A  summary  of  SME  reprocessing   efforts  and  current  status Odele  Coddington,  Gary  Rottman,  Regner Trampedach,  Peter  Pilewskie,   Bill  Barrett   As  part  of  a  larger  team  effort  with  Judith  Lean  (PI),   “How  does  the  Sun’s  Spectrum  Vary?” Outline • Motivation • Recap  of  Year  2  efforts   • • • • Measurement  uncertainty,   Development  of  processing  algorithm  (Processing  Approach  “A”),   Initial  wavelength  calibration  efforts  (Processing  Approach  “B”),   Potential  temperature  dependencies. • Year  3  Efforts   • • • • “cleaning  the  time  series” uncertainty  propagation   Incorporating  a  SORCE/SOLSTICE  reference  spectrum wavelength  calibration • Year  3  Results • Remaining  steps  for  no-­‐cost  extension  period 3rd  S olar  Irradiance  S cience  Team,  8-­‐9  May  2018 Motivation • SME  reanalysis  will  give  an  improved   understanding  of  solar  cycle  variability. • SME  observations  are  potentially  stable   SSI  database  due  to  limited  solar   exposure  and  in-­‐flight  monitoring  of   degradation. • Re-­‐analyzed  SME  observations  may: • constrain  UV  variability,  and  – through   a   model  and  TSI  observations   – further   constrain  visible  and  infrared   SSI  variability.   • This  new  knowledge   would  be  used  to   improve  solar  variability  models. Figure:  Comparisons  of  modeled  and  measured  solar  irradiance   over  solar  cycle  time  scales  along  with  components  of  sunspot   and  facular  influence  (Courtesy  Judith  Lean) 3rd  S olar  Irradiance  S cience  Team,  8-­‐9  May  2018 Year  2  Recap 3rd  S olar  Irradiance  S cience  Team,  8-­‐9  May  2018 Measurement  Uncertainty Quantization  Error  and  Photon  Noise #  of  shifts 5-­‐bits  of  data  (DN) Data  increments  (DN)   “quantization  error” Square  root  of  data  (DN)   “photon  noise” 0 1 2 3 4 5 6 7 0  to  31 32  to  63 64  to  127 128  to  255 256  to  511 512  to  1023 1024  to  2047 2048  to  4095 0  to  5.6 5.7  to  7.9 8  to  11.3 11.3  to  16 16  to  22.6 22.6  to  32 32  to  45 45  to  64 1 2 4 8 16 32 64 128 3rd  S olar  Irradiance  S cience  Team,  8-­‐9  May  2018 Sample  SME  data Data  discretization  more   apparent  at  larger  signal   magnitude.   Different  colors  represent   individual  segments  of   data  collection.   3rd  S olar  Irradiance  S cience  Team,  8-­‐9  May  2018 Two  Algorithm  Processing  Approaches • Approach  “A” L0  data Lyman  alpha assign   uncertainties quality  flag  data subset  by   MUV/FUV  channel filter  for  data   outliers compute  daily   average apply  1  AU   correction 3rd  S olar  Irradiance  S cience  Team,  8-­‐9  May  2018 New  data  product For(an(assigned(Δgp around(central(gp: F’(x)(<(0 • Approach  “B” Begins  with  the  daily   average  spectra  from   Approach  A *Developed  to  focus  on   regions  of  large  solar   variability   and  pronounced   spectral  features   F’(x)(>(0 Identify(stationary(points(by(peaks(or( valleys: Obtain(value(of((floating)(grating(position( Compute  derivative  of  daily   for(local(maximum(or(local(minimum average F’(x)(>(0 F’(x)(<(0 Peak&=&local&maximum Valley&=&local&minimum Identifies&gp corresponding&to&max&DN&(or&a&peak)& Identifies&gp corresponding&to&min&DN&(for&a&valley) For&Data1,&Data2,&and&the&average&of&Data1&and&Data2 Generate(Output(File((for(the(specified(time( Revised&Daily&Solar&Spectral&Irradiance range,((floating)(grating(position,(channel,( Identify  local  extrema  in   Time,&Data1,&Data2,&Avg of&Data&1&and&Data2&(corrected&and& screen(position,(quality(flag): spectrum  by  peaks  of  valleys   uncorrected&to&19AU)&with&associated&uncertainties&(floating)& of  inflection  points as&a&function&of&floating&grating&position.& Lyman  alpha Identify  grating  position  of   peak  or  valley New  Data  product  (using   revised  grating  positions  for   select   portions  of  spectrum) 3rd  S olar  Irradiance  S cience  Team,  8-­‐9  May  2018 Recognizing  and  Correcting  for  a  temperature  effect *We  correct   for  temperature  affects  by  evaluating  (and  correcting)   the  non-­‐zero   slope  of  DN  with   temperature  over   solar  minimum   time   period. Earth  Sun  Distance Earth-­‐viewing  instruments  turned  off   Dec  1 1,  1 986 *Corrected  to  a  constant  Temperature  ( 80  deg C). 3rd  S olar  Irradiance  S cience  Team,  8-­‐9  May  2018 Year  3  Efforts 3rd  S olar  Irradiance  S cience  Team,  8-­‐9  May  2018 “Cleaned”  the  time  series • We  identified  a  small  fraction  of   L0  data  that  was: • Out  of  chronological  order  or, • Duplicate  data  points  or, • Data  points  with  the  same  time   stamp  but  different  data  values. • We  “cleaned”  the  data  by: • Move  the  data  points  to  restore   chronology  or, • Remove  duplicates 3rd  S olar  Irradiance  S cience  Team,  8-­‐9  May  2018 Derived  a  Total  Measurement  Uncertainty Flat  top  =  discretized,  telemetered  value Asymmetric  Gaussian  wings  =  photon   noise Pn(x) • We  formalized  a  continuous   probability  distribution  approach   to  derive  the  variance  of  each   SME  count ( . 𝑉𝑎𝑟 𝑥 =   𝜎 = ) 𝑥 − 𝜇 ( 𝑃 𝑥 𝑑𝑥 /. Terms: 2h  =  size  of  discretization  error A  =  amplitude   of  PDF   𝜔0 and  𝜔( =  width  of  Gaussian  on  each  end  of   flat  top   (photon   noise) 𝜎0( = 𝐴ℎ𝜔0 ℎ 𝜋 + 2𝜔0 + 𝐴𝜔07 𝜋 2 7 + 𝐴ℎ 2 3 𝜎(( 𝜋 2 7 + 𝐴ℎ 2 3 = 𝐴ℎ𝜔( ℎ 𝜋 + 2𝜔( 3rd  S olar  Irradiance  S cience  Team,  8-­‐9  May  2018 + 𝐴𝜔(7 Variances  of  left  and  right  side  of   distribution   are  asymmetric Recognized  the  Value  of  “Bin  Borders” Small  data  gap While  we  don’t  know  with  certainty  the  “true”  data  count  within  a  discretized  bin,  we  do  know  the  data  count   value  when  the  spectrum  passes  through  the  borders  of  neighboring  bins. 3rd  S olar  Irradiance  S cience  Team,  8-­‐9  May  2018 Recognized  the  Value  of  Bin  Borders Small  data  gap While  we  don’t  know  with  certainty  the  “true”  data  count  within  a  discretized  bin,  we  do  know  the  data  count  value  when  the  spectrum   passes  through  the  borders  of  neighboring  bins. 3rd  S olar  Irradiance  S cience  Team,  8-­‐9  May  2018 Developed  a  Reference  Spectrum (for  wavelength  calibration  analysis) • SORCE/SOLSTICE  high-­‐resolution   data  was  the  data  for  our  reference • • • • Courtesy  of  Marty  Snow Wavelength  range  – 158-­‐320  nm Spectral  resolution  – 0.1  nm Grating  step  size  – 0.03  nm • We  convolved  the  high-­‐resolution   SOLSTICE  data  twice  with  a   box  car   window  (width  =  25)  for  a   triangular  smoothing  kernel. Red  line  is  our   “SME/SOLSTICE”  reference  spectrum 3rd  S olar  Irradiance  S cience  Team,  8-­‐9  May  2018 Developed  a  Reference  Spectrum (for  use  in  wavelength  calibration  analysis) Blue  line  is  sample  of  SME  data Red  line  is  our  “SME/SOLSTICE”  reference  spectrum 3rd  S olar  Irradiance  S cience  Team,  8-­‐9  May  2018 Wavelength  Calibration • Goal:  Align  each  segment  of  SME  data  to  the  SME/SOLSTICE  reference   • We  tried  an  abscissa_shift_and_stretch IDL  routine   • results  were  disappointing.   • 7  trials  out  of  100  combinations  of  shift  and  stretch  values  produced   meaningful  results  that  converged  in  fewer  than  1,200  iterations. • Of  those,  no  single  shift/stretch  solution  worked  with  all  segments  of  SME   data. • A  correlation  with  our  SME/SOLSTICE  reference  was  better  with  the  original   SME  data  than  the  shifted  result.   3rd  S olar  Irradiance  S cience  Team,  8-­‐9  May  2018 Wavelength  Calibration:  Cross-­‐correlation  Method • Developed  a  cross-­‐correlation  approach  (developed  with  Regner Trampedach)   • The  grating  position  of  the  maximum  correlation  between  the  segment  of   SME  data  and  the  “SME/SOLTICE”  reference  is  found. • The  location  of  the  maximum  is  interpreted  as  a Δ gp shift   • An  uncertainty in  the  Δ gp shift  is  derived  from  the  width  of  the  correlation   function. 3rd  S olar  Irradiance  S cience  Team,  8-­‐9  May  2018 1981-­‐11-­‐02  00:05:59 Uncertainties  in  Grating  Position  Shift Uncertainty  in  Δ gp (Future  work:  translate  the  Δ gp shift  uncertainty  to  an  irradiance  uncertainty) How  reproducible  is   grating  position  shift   from  day-­‐to-­‐day?? 1-­‐σ from  mean median Shown  here  is  Δ gp from  the  first   505  days. mean Purple   values  are  more  than  3-­‐ σ from   mean  and  are  not  used   in  deriving   median  distribution. 3rd  S olar  Irradiance  S cience  Team,  8-­‐9  May  2018 How  many  grating  position  shifts  have  we  identified? Affected Dates 1983/06/11 1984/10/10 1985/10/03 1986/09/09 1987/04/24 1987/08/16 1987/10/17 1987/10/28 1988/05/05 1988/08/21 3rd  S olar  Irradiance  S cience  Team,  8-­‐9  May  2018 Impacts  of  applying   wavelength  calibration The  white  line  shows  the   average  change  in  DN  after   applying  the  wavelength   correction  relative  to  no   wavelength  correction. (Results  for  the  first  500   days  of  the  mission). To-­‐Do  List • Add  a  quality  assurance  flag  for  data  points  that  have  been   “cleaned”. • Determine  how  reproducible  the  cross-­‐correlation  analysis  is  – In  progress. • Compute  the  Δgp for  the  other  affected  periods  -­‐ In  progress • Translate  Δgp shifts  into  irradiance  uncertainty  – will  do  this   using  the  “SME/SOLSTICE”  reference • Apply  wavelength  corrections  to  FUV  data  – will  use  the  closest   in  time  Δgp shifts  from  the  MUV  channel. • Apply  temperature  correction  (developed  in  Year  2  activities) • Apply  degradation  correction  (cumulative  exposures   determined  in  Year  1  activities). • Define  data  level  definitions  -­‐ In  progress • Evaluate  results • Prepare  new  files  (.netcdf)  for  public  release • Prepare  manuscript  of  results. 3rd  S olar  Irradiance  S cience  Team,  8-­‐9  May  2018