qa/qc of solar spectral uv-measurements: checking for wavelength shifts and spectral shape...
TRANSCRIPT
![Page 1: QA/QC of Solar spectral UV-measurements: Checking for wavelength shifts and spectral shape deviations with focus on error flagging Harry Slaper, PN den](https://reader035.vdocuments.us/reader035/viewer/2022062620/551b6815550346a6148b4aba/html5/thumbnails/1.jpg)
QA/QC of Solar spectral QA/QC of Solar spectral UV-measurements:UV-measurements:
Checking for wavelength shifts and spectral shape deviations with
focus on error flagging
Harry Slaper, PN den Outer, HAJM Reinen
![Page 2: QA/QC of Solar spectral UV-measurements: Checking for wavelength shifts and spectral shape deviations with focus on error flagging Harry Slaper, PN den](https://reader035.vdocuments.us/reader035/viewer/2022062620/551b6815550346a6148b4aba/html5/thumbnails/2.jpg)
Flagging outputFlagging outputShift in two wavelength regions:< 325 nm green 0.1 nm yellow 0.2 nm red> 325 nm green 0.2 nm yellow 0.4 nm red
starting wavelength (SZA dependent) < 2-3% error => GREEN
spike error flag: shape error > 10% =>2-fold error at single wavelength=> YELLOWat 305 nm corresponds to 2-3 % CIE-error
atmospheric variability during the scan:green is subdivided in INSTABLE (shape>3%)HIGHLY INSTABLE (shape>5%)
![Page 3: QA/QC of Solar spectral UV-measurements: Checking for wavelength shifts and spectral shape deviations with focus on error flagging Harry Slaper, PN den](https://reader035.vdocuments.us/reader035/viewer/2022062620/551b6815550346a6148b4aba/html5/thumbnails/3.jpg)
Shift and deconvolution Shift and deconvolution analysis: why?analysis: why?
Spectral instruments show variation in slit functions (FWHM 0.3 - 2.5 nm)=>comparing spectra
High accuracy wavelength calibration required (wavelength shift of 0.03-0.05nm => 1% change in Effective UV)
![Page 4: QA/QC of Solar spectral UV-measurements: Checking for wavelength shifts and spectral shape deviations with focus on error flagging Harry Slaper, PN den](https://reader035.vdocuments.us/reader035/viewer/2022062620/551b6815550346a6148b4aba/html5/thumbnails/4.jpg)
Wavelength Shift AnalysisWavelength Shift AnalysisS T E P 1R a t i o o f m e a s u r e m e n t M a t w a v e l e n g t h ,w i t h t h e s u m o f t w o s u r r o u n d i n gm e a s u r e m e n t s :
MR ( ) = 2 M ( )
M ( - s ) + M ( + s )
S T E P 2S i m i l a r r a t i o f o r s i m u l a t e d m e a s u r e m e n tR M o d ( s l i t c o n v o l u t e d h i g h r e s o l u t i o n ) .
![Page 5: QA/QC of Solar spectral UV-measurements: Checking for wavelength shifts and spectral shape deviations with focus on error flagging Harry Slaper, PN den](https://reader035.vdocuments.us/reader035/viewer/2022062620/551b6815550346a6148b4aba/html5/thumbnails/5.jpg)
Wavelength Shift AnalysisWavelength Shift AnalysisS T E P 3C a l c u l a t e E r r o r R a t i o t a k i n g t h e r a t i o o f t h e m e a s u r e d a n dm o d e l l e d r a t i o ’ s f r o m s t e p s 1 a n d 2 .M i n i m a l i z e t h e E R b y c h o o s i n g o p t i m a l s h i f t :
E R =
( MR ( )
M o dR ( + )2)
( n - 1 )
l n ( )
w h e r e s u m m a t i o n o v e r w a v e l e n g t h i n t e r v a l ( 1 6 n m ) n - 1 n u m b e r o f r a t i o s i n t h e i n t e r v a l t h e w a v e l e n g t h s h i f t
![Page 6: QA/QC of Solar spectral UV-measurements: Checking for wavelength shifts and spectral shape deviations with focus on error flagging Harry Slaper, PN den](https://reader035.vdocuments.us/reader035/viewer/2022062620/551b6815550346a6148b4aba/html5/thumbnails/6.jpg)
Correction for slit function Correction for slit function effectseffects
Measured spectrum: M() Iterative ‘Deconvolution’ technique -
start spectrum S0() (high resol.) - simulate measurement Ms0(); - correct start spectrum: S1() = (M() / Ms0()) S0()
Repeat steps until: S9()
![Page 7: QA/QC of Solar spectral UV-measurements: Checking for wavelength shifts and spectral shape deviations with focus on error flagging Harry Slaper, PN den](https://reader035.vdocuments.us/reader035/viewer/2022062620/551b6815550346a6148b4aba/html5/thumbnails/7.jpg)
Wavelength shift Wavelength shift uncertaintiesuncertainties
Reproducability error < 0.01 nm (FWHM <= 1 nm)
Alignment error 0.01-0.02 nm Absolute acccuracy < 0.01 nm
(depends on Extra Terrestrial)
![Page 8: QA/QC of Solar spectral UV-measurements: Checking for wavelength shifts and spectral shape deviations with focus on error flagging Harry Slaper, PN den](https://reader035.vdocuments.us/reader035/viewer/2022062620/551b6815550346a6148b4aba/html5/thumbnails/8.jpg)
-0.1
-0.08
-0.06
-0.04
-0.02
0
0.02
0.04
0.06
0.08
0.1
30 40 50 60 70 80 90
Solar Zenith Angle (degrees)
Waveeln
gth
sh
ift
(nm
)
shift < 325 nm
shift >325 nm
Solar zenith angle (in)depence of wavelength shiftsusing SHICrivm (spectra from RIVM UV monitoring)
![Page 9: QA/QC of Solar spectral UV-measurements: Checking for wavelength shifts and spectral shape deviations with focus on error flagging Harry Slaper, PN den](https://reader035.vdocuments.us/reader035/viewer/2022062620/551b6815550346a6148b4aba/html5/thumbnails/9.jpg)
Shift and deconvolution Shift and deconvolution analysisanalysis
Enables high accuracy wavelength calibration
Facilitates direct comparison of spectral readings from different instruments/sites
Enables calculation of Standardized spectral UV-data
![Page 10: QA/QC of Solar spectral UV-measurements: Checking for wavelength shifts and spectral shape deviations with focus on error flagging Harry Slaper, PN den](https://reader035.vdocuments.us/reader035/viewer/2022062620/551b6815550346a6148b4aba/html5/thumbnails/10.jpg)
Shift and deconvolution Shift and deconvolution software: SHICrivmsoftware: SHICrivm
wavelength dependent shift standardized spectrum
- shift and slit corrected- shift corrected
effective UV-determination for several Action Spectra
Check on spectral anomalies
![Page 11: QA/QC of Solar spectral UV-measurements: Checking for wavelength shifts and spectral shape deviations with focus on error flagging Harry Slaper, PN den](https://reader035.vdocuments.us/reader035/viewer/2022062620/551b6815550346a6148b4aba/html5/thumbnails/11.jpg)
Further application of Further application of SHICrivmSHICrivm
Determine high resolution quality of ET-spectra (Kitt Peak, Atlas 3)
Slit function; FWHM estimates Analysis of requirements
regarding over/under sampling Facilitate model/measurement
intercomparison
![Page 12: QA/QC of Solar spectral UV-measurements: Checking for wavelength shifts and spectral shape deviations with focus on error flagging Harry Slaper, PN den](https://reader035.vdocuments.us/reader035/viewer/2022062620/551b6815550346a6148b4aba/html5/thumbnails/12.jpg)
Analysis of spectral shape errors Analysis of spectral shape errors (spikes, cloud variability)(spikes, cloud variability)
Use ratio of readings at two adjacent wavelengths and compare with expected ratio (SHIC)
identify shortest wavelength with reliable readings (SHIC and ratio method)
identify spikes and cloud variability
![Page 13: QA/QC of Solar spectral UV-measurements: Checking for wavelength shifts and spectral shape deviations with focus on error flagging Harry Slaper, PN den](https://reader035.vdocuments.us/reader035/viewer/2022062620/551b6815550346a6148b4aba/html5/thumbnails/13.jpg)
relative CIE weighted UV
0
0.05
0.1
0.15
0.2
0.25
0.3
0.35
0 20 40 60 80 100
SZA (degrees)
Fra
ctio
n b
elo
w in
dic
ated
wav
elen
gth
290
295
300
303
305
Data from Blumthaler SUSPEN 1997
![Page 14: QA/QC of Solar spectral UV-measurements: Checking for wavelength shifts and spectral shape deviations with focus on error flagging Harry Slaper, PN den](https://reader035.vdocuments.us/reader035/viewer/2022062620/551b6815550346a6148b4aba/html5/thumbnails/14.jpg)
Ratio of spectra for DEG
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
1.1
1.2
290 300 310 320 330 340 350 360 370 380 390 400
Wavelength
Rat
io
DEG 185 0400/0430
![Page 15: QA/QC of Solar spectral UV-measurements: Checking for wavelength shifts and spectral shape deviations with focus on error flagging Harry Slaper, PN den](https://reader035.vdocuments.us/reader035/viewer/2022062620/551b6815550346a6148b4aba/html5/thumbnails/15.jpg)
Check spectral structure
-30
-20
-10
0
10
20
30
290 300 310 320 330 340 350 360 370 380 390 400
wavelength (nm)
de
via
tio
n (
%)
SHIC
Spectral ratio
![Page 16: QA/QC of Solar spectral UV-measurements: Checking for wavelength shifts and spectral shape deviations with focus on error flagging Harry Slaper, PN den](https://reader035.vdocuments.us/reader035/viewer/2022062620/551b6815550346a6148b4aba/html5/thumbnails/16.jpg)
Check spectral structure
-30
-20
-10
0
10
20
30
290 300 310 320 330 340 350 360 370 380 390 400
wavelength (nm)
dev
iati
on
(%
)
SHIC
Spectral ratio
![Page 17: QA/QC of Solar spectral UV-measurements: Checking for wavelength shifts and spectral shape deviations with focus on error flagging Harry Slaper, PN den](https://reader035.vdocuments.us/reader035/viewer/2022062620/551b6815550346a6148b4aba/html5/thumbnails/17.jpg)
Start wavelength
286
288
290
292
294
296
298
300
302
285 290 295 300 305
SHIC analysed (nm)
Ra
tio
an
aly
sed
ATI 185
Linear (ATI 185)
![Page 18: QA/QC of Solar spectral UV-measurements: Checking for wavelength shifts and spectral shape deviations with focus on error flagging Harry Slaper, PN den](https://reader035.vdocuments.us/reader035/viewer/2022062620/551b6815550346a6148b4aba/html5/thumbnails/18.jpg)
Start wavelength
288
290
292
294
296
298
300
302
285 290 295 300 305
SHIC analysed (nm)
Ra
tio
ba
sed
(n
m)
DEG 185
Linear (DEG 185)
Start wavelength
290
295
300
305
310
315
290 295 300 305 310
SHIC analysed start (nm)
Ra
tio
ba
sed
sta
rt (
nm
)
NLR 185
Linear (NLR 185)
starting wavelength CAT
288
290
292
294
296
298
300
302
304
288 290 292 294 296 298 300 302 304
SHIC based start (nm)
Ra
tio
ba
sed
sta
rt (
nm
)
185 CAT
Linear (185 CAT)
Starting wavelength NZY
288
290
292
294
296
298
300
302
304
306
290 295 300 305
SHIC based start (nm)
Ra
tio
ba
sed
sta
rt (
nm
)
NZY
Linear (NZY)
![Page 19: QA/QC of Solar spectral UV-measurements: Checking for wavelength shifts and spectral shape deviations with focus on error flagging Harry Slaper, PN den](https://reader035.vdocuments.us/reader035/viewer/2022062620/551b6815550346a6148b4aba/html5/thumbnails/19.jpg)
Identification of spikes and Identification of spikes and spectral shape errorsspectral shape errors
Spectral shape errors by comparing readings at subsequent wavelengths with ‘expected’ratio
problem how to distinguish between errors and variability
proposal shape error >0.03 instable conditions; >0.05 highly instable; >0.10 YELLOW (spikes flag)
![Page 20: QA/QC of Solar spectral UV-measurements: Checking for wavelength shifts and spectral shape deviations with focus on error flagging Harry Slaper, PN den](https://reader035.vdocuments.us/reader035/viewer/2022062620/551b6815550346a6148b4aba/html5/thumbnails/20.jpg)
Shape errors versus pyrano variabilitySZA 35-45 RIVM data march-october 2000
0
5
10
15
20
25
30
35
40
0 3 6 9 12 15
shape (10%) error (%)
pyr
ano
var
iab
ility
(%
)
meansd%
R = 0.812 mean SD of 1 min readings
![Page 21: QA/QC of Solar spectral UV-measurements: Checking for wavelength shifts and spectral shape deviations with focus on error flagging Harry Slaper, PN den](https://reader035.vdocuments.us/reader035/viewer/2022062620/551b6815550346a6148b4aba/html5/thumbnails/21.jpg)
Correlation of Correlation ofshape 10% with mean SD
SZA nr mean SD SD <min> SD <min>R R R
0 - 35 692 0.851 0.58 0.68435 - 45 1817 0.812 0.49 0.70745 - 55 1805 0.764 0.48 0.6955 - 65 1857 0.722 0.41 0.64665 - 75 1640 0.634 0.37 excl 1% 0.50575 - 85 1648 0.591 0.48 0.64885 - 90 844 0.611 0.59 0.613
shape derived from spectral shape errorsSD derived from pyranometer variabilitywithin minute or between minutes
Correlation of shape errors (10%) and pyranometervariability (mean of SD within minutes; SD between 1 minute readings)
![Page 22: QA/QC of Solar spectral UV-measurements: Checking for wavelength shifts and spectral shape deviations with focus on error flagging Harry Slaper, PN den](https://reader035.vdocuments.us/reader035/viewer/2022062620/551b6815550346a6148b4aba/html5/thumbnails/22.jpg)
Flagging outputFlagging outputShift in two wavelength regions:< 325 nm green 0.1 nm yellow 0.2 nm red> 325 nm green 0.2 nm yellow 0.4 nm red
starting wavelength (SZA dependent) < 2-3% error => GREEN
spike error flag: shape error > 10% =>2-fold error at single wavelength=> YELLOWat 305 nm corresponds to 2-3 % CIE-error
atmospheric variability during the scan:green is subdivided in INSTABLE (shape>3%)HIGHLY INSTABLE (shape>5%)
![Page 23: QA/QC of Solar spectral UV-measurements: Checking for wavelength shifts and spectral shape deviations with focus on error flagging Harry Slaper, PN den](https://reader035.vdocuments.us/reader035/viewer/2022062620/551b6815550346a6148b4aba/html5/thumbnails/23.jpg)
!SHICrivm version 3.012 10-06-2001; Harry Slaper LSO/RIVM, [email protected]! Quality check shift and structure! sh1 refers to average shift over wavelength interval: 300.00 325.00! sh2 refers to average shift over wavelength interval: 325.00 390.00! start_wave indicates effective UV below lowest wavelength where 5 successive readings! meet criterium for structural checks: 0.250! spike and instability checking above the starting wavelength1061024 sza: 44.842 shift1 flagging : GREEN 0.016 (nm) shift2 flagging : GREEN 0.030 (nm) spike and instability flag: GREEN INSTABLE 3.04 % start wavelength flag : YELLOW 299.04 (nm) efuv below: 0.49 % last wavelength flag : GREEN 379.50 (nm) efuv above: 1.92 %
![Page 24: QA/QC of Solar spectral UV-measurements: Checking for wavelength shifts and spectral shape deviations with focus on error flagging Harry Slaper, PN den](https://reader035.vdocuments.us/reader035/viewer/2022062620/551b6815550346a6148b4aba/html5/thumbnails/24.jpg)
SHIC statusSHIC status
Version 2.79 released october 2000; 3.012 now tested; UNIX version tested
flagging included; faster (1-2 sec per spectrum)
spectral shape checks improved shifts below 300 nm possible
further improvement: output control
![Page 25: QA/QC of Solar spectral UV-measurements: Checking for wavelength shifts and spectral shape deviations with focus on error flagging Harry Slaper, PN den](https://reader035.vdocuments.us/reader035/viewer/2022062620/551b6815550346a6148b4aba/html5/thumbnails/25.jpg)
Next year activitiesNext year activities New release of SHIC december 2001 Application of flagging on subsets of data Use in climatological analysis (calculation of
effective UV in standardized way) improvement of algorithm to identify
atmospheric variability during the scans (OE) implementation at the database (in
collaboration with FMI, Ola Engelsen) uncertainty analysis in relation to FWHM