![Page 1: DART Tutorial Sec’on 8: Dealing with Sampling ErrorDART Tutorial Sec’on 8: Slide 17 Experimen’ng with Lorenz 96 The lorenz_96 domain is mapped to a [0, 1] periodic range. Try](https://reader034.vdocuments.us/reader034/viewer/2022050220/5f66095012f388209625cd75/html5/thumbnails/1.jpg)
TheNa'onalCenterforAtmosphericResearchissponsoredbytheNa'onalScienceFounda'on.Anyopinions,findingsandconclusionsorrecommenda'onsexpressedinthispublica'onarethoseoftheauthor(s)anddonotnecessarilyreflecttheviewsoftheNa'onalScienceFounda'on.
©UCAR2014
DARTTutorialSec'on8:DealingwithSamplingError
![Page 2: DART Tutorial Sec’on 8: Dealing with Sampling ErrorDART Tutorial Sec’on 8: Slide 17 Experimen’ng with Lorenz 96 The lorenz_96 domain is mapped to a [0, 1] periodic range. Try](https://reader034.vdocuments.us/reader034/viewer/2022050220/5f66095012f388209625cd75/html5/thumbnails/2.jpg)
Upda'ngAddi'onalPriorStateVariables
Twoprimaryerrorsources:1.Samplingerrorduetonoise.Canoccurevenifthereisalinearrela'onbetweenvariables.Sampleregressioncoefficientimprecisewithfiniteensembles.2.Linearapproxima'onisinvalid.Ifthereissubstan'alnonlinearityin‘true’rela'onbetweenvariablesoverrangeofpriorensemble.(seesec'on10).Mayneedtoaddressbothissuesforgoodperformance.
DARTTutorialSec'on8:Slide2
![Page 3: DART Tutorial Sec’on 8: Dealing with Sampling ErrorDART Tutorial Sec’on 8: Slide 17 Experimen’ng with Lorenz 96 The lorenz_96 domain is mapped to a [0, 1] periodic range. Try](https://reader034.vdocuments.us/reader034/viewer/2022050220/5f66095012f388209625cd75/html5/thumbnails/3.jpg)
RegressionSamplingError&FilterDivergence
Supposeunobservedstatevariableisknowntobeunrelatedtosetofobservedvariables.Unobservedvariableshouldremainunchanged.
DARTTutorialSec'on8:Slide3
![Page 4: DART Tutorial Sec’on 8: Dealing with Sampling ErrorDART Tutorial Sec’on 8: Slide 17 Experimen’ng with Lorenz 96 The lorenz_96 domain is mapped to a [0, 1] periodic range. Try](https://reader034.vdocuments.us/reader034/viewer/2022050220/5f66095012f388209625cd75/html5/thumbnails/4.jpg)
RegressionSamplingError&FilterDivergence
Supposeunobservedstatevariableisknowntobeunrelatedtosetofobservedvariables.Finitesamplesfromjointdistribu'onwillhavenon-zerocorrela'on.Expected|correl|=0.19for20samples.Aaeroneobserva'on,unobservedvariablemean,standarddevia'onchange.
DARTTutorialSec'on8:Slide4
![Page 5: DART Tutorial Sec’on 8: Dealing with Sampling ErrorDART Tutorial Sec’on 8: Slide 17 Experimen’ng with Lorenz 96 The lorenz_96 domain is mapped to a [0, 1] periodic range. Try](https://reader034.vdocuments.us/reader034/viewer/2022050220/5f66095012f388209625cd75/html5/thumbnails/5.jpg)
RegressionSamplingError&FilterDivergence
Supposeunobservedstatevariableisknowntobeunrelatedtosetofobservedvariables.Unobservedvariableshouldremainunchanged.Unobservedmeanfollowsarandomwalkasmoreobserva'onsareused.
DARTTutorialSec'on8:Slide5
![Page 6: DART Tutorial Sec’on 8: Dealing with Sampling ErrorDART Tutorial Sec’on 8: Slide 17 Experimen’ng with Lorenz 96 The lorenz_96 domain is mapped to a [0, 1] periodic range. Try](https://reader034.vdocuments.us/reader034/viewer/2022050220/5f66095012f388209625cd75/html5/thumbnails/6.jpg)
RegressionSamplingError&FilterDivergence
Supposeunobservedstatevariableisknowntobeunrelatedtosetofobservedvariables.Unobservedvariableshouldremainunchanged.UnobservedS.Dsystema'callydecreases.Expectedchangein|SD|isnega'veforanynon-zerosamplecorrela'on.
DARTTutorialSec'on8:Slide6
![Page 7: DART Tutorial Sec’on 8: Dealing with Sampling ErrorDART Tutorial Sec’on 8: Slide 17 Experimen’ng with Lorenz 96 The lorenz_96 domain is mapped to a [0, 1] periodic range. Try](https://reader034.vdocuments.us/reader034/viewer/2022050220/5f66095012f388209625cd75/html5/thumbnails/7.jpg)
RegressionSamplingError&FilterDivergence
Supposeunobservedstatevariableisknowntobeunrelatedtosetofobservedvariables.Unobservedvariableshouldremainunchanged.UnobservedS.Dsystema'callydecreases.Expectedchangein|SD|isnega'veforanynon-zerosamplecorrela'on.
DARTTutorialSec'on8:Slide7
![Page 8: DART Tutorial Sec’on 8: Dealing with Sampling ErrorDART Tutorial Sec’on 8: Slide 17 Experimen’ng with Lorenz 96 The lorenz_96 domain is mapped to a [0, 1] periodic range. Try](https://reader034.vdocuments.us/reader034/viewer/2022050220/5f66095012f388209625cd75/html5/thumbnails/8.jpg)
RegressionSamplingError&FilterDivergence
Supposeunobservedstatevariableisknowntobeunrelatedtosetofobservedvariables.Unobservedvariableshouldremainunchanged.UnobservedS.Dsystema'callydecreases.Expectedchangein|SD|isnega'veforanynon-zerosamplecorrela'on.
DARTTutorialSec'on8:Slide8
![Page 9: DART Tutorial Sec’on 8: Dealing with Sampling ErrorDART Tutorial Sec’on 8: Slide 17 Experimen’ng with Lorenz 96 The lorenz_96 domain is mapped to a [0, 1] periodic range. Try](https://reader034.vdocuments.us/reader034/viewer/2022050220/5f66095012f388209625cd75/html5/thumbnails/9.jpg)
RegressionSamplingError&FilterDivergence
Supposeunobservedstatevariableisknowntobeunrelatedtosetofobservedvariables.Es'matesofunobservedbecometooconfident.Giveprogressivelylessweighttomeaningfulobs.Eventually,meaningfulobsareessen'allyignored.
DARTTutorialSec'on8:Slide9
![Page 10: DART Tutorial Sec’on 8: Dealing with Sampling ErrorDART Tutorial Sec’on 8: Slide 17 Experimen’ng with Lorenz 96 The lorenz_96 domain is mapped to a [0, 1] periodic range. Try](https://reader034.vdocuments.us/reader034/viewer/2022050220/5f66095012f388209625cd75/html5/thumbnails/10.jpg)
FilterDivergence
Ignoringmeaningfulobserva'onsduetooverconfidenceisatypeofFILTERDIVERGENCE.
Thiswasseeninini'alLorenz96(40-variable)experiment.Thespreadbecamesmall=>thefilterthoughtithadagoodes'mate.Theerrorstayedlargebecausegoodobserva'onswereignored.
DARTTutorialSec'on8:Slide10
![Page 11: DART Tutorial Sec’on 8: Dealing with Sampling ErrorDART Tutorial Sec’on 8: Slide 17 Experimen’ng with Lorenz 96 The lorenz_96 domain is mapped to a [0, 1] periodic range. Try](https://reader034.vdocuments.us/reader034/viewer/2022050220/5f66095012f388209625cd75/html5/thumbnails/11.jpg)
RegressionSamplingError&FilterDivergence
Plotshowsexpectedabsolutevalueofsamplecorrela'onversustruecorrela'on.Errordecreaseswithsamplesizeandforlarger|realcorrela'ons|.
DARTTutorialSec'on8:Slide11
![Page 12: DART Tutorial Sec’on 8: Dealing with Sampling ErrorDART Tutorial Sec’on 8: Slide 17 Experimen’ng with Lorenz 96 The lorenz_96 domain is mapped to a [0, 1] periodic range. Try](https://reader034.vdocuments.us/reader034/viewer/2022050220/5f66095012f388209625cd75/html5/thumbnails/12.jpg)
RegressionSamplingError&FilterDivergence
Plotshowsexpectedabsolutevalueofsamplecorrela'onversustruecorrela'on.Forsmalltruecorrela'ons,errorsares'llundesirablylargeevenfor80memberensembles.
DARTTutorialSec'on8:Slide12
![Page 13: DART Tutorial Sec’on 8: Dealing with Sampling ErrorDART Tutorial Sec’on 8: Slide 17 Experimen’ng with Lorenz 96 The lorenz_96 domain is mapped to a [0, 1] periodic range. Try](https://reader034.vdocuments.us/reader034/viewer/2022050220/5f66095012f388209625cd75/html5/thumbnails/13.jpg)
DealingwithRegressionSamplingError
1.Ignoreit:ifnumberofunrelatedobserva'onsissmallandthereissomewayofmaintainingvarianceinpriors.Wedidthisinthe3and9variablemodels.
2.Uselargerensemblestolimitsamplingerror(testinlorenz_96).
Thiscangetexpensiveforbigproblems.Trymodifyingens_sizein&filter_nml (try40,80,160).Note:Forensemblesizesgreaterthan80,set&filter_nml: perturb_from_single_instance = .true.ThistellsDARTtogenerateini'alensemblemembersusingarandomnumbergenerator,ratherthanreadingthemfromaninputfile(whichonlycontains80membersinthisdirectory).
DARTTutorialSec'on8:Slide13
![Page 14: DART Tutorial Sec’on 8: Dealing with Sampling ErrorDART Tutorial Sec’on 8: Slide 17 Experimen’ng with Lorenz 96 The lorenz_96 domain is mapped to a [0, 1] periodic range. Try](https://reader034.vdocuments.us/reader034/viewer/2022050220/5f66095012f388209625cd75/html5/thumbnails/14.jpg)
DealingwithRegressionSamplingError
1.Ignoreit:ifnumberofunrelatedobserva'onsissmallandthereissomewayofmaintainingvarianceinpriors.Wedidthisinthe3and9variablemodels.
2.Uselargerensemblestolimitsamplingerror(testinlorenz_96).
Thiscangetexpensiveforbigproblems.Trymodifyingens_sizein&filter_nml (try40,80,160).
3.Useaddi'onalaprioriinforma'onaboutrela'onbetween
observa'onsandstatevariables.Don’tletanobserva'onimpactstateiftheyareknowntobeunrelated.
4.Trytodeterminetheamountofsamplingerrorandcorrectforit.
Therearemanywaystodothis;somesimple,somecomplex. DARTTutorialSec'on8:Slide14
![Page 15: DART Tutorial Sec’on 8: Dealing with Sampling ErrorDART Tutorial Sec’on 8: Slide 17 Experimen’ng with Lorenz 96 The lorenz_96 domain is mapped to a [0, 1] periodic range. Try](https://reader034.vdocuments.us/reader034/viewer/2022050220/5f66095012f388209625cd75/html5/thumbnails/15.jpg)
DealingwithRegressionSamplingError
3.Useaddi'onalaprioriinforma'onaboutrela'onbetweenobserva'onsandstatevariables.
Foratmosphericassimila'onproblems:Weightregressionasfunc'onofhorizontaldistancefromobserva'on.Gaspari-Cohn:5thordercompactlysupportedpolynomial.
DARTTutorialSec'on8:Slide15
![Page 16: DART Tutorial Sec’on 8: Dealing with Sampling ErrorDART Tutorial Sec’on 8: Slide 17 Experimen’ng with Lorenz 96 The lorenz_96 domain is mapped to a [0, 1] periodic range. Try](https://reader034.vdocuments.us/reader034/viewer/2022050220/5f66095012f388209625cd75/html5/thumbnails/16.jpg)
DealingwithRegressionSamplingError
3.Useaddi'onalaprioriinforma'onaboutrela'onbetweenobserva'onsandstatevariables.
Canuseotherfunc'onstoweightregression.Unclearwhatdistancemeansforsomeobs./statevariablepairs.ReferredtoasLOCALIZATION.
Halfwidth
DARTTutorialSec'on8:Slide16
![Page 17: DART Tutorial Sec’on 8: Dealing with Sampling ErrorDART Tutorial Sec’on 8: Slide 17 Experimen’ng with Lorenz 96 The lorenz_96 domain is mapped to a [0, 1] periodic range. Try](https://reader034.vdocuments.us/reader034/viewer/2022050220/5f66095012f388209625cd75/html5/thumbnails/17.jpg)
DARTprovidesseverallocaliza'onop'ons
1.Differentshapesforthelocaliza'onfunc'onareavailable.Controlledbyselect_localiza-onin&cov_cutoff_nml.
2.Halfwidthoflocaliza'onfunc'onsetbycutoffin&assim_tools_nml
1=>Gaspari-Cohn
2=>Boxcar
3=>RampedBoxcar
DARTTutorialSec'on8:Slide17
![Page 18: DART Tutorial Sec’on 8: Dealing with Sampling ErrorDART Tutorial Sec’on 8: Slide 17 Experimen’ng with Lorenz 96 The lorenz_96 domain is mapped to a [0, 1] periodic range. Try](https://reader034.vdocuments.us/reader034/viewer/2022050220/5f66095012f388209625cd75/html5/thumbnails/18.jpg)
Experimen'ngwithLorenz96
Thelorenz_96domainismappedtoa[0,1]periodicrange.
TryavarietyofhalfwidthsforaGaspariCohnlocaliza'onbyWealreadyknowthataverylargelocaliza'onhalf-widthdiverges.
Whathappensforaverysmallvalue?
Whathappenswithintermediatevalues(saybetween0.1and0.5)?
Canalsotrychangingtheshape,(bestwith40memberensemble):Tryop'on2or3forselect_localiza-onin&cov_cutoff_nml.
DARTTutorialSec'on8:Slide18
&assim_tools_nml filter_kind = 1 cutoff = 1000000.0 …&filter_nml ens_size = 20 perturb_from_single_instance = .false. …
![Page 19: DART Tutorial Sec’on 8: Dealing with Sampling ErrorDART Tutorial Sec’on 8: Slide 17 Experimen’ng with Lorenz 96 The lorenz_96 domain is mapped to a [0, 1] periodic range. Try](https://reader034.vdocuments.us/reader034/viewer/2022050220/5f66095012f388209625cd75/html5/thumbnails/19.jpg)
DealingwithRegressionSamplingError
4.Trytodeterminetheamountofsamplingerrorandcorrectforit.
Manywaystodothis.DARTimplementsonenaiveway:1.Takesetofincrementsfromagivenobserva'on,2.Supposethisobserva'onandastatevariablearenotcorrelated,3.Computetheexpecteddecreaseinspreadgivennotcorrelated,4.Addthisamountofspreadbackintothestatevariable.
Theexpecteddecreaseinspreadiscomputedbyoff-lineMonteCarlo.Resultsofoff-linesimula'onaretabulatedandapplied.(Thiscanbeaveryusefultechniquewhenyou’reanaly'callyclueless).
Trythisalgorithm:set&assim_tools_nml: spread_restoration = .true.
Howdoesitworkwith20ensemblemembers,nolocaliza'on?
DARTTutorialSec'on8:Slide19
![Page 20: DART Tutorial Sec’on 8: Dealing with Sampling ErrorDART Tutorial Sec’on 8: Slide 17 Experimen’ng with Lorenz 96 The lorenz_96 domain is mapped to a [0, 1] periodic range. Try](https://reader034.vdocuments.us/reader034/viewer/2022050220/5f66095012f388209625cd75/html5/thumbnails/20.jpg)
DealingwithRegressionSamplingError
4.Trytodeterminetheamountofsamplingerrorandcorrectforit.Manywaystodothis.DARTalsoimplementsasamplingerrorcorrec'onalgorithmthatcanreducebutnoteliminateproblems.ThisalgorithmALMOSTALWAYSIMPROVESlargemodelresults.Trythisalgorithm:set&assim_tools_nml: sampling_error_correction = .true.
Howdoesitworkwith20ensemblemembers,nolocaliza'on?
DARTTutorialSec'on8:Slide20
![Page 21: DART Tutorial Sec’on 8: Dealing with Sampling ErrorDART Tutorial Sec’on 8: Slide 17 Experimen’ng with Lorenz 96 The lorenz_96 domain is mapped to a [0, 1] periodic range. Try](https://reader034.vdocuments.us/reader034/viewer/2022050220/5f66095012f388209625cd75/html5/thumbnails/21.jpg)
1. FilteringForaOneVariableSystem2. TheDARTDirectoryTree3. DARTRun>meControlandDocumenta>on4. Howshouldobserva>onsofastatevariableimpactanunobservedstatevariable?
Mul>variateassimila>on.5. ComprehensiveFilteringTheory:Non-Iden>tyObserva>onsandtheJointPhaseSpace6. OtherUpdatesforAnObservedVariable7. SomeAddi>onalLow-OrderModels8. DealingwithSamplingError9. MoreonDealingwithError;Infla>on10. RegressionandNonlinearEffects11. Crea>ngDARTExecutables12. Adap>veInfla>on13. HierarchicalGroupFiltersandLocaliza>on14. QualityControl15. DARTExperiments:ControlandDesign16. Diagnos>cOutput17. Crea>ngObserva>onSequences18. LostinPhaseSpace:TheChallengeofNotKnowingtheTruth19. DART-CompliantModelsandMakingModelsCompliant20. ModelParameterEs>ma>on21. Observa>onTypesandObservingSystemDesign22. ParallelAlgorithmImplementa>on23. Loca'onmoduledesign(notavailable)24. Fixedlagsmoother(notavailable)25. Asimple1Dadvec>onmodel:TracerDataAssimila>on
DARTTutorialIndextoSec'ons
DARTTutorialSec'on8:Slide21
