dart tutorial sec’on 4: how should observaons impact an ...single observed variable, single...

33
The Na’onal Center for Atmospheric Research is sponsored by the Na’onal Science Founda’on. Any opinions, findings and conclusions or recommenda’ons expressed in this publica’on are those of the author(s) and do not necessarily reflect the views of the Na’onal Science Founda’on. ©UCAR DART Tutorial Sec’on 4: How should observa’ons impact an unobserved state variable? Mul’variate assimila’on.

Upload: others

Post on 22-Jul-2020

3 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: DART Tutorial Sec’on 4: How should observaons impact an ...Single observed variable, single unobserved variable. So far, have known observaon likelihood for single variable. Now,

TheNa'onalCenterforAtmosphericResearchissponsoredbytheNa'onalScienceFounda'on.Anyopinions,findingsandconclusionsorrecommenda'onsexpressedinthispublica'onarethoseoftheauthor(s)anddonotnecessarilyreflecttheviewsoftheNa'onalScienceFounda'on.

©UCAR

DARTTutorialSec'on4:Howshouldobserva'onsimpactanunobserved

statevariable?Mul'variateassimila'on.

Page 2: DART Tutorial Sec’on 4: How should observaons impact an ...Single observed variable, single unobserved variable. So far, have known observaon likelihood for single variable. Now,

Singleobservedvariable,singleunobservedvariable.

Sofar,haveknownobserva'onlikelihoodforsinglevariable.Now,supposepriorhasanaddi'onalvariable.Willexaminehowensemblemethodsupdateaddi'onalvariable.Basicmethodgeneralizestoanynumberofaddi'onalvariables.MethodsrelatedtoKalmanfilterinsomesense,butnotdonehere.

DARTTutorialSec'on4:Slide2

Page 3: DART Tutorial Sec’on 4: How should observaons impact an ...Single observed variable, single unobserved variable. So far, have known observaon likelihood for single variable. Now,

Ensemblefilters:Upda'ngaddi'onalpriorstatevariables

3

3.5

4

4.5

5

Unob

serv

ed S

tate

Var

iabl

e

−2 0 2 4Observed Variable

Assumethatallweknowisthepriorjointdistribu'on.Onevariableisobserved.Whatshouldhappentotheunobservedvariable?

DARTTutorialSec'on4:Slide3

Page 4: DART Tutorial Sec’on 4: How should observaons impact an ...Single observed variable, single unobserved variable. So far, have known observaon likelihood for single variable. Now,

Ensemblefilters:Upda'ngaddi'onalpriorstatevariables

33.5

44.5

5

Unob

s.

−2 0 2 4Observed Variable

Assumethatallweknowisthepriorjointdistribu'on.Onevariableisobserved.Updateobservedvariablewithoneofthepreviousmethods.

DARTTutorialSec'on4:Slide4

Page 5: DART Tutorial Sec’on 4: How should observaons impact an ...Single observed variable, single unobserved variable. So far, have known observaon likelihood for single variable. Now,

Ensemblefilters:Upda'ngaddi'onalpriorstatevariables

Assumethatallweknowisthepriorjointdistribu'on.Onevariableisobserved.Updateobservedvariablewithoneofthepreviousmethods.

33.5

44.5

5

Unob

s.

−2 0 2 4Observed Variable

DARTTutorialSec'on4:Slide5

Page 6: DART Tutorial Sec’on 4: How should observaons impact an ...Single observed variable, single unobserved variable. So far, have known observaon likelihood for single variable. Now,

Ensemblefilters:Upda'ngaddi'onalpriorstatevariables

Assumethatallweknowisthepriorjointdistribu'on.Onevariableisobserved.Updateobservedvariablewithoneofthepreviousmethods.

33.5

44.5

5

Unob

s.

−2 0 2 4Observed Variable

DARTTutorialSec'on4:Slide6

Page 7: DART Tutorial Sec’on 4: How should observaons impact an ...Single observed variable, single unobserved variable. So far, have known observaon likelihood for single variable. Now,

Ensemblefilters:Upda'ngaddi'onalpriorstatevariables

Assumethatallweknowisthepriorjointdistribu'on.Onevariableisobserved.Computeincrementsforpriorensemblemembersofobservedvariable.

33.5

44.5

5

Unob

s.

−2 0 2 4Observed Variable

Increments

DARTTutorialSec'on4:Slide7

Page 8: DART Tutorial Sec’on 4: How should observaons impact an ...Single observed variable, single unobserved variable. So far, have known observaon likelihood for single variable. Now,

Ensemblefilters:Upda'ngaddi'onalpriorstatevariables

Assumethatallweknowisthepriorjointdistribu'on.Onevariableisobserved.Computeincrementsforpriorensemblemembersofobservedvariable.

33.5

44.5

5

Unob

s.

−2 0 2 4Observed Variable

Increments

DARTTutorialSec'on4:Slide8

Page 9: DART Tutorial Sec’on 4: How should observaons impact an ...Single observed variable, single unobserved variable. So far, have known observaon likelihood for single variable. Now,

Ensemblefilters:Upda'ngaddi'onalpriorstatevariables

Assumethatallweknowisthepriorjointdistribu'on.Onevariableisobserved.Computeincrementsforpriorensemblemembersofobservedvariable.

33.5

44.5

5

Unob

s.

−2 0 2 4Observed Variable

Increments

DARTTutorialSec'on4:Slide9

Page 10: DART Tutorial Sec’on 4: How should observaons impact an ...Single observed variable, single unobserved variable. So far, have known observaon likelihood for single variable. Now,

Ensemblefilters:Upda'ngaddi'onalpriorstatevariables

Assumethatallweknowisthepriorjointdistribu'on.Onevariableisobserved.Computeincrementsforpriorensemblemembersofobservedvariable.

33.5

44.5

5

Unob

s.

−2 0 2 4Observed Variable

Increments

DARTTutorialSec'on4:Slide10

Page 11: DART Tutorial Sec’on 4: How should observaons impact an ...Single observed variable, single unobserved variable. So far, have known observaon likelihood for single variable. Now,

Ensemblefilters:Upda'ngaddi'onalpriorstatevariables

Assumethatallweknowisthepriorjointdistribu'on.Onevariableisobserved.Computeincrementsforpriorensemblemembersofobservedvariable.

33.5

44.5

5

Unob

s.

−2 0 2 4Observed Variable

Increments

DARTTutorialSec'on4:Slide11

Page 12: DART Tutorial Sec’on 4: How should observaons impact an ...Single observed variable, single unobserved variable. So far, have known observaon likelihood for single variable. Now,

Ensemblefilters:Upda'ngaddi'onalpriorstatevariables

Usingonlyincrementsguaranteesthatifobserva'onhadnoimpactonobservedvariable,theunobservedvariableisunchanged.Highlydesirable!

33.5

44.5

5

Unob

s.

−2 0 2 4Observed Variable

Increments

DARTTutorialSec'on4:Slide12

Page 13: DART Tutorial Sec’on 4: How should observaons impact an ...Single observed variable, single unobserved variable. So far, have known observaon likelihood for single variable. Now,

Ensemblefilters:Upda'ngaddi'onalpriorstatevariables

Assumethatallweknowisthepriorjointdistribu'on.Howshouldtheunobservedvariablebeimpacted?1stchoice:leastsquaresEquivalenttolinearregression.Sameasassumingbinormalprior.

3

3.5

4

4.5

5

Unob

serv

ed S

tate

Var

iabl

e

−2 0 2 4Observed Variable

Increments

DARTTutorialSec'on4:Slide13

Page 14: DART Tutorial Sec’on 4: How should observaons impact an ...Single observed variable, single unobserved variable. So far, have known observaon likelihood for single variable. Now,

Ensemblefilters:Upda'ngaddi'onalpriorstatevariables

Havejointpriordistribu'onoftwovariables.Howshouldtheunobservedvariablebeimpacted?1stchoice:leastsquaresBeginbyfindingleastsquaresfit.

3

3.5

4

4.5

5

Unob

serv

ed S

tate

Var

iabl

e

−2 0 2 4Observed Variable

Increments

DARTTutorialSec'on4:Slide14

Page 15: DART Tutorial Sec’on 4: How should observaons impact an ...Single observed variable, single unobserved variable. So far, have known observaon likelihood for single variable. Now,

Ensemblefilters:Upda'ngaddi'onalpriorstatevariables

Havejointpriordistribu'onoftwovariables.Next,regresstheobservedvariableincrementsontoincrementsfortheunobservedvariable.Equivalenttofirstfindingimageofincrementinjointspace.

3

3.5

4

4.5

5

Unob

serv

ed S

tate

Var

iabl

e

−2 0 2 4Observed Variable

Increments

DARTTutorialSec'on4:Slide15

Page 16: DART Tutorial Sec’on 4: How should observaons impact an ...Single observed variable, single unobserved variable. So far, have known observaon likelihood for single variable. Now,

Ensemblefilters:Upda'ngaddi'onalpriorstatevariables

Havejointpriordistribu'onoftwovariables.Next,regresstheobservedvariableincrementsontoincrementsfortheunobservedvariable.Equivalenttofirstfindingimageofincrementinjointspace.

3

3.5

4

4.5

5

Unob

serv

ed S

tate

Var

iabl

e

−2 0 2 4Observed Variable

Increments

DARTTutorialSec'on4:Slide16

Page 17: DART Tutorial Sec’on 4: How should observaons impact an ...Single observed variable, single unobserved variable. So far, have known observaon likelihood for single variable. Now,

Ensemblefilters:Upda'ngaddi'onalpriorstatevariables

Havejointpriordistribu'onoftwovariables.Next,regresstheobservedvariableincrementsontoincrementsfortheunobservedvariable.Equivalenttofirstfindingimageofincrementinjointspace.

3

3.5

4

4.5

5

Unob

serv

ed S

tate

Var

iabl

e

−2 0 2 4Observed Variable

Increments

DARTTutorialSec'on4:Slide17

Page 18: DART Tutorial Sec’on 4: How should observaons impact an ...Single observed variable, single unobserved variable. So far, have known observaon likelihood for single variable. Now,

Ensemblefilters:Upda'ngaddi'onalpriorstatevariables

Havejointpriordistribu'onoftwovariables.Next,regresstheobservedvariableincrementsontoincrementsfortheunobservedvariable.Equivalenttofirstfindingimageofincrementinjointspace.

3

3.5

4

4.5

5

Unob

serv

ed S

tate

Var

iabl

e

−2 0 2 4Observed Variable

Increments

DARTTutorialSec'on4:Slide18

Page 19: DART Tutorial Sec’on 4: How should observaons impact an ...Single observed variable, single unobserved variable. So far, have known observaon likelihood for single variable. Now,

Ensemblefilters:Upda'ngaddi'onalpriorstatevariables

Havejointpriordistribu'onoftwovariables.Next,regresstheobservedvariableincrementsontoincrementsfortheunobservedvariable.Equivalenttofirstfindingimageofincrementinjointspace.

DARTTutorialSec'on4:Slide19

3

3.5

4

4.5

5

Unob

serv

ed S

tate

Var

iabl

e

−2 0 2 4Observed Variable

Increments

Page 20: DART Tutorial Sec’on 4: How should observaons impact an ...Single observed variable, single unobserved variable. So far, have known observaon likelihood for single variable. Now,

Ensemblefilters:Upda'ngaddi'onalpriorstatevariables

Havejointpriordistribu'onoftwovariables.Regression:Equivalenttofirstfindingimageofincrementinjointspace.Thenprojec'ngfromjointspaceontounobservedpriors.

3

3.5

4

4.5

5

Unob

serv

ed S

tate

Var

iabl

e

−2 0 2 4Observed Variable

Increments

DARTTutorialSec'on4:Slide20

Page 21: DART Tutorial Sec’on 4: How should observaons impact an ...Single observed variable, single unobserved variable. So far, have known observaon likelihood for single variable. Now,

Ensemblefilters:Upda'ngaddi'onalpriorstatevariables

Havejointpriordistribu'onoftwovariables.Regression:Equivalenttofirstfindingimageofincrementinjointspace.Thenprojec'ngfromjointspaceontounobservedpriors.

3

3.5

4

4.5

5

Unob

serv

ed S

tate

Var

iabl

e

−2 0 2 4Observed Variable

Increments

DARTTutorialSec'on4:Slide21

Page 22: DART Tutorial Sec’on 4: How should observaons impact an ...Single observed variable, single unobserved variable. So far, have known observaon likelihood for single variable. Now,

Ensemblefilters:Upda'ngaddi'onalpriorstatevariables

Havejointpriordistribu'onoftwovariables.Regression:Equivalenttofirstfindingimageofincrementinjointspace.Thenprojec'ngfromjointspaceontounobservedpriors.3

3.5

4

4.5

5

Unob

serv

ed S

tate

Var

iabl

e

−2 0 2 4Observed Variable

Increments

DARTTutorialSec'on4:Slide22

Page 23: DART Tutorial Sec’on 4: How should observaons impact an ...Single observed variable, single unobserved variable. So far, have known observaon likelihood for single variable. Now,

Ensemblefilters:Upda'ngaddi'onalpriorstatevariables

Havejointpriordistribu'onoftwovariables.Regression:Equivalenttofirstfindingimageofincrementinjointspace.Thenprojec'ngfromjointspaceontounobservedpriors.

DARTTutorialSec'on4:Slide23

3

3.5

4

4.5

5

Unob

serv

ed S

tate

Var

iabl

e

−2 0 2 4Observed Variable

Increments

Page 24: DART Tutorial Sec’on 4: How should observaons impact an ...Single observed variable, single unobserved variable. So far, have known observaon likelihood for single variable. Now,

Ensemblefilters:Upda'ngaddi'onalpriorstatevariables

Havejointpriordistribu'onoftwovariables.Regression:Equivalenttofirstfindingimageofincrementinjointspace.Thenprojec'ngfromjointspaceontounobservedpriors.3

3.5

4

4.5

5

Unob

serv

ed S

tate

Var

iabl

e

−2 0 2 4Observed Variable

Increments

DARTTutorialSec'on4:Slide24

Page 25: DART Tutorial Sec’on 4: How should observaons impact an ...Single observed variable, single unobserved variable. So far, have known observaon likelihood for single variable. Now,

Ensemblefilters:Upda'ngaddi'onalpriorstatevariables

Nowhaveanupdated(posterior)ensemblefortheunobservedvariable.FiangGaussiansshowsthatmeanandvariancehavechanged.Otherfeaturesofthepriordistribu'onmayalsohavechanged.3

3.5

4

4.5

5

Unob

serv

ed S

tate

Var

iabl

e

−2 0 2 4Obs.

DARTTutorialSec'on4:Slide25

Page 26: DART Tutorial Sec’on 4: How should observaons impact an ...Single observed variable, single unobserved variable. So far, have known observaon likelihood for single variable. Now,

Ensemblefilters:Upda'ngaddi'onalpriorstatevariables

Nowhaveanupdated(posterior)ensemblefortheunobservedvariable.FiangGaussiansshowsthatmeanandvariancehavechanged.Otherfeaturesofthepriordistribu'onmayalsohavechanged.3

3.5

4

4.5

5

Unob

serv

ed S

tate

Var

iabl

e

Prior State Fit

−2 0 2 4Obs.

DARTTutorialSec'on4:Slide26

Page 27: DART Tutorial Sec’on 4: How should observaons impact an ...Single observed variable, single unobserved variable. So far, have known observaon likelihood for single variable. Now,

Ensemblefilters:Upda'ngaddi'onalpriorstatevariables

Nowhaveanupdated(posterior)ensemblefortheunobservedvariable.FiangGaussiansshowsthatmeanandvariancehavechanged.Otherfeaturesofthepriordistribu'onmayalsohavechanged.3

3.5

4

4.5

5

Unob

serv

ed S

tate

Var

iabl

e

Prior State Fit

Posterior Fit

−2 0 2 4Obs.

DARTTutorialSec'on4:Slide27

Page 28: DART Tutorial Sec’on 4: How should observaons impact an ...Single observed variable, single unobserved variable. So far, have known observaon likelihood for single variable. Now,

Ensemblefilters:Upda'ngaddi'onalpriorstatevariables

CRITICALPOINT:Sinceimpactonunobservedvariableissimplyalinearregression,candothisINDEPENDENTLYforanynumberofunobservedvariables!Couldalsodomanyatonceusingmatrixalgebraasintradi'onalKalmanFilter.

3

3.5

4

4.5

5

Unob

serv

ed S

tate

Var

iabl

e

Prior State Fit

Posterior Fit

−2 0 2 4Obs.Anderson,J.L.,2003:

Alocalleastsquaresframeworkforensemblefiltering.Mon.Wea.Rev.,131,634-642

DARTTutorialSec'on4:Slide28

Page 29: DART Tutorial Sec’on 4: How should observaons impact an ...Single observed variable, single unobserved variable. So far, have known observaon likelihood for single variable. Now,

Mul'variateassimila'onwithDART:

Thebasicregressioncodeistrivial:However,DARTadvancedop'onscanobscurethecode.

Seeassimila/on_code/modules/assimila/on/assim_tools_mod.f90subroutine update_from_obs_inc

Togenerateoutputfromamul'variateLorenz_63experiment(thevalueofcutoffispresumedtobelarge–setinSec'on3):

cd models/lorenz_63/work; ./filter

NowdoMatlabdiagnos'cs(seesec'on1).

•  Doesmul'variatedobeker?•  Besuretorecordtheerrorvaluesforcomparison.•  Canyouiden'fyanyobviousperformancedifferences?

DARTTutorialSec'on4:Slide29

Page 30: DART Tutorial Sec’on 4: How should observaons impact an ...Single observed variable, single unobserved variable. So far, have known observaon likelihood for single variable. Now,

Mul'variateassimila'oninLorenz63:

Whathappensifnotallstatevariablesareobserved?

1.  Tryobservingonlyxandy(ignorezobserva'onsfromabove).Inmodels/lorenz_63/workeditinput.nml

&filter_nml … async = 0, adv_ens_command = "./advance_model.csh”,

obs_sequence_in_name = "obs_seq.out”, obs_sequence_out_name = "obs_seq.final”, …

Execute./filtertoproducenewassimila'on.

Lookattheerrorsta's'csand'meserieswithMatlab.

Recordtheerrorandspreadvaluesandcomparetounivariatecase.

DARTTutorialSec'on4:Slide30

Page 31: DART Tutorial Sec’on 4: How should observaons impact an ...Single observed variable, single unobserved variable. So far, have known observaon likelihood for single variable. Now,

Mul'variateassimila'oninLorenz63:

Whathappensifnotallstatevariablesareobserved?

2.  Tryobservingonlyx(ignorey and zobserva'onsfromabove).Inmodels/lorenz_63/workeditinput.nml

&filter_nml … obs_sequence_in_name = "obs_seq.out.xy” …

Execute./filtertoproducenewassimila'on.

Lookattheerrorsta's'csand'meserieswithMatlab.

Recordtheerrorandspreadvaluesandcomparetounivariatecase.

Whatwouldhappenedifwemadethisintoaunivariateassimila'on?&assim_tools_nml filter_kind = 1 cutoff = 1000000.0

DARTTutorialSec'on4:Slide31

Page 32: DART Tutorial Sec’on 4: How should observaons impact an ...Single observed variable, single unobserved variable. So far, have known observaon likelihood for single variable. Now,

Mul'variateassimila'oninLorenz63:

Whathappensifnotallstatevariablesareobserved?

3.  Tryobservingonlyz(ignorex and yobserva'onsfromabove).Inmodels/lorenz_63/workeditinput.nml

&filter_nml … obs_sequence_in_name = "obs_seq.out.x” …&assim_tools_nml … cutoff = 0.00001

Execute./filtertoproducenewassimila'on;lookattheerrorsta's'csand'meserieswithMatlab.

Recordtheerrorandspreadvaluesandcomparetounivariatecase.Dynamicsforxandyaresymmetric;zcanNOTdis'nguishthem.Howdowewantfiltertohandlethis?Doesitdowhatwewantinthiscase?

DARTTutorialSec'on4:Slide32

Page 33: DART Tutorial Sec’on 4: How should observaons impact an ...Single observed variable, single unobserved variable. So far, have known observaon likelihood for single variable. Now,

1.   FilteringForaOneVariableSystem2.   TheDARTDirectoryTree3.   DARTRun>meControlandDocumenta>on4.   Howshouldobserva>onsofastatevariableimpactanunobservedstatevariable?

Mul>variateassimila>on.5.   ComprehensiveFilteringTheory:Non-Iden>tyObserva>onsandtheJointPhaseSpace6.   OtherUpdatesforAnObservedVariable7.   SomeAddi>onalLow-OrderModels8.   DealingwithSamplingError9.   MoreonDealingwithError;Infla>on10.   RegressionandNonlinearEffects11.   Crea>ngDARTExecutables12.   Adap>veInfla>on13.   HierarchicalGroupFiltersandLocaliza>on14.   QualityControl15.   DARTExperiments:ControlandDesign16.   Diagnos>cOutput17.   Crea>ngObserva>onSequences18.   LostinPhaseSpace:TheChallengeofNotKnowingtheTruth19.   DART-CompliantModelsandMakingModelsCompliant20.   ModelParameterEs>ma>on21.   Observa>onTypesandObservingSystemDesign22.   ParallelAlgorithmImplementa>on23.  Loca'onmoduledesign(notavailable)24.  Fixedlagsmoother(notavailable)25.   Asimple1Dadvec>onmodel:TracerDataAssimila>on

DARTTutorialIndextoSec'ons

DARTTutorialSec'on4:Slide33