![Page 1: An Introduction to Nonlinear Principal Component AnalysisAn Introduction to Nonlinear Principal Component Analysis Adam Monahan monahana@uvic.ca School of Earth and Ocean Sciences](https://reader034.vdocuments.us/reader034/viewer/2022052519/5f1cd21dc17edf209e5ec73a/html5/thumbnails/1.jpg)
An Introduction toNonlinear
Principal Component AnalysisAdam [email protected]
School of Earth and Ocean SciencesUniversity of Victoria
An Introduction toNonlinearPrincipal Component Analysis – p. 1/33
![Page 2: An Introduction to Nonlinear Principal Component AnalysisAn Introduction to Nonlinear Principal Component Analysis Adam Monahan monahana@uvic.ca School of Earth and Ocean Sciences](https://reader034.vdocuments.us/reader034/viewer/2022052519/5f1cd21dc17edf209e5ec73a/html5/thumbnails/2.jpg)
OverviewDimensionality reduction
Principal Component AnalysisNonlinear PCA
theoryimplementation
Applications of NLPCALorenz attractorNH Tropospheric LFV
Conclusions
An Introduction toNonlinearPrincipal Component Analysis – p. 2/33
![Page 3: An Introduction to Nonlinear Principal Component AnalysisAn Introduction to Nonlinear Principal Component Analysis Adam Monahan monahana@uvic.ca School of Earth and Ocean Sciences](https://reader034.vdocuments.us/reader034/viewer/2022052519/5f1cd21dc17edf209e5ec73a/html5/thumbnails/3.jpg)
OverviewDimensionality reductionPrincipal Component Analysis
Nonlinear PCAtheoryimplementation
Applications of NLPCALorenz attractorNH Tropospheric LFV
Conclusions
An Introduction toNonlinearPrincipal Component Analysis – p. 2/33
![Page 4: An Introduction to Nonlinear Principal Component AnalysisAn Introduction to Nonlinear Principal Component Analysis Adam Monahan monahana@uvic.ca School of Earth and Ocean Sciences](https://reader034.vdocuments.us/reader034/viewer/2022052519/5f1cd21dc17edf209e5ec73a/html5/thumbnails/4.jpg)
OverviewDimensionality reductionPrincipal Component AnalysisNonlinear PCA
theoryimplementation
Applications of NLPCALorenz attractorNH Tropospheric LFV
Conclusions
An Introduction toNonlinearPrincipal Component Analysis – p. 2/33
![Page 5: An Introduction to Nonlinear Principal Component AnalysisAn Introduction to Nonlinear Principal Component Analysis Adam Monahan monahana@uvic.ca School of Earth and Ocean Sciences](https://reader034.vdocuments.us/reader034/viewer/2022052519/5f1cd21dc17edf209e5ec73a/html5/thumbnails/5.jpg)
OverviewDimensionality reductionPrincipal Component AnalysisNonlinear PCA
theory
implementation
Applications of NLPCALorenz attractorNH Tropospheric LFV
Conclusions
An Introduction toNonlinearPrincipal Component Analysis – p. 2/33
![Page 6: An Introduction to Nonlinear Principal Component AnalysisAn Introduction to Nonlinear Principal Component Analysis Adam Monahan monahana@uvic.ca School of Earth and Ocean Sciences](https://reader034.vdocuments.us/reader034/viewer/2022052519/5f1cd21dc17edf209e5ec73a/html5/thumbnails/6.jpg)
OverviewDimensionality reductionPrincipal Component AnalysisNonlinear PCA
theoryimplementation
Applications of NLPCALorenz attractorNH Tropospheric LFV
Conclusions
An Introduction toNonlinearPrincipal Component Analysis – p. 2/33
![Page 7: An Introduction to Nonlinear Principal Component AnalysisAn Introduction to Nonlinear Principal Component Analysis Adam Monahan monahana@uvic.ca School of Earth and Ocean Sciences](https://reader034.vdocuments.us/reader034/viewer/2022052519/5f1cd21dc17edf209e5ec73a/html5/thumbnails/7.jpg)
OverviewDimensionality reductionPrincipal Component AnalysisNonlinear PCA
theoryimplementation
Applications of NLPCA
Lorenz attractorNH Tropospheric LFV
Conclusions
An Introduction toNonlinearPrincipal Component Analysis – p. 2/33
![Page 8: An Introduction to Nonlinear Principal Component AnalysisAn Introduction to Nonlinear Principal Component Analysis Adam Monahan monahana@uvic.ca School of Earth and Ocean Sciences](https://reader034.vdocuments.us/reader034/viewer/2022052519/5f1cd21dc17edf209e5ec73a/html5/thumbnails/8.jpg)
OverviewDimensionality reductionPrincipal Component AnalysisNonlinear PCA
theoryimplementation
Applications of NLPCALorenz attractor
NH Tropospheric LFV
Conclusions
An Introduction toNonlinearPrincipal Component Analysis – p. 2/33
![Page 9: An Introduction to Nonlinear Principal Component AnalysisAn Introduction to Nonlinear Principal Component Analysis Adam Monahan monahana@uvic.ca School of Earth and Ocean Sciences](https://reader034.vdocuments.us/reader034/viewer/2022052519/5f1cd21dc17edf209e5ec73a/html5/thumbnails/9.jpg)
OverviewDimensionality reductionPrincipal Component AnalysisNonlinear PCA
theoryimplementation
Applications of NLPCALorenz attractorNH Tropospheric LFV
Conclusions
An Introduction toNonlinearPrincipal Component Analysis – p. 2/33
![Page 10: An Introduction to Nonlinear Principal Component AnalysisAn Introduction to Nonlinear Principal Component Analysis Adam Monahan monahana@uvic.ca School of Earth and Ocean Sciences](https://reader034.vdocuments.us/reader034/viewer/2022052519/5f1cd21dc17edf209e5ec73a/html5/thumbnails/10.jpg)
OverviewDimensionality reductionPrincipal Component AnalysisNonlinear PCA
theoryimplementation
Applications of NLPCALorenz attractorNH Tropospheric LFV
Conclusions
An Introduction toNonlinearPrincipal Component Analysis – p. 2/33
![Page 11: An Introduction to Nonlinear Principal Component AnalysisAn Introduction to Nonlinear Principal Component Analysis Adam Monahan monahana@uvic.ca School of Earth and Ocean Sciences](https://reader034.vdocuments.us/reader034/viewer/2022052519/5f1cd21dc17edf209e5ec73a/html5/thumbnails/11.jpg)
Dimensionality Reduction
Climate datasets made up of time series at individualstations/geographical locations
Typical dataset has P ∼ O(103) time seriesOrganised structure in atmosphere/ocean flows
⇒ time series at different locations not independent
⇒ data does not fill out isotropic cloud of points in RP ,but clusters around lower-dimensional surface(reflecting the “attractor”)Goal of dimensionality reduction in climatediagnostics is to characterise such structures inclimate datasets
An Introduction toNonlinearPrincipal Component Analysis – p. 3/33
![Page 12: An Introduction to Nonlinear Principal Component AnalysisAn Introduction to Nonlinear Principal Component Analysis Adam Monahan monahana@uvic.ca School of Earth and Ocean Sciences](https://reader034.vdocuments.us/reader034/viewer/2022052519/5f1cd21dc17edf209e5ec73a/html5/thumbnails/12.jpg)
Dimensionality Reduction
Climate datasets made up of time series at individualstations/geographical locations
Typical dataset has P ∼ O(103) time series
Organised structure in atmosphere/ocean flows⇒ time series at different locations not independent
⇒ data does not fill out isotropic cloud of points in RP ,but clusters around lower-dimensional surface(reflecting the “attractor”)Goal of dimensionality reduction in climatediagnostics is to characterise such structures inclimate datasets
An Introduction toNonlinearPrincipal Component Analysis – p. 3/33
![Page 13: An Introduction to Nonlinear Principal Component AnalysisAn Introduction to Nonlinear Principal Component Analysis Adam Monahan monahana@uvic.ca School of Earth and Ocean Sciences](https://reader034.vdocuments.us/reader034/viewer/2022052519/5f1cd21dc17edf209e5ec73a/html5/thumbnails/13.jpg)
Dimensionality Reduction
Climate datasets made up of time series at individualstations/geographical locations
Typical dataset has P ∼ O(103) time seriesOrganised structure in atmosphere/ocean flows
⇒ time series at different locations not independent
⇒ data does not fill out isotropic cloud of points in RP ,but clusters around lower-dimensional surface(reflecting the “attractor”)Goal of dimensionality reduction in climatediagnostics is to characterise such structures inclimate datasets
An Introduction toNonlinearPrincipal Component Analysis – p. 3/33
![Page 14: An Introduction to Nonlinear Principal Component AnalysisAn Introduction to Nonlinear Principal Component Analysis Adam Monahan monahana@uvic.ca School of Earth and Ocean Sciences](https://reader034.vdocuments.us/reader034/viewer/2022052519/5f1cd21dc17edf209e5ec73a/html5/thumbnails/14.jpg)
Dimensionality Reduction
Climate datasets made up of time series at individualstations/geographical locations
Typical dataset has P ∼ O(103) time seriesOrganised structure in atmosphere/ocean flows
⇒ time series at different locations not independent
⇒ data does not fill out isotropic cloud of points in RP ,but clusters around lower-dimensional surface(reflecting the “attractor”)Goal of dimensionality reduction in climatediagnostics is to characterise such structures inclimate datasets
An Introduction toNonlinearPrincipal Component Analysis – p. 3/33
![Page 15: An Introduction to Nonlinear Principal Component AnalysisAn Introduction to Nonlinear Principal Component Analysis Adam Monahan monahana@uvic.ca School of Earth and Ocean Sciences](https://reader034.vdocuments.us/reader034/viewer/2022052519/5f1cd21dc17edf209e5ec73a/html5/thumbnails/15.jpg)
Dimensionality Reduction
Climate datasets made up of time series at individualstations/geographical locations
Typical dataset has P ∼ O(103) time seriesOrganised structure in atmosphere/ocean flows
⇒ time series at different locations not independent
⇒ data does not fill out isotropic cloud of points in RP ,but clusters around lower-dimensional surface(reflecting the “attractor”)
Goal of dimensionality reduction in climatediagnostics is to characterise such structures inclimate datasets
An Introduction toNonlinearPrincipal Component Analysis – p. 3/33
![Page 16: An Introduction to Nonlinear Principal Component AnalysisAn Introduction to Nonlinear Principal Component Analysis Adam Monahan monahana@uvic.ca School of Earth and Ocean Sciences](https://reader034.vdocuments.us/reader034/viewer/2022052519/5f1cd21dc17edf209e5ec73a/html5/thumbnails/16.jpg)
Dimensionality Reduction
Climate datasets made up of time series at individualstations/geographical locations
Typical dataset has P ∼ O(103) time seriesOrganised structure in atmosphere/ocean flows
⇒ time series at different locations not independent
⇒ data does not fill out isotropic cloud of points in RP ,but clusters around lower-dimensional surface(reflecting the “attractor”)Goal of dimensionality reduction in climatediagnostics is to characterise such structures inclimate datasets
An Introduction toNonlinearPrincipal Component Analysis – p. 3/33
![Page 17: An Introduction to Nonlinear Principal Component AnalysisAn Introduction to Nonlinear Principal Component Analysis Adam Monahan monahana@uvic.ca School of Earth and Ocean Sciences](https://reader034.vdocuments.us/reader034/viewer/2022052519/5f1cd21dc17edf209e5ec73a/html5/thumbnails/17.jpg)
Dimensionality Reduction
Realising this goal has both theoretical and practicaldifficulties:
Theoretical:what is the precise definition of “structure”?how to formulate appropriate statistical model?
Practical:many important observational climate datasetsquite short, with O(10)−O(1000) statisticaldegrees of freedomwhat degree of “structure” can be robustlydiagnosed with existing data?
An Introduction toNonlinearPrincipal Component Analysis – p. 4/33
![Page 18: An Introduction to Nonlinear Principal Component AnalysisAn Introduction to Nonlinear Principal Component Analysis Adam Monahan monahana@uvic.ca School of Earth and Ocean Sciences](https://reader034.vdocuments.us/reader034/viewer/2022052519/5f1cd21dc17edf209e5ec73a/html5/thumbnails/18.jpg)
Dimensionality Reduction
Realising this goal has both theoretical and practicaldifficulties:Theoretical:
what is the precise definition of “structure”?how to formulate appropriate statistical model?
Practical:many important observational climate datasetsquite short, with O(10)−O(1000) statisticaldegrees of freedomwhat degree of “structure” can be robustlydiagnosed with existing data?
An Introduction toNonlinearPrincipal Component Analysis – p. 4/33
![Page 19: An Introduction to Nonlinear Principal Component AnalysisAn Introduction to Nonlinear Principal Component Analysis Adam Monahan monahana@uvic.ca School of Earth and Ocean Sciences](https://reader034.vdocuments.us/reader034/viewer/2022052519/5f1cd21dc17edf209e5ec73a/html5/thumbnails/19.jpg)
Dimensionality Reduction
Realising this goal has both theoretical and practicaldifficulties:Theoretical:
what is the precise definition of “structure”?
how to formulate appropriate statistical model?
Practical:many important observational climate datasetsquite short, with O(10)−O(1000) statisticaldegrees of freedomwhat degree of “structure” can be robustlydiagnosed with existing data?
An Introduction toNonlinearPrincipal Component Analysis – p. 4/33
![Page 20: An Introduction to Nonlinear Principal Component AnalysisAn Introduction to Nonlinear Principal Component Analysis Adam Monahan monahana@uvic.ca School of Earth and Ocean Sciences](https://reader034.vdocuments.us/reader034/viewer/2022052519/5f1cd21dc17edf209e5ec73a/html5/thumbnails/20.jpg)
Dimensionality Reduction
Realising this goal has both theoretical and practicaldifficulties:Theoretical:
what is the precise definition of “structure”?how to formulate appropriate statistical model?
Practical:many important observational climate datasetsquite short, with O(10)−O(1000) statisticaldegrees of freedomwhat degree of “structure” can be robustlydiagnosed with existing data?
An Introduction toNonlinearPrincipal Component Analysis – p. 4/33
![Page 21: An Introduction to Nonlinear Principal Component AnalysisAn Introduction to Nonlinear Principal Component Analysis Adam Monahan monahana@uvic.ca School of Earth and Ocean Sciences](https://reader034.vdocuments.us/reader034/viewer/2022052519/5f1cd21dc17edf209e5ec73a/html5/thumbnails/21.jpg)
Dimensionality Reduction
Realising this goal has both theoretical and practicaldifficulties:Theoretical:
what is the precise definition of “structure”?how to formulate appropriate statistical model?
Practical:
many important observational climate datasetsquite short, with O(10)−O(1000) statisticaldegrees of freedomwhat degree of “structure” can be robustlydiagnosed with existing data?
An Introduction toNonlinearPrincipal Component Analysis – p. 4/33
![Page 22: An Introduction to Nonlinear Principal Component AnalysisAn Introduction to Nonlinear Principal Component Analysis Adam Monahan monahana@uvic.ca School of Earth and Ocean Sciences](https://reader034.vdocuments.us/reader034/viewer/2022052519/5f1cd21dc17edf209e5ec73a/html5/thumbnails/22.jpg)
Dimensionality Reduction
Realising this goal has both theoretical and practicaldifficulties:Theoretical:
what is the precise definition of “structure”?how to formulate appropriate statistical model?
Practical:many important observational climate datasetsquite short, with O(10)−O(1000) statisticaldegrees of freedom
what degree of “structure” can be robustlydiagnosed with existing data?
An Introduction toNonlinearPrincipal Component Analysis – p. 4/33
![Page 23: An Introduction to Nonlinear Principal Component AnalysisAn Introduction to Nonlinear Principal Component Analysis Adam Monahan monahana@uvic.ca School of Earth and Ocean Sciences](https://reader034.vdocuments.us/reader034/viewer/2022052519/5f1cd21dc17edf209e5ec73a/html5/thumbnails/23.jpg)
Dimensionality Reduction
Realising this goal has both theoretical and practicaldifficulties:Theoretical:
what is the precise definition of “structure”?how to formulate appropriate statistical model?
Practical:many important observational climate datasetsquite short, with O(10)−O(1000) statisticaldegrees of freedomwhat degree of “structure” can be robustlydiagnosed with existing data?
An Introduction toNonlinearPrincipal Component Analysis – p. 4/33
![Page 24: An Introduction to Nonlinear Principal Component AnalysisAn Introduction to Nonlinear Principal Component Analysis Adam Monahan monahana@uvic.ca School of Earth and Ocean Sciences](https://reader034.vdocuments.us/reader034/viewer/2022052519/5f1cd21dc17edf209e5ec73a/html5/thumbnails/24.jpg)
Principal Component Analysis
A classical approach to dimensionalityprincipal component analysis (PCA)
Look for M -dimensional hyperplane approximation, optimalin least-squares sense
X(t) =M∑
k=1
〈X(t), ek〉 ek + ε(t)
minimising E {||ε2||}inner product often (not always) simple dot product
Vectors ek are the empirical orthogonal functions (EOFs)
An Introduction toNonlinearPrincipal Component Analysis – p. 5/33
![Page 25: An Introduction to Nonlinear Principal Component AnalysisAn Introduction to Nonlinear Principal Component Analysis Adam Monahan monahana@uvic.ca School of Earth and Ocean Sciences](https://reader034.vdocuments.us/reader034/viewer/2022052519/5f1cd21dc17edf209e5ec73a/html5/thumbnails/25.jpg)
Principal Component Analysis
A classical approach to dimensionalityprincipal component analysis (PCA)
Look for M -dimensional hyperplane approximation, optimalin least-squares sense
X(t) =M∑
k=1
〈X(t), ek〉 ek + ε(t)
minimising E {||ε2||}inner product often (not always) simple dot product
Vectors ek are the empirical orthogonal functions (EOFs)
An Introduction toNonlinearPrincipal Component Analysis – p. 5/33
![Page 26: An Introduction to Nonlinear Principal Component AnalysisAn Introduction to Nonlinear Principal Component Analysis Adam Monahan monahana@uvic.ca School of Earth and Ocean Sciences](https://reader034.vdocuments.us/reader034/viewer/2022052519/5f1cd21dc17edf209e5ec73a/html5/thumbnails/26.jpg)
Principal Component Analysis
A classical approach to dimensionalityprincipal component analysis (PCA)
Look for M -dimensional hyperplane approximation, optimalin least-squares sense
X(t) =M∑
k=1
〈X(t), ek〉 ek + ε(t)
minimising E {||ε2||}
inner product often (not always) simple dot product
Vectors ek are the empirical orthogonal functions (EOFs)
An Introduction toNonlinearPrincipal Component Analysis – p. 5/33
![Page 27: An Introduction to Nonlinear Principal Component AnalysisAn Introduction to Nonlinear Principal Component Analysis Adam Monahan monahana@uvic.ca School of Earth and Ocean Sciences](https://reader034.vdocuments.us/reader034/viewer/2022052519/5f1cd21dc17edf209e5ec73a/html5/thumbnails/27.jpg)
Principal Component Analysis
A classical approach to dimensionalityprincipal component analysis (PCA)
Look for M -dimensional hyperplane approximation, optimalin least-squares sense
X(t) =M∑
k=1
〈X(t), ek〉 ek + ε(t)
minimising E {||ε2||}inner product often (not always) simple dot product
Vectors ek are the empirical orthogonal functions (EOFs)
An Introduction toNonlinearPrincipal Component Analysis – p. 5/33
![Page 28: An Introduction to Nonlinear Principal Component AnalysisAn Introduction to Nonlinear Principal Component Analysis Adam Monahan monahana@uvic.ca School of Earth and Ocean Sciences](https://reader034.vdocuments.us/reader034/viewer/2022052519/5f1cd21dc17edf209e5ec73a/html5/thumbnails/28.jpg)
Principal Component Analysis
A classical approach to dimensionalityprincipal component analysis (PCA)
Look for M -dimensional hyperplane approximation, optimalin least-squares sense
X(t) =M∑
k=1
〈X(t), ek〉 ek + ε(t)
minimising E {||ε2||}inner product often (not always) simple dot product
Vectors ek are the empirical orthogonal functions (EOFs)
An Introduction toNonlinearPrincipal Component Analysis – p. 5/33
![Page 29: An Introduction to Nonlinear Principal Component AnalysisAn Introduction to Nonlinear Principal Component Analysis Adam Monahan monahana@uvic.ca School of Earth and Ocean Sciences](https://reader034.vdocuments.us/reader034/viewer/2022052519/5f1cd21dc17edf209e5ec73a/html5/thumbnails/29.jpg)
Principal Component Analysis
An Introduction toNonlinearPrincipal Component Analysis – p. 6/33
![Page 30: An Introduction to Nonlinear Principal Component AnalysisAn Introduction to Nonlinear Principal Component Analysis Adam Monahan monahana@uvic.ca School of Earth and Ocean Sciences](https://reader034.vdocuments.us/reader034/viewer/2022052519/5f1cd21dc17edf209e5ec73a/html5/thumbnails/30.jpg)
Principal Component Analysis
Operationally, EOFs are found as eigenvectors ofcovariance matrix (in appropriate norm)
PCA optimally efficient characterisation of GaussiandataMore generally: PCA provides optimallyparsimonious data compression for any datasetwhose distribution lies along orthogonal axesBut what if the underlying low-dimensionalstructure is curved rather than straight?(cigars vs. bananas)
An Introduction toNonlinearPrincipal Component Analysis – p. 7/33
![Page 31: An Introduction to Nonlinear Principal Component AnalysisAn Introduction to Nonlinear Principal Component Analysis Adam Monahan monahana@uvic.ca School of Earth and Ocean Sciences](https://reader034.vdocuments.us/reader034/viewer/2022052519/5f1cd21dc17edf209e5ec73a/html5/thumbnails/31.jpg)
Principal Component Analysis
Operationally, EOFs are found as eigenvectors ofcovariance matrix (in appropriate norm)PCA optimally efficient characterisation of Gaussiandata
More generally: PCA provides optimallyparsimonious data compression for any datasetwhose distribution lies along orthogonal axesBut what if the underlying low-dimensionalstructure is curved rather than straight?(cigars vs. bananas)
An Introduction toNonlinearPrincipal Component Analysis – p. 7/33
![Page 32: An Introduction to Nonlinear Principal Component AnalysisAn Introduction to Nonlinear Principal Component Analysis Adam Monahan monahana@uvic.ca School of Earth and Ocean Sciences](https://reader034.vdocuments.us/reader034/viewer/2022052519/5f1cd21dc17edf209e5ec73a/html5/thumbnails/32.jpg)
Principal Component Analysis
Operationally, EOFs are found as eigenvectors ofcovariance matrix (in appropriate norm)PCA optimally efficient characterisation of GaussiandataMore generally: PCA provides optimallyparsimonious data compression for any datasetwhose distribution lies along orthogonal axes
But what if the underlying low-dimensionalstructure is curved rather than straight?(cigars vs. bananas)
An Introduction toNonlinearPrincipal Component Analysis – p. 7/33
![Page 33: An Introduction to Nonlinear Principal Component AnalysisAn Introduction to Nonlinear Principal Component Analysis Adam Monahan monahana@uvic.ca School of Earth and Ocean Sciences](https://reader034.vdocuments.us/reader034/viewer/2022052519/5f1cd21dc17edf209e5ec73a/html5/thumbnails/33.jpg)
Principal Component Analysis
Operationally, EOFs are found as eigenvectors ofcovariance matrix (in appropriate norm)PCA optimally efficient characterisation of GaussiandataMore generally: PCA provides optimallyparsimonious data compression for any datasetwhose distribution lies along orthogonal axesBut what if the underlying low-dimensionalstructure is curved rather than straight?(cigars vs. bananas)
An Introduction toNonlinearPrincipal Component Analysis – p. 7/33
![Page 34: An Introduction to Nonlinear Principal Component AnalysisAn Introduction to Nonlinear Principal Component Analysis Adam Monahan monahana@uvic.ca School of Earth and Ocean Sciences](https://reader034.vdocuments.us/reader034/viewer/2022052519/5f1cd21dc17edf209e5ec73a/html5/thumbnails/34.jpg)
Nonlinear Low-Dimensional Structure
An Introduction toNonlinearPrincipal Component Analysis – p. 8/33
![Page 35: An Introduction to Nonlinear Principal Component AnalysisAn Introduction to Nonlinear Principal Component Analysis Adam Monahan monahana@uvic.ca School of Earth and Ocean Sciences](https://reader034.vdocuments.us/reader034/viewer/2022052519/5f1cd21dc17edf209e5ec73a/html5/thumbnails/35.jpg)
Nonlinear PCAAn approach to diagnosing nonlinear low-dimensionalstructure is Nonlinear PCA (NLPCA)
Goal: find functions (with M < P )
sf : RP → RM , f : RM → RP
such thatX(t) = (f ◦ sf ) (X(t)) + ε(t)
where
E {||ε2||} is minimised
f(λ) ∼ approximation manifold
λ(t) = sf (X(t)) ∼ manifold parameterisation (time series)
An Introduction toNonlinearPrincipal Component Analysis – p. 9/33
![Page 36: An Introduction to Nonlinear Principal Component AnalysisAn Introduction to Nonlinear Principal Component Analysis Adam Monahan monahana@uvic.ca School of Earth and Ocean Sciences](https://reader034.vdocuments.us/reader034/viewer/2022052519/5f1cd21dc17edf209e5ec73a/html5/thumbnails/36.jpg)
Nonlinear PCAAn approach to diagnosing nonlinear low-dimensionalstructure is Nonlinear PCA (NLPCA)
Goal: find functions (with M < P )
sf : RP → RM , f : RM → RP
such thatX(t) = (f ◦ sf ) (X(t)) + ε(t)
where
E {||ε2||} is minimised
f(λ) ∼ approximation manifold
λ(t) = sf (X(t)) ∼ manifold parameterisation (time series)
An Introduction toNonlinearPrincipal Component Analysis – p. 9/33
![Page 37: An Introduction to Nonlinear Principal Component AnalysisAn Introduction to Nonlinear Principal Component Analysis Adam Monahan monahana@uvic.ca School of Earth and Ocean Sciences](https://reader034.vdocuments.us/reader034/viewer/2022052519/5f1cd21dc17edf209e5ec73a/html5/thumbnails/37.jpg)
Nonlinear PCAAn approach to diagnosing nonlinear low-dimensionalstructure is Nonlinear PCA (NLPCA)
Goal: find functions (with M < P )
sf : RP → RM , f : RM → RP
such thatX(t) = (f ◦ sf ) (X(t)) + ε(t)
where
E {||ε2||} is minimised
f(λ) ∼ approximation manifold
λ(t) = sf (X(t)) ∼ manifold parameterisation (time series)
An Introduction toNonlinearPrincipal Component Analysis – p. 9/33
![Page 38: An Introduction to Nonlinear Principal Component AnalysisAn Introduction to Nonlinear Principal Component Analysis Adam Monahan monahana@uvic.ca School of Earth and Ocean Sciences](https://reader034.vdocuments.us/reader034/viewer/2022052519/5f1cd21dc17edf209e5ec73a/html5/thumbnails/38.jpg)
Nonlinear PCAAn approach to diagnosing nonlinear low-dimensionalstructure is Nonlinear PCA (NLPCA)
Goal: find functions (with M < P )
sf : RP → RM , f : RM → RP
such thatX(t) = (f ◦ sf ) (X(t)) + ε(t)
where
E {||ε2||} is minimised
f(λ) ∼ approximation manifold
λ(t) = sf (X(t)) ∼ manifold parameterisation (time series)
An Introduction toNonlinearPrincipal Component Analysis – p. 9/33
![Page 39: An Introduction to Nonlinear Principal Component AnalysisAn Introduction to Nonlinear Principal Component Analysis Adam Monahan monahana@uvic.ca School of Earth and Ocean Sciences](https://reader034.vdocuments.us/reader034/viewer/2022052519/5f1cd21dc17edf209e5ec73a/html5/thumbnails/39.jpg)
Nonlinear PCAAn approach to diagnosing nonlinear low-dimensionalstructure is Nonlinear PCA (NLPCA)
Goal: find functions (with M < P )
sf : RP → RM , f : RM → RP
such thatX(t) = (f ◦ sf ) (X(t)) + ε(t)
where
E {||ε2||} is minimised
f(λ) ∼ approximation manifold
λ(t) = sf (X(t)) ∼ manifold parameterisation (time series)
An Introduction toNonlinearPrincipal Component Analysis – p. 9/33
![Page 40: An Introduction to Nonlinear Principal Component AnalysisAn Introduction to Nonlinear Principal Component Analysis Adam Monahan monahana@uvic.ca School of Earth and Ocean Sciences](https://reader034.vdocuments.us/reader034/viewer/2022052519/5f1cd21dc17edf209e5ec73a/html5/thumbnails/40.jpg)
Nonlinear PCA
s f
f
tλ
X(t)
λ(t) = s f( X(t))
X(t) = ( f o s f)( X(t))
^
From Monahan, Fyfe, and Pandolfo (2003)
An Introduction toNonlinearPrincipal Component Analysis – p. 10/33
![Page 41: An Introduction to Nonlinear Principal Component AnalysisAn Introduction to Nonlinear Principal Component Analysis Adam Monahan monahana@uvic.ca School of Earth and Ocean Sciences](https://reader034.vdocuments.us/reader034/viewer/2022052519/5f1cd21dc17edf209e5ec73a/html5/thumbnails/41.jpg)
Nonlinear PCAAs with PCA, “fraction of variance explained” is ameasure of quality of approximation
PCA is a special case of NLPCAWhen implemented, NLPCA should reduce to PCAif:
data is Gaussiannot enough data is available to robustlycharacterise non-Gaussian structure
An Introduction toNonlinearPrincipal Component Analysis – p. 11/33
![Page 42: An Introduction to Nonlinear Principal Component AnalysisAn Introduction to Nonlinear Principal Component Analysis Adam Monahan monahana@uvic.ca School of Earth and Ocean Sciences](https://reader034.vdocuments.us/reader034/viewer/2022052519/5f1cd21dc17edf209e5ec73a/html5/thumbnails/42.jpg)
Nonlinear PCAAs with PCA, “fraction of variance explained” is ameasure of quality of approximationPCA is a special case of NLPCA
When implemented, NLPCA should reduce to PCAif:
data is Gaussiannot enough data is available to robustlycharacterise non-Gaussian structure
An Introduction toNonlinearPrincipal Component Analysis – p. 11/33
![Page 43: An Introduction to Nonlinear Principal Component AnalysisAn Introduction to Nonlinear Principal Component Analysis Adam Monahan monahana@uvic.ca School of Earth and Ocean Sciences](https://reader034.vdocuments.us/reader034/viewer/2022052519/5f1cd21dc17edf209e5ec73a/html5/thumbnails/43.jpg)
Nonlinear PCAAs with PCA, “fraction of variance explained” is ameasure of quality of approximationPCA is a special case of NLPCAWhen implemented, NLPCA should reduce to PCAif:
data is Gaussiannot enough data is available to robustlycharacterise non-Gaussian structure
An Introduction toNonlinearPrincipal Component Analysis – p. 11/33
![Page 44: An Introduction to Nonlinear Principal Component AnalysisAn Introduction to Nonlinear Principal Component Analysis Adam Monahan monahana@uvic.ca School of Earth and Ocean Sciences](https://reader034.vdocuments.us/reader034/viewer/2022052519/5f1cd21dc17edf209e5ec73a/html5/thumbnails/44.jpg)
Nonlinear PCAAs with PCA, “fraction of variance explained” is ameasure of quality of approximationPCA is a special case of NLPCAWhen implemented, NLPCA should reduce to PCAif:
data is Gaussian
not enough data is available to robustlycharacterise non-Gaussian structure
An Introduction toNonlinearPrincipal Component Analysis – p. 11/33
![Page 45: An Introduction to Nonlinear Principal Component AnalysisAn Introduction to Nonlinear Principal Component Analysis Adam Monahan monahana@uvic.ca School of Earth and Ocean Sciences](https://reader034.vdocuments.us/reader034/viewer/2022052519/5f1cd21dc17edf209e5ec73a/html5/thumbnails/45.jpg)
Nonlinear PCAAs with PCA, “fraction of variance explained” is ameasure of quality of approximationPCA is a special case of NLPCAWhen implemented, NLPCA should reduce to PCAif:
data is Gaussiannot enough data is available to robustlycharacterise non-Gaussian structure
An Introduction toNonlinearPrincipal Component Analysis – p. 11/33
![Page 46: An Introduction to Nonlinear Principal Component AnalysisAn Introduction to Nonlinear Principal Component Analysis Adam Monahan monahana@uvic.ca School of Earth and Ocean Sciences](https://reader034.vdocuments.us/reader034/viewer/2022052519/5f1cd21dc17edf209e5ec73a/html5/thumbnails/46.jpg)
NLPCA: Implementation
Implemented NLPCA using neural networks(convenient, not necessary)
Parameter estimation more difficult than for PCAPCA model is linear in statistical parameters:
Y = MX
so variational problem has unique analytic solutionNLPCA model nonlinear in model parameters, sosolution
may not be uniquemust be found through numerical minimisation
An Introduction toNonlinearPrincipal Component Analysis – p. 12/33
![Page 47: An Introduction to Nonlinear Principal Component AnalysisAn Introduction to Nonlinear Principal Component Analysis Adam Monahan monahana@uvic.ca School of Earth and Ocean Sciences](https://reader034.vdocuments.us/reader034/viewer/2022052519/5f1cd21dc17edf209e5ec73a/html5/thumbnails/47.jpg)
NLPCA: Implementation
Implemented NLPCA using neural networks(convenient, not necessary)Parameter estimation more difficult than for PCA
PCA model is linear in statistical parameters:
Y = MX
so variational problem has unique analytic solutionNLPCA model nonlinear in model parameters, sosolution
may not be uniquemust be found through numerical minimisation
An Introduction toNonlinearPrincipal Component Analysis – p. 12/33
![Page 48: An Introduction to Nonlinear Principal Component AnalysisAn Introduction to Nonlinear Principal Component Analysis Adam Monahan monahana@uvic.ca School of Earth and Ocean Sciences](https://reader034.vdocuments.us/reader034/viewer/2022052519/5f1cd21dc17edf209e5ec73a/html5/thumbnails/48.jpg)
NLPCA: Implementation
Implemented NLPCA using neural networks(convenient, not necessary)Parameter estimation more difficult than for PCAPCA model is linear in statistical parameters:
Y = MX
so variational problem has unique analytic solution
NLPCA model nonlinear in model parameters, sosolution
may not be uniquemust be found through numerical minimisation
An Introduction toNonlinearPrincipal Component Analysis – p. 12/33
![Page 49: An Introduction to Nonlinear Principal Component AnalysisAn Introduction to Nonlinear Principal Component Analysis Adam Monahan monahana@uvic.ca School of Earth and Ocean Sciences](https://reader034.vdocuments.us/reader034/viewer/2022052519/5f1cd21dc17edf209e5ec73a/html5/thumbnails/49.jpg)
NLPCA: Implementation
Implemented NLPCA using neural networks(convenient, not necessary)Parameter estimation more difficult than for PCAPCA model is linear in statistical parameters:
Y = MX
so variational problem has unique analytic solutionNLPCA model nonlinear in model parameters, sosolution
may not be uniquemust be found through numerical minimisation
An Introduction toNonlinearPrincipal Component Analysis – p. 12/33
![Page 50: An Introduction to Nonlinear Principal Component AnalysisAn Introduction to Nonlinear Principal Component Analysis Adam Monahan monahana@uvic.ca School of Earth and Ocean Sciences](https://reader034.vdocuments.us/reader034/viewer/2022052519/5f1cd21dc17edf209e5ec73a/html5/thumbnails/50.jpg)
NLPCA: Implementation
Implemented NLPCA using neural networks(convenient, not necessary)Parameter estimation more difficult than for PCAPCA model is linear in statistical parameters:
Y = MX
so variational problem has unique analytic solutionNLPCA model nonlinear in model parameters, sosolution
may not be unique
must be found through numerical minimisation
An Introduction toNonlinearPrincipal Component Analysis – p. 12/33
![Page 51: An Introduction to Nonlinear Principal Component AnalysisAn Introduction to Nonlinear Principal Component Analysis Adam Monahan monahana@uvic.ca School of Earth and Ocean Sciences](https://reader034.vdocuments.us/reader034/viewer/2022052519/5f1cd21dc17edf209e5ec73a/html5/thumbnails/51.jpg)
NLPCA: Implementation
Implemented NLPCA using neural networks(convenient, not necessary)Parameter estimation more difficult than for PCAPCA model is linear in statistical parameters:
Y = MX
so variational problem has unique analytic solutionNLPCA model nonlinear in model parameters, sosolution
may not be uniquemust be found through numerical minimisation
An Introduction toNonlinearPrincipal Component Analysis – p. 12/33
![Page 52: An Introduction to Nonlinear Principal Component AnalysisAn Introduction to Nonlinear Principal Component Analysis Adam Monahan monahana@uvic.ca School of Earth and Ocean Sciences](https://reader034.vdocuments.us/reader034/viewer/2022052519/5f1cd21dc17edf209e5ec73a/html5/thumbnails/52.jpg)
NLPCA: Parameter EstimationTwo fundamental issues regarding parameterestimation common to all statistical models:
Reproducibilitymodel must be robust to the introduction of newdatanew observations shouldn’t fundamentallychange model
Classifiability:model must be robust to details of optimisationproceduremodel shouldn’t depend on initial parametervalues
An Introduction toNonlinearPrincipal Component Analysis – p. 13/33
![Page 53: An Introduction to Nonlinear Principal Component AnalysisAn Introduction to Nonlinear Principal Component Analysis Adam Monahan monahana@uvic.ca School of Earth and Ocean Sciences](https://reader034.vdocuments.us/reader034/viewer/2022052519/5f1cd21dc17edf209e5ec73a/html5/thumbnails/53.jpg)
NLPCA: Parameter EstimationTwo fundamental issues regarding parameterestimation common to all statistical models:Reproducibility
model must be robust to the introduction of newdatanew observations shouldn’t fundamentallychange model
Classifiability:model must be robust to details of optimisationproceduremodel shouldn’t depend on initial parametervalues
An Introduction toNonlinearPrincipal Component Analysis – p. 13/33
![Page 54: An Introduction to Nonlinear Principal Component AnalysisAn Introduction to Nonlinear Principal Component Analysis Adam Monahan monahana@uvic.ca School of Earth and Ocean Sciences](https://reader034.vdocuments.us/reader034/viewer/2022052519/5f1cd21dc17edf209e5ec73a/html5/thumbnails/54.jpg)
NLPCA: Parameter EstimationTwo fundamental issues regarding parameterestimation common to all statistical models:Reproducibility
model must be robust to the introduction of newdata
new observations shouldn’t fundamentallychange model
Classifiability:model must be robust to details of optimisationproceduremodel shouldn’t depend on initial parametervalues
An Introduction toNonlinearPrincipal Component Analysis – p. 13/33
![Page 55: An Introduction to Nonlinear Principal Component AnalysisAn Introduction to Nonlinear Principal Component Analysis Adam Monahan monahana@uvic.ca School of Earth and Ocean Sciences](https://reader034.vdocuments.us/reader034/viewer/2022052519/5f1cd21dc17edf209e5ec73a/html5/thumbnails/55.jpg)
NLPCA: Parameter EstimationTwo fundamental issues regarding parameterestimation common to all statistical models:Reproducibility
model must be robust to the introduction of newdatanew observations shouldn’t fundamentallychange model
Classifiability:model must be robust to details of optimisationproceduremodel shouldn’t depend on initial parametervalues
An Introduction toNonlinearPrincipal Component Analysis – p. 13/33
![Page 56: An Introduction to Nonlinear Principal Component AnalysisAn Introduction to Nonlinear Principal Component Analysis Adam Monahan monahana@uvic.ca School of Earth and Ocean Sciences](https://reader034.vdocuments.us/reader034/viewer/2022052519/5f1cd21dc17edf209e5ec73a/html5/thumbnails/56.jpg)
NLPCA: Parameter EstimationTwo fundamental issues regarding parameterestimation common to all statistical models:Reproducibility
model must be robust to the introduction of newdatanew observations shouldn’t fundamentallychange model
Classifiability:
model must be robust to details of optimisationproceduremodel shouldn’t depend on initial parametervalues
An Introduction toNonlinearPrincipal Component Analysis – p. 13/33
![Page 57: An Introduction to Nonlinear Principal Component AnalysisAn Introduction to Nonlinear Principal Component Analysis Adam Monahan monahana@uvic.ca School of Earth and Ocean Sciences](https://reader034.vdocuments.us/reader034/viewer/2022052519/5f1cd21dc17edf209e5ec73a/html5/thumbnails/57.jpg)
NLPCA: Parameter EstimationTwo fundamental issues regarding parameterestimation common to all statistical models:Reproducibility
model must be robust to the introduction of newdatanew observations shouldn’t fundamentallychange model
Classifiability:model must be robust to details of optimisationprocedure
model shouldn’t depend on initial parametervalues
An Introduction toNonlinearPrincipal Component Analysis – p. 13/33
![Page 58: An Introduction to Nonlinear Principal Component AnalysisAn Introduction to Nonlinear Principal Component Analysis Adam Monahan monahana@uvic.ca School of Earth and Ocean Sciences](https://reader034.vdocuments.us/reader034/viewer/2022052519/5f1cd21dc17edf209e5ec73a/html5/thumbnails/58.jpg)
NLPCA: Parameter EstimationTwo fundamental issues regarding parameterestimation common to all statistical models:Reproducibility
model must be robust to the introduction of newdatanew observations shouldn’t fundamentallychange model
Classifiability:model must be robust to details of optimisationproceduremodel shouldn’t depend on initial parametervalues
An Introduction toNonlinearPrincipal Component Analysis – p. 13/33
![Page 59: An Introduction to Nonlinear Principal Component AnalysisAn Introduction to Nonlinear Principal Component Analysis Adam Monahan monahana@uvic.ca School of Earth and Ocean Sciences](https://reader034.vdocuments.us/reader034/viewer/2022052519/5f1cd21dc17edf209e5ec73a/html5/thumbnails/59.jpg)
NLPCA: Synthetic Gaussian Data
Synthetic Gaussian dataAn Introduction toNonlinearPrincipal Component Analysis – p. 14/33
![Page 60: An Introduction to Nonlinear Principal Component AnalysisAn Introduction to Nonlinear Principal Component Analysis Adam Monahan monahana@uvic.ca School of Earth and Ocean Sciences](https://reader034.vdocuments.us/reader034/viewer/2022052519/5f1cd21dc17edf209e5ec73a/html5/thumbnails/60.jpg)
Applications of NLPCA: Lorenz Attractor
ScatterplotsAn Introduction toNonlinearPrincipal Component Analysis – p. 15/33
![Page 61: An Introduction to Nonlinear Principal Component AnalysisAn Introduction to Nonlinear Principal Component Analysis Adam Monahan monahana@uvic.ca School of Earth and Ocean Sciences](https://reader034.vdocuments.us/reader034/viewer/2022052519/5f1cd21dc17edf209e5ec73a/html5/thumbnails/61.jpg)
Applications of NLPCA: Lorenz Attractor
1D PCA approximation (60%)An Introduction toNonlinearPrincipal Component Analysis – p. 16/33
![Page 62: An Introduction to Nonlinear Principal Component AnalysisAn Introduction to Nonlinear Principal Component Analysis Adam Monahan monahana@uvic.ca School of Earth and Ocean Sciences](https://reader034.vdocuments.us/reader034/viewer/2022052519/5f1cd21dc17edf209e5ec73a/html5/thumbnails/62.jpg)
Applications of NLPCA: Lorenz Attractor
1D NLPCA approximation (76%)An Introduction toNonlinearPrincipal Component Analysis – p. 17/33
![Page 63: An Introduction to Nonlinear Principal Component AnalysisAn Introduction to Nonlinear Principal Component Analysis Adam Monahan monahana@uvic.ca School of Earth and Ocean Sciences](https://reader034.vdocuments.us/reader034/viewer/2022052519/5f1cd21dc17edf209e5ec73a/html5/thumbnails/63.jpg)
Applications of NLPCA: Lorenz Attractor
2D PCA approximation (94%)An Introduction toNonlinearPrincipal Component Analysis – p. 18/33
![Page 64: An Introduction to Nonlinear Principal Component AnalysisAn Introduction to Nonlinear Principal Component Analysis Adam Monahan monahana@uvic.ca School of Earth and Ocean Sciences](https://reader034.vdocuments.us/reader034/viewer/2022052519/5f1cd21dc17edf209e5ec73a/html5/thumbnails/64.jpg)
Applications of NLPCA: Lorenz Attractor
2D NLPCA approximation (97%)An Introduction toNonlinearPrincipal Component Analysis – p. 19/33
![Page 65: An Introduction to Nonlinear Principal Component AnalysisAn Introduction to Nonlinear Principal Component Analysis Adam Monahan monahana@uvic.ca School of Earth and Ocean Sciences](https://reader034.vdocuments.us/reader034/viewer/2022052519/5f1cd21dc17edf209e5ec73a/html5/thumbnails/65.jpg)
Applications of NLPCA: NH Tropospheric LFV
EOF1
EOF2
10-day lowpass-filtered 500 hPa geopotential height EOFs
An Introduction toNonlinearPrincipal Component Analysis – p. 20/33
![Page 66: An Introduction to Nonlinear Principal Component AnalysisAn Introduction to Nonlinear Principal Component Analysis Adam Monahan monahana@uvic.ca School of Earth and Ocean Sciences](https://reader034.vdocuments.us/reader034/viewer/2022052519/5f1cd21dc17edf209e5ec73a/html5/thumbnails/66.jpg)
Applications of NLPCA: NH Tropospheric LFV
1D NLPCA Approximation: spatial structure(PCA: 14.8%; NLPCA 18.4%)
An Introduction toNonlinearPrincipal Component Analysis – p. 21/33
![Page 67: An Introduction to Nonlinear Principal Component AnalysisAn Introduction to Nonlinear Principal Component Analysis Adam Monahan monahana@uvic.ca School of Earth and Ocean Sciences](https://reader034.vdocuments.us/reader034/viewer/2022052519/5f1cd21dc17edf209e5ec73a/html5/thumbnails/67.jpg)
Applications of NLPCA: NH Tropospheric LFV
1D NLPCA Approximation: pdf of time seriesAn Introduction toNonlinearPrincipal Component Analysis – p. 22/33
![Page 68: An Introduction to Nonlinear Principal Component AnalysisAn Introduction to Nonlinear Principal Component Analysis Adam Monahan monahana@uvic.ca School of Earth and Ocean Sciences](https://reader034.vdocuments.us/reader034/viewer/2022052519/5f1cd21dc17edf209e5ec73a/html5/thumbnails/68.jpg)
Applications of NLPCA: NH Tropospheric LFV
1D NLPCA Approximation: regime maps
An Introduction toNonlinearPrincipal Component Analysis – p. 23/33
![Page 69: An Introduction to Nonlinear Principal Component AnalysisAn Introduction to Nonlinear Principal Component Analysis Adam Monahan monahana@uvic.ca School of Earth and Ocean Sciences](https://reader034.vdocuments.us/reader034/viewer/2022052519/5f1cd21dc17edf209e5ec73a/html5/thumbnails/69.jpg)
Applications of NLPCA: NH Tropospheric LFV
1D NLPCA Approximation: interannual variabilityAn Introduction toNonlinearPrincipal Component Analysis – p. 24/33
![Page 70: An Introduction to Nonlinear Principal Component AnalysisAn Introduction to Nonlinear Principal Component Analysis Adam Monahan monahana@uvic.ca School of Earth and Ocean Sciences](https://reader034.vdocuments.us/reader034/viewer/2022052519/5f1cd21dc17edf209e5ec73a/html5/thumbnails/70.jpg)
NLPCA: Limitations and DrawbacksParameter estimation in NLPCA (as in any nonlinearstatistical model) must be done very carefully toensure robust approximation
⇒ analysis time-consuming, data hungry⇒ insufficiently careful analysis leads to spurious
results (e.g. Christiansen, 2005)Theoretical underpinning of NLPCA is weak
⇒ no “rigorous” theory of sampling variabilityInformation theory may provide new tools with
better sampling propertiesbetter theoretical basis
An Introduction toNonlinearPrincipal Component Analysis – p. 25/33
![Page 71: An Introduction to Nonlinear Principal Component AnalysisAn Introduction to Nonlinear Principal Component Analysis Adam Monahan monahana@uvic.ca School of Earth and Ocean Sciences](https://reader034.vdocuments.us/reader034/viewer/2022052519/5f1cd21dc17edf209e5ec73a/html5/thumbnails/71.jpg)
NLPCA: Limitations and DrawbacksParameter estimation in NLPCA (as in any nonlinearstatistical model) must be done very carefully toensure robust approximation
⇒ analysis time-consuming, data hungry
⇒ insufficiently careful analysis leads to spuriousresults (e.g. Christiansen, 2005)Theoretical underpinning of NLPCA is weak
⇒ no “rigorous” theory of sampling variabilityInformation theory may provide new tools with
better sampling propertiesbetter theoretical basis
An Introduction toNonlinearPrincipal Component Analysis – p. 25/33
![Page 72: An Introduction to Nonlinear Principal Component AnalysisAn Introduction to Nonlinear Principal Component Analysis Adam Monahan monahana@uvic.ca School of Earth and Ocean Sciences](https://reader034.vdocuments.us/reader034/viewer/2022052519/5f1cd21dc17edf209e5ec73a/html5/thumbnails/72.jpg)
NLPCA: Limitations and DrawbacksParameter estimation in NLPCA (as in any nonlinearstatistical model) must be done very carefully toensure robust approximation
⇒ analysis time-consuming, data hungry⇒ insufficiently careful analysis leads to spurious
results (e.g. Christiansen, 2005)
Theoretical underpinning of NLPCA is weak⇒ no “rigorous” theory of sampling variability
Information theory may provide new tools withbetter sampling propertiesbetter theoretical basis
An Introduction toNonlinearPrincipal Component Analysis – p. 25/33
![Page 73: An Introduction to Nonlinear Principal Component AnalysisAn Introduction to Nonlinear Principal Component Analysis Adam Monahan monahana@uvic.ca School of Earth and Ocean Sciences](https://reader034.vdocuments.us/reader034/viewer/2022052519/5f1cd21dc17edf209e5ec73a/html5/thumbnails/73.jpg)
NLPCA: Limitations and DrawbacksParameter estimation in NLPCA (as in any nonlinearstatistical model) must be done very carefully toensure robust approximation
⇒ analysis time-consuming, data hungry⇒ insufficiently careful analysis leads to spurious
results (e.g. Christiansen, 2005)Theoretical underpinning of NLPCA is weak
⇒ no “rigorous” theory of sampling variabilityInformation theory may provide new tools with
better sampling propertiesbetter theoretical basis
An Introduction toNonlinearPrincipal Component Analysis – p. 25/33
![Page 74: An Introduction to Nonlinear Principal Component AnalysisAn Introduction to Nonlinear Principal Component Analysis Adam Monahan monahana@uvic.ca School of Earth and Ocean Sciences](https://reader034.vdocuments.us/reader034/viewer/2022052519/5f1cd21dc17edf209e5ec73a/html5/thumbnails/74.jpg)
NLPCA: Limitations and DrawbacksParameter estimation in NLPCA (as in any nonlinearstatistical model) must be done very carefully toensure robust approximation
⇒ analysis time-consuming, data hungry⇒ insufficiently careful analysis leads to spurious
results (e.g. Christiansen, 2005)Theoretical underpinning of NLPCA is weak
⇒ no “rigorous” theory of sampling variability
Information theory may provide new tools withbetter sampling propertiesbetter theoretical basis
An Introduction toNonlinearPrincipal Component Analysis – p. 25/33
![Page 75: An Introduction to Nonlinear Principal Component AnalysisAn Introduction to Nonlinear Principal Component Analysis Adam Monahan monahana@uvic.ca School of Earth and Ocean Sciences](https://reader034.vdocuments.us/reader034/viewer/2022052519/5f1cd21dc17edf209e5ec73a/html5/thumbnails/75.jpg)
NLPCA: Limitations and DrawbacksParameter estimation in NLPCA (as in any nonlinearstatistical model) must be done very carefully toensure robust approximation
⇒ analysis time-consuming, data hungry⇒ insufficiently careful analysis leads to spurious
results (e.g. Christiansen, 2005)Theoretical underpinning of NLPCA is weak
⇒ no “rigorous” theory of sampling variabilityInformation theory may provide new tools with
better sampling propertiesbetter theoretical basis
An Introduction toNonlinearPrincipal Component Analysis – p. 25/33
![Page 76: An Introduction to Nonlinear Principal Component AnalysisAn Introduction to Nonlinear Principal Component Analysis Adam Monahan monahana@uvic.ca School of Earth and Ocean Sciences](https://reader034.vdocuments.us/reader034/viewer/2022052519/5f1cd21dc17edf209e5ec73a/html5/thumbnails/76.jpg)
NLPCA: Limitations and DrawbacksParameter estimation in NLPCA (as in any nonlinearstatistical model) must be done very carefully toensure robust approximation
⇒ analysis time-consuming, data hungry⇒ insufficiently careful analysis leads to spurious
results (e.g. Christiansen, 2005)Theoretical underpinning of NLPCA is weak
⇒ no “rigorous” theory of sampling variabilityInformation theory may provide new tools with
better sampling properties
better theoretical basis
An Introduction toNonlinearPrincipal Component Analysis – p. 25/33
![Page 77: An Introduction to Nonlinear Principal Component AnalysisAn Introduction to Nonlinear Principal Component Analysis Adam Monahan monahana@uvic.ca School of Earth and Ocean Sciences](https://reader034.vdocuments.us/reader034/viewer/2022052519/5f1cd21dc17edf209e5ec73a/html5/thumbnails/77.jpg)
NLPCA: Limitations and DrawbacksParameter estimation in NLPCA (as in any nonlinearstatistical model) must be done very carefully toensure robust approximation
⇒ analysis time-consuming, data hungry⇒ insufficiently careful analysis leads to spurious
results (e.g. Christiansen, 2005)Theoretical underpinning of NLPCA is weak
⇒ no “rigorous” theory of sampling variabilityInformation theory may provide new tools with
better sampling propertiesbetter theoretical basis
An Introduction toNonlinearPrincipal Component Analysis – p. 25/33
![Page 78: An Introduction to Nonlinear Principal Component AnalysisAn Introduction to Nonlinear Principal Component Analysis Adam Monahan monahana@uvic.ca School of Earth and Ocean Sciences](https://reader034.vdocuments.us/reader034/viewer/2022052519/5f1cd21dc17edf209e5ec73a/html5/thumbnails/78.jpg)
ConclusionsTraditional PCA optimal for dimensionalityreduction only if data distribution falls alongorthogonal axes
Can define nonlinear generalisation, NLPCA, whichcan robustly characterise nonlinear low-dimensionalstructure in datasetsNLPCA approximations can provide afundamentally different characterisation of data thanPCA approximationsImplementation of NLPCA difficult and lacking inunderlying theory; represents a first attempt at a big(and challenging) problem
An Introduction toNonlinearPrincipal Component Analysis – p. 26/33
![Page 79: An Introduction to Nonlinear Principal Component AnalysisAn Introduction to Nonlinear Principal Component Analysis Adam Monahan monahana@uvic.ca School of Earth and Ocean Sciences](https://reader034.vdocuments.us/reader034/viewer/2022052519/5f1cd21dc17edf209e5ec73a/html5/thumbnails/79.jpg)
ConclusionsTraditional PCA optimal for dimensionalityreduction only if data distribution falls alongorthogonal axesCan define nonlinear generalisation, NLPCA, whichcan robustly characterise nonlinear low-dimensionalstructure in datasets
NLPCA approximations can provide afundamentally different characterisation of data thanPCA approximationsImplementation of NLPCA difficult and lacking inunderlying theory; represents a first attempt at a big(and challenging) problem
An Introduction toNonlinearPrincipal Component Analysis – p. 26/33
![Page 80: An Introduction to Nonlinear Principal Component AnalysisAn Introduction to Nonlinear Principal Component Analysis Adam Monahan monahana@uvic.ca School of Earth and Ocean Sciences](https://reader034.vdocuments.us/reader034/viewer/2022052519/5f1cd21dc17edf209e5ec73a/html5/thumbnails/80.jpg)
ConclusionsTraditional PCA optimal for dimensionalityreduction only if data distribution falls alongorthogonal axesCan define nonlinear generalisation, NLPCA, whichcan robustly characterise nonlinear low-dimensionalstructure in datasetsNLPCA approximations can provide afundamentally different characterisation of data thanPCA approximations
Implementation of NLPCA difficult and lacking inunderlying theory; represents a first attempt at a big(and challenging) problem
An Introduction toNonlinearPrincipal Component Analysis – p. 26/33
![Page 81: An Introduction to Nonlinear Principal Component AnalysisAn Introduction to Nonlinear Principal Component Analysis Adam Monahan monahana@uvic.ca School of Earth and Ocean Sciences](https://reader034.vdocuments.us/reader034/viewer/2022052519/5f1cd21dc17edf209e5ec73a/html5/thumbnails/81.jpg)
ConclusionsTraditional PCA optimal for dimensionalityreduction only if data distribution falls alongorthogonal axesCan define nonlinear generalisation, NLPCA, whichcan robustly characterise nonlinear low-dimensionalstructure in datasetsNLPCA approximations can provide afundamentally different characterisation of data thanPCA approximationsImplementation of NLPCA difficult and lacking inunderlying theory; represents a first attempt at a big(and challenging) problem
An Introduction toNonlinearPrincipal Component Analysis – p. 26/33
![Page 82: An Introduction to Nonlinear Principal Component AnalysisAn Introduction to Nonlinear Principal Component Analysis Adam Monahan monahana@uvic.ca School of Earth and Ocean Sciences](https://reader034.vdocuments.us/reader034/viewer/2022052519/5f1cd21dc17edf209e5ec73a/html5/thumbnails/82.jpg)
Acknowledgements
William Hsieh (UBC)Lionel Pandolfo (UBC)John Fyfe (CCCma)Qiaobin Teng (CCCma)Benyang Tang (JPL)
An Introduction toNonlinearPrincipal Component Analysis – p. 27/33
![Page 83: An Introduction to Nonlinear Principal Component AnalysisAn Introduction to Nonlinear Principal Component Analysis Adam Monahan monahana@uvic.ca School of Earth and Ocean Sciences](https://reader034.vdocuments.us/reader034/viewer/2022052519/5f1cd21dc17edf209e5ec73a/html5/thumbnails/83.jpg)
Parameter Estimation in NLPCAAn ensemble approach was taken
For a large number N (∼ 50) of trials:data was randomly split into training andvalidation sets (taking autocorrelation intoaccount)a random initial parameter set was selected
For each ensemble member, iterative minimisationprocedure carried out until either:
error over training data stopped changingerror over validation data started increasing
Method does not look for global error minimum
An Introduction toNonlinearPrincipal Component Analysis – p. 28/33
![Page 84: An Introduction to Nonlinear Principal Component AnalysisAn Introduction to Nonlinear Principal Component Analysis Adam Monahan monahana@uvic.ca School of Earth and Ocean Sciences](https://reader034.vdocuments.us/reader034/viewer/2022052519/5f1cd21dc17edf209e5ec73a/html5/thumbnails/84.jpg)
Parameter Estimation in NLPCAAn ensemble approach was takenFor a large number N (∼ 50) of trials:
data was randomly split into training andvalidation sets (taking autocorrelation intoaccount)a random initial parameter set was selected
For each ensemble member, iterative minimisationprocedure carried out until either:
error over training data stopped changingerror over validation data started increasing
Method does not look for global error minimum
An Introduction toNonlinearPrincipal Component Analysis – p. 28/33
![Page 85: An Introduction to Nonlinear Principal Component AnalysisAn Introduction to Nonlinear Principal Component Analysis Adam Monahan monahana@uvic.ca School of Earth and Ocean Sciences](https://reader034.vdocuments.us/reader034/viewer/2022052519/5f1cd21dc17edf209e5ec73a/html5/thumbnails/85.jpg)
Parameter Estimation in NLPCAAn ensemble approach was takenFor a large number N (∼ 50) of trials:
data was randomly split into training andvalidation sets (taking autocorrelation intoaccount)
a random initial parameter set was selected
For each ensemble member, iterative minimisationprocedure carried out until either:
error over training data stopped changingerror over validation data started increasing
Method does not look for global error minimum
An Introduction toNonlinearPrincipal Component Analysis – p. 28/33
![Page 86: An Introduction to Nonlinear Principal Component AnalysisAn Introduction to Nonlinear Principal Component Analysis Adam Monahan monahana@uvic.ca School of Earth and Ocean Sciences](https://reader034.vdocuments.us/reader034/viewer/2022052519/5f1cd21dc17edf209e5ec73a/html5/thumbnails/86.jpg)
Parameter Estimation in NLPCAAn ensemble approach was takenFor a large number N (∼ 50) of trials:
data was randomly split into training andvalidation sets (taking autocorrelation intoaccount)a random initial parameter set was selected
For each ensemble member, iterative minimisationprocedure carried out until either:
error over training data stopped changingerror over validation data started increasing
Method does not look for global error minimum
An Introduction toNonlinearPrincipal Component Analysis – p. 28/33
![Page 87: An Introduction to Nonlinear Principal Component AnalysisAn Introduction to Nonlinear Principal Component Analysis Adam Monahan monahana@uvic.ca School of Earth and Ocean Sciences](https://reader034.vdocuments.us/reader034/viewer/2022052519/5f1cd21dc17edf209e5ec73a/html5/thumbnails/87.jpg)
Parameter Estimation in NLPCAAn ensemble approach was takenFor a large number N (∼ 50) of trials:
data was randomly split into training andvalidation sets (taking autocorrelation intoaccount)a random initial parameter set was selected
For each ensemble member, iterative minimisationprocedure carried out until either:
error over training data stopped changingerror over validation data started increasing
Method does not look for global error minimum
An Introduction toNonlinearPrincipal Component Analysis – p. 28/33
![Page 88: An Introduction to Nonlinear Principal Component AnalysisAn Introduction to Nonlinear Principal Component Analysis Adam Monahan monahana@uvic.ca School of Earth and Ocean Sciences](https://reader034.vdocuments.us/reader034/viewer/2022052519/5f1cd21dc17edf209e5ec73a/html5/thumbnails/88.jpg)
Parameter Estimation in NLPCAAn ensemble approach was takenFor a large number N (∼ 50) of trials:
data was randomly split into training andvalidation sets (taking autocorrelation intoaccount)a random initial parameter set was selected
For each ensemble member, iterative minimisationprocedure carried out until either:
error over training data stopped changing
error over validation data started increasing
Method does not look for global error minimum
An Introduction toNonlinearPrincipal Component Analysis – p. 28/33
![Page 89: An Introduction to Nonlinear Principal Component AnalysisAn Introduction to Nonlinear Principal Component Analysis Adam Monahan monahana@uvic.ca School of Earth and Ocean Sciences](https://reader034.vdocuments.us/reader034/viewer/2022052519/5f1cd21dc17edf209e5ec73a/html5/thumbnails/89.jpg)
Parameter Estimation in NLPCAAn ensemble approach was takenFor a large number N (∼ 50) of trials:
data was randomly split into training andvalidation sets (taking autocorrelation intoaccount)a random initial parameter set was selected
For each ensemble member, iterative minimisationprocedure carried out until either:
error over training data stopped changingerror over validation data started increasing
Method does not look for global error minimum
An Introduction toNonlinearPrincipal Component Analysis – p. 28/33
![Page 90: An Introduction to Nonlinear Principal Component AnalysisAn Introduction to Nonlinear Principal Component Analysis Adam Monahan monahana@uvic.ca School of Earth and Ocean Sciences](https://reader034.vdocuments.us/reader034/viewer/2022052519/5f1cd21dc17edf209e5ec73a/html5/thumbnails/90.jpg)
Parameter Estimation in NLPCAAn ensemble approach was takenFor a large number N (∼ 50) of trials:
data was randomly split into training andvalidation sets (taking autocorrelation intoaccount)a random initial parameter set was selected
For each ensemble member, iterative minimisationprocedure carried out until either:
error over training data stopped changingerror over validation data started increasing
Method does not look for global error minimum
An Introduction toNonlinearPrincipal Component Analysis – p. 28/33
![Page 91: An Introduction to Nonlinear Principal Component AnalysisAn Introduction to Nonlinear Principal Component Analysis Adam Monahan monahana@uvic.ca School of Earth and Ocean Sciences](https://reader034.vdocuments.us/reader034/viewer/2022052519/5f1cd21dc17edf209e5ec73a/html5/thumbnails/91.jpg)
Parameter Estimation in NLPCAEnsemble member becomes candidate model if
⟨||ε||2
⟩validation
≤⟨||ε||2
⟩training
Candidate models comparedif they share same shape and orientation
⇒ approximation is robustif they differ in shape and orientation
⇒ approximation is not robust
If approximation not robust, model simplified &procedure repeated until robust model found
An Introduction toNonlinearPrincipal Component Analysis – p. 29/33
![Page 92: An Introduction to Nonlinear Principal Component AnalysisAn Introduction to Nonlinear Principal Component Analysis Adam Monahan monahana@uvic.ca School of Earth and Ocean Sciences](https://reader034.vdocuments.us/reader034/viewer/2022052519/5f1cd21dc17edf209e5ec73a/html5/thumbnails/92.jpg)
Parameter Estimation in NLPCAEnsemble member becomes candidate model if
⟨||ε||2
⟩validation
≤⟨||ε||2
⟩training
Candidate models compared
if they share same shape and orientation⇒ approximation is robust
if they differ in shape and orientation⇒ approximation is not robust
If approximation not robust, model simplified &procedure repeated until robust model found
An Introduction toNonlinearPrincipal Component Analysis – p. 29/33
![Page 93: An Introduction to Nonlinear Principal Component AnalysisAn Introduction to Nonlinear Principal Component Analysis Adam Monahan monahana@uvic.ca School of Earth and Ocean Sciences](https://reader034.vdocuments.us/reader034/viewer/2022052519/5f1cd21dc17edf209e5ec73a/html5/thumbnails/93.jpg)
Parameter Estimation in NLPCAEnsemble member becomes candidate model if
⟨||ε||2
⟩validation
≤⟨||ε||2
⟩training
Candidate models comparedif they share same shape and orientation
⇒ approximation is robustif they differ in shape and orientation
⇒ approximation is not robust
If approximation not robust, model simplified &procedure repeated until robust model found
An Introduction toNonlinearPrincipal Component Analysis – p. 29/33
![Page 94: An Introduction to Nonlinear Principal Component AnalysisAn Introduction to Nonlinear Principal Component Analysis Adam Monahan monahana@uvic.ca School of Earth and Ocean Sciences](https://reader034.vdocuments.us/reader034/viewer/2022052519/5f1cd21dc17edf209e5ec73a/html5/thumbnails/94.jpg)
Parameter Estimation in NLPCAEnsemble member becomes candidate model if
⟨||ε||2
⟩validation
≤⟨||ε||2
⟩training
Candidate models comparedif they share same shape and orientation
⇒ approximation is robust
if they differ in shape and orientation⇒ approximation is not robust
If approximation not robust, model simplified &procedure repeated until robust model found
An Introduction toNonlinearPrincipal Component Analysis – p. 29/33
![Page 95: An Introduction to Nonlinear Principal Component AnalysisAn Introduction to Nonlinear Principal Component Analysis Adam Monahan monahana@uvic.ca School of Earth and Ocean Sciences](https://reader034.vdocuments.us/reader034/viewer/2022052519/5f1cd21dc17edf209e5ec73a/html5/thumbnails/95.jpg)
Parameter Estimation in NLPCAEnsemble member becomes candidate model if
⟨||ε||2
⟩validation
≤⟨||ε||2
⟩training
Candidate models comparedif they share same shape and orientation
⇒ approximation is robustif they differ in shape and orientation
⇒ approximation is not robust
If approximation not robust, model simplified &procedure repeated until robust model found
An Introduction toNonlinearPrincipal Component Analysis – p. 29/33
![Page 96: An Introduction to Nonlinear Principal Component AnalysisAn Introduction to Nonlinear Principal Component Analysis Adam Monahan monahana@uvic.ca School of Earth and Ocean Sciences](https://reader034.vdocuments.us/reader034/viewer/2022052519/5f1cd21dc17edf209e5ec73a/html5/thumbnails/96.jpg)
Parameter Estimation in NLPCAEnsemble member becomes candidate model if
⟨||ε||2
⟩validation
≤⟨||ε||2
⟩training
Candidate models comparedif they share same shape and orientation
⇒ approximation is robustif they differ in shape and orientation
⇒ approximation is not robust
If approximation not robust, model simplified &procedure repeated until robust model found
An Introduction toNonlinearPrincipal Component Analysis – p. 29/33
![Page 97: An Introduction to Nonlinear Principal Component AnalysisAn Introduction to Nonlinear Principal Component Analysis Adam Monahan monahana@uvic.ca School of Earth and Ocean Sciences](https://reader034.vdocuments.us/reader034/viewer/2022052519/5f1cd21dc17edf209e5ec73a/html5/thumbnails/97.jpg)
Parameter Estimation in NLPCAEnsemble member becomes candidate model if
⟨||ε||2
⟩validation
≤⟨||ε||2
⟩training
Candidate models comparedif they share same shape and orientation
⇒ approximation is robustif they differ in shape and orientation
⇒ approximation is not robustIf approximation not robust, model simplified &procedure repeated until robust model found
An Introduction toNonlinearPrincipal Component Analysis – p. 29/33
![Page 98: An Introduction to Nonlinear Principal Component AnalysisAn Introduction to Nonlinear Principal Component Analysis Adam Monahan monahana@uvic.ca School of Earth and Ocean Sciences](https://reader034.vdocuments.us/reader034/viewer/2022052519/5f1cd21dc17edf209e5ec73a/html5/thumbnails/98.jpg)
Parameter Estimation in NLPCAProcedure will ultimately yield PCA solution if norobust non-Gaussian structure present
Such a careful procedure necessary to avoid findingspurious non-Gaussian structure
An Introduction toNonlinearPrincipal Component Analysis – p. 30/33
![Page 99: An Introduction to Nonlinear Principal Component AnalysisAn Introduction to Nonlinear Principal Component Analysis Adam Monahan monahana@uvic.ca School of Earth and Ocean Sciences](https://reader034.vdocuments.us/reader034/viewer/2022052519/5f1cd21dc17edf209e5ec73a/html5/thumbnails/99.jpg)
Parameter Estimation in NLPCAProcedure will ultimately yield PCA solution if norobust non-Gaussian structure presentSuch a careful procedure necessary to avoid findingspurious non-Gaussian structure
An Introduction toNonlinearPrincipal Component Analysis – p. 30/33
![Page 100: An Introduction to Nonlinear Principal Component AnalysisAn Introduction to Nonlinear Principal Component Analysis Adam Monahan monahana@uvic.ca School of Earth and Ocean Sciences](https://reader034.vdocuments.us/reader034/viewer/2022052519/5f1cd21dc17edf209e5ec73a/html5/thumbnails/100.jpg)
Applications of NLPCA: Tropical Pacific SST
EOF PatternsAn Introduction toNonlinearPrincipal Component Analysis – p. 31/33
![Page 101: An Introduction to Nonlinear Principal Component AnalysisAn Introduction to Nonlinear Principal Component Analysis Adam Monahan monahana@uvic.ca School of Earth and Ocean Sciences](https://reader034.vdocuments.us/reader034/viewer/2022052519/5f1cd21dc17edf209e5ec73a/html5/thumbnails/101.jpg)
Applications of NLPCA: Tropical Pacific SST
1D NLPCA Approximation: spatial structure
An Introduction toNonlinearPrincipal Component Analysis – p. 32/33
![Page 102: An Introduction to Nonlinear Principal Component AnalysisAn Introduction to Nonlinear Principal Component Analysis Adam Monahan monahana@uvic.ca School of Earth and Ocean Sciences](https://reader034.vdocuments.us/reader034/viewer/2022052519/5f1cd21dc17edf209e5ec73a/html5/thumbnails/102.jpg)
Applications of NLPCA: Tropical Pacific SST
1D NLPCA Approximation: spatial structure
An Introduction toNonlinearPrincipal Component Analysis – p. 33/33