yosuke morishima
DESCRIPTION
Yosuke Morishima. Pattern Recognition and Machine Learning. Chapter 13 : Sequential Data. Contents in latter part. Linear Dynamical Systems What is different from HMM? Kalman filter Its strength and limitation Particle Filter Its simple algorithm and broad application - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Yosuke Morishima](https://reader035.vdocuments.us/reader035/viewer/2022062305/568163dc550346895dd5345b/html5/thumbnails/1.jpg)
Yosuke Morishima
PATTERN RECOGNITION AND MACHINE LEARNINGCHAPTER 13: SEQUENTIAL DATA
![Page 2: Yosuke Morishima](https://reader035.vdocuments.us/reader035/viewer/2022062305/568163dc550346895dd5345b/html5/thumbnails/2.jpg)
Contents in latter part
Linear Dynamical SystemsWhat is different from HMM?
Kalman filterIts strength and limitation
Particle FilterIts simple algorithm and broad application Application in Neuroscience
![Page 3: Yosuke Morishima](https://reader035.vdocuments.us/reader035/viewer/2022062305/568163dc550346895dd5345b/html5/thumbnails/3.jpg)
Dynamical SystemsState space model of Dynamical systems
Now latent variable is continuous rather than discrete
),( 1 wzFz nn
),( vzHx nn
State equation
Observation equation
![Page 4: Yosuke Morishima](https://reader035.vdocuments.us/reader035/viewer/2022062305/568163dc550346895dd5345b/html5/thumbnails/4.jpg)
Dynamical SystemsState space model of Dynamical systems
Now latent variable is continuous rather than discrete
)|( 1nn zzp
)|( nn zxpor
),( 1 wzFz nn
),( vzHx nn
![Page 5: Yosuke Morishima](https://reader035.vdocuments.us/reader035/viewer/2022062305/568163dc550346895dd5345b/html5/thumbnails/5.jpg)
Linear Dynamical SystemsSpecial case of dynamical systems
Gaussian assumption on distribution
Wherez: Latent variablex: ObservationA: Dynamics matrixC: Emission matrixw, v, u: Noise
![Page 6: Yosuke Morishima](https://reader035.vdocuments.us/reader035/viewer/2022062305/568163dc550346895dd5345b/html5/thumbnails/6.jpg)
Linear Dynamical SystemsBayesian form of LDS
Since LDS is linear Gaussian model, joint distribution over all latent and observed variables is simply Gaussian.
![Page 7: Yosuke Morishima](https://reader035.vdocuments.us/reader035/viewer/2022062305/568163dc550346895dd5345b/html5/thumbnails/7.jpg)
Kalman Filter
• Kalman filter does exact inference in LDS in which all latent and observed variables are Gaussian (incl. multivariate Gaussian).
• Kalman filter handles multiple dimensions in a single set of calculations.
• Kalman filter has two distinct phases: Predict and Update
![Page 8: Yosuke Morishima](https://reader035.vdocuments.us/reader035/viewer/2022062305/568163dc550346895dd5345b/html5/thumbnails/8.jpg)
Application of Kalman Filter
Tracking moving object
Blue: True positionGreen: MeasurementRed: Post. estimate
![Page 9: Yosuke Morishima](https://reader035.vdocuments.us/reader035/viewer/2022062305/568163dc550346895dd5345b/html5/thumbnails/9.jpg)
Two phases in Kalman Filter
PredictPrediction of state estimate and estimate covariance
Update
Update of state estimate and estimate covariance with Kaman gain
![Page 10: Yosuke Morishima](https://reader035.vdocuments.us/reader035/viewer/2022062305/568163dc550346895dd5345b/html5/thumbnails/10.jpg)
Estimation of Parameter in LDS
Distribution of Zn-1 is used as a prior for estimation of Zn
Red:Blue: Red:Green:Blue:
Predict Update
![Page 11: Yosuke Morishima](https://reader035.vdocuments.us/reader035/viewer/2022062305/568163dc550346895dd5345b/html5/thumbnails/11.jpg)
Derivation of Kalman Filter
• We use sum-product algorithm for efficient inference of latent variables.
• LDS is continuous case of HMM (sum -> integer)
where
![Page 12: Yosuke Morishima](https://reader035.vdocuments.us/reader035/viewer/2022062305/568163dc550346895dd5345b/html5/thumbnails/12.jpg)
Sum-product algorithm
Solve…
where
where (Kalman Gain Matrix)
![Page 13: Yosuke Morishima](https://reader035.vdocuments.us/reader035/viewer/2022062305/568163dc550346895dd5345b/html5/thumbnails/13.jpg)
What we have estimated?
Predict:
Update:
![Page 14: Yosuke Morishima](https://reader035.vdocuments.us/reader035/viewer/2022062305/568163dc550346895dd5345b/html5/thumbnails/14.jpg)
What we have estimated?
Predicted mean of Zn Predicted mean of XnObserved Xn
Prediction errorPredict:
Update:
![Page 15: Yosuke Morishima](https://reader035.vdocuments.us/reader035/viewer/2022062305/568163dc550346895dd5345b/html5/thumbnails/15.jpg)
Parameter Estimation with EM algorithm
• E step: Given , run the inference algorithm to determine distribution of latent variable
• M step: Given , maximize the complete-data log likelihood function wrt q
Where
where
![Page 16: Yosuke Morishima](https://reader035.vdocuments.us/reader035/viewer/2022062305/568163dc550346895dd5345b/html5/thumbnails/16.jpg)
Limitation of Kalman Filter• Due to assumption of Gaussian distribution in KF,
KF can not estimate well in nonlinear/non-Gaussian problem.
• One simple extension is mixture of Gaussians• In mixture of K Gaussians, is mixture of K
Gaussians, and will comprise mixture of Kn Gaussians. -> Computationally intractable
![Page 17: Yosuke Morishima](https://reader035.vdocuments.us/reader035/viewer/2022062305/568163dc550346895dd5345b/html5/thumbnails/17.jpg)
Limitation of Kalman Filter
• To resolve nonlinear dynamical system problem, other methods are developed.• Extended Kalman filter: Gaussian approximation by
linearizing around the mean of predicted distribution• Particle filter: Resampling method, see later• Switching state-space model: continuous type of
switching HMM
![Page 18: Yosuke Morishima](https://reader035.vdocuments.us/reader035/viewer/2022062305/568163dc550346895dd5345b/html5/thumbnails/18.jpg)
Particle Filter
• In nonlinear/non-Gaussian dynamical systems, it is hard to estimate posterior distribution by KF.
• Apply the sampling-importance-resampling (SIR) to obtain a sequential Monte Carlo algorithm, particle filter.
• Advantages of Particle Filter• Simple algorithm -> Easy to implement• Good estimation in nonlinear/non-Gaussian problem
![Page 19: Yosuke Morishima](https://reader035.vdocuments.us/reader035/viewer/2022062305/568163dc550346895dd5345b/html5/thumbnails/19.jpg)
How to Represent Distribution
Original distribution(mixture of Gaussian)
Gaussian approximation
Approximation by PF(distribution of particle)
Approximation by PF(histogram)
![Page 20: Yosuke Morishima](https://reader035.vdocuments.us/reader035/viewer/2022062305/568163dc550346895dd5345b/html5/thumbnails/20.jpg)
Where’s a landmine?Use metal detector to find a landmine (orange star).
![Page 21: Yosuke Morishima](https://reader035.vdocuments.us/reader035/viewer/2022062305/568163dc550346895dd5345b/html5/thumbnails/21.jpg)
Where’s a landmine?Random survey in the field (red circle).The filter draws a number of randomly distributed estimates, called particles. All particles are given the same likelihood
![Page 22: Yosuke Morishima](https://reader035.vdocuments.us/reader035/viewer/2022062305/568163dc550346895dd5345b/html5/thumbnails/22.jpg)
Where’s a landmine?Get response from each point (strength: size).Assign a likelihood to each particle such that the particular particle can explain the measurement.
![Page 23: Yosuke Morishima](https://reader035.vdocuments.us/reader035/viewer/2022062305/568163dc550346895dd5345b/html5/thumbnails/23.jpg)
Where’s a landmine?Decide the place to survey in next step.Scale the weight of particles to select the particle for resampling
![Page 24: Yosuke Morishima](https://reader035.vdocuments.us/reader035/viewer/2022062305/568163dc550346895dd5345b/html5/thumbnails/24.jpg)
Where’s a landmine?Intensive survey of possible placeDraw random particles based upon their likelihood (Resampling).High likelihood -> more particle; low likelihood -> less particle All particle have equal likelihood again.
![Page 25: Yosuke Morishima](https://reader035.vdocuments.us/reader035/viewer/2022062305/568163dc550346895dd5345b/html5/thumbnails/25.jpg)
Operation of Particle Filter
![Page 26: Yosuke Morishima](https://reader035.vdocuments.us/reader035/viewer/2022062305/568163dc550346895dd5345b/html5/thumbnails/26.jpg)
Algorithm of Particle Filter• Sample representation of the posterior distribution
p(zn|Xn) expressed as a samples {z(l)n} with corresponding weights {w(l)n}.
• Draw samples from mixture distribution
• Use new observation xn+1 to evaluate the corresponding weights .
where
where
![Page 27: Yosuke Morishima](https://reader035.vdocuments.us/reader035/viewer/2022062305/568163dc550346895dd5345b/html5/thumbnails/27.jpg)
Limitation of Particle Filter
• In high dimensional models, enormous particles are required to approximate posterior distribution.
• Repeated resampling cause degeneracy of algorithm.• All but one of the importance weights are close to zero• Avoided by proper choice of resampling method
![Page 28: Yosuke Morishima](https://reader035.vdocuments.us/reader035/viewer/2022062305/568163dc550346895dd5345b/html5/thumbnails/28.jpg)
Application to Neuroeconomics
Estimation of Q value (latent variable) in reward tracking task
Samejima, Science 05
Prew(L)-Prew (R)
![Page 29: Yosuke Morishima](https://reader035.vdocuments.us/reader035/viewer/2022062305/568163dc550346895dd5345b/html5/thumbnails/29.jpg)
Model framework
a: learning rateg: discount factorb: temperature parameterQ: action valuea: actionr: reward
![Page 30: Yosuke Morishima](https://reader035.vdocuments.us/reader035/viewer/2022062305/568163dc550346895dd5345b/html5/thumbnails/30.jpg)
Estimation of Q
![Page 31: Yosuke Morishima](https://reader035.vdocuments.us/reader035/viewer/2022062305/568163dc550346895dd5345b/html5/thumbnails/31.jpg)
Demonstration
• Estimation of Q value in reward seeking taskCode is provided by Kazu Samejima
• Comparison of performance btw with/without resampling.Code is provided by Klaas Enno Stephan