Sharing Features among Dynamical Systems with Beta Processes

Emily B. Fox, Erik B. Sudderth, Michael I. Jordan, Alan S. Willsky

NIPS 2009

Some pictures and slides are adopted from Emily Fox’s paper and slides in NIPS 2009

Outline

• Introduction

• Sticky HDP-HMM

• Beta process autoregressive HMM (BP-AR-HMM)

• Experiment Results

• Conclusion

Introduction

• Problem: To discover and model common behaviors in related time series.

• Jointly model multiple time series– Transfer knowledge between related time series– Uncover interesting relationships

• Standard way to represent the set of behaviors time series exhibit via associated features:

Dynamic Behavior

Binary matrix

If implies that time series i exhibit feature k

Sticky HDP-HMM

• HDP-HMM assumes all time series exhibit the same behaviors in same way

picture is adopted from Emily Fox’s paper in ICML 2008

The beta process distributed measure is

represented by its masses and locations as

BP-AR-HMM Model

Shared set of features or behaviors

Beta process

prior

masses

BP-AR-HMM Model

Transition patterns for object i

Features indicating behaviors of object i

BP-AR-HMM Model

Latent dynamical mode for object i

Switching Vector Autoregressive (VAR) Process with order r

BP-AR-HMM Inference

• Metropolis-Hastings binary feature sampling

• “Birth and dead” reversible jump MCMC (RJMCMC) Proposal Distribution:

is defined similarly, but using where

BP-AR-HMM Inference

• MH acceptance probability is

where

• MH sampling is also used for updating and

Synthetic Experiments

• Five time series are generated from

with

Motion Capture (MoCap) Experiments

• 6 CMU MoCap exercise routines

• Each routine includes some of the following 12 motions: running in place, jumping jacks, arm circles, side twists, knee raises,

squats, punching, up and down, two variants of toe touches, arch over, and a reach out stretch

• Observation: 62-dimension

• 25 chains of the sampler for 15,000 burn-in and 5000 collections

• Chosen MCMC sample minimizes an expected Hamming distance criterion

MoCap Results• Share motions: Jumping jacks, Side twists, Arm circles, Squats• Unique motion: Bend Over, Punching, Toe Touch 1, Toe Touch 2• Split motions: knee raises, up and down, running in place

Comparison with Other Models

Conclusion

• Bayesian nonparametric framework for discovering dynamical features common to multiple time series

• Novel exact sampling algorithm for non-conjugate beta process models