statistical object identification, tracking, and analysis

AISR Briefing // Michael Turmon, JPL 1

Statistical Object Identification, Tracking, and Analysis

Michael Turmon

Jet Propulsion Laboratory/Caltech

AISR Program Meeting

NASA Ames Conference Center

4 April 2005


Object Tracking and Analysis: Overview

Identification: Find the objects in multispectral science images Tracking: Link identified objects in series of images Trajectory Analysis: Model and classify object tracks

Identification

Allow scientists in domains like solar physics and atmosphere & ocean circulation to understand great volumes of temporal data in directly informative terms

Sunspot and Facula Regions in a Solar Quadrant15 November 1998 and the next five days; using MDI imagery

Tracking Trajectory analysis

Move scientists beyond looking at pixels to understanding phenomena


Project Activities

Scope: Demonstrate object analysis technology in three application areas: solar physics, geophysics (GPS), oceans/atmospheres

Highlights (June 2001 – June 2004)– Sunspots tracked over seven years’ images

- 100 GB of imagery distilled into 2500 object histories, < 1GB

- Related high-cadence track dataset also analyzed

– Integration of object tracks with DS9 browser

– Fast Kalman models developed and tested for GPS time series

– New hidden Markov classification of seismograph time series - Developed and used new constrained optimization methods

Future activities– Perform analysis of high-cadence solar data

– Tune segmentation and object tracking models for HMI


Technology Overview

Identification– Per-class mixture models drive Markov random field segmentation

- Trained using combination of expert-provided labels and unclassified pixels

– Aggregate connected components into objects Tracking

– Compute index of object overlap (past -> future)– Associate current objects to past objects to optimize total overlap

Object analysis– Continuous and discrete modeling of object path and characteristics– Basic subroutines are Kalman smoother and forward-backward recursion


Photogram

Magnetogram SNQ

Key:S(pot)F(acula)Q(uiet sun)

Flexible, general methods using statisticalmodels to identify objects in images S

F

Q

1: Experts identify classesin sample images

2: Learned model performsclassification automatically

MagneticField

Lig

htIn

tens

ity

Labeling

Labeling by inferredstatistical model

Q

SF

Identification: Integrating Multimode Imagery

Can not distinguish classes from just one observable– Move beyond ad hoc threshold rules to allow arbitrary class separators

Select model by using sample images labeled by scientists


Unlabeled Data

Labeled Data

sunspot

quiet

facula

faculaquiet

sunspot

Previous Feature->Class Map New Feature->Class Map

Identification: Partly-Labeled Data

Hand Labeling: time-consuming, expensive, asks much of scientists. Data from some feature classes (e.g., background) is easy to identify;

small amounts of labeled data can be obtained with care.- E.g., scatter plot at left: 15K quiet examples + 607 sunspot + 340 facula

- Technical challenge: ensure atypical distribution of labeled data does not affect learned class proportions.

– Developed methods using partly-classified data to bootstrap vast amounts of unlabeled data, seamlessly in same clustering algorithm.

- Selected 100K examples from 10B total, 30K labeled — mostly quiet background.

Yields >20% improvement in sunspot classification accuracy, and >25% improvement in facula classification accuracy.


Identification: ResultsTurmon et al., “Statistical Pattern Recognition for Labeling Solar Active Regions:

Application to SoHO/MDI Imagery,” Astrophysical Journal, March 20 2002, 396-407.


Feature Identification: Publications

The mixture modeling work appeared in:- Mixtures-2001, “Recent Developments in Mixture Modelling,” Hamburg

- Compstat-2004, Prague, as “Symmetric Normal Mixtures”

Work comparing our MDI labelings to other observatories:– Harry Jones (Kitt Peak Nat’l Solar Obs.) & Steve Walton (San Fernando Obs.)

- J. Pap, H. Jones, M. Turmon & L. Floyd, “Study of the SOHO/VIRGO Irradiance Variations using MDI and Kitt Peak images,” Proc. SOHO-11 Workshop, Davos, 2002.

- H.P. Jones, M. Turmon, et al. “A comparison of feature classification methods for modeling solar irradiance variation,” 34th COSPAR Scientific Assembly, 2002.

– Laslo Gyorfi at Debrecen Observatory, Hungary- L. Gyori, T. Baranyi, M. Turmon & J.M. Pap, "Comparison of image-processing

methods to extract sunspots,” Proc. SOHO-11 Workshop, Davos, 2002.

- L. Gyori, T. Baranyi, M. Turmon & J.M. Pap, “Study of differences between sunspot area data determined from ground-based and space-borne observations,” Adv. Space Res., April 2004.

Connections with irradiance- J. M. Pap, M. Turmon et al., “Magnetic Field & Long-Term Solar Irradiance

Variations Over Solar Cycles 21 to 23,” AGU, 2003, San Francisco (poster).


Feature Identification: Infusion

This software will be used in the HMI data pipeline at Stanford– HMI imager will fly on board SDO, the first LWS mission

- http://sdo.gsfc.nasa.gov and http://hmi.stanford.edu

– HMI’s data volume is unprecedented in Solar Physics- 4096x4096 pixel images every 90 seconds

- These data volumes make it more important to focus attention

– HMI/SDO is the successor to MDI/SoHO, on which these results are based

The software has also been baselined by CNES Picard– Picard has been given the go-ahead by CNES for 2007 launch

- http://smsc.cnes.fr/PICARD/

– Picard will measure tiny variations in solar diameter and shape- Active region recognition and rejection is important to its delicate results

– Software must clear ITAR and licensing hurdles


Object Tracking Methods

Associate objects in beforeand after images

Correlation-based tracker– Motion model: deterministic drift plus stochastic uncertainty

– For sunspots or cyclones, have motion and correlation on the sphere

– Correlation measure between a in A and b in B is D(a,b)

Solve assignment problem to match A up to A’:

with P a permutation matrix

Solution by linear programming

For our applications, key is to get deterministic drift correct

A B

Before After


Object Tracking: ExampleM

agn

eto

gra

mL

abel

ing


Object Tracking: Sunspots over seven years

Coordinates, size, intensity of 2500 sunspots from 1996-2003– Over 100-fold reduction in data volume

June through July 1999 shown above– Ordered by central meridian passage (CMP) time

– Successive sightings overplotted; extent indicated by bounding box

Enable quantitative studies of spot taxonomy

La

titu

de

Time (CMP) ––>Zoom view


Object Tracking: High-rate data

We also have tracks from four, three-month periods of high-cadence data from SOHO/MDI

– Continuous telemetry gives full-disk images every minute

– Unprecedented temporal resolution: 1500 images/day or 3GB/day

– Small features we identify and track are tracers for motion of plasma in photosphere

High-cadence data give more samples for each region of interest


Object Analysis Methods: General

Premise: Learn from objects by modeling their evolution as noisy differential equations of several related types

– Hidden Markov models: Finite-state machine controls time series- Divide behaviors into classes according to hidden discrete state u(t)

– Kalman filters: Continuous-state machine controls time series- Explain or model behavior by hidden state vector u(t)

– Track clustering: Discrete variable C selects object type- Extends clustering (the most useful

baseline discovery algorithm) to thetemporal domain

All methods in this family generalizethe basic HMM/Kalman model

– Crucially: subroutine re-use

– Kalman smoother and forward-backward


Object Tracking: Ocean Eddies

Same technology tracks eddies in shallow-water ocean simulation (Toshio M. Chin, JPL)

State (position, size) of two labeled eddies through time, above left Two subclasses of eddies are apparent, above right

statistical object identification, tracking, and analysis

Documents