1

Machine Learning and Data Mining for Automatic Detection and Interpretation of

Solar Events

Jie Zhang (Presenting, Co-I, SCS*) Art Poland (PI, SCS*)Harry Wechsler (Co-I, Computer Science)Kirk Borne (Co-I, SCS*) Oscar Olmedo (student, SCS*)(George Mason University)

*SCS: School of Computational Sciences at GMU

NASA AISRP NASA AMES, Moffett Field, CA April 4 – 6, 2005

2

Introduction• Why solar events?

• Great interests of scientific understanding• Great interests of practical use: the space weather

• What are solar events? Examples

CME FLARE Dimming(coronal mass ejection)

3

Year 1996 – 2004

Flare Count 19176CME Count 8852

Daily Min Max (1996) (2002)Flare 1.0 10CME 0.5 5

Sunspot 20 200

CME/Flare Statistics

4

Objectives

• Our main objective is to develop an automatic system for CME detection, tracking, characterization and source region location

• An automatic system is needed• Timely detection, necessary for space weather forecasting• Objective characterization, removing human bias• Reducing human cost

• Data volume and number of events are enormous• Explosively growth of data (SOHO, STEREO and SDO)

5

Methods

• Image Processing (current work)• Pre-processing• Detection and Tracking• Characterization

• Machine Learning (future work)• Develop robust and efficient algorithms for event detection• Learning Methods

• Statistical learning theory, e.g., Support Vector Machine (SVM)• Performance Evaluation

• Benchmark (catalog by human) • ROC (Receiver Operating Characteristic) curve: hit, miss, or

false-detection

• Data Mining (future work)• Association of events from different sets of observations

• Space, and Time• Physical parameters, e.g., intensity

6

Image Properties

•Find a faint moving object against a strong slow-varying background

•CME, like other astrophysical objects, is optically thin; no hard surface

• An object without fixed shape; an expansion flow

7

Image Processing: Pre-processing

• Calibration• Filtering and Smoothing• Differencing• Polar Transformation

8

Image Processing: Initial Detection

• Finding CME angular expansion

• Projection • Threshold : get core angles• Morphology analysis

• Region Growing• Closing (Dilation + Erosion): join features with narrow gaps• Opening (Erosion + Dilation): remove narrow features

• Finding CME Height•Thresholding on the area of selected angular expansion•Projection along the height

9

Image Processing: a Demo

2002/12/01 – 12/07: 431 images

10

Image Processing: Detection and Tracking

• After the first detection• Set the time stamp, expire after 5 hours• set the targeted tracking region

• Targeted-tracking reduces false detection significantly, e.g., remove contamination of CME trailing outflow

• Cleaning• Remove sporadic detection

•Preliminary Statistics: 2002/12/01 – 2002/12/07•19 CMEs in human catalog•19 CMEs in machine catalog (25 before cleaning)

• hit: 14 (74%)• miss: 5 (26%)• false detection: 5 (26%)

11

Future Plan of this Project

• We are only a few months into this project, which is supported for only one year

• We are seeking a full 3-year funding to fulfill the proposed objectives• Finish all image processing tasks

• C2 (almost done)• C3 (under development)• EIT (under development)

• Use machine learning methods to develop robust algorithms (future)

• Use data mining methods to integrate detections, for the ultimate goal of space weather prediction (future)

• Make a computer-generated event catalog

12

The Future

• Automatic detection of all relevant events in the integrated Sun-Earth connection system• Sun• Solar Corona• Heliosphere• Magnetosphere• Ionosphere

• Virtual X Observatories: contributor and user

• Machine learning and Data Mining for general science discovery