Three New Ideas in SDP-based Manifold Learning

Alexander GrayGeorgia Institute of Technology

College of Computing

FASTlab: Fundamental Algorithmic and Statistical Tools

The FASTlabFundamental Algorithmic and Statistical Tools Laboratory

1. Arkadas Ozakin: Research scientist, PhD Theoretical Physics2. Dong Ryeol Lee: PhD student, CS + Math3. Ryan Riegel: PhD student, CS + Math4. Parikshit Ram: PhD student, CS + Math5. William March: PhD student, Math + CS6. James Waters: PhD student, Physics + CS7. Hua Ouyang: PhD student, CS8. Sooraj Bhat: PhD student, CS9. Ravi Sastry: PhD student, CS10. Long Tran: PhD student, CS11. Michael Holmes: PhD student, CS + Physics (co-supervised)12. Nikolaos Vasiloglou: PhD student, EE (co-supervised)13. Wei Guan: PhD student, CS (co-supervised)14. Nishant Mehta: PhD student, CS (co-supervised)15. Wee Chin Wong: PhD student, ChemE (co-supervised)16. Abhimanyu Aditya: MS student, CS17. Yatin Kanetkar: MS student, CS18. Praveen Krishnaiah: MS student, CS19. Devika Karnik: MS student, CS20. Prasad Jakka: MS student, CS

10 sample tasks

• “Find engines like this one” (querying)• “Plot the distribution of engine sizes and emissions” (density

estimation)• “Predict the lifetime maintenance cost” (regression)• “Predict existence of fault or not” (classification)• “Predict the number of failures next year” (time series analysis)• “Show all engines on a 2-d plot” (dimension reduction)• “Show or remove the unusual engines” (outlier detection)• “Show the different types of engines” (clustering)• “Is this group equivalent to this group?” (two-sample testing)• “What’s the best action to take based on this behavior?”

(reinforcement learning/control)

Types of data:• Sensor measurements• Documents• Database records, etc.

Rankmap

• Can do manifold learning using only ordinal data

Isometric Separation Maps

• Preserve class proximity

Density-Preserving Maps

• Preserve densities, not distances

The problem: big datasets

D

N

M

Could be large: N (#data), D (#features), M (#models)

Dual-tree All-nearest-neighbors

• O(N2) O(N)

Rank-approximate Nearest-neighbor Search

• Distance approximation rank approximation

Multi-scale

Decompositions

e.g. kd-trees

[Bentley 1975], [Friedman, Bentley & Finkel 1977],[Moore & Lee 1995]

How can we compute these efficiently?

A kd-tree: level 1

A kd-tree: level 2

A kd-tree: level 3

A kd-tree: level 4

A kd-tree: level 5

A kd-tree: level 6

Some application highlights

• Our software is being put into the pipelines of the world’s massive-scale science projects– Astronomy sky surveys (LSST, Pan-

STARRS, DES): 1B objects/month– Large Hadron Collider: 1M events/sec

Some application highlights

• Others– McAfee spam blacklisting: 300M emails/day– Supermarket demand forecasting– Algorithmic trading– Audio fingerprint matching– Legal document browsing and search

Software

• MLPACK (C++)– First scalable comprehensive ML library

• MLPACK-db – fast data analytics in relational

databases (SQL Server)

• MLPACK Pro

- Very-large-scale data