manifold learning dimensionality reduction. outline introduction dim. reduction manifold isomap...

Manifold LearningDimensionality Reduction

Outline

Introduction Dim. Reduction Manifold

Isomap Overall procedure Approximating geodesic dist. Dijkstra’s algorithm

Reference

Introduction (dim. reduction)

DimensionalityReduction

LinearPCAMDS

Non-linearIsomap(2000)

LLE(2000)SDE(2005)

Introduction (dim. reduction)

Principal Component Analysis

x∑

Introduction (dim. reduction)

DimensionalityReduction

LinearPCAMDS

Non-linearIsomap(2000)

LLE(2000)SDE(2005)

Introduction (dim. reduction)

Multidimensional Scaling

ChicagoRaleigh

Boston Seattle S.F. Austin Orlando

Chicago 0

Raleigh 641 0

Boston 851 608 0

Seattle 1733 2363 2488 0

S.F. 1855 2406 2696 684 0

Austin 972 1167 1691 1764 1495 0

Orlando 994 520 1105 2565 2458 1015 0

Introduction (dim. reduction)

Introduction (dim. reduction)

DimensionalityReduction

LinearPCAMDS

Non-linearIsomap(2000)

LLE(2000)SDE(2005)

Introduction (manifold)

Linear methods do nothing more than “globally transform”(rotate/translate..) data. Sometimes need to “unwrap” the data first

PCA

Introduction (dim. reduction)

The task of dimensionality reduction is to find a small number of features to represent a large number of observed dimensions.

Introduction (manifold)

Introduction (manifold)

Outline

Introduction Dim. Reduction Manifold

Isomap Overall procedure Approximating geodesic dist. Dijkstra’s algorithm

Reference

Isomap (overall procedure)

Compute fully-connected neighborhood of points for each item (k nearest)

Calculate pairwise Euclidean distances within each neighborhood

Use Dijkstra’s Algorithm to compute shortest path from each point to non-neighboring points

Run MDS on resulting distance matrix

Isomap (Approximating geodesic dist.)

Isomap (Approximating geodesic dist.)

Isomap (Approximating geodesic dist.)

is not much bigger than

Isomap (Approximating geodesic dist.)

is not much bigger than

Isomap (Approximating geodesic dist.)

is not much bigger than

Isomap (Approximating geodesic dist.)

is not much bigger than

Isomap (Approximating geodesic dist.)

Isomap (Dijkstra’s Algorithm)

Greedy breadth-first algorithm to compute shortest path from one point to all other points

Isomap (Dijkstra’s Algorithm)

Greedy breadth-first algorithm to compute shortest path from one point to all other points

Isomap (Dijkstra’s Algorithm)

Greedy breadth-first algorithm to compute shortest path from one point to all other points

Isomap (Dijkstra’s Algorithm)

Greedy breadth-first algorithm to compute shortest path from one point to all other points

Isomap

Isomap

Reference

http://www.cs.unc.edu/Courses/comp290-090-s06/

http://www.cse.msu.edu/~lawhiu/manifold/