Feature Extraction (I)

Data Mining IIYear 2009-10Lluís Belanche Alfredo Vellido

Dimensionality reduction (1)

Dimensionality reduction (2)

Signal representation vs classification

Principal Components Analysis (PCA)

General goal : project the data onto a new subspace so that a maximum of relevant information is preserved

In PCA, relevant information is variance (dispersion).

PCA Theory (1)

PCA Theory (2)

PCA Theory (3)

PCA Theory (4)

Algorithm for PCA

PCA examples (1)

PCA examples (2)

PCA examples (2)

PCA examples (3)

PCA examples (4)

Two solutions: in which sense are they optimal?

1. In the signal representation sense2. In the signal separation sense3. In both4. In none

Other approaches to FE Kernel PCA: perform PCA in xΦ(x), where K(x,y) = < Φ(x), Φ(y)> is a kernel ICA (Independent Components Analysis):

Seeks statistical independence of features (PCA seeks uncorrelated features)

Equivalence to PCA iff features are Gaussian Image and audio analysis brings own methods

Series expansion descriptors (from the DFT, DCT or DST) Moment-based features Spectral features Wavelet descriptors

Cao, J.J. et al. A comparison of PCA, KPCA and ICA for dimensionality reduction. Neurocomputing 55, pp. 321-336 (2003)