matlab project independent component...
TRANSCRIPT
![Page 1: Matlab project Independent component analysisperso.uclouvain.be/pa.absil/EECI2011/Grenoble_Matlab_project.pdf · Matlab project Independent component analysis ... Blind source separation](https://reader031.vdocuments.us/reader031/viewer/2022021514/5b151e087f8b9a467c8dbbc0/html5/thumbnails/1.jpg)
Matlab project
Independent component analysis
Michel JournéeDept. of Electrical Engineering and Computer Science
University of Liège, Belgium
September 2008
![Page 2: Matlab project Independent component analysisperso.uclouvain.be/pa.absil/EECI2011/Grenoble_Matlab_project.pdf · Matlab project Independent component analysis ... Blind source separation](https://reader031.vdocuments.us/reader031/viewer/2022021514/5b151e087f8b9a467c8dbbc0/html5/thumbnails/2.jpg)
2
What is Independent Component Analysis?
![Page 3: Matlab project Independent component analysisperso.uclouvain.be/pa.absil/EECI2011/Grenoble_Matlab_project.pdf · Matlab project Independent component analysis ... Blind source separation](https://reader031.vdocuments.us/reader031/viewer/2022021514/5b151e087f8b9a467c8dbbc0/html5/thumbnails/3.jpg)
3
The cocktail party problem
![Page 4: Matlab project Independent component analysisperso.uclouvain.be/pa.absil/EECI2011/Grenoble_Matlab_project.pdf · Matlab project Independent component analysis ... Blind source separation](https://reader031.vdocuments.us/reader031/viewer/2022021514/5b151e087f8b9a467c8dbbc0/html5/thumbnails/4.jpg)
4
ICA performs a linear projection into independent components
Assumptions
linearityno delaystatistically independent sources
![Page 5: Matlab project Independent component analysisperso.uclouvain.be/pa.absil/EECI2011/Grenoble_Matlab_project.pdf · Matlab project Independent component analysis ... Blind source separation](https://reader031.vdocuments.us/reader031/viewer/2022021514/5b151e087f8b9a467c8dbbc0/html5/thumbnails/5.jpg)
5
ICA performs a linear projection into independent components
number of variablesn
number of componentsp
= SX A
Random vector
Real matrix
Statistically independentrandom variables
n x p
![Page 6: Matlab project Independent component analysisperso.uclouvain.be/pa.absil/EECI2011/Grenoble_Matlab_project.pdf · Matlab project Independent component analysis ... Blind source separation](https://reader031.vdocuments.us/reader031/viewer/2022021514/5b151e087f8b9a467c8dbbc0/html5/thumbnails/6.jpg)
6
ICA performs a linear projection into independent components
=p x N
n x pn x N
number of samplesN
number of variablesn
number of componentsp
A SX
Samples ofstatistically independent
random variables
![Page 7: Matlab project Independent component analysisperso.uclouvain.be/pa.absil/EECI2011/Grenoble_Matlab_project.pdf · Matlab project Independent component analysis ... Blind source separation](https://reader031.vdocuments.us/reader031/viewer/2022021514/5b151e087f8b9a467c8dbbc0/html5/thumbnails/7.jpg)
7
ICA for blind source separation:fECG extraction
ICA
![Page 8: Matlab project Independent component analysisperso.uclouvain.be/pa.absil/EECI2011/Grenoble_Matlab_project.pdf · Matlab project Independent component analysis ... Blind source separation](https://reader031.vdocuments.us/reader031/viewer/2022021514/5b151e087f8b9a467c8dbbc0/html5/thumbnails/8.jpg)
8
ICA for blind source separation:Analysis of EEG
CSF
EEG
![Page 9: Matlab project Independent component analysisperso.uclouvain.be/pa.absil/EECI2011/Grenoble_Matlab_project.pdf · Matlab project Independent component analysis ... Blind source separation](https://reader031.vdocuments.us/reader031/viewer/2022021514/5b151e087f8b9a467c8dbbc0/html5/thumbnails/9.jpg)
9
ICA for EEG
![Page 10: Matlab project Independent component analysisperso.uclouvain.be/pa.absil/EECI2011/Grenoble_Matlab_project.pdf · Matlab project Independent component analysis ... Blind source separation](https://reader031.vdocuments.us/reader031/viewer/2022021514/5b151e087f8b9a467c8dbbc0/html5/thumbnails/10.jpg)
10
ICA for data analysis
Principal directions
Independent directions
![Page 11: Matlab project Independent component analysisperso.uclouvain.be/pa.absil/EECI2011/Grenoble_Matlab_project.pdf · Matlab project Independent component analysis ... Blind source separation](https://reader031.vdocuments.us/reader031/viewer/2022021514/5b151e087f8b9a467c8dbbc0/html5/thumbnails/11.jpg)
11
ICA for denoising
Wiener filtering
ICA filtering
Noisy image
Original image
![Page 12: Matlab project Independent component analysisperso.uclouvain.be/pa.absil/EECI2011/Grenoble_Matlab_project.pdf · Matlab project Independent component analysis ... Blind source separation](https://reader031.vdocuments.us/reader031/viewer/2022021514/5b151e087f8b9a467c8dbbc0/html5/thumbnails/12.jpg)
12
ICA has applications in many areas
� Blind source separation (e.g., biomedical signal processing, radar and mobile communication)
� Data analysis
� Noise reduction
� Feature extraction (image, audio, video representation)
� …
![Page 13: Matlab project Independent component analysisperso.uclouvain.be/pa.absil/EECI2011/Grenoble_Matlab_project.pdf · Matlab project Independent component analysis ... Blind source separation](https://reader031.vdocuments.us/reader031/viewer/2022021514/5b151e087f8b9a467c8dbbc0/html5/thumbnails/13.jpg)
13
ICA is an optimization problem
estimator of statistical independence
![Page 14: Matlab project Independent component analysisperso.uclouvain.be/pa.absil/EECI2011/Grenoble_Matlab_project.pdf · Matlab project Independent component analysis ... Blind source separation](https://reader031.vdocuments.us/reader031/viewer/2022021514/5b151e087f8b9a467c8dbbc0/html5/thumbnails/14.jpg)
14
ICA algorithms compute the unmixing model
Mixing model Unmixing model
![Page 15: Matlab project Independent component analysisperso.uclouvain.be/pa.absil/EECI2011/Grenoble_Matlab_project.pdf · Matlab project Independent component analysis ... Blind source separation](https://reader031.vdocuments.us/reader031/viewer/2022021514/5b151e087f8b9a467c8dbbc0/html5/thumbnails/15.jpg)
15
ICA is an optimization problem
1. Estimation of the statistical independence of the z's:
2. Minimization of the contrast:
![Page 16: Matlab project Independent component analysisperso.uclouvain.be/pa.absil/EECI2011/Grenoble_Matlab_project.pdf · Matlab project Independent component analysis ... Blind source separation](https://reader031.vdocuments.us/reader031/viewer/2022021514/5b151e087f8b9a467c8dbbc0/html5/thumbnails/16.jpg)
16
The contrast presents two inherent symmetries
If are independent,
then are independent,
and as well.
![Page 17: Matlab project Independent component analysisperso.uclouvain.be/pa.absil/EECI2011/Grenoble_Matlab_project.pdf · Matlab project Independent component analysis ... Blind source separation](https://reader031.vdocuments.us/reader031/viewer/2022021514/5b151e087f8b9a467c8dbbc0/html5/thumbnails/17.jpg)
17
The contrast presents two inherent symmetries
If are independent,
then are independent,
and as well.
⇒
invertible diagonal matrix
permutation matrix
![Page 18: Matlab project Independent component analysisperso.uclouvain.be/pa.absil/EECI2011/Grenoble_Matlab_project.pdf · Matlab project Independent component analysis ... Blind source separation](https://reader031.vdocuments.us/reader031/viewer/2022021514/5b151e087f8b9a467c8dbbc0/html5/thumbnails/18.jpg)
18
Determined by Orthogonal ICA
Furthermore, most ICA methods use prewhitening
Determined by SVD of XPrincipal Component Analysis
For any matrix :
W= U S V T
![Page 19: Matlab project Independent component analysisperso.uclouvain.be/pa.absil/EECI2011/Grenoble_Matlab_project.pdf · Matlab project Independent component analysis ... Blind source separation](https://reader031.vdocuments.us/reader031/viewer/2022021514/5b151e087f8b9a467c8dbbc0/html5/thumbnails/19.jpg)
19
In dimension 2…
s1
s2
x1
x2 a
2
a1
=x1
x2
.
![Page 20: Matlab project Independent component analysisperso.uclouvain.be/pa.absil/EECI2011/Grenoble_Matlab_project.pdf · Matlab project Independent component analysis ... Blind source separation](https://reader031.vdocuments.us/reader031/viewer/2022021514/5b151e087f8b9a467c8dbbc0/html5/thumbnails/20.jpg)
20
In dimension 2…
x1
x2
z1
z2
PCA ICA
![Page 21: Matlab project Independent component analysisperso.uclouvain.be/pa.absil/EECI2011/Grenoble_Matlab_project.pdf · Matlab project Independent component analysis ... Blind source separation](https://reader031.vdocuments.us/reader031/viewer/2022021514/5b151e087f8b9a467c8dbbc0/html5/thumbnails/21.jpg)
21
ICA as an optimization on the orthogonal group
Orthogonal ICA (also called prewhitening-based ICA):
The orthogonal group automatically gets rid of the scaling indeterminacy.
with
![Page 22: Matlab project Independent component analysisperso.uclouvain.be/pa.absil/EECI2011/Grenoble_Matlab_project.pdf · Matlab project Independent component analysis ... Blind source separation](https://reader031.vdocuments.us/reader031/viewer/2022021514/5b151e087f8b9a467c8dbbc0/html5/thumbnails/22.jpg)
22
A whole bunch of ICA algorithms…
Contrast
Estimation of the mutual information
Joint diagonalization of cumulant matrices
Diagonalization of cumulant tensors
Non-gaussianity
Constrained covariance
Manifold
Orthogonal group
Stiefel manifold
Oblique manifold
Flag manifold (independent subspace analysis)
Optimization method
Jacobi rotations
Gradient descent
Second-order approaches
![Page 23: Matlab project Independent component analysisperso.uclouvain.be/pa.absil/EECI2011/Grenoble_Matlab_project.pdf · Matlab project Independent component analysis ... Blind source separation](https://reader031.vdocuments.us/reader031/viewer/2022021514/5b151e087f8b9a467c8dbbc0/html5/thumbnails/23.jpg)
23
Outline of the project
� Manifold : Orthogonal group
� Contrast: Joint diagonalization of cumulant matrices
![Page 24: Matlab project Independent component analysisperso.uclouvain.be/pa.absil/EECI2011/Grenoble_Matlab_project.pdf · Matlab project Independent component analysis ... Blind source separation](https://reader031.vdocuments.us/reader031/viewer/2022021514/5b151e087f8b9a467c8dbbc0/html5/thumbnails/24.jpg)
24
Joint diagonalization of a set of matrices
Given m cumulant matrices Ci, minimize
CiW T W =
Diagonalization of one matrix:
![Page 25: Matlab project Independent component analysisperso.uclouvain.be/pa.absil/EECI2011/Grenoble_Matlab_project.pdf · Matlab project Independent component analysis ... Blind source separation](https://reader031.vdocuments.us/reader031/viewer/2022021514/5b151e087f8b9a467c8dbbc0/html5/thumbnails/25.jpg)
25
Joint diagonalization of a set of matrices
Given m cumulant matrices Ci, minimize
Ci
W T W =
Joint diagonalization of m matrices:
![Page 26: Matlab project Independent component analysisperso.uclouvain.be/pa.absil/EECI2011/Grenoble_Matlab_project.pdf · Matlab project Independent component analysis ... Blind source separation](https://reader031.vdocuments.us/reader031/viewer/2022021514/5b151e087f8b9a467c8dbbc0/html5/thumbnails/26.jpg)
26
Outline of the project
� Manifold : Orthogonal group
� Contrast: Joint diagonalization of cumulant matrices
� Optimization method: conjugate gradient
� Applications: blind source separation of images, bioinformatics
![Page 27: Matlab project Independent component analysisperso.uclouvain.be/pa.absil/EECI2011/Grenoble_Matlab_project.pdf · Matlab project Independent component analysis ... Blind source separation](https://reader031.vdocuments.us/reader031/viewer/2022021514/5b151e087f8b9a467c8dbbc0/html5/thumbnails/27.jpg)
27
Separation of images
ICA
![Page 28: Matlab project Independent component analysisperso.uclouvain.be/pa.absil/EECI2011/Grenoble_Matlab_project.pdf · Matlab project Independent component analysis ... Blind source separation](https://reader031.vdocuments.us/reader031/viewer/2022021514/5b151e087f8b9a467c8dbbc0/html5/thumbnails/28.jpg)
28
Analysis of gene expression data
Microarray
Each spot reflects the expression of a gene
Gene expression database
Rows ↔ genes ( ~104)
Columns ↔ experiments ( ~102)
![Page 29: Matlab project Independent component analysisperso.uclouvain.be/pa.absil/EECI2011/Grenoble_Matlab_project.pdf · Matlab project Independent component analysis ... Blind source separation](https://reader031.vdocuments.us/reader031/viewer/2022021514/5b151e087f8b9a467c8dbbc0/html5/thumbnails/29.jpg)
29
Analysis of gene expression data
Microarray
Each spot reflects the expression of a gene
Gene expression database
Rows ↔ genes
Columns ↔ experiments
DNA
mRNA
Protein
![Page 30: Matlab project Independent component analysisperso.uclouvain.be/pa.absil/EECI2011/Grenoble_Matlab_project.pdf · Matlab project Independent component analysis ... Blind source separation](https://reader031.vdocuments.us/reader031/viewer/2022021514/5b151e087f8b9a467c8dbbc0/html5/thumbnails/30.jpg)
30
Such a database is a goldmine for new knowledge about the cellular machinery
� Global picture of the transcriptome under several conditions
� Genes that are coexpressed across similar conditions are very informative
� Identification of interesting structures in the genome
Some interesting questions:
� What does this gene do?
� Which genes are responsible of a phenotype?
� How do the genes act on a phenotype?
![Page 31: Matlab project Independent component analysisperso.uclouvain.be/pa.absil/EECI2011/Grenoble_Matlab_project.pdf · Matlab project Independent component analysis ... Blind source separation](https://reader031.vdocuments.us/reader031/viewer/2022021514/5b151e087f8b9a467c8dbbc0/html5/thumbnails/31.jpg)
31
ICA in case of gene expression data
≈Expression mode(statistically independent)
weigths
![Page 32: Matlab project Independent component analysisperso.uclouvain.be/pa.absil/EECI2011/Grenoble_Matlab_project.pdf · Matlab project Independent component analysis ... Blind source separation](https://reader031.vdocuments.us/reader031/viewer/2022021514/5b151e087f8b9a467c8dbbc0/html5/thumbnails/32.jpg)
32
Analysis of an ovarian cancer database
175 genes
17 tissues
+ some clinical data
![Page 33: Matlab project Independent component analysisperso.uclouvain.be/pa.absil/EECI2011/Grenoble_Matlab_project.pdf · Matlab project Independent component analysis ... Blind source separation](https://reader031.vdocuments.us/reader031/viewer/2022021514/5b151e087f8b9a467c8dbbc0/html5/thumbnails/33.jpg)
33
ICA expression modes are highly correlated with the observed phenotypes
Tissues
Expressionmodes
Pre-menopause
poorly differentiated serous papillary adenocarcinoma
benign mucinous cystadenoma
![Page 34: Matlab project Independent component analysisperso.uclouvain.be/pa.absil/EECI2011/Grenoble_Matlab_project.pdf · Matlab project Independent component analysis ... Blind source separation](https://reader031.vdocuments.us/reader031/viewer/2022021514/5b151e087f8b9a467c8dbbc0/html5/thumbnails/34.jpg)
34
ICA identifies genes likely to be coexpressed for an observed phenotype
E.g. poorly differentiated serous papillary adenocarcinoma (pd-spa)
genes
HLA CLASS IMEMBRANE GLYCOPROTEIN GP130 PLACENTAL-CADHERIN COFILINTIE1
Expression mode 15
![Page 35: Matlab project Independent component analysisperso.uclouvain.be/pa.absil/EECI2011/Grenoble_Matlab_project.pdf · Matlab project Independent component analysis ... Blind source separation](https://reader031.vdocuments.us/reader031/viewer/2022021514/5b151e087f8b9a467c8dbbc0/html5/thumbnails/35.jpg)
35
References
� P.-A. Absil and K. A. Gallivan, Joint diagonalization on the oblique manifold for independent component analysis, ICASSP 2006, 2006.
� P.-A. Absil, R. Mahony, and R. Sepulchre, Optimization algorithms on matrix manifolds, Princeton University Press, 2008.
� F. R. Bach and M. I. Jordan, Kernel independent component analysis, Journal of Machine Learning Research, 3,1-48, 2003.
� J.-F. Cardoso, High-order contrasts for independent component analysis, Neural Computation 11, no. 1, 157–192, 1999.
� P. Comon, Independent Component Analysis, a new concept ?, Signal Processing, Elsevier 36, no. 3, 287–314, Special issue on Higher-Order Statistics, 1994.
� A. Hyvärinen, J. Karhunen, and E. Oja, Independent component analysis, John Wiley & Sons, 2001.
� E.G. Learned-Miller and J.W.Fisher III, ICA using spacings estimates of entropy, Journal of Machine Learning Research, 4, 1271-1295,2003.
![Page 36: Matlab project Independent component analysisperso.uclouvain.be/pa.absil/EECI2011/Grenoble_Matlab_project.pdf · Matlab project Independent component analysis ... Blind source separation](https://reader031.vdocuments.us/reader031/viewer/2022021514/5b151e087f8b9a467c8dbbc0/html5/thumbnails/36.jpg)
36
References
� W. Liebermeister, Linear modes of gene expression determined by independent component analysis, Bioinformatics 18, 51–60, 2002.
� A.-M. Martoglio, J. W. Miskin, S. K. Smith, and D. J. C. MacKay, A decomposition model to track gene expression signatures: preview on observer-independent classification of ovarian cancer, Bioinformatics 18, no. 12, 1617–1624, 2002.
� A. E. Teschendorff, M Journée, P.-A. Absil, R. Sepulchre, andC. Caldas, Elucidating the altered transcriptional programs inbreast cancer using independent component analysis, PLoS Computational Biology 3, Number 8, page 1539-1554, 2007.
![Page 37: Matlab project Independent component analysisperso.uclouvain.be/pa.absil/EECI2011/Grenoble_Matlab_project.pdf · Matlab project Independent component analysis ... Blind source separation](https://reader031.vdocuments.us/reader031/viewer/2022021514/5b151e087f8b9a467c8dbbc0/html5/thumbnails/37.jpg)
37
Schedule
4 Matlab session:
- Wednesday 11:30 – 12:30
- Wednesday 16:30 – 17:30
- Thursday 16:30 – 17:30
- Friday 11:30 – 12:30
![Page 38: Matlab project Independent component analysisperso.uclouvain.be/pa.absil/EECI2011/Grenoble_Matlab_project.pdf · Matlab project Independent component analysis ... Blind source separation](https://reader031.vdocuments.us/reader031/viewer/2022021514/5b151e087f8b9a467c8dbbc0/html5/thumbnails/38.jpg)
38
Good work!