thomas wachtler, te-won lee and terrence j. sejnowski · thomas wachtler, te-won lee and terrence...
TRANSCRIPT
![Page 1: Thomas Wachtler, Te-Won Lee and Terrence J. Sejnowski · Thomas Wachtler, Te-Won Lee and Terrence J. Sejnowski presented by Andrew Rabinovich October 17th, 2002. Goal:Find efficient](https://reader036.vdocuments.us/reader036/viewer/2022071019/5fd33c1711e0241b732ac1c1/html5/thumbnails/1.jpg)
Thomas Wachtler, Te-Won Lee and Terrence J. Sejnowski
presented by Andrew RabinovichOctober 17th, 2002
![Page 2: Thomas Wachtler, Te-Won Lee and Terrence J. Sejnowski · Thomas Wachtler, Te-Won Lee and Terrence J. Sejnowski presented by Andrew Rabinovich October 17th, 2002. Goal:Find efficient](https://reader036.vdocuments.us/reader036/viewer/2022071019/5fd33c1711e0241b732ac1c1/html5/thumbnails/2.jpg)
Goal: Find efficient representation of chromatic sensory information such that its redundancy is reduced
Solution:
![Page 3: Thomas Wachtler, Te-Won Lee and Terrence J. Sejnowski · Thomas Wachtler, Te-Won Lee and Terrence J. Sejnowski presented by Andrew Rabinovich October 17th, 2002. Goal:Find efficient](https://reader036.vdocuments.us/reader036/viewer/2022071019/5fd33c1711e0241b732ac1c1/html5/thumbnails/3.jpg)
12211 niniiifor all i=1,…,n
N
i ii1
![Page 4: Thomas Wachtler, Te-Won Lee and Terrence J. Sejnowski · Thomas Wachtler, Te-Won Lee and Terrence J. Sejnowski presented by Andrew Rabinovich October 17th, 2002. Goal:Find efficient](https://reader036.vdocuments.us/reader036/viewer/2022071019/5fd33c1711e0241b732ac1c1/html5/thumbnails/4.jpg)
• The independent components are assumed statistically independent
• The independent components must have Non-Gaussian distributions
![Page 5: Thomas Wachtler, Te-Won Lee and Terrence J. Sejnowski · Thomas Wachtler, Te-Won Lee and Terrence J. Sejnowski presented by Andrew Rabinovich October 17th, 2002. Goal:Find efficient](https://reader036.vdocuments.us/reader036/viewer/2022071019/5fd33c1711e0241b732ac1c1/html5/thumbnails/5.jpg)
Two random variables yi and yj are said to be independent if information of yi does not give any information on value of yj for i?j
In terms of probability densities:p(y1,y2,…,yn)=p1(y1) p2(y2)… p3(y3)
and expected values:E[g(y1)h(y2)]=E[g(y1)]E[h(y2)]
![Page 6: Thomas Wachtler, Te-Won Lee and Terrence J. Sejnowski · Thomas Wachtler, Te-Won Lee and Terrence J. Sejnowski presented by Andrew Rabinovich October 17th, 2002. Goal:Find efficient](https://reader036.vdocuments.us/reader036/viewer/2022071019/5fd33c1711e0241b732ac1c1/html5/thumbnails/6.jpg)
Two random variables x1 and x2 are uncorrelated if:cov(x1,x2)=E[x1,x2]-E[x1]E[x2]=0
For mx=0(centered data):corr(x1,x2)=E[x1,x2]
![Page 7: Thomas Wachtler, Te-Won Lee and Terrence J. Sejnowski · Thomas Wachtler, Te-Won Lee and Terrence J. Sejnowski presented by Andrew Rabinovich October 17th, 2002. Goal:Find efficient](https://reader036.vdocuments.us/reader036/viewer/2022071019/5fd33c1711e0241b732ac1c1/html5/thumbnails/7.jpg)
![Page 8: Thomas Wachtler, Te-Won Lee and Terrence J. Sejnowski · Thomas Wachtler, Te-Won Lee and Terrence J. Sejnowski presented by Andrew Rabinovich October 17th, 2002. Goal:Find efficient](https://reader036.vdocuments.us/reader036/viewer/2022071019/5fd33c1711e0241b732ac1c1/html5/thumbnails/8.jpg)
Higher-order(past 2nd) cumulants are zero for Gaussian distributions, which is essential for estimation of the ICA model.
![Page 9: Thomas Wachtler, Te-Won Lee and Terrence J. Sejnowski · Thomas Wachtler, Te-Won Lee and Terrence J. Sejnowski presented by Andrew Rabinovich October 17th, 2002. Goal:Find efficient](https://reader036.vdocuments.us/reader036/viewer/2022071019/5fd33c1711e0241b732ac1c1/html5/thumbnails/9.jpg)
4th order cumulant that is a classical measure of Non Gaussianity
kurt(y)=E[y4]-3(E[y2])2
assuming unit variance:kurt(y)=E[y4]-3
Kurtosis for Non Gaussian random variables:
superGaussian(leptokurtic)(+) subGaussian(platykurtic)(-)
![Page 10: Thomas Wachtler, Te-Won Lee and Terrence J. Sejnowski · Thomas Wachtler, Te-Won Lee and Terrence J. Sejnowski presented by Andrew Rabinovich October 17th, 2002. Goal:Find efficient](https://reader036.vdocuments.us/reader036/viewer/2022071019/5fd33c1711e0241b732ac1c1/html5/thumbnails/10.jpg)
Whitening the data, reduces the search space of the unmixing matrix
A’=VA (A’ is orthogonal)E[z1z1
T]=A’E[s1s1T]A’T=A’A’T=I
Whiteness of a zero-mean random vector y1 guarantees uncorrelatedness and unit variance of its components
whitening(sphering)
![Page 11: Thomas Wachtler, Te-Won Lee and Terrence J. Sejnowski · Thomas Wachtler, Te-Won Lee and Terrence J. Sejnowski presented by Andrew Rabinovich October 17th, 2002. Goal:Find efficient](https://reader036.vdocuments.us/reader036/viewer/2022071019/5fd33c1711e0241b732ac1c1/html5/thumbnails/11.jpg)
I. Begin with a uniform distribution:
II. Mix the independent component with A
III. Whiten the data
IV. Perform ICA to extract the rotation
I. II. III. IV.
5 1010 2=
![Page 12: Thomas Wachtler, Te-Won Lee and Terrence J. Sejnowski · Thomas Wachtler, Te-Won Lee and Terrence J. Sejnowski presented by Andrew Rabinovich October 17th, 2002. Goal:Find efficient](https://reader036.vdocuments.us/reader036/viewer/2022071019/5fd33c1711e0241b732ac1c1/html5/thumbnails/12.jpg)
Original Distribution
Mixed Distribution
![Page 13: Thomas Wachtler, Te-Won Lee and Terrence J. Sejnowski · Thomas Wachtler, Te-Won Lee and Terrence J. Sejnowski presented by Andrew Rabinovich October 17th, 2002. Goal:Find efficient](https://reader036.vdocuments.us/reader036/viewer/2022071019/5fd33c1711e0241b732ac1c1/html5/thumbnails/13.jpg)
Gaussian Distribution is rotationally symmetric
No information on the directions of the columns of the mixing matrix A
![Page 14: Thomas Wachtler, Te-Won Lee and Terrence J. Sejnowski · Thomas Wachtler, Te-Won Lee and Terrence J. Sejnowski presented by Andrew Rabinovich October 17th, 2002. Goal:Find efficient](https://reader036.vdocuments.us/reader036/viewer/2022071019/5fd33c1711e0241b732ac1c1/html5/thumbnails/14.jpg)
The variances(energies) of the independent components cannot be determined.
The order of independent components cannot be determined
i iii
i
asaa
x **
![Page 15: Thomas Wachtler, Te-Won Lee and Terrence J. Sejnowski · Thomas Wachtler, Te-Won Lee and Terrence J. Sejnowski presented by Andrew Rabinovich October 17th, 2002. Goal:Find efficient](https://reader036.vdocuments.us/reader036/viewer/2022071019/5fd33c1711e0241b732ac1c1/html5/thumbnails/15.jpg)
• Maximization of Non-Gaussianity- Kurtosis- Negentropy
• Maximization of Likelihood Estimation- InfoMax- Gradient
• Minimization of Mutual Information• Tensorial Methods • Nonlinear Decorrelation & Nonlinear PCA
![Page 16: Thomas Wachtler, Te-Won Lee and Terrence J. Sejnowski · Thomas Wachtler, Te-Won Lee and Terrence J. Sejnowski presented by Andrew Rabinovich October 17th, 2002. Goal:Find efficient](https://reader036.vdocuments.us/reader036/viewer/2022071019/5fd33c1711e0241b732ac1c1/html5/thumbnails/16.jpg)
Non-Gaussian Distribution is chosen, since Gaussian pdf is Completely described by mean and variance(PCA)
![Page 17: Thomas Wachtler, Te-Won Lee and Terrence J. Sejnowski · Thomas Wachtler, Te-Won Lee and Terrence J. Sejnowski presented by Andrew Rabinovich October 17th, 2002. Goal:Find efficient](https://reader036.vdocuments.us/reader036/viewer/2022071019/5fd33c1711e0241b732ac1c1/html5/thumbnails/17.jpg)
Principal Component Analysis (PCA) finds directions of maximal variance in Non-Gaussian data (second-order statistics).
![Page 18: Thomas Wachtler, Te-Won Lee and Terrence J. Sejnowski · Thomas Wachtler, Te-Won Lee and Terrence J. Sejnowski presented by Andrew Rabinovich October 17th, 2002. Goal:Find efficient](https://reader036.vdocuments.us/reader036/viewer/2022071019/5fd33c1711e0241b732ac1c1/html5/thumbnails/18.jpg)
Independent Component Analysis(ICA) finds directions of maximal independence in non-Gaussian data (higher-order statistics).
![Page 19: Thomas Wachtler, Te-Won Lee and Terrence J. Sejnowski · Thomas Wachtler, Te-Won Lee and Terrence J. Sejnowski presented by Andrew Rabinovich October 17th, 2002. Goal:Find efficient](https://reader036.vdocuments.us/reader036/viewer/2022071019/5fd33c1711e0241b732ac1c1/html5/thumbnails/19.jpg)
Mixed Signal: Independent Components:
Mixture 1
Mixture 2
Mixed Signal: Independent Components:
![Page 20: Thomas Wachtler, Te-Won Lee and Terrence J. Sejnowski · Thomas Wachtler, Te-Won Lee and Terrence J. Sejnowski presented by Andrew Rabinovich October 17th, 2002. Goal:Find efficient](https://reader036.vdocuments.us/reader036/viewer/2022071019/5fd33c1711e0241b732ac1c1/html5/thumbnails/20.jpg)
Find efficient representation of chromatic sensory information such that its redundancy is reduced
![Page 21: Thomas Wachtler, Te-Won Lee and Terrence J. Sejnowski · Thomas Wachtler, Te-Won Lee and Terrence J. Sejnowski presented by Andrew Rabinovich October 17th, 2002. Goal:Find efficient](https://reader036.vdocuments.us/reader036/viewer/2022071019/5fd33c1711e0241b732ac1c1/html5/thumbnails/21.jpg)
8 hyperspectral images of natural scenes
![Page 22: Thomas Wachtler, Te-Won Lee and Terrence J. Sejnowski · Thomas Wachtler, Te-Won Lee and Terrence J. Sejnowski presented by Andrew Rabinovich October 17th, 2002. Goal:Find efficient](https://reader036.vdocuments.us/reader036/viewer/2022071019/5fd33c1711e0241b732ac1c1/html5/thumbnails/22.jpg)
RGB vs. Hyperspectral
![Page 23: Thomas Wachtler, Te-Won Lee and Terrence J. Sejnowski · Thomas Wachtler, Te-Won Lee and Terrence J. Sejnowski presented by Andrew Rabinovich October 17th, 2002. Goal:Find efficient](https://reader036.vdocuments.us/reader036/viewer/2022071019/5fd33c1711e0241b732ac1c1/html5/thumbnails/23.jpg)
![Page 24: Thomas Wachtler, Te-Won Lee and Terrence J. Sejnowski · Thomas Wachtler, Te-Won Lee and Terrence J. Sejnowski presented by Andrew Rabinovich October 17th, 2002. Goal:Find efficient](https://reader036.vdocuments.us/reader036/viewer/2022071019/5fd33c1711e0241b732ac1c1/html5/thumbnails/24.jpg)
ICA PCA
![Page 25: Thomas Wachtler, Te-Won Lee and Terrence J. Sejnowski · Thomas Wachtler, Te-Won Lee and Terrence J. Sejnowski presented by Andrew Rabinovich October 17th, 2002. Goal:Find efficient](https://reader036.vdocuments.us/reader036/viewer/2022071019/5fd33c1711e0241b732ac1c1/html5/thumbnails/25.jpg)
Hyperspectral
HCEV
![Page 26: Thomas Wachtler, Te-Won Lee and Terrence J. Sejnowski · Thomas Wachtler, Te-Won Lee and Terrence J. Sejnowski presented by Andrew Rabinovich October 17th, 2002. Goal:Find efficient](https://reader036.vdocuments.us/reader036/viewer/2022071019/5fd33c1711e0241b732ac1c1/html5/thumbnails/26.jpg)
East/WestNorth/South
LMS – Opponent color space
S
L-M
Pixels
(Long, Medium, Short), not Least Means Squares
![Page 27: Thomas Wachtler, Te-Won Lee and Terrence J. Sejnowski · Thomas Wachtler, Te-Won Lee and Terrence J. Sejnowski presented by Andrew Rabinovich October 17th, 2002. Goal:Find efficient](https://reader036.vdocuments.us/reader036/viewer/2022071019/5fd33c1711e0241b732ac1c1/html5/thumbnails/27.jpg)
![Page 28: Thomas Wachtler, Te-Won Lee and Terrence J. Sejnowski · Thomas Wachtler, Te-Won Lee and Terrence J. Sejnowski presented by Andrew Rabinovich October 17th, 2002. Goal:Find efficient](https://reader036.vdocuments.us/reader036/viewer/2022071019/5fd33c1711e0241b732ac1c1/html5/thumbnails/28.jpg)
![Page 29: Thomas Wachtler, Te-Won Lee and Terrence J. Sejnowski · Thomas Wachtler, Te-Won Lee and Terrence J. Sejnowski presented by Andrew Rabinovich October 17th, 2002. Goal:Find efficient](https://reader036.vdocuments.us/reader036/viewer/2022071019/5fd33c1711e0241b732ac1c1/html5/thumbnails/29.jpg)
• Broadband and broad peaked basis functions
• Achromatic and color-opponent basis functions with non-orthogonal opponencydirection
• More accurate and efficient than PCA
![Page 30: Thomas Wachtler, Te-Won Lee and Terrence J. Sejnowski · Thomas Wachtler, Te-Won Lee and Terrence J. Sejnowski presented by Andrew Rabinovich October 17th, 2002. Goal:Find efficient](https://reader036.vdocuments.us/reader036/viewer/2022071019/5fd33c1711e0241b732ac1c1/html5/thumbnails/30.jpg)
Te-Won Lee, Serge Belongie and Terrence J. Sejnowski
![Page 31: Thomas Wachtler, Te-Won Lee and Terrence J. Sejnowski · Thomas Wachtler, Te-Won Lee and Terrence J. Sejnowski presented by Andrew Rabinovich October 17th, 2002. Goal:Find efficient](https://reader036.vdocuments.us/reader036/viewer/2022071019/5fd33c1711e0241b732ac1c1/html5/thumbnails/31.jpg)
• Aapo Hyvarinen, Juha Karhunen and ErkkiOja. Independent Component Analysis. Wiley-Interscience Publication, 2001
• Te-Won Lee, Thomas Wachtler and Terrence J. Sejnowski. Color Opponencyis an efficient representation of spectral properties in natural scenes. Vision Research 42 (2002)
![Page 32: Thomas Wachtler, Te-Won Lee and Terrence J. Sejnowski · Thomas Wachtler, Te-Won Lee and Terrence J. Sejnowski presented by Andrew Rabinovich October 17th, 2002. Goal:Find efficient](https://reader036.vdocuments.us/reader036/viewer/2022071019/5fd33c1711e0241b732ac1c1/html5/thumbnails/32.jpg)
This document was created with Win2PDF available at http://www.daneprairie.com.The unregistered version of Win2PDF is for evaluation or non-commercial use only.