an introduction to blind source separation kenny hild sept. 19, 2001
TRANSCRIPT
![Page 1: An Introduction to Blind Source Separation Kenny Hild Sept. 19, 2001](https://reader035.vdocuments.us/reader035/viewer/2022070410/56649f285503460f94c40f51/html5/thumbnails/1.jpg)
An Introduction to BlindSource Separation
Kenny Hild
Sept. 19, 2001
![Page 2: An Introduction to Blind Source Separation Kenny Hild Sept. 19, 2001](https://reader035.vdocuments.us/reader035/viewer/2022070410/56649f285503460f94c40f51/html5/thumbnails/2.jpg)
Problem Statement
• Communication System• Transmitter
• Medium
• Receiver
1. Data sent is unknown
2. Transfer function of medium may be unknown
3. Interference
![Page 3: An Introduction to Blind Source Separation Kenny Hild Sept. 19, 2001](https://reader035.vdocuments.us/reader035/viewer/2022070410/56649f285503460f94c40f51/html5/thumbnails/3.jpg)
Possible Solutions
• Beamforming• Uses geometric information
• Steer antenna array to a desired angle of arrival
• Filtering• Separate based on frequency information
• Blind Source Separation, BSS• Statistical beamforming
• Steer antenna array to directions based on statistics
![Page 4: An Introduction to Blind Source Separation Kenny Hild Sept. 19, 2001](https://reader035.vdocuments.us/reader035/viewer/2022070410/56649f285503460f94c40f51/html5/thumbnails/4.jpg)
Beamforming
• Suppose• Direction of arrival is 00 azimuth
• s1(n) = s2(n) = cos(wn)
• Transfer functions are pure delays
• Then• y(n) = x1(n) + x2(n-), = 0
• y(n) = cos(wn) + cos(wn)
+ cos(wn + ) + cos(wn + )
• y(n) = 2cos(wn)
+ 2cos(()/2)cos(wn + ()/2)
s1(n)s2(n)
x2(n)x1(n)
![Page 5: An Introduction to Blind Source Separation Kenny Hild Sept. 19, 2001](https://reader035.vdocuments.us/reader035/viewer/2022070410/56649f285503460f94c40f51/html5/thumbnails/5.jpg)
Filtering
• Suppose• Signal is low pass,
noise is white
• Signals are bandpass
• Then• Design LPF to remove
high frequencies
-3 -2 -1 0 1 2 30
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
Frequency (normalized rad/s)
noise
signal
-3 -2 -1 0 1 2 30
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
signal #1signal #2
![Page 6: An Introduction to Blind Source Separation Kenny Hild Sept. 19, 2001](https://reader035.vdocuments.us/reader035/viewer/2022070410/56649f285503460f94c40f51/html5/thumbnails/6.jpg)
Assumptions
• Signals• Overlap in time
• Angle of arrival is unknown – prevents beamforming
• Overlap in frequency – prevents filtering
• Blind Source Separation• Does not assume knowledge of DOA
• Does not require signals to be separable in frequency domain
![Page 7: An Introduction to Blind Source Separation Kenny Hild Sept. 19, 2001](https://reader035.vdocuments.us/reader035/viewer/2022070410/56649f285503460f94c40f51/html5/thumbnails/7.jpg)
Applications
• Early diagnosis of pathology in fetus• Each EKG sensor contains a mixture of signals
• Desire is to separate out fetus’ heartbeat
• Hearing aids• Speech discrimination difficult with multiple speakers
• The observations are the signals at each ear
• Cellular communications• CDMA signals utilize overlapping frequency ranges
• Additional signals, multi-path deteriorate performance
![Page 8: An Introduction to Blind Source Separation Kenny Hild Sept. 19, 2001](https://reader035.vdocuments.us/reader035/viewer/2022070410/56649f285503460f94c40f51/html5/thumbnails/8.jpg)
Types of Mixtures
• Memory• Instantaneous
• Convolutive
• Noise• Linearity• Over/under-determined
![Page 9: An Introduction to Blind Source Separation Kenny Hild Sept. 19, 2001](https://reader035.vdocuments.us/reader035/viewer/2022070410/56649f285503460f94c40f51/html5/thumbnails/9.jpg)
Components of Adaptive Filter
• Topology• Instantaneous
• Convolutive
• Criterion• Optimization method
• Gradient descent
• Fixed point
![Page 10: An Introduction to Blind Source Separation Kenny Hild Sept. 19, 2001](https://reader035.vdocuments.us/reader035/viewer/2022070410/56649f285503460f94c40f51/html5/thumbnails/10.jpg)
Topology
• Over-determined, linear mixture• N > M
• H, W are matrices of ARMA filters• Types of topologies
• Frequency-domain
• Time-domain• Feedforward
• Feedback
• Lattice
![Page 11: An Introduction to Blind Source Separation Kenny Hild Sept. 19, 2001](https://reader035.vdocuments.us/reader035/viewer/2022070410/56649f285503460f94c40f51/html5/thumbnails/11.jpg)
Topology
• For Instantaneous Mixtures• H, W are matrices of constants
• Often W is broken down into 2-3 operations• Dimension reduction, (N x M) matrix D
• Spatial whitening, (N x N) matrix W
• Rotations, (N x N) matrix R
W = RWD (N x M)
x = Hs (M x 1)
y = Wx = WHs (N x 1)
![Page 12: An Introduction to Blind Source Separation Kenny Hild Sept. 19, 2001](https://reader035.vdocuments.us/reader035/viewer/2022070410/56649f285503460f94c40f51/html5/thumbnails/12.jpg)
Topology
• Spatial whitening• Makes outputs uncorrelated
• This is insufficient
• For separation• 4 possible rotations
-5 -4 -3 -2 -1 0 1 2 3 4 5-5
-4
-3
-2
-1
0
1
2
3
4
5
s1(n)
s2(n)
y1(n)y2(n)
![Page 13: An Introduction to Blind Source Separation Kenny Hild Sept. 19, 2001](https://reader035.vdocuments.us/reader035/viewer/2022070410/56649f285503460f94c40f51/html5/thumbnails/13.jpg)
Criterion
• Spatial whitening• x = Wx
• E[xxT] = IN
• W = xx
• J = ij (Rx(i,j) – IN(i,j))2
Rx=
![Page 14: An Introduction to Blind Source Separation Kenny Hild Sept. 19, 2001](https://reader035.vdocuments.us/reader035/viewer/2022070410/56649f285503460f94c40f51/html5/thumbnails/14.jpg)
Criterion
• Indeterminacies• Gain
• Permutation
• Rotations• Find characteristic of sources that is not true for any
mixture
![Page 15: An Introduction to Blind Source Separation Kenny Hild Sept. 19, 2001](https://reader035.vdocuments.us/reader035/viewer/2022070410/56649f285503460f94c40f51/html5/thumbnails/15.jpg)
Criterion
• Nullify correlations• Between nonlinear functions of the outputs
• Nonlinearity can be most any odd function• Cubic
• Hyperbolic tangent
• Requires source pdf’s to be even-symmetric
• Non-linear PCA• If data is sphered, stable points are ICA solution
• Minimizes joint entropy of nonlinear functions of outputs
![Page 16: An Introduction to Blind Source Separation Kenny Hild Sept. 19, 2001](https://reader035.vdocuments.us/reader035/viewer/2022070410/56649f285503460f94c40f51/html5/thumbnails/16.jpg)
Criterion
• Cancellation of HOS• 4th-order (kurtosis) is most common• If y1, y2, y3, y4 can be separated into 2 groups that are
mutually independent, 4th-order cumulant is zero• Must check all 4th-order cumulants• Statistical properties of cumulant estimators are poor
• Central limit theorem• Sum of independent, non-Gaussian sources approaches
Gaussian• Maximize (K-L) distance between marginal pdf and
Gaussian• Must know/estimate the kurtosis for each source
![Page 17: An Introduction to Blind Source Separation Kenny Hild Sept. 19, 2001](https://reader035.vdocuments.us/reader035/viewer/2022070410/56649f285503460f94c40f51/html5/thumbnails/17.jpg)
Criterion
• Maximum Likelihood• Must know/assume source distributions
• Minimize K-L divergence between output pdf’s and known/assumed source pdf’s
• Sensitive to outliers, model mismatch
• Maximize the information flow• Maximize joint entropy of outputs (of the
nonlinearities)
• Nonlinearities should be source cdf’s
• Equivalent to maximum likelihood
![Page 18: An Introduction to Blind Source Separation Kenny Hild Sept. 19, 2001](https://reader035.vdocuments.us/reader035/viewer/2022070410/56649f285503460f94c40f51/html5/thumbnails/18.jpg)
Criterion
• Mutual statistical independence• Oftentimes sources are independent
• Uncorrelatedness does not imply independence
• Canonical criterion
• Difficult to estimate• Solution includes an infinite-limit integral
• Marginal pdf’s estimated by truncated expansion about Gaussian