erpwavelab 1st international summer school in biomedical engineering1st international summer school...
Post on 19-Dec-2015
213 views
TRANSCRIPT
Morten MørupTechnical University of Denmark
ERPWAVELAB 1st International Summer School in Biomedical Engineering August 8, 2006 August 8, 2006
Analyzing the Wavelet Transformed EEG using Non-negative Matrix and Tensor Factorization
An introduction to ERPWAVELAB
ERPWAVELAB 1st International Summer School in Biomedical Engineering August 8 2006 August 8 2006
Morten MørupTechnical University of Denmark
Parts of the work done in collaboration with
Lars Kai Hansen, ProfessorDepartment of Signal Processing
Informatics and Mathematical Modeling,Technical University of Denmark
Sidse M. Arnfred, Dr. Med. PhDCognitive Research Unit
Hvidovre HospitalUniversity Hospital of Copenhagen
Morten MørupTechnical University of Denmark
ERPWAVELAB 1st International Summer School in Biomedical Engineering August 8, 2006 August 8, 2006
•The continuous wavelet transform and measures of the event related ERP in the time-frequency domain•Introduction to NMF and extensions to tensor decompositions (PARAFAC & TUCKER)•Accessing significance•A demonstration of ERPWAVELAB•Discussion
OUTLINE
ERPWAVELAB 1st International Summer School in Biomedical Engineering August 8 2006 August 8 2006
Morten MørupTechnical University of Denmark
time
time
frequency
Continuous Wavelet transform
Complex Morlet wavelet - Real part - Complex part
Absolute value of wavelet coefficient
Captures frequency changes through time
ie
ERPWAVELAB 1st International Summer School in Biomedical Engineering August 8 2006 August 8 2006
Morten MørupTechnical University of Denmark
Continuous Wavelet transform (continued)
epoc
h
chan
nel
time-frequency
epoc
h
chan
nel
time
epochtimechannel X epochfrequencytimechannel X
ERPWAVELAB 1st International Summer School in Biomedical Engineering August 8 2006 August 8 2006
Morten MørupTechnical University of Denmark
The Vector strength
Vectors coherent, i.e. correlated Vectors incoherent, i.e. uncorrelated
Vector strength a measure of coherence
ERPWAVELAB 1st International Summer School in Biomedical Engineering August 8 2006 August 8 2006
Morten MørupTechnical University of Denmark
Measures of the event related ERP in the time-frequency domain
ERSP
WTav
ITPC
avWT
ERPWAVELAB 1st International Summer School in Biomedical Engineering August 8 2006 August 8 2006
Morten MørupTechnical University of Denmark
Measures of the event related ERP in the time-frequency domain (cont.)
Since scalp works as low pass filter it is customary to normalize X before calculating each measure. Frequently used normalizations are:(where tb are points in the baseline region and Tb the total number of baseline samples)
fntfcX
ntfcXT
ntfcXb
b
T
tb
b
),,,(or
),,,(1
),,,(
ERPWAVELAB 1st International Summer School in Biomedical Engineering August 8 2006 August 8 2006
Morten MørupTechnical University of Denmark
ERPWAVELAB demonstration, tutorial dataset 1
ERSP
WTav avWT INDUCED
ITPC
ERPWAVELAB 1st International Summer School in Biomedical Engineering August 8 2006 August 8 2006
Morten MørupTechnical University of Denmark
f’=f, t’=t
ERPWAVELAB 1st International Summer School in Biomedical Engineering August 8 2006 August 8 2006
Morten MørupTechnical University of Denmark
Multi-channel decomposition
time-frequency activities appear to be similar across channels but varying in strength.
Motivates to decompose the activity into similar time-frequency signatures varying in strength in the recording channels.Thus, this form of decomposition is primarily useful for data exploratory purposes giving very easy summaries of what types of activities are present in the data. Only when the measures of interest can be assumed linear and no cancellation between sources are present the decomposition can also reveal the underlying true sources.
≈
ERPWAVELAB 1st International Summer School in Biomedical Engineering August 8 2006 August 8 2006
Morten MørupTechnical University of Denmark
Factor Analysis
dW
dH
Spearman ~1900
VWH
d
Vtests x subjects Wtests x intelligencesHintelligencesxsubject
test
s
Subjects Subjects
test
s
Int.
Int.
Non-negative Matrix Factorization (NMF): VWH s.t. Wi,d,Hd,j0
(~1970 Lawson, ~1995 Paatero, ~2000 Lee & Seung)
ERPWAVELAB 1st International Summer School in Biomedical Engineering August 8 2006 August 8 2006
Morten MørupTechnical University of Denmark
Non-negative matrix factorization (NMF)NMF: VWH s.t. Wi,d,Hd,j0
Multiplicative updates: Let C be a given cost function
Positive termNegative term
ERPWAVELAB 1st International Summer School in Biomedical Engineering August 8 2006 August 8 2006
Morten MørupTechnical University of Denmark
Non-negative matrix factorization (NMF)
(Lee & Seung - 2001)
NMF gives Part based representation (Lee & Seung – Nature 1999)
ERPWAVELAB 1st International Summer School in Biomedical Engineering August 8 2006 August 8 2006
Morten MørupTechnical University of Denmark
The NMF decomposition is not unique
Simplical Cone
NMF only unique when data adequately spans the positive orthant (Donoho & Stodden - 2004)
HWH)(WP)(PWHV -1 ~~
z
y
x
Convex Hull
z
y
x
Positive Orthant
z
yx
Simplical Cones
ERPWAVELAB 1st International Summer School in Biomedical Engineering August 8 2006 August 8 2006
Morten MørupTechnical University of Denmark
Sparse Coding NMF (SNMF)
(Eggert & Körner, 2004) (Mørup & Schmidt, 2006)
ERPWAVELAB 1st International Summer School in Biomedical Engineering August 8 2006 August 8 2006
Morten MørupTechnical University of Denmark
Why sparseness?
• Ensures uniqueness• Eases interpretability
(sparse representation factor effects pertain to fewer dimensions)
• Can work as model selection(Sparseness can turn off excess factors by letting them become zero)
• Resolves over complete representations (when model has many more free variables than data points)
ERPWAVELAB 1st International Summer School in Biomedical Engineering August 8 2006 August 8 2006
Morten MørupTechnical University of Denmark
time-frequency Subjec
ts/Con
dition
/Tria
ls
chan
nel
Often extra modalities such as subjects, conditions and trials are present, consequently the data forms a tensor.
Need for tensor decomposition
ERPWAVELAB 1st International Summer School in Biomedical Engineering August 8 2006 August 8 2006
Morten MørupTechnical University of Denmark
Higher Order Non-negative Matrix Factorization
2dA
1dA
dW
dH
1dA
3dA
1A
3A
2A
C
Factor Analysis PARAFAC TUCKER
D
ddidiii
12121
HWV 3
1
21
321321 di
D
ddidiiii AAA
V 3
3
2
2
33
1
1
2211321321
321J
j
J
jji
J
jjijijjjiii AAACV
= 321
13
1
321
1211111321 ji
J
jjjjijijjjiii AAA
CV
ERPWAVELAB 1st International Summer School in Biomedical Engineering August 8 2006 August 8 2006
Morten MørupTechnical University of Denmark
Uniqueness• Although PARAFAC in general is unique under mild conditions, the
proof of uniqueness by Kruskal is based on k-rank*. However, the k-rank does not apply for non-negativity**.
• TUCKER model is not unique, thus no guaranty of uniqueness.Imposing sparseness useful in order to achieve unique decompositions
Tensor decompositions known to have problems with degeneracy, however when imposing non-negativity degenerate solutions can’t occur***
*) k-rank: The maximum number of columns chosen by random of a matrix certain to be linearly independent. **) L.-H. Lim and G.H. Golub, 2006.***) See L.-H. Lim - http://www.etis.ensea.fr/~wtda/Articles/wtda-nnparafac-slides.pdf
ERPWAVELAB 1st International Summer School in Biomedical Engineering August 8 2006 August 8 2006
Morten MørupTechnical University of Denmark
Example why Non-negative PARAFAC isn’t unique
10
002
½1
½12,
10
10½
½1
½1:
10
002
½1
½12,
10
00½
00
10½
½1
½1:
10
003
00
10
01
012,
10
00
00
10
01
01:
20
00
01
01
11
11,
11
11:
:3
22:
32
12
11
01)
1
2(
11
11
11
11
11
01)
0
1(
11
11
12
01,
11
01,
11
11
21
21
21
21
2
1
)3()2()1(
XX
XX
XX
XX
X
X
AAA
IV
III
II
I
ranknegativeNon
satisfiedFKKKconditionKruskal
diag
diag
CBA
T
T
ERPWAVELAB 1st International Summer School in Biomedical Engineering August 8 2006 August 8 2006
Morten MørupTechnical University of Denmark
PARAFAC model estimation
2dA
1dA
3dA
3
1
21
321321 di
D
ddidiiii AAA
V
V
T123
33
AAZ
ZAV 3
T231
111
AAZ
ZAV
T132
222
AAZ
ZAV
Thus, the PARAFAC model is by the matricizing operationThus, the PARAFAC model is by the matricizing operationestimated straight forward from regular NMF estimationestimated straight forward from regular NMF estimation
JJ BABABABA 2211
ERPWAVELAB 1st International Summer School in Biomedical Engineering August 8 2006 August 8 2006
Morten MørupTechnical University of Denmark
1dA
1A
3A
2A
TUCKER
3
3
2
2
33
1
1
2211321321
321J
j
J
jji
J
jjijijjjiii AAACV
TUCKER model estimation
T123
3
33
AACZ
ZAV 3
T23)1(
1
111
AACZ
ZAV
T132
2
222
AACZ
ZAV
)( 123 AAA CV vecvec
ERPWAVELAB 1st International Summer School in Biomedical Engineering August 8 2006 August 8 2006
Morten MørupTechnical University of Denmark
Algorithm outline (TUCKER) (PARAFAC follows by setting C=I)
ERPWAVELAB 1st International Summer School in Biomedical Engineering August 8 2006 August 8 2006
Morten MørupTechnical University of Denmark
ERPWAVELAB 1st International Summer School in Biomedical Engineering August 8 2006 August 8 2006
Morten MørupTechnical University of Denmark
Accessing Significance
• Comparison to known distribution • Bootstrapping• Cross validation
ERPWAVELAB 1st International Summer School in Biomedical Engineering August 8 2006 August 8 2006
Morten MørupTechnical University of Denmark
Comparison to known distribution
(Mardia, Directional Statistics)
Rayleigh distributedRed: Theoretical mean value of N-½
Black: Mean value estimated by bootstrapping
Normal distributed
Random ITPC and ERPCOH corresponds to a random walk in the complex planethus is Rayleigh distributed.
2:
1:
)(:
2
2
2
2
2
22
mean
ecdf
ex
xfpdf
x
x
ERPWAVELAB 1st International Summer School in Biomedical Engineering August 8 2006 August 8 2006
Morten MørupTechnical University of Denmark
Bootstrapping
1) Randomly select Data from the epochs to form new datasets (each epoch might be represented 0, 1 or several times in the datasets). 2) Calculate the measure of interest for each of these datasets.3) Evaluate the values found to the distribution of values found by the bootstrap datasets.
ERPWAVELAB 1st International Summer School in Biomedical Engineering August 8 2006 August 8 2006
Morten MørupTechnical University of Denmark
Cross validation
• Split dataset into exploratory and confirmatory datasets.
• Find significant activity in exploratory dataset
• See if this activity is also significant in confirmatory dataset
Correcting for multiple comparison by bootstrapping very expensive and often too conservative. Thus Cross validation useful.
ERPWAVELAB 1st International Summer School in Biomedical Engineering August 8 2006 August 8 2006
Morten MørupTechnical University of Denmark
– Dataset generation– Single subject analysis
Artifact rejection in the time frequency domainNMF decompositionCross coherence tracking
– Multi subject analysisClusteringAnalysis of Variance (ANOVA)Tensor decomposition
ERPWAVELAB Tutorial:
The toolbox is free to download from www.erpwavelab.com
ERPWAVELAB 1st International Summer School in Biomedical Engineering August 8 2006 August 8 2006
Morten MørupTechnical University of Denmark
Epilog: Some History of PARAFAC and EEG
• Harshman (1970) (Suggested its use on EEG)• Möcks (1988) (Topographic Component Analysis)
ERP of (channel x time x subject)• Field and Graupe (1991)
ERP of (channel x time x subject)• Miwakeichi et al. (2004)
EEG of (channel x time x frequency)• Mørup et al. (2005)
ERP of ITPC (channel x time x frequency x subject x condition)
ERPWAVELAB 1st International Summer School in Biomedical Engineering August 8 2006 August 8 2006
Morten MørupTechnical University of Denmark
References
Bro, R., 1998. Multi-way Analysis in the Food Industry: Models, algorithms and Applications. Amsterdam, Copenhagen.Bro, R.,Jong, S. D., 1997. A fast non-negativity-constrained least squares algorithm. J. Chemom. 11, 393–401Carroll, J. D. and Chang, J. J. Analysis of individual differences in multidimensional scaling via an N-way generalization of "Eckart-Young" decomposition, Psychometrika 35 1970 283—319Delorme, A.,Makeig, S., 2004. EEGLAB: an open source toolbox for analysis of single-trial EEG dynamics including independent component analysis. J Neurosci Methods 134, 9-21Donoho, D. and Stodden, V. When does non-negative matrix factorization give a correct decomposition into parts? NIPS2003Eggert, J. and Korner, E. Sparse coding and NMF. In Neural Networks volume 4, pages 2529-2533, 2004Field, Aaron S.; Graupe, Daniel “Topographich Component (Parallel Factor) analysis of Multichannel Evoked Potentials: Practical Issues in Trilinear Spatiotemporal Decomposition” Brain Topographa, Vol. 3, Nr. 4, 1991 Fiitzgerald, D. et al. Non-negative tensor factorization for sound source separation. In proceedings of Irish Signals and Systems Conference, 2005Kruskal, J.B. Three-way analysis: rank and uniqueness of trilinear decompostions, with application to arithmetic complexity and statistics. Linear Algebra Appl., 18: 95-138, 1977Harshman, R. A. Foundations of the PARAFAC procedure: Models and conditions for an "explanatory" multi-modal factor analysis},UCLA Working Papers in Phonetics 16 1970 1—84Herrmann, C. S., Grigutsch, M.,Busch, N. A., 2005. EEG oscillations and wavelet analysis.Lathauwer, Lieven De and Moor, Bart De and Vandewalle, Joos MULTILINEAR SINGULAR VALUE DECOMPOSITION.SIAM J. MATRIX ANAL. APPL.2000 (21)1253–1278Lee, D.D. and Seung, H.S. Algorithms for non-negative matrix factorization. In NIPS, pages 556-462, 2000Lee, D.D and Seung, H.S. Learning the parts of objects by non-negative matrix factorization, NATURE 1999Lim, Lek-Heng - http://www.etis.ensea.fr/~wtda/Articles/wtda-nnparafac-slides.pdf
Lim, L.-H. and Golub, G.H., "Nonnegative decomposition and approximation of nonnegative matrices and tensors," SCCM Technical Report, 06-01, forthcoming, 2006.Mardia, K. V.,Jupp, P. E., 1999. Directional Statistics. WILEY & SONS 76-77Miwakeichi, F., Martinez-Montes, E., Valdes-Sosa, P. A., Nishiyama, N., Mizuhara, H., Yamaguchi, Y., 2004. Decomposing EEG data into space-time-frequency components using Parallel Factor Analysis. Neuroimage 22, 1035-1045.
Möcks, J., 1988. Decomposing event-related potentials: a new topographic components model. Biol. Psychol. 26, 199-215.
Mørup, M and Hansen, L.K and Herman, C.S. and Parnas, Josef and Arnfrede, Sidse M. “Parallel Factor Analysis as an exploratory tool for wavelet transformed event –related EEG” NeuroImage 20, 938-947 (2006)
Mørup, M. and Hansen, L.K.and Arnfred, S.M.Decomposing the time-frequency representation of EEG using nonnegative matrix and multi-way factorization Technical report, Institute for Mathematical Modeling, Technical University of Denmark, 2006a
Mørup, M. and Schmidt, M.N. Sparse non-negative matrix factor 2-D deconvolution. Technical report, Institute for Mathematical Modeling, Technical University of Denmark, 2006b
Mørup, M. and Hansen, L.K.and Arnfred, S.M. Algorithms for Sparse Higher Order Non-negative Matrix Factorization (HONMF), Technical report, Institute for Mathematical Modeling, Technical University of Denmark, 2006e
Tamara G. Kolda Multilinear operators for higher-order decompositions technical report Sandia national laboratory 2006 SAND2006-2081.
Tucker, L. R. Some mathematical notes on three-mode factor analysis Psychometrika 31 1966 279—311
Welling, M. and Weber, M. Positive tensor factorization. Pattern Recogn. Lett. 2001