kernel methods for de-noising with neuroimaging application trine julie abrahamsen september 8, 2009
Post on 19-Dec-2015
214 views
TRANSCRIPT
08/09/20092
Overview
• Introduction
• Kernel Principal Component Analysis
• The Pre-image Problem
• Analysis of Cimbi Data
• Conclusions
08/09/20093
Introduction
Introduction - Kernel PCA - The Pre-image Problem - Analysis of Cimbi Data – Conclusions
• Kernel PCA for de-noising
• The pre-image problem is a key aspect in achieving good results
Objective
The over all aim of this project is two-fold
• To investigate the pre-image problem
• Apply kernel methods for de-noising on neuroimaging data
08/09/20094
Introducing Kernels
The idea of kernel methods
Often
Introduction - Kernel PCA - The Pre-image Problem - Analysis of Cimbi Data - Conclusions
The Gaussian kernel
Definition of kernel function:
Mika et al. 1999
Schölkopf et al. 2001
08/09/20095
Kernel Principal Component Analysis
Introduction - Kernel PCA - The Pre-image Problem - Analysis of Cimbi Data - Conclusions
Linear PCA is performed in feature space. Thus, the first PC can be found as the normal direction, v1 , by
All solutions must lie in the span of the training images, hence,
The projection of onto the i’th PC can be found as
While the projection onto the subspace spanned by the first q PCs is given by
Schölkopf et al. 1998
08/09/20096
Kernel PCA - Illustration
Introduction - Kernel PCA - The Pre-image Problem - Analysis of Cimbi Data - Conclusions
PCA
Kernel PCA
08/09/20097
The Pre-Image Problem
Introduction - Kernel PCA - The Pre-image Problem - Analysis of Cimbi Data - Conclusions
The pre-image problem = reconstruction of point in input space from feature space point
Ill-posed due to properties of the -map.
Relax search to find approximate pre-image
Common methods seek to minimize the feature space distance
where
Mika et al. 1999
Schölkopf et al. 1999
08/09/20098
Overview of Current Estimation Schemes
Introduction - Kernel PCA - The Pre-image Problem - Analysis of Cimbi Data - Conclusions
Mika et al.
Kwok & Tsang
Dambreville et al.
08/09/20099
Input Space Distance Regularization
Introduction - Kernel PCA - The Pre-image Problem - Analysis of Cimbi Data - Conclusions
Which is equivalent to minimizing
For RBF kernels the cost function (which should be maximized) reduces to
08/09/200910
Experiments on the USPS Data Set
Introduction - Kernel PCA - The Pre-image Problem - Analysis of Cimbi Data - Conclusions
USPS digits Gaussian noise added Mika et al. Input space regularization
Hull 1994
08/09/200911
Experiments on the USPS Data Set
Introduction - Kernel PCA - The Pre-image Problem - Analysis of Cimbi Data - Conclusions
(a) Kwok & Tsang (2004)
(b) Mika et al. (1999)
(c) Dambreville et al. (2006)
(d) Input Space Reg.
Evaluating the stability by confidence intervals on the mean squared error.
08/09/200912
Introducing the Cimbi Analysis
Introduction - Kernel PCA - The Pre-image Problem - Analysis of Cimbi Data - Conclusions
• De-noising for refining statistical significance
• D = 14 regions representing the frontolimbic area.
• Serotonin receptor binding potential quantified from
PET scans
• Neuroticism, Anxiety, and Vulnerability
• N = 129.
• Initial experiments on training and test sets
www.cimbi.org
Frøkjaer 2008
08/09/200913
Applying Kernel PCA De-noising
Introduction - Kernel PCA - The Pre-image Problem - Analysis of Cimbi Data - Conclusions
Vuln
era
bili
ty Original De-noised
Neuroticism p=0.11 p=0.057*
Anxiety p=0.063* p=0.015**
Vulnerability p=0.039** p=0.018**
08/09/200914
Learning curves
Introduction - Kernel PCA - The Pre-image Problem - Analysis of Cimbi Data - Conclusions
• 100 subsets sampled without replacement
• Data set sizes 5,10,…,125,129
• c chosen as 5th percentile and q chosen so 65% of variance is described
• Dambreville et al.
08/09/200915
Conclusions
Introduction - Kernel PCA - The Pre-image Problem - Analysis of Cimbi Data - Conclusions
• Adding input space distance regularization stabilizes the pre-image with limited sacrifice in terms of de-noising efficiency
• For the Cimbi data Dambreville et al.’s method proved very efficient
• When working on all 129 subjects a remarkable decrease in p-value could be achieved for both neuroticism, vulnerability, and anxiety
• Derive guidelines for choosing the regularization parameter λ
• Introduce other types of regularization
• Investigate other applications of kernel PCA (e.g., outlier detection)
• Improve performance by varying the kernel and its parameters
• Include non-linear adjustment for age and gender
Future Work
08/09/200918
Many local minima with almost equal value
Introduction - Kernel PCA - The Pre-image Problem - Analysis of Cimbi Data - Conclusions
08/09/200919
Distance distortions for non-linear kernels
Introduction - Kernel PCA - The Pre-image Problem - Analysis of Cimbi Data - Conclusions
08/09/200920
Permutation Test
Introduction - Kernel PCA - The Pre-image Problem - Analysis of Cimbi Data - Conclusions
• Non-parametric test
• Re-arrange trait score on BP
• p-value is found as the proportion of sampled permutations where the correlation is greater or equal to the correlation found on the original data.
08/09/200921
Experiments on the USPS Data Set
Introduction - Kernel PCA - The Pre-Image Problem - Analysis of Cimbi Data - Conclusions