gaussian process regression within an active learning scheme
DESCRIPTION
University of Trento Dept . of Information Engineering and Computer Science Italy. GAUSSIAN PROCESS REGRESSION WITHIN AN ACTIVE LEARNING SCHEME. Edoardo Pasolli [email protected]. Farid Melgani [email protected]. IGARSS 2011. Introduction. Supervised regression approach. - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: GAUSSIAN PROCESS REGRESSION WITHIN AN ACTIVE LEARNING SCHEME](https://reader036.vdocuments.us/reader036/viewer/2022062500/56815065550346895dbe62f2/html5/thumbnails/1.jpg)
GAUSSIAN PROCESS REGRESSION WITHINAN ACTIVE LEARNING SCHEME
IGARSS 2011
University of TrentoDept. of Information Engineering and
Computer ScienceItaly
Edoardo Pasolli
Farid [email protected]
n.it
July 28, 2011
![Page 2: GAUSSIAN PROCESS REGRESSION WITHIN AN ACTIVE LEARNING SCHEME](https://reader036.vdocuments.us/reader036/viewer/2022062500/56815065550346895dbe62f2/html5/thumbnails/2.jpg)
Introduction
Supervised regression approach
2
Pre-processing
Feature extraction
RegressionImage/Signal Prediction
Training sample
collection
Training sample quality/quantity
Human expert
Impact on prediction errors
![Page 3: GAUSSIAN PROCESS REGRESSION WITHIN AN ACTIVE LEARNING SCHEME](https://reader036.vdocuments.us/reader036/viewer/2022062500/56815065550346895dbe62f2/html5/thumbnails/3.jpg)
Introduction
Active learning approach for classification problems
3
Trainingof classifier
Active learningmethod
Model of classifier
Learning (unlabeled) set
Labeling of selected samples
Selected samplesafter labeling
Insertion in training set
f1
f2
f1
f2
f1
f2
Selected samples from
learning (unlabeled) set
f2
f1
f2
f1
Human expert
Training (labeled) set
Class 1 Class
3
Class 2
![Page 4: GAUSSIAN PROCESS REGRESSION WITHIN AN ACTIVE LEARNING SCHEME](https://reader036.vdocuments.us/reader036/viewer/2022062500/56815065550346895dbe62f2/html5/thumbnails/4.jpg)
Objective
Propose GP-based active learning strategies for biophysical parameter estimation problems
4
![Page 5: GAUSSIAN PROCESS REGRESSION WITHIN AN ACTIVE LEARNING SCHEME](https://reader036.vdocuments.us/reader036/viewer/2022062500/56815065550346895dbe62f2/html5/thumbnails/5.jpg)
Gaussian Processes (GPs)
Predictive distribution
5
2**** ,~,,| NXf xy
yk 12
** , IXXK nt *
12***
2* ,, kkxx
IXXKk n
t
fy
IN n2,0~ XXKGPf ,,0~
IXXKGPy n2,,0~
noise variance
covariance matrix defined by covariance function
',xxk
![Page 6: GAUSSIAN PROCESS REGRESSION WITHIN AN ACTIVE LEARNING SCHEME](https://reader036.vdocuments.us/reader036/viewer/2022062500/56815065550346895dbe62f2/html5/thumbnails/6.jpg)
Gaussian Processes (GPs)6
Example of predicted function
: predicted value
*: standard deviation of predicted value
*
: training sample
![Page 7: GAUSSIAN PROCESS REGRESSION WITHIN AN ACTIVE LEARNING SCHEME](https://reader036.vdocuments.us/reader036/viewer/2022062500/56815065550346895dbe62f2/html5/thumbnails/7.jpg)
Proposed Strategies7
GPRegression
U’s: Selected
unlabeled samples
Labeling
SelectionInsertion intraining set
Human expert
L: Training set
U: Learning set
L’s: Labeled samples
![Page 8: GAUSSIAN PROCESS REGRESSION WITHIN AN ACTIVE LEARNING SCHEME](https://reader036.vdocuments.us/reader036/viewer/2022062500/56815065550346895dbe62f2/html5/thumbnails/8.jpg)
Proposed Strategies
Minimize covariance measure in feature space (Cov)
8
2
2
2
2
'exp,
l'k f
xxxx : squared
exponential covariance functionsignal
variance length-scale
: training sample: covariance function with respect to training sample
',xxk'x
![Page 9: GAUSSIAN PROCESS REGRESSION WITHIN AN ACTIVE LEARNING SCHEME](https://reader036.vdocuments.us/reader036/viewer/2022062500/56815065550346895dbe62f2/html5/thumbnails/9.jpg)
Proposed Strategies
Minimize covariance measure in feature space (Cov)
9
: covariance measure with respect to all training samples
covf
covf
: covariance function with respect to training sample
ik xx,
ix
: training sample
n
iikf
1cov ,xxx xx
xcov
* minarg f
: selection of samples with minimum values of
![Page 10: GAUSSIAN PROCESS REGRESSION WITHIN AN ACTIVE LEARNING SCHEME](https://reader036.vdocuments.us/reader036/viewer/2022062500/56815065550346895dbe62f2/html5/thumbnails/10.jpg)
Proposed Strategies
Maximize variance of predicted value (Var)
10
kkxxx122 ,
IKk n
t
: predicted value
x: standard deviation of predicted value
x
: training sample
![Page 11: GAUSSIAN PROCESS REGRESSION WITHIN AN ACTIVE LEARNING SCHEME](https://reader036.vdocuments.us/reader036/viewer/2022062500/56815065550346895dbe62f2/html5/thumbnails/11.jpg)
Proposed Strategies
Maximize variance of predicted value (Var)
11
xx 2var f xx
xvar
* maxarg f
varf
varf: variance
: training sample
: selection of samples with maximum values of
![Page 12: GAUSSIAN PROCESS REGRESSION WITHIN AN ACTIVE LEARNING SCHEME](https://reader036.vdocuments.us/reader036/viewer/2022062500/56815065550346895dbe62f2/html5/thumbnails/12.jpg)
Experimental Results
Data set description (MERIS) Simulated acquisitions Objective: estimation of chlorophyll
concentration in subsurface case I + case II (open and coastal) waters
Sensor: MEdium Resolution Imaging Spectrometer (MERIS)
# channels: 8 (412-618 nm) Range of chlorophyll concentration: 0.02-54
mg/m3
12
![Page 13: GAUSSIAN PROCESS REGRESSION WITHIN AN ACTIVE LEARNING SCHEME](https://reader036.vdocuments.us/reader036/viewer/2022062500/56815065550346895dbe62f2/html5/thumbnails/13.jpg)
Experimental Results
Data set description (SeaBAM) Real aquisitions Objective: estimation of chlorophyll
concentration mostly in subsurface case I (open) waters
Sensor: Sea-viewing Wide Field-of-view (SeaWiFS)
# channels: 5 (412-555 nm) Range of chlorophyll concentration: 0.02-32.79
mg/m3
13
![Page 14: GAUSSIAN PROCESS REGRESSION WITHIN AN ACTIVE LEARNING SCHEME](https://reader036.vdocuments.us/reader036/viewer/2022062500/56815065550346895dbe62f2/html5/thumbnails/14.jpg)
Experimental Results
Mean Squared Error
14
MERIS SeaBAM
![Page 15: GAUSSIAN PROCESS REGRESSION WITHIN AN ACTIVE LEARNING SCHEME](https://reader036.vdocuments.us/reader036/viewer/2022062500/56815065550346895dbe62f2/html5/thumbnails/15.jpg)
Experimental Results
Standard Deviation of Mean Squared Error
15
MERIS SeaBAM
![Page 16: GAUSSIAN PROCESS REGRESSION WITHIN AN ACTIVE LEARNING SCHEME](https://reader036.vdocuments.us/reader036/viewer/2022062500/56815065550346895dbe62f2/html5/thumbnails/16.jpg)
Experimental Results
Detailed results
16
Accuracies on 4000 test samples
Method#
trainingsamples
MSE σMSE R2 σR2
Full 1000 0.086 - 0.991 -Initial 50 1.638 0.869 0.849 0.070RanCovVar
1500.5850.3780.184
0.4060.1050.054
0.9380.9610.980
0.0450.0100.005
RanCovVar
3000.2370.2120.095
0.0840.1770.005
0.9750.9770.990
0.0080.0180.000
MERIS
![Page 17: GAUSSIAN PROCESS REGRESSION WITHIN AN ACTIVE LEARNING SCHEME](https://reader036.vdocuments.us/reader036/viewer/2022062500/56815065550346895dbe62f2/html5/thumbnails/17.jpg)
Experimental Results
Detailed results
17
Accuracies on 4000 test samples
Method#
trainingsamples
MSE σMSE R2 σR2
Full 1000 0.086 - 0.991 -Initial 50 1.638 0.869 0.849 0.070RanCovVar
1500.5850.3780.184
0.4060.1050.054
0.9380.9610.980
0.0450.0100.005
RanCovVar
3000.2370.2120.095
0.0840.1770.005
0.9750.9770.990
0.0080.0180.000
MERIS
![Page 18: GAUSSIAN PROCESS REGRESSION WITHIN AN ACTIVE LEARNING SCHEME](https://reader036.vdocuments.us/reader036/viewer/2022062500/56815065550346895dbe62f2/html5/thumbnails/18.jpg)
Experimental Results
Detailed results
18
Accuracies on 459 test samples
Method#
trainingsamples
MSE σMSE R2 σR2
Full 460 1.536 - 0.806 -Initial 60 5.221 2.968 0.526 0.215RanCovVar
1602.9722.2101.818
1.0380.0740.029
0.6820.7450.784
0.0690.0070.003
RanCovVar
3102.0621.6011.573
0.6870.0100.003
0.7530.8000.803
0.0660.0010.000
SeaBAM
![Page 19: GAUSSIAN PROCESS REGRESSION WITHIN AN ACTIVE LEARNING SCHEME](https://reader036.vdocuments.us/reader036/viewer/2022062500/56815065550346895dbe62f2/html5/thumbnails/19.jpg)
Experimental Results
Detailed results
19
Accuracies on 459 test samples
Method#
trainingsamples
MSE σMSE R2 σR2
Full 460 1.536 - 0.806 -Initial 60 5.221 2.968 0.526 0.215RanCovVar
1602.9722.2101.818
1.0380.0740.029
0.6820.7450.784
0.0690.0070.003
RanCovVar
3102.0621.6011.573
0.6870.0100.003
0.7530.8000.803
0.0660.0010.000
SeaBAM
![Page 20: GAUSSIAN PROCESS REGRESSION WITHIN AN ACTIVE LEARNING SCHEME](https://reader036.vdocuments.us/reader036/viewer/2022062500/56815065550346895dbe62f2/html5/thumbnails/20.jpg)
Conclusions
In this work, GP-based active learning strategies for regression problems are proposed
Encouraging performances in terms of convergence speed stability
Future developments extension to other regression approaches
20
![Page 21: GAUSSIAN PROCESS REGRESSION WITHIN AN ACTIVE LEARNING SCHEME](https://reader036.vdocuments.us/reader036/viewer/2022062500/56815065550346895dbe62f2/html5/thumbnails/21.jpg)
GAUSSIAN PROCESS REGRESSION WITHINAN ACTIVE LEARNING SCHEME
IGARSS 2011
University of TrentoDept. of Information Engineering and
Computer ScienceItaly
Edoardo Pasolli
Farid [email protected]
n.it
July 28, 2011