kohonen maps and counterpropagation artificial neural networks toolbox for matlab davide ballabio...
TRANSCRIPT
Kohonen Maps and Counterpropagation
Artificial Neural Networks Toolbox for MATLAB
Davide BallabioMilano Chemometrics and QSAR Research GroupUniversità Milano - Bicocca
Kohonen and CPANN Toolbox for MATLAB
• Kohonen Maps (or Self Organizing Maps) and Counterpropagation Artificial Neural Networks (CPANNs) are two of the most popular Neural Networks (algorithms that simulate the human learning).
• can handle both supervised and unsupervised problems
• Kohonen and CP-ANN toolbox is a collection of MATLAB modules freely available via internet: http://michem.disat.unimib.it/chm
Kohonen maps
• Kohonen Maps (or Self Organizing Maps) can handle unsupervised problems
SOM are made of neurons and need time (iterations) to learn to describe the data.
The time (number of iterations) and the size (number of neurons) must be defined by the user
• Kohonen Maps (or Self Organizing Maps) can handle unsupervised problems
p Kohonen layers(p = number of variables)
Kohonen maps
AAx1
x2
x3
x4
TOP MAP
Kohonen layers
X
Toroidal geometryToroidal geometry
• Kohonen Maps (or Self Organizing Maps) can handle unsupervised problems
Kohonen maps
• Counterpropagation Artificial neural Networks can handle supervised problems (in this case classification problems)
• CPANNs are an evolution of Kohonen Maps
CPANN
• Class unfolding: transform the class vector in a matrix of zeros and ones
CPANN
Multi-y (n x G)Class (n x 1)
112233..GG
1 0 0 … … … 01 0 0 … … … 00 1 0 … … … 00 1 0 … … … 00 0 1 … … … 00 0 1 … … … 0
0 0 0 … … … 10 0 0 … … … 1
unfolding
AAx1
x2
x3
x4
y1
y2
y3
TOP MAP
• Counterpropagation Artificial neural Networks can handle supervised problems (in this case classification problems)
CPANN
Output layers
Kohonen layers
• Counterpropagation Artificial neural Networks can handle supervised problems (in this case classification problems):
CPANN
out x wc si jii
m
min2
1
For each sample, for each epoch:
1) Find the winning neuron
• Counterpropagation Artificial neural Networks can handle supervised problems (in this case classification problems):
CPANN
For each sample, for each epoch:
2) Update the Kohonen weights
oldwxΔw rir
r η
1d
d1
max
learning rate
topological distance
• Counterpropagation Artificial neural Networks can handle supervised problems (in this case classification problems):
CPANN
For each sample, for each epoch:
3) Update the output weights
oldycΔy rir
r η
1d
d1
max
• In order to better understand how variables can characterize the data, we can do Principal Component Analysis on the Kohonen weights:
Kohonen Maps and CPANN
PCA
Weights matrix
N x p
W
2. Eigenvectors (loadings)analyse the variables
variables
neur
ons
1. Eigenvalues:decide how many component we can retain
3. Scoresanalyse the neurons
relate neurons (and samples placed in neurons) and variables in a global way
Kohonen Maps and CPANN toolbox
• Features of the toolbox:• user defined data scaling + automatic range scaling• modules for fitting and validating models• main available settings:
• size and epochs (required)• boundary condition• learning rates can be modified• missing values are handled
• visualize top map with Graphical User Interface• visualize PCA on weights with Graphical User Interface
The toolbox is based on the algorithm explained in: Zupan J, Novic M, Ruisánchez I. Chemometrics and Intelligent Laboratory Systems (1997) 38 1-23.
Kohonen Maps and CPANN toolbox
• Example of application on the ITAOILS dataset:
• 572 olive oil samples, • each sample is described by the percentage composition of 8 fatty acids (variables)• samples belong to 9 different Italian areas of productions (classes)• the final aim of the classification model is the geographical origin determination of the samples.
Data reference:M. Forina, C. Armanino, S. Lanteri, E. Tiscornia, in: Food Research and Data Analysis, Classification of olive oils from their fatty acid composition, Applied Science Publishers, London, 1983.
Kohonen Maps and CPANN toolbox
• Principal Component Analysis for data structure evaluation
Thanks for your attention !
Davide Ballabio
Milano Chemometrics and QSAR Research GroupDepartment of Environmental SciencesUniversità Milano – Bicocca
You can download the Kohonen and CP-ANN toolbox 1.0 here:http://michem.disat.unimib.it/chm
Version 2.0 will be available soon…
Paper in preparation, to be submitted to Chemometrics and Intelligent Laboratory Systems