pattern recognition : clustering and classification richard brereton [email protected]
TRANSCRIPT
CLUSTER ANALYSIS - UNSUPERVISED PATTERN RECOGNITION
•Grouping of objects according to similarity.
•No predefined classes
TAXONOMY
CHEMICAL TAXONOMY
Grouping organisms according to similarity from chemical fingerprints
•DNA base pairs, proteins
•NMR and pyrolysis of extracts
•NIR spectra
SIMILAR PRINCIPLES IN ALL TYPES OF CHEMISTRY
• Chemical archaeology
• Environmental samples
• Food
STEPS IN CLUSTER ANALYSIS
Similarity measures.
Calculate similarity between objects.
Example
Correlation coefficient : higher, more similar
Euclidean distance : smaller, more similar
Euclidean distance
Manhattan distance
Manhattan distance : smaller, more similar
Use correlations for illustration. Group samples.
1. Find most similar, highest correlation.
Objects 2 and 5. 2. Combine them.
3. Work out new correlation of the new object2&5 with the other objects (1,3,4,6).
Linkage methods – determination of new similarity measures of groups.
Several methods.
• Nearest neighbour uses the highest correlation
• Furthest neighbour uses the lowest correlation
• Average linkage uses an average.
Illustrate with nearest neighbour.
Dendrograms
CLUSTER ANALYSIS : SUMMARY
• Similarity measures
• Linkage methods
• Dendrogram
CLASSIFICATION
Many methods.
CONVENTIONAL
LDA (Linear discriminant analysis)
Original statistics : projections
Examples
Orange juices, can we class into origins and can we detect adulteration from NIR spectra?
Class modelling of mussels, can we find which come from polluted site from GC?
Detailed mathematical model
PRINCIPLES : BIVARIATE EXAMPLE
Class A
Class B
line 1
line 2
Class A Class B
centre centre
Often exact cut-off impossible
Class A Class B
centre centre
Class A
Class B
line 1
line 2
Class distance plots
Centre class A
Centre class B
Class distances
Multivariate data : several measurements per class
Example – Fisher Iris data – four measurements per irisPetal width, petal length, sepal width, sepal length
150 Irises, divided into 50 of each species
I. Setosa
I. Versicolor
I. Verginica
SPECIAL DISTANCES USED.
Linear discriminant function between classes A and B
• The first term is simply the difference between the centres of each class – so a more positive value indicates class A.
• The middle term is the inverse of the “pooled variance covariance matrix.
What does this mean? Sometimes measurements are correlated.Sometimes classes are more dispersed.Puts distances on common scale.
•The final term is the measurement for each object.
Discriminant score against sample number : I Versicolor and I Verginica
-35
-30
-25
-20
-15
-10
-5
0
Can shift the scale so that •positive score probably class A, •negative score probably class B.
Note some ambiguities. WAB.
Discriminant score against sample number - adjust for group means
-20
-15
-10
-5
0
5
10
15
Extending to more than 2 classes
Three classes – 2 out of 3 possible discriminant parameters
If we have 3 classes and choose to use WAB and WAC as the
functions, it is easy to see that
•an object belongs to class A if WAB and WAC are both positive,
•an object belongs to class B if WAB is negative and WAC is
greater than WAB, and
•an object belongs to class C if WAC is negative and WAB is
greater than WAC.
WAB
WAC
Class A
Class B
Class C
Mahalanobis distance
Similar idea to the Euclidean distance, i.e. distance to the centre of a class but use the variance covariance matrix for scaling.
0.0
1.0
2.0
3.0
4.0
5.0
0.0 2.0 4.0 6.0 8.0 10.0
Distance to class A
Du
stan
ce t
o c
lass
B
0.0
1.0
2.0
3.0
4.0
5.0
6.0
7.0
8.0
9.0
10.0
0.0 2.0 4.0 6.0 8.0 10.0
Distance to class A
Dis
tan
ce t
o c
lass
B
Class B
Class AOutlier - maybe another class?
Ambiguous
0
1
2
3
4
5
6
7
8
9
10
0 1 2 3 4 5 6 7 8 9 10
I Versicolor I Verginica
0
2
4
6
8
10
12
14
16
18
0 2 4 6 8 10 12 14 16
I Versicolor I Verginica I Serosa
Many classical statistical methods developed first in biology.
Problem for chemists: Mahalanobis distance depends on measurements being more than variables
Spectroscopy, chromatography : often a huge number of measurements per sample.
Solutions
•Variable selection
•PCA prior to performing classification
Many diagnostics
•Modelling power of variables
•Discriminatory power of variables
•Quality of class model
•Probabilities of class membership
•Ambiguous classification : is analytical data good enough?
MANY SOPHISTICATIONS
Large number of methods for classification based on LDA.
•Bayesian methods – based on prior probabilities.
•Methods that try to find optimal groupings before class modelling.
LOTS OF INFORMATION
•Class membership
•Outliers
•Whether another new class
•Is a class well defined or are there subclasses e.g. subspecies or species from different environments
•What measurements are most useful for discrimination. Can we reduce the number of measurements?
•Are there ambiguous samples, and if so do we need more or better measurements?
•Replicates analysis. Is our method sufficiently good for repeatability. Clinical diagnostics.
SIMCA sometimes used in chemometrics as an alternative
•Soft
•Independent
•Modelling of
•Class analogy
Use PCA models
*
Use PCA to model each class independently
•Choose optimal number of PCs
•Use distance from PC model as an indicator of class distance
VALIDATION OF A CLASS MODEL
Procedure. •Establish a training set.•Assess model with a test set.•Use model on real data. Information •Graphical - e.g. diagrams•Quantitative - class distances•Quantitative - probability of membership of a given class.
Training set
Test set
SUMMARY
•Cluster analysis – unsupervised pattern recognition
•Similarity measures
•Linkage
•Dendrograms
•Classification – supervised pattern recognition
•Class models
•Class distances
•Graphical methods