9.54 class 13 - mit9.54/fall14/slides/class13.pdf · what is clustering? •the organization of...
TRANSCRIPT
![Page 1: 9.54 Class 13 - MIT9.54/fall14/slides/Class13.pdf · What is clustering? •The organization of unlabeled data into similarity groups called clusters. •A cluster is a collection](https://reader034.vdocuments.us/reader034/viewer/2022051601/5ad0c47d7f8b9ad24f8e0d65/html5/thumbnails/1.jpg)
9.54 Class 13
Unsupervised learning
Clustering
Shimon Ullman + Tomaso Poggio Danny Harari + Daneil Zysman + Darren Seibert
![Page 2: 9.54 Class 13 - MIT9.54/fall14/slides/Class13.pdf · What is clustering? •The organization of unlabeled data into similarity groups called clusters. •A cluster is a collection](https://reader034.vdocuments.us/reader034/viewer/2022051601/5ad0c47d7f8b9ad24f8e0d65/html5/thumbnails/2.jpg)
Outline
• Introduction to clustering
• K-means
• Bag of words (dictionary learning)
• Hierarchical clustering
• Competitive learning (SOM)
![Page 3: 9.54 Class 13 - MIT9.54/fall14/slides/Class13.pdf · What is clustering? •The organization of unlabeled data into similarity groups called clusters. •A cluster is a collection](https://reader034.vdocuments.us/reader034/viewer/2022051601/5ad0c47d7f8b9ad24f8e0d65/html5/thumbnails/3.jpg)
What is clustering?
• The organization of unlabeled data into similarity groups called clusters.
• A cluster is a collection of data items which are “similar” between them, and “dissimilar” to data items in other clusters.
![Page 4: 9.54 Class 13 - MIT9.54/fall14/slides/Class13.pdf · What is clustering? •The organization of unlabeled data into similarity groups called clusters. •A cluster is a collection](https://reader034.vdocuments.us/reader034/viewer/2022051601/5ad0c47d7f8b9ad24f8e0d65/html5/thumbnails/4.jpg)
Historic application of clustering
![Page 5: 9.54 Class 13 - MIT9.54/fall14/slides/Class13.pdf · What is clustering? •The organization of unlabeled data into similarity groups called clusters. •A cluster is a collection](https://reader034.vdocuments.us/reader034/viewer/2022051601/5ad0c47d7f8b9ad24f8e0d65/html5/thumbnails/5.jpg)
Computer vision application:Image segmentation
![Page 6: 9.54 Class 13 - MIT9.54/fall14/slides/Class13.pdf · What is clustering? •The organization of unlabeled data into similarity groups called clusters. •A cluster is a collection](https://reader034.vdocuments.us/reader034/viewer/2022051601/5ad0c47d7f8b9ad24f8e0d65/html5/thumbnails/6.jpg)
What do we need for clustering?
![Page 7: 9.54 Class 13 - MIT9.54/fall14/slides/Class13.pdf · What is clustering? •The organization of unlabeled data into similarity groups called clusters. •A cluster is a collection](https://reader034.vdocuments.us/reader034/viewer/2022051601/5ad0c47d7f8b9ad24f8e0d65/html5/thumbnails/7.jpg)
Distance (dissimilarity) measures
They are special cases of Minkowski distance:
(p is a positive integer)
ppm
k jkikjip xxd
1
1),(
xx
![Page 8: 9.54 Class 13 - MIT9.54/fall14/slides/Class13.pdf · What is clustering? •The organization of unlabeled data into similarity groups called clusters. •A cluster is a collection](https://reader034.vdocuments.us/reader034/viewer/2022051601/5ad0c47d7f8b9ad24f8e0d65/html5/thumbnails/8.jpg)
Cluster evaluation (a hard problem)
• Intra-cluster cohesion (compactness):– Cohesion measures how near the data points in a
cluster are to the cluster centroid.
– Sum of squared error (SSE) is a commonly used measure.
• Inter-cluster separation (isolation): – Separation means that different cluster centroids
should be far away from one another.
• In most applications, expert judgments are still the key
![Page 9: 9.54 Class 13 - MIT9.54/fall14/slides/Class13.pdf · What is clustering? •The organization of unlabeled data into similarity groups called clusters. •A cluster is a collection](https://reader034.vdocuments.us/reader034/viewer/2022051601/5ad0c47d7f8b9ad24f8e0d65/html5/thumbnails/9.jpg)
How many clusters?
![Page 10: 9.54 Class 13 - MIT9.54/fall14/slides/Class13.pdf · What is clustering? •The organization of unlabeled data into similarity groups called clusters. •A cluster is a collection](https://reader034.vdocuments.us/reader034/viewer/2022051601/5ad0c47d7f8b9ad24f8e0d65/html5/thumbnails/10.jpg)
Clustering techniques
Divisive
![Page 11: 9.54 Class 13 - MIT9.54/fall14/slides/Class13.pdf · What is clustering? •The organization of unlabeled data into similarity groups called clusters. •A cluster is a collection](https://reader034.vdocuments.us/reader034/viewer/2022051601/5ad0c47d7f8b9ad24f8e0d65/html5/thumbnails/11.jpg)
Clustering techniques
![Page 12: 9.54 Class 13 - MIT9.54/fall14/slides/Class13.pdf · What is clustering? •The organization of unlabeled data into similarity groups called clusters. •A cluster is a collection](https://reader034.vdocuments.us/reader034/viewer/2022051601/5ad0c47d7f8b9ad24f8e0d65/html5/thumbnails/12.jpg)
Clustering techniques
Divisive
K-means
![Page 13: 9.54 Class 13 - MIT9.54/fall14/slides/Class13.pdf · What is clustering? •The organization of unlabeled data into similarity groups called clusters. •A cluster is a collection](https://reader034.vdocuments.us/reader034/viewer/2022051601/5ad0c47d7f8b9ad24f8e0d65/html5/thumbnails/13.jpg)
K-Means clustering
• K-means (MacQueen, 1967) is a partitional clusteringalgorithm
• Let the set of data points D be {x1, x2, …, xn},
where xi = (xi1, xi2, …, xir) is a vector in X Rr, and r is the number of dimensions.
• The k-means algorithm partitions the given data into k clusters:
– Each cluster has a cluster center, called centroid.
– k is specified by the user
![Page 14: 9.54 Class 13 - MIT9.54/fall14/slides/Class13.pdf · What is clustering? •The organization of unlabeled data into similarity groups called clusters. •A cluster is a collection](https://reader034.vdocuments.us/reader034/viewer/2022051601/5ad0c47d7f8b9ad24f8e0d65/html5/thumbnails/14.jpg)
K-means algorithm
• Given k, the k-means algorithm works as follows:
1. Choose k (random) data points (seeds) to be the initial centroids, cluster centers
2. Assign each data point to the closest centroid
3. Re-compute the centroids using the current cluster memberships
4. If a convergence criterion is not met, repeat steps 2 and 3
![Page 15: 9.54 Class 13 - MIT9.54/fall14/slides/Class13.pdf · What is clustering? •The organization of unlabeled data into similarity groups called clusters. •A cluster is a collection](https://reader034.vdocuments.us/reader034/viewer/2022051601/5ad0c47d7f8b9ad24f8e0d65/html5/thumbnails/15.jpg)
K-means convergence (stopping) criterion
• no (or minimum) re-assignments of data points to different clusters, or
• no (or minimum) change of centroids, or• minimum decrease in the sum of squared error (SSE),
– Cj is the jth cluster, – mj is the centroid of cluster Cj (the mean vector of all the
data points in Cj), – d(x, mj) is the (Eucledian) distance between data point x
and centroid mj.
k
jC j
j
dSSE1
2),(x
mx
![Page 16: 9.54 Class 13 - MIT9.54/fall14/slides/Class13.pdf · What is clustering? •The organization of unlabeled data into similarity groups called clusters. •A cluster is a collection](https://reader034.vdocuments.us/reader034/viewer/2022051601/5ad0c47d7f8b9ad24f8e0d65/html5/thumbnails/16.jpg)
K-means clustering example:step 1
![Page 17: 9.54 Class 13 - MIT9.54/fall14/slides/Class13.pdf · What is clustering? •The organization of unlabeled data into similarity groups called clusters. •A cluster is a collection](https://reader034.vdocuments.us/reader034/viewer/2022051601/5ad0c47d7f8b9ad24f8e0d65/html5/thumbnails/17.jpg)
K-means clustering example –step 2
![Page 18: 9.54 Class 13 - MIT9.54/fall14/slides/Class13.pdf · What is clustering? •The organization of unlabeled data into similarity groups called clusters. •A cluster is a collection](https://reader034.vdocuments.us/reader034/viewer/2022051601/5ad0c47d7f8b9ad24f8e0d65/html5/thumbnails/18.jpg)
K-means clustering example –step 3
![Page 19: 9.54 Class 13 - MIT9.54/fall14/slides/Class13.pdf · What is clustering? •The organization of unlabeled data into similarity groups called clusters. •A cluster is a collection](https://reader034.vdocuments.us/reader034/viewer/2022051601/5ad0c47d7f8b9ad24f8e0d65/html5/thumbnails/19.jpg)
K-means clustering example
![Page 20: 9.54 Class 13 - MIT9.54/fall14/slides/Class13.pdf · What is clustering? •The organization of unlabeled data into similarity groups called clusters. •A cluster is a collection](https://reader034.vdocuments.us/reader034/viewer/2022051601/5ad0c47d7f8b9ad24f8e0d65/html5/thumbnails/20.jpg)
K-means clustering example
![Page 21: 9.54 Class 13 - MIT9.54/fall14/slides/Class13.pdf · What is clustering? •The organization of unlabeled data into similarity groups called clusters. •A cluster is a collection](https://reader034.vdocuments.us/reader034/viewer/2022051601/5ad0c47d7f8b9ad24f8e0d65/html5/thumbnails/21.jpg)
K-means clustering example
![Page 22: 9.54 Class 13 - MIT9.54/fall14/slides/Class13.pdf · What is clustering? •The organization of unlabeled data into similarity groups called clusters. •A cluster is a collection](https://reader034.vdocuments.us/reader034/viewer/2022051601/5ad0c47d7f8b9ad24f8e0d65/html5/thumbnails/22.jpg)
Why use K-means?
• Strengths:
– Simple: easy to understand and to implement
– Efficient: Time complexity: O(tkn),
where n is the number of data points,
k is the number of clusters, and
t is the number of iterations.
– Since both k and t are small. k-means is considered a linear algorithm.
• K-means is the most popular clustering algorithm.
• Note that: it terminates at a local optimum if SSE is used. The global optimum is hard to find due to complexity.
![Page 23: 9.54 Class 13 - MIT9.54/fall14/slides/Class13.pdf · What is clustering? •The organization of unlabeled data into similarity groups called clusters. •A cluster is a collection](https://reader034.vdocuments.us/reader034/viewer/2022051601/5ad0c47d7f8b9ad24f8e0d65/html5/thumbnails/23.jpg)
Weaknesses of K-means
• The algorithm is only applicable if the mean is defined. – For categorical data, k-mode - the centroid is
represented by most frequent values.
• The user needs to specify k.
• The algorithm is sensitive to outliers– Outliers are data points that are very far away
from other data points.
– Outliers could be errors in the data recording or some special data points with very different values.
![Page 24: 9.54 Class 13 - MIT9.54/fall14/slides/Class13.pdf · What is clustering? •The organization of unlabeled data into similarity groups called clusters. •A cluster is a collection](https://reader034.vdocuments.us/reader034/viewer/2022051601/5ad0c47d7f8b9ad24f8e0d65/html5/thumbnails/24.jpg)
Outliers
![Page 25: 9.54 Class 13 - MIT9.54/fall14/slides/Class13.pdf · What is clustering? •The organization of unlabeled data into similarity groups called clusters. •A cluster is a collection](https://reader034.vdocuments.us/reader034/viewer/2022051601/5ad0c47d7f8b9ad24f8e0d65/html5/thumbnails/25.jpg)
Dealing with outliers
• Remove some data points that are much further away from the centroids than other data points
– To be safe, we may want to monitor these possible outliers over a few iterations and then decide to remove them.
• Perform random sampling: by choosing a small subset of the data points, the chance of selecting an outlier is much smaller
– Assign the rest of the data points to the clusters by distance or similarity comparison, or classification
![Page 26: 9.54 Class 13 - MIT9.54/fall14/slides/Class13.pdf · What is clustering? •The organization of unlabeled data into similarity groups called clusters. •A cluster is a collection](https://reader034.vdocuments.us/reader034/viewer/2022051601/5ad0c47d7f8b9ad24f8e0d65/html5/thumbnails/26.jpg)
Sensitivity to initial seeds
Random selection of seeds (centroids)
Iteration 1 Iteration 2
Random selection of seeds (centroids)
Iteration 1 Iteration 2
![Page 27: 9.54 Class 13 - MIT9.54/fall14/slides/Class13.pdf · What is clustering? •The organization of unlabeled data into similarity groups called clusters. •A cluster is a collection](https://reader034.vdocuments.us/reader034/viewer/2022051601/5ad0c47d7f8b9ad24f8e0d65/html5/thumbnails/27.jpg)
Special data structures
• The k-means algorithm is not suitable for discovering clusters that are not hyper-ellipsoids (or hyper-spheres).
![Page 28: 9.54 Class 13 - MIT9.54/fall14/slides/Class13.pdf · What is clustering? •The organization of unlabeled data into similarity groups called clusters. •A cluster is a collection](https://reader034.vdocuments.us/reader034/viewer/2022051601/5ad0c47d7f8b9ad24f8e0d65/html5/thumbnails/28.jpg)
K-means summary
• Despite weaknesses, k-means is still the most popular algorithm due to its simplicity and efficiency
• No clear evidence that any other clustering algorithm performs better in general
• Comparing different clustering algorithms is a difficult task. No one knows the correct clusters!
![Page 29: 9.54 Class 13 - MIT9.54/fall14/slides/Class13.pdf · What is clustering? •The organization of unlabeled data into similarity groups called clusters. •A cluster is a collection](https://reader034.vdocuments.us/reader034/viewer/2022051601/5ad0c47d7f8b9ad24f8e0d65/html5/thumbnails/29.jpg)
Application to visual object recognition: Dictionary learning (Bag of Words)
![Page 30: 9.54 Class 13 - MIT9.54/fall14/slides/Class13.pdf · What is clustering? •The organization of unlabeled data into similarity groups called clusters. •A cluster is a collection](https://reader034.vdocuments.us/reader034/viewer/2022051601/5ad0c47d7f8b9ad24f8e0d65/html5/thumbnails/30.jpg)
Learning the visual vocabulary
![Page 31: 9.54 Class 13 - MIT9.54/fall14/slides/Class13.pdf · What is clustering? •The organization of unlabeled data into similarity groups called clusters. •A cluster is a collection](https://reader034.vdocuments.us/reader034/viewer/2022051601/5ad0c47d7f8b9ad24f8e0d65/html5/thumbnails/31.jpg)
Learning the visual vocabulary
![Page 32: 9.54 Class 13 - MIT9.54/fall14/slides/Class13.pdf · What is clustering? •The organization of unlabeled data into similarity groups called clusters. •A cluster is a collection](https://reader034.vdocuments.us/reader034/viewer/2022051601/5ad0c47d7f8b9ad24f8e0d65/html5/thumbnails/32.jpg)
Examples of visual words
![Page 33: 9.54 Class 13 - MIT9.54/fall14/slides/Class13.pdf · What is clustering? •The organization of unlabeled data into similarity groups called clusters. •A cluster is a collection](https://reader034.vdocuments.us/reader034/viewer/2022051601/5ad0c47d7f8b9ad24f8e0d65/html5/thumbnails/33.jpg)
Clustering techniques
Divisive
![Page 34: 9.54 Class 13 - MIT9.54/fall14/slides/Class13.pdf · What is clustering? •The organization of unlabeled data into similarity groups called clusters. •A cluster is a collection](https://reader034.vdocuments.us/reader034/viewer/2022051601/5ad0c47d7f8b9ad24f8e0d65/html5/thumbnails/34.jpg)
Hierarchical clustering
![Page 35: 9.54 Class 13 - MIT9.54/fall14/slides/Class13.pdf · What is clustering? •The organization of unlabeled data into similarity groups called clusters. •A cluster is a collection](https://reader034.vdocuments.us/reader034/viewer/2022051601/5ad0c47d7f8b9ad24f8e0d65/html5/thumbnails/35.jpg)
Example: biological taxonomy
![Page 36: 9.54 Class 13 - MIT9.54/fall14/slides/Class13.pdf · What is clustering? •The organization of unlabeled data into similarity groups called clusters. •A cluster is a collection](https://reader034.vdocuments.us/reader034/viewer/2022051601/5ad0c47d7f8b9ad24f8e0d65/html5/thumbnails/36.jpg)
A Dendrogram
![Page 37: 9.54 Class 13 - MIT9.54/fall14/slides/Class13.pdf · What is clustering? •The organization of unlabeled data into similarity groups called clusters. •A cluster is a collection](https://reader034.vdocuments.us/reader034/viewer/2022051601/5ad0c47d7f8b9ad24f8e0d65/html5/thumbnails/37.jpg)
Types of hierarchical clustering
• Divisive (top down) clusteringStarts with all data points in one cluster, the root, then
– Splits the root into a set of child clusters. Each child cluster is recursively divided further
– stops when only singleton clusters of individual data points remain, i.e., each cluster with only a single point
• Agglomerative (bottom up) clusteringThe dendrogram is built from the bottom level by
– merging the most similar (or nearest) pair of clusters
– stopping when all the data points are merged into a single cluster (i.e., the root cluster).
![Page 38: 9.54 Class 13 - MIT9.54/fall14/slides/Class13.pdf · What is clustering? •The organization of unlabeled data into similarity groups called clusters. •A cluster is a collection](https://reader034.vdocuments.us/reader034/viewer/2022051601/5ad0c47d7f8b9ad24f8e0d65/html5/thumbnails/38.jpg)
Divisive hierarchical clustering
![Page 39: 9.54 Class 13 - MIT9.54/fall14/slides/Class13.pdf · What is clustering? •The organization of unlabeled data into similarity groups called clusters. •A cluster is a collection](https://reader034.vdocuments.us/reader034/viewer/2022051601/5ad0c47d7f8b9ad24f8e0d65/html5/thumbnails/39.jpg)
Agglomerative hierarchical clustering
![Page 40: 9.54 Class 13 - MIT9.54/fall14/slides/Class13.pdf · What is clustering? •The organization of unlabeled data into similarity groups called clusters. •A cluster is a collection](https://reader034.vdocuments.us/reader034/viewer/2022051601/5ad0c47d7f8b9ad24f8e0d65/html5/thumbnails/40.jpg)
Single linkage or Nearest neighbor
![Page 41: 9.54 Class 13 - MIT9.54/fall14/slides/Class13.pdf · What is clustering? •The organization of unlabeled data into similarity groups called clusters. •A cluster is a collection](https://reader034.vdocuments.us/reader034/viewer/2022051601/5ad0c47d7f8b9ad24f8e0d65/html5/thumbnails/41.jpg)
Complete linkage or Farthest neighbor
![Page 42: 9.54 Class 13 - MIT9.54/fall14/slides/Class13.pdf · What is clustering? •The organization of unlabeled data into similarity groups called clusters. •A cluster is a collection](https://reader034.vdocuments.us/reader034/viewer/2022051601/5ad0c47d7f8b9ad24f8e0d65/html5/thumbnails/42.jpg)
Divisive vs. Agglomerative
![Page 43: 9.54 Class 13 - MIT9.54/fall14/slides/Class13.pdf · What is clustering? •The organization of unlabeled data into similarity groups called clusters. •A cluster is a collection](https://reader034.vdocuments.us/reader034/viewer/2022051601/5ad0c47d7f8b9ad24f8e0d65/html5/thumbnails/43.jpg)
Object category structure in monkey inferior temporal (IT) cortex
![Page 44: 9.54 Class 13 - MIT9.54/fall14/slides/Class13.pdf · What is clustering? •The organization of unlabeled data into similarity groups called clusters. •A cluster is a collection](https://reader034.vdocuments.us/reader034/viewer/2022051601/5ad0c47d7f8b9ad24f8e0d65/html5/thumbnails/44.jpg)
Kiani et al., 2007
Object category structure in monkey inferior temporal (IT) cortex
![Page 45: 9.54 Class 13 - MIT9.54/fall14/slides/Class13.pdf · What is clustering? •The organization of unlabeled data into similarity groups called clusters. •A cluster is a collection](https://reader034.vdocuments.us/reader034/viewer/2022051601/5ad0c47d7f8b9ad24f8e0d65/html5/thumbnails/45.jpg)
Hierarchical clustering of neuronal response patterns in monkey IT cortex
Kiani et al., 2007
![Page 46: 9.54 Class 13 - MIT9.54/fall14/slides/Class13.pdf · What is clustering? •The organization of unlabeled data into similarity groups called clusters. •A cluster is a collection](https://reader034.vdocuments.us/reader034/viewer/2022051601/5ad0c47d7f8b9ad24f8e0d65/html5/thumbnails/46.jpg)
Competitive learning
![Page 47: 9.54 Class 13 - MIT9.54/fall14/slides/Class13.pdf · What is clustering? •The organization of unlabeled data into similarity groups called clusters. •A cluster is a collection](https://reader034.vdocuments.us/reader034/viewer/2022051601/5ad0c47d7f8b9ad24f8e0d65/html5/thumbnails/47.jpg)
Competitive learning algorithm:Kohonen Self Organization Maps (K-SOM)
![Page 48: 9.54 Class 13 - MIT9.54/fall14/slides/Class13.pdf · What is clustering? •The organization of unlabeled data into similarity groups called clusters. •A cluster is a collection](https://reader034.vdocuments.us/reader034/viewer/2022051601/5ad0c47d7f8b9ad24f8e0d65/html5/thumbnails/48.jpg)
K-SOM example
• Four input data points (crosses) in 2D space.
• Four output nodes in a discrete 1D output space (mapped to 2D as circles).
• Random initial weights start the output nodes at random positions.
![Page 49: 9.54 Class 13 - MIT9.54/fall14/slides/Class13.pdf · What is clustering? •The organization of unlabeled data into similarity groups called clusters. •A cluster is a collection](https://reader034.vdocuments.us/reader034/viewer/2022051601/5ad0c47d7f8b9ad24f8e0d65/html5/thumbnails/49.jpg)
• Randomly pick one input data point for training (cross in circle).
• The closest output node is the winning neuron (solid diamond).
• This winning neuron is moved towards the input data point, while its two neighbors move also by a smaller increment (arrows).
K-SOM example
![Page 50: 9.54 Class 13 - MIT9.54/fall14/slides/Class13.pdf · What is clustering? •The organization of unlabeled data into similarity groups called clusters. •A cluster is a collection](https://reader034.vdocuments.us/reader034/viewer/2022051601/5ad0c47d7f8b9ad24f8e0d65/html5/thumbnails/50.jpg)
• Randomly pick another input data point for training (cross in circle).
• The closest output node is the new winning neuron (solid diamond).
• This winning neuron is moved towards the input data point, while its single neighboring neuron move also by a smaller increment (arrows).
K-SOM example
![Page 51: 9.54 Class 13 - MIT9.54/fall14/slides/Class13.pdf · What is clustering? •The organization of unlabeled data into similarity groups called clusters. •A cluster is a collection](https://reader034.vdocuments.us/reader034/viewer/2022051601/5ad0c47d7f8b9ad24f8e0d65/html5/thumbnails/51.jpg)
K-SOM example
• Continue to randomly pick data points for training, and move the winning neuron and its neighbors (by a smaller increment) towards the training data points.
• Eventually, the whole output grid unravels itself to represent the input space.
![Page 52: 9.54 Class 13 - MIT9.54/fall14/slides/Class13.pdf · What is clustering? •The organization of unlabeled data into similarity groups called clusters. •A cluster is a collection](https://reader034.vdocuments.us/reader034/viewer/2022051601/5ad0c47d7f8b9ad24f8e0d65/html5/thumbnails/52.jpg)
Competitive learning claimed effect
![Page 53: 9.54 Class 13 - MIT9.54/fall14/slides/Class13.pdf · What is clustering? •The organization of unlabeled data into similarity groups called clusters. •A cluster is a collection](https://reader034.vdocuments.us/reader034/viewer/2022051601/5ad0c47d7f8b9ad24f8e0d65/html5/thumbnails/53.jpg)
Hebbian vs. Competitive learning
![Page 54: 9.54 Class 13 - MIT9.54/fall14/slides/Class13.pdf · What is clustering? •The organization of unlabeled data into similarity groups called clusters. •A cluster is a collection](https://reader034.vdocuments.us/reader034/viewer/2022051601/5ad0c47d7f8b9ad24f8e0d65/html5/thumbnails/54.jpg)
Summary
• Clustering has a long history and still is in active research– There are a huge number of clustering algorithms, among them:
Density based algorithm, Sub-space clustering, Scale-up methods, Neural networks based methods, Fuzzy clustering, Co-clustering …
– More are still coming every year
• Clustering is hard to evaluate, but very useful in practice• Clustering is highly application dependent (and to some
extent subjective)• Competitive learning in neuronal networks performs
clustering analysis of the input data