On Mechanism in Clustering

Speaker: Caiming Zhong

04-02-2010

2

Outline

Some main components of a clustering algorithm

A mechanism: Adaptive (Autonomous) scheme, or framework K-Means: single prototype for one cluster Affinity Propagation Multi-prototype based autonomy

Potential topics

3

Main components of a clustering algorithm Distance metric (Similarity measure) Objective function Clustering scheme

4

Main components of a clustering algorithm (cont.) Distance metric (Similarity measure)

Cornerstone for a clustering algorithm. Euclidean distance is the most used, but doesn’t

work some time.

5

Euclidean

vs.

Geodesic

6

Main components of a clustering algorithm (cont.) A similarity measure is not always a metric Conventional similarity measures

7

8

9

10

11

Main components of a clustering algorithm (cont.) Special similarity measures

Point symmetry distance

12

13

14

15

16

17

Main components of a clustering algorithm (cont.) Special similarity measures

Path-based distance (minmax diatance)

18

19

20

21

Densities

22

Main components of a clustering algorithm (cont.) Objective Function

What objective function to be optimized? K-Means: MSE, compactness Path-based: connectivity Point symmetry: Symmetry

23

Main components of a clustering algorithm (cont.) Clustering framework

Split-and-merge Agglomerative Divisive Partitioning Density connectivity …

24

A mechanism: Autonomous framework Generally a clustering process of clustering

scheme stops when a certain criterion is satisfied. The criterion is usually user-specifieduser-specified parameters.

The number of clusters The number of iterations

If the criterion is not a specific threshold, but convergence (the stable state is achieved), we can say “Autonomous frameworkAutonomous framework”

25

A mechanism: Autonomous framework (cont.) K-Means is a typical autonomous framework

Repeatedly move prototypes (representative points of a cluster), until no prototype changed

Affinity propagation

26

27

28

A mechanism: Autonomous framework (cont.)

A multi-prototype clustering algorithm

29

30

31

32

33

34

35

36

37

38

39

Potential topics

Apply existing mechanisms onto Graph (K-MST Graph) , in breeding.

Improve the existing mechanisms. Exploit new mechanism.

40

References

R. XU, D. WUNSCH, Survey of clustering algorithms. IEEE Transactions on Neural Networks, 2005.

M. Su, C. Chou, A modified version of the K-means algorithm with a distance based on cluster symmetry, IEEE Transactions on PAMI, 2001.

S, Bandyopadhyay, S. Saha, GAPS: A clustering method using a new point symmetry-based distance measure, Pattern Recognition, 2007.

B. Fischer, J. Buhmann, Path-Based Clustering for Grouping of Smooth Curves and Texture Segmentation, IEEE Transactions PAMI, 2003.

41

References (cont.)

H. Chang, D. Yeung, Robust path-based spectral clustering, Pattern recognition, 2008.

B. Frey, D. Dueck, Clustering by passing messages between data points, Science, 2007.

M. Liu, X. Jiang, AC. Kot, A multi-prototype clustering algorithm, Pattern Recognition, 2009.

42

Thanks!