Upload File

data mining and statistical learning - lecture 14 clustering methods partitional clustering in...

  • Home
  • Documents
  • Data mining and statistical learning - lecture 14 Clustering methods Partitional clustering in which clusters are represented by their centroids (proc
23
Data mining and statistic al learning - lecture 14 Clustering methods Partitional clustering in which clusters are represented by their centroids (proc FASTCLUS) Agglomerative hierarchical clustering in which the closest clusters are repeatedly merged (proc CLUSTER) Density-based clustering in which core points and associated border points are clustered (proc MODECLUS)

Post on 21-Dec-2015

218 views

Category:

Documents


0 download

Report

TRANSCRIPT

Page 1: Data mining and statistical learning - lecture 14 Clustering methods Partitional clustering in which clusters are represented by their centroids (proc

Data mining and statistical learning - lecture 14

Clustering methods

Partitional clustering in which clusters are represented by their centroids (proc FASTCLUS)

Agglomerative hierarchical clustering in which the closest clusters are repeatedly merged (proc CLUSTER)

Density-based clustering in which core points and associated border points are clustered (proc MODECLUS)

Page 2: Data mining and statistical learning - lecture 14 Clustering methods Partitional clustering in which clusters are represented by their centroids (proc

Data mining and statistical learning - lecture 14

Proc FASTCLUS

Select k initial centroids

Repeat the following until the clusters remain unchanged:

Form k clusters by assigning each point to its nearest centroid

Update the centroid of each cluster

Page 3: Data mining and statistical learning - lecture 14 Clustering methods Partitional clustering in which clusters are represented by their centroids (proc

Data mining and statistical learning - lecture 14

Identification of water samples with incorrect

total nitrogen levels

0

5000

10000

15000

20000

25000

0 5000 10000 15000 20000 25000 30000

Total nitrogen (persulfate) mg/l

To

tal

nit

rog

en (

Kje

ldah

l) m g

/l

Page 4: Data mining and statistical learning - lecture 14 Clustering methods Partitional clustering in which clusters are represented by their centroids (proc

Data mining and statistical learning - lecture 14

Identification of water samples with incorrect total nitrogen levels

- 2-means clustering

0

5000

10000

15000

20000

25000

0 5000 10000 15000 20000 25000 30000

Total nitrogen (persulfate digestion)

To

tal

nit

rog

en (

Kje

ldah

l)

Cluster 1 Cluster 2

Initializationproblems?

Page 5: Data mining and statistical learning - lecture 14 Clustering methods Partitional clustering in which clusters are represented by their centroids (proc

Data mining and statistical learning - lecture 14

Limitations of K-means clustering

1. Difficult to detect clusters with non-spherical shapes

2. Difficult to detect clusters of widely different sizes

3. Difficult to detect clusters of different densities

Page 6: Data mining and statistical learning - lecture 14 Clustering methods Partitional clustering in which clusters are represented by their centroids (proc

Data mining and statistical learning - lecture 14

Proc MODECLUS

Use a smoother to estimate the (local) density of the given dataset

A cluster is loosely defined as a region surrounding a local maximum of the probability density function

Page 7: Data mining and statistical learning - lecture 14 Clustering methods Partitional clustering in which clusters are represented by their centroids (proc

Data mining and statistical learning - lecture 14

Identification of water samples with incorrect

total nitrogen levels

- proc MODECLUS, R = 1000

Smoothing parameter R = 1000

0

5000

10000

15000

20000

25000

0 10000 20000 30000

Tot_N (Kj) mg/l

To

t_N

(p

s)

mg

/l Cluster 1

Cluster 2

Cluster 3

Cluster 4

Cluster 5

Other clusters

What will happen if R is increased?

Page 8: Data mining and statistical learning - lecture 14 Clustering methods Partitional clustering in which clusters are represented by their centroids (proc

Data mining and statistical learning - lecture 14

Identification of water samples with incorrect

total nitrogen levels

- proc MODECLUS, R = 4000

Smoothing parameter R = 4000

0

5000

10000

15000

20000

25000

0 10000 20000 30000

Tot_N (Kj) mg/l

To

t_N

(p

s)

mg

/l

Cluster 1

Cluster 2

Page 9: Data mining and statistical learning - lecture 14 Clustering methods Partitional clustering in which clusters are represented by their centroids (proc

Data mining and statistical learning - lecture 14

Identification of water samples with incorrect

total nitrogen levels

- proc MODECLUS, method 6

0

5000

10000

15000

20000

25000

0 5000 10000 15000 20000 25000 30000

Total nítrogen (persulfate digestion)

To

tal n

itro

ge

n (

Kje

lda

hl) Cluster 1

Cluster 2

Cluster 3

Cluster 4

Cluster 5

Clusters 6 - 18

No cluster assigned

Why did the clustering fail?

Page 10: Data mining and statistical learning - lecture 14 Clustering methods Partitional clustering in which clusters are represented by their centroids (proc

Data mining and statistical learning - lecture 14

Limitations of density-based clustering

1. Difficult to control (requires repeated runs)

2. Collapses in high dimensions

Page 11: Data mining and statistical learning - lecture 14 Clustering methods Partitional clustering in which clusters are represented by their centroids (proc

Data mining and statistical learning - lecture 14

Strength of density-based clustering

Given a sufficiently large sample, nonparametric density-based clustering methods are capable of detecting clusters of unequal size and dispersion and with highly irregular shapes

Page 12: Data mining and statistical learning - lecture 14 Clustering methods Partitional clustering in which clusters are represented by their centroids (proc

Data mining and statistical learning - lecture 14

Identification of water samples with incorrect

total nitrogen levels

- transformed data

-10000

-5000

0

5000

10000

15000

0 5000 10000 15000 20000 25000

Total N (Kj)

To

tal

N (

ps)

-T

ota

l N

(K

j)

Page 13: Data mining and statistical learning - lecture 14 Clustering methods Partitional clustering in which clusters are represented by their centroids (proc

Data mining and statistical learning - lecture 14

Identification of water samples with incorrect

total nitrogen levels

- proc MODECLUS, R = 2000, transformed data

-10000

-5000

0

5000

10000

15000

0 5000 10000 15000 20000 25000

Total N (Kj)

To

tal

N (

ps)

-T

ota

l N

(K

j)

Cluster 1

Cluster 2

Cluster 3-6

Page 14: Data mining and statistical learning - lecture 14 Clustering methods Partitional clustering in which clusters are represented by their centroids (proc

Data mining and statistical learning - lecture 14

Preprocessing

1. Standardization

2. Linear transformation

3. Dimension reduction

Page 15: Data mining and statistical learning - lecture 14 Clustering methods Partitional clustering in which clusters are represented by their centroids (proc

Data mining and statistical learning - lecture 14

Postprocessing

1. Split a cluster• Usually, the cluster with the largest SSE is split

2. Introduce a new cluster centroid• Often the point that is farthest from any cluster center is

chosen

3. Disperse a cluster• Remove one centroid and reassign the points to other

clusters

4. Merge two clusters• Typically, the clusters with the closest centroids are chosen

Page 16: Data mining and statistical learning - lecture 14 Clustering methods Partitional clustering in which clusters are represented by their centroids (proc

Data mining and statistical learning - lecture 14

Profiling website visitors

1. A total of 296 pages at a Microsoft website are grouped into 13 homogenous categories• Initial• Support• Entertainment• Office• Windows• Othersoft• Download• …..

2. For each of 32711 visitors we have recorded how many times they have visited the different categories of pages

3. We would like to make a behavioural segmentation of the users ( a cluster analysis) that can be used in future marketing decisions

Page 17: Data mining and statistical learning - lecture 14 Clustering methods Partitional clustering in which clusters are represented by their centroids (proc

Data mining and statistical learning - lecture 14

Profiling website visitors

- the dataset

client_codeinitial help entertainmentoffice windows othersft download otherint developmenthardware business informationarea10001 1 1 1 0 0 0 0 0 0 0 0 0 010002 1 1 0 0 0 0 0 0 0 0 0 0 010003 2 1 0 0 0 0 0 0 0 0 0 0 010004 0 0 0 0 0 0 0 0 0 0 0 0 110005 0 0 0 0 0 0 0 1 0 0 0 0 010006 2 0 0 0 0 0 0 0 0 0 0 0 010007 0 0 0 1 0 0 0 0 0 0 0 0 010008 1 0 0 0 0 0 0 0 0 0 0 0 010009 0 0 0 0 1 0 1 0 0 0 0 0 010010 1 1 0 1 0 1 0 0 2 0 0 0 010011 2 0 0 3 0 0 0 0 0 0 0 0 010012 0 0 0 0 0 1 0 0 1 0 0 0 010013 0 0 0 0 0 0 1 0 0 0 0 0 010014 0 0 0 0 0 0 0 0 0 0 0 0 110015 0 0 0 0 0 0 0 0 0 1 0 0 010016 0 0 0 0 0 0 0 1 1 0 0 0 010017 1 0 0 0 0 0 0 0 3 0 0 0 010018 1 0 0 0 0 0 0 0 0 0 0 0 010019 4 0 2 1 0 0 2 0 0 1 1 0 010020 0 1 1 1 0 0 1 0 0 0 0 0 010021 3 1 1 1 2 1 1 3 2 0 1 1 0

Why is it necessary to group the pages into categories?

Page 18: Data mining and statistical learning - lecture 14 Clustering methods Partitional clustering in which clusters are represented by their centroids (proc

Data mining and statistical learning - lecture 14

Profiling website visitors

- 10-means clustering

Page 19: Data mining and statistical learning - lecture 14 Clustering methods Partitional clustering in which clusters are represented by their centroids (proc

Data mining and statistical learning - lecture 14

Profiling website visitors

- cluster proximities

Page 20: Data mining and statistical learning - lecture 14 Clustering methods Partitional clustering in which clusters are represented by their centroids (proc

Data mining and statistical learning - lecture 14

Profiling website visitors

- profiles

Page 21: Data mining and statistical learning - lecture 14 Clustering methods Partitional clustering in which clusters are represented by their centroids (proc

Data mining and statistical learning - lecture 14

Profiling website visitors

- Kohonen Map of cluster frequencies

Page 22: Data mining and statistical learning - lecture 14 Clustering methods Partitional clustering in which clusters are represented by their centroids (proc

Data mining and statistical learning - lecture 14

Profiling website visitors

- Kohonen Maps of means by variable and grid cell

Page 23: Data mining and statistical learning - lecture 14 Clustering methods Partitional clustering in which clusters are represented by their centroids (proc

Data mining and statistical learning - lecture 14

Characteristics of Kohonen maps

The centroids vary smoothly over the map• The set of clusters having unusually large (or small) values

of a given variable tend to form connected spatial patterns

Clusters with similar centroids need not be close to each other in a Kohonen map

The sizes of the clusters in Kohonen maps tend to be less variable than those obtained by K-means clustering


Fuzzy Density Based Clustering with Generalized Centroidsvigir.missouri.edu/~gdesouza/Research/Conference... · clustering around the centroids is. Higher values for !produce crisp

CSE601 Mixture Model Clustering - University at Buffalojing/cse601/fa12/materials/... · 2012. 10. 26. · Outline •Basics –Motivation, definition, evaluation •Methods –Partitional

Spring 2003Data Mining by H. Liu, ASU1 4. Clustering Methods Concepts Partitional (k-Means, k-Medoids) Hierarchical (Agglomerative & Divisive, COBWEB)

Centroids & Moments of Inertia of Beam Sectionsfaculty.arch.tamu.edu/media/cms_page_media/4210/NS8sections_3N6... · Centroids & Moments of Inertia of Beam Sections ... Centroids

Partitional Algorithms to Detect Complex Clusters Kernel K-means K-means applied in Kernel space Spectral clustering Eigen subspace of the affinity matrix

K-means clustering Hongning Wang CS@UVa. Today’s lecture k-means clustering – A typical partitional clustering algorithm – Convergence property Expectation

Uncertain Centroid based Partitional Clustering of Uncertain Data

BPEC: Belief-Peaks Evidential Clustering version.pdf · them, partitional clustering has attracted a lot of attention in artificial intelligence communities. Classical hard partitional

CSE 494/598 Lecture-11: Clustering & Classification › ~lmanikon › CSE494-598 › lectures › ... · K-means Clustering Algorithm •Initial centroids are often chosen randomly

CSE601 Partitional Clustering - University at Buffalojing/cse601/fa12/materials/clustering... · – Sometimes the initial centroids will readjust themselves in right way, and sometimes

K-means Clustering - unipi.itdidawiki.cli.di.unipi.it/.../fetch.php/dm/dm2014_clustering_kmeans.pdf · K-means Clustering Partitional clustering approach Each cluster is associated

Using Metaboanalyst– hands‐on...Empirical Bayesian Analysis of Microarray (and Metabolites) (EBAM) Cluster Analysis Hierarchical Clustering: Dendrogram Heatmaps Partitional Clustering:

Clustering: David Arthur, Sergei Vassilvitskii. k-means ...aris.me/.../data-mining-2016/slides/partitional-clustering.pdf · Fig. 2 Effect of an inferior initialization on the k-means

Relative clustering validity criteria: A comparative overviewhcao/paper/icredits/2010...Clustering techniques can be broadly divided into three main types [7]: overlapping, partitional,

Centroids by Integration

CSE601 Clustering Ensemble - University at Buffalo · 2012-10-17 · Outline • Basics – Motivation, definition, evaluation • Methods – Partitional – Hierarchical – Density-based

Graph Based K-Means Clustering - University of Michiganweb.eecs.umich.edu/~hero/Preprints/GallMCSH_revised_final.pdf · Graph Based K-Means Clustering ... popular partitional clustering

MIT - 9.54 Class 139.54/fall14/slides/Class13.pdfK-Means clustering •K-means (MacQueen, 1967) is a partitional clustering algorithm •Let the set of data points D be {x1, x2, …,

Image segmentation by Clustering - · PDF fileimage using 2 centroids) Figure 4 ... of Clusters in K-Means Clustering and Application in Colour Image Segmentation". [7] ... means-like

Last lecture summary. Cluster analysis unsupervised hierarchical clustering – agglomerative – divisive – dendrogram partitional – k-means MITO BANARMFL

Nanda and Panda 2013 - A Survey on Nature Inspired Metaheuristic Algorithms for Partitional Clustering

Centroids Integration

Text Clustering PengBo Nov 1, 2010. Today’s Topic Document clustering Motivations Clustering algorithms Partitional Hierarchical Evaluation

Today’s Topic: Clustering Document clustering Motivations Document representations Success criteria Clustering algorithms Partitional Hierarchical

A Survey of Partitional and Hierarchical Clustering Algorithmsdmkd.cs.wayne.edu/Papers/kmeans13.pdf · A Survey of Partitional and Hierarchical Clustering Algorithms 89 4.2 Partitional

Example - Centroids

Farthest Centroids Divisive Clustering - CSE User Home …saad/PDF/umsi-2008-05.pdf · Farthest Centroids Divisive Clustering ∗ Haw-ren Fang† Yousef Saad† January 28, 2008 Abstract

PARTITIONAL CLUSTERING

9/03Data Mining – Clustering G Dong (WSU) 1 4. Clustering Methods Concepts Partitional (k-Means, k-Medoids) Hierarchical (Agglomerative & Divisive, COBWEB)

Partitional Values1

A Fuzzy Partitional Clustering algorithm with Adaptive

Clustering fundamentals - polito.it · 17 D B M G K-means Clustering – Details Initial centroids are often chosen randomly. Clusters produced vary from one run to another. The centroid

Uncertain Centroid based Partitional Clustering of ...vldb.org/pvldb/vol5/p610_francescogullo_vldb2012.pdf · Uncertain Centroid based Partitional Clustering of Uncertain Data Francesco

Image Segmentation For CT Image With Artefact · 2016. 9. 9. · 3.1.K-MEANS CLUSTERING K-means clustering is an partitional clustering technique which attempts to directly decompose

Clustering Algorithms for Genetic Analysis with GeneMarkerThere are two types of data clustering: hierarchical and partitional. Partitional clustering includes the K-means and Self-Organizing