Upload File

part ii - clustering© prentice hall1 clustering large db most clustering algorithms assume a large...

  • Home
  • Documents
  • Part II - Clustering© Prentice Hall1 Clustering Large DB Most clustering algorithms assume a large data structure which is memory resident. Most clustering
14
© Prentice Hall 1 Part II - Clustering Clustering Large DB Clustering Large DB Most clustering algorithms assume a Most clustering algorithms assume a large data structure which is large data structure which is memory resident. memory resident. Clustering may be performed first Clustering may be performed first on a sample of the database then on a sample of the database then applied to the entire database. applied to the entire database. Algorithms Algorithms – BIRCH BIRCH – DBSCAN DBSCAN – CURE CURE

Upload: clinton-parsons

Post on 16-Dec-2015

219 views

Category:

Documents


2 download

Report

TRANSCRIPT

Page 1: Part II - Clustering© Prentice Hall1 Clustering Large DB Most clustering algorithms assume a large data structure which is memory resident. Most clustering

© Prentice Hall 1Part II - Clustering

Clustering Large DBClustering Large DB

Most clustering algorithms assume a large Most clustering algorithms assume a large data structure which is memory resident.data structure which is memory resident.

Clustering may be performed first on a Clustering may be performed first on a sample of the database then applied to the sample of the database then applied to the entire database.entire database.

AlgorithmsAlgorithms– BIRCHBIRCH– DBSCANDBSCAN– CURECURE

Page 2: Part II - Clustering© Prentice Hall1 Clustering Large DB Most clustering algorithms assume a large data structure which is memory resident. Most clustering

© Prentice Hall 2Part II - Clustering

Desired Features for Large Desired Features for Large DatabasesDatabases

One scan (or less) of DBOne scan (or less) of DB OnlineOnline Suspendable, stoppable, resumableSuspendable, stoppable, resumable IncrementalIncremental Work with limited main memoryWork with limited main memory Different techniques to scan (e.g. Different techniques to scan (e.g.

sampling)sampling) Process each tuple onceProcess each tuple once

Page 3: Part II - Clustering© Prentice Hall1 Clustering Large DB Most clustering algorithms assume a large data structure which is memory resident. Most clustering

© Prentice Hall 3Part II - Clustering

BIRCHBIRCH Balanced Iterative Reducing and Balanced Iterative Reducing and

Clustering using HierarchiesClustering using Hierarchies Incremental, hierarchical, one scanIncremental, hierarchical, one scan Save clustering information in a tree Save clustering information in a tree Each entry in the tree contains Each entry in the tree contains

information about one clusterinformation about one cluster New nodes inserted in closest entry in New nodes inserted in closest entry in

treetree

Page 4: Part II - Clustering© Prentice Hall1 Clustering Large DB Most clustering algorithms assume a large data structure which is memory resident. Most clustering

© Prentice Hall 4Part II - Clustering

Clustering FeatureClustering Feature (N,LS,SS)(N,LS,SS)

– N: Number of points in clusterN: Number of points in cluster– LS: Sum of points in the clusterLS: Sum of points in the cluster– SS: Sum of squares of points in the clusterSS: Sum of squares of points in the cluster

CF TreeCF Tree– Balanced search treeBalanced search tree– Node has CF triple for each childNode has CF triple for each child– Leaf node represents cluster and has CF Leaf node represents cluster and has CF

value for each subcluster in it.value for each subcluster in it.– Subcluster has maximum diameterSubcluster has maximum diameter

Page 5: Part II - Clustering© Prentice Hall1 Clustering Large DB Most clustering algorithms assume a large data structure which is memory resident. Most clustering

© Prentice Hall 5Part II - Clustering

BIRCH AlgorithmBIRCH Algorithm

Page 6: Part II - Clustering© Prentice Hall1 Clustering Large DB Most clustering algorithms assume a large data structure which is memory resident. Most clustering

© Prentice Hall 6Part II - Clustering

Improve ClustersImprove Clusters

Page 7: Part II - Clustering© Prentice Hall1 Clustering Large DB Most clustering algorithms assume a large data structure which is memory resident. Most clustering

© Prentice Hall 7Part II - Clustering

DBSCANDBSCAN

Density Based Spatial Clustering of Density Based Spatial Clustering of Applications with NoiseApplications with Noise

Outliers will not effect creation of cluster.Outliers will not effect creation of cluster. InputInput

– MinPts MinPts – minimum number of points in – minimum number of points in clustercluster

– EpsEps – for each point in cluster there must – for each point in cluster there must be another point in it less than this distance be another point in it less than this distance away.away.

Page 8: Part II - Clustering© Prentice Hall1 Clustering Large DB Most clustering algorithms assume a large data structure which is memory resident. Most clustering

© Prentice Hall 8Part II - Clustering

DBSCAN Density ConceptsDBSCAN Density Concepts

Eps-neighborhood:Eps-neighborhood: Points within Eps Points within Eps distance of a point.distance of a point.

Core point:Core point: Eps-neighborhood dense enough Eps-neighborhood dense enough (MinPts)(MinPts)

Directly density-reachable:Directly density-reachable: A point p is A point p is directly density-reachable from a point q if the directly density-reachable from a point q if the distance is small (Eps) and q is a core point.distance is small (Eps) and q is a core point.

Density-reachable:Density-reachable: A point si density- A point si density-reachable form another point if there is a path reachable form another point if there is a path from one to the other consisting of only core from one to the other consisting of only core points.points.

Page 9: Part II - Clustering© Prentice Hall1 Clustering Large DB Most clustering algorithms assume a large data structure which is memory resident. Most clustering

© Prentice Hall 9Part II - Clustering

Density ConceptsDensity Concepts

Page 10: Part II - Clustering© Prentice Hall1 Clustering Large DB Most clustering algorithms assume a large data structure which is memory resident. Most clustering

© Prentice Hall 10Part II - Clustering

DBSCAN AlgorithmDBSCAN Algorithm

Page 11: Part II - Clustering© Prentice Hall1 Clustering Large DB Most clustering algorithms assume a large data structure which is memory resident. Most clustering

© Prentice Hall 11Part II - Clustering

CURECURE

Clustering Using RepresentativesClustering Using Representatives Use many points to represent a cluster Use many points to represent a cluster

instead of only oneinstead of only one Points will be well scatteredPoints will be well scattered

Page 12: Part II - Clustering© Prentice Hall1 Clustering Large DB Most clustering algorithms assume a large data structure which is memory resident. Most clustering

© Prentice Hall 12Part II - Clustering

CURE ApproachCURE Approach

Page 13: Part II - Clustering© Prentice Hall1 Clustering Large DB Most clustering algorithms assume a large data structure which is memory resident. Most clustering

© Prentice Hall 13Part II - Clustering

CURE AlgorithmCURE Algorithm

Page 14: Part II - Clustering© Prentice Hall1 Clustering Large DB Most clustering algorithms assume a large data structure which is memory resident. Most clustering

© Prentice Hall 14Part II - Clustering

CURE for Large DatabasesCURE for Large Databases


Clustering of Large Designs for Channel-Width Constrained FPGAs

Parallel Clustering for Visualizing Large Scientific Line Datavis.cs.ucdavis.edu/papers/parallel-clustering-scientific-line-data.pdf · Parallel Clustering for Visualizing Large Scientific

Algorithms for clustering large datasets in arbitrary metric spaces

Diffusion and Clustering on Large Graphscseweb.ucsd.edu/~atsiatas/defense_slides.pdf · Diffusion and Clustering on Large Graphs Alexander Tsiatas Final Defense 17 May 2012 . Introduction

A New Density Based Clustering for Large Spatial …perrizo/saturday/ExternalDriv… · Web viewA distribution-based clustering algorithm for mining in large spatial databases. In

Task Clustering Problem with Large Communication Delays

Efficient Evidence Accumulation Clustering for large ... · Efficient Evidence Accumulation Clustering for large datasets/big data ... Palavras-chave: Metodos de agrupamento, EAC,

Clustering - Visualization · Hierarchical clustering for large datasets? • OK for small datasets (e.g.,

CURE: An Efficient Clustering Algorithm for Large Databaseshan/dmclass/cure.pdf · CURE: An Efficient Clustering Algorithm for Large Databases ... the CF-tree. For each ... traversed

Large-scale Single-pass k-Means Clustering at Scale

1. Introduction - University of Calgary in Alberta · 2014-10-17 · Keywords: Spatial Clustering; Constraint-based Clustering; Obstacle. 1. Introduction Clustering large amounts

HALL4 HALL2 HALL1

Diffusion and Clustering on Large Graphscseweb.ucsd.edu/~atsiatas/advancement_slides.pdf · Diffusion and Clustering on Large Graphs Alexander Tsiatas Thesis Proposal / Advancement

Parallel Multivariate Spatio-Temporal Clustering of Large ...sarat/files/Sreepathi_IEEECluster...Parallel Multivariate Spatio-Temporal Clustering of Large Ecological Datasets on Hybrid

Clustering Very Large Multi-dimensional Datasets with MapReduce

Clustering. Introduction Clustering Summarization of large data –Understand the large customer data Data organization –Manage the large customer data

Strategies and Algorithms for Clustering Large Datasets: A ... · Strategies and Algorithms for Clustering Large Datasets: A Review Javier Béjar ... Classical methods for clustering

Iterative Clustering for Energy-Efficient Large-Scale

Quantum Clustering of Large Data Setshorn.tau.ac.il/publications/Thesis_GS.pdf · clustering method. On financial data, our method shows clustering of stocks to different groups,

Network Lasso: Clustering and Optimization in Large Graphsstanford.edu/~boyd/papers/pdf/network_lasso.pdf · Network Lasso: Clustering and Optimization in Large Graphs David Hallac,

CURE: An Efficient Clustering Algorithm for Large Databaseshan/dmclass/cure.pdf · CURE: An Efficient Clustering Algorithm for Large Databases Sudipto Guha* Rajeev Rastogi Kyuseok

Smart-Sample: An E cient Algorithm for Clustering Large

Genetic algorithms for large scale clustering problemscs.uef.fi/sipu/pub/GA-CompJ1997.pdf · 2003. 9. 15. · Genetic algorithms for large scale clustering problems (published in

Scalable Inference Algorithms for Clustering Large Networks

Penalized Clustering of Large-Scale Functional Data …publish.illinois.edu/wenxuan/files/2012/12/ssclust_jasa.pdf · Penalized Clustering of Large-Scale Functional Data With Multiple

Incremental Sub-Trajectory Clustering of Large Moving Object Databases

Distributed approximate spectral clustering for large scale datasets

Scalable and Memory-Efficient Clustering of Large-Scale Social …bigdata.cs.skku.edu/down/ICDM2012_joyce_gem.pdf · 2019-03-07 · Scalable and Memory-E cient Clustering of Large-Scale

Macros for Clustering Large Data Sets when the … for Clustering Large... · Introduction MACROS FOR DETERMINING THE OPTIMAL CLUSTERS IN LARGE DATA SETS B. GERARDIN J.L. MOLLIERE

Scaling EM (Expectation-Maximization) Clustering to Large ...€¦ · Scaling EM Clustering to Large Databases Bradley, Fayyad, and Reina 2 A general view of clustering places it

Energy Balanced Clustering Algorithm in Large Scope WSN

MIXTURE MODEL CLUSTERING FOR VERY LARGE …studentsrepo.um.edu.my/6108/1/Mixture_Model_Clustering_for_Very... · MIXTURE MODEL CLUSTERING FOR VERY LARGE DATA ... 2010 . MIXTURE MODEL

Distributed approximate spectral clustering for large ...cobweb.cs.uga.edu/~squinn/mmd_s15/papers/p223-hefeeda.pdf · Distributed Approximate Spectral Clustering for Large-Scale Datasets

IJCAI13 Paper review: Large-scale spectral clustering on graphs

Clustering: Large Databases in data mining