ant colony data mining

8/6/2019 Ant Colony Data Mining

1/51

MIMI

LL

MAPLD2005/P249

FPGA Co-Processor Enhanced

Ant Colony Systems Data

MiningJason Isaacs and Simon Y. Foo

Machine Intelligence Laboratory

FAMU-FSU College of Engineering

Department of Electrical and Computer Engineering
http://images.google.com/imgres?imgurl=freud.psy.fsu.edu/~chalcraf/downloads/FSU1851Seal.gif&imgrefurl=http://freud.psy.fsu.edu/~chalcraf/downloads/&h=664&w=664&sz=22&tbnid=8MAt2M0o80sJ:&tbnh=136&tbnw=136&start=9&prev=/images%3Fq%3Dfsu%2Bseal%26hl%3Den%26lr%3D%26ie%3DUTF-8


2/51

Isaacs 2

MIMI

LL

MAPLD2005/P249

Presentation Outline

Introduction

Significance of Research

Concise Background on ACS

Summary of Data Mining focused onClustering

Discussion of ACS-based Data Mining

FPGA Co-processor Enhancement Conclusions

Future Work
http://images.google.com/imgres?imgurl=freud.psy.fsu.edu/~chalcraf/downloads/FSU1851Seal.gif&imgrefurl=http://freud.psy.fsu.edu/~chalcraf/downloads/&h=664&w=664&sz=22&tbnid=8MAt2M0o80sJ:&tbnh=136&tbnw=136&start=9&prev=/images%3Fq%3Dfsu%2Bseal%26hl%3Den%26lr%3D%26ie%3DUTF-8


3/51

Isaacs 3

MIMI

LL

MAPLD2005/P249

Project Goal: to design and implement an Ant ColonySystems toolbox for non-combinatorial problem

solving. This toolbox will comprise both hardware and

software based solutions.
http://images.google.com/imgres?imgurl=freud.psy.fsu.edu/~chalcraf/downloads/FSU1851Seal.gif&imgrefurl=http://freud.psy.fsu.edu/~chalcraf/downloads/&h=664&w=664&sz=22&tbnid=8MAt2M0o80sJ:&tbnh=136&tbnw=136&start=9&prev=/images%3Fq%3Dfsu%2Bseal%26hl%3Den%26lr%3D%26ie%3DUTF-8http://images.google.com/imgres?imgurl=freud.psy.fsu.edu/~chalcraf/downloads/FSU1851Seal.gif&imgrefurl=http://freud.psy.fsu.edu/~chalcraf/downloads/&h=664&w=664&sz=22&tbnid=8MAt2M0o80sJ:&tbnh=136&tbnw=136&start=9&prev=/images%3Fq%3Dfsu%2Bseal%26hl%3Den%26lr%3D%26ie%3DUTF-8


4/51

Isaacs 4

MIMI

LL

MAPLD2005/P249

Ant Colony Systems Project Overview

This work aims at advancing fundamental research inAnt Colony Systems.

The major objectives of this project are:

Develop a set of behavior models

Design ACS algorithms for solutions to non-combinatorial

problems

Analyze algorithms for hardware implementations

Implement FPGA Modules CURRENT

Incorporate all modules into a cohesive toolbox


5/51

Isaacs 5

MIMI

LL

MAPLD2005/P249

Introduction to Ant Colony Systems

Ants are model organisms for bio-simulations due to both their relative

individual simplicity and their complex group behaviors.

Colonies have evolved means for collectively performing tasks that are far

beyond the capacities of individual ants. They do so without direct

communication or centralized control Stigmergy. Previous Research: our use of simulated ants to generate random numbers

proved a novel application for ACS.

Prior to 1992, ACS was used exclusively to study real ant behavior.

However, in the last decade, beginning with Marco Dorigos 1992 PhD

Dissertation Optimization, Learning and Natural Algorithms, modeling the way

real ants solve problems using pheromones, ant colony simulations have providedsolutions to a variety of NP-hard combinatorial optimization problems


6/51

Isaacs 6

MIMI

LL

MAPLD2005/P249

ACS Application Area: Data Mining

Ant Colony real-world behaviors applicableto Data Mining: Ant Foraging

Cemetery Organization and Brood Sorting

Division of Labor and Task Allocation

Self-organization and Templates

Co-operative Transport Nest Building


7/51Isaacs 7

MIMI

LL

MAPLD2005/P249

Cemetery Organization and Brood Sorting


8/51Isaacs 8

MIMI

LL

MAPLD2005/P249

Ant Colony Nest Examples


9/51Isaacs 9

MIMI

LL

MAPLD2005/P249

Feature/Object

Classification

Recognize

Clustering

Connection Topology

Store New Object

NEST (Data Warehouse)

ACS Data Mining

NO YES

Update Cognitive Map

Data

Flowchart for the ACS Data Mining System


10/51Isaacs 10

MIMI

LL

MAPLD2005/P249

Knowledge Discovery and

Data Mining

What is Data Mining?

Discovery of useful summaries of data

Also, Data Mining refers to a collection of techniques for

extracting interesting relationships and knowledge hiddenin data.

It is best described as the nontrivial process ofidentifying valid, novel, potentially useful, and ultimatelyunderstandable patterns in data. (Fayyad, et al 1996)


11/51Isaacs 11

MIMI

LL

MAPLD2005/P249

Knowledge Discovery in Databases

Data

Warehouse

Prepared

data

Data

Cleaning

Integration

Selection

TransformationData

Mining

Patterns

Evaluation

Visualization

Knowledge

Knowledge

Base


12/51Isaacs 12

MIMI

LL

MAPLD2005/P249

Typical Tasks in Data Mining

Classification

Prediction

Clustering Association Analysis

Summarization


13/51Isaacs 13

MIMI

LL

MAPLD2005/P249

Clustering

What is Clustering?

Given points in some space, often a high-dimensional space, group the points into a

small number of clusters, each cluster

consisting of points that are near in somesense.


14/51

Isaacs 14

MIMI

LL

MAPLD2005/P249

The k-Means Algorithm

k-means picks k cluster centroids and assigns points to the clusters by picking the

closest centroid to the point in question. As points are assigned to clusters, thecentroid of the cluster may migrate.

For a very simple example of five points in two dimensions. Suppose we assign the

points 1, 2, 3, 4, and 5 in that order, with k = 2. Then the points 1 and 2 are assigned

to the two clusters, and become their centroids for the moment.

When we consider point 3, suppose it is closer to 1, so 3 joins the cluster of 1, whosecentroid moves to the point indicated as a. Suppose that when we assign 4, we find

that 4 is closer to 2 than to a, so 4 joins 2 in its cluster, whose center thus moves to b.

Finally, 5 is closer to a than to b, so it joins the cluster {1,3}, whose centroid moves

to c.


15/51

Isaacs 15

MIMI

LL

MAPLD2005/P249

The k-Means Algorithm

Having located the centroids of the k clusters, we can reassign

all points, since some points that were assigned early may

actually wind up closer to another centroid, as the centroids

move about. If we are not sure of k, we can try different valuesof k until we find the smallest k such that increasing k does not

much decrease the average distance of points to their centroids.


16/51

Isaacs 16

MIMI

LL

MAPLD2005/P249

E= {Oi,, On} Set of n data or objects collected.

Oi = {vi,, vk} Each object is a vector of k numerical attributes.

Vector similarity is measured by Euclidean distance (can use

other: Minkowski, Hamming, or Mahalanobis).

Dmax = max D{Oi, Oj}, where Oi,Oj

E

ACS Notation and Heuristics


17/51

Isaacs 17

MIMI

LL

MAPLD2005/P249

2-D search area, in general, must be at least m

2

n, but experiments haveshown that m2 4n provides good results.

A heap/pileHis considered to be a collection of two or more objects. This

collection is located on a given single cell rather than just spatially connected.

This limitation prevents overlaps.

O1 O2

O4

O5

O3

O5

O4

O2

O3

O1

Spatial pattern cluster Single-cell ranked cluster

ACS Notation and Heuristics


18/51

Isaacs 18

MIMI

LL

MAPLD2005/P249

Dmax is the maximum distance between two objects ofH:

Ocenter is the center of mass of all objects inH: (not necessarily a real

object)

Odissim is the most dissimilar object inH,i.e. which maximizes

Dmean is the mean distance between the objects ofHand the center of

mass Ocenter :

=HO

centeriH

mean

i

HOODn

HD ))(,(1

)(

= HO iHcenter i OnHO1

)(

),(max)(,

max jiHOO

OODHD

ji

=

))((., HOD center

ACS Distance Measures


19/51

Isaacs 19

MIMI

LL

MAPLD2005/P249

ACS Unsupervised Learning and Clustering

Algorithm

Initialize randomly the ant positions

Repeat

For each anti Do

Move anti If anti does not carry any object Then look at 8-cell

neighborhood and pick up object according to pick-upalgorithm

Else (anti is already carrying an object O) look at 8-cell

neighborhood and drop O according to drop-off algorithm

Until stopping criterion


20/51

Isaacs 20

MIMI

LL

MAPLD2005/P249

ACS Data Mining Algorithm

Top Level

1. Load Database

2. Data Compression

3. Object Clustering

4. Clustering of Similar Groups

5. Reevaluate Objects in Groups


21/51

Isaacs 21

MIMI

LL

MAPLD2005/P249

ACS Data Mining Algorithm

Top Level

Load Database

Select Compression Method

Wavelets

Principle Component Analysis

None

Repeat for Max_Iterations1 Object Clustering Begin Ants Redistribute Objects

K-means

Repeat for Max_Iterations2 Clustering of Similar Groups

Ants Redistribute Piles (Clusters) of Objects

K-means

Repeat for Max_Iterations3 Reevaluate Objects in Groups

Ants Redistribute Objects in Clusters with a Probability based on Least Similar ObjectsDistance from the Mean of the Cluster

K-means


22/51

Isaacs 22

MIMI

LL

MAPLD2005/P249

ACS Object Pick-up Algorithm

1. Label 8-cell neighborhood as unexplored

2. Repeat

1. Consider the next unexplored cell c around anti with the following order: cell 1is

NW, cell 2 is N, cell 3 is NE, N is the direction the ant is facing.

2. If c is not empty Then do one of the following:

1. If c contains a single object O, Then load O with probability Pload, Else

2. If c contains a heap of two objects, Then remove one of the two with a probability

Pdestroy, Else

3. If c contains a heap H of more than 2 objects, Then remove the most dissimilar object

Odissim(H) from H provided that

3. Label c as explored

3. Until all 8 cells have been explored or one object has been loaded

removemean

centerdissim

THD

HOHOD>)(

))(),((


23/51

Isaacs 23

MIMI

LL

MAPLD2005/P249

ACS Object Drop-off Algorithm

1. Label 8-cell neighborhood as unexplored

2. Repeat

1. Consider the next unexplored cell c around anti with the following order: cell 1is

NW, cell 2 is N, cell 3 is NE, N is the direction the ant is facing.

1. If c is empty Then drop O in cell with a probability Pdrop, Else

2. If c contains a single object O, Then drop O to create a heap H provided that:

Else

3. If c contains a heap H, Then drop O on H provided that:

2. Label c as explored

3. Until all 8 cells have been explored or carried object has been dropped

createTD

OOD>

max

' ),(

))(),(())(,(HOHODHOOD

centerdissimcenter

ant colony data mining

Documents