data mining ii - 1dl460 filekjell orsborn - udbl - it - uu 25/02/16 2 introductory to spatial mining...
Post on 02-Nov-2019
3 Views
Preview:
TRANSCRIPT
25/02/16 1 Kjell Orsborn - UDBL - IT - UU
DATA MINING II - 1DL460
Spring 2016
A second course in data mining������
http://www.it.uu.se/edu/course/homepage/infoutv2/vt16
Kjell Orsborn���Uppsala Database Laboratory���
Department of Information Technology, Uppsala University, ���Uppsala, Sweden
25/02/16 2 Kjell Orsborn - UDBL - IT - UU
Introductory to Spatial Mining
• Slides from the book by Dr. M. H. Dunham, Data Mining, Introductory and Advanced Topics, Prentice Hall, 2002.
25/02/16 3 Kjell Orsborn - UDBL - IT - UU
Spatial Mining Outline
Goal: Provide an introduction to some spatial mining techniques.���
• Introduction• Spatial Data Overview • Spatial Data Mining Primitives• Generalization/Specialization• Spatial Rules• Spatial Classification• Spatial Clustering
25/02/16 4 Kjell Orsborn - UDBL - IT - UU
Spatial Object
• Contains both spatial and nonspatial attributes.• Must have a location type attributes:
– Coordinates– Latitude/longitude– Zip code– Street address
• May retrieve object using either (or both) spatial or nonspatial attributes.
25/02/16 5 Kjell Orsborn - UDBL - IT - UU
Spatial Data Mining Applications
• Geology• GIS Systems• Environmental Science• Agriculture• Medicine• Robotics• Engineering• May involve both spatial and temporal aspects
25/02/16 6 Kjell Orsborn - UDBL - IT - UU
Spatial Queries
• Spatial selection may involve specialized selection comparison operations:– Near– North, South, East, West– Contained in– Overlap/intersect
• Region (Range) Query – find objects that intersect a given region.• Nearest Neighbor Query – find object close to identified object.• Distance Scan – find object within a certain distance of an identified
object where distance is made increasingly larger.
25/02/16 7 Kjell Orsborn - UDBL - IT - UU
Spatial Data Structures
• Data structures designed specifically to store or index spatial data.
• Often based on B-tree or Binary Search Tree• Cluster data on disk basked on spatial/geographic location.• May represent complex spatial structure by placing the
spatial object in a containing structure of a specific spatial/geographic shape.
• Techniques:– Quad Tree– R-Tree– k-D Tree– X-Tree
25/02/16 8 Kjell Orsborn - UDBL - IT - UU
MBR
• Minimum Bounding Rectangle• Smallest rectangle that completely contains the object
25/02/16 9 Kjell Orsborn - UDBL - IT - UU
MBR Examples
25/02/16 10 Kjell Orsborn - UDBL - IT - UU
Quad Tree
• Hierarchical decomposition of the space into quadrants (MBRs)
• Each level in the tree represents the object as the set of quadrants which contain any portion of the object.
• Each level is a more exact representation of the object.• The number of levels is determined by the degree of
accuracy desired.
25/02/16 11 Kjell Orsborn - UDBL - IT - UU
Quad Tree Example
25/02/16 12 Kjell Orsborn - UDBL - IT - UU
R-Tree
• As with Quad Tree the region is divided into successively smaller rectangles (MBRs).
• Rectangles need not be of the same size or number at each level.
• Rectangles may actually overlap.• Lowest level cell has only one object.• Tree maintenance algorithms similar to those for B-trees.
25/02/16 13 Kjell Orsborn - UDBL - IT - UU
R-Tree Example
25/02/16 14 Kjell Orsborn - UDBL - IT - UU
K-D Tree
• Designed for multi-attribute data, not necessarily spatial• Variation of binary search tree• Each level is used to index one of the dimensions of the
spatial object.• Lowest level cell has only one object• Divisions not based on MBRs but successive divisions of
the dimension range.
25/02/16 15 Kjell Orsborn - UDBL - IT - UU
k-D Tree Example
25/02/16 16 Kjell Orsborn - UDBL - IT - UU
X-tree
• An X-tree (for eXtended node tree) is an index tree structure based on the R-tree used for storing data in many dimensions.
• It appeared in 1996.• Differs from R-trees (1984), R+-trees (1987) and R*-trees (1990) because it
emphasizes prevention of overlap in the bounding boxes, which increasingly becomes a problem in high dimensions.
• In cases where nodes cannot be split without preventing overlap, the node split will be deferred, resulting in super-nodes.
25/02/16 17 Kjell Orsborn - UDBL - IT - UU
Topological Relationships
• Disjoint• Overlaps or Intersects• Equals• Covered by or inside or contained in• Covers or contains
25/02/16 18 Kjell Orsborn - UDBL - IT - UU
Distance between spatial objects
• Distance between points– E.g Euclidean and Manhattan disctance
• Extensions for spatial objects (compare clustering):
25/02/16 19 Kjell Orsborn - UDBL - IT - UU
Progressive refinement
• Make approximate answers prior to more accurate ones.• Filter out data not part of answer• Hierarchical view of data based on spatial relationships• Coarse predicate recursively refined
25/02/16 20 Kjell Orsborn - UDBL - IT - UU
Progressive Refinement
25/02/16 21 Kjell Orsborn - UDBL - IT - UU
Spatial Data Dominant Algorithm
25/02/16 22 Kjell Orsborn - UDBL - IT - UU
STING
• STatistical Information Grid-based• Hierarchical technique to divide area into rectangular cells• Grid data structure contains summary information about
each cell• Hierarchical clustering • Similar to quad tree
25/02/16 23 Kjell Orsborn - UDBL - IT - UU
STING
25/02/16 24 Kjell Orsborn - UDBL - IT - UU
STING Build Algorithm
25/02/16 25 Kjell Orsborn - UDBL - IT - UU
STING Algorithm
25/02/16 26 Kjell Orsborn - UDBL - IT - UU
Spatial Rules
• Characteristic Rule - describes the data– The average family income in Dallas is $50,000.���
• Discriminant Rule - describe the differences between different classes of the data– The average family income in Dallas is $50,000, while in Plano
the average income is $75,000.���
• Association Rule - implications of one set of data by another– The average family income in Dallas for families living near
White Rock Lake is $100,000.
25/02/16 27 Kjell Orsborn - UDBL - IT - UU
Spatial Association Rules
• Either antecedent or consequent must contain spatial predicates.
• View underlying database as set of spatial objects.– compare with a set of transactions in conventional
association analysis• May be created using a type of progressive refinement
– First filter on coarser level using e.g. concept hierarchies or MBR’s
25/02/16 28 Kjell Orsborn - UDBL - IT - UU
Spatial Association Rule Algorithm
25/02/16 29 Kjell Orsborn - UDBL - IT - UU
Spatial Classification
• Partitions spatial objects• May use nonspatial attributes and/or spatial attributes• As in other types of of spatial mining, generalization and
progressive refinement may be used to improve efficiency.
25/02/16 30 Kjell Orsborn - UDBL - IT - UU
ID3 Extension
• Neighborhood Graph constructed for nodes in space – Nodes – objects– Edges – connects neighbors
• Definition of neighborhood varies– e.g distance between objects, topological relationships
between objects and direction relationship• ID3 considers nonspatial attributes of all objects in a
neighborhood (not just one) for classification.
25/02/16 31 Kjell Orsborn - UDBL - IT - UU
Spatial Decision Tree
• Approach similar to that used for spatial association rules.• Spatial objects can be described based on objects close to
them – area around an object called buffer.• Description of class based on aggregation of nearby
objects.• A training set with known classification is used for
building a model based on most relevant predicates (spatial and non-spatial)
25/02/16 32 Kjell Orsborn - UDBL - IT - UU
Spatial Decision Tree Algorithm
25/02/16 33 Kjell Orsborn - UDBL - IT - UU
Spatial Clustering
• Detect clusters of irregular shapes• Use of centroids and simple distance approaches may not
work well.• Clusters should be independent of order of input.
25/02/16 34 Kjell Orsborn - UDBL - IT - UU
Spatial Clustering
25/02/16 35 Kjell Orsborn - UDBL - IT - UU
CLARANS Extensions
• CLARANS – clustering large applications based upon randomized search
• Remove main memory assumption of original CLARANS.• Use spatial index techniques.• Use sampling and R*-tree to identify central objects.• Find cluster for these central objects.• Change cost calculations by reducing the number of
objects examined.• Voronoi diagram applied to define clusters
25/02/16 36 Kjell Orsborn - UDBL - IT - UU
Voronoi
25/02/16 37 Kjell Orsborn - UDBL - IT - UU
SD(CLARANS)
• Spatial Dominant CLARANS• An initial filtering to extract relevant data based on non
spatial data• First clusters spatial components using CLARANS• Examines the non-spatial data within each cluster to derive
a description of that cluster– e.g. to decide upon major type of vegetation within a region
• Uses a learning tool to derive description of cluster.
25/02/16 38 Kjell Orsborn - UDBL - IT - UU
SD(CLARANS) Algorithm
25/02/16 39 Kjell Orsborn - UDBL - IT - UU
DBCLASD
• Extension of DBSCAN• Distribution Based Clustering of LArge Spatial Databases• Assumes items in cluster are uniformly distributed.• Identifies distribution satisfied by distances between
nearest neighbors.• Objects added if distribution is uniform.
25/02/16 40 Kjell Orsborn - UDBL - IT - UU
DBCLASD Algorithm
25/02/16 41 Kjell Orsborn - UDBL - IT - UU
Aggregate proximity• Approximation can be used to identify the characteristics of clusters
by determining features that are close.– features are spatial objects such as rivers, oceans, schools, etc.���
• Aggregate proximity – measure of how close a cluster is to a feature.���
• Aggregate proximity relationship finds the k closest features to a cluster.���
• CRH Algorithm uses different shapes (with increasing accuracy and decreasing efficency) to identify these closest features:– Encompassing Circle– Isothetic Rectangle– Convex Hull
25/02/16 42 Kjell Orsborn - UDBL - IT - UU
CRH
25/02/16 43 Kjell Orsborn - UDBL - IT - UU
Temporal Mining Outline
Goal: examine some temporal data mining issues and approaches.���
• Introduction• Modeling Temporal Events• Time Series• Pattern Detection• Sequences• Temporal Association Rules
25/02/16 44 Kjell Orsborn - UDBL - IT - UU
Temporal Database
• Snapshot – Traditional database• Temporal – Multiple time points• Ex:
25/02/16 45 Kjell Orsborn - UDBL - IT - UU
Temporal Queries
• Query
• Database
• Intersection query
• Inclusion query
• Containment query
• Point query – Tuple retrieved is valid at a particular point
in time.
tsq te
q
tsd te
d
tsq te
q tsd te
d
tsq te
q tsd te
d
tsq te
q tsd te
d
25/02/16 46 Kjell Orsborn - UDBL - IT - UU
Types of Databases
• Snapshot – No temporal support���
• Transaction Time – Supports time when transaction inserted data– Timestamp– Range���
• Valid Time – Supports time range when data values are valid���
• Bitemporal – Supports both transaction and valid time.
25/02/16 47 Kjell Orsborn - UDBL - IT - UU
Modeling Temporal Events
• Techniques to model temporal events.• Often based on earlier approaches• Finite State Recognizer (Machine) (FSR)
– Each event recognizes one character– Temporal ordering indicated by arcs– May recognize a sequence– Require precisely defined transitions between states
• Additional approaches– Markov Model– Hidden Markov Model– Recurrent Neural Network
25/02/16 48 Kjell Orsborn - UDBL - IT - UU
FSR - Finite State Recognizer (Machine)
• Start state 0• Recognizing t shift state to state 1• Recognizing h in state 1 shift state to state 2• Recognizing e in state 2 shift state to state 3 (end node)• Recognizing other characters (*) shift state to state 0
25/02/16 49 Kjell Orsborn - UDBL - IT - UU
Markov Model (MM)
• Directed graph– Vertices represent states– Arcs show transitions between states– Arc has probability of transition– At any time, one state is designated as current state.
• Markov Property – Given a current state, the transition probability is independent of any previous states.
• Applications: speech recognition (nodes-sounds), natural language processing (nodes-words), system availability (nodes-system states)
25/02/16 50 Kjell Orsborn - UDBL - IT - UU
Markov Model
• A transition (arc) is associated with a transition probability, pij, i.e. the probability on an arc that a transition will be made from state i to state j.
• Probabilities can be combined to determine the probability that a pattern will be produced by the MM.
• Transition probabilities are learned during a training phase
25/02/16 51 Kjell Orsborn - UDBL - IT - UU
Hidden Markov Model (HMM)
• Like MM, but states need not correspond to observable states.• HMM models processes that produces as output a sequence of
observable symbols.– Given a sequence of symbols, the HMM can be constructed to produce
and output these symbols.– What is hidden is the state sequence that produced these symbols.
• Associated with each node is the probability of the observation of an event.
• Train HMM to recognize a sequence.• Transition and observation probabilities learned from training set.
25/02/16 52 Kjell Orsborn - UDBL - IT - UU
Hidden Markov Model
• The two states in the example represent two different coins• It is equally likely to toss either of the coins in the next toss • The hidden part of the model is associated with the observable output from each
state– Left coins is fair (head = 0.5, tail = 0.5), Right coin is unfair (head = 0.3, tail = 0.7)
• The hidden probabilities are used to determine what the output from that state will be (H/T) while the public (or transition) probabilities determine the next state that will occur (tossing fair/unfair coin)
Modified from [RJ86]
25/02/16 53 Kjell Orsborn - UDBL - IT - UU
HMM Algorithm
25/02/16 54 Kjell Orsborn - UDBL - IT - UU
HMM Applications
• Given a sequence of events and an HMM, what is the probability that the HMM produced the sequence?
• Given a sequence and an HMM, what is the most likely state sequence which produced this sequence?
25/02/16 55 Kjell Orsborn - UDBL - IT - UU
Recurrent Neural Network (RNN)
• Extension to basic NN• Neuron can obtain input from any other neuron (including
output layer).• Can be used for both recognition and prediction
applications.• Time to produce output unknown• Temporal aspect added by backlinks.
25/02/16 56 Kjell Orsborn - UDBL - IT - UU
RNN
25/02/16 57 Kjell Orsborn - UDBL - IT - UU
Time Series
• Set of attribute values over time• Time Series Analysis – finding patterns in the values.
– Trends– Cycles– Seasonal– Outliers
25/02/16 58 Kjell Orsborn - UDBL - IT - UU
Analysis techniques
• Trend analysis - detecting trends in time series– Smoothing – moving average of attribute values.– Autocorrelation – provides relationships between different
subseries with lag intervals• yearly, seasonal• lag – time difference between related items.• the autocorrelation coefficient rk measures the correlations between
time series values a certain disctance, lag k, apart.• e.g. Pearson’s correlation coefficient r
25/02/16 59 Kjell Orsborn - UDBL - IT - UU
Smoothing
25/02/16 60 Kjell Orsborn - UDBL - IT - UU
Correlation with Lag of 3
25/02/16 61 Kjell Orsborn - UDBL - IT - UU
Similarity between sequences
• Determine similarity between a target pattern, X, and sequence, Y: sim(X,Y)
• Similar to Web usage mining• Similar to earlier word processing and spelling corrector
applications.• Issues:
– Length– Scale– Gaps– Outliers– Baseline
25/02/16 62 Kjell Orsborn - UDBL - IT - UU
Longest Common Subseries
• Find longest subseries they have in common.• Ex:
– X = <10,5,6,9,22,15,4,2>– Y = <6,9,10,5,6,22,15,4,2>– Output: <22,15,4,2>– Sim(X,Y) = l/n = 4/9
25/02/16 63 Kjell Orsborn - UDBL - IT - UU
Similarity based on Linear Transformation• Linear transformation function f
– Convert a value from one series to a value in the second– Algorithms for finding f in “Time-series similarity problems and well-separated
geometric sets”, Bollobas et al, Proceeding SCG '97.
• εf – tolerated difference in results• δ – time value difference allowed
25/02/16 64 Kjell Orsborn - UDBL - IT - UU
Prediction
• Predict future value for time series• Regression may not be sufficient• Stationary vs nonstationary • Statistical Techniques
– ARMA (Autoregressive Moving Average)– ARIMA (Autoregressive Integrated Moving Average)
• Special techniques might be needed for online infinite streams– Algorithms for moving average discussed in stream mining
section
25/02/16 65 Kjell Orsborn - UDBL - IT - UU
Pattern Detection
• Identify patterns of behavior in time series• Speech recognition, signal processing• Can make use of FSR, MM, HMM
25/02/16 66 Kjell Orsborn - UDBL - IT - UU
Pattern detection – string matching
• Find given pattern in sequence• Knuth-Morris-Pratt: Construct FSM• Boyer-Moore: Construct FSM
25/02/16 67 Kjell Orsborn - UDBL - IT - UU
Pattern detection - distance between strings
• Cost to convert one to the other• Transformations
– Match: Current characters in both strings are the same– Delete: Delete current character in input string– Insert: Insert current character in target string into string
25/02/16 68 Kjell Orsborn - UDBL - IT - UU
Pattern detection - distance between strings
25/02/16 69 Kjell Orsborn - UDBL - IT - UU
Frequent Sequence
25/02/16 70 Kjell Orsborn - UDBL - IT - UU
Frequent Sequence Example
• Purchases made by customers• s(<{A},{C}>) = 1/3• s(<{A},{D}>) = 2/3• s(<{B,C},{D}>) = 2/3
25/02/16 71 Kjell Orsborn - UDBL - IT - UU
Frequent Sequence Lattice
25/02/16 72 Kjell Orsborn - UDBL - IT - UU
SPADE
• Sequential Pattern Discovery using Equivalence classes• Identifies patterns by traversing lattice in a top down
manner.• Divides lattice into equivalent classes and searches each
separately.• ID-List: Associates customers and transactions with each
item.
25/02/16 73 Kjell Orsborn - UDBL - IT - UU
SPADE Example
• ID-List for Sequences of length 1:
• Count for <{A}> is 3• Count for <{A},{D}> is 2
25/02/16 74 Kjell Orsborn - UDBL - IT - UU
Θ1 Equivalence Classes
25/02/16 75 Kjell Orsborn - UDBL - IT - UU
SPADE Algorithm
25/02/16 76 Kjell Orsborn - UDBL - IT - UU
Temporal Association Rules
• Transaction has time:<TID, CID, I1, I2, …, Im, ts, te>
• [ts, te] is range of time the transaction is active.• Types:
– Inter-transaction rules– Episode rules– Trend dependencies– Sequence association rules– Calendric association rules
25/02/16 77 Kjell Orsborn - UDBL - IT - UU
Temporal Association Rules Inter-transaction rules
• Intra-transaction association rulesTraditional association Rules
• Inter-transaction association rules– Rules across transactions– Sliding window – How far apart (time or number of transactions)
to look for related itemsets.
25/02/16 78 Kjell Orsborn - UDBL - IT - UU
Temporal Association Rules Episode Rules
• Association rules applied to sequences of events.• Episode – set of event predicates and partial ordering on
them
25/02/16 79 Kjell Orsborn - UDBL - IT - UU
Temporal Association Rules Trend Dependencies
• Association rules across two database states based on time.• Ex: (SSN,=) ⇒ (Salary, ≤)���
���Confidence=4/5���Support=4/36
25/02/16 80 Kjell Orsborn - UDBL - IT - UU
Temporal Association Rules Sequence Association Rules
• Association rules involving sequences• Ex:
<{A},{C}> ⇒ <{A},{D}>Support = 1/3Confidence 1
25/02/16 81 Kjell Orsborn - UDBL - IT - UU
Temporal Association Rules Calendric Association Rules
• Each transaction has a unique timestamp.• Group transactions based on time interval within which
they occur.• Identify large itemsets by looking at transactions only in
this predefined interval.
top related