Örüntü tanıma - pattern recognition
DESCRIPTION
Örüntü tanıma - Pattern Recognition Prepared by Hassa-k A. Moha Istanbul University, Faculty of Computer EngineeringTRANSCRIPT
www.company.com
Pattern Recognition Örüntü Tanıma
ÖĞRENCİ İSİM SOYADI
Hassan ABDI MOHAMED
ÖĞRETİM GÖREVLİSİ
Yrd. Doç. Dr. Zeynep ORMAN
Presentation contents Introduction Definitions (pattern , pattern recognition) Applications of Pattern Recognition Pattern Recognition Models Recognition Techniques
Supervised learning Unsupervised learning
Pattern Learning Methods K-nearest neighbor Hierarchical Clustering
Agglomerative (bottom-up) Divisive (top-down):
K-means Clustering
www.company.com
Overview
Humans have developed highly sophisticated skills for sensing their environment and taking actions according to what they observe, e.g. Recognizing a face, understanding spoken words, reading handwriting, distinguishing fresh food from its smell.
We would like to give similar capabilities to machines.
www.company.com
Overview
• Gestalt Rules of Perception• the whole is more than the sum of its parts•
Proximity - group nearby segments of images together
• Similarity - group similar things• Good continuation - perceive continuous patterns• Closure - fill in the gaps
www.company.com
Overview• Human can recognize the faces without worrying about
the varying illuminations.
• When Implementing such recognition artificially comes, then it becomes a very complex task.
• The fields of artificial intelligence have made this complex task possible.
• a branch of artificial intelligence is known as Pattern Recognition
www.company.com
What ıs a pattern ?• A pattern is a set of objects or phenomena or
concepts where the elements of the set are similar to one another in certain ways or aspects.
• • A pattern is an entity, that could be given a• name, e.g. fingerprint image, handwritten word,
human face, speech signal, DNA sequence
www.company.com
Some examples……
www.company.com
What is Pattern Recognition?Pattern recognition is the procedure of processing and analyzing diverse information(numerical, literal, logical) characterizing the objects or phenomenon, so as to provide descriptions, identifications, classifications and interpretations for them.
decisions about the categories of the patterns
A “Perceive + Process + Prediction” View ıt is the study of how machines can
www.company.com
Cont...• Perceive: Observe the environment (i.e. interact
with the real-world)• Process: Learn to distinguish patterns of interest
from their background• Prediction:make sound and reasonable
decisions about the categories of the patterns
www.company.com
Applications of Pattern Recognition
• Pattern recognition is used in any area of science and engineering that studies the structure of observations.
• It is now frequently used in many applications in manufacturing industry, health care and military.
•The following slides we will se the most common applications that is used for pattern recognition.
www.company.com
Applications of Pattern Recognition
• 1) Character Recognition
– Input: images with characters (normally contaminated with noise)
– Output: the identified character strings
– Useful in scenarios such as automatic license plate recognition (ALPR),optical character recognition (OCR), etc.
www.company.com
Applications of Pattern Recognition
• 2) Speech Recognition – Input: acoustic signal (e.g. sound waves) – Output: contents of the speech
– Useful in scenarios such as speech-to-text (STT), voice command & control, etc.
www.company.com
Applications of Pattern Recognition
• 3) Fingerprint Recognition
– Input: fingerprints of some person – Output: the person’s identity
– Useful in scenarios such as computerized access control, criminal pursuit, etc.
www.company.com
Applications of Pattern Recognition
• 4) Signature Identification
– Input: signature of some person (sequence of dots)– Output: the signatory’s identity
– Useful in scenarios such as digital signature verification, credit card anti-fraud, etc.
www.company.com
Applications of Pattern Recognition
• 5) Face Detection
– Input: images with several people– Output: locations of the peoples’ faces in the image
– Useful in scenarios such as digital camera capturing, video surveillance,etc.
www.company.com
Applications of Pattern Recognition
• 6) Text Categorization – Input: document, web pages, etc.– Output: category of the text, such as political,
economic, military, sports, etc.
– Useful in scenarios such as information retrieval, document organization,etc.
www.company.com
Applications of PR - More
www.company.com
Pattern recognition system
• design model of a pattern recognition system essentially involves the following three steps:-
Data acquisition and preprocessingFeature extractionDecision making
www.company.com
PATTERN RECOGNITION MODELS
• Statistical model pattern recognition systems are based on statistics and probabilities
• Syntactic model structural models for pattern recognition and are based on the relation between
features. Here the patterns are represented by structures
• Template matching model In this model, a template or a prototype of the pattern to be recognized is
available
• Neural network model an artificial neural network (ANN) is a self-adaptive trainable process that is able to
learn and resolve complex problems based on available knowledge.
www.company.com
Classification techniques
• There are two general types of classification techniques: supervised and unsupervised
• classification.• A classification procedure is supervised if the user• • defines the decision rules for each class directly or• • provides training data (class prototypes) for each class to guide the
computer classification.• A classification procedure is unsupervised if• • no training data are required• • the user needs to specify the number of classes (at most)
Example [ Örnek ] Classification and Clustering
Class1 1
Class 2
?
ClassificationClustering
Problem:
How to partition
How many clusters
Problem:
Which class to assign
www.company.com
Pattern recognition and classification• In classification type we already know the
categories of characters, and then classify the handwritten ones into category “A” and category “B”
• In clustering type We do not know the categories of symbols, and then learn the categories and group the symbols accordingly.
Supervised learning
unsupervised learning
www.company.com
Classification Process: Model Construction
TrainingData
NAME RANK YEARS TENUREDMike Assistant Prof 3 noMary Assistant Prof 7 yesBill Professor 2 yesJim Associate Prof 7 yesDave Assistant Prof 6 noAnne Associate Prof 3 no
ClassificationAlgorithms
IF rank = ‘professor’
OR years > 6
THEN tenured = ‘yes’
Classifier(Model)
www.company.com
Classification Process: Model usage in Prediction
Classifier
TestingData
NAME RANK YEARS TENUREDTom Assistant Prof 2 noMerlisa Associate Prof 7 noGeorge Professor 5 yesJoseph Assistant Prof 7 yes
Unseen Data
(Jeff, Professor, 4)
Tenured?
www.company.com
Supervised : K-Nearest Neighbors Method
• the k-Nearest Neighbors algorithm (or k-NN for short) is a non-parametric method used for classification and regression.
• In both cases, the input consists of the k closest training examples in the feature space
• In k-NN classification, the output is a class membership. An object is classified by a majority vote of its neighbors.
• In k-NN regression, the output is the property value for the object. This value is the average of the values of its knearest neighbors.
www.company.com
Supervised : K-Nearest Neighbors Method
• Lazy Learning Algorithm• Defer the decision to generalize beyond the training• examples till a new query is encountered• Whenever When ever we have a new point to classify,
classify, we find its K nearest neighbors from the training data.
• The distance is calculated using one of the following• measures
Euclidean Distance Minkowski Distance Mahalanobis Distance
www.company.com
Supervised : K-Nearest Neighbors Method
• The predicted class is determined from the nearest neighbor list
• classification• take the majority vote of class labels among the k-
nearest neighbor
www.company.com
K-Nearest Neighbors Method
• k-Nearest Neighbors algorithm (or k-NN for short) is a non-parametric method used for classification and regression. The output depends on whether k-NN is used for classification or regression:
• Example of k-NN classification.
www.company.com
Unsupervised : K-means Method
• K-means is a partitional clustering Method• Let the set of data points (or instances) D be
{x1, x2, …, xn},
where xi = (xi1, xi2, …, xir) is a vector in a real-valued space X Rr, and r is the number of attributes (dimensions) in the data.
• The k-means algorithm partitions the given data into k clusters. – Each cluster has a cluster center, called centroid.– k is specified by the user
www.company.com
Unsupervised : K-means Method
Algorithm of K-means Method1) Pick a number (K) of cluster centers (at random)2) Assign every item to its nearest cluster center (e.g.
using Euclidean distance)
3) Move each cluster center to the mean of its assigned items
4) Repeat steps 2,3 until convergence (change in cluster assignments less than a threshold)
www.company.com
K-means example, step 1
k1
k2
k3
X
Y
Pick 3 initialclustercenters(randomly)
www.company.com
K-means example, step 2
32
k1
k2
k3
X
Y
Assigneach pointto the closestclustercenter
www.company.com
K-means example, step 3
X
Y
Moveeach cluster centerto the meanof each cluster
k1
k2
k2
k1
k3
k3
www.company.com
K-means example, step 4
X
Y
Reassignpoints closest to a different new cluster center
Q: Which points are reassigned?
k1
k2
k3
www.company.com
K-means example, step 5
X
move cluster centers to
cluster means
k2
k1
k3
www.company.com
Unsupervised : Hierarchical Clustering Method
Agglomerative (bottom-up):
Start with each document being a single cluster.Eventually all documents belong to the same cluster.
Divisive (top-down):
Start with all documents belong to the same cluster.
Eventually each node forms a cluster on its own.
Does not require the number of clusters k in advance
Needs a termination/readout condition
The final mode in both Agglomerative and Divisive is of no use.
www.company.com
HAC - Hierarchical Agglomerative Clustering
Start with all instances in their own cluster.Until there is only one cluster:
Among the current clusters, determine the two clusters, ci and cj, that are most similar.
Replace ci and cj with a single cluster ci cj
www.company.com
Dendrogram: Document Example
38
• As clusters agglomerate, docs likely to fall into a hierarchy of “topics” or concepts.
d1
d2
d3
d4
d5
d1,d2 d4,d5 d3
d3,d4,d5
www.company.com
Divisive Hierarchical Clustering
• This top-down strategy does the reverse of agglomerative hierarchical.
Four widely used measure for distance between clusters
are as follows, where is the distance
between two objects or points, p and p’ ;
– mi is the mean for clusters, Ci
– ni is the number of objects Ci
1. Minimum Distance:
2. Maximum Distance:
3. Mean Distance:
4. Average Distance:
pp
www.company.com
Divisive Hierarchical Clustering
www.company.com
Measuring the distance of two clusters
A few ways to measure distances of two clusters.Results in different variations of the algorithm.
Single linkComplete linkAverage linkCentroids
www.company.com
END