pattern recognition

CPR 2007-2008 1

Armando Vieira&

Bernardete Ribeiro

2007

/200

8

Artificial Intelligence&

Pattern Recognition

Introduction

CPR 2007-2008 2

Program • Introduction to Artificial Intelligence• Soft Artificial Intelligence• Artificial Neural Networks: theory, training,

applications• Supervised Learning: Mulilayer Perceptron• Unsupervised Learning: Self-Organized Kohonen

Maps• Genetic Algorithms• Applications• Project

CPR 2007-2008 3

Pattern Recognition?

• A pattern is an object, process or event

• A class (or category) is a set of patterns that share common attribute (features) usually from the same information source

• During recognition (or classification) classes are assigned to the objects.

• A classifier is a machine that performs such task

“The assignment of a physical object or event to one of several pre-specified categories” -- Duda & Hart

CPR 2007-2008 4

What is a pattern?What is a pattern?“A pattern is the opposite of a chaos; it is an entity vaguely

defined, that could be given a name.”

CPR 2007-2008 5

Examples of PatternsCristal Patterns: atómic or molecular

Their structures are represented by 3D graphs and can be described by deterministic grammars or formal languages

CPR 2007-2008 6

Patterns of Constellations

Patterns of constellations are represented by 2D planar graphs

Human perception has strong tendency to find patterns from anything. We see patterns from even random noise --- we are more likely to believe a hidden pattern than denying it when the risk (reward) for missing (discovering) a pattern is often high.

Examples of Patterns

CPR 2007-2008 7

Biological Patterns ---morphology

Landmarks are identified from biologic forms and these patterns are then represented by a list of points. But for other forms, like the root of plants,Points cannot be registered crossing instances.

Applications: Biometrics, computacional anatomy, brain mapping, …


CPR 2007-2008 8

Biological Patterns

Landmarks are identified from biologic forms and these patterns are then represented by a list of points.


CPR 2007-2008 9

Music Patterns

Ravel Symphony?


CPR 2007-2008 10

People Recognition

Pat

tern

s B

ehav

ior

?

Funny, Funny


CPR 2007-2008 11

Discovery and Association of Patterns

Statistics show connections between the shape of one’s face (adults) and his/her Character. There is also evidence that the outline of children’s face is related to alcohol abuse during pregnancy.


CPR 2007-2008 12

What are the features?Statistics show connections between the shape of one’s face (adults) and his/her Character.

Examples of PatternsDiscovery and Association of Patterns

CPR 2007-2008 13

We may understand patterns of brain activity and find relationships between brain activities, cognition, and behaviors

Patterns of Brain Activity


CPR 2007-2008 14

Variation Patterns: 1. Expression – geometric deformation 2. illumination--- Photometric deformation 3. Transformation –3D pose 3D 4. Noise and Occlusion


CPR 2007-2008 15

A broad range of texture patterns are generated by stochastic processes.


CPR 2007-2008 16

. How are these patterns represented in human mind?


CPR 2007-2008 17

Speech signals and Hidden Markov models


CPR 2007-2008 18

Natural Language and stochastic grammar..


CPR 2007-2008 19

Object Recognition

Pat

tern

s ev

eryw

her

e ?


CPR 2007-2008 20

Maps Recognition

Pat

tern

s o

f G

lob

al W

arm

ing

?


CPR 2007-2008 21

Finacial Series Pattern Recognition


CPR 2007-2008 22

Ho

w t

o T

rad

e C

har

t P

atte

rns

?Examples of Patterns

CPR 2007-2008 23

Pattern Recognition in Medical Diagnosis

TomografiaTomografia

projecçã

oproj

ecção

retroproje

cção

retroproje

cção

ReconstruçãoReconstrução

Tomos (=corte) +grafos (=escrita, imagem, gráfico)

f(x,y,z)f(x,y,z)

projecçõesprojecçõesp(r,p(r,,z),z)

p(r,p(r,,z),z)f(x,y,z)f(x,y,z)


CPR 2007-2008 24

Optical Character Recognition


CPR 2007-2008 25

Gra

ph

ic A

rts

Escher, who else?


CPR 2007-2008 26

Human Genome

Bea

uti

ful

Pat

tern

s!


CPR 2007-2008 27

• Optical Character

Recognition (OCR)

• Biometrics

• Diagnostic systems

• Military applications

• Handwritten: sorting letters by postal code, input device for PDA‘s.

• Printed texts: reading machines for blind people, digitalization of text documents.

• Face recognition, verification, retrieval. • Finger prints recognition.• Speech recognition.

• Medical diagnosis: X-Ray, EKG analysis.• Machine diagnostics, waster detection.

• Automated Target Recognition (ATR).

• Image segmentation and analysis (recognition from aerial or satelite photographs).

Examples of Applications

CPR 2007-2008 28

Approaches

• Statistical PR: based on underlying statistical model of patterns and pattern classes.

• Neural networks: classifier is represented as a network of cells modeling neurons of the human brain (connectionist approach).

• Structural (or syntactic) PR: pattern classes represented by means of formal structures as grammars, automata, strings, etc.

CPR 2007-2008 29

An example of Pattern Recognition Classification of fish into two classes: salmon and Sea Bass by discriminative method

•“Sorting incoming Fish on a conveyor according to species using optical sensing”

CPR 2007-2008 30

Problem Analysis

– Set up a camera and take some sample images to extract features

• Length• Lightness• Width• Number and shape of fins• Position of the mouth, etc…

This is the set of all suggested features to explore for use in our classifier!

CPR 2007-2008 31

Pattern Recognition Phases

• Preprocess raw data from camera

• Segment isolated fish

• Extract features from each fish (length,width, brightness, etc.)

• Classify each fish

CPR 2007-2008 32

Pattern Recognition Phases

• Preprocessing– Use a segmentation operation to isolate

fishes from one another and from the background

• Information from a single fish is sent to a feature extractor whose purpose is to reduce the data by measuring certain features

• The features are passed to a classifier

CPR 2007-2008 33

• Classification

Select the length of the fish as a possible feature for discrimination

CPR 2007-2008 34

Features and Distributions

The length is a poor feature alone!

Select the lightness as a possible feature.

CPR 2007-2008 35

“Customers do not want sea bass in their cans of salmon”

• Threshold decision boundary and cost relationship

• Move our decision boundary toward smaller values of lightness in order to minimize the cost (reduce the number of sea bass that are classified salmon!)

Task of decision theory

CPR 2007-2008 36

• Adopt the lightness and add the width of the fish

Fish x = [x1, x2]

Lightness Width

CPR 2007-2008 37

Decision/classification Boundaries ?

CPR 2007-2008 38

CPR 2007-2008 39

• We might add other features that are not correlated with the ones we already have. A precaution should be taken not to reduce the performance by adding such “noisy features”

• Ideally, the best decision boundary should be the one which provides an optimal performance such as in the following figure:

CPR 2007-2008 40

CPR 2007-2008 41

• However, our satisfaction is premature because the central aim of designing a classifier is to correctly classify novel input

Issue of generalization!

CPR 2007-2008 42

CPR 2007-2008 43

Occam’s Razor

Entities are not to be multiplied without necessity

William of Occam (1284-1347)

CPR 2007-2008 44

A Complete PR System

CPR 2007-2008 45

Problem Formulation

Measurements Preprocessing

ClassificationFeaturesInputobject

ClassLabel

Basic ingredients:•Measurement space (e.g., image intensity, pressure)•Features (e.g., corners, spectral energy)•Classifier - soft and hard•Decision boundary•Training sample•Probability of error

CPR 2007-2008 46

Design Cycle

1. Feature selection and extraction --- What are good discriminative features?2. Modeling and learning 3. Dimension reduction, model complexity4. Decisions and risks5. Error analysis and validation.6. Performance bounds and capacity.7. Algorithms

CPR 2007-2008 47

CPR 2007-2008 48

• Data Collection

How do we know when we have collected an adequately large and representative set of examples for training and testing the system?

CPR 2007-2008 49

• Feature Choice

Depends on the characteristics of the problem domain. Simple to extract, invariant to irrelevant transformation, insensitive to noise.

CPR 2007-2008 50

• Model Choice

Unsatisfied with the performance of our linear fish classifier and want to jump to another class of model

CPR 2007-2008 51

• Training

Use data to determine the classifier. Many different procedures for training classifiers and choosing models

CPR 2007-2008 52

• Evaluation

Measure the error rate (or performance) and switch from one set of features & models to another one.

CPR 2007-2008 53

• Computational Complexity

What is the trade off between computational ease and performance?

(How an algorithm scales as a function of the number of features, number or training examples, number patterns or categories?)

pattern recognition

Documents

examples of applications

examples of patternsdiscovery

examples of patternshow

examples of patternspatterns

set of patterns

behaviors cpr

occlusion cpr

task cpr