1

Pattern Classification

All materials in these slides were taken from Pattern Classification (2nd ed) by R. O. Duda, P. E. Hart and D. G. Stork, John Wiley & Sons, 2000 with the permission of the authors and the publisher

2

Intro to Pattern Recognition Chapter 1: Sections 1.1-1.6

Machine Perception An Example

Pattern Recognition Systems The Design Cycle

Learning and Adaptation Conclusion

3

Machine Perception

Build a machine that can recognize patterns:

Speech recognition

Fingerprint identification

OCR (Optical Character Recognition)

DNA sequence identification

4

An Example

“Sorting incoming Fish on a conveyor,

according to species using optical sensing”

Sea bassSpecies

Salmon

5

6

Problem Analysis

Take sample images … to extract features Length Lightness Width Number and shape of fins Position of the mouth etc…

= set of features to use in our classifier!

7

Use a segmentation operation to isolate fish from one another and from the background

Information from each single fish is sent to a feature extractor, whose purpose is to reduce the data by measuring certain features

The features are passed to a classifier

Preprocessing

8

9

Select the length of the fish as a possible feature for discrimination

Classification

10

11

The length is a poor feature alone!

Select the lightness as a possible feature.

12

13

Move decision boundary toward smaller values of lightness, to minimize cost reduce number of sea bass classified as salmon!

Threshold decision boundary and cost relationship

Task of decision theory

14

Adopt the lightness and add the width of the fish

Fish xT = [x1, x2]

Lightness Width

15

16

Perhaps add other features not correlated with current ones Warning: “noisy features” will reduce

performance

Best decision boundary == one that provides an optimal performance Eg …

17

“Optimal Performance” ??

18

Goal: Optimal performance on NOVEL data Performance on TRAINING DATA !=

Performance on NOVEL data

Objective: Dealing with Novel Data

Issue of generalization!

19

20

Pattern Recognition Systems

Sensing Using transducer (camera, microphone, …) PR system depends of the bandwidth, the

resolution sensitivity distortion of the transducer

Segmentation and grouping Patterns should be well separated

(should not overlap)

21

22

Feature extraction Discriminative features Want features INVARIANT wrt translation,

rotation, scale. Classification

Using feature vector (provided by feature extractor) to assign given object to a category

Post Processing Exploit context

info not from the target pattern itself to improve performance

23

The Design Cycle

Data collection Feature Choice Model Choice Training Evaluation

Computational Complexity

24

25

How do we know when we have collected an adequately large and representative set of examples for training and testing the system?

Data Collection

26

Depends on characteristics of problem domain.

Ideally… Simple to extract Invariant to irrelevant transformation Insensitive to noise.

Feature Choice

27

If not satisfied with performance (EG, of our fish classifier)

Consider another class of model

Model Choice

28

Use data to obtain good classifier identify best model determine appropriate parameters

Many procedures for training classifiers and choosing models

Training

29

Measure error rate ~= performance

May suggest switching from one set of features to another one

Evaluation

30

Trade-off between computational ease and performance?

How algorithm scales as function of number of features, patterns or categories?

Computational Complexity

31

Learning and Adaptation

Supervised learning

A teacher provides a category label or cost for each pattern in the training set

Unsupervised learning

The system forms clusters or “natural groupings” of the input patterns

32

Conclusion

Reader seems to be overwhelmed by the number, complexity and magnitude of the sub-problems of Pattern Recognition

Many of these sub-problems can indeed be solved

Many fascinating unsolved problems still remain