1

Pattern Recognition:Statistical and Neural

Lonnie C. Ludeman

Lecture 12

Sept 30, 2005

Nanjing University of Science & Technology

2

Lecture 11 Topics

1. M-Class Case and Gaussian Review

2. M-Class Case in Likelihood Ratio Space

3. Example Vector Observation M-Class

3

if equality then decide x from the boundary classes by random choice

MPE and MAP Decision Rule: M-Class Case

Select class Ck

if p( x | Ck ) P(Ck ) > p( x | Cj ) P(Cj )

for all j = 1, 2, … , M j = k

for observed x

Review 1

4

yi(x) = Cij p(x | Cj) P(Cj)j=1

M

if yi(x) < yj(x) for all j = i

Then decide x is from Ci

Bayes Decision Rule: M-Class Case

Review 2

5

if- (x – M

1)TK

1

-1(x – M1) + (x – M

2)TK

2

-1(x – M2)

><

C1

C2

T1 = 2 ln(T ) = 2 lnT + ln - ln

K2

1 2

K1

1 2

T1

Optimum Decision Rule: 2-Class Gaussian

where

And T is the optimum threshold for the typeof performance measure used

K1

K2

Quadratic Processing

Review 3

6

if ( M1 – M

2)T K-1 x >

<C

1

C2

T2

2-Class Gaussian: Special Case 1: K1 = K

2 = K

And T is the optimum threshold for the typeof performance measure used

T2 = ln T + ½ ( M

1

T K-1 M1 – M

2

T K-1 M2)

Equal Covariance Matrices

Linear Processing

Review 4

7

if ( M1 – M

2)T x >

<C

1

C2

T3

2-Class Gaussian: Case 2: K1 = K

2 = K = s2 I

And T is the optimum threshold for the typeof performance measure used

T3 = s2 ln T + ½ ( M

1

T M1 – M

2

T M2)

Equal Scaled Identity Covariance Matrices

Linear Processing

Review 5

8

Q

i(x) = (x – M

j)TK

j

-1(x – Mj) } – 2 ln P(C

j) + ln | K

i |

dMAH

(x , Mj) Bias

Quadratic Operation on observation vector x

2

M- Class General Gaussian - Continued

Select Class Cj if Q

j(x) is MINIMUM

Equivalent statistic: Qj(x) for j = 1, 2, … , M

Review 6

9

Select Class Cj if L

j(x) is MAXIMUM

Lj(x) = M

j

TK-1x – ½ Mj

T

K-1M

j

+ lnP(Cj)

Equivalent Rule for MPE and MAP

M-Class Gaussian – Case 1: K1 = K

2 = … = K

M = K

Dot Product Bias

Linear Operation on observation vector x

Review 7

10

where yi(x) = Cij p(x | Cj) P(Cj)j=1

M

if yi(x) < yj(x) for all j = i

Then decide x is from Ci

Bayes Decision Rule in Likelihood ratio space: M-Class Case derivation

We know that Bayes Decision Rule for the M-Class Case is

11

LM

(x) = p(x | CM

) / p(x | CM

) = 1

Dividing through by p(x | CM) gives sufficient

statistics vi(x) as follows

Therefore the decisin rule becomes

12

Bayes Decision Rule in the Likelihood Ratio Space

The dimension of the Likelihood Ratio Space is always one less than the number of classes

13

Given: Three Classes C1, C

2, and C

3

Example: M-Class case

Nk are statistically independent all classes

Nk ~ N(0,1)

14

Determine:

This problem is an abstraction of a tri-phase communication system

(a) Find the minimum probability of error (MPE) decision rule

(b) Illustrate your decision regions in the observation space

(c) Use your decision rule to classify the observed pattern vector

x =[ 0.4, 0.7]T

(d) Calculate the probability of error P(error)

15

Solution:

Problem is Gaussian with equal scaled identity Covariance Matrices so the optimum decision rule is as follows

(a) Find MPE decision Rule

16

Select class with minimum Li(x)

/

for our example we have

17

Droping the -½ + ln 1/3 as it appears in all the Li(x),

the new statistics s1(x), s

2(x), and s

3(x)can be defined

as

and an equivalent decision rule becomes

18

This decision rule can be rewritten in terms of the observation x as follows

where in the observation space X, R

k is the region where

Ck is decided

19

Decision Region in the Observation Space

X Observation Space

20

(c) the pattern vector x

x = [ x1, x

2 ]T = [ 0.4, 0.7 ]T

x Is a member of R1 therefore x is classified

as coming from class C1

21

(d) Determine the probability of error

P(error) = 1- P(correct)

= 1 - P(correct |C1)P(C

1 )

- P(correct |C2)P(C

2 )

- P(correct |C3)P(C

3 )

22

23

24

P(error) = 0.42728

25

Summary

1. M-Class Case and Gaussian Review

2. M-Class Case in Likelihood Ratio Space

3. Example Vector Observation M-Class

26

End of Lecture 12