306 CHAPTER 8 HYPOTHESIS TESTING

Theorem 8.2

is a maximum a posteriori probability (MAP) hypothesis test. In such aall outcomes s for which P[Hols] > P[H1Is], and Al contains all ouP[Hlls] > P[Hols]. If P[Hols] = P[Hlls], the assignment of s todoes not affect PERR. In Theorem 8.1, we arbitrarily assign s to Ao whenprobabilities are equal. We would have the same probability of error if .-=for all outcomes that produce equal a posteriori probabilities or if we assign .=:.::-:::a.with equal a posteriori probabilities to Ao and others to AI.

Equation (8.8) is another statement of the MAP decision rule. It cn::!::::=probability models that are assumed to be known:

• The a priori probabilities of the hypotheses: P [Ho] and P [Hll,

• The likelihood function of Ho: P[sIHo], \

• The likelihood function of HI: P[sIHIl.

When the outcomes of an experiment yield a random vector X as the deciS::::::':::=:J••can express the MAP rule in terms of conditional PMFs or PDFs. If X isX = Xi to be the .outcome of the experiment. If the sample space S o~:;z C:'!'I:II"

is continuous, we interpret the conditional probabilities by assumingcorresponds to the random vector X in the small volume X ::::: X < x + ds: -""'-----Ix (x)dx. Section 4.9 demonstrates that the conditional probabilities aredensities. Thus in terms of the random variable X, we have the follo . =MAP hypothesis test.

For an experiment that produces a random vector X, the MAP hypothese -

A if PxIHo (x) P [HllXE 01 > ---;

PXIH1 (x) - P [Ho]

. . IxIHo (x) P [HI]Continuous: x E Ao if ::::P [Ho];

IxIHl (x)

Discrete: X E A 1 other.

In these formulas, the ratio of conditional probabilities is referred to as _The formulas state that in order to perform a binary hypothesis test, we obs=-::::~':::=.::l__of an experiment, calculate the likelihood ratio on the left side of the fouu.-..<c,'- _it with a constant on the right side of the formula. We can view the like' -evidence, based on an observation, in favor of Ho: If the likelihood ratio __Ho is more likely than HI. The ratio of prior probabilities, on the right si :.prior to performing the experiment, in favor of HI. Therefore, Theorem - -is the better conclusion if the evidence in favor of Ho, based on the eXIX!'I::::::::::t~-=~_the prior evidence in favor of HI.

In many practical hypothesis tests, including the following example, - ~H====:;:lI_compare the logarithms of the two ratios.

Example 8.6 With probability p, a digital communications system transmits a O. r.======~probability 1 - p. The received signal is either X = -v + N vobit is 0; or v + N volts, if the transmitted bit is 1. The voltage ±[ -~component of the received signal, and N, a Gaussian (0,0') ran ~

310 CHAPTER 8 HYPOTHESIS TESTING

Periorming the same substitutions and simplifications as -

In (qIP[HdCoI)

n E Ao if n 2: n" = 1 + qOtH01)IO = 58.92;in l-qo

l-ql

Therefore, in the minimum cost hypothesis test, Ao = { 2: _-at most 58 disk drives to reach a conclusion regarding Co

drives pass the test, then N 2: 59, and the failure rate' -"'.=-=~--=-error probabilities are:

PFA = P [N ::: 581Ho] = FNIHo (58) = 1 - (1 - 10-

PMISS = P [N 2: 591HrJ = 1 - FNIHI (58) = (1 - 10-: -

The average cost (in dollars) of this rule is

E [C] = P [Ho] PFAClO + P [HI] PMISSCOI= (0.9)(0.0058)(50,000) + (0.1)(0.0022)(

By comparison, the MAP test, which minimizes the probab:=I',3j,3:=:-:=-the expected cost, has an expected cost

E [CMAP] = (0.9)(0.0046)(50,000) + (0.1)(0.0079)(

A savings of $60 may not seem very large. The reason isthe minimum cost test work very well. By comparison, for a - -="'~-.'-'-""testing altogether, each day that the failure rate is qj = 0.11,000 returned drives at an expected cost of $200,000. S'with probability P[HIl = 0.1, the expected cost of a "no te

Neyman-Pearson Test

Given an observation, the MAP test minimizes the probability 0: !C:~~ ~---

hypothesis and the minimum cost test minimizes the cost of etest requires that we know the a priori probabilities P[Hj] of the a:t:::!?=;; t:...••__and the minimum cost test requires that we know in additiontwo types of errors. In many situations, these costs and a priori pITJ:3!:X:::;;;:-or even impossible to specify. In this case, an alternate appro act.tolerable level for either the false alarm or miss probability. This ,C-,~=,.-'"Neyman-Pearson test. The Neyman-Pearson test minimizes PMISS SI:.:~=:J:==_-'-.;dI.probability constraint PPA = a, where a is a constant that indicates cr=:~'iCie::==:::-alarms. Because PPA = P[AIIHo] and PMISS = P[AoIHd are '-'.h..e....- ..•.•.••••.•...'::==~_.the test does not require the a priori probabilities P[Ho] and P[Hl1-Neyman-Pearson test when the decision statistic is a continous randoc; .-~.:-

Theorem 8.4 Neyman-Pearson Binary Hypothesis Test~Based on the decision statistic X, a continuous random vector, the de;:=;c::,,::::

G = PMlSS + ),,(PFA - a)

= to !XIHI (x) dx +)..(1 - to fxlHo (x) dx - a )

= 1 (fxIHI (x) - VXIHo (x») ds: + ),,(1 - a)Ao

For a given x and a, we see that G is minimized if AO includes all x satisfying

(8.27)

(8.28)

8.2 BINARY HYPOTHESIS TESTING 311

mizes PMISS, subject to the constraint PFA = a, is

. IxIHo (x)X E Ao if L (x) = ::::y;

IxIHI (x)X E Al otherwise,

where y is chosen so that fL(x)<y IxIHo(X) dx = a.

Proof Using the Lagrange multiplier method, we define the Lagrange multiplier X and the function

(8.29)

fxlHI (x) - VXIHo (x) :5 o. (8.30)

Note that x is found from the constraint PFA = a. Moreover, we observe that Equation (8.~) implies)..> 0; otherwise, !XIHo(x) - )..!XIHI (x) > 0 for all x and AO = ¢, the empty set, would minimizeG. In this case, PFA = 1, which would violate the constraint that PFA = a. Since X > 0, we canrewrite the inequality (8.30) as L (x) ~ 1/).. = y.

.' ~

In the radar system of Example 8.4, the decision statistic was a random variable X andthe receiver operating curves (ROCs) of Figure 8.2 were generated by adjusting a thresholdxo that specified the sets Ao = {X :s xo} and Al = {X > xo}. Example 8.4 did not questionwhether this rule finds the best ROC, that is, the best trade-off between PMlSS and PFA.

The Neyman-Pearson test finds the best ROC. For each specified value of PFA = a, theNeyman-Pearson test identifies the decision rule that minimizes PMlss.

In the Neyman-Pearson test, an increase in y decreases PMlSS but increases PFA. Whenthe decision statistic X is a continuous random vector, we can choose y so that false alarmprobability is exactly a. This may not be possible when X is discrete. In the discrete case,we have the following version of the Neyman-Pearson test.

;:Morem8.5 Discrete Neyman-Pearson TestBased on the decision statistic X, a decision random vector, the decision rule that minimizesPMISS, subject to the constraint PFA :s a, is

A ifL()PXIHo (x)

X E 0 I X = ::::y;PXIHI (x)

where y is the largest possible value such that LL(x)<y PXIHo(x) dx:s a. ----

X E Al otherwise,

Example 8.10 Continuing the disk drive factory test of Example 8.8, design a Neyman-Pearson testsuch that the false alarm probability satisfies PFA :5 a = 0.01. Calculate the resulting

The Neyman-Pearson test is

. PNIHo (n)n E Ao If L(n) = ~ y;

PNIHj (n)

We see from Equation (8.15) that this is the same as the MAP test with P[H1]/ P[Hreplaced by y. Thus, just like the MAP test, the Neyman-Pearson test must be ~threshold test of the form

n E A 1 otherwise. ( .s.

312 CHAPTER 8 HYPOTHESIS TESTING

miss and false alarm probabilities.

n E Ao if n ~ n*; n E Al otherwise. l (~.:Some algebra would allow us to find the threshold n" in terms of the parameter iHowever, this is unnecessary. It is simpler to choose n" directly so that the test meetsthe false alarm probability constraint

PFA = P [N ::: n" - 11Ho] = FNIHo (n* - 1) = 1 - (1 - Qo)n*-1 ::: a. (8.3

This impliesn" < 1 + In(l - a) = 1 + In(0.99) = 101.49. (8.34

- In(l - qa) In(0.9)

Thus, we can choose n* = 101 and still meet the false alarm probability constrainThe error probabilities are:

PFA = P [N s lOOIHo] = 1 - (1 - 10-4)100= 0.00995, (8.3

PMISS = P [N ~ 1011Hd = (1 _10-1)100 = 2.66.10-5. (8.36)

We see that a one percent false alarm probability yields a dramatic reduction in theprobability of a miss. Although the Neyman-Pearson test minimizes neither the overallprobability of a test error nor the expected cost E[C], it may be preferable to either theMAP test or the minimum cost test. In particular, customers will judge the quality of thedisk drives and the reputation of the factory based on the number of defective drivesthat are shipped. Compared to the other tests, the Neyman-Pearson test results in amuch lower miss probability and far fewer defective drives being shipped.

Maximum Likelihood Test

Similar to the Neyman-Pearson test, the maximum likelihood (ML) test is another methodthat avoids the need for a priori probabilities. Under the ML approach, we treat the hy-pothesis as some sort of "unknown" and choose a hypothesis Hi for which P[sIHiJ, theconditional probability of the outcome s given the hypothesis Hi is largest. The idea behindchoosing a hypothesis to maximize the probability of the observation is to avoid makingassumptions about the a priori probabilities P [Hi]. The resulting decision rule, called themaximum likelihood (ML) rule, can be written mathematically as:

Definition 8.1 Maximum Likelihood Decision RuleFor a binary hypothesis test based on the experimental outcome s E S, the maximum

8.2 BINARY HYPOTHESIS TESTING 313

likelihood (ML) decision rule is

S E Ao if P [sIHo] :::: P [sIHl]; s E Al otherwise.

Comparing Theorem 8.1 and Definition 8.1, we see that in the absence of information aboutthe a priori probabilities P [Hi], we have adopted a maximum likelihood decision rule thatis the same as the MAP rule under the assumption that hypotheses Ho and HI occur withequal probability. In essence, in the absence of a priori information, the ML rule assumesthat all hypotheses are equally likely. By comparing the likelihood ratio to a threshold equalto 1, the ML hypothesis test is neutral about whether Ho has a higher probability than HIor vice versa.

When the decision statistic of the experiment is a random vector X, we can express theML rule in terms of conditional PMFs or PDFs, just as we did for the MAP rule.

orem 8.6 If an experiment produces a random vector X, the ML decision rule states

Discrete: X E Ao if PXIHo (x) ::::1;PX.lHl (x)

. . fx.lHo (x)Continuous: x E Ao if > 1;Ixsn, (x) -

X E A 1 otherwise,

X E A 1 otherwise.

Comparing Theorem 8.6 to Theorem 8.4, when X is continuous, or Theorem 8.5, when Xis discrete, we see that the maximum likelihood test is the same as the Neyman-Pearsontest with parameter y = 1. This guarantees that the maximum likelihood test is optimal inthe limited sense that no other test can reduce PMISS for the same PFA.

In practice, we use a ML hypothesis test in many applications. It is almost as effectiveas the MAP hypothesis test when the experiment that produces outcome s is reliable inthe sense that PERR for the ML test is low. To see why this is true, examine the decisionrule in Example 8.6. When the signal-to-noise ratio 2v/0" is high, the threshold (of thelog-likelihood ratio) is close to 0, which means that the result of the MAP hypothesis testis close to the result of a ML hypothesis test, regardless of the prior probability p.

f.~~!!- Continuing the disk drive test of Example 8.8, design the maximum likelihood test for1 the factory state based on the decision statistic N, the number of drives tested up toand including,the first failure.

The ML hypothesis test corresponds to the MAP test with P[Hol = P[Hll = 0.5. Inths icase, Equation (8.16) implies n" = 66.62 or Ao = {n ::::67}. The conditional errorprobabilities under the ML rule are

PFA = P [N ::: 661Ho] = 1 - (1 - 10-4)66 = 0.0066, (8.37)

PMISS = P [N:::: 671Hl] = (1 - 10-1)66 = 9.55 .10-4. (8.38)

For the [AL test, PERR = 0.0060. Comparing the MAP rule with the ML rule, we seethat the prior information used in the MAP rule makes it more difficult to reject the nullhypothesis, We need only 46 good drives in the MAP test to accept Ho, while in the