juror understanding of random match probabilities dale a. nance case western reserve university...

Juror Understanding of Random Match Probabilities

Dale A. NanceCase Western Reserve University

August, 2007

Focus of Presentation

• What we know about how jurors react to testimony reporting a match between the defendant and the perpetrator and presenting a “random match probability” (RMP)

• “Experiments” assessing juror reactions

Eight Common Hypotheses About Cognitive Error by Jurors

• 1. The Prosecutor’s Fallacy• 2. Neglect of Lab Error• 3. Improper Combination Strategies• 4. Vividness• 5. Defense Attorney’s Fallacy• 6. Defense Attorney’s (Extreme) Fallacy• 7. The Inversion Fallacy• 8. Misaggregation

1. The Prosecutor’s Fallacy

“The chance of a coincidental match with an innocent man is 1 in 40,000.”

What the Expert Says

1. The Prosecutor’s Fallacy

“The chance of a coincidental match with an innocent man is 1 in 40,000.”

What the Expert Says

“The chance that the accused in innocent is 1 in 40,000, so the odds that he is guilty must be 39,999 to 1.”

What the Jurors Think

2. Neglect of Lab Error

“The chance of a coincidental match with an innocent man is 1 in 40,000.”

What the Expert Says

2. Neglect of Lab Error

“The chance of a coincidental match with an innocent man is 1 in 40,000.”

What the Expert Says

“The chance that the accused, though innocent, would be implicated by either coincidence or lab error is 1 in 40,000.”

What the Jurors Think

3. Combination Errors (Averaging)

“The chance of a coincidental match with an innocent man is 1 in 40,000.”

“The chance of a false positive lab error is about 1 in 1,000.”

What the Expert Says

3. Combination Errors (Averaging)

“The chance of a coincidental match with an innocent man is 1 in 40,000.”

“The chance of a false positive lab error is about 1 in 1,000.”

What the Expert Says

“The chance that the accused , though innocent, would be implicated by a coincidental match or lab error is 1 in 20,500.”

What the Jurors Think

4. The Vividness Hypothesis

“The chance of a coincidental match with an innocent man is one in a billion.”

What the Expert Says

4. The Vividness Hypothesis

“The chance of a coincidental match with an innocent man is one in a billion.”

What the Expert Says

“One in a billion! That’s all I need to know. Hang the bastard!”

What the Jurors Think

5. The Defense Attorney’s Fallacy

“The chance of a coincidental match with an innocent man is 1 in 40,000. Yes, out of 12,000,000 adult men, about 300 will match.”

What the Expert Says

5. The Defense Attorney’s Fallacy

“The chance of a coincidental match with an innocent man is 1 in 40,000. Yes, out of 12,000,000 adult men, about 300 will match.”

What the Expert Says

“If 300 men will match, then this DNA evidence tells us nothing. I should just decide the case on the eyewitness evidence.”

What the Jurors Think

6. The Defense Attorney’s (Extreme) Fallacy

“The chance of a coincidental match with an innocent man is 1 in 40,000. Yes, out of 12,000,000 adult men, about 300 will match.”

What the Expert Says

6. The Defense Attorney’s (Extreme) Fallacy

“The chance of a coincidental match with an innocent man is 1 in 40,000. Yes, out of 12,000,000 adult men, about 300 will match.”

What the Expert Says

“If 300 men will match, then the chance the accused is guilty must be only 1 in 300.”

What the Jurors Think

7. The Inversion Fallacy

“The chance of a coincidental match with an innocent man is 1 in 40,000.”

What the Expert Says

7. The Inversion Fallacy

“The chance of a coincidental match with an innocent man is 1 in 40,000.”

What the Expert Says

“The chance that the accused in guilty is just 1 in 40,000. This prosecutor must be from Durham.”

\

What the Jurors Think

8. Misaggregation

“The chance of a coincidental match with an innocent man is 1 in 40,000.”

What the Expert Says

8. Misaggregation

“The chance of a coincidental match with an innocent man is 1 in 40,000.”

What the Expert Says

“Without the DNA evidence, I would place the odds of guilt at 2:1 against. With this DNA evidence, the odds of guilt are about 2:1 for.”

What the Jurors Think

8. Misaggregation: How Bad Is It?For a RMP = 1 in 40,000, and considering

only the chance of:

• Coincidental match, posterior odds should be 40,000 times the prior odds:

• Coincidental match or lab error (at a rate of 1 in 1,000), posterior odds should be about 1000 times the prior:

• Coincidental match, lab error, or other sources of error (like police planting of evidence), assessed by the average juror at about 1 in 50, the posterior should be about 40 times the prior:

PRIOR → POST. ODDS ODDS

1:2 → 20,000:1

1:2 → 500:1

1:2 → 20:1

8. Misaggregation:What Can Be Done About it?

• 1. Give RMP testimony in the form of probabilities focused on the defendant, rather than frequencies focused on the population:

– “The probability that defendant would match if he were innocent is 1 in 40,000.”

rather than

– “1 in 40,000 people in the population share this DNA profile.”

8. Misaggregation:What Can Be Done About it?

• 2. Give testimony explaining the RMP by showing results of hypothetical Bayes’ Rule calculations. For example, with RMP= 1 in 40,000 and ignoring other sources of error:

Prior Probability → Posterior Probability 1/10 of 1% → 97.56% 1% → 99.75% 20% → 99.99% 50% → 99.99% 70% → 99.99%

8. Misaggregation:What Can Be Done About it?

• Incorporating information about lab error rates into the calculation produces lower posterior probabilities:

Prior Prob. → Post. Prob. Post. Prob. (ignoring lab error) (incorp. lab

error)

1/10 of 1% → 97.56% 49.42% 1% → 99.75% 90.79% 20% → 99.99% 99.59% 50% → 99.99% 99.90% 70% → 99.99% 99.96%

Conclusions

• Pro-prosecution fallacies: extant but correctible by argument or by restrictions on form of RMP presentation

• Pro-defense fallacies: extant but of declining importance as RMP becomes very small

• Pro-defense error (misaggregation): serious but

potentially amenable to Bayesian instruction