The Mind of the User

When 98% is more than 100%: How number format affects judgment & decisions

Colleen Roller

UX Matters Columnist

On Decision Architecture

Do you know…

How well do people understand and interpret numeric data?

Does the format of numeric data have an impact?

How should we display numeric data?

This presentation

Reviews research that reveals how

people perceive and use data

Suggests UX design principles and

best practices

A study

Rate the attractiveness of a simple gamble (on a scale from 0-20)

1st group (rating = 9.4)

7 out of 36 chance to win $9

2nd group (rating = 14.9)

7 out of 36 chance to win $9

29 out of 36 chance to lose 5 cents

Bateman/Dent/Peters/Slovic/Starmer -

2007

Judgment & decision making

Must be able to attach meaning to data

Meaning is determined via

Context – a reference point or upper/

lower bound

Comparison – against the context

The ease with which meaning can be derived

dictates the extent to which data will be used.

A study

Which bowl did people prefer:

A. 100 beans, 7 of which are red

B. 10 beans, 1 of which is red

Denes-Raj/Epstein - 1994

A study

Which disease is more dangerous?

A. Kills 1286 out of every 10,000 people

B. Kills 24 out of every 100 people

Yamagishi 1997

A study

Will a mental patient commit a violent

act within 6 months of being

discharged from the hospital?

A. 20 out of every 100 patients commit a

violent act (41% refused to discharge)

B. 20% chance that the patient will commit

a violent act (21% refused to discharge)

Slovic/Monahan/MacGregor - 2000

Quiz

Which is more easily understood:

A. A 30% chance of rain

B. A 3 in 10 chance of rain

30% chance of rain

Common misinterpretations:

It will rain in 30% of the area

It will rain 30% of the time

It will rain on 30% on days like this

Gigerenzer/Edwards - 2003

Problems with probability

What the doctor said:

You have a 30% - 50% chance of

developing a sexual problem

What the patient heard:

In 30% - 50% of your sexual encounters,

something will go awry

Gigerenzer/Edwards - 2003

Summary

People tend to

Comprehend frequencies better than

probability/percent

20 out of 100, rather than 20%

Focus on the numerator (and ignore the

denominator)

9 out of 100 is bigger than 1 out of 10

Different expressions of equivalent data -

e.g., 30 out of 1000 is more than 3 out of 10

A study

Purchase equipment for use in the

event of an airline crash landing:

A. Chance of saving 150 lives

B. Chance of saving 98% of 150 lives

Slovic et al. - 2002

85% is more than 100%

Even 85% of 150 is more than “150”!

Slovic et al. - 2002

Calculations

Probability

1% of car trips result in an

accident. In 55% of the

trips that result in an

accident, the driver is

drunk. In 5% of the car

trips that do not result in

an accident, the driver is

drunk. If the driver is

drunk, what is the

probability of an accident?

Frequency

100 out of 10,000 car trips

result in an accident.

Among the 100 trips that

result in an accident, the

driver is drunk in 55 of

them. Among the 9900 car

trips that don’t result in an

accident, the driver is drunk

in 500 of them. How many

car trips where the driver is

drunk result in an accident?

Study: Rates of return

Allocate money across two investment

funds – stocks v. bonds

Group A: shown 1-yr rates of return 63%

Group B: shown 30-yr rates of return 81%

Benartzi/Thaler, 2001

Data on the page

Versus

Hibbard/Slovic/Peters/Finucane - 2002

61%

75%

No neutral design

What information – and how it is

presented – drives decision outcomes

Numeric ability

Almost half of the general population

has difficulty with simple numeric tasks

National Adult Literacy Survey

Numeracy

Those who are numeric

Readily understand and use numeric data effectively

Those who are non-numeric

Informed less by numbers, and more by other non-numeric sources of info

Number format

Best Practices

Determine the right criteria

Determine what decision criteria

people should be using

Highlight them (salience)

Make it easy to evaluate, compute, &

attach meaning

Frequency v. probability

Convey absolute risks over relative

risks

3 out of 1000 will have a stroke

is better than

50% higher chance of stroke

Don’t use decimals (.03)

Apples to apples comparisons

When presenting various probabilities,

keep the denominator consistent 20 out of 1000 compared to 1 out of 1000

is better than

1 out of 50 compared to 1 out of 1000

Attach meaning

Use labels to show standard of

performance

Example: unacceptable, acceptable,

excellent

Labels provide expert guidance and easy

mental processing

Mapping

A higher number means better quality

Reduces cognitive load

Subtle, but influential

Quiz

Which one results in better

comprehension and better choices:

A. Number of patients per registered nurse

B. Number of registered nurses per 100

patients

Variety of visual display

Consecutive v. random

Schapira/Nattinger/McHorney - 2001

Placement of solid dots

Conveying small risks

To help people understand the

meaning of small risks

Show context by providing a range of

probabilities and risks for comparison

Example: being hit by a car v. x-rays v.

lighting v. asbestos

Emotion v. probability

When consequences are marked by

strong emotion

All or none – sensitive to the possibility

rather than the probability

Final thoughts

Determine what info is most important

What should people base the decision on?

Design for meaning and ease

Via context and comparison

Facilitate easy computation

Test – multiple methods

Test drive, A/B testing, website metrics,

think aloud, observe/probe in usability testing

Questions/discussion

Colleen RollerUX Matters Columnist

on Decision Architecture

http://uxmatters.com/

Reference articles

Simple Tools for Understanding Risks: From Innumeracy to Insight – G.

Gigerenzer & A. Edwards, 2003

Numeracy and Decision Making – E. Peters et al., 2006

Numeracy and the Perception and Communication of Risk – E. Peters, 2008

Strategies for Reporting Health Plan Performance Information to Consumers:

Evidence from Controlled Studies – J. Hibbard, et al., 2002

Risk as Analysis and Risk as Feelings: Some Thoughts about Affect, Reason,

Risk, and Rationality – P. Slovic et al., 2004

Numeracy Skill and the Communication, Comprehension, and Use of Risk-

Benefit Information – E. Peters et al., 2007

Reducing the Influence of Anecdotal Reasoning on People’s Health Care

Decisions: Is a Picture Worth a Thousand Statistics? - A. Fagerlin et al., 2005

Bringing Meaning to Numbers: The Impact of Evaluative Categories on

Decisions – E. Peters et al., 2009

When a 12.86% Mortality is More Dangerous than 24.14%: Implications for

Risk Communication – K Yamagishi, 1997