thinking with probability - trinity college dublin week... · 2013-02-15 · thinking with...

Thinking with Probability

Thinking – fast and slow

Thinking by decomposing events

formalising probability rules

conditional probability

Brain Teasers Tijms Problems, Chap 1

15/02/2013 ST2352 2013 Week 6 1

2 dice rolled; one is 6. What prob other is 6?

Aged 22 Prob of death

2 babies dead. Pr(mother murderer)?

Monty Hall; Choose 1; Learn goat at 2. Pr(car at 3)?

Thinking with Probability Thinking about systems by decomposing events

Formalising probability rules

conditional probability

Forward modelling A makes B more likely

Inverse thinking Have observed B; Pr(A|B)

Evaluating evidence Bayes Rule

Using evidence in prediction Statistical Inference

15/02/2013 ST2352 2013 Week 6 2

Evaluating Evidence with Probs

How does evidence lead to conclusions in situations of uncertainty? Bayes Theorem

Data fusion, use of techniques that combine data from multiple sources and gather that information in order to achieve inferences, which will be more efficient and potentially more accurate than if they were achieved by means of a single source.

Spam

Cancer Screening

Law

…..

3 ST2352 2013 Week 6 15/02/2013

All Probs are Conditional

Probability

Measure of uncertainty about event given info Pr(A wins league, given probs for each match, and indep)

Pr(A wins league, given that somebody wins, and probs, and indep)

Pr(A wins league, given no info, other than probs, and indep)

Pr info(event ) Pr(event | info)

Simple Forward Theory

Subtle Inverse Inference 15/02/2013 ST2352 2013 Week 6 4

Bayes Rule

ST2352 2013 Week 6 5

Pr( ) Pr( | ) Pr( ) Pr( ) Pr( | ) Pr( )

Pr( )Pr( | ) Pr( | )

Pr( )

Aand B A B B B and A B A A

BB A A B

A

Inverting the Conditioning

Multiple Possibilities

1 2

1 1

................

Pr( | )Pr( )Pr( | )

Pr( | )Pr( ) ....... Pr( | )Pr( )

n

i ii

n n

B B OR B OR B

A B BB A

A B B A B B

15/02/2013

Inversion

15/02/2013 ST2352 2013 Week 6 6

2 dice rolled; one is 6. What prob other is 6? 2 dice rolled; first is 6. What prob other is 6?

Serious

Sally Clarke - Sudden Infant Death SID

The case was widely criticised because of the way statistical evidence was misrepresented in the original trial, particularly by Meadow. He stated in evidence as an expert witness that "one sudden infant death in a family is a tragedy, two is suspicious and three is murder unless proven otherwise" (Meadow's law). He claimed that, for an affluent non-smoking family like the Clarks, the probability of a single cot death was 1 in 8,543, so the probability of two cot deaths in the same family was around "1 in 73 million" (8543 × 8543).

ST2352 2013 Week 6 7 15/02/2013

Light

Metro Wed 10 Nov 2010

Teens at risk from hyper-texting

Teenagers who send more than 100 text messages per day are more likely to have had sex, tried drugs, research has revealed.

4200 students at 20 schools; hyper-texting 19.2%

Such teens 43% more likely to have tried alcohol.

ST2352 2013 Week 6 8 15/02/2013

Brain Teasers

Monty Hall Game Show (Tijms, Ch 1, Q11)

One car, behind one of three doors.

Player selects one: say Door 1

Before opening this door

host opens one of two others: say Door 2 GOAT!

host offers chance to change selection.

Issue Is there any point changing?

Evidence in favour of (stay with) chosen door

ST2352 2013 Week 6 9 15/02/2013

ST2352 2013 Week 6 10

Serious Life Expectancy in Ireland

Average age at death = 75

Average age at death = 80

Average age at death, given survival to 60, = 79

http://understandinguncertainty.org/node/272

15/02/2013

ST2352 2013 Week 6 11

Cond Prob for Lifetimes Knowledge of current age impacts uncertainty on age at death

Probability Distribution Poss LiveTimes 1 2 3 4 5 6 Corresp Probs 0.1 0.2 0.3 0.3 0.05 0.05

Pr(death at end day 3, given alive at start day 3)

15/02/2013

ST2352 2013 Week 6 12

Cond Prob for Lifetimes Knowledge of current age impacts uncertainty on age at death

Expt: Choose random component

Poss Lives 1 2 3 4 5 6 Probs 0.1 0.2 0.3 0.3 0.05 0.05

Pr(death at end day 3) = 0.30

Pr(alive at start day 3) = 0.70

Pr(death at end day 3, given alive at start day 3)

Pr(death 'at' 3 AND alive 'at' 3) Pr(death 'at' 3)= =

Pr(ali

What event

ve 'at'

ident ?

3)

ity

P

0.30.43

r(alive 'at' 3) 0.7

Pr(death end day , given alive day 3)

0.3 0.3 0.05 0.05, , , for 3, 4,5,6

0.7 0.7 0.7 0.7

k

k resp

15/02/2013

Odds and Weight of Evidence

ST2352 2013 Week 6 13

Pr(in favour of )

Pr

Pr1

Twice as likely

Home Advantage

Risk Factor

AOdds A

A

Odds A

15/02/2013

Odds and Weight of Evidence

ST2352 2013 Week 6 14

Odds Rule Form for Evidence

Pr( ) Pr( )Pr( | ) Pr( | ) ; Pr( | ) Pr( | )

Pr( ) Pr( )

Pr( | ) Pr( | ) Pr( )

Pr( | ) Pr( | ) Pr( )

A AA E E A A E E A

E E

A E E A A

A E E A A

15/02/2013

Weight of Evidence

ST2352 2013 Week 6 15

12

12 1

2

13

Pr( | ) Pr( | ) Pr( )

Pr( | ) Pr( | ) Pr( )

Car behind door chosen (eg 1);

Host opens door (eg 2)

Prior Odds(A) =

Pr( | ) Pr( | )

Posterior Odds(A)1

Posterior Pr(A) =

A E E A A

A E E A A

A A

E

E A E A

15/02/2013

Weight of Evidence

ST2352 2013 Week 6 16

12

12 1

2

13

Pr( | ) Pr( | ) Pr( )

Pr( | ) Pr( | ) Pr( )

Car behind door chosen (eg 1);

Host opens door (eg 2)

Prior Odds(A) =

Pr( | ) Pr( | )

Posterior Odds(A)1

Posterior Pr(A) =

A E E A A

A E E A A

A A

E

E A E A

15/02/2013

Brain Teasers

• Who is the murderer? (Tijms, Ch 1 Q6)?

Murder committed; know either X or Y – equally likely.

Evidence: actual perp has blood group A

10% of people group A; X is group A

Seek Pr( X is perp | evidence)

ST2352 2013 Week 6 17 15/02/2013

Brain Teasers

Who is the murderer? (Tijms, Ch 1 Q6)?

Murder committed; know either X or Y – equally likely.

Evidence: actual perp has blood group A

10% of people group A; X is group A

Seek Pr( X is perp | evidence)

ST2352 2013 Week 6 18

Define murderer

Define Blood group A left at crime scene

Pr( | )Pr( )Pr( | ) Pr( | )Pr( )

Pr( )

Pr( ) ? Pr( ) ?

Pr( | ) ? Pr( | ) ?

Pr( | ) ?

H X

E

E H HH E E H H

E

H H

E H E H

H E

Events are T /F

15/02/2013

Brain Teasers

Who is the murderer? (Tijms, Ch 1 Q6)?

Murder committed; know either X or Y – equally likely.

Evidence: actual perp has blood group A

10% of people group A; X is group A

Seek Pr( X is perp | evidence)

ST2352 2013 Week 6 19

Define murderer

Define Blood group A left at crime scene

Pr( | )Pr( )Pr( | ) Pr( | )Pr( )

Pr( )

Pr( ) Pr( ) 0.5

Pr( | ) 1; Pr( | ) 0.1

1 0.5 10Pr( | )

1 0.5 0.1 0.5 11

H X

E

E H HH E E H H

E

H H

E H E H

H E

15/02/2013

Weight of Evidence

ST2352 2013 Week 6 20

Evidence Fusion

1 1

1 1

Pr( | ..... ) Pr( | )Pr( | ) Pr( )....

Pr( | ..... ) Pr( | ) Pr( | ) Pr( )

if evidence indep given ,

n n

n n

A E AND AND E E AE A A

A E AND AND E E A E A A

A A

15/02/2013

Serious

Sally Clarke - Sudden Infant Death SID

He claimed that, for an affluent non-smoking family like the Clarks, the probability of a single cot death was 1 in 8,543, so the probability of two cot deaths in the same family was around "1 in 73 million" (8543 × 8543).

ST2352 2013 Week 6 21

2

18543

Pr(2 Cot Deaths | Normal Family) = Pr(Normal Family | 2 Cot Deaths)

Pr(Normal | 2 Deaths) Pr(Normal ) Pr(2 Deaths | Normal)=

Pr(Not normal | 2 Deaths) Pr(Not normal ) Pr(2 Deaths | Not normal)

15/02/2013

Serious

Sally Clarke - Evidence Fusion

ST2352 2013 Week 6 22

Pr(Normal | 2 Deaths) Pr(Normal ) Pr(1 Death | Normal)=

Pr(Not normal | 2 Deaths) Pr(Not normal ) Pr(1 Death | Not normal)

Pr(2 Death | Normal) Pr(2 Death | Not normal)

Pr(2 Deaths | Normal) =

nd

nd

Formally

Pr(2 Death | 1 Death, Normal) Pr(1 Death | Normal)nd st

15/02/2013

Background

• Overlaps completely with ST2351

15/02/2013 ST2352 2013 Week 6 23

Decomposition via Conditional Probs

ST2352 2013 Week 6 24

13

12 2

1

1 2

3

13

13

12

16

16

1

1

3

2

3

13

13

Contestantchooses door1, for example

Car in factbehind each door with equal prob

As car behind door 1quiz master opens 2 or 3 with equal prob

If car behind door 2quiz master MUST open 3 ie prob = 1; sim’ly if behind door 3

3

2

2

31

1

1

11

6

16

13

13

Car

Car

Goat

Goat

Contestantdoes not switch; remains with door 1

Car

Car

Goat

Goat

Contestantswitches with prob 1 to Door:

Prob wins =2

31

3

15/02/2013

Inversion

15/02/2013 ST2352 2013 Week 6 25

Survey of travellers in US finds: 20% have been to Europe; 15% have Amex card and have been to Europe; 55% have neither; What %age of Amex card holders have been to Europe?

Y N Y N

Y 15% Y 15

N 15% N 55

15% 20 100

Europe

Amex

Europe

Amex

Pr | Pr |A E E A

Inversion

Inverse Theory Prob ( 1st Q, given 2nd Q)

15/02/2013 ST2352 2013 Week 6 26

2 1 2 1 1

1 2

2 2

2 2 1 1 2 1 1

Pr Pr | PrPr |

Pr Pr

RecallPr Pr | Pr Pr | Pr

Q AND Q Q Q QQ Q

Q Q

Q Q Q Q Q Q Q

Queens

Inversion

15/02/2013 ST2352 2013 Week 6 27

Y N Y N

Y 15% Y 15

N 15% N 55

15% 20 100

Europe

Amex

Europe

Amex

Pr Pr | Pr

Pr |Pr Pr | Pr Pr | Pr

A E A EAN EE A

A A

D

E E A E E

Event Decomposition

ST2352 2013 Week 6 28

( )

( )

Pr( )

OR

AND

B B Certain Event

A A B B AOR AND OR ANDB A B

A

15/02/2013

1 2

1 2

1 2

................

.......

More generally

.........

...

Pr( )

n

n

n

B OR B OR B Certain Event

A A AND A AND B OR B OR B

A AND B OR A AND B OR A AND B

A

Event Identities Probability Monte Hall

15/02/2013 ST2352 2013 Week 6 29

Choose Correct Door Choose Incorrect Door Certain Event

Win by Staying Choose Correct Door

Win by Switching Choose Incorrect Door

Pr Win by Staying

OR

Pr( ) Pr( ) Pr( ) Pr( )

Important special case

Pr( ) Pr( ) Pr( )

OR AND

O

A B A B A B

A B A B whendis oi tR j n

Addition Rule

Probability Rules Conditional Prob and Independence

1 2

1 2

Pr( ) Pr( | ) Pr( )

Important special case

Pr( ) Pr( ) Pr( )

Pr 2 2

Pr 2 2

A B A B B

A B A B when independent

Die Di

AND

AND

AND

AND

e

Die S

Multiplication Rule

30 ST2352 2013 Week 6

All computed probs: need real world knowledge assumptions about real world Sometimes useful to be explicit what - if

15/02/2013

Chain Rule: Forward Theory

Extension Chain Rule

Pr Pr( ) | )Pr( )

Pr(

Pr( )Pr( ) P

| ) Pr( | )Pr( | )

Pr Pr( | )Pr( | )Pr(

r( | ) Pr( ) Pr( | )Pr( )

Pr( , , ) Pr( | , )Pr( | )P (

)

r )

A AND B ANDC A AND B C C

But A AND B C A B AND

Aand BAand B A B B A B

B

A

C B C

Thus A AND B ANDC A B ANDC

B C

B

A B C B C C

Sp

C C

Pr( )Pr( )Pr( )ecial case if events probabilistically iA B C ndep

15/02/2013 ST2352 2013 Week 6 31

Forward Theory: Decompose

Cards Prob 1st two cards Queen?

By count

By cond prob

Inverse Theory Prob ( 1st Q, given 2nd Q)

15/02/2013 ST2352 2013 Week 6 32

1 2

1 2 2 1 1

,

Pr

Defin

Pr | P

e

r

Q Q

Q ANDQ Q Q Q

Queens

Simpler than combinatorics?

Cards Prob 2nd card is Queen?

Event Identity

15/02/2013 ST2352 2013 Week 6 33

2 2

2whence Pr

Q Q AND

Q

Forward Theory: Decompose

ST2352 2013 Week 6 34

nd2

1

2

Regular pack of cards; no replacement

= 2 card is Q

by consideri

Focus

Decompose

Event Identit

ng

Pr( )

y

Q

Q

mpute QCo

Chance Tree

Simulation equiv?

15/02/2013

Forward Theory: Decompose

Queens

Password

– Draw 8 chars, unif at random, ABC…XYZ123..89

– Reject entire password if any dups within 8 • Alt, reject char if same as any prev accepted char

– What is prob ‘reject password’ (long run prop)

15/02/2013 ST2352 2013 Week 6 35

Forward Theory: Decompose

ST2352 2013 Week 6 36

Password 8 from ABC…XYZ123..89 (unif@rand, w/repl)

– Reject entire password if any dups within 8

8

7 6Pr 8

28.29.30.31.32.33.34.350.42

35

Pr 8 0.58

N N NAccept

N N N

Reject

15/02/2013

Forward Theory: Decompose

40

325.326........365Pr(at least one common birthday) =1-

365

1 0.097 0.903

ST2352 2013 Week 6 37

Password 8 from ABC…XYZ123..89 (unif@rand, w/repl)

– Reject entire password if any dups within 8

– Prob ‘reject password’ Easier: Prob accept

8

8 8 7 1

8 8 7 1

8 7 1 7 6 1 1

8 7 7 5 1

Accept first chars from alphabet of

Seek Pr

Event Identity Password OK = ...

Pr Pr ...

Pr | ... Pr | ... ... Pr

Pr | Pr | ...

Def

7

i

Pr

nek

AND AND

A

A k N

NOT A

A A A A

A A A A

A A A A A A A

ND AN

A A A A A

D

AND AN

N

D

N

6N N

N N

15/02/2013

Birthday problem

Forward Theory: Decompose

ST2352 2013 Week 6 38

Password 8 from ABC…XYZ123..89 (unif@rand, w/repl)

– Reject entire password if any dups within 8

– Prob ‘reject password’ Easier: Prob accept

8 8 7 8 7 1

8 8 7 1

8 7 1 6 7 1 1

8

Accept first chars;

Event Identity Password OK = ...

Pr Pr ...

Pr | ... Pr | ... ... Pr

7 6

Define

Pr

k

AND AND AND

AND AND

AND AN

A k

A A A A A A

A A A A

A A A A A A A

N N N

N N N

NOT A

D

15/02/2013

Birthday problem

Forward Theory: Decompose

ST2352 2013 Week 6 39

1

1...6 1 1

1

Poss values of are 1,2, 6

Pr ; 0, 6( 1)

in terms of

k

k k

k s s

k k k

y k k

k k

s

k

s

AN

S s k

S s p p s k

S s S s Y anyval

Y y S s

D

OR A y

p p

N

p

D

15/02/2013

Forward Recursion by Decomp: Dice

16

Poss values of

1,2, 6;

Pr

k

k

Y

y

Y y

Conditional Independence

15/02/2013 ST2352 2013 Week 6 40

1 1 1 1

1 1 1 1

Formally Pr , | Pr | Pr |

Notation ,

Generalisation of Pr Pr Pr

Example Markov Chain

Pr | Pr | Pr | ,

Pr , | Pr | Pr |

t t t t t t

t t t t t t t

A B C A C B C

A B A AND B

A AND B A B

Y past Y Y Y Y Y

Y Y Y Y Y Y Y

SubsysC

using RAND()C

SubsysA

using RAND()A

SubsysB

using RAND()B

Randomness in AC,BC dependent; eg AC ‘large’ BC ‘large’ (probably) Randomness in AC, BC cond indep, if we know the value of C