probability theor y & uncer tainty

Post on 24-Dec-2021

4 Views

Category:

Documents

0 Downloads

Preview:

Click to see full reader

TRANSCRIPT

AI: 15-780 / 16-731Mar 1, 2007

Probability Theory & UncertaintyRead Chapter 13 of textbook

Michael S. Lewicki ! Carnegie MellonAI: Probability Theory ! Mar 1, 2007

What you will learn today

• fundamental role of uncertainty in AI

• probability theory can be applied to many of these problems

• probability as uncertainty

• probability theory is the calculus of reasoning with uncertainty

• probability and uncertainty in different contexts

• review of basis probabilistic concepts

- discrete and continuous probability

- joint and marginal probability

- calculating probability

• next probability lecture: the process of probabilistic inference

2

Michael S. Lewicki ! Carnegie MellonAI: Probability Theory ! Mar 1, 2007 3

What is the role of probability and inference in AI?

• Many algorithms are designed as if knowledge is perfect, but it rarely is.

• There are almost always things that are unknown, or not precisely known.

• Examples:

- bus schedule

- quickest way to the airport

- sensors

- joint positions

- finding an H-bomb

• An agent making optimal decisions must take into account uncertainty.

Michael S. Lewicki ! Carnegie MellonAI: Probability Theory ! Mar 1, 2007

Probability as frequency: k out of n possibilities

• Suppose we’re drawing cards from a standard deck:

- P(card is the Jack ♥ | standard deck) = 1/52

- P(card is a ♣ | standard deck) = 13/52 = 1/4

• What’s the probability of a drawing a pair in 5-card poker?

- P(hand contains pair | standard deck) =

# of hands with pairs_______________

total # of hands

- Counting can be tricky (take a course in combinatorics)

- Other ways to solve the problem?

• General probability of event given some conditions:

P(event | conditions)

4

Michael S. Lewicki ! Carnegie MellonAI: Probability Theory ! Mar 1, 2007 5

Making rational decisions when faced with uncertainty

• Probability

the precise representation of knowledge and uncertainty

• Probability theory

how to optimally update your knowledge based on new information

• Decision theory: probability theory + utility theory

how to use this information to achieve maximum expected utility

• Consider again the bus schedule. What’s the utility function?

- Suppose the schedule says the bus comes at 8:05.

- Situation A: You have a class at 8:30.

- Situation B: You have a class at 8:30, and it’s cold and raining.

- Situation C: You have a final exam at 8:30.

Michael S. Lewicki ! Carnegie MellonAI: Probability Theory ! Mar 1, 2007

Probability of uncountable events

• How do we calculate probability that it will rain tomorrow?

- Look at historical trends?

- Assume it generalizes?

• What’s the probability that there was life on Mars?

• What was the probability the sea level will rise 1 meter within the century?

• What’s the probability that candidate X will win the election?

6

Michael S. Lewicki ! Carnegie MellonAI: Probability Theory ! Mar 1, 2007

The Iowa Electronic Markets: placing probabilities on single events

• http://www.biz.uiowa.edu/iem/

• “The Iowa Electronic Markets are real-money futures markets in which contract payoffs depend on economic and political events such as elections.”

• Typical bet: predict vote share of candidate X - “a vote share market”

7

Michael S. Lewicki ! Carnegie MellonAI: Probability Theory ! Mar 1, 2007

Political futures market predicted vs actual outcomes

8

Michael S. Lewicki ! Carnegie MellonAI: Probability Theory ! Mar 1, 2007

John Craven and the missing H-Bomb

• In Jan. 1966, used Bayesian probability and subjective odds to locate H-bomb missing in the Mediterranean ocean.

9

Michael S. Lewicki ! Carnegie MellonAI: Probability Theory ! Mar 1, 2007

Probabilistic Methodology

10

0, 1, or 2 parachutes open?

type of collision

prevailing wind direction

Michael S. Lewicki ! Carnegie MellonAI: Probability Theory ! Mar 1, 2007

Clim

atic U

nce

rta

inty

Clim

atic U

nce

rta

inty

Fo

rest

et

al.

, 2

00

1F

ore

st e

t al.

, 2

00

1

Probabilistic assessment of dangerous climate change

11

from Forrest et al (2001)

from Mastrandrea and Schneider (2004)

Michael S. Lewicki ! Carnegie MellonAI: Probability Theory ! Mar 1, 2007

Factoring in Risk Using Decision Theory

12

Step 1Step 1

P(“DAI” = 55.8%)

Step 2Step 2

P(“DAI” = 27.4%Carbon Tax 2050 = $174/Ton

Dangerous Climate Change

Michael S. Lewicki ! Carnegie MellonAI: Probability Theory ! Mar 1, 2007

Uncertainty in vision: What are these?

13

Michael S. Lewicki ! Carnegie MellonAI: Probability Theory ! Mar 1, 2007

Uncertainty in vision

14

Michael S. Lewicki ! Carnegie MellonAI: Probability Theory ! Mar 1, 2007

Edges are not as obvious they seem

15

Michael S. Lewicki ! Carnegie MellonAI: Probability Theory ! Mar 1, 2007

An example from Antonio Torralba

16

What’s this?

Michael S. Lewicki ! Carnegie MellonAI: Probability Theory ! Mar 1, 2007

We constantly use other information to resolve uncertainty

17

Michael S. Lewicki ! Carnegie MellonAI: Probability Theory ! Mar 1, 2007

Image interpretation is heavily context dependent

18

Michael S. Lewicki ! Carnegie MellonAI: Probability Theory ! Mar 1, 2007

This phenomenon is even more prevalent in speech perception

19

• It is very difficult to recognize phonemes from naturally spoken speech when they are presented in isolation.

• All modern speech recognition systems rely heavily on context (as do we).

• HMMs model this contextual dependence explicitly.

• This allows the recognition of words, even if there is a great deal of uncertainty in each of the individual parts.

Michael S. Lewicki ! Carnegie MellonAI: Probability Theory ! Mar 1, 2007

De Finetti’s definition of probability

• Was there life on Mars?

• You promise to pay $1 if there is, and $0 if there is not.

• Suppose NASA will give us the answer tomorrow.

• Suppose you have an oppenent

- You set the odds (or the “subjective probability”) of the outcome

- But your oppenent decides which side of the bet will be yours

• de Finetti showed that the price you set has to obey the axioms of probability or you face certain loss, i.e. you’ll lose every time.

20

Michael S. Lewicki ! Carnegie MellonAI: Probability Theory ! Mar 1, 2007 21

Axioms of probability

• Axioms (Kolmogorov):

0 ! P(A) ! 1

P(true) = 1

P(false) = 0

P(A or B) = P(A) + P(B) " P(A and B)

• Corollaries:

- A single random variable must sum to 1:

- The joint probability of a set of variables must also sum to 1.

- If A and B are mutually exclusive:

P(A or B) = P(A) + P(B)

n∑

i=1

P (D = di) = 1

Michael S. Lewicki ! Carnegie MellonAI: Probability Theory ! Mar 1, 2007

Rules of probability

• conditional probability

• corollary (Bayes’ rule)

22

Pr(A|B) =Pr(A andB)

Pr(B), P r(B) > 0

Pr(B|A)Pr(A) = Pr(A andB) = Pr(A|B)Pr(B)

⇒ Pr(B|A) =Pr(A|B)Pr(B)

Pr(A)

Michael S. Lewicki ! Carnegie MellonAI: Probability Theory ! Mar 1, 2007

Discrete probability distributions

• discrete probability distribution

• joint probability distribution

• marginal probability distribution

• Bayes’ rule

• independence

23

Michael S. Lewicki ! Carnegie MellonAI: Probability Theory ! Mar 1, 2007 24

!"

#$%&'()*+,-,!.."/,012'34,56,7$$'3 8'$9:9(;(<+(=,01:;&+(=<>,?;(23,@"

A*3,B$(1+,C(<+'(9D+($1

E3=(%3,F$',G:H(1),:,I$(1+,2(<+'(9D+($1,$F,7,J:'(:9;3<>

"6 7:H3,:,+'D+*,+:9;3,;(<+(1),:;;,=$G9(1:+($1<,$F,J:;D3<,$F,&$D',J:'(:9;3<,K(F,+*3'3,:'3,7,L$$;3:1,J:'(:9;3<,+*31,+*3,+:9;3,4(;;,*:J3,!7,'$4<M6

!6 N$',3:=*,=$G9(1:+($1,$F,J:;D3</,<:&,*$4,%'$9:9;3,(+,(<6

O6 PF,&$D,<D9<='(93,+$,+*3,:Q($G<,$F,%'$9:9(;(+&/,+*$<3,1DG93'<,GD<+,<DG,+$,"6

RQ:G%;3>,L$$;3:1,J:'(:9;3<,0/,L/,#

.6"."""

.6!S.""

.6"."."

.6.S.."

.6.S"".

.6"..".

.6.S"..

.6O....

!"#$%&'

0

L

#.6.S.6!S

.6". .6.S.6.S

.6".

.6"..6O.

#$%&'()*+,-,!.."/,012'34,56,7$$'3 8'$9:9(;(<+(=,01:;&+(=<>,?;(23,@!

T<(1),+*3,B$(1+

U13,&$D,*:J3,+*3,BC,&$D,=:1,:<H,F$',+*3,%'$9:9(;(+&,$F,:1&,;$)(=:;,3Q%'3<<($1,(1J$;J(1),&$D',:++'(9D+3

!"!

"!"

!"#$%&'()!*+,-

.*+,/./

All the nice looking slides like this one from now on are from Andrew Moore.

Michael S. Lewicki ! Carnegie MellonAI: Probability Theory ! Mar 1, 2007 25

!"

#$%&'()*+,-,!.."/,012'34,56,7$$'3 8'$9:9(;(<+(=,01:;&+(=<>,?;(23,@"

A*3,B$(1+,C(<+'(9D+($1

E3=(%3,F$',G:H(1),:,I$(1+,2(<+'(9D+($1,$F,7,J:'(:9;3<>

"6 7:H3,:,+'D+*,+:9;3,;(<+(1),:;;,=$G9(1:+($1<,$F,J:;D3<,$F,&$D',J:'(:9;3<,K(F,+*3'3,:'3,7,L$$;3:1,J:'(:9;3<,+*31,+*3,+:9;3,4(;;,*:J3,!7,'$4<M6

!6 N$',3:=*,=$G9(1:+($1,$F,J:;D3</,<:&,*$4,%'$9:9;3,(+,(<6

O6 PF,&$D,<D9<='(93,+$,+*3,:Q($G<,$F,%'$9:9(;(+&/,+*$<3,1DG93'<,GD<+,<DG,+$,"6

RQ:G%;3>,L$$;3:1,J:'(:9;3<,0/,L/,#

.6"."""

.6!S.""

.6"."."

.6.S.."

.6.S"".

.6"..".

.6.S"..

.6O....

!"#$%&'

0

L

#.6.S.6!S

.6". .6.S.6.S

.6".

.6"..6O.

#$%&'()*+,-,!.."/,012'34,56,7$$'3 8'$9:9(;(<+(=,01:;&+(=<>,?;(23,@!

T<(1),+*3,B$(1+

U13,&$D,*:J3,+*3,BC,&$D,=:1,:<H,F$',+*3,%'$9:9(;(+&,$F,:1&,;$)(=:;,3Q%'3<<($1,(1J$;J(1),&$D',:++'(9D+3

!"!

"!"

!"#$%&'()!*+,-

.*+,/./

Michael S. Lewicki ! Carnegie MellonAI: Probability Theory ! Mar 1, 2007 26

!"

#$%&'()*+,-,!.."/,012'34,56,7$$'3 8'$9:9(;(<+(=,01:;&+(=<>,?;(23,@"

A*3,B$(1+,C(<+'(9D+($1

E3=(%3,F$',G:H(1),:,I$(1+,2(<+'(9D+($1,$F,7,J:'(:9;3<>

"6 7:H3,:,+'D+*,+:9;3,;(<+(1),:;;,=$G9(1:+($1<,$F,J:;D3<,$F,&$D',J:'(:9;3<,K(F,+*3'3,:'3,7,L$$;3:1,J:'(:9;3<,+*31,+*3,+:9;3,4(;;,*:J3,!7,'$4<M6

!6 N$',3:=*,=$G9(1:+($1,$F,J:;D3</,<:&,*$4,%'$9:9;3,(+,(<6

O6 PF,&$D,<D9<='(93,+$,+*3,:Q($G<,$F,%'$9:9(;(+&/,+*$<3,1DG93'<,GD<+,<DG,+$,"6

RQ:G%;3>,L$$;3:1,J:'(:9;3<,0/,L/,#

.6"."""

.6!S.""

.6"."."

.6.S.."

.6.S"".

.6"..".

.6.S"..

.6O....

!"#$%&'

0

L

#.6.S.6!S

.6". .6.S.6.S

.6".

.6"..6O.

#$%&'()*+,-,!.."/,012'34,56,7$$'3 8'$9:9(;(<+(=,01:;&+(=<>,?;(23,@!

T<(1),+*3,B$(1+

U13,&$D,*:J3,+*3,BC,&$D,=:1,:<H,F$',+*3,%'$9:9(;(+&,$F,:1&,;$)(=:;,3Q%'3<<($1,(1J$;J(1),&$D',:++'(9D+3

!"!

"!"

!"#$%&'()!*+,-

.*+,/./

Michael S. Lewicki ! Carnegie MellonAI: Probability Theory ! Mar 1, 2007

!"

#$%&'()*+,-,!.."/,012'34,56,7$$'3 8'$9:9(;(<+(=,01:;&+(=<>,?;(23,@"

A*3,B$(1+,C(<+'(9D+($1

E3=(%3,F$',G:H(1),:,I$(1+,2(<+'(9D+($1,$F,7,J:'(:9;3<>

"6 7:H3,:,+'D+*,+:9;3,;(<+(1),:;;,=$G9(1:+($1<,$F,J:;D3<,$F,&$D',J:'(:9;3<,K(F,+*3'3,:'3,7,L$$;3:1,J:'(:9;3<,+*31,+*3,+:9;3,4(;;,*:J3,!7,'$4<M6

!6 N$',3:=*,=$G9(1:+($1,$F,J:;D3</,<:&,*$4,%'$9:9;3,(+,(<6

O6 PF,&$D,<D9<='(93,+$,+*3,:Q($G<,$F,%'$9:9(;(+&/,+*$<3,1DG93'<,GD<+,<DG,+$,"6

RQ:G%;3>,L$$;3:1,J:'(:9;3<,0/,L/,#

.6"."""

.6!S.""

.6"."."

.6.S.."

.6.S"".

.6"..".

.6.S"..

.6O....

!"#$%&'

0

L

#.6.S.6!S

.6". .6.S.6.S

.6".

.6"..6O.

#$%&'()*+,-,!.."/,012'34,56,7$$'3 8'$9:9(;(<+(=,01:;&+(=<>,?;(23,@!

T<(1),+*3,B$(1+

U13,&$D,*:J3,+*3,BC,&$D,=:1,:<H,F$',+*3,%'$9:9(;(+&,$F,:1&,;$)(=:;,3Q%'3<<($1,(1J$;J(1),&$D',:++'(9D+3

!"!

"!"

!"#$%&'()!*+,-

.*+,/./

27

Michael S. Lewicki ! Carnegie MellonAI: Probability Theory ! Mar 1, 2007

!"

#$%&'()*+,-,!.."/,012'34,56,7$$'3 8'$9:9(;(<+(=,01:;&+(=<>,?;(23,@"

A*3,B$(1+,C(<+'(9D+($1

E3=(%3,F$',G:H(1),:,I$(1+,2(<+'(9D+($1,$F,7,J:'(:9;3<>

"6 7:H3,:,+'D+*,+:9;3,;(<+(1),:;;,=$G9(1:+($1<,$F,J:;D3<,$F,&$D',J:'(:9;3<,K(F,+*3'3,:'3,7,L$$;3:1,J:'(:9;3<,+*31,+*3,+:9;3,4(;;,*:J3,!7,'$4<M6

!6 N$',3:=*,=$G9(1:+($1,$F,J:;D3</,<:&,*$4,%'$9:9;3,(+,(<6

O6 PF,&$D,<D9<='(93,+$,+*3,:Q($G<,$F,%'$9:9(;(+&/,+*$<3,1DG93'<,GD<+,<DG,+$,"6

RQ:G%;3>,L$$;3:1,J:'(:9;3<,0/,L/,#

.6"."""

.6!S.""

.6"."."

.6.S.."

.6.S"".

.6"..".

.6.S"..

.6O....

!"#$%&'

0

L

#.6.S.6!S

.6". .6.S.6.S

.6".

.6"..6O.

#$%&'()*+,-,!.."/,012'34,56,7$$'3 8'$9:9(;(<+(=,01:;&+(=<>,?;(23,@!

T<(1),+*3,B$(1+

U13,&$D,*:J3,+*3,BC,&$D,=:1,:<H,F$',+*3,%'$9:9(;(+&,$F,:1&,;$)(=:;,3Q%'3<<($1,(1J$;J(1),&$D',:++'(9D+3

!"!

"!"

!"#$%&'()!*+,-

.*+,/./

28

!!

"#$%&'()*+,+!--./+012&34+56+7##&3 8&#9:9';'<*'=+01:;%*'=<>+?;'23+@A

B<'1(+*)3+C#'1*

8D8##&+7:;3E+F+-6@GH@ !"!

"!"

!"#$%&'()!*+,-

.*+,/./

"#$%&'()*+,+!--./+012&34+56+7##&3 8&#9:9';'<*'=+01:;%*'=<>+?;'23+@@

B<'1(+*)3+C#'1*

8D8##&E+F+-6IG-@ !"!

"!"

!"#$%&'()!*+,-

.*+,/./

Michael S. Lewicki ! Carnegie MellonAI: Probability Theory ! Mar 1, 2007

!"

#$%&'()*+,-,!.."/,012'34,56,7$$'3 8'$9:9(;(<+(=,01:;&+(=<>,?;(23,@"

A*3,B$(1+,C(<+'(9D+($1

E3=(%3,F$',G:H(1),:,I$(1+,2(<+'(9D+($1,$F,7,J:'(:9;3<>

"6 7:H3,:,+'D+*,+:9;3,;(<+(1),:;;,=$G9(1:+($1<,$F,J:;D3<,$F,&$D',J:'(:9;3<,K(F,+*3'3,:'3,7,L$$;3:1,J:'(:9;3<,+*31,+*3,+:9;3,4(;;,*:J3,!7,'$4<M6

!6 N$',3:=*,=$G9(1:+($1,$F,J:;D3</,<:&,*$4,%'$9:9;3,(+,(<6

O6 PF,&$D,<D9<='(93,+$,+*3,:Q($G<,$F,%'$9:9(;(+&/,+*$<3,1DG93'<,GD<+,<DG,+$,"6

RQ:G%;3>,L$$;3:1,J:'(:9;3<,0/,L/,#

.6"."""

.6!S.""

.6"."."

.6.S.."

.6.S"".

.6"..".

.6.S"..

.6O....

!"#$%&'

0

L

#.6.S.6!S

.6". .6.S.6.S

.6".

.6"..6O.

#$%&'()*+,-,!.."/,012'34,56,7$$'3 8'$9:9(;(<+(=,01:;&+(=<>,?;(23,@!

T<(1),+*3,B$(1+

U13,&$D,*:J3,+*3,BC,&$D,=:1,:<H,F$',+*3,%'$9:9(;(+&,$F,:1&,;$)(=:;,3Q%'3<<($1,(1J$;J(1),&$D',:++'(9D+3

!"!

"!"

!"#$%&'()!*+,-

.*+,/./

29

!!

"#$%&'()*+,+!--./+012&34+56+7##&3 8&#9:9';'<*'=+01:;%*'=<>+?;'23+@A

B<'1(+*)3+C#'1*

8D8##&+7:;3E+F+-6@GH@ !"!

"!"

!"#$%&'()!*+,-

.*+,/./

"#$%&'()*+,+!--./+012&34+56+7##&3 8&#9:9';'<*'=+01:;%*'=<>+?;'23+@@

B<'1(+*)3+C#'1*

8D8##&E+F+-6IG-@ !"!

"!"

!"#$%&'()!*+,-

.*+,/./

Michael S. Lewicki ! Carnegie MellonAI: Probability Theory ! Mar 1, 2007

!"

#$%&'()*+,-,!.."/,012'34,56,7$$'3 8'$9:9(;(<+(=,01:;&+(=<>,?;(23,@"

A*3,B$(1+,C(<+'(9D+($1

E3=(%3,F$',G:H(1),:,I$(1+,2(<+'(9D+($1,$F,7,J:'(:9;3<>

"6 7:H3,:,+'D+*,+:9;3,;(<+(1),:;;,=$G9(1:+($1<,$F,J:;D3<,$F,&$D',J:'(:9;3<,K(F,+*3'3,:'3,7,L$$;3:1,J:'(:9;3<,+*31,+*3,+:9;3,4(;;,*:J3,!7,'$4<M6

!6 N$',3:=*,=$G9(1:+($1,$F,J:;D3</,<:&,*$4,%'$9:9;3,(+,(<6

O6 PF,&$D,<D9<='(93,+$,+*3,:Q($G<,$F,%'$9:9(;(+&/,+*$<3,1DG93'<,GD<+,<DG,+$,"6

RQ:G%;3>,L$$;3:1,J:'(:9;3<,0/,L/,#

.6"."""

.6!S.""

.6"."."

.6.S.."

.6.S"".

.6"..".

.6.S"..

.6O....

!"#$%&'

0

L

#.6.S.6!S

.6". .6.S.6.S

.6".

.6"..6O.

#$%&'()*+,-,!.."/,012'34,56,7$$'3 8'$9:9(;(<+(=,01:;&+(=<>,?;(23,@!

T<(1),+*3,B$(1+

U13,&$D,*:J3,+*3,BC,&$D,=:1,:<H,F$',+*3,%'$9:9(;(+&,$F,:1&,;$)(=:;,3Q%'3<<($1,(1J$;J(1),&$D',:++'(9D+3

!"!

"!"

!"#$%&'()!*+,-

.*+,/./

30

!"

#$%&'()*+,-,!../0,123'45,67,8$$'4 9'$:;:(<(=+(>,12;<&+(>=?,@<(34,AB

C2D4'42>4,5(+*,+*4,E$(2+

!

!"

#"

!

"!"#

"$%&'()*+",-./

"%*0"$%&'()*+",-./

!

!#!#

1,-.2

1,-.2

12

12132

!

!!

"

"

!"

!!"!!"

#$%&'()*+,-,!../0,123'45,67,8$$'4 9'$:;:(<(=+(>,12;<&+(>=?,@<(34,AF

C2D4'42>4,5(+*,+*4,E$(2+

!

!"

#"

!

"!"#

"$%&'()*+",-./

"%*0"$%&'()*+",-./

!

!#!#

1,-.2

1,-.2

12

12132

!

!!

"

"

!"

!!"!!"

9G8;<4 H,9$$'I,J,.7AFBA,K,.7LF.A,J,.7F/!,,

!"

#$%&'()*+,-,!../0,123'45,67,8$$'4 9'$:;:(<(=+(>,12;<&+(>=?,@<(34,AB

C2D4'42>4,5(+*,+*4,E$(2+

!

!"

#"

!

"!"#

"$%&'()*+",-./

"%*0"$%&'()*+",-./

!

!#!#

1,-.2

1,-.2

12

12132

!

!!

"

"

!"

!!"!!"

#$%&'()*+,-,!../0,123'45,67,8$$'4 9'$:;:(<(=+(>,12;<&+(>=?,@<(34,AF

C2D4'42>4,5(+*,+*4,E$(2+

!

!"

#"

!

"!"#

"$%&'()*+",-./

"%*0"$%&'()*+",-./

!

!#!#

1,-.2

1,-.2

12

12132

!

!!

"

"

!"

!!"!!"

9G8;<4 H,9$$'I,J,.7AFBA,K,.7LF.A,J,.7F/!,,

!"

#$%&'()*+,-,!../0,123'45,67,8$$'4 9'$:;:(<(=+(>,12;<&+(>=?,@<(34,AB

C2D4'42>4,5(+*,+*4,E$(2+

!

!"

#"

!

"!"#

"$%&'()*+",-./

"%*0"$%&'()*+",-./

!

!#!#

1,-.2

1,-.2

12

12132

!

!!

"

"

!"

!!"!!"

#$%&'()*+,-,!../0,123'45,67,8$$'4 9'$:;:(<(=+(>,12;<&+(>=?,@<(34,AF

C2D4'42>4,5(+*,+*4,E$(2+

!

!"

#"

!

"!"#

"$%&'()*+",-./

"%*0"$%&'()*+",-./

!

!#!#

1,-.2

1,-.2

12

12132

!

!!

"

"

!"

!!"!!"

9G8;<4 H,9$$'I,J,.7AFBA,K,.7LF.A,J,.7F/!,,

Michael S. Lewicki ! Carnegie MellonAI: Probability Theory ! Mar 1, 2007

Continuous probability distributions

• probability density function (pdf)

• joint probability density

• marginal probability

• calculating probabilities using the pdf

• Bayes’ rule

31

Michael S. Lewicki ! Carnegie MellonAI: Probability Theory ! Mar 1, 2007 32

!

"#$%&'()*+,+-../0+123&45+67+8##&4 9&#:;:'<'*%+=42>'*'4>?+@<'34+A

1+9=B+#C+1D4&'E;2+1(4>+'2+-...

"#$%&'()*+,+-../0+123&45+67+8##&4 9&#:;:'<'*%+=42>'*'4>?+@<'34+F

1+9=B+#C+1D4&'E;2+1(4>+'2+-...G4*+H+:4+;+E#2*'2I#I>+&;23#D+J;&';:<47

KC+$LMN+'>+;+9&#:;:'<'*%+=42>'*%+BI2E*'#2+C#&+H+*)42O

! " #$

$%&!

"#

$##%!&"' !"

! " #$

$%&#$

%$&'(

&'(!&'("#$)'(%$ $%'

P+.7!F

more of Andrew’s nice slides

Michael S. Lewicki ! Carnegie MellonAI: Probability Theory ! Mar 1, 2007

!

"#$%&'()*+,+-../0+123&45+67+8##&4 9&#:;:'<'*%+=42>'*'4>?+@<'34+A

1+9=B+#C+1D4&'E;2+1(4>+'2+-...

"#$%&'()*+,+-../0+123&45+67+8##&4 9&#:;:'<'*%+=42>'*'4>?+@<'34+F

1+9=B+#C+1D4&'E;2+1(4>+'2+-...G4*+H+:4+;+E#2*'2I#I>+&;23#D+J;&';:<47

KC+$LMN+'>+;+9&#:;:'<'*%+=42>'*%+BI2E*'#2+C#&+H+*)42O

! " #$

$%&!

"#

$##%!&"' !"

! " #$

$%&#$

%$&'(

&'(!&'("#$)'(%$ $%'

P+.7!F

33

Michael S. Lewicki ! Carnegie MellonAI: Probability Theory ! Mar 1, 2007 34

What does p(x) mean?

• It does not mean a probability!

• First of all, it’s not a value between 0 and 1.

• It’s just a value, and an arbitrary one at that.

• The likelihood of p(a) can only be compared relatively to other values p(b)

• It indicates the relative probability of the integrated density over a small delta:

!

"#$%&'()*+,+-../0+123&45+67+8##&4 9&#:;:'<'*%+=42>'*'4>?+@<'34+/A

B;<C'2(+*#+%#D&+>*#E;F)

G 6);*H>+*)4+(D*IJ44<+E4;2'2(+#J+$KLMN

OJ+

*)42+

5)42+;+P;<D4+Q+'>+>;E$<43+J&#E+*)4+3'>*&':D*'#20+%#D+;&4+! *'E4>+;>+<'C4<%+*#+J'23+*);*+Q+'>+RP4&%+F<#>4+*#S+T7A/ *);2+*);*+Q+'>+RP4&%+F<#>4+*#S+T7U-7:

;

!!"

#""!"

!"

"#$%&'()*+,+-../0+123&45+67+8##&4 9&#:;:'<'*%+=42>'*'4>?+@<'34+/V

B;<C'2(+*#+%#D&+>*#E;F)

G 6);*H>+*)4+(D*IJ44<+E4;2'2(+#J+$KLMN

OJ+

*)42+

!!"

#""!"

!"

!$!%$!&

$#%$#&

$"

#$$%

#$$%

& !"

!"#$%

&

Michael S. Lewicki ! Carnegie MellonAI: Probability Theory ! Mar 1, 2007 35

!

"#$%&'()*+,+-../0+123&45+67+8##&4 9&#:;:'<'*%+=42>'*'4>?+@<'34+/A

BC$4D*;*'#2>BEFG+H+*)4+4C$4D*43+I;<J4+#K+&;23#L+I;&';:<4+F

H+*)4+;I4&;(4+I;<J4+54M3+>44+'K+54+*##N+;+I4&%+<;&(4+2JL:4&+#K+&;23#L+>;L$<4>+#K+F

!"

#"$

$!

"!!#! !"

"#$%&'()*+,+-../0+123&45+67+8##&4 9&#:;:'<'*%+=42>'*'4>?+@<'34+/O

BC$4D*;*'#2>BEFG+H+*)4+4C$4D*43+I;<J4+#K+&;23#L+I;&';:<4+F

H+*)4+;I4&;(4+I;<J4+54M3+>44+'K+54+*##N+;+I4&%+<;&(4+2JL:4&+#K+&;23#L+>;L$<4>+#K+F

!"

#"$

$!

"!!#! !"

H+*)4+K'&>*+L#L42*+#K+*)4+>);$4+K#&L43+:%+*)4+;C4>+;23+*)4+:<J4+DJ&I4

H+*)4+:4>*+I;<J4+*#+D)##>4+'K+%#J+LJ>*+(J4>>+;2+J2N2#52+$4&>#2M>+;(4+;23+%#JM<<+:4+K'243+*)4+>PJ;&4+#K+%#J&+4&&#&

BE;(4GHQR7O!A

Michael S. Lewicki ! Carnegie MellonAI: Probability Theory ! Mar 1, 2007

!

"#$%&'()*+,+-../0+123&45+67+8##&4 9&#:;:'<'*%+=42>'*'4>?+@<'34+/A

BC$4D*;*'#2>BEFG+H+*)4+4C$4D*43+I;<J4+#K+&;23#L+I;&';:<4+F

H+*)4+;I4&;(4+I;<J4+54M3+>44+'K+54+*##N+;+I4&%+<;&(4+2JL:4&+#K+&;23#L+>;L$<4>+#K+F

!"

#"$

$!

"!!#! !"

"#$%&'()*+,+-../0+123&45+67+8##&4 9&#:;:'<'*%+=42>'*'4>?+@<'34+/O

BC$4D*;*'#2>BEFG+H+*)4+4C$4D*43+I;<J4+#K+&;23#L+I;&';:<4+F

H+*)4+;I4&;(4+I;<J4+54M3+>44+'K+54+*##N+;+I4&%+<;&(4+2JL:4&+#K+&;23#L+>;L$<4>+#K+F

!"

#"$

$!

"!!#! !"

H+*)4+K'&>*+L#L42*+#K+*)4+>);$4+K#&L43+:%+*)4+;C4>+;23+*)4+:<J4+DJ&I4

H+*)4+:4>*+I;<J4+*#+D)##>4+'K+%#J+LJ>*+(J4>>+;2+J2N2#52+$4&>#2M>+;(4+;23+%#JM<<+:4+K'243+*)4+>PJ;&4+#K+%#J&+4&&#&

BE;(4GHQR7O!A

36

Michael S. Lewicki ! Carnegie MellonAI: Probability Theory ! Mar 1, 2007

!"

#$%&'()*+,-,.""!/,012'34,56,7$$'3 8'$9:9(;(+&,<31=(+(3=>,?;(23,!@

AB%3C+:+($1,$D,:,DE1C+($1!FAGDHIJK,F,+*3,3B%3C+32,L:;E3,$D,DHBJ,4*3'3,B,(=,2':41,D'$M,IN=,2(=+'(9E+($16,

F,+*3,:L3':)3,L:;E3,43N2,=33,(D,43,+$$O,:,L3'&,;:')3,1EM93',$D,':12$M,=:M%;3=,$D,DHIJ

"#

$#%

%!

"!!#!$ !"!"!

P$+3,+*:+,(1,)313':;>

#!$"!#"$ %&$!$& &

%&'()*%#+,-$ . %&

%.'(.**#!+,-$" . %&

#$%&'()*+,-,.""!/,012'34,56,7$$'3 8'$9:9(;(+&,<31=(+(3=>,?;(23,."

Q:'(:1C3'. F,Q:'GIK,F,+*3,3B%3C+32,=RE:'32,2(DD3'31C3,93+4331,B,:12,AGIK, "

#

$#%

$%!

"!!#! !"!" .. !'

F,:M$E1+,&$EN2,3B%3C+,+$,;$=3,(D,&$E,ME=+,)E3==,:1,E1O1$41,%3'=$1N=,:)3,:12,&$EN;;,93,D(132,+*3,=RE:'3,$D,&$E',3''$'/,:12,:==EM(1),&$E,%;:&,$%+(M:;;&

/.'&0*#+,-$1+2 %

37

Michael S. Lewicki ! Carnegie MellonAI: Probability Theory ! Mar 1, 2007

!"

#$%&'()*+,-,.""!/,012'34,56,7$$'3 8'$9:9(;(+&,<31=(+(3=>,?;(23,!@

AB%3C+:+($1,$D,:,DE1C+($1!FAGDHIJK,F,+*3,3B%3C+32,L:;E3,$D,DHBJ,4*3'3,B,(=,2':41,D'$M,IN=,2(=+'(9E+($16,

F,+*3,:L3':)3,L:;E3,43N2,=33,(D,43,+$$O,:,L3'&,;:')3,1EM93',$D,':12$M,=:M%;3=,$D,DHIJ

"#

$#%

%!

"!!#!$ !"!"!

P$+3,+*:+,(1,)313':;>

#!$"!#"$ %&$!$& &

%&'()*%#+,-$ . %&

%.'(.**#!+,-$" . %&

#$%&'()*+,-,.""!/,012'34,56,7$$'3 8'$9:9(;(+&,<31=(+(3=>,?;(23,."

Q:'(:1C3'. F,Q:'GIK,F,+*3,3B%3C+32,=RE:'32,2(DD3'31C3,93+4331,B,:12,AGIK, "

#

$#%

$%!

"!!#! !"!" .. !'

F,:M$E1+,&$EN2,3B%3C+,+$,;$=3,(D,&$E,ME=+,)E3==,:1,E1O1$41,%3'=$1N=,:)3,:12,&$EN;;,93,D(132,+*3,=RE:'3,$D,&$E',3''$'/,:12,:==EM(1),&$E,%;:&,$%+(M:;;&

/.'&0*#+,-$1+2 %

38

Michael S. Lewicki ! Carnegie MellonAI: Probability Theory ! Mar 1, 2007

!!

"#$%&'()*+,+-..!/+012&34+56+7##&3 8&#9:9';'*%+<31='*'3=>+?;'23+-!

?*:12:&2+<3@':*'#1!- A+B:&CDE+A+*)3+3F$3G*32+=HI:&32+2'JJ3&31G3+93*4331+F+:12+KCDE+ "

#

$#%

$%!

"!!#! !"!" ## &!

A+:L#I1*+%#IM2+3F$3G*+*#+;#=3+'J+%#I+LI=*+(I3==+:1+I1N1#41+$3&=#1M=+:(3+:12+%#IM;;+93+J'132+*)3+=HI:&3+#J+%#I&+3&&#&/+:12+:==IL'1(+%#I+$;:%+#$*'L:;;%

! A+?*:12:&2+<3@':*'#1+A+O*%$'G:;P+23@':*'#1+#J+D+J&#L+'*=+L3:1

$#%&'()*+,-.*/ %

)-.*/ $%!

0#%##%!

"#$%&'()*+,+-..!/+012&34+56+7##&3 8&#9:9';'*%+<31='*'3=>+?;'23+--

Q1+-+2'L31='#1=

$RF/%S+A+$&#9:9';'*%+231='*%+#J+&:12#L+@:&':9;3=+RD/TS+:*+

;#G:*'#1+RF/%S

39

Michael S. Lewicki ! Carnegie MellonAI: Probability Theory ! Mar 1, 2007

!!

"#$%&'()*+,+-..!/+012&34+56+7##&3 8&#9:9';'*%+<31='*'3=>+?;'23+-!

?*:12:&2+<3@':*'#1!- A+B:&CDE+A+*)3+3F$3G*32+=HI:&32+2'JJ3&31G3+93*4331+F+:12+KCDE+ "

#

$#%

$%!

"!!#! !"!" ## &!

A+:L#I1*+%#IM2+3F$3G*+*#+;#=3+'J+%#I+LI=*+(I3==+:1+I1N1#41+$3&=#1M=+:(3+:12+%#IM;;+93+J'132+*)3+=HI:&3+#J+%#I&+3&&#&/+:12+:==IL'1(+%#I+$;:%+#$*'L:;;%

! A+?*:12:&2+<3@':*'#1+A+O*%$'G:;P+23@':*'#1+#J+D+J&#L+'*=+L3:1

$#%&'()*+,-.*/ %

)-.*/ $%!

0#%##%!

"#$%&'()*+,+-..!/+012&34+56+7##&3 8&#9:9';'*%+<31='*'3=>+?;'23+--

Q1+-+2'L31='#1=

$RF/%S+A+$&#9:9';'*%+231='*%+#J+&:12#L+@:&':9;3=+RD/TS+:*+

;#G:*'#1+RF/%S

40

Michael S. Lewicki ! Carnegie MellonAI: Probability Theory ! Mar 1, 2007

!"

#$%&'()*+,-,"..!/,012'34,56,7$$'3 8'$9:9(;(+&,<31=(+(3=>,?;(23,"@

A1,",2(B31=($1=

C3+,D/E, 93,:,%:(',$F,G$1+(1H$H=,':12$B,I:'(:9;3=/,:12,;3+,J,93,=$B3,'3)($1,$F,KD/EL,=%:G3M

!!"

#"!"#

$"$#"#%!&'(!"#

!"#!!"##

#$%&'()*+,-,"..!/,012'34,56,7$$'3 8'$9:9(;(+&,<31=(+(3=>,?;(23,"N

A1,",2(B31=($1=

C3+,D/E, 93,:,%:(',$F,G$1+(1H$H=,':12$B,I:'(:9;3=/,:12,;3+,J,93,=$B3,'3)($1,$F,KD/EL,=%:G3M

!!"

#"!"#

$"$#"#%!&'(!"#

!"#!!"##

8K,".OB%)O@.,:12"P..O43()*+O@...L,Q

:'3:,H123',+*3,"R2,=H'F:G3,4(+*(1,+*3,'32,'3G+:1);3

41

Michael S. Lewicki ! Carnegie MellonAI: Probability Theory ! Mar 1, 2007

!"

#$%&'()*+,-,"..!/,012'34,56,7$$'3 8'$9:9(;(+&,<31=(+(3=>,?;(23,"@

A1,",2(B31=($1=

C3+,D/E, 93,:,%:(',$F,G$1+(1H$H=,':12$B,I:'(:9;3=/,:12,;3+,J,93,=$B3,'3)($1,$F,KD/EL,=%:G3M

!!"

#"!"#

$"$#"#%!&'(!"#

!"#!!"##

#$%&'()*+,-,"..!/,012'34,56,7$$'3 8'$9:9(;(+&,<31=(+(3=>,?;(23,"N

A1,",2(B31=($1=

C3+,D/E, 93,:,%:(',$F,G$1+(1H$H=,':12$B,I:'(:9;3=/,:12,;3+,J,93,=$B3,'3)($1,$F,KD/EL,=%:G3M

!!"

#"!"#

$"$#"#%!&'(!"#

!"#!!"##

8K,".OB%)O@.,:12"P..O43()*+O@...L,Q

:'3:,H123',+*3,"R2,=H'F:G3,4(+*(1,+*3,'32,'3G+:1);3

42

Michael S. Lewicki ! Carnegie MellonAI: Probability Theory ! Mar 1, 2007

!"

#$%&'()*+,-,.//!0,123'45,67,8$$'4 9'$:;:(<(+&,=42>(+(4>?,@<(34,.A

B2,.,3(C42>($2>

D4+,E0F, :4,;,%;(',$G,H$2+(2I$I>,';23$C,J;'(;:<4>0,;23,<4+,K,:4,>$C4,'4)($2,$G,LE0FM,>%;H4N

!!"

#"!"#

$"$#"#%!&'(!"#

!"#!!"##

9L,OLC%)P.AMQ!/R. S,OL54()*+P""//MQ!A//R.

T,!,M,U

;'4;,I234',+*4,.P3,>I'G;H4,5(+*(2,+*4,'43,$J;<

#$%&'()*+,-,.//!0,123'45,67,8$$'4 9'$:;:(<(+&,=42>(+(4>?,@<(34,.V

B2,.,3(C42>($2>

D4+,E0F, :4,;,%;(',$G,H$2+(2I$I>,';23$C,J;'(;:<4>0,;23,<4+,K,:4,>$C4,'4)($2,$G,LE0FM,>%;H4N

!!"

#"!"#

$"$#"#%!&'(!"#

!"#!!"##

W;X4,+*4,>%4H(;<,H;>4,$G,'4)($2,K,U,Y4J4'&5*4'4Z7

K4C4C:4',+*;+,5(+*,%'$:;:(<(+&,!0,LE0FM,5(<<,:4,3';52,G'$C,Y>$C45*4'4Z7,

@$77

! !$

%$#

$

%$#

## "

$"$#"#% $!"#

43

Michael S. Lewicki ! Carnegie MellonAI: Probability Theory ! Mar 1, 2007

!"

#$%&'()*+,-,.//!0,123'45,67,8$$'4 9'$:;:(<(+&,=42>(+(4>?,@<(34,.A

B2,.,3(C42>($2>

D4+,E0F, :4,;,%;(',$G,H$2+(2I$I>,';23$C,J;'(;:<4>0,;23,<4+,K,:4,>$C4,'4)($2,$G,LE0FM,>%;H4N

!!"

#"!"#

$"$#"#%!&'(!"#

!"#!!"##

9L,OLC%)P.AMQ!/R. S,OL54()*+P""//MQ!A//R.

T,!,M,U

;'4;,I234',+*4,.P3,>I'G;H4,5(+*(2,+*4,'43,$J;<

#$%&'()*+,-,.//!0,123'45,67,8$$'4 9'$:;:(<(+&,=42>(+(4>?,@<(34,.V

B2,.,3(C42>($2>

D4+,E0F, :4,;,%;(',$G,H$2+(2I$I>,';23$C,J;'(;:<4>0,;23,<4+,K,:4,>$C4,'4)($2,$G,LE0FM,>%;H4N

!!"

#"!"#

$"$#"#%!&'(!"#

!"#!!"##

W;X4,+*4,>%4H(;<,H;>4,$G,'4)($2,K,U,Y4J4'&5*4'4Z7

K4C4C:4',+*;+,5(+*,%'$:;:(<(+&,!0,LE0FM,5(<<,:4,3';52,G'$C,Y>$C45*4'4Z7,

@$77

! !$

%$#

$

%$#

## "

$"$#"#% $!"#

44

Michael S. Lewicki ! Carnegie MellonAI: Probability Theory ! Mar 1, 2007

!"

#$%&'()*+,-,.//!0,123'45,67,8$$'4 9'$:;:(<(+&,=42>(+(4>?,@<(34,.A

B2,.,3(C42>($2>

D4+,E0F, :4,;,%;(',$G,H$2+(2I$I>,';23$C,J;'(;:<4>0,;23,<4+,K,:4,>$C4,'4)($2,$G,LE0FM,>%;H4N

!!"

#"!"#

$"$#"#%!&'(!"#

!"#!!"##

$%&

$$$$'() )

)"&

)"

)#'

)#( $

%

&'(

)*+,-.*+,-

/

#!"# "#%

#$%&'()*+,-,.//!0,123'45,67,8$$'4 9'$:;:(<(+&,=42>(+(4>?,@<(34,.O

B2,C,3(C42>($2>

D4+,LE!0E.0NECM :4,;2,2P+I%<4 $G,H$2+(2I$I>,';23$C,J;'(;:<4>0,;23,<4+,K,:4,>$C4,'4)($2,$G,!C N

#" !!"***""## $+ !'''( *

!! !"!###

**

*

$#$#$####%!"***""#

+$$+

$+

""***"!"***""#***

45

Michael S. Lewicki ! Carnegie MellonAI: Probability Theory ! Mar 1, 2007

!"

#$%&'()*+,-,.//!0,123'45,67,8$$'4 9'$:;:(<(+&,=42>(+(4>?,@<(34,.A

B234%42342C4

BD,E,;23,F,;'4,(234%42342+,+*42,G2$5(2),+*4,H;<I4,$D,E,3$4>,2$+,*4<%,%'43(C+,+*4,

H;<I4,$D,F

!"!"!#"$%&'#$())$ !"#"!#"$% !"#

J%)054()*+,KLM,(234%42342+

#$%&'()*+,-,.//!0,123'45,67,8$$'4 9'$:;:(<(+&,=42>(+(4>?,@<(34,N/

B234%42342C4

BD,E,;23,F,;'4,(234%42342+,+*42,G2$5(2),+*4,H;<I4,$D,E,3$4>,2$+,*4<%,%'43(C+,+*4,

H;<I4,$D,F

!"!"!#"$%&'#$())$ !"#"!#"$% !"#

+*4,C$2+$I'>,>;&,+*;+,;CC4<4';+($2,;23,54()*+,;'4,

(234%42342+

46

Michael S. Lewicki ! Carnegie MellonAI: Probability Theory ! Mar 1, 2007

!"

#$%&'()*+,-,.//!0,123'45,67,8$$'4 9'$:;:(<(+&,=42>(+(4>?,@<(34,.A

B234%42342C4

BD,E,;23,F,;'4,(234%42342+,+*42,G2$5(2),+*4,H;<I4,$D,E,3$4>,2$+,*4<%,%'43(C+,+*4,

H;<I4,$D,F

!"!"!#"$%&'#$())$ !"#"!#"$% !"#

J%)054()*+,KLM,(234%42342+

#$%&'()*+,-,.//!0,123'45,67,8$$'4 9'$:;:(<(+&,=42>(+(4>?,@<(34,N/

B234%42342C4

BD,E,;23,F,;'4,(234%42342+,+*42,G2$5(2),+*4,H;<I4,$D,E,3$4>,2$+,*4<%,%'43(C+,+*4,

H;<I4,$D,F

!"!"!#"$%&'#$())$ !"#"!#"$% !"#

+*4,C$2+$I'>,>;&,+*;+,;CC4<4';+($2,;23,54()*+,;'4,

(234%42342+

47

Michael S. Lewicki ! Carnegie MellonAI: Probability Theory ! Mar 1, 2007

!"

#$%&'()*+,-,.//!0,123'45,67,8$$'4 9'$:;:(<(+&,=42>(+(4>?,@<(34,A!

8B<+(C;'(;+4,DE%4F+;+($2

!!!"#" !"" !"# !"#$

DGH%)054()*+I,JK.L7M0."//N

O*4,F42+'$(3,$P,+*4,F<$B3

#$%&'()*+,-,.//!0,123'45,67,8$$'4 9'$:;:(<(+&,=42>(+(4>?,@<(34,A.

8B<+(C;'(;+4,DE%4F+;+($2

!!!" !" !"$$# !"!"!#"$

48

Michael S. Lewicki ! Carnegie MellonAI: Probability Theory ! Mar 1, 2007

!"

#$%&'()*+,-,.//!0,123'45,67,8$$'4 9'$:;:(<(+&,=42>(+(4>?,@<(34,A!

8B<+(C;'(;+4,DE%4F+;+($2

!!!"#" !"" !"# !"#$

DGH%)054()*+I,JK.L7M0."//N

O*4,F42+'$(3,$P,+*4,F<$B3

#$%&'()*+,-,.//!0,123'45,67,8$$'4 9'$:;:(<(+&,=42>(+(4>?,@<(34,A.

8B<+(C;'(;+4,DE%4F+;+($2

!!!" !" !"$$# !"!"!#"$

49

Michael S. Lewicki ! Carnegie MellonAI: Probability Theory ! Mar 1, 2007

!"

#$%&'()*+,-,.//!0,123'45,67,8$$'4 9'$:;:(<(+&,=42>(+(4>?,@<(34,AA

B4>+,&$C',C234'>+;23(2)!"#$#$#$"%&'("')'*+",-."/0'1"234'(5-&1 !"#"!#" !"!

D1<<,+*4,+(E4F

DG2<&,5*42,H,;23,I,;'4,(234%42342+F

DJ+,K;2,L;(<,4M42,(L,H,;23,I,;'4,(234%42342+F

#$%&'()*+,-,.//!0,123'45,67,8$$'4 9'$:;:(<(+&,=42>(+(4>?,@<(34,AN

O(M;'(;+4,PQ%4K+;+($2

#"" $%$&%&'&!#("&%&( +6,+#6,$"50'1"+6,"-.

#" $%$&%&'%&(%&(" +6,+6,+#6,$

#"" $%$&%&'%!#("%%&( +6,+#6,$"50'1"+6,"-.

# !"!" $%$&%&'%&!#("%&%&( +6,+,+#6,$"50'1"+6,"-.

#$#$#$" !"#"!#" !"!

50

Michael S. Lewicki ! Carnegie MellonAI: Probability Theory ! Mar 1, 2007

!"

#$%&'()*+,-,.//!0,123'45,67,8$$'4 9'$:;:(<(+&,=42>(+(4>?,@<(34,AA

B4>+,&$C',C234'>+;23(2)!"#$#$#$"%&'("')'*+",-."/0'1"234'(5-&1 !"#"!#" !"!

D1<<,+*4,+(E4F

DG2<&,5*42,H,;23,I,;'4,(234%42342+F

DJ+,K;2,L;(<,4M42,(L,H,;23,I,;'4,(234%42342+F

#$%&'()*+,-,.//!0,123'45,67,8$$'4 9'$:;:(<(+&,=42>(+(4>?,@<(34,AN

O(M;'(;+4,PQ%4K+;+($2

#"" $%$&%&'&!#("&%&( +6,+#6,$"50'1"+6,"-.

#" $%$&%&'%&(%&(" +6,+6,+#6,$

#"" $%$&%&'%!#("%%&( +6,+#6,$"50'1"+6,"-.

# !"!" $%$&%&'%&!#("%&%&( +6,+,+#6,$"50'1"+6,"-.

#$#$#$" !"#"!#" !"!

51

Michael S. Lewicki ! Carnegie MellonAI: Probability Theory ! Mar 1, 2007

!"

#$%&'()*+,-,.//!0,123'45,67,8$$'4 9'$:;:(<(+&,=42>(+(4>?,@<(34,AB

C(D;'(;+4,#$D;'(;2E4

!"!#$#"%&'($ !""! #$%#$ !!" ##$$

"!$#"$"%&'($ ))

"""" $%$&'($$ !"" #$$$$

"!$#"$"%&'($ ))

!!!! #%#&'(## !"" #$$$$

#$%&'()*+,-,.//!0,123'45,67,8$$'4 9'$:;:(<(+&,=42>(+(4>?,@<(34,AF

C(D;'(;+4,#$D;'(;2E4

!"!#$#"%&'($ !""! #$%#$ !!" ##$$

"!$#"$"%&'($ ))

"""" $%$&'($$ !"" #$$$$

"!$#"$"%&'($ ))

!!!! #%#&'(## !"" #$$$$

*+,-.%**/01+- %%&

'(()

*$

#

$))!

&

'(()

*$$##$

!"!

"!"*

"" )++,,%))

)

"$"*$""""

!"!"!!#$%

52

Michael S. Lewicki ! Carnegie MellonAI: Probability Theory ! Mar 1, 2007

!"

#$%&'()*+,-,.//!0,123'45,67,8$$'4 9'$:;:(<(+&,=42>(+(4>?,@<(34,AB

C(D;'(;+4,#$D;'(;2E4

!"!#$#"%&'($ !""! #$%#$ !!" ##$$

"!$#"$"%&'($ ))

"""" $%$&'($$ !"" #$$$$

"!$#"$"%&'($ ))

!!!! #%#&'(## !"" #$$$$

#$%&'()*+,-,.//!0,123'45,67,8$$'4 9'$:;:(<(+&,=42>(+(4>?,@<(34,AF

C(D;'(;+4,#$D;'(;2E4

!"!#$#"%&'($ !""! #$%#$ !!" ##$$

"!$#"$"%&'($ ))

"""" $%$&'($$ !"" #$$$$

"!$#"$"%&'($ ))

!!!! #%#&'(## !"" #$$$$

*+,-.%**/01+- %%&

'(()

*$

#

$))!

&

'(()

*$$##$

!"!

"!"*

"" )++,,%))

)

"$"*$""""

!"!"!!#$%

53

Michael S. Lewicki ! Carnegie MellonAI: Probability Theory ! Mar 1, 2007

!"

#$%&'()*+,-,.//!0,123'45,67,8$$'4 9'$:;:(<(+&,=42>(+(4>?,@<(34,AB

#$C;'(;2D4,E2+F(+($2

GHI%)054()*+J,KL.M7N0.O//P

!"#$ !"!"#$ !"

%&&'()$*+ !"

%&&'()$*+ !"

#$%&'()*+,-,.//!0,123'45,67,8$$'4 9'$:;:(<(+&,=42>(+(4>?,@<(34,AQ

#$C;'(;2D4,E2+F(+($2

GHI%)054()*+J,KL.M7N0.O//P

!"#$ !"!"#$ !"

%&&'()$*+ !"

%&&'()$*+ !"

9'(2D(%;<G()42C4D+$'

$R,#

54

Michael S. Lewicki ! Carnegie MellonAI: Probability Theory ! Mar 1, 2007

!"

#$%&'()*+,-,.//!0,123'45,67,8$$'4 9'$:;:(<(+&,=42>(+(4>?,@<(34,AB

#$C;'(;2D4,E2+F(+($2

GHI%)054()*+J,KL.M7N0.O//P

!"#$ !"!"#$ !"

%&&'()$*+ !"

%&&'()$*+ !"

#$%&'()*+,-,.//!0,123'45,67,8$$'4 9'$:;:(<(+&,=42>(+(4>?,@<(34,AQ

#$C;'(;2D4,E2+F(+($2

GHI%)054()*+J,KL.M7N0.O//P

!"#$ !"!"#$ !"

%&&'()$*+ !"

%&&'()$*+ !"

9'(2D(%;<G()42C4D+$'

$R,#

55

Michael S. Lewicki ! Carnegie MellonAI: Probability Theory ! Mar 1, 2007

!"

#$%&'()*+,-,!""./,012'34,56,7$$'3 8'$9:9(;(+&,<31=(+(3=>,?;(23,@A

#$B:'(:1C3,DE1,D:C+=

!!

"

#

$$

%

&''(('

!"!

"!"#

"" $%%&&'$!

!

"#"$#))

))!!"!""#$%

FG'E3,$',D:;=3>,HI,)J& K,",+*31,L,:12,M,:'3,(123%31231+

FG'E3,$',D:;=3>,HI,L,:12,M,:'3,(123%31231+,+*31,)J& K,"

FG'E3,$',D:;=3>,HI,)J& K,)J )& +*31,L,:12,M,:'3,23+3'N(1(=+(C:;;&,'3;:+32

FG'E3,$',D:;=3>,HI,L,:12,M,:'3,23+3'N(1(=+(C:;;&,'3;:+32,+*31,)J& K,)J )&

O$4,C$E;2,&$E,%'$B3,$',2(=%'$B3,+*3=3P

#$%&'()*+,-,!""./,012'34,56,7$$'3 8'$9:9(;(+&,<31=(+(3=>,?;(23,Q"

R313':;,#$B:'(:1C3

!!"!""#$% '((' $%%&&'$ #

"" "#"$#

S3+,! K,TL./L!/,U,LVW 93,:,B3C+$',$I,V C$1+(1E$E=,':12$N,B:'(:9;3=

() ""())( **+,- )'' "%#!

?,(=,:,V,J,V,=&NN3+'(C,1$1X13):+(B3,23I(1(+3,N:+'(J

HI,:;;,2(=+'(9E+($1=,:'3,;(13:';&,(123%31231+,(+,(=,%$=(+(B3,23I(1(+3

HI,+*3,2(=+'(9E+($1=,:'3,;(13:';&,23%31231+,(+,*:=,23+3'N(1:1+,Y3'$

56

Michael S. Lewicki ! Carnegie MellonAI: Probability Theory ! Mar 1, 2007

!"

#$%&'()*+,-,!""./,012'34,56,7$$'3 8'$9:9(;(+&,<31=(+(3=>,?;(23,@A

#$B:'(:1C3,DE1,D:C+=

!!

"

#

$$

%

&''(('

!"!

"!"#

"" $%%&&'$!

!

"#"$#))

))!!"!""#$%

FG'E3,$',D:;=3>,HI,)J& K,",+*31,L,:12,M,:'3,(123%31231+

FG'E3,$',D:;=3>,HI,L,:12,M,:'3,(123%31231+,+*31,)J& K,"

FG'E3,$',D:;=3>,HI,)J& K,)J )& +*31,L,:12,M,:'3,23+3'N(1(=+(C:;;&,'3;:+32

FG'E3,$',D:;=3>,HI,L,:12,M,:'3,23+3'N(1(=+(C:;;&,'3;:+32,+*31,)J& K,)J )&

O$4,C$E;2,&$E,%'$B3,$',2(=%'$B3,+*3=3P

#$%&'()*+,-,!""./,012'34,56,7$$'3 8'$9:9(;(+&,<31=(+(3=>,?;(23,Q"

R313':;,#$B:'(:1C3

!!"!""#$% '((' $%%&&'$ #

"" "#"$#

S3+,! K,TL./L!/,U,LVW 93,:,B3C+$',$I,V C$1+(1E$E=,':12$N,B:'(:9;3=

() ""())( **+,- )'' "%#!

?,(=,:,V,J,V,=&NN3+'(C,1$1X13):+(B3,23I(1(+3,N:+'(J

HI,:;;,2(=+'(9E+($1=,:'3,;(13:';&,(123%31231+,(+,(=,%$=(+(B3,23I(1(+3

HI,+*3,2(=+'(9E+($1=,:'3,;(13:';&,23%31231+,(+,*:=,23+3'N(1:1+,Y3'$

57

Michael S. Lewicki ! Carnegie MellonAI: Probability Theory ! Mar 1, 2007

!"

#$%&'()*+,-,!.."/,012'34,56,7$$'3 8'$9:9(;(+&,<31=(+(3=>,?;(23,@"

A3=+,&$B',B123'=+:12(1)!"#$#$#$"%&'("')'*+",-."/0'1"234'(5-&1 !"#$%"#$!%"#$ !"!

C0;;,+*3,+(D3E

CF1;&,4*31,G,:12,H,:'3,(123%31231+E

CI+,J:1,K:(;,3L31,(K,G,:12,H,:'3,(123%31231+E

#$%&'()*+,-,!.."/,012'34,56,7$$'3 8'$9:9(;(+&,<31=(+(3=>,?;(23,@!

7:')(1:;,<(=+'(9B+($1=

#$

%$"

"&

'&&()() +6,+,

58

Michael S. Lewicki ! Carnegie MellonAI: Probability Theory ! Mar 1, 2007

!!

"#$%&'()*+,+!--./+012&34+56+7##&3 8&#9:9';'*%+<31='*'3=>+?;'23+@A

"#12'*'#1:;+<'=*&'9B*'#1=

!"#

!$%

!

!

!"#$%!!&'!()*)')

+,-

+./00"$12#3,4(2- !%

+5600"$12#3,4(2- !%

+6000"$12#3,4(2- !%

"#$%&'()*+,+!--./+012&34+56+7##&3 8&#9:9';'*%+<31='*'3=>+?;'23+@@

"#12'*'#1:;+<'=*&'9B*'#1=

!"#

!$%

!

!

!"#$%!!&'!()*)')

+,-

+./00"$12#3,4(2- !%

+-

+7-+,-

!%

!$%!$% !

5)%C

59

Michael S. Lewicki ! Carnegie MellonAI: Probability Theory ! Mar 1, 2007

!!

"#$%&'()*+,+!--./+012&34+56+7##&3 8&#9:9';'*%+<31='*'3=>+?;'23+@A

"#12'*'#1:;+<'=*&'9B*'#1=

!"#

!$%

!

!

!"#$%!!&'!()*)')

+,-

+./00"$12#3,4(2- !%

+5600"$12#3,4(2- !%

+6000"$12#3,4(2- !%

"#$%&'()*+,+!--./+012&34+56+7##&3 8&#9:9';'*%+<31='*'3=>+?;'23+@@

"#12'*'#1:;+<'=*&'9B*'#1=

!"#

!$%

!

!

!"#$%!!&'!()*)')

+,-

+./00"$12#3,4(2- !%

+-

+7-+,-

!%

!$%!$% !

5)%C

60

Michael S. Lewicki ! Carnegie MellonAI: Probability Theory ! Mar 1, 2007

!"

#$%&'()*+,-,!../0,123'45,67,8$$'4 9'$:;:(<(+&,=42>(+(4>?,@<(34,AB

C234%42342D4,E4F(>(+43

C+G>,4;>&,+$,%'$F4,+*;+,+*4>4,>+;+4H42+>,;'4,4IJ(F;<42+K

!"!"!#"$%&'#$())$ !"#"!#"$% !"#

!"!*"$%&'#

!"!*"$%&'#

!"!"!#"$%&'#

!"#!"

#"!#"

!"#"!#"

!"

$

!"

$

!"

#$%&'()*+,-,!../0,123'45,67,8$$'4 9'$:;:(<(+&,=42>(+(4>?,@<(34,AL

8$'4,J>4MJ<,>+JMM

N;&4>EJ<4

OP*4>4,D;2,;<<,:4,%'$F43,M'$H,34M(2(+($2>,$2,%'4F($J>,><(34>Q

+!*" !%&

'&!#

&#!#"

!*"

!*#"!#*"

'!"

'!#"'!#" !

!"

!"!*"!*"

!"

#"#!"!#" !

61

Michael S. Lewicki ! Carnegie MellonAI: Probability Theory ! Mar 1, 2007

!"

#$%&'()*+,-,!../0,123'45,67,8$$'4 9'$:;:(<(+&,=42>(+(4>?,@<(34,AB

C234%42342D4,E4F(>(+43

C+G>,4;>&,+$,%'$F4,+*;+,+*4>4,>+;+4H42+>,;'4,4IJ(F;<42+K

!"!"!#"$%&'#$())$ !"#"!#"$% !"#

!"!*"$%&'#

!"!*"$%&'#

!"!"!#"$%&'#

!"#!"

#"!#"

!"#"!#"

!"

$

!"

$

!"

#$%&'()*+,-,!../0,123'45,67,8$$'4 9'$:;:(<(+&,=42>(+(4>?,@<(34,AL

8$'4,J>4MJ<,>+JMM

N;&4>EJ<4

OP*4>4,D;2,;<<,:4,%'$F43,M'$H,34M(2(+($2>,$2,%'4F($J>,><(34>Q

+!*" !%&

'&!#

&#!#"

!*"

!*#"!#*"

'!"

'!#"'!#" !

!"

!"!*"!*"

!"

#"#!"!#" !

62

Michael S. Lewicki ! Carnegie MellonAI: Probability Theory ! Mar 1, 2007 63

Next time: The process of probabilistic inference

1. define model of problem

2. derive posterior distributions and estimators

3. estimate parameters from data

4. evaluate model accuracy

top related