review of bayes’ rule

BIOL 301 Guest Lecture: Reasoning Under Uncertainty

(Intro to Bayes Networks)

Simon D. LevyCSCI Department

8 April 2010

Review of Bayes’ Rule

• From the Product Rule:

• P(A|B) = P(A & B) / P(B)• P(A &B) = P(A|B) * P(B) = P(B|A) * P(A)

Rev. Thomas Bayes (1702-1761)• We derive Bayes’ Rule by

substitution:• P(A|B) = P(A & B) / P(B) = P(B|A) * P(A) / P(B)

Real-World Problems May Involve Many Variables

http://www.bayesia.com/assets/images/content/produits/blab/tutoriel/en/chapitre-3/image026.jpg

Real-World Problems May Involve Many Variables

Variables Are TypicallyOberserved Simultaneously

(Confounded)Fever Ache Virus PNo No No .950No No Yes .002No Yes No .032No Yes Yes .002Yes No No .002Yes No Yes .001Yes Yes No .010Yes Yes Yes .001

So how do we compute P(V=Yes), P(F=Yes & A=No), etc.?

Marginalization

Fever Ache Virus PNo No No .950No No Yes .002No Yes No .032No Yes Yes .002Yes No No .002Yes No Yes .001Yes Yes No .010Yes Yes Yes .001 ______ Sum = .006

P(V=Yes):

Marginalization

Fever Ache Virus PNo No No .950No No Yes .002No Yes No .032No Yes Yes .002Yes No No .002Yes No Yes .001Yes Yes No .010Yes Yes Yes .001 ______ Sum = .003

P(F=Yes & A=No):

Combinatorial Explosion(The “Curse of Dimensionality”)

Assuming (unrealistically) only two values (Yes/No) per variable:

# Variables # of Rows in Table1 22 43 84 165 326 64: :20 1,048,576

Solution: Local Causality + Belief Propagation

Local Causality

Recover Joint From Prior & Posterior

B E P(A)T T .95T F .94F T .29F F .001

P(B).001

P(E).002

B E A ProbT T T .001*.002*.95 = .000001900T T F .001*.002*.05 = .000000100T F T .001*.998*.94 = .000938120T F F .001*.998*.06 = .000059880F T T .999*.002*.29 = .000579420F T F .999*.002*.71 = .001418580F F T .999*.998*.001 = .000997002F F F .999*.998*.999 = .996004998

Belief Propagation

• Consider just B → A → J

• P(J=T | B=T) = P(J=T | A=T) * P(A=T | B=T) • Then use Bayes’ Rule and marginalization to answer more sophisticated queries like

P(B=T | J=F & E=F & M=T)

Multiply-Connected Networks

Wet Grass

Cloudy

Rain Sprinkler

Clustering (“Mega Nodes”)

Cloudy Sprinkler Rain

Sprinkler Rain Wet Grass

Sprinkler Rain

A

B

D

F

E

C G

H

Junction Tree Algorithm (Huang & Darwiche 1994)

A

B

D

F

E

C G

H

A

B

D

F

E

C G

H

Junction Tree Algorithm (Huang & Darwiche 1994)

“Moralize””

A

B

D

F

E

C G

H

Junction Tree Algorithm (Huang & Darwiche 1994)

A

B

D

F

E

C G

H

Junction Tree Algorithm (Huang & Darwiche 1994)

A

B

D

F

E

C G

HTriangulate

Junction Tree Algorithm (Huang & Darwiche 1994)

A

B

D

F

E

C G

H

Junction Tree Algorithm (Huang & Darwiche 1994)

A

B

D

F

E

C G

H

ABD ADE

DEF

ACE CEG

EGH

AD

DE

AE CE

EG

“Message-Passing”

ABD ADE

DEF

ACE CEG

EGH

AD

DE

AE CE

EG

Observe A=T

“Message-Passing”

ABD ADE

DEF

ACE CEG

EGH

AD

DE

AE CE

EG

Pick a cluster containing A:

“Message-Passing”

ABD ADE

DEF

ACE CEG

EGH

AD

DE

AE CE

EG

Pass messages to propagate evidence: