probabilistic reasoning; network-based reasoning

Probabilistic Reasoning;Probabilistic Reasoning;Network-based reasoning Network-based reasoning

Set 7Set 7

ICS 179, Spring ICS 179, Spring 20102010

Chavurah 5/8/2010

Propositional Reasoning

If Alex goes, then Becky goes: If Chris goes, then Alex goes:

Question: Is it possible that Chris goes to

the party but Becky does not?

Example: party problem

BA

A C

e?satisfiabl ,,

theIs

C B, ACBA

theorynalpropositio

= A

= B

= C

= A

Chavurah 5/8/2010

Probabilistic ReasoningParty example: the weather effect

Alex is-likely-to-go in bad weather Chris rarely-goes in bad weather Becky is indifferent but unpredictable

Questions: Given bad weather, which group of

individuals is most likely to show up at the party?

What is the probability that Chris goes to the party but Becky does not?

P(W,A,C,B) = P(B|W) · P(C|W) · P(A|W) · P(W)

P(A,C,B|W=bad) = 0.9 · 0.1 · 0.5

P(A|W=bad)=.9W A

P(C|W=bad)=.1W C

P(B|W=bad)=.5W B

W

P(W)

P(A|W)P(C|W)P(B|W)

B CA

W A P(A|W)

good 0 .01

good 1 .99

bad 0 .1

bad 1 .9

Chavurah 5/8/2010

Mixed Probabilistic and Deterministic networks

P(C|W)P(B|W)

P(W)

P(A|W)

W

B A C

Query:Is it likely that Chris goes to the party if Becky does not but the weather is bad?

PN

CN

Semantics?

Algorithms?),,|,( ACBAbadwBCP

A→B C→A

B A CP(C|W)P(B|W)

P(W)

P(A|W)

W

B A C

A→B C→A

B A C

5

The problem

All men are mortal T

All penguins are birds T

…

Socrates is a man

Men are kind p1

Birds fly p2

T looks like a penguin

Turn key –> car starts P_n

Q: Does T fly?P(Q)?

True propositions

Uncertain propositions

Logic?....but how we handle exceptionsProbability: astronomical

6

12

13

Alpha and beta are events

15

Burglary is independent of EarthquakeBurglary is independent of Earthquake

Earthquake is independent of burglary

20

26

27

28

29

30

31

32

33

34

35

Bayesian Networks: Representation

= P(S) P(C|S) P(B|S) P(X|C,S) P(D|C,B)

lung Cancer

Smoking

X-ray

Bronchitis

DyspnoeaP(D|C,B)

P(B|S)

P(S)

P(X|C,S)

P(C|S)

P(S, C, B, X, D)

Conditional Independencies Efficient Representation

Θ) (G,BN

CPD: C B D=0 D=10 0 0.1 0.90 1 0.7 0.31 0 0.8 0.21 1 0.9 0.1

Bayesian networks

Chapter 14 , Russel and Norvig

Section 1 – 2

Outline

Syntax Semantics

Example Topology of network encodes conditional

independence assertions:

Weather is independent of the other variables Toothache and Catch are conditionally

independent given Cavity

Example I'm at work, neighbor John calls to say my alarm is ringing, but

neighbor Mary doesn't call. Sometimes it's set off by minor earthquakes. Is there a burglar?

Variables: Burglary, Earthquake, Alarm, JohnCalls, MaryCalls

Network topology reflects "causal" knowledge: A burglar can set the alarm off An earthquake can set the alarm off The alarm can cause Mary to call The alarm can cause John to call

Example contd.

Compactness A CPT for Boolean Xi with k Boolean parents has 2k rows for

the combinations of parent values

Each row requires one number p for Xi = true(the number for Xi = false is just 1-p)

If each variable has no more than k parents, the complete network requires O(n · 2k) numbers

I.e., grows linearly with n, vs. O(2n) for the full joint distribution

For burglary net, 1 + 1 + 4 + 2 + 2 = 10 numbers (vs. 25-1 = 31)

SemanticsThe full joint distribution is defined as the product of the

local conditional distributions:P (X1, … ,Xn) = πi = 1 P (Xi | Parents(Xi))

e.g., P(j m a b e)= P (j | a) P (m | a) P (a | b, e) P (b) P (e)

n

Constructing Bayesian networks 1. Choose an ordering of variables X1, … ,Xn

2. For i = 1 to n add Xi to the network select parents from X1, … ,Xi-1 such that

P (Xi | Parents(Xi)) = P (Xi | X1, ... Xi-1)

This choice of parents guarantees:P (X1, … ,Xn) = πi =1 P (Xi | X1, … , Xi-1)

= πi =1P (Xi | Parents(Xi))

(by construction)(chain rule)

Suppose we choose the ordering M, J, A, B, E

P(J | M) = P(J)?

Example

Suppose we choose the ordering M, J, A, B, E

P(J | M) = P(J)?P(A | J, M) = P(A | J)? P(A | J, M) = P(A)?

No

Example

Suppose we choose the ordering M, J, A, B, E

P(J | M) = P(J)?P(A | J, M) = P(A | J)? P(A | J, M) = P(A)? NoP(B | A, J, M) = P(B | A)? P(B | A, J, M) = P(B)? No

Example

Suppose we choose the ordering M, J, A, B, E

P(J | M) = P(J)?P(A | J, M) = P(A | J)? P(A | J, M) = P(A)? NoP(B | A, J, M) = P(B | A)? YesP(B | A, J, M) = P(B)? NoP(E | B, A ,J, M) = P(E | A)?P(E | B, A, J, M) = P(E | A, B)?

No

Example

Suppose we choose the ordering M, J, A, B, E

P(J | M) = P(J)?P(A | J, M) = P(A | J)? P(A | J, M) = P(A)? NoP(B | A, J, M) = P(B | A)? YesP(B | A, J, M) = P(B)? NoP(E | B, A ,J, M) = P(E | A)? NoP(E | B, A, J, M) = P(E | A, B)? Yes No

Example

Example contd.

Deciding conditional independence is hard in noncausal directions

(Causal models and conditional independence seem hardwired for humans!)

Network is less compact: 1 + 2 + 4 + 2 + 4 = 13 numbers needed