bayesian networks chapter 14 section 1 – 2. bayesian networks a simple, graphical notation for...
TRANSCRIPT
Bayesian networks
Chapter 14
Section 1 – 2
Bayesian networks• A simple, graphical notation for conditional
independence assertions and hence for compact specification of full joint distributions
• Syntax:– a set of nodes, one per variable– a directed, acyclic graph (link ≈ "directly
influences")– a conditional distribution for each node given its
parents:P (Xi | Parents (Xi))
Example• Topology of network encodes conditional independence
assertions:
• Weather is independent of the other variables• Toothache and Catch are conditionally independent
given Cavity
Example
• I'm at work, neighbor John calls to say my alarm is ringing, but neighbor Mary doesn't call. Sometimes it's set off by minor earthquakes. Is there a burglar?
• Variables: Burglary, Earthquake, Alarm, JohnCalls, MaryCalls
• Network topology reflects "causal" knowledge:– A burglar can set the alarm off– An earthquake can set the alarm off– The alarm can cause Mary to call– The alarm can cause John to call
Example contd.
Compactness
• If each variable has no more than k parents, the complete network requires O(n · 2k) numbers
• I.e., grows linearly with n, vs. O(2n) for the full joint distribution
• For burglary net, 1 + 1 + 4 + 2 + 2 = 10 numbers (vs. 25-1 = 31)
Semantics
Constructing Bayesian networks
• 1. Choose an ordering of variables X1, … ,Xn
• 2. For i = 1 to n– add Xi to the network– select parents from X1, … ,Xi-1 such that
P (Xi | Parents(Xi)) = P (Xi | X1, ... Xi-1)
This choice of parents guarantees:
P (X1, … ,Xn) = πi =1 P (Xi | X1, … , Xi-1) (chain rule)
= πi =1P (Xi | Parents(Xi)) (by construction)
n
n
• Suppose we choose the ordering M, J, A, B, E
P(J | M) = P(J)?
•
Example
• Suppose we choose the ordering M, J, A, B, E
P(J | M) = P(J)?
P(A | J, M) = P(A | J)? P(A | J, M) = P(A)?
No
•
Example
• Suppose we choose the ordering M, J, A, B, E
P(J | M) = P(J)? No
P(A | J, M) = P(A | J)? P(A | J, M) = P(A)? No
P(B | A, J, M) = P(B | A)?
P(B | A, J, M) = P(B)?
•
Example
• Suppose we choose the ordering M, J, A, B, E
P(J | M) = P(J)?No
P(A | J, M) = P(A | J)? P(A | J, M) = P(A)? No
P(B | A, J, M) = P(B | A)? Yes
P(B | A, J, M) = P(B)? No
P(E | B, A ,J, M) = P(E | A)?
P(E | B, A, J, M) = P(E | A, B)?
Example
• Suppose we choose the ordering M, J, A, B, E
P(J | M) = P(J)?No
P(A | J, M) = P(A | J)? P(A | J, M) = P(A)? No
P(B | A, J, M) = P(B | A)? Yes
P(B | A, J, M) = P(B)? No
P(E | B, A ,J, M) = P(E | A)? No
P(E | B, A, J, M) = P(E | A, B)? Yes
•
Example
Example contd.
• Deciding conditional independence is hard in non-causal directions• (Causal models and conditional independence seem hardwired for
humans!)• Network is less compact
Example contd.
• Deciding conditional independence is hard in non-causal directions• (Causal models and conditional independence seem hardwired for
humans!)• Network is less compact: 1 + 2 + 4 + 2 + 4 = 13 numbers needed
Another example
Another example
Example from Medical Diagnostics
• Network represents a knowledge structure that models the relationship between medical difficulties, their causes and effects, patient information and diagnostic tests
Visit to Asia
Tuberculosis
Tuberculosisor Cancer
XRay Result Dyspnea
BronchitisLung Cancer
Smoking
Patient Information
Medical Difficulties
Diagnostic Tests
Example from Medical Diagnostics
• Relationship knowledge is modeled by deterministic functions, logic and conditional probability distributions
Patient Information
Diagnostic Tests
Visit to Asia
Tuberculosis
Tuberculosisor Cancer
XRay Result Dyspnea
BronchitisLung Cancer
SmokingTuber
Present
Present
Absent
Absent
Lung Can
Present
Absent
Present
Absent
Tub or Can
True
True
True
False
Medical DifficultiesTub or Can
True
True
False
False
Bronchitis
Present
Absent
Present
Absent
Present
0.90
0.70
0.80
0.10
Absent
0.l0
0.30
0.20
0.90
Dyspnea
Example from Medical Diagnostics
• Propagation algorithm processes relationship information to provide an unconditional or marginal probability distribution for each node
• The unconditional or marginal probability distribution is frequently called the belief function of that node
TuberculosisPresentAbsent
1.0499.0
XRay ResultAbnormalNormal
11.089.0
Tuberculosis or CancerTrueFalse
6.4893.5
Lung CancerPresentAbsent
5.5094.5
DyspneaPresentAbsent
43.656.4
BronchitisPresentAbsent
45.055.0
Visit To AsiaVisitNo Visit
1.0099.0
SmokingSmokerNonSmoker
50.050.0
• As a finding is entered, the propagation algorithm updates the beliefs attached to each relevant node in the network
• Interviewing the patient produces the information that “Visit to Asia” is “Visit”• This finding propagates through the network and the belief functions of several nodes are updated
TuberculosisPresentAbsent
5.0095.0
XRay ResultAbnormalNormal
14.585.5
Tuberculosis or CancerTrueFalse
10.289.8
Lung CancerPresentAbsent
5.5094.5
DyspneaPresentAbsent
45.055.0
BronchitisPresentAbsent
45.055.0
Visit To AsiaVisitNo Visit
100 0
SmokingSmokerNonSmoker
50.050.0
TuberculosisPresentAbsent
5.0095.0
XRay ResultAbnormalNormal
18.581.5
Tuberculosis or CancerTrueFalse
14.585.5
Lung CancerPresentAbsent
10.090.0
DyspneaPresentAbsent
56.443.6
BronchitisPresentAbsent
60.040.0
Visit To AsiaVisitNo Visit
100 0
SmokingSmokerNonSmoker
100 0
• Further interviewing of the patient produces the finding “Smoking” is “Smoker”• This information propagates through the network
TuberculosisPresentAbsent
0.1299.9
XRay ResultAbnormalNormal
0 100
Tuberculosis or CancerTrueFalse
0.3699.6
Lung CancerPresentAbsent
0.2599.8
DyspneaPresentAbsent
52.147.9
BronchitisPresentAbsent
60.040.0
Visit To AsiaVisitNo Visit
100 0
SmokingSmokerNonSmoker
100 0
• Finished with interviewing the patient, the physician begins the examination• The physician now moves to specific diagnostic tests such as an X-Ray, which results in a
“Normal” finding which propagates through the network• Note that the information from this finding propagates backward and forward through the arcs
TuberculosisPresentAbsent
0.1999.8
XRay ResultAbnormalNormal
0 100
Tuberculosis or CancerTrueFalse
0.5699.4
Lung CancerPresentAbsent
0.3999.6
DyspneaPresentAbsent
100 0
BronchitisPresentAbsent
92.27.84
Visit To AsiaVisitNo Visit
100 0
SmokingSmokerNonSmoker
100 0
• The physician also determines that the patient is having difficulty breathing, the finding “Present” is entered for “Dyspnea” and is propagated through the network
• The doctor might now conclude that the patient has bronchitis and does not have tuberculosis or lung cancer
Summary
• Bayesian networks provide a natural representation for (causally induced) conditional independence
• Topology + conditional probability = compact representation of joint distribution
• Generally easy for domain experts to construct