A. Darwiche
Query TypesQuery Types
• Pr: – Evidence: Pr(e)– Posterior marginals: Pr(x|e) for every X
• MPE: Most probable instantiation:– Instantiation y such that Pr(y|e) is maximal (Y = E)
• MAP: Maximum a posteriori hypothesis:– Intantiation y such that Pr(y|e) is maximal (Y is subset of E)
A. Darwiche
Pr: Posterior MarginalsPr: Posterior MarginalsBattery Age Alternator Fan Belt
BatteryCharge Delivered
Battery Power
Starter
Radio Lights Engine Turn Over
Gas Gauge
Gas
Fuel Pump Fuel Line
Distributor
Spark Plugs
Engine Start
A. Darwiche
Diagnosis ScenarioDiagnosis ScenarioBattery Age Alternator Fan Belt
BatteryCharge Delivered
Battery Power
Starter
Radio Lights Engine Turn Over
Gas Gauge
Gas
Fuel Pump Fuel Line
Distributor
Spark Plugs
Engine Start
ok on yes no
.001
ok off yes no
.090
A. Darwiche
Battery Age Alternator Fan Belt
BatteryCharge Delivered
Battery Power
Starter
Radio Lights Engine Turn Over
Gas Gauge
Gas
Fuel Pump Fuel Line
Distributor
Spark Plugs
Engine Start
MPE: Most Probable ExplanationMPE: Most Probable Explanation
A. Darwiche
Battery Age Alternator Fan Belt
BatteryCharge Delivered
Battery Power
Starter
Radio Lights Engine Turn Over
Gas Gauge
Gas
Fuel Pump Fuel Line
Distributor
Spark Plugs
Engine Start
MPE: Most Probable ExplanationMPE: Most Probable Explanation
A. Darwiche
MAP: Maximum a Posteriori MAP: Maximum a Posteriori HypothesisHypothesis
Battery Age Alternator Fan Belt
BatteryCharge Delivered
Battery Power
Starter
Radio Lights Engine Turn Over
Gas Gauge
Gas
Fuel Pump Fuel Line
Distributor
Spark Plugs
Engine Start
A. Darwiche
MAP: Maximum a Posteriori MAP: Maximum a Posteriori HypothesisHypothesis
Battery Age Alternator Fan Belt
BatteryCharge Delivered
Battery Power
Starter
Radio Lights Engine Turn Over
Gas Gauge
Gas
Fuel Pump Fuel Line
Distributor
Spark Plugs
Engine Start
A. Darwiche
Battery Age Alternator Fan Belt
BatteryCharge Delivered
Battery Power
Starter
Radio Lights Engine Turn Over
Gas Gauge
Gas
Fuel Pump Fuel Line
Distributor
Spark Plugs
Engine Start
MAP: Maximum a Posteriori MAP: Maximum a Posteriori HypothesisHypothesis
A. Darwiche
false
A B
true
false
A
.3
.7
ØABA
.1truetrue.9true false
ØB
.2falsefalse
.8false true
**
* *
λ~b λ~aλbλa
+
+ +
* * * *
.3 .1 .9 .8 .2 .7
Factoring
A. Darwiche
NotationNotation• A binary variable X:
– is variable with two values (true, false)– x is short notation for X=true– ~x is short notation for X=false
• If X is a variable with parents Y and Z, then:
represents the probability Pr(X=x | Y=y, Z=y)
• If X is a binary variable with parents Y and Z (also binary), then:
represents the probability Pr(X=true | Y=false, Z=true)
A. Darwiche
NotationNotation• An instantiation is a set of variables with their values:
– X=true,Y=false, Z=true is an instantiation– A=a, B=b, C=c is an instantiation
• x, ~y, z is short notation for the instantiationX=true, Y=false, Z=true
• a,b,c is short notation for the instantiation A=a, B=b, C=c
• Two instantiations are inconsistent iff they assign different values to the same variable:– x,~y,z and x,y,z are inconsistent– x,~y,z and a,b,c are consistent
A. Darwiche
Pr(a) = .03 + .27 = .3
Joint Probability DistributionJoint Probability Distribution
false
false
B
.03
.27
A
.56
.14
truetrue
true
false
false
false
Pr
false
true
A. DarwichePr(~b) = .27 + .14 = .41
false
false
B
.03
.27
A
.56
.14
truetrue
true
false
false
false
Pr
false
true
Joint Probability DistributionJoint Probability Distribution
A. Darwiche
F = .03λaλb + .27λaλ~b + .56λ~aλb + .14λ~a λ~b
false
false
B
.03
A
truetrue
true
false
false
false
Pr
false
true
.27
.14
.56
λaλb .03
λaλ~b .27
λ~aλb .56
λ~aλ~b .14
λa λb …are called evidence indicators
F is called the polynomial of the given probability distribution
Evidence IndicatorsEvidence Indicators
A. Darwiche
Computing ProbabilitiesComputing Probabilities
• To compute the probability of instantiation e:Evaluate polynomial F while replacing each indicator
-by 1 if the instantiation is consistent with the indicator;-by 0 if the instantiation is inconsistent with the indicator
• Examples:– Indicator λa is consistent with instantiation a,~b,c
– Indicator λb is inconsistent with instantiation a,~b,c
– Indicator λd is consistent with instantiation a,~b,c
– Indicator λ~d is consistent with instantiation a,~b,c
A. Darwiche
F = .03λaλb + .27λaλ~b + .56λ~aλb + .14λ~a λ~b
Computing ProbabilitiesComputing Probabilities
To compute the probability of instantiation a, ~b:
F(a,~b) = .03*1*0 + .27*1*1 + .56*0*0 + .14*0*1 = .27
To compute the probability of instantiation ~a:
F(~a) = .03*0*1 + .27*0*1 + .56*1*1 + .14*1*1 = .70
A. Darwiche
A B
true
false
A
.3
.7
ØABA
.1truetrue.9true false
ØB
.2falsefalse
.8false true
false
false
BA
truetrue
true
false
false
false
Pr
false
true
.03=.3*.1
.27=.3*.9
56=.7*.8
.14=.7*.2
A. Darwiche
A B
true
false
A ØABA
truetruetrue false
ØB
falsefalsefalse true
θaθ~a
θ b|a
θ~b|a
θ b|~a
θ ~b|~a
false
false
BA
truetrue
true
false
false
false
Pr
false
true
θa θ b|a
θa θ~b|a
θ~a θ b|~a
θ~a θ ~b|~a
A. Darwiche
A B
true
false
A ØABA
truetruetrue false
ØB
falsefalsefalse true
θaθ~a
θ b|a
θ~b|a
θ b|~a
θ ~b|~a
false
false
BA
truetrue
true
false
false
false
Pr
false
true
λaλb θa θ b|a
λaλ~b θa θ~b|a
λ~aλb θ~a θ b|~a
λ~aλ~b θ~a θ ~b|~a
A. Darwiche
A B
true
false
A ØABA
truetruetrue false
ØB
falsefalsefalse true
θaθ~a
θ b|a
θ~b|a
θ b|~a
θ ~b|~a
false
BA
truetrue
true
false
false
false
Pr
false
true
λaλb θa θ b|a
λaλ~b θa θ~b|a
λ~aλb θ~a θ b|~a
λ~aλ~b θ~a θ ~b|~a
λa λb θa θb|a+ λa λ~b θa θ~b|a+ λ~aλb θ~a θb|~a+ λ~a λ~b θ~a θ~b|~a
A. Darwiche
A B
truefalse
A
θaθ~a
ØA
false
B
θ b|aθ~b|a
A
θ b|~aθ ~b|~a
truetruetrue
false
falsefalse
ØB
falsetrue
λa λb θa θb|a+ λa λ~b θa θ~b|a+ λ~aλb θ~a θb|~a+ λ~a λ~b θ~a θ~b|~a
The Polynomial of a Bayesian The Polynomial of a Bayesian NetworkNetwork
A. Darwiche
F = λa λb λc λd θa θb|a θc|a θd|bc +
λa λb λc λ~d θa θb|a θc|a θ~d|bc +
….
A
B
C
D
The Polynomial of a Bayesian The Polynomial of a Bayesian NetworkNetwork
A. Darwiche
Arithmetic CircuitArithmetic Circuit λa λb θa θb|a+ λa λ~b θa θ~b|a+ λ~aλb θ~a θb|~a+ λ~a λ~b θ~a θ~b|~a
+
**
* *
λ~b λ~aλbλa
+ +
* * * *
θa θab θa~b θ~ab θ~a~b θ~a
Factoring
A. Darwiche
**
* *
λ~b λ~aλbλa
+
+ +
* * * *
θa θab θa~b θ~ab θ~a~b θ~a
1 1 1 0
.3
.3 .1 .9 .8 .2 0
.3 0
1 1
Arithmetic CircuitArithmetic Circuit
.3 .1 .9 .8 .2 .7
Pr(a)
A. Darwiche
false
A B
true
false
A
.3
.7
ØABA
.1truetrue.9true false
ØB
.2falsefalse
.8false true
**
* *
λ~b λ~aλbλa
+
+ +
* * * *
θa θb|a θ~b|a θb|~a θ~b|~a θ~a
Factoring
A. Darwiche
Factoring the Polynomial of a Factoring the Polynomial of a Bayesian NetworkBayesian Network
S1
T
S2 S3 Sn…
õtòtQ
i=1n (õsiòsijt+õøsiòø sijt)
+õø tòø tQ
i=1n (õsiòsijø t+õøsiòø sijø t)
A. Darwiche
Primitive Platforms (embedded)
Embedding Probabilistic Embedding Probabilistic Reasoning SystemsReasoning Systems
Sophisticated Platform(desktop)
compiler
Eval Eval EvalA. Circuit A. Circuit A. Circuit
A. Darwiche
TreeWidthTreeWidth(Measures connectivity of Networks)(Measures connectivity of Networks)
Higher treewidth
A. Darwiche
TreeWidthTreeWidth(Measures connectivity of Networks)(Measures connectivity of Networks)
Singly-connected network(polytree)
Multiply-connected networks
A. Darwiche
TreewidthTreewidth• The treewidth of a polytree is m, where m is
the maximum number of parents that any node
• If each node has at most one parent, the polytree is called a tree
• The treewidth of a tree is 1