exact reasoning: conditioning search and hybrids
DESCRIPTION
Exact Reasoning: conditioning search and hybrids. COMPSCI 276 Fall 2007. Probabilistic Inference Tasks. Belief updating: Finding most probable explanation (MPE) Finding maximum a-posteriory hypothesis Finding maximum-expected-utility (MEU) decision. Belief Updating. Smoking. - PowerPoint PPT PresentationTRANSCRIPT
Exact Reasoning:conditioning search and hybrids
COMPSCI 276Fall 2007
CS 276 2
Probabilistic Inference Tasks
X/A
a
*k
*1 e),xP(maxarg)a,...,(a
evidence)|xP(X)BEL(X iii
Belief updating:
Finding most probable explanation (MPE)
Finding maximum a-posteriory hypothesis
Finding maximum-expected-utility (MEU) decision
e),xP(maxarg*xx
)xU(e),xP(maxarg)d,...,(d X/D
d
*k
*1
variableshypothesis: XA
function utilityx variablesdecision
: )( :
UXD
CS 276 3
Belief Updating
lung Cancer
Smoking
X-ray
Bronchitis
Dyspnoea
P (lung cancer=yes | smoking=no, dyspnoea=yes ) = ?
CS 276 4
Conditioning generates the probability tree
0
),|(),|()|()|()()0,(ebcb
cbePbadPacPabPaPeaP
Complexity of conditioning: exponential time, linear space
CS 276 5
Conditioning+Elimination
0
),|(),|()|()|()()0,(edcb
cbePbadPacPabPaPeaP
Idea: conditioning until of a (sub)problem gets small*w
CS 276 6
Loop-cutset decomposition You condition until you get a
polytree
B
CB
F
A
BCB
F
A=0 A=0 A=0
BCB
F
A=1 A=1 A=1
P(B|F=0) = P(B, A=0|F=0)+P(B,A=1|F=0)
A=0 A=1
Loop-cutset method is time exp in loop-cutset sizeAnd linear space
CS 276 9
OR search spaceA
D
B C
E
F
0 1 0 1 0 1 0 1
0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
0 1 0 1
E
C
F
D
B
A 0 1
Ordering: A B E C D F
CS 276 10
AND/OR search space
AOR
0AND 1
BOR B
0AND 1 0 1
EOR C E C E C E C
OR D F D F D F D F D F D F D F D F
AND 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
AND 0 10 1 0 10 1 0 10 1 0 10 1
A
D
B C
E
F
A
D
B
CE
F
Primal graph DFS tree
A
D
B C
E
F
A
D
B C
E
F
CS 276 12
OR vs AND/OR AOR
0AND 1
BOR B
0AND 1 0 1
EOR C E C E C E C
OR D F D F D F D F D F D F D F D F
AND 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
AND 0 10 1 0 10 1 0 10 1 0 10 1
E 0 1 0 1 0 1 0 1
0C 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
F 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
D 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
0B 1 0 1
A 0 1
E 0 1 0 1 0 1 0 1
0C 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
F 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
D 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
0B 1 0 1
A 0 1
AND/OR
OR
A
D
B C
E
F
A
D
B
CE
F
1
1
1
0
1
0
CS 276 13
AND/OR vs. OR
E 0 1 0 1 0 1 0 1
0C 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
F 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
D 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
0B 1 0 1
A 0 1
AOR
0AND 1
BOR B
0AND 1 0 1
EOR C E C E C E C
OR D F D F D F D F D F D F D F D F
AND 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
AND 0 10 1 0 10 1 0 10 1 0 10 1
E 0 1 0 1 0 1 0 1
0C 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
F 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
D 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
0B 1 0 1
A 0 1
E 0 1 0 1 0 1 0 1
0C 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
F 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
D 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
0B 1 0 1
A 0 1
AND/OR
OR
A
D
B C
E
F
A
D
B
CE
F
AND/OR size: exp(4), OR size exp(6)
CS 276 14
AND/OR vs. OR
E 0 1 0 1 0 1 0 1
0C 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
F 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
D 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
0B 1 0 1
A 0 1
AOR
0AND 1
BOR B
0AND 1 0 1
EOR C E C E C E C
OR D F D F D F D F D F D F D F D F
AND 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
AND 0 10 1 0 10 1 0 10 1 0 10 1
E 0 1 0 1 0 1 0 1
0C 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
F 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
D 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
0B 1 0 1
A 0 1
E 0 1 0 1 0 1 0 1
0C 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
F 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
D 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
0B 1 0 1
A 0 1
AND/OR
OR
A
D
B C
E
F
A
D
B
CE
F
No-goods(A=1,B=1)(B=0,C=0)
CS 276 15
AND/OR vs. OR
E 0 1 0 1 0 1
C 1 1 0 1 0 1 1 1
F 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
D 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
0B 1 0
A 0 1
AOR
0AND 1
BOR B
0AND 1 0
EOR C E C E C
OR D F D F D F D F
AND 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
AND 10 1 0 10 1 10 1
E 0 1 0 1 0 1
C 1 1 0 1 0 1 1 1
F 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
D 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
0B 1 0
A 0 1
E 0 1 0 1 0 1
C 1 1 0 1 0 1 1 1
F 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
D 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
0B 1 0
A 0 1
AND/OR
OR
A
D
B C
E
F
A
D
B
CE
F
(A=1,B=1)(B=0,C=0)
CS 276 16
OR space vs. AND/OR space
width heightOR space AND/OR space
time(sec.) nodes backtracks time(sec.) AND nodes OR nodes
5 10 3.154 2,097,150 1,048,575 0.03 10,494 5,247
4 9 3.135 2,097,150 1,048,575 0.01 5,102 2,551
5 10 3.124 2,097,150 1,048,575 0.03 8,926 4,463
4 10 3.125 2,097,150 1,048,575 0.02 7,806 3,903
5 13 3.104 2,097,150 1,048,575 0.1 36,510 18,255
5 10 3.125 2,097,150 1,048,575 0.02 8,254 4,127
6 9 3.124 2,097,150 1,048,575 0.02 6,318 3,159
5 10 3.125 2,097,150 1,048,575 0.02 7,134 3,567
5 13 3.114 2,097,150 1,048,575 0.121 37,374 18,687
5 10 3.114 2,097,150 1,048,575 0.02 7,326 3,663
CS 276 17
AND/OR search tree for graphical models
The AND/OR search tree of R relative to a spanning-tree, T, has:
Alternating levels of: OR nodes (variables) and AND nodes (values)
Successor function: The successors of OR nodes X are all its consistent values along
its path The successors of AND <X,v> are all X child variables in T
A solution is a consistent subtree Task: compute the value of the root node
0
A
B
0
E
OR
AND
OR
AND
OR
AND
C
0
OR
AND
D
0 1
F
0 1
1
D
0 1
F
0 1
0 1
1
E C
0
D
0 1
F
0 1
1
D
0 1
F
0 1
0 1
1
B
0
E C
0
D
0 1
F
0 1
1
D
0 1
F
0 1
0 1
1
E C
0
D
0 1
F
0 1
1
D
0 1
F
0 1
0 1
A
D
B C
E
F
A
D
B
CE
F
CS 276 18
From DFS trees to pseudo-trees (Freuder 85, Bayardo 95)
(a) Graph
4 61
3 2 7 5
(b) DFS treedepth=3
(c) pseudo- treedepth=2
(d) Chaindepth=6
4 6
1
3
2 7
5 2 7
1
4
3 5
6
4
6
1
3
2
7
5
CS 276 19
From DFS trees to Pseudo-trees
a
1
2
a
3
a
4
OR
AND
OR
AND
OR
AND
AND
OR
ba
b c
c
b
4
ba c
c
4
ba c
b
3
a
4
ba c
b
4
ba c
c
4
ba c
c
3
a
4
ba c
b
4
ba c
c
4
ba c
7
a
5
a
6
ba c
b
6
ba c
c
6
ba c
b
5
a
6
ba c
b
6
ba c
c
6
ba c
c
5
a
6
ba c
b
6
ba c
c
6
ba c
a
1
3
a
4
b c
OR
AND
OR
AND
OR
AND
b c
a
2
b ca
b
4
b ca
2
b ca
c
4
b ca
2
b ca
5
a
7
b ca
6
b ca
b
7
b ca
6
b ca
c
7
b ca
6
b ca
DFS tree
depth = 3
pseudo- tree
depth = 2
CS 276 20
Finding min-depth backbone trees
Finding min depth DFS, or pseudo tree is NP-complete, but:
Given a tree-decomposition whose tree-width is w*, there exists a pseudo tree T of G whose depth, satisfies (Bayardo and Mirankar, 1996, bodlaender and Gilbert, 91):
m <= w* log n,
CS 276 21
Generating pseudo-trees from Bucket trees
FA
C
B D
E
E
A
C
B
D
F (AF) (EF)
(A)
(AB)
(AC) (BC)
(AE)
(BD) (DE)
Bucket-tree based on dd: A B C E D F
E
A
C
B
D
F
Induced graph
E
A
C
B
D F
Bucket-tree used as pseudo-tree
AND
AND
AND
AND
0 1
BOR B
0 1 0 1
COR E C E C E C E
OR D F D F D F D F D F D F D F D F
0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
0 10 1 0 10 1 0 10 1 0 10 1
AOR
AND/OR search tree
Bucket-tree
ABE
A
ABC
AB
BDE AEF
bucket-A
bucket-E
bucket-B
bucket-C
bucket-D bucket-F
(AE) (BE)
CS 276 23
AND/OR Search-tree properties (k = domain size, m = pseudo-tree depth. n = number of
variables)
Theorem: Any AND/OR search tree based on a pseudo-tree is sound and complete (expresses all and only solutions)
Theorem: Size of AND/OR search tree is O(n km)Size of OR search tree is O(kn)
Theorem: Size of AND/OR search tree can be bounded by O(exp(w* log n))
Related to: (Freuder 85; Dechter 90, Bayardo et. al. 96, Darwiche 1999, Bacchus 2003)
When the pseudo-tree is a chain we get an OR space
CS 276 24
Tasks and value of nodes V( n) is the value of the tree T(n) for the task:
Counting: v(n) is number of solutions in T(n) Consistency: v(n) is 0 if T(n) inconsistent, 1 othewise. Optimization: v(n) is the optimal solution in T(n) Belief updating: v(n), probability of evidence in T(n). Partition function: v(n) is the total probability in
T(n).
Goal: compute the value of the root node recursively using dfs search of the AND/OR tree.
Theorem: Complexity of AO dfs search is Space: O(n) Time: O(n km) Time: O(exp(w* log n))
CS 276 26
A
C
B
DE
A
D
B C
E
0
A
B
0
E D
0 1
C
0 1
C
0 1 0 1
1
E D
0 1
C
0 1
C
0 1 0 1
1
B
0
E D
0 1
C
0 1
C
0 1 0 1
1
E D
0 1
C
0 1
C
0 1 0 1
Belief-updating on example
CS 276 27
0
A
B
0
E D
0 1
C
0 1
C
0 1 0 1
1
E D
0 1
C
0 1
C
0 1 0 1
1
B
0
E D
0 1
C
0 1
C
0 1 0 1
1
E D
0 1
C
0 1
C
0 1 0 1
PA(0) PA(1)
PB|A(0|0) PB|A(1|0) PB|A(0|1) PB|A(1|1)
PE|AB(0|0,0) PE|AB(1|0,0) PE|AB(0|0,1) PE|AB(1|0,1) PE|AB(0|1,0) PE|AB(1|1,0) PE|AB(0|1,1) PE|AB(1|1,1)
PD|BC(0|0,0)×PC|A(0|0)
A
C
B
DE
A
D
B C
E
CS 276 28
0
A
B
0
E D
0 1
C
0
0 1
1
C
0 1
1
E D
0 1
C
0
0 1
1
C
0 1
P(A=0)
P(B=0|A=0) P(B=1|A=0)
P(E=0|A=0,B=0)
P(D=0|B=0,C=0)×P(C=0|A=0)
P(D=1|B=1,C=1)×P(C=1|A=0)
P(D=0|B=0,C=1)×P(C=1|A=0)
P(D=1|B=0,C=0)×P(C=0|A=0)
P(D=1|B=0,C=1)×P(C=1|A=0)
P(D=0|B=1,C=0)×P(C=0|A=0)
P(D=0|B=1,C=1)×P(C=1|A=0)
P(D=1|B=1,C=0)×P(C=0|A=0)
P(E=1|A=0,B=0) P(E=0|A=0,B=1) P(E=1|A=0,B=1)
A
C
B
DE
A
D
B C
E
CS 276 29
0
A
B
0
E C
0 1
D
0
0 1
1
D
0 1
1
E C
0 1
D
0
0 1
1
D
0 1
P(A=0)
P(B=0|A=0) P(B=1|A=0)
P(E=0|A=0,B=0)
P(D=0|B=0,C=0) P(D=1|B=1,C=1)P(D=0|B=0,C=1) P(D=1|B=0,C=0) P(D=1|B=0,C=1) P(D=0|B=1,C=0) P(D=0|B=1,C=1) P(D=1|B=1,C=0)
P(E=1|A=0,B=0) P(E=0|A=0,B=1) P(E=1|A=0,B=1)
P(C=0|A=0) P(C=1|A=0) P(C=0|A=0) P(C=1|A=0)
(d)
CS 276 30
AND/OR Tree DFS Algorithm (Belief Updating)
AND node: Combination operator (product)
OR node: Marginalization operator (summation)
Value of node = updated belief for sub-problem below
0
A
B
0
E
OR
AND
OR
AND
OR
AND
C
0
OR
AND
D
0 1
1
D
0 1
0 1
1
E C
0
D
0 1
1
D
0 1
0 1
1
B
0
E C
0
D
0 1
1
D
0 1
0 1
1
E C
0
D
0 1
1
D
0 1
0 1
A
D
B C
E
A
D
B
CE
B C D=0
D=1
0 0 .2 .80 1 .1 .91 0 .3 .71 1 .5 .5
),|( CBDP
.7.8 .9 .5 .7.8 .9 .5
Evidence: D=1
A B E=0
E=1
0 0 .4 .60 1 .5 .51 0 .7 .31 1 .2 .8
),|( BAEP
Evidence: E=0
.4 .5 .7 .2
A B=0
B=1
0 .4 .61 .1 .9
)|( ABPA C=
0C=1
0 .2 .81 .7 .3
)|( ACPA P(A
)0 .61 .4
)(AP
.2 .8 .2 .8 .1 .9 .1 .9
.4 .6 .1 .9
.6 .4
.8 .9
.8 .9
.7 .5
.7 .5
.8 .9
.8 .9
.7 .5
.7 .5
.4 .5 .7 .2.88 .54 .89 .52
.352 .27 .623 .104
.3028 .1559
.24408
.3028 .1559
Result: P(D=1,E=0)
CS 276 31
Complexity of AND/OR Tree Search
AND/OR tree OR tree
Space O(n) O(n)
Time
O(n km)O(n kw* log n)
[Freuder & Quinn85], [Collin, Dechter & Katz91], [Bayardo & Miranker95], [Darwiche01]
O(kn)
k = domain sizem = depth of pseudo-treen = number of variablesw*= treewidth
CS 276 32
From Search Trees to Search Graphs
Any two nodes that root identical subtrees (subgraphs) can be merged
CS 276 33
From Search Trees to Search Graphs
Any two nodes that root identical subtrees (subgraphs) can be merged
CS 276 35
AND/OR Tree
A
D
B C
E
F
A
D
B
CE
F
G H
J
K
G
H
J
KAOR
0AND 1
BOR B
0AND 1 0 1
EOR C E C E C E C
OR D F D F D F D F D F D F D F D F
AND
AND 0 10 1 0 10 1 0 10 1 0 10 1
OR
OR
AND
AND
0
G
H H
0 1 0 1
0 1
1
G
H H
0 1 0 1
0 1
0
J
K K
0 1 0 1
0 1
1
J
K K
0 1 0 1
0 1
0
G
H H
0 1 0 1
0 1
1
G
H H
0 1 0 1
0 1
0
J
K K
0 1 0 1
0 1
1
J
K K
0 1 0 1
0 1
0
G
H H
0 1 0 1
0 1
1
G
H H
0 1 0 1
0 1
0
J
K K
0 1 0 1
0 1
1
J
K K
0 1 0 1
0 1
0
G
H H
0 1 0 1
0 1
1
G
H H
0 1 0 1
0 1
0
J
K K
0 1 0 1
0 1
1
J
K K
0 1 0 1
0 1
0
G
H H
0 1 0 1
0 1
1
G
H H
0 1 0 1
0 1
0
J
K K
0 1 0 1
0 1
1
J
K K
0 1 0 1
0 1
0
G
H H
0 1 0 1
0 1
1
G
H H
0 1 0 1
0 1
0
J
K K
0 1 0 1
0 1
1
J
K K
0 1 0 1
0 1
0
G
H H
0 1 0 1
0 1
1
G
H H
0 1 0 1
0 1
0
J
K K
0 1 0 1
0 1
1
J
K K
0 1 0 1
0 1
0
G
H H
0 1 0 1
0 1
1
G
H H
0 1 0 1
0 1
0
J
K K
0 1 0 1
0 1
1
J
K K
0 1 0 1
0 1
CS 276 36
An AND/ORgraph
A
D
B C
E
F
A
D
B
CE
F
G H
J
K
G
H
J
KAOR
0AND 1
BOR B
0AND 1 0 1
EOR C E C E C E C
OR D F D F D F D F D F D F D F D F
AND
AND 0 10 1 0 10 1 0 10 1 0 10 1
OR
OR
AND
AND
0
G
H H
0 1 0 1
0 1
1
G
H H
0 1 0 1
0 1
0
J
K K
0 1 0 1
0 1
1
J
K K
0 1 0 1
0 1
CS 276 37
Merging Based on ContextOne way of recognizing nodes that can be
merged:
context (X) = ancestors of X in pseudo tree that are connected to X, or to descendants of X
[ ]
[A]
[AB]
[AE][BC]
[AB]
A
D
B
EC
F
pseudo tree
A
E
C
B
F
D
A
E
C
B
F
D
CS 276 38
AND/OR Search Graph
A
E
C
B
F
D
A
D
B
EC
F
A B C RABC
0 0 0 10 0 1 10 1 0 00 1 1 11 0 0 11 0 1 11 1 0 11 1 1 0
A B E RABE
0 0 0 10 0 1 00 1 0 10 1 1 11 0 0 01 0 1 11 1 0 11 1 1 0
A E F RAEF
0 0 0 00 0 1 10 1 0 10 1 1 11 0 0 11 0 1 11 1 0 11 1 1 0
B C D RBCD
0 0 0 10 0 1 10 1 0 10 1 1 01 0 0 11 0 1 01 1 0 11 1 1 1
Constraint Satisfaction – Counting Solutions
[ ]
[A]
[AB]
[AE][BC]
[AB]
Context
pseudo tree
context minimal graph
AOR
0AND
BOR
0AND
OR E
OR
AND
AND 0 1
C
D D
0 1 0 1
0 1
1
E
F F
0 1 0 1
0 1
C
D D
0 1 0 1
0 1
1
B
0
E
F F
0 1 0 1
0 1
C
0 1
1
E
0 1
C
0 1
CS 276 39
.8 .9 .7.7.8
AND/OR Tree DFS Algorithm (Belief Updating)
AND node: Combination operator (product)OR node: Marginalization operator (summation)
Value of node = updated belief for sub-problem below
0
A
B
0
E C
0
D
0 1
1
D
0 1
0 1
1
E C
0
D
0 1
1
D
0 1
0 1
1
B
0
E C
0
D
0 1
1
D
0 1
0 1
1
E C
0
D
0 1
1
D
0 1
0 1
B C D=0
D=1
0 0 .2 .80 1 .1 .91 0 .3 .71 1 .5 .5
),|( CBDP
.7.8 .9 .5 .7.8 .9 .5
Evidence: D=1
A B E=0
E=1
0 0 .4 .60 1 .5 .51 0 .7 .31 1 .2 .8
),|( BAEP
Evidence: E=0
.4 .5 .7 .2
A B=0
B=1
0 .4 .61 .1 .9
)|( ABPA C=
0C=1
0 .2 .81 .7 .3
)|( ACPA P(A
)0 .61 .4
)(AP
.2 .8 .2 .8 .1 .9 .1 .9
.4 .6 .1 .9
.6 .4
.9
.8 .9
.5
.7 .5 .8 .9
.5
.7 .5
.4 .5 .7 .2.88 .54 .89 .52
.352 .27 .623 .104
.3028 .1559
.24408
.3028 .1559
Result: P(D=1,E=0)
A
D
B
CE
[ ]
[A]
[AB]
[BC]
[AB]
ContextA
D
B C
E
CS 276 40
.7.8
AND/OR Graph DFS Algorithm (Belief Updating)
0
A
B
0
E C
0
D
0 1
1
D
0 1
0 1
1
E C
0
D
0 1
1
D
0 1
0 1
1
B
0
E C
0 10 1
1
E C
0 10 1
A
D
B C
E
B C D=0
D=1
0 0 .2 .80 1 .1 .91 0 .3 .71 1 .5 .5
),|( CBDP.7.8 .9 .5
Evidence: D=1
A B E=0
E=1
0 0 .4 .60 1 .5 .51 0 .7 .31 1 .2 .8
),|( BAEP
Evidence: E=0
.4 .5 .7 .2
A B=0
B=1
0 .4 .61 .1 .9
)|( ABPA C=
0C=1
0 .2 .81 .7 .3
)|( ACPA P(A
)0 .61 .4
)(AP
.2 .8 .2 .8 .1 .9 .1 .9
.4 .6 .1 .9
.6 .4
.9
.8 .9
.5
.7 .5 .8 .9 .7 .5
.4 .5 .7 .2.88 .54 .89 .52
.352 .27 .623 .104
.3028 .1559
.24408
.3028 .1559
A
D
B
CE
[ ]
[A]
[AB]
[BC]
[AB]
Context
B C Value0 0 .80 1 .91 0 .71 1 .1
Cache table for D
Result: P(D=1,E=0)
CS 276 41
AND/OR context minimal graph
C
0
K
0
H
0
L
0 1
N N
0 1 0 1
F F F
1 1
0 1 0 1
F
G
0 1
1
A
0 1
B B
0 1 0 1
E EE E
0 1 0 1
J JJ J
0 1 0 1
A
0 1
B B
0 1 0 1
E EE E
0 1 0 1
J JJ J
0 1 0 1
G
0 1
G
0 1
G
0 1
M
0 1
M
0 1
M
0 1
M
0 1
P
0 1
P
0 1
O
0 1
O
0 1
O
0 1
O
0 1
L
0 1
N N
0 1 0 1
P
0 1
P
0 1
O
0 1
O
0 1
O
0 1
O
0 1
D
0 1
D
0 1
D
0 1
D
0 1
K
0
H
0
L
0 1
N N
0 1 0 1
1 1
A
0 1
B B
0 1 0 1
E EE E
0 1 0 1
J JJ J
0 1 0 1
A
0 1
B B
0 1 0 1
E EE E
0 1 0 1
J JJ J
0 1 0 1
P
0 1
P
0 1
O
0 1
O
0 1
O
0 1
O
0 1
L
0 1
N N
0 1 0 1
P
0 1
P
0 1
O
0 1
O
0 1
O
0 1
O
0 1
D
0 1
D
0 1
D
0 1
D
0 1
B A
C
E
F G
HJ
D
K M
L
N
OP
C
HK
D
M
F
G
A
B
E
J
O
L
N
P
[AB]
[AF][CHAE]
[CEJ]
[CD]
[CHAB]
[CHA]
[CH]
[C]
[ ]
[CKO]
[CKLN]
[CKL]
[CK]
[C]
(C K H A B E J L N O D P M F G)
C
HK
D
M
F
G
A
B
E
J
O
L
N
P
[AB]
[AF][CHAE]
[CEJ]
[CD]
[CHAB]
[CHA]
[CH]
[C]
[ ]
[CKO]
[CKLN]
[CKL]
[CK]
[C]
(C K H A B E J L N O D P M F G)
CS 276 42
How Big Is the Context?
Theorem: The maximum context size for a pseudo tree is equal to the treewidth of the graph along the pseudo tree.
C
HK
D
M
F
G
A
B
E
J
O
L
N
P
[AB]
[AF][CHAE]
[CEJ]
[CD]
[CHAB]
[CHA]
[CH]
[C]
[ ]
[CKO]
[CKLN]
[CKL]
[CK]
[C]
(C K H A B E J L N O D P M F G)
B A
C
E
F G
HJ
D
K M
L
N
OP
max context size = treewidth
CS 276 43
G
EK
F
L
H
C
BA
M
J
D
Treewidth vs. Pathwidth
EK
L
H
C
A
M
J
ABC
BDEF
BDFG
EFH
FHK
HJ KLM
treewidth = 3 = (max cluster size) - 1
ABC
BDEFG
EFH
FHKJ
KLM
pathwidth = 4 = (max cluster size) - 1
D
G
B
F
TREE
CHAIN
CS 276 44
Complexity of AND/OR Graph Search
AND/OR graph OR graph
Space O(n kw*) O(n kpw*)
TimeO(n kw*)
O(n kpw*)
k = domain sizen = number of variablesw*= treewidthpw*= pathwidth
w* ≤ pw* ≤ w* log n
CS 276 45
AND/OR search algorithms AO(i)
i = the max size of a cache table (i.e. number of variables in a context)
i = 0 i = w* treewidth
Space:
Time:
O(n exp w* )
O(n exp w* )
O( n )
O( exp (w* log n) )
CS 276 46
Searching AND/OR Graphs AO(i): searches depth-first, cache i-context
i = the max size of a cache table (i.e. number of variables in a context)
i=0
i=w*
Space: O(n)
Time: O(exp(w* log n))
Space: O(exp w*)
Time: O(exp w*)Space: O(exp(i) )
Time: O(exp(m_i+i )
i
CS 276 47
Caching
A
D
B C
E
F
A
D
B
CE
F
G H
J
K
G
H
J
KAOR
0AND 1
BOR B
0AND 1 0 1
EOR C E C E C E C
OR D F D F D F D F D F D F D F D F
AND
AND 0 10 1 0 10 1 0 10 1 0 10 1
OR
OR
AND
AND
0
G
H H
0 1 0 1
0 1
1
G
H H
0 1 0 1
0 1
0
J
K K
0 1 0 1
0 1
1
J
K K
0 1 0 1
0 1
0
G
H H
0 1 0 1
0 1
1
G
H H
0 1 0 1
0 1
0
J
K K
0 1 0 1
0 1
1
J
K K
0 1 0 1
0 1
0
G
H H
0 1 0 1
0 1
1
G
H H
0 1 0 1
0 1
0
J
K K
0 1 0 1
0 1
1
J
K K
0 1 0 1
0 1
0
G
H H
0 1 0 1
0 1
1
G
H H
0 1 0 1
0 1
0
J
K K
0 1 0 1
0 1
1
J
K K
0 1 0 1
0 1
0
G
H H
0 1 0 1
0 1
1
G
H H
0 1 0 1
0 1
0
J
K K
0 1 0 1
0 1
1
J
K K
0 1 0 1
0 1
0
G
H H
0 1 0 1
0 1
1
G
H H
0 1 0 1
0 1
0
J
K K
0 1 0 1
0 1
1
J
K K
0 1 0 1
0 1
0
G
H H
0 1 0 1
0 1
1
G
H H
0 1 0 1
0 1
0
J
K K
0 1 0 1
0 1
1
J
K K
0 1 0 1
0 1
0
G
H H
0 1 0 1
0 1
1
G
H H
0 1 0 1
0 1
0
J
K K
0 1 0 1
0 1
1
J
K K
0 1 0 1
0 1
context(D)={D}
context(F)={F}
CS 276 48
Caching
A
D
B C
E
F
A
D
B
CE
F
G H
J
K
G
H
J
KAOR
0AND 1
BOR B
0AND 1 0 1
EOR C E C E C E C
OR D F D F D F D F D F D F D F D F
AND
AND 0 10 1 0 10 1 0 10 1 0 10 1
OR
OR
AND
AND
0
G
H H
0 1 0 1
0 1
1
G
H H
0 1 0 1
0 1
0
J
K K
0 1 0 1
0 1
1
J
K K
0 1 0 1
0 1
context(D)={D}
context(F)={F}
CS 276 52
All four search spaces
E
C
F
D
B
A
0 1 0 1 0 1 0 1
0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
0 1 0 1
0 1
0AND 1
BOR B
0AND 1 0 1
EOR C E C E C E C
OR D F D F D F D F D F D F D F D F
AND 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
AND 0 10 1 0 10 1 0 10 1 0 10 1
0 1 0 1 0 1 0 1
0 1 0 1 0 1 0 1
0 1
0 1 0 1
E
C
F
D
B
A 0 1
0 1
0 1 0 1 0 1
AOR
0AND 1
BOR B
0AND 1 0 1
EOR C E C E C E C
ORD F D F D F D F D F D F D F D F
AND 0 1 0 1
AND0 10 1 0 1 0 1 0 1
Full OR search tree
Full AND/OR search tree
Context minimal OR search graph
Context minimal AND/OR search graph
A
D
B C
E
F
A
D
B
CE
F
CS 276 53
All four search spaces
E
C
F
D
B
A
0 1 0 1 0 1 0 1
0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
0 1 0 1
0 1
0AND 1
BOR B
0AND 1 0 1
EOR C E C E C E C
OR D F D F D F D F D F D F D F D F
AND 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
AND 0 10 1 0 10 1 0 10 1 0 10 1
0 1 0 1 0 1 0 1
0 1 0 1 0 1 0 1
0 1
0 1 0 1
E
C
F
D
B
A 0 1
0 1
0 1 0 1 0 1
AOR
0AND 1
BOR B
0AND 1 0 1
EOR C E C E C E C
ORD F D F D F D F D F D F D F D F
AND 0 1 0 1
AND0 10 1 0 1 0 1 0 1
Full OR search tree
Full AND/OR search tree
Context minimal OR search graph
Context minimal AND/OR search graph
A
D
B C
E
F
A
D
B
CE
F
Time-space
CS 276 54
Available code http://graphmod.ics.uci.edu/group/
Software
CS 276 55
AND/OR w-cutset
A
C
B K
G L
D F
H
M
J
E
AC
B K
G
L
D
FH
M
J
E
A
C
B K
G L
D F
H
M
J
E
C
B K
G
L
D
FH
M
J
E
3-cutset
A
C
B K
G L
D F
H
M
J
E
C
K
G
L
D
FH
M
J
E
2-cutset
A
C
B K
G L
D F
H
M
J
E
L
D
FH
M
J
E
1-cutset
CS 276 56
AND/OR w-cutset
AC
B K
G
L
D
FH
M
J
E
AC
B K
G
L
D
FH
M
J
E
AC
B K
G
L
D
FH
M
J
E
pseudo tree 1-cutset treegrahpical model
CS 276 57
Searching AND/OR Graphs AO(i): searches depth-first, cache i-context
i = the max size of a cache table (i.e. number of variables in a context)
i=0
i=w*
Space: O(n)
Time: O(exp(w* log n))
Space: O(exp w*)
Time: O(exp w*)Space: O(exp(i) )
Time: O(exp(m_i+i )
i
CS 276 58
w-cutset Trees Over AND/OR space
Definition: T_w is a w-cutset tree relative to backbone tree T, iff
T_w is roots T and when removed, yields tree-width w.
Theorem: AO(i) time complexity for backbone T is time
O(exp(i+m_i)) and space O(i), m_i is the depth of the T_i tree.
Better than w-cutset: O(exp(i+c_i)) when c_i is the number of nodes in T_i