and/or search for mixed networks #csp robert mateescu ics280 spring 2004 - current topics in...

AND/OR Search for• Mixed Networks

• #CSP

Robert Mateescu

ICS280 Spring 2004 - Current Topics in Graphical Models Professor Rina Dechter

Mixed Networks

Belief Networks

A

D

B C

E

F

A

D

B C

E

F

)|(),,|(

),|(),|(),|(),( :CPTS

}1,0{:Domains

,,,,, :Variables

AFPBAEP

CBDPACPABPAP

DDDDDD

FEDCBA

FEDCBA

Constraint Networks

),(),(),(),( :Relations

}1,0{:Domains

,,,,, :Variables

EARBCDPAFCRABCR

DDDDDD

FEDCBA

FEDCBA

Mixed NetworksA

D

B C

E

F

A

D

B C

E

F

A

D

B C

E

F

),(),(),(),( :Relations

)|(),,|(

),|(),|(),|(),( :CPTs

}1,0{:Domains

,,,,, :Variables

EARBCDPAFCRABCR

AFPBAEP

CBDPACPABPAP

DDDDDD

FEDCBA

FEDCBA

otherwise ,0

if ,|

xxP

xPxxP

xPB

BB

Μ

Tasks for Mixed Networks

• Tasks for Belief Networks:– Belief updating: evaluating the posterior probability of each

singleton proposition given some evidence;– Most probable explanation (MPE): finding a complete assignment

to all variables having maximum probability given the evidence;– Maximum a posteriori hypothesis (MAP): finding the most likely

assignment to a subset of hypothesis variables given the evidence;

• Tasks for Constraint Networks:– Consistent: Decide if network is consistent– Find solutions: Find one, some or all solutions

• Tasks for Mixed Networks:– Belief updating, MPE, MAP– Constraint Probability Evaluation (CPE): Find the probability of

the constraint query

Auxiliary Network

A

D

B C

E

F

A

D

B C

E

F

)|(),,|(

),|(),|(),|(),( :CPTS

}1,0{:Domains

,,,,, :Variables

AFPBAEP

CBDPACPABPAP

DDDDDD

FEDCBA

FEDCBA

),(),(),(),( :Relations

}1,0{:Domains

,,,,, :Variables

EARBCDPAFCRABCR

DDDDDD

FEDCBA

FEDCBA

A

D

B C

E

FX

Y

Z

Q

otherwise ,0

if ,1)|(

Rt

tXP Saux

Mixed Graph

• The mixed graph is the union of the belief network graph and the constraint network graph

• The moral mixed graph is the union of the moral graph of the belief network and the graph of the constraint network

• Given a mixed graph GM = (GB,GR) of a mixed network M(B,R) where GB is the directed graph of B, and GR is the undirected constraint graph of R, the ancestral graph of Y \in X in GM is the union of GR and the ancestral graph of Y in GB

dm-separation

• Definition: Given a mixed graph, GM and given three subsets of variables W, Y and Z which are disjoint, we say that W and Y are dm-separated given Z in the mixed graph GM, denoted <W,Z,Y>dm, iff in the ancestral mixed graph of WUYUZ, all the paths between W and Y are intercepted by variables in Z.

• Theorem: Given a mixed network M = M(B,R) and its mixed graph GM, then GM is a minimal I-map relative to dm-separation

AND/OR Search for Mixed Networks

A

D

B C

E

F

A

D

B

CE

F

Mixed network DFS (legal) tree

AND/OR Search Space

0

A

B

0

E

OR

AND

OR

AND

OR

AND

C

0

OR

AND

D

0 1

F

0 1

1

D

0 1

F

0 1

0 1

1

E C

0

D

0 1

F

0 1

1

D

0 1

F

0 1

0 1

1

B

0

E C

0

D

0 1

F

0 1

1

D

0 1

F

0 1

0 1

1

E C

0

D

0 1

F

0 1

1

D

0 1

F

0 1

0 1

A

D

B

CE

F

Forward: Expanding an AND node

0

A

B

0

E

OR

AND

OR

AND

OR

AND

C

0

OR

AND

D

0 1

F

0 1

1

D

0 1

F

0 1

0 1

1

E C

0

D

0 1

F

0 1

1

D

0 1

F

0 1

0 1

1

B

0

E C

0

D

0 1

F

0 1

1

D

0 1

F

0 1

0 1

1

E C

0

D

0 1

F

0 1

1

D

0 1

F

0 1

0 1

A

D

B

CE

F

g-value(ANDnode) = ∏ Pi (or 1, if product is empty)

e.g. g-value(<B,1>) = P(B=1|A=0)

Forward: Expanding an OR node

0

A

B

0

E

OR

AND

OR

AND

OR

AND

C

0

OR

AND

D

0 1

F

0 1

1

D

0 1

F

0 1

0 1

1

E C

0

D

0 1

F

0 1

1

D

0 1

F

0 1

0 1

1

B

0

E C

0

D

0 1

F

0 1

1

D

0 1

F

0 1

0 1

1

E C

0

D

0 1

F

0 1

1

D

0 1

F

0 1

0 1

A

D

B

CE

F

g-value(ORnode) = 0

Backward: Propagating g-values

0

A

B

0

E

OR

AND

OR

AND

OR

AND

C

0

OR

AND

D

0 1

F

0 1

1

D

0 1

F

0 1

0 1

1

E C

0

D

0 1

F

0 1

1

D

0 1

F

0 1

0 1

1

B

0

E C

0

D

0 1

F

0 1

1

D

0 1

F

0 1

0 1

1

E C

0

D

0 1

F

0 1

1

D

0 1

F

0 1

0 1

A

D

B

CE

F

OR nodes perform summation

AND nodes perform product

The main idea

• Search algorithms can exploit a wide range of constraint propagation techniques

• When generating AND nodes, consistency is checked according to the desired level of constraint propagation

AND/OR Search (1)

AOR

AND

OR

AND

OR

AND

OR

AND

A

D

B C

E

F

g(A)=0

A

D

B

CE

F

AND/OR Search (2)

0

AOR

AND

OR

AND

OR

AND

OR

AND

1

A

D

B C

E

F

g(A)=0

g(<A,1>)=P(A=1)g(<A,0>)=P(A=0)

A

D

B

CE

F

AND/OR Search (3)

0

A

B

OR

AND

OR

AND

OR

AND

OR

AND

1

A

D

B C

E

F

g(A)=0

g(<A,0>)=P(A=0)g(<A,1>)=P(A=1)

g(B)=0A

D

B

CE

F

AND/OR Search (4)

0

A

B

0

OR

AND

OR

AND

OR

AND

OR

AND

1

1

A

D

B C

E

F

g(A)=0

g(<A,0>)=P(A=0)g(<A,1>)=P(A=1)

g(B)=0

g(<B,0>)=P(B=0|A=0) g(<B,1>)=P(B=1|A=0)

A

D

B

CE

F

AND/OR Search (5)

0

A

B

0

E

OR

AND

OR

AND

OR

AND

C

OR

AND

1

1

A

D

B C

E

F

g(A)=0

g(<A,0>)=P(A=0)g(<A,1>)=P(A=1)

g(B)=0

g(E)=0 g(C)=0

g(<B,0>)=P(B=0|A=0) g(<B,1>)=P(B=1|A=0)

A

D

B

CE

F

AND/OR Search (6)

0

A

B

0

E

OR

AND

OR

AND

OR

AND

C

OR

AND

0

1

1

A

D

B C

E

F

g(A)=0

g(<A,0>)=P(A=0)g(<A,1>)=P(A=1)

g(B)=0

g(E)=0 g(C)=0

g(<E,0>)=P(E=0|A=0,B=0)

g(<B,0>)=P(B=0|A=0) g(<B,1>)=P(B=1|A=0)

A

D

B

CE

F

AND/OR Search (6)

0

A

B

0

E

OR

AND

OR

AND

OR

AND

C

OR

AND

0

1

1

A

D

B C

E

F

g(A)=0

g(<A,0>)=P(A=0)g(<A,1>)=P(A=1)

g(B)=0

g(C)=0

g(E)=P(E=0|A=0,B=0)

g(<B,0>)=P(B=0|A=0) g(<B,1>)=P(B=1|A=0)

A

D

B

CE

F

AND/OR Search (8)

0

A

B

0

E

OR

AND

OR

AND

OR

AND

C

OR

AND

0

1

1

A

D

B C

E

F

g(A)=0

g(<A,0>)=P(A=0)g(<A,1>)=P(A=1)

g(B)=0

g(C)=0

g(<B,0>)= P(B=0|A=0) * P(E=0|A=0,B=0)g(<B,1>)=P(B=1|A=0)

A

D

B

CE

F

Constraint propagation

A

D

B C

E F

G H I K

A

D

B C

E F

G H I K

>

>

>

>

>

>> >

>

All domains are {1,2,3,4}

Constraint checking only

1 2

A

C

3 4

B CB

2 3 4

ED

3 4 3 4

HG

4 4

G I

4

I

D

4

G

ED

2 3 4

FF

4

K

F

4

K

3

K

4

3 4 3 4

4

G

D D

4

K

F F

B

4

D

B

OR

AND

OR

AND

OR

AND

AND

OR

A

D

B C

E F

G H I K

>

>

>

>

>

>> >

>

Forward checking

1 2

A

C

3 4

B CB

2 3 4

ED

3 4 3 4

HG

4 4

G I

4

I

D

4

G

ED

2 3 4

FF

4

K

F

4

K

3

K

4

3 4 3 4

4

G

D D

4

K

F F

B

4

D

B

OR

AND

OR

AND

OR

AND

AND

OR

A

D

B C

E F

G H I K

>

>

>

>

>

>> >

>

Maintaining arc-consistency

1 2

A

C

3 4

B CB

2 3 4

ED

3 4 3 4

HG

4 4

G I

4

I

D

4

G

ED

2 3 4

FF

4

K

F

4

K

3

K

4

3 4 3 4

4

G

D D

4

K

F F

B

4

D

B

OR

AND

OR

AND

OR

AND

AND

OR

A

D

B C

E F

G H I K

>

>

>

>

>

>> >

>

AND/OR vs. OR Space

Linear space algorithms

AND/OR search vs. Bucket Elimination

#CSP

• #CSP – Counting the solutions of a CSP problem is very similar to the CPE task in mixed networks

• If the belief part in a mixed network is empty, we can translate the AND/OR search for mixed networks to an AND/OR search for CSPs.

• OR nodes are initialized with g-value 0• AND nodes are initialized with g-value 1

Parent set and parent separator set

A

D

B

CE

F

Parent set Parent separator set

A

B A AB

C AB ABC

D BC D

E AB E

F AC F

Caching in AND/OR search

• If space is available, parts of the search tree can be cached, transforming the search space into a search graph

• In principle, we can cache at OR level and/or at AND level

• Caching at AND level => use parent separator set• Caching at OR level => use parent set

size(parent separator set) ≤ size(parent set) + 1

ORcache, N20, K3, Nc20 ,Pc4, inst20Time

10% 20% 30% 40% 50% 60% 70%

AUXILIARY BE 0.10110 0.10155 0.10115 0.10025 0.10000 0.08970 0.08805

i=0 AND/OR FC 0.00650 0.01250 0.02450 0.06555 0.22940 1.09355 5.81740AND/OR RFC 0.00350 0.01005 0.02555 0.07660 0.27490 1.33295 6.94850OR FC 0.00505 0.01200 0.02755 0.08670 0.52620 5.49720 65.68775OR RFC 0.00400 0.01255 0.02800 0.09870 0.56040 5.72635 67.94275




ORcache, N20, K3, Nc20 ,Pc4, inst20# of nodes

10% 20% 30% 40% 50% 60% 70%

i=0 AND/OR FC 225 518 1192 3552 16003 106651 735153AND/OR RFC 154 387 1052 3407 15737 106617 735153OR FC 225 519 1203 3810 28079 414463 6533674OR RFC 154 387 1062 3664 27801 414428 6533674




and/or search for mixed networks #csp robert mateescu ics280 spring 2004 - current topics in...

Documents