1

Tree Clustering for Constraint Networks

Chris Reeson Advanced Constraint Processing

Fall 2009

By Rina Dechter & Judea PearlArtificial Intelligence, Oct 1988

Three

Overview

• Contributions of the Paper• Context of the Paper• Algorithms– Tree Clustering– Adaptive Consistency

• Relative Merits• Conclusion• Related algorithms

Three

Contributions of the Paper

• Introduces Tree Clustering (T-C) for backtrack-free search

• Introduces the Adaptive Consistency (A-C) algorithm

• Compares the two algorithms

Three

Context

• Two ways to restrict CSPs to make finding the minimal CSP efficient– Topology of constraint graph– Types of constraints

• T-C & A-C– Target the topology: tree– Are applicable to binary & non-binary CSPs

• Two ways to transform a constraint graph into a tree– Remove redundant arcs in dual graphs (join tree)– Form larger clusters of c-variables (simulates a join tree)

Three

Definitions• Hyper, dual, and primal graphs

h8

h7

h1h5

h3

h4

h6 h2

18 23

9

10

11

4

12 513

78 6

Hypergraph Dual graph Primal graph

18 2 3

9 10 11

412

51378 6

h3h2

h4

h7h6h8

h5

• The discussion focuses on primal graphs for simplicity

From Join Graph to Join Tree

• Join Graph– Start w/ the dual graph, remove redundant edges

while maintaining the connectedness property– Connectedness property: For each two nodes

sharing a variable, there is at least one path of labeled arcs containing the shared variable

• Join Tree– When the join graph is a tree, the dual CSP can

be solved BT free w/ directional arc consistency– What if there isn’t a join tree?

The idea for Tree Clustering

ACE CDE

AEFABCA

CE

EAC

CAE

ACE CDE

AEFABC

CE

AEAC

Three

Motivating ProblemConstraint Graph

D B

AC

EF

G

Domains: {1, 2, 3, 4, 5}

A solution

G > F

C ≠ D

C < A

C < BD < AD < B

F > B E > B

G > E

A ≠ B

2 3

41

43

5

Three

Overview

• Contributions of the Paper• Context of the Paper• Algorithms– Tree Clustering (T-C)– Adaptive Consistency (A-C)

• Relative Merits• Conclusion• Related Algorithms

Three

Tree Clustering (T-C): Idea

• A CSP organized as a join tree can be solved efficiently

• Tree Clustering Algorithm– Solves a CSP by breaking it into subproblems– Triangulates the primal graph– Solves subproblems & combines the solutions

D B

AC

EF

G

D B

AC

EF

G

ABCD

BED

DFE

EFG

BD

DE

EF

Three

ABCD

BED

DFE

EFG

BD

DE

EF

4312, 5312, 5432, …

342, 351, 221, …

234, 415, 153, …

435, 123, 112, …

Tree Clustering (T-C): Algorithm

1. Triangulate the primal graph2. Identify all the maximal cliques

in the primal chordal graph3. Form a join tree4. Solve the subproblems– Each cluster becomes single

variable

5. Solve the tree problem– Perform DAC from leaves to root– Instantiate BT-free from root to

leaves

D B

AC

EF

G

2 3

41

43

5

Three

Tree Clustering (T-C): Costs

1. Given a CSP and its primal graph generate a chordal primal graph: O(n2)

2. Identify all the maximal cliques in the primal chordal graph: O(|E’|)

3. Form the dual graph: O(n)4. Solve the sub problems: O(kr)

where k=domain size5. Solve the tree problem: O(n ∙

t log t)…

D B

AC

EF

G

ABCD

BED

DFE

EFG

BD

DE

EF

4312, 5312, 5432, …

342, 351, 221, …

234, 415, 153, …

435, 123, 112, …

2 3

41

43

5

Three

Tree Clustering (T-C): Total Cost

• Dominated by O(n t log t)∙– t is the largest number of solutions in a cluster,

t ≤ kr

– Time: O(n k∙ r r log k) = O(nr k∙ ∙ r )– Space: O(n k∙ r )

Three

Overview

• Contributions of the Paper• Context of the Paper• Algorithms– Tree Clustering (T-C)– Adaptive Consistency (A-C)

• Relative Merits• Conclusion• Related Algorithms

Three

Adaptive Consistency (A-C)

• An ordered constraint graph is backtrack-free if the level of directional strong consistency along this order is greater than the width of the ordered graph

• Beware– Enforcing i-consistency for i > 2 often requires the

addition of constraints which increase the width

Three

Adaptive Consistency (A-C): Idea

• Given an ordering d,– d-i-consistency is defined recursively– letting i change dynamically from node to node

• (A-C later redefined as bucket elimination)

Three

Adaptive Consistency: Algorithm

1. For i=n downto 1 do Steps 2-42. Compute PARENTS(Xi)

3. Connect all PARENTS(Xi)

4. Perform Consistency(Xi, PARENTS(Xi)) joining the constraints between Xi & its parents

5. Build a solution BT-free in the ordering (X1, …, Xn)

D B

AC

EF

G

C

B

D

E

F

G

A

C

B

D

E

F

G

A

tighten A by 2 consist

ACB join CB join AC to tighten AC by 3c

BE join CB join AC to tighten ACB by 4c

tighten D by 2 consist

EF join DE to tighten DE by 3 consist

GF join GE to tighten EF by 3 consist

Three

Adaptive Consistency: Cost

• Time: O(n exp(W*(d) + 1)), ∙ see Dechter page 109

• Space: O(n k∙ W*(d))

Three

Overview

• Contributions of the Paper• Context of the Paper• Algorithms– Tree Clustering (T-C)– Adaptive Consistency (A-C)

• Relative Merits• Conclusion• Related Algorithms

Three

Relative Merits

• Arcs resulting from triangulation match arcs added by adaptive consistency, for the same ordering

• Every cluster in T-C is represented in A-C by a series of smaller constraints

• Similar bounds– W*(d) + 1 = the size of the largest clique

• A-C eliminates the redundancy of generated solutions

• T-C enumerates all solutions that A-C represents via constraints.

D B

AC

EF

G

C

B

D

E

F

G

A

D B

AC

EF

G

C

B

D

E

F

G

A

Three

Conclusions

• Tree clustering groups c-nodes into a tree capable of supporting query answering backtrack-free

• Useful in systems that need to answer many questions about a dataset and where the environmental conditions undergo local changes

• Recently, researchers have started looking at T-C for solving the CSP, see BTD by Jégou & Terrioux (and others in soft CSPs)

Note On Triangulation• Find the triangulated graph w/ smallest maximum clique: NP-hard• Heuristics

– Operation: when eliminating a node, connect all its neighbors, to form a clique (fill edges)

– H1: choose the node w/ smallest degree– H2: choose the node that, after elimination, yields the smallest number of

fill edges– H3: Given any ordering (e.g., maximal cardinality ordering), moralize the

graph• Elimination order is the reverse of instantiation order• Elimination order of a triangulated graph is called a perfect

elimination scheme– In this ordering, every node is simplicial: forms a clique w/ its neighbors– If you follow elimination order, no fill edges need to be added

Maximal Cardinality Ordering

• An approximation of min. width ordering• Choose a node arbitrarily a simplicial node• Among the remaining nodes, choose the one

that is connected to the maximum number of already chosen nodes, break ties arbitrarily

• Repeat…• Reverse the final order Tsang 6.2.4

Dechter Fig 4.5

Two Additional Algorithms

• Maximal Cliques of the triangulated graph• Join Tree of the triangulated graph