recent development on elimination ordering group 1
Post on 20-Dec-2015
213 views
TRANSCRIPT
![Page 1: Recent Development on Elimination Ordering Group 1](https://reader030.vdocuments.us/reader030/viewer/2022032800/56649d405503460f94a1aafd/html5/thumbnails/1.jpg)
Recent Development on Recent Development on Elimination OrderingElimination Ordering
Group 1Group 1
![Page 2: Recent Development on Elimination Ordering Group 1](https://reader030.vdocuments.us/reader030/viewer/2022032800/56649d405503460f94a1aafd/html5/thumbnails/2.jpg)
Recent Development on Recent Development on Elimination OrderingElimination Ordering
By Florence Lo
Andrew Yan
![Page 3: Recent Development on Elimination Ordering Group 1](https://reader030.vdocuments.us/reader030/viewer/2022032800/56649d405503460f94a1aafd/html5/thumbnails/3.jpg)
Table of ContentsTable of Contents
Problem Statement Background Motivations Approaches to Elimination Orderings An approximation algorithm for
triangulation Q & A
![Page 4: Recent Development on Elimination Ordering Group 1](https://reader030.vdocuments.us/reader030/viewer/2022032800/56649d405503460f94a1aafd/html5/thumbnails/4.jpg)
Problem StatementProblem Statement
Minimizing Total Cost in Vertices Elimination
![Page 5: Recent Development on Elimination Ordering Group 1](https://reader030.vdocuments.us/reader030/viewer/2022032800/56649d405503460f94a1aafd/html5/thumbnails/5.jpg)
BackgroundBackground
BN specifies a complete joint probability distribution (JPD) over all variables. Given the JPD, one can answer all possible inference queries by summing out over irrelevant variables.Assumed all variables have 2 states and there are n variables => O(2n)
![Page 6: Recent Development on Elimination Ordering Group 1](https://reader030.vdocuments.us/reader030/viewer/2022032800/56649d405503460f94a1aafd/html5/thumbnails/6.jpg)
Background (cont)Background (cont)
To make inference more efficient, 2 exact inference algorithms are used
1) Variable Elimination Algorithm (VE)
2) Clique Tree Propagation (CTP)
![Page 7: Recent Development on Elimination Ordering Group 1](https://reader030.vdocuments.us/reader030/viewer/2022032800/56649d405503460f94a1aafd/html5/thumbnails/7.jpg)
Variable Elimination (VE)Variable Elimination (VE)
Use the factored representation of the JPD to do marginalization efficiently. The key idea is to “push sum in” as far as possible when summing out irrelevant termsComplexity of VE is based on the cost of variables elimination (i.e. elimination orderings)
![Page 8: Recent Development on Elimination Ordering Group 1](https://reader030.vdocuments.us/reader030/viewer/2022032800/56649d405503460f94a1aafd/html5/thumbnails/8.jpg)
Clique Tree Propagation Clique Tree Propagation (CTP)(CTP)
Inference in BN is formulated as message passing in a junction tree
First step in CTP is triangulated graph, one way is to eliminate vertices one by one to add the extra edges (i.e. elimination orderings)
![Page 9: Recent Development on Elimination Ordering Group 1](https://reader030.vdocuments.us/reader030/viewer/2022032800/56649d405503460f94a1aafd/html5/thumbnails/9.jpg)
DiscussionDiscussion
Different elimination ordering leads to different cost
NP-hard
![Page 10: Recent Development on Elimination Ordering Group 1](https://reader030.vdocuments.us/reader030/viewer/2022032800/56649d405503460f94a1aafd/html5/thumbnails/10.jpg)
3 Approaches to Establish 3 Approaches to Establish Elimination OrderingsElimination Orderings
1. Elimination Ordering Heuristics
2. Triangulation
3. Simulated Annealing
![Page 11: Recent Development on Elimination Ordering Group 1](https://reader030.vdocuments.us/reader030/viewer/2022032800/56649d405503460f94a1aafd/html5/thumbnails/11.jpg)
Elimination Ordering Elimination Ordering HeuristicsHeuristics
Maximum Cardinality Search (Tarjan et. al. 1985)
Minimum Deficiency Heuristics (Bertele et. al. 1972)
Minimum Degree Heuristics (Rose 1972)
![Page 12: Recent Development on Elimination Ordering Group 1](https://reader030.vdocuments.us/reader030/viewer/2022032800/56649d405503460f94a1aafd/html5/thumbnails/12.jpg)
DiscussionsDiscussions
Easy to implementConcepts are easyLinear time complexityGood approximation to the optimal solution
![Page 13: Recent Development on Elimination Ordering Group 1](https://reader030.vdocuments.us/reader030/viewer/2022032800/56649d405503460f94a1aafd/html5/thumbnails/13.jpg)
TriangulationTriangulation
Objectives
1. Size of the maximal clique is minimum
2. Minimal triangulation
![Page 14: Recent Development on Elimination Ordering Group 1](https://reader030.vdocuments.us/reader030/viewer/2022032800/56649d405503460f94a1aafd/html5/thumbnails/14.jpg)
Minimal TriangulationMinimal TriangulationComputing a minimal triangulation consists in
embedding a given graph with a triangulated
graph by adding a set of edges (called a fill).
If the set of edges added is inclusion-minimal,
the fill is said to be minimal, and the
corresponding triangulated graph is called a
minimal triangulation
![Page 15: Recent Development on Elimination Ordering Group 1](https://reader030.vdocuments.us/reader030/viewer/2022032800/56649d405503460f94a1aafd/html5/thumbnails/15.jpg)
DiscussionDiscussion
Minimal triangulation generally improves the total cost
May stuck in the local minima of the cost function
NP-hard
![Page 16: Recent Development on Elimination Ordering Group 1](https://reader030.vdocuments.us/reader030/viewer/2022032800/56649d405503460f94a1aafd/html5/thumbnails/16.jpg)
Simulated AnnealingSimulated AnnealingStochastic optimization algorithm to find
global minimum cost configuration of NP-complete combinatorial problems with cost functions having many local minima
A combination of deterministic descent search and a Monte Carlo method
Accept cost function increase with positive probability dependent on the state of the search process
![Page 17: Recent Development on Elimination Ordering Group 1](https://reader030.vdocuments.us/reader030/viewer/2022032800/56649d405503460f94a1aafd/html5/thumbnails/17.jpg)
PseudoPseudo-code-codeSelect an initial solution s0
Select an initial temperature t0 > 0Select a temperature reduction function α Repeat repeat randomly select s from N(s0) δ = f(s) – f(s0) If δ < 0 then s0 = s else generate random x uniformly in the range (0, 1) if x < exp(- δ/t) then s0 = s until iteration_count = nrep Set t = α(t)Until stopping condition = trueS0 is the approximation to the optimal solution
![Page 18: Recent Development on Elimination Ordering Group 1](https://reader030.vdocuments.us/reader030/viewer/2022032800/56649d405503460f94a1aafd/html5/thumbnails/18.jpg)
2 Categories of Decisions to 2 Categories of Decisions to makemake
1. Parameters of the annealing algorithm
- t0, cooling schedule governed by nrep and α, stopping conditions
2. Problem specific and involve the choice of the space of feasible solutions, the form of the cost function, neighborhood structure employed
![Page 19: Recent Development on Elimination Ordering Group 1](https://reader030.vdocuments.us/reader030/viewer/2022032800/56649d405503460f94a1aafd/html5/thumbnails/19.jpg)
DiscussionDiscussion
Time consumingExtensive experiments on the control
parametersEnhancement and modifications to speed up
computation time
![Page 20: Recent Development on Elimination Ordering Group 1](https://reader030.vdocuments.us/reader030/viewer/2022032800/56649d405503460f94a1aafd/html5/thumbnails/20.jpg)
An approximation algorithm An approximation algorithm for triangulationfor triangulation
By Ann Becker and Dan Geiger in 1996 Same state space size Optimality criterion
– Cliquewidth, k 2+1 – approximation
= approximation ratio for 3-way vertex cutO(2(2+1)kn•poly(n))
– poly(n) = linear programming Divide and conquer
![Page 21: Recent Development on Elimination Ordering Group 1](https://reader030.vdocuments.us/reader030/viewer/2022032800/56649d405503460f94a1aafd/html5/thumbnails/21.jpg)
The algorithmThe algorithm
Triangulate(G,W,k)If |V| < (2 +1)K then
make a clique out of G
ElseFind a W-decomposition (X,A,B,C) of G wrt (k,)If not found
Return “cliquewidth > k”
WAWA, WBWB, WCWC
Call Triangulate( G[AX], WA,k)
Call Triangulate( G[BX], WB,k)
Call Triangulate( G[CX], WC,k)Make a clique of G[W X]
![Page 22: Recent Development on Elimination Ordering Group 1](https://reader030.vdocuments.us/reader030/viewer/2022032800/56649d405503460f94a1aafd/html5/thumbnails/22.jpg)
Trial and errorTrial and error
Try for k = 1, 2, 3… until success!!!
![Page 23: Recent Development on Elimination Ordering Group 1](https://reader030.vdocuments.us/reader030/viewer/2022032800/56649d405503460f94a1aafd/html5/thumbnails/23.jpg)
Example (k=3,Example (k=3,=1)=1)
![Page 24: Recent Development on Elimination Ordering Group 1](https://reader030.vdocuments.us/reader030/viewer/2022032800/56649d405503460f94a1aafd/html5/thumbnails/24.jpg)
ImprovementImprovement
Processing the input– Simplicial vertex
–Removed repeatedly–Improves the running time
![Page 25: Recent Development on Elimination Ordering Group 1](https://reader030.vdocuments.us/reader030/viewer/2022032800/56649d405503460f94a1aafd/html5/thumbnails/25.jpg)
ImprovementImprovement
Termination condition– |V| < (2+1)k from a clique– Junction tree instead– W as a clique– Approximation may improved
![Page 26: Recent Development on Elimination Ordering Group 1](https://reader030.vdocuments.us/reader030/viewer/2022032800/56649d405503460f94a1aafd/html5/thumbnails/26.jpg)
ImprovementImprovement
Post-processing the output– May not minimal triangulated– Remove some added edges, still triangulated
– Kjaerulff’s algorithm
![Page 27: Recent Development on Elimination Ordering Group 1](https://reader030.vdocuments.us/reader030/viewer/2022032800/56649d405503460f94a1aafd/html5/thumbnails/27.jpg)
In practiceIn practice
Time complexity O(2(2+1)kn•poly(n))|W| < k in most case
– 24.66k 22k
W consists of two subsets– No 3-way vertex cut
Step further for kFind a W-decomposition (X,A,B,C) of G wrt (k,)If not found
Return “cliquewidth > k”
![Page 28: Recent Development on Elimination Ordering Group 1](https://reader030.vdocuments.us/reader030/viewer/2022032800/56649d405503460f94a1aafd/html5/thumbnails/28.jpg)
A tighter boundA tighter bound
l be the size of the largest clique in the output
Test against (2+1)k– The ratio can be smaller than 2+1– Optimal cliquewidth can be larger than k
l/k instead of 2+1Instance-specific posteriori bound
![Page 29: Recent Development on Elimination Ordering Group 1](https://reader030.vdocuments.us/reader030/viewer/2022032800/56649d405503460f94a1aafd/html5/thumbnails/29.jpg)
The weighted problemThe weighted problem
Different state spaces of each vertex– w(v) = log2(state space size)
– w(clique) = sum of w(v) in the clique
Weighted W-decompositionWhen terminate in recursion, run greedy
algorithm (minimum weight heuristics)(2+1)m
![Page 30: Recent Development on Elimination Ordering Group 1](https://reader030.vdocuments.us/reader030/viewer/2022032800/56649d405503460f94a1aafd/html5/thumbnails/30.jpg)
ResultsResults
Medianus I (43 vertices, 110 edges)Compare with the enhanced minimum
weight heuristics– Better when state space increase
l/k = 10/6 (not 3.66)Run time: one or two minutes
![Page 31: Recent Development on Elimination Ordering Group 1](https://reader030.vdocuments.us/reader030/viewer/2022032800/56649d405503460f94a1aafd/html5/thumbnails/31.jpg)
DiscussionDiscussion
O(24.66kn•poly(n))– Polynomial for k = O(log n)
Exponential time for an arbitrary inference
![Page 32: Recent Development on Elimination Ordering Group 1](https://reader030.vdocuments.us/reader030/viewer/2022032800/56649d405503460f94a1aafd/html5/thumbnails/32.jpg)
![Page 33: Recent Development on Elimination Ordering Group 1](https://reader030.vdocuments.us/reader030/viewer/2022032800/56649d405503460f94a1aafd/html5/thumbnails/33.jpg)
What is cliquewidth?What is cliquewidth?
Undirected graphThe size of the largest clique in the junction
free of the graph in which the size of the largest clique is minimized
back
![Page 34: Recent Development on Elimination Ordering Group 1](https://reader030.vdocuments.us/reader030/viewer/2022032800/56649d405503460f94a1aafd/html5/thumbnails/34.jpg)
What is 3-way vertex cut?What is 3-way vertex cut?
A weighted undirected graph3 verticesA set of vertices of minimum weight whose
removal three vertices disconnected4/3 – approximation2 – approximation
back
![Page 35: Recent Development on Elimination Ordering Group 1](https://reader030.vdocuments.us/reader030/viewer/2022032800/56649d405503460f94a1aafd/html5/thumbnails/35.jpg)
What is W-decomposition?What is W-decomposition?
An integer k 1, a real number 1 A graph G = ( V, E ), |V| ( 2 + 1 ) k W V A decomposition ( X, A, B, C ) wrt ( k, )
– |W| < ( + 1 )k– |X| < k– |(WA)X| < ( + 1 )k– |(WB)X| < ( + 1 )k– |(WC)X| < ( + 1 )k
back
![Page 36: Recent Development on Elimination Ordering Group 1](https://reader030.vdocuments.us/reader030/viewer/2022032800/56649d405503460f94a1aafd/html5/thumbnails/36.jpg)
What is decomposition?What is decomposition?
A graph G = ( V, E )A partition ( X, A, B, C )
– A, B – No edges between A, B, C
back