authors : ann becker and dan geiger presented by : igor kviatkovsky
DESCRIPTION
Optimization of Pearl’s Method of Conditioning and Greedy-Like Approximation Algorithm for the Vertex Feedback Set Problem. Authors : Ann Becker and Dan Geiger Presented by : Igor Kviatkovsky. Outline. Introduction The Loop Cutset (LC) problem Weighted Vertex Feedback Set (WVFS) problem - PowerPoint PPT PresentationTRANSCRIPT
Optimization of Pearl’s Method of Conditioning and Greedy-Like
Approximation Algorithm for the Vertex Feedback Set Problem
Authors: Ann Becker and Dan GeigerPresented by: Igor Kviatkovsky
Outline
• Introduction• The Loop Cutset (LC) problem• Weighted Vertex Feedback Set (WVFS) problem• Reduction from LC to WVFS• MGA (Modified Greedy Algorithm)
– 2-approximation for WVFS
Introduction• Pearl’s method of conditioning is one of the known
inference methods for Bayesian networks• Find a set of vertices such that once the corresponding
variables are instantiated, the remaining network is singly–connected– Pearl’s UPDATE-TREE procedure can be applied
A
L
C
I
D
J
B
M
E
K
Fixing value of A & B
L
C
I
J
M
E
K
D
a b b a
Introduction• Fixing value of A & B & L
breaks all loops. But can we choose less variables to break all loops?
• Are there better variables to choose than others?
• Motivation(Geiger&Becker):– choose the vertices which
break more loops– vertices with higher degree are
more likely to break more loops (vertice J with degree 3 breaks 3 loops!)
A
L
C
I
D
J
B
E
K
M
The Loop Cutset Problem
• Definitions:• The underlying graph G of a directed graph D
is the undirected graph formed by ignoring the directions of the edges in D.
• A loop in D is a subgraph of D whose underlying graph is a cycle.
• A vertex v is a sink with respect to a loop Γ if the two edges adjacent to v in Γ are directed into v .
• Every loop must contain at least one vertex that isn’t a sink with respect to that loop.
sink
The Loop Cutset Problem
• Each vertex that isn’t a sink with respect to a loop Γ is called an allowed vertex with respect to Γ.
• A loop cutset of a directed graph D is a set of vertices that contain at least one allowed vertex with respect to each loop in D.
• A minimum loop cutset of a weighted directed graph D is the one for which the weight is minimum.
The Loop Cutset Problem
• Example– L is a sink with respect to the
loop ACILJDA– L is an allowed vertex with
respect to the loop JLMJ– {A,B,L}, {C,J} are valid loop
cutsets of the graph– Suppose equal size of
variables’ domains (=r)• The number of instances
associated with {C,J} is r2
• The number of instances associated with {A,B,L} is r3
• {C,J} is of lower weight than {A,B,L}
A
L
C
I
D
J
B
E
K
M
Weighted Vertex Feedback Set (WVFS)
• Let G(V,E) be an undirected graph.• Let w: V R+ be a weight function on the vertices of
G.• A vertex feedback set of G is a subset of vertices F
in V such that each cycle in G passes through at least one vertex in F.
• A weight of a set of vertices X is w(X)=ΣvЄX w(v).
• A minimum vertex feedback set of weighted graph G is vertex feedback set F* with the minimum weight.
• WVFS problem is finding minimum vertex feedback set of a given weighted graph G having a weight function w.
Reduction from LC to WVFS• Given a weighted directed graph (D,w), splitting
weighted undirected graph Ds with a weight function ws is constructed– Split each vertex v in D into two vertices vin and vout in
Ds , connect vin and vout .– All incoming edges to v become undirected incident
edges with vin .– All outcoming edges from v become undirected
incident edges with vout .– Set ws(vin)=∞ and ws(vout)=w(v).
VoutV Vin
W(v) ∞ W(v)
V Vin
∞
Vout
W(v)
Γ2
Γ1
C1
C2
Algorithm LC• Ψ(X) is a set obtained by replacing each vertex
vin or vout in X by the respective source vertex v in D
• Algorithm LC– Input: A Bayesian network D.– Output: A loop cutset of D.1. Construct the splitting graph Ds with weight function
ws .2. Apply MGA on (Ds , ws) to obtain a vertex feedback
set F.3. Output Ψ(F).
• One-to-one and onto correspondence between loops in D and cycles in Ds
– MGA 2-approximation for WVFS yields LC 2-approximation!
Algorithm GA (Greedy Algorithm)
• Input: A weighted undirected graph G(V,E,w)• Output: A vertex feedback set F.
– F Ø; i 0;– Repeatedly remove all vertices with degree 0 or 1
from V and their adjacent edges from E and insert the resulting graph into G1 .
– While Gi isn’t the empty graph, do:1. Pick a vertex vi for which w(vi)/d(vi) is minimum in Gi
2. F F U {vi}
3. V V \{vi}4. i i + 15. Repeatedly remove all vertices with degree 0 or 1 from
V and their adjacent edges from E and insert the resulting graph into Gi .
Algorithm MGA (Modified GA)– F ' Ø; i 0;– Repeatedly remove all vertices with degree 0 or
1 from V and their adjacent edges from E and insert the resulting graph into G1 .
– While Gi isn’t the empty graph, do:1. Pick a vertex vi for which w(vi)/d(vi) is minimum in Gi
2. F’ F’ U {vi}3. V V \{vi}4. i i + 15. Repeatedly remove all vertices with degree 0 or 1 from
V and their adjacent edges from E and insert the resulting graph into Gi . For every edge e=(u1,u2) removed in this process do:– C(e) w(vi)/d(vi); – w(u1) w(u1) – C(e); – w(u2) w(u2) – C(e);
MGA (Phase 2)• Phase 2
– F F ‘– For i = |F ’| to 1 do
• If every cycle in G that intersects with {vi} also intersects with F \{vi} then,
– F F \{vi}
• After applying phase 2 on the vertex feedback set, all redundant vertices are removed and we get a minimal vertex feedback set
• Before applying phase 2 – 4-approximation
• After applying phase 2 – 2-approximation
MGA - Motivation• In each iteration remove all the isolated nodes
and all the chains– Isolated nodes and chains don’t break singly-
connectedness
• Pick a node with the minimum cost w(vi)/d(vi) to vertex feedback set– Small w(vi) means, it’s worthy taking node vi into the
vertex feedback set since it has a low weight
– Large d(vi) means, it’s worthy taking node vi into the vertex feedback set since it might break a high number of loops (has a high degree), even if it’s weight w(vi) is large
MGA - Motivation• Subtract the effective cost (cost paid per edge)
from the neighboring nodes weights.
• Example:
v2
v4v5
v3
v1
w1=1
c=0.5v2
v4v5
v3
v1
C(e)=0.5
5
v2
v4v5
v3
v1 C(e)=1.25
G1 G2 G3
w1=3
c=1w1=10
c=5
w1=10
c=5
w1=10
c=10/3
w2=9.5
c=9.5/2
w2=2.5
c=2.5/2 w2=10
c=5
w2=10
c=5
F ‘={v1}
F ‘={v1,v2}
F=F ’
C(e)=0.5
C(e)=0.5
C(e)=0.5
C(e)=1.25
C(e)=1.25
C(e)=1.25
F ’=Φ
Performance and Summary
• Theoretical approximation ratio of MGA is 2 (the worst case)
• Average approximation ratio of MGA on randomly generated graphs is 1.22
• Before running the computations of conditioning method, the optimal complexity of its running time can be computed with high precision using MGA
Extra slides – MGA Analysis and approximation ratio proof
MGA - Analysis• F* - a minimum vertex feedback set of G(V,E,w).
• Vertices in F ‘ are {v1,v2,…,vt} when vi are indexed in the order they inserted into F ’.
• wi(v) and di(v) are weight and degree respectively of vertex v in Gi .
• Vi is a set of vertices of Gi .
• Let • Theorem 6 (proved later):
* * and i i i iF F V F F V
*
( ) 2 ( )i i
i iv F v F
d v d v
MGA - Analysis
• Let Γ1(v) be the set of edges in G1 for which at least one endpoint is v.
• From the description of the algorithm for every vertex v
in G1 , , if vЄF
1 ( )
( ) ( )e v
C e w v
1 ( )
( ) ( )e v
C e w v
Theorem 3
• Theorem 3: Algorithm MGA always outputs a vertex feedback set F whose weight is no more than twice the weight of a minimum vertex feedback set F*.
• Proof:– Let i=j+1, then:
( ) ( ) ( ) ( )
( ) ( ) ( ) 1 ( )j j j i i i j i i
j j ij j j i i i i i
w v w v w v c w vc c c
d v d v d v d v
*( ) 2 ( )w F w F
Theorem 3 (Proof)
– By grouping edges according to iterations they are assigned a weight:
– Let
– By definition:
1
1
1( ) 1
( ) ( ) ( ( ) ( ))i
i
i i i j j i j ie v j
C e c d v c d v d v
1 if and 0 if i i i iv F v F
1
1
11 1 ( ) 1 1
( ) ( ) ( ) ( ) ( ( ) ( ))i
t t t i
i i i i i i i j j i j ii i e v i j
w F w v C e c d v c d v d v
1 1 11 2
( ) ( ) ( ) ( )t t t
i i i i j i ji i j i
w F c d v c c d v
( ) ( )i
t
j i j ij i v F
d v d v
*( ) 2 ( )w F w F
1 ( )
( ) ( )i
ie v
C e w v
Theorem 3 (Proof)
1
* *1
* *1
1 1 12
1 1 12
1
11 1
( ) ( ) ( ) ( )
2 ( ) 2( ) ( )
2 ( ) 2 ( )
i
i
i i
t
i i iv F i v F
t
i i iiv F v F
t t
i i i ii iv F v F
w F c d v c c d v
c d v c c d v
c d v c d v
* * * *1 1 1
* * * *1 1
1
11 1 1\
1
11 \
2 ( ) 2 ( ) 2 ( )
2 ( ) ( ( ) ( )) ( )
i i i i
i i i t
t t t
i i i i i ii i iv F F v F v F
t
i i i i i t ti v F F v F v F
c d v c d v c d v
c d v c d v d v c d v
* *1
*
( )
2 ( ) 2 ( ) 2 ( ).e vv F v F
C e w v w F
*( ) 2 ( )w F w F
Theorem 6:
Grouping edges by iterations they are assigned a weight
*
( ) 2 ( )i i
i iv F v F
d v d v
Theorem 6
• Definitions:– Let
– Let dX(v) be the number of edges whose one endpoint is v and the other is a vertex in X.
• Theorem 6: Let G be a weighted graph for which every vertex has a degree strictly greater than 1, F be a minimal vertex feedback set of G and F* be an arbitrary vertex feedback set of G (possibly a minimum weight one), then:
*
( ) 2 ( )v F v F
d v d v
* *\ and \F V F F V F
v XdX(v)=3
Theorem 6 (Proof)• To prove the theorem l.h.s. is divided to 2 terms and
an upper bound for each term is provided.
• Lemma 7: Let G,F and F* be defined as above. Then,
• Proof:– For every set of vertices B in G
– Since d(v) ≥ 2 for each v in G
( ) 2 ( ( ) 2)v F v F
d v F d v
*
( ) 2 ( )v F v F
d v d v
* *2 ( ) 2 2v F
F d v F F F F
* * *
\
( ) 2 ( ) ( ) 2 2 ( ) \v F v F B v F B
d v F F d v d v F F B F F B
*
\
( ) 2 \ 2 ( ) \v F B
d v F B F F B
F
BF*
≥0
Theorem 6 (Lemma 7)
– Since dB(v) ≤ d(v),
– We have to prove that the following hold for some set of vertices B
* * * *2 ( ) 2 2 ( ) 2 2Bv F B v F
F d v F F B F F B d v F F F F
* * * *
* * * *
* *
( ) 2 2 2 2
( ) 2 2
( ) 2 2 ( ( ) 2) 2
Bv F B
Bv F B
B Bv F B v F B
d v F F B F F B F F B F F B
d v F F B F F B F F B F F B
d v F B F B d v F B
*2 ( ( ) 2) 2Bv F B
F d v F B
F
BF*
* *2 ( ) 2 2v F
F d v F F F F
Lemma 7 – Let’s define a set B for which this inequality can be
proven.– F is minimal
• Each vertex in F can be associated with a cycle in G that contains no other vertices of F.
• We define a graph H that consists of the union of these cycles (one cycle per each vertex).
• Definition: A linkpoint is a vertex with degree 2. A branchpoint is a vertex with degree larger than 2.
• Every vertex in F is a linkpoint in H.• Let B the vertices of H.
*2 ( ( ) 2) 2Bv F B
F d v F B
linkpoint branchpoint
F
BF*
Lemma 7
– The proof is constructive• We apply the following procedure on H and showing that
there are terms on the r.h.s. that contribute 2 for each vєF , and weren’t used for other vєF .
– Pick a vertex vєF and follow the two paths p1 and p2 in H from it until the first branchpoint on each path is found.
– There are 3 cases to consider:• Case 1:
– From definition of H:
– dB(b1)-2 ≥ 1, dB(b2)-2 ≥ 1
– There are terms to contribute 2 to r.h.s.
*2 ( ( ) 2) 2Bv F B
F d v F B
1 2,b b Fv
p1 b1
b2
p2
Lemma 7
• Case 2:– If dB(b1)≥ 4
» dB(b1)-2 ≥ 2– If dB(b1)= 3
» dB(b1)-2 = 1» dB(b2)-2 ≥ 1
– There are terms to contribute 2 to r.h.s.• Case 3:
– Isolated cycle– There exists a vertex in F* , that
resides on no other cycle of H.» There are maximum such
cases – There are terms to contribute 2 to
r.h.s.
*2 ( ( ) 2) 2Bv F B
F d v F B
*F B
p3 b2
v b1
p2
p1
v
p1
b1
p2
v
p1
b1
p2
Lemma 7
– Remove the paths p1 and p2 from H obtaining a graph in which still each vertex in F resides on a cycle that contains no other vertices of F.
– Continue till F is exhausted.
*2 ( ( ) 2) 2Bv F B
F d v F B
• Lemma 8: Let G,F and F* be defined as above. Then,
• Proof: Note that
Lemma 7
* *
* * *
( ( ) 2) ( ) ( ( ) 2) ( ( ) 2)F F
v F v F F v F F v F F
d v d v d v d v
* *
* * *
( ( ) 2) ( ) ( ( ) 2) ( ( ) 2)F F
v F v F F v F F v F F
d v d v d v d v
F*
F v2
v4v5
v3
v1
)2-2)+(3-2=(1 =1 -=1=1
Lemma 8
• So if we prove,
then we prove the lemma• The graph induced by is a forest, and since the
number of edges in a forest is smaller than the number of vertices, the following holds
* *
* * *
( ( ) 2) ( ) ( ( ) 2) ( ( ) 2)F F
v F v F F v F F v F F
d v d v d v d v
*F
* *
* *
( ( ) 2) ( ( ) 2) 0F F
v F F v F F
d v d v
*
*
*
*
* *
* *
*( ) / 2
( ( ) 2) 0
( ( ) 2) ( ( ) 2) 0.
Fv F
Fv F
F Fv F F v F F
d v F
d v
d v d v
• To complete the proof , we use bounds obtained from lemmas 7 and 8:
Theorem 6 (Proof)
*
*
*
* *
* *
* * *
*
* * *
* *
( ) 2 ( ( ) 2)
( ) 2 2 ( )
( ( ) 2) ( ( ) 2)
( ) ( ) ( ) ( )
( ) ( ) 2 ( ).
v F v F
Fv F v F F
Fv F F v F F
F Fv Fv F F v F F v F F
Fv F v F v F
d v F d v
d v F F F F d v
d v d v
d v d v d v d v
d v d v d v
*
( ) 2 ( )v F v F
d v d v