approximation algorithms chapter 14: rounding applied to set cover

Approximation AlgorithmsChapter 14: Rounding Applied to Set Cover

Overview

Set Cover– Approximation by simple rounding

• f-approx. algorithm (f: the frequency of the most frequent element).

– Approximation by randomized rounding• O(log n)-approx. algorithm (n: # elements to be covered).

Weighted Vertex Cover– 2-approx. algorithm

• Method based on half-integral solutions of the linear programming

• Each variable takes only 0, 1/2, or 1.

Set Cover

Input– Elements U={a1,…,an},

– Subsets of U:S={S1,…,Sm}.

– Cost function c: S→Q+. Output

– Subsets of S that cover all elements in U s.t. the sum of costs of chosen subsets in S is minimized.

S1 S2 S3S4

S5

2 2 2

1

4

Cost of a subset

Cost:2+2+2=6.

Cost:1+4=5.

Cost:2+1+4=7.

This is not a solutionsince an element is not covered.

a1 a2a3

a4 a5 a6

Set Cover by linear inequalities (1/4)

Objective function– Minimize the sum of costs

of subsets chosen:

Constraints– For covers

• Each element must appear in at least one chosen subset.

– For choosing subsets• Each subset is either chosen

or not chosen.

SS

xSc )(S

)( 1:

UaxSSaS

)( }1,0{ S SxS

S1 S2 S3S4

S5

2 2 2

1

4

Cost of a subset

Cost: 2+2+2=6.

Cost: 1+4=5.

Cost: 2+1+4=7.

a1 a2a3

a4 a5 a6

This is not a solutionsince an element is not covered.

Set Cover by linear inequalities (2/4)

Objective function– Minimize the sum of costs

of subsets chosen.

Constraints– For covers

– For choosing subsets

5432141222 SSSSS xxxxx

.1: ,1:

,1: ,1 :

,1: ,1:

5352

513

4241

65

43

21

SSSS

SSS

SSSS

xxaxxa

xxaxa

xxaxxa

)( }1,0{ S SxS

S1 S2 S3S4

S5

2 2 2

1

4

a1 a2a3

a4 a5 a6

This is not a solutionsince an element is not covered.

Cost of a subset

Cost: 2+2+2=6.

Cost: 1+4=5.

Cost: 2+1+4=7.

Set Cover by linear inequalities (3/4)

Objective function

Constraints– For covers

– For choosing subsets

.60401121212

4122254321

SSSSS xxxxx

.01: ,01:

,01: ,1 :

,01: ,01:

5352

513

4241

65

43

21

SSSS

SSS

SSSS

xxaxxa

xxaxa

xxaxxa

S1 S2 S3S4

S5

2 2 2

1

4

a1 a2a3

a4 a5 a6

.0,0,1,1,154321 SSSSS xxxxx

Cost of a subset

Cost: 2+2+2=6.

Cost: 1+4=5.

Cost: 2+1+4=7.

This is not a solutionsince an element is not covered.

Set Cover by linear inequalities (4/4)

Objective function

Constraints– For covers

– For choosing subsets

.51411020202

4122254321

SSSSS xxxxx

.10: ,10:

,10: ,10 :

,10: ,10:

5352

513

4241

65

43

21

SSSS

SSS

SSSS

xxaxxa

xxaxa

xxaxxa

S1 S2 S3S4

S5

2 2 2

1

4

a1 a2a3

a4 a5 a6

.1,1,0,0,054321 SSSSS xxxxx

Cost of a subset

Cost: 2+2+2=6.

Cost: 1+4=5.

Cost: 2+1+4=7.

This is not a solutionsince an element is not covered.

LP-relaxation

Constraints– Each subset is either

chosen or not chosen.

– It takes a value bet. 0 and 1.

• From the nature of Set Cover, the upper bound can be eliminated.

– It takes a positive value.

)( }1,0{ S SxS

S1 S2 S3S4

S5

2 2 2

1

4

a1 a2a3

a4 a5 a6

)( 10 S SxS

)( 0 S SxS

Cost of a subset

Cost: 2+2+2=6.

Cost: 1+4=5.

Cost: 2+1+4=7.

This is not a solutionsince an element is not covered.

Rounding

To change a natural number into an integer.

x1 x2 x3 x40

1

Solution found by LP

x1 x2 x3 x4

0

1

x1 x2 x3 x4

0

1

Rounded solutionwith a threshold

Probabilisticallyrounded solution

Threshold

Overview

Set Cover– Approximation by simple rounding

• f-approx. algorithm (f: the frequency of the most frequent element).

– Approximation by randomized rounding• O(log n)-approx. algorithm (n: # elements to be covered).

Weighted Vertex Cover– Method based on half-integral solutions of the linear

programming• Each variable takes only 0, 1/2, or 1.• 2-approx. algorithm

Algorithm 14.1

A simple rounding algorithm A1

– f: the frequency of the most frequent element.– 1. Find an optimal solution to the LP-relaxation.

– 2. Pick all sets S for which xS 1/≧ f.

• xS becomes 1 if xS 1/≧ f .

S1 S2

S3

S4

S5

2 2

2

1

3

a1 a2

a3a4 a5

f =2.

Solutionby LP-relax.

Solution byLP-relax.

54321 SSSSS xxxxx

0.5

54321 SSSSS xxxxx

1.0

54321 SSSSS xxxxx

0.5

54321 SSSSS xxxxx

1.0

Rounded solution Rounded solution

Theorem 14.2

A1 (Algorithm 14.1) is a f -approximation algorithm for Set Cover.– We need to consider the following two properties:

• A1 outputs a sound solution, which covers all elements.

• How much is the cost of the solution with A1?

Proof of Theorem 14.2 (1/2)

A1 outputs a sound solution, which covers all elements.– For any element a, there exists a set S s.t. a is in S and

xS 1/≧ f .• From the constraints for covers.

– Therefore, every element is chosen.

At most f

xS の値

From the constraints of covers,the sum of the areas of is at least 1.

1/f

S s.t. a is in S

= f (1/f )=1.

f

Area of

Proof of Theorem 14.2 (2/2)

How much is the cost of A1 (COST) ?

– Let OPTLP (OPTf in the text) be the cost of a solution by the LP-relaxation.

– Let xS be a solution by the LP-relax., and yS rounded one.

• yS ≦ f xS holds since

– xS 1/≧ f, f xS 1=≧ yS if yS=1.

– xS 0, ≧ f xS 0=≧ yS if yS=0.

• Therefore, COST ≦ f OPTLP ≦f OPT..)(OPT,)(COST LP s

Ss

S

xScySc

SS

1/f

f xS xS

1

yS

1/f

f xS xS

1

yS

Example 14.3

A set consists of three connected elements in Vi.

A cost of each set is 1. f = 4. The optimal cost: 2. In the bottom figure, the

cost is 8.

V1 V2 V3

xS=1/4.

xS=1.

Overview

Set Cover– Approximation by simple rounding

• f-approx. algorithm (f: the frequency of the most frequent element).

– Approximation by randomized rounding• O(log n)-approx. algorithm (n: # elements to be covered).

Weighted Vertex Cover– Method based on half-integral solutions of the linear

programming• Each variable takes only 0, 1/2, or 1.• 2-approx. algorithm

Randomized rounding C=φ % C is a collection of picked sets. while (C doesn’t satisfy condition A)

– Find C by a manner explained later.• This C satisfies condition A with prob. more than 1/2.

end-while– [Condition A]

• C is a solution of set cover.• The cost of C is at most OPTLP・ 4clog n.

– c is some constant.

– The expectation T of executing loops in while-statement is at most 2.

.28

13

4

12

2

11 T

How to find C (1/2)

Compute a solution xS of the LP-relaxation. for i=1 to clog n

– Construct a family Ci of picked sets by choosing S with prob xS.

end-for C=∪Ci .

Solution byLP-relaxation

54321 SSSSS xxxxx

1.0

C1

S1

S2

C2

S1

S2

S4

C3

S1

S2

S5

C4

S1

S5

C

S1

S2

S4

S5

How to find C (2/2)

Compute a solution xS of the LP-relaxation. for i=1 to clog n

– Construct a family Ci of picked sets by choosing S with prob xS.

end-for C=∪Ci .

– C is not a set cover with prob. less than 1/4.

– The cost of C is more than OPTLP 4c log n with prob. less than 1/4.

Less than 1/4 Less than 1/4

More than 1/2

Prob. that element a is in Ci

Consider the example below. The prob. P that any set Si containing element a is

P=(1-xS1)(1-xS2)(1-xS3).

– xS1+xS2+xS3 1 from the constraints of covers.≧

P is maximized where xS1=xS2=xS3=1/3.

S1

S2S3

S5

a1

Maximum prob. a is not chosen

.21 dPPP k Suppose an element is in each of k sets.

Let

Fix d, and replace Pk as )( 121 kk PPPdP Then, the partial derivative of log g becomes

To simplify the problem,

)1(1

i

k

i

Pg

)1log(log1

i

k

i

Pg

instead ofmaximize

ki

kii

PP

PPdPP

g

1

1

1

1

1

1

1

1log

11 Pi 0

log g

iP

g

log

+ 0 －

Max

This shows Pi=Pk makes log g maximized.

This property holds for any i, then Pi=d/k.

Under the constraint that d 1, ≧ g takes the max. ((1-1/k)k) where d=1.

d/k

Prob. C is not a set cover Prob. a is not covered by using Ci is at most (1-

1/k)k.

Prob. a is not covered by C is at most (1/e)clogn.– Choose constant c s.t. (1/e)clogn 1/(4≦ n).– c 5 (4/log ≧ ≧ n)+1 (n ≧ 3).

.11

1

111

1

ek

ekk

k

Prob. C is not a set cover

Prob. at least one element is not in C is at most 1/4.

Less than 1/4n

At least one of a1, a2, a3 isnot chosen with prob. less

than 1/4.

Less than 1/4n

Less than 1/4n

a1 is not chosen. a2 is not chosen.

a3 is not chosen.

n=3

The cost of C

)in is Pr(log

)in isPr()in isPr(log

1

i

i

nc

i

CSnc

CSCS

LPOPTlog

)(log

)()in is Pr(log

)()in is Pr()](cost[

nc

Scxnc

ScCSnc

ScCSCE

SS

iS

S

Markov’s inequality (1/2)

Random variable X takes a non-negative value, and the average of X is μ.

x

xx

x

dxxXxP

dxxXxPdxxXxP

dxxXxP

)(

)()(

)(

0

0

≧0

≧ε

X

P(X=x)

x

x=ε

Markov’s inequality (2/2)

Random variable X takes a non-negative value, and the average of X is μ.

).()(

)(

)()(

)(

0

0

XPdxxXP

dxxXxP

dxxXxPdxxXxP

dxxXxP

x

x

xx

x

.)( XP

≧0

≧ε

The value of cost (C)

.OPTlog4

,OPTlog)](cost[),(cost

LP

LP

nc

ncCECX

Apply Markov’s inequality to cost (C).

.)( XP

.4

1

log4OPT

logOPT

log4OPT)log4OPT)(cost(

LP

LP

LPLP

nc

nc

ncncCP

Prob. the cost of C becomes morethan OPTLP4clog n is at most 1/4.

Each of the following two events happens with prob. less than 1/4.– C is not a set cover,

– The cost of C is more than c=OPTLP4clog n. Therefore, the event that C is a set cover and its cost is at

most c is at least 1/2.

Less than 1/4 Less than 1/4

More than 1/2

Overview

Set Cover– Approximation by simple rounding

• f-approx. algorithm (f: the frequency of the most frequent element).

– Approximation by randomized rounding• O(log n)-approx. algorithm (n: # elements to be covered).

Weighted Vertex Cover– Method based on half-integral solutions of the linear

programming• Each variable takes only 0, 1/2, or 1.• 2-approx. algorithm

Weighted vertex cover

Weighted vertex cover– Input: graph with weights on vertices G=(V,E).– Output: A ⊆V.

• For any (u,v)∈E, u∈A or v∈A .

• The sum of weights of v∈A is minimized.

14 1

2 55

14 1

2 55

v1 v2 v3

v4 v5 v6

v1 v2 v3

v4 v5 v6

Weighted vertex cover

Definition– Input: Graph G=(V,E).– Output: A ⊆V.

• For any edge (u,v)∈E, u∈A or v∈A .

• The sum of weights of v∈A is minimized.

14 1

2 55

a1 a2

a3

a6

a4

a5

14 1

2 55

a1 a2

a3

a6

a4

a5

v1 v2 v3

v4 v5 v6

v1 v2 v3

v4 v5 v6

Formulation by linear inequalities

Objective function– Minimize:

Constraints– For covers– For choosing edges

Vv

vxvc )(

Evuxx vu ),( ,1Vvxv },1,0{

14 1

2 55

a1 a2

a3

a6

a4

a5

14 1

2 55

a1 a2

a3

a6

a4

a5

v1 v2 v3

v4 v5 v6

v1 v2 v3

v4 v5 v6

Formulation by linear inequalities

Objective function– Minimize:

Constraints– For covers

– For choosing edges

654321525114 vvvvvv xxxxxx

}.1,0{,,,,,654321vvvvvv xxxxxx

14 1

2 55

a1 a2

a3

a6

a4

a5

14 1

2 55

a1 a2

a3

a6

a4

a5

v1 v2 v3

v4 v5 v6

v1 v2 v3

v4 v5 v6

.1,1,1

,1,1,1

655463

413221

vvvvvv

vvvvvv

xxxxxx

xxxxxx

LP-relaxation

Objective function– Minimize:

Constraints– For covers– For choosing edges

Vv

vxvc )(

Evuxx vu ),( ,1

Vvxv ,0

14 1

2 55

a1 a2

a3

a6

a4

a5

14 1

2 55

a1 a2

a3

a6

a4

a5

v1 v2 v3

v4 v5 v6

v1 v2 v3

v4 v5 v6

LP-relaxation

Objective function– Minimize:

Constraints– For covers

– For choosing edges

654321525114 vvvvvv xxxxxx

.0,,,,,654321vvvvvv xxxxxx

14 1

2 55

a1 a2

a3

a6

a4

a5

14 1

2 55

a1 a2

a3

a6

a4

a5

v1 v2 v3

v4 v5 v6

v1 v2 v3

v4 v5 v6

.1,1,1

,1,1,1

655463

413221

vvvvvv

vvvvvv

xxxxxx

xxxxxx

Extreme point solution

The optimal solution of Linear Programming. The solution which cannot be expressed as

convex combination of two other feasible solution.– Convex combination: A linear equation s.t. the sum of

its coefficients is 1.• z is a convex combination of x and y, where z =0.8x+0.2y.

Feasible solution

Convex combinationof feasible solution

Half-integral solution

Solution of Linear Programming s.t. each value takes 0, 1/2 or 1.

2-approximation algorithm

Compute an extreme point solution x. Choose any vertex s.t its corresponding value

takes 1/2 or 1.– If x is an extreme point solution, each variable takes 0,

1/2, or 1. (Lemma 14.4)

Lemma 14.4

x: a solution of weighted vertex cover obtained by Linear Programming.

If x is not half-integral, x can be expressed as convex combination of two other feasible solution.– x is not an extreme point solution.

– Outline of its proof• Construct y and z s.t. x is not half-integral and x=1/2(y+z).

– x can be expressed by convex combination of y and z.

• Show y and z are feasible solutions.

Proof of Lemma 14.4 (1/3)

Construct other solutions y and z from x, each of them takes 0, 1/2, or 1.

54321 vvvvv xxxxx

1.0x

0.5

1.0y

0.5

1.0z

0.5

+ε

－ ε

－ ε

+ε

V+={v3}.54321 vvvvv xxxxx

54321 vvvvv xxxxx

vi s.t. xvi > 1/2.

V － ={v4}. vi s.t. xvi < 1/2.

+εin y, － εin z.

－ εin y, +εin z.

zyx 2

1holds.

Proof of Lemma 14.4 (2/3)

Are y and z feasible solutions?– In any feasible solution, xu+xv 1 holds.≧

xu

xv

1/2 1

1/2

1

Feasible solution

yu

yv

1/2 1

1/2

1

Change from x to y

zu

zv

1/2 1

1/2

1

Change from x to z

Where εis set to a small value,y and z are feasible solutions.

Proof of Lemma 14.4 (3/3)

When xu＋ xv=1,– xu= xv=1/2.

• yu= yv=zu= zv=1/2 (no change).

– xu=0, xv=1.• yu=0, yv=1, zu=0, zv=1 (no change).

– xu<1/2, xv>1/2.• yu＋ yv= xu+ε ＋ xv -ε =1,• zu＋ zv= xu-ε ＋ xv +ε =1.

Then, y and z are feasible solutions, and any solution can be expressed by a half-integral solution.