approximation algorithms:
DESCRIPTION
Approximation Algorithms:. problems, techniques, and their use in game theory. Éva Tardos Cornell University. What is approximation?. Find solution for an optimization problem guaranteed to have value close to the best possible. How close? additive error: (rare) - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Approximation Algorithms:](https://reader031.vdocuments.us/reader031/viewer/2022020921/56814d4c550346895dba79df/html5/thumbnails/1.jpg)
Approximation Algorithms:
Éva TardosCornell University
problems, techniques, and their use in game theory
![Page 2: Approximation Algorithms:](https://reader031.vdocuments.us/reader031/viewer/2022020921/56814d4c550346895dba79df/html5/thumbnails/2.jpg)
FOCS 2002 2
What is approximation?Find solution for an optimization
problem guaranteed to have value close to the best possible.
How close?• additive error: (rare)
– E.g., 3-coloring planar graphs is NP-complete, but 4-coloring always possible
• multiplicative error: -approximation: finds solution for
an optimization problem within an factor to the best possible.
![Page 3: Approximation Algorithms:](https://reader031.vdocuments.us/reader031/viewer/2022020921/56814d4c550346895dba79df/html5/thumbnails/3.jpg)
FOCS 2002 3
Why approximate?
• NP-hard to find the true optimum
• Just too slow to do it exactly
• Decisions made on-line
• Decisions made by selfish players
![Page 4: Approximation Algorithms:](https://reader031.vdocuments.us/reader031/viewer/2022020921/56814d4c550346895dba79df/html5/thumbnails/4.jpg)
FOCS 2002 4
Outline of talk
Techniques: • Greedy• Local search• LP techniques:• rounding• Primal-dual
Problems:
• Disjoint paths• Multi-way cut and labeling•network design, facility location
Relation to Games– local search price of anarchy– primal dual cost sharing
![Page 5: Approximation Algorithms:](https://reader031.vdocuments.us/reader031/viewer/2022020921/56814d4c550346895dba79df/html5/thumbnails/5.jpg)
FOCS 2002 5
Max disjoint paths problem
Given graph G, n nodes, m edges, and source-sink pairs.
Connect as many as possible via edge-disjoint path.
t
s t
s s
t
t
s
![Page 6: Approximation Algorithms:](https://reader031.vdocuments.us/reader031/viewer/2022020921/56814d4c550346895dba79df/html5/thumbnails/6.jpg)
FOCS 2002 6
Greedy AlgorithmGreedily connect s-t pairs
via disjoint paths, if there is a free path using at most m½ edges:
m½ 4
s t
s s
t
t
s t
If there is no short path at all, take a single long one.
![Page 7: Approximation Algorithms:](https://reader031.vdocuments.us/reader031/viewer/2022020921/56814d4c550346895dba79df/html5/thumbnails/7.jpg)
FOCS 2002 7
Greedy AlgorithmTheorem: m½ –approximation.
Kleinberg’96Proof: One path used can block
m½ better paths m½ 4
s t
s s
t
t
s t
Essentially best possible: m½- lower bound unless P=NP by [Guruswami, Khanna, Rajaraman, Shepherd, Yannakakis’99]
![Page 8: Approximation Algorithms:](https://reader031.vdocuments.us/reader031/viewer/2022020921/56814d4c550346895dba79df/html5/thumbnails/8.jpg)
FOCS 2002 8
Disjoint paths:open problem
Connect as many as pairs possible via paths where 2 paths may share any edge
t
s t
s s
t
t
s
• Same practical motivation• Best greedy algorithm: n½ - (and also m1/3 -) approximation: Awerbuch, Azar, Plotkin’93.
• No lower bound …
![Page 9: Approximation Algorithms:](https://reader031.vdocuments.us/reader031/viewer/2022020921/56814d4c550346895dba79df/html5/thumbnails/9.jpg)
FOCS 2002 9
Outline of talk
Techniques: • Greedy• Local search• LP techniques:• rounding• Primal-dual
Problems:
• Disjoint paths• Multi-way cut and labeling•network design, facility location
Relation to Games– local search Price of anarchy– primal dual Cost sharing
![Page 10: Approximation Algorithms:](https://reader031.vdocuments.us/reader031/viewer/2022020921/56814d4c550346895dba79df/html5/thumbnails/10.jpg)
FOCS 2002 10
Multi-way Cut ProblemGiven:
– a graph G = (V,E) ;– k terminals {s1, …, sk}– cost we for each edge e
Goal: Find a partition that separates terminals, and minimizes the cost
{e separated} weSeparated edgess1
s2
s3
s4
![Page 11: Approximation Algorithms:](https://reader031.vdocuments.us/reader031/viewer/2022020921/56814d4c550346895dba79df/html5/thumbnails/11.jpg)
FOCS 2002 11
Greedy Algorithm
For each terminal in turn– Find min cut separating si
from other terminals The first cut
The next cut
s2
s1
s4
s3
s2
s1
s4
s3
![Page 12: Approximation Algorithms:](https://reader031.vdocuments.us/reader031/viewer/2022020921/56814d4c550346895dba79df/html5/thumbnails/12.jpg)
FOCS 2002 12
Theorem: Greedy is a2-approximation
Proof: Each cut costs at most the optimum’s cut [Dahlhaus, Johnson, Papadimitriou, Seymour, and Yannakakis’94]
Cuts found by algorithm:
Optimum partition
Selected cuts, cheaper than optimum’s cut, but
each edge in optimum is counted twice.
s4
s3
s2
s1
![Page 13: Approximation Algorithms:](https://reader031.vdocuments.us/reader031/viewer/2022020921/56814d4c550346895dba79df/html5/thumbnails/13.jpg)
FOCS 2002 13
Multi-way cuts extension
Given: – graph G = (V,E), we0 for e E– Labels L={1,…,k} – Lv L for each node v
Objective: Find a labeling of nodes such that each node v assigned to a label in Lv and it minimizes cost {e separated} we
Separated edges
part 1
part 2
part 3
part 4
![Page 14: Approximation Algorithms:](https://reader031.vdocuments.us/reader031/viewer/2022020921/56814d4c550346895dba79df/html5/thumbnails/14.jpg)
FOCS 2002 14
Example
Does greedy work? For each terminal in turn
– Find min cut separating si from other terminals
Blue or greenRed
or g
reen
Red or blue
cheapmediumexpensive
s3
s1
s2
![Page 15: Approximation Algorithms:](https://reader031.vdocuments.us/reader031/viewer/2022020921/56814d4c550346895dba79df/html5/thumbnails/15.jpg)
FOCS 2002 15
Greedy doesn’t workGreedyFor each terminal in turn
– Find min cut separating si from other terminals
The first two cuts:
Remaining part not valid!
Blue or greenRed
or g
reen
Red or blues2
s1
s3
![Page 16: Approximation Algorithms:](https://reader031.vdocuments.us/reader031/viewer/2022020921/56814d4c550346895dba79df/html5/thumbnails/16.jpg)
FOCS 2002 16
Local search
[Boykov Veksler Zabih CVPR’98] 2-approximation
1. Start with any valid labeling.
2. Repeat (until we are tired):a. Choose a color c.b. Find the optimal move where a
subset of the vertices can be recolored, but only with the color c.
(We will call this a c-move.)
![Page 17: Approximation Algorithms:](https://reader031.vdocuments.us/reader031/viewer/2022020921/56814d4c550346895dba79df/html5/thumbnails/17.jpg)
FOCS 2002 17
A possible -move
Thm [Boykov, Vekler, Zabih] The best -move can be found via an (s,t) min-cut
![Page 18: Approximation Algorithms:](https://reader031.vdocuments.us/reader031/viewer/2022020921/56814d4c550346895dba79df/html5/thumbnails/18.jpg)
FOCS 2002 18
Idea of the flow networkfor finding a -move
s = all other terminals: retain current color
sc = change color to c =
G
![Page 19: Approximation Algorithms:](https://reader031.vdocuments.us/reader031/viewer/2022020921/56814d4c550346895dba79df/html5/thumbnails/19.jpg)
FOCS 2002 19
Theorem: local optimum is a 2-approximation
Partition found by algorithm:
Cuts used by optimum
The parts in optimum each give a possible local move:
![Page 20: Approximation Algorithms:](https://reader031.vdocuments.us/reader031/viewer/2022020921/56814d4c550346895dba79df/html5/thumbnails/20.jpg)
FOCS 2002 20
Theorem: local optimum is a 2-approximation
Partition found by algorithm:
Possible move using the optimum
Changing partition does not help current cut cheaperSum over all colors:
Each edge in optimum counted twice
![Page 21: Approximation Algorithms:](https://reader031.vdocuments.us/reader031/viewer/2022020921/56814d4c550346895dba79df/html5/thumbnails/21.jpg)
FOCS 2002 21
Metric labeling classification
open problemGiven:
– graph G = (V,E); we0 for e E– k labels L– subsets of allowed labels Lv – a metric d(.,.) on the labels.
Objective: Find labeling f(v)Lv for each node v to minimizee=(v,w) we d(f(v),f(w))
Best approximation known: O(ln k ln ln k) Kleinberg-T’99
![Page 22: Approximation Algorithms:](https://reader031.vdocuments.us/reader031/viewer/2022020921/56814d4c550346895dba79df/html5/thumbnails/22.jpg)
FOCS 2002 22
Outline of talk
Techniques: • Greedy• Local search• LP techniques:• rounding• Primal-dual
Problems:
• Disjoint paths• Multi-way cut and labeling•network design, facility location
Relation to Games– local search Price of anarchy– primal dual Cost sharing
![Page 23: Approximation Algorithms:](https://reader031.vdocuments.us/reader031/viewer/2022020921/56814d4c550346895dba79df/html5/thumbnails/23.jpg)
FOCS 2002 23
Using Linear Programs for multi-way cuts
Using a linear program = fractional cut probabilistic assignment of
nodes to parts
?
Idea: Find “optimal” fractional labeling via linear programming
Label ? as : ½ + ½
![Page 24: Approximation Algorithms:](https://reader031.vdocuments.us/reader031/viewer/2022020921/56814d4c550346895dba79df/html5/thumbnails/24.jpg)
FOCS 2002 24
Fractional Labeling
Variables:0 xva 1 p=node, a=label in Lv
– xva fraction of label a used on node v
Constraints:
xva = 1aLv
for all nodes v V
– each node is assigned to a label
cost as a linear function of x: we ½ |xua - xva |
e=(u,v) aL
![Page 25: Approximation Algorithms:](https://reader031.vdocuments.us/reader031/viewer/2022020921/56814d4c550346895dba79df/html5/thumbnails/25.jpg)
FOCS 2002 25
From Fractional x to multi-way cut
The Algorithm (Calinescu, Karloff, Rabani, ’98, Kleinberg-T,’99)
While there are unassigned nodes• select a label a at random
xva
1
u v Unassigned nodes
![Page 26: Approximation Algorithms:](https://reader031.vdocuments.us/reader031/viewer/2022020921/56814d4c550346895dba79df/html5/thumbnails/26.jpg)
FOCS 2002 26
The Algorithm (Cont.)
While there are unassigned nodes– select a label a at random
xva
1
u vUnassigned nodes
select 0 1 at randomassign all unassigned nodes v to selected label a if xva
![Page 27: Approximation Algorithms:](https://reader031.vdocuments.us/reader031/viewer/2022020921/56814d4c550346895dba79df/html5/thumbnails/27.jpg)
FOCS 2002 27
Why Is This Choice Good?
select 0 1 at randomassign all unassigned nodes v to
selected label a if xva
Note:• Probability of assigning node v to
label a is xva • Probability of separating nodes u
and v in this iteration is |xua – xva |
xpa
1
p qUnassigned nodes
![Page 28: Approximation Algorithms:](https://reader031.vdocuments.us/reader031/viewer/2022020921/56814d4c550346895dba79df/html5/thumbnails/28.jpg)
FOCS 2002 28
From Fractional x to Multi-way cut (Cont.)
Theorem: Given a fractional x, we find multi-way cut with expected
separation cost 2 (LP cost of x)
Corollary: if x is LP optimum . 2-approximation
Calinescu, Karloff, Rabani, ’98 1.5 approximation for multi-way cut
(does not work for labeling)
Karger, Klein, Stein, Thorup, Young’99 improved bound 1.3438..
![Page 29: Approximation Algorithms:](https://reader031.vdocuments.us/reader031/viewer/2022020921/56814d4c550346895dba79df/html5/thumbnails/29.jpg)
FOCS 2002 29
Outline of talk
Techniques: • Greedy• Local search• LP techniques:• rounding• Primal-dual
Problems:
• Disjoint paths• Multi-way cut and labeling•network design, facility location
Relation to Games– local search Price of anarchy– primal dual Cost sharing
![Page 30: Approximation Algorithms:](https://reader031.vdocuments.us/reader031/viewer/2022020921/56814d4c550346895dba79df/html5/thumbnails/30.jpg)
FOCS 2002 30
Metric Facility Location
F is a set of facilities (servers).D is a set of clients.
cij is the distance between any i and j in D F.
Facility i in F has cost fi.
clientfacility5
4
23
![Page 31: Approximation Algorithms:](https://reader031.vdocuments.us/reader031/viewer/2022020921/56814d4c550346895dba79df/html5/thumbnails/31.jpg)
FOCS 2002 31
Problem Statement
We need to:1) Pick a set S of facilities to open. 2) Assign every client to an open
facility (a facility in S).
Goal: Minimize cost of S + p dist(p,S).
clientfacility5
4
23
openedfacility
![Page 32: Approximation Algorithms:](https://reader031.vdocuments.us/reader031/viewer/2022020921/56814d4c550346895dba79df/html5/thumbnails/32.jpg)
FOCS 2002 32
What is known?
All techniques can be used:• Clever greedy [Jain, Mahdian,
Saberi ’02]
• Local search [starting with Korupolu, Plaxton, and Rajaraman ’98], can handle capacities
• LP and rounding: [starting with Shmoys, T, Aardal ’97]
Here: primal-dual [starting with Jain-Vazirani’99]
![Page 33: Approximation Algorithms:](https://reader031.vdocuments.us/reader031/viewer/2022020921/56814d4c550346895dba79df/html5/thumbnails/33.jpg)
FOCS 2002 33
What is the primal-dual method?
• Uses economic intuition from cost sharing– For each requirement, like
aLv xva = 1, someone has to pay to make it true…
• Uses ideas from linear programming:– dual LP and weak duality– But does not solve linear
programs
![Page 34: Approximation Algorithms:](https://reader031.vdocuments.us/reader031/viewer/2022020921/56814d4c550346895dba79df/html5/thumbnails/34.jpg)
FOCS 2002 34
Dual Problem: Collect Fees
Client p has a fee αp (cost-share)
Goal: collect as much as possible max p αp
Fairness: Do no overcharge: for any subset A of clients and any possible facility i we must have p A [αp – dist(p,i)] fi
amount client p would contribute to building facility i.
![Page 35: Approximation Algorithms:](https://reader031.vdocuments.us/reader031/viewer/2022020921/56814d4c550346895dba79df/html5/thumbnails/35.jpg)
FOCS 2002 35
Exact cost-sharing
• All clients connected to a facility
• Cost share αp covers connection costs for each client p
• Costs are “fair”• Cost fi of selecting a facility i is
covered by clients using it
p αp = f(S)+ p dist(p,S) , and
both facilities are fees are optimal
![Page 36: Approximation Algorithms:](https://reader031.vdocuments.us/reader031/viewer/2022020921/56814d4c550346895dba79df/html5/thumbnails/36.jpg)
FOCS 2002 36
Approximate cost-sharing
Idea 1: each client starts unconnected, and with fee αp=0
Then it starts raising what it is willing to pay to get connected
• Raise all shares evenly αExample:
= client= possible facility with its cost
4 4
4
![Page 37: Approximation Algorithms:](https://reader031.vdocuments.us/reader031/viewer/2022020921/56814d4c550346895dba79df/html5/thumbnails/37.jpg)
FOCS 2002 37
Primal-Dual Algorithm (1)
• Each client raises his fee α evenly what it is willing to pay
α = 1
Its α =1 share could be used towards building a connection to either facility
4 4
![Page 38: Approximation Algorithms:](https://reader031.vdocuments.us/reader031/viewer/2022020921/56814d4c550346895dba79df/html5/thumbnails/38.jpg)
FOCS 2002 38
Primal-Dual Algorithm (2)
• Each client raises evenly what it is willing to pay
Starts contributing towards facility cost
α = 2
4 4
![Page 39: Approximation Algorithms:](https://reader031.vdocuments.us/reader031/viewer/2022020921/56814d4c550346895dba79df/html5/thumbnails/39.jpg)
FOCS 2002 39
Primal-Dual Algorithm (3)
• Each client raises evenly what it is willing to pay
Three clients contributing
α = 3
4 4
![Page 40: Approximation Algorithms:](https://reader031.vdocuments.us/reader031/viewer/2022020921/56814d4c550346895dba79df/html5/thumbnails/40.jpg)
FOCS 2002 40
Primal-Dual Algorithm (4)
Open facility, when cost is covered by contributions
4
clients connected to open facility
Open facility
α = 3
4
![Page 41: Approximation Algorithms:](https://reader031.vdocuments.us/reader031/viewer/2022020921/56814d4c550346895dba79df/html5/thumbnails/41.jpg)
FOCS 2002 41
Primal-Dual Algorithm: Trouble
Trouble: – one client p connected to
facility i, but contributes to also to facility j
4
Open facility
α = 3
4i j
p
![Page 42: Approximation Algorithms:](https://reader031.vdocuments.us/reader031/viewer/2022020921/56814d4c550346895dba79df/html5/thumbnails/42.jpg)
FOCS 2002 42
Primal-Dual Algorithm (5)
Close facility j: will not open this facility.
Will this cause trouble?• Client p is close to both i and j
facilities i and j are at most 2α from each other.
4
Open facility
α = 3
4i j
p
ghost
![Page 43: Approximation Algorithms:](https://reader031.vdocuments.us/reader031/viewer/2022020921/56814d4c550346895dba79df/html5/thumbnails/43.jpg)
FOCS 2002 43
4
Primal-Dual Algorithm (6)
Not yet connected clients raise their fee evenly
Until all clients get connected
4
no not need to pay more than 3
Open facility
α =6 α =3
α =3 α =3
ghost
![Page 44: Approximation Algorithms:](https://reader031.vdocuments.us/reader031/viewer/2022020921/56814d4c550346895dba79df/html5/thumbnails/44.jpg)
FOCS 2002 44
Feasibility + fairness ?? All clients connected to
a facility Cost share αp covers
connection costs of client p
Cost fi of opening a facility i is covered by clients connected to it
• ?? Are costs “fair” ??
![Page 45: Approximation Algorithms:](https://reader031.vdocuments.us/reader031/viewer/2022020921/56814d4c550346895dba79df/html5/thumbnails/45.jpg)
FOCS 2002 45
a set of clients A, and any possible facility i we have
p A [αp – dist(p,i)] fi
– Why? we open facility i if there is enough contribution, and do not raise fees any further
But closed facilities are ignored! and may violate fairness
Are costs “fair”??
44
open facility
closed facility, ignored
![Page 46: Approximation Algorithms:](https://reader031.vdocuments.us/reader031/viewer/2022020921/56814d4c550346895dba79df/html5/thumbnails/46.jpg)
FOCS 2002 46
Fair till it reaches a “ghost” facility.
Let α’q αq be the fee till a ghost facility is reached
Are costs “fair”??
44
open facility
Closed facility, ignored
cause of closing
ji
p
α’q=4
![Page 47: Approximation Algorithms:](https://reader031.vdocuments.us/reader031/viewer/2022020921/56814d4c550346895dba79df/html5/thumbnails/47.jpg)
FOCS 2002 47
Feasibility + fairness ?? All clients connected to a
facility Cost share αp covers
connection costs for client p Cost αp also covers cost of
selected a facilities Costs α’p are “fair”How much smaller is α’ α ??
44
p
![Page 48: Approximation Algorithms:](https://reader031.vdocuments.us/reader031/viewer/2022020921/56814d4c550346895dba79df/html5/thumbnails/48.jpg)
FOCS 2002 48
How much smaller is α’ α?
q client met ghost facility j j became a ghost due to client p
qi
p stopped raising its share first αp α’q αq
Recall dist(i,j) 2 αp, soαq α’q +2 αp 3α’q
44
p
j
![Page 49: Approximation Algorithms:](https://reader031.vdocuments.us/reader031/viewer/2022020921/56814d4c550346895dba79df/html5/thumbnails/49.jpg)
FOCS 2002 49
Primal-dual approximation
The algorithm is a 3-approximation algorithm for the facility location problem
[Jain-Vazirani’99, Mettu-Plaxton’00]Proof: Fairness of the α’p fees
p α’p min cost [max min]
cost-recovery:f(S) + p dist(p,S) = p αp
α 3α’q
3-approximation algorithm
![Page 50: Approximation Algorithms:](https://reader031.vdocuments.us/reader031/viewer/2022020921/56814d4c550346895dba79df/html5/thumbnails/50.jpg)
FOCS 2002 50
Outline of talk
Techniques: • Greedy• Local search• LP techniques:• rounding• Primal-dual
Problems:
• Disjoint paths• Multi-way cut and labeling•network design, facility location
Relation to Games– primal dual Cost sharing– local search Price of anarchy
![Page 51: Approximation Algorithms:](https://reader031.vdocuments.us/reader031/viewer/2022020921/56814d4c550346895dba79df/html5/thumbnails/51.jpg)
FOCS 2002 51
primal dual Cost sharing
Dual variables αp are natural cost-shares:
Recall: fair = no set is overcharged
= core allocationp Aαp – dist(p,i) fi for all A and i.
[Chardaire’98; Goemans-Skutella’00] strong connection between core cost-allocation and linear programming dual solutions
See also Shapley’67, Bondareva’63 for other games
![Page 52: Approximation Algorithms:](https://reader031.vdocuments.us/reader031/viewer/2022020921/56814d4c550346895dba79df/html5/thumbnails/52.jpg)
FOCS 2002 52
Primal-Dual Cost-sharing
Primal dual = for each requirement someone willing to pay to make it true
Cost-sharing: only players can have shares.
• Not all requirements are naturally associated with individual players.
• Real players need to share the cost.
![Page 53: Approximation Algorithms:](https://reader031.vdocuments.us/reader031/viewer/2022020921/56814d4c550346895dba79df/html5/thumbnails/53.jpg)
FOCS 2002 53
primal dual Cost sharing
Fair no subset is overcharged
Stronger desirable property: population monotone (cross-monotone):
Extra clients do not increase cost-shares.
• Spanning-tree game: [Kent and Skorin-Kapov’96 and Jain Vazirani’01]
• Facility location, single source rent-or-buy [Pal-T’02]
![Page 54: Approximation Algorithms:](https://reader031.vdocuments.us/reader031/viewer/2022020921/56814d4c550346895dba79df/html5/thumbnails/54.jpg)
FOCS 2002 54
Local search (for facility location)
Local search: simple search steps to improve objective:
• add(s) adds new facility s• delete(t) closes open facility t• swap(s,t) replaces open facility
s by a new facility t
Key to approximation bound:How bad can be a local optima?3-approximation [Charikar, Guha’00]
![Page 55: Approximation Algorithms:](https://reader031.vdocuments.us/reader031/viewer/2022020921/56814d4c550346895dba79df/html5/thumbnails/55.jpg)
FOCS 2002 55
Local search Price of anarchy in games
Price of anarchy: facilities are operated by separate selfish agents
Agents open/close facilities when it benefits their own objective.
Agent’s “best response” dynamic:• Simple local steps analogous to
local search.Price of anarchy: • How bad can be a stable state?• 2-approximation in a related
maximization game: [Vetta’02]
![Page 56: Approximation Algorithms:](https://reader031.vdocuments.us/reader031/viewer/2022020921/56814d4c550346895dba79df/html5/thumbnails/56.jpg)
FOCS 2002 56
Conclusions for approximation
Greedy, Local search• clever greedy/local steps can
lead to great resultsPrimal-dual algorithms• Elegant combinatorial methods• Based on linear programming
ideas, but fast, avoids explicitly solving large linear programs
Linear programming• very powerful tool, but slow to
solveInteresting connections to
issues in game theory