algorithm design - heinz nixdorf institut · 2012-05-31 · friedhelm meyer auf der heide 1 heinz...

Friedhelm Meyer auf der Heide 1

HEINZ NIXDORF INSTITUTE

University of Paderborn

Algorithms and Complexity

Welcome to the course

Algorithm Design

Summer Term 2011

Friedhelm Meyer auf der Heide

Lecture 13, 15.7.2011




Algorithms and ComplexityTopics

- Divide & conquer

- Dynamic programming

- Greedy Algorithms

- Randomized Algorithms

- Approximation Algorithms

- Online Algorithms





Chapter 1:

Divide & Conquer




Algorithms and ComplexityDivide & Conquer

Divide the problem in a number of smaller subproblems.

Conquer the subproblems by solving them recursively.

Combine the results of the subproblem into a solution

of the original problem.

Our Examples:

Integer multiplication, matrix multiplication (HW), closest pair

of points in the plane, Fast Fourier transform, ….(HW)





Chapter 2:

Dynamic Progamming




Algorithms and ComplexityDynamic programming

You probably know dynamic programming algorithms for

optimal evaluation of the product of many matrices, and for the

word problem for context-free languages (CYK-algorithm)

Examples:

String Distance, Shortest Paths, Knapsack, Minimum Weight

Triangulation (HW), Optimal Search Trees (HW)





Chapter 3:

Greedy Algorithms




Algorithms and ComplexityGreedy Algorithms

Examples:Activity Selection (or Interval Scheduling), Scheduling to Minimize Lateness

Theoretical Foundations of the Greedy Method: Matroid TheoryExamples for Matroids: Linear independent vectors, Minimum Spanning Trees, Bipartite matching with maximum left side





For which problems are Greedy-Algorithms

optimal?

Consider a finite set E and a system U of subsets of E. (E,U) is called a

subset-system, if the following holds:

(i) ? 2 U

(ii) For each B 2 U, also each subset of B is in U.

B2 U is called maximal, if no proper superset of B is in U.

The Optimization problem corresponding to (E,U) is :

Given a weight function w:EQ+ ,compute a maximal set B 2 U with

maximizes w(B) = §e2 B w(e).

(Minimization problems are described analagously.)





Systems of Subsets and the

Canonical Greedy Algorithm

Canonical Greedy ((E,U))

(1) Sort E such that w(e1) ¸ … ¸ w(en).

(2) B?.

(3) For k=1 to n

if B [ {ek} 2 U then B B [ {ek}

(4) Return B

For which types of subset-systems is the canonical Greedy

algorithm optimal?




Algorithms and ComplexityExample 2

Problem: Minimum spanning tree (MST)

Input: a graph G=(V,E) with positive edge weights w:EQ+ .

Output: a spanning tree with minimum weight.

MST is a subset-system:

- E : the set of edges,

- U : all subsets of E that do not contain a cycle

form a subset-system




Algorithms and ComplexityGreedy-Algorithms und Matroids

A system of subsets (E,U) is a matroid, if in addition, the following exchange

property holds:

(iii) For all A,B 2 U with |A|<|B|, there is x2 B-A such that A[{x} 2 U.

Remark: All maximal sets of a matroid have the same size. (homework)

We will see:

- The system of subsets for 0-1 knapsack (Example 1) is not a matroid.

- The systems of subsets for examples 0,2,3 are a matroids.

Theorem: Consider a system of subsets (E,U). canonical Greedy is optimal

for (E,U) for every weight function w, if and only if (E,U) is a matroid.

Kruskal‟s algorithm is optimal for MST (example 2)!





Chapter 4.

Randomised Algorithms




Algorithms and ComplexityRandomized Algorithms

… use the outcome of random experiments (“coin flips”) to guide the execution of the algorithm.

For a fixed input, runtime and output may be random variables.




Algorithms and ComplexityRandomized Algorithms

Examples:

Quicksort, Skip Lists, Contention Resolution (symmetry -breaking), Global Minimum Cut (contraction algorithm), Random Variables and their Expectations (Guessing Cards, Coupon Collector)

Chernoff Bounds, example Load Balancing




Algorithms and ComplexityCuts in graphs

Let G=(V,E) be a connected, undirected graph.

A cut in G is defined by a disjoint partition of V into two sets A

and B. Its size is the number of edges between A and B.

Goal: Compute a cut with minimum size.

A cut of size 6.

A B




Algorithms and ComplexityTools from Probability Theory: Chernoff Bounds

Prob(|X-¹|)¸ ± ¹) is „inverse exponential“ in n !!!!





Chapter 5.

Approximation Algorithms





26

Approximation Algorithms

Examples:

Greedy Techniques (Load-Balancing, Center Selection )

Pricing Method (Vertex Cover)

Linear Programming and Rounding (Vertex Cover, generalized Load-Balancing )

Polynomial Time Approximation Scheme (Knapsack )





15.07.2011 29




Algorithms and ComplexityApproximation via “rounding” linear programs





minimize





Chapter 6.

Online Algorithms




Algorithms and ComplexityOnline Algorithms

An online algorithm is one that can process its input piece-by-piece,

without having the entire input available from the start.

In contrast, an offline algorithm is given the whole problem data from the

beginning and is required to output an answer which solves the problem

at hand.

Input: a sequence of requests.

Task: process the requests as efficiently as possible

Online:

i„th request has to be processed before future requests are known

Offline:

All requests are known in advance




Algorithms and ComplexityHow to measure quality of online algorithms?

1. Assume some a priori knowledge about request

sequence, e.g., „requests are chosen randomly“

2. Assume worst case measure, compare online cost to

offline cost

Online : standard competitive analysis – competitive ratio

Online randomized:





Examples:

Paging, page migration, data management in

networks





Page migration – Classical online problem

• processors connected by a network

There are costs of communication associated with each

edge. Cost of communication between pair of nodes =

cost of the cheapest path between these nodes.

• Costs of communication fulfill the triangle inequality.

Page Migration Model (1)

v1

v2

v3

v4

v7

v6

v5





Alternative view:

• processors in a metric space

Indivisible memory page of size in the local memory of

one processor (initially at )

Page Migration Model (2)

v1

v2

v3

v4

v7

v6

v5




Algorithms and ComplexityPage Migration Model (3)

Input: sequence of processors,

dictated by a request adversary

- processor which wants to access (read or write)

one unit of data from the memory page.

After serving a request an algorithm may move the page

to a new processor.

v1

v2

v3

v4

v7

v6

v5




Algorithms and ComplexityPage Migration (cost model)

Cost model:

The page is at node .

Serving a request issued at costs .

Moving the page to node costs .




Algorithms and ComplexityA randomized algorithm

Memoryless coin-flipping algorithm CF [Westbrook 92]

Theorem: CF is 3-competitive against an adaptive-online

adversary (may see the outcomes of the coinflips).

Remark: This ratio is optimal against adaptive-online

adversary.

In each step, after serving a request issued at ,

move page to with probability .




Algorithms and ComplexityDeterministic algorithm

Algorithm Move-To-Min (MTM) [Awerbuch, Bartal, Fiat 93]

Theorem: MTM is 7-competitive

Remark: The currently best deterministic algorithm achieves

competitive ratio of 4.086

After each steps, choose to be the node

which minimizes , and move to .

( is the best place for the page in the last steps)





Data management in networks





Scenario

Networks have low bandwidth, global objects are small, access is

fine grained.

• typical for parallel processor networks, partially also for the internet.

• bottleneck: link-congestion

task: distribute global objects (maybe dynamically) among

processors such that

• an application (sequence of read/write access to global

variables) can be executed using small link-congestion

• storage overhead is small.

- Exploit Locality -





Basic Strategy

Design strategy for trees

Produce strategy for target-network by tree embedding





Dynamic Model

Application: Sequence of read / write requests from processors

to objects. Each processor decides solely based on

its local knowledge.

distributed online-strategy

Goal: Develop strategy that produces only by a factor c more

congestion than an optimal offline strategy.

c-competitive strategy

(and by a factor m more storage per processor

(m, c) – competitive strategy )





Dynamic strategy for trees

v writes to x :

v creates (or updates) copy of x in v,

and invalidates all other copies (consistency!)

v reads x:

v reads the closest copy of x and creates copies in

every processor on the path back to v.

(Remark: Data Tracking in trees is easy!)





Example and Analysis

Consider phase write (v0), read (v1), read (v2), ... , read (vk-1), write (vk)

v0







v0

v1







v0

v1

v2







v0

v1

v2

v3







v0

v1

v2

v3

vk







v0

v1

v2

v3

Each strategy has to use each link of the red subtree at least once.

Our strategy uses each of these links at most three times.

Strategy is 3-competitive for trees

vk





Other networks

Idea: Simulate suitable tree in target-network M.

tree embedding:

Goals: - small dilation

(in order to reduce overall load)

- randomized embedding

(in order to reduce congestion)

Goals contradict?!?





Tree embedding

Example: nxn-mesh1 2 30

M„

v

• leaves: nodes of the mesh

• link-capacity: # links leaving the submesh

Randomized, locality preserving embedding!





Result for meshes

The static and dynamic strategies are

O (log(n))-competitive in nxn-meshes, w.h.p.

Finding an optimal static placement for several variables is

NP-hard already on 3x3-meshes.





Some Results

competitive ratio w.h.p.

d-dimensional meshes O(d log n)

Fat Trees O(log n)

Hypercubes O(log n)

SE Networks

De Bruijn Networks

Direct Butterflies

Indirect Butterflies

Arbitrary Networks

O(log n)

O(log n)

O(log n)

O(1)

polylog n (Räcke 2002)




Algorithms and ComplexityI wish you ….

… not too much

stress with exams

and some time for relaxing holydays.





Friedhelm Meyer auf der Heide

Heinz Nixdorf Institute

& Computer Science Department


Fürstenallee 11

33102 Paderborn, Germany

Tel.: +49 (0) 52 51/60 64 80

Fax: +49 (0) 52 51/62 64 82

E-Mail: [email protected]

http://www.upb.de/cs/ag-madh

Thank you for

your attention!

algorithm design - heinz nixdorf institut · 2012-05-31 · friedhelm meyer auf der heide 1 heinz...

Documents