1

15.053 April 3, 2007

z The maximum flow problem

z See class notes on website.

Quotes of the day

You get the maxx for the minimum at TJ Maxx.

-- ad for a clothing store

This was the most unkindest cut of all

-- Shakespeare in Julius Caesar Act 3

2

Goal of this lecture

z Introduce the maximum flow problem

z Discuss applications of flows

z Present the Ford-Fulkerson max flow algorithm

z Present the max-flow min-cut result

3

4

The Maximum Flow Problem

z Directed Graph G = (N, A). – Source s – Sink t – Capacities uij on arc (i,j) – Maximize the flow out of s, subject to – Flow out of i = Flow into i, for i ≠ s or t.

A Network with Arc Capacities

s

1

t

2

4

1

2

3

1

1 2

3

4

The max flow problem is to send as much flow from the source node s to the sink node t as possible subject to the following constraints: 1. The flow into node i is the flow out of node i for nodes i ≠ s, t. 2. The flow xij on arc (i, j) satisfies 0 ≤ xij ≤ uij, where uij is the capacity of arc (i, j).

5

Feasible flows

z Flow in arc (i, j) is at most its capacity z Total flow into node 1 = total flow out of node z Total flow into node 2 = total flow out of node 2

A feasible flow

s

1

t

2

1

1

1

1

0

1 2

3

4

capacities

For most of the lecture, we will represent capacities using the color scheme given here and given line thicknesses. We do this so that our networks are not so “busy” with so many numbers. It is confusing to have to look at both capacities and flows at the same time.

6

Two Infeasible flows

s

1

t

2

1

2

1

2

0

1 2

3

4

capacities

The flow out of node 1 exceeds the flow in. s

1

t

2

1

0

1

1

1

The capacity of arc (s, 2) is violated.

This slide gives the two ways that a flow can be infeasible other than the flow being negative.

7

A feasible flow

s

1

t

2

2

0

1

1

1

1 2

3

4

capacities

A maximum flow

s

1

t

2

3

1

2

2

1

On representing capacities

z Usually capacities are just listed as numbers.

z We are using colors and thicknesses in this lecture to make the animations easier to follow.

z On any homework set, just use numbers. Don’t use colors or thicknesses.

8

An application of maximum flow

z In a college dormitory, students want to download a movie. They want to know how many minutes of the movie they can download each second.

s

1

t

2

This is a straightforward application of max flow in that we want to send as many packets per second from the download site to the dorm.

9

s

1

t

2

10

6

8

10

1

10

An application of maximum flow

Starbucks has decided to create a slurry pipeline to send coffee from Seattle to retailers around the country. How much coffee can it ship to MIT every day?

1

2

4

1 3

2 1s MIT

This is a silly application. It is motivated by an April fools joke by NPR radio about 10 years ago. They had a story that sounded fully serious about a pipeline for shipping hot coffee from Seattle to the East Coast. They went into great detail about the legal rights concerning putting pipes on people’s properties and about the expenses involved.

The maximum matching problem

M

I

CT

D

CY G

P

M

Can you match the stars of Grey’s Anatomy so as to get four pairs, with each man and women matched exactly once?

http://abc.go.com/primetim e/greysanatomy/bios

For fans of Grey’s Anatomy, please do not be too concerned about the possible matches, which are only loosely based on the show. I wanted to create a problem in which the maximum number of matched pairs would be 3. But if I used real relationships from Grey’s Anatomy, it would be very simple to come up with solutions in which all four women were matched to the men.

11

12

Transforming Matching into Max Flow

M

I

CT

CY

D

G

P

M

s t

All arcs have capacity 1.

Find the max flow.

To create a max flow from the matching problem, we create a source node s with arcs from the source node to the women. And we add a sink node t with arcs from the men to the sink node. And we direct the original arcs from the women to the men.

We put a capacity of 1 on each arc. Now a flow of k units from s to t corresponds to a matching of k pairs of persons. In this case, the maximum flow from s to t is 3, and it corresponds to matching three women to three men.

What is the maximum number of arc-disjoint s-t paths?

Two paths are arc disjoint if they have no arc in common.

Solve a max flow problem where each capacity is 1.

s t

1

2

3

4

5

6

7

8

9

10

11

12

There are 3 arc-disjoint s-t paths

There are some communication networks where communication is especially critical. One way of protecting communication from a source node s to a sink node t is to make sure that there are multiple disjoint paths on which communication can be sent. In this case, there are three arc-disjoint paths on which communication can be sent.

Two paths are called arc-disjoint if the two paths do not share an arc in common.

13

14

Representing the Max Flow as an LP

Representing Arc flows

s

1

t

2

xs1

xs2

x1t

x2t

x12

1 2

3

4

capacities

max v s.t. xs1 + xs2 = v

0 ≤ xs1 ≤ 4

xs1 = x12 + x1t

xs2 + x12 = x2t

s.t. -x1t – x2t = -v

0 ≤ xs2 ≤ 1 0 ≤ x12 ≤ 1 0 ≤ x1t ≤ 2 0 ≤ x2t ≤ 3

Here we represent the max flow problem as a linear programming problem.

It is common for the maximum flow to be represented by a variable v.

15

Flow out of i - Flow into i = 0 for i ≠ s or t. Let v = flow out of s and into t.

max v s.t xs1 + xs2 = v

max v

s.t. Σj xsj = v -xs1 + x12 + x1t = 0 -xs2 - x12 + x2t = 0

Σj xij – Σj xji = 0 for each i ≠ s or t

-x1t - x2t = -v 0 ≤ xij ≤ uij for all (i,j) 0 ≤ xij ≤ uij for all (i,j)

s.t. - Σi xit = -v

max v s.t. xs1 + xs2 = v

xs1 = x12 + x1t

xs2 + x12 = x2t

s.t. -x1t – x2t = -v

Here is the same formulation. We then generalize it to a max flow problem on an arbitrary graph.

A path with capacity 2

16

The Ford-Fulkerson Algorithm

s

1

t

2

4

1

2

3

1

Essential Subroutine: sending flow on a path from s to t. Such a path is called an augmenting path.

The capacity of the path is the smallest capacity of an arc on the path.

1 2

3

4

capacities

A path with capacity 1

What is the remaining augmenting path?

In the early 1950’s, Ford and Fulkerson worked for the Rand Corporation. They became interested in the max flow problem and other flow problems, and ultimately became the leading researchers in this area.

The Ford-Fulkerson max flow algorithm is known as an augmenting path algorithm. At each iteration, the algorithm maintains a feasible flow. The algorithm then successively gets better flows by sending flow along paths from s to t.

Here we introduce an augmenting path. We subtly change the definition of an augmenting path a few slides from now.

On the FF Max Flow Algorithm

s

1

t

2

2

0

1

1

1

1 2

3

4

capacities The network with flows

The Ford-Fulkerson Algorithm will create a series of larger and larger flows. Consider, e.g., the flow x’ above.

Look at (1, 2). x’12 = 1; for any flow, 0 ≤ x12 ≤ 1.

If we start with x’, then we can decrease the flow in (1, 2) by 1 and cannot increase the flow in (1, 2)

The FF algorithm maintains a feasible flow at each iteration. (This is analogous to the simplex algorithm maintaining a basic feasible solution at each iteration; however, the FF algorithm is not a special case of the simplex algorithm.)

For the algorithm to proceed, we keep track of how much the flow can change in each arc at subsequent iterations, starting from the feasible flow at this iteration.

17

Focusing on How Flows Can Change

s

1

t

2

2

0

1

1

1

1 2

3

4

capacities The network with flows

Look at (2, t)

t

2

1

If we start with a flow of 1 in (2, t), we can increase the flow by as much as 2, and we can decrease the flow by as much as 1.

To illustrate the idea of how flows can change, suppose we start with the feasible flow given in the diagram. Then let us focus on arc (2, t) to illustrate the possible changes.

You may ask whether the algorithm would actually reduce the flow in (2, t). The answer is “no”. But the purpose of this slide and the next few slides is to illustrate how flows can change and the residual network. So, please accept for now the possibility that flow in (2, t) could decrease.

18

19

The Original and Residual Networks

t

2

1

Original Network

t

2

The Residual Network We can send as many as 2 units from 2 to t and send as many as one unit from t to 2.

Decreasing flow in (2, t) by 1 is mathematically equivalent to sending one unit of flow from t to 2.

Currently the flow in arc (2, t) is 1 and the capacity is 2. This permits us to increase the flow in (2, t) by at most one additional unit.

We can also modify the current flow by decreasing the flow in (2, t) from 1 to 0. This corresponds to being permitted to send negative one unit of flow in (2, t). Mathematically, this is equivalent to being permitted to send as much as 1 unit of flow from node t to node 1.

There are alternative ways of representing the possibility of decreasing the flow in (2, t) by as much as one. Rather than permitting negative flows to be sent, we add the backward arc (t, 2) and give it a capacity of 1. You should probably first convince yourself that reducing the flow in (2, t) by 1 is equivalent to sending one unit of flow from node t to node 2.

20

Focusing on How Flows Can Change

s

1

t

2

2

0

1

1

1

1 2

3

4

capacities The network with flows

s

1

t

2

1 2

3

4

Residual capacities The Residual Network

Rather than work with the original capacities and with the current flows at a given iteration, we will work with the “residual network”. The residual capacities either reflect that the current flow can be increased (such as the arc (2, t) with a residual capacity of 2) or else the residual capacities reflect that the current flow can be decreased. For example, the flow in (2, t) can be reduced from 1 to 0, and this is represented by an arc (t, 2) with a residual capacity of 1.

21

The Original and Residual Networks

t

2

1

Original Network

t

2

The Residual Network The residual capacities are

rij = uij – xij

rji = xij

In general, suppose that capacity is uij and the “current flow” is xij. (The current flow is the flow at some stage of the algorithm.)

This is how residual capacities are defined in terms of the original capacities and the current flow at an iteration.

1

22

Changes in the residual Network

t

2

t

2

Original Network

The Residual Network The residual capacities are

rij = uij – xij

rji = xij

Suppose that we increase the flow in (2, t) from 1 to 2. What happens in the residual network.

2

Notice that if we send a unit of flow in (2, t) then the residual capacity of (2, t) will decrease by one and the residual capacity of (t, 2) will increase by 1.

Similarly, if we were to reduce the flow in (2, t) by 1, this corresponds to sending one unit of flow in (t, 2) in the residual network. In this case, the residual capacity of (t, 2) would decrease by one, and the residual capacity of (2, t) would increase by 1.

The Ford-Fulkerson Algorithm

Essential Subroutine: sending flow on a path from s to t in the residual network G(x). Such a path is called an augmenting path. The capacity of the path is the smallest residual capacity of an arc on the path.

1 2

3

4

Residual capacities

s

1

t

2

The Residual Network

What other path could have been selected?

Now that you are familiar with the concept of “residual network” we can talk about the Ford-Fulkerson augmenting path algorithm. It’s actually a very simple algorithm if described in terms of the residual network. We first describe augmenting paths in terms of the residual network.

23

The Ford Fulkerson Maximum Flow Algorithm

Begin x := 0; while there is some directed path from

s to t in G(x) do begin

let P be a path from s to t in G(x); send δ(P) units of flow along P; update x and G(x);

end

end {the flow x is now maximum}.

The FF algorithm iteratively sends flows along augmenting paths until there is no augmenting path remaining.

24

25

Augmenting Path Theorem

z A flow is a maximum s-t flow if and only if there is no augmenting path from s to t in the residual network.

Mental Break: More people who got it wrong.

Nobody now fears that a Japanese fleet could deal an unexpected blow at our Pacific possessions... Radio makes surprise impossible.

Josephus Daniels, 1922 --

While theoretically and technically television may be feasible, commercially and financially I consider it an impossibility, a development of which we need waste little time dreaming.

Lee De Forest, 1926 --

What can be more palpably absurd than the prospect held out of locomotives travelling twice the speed of stagecoaches?

Quarterly Review, 1825 --26

The OR group at CN Rail was asked by the management to find a way to decrease the number of train crashes. After exhaustive statistical analysis and computer simulation they found that with large statistical significance the car most involved in crashes was the last car of the train. Their recommendation to management: remove the last car from all the CN trains.

-- Source :Ira Hammerman, ORCS-L 3/19/95.

It can be taken for granted that before 1980 ships, aircraft, locomotives and even automobiles will be atomically fueled.

-- David Sarnoff, 1955

27

28

Ford-Fulkerson Max Flow

This is the original network, and the original residual network.

4

11

2

21

2

3

3

1

s

2

4

5

3

t

1 2

3

4

residual arc

We illustrate the FF augmenting path algorithm on this network. The animation is contained in the file entitled FF Max Flow.ppt.

29

Ford-Fulkerson Max Flow 1

1

2

2

2

1

2 s

2

4

5

3

t

Here is the optimal flow

The algorithm terminates after five iterations with this maximum flow. In fact, it could have had fewer augmentations if we had chosen different augmenting paths.

Comments on the number of augmentations in the Ford-Fulkerson Max Flow Algorithm z If a “good” augmenting path is found, the

algorithm can be efficient – fewest number of arcs –augmentation with largest capacity

z It can be terrible if one is permitted to choose “bad” augmenting paths

z But it is finite so long as the data is all integer valued. And the optimum flows are integral so long as the data is integral.

The FF augmenting path algorithm can be painfully slow. Even with a network with only 5 arcs, it is possible to take trillions of iterations if the augmenting paths are chosen stupidly. Fortunately, there are two simple rules for choosing the augmenting paths that lead to provably efficient implementations of the max flow algorithm.

30

s

2 5

31

Cuts in Networks

A cut in a network is a partition of the nodes into two parts, denoted S and T.

4

11

2

21

2

3

3

1

4

3

t

1 2

3

4

residual arc

An s-t cut in a network is a cut in which s ∈ S, and t ∈ T.

s

2 5

The duality theory for max flows is very famous. Important examples of it predate duality for linear programming. Here we introduce the concept of cuts. In particular, we focus on cuts that separate the source node s from the sink node t. We call such cuts: s-t cuts.

s

2 5

32

Cuts in Networks

The arcs of the cut (S, T) are the arcs originating in S and ending in T.

4

11

2

21

2

3

3

1

4

3

t

1 2

3

4

residual arc

The capacity of the cut (S, T) is the sum of the capacity of the arcs of the cut.

s

2 5

The capacity of (S, T) is 7.

The capacity of the cut (S, T) is the maximum amount of flow that can be sent from S to T, and thus is an upper bound on the amount of flow that can be sent from s to t.

9 5

s

2

4

5

3

33

Cuts in Networks

What is the capacity of (S, T) if S = {s, 2, 3}?

4

11

2

21

2

3

3

1

t

1 2

3

4

residual arc

s

2

3

What is the capacity of (S, T) if S = {s, 2, 3, 4, 5}?

4

5

As with many concepts, it helps to work out a few examples to make sure that you know the definition.

s

2

34

Cut Duality

The capacity of every s-t cut is at least the value of the maximum flow.

4

11

2

21

2

3

3

1

4

5

3

t

1 2

3

4

residual arc

All flow from s to t must at some point cross from S to T.

s

2S

So, the max flow is at most 9.

Since all flow from s to t much eventually be sent from S to T, the capacity of (S, T) is an upper bound on the max flow sent from s to t.

t

35

T

Max Flow = Min Cut

The maximum flow from s to t is always equal to the minimum capacity of an s-t cut.

1

2

2

4

1

1

2

3

3

1

s

2

4

5

3

1 2

3

4

residual arc

t

Min Cut: S = {s, 2, 3, 4, 5}, T = {t}. Capacity = 5.

The max flow is always equal to the min cut. In some sense, this is a special case of linear programming duality. But for the purposes of 15.053, it is sufficient to view it as analogous to LP duality.

I

CT

G

s

36

Max Flow = Min Cut

M

CY

D

P

M t

All arcs have capacity 1.

I

CT

G

s

A cut with capacity 3

Recall that the max flow from s to t is 3 in this example, and so the maximum size of a matching is 3. By the max-flow min-cut theorem, there is an s-t cut with capacity 3. The s-t cut with capacity 3 is illustrated on this diagram. If we were to eliminate the three arcs of the cut, then there would be no path from s to t at all. In general, eliminating all arcs of any s-t cut will eliminate all paths from s to t in the network.

s

1

37

A Network with Arc Capacities

t

2

4

3

Max Flow = Min Cut

1

2

1s

1

Min cut: S = {1, 2}. Capacity = 4.

Here is another illustration of max flow = min cut.

s

3

4 8

38

Max Flow = Min Cut

t

1

2

5

6

7

9

10

11

12

Min cut: S = {s, 3, 4, 8}. Capacity = 3.

s

4 8

3

Here is another illustration of max flow = min cut.

39

A simple and very bad example

s

1

2

t

M M

MM 1

This example shows that it is possible to have 2M augmentations in this graph with four nodes and five arcs if one chooses augmentations in a perverse manner, as illustrated on the next slides. if M = 1 trillion, this would result in 2 trillion augmentations.

At the same time, it is possible to find a max flow in 2 augmentations if the augmenting paths are s-1-t and s-2-t.

40

After 1 augmentation

s

1

2

t

M-1 M

M-1M 1

1

1

41

After two augmentations

s

1

2

t

M-1 M-1

M-1M-1 1

1

1

1

1

42

After 3 augmentations

s

1

2

t

M-2 M-1

M-2M-1 1

2

2

1

1

43

And so on

44

After 2M augmentations

s

1

2

t M

M

M

M

1

Initial Application of Flows

z Military operations in Europe – General F. S. Ross developed the model – brought the model to Ford and Fulkerson at Rand – Evaluate the capacity of the Eastern European rail

network to support a conventional war.

45

46

And now, it’s time for …..

47

Summary

z Ford-Fulkerson Algorithm

z Duality Theory: Max Flow = Min Cut