Dynamic Programming

1

Content

• What is Dynamic Programming?• Knapsack Problem• All-Pairs Shortest Path Problem

2

Dynamic Programming

What is Dynamic Programming?

3

What is Dynamic Programming?

• Dynamic Programming (DP) tends to break the original problem to sub-problems, i.e., in a smaller size

• The optimal solution in the bigger sub-problems is found through a retroactive formula which connects the optimal solutions of sub-problems.

• Used when the solution to a problem may be viewed as the result of a sequence of decisions.

4

Properties for Problems Solved by DP

• Simple Subproblems

– The original problem can be broken into smaller subproblems with the same structure

• Optimal Substructure of the problems

– The solution to the problem must be a composition of subproblem solutions (the principle of optimality)

• Subproblem Overlap

– Optimal subproblems to unrelated problems can contain subproblems in common

5

The Principle of Optimality

• The basic principle of dynamic programming• Developed by Richard Bellman• An optimal path has the property that whatever

the initial conditions and control variables (choices) over some initial period, the control (or decision variables) chosen over the remaining period must be optimal for the remaining problem, with the state resulting from the early decisions taken to be the initial condition.

6

Example: Shortest Path Problem

10

5

3

GoalGoalStartStart

7

25

28

40

Example: Shortest Path Problem

10

5

3

StartStart GoalGoal

8

25

28

40

Example: Shortest Path Problem

10

5

3

StartStart GoalGoal

9

Recall Greedy Method forShortest Paths on a Multi-stage Graph

• Problem– Find a shortest path from v0 to v3

Is the greedy solution optimal?

10

Recall Greedy Method forShortest Paths on a Multi-stage Graph

• Problem– Find a shortest path from v0 to v3

Is the greedy solution optimal?

The optimal path

The optimal path

11

Example Dynamic Programming

min 1,1 3

min 1,2 3min 0 3

min 1,3 3

min 1,4 3

( , )

( , )( , )

( , )

( )

3

5

,

1

7

d v v

d v vd v v

d v v

d v v

12

Dynamic Programming

Knapsack Problem

13

Knapsack Problems

• Given some items, pack the knapsack to get the maximum total value. Each item has some weight and some benefit. Total weight that we can carry is no more than some fixed capacity.

• Fractional knapsack problem

– Items are divisible: you can take any fraction of an item.

– Solved with a greedy algorithm.

• 0-1 knapsack problem

– Items are indivisible; you either take an item or not.

– Solved with dynamic programming.14

0-1 Knapsack Problem

• Given a knapsack with maximum capacity W, and a set S consisting of n items

• Each item i has some weight wi and benefit value bi (all wi and W are integer values)

• Problem: How to pack the knapsack to achieve maximum total value of packed items?

Why it is called a 0-1 Knapsack Problem?Why it is called a 0-1 Knapsack Problem?

15

Example: 0-1 Knapsack Problem

Which boxes should be chosen to maximize the amount of money while still keeping the overall weight under 15 kg ?

16

Example: 0-1 Knapsack Problem

1 {0,1}x

2 {0,1}x

3 {0,1}x

5 {0,1}x

4 {0,1}x

Objective Function

Unknowns or Variables

Constraints

's, 1, ,5ix i

, ,0,1 1, 5ix i

1 2 3 4 54 2 10 2 1

Maximize

x x x x x

1 2 3 4 512 1

Subject t

154 2

o

1x x x x x

17

Formulation: 0-1 Knapsack Problem

1

1

Maximize

Subject t

{0,1}

o

n

i ii

n

i ii

i

b x

w x W

x

18

0-1 Knapsack Problem: Brute-Force Approach

• Since there are n items, there are 2n

possible combinations of items.

• We go through all combinations and

find the one with maximum value and

with total weight less or equal to W

• Running time will be O(2n)

19

DP Define Subproblem

• Suppose that items are labeled 1,..., n.

• Define a subproblem, say, Pk as to finding an optimal solution for items in Sk = {1, 2,..., k}.

original problem is Pn.

• Is such a scheme workable? • Is the principle of optimality held?

20

A Counterexample

1. 2kgs, 3$

2. 3kgs, 4$

3. 4kgs, 5$

4. 5kgs, 8$

5. 9kgs, 10$20 kgs

P1

P2

P3

P4

P5

21

A Counterexample

Sub-problem

Optimum

Value

P1 1 3$

P2 1, 2 7$

P3 1, 2, 3 12$

P4 1, 2, 3, 4 20$

P5 1, 3, 4, 5 26$

1. 2kgs, 3$

2. 3kgs, 4$

3. 4kgs, 5$

4. 5kgs, 8$

5. 9kgs, 10$20 kgs

22

A Counterexample

Sub-problem

Optimum

Value

P1 1 3$

P2 1, 2 7$

P3 1, 2, 3 12$

P4 1, 2, 3, 4 20$

P5 1, 3, 4, 5 26$

1. 2kgs, 3$

2. 3kgs, 4$

3. 4kgs, 5$

4. 5kgs, 8$

5. 9kgs, 10$20 kgs

Solution for P4 is not part of the solution for P5 !!!

23

DP Define Subproblem

• Suppose that items are labeled 1,..., n.

• Define a subproblem, say, Pk as to finding an optimal solution for items in Sk = {1, 2,..., k}.

original problem is Pn.

• Is such a scheme workable? • Is the principle of optimality held?

24

DP Define Subproblem

• Suppose that items are labeled 1,..., n.

• Define a subproblem, say, Pk,w as to finding an optimal solution for items in Sk = {1, 2,..., k} and with total weight no more than w. original problem is Pn,W.

• Is such a scheme workable? • Is the principle of optimality held?

New version

25

DP Principle of Optimality

1,

,1, 1,

if

max , if k

k w k

k wk w k w w k k

B w wB

B B b w w

Denote the benefit for the optimal solution of Pk,w as Bk,w.

26

DP Principle of Optimality

1,

,1, 1,

if

max , if k

k w k

k wk w k w w k k

B w wB

B B b w w

Denote the benefit for the optimal solution of Pk,w as Bk,w.

In this case, it is impossible to include the

kth object.

In this case, it is impossible to include the

kth object.

Not include

the kth object

include

the kth objectThere are two

possible choices.

There are two possible choices.

27

Example

w k 0 1 2 3 4

0

1

2

3

4

5

1. 2kgs, 3$

2. 3kgs, 4$

3. 4kgs, 5$

4. 5kgs, 6$5 kgs

1,

,1, 1,

if

max , if k

k w k

k wk w k w w k k

B w wB

B B b w w

28

1,

,1, 1,

if

max , if k

k w k

k wk w k w w k k

B w wB

B B b w w

Example

w k 0 1 2 3 4

0

1

2

3

4

5

1. 2kgs, 3$

2. 3kgs, 4$

3. 4kgs, 5$

4. 5kgs, 6$5 kgs

0 0 0 0 0

0

0

0

0

0

Step 1. Setup table and initialize base conditions.

29

1,

,1, 1,

if

max , if k

k w k

k wk w k w w k k

B w wB

B B b w w

Example

w k 0 1 2 3 4

0

1

2

3

4

5

1. 2kgs, 3$

2. 3kgs, 4$

3. 4kgs, 5$

4. 5kgs, 6$5 kgs

0 0 0 0 0

0

0

0

0

0

0

3

3

3

3

Step 2. Fill all table entries progressively.

30

1,

,1, 1,

if

max , if k

k w k

k wk w k w w k k

B w wB

B B b w w

Example

w k 0 1 2 3 4

0

1

2

3

4

5

1. 2kgs, 3$

2. 3kgs, 4$

3. 4kgs, 5$

4. 5kgs, 6$5 kgs

0 0 0 0 0

0

0

0

0

0

0

3

3

3

3

0

3

4

4

7

Step 2. Fill all table entries progressively.

31

1,

,1, 1,

if

max , if k

k w k

k wk w k w w k k

B w wB

B B b w w

Example

w k 0 1 2 3 4

0

1

2

3

4

5

1. 2kgs, 3$

2. 3kgs, 4$

3. 4kgs, 5$

4. 5kgs, 6$5 kgs

0 0 0 0 0

0

0

0

0

0

0

3

3

3

3

0

3

4

4

7

0

3

4

5

7

Step 2. Fill all table entries progressively.

32

1,

,1, 1,

if

max , if k

k w k

k wk w k w w k k

B w wB

B B b w w

Example

w k 0 1 2 3 4

0

1

2

3

4

5

1. 2kgs, 3$

2. 3kgs, 4$

3. 4kgs, 5$

4. 5kgs, 6$5 kgs

0 0 0 0 0

0

0

0

0

0

0

3

3

3

3

0

3

4

4

7

0

3

4

5

7

0

3

4

5

7

Step 2. Fill all table entries progressively.

33

1,

,1, 1,

if

max , if k

k w k

k wk w k w w k k

B w wB

B B b w w

Example

w k 0 1 2 3 4

0

1

2

3

4

5

1. 2kgs, 3$

2. 3kgs, 4$

3. 4kgs, 5$

4. 5kgs, 6$5 kgs

0 0 0 0 0

0

0

0

0

0

0

3

3

3

3

0

3

4

4

7

0

3

4

5

7

0

3

4

5

7

Step 3. Trace back

34

Pseudo-Polynomial Time Algorithm

• The time complexity for 0-1 knapsack using DP is O(Wn).

• Not a polynomial-time algorithm if W is large.

• This is a pseudo-polynomial time algorithm.

35

Dynamic Programming

All-PairsShortest Path Problem

36

All-Pairs Shortest Path Problem

• Given weighted graph G(V,E), we

want to determine the cost dij of

the shortest path between each

pair of nodes in V.

37

Vk

Floyd's Algorithm

• Let be the minimum cost of a path from node i to node j, using only nodes in Vk={v1,…,vk}.

kijd

kk

Vk1

ii jj1kijd

1kikd

1kkjd

1 1 1min , 0

0

k k kij ik kjk

ij

ij

d d d kd

c k

The all-pairs shortest path problem is to find all paths with costs

nij ijd d i j

38

Floyd's Algorithm

1 1 1min , 0

0

k k kij ik kjk

ij

ij

d d d kd

c k

Input Parameter: DOutput Parameter: D, nextall_paths(D, next) n = D.NumberOfRows;// initialize next: if no intermediate// vertices are allowed next[i][j] = jfor i = 1 to n

for j = 1 to nnext[i][j] = j;

for k = 1 to n // compute D(k) for i = 1 to n for j = 1 to n if (D[i][k] + D[k][j] < D[i][j]) D[i][j] = D[i][k] + D[k][j];

next[i][j] = next[i][k];

Input Parameter: DOutput Parameter: D, nextall_paths(D, next) n = D.NumberOfRows;// initialize next: if no intermediate// vertices are allowed next[i][j] = jfor i = 1 to n

for j = 1 to nnext[i][j] = j;

for k = 1 to n // compute D(k) for i = 1 to n for j = 1 to n if (D[i][k] + D[k][j] < D[i][j]) D[i][j] = D[i][k] + D[k][j];

next[i][j] = next[i][k];

O(n3)O(n3)39

Floyd's Algorithm

Input Parameters: next, i, jOutput Parameters: Noneprint_path(next, i, j) { // if no intermediate vertices, just

// print i and j and return if (j == next[i][j]) { print(i + “ ” + j); return; } // output i and then the path from the vertex // after i (next[i][j]) to j print(i + “ ”); print_path(next,next[i][j], j);}

Input Parameters: next, i, jOutput Parameters: Noneprint_path(next, i, j) { // if no intermediate vertices, just

// print i and j and return if (j == next[i][j]) { print(i + “ ” + j); return; } // output i and then the path from the vertex // after i (next[i][j]) to j print(i + “ ”); print_path(next,next[i][j], j);}

40