06) # dynamic programming

8/3/2019 06) # Dynamic Programming

1/53

Dynamic Programming


2/53

Dynamic Programming

"Dynamic programming algorithm stores the results forsmall subproblems and looks them up, rather than

recomputing them, when it needs them later to solvelarger subproblems"

Steps in developing a dynamic programmingalgorithm

Characterize the structure of an optimal solution Recursively define the value of an optimal solution

Compute the value of an optimal solution in a bottom-up fashion

Construct an optimal solution from computedinformation


3/53

Matrix chain multiplication

Problem: We are given a sequence(chain) of n matrices to bemultiplied, we wish to compute theproduct

A1A2.AnSince matrix multiplication is associative

so all ways of parenthesization willyield the same answer.

The product of A1A2A3A4 can beparenthesized in five distinct ways:


4/53

(A1(A2(A3A4)))

(A1((A2A3)A4))

((A1A2)(A3A4))((A1(A2A3))A4)

(((A1A2)A3)A4)

Although answer by every one will be

same, but the cost of evaluating variesdrastically.


5/53

Example:

A1:10 100

A2 : 100 5

A3: 5 50

((A1A2)A3):total cost = 10.100

.5+10.5.50= 7500

(A1(A2A3)):total cast 100.5.50 +10.100.50= 75000


6/53

Matrix chain multiplication problem:

Given a chain of n matriceswhere for i = 1,2n; matrix Ai hasdimension pi-1x pi,

fully parenthesize the product A1

A2

Anin a way that minimizes the number of

scalar multiplications.

P(n) : no of alternative parenthesizations

P(n) = k=1n-1P(k)P(n-k)


7/53

Solution of is (4n/n3/2)

i.e exponential in n

And so brute force method of exhaustivesearch is a poor strategy.

Let us use Dynamic programmingapproach

Let us divide the problem as:for i


8/53

To evaluate Ai..j We must find an index k

Ai..j = Ai..k Ak+1..j for some k in between I and j.

Total cost to get Ai..j =cost of computing Ai..k + cost of Ak+1..j

+cost of combining

Optimal cost of Ai..j can be obtained by

getting each of Ai..k and Ai+1..j optimally.

Thus optimal solution of the original problem

can be obtained from the optimal solutions ofsubproblems.


9/53

Recursive solution

Let m[i ,j] = minimum no. of multiplicationsneeded to compute the matrix Ai..j.

So answer of the given problem

m[1,n].

Now m[i,j] can be defined recursively as:

m[i , i] = 0

Let k be the index for the optimal solution

m[i,j] = m[I ,k ] + m[k+1 , j] +pi-1pkpj

R i d fi i i f i i f


10/53

Recursive definition for minimum cost pfparenthesizing the product A1A2.Anbecomes:

0 if i=j

m[i ,j] = min{m[i ,k] + m[k+1 ,j] + pi-1pkpj


11/53

To keep track of successive decisions let us useS[i j] to denote the value of index k from where

sequence AiAj is to be partitioned.

Thusm[i,j] = m[i,k] + m[k+1, j] + pi-1pkpj if I


12/53

for finding m[i ,j] optimal results for Ai..kand Ak+1..j are used and

size of Ai..kIs k-i+1 which is less than j-i+1

Also size ofAk+1..jis j-k which is also less than j-i+1

Table m[i,j] is filled by bottom upapproach.


13/53


14/53


15/53


16/53


17/53

3

3

3

5

5

3

3

3

4

33

3

1

21

1

2

3

4

5

2

3

4

5

6 i

j


18/53

Optimal solution

Optimal way to multiplication of thesesix matrices

((A1(A2A3))((A4A5)A6))


19/53

Exercise

M1 10 x 20

M2 20 x 50

M3 50 x 1 M4 = 1 x 100

Find optimal way of multiplying usingDynamic programming


20/53

Solution

Min no of multiplication = 2200

( M1 x (M 2 x M3)) x M4


21/53

Example Let us begin with a simple capital budgeting

problem.

A corporation has $5 million to allocate to itsthree plants for possible extension.

Each plant has submitted a number ofproposals .

Each proposal gives the cost of expansion(c) annd total revenue expected( r ).


22/53

Optimal Binary search Tree

Let the set of identifiers be


23/53

Let the set of identifiers be

= {for , do , while. int, if}

Binary search treesAverage no of searches (if assumed equal probability of occurrence)

= 12/5 11/5

for

if

int

whiledo

for

int

whileif

do


24/53

General case with differentprobabilities

Let us take the set of keys {a1, a2.an}

Where a1


25/53

do

int

while

for

if

for

int

whileif

do


26/53

Optimal search tree : expected cost isminimum

class Ei = set of x s.t. ai < x< ai+1En = set of x s.t. x > an

E0 = set of x s.t. x < a0objective:

minimize (p(i) * level (ai) + q(i)(level(Ei) -

1))


27/53

while

do

if

if

whiledo

a b

Draw all binary search trees for set

{ while ,if , do} and find expected costs


28/53

do

if

while

while

if

do

if

while

do

c d

e


29/53

Ex: pi = 1/7 =qi p=(0.5, 0.1 , 0.05)

q= (0.15, 0.1, 0.05,0.05)

A 15/7 2.65

B 13/7 1.9

C 15/7 1.5D 15/7 2.05

E 15/7 1.6


30/53

To apply dynamic programming approachwe need to view the construction of such a

tree as the result of a sequence ofdecisions.

A possible approach would be to make adecision :as to which of the ai s should be

assigned to the root node of the tree.

If we choose ak as root then internal nodesa1, a2, ak-1 as well as external nodes

for the classes E0, E1,Ek-1 will lie in theleft subtree l of the root. Remaining will be


31/53

be in the right subtree r.

Define

cost(l) =1i


32/53

ak

l r

To consider costs from ak

some extra cost isto be added

Let w( i, j) = qi + l=i+1j (q(l) + p(l))

w(i ,j) is the extra cost for the tree

containing nodes(ai+1,,..aj)


33/53

Expected cost for the tree with ak as root

p(k) + cost (l) +cost( r) + w(0 , k-1)+ w( k ,n)

W(0, k-1) = q0 + l=-1k-1 (q(l) + p(l))

W(k , n) = qk+ l=k+1n (q(l) +p(l))


34/53

p(k) + w(0,k-1) + w(k,n) = pk + q0+k-1(p i+

qi)

+ qk+ k+1 (pi + qi)

= q0+ 1k (pi+qi) + qk+ k+1n (pi+qi)

so to minimize whole cost of the tree

cost(l) as well as cost ( r)

both should individually be minimized.

L


35/53

Let

C(i, j) cost of optimal binary search tree tij

containing ai+1 aj and Ei .EjCost (l) = C( 0 , k-1)

Cost (r ) = c(k , n)

Total cost of t0nPk + c(0, k-1) + c(k , n)+ w(0 , k-1) + w( k , n)

So

C(0,n) = min{c(0,k-1) + c(k,n) +pk+ w(0,k-1)

1kn +w(k, n)}

C(i j) min{c(i k 1) c(k j) p w(i k 1)


36/53

C(i ,j) = min{c(i, k-1)+c(k, j) + pk+ w(i, k-1) +i


37/53

C(1,4) =w(1,4)+min{c(1,1)+c(2,4);c(1,2)+c(3,2);

c(1,3)+c(4,4)}

Example: n 4


38/53

Example: n=4

(a1,a2,a3,a4) =( do, if, int, while)

LetP=(3,3,1,1)

Q=(2,3,1,1,1)

W(i,i)= qiC(i, i)= 0

And r( i,i)= 0

W(i,j) = pj + qj + w(i, j-1)


39/53

A

3,0,0 1,0,0

8,8,1

2,0,0 1,0,0 1,0,0

7,7,2 3,3,3 3,3,4

12,19,1

9,12,2

5,8,3

14,25,2

11,19,2

16,32,2

Wi,i+j

Ci,i+j

Ri,i+j

0 1 2 3 40

1

2

3

4


40/53

Optimal binary search tree

if

intdo

while

R(0,4)=2 i.e. if

R(2,4)=3

i.e. int

(a1,a2,a3,a4) =( do, if, int, while)

E i


41/53

a0 < a2


42/53

0 0

1 4

- -

- -

0 0

1 8

2 9

3 12

0 0

1 5

2 6

- -

1

2

3

4

Plant 3

C3 r3

Plant2

C2 r2

Plant1

C1 r1Proposal

Each plant will only be permitted to work on one


43/53

p y pof its proposals.

The goal is to maximize the firms revenueresulting from the allocation of the $ 5 million.

A straight forward way to solve this is to try all

possibilities and choose the best.In this case there are only 3 4 2 = 24 waysof allocating the money.

Many of these are infeasible.

For example proposals 3 , 4 and 1 for the threeplants costs $6 million

Let us break the problem into three


44/53

pstages:

Each stage represents the moneyallocated to a single plant.

So stage one represents the moneyallocated to plant 1

Stage 2 the money to plant 2

Stage 3 the money allocated to plant 3

Each stage is divided into states.A state

encompasses the information requiredto go from one stage to the next

{0,1,2,3,4,5}: the amount of money spenton plant 1.


45/53

on plant 1.

{0,1,2,3,4,5}: the amount of money spenton plants 1 and 2

{5}: the amount of money spent on plants1,2 and 3.

Associated with each state is a revenue.Note that to make a decision at stage 3, it

is only necessary to know how muchwas spent on plants 1 and 2, not how it

was spent.

Then the optimal therevenue

If the availablecapital is


46/53

1 0

2 53 6

4 6

3 6

3 6

proposal is for stage1 is

p

We are now ready to compute for stage 2.


47/53

y p g

In this case we want to find the best

solution for both plants 1 and 2.We want to calculate the best revenue for

a given amount for plants 1 and 2.

We simply go through all the plant 2proposals, allocate the given amount offund to it and use the above table to seehow plant 1 will spend the remainder.

Suppose we want to determine bestallocation for state value 4


48/53

allocation for state value 4.

1 Proposal 1 of 2 gives revenue 0, leaves

4 for plant 1 whichreturns 6: total:6

2 proposal 2 of 2 gives revenue 8 leaves 2for plant 1 whichreturns 6: total:14

3 proposal 3 of 2 gives revenue 9, leaves1 for plant 1 which

returns 5: total:64 proposal 4 of 2 gives revenue 12, leaves

0 for plant 1 whichreturns 0: total:12


49/53

0

58

13

14

17

1

12

2

2 or 3

4

0

12

3

4

5

And therevenue for

stages 1and 2 is

Then theoptimal

proposal is

If availablecapital x2

We can now go on to stage 3.


50/53

The only value we are interested in is x3 =

5.One again we go through all theproposals for this stage, determine theamount of money remaining and use

the above table to decide the value forprevious stages.

Proposal 1 of plant 3 gives revenue 0 and

leaves 5 to prev. stage giving revenue =17

Proposal 2 of plant 3 gives revenue 4leaves 4 prev stages give 14 so total

=

Here calculations are done recursively.


51/53

Stage 2 calculations are based on stage1,

Stage 3 only on stage 2.

In fact given at a state all future decisionsare made independent of how you got

to the state.This is principle of optimality.

Dynamic programming rests on this

assumption.


52/53

Principle of Optimality

AN OPTIMAL SEQUENCE OFDECISIONS HAS THE PROPERTYTHAT WHATEVER THE INITIAL

STATE AND DECISIONS ARE, THEREMAINING DECISIONS MUSTCONSTITUTE AN OPTIMALDECISION SEQUENCE WITH

REGARD TO HE STATE RESULTINGFROM THE FIRST DECISION.

Denote r(kj) the revenue for proposal kj atstage j and by c(kj) the corresponding


53/53

stage j and by c(kj) the correspondingcost.

Let fj(xj) be the revenue of state xj instage j

06) # dynamic programming

Documents