Dynamic Optimization (Math-S-401)

Professor: Bram De RockTeaching assistant: Barnabe Walheer

This document contains some relevant information to study the course DynamicOptimization. Firstly it lists some background references that can be useful to refreshthe mathematical concepts that a bachelor in economics should master. Secondly,it provides a short description, with corresponding motivation, of the three parts.For each part it also contains some specific references that can help the interestedstudents to deepen their knowledge; this material is not part of the exam. Finally,for each topic it presents some exercises that allow the students to improve and testtheir knowledge; these exercises are similar to the ones for the written exam.

Course information

• Title: Dynamic Optimization

• Code: Math-S-401

• Credits: 5 ECTS

• Teaching language: English

• Course website: http://mathecosolvay.wordpress.com/

• Teaching load: 48 hours in the second semester

• Exam:

– Practical part (40%): written exam consisting of exercises

– Theoretical part (60%): oral exam consisting of 2 questions related to thetheory

– Both exams are open book; i.e. lecture notes and solutions of the providedexercises

1

Background

ContentThe students are assumed to be familiar with the mathematical subjects discussedin the first two years of the bachelor in economics at Universite libre de Brux-elles; i.e. Mathematique generale : analyse et algebre lineaire (MATH-S101) andMathematique: fonctions de plusieurs variables (MATH-S201). Below we list somebackground references that can be used to refresh this material.

References

• Chiang, A.C. and K. Wainwright, “Fundamental Methods of Mathematical Eco-nomics”, Economic series, McGraw-Hill.

• Gassner, M. “Mathematique generale”, first year course.

• Gassner, M. “Mathematique: fonctions de plusieurs variables”, second yearcourse.

• Luderer, B., V. Nollau and K. Vetters, “Mathematical Formulas for Economists”,Springer, New York (www.springerlink.com).

• Simon, C.P. and L. Blume, “Mathematiques pour economistes”, ouvertureseconomiques, De Boeck Universite.

• Simon, C.P. and L. Blume “Mathematics for Economists”, Norton & Company,New York.

• Sydsaeter, K., A. Strom and P. Berck, “Economists’ Mathematical Manual”,Springer, New York (www.springerlink.com).

2

Part I: static optimization

ContentIn this part we discuss the necessary and sufficient conditions for static optimizationproblems. That is, we characterize these points that optimize a (differentiable) objec-tive function while possibly facing some equality constraints, inequality constraintsand/or positivity constraints. This leads us to the so-called Lagrangian function andthe well-know Kuhn-Tucker conditions. We also study the envelope theorem andshow how we can use it to obtain some comparative statics.

Table of contents

I.1 Free optimization

I.2 Constrained optimization with equality constraints: the Lagrangian function

I.3 Constrained optimization with inequalities: the Kuhn-Tucker conditions

I.4 The envelope theorem: comparative statics

References

• Chiang, A.C. and K. Wainwright, Chapters 9-13 of “Fundamental Methods ofMathematical Economics”, Economic series, McGraw-Hill.

• Gassner, M., Chapters 1 and 3 of “Principles of static and dynamic optimiza-tion”, handwritten notes available upon request.

• Simon, C.P. and L. Blume, Chapters 16-19 of “Mathematics for Economists”,Norton & Company, New York.

Exercises

• Exercise 1

Classify the stationary points (maximum, minimum and saddle point) of:

(a) f(x, y, z) = x2 + x2y + y2z + z2 − 4z

(b)f(x1, x2, x3) = x21 + x22 + 3x23 − x1x2 + 2x1x3 + x2x3

(c)f(x1, x2, x3, x3) = 20x2 + 48x3 + 6x4 + 8x1x2 − 4x21 − 12x23 − x24 − 4x32

3

• Exercise 2

Suppose Q = F (K,L) denotes the output of a firm when the input of capitalis K with price r and that of labour is L with price w and that the firm has mto spend.

(a) Find the budget constraint.

(b) Solve the optimization problem.

(c) Suppose now that Q = 120KL, r = 2, w = 5; find the optimal solution.

(d) What is the solution if m = 100?

(e) What is the solution if m = 101?

(f) Make a link between the Lagrange multiplier λ and the increase of m from(d) to (e).

• Exercise 3

Each week an individual consumes quantities x and y of two goods and worksfor l hours. These quantities are chosen to maximize the utility function:

U(x, y, l) = α lnx+ β ln y + γ ln(L− l) with α + β + γ = 1

The price of the two goods are p and q. The wage per hour is w and theindividual has an unearned income m.

(a) Find the budget constraint.

(b) Solve the optimization problem.

(c) Discuss the two following cases: (i) m ≤ µwL and (ii) m > µwL whereµ = (α + β)/(1− α− β).

• Exercise 4

A statistical problem requires solving:

4

min a21x21 + a22x

22 + · · ·+ a2nx

2n subject to x1 + x2 + · · ·+ xn = 1

Here all constants ai are nonzero.

(a) Solve the problem, taking it for granted that the minimum value exists.

(b) What is the solution if one of the ai’s is zero?

• Exercise 5

Consider the following problem:

min f(x, y, z) = (y + z − 3)2 subject to

x2 + y + z = 2x+ y2 + 2z = 2

(a) Solve the problem.

(b) It is tempting to believe that the solution which does not solve the mini-mization problem must solve the maximization problem. Is it the case?

• Exercise 6

Suppose a firm earns revenue R(Q) = aQ − bQ2 and incurs cost C(Q) =aQ+ bQ2 where all the constants are positive. The firm maximizes profit π(Q)subject to the constraint Q ≥ 0.

Solve this problem and find conditions to bind at the optimum.

• Exercise 7

Consider the cost minimization of a firm:

minC = rK + wL subject to F (K,L) = Q

(a) Find: (i)∂C∗

∂r, (ii)∂C

∗

∂w, (iii)∂C

∗

∂Q.

(b) What can you conclude on the Lagrangian multiplier?

5

(c) Prove that ∂K∗

∂w= ∂L∗

∂r.

• Exercise 8

A firm uses positive quantities K and L units of two inputs to produce√KL

units of a product. The input factor costs are r and w per unit, respectively.The firm wants to minimize the costs of producing at least Q units.

(a) Formulate the nonlinear programming problem.

(b) Write down the Kuhn-Tucker conditions.

(c) Solve these conditions to determine K∗ and L∗ as functions of r, w and Q.

(d) Give the minimum cost function C∗(r, w,Q) = rK∗ + wL∗.

(e) Find (i)∂C∗

∂r, (ii)∂C

∗

∂w, (iii)∂C

∗

∂Q.

(f) Verify that ∂C∗

∂r= K∗ and ∂C∗

∂W= L∗.

(g) Give an economic interpretation of (f).

6

Part II: solving difference and differential equations

ContentIn this part we study economic dynamics, meaning that we want to describe howeconomic variables evolve over time. An important issue in this respect is the factthat we consider time to be discrete or continuous. We mainly focus on time beingcontinuous and show how we can solve several types of differential equations. Asa preliminary to solving these type of equations we first study integrals. We endthis part by studying difference equations and we discuss the similarity to solvingdifferential equations.

Table of contents

II.1 Integrals

II.2 Solving differential equations

III.3 Solving difference equations

References

• Chiang, A.C. and K. Wainwright, Chapters 14-19 of “Fundamental Methods ofMathematical Economics”, Economic series, McGraw-Hill.

• Gassner, M., Chapters 2 and 4 of “Principles of static and dynamic optimiza-tion”, handwritten notes available upon request.

• Hoy, M., J. Livernois, C. McKenna, R. Rees and T. Stengos, Chapters 17-24of “Mathematics for Economics”, MIT press, Cambridge, Massachusetts.

• Sydsaeter, K. and P. Hammond, Chapter 9 of “Essential mathematics for Eco-nomic Analysis ”, Prentice hall, Essex.

Exercises

1 Integrals

7

• Exercise 1

Find the following integrals

(a)∫

ln(x)dx

(b)∫x ln(x)dx

(c)∫

x3

(1+x2)3dx

(d)∫

(a− bt+ ct2)e−rtdt

(e)∫x2exdx

(f)∫

sin(x)x2dx

(e)∫

sin(x)e2xdx

• Exercise 2

Evaluate the following integrals:

(a)∫ 1

02xex

2dx

(b)∫ 1

0(x2 + a)2dx

(c)∫ 10

0dx

(x−1)

(d)∫ 2

0dx√4−x

• Exercise 3

(a)∫∞1

1√xdx

(b)∫ 0

−∞ exdx

(c)∫ a0

xdx√a2−x2

• Exercise 4

A theory of investment has used a function W defined for all T > 0 by:

8

W (T ) =K

T

∫ T

0

e(−θt)dt

with K and θ positive constants

Evaluate the integral, and then prove that W (T ) takes values in the interval(0, K) and is strictly decreasing (Hint: ex > 1 + x, ∀x).

• Exercise 5

The consumer surplus is given by: CS =∫ Q∗0

(f(Q)− P ∗)dQ.

The producer surplus is given by: PS =∫ Q∗0

(P ∗ − g(Q))dQ.

(a) What are f(Q) and g(Q)?

(b) Take f(Q) = 50 − 0.1Q and g(Q) = 0.2Q + 20. Compute CS and PS.Sketch the graphs of the two surplus.

(c) Redo the exercise with f(Q) = 6000/(Q+ 50) and g(Q) = Q+ 10.

• Exercise 6

Let K(t) denote the capital stock of an economy at time t and I(t) the netinvestment at time t. I(t) is given by the rate of increase K(t).

If I(t) = 3t2 + 2t+ 5,

(a) What is the total increase in the capital stock during the interval t = 0 tot = 5.

(b) If K(t0) = K0 and K(t1) = K1, find an expression for the total increase inthe capital stock from time t = t0 to t = t1.

• Exercise 7

A probability density function f is defined for all x by:

f(x) =λae−λx

(e−λx + a)2

(a) Show that F (x) is an indefinite integral of f(x).

9

(b) Determine limx→∞ F (x) and limx→−∞ F (x).

(c) Show that F (x) is strictly increasing.

(d) Show that∫ x−∞ f(t)dt = F (x).

(e) Show that F has an inflection point.

(f) Compute∫ +∞−∞ f(x)dx.

• Exercise 8

Find f(x) if f ′′(x) = x−2 + x3 + 2, f(1) = 0 and f ′(1) = 1/4.

• Exercise 9

Compute the area of the triangle defined by:

D = (x, y) | 0 ≤ x ≤ 2; 0 ≤ y ≤ x/2

Compute now, the area defined by:

D′ = (x, y) | 2y ≤ x ≤ 2; 0 ≤ y ≤ 1

What can you conclude? Explain.

• Exercise 10

Find the volume of the solid Ω over the rectangle 0 ≤ x ≤ 1, 0 ≤ y ≤ 3 andbounded by the xy−plane and the plane z = x+ y.

• Exercise 11

Let D be the region characterized by 0 ≤ y ≤ 1/2, and for any such y, 4y ≤x ≤ 1 + 2y. Find

∫∫D

x1+y2

dxdy.

• Exercise 12

The average value (AV ) of the function f on the region D is given by:

10

AV =

∫∫D

f(x, y)dydx∫∫D

dydx

Compute the average value of the money hold in the state of Kansas knowingthat this state can be modeled as a rectangle (0 ≤ x ≤ 500, 0 ≤ y ≤ 300) andthat the money hold can be modeled as M(x, y) = 10−2x− 10−7x2y.

2 Solving differential equations

• Exercise 1

Let X(t) denote the national product, K(t) the capital stock and L(t) thenumber of workers in a country at time t. Suppose for all t ≥ 0, X =AK(1−α)Lα, K = sX and L = L0e

λt. Here, all the constants are positive.

(a) Find the solution when K(0) = K0 > 0.

(b) Compute the limit for K/L as t→∞.

(c) Compute the limit for X/L as t→∞.

(d) Solve the problem if L = b(t+ a)p.

(e) Compute the limit for K/L as t→∞ with L defined in (d).

• Exercise 2

Assume that the price P of a good varies with time, and that P = λ[D(P ) −S(P )].

(a) Find the price if the demand is a− bP and the supply is c+ dP .

(b) What can you say on P if t tend to infinity.

(c) Give an economic interpretation of (b).

11

• Exercise 3

Solve the following economic growth model:

K = γ1bKα + γ2K

(a) First as a separable equation.

(b) Second by considering it as a Bernoulli equation.

• Exercise 4

A model includes the following system:

π(t) = απ(t)− σ(t)σ(t) = π(t)− 1

βσ(t)

Find the general solution when α + 1β> 2.

• Exercise 5

Consider the Harrod-Domar growth model:

Y = C + I

I = vdY

dt+ I

C = γY

(a) Interpret the three equations and give the restrictions on the parameters.

(b) By substituting, obtain a differential equation for Y .

(c) Solve this equation.

(d) Obtain and discuss a stability condition (that is a condition on the param-eters to ensure that Y tends to an equilibrium value).

12

• Exercise 6

Solve the following differential equations for the specific initial conditions:

(a) x+ 2x+ x = t2, x(0) = 0, x(0) = 1

(b) x+ 4x = 4t+ 1, x(π/2) = 0, x(π/2) = 0

• Exercise 7

Find x(t) if x(0) = 0, x(0) = 1, x(0) = 0 and...x − x− x+ x = 8te−t.

• Exercise 8

Show that the solutions of the following system are solutions of a second-orderdifferential equation:

x1(t) = ax1(t) + bx2(t) + f1(t)x2(t) = cx1(t) + dx2(t) + f2(t)

3 Solving difference equations

• Exercise 1

Find the solutions of the following difference equations with the given value ofx0:

(a) xt+1 = 2xt + 4, x0 = 1

(b) 3xt+1 = xt + 2, x0 = 2

(c) 2xt+1 + 3xt + 2 = 0, x0 = −1

(c) xt+1 − xt + 3 = 0, x0 = 3

• Exercise 2

Consider the difference equation xt =√xt−1 − 1 with x0 = 5.

(a) Compute x1, x2 and x3.

13

(b) Compute x4; what can you conclude? Explain.

• Exercise 3

Prove that the following solutions are the general solutions of the followingdifference equations:

(a) xt = A+Bt and xt+2 − 2xt+1 + xt = 0

(b) xt = A3t +B4t and xt+2 − 7xt+1 + 12xt = 0

(c) xt = A2t +Bt2t + 1 and xt+2 − 4xt+1 + 4xt = 1

• Exercise 4

Find the general solutions of the following difference equations:

(a) xt+2 − 6xt+1 + 8xt = 0

(b) xt+2 − 8xt+1 + 16xt = 0

(c) xt+2 + 2xt+1 + 3xt = 0

(b) 3xt+2 + 2xt = 4

(b) xt+2 + 2xt+1 + xt = 9.2t

(b) xt+2 − 3xt+1 + 2xt = 3.5t + sin 12πt

• Exercise 5

A model of location uses the difference equation:

Dn+2 − 4(ab+ 1)Dn+1 + 4a2b2Dn = 0, n = 0, 1, . . .

where a and b are constants, and Dn is the unknown function. Find the solutionof this equation.

• Exercise 6

A economic model is based on the following system:

14

Ct = cYt−1, Kt = σYt−1, Yt = Ct +Kt −Kt−1

(a) Give an economic interpretation of the equations

(b) Derive a second order difference equation for Yt

• Exercise 7

Find the solutions of the following difference equations:

(a) xt+3 − 3xt+1 + 2xt = 0

(b) xt+4 + 2xt+2 + xt = 8

• Exercise 7

Solve the following systems (t = 0, 1, . . . ):

(a) xt+1 = 12xt + 1

3yt, yt+1 = 1

2xt + 2

3yt

(b) xt+1 = −yt−zt+1, yt+1 = −xt−zt+t, zt+1 = −xt−yt+2t, x0 = y0 = 0, z0 = 1

(c) xt+1 = ayt + ckt, yt+1 = bxt + dkt(a > 0, b > 0, k2 6= ab)

15

Part III: dynamic optimization

ContentIn this part we study how to optimize an objective function that takes into accountthe economic dynamics of the variables at hand. The used techniques do of courseagain depend on time being discrete or continuous. In the continuous frameworkwe discuss the calculus of variation method, which will lead to the Euler-Lagrangeconditions and optimal control theory that departs from the maximum principle. Atthe end of this part we briefly touch upon dynamic programming and we introducethe so-called Euler and Bellman equation.

Table of contents

III.1 Calculus of variation

III.2 Optimal control theory

III.3 Dynamic programming

References

• Chiang, A.C., “Elements of Dynamic Optimization”, McGraw-Hill.

• Chiang, A.C. and K. Wainwright, Chapter 20 of “Fundamental Methods ofMathematical Economics”, Economic series, McGraw-Hill.

• Gassner, M., Chapter 5 of “Principles of static and dynamic optimization”,handwritten notes available upon request.

• Hoy, M., J. Livernois, C. McKenna, R. Rees and T. Stengos, Chapter 25 of“Mathematics for Economics”, MIT press, Cambridge, Massachusetts.

• Kamien, M.I. and N.L. Schwartz,“Dynamic Optimization - the Calculus ofVariations and Optimal Control in Economics and Management”, advancedtextbooks in economics, Ed. C.J. Bliss and M.D. Intriligator, North Holland.

• Sydsaeter, K, P. Hammond, A, Seierstad and A, Storm “Further mathematicsfor Economic Analysis ”, Prentice hall, Essex.

16

Exercises

1 Calculus of variation

• Exercise 1

Find the Euler equation associated with J(x) =∫ t1t0F (t, x, x) dt when:

(a) F (t, x, x) = x2 + x2 + 2xet

(b) F (t, x, x) = [(x− x)2 + x2]e−at

(c) F (t, x, x) = −ex−ax

(d) F (t, x, x) = 2tx+ 3xx+ tx2

• Exercise 2

A monopolist’s production of a commodity per unit of time is x(t). Supposeb(x) is the associate cost function. At time t, let D(p(t), p(t)) be the demandfor the commodity per unit of time when the price is p(t). If production atany time is adjusted to meet demand, the monopolists total profit in the timeinterval is given by:

π(t) =

∫ T

0

[pD(p, p)− b(D(p, p))] dt

Suppose that p(0) and p(T ) are given. The monopolists natural problem is tofind a price function p(t) that maximizes the total profit.

(a) Find the Euler equation

(b) Let b(x) = αx2+βx+γ and x = D(p, p) = Ap+Bp+C where α, β, γ, B,C >0 and A < 0. Find the general solution.

• Exercise 3

Solve the following problem if:

17

max

∫ 1

0

(10− x2 − 2xx− 5x2)e−t dt

(a) x(0) = 0 and x(1) = 1

(b) x(0) = 0 and x(1) is free

(c) x(0) = 0 and x(1) ≥ 2

• Exercise 4

Find the Euler equation and solve the following problem:

min

∫ T

0

(α1Y2 + α2G

2)e−t dt, Y = r1Y − r2G, Y (0) = Y0, and, Y (T )free

where all the constants are positive and given.

• Exercise 5

Consider the simple macroeconomic problem of trying to steer the state y(t)of the economy over the course of a planning period [0, T ] toward the desiredlevel y, independent of t, by means of the control u(t), where y(t) = u(t).Because using the control is costly, the objective is to minimize the integral∫ T0

[(y(t)− y) + c(u(t))2] dt with y(T ) = y, where c is a positive constant.

(a) Find the Euler equation (Hint: define x(t) = y(t) − y then x(0) = x0 andx(T ) = 0)

(b) Give the general solution

(c) Solve the problem if x(0) = x0 and x(T ) is unrestricted

(d) What happens to the terminal state x∗(T ) as the horizon T →∞ and alsoas c→∞

18

• Exercise 6

Consider the Ramsey model:

min

∫ T

0

U(f(K, t)− K − δK, t) dt,with, K(0) = K0, and, K(T ) = KT

(a) Find the Euler equation

(b) Deduce an expression for the corresponding relative rate of change of con-

sumption CC

2 Optimal control theory

• Exercise 1

Solve the following problem using the Hamiltonian:

(a) max∫ 2

0[etx(t)− u(t)2]dt, x(t) = −u(t), x(0) = 0, x(2) free

(b) max∫ 1

0[1− u(t)2]dt, x(t) = x(t) + u(t), x(0) = 1, x(1) free

(c) max∫ 1

0[x(t) + u(t)2]dt, x(t) = −u(t), x(0) = 0, x(1) free

(d) max∫ 10

0[1− 4x(t)− 2u(t)2]dt, x(t) = u(t), x(0) = 0, x(10) free

(e) max∫ T0

[x(t)− u(t)2]dt, x(t) = x(t) + u(t), x(0) = 0, x(T ) free

• Exercise 2

Solve the following problems using the Hamiltonian:

(a) max∫ 1

0x(t)dt, x(t) = x(t) + u(t);x(0) = 0, x(1) free, u ∈ [−1, 1]

(b) max∫ T0x(t)dt, x(t) = u(t);x(0) = 0, x(1) free, u ∈ [0, 1]

(c) max∫ T0−(u(t)2 + x(t)2)dt, x(t) = au(t);x(0) = 1, x(1) free, u ∈ [0, 1] when

a < 0

19

(d) max∫ T0−(u(t)2 + x(t)2)dt, x(t) = au(t);x(0) = 1, x(1) free, u ∈ [0, 1] when

a ≥ 0

(e) max∫ 2

0(−2u(t) + x(t)2)dt, x(t) = u(t);x(0) = 1, x(2) free, u ∈ [0, 1]

• Exercise 3

Solve the following problems using the Hamiltonian:

max∫ 1

0[1− tx(t)− u(t)2]dt, x(t) = u(t);x(0) = x0, x(T ) free and

(a) u ∈ R

(b) u ∈ [0, 1]

(c) u ∈ [−1, 1]

• Exercise 4

Solve the following problems using the Hamiltonian and compute the corre-sponding value of the objective function:

(a) max∫ 10

0x(t)dt, x(t) = u(t);x(0) = 0, x(10) = 2, u ∈ [0, 1]

(b) max∫ T0x(t)dt, x(t) = u(t);x(0) = x0, x(T ) = x1, u ∈ [0, 1] with x0 < x1 <

x1 + T

• Exercise 5

A firm has an order of B units of a commodity to be delivered at time T . Letx(t) be the stock at time t and ax(t) the cost per unit of time. The increasein x(t) which equals production per unit of time, is u(t) = x(t). Assume thatthe total cost of production per unit of time is equal to b(u(t)2). Here a and bare positive constants. The firms cost minimization problem is:

min

∫ T

0

[ax(t) + b(u(t)2)]dt, x(t) = u(t), x(0) = 0, x(T ) = B, u(t) ≥ 0

Find the only possible solution to the problem and explain why it really is asolution (Hint: distinguish the cases B ≥ aT 2/4b and B < aT 2/4b).

20

• Exercise 6

Consider the following problem:

max∫ T0

[U(x(t))− bx(t)− gz(t)]dt, z(t) = ax(t), z(0) = z0, z(T ) free

where U(x) is the utility enjoyed by the society consuming x, whereas b(x) isthe total cost and z(t) is the stock of pollution of time t. The control variableis x(t), whereas z(t) is the state variable. The constants a and g are positive.

(a) Write down the conditions of the maximum principle.

(b) Prove that if x∗(t) > 0 solves the problem, then U ′(x∗(t)) = b′(x∗(t)) +ag(T − t).

• Exercise 7

Solve the following problem by using both the calculus of variations and controltheory:

(a) max∫ 1

0(2xe−t − 2xx− x2)dt;x(0) = 0, x(1) = 1

(b) max∫ 2

0(3− x2 − 2x2)dt;x(0) = 1, x(2) ≥ 4

(c) max∫ 1

0(−2x− x2)dt;x(0) = 1, x(1) = 0

• Exercise 8

At time t = 0 an oil field is known to contains x barrels of oil. It is desired toextract all of the oil during a given time interval [0, T ]. If x(t) is the amount ofoil left et time t, then −x is the extraction rate. Assume that the world marketprice per barrel is aeαt with a and α positive. The extraction costs per unit oftime are assumed to be x(t)2eβt with β positive.

(a) Determine the profit per unit of time π(t).

(b) Solve the following problem max∫ T0π(t)e−rtdt with x(0) = x and x(T ) = 0

using the control theory.

(c) Show that at the optimum ∂π∂x

= ce−rt for some constant c.

(d) Find the same result using the Euler equation.

• Exercise 9

21

Consider the following problem in economic growth theory:

max

∫ T

0

(1− s(t))eρtf(k(t))e−δtdt

with k(t) = s(t)eρtf(k(t))− λk(t), k(0) = k0, k(T ) ≥ kT > k0, 0 ≤ s(t) ≤ 1

Here k(t) (the capital stock) is the state variable and s(t) (the savings rate)is the control variable. Suppose the production function f(k) > 0 wheneverk ≥ k0e

−λt, that f ′(k) > 0 and that all constants are positive.

(a) Suppose (k∗(t), s∗(t)) solves the problem. Write down the conditions in themaximum principle (call the shadow price p(t))

(b) What are the values of s∗(t)?

(c) Put ρ = 0, f(k) = ak, a > 0, δ = 0 and λ = 0. Suppose that T > 1/aand that k0e

aT > kT . Find the only possible solution to the problem (Hint:distinguish between two different cases)

(d) Compute the value function V (k0, kT , T ) in case (c) for the two cases

(e) Verify that ∂V∂k0

= p(0) for the two cases

(f) Verify that ∂V∂kT

= −p(T ) for the two cases

(g) Verify that ∂V∂T

= H∗(T ) for the two cases

• Exercise 10

Given the positive constant T , find the only possible solution to the problem:

max∫ T0

(2x2e−2t − uet)dt; x(t) = u(t)et, x(0) = 1, x(T ) free and u ∈ [0, 1]

Compute the value function V (T ) and verify that V ′(T ) = H∗(T )

• Exercise 11

Consider the problem max∫ 1

0uxdt, x = 0, x(0) = x0, x(1) free and u ∈ [0, 1]

(a) Prove that if x0 < 1, then the optimal control is u∗ = 0 and if x0 > 0, thenthe optimal control is u∗ = 1.

(b) Show that the value function V (x0) is not differentiable at x0 = 0.

22

• Exercise 12

Consider the problem max∫ T0

(x− 1/2u2)dt, x = u, x(0) = x0, x(T ) free

(a) Solve the problem (call the shadow price p(t))

(b) Compute the optimal value V (x0, T )

(c) Verify that ∂V∂xT

= p(0)

(d) Verify that ∂V∂T

= H∗(T )

• Exercise 13

Suppose a person inherits 1 million. He decides to retire and take as his incomethe interest payments he gets from investing this money. Let W (t) be his wealthat time t and suppose the rate of return on invested money is 5 %.

(a) Determine his income function

The income can either be consumed or reinvested, then W ′(t) = Y (t) − C(t).Suppose the consumer’s utility function is U(t) = logC(t) and his personalrate of time preference is 2 %. He expects to live another 40 years and wantsto leave a bequest of half million to his children.

(b) What are the state and control variables?

(c) Set up the dynamic optimization problem

(d) What are the necessary conditions for an optimal solution?

(e) Are these conditions sufficient?

• Exercise 14

A Calculus of variations problem is of the form:

max∫ baF (x(t), x(t), t)dt with x(a) = xa and x(b) = xb.

(a) Reformulate this as an optimal control problem letting x(t) = u(t).

(b) Form the Hamiltonian of this problem.

23

(c) Obtain the necessary conditions for an optimal solution.

(d) By differentiating ∂H∂u

with respect to t, show that these conditions implythe Euler’s equation.

• Exercise 15

Suppose a consumer’s income is derived from his/her wealth according to:Y = rW . This income is either consumed or invested: W ′ = I = Y − C. Theconsumer’s initial wealth is W (0) = W0 and at the end of his/her expectedlifetime (i.e. t = T ), we must have W (T ) ≥ 0. The consumer’s initial utility

function is U(C(t)) and the consumer seeks to maximize:∫ T0U(C(t))e−δtdt.

Suppose that the utility function is logarithmic.

(a) Formulate the problem facing the consumer.

(b) Show that the Hamiltonian is concave in W and C.

(c) Write down the necessary conditions for an optimal solution.

(d) Are these conditions sufficient?

(e) Show that the optimal solution for consumption is given by C∗(t) = 1/φ0e(t(r−δ))

where φ0 is a constant of integration.

(f) Show that the transversality condition requires that φ0 > 0 and this condi-tion requires that φ(T ) > 0 and W (T ) = 0.

(g) What condition do we need for optimal consumption to rise (fall) over time?

• Exercise 16

Let x(t) denote the stock of a limited resource which at time t is being extractedat the rate of u(t) so x = −u. This resource is then transformed into a con-sumption good via the production function: c(u) and the consumption good inturn yields utility for the community given by U(c). At reasonable extractionrates, the resource is calculated to last the community at least one hundredyears. The community seeks to find the extraction rate that maximizes utilityover that period.

(a) Formulate the problem facing the community.

24

(b) Let c(u) = auα with 0 < α < 1, U(c) = log c and x(0) = 1000; give thenecessary and sufficient conditions.

(c) Find the optimal extraction rate.

• Exercise 17

Solve the following problem using the current value formulation:

(a) max∫ 20

0(4K − u2)e−0.25tdt, K = −0.25K + u,K(0) = K0, K(20) free, u ≥ 0

(b) max∫ T0

(−x2 − 1/2u2)e−2tdt, x = x+ u, x(0) = 1, x(T ) free

(c) max∫ 5

0[10u(t)−(u(t)2+2))]e−0.1tdt, x(t) = −u(t);x(0) = 10, x(5) ≥ 0, u ≥ 0

(d) max∫ 1

0(−2x− x2)dt;x(0) = 1, x(1) = 0

• Exercise 18

Solve the following problem:

(a) max∫∞0

(lnu)e−0.2tdt; x = 0.1x− u, x(0) = 10, limt→∞ x(t) ≥ 0, u > 0

(b) max∫∞0−u2e−rtdt; x = ue( − at), x(0) = 0, limt→∞ x(t) ≥ K with a >

r/2 > 0 and K > 0

(c) max∫∞−1(x− u)e−tdt; x = ue( − t), x(−1) = 0, x(∞) free and u ∈ [0, 1]

(d) max∫∞0x(2− u)e−tdt; x = uxe( − t), x(0) = 1, x(∞) free and u ∈ [0, 1]

• Exercise 19

Consider the following problem:

max∫∞0

11−δ [rA(t)+w−u(t)]1−δe−ρtdt; A(t) = u(t), A(0) = A0 > 0, limt→∞A(t) ≥

−w/r with 0 < δ < 1 and 0 < r < ρ

(a) Solve the problem (use λ(t) to denote the shadow price

(b) Compute the optimal value V of the objective function

(c) How does V change when ρ increase?

(d) Show that ∂V∂A0

= λ(0)

25

• Exercise 20

Consider the variational problem with an integral constraint:

max

∫ t1

t0

F (t, x, x)dt, x(t0) = x0, x(t1) = x1,

∫ t1

t0

G(t, x, x)dt = K

Transform the problem to a control problem with one control variable (u = x)and two state variables x = x(t) and y(t) =

∫ tt0G(τ, x(τ), x(τ))dτ

• Exercise 21

Solve the following problem:

(a) max∫ T0

(x(t) + y(t)− 1/2u(t)2)dt,

x(t) = y(t), x(0) = 0, x(T ) freey(t) = u(t), y(0) = 0, y(T ) free

(b) max∫ 4

0(10− x1 + u)dt,

x1(t) = x2(t), x1(0) = 2, x1(4) freex2(t) = u(t), x2(0) = 0, x2(4) free

, u ∈ [−1, 1]

(c) max∫ T0

(12x1+

15x2−u1−u2)dt,

x1(t) = u1(t), x1(0) = 0, x1(T ) free , u1 ∈ [0, 1]x2(t) = u2(t), x2(0) = 0, x2(T ) free , u2 ∈ [0, 1]

(d) max∫ T0

(12x1 + 1

5x2 − u1 − u2 + 3x1(T ) + 2x2(T ))dt,

x1(t) = u1(t), x1(0) = 0, x1(T ) free , u1 ∈ [0, 1]x2(t) = u2(t), x2(0) = 0, x2(T ) free , u2 ∈ [0, 1]

with T > 5.

(e) max∫ T0

(x2 + c(1− u1 − u2))dt,x1(t) = au1(t), x1(0) = x01, x1(T ) free , u1 ≥ 0x2(t) = au2(t) + bx1(t), x2(0) = x02, x2(T ) free , u2 ≥ 0

with u1 + u2 ≤ 1 and T − c/a > T − 2/b > 0.

• Exercise 22

Consider the problem:

max∫ T0U(c(t))e−rtdt,

K(t) = f(K(t), u(t))− c(t), K(0) = K0, K(T ) = KT

x(t) = −u(t), x(0) = x0, x(T ) = 0

26

Where u(t) ≥ 0 and c(t) ≥ 0. Here K(t) denote the capital stock, x(t) is thestock of a natural resource, c(t) is the consumption, u(t) the rate of extraction,U the utility function and f the production functions.

(a) What is (are) the state variable(s)?

(b) What is (are) the control variable(s)?

(c) Write down the conditions of the maximum principle assuming u(t) andc(t) are strictly positive at the optimum

(d) Derive from these conditions that:

(i) CC

=r−f ′K(K,u)

ωwhere ω is the elasticity of the marginal utility

(ii) ddt

(f ′u(K, u)) = f ′K(K, u)f ′u(K, u)

• Exercise 23

Draw the phase diagram of the following problem (call λ the shadow price)

(a) max∫∞0

(x− u2)e−0.1tdt, x = −0.4x+ u, x(0) = 1, x(∞) free, u ∈ (0,∞), inthe λx−space

(a) max∫∞0

11−σC

1−σe−rtdt, K = aK−bK2−C,K(0) = K0 > 0 where a > r > 0and σ > 0 and with K(t) ≥ 0 for all t, in the λK−space

(a) max∫∞0

(ax − 1/2u2)e−rtdt, x = −bx + u, x(0) = x0, x(∞) free and allconstants are positive, in the λx−space

(a) max∫ T0

lnC(t)e−rtdt, K = AKα − C,K(0) = K0, K(T ) = KT where r > 0and A > 0 and α ∈ (0, 1) for all t, in the CK−space

• Exercise 24

Draw a phase diagram, find the equilibrium points and draw the direction ofmotion of the following system:

(a)

x = yy = −2x− y

(b)

K = aK − bK2 − CC = w(a− 2bK)C

where a, b, w > 0 and K,C ≥ 0

27

(c)

x = yy = x

(d)

x = x+ yy = x− y

(e)

x = x− 4yy = 2x− 5y

(f)

x = x(k − ay)y = y(−h+ bx)

where a, b, h, k > 0 and x, y > 0

(g)

K = AKa − Cy = C(aAKa−1 − r) where a,A, r > 0, a < 1 and K,C ≥ 0

(h)

x = −xy = −xy − y2

(i)

x = 1

2x+ y

y = y − 2

(k)

x = x(y − x

2− 2)

y = y(1− y2x

)

(a)

x = y2 − xy = 25

4− y2 − (x− 1

4)2

3 Dynamic programming

28