bea400 microeconomics lecture 11 module 5:...

BEA400 Microeconomics – Lecture 11

© University of Tasmania - CRICOS PROVIDER CODE 00586B 1

BEA400 Microeconomics

Lecture 11

Module 5: Choice Over Time with Uncertainty

Lecture 11: Stochastic Processes, Ito’s Lemma and Stochastic Optimal Control

• Stochastic Processes

The Pure Weiner Process or Brownian Motion Scaling the variance of the Process Weiner Process or Brownian Motion with Drift Markovian Process General Weiner Process or Brownian Motions

• Stochastic Integration • Ito’s Lemma

Demonstration of Ito’s Lemma Some Properties Ito’s Lemma Derivation of Ito’s Lemma

• Applications of Ito’s Lemma

A Stochastic Rate of Inflation Real Rate of Return with Stochastic Inflation Black-Scholes Option Pricing

• Stochastic Optimal Control Theory

Hamilton-Jacobi-Bellman (HJB) Equation Optimal Extraction of an Uncertain Non-Renewable Resource Consumption-Savings Decision with Risky Income Consumption-Savings Decision with optional Risky Asset

• Logarithmic Utility • HARA Utility

Reading: I haven’t found a text that I’m happy with for this part of the course, so lecture notes should suffice. You could try: Malliaris, A.G. and W.A Brock, Stochastic Methods in Economics and Finance, North-Holland, 1982. Chapters 2, 3 and 4.



Stochastic Processes 1. The Pure Weiner Process or Brownian Motion A Pure or Basic Weiner Process or Brownian Motion refers to a continuous time stochastic process where a random variable at time t, ( ) = tx t x which evolves over a small interval of time dt , according to a stochastic differential equation, =dx z dt

where z is a standard normal with mean zero and standard deviation of 1.

Since z is normally distributed it follows that dx will be normally distributed with an

expected value of zero and standard deviation of dt . Thus x is changing randomly

by dx since it depends on z dt

( )~ 0,dx N dt

That is dx is be normally distributed with ( ) 0=E dx and ( ) =Var dx dt (and

( ) =SD dx dt )

The reason that dx is scaled with dt is that any other choice for the magnitude of

dx would lead to a problem that is either meaningless or trivial when we consider

what happens at the limit when 0dt → .

Also if dx were not scaled in this way, the variance of the random walk would have a

limiting value of 0 or ∞ .

Now let us consider dx over two very small but consecutive time periods.

Consider two time periods 0t and 1t , such that 0 1= +T t t with corresponding values

of x, ( )0 0=x t x and ( )1 1=x t x and changes ( )0 0=dx t dx and ( )1 1=dx t dx



With the change over both periods as 0 1= +Tdx dx dx then the expected value of dx

over both periods is

( ) ( ) ( )0 1 0 0 0= + = + =TE dx E dx E dx

and variance

( ) ( ) ( ) ( )0 1 0 12 ,= + +TVar dx Var dx Var dx Cov dx dx

If we assume that the values of z are independent over time then dx in any period of

time is independent of dx in all other periods and the variance is

( ) ( ) ( )0 1

0 1

= +

= +

=

TVar dx Var dx Var dxdt dtdT

More generally over a long time period T,

( ) 0=TE dx and variance ( ) = = ∑TVar dx dT dt and ( ) = ∑SD dx dt

that is the variance of the Weiner process over the time period T, is equal to the sum

of the changes in time periods.

This allows us to consider the value x over discrete changes in time 1=dt

Consider a starting point 0s = then at time T, after T changes of 1=dt then the

variance of Tdx

( )0 0

1= =

= = = =∑ ∑T T

Ts s

Var dx dT dt T

At the starting point 0s = , so with 0=sx x the change in x over T, Tdx is

0= −T Tdx x x



and

0= +T Tx x dx

If the process starts at zero, 0 0= =sx x , then the value of x in an period T > s > 0, is

given by

0= +

=T T

T

x x dxdx

Thus from

( )~ 0,dx N dt

( ) 0=E dx and variance ( ) =Var dx dt ( ) =SD dx dt

with the assumptions of 0s = , 0 0= =sx x and 1=dt

we obtain that x in any future period T is

( )~ 0,Tx N T

( ) 0=TE x and variance ( ) =TVar x T ( ) =TSD x T

and for any time period in between s and T, t ( )0,tx N t

2. Scaling the variance of the Basic Weiner Process The Basic Weiner Process can be enhanced to scale the standard deviation of the

random variable tx by σ

σ=dx z dt .

In which case ( )~ 0,tx N tσ

Note that σ can be modelled to depend on time, t, and the value of tx such that

( ),σ σ= tt x



3. Weiner Process or Brownian Motion with Drift The Basic Weiner Process can also be enhanced by adding a drift or growth term, µ

to the stochastic differential equation.

µ σ= +dx dt z dt In which case ( )~ ,tx N t tµ σ

Note that as with σ , µ can be modelled to depend on time, t, and the value of tx

such that ( ),µ µ= tt x

4. Markovian Process A Markovian process is one where the probability values of future values of x

conditional on being at time t, only depend upon the current value of x and no other

information.

( )( ) ( ) ( )( ), , tP x t t P x t t x t x= =

While this might seem rather restrictive it can be modified to allow a fixed amount of

past information. The General Weiner Process described below is an example of a

Markovian process.

5. General Weiner Processes or Brownian Motions General Weiner Process or Brownian Motion refers to a continuous time stochastic

process where a random variable ( ) = tx t x evolves over time, t, according to some

stochastic differential equation:

( ) ( ), ,µ σ= +t tdx t x dt t x z dt

This also called an Ito Process.



Stochastic Integration Stochastic Integration was developed by Ito(1944) who generalised the stochastic

integral first introduced by Weiner (1923)

Consider for which a stochastic process ( ), ,dx t x z with deterministic component and

a random component which follows a Standardised Weiner process z .

( ) ( ), ,µ σ= +t tdx t x dt t x z dt

Stochastic Integration transforming the above into an integral equation we get,

( ) ( ) ( )= + +∫ ∫0 0

0 , ,t t

t s sx x s x ds s x z dsµ σ

We have encountered integrals of the form ( )0

,µ∫t

ss x ds before but how do we cope

with ( )0

,σ∫t

ss x z dt which does not exist? Ito’s Lemma!



Ito’s Lemma Suppose x follows a Brownian Motion with

( ) ( ), ,µ σ= +t tdx t x dt t x z dt and

( )22 2,σ= tdx t x z dt as ( )2 0dt = and 0× =dz dt and ( )2dz dt=

If the dynamics of ( )x t can be written by an Ito Process, then the dynamics of well-

behaved function of ( )x t that describe its distribution, ( )= ,t ty F t x will also be described by an Ito Process. Ito’s Lemma gives: 21

2x t xxdy F dx F dt F dx= + +

( )212t x xx xdy F F F dt F dzµ σ σ= + + +

and expectation

[ ] ( ) [ ]( )

212

212

/ /t x xx x

t x xx

E dy dt F F F F E dz dt

F F F

µ σ σ

µ σ

= + + +

= + + as [ ] 0E dz =

[ ] ( )

( )( ) ( )

( )

212

212

22 2 2 2 2 2 21 12 2

22 2 2 2 212

t x xx x

t x xx

t x xx x t x xx x

t x xx x

E dy E F F F dt F dz

F F F dt

E dy E F F F dt F F F F dzdt F dz

F F F dt F E dz

µ σ σ

µ σ

µ σ σ µ σ σ

µ σ σ

⎡ ⎤= + + +⎣ ⎦

= + +

⎡ ⎤⎡ ⎤ = + + + + + +⎣ ⎦ ⎢ ⎥⎣ ⎦

⎡ ⎤= + + + ⎣ ⎦

[ ] 2 2 2

2 2

x

x

Var dy F E dz

F

σ

σ

⎡ ⎤= ⎣ ⎦=

as 2 1E dz⎡ ⎤ =⎣ ⎦



Demonstration of Ito’s Lemma Suppose x follows a Brownian Motion with

0 0dx xdt xdzµ σ= + and 2 2 2 2 2 2

0 0dx x dz x dtσ σ= = as ( )2 0dt = , 0dz dt× = and ( )2dz dt= Consider ( )lny x t= Ito’s Lemma 21


212 2

1 1dy dx dxx x

⎛ ⎞= + −⎜ ⎟⎝ ⎠

Note that 2 2 2 2 2 2

0 0dx x dz x dtσ σ= = ,

( ) 2 2

0 0 02

20 0 0

1 12

12

dy xdt xdz x dtx x

dt dz

µ σ σ

µ σ σ

= + −

⎛ ⎞= − +⎜ ⎟⎝ ⎠

which can be written

( ) ( ) ( )210 0 02

0 0

0t t

y t y dt dzµ σ σ= + − +∫ ∫

which gives

( ) ( ) ( ) ( )210 0 020y t y t z tµ σ σ= + − +

Now substituting back ( )lny x t= by using ( )ye x t= and ( ) ( )00 yx e= gives

( ) ( ) ( ) ( )210 0 020 expx t x t z tµ σ σ⎡ ⎤= − +⎣ ⎦

Some Properties Ito’s Lemma

( )1 1 2 1 1 1 2 20 0 0

T T T

a a dz a dz a dzσ σ σ σ+ = +∫ ∫ ∫

( ) ( ) ( )2

2

0 0

T T

E t dz t E t dtσ σ⎡ ⎤

⎡ ⎤=⎢ ⎥ ⎣ ⎦⎣ ⎦∫ ∫ if ( ) 2

0

T

E s dsσ⎡ ⎤ < ∞⎣ ⎦∫



Derivation of Ito’s Lemma (Not examinable) A Taylor series expansion of ( ) ( )( ),y t F t x t= around 0y gives dy The expected value of dy can be computed (noting that [ ] 0E dz = ) as [ ] ( ) ( ) ( ) ( )21

0 0 0 0 02 higher order termst t tE y y F x E x x F x E x x⎡ ⎤′ ′′⎡ ⎤− = − + − +⎣ ⎦ ⎣ ⎦

since ( )0tE x x tµ⎡ ⎤− =⎣ ⎦ and ( ) [ ]⎡ ⎤− = − + = +⎣ ⎦

2 2 2 2 2 20 0t tE x x V x x t t tµ σ µ

[ ] ( ) ( )( )2 2 21

0 0 02 higher order termstE y y F x t F x t tµ σ µ′ ′′− = + + + with the variance of 0ty y− ignoring higher order terms and using ( )2 0dt = ,

( )2dz dt= and 0dz dt× =

so by setting t =1 allows the change in y to be given by

( ) ( ) ( )2

0 0 02dy F x F x dt F x dzσµ σ

⎛ ⎞′ ′′ ′= + +⎜ ⎟

⎝ ⎠

or if F also depends on t then

( )21

2t x xx xdy F F F dt F dzµ σ σ= + + +

( ) ( ) ( )212t x xx xy t F F F t F z tµ σ σ= + + +

and finally dividing by dt and taking the expectation (noting [ ] 0E dz = ) gives

[ ] ( ) [ ]( )

212

212

/ /t x xx x

t x xx

E dy dt F F F F E dz dt

F F F

µ σ σ

µ σ

= + + +

= + +



Applications of Ito’s Lemma A Stochastic Rate of Inflation

Suppose the rate of inflation is given by a basic Brownian Motion 0 0dP dt dzP

µ σ= +

We can easily find the mean and variance of such a series,

[ ] [ ]µ σ

µ

⎡ ⎤ = +⎢ ⎥⎣ ⎦⎡ ⎤ =⎢ ⎥⎣ ⎦

0 0

0

dPE E dt E dzP

dPE dtP

The average proportionate change in prices is expected to be µ0 multiplied by the time period over which it is considered. This can result can be re-expressed as the

[ ]

µ

µ

π µ

⎡ ⎤ =⎢ ⎥⎣ ⎦⎡ ⎤

=⎢ ⎥⎣ ⎦

=

&

&

0

0

0

/dP dtEP

dPEP

E

Thus the expected continuos rate of change in inflation is µ0

[ ] [ ]{ }

[ ] [ ]

µ µ σ σ µ

σ

⎡ ⎤ ⎧ ⎫⎡ ⎤ ⎛ ⎞ ⎡ ⎤= −⎢ ⎥ ⎨ ⎬⎜ ⎟⎢ ⎥ ⎢ ⎥⎣ ⎦ ⎝ ⎠ ⎣ ⎦⎩ ⎭⎢ ⎥⎣ ⎦⎡ ⎤ ⎡ ⎤ ⎡ ⎤= + + −⎢ ⎥ ⎣ ⎦ ⎣ ⎦⎣ ⎦⎡ ⎤ = = = =⎢ ⎥⎣ ⎦

22

22 2 2 20 0 0 0 0

2 20

as E 0, E 0 and

dP dP dPVar E EP P P

dPVar E dt E dt dz E dz E dtP

dPVar dt dz dt dz dt dzP

The variance in the proportionate change in prices is σ 20 multiplied by the time period

over which it is considered.

[ ]

[ ]

σ

σπ

σπ

⎡ ⎤ =⎢ ⎥⎣ ⎦

=

= =

&

&

20

20

0

/

as

dP PVardt dt

Vardt

SD dz dtdz



While the Brownian Motion can readily provide the mean and variance for the

proportionate change in prices if we want to know something about the price level in

any particular period we will need to solve the differential equation

µ σ= +0 0dP Pdt Pdz . The presence of P, which depends on time, in the drift term is

no problem as we have learnt to solve such differential equations previously, but its

presence with the dz term requires the use Ito’s Lemma in order to solve for P(t).

Let ( ) ( )lny t P t= then by Ito’s Lemma 21


µ σ σ

µ σ σ

⎛ ⎞= + −⎜ ⎟⎝ ⎠

= + −

⎛ ⎞= − +⎜ ⎟⎝ ⎠

212 2

2 20 2

20 2

1 1

1 1 12

12

dy dP dPP P

Pdt Pdz P dtP P P

dt dz

All terms are independent of time and so can easily be integrated to provide y(t)

( ) 20 2

12

y t t zµ σ σ⎛ ⎞= − +⎜ ⎟⎝ ⎠

( ) ( ) ( ) ( ) ( )21

0 0 020 expy tP t e P t z tµ σ σ⎡ ⎤= = − +⎣ ⎦

Real Rate of Return with Stochastic Inflation If we also have an asset returning a nominal return of Q period that is growing at r then

dP Pdt Pdzµ σ= + and dQ rQdt=

Then defining QqP

= the real return then Ito’s lemma gives

2 2 2

2 22 2

1 22

q q q q q qdq dt dP dQ dP dQ dQdPt P Q P Q Q P

⎛ ⎞∂ ∂ ∂ ∂ ∂ ∂= + + + + +⎜ ⎟∂ ∂ ∂ ∂ ∂ ∂ ∂⎝ ⎠

Since ( )2 0dt = , ( )2dz dt= and , dQdP and 2dQ equal zero. Which allows



( )

( )

⎛ ⎞= − + + ⎜ ⎟⎝ ⎠

⎛ ⎞= − + + + ⎜ ⎟⎝ ⎠

= − + −

2 22 3

22

2

1 1 22

1

Q Qdq dP dQ P dtP P PQ QPdt Pdz rQdt dtP P P

dq r dt dzq

σ

µ σ σ

µ σ σ

since =QqP

Black-Scholes Option Pricing Suppose there are three assets. The first with mean rate of return

1pµ and SD 1pσ ,

the second with a mean rate of return 2pµ and SD

2pσ and the third a risk free asset that earns a rate of return r per period. The nominal value of the portfolio is

( ) ( ) ( ) ( ) ( ) ( )1 1 2 2 3v t n t p t n t p t np t= + +

( ) ( ) ( )1 11 1 1p pdp t p t dt p t dzµ σ= + and ( ) ( ) ( )

2 22 2 2p pdp t p t dt p t dzµ σ= +

( ) ( )( ) ( )

1 1 2 2

1 1 2 2

1 1 2 2 3

1 1 2 2 3

31 1 2 2

p p p p

p p p p

dv n dp n dp dnp

n p dt dz n p dt dz r np dt

npn p n pdv dt dz dt dz r dtv v v v

µ σ µ σ

µ σ µ σ

= + +

= + + + +

= + + + +

Defining 1w , 2w and 3w as the shares the three assets from of the portfolio’s value

so that 1 11

n pwv

= , 2 22

n pwv

= and 33 1 21npw w w

v= = − − so that the above equation

can be written

( ) ( ) ( )= + + + + −1 1 2 21 2 1 21-w p p p p

dv w dt dz w dt dz w r dtv

µ σ µ σ

Defining 1w , 2w for a riskless portfolio

1 21 2var 0p pdv w dz w dzv

σ σ⎡ ⎤ = + =⎢ ⎥⎣ ⎦

then 1 21 2 p pw dz w dzσ σ= − 2

1

1

2

p

p

ww

σσ

⇒ = −

( )1 21 2 1 2 1-wp p

dv w dt w dt r w dtv

µ µ= + + −



since dv rdtv

= for the riskless portfolio

( ) ( )( )1 21 2 0p pw r w r dtµ µ− + − = ( )( )

2

1

1

2

p

p

rww r

µ

µ

−⇒ = −

−

Combining the two

( )( )

−= − = −

−22

1 1

1

2

pp

p p

rww r

µσσ µ

and so

( ) ( )− −

=1 2

1 2

p p

p p

r rµ µ

σ σ

Thus the rate of return per unit of risk must be equal for the two risk assets. If asset 2 is stock option whose price is determined by ( )2 1,p F p t= and

1 11 p pdp dt dzµ σ= + Using Ito’s Lemma and that the rate of return per unit of risk must be equalised

( )= + +

= + + +

= + − +

1 1 1

1 1 1 1 1 1 1

1 1 1 1

212 1 12

2 212 12

2 212 1 12

t p p p

t p p p p p p p

t p p p p

dp F dt F dp F dp

dp F F F p dt F

dp F rp F rF F p

µ σ σ

σ

If the option can only be exercised at terminal time T with exercise price Tp the boundary condition ( )0, , -F s s s T t= = and ( ) [ ]1 1, max 0, TF p T p p= − Then Black and Scholes (1973) and Merton (1973) demonstrated that

( ) ( ) ( )

( )

−

−

−∞

= +

⎡ ⎤⎛ ⎞= + +⎢ ⎥⎜ ⎟⎜ ⎟⎢ ⎥⎝ ⎠⎣ ⎦

= −

= ∫

1

1

1

1

2

12

22 1 1

21

/ 2

, , , 1 e 2

11 ln2

2 1

12

rsp

p

T p

p

ys

p s p r p d d

pd r sp s

d d s

y e ds

σ φ φ

σ

σ

σ

φπ

Note that ( )yφ is the cumulative normal distribution



Stochastic Optimal Control Theory Previously our equation of motion was written,

( ), ,y f u y t•

=

Now consider a stochastic differential equation where ( ), ,y f u y t•

= is the

deterministic (non-random) component and ( ), ,t y u dzσ is the stochastic component

with ( ), ,t y uσ being a deterministic function and dz being a Brownian motion

increment.

The problem then becomes

( )( ) ( ) ( )( )u

0

, Max E , ,T

V y t T F t y t u t dt⎡ ⎤

= ⎢ ⎥⎣ ⎦∫

subject to

( ) ( ), , , ,dy f t y u dt t y u dzσ= +

The optimal value of the maximised problem can be written as

( ) ( ) ( )

( )( ) ( )

0

0 0

0 0 0 0u

0 0u

, Max , , , ,

Max , , ,

t t T

t t t

V y t E F t y u dt F t y u dt

E F t y u t t V y y t t

+∆

+∆

⎡ ⎤= +⎢ ⎥

⎢ ⎥⎣ ⎦⎡ ⎤= ∆ + + ∆ + ∆⎣ ⎦

∫ ∫

( ) ( ) ( ){ }u

, Max , , V y+ , tV y t E F t y u t y t≅ ∆ + ∆ + ∆

Note that following our previous section on Ito’s Lemma the above stochastic

differential equation can be rewritten as,

( ) ( ) ( )

212

212, , , , , ,

t y yy

t y yy y

dV V V dy V dy

V V f t y u dt V t y u dt V t y u dzσ σ

= + +

= + + +

Assuming that ( ),V y t is twice differentiable we expand the function on the right

around ( ),y t by Talyor Series expansion:

( ) ( ) ( ) ( )2 21 12 2V y+ , t V y, . .y yy t tt yty t t V y V y V t V t V y t h o t∆ + ∆ = + ∆ + ∆ + ∆ + ∆ + ∆ ∆ +



Inserting ( ) ( ), , , ,y f t y u t t y u zσ∆ = ∆ + ∆ , using ( )2 0t∆ = , ( )2z t∆ = ∆ and 0z t∆ ∆ = and then simplifying gives.

( ) ( ) ( )( ) ( ) ( ) ( ){ }212u

, Max , , V y, , , , ,t y yy yV y t E F t y t u t t t V V f t y u V t y u t V zσ σ⎡ ⎤≅ ∆ + + + + ∆ + ∆⎣ ⎦

Now tale the expectation of the above, the only stochastic term is z∆ and its

expectation is ( ) 0E z∆ = . Then subtract ( ),V y t from both sides and divide through

by t∆ and let 0t∆ → .

( ) ( )( ) ( ) ( ){ }212u

0 Max , , , , , ,t y yyF t y t u t V V f t y u V t y uσ= + + +

( ) ( )( ) ( ) ( )212u

Max , , , , , ,t y yyV F t y t u t V f t y u V t y uσ⎡ ⎤− = + +⎣ ⎦

Hamilton-Jacobi-Bellman (HJB) Equation This is the Hamilton-Jacobi-Bellman (HJB) equation of stochastic control theory

( ) ( )( ) ( ) ( )212u

Max , , , , , ,t y yyV F t y t u t V f t y u V t y uσ⎡ ⎤− = + +⎣ ⎦

Note that the co-state variable λ is yV so that ( )( ) ( ), ,

y

V y t TV t

y tλ

∂= =

∂, differentiating

with respect to the state variable gives ( )( )( )

( )( )

2

2

, ,yy y

V y t T tV

y ty t

λλ

∂ ∂= = =

∂∂ so that the HJB

equation can be written ( ) ( )( ) ( ) ( )212u

Max , , , , , ,t yV F t y t u t f t y u t y uλ λ σ⎡ ⎤− = + +⎣ ⎦ .

If the transformed Hamiltonian function ( ), , , yH H u y λ λ=% % is

( ) ( )( ) ( ) ( )212, , , , , ,yH F t y t u t f t y u t y uλ λ σ= + +%

Then u

MaxtV H− = %

Assuming 0Hu

∂=

∂

% can be solved for the optimal choice ( )* * , , yu u y λ λ= then

inserting into the HJB

( )* , ,t yV H y λ λ− = % .



since ( )* * , , yu u y λ λ=

( ) ( ), , , , , ,y ydy f t y dt t y dzλ λ σ λ λ= + Then since ( )* , , , yH f t yλ λ λ=%

( )λ σ= +% *dy H dt dz

It can also be shown by using the definition of yVλ = , the stochastic equation of motion for y the state variable and Ito’s lemma that

( )λ σ λ= − +% *y yd H dt dz then optimal HJB conditions are:

0Hu

∂=

∂

% Equation for optimal choice u.

( )λ σ λ= − +% *y yd H dt dz Equation of motion for the co-state variable.

( )λ σ= +% *dy H dt dz Equation of motion for the state variable y.

( )( ), 0T y Tλ = Endpoint restriction.



Consumption-Savings Decision with Risky Income

Suppose that each period t the consumer receives an income ( )y t per period t it

grows at constant rate of Ydy µ= but with random component Y ydzσ where dz is a

standardised Weiner process.

y ydy ydt ydzµ σ= +

We could include y as a state variable, with a corresponding co-state variable and

apply the rules of Stochastic Optimal Control. However it will be much simpler for us

to use Ito’s Lemma on dy first (see stochastic rate of inflation example) and replacing

y with:

( ) ( ) ( ) ( )2120 exp y y yy t y t z tµ σ σ⎡ ⎤= − +⎣ ⎦

so that the wealth evolution now explicitly includes the variance of income.

( ) ( ) ( )( ) ( ) ( ) ( ) ( )

( ) ( ) ( ) ( )•

= + −

⎡ ⎤= + − + −⎣ ⎦⎡ ⎤ ⎡ ⎤= + − −⎢ ⎥ ⎣ ⎦⎣ ⎦

212

212

0 exp

0 exp

y y y

y y

dw r w t dt y t dt c t dt

r w t dt y t z t dt c t dt

E w r w t y t c t

µ σ σ

µ σ

since ( )⎡ ⎤ =⎣ ⎦ 0E z t

with terminal conditions ( ) ( )00 , w 0w w T= = .

Proceeding in this manner we can simply use the non-stochastic version of the

Hamiltonian, which for this problem is

( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( )( )δλ λ µ σ σ− ⎡ ⎤⎡ ⎤ = + + − + −⎣ ⎦ ⎣ ⎦21

2, , , ln 0 expty y yH t w t c t e c t t rw t y t z t c t

with Hamiltonian Conditions

H1: ( ) ( ) ( )−∂

= − =∂

1 0tH e tc t c t

δ λ ⇒ ( ) ( )−=

1 tc t et

δ

λ

H2: ( ) ( ) ( ) dw tH w r w t c t

dtλ∂

= = = −∂

& ⇒

( ) ( ) ( )21

2

0 0y y r trt rtw t e w y e dt e c t dt

µ σ⎡ ⎤− −− −⎣ ⎦= + −∫ ∫



H3: ( )( ) ( )d t H r t

dt w tλ

λ λ∂= = − = −

∂& ⇒ ( ) ( )0 rtt eλ λ −=

( )( ) ( )

( )

2120

0 212

0

1

1

y y r T rT

y y

T

yw e w T er

ce

µ σ

δ

δµ σ

⎡ ⎤− − −⎣ ⎦

−

⎛ ⎞⎛ ⎞⎜ ⎟+ − −⎜ ⎟⎜ ⎟− − ⎝ ⎠⎝ ⎠=−

to go with ( ) ( )−= 0r tc t c e δ and

( ) ( ) ( )( )21

20 00 21

2

1 1 y y r trt rt t rt

y y

c yw t w e e e e er

µ σδ

δ µ σ

⎡ ⎤− −− ⎣ ⎦⎛ ⎞= − − + −⎜ ⎟− − ⎝ ⎠

to give the solution

for the control variable, consumption and state variable wealth.

We can see that income growth adds to initial consumption in that it effectively

reduces the rate of discounting on future income by its growth.

While income uncertainty reduces initial consumption in that it effectively increases

the rate of discounting on present value of future income due to its uncertainty.

If we had proceeded without solving for dy first, we would have had to use the

stochastic version of the Hamiltonian, which in this problem would be

( ) ( ) ( ) ( ) ( ) ( )( ) 2 212, , , , lnt

w y w yH t w t c t e c t t rw t c t yδλ λ λ µ λ σ−⎡ ⎤ = + + − +⎣ ⎦

and its first order conditions.



Consumption-Savings Decision with Optional Risky Asset Logarithmic Utility

Ignoring income for the moment, assume that there is a risk asset with rate of return

ar and SD of aσ then evolution of wealth becomes.

( ) ( )( ) ( )( ) ( ) ( )

1 a a

a a

dw w t a r ar c t dt aw dz

rw t a r r w c t dt aw dz

σ

σ

⎡ ⎤= − + − +⎣ ⎦⎡ ⎤= + − − +⎣ ⎦

The Maximum Value Function is

( )( )( )

( )0

, lnT t

c tV y t T Max e c t dtδ−= ∫

so that ( ) ( )( ) ( ) ( )212u

Max , , , , , ,t y yyV F t y t u t V f t y u V t y uσ⎡ ⎤− = + +⎣ ⎦ the stochastic optimal

control problem is,

( ) ( )( ) ( )( ) ( ) ( ) ( ){ }− ⎡ ⎤− = + + − − +⎣ ⎦

212,

Max lntt w a a wwc t a t

V e c t V r a r r w t c t aw Vδ σ

Or replacing wV λ= and defining the stochastic Hamiltonian as

( ) ( ) ( ) ( ) ( )( )( ) ( ) ( ) ( ) ( )( )212, , , , , lnt

w a w aH t w t c t a t e c t r a t r r w t c t a t w tδλ λ λ λ σ− ⎡ ⎤⎡ ⎤ = + + − − +⎣ ⎦ ⎣ ⎦%

( ) ( )[ ]{ }= %

,Max , , , , , wc t a t

V H t w c aδ λ λ

Examine the Hamiltonian Conditions (removing time (t) from the variables for clarity).

H1A: 2 2 0a w aH r w rw awa

λ λ λ σ∂= − + =

∂

% ⇒

( )2* a

w a

r ra

wλλ σ

−= −

H1: 1 0tH ec c

δ λ−∂= − =

∂

% ⇒ 1 tc e δ

λ−=

H1 implies 1 0tH ec c

δ λ−∂= − =

∂

%



////

c w

c w

c ww

cww

c ww

c w

cc ww

c w

F V

F Vw w

c F Vw c

F wVc c

F Vc w wwc F V c c

F V w wc wF V c c

λ= =

∂ ∂=

∂ ∂∂ ∂

=∂ ∂

∂ ∂=

∂ ∂∂ ∂

− = −∂ ∂

∂− = −

∂

where ww w

w

Vw w

Vλλ

− = − is the Arrow-Pratt Measure Relative Risk Aversion RRA of

wealth

//

w wc c

∂∂

the elasticity of wealth for consumption

cc c

c c

F F ccF c F

∂− = −

∂ the intertemporal elasticity of substitution

Thus the intertemporal elasticity of substitution is equal to the product of Arrow-Pratt

Measure Relative Risk Aversion for wealth and the elasticity of wealth for

consumption.

Or

Thus the Arrow-Pratt Measure Relative Risk Aversion is equal to the product of and

the elasticity of consumption for wealth and the intertemporal elasticity of substitution.

Since ww w

w

Vw w

Vλλ

− = − the Arrow-Pratt Measure Relative Risk Aversion the optimal

proportion invested in the risky asset is

( )( )

( )2 2

1* a a

w a a

r r r ra

RRA wwλλ σ σ

− −= − =



Thus the proportion invested in the risky asset is inversely related to the Relative

Risk Aversion of wealth and positively related to assets return over the risk free asset

and negatively to its variance. Thus if RRA is constant then a* will also be constant

over life.

/ /

cww

cc c ww w

F wVc c

F F V V

∂ ∂=

∂ ∂= −

For ( )lnc t the intertemporal elasticity of substitution is

2

1 1cc c

c c

F F c cc cF c F c

−

−

∂ −− = − = − × =

∂

( ) /1/

w wRRA wc c

∂=

∂ since 1w

c∂

= −∂

and so ( )1 c RRA ww

= − thus the Arrow-

Pratt Measure Relative Risk Aversion for logarithmic utility is ( ) wRRA wc

= − and

Returning to the other Hamiltonian Conditions

H2: ( ) ( ) ( )( ) ( ) ( )a

dw tH w rw t a t r r w t c tdtλ

∂= = = + − −

∂&

H3: ( )

( ) ( ) ( )( )( ) ( ) ( )2 2a w a

d t H t r a t r r a t w tdt w tλ

λ λ λ σ∂= = − = − + − −

∂&

Inserting H1A:( )

2* a

w a

r ra

wλλ σ

−= − into H2 and H3

H2: ( ) ( )( )

( ) ( ) ( ) ( )

( ) ( )( ) ( )

2

2

2

aa

w a

a

w a

t r rw rw t r r w t c t

w t

t r rrw t c t

λλ σ

λλ σ

−= − − −

−= − −

&



Since

( ) ww

w

w

w

VRRA w w

vw wc

c

λλ

λλ

= −

− = −

=

( ) ( ) ( )2

2 1a

a

r rw rw t c t

σ

⎛ ⎞−⎜ ⎟= − +⎜ ⎟⎝ ⎠

&

Now H3:

H3:

( ) ( )( )( ) ( ) ( )( )

( ) ( )

( ) ( ) ( )( )

( )( )

( )

( )

2

22 2

2 2 2 2

2 2

a aa w a

w a w a

a a

ww a a

t r r t r rt r r r w t

w t w t

t r r t r rt r

w tw t

r t

λ λλ λ λ σ

λ σ λ σ

λ λλ

λλ σ σ

λ λ

⎛ ⎞ ⎛ ⎞− −= − + − − −⎜ ⎟ ⎜ ⎟⎜ ⎟ ⎜ ⎟

⎝ ⎠ ⎝ ⎠

− −= − + −

= −

&

&

Thus ( ) ( )0 rtt eλ λ −= and H1 is now ( ) ( )−=0

1 r tc t e δ

λ so ( ) ( )

100

cλ

= thus

( ) ( ) ( )0 r tc t c e δ−=

( ) ( ) ( ) ( )2

20 1r t a

a

r rw rw t c e δ

σ−

⎛ ⎞−⎜ ⎟= − +⎜ ⎟⎝ ⎠

&

Denote ( )2

2 1a

a

r rσ

⎛ ⎞−⎜ ⎟= +⎜ ⎟⎝ ⎠

l

( ) ( )0

r tw rw t c e δ−− = −& l

( )( ) 0rt te w rw t c e δ− −− = −& l

Using ( )( )

( )= +∫ '

f xf x ee dx A

f x to integrate the RHS 1

0

1ts t

s

e ds e Aδ δ

δ− −

=

= − +∫ gives



( )( )

( )

( )

00

0 1 2

0

1

1

trt s

s

rt t

rt t

e w rw t c e ds

w t e c e A A

w t e A c e

δ

δ

δ

δ

δ

− −

=

− −

− −

− = −

= + −

= +

∫ ∫& l

l

l

setting t=0

( ) 010A w cδ

= − l

( ) ( ) ( )010 1rt tw t e w c e δ

δ− −= − −l

setting t = T

( ) ( ) ( )( )( )

( )

0

00

10 1

1 1

rT T

rT

T

w T e w c e

w w T ec

e

δ

δ

δ

δ

− −

−

−

= − −

−=

−

l

l

Note 20, 0a ar σ

∂ ∂> <

∂ ∂l l thus initial consumption 0 00, 0

a a

c cr σ

∂ ∂< >

∂ ∂

Consumption growth is the same ( ) ( ) ( )0 r tc t c e δ−=

While the PV of wealth ( ) ( ) ( )010 1rt tw t e w c e δ

δ− −= − −l will be higher for higher ar

and lower 2aσ .

Since the intertemporal elasticity of substitution (Arrow-Pratt degree of relative risk

aversion for consumption) is constant and equal to one. In ( ) wRRA wc

= − there is

no parameter to describe degree of degree of risk aversion for wealth with the

logarithmic utility function other than the wealth and consumption.



Hyperbolic Absolute Risk Aversion (HARA) class of utility function

However if we choose a different functional form for utility the picture is different. If

we consider a Hyperbolic Absolute Risk Aversion (HARA) class of utility function.

( )( ) ( )1u c t c t θ

θ=

For this HARA utility function the intertemporal elasticity of substitution is

( ) 2

1

11c cc

c c

cF Fc c cc F F c

θ

θ

θθ

−

−

−∂− = − = − × = −∂

( ) /1/

w wRRA wc c

θ ∂− =

∂ since 1w

c∂

= −∂

and so ( )1 c RRA ww

θ− = − thus the

Arrow-Pratt Measure Relative Risk Aversion for simple HARA utility is

( ) ( )1 wRRA wc

θ= − −

The corresponding stochastic Hamiltonian is

( ) ( ) ( ) ( ) ( ) ( )( )( ) ( ) ( ) ( ) ( )( )212

1, , , , , tw a w aH t w t c t a t e c t t r a t r r w t c t a t w tθδλ λ λ λ σ

θ− ⎡ ⎤⎡ ⎤ = + + − − +⎣ ⎦ ⎣ ⎦

%

H1A: 2 2 0a w aH r w rw awa

λ λ λ σ∂= − + =

∂

% ⇒

( )2* a

w a

r ra

wλλ σ

−= −

H1: 1 0tH e cc

δ θ λ− −∂= − =

∂

% ⇒

11

1 tc eθ

δ

λ

−

−⎛ ⎞= ⎜ ⎟⎝ ⎠

H3:

( ) ( )( )( ) ( ) ( )( )

( ) ( )

( ) ( ) ( )( )

( )( )

( )

( )

2

22 2

2 2 2 2

2 2

a aa w a

w a w a

a a

ww a a

t r r t r rt r r r w t

w t w t

t r r t r rt r

w tw t

r t

λ λλ λ λ σ

λ σ λ σ

λ λλ

λλ σ σ

λ λ

⎛ ⎞ ⎛ ⎞− −= − + − − −⎜ ⎟ ⎜ ⎟⎜ ⎟ ⎜ ⎟

⎝ ⎠ ⎝ ⎠

− −= − + −

= −

&

&

Thus ( ) ( )0 rtt eλ λ −= and H1 is now ( ) ( )( )

11

10

r tc t eθ

δ

λ

−

−⎛ ⎞= ⎜ ⎟⎜ ⎟⎝ ⎠

so ( ) ( )

11

100

cθ

λ

−

⎛ ⎞= ⎜ ⎟⎜ ⎟⎝ ⎠

thus

( ) ( ) ( )( )1

1

0 r tc t c eθ

δ−

−=



Since

( )

( )

( )

1

1

ww

w

w

w

VRRA w

vww

cc

λθ

λλ

θ λ

= −

− − = −

=−

Then substituting ( )1w

cλλ θ

=−

into w& to give

( ) ( ) ( ) ( )2

21a

w a

c t r rw rw t c t

θ λ σ−

= − −−

&

( ) ( ) ( )( )

2

2

1 11

a

a

r rw rw t c t

θσ

⎛ ⎞−⎜ ⎟= − +⎜ ⎟−⎝ ⎠

&

And substituting ( ) ( ) ( )( )1

1

0 r tc t c eθ

δ−

−=

( ) ( ) ( )( ) ( )( )

11

2

2

10 11

r t a

a

r rw rw t c e

θδ

θσ

−−

⎛ ⎞−⎜ ⎟= − +⎜ ⎟−⎝ ⎠

&

Denote ( )

( )

2

2

1 11

a

a

r rθσ

⎛ ⎞−⎜ ⎟= +⎜ ⎟−⎝ ⎠

l

( )( )1

0

rt

w rw t c eδθ−

−− = −& l

( )( )( )1

0

rr t

rte w rw t c eδθ

⎛ ⎞−−⎜ ⎟⎜ ⎟−− ⎝ ⎠− = −& l

Using ( )( )

( )= +∫ '

f xf x ee dx A

f x to integrate the RHS

( ) ( )1 1

10

1r rt s t

s

e ds e Ar

δ δθ θθ

δ

− −

− −

=

−= +

−∫ gives



( )( )( )

( )( )

( )( )

10

0

10 1 2

10

1

1

rt r srt

s

rr t

rt

rr t

rt

e w rw t c e ds

w t e c e A Ar

w t e A c er

δθ

δθ

δθ

θθ δ

θθ δ

⎛ ⎞−−⎜ ⎟⎜ ⎟−− ⎝ ⎠

=

⎛ ⎞−−⎜ ⎟⎜ ⎟−− ⎝ ⎠

⎛ ⎞−−⎜ ⎟⎜ ⎟−− ⎝ ⎠

− = −

−= − + −

−

−= −

−

∫ ∫& l

l

l

setting t=0

( ) 010A w c

rθ

θ δ−

= +−

l

( ) ( ) 10

10 1r trtw t e w c e

r

θ δθθ

θ δ

−− −

⎛ ⎞−= + −⎜ ⎟

− ⎝ ⎠l

Note that if θ=0 the logarithmic case

( ) ( ) ( )010 1rt tw t e w c e δ

δ− −= − −l

If t=T then we can use the terminal condition for wealth to remove 0c from the

problem. If there was no terminal condition on wealth we would need to use the

transversality conditions, (such as = 0Tλ if the terminal state is free) together with

the equation of motion of the co-state variable to provide 0λ and so 0c .

( ) ( )

( )( )

10

00

1

10 1

1 1

r TrT

rT

r T

w T e w c er

w w T ec

er

θ δθ

θ δθ

θθ δ

θθ δ

−− −

−

−−

⎛ ⎞−= + −⎜ ⎟

− ⎝ ⎠

−=

⎛ ⎞−−⎜ ⎟

− ⎝ ⎠

l

l

bea400 microeconomics lecture 11 module 5:...

Documents