© 2007 warren b. powell slide 1 the dynamic energy resource model lawrence livermore national...

© 2007 Warren B. Powell Slide 1

The Dynamic Energy Resource Model

Lawrence Livermore National LaboratoriesSeptember 24, 2007

Warren PowellAlan Lamont

Jeffrey StewartAbraham George

© 2007 Warren B. Powell, Princeton University


The dynamic energy resource model

Questions:» How will the market evolve in terms of the adoption of

competing energy technologies?• How many windmills, and where?• How much ethanol capacity?• How will the capacity of coal, natural gas and oil evolve?

» What government policies should be implemented?• Carbon tax? Cap and trade?• Tax credits for windmills and solar panels?• Tax credits for ethanol?

» Where should we invest R&D dollars?• Ethanol or hydrogen?• Batteries or windmills?• Hydrogen production, storage or conversion?



Features we need:» Multiple time scales

• Model will plan decades into the future, but reflect decisions and processes that occur on hourly, daily, seasonal and yearly levels.

» Multiple forms of uncertainty• We will model dynamic information processes that describe

the evolution of technology, climate, weather, prices and wind.

» Multiple levels of spatial granularity• The model will be able to run at different levels of spatial

aggregation, capturing the geographic substitution of different types of energy.

» Multi-attribute representation of markets• We want to be able to distinguish energy demands to capture

usage and lifestyle patterns.


Outline

A deterministic modelA stochastic, dynamic energy modelADP for energy capacity management


Outline



Deterministic models

Deterministic, linear programming-based models» Basic model:

» Features:• Can model flows of energy and substitution of energy

resources over time.• Assumes a deterministic view of the world (everything is

known now).

1 1min subject to , 0t t t t t t t tt

c x A x B x R x



A single large linear program:

TimeSpace



Limitations» Unable to model uncertainty in technology, climate,

prices.

» Unable to model activities at a high level of detail. Large linear program limits the number of rows, which grows rapidly as we use finer representations of resources and markets.

» Traditionally uses a discrete time representation, making it hard to handle fine time scales (e.g. hourly) over long horizons (e.g. 50 years).


Outline



Stochastic, dynamic model

The state of a resource:

Capacity of facilities

Location

Cost

Carbon output

Age

Reserves

ta



Modeling multiple energy resources:» The attributes of a single resource:

» The resource state vector:

» The information process:

The attributes of a single resource

The attribute space

a

a

A

ˆ The change in the number of resources with

attribute .taR

a

The number of resources with attribute

The resource state vector

ta

t ta a

R a

R R

A



Modeling market demands:» The attributes of a single type of demand:

» The demand state vector:

» The information process:ˆ The change in the number of demands with

attribute .tbD

b

The attributes of a demand to be served.

The attribute space

b

b

B

The number of demands with attribute

The demand state vector

tb

t tb b

D b

D D

B



The system state:

, , System state, where:

Resource state (how much capacity, reserves)

Market demands

"system parameters"

State of the technology (costs, pe

t t t t

t

t

t

S R D

R

D

rformance)

Climate, weather (temperature, rainfall, wind)

Government policies (tax rebates on solar panels)

Market prices (oil, coal)



The three states of our system» The state of a single resource/entity

» The resource state vector

» The system state vector

1

2

3

t

t t

t

a

a a

a

1

2

3

ta

t ta

ta

R

R R

R

, ,t t t tS R D



The decision variable:

New capacity

Retired capacity

:

Type

Location

Technology

t

for eachx



Exogenous information:

ˆ ˆ ˆNew information = , ,t t t tW R D

where:

ˆ Exogenous changes in capacity, reserves

ˆ New demands for energy from each source

ˆ Exogenous changes in parameters.

Change in technology

Ch

t

t

t

R

D

ange in climate/weather

Change in prices/market supplies


Stochastic, dynamic model Hourly

» Daily temperature variation» Wind» Equipment failures

Daily» Fluctuation in spot prices» Short term demand» Major weather events» Transportation delays (movement of coal and oil)

Monthly» Seasonal variation (temperature, water flow for hydro, population shifts)» Medium term weather patterns» Significant supply disruptions (major hurricane, wars)

Yearly» Changes in technology» Demand patterns (SUV’s)» Long term climate cycles (including global warming)» Spatial patterns in population growth» New supply discoveries (major oil fields)» Intervention of foreign governments in markets» Long term supply contracts

tW



The transition function

1 1( , , )Mt t t t tS S S x W S

t t+1



Our strategy:» Basic model:

» Features:• Simulation-based – We simulate forward in time using a very

general-purpose transition model.• Handles virtually any form of uncertainty.• Can use a range of policies for different types of decisions,

from simple dispatch rules to more sophisticated policies that look into the future.

1 1

( )

, ,

t t t

Mt t t t

x X S

S S S x W

Make a decision using policy

Update state using system model


Information and decisionsInformation

T – changes in technologyS – changes in energy suppliesP – changes in energy prices

W – Weather

DecisionsI – Changes in energy capacity infrastructure (new plants, new fields)S – Short term changes in suppliesR – R&D investmentsM – Market response

T T T T

W WW W WW W WW W WW W WWW WWS S S S S S SSP P PP P P P P P P P P P P P P P P P

I I I IR RR R R R R R R RS S S S S S S S S S S S S S S S S SM M M M M M M M M


Making decisions

Dispatch decisions:» Use the technology with the lowest marginal cost.

» Small linear program to handle substitution of different types of power.

GENERATORS MARKETS

2

1

4

3

A

C

B

I

II

III

ENERGY SOURCES


Making decisions

Hydro power management» Forecast inflow and outflow to reservoirs to determine

amount available for generating electricity

r0l

1l

L

1 2 3 t

r0l

1l

L

1 2 3 t

r

0l

1l

1l

L

1 2 3 t

2lr

0l

1l

1l

L

1 2 3 t

2l r

1l2l

2l

0l

L

1 2 3 t

r

1l2l

2l

0l

L

1 2 3 t


Making decisions

Purchasing new capacity:» A decision to add capacity in year t changes the

capacity available in year t+1.

» Resource transition function

Resource state vector at time .t ta a AR R t

1 1ˆ

t t t tR R x R

Resource state vector

Capacity change decisions

Exogenous changes to resources


Making decisions

Purchasing new capacity:» We want our capacity acquisition decisions to mimic the

intelligence that companies/financial markets make.

» We propose to “simulate Wall St.” by solving the capacity acquisition problem as an optimization problem to find the policy that solves:

» The optimal policy is characterized by Bellman’s equation:

» Problem: Solving this equation is computationally intractable because of the “three curses of dimensionality.”

min ( , ( )) t t t tt

E C S X S

1 1( ) min ( , ) ( )tt t x t t t t tV S C S x EV S


Approximate dynamic programming

Solving the dynamic program using approximate dynamic programming:» Step 1: Break transition into two steps:

» Step 2: Formulate value function around post-decision state:

» Step 3: Replace value function with approximation

» Step 4: Design strategy for updating the approximation

1 1

Post-decision state variable

ˆ New pre-decision state variable

xt t t

xt t t

R R x

R R R

( ) min ( , ) ( )t

x xt t x t t t t tV S C S x V R

( ) arg min ( , ) ( )t

xt t x t t t t tX S C S x V R


Part VII - CASTLE Lab NewsCASTLE Lab News

New Modeling Language Captures Complexities of Real-World Operations!

75 cents

Spans the gap betweensimulation and optimization.

CASTLE Lab announced the development of a powerful new simulation environment for modeling complex operations in transportation and logistics. The dissertation of Dr. Joel Shapiro, it offers the flexibility of simulation environments, but the intelligence of optimization. The modeling language will allow managers to quickly test continued on page 3

Thursday, March 2, 1999


Outline



ADP for energy resource management

oiltx

2008

oiltR ˆ oil

tD ˆ oiltˆ oil

tR

New information 2009

1oiltR 1

oiltx 1

ˆ oiltD 1

ˆ oilt 1

ˆ oiltR

New information

windtxwind

tR ˆ windtD ˆ wind

tˆ windtR 1

windtR 1

windtx 1

ˆ windtD 1

ˆ windt 1

ˆ windtR

coaltxcoal

tR ˆ coaltD ˆ coal

tˆ coaltR 1

coaltR 1

coaltx 1

ˆ coaltD 1

ˆ coalt 1

ˆ coaltR

corntxcorn

tR ˆ corntD ˆ corn

tˆ corntR 1

corntx 1

corntR 1

ˆ corntD 1

ˆ cornt

ˆ corntR


We have to allocate resources before we know the demands for different types of energy in the future:



We use value function approximations of the future to make decisions now:



,,1x ntR

,,2x ntR

,,3x ntR

,,4x ntR

,,5x ntR

This determines how much capacity to provide:



,1ˆ ( )ntv

,2ˆ ( )ntv

,3ˆ ( )ntv

,4ˆ ( )ntv

,5ˆ ( )ntv

Marginal value:

,,1x ntR

,,2x ntR

,,3x ntR

,,4x ntR

,,5x ntR



1, 1,( )xt AB t ABV R

,1,

x nt ABR

Using the marginal values, we iteratively estimate piecewise linear functions.



R1t

ktv

ktv

Right derivativeLeft derivative


,1,

x nt ABR




R1t

( 1)ktv ( 1)k

tv


,1,

x nt ABR




Piecewise linear, separable value function approximations:

Piecewise linear, separable:

( ) ( )t t tl tll

V R V R

L




t



Features» Extremely flexible

• Simulation-based modeling is able to handle high level of detail about energy resources and demands, time scales and uncertainties.

• Can handle mixed policies: – Myopic policies for dispatch problem– Rolling horizon procedures for hydro– Dynamic programming-based policies for capacity

acquisition

» Challenges• Value function approximations have to be designed to handle

the state of technology and climate.• Strategies have to be designed to guide the system to reach

different goals.• Measures for evaluating solution quality (is it realistic? near-

optimal?) need to be designed.



Uncertainties (exogenous information processes)» Technology:

• Carbon sequestration• The cost of batteries, fuel cells, solar panels• The storage of hydrogen, efficiency of solar panels, …

» Climate: • Global and regional temperatures• Changing patterns of snow storage on mountains• Wind patterns

» Markets: • Global supplies of oil and natural gas• International consumption patterns• Domestic purchasing behaviors (SUV’s?)• Tax policies• The price of oil and natural gas



Alternative ways of solving large stochastic optimization problems:» Simulation using myopic policies – Using rules to determine

decisions based on the current state of the system. Rules are hard to design, and decisions now do not consider the impact on the future.

» Deterministic optimization – Ignores uncertainty (and problems are still very large scale).

» Rolling horizon procedures – Uses point estimates of what might happen in the future. Will not produce robust behaviors.

» Stochastic programming – Cannot handle multiple sources of uncertainty over multiple time periods.

» Markov decision processes – Discrete state, discrete action will not scale (“curse of dimensionality”)


Dynamic energy resource management Proposed approach: Approximate dynamic

programming» Our research combines mathematical programming, simulation

and statistics in a dynamic programming framework.• Math programming handles high-dimensional decisions.• Simulation handles complex dynamics and high-dimensional

information processes.• Statistical learning is used to improve decisions iteratively.• Solution strategy is highly intuitive – tends to mimic human behavior.

» Features:• Scales to very large scale problems.• Easily handles complex dynamics and information processes.• Rigorous theoretical foundation

» Research challenge:• Calibrating the model.• Designing high quality policies using the tools of approximate

dynamic programming.• Evaluating the quality of these policies.

© 2007 warren b. powell slide 1 the dynamic energy resource model lawrence livermore national...

Documents

dynamic energy modeladp

dynamic modelthe state

flows of energy

dynamic modelthe system

dynamic information

systemthe state

ethanol capacity

demand state vector