1

Leigh Tesfatsion Professor of Econ, Math, and Electrical and Computer Engineering

Iowa State University, Ames, Iowa http://www.econ.iastate.edu/tesfatsi/

tesfatsi@iastate.edu

Presentation Slides: Last Revised 6/7/10Presentation Slides: Last Revised 6/7/10

Testing the Performance of Restructured Electric Power Markets

Illustrative Experimental FindingsIllustrative Experimental Findings

2

Project Directors: Leigh Tesfatsion (Prof. of Econ, Math, & ECpE, ISU) Dionysios Aliprantis (Ass’t Prof. of ECpE, ISU) David Chassin (Staff Scientist, PNNL/DOE)

Research Assoc’s: Dr. Junjie Sun (Fin. Econ, OCC, U.S. Treasury, Wash, D.C.) Dr. Hongyan Li (Consulting Eng., ABB Inc., Raleigh, NC)

Research Assistants:Research Assistants: Huan Zhao (Econ PhD Candidate, ISU);

Chengrui Cai (ECpE PhD Student, ISU);

Pedram Jahangiri (ECpE PhD Student, ISU);

Auswin Thomas (ECpE MS Student, ISU).

** Currently supported by grants from DOE at Pacific Northwest Currently supported by grants from DOE at Pacific Northwest National Lab (PNNL) & from the Electric Power Research CenterNational Lab (PNNL) & from the Electric Power Research Center

Project:Project: Integrated Wholesale/Retail Power Integrated Wholesale/Retail Power System Operation with SmartSystem Operation with Smart--Grid FunctionalityGrid Functionality

3

Wholesale & Retail Power System Operations

4

Integrated Wholesale/Retail System Operations

WholesaleTest bed (AMES)

developed by ISU Team

RetailTest bed (GridLAB-D)developed by DOE/PNNL

Seaming in ProgressSeaming in Progress

x

x

Bilateral Contracts

5

Meaning of “Smart Grid Functionality”?

For our project purposes:

SmartSmart--grid functionalitygrid functionality = Service-oriented grid enhancements permitting more responsiveness to needs and preferences of retail consumers.

Example:Example: Introduction of advanced metering and other technologies for support of flexible retail contracting and choice of suppliers by retail consumers

6

Project Context:Project Context: North American North American restructuring of wholesale power marketsrestructuring of wholesale power markets

In April 2003 the U.S. Federal Energy Regulatory Commission (FERC) proposed adoption of a wholesale power market design with particular core features.

Over 50% of North American generation now operates under some variant of the FERC design.

Adopters to Date: New York (NY-ISO), mid-Atlantic states (PJM), New England (ISO-NE), Midwest/Manitoba (MISO), Texas (ERCOT), Southwest (SPP), and California (CAISO)

Note:Note: Ontario (IESO), Alberta (AESO) , and other Canadian provinces have not fully adopted FERC design

7

Regions Operating Under Some Version of FERC Design http://www.ferc.gov/industries/electric/indus-act/rto/rto-map.asp

8

Core Features of FERCCore Features of FERC’’s Market Designs Market Design

• Market to be managed by an independent systemoperator (ISO) having no ownership stake

• Two-settlement system: Concurrent operation of day-ahead (forward) & real-time (spot) markets

• Transmission grid congestion managed via Locational Marginal Prices (LMPs), where LMP at bus k = least cost of servicing 1 additional MW of power at bus k

• Oversight & market power mitigation by outside agency

Has led in practice to complicated systems difficult to analyze by standard analytical & statistical tools !

9

Example: Complex MISO Market OrganizationBusiness Practices Manual 001-r1 (1/6/09)

Two-Settlement Power Market System under LMP

Core ofCore ofFERC designFERC design

Ss

AMES project to date

Xx

Xx

10

Project Work to Date: Wholesale LevelProject Work to Date: Wholesale Level

• Development and open-source release of AMES (Agent-based Modeling of Electricity Systems)

• AMES = Computational test bed incorporating core features of the FERC wholesale power market design

• Used to test performance under FERC design

• Used to test performance under modifications of design

AMES Homepage (code/manual/publications):AMES Homepage (code/manual/publications):http://www.econ.iastate.edu/tesfatsi/AMESMarketHome.htm

11

AMES (V2.05) Architecture AMES (V2.05) Architecture (based on business practices manuals for MISO/ISO(based on business practices manuals for MISO/ISO--NE)NE)

Two-settlement systemDay-ahead market (double auction, financial contracts) Real-time market (settlement of differences)

AC transmission gridGeneration Companies (GenCos)Generation Companies (GenCos) & LoadLoad--Serving Entities (LSEs)Serving Entities (LSEs)located at user-specified transmission busesGrid congestion managed via Locational Marginal Prices (Locational Marginal Prices (LMPsLMPs))LMP at bus k LMP at bus k = Least cost of servicing one additional MW of = Least cost of servicing one additional MW of power at bus kpower at bus k.

Independent System Operator (ISO)System reliability assessmentsDay-ahead scheduling via bid/offer bid/offer based optimal power flow (OPF)based optimal power flow (OPF)Real-time dispatch

TradersGenCos (sellers) LSEs (buyers)LearningLearning capabilities

12

AMES Modular & Extensible Architecture (Java)AMES Modular & Extensible Architecture (Java)

Market protocols & AC transmission grid structure― Graphical user interface (GUI) & modularized class structure

permit easy experimentation with alternative parameter settings and alternative institutional/grid constraints

Learning representations for traders― Java Reinforcement Learning Module (JReLM) ― “Tool box” permitting experimentation with a wide variety

of learning methods (Roth-Erev, Temp Diff/Q-learning,…)

Bid/offer-based optimal power flow formulation― Java DC Optimal Power Flow Module (DCOPFJ)― Permits experimentation with various DC OPF formulations

Output displays and dynamic test cases― Customizable chart/table displays & 5-bus/30-bus test cases

13

Illustrative AMES Experimental Illustrative AMES Experimental Findings for a 5Findings for a 5--Bus Test CaseBus Test Case

Definition: Definition: Incentive misalignmentIncentive misalignment →→Institutional design fails to align incentives of market participants with efficiency objectives (non-wastage of resources) and/or welfare objectives (socially desirable distribution of net surplus)

Experiments Reported Below:Experiments Reported Below: Incentive misalignment problems under FERC wholesale power market design for a range of experimental treatments:

• Generator learning [intensive parameter sweep]

• Sensitivity of wholesale demand to price [0 to 100%]

14

G4

G2G1 G3

G5

Bus 1 Bus 2 Bus 3

LSE 3LSE 3Bus 4Bus 5

LSELSE 11 LSE 2LSE 2

Five GenCo sellers G1,…,G5 and three LSE buyers LSE 1, LSE 2, LSE 3

5-Bus Transmission Grid(Used in many ISO business practice/training manuals)

$$$250 MW Capacity$

$ $ $$

15

GenCo True Cost & Capacity Attributes

15

16

Activities of AMES ISO During Each Operating Day D:Activities of AMES ISO During Each Operating Day D:Timing Adopted from Midwest ISO (MISO)Timing Adopted from Midwest ISO (MISO)

16

00:00

11:00

16:00

23:00

Real-Time

Market (RTM)

forday D

Day-Ahead Market (DAM)for day D+1

ISO collects bids/offers from LSEs and GenCos

ISO evaluates LSE demand bids and GenCo supply offers

ISO solves D+1 DC OPF and posts D+1 dispatch

and LMP schedule

Day-ahead settlementReal-time settlement

17

AMES ISO (Market Operator)Public Access:Public Access:

//// Public MethodsPublic MethodsgetWorldEventSchedule( clock time,… );getMarketProtocols( bid/offer reporting, settlement,… );Methods for receiving data;Methods for retrieving stored ISO data;

Private Access:Private Access://// Private MethodsPrivate Methods

Methods for gathering, storing, posting, & sending data;Method for solving hourly DC optimal power flow;Methods for posting schedules and carrying out settlements;Methods for implementing market power mitigation;

//// Private DataPrivate DataHistorical data (e.g., cleared bids/offers, market prices,…);Address book (communication links);

18

AMES ISO Solves Hourly DC Optimal Power Flow (OPF)AMES ISO Solves Hourly DC Optimal Power Flow (OPF)GenCos report hourly supply offers and LSEs report fixed &

price-sensitive hourly demand bids to ISO for day-ahead market

Subject to

Fixed and price-sensitive demand bids for LSE j

LSE gross buyer surplus

GenCo-reported total avoidable costs

R R

RUi

Max Max0 SLMaxjPurchase capacityinterval for LSE j

Dual variable for this bus-k balance constraint givesLMP for bus k

Operating capacityinterval for GenCo i

PUkm = Thermal limit for branch km

19

AMES ISO Solves DCAMES ISO Solves DC--OPF via DCOPFJ ModuleOPF via DCOPFJ Module

19

DC-OPF raw data (SI)

DCOPFJ Shell

Per Unit conversion

Form SCQP matrices

QuadProgJ: An SCQP solver

Per Unit SCQP output

Solution output (SI)

20

AMES AMES Generation Company (Seller)Generation Company (Seller)Public Access:Public Access:

// Public Methods// Public MethodsgetWorldEventSchedule( clock time,… );getMarketProtocols( ISO market power mitigation,… );Methods for receiving data;Methods for retrieving GenCo data;

Private Access:Private Access://// Private MethodsPrivate Methods

Methods for gathering, storing, and sending data;Methods for calculating own expected & actual net earnings;Method for updating own supply offers (LEARNING);

//// Private DataPrivate DataOwn capacity, grid location, cost function, current wealth… ;Data recorded about external world (prices, dispatch,…);Address book (communication links);

21

AMES GenCos are learners who report strategic supply offers to ISO for DAM

22

In 5-bus study, AMES GenCos use VRE learning (version of Roth-Erev stochastic reinforcement learning)

Each GenCo maintains action choice propensities q, normalized tochoice probabilities Prob, to choose actions (supply offers). Agood (bad) reward rk resulting from an action ak results in an increase (decrease) in both qk and Probk.

Action Choice a1

Action Choice a2

Action Choice a3

Choice Propensity q1 Choice Probability Prob1

Choice Propensity q2

Choice Propensity q3

Choice Probability Prob2

Choice Probability Prob3

rupdatechoose normalize

23

VRE Updating of Action PropensitiesVRE Updating of Action Propensities

Parameters:Parameters:• qj(1) Initial propensity• Experimentation• φ Recency (forgetting)

Variables:Variables:• aj Current action choice• qj Propensity for action aj• ak Last action chosen• rk Reward for action ak• t Current time step• N Number of actions

Xxxx

xxxx

Xxx

xxx

Xxxxxx

Ej(ε,N,k,t)

Ej( ε,N,k,t)

ε

24

From VRE Propensities to ProbabilitiesGibbsGibbs--Boltzmann ProbabilityBoltzmann Probability

qj(t) = “propensity” to choose action j at time tN = Total number of available actionsT = Temperature (“cooling”) parameter

Probj(t) = Probability of choosing action j at time t

N

n=1

e qn(t)/T

Xxx

xxxxProbj(t)

25

AMES GenCo learning implemented via AMES GenCo learning implemented via JReLMJReLM module module (Java Reinforcement Learning Module developed by(Java Reinforcement Learning Module developed byCharles J. Gieseler, Comp Sci M.S. Thesis, 2005)

Market Simulation

Learning Agent

26

AMES Load-Serving Entity (Buyer)Public Access:Public Access:// Public MethodsPublic Methods

getMarketProtocols(posting, trade, settlement);getMarketProtocols(ISO market power mitigation);Methods for receiving data;Methods for retrieving LSE data;

Private Access:Private Access:// Private MethodsPrivate Methods

Methods for gathering, storing, and sending data;Methods for calculating own expected & actual net earnings;

// Private DataPrivate DataOwn downstream demand, grid location, current wealth…;Data recorded about external world (prices, dispatch,…);Address book (communication links);

27

AMES LSE Hourly Demand-Bid Formulation

Hourly demand bid for each LSE j 

Fixed + Price‐Sensitive Demand Bid

FixedFixed demand bid = pFLj (MWs)

PricePrice‐‐sensitivesensitive demand bid 

= Inverse demand function for real power pSLj (MWs) overa purchase capacity interval:

Fj(pSLj)  =  cj ‐ 2dj pSLj

0  ≤ pSLj ≤ SLMaxj

28

R Measure for Systematically Varying the Amount of Demand-Bid Price Sensitivity

28

R = SLMaxSLMaxjj /[ ppFFLjLj + SLMaxSLMaxjj ]

For LSE j in Hour H:pFpFLjLj = Fixed demand for real power (MWs)SLMaxSLMaxjj = Maximum potential price-sensitive demand (MWs)

$/MWh

R=0.0 R=0.5 R=1.0

$/MWh $/MWh

SLMaxjSLMaxjpFLj pFLj

pLj pLj pLj

D D

D

(100% Fixed Demand) (100% Price-Sensitive Demand)

29

LSE Hourly Fixed Demands for R=0.0LSE Hourly Fixed Demands for R=0.0

17 = Peak demand hour

30

First Experiments: Avg GenCo net earningsGenCo net earnings (Day 1000) for R=0 under varied learning parameter settings

Small beta ≅“zero-intelligence”budget-constrainedtrading.

Learning matters !

= Sweet spotregion

= Selected

31

Lerner Index (LI): Measure of Market Power

Given binding capacity constraint on GenCo i, can have LIi > 0 withoutexercise of market power by GenCo i.

32

Second Experiments: Avg LMP with/without GenCo learning as demand varies from R=0 (100% fixed) to R=1 (100% price sensitive)

With GenCo Learning(Day 1000)

R R

Avg LMP (locational marginal price) Avg LI (Lerner Market Power Index)

33

Single-Run Illustration of Findings for R=0.0 (100% Fixed Demand)

W/O Gen Learning (Day 1000) With Gen Learning (Day 1000)

34

Summary of Findings from the First TwoSummary of Findings from the First TwoTwoTwo Sets of Experiments Sets of Experiments ((Li,Sun,TesfatsionLi,Sun,Tesfatsion, 2009), 2009)

BOTTOM LINE:

Even with 100% priceEven with 100% price--sensitive demand bids (R=1.0),sensitive demand bids (R=1.0),prices are much higher under GenCo learning due toprices are much higher under GenCo learning due tostrategic supply offers (capacity withholding).strategic supply offers (capacity withholding).

NEEDED:Countervailing powerCountervailing power == AActive LSE demand bidding as well as active GenCo supply offers at wholesale level to to support more competitive pricing at wholesale.support more competitive pricing at wholesale.

POSSIBLE MEANS:Integrated wholesale/retail restructuringIntegrated wholesale/retail restructuring providing providing array of pricearray of price--sensitive retail contracts and permitting sensitive retail contracts and permitting retail consumers to select their LSE suppliers retail consumers to select their LSE suppliers

35

Unfortunately, since 2000 Enron scandal, retail restructuring has been slowed

Source: www.eia.doe.gov/cneaf/electricity/page/restructuring/restructure_elect.html

(Jan 2010)Retail

36

Texas (ERCOT) has moved furthest towards full restructuring → Our project case study

Wholesale

Retail

x

x

Bilateral Contracts

37

Third Set of Experiments: Net surplus extraction by ISOs in day-ahead energy markets under Locational Marginal Pricing (LMP)

Day-ahead market activitieson a typical operating day D:

38

ISO Net Surplus Extraction ExampleISO Net Surplus Extraction Example

Cleared load = pFL. LSE at

bus 2 pays LMP2 > LMP1 for each unit of pF

L. M units of pF

L are supplied by cheaper G1 at bus 1 who receives only LMP1 per unit.

ISO collects difference:

ISO Net SurplusISO Net Surplus= [ LSE Payments

– GenCo Revenues ]

= M x [LMP2 – LMP1]

39

ISO Net Surplus Extraction Example ISO Net Surplus Extraction Example …… Cont.Cont.

ISO Net Surplus:ISO Net Surplus:INS=M x [LMP2 – LMP1]

GenCo Net Surplus:GenCo Net Surplus:Area S1 + Area S2

LSE Net Surplus:LSE Net Surplus:Area B

Total Net Surplus:Total Net Surplus:TNSTNS == [INS+S1+S2+B]

ISO Objective (OPF):ISO Objective (OPF):maximize TNS subject to

trans/gen constraints.

40

55--Bus Test Case Results Without GenCo LearningBus Test Case Results Without GenCo Learning::Daily ISO net surplus as LSE demand varies from

R=0.0 (100% fixed) to R=1.0 (100% price sensitive)

40

R Value

$

41

5-Bus Test Case Results With GenCo VRE Learning:Mean ISO net surplus on Day 1000 as LSE demand varies from R=0.0 (100% fixed) to R=1.0 (100% price sensitive)

41

R Value

$

42

Empirical Comparisons

From PJM 2008 report:ISO net surplus from day-ahead market: $2.66 billion$2.66 billion

From MISO 2008 report:ISO net surplus from day-ahead market: $500$500 millionmillion

From CAISO 2008 report:ISO net surplus from day-ahead inter-zonal congestion charges: $176 million$176 million.

From ISO-NE 2008 report:Combined ISO net surplus for real-time and day-ahead markets: $121 million$121 million.

42

43

Summary of Findings from ISO Net Surplus Experiments (Li/Tesfatsion, 2010)

ISO net surplus extractions not well aligned with market efficiency

Treatments resulting in greater GenCo economic capacity withholding (hence higher & more volatile LMPs) also result in greater ISO & GenCo net surplus

ISO net surplus collections should be allocated for ex ante remedy of structural/behavioral problems that encourage GenCo economic capacity withholding.

Should not be used ex post for LMP payment offsets and LMP risk hedge support (current norm)

44

OnOn--Line ResourcesLine Resources

Presentation SlidesPresentation Slideswww.econ.iastate.edu/classes/econ458/tesfatsion/WPMExperiments45www.econ.iastate.edu/classes/econ458/tesfatsion/WPMExperiments458.pdf8.pdf

AMES Test Bed Homepage AMES Test Bed Homepage (Code/Manual/Publications)(Code/Manual/Publications)www.econ.iastate.edu/tesfatsi/AMESMarketHome.htm

AgentAgent--Based Electricity Market ResearchBased Electricity Market Researchwww.econ.iastate.edu/tesfatsi/aelectric.htm

AgentAgent--Based Computational Economics HomepageBased Computational Economics Homepagewww.econ.iastate.edu/tesfatsi/ace.htmwww.econ.iastate.edu/tesfatsi/ace.htm