Savings Are What You Define Them To Be:The “Baseline” Issue

Tom EckmanManager, Conservation Resources

Northwest Power and Conservation Council

2

Defining Savings and Baselines• Savings - The difference in energy use between the baseline and

post (after measure delivery) periods, which is caused by the installation/use of a measure or is attributable to one or more “mechanisms” (e.g, programs, codes and standards, etc).

• Baselines Can Be:– Current/common practice = baseline is used if the measure affects

systems, equipment or practices that are at the end of their useful life. In this case baseline use is defined by the recent typical choices of eligible end users in purchasing new equipment and services.

– Pre-conditions = baseline is used when the measure-affected equipment or practice still has remaining useful life. In this case baseline use is defined by typical existing conditions.

Baseline Types

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 200

5

10

15

20

25

30

Pre-Condition Current Practice Stock Average Efficient Case

Year

Ener

gy U

se

Current Practice Baseline Savings

Pre-Condition Baseline Savings

Baseline Types – The “Special One”Because it Comes in Two-Parts

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 200

5

10

15

20

25

30

Pre-Condition Current Practice Stock Average Efficient Case

Year

Ener

gy U

se

Current Practice Baseline Savings for EUL

Early Retirement Baseline Savings for RUL

Persistent savings is only equivalent to savings over “current practice,” and there’s a risk in might be less

Savings Are A Function of Perspective

• Resource Planning Perspective – Savings are the difference between forecast energy use and efficient case energy use

• Program Planning and Evaluation Perspective – Savings are the difference between the energy use that would have occurred “absent the program” and the energy use of program participants*

*Including spillover and market effects

slide 6

Resource Planning Perspective Focuses on Total Energy Use Reduction From All “Forces”

2010

2011

2012

2013

2014

2015

2016

2017

2018

2019

2020

2021

2022

2023

2024

2025

2026

2027

2028

2029

0

50,000

100,000

150,000

200,000

250,000

PNW Load w/o EfficiencyPNW Loads w/ Efficiency

GW

H/Y

R

7

Program Planning and Evaluation Perspective Attempts to Determine Attribution To Each “Force”

Source: Jayaweera, PhD, Tina and Hossein Haeri, PhD. “Methodology for Quantifying Market-Induced, Non-Programmatic Savings.” Prepared by Cadmus for BPA. April 1, 2011.

The Relationship Between Newtonian Physics and “Attribution”

• Newton’s First Law: Corpus omne perseverare in statu suo quiescendi vel movendi uniformiter in directum, nisi quatenus a viribus impressis cogitur statum illum mutare.

• Translation 1 by Dave Berry – A body at rest will remain at rest until 8:45 p.m. The night before the science-fair project is due, at which point the body will come rushing to the body’s parents, who are already in their pajamas, and shout, “I just remembered the science fair is tomorrow and we gotta go to the store right now!”

• Translation 2 - An object remains at rest or in motion unless acted upon by an external “force”

A Power Planner’s View of the Derivation of Attribution & “Net-to-Gross” Ratios

slide 10

A Power Planner’s View of Attribution and Net-To-Gross

• We Don’t Care Who Was Responsible for Screwing in the CFL

• We Do Need to Know That– It got installed– How much electricity

it will use– When it will use

electricity – How long it will

function

Implications for EM&V

• There is no such thing as “naturally occurring” efficiency improvements

• An external “force” must be exerted to move inefficient conditions to efficient conditions

• “Net-to-gross” evaluations attempt to quantify the magnitude of each “force”

• The choice of a “baseline” is not an assignment of “attribution”– But it does matter!

Should The Resource Planning and Program Planning “Perspectives” Use Different Baselines?

• No – If the reported savings are used to determine the need for additional resources

• Maybe – If the reported savings are used to determine compliance with an “EERS” (Energy Efficiency Resource Standard) which was set independent of resource need

• Yes – I can’t think of any logical reason

Why Both Perspectives Should Use Current/Common Practice Baselines

• Consistency with Load Forecast– The energy use of an appliance or piece of

equipment that form the basis for load forecast should also be the basis for determining the remaining potential for conservation

– Determining whether all or only a portion of the savings attributed to a program actually impact the need for resources cannot be done without comparison to what was explicitly or implicitly assumed in forecasting that need

Why Both Perspectives Should Use Current/Common Practice Baselines

• Experimental Design - Determining what is occurring prior to program launch is a better measure of what would have occurred absent the program (i.e., the counterfactual) than evaluations made after the program has influenced the market

Why Both Perspectives Should Use Current/Common Practice Baselines

• Establishing common practice baselines requires information on current market conditions prior to program or project design and implementation.

– As a result, assessments of both the need for and design of market intervention programs can be conducted prior to implementation.

– This reduces the probability of initiating programs where standard practices is already efficient and increases the likelihood of the targeting of niche markets that are not.

Why Both Perspectives Should Use Current/Common Practice Baselines

Use of a common practice baseline generally results in much smaller differences between gross and net savings (i.e., higher NTG ratios). As a result, regulatory processes, such as cost recovery proceedings, tend to be less acrimonious

Why Both Perspectives Should Use Current/Common Practice Baselines

• Use of Common Practice baselines tests whether incremental savings and further investments in energy efficiency are cost-effective to pursue, regardless of who gets credit for them.

Use of Common Practice Baselines Does Not Protect Against Self-Selection Bias

• However, the market research or other data analysis that must be carried out in order to estimate a common practice baseline can and should inform program designs so they more effectively target consumers who are not already adopting the efficient technology or practice

But, Poorly Designed Programs Are Still Possible!

Use of Common Practice Baselines Does Not Capture Non-Participant Spillover

• On the other hand, very few NTG evaluations capture non-participant spillover, so whether this is better or worse than the current majority paradigm isn’t clear.

Use of Common Practice Baselines Does Not Give Credit For Market Effects of Prior Programs

• On the other hand, very few NTG evaluations capture market effects of prior programs, so whether this is better or worse than the current majority paradigm isn’t clear.