The Saga Continues:Measure Interactions for

Residential HVAC and Wx measures

Regional Technical ForumApril 23, 2014

HistoryMr. Toad’s Wild Ride AGAIN?

• The way we used to (currently) handle measure interactions:– “Last Measure In”

• After the calibration, we searched for a better method– First, we recommended “Option 3”,

• An improvement over Last Measure In allowed by RBSA (set the starting point) and necessitated by the SEEM calibration (energy use non-linear with UA).

– Then we flipped to “Participant or Population Data”,• Met our “indicators” to the “guiding principle”: first-year and long-term savings, and

based on observable data. (Option 3 didn’t meet the indicators.)– Now we’re proposing flopping back to Option 3

• More specifically, allowing it for Residential Weatherization and HVAC measures. The Participant and Population Data method still has merit.

2

3

Reminder: “Participant or Population Data”Slides from February meeting:

For Res Wx/HVAC, method either requires a lot of data, or a lot of guessing.

An Attempted Use of Participant Data

4

• ETO Dataset– 2013 Home Energy Rating (HER) Program• 140 of 2,000 audited homes had measures we’re

interested in installed (Wx, HVAC)– Data Collected• Audits provide insulation bins

– Attics: < or > R11– Walls: < or > R4– Floors: Insulated or Not– Windows: Single, Double metal, Double wood

• Measure installs provide more detail– Amount of insulation pre/post, square footage (for the

portion of the component worked on)

5

Starting Point

0102030405060708090100

0%

20%

40%

60%

80%

100%

Measure or Category

Attic Insulation Levels(Measures in ETO's HER 2013 Dataset)

Percent R-11 or less Percent More than R-11 Count

0102030405060708090100

0%

20%

40%

60%

80%

100%

Measure or Category

Wall Insulation Levels(Measures in ETO's HER 2013 Dataset)

Percent R-4 or less Percent More than R-4 Count

0102030405060708090100

0%

20%

40%

60%

80%

100%

Measure or Category

Floor Insulation Levels(Measures in ETO's HER 2013 Dataset)

Percent No Floor Insulation Percent Floor Insulation Installed Count

0102030405060708090100

0%

20%

40%

60%

80%

100%

Measure or Category

Windows(Measures in ETO's HER 2013 Dataset)

Percent Single Pane Percent Multi-Pane Metal Percent Multi-Pane Wood Count

6

Next Step: Adjust for Concurrent Measures

• Staff came up with a method and the ETO data can make it work.

7

ETO Data: Concurrent Measures

"Main measure" "Concurrent measure"When _____ measure is installed, how often is the _____ measure also installed, Frequency

CEILINGINSULATE FLOORINSULATE 61%CEILINGINSULATE WALLINSULATE 28%CEILINGINSULATE WINDOWS 7%CEILINGINSULATE AIRSEALING 16%CEILINGINSULATE DUCTINSULATE 14%CEILINGINSULATE DHP 3%FLOORINSULATE CEILINGINSULATE 56%FLOORINSULATE WALLINSULATE 7%FLOORINSULATE WINDOWS 5%FLOORINSULATE AIRSEALING 10%FLOORINSULATE DUCTINSULATE 39%FLOORINSULATE DHP 2%WALLINSULATE CEILINGINSULATE 63%WALLINSULATE FLOORINSULATE 43%WALLINSULATE WINDOWS 17%WALLINSULATE AIRSEALING 20%WALLINSULATE DUCTINSULATE 7%WALLINSULATE DHP 3%

WINDOWS CEILINGINSULATE 10%WINDOWS FLOORINSULATE 7%WINDOWS WALLINSULATE 10%WINDOWS AIRSEALING 6%WINDOWS DUCTINSULATE 6%WINDOWS DHP 0%

AIRSEALING CEILINGINSULATE 55%AIRSEALING FLOORINSULATE 57%AIRSEALING WALLINSULATE 30%AIRSEALING WINDOWS 15%AIRSEALING DUCTINSULATE 35%AIRSEALING DHP 10%

DUCTINSULATE CEILINGINSULATE 34%DUCTINSULATE FLOORINSULATE 70%DUCTINSULATE WALLINSULATE 7%DUCTINSULATE WINDOWS 10%DUCTINSULATE AIRSEALING 24%DUCTINSULATE DHP 0%

DHP CEILINGINSULATE 10%DHP FLOORINSULATE 8%DHP WALLINSULATE 5%DHP WINDOWS 0%DHP AIRSEALING 10%DHP DUCTINSULATE 0%

8

Adjusted Baseline

0102030405060708090100

0%

20%

40%

60%

80%

100%

Measure or Category

Adjusted for Concurrent Measures - Attic Insulation Levels(Measures in ETO's HER 2013 Dataset)

Percent R-11 or less Percent More than R-11 Count

0102030405060708090100

0%

20%

40%

60%

80%

100%

Measure or Category

Adjusted for Concurrent Measures - Wall Insulation Levels(Measures in ETO's HER 2013 Dataset)

Percent R-4 or less Percent More than R-4 Count

0102030405060708090100

0%

20%

40%

60%

80%

100%

Measure or Category

Adjusted for Concurrent Measures - Floor Insulation Levels(Measures in ETO's HER 2013 Dataset)

Percent No Floor Insulation Percent Floor Insulation Installed Count

0102030405060708090100

0%

20%

40%

60%

80%

100%

Measure or Category

Adjusted for Concurrent Measures - Floor Insulation Levels(Measures in ETO's HER 2013 Dataset)

Percent Single Pane Percent Multi-Pane Metal Percent Multi-Pane Wood Count

9

Next Step

• Average these baselines and apply them to the prototypes to calculate measure savings.

• But Wait. Can we still average the baselines and perform just one run in SEEM?

10

(1,000)

-

1,000

2,000

3,000

4,000

5,000

6,000

7,000

Uninsulated Insulated 25%Uninsluated

50%Uninsulated

75%Uninsulated

Heati

ng E

nerg

y Sa

ving

s (kW

h/yr

)Energy Savings Compared

(2200 sq.ft Crawlspace house in Portland)

SEEM Savings

SEEM Weighted Afterwards

Phase I Savings

Phase I Weighted Afterwards

11

Measure: Attic R0 to R38

0.00

0.20

0.40

0.60

0.80

1.00

1.20

0

10000

20000

30000

40000

50000

60000

0.00 0.10 0.20 0.30 0.40

Adju

stm

ent F

acto

r

Heati

ng E

nerg

y U

se E

stim

ate

(kw

h/ye

ar)

House Uo (UA divided by sum of heat loss surface areas)

Heating Zone 1, 2200 ft2 Prototype

SEEMAdjusted (Phase 1)P1 Adjustment Factor

12

13

Trouble with the “Participant or Population Data Method”

• Application of the ETO dataset taught two things about this method:– Analysis Paralysis

• SEEM calibration requires modeling each house’s starting and stopping condition. Averaging of insulation u-values doesn’t give accurate answers.

– Extreme Data Collection• Since houses coming into program vary over time,

and since program designs vary across programs and over time, we’d likely need data quite often (annually?) to stay true to this method.

• We need to know fairly well the u-values of each component, and infiltration levels.

So Now What?

14

15Renewed Interest in Option 3 … But what’s Option 3, again? Slides from October RTF Presentation:

Comparing the MethodsSelection Criteria

Participant or Population Data

MethodBilling Analysis

Method Option 3 Method

Usefulness to Programs

UES for Individual Measures

YesMaybe (may require larger sample, additional

analysis, assumptions)Yes

Data Collection Level of Effort

High (Detailed whole-house audit on a representative sample of program participants, ongoing

collection)

Low/Moderate No Data Collection*

Analysis

Timing 1 to 5 years 1 to 5 years 5 years +Performed By RTF Utilities RTF

Level of Effort High (at first, then depends on consistency of data quality/type) Moderate Low (centralized

analysis)

Accuracy

1st-year Savings

Moderate (depends on quality and representativeness of data; also depends on calibration’s alignment with measures)

High (for the participants we didn’t have to throw out);

Low (for participants with low statistical reliability.)

Low

Long-Term Savings

Any (depends on whether calibration represents long-term

conditions)

Any (depends on whether billing period represents long-term

conditions)

Any (depends on long-term success of achieving

“full measure package” and whether calibration

represents long-term conditions)

16

Slide from February RTF Presentation:

17Wait! What about the “Guiding Principle” and its “Indicators”?Slides from January RTF Presentation:

Do we need a 4th “Indicator”?• 4th Indicator: – Reasonable Level of Effort

• The method isn’t overly burdensome for programs or the RTF; it balances:– an appropriate level of effort for the savings potential, with

(Note level of effort includes data collection and analysis efforts today, as well as in the future to maintain the measure)

– validity of savings estimate, with– usefulness to the Region.

• Guidelines allows for this under “Best Practice”– “A best practice savings estimate is one that relies on the

best practical and reliable data collection and estimation methods. Practical means that the required data collection and estimation can be carried out with proven techniques and resources deemed reasonable by the RTF…”

18

19

Staff Recommendation• Proposal

– For handling Measure Interactions associated with the Residential Weatherization and HVAC measures, use “Option 3” where measure identifiers cannot be used.

• Justification– Option 3, could be considered “Best Practice”. While not perfect, it is

an appropriate blend of effort and accuracy, while still providing UES’s useful for planning.• It would be difficult to justify the “Participant or Population Data” method

over Impact Evaluation (billing analysis) since Impact Evaluation provides equivalent accuracy at less cost. But Impact Evaluation makes planning more difficult since it doesn’t directly provide a UES for each individual measure.

• Option 3’s short term inaccuracies are probably ok considering the Region’s needs are more long-term.

• Option 3’s long term inaccuracies are less significant the more program houses achieve the “full measure package” over time.

20

Decision

• “I _____ move the RTF approve the following method of accounting for measure interactions in Residential Weatherization and HVAC UES measures: (choose one)– A. Option 3 (justification: Best Practice);– B. “Participant or Population Data” (no change); or– C. Stop providing UES measures, instead require

impact evaluation.”

If we choose Option 3…

21

Slides from Staff Update at November RTF Meeting:

Slide from October RTF Presentation:

Defining “Full Measure Package” 22

Defining “Full Measure Package”• We still have more work to do; there

wasn’t consensus with the subcommittee.• Staff has come up with a new proposal to

adjust the full measure package definition for each Characteristic Scenario based on cost-effectiveness of measure installation. Our goal is to match what programs encourage for participating houses.– For example, in houses with R-30 in the

attic, it’s not cost-effective to add insulation. For that suite of Characteristic Scenario’s, Option 3 would assume the existing attic insulation defines the full measure package (not R-38).

• Next Steps: Staff’s suggestion is for staff to work on this proposal and bring it back to the full RTF once it’s been fully analyzed (or if we run into further major roadblocks).

23

0.00

0.20

0.40

0.60

0.80

1.00

1.20

0

10000

20000

30000

40000

50000

60000

0.00 0.10 0.20 0.30 0.40

Adju

stm

ent F

acto

r

Heati

ng E

nerg

y U

se E

stim

ate

(kw

h/ye

ar)

House Uo (UA divided by sum of heat loss surface areas)

Heating Zone 1, 2200 ft2 Prototype

SEEMAdjusted (Phase 1)

P1 Adjustment Factor

Any Suggestions?Re-naming the “Option 3” method

• Looking for a descriptive name• Some Options:– Adjusted Last-Measure-In– Pro-rated Additive Savings– Others?

24