Microsoft Word - 37-RTO workpaper_041816.docx
XX-XX: Behavioral Modification: Home Energy Reports
Residential (Single and Multi-Family)
Electric and Natural Gas
Issued: 10/26/2020
Prepared by: Guidehouse Inc,
Version: 1.1
Michigan Residential Measures1Version 1.1
Document Revision History
Version #
Revision Date
Description of Revisions and Affected Sections
Edited By
1.1
10/26/2020
· Added requirement for number of reports to section 4.
· Updated required components in Section 4.1 based on conversations with Oracle on what key components have appeared in all past reports.
· Confirmed that multi-family residences are eligible for the program and were included in calibration. Edited language on title page and in Section 4.2.
· Clarified relationship between coincidence factor and demand savings factor in section 9.
Guidehouse
EWR Collaborative Review and Decision
Date
Description of Collaborative Decision
Lead
Table of Contents
1.New Measure and Update to Existing Measure11.1 Updates to Existing Measures11.2 New Measures31.3 Missing Measures32.Purpose and Measure Background33.Measure Description44.Measure Specification & Savings Summary44.1 Measure Specification44.2 Requirements for Application44.3 Savings Summary55.Baseline and Proposed Improvement Description75.1 Baseline Measure, Practice, or Technology75.2 Improved Measure, Practice, or Technology86.Methodology and Assumptions86.1 Data96.2 Energy Savings106.2.1 Regression Analysis106.2.2 Annualizing Savings106.2.3 Potential Savings Overlap (or Uplift Adjustment)116.3 Coincident Peak Demand Savings126.3.1 Regression Analysis126.3.2 Potential Savings Overlap (or Uplift Adjustment)126.3.3 Demand Savings Factor137.Estimated Savings over Baseline138.Potential Savings Overlap149.Coincidence Factor1410.Measure Life1411.Measure Cost1412.Any Recurring Cost1413.Relevant Codes and Standards1414.Ongoing EM&V, Research, and Calibration Planning1415.Sources of Information1516.Attachments15
Michigan Behavior Resource Manual Residential Measures11Version 1.1
1. New Measure and Update to Existing Measure
Based on the results of a calibration study, this whitepaper proposes:
(1) Updates to existing electric and gas measures based on the results of the calibration study. In some cases, there was insufficient data to calibrate the energy savings estimate. In these cases, this whitepaper proposes updates to the existing measures just to account for the calibrated uplift adjustment factors and the demand savings factor.
(2) New measure submissions resulting from the introduction of new electric usage bands (splitting the >11k kWh usage band into an 11k-13k kWh usage band and a >13k kWh usage band) and a new gas usage band (601-900 therms), or where a BRM value did not exist but for which there was sufficient data to calibrate.
(3) In some cases, a usage band/year combination does not exist in the BRM and there was insufficient data to calibrate. In these cases, this whitepaper proposes that a utility would claim savings using the last year available in the BRM for that usage band.
The following table identifies the type of proposed update for each measure:
Fuel
Usage Band
Year 1
Year 2
Year 3
Year 4
Year 5
Year 6
Year 7
Year 8
Electric
5k-7k kWh
CV
CV
CV
CV
CV
CV
Electric
7k-9k kWh
CV
CV
CV
CV
CV
PCV
CV
Electric
9k-11k kWh
CV
CV
CV
CV
CV
CV
PCV
CV
Electric
11k-13k kWh
NM
NM
NM
NM
NM
Electric
>13k kWh
NM
NM
NM
NM
NM
NM
Gas
601-900 Therms
NM
NM
NM
NM
Gas
901-1200 Therms
CV
CV
CV
CV
CV
CV
CV
Gas
>1200 Therms
CV
CV
CV
CV
PCV
PCV
PCV
CV
CV = Calibrated Value: Update to existing measure based on results of the calibration study
PCV = Partially Calibrated Value: Update to existing measure based on results of the calibrated uplift and demand savings factor only
NM = New Measure: New measure submission
Blank: Missing value
1.1 Updates to Existing Measures
This whitepaper proposes updates to the following existing electric and gas measures based on the results of calibration.
Measure Code[footnoteRef:2] [2: The measure code uses the following structure: MeasureType-Sector-MeasureCategory-MeasureNumber-FuelType-Climate-BuildingType-System-Vintage-MeasureVersion. All measures included in this Behavior Resource Manual will be Measure Type “B”. ]
Electric/ Gas
Behavior Measure Description
B-RE-MS-000003-E-XX-XX-XX-XX-01
Electric
Behavior Modification: Home Energy Reports, 9k-11k kWh Annual Usage (1st year)
B-RE-MS-000004-E-XX-XX-XX-XX-01
Electric
Behavior Modification: Home Energy Reports, 7k-9k kWh Annual Usage (1st year)
B-RE-MS-000005-E-XX-XX-XX-XX-01
Electric
Behavior Modification: Home Energy Reports, 5k-7k kWh Annual Usage (1st year)
B-RE-MS-000007-E-XX-XX-XX-XX-01
Electric
Behavior Modification: Home Energy Reports, 9k-11k kWh Annual Usage (2nd year)
B-RE-MS-000008-E-XX-XX-XX-XX-01
Electric
Behavior Modification: Home Energy Reports, 7k-9k kWh Annual Usage (2nd year)
Measure Code
Electric/ Gas
Behavior Measure Description
B-RE-MS-000009-E-XX-XX-XX-XX-01
Electric
Behavior Modification: Home Energy Reports, 5k-7k kWh Annual Usage (2nd year)
B-RE-MS-000010-E-XX-XX-XX-XX-01
Electric
Behavior Modification: Home Energy Reports, 9k-11k kWh Annual Usage (3rd year)
B-RE-MS-000011-E-XX-XX-XX-XX-01
Electric
Behavior Modification: Home Energy Reports, 7k-9k kWh Annual Usage (3rd year)
B-RE-MS-000012-E-XX-XX-XX-XX-01
Electric
Behavior Modification: Home Energy Reports, 9k-11k kWh Annual Usage (4th year)
B-RE-MS-000013-E-XX-XX-XX-XX-01
Electric
Behavior Modification: Home Energy Reports, 7k-9k kWh Annual Usage (4th year)
B-RE-MS-000014-E-XX-XX-XX-XX-01
Electric
Behavior Modification: Home Energy Reports, 9k-11k kWh Annual Usage (5th year)
B-RE-MS-000015-E-XX-XX-XX-XX-01
Electric
Behavior Modification: Home Energy Reports, 7k-9k kWh Annual Usage (5th year)
B-RE-MS-000016-E-XX-XX-XX-XX-01
Electric
Behavior Modification: Home Energy Reports, 9k-11k kWh Annual Usage (6th year)
B-RE-MS-000017-E-XX-XX-XX-XX-01
Electric
Behavior Modification: Home Energy Reports, 7k-9k kWh Annual Usage (6th year)
B-RE-MS-000018-E-XX-XX-XX-XX-01
Electric
Behavior Modification: Home Energy Reports, 9k-11k kWh Annual Usage (7th year)
B-RE-MS-000019-E-XX-XX-XX-XX-01
Electric
Behavior Modification: Home Energy Reports, 7k-9k kWh Annual Usage (7th year)
B-RE-MS-000020-G-XX-XX-XX-XX-01
Gas
Behavior Modification: Home Energy Reports, 900-1200 Therms Annual Usage (1st year)
B-RE-MS-000021-G-XX-XX-XX-XX-01
Gas
Behavior Modification: Home Energy Reports, >1200 Therms Annual Usage (1st year)
B-RE-MS-000022-G-XX-XX-XX-XX-01
Gas
Behavior Modification: Home Energy Reports, 900-1200 Therms Annual Usage (2nd year)
B-RE-MS-000023-G-XX-XX-XX-XX-01
Gas
Behavior Modification: Home Energy Reports, >1200 Therms Annual Usage (2nd year)
B-RE-MS-000024-G-XX-XX-XX-XX-01
Gas
Behavior Modification: Home Energy Reports, 900-1200 Therms Annual Usage (3rd year)
B-RE-MS-000025-G-XX-XX-XX-XX-01
Gas
Behavior Modification: Home Energy Reports, >1200 Therms Annual Usage (3rd year)
B-RE-MS-000026-G-XX-XX-XX-XX-01
Gas
Behavior Modification: Home Energy Reports, 900-1200 Therms Annual Usage (4th year)
B-RE-MS-000027-G-XX-XX-XX-XX-01
Gas
Behavior Modification: Home Energy Reports, >1200 Therms Annual Usage (4th year)
B-RE-MS-000028-G-XX-XX-XX-XX-01
Gas
Behavior Modification: Home Energy Reports, 900-1200 Therms Annual Usage (5th year)
B-RE-MS-000029-G-XX-XX-XX-XX-01
Gas
Behavior Modification: Home Energy Reports, >1200 Therms Annual Usage (5th year)
B-RE-MS-000030-G-XX-XX-XX-XX-01
Gas
Behavior Modification: Home Energy Reports, 900-1200 Therms Annual Usage (6th year)
B-RE-MS-000031-G-XX-XX-XX-XX-01
Gas
Behavior Modification: Home Energy Reports, >1200 Therms Annual Usage (6th year)
B-RE-MS-000032-G-XX-XX-XX-XX-01
Gas
Behavior Modification: Home Energy Reports, 900-1200 Therms Annual Usage (7th year)
B-RE-MS-000033-G-XX-XX-XX-XX-01
Gas
Behavior Modification: Home Energy Reports, >1200 Therms Annual Usage (7th year)
1.2 New Measures
This whitepaper proposes the addition of two new usage bands: (1) splitting the current >11k kWh annual usage band into an 11k-13k kWh and >13k kWh annual usage band, and (2) adding a new 600-900 therm annual usage band. In addition, we propose new values where values did not exist within the BRM but for which there was sufficient data to calibrate.
Electric/ Gas
Behavior Measure Description
Electric
Behavior Modification: Home Energy Reports, >13k kWh Annual Usage (1st year)
Electric
Behavior Modification: Home Energy Reports, 11k-13k kWh Annual Usage (1st year)
Electric
Behavior Modification: Home Energy Reports, >13k kWh Annual Usage (2nd year)
Electric
Behavior Modification: Home Energy Reports, 11k-13k kWh Annual Usage (2nd year)
Electric
Behavior Modification: Home Energy Reports, >13k kWh Annual Usage (3rd year)
Electric
Behavior Modification: Home Energy Reports, 11k-13k kWh Annual Usage (3rd year)
Electric
Behavior Modification: Home Energy Reports, 5k-7k kWh Annual Usage (3rd year)
Electric
Behavior Modification: Home Energy Reports, >13k kWh Annual Usage (4th year)
Electric
Behavior Modification: Home Energy Reports, 11k-13k kWh Annual Usage (4th year)
Electric
Behavior Modification: Home Energy Reports, 5k-7k kWh Annual Usage (4th year)
Electric
Behavior Modification: Home Energy Reports, >13k kWh Annual Usage (5th year)
Electric
Behavior Modification: Home Energy Reports, 11k-13k kWh Annual Usage (5th year)
Electric
Behavior Modification: Home Energy Reports, 5k-7k kWh Annual Usage (5th year)
Electric
Behavior Modification: Home Energy Reports, >13k kWh Annual Usage (6th year)
Electric
Behavior Modification: Home Energy Reports, 5k-7k kWh Annual Usage (6th year)
Electric
Behavior Modification: Home Energy Reports, 9k-11k kWh Annual Usage (8th year)
Gas
Behavior Modification: Home Energy Reports, 600-900 Therms Annual Usage (1st year)
Gas
Behavior Modification: Home Energy Reports, 600-900 Therms Annual Usage (2nd year)
Gas
Behavior Modification: Home Energy Reports, 600-900 Therms Annual Usage (3rd year)
Gas
Behavior Modification: Home Energy Reports, 600-900 Therms Annual Usage (5th year)
Gas
Behavior Modification: Home Energy Reports, >1200 Therms Annual Usage (8th year)
1.3 Missing Measures
This whitepaper proposes that when a usage band/year combination does not exist in the BRM, utilities should claim savings using the last year available in the BRM for that usage band. For example, if a wave in Year 4 was in the 600-900 therm usage band, the utility would claim the Year 3 value as the Year 4 value does not exist.
2. Purpose and Measure Background
This whitepaper reports the results of a 2020 calibration of the Behavior Modification: Home Energy Report measure. All usage bands with available data for a given program year were calibrated and this whitepaper proposes updated or new savings values accordingly. The calibration study also calibrated the demand savings factor for this measure.
Note, this whitepaper should not serve as an evaluation guidance document for custom evaluation of behavior modification measures. Additionally, any new measure submissions for the Behavior Modification: Home Energy Report measure should include descriptions of how the new measure values were estimated.
3. Measure Description
The Home Energy Reports (HER) program provides residential households accurate and timely information on their energy consumption through a variety of communication methods to change the consumers’ energy usage behavior.
·
The HER program is organized around two concepts. First, to motivate consumers through normative messaging to change their energy-use behavior. Personalized neighbor comparisons based on the size of the home, location, and heating fuel type—among other criteria—give households a motivational benchmark for their energy usage. Second, to provide them with salient, personalized advice to capitalize on this motivation to use less energy and save money.
Program reports are delivered through direct mail and are often supplemented with digital communications such as email, web, mobile phones, and social networks. This platform approach ensures that all households can access this information in the most effective way.
Applicable Building Type(s)
Applicable Sub-categories
Applicable Fuel Type(s)
Affected End-Use Load(s)
Electricity
Natural Gas
Space Heating
Space Cooling
Water Heating
Lighting
Appliance
Process
Other
Residential
n/a
ü
ü
ü
ü
ü
ü
ü
ü
4. Measure Specification & Savings Summary4.1 Measure Specification
HERs are a tool that can be deployed to residential customers to drive a behavioral change to reduce home energy consumption. To qualify for this measure, a program must send recurring reports by direct mail targeting a minimum of four reports per year.[footnoteRef:3] A program can supplement with digital communication but an all-digital program would not qualify for this measure. Key information required to qualify for the measure includes, at a minimum: [3: A wave might not reach four reports in a program year if it is launched late in the year but should reach four reports in the first 12-months after wave launch.]
1. Comparison of the customer’s home energy use to energy usage of other homes
2. Suggested actions the customer can take to improve energy efficiency
In addition, the program must have a simple opt-out process.
4.2 Requirements for Application
The Behavior Modification: Home Energy Report measure is restricted to residential customers.
4.3 Savings Summary
Electric Measures
Measure Code[footnoteRef:4] [4: The measure code uses the following structure: MeasureType-Sector-MeasureCategory-MeasureNumber-FuelType-Climate-BuildingType-System-Vintage-MeasureVersion. All measures included in this Behavior Resource Manual will be Measure Type “B”. For existing measures, the measure version has been updated from “01” to “02”. For new and missing measures, the measure code must be assigned.]
Year
Usage Band
Electric Savings per Customer
Coincident Peak Demand Savings
1st Year
>13k kWh Annual Usage
1.13%
0.88%
1st Year
11k-13k kWh Annual Usage
0.90%
0.70%
B-RE-MS-000003-E-XX-XX-XX-XX-02
1st Year
9k-11k kWh Annual Usage
0.78%
0.61%
B-RE-MS-000004-E-XX-XX-XX-XX-02
1st Year
7k-9k kWh Annual Usage
0.72%
0.56%
B-RE-MS-000005-E-XX-XX-XX-XX-02
1st Year
5k-7k kWh Annual Usage
0.12%
0.09%
2nd Year
>13k kWh Annual Usage
1.88%
1.47%
2nd Year
11k-13k kWh Annual Usage
1.42%
1.11%
B-RE-MS-000007-E-XX-XX-XX-XX-02
2nd Year
9k-11k kWh Annual Usage
1.36%
1.06%
B-RE-MS-000008-E-XX-XX-XX-XX-02
2nd Year
7k-9k kWh Annual Usage
1.27%
0.99%
B-RE-MS-000009-E-XX-XX-XX-XX-02
2nd Year
5k-7k kWh Annual Usage
0.59%
0.46%
3rd Year
>13k kWh Annual Usage
2.19%
1.71%
3rd Year
11k-13k kWh Annual Usage
1.66%
1.29%
B-RE-MS-000010-E-XX-XX-XX-XX-02
3rd Year
9k-11k kWh Annual Usage
1.50%
1.17%
B-RE-MS-000011-E-XX-XX-XX-XX-02
3rd Year
7k-9k kWh Annual Usage
1.29%
1.01%
3rd Year
5k-7k kWh Annual Usage
0.87%
0.68%
4th Year
>13k kWh Annual Usage
2.11%
1.65%
4th Year
11k-13k kWh Annual Usage
1.84%
1.44%
B-RE-MS-000012-E-XX-XX-XX-XX-02
4th Year
9k-11k kWh Annual Usage
1.42%
1.11%
B-RE-MS-000013-E-XX-XX-XX-XX-02
4th Year
7k-9k kWh Annual Usage
1.81%
1.41%
4th Year
5k-7k kWh Annual Usage
0.77%
0.60%
5th Year
>13k kWh Annual Usage
2.19%
1.71%
5th Year
11k-13k kWh Annual Usage
1.54%
1.20%
B-RE-MS-000014-E-XX-XX-XX-XX-02
5th Year
9k-11k kWh Annual Usage
1.24%
0.97%
B-RE-MS-000015-E-XX-XX-XX-XX-02
5th Year
7k-9k kWh Annual Usage
1.64%
1.28%
5th Year
5k-7k kWh Annual Usage
0.89%
0.69%
6th Year
> 13 kWh Annual Usage
1.98%
1.54%
6th Year
11k-13k kWh Annual Usage
*
*
B-RE-MS-000016-E-XX-XX-XX-XX-02
6th Year
9k-11k kWh Annual Usage
1.59%
1.24%
B-RE-MS-000017-E-XX-XX-XX-XX-02
6th Year
7k-9k kWh Annual Usage
1.61%
1.25%
6th Year
5k-7k kWh Annual Usage
0.93%
0.73%
7th Year
> 13 kWh Annual Usage
*
*
7th Year
11k-13k kWh Annual Usage
*
*
B-RE-MS-000018-E-XX-XX-XX-XX-02
7th Year
9k-11k kWh Annual Usage
2.58%
2.01%
B-RE-MS-000019-E-XX-XX-XX-XX-02
7th Year
7k-9k kWh Annual Usage
1.98%
1.54%
7th Year
5k-7k kWh Annual Usage
*
*
8th Year
> 13 kWh Annual Usage
*
*
8th Year
11k-13k kWh Annual Usage
*
*
8th Year
9k-11k kWh Annual Usage
2.02%
1.58%
8th Year
7k-9k kWh Annual Usage
*
*
8th Year
5k-7k kWh Annual Usage
*
*
Note: Energy savings quoted as percentage of baseline energy consumption.
* These are the missing values described in Section 1.3. For these values, this whitepaper recommends the utility claim the
last year available in the BRM for that usage band. For example, if a utility would like to claim savings for a wave in Year 7 in
the 5k – 7k kWh band, we recommend using the Year 6 value. These values could be explicitly included in the BRM or a
statement of how to handle them could be included.
Gas Measures
Measure Code
Year
Usage Band
Gas Savings per Customer
1st Year
600-900 Therms Annual Usage
0.27%
B-RE-MS-000020-G-XX-XX-XX-XX-02
1st Year
900-1200 Therms Annual Usage
0.47%
B-RE-MS-000021-G-XX-XX-XX-XX-02
1st Year
>1200 Therms Annual Usage
0.41%
2nd Year
600-900 Therms Annual Usage
0.45%
B-RE-MS-000022-G-XX-XX-XX-XX-02
2nd Year
900-1200 Therms Annual Usage
0.68%
B-RE-MS-000023-G-XX-XX-XX-XX-02
2nd Year
>1200 Therms Annual Usage
0.74%
3rd Year
600-900 Therms Annual Usage
0.51%
B-RE-MS-000024-G-XX-XX-XX-XX-02
3rd Year
900-1200 Therms Annual Usage
0.71%
B-RE-MS-000025-G-XX-XX-XX-XX-02
3rd Year
>1200 Therms Annual Usage
0.73%
4th Year
600-900 Therms Annual Usage
*
B-RE-MS-000026-G-XX-XX-XX-XX-02
4th Year
900-1200 Therms Annual Usage
0.74%
B-RE-MS-000027-G-XX-XX-XX-XX-02
4th Year
>1200 Therms Annual Usage
0.82%
5th Year
600-900 Therms Annual Usage
0.27%
B-RE-MS-000028-G-XX-XX-XX-XX-02
5th Year
900-1200 Therms Annual Usage
0.75%
B-RE-MS-000029-G-XX-XX-XX-XX-02
5th Year
>1200 Therms Annual Usage
1.04%
6th Year
600-900 Therms Annual Usage
*
B-RE-MS-000030-G-XX-XX-XX-XX-02
6th Year
900-1200 Therms Annual Usage
0.73%
B-RE-MS-000031-G-XX-XX-XX-XX-02
6th Year
>1200 Therms Annual Usage
0.69%
7th Year
600-900 Therms Annual Usage
*
B-RE-MS-000032-G-XX-XX-XX-XX-02
7th Year
900-1200 Therms Annual Usage
0.89%
B-RE-MS-000033-G-XX-XX-XX-XX-02
7th Year
>1200 Therms Annual Usage
0.78%
8th Year
600-900 Therms Annual Usage
*
8th Year
900-1200 Therms Annual Usage
*
8th Year
>1200 Therms Annual Usage
0.82%
Note: Energy savings quoted as percentage of baseline energy consumption.
* These are the missing values described in Section 1.3. For these values, this whitepaper recommends the utility claim the last year available in the BRM for that usage band. For example, if a utility would like to claim savings for a wave in Year 4 in the 600 – 900 therms band, we recommend using the Year 3 value. These values could be explicitly included in the BRM or a statement of how to handle them could be included.
5. Baseline and Proposed Improvement Description5.1 Baseline Measure, Practice, or Technology
The baseline measure is a control group customer that does not receive energy usage feedback and benchmarking from a direct mail or email HER.
5.2 Improved Measure, Practice, or Technology
The behavior modification report is an improved measure in that it provides customer-specific energy usage feedback, benchmarking and tips to identify potential savings opportunities, thus altering participant behavior and energy consumption. See Section 4 for specific measure requirements.
6. Methodology and Assumptions
The HER program, implemented by the investor-owned utilities in Michigan, employs a randomized controlled trial (RCT) design that allows for the accurate and unbiased measurement of program impacts. In an RCT design, eligible households are randomly assigned to either a treatment group that receives HERs or a control group that does not receive them. A sound randomization leads to treatment and control groups that are statistically equivalent along observed and unobserved characteristics with the sole difference being that the treatment group receives HERs, and the control group does not. Therefore, any difference in energy usage observed between the treatment and control group can be attributed to the HERs. The RCT program design is summarized in Figure 1.
Figure 1: Experimental Setup for a Randomized Controlled Trial
Experimental design is considered the “gold standard” of program evaluation and an RCT is recognized as an appropriate method for evaluation in the United States Department of Energy’s Uniform Methods Project and in an evaluation protocol report authored by the State and Local Energy Efficiency (SEE) Action Network in 2012. Experimental designs eliminate self-selection bias, wherein customers who choose to join a program are different from those who choose not to join, making the savings estimates from experimental designs known to be unbiased.
The remainder of this section discusses the data used for calibration and then the methodology for the energy and coincident peak demand savings results.
6.1 Data
For this calibration study, Guidehouse included data for over 1.2 million HER participants and just under 400,000 controls across 22 program waves for DTE Energy and Consumers Energy. For DTE Energy, data spanned from 2010 to 2019. For Consumers Energy, data was only included from 2011 to early 2015.[footnoteRef:5] The number of participants and controls in the evaluation of savings from behavior modification varies with the program year and fuel type. Figure 2 shows how many HER participants and controls Guidehouse included in the calibration study for each usage band/program year combination. [5: Newer data was not incorporated for Consumers Energy waves due to a gap in the program in 2015 and then changing implementers between 2016 and present. ]
Figure 2: Summary of Uplift Adjustment Factor
Source: Guidehouse
For demand savings, Guidehouse was only able to include DTE Energy waves that launched after 2016 due to AMI data limitations. The following table shows the waves, years, and customer counts included in the demand calibration.
Wave
Utility
Fuel
Start Date
PY1
PY2
PY3
PY4
Participants*
Controls*
dte_201602_d
Dual
DTE
2/1/2016
Y
Y
Y
Y
18,915
9,255
dte_201602_e
Elec
DTE
2/1/2016
Y
Y
16,830
9,665
dte_201606_d
Dual
DTE
6/1/2016
Y
Y
Y
Y
9,188
6,453
dte_201606_e
Elec
DTE
6/1/2016
Y
Y
19,555
12,739
dte_201710_d
Dual
DTE
10/1/2017
Y
Y
27,348
9,928
dte_201803_d
Dual
DTE
3/1/2018
Y
Y
25,676
11,789
dte_201901_d
Dual
DTE
1/1/2019
Y
23,296
9,750
* Counts represent unique number of customers included in regression analysis for demand savings across all years.
Source: Guidehouse
6.2 Energy Savings
For calibration, the energy savings resulting from the HER program were calculated using a regression-based ex post evaluation grouping individual program waves together into a single regression for each usage band and program year.[footnoteRef:6] These regressions compare participant energy use to a counterfactual, baseline usage scenario in the absence of the program. In the case of an RCT, baseline usage is determined by the control group. The estimates are then annualized and adjusted to account for savings overlap. [6: As noted in Section 2, this whitepaper is not an evaluation guidance document for custom evaluation of behavior modification measures. Evaluation of individual program waves (as opposed to usage band groups) may utilize different regression specifications.]
6.2.1 Regression Analysis
Guidehouse used a lagged dependent variable (LDV) model to estimate daily per participant savings from HERs, by usage band and program year, using monthly billing data. Formally, the model is:
Where,
is the average daily usage for household i during month t
is the average daily usage for household i during the same month as time t lagged to the pre-program year[footnoteRef:7] [7: For DTE’s leading wave (which launched in July 2011) data was missing for more than 90% of the wave’s customers in the first three months of the pre-period (July-September 2010). To address the missing data, Guidehouse imputed pre-period data for this wave (but no other waves). Imputing the pre-period data involved assigning each customer missing a given pre-period usage observation the average value for all customers who have data in the same month of the pre-period. For example, all customers missing usage data in July 2010 were assigned the average for all customers with data for that month.]
comprise a set of month-of-year indicators, which equal 1 if t falls in month-of-year j, and 0 otherwise
is a binary variable that equals 1 if household i is in the treatment group and 0 otherwise
is the number of cooling degree days for household i during month t
is the number of heating degree days for household i during month t
comprise a set of wave indicators, which equal 1 if household i is in wave w and 0 otherwise
is a binary variable that equals 1 if household i is a DTE customer and 0 if a Consumers Energy customer
is the cluster-robust error term for household i in time t
New measure submissions to the BRM for the Behavioral Modifications: Home Energy Reports measure should use an LDV model specification.
6.2.2 Annualizing Savings
To get one annualized savings value, average daily savings is multiplied by 365 days.
6.2.3 Potential Savings Overlap (or Uplift Adjustment)
HERs may increase participation in other utility energy efficiency programs; this additional participation is known as efficiency program uplift and could result in double counted savings if not addressed. To avoid double counting savings, the savings from program uplift are subtracted from behavior modification report savings and attributed to the lifted program.
Chapter 17 of the Uniform Methods Project protocols identifies the following three steps for measuring savings overlap from programs with individual customer tracking data[footnoteRef:8] using a difference-in-difference (DID) statistic: [8: Guidehouse also considered uplift into upstream lighting programs, but evidence from 2018 and 2019 DTE Energy HER program surveys suggests that treatment and control customers purchase LED bulbs at almost identical rates. This evidence indicates that no upstream lighting uplift adjustment is necessary.]
1. Match the [behavior-based] program treatment and control group subjects to the utility energy efficiency program tracking data.
2. Calculate the uplift savings per treatment group subject as the difference between treatment and control groups in average efficiency program savings per subject, where the savings are obtained from the utility tracking database of installed measures. (The averages should be calculated over all treatment group subjects and all control group subjects, not just those who participated in efficiency programs.)
3. Multiply the uplift savings per treatment group customer by the number of subjects who were in the treatment group to obtain the total uplift savings. (pg. 37)
The general process for removing double counted savings from programs with individual customer tracking data from the savings estimate out of the regression analysis is described in Figure 3. The green shapes outline the steps for the regression analysis, the yellow shapes outline the steps for the double counting analysis. The equation at the bottom of the figure shows that double counted savings are removed from savings coming out of the regression analysis to get the savings value without double counted savings which is recommended for the BRM.
Figure 3: Double Counting Adjustment Process for Programs with Individual Customer Tracking Data
Source: Guidehouse
The calibration study calculated uplift for all DTE Energy waves using a difference-in-difference (DID) statistic based on pre-period and post-period average savings from other energy efficiency programs with individual customer tracking data.[footnoteRef:9] These estimates were combined with uplift estimates from the prior Consumers Energy evaluations[footnoteRef:10] and weighted by the proportion of participants from each utility. [9: Estimates were based on evaluations of program tracking data for all programs and program years where data was available. The other program savings values were adjusted for an installation rate adjustment factor (IRAF) and a net-to-gross ratio (NTGR) to reflect the fact that HER savings produced by the regression inherently account for these items.] [10: The program uplift estimates for Consumers Energy were based on a simple difference between treatment and control participation rates (not savings) during the program year.]
Guidehouse only accounted for cases of positive uplift (i.e., where HER increased participation in other programs) for a given wave and program year. Cases where uplift is negative (i.e., where HER decreased participation in other programs) did not lead to an adjustment to the savings. There is no issue of double counting savings at the portfolio level for negative uplift which is why it was not adjusted for.[footnoteRef:11] [11: The issue of negative uplift is a baseline issue. If negative uplift occurs and you are trying to estimate savings for the HER program in the absence of all other programs, negative uplift should be added back to the HER savings to account for the fact that the control customers’ usage (which represents the baseline) is too low. However, when estimating HER while considering all of the other programs, this adjustment is not appropriate.]
Figure 4 shows the uplift adjustment found in this calibration study and the equation for adjusting the regression-savings.
Figure 4: Uplift Adjustment Factor
Source: Guidehouse
6.3 Coincident Peak Demand Savings
6.3.1 Regression Analysis
Guidehouse estimated demand savings using the same regression model presented in Section 6.2.1 above but including only coincident peak data.[footnoteRef:12] Guidehouse then estimated an analogous energy savings model using the same HER waves and program years. [12: The coincident peak is defined as 3-6pm on the three hottest, consecutive, non-holiday weekdays in July.]
6.3.2 Potential Savings Overlap (or Uplift Adjustment)
For coincident peak demand savings, Guidehouse excluded customers in DTE Energy’s demand response program to avoid double counting with demand response event savings.
6.3.3 Demand Savings Factor
A demand savings factor was calculated based on the following equation.[footnoteRef:13] [13: The energy and demand savings in this equation are after adjusting for uplift as described in Section 6.2.3 and 6.3.2.]
Guidehouse found a demand savings factor of 0.78.
The reported coincident peak demand savings are calculated by multiplying the energy savings rate by the demand savings factor as shown in the following equation.
7. Estimated Savings over Baseline
This section provides information and formulas to calculate energy and coincident peak demand savings for HERs, along with an illustrative example.
Estimating energy savings requires the calculation of three input values:
· Average Annual Energy Usage – calculated using billing data from the 12-month program year prior to the program year for which savings are being estimated for customers randomly assigned to the HER control group.[footnoteRef:14] [14: For example, if savings are being claimed for PY2019, average annual energy usage would be calculated using control customers from January to December 2018.]
· Savings Rate – Average Annual Energy Usage along with year[footnoteRef:15] determine the appropriate BRM percent savings rate to be applied for both energy savings and coincident peak demand savings. [15: Since savings are annualized, the year a wave launches (regardless of the month) is year 1, the next year would be year 2, and so on. For example, if a wave launched in June 2017, PY2017 would be year 1, PY2018 would be year 2, PY2019 would be year 3, etc. ]
· Number of Active Participants – the number of HER participants that had an active utility account at the premise where they receive HERs as of the end of the program year.[footnoteRef:16] [16: Note that participant households who opt out of the HER program should be included in this count as they are included in determining savings to preserve the validity of the RCT.]
Estimating coincident peak demand savings requires one additional input:
· Average Coincident Peak Demand – calculated using hourly demand data during the coincident peak from July of the program year prior to the program year for which savings are being estimated for customers randomly assigned to the HER control group.[footnoteRef:17] [17: For example, if savings are being claimed for PY2019, average coincident peak demand would be calculated using control customers during 2018’s coincident peak.]
These values are used to estimate savings using the following equations:
Example Savings Calculation
Consider a dual fuel HER wave with 10,000 active participants in the first program year, average annual electric usage of 8,000 kWh, 5 coincident peak kW and 1,300 Therms for the control group in the prior program year.
· Electric Energy Savings are:
576,000 kWh = 8,000 kWh * 0.72% * 10,000
· Coincident Peak Demand Savings are:
280 kW = 5 kW * 0.56% * 10,000
· Gas Energy Savings are:
53,300 Therms = 1,300 Therms * 0.41% * 10,000
8. Potential Savings Overlap
Refer to Sections 6.2.3 and 6.3.2.
9. Coincidence Factor
The rate of coincident peak savings is 1 for all measures in this whitepaper.[footnoteRef:18] [18: The demand savings factor of 0.78 was determined by calculating the ratio of coincident peak demand savings relative to energy savings during the coincident peak period. As a result, the demand savings rate (calculated as the energy savings rate x 0.78) represents coincident peak demand savings rate. As such, the coincidence factor is 1.0.]
10. Measure Life
The measure life is 1 year for all measures in this whitepaper.
11. Measure Cost
$6.91 per measure (i.e., cost to deliver approximately four print HERs to one participant for one year)
12. Any Recurring Cost
There are no recurring costs associated with this measure.
13. Relevant Codes and Standards
There are no relevant codes or standards associated with this measure.
14. Ongoing EM&V, Research, and Calibration Planning
· Year 9 savings to be re-submitted by Oracle and DTE in accordance with new modelling guidelines from the Michigan Public Service Commission.
· Year 10 savings to be submitted by Oracle and DTE for 2022 BRM.
· In the absence of significant changes in program design or delivery, savings values for Year 6 and up should be calibrated in 3-4 years. Significant changes in program design or delivery may warrant calibration sooner or covering more years.
15. Sources of Information
· U.S. Department of Energy, “Chapter 17: Residential Behavior Protocol,” in The Uniform Methods Project: Methods for Determining Energy Efficiency Savings for Specific Measures, 2015. Accessed 22 March 2017, https://energy.gov/sites/prod/files/2015/02/f19/UMPChapter17-residential-behavior.pdf.
· SEE Action Network, “Evaluation, Measurement, and Verification (EM&V) of Residential Behavior-Based Energy Efficiency Programs: Issues and Recommendations,” 2012. Accessed 22 March 2017, https://www4.eere.energy.gov/seeaction/system/files/documents/emv_behaviorbased_eeprograms.pdf.
16. Attachments
· Attachment A: “Home Energy Report Measure Calibration”, presentation given to Michigan Public Service Commission on October 5, 2020
· File Name: Attachment_A_MI BRM Calibration Results 2020-10-05 - Presentation to MPSC.pdf
Attachment_A_MI
BRM Calibration Results 2020-10-05 - Presentation to MPSC.pdf
Home Energy Report Measure CalibrationPresentation to the
Michigan Public Service Commission
October 5, 2020
1©2020 Guidehouse Inc. All Rights Reserved
1. Measure Description Slide 2
2. Motivation Slide 3
3. Methodology Slide 4
4. Data Slide 11
5. Model Comparison Results Slide 15
6. BRM Savings Value Recommendations Slide 20
7. Next Steps Slide 25
8. Appendices Slide 27
Agenda
2©2020 Guidehouse Inc. All Rights Reserved
HERs change energy use behavior1 through two primary
mechanisms:
1. Motivates residential customers through normative messaging to
change their behavior. Personalized neighbor comparisons based
on home size, location and energy type—among other criteria—
give households a motivational benchmark for their energy usage.
2. Provides residential customers with salient, personalized advice
to capitalize on this motivation to use less energy and save
money.
HERs are delivered through direct mail and are often supplemented
with digital communications such as email, the web, telephones,
mobile phones, and social networks. This platform approach ensures
all households have access to the information.
1. Measure DescriptionHome Energy Reports (HERs) seek to achieve energy savings by providing households accurate
monthly electric and/or gas usage information, motivating a change in energy use behavior.
Figure 1. Sample Home Energy Report
Source: DTE Energy’s HER Program Implemented by Oracle 1 Allcott, H. Social norms and energy conservation. Journal of Public Economics (2011), Volume 95, Issues
9-10: 1082-1095.
3©2020 Guidehouse Inc. All Rights Reserved
The primary drivers of conducting calibration
research on the HER measure at this time are:
– These measures have not been calibrated
since 2017 at which time values for Year 1 through
Year 5 were calibrated using data from all DTE
Energy (DTE) and Consumers Energy (CE)1
commercialized waves, with a few exceptions.2
– Since that time, savings values for Years 6-9 have
been added or proposed for inclusion in the BRM
based on results from DTE’s 2011 pilot wave.
– The peak demand savings factor in the BRM has
never been calibrated. It is currently 1.5 times
energy savings.
2. MotivationThe HER measure in the Behavior Resource Manual (BRM) was last calibrated using data through
2016.
1 Oracle’s file names abbreviate Consumers Energy as CMS so this appears
in their wave naming conventions throughout this deck.2 Year 5 9-11k kWh and Year 5 >1200 therms were not calibrated in the prior
study due to insufficient data. The DTE_201504_D wave was not included in
the gas calibration due to usage being below the 900 CCF cutoff. The
CMS_201204_E_BC, CMS_201203_G, and CMS_201403_D waves were
not included in the study due to the lack of zip code data for these waves.
Fuel
Type
Usage
Band
Year
1
Year
2
Year
3
Year
4
Year
5
Year
6
Year
7
Year
8
Year
9
Elec5k-7k
kWh0.70% 0.77%
Elec7k-9k
kWh0.84% 1.50% 1.82% 1.25% 2.01% 1.70% 2.20% * *
Elec9k-11k
kWh1.08% 1.52% 1.77% 2.27% 2.18% 2.12% 2.73% * *
Elec>11k
kWh1.20% 1.78%
Gas900-1200
Therms0.34% 0.53% 0.91% 0.86% 0.66% 0.73% 0.82% * *
Gas>1200
Therms0.43% 0.60% 0.57% 0.66% 1.09% 0.73% 0.82% * *
Table 1. Current BRM Usage Bands and Savings Rates
*Values have been proposed by DTE/Oracle for these usage bands.
Source: MI BRM
4©2020 Guidehouse Inc. All Rights Reserved
3. MethodologyThis calibration study estimates energy savings by usage band and program delivery year (i.e., Year
1, Year 2) using monthly billing data for the DTE (between 2010 and 2019) and CE (between 2011
and early 2015)1 programs offering HERs.2
1 New data was not incorporated for CE waves due to a gap in the program in 2015 and then changing implementers between 2016 and present. 2 Each HER wave/program year included in the calibration analysis had 12 months of post-program data available for the applicable program year.3 State and Local Energy Efficiency (SEE) Action Network. 2012. Evaluation, Measurement, and Verification (EM&V) of Residential Behavior-Based Energy Efficiency Programs: Issues and
Recommendations. Prepared by A. Todd, E. Stuart, S. Schiller, and C. Goldman, Lawrence Berkeley National Laboratory. http://behavioranalytics.lbl.gov.
Stewart, J. and A. Todd. Chapter 17: Residential Behavior Evaluation Protocol. In NREL’s Uniform Methods Project Protocols.
Note: Complete model specifications are shown in Appendix A.
DTE and CE implement their HER programs as randomized controlled trials (RCTs), wherein customers are randomly assigned to treatment and control groups (Figure 2). This program design is known to produce unbiased estimate of program impacts.2
Because customers are randomly assigned into a treatment group or a control group, they are expected to be equivalent in every way expect program treatment - in this case, receipt of the report. As such, any differences in usage between the treatment group and the control group observed in the program period are necessarily the result of the program.
Source: SEE Action Report3
Figure 2. Illustration of an RCT
5©2020 Guidehouse Inc. All Rights Reserved
3. MethodologyGuidehouse estimated savings using both a lagged dependent variable (LDV) model and a linear
fixed effect regression (LFER) to inform which model class should be used going forward.
LDV and LFER are two model classes used for evaluating savings for HER programs.
• Both are endorsed by the Uniform Methods Project (UMP)1 and, because of the RCT, are expected to produce unbiased
savings estimates.
• The LDV model is more commonly used by evaluators for claiming savings, although most evaluators estimate multiple
models for robustness and investigate if the models are not producing consistent results.
Each of these model classes account for naturally occurring randomization imbalances that lead to differences in usage
between the treatment and control groups in the pre-program period (i.e., before reports are sent), but they do so in different
ways. In a balanced RCT, these two model classes are expected to result in unbiased and similar (i.e., not statistically
different) results.
Guidehouse estimated models both for (1) each individual wave/program year to test differences between the two models,
and (2) for each usage band/program year (combining individual waves in the same usage band into one model) for
calibrating the BRM.
1 Stewart, J. and A. Todd. Chapter 17: Residential Behavior Evaluation Protocol. In NREL’s Uniform Methods Project Protocols.
6©2020 Guidehouse Inc. All Rights Reserved
𝐴𝐷𝑈𝑖𝑡= 𝜶𝒊 + 𝛽1𝑃𝑜𝑠𝑡𝑡 ∙ 𝑇𝑟𝑒𝑎𝑡𝑚𝑒𝑛𝑡𝑖 +
𝐽
𝛽2𝑗𝑌𝑟𝑀𝑜𝑗𝑡 + 𝛽3𝐶𝐷𝐷_𝑎𝑛𝑑_𝑜𝑟_𝐻𝐷𝐷𝑖𝑡
+𝛽4𝑃𝑜𝑠𝑡𝑡 + 𝜀𝑖𝑡
• LFER controls for differences by including a customer-specific fixed
effect (a single value for each customer) which controls for all time
invariant observable and unobservable characteristics.
• The fixed effect does not control for time varying characteristics (for
example, if usage is highly seasonal).
3. MethodologyEquation 1 and 2 show the model specifications Guidehouse estimated for this calibration study.
𝐴𝐷𝑈𝑖𝑡=
𝐽
𝛽1𝑗𝑌𝑟𝑀𝑜𝑗𝑡 +
𝐽
𝛽2𝑗𝑌𝑟𝑀𝑜𝑗𝑡 ∙ 𝑨𝑫𝑼𝒍𝒂𝒈𝒊𝒕 + 𝛽3𝑇𝑟𝑒𝑎𝑡𝑚𝑒𝑛𝑡𝑖
+ 𝛽4𝐶𝐷𝐷_𝑎𝑛𝑑_𝑜𝑟_𝐻𝐷𝐷𝑖𝑡 +
𝑊
𝛽5𝑊𝑎𝑣𝑒𝑖𝑤 + 𝛽6𝑈𝑡𝑖𝑙𝑖𝑡𝑦𝑖 + 𝜀𝑖𝑡
• LDV controls for differences between the treatment and control
groups by including lagged usage (from the pre-period) as an
explanatory variable.
• The lagged usage does a good job of controlling for differences in
usage over time.
• Time invariant customer characteristics must be explicitly added to
the model to be accounted for. With a RCT these characteristics
are expected to be well-balanced between the treatment and
control groups.
Equation 1. LDV Model1 Equation 2. LFER Model2
1 This is the LDV specification used for this calibration study; it is one example within the broader class of LDV models. Note 𝛽5 and 𝛽6 are included in pooled specifications (where different waves are modelled together) but are not included in single wave regressions as all customers would have the same value for wave and utility.2 This is the LFER specification used for this calibration study; it is one example within the broader class of LFER models. Note 𝛽4 is included in pooled specifications (where different waves are modelled together) but is not included in single wave regressions as Post is perfectly collinear with the year-month dummies in 𝛽2 when waves are modelled individually.
7©2020 Guidehouse Inc. All Rights Reserved
3. MethodologyThe figures below illustrate how the lagged usage and the fixed effect work for one individual
customer. Note, that both models produce an unbiased estimate of savings.
Figure 3. LDV Model Illustration Figure 4. LFER Model Illustration
Source: Guidehouse
0
0.5
1
1.5
2
2.5
3
3.5
4
4.5
5
201
9-0
1
201
9-0
2
201
9-0
3
201
9-0
4
201
9-0
5
201
9-0
6
201
9-0
7
201
9-0
8
201
9-0
9
201
9-1
0
201
9-1
1
201
9-1
2
202
0-0
1
202
0-0
2
202
0-0
3
202
0-0
4
202
0-0
5
202
0-0
6
202
0-0
7
202
0-0
8
202
0-0
9
202
0-1
0
202
0-1
1
202
0-1
2
Pre Post
Ave
rage
Da
ily U
sage
(T
he
rms)
Avg Daily Use (Therms) Fixed Effect
Note, the LDV model only includes unique observations of the dependent variable (average
daily usage) for the post period and the pre-period enters as an independent variable (lagged
usage). Thus, in estimating one year of savings, the model includes 12 observations.
Note, the LFER model includes unique observations of the dependent variable (average
daily usage) for both the pre and post periods. Thus, in estimating one year of savings,
the model includes 24 observations.
0
0.5
1
1.5
2
2.5
3
3.5
4
4.5
5
202
0-0
1
202
0-0
2
202
0-0
3
202
0-0
4
202
0-0
5
202
0-0
6
202
0-0
7
202
0-0
8
202
0-0
9
202
0-1
0
202
0-1
1
202
0-1
2
Post
Ave
rage
Da
ily U
sage
(T
he
rms)
Avg Daily Use (Therms) Lagged Usage
8©2020 Guidehouse Inc. All Rights Reserved
3. MethodologyHERs may increase participation in other energy efficiency programs (also referred to as program
uplift). To avoid double-counting, the savings associated with program uplift are subtracted from the
HER program and attributed to the lifted program measures.
Figure 5. Report Savings Calculation
1Stewart, J. and A. Todd. Chapter 17: Residential Behavior Evaluation Protocol. In NREL’s
Uniform Methods Project Protocols.2 For example, HER treatment customers may participate in a weatherization program at the
same rate as HER controls, but when they participate, they may install more insulation leading
to higher savings.
Report Savings w/
Double Counted
Savings
Downstream
Double Counting
Adjustment
Factor
Report Savings w/out
Double Counted
Savings
Reg.
Savings
Estimate5.7%
BRM
Reported
Savings
Elec
Gas
Reg.
Savings
Estimate9.4%
BRM
Reported
Savings1
1
• The calibration study calculated uplift for all DTE waves using a
difference-in-difference (DID) statistic based on pre-period and
post-period average savings from other energy efficiency
programs. These estimates were combined with uplift estimates
from the prior CE evaluations and weighted by the proportion of
participants from each utility.
– This approach differs from the method used in the prior
calibration study in that:
1. It measures uplift associated with increased net savings
within a program rather than just increased participation in
accordance with UMP chapter 17.1,2
2. It removes estimates of negative uplift for a given wave /
program year, as negative uplift is a baseline issue rather
than a double counting issue.
• Compared to the prior calibration study uplift increased from
0.17% for electric and 5.04% for gas.
Source: Guidehouse
9©2020 Guidehouse Inc. All Rights Reserved
3. MethodologyMichigan-specific data, as well as secondary data from other jurisdictions, does not provide
evidence of lift in upstream lighting.
1 Ohio is conducting a study looking into upstream lighting uplift, but results are not expected
until late 2020.2 See Appendix B for references to all the studies shown in notes 3-8.
Jurisdiction1,2 Approach
National Grid
New York
Conducted a survey in 2014 in which they found near-zero
joint savings with upstream lighting that were not statistically
significant.3
PSE Applies a value based on a 2015 survey of treatment/control.
The evaluation estimates uplift for LEDs of 2.5-15 kWh per
household (results are not statistically significant).4
PG&E Calculates an adjustment value for LEDs by wave, based on
surveys of treatment/control. The 2015 evaluation estimated
uplift of 0-1 kWh per household for LEDs.5
SDG&E and
SCE
Use adapted versions of PG&E’s results.6, 7
Pennsylvania Adopted an adjustment factor into the TRM based on the
PSE (2015) and PG&E (2013) studies, and professional
judgement by the statewide evaluator. Year 1 adjustment
factor is 0.75% increased by 0.75 percentage points up to
Year 4.8
Maryland Adjust based on the Pennsylvania TRM.
• 2018 and 2019 surveys for DTE’s HER program show that
treatment and control customers purchase LED bulbs at almost
identical rates suggesting no change in participation in upstream
lighting because of the reports (Appendix B).
• Relatively few jurisdictions/utilities account for upstream lighting.
Of those that do (Table 2), adjustments are based on differences in
bulb purchases between treatment/control groups. HERs are not
found to consistently result in more bulb purchases in treatment
customers relative to the control group across these studies.
• In addition, upstream lighting is expected to make up less of the
utility’s portfolios in the coming years as the baseline shifts
reducing the potential for lift in upstream lighting.
Table 2. Upstream Lighting Approaches
Source: Guidehouse literature review
10©2020 Guidehouse Inc. All Rights Reserved
• Due to AMI data limitations, calibration of demand savings only included DTE waves launched in 2016 or later.
– Uplift for DTE’s demand response program was accounted for in these estimates by excluding customers in the demand response
programs.
• Guidehouse estimated a demand savings regression model for all waves with available AMI data and an analogous energy
savings regression model for the same waves. These regression specifications are included in Appendix A.
– The demand savings model only includes data for 3-6pm on the three hottest, consecutive, non-holiday weekdays in July.
• We calculated a demand savings factor by comparing the demand savings to the energy savings using the following
equation:
Demand Savings Factor =% 𝐷𝑒𝑚𝑎𝑛𝑑 𝑆𝑎𝑣𝑖𝑛𝑔𝑠
% 𝐸𝑛𝑒𝑟𝑔𝑦 𝑆𝑎𝑣𝑖𝑛𝑔𝑠
• The resulting demand savings factor can replace the current 1.5x demand savings factor used for all waves. The waves
used for demand calibration cover all but the highest usage band (>13 MWh).
3. MethodologyDemand savings have not been previously calibrated in the BRM and there is currently an assumed
1.5x demand savings factor. This study used available data to calibrate that demand savings factor.
Note: Due to data limitations the calibration study is unable to determine whether the demand savings factor varies by usage band. As a result, we assume a single demand savings factor for all
waves.
11©2020 Guidehouse Inc. All Rights Reserved
• In scoping this study, a question arose of whether older
program waves savings values were pertinent to the
current/future program given changes in design and time
period.
• Guidehouse found that, within a given usage band, savings for
older waves were neither systematically higher or lower than
newer waves.
• Guidehouse also reviewed the report layout and content and
found that although it has changed some over time the
information is largely similar.
• Report frequency has changed with print reports becoming
less frequent and email reports more frequent, but without
systematic changes in savings we didn’t conclude that this
warrants wave exclusion.
– DTE waves average between 3 and 6 print reports per year.
• Based on this review, Guidehouse included all waves with
data in a given year in our calibration.
4. DataGuidehouse included data from 1,256,027 treatment customer and 378,997 control customers across
22 program waves for DTE (spanning 2010-2019) and CE (spanning 2011-early 2015).
Waves Included in 2017
Calibration StudyNew Waves
CMS_201105_D
CMS_201203_D
CMS_201204_E_MUSK
CMS_201303_E
DTE_201107_D
DTE_201309_D
DTE_201309_E
DTE_201401_D
DTE_201401_E
DTE_201504_D
DTE_201504_E
DTE_201602_D*
DTE_201602_E*
DTE_201602_G
DTE_201606_D*
DTE_201606_E*
DTE_201610_G
DTE_201710_D*
DTE_201710_G
DTE_201711_G
DTE_201803_D*
DTE_201803_G
DTE_201901_D*
Table 3. Waves Included in Calibration1
* Wave included in demand savings factor calibration. Due to AMI data availability, only
DTE electric customers in waves starting after 2016 were included.1 Appendix D has more information about the included waves including customer counts
and program years included. This appendix also includes information about which
waves were excluded and why.Note: More information can be found in Appendix C.
Source: Guidehouse analysis
12©2020 Guidehouse Inc. All Rights Reserved
4. DataIn addition to calibrating savings based on existing BRM usage bands, this study also introduces
usage bands of greater than 13 MWh (breaks the current high end of >11 MWh into 11-13 and >13
MWh instead), and 600-900 Therms. Not all deemed savings values will be calibrated.
Fuel Usage Band
Electric 5 to 7 MWh
Electric 7 to 9 MWh
Electric 9 to 11 MWh
Electric > 11 MWh
Existing BRM
Usage BandsProposed BRM
Usage Bands
Table 4. Existing and Proposed Usage Bands
Fuel Usage Band
Gas 900-1200 Therms
Gas >1200 Therms
Fuel Usage Band
Electric 5 to 7 MWh
Electric 7 to 9 MWh
Electric 9 to 11 MWh
Electric 11 to 13 MWh
Electric > 13 MWh
Fuel Usage Band
Gas 600-900 Therms
Gas 900-1200 Therms
Gas >1200 ThermsSource: Guidehouse analysis
• Guidehouse proposes splitting the highest electric usage band
currently in the BRM (>11 MWh) in two bands (11-13 MWh and
>13 MWh).
– Both bands are large enough to always produce statistically
significant savings estimates (i.e., statistically different from
zero with 90% confidence).
– The >13 MWh band has higher savings than the 11-13 MWh
band in each program year where both bands have savings
estimates. The savings are or are close to statistically
different between the two bands in most years.
• Guidehouse proposes adding a lower usage band for gas from
600-900 therms.
– Four waves fall into this band and we see that they produce
savings that are statistically different from zero.
13©2020 Guidehouse Inc. All Rights Reserved
4. DataBetween DTE and CE, Guidehouse included 22 waves across 8 program years in the calibration
study.
Table 5. Number of Waves per Usage Band by Year1
Fuel Usage Band
Year
1
Year
2
Year
3
Year
4
Year
5
Year
6
Year
7
Year
8
Year
9
Electric 5 to 7 MWh 3 2 1 1 1 1
Electric 7 to 9 MWh 4 6 3 2 2 * 1 * *
Electric 9 to 11 MWh 5 5 5 4 1 3 * 1 *
Electric 11 to 13 MWh 4 2 2 1 1
Electric > 13 MWh 3 3 1 1 1 1
Gas 600 to 900 Therms 3 4 1 1
Gas 900 to 1200 Therms 8 7 6 5 3 3 1 * *
Gas > 1200 Therms 5 4 3 3 * * * 1 *
* These are deemed in the BRM (or have been proposed by DTE/Oracle) but there is not enough data to
calibrate them in this study.
1Refer to Appendix D for additional information on specific DTE and CE cohorts included in the calibration study.
Source: Guidehouse analysis
14©2020 Guidehouse Inc. All Rights Reserved
4. DataThe earlier years have more customers in the calibration than the later years. The savings estimates
for program year/band combinations with fewer customers are less precise.
Figure 6. Customer Counts by Usage Band and Program Year
Source: Guidehouse analysis
15©2020 Guidehouse Inc. All Rights Reserved
• Both the individual wave/program year regressions
and the pooled usage band/program year regressions
do not exhibit statistically significant differences
between the two model specifications.
• The point estimates of savings can be practically
different even if they are not statistically different.
There is no way to know which estimate is “correct”
as they are both unbiased estimates of savings.
• Importantly, we see that neither model is
systematically higher or lower than the other (51%
higher for LDV), such that choosing one model will
result in higher savings for some wave/program years
and lower savings for others.
• We see that the LDV model has slightly more precise
estimates than the LFER model. The difference is
about 5%.
5. Model Comparison ResultsGuidehouse found the point estimates of electric savings were never statistically different between
the LDV and LFER models.
Figure 7. LDV vs. LFER Electric Savings by Wave and Program Year
Source: Guidehouse analysis
Note, each data point in the figure represents the savings estimate (LDV, LFER) for one
wave/program year. As such, the diagonal line represents cases where the LDV and
LFER savings are identical.
-0.2
0
0.2
0.4
0.6
0.8
1
1.2
-0.2 0 0.2 0.4 0.6 0.8 1 1.2
LF
ER
Sa
vin
gs (
kW
h/d
ay)
LDV Savings (kWh/day)
16©2020 Guidehouse Inc. All Rights Reserved
• Similar to electric, both the individual wave/program year
regressions and the pooled usage band/program year
regressions do not exhibit statistically significant
differences between the two models.
• The LDV model is higher than the LFER model more
frequently for gas. However, most of the unbalanced
estimates belong to DTE’s leading wave (201107).
– This aligns with results from Oracle showing large
differences between the LDV and LFER models for
this wave.
• We see that the LDV model has slightly more precise
estimates than the LFER model. The difference is about
5% to 10%.
5. Model Comparison ResultsGuidehouse found the point estimates of gas savings were never statistically different between the
LDV and LFER models.
Figure 8. LDV vs. LFER Gas Savings by Wave and Program Year
Source: Guidehouse analysis
Note, each data point in the figure represents the savings estimate (LDV, LFER) for one
wave/program year. As such, the diagonal line represents cases where the LDV and
LFER savings are identical.
0
0.01
0.02
0.03
0.04
0.05
0 0.01 0.02 0.03 0.04 0.05
LF
ER
Sa
vin
gs (
the
rms/d
ay)
LDV Savings (therms/day)
dte_201107_d
17©2020 Guidehouse Inc. All Rights Reserved
• Guidehouse’s LDV and LFER handle the missing pre-period data in different ways such that the two models end up using different data sets
to estimate savings.
– The LDV model drops data in the post-period if the pre-period data is missing. This means LDV is using only 9 months of post data to estimate savings.
– The LFER model only drops the pre-period data. This means LFER is correctly using the full 12 months of post data to estimate the savings.
5. Model Comparison Results – DTE 2011 WaveGuidehouse determined a primary cause of the differences for the DTE 201107 was missing pre-
period data (more than 90% of customers were missing Jul-Sep 2010 data).
0
0.5
1
1.5
2
2.5
3
3.5
4
4.5
201
2-0
7
201
2-0
8
201
2-0
9
201
2-1
0
201
2-1
1
201
2-1
2
201
3-0
1
201
3-0
2
201
3-0
3
201
3-0
4
201
3-0
5
201
3-0
6Post
Ave
rage
Da
ily U
sa
ge
(T
he
rms)
Avg Daily Use (Therms) Lagged Usage
0
0.5
1
1.5
2
2.5
3
3.5
4
4.5
201
0-0
7
201
0-0
8
201
0-0
9
201
0-1
0
201
0-1
1
201
0-1
2
201
1-0
1
201
1-0
2
201
1-0
3
201
1-0
4
201
1-0
5
201
1-0
6
201
2-0
7
201
2-0
8
201
2-0
9
201
2-1
0
201
2-1
1
201
2-1
2
201
3-0
1
201
3-0
2
201
3-0
3
201
3-0
4
201
3-0
5
201
3-0
6
Pre Post
Ave
rage
Da
ily U
sage
(T
he
rms)
Avg Daily Use (Therms) Fixed Effect
Figure 9. LDV Model Illustration w/ Missing Pre-Period Data Figure 10. LFER Model Illustration w/ Missing Pre-Period Data
Source: Guidehouse
18©2020 Guidehouse Inc. All Rights Reserved
5. Model Comparison Results – DTE 2011 WaveTo address the missing data, Guidehouse imputed pre-period data for the LDV model for the DTE
201107 wave so each model could estimate savings using 12 months of post-period data. Figure 12
shows LDV (with imputed data for DTE 201107) and LFER (with original data) are more similar.
Source: Guidehouse analysis
Figure 11. Original Data Figure 12. Missing Pre-Data Imputed for
LDV Model for DTE 201107*
-0.2
0
0.2
0.4
0.6
0.8
1
-0.2 0 0.2 0.4 0.6 0.8 1LF
ER
Savin
gs (
kW
h/d
ay)
LDV Savings (kWh/day)
0
0.01
0.02
0.03
0.04
0.05
0 0.01 0.02 0.03 0.04 0.05LF
ER
Savin
gs (
therm
s/d
ay)
LDV Savings (therms/day)
* Note in these graphics only the LDV model for the DTE
201107 wave (in orange) is re-estimated. The LFER model for
this wave does not change. All other waves (green dots) also
remain identical to the original.
Ele
ctr
icG
as
• For the LDV model, imputing the pre-
period data involved assigning each
customer missing a given pre-period
usage observation the average value for
all customers who have data in the same
month of the pre-period.
– For example, all customers missing
usage data in July 2010 were assigned
the average for all customers with data
for that month.
• Imputing data for LDV brings the models
closer together although there are still
some differences.
– These are not statistically significant
differences and in practical terms we
are discussing hundredths of therms
per day.
-0.2
0
0.2
0.4
0.6
0.8
1
-0.2 0 0.2 0.4 0.6 0.8 1LF
ER
Savin
gs (
kW
h/d
ay)
LDV Savings (kWh/day)
0
0.01
0.02
0.03
0.04
0.05
0 0.01 0.02 0.03 0.04 0.05LF
ER
Savin
gs (
therm
s/d
ay)
LDV Savings (therms/day)
19©2020 Guidehouse Inc. All Rights Reserved
5. Model Comparison ResultsTable 6 shows options to move forward with the modelling for the DTE 201107 wave. Both options
result in an unbiased estimate of annual savings.
Options Implication for BRM Pros Cons
1 LFER, original
data for all
waves
LFER would be selected as the model
approach for calibration and all new measure
submissions
- Does not require any imputation of
missing data
- Achieves consistency in data cleaning
steps across all waves (i.e., no data
cleaning exceptions/additions are required
for DTE 201107 with this approach)
- Results in unbiased standard errors for
all waves
- Would achieve a result in which the model class
used (LFER) is inconsistent with most other
jurisdictions
- Across the program, standard errors are higher
for the LFER model than the LDV (not
considering imputed data)
2 LDV, imputed
data for DTE
201107
LDV would be selected as the model approach
for calibration and all new measure
submissions
Pre-period data would be imputed for the
missing portion of the DTE 201107 wave, but
would not be for other waves. BRM would
specify how to do this
- Moves BRM to using LDV model class
rather than LFER model class aligning with
most other jurisdictions
- Across the program, standard errors are
lower for the LDV model than for LFER
(not considering imputed data)
- Imputing data biases the standard error
downward (i.e., implies the estimate is more
precise than it is) for the DTE 201107 wave by
giving the model more information than we
actually have
- Requires additional data processing steps of
imputing data for DTE 201107 wave relative to all
other waves
Table 6. Model Options for DTE 201107 Wave
Source: Guidehouse analysis
20©2020 Guidehouse Inc. All Rights Reserved
In general, savings increase and then level off for each usage band over time and are generally higher for higher usage bands than lower usage
bands. These results are consistent with evaluated results across many jurisdictions reflecting program ramp-up and the larger savings
opportunities among higher users.
6. Calibrated BRM Savings Values - LDVCalibrated BRM values, accounting for uplift, are shown below from the LDV model (with imputed
data for the DTE 201107 wave).
Table 7. Calibrated BRM Usage Bands - LDV
Fuel Usage Band Year 1 Year 2 Year 3 Year 4 Year 5 Year 6 Year 7 Year 8 Year 9
Elec 5-7 MWh 0.12% 0.59% 0.87% 0.77% 0.89% 0.93%
Elec 7-9 MWh 0.72% 1.27% 1.29% 1.81% 1.64% * 1.98% ** **
Elec 9-11 MWh 0.78% 1.36% 1.50% 1.42% 1.24% 1.59% * 2.02% **
Elec 11-13 MWh 0.90% 1.42% 1.66% 1.84% 1.54%
Elec >13 MWh 1.13% 1.88% 2.19% 2.11% 2.19% 1.98%
Gas 601 - 900 Therm 0.27% 0.45% 0.51% 0.27%
Gas 901 - 1200 Therm 0.47% 0.68% 0.71% 0.74% 0.75% 0.73% 0.89% ** **
Gas >1200 Therm 0.41% 0.74% 0.73% 0.82% * * * 0.82% **
* These savings values have not changed from the deemed values in the current BRM as there was not enough data to calibrate them.
**Values have been proposed by DTE/Oracle for these usage bands.
Data for the DTE 201107 wave is imputed for the missing portion of the pre-period.
Note, when a usage band/year combination does not exist in the BRM for a wave for which savings are being claimed, we recommend the utility claim the last year available
in the BRM for that usage band. For example, if a utility would like to claim savings for a wave in Year 7 in the 5 – 7 MWh band, we recommend using the Year 6 value,
0.93%.
Source: Guidehouse analysis
21©2020 Guidehouse Inc. All Rights Reserved
Similar to the LDV results, savings increase and then level off for each usage band over time and are generally higher for higher usage bands
than lower usage bands. These results are consistent with evaluated results across many jurisdictions reflecting program ramp-up and the
larger savings opportunities among higher users.
6. Calibrated BRM Savings Values - LFERCalibrated BRM values, accounting for uplift, are shown below from the LFER model.
Table 8. Calibrated BRM Usage Bands - LFER
Fuel Usage Band Year 1 Year 2 Year 3 Year 4 Year 5 Year 6 Year 7 Year 8 Year 9
Elec 5-7 MWh 0.13% 0.57% 0.89% 0.81% 0.83% 0.88%
Elec 7-9 MWh 0.70% 1.11% 1.29% 1.78% 1.62% * 2.33% ** **
Elec 9-11 MWh 0.79% 1.45% 1.59% 1.64% 1.40% 1.73% * 2.29% **
Elec 11-13 MWh 0.96% 1.42% 1.67% 1.75% 1.41%
Elec >13 MWh 1.27% 1.98% 2.10% 1.99% 2.02% 1.95%
Gas 601 - 900 Therm 0.31% 0.36% 0.60% 0.35%
Gas 901 - 1200 Therm 0.44% 0.69% 0.65% 0.70% 0.72% 0.65% 0.59% ** **
Gas >1200 Therm 0.45% 0.75% 0.68% 0.81% * * * 0.58% **
* These savings values have not changed from the deemed values in the current BRM as there was not enough data to calibrate them.
**Values have been proposed by DTE/Oracle for these usage bands.
Note, when a usage band/year combination does not exist in the BRM for a wave for which savings are being claimed, we recommend the utility claim the last year available in
the BRM for that usage band. For example, if a utility would like to claim savings for a wave in Year 7 in the 5 – 7 MWh band, we recommend using the Year 6 value, 0.88%.
Source: Guidehouse analysis
22©2020 Guidehouse Inc. All Rights Reserved
6. Comparison of Calibrated Savings Values to the BRMComparisons of calibrated values from the LDV model (with imputed data for the DTE 201107 wave)
and LFER model to the current BRM values for electric waves are shown below. Generally the newly
calibrated values are lower than the current BRM.Figure 13. Difference between BRM and Newly Calibrated Savings Rates - Electric
Note, while some calibrated results are not statistically different from zero, we recommend including the point estimate in the BRM as it is the best estimate available for the usage band/year at this time.
Source: Guidehouse analysis
23©2020 Guidehouse Inc. All Rights Reserved
6. Comparison of Calibrated Savings Values to the BRMComparisons of calibrated values from the LDV model (with imputed data for the DTE 201107 wave)
and LFER model to the current BRM values for gas waves are shown below. The newly calibrated
values are mixed compared to the current BRM.
Figure 14. Difference between BRM and Newly Calibrated Savings Rates - Gas
Note, while some calibrated results are not statistically different from zero, we recommend including the point estimate in the BRM as it is the best estimate available for the usage band/year at this time.
Source: Guidehouse analysis
24©2020 Guidehouse Inc. All Rights Reserved
6. BRM Savings Value RecommendationUsing either the LDV or LFER model, Guidehouse found a demand savings factor near or below 1.
Table 9. Calibrated BRM Demand Savings Factor
Model Energy
Savings
Demand
Savings
Demand
Savings Factor
LDV 0.86% 0.67% 0.78
LFER* 0.69% 0.70% 1.02
* The LFER energy savings are based on a weighted average of individual
wave / program year results.
Source: Guidehouse analysis
25©2020 Guidehouse Inc. All Rights Reserved
• Reach consensus regarding model specification options outlined on slide 19.
• Develop and submit a modified measure workpaper for review by the EWR Collaborative’s Technical
Subcommittee, including:
– Updated energy and demand savings values based on calibration
– Specify the demand savings factor
– Provide additional specifications on the measure description (in particular, report frequency)
– Updates to the description for how to determine a wave’s usage band for calculating savings
– Model specification requirements for future new measure submissions, including how to handle missing
data
• Recommend calibration of later years’ savings values in 3-4 years when an additional 5-6 electric waves
and 2-3 gas waves could be included.
7. Next StepsGuidehouse will develop an updated measure workpaper for the BRM.
26©2020 Guidehouse Inc. All Rights Reserved
Contact
©2020 Guidehouse Inc. All rights reserved. This content is for
general information purposes only, and should not be used as
a substitute for consultation with professional advisors.
Carly Olig
Associate Director
Debbie Brannan
Director
Kathleen Ward
Managing Consultant
Cherish Smith
Associate Director
Rachel Marty
Managing Consultant
Jackson Lines
Senior Consultant
27©2020 Guidehouse Inc. All Rights Reserved
Appendices
28©2020 Guidehouse Inc. All Rights Reserved
Where:
• i indexes the customer
• t indexes time
• w indexes program wave
• 𝐴𝐷𝑈𝑖𝑡 is the customer’s average daily energy consumption during time t
• 𝐴𝐷𝑈𝑙𝑎𝑔𝑖𝑡 is the customer’s average daily energy consumption during the same month as time t lagged to the pre-program year
• 𝑌𝑟𝑀𝑜𝑗𝑡 comprise a set of month-of-year indicators, which equal 1 if
t falls in month-of-year j, and 0 otherwise
• 𝑇𝑟𝑒𝑎𝑡𝑚𝑒𝑛𝑡𝑖 is a binary indicator that equals 1 if customer i is atreatment customer, and 0 otherwise
• 𝐶𝐷𝐷𝑖𝑡 are the cooling degree-days during time t for customer i
• 𝐻𝐷𝐷𝑖𝑡 are the heating degree-days during time t for customer i
• 𝑊𝑎𝑣𝑒𝑖𝑤 is a binary indicator that equals 1 if customer i falls inwave w, and 0 otherwise (only included in pooled models)
• 𝑈𝑡𝑖𝑙𝑖𝑡𝑦𝑖 is a binary indicator that equals 1 if customer i is a DTE customer, and 0 otherwise (only included in pooled models)
• The 𝛽1 − 𝛽6 are unknown parameters to be estimated
• 𝜀𝑖𝑡 is a mean-zero disturbance term
Appendix A. Energy Regression Model - LDVAn LDV model was used to calculate energy savings.
𝐴𝐷𝑈𝑖𝑡=
𝐽
𝛽1𝑗𝑌𝑟𝑀𝑜𝑗𝑡 +
𝐽
𝛽2𝑗𝑌𝑟𝑀𝑜𝑗𝑡 ∙ 𝐴𝐷𝑈𝑙𝑎𝑔𝑖𝑡 + 𝛽3𝑇𝑟𝑒𝑎𝑡𝑚𝑒𝑛𝑡𝑖 + 𝛽4𝐶𝐷𝐷_𝑎𝑛𝑑_𝑜𝑟_𝐻𝐷𝐷𝑖𝑡 +
𝑊
𝛽5𝑊𝑎𝑣𝑒𝑖𝑤
+ 𝛽6𝑈𝑡𝑖𝑙𝑖𝑡𝑦𝑖 + 𝜀𝑖𝑡
29©2020 Guidehouse Inc. All Rights Reserved
Where:
• i indexes the customer
• t indexes time
• 𝐴𝐷𝑈𝑖𝑡 is the customer’s average daily energy consumption during time t
• 𝛼𝑖 is the customer specific fixed effect capturing all time-invariant observable and unobservable customer characteristics
• 𝑃𝑜𝑠𝑡𝑡 is a binary indicator that equals 1 if observation t for customer i is after customer i’s HER wave started (only included
independently of treatment in pooled models)
• 𝑇𝑟𝑒𝑎𝑡𝑚𝑒𝑛𝑡𝑖 is a binary indicator that equals 1 if customer i is atreatment customer, and 0 otherwise
• 𝑌𝑟𝑀𝑜𝑗𝑡 comprise a set of month-of-year indicators, which equal 1 if
t falls in month-of-year j, and 0 otherwise
• 𝐶𝐷𝐷𝑖𝑡 are the cooling degree-days during time t for customer i
• 𝐻𝐷𝐷𝑖𝑡 are the heating degree-days during time t for customer i
• The 𝛽1 − 𝛽4 are unknown parameters to be estimated
• 𝜀𝑖𝑡 is a mean-zero disturbance term
Appendix A. Energy Regression Model - LFERAn LFER model was used to calculate energy savings.
𝐴𝐷𝑈𝑖𝑡= 𝛼𝑖 + 𝛽1𝑃𝑜𝑠𝑡𝑡 ∙ 𝑇𝑟𝑒𝑎𝑡𝑚𝑒𝑛𝑡𝑖 +
𝐽
𝛽2𝑗𝑌𝑟𝑀𝑜𝑗𝑡 + 𝛽3𝐶𝐷𝐷_𝑎𝑛𝑑_𝑜𝑟_𝐻𝐷𝐷𝑖𝑡 + 𝛽4𝑃𝑜𝑠𝑡𝑡 + 𝜀𝑖𝑡
30©2020 Guidehouse Inc. All Rights Reserved
• The model for demand savings used only the data from the coincident peak demand period (i.e., the three
hottest, consecutive weekdays in July from 3 to 6 p.m.).
• For the LDV model, the usage lag (ADUlagit) is defined as the average usage during the same hour on the
three peak days from the pre-program year.
Appendix A. Demand Regression ModelAnalogous models were used to calculate demand savings.
31©2020 Guidehouse Inc. All Rights Reserved
Appendix B. Upstream Lighting - DTE 2019 Survey1
Over three-quarters of respondents said they purchased small energy efficiency devices in PY2019. LED light
bulbs were the most common measure purchased by over 90 percent of small energy efficiency device
purchasers. There was little difference between recipient and control group customers in rates of purchasing
the different small energy efficient devices.Small EE Devices
» Over three-quarters of HER recipient (76 percent) and control (78 percent) group customers reported purchasing small energy efficient devices in 2019.
» LED light bulbs were the most common measures installed, with over 90 percent of small energy efficiency device purchasers reporting buying this measure. The difference in purchase rates between recipient and control group customers was not statistically significant (Figure 1-12). The majority of treatment and control customers purchase bulbs from participating Upstream Lighting retailers. These findings do not support the hypothesis that HER savings are primarily attributable to lighting.
» Rates of purchasing all other small energy efficient devices were similar for treatment and control group customer.
FIGURE B-1. HOME ENERGY REPORT
Small EE Purchases (Recipient vs. Control)2, n=1165
3%
4%
11%
12%
12%
15%
95%
3%
4%
12%
10%
14%
15%
94%
Other non-connected devices
Other smart home devices
Smart outlets
Smart or WiFi-enabled thermostat
Advanced power strip
Programmable thermostat
LED lightbulbs
Recipient (n=588) Control (n=577)
2 Survey results were weighted by HER wave to ensure that survey sample was
representative of the program population. See slide 22 in PY2019 report for more detail on how wave-based weights were constructed.
Source: 2019 year-end HER recipient and control survey data.
1 Due to changes to the program survey, results should not be compared across 2018
and 2019. However, the HER recipient and control groups can be accurately compared
within each year.
32©2020 Guidehouse Inc. All Rights Reserved
Appendix B. Upstream Lighting - DTE 2018 Survey1LED bulbs remain the most popular energy efficient purchase among respondents who reported making an
energy efficient purchase in the last 12 months.
• Generally, HER recipients report engaging in energy efficient
purchases more often than their control counterparts (Figure 4-16).
• The following energy efficient appliance are those which HER
recipients reported purchases at a higher rate than their control
counterparts:
– Freezers, 80 percent confidence
– Furnace Fans, 80 percent confidence
– Insulation, 80 percent confidence
– Other non-LED lighting, 90 percent confidence
• LED bulbs are the most common energy efficient item purchased
by both HER recipients and their control counterparts at almost
identical rates which may suggest that the HER tips designed to
motivate purchases of LED bulbs are not effective. It is also
possible that general consumer preferences of DTE customers has
shifted towards LED bulbs and thus there is no purchasing habit
modification potential.
Energy efficient appliance purchasesFigure B-2. HOME ENERGY REPORT
Reported Energy Efficiency Purchases by
Appliance (Recipient vs. Control), n = 210
41%
17%
15%
13%
7%
9%
10%
9%
5%
6%
3%
41%
18%
11%
8%
10%
7%
6%
6%
3%
3%
1%
0% 10% 20% 30% 40% 50%
LED Light Bulbs
Electronics
Programmable Thermostat
Other Lights**
Clothes Dryer
Dishwasher
Insulation*
Water Heater
Dehumidifier
Furnace Fan*
Freezer*
Control Treatment
Source: 2018 monthly HER recipient survey data
1 Due to changes to the program survey, results should not be compared across 2018
and 2019. However, the HER recipient and control groups can be accurately compared
within each year.
33©2020 Guidehouse Inc. All Rights Reserved
3 DNV-GL. 2014. National Grid Residential Building Practices and Demonstration Program Evaluation, Final Results.
http://viget.Opower.com/company/library/verification-reports.
4 DNV-GL. 2018. PSE Home Energy Reports Program. 2017 Impact Evaluation – Final Report. https://conduitnw.org/Pages/File.aspx?rid=4415
5 DNV-GL. 2017. Review and Validation of 2015 Pacific Gas and Electri