Download - Performance Metrics for Weatherization
![Page 1: Performance Metrics for Weatherization](https://reader036.vdocuments.us/reader036/viewer/2022062720/568132af550346895d996130/html5/thumbnails/1.jpg)
Performance Metrics for Weatherization
David Carroll
State WAP Evaluations
ACI Home Performance Conference
April 30, 2014
![Page 2: Performance Metrics for Weatherization](https://reader036.vdocuments.us/reader036/viewer/2022062720/568132af550346895d996130/html5/thumbnails/2.jpg)
What is Performance Measurement?
2
![Page 3: Performance Metrics for Weatherization](https://reader036.vdocuments.us/reader036/viewer/2022062720/568132af550346895d996130/html5/thumbnails/3.jpg)
Step 1 - Goals• What are your goals?
– Are you trying to serve a certain population?– Are you trying to affect …
• Energy use?
• Energy costs?
• Payment problems?
• Emissions?
• Health problems?
– How will you know when you are successful?
3
![Page 4: Performance Metrics for Weatherization](https://reader036.vdocuments.us/reader036/viewer/2022062720/568132af550346895d996130/html5/thumbnails/4.jpg)
Step 2 – Measurement• Overall Research
– Who are you serving?– How are you serving them?– What are they saving?– What other benefits are you delivering?
• Comparative Research– What approach maximizes energy, cost,
payment, emissions, health impacts?
4
![Page 5: Performance Metrics for Weatherization](https://reader036.vdocuments.us/reader036/viewer/2022062720/568132af550346895d996130/html5/thumbnails/5.jpg)
Step 3 - Assessment
• Assessment– What are your goals for your program?– Does your performance match your goals?– How can you better align your performance
with your goals?
5
![Page 6: Performance Metrics for Weatherization](https://reader036.vdocuments.us/reader036/viewer/2022062720/568132af550346895d996130/html5/thumbnails/6.jpg)
Step 4 – Improvement
• Strategies– Changing who you serve
• Targeting
• Partnerships
– Changing what you deliver• Assessments
• Measures
6
![Page 7: Performance Metrics for Weatherization](https://reader036.vdocuments.us/reader036/viewer/2022062720/568132af550346895d996130/html5/thumbnails/7.jpg)
Step 4 - Improvement
• Strategies (continued)– Changing your performance
• Comparative analysis
• Mentoring / partnerships
– Changing funding available• Leveraging
• Advocacy
7
![Page 8: Performance Metrics for Weatherization](https://reader036.vdocuments.us/reader036/viewer/2022062720/568132af550346895d996130/html5/thumbnails/8.jpg)
A Comprehensive Approach to Performance Measurement
8
![Page 9: Performance Metrics for Weatherization](https://reader036.vdocuments.us/reader036/viewer/2022062720/568132af550346895d996130/html5/thumbnails/9.jpg)
Benchmark
• Use data on ….– Population / Census Data (ACS)– Recipients / LIHEAP or Energy Assistance– WX Clients / Demographics, Housing Units,
Baseline Conditions
9
![Page 10: Performance Metrics for Weatherization](https://reader036.vdocuments.us/reader036/viewer/2022062720/568132af550346895d996130/html5/thumbnails/10.jpg)
Observe
• Observations of ….– Intake– Audits– Service Delivery– Inspections
10
![Page 11: Performance Metrics for Weatherization](https://reader036.vdocuments.us/reader036/viewer/2022062720/568132af550346895d996130/html5/thumbnails/11.jpg)
Measure
• Collect data on ….– Installed Measures– Post-WX Conditions– Pre/Post Energy Usage– Client Outcomes – Payments, Housing, Health
11
![Page 12: Performance Metrics for Weatherization](https://reader036.vdocuments.us/reader036/viewer/2022062720/568132af550346895d996130/html5/thumbnails/12.jpg)
Analyze
• Study ….– Gross changes in weather-normalized usage– Net changes using a comparison group– Variations in outcomes by key client, building,
and program factors
12
![Page 13: Performance Metrics for Weatherization](https://reader036.vdocuments.us/reader036/viewer/2022062720/568132af550346895d996130/html5/thumbnails/13.jpg)
Assessment
• Investigate…– Why are some subgroups seeing high impacts?– Why are some subgroups seeing low impacts?– What strategies might be tested to better align
outcomes with goals?
13
![Page 14: Performance Metrics for Weatherization](https://reader036.vdocuments.us/reader036/viewer/2022062720/568132af550346895d996130/html5/thumbnails/14.jpg)
Who are you Serving? Example of using Census and LIHEAP
data as a Benchmark
14
![Page 15: Performance Metrics for Weatherization](https://reader036.vdocuments.us/reader036/viewer/2022062720/568132af550346895d996130/html5/thumbnails/15.jpg)
Minnesota Households - 2010
About one in four Minnesota households were income-eligible for the LIHEAP Program in 2010.
15
Eligibility Group Number of Households Percent of Households
Not Income Eligible 1,564,187 75%
Income Eligible 527,359 25%
All Households 2,091,546 100%
Source: 2010 American Community Survey
![Page 16: Performance Metrics for Weatherization](https://reader036.vdocuments.us/reader036/viewer/2022062720/568132af550346895d996130/html5/thumbnails/16.jpg)
Minnesota LIHEAP and WAP Recipients in 2010
About 38% of income-eligible households received LIHEAP and 2% received WAP.
National averages are 17% and 0.5%, respectively.
16
Received BenefitsLIHEAP
RecipientsPercent of
HouseholdsWAP
RecipientsPercent of
Households
Yes 199,969 38% 11,207 2%
No 327,390 62% 516,152 98%
Income-Eligible Households
527,359 100% 527,359 100%
Source: 2010 American Community Survey and 2010 MN LIHEAP/WAP Database
![Page 17: Performance Metrics for Weatherization](https://reader036.vdocuments.us/reader036/viewer/2022062720/568132af550346895d996130/html5/thumbnails/17.jpg)
Housing Unit Type
LIHEAP is less likely to serve households in Large Multi-Family buildings. WAP serves very few households in Multi-Family buildings of any type.
17
Housing Unit Type Low-Income LIHEAP Recipients WAP Recipients
Single Family 52% 59% 80%
Mobile Home 5% 11% 14%
Small Multi-Family 8% 7% 3%
Large Multi-Family 35% 23% 3%
Total 100% 100% 100%
Source: 2010 American Community Survey and 2010 MN LIHEAP/WAP Database
![Page 18: Performance Metrics for Weatherization](https://reader036.vdocuments.us/reader036/viewer/2022062720/568132af550346895d996130/html5/thumbnails/18.jpg)
Housing Unit Type
LIHEAP is less likely to serve households in Large Multi-Family buildings. WAP serves very few households in Multi-Family buildings of any type.
18
Housing Unit Type Low-Income LIHEAP Recipients WAP Recipients
Single Family 52% 59% 80%
Mobile Home 5% 11% 14%
Small Multi-Family 8% 7% 3%
Large Multi-Family 35% 23% 3%
Total 100% 100% 100%
Source: 2010 American Community Survey and 2010 MN LIHEAP/WAP Database
![Page 19: Performance Metrics for Weatherization](https://reader036.vdocuments.us/reader036/viewer/2022062720/568132af550346895d996130/html5/thumbnails/19.jpg)
Ownership Status
LIHEAP serves both owners and renters. WAP is less likely to serve renters.
19
Ownership Status Low-Income LIHEAP Recipients WAP Recipients
Owner 46% 44% 84%
Renter/Other 54% 56% 16%
Total 100% 100% 100%
Source: 2010 American Community Survey and 2010 MN LIHEAP/WAP Database
![Page 20: Performance Metrics for Weatherization](https://reader036.vdocuments.us/reader036/viewer/2022062720/568132af550346895d996130/html5/thumbnails/20.jpg)
Ownership Status – Single Family Units
Even if we control for housing unit type, WAP is more likely to serve owners than renters.
20
Ownership Status Low-Income LIHEAP Recipients WAP Recipients
Owner 75% 60% 87%
Renter/Other 25% 40% 13%
Total 100% 100% 100%
Source: 2010 American Community Survey and 2010 MN LIHEAP/WAP Database
![Page 21: Performance Metrics for Weatherization](https://reader036.vdocuments.us/reader036/viewer/2022062720/568132af550346895d996130/html5/thumbnails/21.jpg)
Ownership Status of WAP Participants – Single Family Units
Control for housing unit type reduces that variation in service to renters by substate area.
21
Ownership Status
State NE NW Central Metro SE SW
Owner 87% 93% 94% 89% 80% 88% 88%
Renter/ Other
13% 7% 6% 11% 20% 12% 12%
Total 100% 100% 100% 100% 100% 100% 100%
Source: 2010 American Community Survey and 2010 MN LIHEAP/WAP Database
![Page 22: Performance Metrics for Weatherization](https://reader036.vdocuments.us/reader036/viewer/2022062720/568132af550346895d996130/html5/thumbnails/22.jpg)
How are you serving them? Example of furnishing details for
the state, region, and agency.
22
![Page 23: Performance Metrics for Weatherization](https://reader036.vdocuments.us/reader036/viewer/2022062720/568132af550346895d996130/html5/thumbnails/23.jpg)
23
Table 4 PY 2010 Clients in Single Family Homes
Housing Unit Characteristics by Region and Agency
Statistic STATE Central Region
Lake Wobegone
Number of Units 8,863 1,292 100
Housing Unit
Median Heated Space 1,340 1,235 1,200
Mean Heated Space 1,451 1,336 1,350
% One Story 40% 46% 49%
Housing Vintage
% pre 1940 38% 38% NC
% 1940-1969 33% 27% NC
% 1970 or later 29% 35% NC
PreWX Status
Mean CFM 50 2,337 2,163 2,855 NC=Not Collected
![Page 24: Performance Metrics for Weatherization](https://reader036.vdocuments.us/reader036/viewer/2022062720/568132af550346895d996130/html5/thumbnails/24.jpg)
24
Table 7 PY 2010 Clients in Single Family Homes
Air Sealing and Shell Measures by Region and Agency
Statistic STATE Central Region
Lake Wobegone
Number of Units 8,863 1,292 100
Air Sealing
< 500 CFM Reduction 54% 58% 50%
500 - < 1000 CFM Reduction 23% 20% 15%
1000+ CFM Reduction 19% 17% 32%
No Information 5% 5% 3%
Any bypass sealing or caulking 81% 75% 82%
Attic Insulation
% Installed 86% 83% 90%
Wall Insulation
% Installed 36% 36% 39%
Other Insulation
% Floor insulation 5% 7% 6%
% Rim/band joist insulation 19% 41% 15%
% Foundation insulation 6% 6% 10%
Mean CFM reduction 607 548 840
% Installed Mechanical Ventilation 46% 57% 55%
![Page 25: Performance Metrics for Weatherization](https://reader036.vdocuments.us/reader036/viewer/2022062720/568132af550346895d996130/html5/thumbnails/25.jpg)
How much energy are you saving and how does that vary by important program factors?
25
![Page 26: Performance Metrics for Weatherization](https://reader036.vdocuments.us/reader036/viewer/2022062720/568132af550346895d996130/html5/thumbnails/26.jpg)
Table 1WAP Energy Impacts for Single Family Site-Built Homes
Gross and Net Gas Savings by End Use (therms/year)
Group/Breakout # HomesGas Use Pre-WAP
Gas Use Post-WAP
Gross Savings Net Savings % of Pre
Total Use 937 983 845 138 (±10)
130 (±11) 13.2% (±1.1%) Comparison 1,295 929 921 8 (±4)
Heating Use 937 775 658 117 (±9)
100 (±11) 13.0% (±1.4%)
Comparison 1,295 741 724 16 (±6)
Baseload Use 937 208 187 21 (±7)
30 (±8) 14.2% (±17)
Comparison 1,295 188 196 -8 (±5)
![Page 27: Performance Metrics for Weatherization](https://reader036.vdocuments.us/reader036/viewer/2022062720/568132af550346895d996130/html5/thumbnails/27.jpg)
Table 3WAP Energy Impacts for Single Family Site-Built Homes
Gas Savings for Homes with Natural Gas Main HeatBy Measure Combination (therms/year)
Group/Breakout # HomesGas Use Pre-
WAP Net Savings % of Pre
No Major Measures 202 877 37 (±15) 4.2% (±1.7%)
One Major Measure 298 957 121 (±17) 12.7% (±1.8%)
Two Major Measures 211 1,003 162 (±20) 16.1% (±2.0%)
Three Major Measures 115 1,111 236 (±29) 21.2% (±2.6%)
All Four Major Measures 33 1,179 382 (±71) 32.4% (±6.1%)
![Page 28: Performance Metrics for Weatherization](https://reader036.vdocuments.us/reader036/viewer/2022062720/568132af550346895d996130/html5/thumbnails/28.jpg)
Table 4
WAP Energy Impacts for Single Family Site-Built HomesNet Gas Savings for Natural Gas Main Heat by Pre-Weatherization
Gas Usage (therms/year)
Pre-WAP Gas Use (therms/yr)
Major Measures # Homes
Gas Use Pre-WAP Net Savings % of Pre
All Clients 1.4 937 983 130 (±10) 13.2% (±1.1%)
<750 th/yr. 1.1 245 640 64 (±12) 10.0% (±1.9%)
750-1000 1.3 296 880 105 (±14) 12.0% (±1.6%)
1000-1250 1.6 226 1,097 142 (±22) 12.9% (±2.0%)
1250-1500 1.6 101 1,355 219 (±42) 16.2% (±3.1%)
>=1500 th/yr. 2.0 69 1,731 269 (±65) 15.6% (±3.7%)
![Page 29: Performance Metrics for Weatherization](https://reader036.vdocuments.us/reader036/viewer/2022062720/568132af550346895d996130/html5/thumbnails/29.jpg)
Table 12Gas Impact Results by Agency for Gas Heated
Single Family Site-Built Homes
Agency IDGas Use Pre-WAP Net Savings % of Pre
# of Measures
A 1,077 187 (± 21) 17.3% (±1.9) 2.2B 992 122 (± 32) 12.3 % (±3.2) 0.9C 1,028 119 (± 40) 11.6% (±3.9) 1.2D 948 118 (± 19) 12.4% (±2.0) 1.3E 1,012 113 (± 64) 11.1% (±6.3) 0.7F 937 109 (± 44) 11.6% (±4.7) 1.2G 945 107 (± 50) 11.3% (±5.3) 0.9H 875 94 (± 26) 10.7% (±3.0) 0.7I 929 94 (± 41) 10.1% (±4.4) 1.1J 889 58 (± 27) 6.5% (±3.0) 0.5
Total 983 130 (±11) 13.2% (±0.7) 1.4
![Page 30: Performance Metrics for Weatherization](https://reader036.vdocuments.us/reader036/viewer/2022062720/568132af550346895d996130/html5/thumbnails/30.jpg)
Make sure you consider other program impacts.
30
![Page 31: Performance Metrics for Weatherization](https://reader036.vdocuments.us/reader036/viewer/2022062720/568132af550346895d996130/html5/thumbnails/31.jpg)
Table 26Emissions Benefits
Estimated Value of Avoided Emissions per Home for Single Family Site-Built Homes (2013 Dollars)
Pollutant First Year Lifetime
CO2 Equivalent $53.65 $1,009.63
NOX $23.45 $432.11
PM2.5 $2.49 $42.85
SO2 $44.48 $684.39
VOC $0.41 $8.25
Total $124.49 $2,177.22
31
![Page 32: Performance Metrics for Weatherization](https://reader036.vdocuments.us/reader036/viewer/2022062720/568132af550346895d996130/html5/thumbnails/32.jpg)
Dwelling Quality Indicators
32
IndicatorPre-Audit Incidence
Post-WX Incidence
Percent Change
Home sometimes at unhealthy temperature
20% 9% -55%
Observed standing water in home
33% 19% -42%
Frequent mildew odor or musty smell
29% 16% -45%
Home somewhat or very infested with insects
25% 17% -32%
Home has CO monitor 41% 74% +80%
![Page 33: Performance Metrics for Weatherization](https://reader036.vdocuments.us/reader036/viewer/2022062720/568132af550346895d996130/html5/thumbnails/33.jpg)
Affordability Indicators
33
IndicatorPre-Audit Incidence
Post-WX Incidence
Change
Got disconnect notice 45% 33% -26%
Had natural gas, electricity, or bulk fuel service terminated
18% 9% -50%
Went without food to pay energy bills
34% 23% -33%
Went without food in the last four weeks
9% 6% -33%
Used high interest loan to pay energy bills
20% 12% -40%