9. evaluation and ee indicators
Post on 16-Oct-2021
0 Views
Preview:
TRANSCRIPT
IEA 2019. All rights reserved.
9. Evaluation and EE Indicators
John Dulac
Bangkok, 3 April 2019
IEA #energyefficientworld
2IEA 2019. All rights reserved.
9. Evaluation and EE Indicators
9. Evaluation and EE Indicators
Trainer(s): John Dulac
Scenario: The national government wants to know how effective the
energy efficiency programmes have been and wants to compare different
cities.
Question: How do you develop indicators that properly measures the
benefits of your programmes?
3IEA 2019. All rights reserved.
Training Overview
1. Why Evaluate?
• Determine impact, provide insight, determine value for money
2. Using indicators
• Value of indicators
• Analyses that can be done (performance, demand, decomposition)
• Examples of indicators in urban areas: IEA, ESMAP
3. Activity: Telling the story
4. What are the steps?
• What needs to be tracked, define the indicators
• Assess data and tell the story
• Embedding evaluation in project planning
10 mins
15 mins
30 mins
15 mins
4IEA 2019. All rights reserved.
1. Why Evaluate?
5IEA 2019. All rights reserved.
1. Why Evaluate?
Your peers in the Indicators and Evaluation course will specialise more on this topic.
It is important to understand how you can use evaluation into your specialty in the
urban environment
Where to start? Tools What are the steps?
6IEA 2019. All rights reserved.
1. Why Evaluate?
Where to start? Tools What are the steps?
Process, how did
it go?
Impact, what did
we achieve?
Economic, did we
get value for
money?
• Regulators
• NGOs and public
• Programme
managers
• Partners
• Funders
• Treasury
7IEA 2019. All rights reserved.
1. Why Evaluate?
Where to start? Tools What are the steps?
Impact, what did
we achieve?
Energy efficiency policies affect
multiple aspects of society and
the economy of interest to
stakeholders
These are some of the multiple
benefits or impacts that must be
considered in evaluation
Are these impacts being
considered in your evaluation
reports?
Energy savings GHG
reduction
Energysecurity
Energy delivery
Assetvalues
Householdsavings
Publicbudgets
Resourceefficiency
Employment Productivity
Health and well-being
Povertyalleviation
Macro-economicimpacts
Energyprices
Air quality
IEA’s Multiple Benefits Diagram
https://www.iea.org/publications/freepublication
s/publication/Multiple_Benefits_of_Energy_Efficie
ncy.pdf
8IEA 2019. All rights reserved.
1. Why Evaluate?
Where to start? Tools What are the steps?
Process, how did
it go?
Economic,
Social, Political
climate
Organizations
engaged
Business or
Execution
models used
Problems
encountered
Programme Delivery
Evaluation also provides insight on how the programme was delivered
9IEA 2019. All rights reserved.
1. Why Evaluate?
Where to start? Tools What are the steps?
Economic, did we
get value for
money?
Evaluation also provides metrics in comparing costs to benefits
Source: https://www.theicct.org/publications/cost-benefit-analysis-brazils-heavy-duty-emission-standards-p-8
10IEA 2019. All rights reserved.
2. Using indicators
11IEA 2019. All rights reserved.
2. Using indicators. Value of indicators
Target
• reinforces the message of the targets
Progress
• keeps everyone aware of their progress and whether they are on-track to meet the goals
Motivation
• reinforces competition and provides evidence for stronger policies when necessary
Where to start? Tools What are the steps?
12IEA 2019. All rights reserved.
2. Using indicators. Value of indicators
Motivation
• reinforces competition and provides evidence for stronger policies when necessary
Where to start? Tools What are the steps?
• For example: AEEE’s EE preparedness report
Source State-EE-Preparedness-Index-FINAL_July2018.pdf
13IEA 2019. All rights reserved.
2. Using indicators. Value of indicators
Motivation
• reinforces competition and provides evidence for stronger policies when necessary
Where to start? Tools What are the steps?
• For example: ESMAP TRACE tool
Source ESMAP TRACE 2.0
14IEA 2019. All rights reserved.
2. Using indicators. Analyses that can be done
Where to start? Tools What are the steps?
Performance Metrics
• Measures changes in energy intensities
Demand Analysis
• Analysing demands and projecting possible futures
Decomposition
• Break down energy use into individual factors to help determine where best to address future policy concerns.
15IEA 2019. All rights reserved.
2. Using indicators. Analyses that can be done
Where to start? Tools What are the steps?
Performance Metrics
• Measures changes in energy intensities
• Main energy use divided by main driver
• More aggregated, the better
• Water Energy Intensity kWhr/million liters
0
200
400
600
800
1000
Water Source and
conveyance
Treatment Distribution Total
En
erg
y i
nte
nsi
ty (
kW
hr/
mil
lio
n
lite
rs)
Source ACEEE Survey on Energy Use in Water
16IEA 2019. All rights reserved.
2. Using indicators. Analyses that can be done
Where to start? Tools What are the steps?
Demand Analysis
• Analysing demands and projecting possible futures
• Change each factor and see how they affect future trends
• What-if analysis: Proportional distribution of energy savings by holding only changing one indicator at a time
Source Energy Technology Perspectives
17IEA 2019. All rights reserved.
2. Using indicators. Analyses that can be done
Where to start? Tools What are the steps?
Decomposition
• Break down energy use into individual factors to help determine where best to address future policy concerns.
• Sample Municipality X, street lighting performance
0.0
50.0
100.0
150.0
200.0
250.0
300.0
350.0
400.0
2000 Energy
use
Activity
km of lit
roads
Structure 1
density of
lighting
Structure 2
share of tech
Efficiency 2017 Energy
use
Fin
al
En
erg
y C
on
sum
pti
on
, G
Wh
r
18IEA 2019. All rights reserved.
2. Using indicators. Example: IEA indicators
Where to start? Tools What are the steps?
Manuals on statistics Manuals on policymaking
https://webstore.iea.org/energy-efficiency-
indicators-fundamentals-on-statistics
https://webstore.iea.org/energy-efficiency-indicators-
essentials-for-policy-making
19IEA 2019. All rights reserved.
2. Using indicators. Example: IEA indicators
Where to start? Tools What are the steps?
Online Course
20IEA 2019. All rights reserved.
2. Using indicators. Example: Worldbank
Where to start? Tools What are the steps?
https://www.esmap.org/node/235?resources
21IEA 2019. All rights reserved.
3. Activity
22IEA 2019. All rights reserved.
3. Activity
• What story can you tell about these figures?
0.00
10.00
20.00
30.00
40.00
50.00
60.00
70.00
80.00
0.00
5.00
10.00
15.00
20.00
25.00
30.00
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 m3 w
ate
r co
nsu
mp
tio
n p
er
year
Po
pu
lati
on
per
year
Population 10^6 capita Water per capita m3/capita
0
500
1000
1500
2000
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18
Site 1 (groundwater) Site 2 (groundwater)
Site 3 (wastewater reuse) Site 4 (river)
Site 5 (desalination)
0.0
200.0
400.0
600.0
800.0
1000.0
1200.0
1400.0
1600.0
2000 Energy
use
Activity
population
Structure 2
share of
water
treatment
plants
Efficiency Structure 1
water
demand per
capita
2017 Energy
use
Fin
al
En
erg
y C
on
sum
pti
on
, G
Wh
r
23IEA 2019. All rights reserved.
3. Activity
• What story can you tell about these figures?
0.0
50.0
100.0
150.0
200.0
250.0
300.0
350.0
400.0
2000 Energy
use
Activity
km of lit
roads
Structure 1
density of
lighting
Structure 2
share of tech
Efficiency 2017 Energy
use
Fin
al En
erg
y C
on
sum
pti
on
, G
Wh
r
0
100000
200000
300000
400000
500000
600000
700000
800000
900000
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18
Nu
mb
er
of
Un
its
MV HPSV LPSV T5 LED
24IEA 2019. All rights reserved.
3. What are the steps?
25IEA 2019. All rights reserved.
4. Steps in the process
• What story should be told?
• What were the objectives?
• What are the risks?
Step 1: Identify
what needs to be
tracked
• What performance metrics can you use?
• What data is needed?
Step 2: Define the
tracking indicators
• What analysis method should you use?Step 3: Assess the
data
• How do you visualise the results?
• How would it vary across countries?
Step 4: Tell the
story
Where to start? Tools What are the steps?
26IEA 2019. All rights reserved.
www.iea.orgIEA #energyefficientworld
top related