i workshop wind globalgeo e 3tier - matt
TRANSCRIPT
Selection bias & the value of certainty
Presented by:
Matthew Hendrickson, Sr. Dir. Of Assessment
Renewable Energy
Risk Analysis
Powered by weather science, 3TIER helps the
global energy market manage renewable
energy risk—for the next 5 minutes through
the next 50 years.
3TIER is Risk Management
What to do about Risk?
What is Risk?
» Risk is often described as uncertainty.
» A typical wind consultant will describe uncertainty as the
standard deviation of error around the primary estimate.
A few questions
» Do we know what this means?
» How does knowledge of risk guide our decisions?
σ = ?
US Wind Industry Underperformance
In 2008, reports by three US consultants
show that US Wind Industry is
underperforming estimates by 10-11%.
1. “Understanding and Closing the Gap on Plant Performance” – Eric White, AWS Truewind, AWEA WindPower 2008
2. “Project Underperformance: 2008 Udate” – Steve Jones, DNV-GEC, AWEA WindPower 2008
3. “Validation of Energy Predictions by Comparison to Actual Performance, Clint Johnson, AWEA WindPower 2008
Underperformance explained
» No single factor described as major source. Rather a list of
possibilities has been suggested.
› Wind farm availability
› Inter-annual variability
› Turbine performance
› Wake effects
› Wind flow modeling
› Measurement bias
» Advancements in every one of the these fields has taken
place and US performance gap is closing.
Seems like a bad case of luck
» With so many sources, there is a nagging feeling like much
of what is wrong is luck.
» Could it be luck?
» Or something much more predictable…
» Perhaps it could be Selection Bias
Selection Bias Game
Project Selection Game
» You work for a large wind developer working in an active
market.
» As Chief Investment Officer, you are tasked to manage
your companies wind investments.
» All of your investments come in the form of winning
RFP’s to sell power in a power purchase agreement
(PPA). With a PPA, your company can build the project.
» Each project costs $100 in capital. Your companies goal
is to make a desired return of $20 for each $100.
Game mechanics
» Each Investor must pick a project and bid a price that they
think will both win and yield a desired return.
» Lowest price wins.
» True NCF is hidden.
Windy Hill Desert Breeze Ocean Front Gusty Pass Bent CornTrue NCF (hidden) 36.3% 35.0% 37.7% 32.5% 37.9%
Uncrtainty 8.3% 9.2% 9.4% 7.7% 7.5%
UncertainNCF 34.6% 27.9% 39.6% 35.2% 42.9%
$100 56.10 68.18 47.13 55.13 41.21
$110 60.77 72.85 51.80 59.80 45.88
$120 65.45 77.52 56.47 64.47 50.56
$130 70.12 82.19 61.15 69.15 55.23
$140 74.79 86.87 65.82 73.82 59.90
$150 79.46 91.54 70.49 78.49 64.57
Game mechanics
» Your desired return is $120.
» But your CEO becomes very unhappy with returns less than
$110. Since you can’t be fired, he will take investment
capital away from you for low returns and put it somewhere
else.
» He will take away $4 of capital for every $1 below the
threshold return.
» Each projects return gets added to an investors capital.
» The investor with the most capital at the end of the game
wins!
Game mechanics Windy Hill Desert Breeze Ocean Front Gusty Pass Bent Corn
True NCF (hidden) 36.3% 35.0% 37.7% 32.5% 37.9%
Uncrtainty 8.3% 9.2% 9.4% 7.7% 7.5%
UncertainNCF 34.6% 27.9% 39.6% 35.2% 42.9%
$100 56.10 68.18 47.13 55.13 41.21
$110 60.77 72.85 51.80 59.80 45.88
$120 65.45 77.52 56.47 64.47 50.56
$130 70.12 82.19 61.15 69.15 55.23
$140 74.79 86.87 65.82 73.82 59.90
$150 79.46 91.54 70.49 78.49 64.57
» Watch out for uncertainty!
» It might make a project look better or worse than it really is.
» Each projects true NCF varies. That might be a windy
project or it might be a dud.
Play the game
Launch Game
Selection bias model
» Selection model created that simulated selection
» Factors included
› Energy Estimate Uncertainty (Estimator of Risk)
› Weight of NCF in Selection Decision (How much do you care?)
› Market Pressure (Forces outside of your control)
› Selection Pool Size (How many choices?)
» Model ran iteratively across range of values
» Simulated decision process of investor
Selection Model Results
Assumes 30% NCF Weight & 5 project selection pool
Selection Model Results
Assumes 9% uncertainty, 1.1x market threshold, 30% NCF Weight & 5 project selection pool
How to protect yourself
Conclusion – whenever project return is a consideration
(almost always), selection bias will be present.
» Act with conservatism
› Apply conservative factors where appropriate
› But these factors should be transparent to avoid double-dipping
» System must resist uncertainty
› Investing on the P50 level flawed
› Pick a risk level (PX) that is comfortable and stick with it. (e.g.
Invest at the probability level such that your P95 scenerio is at least
a break even scenerio)