gsa-wa perth 2006
Post on 12-May-2015
635 Views
Preview:
DESCRIPTION
TRANSCRIPT
Oliver Kreuzer
Centre for Exploration TargetingThe University of Western Australia
Risk, Uncertainty and Bias:
Rulers over Exploration
Success and Failure
Acknowledgements
Mike Etheridge, Maureen McMahonGEMOC Key Centre, Macquarie University
Colin Wastell, Gillian LucasDepartment of Psychology, Macquarie University
Presentation outline
Aspects of our business Performance, low base rate situation, low probability of success
Risk, uncertainty and decision analysis Definitions of risk and uncertainty
What is decision analysis?
The psychology of decision-making Common heuristics and biases
What is their impact on the process of decision-making?
Outlook What can we learn from the petroleum industry?
Mineral exploration Business aspects
Randolph (2002)
Mineral exploration Business aspects
Bosma (2003)
Mineral exploration Business aspects
Economic activity
As such expected to provide acceptable returns to investors
However, probability of success so low and geological uncertainty so
high that it has proven difficult to manage for financial success
Mineral exploration Business aspects
At best a break-even proposition
Schodde (2003, 2004)
• Compiled NPVs of 109 major Australian gold projects (1985–2003)
• NPVs = $4.74 billion; costs of finding / evaluating $4.64 billion
• Average return of $1.02 per $1 dollar spent on exploration
Leveille & Doggett (in press, Economic Geology Special Publication)
• Measured costs + returns from 65 Chilean copper projects (1950–2004)
• Only 14 generated sufficient returns to offset their exploration costs
• Overall return below breakeven
Mineral exploration Business aspects
Problem: Low base rate situation
Exploration is an example of a low base-rate situation, i.e. there is a
low rate of occurrence of ore deposits in individual targets
High number of drill holes per discovery
Based on Schodde (2003)
Data exclude follow-up drilling!
Mineral exploration Business aspects
Low chance of proceeding to the next stage
Rio Tinto
100%
10%
0.3%
0.06%
0.03%
Kennecott
10%
10%
10%
Project generation
Build an expert team for the belt
Establish data base and management system
Define prospect risks
Test presence of mineralizing system
Test geologic and mineralization models
Test geologic information
Test potential of mineralizing system
Establish size and grade potentail
Establish controls on grade distribution
Feasibility
Determine project costs
Determine NPV
Stage Milestones / Aims
Prospect definition(reconnaissance)
Systematic drilltesting of targets
Resource delineation
Select and acquire ground in well endowed belts
Build area knowledge
Test continuity
Establish economic / metallurgical parameters
Mineral exploration Business aspects
Measuring Exploration Success in a Brownfields Environment(Laverton District, WA)
0.00
0.20
0.40
0.60
0.80
1.00
A B C D E F
Stage
Pro
ba
bili
ty
0.0
5.0
10.0
15.0
20.0
25.0
30.0
A$
Mill
ion
Total expenditure Average cost per prospect Probability of advancing from previous stage
Lord et al. (2001)
Mineral exploration Business aspects
Parry (2001)
Mineral exploration Business aspects
Parry (2001)
Observation 1
For a some companies exploration has been very lucrative; huge
profits were made when they reached the ultimate goal of mining
success
However, on average, mineral exploration appears to be a break-
even proposition – or worse…
The studies of Schodde and Leveille & Doggett illustrate that we
need to measure exploration performance if we want to improve it
• E.g. Schodde (2003): As a rule of thumb, we should aim to find gold for
less than A$15/oz. This is twice as good as the current average.”
Risk
Variability of possible returns
As measured by their standard deviation
• Risk includes but is not limited to chance of making a loss
• Risk equals opportunity
Probability of failure
PFailure = 1 – PSuccess
• Risk can be estimated if we can assign a value to PSuccess
• Risk can be reduced if we can find ways of improving our PSuccess
e.g. Singer & Kouda (1998), Guj (2005)
)()(( 21 iPmeanxSUMSD ini
Uncertainty
Definition
A measure of our inability to assign a single value to risk
Types of uncertainty
Inherent natural variability of geologic objects and processes
Conceptual and model uncertainty
Errors / inaccuracies / biases that occur when we sample,
observe, measure or mathematically evaluate geological data
e.g. Bardossy & Fodor (2001), Purvis (2003)
Uncertainty
Most decisions we make in mineral exploration are
made under conditions of significant uncertainty
the performance of ourtargeting tools
Imperfect knowledge ofof geological systems
the tenement or prospectgeology
our exploration / ore depositmodels
interpretation of targetingparameters
the grade, continuity orextent of mineralisation
controls on localisation ofmineralisation
Inherent variability of geo-logical objects / processes
Limitations and biases ofgeological investigations
Uncertainty about
Uncertainty
Uncertainty has rarely been estimated or quantified for our models,
maps or sections
In fact, many geological products imply a level of certainty that is
simply unrealistic
This is a major impediment to mineral exploration If we don’t estimate or determine uncertainty we won’t be able to
quantify and evaluate exploration risk
Figures from Shatwell (2003)
Decision analysis
Identify what choices or alternatives are available
Identify the possible outcomes for each alternative
Estimate the value of each possible outcome
Estimate the probability of each possible outcome
Calculate the weighted average value for each choice
Make the decision
e.g. Newendorp & Schuyler (2000)
Decision analysis
Does not eliminate or reduce risk
Helps us to evaluate, quantify and understand risk
Helps us choose the alternative that offers the best risk / reward ratio
Does not replace professional judgment
Helps us to communicate geological risks and uncertainties
• without ambiguity, and
• in terms of probabilistic and monetary values
e.g. Newendorp & Schuyler (2000)
Decision analysis
Is decision analysis only for the majors?
To expensive (software, consultant fees) and too time consuming
(compilation of input values) to be practical for juniors?
In my opinion – No.
Juniors face the same risk and uncertainty as the majors
The junior business model is even more vulnerable to gambler’s ruin
(limited risk capital, limited diversity of portfolio, few projects)
A quick and dirty analysis is still better than failure to manage risk
Observation 2
Mineral exploration is a business bedeviled by uncertainty
Yet, many of our outputs and decision-making processes imply a
level of confidence that is simply unrealistic
For effective, formal risk management to take place we have to
estimate, measure or calculate geological uncertainties
Decision analysis provides us with simple, effective tools for choosing
the best course of action under conditions of uncertainty
Psychology of decision-making
Intuitive
The inherent geological complexities and uncertainties in exploration
clash with rational decision-making
Hence, we tend to rely extensively on intuitive thinking and judgment
Biased
This Intuitive thinking is subject to a well understood set of mental
short cuts (heuristics) and systematic errors (biases)
Psychology of decision-making
The Two-Systems View Recognizes that we use 2 main types of cogitive process
System 1
IntuitionSystem 2
Reasoning
FastParallelAutomaticEffortlessAssociativeSlow-learningEmotional
SlowSerialControlledEffortfulRule-governedFlexibleNeutral
Pro
ce
ss
e.g. Kahneman (2003)
Psychology of decision-making
A stamp and an envelope cost $1.10 in total.
The stamp costs $1 more than the envelope.
How much does the envelope cost?
e.g. Kahneman (2003)
Psychology of decision-making
Most people intuitively answer 10 cents
$1.10 separates naturally into $1 and 10 cents
10 cents is about the right magnitude
But, envelope = 5 cents, stamp = $1.05
Implications of such cognitive tests
Monitoring of System 1 by System 2 is generally quite lax
We tend to offer answers without checking them
We are not used to thinking hard and often trust a plausible judgment
that quickly comes to mind
e.g. Kahneman (2003)
Heuristics
What are heuristics? Rules of thumb or mental shortcuts
Pros Very effective, automatic processes
Reduce the time and effort of decision-making
Lead to reasonable decisions in many situations
Cons Frequently bias our perception impact on System 1
Cause severe and systematic errors of judgment
Worse when we are under time pressure / multitasking
e.g. Kahneman (2003)
Heuristics
Common types of heuristics
Representiveness
Framing
Anchoring and adjustment
Availability
e.g. Kahneman (2003)
Heuristics Representativeness
Representativeness heuristic
Our tendency to overgeneralize from a few characteristics or
observations
We often judge whether an object (X) belongs to a particular class
(Y) by how representative (or similar) X is of Y
Source of multiple biases
Base rate neglect
Gambler’s ruin
Overconfidence
e.g. Kahneman (2003)
Heuristics Representativeness
Base Rate Neglect: an example
We know that 1 in 100 targets delivers a gold discovery
A new targeting method has been developed
It is practical only over small areas (i.e. known targets)
• Generates an anomaly in 90% of test cases over known deposits
• Delivers a null result in 90% of test cases in barren areas
Exploration companies run it over a total of 1,000 targets
What is the likelihood that it will correctly identify a deposit?
Example based on Nick Hayward (BHP Billiton), 2003 AIG Symposium
Example based on Nick Hayward (BHP Billiton), 2003 AIG Symposium
Heuristics Representativeness
Answer: 8.3%
True Positives : Total Positives = 9 : 108 = 0.083
# of Targets
= 1000
Targets = deposit
= 10
Target ≠ deposits
= 990
Anomalies
= 108
True positives
= 9
False positives
= 99
No anomaly
= 892
False negatives
= 1
True negatives
= 891
Heuristics Representativeness
Exploration: example of a low base rate situation
Base rates should be the main factor in our estimations
However, we tend to ignore prior probabilities when other targeting
parameters seem more relevant
Consequences
Our targeting models need to focus on those parameters that have
relatively low false positive rates
Wasting time and money on false positives is one of our industry’s
main contributors to poor performance
e.g. Hronsky (2004), Etheridge (2004)
Heuristics Representativeness
Gambler’s ruin (gambler’s fallacy)
Wins are perceived more likely after we suffered a string of losses
Example: tossing a fair coin
After H turned up 9× in a row, is it more likely that T will turn up next?
No, the odds are exactly the same for every single toss
Each toss of the coin is an independent event
The coin has no memory of the past 9 tosses
e.g. Busenitz & Barney (1997), Roney & Trick (2003)
Heuristics Representativeness
Small sample of tosses very likely for the number of H and T
outcomes to be unequal
Only in the long run will those outcomes equalize
Example: probability of gambler’s ruin
Sufficient capital for 5 trials, each @ Psuccess = 0.1 (or 10%)
What is the probability of at least 1 success in 5 trials?
Equation:
e.g. Busenitz & Barney (1997), Roney & Trick (2003); Example by Guj (2005)
xnxnx
nx PPCP )1()(
Heuristics Representativeness
• Where (Cnx) = n! / [x! × (n – x)!]
• (P15 ) = [(5! / 1! × 4!) × 0.1 × 0.94 + … + (5! / 4! × 1!) × 0.14 × 0.9 = 0.4099
• PGambler’s ruin = 1 – 0.4099 = 0.5901 or 59% chance of going bust!
• If PSuccess = 0.01 PGambler’s ruin = 0.9509 or 95% chance of failure!
Consequences
Spending too much on too few prospects is extremely risky
A streak of bad luck does not mean that we are due for successExample by Guj (2005)
Heuristics Framing
Framing heuristic
Our tendency to process information depending on how this
information is presented (or framed)
Consequences
Most judgements and decisions are guided by information derived
from the rarest events in our business – discoveries
We should start thinking outside the box by framing decisions with
information derived from the bulk of our projects – those that failed
e.g. Kahneman (2003)
Heuristics Anchoring and adjustment
Anchoring and adjustment heuristic
We tend to base our initial estimates on any value we have at
hand (anchor), regardless of its relevance
We then adjust our estimate until we reach a final value
Our adjustments are typically insufficient, narrow and biased
towards the value of the anchor
e.g. Kahneman (2003), Welsh et al. (2005)
Heuristics Anchoring and adjustment
Consequences
Strong anchoring to specific exploration models means we
are less likely to find something that is different
We drill our best target in a project first; but when it fails, we
often lower our standards to justify drilling lesser quality
targets
Observation 3
Even after decades of cognitive research we continue to assume that
our intuition, experience and intelligence will guide us toward the best
possible decision under conditions of uncertainty
Yet, the opposite is true: we are prone to cognitive biases that
frequently prevent us from choosing the optimal course of action
Moreover, the situations of greatest uncertainty are the ones where
poor judgment is most likely to result in failure
Awareness of our limitations is the first critical step in developing
good decision-making procedures
cf. Bratvold et al. (2002), Purvis (2003)
Outlook The petroleum example
So, where should we go from here?
We could, for example, look at how our colleagues in petroleum
exploration have changed the fortunes of their industry
What can we learn from the petroleum example?
• That disciplined management of risk and uncertainty can generate value
and turn an industry around
• That prediction and visualization of subsurface geology can improve
success rates
• That holistic geological models that focus on “where” rather than “how”
can reduce uncertainty
Outlook The petroleum example
BP exploration 1983–2002
0%
10%
20%
30%
40%
50%
60%
70%
80%
1983
1985
1987
1989
1991
1993
1995
1997
1999
2001
Economic success rate
High-risk wells
Late 90’s
High-risk wells ~ 10%
Success rate > 50%
Onset of formalrisk assessment
Late 80’s
High-risk wells > 50%
Success rate < 20% Glenn McMaster (BP), 2003 SPE Distinguished Lecturer
Before
After
Outlook The petroleum example
Figures from Jones & Hillis (2003), Etheridge (2004), Cockcroft (2005)
Management ofrisk and uncertainty
Process-basedmodels
Visualization of subsurface geology
Outlook Probabilistic ore systems models
Risk management and ore deposit modeling
Holistic, flexible and process-based
• build on the petroleum and mineral systems approach (Geoscience Australia)
Probabilistic
• assign probabilities to critical success factors
• multiplication rather than addition of critical success factors to eliminate those areas
where one or more of these factors are absent
• value distributions instead of single values
Calibrated
• multiple realizations
• statistical assessment of sensitivity of outputs
Outlook Probabilistic ore systems models
Link models to decision structures + GIS E.g. decision trees, Monte Carlo simulation
0.3 0.5 0.7 0.9 1.1
5% 90% 5% .4824 .9174
0.3 0.5 0.7 0.9 1.1
Mean=0.700001
Distribution for Probability of trap being in the positi...
0.000
0.500
1.000
1.500
2.000
2.500
3.000
Mean=0.700001
0.3 0.5 0.7 0.9 1.1
EV outcome for comparison of potential project risks and rewards, regardless of project type, stage or location
“The successful explorers over the next decade will be those that
embrace effective risk management”
Marcus Randolph
President Diamonds and Specialty Products, BHP-Billiton, 2003
“After all, the risk in discovery is still the greatest single risk”
Siegfried Muessig
The Art of Exploration: SEG Presidential Address, 1978
“What we need in all our endeavors … is responsible risk taking
and what we want are the rewards of such responsibility”
Paul Bailly
Risk and the Economic Geologist: SEG Presidential Address, 1982
top related