risk assessment tor stålhane ntnu / idi. what is risk - 1 risks are characterized by three factors:...
Post on 22-Dec-2015
219 views
TRANSCRIPT
Risk assessment
Tor Stålhane
NTNU / IDI
What is risk - 1
Risks are characterized by three factors:
• They are concerned with events that may – or may not – happen in the future.
• The events are identifiable but their effect and probability are uncertain.
• The outcome of the events can be influenced by our actions
What is risk - 2
A risk is something that can be a problem in the future. It is defined by two parameters
• The probability - p. What is the probability that the risk will become a problem?
• The consequences - C. What will happen if the risk becomes a problem?
The risk – R – is defined as R = C*p
How large is the risk - 1
In order to find the size of a risk, we need values for p and C.
In some cases we can estimate these values from historical data but in most cases we will have to use expert opinions or other subjective data sources.
It is not always possible – or meaningful – to assign a numerical value to a consequence, e.g. loss of lives.
How large is the risk - 2
Even though assessment is a subjective activity it is not about throwing out any number that you like.
To be useful, an assessment must be• Based on relevant experience.• Anchored in real world data, e.g. “How bad can it
get?”• The result of a documented and agreed-upon
process. Having a process makes it possible to later improve the process based on experiences.
Assessing risk
The quality of an assessment increases when the background info gets more specific.
Don’t ask: “What is the consequence of X?” or “What is the probability of Y?”
It is better to ask: “What is the consequence of X in scenario S?” or “What is the probability of Y in scenario S?”
Assessment and scenarios - 1
If the probability of scenario Si is p(Si), and pi and Ci are the probability and consequence of an accident in scenario Si, we have that:
iii CpSpR )(
The method is critically dependent on the • Quality of the scenario descriptions• Independence of the scenarios
Assessment and scenarios - 2
We can improve our assessments even more if we do not ask for consequences in general but for consequences for one particular asset. Thus, in scenario i we have consequence Cj,i for asset j.
jijii CpSpR ,)(
Assessing C and p
We can assess consequences and probabilities in several ways:
• Textual categories – e.g. High, Medium, Low.• Numerical categories – e.g. values from 1 to 10. • Value intervals.• Statistical distributions.
Textual categories – 1
When using categories, it is important to give a short description as to what each category implies. E.g. it is not enough to say “High consequences”. We must relate it to something already known, e.g.
• Project size
• Company turn-over
• Company profit
Textual categories – 2
Two simple examples:
• Consequences: we will use the category “High” if the consequence will gravely endanger the profitability of the project.
• Probability: we will use the category “Low” if the event can occur but only in extreme cases.
The CORAS consequence table
Consequence values
Category Insignificant Minor Moderate Major Catastrophic
Measuredrelated toincome
0.0 – 0.1% 0.1 – 1.0% 1 – 5% 5 – 10% 10 – 100%
Measuredloss due toimpact onbusiness
No impact onbusiness. Minor delays
Lost profits
Reduce theresources of oneor moredepartmentsLoss of a coupleof customers
Close downdepartments orbusinesssectors
Out ofbusiness
The CORAS frequency tableFrequency values
Category Rare Unlikely Possible LikelyAlmostcertain
Number ofUnwantedincidents perYear
1/100 1/100 – 1/50 1/50 - 1 1 - 12 > 12
Number ofUnwantedincidents perDemand
1/1000 (1/500) 1/50 (1/25) 1/1
Interpretationof number ofdemands
UnwantedincidentneverOccurs
Eachthousandtime thesystem isused
Each fivetimes thesystem isused
Each tenthtime thesystem isused
Everysecondtime thesystem isused
Consequence and probability - 1
Consequence
Probability H M L
H H H M
M H M L
L M L L
Consequence and probability - 2
The multiplication table is used to rank risks. It can not tell us how large they are.
We should only use resources on risk that are above a certain, predefined level.
Numerical categories -1
We can use numbers instead of names. This does not make the assessment more precise but will free us from the need to define a multiplication table in order to identify risks.
In principle we can use any numbers. The best solution is, however, to just assign number to the three aforementioned categories
Numerical categories – 2
The following values are often used in practice, both for consequences, benefits and probabilities:
• 10 – high
• 4 – medium
• 1 – low
Thus, a medium consequence and a low probability will give a risk of 4*1 = 4.
Numerical categories – 3
Consequence
Probability H / 10 M / 3 L / 1
H / 10 H / 100 H / 30 M / 10
M / 3 H / 30 M / 9 L / 3
L / 1 M / 10 L / 3 L / 1
Value intervals
If we have more info available we can give better estimates. Even though we cannot give exact values, we can give our assessments as intervals.
An interval has a start and an end value – denoted a and b. We denote the interval I as I = [a, b]
In our case, the width of the interval is a measure of our uncertainty.
Simple interval arithmetic
As long as all interval limits are positive, we can write:
• I = I1 * I2, I = [a1*a2, b1*b2] • I = I1 + I2, I = [a1 + a2, b1 + b2] • I = I1 - I2, I = [a1 - a2, b1 - b2] • I = I1 / I2, I = [a1*b2, b1/a2]
If we use intervals for consequence (C) and probability (p) we get
R = [C1*p1,C2*p2]
Statistical distributions - 1
We can use statistical distribution for C and p. In this case, the distributions are used to show our uncertainty.
Practical solutions could be:
• Beta distribution for p
• Gamma distribution for C
Statistical distributions - 2
Based on the distributions of p and C, we can compute the distribution of the risk in three ways:
• Mellin transforms
• Monte Carlo simulation
• Approximation methods
We will only look at the third alternative.
Statistical distributions - 3
The following approximation holds:
)()(
)(),...(),()(
),,...,(
1
2
21
21
i
n
i i
n
jin
xVarx
fYVar
xExExEfYE
xindxxxxfY
Risk approximation
Using the expressions from the previous slide we get the following approximations:
)()()()()(
)()()(22 CVarpEpVarCERVar
CEpERE
It is now straight forward to find the expected value and variance for R
Simple risk assessment
In order to a simple risk assessment we need to identify:
• Dangerous events
• Each event’s – consequence – C– probability – p
• Possible barriers – changes or controls
• Person responsible for each risk - Resp.
Simple risk table
Event C p R Barriers Resp
Events
We start by identifying dangerous events. The simple way to do this is to use brainstorming – just sit down and envisage your worst nightmares related to the activities under consideration.
Be realistic – only consider things that you believe can happen.
Barriers
Barriers can be realized through:
• Prevention – we change the system so that the event cannot occur.
• Mitigation – we can– change the system in order to reduce the
event’s probability or consequences.– define activities that will reduce the problems
if the event occurs.
Bar
rier
1 Bar
rier
2 Bar
rier
3 Bar
rier
4 Bar
rier
5 Bar
rier
6
Risk Prob. Event
Prevention barriersPrevent risk from becoming a problem
Handling barriersPrevent event from having bad consequences
Reduction barriersReduce effect of event
Benefits
It is important to bear in mind that:
• We usually expect to gain something through change – new products, new ways to work etc.
• Risks stem from changes.
• Reducing risk is a cost factor
We need to look at the total picture.
The total picture - 1
The total picture of the situation shows the risks and the benefits that stem from a planned change.
This is not a mechanism that can be used to identify the best solution.
It is, however, an important input when we want to make a decision.
The total picture - 2
The total picture shows risks and benefits. Risk can be shown in two ways:
1. Unmitigated risks
2. Mitigated risks – include the effect of risk reduction activities, e.g. barriers. This can be done by
– Modifying the risk assessment– Indicate how the risk will move in the
diagram
Consequences and benefits
B
HReduced number of MMI-related defects
M
L
p L M H
C
LExtra work needed for MMI-specification
M
H
Unmitigated risks
B
HReduced number of MMI-related defects
M
L
p L M H
C
LExtra work needed for MMI-specification
M
HLarge disagreements between designers and MMI experts
Partnership does not work
The mitigation effect
B
HReduced number of MMI-related defects
M
L
p L M H
C
LExtra work needed for MMI-specification
M
HLarge disagreements between designers and MMI experts
Partnership does not work
1
2
Including benefits
B
HReduced number of MMI-related defects
Better MMI for existing products
Better MMI requirements will reduce imp. costs
M
L
p L M H
C
L Extra work needed for MMI-specification
M
HLarge disagreements between designers and MMI experts
Partnership does not work
1
2
C and p as intervals - 1Benefit
Consequence
p
C and p as intervals - 2Benefit
Consequence
p
Mitigation effectCost of mitigationand benefits’ value and probability
Increased value or probability
The tyranny of “either – or”
All too often we are confronted by the statement that we can get only get X if we are willing to suffer Y.
This is the wrong attitude. The right attitude is that we will
1. Do what is needed to get X
2. Perform activities that will remove or reduce the bad effects of Y.
Leverage
Leverage is a prioritizing mechanism:
Leverage = (Benefit – Cost) / Cost
Leverage will prioritize activities with
• Large net benefits
• Small costs
Extended risk table -1
We can use cause – consequence chains or event trees for a risk to identify the best place to insert a barrier.
For each barrier, we need to assess:
• Cost - the cost of implementing it. We will use the scale H = 10, M = 3 and L = 1.
• E – how effective is the barrier? We will use the scale h = 1.0, m = 0.5 and l = 0.2
Extended risk table - 2
Event C p R Barrier Cost E L Resp.
Barrier leverage
Leverage = (C*p*E – Cost) / Cost
The leverage will prioritize barriers which:
• Have low costs – Cost is small
• Have high efficiency – E is large
• Attack important risks – C*p is high
Barrier – example Event Cons
.p R Mitigation E Cost
LResp
Partnership doesnot work – businessconflicts
10 3 30
Do a thorough researchon selected partner’sbusiness goals
0.5 10 0.5
John
Customers do notprioritize projectparticipation 10 3 30
State the conditions andconsequences of customerparticipation in thecontract
1.0 3 9.0
Pete
Some comments on barriers
It is important to remember that:• Each risk will usually need a different barrier – a
barrier that works against one risk can be valueless against another risk.
• It is important to consider the three main barrier strategies:– Prevent the risk from becoming a problem– Control the problem to avoid the consequences– Reduce the consequences
ALARP and GALE
There are two competing principles in the assessment of risk:
• ALARP – As Low As Reasonably Possible- We have done all that is reasonable to prevent problems and dangers.
• GALE – Globally At Least Equivalent. E.g. introducing a new process will not increase the risks compared to what it is today.
ALARP
ALARP requires that we analyze each risk separately and then implement mitigation activities.
A reasonable goal is to reduce each risk until the extra mitigation costs exceed the value of the risk reduction achieved.
All that we have seen up till now fits into an ALARP policy .
GALE
GALE requires us to look at the total risk of a change. In this way we can start by attacking the cheapest risk or the risk with the largest leverage.
The problem with the GALE principle is that we need to perform arithmetic on risks. E.g. we need to decide how many medium risks we need before we have a large risk
ALARP vs. GALE
The one important thing with using the GALE principle is that it forces us to ask “What is the current risk level?”
All too often we act as it the current way of doing things is risk free and all risk stems from changes.
This stance is enforced by the human tendency to underestimate the risk of status quo.
Using GALE
Important points
• GALE is a method for risk analysis. Benefits must be included elsewhere
• We need to look at both our current risk and the risk resulting from the proposed changes.
• Always perform a sensitivity analyses.
Risk – status quo vs. change
In many cases, maybe even in most of them, we do risk assessment because we want to compare two or more alternatives, e.g.:
• Status quo – no changes
• One or more changes - improvements
Event identification
• All significant dangerous events must have been identified.
• There must be a minimal overlap between the dangerous events .
• There must be a maximum of commonality between the dangerous events considered for the status quo and for the system after the proposed changes
The three event sets
The previous rules split the dangerous events into three sets – dangerous events that:
• Apply both to the status quo and to the new system.
• Are unique to the status quo
• Are unique to the new system
GALE and risk assessment - 1
GALE uses the following parameters for risk assessment:
• FE – the event frequency
• PE – the probability that the event will lead to an accident
• S – the severity score of an event
GALE and risk assessment - 2
We can compute individual and accumulated risk indices:
IE = FE + PE + S
IGR = log Sumi(10I)
IE is the risk index for a hazardous event
IGR is the global risk index
The GALE scoring scheme
The scoring scheme of GALE • Focuses on deviations from current
average. This is reasonable, given that it is mainly concerned with comparing status quo to a new situation.
• Must be tailored to each situation. The next slide shows an example from road safety. We need a scheme adapted to SPI.
Road safety - frequency score for event
Frequency classification
Occurrences / year on M42 ATM section FE
Very frequent 10000 Hourly 6
Frequent 1000 A few times a day 5
Probable 100 Every few days 4
Occasional 10 Monthly 3
Remote 1 Annually 2
Improbable 0.1 Every 10 years 1
Incredible 0.01 Every 100 years 0
Frequency score for event Frequency
classificationOccurrences per project FE
Very frequent 200 Every project 6
Frequent 100 Every few projects 5
Probable 40 Every 10th project 4
Occasional 10 Every 100th project 3
Remote 1 A few times in the company’slifetime
2
Improbable 0.2 One or two times during thecompany’s lifetime
1
Incredible 0.01 Once in the company’slifetime
0
Probability score for event
Classification Interpretation PE
Probable It is probable that this event, if it occurs, will cause a problem 3
Occasional The event, if it occurs, will occasionally cause a problem 2
Remote There is a remote chance that this event, if it occurs, will cause a problem
1
Improbable It is improbable that this event, if it occurs, will cause a problem 0
Severity score for event
Severityclassification
Interpretation S
Severe The portion of occurring problems thathave serious consequences is muchlarger than average
2
Average The portion of occurring problems thathave serious consequences is similarto our average
1
Minor The portion of occurring problems thathave serious consequences is muchlower than average
0
Sensitivity analysis
The global risk index is made of many indices. Each index will have a certain degree of uncertainty connected to it.
Usually, a few indices will have a large influence on the result while the rest will have but little influence.
Pareto’s rule applies - we need to identify the few important indices.
Important things to remember - 1The most important things to remember:• Risk assessment is by its nature subjective. • Use group techniques and include all
stakeholders• Use simple techniques so that you do not
exclude one or more stakeholders• Anchor it in experience and available data will,
however, improve the quality• Subjective values like “High” must be anchored
in each company’s reality. One company’s “High” may be another company’s “Low”.
Important things to remember - 2
• Include the effect of choosing status quo in all risk analyses.
• Always include opportunities• Consider the three barrier categories –
prevention, handling and reduction• Rank risks and opportunities according to
their leverage• The results from a risk assessment is just
one of several inputs to a decision