sensitivity and importance analysis risk analysis for water resources planning and management...
TRANSCRIPT
Sensitivity and Importance Analysis
Risk Analysis for Water Resources Planning and Management
Institute for Water Resources
2008
Sensitivity Analysis DefinedStudy of how the variation in the output of a model can be apportioned, qualitatively or quantitatively, to different “sources of variation”
Including assumptionsInput uncertaintyScenario/model uncertainty
The PointComplex analysis may have dozens of input and output variables that are linked by a system of equationsAnalysts and decision makers must understand the relative importance of the components of an analysisSome outcomes and decisions are sensitive to minor changes in assumptions and input values
Sensitivity AnalysisIf it is not obvious which assumptions and uncertainties most affect outputs, conclusions and decisions the purpose of sensitivity analysis is to systematically find this out
Systematic Investigation of…
Future scenariosModel parametersModel inputsAssumptionsModel functional form
Why Sensitivity Analysis?Provides understanding of how output variables respond to changes in model inputs Increases confidence in analysis and its predictions
Assumptions SensitivityList the key assumptions (scenarios) of your analysisExplore what happens as you change/drop each one individually
Do your answers change?
Challenging assumptions can be effective sensitivity analysis
Sensitivity Analysis MethodsDeterministic one-at-a-time analysis of each factorDeterministic joint analysisScenario analysisSubjective estimatesParametric analysis--range of valuesProbabilistic analysis can be used for importance analysis
One-At-A-Time AnalysisHold each parameter constant
Expected valueRepresentative value
Let one input varyAssumptionInputParameter
Common, useful, dangerous
One-At-A-Time AnalysisDo not equate magnitude with influenceA=U(107,108), B=U(2,6)C = A + B; A dominatesC = AB; B dominates
One-At-A-Time AnalysisDependence and branching in model creates flaws with this logic
If A<50 thenC = B + 1Else C = B100
What value do we set A equal to?
Joint AnalysisChange combinations of variables at same timeEnables analysts to take dependencies explicitly into accountCan have same limitations as OAAT analysis
Subjective EstimatesSubjective estimates of uncertain values can be used to identify threshold values of importance to the risk assessment
Range of ValuesA specific (not subjective) range of values is used
E.g., 10th, 50th, 90th percentiles
Ceteris paribus approachAll possible combinations approach
All 10th percentiles, 10th with 90th and so on
Importance AnalysisHow much does each model input contribute to the variation in the output?Typically a few key inputs account for most output variation
These are your important inputs.
Not particularly good at identifying nonlinear or multivariate relationships
Habitat Units CreatedP
rob
ab
ility
HUs
0.000
0.005
0.010
0.015
0.020
0.025
0.030
0.035
0.040
30 60 90 120
Regression Sensitivity for Grand Total HUs/D14
Std b Coefficients
B=>15 Degrees C / Use this.../L10 .071
V5: water temperature / Me.../H9 .073
V6: Dissolved oxygen / Mea.../N22-.075
V15: Pool class / Use this.../O26-.082
V6: Dissolved oxygen / Mea.../H22-.105
B=>15 Degrees C / Use this.../O10-.106
V15: Pool class / Use this.../H26-.117
V5: water temperature / Me.../N9 .147
V6: Dissolved oxygen / Mea.../B22-.157
V5: water temperature / Me.../E9 .16
V5: water temperature / Me.../K9 .162
A=resident rainbow trout /.../X6-.163
B=>15 Degrees C / Use this.../U10-.194
B=>15 Degrees C / Use this.../E10 .22
A=resident rainbow trout /.../L6-.287
A=resident rainbow trout /.../R6-.45
-1 -0.75 -0.5 -0.25 0 0.25 0.5 0.75 1
Statistical MethodsApportion variation in output to inputs via
Regression analysisAnalysis of varianceResponse surface methodsFourier amplitude sensitivity test (FAST)Mutual information index (MII)Classification and regression trees (CART)
So What?When decision is sensitive to changes or uncertainties within realm of possibility then more precision and additional information may be required
More data (research)Better modelsConservative risk management
Take Away Points“What if” analysis is essential to good risk assessmentSystematic investigations of model parameters, model inputs, assumptions, model functional formEssential to good risk management
One-At-A-Time Analysis akaNominal range sensitivity analysis (NRSA) .Individually varying one model input across its range of plausible values holding all other inputs at nominal or base-case values Resulting difference in model output is called the sensitivity or swing weight of model to the varied input
Automatic Differentiation (AD). Systematic evaluation of partial derivative of model output with respect to a given model inputSimilar to NRSA
Only an arbitrarily small change is considered, rather than a possible range of values
Provides indication of local sensitivity.