sensit (1)
Post on 05-Apr-2018
219 Views
Preview:
TRANSCRIPT
8/2/2019 sensit (1)
http://slidepdf.com/reader/full/sensit-1 1/125
Sensitivity Analysis:a Validation andVerification Tool
Terry BahillSystems and Industrial EngineeringUniversity of Arizona
Tucson, AZ 85721-0020terry@sie.arizona.eduCopyright ©, 1993-2009 BahillThis file is located athttp://www.sie.arizona.edu/sysengr/slides/
8/2/2019 sensit (1)
http://slidepdf.com/reader/full/sensit-1 2/125
4/12/2012 2
ReferencesSmith, E. D., Szidarovszky, F., Karnavas, W. J. and
Bahill, A. T., Sensitivity analysis, a powerful systemvalidation technique, The Open Cybernetics and Systemics
Journal,
http://www.bentham.org/open/tocsj/openaccess2.htm,
2: 39-56, 2008, doi: 10.2174/1874110X00802010039
W. J. Karnavas, P. Sanchez and A. T. Bahill, Sensitivity
analyses of continuous and discrete systems in the timeand frequency domains, IEEE Trans. Syst. Man.
Cybernetics, SMC-23(2), 488-501, 1993.
© 2009 Bahill
8/2/2019 sensit (1)
http://slidepdf.com/reader/full/sensit-1 3/125
4/12/2012 3
You should perform asensitivity analysis anytime you
create a model write a set of requirements
design a system
make a decision
do a tradeoff study
originate a risk analysis
want to discover the cost drivers
© 2009 Bahill
8/2/2019 sensit (1)
http://slidepdf.com/reader/full/sensit-1 4/125
4/12/2012 4
In a sensitivity analysis change
the values of
inputs
parameters
architectural features measure changes in
outputs
performance indices
© 2009 Bahill
8/2/2019 sensit (1)
http://slidepdf.com/reader/full/sensit-1 5/125
4/12/2012 5
A sensitivity analysis can be used to validate a model,
warn of unrealistic model behavior, point out important assumptions,
help formulate model structure, simplify a model,
suggest new experiments, guide future data collection efforts, suggest accuracy for calculating parameters,
adjust numerical values of parameters,
choose an operating point, allocate resources,
detect critical criteria, suggest manufacturing tolerances,
identify cost drivers.© 2009 Bahill
8/2/2019 sensit (1)
http://slidepdf.com/reader/full/sensit-1 6/125
4/12/2012 6
History: the earliest sensitivity analyses The genetics studies on the pea by Gregor
Mendel, 1865. The statistics studies on the Irish hops crops by
Gosset (reported under the pseudonymStudent), ca 1890.
© 2009 Bahill
8/2/2019 sensit (1)
http://slidepdf.com/reader/full/sensit-1 7/1254/12/2012 7
Classes of sensitivity functions Analytic
for well defined systems usually partial derivatives
Empirical
show sensitivity to parameters
observe system changes whenparameters are changed
works for an unmodeled system
© 2009 Bahill
8/2/2019 sensit (1)
http://slidepdf.com/reader/full/sensit-1 8/1254/12/2012 8
Types of sensitivity functions
Analytic
absolute
relative
semirelative
Empirical direct observation
sinusoidal variation of parameters
design of experiments
Excel, using what-if analysis
© 2009 Bahill
8/2/2019 sensit (1)
http://slidepdf.com/reader/full/sensit-1 9/1254/12/2012 9
The absolute-sensitivity function
The absolute-sensitivity of the
function F to variations in theparameter is
It should be evaluated at the
normal operating point
(NOP).
NOP
F F
S
© 2009 Bahill
8/2/2019 sensit (1)
http://slidepdf.com/reader/full/sensit-1 10/1254/12/2012 10
Examples Absolute-sensitivity functions are used
to calculate changes in the output due to changesin the inputs or system parameters
to see when a parameter has its greatest effect
in adaptive control systems
© 2009 Bahill
8/2/2019 sensit (1)
http://slidepdf.com/reader/full/sensit-1 11/1254/12/2012 11
A process model example
x and y are inputs and A to F model system
parameters. The output z is love potion number 9.The normal operating point is
What is the easiest way to increasethe quantity of z?
This sounds like a problem for absolute-sensitivityfunctions.
2 2 z Ax By Cxy Dx Ey F
0 0 0 0
0 0 0 0
( , ) (1,1), 1, 2,
3, 5, 7, 8,
x y A B
C D E F
0 2. z
© 2009 Bahill
8/2/2019 sensit (1)
http://slidepdf.com/reader/full/sensit-1 12/125
Absolute-sensitivity functions*
4/12/2012 12 © 2009 Bahill
0 0 0 0 0NOP
0 0 0 0 0
NOP
2 0,
2 0.
z
x
z
y
z
S A x C y D x
zS B y C x E
y
2
0NOP
2
0
NOP
0 0
NOP
0
NOP
0
NOP
NOP
1,
1,
1,
1,
1,
1,
z
A
z
B
zC
z
D
z
E
z
F
z
S x A
zS y
B
zS x yC
zS x
D
zS y
E
zS
F
8/2/2019 sensit (1)
http://slidepdf.com/reader/full/sensit-1 13/1254/12/2012 13
What about interactions? Change two parameters at the same time.
Interactions can be bigger than the first-order effects.
Non-steroidal, anti-inflammatory drugs (NSAID) such
as Ibupropren and Aleve have dangerous interactionswith angiotensin converting enzymes, which effectthe kidneys and lower blood pressure. No
pharmacists would allow you to take both.
My mother once cleaned the toilet with sodium
hypochlorite (Clorox bleach) and ammonia. Itproduced chorine gas.
Alcohol and barbiturates are much more dangerousif mixed.
© 2009 Bahill
8/2/2019 sensit (1)
http://slidepdf.com/reader/full/sensit-1 14/125
Cooperation The performance of a system could be greater than
the sum of its subsystems (cooperation). Alone, neither a human nor a knife can slice bread.
Together a blind person and a Seeing Eye dog dobetter than either alone.
A pair of chopsticks performs more than twice aswell as an individual chopstick.
Two lions chasing a Thompkins Gazelle are more
than twice as likely to catch it, than a single lion.
4/12/2012 © 2009 Bahill14
8/2/2019 sensit (1)
http://slidepdf.com/reader/full/sensit-1 15/125
4/12/2012 15
Interactions in the process control model Change two parameters at the same time.
Mixed partial derivatives can be bigger than first-order partial derivatives.
Of the 64 possible second-partial derivatives, onlythe following are nonzero.
© 2009 Bahill
8/2/2019 sensit (1)
http://slidepdf.com/reader/full/sensit-1 16/125
Interactions*
4/12/2012 16 © 2009 Bahill
2
2
2
NOP
2
0
NOP
2
02
NOP
2
02
NOP
1,
3,
2 2,
2 4.
z
y E
z
x y
z
x
z
y
zS
y E
zS C
x y z
S A x
zS B
y
2
0
NOP
2
0
NOP
2
NOP
2
0
NOP
2
0
NOP
2 2,
1,
1,
2 2,
1,
z
x A
z
x C
z
x D
z
y B
z
y C
z
S x x A
zS y
x C
zS
x D
z
S y y B
zS x
y C
8/2/2019 sensit (1)
http://slidepdf.com/reader/full/sensit-1 17/1254/12/2012 © 2009 Bahill17
Table 1. Effects of Individual and Combined Parameter Changes
for some second-order interaction terms with delta of 1.0, where
0 x x x etc.
FunctionsNormal
values
Valuesincreased
by one
unit
New z
values z
Total
change
in z
( , ) f x A A=1
x=1
A=2
x=27 5 5
0( , ) f x A A=1 A=2 3 1 2 z 0( , ) f x A x=1 x=2 3 1
0 0( , ) f x A A=1
x=12 0
( , ) f y B B=2
y=1
B=3
y=28 6 6
0( , ) f y B B=2 B=3 3 13 z
0( , ) f y B y=1 y=2 4 2
0 0( , ) f y B B=2
y=12 0
The purpose of
this slide is to
show the affectsof interactions,
without using
mathematics.
We will now usethese data to
estimate the value
of one of the
mixed-partial
derivatives
8/2/2019 sensit (1)
http://slidepdf.com/reader/full/sensit-1 18/125
Estimate themixed-second-partial derivative
4/12/2012 © 2009 Bahill18
2
0 0 0 0 0 0( , ) ( , ) ( , ) ( , ) ( , ) f f f f f
2 7 3 3 23
1
z
x A
This is the wrong answer.Analytically we found that the correct value is 2.
8/2/2019 sensit (1)
http://slidepdf.com/reader/full/sensit-1 19/125
A smaller step size, 0.01
4/12/2012 © 2009 Bahill19
Use the following general equation from Smith, Szidarovszky, Karnavas and Bahill [2008].
0 0 0 0 0 0 0 0
2 2
0 0 0 0
( , ) ( , ) ( , )( ) ( , )( )
1( , )( ) 2 ( , )( )( ) ( , )( )
2!
x y
xx xy yy
f x y f x y f x y x x f x y y y
f x x f x x y y f y y
Converting to find the value if we change x and A yields
0 0 0 0 0 0 0 0
2 2
0 0 0 0
( , ) ( , ) ( , )( ) ( , )( )1
( , )( ) 2 ( , )( )( ) ( , )( )2!
x A
xx xA AA
f x A f x A f x A x x f x A A A
f x x f x x A A f A A
Now, using the symbols that we used in our absolute sensitivity functions, we can write
2 2
0 0 0 0
2 2 2
( , ) ( , )
1 ( , ) 2 ( , ) ( , )2!
z z
x A
z z z
x A x A
z S x A S x A
S S S
Inserting numbers we get
1
0 *0.01 1* 0.01 2 *0.0001 2 *2 *0.0001 0.01 30 02
z
This delta z will now be put into row 2 column 5 of the following table.
z
8/2/2019 sensit (1)
http://slidepdf.com/reader/full/sensit-1 20/125
4/12/2012 © 2009 Bahill20
Table 2. Effects of Individual and Combined Parameter Changes for
some second-order interaction terms with delta of 0.01, where
0 x x x etc.
FunctionsNormal
values
Valuesincreased
by delta
New z
values z
Total change
in z
( , ) f x A A=1
x=1
A=1.01
x=1.012.0103 0.0103 0.0103
0( , ) f x A A=1 A=1.01 2.0100 0.01000.0101 z
0( , ) f x A x=1 x=1.01 2.0001 0.0001
0 0( , ) f x A A=1
x=12.0000 0
( , ) f y B B=2
y=1
B=2.01
y=2.012.0104 0.0104 0.0104
0( , ) f y B B=2 B=2.01 2.0100 0.01000.0102 z
0( , ) f y B y=1 y=2.01 2.0002 0.0002
0 0( , ) f y B B=2
y=12.0000 0
8/2/2019 sensit (1)
http://slidepdf.com/reader/full/sensit-1 21/125
4/12/2012 © 2009 Bahill21
Estimate the mixed-second-partial derivative, according to this formula from Smith,
Szidarovszky, Karnavas and Bahill [2008].
2
0 0 0 0 0 0( , ) ( , ) ( , ) ( , ) ( , ) f f f f f
Table 3. Values to be used in estimating the second
partial derivative
TermsParameter values with a
0.01 step size
Function
values
( , ) f A =1.01
x=1.01
2.0103
0( , ) f A =1.00
x =1.012.0100
0( , ) f A =1.01
x =1.002.0001
0 0( , ) f A =1.00
x =1.002.0000
2 2.0103 2.0100 2.0001 2.0000 0.00022
0.01*0.01 0.0001
z
x A
This is the same value that we computed analytically.
8/2/2019 sensit (1)
http://slidepdf.com/reader/full/sensit-1 22/125
4/12/2012 © 2009 Bahill22
Table 4. Effects of Individual and Combined Parameter Changes for some third-
order interaction terms, delta = 0.001
FunctionsNormal
values
Values
increased
by delta
New z
values z Total change in z
( , ) f x A A=1
x=1
A=1.001
x=1.0022.0108 0.0108 0.0108
0( , ) f x A A=1 A=1.001 2.0100 0.0100
0.0102 z 0( , ) f x A x=1 x=1.001 2.0001 0.0001
0( , ) f x A x=1 x=1.001 2.0001 0.0001
0 0( , ) f x A A=1 x=1
2.0000 0
( , ) f y B B=2
y=1
B=2.001
y=1.0022.0112 0.0112 0.0112
0( , ) f y B B=2 B=2.001 2.0100 0.0100
0.0104 z 0( , ) f y B y=1 y=1.001 2.0002 0.0002
0( , ) f y B y=1 y=1.001 2.0002 0.0002
0 0( , ) f y B B=2
y=12.0000 0
8/2/2019 sensit (1)
http://slidepdf.com/reader/full/sensit-1 23/125
4/12/2012 23
Third-order partial derivatives Once again the interaction affect is larger than the
sum of the individual changes. But at least the third-order terms are smaller than
the first and second-order terms.
Three of the third-order partial derivatives aregreater than zero.
All of the fourth-order partial derivatives are zero.
© 2009 Bahill
8/2/2019 sensit (1)
http://slidepdf.com/reader/full/sensit-1 24/125
Minisummary The purposes of this section were
to show the bad affects of too large of a step size
to show how to calculate derivatives analytically
and to estimate derivatives numerically
to show that interactions are important
to show how to consider third- and forth-orderderivatives.
4/12/2012 © 2009 Bahill24
8/2/2019 sensit (1)
http://slidepdf.com/reader/full/sensit-1 25/125
4/12/2012 25
A pendulum clock 1I have a grandfather clock in Tucson that I would
like to move to a cabin up on Mount Lemmon.But I’ve been told that the changes in
temperature and altitude will make it inaccurate.Which will be the bigger culprit?
The period of oscillation of a pendulum is
A one-meter pendulum has a two-second
period. If the temperature changes by T , thenthe length becomes
Use the absolute-sensitivity function of P withrespect to T to calculate how many seconds perday the clock will gain.
glP / 2
0 (1 )T l l k T
© 2009 Bahill
8/2/2019 sensit (1)
http://slidepdf.com/reader/full/sensit-1 26/125
4/12/2012 26
A pendulum clock 2The absolute-sensitivity function is
The coefficient of expansion of a brass rod is
At the normal operating point T=0 and l=l 0, so
Mt. Lemmon is 2000 meters higher than Tucson and temperaturechanges 5ºC per 1000 m. So T=-10ºC.
Therefore, the change in period is
The pendulum will gain 8.6 seconds per day*.
NOP NOP
2 /
(1
P T
T
T
l g l k
T g l k T S
50 0 / 2 10 sec/ C
P
T T k glS
5
2 10 /°CT k
42 10 secP
T P S T
© 2009 Bahill
8/2/2019 sensit (1)
http://slidepdf.com/reader/full/sensit-1 27/125
4/12/2012 27
A pendulum clock 3However, the gravitational acceleration constant depends on
altitude (H)
Therefore the period becomes
and the absolute-sensitivity of P with respect to H is
For this equation the normal operating point is sea level, soH=0 and g 0=9.78. So,
6
0 3 10 , where is in meters.g g H H
0
2
H
l
P g k H
3
0NOP
P H H
H
k l
Sg k H
© 2009 Bahill
8/2/2019 sensit (1)
http://slidepdf.com/reader/full/sensit-1 28/125
4/12/2012 28
A pendulum clock 4
Going from Tucson to Mount Lemmon H=2000, so
The clock loses 26 seconds per day.
Although changes due to temperature and altitude are in the
opposite direction, they do not cancel each other out, becausechanges due to altitude are bigger.
0 7
3
0
3 10 s/m H P
H
k l
Sg
46 10 secP H P S H
© 2009 Bahill
8/2/2019 sensit (1)
http://slidepdf.com/reader/full/sensit-1 29/125
4/12/2012 29
Single pole with time delay 1
Use an absolute-sensitivity function to find when the
parameter K has the greatest effect on the step
response of the system. The step response is
( )
( ) ( ) 1
sY s Ke
M s R s s
( )
1
s
sr
KesY
s s
© 2009 Bahill
8/2/2019 sensit (1)
http://slidepdf.com/reader/full/sensit-1 30/125
4/12/2012 30
Single pole with time delay 2The absolute-sensitivity function of the step-response
with respect to K is
which transforms into
K has its greatest effect when the response reachessteady-state.
1τ)(
0
θ0
ss
esY
s
K S sr
0 0( ) / ( ) 1sr
t
K
yt eS
© 2009 Bahill
8/2/2019 sensit (1)
http://slidepdf.com/reader/full/sensit-1 31/125
4/12/2012 31
Types of sensitivity functions
Analytic
absolute
relative
semirelative
Empirical direct observation
sinusoidal variation of parameters
design of experiments
Excel, using what-if analysis
© 2009 Bahill
8/2/2019 sensit (1)
http://slidepdf.com/reader/full/sensit-1 32/125
4/12/2012 32
The relative-sensitivity function The relative-sensitivity of the function F to
variations in the parameter is
Relative-sensitivity functions are used tocompare parameters.
% change in
% change in
F F F F S
0
0NOP
F F
F S
© 2009 Bahill
8/2/2019 sensit (1)
http://slidepdf.com/reader/full/sensit-1 33/125
4/12/2012 33
Process control modelWhat is the easiest way (smallest percent change in an
operating point parameter) to increase the quantity of z that is being produced? Now this problem seemsappropriate for relative-sensitivity functions.
© 2009 Bahill
8/2/2019 sensit (1)
http://slidepdf.com/reader/full/sensit-1 34/125
Relative-sensitivity functions
4/12/2012 34 © 2009 Bahill
0 0
0
NOP 0 0
0 0
NOP 0 0
0 00 0 0 0 0
NOP 0 0
0 00 0 0 0 0
0 0NOP
3.5,
4,
2 0,
2 0.
z
E
z
F
z
x
z
y
z E E
S y E z z
z F F S
F z z
z x x
S A x C y D x z z
z y yS B y C x E
y z z
20 0
0NOP 0 0
20 00
NOP 0 0
0 00 0
NOP 0 0
0 00
NOP 0 0
0.5,
1,
1.5,
2.5,
z
A
z
B
zC
z
D
z A A
S x A z z
z B BS y
B z z
z C C S x yC z z
z D DS x
D z z
8/2/2019 sensit (1)
http://slidepdf.com/reader/full/sensit-1 35/125
4/12/2012 35
F is most important Therefore, we should increase F if we wish to
increase z. What about interactions? Could we do better by
changing two parameters at the same time?
© 2009 Bahill
8/2/2019 sensit (1)
http://slidepdf.com/reader/full/sensit-1 36/125
Interactions*
4/12/2012 36 © 2009 Bahill
2 2 2
0 0 0 0
2 2 2
0 0NOP
2
0 0 0 0 0
2 2
0 0NOP
2
0 0
2
0NOP
2 2
0 0 0 0
2 2
0 0NOP
2
0 0 0 0 0
2 20 0NOP
2 2*1 *10.5,
2
0.75,
1.25,
21.0,
0.75,
z
x A
z
x C
z
x D
z
y B
z
y C
z x A x AS
x A z z
z x C x y C S
x C z z
z x DS
x D z
z y B y BS
y B z z
z y C x y C S
y C z z
2
2
2
0 0
2
0NOP
2
0 0 0 0 0
2 2
0 0NOP
2 2 2
0 0 0
2 2 2
0 0NOP
2 2 2
0 0 0
2 2 2
0 0NOP
1.75,
0.75,
20.5,
21.0.
z
y E
z
x y
z
x
z
y
z y E S
y E z
z x y x y C S
x y z z
z x x AS
x z z z y y B
S y z z
8/2/2019 sensit (1)
http://slidepdf.com/reader/full/sensit-1 37/125
4/12/2012 37
Mini-summary Using absolute-sensitivity functions, the second- and
third-order terms, e. g.
were the most important, but using relative-
sensitivity functions, F was the most importantparameter.
The absolute-sensitivity functions show the mostimportant parameters for a fixed size change in theparameters
The relative-sensitivity functions show the mostimportant parameters for a certain percent change in
the parameters.
3
2and
z
x A
2
2
z
y
© 2009 Bahill
8/2/2019 sensit (1)
http://slidepdf.com/reader/full/sensit-1 38/125
4/12/2012 38
Relatively the most important
0 0
NOP 0 04
z
F
F F zS F z z
© 2009 Bahill
8/2/2019 sensit (1)
http://slidepdf.com/reader/full/sensit-1 39/125
4/12/2012 39
The operating point* In the process control example, if the operating point
is changed from (1, 1) to (10, 10), then the output z becomes most sensitive (relatively) to the input y .
At the operating point (1, 1), the output is not
sensitive to the inputs, which means we could
twiddle with the inputs forever and not be able tocontrol the output. Therefore, this is not a desirable
operating point.*
© 2009 Bahill
8/2/2019 sensit (1)
http://slidepdf.com/reader/full/sensit-1 40/125
More examples of relative sensitivity functions
4/12/2012 © 2009 Bahill40
8/2/2019 sensit (1)
http://slidepdf.com/reader/full/sensit-1 41/125
4/12/2012 41
Double pole with time-delay
Which of the parameters is
most important?
2
( )
( ) ( ) (τ 1)
sY s Ke
M s R s s
0
0NOP
1 M
K M K
M K S
0
0
0
θ NOP
θθ
θ
M M s
M S
0
0
0
NOP 0
2ττ τ 1
M s M
M sS
© 2009 Bahill
8/2/2019 sensit (1)
http://slidepdf.com/reader/full/sensit-1 42/125
4/12/201242
Frequency domain output
8/2/2019 sensit (1)
http://slidepdf.com/reader/full/sensit-1 43/125
4/12/2012 43
Results For low frequencies, K is biggest
For mid-frequencies, is biggest
For high frequencies, is biggest
© 2009 Bahill
A i l l d l t
8/2/2019 sensit (1)
http://slidepdf.com/reader/full/sensit-1 44/125
A simple closed loop system
4/12/2012 44 © 2009 Bahill
Transfer function( )
( )
( )
Y s K M s
R s s K
Time-domain step-response
( ) 1Kt
sr y t e
Time-domain relative-sensitivity function0
0 0
0 0
1 1
sr
K t y Kt
K K t K t NoP
K K teS te
e e
Frequency-domain step-response
( )( )
sr
K Y s
s s K
Frequency-domain relative-sensitivity function
0
ˆ ( )sr Y
K
sS ss K
Take the inverse Laplace transform0
0ˆ ( )sr Y K t
K S t K e
where is the unit impulse.
8/2/2019 sensit (1)
http://slidepdf.com/reader/full/sensit-1 45/125
4/12/2012 45 © 2009 Bahill
Time domain output
8/2/2019 sensit (1)
http://slidepdf.com/reader/full/sensit-1 46/125
4/12/2012 46
An expert systemSensitivity functions need not be functions of
time or frequency.If premise1 = true (CF 1)
and premise2 = true (CF 2)
or premise3 = true (CF 3
)
then conclusion = true CF 4.
The certainty of the rule uses the minimum of
the certainties of the AND clauses.
© 2009 Bahill
Certainty factor domain output
8/2/2019 sensit (1)
http://slidepdf.com/reader/full/sensit-1 47/125
4/12/2012 47 © 2009 Bahill
If CF 1 < CF 2, then the final certainty factorbecomes
3 41 4 1 4
100100 100 10,000 f
CF CF CFCF CFCF
CF
The relative-sensitivity functions are
1
3 4 104
0NOP
110,000 100
f CF
CF f
CF CF CF CF S
CF
20
f CF
CF S
3
301 4 4
0NOP
1 10,000 100
f CF
CF
f
CF CFCF CF S CF
4
3 1 3 4 401
0NOP
2
100 100 1,000,000
f CF
CF
f
CF CFCF CF CF CF S
CF
8/2/2019 sensit (1)
http://slidepdf.com/reader/full/sensit-1 48/125
4/12/2012 48 © 2009 Bahill
Assume 1 3 4 280 and 81CF CF CF CF
Then 0 87 f CF and
10.26
f CF
CF S
20
f CF
CF S
30.26
f CF
CF S
40.53
f CF
CF S
Changes in CF 4 are twice as importantas changes in CF 1 or CF 3 and increases in CF 2 have no effect.
8/2/2019 sensit (1)
http://slidepdf.com/reader/full/sensit-1 49/125
4/12/2012 49
Limitations of therelative-sensitivity function
L x t y t Lx t Ly t
0
NOP 0
( ) ( ) ( )
( )
f t f t f t f t S
f t
0
NOP 0
( ) ( ) ( )ˆ( )
F s F s F s F sS
F s
Because
ˆ f t F s
S S we have two functions
© 2009 Bahill
8/2/2019 sensit (1)
http://slidepdf.com/reader/full/sensit-1 50/125
4/12/201250
Disadvantages of relative-sensitivity functions
different in time and frequency domains cannot use Laplace transforms to get time-
domain solution
division by zero problem
0
0NOP
F F
F S
© 2009 Bahill
8/2/2019 sensit (1)
http://slidepdf.com/reader/full/sensit-1 51/125
4/12/2012 51
Types of sensitivity functions
Analytic
absolute relative
semirelative
Empirical direct observation
sinusoidal variation of parameters
design of experiments
Excel, using what-if analysis
© 2009 Bahill
8/2/2019 sensit (1)
http://slidepdf.com/reader/full/sensit-1 52/125
4/12/2012 52
Double pole with time delay
© 2009 Bahill
8/2/2019 sensit (1)
http://slidepdf.com/reader/full/sensit-1 53/125
4/12/2012 53
The semirelative-sensitivity function The semirelative-sensitivity of the function F to
variations in the parameter is
0NOP
F F S
© 2009 Bahill
8/2/2019 sensit (1)
http://slidepdf.com/reader/full/sensit-1 54/125
Tradeoff study
4/12/2012 54 © 2009 Bahill
A Generic Tradeoff Study
CriteriaWeight of
ImportanceAlternative-1 Alternative-2
Criterion-1 Wt1 S11 S12 Criterion-2 Wt2 S21 S22 AlternativeRating
Sum1 Sum2
A Numeric Example of a Tradeoff StudyAlternatives
CriteriaWeight of
ImportanceUmpire’s
AssistantSeeing
Eye DogAccuracy 0.75 0.67 0.33
Silence ofSignaling
0.25 0.83 0.17
Sum of weighttimes score
0.71The
winner0.29
1 1 11 2 21 2 1 12 2 22
andSum Wt S Wt S Sum Wt S Wt S
8/2/2019 sensit (1)
http://slidepdf.com/reader/full/sensit-1 55/125
4/12/2012 55
Which parameters could changethe recommendations?
Use this performance index
Compute the semirelative-sensitivity functions.
1 1 2
1 11 2 21 1 12 2 22 0.420
PI Sum Sum
Wt S Wt S Wt S Wt S
© 2009 Bahill
8/2/2019 sensit (1)
http://slidepdf.com/reader/full/sensit-1 56/125
Semirelative-sensitivity functions*
4/12/2012 56 © 2009 Bahill
1
2
11
21
12
22
11 12 1
21 22 2
1 11
2 21
1 12
2 22
0.26
0.16
0.50
0.21
-0.25
-0.04
F
Wt
F
Wt
F
S
F
S
F S
F
S
S S S Wt
S S S Wt
S Wt S
S Wt S
S Wt S
S Wt S
8/2/2019 sensit (1)
http://slidepdf.com/reader/full/sensit-1 57/125
A sensitivity matrix
4/12/2012 57 © 2009 Bahill
Analytic semirelative-sensitivity function values for PI 1, the
difference of the alternative ratings
Alternatives
CriteriaWeight of
Importance
Umpire’s
Assistant
Seeing Eye
Dog
Accuracy 1
1
PI
Wt S = 0.26 1
11
PI
SS = 0.50 1
12
PI
SS = -0.25
Silence of
Signaling1
2
PI
Wt S = 0.16 1
21
PI
SS = 0.21 1
22
PI
SS = -0.04
A nice way todisplay the sensitivities
8/2/2019 sensit (1)
http://slidepdf.com/reader/full/sensit-1 58/125
4/12/2012 58
What about interactions?The semirelative-sensitivity function of PI1 for the
interaction of Wt1 and S11 is
which is as big as the first-order terms.
1 11 0 0 0 0
2
1 11 1 11
1 11 NOP
0.5025F
Wt S
F S Wt S Wt S
Wt S
© 2009 Bahill
8/2/2019 sensit (1)
http://slidepdf.com/reader/full/sensit-1 59/125
4/12/2012 59
Interactions for PI1
So interactions are important.
Semirelative Sensitivity Values Showing Interaction Effects
Function Normalvalues
Valuesincreasedby 10%
New Fvalues
F Total changein z
1
F
Wt S 1Wt =0.75 1Wt =0.82 0.446 0.026
11
F
SS 11
S =0.67 11S =0.74 0.470 0.0500.076F
1 11
F
Wt SS
1Wt =0.75
11S =0.67
1Wt =0.82
11S =0.74
0.501 0.081 0.081
© 2009 Bahill
8/2/2019 sensit (1)
http://slidepdf.com/reader/full/sensit-1 60/125
4/12/2012 60
A new performance index A problem with performance index PI1 is that if s11=s12,
then the sensitivity with respect to Wt1 becomes equalto zero.
Mathematically this is correct, but logically it is wrong.
Another problem is that the sensitivity with respect to
Wt1 does not depend on scores for the nonwinningalternatives and we do want the sensitivities to dependon the other parameters.
The following performance index solves both of these
problems.
3
1 1
1 n m
i ij
i j
PI Wt S
m
© 2009 Bahill
8/2/2019 sensit (1)
http://slidepdf.com/reader/full/sensit-1 61/125
4/12/2012 61
Semirelative sensitivity functions
3 3
1NOP
1i
m
PI Wt i i ij
ji
PI S Wt Wt SWt m
3
ij
i ijPI S
Wt SSm
© 2009 Bahill
8/2/2019 sensit (1)
http://slidepdf.com/reader/full/sensit-1 62/125
Sensitivity matrix for PI3
Table VI. Analytic semirelative-sensitivity function values
for PI 3, the sum of all weight times scoresAlternatives
CriteriaWeight of
Importance
Umpire’s
Assistant
Seeing Eye
Dog
Accuracy of
the call
3
1
0.38PI
Wt S 3
11 0.25
PI
SS 3
12
0.12PI
S
S
Silence of
Signaling3
20.13PI
Wt S 3
210.10PI
SS 3
220.02PI
SS
4/12/2012 © 2009 Bahill62
8/2/2019 sensit (1)
http://slidepdf.com/reader/full/sensit-1 63/125
4/12/2012 63
It is not difficultAlthough these equations may look formidable, they
are easy to compute with a spreadsheet. For example
is merely the sum of the weight times scores in column
k and this is already in the spreadsheet. Furthermore,because
and the rest of the second order sensitivities are zero,Table VI is complete: it has all of the sensitivities in it.
1
n
k kj
k
Wt S
3 3
ij i ij
PI PI
S Wt SS S
© 2009 Bahill
8/2/2019 sensit (1)
http://slidepdf.com/reader/full/sensit-1 64/125
4/12/2012 64
What about interactions?
Yes, We do have to worry about interactions,because this is bigger than most of the first order
sensitivities.
3
1 11 0 0
2
3 1 111 11
1 11 NOP0.25
PI
Wt S
PI Wt S
S Wt SWt S m
© 2009 Bahill
8/2/2019 sensit (1)
http://slidepdf.com/reader/full/sensit-1 65/125
4/12/2012 65
Interaction Sensitivity MatrixTable VII. Analytic semirelative-sensitivity function values for
the interactions of PI 3
Alternatives
CriteriaWeight of
Importance
Umpire’sAssistant
Seeing Eye
Dog
Accuracy of
the call3
1 110.25
PI
Wt SS 3
1 120.12
PI
Wt SS
Silence of
Signaling3
2 210.10PI
Wt SS 3
2 220.02
PI
Wt SS
These cells contain the same numerical values as Table VI.
© 2009 Bahill
Tradeoff studies are hierarchical
8/2/2019 sensit (1)
http://slidepdf.com/reader/full/sensit-1 66/125
Tradeoff studies are hierarchical
4/12/2012 66 © 2009 Bahill
The structure of a hierarchical tradeoff study
Criteria
Normalized
Criteria
Weights
Subcriteria
Normalized
Subcriteria
Weights
Scores for
Alternative-1
Scores for
Alternative-2
Performance (1)CW
Subcriteria-1 (1)
1Wt (1)
11S (1)
12S
Subcriteria-2 (1)
2Wt (1)
21S (1)
22S
Subcriteria-3 (1)
3Wt (1)
31S (1)
32S
Subcriteria-4 (1)
4Wt (1)
41S (1)
42S
Cost(2)
CW Subcriteria-1 (2 )
1Wt (2 )
11S (2 )
12S
Subcriteria-2 (1)
2Wt (2 )
21S (2 )
22S
Schedule (3)CW
Subcriteria-1 (1)
1Wt (3)
11S (3)
12S
Subcriteria-2 (3)
2Wt (3)
21S (3)
22S
Risk (4)CW
Subcriteria-1 (4 )
1Wt (4 )
11S (4 )
12S
Subcriteria-2 (4 )
2Wt (4 )
21S
(4 )
22S
Subcriteria-3 (4 )
3Wt (4 )
31S (4 )
32S
Alternative
Ratings1Sum 2Sum
8/2/2019 sensit (1)
http://slidepdf.com/reader/full/sensit-1 67/125
4/12/2012 67
A new performance index, PI5Because most of tradeoff studies are hierarchical, in the
Spin Coach and the PopUp Coach I used thisperformance index
( )( ) ( ) ( )
5
1 1 1
1 n lk ml l l
i ij
l i j
PI CW Wt S
m
5( )
( )( ) ( ) ( )
1 1
1l
n l mPI l l l
i ijCW i j
S CW Wt Sm
5 ( ) ( ) ( )1ij
PI l l l
S i ijS CW Wt S
m
© 2009 Bahill
Single pole with time delay (when)
8/2/2019 sensit (1)
http://slidepdf.com/reader/full/sensit-1 68/125
4/12/2012 68
Single pole with time delay (when)Transfer function
( )( )
( ) 1
sY s Ke
M s
R s s
Step-response
( )( 1)
s
sr
KeY s
s s
Semirelative-sensitivity functions0
0
0( 1)sr
sY
K
K eS
s s
0
0 0
0( 1)sr
sY K e
Ss
0
0 0
2
0( 1)sr
sY K e
Ss
Which (for 0t ) transform to0 0( )
0( ) (1 )sr y t
K S t K e
0 0( )
0 0( )sr y t S t K e
0 0( )
0 0 0( ) ( )sr y t S t K t e
© 2009 Bahill
8/2/2019 sensit (1)
http://slidepdf.com/reader/full/sensit-1 69/125
4/12/2012 69 © 2009 Bahill
h d hi h
8/2/2019 sensit (1)
http://slidepdf.com/reader/full/sensit-1 70/125
4/12/2012 70
What does this teach us? If the model does not match the physical system in
the early part of the step response, then adjust thetime-delay of the model.
For steady state …
In the middle …
© 2009 Bahill
T f i i i f i
8/2/2019 sensit (1)
http://slidepdf.com/reader/full/sensit-1 71/125
4/12/2012 71
Types of sensitivity functions
Analytic
absolute relative
semirelative
We have just examined the analytic sensitivity
functions. We are now ready to look at theempirical sensitivity functions.
Empirical
direct observation
sinusoidal variation of parameters
design of experiments
Excel, using what-if analysis
© 2009 Bahill
E i i l i i i f i
8/2/2019 sensit (1)
http://slidepdf.com/reader/full/sensit-1 72/125
4/12/2012 72
Empirical sensitivity functions1
The method of direct observation
can be preformed on real-world systems
models of those systems
simulations of those models
© 2009 Bahill
8/2/2019 sensit (1)
http://slidepdf.com/reader/full/sensit-1 73/125
4/12/2012 73 © 2009 Bahill
8/2/2019 sensit (1)
http://slidepdf.com/reader/full/sensit-1 74/125
4/12/2012 74 © 2009 Bahill
RealSystem
Model ofReal
System
ComputerSimulationof Model
Modelers ModelersGood
ModelersGood
Modelers
M a t h e m a t i c i a n
s E x p e r i m e n t a l i s t s
E ti ti d i ti
8/2/2019 sensit (1)
http://slidepdf.com/reader/full/sensit-1 75/125
4/12/2012 75
Estimating derivatives
are small, then the second term on the right can beneglected.
0If (x-x ) and ( ) f
© 2009 Bahill
T d ff t d l
8/2/2019 sensit (1)
http://slidepdf.com/reader/full/sensit-1 76/125
4/12/2012 76
Tradeoff study exampleFor a +10% parameter change, the semirelative-
sensitivity function is
This is very easy to compute.
Tradeoff Study Matrix with S11 Increased by 5%Criteria
Weight ofImportance
Umpire’sAssistant
SeeingEye dog
Accuracy 0.75 0.74 0.33Silence of Signaling 0.25 0.83 0.17
Sum of weight timesscore 0.76 0.29
0 0
0
100.1
F F F S F
© 2009 Bahill
S iti it t i
8/2/2019 sensit (1)
http://slidepdf.com/reader/full/sensit-1 77/125
Sensitivity matrix
4/12/2012 © 2009 Bahill77
Table X. Numerical estimates for semirelative-sensitivity function
for PI 3, the sum of the alternative ratings squared, for a plus 10%
parameter perturbationAlternatives
CriteriaWeight of
Importance
Umpire’s
Assistant
Seeing Eye
Dog
Accuracy of
the call3
10.38PI
Wt S 3
110.25
PI
SS 3
120.12PI
SS
Silence of
Signaling3
20.13
PI
Wt S 3
210.10PI
SS 3
220.02PI
SS
These are the same resultsthat were obtained in the analytic
semirelative sensitivity section.
B t h t b t th d d t ?
8/2/2019 sensit (1)
http://slidepdf.com/reader/full/sensit-1 78/125
4/12/2012 78
But what about the second-order terms?Namely
When using the sum of weighted scores combining function
the second derivatives are all zero. So our estimations are all
right. This is not true for the product combining function,
most other combining functions (See Daniels, Werner and
Bahill [2001] for explanations of other combining functions.)or other performance indices.
In particular let’s try PI3.
2
0
( )( )
2!
f x x
1 1 11 2 21 2 1 12 2 22andSum Wt S Wt S Sum Wt S Wt S
1 2 1 2
1 11 21 2 12 22andWt Wt Wt Wt F S S F S S
© 2009 Bahill
D i ti f f ti f t i bl
8/2/2019 sensit (1)
http://slidepdf.com/reader/full/sensit-1 79/125
4/12/2012 79
Derivative of a function of two variables
Let us examine the second-order terms, those insidethe { }, for two reasons
to see if they are large and must be included incomputing the first derivative
to estimate the effects of interactions on thesensitivity analysis
0 0 0 0 0 0 0 0
2 20 0 0 0
( , ) ( , ) ( , )( ) ( , )( )
1 ( , )( ) 2 ( , )( )( ) ( , )( )2!
x y
xx xy yy
f x y f x y f x y x x f x y y y
f x x f x x y y f y y
© 2009 Bahill
I t ti
8/2/2019 sensit (1)
http://slidepdf.com/reader/full/sensit-1 80/125
4/12/2012 80
InteractionsPreviously we derived the analytic semirelative-
sensitivity function for the interaction of Wt1 and S11 as,
which is as big as the first-order semirelative-sensitivityfunctions.
0 03
1 11 0 0
21 113
1 11
1 11 NOP
0.25PI
Wt S
Wt SPI S Wt S
Wt S m
© 2009 Bahill
Interactions
8/2/2019 sensit (1)
http://slidepdf.com/reader/full/sensit-1 81/125
4/12/2012 81
InteractionsFor a 10% change in parameter values, a simple-minded
approximation is
using our tradeoff study values we get
This does not match the analytic value.
What went wrong?
2
2
0 0 0 0
0 0
100.1 0.1
F F F F
S F
3
1 11
210 0.424PI
Wt SS F
© 2009 Bahill
Maybe the step size is too big
8/2/2019 sensit (1)
http://slidepdf.com/reader/full/sensit-1 82/125
4/12/2012 82
Maybe the step size is too big Let’s reduce the perturbation step size to 0.1%?
This is closer, but it is still too big.
2
2
0 0 0 0
0 0
10000.001 0.001
F F F F S F
3
1 11
21000 0.393PI
Wt SS F
© 2009 Bahill
What went wrong?
8/2/2019 sensit (1)
http://slidepdf.com/reader/full/sensit-1 83/125
4/12/2012 83
What went wrong?In the previous computations, we
changed both parameters at thesame time and then compared thevalue of the function to the value of the function at its normal operating
point. However, this is not thecorrect estimation for the second-partial derivative.
© 2009 Bahill
Estimating the second partials
8/2/2019 sensit (1)
http://slidepdf.com/reader/full/sensit-1 84/125
4/12/2012 84
Estimating the second partials1
To estimate the second-partial derivatives we should
start with2
0 0 0 0 0( , ) ( , ) ( , ) f f f
0 0 0 02
0 0
( , ) ( , ) ( , ) ( , )( , )
f f f f f
2
0 0 0 0 0 0( , ) ( , ) ( , ) ( , ) ( , ) f f f f f
0
© 2009 Bahill
Estimating the second partials
8/2/2019 sensit (1)
http://slidepdf.com/reader/full/sensit-1 85/125
Estimating the second partials2
4/12/2012 85 © 2009 Bahill
Values to be Used in Estimating the Second Derivative
Terms
Parameter values with a 0.1% step size,
that is 1Wt =0.00075 and 11S =0.00067Functionvalues
( , ) f 1Wt =0.75075
11S =0.67067
0.50063
0( , ) f 11S =0.67067 0.50025
0( , ) f 1Wt =0.75075 0.50038
0 0( , ) f 1Wt =0.75000
11S =0.670000.50000
2
3
1 11
0.50063 0.50038 0.50025 0.500000.5
*0.00075*0.00067
PI
Wt S m
Estimating the sensitivity functions
8/2/2019 sensit (1)
http://slidepdf.com/reader/full/sensit-1 86/125
4/12/2012 86
Estimating the sensitivity functionsTo get the semirelative-sensitivity function we multiply
the second-partial derivative by the normal values of Wt1 and S11 to get
Now, this is the same result that we derived in the
analytic semirelative sensitivity section.
3
1 11 0 0 0 0
2
31 11 1 11
1 11 NOP
0.5 0.25PI
Wt S
PI S Wt S Wt S
Wt S
© 2009 Bahill
Lessons learned
8/2/2019 sensit (1)
http://slidepdf.com/reader/full/sensit-1 87/125
4/12/2012 87
Lessons learned For a tradeoff study using the sum combining
function and a simple performance index,anything works.
Otherwise, the perturbation step size should be
small. Five and 10% perturbations are not
acceptable. It is incorrect to estimate the second partial
derivative by changing two parameters at thesame time and then comparing that value of the
function to the value of the function at itsnormal operating point. Estimating secondderivatives requires evaluation of four not two
numerator terms.
© 2009 Bahill
Sensitivity analysis of a risk analysis
8/2/2019 sensit (1)
http://slidepdf.com/reader/full/sensit-1 88/125
4/12/2012 88
Sensitivity analysis of a risk analysisLet
be the probability of occurrence, the severity, and therisk, for the jth failure mode. Risk is
Use the performance index* and
calculate the semirelative-sensitivity functions
The largest sensitivities are always those for the largestrisk. This means that we should spend extra time andeffort estimating the probability and severity of thehighest ranked risk, which seems eminently sensible.
, and j j jP S R
j j j R P S
1
n
j
j
PI R
0 j
PI
P j j jS S P R 0 j
PI
S j j jS R S R
© 2009 Bahill
Linearity
8/2/2019 sensit (1)
http://slidepdf.com/reader/full/sensit-1 89/125
4/12/2012 89
Linearity Although the model is linear, the
sensitivity functions are not.
© 2009 Bahill
8/2/2019 sensit (1)
http://slidepdf.com/reader/full/sensit-1 90/125
4/12/2012 90 © 2009 Bahill
Types of sensitivity functions
8/2/2019 sensit (1)
http://slidepdf.com/reader/full/sensit-1 91/125
4/12/2012 91
Types of sensitivity functions
Analytic
absolute relative
semirelative
Empirical
direct observation
sinusoidal variation of parameters
design of experiments
Excel, using what-if analysis
© 2009 Bahill
Empirical sensitivity functions
8/2/2019 sensit (1)
http://slidepdf.com/reader/full/sensit-1 92/125
4/12/2012 92
Empirical sensitivity functions2
Sinusoidal variation of parameters,
also called frequency-domain experiments
response-surface methodology
© 2009 Bahill
Sinusoidal variation of parameters
8/2/2019 sensit (1)
http://slidepdf.com/reader/full/sensit-1 93/125
4/12/2012 93
Sinusoidal variation of parameters Make two runs of the system
1. all parameters are set at their normal values
2. all parameters are modulated sinusoidally
Compute the power spectrum of each
Form the ratio of the two spectra at each frequency
Spikes will be observed at the modulation frequencies
at frequencies related to nonlinearities
at frequencies related to product effects
© 2009 Bahill
A first-order negative-feedback system
8/2/2019 sensit (1)
http://slidepdf.com/reader/full/sensit-1 94/125
4/12/2012 94
A first order, negative feedback system
K ______ s + A
H
R (s ) Y (s )+
-
© 2009 Bahill
A first-order negative-feedback control
8/2/2019 sensit (1)
http://slidepdf.com/reader/full/sensit-1 95/125
4/12/2012 95
A first order, negative feedback, controlsystem (continued)
Input: one Hertz, unit-amplitude sinusoid
Duration: one second Sampling: 2048 evenly spaced samples
Modulation frequencies: 5, 30 and 170 Hz
HK AsK
s RsY s M
)()()(
© 2009 Bahill
Modulation equations
8/2/2019 sensit (1)
http://slidepdf.com/reader/full/sensit-1 96/125
4/12/2012 96
Modulation equations A = 0.1 (1 + 0.5 sin (5 2t))
H = 50 (1 + 0.5 sin (30
2t))
K = 1.0 (1 + 0.5 sin (170 2t))
© 2009 Bahill
Step response with modulated parameters
8/2/2019 sensit (1)
http://slidepdf.com/reader/full/sensit-1 97/125
Step response with modulated parameters
4/12/2012 97 © 2009 Bahill
Location of spikes
8/2/2019 sensit (1)
http://slidepdf.com/reader/full/sensit-1 98/125
4/12/2012 98
Location of spikes If a parameter is modulated at a we expect to see
power at a
If there is a parabolic nonlinearity we also expect powerat 2 a
because 2 sin2 x = 1 - cos 2 x
If the system is sensitive to the product of twoparameters modulated at a and b, then we expectpower at a b because 2 sin x sin y = cos( x-y ) - cos( x+y )
These product terms are called interactions.
© 2009 Bahill
8/2/2019 sensit (1)
http://slidepdf.com/reader/full/sensit-1 99/125
4/12/2012 99 © 2009 Bahill
8/2/2019 sensit (1)
http://slidepdf.com/reader/full/sensit-1 100/125
4/12/2012 100
Split peaks are due to the one Hertz input.
This technique probably produces relative-sensitivities.
© 2009 Bahill
The semirelative-sensitive functions*
8/2/2019 sensit (1)
http://slidepdf.com/reader/full/sensit-1 101/125
The semirelative sensitive functions
4/12/2012 101 © 2009 Bahill
1.50
~22
00 1.0
000
s H K As
AK
S
ss M
K
1.50
~22
0 50
000
20
s H K As
H K
S
M
H
1.50
~22
00 1.0
000
s H K As
AK S
M
A
An M/M1 queue
8/2/2019 sensit (1)
http://slidepdf.com/reader/full/sensit-1 102/125
4/12/2012 102
An M/M1 queue service rate =
= 0.8 + 0.2 sin(46/4096 2t) arrival rate =
= 0.4 + 0.2 sin(4/4096 2t)
© 2009 Bahill
An M/M1 queue
8/2/2019 sensit (1)
http://slidepdf.com/reader/full/sensit-1 103/125
4/12/2012 103
An M/M1 queue service rate =
= 0.8 + 0.2 sin(46/4096 2t) arrival rate =
= 0.4 + 0.2 sin(4/4096 2t)
© 2009 Bahill
8/2/2019 sensit (1)
http://slidepdf.com/reader/full/sensit-1 104/125
4/12/2012 104 © 2009 Bahill
8/2/2019 sensit (1)
http://slidepdf.com/reader/full/sensit-1 105/125
4/12/2012 105
An M/M1 queue is a low pass filter
The interaction peaks are not the same height.
© 2009 Bahill
Bode diagram of a low-pass filter
8/2/2019 sensit (1)
http://slidepdf.com/reader/full/sensit-1 106/125
g p
4/12/2012 106 © 2009 Bahill
Some problems with sinusoidal
8/2/2019 sensit (1)
http://slidepdf.com/reader/full/sensit-1 107/125
4/12/2012 107
p variation of parameters
There must be an input signal. Shape of the spikes depends on parameters of the input
signal.
The output cannot be stationary.
The frequency response of the system (e.g. low-pass,
high-pass, resonance, etc.) must be known.
The range of linearity of the system must be known.*
Parameters of the FFT and the windows must beunderstood.
© 2009 Bahill
8/2/2019 sensit (1)
http://slidepdf.com/reader/full/sensit-1 108/125
4/12/2012 108 © 2009 Bahill
Types of sensitivity functions
8/2/2019 sensit (1)
http://slidepdf.com/reader/full/sensit-1 109/125
4/12/2012 109
yp y
Analytic
absolute relative
semirelative
Empirical
direct observation sinusoidal variation of parameters
design of experiments (DoE)
Excel, using what-if analysis
© 2009 Bahill
Empirical sensitivity functions3
8/2/2019 sensit (1)
http://slidepdf.com/reader/full/sensit-1 110/125
4/12/2012 110
p y 3
Design of experiments (DoE) should be used when
experiments are expensive. When doing a Taguchi 3-level design pick a normal
value, a high value and a low value.
If the high and low are some percentage change,
then you are doing a relative sensitivity analysis. If the high and low are plus and minus a unit, then
you are doing an absolute sensitivity analysis.
Alternatively, the high and low could be realistic
design options, in which case it does notcorrespond to any of our sensitivity functions.
© 2009 Bahill
Altitude 0 ft
9%
Percent change in air density
over the parameter ranges tobe expected for a typical July
afternoon in United States
8/2/2019 sensit (1)
http://slidepdf.com/reader/full/sensit-1 111/125
4/12/2012 111 © 2009 Bahill
Low High
Temperature
Relative Humidity
Barometric Pressure P e r c e n t C h a n g e i n A
i r D e n s i t y
70 ºF 2600 feet
85 ºF50%
760 mm Hg
90%745 mm Hg
10%775 mm Hg
100 ºF
5200 feet
1%
3%
5%
7%
-1%
-3%
-5%
-7%
-9%
ballparks.
Medium
Types of sensitivity functions
8/2/2019 sensit (1)
http://slidepdf.com/reader/full/sensit-1 112/125
4/12/2012 112
yp y
Analytic
absolute relative
semirelative
Empirical
direct observation sinusoidal variation of parameters
design of experiments
Excel, using what-if analysis
© 2009 Bahill
The Pinewood Derby
8/2/2019 sensit (1)
http://slidepdf.com/reader/full/sensit-1 113/125
y
4/12/2012 113 © 2009 Bahill
The most important parameters
8/2/2019 sensit (1)
http://slidepdf.com/reader/full/sensit-1 114/125
4/12/2012 114
in the Pinewood Derby
baseline for Overall Happiness scoringfunction
baseline for Percent Happy Scoutsscoring function
importance weight for OverallHappiness evaluation criterion
baseline for Number of Repeat Racesscoring function
input value for Percent Happy Scouts
evaluation criterion input value for Number of Repeat
Races evaluation criterion
© 2009 Bahill
Of the 89 parameters only 3 couldh h f d l
8/2/2019 sensit (1)
http://slidepdf.com/reader/full/sensit-1 115/125
4/12/2012 115
p ychange the preferred alternative
1. The Tradeoff Function with 90% for Performance and 10% for
Utilization of Resources the preferredalternative was a round robin with besttime scoring
with 57% for Performance and 43% forResources the preferred alternativeswitched to the double eliminationtournament
© 2009 Bahill
Sensitivity Analysis of Pinewood Derby (simulation data)0 9
8/2/2019 sensit (1)
http://slidepdf.com/reader/full/sensit-1 116/125
4/12/2012 116 © 2009 Bahill
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1
Performance Weight
O v e r a
l l S c o r e
Single elimination
Double elimination
Round robin, mean-time
Round robin, best-time
Round robin, points
Sensitivity of Pinewood Derby (prototype data)
8/2/2019 sensit (1)
http://slidepdf.com/reader/full/sensit-1 117/125
4/12/2012 117 © 2009 Bahill
0.5
0.55
0.6
0.65
0.7
0.75
0.8
0.85
0.9
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1
Performance Weight
O v e r a l l S c o r e
Double elimination
Round robin, best-time
Round robin, points
Of the 89 parameters only 3 couldh h f d l i
8/2/2019 sensit (1)
http://slidepdf.com/reader/full/sensit-1 118/125
4/12/2012 118
change the preferred alternative2
2. The slope of thePercent Happy Scouts scoring function
3. The baseline for the
Percent Happy Scouts scoring function
© 2009 Bahill
8/2/2019 sensit (1)
http://slidepdf.com/reader/full/sensit-1 119/125
4/12/2012 119 © 2009 Bahill
Validation1
8/2/2019 sensit (1)
http://slidepdf.com/reader/full/sensit-1 120/125
4/12/2012 120
If a system (or its model) is very sensitive to
parameters over which the customer has no control,then it may be the wrong system for that customer.
If the sensitivity analysis reveals the most important
parameters and that result is a surprise, then it may be
the wrong system. If a system is more sensitive to its parameters than to
its inputs, then it may be the wrong system or thewrong operating point.
If the sensitivities of the model are different from thesensitivities of the physical system, then it may be the
wrong model.
© 2009 Bahill
Validation2
8/2/2019 sensit (1)
http://slidepdf.com/reader/full/sensit-1 121/125
4/12/2012 121
If you delete a requirement, then your completeness
measure (a traceability matrix) should show a vacuity. After you make a decision, do a sensitivity analysis and
see if changing a parameter would change your
decision.
Domain experts should agree with the sensitivityanalysis about which criteria in a tradeoff study are themost important.
Domain experts should agree with the sensitivity
analysis about which risks are the most important. Do a sensitivity analysis of prioritized lists: see if
changing the most important criteria would change theprioritization.
© 2009 Bahill
Verification
8/2/2019 sensit (1)
http://slidepdf.com/reader/full/sensit-1 122/125
4/12/2012 122
Unplanned excessive sensitivity to any parameter is a
verification mistake. Sensitivity to interactions should be flagged and
studied: such interactions may be unexpected and
undesirable.
© 2009 Bahill
Résumé
8/2/2019 sensit (1)
http://slidepdf.com/reader/full/sensit-1 123/125
4/12/2012 123
After you
build a model, or write a set of requirements, or
do a tradeoff study, or
design a system,
you should study that thing to see if itmakes sense.
One of the best ways to study a thing iswith a sensitivity analysis.
© 2009 Bahill
Describe this talk to your Vice President
8/2/2019 sensit (1)
http://slidepdf.com/reader/full/sensit-1 124/125
Professor Bahill modeled our potion production
process and did a sensitivity analysis of it. So we nowhave a better understanding of our process. Hissensitivity analysis accounts for nonlinearities andparameter interactions. His equations for estimating
parameters are correct (because Szidarovszkyderived them). This analysis shows which parametersare the most important for making our potent.
I recommend that we name our potion Love Potion
Number Nine. We should buy the copyright for that
song and play it in the background of our TVcommercials.
Play an audio clip.4/12/2012 © 2009 Bahill124
8/2/2019 sensit (1)
http://slidepdf.com/reader/full/sensit-1 125/125
top related