chapters 19 – 20: dealing with uncertainties tps example: uncertain outcome decision rules for...
TRANSCRIPT
![Page 1: Chapters 19 – 20: Dealing With Uncertainties TPS Example: Uncertain outcome Decision rules for complete uncertainty Utility functions Expected value of](https://reader038.vdocuments.us/reader038/viewer/2022110320/56649ca75503460f9496a2f0/html5/thumbnails/1.jpg)
Chapters 19 – 20:Dealing With Uncertainties
• TPS Example: Uncertain outcome• Decision rules for complete uncertainty• Utility functions• Expected value of perfect information• Statistical decision theory• The value-of-information procedure
![Page 2: Chapters 19 – 20: Dealing With Uncertainties TPS Example: Uncertain outcome Decision rules for complete uncertainty Utility functions Expected value of](https://reader038.vdocuments.us/reader038/viewer/2022110320/56649ca75503460f9496a2f0/html5/thumbnails/2.jpg)
TPS Decision Problem 7OS Development Options:• Option B-Conservative (BC)– Sure to work (cost=$600K)– Performance = 4000 tr/sec
• Option B-Bold (BB)– If successful, Cost=$600K
Perf.= 4667 tr/sec– If not, Cost=$100K
Perf.= 4000 tr/secWhich option should we choose?
![Page 3: Chapters 19 – 20: Dealing With Uncertainties TPS Example: Uncertain outcome Decision rules for complete uncertainty Utility functions Expected value of](https://reader038.vdocuments.us/reader038/viewer/2022110320/56649ca75503460f9496a2f0/html5/thumbnails/3.jpg)
Operating System Development Options
Option BB (Bold)
Successful Not Successful
Option BC (Conservative)
Option A
Performance (tr/sec) 4667 4000 4000 2400
Value ($0.3K per tr/sec) 1400 1200 1200 720
Basic cost 600 600 600 170
Total cost 600 700 600 170
Net value NV 800 500 600 550
NV relative to Option A 250 -50 50 0
![Page 4: Chapters 19 – 20: Dealing With Uncertainties TPS Example: Uncertain outcome Decision rules for complete uncertainty Utility functions Expected value of](https://reader038.vdocuments.us/reader038/viewer/2022110320/56649ca75503460f9496a2f0/html5/thumbnails/4.jpg)
Payoff Matrix for OS Options BB, BC
State of Nature
Alternative Favorable Unfavorable
BB (Bold) 250 -50
BC (Conservative)
50 50
![Page 5: Chapters 19 – 20: Dealing With Uncertainties TPS Example: Uncertain outcome Decision rules for complete uncertainty Utility functions Expected value of](https://reader038.vdocuments.us/reader038/viewer/2022110320/56649ca75503460f9496a2f0/html5/thumbnails/5.jpg)
Decision Rules for Complete Uncertainty
• Maximin Rule. Determine minimum payoff for each option. Choose option maximizing the minimum payoff.
• Maximax Rule. Determine max-payoff for each option. Choose option maximizing max-payoff.
• Laplace Rule. Assume all states of nature equally likely. Choose option with maximum expected value.
BB = 0.5(250) + 0.5(-50) = 100 BC = 0.5(250) + 0.5(50) = 50
![Page 6: Chapters 19 – 20: Dealing With Uncertainties TPS Example: Uncertain outcome Decision rules for complete uncertainty Utility functions Expected value of](https://reader038.vdocuments.us/reader038/viewer/2022110320/56649ca75503460f9496a2f0/html5/thumbnails/6.jpg)
Difficulty with Laplace Rule
Favorable U1 U2 Expected Value
BB 250 -50 -50 50
BC 50 50 50 50
U1 : performance failure
U2 : reliability failure
![Page 7: Chapters 19 – 20: Dealing With Uncertainties TPS Example: Uncertain outcome Decision rules for complete uncertainty Utility functions Expected value of](https://reader038.vdocuments.us/reader038/viewer/2022110320/56649ca75503460f9496a2f0/html5/thumbnails/7.jpg)
Breakeven Analysis• Treat uncertainty as a parameter
P= Prob (Nature is favorable)• Compute expected values as f(P)
EV (BB) = P(250) + (1-P)(-50) = -50 + 300PEV (BC) = P(50) + (1-P)(50) = 50
• Determine breakeven point P such thatEV (BB) = EV (BC): -50 + 300P = 50, P=0.333
If we feel that actually P>0.333, Choose BB P<0.333, Choose BC
![Page 8: Chapters 19 – 20: Dealing With Uncertainties TPS Example: Uncertain outcome Decision rules for complete uncertainty Utility functions Expected value of](https://reader038.vdocuments.us/reader038/viewer/2022110320/56649ca75503460f9496a2f0/html5/thumbnails/8.jpg)
Utility Functions
Is a Guaranteed $50K
The same as
A 1/3 chance for $250K
and A 2/3 chance for -$50K?
![Page 9: Chapters 19 – 20: Dealing With Uncertainties TPS Example: Uncertain outcome Decision rules for complete uncertainty Utility functions Expected value of](https://reader038.vdocuments.us/reader038/viewer/2022110320/56649ca75503460f9496a2f0/html5/thumbnails/9.jpg)
Utility Function for Two Groups of Managers
Utility
1 2 3 4 5-1-2-3
10
20
-10
-20
Payoff ($M)
Group 1
Group 2
![Page 10: Chapters 19 – 20: Dealing With Uncertainties TPS Example: Uncertain outcome Decision rules for complete uncertainty Utility functions Expected value of](https://reader038.vdocuments.us/reader038/viewer/2022110320/56649ca75503460f9496a2f0/html5/thumbnails/10.jpg)
Possible TPS Utility Function
10
20
-10
-20
-100 100 200 300
(-80,-25)
(-50,-20)
(50,5) (100,10)(170,15)
(250,20)
![Page 11: Chapters 19 – 20: Dealing With Uncertainties TPS Example: Uncertain outcome Decision rules for complete uncertainty Utility functions Expected value of](https://reader038.vdocuments.us/reader038/viewer/2022110320/56649ca75503460f9496a2f0/html5/thumbnails/11.jpg)
Utility Functions in S/W Engineering
• Managers prefer loss-aversion– EV-approach unrealistic with losses
• Managers’ U.F.’S linear for positive payoffs– Can use EV-approach then
• People’s U.F.’S aren’t– Identical– Easy to predict– Constant
![Page 12: Chapters 19 – 20: Dealing With Uncertainties TPS Example: Uncertain outcome Decision rules for complete uncertainty Utility functions Expected value of](https://reader038.vdocuments.us/reader038/viewer/2022110320/56649ca75503460f9496a2f0/html5/thumbnails/12.jpg)
TPS Decision Problem 7
•Available decision rules inadequate
•Need better information
Info. has economic value
State of Nature
Alternative Favorable Unfavorable
BB (Bold) 250 -50
BC (Conservative) 50 50
![Page 13: Chapters 19 – 20: Dealing With Uncertainties TPS Example: Uncertain outcome Decision rules for complete uncertainty Utility functions Expected value of](https://reader038.vdocuments.us/reader038/viewer/2022110320/56649ca75503460f9496a2f0/html5/thumbnails/13.jpg)
Expected Value of Perfect Information (EVPI)
Build a Prototype for $10K– If prototype succeeds, choose BB
Payoff: $250K – 10K = $240K– If prototype fails, choose BC
Payoff: $50K – 10K = $40KIf equally likely,
Ev = 0.5 ($240K) + 0.5 ($40K) = $140K
Could invest up to $50K and do better than before – thus, EVPI = $50K
![Page 14: Chapters 19 – 20: Dealing With Uncertainties TPS Example: Uncertain outcome Decision rules for complete uncertainty Utility functions Expected value of](https://reader038.vdocuments.us/reader038/viewer/2022110320/56649ca75503460f9496a2f0/html5/thumbnails/14.jpg)
However, Prototype Will Give Imperfect Information
That is,P(IB|SF) = 0.0
Investigation (Prototype) says choose bold state of nature: Bold will fail
P(IB|SS) = 1.0Investigation (prototype) says choose bold
state of nature: bold will succeed
![Page 15: Chapters 19 – 20: Dealing With Uncertainties TPS Example: Uncertain outcome Decision rules for complete uncertainty Utility functions Expected value of](https://reader038.vdocuments.us/reader038/viewer/2022110320/56649ca75503460f9496a2f0/html5/thumbnails/15.jpg)
Suppose we assess the prototype’s imperfections asP(IB|SF) = 0.20, P(IB|SS) = 0.90
And Suppose the states of nature are equally likelyP(SF) = 0.50 P(SS) = 0.50
We would like to compute the expected value of using the prototypeEV(IB,IC) = P(IB) (Payoff if use bold)
+P(IC) (Payoff if use conservative)
= P(IB) [ P(SS|IB) ($250K) + P(SE|IB) (-$50K) ] + P(IC) ($50K)
But these aren’t the probabilities we know
![Page 16: Chapters 19 – 20: Dealing With Uncertainties TPS Example: Uncertain outcome Decision rules for complete uncertainty Utility functions Expected value of](https://reader038.vdocuments.us/reader038/viewer/2022110320/56649ca75503460f9496a2f0/html5/thumbnails/16.jpg)
How to get the probabilities we need
P(IB) = P(IB|SS) P(SS) + P(IB|SF) P(SF)
P(IC) = 1 – P(IB)
P(SS|IB) = P(IB|SS) P(SS) (Bayes’ formula)
P(IB)
P(SF|IB) = 1 – P (SS|IB)
P(SS|IB) = Prob (we will choose Bold in a state of nature where it will succeed)
Prob (we will choose Bold)
![Page 17: Chapters 19 – 20: Dealing With Uncertainties TPS Example: Uncertain outcome Decision rules for complete uncertainty Utility functions Expected value of](https://reader038.vdocuments.us/reader038/viewer/2022110320/56649ca75503460f9496a2f0/html5/thumbnails/17.jpg)
Net Expected Value of PrototypePROTO
COST, $KP (PS|SF) P (PS|SS) EV, $K NET EV,
$K
0 60 0
5 0.30 0.80 69.3 4.3
10 0.20 0.90 78.2 8.2
20 0.10 0.95 86.8 6.8
30 0.00 1.00 90 0
8
4
010 20 30
NE
T E
V, $
K
PROTO COST, $K
![Page 18: Chapters 19 – 20: Dealing With Uncertainties TPS Example: Uncertain outcome Decision rules for complete uncertainty Utility functions Expected value of](https://reader038.vdocuments.us/reader038/viewer/2022110320/56649ca75503460f9496a2f0/html5/thumbnails/18.jpg)
Conditions for Successful Prototyping (or Other Info-Buying)
1. There exist alternatives whose payoffs vary greatly depending on some states of nature.
2. The critical states of nature have an appreciable probability of occurring.
3. The prototype has a high probability of accurately identifying the critical states of nature.
4. The required cost and schedule of the prototype does not overly curtail its net value.
5. There exist significant side benefits derived from building the prototype.
![Page 19: Chapters 19 – 20: Dealing With Uncertainties TPS Example: Uncertain outcome Decision rules for complete uncertainty Utility functions Expected value of](https://reader038.vdocuments.us/reader038/viewer/2022110320/56649ca75503460f9496a2f0/html5/thumbnails/19.jpg)
Pitfalls Associated by Success Conditions
1. Always build a prototype or simulation– May not satisfy conditions 3,4
2. Always build the software twice– May not satisfy conditions 1,2
3. Build the software purely top-down– May not satisfy conditions 1,2
4. Prove every piece of code correct– May not satisfy conditions 1,2,4
5. Nominal-case testing is sufficient– May need off-nominal testing to satisfy conditions
1,2,3
![Page 20: Chapters 19 – 20: Dealing With Uncertainties TPS Example: Uncertain outcome Decision rules for complete uncertainty Utility functions Expected value of](https://reader038.vdocuments.us/reader038/viewer/2022110320/56649ca75503460f9496a2f0/html5/thumbnails/20.jpg)
Statistical Decision Theory: Other S/W Engineering Applications
How much should we invest in:– User information gathering– Make-or-buy information– Simulation– Testing– Program verification
How much should our customers invest in:– MIS, query systems, traffic models, CAD,
automatic test equipment, …
![Page 21: Chapters 19 – 20: Dealing With Uncertainties TPS Example: Uncertain outcome Decision rules for complete uncertainty Utility functions Expected value of](https://reader038.vdocuments.us/reader038/viewer/2022110320/56649ca75503460f9496a2f0/html5/thumbnails/21.jpg)
The Software Engineering Field Exists Because
Processed Information Has Value