topic 7 - uncertainty mgmt.pptx

Uncertainty Management

SCO 6041 – Class 7

Uncertainty Management

• Agenda

– PERT Analysis

– Monte-Carlo Analysis

The Role of Chance …

Source: Dilbert Comic Strips Archivehttp://www.dilbert.com/strips/

http://www.dilbert.com/strips/

A Roadmap

Uncertainty Risks Ambiguity

An understanding that important aspects of the project are not determined, but rather random variables with probability distributions. Examples: payoffs, task durations, etc.

A knowledge that the project depends on the outcomes of certain events in the future that can be fully described by their probability distributions. Examples: Market entry of competitors, outcome of drug trials, etc.

An awareness that the project is influenced by unforeseeable events, which cannot be predicted, and are not even recognized at the time of project planning.

Focus of this chapter

Key Ideas of This Class

• Instead of taking task durations as determined, we treat them as random variables

• We can then calculate the likelihood of finishing the project by a certain time

• This can serve as a basis for determining the ‘right’ project buffer

Topic Objective: Establish better estimates of task duration, buffers, and project completion

Central Assumption

0

0.01

0.02

0.03

0.04

0.05

0.06

5 6 7 8 9 10

Prob

abili

ty

Duration

Task Durations follow a Beta Distribution (right skewed)

Maximum Time (Pessimistic Estimate)

Minimum Time (Optimistic Estimate)

Mode(Most likely time)

Mean(Average Time)

Bounded by Min & Max (in contrast to Normal distribution)

0

0.01

0.02

0.03

0.04

0.05

0.06

5 6 7 8 9 10

Prob

abili

ty

Duration

Central AssumptionTask Durations follow a Beta Distribution (centered)



Mean(Average Time)

0

0.01

0.02

0.03

0.04

0.05

0.06

0.07

5 6 7 8 9 10

Prob

abili

ty

Duration

Central AssumptionTask Durations follow a Beta Distribution (left skewed)



Mean(Average Time)

Mode(Most likely time)

9

PERT

• Program Evaluation and Review Technique (PERT)

• Developed by U.S. Navy in 1950’s– Navy Special Projects Office– Polaris submarine weapon system

“…. With units of time as a common denominator, PERT quantifies knowledge about the uncertainties involved in developmental programs requiring effort at the edge of, or beyond, current knowledge of the subject – effort for which little or no previous experience exists. ….”

— Willard Fazar (Head, Program Evaluation Branch, Special Projects Office, U. S. Navy), The American Statistician, April 1959.

PERT Analysis

Step 1: Instead of asking for task durations, ask for optimistic, pessimistic and most likely

Task Optimistic Most Likely PessimisticA 1 2 3B 7 8 9C 11 13 15D 8 10 12E 10 15 23F 4 7 14G 2 4 11H 3 4 7I 16 21 35J 4 5 6K 2 2 2L 2 2 2M 8 10 14N 1 1 1

Optimistic = Minimum Time

Most Likely = Mode

Pessimistic = Maximum Time

PERT Analysis

Step 2: Calculate the mean and variance of each task duration using the following formulas:

4*6

Optimistic Pessimistic MostLikely

22

36Pessimistic Optimistic

PERT Analysis

Step 2: Calculate the mean task duration, and the variance of each task duration

Task Optimistic Most Likely Pessimistic Mean VarianceA 1 2 3 2 .11B 7 8 9 8 .11C 11 13 15 13 .44D 8 10 12 10 .44E 10 15 23 15.5 4.69F 4 7 14 7.67 2.78G 2 4 11 4.83 2.25H 3 4 7 4.33 .44I 16 21 35 22.5 10.03J 4 5 6 5 .11K 2 2 2 2 0L 2 2 2 2 0M 8 10 14 10.33 1N 1 1 1 1 0

PERT AnalysisStep 3: Schedule the Project Using Mean Durations

Task A2 days0 slack

Task B8 days0 slack

Task C13 days0 slack

Task D10 days0 slack

Task E15.5 days

3 slack

Task F7.67 days

3 slack

Task G4.83 days

3 slack

Task J5 days

42.5 slack

Task H4.33 days

0 slack

Task I22.5 days

0 slack

Task K2 days

20.5 slack

Task L2 days

45.8 slack

Task M42.5 days44 slack

Task N1 day

0 slack

0 2

0 2

2 17.5

5 20.5

2 10

2 10

10 23

10 23

17.5 25.2

20.5 28.2

23 33

23 33

25.2 30

28.2 33

2 7

44.5 49.5

33 37.3

33 37.3

37.3 59.8

37.3 59.8

37.3 39.3

57.8 59.8

7 9

55.8 57.8 7 17.3

49.5 59.8

59.8 60.8

59.8 60.8

ES EF

LS LF

Ear

liest

S

tart

Ear

liest

Fi

nish

Note: I have assumed that a second Product Architect was hired (no resource conflict) and that K has to come after L (no resource conflict for product manager). See case.

PERT AnalysisStep 4: Calculate Mean & Standard Deviation of Critical Path

Task A2 days0 slack

Task B8 days0 slack

Task C13 days0 slack

Task D10 days0 slack

Task H4.33 days

0 slack

Task I22.5 days

0 slack

Task N1 day

0 slack

0 2

0 2

2 10

2 10

10 23

10 23

23 33

23 33

33 37.3

33 37.3

37.3 59.8

37.3 59.8

59.8 60.8

59.8 60.8

Mean(μ) 2 8 13+ + 10+ 4.33+ 22.5+ 1+ 60.8=

Variance(σ2) .11 .11 .44+ + .44+ .44+ 10.03+ 0+ 11.6=

Standard Deviation

(σ)3.4√𝟏𝟏 .𝟔 =Assume independence between tasks

(allows variances to be additive)

PERT AnalysisStep 5: Plot cumulative normal distribution

50 54.000000000000158.000000000000162.000000000000266.000000000000270.00000000000030

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

Project Duration

Cum

ulati

ve P

roba

bilit

y

PERT Analysis

Hint: To plot the cumulative normal distribution in Excel, create (and then plot) the following Table:

X-Axis Y-Axis

50 =NORM.DIST(50,μ,σ,1)


… …


PERT Analysis

Step 5: Why Normal Distribution?

Central Limit Theorem: The sum of several random variables with any (though identical) distribution is approximately normally distributed


50 54.200000000000158.400000000000162.600000000000266.80000000000020

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

Project Duration

Cum

ulat

ive

Prob

abili

ty

There is a 90% chance of the project finishing in less than 65 days.


50 54.200000000000158.400000000000162.600000000000266.80000000000020

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

Project Duration

Cum

ulat

ive

Prob

abili

ty There is a 70% chance of the project finishing in less than 63 days.

In Excel, use =NORMINV(0.70,60.8,3.4)

PERT AnalysisStep 6: Understand Uncertainty

61

Likelihood of Finishing the Project

on Time

Total Time until Project is Due

50%

62 60%

63 70%

64 80%

65 90%

69 99%

PERT AnalysisStep 7: Planning Project Buffers

61

Likelihood of Finishing the Project

on Time


50%

6260%

6370%

6480%

6590%

6999%

0

Project Buffer (Days)

1

2

3

4

8

PERT - Criticisms

• PERT assumes that the critical path remains constant – but uncertainty in task durations implies that critical paths may change

• PERT assumes that task durations are independent of each other – but since the same people were involved in creating different estimates, and since the same resources may be involved in different tasks, this assumption may not hold

Monte Carlo Simulations

• A lot of times we have to make assumptions simply to know what a distribution looks like. If uncertain events interact in complex ways, we often have no clue what the real underlying distribution looks like – but we can simulate it!


• Example of a known distribution: Rolling a dice

1 2 3 4 5 6

-0.0499999999999997

2.91433543964104E-16

0.0500000000000003

0.1

0.15

0.2

0.25

0.3

0.35

Outcome

Prob

abili

ty

Expected Value: 3.5


• What does the distribution of the maximum of two dice look like?

D1 D2 Score D1 D2 Score D1 D2 Score D1 D2 Score D1 D2 Score D1 D2 Score

1 1 1 2 1 2 3 1 3 4 1 4 5 1 5 6 1 6

1 2 2 2 2 2 3 2 3 4 2 4 5 2 5 6 2 6

1 3 3 2 3 3 3 3 3 4 3 4 5 3 5 6 3 6

1 4 4 2 4 4 3 4 4 4 4 4 5 4 5 6 4 6

1 5 5 2 5 5 3 5 5 4 5 5 5 5 5 6 5 6

1 6 6 2 6 6 3 6 6 4 6 6 5 6 6 6 6 6


• What does the distribution of the maximum of two dice look like?

1 2 3 4 5 60.00

0.05

0.10

0.15

0.20

0.25

0.30

0.35

Outcome

Prob

abili

ty

Expected Value: 4.47


• Suppose I offer you the following gamble: I get to roll a dice. You get to roll two dice, record the maximum, then re-roll the dice, record the minimum, and your score is the maximum-minimum. If I beat you, I get $1 from you, if you beat me, you get $1 from me.

Is this a good gamble?

• We can simulate this by throwing a lot of dice and recording the results. Fortunately, there is software that can do this for us …


• This is the Probability Distribution of your Score


• Your Probability of Winning: 23%• Your Probability of Drawing: 13%• Your Probability of Loosing: 64%

Simulating the Echelon Release Project

• Histogram


• Parameter Estimate Comparison

60.8

Monte Carlo SimulationPERT

61.3Mean

Standard Deviation 3.4 3.8


• Project Duration

61

PERT Likelihood of Finishing on Time


50%

62 60%

63 70%

64 80%

65 90%

69 99%

Monte Carlo Likelihood of

Finishing on Time

50%

59%

68%

75%

82%

97%

PERT underestimates the project duration, since the method ignores that critical paths can change!

The Path-Changing Issue …

Task

Task

Task

Task

Task

Task

Task

Project A

Task

Task

Task

Task

Task

Task

Task

Project B

In which project would you be more distrustful of PERT estimates?


• Likelihood of Being Critical (Criticality)

A

Likelihood of Being CriticalTask

100%B 80%C 80%D 80%E 20%F 20%G 20%H 100%I 100%J 0%K 0%L 0%

M 0%N 100%

These estimates give you a good idea which tasks are critical and require more intense monitoring and control

Simulating the Echelon Release Project• Contribution to Variance in Project Duration (Cruciality)

A

ContributionTask

1.1%B 0.5%C 2.9%D 3.3%E 2.1%F 0.8%G 0.5%H 3.7%I 84.9%J 0%K 0%L 0%

M 0%N 0%

These estimates give you a good idea which tasks drive uncertainty in project duration, and therefore where to target risk reduction efforts

0

1

2

3

4

5

6

0 1 2 3 4 5 6 7 8

Dura

tion

Task

B

Duration Task A

0123456789

10

0 1 2 3 4 5 6 7

Dura

tion

Task

B

Duration Task A

Task Correlations

0

2

4

6

8

10

12

14

0 1 2 3 4 5 6 7 8

Dura

tion

Task

B

Duration Task A

High, positive correlation

No correlation

High, negative correlation

- Estimates were made by same person- Problems from one task transfer to the

next- Same resources work on multiple tasks

- Tasks are independent and their forecasts were not prepared by the same person

- Task boundaries are not well defined- Team members will try to pre-empt

other people’s work

Variance and CorrelationsTwo Independent Random Variables

7 7.4 7.8 8.2 8.6 9 9.4 9.810.2

10.6 1111.4

11.812.2

12.6 130

0.050.1

0.150.2

0.250.3

0.350.4

0.45

7 7.4 7.8 8.2 8.6 9 9.4 9.810.2

10.6 1111.4

11.812.2

12.6 130

0.050.1

0.150.2

0.250.3

0.350.4

0.45

1717.4

17.818.2

18.6 1919.4

19.820.2

20.6000000000001

21.0000000000001

21.4000000000001

21.8000000000001

22.2000000000001

22.6000000000001

23.00000000000010

0.050.1

0.150.2

0.250.3

0.350.4

0.45

μ1 = 10, σ1 = 1 μ2 = 10, σ2 = 1

μ = 20, σ = 1.41

1 2

2 21 2

10 10 20

2 1.41

Variance and CorrelationsTwo Correlated Random Variables

7 7.4 7.8 8.2 8.6 9 9.4 9.8 10.2

10.6 11 11

.411

.812

.212

.6 130

0.050.1

0.150.2

0.250.3

0.350.4

0.45

7 7.4 7.8 8.2 8.6 9 9.4 9.8 10.2

10.6 11 11

.411

.812

.212

.6 130

0.050.1

0.150.2

0.250.3

0.350.4

0.45μ1 = 10, σ1 = 1 μ2 = 10, σ2 = 1

μ = 20, σ = 1.73

1 2

2 2 21 2 12

21

1

10 10 20

2

2 1

2 1

3 1.73

ρ = 0.5

1717.4

17.818.2

18.6 1919.4

19.820.2

20.6000000000001

21.0000000000001

21.4000000000001

21.8000000000001

22.2000000000001

22.6000000000001

23.00000000000010

0.050.1

0.150.2

0.250.3

0.350.4

0.45

Variance and CorrelationsTwo Correlated Random Variables

• Summing independent random variables reduces the standard deviation, since summing always means that high and low values cancel out

• As the correlation between random variables increases, so does the standard deviation of their sum, since a higher correlation implies less ‘canceling out’ of high and low values

40

A Note on Correlation Matrices

• Suppose that the correlation between A and B is 0.9, and the correlation between B and C is 0.9. Could A and C be uncorrelated?

• You can easily end up with ‘inconsistent’ correlation matrices. Crystal Ball will ‘fix’ them for you by altering your correlations to be as close as possible to a valid correlation matrix.


• Project Duration

61

PERT LikelihoodTotal Time until Project is Due

50%

62 60%

63 70%

64 80%

65 90%

69 99%

Monte Carlo Likelihood

50%

59%

68%

75%

82%

97%

Ignoring correlations between task durations also underestimates the project duration

Monte Carlo Likelihood with

Correlations

52%

58%

64%

69%

74%

88%

PERT or Monte Carlo?

• PERT analysis is easy to carry out and understand– Available in MS Project– Can be easily done in Excel

• Monte Carlo simulations can be more accurate and allow more modeling flexibility– Requires specialized software– Requires some spreadsheet development

Key Take-Aways

• Uncertainty– Simple variations in key project parameters can be

easily incorporated into planning

• PERT Analysis– Simple and reasonably effective

• Monte Carlo Simulations– Sophisticated and flexible

topic 7 - uncertainty mgmt.pptx

Documents