succeeding with software in a complex, chaotic world · construx® stevemcc@construx.com construx...

Construx®

Tools for Dealing with Uncertainty

Succeeding with Software in

a Complex, Chaotic World

Construx®

These presentation materials are © 1996-2018

Construx Software Builders, Inc.

All Rights Reserved. No part of the contents of this

presentation may be reproduced or transmitted in any

form or by any means without the written permission

of Construx Software Builders, Inc.

Copyright Notice

Construx®

stevemcc@construx.com

Construx

10900 NE 8th Street, #1350

Bellevue, WA 98004

+1 (866) 296-6300

www.construx.com

Steve

McConnell

Introduction to

Uncertainty

5 What Do I Mean by “Uncertainty”?

► Unknowns (more on this later)

► Variability (in individual productivity, team productivity, defect

rates, interpretation of functionality, etc.,etc.)

► Risk (financial definition of “Risk”)

► Literal “uncertainty”—“The state of not able to be relied on, not

definite”

6 In other words,

Uncertainty ≈

Unknowns ≈ Variability ≈ Risk ≈ Unreliability

7 A Case Study in Uncertainty

An Early Extreme

Programming

Project

8 A major software products

company adopted XP very early

Team grew to 100+ people

9 Project Characteristics

Onsite coaches

Onsite customer

Focus on always doing the “next

most important thing”

Short iterations

Delivery of working software

every iteration

10 This was a Well-Run Project!

This was a highly disciplined,

rigorous, well-supported

Extreme Programming project

This project should have been

successful (in some sense)

11 Project Result

After 18 months the project was

cancelled

Company wrote off $90 million

Company laid off 200 staff

The company said,

“We are never doing another

Extreme Programming project

again”

12 Cause of Failure?

► The discipline of “always do the next most useful thing” was not

converging to a viable product

► After 18 months the team had failed to converge to anything the

company thought was coherent enough to sell

► The company ultimately lost confidence that the team would ever

produce a coherent product

13 An Ineffective Approach to Uncertainty

► This is not a criticism of Extreme Programming,

per se

► It is a criticism of the way that uncertainty was addressed on this

particular project (especially in product concept & requirements)

14 Variations on a Theme

► This basic pattern of projects failing because of mishandled

uncertainty has been repeated thousands of times on projects of

all kinds (Scrum, Kanban, Lean, Waterfall, Scrummerfall, etc.)

Software Projects are Beset by

Uncertainty From All Sides

16 Example Sources of Uncertainty

Acquisitions

Additional customers

Changes in implementation

Architecture changes

Art requirements

Budget changes

Budget cuts

Bug fixes

Bugs being found during late

testing

Business crime (e.g., embezzlement)

Business logic / regulatary changes

Change in market requirements

Change in priorities

Change in quality criteria

Change in technology

Change in your company's credit rating

(i.e., staff size changes, resource

changes)

Changes due to scale (10 users =>

10,000 users)

Changes in clients or third parties

Changes in in-house or third-party tools

Changes in requirements

Changes in staffing

17 Example Sources of Uncertainty

Changes to delivery time (i.e., the

deadline changes)

Changing customers

Changing graphics

Code drop from the vendor

Code reuse

Coding errors

Communication protocol changes

between client and server

Company culture changes (e.g., if

you get acquired)

Company direction

Competition features change

Compiler changes (toolset changes in

general)

Compliance changes

Cost reduction and scope reduction

Target customer changes

Dependencies change

Dependencies not fulfilling promises

Depracation of old libraries (client

software)

End of life for certain parts

18 Example Sources of Uncertainty

End of life of system you're

working on

End or support contracts for

supported technologies

Environment (client

environment)

External APIs change

Facebook APIs change

Design approach fails

Financial benefit (need to

release a new version sooner

than expected for cash flow)

Funding changes

Infrastructure/framework changes

Integration changes (system integration)

Integration of acquired company's code

base to fit your standards and what not

Integration with another code base

Internationalization issues (language,

currency)

IP infringement

Lead developer resigns

Management changes

New research that can affect your existing

stuff

Where does Uncertainty come from?

It comes from developing software in our

complex, chaotic world.

Role of Uncertainty

21 Role of Uncertainty

Uncertainty is a common blind spot—

Most projects don’t realize that Uncertainty

is a factor that needs to be addressed

22 Uncertainty is Usually Not Addressed Well

Common responses to Uncertainty tend to be ineffective

► Plan everything! (Waterfall)

► Plan nothing! (Early Agile)

► Don’t have a plan! (Since the beginning of time)

These are caricatures, but too close to the truth for comfort

Talk Roadmap

24 Understanding Software Projects Lectures

25

Size Human

Variation

Defects Uncertainty

The Four Factors Lifecycle Model

26 Talk Roadmap

► Introduction

► Categories of Uncertainty

► Tools for Dealing with Uncertainty

Uncertainty and Size

Uncertainty and Defects

Uncertainty and Human Variation

Uncertainty and the SLM

Uncertainty and the Four

Factors Lifecycle Model

► Conclusions

Categories of Uncertainty

Tool:

Categories of Uncertainty

29 Categories of Uncertainty

Known Unknown

Knowable Unknowable

Static Situation Dynamic/Emergent Situation

Current State Future State

Common Cause Special Cause

30 Categories of Uncertainty

Some categories of uncertainty are more problematic than others

31 Categories of Uncertainty

Known Unknown

Knowable Unknowable

Static Situation Dynamic/Emergent Situation

Current State Future State

Common Cause Special Cause

32 Categories of Uncertainty

Known Unknown

Knowable Unknowable

Static Situation Dynamic/Emergent Situation

Current State Future State

Common Cause Special Cause

33 Categories of Uncertainty

Known Unknown

Knowable Unknowable

Static Situation Dynamic/Emergent Situation

Current State Future State

?

Common Cause Special Cause

34 Categories of Uncertainty

Known Unknown

Knowable Unknowable

Static Situation Dynamic/Emergent Situation

Current State Future State

?

Common Cause Special Cause

35 Categories of Uncertainty

Known Unknown

Knowable Unknowable

Static Situation Dynamic/Emergent Situation

Current State Future State

?

Common Cause Special Cause

36 Summary of Categories of Uncertainty

Again, some categories of uncertainty (and combinations) are more

problematic than others

Tool:

Essential vs. Accidental Uncertainty

38 Essential vs. Accidental Uncertainty

Adapting Fred Brooks’ distinction between Essential and Accidental,

“Essential” Uncertainty inherent in the problem we’re trying to

solve

“Accidental” Uncertainty created by our solution

39 Examples of Accidental Uncertainty

We decide to hire new staff with unknown capabilities

We decide to develop functionality in an area where our users don’t

quite know what they want

We decide to try a new design approach

We decide to use a new programming language

We decide to use a new methodology

All the above, at the same time!

40 Accidental Uncertainty

This is Uncertainty that we take on, through our decisions, and which

in many cases is discretionary

Tools for Dealing with

Uncertainty

The Interaction Between

Uncertainty and Size

Human

Variation

Defects

Size

Uncertainty

43 Uncertainty and Size

The larger the project is, the disproportionately larger impact

uncertainty will have

44 Cocomo II and Uncertainty – Small Project

Cumulative Effect of

Uncertainty Factors:

5x

45 Cocomo II and Uncertainty – Large Project

Cumulative Effect of

Uncertainty Factors:

8x

Tool:

Keep Projects Small

47 Project Outcomes by Project Size

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

10 FP

500-1000 LOC

100 FP

5K-10K LOC

1,000 FP

50K-100K LOC

10,000 FP

500K-1M LOC

100,000 FP

5M-10M LOC

Perc

enta

ge

Failed

Late

On Time or Early

Data from Economics of Software Quality, Capers Jones, 2012

48 Essential vs. Accidental Uncertainty and Size

Size is often an Essential

source of uncertainty, but

… sometimes …

it’s Accidental

49 Final Word on Uncertainty and Size

High Uncertainty and Large Size is a Bad Combination

50 Remember That Early XP Project?

Known Unknown

Knowable Unknowable

Static Situation Dynamic/Emergent Situation

Current State Future State

Now Add Large Size to This (200 Staff)

Common Cause Special Cause

The Interaction Between

Uncertainty and Defects

Size Human

Variation

Defects Uncertainty

Tool:

Intellectual Phase Profile

53 Intellectual Phase Profile for Projects

and Activities

Schedule

Focus

Discovery

(Requirements)

Invention

(Design) Construction

(Implementation)

54 Intellectual Phase Profile

Overlap = Wicked Problem

Schedule

Focus

Discovery

(Requirements)

Invention

(Design) Construction

(Implementation)

“Wickedness”

(not just iteration)

55 Intellectual Phase Profile

Schedule

Focus

Discovery

(Requirements)

Invention

(Design) Construction

(Implementation)

Overlap =

►Dependencies

► Potential for work

that will need to

be redone

► Potential for

defects

► Risk

►Uncertainty

56 Intellectual Phase Profile and Defects

Overlap = Uncertainty = Defect Potential

57 Uncertainty and Defects

Uncertainty handled badly Rework (aka Defects)

Plan wrong

Experiment wrong

Design wrong

Build wrong

Build wrong thing

58 Uncertainty and Defects

Uncertainty Defects Extra Cost, Longer Schedules

This becomes a vicious circle, with

Extra unplanned work More uncertainty

59 The Intellectual Phase Profile Provides

Key Questions to Ask

► How much uncertainty do I expect in requirements?

► How much uncertainty do I expect in design?

► How much interaction should I expect between requirements and

design, wrt uncertainty?

► How much uncertainty should I expect in construction?

► How much interaction should I expect between construction,

requirements and design, wrt uncertainty?

60 Variations in Sources of Uncertainty-

Concentrate on High Focus Areas

61 Expect More Defects!

Expect more defects in the

areas of high uncertainty

Focus

High Focus

Area

High Activity

Overlap

(wickedness) in a

High Focus Area

= High Defect

Potential

62 Ask About Essential vs. Accidental Uncertainty

How much Uncertainty is inherent in Discovery, Invention, and

Construction (“Essential”) vs. how much is being created by your

team’s solution? (“Accidental”)?

Avoid Accidental Uncertainty

63 An Example of Accidental Uncertainty

Related to Defects

“Defect Detection Gap”

Defect creation

Defect removal

Well-Run Project

Time Time

Cumulative

Defects

Typical Project

This area is

accidental

uncertainty This is too,

but better

controlled

The Interaction Between

Uncertainty and Human Variation

Size

Defects

Human

Variation

Uncertainty

65 Uncertainty and Human Variation

How much uncertainty can you safely take on?

Tool:

MSF Risk Offset

67 MSF =

Motorcycle Safety

Foundation

68 MSF Risk

Offset Tool

69

Resu

lt

What is Your Risk Offset?

Skill Scale:

High

Skill Scale:

Low

Risk Scale:

High

Risk Scale:

Low

70

Resu

lt

What is Your Risk Offset?

Skill Scale:

High

Skill Scale:

Low

Average Risk with Higher Skill:

Good Result

Average Risk with Average

Skill: Average Result

Average Risk with Lower Skill:

Bad Result

Average Risk with Lower Skill:

Worse Result

Low Risk with Higher Skill:

Best Result Risk Scale:

High

Risk Scale:

Low

71 What Are The

Software

Implications

of the MSF

Risk Offset

Tool?

72

Resu

lt

Software Implications: All These Apply!

Average Uncertainty with

Higher Skill: Good Result

Average Uncertainty with

Average Skill: Average Result

Average Uncertainty with

Lower Skill: Bad Result

Average Uncertainty with

Lower Skill: Worse Result

Low Uncertainty with Higher

Skill: Best Result Skill:

High

Skill:

Low

Uncertainty:

High

Uncertainty:

Low

73

Skill:

High

Skill:

Low

Uncertainty:

High

Uncertainty:

Low

Resu

lt

Software Implications

Typical Result With an Average Skill Level?

Average Result ?

74

Skill:

High

Skill:

Low

Uncertainty:

High

Uncertainty:

Low

Resu

lt

Software Implications

Best Result With an Average Skill Level?

Good Result

?

Best Result is Not Attainable at

this Skill Level

75

Skill:

High

Skill:

Low

Uncertainty:

High

Uncertainty:

Low

Resu

lt

Software Implications

Result With a High Skill Level?

Best Result is Readily

Attainable at this Skill Level

Good Result is Attainable

Even With High Risk ?

76

Skill:

High

Skill:

Low

Uncertainty:

High

Uncertainty:

Low

Resu

lt

Software Implications

Best Result with a High Risk/Uncertainty Level?

Good Result ?

Bad Result

Best Result is Not Attainable at

this Risk/Uncertainty Level

77

Skill:

High

Skill:

Low

Uncertainty:

High

Uncertainty:

Low

Resu

lt

Software Implications

Best Result with a Low Skill Level?

Bad Result

Bad Result (Best Possible

Result with Low Skill Level) ?

78

Skill:

High

Skill:

Low

Uncertainty:

High

Uncertainty:

Low

Resu

lt

XP Case Study

Bad Result

?

?

79

Skill:

High

Skill:

Low

Uncertainty:

High

Uncertainty:

Low

Resu

lt

XP Case Study

?

?

Even Worse Result

Extending the MSF Tool

81

Resu

lt

Uncertainty and Size

Size:

Small

Size:

Large

Average Uncertainty with

Small Size: Good Result

Average Uncertainty with

Average Size: Average Result

Average Uncertainty with

Med-Large Size: Bad Result

High Uncertainty with

Med-Large Size: Worse Result

Low Uncertainty with

Small-Medium Size: Best Result Uncertainty:

High

Uncertainty:

Low

82 Extending the MSF Tool

We can create analogous diagrams with any pair of factors from the

Four Factors Lifecycle Model

Size Human

Variation

Defects Uncertainty

The Interaction Between

Uncertainty and the SLM

Size Human

Variation

Defects Uncertainty

84 Interaction Between Uncertainty and the SLM

► More uncertainty calls for tighter feedback loops, i.e., more

iteration

► Briefly, …

85 The Lifecycle Building Block

Require-

ments

Archi-

tecture

Construction System

Test

86 Variation: Length of Iteration

87 Variation: Iterating Within an Activity

88 Variation: Iterating Across Activities

89 Interaction Between Uncertainty and the SLM

► Overall, iteration is a powerful tool for addressing Uncertainty

► This is covered extensively in the USP Lecture series

Uncertainty and the Whole Four

Factors Software Lifecycle Model

Size Human

Variation

Defects Uncertainty

Tool: Four Factors Lifecycle Model Radar Chart

92 Four Factors Lifecycle Model Radar Chart

The kind of Uncertainty we’re talking about here is the Uncertainty

of, “Should we do this project?”

93 Four Factors

Lifecycle Model

Radar Chart

Size

Defects

(Defect Potential)

Uncertainty Human

Variation

4

3

2

1

94 Proposed Radar

Chart Scale (Draft)

15+ Green

13-14 Yellow

11-12 Orange

≤10 Red

Defects

(Defect Potential)

Uncertainty Human

Variation

Size

4

3

2

1

95

4

3

2

1

Ideal Project

Score: 16, Green

Size

Defects

(Defect Potential)

Uncertainty Human

Variation

96

4

3

2

1

Worst Possible

Project

Score: 4, Red

Defects

(Defect Potential)

Uncertainty Human

Variation

Size

97

4

3

2

1

Best Possible

Project With High

Uncertainty

Score: 13,

Yellow

Defects

(Defect Potential)

Uncertainty Human

Variation

Size

98

4

3

2

1

Best Possible

Project With

Large Size

Score: 13,

Yellow

Defects

(Defect Potential)

Uncertainty Human

Variation

Size

99

4

3

2

1

Best Possible Project

With Both High

Uncertainty and

Large Size

Score: 10,

Red

Defects

(Defect Potential)

Uncertainty Human

Variation

Size

100

4

3

2

1

XP Case Study

Project

Score: ~10, Red

Defects

(Defect Potential)

Uncertainty Human

Variation

Size

New methodology

New product

Lots of requirements

volatility

200 staff

>18 months

Large team

(regression to

mean)

Team new to XP

Extensive

coaching

Disciplined use of XP’s

quality practices

101

4

3 2

1

What Do We Do With Our Score?

Suggested actions

Green Proceed!

Yellow Take steps to reduce the

score, if possible

Orange Proceed only if reducing

the score to yellow or green

is possible

Red Proceed only if reducing

the score to yellow or green

is possible

Defects

(Defect Potential)

Uncertainty Human

Variation

Size

3

2

1

Conclusions

103 Summary

► Software projects with high uncertainty are common

► Software projects with high uncertainty are difficult

► Numerous tools exist to help understand and deal with

uncertainty

104 These Tools Help Understand Uncertainty

► Categories of Uncertainty

► Essential vs. Accidental Uncertainty

► Project Size and Cocomo II’s Uncertainty-Related Factors

► Defects and Intellectual Phase Profiles

► Human Variation and MSF Uncertainty Offset Tool

► Software Lifecycle Model and focused iteration

► Four Factors Lifecycle Model Radar Chart

105 These Additional Tools Can Help Too

► Hardest Part First

► Cone of Uncertainty

► Cynefin

► Feature Build Up

► Prototyping

► Spikes

► Defect Detection Gap

Size Human

Variation

Defects Uncertainty

And Much More!

106 See Much More Detail in the Understanding

Software Projects Lectures!

Construx®

hello@construx.com

Construx

10900 NE 8th Street, Suite 1350

Bellevue, WA 98004

+1 (866) 296-6300

www.construx.com

We believe that each and every software

project can be successful.

We believe that developing the professional

skills of companies, teams, and individuals is

the best way to make software projects

more successful.

For information about our training and

consulting services, contact

stevemcc@construx.com.