agile way of dealing with uncertainty in a complex adaptive world

Naresh Jainnaresh@agilefaqs.com

twitter: @nashjain

http://nareshjain.com

1Saturday 1 September 2012

Agile Way of dealing with Uncertainty in a

Complex Adaptive World

twitter: @nashjain

Video on Selective Attention http://www.youtube.com/watch?v=vJG698U2Mvo

Selec%ve A)en%on

The process by which a person can selec%vely pick out one s%mulus from a mixture of s%muli occurring simultaneously.

Which line is the longest?

Müller-Lyer optical illusion

Which Orange Circle is bigger?

Ebbinghaus optical illusion

Pet Plant Research @ an elderly Nursing Home

Student Volunteers @ Nursing Home

What would you prefer?

A lo)ery %cket with a random number

or a number you’ve picked?

In the Casino, if you toss the dice

or someone else tosses the dice?

In the Casino, if you toss the dice

or someone else tosses the dice?

Do you think it will make any difference?

Illusion of Control

Our desire to control is so powerful that the feeling of being in control is so rewarding that people oLen act as though they can control the uncontrollable.

Electrical Shock Research

High Volts Shock Group vs. Low Volts Shock Group

5 shocks of 5 volts each vs. 3 shocks of 2-‐4 volts each

Every 10 seconds vs. Random %me interval

Electrical Shock Research

High Volts Shock Group vs. Low Volts Shock Group

5 shocks of 5 volts each vs. 3 shocks of 2-‐4 volts each

Every 10 seconds vs. Random %me intervalWhich Group would have sweated more, whose heart beats would be faster and who claimed

to be more afraid?

Uncertainty

Why?Our desire to Control

How do we deal with Uncertainty?

Predictability Paradox

How to Organize a Children's Party?

A video by Dave Snowdenhttp://www.youtube.com/watch?v=Miwb92eZaJg

Why is there only ONE Toyota or Apple today?

Products & Processes are like haircutsCopying someone else’s rarely works

Retrospec%ve Coherence

Hindsight does not lead to foresight!

Tech Talks!

Taking ownership of a simple process

Adapted from Jeff Patton

The Ball Point Game

Your goal:

As a team predictably "process" the most number of balls in a round by passing a ball to each member

You have 3 rounds to get the best score you can

The Ball Point Game

Your goal:

Simple structure:

Predict the number of balls you can process

Pass balls for 2 minutes (no more, no less)

Take 2 minute to discuss and improve your strategy

The Ball Point Game

Your goal:

Simple structure:

Predict the number of balls you can process

Pass balls for 2 minutes (no more, no less)

Take 2 minute to discuss and improve your strategy

Simple rules:

Everyone must touch the ball for it to be “done”

The ball must have “air %me” -‐ it must be tossed or dropped between team members

Core Agile concepts learned?

Adapted from Jeff Patton29Saturday 1 September 2012

Ideal processes use a simple framework -‐ like a game

Changing your strategies & tac7cs, not the framework, allow you to improve

Process improvement comes from change

Skill improvement come from prac7ce

Certain kind of es%mates improves with frequent measurement

Velocity is agile’s language for measuring throughput

Visibility of work helps us make improvement decisions

Reflec7on: observing, measuring & changing is the means for process improvement

Team work is an individual skill

“Simple, clear purpose and principles give rise to complex

and intelligent behavior.

Complex rules and regulations give rise to simple

and stupid behavior.”

Dee Hock30Saturday 1 September 2012

Your SoLware Development Game?

What would be:

Your goal

Simple structure

Simple rules

The Agile Game

Your goal: As a team, predictably deliver max value to users & stakeholders

The Agile Game

Simple structure: As a team, set a goal & plan to accomplish the work

Deliver working solu%on by the end of a fixed cycle

Reflect & improve your Product, Plan, People and Process

The Agile Game

Simple structure: As a team, set a goal & plan to accomplish the work

Deliver working solu%on by the end of a fixed cycle

Reflect & improve your Product, Plan, People and Process

Simple rules:Whole team works together & takes responsibility for the outcome

Progress and quality must be kept visible

Finished work (working solu%on) is the only measure of progress

Agile Origins

SoLware Engineering?

Crea%ng SoLware is a CraL.

Conver%ng source code to executable is the engineering bit.

“Software Engineering is the application ofa systematic, disciplined, quantifiableapproach to development, operation andmaintenance of software: that is, theapplication of engineering to software.”

IEEE Standard Computer Dictionary,

ISBN 1-55937-079-3, 1990

IEEE defines SoLware Engineering as...

Who used SoLware Engineering?

For the space shuttle’s operating system37Saturday 1 September 2012

Some Sta%s%csNASA’s Defect Density

The last 11 versions of the space shu)le’s 420,000 line systems had a total of 17

defects.

The last 11 versions of the space shu)le’s 420,000 line systems had a total of 17

defects.

One More Data Point

Another real soLware engineering project

Safeguard - Ballistic Missile Defense System

1969-‐1975, 5407 person years

Hardware designed at the same Ame as soBware specs being wriDen

Late changes in requirements not an opAon

reqmts20 %

design20 %

code &unit test

integrationtesting42 %

1969-‐1975, 5407 person years

Hardware designed at the same Ame as soBware specs being wriDen

Late changes in requirements not an opAon

reqmts20 %

design20 %

code &unit test

integrationtesting42 %

Did it Succeed?

Safeguard Ballis%c Missile Defense System…

Revised Project Statistics

Revised Project StatisticsThe project was delivered according to specifica%ons

Cost: $25 Billion (not adjusted)

1969-‐1975, 5407 person years

Operational for 133 days - Project terminated in 1978

1969-‐1975, 5407 person years

‘By the time the 6-year anti-missile system project was completed, the new missiles were faster than the anti-

missile missiles’

Operational for 133 days - Project terminated in 1978

Where do things go wrong?

Requirements are stable

Technology is well known and mature

Everything goes as expected/planned

We’ve a great deal of expertise having

done the same thing many times before

Heavy weight methods work well when the previous

points are valid

Projects with those characteristics are few

and far between.

Heavy Weight Methodologies

Heavy weight methodologies work in some instances, but there are high costs, and the risk in using them in dynamic environments is high.

Heavy Weight Methodologies

The Business Case for Agile Development

SucceededFailedChallenged

Chaos Report 2006. Standish Group

We need to do be)er than this ….

IT Projects

Project Overruns….

Always7%

OLen13%

Some%mes16%

Rarely19%

Never45%

Standish Group study reported at XP2002 by Jim Johnson, Chairman

O@en or Always Used: 20%

Rarely or NeverUsed: 64%

Feature Use

How significant is requirements change on a project? “The average project has 30% requirements change”

Can We Predict What We Need ?

Why Agile?

Albert Einstein55Saturday 1 September 2012

Albert Einstein

A perfection of means, and confusion of aims, seems to be

our main problem.

Process is a placebo

Jared spool’s tricks to Dogma conAnuum arranges terminology from improvisaAon to atrophy

Process is a placebo

Process is built on values and principles and tailored to fit its

context

Src: Jeff Patton57Saturday 1 September 2012

Traditional cost profile

Lower cost of change curve

Agile system cost profile

Traditional cost profile

Lower cost of change curve

“I’m glad we’re all agreed then.”

Clear communica%on is the founda%on

“Ah...”

Get mental models out on the table

“Ah!”

Convergence through itera%on

“I’m glad we’re all agreed then.”

A genuinely shared understanding

Tradi%onal soLware development fixes scope then es%mates to figure out %me and cost

Traditional software

development

Time Cost(resources)

development

Agile development fixes %me and cost, then leverages itera%on and incremen%ng to maximize scope

development

Agile software development

development

TimeCost

(resources)

development

TimeCost

(resources)

Leverage a shared understanding of desired product goals to minimize scope while maximizing value

development

TimeCost

(resources)

Leverage a shared understanding of desired product goals to minimize scope while maximizing value

development

TimeCost

(resources)

Target business goals & outcomesSrc: Jeff Patton

Building Quality into the Process

Toyoda Loom

Source: Beyond Agile Software Development Becoming Lean, Mary Poppendieck, Poppendieck.llc

Utilization (%)

Focus on Throughput

Tradi%onal Process

Applying Lean Principles to SoLware Development

End-to-End small slices of work

End-to-End small slices of work 20 % done = 100 % usable

Fix / Integrate $

DesignSpecifications

Use Cases / Functional Specs

Requirements Gathering

Project Plan/Estimation

Inception

Lean Principles applied to SoLware Development

Itera%ve

Incremental

Itera%ve AND Incremental

• Mix the strategies:–Iterate to find and improve soluAons

–Increment to add funcAonality

AgileBirth of a new Software Movement!

Agile has evolved over many years

Src: Jeff Patton

FDD | Feature Driven Development (Jeff DeLuca)

DSDM | Dynamic System Development Method (Dane Faulkner)

Adaptive Software Development (Jim Highsmith)

Crystal (Alistair Cockburn)

SCRUM (Ken Schwaber)

XP | Extreme Programming (Kent Beck)

Lean Software Development (Mary Poppendieck)

Pragmatic

Crystal Lean

Adaptive

Agile Umbrella

Agile Manifesto

“We are uncovering better ways of developing software by doing it and helping others do it. Through this work we have come to value:

Agile Manifesto

– Individuals and interactions OVER processes and tools.

Agile Manifesto

– Individuals and interactions OVER processes and tools. – Working software OVER comprehensive documentation.

Agile Manifesto

– Individuals and interactions OVER processes and tools. – Working software OVER comprehensive documentation. – Customer collaboration OVER contract negotiation.

Agile Manifesto

– Individuals and interactions OVER processes and tools. – Working software OVER comprehensive documentation. – Customer collaboration OVER contract negotiation. – Responding to change OVER following a plan.

Agile Manifesto

– Individuals and interactions OVER processes and tools. – Working software OVER comprehensive documentation. – Customer collaboration OVER contract negotiation. – Responding to change OVER following a plan.

That is, while there is value in the items on the right, we value the items on the left more.”

Agile Manifesto Principles

Our highest priority is to satisfy the customer through early and

continuous delivery of valuable software.

Welcome changing requirements, even late in development. Agile processes

harness change for the customer's competitive advantage.

Deliver working software frequently, from a couple of

weeks to a couple of months, with a preference to the shorter

timescale.

Business people and developers must work together daily

throughout the project.

Build projects around motivated

individuals. Give them the environment

and support they need, and trust them to get the job done.

The most efficient and effective method of conveying information to and within a development team is face-to-face conversation.

Working software is the primary measure of progress.

Agile processes promote sustainable development. The

sponsors, developers, and users should be able to maintain a constant pace

indefinitely.

Simplicitythe art of maximizing the amount of

work not doneis essential.

Continuous attention to technical excellence and good design

enhances agility.

The best architectures, requirements, and designs emerge

from self-organizing teams.

At regular intervals, the team reflects on how to become more effective, then tunes and adjusts

its behavior accordingly.

It turns out...

Ziv's law -‐ specifica%ons will never be fully understood.

It turns out...

Humphrey's law -‐ the user will never know what they want un%l aLer the system is in produc%on (maybe not even then)

It turns out...

Wegner's lemma -‐ an interac%ve system can never be fully specified nor can it ever be fully tested.

It turns out...

Langdon's lemma -‐ soLware evolves more rapidly as it approaches chao%c regions (taking care not to spill over into chaos)

It turns out...

Langdon's lemma -‐ soLware evolves more rapidly as it approaches chao%c regions (taking care not to spill over into chaos)

Any association of predictive or defined processes with Agile is an exercise in futility. - Jeff

Treat agile principles as “proper%es” you use to assess process health

Performing a simple process health checkup: h)p://www.s%ckyminds.com/s.asp?F=S15474_COL_2

1. Frequent delivery

2. ReflecAve improvement

3. Close communicaAon

4. Focus

5. Personal safety

4. Focus

5. Personal safety

6. Easy access to experts

4. Focus

5. Personal safety

7. Strong technical environment

4. Focus

5. Personal safety

8. Sunny day visibility

4. Focus

5. Personal safety

9. Regular cadence

4. Focus

5. Personal safety

9. Regular cadence

10.High energy

4. Focus

5. Personal safety

9. Regular cadence

10.High energy

11.Empowered teams

4. Focus

5. Personal safety

9. Regular cadence

10.High energy

11.Empowered teams

12.DisrupAve change

Our Team Rooms

Our plans looks like this

Source : ThoughtWorks

some more plans…

src: ThoughtWorks India98Saturday 1 September 2012

src: ThoughtWorks India

Work or Fun or Both?

src: ThoughtWorks India

Work or Fun or Both?

Agile Evolution

Pragmatic

Crystal Lean

Adaptive

Agile Umbrella

Agile become...

Balance discovery with delivery

Delivery: building product right

Discovery: understanding the right product to

Then came along...

LeanAgile-UX

ProductDiscovery

Agile Ecosystem

High Level View of an Agile Process

Then came along...

Agile-UX

ProductDiscovery

Agile Ecosystem

ScrumLean

Kanban

Where did Agile Originate?

Problem

Known Unknown

Src: Eric Ries

Where Agile appears to work best?

Problem

Known Unknown

Src: Eric Ries

Problem

Known Unknown

l e ??

Src: Eric Ries

Kaizen vs. Kaikaku

Currently...

LeanAgile-UX

ProductDiscovery

Dev-OPs

LeanStartup

AgileEcosystem

KanbanScrum

The Future

XPAgile

Agile-UX

ProductDiscovery

Dev-OPs

Lean Startup

Costumer Development

ContinuousDelivery

KanbanScrum Lean

Organizations have habits, and they will stick to their habits even at the risk of their own

survival.Brad Anderson, CEO, Best Buy

Organizational structures have a short life... Nobody likes to reorganize, and you

always run the risk that you distract your employees and lose focus on customers.

But if you don't do it, you lose your competitive edge.

Nancy McKinstry, CEO, Wolters Kluwer

Innovation

Metrics Mess

Metrics MessKnowledge Islands

Be careful not to…

twitter: @nashjain

Ques%ons?123Saturday 1 September 2012

twitter: @nashjain

Ques%ons?123Saturday 1 September 2012

agile way of dealing with uncertainty in a complex adaptive world

number of balls

low volts shock group

random number

ball point gameyour

ball point gamesaturday

pass balls

core agile concepts

number youve

Technology

business uncertainty agile india

dealing with uncertainty: from agile to antifragile

dealing with estimation, uncertainty, risk, and commitment

dealing with climatic uncertainty - remediation … ·...

dealing with medical uncertainty

cs 326a: motion planning kynodynamic planning + dealing with...

dealing with uncertainty

robust workforce planning: dealing with uncertainty

dealing with uncertainty - our business world

dealing with uncertainty: concepts and tools · dealing...

agile project outsourcing - dealing with rfp and rfi

dealing with auditors: helping them understand agile

dealing with uncertainty: implications for fisheries ... ·...

tools for dealing with uncertainty, ambiguity, and...

corporate finance lecture 5. topics covered decision trees...

dealing with valuation uncertainty - property funds ·...

dealing with uncertainty in emerging assessments

genetic counseling: dealing with uncertainty · genetic...

the big five for founding - dealing with uncertainty

chapter 3 dealing with demand uncertainty us- ing … ·...