Download - Deferring the Last Responsible Moment
Deferring the Last Responsible Moment
Eoin Woods, [email protected]
@eoinwoodzwww.eoinwoods.info
Join the conversation on Twitter:
@SoftArchConf #SoftwareArchitect2015
20151015.3
2
2
Introductions
Eoin Woods
• CTO at Endava
• Career has spanned products and applications• Architecture and software engineering• Bull, Sybase, InterTrust• BGI (Barclays) and UBS• Endava
• Constantly fighting tendency to make decisions too early!
3
3
Content
Introducing the Last Responsible Moment
Introducing Real Options
Balancing Early and Late Decision Making
Tactics to Allow Decisions to be Delayed
Summary
5
5
Roots in Lean Thinking
Lean working systematically avoids waste
• Initially applied to manufacturing
• Honda and Toyota applied it toproduct development
• In the 1990s “Lean Construction Institute” formed
• Applied to software from early 2000s• Mary and Tom Poppendieck
• Fits well with many of Agile’s principles
6
6
Key Principles for Lean Working
Key principles
1. Eliminate waste
2. Amplify learning
3. Decide as late as possible
4. Deliver as fast as possible
5. Empower the team
6. Build quality in
7. See the whole
7
7
The Last Responsible Moment
A term coined by Lean Construction authors in ~2000
• LCI’s white paper series by Howell, Ballard & Zabell
A single conceptual design will normally be selected before the end of [the lean design] phase because the last responsible moment for making that decision will have usually passed. Design decisions will be deferred until the last responsible moment if doing so offers an opportunity to increase customer value.
8
8
The Last Responsible Moment
Kenneth Rubin’s definition for software development
• Essential Scrum, 2012
A strategy of not making a premature decision but instead delaying commitment and keeping important and irreversible decisions open until the cost of not making a decision becomes greater than the cost of making a decision.
9
9
Last Responsible Moment - Opinions
… it isn’t good advice, in fact it isn’t even a meaningful phrase; what is worse, if it were meaningful, it isn’t actionable; and if it were actionable, then it wouldn’t be good advice-- Alistair Cockburnhttp://alistair.cockburn.us/Last+Responsible+Moment+reconsidered
The problem with “LRM” is that when asking someone to defer a commitment, asking them to simply “defer the commitment” to the “Last responsible moment” leaves them with a lot of uncertainty and very little control.-- Chris Matts
Its not necessary to be able to quantify the costs and benefits very accurately because any evaluation is going to be better than none!-- Carl Scotland
http://wirfs-brock.com/blog/2011/01/18/agile-architecture-myths-2-architecture-decisions-should-be-made-at-the-last-responsible-moment/
I’m not a good enough of a designer (or maybe I am too much of an optimist) to know when the last responsible moment is. Just having a last-responsible moment mindset leaves me open to making late decisions. I’m sure this is not what Mary and Tom intended at all.-- Rebecca Wirfs-Brock
10
10
Difficulties with the Last Responsible Moment
The problem is … simple to say, difficult to use
• Actually spotting “the moment” is difficult
• Very easy to realise it passed some time ago
• How do you know when the cost of deferring is greater than the cost of committing?
• Usually a period rather than a “moment”
For example, you want to decide which JS UI framework to use … when is the last moment for this decision?
12
12
Real Options
An extended model for LRM is “Real Options”
• Based on the idea of a financial option
• Defined by Chris Matts and Olav Maassen in ~2007
A real option – “never commit without knowing why”
• An option but not an obligation for a future decision
• Has a “value”
• Has an “expiry” (date or other condition)
• Has a “cost to buy” and a “cost to exercise”
Introduction from Matts and Maassen: http://www.infoq.com/articles/real-options-enhance-agility
13
13
Defining a Real Option
The constituent of a Real Option are:
• Value – what benefit will exercising this option have for the organisation? Could be time, money, opportunity, …
• Expiry – what conditions mean that the option can no longer be exercised? A project date or when something happens (e.g. remaining budget falls to a point)
• Cost to Buy – what do you need to do in order to have the option? PoCs? Research? Analysis? Design change?
• Cost to Exercise – once you commit to this option, what is the cost to achieve it? Development work? Changes in Operations? New testing? Additional analysis? Licenses?
14
14
An Example of a Real Option
Suppose we need an option on a database …
The Optionthan aTo use a document oriented database (rather
relational database)
The Value Reduced development cost (~15%) if database model matches problem domain
The Expiry Time of first production release less time taken to replace DB access layer & deploy chosen database
Purchase Cost A DB access layer (mocked) plus research to prove that both dbs could be deployed
Exercise Cost dbWrite DB access layer, design operational environment, deploy db
15
15
Example of a Real Option (ii)
Time
Cost/Value
R0
DB Deploy Time
Data AccessLayer Build Time
Value
Cost
Expiry
Benefit
T0
16
16
Using Real Options
To create and manage a Real Option
1. Identify your options for a decision
2. Identify the conditions defining the last decision point• e.g. decision point = deadline - implementation time• e.g. decision point = resource0 < Value1
3. Keep learning and looking for options until decision point
4. Move back decision point where possible
5. When decision point arrives, act as quickly as possible• Real Options ≠ procrastination!
(More on moving back the decision point in the
next section.)
17
17
How Real Options Help with Decision Making
The value of Real Options
• Real Options overcome natural tendency to decide early• Makes a “no decision” position clear• Requires precision about when a decision is needed• Overcomes aversion to uncertainty
• Real Options encourage thinking about alternatives
• Real Options force clarity in the options available
• Real Options encourage gathering information
Combination of factors create likelihood of better decisions
19
19
Balancing Late vs Early Decisions
Not every decision can (or should) be delayed
• Like any idea deferring decisions can be over used
• Little benefit in deferring decisions with low impact• Decisions that can be changed at low cost• Decisions that have little tangible impact on the outcome• Decisions that aren’t going to affect customer value
• Making no decisions early will just stall progress
• Many people feel reassured when a decision is made• “any decision is better than no decision”• Real Options helps decision making to be more transparent
• Lack of a decision may block others from progressing
20
Early Decisions• Harder to validate due to
less information• Have a tendency to
optimise unwisely• Have a tendency to be
wasteful (YAGNI)• Often look decisive• Can be (falsely?)
reassuring
Late Decisions• Maximise information
available for decision• Avoid premature
optimisation problems• Can block others from
progressing• Can overwhelm decision
making if too many• Need communicated• Can cause worry about
lack of direction
Late vs Early Decisions
Remember need for communication, clarity and reasoning when making late decisions to avoid looking indecisive or unsettling people – psychological factors
23
23
Early Decision – NATS ATC Software
http://www.computerweekly.com/feature/A-brief-history-of-an-air-traffic-control-system
http://nats.aero
25
25
Tactics to Defer Decisions
(Re)Organise for Change
• Run projects that can absorb change as late as possible
Architect for Change
• Design systems that can be changed regularly and reliably
Design for Change
• Implement software that limits the impact of change
26
26
(Re)Organise for Change
Organise project to absorb change and allow late decisions
• Domain Analysis
• Know the domain to identify the options for late decisions
• Early Sharing of Information
• Don’t wait for perfection before validating
• Minimum Viable Product
• Build the smallest thing possible to gain more knowledge
• Set Based Design
• Back enough options to allow late selection of the solution
27
Benefit• Good domain knowledge
identifies alternatives• Domain analysis leads to
new options• Collaboration across
business and developers
Cost• Significant time and
effort to develop• Requires specialisation
Domain Analysis
www.uscis.gov
28
Benefit• Allows collaboration to
identify options• Early feedback to spot
problems early• Clarity on direction
Cost• Time for communication
Share Information Early
29
29
Minimum Viable Product
Don’t build anything that isn’t essential
• A MVP is the product level version of “do the simplest thing that could possibly work”
• The simplest set of features that could meet the requirement
• Allows for rapid delivery, cost effective feedback loop and affordable mistakes to be made, so facilitates learning
• The key is to decide what the minimum really is
30
Benefit• Smallest number of
decisions made each time
• Options stay open until feature needed
Cost• Effort to fit approach
into many organisations• Difficult to “sell” without
a defined end-state
Minimum Viable Product
cloudmanic.com
31
31
Set Based Design
Why choose before the end?
• Set based design develops a set of solutions for each problem where the best solution isn’t clear at the start
• e.g. a “safe” solution, a “likely” solution & an “innovative” solution
• As time progresses stop work on solutions that don’t work well or where their LRM has passed
• Overall project assembled from “winning” solutions
32
Benefit• Hedges options right
through project• Latest decision possible
Cost• Duplicated effort• Integration difficult if
multiple choices• Can be confusing
without careful management
Set Based Design (ii)
33
33
Architect for Change
Design a system that can be changed regularly, reliably
• Separate Concerns• Limit items to be changed
• Avoid Duplication (DRY)
• Limit impact of change
• Dependency Management• Know the impact of change
• Variation Points• Manage a set of options through the lifecycle
• Testing and Continuous Delivery• Enable reliable and rapid change
34
Benefit• Avoid duplication• Change isolation• Simpler components• Adds options for reuse &
extension
Cost• Design effort• Refactoring
Separate Concerns (Cohesion)
35
Benefit• Avoids wasted effort• Easier change• Efficiency
Cost• Effort• Refactoring impact
Avoid Duplication (“Don’t Repeat Yourself”)
deviq.com/don-t-repeat-yourself
36
Benefit• Find change blackspots• Avoid high coupling• Structural insight
Cost• Time & prioritisation• Refactoring• Tools
Dependency Management (Coupling)
37
37
Variation Points
Variation Points build options into design thinking
• An idea from product line engineering
• Design software as a set of assets designed for use in different situations
• Explicit variation points in the (functional) design
• Makes existence, constraints and cost of options clear
• Promotes variability to be a feature of the system
• Document & test the variation points for product owner
38
38
Variation Points (ii)
Example in a payment processing system
• Identify explicit options for variation of:• transaction authorisation mechanism• transaction message transport• security mechanisms
• Product owner can now decide how and when to invest in these options
• Variation points are enablers of a set of Real Options
• Key is explicit design and management of the variation point set
39
Benefit• First class options in the
design• Excellent visibility of
options for PO
Cost• Up front design effort• Implementation effort• Requires a little
clairvoyance (i.e. domain knowledge)
Variation Points (iii)
40
Benefit• Confident change• Reliable change• Rapid delivery of
changes to production
Cost• Implementation cost• Initial effort• Customer commitment
Test Automation and Continuous Delivery
derberg.github.io
41
41
Design for Change
Design a system that can be changed regularly, reliably
• Interfaces and Abstractions
• Encourage substitutability
• Parameterisation
• Create logic that expects to delegate responsibility for details
• Encapsulate Variation
• Limit the impact of changing a decision
42
Benefit• Delay implementation
decision point• Substitutability (LSP)
allows future options
Cost• Complexity• Design effort• Mocking effort• LCD risks
Oracle Adapter
MSSQL Adapter
«interface»Database Adapter
Interfaces and Abstractions
43
Benefit• Delays implementation
decision• Allows build and test of
parameterised modules• Allows decision to be
changed repeatedly
Cost• Complexity• Design effort
Parameterisation
setScoreCalculator(ScoreCalculator sc)
Credit Score Reporting
Customer List
Rule Based Calculator
Model Based Calculator
OneR CalculatorBayesian
Calculator
44
Benefit• Localised change impact• Predictable and reliable
change• Avoiding duplication
Cost• Vigilance• Refactoring as
knowledge improves• Structural complexity (?)
Modularise and Encapsulate Variation
Payroll Manager
«interface»Hourly Rate Calculator
Hourly Paid Hourly Rate Calculator
Salary Based Hourly Rate Calculator
46
46
Delaying the Last Responsible Moment
An enterprise system for compliance reporting
• Receives all the business transactions for the company
• Analyses, sorts, classifies the business activity
• Contains a configurable rule set for report generation
• Sends real time report feeds to some regulators
• Generates document reports for other regulators
• Provides controls such as “4 eyes” checks on dispatch
• Needed “immediately” due to regulatory inspection
47
47
Regulatory Reporting System
Project organisation to
allow late change?
Architecture for delaying change?
Design to defer the last responsible
moment?
Regulatory Reporting System
Reporting Analysts
Supervisors
Business Transactions
Regulator
Activity Reports
48
48
Regulatory Reporting System
Project organisation ideas:
• What is a minimum viable reporting system?
• What will the regulator put up with? (Domain knowledge)
• Which regulator first? (Domain knowledge)
• Develop a simple file generator with manual processes alongside a fully automated system!
49
49
Regulatory Reporting System
Architecture ideas:
• What variation points will we need?• Report type (data included, data items, ….)• Regulator specific calculations for report items (e.g. NPV)• Transport for dispatch of reports
• Continuous delivery to improve system every couple of days – allows better negotiation with the regulators
• Automated testing of reports to avoid manual test cycle delays
• Stress cohesion and low coupling to reuse and replace pieces of the process as the real requirements emerge
50
50
Regulatory Reporting System
Design ideas:
• Encapsulate all of the variation points we identified to allow rapid safe extension
• Parameterise the report generation process to allow regulator specific calculations to be “injected”
52
52
Summary
Decisions need information so making them late helps
• The last responsible moment is a difficult concept• Hard to measure and use
• Real Options add some important concepts to the idea• Value, cost, expiry
• It is often possible to move the “expiry moment”• Organise for Change• Architect for Change• Design for Change
53
53
Summary (ii)
Moving the moment an option expires
• Organise for Change
• Domain Analysis, Share information early,
MVP, Set-based design
• Architect for Change
• Separate Concerns, Avoid duplication, Manage dependencies,
Variation points, Testing and continuous delivery
• Design for Change
• Interfaces and abstractions, Parameterisation,
Encapsulate variation
The tactics aren’t novel but using them to delay commitment may be
54
54
Resources
http://www.infoq.com/articles/real-options-enhance-agility
http://www.agilecoach.net
55
Thank you
QUALITY. PRODUCTIVITY. INNOVATION.
Eoin WoodsEndava
+44 207 367 1000
en_ewoods