uw agile cp202 class 3 managing software debt
DESCRIPTION
TRANSCRIPT
UW Agile CP202 Class 3:Managing Software DebtContinued Delivery of High Value as Systems Age
Chris SterlingTechnology Consultant / Agile Coach /
Certified Scrum TrainerSterling Barton, LLC
Web: www.SterlingBarton.comEmail: [email protected]
Blog: www.GettingAgile.com Follow Me on Twitter: @csterwa
Hash Tag for Presentation: #swdebt
1Monday, May 10, 2010
Topics Being Covered
Problems Found with Aging Software
Software Debt Explained
Technical Debt
Quality Debt
Configuration Management Debt
Design Debt
Platform Experience Debt
The Wrap Up
A Story of What is Possible
2Monday, May 10, 2010
Problems Found with Aging Software
Software gets difficult to add features to as it ages
Business expectations do not lessen as software ages
Software must remain maintainable and changeable to meet needs of business over time
3Monday, May 10, 2010
Lack of emphasis on software quality attributes contributes to
decay
4Monday, May 10, 2010
Like-to-Like Migrations
“It will be easy since we worked on the original version” - although we understand the domain we will be fighting with new features, technology, tools, and processes
“We don’t have any other options” - Refactoring and test automation are potential alternatives to like-to-like migrations.
5Monday, May 10, 2010
Limited Platform Expertise
Risk and costs increase as expertise becomes more limited for aging software platforms.
6Monday, May 10, 2010
Costs for Release Stabilization
0
125000
250000
375000
500000
Release 1Release 2
Release 3Release 4
Release 5Release 6
Cost of Fixing Defects Cost for Feature Dev
7Monday, May 10, 2010
Regression Costs - Manual vs.
8Monday, May 10, 2010
Extreme Specialization
Knowledge and capability to maintain legacy software decays with time
Costs to maintain rarely used software platforms are higher
Leads to waiting for people in specialized roles to finish their tasks in support of development effort
9Monday, May 10, 2010
Software DebtCreeps into software slowly and leaves organizations with liabilities
10Monday, May 10, 2010
Software Debt Creeps In
Shows a relatively new system with little software debt accrued.
11Monday, May 10, 2010
Software Debt Creeps In
An aging software system slowly incurs significant debt in multiple functional areas.
12Monday, May 10, 2010
Software Debt Creeps In
An older system has accrued significant debt in all functional areas and components.
13Monday, May 10, 2010
Managing Software Debt – an
14Monday, May 10, 2010
Managing Software Debt
It is impossible to stop software debt from creeping into our software
Managing debt in software is based on putting frequent feedback mechanisms in place for code, quality assurance, configuration management, design, and organization of people on project team
Feedback mechanisms should be frequent, automated, and easy to use to support their continued use or modified to new needs
15Monday, May 10, 2010
Types of Software Debt
Technical Debt
Quality Debt
Configuration Management Debt
Design Debt
Platform Experience Debt
16Monday, May 10, 2010
Technical DebtIssues in software implementation that will impede future development
if left unresolved
17Monday, May 10, 2010
* Ward Cunningham on “Technical Debt”
Technical Debt includes those internal things that you choose not to do now, but which will impede future development if left undone. This includes deferred refactoring.
Technical Debt doesn’t include deferred functionality, except possibly in edge cases where delivered functionality is “good enough” for the customer, but doesn’t satisfy some standard (e.g., a UI element that isn’t fully compliant with some UI standard).
* Ward Cunningham - “Technical Debt” - http://c2.com/cgi/wiki?TechnicalDebt
18Monday, May 10, 2010
My Definition of “Technical Debt”
“Technical debt is the decay of component and inter-component behavior when the application
functionality meets a minimum standard of satisfaction for the customer.”
19Monday, May 10, 2010
Test-Driven Development
Test-Driven Development (TDD)
The “Flow”
Integrating with Team
Difficulties with TDD
Example TDD Session
Refactoring
20Monday, May 10, 2010
Questions for You...
How many of you are practicing TDD?
What issues are you having currently with TDD?
21Monday, May 10, 2010
TDD - Basic “Flow”
22Monday, May 10, 2010
TDD - Basic “Flow”
Write Failing Test
22Monday, May 10, 2010
TDD - Basic “Flow”
Write Failing Test
Make Test Pass
22Monday, May 10, 2010
TDD - Basic “Flow”
Write Failing Test
Make Test Pass
Refactor to Acceptable Design
22Monday, May 10, 2010
TDD - Basic “Flow”
Write Failing Test
Make Test Pass
Refactor to Acceptable Design
22Monday, May 10, 2010
Steps of TDD “Flow”
1. Write a programmer test
2. Run the programmer test and it should fail (red bar)
3. Write just enough code to make failing test pass
4. Run programmer test successfully (green bar)
5. Refactor code to an acceptable design (green bar)
RISK: Pushing out refactoring to add more code first. Don’t forget to refactor frequently to an acceptable design.
23Monday, May 10, 2010
Be a User --“What should the software do next
This question helps you to decide what the next programmer test should model
24Monday, May 10, 2010
Integrating with Team
25Monday, May 10, 2010
Integrating with Team
Write Failing Test
25Monday, May 10, 2010
Integrating with Team
Write Failing Test
Make Test Pass
25Monday, May 10, 2010
Integrating with Team
Write Failing Test
Make Test Pass
Integrate with Team
25Monday, May 10, 2010
Integrating with Team
Write Failing Test
Make Test Pass
Refactor to Acceptable Design
Integrate with Team
25Monday, May 10, 2010
Integrating with Team
Write Failing Test
Make Test Pass
Refactor to Acceptable Design
Integrate with Team
25Monday, May 10, 2010
Integrating with Team
Write Failing Test
Make Test Pass
Refactor to Acceptable Design
Integrate with Team
25Monday, May 10, 2010
Need-Driven Design
26Monday, May 10, 2010
Test-Driven Development
What Causes TDD to be so Difficult to Implement Well?
27Monday, May 10, 2010
Impediments to TDD
This simple technique for developing software, when used in a disciplined manner, will enable individuals and teams to improve software quality. The discipline necessary to do TDD is not easily attainable. Following is a list of environmental issues that lowers chances of effectively implementing TDD approach.
28Monday, May 10, 2010
Schedule Pressure
Pressure from management and stakeholders to release based on an unreasonable plan. Integrity of the software is always sacrificed when the plan is inflexible and unwilling to incorporate reality.
29Monday, May 10, 2010
Lack of Passion
Lack of passion for learning and implementing TDD effectively on Team. The amount of discipline required makes passion extremely helpful.
30Monday, May 10, 2010
Insufficient Experience
Insufficient team member ratio who have experience doing TDD well. When there is little experience on or coaching for the team success is minimal.
31Monday, May 10, 2010
Legacy Code
If the software’s design is poor or is difficult to test then finding a starting point could seem impossible. If legacy code is large and contains no, or only minimal, test coverage then disciplined TDD will not show visible results for some time.
32Monday, May 10, 2010
Introducing TDD Approach
To successfully adopt TDD, it is important to manage these environmental issues. This could include managing expectations, providing the Team support from a coach, and enabling sufficient learning of tools and techniques while working with an existing code base.
33Monday, May 10, 2010
Test-Driven Development
Quick Example
34Monday, May 10, 2010
Jitter – Example TDD SessionFake micro-blogging tool named “Jitter” is made by Seattle-based fictitious company that focuses on enabling coffee injected folks to write short messages and have common online messaging shorthand expanded for easy reading. The user story we are working on is:
So it is easier to read their kid’s messages, Mothers want to automatically expand common shorthand notation
The acceptance criteria for this user story are:
LOL, AFAIK, and TTYL are expandable
Expand lower and upper case versions of shorthand
35Monday, May 10, 2010
Expand LOL to “laughing out loud”
public class WhenUsersWantToExpandMessagesThatContainShorthandTest {
@Test
public void shouldExpandLOLToLaughingOutLoud() {
JitterSession session = mock(JitterSession.class);
when(session.getNextMessage()).thenReturn("Expand LOL please");
MessageExpander expander = new MessageExpander(session);
assertThat(expander.getNextMessage(),
equalTo("Expand laughing out loud please"));
}
}
public class MessageExpander {
public String getNextMessage() {
String msg = session.getNextMessage();
return msg.replaceAll("LOL", "laughing out loud");
}
36Monday, May 10, 2010
But wait…what if…?
What if LOL is written in lower case?
What if it is written as “Lol”? Should it be expanded?
What if some variation of LOL is inside a word?
What if characters surrounding LOL are symbols, not letters?
Write these down as upcoming programmer tests as comments so I don’t forget them.
// shouldExpandLOLIfLowerCase
// shouldNotExpandLOLIfMixedCase
// shouldNotExpandLOLIfInsideWord
// shouldExpandIfSurroundingCharactersAreNotLetters
37Monday, May 10, 2010
Expand LOL If Lower Case
@Test
public void shouldExpandLOLIfLowerCase() {
when(session.getNextMessage()).thenReturn("Expand lol please");
MessageExpander expander = new MessageExpander(session);
assertThat(expander.getNextMessage(),
equalTo("Expand laughing out loud please"));
}
This forced use of java.util.regex.Pattern to handle case insensitivity.
public String getNextMessage() {
String msg = session.getNextMessage();
return Pattern.compile("LOL”, Pattern.CASE_INSENSITIVE)
.matcher(msg).replaceAll("laughing out loud");
}
38Monday, May 10, 2010
Don’t Expand “Lol” – Mixed-Case
@Test
public void shouldNotExpandLOLIfMixedCase() {
String msg = "Do not expand Lol please";
when(session.getNextMessage()).thenReturn(msg);
MessageExpander expander = new MessageExpander(session);
assertThat(expander.getNextMessage(), equalTo(msg));
}
This forced me to stop using Pattern.CASE_INSENSITIVE flag in pattern compilation. Only use “LOL” or “lol” for replacement criteria.
public String getNextMessage() {
String msg = session.getNextMessage();
return Pattern.compile("LOL|lol").matcher(msg)
.replaceAll("laughing out loud");
}
39Monday, May 10, 2010
Don’t Expand “LOL” If Inside Word
@Test
public void shouldNotExpandLOLIfInsideWord() {
String msg = "Do not expand PLOL or LOLP or PLOLP please";
when(session.getNextMessage()).thenReturn(msg);
MessageExpander expander = new MessageExpander(session);
assertThat(expander.getNextMessage(), equalTo(msg));
}
The pattern matching is now modified to use spaces around each variation of valid LOL shorthand.
return Pattern.compile("\\sLOL\\s|\\slol\\s").matcher(msg)
.replaceAll("laughing out loud");
40Monday, May 10, 2010
Expand “LOL” If Not Inside Word
@Test
public void shouldExpandIfSurroundingCharactersAreNotLetters() {
when(session.getNextMessage()).thenReturn("Expand .lol! please");
MessageExpander expander = new MessageExpander(session);
assertThat(expander.getNextMessage(),
equalTo("Expand .laughing out loud! please"));
}
The final implementation of pattern matching code looks as follows.
return Pattern.compile("\\bLOL\\b|\\blol\\b").matcher(msg)
.replaceAll("laughing out loud");
41Monday, May 10, 2010
Merciless Refactoring
Refactoring: a disciplined technique for restructuring an existing body of code, altering its internal structure without changing its external behavior.*
Merciless: having or showing no [mercy - showing great kindness toward the distressed]
Relieve your distressed code through kindness and disciplined restructuring
* From http://www.refactoring.com/
42Monday, May 10, 2010
Where to Start Refactoring?Does this change directly affect feature I am working on?
Would change add clarity for feature implementation?
Will change add automated tests where there are none?
If “yes” to any question above, ask following question to decide if you should work on it now:
At first glance, does refactoring look like a large endeavor involving significant portions of the software’s components?
43Monday, May 10, 2010
When to Stop Refactoring?
Am I refactoring code not directly affected by feature?
Is other code directly affected by feature I am working on that has not been refactoring sufficiently?
If refactoring is exploding feature estimate given to Customer then I should bring it up to Team to decide how we should progress
If Team decides that refactoring can be absorbed into current iteration without affecting delivery on our commitments then continue refactor
44Monday, May 10, 2010
Automate Testing to Support
Refactoring cannot be done effectively without automated tests surrounding code
Start by creating automated test which fails
If difficult to create at unit level look at automated acceptance tests from functional perspective
Over time look for ways to create automated unit tests
45Monday, May 10, 2010
Automated Tests
Application
46Monday, May 10, 2010
Automated Tests
New Feature
Application
46Monday, May 10, 2010
Automated Tests
New Feature
Application
Design Changes
46Monday, May 10, 2010
Automated Tests
New Feature
Application
Design ChangesAutomatedRegressionTest Run
46Monday, May 10, 2010
Automated Tests
New Feature
Application
Design ChangesAutomatedRegressionTest Run
46Monday, May 10, 2010
Principles of Agile Software Quality
The system always runs
No code is written without a failing test
Zero post-iteration bugs
* From “Flawless Iterations” presented by Alex Pukinskis at Agile 2005
47Monday, May 10, 2010
Quality DebtA lack of quality, either technical or functional, will lessen value per
feature added over time
48Monday, May 10, 2010
Accrual of Quality Debt with Releases
49Monday, May 10, 2010
Break/Fix Only Prolongs the Agony
50Monday, May 10, 2010
Effect of Project Constraints on
51Monday, May 10, 2010
Effect of Project Constraints on
51Monday, May 10, 2010
Acceptance Test-Driven Development
52Monday, May 10, 2010
A Fit Case StudyCost reduction using Fit for test automation and data conversion
53Monday, May 10, 2010
Manual Regression Testing
Testing was taking 75 person hours during 2 full test runs consisting of:
Comprehensive manual regression testing
Data conversion and validation
Cost for testing was $17,000 each iteration
54Monday, May 10, 2010
Introducing Fit into Testing Process
After 8 iterations team had introduced healthy amount of Fit fixtures and automated tests
Reduced 70+ hour test runtime down to 6 hours which now included:
Fit automated regression testing
Data conversion and validation automated with Fit fixtures
Reduced cost of testing each iteration from $17,000 to $7,000
55Monday, May 10, 2010
Configuration Management DebtCreating unpredictable and error-prone release management
56Monday, May 10, 2010
Traditional Source Control
57Monday, May 10, 2010
Traditional Source Control
Main Branch
57Monday, May 10, 2010
Traditional Source Control
Main Branch
Version 1Branch
Integrate forVersion 2
CodeComplete
57Monday, May 10, 2010
Traditional Source Control
Main BranchDebt
Death March
Version 1Branch
Integrate forVersion 2
CodeComplete
57Monday, May 10, 2010
Traditional Source Control
Main BranchDebt
Death March {Debt accrues quickly within stabilization periods
Version 1Branch
Integrate forVersion 2
CodeComplete
57Monday, May 10, 2010
Flexible Source Control Management
58Monday, May 10, 2010
Flexible Source Control Management
Main Branch
58Monday, May 10, 2010
Flexible Source Control Management
Main Branch
Version 1
58Monday, May 10, 2010
Flexible Source Control Management
Main Branch
Version 1 Version 2
58Monday, May 10, 2010
Flexible Source Control Management
Main Branch
Version 1 Version 2{Not Easy! Must have proper infrastructure to do this.
58Monday, May 10, 2010
Continuous Integration
59Monday, May 10, 2010
Scaling to Multiple Integrations
60Monday, May 10, 2010
Design DebtDesign decays when not attended to so design your software
continually
61Monday, May 10, 2010
* Abuse User Stories
Implement Securityfor User Information
* From “User Stories Applied” presented by Mike Cohn Agile 2006
62Monday, May 10, 2010
* Abuse User Stories
Implement Securityfor User Information
As a Malicious Hacker I want to steal credit card information so that I can make fraudulent charges
* From “User Stories Applied” presented by Mike Cohn Agile 2006
62Monday, May 10, 2010
Platform Experience DebtSilos of knowledge and increased specialization will increase cost of
maintenance over time
63Monday, May 10, 2010
Team Configuration Patterns
Virtual Architect Pattern
Integration Team Pattern
Component Shepherd Pattern
Team Architect Pattern
64Monday, May 10, 2010
Virtual Architect Pattern
EnterprisePlanning
65Monday, May 10, 2010
Virtual Architect Pattern
Pros
Share architecture ideas and needs across teams
Based on verbal communication
Cons
Usually singles out special Team Member role
Could lead to top-down architecture decisions
IT may gain extensive influence and begin to run Product
66Monday, May 10, 2010
Integration Team Pattern
Feature Development
Integrate Features
All features are implemented
and integrated every iteration
67Monday, May 10, 2010
Integration Team Pattern
Pros
Reduces complexity on Feature Teams
Forces delivery from Integration Team instead of interface and deployment designs
Cons
Perpetuates specialized roles
Don’t always work on highest value Product Backlog items
68Monday, May 10, 2010
Component Shepherd Pattern
69Monday, May 10, 2010
Component Shepherd Pattern
Pros
Share more knowledge within organization to minimize platform experience debt
Work on highest value Product Backlog items
Cons
Not always optimal as using individual knowledge
Difficult to learn multiple systems across Teams
70Monday, May 10, 2010
Team Architect Pattern
71Monday, May 10, 2010
Team Architect Pattern
Pros
Team owns architecture decisions
Decisions are made close to implementation concerns
Cons
May not have appropriate experience on Team
Team could get “stuck” on architecture decisions
72Monday, May 10, 2010
What is possible?High quality can be attained and should enable accelerated feature
delivery at same time.
73Monday, May 10, 2010
A Story: Field Support Application
2000+ users access application each day
Application supports multiple perspectives and workflows from Field Support Operations to Customer Service
Team of 5 people delivering features on existing Cold Fusion platform implementation
Migrating to Spring/Hibernate in slices while delivering valuable features
36 2-week Sprints, 33 production releases, and only 1 defect found in production
So, what was the defect you say? Let me tell you…
74Monday, May 10, 2010
Lets wrap this up...What should I take away from this?
75Monday, May 10, 2010
Principles for Managing Software Debt
Maintain one list of work
Emphasize quality
Evolve tools and infrastructure continually
Improve system design always
Share knowledge across the organization
And most importantly, get the right people to work on your software!
76Monday, May 10, 2010