how automation reveals technical debt
Post on 11-May-2015
1.069 Views
Preview:
TRANSCRIPT
© 2013 IBM Corporation
How Automation Reveals Technical Debt
© 2013 IBM Corporation
Eric’s BioI’m a DevOps Evangelist at UrbanCode where I helps customers get the most out of their build, deploy and release processes.
I have automation experience throughout the application life-cycle in roles as a developer, test automation engineer, and support engineer. For the last 9 years, I’ve been focused on CI, CD and DevOps
Eric Minickeric@urbancode.com@EricMinick
© 2013 IBM Corporation
Technical Debt
© 2013 IBM Corporation
Why do we accumulate technical debt?
We leverage technical debt to deliver more faster.
De-leveraging is rarely accounted for in project planning.
Green-Shifting*
Scope Time
Resources
* http://www.drdobbs.com/191600661
© 2013 IBM Corporation
Why do we care? Paying interest
Now Later Later Still Much Later0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Debt PaidInterest PaidValue Delivered
Debt
© 2013 IBM Corporation
Why do we care? Or paying our debts
Now Later Later Still Much Later0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Debt PaidInterest PaidValue Delivered
Debt
© 2013 IBM Corporation
Pay it now, or pay more later
NowLa
ter
Late
r Still
Muc
h La
ter
Debt Paid
Interest Paid
Value De-livered
NowLa
ter
Late
r Still
Muc
h La
ter
Debt Paid
Interest Paid
Value De-livered
© 2013 IBM Corporation
Why should we care?
Baggage that slows the team
Lack of automated tests lengthen QA cycles
Fear of merging work
Unrefactored code slow to work with
Slow build / deploy processes delay learning and release pace
Quality issues
Lack of tests results in buggier code
Releases are error prone and lead to unnecessary outages
© 2013 IBM Corporation
The limits of what we know
Known Knowns: Bugs confirmed and tracked
Known Unknowns: Undiscovered bugs
Unknown unknowns: One of our project teams is using a GPL’d library making their product impossible to ship
© 2013 IBM Corporation
Where Automation Helps
“Testing” for debt: Automated tests, code scans and reports can help identify (and quantify) problems.
Automation as a learning experience: The act of automating brings surprises.
© 2013 IBM Corporation
Testing for Debt
Continuous Integration (multi-component)
Code Inspection
Watching Trends
© 2013 IBM Corporation
Testing for Debt: Continuous Integration
On code commit, build and test the software
Roll up changes to build-time or runtime dependencies and test those too to identify API incompatibilities
© 2013 IBM Corporation
Testing for Debt: Code Inspection
Code Reviews – Rapidly detect issues
Static Code Analysis – Tool based checks for bugs, code duplication, code theft, & non-compliance with dev standards.
© 2013 IBM Corporation
Testing for Debt: Code Inspection
Code Reviews – Rapidly detect issues
Static Code Analysis – Tool based checks for bugs, code duplication, code theft, & non-compliance with dev standards.
© 2013 IBM Corporation
Testing for Debt: Watching Trends
© 2013 IBM Corporation
More visualizations: Sonar
http://nemo.sonarsource.org/dashboard/index/327690?did=6
© 2013 IBM Corporation
An example safety net
Continuous build & unit test
Nightly slow tests / code scans
Emails identifying new issues – ideally tied against source code changes
Regular inspection of trends
Bugs / Stories entered around issues
© 2013 IBM Corporation
Automation as a learning experience
Implementing the safety net helps us discover unacknowledged debt
© 2013 IBM Corporation
Tests? What tests?
© 2013 IBM Corporation
Automation Examples: The Build
Build time dependencies not understood
Build scripts missing or incomplete
“Magic build server” anti-pattern
http://www.urbancode.com/html/resources/webinars/Role_of_Binary_repositories_in_Software_Configuration_Management.html
© 2013 IBM Corporation
Automation Examples: Deployment
Deployment scripts scarce
“Special Instructions” with most deployments indicate non-repeatable process
Environmental differences handled poorly
Separation of duties less than real
© 2013 IBM Corporation
Automation Example: Test Automation
Requirements less well understood
Existing tests are few, stale, un-optimized
Application not architected for testability
© 2013 IBM Corporation
Expect the unexpected when automating
At scale, Green-Shifting, has hidden issues
Include these discoveries in ROI estimations for automation (positively and negatively)
This is a happy side effect
© 2013 IBM Corporation
Start making decisions
© 2013 IBM Corporation
Direct and indirect automation benefits
Direct: We tested for problems and found them.
Indirect: In attempting to be more efficient, we automated, and accidently discovered problems.
© 2013 IBM Corporation
Automating for the team
Provide a “safety net” to detect and recognize issues.
Diagnose and repair lack of repeatability in build-deploy-test
Quantify accumulating debt in support of fighting scope creep
Team level tooling is fine
© 2013 IBM Corporation
Automating the enterprise
Benefits extend beyond aggregate team level benefits
Central Automation and Reporting gets us:
–Identify who can use shared configuration
–Who has tests, who doesn’t
–Who is using what tools
–Build / deploy failure rates
© 2013 IBM Corporation
Stories from customers
© 2013 IBM Corporation
Favorite Examples: Deployment Failures
Debt: High rate of deployment failures a problem
Interest: QA productivity is getting hurt & lengthened time to market
Goal: Reduce failure rate from 40% to 5%
Approach: Avoid manually fixing a deployment. Fix the automation and redeploy.
Enforcement: Monthly / weekly spreadsheet of success to CIO with a six month plan.
© 2013 IBM Corporation
Favorite Examples: “End of Day” Commits
Debt: Developers commit breaking changes at the end of the day
Interest: Code base broken in morning, or other people stay up late to fix it.
Goal: Avoid other devs staying late to clean up
Approach: Report on failures, and react to negative patterns as a team.
Enforcement: Social pressures
© 2013 IBM Corporation
© 2013 IBM Corporation
Favorite Examples: Low numbers of tests
Problem: Unit testing discipline breaking down over time
Goal: Maintain high or improving coverage
Approach: Standard coverage tools and an emphasis on upward trends.
Enforcement: Trending report
on monitor over CIO’s door
© 2013 IBM Corporation
No hiding!No greenshifting
© 2013 IBM Corporation
Summary
We accumulate technical debt as we race to deliver more, faster.
This causes us to eventually release less, slower, with worse quality
Automation directly and indirectly helps us find issues.
There are benefits at both team and enterprise levels.
© 2013 IBM Corporation
More References
http://urbancode.com/resources
Enterprise CD Maturity Model
Lean Build & Deployment Automation
Managing Release Risks with Metrics
Blogs.urbancode.com
Twitter.com/UrbanCode
facebook.com/IBMUrbanCodeProduc
Slideshare.net/Urbancode
© 2013 IBM Corporation
Yes, we sell products for this
uBuild
–Build management and continuous integration
uDeploy
–Deployment and release automation
uRelease
–Release management tool for planning and executing big releases
... And IBM’s amazing portfolio of CLM, testing tools, service virtualization, provisioning, etc, etc.
© 2013 IBM Corporation
Questions?
eminick@us.ibm.com@EricMinick, @UrbancodeSlideshare.net/Urbancode
top related