why #opendx?
Post on 22-Mar-2017
145 Views
Preview:
TRANSCRIPT
Gradle Summit 2016
How to Measure Your #GradleDX
Janelle Klein
, Developer, Consultant, CTO @Specialized in Statistical Process Control (SPC) and Supply Chain Optimization from Lean Manufacturing (data geek)
Continuous Delivery infrastructure, automation strategy & technical mentorship
Janelle Klein
Who Am I?
How to Measure the PAIN in Software Development
Janelle Klein
Author of “Idea Flow”
leanpub.com/ideaflow
Yay!
Tour SpeakerFounder of
newiron.com
What is This Talk About?
#OpenDXAn Open Standard for Measuring PAIN
Universal definition of “Better”
(Specification for Data Collection)
Developer Experience
#GradleDX is a Pilot Project
#OpenDX
within
#GradleDX
Feedback LoopCommunity Pain Sensor
Tools
Pilot: Supplier Partnership
#GradleDX Community Project
Plugin
Plugin
Plugin
Plugin
Why does partnership matter in a software supply-chain?
My Codeis AWESOME!
100% Code Coverage
Low Cyclomatic Complexity
Usability Problems
This is PAINFUL!
Depend On API
WTF?!
WTF?!
WTF?!
WTF?!
WTF?! WTF?!
My Codeis AWESOME!
100% Code Coverage
Low Cyclomatic Complexity
This is PAINFUL!
Depend On API
WTF?!
WTF?!
WTF?!
WTF?!
WTF?! WTF?!
We can both be looking at the exact same code, and have a totally different experience.
100% Code Coverage
Low Cyclomatic Complexity
Usability Problems
Depend On API
WTF?!
WTF?!
WTF?!
WTF?!
WTF?! WTF?!
Here’s what caused me PAIN
Painful Experience
Oh…I can fix that!
100% Code Coverage
Low Cyclomatic Complexity
Depends On API
WTF?!
WTF?!
WTF?!
WTF?!
WTF?! WTF?!
Oh…I can fix that!
100% Code Coverage
Low Cyclomatic Complexity
Depend On API
Oh…I can fix that!
No More Pain
Thisis AWESOME!
100% Code Coverage
Low Cyclomatic Complexity
Depend On API
Yay! It’s AWESOME!
No More Pain
This is AWESOME!
Visibility Facilitates the Conversation
Application
3rd Party Dependencies
Local Components
Usability Problems
90% of our Software is HERE
Application
3rd Party Dependencies
Local Components
Usability Problems
90% of our Software is HERE
This is PAINFUL!
Bad Software has become the new Environment Pollution
Cost &
Risk
Human Limitations
We are HERE:
Difficulty of Work
What’s Causing our Pain? Cumulative Usability Problems in the Software Supply Chain
Alert!
Alert!
Our Application
PRESSURE for better quality parts PAIN
What if we started measuring PAIN across the software supply chain?
This is PAINFUL!
Everything is AWESOME!
Our Application
PRESSURE for better quality parts
What if we started measuring PAIN across the software supply chain?
Feedback LoopCommunity Pain Sensor
Tools
#GradleDX: Pilot for Supply-Chain Feedback Loop
Plugin
Plugin
Plugin
Plugin
Pilot: Supplier Partnership
Why Should YOU Care?
source: Domics - “What’s a Shrug?”
Every few years we rewrite our software…
Start%Over%
Unmaintainable%So0ware%
Across the Industry
We Start with the Best of Intentions
High Quality Code
Low Technical Debt
Automated Deployment
Good Code Coverage
Then This Happens!
PAIN
The Classic Story of Project Failure
Problems get deferred
Builds start breaking
Releases get chaotic
Productivity slows to a crawl
Developers begging for time
It’s never enough
Project Meltdown
Our Solution…
Until we can’t take it anymore.
On to greener pastures…
Has anyone struggled with the challenge of business pressure?
Has anyone ever tried to fix this problem within their organization?
Has anyone ever succeeded?
This is the biggest cause of FAILURE in our industry:
Because PAIN Doesn’t Show Up
on a Balance Sheet
If we don’t MEASURE it,
Why?
they don’t SEE it.
“Better”“Better”
What if you could get managers and developers all pulling the same direction in your company?
ManagersDevelopers
“Better”“Better”
What if we could solve this problem in ALL of our companies?
ManagersDevelopers
Two Options:
Option 1
Stay the Course
Here’s the Catch:
In order for this to work, we have to start working together as a community.
Option 2
Build Happiness
or
Let’s Do This
LEARN YOUR WAY TO AWESOME.
Free to Join Industry Peer Mentorship Network
1. Make the PAIN visible
2. Breakdown the causes into patterns
3. Help each other to learn & improve
We are working together to INVENT solutions for:
Community Analytics Platform
Process Control, Supply-Chain Optimization &
Throughput Accounting for
Software Organizations
#OpenDXAn Open Standard for Measuring PAIN
Based on:
The Goal Eliyahu M. Goldratt
“Throughput Accounting”
System Productivity =Rate of Revenue
Rate of Investment
Theory of Constraints Optimizing Flow in a Supply Chain System
“Throughput Accounting”
System Productivity =Rate of Revenue
Rate of Investment
Throughput Accounting measures how efficiently we turn effort into money.
Our PAIN Matters
How Do We Measure PAIN?
Great Team Disciplined with Best Practices Constantly Working on Improvements+
Project FAILURE
About 8 Years Ago…
ApplicationTests
Our Test Suite
ApplicationTests
Our Test Suite
“What are we supposed to do?!”
The Retrospective
“What are we supposed to do?!”
Our biggest problem
“Well, if we don’t understand a problem, we should
collect data.”
The Retrospective
Our biggest problem
“What data would help us understand the problem?”
“What are we supposed to do?!”
Technical Debt Mistakes
I thought the main obstacle was Technical Debt
?Most of our mistakes were in the
most well-written parts of the code.
Mistakes
We made significantly more mistakes in code that we didn’t write ourselves.
Lower Familiarity
More Mistakes=
There had to be more to the story...
Complex(So*ware(
PAIN
This is what I knew...
What made development feel painful?
Unexpected Behavior
Problem Resolved
Tracking Painful Interaction with the Code (Friction)
Troubleshooting
Progress
5 hours and 18 minutes of troubleshooting...
PAINFUL
The amount of PAIN was caused by…
Likeliness(of((Unexpected(Behavior(
Cost(to(Troubleshoot(and(Repair(
High(Frequency(Low(Impact(
Low(Frequency(Low(Impact(
Low(Frequency(High(Impact(
PAIN(
What Causes Unexpected Behavior (likeliness)?
What Makes Troubleshooting Time-Consuming (impact)?
Semantic Mistakes
Stale Memory Mistakes
Association Mistakes
Bad Input Assumption
Tedious Change Mistakes
Copy-Edit Mistakes
Transposition Mistakes
Failed Refactor Mistakes
False Alarm
Non-Deterministic Behavior
Ambiguous Clues
Lots of Code Changes
Noisy Output
Cryptic Output
Long Execution Time
Environment Cleanup
Test Data Creation
Using Debugger
Most of the pain was caused by human factors.
What causes PAIN?
What Causes Unexpected Behavior (likeliness)?
What Makes Troubleshooting Time-Consuming (impact)?
Non-Deterministic Behavior
Ambiguous Clues
Lots of Code Changes
Noisy Output
Cryptic Output
Long Execution Time
Environment Cleanup
Test Data Creation
Using Debugger
What causes PAIN?
Most of the pain was caused by human factors.
Semantic Mistakes
Stale Memory Mistakes
Association Mistakes
Bad Input Assumption
Tedious Change Mistakes
Copy-Edit Mistakes
Transposition Mistakes
Failed Refactor Mistakes
False Alarm
What Causes Unexpected Behavior (likeliness)?
What Makes Troubleshooting Time-Consuming (impact)?
Non-Deterministic Behavior
Ambiguous Clues
Lots of Code Changes
Noisy Output
Cryptic Output
Long Execution Time
Environment Cleanup
Test Data Creation
Using Debugger
What causes PAIN?
Semantic Mistakes
Stale Memory Mistakes
Association Mistakes
Bad Input Assumption
Tedious Change Mistakes
Copy-Edit Mistakes
Transposition Mistakes
Failed Refactor Mistakes
False Alarm
Most of the pain was caused by human factors.
What Causes Unexpected Behavior (likeliness)?
What Makes Troubleshooting Time-Consuming (impact)?
Non-Deterministic Behavior
Ambiguous Clues
Lots of Code Changes
Noisy Output
Cryptic Output
Long Execution Time
Environment Cleanup
Test Data Creation
Using Debugger
What causes PAIN?
PAIN is a consequence of how we interact with the code.
Semantic Mistakes
Stale Memory Mistakes
Association Mistakes
Bad Input Assumption
Tedious Change Mistakes
Copy-Edit Mistakes
Transposition Mistakes
Failed Refactor Mistakes
False Alarm
PAIN occurs during the process of understanding and extending the software
Complex(So*ware(
PAIN
Not the Code.
Optimize “Idea Flow”
We deleted everything…
The automation wasn’t solving the problem!
What are the specific risks in this release?
How can we mitigate the risks?
Handful of manual tests
Custom Tools
+
Time-Bound Releases
RISK-Bound Releases
Release Size
(weeks)
Time
Defect Rates
RISK-Bound Releases
Continuous Delivery is about learning how to effectively manage risk
My team spent tons of time working on improvements that didn’t make much difference.
We had tons of automation, but the automation didn’t catch our bugs.
My team spent tons of time working on improvements that didn’t make much difference.
We had well-modularized code,
but it was still extremely time-consuming to troubleshoot defects.
The hard part isn’t solving the problems it’s identifying the right problems to solve.
“What are the specific problems that are causing the team’s pain?”
Complex(So*ware(
PAIN
Measure Friction in “Idea Flow”
OpenDX measures the “friction” that occurs during a developer’s experience
The Rhythm of “Idea Flow”
Write a little code.
Work out the kinks.
Write a little code.
Work out the kinks.
Write a little code.
Work out the kinks.
Conflict Confirm
Rework'
Learn'
Validate(
Modify'
Progress Loop Conflict Loop
Troubleshoot'
Write a little code. Work out the kinks.
The Rhythm of “Idea Flow”
On Intelligence Jeff Hawkins
Confirmed Expectations
Our brain works like a giant prediction engine…
Violated Expectations
Our brain works like a giant prediction engine…
Conflict Confirm
Rework'
Learn'
Validate(
Modify'
Progress Loop Conflict Loop
Troubleshoot'
The Rhythm of “Idea Flow”
“Friction” in Idea Flow
Conflict Confirm
Rework'
Learn'
Validate(
Modify'
Progress Loop Conflict Loop
Troubleshoot'
“Idea Flow Map”
“Friction”
Idea Flow Mapping Tools
(Open Source, Supported GA ~July 2016)github.com/ideaflow/tools
Typical Idea Flow Maps
Single Problem
Multi-Problem
We can read Visual Indicators in Idea Flow Maps
Le#$Atrium$
Le#$Ventricle$
Right$Ventricle$
Right$Atrium$
What’s$causing$this$pa7ern?$
Similar to how an EKG helps doctors diagnose heart problems...
...Idea Flow Maps help developers diagnose software problems.
Problem-Solving Machine
We can read Visual Indicators in Idea Flow Maps
= Solution Strategy
The “Heart” of Software Development(the problem-solving machine)
Based on: Grandon Gill and Richard Hicks models of Task Complexity and John Boyd’s OODA Loop
= Solution Strategy
The “Heart” of Software Development(the problem-solving machine)
Choose a general strategy
= Solution Strategy
The “Heart” of Software Development(the problem-solving machine)
Understand the system
= Solution Strategy
The “Heart” of Software Development(the problem-solving machine)
Code & work out the kinks
= Solution Strategy
The “Heart” of Software Development(the problem-solving machine)
Back to the drawing board
The symptoms we experience depend on where the disruptions are in the problem-solving process
Trial and Error
The Friction is Here
Troubleshooting
Progress
Learning
Rework
Assumption Risk (Rework)
Likelihood)of))making)a))
Bad)Assump4on)
Cost)to)Correct)Decisions)
High)Uncertainty)Low)Delay)
Low)Uncertainty)Low)Delay)
Low)Uncertainty)High)Delay)
PAIN)
Struggling with Understanding the Code
The Friction is Here
Troubleshooting
Progress
Learning
Rework
Familiarity Risk (Learning)
Likelihood)of))working)with)Unfamiliar)
Code)
Cost)to)Learn)
High)Frequency)Easy)to)Learn)
Low)Frequency)Easy)to)Learn)
Low)Frequency)Hard)to)Learn)
PAIN)
Struggling with Feedback
The Friction is Here
Troubleshooting
Progress
Learning
Rework
Quality Risk (Troubleshooting)
Likelihood)of))Unexpected)Behavior)
Cost)to)Troubleshoot)and)Repair)
High)Frequency)Low)Impact)
Low)Frequency)Low)Impact)
Low)Frequency)High)Impact)
PAIN)
”Friction” is the time spent on:
x
Troubleshooting
x
Learning
x
Rework
Quality Risk Familiarity Risk Assumption Risk
“What caused the pain in this case?”
Categorize the Problems with #HashTags
#ReportingEngine
#Hibernate
#MergeHell
1. Problem A
2. Problem B
3. Problem C
1000 hours/month
Add up the Pain by Category
What’s the biggest problem to solve?
Case studies.
Troubleshooting
Progress
Learning
7:070:00
0:00 19:52
12 year old project after all original developers left.
Case Study: Huge Mess with Great Team
70-90% of dev capacity on “friction”
The Team’s Improvement Focus: Increasing unit test coverage by 5%
Case Study: Huge Mess with Great Team
1. Test Data Generation
2. Merging Problems
3. Repairing Tests
1000 hours/month
The Biggest Problem: ~700 hours/month generating test data
18 months after a Micro-Services/Continuous Delivery rewrite.
Troubleshooting
Progress
Learning40-60% of dev capacity on “friction”
0:00 28:15
12:230:00
Case Study: From Monolith to Microservices
The Architecture Looked Good on Paper
Team A Team B Team C
Complexity Moved Here
We Don’t Have TIME To Fix It!
The Cost of Escalating Risk
0%
100%
Release 1 Release 2 Release 3
Troubleshooting
Progress
Learning
Percentage Capacity spent on Troubleshooting (red) and Learning (blue)
(extrapolated from samples)
0%
100%
Release 1 Release 2 Release 3
Percentage Capacity spent on Troubleshooting (red) and Learning (blue)
Figure out what to do Learning is front-loaded
Troubleshooting
Progress
Learning
The Cost of Escalating Risk
0%
100%
Release 1 Release 2 Release 3
Percentage Capacity spent on Troubleshooting (red) and Learning (blue)
Rush Before the Deadline Validation is Deferred
Troubleshooting
Progress
Learning
The Cost of Escalating Risk
0%
100%
Release 1 Release 2 Release 3
Percentage Capacity spent on Troubleshooting (red) and Learning (blue)
Pain Builds Baseline friction keeps rising
Troubleshooting
Progress
Learning
The Cost of Escalating Risk
0%
100%
Release 1 Release 2 Release 3
Percentage Capacity spent on Troubleshooting (red) and Learning (blue)
Chaos Reigns Unpredictable work stops
fitting in the timebox
Troubleshooting
Progress
Learning
The Cost of Escalating Risk
Over 1000 Hours/Month in Downtime!
They tried to solve their design problems with more test automation.
25 developers down for 2 days
(It didn’t work)
Lots of Friction in BRAND NEW code:
27:15
“WHY?”
Application
3rd Party Dependencies
Local Components
Usability Problems
90% of our Software is HERE
This is PAINFUL!
“The Idea Flow Factory” (supply chain model)
Optimize the Rate of Idea Flow
Across ANY Software Supply Chain
Build Test Deploy
Typical manufacturing metaphor mapping:
Customer
The Problem: Focus on short-term effects
Software Supply Chain as the Factory Floor
Customer
Software Supply Chain as the Factory Floor
Customer
Dev = Operator “flows” the components
Architecture Changes Create Bottlenecks
Customer
Lack of Familiarity Creates Bottlenecks
WTF?!
Customer: Big Kahuna
Customers: Seg 2 Seg 3
Customers: Seg 2 Seg 4 Seg 5
Connect Maintenance Effort to Revenue through Software Dependencies
“Big Kahuna”
Effort
Revenue
“Throughput Accounting” Metrics
Cost of Support vs Revenue per Customer Segment (%)
Effort
Revenue
“Throughput Accounting” Metrics
Cost of Support vs Revenue per Customer Segment (%)
Breakdown of Development Capacity
If you were an investor: wouldn’t YOU want to reduce friction?
“Throughput Accounting” Metrics
With #OpenDX, we can apply the entire suite of Risk Management Tools from Lean Manufacturing
Statistical Process Control (SPC)
Optimal Friction
Upper Control Limit X"
“Out of Control”
20min
0m
Average Pain per Incident
Target
Control Limit
Supply Chain Optimization
Throughput Accounting
We have the power to solve this problem:
If we choose to do it.
Application
3rd Party Dependencies
Local Components
Usability Problems
This is PAINFUL!
We have the power to solve this problem:
Two Options:
Option 1
Stay the Course
Option 2
Build Happiness
or
Let’s Do This
Miko Matsumura: “Build Happiness” is a philosophy that happiness must be built.
Miko: “That’s not a happy elephant!”
Miko: “That’s not a happy elephant!”
Focus on the PAIN & Build Happiness
Janelle Kleinopenmastery.org @janellekz
Check out openmastery.org for details.
Read my Book.
Think About It.
FREE with MembershipBuy It
How to Measure the PAIN in Software Development
Janelle Klein
Learn more at www.gradle.org
top related