waterloo agile lean p2p group creating the technical plumbing to support agile tools and techniques...
TRANSCRIPT
Waterloo Agile Lean P2P Group
Creating the Technical Plumbing to Support Agile Tools and Techniques from the Muddy Trenches
Leon KehlNovember 22, 2011Waterloo Ontario
Agenda
A Tale of Two Agilites
How Agile are you and your company?
The Philosophy and Practise of Systems Design
Peer to Peer Discussion
Examples from the Trenches
Dialectic Peer to Peer Discussion Revisited
Q/A
How are Agile are you?
How would you rate your Agile knowledge (1-10)?
How would you rate your companies Agile practices (1-10)?
Agile Top 10 Survey
Survey Says: Agile Works in Practice. Sept 2006 – Dr. Dobbs Journal
Agile Top 10 Survey
Survey Says: Agile Works in Practice. Sept 2006 – Dr. Dobbs Journal
Trivial Pursuit
Who knows what happened to the original C3 project at Chrysler?
Trivial Pursuit
The plan was to roll out the system to different payroll 'populations' in stages, but C3 never managed to make another release despite two more years' development. The C3 system only ever paid 10,000 people (target was 85,000)
Chrysler was bought out by Daimler-Benz in 1998. DaimlerChrysler stopped the C3 project on 1 February 2000.
Frank Gerhardt, a manager at the company, announced to the XP conference in 2000 that DaimlerChrysler had de facto banned XP after shutting down C3; however, some time later DaimlerChrysler resumed the use of XP.
The Knowing – Doing Gap
• Each year $60 Billion spent on training
• TQM Training at five companies. Four weren’t using it and last limited use only
• At Toyota “Many plants have put in an andon cord .. to stop the assembly line. A 5-year old can pull the cord. But it takes a lot of effort to drive the right philosophies”
Causes
• Knowing “What to Do is Not Enough
• When Talk Substitutes for Action
• When Memory Is a Substitute for Thinking
• When Fear Prevents Acting on Knowledge
• When Measurement Obstructs Good Judgement
• When Internal Competition Turns Friends into Enemies
Computers in Context: The Philosophy and Practice of System Design
They examine key assumptions about the role of software developers, and their relationship to culture and work in a way which touches everyday practice and which can subtly transform it
Thinking
• Hard Systems Thinking
• Soft Systems Thinking
• Dialectic Systems Thinking
Hard Systems Thinkingaka Waterfall
• A hierarchically organized set of elements
• Everything that can be said in principle is contained in all elements and properties
• Functional Analysis from the Top Down
• Try to create a “true” representation of the world
• “when the map and the terrain disagree the error is in the terrain”
Soft Systems Thinkingaka Agile
• Systems are in our minds, they are perspectives that we change and improve by being confronted with other perspectives
• Design is seen as learning
• Perspectives are communicated, shared and negotiated to create shared interest in change
• Requires substantial experience and professionalism by practitioners
Manifesto for Agile Software Development
Individuals and interactions over processes and tools
Working software over comprehensive documentation
Customer collaboration over contract negotiation
Responding to change over following a plan
Dialectic Systems Thinking
• Originates with Socrates, used by Marx, Nietzsche and Freud
• Face a network of dynamically changing contradictions
• Resolving some conflicts requires changing perceptions but also resources, structures, systems and power relationships
• The world rather than people’s perceptions of it, is our primary source for learning
• World suffused with conflicts, dark forces, frightening jungle of different interests, struggles for power
Examples of Contradictions
• Developers are told to write unit tests because they will be more productive, but believe they don’t add value and always do them after development
• Customers will report problems and developers will say they are impossible
• Management wants to become Agile without changing legacy software, developers or development practices
• Development is told they are too slow but can’t make wholesale changes to speed things up
• Code quality is supposed to be a priority but developers can go to anyone for code reviews
• Customers and management complain about bugs, but automated unit tests aren’t a priority
Peer to Peer Support
• Split into pairs
• Explain to your partner a current or recent situation where you are seeing a knowing/doing gap or a contradiction relating to Agile best practices.
• Take about five minutes and then switch.
• As the listener, ask questions try to understand the network of contradictions present in the situation
Peer to Peer Support
• Did the stories you heard support these ideas of a knowing/doing gap and the idea of dialectic thinking?
• Do you have any observations you want to share?
Dialectic Systems Thinking Creating an Intervention
• Contradictions are opportunities for intervention
• The challenge is to understand and change established traditions in the user organization as well as in the project group and in the development organization as a whole
• The interventionist paradigm suggests we actively participate in organizational games
• Remain a problem solver relying on both construction and evolution approaches
Enough PhilosophyLet’s Get Practical
• So you want to change the system?
• What tools and techniques can you use?
• What ideally are you trying to achieve?
Enough PhilosophyLet’s Get Practical
• So you want to change the system?
• What tools and techniques can you use?
• What ideally are you trying to achieve?
• Well it depends. Dialectic Systems thinking would imply each situation is unique and presents its own contradictions to exploit
Agile? At Google
Google is an exceptionally disciplined company, from a software-engineering perspective. They take things like unit testing, design documents and code reviews more seriously than any other company I've even heard about. They work hard to keep their house in order at all times, and there are strict rules and guidelines in place that prevent engineers and teams from doing things their own way. The result: the whole code base looks the same, so switching teams and sharing code are both far easier than they are at other places.
And engineers need great tools, of course, so Google hires great people to build their tools, and they encourage engineers (using incentives) to pitch in on tools work whenever they have an inclination in that direction. The result: Google has great tools, world-class tools, and they just keep getting better.
Case #1
• 20 year old legacy system, ported from Unix paired with another legacy system for another purpose
• Supported by developers with long term service, but weak development skills
• “Unit tests” consisted of a 100 page Word document that got added to when functionality changed
• Product management wanted to support another higher speed hardware device for new markets
• Any testing required minutes to hours of setup and physical devices
Solution #1
• Worked on system to develop understanding of business space, software, skillsets
• Developed proposal to rewrite from the ground up utilizing similar technology but with Agile and working prototype in two months. Chutzpah required
• Then got green light for full development
• Used simulators, automated unit tests, continuous integration. Used younger more “flexible” resources.
• Didn’t attempt to understand all existing code, only what it was intended to do
• Was able to accommodate new device into design
Obstacles #1Considerable pressure to use existing components, algorithms,
resources because that was what was always done:
• Ran as silently as possible, avoided seeking advice from existing “experts” or using “experienced” resources
• Used data from the field, run through previous and new algorithms to demonstrate poor performance and in fact crashes from existing libraries
• Measured performance of standard logging to demonstrate inefficiencies
• Used continuous integration, automated unit tests, automated integration tests, code coverage, automated document generation etc. to minimize potential criticisms and to solidify management approval
Case #2
• System was recently developed, using Agile with automated unit tests, code coverage and nightly builds
• A number of team members left, replacements weren't as concerned about unit tests saw them as overhead
• Build often broken, automated unit tests were allowed to fail without fixing, time to run increased significantly to almost an hour
• Little management support for improvements, focus on adding features, bug fixes
Solution #2
• Got buy-in from receptive team members for continuous integration
• Split unit tests into majority which could run quickly and those that couldn't
• Grabbed a spare PC to act as a integration server
• Fixed all the unit tests and got them passing
• Emails and public shaming kept unit tests passing
Case #3
Older thick client application subsystem, responsible for fetching, caching and displaying large quantities from server
Although other subsystems had automated unit tests for a number of years, no real tests existed
Complicated design, difficult and expensive changes, lots of bugs, chief designer left company, remaining resources had weak knowledge
Desire from management for better, stronger faster but can't find resources or solutions
Solution #3
• Using nothing more than sophisticated than Task Manager was able to demonstrate caching wasn't working for large volume cases where users were reporting problems
• Problems were worse in high latency, lower bandwidth situations. Nobody knew because of lack of instrumentation, despite large test groups
• Pitched a skunkworks project to reimplement fetching and caching subsystem using Agile best practices: unit tests, test driven development etc.
• Demonstrated a working prototype that addressed all outstanding issues
• Not yet implemented yet, but multiyear project to extend existing caching to local disk abandoned during final internal testing due to major performance problems
Case #4 Relatively new .Net application, had unit tests
continuous integration, worked on by large number of developers over the years including contractors Lots of bugs, fixes and changes tended to cause more bugs, expensive to change anything
Use of a limited number of design patterns, complaints that there has to be a better ways. Frustration from developers and business partners. Changes consisted basically of jamming in new features under pressure
Solution #4• Use Code Coverage tool to measure code coverage of unit
tests and showed that coverage was only 3%. Analyzed code repository commit records to show recent developer activity was probably only 1%
• Examined code changes for recent project and existing code. Found poor existing designs, inadequate code reviews, violation of basic OO principles.
Identified one common base class intended to be simple wrapper class was over 11,000 lines. No unit tests and in fact impossible to unit test. Had been worked on by everyone.
• Pitched prototyping rewrite to sympathetic management looking for solutions to improve productivity and quality
• Developed prototype with simplified design, dramatic reduction in size and complexity. Used Agile best practices: Complete unit tests, measured with code coverage tools
Continuous Integration
Builds would be broken for days, then everyone would commit again breaking for days
Lava lamps were introduced because they were cool
Side effect was suddenly failures were visibleto management, reality was exposed
Created the opportunity to redo slow flaky build process created by “experts”
Impact of Change
Dropped build times to 20 minutes
Eliminated build problems
Broken builds became rareoccurrences
Recommended Practices
Agile is people centric so success depends on them. Seek out allies, try not to create enemies!
Minimize dependence and interaction with “experienced” resources that created the broken system or process. Better to remain strangers than become enemies
Don't rely on management to make things happen or enforce team norms. Use dialectic thinking
Skunkworks if necessary
Data is your friend. Make visible to the team and management what you want changed. Turn over the rocks!
Recommended Practices
Appraise the resources carefully. An experienced resource that helped create a broken system or process should be avoided. If they were good they would have fixed it or left. Best is a more junior resource who has knowledge and wants change
Need to assess code quality, design, warts and all in detail to understand weaknesses gaps, possibilities. Don't need to understand everything, just how to exploit that knowledge
Fully embrace Agile best practices, because it makes it harder to argue against change. Focus on universal best practices
In general avoid piecemeal approach to broken legacy systems. Try to focus on larger self contained changes.
Patience and timing are everything and failure leads to success
Peer to Peer Support
• Find your partner again
• Based on what you've just heard what strategies/tactics could you use to make a change in the original situation?
• Take about five minutes and then switch.
• This time both of you dialogue trying to come up with solutions to each person's situation.
Anything to share?Any questions?
In conclusion
The significant problems we face cannot be solved at the same level of thinking we were at when we created them
Things should be as simple as possible, but not simpler
I am neither especially clever nor especially gifted. I am only very, very curious
Albert Einstein