3/18/2013
A real-life overview of Agile workflow practicesMichael Toppa @mtoppa www.toppa.com
Mike Toppa
19 years of experience in web development, project management, and functional management
✤ Currently: Rails Engineer, ElectNext
✤ Previously:
✤ Director of Development, WebDevStudios
✤ Director of Web Applications, U Penn School of Medicine
✤ Web developer at: Georgetown University, Stanford University, E*Trade, and Ask Jeeves
Overview
✤ Theory
✤ Waterfall vs Agile
✤ Practice
✤ My experiences at U Penn
✤ ...and ElectNext
So what’s the problem?
Theory
Quality Features
Fast Schedule
LowCost
The software development dilemma
Pick any two
I’ve explained the triangle to dozens of clients over the yearsProgramming is not magic
Traditional “Waterfall” Approach
Source
Features determine the cost and scheduleDefine all requirements up frontLogically break down the workEstimate the effort / durationsPlan out all the work And only then begin the development…while trying to limit any change that will threaten the plan.
A perspective on the traditional approach
Source
Information is lost in handoffs
Source
“I find your lack of faith disturbing”
No opportunity for feedback
Source
Over-production of features
Source
Ask customers what they want at the beginning, when they really don’t knowPenalize them for adding things laterTCL example
“The main thing that pushed Agile and Scrum was that the success rate on traditional projects was terrible; it was 45%. If that was a car-manufacturing place, that would mean you’d throw out every other car you built.”
Ken Schwaber, co-creator of Scrum, 6/21/2011
Low success rate
Source
Source
Cost in software projects means manpowerBrooks’ Law: adding more manpower makes late projects later
Source
Agile: frequent feedback
Source
Every iteration ends with a retrospectiveAlso, feedback from clients during iteration review
Agile: “inspect and adapt”
Source
Single loop learning is “how can we do better”?Double loop learning is “Why do we believe that?”Double loop learning means challenging fundamental assumptions
Agile: incremental and iterative
Source
Develop systems in small portions at a time (incremental), through repeated cycles (iterative), and take advantage of what was learned in each cycle (for both features and implementation)
The Agile workflow
Source
Why?
✤ The pace of business keeps getting faster
✤ Feedback is essential
✤ Time is scarce
✤ Things will change
Incremental development:slice vertically, not horizontally
Source
This is where developers unfamiliar with Agile freak out
How do you develop a UI or a database in pieces? This seems like it would lead to a giant mess. Remember the iterative part - we can sketch out the overall design, but we build incrementally, fleshing out the details of what’s needed soon
It is possible, it is practical, and there are a lot of people doing it.
It's actually the opposite of messy hacking. Doing it well requires a very disciplined development process, and strong application design skills, as you are trying to maintain a solid application design while always being ready to adapt to change.
The Agile Umbrella
Source
Agile is a mindset and a set of values. There are multiple methodologies that implement it. I’ll focus on the most popular one, Scrum
Scrum: overview
Source
Scrum has 3 roles
✤ Product owner
✤ Scrum master
✤ Self-organizing, cross-functional team
Scrum role: Product Owner
Responsible for what the team will work on,
and setting priorities,
but not how the work is done
Responsible for what the team will work on, but not how the work is doneWorks closely with clients to understand their needsGathers and writes business requirements in small pieces, called “user stories”Based on client needs, sets priorities for the teamDoes not have authority over technical design decisionsCannot tell an individual team member what to doA good Product Owner is: available, business-savvy, communicative, decisive, empowered
Scrum role: Scrum Master
A “servant-leader” for the team
A “servant-leader” for the team - analogous to a physical trainerCan coach and evangelize, but not issue commandsBut does have authority over the Scrum processRemoves obstacles for the teamA good Scrum Master is: responsible, humble, collaborative, committed, influential, and knowledgeable
The team: self-organizingand cross-functional
Source
Cross-functional team takes collective responsibility for estimating the work, and doing the workDoing it in the priority order they are asked to followKeeping quality high by working together (inspecting each other's code, discussing best technical solutions, etc)
So who’sthe boss?
Source
Personnel management exists completely outside this structure.Works best in relatively “flat” organizations where people are given autonomy and achievable goalsAntithetical to top-down, command and control hierarchiesMore on the this later
Practice
My experiences at U Penn
✤ 15 web developers and designers
✤ Ad-hoc development process: hadn’t heard of waterfall or agile
✤ Independent: developers worked alone on projects (a huge business risk)
✤ Customer service oriented: say “yes” to everything
✤ Focused on fast delivery
The team
The situation
✤ Ever growing number of clients and projects
✤ Hiring freeze
✤ Sparse project management, little documentation
✤ Ambiguous lines of authority
✤ Reactive planning, daily firefighting
✤ No one in a position to prioritize -> politicized decision making
Mike: Why Agile? Why Scrum?
✤ Maintain frequent interactions with clients
✤ Enable planning
✤ Reduce risk
✤ Improve quality
Maintain frequent interactions with clients - Provides quick feedback, Existing good relationshipsEnable planning- make commitments, and meet them, Reduce need for firefightingReduce risk- Have more than one person know a projectImprove Quality- Reduce misunderstandings, Reduce missed requirements, Have fewer bugs
I expected our scrum adoption to look something like this...
Source
I read Mike Cohn’s book Succeeding with Agile: Software Development using ScrumI taught the team core scrum concepts, I explained the new process to clientsA small team did a pilot project first, Then the whole group switched
But it turned out like this...
Source
Team didn’t like it, and clients didn’t like it. It felt like just adding a new set of work and processes on top of the existing work
The Scrum Promise
“In my Scrum classes I warn attendees of what I call the Scrum Promise: If you adopt Scrum, there will be a day you come into the office nearly in tears over how hard the change can be. This is because Scrum doesn’t solve problems, it uncovers them and puts them in our face. Then, through hard work we address them.”
– Mike Cohn, Agile Trainer
But what was really going on was that it was bringing previously unrecognized and unarticulated problems to the surface
So I brought in a Scrum coach
Source
A skilled external coach is often key for driving change - they bring a wide range of experience and can see things objectivelyIf you’ve never led an Agile transition before, it’s surprisingly easy to do it wrongYou need at least enough management support to pay the coachYou need to make sure you’re bringing in someone good
Problem #1: false start
Source
Throwing people together and calling them a team doesn’t make them a team- stepping on each other's toes in the code, not familiar with each others’ projects, some unwilling to share workI misunderstood the roles:- The clients were acting as their own product owners, The scrum masters were our former project managers, and continued doing traditional project management- I had several “scrum-buts” - aspects of our process where we didn't follow scrum and stuck to how we always had worked before
Problem #2: too much work
9 developers, 2 product owners, and me supporting- 22 clients with 124 applications3 designers and 1 product owner supporting- about 200 static content web sitesTaking inventory itself was a huge undertaking
Estimating: story points and planning poker
Photo by Kelly HiranoUsed with permission
- How did we generate that chart?- The teams give “story points” to the work by playing planning poker (the work is expressed in a format called “user stories”)- People are bad at estimating time, but we're good at estimating relative size or difficulty- Team based estimates are more accurate than estimates by individuals
Velocity enables scheduling and “sustainable pace”
Source
After a few sprints, our teams had velocities, which allowed us to make time estimates for projects.Also gave us a solid basis for not saying “yes” to every new work request.And this is key to the Agile goal of “sustainable pace”
Problem #3: task switching
Source
Context switching between two projects eats about 20% of a full-time worker’s schedule
Problem #4: Misalignment of authority and responsibility
Cartoon by Mike LynchUsed with permission
- Following this advise lets you cover yourself politically, and is a great way to make everyone who works for you miserable- I've found that misalignment of authority and responsibility can explain a lot of dysfunction that happens in organizations- When you have responsibility for your work but not enough authority over it, you will feel like a cog in machine
What makes a job enjoyable?
✤ Autonomy
✤ Reward for effort
✤ Challenging/complex work
“Work that fulfills these three criteria is meaningful.”
– Malcolm Gladwell, “Outliers: The Story of Success”
In the end...
✤ Team improved practices, quality, and predictability
✤ Attempts to better align authority and responsibility with clients (by means of creating an advisory board) failed
Practice
My experiences at ElectNext
The team
✤ 6 person company -> freedom of action
✤ Spread out across 3 cities - meetings are online
✤ Team was already doing a couple Agile practices: daily stand-ups and weekly sprints
✤ But not other practices -> some confusion around goals and workflow
✤ No external clients: designing and selling our product
✤ Highly collaborative, can-do team
1 CEO, 2 business developers, 1 product manager, 2 engineersDaily standup, weekly planning, but no longer range planning, no retrospectives and no specific agile roles
Refinements
✤ Added retrospectives and setting monthly goals
✤ Stabilized engineering velocity
✤ Decided to not have product owner and scrum master roles for now
✤ But more defined roles may be needed as we grow
Our daily standup
Each part of the company has a separate section of the document- more detailed tracking is does separately
References: overviews and product management
✤ “Succeeding with Agile: Software Development Using Scrum” and “Agile Estimating and Planning” by Mike Cohn
✤ “Specification by Example” and “Impact Mapping” by Gojko Adzic
✤ “Kanban: Successful Evolutionary Change for Your Technology Business” by David J. Anderson
✤ “The Lean Startup” by Eric Ries
References: technical practices
✤ “Clean Code” and “Agile Software Development, Principles, Patterns, and Practices” by Bob Martin
✤ “Refactoring: Improving the Design of Existing Code” by Martin Fowler
✤ “Agile Database Techniques” by Scott Ambler
✤ “The Rspec Book: Behaviour-Driven Development with RSpec, Cucumber, and Friends” by David Chelimsky
✤ “Working Effectively with Legacy Code” by Michael Feathers
References: Agile beyond software development✤ Angry Dinosaurs: Accelerating Change and Institutional
Incompetence
✤ Cory Ondrejka, Wharton Web Conference, 2010
✤ Let it Roll: Why more companies are abandoning budgets in favor of rolling forecasts
✤ CFO Magazine, May 1, 2011
✤ The Insourcing Boom
✤ Atlantic Magazine, December 2012
3/18/2013
Any questions?Michael Toppa @mtoppa www.toppa.com