limited wip meeting presentation - the phoenix project book review
DESCRIPTION
London Limited WIP community meeting in October. A review of a great book "The Phoenix Project".TRANSCRIPT
The Phoenix Project: A Novel About IT, DevOps, and Helping Your
Business Win.
Poll
● How many have read “The Phoenix Project”?● What is the cycle time in your org from code to
production?● How many of you have Ops represented in Dev
teams?– How long has Ops people been embedded in Dev
teams?
Who is here today?
● Why are you here?– Name
– Company
– Role
– Interest in book / event / Limited WIP
– What do you want to get out of tonight?
My Objective
● Get more people to read “The Phoenix Project”– Make a difference to more people
● Provide some thinking tools and references– Speed up your learning and adoption
● Learn from you and your experiences
Why is this important?
High Performing DevOps Teams ● They’re more agile
– 30x more frequent deployments
– 8,000x faster cycle time than their peers
● They’re more reliable– 2x the change success rate
– 12x faster MTTR ● MTTR = Mean time to Restore/Repair/recovery
Source: Puppet Labs 2012 State Of DevOps: http://puppetlabs.com/2013-state-of-devops-infographic
7
Buzz about book first caught my attention
Top 100 Books
● http://www.noop.nl/2013/08/top-100-agile-books-edition-2013.html
●
What does Gene Kim Say about the book?
● http://www.youtube.com/watch?v=FpEd-QKVIrw 5 min
The Phoenix Project - The Authors
● Gene Kim– Gene Kim is a multiple award-winning entrepreneur, the founder
and former CTO of Tripwire and a researcher. He is passionate about IT operations, security and compliance, and how IT organizations successfully transform from “good to great”.
● George Spafford– George is an author and speaker, consulting and conducting
training on strategy, IT management, information security and overall service improvement in the U.S., Canada, Australia, New Zealand and China. Co-author of “The Visible Ops Handbook” and “Visible Ops Security,” George is a certified ITIL Expert, TOCICO Jonah and a Certified Information Systems Auditor (CISA).
● Kevin Behr– Kevin Behr is the founder of the Information Technology
Process Institute (ITPI) and the CTO of Assemblage Pointe. Kevin has twenty years of IT management experience and is a mentor and advisor to Chief Executive Officers and Chief Information Officers.
Book Introduction
● Bill is an IT manager at Parts Unlimited. It’s Tuesday morning and on his drive into the office, Bill gets a call from the CEO.
● The company’s new IT initiative, code named Phoenix Project, is critical to the future of Parts Unlimited, but the project is massively over budget and very late. The CEO wants Bill to report directly to him and fix the mess in ninety days or else Bill’s entire department will be outsourced.
● With the help of a prospective board member and his mysterious philosophy of The Three Ways, Bill starts to see that IT work has more in common with manufacturing plant work than he ever imagined. With the clock ticking, Bill must organize work flow streamline interdepartmental communications, and effectively serve the other business functions at Parts Unlimited.
● In a fast-paced and entertaining style, three luminaries of the DevOps movement deliver a story that anyone who works in IT will recognize. Readers will not only learn how to improve their own IT organizations, they’ll never view IT the same way again.
Cast at Parts Unlimited
● Parts Unlimited: Business Executives – Steve Masters, CEO, acting CIO
– Dick Landry, CFO
– Sarah Moulton, SVP of Retail Operations
– Maggie Lee, Senior Director of Retail Program Management
– Bill Palmer, VP of IT Operations , former Director of Midrange Technology Operations
– Wes Davis, Director of Distributed Technology Operations
– Brent Geller, Lead Engineer
– Patty McKee, Director of IT Service Support
– John Pesche, Chief Information Security Officer (CISO)
– Chris Allers, VP of Application Development
● Parts Unlimited: Board – Bob Strauss, Lead Director, former Chairman, former CEO
– Erik Reid, Board Candidate
– Nancy Mailer, Chief Audit Executive
Why a novel?● Who has read “The Goal”?
● Storytelling Is The Most Effective Means Of Creating A Shared Understanding
● The pattern is everywhere: The core of the solution-selling, good copy-writing and the best TED talks: Describe the problem, show its significance, describe how the problem is solved and the value it will creates.
● Storytelling becomes very important when the problem we’re trying to point out is not fully recognized, or when the solution being proposed flies in the face of common wisdom.
http://itrevolution.com/learn-more-about-concepts-in-phoenix-project/http://www.youtube.com/watch?v=oP3c1h8v2ZQ
Time LineC
P%
S
t ory
Ti m
e1
154%
02 S
ep2
257%
02 S
ep3
3611
%02
Sep
447
14%
03 S
ep5
6218
%04
Sep
674
22%
05 S
ep7
8425
%05
Sep
894
28%
08 S
ep9
104
31%
09 S
ep10
111
33%
11 S
ep11
119
35%
11 S
ep12
125
37%
12 S
ep13
138
41%
15 S
ep14
146
43%
16 S
ep15
154
46%
17 S
ep16
166
49%
18 S
ep17
177
52%
22 S
ep18
182
54%
23 S
ep19
188
56%
23 S
ep20
203
60%
26 S
ep21
216
64%
26 S
ep22
223
66%
29 S
ep23
233
69%
07 O
c t24
238
70%
11 O
c t25
246
73%
14 O
c t26
255
75%
17 O
c t27
264
78%
21 O
c t28
273
81%
27 O
c t29
282
83%
03 N
ov30
293
87%
03 N
ov31
299
88%
03 N
ov32
307
91%
10 N
ov33
314
93%
11 N
ov34
321
95%
28 N
ov35
328
97%
09 J
an
Problems
How to create a “Problem”?
The ERM framework by the Commission of Sponsoring Organizations of the Treadway Commission (COSO) provides a more disciplined and consistent standard against which to implement and assess a company’s ERM program.
http://www.coso.org/documents/coso_erm_executivesummary.pdfhttp://www.youtube.com/watch?v=b7JldvsY7Ac
Strategy
● Strategy: as embodied by the Phoenix Project, which the entire future of the company depends upon. It requires that IT be a core competency.
Reporting
● Accurate financial reporting: as embodied by the third year repeat audit findings around the IT general controls, but also the loss of accounts payable and inventory management systems which prevents the closing the financial books at the end of quarter. Even the payroll failure resulted in inaccurate payroll numbers, which led to financial reporting errors.
Compliance
● Compliance with laws and regulations (e.g., SOX-404, PCI DSS): failing the SOX-404 audit results in an adverse footnotes by the external auditor in the SEC 10-K statement, and there are grave implications for failing the PCI DSS assessment.
Operations
● Operations (e.g., IT application and project delivery, IT operations, information security)
● The Project Phoenix was $20 million over-budget and three years late, and when it finally was deployed into production, everyone would have been better off it hadn’t. Furthermore, virtually all of the critical business systems that run daily operations require that IT services be running correctly, which they often weren’t.
Generic problems – Downward Spirals
● Fragile artefacts become more fragile● Technical debt grows● Date-driven application projects focus only on features, sacrificing
non-functional requirements, which results in more fragile artifacts in production
● Application deployment take longer, more difficult, getting gets worse● IT Ops is stuck fire fighting and therefore cannot do preventive work
or new projects● Long feature delivery cycle times result in more political decision
making, meaning more focus on features (vs. non-functional requirements)
Hand out time line
Breakthrough 1 - 3
● Breakthrough 1: discovering too much WIP in IT Operations and gob-smacking amounts of reliance on Brent for both project and recovery work
● Breakthrough 2: elevating preventive work to prevent unplanned work (especially for Brent) and making work visible
● Breakthrough 3: throttling the flow of work into IT Operations by the project freeze (subordinate)
Breakthrough 4 - 6
● Breakthrough 4: identifying and limiting the flows of work to Brent (it was genuinely surprising to find out after the fact at how much this resembles the green/red tag pattern used in “The Goal”) Kanban Board
● Breakthrough 5: documenting and defining standardized work, and managing handoffs to reduce time spent in queue (through use of kanbans), and further reducing reliance on Brent
● Breakthrough 6: correctly identifying reliance on IT systems to hit Dick’s corporate objectives (via Gartner RVM model) and correctly scoping audit and infosec (via GAIT and GAIT-R)
Breakthrough 7 & 8
● Breakthrough 7: building DevOps flow of work to better design in non-functional requirements in Dev, reduce batch sizes and enable single-piece flow through Development and IT Operations and better enable operational resilience (replicating the work described by Jez Humble, David Farley, Paul Hammond and John Allspaw)
● Breakthrough 8: when they discover the constraint moves outside the organization, they bring back those resources in-house (replicating one of my favourite Theory of Constraints plays: when there’s idle plant capacity, it’s often cheaper to fabricate parts in-house than to pay a supplier. Lovely.)
4 types of work
● Business project work, ● IT project work, ● changes and ● unplanned work.
Thee Three Ways
● Way No. 1: The flow● Way No. 2: Consistent feedback● Way No. 3: Continual learningPage 99 – Erik Introduces the concept.
Page 297 – Bill learns how important The Three Ways are.
“But I learned that the practices that Allspaw and Hammond espoused are the inevitable outcome of applying the Three Ways to the IT value stream. It totally changed how we managed IT and it saved our company. “How did they do it?” I ask, dumbfounded.”
Way No. 1: The flow
1. Understand the flow of work. ● Without understanding, any changes will have random
effects.
2. Always increase the flow
3. Never pass on defects downstream
4. Never allow local optimization to cause global degradation
5. Achieve profound understanding of the entire system
Way No 2: Consistent Feedback
● Understand and respond to the needs of all customers, both internal and external.
● Shorten and amplify all feedback loops. – Kim said this rule is modelled after the Toyota Lean
manufacturing line, where any employee can stop the line immediately if they see any defect.
● Create quality at the source. – This means pro-actively building quality into everything. There's
no need for developers to rely on QA to find their errors.
● Create and embed knowledge where we need it.
Way No. 3: Continual learning
● Create a culture that encourages experimentation
● Learns from failure● Recognizes repetition as the prerequisite to
mastery
Thanks
● Thanks Barclays Bank for providing the venue● Thanks to the authors of “The Phoenix Project”● Thanks for your participation
References
● http://itrevolution.com/learn-more-about-concepts-in-phoenix-project/
● http://itrevolution.com/a-personal-reinterpretation-of-the-three-ways/
● Free 170 page excerpt:http://itrevolution.com/the-phoenix-project-excerpt/
●
● http://slideshare.net/realgenekim
● TOCICO http://www.tocico.org
● Dr Holt's TOC courses at Washington State University
– http://public.wsu.edu/~engrmgmt/holt/WSU-TOC-Flyer-Web.htm