agile & devops : gaming - hatech devops services, a ... · “the devops movement addresses the...
TRANSCRIPT
ENABLING ORGANIZATIONALAGILITY
ORGAGILITY
Agile & DevOps : Gaming
Copyright © 2012-2017, OrgAgility. All Rights Reserved.
QUESTIONQuick poll
How many people here are familiar with and have used agile techniques at work?How many of you have worked in companies or on projects that tried agile and failed?
1Copyright © 2012-2017, OrgAgility. All Rights Reserved.
Alex SinghEmail: [email protected]: http://www.orgagility.comBlog: http://orgagility.wordpress.comTel: +1 949-667-1008Tel: +1 646-833-8001
2
Jon HathawayEmail: [email protected]: http://hatech.ioBlog: http://hatech.ioTel: +1 702 875 2330Tel: +1 702 389 8160
3JH
ENABLING ORGANIZATIONALAGILITY
ORGAGILITY
What We DoTransform your business—operations, product and development, working together as one integrated agile team—and drive towards purpose via continuous innovation, discovery, delivery, learning and improvement.
ENABLING ORGANIZATIONALAGILITY
ORGAGILITY
What We BelieveWe are committed to creating environments where people can experience joy, self-expression, partnership, and satisfaction arising from the completion of challenging and meaningful work.
Agenda
• Lean-Agile guiding principles• Agile journey at a gaming company• Q&A
Copyright © 2012-2017, OrgAgility. All Rights Reserved. 6
Positive results, but …Industry Week/MPI Census of Manufacturers, 2008
• Fully achieved their objectives: 2%
• Significant result: 24%
• Not making good progress: 74%
Ken Schwaber, 02/2008
• “I estimate that 75% of organizations using Scrum will not succeed in getting the benefits that they hope for from it.”
8Copyright © 2012-2017, OrgAgility. All Rights Reserved.
Top 5 reasons for agile adoption struggles
• Lack of leadership support• Organization culture at
odds with lean-agile values
• Lack of systems thinking• Ignoring people
development• Cargo cult agile
9Copyright © 2012-2017, OrgAgility. All Rights Reserved.
Simplicity is the ultimate sophistication.
10Copyright © 2012-2017, OrgAgility. All Rights Reserved.
Leonardo Da Vinci
AgileCollection of practicesfocused on iteratively producing functionality
Copyright © 2012-2017, OrgAgility. All Rights Reserved. 11
A framework within which people can address complex adaptive problems, while productively and creatively delivering products of the highest possible value
Agility… predictably and
frequently delivering value increments to
customers
… that enables …
Image Source: Westminster 2017 12
Copyright © 2012-2017, OrgAgility. All Rights Reserved. 13
Agile needn’t be complicated
It’s not about the practices, the charts and the graphs, but about the smiles!
Outcomes
Clarity of Work
Incremental & Iterative Delivery
of Value & Results
Engaged Customer
Accountable Teams
Ship and Reflect —Responsive & Able
to Reorient Quickly
Copyright © 2012-2017, OrgAgility. All Rights Reserved. 14
Agile needn’t be complicated
Intake
1st Guiding Principle: Value
15Copyright © 2012-2017, OrgAgility. All Rights Reserved.
Increase Value
Deliver value early and often
Make it easier, better, faster, cheaper for customers
Engage employees to: • Increase Safety• Increase Morale• Reduce Cost• Improve Productivity
2nd Guiding Principle: Flow
16Copyright © 2012-2017, OrgAgility. All Rights Reserved.
Improve Flow
Optimize end-to-end flow for faster delivery
Minimize delays, queues, non-value add overhead
Tactics: • Visualize & limit WIP• Pull work• Reduce batch size and
manage queue lengths
3rd Guiding Principle: Quality
17Copyright © 2012-2017, OrgAgility. All Rights Reserved.
Enhance QualityDiscover
quality through fast feedback
Create useful, usable, desirable products
Tactics: • Focus (reduce WIP)• Accelerate feedback• Collaborate• Prevent defects (rework)
Lean-Agile Transformation Objectives
Visibility
Predictability
Time to Market
Value / Outcome Driven
Enterprise Agility
Become proficient at one step before ascending to the next
18Copyright © 2012-2017, OrgAgility. All Rights Reserved.
Different goals at different times in your transformation
Your goal is not to do a set of practices, but to improve the way you do what you do.
Alex Singh
PROCESS IMPROVEMENTS DRIVE OUT WASTE AND INCREASE EFFECTIVENESSImprove your process
19Copyright © 2012-2017, OrgAgility. All Rights Reserved.
Practices are designed to solve a problem in a context. When challenges occur, look at the intention of the practice. Then, using principles, look for an alternative to the practice.Finally, decide what to do next, but define the expected outcomes before starting the experiment (or implementing the countermeasure)
Alex Singh & Al Shalloway
BUT FIRST GRASP THE PURPOSE OF THE PRACTICEPractices Shouldn’t be Inviolable!
20Copyright © 2012-2017, OrgAgility. All Rights Reserved.
Success is the ability to …
Continue improvements independently:• Holistically see problems and inefficiencies • Determine root causes and fix them
permanently• Share knowledge, ideas, practices laterally
Copyright © 2012-2017, OrgAgility. All Rights Reserved. 21
(Platform) At the start …
Goal: Use the cloud to allow remote monitoring of, and deployment to, slot machines on a casino floor.
Copyright © 2012-2017, OrgAgility. All Rights Reserved. 23JH
Where’s the beef?
Investment in architecture must provide some benefit in the short to medium term
for buy-in and continued funding.Image source: https://xkcd.com/974/
JH
Top Challenges
1. Non-collaborative culture
2. Poor technical leadership
3. Lack of “production” mindset
25Copyright © 2012-2017, OrgAgility. All Rights Reserved. JH
(Platform) At the start …
• 18-months and no value• No end in sight
Copyright © 2012-2017, OrgAgility. All Rights Reserved. 26JH
I often say that when you can measure what you are speaking about, and express it in numbers, you know something about it; but when you cannot measure it, when you cannot express it in numbers, your knowledge is of a meager and unsatisfactory kind.
William Thomson (Lord Kelvin)
IF YOU CAN NOT MEASURE IT, YOU CAN NOT IMPROVE ITTo measure is to know
27Copyright © 2012-2017, OrgAgility. All Rights Reserved.
(Platform) Summary …
• 3 teams in 3 cities — low trust• Wrong people in wrong roles• No Operations involvement• No DevOps team or mentality• No representation from Legal
and Regulators
Copyright © 2012-2017, OrgAgility. All Rights Reserved. 28JH
Step 1: Visibility (Platform)
Deliver one sliver of functionality cleanly end-to-end to learn about:• Process — activities, flow, and
blockages• Teamwork — structure and
swarming• Intra-team collaboration
Copyright © 2012-2017, OrgAgility. All Rights Reserved. 29JH
Step 2: Requirement Clarity (Platform)
• Agile users stories — defined outcomes and tasks for all work
• Relative story point estimation• Single master backlog
– Component teams implement related features in concert
Copyright © 2012-2017, OrgAgility. All Rights Reserved. 30JH
What is a Story?Describes functionality valuable to a stakeholder of a system or product.
A short description of what the product should do, emphasizing user goals(not system attributes).
As an <user type>I want to <do something>So that I can <accomplish something of value>
I’ll know this is done when• A• B• C
Copyright © 2012-2017, OrgAgility. All Rights Reserved. 31
Step 3: Build Trust (Platform)
• Work together in time boxes– Short-term goals—build vertical
slices of same functionality– Cross-city pairing– Attend each others’ planning
and review sessions
• Zero defects acceptedCopyright © 2012-2017, OrgAgility. All Rights Reserved. 32
The Other Problem
IT, integration engineering, systems engineering, systems support, operations:
• Overlapping interests• No clearly defined ownership
35Copyright © 2012-2017, OrgAgility. All Rights Reserved.
Typical Engineering Skillset distribution
36
0
1
2
3
4
5
6
7
8
9Open Source Tools
Open Source ProgrammingCoding
Scripting
Test Driven Development
Unit Testing
Automation Tools
Data Management Skills
Business Focused
Measurement and Metrics
Continual Integration
Continual Deployment
Continual Delivery
Backup/RestoreDisaster Recovery
Failover ClusteringSystems HA
Systems Scaling
Performance Tuning
Hardware Management
Hypervisor Management
Systems Monitoring
Application Monitoring
Systems Recovery
Traffic Management
DDOS Mitigation
Orchestration
Bare Metal Deployment
Change ControlRelease Management
Engineering
PrimaryFocus:DrivechangeDriveinnovation
Copyright © 2012-2017, OrgAgility. All Rights Reserved. JHJH
Typical Operations Skillset distribution
37
0
1
2
3
4
5
6
7
8
9Open Source Tools
Open Source ProgrammingCoding
Scripting
Test Driven Development
Unit Testing
Automation Tools
Data Management Skills
Business Focused
Measurement and Metrics
Continual Integration
Continual Deployment
Continual Delivery
Backup/RestoreDisaster Recovery
Failover ClusteringSystems HA
Systems Scaling
Performance Tuning
Hardware Management
Hypervisor Management
Systems Monitoring
Application Monitoring
Systems Recovery
Traffic Management
DDOS Mitigation
Orchestration
Bare Metal Deployment
Change ControlRelease Management
Operations
PrimaryFocus:ProvidestabilityProvidereliability
Copyright © 2012-2017, OrgAgility. All Rights Reserved. JH
Typical Support Skillset distribution
38
0
1
2
3
4
5
6
7
8
9Open Source Tools
Open Source ProgrammingCoding
Scripting
Test Driven Development
Unit Testing
Automation Tools
Data Management Skills
Business Focused
Measurement and Metrics
Continual Integration
Continual Deployment
Continual Delivery
Backup/RestoreDisaster Recovery
Failover ClusteringSystems HA
Systems Scaling
Performance Tuning
Hardware Management
Hypervisor Management
Systems Monitoring
Application Monitoring
Systems Recovery
Traffic Management
DDOS Mitigation
Orchestration
Bare Metal Deployment
Change ControlRelease Management
Systems Support
PrimaryFocus:ProvideconsistencyProvideconfidence
Copyright © 2012-2017, OrgAgility. All Rights Reserved. JH
DevOps — Multidiscipline Skillset
39
0
1
2
3
4
5
6
7
8
9Open Source Tools
Open Source ProgrammingCoding
Scripting
Test Driven Development
Unit Testing
Automation Tools
Data Management Skills
Business Focused
Measurement and Metrics
Continual Integration
Continual Deployment
Continual Delivery
Backup/RestoreDisaster Recovery
Failover ClusteringSystems HA
Systems Scaling
Performance Tuning
Hardware Management
Hypervisor Management
Systems Monitoring
Application Monitoring
Systems Recovery
Traffic Management
DDOS Mitigation
Orchestration
Bare Metal Deployment
Change ControlRelease Management
DevOps
PrimaryFocus:CreaterevenueMeetexpectations
Copyright © 2012-2017, OrgAgility. All Rights Reserved. JH
The Other Problem
40Copyright © 2012-2017, OrgAgility. All Rights Reserved.
DEV OPS
Production Releases
Change Requests
The Other Problem
• Production releases considered risky• Strained collaboration between
development and operations• Development not appreciating the
realities of operations• Operations not understanding the new
development strategies
41Copyright © 2012-2017, OrgAgility. All Rights Reserved.
What we wanted …
1. Streamline the end-to-end flow2. Reduce the feedback cycle3. Flexible people4. Multidisciplinary “generalizing specialists”5. Standardized infrastructure6. Automation and tools7. Standardized development guidelines8. You build it, you run it
42Copyright © 2012-2017, OrgAgility. All Rights Reserved.
Step 4: Build a DevOps Culture
43Copyright © 2012-2017, OrgAgility. All Rights Reserved.
DEV OPS
Extend delivery to production
Timely operations feedback to teams
Embed project knowledge into Operations
Embed Operations knowledge into projects
“The DevOps movement addresses the dysfunction that results from organizations composed of functional silos. Thus, creating another functional silo that sits between Development and Operations is clearly a poor (and ironic) way to try and solve these problems. DevOpsproposes instead strategies to create better collaboration between functional silos, or doing away with the functional silos altogether and creating cross-functional teams (or some combination of these approaches).”
44Source: http://continuousdelivery.com/2012/10/theres-no-such-thing-as-a-devops-team/
Jez Humble
Step 4: Build a DevOps Culture
Goals:
• Automate deployment – existing Windows applications
• Repeatability, supportability, reliability
Copyright © 2012-2017, OrgAgility. All Rights Reserved. 45
Step 4: Build a DevOps Culture
Business Quality
IT Operations Development
CALMS
46Copyright © 2012-2017, OrgAgility. All Rights Reserved.
Step 4: Build a DevOps Culture
• Culture
• Automation
• Lean
• Measurements
• SharingJohn Willis and Damon Edwards 47
Step 4: Build a DevOps Culture
Goals:
• Improve other teams • not become another silo!
• Foster deep communication
Copyright © 2012-2017, OrgAgility. All Rights Reserved. 48
Step 4: Build a DevOps Culture
• Sit together + AV bridge
• Attitude not just skill
• AmbassadorsCopyright © 2012-2017, OrgAgility. All Rights Reserved. 49
Step 5: DevOps Requirements
• Agile users stories• Story point estimates• Add functionality iteratively–Start with a basic blueprint–Augment & build on functionality
50Copyright © 2012-2017, OrgAgility. All Rights Reserved.
Step 5: DevOps Requirements
• Monthly goals• Next most important 2-3 blueprints• Complete blueprints in a week
(longer ones were rare but OK)
51Copyright © 2012-2017, OrgAgility. All Rights Reserved.
Step 6: DevOps Teamwork
56Image Source: http://andon2013.blogspot.com/2013/11/conclusiones.html
Swarm on blueprints (i.e., work)Face-to-face discussions
Step 6: DevOps Teamwork
57Copyright © 2012-2017, OrgAgility. All Rights Reserved.
• Partnership on the goals
• Group accountabilityIT, integration engineering, systems engineering, systems support, operations
Step 6: DevOps Teamwork
• Quick daily stand-up• Knowledge sharing– Blogs– Technical demos
• Improvement backlog
58Copyright © 2012-2017, OrgAgility. All Rights Reserved.
Step 6: DevOps Teamwork
• Never go home angry• Have fun– Poker daily– Shooting range frequently
• Snacks, donuts, weekly team lunch
59Copyright © 2012-2017, OrgAgility. All Rights Reserved.
Step 6: DevOps Teamwork
60Image Source: http://andon2013.blogspot.com/2013/11/conclusiones.html
Problem Swarming:• Abnormality• Visualize problem• Positive & timely
response• Joint PDCA• Back to standard
Step 7: DevOps Holistic View
• Min and Max limits on work process steps– Lowered gradually to increase flow and teamwork– No feast, no famine
• Step out of your comfort zone• Stories can exit process step only when exit
criteria are met (done)• Follow the work sequencing guidance
61Copyright © 2012-2017, OrgAgility. All Rights Reserved.
DevOps Work Priority Guidance
1. Fix an unhealthy increment (broken build, recently introduced defects, etc.)
2. Help remove a flag (Block)3. Help an existing swarm move their work forward4. Refill a queue (if low before the bottleneck) but
don’t exceed the WIP Limit5. Start a new item. However, if all at limits, join an
existing swarm
Copyright © 2012-2017, OrgAgility. All Rights Reserved. 62
Step 8: Integrate Platform + DevOps
Goal: end-to-end development and delivery• Developed, locally tested, integrated,
validated, and deployed• Platform stories and DevOps blueprints in
concert• Platform team representation on DevOps
63Copyright © 2012-2017, OrgAgility. All Rights Reserved.
Step 8: Integrate Platform + DevOps
• Feedback from business on completion of each platform story and DevOps blueprint
• Friday demos — show everything worked on that week– Team only first; stakeholders after 3 weeks,
real customers in attendance after 2-months– Tie everything back to a business goal — tell a
story
64Copyright © 2012-2017, OrgAgility. All Rights Reserved.
Results
65Copyright © 2012-2017, OrgAgility. All Rights Reserved.
0
1
2
3
4
5Identical Environments
Repeatability
Application Monitoring
Fabric Monitoring
Application Logging
Fabric Logging
Code Documentation
API Documentation
Automated Functional Test
ProvisioningDeployment
Min Downtime
HA
Release Management
Dependency Management
Auto Build
Versioning
Source Control
Branching Strategy
Load Test
Continuous Delivery Roadmap - Platform
JH
Results
• Reduced time to market• Reduced cycle time• Higher customer satisfaction (via NPS)• Reduced mean time to restore service
(MTTRS)• Increased “sharing”• Increased morale
Copyright © 2012-2017, OrgAgility. All Rights Reserved. 66
Agileisn’t…….
• Aboutthe tools youuse
• Aboutthecloud youdeployinto
• Aboutpleasingmanagement
• Aboutdoingsomethingcool
JH
Agileis……Simply– Agile isn’ttheGoal:BetteristheGoal!
GettingsomethingintoProduction:• Thatisvaluable• Thatishighquality• Thatisreliable• Thatismaintainable• Hasfewer bugs
JH
ToolsvsMethodsvsProcesses
Adoptingtoolswill:- Showyouwhereyouarebroken,faster- Automatebad processes- Forces youtoworkinawaythetool ‘wants’to
Toolsshouldbeadoptedwhen:- Youhavea‘good’ processthatneedstobespedup- Youwantarepeatable ‘good’process- Youwanttofind outwherethingsarebroken,faster
JH
SoWheredoesAWSfit?
BuiltforScale– nowaitingfor‘environments’
Automation – consistencyoverspeed
Pay-as-you-go – ReducedCapex,speedofsetup
JH
SpeedinguptheFeedbackLoop
AWSCodePipeline – CIWorkflow
AWSCodeBuild – compile,build,text
AWSCodeDeploy – automateddeployment
AWSCodeCommit – ManagedGITrepositories
JH
Compliance?
AWSEC2SystemsManager– track,audit,patch
AWSConfig – RulesbasedVerificationandNotify
AWS CloudTrail – APIActivityUsageandTracking
JH
Is this item:• The most valuable
thing to be working on?
• Going to solve a real customer need?
• Necessary? Necessary now?
• Desirable, valuable, feasible, viable?• How do we know
that?• Going to help us learn
and improve?
Will this action:• Help us finish and deliver
stories faster? How so?• Enable smoother
uninterrupted flow of value?• Improve how
customer/supplier pairs function?
• Simplify our process?• Help us inspect and adapt
sooner?• Improve flow but sacrifice
focus on value delivery?• Eliminate waste but impede
flow?
Are we:• Negotiating features
instead of quality?• Sacrificing quality for
expediency?• Exposing wastes,
inefficiencies, weaknesses, quickly?
• Solving the real problem and tackling the root cause(s)?
• Accelerating feedback?• Preventing the
introduction of defects
Guiding Questions