continuous delivery of a cloud deployment at a large telecommunications provider
TRANSCRIPT
© 2013 IBM Corporation
Continuous Delivery of a Cloud Deployment at a Large Telecommunications ProviderM Kevin McHugh @mkmchu
Eduardo Patrocinio @patrocinio
1717 – February 25, 2014
1
Agenda
Introduction
Continuous Delivery – what is it, how did we do it
Wrap up
Questions
2
Agenda
Introduction
Continuous Delivery – what is it, how did we do it
Wrap up
Questions
3
Solution Architecture
4
Agenda
Introduction
Continuous Delivery – what is it, how did we do it
Wrap up
Questions
5
Definition
Continuous Delivery (CD) is a design practice used in software development to
automate and improve the process of software delivery. Techniques such as automated testing, continuous integration and continuous deployment allow software to be
developed to a high standard and easily packaged and deployed to test environments, resulting in the ability to rapidly, reliably and repeatedly push out enhancements and bug fixes to customers at low risk and with minimal manual overhead. The technique was one of the assumptions of extreme programming but at an enterprise level has developed into a discipline of its own, with job descriptions for roles such as "buildmaster" calling for CD skills as mandatory.
(Source: Wikipedia 1/2014 - http://en.wikipedia.org/wiki/Continuous_delivery)
The ability to build a releasable asset with nearly trivial effort
What about Continuous Deployment?
6
Components
Method
Continuous Integration
Process
Assets
Automated Test
Automated Build of deployable asset
Tools
7
Method: Agile
7
Team Member
Team Member
Customer
Team Member
Team Lead
Agile
�Value-driven lifecycle with regular production of working software
�Small, co-located team developing straightforward software
�Goal is to develop a high-quality system in an evolutionary, collaborative, and self-organizing manner
�Pre-requisite: can build a product backlog
Agile
Scrum Roles (pigs)
�Product Owner
– Owns the product backlog
– Chooses what work to do next
�Team Member
– Creates the solution
– Owns estimation of difficulty of items in the backlog
– Constrains how much work can be done
�Team Lead (Scrum Master)
– Removes obstacles
– Status reporting
8
Agile terms and how the team operates
Backlog Grooming
Sprint Planning
Scrums
Velocity
Sprint Demo
Sprint Retrospective
Sample User Stories and point assignments:
8
Story
Points Summary
13 pts As a Bank Customer, I want to apply for a loan, so that I can…
8 pts As a Bank Customer, I want to deposit cash
5 pts As a Bank Customer, I want to change my PIN.
3 pts As a Bank Customer, I want to withdraw cash
1 pt As a Bank Customer, I check my account balance
5 pts As a Bank Customer, I want to buy postage stamps
AgileMethod: Tenants of Agile and its iterative cycle
Sprint 1 Sprint 2
Story Points DeliveredStory Points Delivered Story Points DeliveredStory Points Delivered
Velocity (rolling average of points delivered)Velocity (rolling average of points delivered)
9
Agile adoption – adoption by execution
Two-week sprints
Single Product Owner
Scrum Master
Rank user stories
Set up the meetings
Method: What we did Agile
Sprint 1 Sprint 2
Story Points DeliveredStory Points Delivered Story Points DeliveredStory Points Delivered
Velocity (rolling average of points delivered)Velocity (rolling average of points delivered)
1010
Sprint by Sprint view of demonstrated progress of feature delivery
Agile
Release 1.0 - Functional Story Point Burndown
0
50
100
150
200
250
300
350
400
450
Sprin
t 1Spr
int 2
Sprin
t 3Spr
int 4
Sprin
t 5Spr
int 6
Sprin
t 7Spr
int 8
Time
Sto
ry P
oin
ts
Backlog Size
Remaining Backlog
Sprint Velocity
Projected Pts
Planned Pts
Method: Sample Burndown – Measured delivery
11
Continuous Integration
Continuous integration (CI) is the practice, in software engineering, of merging all developer working copies with a shared mainline several times a day.
(Source: Wikipedia 1/29/2014 - http://en.wikipedia.org/wiki/Continuous_integration)
Develop features or fix defects on isolated branches
Merge branches onto the main
Identify points in time on the main that represent a viable code drop
This all drives the ability to have a stable, automatically tested, “main” which can be sent forward for Test, QA, and Production deployment
12
Continuous Integration – what we did
Initially
Work space isolation obtained via Business Process Manager (BPM) locking of a business process
The team is working on the main
This limited / prevented concurrent engineering
Led to Sprint-fall
Evolved into
Paired programmers working on separate tool kits, and backend processes
13
Process: 30,000 ft view
Development Test QA Production
Main
Feature 455
Unit Test
Asset Mgmt deploy deploy deploy
Build
Customer E2E Automated Test
- REST API
Customer Manual Test
- UAT
Automated Test
IBM Automated Test
- REST API
- UI
IBM Automated Test
- REST API
- UI
Customer Manual Test
- UAT
14
Assets
Assets include any element that the team can change on any of the systems supporting the project. For example:
A WAR file
An executable
An environment variable
A database definition
15
Assets – what we did
BPM process application and tool kitTWX files
PatternThe topology of an application
PatchThose changes to SCO software and/or associated configuration changes
ConfigurationThose changes to the systems supporting the SCO product
Configuration of the VMs internally and from a VM mgmt standpoint
ImagesThe building block for a pattern
16
Rational Team Concert
Assets – what we did
Development
Exported as .twxfile, then checked
into RTC along with a change
description
Import .twx from RTC
QATest Production
17
Automated Test
Automated test provides the means to assess the quality of the main branch or the deployed release
Through the use of Automated Test, the Continuous Delivery framework can then decide of a particular release candidate is of sufficient quality to be promoted to the next level
18
Automated Test – what we did
Test Architecture
Automated REST API Test
Deliver a solution that can be accessed via REST API
Verify that, through the REST API, all features work
Automated User Interface (UI) Test
Confirm that the solution works with the developed user interface
19
Automated Build of Deployable Asset
Automated build provides the means to collect the set of assets from the development system such that they can later be deployed to a target system
Automated deploy provides the means to place the assets onto a target system for test and use
20
Automated Build of Deployable Asset – what we did
Use SmartCloud Orchestrator (SCO), REST based export/import feature
Implemented a persistence layer to encapsulate differences between deployment environments
Leverage build and version control tools
Control asset versions
Initiate automated testing
Track test results
21
Tools to integrate and support
Integrate SCM and Build management tools toSpeed the development
Deliver to the customer control systems
Automated TestTest the User Interface – record and replay UI tool
Test the REST Interface – test harness to invoke API, record results
Automated BuildBPM – REST API based extract from SCO
Patches, etc handled more conventionally
Automated DeployBPM – REST API based import to SCO
Patches, etc handled more conventionally
22
Tools to integrate and support – what we did
23
Tools to integrate and support – what we did
24
Agenda
Introduction
Continuous Delivery – what is it, how did we do it
Wrap up
Questions
25
Components
Method
Continuous Integration
Process
Assets
Automated Test
Automated Build of deployable asset
Tools
26
…In Retrospect
Defect zone vs. Story zone (defect backlog vs. story backlog)
Team commitment to Agile and CD – No boycotts
Sprint granularity of CD vs. Story level of CD
Deliver both Product and Services
Difficulty grasping task ownership and delivery
27
Questions
28
Thank you!!