![Page 1: Test (and More) Patterns for Continuous Software Delivery](https://reader033.vdocuments.us/reader033/viewer/2022051818/54b6e1534a7959265b8b456b/html5/thumbnails/1.jpg)
�
AW12 Concurrent�Session�11/13/2013�3:45�PM�
�����
"Test (and More) Patterns for Continuous Software Delivery"
���
Presented by:
Andy Singleton Assembla
���������
Brought�to�you�by:��
��
340�Corporate�Way,�Suite�300,�Orange�Park,�FL�32073�888Ͳ268Ͳ8770�ͼ�904Ͳ278Ͳ0524�ͼ�[email protected]�ͼ�www.sqe.com
![Page 2: Test (and More) Patterns for Continuous Software Delivery](https://reader033.vdocuments.us/reader033/viewer/2022051818/54b6e1534a7959265b8b456b/html5/thumbnails/2.jpg)
Andy Singleton Assembla
Andy Singleton is the founder of Assembla, a SaaS company providing tools for distributed agile development teams. Andy moved Assembla from bi-weekly releases to multiple daily releases. His forthcoming book Unblock! A Guide to the New Continuous Agile describes workflows for product management, task management, coding, and testing. Previously, Andy founded PowerSteering Software, a provider of enterprise project management software, and Cambridge Interactive, an eBusiness consultancy. He developed financial information systems for Reuters, Bloomberg, SNL Kagan, and Thomson. During his career, Andy has produced more than twenty major software and financial information products.
![Page 3: Test (and More) Patterns for Continuous Software Delivery](https://reader033.vdocuments.us/reader033/viewer/2022051818/54b6e1534a7959265b8b456b/html5/thumbnails/3.jpg)
Be#er%So(ware%East%From%Andy%Singleton,%h#p://andysingleton.com%
www.assembla.com%%
![Page 4: Test (and More) Patterns for Continuous Software Delivery](https://reader033.vdocuments.us/reader033/viewer/2022051818/54b6e1534a7959265b8b456b/html5/thumbnails/4.jpg)
In%this%session%! IntroducAon%to%CD,%benefits%of%CD,%and%ways%to%pitch%it%! MulAple%code%and%test%pa#erns%to%choose%from%! How%to%get%the%automated%tests%you%need%! Changes%in%the%role%of%QA%%
![Page 5: Test (and More) Patterns for Continuous Software Delivery](https://reader033.vdocuments.us/reader033/viewer/2022051818/54b6e1534a7959265b8b456b/html5/thumbnails/5.jpg)
Program% Launch%
Releases%
Plan% Program% Test% Doc% Deploy%
Skip%Automate%&%Blend% Lag% Automate%
Pull%
CI%&%CD%
End%up%with%
Waterfall%to%ConAnuous%
Measure%
![Page 6: Test (and More) Patterns for Continuous Software Delivery](https://reader033.vdocuments.us/reader033/viewer/2022051818/54b6e1534a7959265b8b456b/html5/thumbnails/6.jpg)
Assembla%in%2011%! “Scrumban”%with%iteraAve%releases,%but%conAnuous%planning%to%accommodate%a%distributed%team.%
! Releases%took%longer%as%system%got%bigger%and%there%was%more%to%test.%2%weeks%V>%3%weeks%
! Bugs%in%producAon.%%2%days%for%fixes.%%Stressful)! CompeAtors%achieved%faster%velocity%with%conAnuous%delivery%
! Made%a%study%of%conAnuous%methods%with%our%own%team,%customers,%and%tools.%
! Now%–%releasing%about%250%Ames%per%month.%%Fewer%bugs.%%Much%less%stress.%
![Page 7: Test (and More) Patterns for Continuous Software Delivery](https://reader033.vdocuments.us/reader033/viewer/2022051818/54b6e1534a7959265b8b456b/html5/thumbnails/7.jpg)
You%can%do%it%incrementally%1. Release%more%frequently%2. Improve%%%%%%%
![Page 8: Test (and More) Patterns for Continuous Software Delivery](https://reader033.vdocuments.us/reader033/viewer/2022051818/54b6e1534a7959265b8b456b/html5/thumbnails/8.jpg)
When%to%use%ConAnuous%Delivery?%! You%provide%an%online%service.%%CompeAAve%pressure%will%force%you%to%conAnuous:%Office%365%vs%Google%Docs.%
! You%provide%any%service%with%so(ware%inside%! Your%release%Ames%are%gedng%longer,%or%the%release%process%is%stressful%
! You%are%developing%a%new%product%with%lean%startup%and%MVP%techniques%
! You%have%a%big%project%with%a%lot%of%contributors%! This%applies%to%hardware,%also%
![Page 9: Test (and More) Patterns for Continuous Software Delivery](https://reader033.vdocuments.us/reader033/viewer/2022051818/54b6e1534a7959265b8b456b/html5/thumbnails/9.jpg)
Benefits%for%the%test%process%! Eliminate%stress%from%the%release%process.%%There%is%never%a%big%batch%of%stuff%to%test%under%pressure.%%
! Add%or%remove%test%layers%to%increase%quality%or%increase%speed%
! ConAnuous%integraAon%removes%the%stress%of%acquiring%and%fidng%components%from%many%teams%and%suppliers%
! Your%CEO%wonders%why%the%compeAAon%is%releasing%faster%and%moving%faster%
%
![Page 10: Test (and More) Patterns for Continuous Software Delivery](https://reader033.vdocuments.us/reader033/viewer/2022051818/54b6e1534a7959265b8b456b/html5/thumbnails/10.jpg)
ConAnuous%Delivery%Dial%
Dev
Business
Ops
DEV TEST UAT PROD
CONTINUOUS DELIVERY
RELEASE TRAIN
Self-Service
Controlled
INTEG STAGING
From%Steve%Brodie%and%Rohit%Jainendran%
![Page 11: Test (and More) Patterns for Continuous Software Delivery](https://reader033.vdocuments.us/reader033/viewer/2022051818/54b6e1534a7959265b8b456b/html5/thumbnails/11.jpg)
The%big%quesAon:%How%to%test?%! We%release%so(ware%in%batches%so%that%we%can%test%it.%%That%is%the%whole%reason%for%doing%it.%%We%test%so(ware%%“release%candidates”%to%make%sure%everything%works%together.%
! In%conAnuous%delivery,%we%might%get%as%li#le%at%10%minutes%to%test%a%release%candidate%
! If%we%can’t%test%the%complete%version,%how%do%we%guarantee%quality?%%Is%it%magic?%
![Page 12: Test (and More) Patterns for Continuous Software Delivery](https://reader033.vdocuments.us/reader033/viewer/2022051818/54b6e1534a7959265b8b456b/html5/thumbnails/12.jpg)
Test%Layering%–%A%Golden%Ticket%Monitor%your%released%so(ware:%Errors,%Usage%volume,%usage%pa#erns,%user%feedback%
QA%System%with%Human%test%consultants%
Code%review:%Both%a%manual%test,%and%a%place%to%ask%for%test%scripts.%
ConAnuous%integraAon:%Run%automated%tests%before%using%human%review%Ame%
Unit%tests%in%the%development%environment%
Switch%new%features%and%architecture%
![Page 13: Test (and More) Patterns for Continuous Software Delivery](https://reader033.vdocuments.us/reader033/viewer/2022051818/54b6e1534a7959265b8b456b/html5/thumbnails/13.jpg)
Go%Both%Ways%
Veloc i ty )
Quality)
Increase)Quality)
Increase)Velocity)
![Page 14: Test (and More) Patterns for Continuous Software Delivery](https://reader033.vdocuments.us/reader033/viewer/2022051818/54b6e1534a7959265b8b456b/html5/thumbnails/14.jpg)
Switch%and%Unveil%
Hidden%Programmer%sees%a%change%locally.%Change%is%tested%in%the%%main%version%but%not%seen.%
Test% Story%Owner%and%testers%see%the%change%on%test%systems.%
Beta%Insiders%see%it%and%use%it.%%Story%Owner%can%show%it%to%users%for%feedback%or%A/B%tesAng.%
UNVEIL!% The%big%event.%%Communicate%with%all%users.%%Measure%reacAon.%
One%code%version%
No%special%test%builds%
No%longVrunning%branches%
![Page 15: Test (and More) Patterns for Continuous Software Delivery](https://reader033.vdocuments.us/reader033/viewer/2022051818/54b6e1534a7959265b8b456b/html5/thumbnails/15.jpg)
![Page 16: Test (and More) Patterns for Continuous Software Delivery](https://reader033.vdocuments.us/reader033/viewer/2022051818/54b6e1534a7959265b8b456b/html5/thumbnails/16.jpg)
Code%and%Test%Pa#erns%! Centralized%
! TradiAonal%process%with%no%branches%finds%and%fixes%problems%as%early%as%possible%
! Distributed%! Release%every%change%with%its%own%branch%and%test%
! Temporary%branches%! Efficient%review,%combines%benefits%of%centralized%and%distributed%
! MAXOS%! Use%centralized%conAnuous%integraAon%to%manage%a%massively%scalable%IT%system%
![Page 17: Test (and More) Patterns for Continuous Software Delivery](https://reader033.vdocuments.us/reader033/viewer/2022051818/54b6e1534a7959265b8b456b/html5/thumbnails/17.jpg)
Centralized%CI/CD%Contributor%Commits%–%“as%early%as%possible”%to%find%problems%
ConAnuous%IntegraAon%tests%
Fail%V%alarm%
Release%Candidate%Test%System% Release%
QA%TesAng%
Pass%
![Page 18: Test (and More) Patterns for Continuous Software Delivery](https://reader033.vdocuments.us/reader033/viewer/2022051818/54b6e1534a7959265b8b456b/html5/thumbnails/18.jpg)
Distributed%ConAnuous%Delivery%Contributor%Commits%
Branch%or%Fork%
Deployed%version%
Peer%review%merge%requests%
Other%contribuAons%merged%and%released%“as%late%as%possible”%
QA%Consults%Pass%Final%Auto%Test?%
Merge%back%Current%
Deploy%
![Page 19: Test (and More) Patterns for Continuous Software Delivery](https://reader033.vdocuments.us/reader033/viewer/2022051818/54b6e1534a7959265b8b456b/html5/thumbnails/19.jpg)
Distributed:%MulAple%Test%Systems%Contributor%1%
Contributor%2%
ProducAon%Revision%
Release%AnyAme%
CI%System%QA%Team%
Test%System%1%
Test%System%2%
![Page 20: Test (and More) Patterns for Continuous Software Delivery](https://reader033.vdocuments.us/reader033/viewer/2022051818/54b6e1534a7959265b8b456b/html5/thumbnails/20.jpg)
Assembla%spins%up%test%servers%
![Page 21: Test (and More) Patterns for Continuous Software Delivery](https://reader033.vdocuments.us/reader033/viewer/2022051818/54b6e1534a7959265b8b456b/html5/thumbnails/21.jpg)
Test%and%Review%on%Temp%Branches%
Contributor%Commits%
Auto%tests%
Mainline%
Review% Auto%tests% Review%
• Github%style%–%Pull%requests%or%merge%requests%going%to%CI%system%• Gerrit%style%–%Temporary%branches%for%each%contribuAon%• Assembla%git%supports%both%–%merge%requests,%and%temp%branches%%created%%
by%a%push%to%a%protected%branch.%
Test%before%spending%human%Ame%on%review%
![Page 22: Test (and More) Patterns for Continuous Software Delivery](https://reader033.vdocuments.us/reader033/viewer/2022051818/54b6e1534a7959265b8b456b/html5/thumbnails/22.jpg)
The%Services%Megatrend%Desktop Web App Cloud Services
App
DB
Service
Service
Service
![Page 23: Test (and More) Patterns for Continuous Software Delivery](https://reader033.vdocuments.us/reader033/viewer/2022051818/54b6e1534a7959265b8b456b/html5/thumbnails/23.jpg)
Matrix%of%Services%V%MAXOS%
PrioriAzed%Backlog%
Current%Work%
Each%team%releases%when%ready%
Service%team%IntegraAon%test%env%
ProducAon%service%
Service%team%IntegraAon%test%env%
ProducAon%service%
Service%team%IntegraAon%test%env%
ProducAon%service%
Feedback%on%speed,%errors,%usage,%and%requests%
Organize%by%tesAng%as%one%system%
![Page 24: Test (and More) Patterns for Continuous Software Delivery](https://reader033.vdocuments.us/reader033/viewer/2022051818/54b6e1534a7959265b8b456b/html5/thumbnails/24.jpg)
Coordinate%without%big%meeAngs%
PrioriAzed%Backlog%
Current%Work%
Service%team%IntegraAon%test%env%
Service%team%IntegraAon%test%env%
Service%team%IntegraAon%test%env%
ConAnuous%IntegraAon%between%%latest%dev%version%of%each%service%
• ConAnuous%integraAon%helps%teams%coordinate.%
• See%dependencies%between%“producers”%and%“consumers”%
• Errors%and%conflicts%show%related%team%contact%info%
• MeeAngs%and%changes%negoAated%between%two%teams,%not%many%
![Page 25: Test (and More) Patterns for Continuous Software Delivery](https://reader033.vdocuments.us/reader033/viewer/2022051818/54b6e1534a7959265b8b456b/html5/thumbnails/25.jpg)
![Page 26: Test (and More) Patterns for Continuous Software Delivery](https://reader033.vdocuments.us/reader033/viewer/2022051818/54b6e1534a7959265b8b456b/html5/thumbnails/26.jpg)
Role:%Developer%! Developers%have%more%power%and%responsibility.%! Developers%have%more%responsibility%for%tesAng.%! Developers%(not%QA%or%PM)%decide%when%to%release.%This)is)a)strong)finding.%! IncenAves%are%correct.%%Developer%might%have%to%come%back%from%Friday%night%beers%to%fix%a%problem.%%This%provides%a%moAvaAon%to%make%good%decisions%and%automate%tesAng.%
! Features%can%be%released%but%hidden.%%Product%Managers%and%Marketers%will%unveil%when%they%are%ready.%Unblock!%
![Page 27: Test (and More) Patterns for Continuous Software Delivery](https://reader033.vdocuments.us/reader033/viewer/2022051818/54b6e1534a7959265b8b456b/html5/thumbnails/27.jpg)
Role:%QA%! QA%is%a%consultant.%! QA%gets%more%respect.%%Developers%have%to%ASK%for%service.%
! Developers%do%more%of%the%tesAng%work.%%They%should%organize%reviews%and%automated%tests%so%that%bugs%don’t%go%through%into%the%manual%test%process.%
! QA%has%more%Ame%to%invesAgate%usability%! QA%monitors%producAvity%and%quality%metrics%
![Page 28: Test (and More) Patterns for Continuous Software Delivery](https://reader033.vdocuments.us/reader033/viewer/2022051818/54b6e1534a7959265b8b456b/html5/thumbnails/28.jpg)
IncenAves%for%ConAnuous%Flow%You%don’t%need%culture%change.%%You%only%need%to%release%more%frequently%1. If%you%do%code)review,%you%can%get%automated.%No%
browbeaAng%and%cajoling%is%required.%%Developers%will%ask%for%tests%when%they%review.%
2. If%developers%decide%to%release,%they%will%take%more%responsibility%for%tesAng%and%automated%tests.%
3. If%you%release%more%frequently,%developers%will%quickly%learn%not%to%break%the%build%
4. If%PM’s%take%unveil)and)measurement%responsibility,%they%will%make%be#er%features%
![Page 29: Test (and More) Patterns for Continuous Software Delivery](https://reader033.vdocuments.us/reader033/viewer/2022051818/54b6e1534a7959265b8b456b/html5/thumbnails/29.jpg)
Our%Master%Plan%1. Release%more%frequently%2. Improve%%%%%%%
![Page 30: Test (and More) Patterns for Continuous Software Delivery](https://reader033.vdocuments.us/reader033/viewer/2022051818/54b6e1534a7959265b8b456b/html5/thumbnails/30.jpg)
www.conAnuousagile.com/unblock%