test (and more) patterns for continuous software delivery
DESCRIPTION
Top web companies employ continuous delivery of software to build and deploy systems faster and gain a marked competitive advantage. You can do it, too! Andy Singleton shares the patterns for testing in real time that result in more frequent and more reliable releases. He explains why you will have to invest seriously in automated tests and shares experiences developing the most time-efficient types of automated tests, setting up a social structure to get the tests you need, and employing existing layers of testing and production monitoring. Then, Andy goes on to describe the strategies and techniques within the development/delivery process that enable continuous delivery: developer code review workflows; specific ways to use Git, Perforce, and Subversion for version control; continuous integration approaches; and more. Join a discussion on the ways continuous delivery changes the roles of developers and testers and accelerates the value you deliver to your business.TRANSCRIPT
�
AW12 Concurrent�Session�11/13/2013�3:45�PM�
�����
"Test (and More) Patterns for Continuous Software Delivery"
���
Presented by:
Andy Singleton Assembla
���������
Brought�to�you�by:��
��
340�Corporate�Way,�Suite�300,�Orange�Park,�FL�32073�888Ͳ268Ͳ8770�ͼ�904Ͳ278Ͳ0524�ͼ�[email protected]�ͼ�www.sqe.com
Andy Singleton Assembla
Andy Singleton is the founder of Assembla, a SaaS company providing tools for distributed agile development teams. Andy moved Assembla from bi-weekly releases to multiple daily releases. His forthcoming book Unblock! A Guide to the New Continuous Agile describes workflows for product management, task management, coding, and testing. Previously, Andy founded PowerSteering Software, a provider of enterprise project management software, and Cambridge Interactive, an eBusiness consultancy. He developed financial information systems for Reuters, Bloomberg, SNL Kagan, and Thomson. During his career, Andy has produced more than twenty major software and financial information products.
Be#er%So(ware%East%From%Andy%Singleton,%h#p://andysingleton.com%
www.assembla.com%%
In%this%session%! IntroducAon%to%CD,%benefits%of%CD,%and%ways%to%pitch%it%! MulAple%code%and%test%pa#erns%to%choose%from%! How%to%get%the%automated%tests%you%need%! Changes%in%the%role%of%QA%%
Program% Launch%
Releases%
Plan% Program% Test% Doc% Deploy%
Skip%Automate%&%Blend% Lag% Automate%
Pull%
CI%&%CD%
End%up%with%
Waterfall%to%ConAnuous%
Measure%
Assembla%in%2011%! “Scrumban”%with%iteraAve%releases,%but%conAnuous%planning%to%accommodate%a%distributed%team.%
! Releases%took%longer%as%system%got%bigger%and%there%was%more%to%test.%2%weeks%V>%3%weeks%
! Bugs%in%producAon.%%2%days%for%fixes.%%Stressful)! CompeAtors%achieved%faster%velocity%with%conAnuous%delivery%
! Made%a%study%of%conAnuous%methods%with%our%own%team,%customers,%and%tools.%
! Now%–%releasing%about%250%Ames%per%month.%%Fewer%bugs.%%Much%less%stress.%
You%can%do%it%incrementally%1. Release%more%frequently%2. Improve%%%%%%%
When%to%use%ConAnuous%Delivery?%! You%provide%an%online%service.%%CompeAAve%pressure%will%force%you%to%conAnuous:%Office%365%vs%Google%Docs.%
! You%provide%any%service%with%so(ware%inside%! Your%release%Ames%are%gedng%longer,%or%the%release%process%is%stressful%
! You%are%developing%a%new%product%with%lean%startup%and%MVP%techniques%
! You%have%a%big%project%with%a%lot%of%contributors%! This%applies%to%hardware,%also%
Benefits%for%the%test%process%! Eliminate%stress%from%the%release%process.%%There%is%never%a%big%batch%of%stuff%to%test%under%pressure.%%
! Add%or%remove%test%layers%to%increase%quality%or%increase%speed%
! ConAnuous%integraAon%removes%the%stress%of%acquiring%and%fidng%components%from%many%teams%and%suppliers%
! Your%CEO%wonders%why%the%compeAAon%is%releasing%faster%and%moving%faster%
%
ConAnuous%Delivery%Dial%
Dev
Business
Ops
DEV TEST UAT PROD
CONTINUOUS DELIVERY
RELEASE TRAIN
Self-Service
Controlled
INTEG STAGING
From%Steve%Brodie%and%Rohit%Jainendran%
The%big%quesAon:%How%to%test?%! We%release%so(ware%in%batches%so%that%we%can%test%it.%%That%is%the%whole%reason%for%doing%it.%%We%test%so(ware%%“release%candidates”%to%make%sure%everything%works%together.%
! In%conAnuous%delivery,%we%might%get%as%li#le%at%10%minutes%to%test%a%release%candidate%
! If%we%can’t%test%the%complete%version,%how%do%we%guarantee%quality?%%Is%it%magic?%
Test%Layering%–%A%Golden%Ticket%Monitor%your%released%so(ware:%Errors,%Usage%volume,%usage%pa#erns,%user%feedback%
QA%System%with%Human%test%consultants%
Code%review:%Both%a%manual%test,%and%a%place%to%ask%for%test%scripts.%
ConAnuous%integraAon:%Run%automated%tests%before%using%human%review%Ame%
Unit%tests%in%the%development%environment%
Switch%new%features%and%architecture%
Go%Both%Ways%
Veloc i ty )
Quality)
Increase)Quality)
Increase)Velocity)
Switch%and%Unveil%
Hidden%Programmer%sees%a%change%locally.%Change%is%tested%in%the%%main%version%but%not%seen.%
Test% Story%Owner%and%testers%see%the%change%on%test%systems.%
Beta%Insiders%see%it%and%use%it.%%Story%Owner%can%show%it%to%users%for%feedback%or%A/B%tesAng.%
UNVEIL!% The%big%event.%%Communicate%with%all%users.%%Measure%reacAon.%
One%code%version%
No%special%test%builds%
No%longVrunning%branches%
Code%and%Test%Pa#erns%! Centralized%
! TradiAonal%process%with%no%branches%finds%and%fixes%problems%as%early%as%possible%
! Distributed%! Release%every%change%with%its%own%branch%and%test%
! Temporary%branches%! Efficient%review,%combines%benefits%of%centralized%and%distributed%
! MAXOS%! Use%centralized%conAnuous%integraAon%to%manage%a%massively%scalable%IT%system%
Centralized%CI/CD%Contributor%Commits%–%“as%early%as%possible”%to%find%problems%
ConAnuous%IntegraAon%tests%
Fail%V%alarm%
Release%Candidate%Test%System% Release%
QA%TesAng%
Pass%
Distributed%ConAnuous%Delivery%Contributor%Commits%
Branch%or%Fork%
Deployed%version%
Peer%review%merge%requests%
Other%contribuAons%merged%and%released%“as%late%as%possible”%
QA%Consults%Pass%Final%Auto%Test?%
Merge%back%Current%
Deploy%
Distributed:%MulAple%Test%Systems%Contributor%1%
Contributor%2%
ProducAon%Revision%
Release%AnyAme%
CI%System%QA%Team%
Test%System%1%
Test%System%2%
Assembla%spins%up%test%servers%
Test%and%Review%on%Temp%Branches%
Contributor%Commits%
Auto%tests%
Mainline%
Review% Auto%tests% Review%
• Github%style%–%Pull%requests%or%merge%requests%going%to%CI%system%• Gerrit%style%–%Temporary%branches%for%each%contribuAon%• Assembla%git%supports%both%–%merge%requests,%and%temp%branches%%created%%
by%a%push%to%a%protected%branch.%
Test%before%spending%human%Ame%on%review%
The%Services%Megatrend%Desktop Web App Cloud Services
App
DB
Service
Service
Service
Matrix%of%Services%V%MAXOS%
PrioriAzed%Backlog%
Current%Work%
Each%team%releases%when%ready%
Service%team%IntegraAon%test%env%
ProducAon%service%
Service%team%IntegraAon%test%env%
ProducAon%service%
Service%team%IntegraAon%test%env%
ProducAon%service%
Feedback%on%speed,%errors,%usage,%and%requests%
Organize%by%tesAng%as%one%system%
Coordinate%without%big%meeAngs%
PrioriAzed%Backlog%
Current%Work%
Service%team%IntegraAon%test%env%
Service%team%IntegraAon%test%env%
Service%team%IntegraAon%test%env%
ConAnuous%IntegraAon%between%%latest%dev%version%of%each%service%
• ConAnuous%integraAon%helps%teams%coordinate.%
• See%dependencies%between%“producers”%and%“consumers”%
• Errors%and%conflicts%show%related%team%contact%info%
• MeeAngs%and%changes%negoAated%between%two%teams,%not%many%
Role:%Developer%! Developers%have%more%power%and%responsibility.%! Developers%have%more%responsibility%for%tesAng.%! Developers%(not%QA%or%PM)%decide%when%to%release.%This)is)a)strong)finding.%! IncenAves%are%correct.%%Developer%might%have%to%come%back%from%Friday%night%beers%to%fix%a%problem.%%This%provides%a%moAvaAon%to%make%good%decisions%and%automate%tesAng.%
! Features%can%be%released%but%hidden.%%Product%Managers%and%Marketers%will%unveil%when%they%are%ready.%Unblock!%
Role:%QA%! QA%is%a%consultant.%! QA%gets%more%respect.%%Developers%have%to%ASK%for%service.%
! Developers%do%more%of%the%tesAng%work.%%They%should%organize%reviews%and%automated%tests%so%that%bugs%don’t%go%through%into%the%manual%test%process.%
! QA%has%more%Ame%to%invesAgate%usability%! QA%monitors%producAvity%and%quality%metrics%
IncenAves%for%ConAnuous%Flow%You%don’t%need%culture%change.%%You%only%need%to%release%more%frequently%1. If%you%do%code)review,%you%can%get%automated.%No%
browbeaAng%and%cajoling%is%required.%%Developers%will%ask%for%tests%when%they%review.%
2. If%developers%decide%to%release,%they%will%take%more%responsibility%for%tesAng%and%automated%tests.%
3. If%you%release%more%frequently,%developers%will%quickly%learn%not%to%break%the%build%
4. If%PM’s%take%unveil)and)measurement%responsibility,%they%will%make%be#er%features%
Our%Master%Plan%1. Release%more%frequently%2. Improve%%%%%%%
www.conAnuousagile.com/unblock%