winops conf 2016 - matteo emili - development and qa dilemmas in devops
TRANSCRIPT
![Page 1: WinOps Conf 2016 - Matteo Emili - Development and QA Dilemmas in DevOps](https://reader034.vdocuments.us/reader034/viewer/2022051708/588ae0ac1a28abab6c8b541f/html5/thumbnails/1.jpg)
Development and QA dilemmas in DevOpsMatteo Emilitwitter.com/MattVSTS || mattvsts.blogspot.com
![Page 2: WinOps Conf 2016 - Matteo Emili - Development and QA Dilemmas in DevOps](https://reader034.vdocuments.us/reader034/viewer/2022051708/588ae0ac1a28abab6c8b541f/html5/thumbnails/2.jpg)
Who am I?
Visual Studio and Development Technologies MVP (ALM)
Professional Scrum Master I Systems Engineering Advisor @ Dell Software
Taunton Developers Meetup London Microsoft DevOps Meetup Other communities overseas
![Page 3: WinOps Conf 2016 - Matteo Emili - Development and QA Dilemmas in DevOps](https://reader034.vdocuments.us/reader034/viewer/2022051708/588ae0ac1a28abab6c8b541f/html5/thumbnails/3.jpg)
CD Cycle
REQUIREMENTSPRODUCTBACKLOG
OPSBACKLOG
MonitorImplement
WORKING SOFTWARE
DefineIdeation
DevelopIdea to working software
OperateWorking software in productionValue realization
![Page 4: WinOps Conf 2016 - Matteo Emili - Development and QA Dilemmas in DevOps](https://reader034.vdocuments.us/reader034/viewer/2022051708/588ae0ac1a28abab6c8b541f/html5/thumbnails/4.jpg)
Are we all ready for it?
![Page 5: WinOps Conf 2016 - Matteo Emili - Development and QA Dilemmas in DevOps](https://reader034.vdocuments.us/reader034/viewer/2022051708/588ae0ac1a28abab6c8b541f/html5/thumbnails/5.jpg)
Sometimes things are the other way round…
The first thing to do should be creating a Minimum Viable Product, including the infrastructure
The Pipeline must work from the very first moment Dev and Test environments should be disposable,
infrastructure must be idempotent Testing can happen in Production
![Page 6: WinOps Conf 2016 - Matteo Emili - Development and QA Dilemmas in DevOps](https://reader034.vdocuments.us/reader034/viewer/2022051708/588ae0ac1a28abab6c8b541f/html5/thumbnails/6.jpg)
It is a shocking change for some
“I was frightened by not feeling a safety net” “Does it actually work?” “The feeling is not good at first” “We don’t want to be accountable for all of this” “It can’t work!” “How do we know what to build?” “It is hard to get the grip with these tools”
![Page 7: WinOps Conf 2016 - Matteo Emili - Development and QA Dilemmas in DevOps](https://reader034.vdocuments.us/reader034/viewer/2022051708/588ae0ac1a28abab6c8b541f/html5/thumbnails/7.jpg)
PrerequisiteThe team must buy into DevOps!
![Page 8: WinOps Conf 2016 - Matteo Emili - Development and QA Dilemmas in DevOps](https://reader034.vdocuments.us/reader034/viewer/2022051708/588ae0ac1a28abab6c8b541f/html5/thumbnails/8.jpg)
Scenario #0
Use your infrastructure!
Leverage on Infrastructure As Code (DSC, Azure RM, Amazon CloudFormations) to remove friction when factoring the infrastructure in
Spend time on defining what you need to have, and the provider (Azure, AWS, local deployment service) will deploy that for you
Idempotent and resilient – if there is something wrong, redeploy
Less friction here means no barrier between Dev and Ops, a fundamental (and time saving) change!
![Page 9: WinOps Conf 2016 - Matteo Emili - Development and QA Dilemmas in DevOps](https://reader034.vdocuments.us/reader034/viewer/2022051708/588ae0ac1a28abab6c8b541f/html5/thumbnails/9.jpg)
Scenario #1
Database schema change
You must bring a database schema change in production
Cloud or mission critical web service Transparent for your users, regardless of public-facing
or internal change No downtime for migration – it must work!
![Page 10: WinOps Conf 2016 - Matteo Emili - Development and QA Dilemmas in DevOps](https://reader034.vdocuments.us/reader034/viewer/2022051708/588ae0ac1a28abab6c8b541f/html5/thumbnails/10.jpg)
![Page 11: WinOps Conf 2016 - Matteo Emili - Development and QA Dilemmas in DevOps](https://reader034.vdocuments.us/reader034/viewer/2022051708/588ae0ac1a28abab6c8b541f/html5/thumbnails/11.jpg)
The typical approach
Develop everything in a separate environment Countless hours of tests Big-bang migration then new feature for all the users Big release date, people freak out …lots of fingers crossed!
![Page 12: WinOps Conf 2016 - Matteo Emili - Development and QA Dilemmas in DevOps](https://reader034.vdocuments.us/reader034/viewer/2022051708/588ae0ac1a28abab6c8b541f/html5/thumbnails/12.jpg)
Shhh! Don’t make noise!
There is no such approach in DevOps Each big change must be split in smaller changes
They are often invisible to the user Silent deployments can happen across multiple
sprints If something breaks, rollback Start using your MVP straight away!
![Page 13: WinOps Conf 2016 - Matteo Emili - Development and QA Dilemmas in DevOps](https://reader034.vdocuments.us/reader034/viewer/2022051708/588ae0ac1a28abab6c8b541f/html5/thumbnails/13.jpg)
DemoInfrastructure as CodeDatabase schema change
![Page 14: WinOps Conf 2016 - Matteo Emili - Development and QA Dilemmas in DevOps](https://reader034.vdocuments.us/reader034/viewer/2022051708/588ae0ac1a28abab6c8b541f/html5/thumbnails/14.jpg)
Scenario #2
Feature Flags
Championed by Martin Fowler, Feature Flags are around since at least 2009
Pretty much all services nowadays use them Flickr, Amazon, Microsoft, Facebook, Google, Netflix … They are used to enable or disable certain areas of
your application to your users
![Page 15: WinOps Conf 2016 - Matteo Emili - Development and QA Dilemmas in DevOps](https://reader034.vdocuments.us/reader034/viewer/2022051708/588ae0ac1a28abab6c8b541f/html5/thumbnails/15.jpg)
“…wait. Does it mean Microservices and all the new stuff around?”
No, it doesn’t! You can use Feature Flags with all the architectures It is a practice, not a product It ties in with the previous scenario and makes testing
easier
![Page 16: WinOps Conf 2016 - Matteo Emili - Development and QA Dilemmas in DevOps](https://reader034.vdocuments.us/reader034/viewer/2022051708/588ae0ac1a28abab6c8b541f/html5/thumbnails/16.jpg)
How?
Practically speaking, it is about a Configuration Database coupled with Identifiers
The Configuration Database can be a csv file, XML file, key-value model, a database (!)
Identifiers are areas you will easily split your applications into
They often go well together with Feature/Epics or even User Stories
Countless OSS libraries
![Page 17: WinOps Conf 2016 - Matteo Emili - Development and QA Dilemmas in DevOps](https://reader034.vdocuments.us/reader034/viewer/2022051708/588ae0ac1a28abab6c8b541f/html5/thumbnails/17.jpg)
It doesn’t come for free
It is expensive (in terms of time, and sometimes refactoring) to build it up from scratch in an existing application
Not just enable/disable – logging, kill switch, selective adoption, rollouts are all things to implement that takes time
OSS libraries help, as there is a choice for each platform, but there is still an implementation cost to keep in mind
![Page 18: WinOps Conf 2016 - Matteo Emili - Development and QA Dilemmas in DevOps](https://reader034.vdocuments.us/reader034/viewer/2022051708/588ae0ac1a28abab6c8b541f/html5/thumbnails/18.jpg)
DemoFeature Flags with LaunchDarkly
![Page 19: WinOps Conf 2016 - Matteo Emili - Development and QA Dilemmas in DevOps](https://reader034.vdocuments.us/reader034/viewer/2022051708/588ae0ac1a28abab6c8b541f/html5/thumbnails/19.jpg)
Scenario #3
A/B Testing, MVT Testing…is it about testing?
Little changes for QA in DevOps, except for automation
What is really new are practices of Testing in Production - extremely common especially with Web Applications
You deploy version B of your A application (or a permutation of possibilities), direct a percentage of the traffic towards it and compare the adoption results
Implementing them is extremely easy, getting value out of them is hard!
![Page 20: WinOps Conf 2016 - Matteo Emili - Development and QA Dilemmas in DevOps](https://reader034.vdocuments.us/reader034/viewer/2022051708/588ae0ac1a28abab6c8b541f/html5/thumbnails/20.jpg)
A bit of Lean…
"There is no such thing as a free lunch“ – technology alone is not going to deliver a successful product
My take is to adopt the Build-Measure-Learn loop Start with a Minimum Viable Product and measure its
usage with appropriate metrics Build on top of it, discard what doesn’t work
![Page 21: WinOps Conf 2016 - Matteo Emili - Development and QA Dilemmas in DevOps](https://reader034.vdocuments.us/reader034/viewer/2022051708/588ae0ac1a28abab6c8b541f/html5/thumbnails/21.jpg)
Getting visibility on what is actually happening
Is my application performing as expected? What about availability? Are my users having a good experience?
Above all, you need better, faster and more direct feedback
![Page 22: WinOps Conf 2016 - Matteo Emili - Development and QA Dilemmas in DevOps](https://reader034.vdocuments.us/reader034/viewer/2022051708/588ae0ac1a28abab6c8b541f/html5/thumbnails/22.jpg)
Learn from your users and improve your product
In the Build-Measure-Learn loop the Measure stage is the most important one
Metrics will tell if you are going in the right direction, so it is critical to ‘get it right’
Extend your product to include what really matters Get insights on your users before they react Proactively understand potential problems
You can analyse your data and understand if your application is behaving as expected
![Page 23: WinOps Conf 2016 - Matteo Emili - Development and QA Dilemmas in DevOps](https://reader034.vdocuments.us/reader034/viewer/2022051708/588ae0ac1a28abab6c8b541f/html5/thumbnails/23.jpg)
It is different from logging
We are not talking about plain application logs What user actions are meaningful to your business
goals? Is the telemetry data answering this question? Is there any bottlenecks which is penalising the
business? You need to have visibility on user events
Actionable insights are mapped into behaviours (what content is used, not how)
E.g.: when a user enters in a certain page/area, log on/off, Favourites, social stats, errors…
![Page 24: WinOps Conf 2016 - Matteo Emili - Development and QA Dilemmas in DevOps](https://reader034.vdocuments.us/reader034/viewer/2022051708/588ae0ac1a28abab6c8b541f/html5/thumbnails/24.jpg)
You won’t do that from scratch
It is too expensive to build something from scratch You need to use an analytics provider The development experience must be cross-platform,
unobtrusive at first and then customisable with an SDK
Easily add custom telemetry wherever needed
![Page 25: WinOps Conf 2016 - Matteo Emili - Development and QA Dilemmas in DevOps](https://reader034.vdocuments.us/reader034/viewer/2022051708/588ae0ac1a28abab6c8b541f/html5/thumbnails/25.jpg)
DemoLearn from your users
![Page 26: WinOps Conf 2016 - Matteo Emili - Development and QA Dilemmas in DevOps](https://reader034.vdocuments.us/reader034/viewer/2022051708/588ae0ac1a28abab6c8b541f/html5/thumbnails/26.jpg)
So?
These scenarios are very common when approaching DevOps
Remember: DevOps is not the silver bullet, teams must buy-in into the process
Collaboration is key to success Use data to take decisions Don’t be afraid!