how to choose tools for devops and continuous delivery - #doxlon
DESCRIPTION
With an ever-increasing array of tools and technologies claiming to 'enable DevOps', how do we know which tools to try or to choose? In-house, open source, or commercial? Ruby or shell? Dedicated or plugins? It transpires that highly collaborative practices such as DevOps and Continuous Delivery require new ways of assessing tools and technologies in order to avoid creating new silos. Matthew Skelton shares his recent experience of helping many different organisations to evaluate and select tools to facilitate DevOps; the recommendations may surprise you. A talk given at DevOps Exchange (#doxlon) meetup group on 24th July 2014: http://www.meetup.com/DevOps-Exchange-London/events/194288152/TRANSCRIPT
![Page 1: How to choose tools for DevOps and Continuous Delivery - #doxlon](https://reader034.vdocuments.us/reader034/viewer/2022051608/540d8f6c8d7f72767e8b4a63/html5/thumbnails/1.jpg)
How to choose tools for DevOps and
Continuous DeliveryMatthew Skelton, Skelton Thatcher Consulting
DevOps Exchange meetup group, 24 July 2014, London, UK
#doxlon
![Page 2: How to choose tools for DevOps and Continuous Delivery - #doxlon](https://reader034.vdocuments.us/reader034/viewer/2022051608/540d8f6c8d7f72767e8b4a63/html5/thumbnails/2.jpg)
Collaboration
Learning
Singleton tools
Conway’s Law
![Page 3: How to choose tools for DevOps and Continuous Delivery - #doxlon](https://reader034.vdocuments.us/reader034/viewer/2022051608/540d8f6c8d7f72767e8b4a63/html5/thumbnails/3.jpg)
Matthew Skelton
•15 years building & operating software systems
•Cybernetics + Neuroscience• control engineering
• psychology
• ‘network’ interactions
@matthewpskelton
![Page 4: How to choose tools for DevOps and Continuous Delivery - #doxlon](https://reader034.vdocuments.us/reader034/viewer/2022051608/540d8f6c8d7f72767e8b4a63/html5/thumbnails/4.jpg)
PIPELINE Conference
London Continuous Delivery meetupgroup (#londoncd)
![Page 5: How to choose tools for DevOps and Continuous Delivery - #doxlon](https://reader034.vdocuments.us/reader034/viewer/2022051608/540d8f6c8d7f72767e8b4a63/html5/thumbnails/5.jpg)
Help orgs to adopt and sustain good engineering practices
Interim CTO/Head of X, tech strategy, architecture, workshops, delivery
![Page 6: How to choose tools for DevOps and Continuous Delivery - #doxlon](https://reader034.vdocuments.us/reader034/viewer/2022051608/540d8f6c8d7f72767e8b4a63/html5/thumbnails/6.jpg)
Recent clients
•Tourism
•Betting & gambling
•Travel booking
•Financial data & reporting
•Healthcare
![Page 7: How to choose tools for DevOps and Continuous Delivery - #doxlon](https://reader034.vdocuments.us/reader034/viewer/2022051608/540d8f6c8d7f72767e8b4a63/html5/thumbnails/7.jpg)
Common themes•Online since ~1999
•Successful in their sector
•Large, central core database(s)
•Non-aligned goals
•Need to adopt DevOps and Continuous Delivery
![Page 8: How to choose tools for DevOps and Continuous Delivery - #doxlon](https://reader034.vdocuments.us/reader034/viewer/2022051608/540d8f6c8d7f72767e8b4a63/html5/thumbnails/8.jpg)
Common needs
•Technology selection ticklists
•Team interactions
•Collaboration opportunities
•Tools as catalysts
![Page 9: How to choose tools for DevOps and Continuous Delivery - #doxlon](https://reader034.vdocuments.us/reader034/viewer/2022051608/540d8f6c8d7f72767e8b4a63/html5/thumbnails/9.jpg)
![Page 10: How to choose tools for DevOps and Continuous Delivery - #doxlon](https://reader034.vdocuments.us/reader034/viewer/2022051608/540d8f6c8d7f72767e8b4a63/html5/thumbnails/10.jpg)
Continuous Delivery
•A scientific approach to changing software systems (Dave Farley)
•Regular, rapid, repeatable, reliable changes
![Page 11: How to choose tools for DevOps and Continuous Delivery - #doxlon](https://reader034.vdocuments.us/reader034/viewer/2022051608/540d8f6c8d7f72767e8b4a63/html5/thumbnails/11.jpg)
Continuous Delivery
•Quality
•Safety
•Reliability
•Psychology
•Effectiveness
![Page 12: How to choose tools for DevOps and Continuous Delivery - #doxlon](https://reader034.vdocuments.us/reader034/viewer/2022051608/540d8f6c8d7f72767e8b4a63/html5/thumbnails/12.jpg)
![Page 13: How to choose tools for DevOps and Continuous Delivery - #doxlon](https://reader034.vdocuments.us/reader034/viewer/2022051608/540d8f6c8d7f72767e8b4a63/html5/thumbnails/13.jpg)
DevOps“Highly effective, daily collaboration between software developers and IT
operations people to produce relevant, working systems” *
*also QA/Testing, IT Service Desk, Programme Management,
Commercial, Marketing, etc.
![Page 14: How to choose tools for DevOps and Continuous Delivery - #doxlon](https://reader034.vdocuments.us/reader034/viewer/2022051608/540d8f6c8d7f72767e8b4a63/html5/thumbnails/14.jpg)
Not DevOps
“Automation”
“Build & Release”
“Infrastructure Development”
“System Administration”
![Page 15: How to choose tools for DevOps and Continuous Delivery - #doxlon](https://reader034.vdocuments.us/reader034/viewer/2022051608/540d8f6c8d7f72767e8b4a63/html5/thumbnails/15.jpg)
![Page 16: How to choose tools for DevOps and Continuous Delivery - #doxlon](https://reader034.vdocuments.us/reader034/viewer/2022051608/540d8f6c8d7f72767e8b4a63/html5/thumbnails/16.jpg)
Collaboration
![Page 17: How to choose tools for DevOps and Continuous Delivery - #doxlon](https://reader034.vdocuments.us/reader034/viewer/2022051608/540d8f6c8d7f72767e8b4a63/html5/thumbnails/17.jpg)
1. Version Control
![Page 18: How to choose tools for DevOps and Continuous Delivery - #doxlon](https://reader034.vdocuments.us/reader034/viewer/2022051608/540d8f6c8d7f72767e8b4a63/html5/thumbnails/18.jpg)
![Page 19: How to choose tools for DevOps and Continuous Delivery - #doxlon](https://reader034.vdocuments.us/reader034/viewer/2022051608/540d8f6c8d7f72767e8b4a63/html5/thumbnails/19.jpg)
![Page 20: How to choose tools for DevOps and Continuous Delivery - #doxlon](https://reader034.vdocuments.us/reader034/viewer/2022051608/540d8f6c8d7f72767e8b4a63/html5/thumbnails/20.jpg)
![Page 21: How to choose tools for DevOps and Continuous Delivery - #doxlon](https://reader034.vdocuments.us/reader034/viewer/2022051608/540d8f6c8d7f72767e8b4a63/html5/thumbnails/21.jpg)
![Page 22: How to choose tools for DevOps and Continuous Delivery - #doxlon](https://reader034.vdocuments.us/reader034/viewer/2022051608/540d8f6c8d7f72767e8b4a63/html5/thumbnails/22.jpg)
![Page 23: How to choose tools for DevOps and Continuous Delivery - #doxlon](https://reader034.vdocuments.us/reader034/viewer/2022051608/540d8f6c8d7f72767e8b4a63/html5/thumbnails/23.jpg)
![Page 24: How to choose tools for DevOps and Continuous Delivery - #doxlon](https://reader034.vdocuments.us/reader034/viewer/2022051608/540d8f6c8d7f72767e8b4a63/html5/thumbnails/24.jpg)
2. Deployment Pipeline
![Page 25: How to choose tools for DevOps and Continuous Delivery - #doxlon](https://reader034.vdocuments.us/reader034/viewer/2022051608/540d8f6c8d7f72767e8b4a63/html5/thumbnails/25.jpg)
![Page 26: How to choose tools for DevOps and Continuous Delivery - #doxlon](https://reader034.vdocuments.us/reader034/viewer/2022051608/540d8f6c8d7f72767e8b4a63/html5/thumbnails/26.jpg)
![Page 27: How to choose tools for DevOps and Continuous Delivery - #doxlon](https://reader034.vdocuments.us/reader034/viewer/2022051608/540d8f6c8d7f72767e8b4a63/html5/thumbnails/27.jpg)
![Page 28: How to choose tools for DevOps and Continuous Delivery - #doxlon](https://reader034.vdocuments.us/reader034/viewer/2022051608/540d8f6c8d7f72767e8b4a63/html5/thumbnails/28.jpg)
Challenges were:
•Limited Git skills in Service team
•Manual deployments
•‘Snowflake’ servers
•No CI
•Risks: security, automation, etc
![Page 29: How to choose tools for DevOps and Continuous Delivery - #doxlon](https://reader034.vdocuments.us/reader034/viewer/2022051608/540d8f6c8d7f72767e8b4a63/html5/thumbnails/29.jpg)
http://bit.ly/thetrainline-weekly-deploy
![Page 30: How to choose tools for DevOps and Continuous Delivery - #doxlon](https://reader034.vdocuments.us/reader034/viewer/2022051608/540d8f6c8d7f72767e8b4a63/html5/thumbnails/30.jpg)
What we did•Built a walking skeleton pipeline
•Modelled security roles and stages
• Included manual steps (at first)
•Walked people through steps
•Finally: opened firewall so everyone could see the UI
![Page 31: How to choose tools for DevOps and Continuous Delivery - #doxlon](https://reader034.vdocuments.us/reader034/viewer/2022051608/540d8f6c8d7f72767e8b4a63/html5/thumbnails/31.jpg)
Results
•Security: happy
•Service team: relieved (& happy)
•Developers: won over (& happy)
•Business: surprised (& happy)
![Page 32: How to choose tools for DevOps and Continuous Delivery - #doxlon](https://reader034.vdocuments.us/reader034/viewer/2022051608/540d8f6c8d7f72767e8b4a63/html5/thumbnails/32.jpg)
3. Log Aggregation
![Page 33: How to choose tools for DevOps and Continuous Delivery - #doxlon](https://reader034.vdocuments.us/reader034/viewer/2022051608/540d8f6c8d7f72767e8b4a63/html5/thumbnails/33.jpg)
LogStash
•ElasticSearch+ LogStash+ Kibana•ELK
•In Production, Pre-Prod, Test
•On developer machines!!!
![Page 34: How to choose tools for DevOps and Continuous Delivery - #doxlon](https://reader034.vdocuments.us/reader034/viewer/2022051608/540d8f6c8d7f72767e8b4a63/html5/thumbnails/34.jpg)
http://rashidkpc.github.io/Kibana/images/screenshots/searchss.png
![Page 35: How to choose tools for DevOps and Continuous Delivery - #doxlon](https://reader034.vdocuments.us/reader034/viewer/2022051608/540d8f6c8d7f72767e8b4a63/html5/thumbnails/35.jpg)
Collaboration & tool choice
Value collaboration as a key criterion
Orthogonal to main purpose (?)
“How does [the use of] this tool help people to collaborate?”
![Page 36: How to choose tools for DevOps and Continuous Delivery - #doxlon](https://reader034.vdocuments.us/reader034/viewer/2022051608/540d8f6c8d7f72767e8b4a63/html5/thumbnails/36.jpg)
Learning
![Page 37: How to choose tools for DevOps and Continuous Delivery - #doxlon](https://reader034.vdocuments.us/reader034/viewer/2022051608/540d8f6c8d7f72767e8b4a63/html5/thumbnails/37.jpg)
![Page 38: How to choose tools for DevOps and Continuous Delivery - #doxlon](https://reader034.vdocuments.us/reader034/viewer/2022051608/540d8f6c8d7f72767e8b4a63/html5/thumbnails/38.jpg)
![Page 39: How to choose tools for DevOps and Continuous Delivery - #doxlon](https://reader034.vdocuments.us/reader034/viewer/2022051608/540d8f6c8d7f72767e8b4a63/html5/thumbnails/39.jpg)
![Page 40: How to choose tools for DevOps and Continuous Delivery - #doxlon](https://reader034.vdocuments.us/reader034/viewer/2022051608/540d8f6c8d7f72767e8b4a63/html5/thumbnails/40.jpg)
![Page 41: How to choose tools for DevOps and Continuous Delivery - #doxlon](https://reader034.vdocuments.us/reader034/viewer/2022051608/540d8f6c8d7f72767e8b4a63/html5/thumbnails/41.jpg)
Learning & tool choice
Bring people with you
Appreciate current skills
Prefer achievable gains now
Avoid fear of too-scary tools
![Page 42: How to choose tools for DevOps and Continuous Delivery - #doxlon](https://reader034.vdocuments.us/reader034/viewer/2022051608/540d8f6c8d7f72767e8b4a63/html5/thumbnails/42.jpg)
Singleton tools(or the ‘Prize Bull’ approach)
![Page 43: How to choose tools for DevOps and Continuous Delivery - #doxlon](https://reader034.vdocuments.us/reader034/viewer/2022051608/540d8f6c8d7f72767e8b4a63/html5/thumbnails/43.jpg)
![Page 44: How to choose tools for DevOps and Continuous Delivery - #doxlon](https://reader034.vdocuments.us/reader034/viewer/2022051608/540d8f6c8d7f72767e8b4a63/html5/thumbnails/44.jpg)
Singleton tools
•Special database server
•Costly log aggregation
•Costly monitoring
•Server configuration
![Page 45: How to choose tools for DevOps and Continuous Delivery - #doxlon](https://reader034.vdocuments.us/reader034/viewer/2022051608/540d8f6c8d7f72767e8b4a63/html5/thumbnails/45.jpg)
http://www.walpapershddownload.com/highland-cattle-wallpapers/
![Page 46: How to choose tools for DevOps and Continuous Delivery - #doxlon](https://reader034.vdocuments.us/reader034/viewer/2022051608/540d8f6c8d7f72767e8b4a63/html5/thumbnails/46.jpg)
“Better features”?
Optimise globally across the teams that need to collaborate
![Page 47: How to choose tools for DevOps and Continuous Delivery - #doxlon](https://reader034.vdocuments.us/reader034/viewer/2022051608/540d8f6c8d7f72767e8b4a63/html5/thumbnails/47.jpg)
Singleton tool
Breaks feedback (learning) loop from Production
Makes CI/CD more difficult
Underestimates value of collaboration and learning
![Page 48: How to choose tools for DevOps and Continuous Delivery - #doxlon](https://reader034.vdocuments.us/reader034/viewer/2022051608/540d8f6c8d7f72767e8b4a63/html5/thumbnails/48.jpg)
Conway’s Law
![Page 49: How to choose tools for DevOps and Continuous Delivery - #doxlon](https://reader034.vdocuments.us/reader034/viewer/2022051608/540d8f6c8d7f72767e8b4a63/html5/thumbnails/49.jpg)
Mel Conway, 1968
“organizations which design systems ... are constrained to produce designs which are copies of the communication structures of these organizations”
http://www.melconway.com/Home/Conways_Law.html
![Page 50: How to choose tools for DevOps and Continuous Delivery - #doxlon](https://reader034.vdocuments.us/reader034/viewer/2022051608/540d8f6c8d7f72767e8b4a63/html5/thumbnails/50.jpg)
![Page 51: How to choose tools for DevOps and Continuous Delivery - #doxlon](https://reader034.vdocuments.us/reader034/viewer/2022051608/540d8f6c8d7f72767e8b4a63/html5/thumbnails/51.jpg)
DevOps Topologies
![Page 52: How to choose tools for DevOps and Continuous Delivery - #doxlon](https://reader034.vdocuments.us/reader034/viewer/2022051608/540d8f6c8d7f72767e8b4a63/html5/thumbnails/52.jpg)
DevOps Topologies
![Page 53: How to choose tools for DevOps and Continuous Delivery - #doxlon](https://reader034.vdocuments.us/reader034/viewer/2022051608/540d8f6c8d7f72767e8b4a63/html5/thumbnails/53.jpg)
DevOps Topologies
![Page 54: How to choose tools for DevOps and Continuous Delivery - #doxlon](https://reader034.vdocuments.us/reader034/viewer/2022051608/540d8f6c8d7f72767e8b4a63/html5/thumbnails/54.jpg)
DevOps Topologies
![Page 55: How to choose tools for DevOps and Continuous Delivery - #doxlon](https://reader034.vdocuments.us/reader034/viewer/2022051608/540d8f6c8d7f72767e8b4a63/html5/thumbnails/55.jpg)
Conway & Tool Choice
See the organisation as a system
Separate tools for separate teams
Shared tools for collaborative teams
http://bit.ly/DevOpsTopologies
![Page 56: How to choose tools for DevOps and Continuous Delivery - #doxlon](https://reader034.vdocuments.us/reader034/viewer/2022051608/540d8f6c8d7f72767e8b4a63/html5/thumbnails/56.jpg)
Conway’s Law•Allan Kelly - @allankellynet
•https://vimeo.com/channels/londoncd
![Page 57: How to choose tools for DevOps and Continuous Delivery - #doxlon](https://reader034.vdocuments.us/reader034/viewer/2022051608/540d8f6c8d7f72767e8b4a63/html5/thumbnails/57.jpg)
![Page 58: How to choose tools for DevOps and Continuous Delivery - #doxlon](https://reader034.vdocuments.us/reader034/viewer/2022051608/540d8f6c8d7f72767e8b4a63/html5/thumbnails/58.jpg)
How to choose tools for DevOps
Value collaboration aspects
Avoid a learning mountain: evolve tooling
Avoid Production-only tools
Consider Conway’s Law
(this list is incomplete!)
![Page 59: How to choose tools for DevOps and Continuous Delivery - #doxlon](https://reader034.vdocuments.us/reader034/viewer/2022051608/540d8f6c8d7f72767e8b4a63/html5/thumbnails/59.jpg)
Further readingHighOps Operability eBook
bit.ly/OperabilityEbook
Build Quality In
buildqualityin.com
![Page 60: How to choose tools for DevOps and Continuous Delivery - #doxlon](https://reader034.vdocuments.us/reader034/viewer/2022051608/540d8f6c8d7f72767e8b4a63/html5/thumbnails/60.jpg)
Thank youmatthewskelton.net / @matthewpskelton
skeltonthatcher.com
HT: @Squire_Matt, @alan_parkinson