Download - OSDC 2014: Jonathan Clarke - Rudder
![Page 1: OSDC 2014: Jonathan Clarke - Rudder](https://reader034.vdocuments.us/reader034/viewer/2022042601/54c6715e4a795913618b4676/html5/thumbnails/1.jpg)
Normation – CC-BY-SAnormation.com
Rudder
A powerful and structuredCFEngine framework
Jonathan CLARKE – [email protected]@jooooooon42 (that's 7 'o's)
![Page 2: OSDC 2014: Jonathan Clarke - Rudder](https://reader034.vdocuments.us/reader034/viewer/2022042601/54c6715e4a795913618b4676/html5/thumbnails/2.jpg)
Normation – CC-BY-SAnormation.com 2
www.rudder.cmWho am I?
● Jonathan Clarke
● Title: Co-founder & Product lead at Normation
● Origins: Sysadmin, infrastructure management
● Now: Automation + “running a company”-stuff
● Contributor to free software:
– Co-creator of Rudder
– Contributor to CFEngine, OpenLDAP
● Co-organizer of events:
![Page 3: OSDC 2014: Jonathan Clarke - Rudder](https://reader034.vdocuments.us/reader034/viewer/2022042601/54c6715e4a795913618b4676/html5/thumbnails/3.jpg)
Normation – CC-BY-SAnormation.com 3
www.rudder.cmIntro
This presentationis about Lego
Photo CC BY-NC-SA 2.0 from https://www.flickr.com/photos/dillpixel/
![Page 4: OSDC 2014: Jonathan Clarke - Rudder](https://reader034.vdocuments.us/reader034/viewer/2022042601/54c6715e4a795913618b4676/html5/thumbnails/4.jpg)
Normation – CC-BY-SAnormation.com 4
www.rudder.cmIntro
Reminder
Photo CC BY-NC-SA 2.0 from https://www.flickr.com/photos/evaekeblad/ Photo CC BY-SA 2.0 from https://www.flickr.com/photos/georgivar/
![Page 5: OSDC 2014: Jonathan Clarke - Rudder](https://reader034.vdocuments.us/reader034/viewer/2022042601/54c6715e4a795913618b4676/html5/thumbnails/5.jpg)
Normation – CC-BY-SAnormation.com 5
www.rudder.cmBackground
A bunch of ops consultants
● From “plain old” infrastructure to configuration management● Multiple companies: small, large & huge● 5-10 years of doing this
We always got the same takeaways
![Page 6: OSDC 2014: Jonathan Clarke - Rudder](https://reader034.vdocuments.us/reader034/viewer/2022042601/54c6715e4a795913618b4676/html5/thumbnails/6.jpg)
Normation – CC-BY-SAnormation.com 6
www.rudder.cmTakeaway #1: Automated configuration rocks!
Automated configuration rocks!
ScalableManage 1 to > 100000 servers the same way
Save timeDeploy faster & be more responsive to changes
Improve reliabilityAvoid manual errors, harmonize configurations
The proper way
to manage systems
![Page 7: OSDC 2014: Jonathan Clarke - Rudder](https://reader034.vdocuments.us/reader034/viewer/2022042601/54c6715e4a795913618b4676/html5/thumbnails/7.jpg)
Normation – CC-BY-SAnormation.com 7
www.rudder.cmTakeaway #2: Getting everyone on board?
Getting everyoneon board for CM is hard
Frustration“I can do it quicker by hand or with a shell script”
Steep learning curveNew concepts, non obvious syntaxes, paradigm, ...
Lack of motivation“What do I have to gain from using this tool?”
![Page 8: OSDC 2014: Jonathan Clarke - Rudder](https://reader034.vdocuments.us/reader034/viewer/2022042601/54c6715e4a795913618b4676/html5/thumbnails/8.jpg)
Normation – CC-BY-SAnormation.com 8
www.rudder.cmFeedback #2: CFEngine is hard!
Getting started from lots of bricks is daunting.
Photo CC BY-NC-SA 2.0 from https://www.flickr.com/photos/strutta/
![Page 9: OSDC 2014: Jonathan Clarke - Rudder](https://reader034.vdocuments.us/reader034/viewer/2022042601/54c6715e4a795913618b4676/html5/thumbnails/9.jpg)
Normation – CC-BY-SAnormation.com 9
www.rudder.cmWhat can we do?
So how comeso many projects
do work out?
![Page 10: OSDC 2014: Jonathan Clarke - Rudder](https://reader034.vdocuments.us/reader034/viewer/2022042601/54c6715e4a795913618b4676/html5/thumbnails/10.jpg)
Normation – CC-BY-SAnormation.com 10
www.rudder.cmWhat can we do?
Thanks to a hero!
So how comeso many projects
do work out?
Photo CC BY-NC-ND 2.0 from https://www.flickr.com/photos/mwboeckmann/
![Page 11: OSDC 2014: Jonathan Clarke - Rudder](https://reader034.vdocuments.us/reader034/viewer/2022042601/54c6715e4a795913618b4676/html5/thumbnails/11.jpg)
Normation – CC-BY-SAnormation.com 11
www.rudder.cmWhat can we do?
Poor configuration management hero...
![Page 12: OSDC 2014: Jonathan Clarke - Rudder](https://reader034.vdocuments.us/reader034/viewer/2022042601/54c6715e4a795913618b4676/html5/thumbnails/12.jpg)
Normation – CC-BY-SAnormation.com 12
www.rudder.cmWhat can we do?
Poor configuration management hero...
Hey, I'm trying to do this thing in config management,but I can't it to work, can you help me?
![Page 13: OSDC 2014: Jonathan Clarke - Rudder](https://reader034.vdocuments.us/reader034/viewer/2022042601/54c6715e4a795913618b4676/html5/thumbnails/13.jpg)
Normation – CC-BY-SAnormation.com 13
www.rudder.cmWhat can we do?
Poor configuration management hero...
Hi, this is the supervision team.I'm sorry to disturb you at night, but we've got this error
in production, and I think it's related to a change in the CM tool,but I don't understand it. Can you help me?
![Page 14: OSDC 2014: Jonathan Clarke - Rudder](https://reader034.vdocuments.us/reader034/viewer/2022042601/54c6715e4a795913618b4676/html5/thumbnails/14.jpg)
Normation – CC-BY-SAnormation.com 14
www.rudder.cmWhat can we do?
![Page 15: OSDC 2014: Jonathan Clarke - Rudder](https://reader034.vdocuments.us/reader034/viewer/2022042601/54c6715e4a795913618b4676/html5/thumbnails/15.jpg)
Normation – CC-BY-SAnormation.com 15
www.rudder.cmWhat can we do?
How can we help?
This is clearly a problem.
![Page 16: OSDC 2014: Jonathan Clarke - Rudder](https://reader034.vdocuments.us/reader034/viewer/2022042601/54c6715e4a795913618b4676/html5/thumbnails/16.jpg)
Normation – CC-BY-SAnormation.com 16
www.rudder.cm
Steep learning curveNew concepts, non obvious syntaxes, paradigm, ...
Approach
1) Separate content and controls
2) Provide access to key parameters without having to edit {CFEngine,Puppet,Chef} code
![Page 17: OSDC 2014: Jonathan Clarke - Rudder](https://reader034.vdocuments.us/reader034/viewer/2022042601/54c6715e4a795913618b4676/html5/thumbnails/17.jpg)
Normation – CC-BY-SAnormation.com 17
www.rudder.cm
Lack of motivation“What do I have to gain from using this tool?”
Approach
1) Show the benefits to all users
2) Provide nice reports showing what works, how many machines are impacted
![Page 18: OSDC 2014: Jonathan Clarke - Rudder](https://reader034.vdocuments.us/reader034/viewer/2022042601/54c6715e4a795913618b4676/html5/thumbnails/18.jpg)
Normation – CC-BY-SAnormation.com 18
www.rudder.cm
Frustration“I can do it quicker by hand or with a shell script”
Approach
1) Make it easy and quick to achieve success
2) Provide ready-to-use configuration techniques and share in-house ones simply
![Page 19: OSDC 2014: Jonathan Clarke - Rudder](https://reader034.vdocuments.us/reader034/viewer/2022042601/54c6715e4a795913618b4676/html5/thumbnails/19.jpg)
Normation – CC-BY-SAnormation.com 19
www.rudder.cmWhy Rudder?
Make configuration management easyand increase its adoption
Extend benefitsof
configuration managementto
a wider population
ManagersJunior
sysadminsNon
experts
Lower entry barrierto
learn and use
configuration management
Easy to use Highly powerful
![Page 20: OSDC 2014: Jonathan Clarke - Rudder](https://reader034.vdocuments.us/reader034/viewer/2022042601/54c6715e4a795913618b4676/html5/thumbnails/20.jpg)
Normation – CC-BY-SAnormation.com 20
www.rudder.cm
Sane defaults, always configurable
Philosophy
Core principles
Plug and play
SmartEasy
Extensible& CustomizableOpen source
![Page 21: OSDC 2014: Jonathan Clarke - Rudder](https://reader034.vdocuments.us/reader034/viewer/2022042601/54c6715e4a795913618b4676/html5/thumbnails/21.jpg)
Normation – CC-BY-SAnormation.com 21
www.rudder.cmKey points
Specifically designed forautomation & compliance
Pre-packaged for:Linux, UNIX, Windows, Android
Open Source
Simplified user experiencevia a Web UI
Graphical reportingBased on CFEngine 3(don't reinvent the wheel!)
Vagrant config to test:https://github.com/normation/rudder-vagrant/
![Page 22: OSDC 2014: Jonathan Clarke - Rudder](https://reader034.vdocuments.us/reader034/viewer/2022042601/54c6715e4a795913618b4676/html5/thumbnails/22.jpg)
Normation – CC-BY-SAnormation.com 22
www.rudder.cmWhat can we do?
Right! Show me already!
![Page 23: OSDC 2014: Jonathan Clarke - Rudder](https://reader034.vdocuments.us/reader034/viewer/2022042601/54c6715e4a795913618b4676/html5/thumbnails/23.jpg)
Normation – CC-BY-SAnormation.com 23
www.rudder.cmOverview
Simplified configuration
![Page 24: OSDC 2014: Jonathan Clarke - Rudder](https://reader034.vdocuments.us/reader034/viewer/2022042601/54c6715e4a795913618b4676/html5/thumbnails/24.jpg)
Normation – CC-BY-SAnormation.com 24
www.rudder.cmOverview
Built-in reporting
![Page 25: OSDC 2014: Jonathan Clarke - Rudder](https://reader034.vdocuments.us/reader034/viewer/2022042601/54c6715e4a795913618b4676/html5/thumbnails/25.jpg)
Normation – CC-BY-SAnormation.com 25
www.rudder.cmOverview
Built-in reporting
![Page 26: OSDC 2014: Jonathan Clarke - Rudder](https://reader034.vdocuments.us/reader034/viewer/2022042601/54c6715e4a795913618b4676/html5/thumbnails/26.jpg)
Normation – CC-BY-SAnormation.com 26
www.rudder.cmOverview
Complete tracability
![Page 27: OSDC 2014: Jonathan Clarke - Rudder](https://reader034.vdocuments.us/reader034/viewer/2022042601/54c6715e4a795913618b4676/html5/thumbnails/27.jpg)
Normation – CC-BY-SAnormation.com 27
www.rudder.cmDesign choices
Design choices
![Page 28: OSDC 2014: Jonathan Clarke - Rudder](https://reader034.vdocuments.us/reader034/viewer/2022042601/54c6715e4a795913618b4676/html5/thumbnails/28.jpg)
Normation – CC-BY-SAnormation.com 28
www.rudder.cmDesign choices: CFEngine
#1: Why CFEngine?
![Page 29: OSDC 2014: Jonathan Clarke - Rudder](https://reader034.vdocuments.us/reader034/viewer/2022042601/54c6715e4a795913618b4676/html5/thumbnails/29.jpg)
Normation – CC-BY-SAnormation.com 29
www.rudder.cmDesign choices: CFEngine
CFEngine rocksMulti-platformLinux, Android, BSD, AIX, HP-UX, Solaris, Windows...
Open SourceGPLv3
Small footprint, scalableA few MB of RAM,just seconds to run...
Continuous checkingAgent based approach,no push
Resilient to errorsNetwork outages, failures,unavailable resources...
![Page 30: OSDC 2014: Jonathan Clarke - Rudder](https://reader034.vdocuments.us/reader034/viewer/2022042601/54c6715e4a795913618b4676/html5/thumbnails/30.jpg)
Normation – CC-BY-SAnormation.com 30
www.rudder.cmDesign choices: CFEngine
Continuous checkingEvery 5 minutes
Multi-platformLinux, Unix, Windows, Android...
Separate configuration from implementation
ReportingDone after the checks, separate process
High freqency, trust in compliance reporting
Reuse implementations, less bugs, shared code...Clear separation of roles
Cover as many systems as possible
Avoid bottleneckDifferent report types
![Page 31: OSDC 2014: Jonathan Clarke - Rudder](https://reader034.vdocuments.us/reader034/viewer/2022042601/54c6715e4a795913618b4676/html5/thumbnails/31.jpg)
Normation – CC-BY-SAnormation.com 31
www.rudder.cmDesign choices: Network architecture
#2: Network architecture?
![Page 32: OSDC 2014: Jonathan Clarke - Rudder](https://reader034.vdocuments.us/reader034/viewer/2022042601/54c6715e4a795913618b4676/html5/thumbnails/32.jpg)
Normation – CC-BY-SAnormation.com 32
www.rudder.cmDesign choices: Network architecture
Rudder server
Node Node Node
TCP - port 5309File metadata and files
Authentication and encryption (SSL)
TCP ports 80 and 514HTTP and syslog
Node Node
Isolated networkRelay server
Download info
→ Built upon CFEngine network architecture
All connections go→from nodes to server
Pull-based approach→
![Page 33: OSDC 2014: Jonathan Clarke - Rudder](https://reader034.vdocuments.us/reader034/viewer/2022042601/54c6715e4a795913618b4676/html5/thumbnails/33.jpg)
Normation – CC-BY-SAnormation.com 33
www.rudder.cmDesign choices: Workflow
#3: Typical usage
![Page 34: OSDC 2014: Jonathan Clarke - Rudder](https://reader034.vdocuments.us/reader034/viewer/2022042601/54c6715e4a795913618b4676/html5/thumbnails/34.jpg)
Normation – CC-BY-SAnormation.com 34
www.rudder.cmDesign choices: Workflow
Management
Definepolicy
Changes(fixes, upgrades...)
c c
Community Expert
Sysadmins
Configureparameters
Configuration agent
Initial applicationContinuous verification
REP
OR
TIN
G
Technical abstraction(method vs parameters)
![Page 35: OSDC 2014: Jonathan Clarke - Rudder](https://reader034.vdocuments.us/reader034/viewer/2022042601/54c6715e4a795913618b4676/html5/thumbnails/35.jpg)
Normation – CC-BY-SAnormation.com 35
www.rudder.cmDesign choices: Central validation
#4: Central validation
![Page 36: OSDC 2014: Jonathan Clarke - Rudder](https://reader034.vdocuments.us/reader034/viewer/2022042601/54c6715e4a795913618b4676/html5/thumbnails/36.jpg)
Normation – CC-BY-SAnormation.com 36
www.rudder.cmDesign choices: Central validation
Validation workflow
![Page 37: OSDC 2014: Jonathan Clarke - Rudder](https://reader034.vdocuments.us/reader034/viewer/2022042601/54c6715e4a795913618b4676/html5/thumbnails/37.jpg)
Normation – CC-BY-SAnormation.com 37
www.rudder.cmDesign choices: Central validation
Validation workflow● States:
● Pending validation
– Can be sent to: Pending deployment, Deployed, Cancelled.
● Pending deployment
– The change was validated, but now require to be deployed. Can be sent to: Deployed, Cancelled.
● Deployed
– The change is deployed. This is a final state, it can’t be moved anymore.
● Cancelled
– The change was not approved. This is a final state, it can’t be moved anymore.
![Page 38: OSDC 2014: Jonathan Clarke - Rudder](https://reader034.vdocuments.us/reader034/viewer/2022042601/54c6715e4a795913618b4676/html5/thumbnails/38.jpg)
Normation – CC-BY-SAnormation.com 38
www.rudder.cmDemonstration
Demo!
![Page 39: OSDC 2014: Jonathan Clarke - Rudder](https://reader034.vdocuments.us/reader034/viewer/2022042601/54c6715e4a795913618b4676/html5/thumbnails/39.jpg)
Normation – CC-BY-SAnormation.com 39
www.rudder.cmExtending & Customizing
Extending & Customizing
![Page 40: OSDC 2014: Jonathan Clarke - Rudder](https://reader034.vdocuments.us/reader034/viewer/2022042601/54c6715e4a795913618b4676/html5/thumbnails/40.jpg)
Normation – CC-BY-SAnormation.com 40
www.rudder.cmExtension
Techniques
Implemented inCFEngine syntax
+ metadata for
web configuration
Nodes
Search criteria oninventory data
Hardware/OS/Network/Software/Node name/
...
Directives
Rules
Apply Directives to a Group
Groups
Sysadmins
c c
Manager or sysadmins
Expert
Community
![Page 41: OSDC 2014: Jonathan Clarke - Rudder](https://reader034.vdocuments.us/reader034/viewer/2022042601/54c6715e4a795913618b4676/html5/thumbnails/41.jpg)
Normation – CC-BY-SAnormation.com 41
www.rudder.cmExtension
Techniques
Implemented inCFEngine syntax
+ metadata for
web configuration
Nodes
Search criteria oninventory data
Hardware/OS/Network/Software/Node name/
...
Directives
Rules
Apply Directives to a Group
Groups
Sysadmins
c c
Manager or sysadmins
Expert
Community
![Page 42: OSDC 2014: Jonathan Clarke - Rudder](https://reader034.vdocuments.us/reader034/viewer/2022042601/54c6715e4a795913618b4676/html5/thumbnails/42.jpg)
Normation – CC-BY-SAnormation.com 42
www.rudder.cmExtension
Techniques
Implemented inCFEngine syntax
+ metadata for
web configuration
Nodes
Search criteria oninventory data
Hardware/OS/Network/Software/Node name/
...
Directives
Rules
Apply Directives to a Group
Groups
Sysadmins
c c
Manager or sysadmins
Expert
Community
Write any configuration you like in a Techniqueand share them with co-workersby exposing a selection of parameters
![Page 43: OSDC 2014: Jonathan Clarke - Rudder](https://reader034.vdocuments.us/reader034/viewer/2022042601/54c6715e4a795913618b4676/html5/thumbnails/43.jpg)
Normation – CC-BY-SAnormation.com 43
www.rudder.cmResult
Example === 1000 words
With ncf (see http://www.ncf.io)
![Page 44: OSDC 2014: Jonathan Clarke - Rudder](https://reader034.vdocuments.us/reader034/viewer/2022042601/54c6715e4a795913618b4676/html5/thumbnails/44.jpg)
Normation – CC-BY-SAnormation.com 44
www.rudder.cmResult
Example === 1000 words
With ncf + Rudder variables
![Page 45: OSDC 2014: Jonathan Clarke - Rudder](https://reader034.vdocuments.us/reader034/viewer/2022042601/54c6715e4a795913618b4676/html5/thumbnails/45.jpg)
Normation – CC-BY-SAnormation.com 45
www.rudder.cmOnline documentation
http://www.ncf.io/pages/reference.html
![Page 46: OSDC 2014: Jonathan Clarke - Rudder](https://reader034.vdocuments.us/reader034/viewer/2022042601/54c6715e4a795913618b4676/html5/thumbnails/46.jpg)
Normation – CC-BY-SAnormation.com 46
www.rudder.cmCurrent status
Project is now reliable & scalableBut needs more Techniques
Ohloh statistics:
Source: http://www.ohloh.net/p/rudder-project
h
![Page 47: OSDC 2014: Jonathan Clarke - Rudder](https://reader034.vdocuments.us/reader034/viewer/2022042601/54c6715e4a795913618b4676/html5/thumbnails/47.jpg)
Normation – CC-BY-SAnormation.com
Questions?
Check it out on:http://www.rudder.cm/
Jonathan CLARKE – [email protected]@jooooooon42 (that's 7 'o's)