chef actions: delightful near real-time activity tracking!
DESCRIPTION
Actions is a new analytics add-on for Enterprise Chef, letting you see who did what on your Chef Server.TRANSCRIPT
![Page 1: Chef Actions: Delightful near real-time activity tracking!](https://reader038.vdocuments.us/reader038/viewer/2022103013/53ed9f648d7f7289708b5dc6/html5/thumbnails/1.jpg)
Chef ActionsActions: Delightful near real-time activity tracking!
Allen Goodman (@goodmanio), Software Engineer, ChefJames Casey (@jamesc_000), Engineering Lead, Chef
![Page 2: Chef Actions: Delightful near real-time activity tracking!](https://reader038.vdocuments.us/reader038/viewer/2022103013/53ed9f648d7f7289708b5dc6/html5/thumbnails/2.jpg)
A first look at Chef Actions
Chef Actions answers questions about what is happening on your Chef Server
• What changed on your Chef Server ?• Clients, Cookbooks, Data Bags, Environments, Nodes, Roles
• Who changed it ?
• What did they do ?• Create, Update, Delete
• When did they do it ?
![Page 3: Chef Actions: Delightful near real-time activity tracking!](https://reader038.vdocuments.us/reader038/viewer/2022103013/53ed9f648d7f7289708b5dc6/html5/thumbnails/3.jpg)
Principles
• Provide a read-only view of what happened• Road to audit
• Allow to react to events as they happen
• Also, enable after the fact investigation• “What happened just before nodes started failing runs?”• “When did our systems gets patched for Heartbleed?”
![Page 4: Chef Actions: Delightful near real-time activity tracking!](https://reader038.vdocuments.us/reader038/viewer/2022103013/53ed9f648d7f7289708b5dc6/html5/thumbnails/4.jpg)
Architecture
![Page 5: Chef Actions: Delightful near real-time activity tracking!](https://reader038.vdocuments.us/reader038/viewer/2022103013/53ed9f648d7f7289708b5dc6/html5/thumbnails/5.jpg)
Chef Analytics’ Web Application
![Page 6: Chef Actions: Delightful near real-time activity tracking!](https://reader038.vdocuments.us/reader038/viewer/2022103013/53ed9f648d7f7289708b5dc6/html5/thumbnails/6.jpg)
![Page 7: Chef Actions: Delightful near real-time activity tracking!](https://reader038.vdocuments.us/reader038/viewer/2022103013/53ed9f648d7f7289708b5dc6/html5/thumbnails/7.jpg)
Event Processing
![Page 8: Chef Actions: Delightful near real-time activity tracking!](https://reader038.vdocuments.us/reader038/viewer/2022103013/53ed9f648d7f7289708b5dc6/html5/thumbnails/8.jpg)
![Page 9: Chef Actions: Delightful near real-time activity tracking!](https://reader038.vdocuments.us/reader038/viewer/2022103013/53ed9f648d7f7289708b5dc6/html5/thumbnails/9.jpg)
Message Format from Chef{ "entity_name": "2.8.1", "entity_type": "version", "parent_name": "apache2", "parent_type": "cookbook”, "task": "update", "organization_name": ”my_corp", "recorded_at": "20014-04-05T18:26:13Z", "remote_hostname": ”33.33.33.10", "remote_request_id": "D5974D8A-E961-4845-8D4F-5BB92DAD1BD5", "service_hostname": ”private-chef-fe.example.com”, "request_id": "hY2UqBZTuqAr65M8t4FmYAD5RUjbnhpBhubxBZqVZJZgNX7x", "requestor_name": ”bob", "requestor_type": ”user”, "user_agent": "Chef Client/11.12.0"}
![Page 10: Chef Actions: Delightful near real-time activity tracking!](https://reader038.vdocuments.us/reader038/viewer/2022103013/53ed9f648d7f7289708b5dc6/html5/thumbnails/10.jpg)
Event Processing Architecture
![Page 11: Chef Actions: Delightful near real-time activity tracking!](https://reader038.vdocuments.us/reader038/viewer/2022103013/53ed9f648d7f7289708b5dc6/html5/thumbnails/11.jpg)
Classification
• Static and Dynamic Analysis• Conformance
• Are conformant cookbooks less susceptible to failure?• Coverage
• Is coverage correlated with success? • Style
• Does my preferred style require more maintenance? • Syntactic and Semantic Mistakes
• Is this resource broken?
• Supervised Learning• Predicting Imminent and Long-term Problems
• Why does my infrastructure break?
![Page 12: Chef Actions: Delightful near real-time activity tracking!](https://reader038.vdocuments.us/reader038/viewer/2022103013/53ed9f648d7f7289708b5dc6/html5/thumbnails/12.jpg)
Notifications
IRC
notifications: irc: - "chat.freenode.net#chef" template: - "%{resource} %{action}d!” skip_join: true use_notice: true
notifications: email: - [email protected] - [email protected] on_update: never
Campfire, HipChat, Slack, et al.
![Page 13: Chef Actions: Delightful near real-time activity tracking!](https://reader038.vdocuments.us/reader038/viewer/2022103013/53ed9f648d7f7289708b5dc6/html5/thumbnails/13.jpg)
Webhooks
• POST a full copy of a message to an external service
• Optionally contains copy of object• E.g. full node object or role
• Ships with a stub service for you start from
• Usage:• Monitoring – notified on create/delete• CMDB – monitor software/OS version
changes for compliance
Webhooks
hooks: endpoint: - https://my-service.example.com/ingest with_object: true
![Page 14: Chef Actions: Delightful near real-time activity tracking!](https://reader038.vdocuments.us/reader038/viewer/2022103013/53ed9f648d7f7289708b5dc6/html5/thumbnails/14.jpg)
Creating Metricsmetrics: - id: james - "What’s up with James’ weird hours?” - actions: - create - delete - update - actors: - "james" - resources: - cookbook - statistics: - frequency - mean - window - hourly
![Page 15: Chef Actions: Delightful near real-time activity tracking!](https://reader038.vdocuments.us/reader038/viewer/2022103013/53ed9f648d7f7289708b5dc6/html5/thumbnails/15.jpg)
Persistence
![Page 16: Chef Actions: Delightful near real-time activity tracking!](https://reader038.vdocuments.us/reader038/viewer/2022103013/53ed9f648d7f7289708b5dc6/html5/thumbnails/16.jpg)
Query Parameters
/actions?tasks=create, delete, update&types=environment&name=prod&before=2014-04-17&since=2014-04-10
![Page 17: Chef Actions: Delightful near real-time activity tracking!](https://reader038.vdocuments.us/reader038/viewer/2022103013/53ed9f648d7f7289708b5dc6/html5/thumbnails/17.jpg)
Response{ "id": "e1e2c057-b18f-46db-8c84-7d87efc07260", "tags": [ "Knife” ], "recorded_at": "2014-04-13T04:38:03.000Z", "remote_hostname": "76.115.172.22", "remote_request_id": "d57845ce-3b48-4070-9ed4-4b4333d112cb", "request_id": "g3IAA2QAEGVyY2hlZkAxMjcuMC4wLjEDAAKEjwAAAAkAAAAA", "service_hostname": "action-rspreprod-i-c63b3e97.opscode.us", "task": "delete", "url": "/activities/e1e2c057-b18f-46db-8c84-7d87efc07260", "user_agent": "Chef Knife/11.12.2 (ruby-1.9.3-p194; ohai-7.0.2; x86_64-darwin11.4.2; +http://opscode.com)", "requestable": { "name": "bob", "type": "user”, "url": "/users/bob”, "properties": {"email": "[email protected]", "gravatar_id": "55bf19f1b9fa6ac9ab755488e29a5779”, "avatar_url": “https://gravatar.com/avatar/55bf19f1b9fa6ac9ab755488e29a5779.png?default=mm” }, "entity": { "name": "prod", "parent": null, "type": "environment”, "url": "/organizations/chef/environments/prod”, "manage_url": https://manage-chefconf-demo.opscode.com/organizations/chef/environments/prod } }
![Page 18: Chef Actions: Delightful near real-time activity tracking!](https://reader038.vdocuments.us/reader038/viewer/2022103013/53ed9f648d7f7289708b5dc6/html5/thumbnails/18.jpg)
Ruby Exampleclient = Actions::Client.new(ACTIONS_KEY)
actions = client.actions.where action: %i(create delete update), actor: 'james' resource: :cookbook, window: { recorded_at: :hourly }
{ frequency: actions.frequency, mean: actions.mean }
![Page 19: Chef Actions: Delightful near real-time activity tracking!](https://reader038.vdocuments.us/reader038/viewer/2022103013/53ed9f648d7f7289708b5dc6/html5/thumbnails/19.jpg)
R Exampleinstall.packages('chef-actions')
actions = chef.actions.where(action %in% ["create", "delete", "update"]) .where(actor %in% ["james"]) .where(resource %in% ["cookbook"]) .window(recorded_at, "hourly")
summary(actions)
![Page 20: Chef Actions: Delightful near real-time activity tracking!](https://reader038.vdocuments.us/reader038/viewer/2022103013/53ed9f648d7f7289708b5dc6/html5/thumbnails/20.jpg)
Integration points
• RabbitMQ• Consume raw messages in real-time out of Chef Server
• Notifications• Send formatted messages to external services in real-time
• Webhooks• Send whole message with annotations to your external service in real-time
• REST API• Query-only view of historical data
![Page 21: Chef Actions: Delightful near real-time activity tracking!](https://reader038.vdocuments.us/reader038/viewer/2022103013/53ed9f648d7f7289708b5dc6/html5/thumbnails/21.jpg)
Deployment
![Page 22: Chef Actions: Delightful near real-time activity tracking!](https://reader038.vdocuments.us/reader038/viewer/2022103013/53ed9f648d7f7289708b5dc6/html5/thumbnails/22.jpg)
Deployment topologies
• Supports same HA architecture as Enterprise Chef• Backend
• PostgreSQL• Frontend
• Analytics pipeline, Query API, Ingest service
• Recommended to deploy on separate hardware than Enterprise Chef
![Page 23: Chef Actions: Delightful near real-time activity tracking!](https://reader038.vdocuments.us/reader038/viewer/2022103013/53ed9f648d7f7289708b5dc6/html5/thumbnails/23.jpg)
Omnibus
• New add-on chef-analytics• Delivered as a single omnibus package• Hosted on separate domain
• E.g. analytics.getchef.com• Only interactions with Private Chef
• RabbitMQ configuration details• Manage root URL for generation of links
![Page 24: Chef Actions: Delightful near real-time activity tracking!](https://reader038.vdocuments.us/reader038/viewer/2022103013/53ed9f648d7f7289708b5dc6/html5/thumbnails/24.jpg)
Futures
![Page 25: Chef Actions: Delightful near real-time activity tracking!](https://reader038.vdocuments.us/reader038/viewer/2022103013/53ed9f648d7f7289708b5dc6/html5/thumbnails/25.jpg)
What’s Next?
• Permissions – ACL changes, group join/leave• Object Diffs• Live Feed• Aggregation (all actions from a chef-client run, berkshelf, knife run)• Knife instrumentation • Cloud plugins
![Page 26: Chef Actions: Delightful near real-time activity tracking!](https://reader038.vdocuments.us/reader038/viewer/2022103013/53ed9f648d7f7289708b5dc6/html5/thumbnails/26.jpg)
Q1 Q2 Q3 Q4Account Improvements Large Customer Features Service Provider Features HA Restructuring
Chef Actions and Run History Compliance Reporting Compliance Content
Windows Container Improvements Red Hat
The right premium features
• Make existing differentiation more accessible, address concerns of our largest customers• Extend our analytics capability to provide best of breed compliance reporting• Ensure we retain a lead on Windows support• Embrace containers as a first class part of our ecosystem
![Page 27: Chef Actions: Delightful near real-time activity tracking!](https://reader038.vdocuments.us/reader038/viewer/2022103013/53ed9f648d7f7289708b5dc6/html5/thumbnails/27.jpg)
Event Processing Architecture
![Page 28: Chef Actions: Delightful near real-time activity tracking!](https://reader038.vdocuments.us/reader038/viewer/2022103013/53ed9f648d7f7289708b5dc6/html5/thumbnails/28.jpg)
Chef Actions
• We know there is valuable data locked up inside your Chef Server• Chef Actions exposes that data to you via a variety of APIs • We have built a flexible architecture that will allow us to expose more and more data
• Available today – Limited availability• No release schedule – continuous updates• We’ll rapidly open to more and more customers
• Defined roadmap for 2014 for compliance and audit
![Page 29: Chef Actions: Delightful near real-time activity tracking!](https://reader038.vdocuments.us/reader038/viewer/2022103013/53ed9f648d7f7289708b5dc6/html5/thumbnails/29.jpg)