vertebra: bringing the cloud down to earth
DESCRIPTION
TRANSCRIPT
![Page 1: Vertebra: Bringing The Cloud Down To Earth](https://reader033.vdocuments.us/reader033/viewer/2022051817/5484f679b47959190d8b4d05/html5/thumbnails/1.jpg)
VERTEBRABringing the Cloud Down To Earth
Jayson VantuylChief Systems Architect
Engine Yard, Inc.
☁☔
![Page 2: Vertebra: Bringing The Cloud Down To Earth](https://reader033.vdocuments.us/reader033/viewer/2022051817/5484f679b47959190d8b4d05/html5/thumbnails/2.jpg)
Who Is This Guy?
Jayson Vantuyl
Chief Systems Architect / Founder
Architecting Large Networks Since 1994
Engine Yard, Inc.
Large Hosting Provider focused on hosting Ruby Applications
Managing Over 2,000 Virtual Servers
![Page 3: Vertebra: Bringing The Cloud Down To Earth](https://reader033.vdocuments.us/reader033/viewer/2022051817/5484f679b47959190d8b4d05/html5/thumbnails/3.jpg)
The Classic Blockers
My existing resources are not in the cloud.
A critical component is not in the cloud.
Critical components don’t fit together.
I’m on a budget: small first, scale later.
My boss wants it all (or some of it) in-house.
![Page 4: Vertebra: Bringing The Cloud Down To Earth](https://reader033.vdocuments.us/reader033/viewer/2022051817/5484f679b47959190d8b4d05/html5/thumbnails/4.jpg)
Fog Computing
It’s a Cloud, but it’s “down to Earth”. Fog, get it?
Focus on What, Not Where
Clouds Without Borders
Don’t Rain On My Parade
A Sky Full Of Clouds
![Page 5: Vertebra: Bringing The Cloud Down To Earth](https://reader033.vdocuments.us/reader033/viewer/2022051817/5484f679b47959190d8b4d05/html5/thumbnails/5.jpg)
The Nuts and Bolts
Agents
XMPP
Discovery
Security
XMPP
Security
&
Discovery
Agents
Resource
Database
Workflow
Engine
![Page 6: Vertebra: Bringing The Cloud Down To Earth](https://reader033.vdocuments.us/reader033/viewer/2022051817/5484f679b47959190d8b4d05/html5/thumbnails/6.jpg)
XMPP
XML Messaging and Presence Protocol
IETF Standard (RFC 3920, 3921)
Each Connection Is Essentially Two Streaming XML Documents
Authentication
Transport Security
Ordered Message Delivery, Presence
Many Implementations, Designed To Be Extensible
Server-to-Server Security via Dial-Back
Application-Level Reliability via SRV DNS Records
![Page 7: Vertebra: Bringing The Cloud Down To Earth](https://reader033.vdocuments.us/reader033/viewer/2022051817/5484f679b47959190d8b4d05/html5/thumbnails/7.jpg)
Resources
Every point of control in the system is described by a set of a “resources” (much like a URL).
These resources need not be unique in the cloud.
They are organized into hierarchies.
They are the point of abstraction.
![Page 8: Vertebra: Bringing The Cloud Down To Earth](https://reader033.vdocuments.us/reader033/viewer/2022051817/5484f679b47959190d8b4d05/html5/thumbnails/8.jpg)
Some Possible Resources
Each box is an agent that offers operations on the listed resources
Geography
Organizational Structure
Security Clearance
Type Of Service
[email protected]:/us/ny/new_york/sales/copy_writer
[email protected]:/ca/qc/montreal/sales/rep
[email protected]:/us/dc/washington/research/weapons/clearance/s/clearance/c/clearance/ers/clearance/rs
[email protected]:/us/fl/tampa/ops/helpdesk
[email protected]:/net/tcpip/server/http/blog
[email protected]:/net/tcpip/server/smtp/service/mailer
[email protected]:/hr/ca
[email protected]:/security/ca
[email protected]:/security/us
[email protected]:/hr/us
![Page 9: Vertebra: Bringing The Cloud Down To Earth](https://reader033.vdocuments.us/reader033/viewer/2022051817/5484f679b47959190d8b4d05/html5/thumbnails/9.jpg)
Operations
Fundamental Unit Of Work In Vertebra
Shaped Like Remote Procedure Call
Receives Parameters In A Namespace
Discover Agents That Provide The Resources
Request — Ack — Result* — Done
Scope (i.e. shotgun or sniper?)
![Page 10: Vertebra: Bringing The Cloud Down To Earth](https://reader033.vdocuments.us/reader033/viewer/2022051817/5484f679b47959190d8b4d05/html5/thumbnails/10.jpg)
Example Operations
Completely Unrealistic Example
Let’s Fire Some People, The Entire Sales Dept.
Then Blog About It
[email protected]:/us/ny/new_york/sales/copy_writer
[email protected]:/ca/qc/montreal/sales/rep
[email protected]:/net/tcpip/server/http/blog
[email protected]:/security/ca
[email protected]:/security/us
[email protected]:/manage/ca
[email protected]:/manage/us
fire(who=/manage/sales)
kick_out(who=/security/sales)
add_entry(site=/blog,message=”Fired Sales!”)
instant_message(who=/sales,msg=”Goodbye.”)
![Page 11: Vertebra: Bringing The Cloud Down To Earth](https://reader033.vdocuments.us/reader033/viewer/2022051817/5484f679b47959190d8b4d05/html5/thumbnails/11.jpg)
An Ominous Message
“who” is a resource, discovery is done on /sales
We discover [email protected] and [email protected]
Both get “instant_message” operation, and the agent on their computer delivers the message, “Goodbye.”
[email protected]:/us/ny/new_york/sales/copy_writer
[email protected]:/ca/qc/montreal/sales/rep
[email protected]:/net/tcpip/server/http/blog
[email protected]:/security/ca
[email protected]:/security/us
[email protected]:/manage/ca
[email protected]:/manage/us
fire(who=/manage/sales)
kick_out(who=/security/sales)
add_entry(site=/blog,message=”Fired Sales!”)
instant_message(who=/sales,msg=”Goodbye.”)
![Page 12: Vertebra: Bringing The Cloud Down To Earth](https://reader033.vdocuments.us/reader033/viewer/2022051817/5484f679b47959190d8b4d05/html5/thumbnails/12.jpg)
You’re Fired!
“who” is a resource, discovery is done on/manage/sales
We discover us_hr and ca_hr, which both offer/manage
the “fire” operation goes to the the HR departments for both countries
[email protected]:/us/ny/new_york/sales/copy_writer
[email protected]:/ca/qc/montreal/sales/rep
[email protected]:/net/tcpip/server/http/blog
[email protected]:/security/ca
[email protected]:/security/us
[email protected]:/manage/ca
[email protected]:/manage/us
fire(who=/manage/sales)
kick_out(who=/security/sales)
add_entry(site=/blog,message=”Fired Sales!”)
instant_message(who=/sales,msg=”Goodbye.”)
![Page 13: Vertebra: Bringing The Cloud Down To Earth](https://reader033.vdocuments.us/reader033/viewer/2022051817/5484f679b47959190d8b4d05/html5/thumbnails/13.jpg)
Kick ‘Em Out!
“who” is a resource, discovery is done on/security/sales
We discover us_sec and ca_sec, which provide/security
Security (in both countries) gets the operation to kick out everybody that matches /sales
[email protected]:/us/ny/new_york/sales/copy_writer
[email protected]:/ca/qc/montreal/sales/rep
[email protected]:/net/tcpip/server/http/blog
[email protected]:/security/ca
[email protected]:/security/us
[email protected]:/manage/ca
[email protected]:/manage/us
fire(who=/manage/sales)
kick_out(who=/security/sales)
add_entry(site=/blog,message=”Fired Sales!”)
instant_message(who=/sales,msg=”Goodbye.”)
![Page 14: Vertebra: Bringing The Cloud Down To Earth](https://reader033.vdocuments.us/reader033/viewer/2022051817/5484f679b47959190d8b4d05/html5/thumbnails/14.jpg)
Gloat!
“site” is a resource, discovery is performed on/blog
We discover [email protected], which provides /blog
The operation to post the blog entry goes to the web server agent.
[email protected]:/us/ny/new_york/sales/copy_writer
[email protected]:/ca/qc/montreal/sales/rep
[email protected]:/net/tcpip/server/http/blog
[email protected]:/security/ca
[email protected]:/security/us
[email protected]:/manage/ca
[email protected]:/manage/us
fire(who=/manage/sales)
kick_out(who=/security/sales)
add_entry(site=/blog,message=”Fired Sales!”)
instant_message(who=/sales,msg=”Goodbye.”)
![Page 15: Vertebra: Bringing The Cloud Down To Earth](https://reader033.vdocuments.us/reader033/viewer/2022051817/5484f679b47959190d8b4d05/html5/thumbnails/15.jpg)
Get Your Head In The Cloud
Operations in many agents with one request
Replace the “sales” entry with “us” to fire the whole US workforce
Replace “sales” with “us/ny” to fire the New York office
Can Subdivide the HR office any time and just offer the correct resources
![Page 16: Vertebra: Bringing The Cloud Down To Earth](https://reader033.vdocuments.us/reader033/viewer/2022051817/5484f679b47959190d8b4d05/html5/thumbnails/16.jpg)
Get Your Head In The Cloud
The agents are abstracted.
HR in the US might be done by human with some CRM.
HR in Canada might be automated, or might be paper-only, with people doing the work.
Same for security, and the web server.
![Page 17: Vertebra: Bringing The Cloud Down To Earth](https://reader033.vdocuments.us/reader033/viewer/2022051817/5484f679b47959190d8b4d05/html5/thumbnails/17.jpg)
Reduced To Set NotationIn the abstract, a resource is a set that maps to a set of agents
Resources may be singly inherited
The set that maps to a resource is a subset of the set that maps to its parent
There are useful operations on sets of resources, such as “coverage”
For any operation, the set of discovered agents is the union of the sets mapped to by the intersection of the set of requested resources and set of actual resources
![Page 18: Vertebra: Bringing The Cloud Down To Earth](https://reader033.vdocuments.us/reader033/viewer/2022051817/5484f679b47959190d8b4d05/html5/thumbnails/18.jpg)
What Do We Get Out Of This?
Computational Patterns
map-reduce
traditional RPC
message passing
job queue
observer
Component Abstraction
Same Resources, Different Code
Same Resources, Same Code
Different Resources, Same Code
![Page 19: Vertebra: Bringing The Cloud Down To Earth](https://reader033.vdocuments.us/reader033/viewer/2022051817/5484f679b47959190d8b4d05/html5/thumbnails/19.jpg)
What I Didn’t Tell You About
Responses from Operations and How They Aggregate
The RBAC Security Implementation
The Workflow Engine
The Resource-Keyed Database
Intrinsic I18n
Extensible Data Marshaling
Resource Advertising
Federating Clouds
Fault-Tolerant, Distributed Core Implementation
Monitoring Protocol
Job Management Protocol
Distributed Audit System
Distributed Log System
![Page 20: Vertebra: Bringing The Cloud Down To Earth](https://reader033.vdocuments.us/reader033/viewer/2022051817/5484f679b47959190d8b4d05/html5/thumbnails/20.jpg)
Reliability, Ease of Use, Scalability
This innovation brought to you by