![Page 1: Better, Faster, Cheaper Infrastructure: Apache CloudStack and Riak CS](https://reader038.vdocuments.us/reader038/viewer/2022103110/54b73eb14a795966598b46f7/html5/thumbnails/1.jpg)
BETTER, FASTER, CHEAPER INFRASTRUCTURE: APACHE
CLOUDSTACK + RIAK CSJohn Burwell
([email protected] | @john_burwell)
Friday, November 8, 13
![Page 2: Better, Faster, Cheaper Infrastructure: Apache CloudStack and Riak CS](https://reader038.vdocuments.us/reader038/viewer/2022103110/54b73eb14a795966598b46f7/html5/thumbnails/2.jpg)
Who Am I
• Consulting Engineer @ Basho Technologies
• Member, Apache CloudStack PMC
• Ran operations and designed automated provisioning for hybrid analytic/virtualization clouds
• Led architectural design and server-side development of a SaaS physical security platform
Friday, November 8, 13
![Page 3: Better, Faster, Cheaper Infrastructure: Apache CloudStack and Riak CS](https://reader038.vdocuments.us/reader038/viewer/2022103110/54b73eb14a795966598b46f7/html5/thumbnails/3.jpg)
... but I submitted a ticket.
Friday, November 8, 13
![Page 4: Better, Faster, Cheaper Infrastructure: Apache CloudStack and Riak CS](https://reader038.vdocuments.us/reader038/viewer/2022103110/54b73eb14a795966598b46f7/html5/thumbnails/4.jpg)
Shifting Role Of IT
From Cost Center To Business Opportunity
Friday, November 8, 13
![Page 5: Better, Faster, Cheaper Infrastructure: Apache CloudStack and Riak CS](https://reader038.vdocuments.us/reader038/viewer/2022103110/54b73eb14a795966598b46f7/html5/thumbnails/5.jpg)
Shifting Role Of IT
From Cost Center To Business Opportunity
Infrastructure Be Boring
Friday, November 8, 13
![Page 6: Better, Faster, Cheaper Infrastructure: Apache CloudStack and Riak CS](https://reader038.vdocuments.us/reader038/viewer/2022103110/54b73eb14a795966598b46f7/html5/thumbnails/6.jpg)
Better
• Resilience
• Scalability
• Service
Friday, November 8, 13
![Page 7: Better, Faster, Cheaper Infrastructure: Apache CloudStack and Riak CS](https://reader038.vdocuments.us/reader038/viewer/2022103110/54b73eb14a795966598b46f7/html5/thumbnails/7.jpg)
Faster
• Resource Provisioning
• Systems
• Failure recovery
Friday, November 8, 13
![Page 8: Better, Faster, Cheaper Infrastructure: Apache CloudStack and Riak CS](https://reader038.vdocuments.us/reader038/viewer/2022103110/54b73eb14a795966598b46f7/html5/thumbnails/8.jpg)
Cheaper
• Hardware
• Labor
• Hosting
Friday, November 8, 13
![Page 9: Better, Faster, Cheaper Infrastructure: Apache CloudStack and Riak CS](https://reader038.vdocuments.us/reader038/viewer/2022103110/54b73eb14a795966598b46f7/html5/thumbnails/9.jpg)
Composable Infrastructure
• Resource Pooling
• Rapid Elasticity
• Self Service
• Metering/Charge Back
Friday, November 8, 13
![Page 10: Better, Faster, Cheaper Infrastructure: Apache CloudStack and Riak CS](https://reader038.vdocuments.us/reader038/viewer/2022103110/54b73eb14a795966598b46f7/html5/thumbnails/10.jpg)
Composable Infrastructure
• Resource Pooling
• Rapid Elasticity
• Self Service
• Metering/Charge Back
Late-binding of hardware roles
Friday, November 8, 13
![Page 11: Better, Faster, Cheaper Infrastructure: Apache CloudStack and Riak CS](https://reader038.vdocuments.us/reader038/viewer/2022103110/54b73eb14a795966598b46f7/html5/thumbnails/11.jpg)
Got It ... Why Go Private?
• Leverage existing infrastructure investment
• Customization and Control
• Regulatory/Security Requirements
• Cost
Friday, November 8, 13
![Page 12: Better, Faster, Cheaper Infrastructure: Apache CloudStack and Riak CS](https://reader038.vdocuments.us/reader038/viewer/2022103110/54b73eb14a795966598b46f7/html5/thumbnails/12.jpg)
Doesn’t vSphere/XCP/XenServer already do that?
Friday, November 8, 13
![Page 13: Better, Faster, Cheaper Infrastructure: Apache CloudStack and Riak CS](https://reader038.vdocuments.us/reader038/viewer/2022103110/54b73eb14a795966598b46f7/html5/thumbnails/13.jpg)
Friday, November 8, 13
![Page 14: Better, Faster, Cheaper Infrastructure: Apache CloudStack and Riak CS](https://reader038.vdocuments.us/reader038/viewer/2022103110/54b73eb14a795966598b46f7/html5/thumbnails/14.jpg)
Hopefully not ...
Friday, November 8, 13
![Page 15: Better, Faster, Cheaper Infrastructure: Apache CloudStack and Riak CS](https://reader038.vdocuments.us/reader038/viewer/2022103110/54b73eb14a795966598b46f7/html5/thumbnails/15.jpg)
Friday, November 8, 13
![Page 16: Better, Faster, Cheaper Infrastructure: Apache CloudStack and Riak CS](https://reader038.vdocuments.us/reader038/viewer/2022103110/54b73eb14a795966598b46f7/html5/thumbnails/16.jpg)
Traditional Workload
Management
Network
HypervisorCluster
HypervisorCluster
Shared����������� ������������������ Storage
• Reliable
• Fault tolerant
• Specialized hardware
• IT managed provisioning
• Static scale out model
Friday, November 8, 13
![Page 17: Better, Faster, Cheaper Infrastructure: Apache CloudStack and Riak CS](https://reader038.vdocuments.us/reader038/viewer/2022103110/54b73eb14a795966598b46f7/html5/thumbnails/17.jpg)
Traditional Workload
Management
Network
HypervisorCluster
HypervisorCluster
Shared����������� ������������������ Storage
• Reliable
• Fault tolerant
• Specialized hardware
• IT managed provisioning
• Static scale out model
Focused on resource consolidation not agility
Friday, November 8, 13
![Page 18: Better, Faster, Cheaper Infrastructure: Apache CloudStack and Riak CS](https://reader038.vdocuments.us/reader038/viewer/2022103110/54b73eb14a795966598b46f7/html5/thumbnails/18.jpg)
Software Is Eating Hardware
• Moore’s Law has pulled fault tolerance up the stack
• Commodity Hardware
• Reduced cost
• Services more workload types
• Dynamic resource allocation
Friday, November 8, 13
![Page 19: Better, Faster, Cheaper Infrastructure: Apache CloudStack and Riak CS](https://reader038.vdocuments.us/reader038/viewer/2022103110/54b73eb14a795966598b46f7/html5/thumbnails/19.jpg)
Software Is Eating Hardware
• Moore’s Law has pulled fault tolerance up the stack
• Commodity Hardware
• Reduced cost
• Services more workload types
• Dynamic resource allocation
Specialized Hardware Reduces Agility
Friday, November 8, 13
![Page 20: Better, Faster, Cheaper Infrastructure: Apache CloudStack and Riak CS](https://reader038.vdocuments.us/reader038/viewer/2022103110/54b73eb14a795966598b46f7/html5/thumbnails/20.jpg)
Cloud Workload• Reliable
• Fault Tolerant
• Commodity Hardware
• Self-Service provisioning
• Control Plane -> Dynamic scale out model
SDN
Rack Rack
Object����������� ������������������ Storage
App App
Friday, November 8, 13
![Page 21: Better, Faster, Cheaper Infrastructure: Apache CloudStack and Riak CS](https://reader038.vdocuments.us/reader038/viewer/2022103110/54b73eb14a795966598b46f7/html5/thumbnails/21.jpg)
Cloud Workload• Reliable
• Fault Tolerant
• Commodity Hardware
• Self-Service provisioning
• Control Plane -> Dynamic scale out model
SDN
Rack Rack
Object����������� ������������������ Storage
App App
Resource optimization increases agility
Friday, November 8, 13
![Page 22: Better, Faster, Cheaper Infrastructure: Apache CloudStack and Riak CS](https://reader038.vdocuments.us/reader038/viewer/2022103110/54b73eb14a795966598b46f7/html5/thumbnails/22.jpg)
TraditionalWorkload
CloudWorkload
Reality
Friday, November 8, 13
![Page 23: Better, Faster, Cheaper Infrastructure: Apache CloudStack and Riak CS](https://reader038.vdocuments.us/reader038/viewer/2022103110/54b73eb14a795966598b46f7/html5/thumbnails/23.jpg)
Complete Private Cloud
+
Compute Services Object Storage Services
Friday, November 8, 13
![Page 24: Better, Faster, Cheaper Infrastructure: Apache CloudStack and Riak CS](https://reader038.vdocuments.us/reader038/viewer/2022103110/54b73eb14a795966598b46f7/html5/thumbnails/24.jpg)
What Is Apache CloudStack?
§Open source software that deploys and manages large networks of virtual machines, as a highly available, highly scalable Infrastructure as a Service (IaaS) cloud computing platform.
§CloudStack is a Top-Level Project at the Apache Software Foundation.
Friday, November 8, 13
![Page 25: Better, Faster, Cheaper Infrastructure: Apache CloudStack and Riak CS](https://reader038.vdocuments.us/reader038/viewer/2022103110/54b73eb14a795966598b46f7/html5/thumbnails/25.jpg)
What Is Apache CloudStack?
§Open source software that deploys and manages large networks of virtual machines, as a highly available, highly scalable Infrastructure as a Service (IaaS) cloud computing platform.
§CloudStack is a Top-Level Project at the Apache Software Foundation.
Friday, November 8, 13
![Page 26: Better, Faster, Cheaper Infrastructure: Apache CloudStack and Riak CS](https://reader038.vdocuments.us/reader038/viewer/2022103110/54b73eb14a795966598b46f7/html5/thumbnails/26.jpg)
~300: Acknowledged production implementations
~43,000: Hosts in a single CloudStack instance
~85: Project committers
Friday, November 8, 13
![Page 27: Better, Faster, Cheaper Infrastructure: Apache CloudStack and Riak CS](https://reader038.vdocuments.us/reader038/viewer/2022103110/54b73eb14a795966598b46f7/html5/thumbnails/27.jpg)
Apache CloudStack Users
Friday, November 8, 13
![Page 28: Better, Faster, Cheaper Infrastructure: Apache CloudStack and Riak CS](https://reader038.vdocuments.us/reader038/viewer/2022103110/54b73eb14a795966598b46f7/html5/thumbnails/28.jpg)
Anatomy Of CloudStack• Management Server Farm
• Management and provisioning tasks
• Zone
• Collection of pods, network offerings and secondary storage
• Pod
• Collection of clusters in the same failure boundary
• Cluster
• A grouping of hosts and their associated storage
• Hosts
• Servers onto which services will be provisioned
• Primary Storage
• VM disk storage
• Network
• Logical network associated with service offerings
• Secondary Storage
• Template, snapshot and ISO storage
Zone
CloudStack Pod
Cluster
Host
Host
Network
PrimaryStorage
VM
VM
CloudStack Pod
Cluster
SecondaryStorage
Friday, November 8, 13
![Page 29: Better, Faster, Cheaper Infrastructure: Apache CloudStack and Riak CS](https://reader038.vdocuments.us/reader038/viewer/2022103110/54b73eb14a795966598b46f7/html5/thumbnails/29.jpg)
Management Server
• Orchestrates infrastructure management
• A single management server instance can manage 10,000 hosts
• For production, run n +1 management servers with at least one instance in each zone
• Cluster MySQL
Friday, November 8, 13
![Page 30: Better, Faster, Cheaper Infrastructure: Apache CloudStack and Riak CS](https://reader038.vdocuments.us/reader038/viewer/2022103110/54b73eb14a795966598b46f7/html5/thumbnails/30.jpg)
Control And Data Planes
• Automation control of resources (e.g. hypervisors, storage, switches, etc)
• Small, linear bandwidth consumption
• Low latency
• Transfer of data between infrastructure components (e.g. templates, snapshots, etc)
• Large, variable bandwidth consumption
• High throughput
Friday, November 8, 13
![Page 31: Better, Faster, Cheaper Infrastructure: Apache CloudStack and Riak CS](https://reader038.vdocuments.us/reader038/viewer/2022103110/54b73eb14a795966598b46f7/html5/thumbnails/31.jpg)
Control And Data Plane Scalability
• Separate the control and data plane compute and network traffic
• Data transfer operations offloaded to system virtual machines
• Separate management and storage networks
Friday, November 8, 13
![Page 32: Better, Faster, Cheaper Infrastructure: Apache CloudStack and Riak CS](https://reader038.vdocuments.us/reader038/viewer/2022103110/54b73eb14a795966598b46f7/html5/thumbnails/32.jpg)
Primary Vs Secondary Storage
• Random access read/write
• Changes often
• Zone Scoped
• Focused on throughput
• Write once/read many
• Append only
• Zone or Region Scoped
• Focused on reliability
Friday, November 8, 13
![Page 33: Better, Faster, Cheaper Infrastructure: Apache CloudStack and Riak CS](https://reader038.vdocuments.us/reader038/viewer/2022103110/54b73eb14a795966598b46f7/html5/thumbnails/33.jpg)
Primary Vs Secondary Storage
• Random access read/write
• Changes often
• Zone Scoped
• Focused on throughput
• Write once/read many
• Append only
• Zone or Region Scoped
• Focused on reliability
Secondary Storage is the lifeboat.
Friday, November 8, 13
![Page 34: Better, Faster, Cheaper Infrastructure: Apache CloudStack and Riak CS](https://reader038.vdocuments.us/reader038/viewer/2022103110/54b73eb14a795966598b46f7/html5/thumbnails/34.jpg)
What Is Riak CS?• Simple, reliable storage of large objects accessed using the S3 or Swift
API
• Objects are stored in buckets with a unique key
• Multi-tenant security model with per object ACLs
• Usage Accounting and Reporting
• Open Source (Apache2 Licensed)
Friday, November 8, 13
![Page 35: Better, Faster, Cheaper Infrastructure: Apache CloudStack and Riak CS](https://reader038.vdocuments.us/reader038/viewer/2022103110/54b73eb14a795966598b46f7/html5/thumbnails/35.jpg)
What Is Riak CS?• Simple, reliable storage of large objects accessed using the S3 or Swift
API
• Objects are stored in buckets with a unique key
• Multi-tenant security model with per object ACLs
• Usage Accounting and Reporting
• Open Source (Apache2 Licensed)
Your Own Private S3
Friday, November 8, 13
![Page 36: Better, Faster, Cheaper Infrastructure: Apache CloudStack and Riak CS](https://reader038.vdocuments.us/reader038/viewer/2022103110/54b73eb14a795966598b46f7/html5/thumbnails/36.jpg)
Anatomy Of Riak CS
• S3/Swift API Proxy + Riak key-value store
• Peer-to-Peer Design: Any node can service any read or accept any write request
• Maintains multiple replicas (default 3) spread across multiple nodes
• Ops Friendly
Friday, November 8, 13
![Page 37: Better, Faster, Cheaper Infrastructure: Apache CloudStack and Riak CS](https://reader038.vdocuments.us/reader038/viewer/2022103110/54b73eb14a795966598b46f7/html5/thumbnails/37.jpg)
Anatomy Of Riak CS
• S3/Swift API Proxy + Riak key-value store
• Peer-to-Peer Design: Any node can service any read or accept any write request
• Maintains multiple replicas (default 3) spread across multiple nodes
• Ops Friendly
Horizontally scaleable storage on commodity hardware
Friday, November 8, 13
![Page 38: Better, Faster, Cheaper Infrastructure: Apache CloudStack and Riak CS](https://reader038.vdocuments.us/reader038/viewer/2022103110/54b73eb14a795966598b46f7/html5/thumbnails/38.jpg)
Riak CS Use Cases
• Secondary Storage for VM snapshot and template data
• Database Backup Storage
• Serving Static Content
• Application Log Storage
Friday, November 8, 13
![Page 39: Better, Faster, Cheaper Infrastructure: Apache CloudStack and Riak CS](https://reader038.vdocuments.us/reader038/viewer/2022103110/54b73eb14a795966598b46f7/html5/thumbnails/39.jpg)
Achieving Commodity Reliability
• Create a zone per power source/backbone network connection
• Establish independent primary storage in each zone
• Utilize intra-datacenter block-level replication between zones
• Distribute Riak CS nodes evenly across zones
Friday, November 8, 13
![Page 40: Better, Faster, Cheaper Infrastructure: Apache CloudStack and Riak CS](https://reader038.vdocuments.us/reader038/viewer/2022103110/54b73eb14a795966598b46f7/html5/thumbnails/40.jpg)
The Complete Private Cloud
Management
Network
HypervisorCluster
HypervisorCluster
Shared����������� ������������������ Storage
SDN
Rack Rack
Riak����������� ������������������ CS
App App
Traditional����������� ������������������ Cluster Cloud����������� ������������������ Workload
Friday, November 8, 13
![Page 41: Better, Faster, Cheaper Infrastructure: Apache CloudStack and Riak CS](https://reader038.vdocuments.us/reader038/viewer/2022103110/54b73eb14a795966598b46f7/html5/thumbnails/41.jpg)
Up And Running With Riak CS
• Learn more @ http://basho.com/riak-cloud-storage
• Grab it @ http://docs.basho.com/riakcs/latest/riakcs-downloads/
• Follow us @basho
• Help from the [email protected] mailing list or on FreeNode @ #riak
• Contribute @ http://github.com/basho/riak_cs
Friday, November 8, 13
![Page 42: Better, Faster, Cheaper Infrastructure: Apache CloudStack and Riak CS](https://reader038.vdocuments.us/reader038/viewer/2022103110/54b73eb14a795966598b46f7/html5/thumbnails/42.jpg)
Join Our Community
• Learn more @ http://cloudstack.apache.org
• Grab it @ http://cloudstack.apache.org/downloads.html
• Follow us @cloudstack
• Help from the [email protected] mailing list or on Freenode @ #cloudstack
• Contribute on the [email protected] mailing list or on Freenode @ #cloudstack-dev
Friday, November 8, 13
![Page 43: Better, Faster, Cheaper Infrastructure: Apache CloudStack and Riak CS](https://reader038.vdocuments.us/reader038/viewer/2022103110/54b73eb14a795966598b46f7/html5/thumbnails/43.jpg)
Thoughts? Questions?
Friday, November 8, 13
![Page 44: Better, Faster, Cheaper Infrastructure: Apache CloudStack and Riak CS](https://reader038.vdocuments.us/reader038/viewer/2022103110/54b73eb14a795966598b46f7/html5/thumbnails/44.jpg)
Thank you!
Deck http://slideshare.net/JohnBurwell1Follow @ john_burwell
Email [email protected]
Friday, November 8, 13