Container as a Service support for Apache libcloud

Anthony Shaw, Apache LibCloud PMC

Overview of the LibCloud project

Python library, supports 2.5+, 3.0+ Formed in 2009 Graduated to Apache TLP in May 2011 200k+ downloads a month on PyPi 154 contributors

LibCloud is an open-source library providing a single interface to communicate with multiple clouds, public or private. Supports IaaS, LBaaS, DNS and Storage.

About me

@anthonypjshaw

@tonybaloney

anthonyshaw@apache.org

Based in Sydney, Australia

Head of Innovation for Dimension Data

Why should you use LibCloud?My workloads run in private and public clouds, I’ve got scripts for

both

LibCloud supports all major public clouds and private

hypervisor APIs. Consolidate your scripts into 1 tool

Supported Clouds (IaaS)

Current driverslibcloud.computeCompute (v0.1.0)

• Support for nodes, node images, locations, states• 52 providers including every major cloud provider in the market. Plus local services like Vmware, OpenStack, libvirt

libcloud.dnsDNS (v0.6.0)• Support for zones, records, recordtypes• 19 providers including CloudFlare, DigitalOcean, DNSimple, GoDaddy, Google DNS, Linode, Rackspace, Amazon R53, Zerigo

libcloud.storageObject Storage (v0.5.0)• Support for containers and objects• 11 providers including Amazon S3, Azure Blobs, Google storage, CloudFiles, OpenStack Swift

libcloud.loadbalancerLoad Balancer (v0.5.0)• Support for nodes, balancers, listeners and algorithms• 11 providers including CloudStack, Dimension Data, Amazon ELB, Google GCE LB, SoftLayer LB

libcloud.backupBackup (v0.20.0)• Support for backup targets, recovery points and jobs• 3 providers, Dimension Data, Amazon EBS snaps, Google snaps

Why would we need a container API? The API is for Container-as-a-Service providers, these new

types of cloud services offer container management and hosting as a service.

The new services are already providing proprietary APIs, giving the need for a tool like Libcloud if you want to provision to any cloud provider

Isn’t Docker a standard?Well, yes and no.Docker has been the main technology adopted by these providers as the host system for the containers and also as the specification of the containers themselves.But, Docker is not a provisioning system, it is a virtualization host. Also there are alternatives, like CoreOS Rkt.

Container DriversList container images, or load from external systems

Create clusters for load-balanced containers (where supported)

List, destroy, start and stop persistent containersContainer

Driver

Basic functionality

Containers Container Images

Cluster functionality (optional)

Clusters

Extended functions

Example Codefrom libcloud.container.providers import get_driverfrom libcloud.container.types import Provider

Cls = get_driver(Provider.DOCKER)driver = Cls('user', 'api key')

image = driver.install_image('tomcat:8.0')container = driver.deploy_container('tomcat', image)

container.restart()

Docker Registry The Docker Registry API is used by services like Amazon ECR, the

Docker Hub website and by anyone hosting their own Docker registry.

It doesn’t belong to a particular driver, so is a utility class Some providers, like Amazon ECR have a factory method to provide

a registry client Images from docker registry can be sent to the deploy_container

method for any driver.from libcloud.container.utils.docker import HubClient hub = HubClient() image = hub.get_image('ubuntu', 'latest')

# Get a Registry API client for an existing repositoryclient = conn.ex_get_registry_client('my-image')image = client.get_image('ubuntu', 'latest')

Driver : Docker

from libcloud.container.types import Providerfrom libcloud.container.providers import get_driver

cls = get_driver(Provider.DOCKER)

conn = cls(host='https://198.61.239.128', port=4243, key_file='key.pem', cert_file='cert.pem')

conn.list_containers()

Extra functionality: • Get logs for container• Delete an image• List processes running inside a container• Rename a container• Search for images on docker.io

Base functionality• Install an image from docker hub, or a private

repository• Deploy a container from image• Start, Stop, Restart, Delete a container

Driver : Joyent Triton

from libcloud.container.types import Providerfrom libcloud.container.providers import get_driver

cls = get_driver(Provider.JOYENT)

conn = cls(host='us-east-1.docker.joyent.com', port=2376, key_file='key.pem', cert_file='~/.sdc/docker/admin/ca.pem')

conn.list_containers()

Extra functionality: • Get logs for container• Delete an image• List processes running inside a container• Rename a container• Search for images on docker.io

Base functionality• Install an image from docker hub, or a private

repository• Deploy a container from image• Start, Stop, Restart, Delete a container

Joyent provide hosted and managed Docker hosts as a service

Driver : Amazon ECS

from libcloud.container.types import Providerfrom libcloud.container.providers import get_drivercls = get_driver(Provider.ECS)conn = cls(access_id='SDHFISJDIFJSIDFJ', secret='THIS_IS)+_MY_SECRET_KEY+I6TVkv68o4H', region='ap-southeast-2')conn.list_containers()

Extra functionality: • Create a service (a collection of containers in a

cluster).• Describe existing services• Get ECR registry client

Base functionality• Install an image from docker hub, or a private

repository, or from Amazon ECR (registry)• Set CPU and memory reservations for

containers• Deploy a container from image• Start, Stop, Restart, Delete a container

Elastic Container Service is a container-as-a-service feature of AWS

Cluster functionality• Create a cluster• Deploy a container into a cluster• Destroy an existing cluster

Driver : Google Kubernetes

from libcloud.container.types import Providerfrom libcloud.container.providers import get_drivercls = get_driver(Provider.KUBERNETES)conn = cls(key='my_username', secret='THIS_IS)+_MY_SECRET_KEY+I6TVkv68o4H', host='126.32.21.4')conn.list_containers()

Extra functionality: • Create namespaces• Deploy pods into namespaces• Destroy pods

Base functionality• Install an image from docker hub, or a private

repository• Deploy a container from image• Discover containers within all pods

Kubernetes is an open source orchestration system for Docker containers. It handles scheduling onto nodes in a compute cluster and actively manages workloads to ensure that their state matches the users declared intentions. It groups the containers which make up an application into logical units for easy management and discovery.

Cluster functionality• Create a cluster (a namespace)• Deploy a container into a cluster• Destroy an existing cluster

What now? Support for Google Cloud’s Container Engine (based on

Kubernetes so most of the way there) https://cloud.google.com/container-engine/

Extend Kubernetes support to allow multiple containers to be provisioned to a single pod

Create a CoreOS/Rkt driver Support other providers as they join the market

The Libcloud ecosystem

Mix and match

Compute Storage DNS

Application workload

To take full advantage of the LibCloud ecosystem, deploy your application across multiple providers, choose the best platform(s) for the job.

Compute DNS Storage Load Balancer

Other ways of consuming LibCloudOrchestration Tools Management UIs

Development Tooling

Salt Stack CloudLeverage the flexibility and breadth of the LibCloud driver support from Salt Stack

Contributing to LibCloud

Fork + code

Raise Pull Request

Merge!Test + flake

github.com/apache/libcloud