docker meetup - melbourne 2015 - kubernetes deep dive
TRANSCRIPT
Networking & Storage Deep Dive
Ken ThompsonSenior Cloud Solution ArchitectMelbourne Docker Meetup - October 2015
KUBERNETES: THE HELMSMEN
Melbourne Docker Meetup Oct 2015 - Kubernetes Networking & Storage Deep Dive2
AgendaKubernetes Deep Dive
● Kubernetes Networking● Kubernetes Persistent Volumes● Live Demos:
– Persistent Storage– Something Fun!...
Melbourne Docker Meetup Oct 2015 - Kubernetes Networking & Storage Deep Dive3
Container orchestration at scale
KUBERNETES
● Scale workloads across a cluster ofcontainer hosts
● Declarative state of environmentthat gets managed
● Intelligent policy-based scheduling● Multi-host networking support (with
OpenShift-SDN/OvSwitch etc.)● Multi-host shared storage for
containers● Image update mechanisms (with
OpenShift Deployment Configs -rolling update, a-b, etc.)
Melbourne Docker Meetup Oct 2015 - Kubernetes Networking & Storage Deep Dive
4
KUBERNETES DEEP DIVE:NETWORKING
Melbourne Docker Meetup Oct 2015 - Kubernetes Networking & Storage Deep Dive5
DOCKER IN-HOST NETWORKINGSuitable for single-host, not for multi-host
● Docker assigns an IP to eachcontainer
● Its local and private only by default● Complex for multi-host● Plugins starting to become available● Now we have a container..hold on!
– What happens If we want tohave containers sharingfilesystem
– Or IPC communications
– Or localhost communications(mysql & phpmyadmin for eg.)
eth0
Linux bridge: docker0: 172.17.42.1/16
vethXXX
172.17.x.x/16
container
eth0
vethXXX
172.17.x.y/16
container
eth0
Melbourne Docker Meetup Oct 2015 - Kubernetes Networking & Storage Deep Dive6
KUBERNETES IN-HOST NETWORKINGKubernetes Pods
● Pods (aka gears in OSE v2)– Pod typically runs only one
container for independent scaling,with exceptions such as “sidecars”.Ie. MySQL & phpMyAdmin
– Same IP / network namespace
– Same IPC
– Ability to share disk
– Scheduled on same machine
– Atomic unit
– Can listen (to other containers)on127.0.0.1 or 0.0.0.0
Melbourne Docker Meetup Oct 2015 - Kubernetes Networking & Storage Deep Dive7
INTER-HOST NETWORKINGSolutions Available
PROGRAMMABLEINFRA
● GCE – each host gets asubnet of 256 Ips
● AWS 40-100 Ips perhost
● Custom enterprisenetworking
OVERLAY NETWORKS
● Flannel (ie. AtomicHost) – subnet withsimple overlay.
● Others solutionsavailable like Weave,etc.
Open vSwitch
● OpenStack
● Very largedeployments
● Programmable
● OpenShift-SDN usesthis
Melbourne Docker Meetup Oct 2015 - Kubernetes Networking & Storage Deep Dive8
● Kubernetes assigns 10.1.x.0/24 subnet to pods in each node● Gateway IP 10.1.x.1 is assigned to lbr0● Out of the box with OpenShift, Open vSwitch VXLAN overlay is used to connect nodes/pods● Flat network, allow subdivision via flows
OPENSHIFT-SDN NETWORKING
OVS: br0
vxlan0
eth0
vovsbr
Linux bridge: lbr0: 10.1.0.1/24
vlinuxbr
veth pair
vethXXX
Pod10.1.0.2
eth0
vethXXX
Pod10.1.0.3
eth0VXLAN
VXLAN overlay
OVS: br0
vxlan0
eth0
vovsbr
Linux bridge: lbr0: 10.1.1.1/24
vlinuxbr
veth pair
vethXXX
Pod10.1.1.2
eth0
vethXXX
Pod10.1.1.3
eth010.1.1.0/24
VXLAN
Melbourne Docker Meetup Oct 2015 - Kubernetes Networking & Storage Deep Dive9
● OpenShift-SDN or similar overlays allow communications within a cluster, but how doaccess from outside; ie. WWW?
● Router Pod – Runs HAproxy, proxies incoming traffic through SDN – HTTP/S Only
OPENSHIFT-SDN ROUTING
Melbourne Docker Meetup Oct 2015 - Kubernetes Networking & Storage Deep Dive
10
KUBERNETES DEEP DIVE:STORAGE
Melbourne Docker Meetup Oct 2015 - Kubernetes Networking & Storage Deep Dive11
DOCKER STORAGE CONSTRAINTS KUBERNETES GOALS
CONTAINER STORAGE
1) Containers are ephemeral(stateless), once they disappear, sodoes the data
2) You can mount the host filesystem,but
1) What about loss of host?
2) How do you scale across1000s of hosts?
3) How is this data shared?
1) Allow administrators to describeavailable storage
2) Allow application developers todiscover and request persistentstorage
3) Persistent storage should beavailable without being closelybound to a particular disk, server,network or storage device
4) Support iSCSI, NFS, EBS, Gluster,Ceph… and many more!
Problem Statement
Melbourne Docker Meetup Oct 2015 - Kubernetes Networking & Storage Deep Dive12
PERSISTENT VOLUME CLAIMS
● Admins add PVs to cluster [1]● Like Pods, PVClaims live in a
namespace● Pods [3] and Pvclaims [2] are
requests for resources by users● Users request resources with
limits, like cpu usage by acontainer or storage capacity of avolume [2]
● OpenShift binds requests toavailable resources
● Multiple access modes(RWO,ROM,RWM)
Requests for storage
kind: PersistentVolumeClaimapiVersion: v1metadata: name: myclaimspec: accessModes: - ReadWriteOnce resources: requests: storage: 8Gi
kind: PodapiVersion: v1metadata: name: mypodspec: containers: - name: myfrontend image: dockerfile/nginx volumeMounts: - mountPath: "/var/www/html" name: mypd volumes: - name: mypd persistentVolumeClaim: claimName: myclaim
POST:kind: PersistentVolumeapiVersion: v1metadata: name: pv0001spec: capacity: storage: 10 nfs: Server: srv.com path: /data/path
1 2
3
Melbourne Docker Meetup Oct 2015 - Kubernetes Networking & Storage Deep Dive13
DB
Host 1 Host 2
Storage Pool
Kubernetes Persistent Volumes
CONTAINER STORAGE
mysql-claim
pv0001
Network Storage NFSiSCSI EBS RBD
2. Dev “Claims” PV
1. Admin creates PV
Melbourne Docker Meetup Oct 2015 - Kubernetes Networking & Storage Deep Dive14
DB
Host 1 Host 2
DB
What happens to a container & its storage when a node is lost?
CONTAINER STORAGE
Storage Pool
mysql-claim
pv0001
Network Storage NFSiSCSI EBS RBD
Melbourne Docker Meetup Oct 2015 - Kubernetes Networking & Storage Deep Dive15
What happens to a container & its storage when a node is lost?
CONTAINER STORAGE
DB
Host 1 Host 2
DB
Storage Pool
mysql-claim
pv0001
Network Storage NFSiSCSI EBS RBD
Melbourne Docker Meetup Oct 2015 - Kubernetes Networking & Storage Deep Dive
16
Storage Demo(https://github.com/openshift/origin/tree/master/examples/wordpress)
& HexBoard Demo
(https://www.youtube.com/watch?v=wWNVpFibayA&feature=youtu.be&t=24m25s)
Melbourne Docker Meetup Oct 2015 - Kubernetes Networking & Storage Deep Dive17
Free Kubernetes eBookOpenShift Enterprise Test-Drive
(Free 8 hour Environment on AWS)
Questions?
http://red.ht/1NbW2wi http://red.ht/1MQVgqb
Plus some great ways to get started with Kubernetes...
Melbourne Docker Meetup Oct 2015 - Kubernetes Networking & Storage Deep Dive19
WHAT ARE LINUX CONTAINERS?Software packaging concept that typically includes an application and all ofits runtime dependencies
● Easy to deploy and portable across host systems
● Isolates applications on a host operating system
● In RHEL, this is done through:● Control Groups (cgroups)● Kernel namespaces● SELinux, sVirt, iptables● Docker
HOST OS
SERVER
CONTAINER
LIBS
APP
Melbourne Docker Meetup Oct 2015 - Kubernetes Networking & Storage Deep Dive20
TRADITIONAL OS VS. CONTAINERS
Traditional OS Containers
HARDWARE
HOST OS
HARDWARE
HOST OS
CONTAINER
LIBS
APP A
LIBS A LIBS B LIBS LIBS
APP A APP BCONTAINER
LIBS
APP B
Packaged dependencies = faster boot times + greater portability