container and kubernetes 101 - eventkaddy cms · container and kubernetes 101 for vsphere admins...
TRANSCRIPT
#vmworld
Container and Kubernetes 101 for vSphere Admins
Robbie Jerrom, Tech. Lead – Applications Transformation NEMEANick Kenny, Applications Transformation SME
CNA1816BE
#CNA1816BE
VMworld 2018 Content: Not for publication or distribution
Disclaimer
2©2018 VMware, Inc.
This presentation may contain product features orfunctionality that are currently under development.
This overview of new technology represents no commitment from VMware to deliver these features in any generally available product.
Features are subject to change, and must not be included in contracts, purchase orders, or sales agreements of any kind.
Technical feasibility and market demand will affect final delivery.
Pricing and packaging for any new features/functionality/technology discussed or presented, have not been determined.
VMworld 2018 Content: Not for publication or distribution
Agenda
3©2018 VMware, Inc.
Introductions
Containers and Docker 101
Kubernetes 101
PKS 101
Automating Everything
VMworld 2018 Content: Not for publication or distribution
4©2018 VMware, Inc.
Who are we...
Robbie Jerrom
Nick KennyApplications Transformation SME
Tech. Lead –Applications Transformation NEMEA
@robbiej
VMworld 2018 Content: Not for publication or distribution
5©2018 VMware, Inc.
Containers and Docker 101Click to edit optional subtitle
VMworld 2018 Content: Not for publication or distribution
©2018 VMware, Inc. 6
Hardware
OS Kernel
OS File system
Use
rsp
ace
Container
Ap
p p
roce
ss
Ap
p p
roce
ss
Ap
p p
roce
ss
Ap
p p
roce
ss
Ap
p p
roce
ss
Container
Ap
p p
roce
ss
Ap
p p
roce
ss
Linux Containers 101
6
OS-level Isolation• Isolation at individual kernel
subsystem level (e.g. filesystem, process table, etc)
• User-level process (LXC, libcontainer) orchestrates these subsystems to create a container
Existed for Many Years• Solaris Zones, FreeBSD Jails, OpenVZ
Why?• Process isolation
• Reproducible environment
• Dependency bundlingVMworld 2018 Content: Not for publication or distribution
7©2018 VMware, Inc.
Docker made containers easy for Developers
VMworld 2018 Content: Not for publication or distribution
8©2018 VMware, Inc.
Application
Operating System & Dependencies
PhysicalInfrastructure
Application
Operating System & Dependencies
PhysicalInfrastructure
Containers and VMs
App Dependencies
Compute | Net | Sec | Storage
OS Abstraction
Container Host OS
Container
Compute | Net | Sec | Storage
Hard Problem
Easier
Ubiquitous
Ubiquitous
Configuration Management
VMworld 2018 Content: Not for publication or distribution
9©2018 VMware, Inc.
Container Registry
Repository of Container Images
Anatomy of Building and Running a Container (Redis DB)*
FROM: Ubuntu 14.04
RUN apt-get redis
EXPOSE 6379
CMD [“/user/sbin/redis..]
Minimal Linux “Container Host”
DockerEngine
Running Container
RedisDB
Tools, Libs, SW
#docker build#docker push
#docker run redis
RedisDB
Tools, Libs, SW
Dockerfile
RedisDB
Tools, Libs, SW
VMVMworld 2018 Content: Not for publication or distribution
10©2018 VMware, Inc.
Lets look inside a containerFrom just a single file…
Demo
VMworld 2018 Content: Not for publication or distribution
11©2018 VMware, Inc.
Business App
Docker and Kubernetes
Docker Engine – Executes Container Images
• Manual, no fault tolerance, hard to scale, etc
Scheduling, provisioning, and resource management of multiple containers
• Docker, Mesos Kubernetes Support• AWS, Azure, Google Kubernetes Services
$docker run container_web
$docker run container_web
$docker run container_LB
$docker run container_DB
$kubctl create –f App.yaml
The “App”
Wanted: Container Orchestrator!
Kubernetes Cluster
Docker ContainersOne at a Time
Kubernetes (aka K8s) Orchestrating Multiple Containers
VMworld 2018 Content: Not for publication or distribution
13©2018 VMware, Inc.
Kubernetes ‘K8S’
VMworld 2018 Content: Not for publication or distribution
©2018 VMware, Inc. 14
Kubernetes (basic) Architecture
KubernetesMaster
Container
Pod
Worker Node
Docker Engine
Container
Pod
Container
Pod
Kub
elet
kube-proxy
Container
Pod
Worker Node
Docker Engine
Container
Pod
Container
Pod
Kub
elet
kube-proxy
User Network
API Service
kubectl
VMworld 2018 Content: Not for publication or distribution
16©2018 VMware, Inc.
SidecarContainer
Container
Pod
Pod
one or more application containers that are tightly coupled, sharing network and storage.
Example: a web front-end Pod that consists of an NGINX container and a telegraph container. The NGINX container is providing you a frontend webpage and the telegraph container (Sidecar) is sending NGINX metrics to Wavefrontfor monitoring.
Kubernetes (basic) Architecture
VMworld 2018 Content: Not for publication or distribution
17©2018 VMware, Inc.
ReplicaSet
Extends Pod resource to run and maintain a specific number of copies of a pod.
Container
Container
Pod
Container
Container
PodReplica Set
Kubernetes (basic) Architecture
VMworld 2018 Content: Not for publication or distribution
18©2018 VMware, Inc.
Container
Container
Pod
Deployment
a controller that ensures a set number of replicas of a Pod is running and provides update and upgrade workflows for your Pods.
Example: cloud native Node app that scales horizontally and upgrades 2 pods at a time.
Container
Container
PodReplica Set Deployment
Kubernetes (basic) Architecture
VMworld 2018 Content: Not for publication or distribution
19©2018 VMware, Inc.
Demo
Lets go deploy something… From just two files…
VMworld 2018 Content: Not for publication or distribution
20©2018 VMware, Inc.
*Batteries not included
VMworld 2018 Content: Not for publication or distribution
25©2018 VMware, Inc.
VMware PKS
VMworld 2018 Content: Not for publication or distribution
©2018 VMware, Inc. 26
VMware PKS
A turnkey solution to provision, operate and manage enterprise grade Kubernetes clusters.
+
Fully supportedKubernetes distribution
Latest Stable version of Kubernetes
Deep integration with NSX-T for networking and security
Runs on vSphere and GCP
VMworld 2018 Content: Not for publication or distribution
27©2018 VMware, Inc.
Challenges of Running Containers in Production
Source: CNCF user Survey, December 2017
0%
5%
10%
15%
20%
25%
30%
35%
40%
45%
50%
Difficultychoosing an
orchestrationsolution
Reliability ScalingDeployments
Logging Complexity Networking Monitoring Storage SecurityVMworld 2018 Content: Not for publication or distribution
28©2018 VMware, Inc.
A Comprehensive Kubernetes Solution
Image Registry
Framework Lifecycle Management
Security and Networking
Storage Persistence
Virtual Infrastructure
Physical Infrastructure
Mo
nito
ring
, Lo
gg
ing
, Ana
lyti
cs
Cluster Health Monitoring, Healing and Lifecycle Management
Scheduling, Orchestration, Service Creation
vCloud Storage Provider
vSphere | NSX | VSAN
Physical Infrastructure
NSX
BOSH (K8s LCM)
BOSH (cluster LCM)
Upstream Kubernetes
Harbor
vRe
aliz
eS
uite
& W
ave
Fro
nt
PKS
VMworld 2018 Content: Not for publication or distribution
29©2018 VMware, Inc.
PKS Technical Overview - BOSH
Physical Infrastructure
BOSH
NSX-T
ServiceBroker(s)
vSANvSphere
etcd worker
Container Registry
master etcd workermaster
PKS Control Plane
Kubernetes Cluster Kubernetes Cluster
Wavefrontby VMware
vRealizeAutomation
vRealizeLog Insight
vRealizeOperations
vRealizeNetwork Insight
VMworld 2018 Content: Not for publication or distribution
30©2018 VMware, Inc.
Can
al
Kubernetes Networking Options
L2L3 (North/South)
L4 – Security PolicyLoad Balancing
VM & K8s Connectivity
End-to-End Configuration & troubleshooting
Ops tools & central stats
FlannelL2 only (East/West Pod Traffic)
CalicoL3/L4 (IP Tables)
NGINX/HA ProxyLoad balancing
No End-to-End configuration & troubleshooting
New ops tools + Different locations for stats
NSX-T
Only for Kubernetes Networking
VMworld 2018 Content: Not for publication or distribution
31©2018 VMware, Inc.
Declarative Container Networking & Security @ ScaleWith supporting operational tools
NSX-T Operational Tools
• Traceflow
• Port Mirroring
• Port Connection Tool
• Spoofguard
• Syslog
• Port Counters
• IPFIX
31
NSX-T Traceflow
VMworld 2018 Content: Not for publication or distribution
High Availability. No out-of-the-box fault-tolerance for the cluster components themselves (masters and etcd nodes).
Scaling. Kubernetes clusters handle scaling the pod/service within the Nodes, but doesn’t provide a mechanism to scale Masters & etcd VMs.
Health checks and Healing. The Kubernetes cluster does routine health checks for the health of Nodes only.
Lifecycle Management. Rolling upgrades on a large fleet of clusters is hard. Who manages the system it runs on?
Kubernetes is Difficult for Day 2 Ops …
VMworld 2018 Content: Not for publication or distribution
©2018 VMware, Inc. 34
PKS Technical Overview - BOSH
Physical Infrastructure
BOSH
NSX-T
ServiceBroker(s)
vSANvSphere
etcd worker
Container Registry
master etcd workermaster
PKS Control Plane
Kubernetes Cluster Kubernetes Cluster
Wavefrontby VMware
vRealizeAutomation
vRealizeLog Insight
vRealizeOperations
vRealizeNetwork Insight
VMworld 2018 Content: Not for publication or distribution
©2018 VMware, Inc. 35
VMware GCP AWS
AvailabilityZone
master
etcd
worker
etcd etcd
master
worker worker
master
etcd
worker
etcd etcd
master
worker worker
AvailabilityZone
AvailabilityZone
BOSH
Health Monitor
Health Monitor
worker workerPatch
K8sNewVer
Release
Repair
Lifecycle Management Using BOSH
Deploy
ScaleUpgradePatchRepair
Day 1
Day 2
workerworker
PK
S C
ont
rol P
lane
CVE
VMworld 2018 Content: Not for publication or distribution
36©2018 VMware, Inc.
Monitoring & Logging @ ScaleWho needs what?
Infra K8s Containers Apps ApplicationDev/Ops Owner
Platform Reliability Engineer
vRLI
vRops Wavefront
VMworld 2018 Content: Not for publication or distribution
37©2018 VMware, Inc.
PRE
Infrastructure
Compute Network Monitoring
Security Storage
Connecting K8s to Infrastructure
Kubernetes Cluster
vSphere NSX Wavefront
NSX Datastores
SRE / PRE Rolemaps Kubernetes capabilities to a
given infrastructure
Load Balancer
Storage Requirements
Availability Zone
Security Policy
Application Metrics
ELK Spark Nth App
K8s API
App Devinteracts with native
Kubernetes constructs‘kubectl’ & .yml
AppDev
VMworld 2018 Content: Not for publication or distribution
38©2018 VMware, Inc.
One Last Demo
VMworld 2018 Content: Not for publication or distribution
39©2018 VMware, Inc.
Kubernetes looks after the containers… that deliver the apps..
PKS looks after KubernetesVMworld 2018 Content: Not for publication or distribution
DON’T FORGET TO FILL OUT YOUR SURVEY.
#vmworld #CNA1816BE
VMworld 2018 Content: Not for publication or distribution
THANK YOU!
#vmworld #CNA1816BE
VMworld 2018 Content: Not for publication or distribution