© 2019 diamanti | confidential | do not …...© 2019 diamanti | confidential | do not distribute 4...

© 2019 DIAMANTI | CONFIDENTIAL | DO NOT DISTRIBUTE 2

Challenges in Deploying Kubernetes on Hyperconverged Infrastructure (HCI)


Presenters

Naren NarendraDirector

Product Marketing

Naveen SethFounding Engineer

Hiral PatelFounding Engineer


Agenda

1. Background with AWS EC2 evolution - example

2. Requirements for Kubernetes on bare metal HCI

3. Demo

4. Q&A


Picture source: Evolution of the EC2 Host, https://bit.ly/2lV2WTk (@awsgeek, Jerry Hargrove, AWS)1 https://perspectives.mvdirona.com/2009/10/vl2-a-scalable-and-flexible-data-center-network/ (James Hamilton, DE, AWS, 2009)

Networking is, in effect, “in the way” and blocking the efficient optimization of the most valuable resources in the data center. 1

The Beginning


Picture source: Evolution of the EC2 Host, https://bit.ly/2lV2WTk (@awsgeek, Jerry Hargrove, AWS)Reference blog: https://perspectives.mvdirona.com/2019/02/aws-nitro-system/ (James Hamilton, DE, AWS, Feb 2019)

1GHDD

1G/10GSSD

10G/100GNVMe


Source: Powering Next-Gen EC2 Instances: Deep Dive into the Nitro System - AWS re:Invent 2018, https://bit.ly/2ltCPmaReference blog: https://perspectives.mvdirona.com/2019/02/aws-nitro-system/ (James Hamilton, DE, AWS, Feb 2019)

AWS Nitro Cards


● Host is heavily taxed on utilization and performance, applications starve

● Noisy neighbors● Some NW acceleration● Very High TCO

Hyperconverged Infrastructure (HCI) Evolution

HYPERVISOR

STORAGE

NETWORKING

VM VM VM VM ….

Hyperconverged 1.0

HYPERVISOR

STORAGE

NETWORKING

VM VM VM VM ….

Hyperconverged 1.5

HYPERVISOR

NETWORKING

VM VM VM VM ….

Hyperconverged 2.0

STORAGE

NETWORKING

Hyperconverged 3.0

STORAGE

C1 C2 C3

C4 C5 C6

C7 Cn

….

SO

FTW

AR

E

H

AR

DW

AR

E


● Noisy neighbors● No SLA guarantee● Highest TCO


● Noisy neighbors● Storage offload, NW

acceleration● High TCO

● Cloud native● Applications get >95% host

utilization, Hypervisor removed● Storage and NW acceleration● No noisy neighbors● Guaranteed SLA (Latency, BW)● Inherently secure● Lowest TCO

(Outpost + Nitro)


NETWORKING

Hyperconverged 3.0

STORAGE

C1 C2 C3

C4 C5 C6

C7 Cn

….

NETWORKING

Hyperconverged 3.0+

STORAGE

C1 C2

C4 C5

C7

….

C3

C3

KVM1

KVM2

Bare-Metal Containers

+ Container-Native

Virtualization

“Container-Native Virtualization”


HCI Requirements


HCI Requirements for Containerized Applications

Multiple applications on a node ⇒ Guaranteed SLAs (Jitter Free)

NETWORKING

STORAGE

● SR-IOV○ Hardware queues

● Performance Tiers○ Min guarantees○ Max limits

● SR-IOV○ Hardware queues

● Performance Tiers○ Min guarantees○ Max limits

Networking Storage


HCI Requirements for Kubernetes

NETWORKING

STORAGE

NETWORKING

STORAGE

NETWORKING

STORAGE

Multiple application instances across a Kubernetes cluster

● Static and Dynamic endpoint provisioning

● Multiple endpoints● Endpoint visibility● Separation of control

and data planes● Availability zones

● Static and Dynamic provisioning

● Synchronous mirroring● Snapshots / Restore● Backup / Restore● Availability zones

Networking (CNI) Storage (CSI)


HCI Requirements for Kubernetes Clusters

NETWORKING

STORAGE

NETWORKING

STORAGE

NETWORKING

STORAGE

NETWORKING

STORAGE

NETWORKING

STORAGE

NETWORKING

STORAGE

High Availability Zones ⇒ Campus Clusters

● Zone aware subnet management ● Zone aware mirror placement



HCI Requirements for “yet-to-be-containerized” Legacy Applications

NETWORKING

STORAGE

NETWORKING

STORAGE

NETWORKING

STORAGE

KVM

KVM

KVM

KVM

KVM

KVM

“Container-Native Virtualization”

● Feature parity● Performance parity● Kubernetes managed

● Feature parity● Performance parity● Kubernetes managed



HCI Requirements for Kubernetes Beyond Networking and Storage

Cloud Native Networking

Cloud Native Storage

CSI

CNI

ORCHESTRATION

RUNTIME

GKE AKS SECURITY (RBAC, LDAP, AD)

OPERATING SYSTEM

ORCHESTRATION

CONTAINER RUNTIME

MANAGEMENTMONITORING

LOGGING

SERVICE DISCOVERY

ROLE BASED ACCESS CONTROL

NETWORKING

STORAGE

CONFIGURATION MANAGEMENT

IMAGE REGISTRY

NETWORKING STORAGE


Demo


Demo Flow

1. Demo setup

2. Deploy multi-instance WordPress application using Kubernetes

3. Deploy KVM for legacy application (yet-to-be-containerized) using Kubernetes

4. I/O isolation with QoS for performance


Demo Flow

1. Demo setup





Diamanti Demo Cluster

ETCDKUBERNETES APISERVER

ETCD ETCD

CONTROLLER MANAGERSCHEDULER

SCHEDULER EXTENSIONSAPISERVER/CONTROLLERS

Master Components

NODE AGENT + PLUGINS

KUBELET KUBELET

NODE AGENT + PLUGINS NODE AGENT + PLUGINS

KUBELETINGRESS INGRESS Node Agents

Node2Node1 Node3

INGRESS

KVM CONTROLLER


Demo Flow

1. Demo setup





WordPress Application Deployment

ETCD

MYSQL

WORDPRESS

KUBERNETES APISERVERETCD ETCD



Master Components


KUBELET KUBELET


KUBELET

Third Party Backup Storage

WORDPRESS

INGRESS INGRESS Node Agents

S1S2

Diamanti IO Layer

INGRESS

Node2Node1 Node3

WORDPRESS

Via Storage Classes in Kubernetes


Demo Flow

1. Demo setup





Container-Native Virtualization

● A VM runs inside a container

● Can co-exist with container workloads/pods

● Based on KVM

● Uses Kubernetes as Orchestrator

● Consistent I/O isolation and quality of service for containers and VMs using PCI pass-through


Container-Native Virtualization(CRD) and Pod Deployment on Kubernetes

API Server

KVM Controller

Kubectl / Proxy Kubelet

Pod (per VM)

Container

VM

Guest OS

App

RC

NGINX

Network controller Storage controller

Pod

Scheduler

Diamanti API Server

KVM

Pod


KVM Deployment


ETCD ETCD



Master Components


KUBELET KUBELET


KUBELET

Third Party Backup Storage

INGRESS INGRESS Node Agents

S1S2

Diamanti IO Layer

INGRESS

KVM CONTROLLER

KVM

Node2Node1 Node3


Demo Flow

1. Demo setup





“Isolation” with “QoS” for Network and Storage


ETCD ETCD




KUBELET KUBELET


KUBELETINGRESS INGRESS INGRESS

KVM CONTROLLER

Node2Node1 Node3


KUBELETINGRESS

NETWORKING

STORAGE

NETWORKING

STORAGE

NETWORKING

STORAGE

NETWORKING

STORAGE

C3KVM1 C3KVM

2 C3KVM3

C3KVM4 C3KVM

9

….

C1 C2 C3

C4 C9

….

C10 C11 C12

C13 C18

….

C19 C20 C21

C22 C27

….

Node4


Q&A


To Learn More...

● www.diamanti.com

● [email protected]

● @diamanticom

● https://www.linkedin.com/company/diamanti


Thank You!

© 2019 diamanti | confidential | do not …...© 2019 diamanti | confidential | do not distribute 4...

Documents