openebs hangout#1 - introduction to openebs , an opensource containerized storage for containers

Containerized Storage for Containers

Online Hangout - 18 May 2017@OpenEBS

Agenda

OpenEBS Overview

OpenEBS Project Updates

Demo - K8s 1.6 - Sandbox Setup

Demo - OpenEBS Playbooks

Q/A

Nevermind, how do I play with it?

wget https://raw.githubusercontent.com/openebs/openebs/master/k8s/lib/vagrant/test/k8s/1.6/Vagrantfile

vagrant up

K8s master K8s minions

https://github.com/openebs/openebs/tree/master/k8s/lib/vagrant/test/k8s/1.6

https://raw.githubusercontent.com/openebs/openebs/master/k8s/lib/vagrant/test/k8s/1.6/Vagrantfile

What is OpenEBS ?

➔ OpenEBS is containerized storage for containers

➔ OpenEBS solves the persistent storage issue for containers

Legacy

storage

X

Application

Pod

★ Application pod management

automatically takes care of storage

volume management

Containerized storage - Easy upgrades

Monolithic Storage Kernel

iSCSI Software

A

ReplicationSnapshot

EncryptionB

QoS&

MgmtC

volume1

volume2

Containerized Storage Kernel

(OpenEBS)

volume1

Storage ContainerVSM 1

SoftwareA, B, C

volume2

Storage ContainerVSM 2

SoftwareA’, B’, C

Storage software is same for all

volumes

What is the vision ?

OpenEBS secures and serves

the data for dynamic container

based environments, making it

as easy and safe to run

persistent workloads on these

environments as it is to run

stateless workloads.

Manifests express intent

K8S used with special care for apps requiring

persistence because they require brittle tight coupling.

Container 1 Container 2 Container 2

Hard wired connections via plug-in

stateless stateful

NAS SAN S3 NAS SAN S3

Without OpenEBS DevOps workflow is broken


stateless stateful

No changes to DevOps workflow even for containers requiring

persistence. Users manifest their intent and the storage and

storage controllers adjust automatically as needed.

OpenEBSPolicy engine

Remote Storage

Local Storage

OpenEBS Storage Hosts

OpenEBS VSMs

OpenEBS Storage ClusterEBS

APIs

Containers and underlying storage,

local on host or dedicated storage

pods OR remote S3 or EBS storage

all grouped into a storage cloud that

just works. Billing by AWS.


With OpenEBS the workflow works

DevOps operations with OpenEBS

OpenEBS got the storage ops off my back!

Because of OpenEBS we are rolling into

production.

How is OpenEBS built ?Storage containerization

Storage orchestration

Deep storage operations

Core Storage features

is built using

is integrated with

is built into (OpenEBS Maya)

is built into (OpenEBS)

Smart Capacity Management Volume Latency Monitoring and Auto-healingAdapt to varying workloads - Application Aware Storage (Zones)AWS EBS Like APIsStorage Analytics!

Enterprise HA Intelligent QoSDistributed Scaleout Smart Tiering for higher performance

Introduction to the OpenEBS team

2008 20162012 2017

Why this team?

CloudByte ElastiStor

Enterprise Storage Servers! ( SAN/NAS ) Containerized storage &

Enterprise QoS

Team :

● Deep storage Experience

● Deep DevOps Experience

● Done 100s of production

storage upgrades in the

field

Disk Management Layer

Core Storage Features

HA/Multipath Layer

Provisioning/SDS

S

C

A

L

E

P

E

R

F

O

R

M

A

N

C

E

Multi Tenant and RBAC

Monitoring/Alerting

QoS

Typical deployment

workflow

Typical OpenEBS Workflow - 1


1. Start with an existing or new K8s Cluster



2. Download and Apply OpenEBS Operator






3. OpenEBS Maya Services are Launched


K8s master K8s minions Application Pod (with OpenEBS FlexVolume)




4. Define in application yaml to use OpenEBS Storage







5. (VSM) Storage Containers are Launched and Storage is mounted.

Data







5. (VSM) Storage Containers are Launched and Storage is mounted.

Data

iSCSI

6. Application is launched consuming OpenEBS Storage

OpenEBS Summary

OpenEBS - Word of mouth

Containerized storage for containers

Calico / Flannel owns networking for K8S. OpenEBS owns

storage.

The only open source solution built for this. CEPH is not built

to be containerized.

I run (use) OpenEBS just like I run (use) K8S (EBS).

OpenEBS got the storage ops off my back! Because of OpenEBS

we are rolling into production.

K8s 1.6 SandBox Demo

Quickly Setup K8s 1.6

Vagrant and VirtualBox

Ubuntu 16.04

Configured with Weave and OpenEBS

DevOps Workflow requires Sandbox

Checkout

CI

Deploy

Modify Code

Run in Sandbox

Validate

Commit

K8s SandBox for DevOps


1. Multi-node k8s cluster for testing - HA/Scalability aspects

2. Configured with the choice of Networking and Storage

3. Ability to recreate setup with required versions (locally or cloud)

Setting up Kubernetes

minikube kubeadm

Laptop CloudOn-premise

GCE

ACSIBM BCS

AWS EC2

CenturyLink

conjure kargo

20+ options minishift

openshiftkubeadm

Setting up Kubernetes with kubeadm

1. Setup Master 2. Configure Networking 3. Join Nodes

kubeadm init kubectl apply -- kubeadm join

Install out-of-tree

plugins

Gets the Latest from

repositories, which

may not be

compatible

Huge Network

Dependency

May require some tweaks

based on the plugins

Maintaining the version

compatibility with other nodes

Plugins may require additional

s/w packages installed

Kubernetes - Versioning / Compatibility

APIs and Features in K8s : - v1alpha, v1beta, v1- v2alpha, v2beta, v2

kubernetes changes :- more secure - API server accessible only on 6443- On kubelet, volume attach/detach controller is default option for storage- FlexVolume (mentioned as alpha) is not backward compatible..- Dependency on nslookup of localhost

kubeadm changes : - kubeadm init options : --api-advertise-addresses, --kubernetes-version

- kubeadm join token is stored in secrets (not in cluster info)

Multi-node Kubernetes Sandbox

wget https://raw.githubusercontent.com/openebs/openebs/master/k8s/lib/vagrant/test/k8s/1.6/Vagrantfile

vagrant up


https://github.com/openebs/openebs/tree/master/k8s/lib/vagrant/test/k8s/1.6

https://raw.githubusercontent.com/openebs/openebs/master/k8s/lib/vagrant/test/k8s/1.6/Vagrantfile

Kubernetes - Boxes

Creating new Kubernetes Boxes

Step by step “storage provisioning”

Placeholder for step by step

BOX 1: First thanks to X we get the environment up and running. If you already have the

environment, you may not need this.

BOX 2: Let’s walk around the environment. Similar to slide 33 (Demo set-up).

BOX 3: Let’s check in some YAML that shows that k8s pod definition requires more storage

BOX 4: Checked in and see it now requesting.

BOX 5: Now serving storage

Step by step “behavior under load”

Placeholder for step by step

BOX 1: First thanks to X we get the environment up and running. If you already have the

environment, you may not need this.

BOX 2: Let’s walk around the environment. Similar to slide 33 (Demo set-up).

BOX 3: Let’s create some load (again show how)

BOX 4: Let’s break it! (show how we kill containers and yet workload continues).

Creating new Kubernetes Boxes

Pod Yaml - Stateful App

Demo - Setup

HTTPS

K8s master

K8s minions

FlexVolume

(OpenEBS Driver)

OpenEBS Maya

MasterOpenEBS Storage Hosts

OpenEBS VSMs / Storage Pod iSCSI/Data

OpenEBSProject Updates

OpenEBS 0.2 Released

OpenEBS 0.3 Updates

Other Updates...

OpenEBS 0.2 ( Released! )https://github.com/openebs/openebs/releases/tag/v0.2

(Beta) Integrated with Kubernetes

(Beta)High Availability via Sync Replication

(Beta) Block Storage (available via iSCSI)

(Alpha) Data Protection ( Local Snapshots )

(Alpha) Data Backup with Amazon S3

OpenEBS 0.3 ( In-Progress )https://github.com/openebs/openebs/wiki/Project-Tracker

OpenEBS Hyper-converged with Kubernetes 1.6

Support for adding additional disks as block-data-stores

Implement CI for running automated storage high-availability tests.

Provide usage examples for running Percona/MySQL and ElasticSearch Apps on OpenEBS Storage

Support for running on Fedora, CentOS

Other Updates...

Evan Powell presented a talk on state of storage and OpenEBS at (http://storageconference.us/)

Co-hosting Golang Meetup in Bangalore

OpenEBS on Quora (https://www.quora.com/topic/OpenEBS )

Exploring ways to improve productivity for operations and developers:

OpenStack Kolla on K8s and OpenEBS

Running Data Science notebooks on K8s and OpenEBS

We are hiring!!!

Building On-premise EBS

Demo

Using Ansible and OpenEBS

Convert bare-metal hosts or VMs

into Storage Nodes

Demo - Consuming EBS-Like storage

for Docker Application

Building On-premise EBS-Like Solution

EBS - Like Storage

Build EBS-Like Storage Solution on premise using BareMetal

VMs or Hosts connected with storage disks.

OpenEBS

Maya

Master

OpenEBS Storage Hosts

Installation and Provisioning automated via Ansible

OpenEBS Playbooks Overview

OpenEBS Playbooks - Demo

Used for OpenEBS end-to-end

testing.

Automatically Generate the inventory

(hosts files)

Setup OpenEBS Storage using ESX

VMs

Provision Storage for a Container

running IO Meter.

Q&A

Join the community

#slackSlack.openebs.io

blog.openebs.io

@openebs

We have reached the End of the hangout !

Thank you for joining us !

See you at the next hangout !

What do you think?

https://github.com/openebs

Join the community

#slackSlack.openebs.io

blog.openebs.io

@openebs

openebs hangout#1 - introduction to openebs , an opensource containerized storage for containers

Technology