Persistent Storage for Linux ContainersGrant Della VecchiaSenior Storage Solutions Architect, Red Hat

Linux Containers in brief

A Software packaging concept that typically includes an application and all of its runtime dependencies

• Higher Quality Software

Releases

• Shorter Test Cycles

• Easier Application

Management

HOST OS

SERVER

CONTAINER

LIBS

APP

CONTAINER

LIBS

APP

Benefits

Greater Portability, Automation, and Integration

Code Registry

Push image

Code & Build Test Deploy

Pull image

The Red Hat Stack – From PaaS to Storage

DevOps Tools and User Experience

Language Runtimes and Middleware

Databases and Other Services

Container Orchestration and Management

Container API

Storage

Container Host

Typical workloads for Containers

Base: 194 IT operations and development decision-makers at enterprises in APAC, EMEA, and North America Source: A commissioned study conducted by Forrester Consulting on behalf of Red Hat, January 2015

“Forwhichworkloadsorapplicationusecaseshaveyouused/doyouanticipatetousecontainers?”

Scalable,CostEffective,DistributedStorageforContainers

Why Containers require persistent Storage?

• Web & presentation layers should be stateless ... but ...

• That’s probably < 5% of all application instances in a DC

• Most other infrastructure applications / services require persistence for storing application and configuration data

• Building a different infrastructure for stateful vs. stateless apps ?

• Modern IT requires Standardization (e.g. Cloud Computing)

• Software-defined DC is a hybrid cloud enabler

The requirements for Container Storage?

• Scalable – Scale out capacity & performance

• Resilient – Likely there will be important data

• Flexible – Allow different application access profiles

• Software-defined – To allow flexible deployment models

• Open – For customer choice and lowest TCO

RED HATSTORAGE➔

Persistent Storage requires Redundancy

• Red Hat Storage includes important data redundancy features

• 2-way and 3-way replication

• Erasure Coding

• Geo-Replication

• Snapshots

Current Options for Persistent Storage

Shared Filesystems:• NFS• GlusterFS

Block Storage:• Ceph RBD• iSCSI or FC• GCE Persistent Disk• AWS Elastic Block Store

Storage Options ComparisonNFS-Filer GlusterFS Ceph

RBDiSCSI / FC GCE PD AWS EBS

Scalability - + ++ - ++ ++

Availability + ++ ++ + + +

Cost + + + ++ - -

Deployment Flexibility

- ++ ++ - - -

Data Privacy

++ ++ ++ ++ -- --

Sharing Data

++ ++ +/- - - -

Use Cases Generic Data Sharing

Generic Data Sharing

Very Large Deployments, Object Store

High Performance

Block

Cheap Block Storage

Cheap Block Storage

Configuration Examples

Node 1NGINX

Container

Node 2NGINX

Container

Persistent Storage for OSE/RHEL Atomic

Node 1NGINX

Container

Node 3MySQL

Container

Filesh

are

Filesh

are

BlockDevice/

ObjectS

tore

AtomicHost Atomic HostOPENSHIFTOPENSHIFT

Gluster

Ceph

Gluster

Host OS

Container Container

Host OS

Container Container

/share(Gluster/NFS)

Container Volume mapping options

Gluster Volume

Brick 1 Brick 2 Brick 3 Brick n

/sha

re(G

lust

er/N

FS)

/share/sub1➔ /data

/share/sub2➔ /data

Host OS

Container Container

Host OS

Container Container

Mapping Ceph RBDs to containers

/shared➔/data

Ceph Pool

OSD 1

OSD 2

OSD 3

OSD n

Ceph

RBD

+

FS Ceph

RBD

+FS

/host-share➔/data /local-mnt

/shared➔/data/host-s

hare

Ceph

RBD

+FS

Hyperconverged Containers and Storage

• Lower TCO

• Unified Orchestration

• Ease of Use

• Greater control

ContainerJBoss

NGINXNGINX ContainerMASTER

KubernetesNode 2

ContainerElastic

SparkMongoDB Container

Kubernetes Node 3

Redis

PostgreSQL Container

Kubernetes Node 1

Kubernetes Node 4

App Container App Container App Container App Container

Containerized Gluster Storage

/shared-volume /shared-volume

Gluster Container(privileged)

/bricks/brick2

Gluster Container(privileged)

/bricks/brick1

/shared-volume(Gluster-Fuse or NFS)

Host OS Host OS

Gluster Volume

/shared-volume /shared-volume

Container Container Container Container

Containerized Ceph Storage

/local-mnt1

Container

RHCS

Container

RHCS

Ceph RBDs (and Object Storage)

Host OS Host OS

Ceph Pool

/local-mnt2 /local-mnt3 /local-mnt4

Flexible Deployment Options

x86 x86 x86

Physical

VM VM VM

Virtualized

VM VM VM

Public / Private Cloud

OpenStack,AWS, Azure, GCE VMware, RHEV, KVM

Container

JBoss

NGINX

NGINX Container

MASTER

Kubernetes Node 2

Container

Elastic

Spark

MongoDB Container

Kubernetes Node 3

Redis

PostgreSQL Container

Kubernetes Node 1

Kubernetes Node 4

Container

JBoss

NGINX

NGINX Container

MASTER

Kubernetes Node 2

Container

Elastic

Spark

MongoDB Container

Kubernetes Node 3

Redis

PostgreSQL Container

Kubernetes Node 1

Kubernetes Node 4

Container

JBoss

NGINX

NGINX Container

MASTER

Kubernetes Node 2

Container

Elastic

Spark

MongoDB Container

Kubernetes Node 3

Redis

PostgreSQL Container

Kubernetes Node 1

Kubernetes Node 4

CustomerCaseStudies

Customer Case Study: CapitalOne

Business Challenge: • A leading diversified bank with 65 million customers• Fast growing business and customer base• Need to be disruptive and different• Analytics plays a big role in growth strategy

Solution Description:• Predefined docker images with a wide variety of analytics

tools• Self-service Portal for developers to pick and instantiate• Integrated monitoring and metrics• Automated lifecycle management of containers• High availability through MESOS• Shared and consolidated Storage Platform with Gluster

Solution Benefits:• More agile application development• Larger choice of technologies• Optimal resource usage and

performance

Customer Case Study: CapitalOne

Customer Case Study: Verizon

Dockercontainersrunnin

Business Challenge: • Verizon is the largest provider of cellular services in the United States with

more than 100 million subscribers. Pretty much every subscriber takes ‘selfies’ and snaps of the kittens and these need to be backed up. Whilst many users use Apple’s iCloud, or Yahoo’s Flickr, Verizon also has their own branded Cloud offering – Verizon Cloud.

• Verizon wished to use a file-based format rather than converting all these images to Objects and to have an extremely efficient architecture in terms of the use of their server infrastructure.

Solution Description:• Simplified Deployment via containers• Seamless upgrade and rollback• Dedicated and Containerized Gluster for FSaaS• Dynamic scaling (up) of capacity as needed• Performance and health metrics collection via container monitoring agents

and sending alerts.

Solution Benefits: • Quicker time to market for new services• Increased subscriber ‘stickiness’ and improving customer

satisfaction. • Reduced infrastructure Costs• Infrastructure can grow as demand grows

ETH/IBapp app app

app app app

Converged Computing Architecture Mixed App and Storage Workloads App-onlyservers

Convergedservers

app

app +

Storage-onlyservers

app app app app

app app app app

app app app app

app app

app

Storage stack imposes only 3% - 10% load on compute processing

• Applications and storage stacks can co-exist on same compute substrate.

• This achieves higher server

utilization and lower operational costs across the cloud

Customer Case Study: Verizon

Summary

• Most containerized applications will require Persistent Storage

• Software-defined Storage allows hyper-convergence for applications and

storage

• Red Hat Storage and OpenShift Enterprise provide a complete PaaS

solution with full deployment flexibility from on-premise to hybrid clouds

Thank You