red hat storage day seattle: why software-defined storage matters

WHY SOFTWARE-DEFINED STORAGEMATTERSRoss TurkRed HatOctober 2016

2

THE DATA STORAGE SHORTFALL

Data stores are growing exponentially, while IT budgets are not

HDDs are becoming more dense, but $/GB decline is slowing

Software and hardware advances are needed to close the gap

GROWTH OF DATA

IT STORAGE BUDGETS

20202010

3

APPLIANCES AREN’T ENOUGH

Complexity hidden from end users, along with flexibility

Vendor lock-in leads to pricing premium

Price premium over constituent components is difficult to sustain

THE TRADITIONAL APPROACH TO STORAGE

ADMINS

MANAGEMENT INTERFACE

DATA

NFS/SMB/ISCSI ENDPOINTS

STANDARD SERVERS AND MEDIA

PROPRIETARY SOFTWARE

4

PUBLIC CLOUD STORAGE ISN’T ENOUGH

Complexity still hidden from end users, pay-as-you-go pricing

Fastest-growing segment of IT storage budgets

Mostly built with proprietary software

CONVENIENT BUT LIMITEDADMIN

S

MANAGEMENT INTERFACE

DATA

CLOUD STORAGE APIS

STANDARD SERVERS AND MEDIA

LINUX + PROPRIETARY SOFTWARE

5

THE INDUSTRY IS RETHINKING STORAGE

38% of IT decision makers

report inadequate storage

capabilities as one of their top

three weekly pain points

70% of IT decision makers admit that their organization’s

current storage can’t cope with

emerging workloads

98% of IT decision makers believe a more agile storage solution could benefit their organization

Vanson Bourne Ltd: Storage limitations, frustrations, and coping with future needs, Red Hat Storage research results, June 2016

6

FLEXIBILITY IS CRUCIAL

7

STORAGE IS EVOLVINGOPEN, SOFTWARE-DEFINED STORAGE

Standardized, unified, open platforms

Standard Computersand Disks

Ceph Gluster +++

Control Plane (API, GUI)

Custom GUI

Proprietary Hardware

Proprietary Software

Custom GUI

Proprietary Hardware

Proprietary Software

USER USER USER

ADMIN

ADMIN

ADMIN

Custom GUI

Proprietary Hardware

Proprietary Software

USER ADMIN

Stan

dard

Hard

ware

Open

So

urce

Softw

are

TRADITIONAL STORAGEComplex proprietary silos

WHAT IS SOFTWARE-DEFINED STORAGE?

9

WHAT IS SOFTWARE-DEFINED STORAGE?

SERVER-BASED CENTRALIZED CONTROL OPEN ECOSYSTEM

10

SERVER-BASED STORAGEThe use of software and standard hardware to provide services traditionally provided by single-purpose storage appliances (similar to server virtualization, which uses software to emulate servers), providing increased agility and efficiency

Appliance

USER

Appliance

USER

Appliance

USER

Distributed Cluster of Services

USER USER USER

11

VIRTUALIZED STORAGE SCALES BETTER

Storage Appliance

+ + + + + +

+ + + + + + + + + +

+ + + + + + + + + +

COMPUTE NODE

COMPUTE NODE

STORAGENODE

+

12

SAN/NAS IS ON THE DECLINE

• Storage capacity shipped inside servers has grown since 2010, and is now the majority

• Capacity shipped in SAN/NAS is declining, reflecting the use of server-based storage technologies by web-scale firms

Changing workloads drive the need for flexible, economical server-based storage.

WW DEPLOYED CAPACITY (TB)100%

80%

60%

40%

20%

0%

2010 2011 2012 2013 2014 2015 2016

EXTERNAL CAPACITY INTERNAL CAPACITY

13

WHAT IS SOFTWARE-DEFINED STORAGE?

CENTRALIZED CONTROLSERVER-BASED OPEN ECOSYSTEM

14

STORAGE ORCHESTRATIONThe ability to provision, grow, shrink, and decommission storage resources on-demand and programmatically, providing increased control and integration of storage into a software-defined data center

Web Console API Command LineA browser interface

designed formanaging distributed

storage

A full API for automation and

integration with outside systems

A robust, scriptable command-line

interface for expert operators

Full life cycle management for distributed, software-defined data services

PROVISION INSTALL CONFIGURE TUNE MONITOR

15

WHAT IS SOFTWARE-DEFINED STORAGE?

OPEN ECOSYSTEMSERVER-BASED CENTRALIZED CONTROL

WHY SOFTWARE-DEFINEDSTORAGE MATTERS

17

WHY THIS MATTERS

PROPRIETARYHARDWARE

Common,off-the-shelf hardware

Lower cost, standardized supply chain

18

INDUSTRY STANDARD HARDWAREStandardization makes storage more convenientCustomers can build clusters using standard hardware from existing vendors that’s perfect for their workload.

• Clusters can be performance-optimized, capacity-optimized, or throughput-optimized

• Need capacity? Add more disks. Too slow? Add more servers.

• Clusters can become larger or smaller with no downtime

19

WHY THIS MATTERS

PROPRIETARYHARDWARE

SCALE-UPARCHITECTURE

Common,off-the-shelf hardware

Scale-outarchitecture

Lower cost, standardized supply chain

Increased operational flexibility

20

PERFORMANCE THAT SCALESPerformance should scale up as capacity does

Software-defined storage intelligently uses hardware to provide performance at very large scale.

• Traditional appliances perform better when they are empty than they do when they are full of disks

• Performance in software-defined storage clusters improves as clusters get larger, not the other way around

• Intel, SanDisk, Fujitsu, and Mellanox regularly contribute performance optimizations

PERF

ORM

ANCE

21

WHY THIS MATTERS

PROPRIETARYHARDWARE

SCALE-UPARCHITECTURE

HARDWARE-BASEDINTELLIGENCE

Common,off-the-shelf hardware

Scale-outarchitecture

Software-basedintelligence

Lower cost, standardized supply chain

Increased operational flexibility

More programmability, agility,and control

22

THE ROBUSTNESS OF SOFTWARESoftware can do things hardware can’t

Storage services based on software are more flexible than hardware-based implementations

• Can be deployed on bare metal, inside containers, inside VMs, or in the public cloud

• Can deploy on a single server, or thousands, and can be upgraded and reconfigured on the fly

• Grows and shrinks programmatically to meet changing demands

23

WHY THIS MATTERS

PROPRIETARYHARDWARE

SCALE-UPARCHITECTURE

HARDWARE-BASEDINTELLIGENCE

CLOSED DEVELOPMENTPROCESS

Common,off-the-shelf hardware

Scale-outarchitecture

Software-basedintelligence

Open developmentprocess

Lower cost, standardized supply chain

Increased operational flexibility

More programmability, agility,and control

More flexible, well-integratedtechnology

24

• Contributions from Intel, SanDisk, CERN, and Yahoo

• Presenting Ceph Days in cities around the world and quarterly virtual Ceph Developer Summit events

• Over 11M downloads in the last 12 months

• Increased development velocity, authorship, and discussion has resulted in rapid feature expansion

OPEN ECOSYSTEMS

97 AUTHORS/MO

2,453 COMMITS/MO

260 POSTERS/MO

33 AUTHORS/MO

97 COMMITS/MO

138 POSTERS/MO

25

DISRUPTION IN THE STORAGE INDUSTRY

PUBLIC CLOUD STORAGE

TRADITIONALAPPLIANCES

SOFTWARE-DEFINED STORAGE

better

faster

more

less

weaker

limited

better

faster

less

more

stronger

broad

COST EFFICIENCY

PROVISIONING

VENDOR LOCK-IN

SKILL REQUIRED

GOVERNANCE

DEPLOYMENT OPTIONS

27

THE RED HAT STORAGE PORTFOLIO

Gluster management

Ceph data service

Gluster data service

Ceph management

OPE

N S

OU

RCE

SOFT

WAR

EST

AND

ARD

HAR

DW

ARE

Share-nothing, scale-out architecture provides durability and adapts to changing demands

Self-managing and self-healing features reduce operational overhead

Standards-based interfaces and full APIs ease integration with applications and systems

Supported by the experts at Red Hat

28

OVERVIEW: RED HAT GLUSTER STORAGE

• Purpose-built as a scale-out file store with a straightforward architecture suitable for public, private, and hybrid cloud

• Simple to install and configure, with a minimal hardware footprint

• Offers mature NFS, SMB and HDFS interfaces for enterprise use

Intuit uses Red Hat Gluster Storage to provide flexible, cost-effective storage for its industry-leading financial offerings

Nimble file storage for petabyte-scale workloads

Container-Native Storage• Persistent storage• Containerized storageEnterprise File Sharing• Media streaming• Active ArchivesEnterprise VirtualizationRich Media and Archival

CUSTOMER HIGHLIGHT: INTUIT

TARG

ET U

SE

CASE

S

29

OVERVIEW: RED HAT CEPH STORAGE

• Built from the ground up as a next-generation storage system, based on years of research and suitable for powering infrastructure platforms

• Highly tunable, extensible, and configurable, with policy-based control and no single point of failure

• Offers mature interfaces for block and object storage for the enterprise

Cisco uses Red Hat Ceph Storage to deliver storage for next-generation cloud services

Powerful distributed storage for the cloud and beyond

Cloud Infrastructure• VM storage with

OpenStack® Cinder, Glance Keystone, Manila, and Nova

• Object storage for tenant apps

Rich Media and ArchivalS3-compatible object storage

CUSTOMER HIGHLIGHT: CISCO

TARG

ET U

SE

CASE

S

30

STRENGTH: MASSIVELY DISTRIBUTED DATA

TARGET WORKLOAD: OPENSTACK

31

Allows for instantaneous parallel creation of VMs at massive scaleIntegrates easily and tightly with OpenStack Cinder, Glance, Nova, Keystone, and ManilaOffers instant backup capabilitiesProvides persistent object, file, and database storage for applications

32

STRENGTH: PETABYTE SCALE

TARGET WORKLOAD: OBJECT STORAGE

33

Stores unstructured data at web scale, using standard hardware Works with standard object APIs for a wide range of compatibility Spans multiple regions with no single point of failure Ideal for active archives, big data archives, and content libraries

34

STRENGTH: DEPLOYMENT FLEXIBILITY

HOSTS VIRTUAL MACHINE

S

PRIVATE CLOUD

CONTAINERSPUBLIC CLOUD

HYBRID CLOUD

TARGET WORKLOAD: CONTAINERS

35

Deploys storage alongside applications in containersLowers TCO by increasing utilization of resourcesUnifies container and storage orchestrationAllows for rapid adjustments to compute/storage ratio

CONTAINERNGINIX

CONTAINERMASTE

R

Node 2 Node 3

Node 1

Node 4

NGINIX CONTAINER

POSTGRES CONTAINER

36

STORAGE AT RED HAT

PHYSICAL VIRTUAL PRIVATE CLOUD CONTAINERS PUBLIC CLOUD