red hat storage 2014 - product(s) overview
DESCRIPTION
Red Hat Storage - Software Defined Storage solutions for File, Block and Object purposes.TRANSCRIPT
![Page 1: Red Hat Storage 2014 - Product(s) Overview](https://reader033.vdocuments.us/reader033/viewer/2022060116/557d5f5ed8b42abf3d8b501c/html5/thumbnails/1.jpg)
RED HAT STORAGELIBERATE YOUR INFORMATION
Marcel Hergaarden
Solution Architect, Red Hat
Tuesday, October 28, 2014
![Page 2: Red Hat Storage 2014 - Product(s) Overview](https://reader033.vdocuments.us/reader033/viewer/2022060116/557d5f5ed8b42abf3d8b501c/html5/thumbnails/2.jpg)
Agenda
● Red Hat Storage and Inktank Ceph
● Software Defined concept
● Setup Hierarchy
● Storage Topology Types
● Storage for Openstack
● RHS 3.0 New Features
● Inktank Ceph introduction
![Page 3: Red Hat Storage 2014 - Product(s) Overview](https://reader033.vdocuments.us/reader033/viewer/2022060116/557d5f5ed8b42abf3d8b501c/html5/thumbnails/3.jpg)
Inktank Ceph
● April 2014: Red Hat acquires Inktank Ceph
![Page 4: Red Hat Storage 2014 - Product(s) Overview](https://reader033.vdocuments.us/reader033/viewer/2022060116/557d5f5ed8b42abf3d8b501c/html5/thumbnails/4.jpg)
Future Red Hat Storage: 2 Flavours
● Red Hat Storage – Gluster edition
Mostly used for filebased storage purposesCan also be used as Virtualization Store or Object Storage
● Red Hat Storage – Ceph Edition
Positioned as defacto storage platform for OpenStackCeph offers block- and objectbased access
![Page 5: Red Hat Storage 2014 - Product(s) Overview](https://reader033.vdocuments.us/reader033/viewer/2022060116/557d5f5ed8b42abf3d8b501c/html5/thumbnails/5.jpg)
Red Hat Storage Positioning
FILE OBJECT BLOCK
RED HAT STORAGE
CEPH
Best Scale-Out NAS
Best Object Store Kernel-supported& exposed
Not yet available
SWIFT-basedFile + Object access
Through API-only(libqemu)
![Page 6: Red Hat Storage 2014 - Product(s) Overview](https://reader033.vdocuments.us/reader033/viewer/2022060116/557d5f5ed8b42abf3d8b501c/html5/thumbnails/6.jpg)
What means Software Defined Storage ?
● RHS is a software solution, not an appliance with disks
![Page 7: Red Hat Storage 2014 - Product(s) Overview](https://reader033.vdocuments.us/reader033/viewer/2022060116/557d5f5ed8b42abf3d8b501c/html5/thumbnails/7.jpg)
Open Software-Defined Storage● Stable scale-out storage platform
● Runs On-premise, in Private- and in Public Cloud
SCALE-OUT STORAGEARCHITECTURE
PHYSICAL
Standard x86 systemsScale-out NAS solutions
VIRTUAL
Include idle or legacy resources
CLOUD
EBSEBS
PERSISTENT DATA STORES
Physical Virtual Cloud
Red Hat Storage Server: Software Defined Storage Platform
Continuous Storage Platform
Converged Compute and Storage
Red Hat Storage Server: Software Defined Storage Platform
![Page 8: Red Hat Storage 2014 - Product(s) Overview](https://reader033.vdocuments.us/reader033/viewer/2022060116/557d5f5ed8b42abf3d8b501c/html5/thumbnails/8.jpg)
ENTERPRISEMOBILITY
Increase Data, Application and Infrastructure Agility
CLOUDAPPLICATIONS
CONVERGED COMPUTE AND STORAGE
OPEN, SOFTWARE-DEFINED STORAGE PLATFORM
SCALE-OUT STORAGEARCHITECTURE
PHYSICAL
Standard x86 systemsScale-out NAS solutions
VIRTUAL
Include idle or legacy resources
CLOUD
EBSEBS
BIG DATAWORKLOADS
ENTERPRISEAPPLICATIONS
DATA SERVICES
PERSISTANT DATA STORES
InktankCephEnterprise
FILE SERVICES OPEN OBJECT APIsBLOCK IO
![Page 9: Red Hat Storage 2014 - Product(s) Overview](https://reader033.vdocuments.us/reader033/viewer/2022060116/557d5f5ed8b42abf3d8b501c/html5/thumbnails/9.jpg)
COMPUTECOMPUTE
SOFTWARE-DEFINED / BASED
COMPUTE(Virtualization)
SOFTWARE-DEFINED / BASED
COMPUTE(Virtualization)
STORAGESTORAGE
SOFTWARE-DEFINED / BASED
STORAGE
SOFTWARE-DEFINED / BASED
STORAGE
NETWORKINGNETWORKING
SOFTWARE-DEFINED / BASED
NETWORKING
SOFTWARE-DEFINED / BASED
NETWORKING
ENVIRONMENTALENVIRONMENTAL
SOFTWARE-DEFINED / BASED
FACILITIES
SOFTWARE-DEFINED / BASED
FACILITIES
DATA CENTER FABRIC DATA CENTER FABRIC
Cornerstone of the modern SOFTWARE DEFINED DATACENTER
![Page 10: Red Hat Storage 2014 - Product(s) Overview](https://reader033.vdocuments.us/reader033/viewer/2022060116/557d5f5ed8b42abf3d8b501c/html5/thumbnails/10.jpg)
Red Hat Storage design philosophy
● Runs on X86 commodity hardware systems
● Agnostic deployment (on-premise, virtual, cloud)
● Provides a single namespace storage capacity
● Elastic storage pool – grow or shrink online as needed
● Linear scaling – either scale-up as scale-out
● Components can be subject to hardware failures
![Page 11: Red Hat Storage 2014 - Product(s) Overview](https://reader033.vdocuments.us/reader033/viewer/2022060116/557d5f5ed8b42abf3d8b501c/html5/thumbnails/11.jpg)
Scale-out Software-Defined Architecture
1TB
4TB
Scale out performance, capacity, and availability
Scal
e up
cap
acity
SINGLE GLOBAL NAMESPACE
...
...SERVER(CPU/MEM)
............ ... ...
Global namespace
Aggregates CPU, memory, network capacity
Deploys on RHEL-supported servers and directly connected storage
Scale out linearly
Scale out performance and capacity as needed
![Page 12: Red Hat Storage 2014 - Product(s) Overview](https://reader033.vdocuments.us/reader033/viewer/2022060116/557d5f5ed8b42abf3d8b501c/html5/thumbnails/12.jpg)
VOLUME
VARIETY
SCALE
PORTABILITY
New data problems
![Page 13: Red Hat Storage 2014 - Product(s) Overview](https://reader033.vdocuments.us/reader033/viewer/2022060116/557d5f5ed8b42abf3d8b501c/html5/thumbnails/13.jpg)
Business data growth estimates in 2014
Virtualization
StandardGrowth
MobileComputing
+
Big Data
SocialNetworks
Internet ofThings
+
CloudComputing
50% 100%
20142013
![Page 14: Red Hat Storage 2014 - Product(s) Overview](https://reader033.vdocuments.us/reader033/viewer/2022060116/557d5f5ed8b42abf3d8b501c/html5/thumbnails/14.jpg)
What happens in an Internet minute ?
![Page 15: Red Hat Storage 2014 - Product(s) Overview](https://reader033.vdocuments.us/reader033/viewer/2022060116/557d5f5ed8b42abf3d8b501c/html5/thumbnails/15.jpg)
The Challenge
Exponential Growth of Data
IT Storage Budget
Existing systems don’t scale and are notbuilt / optimized forunstructured data
Increasing cost and complexity
Need to invest in newplatforms ahead of time 2010 2020
![Page 16: Red Hat Storage 2014 - Product(s) Overview](https://reader033.vdocuments.us/reader033/viewer/2022060116/557d5f5ed8b42abf3d8b501c/html5/thumbnails/16.jpg)
Red Hat Storage Setup Topology
RHS Operating System RHS Operating System
Brick #1 Brick #2
![Page 17: Red Hat Storage 2014 - Product(s) Overview](https://reader033.vdocuments.us/reader033/viewer/2022060116/557d5f5ed8b42abf3d8b501c/html5/thumbnails/17.jpg)
Red Hat Storage Setup Topology
RHS Operating System RHS Operating System
SMB 2.0
![Page 18: Red Hat Storage 2014 - Product(s) Overview](https://reader033.vdocuments.us/reader033/viewer/2022060116/557d5f5ed8b42abf3d8b501c/html5/thumbnails/18.jpg)
server1:/exp1 server2:/exp1
DISTRIBUTED VOLUME
FILE 1 FILE 2 FILE 3
BRICK BRICK
MOUNT POINT
server1:/exp1 server2:/exp1
DISTRIBUTED VOLUME
BRICK BRICK
Red Hat Storage: Distributed Volume
![Page 19: Red Hat Storage 2014 - Product(s) Overview](https://reader033.vdocuments.us/reader033/viewer/2022060116/557d5f5ed8b42abf3d8b501c/html5/thumbnails/19.jpg)
MOUNT POINT
server1:/exp1 server2:/exp1
REPLICATED VOLUME
FILE 1 FILE 2
BRICK BRICK
Red Hat Storage: Replicated Volume
![Page 20: Red Hat Storage 2014 - Product(s) Overview](https://reader033.vdocuments.us/reader033/viewer/2022060116/557d5f5ed8b42abf3d8b501c/html5/thumbnails/20.jpg)
MOUNT POINT
Replicated Volume 0
DISTRIBUTED VOLUME
FILE 1 FILE 2
BRICK(exp1)
Replicated Volume 1
BRICK(exp2)
server1 server2
BRICK(exp3)
BRICK(exp4)
server3 server4
Red Hat Storage: Distributed Replicated Volume
![Page 21: Red Hat Storage 2014 - Product(s) Overview](https://reader033.vdocuments.us/reader033/viewer/2022060116/557d5f5ed8b42abf3d8b501c/html5/thumbnails/21.jpg)
Featured use-cases
● Scalable Storagelibrary: Into Petabytes scale
● VM store for RHEV (Red Hat Virtualization)
● Target store for Backup and Archiving (Commvault)
● Storage infra for OpenStack: Cinder, Glance & Swift
● Storage for Fileservice and/or data archives
● Storage for (very) Large files, also Big Data purposes
● Storage for Multimedia purposes
● Windows support: File service and Active Directory
![Page 22: Red Hat Storage 2014 - Product(s) Overview](https://reader033.vdocuments.us/reader033/viewer/2022060116/557d5f5ed8b42abf3d8b501c/html5/thumbnails/22.jpg)
Targetstore for CommVault Simpana
![Page 23: Red Hat Storage 2014 - Product(s) Overview](https://reader033.vdocuments.us/reader033/viewer/2022060116/557d5f5ed8b42abf3d8b501c/html5/thumbnails/23.jpg)
CommVault SimpanaDatastreams benefits
![Page 24: Red Hat Storage 2014 - Product(s) Overview](https://reader033.vdocuments.us/reader033/viewer/2022060116/557d5f5ed8b42abf3d8b501c/html5/thumbnails/24.jpg)
Red Hat Storage inside Openstack
![Page 25: Red Hat Storage 2014 - Product(s) Overview](https://reader033.vdocuments.us/reader033/viewer/2022060116/557d5f5ed8b42abf3d8b501c/html5/thumbnails/25.jpg)
Red Hat Storage 3.0
![Page 26: Red Hat Storage 2014 - Product(s) Overview](https://reader033.vdocuments.us/reader033/viewer/2022060116/557d5f5ed8b42abf3d8b501c/html5/thumbnails/26.jpg)
New key features in Red Hat Storage 3.0
Enhanced Data Protection● Snapshots of Gluster volumes
● Consistent Point-in-Time copies of data
● Help to improve Disaster Recovery Use-Case;
● Create multiple consistent Point-in-Time copies during aday
● Roll-back within minutes to the last Snapshot in case of aVirus-attack, Admin-error, etc.
● Doesn't replace Backup/Recovery but enhances it
![Page 27: Red Hat Storage 2014 - Product(s) Overview](https://reader033.vdocuments.us/reader033/viewer/2022060116/557d5f5ed8b42abf3d8b501c/html5/thumbnails/27.jpg)
Cluster Monitoring
● Nagios-based RHS-Cluster health and performance information
● 3 different deployment options;
● Nagios web-frontend standalone
● Agent-only for integrating in existing Nagios environments
● As a RHS-Console plugin
New key features in Red Hat Storage 3.0
![Page 28: Red Hat Storage 2014 - Product(s) Overview](https://reader033.vdocuments.us/reader033/viewer/2022060116/557d5f5ed8b42abf3d8b501c/html5/thumbnails/28.jpg)
Other Important enhancements in RHS 3.0Deep Hadoop Integration
HDFS-compatible filesystem eliminates overhead of data movement
![Page 29: Red Hat Storage 2014 - Product(s) Overview](https://reader033.vdocuments.us/reader033/viewer/2022060116/557d5f5ed8b42abf3d8b501c/html5/thumbnails/29.jpg)
Flexibility at each phase of your processing pipeline
Data scientist, Programmers, Business Analyst
Data fromAny Source
ApacheHadoop
(MapReduce/Pig/Hive/Hbase, etc)
Red Hat Storage
Posix
Any LinuxTool or
Application(grep, sed, awk, find,
python, etc)
Posix HDFS
Commodity Hardware
Load Pre-process Analyze Export
Data to Any Source
Post-process
Posix Posix
If Necessary... If Necessary...
Any LinuxTool or
Application(grep, sed, awk, find,
python, etc)
![Page 30: Red Hat Storage 2014 - Product(s) Overview](https://reader033.vdocuments.us/reader033/viewer/2022060116/557d5f5ed8b42abf3d8b501c/html5/thumbnails/30.jpg)
Other Important enhancements in RHS 3.0
Enhanced Capacity● Up to 60 disks per RHS node => lower TCO ● Up to ~205TB per node, netto usable capacity● Clustersize up to 120 nodes (was 64 nodes)
Maintainability● None-disruptive upgrades
Introduction of new package delivery options● Red Hat Storage Starter Pack SKU
![Page 31: Red Hat Storage 2014 - Product(s) Overview](https://reader033.vdocuments.us/reader033/viewer/2022060116/557d5f5ed8b42abf3d8b501c/html5/thumbnails/31.jpg)
Other Important enhancements in RHS 3.0
Brick resource changes
SSD disks as brick● SSD’s are now officially supported for the use as brick component
SAN resources● SAN disk resources may be used as brick (architecture review req.)
![Page 32: Red Hat Storage 2014 - Product(s) Overview](https://reader033.vdocuments.us/reader033/viewer/2022060116/557d5f5ed8b42abf3d8b501c/html5/thumbnails/32.jpg)
● Intuitive user interface● Manages massive
scale out● Installation and
configuration● Volume management● On-premise and
public cloud● Integrates with RHEV-
M
Red Hat Storage Gluster edition consolesimplified management
![Page 33: Red Hat Storage 2014 - Product(s) Overview](https://reader033.vdocuments.us/reader033/viewer/2022060116/557d5f5ed8b42abf3d8b501c/html5/thumbnails/33.jpg)
Inktank Ceph Enterprise 1.2
![Page 34: Red Hat Storage 2014 - Product(s) Overview](https://reader033.vdocuments.us/reader033/viewer/2022060116/557d5f5ed8b42abf3d8b501c/html5/thumbnails/34.jpg)
Key Themes in Inktank Ceph Enterprise v1.2
Enterprise Readiness● RADOS Management● User Quotas● RHEL 7 support
Lower TCO● Erasure Coding
Performance● Primary OSD Affinity● Cache Tiering● Key/Value OSD-backend
![Page 35: Red Hat Storage 2014 - Product(s) Overview](https://reader033.vdocuments.us/reader033/viewer/2022060116/557d5f5ed8b42abf3d8b501c/html5/thumbnails/35.jpg)
Intro to Ceph Storage
![Page 36: Red Hat Storage 2014 - Product(s) Overview](https://reader033.vdocuments.us/reader033/viewer/2022060116/557d5f5ed8b42abf3d8b501c/html5/thumbnails/36.jpg)
Ceph RADOS
RADOSA software-based, reliable, autonomous, distributed object store comprised ofself-healing, self-managing, intelligent storage nodes and lightweight monitors
Reliable Autonomous Distributed Object Store
![Page 37: Red Hat Storage 2014 - Product(s) Overview](https://reader033.vdocuments.us/reader033/viewer/2022060116/557d5f5ed8b42abf3d8b501c/html5/thumbnails/37.jpg)
Ceph LIBRADOS
LIBRADOSA library allowing apps to directly access RADOS (C, C++, Java, Python, Ruby, PHP)
RADOSA software-based, reliable, autonomous, distributed object store comprised ofself-healing, self-managing, intelligent storage nodes and lightweight monitors
Library to access Rados
![Page 38: Red Hat Storage 2014 - Product(s) Overview](https://reader033.vdocuments.us/reader033/viewer/2022060116/557d5f5ed8b42abf3d8b501c/html5/thumbnails/38.jpg)
Ceph Unified Storage
RGWA web services gateway forobject storage, compatible
with S3 and Swift
LIBRADOSA library allowing apps to directly access RADOS (C, C++, Java, Python, Ruby, PHP)
RADOSA software-based, reliable, autonomous, distributed object store comprised ofself-healing, self-managing, intelligent storage nodes and lightweight monitors
RBDA reliable, fully-distributed
block device with cloudplatform integration
CEPHFSA distributed file system
with POSIX semantics andscale-out metadata
management
APP HOST/VM CLIENT
![Page 39: Red Hat Storage 2014 - Product(s) Overview](https://reader033.vdocuments.us/reader033/viewer/2022060116/557d5f5ed8b42abf3d8b501c/html5/thumbnails/39.jpg)
Ceph Object Storage Daemons
FS
DISK
OSD
DISK
OSD
FS
DISK
OSD
FS
DISK
OSD
FS
btrfsxfsext4
M
M
M
![Page 40: Red Hat Storage 2014 - Product(s) Overview](https://reader033.vdocuments.us/reader033/viewer/2022060116/557d5f5ed8b42abf3d8b501c/html5/thumbnails/40.jpg)
Ceph RADOS cluster
APPLICATION
M M
M M
M
RADOS CLUSTER
![Page 41: Red Hat Storage 2014 - Product(s) Overview](https://reader033.vdocuments.us/reader033/viewer/2022060116/557d5f5ed8b42abf3d8b501c/html5/thumbnails/41.jpg)
Ceph RADOS Components
OSDs:
10s to 10000s in a cluster
One per disk (or one per SSD, RAID group…)
Serve stored objects to clients
Intelligently peer for replication & recovery
Monitors:
Maintain cluster membership and state
Provide consensus for distributed decision-making
Small, odd number
These do not serve stored objects to clientsM
![Page 42: Red Hat Storage 2014 - Product(s) Overview](https://reader033.vdocuments.us/reader033/viewer/2022060116/557d5f5ed8b42abf3d8b501c/html5/thumbnails/42.jpg)
Ceph CRUSH algorithm Dynamic Data Placement
CRUSH:
Pseudo-random placement algorithm
Fast calculation, no lookup
Repeatable, deterministic
Statistically uniform distribution
Stable mapping
Limited data migration on change
Rule-based configuration
Infrastructure topology aware
Adjustable replication
Weighting
![Page 43: Red Hat Storage 2014 - Product(s) Overview](https://reader033.vdocuments.us/reader033/viewer/2022060116/557d5f5ed8b42abf3d8b501c/html5/thumbnails/43.jpg)
BLOCKSTORAGE
OBJECTSTORAGE
Equivalent toAmazon S3
Equivalent toAmazon EBS
FILESYSTEM
Not yetEnterprisesupported
Ceph Unified Storage
![Page 44: Red Hat Storage 2014 - Product(s) Overview](https://reader033.vdocuments.us/reader033/viewer/2022060116/557d5f5ed8b42abf3d8b501c/html5/thumbnails/44.jpg)
BLOCKSTORAGE
OBJECTSTORAGE
Equivalent toAmazon S3
Equivalent toAmazon EBS
Ceph Unified Storage
![Page 45: Red Hat Storage 2014 - Product(s) Overview](https://reader033.vdocuments.us/reader033/viewer/2022060116/557d5f5ed8b42abf3d8b501c/html5/thumbnails/45.jpg)
Ceph with OpenStack
OPEN STACK
KEYSTONE APISWIFT
APICINDER
APIGLANCE API NOVA
API
CEPH STORAGE CLUSTER(RADOS)
CEPH OBJECT GATEWAY(RGW)
CEPH BLOCK DEVICE(RBD)
HYPERVISOR(Qemu/KVM)
![Page 46: Red Hat Storage 2014 - Product(s) Overview](https://reader033.vdocuments.us/reader033/viewer/2022060116/557d5f5ed8b42abf3d8b501c/html5/thumbnails/46.jpg)
Ceph as Cloud Storage
WEB APPLICATION
APP SERVER APP SERVER APP SERVER
CEPH STORAGE CLUSTER(RADOS)
CEPH OBJECT GATEWAY(RGW)
CEPH OBJECT GATEWAY(RGW)
APP SERVER
S3/Swift S3/Swift S3/Swift S3/Swift
![Page 47: Red Hat Storage 2014 - Product(s) Overview](https://reader033.vdocuments.us/reader033/viewer/2022060116/557d5f5ed8b42abf3d8b501c/html5/thumbnails/47.jpg)
Ceph Cloud Storage including DR
WEB APPLICATION
APP SERVER
CEPH OBJECT GATEWAY(RGW)
CEPH STORAGE CLUSTER(US-EAST)
WEB APPLICATION
APP SERVER
CEPH OBJECT GATEWAY(RGW)
CEPH STORAGE CLUSTER(EU-WEST)
![Page 48: Red Hat Storage 2014 - Product(s) Overview](https://reader033.vdocuments.us/reader033/viewer/2022060116/557d5f5ed8b42abf3d8b501c/html5/thumbnails/48.jpg)
Ceph Web Scale Applications
WEB APPLICATION
APP SERVER APP SERVER APP SERVER
CEPH STORAGE CLUSTER(RADOS)
APP SERVER
NativeProtocol
NativeProtocol
NativeProtocol
NativeProtocol
![Page 49: Red Hat Storage 2014 - Product(s) Overview](https://reader033.vdocuments.us/reader033/viewer/2022060116/557d5f5ed8b42abf3d8b501c/html5/thumbnails/49.jpg)
Ceph Cold Storage
APPLICATION
CACHE POOL (REPLICATED)
BACKING POOL (ERASURE CODED)
CEPH STORAGE CLUSTER
![Page 50: Red Hat Storage 2014 - Product(s) Overview](https://reader033.vdocuments.us/reader033/viewer/2022060116/557d5f5ed8b42abf3d8b501c/html5/thumbnails/50.jpg)
Ceph management: Calimaris
![Page 51: Red Hat Storage 2014 - Product(s) Overview](https://reader033.vdocuments.us/reader033/viewer/2022060116/557d5f5ed8b42abf3d8b501c/html5/thumbnails/51.jpg)
![Page 52: Red Hat Storage 2014 - Product(s) Overview](https://reader033.vdocuments.us/reader033/viewer/2022060116/557d5f5ed8b42abf3d8b501c/html5/thumbnails/52.jpg)
Hands-on Red Hat Storage workshop
Red Hat Storage testing on Amazon Web Services(AWS)
https://engage.redhat.com/aws-test-drive-201308271223
![Page 53: Red Hat Storage 2014 - Product(s) Overview](https://reader033.vdocuments.us/reader033/viewer/2022060116/557d5f5ed8b42abf3d8b501c/html5/thumbnails/53.jpg)
THANK YOU
“...A DISRUPTIVE ANDUNSTOPPABLE FORCE.”
–IDC REPORT