open storage in the enterprise - red...
TRANSCRIPT
![Page 1: Open Storage in the Enterprise - Red Hatpeople.redhat.com/dblack/lceu2013/open_storage_in_the_enterprise-20131023.pdf18 DUSTIN L. BLACK, RHCA Implementation HP DL170e nodes with local](https://reader034.vdocuments.us/reader034/viewer/2022042220/5ec659b5c0696a74ce35303c/html5/thumbnails/1.jpg)
Open Storage in the Enterprise
With GlusterFS and Red Hat Storage
Dustin L. Black, RHCASr. Technical Account Manager & Team LeadRed Hat Global Support Services
LinuxCon Europe -- 2013-10-23
![Page 2: Open Storage in the Enterprise - Red Hatpeople.redhat.com/dblack/lceu2013/open_storage_in_the_enterprise-20131023.pdf18 DUSTIN L. BLACK, RHCA Implementation HP DL170e nodes with local](https://reader034.vdocuments.us/reader034/viewer/2022042220/5ec659b5c0696a74ce35303c/html5/thumbnails/2.jpg)
DUSTIN L. BLACK, RHCA2
Dustin L. Black, RHCA
Sr. Technical Account Manager
Red Hat, Inc.
@dustinlblack
![Page 3: Open Storage in the Enterprise - Red Hatpeople.redhat.com/dblack/lceu2013/open_storage_in_the_enterprise-20131023.pdf18 DUSTIN L. BLACK, RHCA Implementation HP DL170e nodes with local](https://reader034.vdocuments.us/reader034/viewer/2022042220/5ec659b5c0696a74ce35303c/html5/thumbnails/3.jpg)
DUSTIN L. BLACK, RHCA3
#whatis TAM
● Premium named-resource support● Proactive and early access● Regular calls and on-site engagements● Customer advocate within Red Hat and upstream● Multi-vendor support coordinator● High-touch access to engineering● Influence for software enhancements● NOT Hands-on or consulting
![Page 4: Open Storage in the Enterprise - Red Hatpeople.redhat.com/dblack/lceu2013/open_storage_in_the_enterprise-20131023.pdf18 DUSTIN L. BLACK, RHCA Implementation HP DL170e nodes with local](https://reader034.vdocuments.us/reader034/viewer/2022042220/5ec659b5c0696a74ce35303c/html5/thumbnails/4.jpg)
DUSTIN L. BLACK, RHCA4
What is GlusterFS?
● Scalable, general-purpose storage platform● POSIX-y Distributed File System● Object storage (swift)● Distributed block storage (qemu)● Flexible storage (libgfapi)
● No Metadata Server● Heterogeneous Commodity Hardware● Standards-Based – Clients, Applications, Networks● Flexible and Agile Scaling
● Capacity – Petabytes and beyond● Performance – Thousands of Clients
![Page 5: Open Storage in the Enterprise - Red Hatpeople.redhat.com/dblack/lceu2013/open_storage_in_the_enterprise-20131023.pdf18 DUSTIN L. BLACK, RHCA Implementation HP DL170e nodes with local](https://reader034.vdocuments.us/reader034/viewer/2022042220/5ec659b5c0696a74ce35303c/html5/thumbnails/5.jpg)
DUSTIN L. BLACK, RHCA5
Fuse Native NFS
libgfapi
Samba QEMU application
Network Interconnect
![Page 6: Open Storage in the Enterprise - Red Hatpeople.redhat.com/dblack/lceu2013/open_storage_in_the_enterprise-20131023.pdf18 DUSTIN L. BLACK, RHCA Implementation HP DL170e nodes with local](https://reader034.vdocuments.us/reader034/viewer/2022042220/5ec659b5c0696a74ce35303c/html5/thumbnails/6.jpg)
DUSTIN L. BLACK, RHCA6
What is Red Hat Storage?
● Enterprise Implementation of GlusterFS● Software Appliance● Bare Metal Installation● Built on RHEL + XFS● Subscription Model● Storage Software Appliance
● Datacenter and Private Cloud Deployments● Virtual Storage Appliance
● Amazon Web Services Public Cloud Deployments
![Page 7: Open Storage in the Enterprise - Red Hatpeople.redhat.com/dblack/lceu2013/open_storage_in_the_enterprise-20131023.pdf18 DUSTIN L. BLACK, RHCA Implementation HP DL170e nodes with local](https://reader034.vdocuments.us/reader034/viewer/2022042220/5ec659b5c0696a74ce35303c/html5/thumbnails/7.jpg)
DUSTIN L. BLACK, RHCA7
release
iterate
debugenhance
upstream
glusterfs-3.4
release
![Page 8: Open Storage in the Enterprise - Red Hatpeople.redhat.com/dblack/lceu2013/open_storage_in_the_enterprise-20131023.pdf18 DUSTIN L. BLACK, RHCA Implementation HP DL170e nodes with local](https://reader034.vdocuments.us/reader034/viewer/2022042220/5ec659b5c0696a74ce35303c/html5/thumbnails/8.jpg)
Open Storage in the EnterpriseWith GlusterFS and Red Hat Storage
Use Case:Media Storage via Object Interface
![Page 9: Open Storage in the Enterprise - Red Hatpeople.redhat.com/dblack/lceu2013/open_storage_in_the_enterprise-20131023.pdf18 DUSTIN L. BLACK, RHCA Implementation HP DL170e nodes with local](https://reader034.vdocuments.us/reader034/viewer/2022042220/5ec659b5c0696a74ce35303c/html5/thumbnails/9.jpg)
DUSTIN L. BLACK, RHCA9
Goals
● Media file storage for customer-facing app● Drop-in replacement for legacy object backend● 1PB plus 1TB/day growth rate● Minimal resistance to increasing scale● Multi-protocol capable for future services● Fast transactions for fingerprinting and transcoding
![Page 10: Open Storage in the Enterprise - Red Hatpeople.redhat.com/dblack/lceu2013/open_storage_in_the_enterprise-20131023.pdf18 DUSTIN L. BLACK, RHCA Implementation HP DL170e nodes with local](https://reader034.vdocuments.us/reader034/viewer/2022042220/5ec659b5c0696a74ce35303c/html5/thumbnails/10.jpg)
DUSTIN L. BLACK, RHCA10
Implementation
● 12 Dell R710 nodes + MD1000/1200 DAS● Growth of 6 -> 10 -> 12 nodes
● ~1PB in total after RAID 6● GlusterFS Swift interface from OpenStack● Built-in file+object simultaneous access● Multi-GBit network with segregated backend
![Page 11: Open Storage in the Enterprise - Red Hatpeople.redhat.com/dblack/lceu2013/open_storage_in_the_enterprise-20131023.pdf18 DUSTIN L. BLACK, RHCA Implementation HP DL170e nodes with local](https://reader034.vdocuments.us/reader034/viewer/2022042220/5ec659b5c0696a74ce35303c/html5/thumbnails/11.jpg)
DUSTIN L. BLACK, RHCA11
1PB+
UFO/Swift
Backend Network
Load Balancer
User-Facing Application
Fingerprinting & Transcoding
![Page 12: Open Storage in the Enterprise - Red Hatpeople.redhat.com/dblack/lceu2013/open_storage_in_the_enterprise-20131023.pdf18 DUSTIN L. BLACK, RHCA Implementation HP DL170e nodes with local](https://reader034.vdocuments.us/reader034/viewer/2022042220/5ec659b5c0696a74ce35303c/html5/thumbnails/12.jpg)
Open Storage in the EnterpriseWith GlusterFS and Red Hat Storage
Use Case:Self-Service Provisioning with Accounting and Chargeback
![Page 13: Open Storage in the Enterprise - Red Hatpeople.redhat.com/dblack/lceu2013/open_storage_in_the_enterprise-20131023.pdf18 DUSTIN L. BLACK, RHCA Implementation HP DL170e nodes with local](https://reader034.vdocuments.us/reader034/viewer/2022042220/5ec659b5c0696a74ce35303c/html5/thumbnails/13.jpg)
DUSTIN L. BLACK, RHCA13
Goals
● Add file storage provisioning to existing self-service virtualization environment● Automate the administrative tasks
● Multi-tenancy● Subdivide and limit usage by corporate divisions
and departments● Allow for over-provisioning● Create a charge-back model
● Simple and transparent scaling
![Page 14: Open Storage in the Enterprise - Red Hatpeople.redhat.com/dblack/lceu2013/open_storage_in_the_enterprise-20131023.pdf18 DUSTIN L. BLACK, RHCA Implementation HP DL170e nodes with local](https://reader034.vdocuments.us/reader034/viewer/2022042220/5ec659b5c0696a74ce35303c/html5/thumbnails/14.jpg)
DUSTIN L. BLACK, RHCA14
Implementation
● Dell R510 nodes with local disk● ~30TB per node as one XFS filesystem● Bricks are subdirectories of the parent filesystem
● Volumes are therefore naturally over-provisioned● Quotas* placed on volumes to limit usage and
provide for accounting and charge-back● Only 4 gluster commands needed to allocate and
limit a new volume; Easily automated
*New feature in RHS 2.1; Coming in GlusterFS 3.5
![Page 15: Open Storage in the Enterprise - Red Hatpeople.redhat.com/dblack/lceu2013/open_storage_in_the_enterprise-20131023.pdf18 DUSTIN L. BLACK, RHCA Implementation HP DL170e nodes with local](https://reader034.vdocuments.us/reader034/viewer/2022042220/5ec659b5c0696a74ce35303c/html5/thumbnails/15.jpg)
DUSTIN L. BLACK, RHCA15
quota1
vol1
quota2
vol2
q3
vol3
quota4
vol4
quota5
vol5
One backend filesystem is sub-divided into directories
that are used as bricks
Over-Provisioned
Volumes created from the bricks share a total
amount of available space
/xfs/brick1/xfs/brick2/xfs/brick3/xfs/brick4/xfs/brick5
XFS Filesystem
Quotas are set on a volume level and are independent of each other and of
the underlying available space
# gluster volume create# gluster volume start# gluster volume quota enable# gluster volume quota limit-usage
x
![Page 16: Open Storage in the Enterprise - Red Hatpeople.redhat.com/dblack/lceu2013/open_storage_in_the_enterprise-20131023.pdf18 DUSTIN L. BLACK, RHCA Implementation HP DL170e nodes with local](https://reader034.vdocuments.us/reader034/viewer/2022042220/5ec659b5c0696a74ce35303c/html5/thumbnails/16.jpg)
Open Storage in the EnterpriseWith GlusterFS and Red Hat Storage
Use Case:NoSQL Backend with SLA-BoundGeo-Replication
![Page 17: Open Storage in the Enterprise - Red Hatpeople.redhat.com/dblack/lceu2013/open_storage_in_the_enterprise-20131023.pdf18 DUSTIN L. BLACK, RHCA Implementation HP DL170e nodes with local](https://reader034.vdocuments.us/reader034/viewer/2022042220/5ec659b5c0696a74ce35303c/html5/thumbnails/17.jpg)
DUSTIN L. BLACK, RHCA17
Goals
● Replace legacy database key/blob architecture● Divide and conquer
● NoSQL layer for key/pointer● Scalable storage layer for blob payload
● Active/Active sites with 30-minute replication SLA● Performance tuned for small-file WORM patterns
![Page 18: Open Storage in the Enterprise - Red Hatpeople.redhat.com/dblack/lceu2013/open_storage_in_the_enterprise-20131023.pdf18 DUSTIN L. BLACK, RHCA Implementation HP DL170e nodes with local](https://reader034.vdocuments.us/reader034/viewer/2022042220/5ec659b5c0696a74ce35303c/html5/thumbnails/18.jpg)
DUSTIN L. BLACK, RHCA18
Implementation
● HP DL170e nodes with local disk● ~4TB per node● Cassandra replicated NoSQL layer for key/pointer● GlusterFS parallel geo-replication* for data payload
site copy exceeding SLA standards● Worked with Red Hat Engineering to modify
application data patterns for better small-file performance
*New feature in RHS 2.1; Coming in GlusterFS 3.5
![Page 19: Open Storage in the Enterprise - Red Hatpeople.redhat.com/dblack/lceu2013/open_storage_in_the_enterprise-20131023.pdf18 DUSTIN L. BLACK, RHCA Implementation HP DL170e nodes with local](https://reader034.vdocuments.us/reader034/viewer/2022042220/5ec659b5c0696a74ce35303c/html5/thumbnails/19.jpg)
DUSTIN L. BLACK, RHCA19
/gluster1
/gluster2
master_vol
slave_vol
key1>pointer1
key2>pointer2
key1>pointer1
key2>pointer2
master_vol
slave_vol
/gluster1
/gluster2
![Page 20: Open Storage in the Enterprise - Red Hatpeople.redhat.com/dblack/lceu2013/open_storage_in_the_enterprise-20131023.pdf18 DUSTIN L. BLACK, RHCA Implementation HP DL170e nodes with local](https://reader034.vdocuments.us/reader034/viewer/2022042220/5ec659b5c0696a74ce35303c/html5/thumbnails/20.jpg)
Questions?
Open Storage in the EnterpriseWith GlusterFS and Red Hat Storage
![Page 21: Open Storage in the Enterprise - Red Hatpeople.redhat.com/dblack/lceu2013/open_storage_in_the_enterprise-20131023.pdf18 DUSTIN L. BLACK, RHCA Implementation HP DL170e nodes with local](https://reader034.vdocuments.us/reader034/viewer/2022042220/5ec659b5c0696a74ce35303c/html5/thumbnails/21.jpg)
21
Do it!
● Build a test environment in VMs in just minutes!● Get the bits:
● Fedora 19 has GlusterFS packages natively● RHS 2.1 ISO available on the Red Hat Portal● Go upstream: www.gluster.org
![Page 22: Open Storage in the Enterprise - Red Hatpeople.redhat.com/dblack/lceu2013/open_storage_in_the_enterprise-20131023.pdf18 DUSTIN L. BLACK, RHCA Implementation HP DL170e nodes with local](https://reader034.vdocuments.us/reader034/viewer/2022042220/5ec659b5c0696a74ce35303c/html5/thumbnails/22.jpg)
Thank You!
● [email protected] / @dustinlblack
● RHS:
www.redhat.com/storage/
● GlusterFS:
www.gluster.org
● TAM: access.redhat.com/support/offerings/tam/
@Glusterorg
@RedHatStorage
Gluster
Red Hat Storage
Slides Available at: http://people.redhat.com/dblack
Open Storage in the EnterpriseWith GlusterFS and Red Hat Storage