emc world 2016 - code.15 better together: scale-out databases on scale-out storage
TRANSCRIPT
1© Copyright 2016 EMC Corporation. All rights reserved. 1© Copyright 2016 EMC Corporation. All rights reserved.
BETTER TOGETHER: SCALE-OUT DATABASES ON SCALE-OUT STORAGECHRIS DUCHESNE, DEVELOPER ADVOCATE, EMC {code}
2© Copyright 2016 EMC Corporation. All rights reserved.
SCALE-OUT DATABASES
3© Copyright 2016 EMC Corporation. All rights reserved.
• Dataset Size• High Availability• Performance Requirements• Flexible Design• Everyone else seems to be doing it
WHY SCALE-OUT DATABASES?
4© Copyright 2016 EMC Corporation. All rights reserved.
• High performance dedicated servers– 16 - 32 CPU cores– 32 - 512 GB memory– 500 - 4000 GB of SSDs
• 3 to 16 nodes per database / cluster• Usually not source of record
SCALE-OUT DATABASE CONFIGURATIONS
5© Copyright 2016 EMC Corporation. All rights reserved.
• Operational Flexibility– Adding/Removing Nodes– Migrating Nodes– Adding Storage Capacity– Maintenance– Prepare for Performance Elasticity
• Test/Dev/QA Environments– Quick Spin-up / Spin-down– High performance?
CHALLENGES WITH SCALE-OUT DATABASES
6© Copyright 2016 EMC Corporation. All rights reserved.
• More and more enterprises are creating DBaaS (Database as a Service) teams, including scale-out databases• Multiple types of databases compounded by many
production instances == harder to contain• Need to find ways to simplify management of scale-
out databases
WHAT ABOUT SCALE-OUT DBAAS
7© Copyright 2016 EMC Corporation. All rights reserved.
• Maintenance on a server?– Unmount storage and mount on another server
• Need more performance?– ScaleIO can scale to millions of IOPs
• Need to add more storage to a node?– Remotely grow storage capacity easily
• Need to add more servers?– Easier to stick to standard servers without the local SSD
requirements
REDUCE COMPLEXITY WITH SCALEIO
8© Copyright 2016 EMC Corporation. All rights reserved. 8© Copyright 2016 EMC Corporation. All rights reserved.
DEMOBENEFITS OF SCALE-OUT DATABASES ON SCALEIO
9© Copyright 2016 EMC Corporation. All rights reserved.
SCALEIO SDC & SDSScaleIO Data Client
• Lightweight device driver• Allows ScaleIO block devices to be
presented to application server
ScaleIO Data Server
• Manages storage capacity on the application/storage server
• Provides backend access of devices to the SDCs
10© Copyright 2016 EMC Corporation. All rights reserved.
HOW DOES IT WORK?
CPU
RAM
Local Disk (HDD)
Local Disk (SSD)
CPU CPU CPU CPU
RAM
Local Disk (HDD)
Local Disk (SSD)
CPU CPU CPU
ScaleIO – SDS(Virtualized Storage Array)
CPU
RAM
Local Disk (HDD)
Local Disk (SSD)
CPU CPU CPU
ScaleIO – SDS(Virtualized Storage Array)
SDC SDCSDC
SDS SDS SDS
11© Copyright 2016 EMC Corporation. All rights reserved.
Two-Layer Hyper-Converged
DEPLOYMENT OPTIONS
ETH
S S S
C C C
S S S
APPS
STORAGE
MORESTORAGE
ETH
S S S
S S S
S S S
C C C
C C C
C C C
APPS & STORAGE
APPS & STORAGE
APPS & STORAGE
12© Copyright 2016 EMC Corporation. All rights reserved.
ARCHITECTURE – STANDARD
13© Copyright 2016 EMC Corporation. All rights reserved.
ARCHITECTURE – SCALEIO STORAGE
14© Copyright 2016 EMC Corporation. All rights reserved. 14© Copyright 2016 EMC Corporation. All rights reserved.
BENEFITS OF SCALE-OUT STORAGE• Equivalent or better performance when compared
to high performance local storage• Easy to manage – this isn’t fibre channel• Simplify recovery from downed nodes• Easily Add/Remove additional nodes
15© Copyright 2016 EMC Corporation. All rights reserved.
• Deploy Scale-Out Databases in Containers with Docker– Run multiple database instances on each server– Maintain close to bare-metal performance
• Utilize Advanced Schedulers to Automate Deployments– Mesos / Kubernetes / Swarm / Rancher
• Link Scale-out Storage to Containers with REX-Ray– Simplify using persistent storage in containerized environments
HOW CAN WE IMPROVE THINGS FURTHER?
16© Copyright 2016 EMC Corporation. All rights reserved.
CONTAINERS VS VIRTUAL MACHINES
17© Copyright 2016 EMC Corporation. All rights reserved.
PERSISTENT STORAGE WITH REX-RAY• Stateless -> Stateful transition• Multi-Platform Storage Management
integrated with Docker, Mesos, Marathon & Kubernetes
• OSS• https://github.com/emccode/rexray
18© Copyright 2016 EMC Corporation. All rights reserved.
ARCHITECTURE – CONTAINERS AND SCALEIO
19© Copyright 2016 EMC Corporation. All rights reserved.
ARCHITECTURE – HYPER-CONVERGED
20© Copyright 2016 EMC Corporation. All rights reserved. 20© Copyright 2016 EMC Corporation. All rights reserved.
DEMOCONTAINERIZED SCALE-OUT DATABASES ON SCALEIO
21© Copyright 2016 EMC Corporation. All rights reserved. 21© Copyright 2016 EMC Corporation. All rights reserved.
CHALLENGES - CONTAINERS & SCHEDULERS• Many new technologies• Proven in production?• 10GbE a requirement• Additional storage redundancy
22© Copyright 2016 EMC Corporation. All rights reserved.
@EMCcode@ChrisDuchesne
community.emccode.comCome visit us at
Booth #1044 or in the vLab
Questions?
23© Copyright 2016 EMC Corporation. All rights reserved. 23© Copyright 2016 EMC Corporation. All rights reserved.