database cluster as a service in openstack

Database Cluster as a Service in OpenStackIntegrated, Scalable, Highly Available and Secure

Ivan Zora>, CTO -‐ 14.11.2013

Thursday, 14 November 13

Who is Ivan

?Thursday, 14 November 13

SkySQL•Leading provider of open source databases, services and solutions

•Home for the founders and the original developers of the core of MySQL

•The creators of MariaDB, the drop-‐in, innovative replacement of MySQL


MySQL , MariaDB and Percona Serverin the Cloud


The Status of MySQLMySQL

•From Oracle, acquired with Sun in 2010

•Trademark, code, documentation, bugs are available at *.mysql.com

•5.6 GA now•5.7 DMR available for testing

MariaDB

•5.5 GA: branch from Oracle with add-‐ons from Percona and other key contributors

•10.0 Alpha: fork from Oracle with add-‐ons and new storage engines

Percona

•5.6 GA: branch from Oracle with add-‐ons from MariaDB and other key contributors

5Thursday, 14 November 13

More in-‐depth featuresMySQL

•5.6• Scales up to 48 CPU threads• Performance Schema• Memcached plugin for InnoDB• InnoDB API• InnoDB Full Text Indexing• Replication with Global Transaction ID• HA utilities• Improved DDL online operations• Transportable tablespaces• Subquery optimization• Batched key access and multi-‐range read

• Thread pool• PAM Authentication• Audit plugin

MariaDB•5.5

• Subquery optimization• Batched key access and multi-‐range read

• Index condition pushdown• and many, many other improvements in the optimizer...

• SphinxSE and ScaleDB engines• Handler Socket• Dynamic and virtual columns• Group Commit for binary log• Microsecond Support• GIS operations• XtraDB Engine• PAM Authentication• Thread pool• Audit plugin

Percona•5.6

• Porting of the MySQL 5.6 features• XtraDB 5.6• Per-‐table/index/user/client performance counters

• FlashCache support


More in-‐depth featuresMySQL

•5.7• Improved DDL online operations• EXPLAIN improvements• Improved triggers• MySQL Client > syslog• Parallel thread replication• EXPLAIN in JSON format• EXPLAIN for running statements• Subquery optimization• Loss-‐less MySQL Replication• Multi-‐source replication

MariaDB

•10.0• Includes XtraDB 5.6• Performance schema• Fusion I/O atomic writes• Further optimizer improvements• Improved DDL online operations• Replication with Global Transaction ID• Parallel thread replication• Multi-‐source replication• EXPLAIN for running threads and logged queries

• Engine-‐independent table statistics• Per-‐engine table attributes• Per-‐thread memory usage

• TokuDB, Spider, CASSANDRA and CONNECT storage engine


High AvailabilityMySQL

•MySQL Replication with HA utilities

•DRBD from Linbit•Shared storage with Oracle Enterprise Linux

MariaDB•MySQL Replication with MHA and Pacemaker

•MariaDB Galera Cluster Synchronous Replication with Codership Galera

•Support from SkySQL for:•DRBD•Shared storage

Percona•MySQL Replication with Percona Replication Manager

•Percona XtraDB ClusterSynchronous Replication with Codership Galera

•Support for:•DRBD•Shared storage


ScalabilityMySQL

• Improved InnoDB scalability•Group commit, parallel and multi-‐source replication in 5.7

•MySQL Replication for read scalability

•MySQL Cluster for high volume/small transactions

MariaDB• In MariaDB 5.5:

• XtraDB 5.5• Group commit•MySQL Replication for read scalability

• TokuDB and ScaleDB engine

• Plus, in MariaDB 10:• XtraDB 5.6• Parallel and multi-‐source replication• Spider and Cassandra engine

• MariaDB Galera Cluster for synchronous replication

• MySQL Cluster for high volume/small transactions

Percona

•XtraDB 5.6•Group commit•MySQL Replication for read scalability

•Percona XtraDB Cluster for synchronous replication


Ease of UseMySQL

•MySQL Replication Utilities

•Enterprise Monitoring

•Online Backup•MySQL Workbench

MariaDB

•MHA

•Enterprise Monitor

•Xtrabackup•Query Editor•MariaDB Manager

Percona

•Percona Replication Manager

•Nagios and Cacti plugins•Xtrabackup•Percona Toolkit


MySQL in Rackspace

•Rackspace Cloud Database•MySQL Community 5.1•Up to 16GB RAM, 150GB storage

•Standard Servers•Up to 8 vCPUs, 30GB RAM, 1.2TB storage•BYODBMS


MySQL in HPCloud

•Manage a single DB instance through the REST API

•Standard Servers•Up to 8 vCPUs, 30GB RAM, 960GB storage•BYODBMS

•Marketplace•GenieDB•MariaDB ScaleDB


MySQL in AWS

•Amazon RDS•MySQL Community 5.1, 5.5 & 5.6•Reserved instances, provisioned IO and Flash storage•32 vCPUs, 244GB Memory, 10GB network

•Automatic backups, Storage replication, MySQL Replication•Limited tuning

•Standard Servers and Marketplace•Similar instances•MariaDB on CentOS, Debian and Ubuntu•GenieDB


MySQL and OpenStackThursday, 14 November 13

MySQL [or MariaDB, or Percona] in OpenStack

•Used as repository -‐ alternative to SQLite•Increased portability and availability

•High Availability is the key factor:•MySQL Replication•DRBD•Galera


MySQL Replication

16

DatabaseDatabase

Database DatabaseDatabase

Database

Node Manager

HA Manager

Control Network Compute


DRBD

17


Database Database Database Database


Galera Cluster

18


Database Database DatabaseDatabase Database Database


MySQL as a ServiceThursday, 14 November 13

Automatic Provisioning

•REST-‐based for interoperation

•GUI-‐assisted for end-‐users

•From server to cluster


The false promise of the elasticity•Distributed MySQL is not something you can have out of the box•Or not for all your applications

21

Read-Write

Read-Only Read-Only

binlog

relaylog relaylog relaylog relaylog

DatabaseDatabase

Database Database Database

LU

LU LU LU LU

SHARD0-99

SHARD 100-199

SHARD 200-299

SHARD 300-399

1 2 3 4M


DBaaS vs PaaS

22

Database Database

vs.


Storage

•Backups on Swift•Full or incremental

•Moving tablespaces in Cinder•For high availability•For backups and archiving•For maintenance

23

in01

DatabaseDatabase

in02

Database

in03

Database

in04

Database Database

in05

Database


Networking

•Multiple NICs and floating IPs

•Public IPs

•Local proxies vs enhanced connectors

24

Database Database Database Database Database Database


Security

•LDAP & Keystone

•Integration with MySQL•LDAP / OpenLDAP•MySQL Security Plugin using OpenLDAP

25Database

Keystone

LDAP


MariaDB Cluster in OpenStackThursday, 14 November 13

What’s new with MariaDB

27

MariaDB Manager MariaDB Cluster

MariaDBServer(s)

MHA Galera

GUI

API

Monitor

MaxScale

Configuration& Provisioning


SkySQLMaxScale

28

MaxScaleCore

Data Stores

Client Applications

Backend Backend Backend

Frontend Frontend Frontend Frontend Frontend

Clie

nt

Clie

nt

Clie

nt

Clie

nt

Clie

nt

Router

Router

Router

Monitor

Monitor

Auth

Auth

Auth

Service Service Service

Listener Listener Listener Listener

Server Server Server ServerServer


IntroducingMariaDBManager

29

RESTful(APICore

Monitor/ManagementInterface



Monitor(DataProvider

Agent Agent Agent

MariaDB(Galera(Cluster


MariaDB Galera Cluster -‐ State


Provision a new node

31

HTTP POST method -‐ Create a nodeURI:

• .../restfulapi/system/systemid/node• Parameters:• name• hostname• publicip, privateip, port• instanceid• dbusername, dbpassword• ...

1



32



1

HTTP POST method -‐ Run the commandConnect to the new nodeURI:

• .../restfulapi/command/connect• Parameters:• systemid• nodeid• rootpassword

2



33



1



2

HTTP POST method -‐ Run the commandProbe the state of the nodeURI:

• .../restfulapi/command/probe• Parameters:• systemid• nodeid

3



34



1



2

HTTP POST method -‐ Run the commandProbe the state of the nodeURI:

• .../restfulapi/command/probe• Parameters:• systemid• nodeid

3

HTTP POST method -‐ Run the commandProvision the nodeURI:

• .../restfulapi/command/provision• Parameters:• systemid• nodeid

4


Start a node

35

HTTP POST method -‐ Run the commandStart the nodeURI:

• .../restfulapi/command/start• Parameters:• systemid• nodeid


Retrieve the status of a cluster

36

• .../restfulapi/system/systemid


Monitor the # of connections of a node

37

• ../res^ulapi/system/systemid/node/nodeid/monitor/{monitorid}/data


What’s coming•MariaDB Replication and MHA

•MariaDB 10

•MaxScale integration

•Juju charms for MariaDB Cluster and MariaDB Manager

•Puppet & Chef manifests and recipes•MariaDB and Trove

•Trove for MariaDB Manager38


For More Information...

39

• MariaDB Enterprise:www.mariadb.orgwww.mariadb.com

• MariaDB Manager and MaxScale:http://downloads.skysql.com/archives

• Bugs: http://bugs.skysql.com


http://www.mariadb.com




http://downloads.skysql.com/archives




http://bugs.skysql.com

http://bugs.skysql.com

Thank You!

[email protected]

www.slideshare.net/izorattiwww.skysql.com

Pictures courtesy of:•www.pluralsight.com


mailto:[email protected]








http://www.interflora.co.uk

http://www.interflora.co.uk

database cluster as a service in openstack

Technology

moreindepthfeatures

features xtradb5

drbd sharedstorage supportfor

indexconditionpushdown

percona portingofthemysql5

threadmemoryusage tokudb

debianandubuntu geniedb

users fromservertocluster20