mariadb prez by joffrey michaie

107
SkySQL AB © 2013 Confidential Principal Consultant Joffrey MICHAÏE Linux User Group of Mauritius SkySQL Ab / MariaDB

Upload: ishwon

Post on 22-Jun-2015

1.061 views

Category:

Technology


9 download

DESCRIPTION

MariaDB/MySQL presentation by Joffrey Michaie at the Linux User Group of Mauritius meeting on 15 August 2014, Flying Dodo, Bagatelle.

TRANSCRIPT

Page 1: MariaDB Prez by Joffrey Michaie

SkySQL AB © 2013 Confidential

Principal Consultant

Joffrey MICHAÏE

Linux User Group of MauritiusSkySQL Ab / MariaDB

Page 2: MariaDB Prez by Joffrey Michaie

SkySQL AB © 2013 Confidential

Agenda

•SkySQL General Presentation

•High-Availability Solutions for MySQL and MariaDB

•MariaDB history and 5.x presentation

•MariaDB 10 new features

•Questions / Answers

Page 3: MariaDB Prez by Joffrey Michaie

SkySQL AB © 2013 Confidential

Me, myself and I - Joffrey MICHAIE

• Unix Sysadmin from 2002 to 2009

• Joined MySQL Ab (Sun Microsystems) in 2009 as Junior consultant

• Joined SkySQL Ab in 2010 as Senior Consultant

• Principal Consultant since 2013

Page 4: MariaDB Prez by Joffrey Michaie

SkySQL AB © 2013 Confidential

Principal Duties

• Onsite and Remote Consulting• Performance tuning• Architecture and design• Migration• High availability• MySQL Cluster™ Consulting

Page 5: MariaDB Prez by Joffrey Michaie

SkySQL AB © 2013 Confidential

Live my life ? 2009-2014

Page 6: MariaDB Prez by Joffrey Michaie

SkySQL Ab 2012 Confidential

The MySQL AB Story

• MySQL

– #1 Open Source DBMS Worldwide

– 50K downloads per day since 2003

– +13M active installations

• MySQL AB

– Founded by Monty Widenius & David Axmark

– My (pronounced Muu) is Monty's daugther

3

Page 7: MariaDB Prez by Joffrey Michaie

SkySQL Ab 2012 Confidential

MySQL Founders turned SkySQL InvestorsThe shirts however are still as bad ...

Page 8: MariaDB Prez by Joffrey Michaie

SkySQL Ab 2012 Confidential

MySQL AB

• Founded 1995

• 2007 : $100M. Preparation for IPO - Nasdaq

• Jan 2008 : Bought by Sun for $1B

• May 2009 : Sun bought by Oracle for $6B

• European Commission Enquiry

– DBMS Competition

• Oracle made some promises

• Green light just in time for Xmas 2009

5

Page 9: MariaDB Prez by Joffrey Michaie

SkySQL Ab 2012 Confidential

The Oracle promises to the EU

6

• Continued Availability of Storage Engine APIs

• Non-assertion against any 3rd party or vendor for the

implementation of storage engines under GPL

• Same licence commitment for commercial customers

• All MySQL® improvement licenses under the GPL

• Non-mandatory support

• Increased R&D

• MySQL® Customer Advisory Board

• MySQL® Storage Engine Advisory Board

• MySQL® Reference Manual

• Maintain customer’s choice to buy yearly support

Page 10: MariaDB Prez by Joffrey Michaie

SkySQL Ab 2012 Confidential

SkySQL in 60 Seconds

8

The employees are ex-MySQL (actually >90%)

The founders are ex-MySQL executives The investors include the original founders of MySQL

Under Oracle's stewardship, a lot of MySQL customers are facing reduced service levels, price rises

& vendor lock-in

We offer 24/7 worldwide MySQL & MariaDB support, consulting & training for MySQL as well as

products designed to better deploy MySQL in the Cloud

– 85% of our revenues come from 24/7, 30-min-SLA support contracts

Operational since October 2010, we have over 350 customers incl. Amadeus, Deutsche Börse,

Deutsche Telekom, Juniper Networks, Pitney Bowes, Time Warner, Virgin Mobile & Vodafone

Page 11: MariaDB Prez by Joffrey Michaie

SkySQL Ab 2012 Confidential

Page 12: MariaDB Prez by Joffrey Michaie

SkySQL Ab 2012 Confidential

SkySQL Group (currently ~70 people worldwide)

10

CEO - Patrik Sallner - Cloud & SaaS specialist (previously Nokia & F-Secure)

CTO - Ivan Zoratti - Ex. Dir. Field Services EMEA @ MySQL AB

EVP Product Man. - Kaj Arnö – Ex VP Engineering & VP Community MySQL AB

VP Support - Dean Ellis – Ex Director WW Support MySQL AB

VP Sales - Michael Carney – Ex. SEUR Sales Manager @ MySQL AB

VP Sales EMEA - Magnus Stenberg – Ex. NEUR Sales Manager @ MySQL AB

Engineering - Monty Program ~20 MySQL developers (including 8 of the 10 key core developers) lead by

Monty Widenius (MySQL founder)

Page 13: MariaDB Prez by Joffrey Michaie

SkySQL Ab 2012 Confidential

SkySQL is the alternative to Oracle

11

Page 14: MariaDB Prez by Joffrey Michaie

SkySQL Ab 2012 Confidential

SkySQL – Partner Eco-system

L3 Support Contracts

NRE Deals

End

Customer

L3 Support

Corrections

Hot Fixes

Monitoring

Consulting

Training

Corrections

POs

Contracts

Support Tickets

PR

SkySQL 24/7 WW SupportSkySQL 24/7 WW Support

L1, L2, L3 Support

Corrections

Hot Fixes

L1, L2, L3 Support

Contracts and NRE Deals

MHAMHA

Included in Advanced subscriptionIncluded in Advanced subscription

Page 15: MariaDB Prez by Joffrey Michaie

SkySQL&AB&©&2013&Confiden4al&

Introduc4on&to&HA&

“High&availability&is&a&system&design&protocol&and&associated&implementa6on&that&ensures&a&certain&degree&of&opera6onal&con6nuity&during&a&given&

measurement&period”&

Page 16: MariaDB Prez by Joffrey Michaie

SkySQL&AB&©&2013&Confiden4al&

Up4me,&Down4me,&9s&

& &90%& & & & & &1&nine&& & & &36.5&days&/&year &&

& &99%& & & & & &2&nines& & & &3.65&days&/&year&

& &99.9%&& & & & &3&nines& & & &8.76&hours&/&year&

& &99.99%& & & & &4&nines& & & &52&minutes&/&year&

& &99.999%& & & & &5&nines& & & &5&minutes&/&year&

& &99.9999%&& & & &6&nines& & & &31&seconds&/&year&

Availability&=&up4me&/&(up4me&+&down4me)&

Availability&=&MTBF&/&(MTBF&+&MTTR)&

Page 17: MariaDB Prez by Joffrey Michaie

SkySQL&AB&©&2013&Confiden4al&

Terminology&

• Synchronous&vs.&Asynchronous&• SharedHDisk&vs.&SharedHNothing&vs.&SharedHMemory&

• Single&Point&Of&Failure&H&SPOF&• Failover&vs.&Switchover&• Split&Brain&• Node&Fencing,&STONITH,&Quorum&

Page 18: MariaDB Prez by Joffrey Michaie

SkySQL&AB&©&2013&Confiden4al&

Designing&for&HA&

• Which&level&of&availability&do&I&need?&

• How&many&nines?&

• Do&I&require&no&loss&of&data?&• Could&I&loose&some&transac4ons?&

• Will&my&users&no4ce&or&care?&

• Do&I&need&automa4c&failover&or&is&manual&switchover&ok?&

• How&do&I&test&this?&• Can&I&provide&a&reasonable&service&when&X&is&down?&• Replace&X&with&each&component&of&the&service&

Page 19: MariaDB Prez by Joffrey Michaie

SkySQL&AB&©&2013&Confiden4al&

MySQL&Replica4on&H&Asynchronous&

• Asynchronous:&3&Phases&1.  Commit&and&write&to&binlog&on&Master&

2.  Ship&changes&to&relay&log&on&slave&3.  Apply&changes&on&slave&&

• Master&H>&slave&rela4onship&

• MonoHthreaded&on&slaves&un4l&MySQL&5.5&

• MySQL&5.6&allows&mul4Hthreaded&

• No&conflict&resolu4on&• MasterHmaster&replica4on&or&circular&

replica4on&need&applica4on&logic&

&

Master&

Slave(s)&

Page 20: MariaDB Prez by Joffrey Michaie

SkySQL&AB&©&2013&Confiden4al&

MySQL&Replica4on&–&SemiHSynchronous&

• Added&as&a&plugin&in&MySQL®&5.5&

• Ensures&that&changes&have&been&shipped&to&at&least&one&slave&(or&4meouts)&&

• A&COMMIT&on&the&master&waits&for&ONE&

Slave&to&acknowledge&the&transac4on&

• Important:&The&Master&does&not&wait&for&the&

Slave&to&execute&the&transac4on,&only&to&write&

it&to&the&relay&log&

• So&the&Slave&SQL&Thread&may&s4ll&lag&behind&

the&Master&and&queries&to&the&Slave&may&s4ll&

return&old&data&

• Poten4ally&adds&latency&to&queries&

Master&

Slave(s)&

Page 21: MariaDB Prez by Joffrey Michaie

SkySQL&AB&©&2013&Confiden4al&

MySQL®&5.6&–&New&Replica4on&Features&

• Global&Transac4on&ID&(GTID)&• Makes&it&easy&to&automate&failover&and&slave&promo4on&

• Replica4on&failover&and&admin&u4li4es&

• Mul4Hthreaded&slaves&

• Replica4on&event&checksums&

• TimeHdelayed&replica4on&

Page 22: MariaDB Prez by Joffrey Michaie

SkySQL&AB&©&2013&Confiden4al&

MariaDB&10.0&–&New&Replica4on&Features&

• Mul4Hsource&replica4on&

• Global&Transac4on&ID&(GTID)&• Different&implementa4on&from&MySQL&5.6&

• GTID&per&domain&instead&of&server&

Page 23: MariaDB Prez by Joffrey Michaie

SkySQL&AB&©&2013&Confiden4al&

MHA&

Master&High&Availability&Manager&

• Automates&master&failover&and&slave&

promo4on&

• Monitors&the&master&or&can&integrate&with&

Pacemaker/Heartbeat&

• Failover&is&an&online&opera4on&• Also&allows&manual&switchover&

• Short&down4me:&open&a&few&seconds&

• MySQLHReplica4on&consistency&

• No&performance&penalty&

• Drop&in&solu4on&on&exis4ng&deployment&

&

Page 24: MariaDB Prez by Joffrey Michaie

SkySQL&AB&©&2013&Confiden4al&

MHA&

Failover&Process&

1.  Aiempt&to&contact&MySQL&master&server&

by&SSH&

2.  If&master&server&is&alive&access&the&binary&

log&and&recover&events&

3.  Find&the&slave&with&the&most&advanced&

relay&log&

4.  Sync&all&slaves&to&the&latest&available&binlog&event&

5.  STONITH&master&if&necessary&

6.  Promote&slave&to&master&

Page 25: MariaDB Prez by Joffrey Michaie

SkySQL&AB&©&2013&Confiden4al&

Galera&/&MariaDB&Galera&Cluster&

• Provides&“virtually”&synchronous&replica4on&

• Works&with&InnoDB&

• No&slave&lag&• Transac4ons&are&validated&by&slaves&upon&commit&

• Cer4fica4on&and&quorum&

• Transac4ons&may&be&rolled&back&

at&this&stage&

• Mul4Hmaster&or&masterHslave&possible&

Galera Replication

wsrep

MySQL

wsrep

MySQL

wsrep

MySQL

Clients

Page 26: MariaDB Prez by Joffrey Michaie

SkySQL&AB&©&2013&Confiden4al&

Galera&–&Transac4on&Process&

• Transac4on&is&processed&locally&up&to&commit&4me&

Galera Replication

wsrep

MySQL

wsrep

MySQL

wsrep

MySQL

trx

Client

Page 27: MariaDB Prez by Joffrey Michaie

SkySQL&AB&©&2013&Confiden4al&

Galera&–&Transac4on&Process&

• Transac4on&is&processed&locally&up&to&commit&4me&

• Transac4on&is&replicated&to&whole&cluster&

Galera Replication

wsrep

MySQL

wsrep

MySQL

wsrep

MySQL

Client

trx

wsws

Page 28: MariaDB Prez by Joffrey Michaie

SkySQL&AB&©&2013&Confiden4al&

Galera&–&Transac4on&Process&

• Transac4on&is&processed&locally&up&to&commit&4me&

• Transac4on&is&replicated&to&whole&cluster&

• Client&gets&OK&status&

Galera Replication

wsrep

MySQL

wsrep

MySQL

wsrep

MySQL

trx

Client

Page 29: MariaDB Prez by Joffrey Michaie

SkySQL&AB&©&2013&Confiden4al&

Galera&–&Transac4on&Process&

• Transac4on&is&processed&locally&up&to&commit&4me&

• Transac4on&is&replicated&to&whole&cluster&

• Client&gets&OK&status&• Transac4on&is&applied&in&slaves&

Galera Replication

wsrep

MySQL

wsrep

MySQL

wsrep

MySQL

Client

trx trx

Page 30: MariaDB Prez by Joffrey Michaie

SkySQL&AB&©&2013&Confiden4al&

SharedHdisk&Solu4on&

• Ac4ve&–&Passive&replica4on&• Failover&requires&MySQL&crash&recovery&(and&

open&file&system&crash&recovery)&

• Combined&with&Pacemaker/Heartbeat&for&

automa4c&failover&

• Virtual&IP&most&open&used&to&fail&over&

• In&theory&the&SAN&is&a&SPOF&

Page 31: MariaDB Prez by Joffrey Michaie

SkySQL&AB&©&2013&Confiden4al&

DRBD®&&

• Synchronous&replica4on&(three&modes)&

• Ac4ve&–&Passive&replica4on&• Failover&requires&MySQL&crash&recovery&(and&

open&file&system&crash&recovery)&

• Combined&with&Pacemaker/Heartbeat&for&

automa4c&failover&

• Virtual&IP&most&open&used&to&fail&over&

• STONITH&or&other&fencing&mechanism&

needed&to&avoid&splitHbrain&scenarios&

• Available&on&Linux&

Page 32: MariaDB Prez by Joffrey Michaie

SkySQL&AB&©&2013&Confiden4al&

MySQL&Cluster&H&Features&

• Synchronous&replica4on&between&nodes&

• Through&TwoHPhase&Commit&Protocol&

• ACID&transac4ons&• Row&level&locking&• Shared&nothing&architecture&• No&single&point&of&failure&

• Automa4c&failover&

• InHmemory&storage&

• Some&data&can&be&stored&on&disk&

• Checkpoin4ng&to&disk&for&durability&• Two&types&of&indexes&• Ordered&THtrees&• Unique&hash&indexes&

• Online&opera4ons&• Add&node&groups&• Sopware&upgrade&• Some&table&altera4ons&

Page 33: MariaDB Prez by Joffrey Michaie

SkySQL&AB&©&2013&Confiden4al&

MySQL&Cluster&H&Architecture&

Page 34: MariaDB Prez by Joffrey Michaie

SkySQL&AB&©&2013&Confiden4al&

Where&does&MySQL&Cluster&fit?&

• High&demands&on&availability&(5&nines)&

• You&need&write&scalability&• You&have&3&or&more&”machines”&available&

• Where&the&queries&and&data&model&are&simple&

• When&the&data&fits&in&memory&

• When&you&have&skilled&people&

Page 35: MariaDB Prez by Joffrey Michaie

SkySQL&AB&©&2013&Confiden4al&

Geographical&Replica4on&

• For&geographical&(mul4Hsite)&redundacy&

synchronous&solu4ons&are&open&not&

desirable&

• Standard&Replica4on&can&be&combined&

with&any&synchronous&solu4on&locally&

• MySQL&Cluster&

• DRBD&/&shared&disk&• Galera&

Page 36: MariaDB Prez by Joffrey Michaie

SkySQL&AB&©&2013&Confiden4al&

Thank&you!&

MySQL is a registered trademark of Oracle and/or its affiliates. MariaDB is a registered trademark of Monty Program Ab. SkySQL and the SkySQL logo are trademarks of SkySQL Inc. or SkySQL Ab. SkySQL is not affiliated with MySQL. All other company and product names may be trademarks or service marks of their respective owners.

SkySQL&Ab&

www.skysql.com&

www.facebook.com/skysql&

www.linkedin.com/company/skysql&

Joffrey&Michaie&

[email protected]&

Page 37: MariaDB Prez by Joffrey Michaie

What is MariaDB?

• A branch of MySQL – MariaDB is a backward compatible, drop-in replacement

for the MySQL Database Server

• Open Source – The source code for MariaDB is publicly available from

Launchpad – All code in MariaDB is open source

• No closed source modules

– Open bugs database

10.09.2012

SkySQL Ab 2011 Confidential 2

Page 38: MariaDB Prez by Joffrey Michaie

A Brief History

• First version of what would become MySQL created by Monty Widenius 1983

• MySQL AB founded in 1995 • MySQL AB acquired by Sun Microsystems

early 2008 for $1bn • Monty Program founded in late 2008 • Oracle acquire Sun April 2009 • MariaDB 5.1 released in February 2010

10.09.2012

SkySQL Ab 2011 Confidential 3

Page 39: MariaDB Prez by Joffrey Michaie

Who’s Behind MariaDB

• Monty Program – ~30 core MySQL developers – Located around the world

• Community

10.09.2012

SkySQL Ab 2011 Confidential 4

Page 40: MariaDB Prez by Joffrey Michaie

Community and MariaDB

• The goal of MariaDB is to provide a community developed, stable and free database

• MariaDB takes community contributions • Many MariaDB Captains outside of Monty

Program (64% MP, 36% outside) • Open development model

10.09.2012

SkySQL Ab 2011 Confidential 5

Page 41: MariaDB Prez by Joffrey Michaie

MariaDB 5.1 – Feb 2010

• Storage Engines – PBXT* – XtraDB – FederatedX – Aria

• Bugfixes • Removal of Mutexes

10.09.2012

SkySQL Ab 2011 Confidential 6

MariaDB 5.1 = MySQL 5.1 + the following:

Page 42: MariaDB Prez by Joffrey Michaie

MariaDB 5.1

• Microsecond support – Slow query log, SHOW PROCESSLIST etc

• Thread pool* – Pool of threads instead of one thread /connection

10.09.2012

SkySQL Ab 2011 Confidential 7

MariaDB [(none)]> select id, time, time_ms, command, state -> from information_schema.processlist, (select sleep(2)) t; +----+------+----------+---------+-----------+ | id | time | time_ms | command | state | +----+------+----------+---------+-----------+ | 37 | 2 | 2000.493 | Query | executing | +----+------+----------+---------+-----------+ 1 row in set (2.00 sec)

Page 43: MariaDB Prez by Joffrey Michaie

Table Elimination

10.09.2012

SkySQL Ab 2011 Confidential 8

create view actors as select ACNAM_Name, ACDOB_birthdate, ACRAT_rating from ac_anchor left join ac_name on ac_anchor.AC_ID=ac_name.AC_ID left join ac_birthdate on ac_anchor.AC_ID=ac_birthdate.AC_ID left join ac_rating on (ac_anchor.AC_ID=ac_rating.AC_ID and ac_rating.ACRAT_fromdate = (select max(sub.ACRAT_fromdate) from ac_rating sub where sub.AC_ID = ac_rating.AC_ID))

select ACRAT_rating from actors where ACNAM_name='Gary Oldman'

Actor

Name Birth date

Rating

1 n

Page 44: MariaDB Prez by Joffrey Michaie

Table Elimination

10.09.2012

SkySQL Ab 2011 Confidential 9

create view actors as select ACNAM_Name, ACDOB_birthdate, ACRAT_rating from ac_anchor left join ac_name on ac_anchor.AC_ID=ac_name.AC_ID left join ac_birthdate on ac_anchor.AC_ID=ac_birthdate.AC_ID left join ac_rating on (ac_anchor.AC_ID=ac_rating.AC_ID and ac_rating.ACRAT_fromdate = (select max(sub.ACRAT_fromdate) from ac_rating sub where sub.AC_ID = ac_rating.AC_ID))

select ACRAT_rating from actors where ACNAM_name='Gary Oldman'

Actor

Name Birth date

Rating

1 n

x

Page 45: MariaDB Prez by Joffrey Michaie

MariaDB 5.2 – Nov 2010

• Pluggable authentication – Authentication handled by plugins – PAM plugin included

• User statistics – CLIENT_STATISTICS – USER_STATISTICS – INDEX_STATISTICS – TABLE_STATISTICS

10.09.2012

SkySQL Ab 2011 Confidential 10

MariaDB 5.2 = MariaDB 5.1 + the following:

Page 46: MariaDB Prez by Joffrey Michaie

MariaDB 5.2

• Virtual Columns – PERSISTENT or VIRTUAL

• Sphinx Storage Engine – Allows access to Sphinx through MySQL

• Segmented MyISAM key cache – Key cache divided into different segments – Allows for better key cache concurrency – Between 1 and 64 segments

10.09.2012

SkySQL Ab 2011 Confidential 11

Page 47: MariaDB Prez by Joffrey Michaie

Segmented Key Cache

10.09.2012

SkySQL Ab 2011 Confidential 12

Page 48: MariaDB Prez by Joffrey Michaie

MariaDB 5.3 – Jan 2012

• Handler socket – Direct access to InnoDB/XtraDB storage layer – No SQL statements

• Simple CRUD operations on tables

– Can be match faster for large batch operations

• Dynamic columns – Allows you to create columns with dynamic

content – Basically a blob with handling functions

10.09.2012

SkySQL Ab 2011 Confidential 13

MariaDB 5.3 = MariaDB 5.2 + the following:

Page 49: MariaDB Prez by Joffrey Michaie

Dynamic Columns

10.09.2012

SkySQL Ab 2011 Confidential 14

MariaDB [test]> create table t1 (id int auto_increment primary key, -> name varchar(40), -> type enum ("shirt", "phone", "computer"), -> price decimal(10,2), -> dynstr mediumblob); Query OK, 0 rows affected (0.11 sec) MariaDB [test]> insert into t1 (name, type, price, dynstr) values -> ("Funny shirt", "shirt", 10.0, COLUMN_CREATE(1, "blue", 10, "XL")), -> ("nokia", "phone", 649, COLUMN_CREATE(1, "black", 2, "touchscreen")), -> ("htc Desire hd", "phone", 579, COLUMN_CREATE(1, "black", 3, "Android")), -> ("BM/Lenovo Thinkpad X60s", "computer", 419, COLUMN_CREATE(1, "black", 3, "Linux")); Query OK, 4 rows affected (0.04 sec) Records: 4 Duplicates: 0 Warnings: 0 MariaDB [test]> select id, name, type, price, length(dynstr) as len, column_list(dynstr) as list from t1; +----+-------------------------+----------+--------+------+------+ | id | name | type | price | len | list | +----+-------------------------+----------+--------+------+------+ | 1 | Funny shirt | shirt | 10.00 | 17 | 1,10 | | 2 | nokia | phone | 649.00 | 27 | 1,2 | | 3 | htc Desire hd | phone | 579.00 | 23 | 1,3 | | 4 | BM/Lenovo Thinkpad X60s | computer | 419.00 | 21 | 1,3 | +----+-------------------------+----------+--------+------+------+ 4 rows in set (0.03 sec)

Page 50: MariaDB Prez by Joffrey Michaie

Dynamic Columns

10.09.2012

SkySQL Ab 2011 Confidential 15

MariaDB [test]> select name from t1 where COLUMN_GET(dynstr, 1 as char(10)) = "black"; +-------------------------+ | name | +-------------------------+ | nokia | | htc Desire hd | | BM/Lenovo Thinkpad X60s | +-------------------------+ 3 rows in set (0.01 sec) MariaDB [test]> MariaDB [test]> select name, COLUMN_GET(dynstr, 1 as char(10)) from t1 where COLUMN_EXISTS(dynstr, 1); +-------------------------+-----------------------------------+ | name | COLUMN_GET(dynstr, 1 as char(10)) | +-------------------------+-----------------------------------+ | Funny shirt | blue | | nokia | black | | htc Desire hd | black | | BM/Lenovo Thinkpad X60s | black | +-------------------------+-----------------------------------+ 4 rows in set (0.00 sec)

Page 51: MariaDB Prez by Joffrey Michaie

MariaDB 5.3

• Replication enhancements – Original statement logged with RBR events – Checksum for binlog events – RBR fixed for tables with no PK – Consistent snapshot between storage engines

• User feedback plugin • Extended OpenGIS SFS

10.09.2012

SkySQL Ab 2011 Confidential 16

MariaDB 5.3 = MariaDB 5.2 + the following:

Page 52: MariaDB Prez by Joffrey Michaie

Progress Report

• Progress report for ALTER TABLE

10.09.2012

SkySQL Ab 2011 Confidential 17

MariaDB [employees]> alter table salaries engine = maria; Stage: 1 of 2 'copy to tmp table' 17.55% of stage done

MariaDB [employees]> select id, user, db, command, state, -> time_ms, progress from information_schema.processlist; +---------+-------------------+-----------+----------+ | command | state | time_ms | progress | +---------+-------------------+-----------+----------+ | Query | copy to tmp table | 23407.131 | 17.551 | +---------+-------------------+-----------+----------+ 1 row in set (0.47 sec)

Page 53: MariaDB Prez by Joffrey Michaie

Optimizer Enhancements

1. Sub-query optimizations 2. Join additions 3. Optimizations for derived tables and views 4. Disk access optimization 5. Optimizer control

10.09.2012

SkySQL Ab 2011 Confidential 18

Page 54: MariaDB Prez by Joffrey Michaie

Optimizer Enhancements

1. Sub-query optimizations – Semi-join subquery optimization – Materialization for non-correlated IN-queries – Sub-query cache

10.09.2012

SkySQL Ab 2011 Confidential 19

Subqueries are finally usable in practice!

It is no longer necessary to rewrite subqueries manually into joins or into separate queries

Page 55: MariaDB Prez by Joffrey Michaie

Optimizer Enhancements

2. Join additions – Block Nested-Loop-Joins for outer joins – Block Hash-Joins – Batch-Key-Access

10.09.2012

SkySQL Ab 2011 Confidential 20

Before MariaDB 5.3 only Nested-Loop-Joins available

Page 56: MariaDB Prez by Joffrey Michaie

Optimizer Enhancements

3. Optimizations for derived tables and views – Mergeable derived tables processed like VIEWs – Optimizer can create indexes over materialized

derived tables

4. Disk access optimization – Index Condition Pushdown – Multi-Range-Read optimization (MRR)

5. Optimizer control – @@optimizer_switch for all new options

10.09.2012

SkySQL Ab 2011 Confidential 21

Page 57: MariaDB Prez by Joffrey Michaie

Group Commit

10.09.2012

SkySQL Ab 2011 Confidential 22

XtraDB / InnoDB

Page 58: MariaDB Prez by Joffrey Michaie

Group Commit

10.09.2012

SkySQL Ab 2011 Confidential 23

COMMIT

Binary log

XtraDB / InnoDB

3 fsyncs / transaction

2

Page 59: MariaDB Prez by Joffrey Michaie

Group Commit

10.09.2012

SkySQL Ab 2011 Confidential 24

Page 60: MariaDB Prez by Joffrey Michaie

MariaDB 5.5 – April 2012

• New optimized thread pool implementation – Pool size dynamic – Different implementation on Linux and Windows

• Fine grained tuning possible on Linux

• @@skip_replication • LIMIT ROWS EXAMINED

10.09.2012

SkySQL Ab 2011 Confidential 25

MariaDB 5.5 = MySQL 5.5 + MariaDB 5.3 features + the following:

MariaDB [employees]> SELECT * from t1, t2 -> LIMIT 10 ROWS EXAMINED 1000;

Page 61: MariaDB Prez by Joffrey Michaie

Next MariaDB

• Global transaction Id (from MySQL 5.6) • Multi-source replication • New improved InnoDB (from MySQL 5.6) • Cassandra storage engine

10.09.2012

SkySQL Ab 2011 Confidential 26

All highly speculative

Vote for features on http://www.skysql.com/content/ new-server-functionality-have-your-say

Page 62: MariaDB Prez by Joffrey Michaie

A branch or a fork?

• MySQL 5.1 CE = MySQL EE <-> MariaDB 5.1 • MariaDB 5.2 and 5.3 <-> MySQL 5.1 CE & EE • MySQL 5.5 CE != MySQL 5.5 EE

– Commercial extensions – MariaDB 5.5 implements same features

• Merged nonetheless

• 5.6 ? 10.09.2012

SkySQL Ab 2011 Confidential 27

Page 63: MariaDB Prez by Joffrey Michaie

Getting MariaDB

• http://mariadb.org is the main place • Available via OpenSUSE build services • Also available via

– Gentoo – FreeBSD – Homebrew – Slackware – ArchLinux

• yum and apt repos available from MP 10.09.2012

SkySQL Ab 2011 Confidential 28

Page 64: MariaDB Prez by Joffrey Michaie

Support & Services

• Monty Program does NRE • Enterprise level support available from SkySQL

– Monty Program providing L3

• Training, consulting etc available from SkySQL • Others providing MariaDB services:

http://mariadb.org/service-providers/

10.09.2012

SkySQL Ab 2011 Confidential 29

Page 65: MariaDB Prez by Joffrey Michaie

More Information

• Downloads: http://mariadb.org/ • Mailing lists on launchpad • #maria on freenode • Knowledgbase: http://kb.askmonty.org/ • Support:

http://www.skysql.com/products/skysql-enterprise

• Training & consulting: http://www.skysql.com/services/mysql/overview

• MariaDB book: MariaDB Crash Course by Ben Forta 10.09.2012

SkySQL Ab 2011 Confidential 30

Page 66: MariaDB Prez by Joffrey Michaie

Who’s using it?

Page 67: MariaDB Prez by Joffrey Michaie

Other MariaDB projects

• MariaDB Galera Cluster

• Connectors/Drivers– MariaDB C Client Library (LGPL)

– MariaDB JDBC driver (LGPL)

• MariaDB Audit Plugin

Page 68: MariaDB Prez by Joffrey Michaie

10.0 new features

• Replication enhancements

• Storage engines

• Optimizer enhancements

• Administration improvements

• More features, optimizations, syntax

extensions, …

• Merges from 5.6

Page 69: MariaDB Prez by Joffrey Michaie

Replication features

• Global Transaction ID

• Crash-safe slave

• Multi-Source Replication

• Parallel Slave

Page 70: MariaDB Prez by Joffrey Michaie

Global Transaction ID

• Monitor replication progress in complex topologies

• Simple fail-over and slave promotion

A

B C

D E F

Page 71: MariaDB Prez by Joffrey Michaie

Global Transaction ID

• Monitor replication progress in complex topologies

• Simple fail-over and slave promotion

A

B C

D E F

Page 72: MariaDB Prez by Joffrey Michaie

Global Transaction ID

• Monitor replication progress in complex topologies

• Simple fail-over and slave promotion

A

B C

D E F

Page 73: MariaDB Prez by Joffrey Michaie

Global Transaction ID

• Monitor replication progress in complex topologies

• Simple fail-over and slave promotion

A

B C

D E F

Page 74: MariaDB Prez by Joffrey Michaie

Global Transaction ID

• Monitor replication progress in complex topologies

• Simple fail-over and slave promotion

A

B C

D E F

Page 75: MariaDB Prez by Joffrey Michaie

Global Transaction ID – Commands

Setting up a new slave:

Switch a slave to use GTID:

Change master:

Page 76: MariaDB Prez by Joffrey Michaie

Crash-safe slave

• Global Transaction ID enabled

• GTID position stored in InnoDB table

Conclusio

n: In M

ariaDB 10.0

use GTID

for y

our replic

ation

setups

Page 77: MariaDB Prez by Joffrey Michaie

Multi-Source Replication

• Replicating from many masters to one

slave:

• Compatible with GTID

• Contributed by Lixun Peng, Taobao

Page 78: MariaDB Prez by Joffrey Michaie

Multi-source - Commands

Setting up multi-source replication:

All commands take a slave name:

Or use a default connection:

Page 79: MariaDB Prez by Joffrey Michaie

Parallel Slave

• Automatically detects independent

transactions

• Parallel even within the same table

• Adapts to master load

• Preserves commit order

Page 80: MariaDB Prez by Joffrey Michaie

New storage engines

• CassandraSE

• Connect

• Sequence

• Spider

• TokuDB

Page 81: MariaDB Prez by Joffrey Michaie

Cassandra Storage Engine

• Access Cassandra from MariaDB with SQL

• Flexible schema, using “dynamic columns”

• Joins with non-Cassandra tables

• Transparent data mapping

• Many MariaDB nodes — one Cassandra ring

Page 82: MariaDB Prez by Joffrey Michaie

Connect Storage Engine

• CSV and Federated on steroids

• CSV, BIN, FMT, INI, XML, DBF

• column store

• ODBC, MySQL

• TBL (similar to MERGE)

• OEM (external library)

• Flexible table structure specification

• Developed by Olivier Bertrand

Page 83: MariaDB Prez by Joffrey Michaie

Sequence Storage Engine

• Completely virtual read-only tables

• Magically created on demand, magically

disappearing, no .frm files:

• Supports indexes, transactions, XA

Page 84: MariaDB Prez by Joffrey Michaie

Sequence: Example

• Prime numbers below 50:

Page 85: MariaDB Prez by Joffrey Michaie

Spider Storage Engine

• Federated-like storage engine

• Sharding, partitioning

• Transactions, XA

• High-Availability Cluster

• MySQL, HandlerSocket, Oracle

• Developed by Kentoku Shiba

Page 86: MariaDB Prez by Joffrey Michaie

Spider Storage Engine

Page 87: MariaDB Prez by Joffrey Michaie

TokuDB Storage Engine

• When you have billions of rows and terabytes

of data

• 10x to 20x faster INSERTs

• Compression — space saving up to 90%

• Online ALTER TABLE

• Both HDD and Flash

• Developed by Tokutek

Page 88: MariaDB Prez by Joffrey Michaie

Unique Use Case Example

• CONNECT Storage Engine (XML, ODBC,

etc.)

• CONNECT to Oracle (via ODBC), join results

from Cassandra and store the result in

InnoDB– Yes, collaboration between Oracle, Cassandra

and MariaDB is possible today

• Remember to turn on engine condition

pushdown

Page 89: MariaDB Prez by Joffrey Michaie

Optimizer

• Engine independent statistics

• Histogram-based statistics

• Subquery optimizations

Page 90: MariaDB Prez by Joffrey Michaie

Optimizer: Statistics

• Engine independent– stored in mysql database

– stable, uniform, controlled

– for indexes and columns

– used instead of or together with engine

statistics

• Histograms

Page 91: MariaDB Prez by Joffrey Michaie

Subqueries

• EXISTS-to-IN

Can be rewritten into

• Semi-join EXISTS

Page 92: MariaDB Prez by Joffrey Michaie

Administration

• SHOW EXPLAIN

• SHOW PLUGIN SONAME

• SHUTDOWN

• KILL QUERY ID

• Per-connection memory accounting

• Roles

Page 93: MariaDB Prez by Joffrey Michaie

SHOW EXPLAIN

• Shows EXPLAIN for a query running in

another thread

Page 94: MariaDB Prez by Joffrey Michaie

SHOW PLUGIN SONAME

• Information about not installed plugins

• LIKE 'soname', WHERE …

• INFORMATION_SCHEMA.ALL_PLUGINS

Page 95: MariaDB Prez by Joffrey Michaie

SHUTDOWN

• SQL statement to shut the server down

• Same as mysqladmin shutdown

• Requires SHUTDOWN privilege

Page 96: MariaDB Prez by Joffrey Michaie

KILL QUERY ID

• Abort a running query

• Uses query id, not thread id

• Avoid nasty race conditions

Page 97: MariaDB Prez by Joffrey Michaie

Per-connection memory usage

• Monitor memory usage per connection:

• Contributed by Lixun Peng, Taobao

Page 98: MariaDB Prez by Joffrey Michaie

Roles

• SQL standard implementation of roles– CREATE ROLE, DROP ROLE

– GRANT role TO user, GRANT role TO role

– SET ROLE

– DEFINER=role

– CURRENT_ROLE

– INFORMATION_SCHEMA tables

• GSoC–2013, Vicentiu Ciorbaru

Page 99: MariaDB Prez by Joffrey Michaie

Misc. enhancements

• DELETE … RETURNING

• ALTER … IF [NOT] EXISTS

• Named dynamic columns

• Table discovery, assisted table

discovery

• Multiple GET_LOCK– Contributed by Konstantin Osipov

• New regular expression library– GSoC–2013, Sudheera Palihakkara

Page 100: MariaDB Prez by Joffrey Michaie

Merged from 5.6

• InnoDB

• Performance_schema

• Online ALTER TABLE

• ORDER BY … LIMIT optimization

• Privileges on temporary tables

• Character set extensions

• GET DIAGNOSTICS

• TO_BASE64(), FROM_BASE64()

Page 101: MariaDB Prez by Joffrey Michaie

MARIADB AUDIT PLUGIN

Page 102: MariaDB Prez by Joffrey Michaie

MariaDB Audit Plugin

• Track user access to

server and data

• Track all type of access

to data in tables

• Download the MariaDB

Audit Plugin Library

from http://

www.skysql.com/downlo

ads/mariadb-audit-plu

gin-beta

Suddenly it’s straight forward to comply with laws and industry standards

Logged information:• Timestamp• Server-Host• User Client-Host• Connection-ID• Query-ID• Operation• Database Table• Error-Code• ...and more…

Page 103: MariaDB Prez by Joffrey Michaie

Configuration

Page 104: MariaDB Prez by Joffrey Michaie

Logging

• Connect events

– Logs connects, disconnects, and failed connects + error code

• Query events

– Executed queries will be logged with user information

– Queries are logged exactly as executed and might reveal sensitive

data in the logs

– Also queries that cannot be executed will be logged, e.g. user

doesn’t have the rights or a syntax fault

• Table events

– Only available in MariaDB 5.5.31 or newer (not available in

MySQL)

– Views, Stored Procedures, Functions and Triggers can access

table data without showing it in the executed query

Page 105: MariaDB Prez by Joffrey Michaie

Logging

Page 106: MariaDB Prez by Joffrey Michaie

Contact us

• MariaDB.org and MariaDB.com

• Knowledge base: http://mariadb.com/kb

• Mailing lists:

[email protected]

[email protected]

• IRC: #maria channel on FreeNode

• Jira: http://mariadb.org/jira

• Rasmus: [email protected]

Page 107: MariaDB Prez by Joffrey Michaie

Thank you!