froscon2011: how i learned to use sql and then learned not to use it
DESCRIPTION
Keynote for the Open DB Camp track (developer room) at Froscon2011. The point is to compare the history of MySQL with the evolution of new NoSQL systems.TRANSCRIPT
![Page 1: Froscon2011: How i learned to use sql and then learned not to use it](https://reader036.vdocuments.us/reader036/viewer/2022081403/5558c357d8b42a995d8b456b/html5/thumbnails/1.jpg)
2011-07-26 OSCON 2011 1
How I learned to use SQLand then learned not to use it
Henrik Ingo
Froscon2011-08-20
![Page 2: Froscon2011: How i learned to use sql and then learned not to use it](https://reader036.vdocuments.us/reader036/viewer/2022081403/5558c357d8b42a995d8b456b/html5/thumbnails/2.jpg)
2011-07-26 OSCON 2011 2
Henrik Ingo
open source technology and strategy specialist
active in MySQL, Drupal communities
current: Senior Performance Architect at Nokia Ovi
worked in mobile and LAMP with business management, sales, R&D
author of "Open Life: The Philosophy of Open Source"
![Page 3: Froscon2011: How i learned to use sql and then learned not to use it](https://reader036.vdocuments.us/reader036/viewer/2022081403/5558c357d8b42a995d8b456b/html5/thumbnails/3.jpg)
2011-07-26 OSCON 2011 3
1994
![Page 4: Froscon2011: How i learned to use sql and then learned not to use it](https://reader036.vdocuments.us/reader036/viewer/2022081403/5558c357d8b42a995d8b456b/html5/thumbnails/4.jpg)
2011-07-26 OSCON 2011 4
1994
USSR fell 4 years ago
Finland yet to win first gold medal in hockey
Windows 3.0 replacing MS-DOS
Windows 95 did not yet exist
I learn how to write .bat scripts from my dad
Word 6.0 replacing WordPerfect 5.1
Excel 5.0 replacing Lotus
I had not yet used Internet
![Page 5: Froscon2011: How i learned to use sql and then learned not to use it](https://reader036.vdocuments.us/reader036/viewer/2022081403/5558c357d8b42a995d8b456b/html5/thumbnails/5.jpg)
2011-07-26 OSCON 2011 5
1994
C++ & Visual Basic
LANPrinter Database
Client - Server architecture
SQLStandard Interpreted
Flexible and expressive command line env
"SQL for secretaries"Good for English speakers
2000: Bad for IDE w IntelliSense
Stored procedures rule
DBA is king of business logic
![Page 6: Froscon2011: How i learned to use sql and then learned not to use it](https://reader036.vdocuments.us/reader036/viewer/2022081403/5558c357d8b42a995d8b456b/html5/thumbnails/6.jpg)
2011-07-26 OSCON 2011 6
1997
![Page 7: Froscon2011: How i learned to use sql and then learned not to use it](https://reader036.vdocuments.us/reader036/viewer/2022081403/5558c357d8b42a995d8b456b/html5/thumbnails/7.jpg)
2011-07-26 OSCON 2011 7
1997
![Page 8: Froscon2011: How i learned to use sql and then learned not to use it](https://reader036.vdocuments.us/reader036/viewer/2022081403/5558c357d8b42a995d8b456b/html5/thumbnails/8.jpg)
2011-07-26 OSCON 2011 8
Learning SQL
NoSQL advocates say:
SQL is hard to learn
To really scale, you must de-normalize
Most people don't get normalization right
Impedance mismatch between oo and sql
I just need a simple key-value store
Henrik says:
MS Access easy to learn
Darn, I always got normalization
I know, teaching n:n relations was always fun
INSERT INTO t ... serialize($obj)
SELECT value FROM tWHERE key=?;
![Page 9: Froscon2011: How i learned to use sql and then learned not to use it](https://reader036.vdocuments.us/reader036/viewer/2022081403/5558c357d8b42a995d8b456b/html5/thumbnails/9.jpg)
2011-07-26 OSCON 2011 9
1997
WorldWideWebFree Software
1998: Open Source
LAMP stackWhere P = Perl, runs +90% of web "apps"
Oracle doesn't support Linux
PostgreSQL doesn't support Windows
MiniSQL has poor performance
Wish I had a simple data store for my web apps that was really fast and scalable and Free Software...
![Page 10: Froscon2011: How i learned to use sql and then learned not to use it](https://reader036.vdocuments.us/reader036/viewer/2022081403/5558c357d8b42a995d8b456b/html5/thumbnails/10.jpg)
2011-07-26 OSCON 2011 10
1997
![Page 11: Froscon2011: How i learned to use sql and then learned not to use it](https://reader036.vdocuments.us/reader036/viewer/2022081403/5558c357d8b42a995d8b456b/html5/thumbnails/11.jpg)
2011-07-26 OSCON 2011 11
1997
ERROR: Could not connect to MySQL: ERROR 1040: Too many connections
"Slashdotted"Web jargon ca 1997."Your site is down becauseit isn't web scale."
![Page 12: Froscon2011: How i learned to use sql and then learned not to use it](https://reader036.vdocuments.us/reader036/viewer/2022081403/5558c357d8b42a995d8b456b/html5/thumbnails/12.jpg)
2011-07-26 OSCON 2011 12
History of MySQL
1985... ISAM key-value store - Trivia: Access is also ISAM db
SQL support added (HTML)
Data Warehousing: bulk loads, reporting. - Flexible attitude to errors during insert
ODBC (Excel) - BI tool #1
1995.. Released under FOSS-like license
![Page 13: Froscon2011: How i learned to use sql and then learned not to use it](https://reader036.vdocuments.us/reader036/viewer/2022081403/5558c357d8b42a995d8b456b/html5/thumbnails/13.jpg)
2011-07-26 OSCON 2011 13
1997
"Read-mostly" - No MVCC - Not transactional
Not crash safe
Scales well for 2-3 CPUs - This will work well for the next 10 years
Not ANSI SQL
![Page 14: Froscon2011: How i learned to use sql and then learned not to use it](https://reader036.vdocuments.us/reader036/viewer/2022081403/5558c357d8b42a995d8b456b/html5/thumbnails/14.jpg)
2011-07-26 OSCON 2011 14
1997
This site is currently offline for nightly backup. Please come back after 15 minutes.
(No online backups)
![Page 15: Froscon2011: How i learned to use sql and then learned not to use it](https://reader036.vdocuments.us/reader036/viewer/2022081403/5558c357d8b42a995d8b456b/html5/thumbnails/15.jpg)
2011-07-26 OSCON 2011 15
1997-2007
InnoDB and BDB - Transactions!
SAP partnership - 4 CPU servers - Support SQL-92 "enterprise" features - Oracle acquires InnoDB and BDB
Replication - Statement based - Asynchronous
Online backup tool
![Page 16: Froscon2011: How i learned to use sql and then learned not to use it](https://reader036.vdocuments.us/reader036/viewer/2022081403/5558c357d8b42a995d8b456b/html5/thumbnails/16.jpg)
2011-07-26 OSCON 2011 16
2007
We want a database that - does not conform to SQL standard - has no transactions - is not crash safe - impossible to do consistent backups
- but scales to more than 4 cores!
Users are not happy!
![Page 17: Froscon2011: How i learned to use sql and then learned not to use it](https://reader036.vdocuments.us/reader036/viewer/2022081403/5558c357d8b42a995d8b456b/html5/thumbnails/17.jpg)
2011-07-26 OSCON 2011 17
You see where I'm going, don't you...
![Page 18: Froscon2011: How i learned to use sql and then learned not to use it](https://reader036.vdocuments.us/reader036/viewer/2022081403/5558c357d8b42a995d8b456b/html5/thumbnails/18.jpg)
2011-07-26 OSCON 2011 18
Sorry, that was a cheap shot...
...I just couldn't resist :-)
![Page 19: Froscon2011: How i learned to use sql and then learned not to use it](https://reader036.vdocuments.us/reader036/viewer/2022081403/5558c357d8b42a995d8b456b/html5/thumbnails/19.jpg)
2011-07-26 OSCON 2011 19
It really started with...
![Page 20: Froscon2011: How i learned to use sql and then learned not to use it](https://reader036.vdocuments.us/reader036/viewer/2022081403/5558c357d8b42a995d8b456b/html5/thumbnails/20.jpg)
2011-07-26 OSCON 2011 20
Actual questions from Memcache users, ca 2007:
How can I dump all records from a memcached instance?
How can I make memcached highly available?
Backups?
![Page 21: Froscon2011: How i learned to use sql and then learned not to use it](https://reader036.vdocuments.us/reader036/viewer/2022081403/5558c357d8b42a995d8b456b/html5/thumbnails/21.jpg)
2011-07-26 OSCON 2011 21
Answers from Memcache developers
It's a cache. Please use ittogether with a real database!
Why would you ever want to do that?
No no no, you're doing it wrong!
![Page 22: Froscon2011: How i learned to use sql and then learned not to use it](https://reader036.vdocuments.us/reader036/viewer/2022081403/5558c357d8b42a995d8b456b/html5/thumbnails/22.jpg)
2011-07-26 OSCON 2011 22
So clearly there was a market demand...
![Page 23: Froscon2011: How i learned to use sql and then learned not to use it](https://reader036.vdocuments.us/reader036/viewer/2022081403/5558c357d8b42a995d8b456b/html5/thumbnails/23.jpg)
2011-07-26 OSCON 2011 23
Things NoSQL guys do really well
No SQL (parsing)
Schemaless = Win! for agile development
HA with quorum consistency: R + W > N
Transparent sharding
Graph databases
N:N relationships, what's the big deal?
Actually makes sense to give up on SQL!
Map Reduce
Bypass ETL, get clean data
Implemented in Java or Python (or Erlang)
![Page 24: Froscon2011: How i learned to use sql and then learned not to use it](https://reader036.vdocuments.us/reader036/viewer/2022081403/5558c357d8b42a995d8b456b/html5/thumbnails/24.jpg)
2011-07-26 OSCON 2011 24
Best of both worlds
NoSQLNo SQL.
Simple key-value store.
...and secondary indexes
Quorum consistency.
Graph databases
Map Reduce against text files
Java and Python.
MySQLHandlerSocketMemcache API, NDB API
BLOBSELECT v FROM ... WHERE k=?
Functional indexesVirtual columns, etc...
Synchronous replicationGalera, NDB
Damn N:N relations!
Map Reduce against RDBMS
C++ can be done rightDrizzle
![Page 25: Froscon2011: How i learned to use sql and then learned not to use it](https://reader036.vdocuments.us/reader036/viewer/2022081403/5558c357d8b42a995d8b456b/html5/thumbnails/25.jpg)
2011-07-26 OSCON 2011 25
Clayton Christensen on Disruptive innovation
![Page 26: Froscon2011: How i learned to use sql and then learned not to use it](https://reader036.vdocuments.us/reader036/viewer/2022081403/5558c357d8b42a995d8b456b/html5/thumbnails/26.jpg)
2011-07-26 OSCON 2011 26
Clayton Christensen on Disruptive innovation
First they ignore you
Then they laugh at you
Then they fight you
Then you win
- Gandhi
![Page 27: Froscon2011: How i learned to use sql and then learned not to use it](https://reader036.vdocuments.us/reader036/viewer/2022081403/5558c357d8b42a995d8b456b/html5/thumbnails/27.jpg)
2011-07-26 OSCON 2011 27
It used to be
![Page 28: Froscon2011: How i learned to use sql and then learned not to use it](https://reader036.vdocuments.us/reader036/viewer/2022081403/5558c357d8b42a995d8b456b/html5/thumbnails/28.jpg)
2011-07-26 OSCON 2011 28
The future is
![Page 29: Froscon2011: How i learned to use sql and then learned not to use it](https://reader036.vdocuments.us/reader036/viewer/2022081403/5558c357d8b42a995d8b456b/html5/thumbnails/29.jpg)
2011-07-26 OSCON 2011 29
The future is
All Open SourceAll Open Source