highly available database clusters with jdbc emmanuel.cecchet@emicnetworks.com
Post on 14-Jan-2016
212 Views
Preview:
TRANSCRIPT
Highly available database clusters with JDBC
emmanuel.cecchet@emicnetworks.com
http://c-jdbc.objectweb.org/ - c-jdbc@objectweb.org ObjectWebCon’05 - 2 - 19/01/2005
Scaling the database tier – Alternative 1 (SMP)
Internet
Web frontend
App. server
Well-known database
vendor here
Database
Well-known hardware +database vendors here
Cons Cost Scalability limit
http://c-jdbc.objectweb.org/ - c-jdbc@objectweb.org ObjectWebCon’05 - 3 - 19/01/2005
Scaling the database tier – Alternative 2 (shared disks)
Internet
Web frontend
App. server Database Disks
Another well-knowndatabase vendor
Consstill expensive hardwareavailability
http://c-jdbc.objectweb.org/ - c-jdbc@objectweb.org ObjectWebCon’05 - 4 - 19/01/2005
Internet
scalabilityhigh availabilitywithout modifying the client applicationdatabase vendor independenton commodity hardware
Database clustering middleware
JDBC
JDBCJDBC
http://c-jdbc.objectweb.org/ - c-jdbc@objectweb.org ObjectWebCon’05 - 5 - 19/01/2005
RAIDb concept
Redundant Array of Inexpensive DatabasesRAIDb controller
gives the view of a single database to the clientbalance the load on the database backends
RAIDb levels offers various tradeoff of performance and fault tolerance
http://c-jdbc.objectweb.org/ - c-jdbc@objectweb.org ObjectWebCon’05 - 6 - 19/01/2005
RAIDb levels
RAIDb-0partitioningno duplication and no fault toleranceat least 2 nodes
table 2 & 3 table ...
RAIDb controller
table n-1table 1 table n
SQL requests
http://c-jdbc.objectweb.org/ - c-jdbc@objectweb.org ObjectWebCon’05 - 7 - 19/01/2005
RAIDb levels
RAIDb-1mirroringperformance bounded by write broadcastat least 2 nodes
Full DB Full DB
RAIDb controller
Full DBFull DB Full DB
SQL requests
http://c-jdbc.objectweb.org/ - c-jdbc@objectweb.org ObjectWebCon’05 - 8 - 19/01/2005
RAIDb levels
RAIDb-2partial replicationat least 2 copies of each table for fault toleranceat least 3 nodes
table x table y
RAIDb controller
table x & yFull DB table z
SQL requests
http://c-jdbc.objectweb.org/ - c-jdbc@objectweb.org ObjectWebCon’05 - 9 - 19/01/2005
C-JDBC overview
Middleware implementing RAIDb100% Java implementationopen source (LGPL)
Two componentsgeneric JDBC driver (C-JDBC driver)C-JDBC Controller
Read-one, Write all approachprovides eager (strong) consistency
Supports heterogeneous databases
http://c-jdbc.objectweb.org/ - c-jdbc@objectweb.org ObjectWebCon’05 - 10 - 19/01/2005
architectural overview
JVM
Applicationserver
C-JDBC JDBC driver
C-JDBCcontroller
JVM
MySQL JDBC driver
MySQLdatabase
http://c-jdbc.objectweb.org/ - c-jdbc@objectweb.org ObjectWebCon’05 - 11 - 19/01/2005
Inside the C-JDBC Controller
XML configuration file
C-JDBC Controller
MySQL
C-JDBC driver
Java client program
(Servlet, EJB, ...)
XML engine
MySQL
Virtual database
Database Backend
Connection Manager
Database Backend
Connection Manager
Request Manager
Request Cache
Scheduler
Load balancer
MySQL JDBC driver
MySQL JDBC driver
Configuration &
administrationAdministration console
RMIRMI
Recovery Log
Authentication Manager
MySQL
Database Backend
Connection Manager
MySQL JDBC driver
Virtual database
Database Backend
Connection Manager
Database Backend
Connection Manager
Request Manager
Request Cache
Scheduler
Load balancer
Recovery Log
Authentication Manager
Oracle
Oracle JDBC driver
Oracle JDBC driver
Oracle
SocketsSockets
JMX
http://c-jdbc.objectweb.org/ - c-jdbc@objectweb.org ObjectWebCon’05 - 12 - 19/01/2005
Scaling existing databases
add open source databases for more fault tolerance and better performances
rules for on-the-fly query rewriting to handle heterogeneity
MySQL MySQL
C-JDBC Controller RAIDb-2
MySQL JDBC driver
MySQLOracle
Java client program
C-JDBC driver
JVM
C-JDBC driver
C-JDBC driver
JVM
JVM
Oracle JDBC driver
EJB Container JOnAS, WebLogic,
JBoss, WebSphere, ...
Servlet container Tomcat, Jetty, ...
http://c-jdbc.objectweb.org/ - c-jdbc@objectweb.org ObjectWebCon’05 - 13 - 19/01/2005
TPC-W benchmark(Amazon.com)
Nearly linear speedups with the shopping mix
0
200
400
600
800
1000
1200
1400
1600
0 1 2 3 4 5 6
Number of nodes
Th
rou
gh
pu
t in
req
ues
ts p
er m
inu
te
Single Database
Full replication
Partial replication
http://c-jdbc.objectweb.org/ - c-jdbc@objectweb.org ObjectWebCon’05 - 14 - 19/01/2005
Horizontal & Vertical scalability
DB 6DB 5
DB native JDBC driver
DB 7
C-JDBC driver
DB 1 DB 2
DB native JDBC driver
DB 3
DB native JDBC driver
DB 4
C-JDBC controller Full replication
C-JDBC controller Full replication
C-JDBC controller Full replication
C-JDBC controller Full replication
C-JDBC driverJVM
Client program
C-JDBC driver
JVM
Client program
C-JDBC driver
JVM
Client program
C-JDBC driver
C-JDBC driver
adapt and evolve according to user needs
advanced caching functionalities
dynamic adding of database backends
fault tolerant transaction log
integrated backup and checkpointing
grid & edge-side servers support
http://c-jdbc.objectweb.org/ - c-jdbc@objectweb.org ObjectWebCon’05 - 15 - 19/01/2005
Administration
JMX Graphical Consolecluster administrationbackup/checkpoint mgtcluster monitoringSQL profilingintegrated SQL console (iSQL)
Administration APIintegration in any existing administration infrastructureJMX notifications for all internal eventsJMX MBeans for monitoring
http://c-jdbc.objectweb.org/ - c-jdbc@objectweb.org ObjectWebCon’05 - 16 - 19/01/2005
J2EE end-to-end reliability
ObjectWeb exclusivityOn-demand availability
no SPOF, transparent failover
On-demand scalability scalability, flexibility
Internet
http://c-jdbc.objectweb.org/ - c-jdbc@objectweb.org ObjectWebCon’05 - 17 - 19/01/2005
C-JDBC today
Web site~200.000 hits/month>28.000 downloadsEU (18 countries) 36%, US 28%, Japan 12%, China
5%, Canada 4%, Australia 4%, India 3%, Brazil 2%, …
Community27 committers both industrial & academicsc-jdbc@objectweb.org: >200 subscribers, 200-300
msgs/monthtranslation in japanese, italian, chinese, turkish, french ,
german
RPM on JPackage.orgProfessional support by Emic Networks
http://c-jdbc.objectweb.org/ - c-jdbc@objectweb.org ObjectWebCon’05 - 18 - 19/01/2005
What’s next?
Tribe (.objectweb.org)replacement for JGroupsuniform total order broadcast optimized for clusters
LeWYS (.objectweb.org)hardware and software monitoringmonitoring repository
Distributed query executionOptimized support for edge-side servers and
interconnected clusters
http://c-jdbc.objectweb.org/ - c-jdbc@objectweb.org ObjectWebCon’05 - 19 - 19/01/2005
Demos
Tuesday 18INRIA booth: 12.00 to 1pmObjectWeb booth: 2pm to 2.30pm
Wednesday 19INRIA booth: 10.30am to 11amObjectWeb booth: 1pm to 1.30pm
http://c-jdbc.objectweb.org/ - c-jdbc@objectweb.org ObjectWebCon’05 - 20 - 19/01/2005
Q&A_________
Thanks to all users and contributors ...
http://c-jdbc.objectweb.org
Bonus slides
http://c-jdbc.objectweb.org/ - c-jdbc@objectweb.org ObjectWebCon’05 - 22 - 19/01/2005
Current limitations
JDBC onlyDistributed joinsUpdatable ResultSetsXA support through XAPool onlytransparent controller failover not supported
when using horizontal scalability with JGroupsnetwork partition/reconciliation not supported
top related