Database High Availability Using SHADOWSystems
Jaemyung Kim, Kenneth Salem, Khuzaima Daudjee,Ashraf Aboulnaga, and Xin Pan
University of Waterloo
SoCC 2015
SHADOW Hot Standby HA for Cloud
How can we exploitshared persistent storage
to build betterhighly available database systems?
2
Overview
1 Standalone and Hot Standby Failure Handling
2 Shared Storage in Cloud Settings
3 SHADOW: Hot Standby HA for Cloud
4 Performance Evaluation
5 Conclusion
3
Example DBMS Setting
x DBMSbu↵er pool
Log DB
BEGIN CHECKPOINTEND CHECKPOINT
C
nodefailure
restart recovery
4
Standalone Restart Recovery
x DBMSbu↵er pool
Log DB
BEGIN CHECKPOINTEND CHECKPOINT
C
nodefailure
restart recovery
4
Standalone Restart Recovery
x DBMSbu↵er pool
Log DB
BEGIN CHECKPOINTEND CHECKPOINT
C
nodefailure
restart recovery
4
Typical Shared-Nothing Hot Standby
Active DBMSbu↵er pool
Log DB
Standby DBMS
bu↵er pool
Log DB
x
nodefailure
failover
BEGIN CHECKPOINTEND CHECKPOINT
C C
5
Hot Standby Failure and Failover
Active DBMSbu↵er pool
Log DB
Standby DBMS
bu↵er pool
Log DB
x
nodefailure
failover
BEGIN CHECKPOINTEND CHECKPOINT
C C
5
Hot Standby Failure and Failover
Active DBMSbu↵er pool
Log DB
Standby DBMS
bu↵er pool
Log DB
x
nodefailure
failover
BEGIN CHECKPOINTEND CHECKPOINT
C C
5
Hot Standby Is Widely Used
Active DBMSbu↵er pool
Log DB
Standby DBMS
bu↵er pool
Log DB
x
nodefailure
failover
BEGIN CHECKPOINTEND CHECKPOINT
C C
5
Typical Hot Standby High Availability
Storage in Cloud?
Active DBMSbu↵er pool
Log DB
Standby DBMS
bu↵er pool
Log DB
Persistent Storage = Reliable Shared StorageHow can we exploit shared persistent storageto build better highly available database systems?
6
Shared Storage in Cloud Settings
Storage in Cloud?
Active DBMSbu↵er pool
Log DB
Standby DBMS
bu↵er pool
Log DB
Persistent Storage = Reliable Shared StorageHow can we exploit shared persistent storageto build better highly available database systems?
6
Shared Storage in Cloud Settings
Storage in Cloud?
Active DBMSbu↵er pool
Log DB
Standby DBMS
bu↵er pool
Log DB
Persistent Storage = Reliable Shared Storage
How can we exploit shared persistent storageto build better highly available database systems?
6
Shared Storage in Cloud Settings
Storage in Cloud?
Active DBMSbu↵er pool
Log DB
Standby DBMS
bu↵er pool
Log DB
Persistent Storage = Reliable Shared Storage
How can we exploit shared persistent storageto build better highly available database systems?
6
SHADOW: Hot Standby HA for Cloud
Active DBMSbu↵er pool
Standby DBMS
bu↵er pool
x
nodefailure
failover
DB DBLog Log
Log
async rep
Xwrite-o✏oading coordinatedcheckpoint
Recycled Hot Standby HA in Cloud
BEGIN CHECKPOINTEND CHECKPOINT
C
Simplicity: pushes responsibility for durabilityand replication into the storage system
Flexibility: decouples database replication fromDBMS replication
Performance: write-o✏oading (logging andcheckpointing)
E�ciency: less I/O bandwidth
7
SHADOW: Single Logical Log
Active DBMSbu↵er pool
Standby DBMS
bu↵er pool
x
nodefailure
failover
DB DB
Log
LogLog
async rep
Xwrite-o✏oading coordinatedcheckpoint
Single Logical Log
BEGIN CHECKPOINTEND CHECKPOINT
C
Simplicity: pushes responsibility for durabilityand replication into the storage system
Flexibility: decouples database replication fromDBMS replication
Performance: write-o✏oading (logging andcheckpointing)
E�ciency: less I/O bandwidth
7
SHADOW: Single Logical Database
Active DBMSbu↵er pool
Standby DBMS
bu↵er pool
x
nodefailure
failover
DB DB
Log
LogLog
async rep
Xwrite-o✏oading coordinatedcheckpoint
Single Logical Database
BEGIN CHECKPOINTEND CHECKPOINT
C
Simplicity: pushes responsibility for durabilityand replication into the storage system
Flexibility: decouples database replication fromDBMS replication
Performance: write-o✏oading (logging andcheckpointing)
E�ciency: less I/O bandwidth
7
SHADOW: Hot Standby HA for Cloud
Active DBMSbu↵er pool
Standby DBMS
bu↵er pool
x
nodefailure
failover
DB
DB
Log Log
Log
async rep
Xwrite-o✏oading coordinatedcheckpoint
SHADOW High Availability
BEGIN CHECKPOINTEND CHECKPOINT
C
Simplicity: pushes responsibility for durabilityand replication into the storage system
Flexibility: decouples database replication fromDBMS replication
Performance: write-o✏oading (logging andcheckpointing)
E�ciency: less I/O bandwidth
7
SHADOW: Hot Standby HA for Cloud
Active DBMSbu↵er pool
Standby DBMS
bu↵er pool
x
nodefailure
failover
DB
DB
Log Log
Log
async rep
Xwrite-o✏oading coordinatedcheckpoint
BEGIN CHECKPOINTEND CHECKPOINT
C
Simplicity: pushes responsibility for durabilityand replication into the storage system
Flexibility: decouples database replication fromDBMS replication
Performance: write-o✏oading (logging andcheckpointing)
E�ciency: less I/O bandwidth
7
SHADOW: Node Failure and Failover
Active DBMSbu↵er pool
Standby DBMS
bu↵er pool
x
nodefailure
failover
DB
DB
Log Log
Log
async rep
Xwrite-o✏oading coordinatedcheckpoint
BEGIN CHECKPOINTEND CHECKPOINT
C
Simplicity: pushes responsibility for durabilityand replication into the storage system
Flexibility: decouples database replication fromDBMS replication
Performance: write-o✏oading (logging andcheckpointing)
E�ciency: less I/O bandwidth
7
SHADOW: Node Failure and Failover
Active DBMSbu↵er pool
Standby DBMS
bu↵er pool
x
nodefailure
failover
DB
DB
Log Log
Log
async rep
Xwrite-o✏oading coordinatedcheckpoint
BEGIN CHECKPOINTEND CHECKPOINT
C
Simplicity: pushes responsibility for durabilityand replication into the storage system
Flexibility: decouples database replication fromDBMS replication
Performance: write-o✏oading (logging andcheckpointing)
E�ciency: less I/O bandwidth
7
Advantages of SHADOW HA
Active DBMSbu↵er pool
Standby DBMS
bu↵er pool
x
nodefailure
failover
DB
DB
Log Log
Log
async rep
Xwrite-o✏oading coordinatedcheckpoint
BEGIN CHECKPOINTEND CHECKPOINT
C
Simplicity: pushes responsibility for durabilityand replication into the storage system
Flexibility: decouples database replication fromDBMS replication
Performance: write-o✏oading (logging andcheckpointing)
E�ciency: less I/O bandwidth
7
Advantages of SHADOW HA
Active DBMSbu↵er pool
Standby DBMS
bu↵er pool
x
nodefailure
failover
DB
DB
Log Log
Log
async rep
Xwrite-o✏oading coordinatedcheckpoint
BEGIN CHECKPOINTEND CHECKPOINT
C
Simplicity: pushes responsibility for durabilityand replication into the storage system
Flexibility: decouples database replication fromDBMS replication
Performance: write-o✏oading (logging andcheckpointing)
E�ciency: less I/O bandwidth
7
Advantages of SHADOW HA
Active DBMSbu↵er pool
Standby DBMS
bu↵er pool
x
nodefailure
failover
DB
DB
Log Log
Log
async rep
Xwrite-o✏oading coordinatedcheckpoint
BEGIN CHECKPOINTEND CHECKPOINT
C
Simplicity: pushes responsibility for durabilityand replication into the storage system
Flexibility: decouples database replication fromDBMS replication
Performance: write-o✏oading (logging andcheckpointing)
E�ciency: less I/O bandwidth
7
Advantages of SHADOW HA
Active DBMSbu↵er pool
Standby DBMS
bu↵er pool
x
nodefailure
failover
DB
DB
Log Log
Log
async rep
Xwrite-o✏oading coordinatedcheckpoint
BEGIN CHECKPOINTEND CHECKPOINT
C
Simplicity: pushes responsibility for durabilityand replication into the storage system
Flexibility: decouples database replication fromDBMS replication
Performance: write-o✏oading (logging andcheckpointing)
E�ciency: less I/O bandwidth
7
Advantages of SHADOW HA
Active DBMSbu↵er pool
Standby DBMS
bu↵er pool
x
nodefailure
failover
DB
DB
Log Log
Log
async rep
Xwrite-o✏oading coordinatedcheckpoint
BEGIN CHECKPOINTEND CHECKPOINT
C
Simplicity: pushes responsibility for durabilityand replication into the storage system
Flexibility: decouples database replication fromDBMS replication
Performance: write-o✏oading (logging andcheckpointing)
E�ciency: less I/O bandwidth
7
Experimental Methodology
Compare SHADOW System with two baselinesStandalone (SA): single DBMS with restart recovery (varyingcheckpint interval)Synchronous Replication (SR): two replicated DBMSes (nativePostgreSQL implementation)
TPC-C Benchmark (100 Warehouses), no think time
PostgreSQL 9.3 (database fits in memory)
Linux kernel 3.2.0-56
Amazon EC2 with Elastic Block Store (EBS)
8
TPC-C Benchmark Throughtput
0
10000
20000
30000
40000
50000
60000
SAD SA10 SA
Thro
ughput (t
pm
C)
Standalone
Amazon EC2 c3.4xlarge instances, 100WH TPC-C workloaddatabase fits in memory (PostgreSQL 9.3, Linux kernel 3.2.0-56)
Hot Standby
9
TPC-C Benchmark Throughtput
0
10000
20000
30000
40000
50000
60000
SAD SA10 SA SR
Thro
ughput (t
pm
C)
Standalone
Amazon EC2 c3.4xlarge instances, 100WH TPC-C workloaddatabase fits in memory (PostgreSQL 9.3, Linux kernel 3.2.0-56)
Hot Standby
9
TPC-C Benchmark Throughtput
0
10000
20000
30000
40000
50000
60000
SAD SA10 SA SR SHADOW
Thro
ughput (t
pm
C)
Standalone
Amazon EC2 c3.4xlarge instances, 100WH TPC-C workloaddatabase fits in memory (PostgreSQL 9.3, Linux kernel 3.2.0-56)
Hot Standby
9
Variability of TPC-C Throughput
0
10000
20000
30000
40000
50000
60000
SAD SA10 SA
tpm
C
Standalone
Box-and-whisker plot (Q1,Q2,Q3)Ten second interval (new order transactions per second x 60))
0
10000
20000
30000
40000
50000
60000
SAD SA10 SA SR SHADOW
tpm
C
Hot Standby
10
Conclusion
SHADOW Hot Standby HA for CloudHow to exploit shared storage to build better high availabledatabase systems?
Single logical copy of the database and logPushes responsibility for replication out of the DBMS and intothe underlying storage tierDecouples database replication from DBMS replication
Outperforms PostgreSQL’s native replication on a TPC-Cbenchmark
Stabler throughput(tpmC) over time
Geographical limitationReplication coverage is limited to shared storage coverage
11
Mahalo!
12