distributing data the aerospike way

DISTRIBUTING DATA THE

AEROSPIKE WAYYoung PaikDirector, Sales [email protected]

July 24, 2013

mailto:[email protected]

…OR Why Is This Line Taking So Long?

© 2013 Aerospike. All rights reserved. Pg. 2

A Database Is Like A Conference Registration Line

➤ The goal is to get as many people through the line as quickly as possible.

➤ Registrants must get their own registration package, not just anyone’s.


Scaling Throughput

All databases (and registration systems) have limits to performance. The real question is how do you go beyond your current limits.

There are two basic strategies: ➤ Vertical scaling – upgrade single server➤ Horizontal scaling – distribute to multiple

servers


Vertical Scaling

Vertical scaling means that if a small server can handle some traffic … a big one can handle more. This is true … to a point.


What’s Wrong With Vertical Scaling?➤ Expensive➤ Still need to deal with failover.

What happens if your DB goes down? What happens if you need to upgrade?

➤ May still not meet the storage/speed requirements

➤ Still a single point of failure


Horizontal Scaling

Horizontal Scaling means that in some way the load will be distributed among many servers.


What Do You Want From Horizontal Scaling?➤ Hide the complexity of distribution.➤ Linear scalability.➤ Better service availability.➤ Deal with meteor strike on your data

center.


Different Distribution Models

Distributed databases will place different data on different nodes.

Some common methods:➤ Simple sharding➤ Hashed sharding➤ Master-slave ➤ Smart partitioning


Simple Sharding


Simple Sharding

Clients know which node has the data.


Simple Sharding

What happens if a node fails?


Simple Sharding

Pros Cons+ Easy to set up. Clients are

written with a knowledge of how the data is distributed.

+ Servers aren’t coordinated, so no intra-cluster communication is necessary.

- May lead to imbalance and hot nodes

- If a node fails, the data on that node is unavailable.

- Adding new nodes requires reconfiguration on the clients and re-shuffling of data on the server, resulting in service down time.

- Replication must be handled separately.


Hashed Sharding


Hashing Sharding

➤ The key can be hashed using a hashing algorithm to create a seemingly random string

➤ The first several characters of the hash can be used to determine the node for that data.

PaikPaik

C820G3KH15HH3KASD43S

C820G3KH15HH3KASD43S


Instead of using the actual key value, use a hash to randomize how the data is distributed.

Hashed Sharding

Hashed sharding will balance data and load.


Hashed Sharding

But has the same problem on a node failure.


Hashed Sharding

Pros Cons+ Easy to set up. Clients are

written with a knowledge of how the data is distributed.

+ Servers aren’t coordinated, so no intra-cluster communication is necessary.

+ Data/traffic is now balanced.

- If a node fails, the data on that node is unavailable.

- Adding new nodes requires reconfiguration on the clients and re-shuffling of data on the server, resulting in service down time.

- Replication must be handled separately.


Master-Slave


Master-Slave

Master coordinates connection with slave nodes.


Master-Slave Sharding

Pros Cons+ Relatively simple setup with

master controlling distribution.

+ Replication can be set up to go to backup node. Master is responsible for coordinating.

- Adding new nodes requires reconfiguration on the master and often manual re-shuffling of data on the server, resulting in service down time.

- Requires multiple network connections.

- Single point of failure: the master. Some databases like Mongo require 3 masters (called configuration servers) where 2 will be backups for the main one.


Smart PartitioningThe Aerospike Way


Smart Partitioning

Every registrant knows where to go.


Map

Smart Partitioning

And, every registrant knows where to go if a node fails.


Map

Smart Partition Architecture


Cluster creates a map of how data is distributed, called a partition map.

Combine features from other architectures to create a map.

Smart Partitioning

➤ Every key is hashed using the RIPEMD160 hash function

➤ The creates a fixed 160 bits (20 bytes) string.

➤ 12 bits of this hash are used to identify the partition id

➤ There are 4096 partitions

➤ Are distributed among the nodes

PaikPaik

182023kh15hh3kahdjsh182023kh15hh3kahdjsh

PartitionID

Master node

Replica node

… 1 4

1820 2 3

1821 3 2

4096 4 1


Aerospike uses a partition table

Smart Partitioning

For simplicity, let’s take a 3 node cluster with only 9 partitions and a replication factor of 2.


Smart Partitioning

Pros Cons+ Relatively simple setup, with the

cluster determining data distribution.

+ Balanced distribution.

+ No single point of failure.

+ Replication is automatic and immediate.

+ Failover is automatic and immediate.

+ Rebalancing is automatic and immediate.

+ An arbitrary number of nodes can be added to increase capacity.

+ True 24x7 uptime. Cluster can be upgraded on a rolling basis.

- Application must be written using smart API.


What Do You Want From Horizontal Scaling?➤ Hide the complexity of distribution.

Balanced data distribution. No “hot nodes.” Automatic client reconfiguration. No need to manually

reconfigure/restart clients.➤ Linear scalability.

Easy to calculate needed capacity. Cluster can be an arbitrary number of nodes.

➤ Better service availability. 24 x 7 uptime. No downtime, even for “routine”

maintenance. No single point of failure. Automatic replication of data. Automatic failover. Automatic rebalancing when nodes fail. Automatic rebalancing when adding nodes.

➤ Deal with a catastrophe on your data center.© 2013 Aerospike. All rights reserved. Pg. 29

So can you deal with a meteor hitting my data center?


But what about a meteor strike?

Multi-Datacenter Architecture


Data Center 1

Data Center 2

Data Center 3

Cross Data Center Replication (XDR)➤ Asynchronous replication for long link delays

and outages

➤ Namespaces configured to replicate to a destination cluster – master / slave, including star and ring

➤ Replication process Transaction journal on partition master and

replica XDR process writes batches to destination Transmission state shared with source replica Retransmission in case of network fault When data arrives back at originating cluster,

transaction ID matching prevents subsequent application and forwarding

➤ In master / master replication, conflict resolution via multiple versions, or timestamp

© 2013 Aerospike. All rights reserved. Confidential Pg. 32

Caveats for Evaluating Distributed Databases

When testing new databases:➤ Make sure to test to an appropriate scale.➤ Beware of inconsistent performance in a

cloud environment.➤ If a database has caching, make sure your

use is realistic.


THANK YOU


distributing data the aerospike way

Technology

sales engineering aerospike

different data

shuffling of data

simple sharding clients

data center

hashing sharding

hashed sharding pros

node failure