scale out your graph across servers and clouds with orientdb

Scale Out Your Graph Across Servers and Clouds

with OrientDB

#gdsf17

Luca Garulli, Founder and CEO @lgarulli GraphDay - San Francisco - June 17, 2017

Copyright (c) - OrientDB LTD 2

We all want the same thing: an open source GraphDB that is

Fast, Flexible, Scalable and … Unbreakable!

Complexity

ability

Single Thread

Multi Thread

Distributed

Systems/Complexity

Master Node

Auto-Discovery

I’m the only one!

Auto-Discovery

Master Node

Connected!

Master Node

Updated distributed configuration is broadcasted to

all the connected clients

Clients See the Distributed Configuration

Master Node

Auto-reconnect in Case of Failure

In case of failure, the clients auto-reconnect to

the available nodes

Master Node

DBs are automatically deployed

to the newly joined nodes

Database Auto-deploy

Replication: Under the Hood Client commits a transaction

Master Node

HA Queue

Transaction

Requests

Master Node

HA Queue

Replication: Under the Hood Response Handling

WriteQuorum = 2

Sends OK

Requests

Responses

Replication: Under the Hood Fix the unaligned node

11111111111111

Master Node

HA Queue

Requests

Responses

Commit

Replication: Under the Hood 2pc messages

12121212121212

Fix (response != quorum)

Commit (response == quorum)

Rollback (quorum not reached)

Master Node A

Optimistic MVCC Distributed Transaction

(2) tx task

(4) return result

(7) asynch* commit, rollback or fix

(1) Lock all records in order

+ Execute TX locally

Master Node B

(3) Lock all records in order

+ Execute TX locally

(5) Check results

+ unlock all records

(8) Execute the message and Unlock all records

C Node A becomes the transaction coordinator

(6) send response back to the client

Consistency- Distributed Locks don’t block reads - During the transaction the old version of the

record is retrieved - 2pc message is asynchronous, so a record could

not be updated yet on server X - If you need higher consistency, don’t use load

balancer policy ROUND_ROBIN_REQUEST

{ "autoDeploy": true, "writeQuorum": “majority”, "newNodeStrategy": “static”, "clusters": { "*": { "servers": ["<NEW_NODE>"] } } }

Default Configuration (json)

deploys the database automatically on new

writeQuorum=2 means at least 2

nodes must agree on writes. Set it to the majority to ensure

consistency

newNodeStrategy: static (default) or

dynamic. Static = once nodes join, they are

part of configuration

clusters contain the distributed

configuration per cluster

cluster “*” represents the default cluster

configuration. Any new node will join,

containing all the clusters

Custom Setting per Cluster

cluster “customer” extends the default configuration with a custom writeQuorum

{ "autoDeploy": true, "writeQuorum": “majority”, "newNodeStrategy": “static”, "clusters": { "*": { "servers": ["<NEW_NODE>"] }, "customer": { "writeQuorum": 3, "servers": ["<NEW_NODE>"] } } }

Copyright (c) - OrientDB LTD

What about scalability?

Performance with Reads

2222222222

Master Node

10,000 req/sec

Full replication provides linear scalability on reads,

but what about scalability on writes?

Performance with Writes

2424242424

Master Node

12,000 req/sec

2525252525

Master Node

7000 req/sec

2626262626

Master Node

5000 req/sec

The more replicas you

configure, the more the propagation cost

will impact performance, based

on writeQuorum

In order to scale up writes, you need

sharing + replication

We used a solution similar to RAID for Hard Drives

Sharding in OrientDB v2.2

Assign 1 Cluster per Node

customer_usa customer_europe customer_china

Master Node usa

Master Node europe

Master Node china

Customer

Master Node usa

Master Node china

Master Node europe

RAID for Databases

303030

customer_usa customer_europe customer_china

Customer

customer_europecustomer_usacustomer_china

Replica factor = 2

Master Node usa

Master Node china

Master Node europe

Traversal with Spark (Pregel)

313131

https://spark.apache.org/docs/latest/graphx-programming-guide.html#pregel-api

Sharding ConfigurationLEGEND: X = Owner, o = Copy +---------------+-----------+----------+-------+-------+-------+ | | | |MASTER |MASTER |MASTER | | | | |ONLINE |ONLINE |ONLINE | +---------------+-----------+----------+-------+-------+-------+ |CLUSTER |writeQuorum|readQuorum| usa |europe |china | +---------------+-----------+----------+-------+-------+-------+ |* | 2 | 1 | X | | | |customer_usa | 2 | 1 | X | | o | |customer_europe| 2 | 1 | o | X | | |customer_china | 2 | 1 | | o | X | +---------------+-----------+----------+-------+-------+-------+

Sharding Configuration

first node in list is the “owner” for that

cluster

{ "clusters": { “customer_usa": { "servers": [“usa”, “china”] }, “customer_europe": { "servers": [“europe”, “usa”] }, “customer_china": { "servers": [“china”, “europe”] } } }

Static OwnerThe owner can be static. If the

owner is not online, new records can’t be

inserted in the cluster

{ "clusters": { "client_usa": { "owner": "usa", "servers" : [ "usa", "europe", "asia" ] } } } This assures

the owner is not assigned

dynamically

Master Node

5000 req/sec

Performance with Sharding

Master Node

5000 req/sec

Master Node

5000 req/sec

Master Node

5000 req/sec

Master Node

5000 req/sec

Master Node

5000 req/sec

Each write is replicated on only 2 nodes

Master Node

C Master Node

Master Node

Linear and Elastic Scalability on both Read & Writes!

Replica only Nodes

Replica NodeMaster

Node usa

Master Node usa

Replica NodeReplica NodeReplica NodeReplica NodeReplica NodeReplica NodeReplica NodeReplica NodeReplica NodeReplica NodeReplica Node

Replica servers don’t concur in writeQuorum

Server Role Configuration

“*” by default each node is a master.

Following nodes are replica only

{ "servers": { "*": “master”, “usa_r1": “replica", “usa_r2": “replica", “usa_r3": “replica", “europe_r1": “replica", “china1_r1": "replica" } }

Master Node

Load-Balancing on Client Side

Master Node

Load-Balancing Configuration

final OrientGraphFactory factory = new OrientGraphFactory(“remote:localhost/demo");

factory.setConnectionStrategy( OStorageRemote.CONNECTION_STRATEGY.ROUND_ROBIN_CONNECT);

OrientGraph graph = factory.getTx();

Available strategies: - STICKY, - ROUND_ROBIN_CONNECT, - ROUND_ROBIN_REQUEST

What if some records are not aligned

for **ANY** reason?

OrientDB Auto-Repairer

- Executed in chain - It works in batch of 50 records per time (configurable) - Strategies: - Quorum: checks if it meets the configured write quorum - Content: checks the content - Majority: checks if at least there is a majority - Version: gets the higher version - DC (EE only). Example: dc{winner:asia}

Master Node

Auto-Repair Flow

(1) tx read [#10:33,#43:90,#12:23]

(3) return [{#10:33 v1},{#43:90 v5},{#12:23 v9}]

(2) lock [#10:33,#43:90,#12:

23](4) fix [{#43:90 v6}]

Dynamic Timeouts (from v2.2.18)

orientdb> HA STATUS -latency -output=text

REPLICATION LATENCY AVERAGE (in milliseconds) +-------+-----+------+-----+ |Servers|node1|node2*|node3| +-------+-----+------+-----+ |node1 | | 0.60| 0.43| |node2* | 0.39| | 0.38| |node3 | 0.35| 0.53| | +-------+-----+------+-----+

Cross Datacenter & Cloud

Dublin

Austin

Cross Data Centre and Cloud replication. Data

Centers don’t need to have the same number

of servers

Data-Center (Enterprise only)"dataCenters": { "rome": { "writeQuorum": "all", "servers": [ "europe-0", "europe-1", "europe-2" ] }, "austin": { "writeQuorum": "all", "servers": [ "usa-0", "usa-1", "usa-2" ] } }

Performance

Yahoo Benchmark 3 Nodes

OrientDB is 2x-3x faster than Cassandra on the

same HW/SW configuration, same workload

Ops/sec

OrientDB v3.1 (Q1 2018)- DHT-like algorithm = Distributed Sharded index

- Native support for batch-traversal (Pregel-like) without Spark

- Background refactoring of the Graph based on usage statistics

Thank you!@lgarulli orientdb.com

scale out your graph across servers and clouds with orientdb

Software

orientdb - tutorialspoint.com · orientdb also belongs to...

geospatial graphs made easy with orientdb - codemotion spain

orientdb distributed architecture 1.1

orientdb - the first few days in five(ish) minutes

€¦ · web vieworientdb - quick guide. orientdb -...

orientdb: unlock the value of document data relationships

orientdb document or graph? select the right model (old...

orientdb: a document-graph database ready for the cloud -...

orientdb & node.js overview - js.everywhere() kw

orientdb codemotion 2014

the database designed for the modern world -...

orientdb & hazelcast: in-memory distributed graph database

geospatial graphs made easy with orientdb - codemotion...

orientdb – the multi-model and graph database ·...

geecon prague 2016 - geospatial graphs made easy with...

graph databases and orientdb

fully reactive - from data to ui with orientdb + node.js +...

orientdb & lucene

orientdb the database for the web 1.1

securing servers in public and hybrid clouds