crate shared nothing web backends - web backend meetup may 2014

Shared Nothing Web Backends

Matthias Wahl - developer @ crate.io

Agenda

Shared Nothing

Crate

Shared Nothing Architectures using Crate

Cluster Workout

Prepare for Workout

1.) bash -c "$(curl -L try.crate.io)"

https://cdn.crate.io/downloads/releases/crate-0.38.2.tar.gz

2.) sh$ bin/crate

0.) share our “craty” WLAN pw: select*fromcrate;

Prepare for Workout

3.) start up the twitter tutorial

Shared Nothing

Shared Nothing

Wikipedia: !

a distributed computing architecture in which each

node is independent and self-sufficient, and there is no

single point of contention across the system. More

specifically, none of the nodes share memory or disk

storage.

Shared Nothing

Origin:

!

Micheal Stonebraker

“The Case for Shared Nothing” (1986)

!

http://pdf.aminer.org/000/255/770/

the_case_for_shared_nothing.pdf

Shared Nothing

Principle of most functional languages

Shared state/resources (memory) is what makes threading hard

No SPOF

Easy concurrency

DB

Cache

App App App App

DB - Replica

Cache Cache

Bad Idea

DB

Cache

App App App App

DB - Replica

Cache Cache

Bad Idea

Crate

Austria and Berlin based Startup

founded 2013

open source since april https://github.com/crate/crate

Crate

shared nothing massively scalable datastore

standing on the shoulders of giants

Crate

Crate

automatic sharding and replication

(semi-) structured models

blob support

SQL query language

Crate

powerful fulltext search capabilities

complex aggregations in near real-time

- no joins

Crate

crate-java

crate-jdbc

crate-python (+SQLAlchemy)

crate-ruby

…

CRATE Clients

Shared Nothing with Crate

data and query execution is distributed among all nodes

no master/slave - all nodes are equal

automatic sharding & replication

Shared Nothing with Crate

example architecture

crate

app

crate

app

crate

app

…

Load Balancer

Shared Nothing with Crate

horizontally scalable as hell

still flexible enough for complex applications

high availability for your whole stack

Shared Nothing with Crate

create table blogpost ( id string primary key, created timestamp, text string, creator string )

Shared Nothing with Crate

from sqlalchemy import Column, String, DateTime from microblog.model import Base import uuid !!def genuuid(): return str(uuid.uuid4()) !!class BlogPost(Base): ! __tablename__ = 'blogpost' ! id = Column(String, default=genuuid, primary_key=True) text = Column('text', String, nullable=False) creator = Column('creator', String, nullable=False) created = Column('created', DateTime, nullable=False)

Shared Nothing with Crate

@rpcmethod_route(request_method="GET") def list(self): """ Return all blogposts """ query = DBSession.query(BlogPost).order_by(BlogPost.created.desc()) blogposts = query.all() result = [] for post in blogposts: result.append({'id': post.id, 'created': post.created.isoformat(), 'text': post.text, 'creator': post.creator}) return {"data": {"blogposts": result}}

Shared Nothing with Crate

get into detail: !

http://lovelysystems.github.io/lovely.microblog/

Cluster Workout!!!

CRATE

Thank you

web: https://crate.io/

github: https://github.com/crate

twitter: @cratedata

IRC: #crate

stackoverflow tag: cratedata