scalable database, scalable language @ jdc 2013

102

Upload: maciek-prochniak

Post on 29-Nov-2014

96 views

Category:

Software


0 download

DESCRIPTION

Scalable database, Scalable language @ JDC 2013

TRANSCRIPT

Page 1: Scalable database, Scalable language @ JDC 2013
Page 2: Scalable database, Scalable language @ JDC 2013

Maciek Próchniak

Algebraic topologyhocolim

Group cohomology

Monads

GWT

TouKCamel

OSGi

CQRS

Scala

OrientDB

Page 3: Scalable database, Scalable language @ JDC 2013
Page 4: Scalable database, Scalable language @ JDC 2013

Poland, Egypt?

USA is scared :)

Page 5: Scalable database, Scalable language @ JDC 2013

scalable database

OrientDB

scalable language

Scala

Page 6: Scalable database, Scalable language @ JDC 2013

" ... a general purpose programming language designed to express common programming patterns in a concise, elegant, and type-safe way"

Scala about Scala

Page 7: Scalable database, Scalable language @ JDC 2013

Potential client about Scala

"tiny market share and require a very specific type of developers, which is more likely to be found in an academic, rather than business environment"

Page 8: Scalable database, Scalable language @ JDC 2013

OrientDB about OrientDB

"... an Open Source NoSQL DBMS with both the features of Document and Graph DBMSs.

"... written in Java and it's amazing fast: can store up to 150,000 records per second on common hardware"

Page 9: Scalable database, Scalable language @ JDC 2013

Potential client about OrientDB

"OrientDB is another experimental product which is closely tied to European academia, much like Scala"

"Moreover, choosing a document-oriented database for a typical business/commercial application is simply an architectural mistake."

Page 10: Scalable database, Scalable language @ JDC 2013

Really?

Page 11: Scalable database, Scalable language @ JDC 2013
Page 12: Scalable database, Scalable language @ JDC 2013

OracleMySQL

PostgreSQL

MongoCouchRiak

Neo4J

Page 13: Scalable database, Scalable language @ JDC 2013

OracleMySQL

PostgreSQL

MongoCouchRiak

Neo4J

?

Page 14: Scalable database, Scalable language @ JDC 2013

OracleMySQL

PostgreSQL

MongoCouchRiak

Neo4J

OrientDB

Page 15: Scalable database, Scalable language @ JDC 2013

What's the real difference?

● Architecture

● Data model

● Data access

Page 16: Scalable database, Scalable language @ JDC 2013

What's the real difference?

● Architecture

● Data model

● Data access

Page 17: Scalable database, Scalable language @ JDC 2013

Consistency

Partition tolerance

Availability

Page 18: Scalable database, Scalable language @ JDC 2013

P - do you really need it?

Page 19: Scalable database, Scalable language @ JDC 2013

NoSQL is about scalability?

● Replication

● Sharding

- TBD

Page 20: Scalable database, Scalable language @ JDC 2013

Need more consistency?

● Transactions

● Optimistic locking

● MVCC

Page 21: Scalable database, Scalable language @ JDC 2013

NoSQL written in Java

● No additional software

● We all know how to tune JVM

... Don't we?

Page 22: Scalable database, Scalable language @ JDC 2013

Architecture

Page 23: Scalable database, Scalable language @ JDC 2013

Architecture - in memory

Page 24: Scalable database, Scalable language @ JDC 2013

Architecture - embedded

Page 25: Scalable database, Scalable language @ JDC 2013

Architecture - remote server

HTTPBinary

Page 26: Scalable database, Scalable language @ JDC 2013

memory/local/remote

Page 27: Scalable database, Scalable language @ JDC 2013

Impact: testability

Unit

Functional

Integration

H2/??

Mock

DevBase

Page 28: Scalable database, Scalable language @ JDC 2013

Impact: testability

Unit

Functional

Integration

memory:/local:

mock/memory:

local:/remote:

Page 29: Scalable database, Scalable language @ JDC 2013

What's the real difference?

● Architecture

● Data model

● Data access

Page 30: Scalable database, Scalable language @ JDC 2013

Meet our domain

Customer

Address

Customer

EmailAddress

Branch

Email

Page 31: Scalable database, Scalable language @ JDC 2013

How would you model that?

Address

Email

Branch Customer 1..* 1..*

1..*

Page 32: Scalable database, Scalable language @ JDC 2013

Aggregate - consistency boundary

Aggregate: A collection of objects that are bound together by a root entity, otherwise known as an aggregate root.

Page 33: Scalable database, Scalable language @ JDC 2013

Aggregate

The aggregate root guarantees the consistency of changes being made within the aggregate by forbidding external objects from holding references to its members.

Page 34: Scalable database, Scalable language @ JDC 2013

Data model

Customer

Address

Customer

Email

Page 35: Scalable database, Scalable language @ JDC 2013

Modification

Customer

Address

Customer

Email

Page 36: Scalable database, Scalable language @ JDC 2013

Customer in OrientDB

Page 37: Scalable database, Scalable language @ JDC 2013

When document is not enough

Page 38: Scalable database, Scalable language @ JDC 2013

Data model

Customer

Address

Customer

Email

Page 39: Scalable database, Scalable language @ JDC 2013

Customer

Modification

Address

Customer

Email

Page 40: Scalable database, Scalable language @ JDC 2013
Page 41: Scalable database, Scalable language @ JDC 2013

Graph - direct links

Customer Customer

Customer Customer

Page 42: Scalable database, Scalable language @ JDC 2013

Documents

connected into

Graph

Page 43: Scalable database, Scalable language @ JDC 2013

Connections in OrientDB

Page 44: Scalable database, Scalable language @ JDC 2013

New possibilities

Europe

Poland France Romania

Warsaw Lublin Cracow Łódź

Page 45: Scalable database, Scalable language @ JDC 2013

New possibilities

2012

Jan Feb March

1 2 3 4

Page 46: Scalable database, Scalable language @ JDC 2013

OrientDB data model can reflect your domain model

better than document or relational bases

Page 47: Scalable database, Scalable language @ JDC 2013

What's the real difference?

● Architecture

● Data model

● Data access

Page 48: Scalable database, Scalable language @ JDC 2013

Let's get our hands dirty

Page 49: Scalable database, Scalable language @ JDC 2013

Scala

● implicit conversions, parameters

● typeclasses

● for-comprehension

● infix notation

● case classes

Page 50: Scalable database, Scalable language @ JDC 2013

Connecting - classic template

Page 51: Scalable database, Scalable language @ JDC 2013

Connecting - classic - PROS

● Everybody knows it

● Simple

● Can easily do DI

● AOP

Page 52: Scalable database, Scalable language @ JDC 2013

Composability?

Page 53: Scalable database, Scalable language @ JDC 2013

Side effects?

Page 54: Scalable database, Scalable language @ JDC 2013

Making DB interaction explicit

● Type constructortrait WithConnection[T]

Page 55: Scalable database, Scalable language @ JDC 2013

Making DB interaction explicit

● Trivial interactiondef pt[T](a:T) : WithConnection[T]

Page 56: Scalable database, Scalable language @ JDC 2013

Making DB interaction explicit

● Composing interactions def fmap[T](a:WithConnection[T],map:T=>WithConnection[H]) :

WithConnection[H]

Page 57: Scalable database, Scalable language @ JDC 2013

Example

Page 58: Scalable database, Scalable language @ JDC 2013

Example

Page 59: Scalable database, Scalable language @ JDC 2013

for-comprehension

Page 60: Scalable database, Scalable language @ JDC 2013

Composition

Page 61: Scalable database, Scalable language @ JDC 2013

for-comprehension

Page 62: Scalable database, Scalable language @ JDC 2013

Don't want to scary you...

... but I smell a monad here!

Page 63: Scalable database, Scalable language @ JDC 2013

Scala - loading documents

Page 64: Scalable database, Scalable language @ JDC 2013

Browsing documents - POJSON

Page 65: Scalable database, Scalable language @ JDC 2013

Browsing documents - POJSON

Page 66: Scalable database, Scalable language @ JDC 2013

Updating records

update customer set name = 'Maciej' where name = 'Maciek'

update customer add friends = #8:2

update customer put contactDetail = 'email2', '[email protected]'

Page 67: Scalable database, Scalable language @ JDC 2013

Updating records - lens like

Page 68: Scalable database, Scalable language @ JDC 2013

Updating records - lens like

Page 69: Scalable database, Scalable language @ JDC 2013

Developers

know

SQL

Page 70: Scalable database, Scalable language @ JDC 2013

ExtendedSQL

select from customer where name = 'maciek'

select labels[0] from customer where name = 'maciek'

select address.street from customer where contactDetails.email = '[email protected]'

Page 71: Scalable database, Scalable language @ JDC 2013

ExtendedSQL

select from customers where trucks contains (capacity = 2)

select from suppliers where

trucks[capacity = 5].size() > 3

Page 72: Scalable database, Scalable language @ JDC 2013

OrientDB

NotSoNoSQL

Page 73: Scalable database, Scalable language @ JDC 2013

Scala queries

Page 74: Scalable database, Scalable language @ JDC 2013

Graph queries

Page 75: Scalable database, Scalable language @ JDC 2013

Developers

know

SQL

Page 76: Scalable database, Scalable language @ JDC 2013

Traverse command

traverse customer.friends from #8:0 while $depth < 3

select from ( traverse customer.friends from #8:0 while $depth < 10 )where label contains nice

Page 77: Scalable database, Scalable language @ JDC 2013

Traverse performance

Local operation - no joins

Constant lookup time

Page 78: Scalable database, Scalable language @ JDC 2013

Traverse with scala predicates

Page 79: Scalable database, Scalable language @ JDC 2013

Traverse with scala predicates

Page 80: Scalable database, Scalable language @ JDC 2013

What about higher level access?

● ORM anybody?

● No need for partial updates, dirty checking?

Page 81: Scalable database, Scalable language @ JDC 2013

JSON under the hood

Pluggable serialization strategies:

● Jackson

● Lift-Json

● ...

Page 82: Scalable database, Scalable language @ JDC 2013

Scala serializer

Page 83: Scalable database, Scalable language @ JDC 2013

Typesafe queries?

Scala

Language

Intergration

Connectivity

Kit

Page 84: Scalable database, Scalable language @ JDC 2013
Page 85: Scalable database, Scalable language @ JDC 2013

Lifted embedding

Page 86: Scalable database, Scalable language @ JDC 2013

Lifted embedding - meta

Page 87: Scalable database, Scalable language @ JDC 2013

Lifted embedding - meta

Page 88: Scalable database, Scalable language @ JDC 2013

Lifted embedding - meta

Page 89: Scalable database, Scalable language @ JDC 2013

Lifted embedding - query

Page 90: Scalable database, Scalable language @ JDC 2013

Lifted embedding - query

Page 91: Scalable database, Scalable language @ JDC 2013

Can we do better?

Page 92: Scalable database, Scalable language @ JDC 2013

Direct embedding - meta

Page 93: Scalable database, Scalable language @ JDC 2013

Direct embedding - query

Page 94: Scalable database, Scalable language @ JDC 2013

Direct embedding - query

Page 95: Scalable database, Scalable language @ JDC 2013

Type safety?

Page 96: Scalable database, Scalable language @ JDC 2013

Functional Programming patterns help to create

flexible, robust data access code

Page 97: Scalable database, Scalable language @ JDC 2013

Call for action!

● Try it!

● Use it

● Find bugs ;)

● Spread the word

Page 98: Scalable database, Scalable language @ JDC 2013

https://github.com/nuvolabase/orientdb/

https://github.com/mproch/scala-orientdb

https://github.com/mproch/slick-orientdb

Page 99: Scalable database, Scalable language @ JDC 2013

Take away:

● OrientDB data model can reflect your domain model better than document or relational bases

● Functional Programming patterns can help create flexible, robust data access code

Page 100: Scalable database, Scalable language @ JDC 2013

شكرا

Page 101: Scalable database, Scalable language @ JDC 2013

[email protected]://touk.plhttp://mproch.blogspot.com