an intro to neo4j and some use cases (jfokus 2011)

Neo4j- or, why graph dbs are teh

AWESOME!!!11

oh, and also... USE CASES!!!

Emil EifremNeo Technology

#neo4j@[email protected]

What's the plan?

An intro to graph databases and Neo4j

Brief review of some use cases

Conclusion and Q&A

Graph DBs& Neo4j intro

The Graph DB model: representationCore abstractions:

NodesRelationships between nodesProperties on both

name = “Emil”age = 29sex = “yes”

type = KNOWStime = 4 years

type = carvendor = “SAAB”model = “95 Aero”

11 22

33

Example: The Matrix

name = “Thomas Anderson”age = 29

11

name = “The Architect”

4242

CODED_BY

disclosure = public

name = “Cypher”last name = “Reagan”

disclosure = secretage = 6 months

name = “Agent Smith”version = 1.0blanguage = C++

33

1313

KNOWS KNOWS

name = “Morpheus”rank = “Captain”occupation = “Total badass”

age = 3 days

name = “Trinity”

77

22

KNOWS

KNOWS

KNO

WS

Code (1): Building a node spaceGraphDatabaseService graphDb = ... // Get factory

// Create Thomas 'Neo' AndersonNode mrAnderson = graphDb.createNode();mrAnderson.setProperty( "name", "Thomas Anderson" );mrAnderson.setProperty( "age", 29 );

// Create MorpheusNode morpheus = graphDb.createNode();morpheus.setProperty( "name", "Morpheus" );morpheus.setProperty( "rank", "Captain" );morpheus.setProperty( "occupation", "Total bad ass" );

// Create a relationship representing that they know each othermrAnderson.createRelationshipTo( morpheus, RelTypes.KNOWS );// ...create Trinity, Cypher, Agent Smith, Architect similarly

Code (1): Building a node spaceGraphDatabaseService graphDb = ... // Get factoryTransaction tx = graphDb.beginTx();

// Create Thomas 'Neo' AndersonNode mrAnderson = graphDb.createNode();mrAnderson.setProperty( "name", "Thomas Anderson" );mrAnderson.setProperty( "age", 29 );

// Create MorpheusNode morpheus = graphDb.createNode();morpheus.setProperty( "name", "Morpheus" );morpheus.setProperty( "rank", "Captain" );morpheus.setProperty( "occupation", "Total bad ass" );

// Create a relationship representing that they know each othermrAnderson.createRelationshipTo( morpheus, RelTypes.KNOWS );// ...create Trinity, Cypher, Agent Smith, Architect similarly

tx.commit();

Code (1b): Defi ning RelationshipTypes// In package org.neo4j.graphdbpublic interface RelationshipType{ String name();}

// In package org.yourdomain.yourapp// Example on how to roll dynamic RelationshipTypesclass MyDynamicRelType implements RelationshipType{ private final String name; MyDynamicRelType( String name ){ this.name = name; } public String name() { return this.name; }}

// Example on how to kick it, static-RelationshipType-likeenum MyStaticRelTypes implements RelationshipType{ KNOWS, WORKS_FOR,}

Whiteboard friendly

Björn Big Car

DayCare

Björn

owns

drivesbuild

The Graph DB model: traversalTraverser framework for high-performance traversing across the node space

name = “Emil”age = 31sex = “yes”

type = KNOWStime = 4 years

type = carvendor = “SAAB”model = “95 Aero”

11 22

33

Example: Mr Anderson’s friends


11


4242

CODED_BY

disclosure = public




33

1313

KNOWS KNOWS


age = 3 days


77

22

KNOWS

KNOWS

KNO

WS

Code (2): Traversing a node space

// Instantiate a traverser that returns Mr Anderson's friendsTraverser friendsTraverser = mrAnderson.traverse(

Traverser.Order.BREADTH_FIRST,StopEvaluator.END_OF_GRAPH,ReturnableEvaluator.ALL_BUT_START_NODE,RelTypes.KNOWS,Direction.OUTGOING );

// Traverse the node space and print out the resultSystem.out.println( "Mr Anderson's friends:" );for ( Node friend : friendsTraverser ){

System.out.printf( "At depth %d => %s%n",friendsTraverser.currentPosition().getDepth(),friend.getProperty( "name" ) );

}

$ bin/start-neo-exampleMr Anderson's friends:

At depth 1 => MorpheusAt depth 1 => TrinityAt depth 2 => CypherAt depth 3 => Agent Smith$

friendsTraverser = mrAnderson.traverse( Traverser.Order.BREADTH_FIRST, StopEvaluator.END_OF_GRAPH, ReturnableEvaluator.ALL_BUT_START_NODE, RelTypes.KNOWS, Direction.OUTGOING );




disclosure = public

age = 3 days





77

22

33

1313

4242

11KNOWS KNOWS CODED_BYKNOWS

KNOWSKN

OW

S

Example: Friends in love?




disclosure = public





77

22

33

1313

4242


KNOWS

KNO

WS

LOVES

Code (3a): Custom traverser

// Create a traverser that returns all “friends in love”Traverser loveTraverser = mrAnderson.traverse(

Traverser.Order.BREADTH_FIRST,StopEvaluator.END_OF_GRAPH,new ReturnableEvaluator(){

public boolean isReturnableNode( TraversalPosition pos ){

return pos.currentNode().hasRelationship( RelTypes.LOVES, Direction.OUTGOING );

}},RelTypes.KNOWS,Direction.OUTGOING );

Code (3a): Custom traverser

// Traverse the node space and print out the resultSystem.out.println( "Who’s a lover?" );for ( Node person : loveTraverser ){

System.out.printf( "At depth %d => %s%n",loveTraverser.currentPosition().getDepth(),person.getProperty( "name" ) );

}

new ReturnableEvaluator(){ public boolean isReturnableNode( TraversalPosition pos) { return pos.currentNode(). hasRelationship( RelTypes.LOVES, Direction.OUTGOING ); }},

$ bin/start-neo-exampleWho’s a lover?

At depth 1 => Trinity$




disclosure = public





77

22

33

1313

4242


KNOWSKN

OW

SLOVES

Bonus code: domain modelHow do you implement your domain model?Use the delegator pattern, i.e. every domain entity wraps a Neo4j primitive:

// In package org.yourdomain.yourappclass PersonImpl implements Person{ private final Node underlyingNode; PersonImpl( Node node ){ this.underlyingNode = node; }

public String getName() { return (String) this.underlyingNode.getProperty( "name" ); } public void setName( String name ) { this.underlyingNode.setProperty( "name", name ); }}

Domain layer frameworksQi4j (www.qi4j.org)

Framework for doing DDD in pure Java5Defi nes Entities / Associations / Properties

Sound familiar? Nodes / Rel’s / Properties!Neo4j is an “EntityStore” backend

Jo4neo (http://code.google.com/p/jo4neo)

Annotation drivenWeaves Neo4j-backed persistence into domain objects at runtime

S pring Data G raph (http://www.springsource.org/spring-data)

Collaboration with SpringSourceAnnotation driven, AspectJ based

Neo4j system characteristicsDisk-based

Native graph storage engine with custom binary on-disk format

TransactionalJTA/JTS, XA, 2PC, Tx recovery, deadlock detection, MVCC, etc

Scales up

Many billions of nodes/rels/props on single JVMRobust

7+ years in 24/7 production

M'kay... but what are some

use cases?

Social data (customer: e.g. EU largest social network)

name = “Mike”age = 29

11

disclosure = public

name = “Charlie”last_name = “Runkle”

name = “Dani”last_name = “California”age = 27

33

1313

KNOWS KNOWS

name = “Hank”last_name = “Moody”age = 42

age = 3 days

name = “Karen”

77

22

KNOWS

KNOWSKN

OW

S

name = “Marcy Runkle”

4242

KNOWS

Just a social graph?

Spatial data (customer: large telecom company)

name = “Omni Hotel”lat = 3492848long = 283823423

11

length = 7 miles

name = ...lat, long = ...

name = “Swedland”lat = 23410349long = 2342348852

33

1313

ROAD ROOOAD

name = “The Tavern”lat = 1295238237long = 234823492

length = 3 miles

name = ...

77

22

ROAD

ROADR

OAD

name = ...

4242

ROAD

Social? Spatial? … Social AND spatial!

Social AND spatial data (customer: LBS)

name = “Omni Hotel”lat = 3492848long = 283823423

11

weight = 10

name = “Emil”beer_qual = expert

name = “Maria”age = 30beer_qual = non-existant

33

1313

LIKES SIBLING


length = 3 miles

name = ...

77

22

ROAD

ROADR

OAD

name = “Peter”

4242

KNOWS

Financial data (customer: international bank)

name = “Mr Godfather”karma = veeeery-lowcash = more-than-you

11

amount = $1000

name = “Emil”cash = always-too-li'l

title = “ATM @ Wall St”id = 230918484233cash_left = 384204

33

1313

TRANSFER WITHDRAW


amount = $1000

name = ...

77

22

OWNS

DEPOSITTR

ANSF

ER

name = ...

4242

WITHDRAW

Cute. But what about performance?

# nodes query timeRelational database (MySQL) 1 000 2 000 msGraph database (Neo4j) 1 000 2 msGraph database (Neo4j) 1 000 000 2 ms

pathExists(a, b, 5)

name = “Mr Godfather”karma = veeeery-lowcash = more-than-you

11

amount = $1000

name = “Emil”cash = always-too-li'l

title = “ATM @ Wall St”id = 230918484233cash_left = 384204

33

1313

TRANSFER WITHDRAW


amount = $1000

name = ...

77

22

WITHDRAW

DEPOSITTR

ANSF

ER

name = ...

4242

WITHDRAW

How ego are you? (aka other impls?)Franz’ A lleg roG raph (http://agraph.franz.com)

Proprietary, Lisp, RDF-oriented but real graphdb

Sones g raphDB (http://sones.com)

.NET, open source version available

Twitter's FlockDB (http://github.com/twitter/flockdb)

Twitter's graph database for large and shallow graphs

Sparsity Technologies Dex (http://sparsitytechnologies.com)

New on the scene, spun out of Spanish university

Objectivity InfiniteG raph (http://infinitegraph.com)

Graph db on-top-of OODB

ConclusionGraphs && Neo4j => teh awesome!Available NOW under AGPLv3 / commercial license

AGPLv3: “if you’re open source, we’re open source”If you have proprietary software? Must buy a commercial licenseBut the first one is free!

Downloadhttp://neo4j.org

Feedbackhttp://lists.neo4j.org

The N eo4j teamis hiring !

Do you see the Matrix? Apply now.

Questions?

Image credit: lost again! Sorry :(

http://neotechnology.com

an intro to neo4j and some use cases (jfokus 2011)

Documents