graph databases

Graph Databases

Josh Adell <josh.adell@gmail.com>20110719

Who am I?

• Software developer: PHP, Javascript, SQL• http://www.dunnwell.com• Fan of using the right tool for the job

The Problem

The Solution?

> -- Given "Keanu Reeves" find a connection to "Kevin Bacon"> SELECT ??? FROM cast WHERE ???

+---------------------------------------------------------------------+| actor_name                 | movie_title                            |+============================+========================================+| Jennifer Connelley         | Higher Learning                        |+----------------------------+----------------------------------------+| Laurence Fishburne         | Mystic River                           |+----------------------------+----------------------------------------+| Laurence Fishburne         | Higher Learning                        |+----------------------------+----------------------------------------+| Kevin Bacon                | Mystic River                           |+----------------------------+----------------------------------------+| Keanu Reeves               | The Matrix                             |+----------------------------+----------------------------------------+| Laurence Fishburne         | The Matrix                             |+----------------------------+----------------------------------------+

Find Every Actor at Each Degree

> -- First degree> SELECT actor_name FROM cast WHERE movie_title IN (SELECT DISTINCT movie_title FROM cast WHERE actor_name='Kevin Bacon')

> -- Second degree> SELECT actor_name FROM cast WHERE movie_title IN (SELECT DISTINCT movie_title FROM cast WHERE actor_name IN (SELECT actor_name FROM cast WHERE movie_title IN (SELECT DISTINCT movie_title FROM cast WHERE actor_name='Kevin Bacon')))

> -- Third degree> SELECT actor_name FROM cast WHERE movie_title IN(SELECT DISTINCT movie_title FROM cast WHERE actor_name IN (SELECT actor_name FROM cast WHERE movie_title IN (SELECT DISTINCT movie_title FROM cast WHERE actor_name IN (SELECT actor_name FROM cast WHERE movie_title IN (SELECT DISTINCT movie_title FROM cast WHERE actor_name='Kevin Bacon'))))

The Truth

Relational databases aren't very good with relationships

Data

RDBMs

RDBs Use Set Math

The Real Problem

Finding relationships across multiple degrees of separation    ...and across multiple data types    ...and where you don't even know there is a relationship

The Real Solution

Computer Science Definition

A graph is an ordered pair G = (V, E) where V is a set of vertices and E is a set of edges, which are pairs of vertices.

Some Graph DB Vocabulary

• Node: vertex• Relationship: edge• Property: meta-datum attached to a node or relationship• Path: an ordered list of nodes and relationships• Index: node or relationship lookup table

Relationships are First-Class Citizens

• Have a type• Have properties• Have a direction

o Domain semanticso Traversable in any direction

Graph Examples

Relational Databases are Graphs!

New Solution to the Bacon Problem

$keanu = $actorIndex->find('name', 'Keanu Reeves');$kevin = $actorIndex->find('name', 'Kevin Bacon');

$path = $keanu->findPathTo($kevin);

Some Graph Use Cases

• Social networking• Manufacturing• Map directions• Fraud detection• Multi-tenancy

Modelling a Domain with Graphs

• Graphs are "whiteboard-friendly"• Nouns become nodes• Verbs become relationships• Properties are adjectives and adverbs

Audience Participation!

• Neo Technologies• http://neo4j.org• Embedded in Java applications• Standalone server via REST• Plugins: spatial, lucene, rdf

• http://github.com/jadell/Neo4jPHP

Using the REST client

$client = new Client(new Transport());

$customer = new Node($client);$customer->setProperty('name', 'Josh')->save();

$store = new Node($client);$store->setProperty('name', 'Home Despot')      ->setProperty('location', 'Durham, NC')->save();

$order = new Node($client);$order->save();$item = new Node($client);$item->setProperty('item_number', 'Q32-ESM')->save();

$order->relateTo($item, 'CONTAINS')->save();$customer->relateTo($order, 'BOUGHT')->save();$store->relateTo($order, 'SOLD')->save();

$customerIndex = new Index($client, Index::TypeNode, 'customers');$customerIndex->add($customer, 'name', $customer->getProperty('name'));$customerIndex->add($customer, 'rating', 'A++');

Graph Mining

• Paths• Traversals• Ad-hoc Queries

Path Finding

• Find any connection from node A to node B• Limit by relationship types and/or direction• Path finding algorithms: all, simple, shortest, Dijkstra

$customer = $customerIndex->findOne('name', 'Josh');$item = $itemIndex->findOne('item_number', 'Q32-ESM');

$path = $item->findPathsTo($customer)             ->setMaxDepth(2)             ->getSinglePath();

foreach ($path as $node) {    echo $node->getId() . "\n";}

Traversal

• Complex/Custom path finding• Base next decision on previous path

$traversal = new Traversal($client);$traversal->setOrder(Traversal::OrderDepthFirst)->setUniqueness(Traversal::UniquenessNodeGlobal)->setPruneEvaluator('javascript','(function traverse(pos) {     if (pos.length() == 1 && pos.lastRelationship.getType() == "CONTAINS") {        return false;    } else if (pos.length() == 2 && pos.lastRelationship.getType() == "BOUGHT") {        return false;    }    return true;})(position)')->setReturnFilter('javascript',    'return position.endNode().getProperty('type') == 'Customer;');

$customers = $traversal->getResults($item, Traversal::ReturnTypeNode);

• Uses mathematical notation approach• Complex traversal behaviors, including backtracking• https://github.com/tinkerpop/gremlin/wiki

m = [:]g.v(1).out('likes').in('likes').out('likes').groupCount(m)m.sort{a,b -> a.value <=> b.value}

Cypher

• "What to find" vs. "How to find"

$query = 'START item=(1)MATCH (item)<-[:CONTAINS]-(order)<-[:BOUGHT]-(customer)RETURN customer';

$cypher = new Cypher\Query($client, $query);$customers = $cypher->getResultSet();

Cypher Syntax

START item = (1)                        START item = (1,2,3)START item = (items, 'name:Q32*')       START item = (1), customer = (2,3)

MATCH (item)<--(order)                  MATCH (order)-->(item)MATCH (order)-[r]->(item)                              MATCH ()--(item)MATCH    (supplier)-[:SUPPLIES]->(item)<-[:CONTAINS]-(order),    (customer)-[:RATED]->(item)WHERE customer.name = 'Josh' and s.coupon = 'freewidget'

RETURN item, order                      RETURN customer, item, r.ratingRETURN r~TYPE                                                      RETURN COUNT(*)ORDER BY customer.name DESC             RETURN AVG(r.rating)LIMIT 3 SKIP 2

Cypher - All Together Now

// Find the top 10 `widget` ratings by customers who bought AND rated// `widgets`, and the supplier

START item = (items, 'name:widget')MATCH (item)<--(order)<--(customer)-[r:RATED]->(item)<--(supplier)RETURN customer, r.rating, supplier ORDER BY r.rating DESC LIMIT 10

Tools

• Neoclipse• Webadmin

Are RDBs Useful At All?

• Aggregation• Ordered data• Truly tabular data• Few or clearly defined relationships

Questions?

Resources

• http://neo4j.org• http://docs.neo4j.org• http://www.youtube.com/watch?v=UodTzseLh04

o Emil Eifrem (Neo Tech. CEO) webinaro Check out around the 54 minute mark

• http://github.com/jadell/Neo4jPHP

• http://joshadell.com• josh.adell@gmail.com• @josh_adell• Google+, Facebook, LinkedIn