gdm 2011 - neo4j and real world apps

28
Peter Neubauer Neo Technology #neo4j @peterneubauer [email protected]

Upload: peter-neubauer

Post on 16-May-2015

8.596 views

Category:

Technology


1 download

DESCRIPTION

There is a lot of research happening in graph databases these days. This is a

TRANSCRIPT

Page 1: GDM 2011 - Neo4j and real world apps

Peter NeubauerNeo Technology

#neo4j@[email protected]

Page 2: GDM 2011 - Neo4j and real world apps

Neo4j – what is it?

Java based, embeddable, data-localGPL/AGPLACID, JTA compliantIndexing framework24/7 since 2003High Availability clustering supportGreat communityTinkerpop pipes processing stack

Page 3: GDM 2011 - Neo4j and real world apps

Building a node space (core API)GraphDatabaseService graphDb = ... // Get factory

// Create Thomas 'Neo' AndersonNode mrAnderson = graphDb.createNode();mrAnderson.setProperty( "name", "Thomas Anderson" );mrAnderson.setProperty( "age", 29 );

// Create MorpheusNode morpheus = graphDb.createNode();morpheus.setProperty( "name", "Morpheus" );morpheus.setProperty( "rank", "Captain" );morpheus.setProperty( "occupation", "Total bad ass" );

// Create a relationship representing that they know each othermrAnderson.createRelationshipTo( morpheus, RelTypes.KNOWS );// ...create Trinity, Cypher, Agent Smith, Architect similarly

Page 4: GDM 2011 - Neo4j and real world apps

Building a node spaceGraphDatabaseService graphDb = ... // Get factoryTransaction tx = graphdb.beginTx();

// Create Thomas 'Neo' AndersonNode mrAnderson = graphDb.createNode();mrAnderson.setProperty( "name", "Thomas Anderson" );mrAnderson.setProperty( "age", 29 );

// Create MorpheusNode morpheus = graphDb.createNode();morpheus.setProperty( "name", "Morpheus" );morpheus.setProperty( "rank", "Captain" );morpheus.setProperty( "occupation", "Total bad ass" );

// Create a relationship representing that they know each othermrAnderson.createRelationshipTo( morpheus, RelTypes.KNOWS );// ...create Trinity, Cypher, Agent Smith, Architect similarlytx.commit();

Page 5: GDM 2011 - Neo4j and real world apps

Code (2): Traversing a node space

// Instantiate a traverser that returns Mr Anderson's friendsTraverser friendsTraverser = mrAnderson.traverse(

Traverser.Order.BREADTH_FIRST,StopEvaluator.END_OF_GRAPH,ReturnableEvaluator.ALL_BUT_START_NODE,RelTypes.KNOWS,Direction.OUTGOING );

// Traverse the node space and print out the resultSystem.out.println( "Mr Anderson's friends:" );for ( Node friend : friendsTraverser ){

System.out.printf( "At depth %d => %s%n",friendsTraverser.currentPosition().getDepth(),friend.getProperty( "name" ) );

}

Page 6: GDM 2011 - Neo4j and real world apps

Rubygem install neo4j

require ”rubygems”require 'neo4j'

class Person include Neo4j::NodeMixin property :name, :age, :occupation index :name has_n :friendsend

Neo4j::Transactoin.run do neo = Person.new :name=>'Neo', :age=>29 morpheus = Person.new :name=>'Morpheus', :occupation=>'badass' neo.friends << morpheusend

neo.friends.each {|p|...}

Page 7: GDM 2011 - Neo4j and real world apps

Neo4j 1.3 news

GPL Community Edition128 Billion primitives address spaceMore graph algosShort string → longWeb visualizationGremlin 0.9 built-inHA improvements

Page 8: GDM 2011 - Neo4j and real world apps

Web admin

Page 9: GDM 2011 - Neo4j and real world apps

Neo4j High Availability

Page 10: GDM 2011 - Neo4j and real world apps

Neo4j – what do I do with it?

Network ManagementMaster Data ManagementSocialFinanceSpatialOther

BioinformaticsRDFRouting

Page 11: GDM 2011 - Neo4j and real world apps

Call Data Records (CDR)Forming a graphLocation basedPossible uses:

Find clusters (better plans)Build social connectionsFind influencers

Page 12: GDM 2011 - Neo4j and real world apps

Financial data – fraud detection

name = “Mr Godfather”karma = veeeery-lowcash = more-than-you

11

amount = $1000

name = “Emil”cash = always-too-li'l

title = “ATM @ Wall St”id = 230918484233cash_left = 384204

33

1313

TRANSFER WITHDRAW

name = “The Tavern”lat = 1295238237long = 234823492

amount = $1000

name = ...

77

22

OWNS

DEPOSITTR

ANSF

ER

name = ...

4242

WITHDRAW

Page 13: GDM 2011 - Neo4j and real world apps

Routing

Page 14: GDM 2011 - Neo4j and real world apps

Social graphs

RecommendationsLocation based servicesInfluencersShortest path

Page 15: GDM 2011 - Neo4j and real world apps

Recommendations and big graphsGlobal heuristics

Page rankLocal recommendations

Shortest pathsHammock functionsRandom walksDijkstra, A*, Shooting star etc

Page 16: GDM 2011 - Neo4j and real world apps

Impact Analytics, CMDB, Network Management, Provisioning

Page 17: GDM 2011 - Neo4j and real world apps

Impact Analytics, CMDB, Network Management, Provisioning

Page 18: GDM 2011 - Neo4j and real world apps

Master Data Management

Page 19: GDM 2011 - Neo4j and real world apps

Multiple indexes - GIS

Page 20: GDM 2011 - Neo4j and real world apps

Neo4j dynamic layers

Layer1

Layer2

Layer3

GeometryEncoder

DynamicQuery

DynamicStyles

DynamicMeta-Inf

Connected domain data Neo4j Spatial GIS and Spatial stacks

Page 21: GDM 2011 - Neo4j and real world apps

OpenStreetMap

Page 22: GDM 2011 - Neo4j and real world apps

Network Topology analysis

Analytics of network coverage and frequenciesCell towersDrive dataInfrastructure

AnalyticsSpatial signal strengthAntenna placement and azimuthFrequency planningNetwork differences over timeReporting and charting

Page 23: GDM 2011 - Neo4j and real world apps

Cell network analysis

Page 24: GDM 2011 - Neo4j and real world apps

Cell network analysis

Page 25: GDM 2011 - Neo4j and real world apps

Simulations (Energy market)

agents, markets, power plants, bids, substances, technologies

Page 26: GDM 2011 - Neo4j and real world apps

Relevant research areasDistributed traversals

Parallel (BSP) and Local (Neo4j)Crossing shardsMultithreaded

Graph shardingUpfront sharding utilsRuntime sharding algosPartial sharding/replication

Parallel garbage collection under high loadBig graph algos and heuristics

Page 27: GDM 2011 - Neo4j and real world apps

Questions?

Image credit: lost again! Sorry :(

Page 28: GDM 2011 - Neo4j and real world apps

http://neotechnology.com