the graphdb landscape and sones - odbms.org graphdb landscape and sones nosql frankfurt , 9/28/2010...
Post on 19-Apr-2018
218 Views
Preview:
TRANSCRIPT
NoSQL Frankfurt , 9/28/2010Achim Friedland <achim@sones.de> 1NoSQL Frankfurt , 9/28/2010Achim Friedland <achim@sones.de>
Frankfurtco-located to ICOODB 2010
28. September 2010Frankfurt, Germany
The GraphDB Landscapeand sones
1NoSQL Frankfurt , 9/28/2010Achim Friedland <achim@sones.de>
Photo: Gephi, flickr
NoSQL Frankfurt , 9/28/2010Achim Friedland <achim@sones.de> 2
Photo: litlnemo, flickr
2NoSQL Frankfurt , 9/28/2010Achim Friedland <achim@sones.de>
NoSQL Frankfurt , 9/28/2010Achim Friedland <achim@sones.de> 3
• 43 years Object Orientation / OOP
• 35 years Relational Databases / SQL
• 21 years Semantic Web / RDF
Photo: litlnemo, flickr
3NoSQL Frankfurt , 9/28/2010Achim Friedland <achim@sones.de>
NoSQL Frankfurt , 9/28/2010Achim Friedland <achim@sones.de> 4
• 43 years Object Orientation / OOP
• 35 years Relational Databases / SQL
• 21 years Semantic Web / RDF
Photo: litlnemo, flickr
4NoSQL Frankfurt , 9/28/2010Achim Friedland <achim@sones.de>
…but still we do not know where
our files have gone!
NoSQL Frankfurt , 9/28/2010Achim Friedland <achim@sones.de> 5
Our wish list…
5NoSQL Frankfurt , 9/28/2010Achim Friedland <achim@sones.de>
Photo: Chris Westermeyer, flickr
NoSQL Frankfurt , 9/28/2010Achim Friedland <achim@sones.de> 6
Simple management of structured, semi-structured and unstructured
information
6NoSQL Frankfurt , 9/28/2010Achim Friedland <achim@sones.de>
Photo: litlnemo, flickr
NoSQL Frankfurt , 9/28/2010Achim Friedland <achim@sones.de> 7
Simple management of structured, semi-structured and unstructured
information
7NoSQL Frankfurt , 9/28/2010Achim Friedland <achim@sones.de>
Photo: litlnemo, flickr
SQL
XML/JSON binaries
NoSQL Frankfurt , 9/28/2010Achim Friedland <achim@sones.de> 8
Simple recursively linked information models
8NoSQL Frankfurt , 9/28/2010Achim Friedland <achim@sones.de>
Photo: Robbert van der Steeg, flickr
NoSQL Frankfurt , 9/28/2010Achim Friedland <achim@sones.de> 9
Simple recursively linked information models
9NoSQL Frankfurt , 9/28/2010Achim Friedland <achim@sones.de>
Photo: Robbert van der Steeg, flickr
• <a href = “…”>• metadata e.g. EXIF
NoSQL Frankfurt , 9/28/2010Achim Friedland <achim@sones.de> 1010NoSQL Frankfurt , 9/28/2010Achim Friedland <achim@sones.de>
Photo: Khem, flickr
Traversing linked information, finding shortest-paths, do
semantic partitions
NoSQL Frankfurt , 9/28/2010Achim Friedland <achim@sones.de> 11
Photo: NASA, flickr
11NoSQL Frankfurt , 9/28/2010Achim Friedland <achim@sones.de>
Recommendation and discovery
of potentially interessting linked information
NoSQL Frankfurt , 9/28/2010Achim Friedland <achim@sones.de> 12
Photo: NASA, flickr
12NoSQL Frankfurt , 9/28/2010Achim Friedland <achim@sones.de>
Recommendation and discovery
of potentially interessting linked information
socialpersonal item-related
NoSQL Frankfurt , 9/28/2010Achim Friedland <achim@sones.de> 13
Photo: squacco, flickr
13NoSQL Frankfurt , 9/28/2010Achim Friedland <achim@sones.de>
Consistency criteria and indices for simple attributes up to complex
subgraph structures
NoSQL Frankfurt , 9/28/2010Achim Friedland <achim@sones.de> 1414NoSQL Frankfurt , 9/28/2010Achim Friedland <achim@sones.de>
Photo: Cedrix Thual, flickr
Versioned information and versioned information schemata
NoSQL Frankfurt , 9/28/2010Achim Friedland <achim@sones.de> 1515NoSQL Frankfurt , 9/28/2010Achim Friedland <achim@sones.de>
Photo: Birger Hoppe, flickr
Good integration into state-of-the-art programming concepts
(no Object-Relational-Mappers!)
NoSQL Frankfurt , 9/28/2010Achim Friedland <achim@sones.de> 1616NoSQL Frankfurt , 9/28/2010Achim Friedland <achim@sones.de>
Photo: Chuck “Caveman” Coker, flickr
Can graph-databases help?
NoSQL Frankfurt , 9/28/2010Achim Friedland <achim@sones.de> 17
Photo: Jeffrey Beall, flickr
Graph-Databases?
Graph = ( Vertics, Edges )
17NoSQL Frankfurt , 9/28/2010Achim Friedland <achim@sones.de>
A graph database is a database that uses graph structures with nodes, edges, and properties to represent and store information. General graph
databases that can store any graph are distinct from specialized graph databases such as triple stores and
network databases.http://en.wikipedia.org/wiki/Graph_database
NoSQL Frankfurt , 9/28/2010Achim Friedland <achim@sones.de> 18
Photo: Jeffrey Beall, flickr
Graph-Databases?
Graph = ( Vertics, Edges )
R. Angles, C. Gutierrez: Survey of graph database models,
ACM Computing Surv. 40, 1 (Feb. 2008), 1-39
Adrian Silvescu, Donia Caragea, Anna Atramentov:
Graph Databases, Technical Report. May 2002
Mark Gemis, Jan Paredaens, Inge Thyssens, und Jan Van den
Bussche: GOOD: A Graph-Oriented Object Database System, 1993;
In: Proceedings of the1993 ACM SIGMOD Intern. Conf. on Management of Data (Washington D.C., US, May 1993), 505-510.
18NoSQL Frankfurt , 9/28/2010Achim Friedland <achim@sones.de>
NoSQL Frankfurt , 9/28/2010Achim Friedland <achim@sones.de> 19
Photo: Gephi, flickr
The Property-Graph
FriendsAliceID = 1
Alter = 21
BobID = 2
Alter = 23since = 2009/09/21
NoSQL Frankfurt , 9/28/2010Achim Friedland <achim@sones.de> 19
Properties are key-values pairse.g. <String, Object> or <AttributeKey, Object>
NoSQL Frankfurt , 9/28/2010Achim Friedland <achim@sones.de> 20
Photo: Gephi, flickr
The Property-Graph
NoSQL Frankfurt , 9/28/2010Achim Friedland <achim@sones.de>
Focus on structured tables and their relational consistency constraints. Does not auto-scale.
vs. Relational Databases
Table A Table BRelation
Table
20
NoSQL Frankfurt , 9/28/2010Achim Friedland <achim@sones.de> 21
Photo: Gephi, flickr
The Property-Graph
Reference / PointerObject Object
NoSQL Frankfurt , 9/28/2010Achim Friedland <achim@sones.de>
Focus on structured or dynamic objects.No (simple way for) edge properties.
vs. Object-Oriented Model
21
NoSQL Frankfurt , 9/28/2010Achim Friedland <achim@sones.de> 22
Photo: Gephi, flickr
The Property-Graph
PredicateSubject Object
NoSQL Frankfurt , 9/28/2010Achim Friedland <achim@sones.de>
Focus on triples or quads.No (simple way for) edge properties.
vs. Semantic Web / RDF
22
NoSQL Frankfurt , 9/28/2010Achim Friedland <achim@sones.de> 23
Photo: Gephi, flickr
The Property-Graph
Friends EnemiesAlice Bob
Hobby Hobby
Guru
Crypto
NoSQL Frankfurt , 9/28/2010Achim Friedland <achim@sones.de>
Bruce
Eve
Enemies
observes
23
A directed, attributed, multi-relational graphas central data structure for graph-databases.
since = 2009/11/02
NoSQL Frankfurt , 9/28/2010Achim Friedland <achim@sones.de> 24
The Property-Graph
NoSQL Frankfurt , 9/28/2010Achim Friedland <achim@sones.de> 24
Photo: Jeffrey Beall, flickr
The Property-Graph Model
http://wiki.github.com/tinkerpop/blueprints/property-graphmodel
Rodriguez, M.A., Neubauer, P., “Constructions from Dots and Lines,”
Bulletin of the American Society for Information Science and
Technology, American Society for Information Science and Technology,
volume 36, August 2010.
…much more at http://markorodriguez.com ;)
Currently no academically founded or well-definedgraph data model, but…
NoSQL Frankfurt , 9/28/2010Achim Friedland <achim@sones.de> 25
Photo: Gephi, flickr
The Property-Graph
NoSQL Frankfurt , 9/28/2010Achim Friedland <achim@sones.de> 25
// Neo4J example
Node _alice = graphdb.createNode();_alice.setProperty("Name", "Alice");_alice.setProperty("Age", 21);
Node _bob = graphdb.createNode();_alice.setProperty("Name", „Bob");_alice.setProperty("Age", 23);
_alice.createRelationshipTo(_bob, Friends);_bob.createRelationshipTo(_alice, Friends);
NoSQL Frankfurt , 9/28/2010Achim Friedland <achim@sones.de> 26
Photo: Gephi, flickr
The Property-Graph
NoSQL Frankfurt , 9/28/2010Achim Friedland <achim@sones.de> 26
// sones gql example
CREATE VERTEX UserADD ATTRIBUTES (String Name, SET<User> Friends)INDICES (Name)
INSERT INTO User VALUES (Name = "Alice", Age = 21)INSERT INTO User VALUES (Name = "Bob", Age = 23)
LINK User(Name = ‘Alice') VIA Friends TO User(Name = ‘Bob')LINK User(Name = ‘Bob') VIA Friends TO User(Name = ‘Alice‘)
NoSQL Frankfurt , 9/28/2010Achim Friedland <achim@sones.de> 27
Photo: Gephi, flickr
Benefits of Graph-Databases
• Explicit graph data model– No implicit model like within relational dbs
– In the spirit of NoSQL:“One database for one data model”
– GraphDB gets a change to understand the semantics and our intention
27NoSQL Frankfurt , 9/28/2010Achim Friedland <achim@sones.de>
NoSQL Frankfurt , 9/28/2010Achim Friedland <achim@sones.de> 28
Photo: Gephi, flickr
Benefits of Graph-Databases
• Index-free adjacency– No global adjacency index as each vertex acts
as its own “mini-index”
– The speed for traversing from one node to another is independed of the size of the graph( Data locality and improved scalability )
– Optimized for random access patterns
28NoSQL Frankfurt , 9/28/2010Achim Friedland <achim@sones.de>
NoSQL Frankfurt , 9/28/2010Achim Friedland <achim@sones.de> 29
Photo: Gephi, flickr
Neo4J
sones
DEX
InfoGrid
InfiniteGraph
HyperGraphDB
OrientDB
Filament
VertexDB
29NoSQL Frankfurt , 9/28/2010Achim Friedland <achim@sones.de>
Graph-Database landscape
Marklogic?
NoSQL Frankfurt , 9/28/2010Achim Friedland <achim@sones.de> 30
Photo: Gephi, flickr
Data model
DEX
sones
Neo4J
InfoGrid
InfiniteGraph
HyperGraphDB
OrientDB
Filament
VertexDB
30NoSQL Frankfurt , 9/28/2010Achim Friedland <achim@sones.de>
Property-Graph
No Edge-Properties
Schemata
HyperGraphs
NoSQL Frankfurt , 9/28/2010Achim Friedland <achim@sones.de> 31
Photo: Gephi, flickr
Query methods
Neo4J
sones
DEX
InfoGrid
InfiniteGraph
HyperGraphDB
OrientDB
Filament
VertexDB
31NoSQL Frankfurt , 9/28/2010Achim Friedland <achim@sones.de>
Traverser API
Gremlin
Adapted SQL
GraphQL
HGQuery API
NoSQL Frankfurt , 9/28/2010Achim Friedland <achim@sones.de> 32
Photo: Gephi, flickr
Programming languages
Neo4J
sones
DEX
InfoGrid
InfiniteGraph
HyperGraphDB
OrientDB
Filament
VertexDB
C++
Java
C#
C
JS
Lua
32NoSQL Frankfurt , 9/28/2010Achim Friedland <achim@sones.de>
NoSQL Frankfurt , 9/28/2010Achim Friedland <achim@sones.de> 33
Photo: Gephi, flickr
Neo4J
sonesDEX
InfoGrid
InfiniteGraph
HyperGraphDBOrientDB
FilamentVertexDB
“Free Lunch”
AGPLv3
LGPL
BSD
Apache 2.0
$$$
33NoSQL Frankfurt , 9/28/2010Achim Friedland <achim@sones.de>
License models
NoSQL Frankfurt , 9/28/2010Achim Friedland <achim@sones.de> 34
Photo: Gephi, flickr
Neo4J
sones DEX
InfoGrid
InfiniteGraph
HyperGraphDB
OrientDB
FilamentVertexDB
34NoSQL Frankfurt , 9/28/2010Achim Friedland <achim@sones.de>
Protocols
REST/JSON
REST/XML
HTTP
NoSQL Frankfurt , 9/28/2010Achim Friedland <achim@sones.de> 35
• URL• Goals• APIs
• Transactions• Repl./Scaling• Persistency
http://www.neo4j.orgEmbedded and spatial DBMSJava, JRuby, Ruby, Python, Scala, Clojure, C#, ErlangConcurrent reads, Sync on nodesMaster-Slave, Master FailoverProprietary ondisk format
Neo4J
Photo: Gephi, flickr
35NoSQL Frankfurt , 9/28/2010Achim Friedland <achim@sones.de>
NoSQL Frankfurt , 9/28/2010Achim Friedland <achim@sones.de> 36
• URL• Goals
• Transactions• Repl./Scaling• Persistency
http://www.infogrid.orgDistributed GraphDB and Web-Application FrameworkYesp2p-replication, partitioningProprietary, RDBMS, Hadoop
InfoGrid
Photo: Gephi, flickr
36NoSQL Frankfurt , 9/28/2010Achim Friedland <achim@sones.de>
MeshBase _GDB = StoreMeshBase.create(_MySQLStore);MeshObject _xkcd = _GDB.getMeshObjectLifecycleManager().createMeshObject();_xkcd.setProperty("Name", "xkcd");_xkcd.setProperty("Url", "http://www.xkcd.com");_xkcd.relate(_good);
NoSQL Frankfurt , 9/28/2010Achim Friedland <achim@sones.de> 37
• URL• Goals
• Transactions• Repl./Scaling• Persistency
http://www.dama.upc.eduHigh performance and integration of various information sourcesn/an/aProprietary ondisk format
DEX
Photo: Gephi, flickr
37NoSQL Frankfurt , 9/28/2010Achim Friedland <achim@sones.de>
long xkcd = _GDB.newNode(Resource);_GDB.setAttribute(xkcd, Name, "xkcd");_GDB.setAttribute(xkcd, Url, "http://xkcd.com");int Tags = _GDB.newUndirectedEdgeType("Tags");long e2 = _GDB.newEdge(xkcd, good, Tags);
NoSQL Frankfurt , 9/28/2010Achim Friedland <achim@sones.de> 38
• URL• Goals• Transactions• Repl./Scaling• Persistency
http://www.kobrix.comAI, NLP, Semantic WebACI(D), STMp2pBerkeley DB
HyperGraphDB
Photo: Gephi, flickr
38NoSQL Frankfurt , 9/28/2010Achim Friedland <achim@sones.de>
// Note: Website(), Tag() are POJOHGHandle _xkcd = _GDB.add(new Website("xkcd", "http://www.xkcd.com"));HGHandle _good = _GDB.add(new Tag("good");HGHandle _xkcd2good = _GDB.add(new HGPlainLink(_xkcd, _good));
NoSQL Frankfurt , 9/28/2010Achim Friedland <achim@sones.de> 39
• URL• Goals• Transactions• Repl./Scaling• Persistency
http://www.infinitegraph.comVery large graph databasesDistributed transactionsObjectivity/DBObjectivity/DB
Infinite Graph
Photo: Gephi, flickr
39NoSQL Frankfurt , 9/28/2010Achim Friedland <achim@sones.de>
// Note: Website(), Tag() extend BaseVertex; Edge() extends BaseEdgeWebsite xkcd = new Website("xkcd", "http://xkcd.com/"); _GDB.addVertex(xkcd);Tag good = new Tag("good"); _GDB.addVertex(good);xkcd.addEdge (new Edge(), good, EdgeKind.BIDIRECTIONAL);
NoSQL Frankfurt , 9/28/2010Achim Friedland <achim@sones.de> 40
• URL• Goals
• Transactions• Repl./Scaling• Persistency
http://www.orienttechnologies.comProviding a KV-Store, DocumentDB
and a GraphDB in one solutionACID, MVCCn/aProprietary ondisk format
OrientDB
Photo: Gephi, flickr
40NoSQL Frankfurt , 9/28/2010Achim Friedland <achim@sones.de>
OGraphVertex _xkcd = _GDB.createVertex().set("Name", "xkcd").set("Url", "http://www.xkcd.com");
OGraphVertex _good = _GDB.createVertex().set("Name", "good");_xkcd.link (_good);
NoSQL Frankfurt , 9/28/2010Achim Friedland <achim@sones.de> 41
• URL• Goals• Transactions• Repl./Scaling• Persistency
http://www.sones.deManagement of linked (binary) dataACID, MVCC (alpha)p2p (alpha)Proprietary file system
sones
Photo: Gephi, flickr
41NoSQL Frankfurt , 9/28/2010Achim Friedland <achim@sones.de>
NoSQL Frankfurt , 9/28/2010Achim Friedland <achim@sones.de> 42
Photo: Shayne Kaye, flickr
User Friend
AliceID = 1
Alter = 21
BobID = 2
Alter = 23since = 2009/09/21
42NoSQL Frankfurt , 9/28/2010Achim Friedland <achim@sones.de>
sones Property-Hypergraph
User Friend CarolID = 3
Alter = 20since = 1997/04/11
SET<User> Friends
SetMaxNumber = 12
NoSQL Frankfurt , 9/28/2010Achim Friedland <achim@sones.de> 43
Photo: Shayne Kaye, flickr
User Friend BobID = 2
Alter = 23since = 2009/09/21
43NoSQL Frankfurt , 9/28/2010Achim Friedland <achim@sones.de>
sones Property-Hypergraph
User Friend CarolID = 3
Alter = 20since = 1997/04/11
SET<User> Friends
SetMaxNumber = 12
AliceID = 1
Alter = 21
NoSQL Frankfurt , 9/28/2010Achim Friedland <achim@sones.de> 44
Photo: Shayne Kaye, flickr
• Extensions to the Property Graph model
• Multiple edges are be grouped by a hyperedge
• Hyperedge may have their own properties
• Properties may include code as data(e.g. C#: Func<…>, ExpressionTrees)
• Allows calculations be done among the set of edges (GetMinWeight, SetMaxNumber, …)
sones Property-Hypergraph
44NoSQL Frankfurt , 9/28/2010Achim Friedland <achim@sones.de>
NoSQL Frankfurt , 9/28/2010Achim Friedland <achim@sones.de> 45
Photo: Shayne Kaye, flickr
sones Special Edges and Properties
45NoSQL Frankfurt , 9/28/2010Achim Friedland <achim@sones.de>
Vertex
PropertyStream DefaultEdition
LatestRevision
20100921…
Copy1
Copy2
20100918… Copy1
FileStream DefaultEdition
LatestRevision
20100815… Copy1
NoSQL Frankfurt , 9/28/2010Achim Friedland <achim@sones.de> 46
Photo: Shayne Kaye, flickr
sones Graph Query Language
46NoSQL Frankfurt , 9/28/2010Achim Friedland <achim@sones.de>
FROM User SELECT User.Friends.Friends.Name
• “SQL for graphs”
• Its goal is to provide a very user-friendly language forad-hoc graph queries
• Functions and aggregates are type-safe and can be extended by your own plug-ins, e.g.
• SELECT COUNT(User.Friends)
• SELECT User.Friends.Random(2)
• SELECT User.Friends.Name.Substring(2,5)
NoSQL Frankfurt , 9/28/2010Achim Friedland <achim@sones.de> 47
Photo: jonathanpercy, flickr
• Get outgoing edges, where their label is equal to ‘created’
• Get their incoming vertices
• Get incoming edges, where their labels equals ‘created’
• Get outgoing vertices, except the starting vertex
• Get their name property
Gremlin
47NoSQL Frankfurt , 9/28/2010Achim Friedland <achim@sones.de>
./outE[@label=‘created’]/inV/inE[@label=‘created’]/outV[g:except($_)]/@name
top related