nosql @ codemash 2010
DESCRIPTION
Overview of the NoSQL movement, given at CodeMash 2010TRANSCRIPT
bensco'ield – viget labscodemash14 january 2010
NoSQLDeath to Relational Databases(?)
1
Motivations
2
Performance
3
Scalability
4
Meh
5
Flexibility
6
Complexity
7
“Comics” Is Hard
8
9
Functionality
10
Charlie Chaplin
Jet Li
Marian Collier
Hank Mann
11
Taxonomy
12
Key-‐Value Stores
13
distributed hash tables
14
PerformanceScalabilityFlexibilityComplexityFunctionality
highhighhighnonevariable (none)
15
DynamoGT.MPStoreRedis
16
Column-‐Oriented Stores
17
semi-‐structured
18
PerformanceScalabilityFlexibilityComplexityFunctionality
highhighmoderatelowminimal
19
BigTableCassandraHBase
20
Document-‐Oriented Stores
21
also semi-‐structured
22
PerformanceScalabilityFlexibilityComplexityFunctionality
highvariable (high)highlowvariable (low)
23
CouchDBMongoDBRDDBRiak
24
Graph Databases
25
graph theory
26
PerformanceScalabilityFlexibilityComplexityFunctionality
variablevariablehighhighgraph theory
27
ActiveRDF AllegroGraphNeo4J
28
Relational Databases
29
PerformanceScalabilityFlexibilityComplexityFunctionality
variablevariablelowmoderaterelational algebra
30
Examples
31
Redis
32
Data Typesstringslistssetssorted sets
33
In-‐Memoryperiodic snapshots / append only 'ilemaster-‐slave replicationmemory-‐bound
34
require 'redis'
gl = Redis.new
# A stringgl['name'] = 'Kyle Rayner'gl['name']gl.delete('name')
# A listgl.push_tail 'to-dos', 'Lose Ion power'gl.push_tail 'to-dos', 'Mourn dead loved ones'gl.push_tail 'to-dos', 'Blow up zombie lanterns'
gl.list_range('to-dos', 0, -1)
35
Tokyo Cabinet
36
Data Typesbinary datastrings
37
Tables!?
38
Related Projectstyrantdystopiapromenade
39
require 'rufus/tokyo'
# Key-valuejli = Rufus::Tokyo::Cabinet.new('jl.tch')jli['members'] = [ 'Batman', 'Black Canary', 'Blue Beetle', 'Captain Marvel', 'Doctor Light', 'Doctor Fate', 'Guy Gardner', 'Martian Manhunter', 'Mister Miracle'].to_yaml
YAML.load(jli['members'])
40
require 'rufus/tokyo'
# Tablebig7 = Rufus::Tokyo::Table.new('big7.tct')
big7['s'] = {'name' => 'Superman', 'role' => 'deus ex machina'}big7['b'] = {'name' => 'Batman', 'role' => 'mastermind'}big7['gl'] = {'name' => 'Green Lantern', 'role' => 'space cop'}big7['f'] = {'name' => 'Flash', 'role' => 'speedster'}big7['mm'] = {'name' => 'Martian Manhunter', 'role' => '?'}big7['ww'] = {'name' => 'Wonder Woman', 'role' => 'hitter'}big7['a'] = {'name' => 'Aquaman', 'role' => 'fish-talking'}
big7.query {|q| q.add_condition 'role', :streq, 'fish-talking'}
41
Cassandra
42
GenealogyDynamoBigTable
43
Column-‐Orientedcolumnssupercolumnscolumn families
44
Distributedautomatic replicationeventual consistencyeasy expansion
45
Availabilityweak readsquorum reads
46
require 'cassandra'
op = Cassandra.new('OnePiece')
op.insert(:People, '1', {'name' => 'Luffy'})op.insert(:People, '2', {'name' => 'Crocodile'})op.insert(:People, '3', {'name' => 'Mr. 3'})
op.insert(:Fights, '1', {'opponents' => {UUID.new => '2'}})op.insert(:Fights, '1', {'opponents' => {UUID.new => '3'}})
luffy_fights = op.get(:Fights, '1', 'opponents')luffy_fights.map {|t, opp| op.get(:People, opp, 'name')}
47
CouchDB
48
Web-‐InspiredJSON storageHTTP / RESTful interface
49
Viewsprede'ined, updated incrementallyjavascript for map/reduce
50
Updatesfull, including embedded documents
51
require 'couchrest'
konoha = CouchRest.database!('http://127.0.0.1:5984/konoha')naruto = konoha.save_doc { 'name' => 'Naruto Uzumaki', 'chakra' => 'wind'}shikamaru = konoha.save_doc { 'name' => 'Shikamaru Nara', 'chunin' => true}
konoha.save_doc { '_id' => '_design/first', :views => { :chunin => { :map => 'function(doc){if(doc.chunin){emit(null, doc);}}' } }}
puts konoha.views('first/chunin')['rows'].inspect
52
MongoDB
53
Storagebinary JSON documents
54
Accessnative clients
55
Queriesdynamicindex-‐based
56
Updatesallows partial updates
57
require 'mongo'
avengers = Mongo::Connection.new.db('avengers')members = avengers.collection('members')
members.insert {'name' => 'Ant-Man'}members.insert {'name' => 'Hulk'}members.insert {'name' => 'Iron Man'}members.insert {'name' => 'Thor'}members.insert {'name' => 'Wasp'}
members.create_index('name')
pym = members.find {'name' => 'Ant-Man'}pym['name'] = 'Giant-Man'pym.save
members.remove {'name' => 'Hulk'}
members.insert {'name' => 'Captain America'}
58
Riak
59
also Web-‐InspiredJSON storageHTTP / RESTful interfacelinks for relationships
60
Decentralizedno privileged nodes
61
Con'igurablestore / read / write
62
require 'jiak'
jc = JiakClient.new('127.0.0.1', 8098)jc.set_bucket_schema('supervillains', { 'allowed_fields' => ['name', 'alias', 'power']})
jc.store({ 'bucket' => 'supervillains', 'key' => 'Normie', 'object' => { 'name' => 'Norman Osborn', 'alias' => 'Green Goblin', 'power' => 'Supreme jerkfacedness' }, 'links' => []})
kth = jc.fetch('supervillains', 'Normie')
63
Neo4J
64
Structurenodes and edgeskey-‐value pairs
65
Querieslucenegremlin
66
require 'neo4j'
class Person include Neo4j::NodeMixin property :name, :mutant index :name, :mutant has_n :crushes has_n :hookups has_n :marriages def initialize(name, mutant = true) name = name mutant = mutant endend
67
Neo4j::Transaction.run do magneto = Person.new('Magneto') esme = Person.new('Esme') rogue = Person.new('Rogue') magda = Person.new('Magda', false) wasp = Person.new('Wasp', false) magneto.crushes << wasp magneto.hookups << rogue magneto.marriages << magda esme.crushes << magneto rogue.hookups << magneto magda.marriages << magnetoend
68
magneto = Person.find(:name => 'Magneto')
# Who likes Magneto?magneto.relationships.incoming(:crushes).nodes
# Which non-mutants has Magneto dated?magneto.hookups{ !mutant? }.to_a
69
Simulations
70
Structure
71
people{ ‘name’:‘Jimmy Olsen’ ‘title’:‘Superman’s Pal’ ‘company_id’:12441}
companies{ _id:12441 ‘name’:‘Daily Planet’}
72
Lack of Structure
73
mysql> SELECT * FROM people LIMIT 1 \G*************************** 1. row *************************** id: 1content: --- company: Daily Planetname: Jimmy Olsentitle: Superman’s Pal
74
But wait!friendfeedfriendly
75
mysql> desc people;+-------+-------------+------+-----+---------+-------+| Field | Type | Null | Key | Default | Extra |+-------+-------------+------+-----+---------+-------+| id | int(11) | YES | | NULL | || name | varchar(50) | YES | | NULL | |+-------+-------------+------+-----+---------+-------+
mysql> desc attributes;+-----------+--------------+------+-----+---------+-------+| Field | Type | Null | Key | Default | Extra |+-----------+--------------+------+-----+---------+-------+| id | int(11) | YES | | NULL | || person_id | int(11) | YES | | NULL | || attribute | varchar(50) | YES | | NULL | || value | varchar(100) | YES | | NULL | |+-----------+--------------+------+-----+---------+-------+
76
Not Only SQL
77
Polyglot Persistence
78
Caching
79
Already in Usememcached
80
Queues
81
Long-‐running processesresque
82
Logging
83
Rails Log Replacementhttp://github.com/peburrows/mongo_db_logger
84
Hybrid Domains
85
different domains
86
Publishinge-‐commercedocuments
87
Datinge-‐commercesocial graph
88
different scales
89
Photo Sharinguser accountsuploaded photos
90
Next Steps
91
Explore
92
Database Listhttp://internetmindmap.com/database_software
NoSQL Google Grouphttp://groups.google.com/group/nosql-‐discussion
93
Ignore the Database
94
Logical Modeling Firstbe mindful
95
Change the Default
96
Application Templatesstart with something new
97
@bsco'ieldben.sco'[email protected]://spkr8.com/bsco'ieldhttp://viget.com/extendhttp: //bensco'ield.com
bensco'ield
98