riak - from small to large
Post on 08-May-2015
5.167 Views
Preview:
DESCRIPTION
TRANSCRIPT
Rusty Klophaus (@rklophaus)Basho Technologies
Riak from Small to Large
Strange Loop · October 2010
Riak is a Dynamo-inspired,open-sourced, key/value datastorebuilt to scale predictably and easily.
2
Riak is a Dynamo-inspired,open-sourced, key/value datastorebuilt to scale predictably and easily.
3
Your ops guy: calm, relaxed, and wearing a party hat.
NoSQL
4
5http://www.flickr.com/photos/ghewgill/129950124
NoSQL has changed like Keanu hasn’t.
6http://www.youtube.com/watch?v=nEubt6HpGhs
Companies, Funding, Customers, Awareness...
7http://www.flickr.com/photos/free-stock/4791385567
“NoSQL? Yeah, it’s this really cool technology, you probably haven’t heard of it yet.”
8http://www.flickr.com/photos/slava/285199203
“Johnson! Get me a memo on this ‘No sequal’ thing!”
9http://www.flickr.com/photos/proimos/4045973322
Why?
10
Because NoSQL is driven by narcissism, voyeurism, and
materialism.
The catalyst for NoSQL is the data
generated or consumedby humans.
12
13
Photos, music, and video...
14
Constantly connected devices...
15http://www.flickr.com/photos/hanspoldoja/5001818922
Social Networks...
16http://www.flickr.com/photos/mega/7358278
Your Grandmama...
Requirements That Confound an RDBMS
17
Large Data ObjectsBLOBs?
High-Availability SystemsBi-Directional Replication? Log shipping?
High-Capacity SystemsShards?
Vertical Partitioning?
Changing SchemasAuto-generated DDL?
Generic Columns?
Non-Standard AccessRecursive SQL?
The roots of NoSQL are practical, not academic.
18
Scratch an Itch Build a Company
Turning Point
19
➮
Four Important Things
20
1. Broad KnowledgeLearn from the past, both DB *and* non-DB worlds.
• VP Technical Services, Akamai • Senior Architect, Akamai• Chief of Operations, Snapfish• Release Manager, Engine Yard• Bob Ippolito (CTO/CoFounder MochiMedia)• Dr. Eric Brewer (CAP Theorum)
Four Important Things
21
2. Strong CommunityStay in touch with real problems.
Mark PhillipsBasho Community Manager
Four Important Things
22
3. AgilityEvolve to solve new problems.
Four Important Things
23
4. RestraintNo solution can do it all well.
Back to Riak
24
What characteristics of Riak become important at different cluster sizes?
25
Single Box Riak
26
Simplicity
Single Box RiakSimple Key/Value Store, Flexible Schema
27
client = RiakClient(host, port)bucket = client.bucket('mybucket')
obj = bucket.new('foo', 'bar')obj.store()
obj2 = bucket.get(‘foo’)
Clients in Many Languages
Single Box Riak
28
Predictable Development Interface
Single Box Riak
29
Riak
Riak Riak
RiakRiak
Riak
===
Development Production
Configurable Buckets
Single Box Riak
30
!=
Profile
Bucket
Audio
Bucket
Links (Lightweight Data Relations)
Single Box Riak
31
Profile Projects Comments
Small Riak Cluster
32
Parallelism and Power
Small Riak ClusterExpanding the Cluster
33
bin/riak-admin join riak@hostname
Small Riak ClusterExpanding the Cluster
34
bin/riak-admin join riak@hostname
Small Riak ClusterExpanding the Cluster
34
bin/riak-admin join riak@hostname
Hello ladies, can your datastore grow like mine?
*monocle smile*
Small Riak ClusterDistributed Queries
35
Small Riak ClusterDistributed Queries
36
Small Riak ClusterJavascript-Based Map/Reduce
37
Keys
Map
Map
Map
Map
Map
Reduce
Reduce
Reduce
...
Output
Small Riak ClusterWell-Behaved HTTP
38
Riak Riak
RiakRiak
Riak
nginx
Squid
ClientClient
Client
Browser
nginx proxy config - http://gist.github.com/323048
Small Riak ClusterRiak Full-Text Search
39nginx proxy config - http://gist.github.com/323048
bin/search-cmd install mybucket
Riak Riak
RiakRiak
Riak
"converse AND category:shoes"
Large Riak Cluster
40
Operations
No Special Nodes
Large Riak Cluster
41
SpecialNodes
Laugh at Machine Failure
Large Riak Cluster
42
Riak Riak
RiakRiak
RiakX
Scale by Adding Nodes
Large Riak Cluster
43
Riak Riak
RiakRiak
Riak
Riak
Riak
Riak
Riak
Riak
Riak
Riak
Packages or Self-Contained Installation
Large Riak Cluster
44
sudo dpkg -i riak_0.13.0-2_amd64.deb
-or-
unzip riak.zipriak/bin/riak start
$0.00
$0.00Order now, and we’ll include an
extra Riak for free!*
*Offer limited to open-source editions of Riak.
• On-Call Support 24x7x365 • Management Tools• SNMP Monitoring• Multi-site Replication
Enterprise ($$$ / ~10’s of boxes)
47
• On-Call Support 24x7x365• Management Tools • SNMP Monitoring• Multi-site Replication
Enterprise ($$$ / ~10’s of boxes)
48
Thanks! Questions?Next Steps
Read more at http://wiki.basho.com
Download binaries from http://downloads.basho.com
Get source code from http://hg.basho.com
Join the public mailing list at riak-users@lists.basho.com
Mailing list archives at http://riak.markmail.org
Thanks!
Rusty Klophaus (@rklophaus)
Mark Phillips (@pharkmillups)
49
Additional Slides
TutorialGet Riak
Connect with a Client
Store Data
Store an Object with Links
Linkwalking
Map/Reduce
51
Get RiakDownloadhttp://downloads.basho.com/riak
Start Riakcd riakbin/riak start
52
Connect with Python# Code is at http://hg.basho.comimport riak
# Connectclient = riak.RiakClient('127.0.0.1', 8098)
53
Store Datamybucket = client.bucket('mybucket')
# Create an object...obj = mybucket.new('myobject')obj.set_data({ 'foo' : 1, 'bar' : 2 })obj.store()
# Read the object...obj = mybucket.get('myobject')print obj.get_data()
# Or, open a web browser...http://127.0.0.1:8098/riak/mybucket/myobject
54
Store an Object with Linksbands = client.bucket('bands')albums = client.bucket('albums')members = client.bucket('members')
# Store a band, link to album and members...obj = bands.new('Winger') \ .add_link(albums.new('Pull', 1275922).store()) \ .add_link(albums.new('IV', 542731).store()) \ .add_link(albums.new('Karma', 200170).store()) \ .add_link(members.new('Kip Winger').store()) \ .add_link(members.new('Reb Beach').store()) \ .add_link(members.new('John Roth').store()) \ .add_link(members.new('Rod M.').store()) \ .store()
55
Linkwalking# Get the albums...albums = obj.link('albums').run()
# Get the songs (assumes data is present)...songs = obj.link('albums').link('songs').run()
# Get the members...members = riak.MapReduce(client) \ .add('bands', 'Winger') \ .link('members') \ .run()
56
Map/Reduce# Count the number of sales...result = obj \ .link('albums') \ .map("function(v) { return [v.values[0].data]; }") \ .reduce("Riak.reduceSum") \ .run()
57
top related