node riak frank06

8/9/2019 Node Riak Frank06

1/49


2/49

Building a web-basedcollaborative e-readingplatform

J2EE veteran

Trumpetless wannabe trumpet player

@frank06 on the interwebs


3/49

Lets look back, shall we?


4/49

printf("What is your name?");

gets(string);printf("Your name is: %s", string);

Does this sound familiar?

...it is how we learn how to program


5/49

So then we go build our first webapp...

(with the first language we come across)


6/49

Lets imagine a bunch of people

trying to access your website(run that code) at the same time


7/49

Its like hosting a partyand provide only one toilet

hopechurchokc / flickr


8/49

L1 cache: 3 cycles

L2 cache: 14 cycles

RAM: 250 cycles

Disk: 41,000,000 cycles

Network: 240,000,000 cycles


9/49

L1 L2 RAM Disk Network


10/49

In other words, reaching RAM is like going from

here to the Red Light District.

Accessing the network is like going to the moon.


11/49

$result=mysql_query($query);

So... what is your program stilldoing while MySQL goes fetch

Neil Armstrong?


12/49


13/49

But we have threads!(to do other stuff while we wait)


14/49

Programming with threads

Race conditions

Coarse-grained locks more blocking

Fine-grained locks more complexity

Risk for deadlocks

Hard to reason about and therefore to getright

Context switching overhead


15/49

Programming with threads


16/49

A look at Apache and Nginx

http://blog.webfaction.com/a-little-holiday-present

Memory vs concurrent connections
http://blog.webfaction.com/a-little-holiday-presenthttp://blog.webfaction.com/a-little-holiday-present


17/49

A look at Apache and Nginx

1 thread per connection islimiting for massive concurrency

Apache uses 1 thread per connection

Nginx doesnt use threads

it runs an event loop

small memory allocation per connection


18/49

Especially relevant when dealing with I/O

All code runs in a single thread

No need for multithreading no locks!


19/49

window.onload=function() { alert("Apollo 11 landed!")}

Rings a bell?


20/49


21/49

Fortunately, Google has beendeveloping a brand new

Javascript VM for Chrome...

Its robust, blazingly fast, and open-source:

V8


22/49

A set of bindings to Googles V8 Javascript VM

A purely evented, non-blocking infrastructurethat makes it super simple to build highlyconcurrent programs

Ability to handle thousands of concurrentconnections with minimal overhead on asingle process


23/49

I love Ruby

I can use EventMachine for async

And, after all, my Rails app doesnt haveyouporns traffic

Why should I care?


24/49

Libraries like eventmachine will never betruly intuitive to use, because event-

driven I/O is enough of a fundamentalshift that it requires deep language

integration. Javascript, it turns out, is a

fundamentally event-driven languagebecause of its origins in the browser

Adam Wiggins Heroku


25/49

"Threads should be used by experts only"

Javascript is perfect for event loops with firstclass function objects and closures

It is arguably the most popular programming

languageFull Javascript stack?


26/49

$.ajax({url: '/api/feedme',

success: function(data) { $('.result').html(data);

}});

Client-side


27/49

var socket =newio.Socket('localhost');socket.connect();

socket.send('some data');

socket.addEvent('message', function(data){

$('.result').html(data);});

Client-side even better

WebSockets through Socket.IO-node


28/49

http.createServer(function (request, response) {

response.writeHead(200, {'Content-Type': 'text/html'});

response.end('I am back!');}).listen(8000);

Server-side node.js


29/49

Database

?


30/49

Content-agnostic key/value store

REST API embraces HTTP

Javascript Map/Reduce

Distributed, assume that failures will happen

Linearly scalable


31/49

throughput cost predictability

Both up and down; less headaches foroperations and development


32/49

db.save('astronauts', 'neil', {name: "Neil Armstrong",

retired: true,daysInSpace: 8,missions: ['Apollo 11', 'Gemini 8']

})()

riak-js http://github.com/frank06/riak-jsAvailable for node.js and jQuery
http://github.com/frank06/riak-jshttp://github.com/frank06/riak-js


33/49

varmap=function(v, keydata, args) {

var result = [],

doc = Riak.mapValuesJson(v)[0]

doc.missions.forEach(function(mission) {

if (mission.match(newRegExp

(args.mission))) result.push(doc)

})

return result;}

Map/Reduce jobs can be written in Javascriptand submitted via the HTTP interface


34/49

db.mapReduce({

inputs: "astronauts",

query: [{ map:

{ source: map,arg: { mission: "Apollo" } }

}]

})(function(results) { // ... });

Bring the computation to the data mapphases are executed in parallel

Aggregation happens in one node


35/49

db.get('astronauts', 'neil', {r: 2})(

function(neil) { $('.result').html(neil)

})

Tunable N/R/W values to tweak CAP behaviour

Eventual consistency: Brief sacrifices of consistency infailure conditions

Choose your own fault tolerance/performance tradeoff


36/49

Consistency: Reads and writes reflect a

globally consistent system state

Availability: System is available for readsand writes

Partition tolerance: System can handlethe failure of individual parts


37/49

No real-world data store can servecompletely consistent data while

being 100% available and handling

disconnected networks


38/49

Internet

NGINX

Rails Rails Rails Rails Node


39/49

Collaborative platform for studying

Tens of thousands of books alongside with user

generated content

Highlighting, note-taking, sharing

Web-based, use it anywhere: laptop, phone, iPad

HTML5 + Javascript + node.js + Riak (and Rails!)

Expect a beta release in October


40/49

Riak is developer- and ops-friendly: it scalesdown to your laptop as easily as up to a cluster

especially during exams period!Allows us to store multimedia assets just as JSON

Lucene-like search coming soon

Node is used for Ajax calls and WebSocket

Rails for the rest (its convenient and mature)


41/49

Cutting-edge technologies are not bug-free

Riak still has some rough edges (some in terms of

performance)

node is approaching its first stable version

Async JS code can get boomerang-shaped


42/49

db.save(bucket, doc, content)(function(response, meta) {db.get(bucket, doc)(function(response2) {

assert.equal(response2, content);db.remove(bucket, doc)(function() {db.get(bucket, doc)(null, function(r, meta3) {assert.equal(404, meta3.statusCode);db.get(bucket, other)(function() {

// ...

})});

});});

});

Boomerang-shaped code


43/49

Address boomerang-shaped code

Step( functionreadSelf() {

fs.readFile(__filename, this);}, functioncapitalize(err, text) { if (err) { throw err;

}

return text.toUpperCase();},

functionshowIt(err, newText) {sys.puts(newText);

});


44/49

is a new language inspired by Javascript and Ruby

grade: (student) ->

if student.excellent_work

"A+" elseif student.okay_stuff

if student.tried_hard then"B"else"B-"

else

"C"

eldest:if 24 > 21 then"Liz"else"Ike"


45/49

compiles down to Javascriptvar eldest, grade;

grade = function(student) { if (student.excellent_work) {

return "A+";

} elseif (student.okay_stuff) { if (student.tried_hard) { return"B";

} else { return"B-";

}} else {

return "C";}

};eldest = 24 > 21 ? "Liz" : "Ike";

And can be run on node!


46/49

Theres no doubt about it.

Javascript as a platform

is serious stuff.

Full JS stacks will becomemore and more popular.


47/49

Maybe its about timewe started teaching this?

puts("Enter your spacecraft:")gets(function(s) {

puts("Youre flying your " + s)

})


48/49


49/49

http://nodejs.org

http://howtonode.org

http://nodejs.org/cinco_de_node.pdf

http://www.slideshare.net/marcusf/nonblocking-io-event-loops-and-nodejs

http://dailyjs.com/2010/05/26/realie

http://wiki.basho.com

http://jashkenas.github.com/coffee-script/

http://widescript.com

http://franciscotreacy.com
http://widescript.com/http://jashkenas.github.com/coffee-script/http://wiki.basho.com/http://wiki.basho.com/http://dailyjs.com/2010/05/26/realiehttp://www.slideshare.net/marcusf/nonblocking-io-event-loops-and-nodejshttp://nodejs.org/cinco_de_node.pdfhttp://howtonode.org/http://nodejs.org/http://widescript.com/http://widescript.com/http://widescript.com/http://widescript.com/http://jashkenas.github.com/coffee-script/http://jashkenas.github.com/coffee-script/http://wiki.basho.com/http://wiki.basho.com/http://dailyjs.com/2010/05/26/realiehttp://dailyjs.com/2010/05/26/realiehttp://www.slideshare.net/marcusf/nonblocking-io-event-loops-and-nodejshttp://www.slideshare.net/marcusf/nonblocking-io-event-loops-and-nodejshttp://www.slideshare.net/marcusf/nonblocking-io-event-loops-and-nodejshttp://www.slideshare.net/marcusf/nonblocking-io-event-loops-and-nodejshttp://nodejs.org/cinco_de_node.pdfhttp://nodejs.org/cinco_de_node.pdfhttp://howtonode.org/http://howtonode.org/http://nodejs.org/http://nodejs.org/

node riak frank06

Documents