geekcamp sg 2009 - couchapps with couchdb
DESCRIPTION
Introduction to CouchDB and CouchApps. Example of a Rails + CouchApp hybrid built called i.canhazthread.comTRANSCRIPT
#geekcampsg 2009CouchApps - Applications built on CouchDB
Arun Thampi@iamclovin
http://mclov.in
Saturday, August 22, 2009
About myself
• Ruby Developer + Dabbler at Wego.com
• The kick-ass travel meta-search engine
• Work with great set of people
@chuyeow@winstonyw @reds @mongeslani
@jefflimar
@phungleson @garytheis@ijuzwannatwit
Saturday, August 22, 2009
CouchDB FTW• Schema-less Key-Value based
document store
• Erlang
• Speaks HTTP / REST
• Map Reduce
• JSON
• Social Media
• MVCC (No Locks)
• Replication
• Scaling out of the box (HAProxy/Nginx)
• Full Text Search (Lucene/Sphinx)
• Attachments (binary files)
• Web Interface (Futon)
Saturday, August 22, 2009
Profit
Saturday, August 22, 2009
CouchDB - Built for the Web
{ “_id” : “abc”, “_rev” : “abc123def”, “text” : “Hello World”, “screen_name” : “mclovin”, “timestamp” : 123232, “type” : “thread”, “replies”: [ “reply 1”, “reply 2”, “reply 3”]}
Saturday, August 22, 2009
CouchDB - Built for the webInserting a document HTTP PUT
Retrieving a document HTTP GET
Deleting a document HTTP DELETE
Updating a document HTTP POST
Scaling Replication + load balancing
Querying ?
Saturday, August 22, 2009
Saturday, August 22, 2009
Map Reduce with Javascript Views(No I wasn’t making that up)
Document 1
Document 2
Document n
Map Function
[Key 1, Value 1]
Reduce Function[Key 2, Value 2]
[Key k, Value k]
Value X
Saturday, August 22, 2009
Example Map Reduce
function(doc) { if(doc.type == “customer”) { emit(doc.first_name, 1); }}
Map Function
Saturday, August 22, 2009
Example Map Reduce (contd)Reduce Function
function(keys, values, re) { return sum(values);}
Saturday, August 22, 2009
Example Map Reduce (contd){ .. { “key”: “Seth”, “value”: 1 }, { “key”: “Fogell”, “value”: 1 } ..}
reduce([“Seth”, “Fogell”], [1, 1])
Saturday, August 22, 2009
CouchApps
• HTTP / HTML / Javascript is the language of the web
• Coincidentally, that’s what CouchDB speaks too
• Why not have CouchDB apps serve apps?
Saturday, August 22, 2009
CouchApps - Why?
• Barriers to entry to develop powerful data-backed applications is lowered
• Replication enables load-balancing + scaling
• No Impedance Mismatch - Data is Javascript, your views are in Javascript, your HTML pages are rendered in Javascript
Saturday, August 22, 2009
CouchApps
• Available at http://github.com/couchapp/couchapp
• couchapp generate icanhazthread
• couchapp push icanhazthread http://127.0.0.1:5984/icanhazthread
Saturday, August 22, 2009
CouchApps - List & Show Functions
• A view is queried and the results are passed to a list function
• List Function then renders the results of your view in whatever format you please: HTML, JSON, XML, RSS, etc
• In a twisted way, its like a controller + view concept in an MVC framework
Model Schema-less JSON
Controller Map/Reduce View
View List/Show
Saturday, August 22, 2009
Introducing i.canhazthread.com
• Inspired (in fact, cloned) from a.tinythread.com by Joshua Schacter
• Backend powered by a CouchApp (http://github.com/arunthampi/icanhazthread)
• Uses Rails to do Twitter authentication + HTML rendering
Saturday, August 22, 2009
icanhazthread.com
nginx
Rails Mongrels CouchDB
CouchDB API [RSS, Realtime]
Saturday, August 22, 2009
Document Structure
{ “_id” : “abc”, “_rev” : “abc123def”, “text” : “Hello World”, “screen_name” : “mclovin”, “timestamp” : 123232, “type” : “thread”}
{ “_id” : “def”, “_rev” : “234abcdef”, “text” : “oh hai”, “screen_name” : “snehamenon”, “timestamp” : 1232310, “type” : “reply”, “thread_id” : “abc”}
Thread Reply
Saturday, August 22, 2009
Interesting Map/Reduce problem(at least for me)
• Interesting because replies are not contained within a thread
• Also, because threads on the homepage need to be sorted according to timestamp (newest ones first)
• Views need to be commutative & associative
Saturday, August 22, 2009
Show me the code
function(doc) { if(doc.type == ‘thread’) { emit([doc.id, doc.time_created], doc); } else if(doc.type == ‘reply’) { emit([doc.thread_id, doc.time_created], doc); }}
Map Function (yawn)
Saturday, August 22, 2009
Show me the code
[K1, V1] [K2, V2]
[K1, RV1]
[K1, RV2]
group_level =1
[K1, Summary for Thread 1]
[K2, Summary for Thread 2]
num_replies: 2
num_replies: 0
Sorting?
Thread Reply Result
Reduce Function
Saturday, August 22, 2009
Future Directions
• Make icanhazthread real-time (Utilize the _changes API in CouchDB)
• RSS Feeds (Dead simple with the CouchApp list)
• etc etc
Saturday, August 22, 2009
Resources
• CouchDB Wiki: http://wiki.couchdb.org
• CouchDB Mailing lists (user + dev)
• Blogs: Chris Anderson, Harish Mallipeddi, Jan Lenhardt
• CouchDB Futon Test Suite (http://127.0.0.1:5984/_utils/couch_tests.html)
• Other CouchApps: Toast, Chris Anderson’s blog
Saturday, August 22, 2009
Thank you
Q & A
Saturday, August 22, 2009