redis - a trial by fire

Fred Oliveira@f on twitter and

fred@webreakstuff.com on that older thing

A trial by fire. Also, probably myfirst presentation all set in a

monospaced font.

$ mkdir redis && cd redis$ wget http://6rq8.sl.pt$ tar -zxvpf redis-2.0.4.tar.gz$ cd redis-2.0.4$ make

# now give it a minute

$ whatis redis

$ whatis redisredis is an advanced, fast, persistent key-value database

$ whatis redisredis is an advanced, fast, persistent key-value database, developed by Salvatore Sanfilippo (@antirez)

$ whatis redisredis is an advanced, fast, persistent key-value database, developed by Salvatore Sanfilippo (@antirez), who was recently hired by VMWare (to keep working on redis).

$ whereis redis

$ whereis redisplaces like github, engineyard, forrst, craigslist

$ whereis redisplaces like github, engineyard, forrst, craigslist, and (I’ll assume) a bunch of places we won’t hear about.

$ whereis redisplaces like github, engineyard, forrst, craigslist, and (I’ll assume) a bunch of places we won’t hear about.

(mostly because regardless of how sexy you think a key-value storage engine is, not a lot of people talk about what they actually use.)

$ ./redis-cliredis> set currentslide “What’s it good for?”OK

“I see Redis definitely more as a flexible tool than as a solution specialized to solve a specific problem: his mixed soul of cache, store, and messaging server shows this very well.” - Salvatore Sanfilippo

Possible use cases:

* Caching mechanism (as a substitute for memcached that doesn’t just store blobs)* Job management, queueing systems (github’s resque or Octobot are two good examples)* IPC (easy to setup up a message passing system, locally or over a network)* Locking mechanisms (with commands like setnx and blpop/brpop)* Session/Token data store

At WBS, we built a node+redis-based chat server for a client. Super fast and easily scalable. Also, of trivial implementation.

Then we built a realtime node/redis/WS based voting system. Pain points: actually, the only pain point were browsers that didn’t support Websockets.

It is very likely that we end up porting many of our current support processes (queues, background job management, caching) for our products to Redis, mostly for its speed and flexibility.

$ ./redis-cliredis> set currentslide “Crash course!”OK

$ ./redis-cliredis> set currentslide “Redis Data Types”OK

first, a bit about keys

* binary-safe strings* stay away from using whitespace* object:property is a good convention* SHA hashes are great keys

examples:

* user:name* lastmessage* 464A1E96B2D217EBE87449FA8B70E6C7D112560C

redis supports:

* strings* lists* sets (+ sorted sets)* hashes

* strings

set rickroll “never gonna give you up”OKget rickroll“never gonna give you up”

* strings

set count 10 <-- set key OKget count <-- get key“10”setnx count 9999 <-- set if doesn’t exist(integer) 0incr count <-- increment key (returns integer)(integer) 11del count <-- deletes the key(integer) 1expire count 120 <-- key TTL in secs(integer) 1

redis supports:

* strings* lists* sets (+ sorted sets)* hashes

redis supports:

* strings* lists <-- ordered list of binary-safe strings. implemented on top of the idea for a linked list (not an array)* sets (+ sorted sets)* hashes

redis supports:

* strings* lists <-- ordered list of binary-safe strings. implemented on top of the idea for a linked list (not an array) <-- getting an element by index is slow, but adding to edges is super fast (as it should in a db).* sets (+ sorted sets)* hashes

* lists

rpush nevergonna “let you down” <-- push right(integer 1)rpush nevergonna “run around and desert you” (integer 2)rpush nevergonna “make you cry”(integer 3)rpush nevergonna “say goodbye”(integer 4)rpush nevergonna “tell a lie and hurt you”(integer 5)lpush nevergonna “give you up” <-- push left(integer 6)

* lists (indexes)

[“a”, “b”, “c”, “d”, “e”]

* lists (indexes)

[“a”, “b”, “c”, “d”, “e”] 0 1 2 3 4

* lists (indexes)

[“a”, “b”, “c”, “d”, “e”] 0 1 2 3 4 -5 -4 -3 -2 -1

* lists

rpush nevergonna “let you down” <-- push right(integer 1)rpush nevergonna “run around and desert you” (integer 2)rpush nevergonna “make you cry”(integer 3)rpush nevergonna “say goodbye”(integer 4)rpush nevergonna “tell a lie and hurt you”(integer 5)lpush nevergonna “give you up” <-- push left(integer 6)

* lists

lrange 0 2 <-- 3 first elements1. “give you up” 2. “let you down”3. “run around and desert you”

lrange 0 -1 <-- first to last1. “give you up” 2. “let you down”3. “run around and desert you”4. “make you cry”5. “say goodbye”6. “tell a lie and hurt you”

* lists

llen nevergonna <-- list length(integer 6) lpop nevergonna <-- pop from left“give you up” lpop nevergonna“let you down”rpop nevergonna <-- pop from right“tell a lie and hurt you”

* lists

lindex nevergonna 0 <-- element at index“give you up”ltrim nevergonna 0 2 <-- trims list to [0,1,2]OK

other commands:lrem <-- removes x instances of a valueblpop/brpop <-- blocking pop operation. if list is empty, waits for another client to lpush or rpush.sort <-- sorts a list given certain criteria (sorts numeric lists in ASC by default)rpoplpush <-- removes from right, adds to left

redis supports:

* strings* lists* sets (+ sorted sets)* hashes

redis supports:

* strings* lists* sets (+ sorted sets) <-- unordered collection of binary-safe strings. no duplicate elements in sets.* hashes

redis supports:

* strings* lists* sets (+ sorted sets) <-- unordered collection of binary-safe strings. no duplicate elements in sets. <-- you can do interesting things like intersections, unions and verifying if a key is a member of a set.* hashes

* sets

sadd beatles “george” <-- add georgesadd beatles “paul” <-- add paulsadd beatles “john” <-- add johnsadd beatles “ringo” <-- add ringo

smembers beatles <-- list set members1. “ringo”2. “john”3. “george”4. “paul”

* sets

sismember beatles “john” <-- check presence(integer) 1sismember beatles “yoko”(integer) 0

* sets

implementing things like tags becomes trivial:

sadd taggedwith:design “@jasonsantamaria”sadd taggedwith:design “@zeldman”sadd taggedwith:design “@f”

sadd taggedwith:dev “@ezra”sadd taggedwith:dev “@dhh”sadd taggedwith:dev “@f”

sinter taggedwith:design taggedwith:dev^ intersect both sets1. "@f"

* (sorted) sets

sorted sets are much like regular sets, except each one member has an associated score. Redis uses the score to sort between the elements in the set.

zadd pleague 28 “FC Porto”zadd pleague 18 “V. Guimarães”zadd pleague 18 “Benfica”zadd pleague 16 “Nacional”zadd pleague 15 “Académica”

* (sorted) sets

zrank pleague “FC Porto” <-- rank, low to high(integer) 4zrevrank pleague "FC Porto" <-- high to low(integer) 0

zrange pleague 0 2 <-- returns 1st to 3rd1. "Academica"2. "Nacional"3. "Benfica"zrevrange pleague 0 2 <-- high to low1. "FC Porto"2. (...)

* (sorted) sets

zcard pleague <-- return cardinalityzscore pleague “Benfica” <-- return the scorezcount pleague 16 18 <-- returns the number of elements with score between 16 and 18zremrangebyrank <-- remove by rank rangezremrangebyscore <-- remove by rank score

redis supports:

* strings* lists* sets (+ sorted sets)* hashes

* hashes

useful in the case of web applications, where storing objects is important. Implementation is on top of a hash table.

Small hashes, however (with a limited number of fields and size for values) are implemented differently in order to be more memory efficient (to compensate for the overhead of a traditional hash-table implementation).

* hashes

hset u:fred name “fred”hset u:fred age “27”hset u:fred email “fred@webreakstuff.com”

hkeys u:fred <-- return all the keyshvals u:fred <-- return all the valueshgetall u:fred <-- key(1), value(1)..key(n)..

hincr u:fred age 1 <-- age becomes age++

$ ./redis-cliredis> set currentslide “Publish/Subscribe”OK

* pub/sub

Redis natively supports the publish/subscribe messaging pattern. In the pattern, publishers don’t know about subscribers. Subscribers express interest in channels and receive messages in those channels, with no knowledge of publishers.

* pub/sub

subscribe messagesReading messages... (press Ctrl-c to quit)

elsewhere:publish messages “hello world”(integer) 1

back in the first client:1. “message”2. “messages”3. “hello world”

* pub/sub

possible use cases:* alert systems* chat functionality* (insert awesomeness)

$ ./redis-cliredis> set currentslide “A few considerations”OK

* While the primitives in Redis are pretty fast, how fast your system actually performs boils down to how you design the way you store data.

* Remember there’s no guarantee that data has been written to disk when a command returns.

* Always design based on how you query.

* Also, you’ll be building your indexes manually. This means your data will be replicated in a few places. What they taught in your DB classes may not apply. De-normalization is common, natural and necessary here.

Most previous considerations apply to NoSQL in general. Relational data makes sense in relational databases, whereas non-relational data is perfect for NoSQL. Don’t treat NoSQL in general or Redis in particular as a silver bullet.

(although Redis is pretty damn shiny)

$ ./redis-cliredis> set currentslide “Finally,”OK

This talk didn’t focus on:

* Scaling redis (trivial with slaveof)* How fast it is (pretty speedy)* Comparisons between Redis and key value stores (because that would be flamebait)

Also, it didn’t talk about

* Clustering Redis (Salvatore himself will talk about that tomorrow at 3PM on Stage C. He’s a smart dude, you should go listen to what he has to say)

...and it didn’t mention,

* Your brain! (I’m going to be speaking about that sexy beast between your ears tomorrow at 2PM on Stage C. I like to pretend I’m a smart dude, you should probably go listen to what I have to say)

redis> exit>> fred@oreo [~/Projects/redis-2.0.4] $ _

# thank you!# email me at fred@webreakstuff.com# or follow me on twitter - @f