Redis – Memory is the new Disk

•Deepak Mittal• IntelliGrape Software

2

Agenda• What is Redis• Available clients• Data types• Operations on data types• Performance• Persistence• Sweet spots• Design considerations / Best practices• Adopters

3

About me?• Deepak Mittal• CEO of IntelliGrape Software• 13 years of Software development

experience• Interests : Performance tuning of web-

applications and teams

4

Poll • Familiarity with NoSQL? • Memcache?• Redis?

5

What is Redis

Key/value storelike

Memcached on steroids

6

What is Redis

In-memory NoSQL databasebacked by disk

7

What is Redis

Collection of data structures exposed

over the network

8

What is Redis

Not just Strings!

Keys can containStrings, Hashes, Lists, Sets

andSorted Sets

9

What is Redis

10

A Brief History of Redis• Started in 2009 by Salvatore Sanfillipo• VMWare hires Salvatore & Pieter Noordhuis• Financially supported and promoted by

VMWare

11

Redis Characteristics• Is single-threaded• Has queryble key namespace• Is extremely fast (~100 K

operations/second)• Is easy to install (no dependencies)• Is simple and flexible• Stores everything in memory• Written in ANSI C and BSD licensed (free,

open)

12

Language Support

Ruby, Python, PHP, Erlang,Tcl, Perl, Lua, Java, Scala,

Clojure, C#, C/C++,JavaScript/Node.js, Haskell,

IO, Go

13

Data Types• Strings (Integers)• Lists• Hashes• Sets• Sorted Sets

14

Thinking in Redis• This is not an RDBMS• Its all about the KEYS, and NOT the values• There is no querying on values• There are no indexes• There are no schemas

Operations on Strings/Integers• SET key value• GET key• MGET / MSET • INCR / DECR• INCRBY DECRBY • APPEND key value• SETNX key value• GETSET key value

16

Installation

$ wget http://redis.googlecode.com/files/redis-2.4.2.tar.gz

$ tar xzf redis-2.4.2.tar.gz

$ cd redis-2.4.2

$ make

$ ./src/redis-server

17

Atomicity of Commands• All of the built-in commands of Redis are

atomic• Commands can be chained together

(pipelined) to create complex atomic transactions

• Redis is single threaded and therefore, no locking is necessary

• In other words, commands like INCR won’t tread on each other’s toes coming from multiple clients simultaneously!

18

Key Expiration• When caching, we don’t want things to live

forever• Any item in Redis can be made to expire

after or at a certain time

EXPIRE my_key 60

EXPIREAT my_key 1293840000

Operations on Lists• LPUSH/RPUSH key value [value ...]

• LPOP/RPOP key

• LRANGE key start stop

• LSET key index value

• LLEN key

• LTRIM key start stop

• LINSERT key before|after pivot value

• BLPOP key [key ...] timeout

• LREM key count value19

Operations on Sets• SADD key member

• SPOP key

• SMEMBERS key

• SINTER key [key ...]

• SUNION key [key ...]

• SISMEMBER key member

• SCARD key

20

Operations on Sorted Sets• ZADD key score member [score] [member]

• ZCARD key

• ZCOUNT key min max

• ZINCRBY key increment member

• ZRANK key member

• ZRANGE key start stop [WITHSCORES]

• ZSCORE key member

21

Operations on Hashes• HSET key field value

• HGET key field

• HKEYS key

• HVALS key

• HLEN key

• HEXISTS key field

• HDEL key field [field ...]

22

Operations on Keys• TYPE key

• TTL key

• KEYS pattern

• EXPIRE key seconds

• EXPIREAT key timestamp

• EXISTS key

• DEL key

23

Redis Administration Commands• MONITOR• SAVE / BGSAVE• INFO• DBSIZE• FLUSHALL• CONFIG SET parameter value

24

25

Performance

We added two million items into a list in 1.28 seconds, with a networking layer between

us and the server

Salvatore Sanfilippo, Creator

26

Performance• Almost same for read and write, handles

writes faster than read• 100 K ops/second • Performance depends a lot on – Hardware– Persistence configuration – Operation complexity– Payload size– Batch size (pipelining)

27

Redis-benchmark

28

Demo• Redis CLI• Getting Help• Operations on Strings• Key Expiry• Operations on Lists, Set• Groovy program operating on Sorted

Set and Hash• MONITOR / INFO / DBSIZE / TYPE• Redis benchmark

29

Data Durability

Dump data to disk after certain conditions are met

OR

Manually

AND/OR

Append Only File (AOF)

30

Advantages of Persistence• On server restart, Redis re-populates its

cache from files stored on disk• No issue of “cache warm-up”• It is possible to back-up and replicate the

files/cache

31

Memory Requirements?• Depends upon the payload size, and the

data-type used • Instagram reports that they stored 300

million key-value pairs in about 5 GB of memory => 16 MB/million pairs (they also report that Memcache requires 52 MB for the same million keys)

32

Sweet Spots / Use-cases• Cache Server• Stats collection• Tag Cloud• Most read / shared articles• Share state between processes / Session

Tokens• Auto-completion• API rate limiting• Activity Feeds• Job Queue (LPUSH & RPOP)• URL Shortener mapping

33

Case Studies• Video Marketing Platform– Real-time analytics– Stats for usage reports to clients

• Content Publishing application– Tag Cloud – Top Read Articles– URL Shortening information

• Social Media Marketing application– Daily, Monthly and overall stats collection– List of online users

34

Design Considerations / Best Practices• Avoid excessively long keys• Use consistent key namespace

nomenclature• Have human readable keys• Since there is no querying on values, keys

should be thought-out before hand• Store data based on use-cases. Ad-hoc

queries can-be very expensive • Multiple databases are useful for

segmenting key namespaces, especially where key queries are needed

35

Design Considerations / Best Practices• All data must fit in memory• Data duplication/redundancy is OK• Persistence strategy should be chosen

based on the kind of data that you are storing and performance trade-off must be understood

• Good for write-heavy, frequently changing and non-relational data

• Polyglot-persistence is a smart choice• Start by using Redis to augment, and then

replace

36

Some Adopters• Stackoverflow• Craiglist• Github• Digg• Instagram• Blizzard Entertainment

37

Learning Resources• http://redis.io/commands• http://try.redis-db.org• Redis CLI

38

Advanced Features• Master-slave replication• Pub / Sub• Transactions• Using multiple databases• Commands pipelining• Clustering / Sharding• LUA scripting

39

What is Redis• Simple & Easy to use (Key / Value, data-

types map to programming languages)• Flexible (Supports commonly used data-

types• Fast (100 Kops / second)• Powerful (data-types, operations on data-

types)• Safe (persistence)• Easy to use (various client libraries)• Widely adopted (Digg, Craiglist, Github …)

40

Discussion

Now that you understand Redis and its capabilities, what use-cases do come to

your mind?

What are your concerns around usage of Redis?