redis training for java software engineers

© All rights reserved: Moshe Kaplan© All rights reserved: Moshe Kaplan Redis for Java Software Engineers© All rights reserved: Moshe Kaplan© All rights reserved: Moshe Kaplan Redis for Java Software Engineers

Redisfor Java Software Engineers

Copyrights © Moshe [email protected]


Redisfor Java Software Engineers

Moshe KaplanScale Hacker

http://top-performance.blogspot.comhttp://blogs.microsoft.co.il/vprnd

© All rights reserved: Moshe Kaplan© All rights reserved: Moshe Kaplan Redis for Java Software Engineers

About Me: It’s all About

3

Scale


NOSQL MARKETIntroduction

4


When Should I Choose NoSQL?

• Eventually Consistent

• Document Store

• Key Value

5

http://guyharrison.squarespace.com/blog/tag/nosql


Key Value Store

• insert

• get

• multiget

• remove

• truncate

6

<Key, Value>http://wiki.apache.org/cassandra/API


Redis

• Very simple protocol (SMTP like)

• Amazing Performance (60Kqps ops on 1 CPU machine)

• Persistency to disk

• Very little security


HELLO. MY NAME IS REDISIntroduction

8


9

#10 Most Popular DB Engine

http://db-engines.com/en/ranking


Who is Using Redis?


Who is Behind Redis


Installation: Give Yourself 5min• sudo apt-get install -y build-essential tcl8.5

• wget http://download.redis.io/releases/redis-stable.tar.gz

• tar xzf redis-stable.tar.gz

• cd redis-stable

• make

• sudo make install

• cd utils

• sudo ./install_server.sh

• sudo service redis_6379 start

• redis-cli


Redis and Data Models


Redis Data ModelsData Types What?

Binary-safe strings Strings

Lists Strings List

Sets Unique unsorted strings

Sorted Sets Score field

Hashes O(1)

Bit Arrays/Bitmaps Bit operations on set

HyperLogLogs Estimate set cardinality


Key Design

• Keep keys short

• But not too short (readable, existing overhead)• Use Convention:

• comment:1234:reply.to


Setssimple storage


Basic Sets> SET counter 100

OK

> GET counter

(integer) 100

> inc counter

(integer) 101

> incby counter 50

(integer) 151

DECR DECRBY


Multi Set

> MSET counter 100 cnt2 102 cnt3 103

OK

> MGET counter cnt2 cnt 3

(integer) 100

(integer) 102

(integer) 103


Existence

> exists counter

(integer) 1

> type counter

integer

> del counter

(integer) 1


TTL Keys> expire counter 5

(integer) 1

> set cnt2 ex 5

OK

> ttl cnt

5

> persist cnt2

OK

Will Expire in 5 secondsWill Expire in 5 seconds

Will Expire in 5 secondsWill Expire in 5 seconds

How much time was left?How much time was left?

Remove TTLRemove TTL


Linked Listsa basis for queues


Existence

> Adding/Removing for end/start: O(1)

> Getting other items: O(N)

LPUSH/RPUSH => Add items (Head/Tail)

LPOP/RPOP => Take items (Head/Tail)

LRANGE/RRANGE => Get items (Head/Tail)


Blocking Operations

Best for Producer/Consumer (Q)

If list is empty,calls are blocked

> BRPOP <LIST> <# of Seconds>

> BLPOP <LIST> <# of Seconds>


Capped Lists

Define how many items we want to have

> LPUSH mylist <some element>

> LTRIM mylist 0 999


Hashesa column like data store


Existence> hmset user:1000 username antirez birthyear 1977 OK > hget user:1000 username "antirez" > hget user:1000 birthyear"1977" > hgetall user:1000 1) "username" 2) "antirez" 3) "birthyear" 4) "1977"


Setsa basis for social and permissions


Set Ops> sismember myset 3

(integer) 1

> sadd news:1000:tags 1 2 5 77

(integer) 4

> sadd tag:1:news 1000

(integer) 1

> sinter news:1000:tags news:2000:tags

,,,,


Ordered Sets (Z)A leader board solution


Set Ops

> zadd hackers 1940 "Alan Kay" (integer) 1

> zadd hackers 1957 "Sophie Wilson" (integer 1)

> zrange hackers 0 -1

> zremrangebyscore hackers 1940 1960


BitMapsEfficient Configuration and

Permissions


Set Ops

> setbit key 10 1

(integer) 1

> getbit key 10

(integer) 1

> bitcount key


Advanced Operations


Scan: You can iterate all keys!

Scan

SScan

HScan

ZScan


Lua Scripting: You have script engine

> eval "return redis.call('set','foo','bar')" 0

OK


Pub/Sub

Very nice tool, but not as good as Kafka and RabbitMQ

SUBSCRIBE first second

PUBLISH second Hello

UNSUBSCRIBE

PSUBSCRIBE news.*


SortingSORT mylist

SORT mylist DESC

SORT mylist ALPHA

SORT mylist BY weight_*

SORT mylist BY nosort

SORT mylist BY weight_* GET object_*

SORT mylist BY weight_* GET object_* GET #

SORT mylist BY weight_* STORE resultkey

SORT mylist BY weight_*->fieldname GET object_*->fieldname


Transactions1. Atomic

2. Blocking all other operations

3. All or Nothing

MULTI: Start transaction

EXEC: Commit

DISCARD: “Roll back” (only before EXEC, nothing done)

WATCH: Don’t use in multi client (race conditions)


Transaction by Example> MULTI

OK

> INCR foo

QUEUED

> INCR bar

QUEUED

> EXEC

1) (integer) 1

2) (integer) 1


Java Programming


Many Options


Securing Redis


Authentication

• By default: None

• Enable: requirepass in the config file

• Authenticate: AUTH <PASSWORD>

• Simple, Clear text

• No measures to prevent brute force


Communication Encryption

• SSL Communication encryption using stunnel/spiped

• Webdis web front layer

• Encrypt the data in the app level


Persistency


The Good, bad and Evil• RDB

• Point in time snapshots

• Backup before critical operation

• AOF

• On every write to log (or once a seocnd)

• Log rerun at startup

• None

• In memory only solution


CACHING

47


Don’t Use Caching

48


If You Have To

49


Invalidation Can Be Nightmare

50

http://luauf.com/2009/06/08/%C2%BFque-es-memcached/


Geo Load Balancing Can be Worse

51


Finally, Recovery May Not Be Better

52


KISS

53

http://marriagelifeministries.org/?p=962


Thank You !Moshe Kaplan

[email protected]

redis training for java software engineers

Technology