redis - uni-weimar.de · when aof gets to big redis rewrites it from in-memory data new aof is...

Redis

Jan Graßegger

2014/07/03

Redis

A persistent distributed in-memory database

2

Redis

A persistent distributed in-memory database

3

Persistence

Database Write Operation

1. Client sends a write command to the database

2. Database receives the write

3. Database calls the system call that writes the data on disk

4. Operating system transfers the write buffer to the disk controller

5. Disk controller writes data into a physical media

5

It`s all about data safety

Relevant POSIX API commands

• write Gives data safety against process failure

• fsync Gives data safety against system failure (with restrictions)

6

and data corruption

Three levels of safety against corruption

• Databases that do not care about

• Databases that use a log of operations to recover a consistent state after a failure

• Databases that never modify already written data. Append only

7

Redis

Two persistence strategies

• Snapshots

• Append Only File (AOF)

8

Snapshots (1)

Dump of the whole database

Creation interval is configurable and depends on a time interval and minimum number of keys changed

Every snapshot is created from in-memory data

New snapshot created as temporary file and renamed if write operation was successful

9

Snapshots (2)

On failure database changes longer ago than the configured time interval might be lost

Snapshots are created in a background task. Do not affect database performance

10

AOF (1)

Appends change operations to one file

No changing, just appending

Always growing file

Commands only get logged if they changed data

11

AOF (2)

Three levels of durability

• appendnfsync no Acknowledge write only after change was transferred to the kernel (POSIX write)

• appendnfsync everysec Write is flushed every second via POSIX fsync

• appendnfsync always Command is written and synced to disk before acknowledging to the client

12

AOF (3)When AOF gets to big Redis rewrites it from in-memory data

New AOF is wrote to temporary file

When disk was synced temporary file is renamed to old file

If data is added during this process. New data appends to old file. Differences between old and new are buffered

13

Distribution

Distribution

Two aspects of distribution

• Replication

• Partitioning

15

Replication (1)

One master node

Multiple slave nodes

!

Non-blocking on master side

Optional non-blocking on slave side

Slaves are read-only by default

16

Replication (2)

17

Slave Master

SYNC

„Bulk transfer“

COMMAND 1

OKCOMMAND 1

Client

COMMAND 2

OKCOMMAND 2

What about your last talk?

In the last talk I said:

Redis does not support sharding/partitioning.

!

That was right, but…

18

Partitioning(1)

Redis itself does not support partitioning!

There exists several clients or extensions that support it

• Redis cluster

• Twemproxy

• Redis-rb/Predis

19

Partitioning(2)

Disadvantages

• Multiple key operations not supported (intersection)

• Adding and removing nodes can be complex

20

Performance

What Redis claimsIntel(R) Xeon(R) CPU E5520 @ 2.27GHz (with pipelining)

• 552,028 SET operations per second

• 707,463 GET operations per second

!

Intel(R) Xeon(R) CPU E5520 @ 2.27GHz (without pipelining)

• 122,556 SET operations per second

• 123,601 GET operations per second22

First Benchmark1,000,000 million requests (random keys/values) on an Intel Core2Quad Q9550 @ 2,83GHz

23

operations per second

AOF off AOF on

pipelining off 55,919.30 SET 54,755.52 GET

54,677.68 SET 54,427.70 GET

pipelining on 359,195.41 SET 455,373.38 GET

126,087.50 SET 455,580.88 GET