redis - uni-weimar.de · when aof gets to big redis rewrites it from in-memory data new aof is...
TRANSCRIPT
Redis
Jan Graßegger
2014/07/03
Redis
A persistent distributed in-memory database
2
Redis
A persistent distributed in-memory database
3
Persistence
Database Write Operation
1. Client sends a write command to the database
2. Database receives the write
3. Database calls the system call that writes the data on disk
4. Operating system transfers the write buffer to the disk controller
5. Disk controller writes data into a physical media
5
It`s all about data safety
Relevant POSIX API commands
• write Gives data safety against process failure
• fsync Gives data safety against system failure (with restrictions)
6
and data corruption
Three levels of safety against corruption
• Databases that do not care about
• Databases that use a log of operations to recover a consistent state after a failure
• Databases that never modify already written data. Append only
7
Redis
Two persistence strategies
• Snapshots
• Append Only File (AOF)
8
Snapshots (1)
Dump of the whole database
Creation interval is configurable and depends on a time interval and minimum number of keys changed
Every snapshot is created from in-memory data
New snapshot created as temporary file and renamed if write operation was successful
9
Snapshots (2)
On failure database changes longer ago than the configured time interval might be lost
Snapshots are created in a background task. Do not affect database performance
10
AOF (1)
Appends change operations to one file
No changing, just appending
Always growing file
Commands only get logged if they changed data
11
AOF (2)
Three levels of durability
• appendnfsync no Acknowledge write only after change was transferred to the kernel (POSIX write)
• appendnfsync everysec Write is flushed every second via POSIX fsync
• appendnfsync always Command is written and synced to disk before acknowledging to the client
12
AOF (3)When AOF gets to big Redis rewrites it from in-memory data
New AOF is wrote to temporary file
When disk was synced temporary file is renamed to old file
If data is added during this process. New data appends to old file. Differences between old and new are buffered
13
Distribution
Distribution
Two aspects of distribution
• Replication
• Partitioning
15
Replication (1)
One master node
Multiple slave nodes
!
Non-blocking on master side
Optional non-blocking on slave side
Slaves are read-only by default
16
Replication (2)
17
Slave Master
SYNC
„Bulk transfer“
COMMAND 1
OKCOMMAND 1
Client
COMMAND 2
OKCOMMAND 2
What about your last talk?
In the last talk I said:
Redis does not support sharding/partitioning.
!
That was right, but…
18
Partitioning(1)
Redis itself does not support partitioning!
There exists several clients or extensions that support it
• Redis cluster
• Twemproxy
• Redis-rb/Predis
19
Partitioning(2)
Disadvantages
• Multiple key operations not supported (intersection)
• Adding and removing nodes can be complex
20
Performance
What Redis claimsIntel(R) Xeon(R) CPU E5520 @ 2.27GHz (with pipelining)
• 552,028 SET operations per second
• 707,463 GET operations per second
!
Intel(R) Xeon(R) CPU E5520 @ 2.27GHz (without pipelining)
• 122,556 SET operations per second
• 123,601 GET operations per second22
First Benchmark1,000,000 million requests (random keys/values) on an Intel Core2Quad Q9550 @ 2,83GHz
23
operations per second
AOF off AOF on
pipelining off 55,919.30 SET 54,755.52 GET
54,677.68 SET 54,427.70 GET
pipelining on 359,195.41 SET 455,373.38 GET
126,087.50 SET 455,580.88 GET