webinar: redis modules - percona · 2 who we are the open source home and commercial provider of...
TRANSCRIPT
Extend Redis with ModulesItamar Haber
2
Who We Are
The open source home and commercial provider of Redis
Open source. The leading in-memory database platform, supporting any high performance OLTP or OLAP use case.
[email protected]@itamarhaberChief Developer Advocate at Redis Labs
~10 Things About Redis
4
1.Redis: REmote DIctionary Server2./ rɛdɪs/: “red-iss”3.OSS: http://github.com/antirez/redis4.3-clause BSD-license: http://redis.io5. In-memory: read/write from/to RAM6.A database for: 5 data structures7.And: 4 (+1) more specialized ones
5
8.Developed & maintained: (mostly) Salvatore Sanfilippo (a.k.a. @antirez) and his OSS team at @RedisLabs
9.History: v1.0 August 9th, 2009… v3.2.4 Septmeber 26th, 2016
10.“The Leatherman™ of Databases”:mostly used as a DB, cache & broker
6
11.A couple or so of extra features:(a) atomicity; (b) blocking wait;(c) configurable persistence;(d) data expiration and (e) eviction; as well as transactions, PubSub, Luascripts, high availability & clustering
12.Next version (v4.0): MODULES!
7
Redis 1011. Redis is “NoSQL”0. No (explicit) schema, access by key1. Key -> structure -> data
8
Redis data stratav1.0 Strings
ListsSets
v1.2 SortedSets
v2.0 Hashes
v2.2 Bit arraysv2.8.9 HyperLogLogsv3.2 Geo Sets
Bit fieldsv4 MODULES!
and a brain!
9
How to Redis in 3 steps:1. 150 OSS clients in 50 languages, e.g:
Java, Node.js, .NET, Python, Ruby…2. You make a request, i.e.:
PING3. The server replies, i.e.g:
PONG
10
~$ redis-cli127.0.0.1:6379> SET counter 1OK127.0.0.1:6379> GET counter"1"127.0.0.1:6379> INCRBY counter 1(integer) 2127.0.0.1:6379> APPEND counter b||!2b(integer) 7127.0.0.1:6379> GETSET counter "\x00Hello\xffWorld""2b||!2b"127.0.0.1:6379>
The Evolution of Versatility
12
Flexibility: model (almost) anything with basic “building blocks” and simple rules (v0.0.1)Composability: transactions (v1.2) and server-side Lua scripts (v2.6)Extensibility: modules (v4) for adding custom data structures and commands
MODULES! (a.k.a plugins)
14
Redis before modules:1. Redis is ubiquitous for fast data, fits
lots of cases (Swiss™ Army knife)2. Some use cases need special care3. Open source has its own agenda
So what can you do? FR, PR or fork
15
Redis with modules:1. Core still fits lots of cases2. Module extensions for special cases3. A new community-driven ecosystem4. “Give power to users to go faster”5. … and not only to users …What to expect? Nothing’s impossible!
16
Redis modules are:1. Dynamically (server-)loaded libraries2. Future-compatible3. (are mostly) written in C4. (nearly) as fast as the core5. GA Q4 2016
17
Modules let you:1. Process: where the data is at2. Compose: call core & other modules3. Extend: add new data structures
and commands
18
redis> ECHO "Alpha""Alpha"redis> MODULE LOAD example.soOKredis> EXAMPLE.ECHO "Bravo""Bravo"redis> ^C
~$ wc example.c13 46 520 example.c~$ gcc -fPIC -std=gnu99 -c -o example.o example.c~$ ld -o example.so example.o -shared -Bsymbolic -lc
core command
module library
“new” command
Redis Modules API
20
The API1. Where most of the effort was made2. Abstracts & isolates Redis’ internals3. The server’s (C-) binding contract4. Will not be broken once released5. Exposes three conceptual layers
21
Modules API layers1.Operational: admin, memory, disk,
replication, arguments, replies…2.High-level: client-like access to core
and modules’ commands3.Low-level: (almost) native access to
core data structures memory
22
user app
Redis client
Redis
core
data
GET foo
"bar"
101010
010101
101010
23
user
101010
010101
101010
High
level
API
appmodule
24
user
101010
010101
101010
app
Low
level
API
The Benchmark (Why Bother with Modules?)
26
1.2 1.251.05
0.1
seconds
Time needed
for summing
1,000,000
Sorted Set
scores Python
(local)
Lua API
high low
27
Some exemplary modules:• RediSearch – full text search• countminsketch – streaming counter• redablooms – Bloom filters
• redis-tdigest – rank estimator
28
redismodules.com: Redis Module Hub
29
What Is The Hub
1.Modules developed by: anyone2.Certified by: Redis Labs3.Licenses: Open Source & Commercial4. (will be) Distributed via: Redis Cloud
and Redis Labs Enterprise Cluster5.Where: redismodules.com
30
Neural Redis moduleAlpha
• A brain neural network• Simple to use, will be part of Redis• Online (threaded) training• https://github.com/antirez/neural-
redis
31
redis> NN.CREATE net REGRESSOR 2 3 -> 1 …(integer) 13redis> NN.OBSERVE net 1 2 -> 3redis> NN.OBSERVE net 4 5 -> 9redis> NN.OBSERVE net 3 4 -> 7redis> NN.OBSERVE net 1 1 -> 2… repeat ~6 more times
redis> NN.TRAIN net AUTOSTOPredis> NN.RUN net 1 11) "2.0776522297040843"redis> NN.RUN net 3 21) "5.1765427204933099"
Teaching it
how to add
numbers
32
Build your own (module)• Get Redis (unstable branch)• Read the docs under src/modules• An SDK to jumpstart:
https://github.com/RedisLabs/RedisModulesSDK
• Start coding
33
Hack for Fun and Profit• Online: Oct 17th
San Francisco & Tel-Aviv: Nov 11th
• World fame & cash prizes!• Register at:
https://www.hackerearth.com/sprints/redislabs-hackathon-global/
Thank you
Further Reading
36
1. The Redis Open Source Project Website – http://redis.io2. Redis source code on GitHub – http://github.com/antirez/redis3. Getting started:
1. An introduction to Redis data types and abstractions –http://redis.io/topics/data-types-intro
2. Try Redis (in your browser) – http://try.redis.io3. Karl Seguin’s The Little Redis Book –
http://openmymind.net/2012/1/23/The-Little-Redis-Book/4. Josiah Carlson’s Redis In Action – https://redislabs.com/ebook/redis-in-
action
4. Redis documentation – http://redis.io/documentation5. Redis commands – http://redis.io/commands6. Redis community – http://redis.io/community7. Redis Watch newsletter – https://redislabs.com/redis-watch-archive
37
8. Redis Loadable Modules System – http://antirez.com/news/1069. Introduction to Redis Modules API –
https://github.com/antirez/redis/blob/unstable/src/modules/INTRO.md
10. Redis Modules API reference –https://github.com/antirez/redis/blob/unstable/src/modules/API.md
11. Creating a redis Module in 15 lines of code! –https://gist.github.com/dvirsky/83fc32366d5ad82fc3dca47ed2704377