cassandra drivers and libraries
DESCRIPTION
Cassandra Driver Architecture and LibrariesTRANSCRIPT
![Page 1: Cassandra drivers and libraries](https://reader034.vdocuments.us/reader034/viewer/2022042614/559bda621a28ab223a8b4584/html5/thumbnails/1.jpg)
@doanduyhai
Cassandra Drivers And Tools DuyHai DOAN, Technical Advocate
![Page 2: Cassandra drivers and libraries](https://reader034.vdocuments.us/reader034/viewer/2022042614/559bda621a28ab223a8b4584/html5/thumbnails/2.jpg)
@doanduyhai
Agenda!
2
Drivers • architecture, policies, Java driver API
DevCenter (live coding demo!) Cassandra Unit (+ live coding demo!) Object Mapper module (+ live coding demo!) Achilles Object Mapper (+ live coding demo!)
![Page 3: Cassandra drivers and libraries](https://reader034.vdocuments.us/reader034/viewer/2022042614/559bda621a28ab223a8b4584/html5/thumbnails/3.jpg)
Cassandra Drivers Architecture!
Architecture!Policies!
Java driver API!
![Page 4: Cassandra drivers and libraries](https://reader034.vdocuments.us/reader034/viewer/2022042614/559bda621a28ab223a8b4584/html5/thumbnails/4.jpg)
@doanduyhai
Drivers list!
4
• Java • C# • Python • Node.js • Ruby (1.0.0.rc1) • C++ (beta) • ODBC (beta) • Clojure (community) • Go (community) • PHP (to be announced)
![Page 5: Cassandra drivers and libraries](https://reader034.vdocuments.us/reader034/viewer/2022042614/559bda621a28ab223a8b4584/html5/thumbnails/5.jpg)
@doanduyhai
Connection pooling!
5
n2
n3
n4
Driver
Pool1
Pool2
Pool3
Client Thread1
Client Thread2
Client Thread3
![Page 6: Cassandra drivers and libraries](https://reader034.vdocuments.us/reader034/viewer/2022042614/559bda621a28ab223a8b4584/html5/thumbnails/6.jpg)
@doanduyhai
Connection pooling!
6
n2
n3
n4
Driver
Pool1
Pool2
Pool3
Client Thread1
Client Thread2
Client Thread3
1
2 3
4 5
6
![Page 7: Cassandra drivers and libraries](https://reader034.vdocuments.us/reader034/viewer/2022042614/559bda621a28ab223a8b4584/html5/thumbnails/7.jpg)
@doanduyhai
Request Pipelining!
7
Client Cassandra
![Page 8: Cassandra drivers and libraries](https://reader034.vdocuments.us/reader034/viewer/2022042614/559bda621a28ab223a8b4584/html5/thumbnails/8.jpg)
@doanduyhai
Request Pipelining!
8
Client Cassandra
StreamID
StreamID
![Page 9: Cassandra drivers and libraries](https://reader034.vdocuments.us/reader034/viewer/2022042614/559bda621a28ab223a8b4584/html5/thumbnails/9.jpg)
@doanduyhai
Nodes Discovery!
9
n2
n3
n4
n5
n6
n7
n8
n1 Driver
Control Connection
![Page 10: Cassandra drivers and libraries](https://reader034.vdocuments.us/reader034/viewer/2022042614/559bda621a28ab223a8b4584/html5/thumbnails/10.jpg)
@doanduyhai
Round Robin Load Balancing!
10
n2
n3
n4
n5
n6
n7
n8
n1 Client 1
2 3
4
![Page 11: Cassandra drivers and libraries](https://reader034.vdocuments.us/reader034/viewer/2022042614/559bda621a28ab223a8b4584/html5/thumbnails/11.jpg)
@doanduyhai
DC Aware Load Balancing!
11
Client1 DC1
DC2 Client2
⤫
![Page 12: Cassandra drivers and libraries](https://reader034.vdocuments.us/reader034/viewer/2022042614/559bda621a28ab223a8b4584/html5/thumbnails/12.jpg)
@doanduyhai
DC Aware Load Balancing!
12
Client1 DC1
DC2 Client2
⤫
![Page 13: Cassandra drivers and libraries](https://reader034.vdocuments.us/reader034/viewer/2022042614/559bda621a28ab223a8b4584/html5/thumbnails/13.jpg)
@doanduyhai
Token Aware Load Balancing!
13
n2
n3
n4
n5
n6
n7
n8
n1 Client
1
2
3
⤫
![Page 14: Cassandra drivers and libraries](https://reader034.vdocuments.us/reader034/viewer/2022042614/559bda621a28ab223a8b4584/html5/thumbnails/14.jpg)
@doanduyhai
Combining Load Balancing Policies!
14
Load Balancing Policy
Round Robin DC Aware Round Robin
Token Aware
extends
wraps
Default config
![Page 15: Cassandra drivers and libraries](https://reader034.vdocuments.us/reader034/viewer/2022042614/559bda621a28ab223a8b4584/html5/thumbnails/15.jpg)
@doanduyhai
Automatic Failover!
15
n2
n3
n4
Driver
Pool1
Pool2
Pool3
Client Thread
⤫
1
2 3
4 5
6 7 8
![Page 16: Cassandra drivers and libraries](https://reader034.vdocuments.us/reader034/viewer/2022042614/559bda621a28ab223a8b4584/html5/thumbnails/16.jpg)
@doanduyhai
Other policies!
16
Retry policy • write/read timeout • node unavailable Reconnection policy • constant schedule • exponential schedule
![Page 17: Cassandra drivers and libraries](https://reader034.vdocuments.us/reader034/viewer/2022042614/559bda621a28ab223a8b4584/html5/thumbnails/17.jpg)
@doanduyhai
Statements!
17
Plain statement • convenient, one-off query • plain string ☞ parsing overhead
INSERT INTO user(login, name, age) VALUES(‘jdoe’, ‘John DOE’, 33)’;
![Page 18: Cassandra drivers and libraries](https://reader034.vdocuments.us/reader034/viewer/2022042614/559bda621a28ab223a8b4584/html5/thumbnails/18.jpg)
@doanduyhai
Statements!
18
Prepared statements • avoid parsing overhead • query structure should be known ahead of time • bound values • named parameters
INSERT INTO user(login, name, age) VALUES(?, ?, ?)’;
INSERT INTO user(login, name, age) VALUES(:login, :name, :age)’;
![Page 19: Cassandra drivers and libraries](https://reader034.vdocuments.us/reader034/viewer/2022042614/559bda621a28ab223a8b4584/html5/thumbnails/19.jpg)
@doanduyhai
Statements!
19
Parameterized statements • same as plain statement • pass bound values as bytes ☞ avoid ser/deser of values
INSERT INTO user(login, name, age) VALUES(?, ?, ?)’;
![Page 20: Cassandra drivers and libraries](https://reader034.vdocuments.us/reader034/viewer/2022042614/559bda621a28ab223a8b4584/html5/thumbnails/20.jpg)
@doanduyhai
Java Driver!
20
Reference implementation Base on asynchronous Netty library Configurable policies Query tracing support Client-node compression & SSL
![Page 21: Cassandra drivers and libraries](https://reader034.vdocuments.us/reader034/viewer/2022042614/559bda621a28ab223a8b4584/html5/thumbnails/21.jpg)
@doanduyhai
Maven dependency!
21
<dependency> <groupId>com.datastax.cassandra</groupId> <artifactId>cassandra-driver-core</artifactId> <version>2.1.3</version> </dependency>
depends on Netty, Guava, Metrics
Available on Maven Central
![Page 22: Cassandra drivers and libraries](https://reader034.vdocuments.us/reader034/viewer/2022042614/559bda621a28ab223a8b4584/html5/thumbnails/22.jpg)
@doanduyhai
Connect and Write!
22
Cluster cluster = Cluster.builder() .addContactPoints("127.0.0.1", “another-host").build();
seed nodes (IP or DNS name)
Session session = cluster.connect("my_keyspace");
session.execute("INSERT INTO user (user_id, name, email) VALUES (12345, 'johndoe', '[email protected]’)");
![Page 23: Cassandra drivers and libraries](https://reader034.vdocuments.us/reader034/viewer/2022042614/559bda621a28ab223a8b4584/html5/thumbnails/23.jpg)
@doanduyhai
Configuration!
23
Cluster cluster = Cluster.builder() .addContactPoints("127.0.0.1", “another-host") .withLoadBalancingPolicy(new DCAwareRoundRobinPolicy("DC1") .withRetryPolicy(DowngradingConsistencyRetryPolicy.INSTANCE) .withReconnectionPolicy(new ConstantReconnectionPolicy(1000)) .build();
![Page 24: Cassandra drivers and libraries](https://reader034.vdocuments.us/reader034/viewer/2022042614/559bda621a28ab223a8b4584/html5/thumbnails/24.jpg)
@doanduyhai
Read!
24
ResultSet resultSet = session.execute("SELECT * FROM user");
List<Row> rows = resultSet.all();
for (Row row : rows) { Long userId = row.getLong("user_id"); String name = row.getString("name"); String email = row.getString("email"); }
Stateless ☞ thread-safe
![Page 25: Cassandra drivers and libraries](https://reader034.vdocuments.us/reader034/viewer/2022042614/559bda621a28ab223a8b4584/html5/thumbnails/25.jpg)
@doanduyhai 25
Asynchronous Read!
ResultSetFuture future = session.executeAsync("SELECT * FROM user");
ResultSet resultSet = future.get(); //blocking call
List<Row> rows = resultSet.all();
for (Row row : rows) { Long userId = row.getLong("user_id"); String name = row.getString("name"); String email = row.getString("email"); }
![Page 26: Cassandra drivers and libraries](https://reader034.vdocuments.us/reader034/viewer/2022042614/559bda621a28ab223a8b4584/html5/thumbnails/26.jpg)
@doanduyhai 26
Asynchronous Read with CallBack!
ResultSetFuture future = session.executeAsync("SELECT * FROM user");
future.addListener(new Runnable() { public void run() { // Process the results here } }, executor);
executor = Executors .newCachedThreadPool(); or
executor = Executors .sameThreadExecutor();
![Page 27: Cassandra drivers and libraries](https://reader034.vdocuments.us/reader034/viewer/2022042614/559bda621a28ab223a8b4584/html5/thumbnails/27.jpg)
@doanduyhai 27
Query Builder!
Query query = QueryBuilder .select() .all() .from( "my_keyspace", "user") .where(eq("login", "jdoe"));
query.setConsistencyLevel(ConsistencyLevel.ONE);
ResultSet rs = session.execute(query);
![Page 28: Cassandra drivers and libraries](https://reader034.vdocuments.us/reader034/viewer/2022042614/559bda621a28ab223a8b4584/html5/thumbnails/28.jpg)
@doanduyhai
Old manual paging!
28
Some time you need to fetch all table content Manual paging:
SELECT * FROM users WHERE token(login) >= token(<last_fetched_login>) LIMIT 100;
![Page 29: Cassandra drivers and libraries](https://reader034.vdocuments.us/reader034/viewer/2022042614/559bda621a28ab223a8b4584/html5/thumbnails/29.jpg)
@doanduyhai
New automatic paging!
29
n2
n3
n4
n5
n6
n7
n8
n1 Driver
Query
Page 1 + paging state 1
![Page 30: Cassandra drivers and libraries](https://reader034.vdocuments.us/reader034/viewer/2022042614/559bda621a28ab223a8b4584/html5/thumbnails/30.jpg)
@doanduyhai
New automatic paging!
30
n2
n3
n4
n5
n6
n7
n8
n1 Driver
Query
Page 2 + paging state 2
![Page 31: Cassandra drivers and libraries](https://reader034.vdocuments.us/reader034/viewer/2022042614/559bda621a28ab223a8b4584/html5/thumbnails/31.jpg)
@doanduyhai
New automatic paging!
31
Paging state ≈ stateless cookie Resilient to node failure
![Page 32: Cassandra drivers and libraries](https://reader034.vdocuments.us/reader034/viewer/2022042614/559bda621a28ab223a8b4584/html5/thumbnails/32.jpg)
@doanduyhai
Paging during node failure!
32
n2
n3
n4
n5
n6
n7
n8
n1 Driver
Query
Page 1 + paging state 1
![Page 33: Cassandra drivers and libraries](https://reader034.vdocuments.us/reader034/viewer/2022042614/559bda621a28ab223a8b4584/html5/thumbnails/33.jpg)
@doanduyhai
Paging during node failure!
33
n2
n3
n4
n5
n6
n7
n8
n1 Driver
Query
Page 2 + paging state 2
⤫
![Page 34: Cassandra drivers and libraries](https://reader034.vdocuments.us/reader034/viewer/2022042614/559bda621a28ab223a8b4584/html5/thumbnails/34.jpg)
Q & R
! " !
![Page 35: Cassandra drivers and libraries](https://reader034.vdocuments.us/reader034/viewer/2022042614/559bda621a28ab223a8b4584/html5/thumbnails/35.jpg)
Dev Center Demo
![Page 36: Cassandra drivers and libraries](https://reader034.vdocuments.us/reader034/viewer/2022042614/559bda621a28ab223a8b4584/html5/thumbnails/36.jpg)
@doanduyhai
Cassandra Unit!
36
• start an embedded Cassandra server
• useful for unit testing
• mature project, created at August 5th 2011
• designed around Thrift & Hector
• propose a JUnit rule for CQL
![Page 37: Cassandra drivers and libraries](https://reader034.vdocuments.us/reader034/viewer/2022042614/559bda621a28ab223a8b4584/html5/thumbnails/37.jpg)
Cassandra Unit Demo
![Page 38: Cassandra drivers and libraries](https://reader034.vdocuments.us/reader034/viewer/2022042614/559bda621a28ab223a8b4584/html5/thumbnails/38.jpg)
@doanduyhai
Java Driver Object Mapper!
38
• simple mapper
• KISS
• annotations à-la JPA (but no JPA dependencies)
• templating system à-la Spring Data
![Page 39: Cassandra drivers and libraries](https://reader034.vdocuments.us/reader034/viewer/2022042614/559bda621a28ab223a8b4584/html5/thumbnails/39.jpg)
@doanduyhai
Java Driver Object Mapper!
39
@Table(keyspace = "mapper_module", name = "users") public class User {
@PartitionKey private String login;
private String name; // getters and setters omitted... }
Mapping
![Page 40: Cassandra drivers and libraries](https://reader034.vdocuments.us/reader034/viewer/2022042614/559bda621a28ab223a8b4584/html5/thumbnails/40.jpg)
@doanduyhai
Java Driver Object Mapper!
40
MappingManager manager = new MappingManager(session); Mapper mapper = manager.mapper(User.class);
User user = mapper.get("[email protected]");
mapper.saveAsync(new User("[email protected]"));
mapper.delete("[email protected]");
Usage
![Page 41: Cassandra drivers and libraries](https://reader034.vdocuments.us/reader034/viewer/2022042614/559bda621a28ab223a8b4584/html5/thumbnails/41.jpg)
@doanduyhai
Java Driver Object Mapper!
41
@Accessor interface UserAccessor { @Query("SELECT * FROM users LIMIT :max") Result<User> firstNUsers(@Param("max") int limit); }
Accessors (SpringData template-like) definition
![Page 42: Cassandra drivers and libraries](https://reader034.vdocuments.us/reader034/viewer/2022042614/559bda621a28ab223a8b4584/html5/thumbnails/42.jpg)
@doanduyhai
Java Driver Object Mapper!
42
Accessors usage
UserAccessor accessor = manager.createAccessor(UserAccessor.class); List<User> users = accessor.firstNUsers(10).all(); for (User user : users) { System.out.println( profile.getAddress().getZip() ); }
![Page 43: Cassandra drivers and libraries](https://reader034.vdocuments.us/reader034/viewer/2022042614/559bda621a28ab223a8b4584/html5/thumbnails/43.jpg)
Java Driver Object Mapper Demo
![Page 44: Cassandra drivers and libraries](https://reader034.vdocuments.us/reader034/viewer/2022042614/559bda621a28ab223a8b4584/html5/thumbnails/44.jpg)
@doanduyhai
Achilles!
44
Why ? • started in late 2012, when mapper module did not exists
• more involved and more features than the mapper module
• different annotations set (may converge)
![Page 45: Cassandra drivers and libraries](https://reader034.vdocuments.us/reader034/viewer/2022042614/559bda621a28ab223a8b4584/html5/thumbnails/45.jpg)
Achilles Demo
![Page 46: Cassandra drivers and libraries](https://reader034.vdocuments.us/reader034/viewer/2022042614/559bda621a28ab223a8b4584/html5/thumbnails/46.jpg)
@doanduyhai
Dirty Checking!
46
Dirty checking, why is it important ? • 1 user ≈ 8 mutable fields
• × n denormalizations = n update combinations
• and not even counting multiple fields updates …
![Page 47: Cassandra drivers and libraries](https://reader034.vdocuments.us/reader034/viewer/2022042614/559bda621a28ab223a8b4584/html5/thumbnails/47.jpg)
@doanduyhai
Dirty Checking!
47
• Are you going to manually generate n prepared statements for all possible updates ?
• Or just use dynamic plain string statements and get some perf
penalty ?
![Page 48: Cassandra drivers and libraries](https://reader034.vdocuments.us/reader034/viewer/2022042614/559bda621a28ab223a8b4584/html5/thumbnails/48.jpg)
@doanduyhai
Dirty Checking!
48
//No read-before-write ContactEntity proxy = manager.forUpdate(ContactEntity.class, contactId); proxy.setFirstName(…); proxy.setLastName(…); //type-safe updates proxy.setAddress(…);
manager.update(proxy);
![Page 49: Cassandra drivers and libraries](https://reader034.vdocuments.us/reader034/viewer/2022042614/559bda621a28ab223a8b4584/html5/thumbnails/49.jpg)
@doanduyhai
Dirty Checking!
49
Proxy Setters interception
Empty Entity
DirtyMap
PrimaryKey
![Page 50: Cassandra drivers and libraries](https://reader034.vdocuments.us/reader034/viewer/2022042614/559bda621a28ab223a8b4584/html5/thumbnails/50.jpg)
@doanduyhai
Dirty Checking!
50
• Dynamic statement generation
UPDATE contacts SET firstname=?, lastname=?,address=? WHERE contact_id=?
prepared statements are cached, of course
![Page 51: Cassandra drivers and libraries](https://reader034.vdocuments.us/reader034/viewer/2022042614/559bda621a28ab223a8b4584/html5/thumbnails/51.jpg)
@doanduyhai
Main API!
51
manager.insert(entity) manager.update(entity) manager.remove(entity) manager.find(Entity.class, primaryKey)
![Page 52: Cassandra drivers and libraries](https://reader034.vdocuments.us/reader034/viewer/2022042614/559bda621a28ab223a8b4584/html5/thumbnails/52.jpg)
@doanduyhai
Advanced Features!
52
Counter Batch mode Strategies (insert, naming) Options Asynchronous
![Page 53: Cassandra drivers and libraries](https://reader034.vdocuments.us/reader034/viewer/2022042614/559bda621a28ab223a8b4584/html5/thumbnails/53.jpg)
@doanduyhai
Documentation!
53
Comprehensive Github WIKI Twitter-clone demo app (demo.achilles.io) Versioned documentation (HTML & PDF) JavaDoc
![Page 54: Cassandra drivers and libraries](https://reader034.vdocuments.us/reader034/viewer/2022042614/559bda621a28ab223a8b4584/html5/thumbnails/54.jpg)
@doanduyhai
RoadMap!
54
C* 2.1 user defined types (UDT) Query-templates à-la Spring Data Reactive ? (RxJava) ElasticSearch integration (@olivierbourgain)
![Page 55: Cassandra drivers and libraries](https://reader034.vdocuments.us/reader034/viewer/2022042614/559bda621a28ab223a8b4584/html5/thumbnails/55.jpg)
Q & R
! " !