reactive databases for big data applications
TRANSCRIPT
Humberto Rodríguez Avila, Wolfgang de Meuter, Elisa González Boix
Reactive Databases for BigData Applications
Graph-TA 4th EditionBarcelona, 2016
Context
1REACTIVE DATABASES FOR BIGDATA APPLICATIONS/HUMBERTO RODRIGUEZ AVILA
Reactive Programming
2
Server-Side
REACTIVE DATABASES FOR BIGDATA APPLICATIONS/HUMBERTO RODRIGUEZ AVILA
Behaviours
8
5
7
15
+2 *2
+
Dependency graphExpression
x =#(a+2)y =#(b*2)
z =#x#+#y a
x y
z
Sub<expressions
4 b
NO application in Database Contexts
Databases
3REACTIVE DATABASES FOR BIGDATA APPLICATIONS/HUMBERTO RODRIGUEZ AVILA
PubNubPusher
Databases
4REACTIVE DATABASES FOR BIGDATA APPLICATIONS/HUMBERTO RODRIGUEZ AVILA
r.table(“users”).filter( r.row(“age”).lt(18) .changes().run(conn, function(err, cursor) { // update application layer
})
• Collaborative web and mobile apps • Streaming analytics apps • Multiplayer games • Realtime marketplaces • Connected devices
Limited reactive capabilities
Notifications through callbacks
Changefeedsexample
NO shares intermediate results
5REACTIVE DATABASES FOR BIGDATA APPLICATIONS/HUMBERTO RODRIGUEZ AVILA
Reactive Programming
Databases
Reactive BigData applications
+
Architecture overview
6REACTIVE DATABASES FOR BIGDATA APPLICATIONS/HUMBERTO RODRIGUEZ AVILA
Front-end Component Back-end Component
Reactive Language
Language Integrated Query
Graph Dependency Manager
Reactive Database
NoSQL Store
Rules Engine NotificationManager
BigData Analysis
Front-end component
8REACTIVE DATABASES FOR BIGDATA APPLICATIONS/HUMBERTO RODRIGUEZ AVILA
var dbBooks = from b in R.Books where b.tag = “database” select b;
var reactDbBooks = from b in R.Books where b.tag = “database” select b observe as “database books”;
DataSource
Declarative syntax1
2
Reactive Query
3 var response = R.unregisterQuery(“database books”); true | Error
Reactive Language
Language Integrated Query
Graph Dependency Manager
Back-end component
7REACTIVE DATABASES FOR BIGDATA APPLICATIONS/HUMBERTO RODRIGUEZ AVILA
Rules Engine
Working Memory
RuleSet
NoSQL In-Memory Database
Persistent
- Rete-based- Node sharing- Accumulate functions
NoSQL Store1
2
NotificationManager
Socket-based
3triggers
Graph Dependency Manager
Reactive Language
BigData Analysis4
Back-end component
7REACTIVE DATABASES FOR BIGDATA APPLICATIONS/HUMBERTO RODRIGUEZ AVILA
var reactDbBooks = from c in R.Customer join!a!in R.Account on c.id == a.id_costumer where c.age < 20 and a.balance <= 100 select c observe as “customers C”;
1 2 Top node
Customer Account
age < 20 balance <= 100
age < 20 & balance <= 100
Object-type nodeAlpha nodeJoin nodeTerminal node
Future Work
7REACTIVE DATABASES FOR BIGDATA APPLICATIONS/HUMBERTO RODRIGUEZ AVILA
- Implement of garbage collector in the rule engine
- Apply query optimisations techniques to the Rete-Graph
- Integration of our Riak extension with Apache Spark
- Implementation of reactive constructs for Apache Spark
- Finish the implementation of our reactive-query language
Garbage collector
Apache Spark
…
reactiveconstr…Query
optimi…
Contact
Humberto Rodríguez [email protected]
Prof. Dr. Elisa Gonzalez [email protected]
Prof. Dr. Wolfgang De [email protected]
14http://soft.vub.ac.be http://vub.ac.be
Humberto Rodríguez Avila, Wolfgang de Meuter, Elisa González Boix
Reactive Databases for BigData Applications
Graph-TA 4th EditionBarcelona, 2016