Download - Nosql Ug Paris Imdg in Action With Coherence Without Transactions Chapter 110525091639 Phpapp02
7/28/2019 Nosql Ug Paris Imdg in Action With Coherence Without Transactions Chapter 110525091639 Phpapp02
http://slidepdf.com/reader/full/nosql-ug-paris-imdg-in-action-with-coherence-without-transactions-chapter-110525091639 1/76
In Memory Data Grid in Action
with Oracle Coherence
for Paris NoSQL User Group
Cyrille Le Clerc
Transactions chapter will be presented
during another session
Wednesday, May 25, 2011
7/28/2019 Nosql Ug Paris Imdg in Action With Coherence Without Transactions Chapter 110525091639 Phpapp02
http://slidepdf.com/reader/full/nosql-ug-paris-imdg-in-action-with-coherence-without-transactions-chapter-110525091639 2/76
Speaker
2
Cyrille Le Clerc
@cyrilleleclerc
blog.xebia.fr
Open Source(Apache CXF, ...)
In Memory Data Grid
Large Scale
“you build it, you run it”
Wednesday, May 25, 2011
7/28/2019 Nosql Ug Paris Imdg in Action With Coherence Without Transactions Chapter 110525091639 Phpapp02
http://slidepdf.com/reader/full/nosql-ug-paris-imdg-in-action-with-coherence-without-transactions-chapter-110525091639 3/76
3
Once upon a time...
Wednesday, May 25, 2011
7/28/2019 Nosql Ug Paris Imdg in Action With Coherence Without Transactions Chapter 110525091639 Phpapp02
http://slidepdf.com/reader/full/nosql-ug-paris-imdg-in-action-with-coherence-without-transactions-chapter-110525091639 4/76
4
- Released Coherence in 2001
- Started as a distributed cache
- Released Gigaspaces XAP in 2001
- Started as a data grid
On the Financial side
• Very low latency
• Rich queries & transactions
• Scalability
• Data consistency
Needs within financial market :
Wednesday, May 25, 2011
7/28/2019 Nosql Ug Paris Imdg in Action With Coherence Without Transactions Chapter 110525091639 Phpapp02
http://slidepdf.com/reader/full/nosql-ug-paris-imdg-in-action-with-coherence-without-transactions-chapter-110525091639 5/76
5
Let’s define an In Memory Data Grid ...
Wednesday, May 25, 2011
7/28/2019 Nosql Ug Paris Imdg in Action With Coherence Without Transactions Chapter 110525091639 Phpapp02
http://slidepdf.com/reader/full/nosql-ug-paris-imdg-in-action-with-coherence-without-transactions-chapter-110525091639 6/76
Let’s define an In Memory Data Grid
6
eXtreme Scale
This is an In Memory Data Grid
Wednesday, May 25, 2011
7/28/2019 Nosql Ug Paris Imdg in Action With Coherence Without Transactions Chapter 110525091639 Phpapp02
http://slidepdf.com/reader/full/nosql-ug-paris-imdg-in-action-with-coherence-without-transactions-chapter-110525091639 7/76
Let’s define an In Memory Data Grid
7
This is Network Attached Memory
Wednesday, May 25, 2011
7/28/2019 Nosql Ug Paris Imdg in Action With Coherence Without Transactions Chapter 110525091639 Phpapp02
http://slidepdf.com/reader/full/nosql-ug-paris-imdg-in-action-with-coherence-without-transactions-chapter-110525091639 8/76
Similarities with NoSQL document orientedPartitioned, distributed Hastable, schema-less, value is not opaque,scale-out scalability
Very fast
In memory (persistence coming), business logic inside the data
Consistent and AvailableTransactional, redundant
Written in Java, data are POJOsNot necessary
Clients in Java, Microsoft, etc
8
Let’s define an In Memory Data Grid
Wednesday, May 25, 2011
7/28/2019 Nosql Ug Paris Imdg in Action With Coherence Without Transactions Chapter 110525091639 Phpapp02
http://slidepdf.com/reader/full/nosql-ug-paris-imdg-in-action-with-coherence-without-transactions-chapter-110525091639 9/76
9
Use cases for this presentation
Wednesday, May 25, 2011
7/28/2019 Nosql Ug Paris Imdg in Action With Coherence Without Transactions Chapter 110525091639 Phpapp02
http://slidepdf.com/reader/full/nosql-ug-paris-imdg-in-action-with-coherence-without-transactions-chapter-110525091639 10/76
Train Booking System
10
trains, stations,seats, booking and
passengers
Wednesday, May 25, 2011
7/28/2019 Nosql Ug Paris Imdg in Action With Coherence Without Transactions Chapter 110525091639 Phpapp02
http://slidepdf.com/reader/full/nosql-ug-paris-imdg-in-action-with-coherence-without-transactions-chapter-110525091639 11/76
eCommerce Web Site
11
warehouse stocks
231
2
canon-eos: 1ipod : 1headphone : 1
iphone: 1
...
ipad : 1
iphone: 1
barbie : 1
iphone: 1cabbage-doll: 1
121
311
12
264
637
{
"name": "Barbie Computer",
"stock": 637,
"weigth" : 200
}
warehouse &customers shopping carts
Wednesday, May 25, 2011
7/28/2019 Nosql Ug Paris Imdg in Action With Coherence Without Transactions Chapter 110525091639 Phpapp02
http://slidepdf.com/reader/full/nosql-ug-paris-imdg-in-action-with-coherence-without-transactions-chapter-110525091639 12/76
12
In Memory Data Grids Key Principles
Wednesday, May 25, 2011
7/28/2019 Nosql Ug Paris Imdg in Action With Coherence Without Transactions Chapter 110525091639 Phpapp02
http://slidepdf.com/reader/full/nosql-ug-paris-imdg-in-action-with-coherence-without-transactions-chapter-110525091639 13/76
Store Everything in a Mainframe !
13
3 To of RAM
80 x 5.2 GHtz cores Much more than $1,000,000
IBM z11
http://ibm.com/
Wednesday, May 25, 2011
7/28/2019 Nosql Ug Paris Imdg in Action With Coherence Without Transactions Chapter 110525091639 Phpapp02
http://slidepdf.com/reader/full/nosql-ug-paris-imdg-in-action-with-coherence-without-transactions-chapter-110525091639 14/76
Spread on Inexpensive Servers
14
Mainframe Cheap Servers !http://1userverrack.net/
http://ibm.com/
Wednesday, May 25, 2011
7/28/2019 Nosql Ug Paris Imdg in Action With Coherence Without Transactions Chapter 110525091639 Phpapp02
http://slidepdf.com/reader/full/nosql-ug-paris-imdg-in-action-with-coherence-without-transactions-chapter-110525091639 15/76
Partition Data
15
MainFrame
Small servers
Partition gamma
Partition beta
Partition alpha
Partition for scalability
Wednesday, May 25, 2011
7/28/2019 Nosql Ug Paris Imdg in Action With Coherence Without Transactions Chapter 110525091639 Phpapp02
http://slidepdf.com/reader/full/nosql-ug-paris-imdg-in-action-with-coherence-without-transactions-chapter-110525091639 16/76
Duplicate Data
16
s y n c
s y
n c h ro ni z a t i o n
Duplicate data for high availability
Partition alpha
Master
Standby Backup
Wednesday, May 25, 2011
7/28/2019 Nosql Ug Paris Imdg in Action With Coherence Without Transactions Chapter 110525091639 Phpapp02
http://slidepdf.com/reader/full/nosql-ug-paris-imdg-in-action-with-coherence-without-transactions-chapter-110525091639 17/76
17
Data Access Patterns
Wednesday, May 25, 2011
7/28/2019 Nosql Ug Paris Imdg in Action With Coherence Without Transactions Chapter 110525091639 Phpapp02
http://slidepdf.com/reader/full/nosql-ug-paris-imdg-in-action-with-coherence-without-transactions-chapter-110525091639 18/76
Data Access Patterns
This is not traditional Java EE coding style !
Can apply very complex business logic inside thedata
18
Stored Procedures Style
Change management challenge !
Wednesday, May 25, 2011
7/28/2019 Nosql Ug Paris Imdg in Action With Coherence Without Transactions Chapter 110525091639 Phpapp02
http://slidepdf.com/reader/full/nosql-ug-paris-imdg-in-action-with-coherence-without-transactions-chapter-110525091639 19/76
19
Pattern : Targeted Operation
Wednesday, May 25, 2011
7/28/2019 Nosql Ug Paris Imdg in Action With Coherence Without Transactions Chapter 110525091639 Phpapp02
http://slidepdf.com/reader/full/nosql-ug-paris-imdg-in-action-with-coherence-without-transactions-chapter-110525091639 20/76
Pattern: Targeted Operation
20
Partition gamma
Search Trains
Partition beta
Search Trains
Partition alpha
Search Trains
{
"train-id": "tgv-3071-20110512",
"time" : 2011/05/12 12:15,
"departure" : "Paris",
"arrival" : "Marseille",
"seats" : 3,
}
Book Train Tickets
“train-id” is indexed
Wednesday, May 25, 2011
7/28/2019 Nosql Ug Paris Imdg in Action With Coherence Without Transactions Chapter 110525091639 Phpapp02
http://slidepdf.com/reader/full/nosql-ug-paris-imdg-in-action-with-coherence-without-transactions-chapter-110525091639 21/76
21
Pattern : Map Reduce Style Operation
Wednesday, May 25, 2011
7/28/2019 Nosql Ug Paris Imdg in Action With Coherence Without Transactions Chapter 110525091639 Phpapp02
http://slidepdf.com/reader/full/nosql-ug-paris-imdg-in-action-with-coherence-without-transactions-chapter-110525091639 22/76
Pattern: Map Reduce
22
Partition gamma
Search Trains
Partition beta
Search Trains
Partition alpha
Search Trains
{
"departure": "Paris",
"arrival": "Marseille",
"time" : 2011/05/12 12:00,
"seats" : 3,
}
Distributed “Search Train Ticket” Wednesday, May 25, 2011
7/28/2019 Nosql Ug Paris Imdg in Action With Coherence Without Transactions Chapter 110525091639 Phpapp02
http://slidepdf.com/reader/full/nosql-ug-paris-imdg-in-action-with-coherence-without-transactions-chapter-110525091639 23/76
Pattern: Map Reduce
23
Partition gamma
Search Trains
Partition beta
Search Trains
Partition alpha
Search Trains
{
"Paris -> Marseille : 12:15",
"Paris -> Marseille : 13:15"
}
Distributed “Search Train Ticket”
{ #NONE# }
{
"Paris -> Lyon -> Marseille : 12:40"
}
Wednesday, May 25, 2011
7/28/2019 Nosql Ug Paris Imdg in Action With Coherence Without Transactions Chapter 110525091639 Phpapp02
http://slidepdf.com/reader/full/nosql-ug-paris-imdg-in-action-with-coherence-without-transactions-chapter-110525091639 24/76
Pattern: Map Reduce
24
Partition gamma
Search Trains
Partition beta
Search Trains
Partition alpha
Search Trains
Distributed “Search Train Ticket”
{
"Paris -> Marseille : 12:15",
"Paris -> Lyon -> Marseille : 12:40",
"Paris -> Marseille : 13:15"
}
Wednesday, May 25, 2011
7/28/2019 Nosql Ug Paris Imdg in Action With Coherence Without Transactions Chapter 110525091639 Phpapp02
http://slidepdf.com/reader/full/nosql-ug-paris-imdg-in-action-with-coherence-without-transactions-chapter-110525091639 25/76
Data Access Patterns
This is not traditional Java EE coding style
Don’t forget “Map Reduce” = “Distributed TableScan”
25
Use Indexes
Change management
Wednesday, May 25, 2011
7/28/2019 Nosql Ug Paris Imdg in Action With Coherence Without Transactions Chapter 110525091639 Phpapp02
http://slidepdf.com/reader/full/nosql-ug-paris-imdg-in-action-with-coherence-without-transactions-chapter-110525091639 26/76
26
CAP Theorem & In Memory Data Grids
Wednesday, May 25, 2011
7/28/2019 Nosql Ug Paris Imdg in Action With Coherence Without Transactions Chapter 110525091639 Phpapp02
http://slidepdf.com/reader/full/nosql-ug-paris-imdg-in-action-with-coherence-without-transactions-chapter-110525091639 27/76
CAP Theorem and In Memory Data Grid
27
Consistency
Availability
Partition
Tolerance
Only 2 of these 3 properties can be
achieved at any given
moment in timeBrewer’s Conjecture
http://lpd.epfl.ch/sgilbert/pubs/BrewersConjecture-SigAct.pdf
Wednesday, May 25, 2011
7/28/2019 Nosql Ug Paris Imdg in Action With Coherence Without Transactions Chapter 110525091639 Phpapp02
http://slidepdf.com/reader/full/nosql-ug-paris-imdg-in-action-with-coherence-without-transactions-chapter-110525091639 28/76
CAP Theorem and In Memory Data Grid
28
Consistency
Availability
Partition
Tolerance
Only 2 of these 3 properties can be
achieved at any given
moment in timeBrewer’s Conjecture
http://lpd.epfl.ch/sgilbert/pubs/BrewersConjecture-SigAct.pdf
Data Grids
Wednesday, May 25, 2011
7/28/2019 Nosql Ug Paris Imdg in Action With Coherence Without Transactions Chapter 110525091639 Phpapp02
http://slidepdf.com/reader/full/nosql-ug-paris-imdg-in-action-with-coherence-without-transactions-chapter-110525091639 29/76
Cross Data Center Data Consistency
29
TokyoNew York
London
World wide replication
for financial marketWednesday, May 25, 2011
7/28/2019 Nosql Ug Paris Imdg in Action With Coherence Without Transactions Chapter 110525091639 Phpapp02
http://slidepdf.com/reader/full/nosql-ug-paris-imdg-in-action-with-coherence-without-transactions-chapter-110525091639 30/76
Cross Data Center Data Consistency
30
West Coast
East Coast
{
"name": "Barbie Computer",
"stock": 147,
"weigth" : 200
}
{ "name": "Barbie Computer",
"stock": 147,
"weigth" : 200
}
Warehouse stocks
Wednesday, May 25, 2011
7/28/2019 Nosql Ug Paris Imdg in Action With Coherence Without Transactions Chapter 110525091639 Phpapp02
http://slidepdf.com/reader/full/nosql-ug-paris-imdg-in-action-with-coherence-without-transactions-chapter-110525091639 31/76
Cross Data Center Data Consistency
31
propagation delay !
West Coast
East Coast
{
"name": "Barbie Computer",
"stock": 147,
"weigth" : 200
}
set stock to 146
{ "name": "Barbie Computer",
"stock": 147,
"weigth" : 200
}
Wednesday, May 25, 2011
7/28/2019 Nosql Ug Paris Imdg in Action With Coherence Without Transactions Chapter 110525091639 Phpapp02
http://slidepdf.com/reader/full/nosql-ug-paris-imdg-in-action-with-coherence-without-transactions-chapter-110525091639 32/76
Cross Data Center Data Consistency
32
West Coast
East Coast
{
"name": "Barbie Computer",
"stock": 147,
"weigth" : 200
}
set stock to 146
{ "name": "Barbie Computer",
"stock": 147,
"weigth" : 200
}
set weight 175
reconciliation API needed !
Wednesday, May 25, 2011
7/28/2019 Nosql Ug Paris Imdg in Action With Coherence Without Transactions Chapter 110525091639 Phpapp02
http://slidepdf.com/reader/full/nosql-ug-paris-imdg-in-action-with-coherence-without-transactions-chapter-110525091639 33/76
Cross Data Center Data Consistency
33
West Coast
East Coast
{
"name": "Barbie Computer",
"stock": 147,
"weigth" : 200
}
set stock to 146
{ "name": "Barbie Computer",
"stock": 147,
"weigth" : 200
}
set weight 175
Network partitioning
Wednesday, May 25, 2011
7/28/2019 Nosql Ug Paris Imdg in Action With Coherence Without Transactions Chapter 110525091639 Phpapp02
http://slidepdf.com/reader/full/nosql-ug-paris-imdg-in-action-with-coherence-without-transactions-chapter-110525091639 34/76
34
Data Modeling
Wednesday, May 25, 2011
7/28/2019 Nosql Ug Paris Imdg in Action With Coherence Without Transactions Chapter 110525091639 Phpapp02
http://slidepdf.com/reader/full/nosql-ug-paris-imdg-in-action-with-coherence-without-transactions-chapter-110525091639 35/76
Data Modeling
Dominant Question Driven Design
Constrained Tree Schema
Denormalized
35
Opposite to Relational which is Domain Driven Design
Because RPC matters
Due to dominant questions and CTS
Wednesday, May 25, 2011
7/28/2019 Nosql Ug Paris Imdg in Action With Coherence Without Transactions Chapter 110525091639 Phpapp02
http://slidepdf.com/reader/full/nosql-ug-paris-imdg-in-action-with-coherence-without-transactions-chapter-110525091639 36/76
Data Modeling
36
TrainStop
date
TrainStation
code
name
Traincode
type
Seat
number
price
Booking
reduction
Passenger
name
Typical relational data model
Wednesday, May 25, 2011
7/28/2019 Nosql Ug Paris Imdg in Action With Coherence Without Transactions Chapter 110525091639 Phpapp02
http://slidepdf.com/reader/full/nosql-ug-paris-imdg-in-action-with-coherence-without-transactions-chapter-110525091639 37/76
Data Modeling
37
Find the root entity and denormalize
TrainStop
date
Seat
number
price
Booking
reduction
Passenger
name
R e f e r e n c e d a t a
D u p l i c a t e d i n e a c h g r i d n o d e
TrainStation
code
name
R o o t e n t i
t y
Partitioning readyentities tree
Train
code
type
Wednesday, May 25, 2011
7/28/2019 Nosql Ug Paris Imdg in Action With Coherence Without Transactions Chapter 110525091639 Phpapp02
http://slidepdf.com/reader/full/nosql-ug-paris-imdg-in-action-with-coherence-without-transactions-chapter-110525091639 38/76
Data Modeling
38
Remove unused data
TrainStop
date
Seat
number
price
Booking
reduction
Passenger
name
booked
TrainStation
codename
Train
code
type
Partitioned
Replicated
Wednesday, May 25, 2011
7/28/2019 Nosql Ug Paris Imdg in Action With Coherence Without Transactions Chapter 110525091639 Phpapp02
http://slidepdf.com/reader/full/nosql-ug-paris-imdg-in-action-with-coherence-without-transactions-chapter-110525091639 39/76
Data Modeling
39
TrainStop
date
TrainStation
code
name
Seat
number
price
bookedTrain
codetype
Data Grid Ready data structure
Partitioned
Replicated
Wednesday, May 25, 2011
7/28/2019 Nosql Ug Paris Imdg in Action With Coherence Without Transactions Chapter 110525091639 Phpapp02
http://slidepdf.com/reader/full/nosql-ug-paris-imdg-in-action-with-coherence-without-transactions-chapter-110525091639 40/76
40
Data Modeling is Hard !
Wednesday, May 25, 2011
7/28/2019 Nosql Ug Paris Imdg in Action With Coherence Without Transactions Chapter 110525091639 Phpapp02
http://slidepdf.com/reader/full/nosql-ug-paris-imdg-in-action-with-coherence-without-transactions-chapter-110525091639 41/76
Data Modeling is Hard !
41
Two root entities for thesame MoneyTransfer !
from to
CashWitdrawal
date
amount
MoneyTransfer
id
date
amount
Account
number
CashWitdrawal
date
amount
Account
number
Wednesday, May 25, 2011
7/28/2019 Nosql Ug Paris Imdg in Action With Coherence Without Transactions Chapter 110525091639 Phpapp02
http://slidepdf.com/reader/full/nosql-ug-paris-imdg-in-action-with-coherence-without-transactions-chapter-110525091639 42/76
Data Modeling is Hard !
42
CashWitdrawal
date
amount
CashWitdrawal
date
amount
MoneyTransferIn
id
date
amount
MoneyTransferOut
id
date
amount
Account
number
Account
number
Split MoneyTransfer
Wednesday, May 25, 2011
7/28/2019 Nosql Ug Paris Imdg in Action With Coherence Without Transactions Chapter 110525091639 Phpapp02
http://slidepdf.com/reader/full/nosql-ug-paris-imdg-in-action-with-coherence-without-transactions-chapter-110525091639 43/76
Data Modeling is Hard !
43
CashWitdrawal
date
amount
MoneyTransferOut
id
date
amount
Account
number
CashWitdrawal
date
amount
MoneyTransferIn
id
date
amount
Account
number
Split MoneyTransfer
Wednesday, May 25, 2011
7/28/2019 Nosql Ug Paris Imdg in Action With Coherence Without Transactions Chapter 110525091639 Phpapp02
http://slidepdf.com/reader/full/nosql-ug-paris-imdg-in-action-with-coherence-without-transactions-chapter-110525091639 44/76
Data Modeling is Hard !
44
CashWitdrawal
date
amount
MoneyTransferOut
id
date
amount
MoneyTransferIn
id
date
amount
Account
number
Data Grid Ready data structure
Wednesday, May 25, 2011
7/28/2019 Nosql Ug Paris Imdg in Action With Coherence Without Transactions Chapter 110525091639 Phpapp02
http://slidepdf.com/reader/full/nosql-ug-paris-imdg-in-action-with-coherence-without-transactions-chapter-110525091639 45/76
45
Grid Internals
Wednesday, May 25, 2011
7/28/2019 Nosql Ug Paris Imdg in Action With Coherence Without Transactions Chapter 110525091639 Phpapp02
http://slidepdf.com/reader/full/nosql-ug-paris-imdg-in-action-with-coherence-without-transactions-chapter-110525091639 46/76
Data Serialization
Used for data transfer and byte oriented storage
Hot topic like Apache Thrift, Apache Avro, GoogleProtocol Buffer
46
Must support evolvable data structure
Wednesday, May 25, 2011
7/28/2019 Nosql Ug Paris Imdg in Action With Coherence Without Transactions Chapter 110525091639 Phpapp02
http://slidepdf.com/reader/full/nosql-ug-paris-imdg-in-action-with-coherence-without-transactions-chapter-110525091639 47/76
Data Storage
Store Java Beans in the grid
Store byte arrays in the grid
47
No need to unmarshall for inprocess operations
Beware of garbage collector !
Pay unmarshalling at each read and write
Slightly more garbage collector friendly
Low-level / byte-oriented APIs to read data
Wednesday, May 25, 2011
7/28/2019 Nosql Ug Paris Imdg in Action With Coherence Without Transactions Chapter 110525091639 Phpapp02
http://slidepdf.com/reader/full/nosql-ug-paris-imdg-in-action-with-coherence-without-transactions-chapter-110525091639 48/76
Communication Protocols
UDP Multi Cast (Coherence, Gigaspaces)
TCP/IP (Websphere eXtreme Scale)
48
Wednesday, May 25, 2011
7/28/2019 Nosql Ug Paris Imdg in Action With Coherence Without Transactions Chapter 110525091639 Phpapp02
http://slidepdf.com/reader/full/nosql-ug-paris-imdg-in-action-with-coherence-without-transactions-chapter-110525091639 49/76
Topology
Partitions made of shards : 1 primary + 0..*backups)
Dynamic shards location (changes at runtime andat restart)
Can use dedicated “directory servers” or embed it
in the “data nodes”
49
Wednesday, May 25, 2011
7/28/2019 Nosql Ug Paris Imdg in Action With Coherence Without Transactions Chapter 110525091639 Phpapp02
http://slidepdf.com/reader/full/nosql-ug-paris-imdg-in-action-with-coherence-without-transactions-chapter-110525091639 50/76
JVM and Memory
Many editors recommend tiny 1.4 Go JVM !
More than ten JVM per server
50
Garbage collector hell
Management hell
More and more IMDG support large heaps
Wednesday, May 25, 2011
7/28/2019 Nosql Ug Paris Imdg in Action With Coherence Without Transactions Chapter 110525091639 Phpapp02
http://slidepdf.com/reader/full/nosql-ug-paris-imdg-in-action-with-coherence-without-transactions-chapter-110525091639 51/76
51
APIs
Wednesday, May 25, 2011
7/28/2019 Nosql Ug Paris Imdg in Action With Coherence Without Transactions Chapter 110525091639 Phpapp02
http://slidepdf.com/reader/full/nosql-ug-paris-imdg-in-action-with-coherence-without-transactions-chapter-110525091639 52/76
Raw Java Mapping with Oracle Coherence
52
hand-coded serialization JUnit is your friend !
public class Train extends AbstractEvolvable implements PortableObject {
enum Type {
HIGH_SPEED, NORMAL
}
/** Key of the Cache */
String code;
/** Indexed */
String name;
Type type;
List<Seat> seats = new ArrayList<Seat>();
int version;
List<TrainStop> trainStops = new ArrayList<TrainStop>();
@Override
public int getImplVersion() {
return 1;
}
@Override
public void readExternal(PofReader pofReader) throws IOException {
this.code = pofReader.readString(0);
this.name = pofReader.readString(1);
this.type = (Type) pofReader.readObject(2);
pofReader.readCollection(3, this.seats);
pofReader.readCollection(4, this.trainStops); this.version = pofReader.readInt(5);
}
@Override
public void writeExternal(PofWriter pofWriter) throws IOException {
pofWriter.writeString(0, this.code);
pofWriter.writeString(1, this.name);
pofWriter.writeObject(2, this.type);
pofWriter.writeCollection(3, this.seats, Seat.class);
pofWriter.writeCollection(4, this.trainStops, TrainStop.class);
pofWriter.writeInt(5, this.version);
}}
TrainStopdate
Seat
number
price
bookedTrain
code
type
Wednesday, May 25, 2011
JPA Style Mapping with Websphere eXtreme
7/28/2019 Nosql Ug Paris Imdg in Action With Coherence Without Transactions Chapter 110525091639 Phpapp02
http://slidepdf.com/reader/full/nosql-ug-paris-imdg-in-action-with-coherence-without-transactions-chapter-110525091639 53/76
JPA Style Mapping with Websphere eXtremeScale
53
sub entities can havecross relations
@Entity(schemaRoot=true)
public class Train {
@Id
String code;
@Index
@Basic
String name;
@OneToMany(cascade=CascadeType. ALL)
List<Seat> seats = new ArrayList<Seat>();
@Version
int version;
...}
TrainStopdate
Seat
number
price
bookedTrain
code
type
Wednesday, May 25, 2011
7/28/2019 Nosql Ug Paris Imdg in Action With Coherence Without Transactions Chapter 110525091639 Phpapp02
http://slidepdf.com/reader/full/nosql-ug-paris-imdg-in-action-with-coherence-without-transactions-chapter-110525091639 54/76
Map API with Oracle Coherence
54
NamedCache trainCache = CacheFactory.getCache("train-cache");
/** Save */
void persist(Train train) {
trainCache.put(train.getCode(), train);
}
/** Find by key */
Train findByCode(String code) {
return (Train) trainCache.get(code);
}
/** Find by Query Language */
Train findByTrainName(String name) {
Filter filter = QueryHelper.createFilter("name = :name" ,
Collections.singletonMap("name", name));
Set<Map.Entry<String, Train>> trainEntrySet = trainCache.entrySet(filter);
if (trainEntrySet.isEmpty()) {
return null;} else {
return trainEntrySet.iterator().next().getValue();
}
}
Map API
Wednesday, May 25, 2011
7/28/2019 Nosql Ug Paris Imdg in Action With Coherence Without Transactions Chapter 110525091639 Phpapp02
http://slidepdf.com/reader/full/nosql-ug-paris-imdg-in-action-with-coherence-without-transactions-chapter-110525091639 55/76
JPA Style with Websphere eXtreme Scale
55
/** Save */void persist(Train train) {
entityManager.persist(train);
}
/** Find by key */
Train findByCode(String code) {
return (Train) entityManager.find(Train.class, code);}
/** Query Language */
Train findByTrainName(String name) {
Query q = entityManager.createQuery("select t from Train t where t.name=:name");
q.setParameter("name", name);
return (Train) q.getSingleResult();
}
JPA Style Entity Manager
Wednesday, May 25, 2011
7/28/2019 Nosql Ug Paris Imdg in Action With Coherence Without Transactions Chapter 110525091639 Phpapp02
http://slidepdf.com/reader/full/nosql-ug-paris-imdg-in-action-with-coherence-without-transactions-chapter-110525091639 56/76
Creating Indexes
56
Map reduce (without index) = Distributed Table Scan !
Wednesday, May 25, 2011
7/28/2019 Nosql Ug Paris Imdg in Action With Coherence Without Transactions Chapter 110525091639 Phpapp02
http://slidepdf.com/reader/full/nosql-ug-paris-imdg-in-action-with-coherence-without-transactions-chapter-110525091639 57/76
Indexes with Oracle Coherence
57
class Train {
String name;
Collection<String> getTrainStationsCodes() {
return Collections2.transform(trainStops, ...);
}
...
}
{
NamedCache trainCache = CacheFactory.getCache("train-cache");
trainCache.addIndex(new ReflectionExtractor("getName"), false, null);
trainCache.addIndex(new ReflectionExtractor("getTrainStationsCodes"), false, null);
}
Wednesday, May 25, 2011
d i h b h S l
7/28/2019 Nosql Ug Paris Imdg in Action With Coherence Without Transactions Chapter 110525091639 Phpapp02
http://slidepdf.com/reader/full/nosql-ug-paris-imdg-in-action-with-coherence-without-transactions-chapter-110525091639 58/76
Indexes with Websphere eXtreme Scale
58
@Entity(schemaRoot=true)
class Train {
@Index
@Basic
String name;
@Index
Collection<String> getTrainStationsCodes() {
return Collections2.transform(trainStops, ...);
}
...
}
Query query = em.createQuery("select t from Train t where t.name=:name");
query.getPlan();
eXtreme Scale
for q2 in Train ObjectMap using INDEX on name = ( ?name)
filter ( q2.c[0] = ?name )
returning new Tuple( q2 )
This is an execution plan
Wednesday, May 25, 2011
M API
7/28/2019 Nosql Ug Paris Imdg in Action With Coherence Without Transactions Chapter 110525091639 Phpapp02
http://slidepdf.com/reader/full/nosql-ug-paris-imdg-in-action-with-coherence-without-transactions-chapter-110525091639 59/76
More APIs
Another Java EE versus Spring battle ?
JSR 347 Data Grids vs. Spring Data
59
Unified API ontop of NoSQL stores ?
Serialization / Object to Tuple Mapping API ?
Wednesday, May 25, 2011
7/28/2019 Nosql Ug Paris Imdg in Action With Coherence Without Transactions Chapter 110525091639 Phpapp02
http://slidepdf.com/reader/full/nosql-ug-paris-imdg-in-action-with-coherence-without-transactions-chapter-110525091639 60/76
60
Data Grid <-> Relational Database Interactions
Wednesday, May 25, 2011
D t G id R l ti l D t b
7/28/2019 Nosql Ug Paris Imdg in Action With Coherence Without Transactions Chapter 110525091639 Phpapp02
http://slidepdf.com/reader/full/nosql-ug-paris-imdg-in-action-with-coherence-without-transactions-chapter-110525091639 61/76
Data Grid <-> Relational Database
61
Data Grids are “In Memory” -> we need to persist data on disk !
Wednesday, May 25, 2011
D t G id < > R l ti l D t b
7/28/2019 Nosql Ug Paris Imdg in Action With Coherence Without Transactions Chapter 110525091639 Phpapp02
http://slidepdf.com/reader/full/nosql-ug-paris-imdg-in-action-with-coherence-without-transactions-chapter-110525091639 62/76
Data Grid <-> Relational Database
62
update / insert / delete
“select directly modified in DB”
Wednesday, May 25, 2011
D t G id < > R l ti l D t b
7/28/2019 Nosql Ug Paris Imdg in Action With Coherence Without Transactions Chapter 110525091639 Phpapp02
http://slidepdf.com/reader/full/nosql-ug-paris-imdg-in-action-with-coherence-without-transactions-chapter-110525091639 63/76
Data Grid <-> Relational Database
63
backend DB
Highly available write behind queues + SQL batched statements
Data Grid -> Relational Database
Wednesday, May 25, 2011
D t G id < > R l ti l D t b
7/28/2019 Nosql Ug Paris Imdg in Action With Coherence Without Transactions Chapter 110525091639 Phpapp02
http://slidepdf.com/reader/full/nosql-ug-paris-imdg-in-action-with-coherence-without-transactions-chapter-110525091639 64/76
Data Grid <-> Relational Database
64
TrainStop
date
TrainStation
code
name
Seat
number
price
bookedTrain
code
type
Constrained Tree Schema <-> Relational Impedance Mismatch
Data Grid -> Relational Database
Wednesday, May 25, 2011
Data G id < > Relational Database
7/28/2019 Nosql Ug Paris Imdg in Action With Coherence Without Transactions Chapter 110525091639 Phpapp02
http://slidepdf.com/reader/full/nosql-ug-paris-imdg-in-action-with-coherence-without-transactions-chapter-110525091639 65/76
Data Grid <-> Relational Database
DB writes MUST succeed !
65
Align the database on the Data Grid model !
Denormalize the database
Remove the foreign keys, use same PKs in DB and data grid
Support unordered SQL statements
Prefer raw SQL rather than reused business logic
Wednesday, May 25, 2011
Data Grid < > Relational Database
7/28/2019 Nosql Ug Paris Imdg in Action With Coherence Without Transactions Chapter 110525091639 Phpapp02
http://slidepdf.com/reader/full/nosql-ug-paris-imdg-in-action-with-coherence-without-transactions-chapter-110525091639 66/76
Data Grid <-> Relational Database
66
backend DB
Data Grid Originated Scheduled Refresh (Oracle System Change Number, etc)
select * from train where last_modif > ?
Relational Database -> Data Grid
Wednesday, May 25, 2011
Data Grid < > Relational Database
7/28/2019 Nosql Ug Paris Imdg in Action With Coherence Without Transactions Chapter 110525091639 Phpapp02
http://slidepdf.com/reader/full/nosql-ug-paris-imdg-in-action-with-coherence-without-transactions-chapter-110525091639 67/76
Data Grid <-> Relational Database
67
backend DB
Database Originated Push JMS = durable subscription
(Oracle Database Change Notification, etc)
Relational Database -> Data Grid
Wednesday, May 25, 2011
Data Grid < > Relational Database
7/28/2019 Nosql Ug Paris Imdg in Action With Coherence Without Transactions Chapter 110525091639 Phpapp02
http://slidepdf.com/reader/full/nosql-ug-paris-imdg-in-action-with-coherence-without-transactions-chapter-110525091639 68/76
Data Grid <-> Relational Database
In Memory -> prepare for reloading aftermaintenance operations !
Prepare consistency checkers
68
Need for “graceful shutdown with disk persistence”
Wednesday, May 25, 2011
7/28/2019 Nosql Ug Paris Imdg in Action With Coherence Without Transactions Chapter 110525091639 Phpapp02
http://slidepdf.com/reader/full/nosql-ug-paris-imdg-in-action-with-coherence-without-transactions-chapter-110525091639 69/76
69
Transactions
Wednesday, May 25, 2011
7/28/2019 Nosql Ug Paris Imdg in Action With Coherence Without Transactions Chapter 110525091639 Phpapp02
http://slidepdf.com/reader/full/nosql-ug-paris-imdg-in-action-with-coherence-without-transactions-chapter-110525091639 70/76
70
We didn’t have the time to talk abouttransaction.
Another session is planned at Paris No SQLUser Group for this.
Wednesday, May 25, 2011
7/28/2019 Nosql Ug Paris Imdg in Action With Coherence Without Transactions Chapter 110525091639 Phpapp02
http://slidepdf.com/reader/full/nosql-ug-paris-imdg-in-action-with-coherence-without-transactions-chapter-110525091639 71/76
71
Let’s go live !
Wednesday, May 25, 2011
Data Grids and Operations
7/28/2019 Nosql Ug Paris Imdg in Action With Coherence Without Transactions Chapter 110525091639 Phpapp02
http://slidepdf.com/reader/full/nosql-ug-paris-imdg-in-action-with-coherence-without-transactions-chapter-110525091639 72/76
Data Grids and Operations
Standard packaging?
Limited Management
Limited debugging tools
JVM pandemia
72
Do It Yourself (layout, scripts, etc)
Do It Yourself (stop/start, detecting data loss, etc)
Dozens of JVM to manage !
Do It Yourself (debugging consoles, troubleshooting agents)
Wednesday, May 25, 2011
Data Grids and Operations
7/28/2019 Nosql Ug Paris Imdg in Action With Coherence Without Transactions Chapter 110525091639 Phpapp02
http://slidepdf.com/reader/full/nosql-ug-paris-imdg-in-action-with-coherence-without-transactions-chapter-110525091639 73/76
Data Grids and Operations
Dev / Ops collaboration is required
Experts only !
73
Wednesday, May 25, 2011
7/28/2019 Nosql Ug Paris Imdg in Action With Coherence Without Transactions Chapter 110525091639 Phpapp02
http://slidepdf.com/reader/full/nosql-ug-paris-imdg-in-action-with-coherence-without-transactions-chapter-110525091639 74/76
74
The right tool for the right job
Wednesday, May 25, 2011
The right tool for the right job
7/28/2019 Nosql Ug Paris Imdg in Action With Coherence Without Transactions Chapter 110525091639 Phpapp02
http://slidepdf.com/reader/full/nosql-ug-paris-imdg-in-action-with-coherence-without-transactions-chapter-110525091639 75/76
The right tool for the right job
Incredibly fast ! Even with transactions !
Scalable
Good at data replication (when it implements it)
Very geeky on both dev and ops side
“Quite” expensive
75
Not an enterprise grade data store
Reconciliation api, etc
Requires very skilled people + change management
If you solve the data loading issue
Wednesday, May 25, 2011
Questions / Answers
7/28/2019 Nosql Ug Paris Imdg in Action With Coherence Without Transactions Chapter 110525091639 Phpapp02
http://slidepdf.com/reader/full/nosql-ug-paris-imdg-in-action-with-coherence-without-transactions-chapter-110525091639 76/76
?
Questions / Answers