telecom universal datastatesharingfabric
Post on 14-Dec-2014
146 Views
Preview:
DESCRIPTION
TRANSCRIPT
A universal Data & State sharing FabricMarch 2013
Shay HassidimDeputy CTO
1
2
Agenda
• Company Snapshot• Shared Data & State Fabric
– Problem Statement – Existing Solutions – Pros & Cons– IMDG – Fundamentals & Case Study
GigaSpaces – Telco Customers and Partners
3
Founded 2000 , 200 Customers , Sales: US , EU , APAC , RND: TLV , Main Investor: Intel
4
Problem Description• Telecom systems internal systems currently use a variety of
MOM (AMQP, XMPP) , IM protocols , Network based protocols, Application level protocols (SIP), HTML based protocols (Rest, SOAP) or DB (RDBMS, No SQL) to communicate with each other.– Introduce complexity, cost/ROI concerns, increased time to
market, impact customer satisfaction
• A universal data & state sharing fabric connecting the different products / systems / components is required– Real time response– Total reliability– Scalable & Highly Available
5
Possible Solutions - #1:Database
• Pros– Everybody knows databases. No need to educate the
developers.• Cons
– Users need to reshape the data structure to use a specific model (Table , Document). Lock-in.
– No proactive event based messaging fabric. Requires external messaging integration
– Management overhead– Not designed for real-time communication. Mostly
disk based storage medium.
Possible Solutions - #2:ESB
• Pros– Designed for data transformation and
routing between alien systems – Protocol translation focused
• Cons– Does not maintain state. No state sharing.
Getting the latest state requires a DB/Cache– Not designed for real-time communication
6
7
What if we could combine both DB and ESB but avoid all the
cons?+ =
Enter IMDG• Roots in the Tuple Space concept, RDBMS, OODBMS,
MOM• Pros
– State sharing in real time in Massive scale– Built-in proactive event based fabric
• Fused to the engine core. No external subsystem– Support both Key/Value, SQL Based data access and
Message deliver modes with the same API• Topic , Queue, Pub/Sub
– Support locality of reference• Collocate Data and business logic
• Cons– No full SQL Query support as a RDBMS– Requires some education/ramp up
8
9
GigaSpaces IMDG Basic Operations
Application
Space
Take
Application
Space
Read
Application
Space
WriteMultiple
Application
Space
Write
Application
Space
ReadMultiple
Application
Space
TakeMultiple
Application
Space
Execute
Application
Space
Notify
http://www.gigaspaces.com/docs/JavaDoc9.1/org/openspaces/core/GigaSpace.html
Application
Space
Changeset
putInMapincrement
deccrementaddToCollection
10
POJOJavaSpaceMapJPAJMSJDBC
IMDG Real-Time InteroperabilityDocument
IMDG
Application-IMDG Topologies
ApplicationProcess
IMDG Process
Remote call
Application Process
IMDG Instance
Application
Local call
Remote
Collocated
11
0
50
100
150
200
250
300
5
200
10
300
180
GigaSpaces IMDG Latency BenchmarkM
icro
seco
nds
IMDG Real-Time Low-Latency Performance
http://blog.gigaspaces.com/2010/12/06/possible-impossibility-the-race-to-zero-latency/
Async operationNo Serialization
No Network usage Async operation
12
Benchmark using Cisco UCS , 2.93GHz CPU
13
Cassandra vs. GigaSpaces
1 2 3 4 5 6 7 8 100
10,000,000
20,000,000
30,000,000
40,000,000
50,000,000
0
5,000
10,000
15,000
20,000
25,000
Cassandra vs. GigaSpaces Read Benchmark
# of Client Threads
Gig
aSpa
ces
TPS
Cass
andr
a TP
S
1 2 3 4 5 6 7 8 100
200,000
400,000
600,000
800,000
1,000,000
1,200,000
1,400,000
020,00040,00060,00080,000100,000120,000140,000160,000180,000
Cassandra vs. GigaSpaces Write Benchmark
# of Client Threads
Gig
aSpa
ces
TPS
Cass
andr
a TP
S
GigaSpaces is 1000-2000 times faster
GigaSpaces is 3-7 times faster
Benchmark using Cisco UCS , 2.93GHz CPU
ESB Performance
14
http://esbperformance.org/display/comparison/ESB+Performance - August 2012
2 orders of magnitude slower than IMDG
15
IMDG Use Cases
• Database Performance, Scaling • State replication for correlation• Distributed Stateful Processing• State replication over the WAN
• Description– Device activity history – Last n activities
• Challenge– Central DB bottleneck
• Desired solution– Use a distributed cache to front end the database– Preload the data from the DB– Keep the DB in-synch
• Requirements– High consistency– High Performance (<10,000 TX/Sec)
Database Performance, Scaling
16
• Use IMDG to front-end the database
• Keep the DB in-synch– Load the data into memory
using Object based model– Write behind all updates
• Use the IMDG as the system of record– Rich Query (SQL,..)– Transaction support
Database Performance, Scaling Solution Architecture
17
Application
Database 1
load
Database 2 Database 3
Write behind
• Description– State correlation
• Challenge– High availability
• Desired solution– Peer model , multi-master architecture
• Requirements– CAP based consistency
State Replication for correlation
18
• Store correlation state in the IMDG
• Can use replicated or partitioned IMDG topology
• Can use Client-side cache for greater performance optimization of read operation
State Replication for correlationSolution Architecture
19
Application
Application
Application
• Description– Workflow of distributed process– 5-10 steps per process
• Challenge– State information is too big to transfer
• Desired solution– Shared consistent state
• Requirements– Scale to deal with large amount of nodes (>1000)
Distributed Stateful Processing
20
• Store workflow state in the IMDG
• Use pub/sub model to synchronize the state
• Use Template matching / SQL for querying the state of a particular object in the IMDG
Distributed Stateful ProcessingSolution Architecture
POCESS A
STATE A PRCESS BSTATE
BPROCESS
CSTATE
C
21
22
Sharing Device State Across Different DC
• Description– Systems/devices running across different data centers– Need to share state in real time
• Challenge– Requires reliable and scalable data replication over the WAN
• Desired solution– Simple secured point2point state replication over the WAN
• Requirements– Support low-bandwidth , high-latency networks
Replication over the WAN SupportSolution Architecture
London
Hong Kong
New York
23
GigaSpaces IMDG WAN Replication Features
24
Any Replication Topology
High Availability
Data Filtering
Cloud Enabled
Fully Transactional
Custom Conflict Resolution
Interoperable
Single Click Bootstrapping
Security Optimized Connection
XAP – One Product for In Memory Computing
Java-.NET-C++Spring, JPA,JMS
JDBC, MapSchema-Free
One Model for all components
-Clustering -Security
-HA -OA&M
Virtualize All Middleware Components
Customize Application
Management Rules &
Workflows
Real-Time Automated Deployment Monitoring
Management
25
Consistent Management & Monitoring Module (“Cloudify”)
26
Application recipe uses domain-specific language (DSL) to describe the application life cycle
All necessary plumbing provided out of the box
Configuration and setup separate from the application
recipe
A Typical App…
28
Extensive Platform Support
29
Summary – GigaSpaces IMDG
• Designed for real-time data-driven interoperability
• Management of data-centric architecture made easy
• Supports high-end, complex applications• Blazing fast. Highly-Available.
30
Thank You!www.gigaspaces.com
top related