infinispan in 50 minutes - jboss...•5.2 - february 2013 •non-blocking state transfer...
TRANSCRIPT
![Page 1: Infinispan in 50 minutes - JBoss...•5.2 - February 2013 •non-blocking state transfer •cross-site replication •rolling upgrades for hotrod-clients •map/reduce marked stable](https://reader035.vdocuments.us/reader035/viewer/2022070822/5f26b7cb51d8ec66c205aed6/html5/thumbnails/1.jpg)
![Page 2: Infinispan in 50 minutes - JBoss...•5.2 - February 2013 •non-blocking state transfer •cross-site replication •rolling upgrades for hotrod-clients •map/reduce marked stable](https://reader035.vdocuments.us/reader035/viewer/2022070822/5f26b7cb51d8ec66c205aed6/html5/thumbnails/2.jpg)
Infinispan in 50 minutesSanne Grinovero
![Page 3: Infinispan in 50 minutes - JBoss...•5.2 - February 2013 •non-blocking state transfer •cross-site replication •rolling upgrades for hotrod-clients •map/reduce marked stable](https://reader035.vdocuments.us/reader035/viewer/2022070822/5f26b7cb51d8ec66c205aed6/html5/thumbnails/3.jpg)
Sanne Grinovero•Senior Software Engineer at Red Hat•Hibernate team
• lead of Hibernate Search• Hibernate OGM
•Infinispan• Search, Query and Lucene integrations
•Apache Lucene•JGroups
Who’s this guy?
![Page 4: Infinispan in 50 minutes - JBoss...•5.2 - February 2013 •non-blocking state transfer •cross-site replication •rolling upgrades for hotrod-clients •map/reduce marked stable](https://reader035.vdocuments.us/reader035/viewer/2022070822/5f26b7cb51d8ec66c205aed6/html5/thumbnails/4.jpg)
Agenda•What is Infinispan•API•Key features•Three use cases•The path ahead
![Page 5: Infinispan in 50 minutes - JBoss...•5.2 - February 2013 •non-blocking state transfer •cross-site replication •rolling upgrades for hotrod-clients •map/reduce marked stable](https://reader035.vdocuments.us/reader035/viewer/2022070822/5f26b7cb51d8ec66c205aed6/html5/thumbnails/5.jpg)
Infinispanl Open source highly scalable data grid platform
l Distribution or Replicationl Sync or Asyncl Transactionall Persists contents using a CacheLoader
− Write-through or write-behind− Shared or per cluster node
l Hibernate second-level cachel State of the art eviction strategies
![Page 6: Infinispan in 50 minutes - JBoss...•5.2 - February 2013 •non-blocking state transfer •cross-site replication •rolling upgrades for hotrod-clients •map/reduce marked stable](https://reader035.vdocuments.us/reader035/viewer/2022070822/5f26b7cb51d8ec66c205aed6/html5/thumbnails/6.jpg)
Distributed Data Structure
Storage Engine (RAM
+Overflow)
Transaction
Query
Map/Reduce
Monitoring
Transport (JGroups)
Memcached Hotrod Rest
Node (JVM process)
Storage Engine (RAM
+Overflow)
Transaction
Query
Map/Reduce
Monitoring
Transport (JGroups)
Memcached Hotrod Rest
Node (JVM process)
TCP/UDP
RemoteClient
TCPTCP
![Page 7: Infinispan in 50 minutes - JBoss...•5.2 - February 2013 •non-blocking state transfer •cross-site replication •rolling upgrades for hotrod-clients •map/reduce marked stable](https://reader035.vdocuments.us/reader035/viewer/2022070822/5f26b7cb51d8ec66c205aed6/html5/thumbnails/7.jpg)
High Availability•Memory is volatile•Make redundant copies
•Total replication (Replication Mode)•Partial replication (Distribution Mode)
• numOwners•Topology changes
•Nodes will crash!•Re-arrange state
![Page 8: Infinispan in 50 minutes - JBoss...•5.2 - February 2013 •non-blocking state transfer •cross-site replication •rolling upgrades for hotrod-clients •map/reduce marked stable](https://reader035.vdocuments.us/reader035/viewer/2022070822/5f26b7cb51d8ec66c205aed6/html5/thumbnails/8.jpg)
Elasticity•Expect
•Node additions•Node removals
•Topology changes •are totally consistent•do not "stop the world"
![Page 9: Infinispan in 50 minutes - JBoss...•5.2 - February 2013 •non-blocking state transfer •cross-site replication •rolling upgrades for hotrod-clients •map/reduce marked stable](https://reader035.vdocuments.us/reader035/viewer/2022070822/5f26b7cb51d8ec66c205aed6/html5/thumbnails/9.jpg)
![Page 10: Infinispan in 50 minutes - JBoss...•5.2 - February 2013 •non-blocking state transfer •cross-site replication •rolling upgrades for hotrod-clients •map/reduce marked stable](https://reader035.vdocuments.us/reader035/viewer/2022070822/5f26b7cb51d8ec66c205aed6/html5/thumbnails/10.jpg)
API•Key-value store
• java.util.concurrent.ConcurrentHashMap•JSR-107 compliant•CDI support•Spring support
![Page 11: Infinispan in 50 minutes - JBoss...•5.2 - February 2013 •non-blocking state transfer •cross-site replication •rolling upgrades for hotrod-clients •map/reduce marked stable](https://reader035.vdocuments.us/reader035/viewer/2022070822/5f26b7cb51d8ec66c205aed6/html5/thumbnails/11.jpg)
Integration•Hibernate integration
•2nd level cache•Lucene integration
•Query• index storage
•ModeShape•CapeDwarf•CacheStores
•Cassandra, MongoDB, ...
![Page 12: Infinispan in 50 minutes - JBoss...•5.2 - February 2013 •non-blocking state transfer •cross-site replication •rolling upgrades for hotrod-clients •map/reduce marked stable](https://reader035.vdocuments.us/reader035/viewer/2022070822/5f26b7cb51d8ec66c205aed6/html5/thumbnails/12.jpg)
Key features•Transactions•Performance•Persistence•Map/Reduce•Querying
![Page 13: Infinispan in 50 minutes - JBoss...•5.2 - February 2013 •non-blocking state transfer •cross-site replication •rolling upgrades for hotrod-clients •map/reduce marked stable](https://reader035.vdocuments.us/reader035/viewer/2022070822/5f26b7cb51d8ec66c205aed6/html5/thumbnails/13.jpg)
Big Data / Fast Data ?
![Page 14: Infinispan in 50 minutes - JBoss...•5.2 - February 2013 •non-blocking state transfer •cross-site replication •rolling upgrades for hotrod-clients •map/reduce marked stable](https://reader035.vdocuments.us/reader035/viewer/2022070822/5f26b7cb51d8ec66c205aed6/html5/thumbnails/14.jpg)
Transactions•JTA transactions support
•XA or Synchronization based enlistment• recovery!
•Local transactions (batching)•Transactional modes
•optimistic •pessimistic• coming: ergonomics!
![Page 15: Infinispan in 50 minutes - JBoss...•5.2 - February 2013 •non-blocking state transfer •cross-site replication •rolling upgrades for hotrod-clients •map/reduce marked stable](https://reader035.vdocuments.us/reader035/viewer/2022070822/5f26b7cb51d8ec66c205aed6/html5/thumbnails/15.jpg)
Persistence/Cache Store•Used for
•durability• increased storage•warm caches
•Various implementations•HBase, Cassandra, BDBJE, JDBC, MongoDB, File, Cloud,
Remote / Infinispan, ...•Extensible
•easy!
![Page 16: Infinispan in 50 minutes - JBoss...•5.2 - February 2013 •non-blocking state transfer •cross-site replication •rolling upgrades for hotrod-clients •map/reduce marked stable](https://reader035.vdocuments.us/reader035/viewer/2022070822/5f26b7cb51d8ec66c205aed6/html5/thumbnails/16.jpg)
Network Performance & Reliability
![Page 17: Infinispan in 50 minutes - JBoss...•5.2 - February 2013 •non-blocking state transfer •cross-site replication •rolling upgrades for hotrod-clients •map/reduce marked stable](https://reader035.vdocuments.us/reader035/viewer/2022070822/5f26b7cb51d8ec66c205aed6/html5/thumbnails/17.jpg)
Map/Reduce example
![Page 18: Infinispan in 50 minutes - JBoss...•5.2 - February 2013 •non-blocking state transfer •cross-site replication •rolling upgrades for hotrod-clients •map/reduce marked stable](https://reader035.vdocuments.us/reader035/viewer/2022070822/5f26b7cb51d8ec66c205aed6/html5/thumbnails/18.jpg)
Map/Reduce
![Page 19: Infinispan in 50 minutes - JBoss...•5.2 - February 2013 •non-blocking state transfer •cross-site replication •rolling upgrades for hotrod-clients •map/reduce marked stable](https://reader035.vdocuments.us/reader035/viewer/2022070822/5f26b7cb51d8ec66c205aed6/html5/thumbnails/19.jpg)
Map/Reduce
![Page 20: Infinispan in 50 minutes - JBoss...•5.2 - February 2013 •non-blocking state transfer •cross-site replication •rolling upgrades for hotrod-clients •map/reduce marked stable](https://reader035.vdocuments.us/reader035/viewer/2022070822/5f26b7cb51d8ec66c205aed6/html5/thumbnails/20.jpg)
Infinispan on cloudsl Cloud-friendly autodiscovery strategies
l S3_PING, JDBC_PING, TCP_PINGl Fully dynamic clustering
l Add or kill nodes on the flyl Using a CacheLoader, scale back to zero nodes without data loss
![Page 21: Infinispan in 50 minutes - JBoss...•5.2 - February 2013 •non-blocking state transfer •cross-site replication •rolling upgrades for hotrod-clients •map/reduce marked stable](https://reader035.vdocuments.us/reader035/viewer/2022070822/5f26b7cb51d8ec66c205aed6/html5/thumbnails/21.jpg)
Querying•Based on Hibernate Search/Lucene•Objects in the grid are indexed
• index management
![Page 22: Infinispan in 50 minutes - JBoss...•5.2 - February 2013 •non-blocking state transfer •cross-site replication •rolling upgrades for hotrod-clients •map/reduce marked stable](https://reader035.vdocuments.us/reader035/viewer/2022070822/5f26b7cb51d8ec66c205aed6/html5/thumbnails/22.jpg)
Annotate your model@Indexedpublic class Book implements Serializable {
@Field String title; @Field String author; @Field String editor;
public Book(String title, String author, String editor) { this.title = title; this.author = author; this.editor = editor; }
}
![Page 23: Infinispan in 50 minutes - JBoss...•5.2 - February 2013 •non-blocking state transfer •cross-site replication •rolling upgrades for hotrod-clients •map/reduce marked stable](https://reader035.vdocuments.us/reader035/viewer/2022070822/5f26b7cb51d8ec66c205aed6/html5/thumbnails/23.jpg)
Run a QuerySearchManager qf = Search.getSearchManager(cache); Query query = qf.buildQueryBuilderForClass(Book.class) .get() .phrase() .onField("title") .sentence("in action") .createQuery(); List<Object> list = qf.getQuery(query).list();
![Page 24: Infinispan in 50 minutes - JBoss...•5.2 - February 2013 •non-blocking state transfer •cross-site replication •rolling upgrades for hotrod-clients •map/reduce marked stable](https://reader035.vdocuments.us/reader035/viewer/2022070822/5f26b7cb51d8ec66c205aed6/html5/thumbnails/24.jpg)
Use cases•Local cache•Cluster of caches•Autonomous data store
•access protocols
![Page 25: Infinispan in 50 minutes - JBoss...•5.2 - February 2013 •non-blocking state transfer •cross-site replication •rolling upgrades for hotrod-clients •map/reduce marked stable](https://reader035.vdocuments.us/reader035/viewer/2022070822/5f26b7cb51d8ec66c205aed6/html5/thumbnails/25.jpg)
Local cache
![Page 26: Infinispan in 50 minutes - JBoss...•5.2 - February 2013 •non-blocking state transfer •cross-site replication •rolling upgrades for hotrod-clients •map/reduce marked stable](https://reader035.vdocuments.us/reader035/viewer/2022070822/5f26b7cb51d8ec66c205aed6/html5/thumbnails/26.jpg)
Features of a local cache•Eviction•Expiry•Write through, write behind•Preloading•Notifications•Statistics
![Page 27: Infinispan in 50 minutes - JBoss...•5.2 - February 2013 •non-blocking state transfer •cross-site replication •rolling upgrades for hotrod-clients •map/reduce marked stable](https://reader035.vdocuments.us/reader035/viewer/2022070822/5f26b7cb51d8ec66c205aed6/html5/thumbnails/27.jpg)
Local cache not good enough...
![Page 28: Infinispan in 50 minutes - JBoss...•5.2 - February 2013 •non-blocking state transfer •cross-site replication •rolling upgrades for hotrod-clients •map/reduce marked stable](https://reader035.vdocuments.us/reader035/viewer/2022070822/5f26b7cb51d8ec66c205aed6/html5/thumbnails/28.jpg)
Cluster of caches
![Page 29: Infinispan in 50 minutes - JBoss...•5.2 - February 2013 •non-blocking state transfer •cross-site replication •rolling upgrades for hotrod-clients •map/reduce marked stable](https://reader035.vdocuments.us/reader035/viewer/2022070822/5f26b7cb51d8ec66c205aed6/html5/thumbnails/29.jpg)
Limitations of embedded mode
•Client is affected by cache topology changes•Shared resources•Tier management
• incompatible JVM tuning•security•garbage collection
•Non-JVM clients
![Page 30: Infinispan in 50 minutes - JBoss...•5.2 - February 2013 •non-blocking state transfer •cross-site replication •rolling upgrades for hotrod-clients •map/reduce marked stable](https://reader035.vdocuments.us/reader035/viewer/2022070822/5f26b7cb51d8ec66c205aed6/html5/thumbnails/30.jpg)
Cache Servers
![Page 31: Infinispan in 50 minutes - JBoss...•5.2 - February 2013 •non-blocking state transfer •cross-site replication •rolling upgrades for hotrod-clients •map/reduce marked stable](https://reader035.vdocuments.us/reader035/viewer/2022070822/5f26b7cb51d8ec66c205aed6/html5/thumbnails/31.jpg)
Client/Server Protocols•REST•Memcached•Hotrod
• “proprietary”• java, python, ruby
![Page 32: Infinispan in 50 minutes - JBoss...•5.2 - February 2013 •non-blocking state transfer •cross-site replication •rolling upgrades for hotrod-clients •map/reduce marked stable](https://reader035.vdocuments.us/reader035/viewer/2022070822/5f26b7cb51d8ec66c205aed6/html5/thumbnails/32.jpg)
Lucene Index Stored in Infinispan
![Page 33: Infinispan in 50 minutes - JBoss...•5.2 - February 2013 •non-blocking state transfer •cross-site replication •rolling upgrades for hotrod-clients •map/reduce marked stable](https://reader035.vdocuments.us/reader035/viewer/2022070822/5f26b7cb51d8ec66c205aed6/html5/thumbnails/33.jpg)
Single Node Performance Idea
![Page 34: Infinispan in 50 minutes - JBoss...•5.2 - February 2013 •non-blocking state transfer •cross-site replication •rolling upgrades for hotrod-clients •map/reduce marked stable](https://reader035.vdocuments.us/reader035/viewer/2022070822/5f26b7cb51d8ec66c205aed6/html5/thumbnails/34.jpg)
Multi-Node Setup
![Page 35: Infinispan in 50 minutes - JBoss...•5.2 - February 2013 •non-blocking state transfer •cross-site replication •rolling upgrades for hotrod-clients •map/reduce marked stable](https://reader035.vdocuments.us/reader035/viewer/2022070822/5f26b7cb51d8ec66c205aed6/html5/thumbnails/35.jpg)
Performance Warnings• Set Lucene's maximum segment size to fit in LuceneDirectory chunk_size to avoid readlocks• Verify blob sizes fit in JGroups network packets• Check for CacheStores “sweet spot” size
![Page 36: Infinispan in 50 minutes - JBoss...•5.2 - February 2013 •non-blocking state transfer •cross-site replication •rolling upgrades for hotrod-clients •map/reduce marked stable](https://reader035.vdocuments.us/reader035/viewer/2022070822/5f26b7cb51d8ec66c205aed6/html5/thumbnails/36.jpg)
The Path Ahead• 5.1.x.Final• 5.2 - February 2013
• non-blocking state transfer• cross-site replication• rolling upgrades for hotrod-clients• map/reduce marked stable
• 5.3 - Spring 2013• state transfer between sites
• 6.0• Beyond the current amazing performance• Scalable index writing• Advanced CacheStores
![Page 37: Infinispan in 50 minutes - JBoss...•5.2 - February 2013 •non-blocking state transfer •cross-site replication •rolling upgrades for hotrod-clients •map/reduce marked stable](https://reader035.vdocuments.us/reader035/viewer/2022070822/5f26b7cb51d8ec66c205aed6/html5/thumbnails/37.jpg)
JBoss Data Grid (JDG)•JBoss Data Grid•Build on top of Infinispan•http://www.jboss.com/edg
![Page 38: Infinispan in 50 minutes - JBoss...•5.2 - February 2013 •non-blocking state transfer •cross-site replication •rolling upgrades for hotrod-clients •map/reduce marked stable](https://reader035.vdocuments.us/reader035/viewer/2022070822/5f26b7cb51d8ec66c205aed6/html5/thumbnails/38.jpg)
Thank you!•infinispan.blogspot.com•@Infinispan•@SanneGrinovero