#geodesummit - integration & future direction for spring cloud data flow & geode
TRANSCRIPT
Spring Cloud Data Flow + Geode
Sabby Anandan | Product Manager | @sabbyanandan
Stream Batch
Spring Cloud Data Flow
Spring Cloud Stream Spring Cloud Task
Shell; DSL; REST-APIs Drag & Drop UI Security
OOTB Connectors Reactive Data Science
DataflowServer
Admin / Flo UI
Shell
CURL ??XStream/Task Spring Boot Apps
YARN
Why are we here?Data Pipelines requiring:
• Low latency and in-memory processing • High Throughput SLAs • Correlation between reference-data and data-in-flight
• Frequent data-shuffling
http | transform | log
| = ?
http | transform | log
| = Binder
Binders
Region Data Buckets
http | transform | log
Geode Cluster
transform-processor.jar PARTITION
transform-processor.jar PARTITION
log-sink.jar PARTITION
log-sink.jar PARTITION
http-source.jar PARTITION_PROXYhttp-source.jar PARTITION_PROXY
What’s next?
•+/- scaling and automatic re-partitioning •Stream / Task metadata-repository •Key-Value store for OOTB Counters •Partition level local-state and SQL-like stream processing
11
Join the Apache Geode Community! • Check out http://geode.incubator.apache.org
• Subscribe: [email protected]
• Download: http://geode.incubator.apache.org/releases/