the enterprise service bus is dead! long live the enterprise service bus, rimas silkaitis
TRANSCRIPT
![Page 1: The Enterprise Service Bus is Dead! Long live the Enterprise Service Bus, Rimas Silkaitis](https://reader031.vdocuments.us/reader031/viewer/2022020213/58f9ad8d760da3da068b997a/html5/thumbnails/1.jpg)
The Enterprise Kafka Summit 2016
Rimas Silkaitis
![Page 2: The Enterprise Service Bus is Dead! Long live the Enterprise Service Bus, Rimas Silkaitis](https://reader031.vdocuments.us/reader031/viewer/2022020213/58f9ad8d760da3da068b997a/html5/thumbnails/2.jpg)
how Heroku leveraged Kafka to realize the original vision of the enterprise service bus (esb)
![Page 3: The Enterprise Service Bus is Dead! Long live the Enterprise Service Bus, Rimas Silkaitis](https://reader031.vdocuments.us/reader031/viewer/2022020213/58f9ad8d760da3da068b997a/html5/thumbnails/3.jpg)
Rimas Silkaitis
Product
![Page 4: The Enterprise Service Bus is Dead! Long live the Enterprise Service Bus, Rimas Silkaitis](https://reader031.vdocuments.us/reader031/viewer/2022020213/58f9ad8d760da3da068b997a/html5/thumbnails/4.jpg)
Heroku Postgres Heroku Redis
![Page 5: The Enterprise Service Bus is Dead! Long live the Enterprise Service Bus, Rimas Silkaitis](https://reader031.vdocuments.us/reader031/viewer/2022020213/58f9ad8d760da3da068b997a/html5/thumbnails/5.jpg)
What is an ESB?
![Page 6: The Enterprise Service Bus is Dead! Long live the Enterprise Service Bus, Rimas Silkaitis](https://reader031.vdocuments.us/reader031/viewer/2022020213/58f9ad8d760da3da068b997a/html5/thumbnails/6.jpg)
Analyst Definitions
![Page 7: The Enterprise Service Bus is Dead! Long live the Enterprise Service Bus, Rimas Silkaitis](https://reader031.vdocuments.us/reader031/viewer/2022020213/58f9ad8d760da3da068b997a/html5/thumbnails/7.jpg)
ESBs receive service consumption requests, route the requests to the correct service provider, transform the requests to a format compatible with the service provider, wait for the results, and deliver them back to the service consumer.
IDChttp://www.idc.com/2010st/esbcm.html
![Page 8: The Enterprise Service Bus is Dead! Long live the Enterprise Service Bus, Rimas Silkaitis](https://reader031.vdocuments.us/reader031/viewer/2022020213/58f9ad8d760da3da068b997a/html5/thumbnails/8.jpg)
Uses a bus architecture for scalability and reliability, establishes a unified communications channel, provides integration, mediation and communication services and is standards based.
http://redmonk.com/public/OpenSourceESBs.pdf
Redmonk
![Page 9: The Enterprise Service Bus is Dead! Long live the Enterprise Service Bus, Rimas Silkaitis](https://reader031.vdocuments.us/reader031/viewer/2022020213/58f9ad8d760da3da068b997a/html5/thumbnails/9.jpg)
Vendor Definitions
![Page 10: The Enterprise Service Bus is Dead! Long live the Enterprise Service Bus, Rimas Silkaitis](https://reader031.vdocuments.us/reader031/viewer/2022020213/58f9ad8d760da3da068b997a/html5/thumbnails/10.jpg)
Oracle Service Bus transforms complex and brittle architectures into agile integration networks by connecting, virtualizing, and managing interactions between services and applications.
http://www.oracle.com/technetwork/middleware/service-bus/overview/index.html
Oracle
![Page 11: The Enterprise Service Bus is Dead! Long live the Enterprise Service Bus, Rimas Silkaitis](https://reader031.vdocuments.us/reader031/viewer/2022020213/58f9ad8d760da3da068b997a/html5/thumbnails/11.jpg)
Mule ESB is a lightweight Java-based enterprise service bus (ESB) and integration platform that allows developers to connect applications together quickly and easily, enabling them to exchange data.
https://www.mulesoft.com/resources/esb/what-mule-esb
Mulesoft
![Page 12: The Enterprise Service Bus is Dead! Long live the Enterprise Service Bus, Rimas Silkaitis](https://reader031.vdocuments.us/reader031/viewer/2022020213/58f9ad8d760da3da068b997a/html5/thumbnails/12.jpg)
How should we define it?
![Page 13: The Enterprise Service Bus is Dead! Long live the Enterprise Service Bus, Rimas Silkaitis](https://reader031.vdocuments.us/reader031/viewer/2022020213/58f9ad8d760da3da068b997a/html5/thumbnails/13.jpg)
¯\_(ツ)_/¯
![Page 14: The Enterprise Service Bus is Dead! Long live the Enterprise Service Bus, Rimas Silkaitis](https://reader031.vdocuments.us/reader031/viewer/2022020213/58f9ad8d760da3da068b997a/html5/thumbnails/14.jpg)
App
App
App
App
Finance
HR
BizOps
Inventory
![Page 15: The Enterprise Service Bus is Dead! Long live the Enterprise Service Bus, Rimas Silkaitis](https://reader031.vdocuments.us/reader031/viewer/2022020213/58f9ad8d760da3da068b997a/html5/thumbnails/15.jpg)
ESBRouting
Mediation
Messaging
EventProcessing
ServiceOrchestration
![Page 16: The Enterprise Service Bus is Dead! Long live the Enterprise Service Bus, Rimas Silkaitis](https://reader031.vdocuments.us/reader031/viewer/2022020213/58f9ad8d760da3da068b997a/html5/thumbnails/16.jpg)
ESB
Push
Centralized
![Page 17: The Enterprise Service Bus is Dead! Long live the Enterprise Service Bus, Rimas Silkaitis](https://reader031.vdocuments.us/reader031/viewer/2022020213/58f9ad8d760da3da068b997a/html5/thumbnails/17.jpg)
Architecture
![Page 18: The Enterprise Service Bus is Dead! Long live the Enterprise Service Bus, Rimas Silkaitis](https://reader031.vdocuments.us/reader031/viewer/2022020213/58f9ad8d760da3da068b997a/html5/thumbnails/18.jpg)
API
CLI GIT Dashboard
Data Runtime BizOps
![Page 19: The Enterprise Service Bus is Dead! Long live the Enterprise Service Bus, Rimas Silkaitis](https://reader031.vdocuments.us/reader031/viewer/2022020213/58f9ad8d760da3da068b997a/html5/thumbnails/19.jpg)
$ heroku addons:create heroku-postgresql:standard-0 -a sushiCreating postgresql-reticulated-74861... done, ($50.00/month)Adding postgresql-reticulated-74861 to neovintage... doneSetting HEROKU_POSTGRESQL_GRAY_URL and restarting sushi... done, v249The database should be available in 3-5 minutes. ! The database will be empty. If upgrading, you can transfer ! data from another database with pg:copy.Use `heroku pg:wait` to track statusUse `heroku addons:docs heroku-postgresql` to view documentation.
![Page 20: The Enterprise Service Bus is Dead! Long live the Enterprise Service Bus, Rimas Silkaitis](https://reader031.vdocuments.us/reader031/viewer/2022020213/58f9ad8d760da3da068b997a/html5/thumbnails/20.jpg)
API
CLI GIT Dashboard
Data Runtime BizOps
![Page 21: The Enterprise Service Bus is Dead! Long live the Enterprise Service Bus, Rimas Silkaitis](https://reader031.vdocuments.us/reader031/viewer/2022020213/58f9ad8d760da3da068b997a/html5/thumbnails/21.jpg)
API
CLI GIT Web
Data Runtime BizOps
THE API IS OUR ESB!
![Page 22: The Enterprise Service Bus is Dead! Long live the Enterprise Service Bus, Rimas Silkaitis](https://reader031.vdocuments.us/reader031/viewer/2022020213/58f9ad8d760da3da068b997a/html5/thumbnails/22.jpg)
Problems Emerge
• Bottleneck in getting features released that touch API
• Difficult to get Analytics across the entire Org
• Hiring has become difficult
• API is down. So is the entire platform.
![Page 23: The Enterprise Service Bus is Dead! Long live the Enterprise Service Bus, Rimas Silkaitis](https://reader031.vdocuments.us/reader031/viewer/2022020213/58f9ad8d760da3da068b997a/html5/thumbnails/23.jpg)
![Page 24: The Enterprise Service Bus is Dead! Long live the Enterprise Service Bus, Rimas Silkaitis](https://reader031.vdocuments.us/reader031/viewer/2022020213/58f9ad8d760da3da068b997a/html5/thumbnails/24.jpg)
PUSH vs PULLhttp://kafka.apache.org/documentation.html#design_pull
![Page 25: The Enterprise Service Bus is Dead! Long live the Enterprise Service Bus, Rimas Silkaitis](https://reader031.vdocuments.us/reader031/viewer/2022020213/58f9ad8d760da3da068b997a/html5/thumbnails/25.jpg)
PUSH
![Page 26: The Enterprise Service Bus is Dead! Long live the Enterprise Service Bus, Rimas Silkaitis](https://reader031.vdocuments.us/reader031/viewer/2022020213/58f9ad8d760da3da068b997a/html5/thumbnails/26.jpg)
PULLProducers Consumers
![Page 27: The Enterprise Service Bus is Dead! Long live the Enterprise Service Bus, Rimas Silkaitis](https://reader031.vdocuments.us/reader031/viewer/2022020213/58f9ad8d760da3da068b997a/html5/thumbnails/27.jpg)
ESB Redefinition
• Pull over Push
• Consumers dictate their own consumption rate
• Independent development, scaling and scheduling
• Stop the world processing doesn’t exist
![Page 28: The Enterprise Service Bus is Dead! Long live the Enterprise Service Bus, Rimas Silkaitis](https://reader031.vdocuments.us/reader031/viewer/2022020213/58f9ad8d760da3da068b997a/html5/thumbnails/28.jpg)
CLI GIT Dashboard
API Data Runtime BizOps
![Page 29: The Enterprise Service Bus is Dead! Long live the Enterprise Service Bus, Rimas Silkaitis](https://reader031.vdocuments.us/reader031/viewer/2022020213/58f9ad8d760da3da068b997a/html5/thumbnails/29.jpg)
Other Use Cases Around Heroku
![Page 30: The Enterprise Service Bus is Dead! Long live the Enterprise Service Bus, Rimas Silkaitis](https://reader031.vdocuments.us/reader031/viewer/2022020213/58f9ad8d760da3da068b997a/html5/thumbnails/30.jpg)
Operational Experience
![Page 31: The Enterprise Service Bus is Dead! Long live the Enterprise Service Bus, Rimas Silkaitis](https://reader031.vdocuments.us/reader031/viewer/2022020213/58f9ad8d760da3da068b997a/html5/thumbnails/31.jpg)
aggregation
router
aggregation
router
metrics-ingest
router
metrics-ingest
router
ZK postgres
metrics-ingest
router
aggregation
router
aggregation
router
aggregation
router
sink
router
aggregation
router
aggregation
router
metrics-api
router
![Page 32: The Enterprise Service Bus is Dead! Long live the Enterprise Service Bus, Rimas Silkaitis](https://reader031.vdocuments.us/reader031/viewer/2022020213/58f9ad8d760da3da068b997a/html5/thumbnails/32.jpg)
$ heroku kafka:list -a metaas-addons=== Kafka Topics on KAFKA_URL
Name Messages Traffic─────────────────────────────────── ──────────────
─────────────ROUTER_REQUEST.v1 132,823.12/sec 14.6 mb/sec
DYNO_MEMORY.v1 1,478.97/sec 187 kb/secDYNO_LOAD.v1 1,468.01/sec 153 kb/sec
ROUTER_STATUS_SUMMARY.v1 987.4/sec 28.3 kb/secSYSLOG.v1 691.9/sec 76.8 kb/sec
ROUTER_LATENCY_SUMMARY.v1 678.84/sec 37.5 kb/secROUTER_ERROR.v1 476.12/sec 93.9 kb/sec
DYNO_ERROR.v1 354.06/sec 38.5 kb/secDYNO_LOAD_SUMMARY.v1 241.33/sec 9.73 kb/sec
DYNO_MEMORY_SUMMARY.v1 219.26/sec 25.1 kb/secDYNO_ERROR_SUMMARY.v1 73.49/sec 1.83 kb/sec
ROUTER_ERROR_SUMMARY.v1 12.35/sec 276 bytes/sec__consumer_offsets 4.1/sec 405 bytes/sec
ROUTER_STATUS_PARTIAL_SUMMARY.v1 0.0/sec 0 bytes/secROUTER_LATENCY_PARTIAL_HISTOGRAM.v1 0.0/sec 0 bytes/sec
ROUTER_REQUESTS.v1 0.0/sec 0 bytes/sec
![Page 33: The Enterprise Service Bus is Dead! Long live the Enterprise Service Bus, Rimas Silkaitis](https://reader031.vdocuments.us/reader031/viewer/2022020213/58f9ad8d760da3da068b997a/html5/thumbnails/33.jpg)
What do we need to do next?
![Page 34: The Enterprise Service Bus is Dead! Long live the Enterprise Service Bus, Rimas Silkaitis](https://reader031.vdocuments.us/reader031/viewer/2022020213/58f9ad8d760da3da068b997a/html5/thumbnails/34.jpg)
Schema Registry
![Page 35: The Enterprise Service Bus is Dead! Long live the Enterprise Service Bus, Rimas Silkaitis](https://reader031.vdocuments.us/reader031/viewer/2022020213/58f9ad8d760da3da068b997a/html5/thumbnails/35.jpg)
![Page 36: The Enterprise Service Bus is Dead! Long live the Enterprise Service Bus, Rimas Silkaitis](https://reader031.vdocuments.us/reader031/viewer/2022020213/58f9ad8d760da3da068b997a/html5/thumbnails/36.jpg)
Security
![Page 37: The Enterprise Service Bus is Dead! Long live the Enterprise Service Bus, Rimas Silkaitis](https://reader031.vdocuments.us/reader031/viewer/2022020213/58f9ad8d760da3da068b997a/html5/thumbnails/37.jpg)
![Page 38: The Enterprise Service Bus is Dead! Long live the Enterprise Service Bus, Rimas Silkaitis](https://reader031.vdocuments.us/reader031/viewer/2022020213/58f9ad8d760da3da068b997a/html5/thumbnails/38.jpg)
as a Service
![Page 39: The Enterprise Service Bus is Dead! Long live the Enterprise Service Bus, Rimas Silkaitis](https://reader031.vdocuments.us/reader031/viewer/2022020213/58f9ad8d760da3da068b997a/html5/thumbnails/39.jpg)
Creating kafka-lively-27896... done, (free)Adding kafka-lively-27896 to sushi... doneKafka cluster has been created and will be available shortlyUse `heroku addons:docs heroku-kafka` to view documentation.
=== KAFKA_URLName: kafka-silhouetted-79119Created: 2016-03-24 21:21 UTCPlan: Beta 3Status: availableVersion: 0.9.0.0Topics: 0 topics (see heroku kafka:list)Connections: 0 consumers (0 applications)Messages: 0.0 messages/secTraffic: 0 Bytes/sec in / 0 Bytes/s out
$ heroku kafka:info -a sushi
$ heroku addons:create heroku-kafka:beta-3 -a sushi
![Page 41: The Enterprise Service Bus is Dead! Long live the Enterprise Service Bus, Rimas Silkaitis](https://reader031.vdocuments.us/reader031/viewer/2022020213/58f9ad8d760da3da068b997a/html5/thumbnails/41.jpg)
Come See Us At Our BoothWe’re Hiring!