monitoring apache kafka with confluent control center

11

Confluent Control CenterMonitoring Apache Kafka - March 2017

22

Attend the whole series!

Simplify Governance for Streaming Data in Apache KafkaDate: Thursday, April 6, 2017Time: 9:30 am - 10:00 am PT | 12:30 pm - 1:00 pm ETSpeaker: Gwen Shapira, Product Manager, Confluent

Using Apache Kafka to Analyze Session WindowsDate: Thursday, March 30, 2017Time: 9:30 am - 10:00 am PT | 12:30 pm - 1:00 pm ETSpeaker: Michael Noll, Product Manager, Confluent

Monitoring and Alerting Apache Kafka with Confluent Control CenterDate: Thursday, March 16, 2017Time: 9:30 am - 10:00 am PT | 12:30 pm - 1:00 pm ETSpeaker: Nick Dearden, Director, Engineering and Product

Data Pipelines Made Simple with Apache KafkaDate: Thursday, March 23, 2017Time: 9:30 am - 10:00 am PT | 12:30 pm - 1:00 pm ETSpeaker: Ewen Cheslack-Postava, Engineer, Confluent

https://www.confluent.io/online-talk/online-talk-series-five-steps-to-production-with-apache-kafka/

What’s New in Apache Kafka 0.10.2 and Confluent 3.2

Date: Thursday, March 9, 2017Time: 9:30 am - 10:00 am PT | 12:30 pm - 1:00 pm ETSpeaker: Clarke Patterson, Senior Director, Product Marketing

33

Confluent: Enterprise Streaming Platform based on Apache Kafka™

Database Changes

Log Events loT Data Web

Events …

CRM

Data Warehouse

Database

Hadoop

DataIntegration

…

Monitoring

Analytics

Custom Apps

Transformations

Real-time Applications

…

Apache  Open  Source Confluent  Open  Source Confluent  Commercial

Open

Enterprise Ready

Trusted

Complete

Confluent Platform

Confluent

Apache  Kafka™

Data  CompatibilitySchema  Registry

Monitoring  &  AdministrationConfluent  Control  Center

OperationsReplicator  |  Auto  Data  Balancing

Development  and  ConnectivityClients  |  Connectors

44

Cluster and Application Health

How do I manage and monitor my streaming platform at

scale?Apache  Open  Source Confluent  Open  Source Confluent  Commercial

Confluent Platform

Confluent

Apache  Kafka™

Data  CompatibilitySchema  Registry

Monitoring  &  AdministrationConfluent  Control  Center

OperationsReplicator  |  Auto  Data  Balancing

Development  and  ConnectivityClients  |  Connectors

55

Holistic Monitoring

Start with the basics• Are my brokers up ?• Is something maxed out ?• Anything unusual happening ?

Progress to application level SLAs• Are all my messages being delivered ?• Did they arrive within their latency SLAs ?• How do I know nothing was corrupted in

transit ?

66

Making Sense of Kafka Metrics

Correlation does not imply causation

77

Broker Cluster & Topic Health

Cluster health dashboards• Monitor the health of your Kafka clusters

and get alerts if any problems occur• Measure system load, performance,

and operations• View aggregate statistics or drill down

by broker or topic

88

End-to-End Stream Monitoring

Did you

leave?

Did you

arrive?

Message Delivery Assurance• Monitor data delivery across multiple clusters• Narrow down problems to different applications,

clusters, and components• Get alerts if any part of your data pipeline slows

down or stops

99

Did you

leave?

Did you

arrive?

Message Delivery Assurance• Monitor data delivery across multiple clusters• Narrow down problems to different applications,

clusters, and components• Get alerts if any part of your data pipeline slows

down or stops

1010

Time for a Demo

1111

https://www.confluent.io/product/

https://www.confluent.io/download/

1212

Attend the whole series!

Simplify Governance for Streaming Data in Apache KafkaDate: Thursday, April 6, 2017Time: 9:30 am - 10:00 am PT | 12:30 pm - 1:00 pm ETSpeaker: Gwen Shapira, Product Manager, Confluent

Using Apache Kafka to Analyze Session WindowsDate: Thursday, March 30, 2017Time: 9:30 am - 10:00 am PT | 12:30 pm - 1:00 pm ETSpeaker: Michael Noll, Product Manager, Confluent

Monitoring and Alerting Apache Kafka with Confluent Control CenterDate: Thursday, March 16, 2017Time: 9:30 am - 10:00 am PT | 12:30 pm - 1:00 pm ETSpeaker: Nick Dearden, Director, Engineering and Product

Data Pipelines Made Simple with Apache KafkaDate: Thursday, March 23, 2017Time: 9:30 am - 10:00 am PT | 12:30 pm - 1:00 pm ETSpeaker: Ewen Cheslack-Postava, Engineer, Confluent

https://www.confluent.io/online-talk/online-talk-series-five-steps-to-production-with-apache-kafka/

What’s New in Apache Kafka 0.10.2 and Confluent 3.2

Date: Thursday, March 9, 2017Time: 9:30 am - 10:00 am PT | 12:30 pm - 1:00 pm ETSpeaker: Clarke Patterson, Senior Director, Product Marketing

1313

Discount code: kafcom17  Use the Apache Kafka community discount code to get $50 off  www.kafka-summit.orgKafka Summit New York: May 8Kafka Summit San Francisco: August 28

Presented by