monitoring apache kafka with confluent control center

13
1 Confluent Control Center Monitoring Apache Kafka - March 2017

Upload: confluent

Post on 05-Apr-2017

124 views

Category:

Software


0 download

TRANSCRIPT

Page 1: Monitoring Apache Kafka with Confluent Control Center

11

Confluent Control CenterMonitoring Apache Kafka - March 2017

Page 2: Monitoring Apache Kafka with Confluent Control Center

22

Attend the whole series!

Simplify Governance for Streaming Data in Apache KafkaDate: Thursday, April 6, 2017Time: 9:30 am - 10:00 am PT | 12:30 pm - 1:00 pm ETSpeaker: Gwen Shapira, Product Manager, Confluent

Using Apache Kafka to Analyze Session WindowsDate: Thursday, March 30, 2017Time: 9:30 am - 10:00 am PT | 12:30 pm - 1:00 pm ETSpeaker: Michael Noll, Product Manager, Confluent

Monitoring and Alerting Apache Kafka with Confluent Control CenterDate: Thursday, March 16, 2017Time: 9:30 am - 10:00 am PT | 12:30 pm - 1:00 pm ETSpeaker: Nick Dearden, Director, Engineering and Product

Data Pipelines Made Simple with Apache KafkaDate: Thursday, March 23, 2017Time: 9:30 am - 10:00 am PT | 12:30 pm - 1:00 pm ETSpeaker: Ewen Cheslack-Postava, Engineer, Confluent

https://www.confluent.io/online-talk/online-talk-series-five-steps-to-production-with-apache-kafka/

What’s New in Apache Kafka 0.10.2 and Confluent 3.2

Date: Thursday, March 9, 2017Time: 9:30 am - 10:00 am PT | 12:30 pm - 1:00 pm ETSpeaker: Clarke Patterson, Senior Director, Product Marketing

Page 3: Monitoring Apache Kafka with Confluent Control Center

33

Confluent: Enterprise Streaming Platform based on Apache Kafka™

Database Changes

Log Events loT Data Web

Events …

CRM

Data Warehouse

Database

Hadoop

DataIntegration

Monitoring

Analytics

Custom Apps

Transformations

Real-time Applications

Apache  Open  Source Confluent  Open  Source Confluent  Commercial

Open

Enterprise Ready

Trusted

Complete

Confluent Platform

Confluent

Apache  Kafka™

Data  CompatibilitySchema  Registry

Monitoring  &  AdministrationConfluent  Control  Center

OperationsReplicator  |  Auto  Data  Balancing

Development  and  ConnectivityClients  |  Connectors

Page 4: Monitoring Apache Kafka with Confluent Control Center

44

Cluster and Application Health

How do I manage and monitor my streaming platform at

scale?Apache  Open  Source Confluent  Open  Source Confluent  Commercial

Confluent Platform

Confluent

Apache  Kafka™

Data  CompatibilitySchema  Registry

Monitoring  &  AdministrationConfluent  Control  Center

OperationsReplicator  |  Auto  Data  Balancing

Development  and  ConnectivityClients  |  Connectors

Page 5: Monitoring Apache Kafka with Confluent Control Center

55

Holistic Monitoring

Start with the basics• Are my brokers up ?• Is something maxed out ?• Anything unusual happening ?

Progress to application level SLAs• Are all my messages being delivered ?• Did they arrive within their latency SLAs ?• How do I know nothing was corrupted in

transit ?

Page 6: Monitoring Apache Kafka with Confluent Control Center

66

Making Sense of Kafka Metrics

Correlation does not imply causation

Page 7: Monitoring Apache Kafka with Confluent Control Center

77

Broker Cluster & Topic Health

Cluster health dashboards• Monitor the health of your Kafka clusters

and get alerts if any problems occur• Measure system load, performance,

and operations• View aggregate statistics or drill down

by broker or topic

Page 8: Monitoring Apache Kafka with Confluent Control Center

88

End-to-End Stream Monitoring

Did you

leave?

Did you

arrive?

Message Delivery Assurance• Monitor data delivery across multiple clusters• Narrow down problems to different applications,

clusters, and components• Get alerts if any part of your data pipeline slows

down or stops

Page 9: Monitoring Apache Kafka with Confluent Control Center

99

Did you

leave?

Did you

arrive?

Message Delivery Assurance• Monitor data delivery across multiple clusters• Narrow down problems to different applications,

clusters, and components• Get alerts if any part of your data pipeline slows

down or stops

Page 10: Monitoring Apache Kafka with Confluent Control Center

1010

Time for a Demo

Page 11: Monitoring Apache Kafka with Confluent Control Center

1111

https://www.confluent.io/product/

https://www.confluent.io/download/

Page 12: Monitoring Apache Kafka with Confluent Control Center

1212

Attend the whole series!

Simplify Governance for Streaming Data in Apache KafkaDate: Thursday, April 6, 2017Time: 9:30 am - 10:00 am PT | 12:30 pm - 1:00 pm ETSpeaker: Gwen Shapira, Product Manager, Confluent

Using Apache Kafka to Analyze Session WindowsDate: Thursday, March 30, 2017Time: 9:30 am - 10:00 am PT | 12:30 pm - 1:00 pm ETSpeaker: Michael Noll, Product Manager, Confluent

Monitoring and Alerting Apache Kafka with Confluent Control CenterDate: Thursday, March 16, 2017Time: 9:30 am - 10:00 am PT | 12:30 pm - 1:00 pm ETSpeaker: Nick Dearden, Director, Engineering and Product

Data Pipelines Made Simple with Apache KafkaDate: Thursday, March 23, 2017Time: 9:30 am - 10:00 am PT | 12:30 pm - 1:00 pm ETSpeaker: Ewen Cheslack-Postava, Engineer, Confluent

https://www.confluent.io/online-talk/online-talk-series-five-steps-to-production-with-apache-kafka/

What’s New in Apache Kafka 0.10.2 and Confluent 3.2

Date: Thursday, March 9, 2017Time: 9:30 am - 10:00 am PT | 12:30 pm - 1:00 pm ETSpeaker: Clarke Patterson, Senior Director, Product Marketing

Page 13: Monitoring Apache Kafka with Confluent Control Center

1313

Discount code: kafcom17  Use the Apache Kafka community discount code to get $50 off  www.kafka-summit.orgKafka Summit New York: May 8Kafka Summit San Francisco: August 28

Presented by