kafka streams + deep learning - rainfocus streams + deep learning ... mqtt iot iphone app kafka go...

1Confidential

Kafka Streams + Deep Learning TensorFlow and H2O Applied to Real Time Streaming Applications

Kai WaehnerTechnology Evangelist

kontakt@kai-waehner.de

LinkedIn

@KaiWaehner

www.kai-waehner.de

2Apache Kafka and Deep Learning – Kai Waehner

Agenda

1) Deep Learning and Real Time World2) Deep Learning and the Apache Kafka Ecosystem3) Building Neural Networks with TensorFlow and H2O4) Deployment of Neural Networks with Kafka Streams

3Apache Kafka and Deep Learning – Kai Waehner

Agenda

1) Deep Learning and Real World Applications2) Deep Learning and the Apache Kafka Ecosystem3) Building Neural Networks with TensorFlow and H2O4) Deployment of Neural Networks with Kafka Streams

4Apache Kafka and Deep Learning – Kai Waehner

Machine Learning

... allows computers to find hidden insights without being explicitly programmed where to look.

Machine Learning• Decision Trees• Naïve Bayes• Clustering• Neural Networks• etc.

Deep Learning• CNN• RNN• Autoencoder• etc.

5Apache Kafka and Deep Learning – Kai Waehner

Real World Examples of Deep Learning

Language Translation

https://recombu.com/mobile/article/google-instant-voice-translation-the-end-of-franglais_M11397.htmlhttp://wellroundedfashion.com/2015/01/13/trendy-tech-camouflage-from-facial-detection/http://www.washington.edu/news/2017/07/11/lip-syncing-obama-new-tools-turn-audio-clips-into-realistic-video/https://www.technologyreview.com/s/602796/starcraft-will-become-the-next-big-playground-for-ai/

Speech and Video Generation

Picture Analysis (Face, Location, Items, …)

Intelligent Decision Making

6Apache Kafka and Deep Learning – Kai Waehner

Deep Learning to Analyze and Act on Critical Business Moments

Seconds Minutes Hours

Price Optimization

Predictive Maintenance

Fraud Detection

Cross Selling

Transportation Rerouting

Customer Service

Inventory Management

Windows of Opportunity

7Apache Kafka and Deep Learning – Kai Waehner

Live Demo

Deep Learning in Actionhttp://playground.tensorflow.org/

8Apache Kafka and Deep Learning – Kai Waehner

Agenda

1) Deep Learning and Real World Applications2) Deep Learning and the Apache Kafka Ecosystem3) Building Neural Networks with TensorFlow and H2O4) Deployment of Neural Networks with Kafka Streams

9Apache Kafka and Deep Learning – Kai Waehner

Apache Kafka Ecosystem and Deep Learning

Kafka Streams

Kafka Connect

Rest Proxy

Schema Registry

Go / .NET / Python Kafka Producer

KSQL

Kafka Streams

STREAMING PLATFORM

BIG DATA ANALYTICS

Oracle DB

CoaP IoT

Kafka Java Client

…..

HP Vertica

KAFKA

H2O.ai, TensorFlow

Batch

Real Time

ConfluentREST Proxy

MQTT IoT

iPhone App

KafkaGo Client

CK OA NF NK EA C

T

HIVE

Grafana

Kafka

Java EE Web App

Spark

CK OA NF NK EA C

T

Confluent Schema Registry

Kafka Streams

H2O.ai

Mesos

Kafka Streams

TensorFlow

Kubernetes

Avro

Avro

1) Data Producer2) Analytics Platform3) Streaming Platform4) Data Consumer

11Apache Kafka and Deep Learning – Kai Waehner

Agenda

1) Deep Learning and Real World Applications2) Deep Learning and the Apache Kafka Ecosystem3) Building Neural Networks with TensorFlow and H2O4) Deployment of Neural Networks with Kafka Streams

12Apache Kafka and Deep Learning – Kai Waehner

Languages, Frameworks and Tools for Deep Learning

There is no Allrounder!

Portable Format for Analytics (PFA)

13Confidential

Deep Learning with H2O.ai

H2O Engine

R / Python /Scala / Flow UI

Java Code

14Confidential

H2O Deep Water (TensorFlow, MXNet, …)

https://h2o-release.s3.amazonaws.com/h2o/rel-vapnik/1/docs-website/h2o-docs/booklets/DeepWaterBooklet.pdf

Deep Water(H2O + TensorFlow) Pre-Defined Networks

+User-Defined Networks

15Apache Kafka and Deep Learning – Kai Waehner

Live Demo

Use Case: Airline Flight Delay Prediction

Machine Learning Algorithm:Deep Learningusing Neural Networks

Technology:H2O.ai, TensorFlow

16Apache Kafka and Deep Learning – Kai Waehner

Agenda

1) Deep Learning and Real World Applications2) Deep Learning and the Apache Kafka Ecosystem3) Building Neural Networks with TensorFlow and H2O4) Deployment of Neural Networks with Kafka Streams

17Apache Kafka and Deep Learning – Kai Waehner

Stream Processing

Data at Rest Data in Motion

18Apache Kafka and Deep Learning – Kai Waehner

Stream Processing Pipeline

APIs

Adapters / Channels

Integration

Messaging

Stream Ingest

Transformation

Aggregation

Enrichment

Filtering

StreamPreprocessing

Process Management

Analytics (Real Time)

Applications& APIs

Analytics / DW Reporting

StreamOutcomes

• Contextual Rules

• Windowing

• Patterns

• Analytics

• Machine Learning

• …

Stream Analytics

Index / SearchNormalization

Applying an Analytic Modelis just a piece of the puzzle!

19Apache Kafka and Deep Learning – Kai Waehner

Kafka Streams (shipped with Apache Kafka)

Map, filter, aggregate,

apply analytic model,

„any business logic“

Input Stream(Kafka Topic)

Kafka Cluster

Output Stream(Kafka Topic)

Kafka Cluster

Stream ProcessingMicroservice

(Kafka Streams)

Deployed AnywhereJava App, Docker,

Kubernetes, Mesos,“you-name-it”

20Apache Kafka and Deep Learning – Kai Waehner

When to use Kafka Streams for Stream Processing?

21Apache Kafka and Deep Learning – Kai Waehner

A complete streaming microservices, ready for production at large-scale

WordCount

App configuration

Define processing(here: WordCount)

Start processing

22Apache Kafka and Deep Learning – Kai Waehner

Use Case: Airline Flight Delay Prediction

Machine Learning Algorithm:Neural Network built with H2O and TensorFlow

Streaming Platform:Apache Kafka and Kafka Streams

Live Demo

23Apache Kafka and Deep Learning – Kai Waehner

H2O.ai Model + Kafka Streams

Filter

Map

1) Create H2O DL model

2) Configure Kafka Streams Application

3) Apply H2O DL model to Streaming Data

4) Start Kafka Streams App

24Apache Kafka and Deep Learning – Kai Waehner

Github Examples: Kafka + Deep Learning

https://github.com/kaiwaehner/kafka-streams-machine-learning-examples

1) git clone … 2) mvn clean package … 3) look at implementations and unit tests

25Apache Kafka and Deep Learning – Kai Waehner

Online Model Training with Apache Kafka and Kafka Streams

How to improve models?

1. Manual Update

2. Automated Batch

3. Real Time

Your choice… All possible with Kafka!

26Apache Kafka and Deep Learning – Kai Waehner

Caveats for Online Model Training

• Processes and infrastructure not ready

• Validation needed before production

• Slows down the system

• Only a few ML implementations à Build your own!

• Many use cases do not need it

à Do it only when really needed!

27Apache Kafka and Deep Learning – Kai Waehner

Key Take-Aways

à Data Scientist and Developers have to work together continuously (org + tech!)

à Mission critical, scalable production deployment is key for success of Machine Learning projects

à Apache Kafka Ecosystem for Batch and Real Time Machine Learning (Training, Inference, Monitoring)

28Apache Kafka and Deep Learning – Kai Waehner

Confluent KSQL à www.confluent.io/product/ksqlUse SQL-like queries to write stream processing applications with Kafka Streams

Leverage Kafka and Kafka Streams from anywhere without coding• Streaming ETL• Anomaly Detection• Real Time Monitoring• etc.

SELECT STREAMCEIL(timestamp TO HOUR) AS timeWindow, productId, COUNT(*) AS hourlyOrders, SUM(units) AS units

FROM Orders GROUP BY CEIL(timestamp TO HOUR), productId;

timeWindow | productId | hourlyOrders | units------------+-----------+--------------+-------

08:00:00 | 10 | 2 | 508:00:00 | 20 | 1 | 809:00:00 | 10 | 4 | 2209:00:00 | 40 | 1 | 45

... | ... | ... | ...

à … maybe also use it to apply analytic models via user defined functions? J

29Apache Kafka and Deep Learning – Kai Waehner

Kai WaehnerTechnology Evangelist

kontakt@kai-waehner.de@KaiWaehnerwww.kai-waehner.deLinkedIn

Questions? Feedback?Please contact me!