TENSORFLOW + SPARK DATAFRAMES=

TENSORFRAMES

Tallinn Advanced Java MeetupOct 24, 2016Chris Fregly

Research Scientist @ PipelineIO

Thank You for Hosting, Planet OS!!

WHO AM I

Chris Fregly

• Currently

Research Scientist @ PipelineIO (http://pipeline.io)

Contributor @ Apache Spark

Committer @ Netflix Open Source

Founder @ Advanced Spark and TensorFlow Meetup

Author @ Advanced Spark (http://advancedspark.com)

Creator @ PANCAKE STACK (http://pancake-stack.com)

• Previously

Streaming Data Engineer @ Netflix, Databricks, IBM Spark

ADVANCED SPARK AND TENSORFLOWMEETUP

4,600 Members+

Top 4 Spark Meetup!

Github Repo Stars + Forks

DockerHub Repo Pulls

CURRENT PIPELINE.IO RESEARCH

• Model Deploying and Testing

• Model Scaling and Serving

• Online Model Training

• Dynamic Model Optimizing

PIPELINE.IO DELIVERABLES

• 100% Open Source!!

• Github:• https://github.com/fluxcapacitor/

• DockerHub• https://hub.docker.com/r/fluxcapacitor

PIPELINE.IO WORKSHOPS

AGENDA

• Neural Networks

• GPUs

• Tensorflow

• TensorFrames

WHAT ARE NEURAL NETWORKS?

• Like All Machine Learning, Goal is to Minimize Loss (Error)• Mostly Supervised Learning Classification• Many labeled training samples exist

• Training• Step 1: Start with Random Guesses for Input Weights

• Step 2: Calculate Error Against Labeled Data

• Step 3: Determine Gradient Amount and Direction (+ or -)• Step 4: Back-propagate Gradient to Update Each Input Weight

• Step 5: Repeat Step 1 until Convergence or Max Epochs Reached

BACK PROPAGATION

http://kratzert.github.io/2016/02/12/understanding-the-gradient-flow-through-the-batch-normalization-layer.html

Chain Rule

CONVOLUTIONAL NEURAL NETWORKS

• Apply Many Layers (aka. Filters) to Input

• Each Layer/Filter Picks up on Features

• Features not necessarily human-grokkable

• Brute Force –Try Diff numLayers & layerSizes

• Filter Examples

• 3 Color Filters: RGB

• Moving AVG for Time Series

MY FAVORITE USE CASE – STITCH FIX

StitchFix Strata Conf SF 2016:Using Deep Learning to Create New Clothing Styles!

RECURRENT NEURAL NETWORKS

Maintain State

Enables Learning of Sequential Patterns

Uses for Text/NLP Prediction

CHARACTER RNNS

Preserving State differentiates between

1st and 2nd ‘l’to improve prediction

AGENDA

• Neural Networks

• GPUs

• Tensorflow

• TensorFrames

CPU VS GPU

• Fundamentally Different than CPUs

• Therefore, GPU/CUDA Programming Fundamentally Different

SAME INSTRUCTION, MULTIPLE DATA

MINIMIZE DATA DEPENDENCIES

• More natural for structured, independent data

• Tasks perform identical instructions in parallel on same-structured data

• Reduce data dependencies as they limit parallelism

Previous Instruction Previous Loop Iteration

MEMORY AND CORES

EXPLORE YOUR SURROUNDINGS

`nvidia-smi`

AGENDA

• Neural Networks

• GPUs

• Tensorflow

• TensorFrames

WHAT IS TENSORFLOW?

• Google Open Source General Purpose Numerical Computation Engine• Happens to be Good for Neural Networks!

• Tooling• Tensorboard (port 6006 == `goog` upside down!) à

• DAG-based like Spark!• Computation graph is logical plan• Stored in Protobuf ’s• Tensorflow converts logical to physical plan

• Lots of Libraries• TFLearn (Tensorflow’s Scikit-learn Impl)• Tensorflow Serving (Prediction Layer) à

• Distributed and GPU-Optimized CUDA/cuDNN

DEMO!

AWS + Docker + GPU + Docker + Tensorflow

DEMO!

Tensorflow Serving

AGENDA

• Neural Networks

• GPUs

• Tensorflow

• TensorFrames

WHAT ARE TENSORFRAMES?

• Bridge between Spark (JVM) and Tensorflow (C++)

• Python and Scala Bindings for Application Code

• Uses JavaCPP for JNI-level Integration

• Must Install TensorFrames C++ Runtime Libs on All Spark Workers

• Developed by Old Co-worker @ Databricks, Tim Hunter• PhD in Tensors – He’s ”Mr.. Tensor”

WHY TENSORFRAMES?

• Why Not?!

• Best of Both Worlds: Legacy Spark Support + Tensorflow

• Mix and Match Spark ML + Tensorflow AI on Same Data

• Tensorflow is DAG-based Similar to Spark

• Enables Data-Parallel Model Training

DATA-PARALLEL MODEL TRAINING

• Large Dataset are Partitioned Across HDFS Cluster

• Computation Graph (Logical Plan) Passed to Spark Workers

• Workers Train on Each Data Partition in Parallel

• Workers Periodically Aggregate (ie. AVG) Results

• Aggregations happen in “Parameter Server”

• Spark Master/Driver is Parameter Server

Computation Graph (logical plan) is passed to every Spark

TENSORFLOW + MULTIPLE HOSTS/GPUS

Multi-GPU, Data-Parallel Training

Step 1: CPU transfers model replica and (initial) gradients to each GPU

Step 2: CPU synchronizes and waits for all GPUs to process batch

Step 3: CPU copies all training results (gradients) back from GPU

Step 4: CPU averages gradients from all GPUs

Step 5: Repeat Step 1 with (new) gradients

Code

https://github.com/tensorflow/tensorflow/blob/master/tensorflow/models/image/cifar10/cifar10_multi_gpu_train.py

TENSORFRAME PERFORMANCE

• Depends on Algorithm and Dataset, of course!

• TensorFrames Require Extra Serialization JVM <-> C++

• What about Python Serialization from Python Bindings?

• Should be minimal unless using Python UDFs

• PySpark keeps small logical plan in Python layer

• Physical operations happen in JVM (except Python UDFs!)

DEMO!

TensorFrames in Python and Scala

THANK YOU!!

Chris Fregly, Research Scientist @ PipelineIO

• LinkedIn: https://linkedin.com/in/cfregly

• Twitter: @cfregly

http://pipeline.io