Enterprise Deep Learning Workflows

With DL4JJosh Patterson

Strata Singapore 2015

Presenter: Josh Patterson

Past

Research in Swarm Algorithms: Real-time optimization techniques in mesh sensor networks

TVA / NERC: Smartgrid, Sensor Collection, and Big Data

Cloudera: Principal SA, Working with Fortune 500

Today

Patterson Consulting

Skymind (Advisor)

josh@pattersonconsultingtn.com / @jpatanoogaDL4J Co-creator, Co-Author on Upcoming Oreilly Book “Deep

Learning: A Practitioner’s Approach”

Topics

• What is Deep Learning?• Operating in the Enterprise• DL4J• Enterprise Grade Deep Learning Workflows

WHAT IS DEEP LEARNING?

We Want to be able to recognize Handwriting

This is a Hard Problem

Automated Feature Engineering

• Deep Learning can be thought of as workflows for automated feature construction– Where previously we’d consider each stage in the

workflow as unique technique• Many of the techniques have been around for

years– But now are being chained together in a way that

automates exotic feature engineering• As LeCun says:– “machines that learn to represent the world”

These are the features learned at each neuron in a Restricted Boltzmann Machine (RBMS)

These features are passed to higher levels of RBMs to learn more complicated things.

Part of the “7” digit

Learning Progressive Layers

Deep Learning Architectures

• Deep Belief Networks– Most common architecture

• Convolutional Neural Networks– State of the art in image classification

• Recurrent Networks– Timeseries – Any sequence of input vectors

• Recursive Networks– Text / image– Can break down scenes

ENTERPRISEOperating in the

Enterprise User Types• Data Businesspeople

– product and profit-focused– manager with a technical bent.– engineering degree paired with an MBA

• Data Creatives – jacks-of-all-trades– artists or hackers

• Data Developers– write software to do analytic, statistical, and machine learning tasks– computer science degrees, often work with so-called “big data”

• Data Researchers– PhDs - apply their scientific training,– Use of mathematical tools yields valuable insights and products

Python Island

The “Hotel California” of Enterprise Data Science

Issues in Machine Learning

• Data Gravity– We need to process the data in workflows where the data lives

• If you move data you don’t have big data

– Even if the data is not “big” we still want simpler workflows• Integration Issues

– Ingest, ETL, Vectorization, Modeling, Evaluation, and Deployment issues

– Most ML tools are built with previous generation architectures in mind

• Legacy Architectures– Parallel iterative algorithm architectures are not common

So What Do We Need For the Enterprise?

• Ability to work with small and big data easily– Don’t want to change tooling because we moved to

Hadoop• Ability to not get caught up in things like

vectorization and ETL– Need to focus on better models, less on minutia– (Knowing data is still important)

• Ability to experiment with lots of models– Yet be able to take something to production without a

huge headache / re-write

DL4JNext Generation Deep Learning with

DL4J• “The Hadoop of Deep Learning”

– Command line driven– Java, Scala, and Python APIs– ASF 2.0 Licensed

• Java implementation– Parallelization (Yarn, Spark)– GPU support

• Also Supports multi-GPU per host

• Runtime Neutral– Local– Hadoop / YARN + Spark– AWS

• https://github.com/deeplearning4j/deeplearning4j

DL4J Suite of Tools

• DL4J– Main library for deep learning

• Canova– Vectorization library

• ND4J– Linear Algebra framework – Swappable backends (JBLAS, GPUs):

• http://www.slideshare.net/agibsonccc/future-of-ai-on-the-jvm

• Arbiter– Model evaluation and testing platform

Github Stars• Caffe: 7,728• Theano: 2,502• DL4J: 1,663

Other Popular Projects• Mesos: 1,885• Kafka: 2,286

DL4J API Example SVMLightRecordReader recordReader = new SVMLightRecordReader(); recordReader.initialize(svm_conf, new FileSplit( svmLightDemo )); DataSetIterator iterator = new RecordReaderDataSetIterator( recordReader, batchSize, 16, 2 );

SVMLightRecordReader recordReaderTestData = new SVMLightRecordReader(); recordReaderTestData.initialize(svm_conf, new FileSplit( new File( ”./data/full_balanced_svmlight_dataset.txt" ) ) ); DataSetIterator iterator_test = new RecordReaderDataSetIterator( recordReaderTestData, testbatchSize, 16, 2 ); DataSet next_test = iterator_test.next(); MultiLayerConfiguration conf = new NeuralNetConfiguration.Builder() .seed(123) .optimizationAlgo(OptimizationAlgorithm.LINE_GRADIENT_DESCENT) .iterations( iterations ) .list(3) .layer(0, new DenseLayer.Builder() .nIn( numColumns ).nOut(80) .weightInit(WeightInit.XAVIER) .activation("relu") .build()) .layer(1, new DenseLayer.Builder() .nIn( 80 ).nOut(60) .weightInit(WeightInit.XAVIER) .activation("relu") .build()) .layer(2, new org.deeplearning4j.nn.conf.layers.OutputLayer.Builder(LossFunctions.LossFunction.MCXENT) .nIn(60).nOut(2) .activation("softmax") .weightInit(WeightInit.XAVIER) .build()) .backprop(true) .build();

MultiLayerNetwork model = new MultiLayerNetwork(conf); model.init(); model.fit(iterator);

DEEP LEARNING WORKFLOWSEnterprise Grade

Building Deep Learning Workflows

• Need to have flexibility where we build / apply the model– Local – AWS– Spark / Hadoop

• We need to get data from a raw format into a baseline raw vector– Model the data– Evaluate the Model

• Traditionally these are all tied together in one tool– But this is a monolithic pattern– We’d like to apply the unix principles here

• The DL4J Suite of Tools lets us do this

Loading Existing ModelsString jsonModelConf = loadTextFileFromDisk( pathToModelJSON );

MultiLayerConfiguration confFromJson = MultiLayerConfiguration.fromJson( jsonModelConf );

FSDataInputStream hdfsInputStream_ModelParams = hdfs.open(new Path( hdfsPathToModelParams ));

DataInputStream dis = new DataInputStream( hdfsInputStream_ModelParams );INDArray newParams = Nd4j.read( dis );dis.close();MultiLayerNetwork savedNetwork = new MultiLayerNetwork( confFromJson );savedNetwork.init();savedNetwork.setParameters(newParams);

DL4J Spark JobMultiLayerConfiguration confFromJson = MultiLayerConfiguration.fromJson( jsonBuffer.toString() );MultiLayerNetwork savedNetwork = new MultiLayerNetwork( confFromJson );savedNetwork.init();

RDD<LabeledPoint> evaluate_svmLight_data_rdd = MLUtils.loadLibSVMFile( sc.sc(), hdfsFilesToTrainModel, true, parsedSVMLightFeatureColumns );

JavaRDD<LabeledPoint> evaluate_svmLight_data_JavaRDD = JavaRDD.fromRDD( evaluate_svmLight_data_rdd, ClassManifestFactory.fromClass(LabeledPoint.class) );

SparkDl4jMultiLayer trainLayerSpark = new SparkDl4jMultiLayer( sc.sc(), confFromJson );

MultiLayerNetwork trainedNetwork = trainLayerSpark.fit( sc, evaluate_svmLight_data_JavaRDD );

final SparkDl4jMultiLayer completedModelFromSparkToSave = new SparkDl4jMultiLayer(sc.sc(),trainedNetwork);

https://github.com/deeplearning4j/dl4j-spark-cdh5-examples

Turn on GPUs and Spark

<dependency> <groupId>org.deeplearning4j</groupId> <artifactId>dl4j-spark</artifactId> <version>${dl4j.version}</version> </dependency>

<dependency> <groupId>org.nd4j</groupId> <artifactId>nd4j-jcublas-7.0</artifactId> <version>${nd4j.version}</version> </dependency>

Modeling UCI Data: Iris

• We then need to build a deep learning model over the data– We can use the DL4J lib to do this

• We need to vectorize the data– Possibly with some per column transformations

Vectorizing Data5.1,3.5,1.4,0.2,Iris-setosa4.9,3.0,1.4,0.2,Iris-setosa4.7,3.2,1.3,0.2,Iris-setosa7.0,3.2,4.7,1.4,Iris-versicolor

Becomes

0.0 1:0.1666666666666665 2:1.0 3:0.021276595744680823 4:0.00.0 1:0.08333333333333343 2:0.5833333333333334 3:0.021276595744680823 4:0.00.0 1:0.0 2:0.7500000000000002 3:0.0 4:0.01.0 1:0.9583333333333335 2:0.7500000000000002 3:0.723404255319149 4:0.5217391304347826

Canova for Command Line Vectorization

• Library of tools to take – Audio– Video– Image– Text– CSV data

• And convert the input data into vectors in a standardized format (SVMLight, others)– Adaptable with custom input/output formats

• Open Source, ASF 2.0 Licensed– https://github.com/deeplearning4j/Canova– Part of DL4J suite

Workflow Configuration (iris_conf.txt)

canova.input.header.skip=falsecanova.input.statistics.debug.print=falsecanova.input.format=org.canova.api.formats.input.impl.LineInputFormat

canova.input.directory=src/test/resources/csv/data/uci_iris_sample.txtcanova.input.vector.schema=src/test/resources/csv/schemas/uci/iris.txtcanova.output.directory=/tmp/iris_unit_test_sample.txt

canova.output.format=org.canova.api.formats.output.impl.SVMLightOutputFormat

Iris Canova Vector Schema@RELATION UCIIrisDataset@DELIMITER ,

@ATTRIBUTE sepallength NUMERIC !NORMALIZE @ATTRIBUTE sepalwidth NUMERIC !NORMALIZE @ATTRIBUTE petallength NUMERIC !NORMALIZE @ATTRIBUTE petalwidth NUMERIC !NORMALIZE @ATTRIBUTE class STRING !LABEL

Model UCI Iris From CLI./bin/canova vectorize -conf /tmp/iris_conf.txt File path already exists, deleting the old file before proceeding...Output vectors written to: /tmp/iris_svmlight.txt

./bin/dl4j train –conf /tmp/iris_conf.txt

[ …log output… ]

./bin/arbiter evaluate –conf /tmp/iris_conf.txt

[ …log output… ]

Questions?

Thank you for your time and attention

“Deep Learning: A Practitioner’s Approach” (Oreilly, Q2 2016)