Download - Deep learning on Hadoop/Spark -NextML
DEEP
LEARNING
ON HADOOP
ADAM GIBSON // DL4J
presentation@
Scale out Deep Learning on YARN
Next.ML
deeplearning4j.org
presentation@
ADAM / DL4J
● Adam:
○ DL specialist, in residence at Galvanize
○ Head programmer in DL4J project
● SkyMind
○ Commercial support arm of Deeplearning4j
● Deep Learning for Java (DL4J)
○ a distributed, open-source deep-learning framework
for the JVM
○ has most major deep neural nets
deeplearning4j.org
presentation@
OVERVIEW
● What is Deep Learning?
● DL major types of Neural Networks:
a. Feed-forward NN
b. Recurrent NN
c. Convolutional NN
d. Recursive NN
● Scaling DL4J with HADOOP
● Demo
deeplearning4j.org
presentation@
WHAT’S IT GOOD FOR?
● Recommendation engines: anticipate what you will buy or click
● Anomaly detection:
○ fraud in e-commerce
○ tumors in X-rays
○ loans’ likely to default
● Signal processing:
○ customer lifetime value estimation
○ necessary inventory
○ approaching market crash
● Image
deeplearning4j.org
presentation@
FACIAL RECOGNITION &
FEATURE HIERARCHY
DEEP means “high level
abstractions achieved
by using model
architectures composed
of multiple non-linear
transformations”, typically
traditional Machine
Learning algorithms,
mostly Neural-Network
based
(source: Wikipedia)
deeplearning4j.org
presentation@
LEARNING PROPERTIES OF DL
● They can learn unsupervised data
● They save data scientists months of work
● They eat vectors and learn patterns based on occurrence
statistics
● They can handle millions of parameters at a time
● After training, they can be deployed for fast classification
● High level representation are small vectors, achieving
deeplearning4j.org
presentation@
FEED FORWARD NN
● Perceptron (1957)
● Back-prop learning
● No loops
● Can have different
topologies, typically
multi layered
deeplearning4j.org
presentation@
RECURRENT NN
● Loops
● Temporal
behavior
● Used for
temporal
series
deeplearning4j.org
presentation@
CONVOLUTIONAL NN
Specific type of Feed-Forward NN, impressive recent results
● Neocognitron (Fukushima’80)
● Improvement (LeCun’98)
● Google Brain (Ng’11)
deeplearning4j.org
presentation@
RECURSIVE NN
● Recursive AutoencoderDeals with variable-length input, has a topological structure, good for NLP
deeplearning4j.org
presentation@
SGD: Serial vs. Parallel
Model
Training Data
Worker 1
Master
Partial
Model
Global Model
Worker 2
Partial Model
Worker N
Partial
Model
Split 1 Split 2 Split 3
…
deeplearning4j.org
presentation@
MapReduce vs. Parallel iterative
Input
Output
Processor Processor Processor
Superstep 1
Processor Processor
Superstep 2
. . .
Processor