machine intelligence at google scale: vision/speech api
TRANSCRIPT
![Page 1: Machine Intelligence at Google Scale: Vision/Speech API](https://reader034.vdocuments.us/reader034/viewer/2022042800/58a1a2801a28abf84d8c4131/html5/thumbnails/1.jpg)
Machine Intelligence at Google Scale:
Vision/Speech API, TensorFlow and Cloud ML
![Page 2: Machine Intelligence at Google Scale: Vision/Speech API](https://reader034.vdocuments.us/reader034/viewer/2022042800/58a1a2801a28abf84d8c4131/html5/thumbnails/2.jpg)
+Kazunori Sato@kazunori_279
Kaz Sato
Staff Developer AdvocateTech Lead for Data & AnalyticsCloud Platform, Google Inc.
![Page 3: Machine Intelligence at Google Scale: Vision/Speech API](https://reader034.vdocuments.us/reader034/viewer/2022042800/58a1a2801a28abf84d8c4131/html5/thumbnails/3.jpg)
What we’ll cover
Deep learning and distributed training
Large scale neural network on Google Cloud
Cloud Vision API and Speech API
TensorFlow and Cloud Machine Learning
![Page 4: Machine Intelligence at Google Scale: Vision/Speech API](https://reader034.vdocuments.us/reader034/viewer/2022042800/58a1a2801a28abf84d8c4131/html5/thumbnails/4.jpg)
Deep Learning andDistributed Training
![Page 5: Machine Intelligence at Google Scale: Vision/Speech API](https://reader034.vdocuments.us/reader034/viewer/2022042800/58a1a2801a28abf84d8c4131/html5/thumbnails/5.jpg)
![Page 6: Machine Intelligence at Google Scale: Vision/Speech API](https://reader034.vdocuments.us/reader034/viewer/2022042800/58a1a2801a28abf84d8c4131/html5/thumbnails/6.jpg)
From: Andrew Ng
![Page 7: Machine Intelligence at Google Scale: Vision/Speech API](https://reader034.vdocuments.us/reader034/viewer/2022042800/58a1a2801a28abf84d8c4131/html5/thumbnails/7.jpg)
DNN = a large matrix ops
a few GPUs >> CPU(but it still takes days to train)
a supercomputer >> a few GPUs(but you don't have a supercomputer)
You need Distributed Training on the cloud
![Page 8: Machine Intelligence at Google Scale: Vision/Speech API](https://reader034.vdocuments.us/reader034/viewer/2022042800/58a1a2801a28abf84d8c4131/html5/thumbnails/8.jpg)
Google Brain.Large scale neural network on Google Cloud
![Page 9: Machine Intelligence at Google Scale: Vision/Speech API](https://reader034.vdocuments.us/reader034/viewer/2022042800/58a1a2801a28abf84d8c4131/html5/thumbnails/9.jpg)
![Page 10: Machine Intelligence at Google Scale: Vision/Speech API](https://reader034.vdocuments.us/reader034/viewer/2022042800/58a1a2801a28abf84d8c4131/html5/thumbnails/10.jpg)
Enterprise
Google Cloud is
The Datacenter as a Computer
![Page 11: Machine Intelligence at Google Scale: Vision/Speech API](https://reader034.vdocuments.us/reader034/viewer/2022042800/58a1a2801a28abf84d8c4131/html5/thumbnails/11.jpg)
Jupiter network
10 GbE x 100 K = 1 Pbps
Consolidates servers with
microsec latency
![Page 12: Machine Intelligence at Google Scale: Vision/Speech API](https://reader034.vdocuments.us/reader034/viewer/2022042800/58a1a2801a28abf84d8c4131/html5/thumbnails/12.jpg)
Borg
No VMs, pure containers
10K - 20K nodes per Cell
DC-scale job scheduling
CPUs, mem, disks and IO
![Page 13: Machine Intelligence at Google Scale: Vision/Speech API](https://reader034.vdocuments.us/reader034/viewer/2022042800/58a1a2801a28abf84d8c4131/html5/thumbnails/13.jpg)
13
Google Cloud +
Neural Network =
Google Brain
![Page 14: Machine Intelligence at Google Scale: Vision/Speech API](https://reader034.vdocuments.us/reader034/viewer/2022042800/58a1a2801a28abf84d8c4131/html5/thumbnails/14.jpg)
The Inception model (GoogLeNet, 2015)
![Page 15: Machine Intelligence at Google Scale: Vision/Speech API](https://reader034.vdocuments.us/reader034/viewer/2022042800/58a1a2801a28abf84d8c4131/html5/thumbnails/15.jpg)
What's the scalability of Google Brain?
"Large Scale Distributed Systems for Training Neural
Networks", NIPS 2015
○ Inception / ImageNet: 40x with 50 GPUs
○ RankBrain: 300x with 500 nodes
![Page 16: Machine Intelligence at Google Scale: Vision/Speech API](https://reader034.vdocuments.us/reader034/viewer/2022042800/58a1a2801a28abf84d8c4131/html5/thumbnails/16.jpg)
Large-scale neural network for everyone
![Page 17: Machine Intelligence at Google Scale: Vision/Speech API](https://reader034.vdocuments.us/reader034/viewer/2022042800/58a1a2801a28abf84d8c4131/html5/thumbnails/17.jpg)
![Page 18: Machine Intelligence at Google Scale: Vision/Speech API](https://reader034.vdocuments.us/reader034/viewer/2022042800/58a1a2801a28abf84d8c4131/html5/thumbnails/18.jpg)
![Page 19: Machine Intelligence at Google Scale: Vision/Speech API](https://reader034.vdocuments.us/reader034/viewer/2022042800/58a1a2801a28abf84d8c4131/html5/thumbnails/19.jpg)
![Page 20: Machine Intelligence at Google Scale: Vision/Speech API](https://reader034.vdocuments.us/reader034/viewer/2022042800/58a1a2801a28abf84d8c4131/html5/thumbnails/20.jpg)
Pre-trained models. No ML skill required
REST API: receives images and returns a JSON
$2.5 or $5 / 1,000 units (free to try)
Public Beta - cloud.google.com/vision
Cloud Vision API
![Page 21: Machine Intelligence at Google Scale: Vision/Speech API](https://reader034.vdocuments.us/reader034/viewer/2022042800/58a1a2801a28abf84d8c4131/html5/thumbnails/21.jpg)
![Page 22: Machine Intelligence at Google Scale: Vision/Speech API](https://reader034.vdocuments.us/reader034/viewer/2022042800/58a1a2801a28abf84d8c4131/html5/thumbnails/22.jpg)
2222
Demo
![Page 23: Machine Intelligence at Google Scale: Vision/Speech API](https://reader034.vdocuments.us/reader034/viewer/2022042800/58a1a2801a28abf84d8c4131/html5/thumbnails/23.jpg)
Pre-trained models. No ML skill required
REST API: receives audio and returns texts
Supports 80+ languages
Streaming or non-streaming
Limited Preview - cloud.google.com/speech
Cloud Speech API
![Page 24: Machine Intelligence at Google Scale: Vision/Speech API](https://reader034.vdocuments.us/reader034/viewer/2022042800/58a1a2801a28abf84d8c4131/html5/thumbnails/24.jpg)
2424
Demo Video
![Page 25: Machine Intelligence at Google Scale: Vision/Speech API](https://reader034.vdocuments.us/reader034/viewer/2022042800/58a1a2801a28abf84d8c4131/html5/thumbnails/25.jpg)
TensorFlow
![Page 26: Machine Intelligence at Google Scale: Vision/Speech API](https://reader034.vdocuments.us/reader034/viewer/2022042800/58a1a2801a28abf84d8c4131/html5/thumbnails/26.jpg)
The Machine Learning Spectrum
TensorFlow Cloud Machine Learning Machine Learning APIs
Industry / applications
Academic / research
![Page 27: Machine Intelligence at Google Scale: Vision/Speech API](https://reader034.vdocuments.us/reader034/viewer/2022042800/58a1a2801a28abf84d8c4131/html5/thumbnails/27.jpg)
Google's open source library for
machine intelligence
tensorflow.org launched in Nov 2015
The second generation
Used by many production ML projects
What is TensorFlow?
![Page 28: Machine Intelligence at Google Scale: Vision/Speech API](https://reader034.vdocuments.us/reader034/viewer/2022042800/58a1a2801a28abf84d8c4131/html5/thumbnails/28.jpg)
What is TensorFlow?
Tensor: N-dimensional array
Flow: data flow computation framework (like MapReduce)
For Machine Learning and Deep Learning
Or any HPC (High Performance Computing) applications
![Page 29: Machine Intelligence at Google Scale: Vision/Speech API](https://reader034.vdocuments.us/reader034/viewer/2022042800/58a1a2801a28abf84d8c4131/html5/thumbnails/29.jpg)
# define the networkimport tensorflow as tfx = tf.placeholder(tf.float32, [None, 784])W = tf.Variable(tf.zeros([784, 10]))b = tf.Variable(tf.zeros([10]))y = tf.nn.softmax(tf.matmul(x, W) + b)
# define a training stepy_ = tf.placeholder(tf.float32, [None, 10])xent = -tf.reduce_sum(y_*tf.log(y))step = tf.train.GradientDescentOptimizer(0.01).minimize(xent)
![Page 30: Machine Intelligence at Google Scale: Vision/Speech API](https://reader034.vdocuments.us/reader034/viewer/2022042800/58a1a2801a28abf84d8c4131/html5/thumbnails/30.jpg)
# initialize sessioninit = tf.initialize_all_variables()sess = tf.Session()sess.run(init)
# trainingfor i in range(1000): batch_xs, batch_ys = mnist.train.next_batch(100) sess.run(step, feed_dict={x: batch_xs, y_: batch_ys})
![Page 31: Machine Intelligence at Google Scale: Vision/Speech API](https://reader034.vdocuments.us/reader034/viewer/2022042800/58a1a2801a28abf84d8c4131/html5/thumbnails/31.jpg)
![Page 32: Machine Intelligence at Google Scale: Vision/Speech API](https://reader034.vdocuments.us/reader034/viewer/2022042800/58a1a2801a28abf84d8c4131/html5/thumbnails/32.jpg)
Portable● Training on:
○ Data Center
○ CPUs, GPUs and etc
● Running on:
○ Mobile phones
○ IoT devices
![Page 33: Machine Intelligence at Google Scale: Vision/Speech API](https://reader034.vdocuments.us/reader034/viewer/2022042800/58a1a2801a28abf84d8c4131/html5/thumbnails/33.jpg)
TensorBoard: visualization tool
![Page 34: Machine Intelligence at Google Scale: Vision/Speech API](https://reader034.vdocuments.us/reader034/viewer/2022042800/58a1a2801a28abf84d8c4131/html5/thumbnails/34.jpg)
Cloud Machine Learning
![Page 35: Machine Intelligence at Google Scale: Vision/Speech API](https://reader034.vdocuments.us/reader034/viewer/2022042800/58a1a2801a28abf84d8c4131/html5/thumbnails/35.jpg)
Fully managed, distributed training and prediction
for custom TensorFlow graph
Supports Regression and Classification initially
Integrated with Cloud Dataflow and Cloud Datalab
Limited Preview - cloud.google.com/ml
Cloud Machine Learning (Cloud ML)
![Page 36: Machine Intelligence at Google Scale: Vision/Speech API](https://reader034.vdocuments.us/reader034/viewer/2022042800/58a1a2801a28abf84d8c4131/html5/thumbnails/36.jpg)
![Page 37: Machine Intelligence at Google Scale: Vision/Speech API](https://reader034.vdocuments.us/reader034/viewer/2022042800/58a1a2801a28abf84d8c4131/html5/thumbnails/37.jpg)
Distributed Training with TensorFlow
![Page 38: Machine Intelligence at Google Scale: Vision/Speech API](https://reader034.vdocuments.us/reader034/viewer/2022042800/58a1a2801a28abf84d8c4131/html5/thumbnails/38.jpg)
● CPU/GPU scheduling
● Communications
○ Local, RPC, RDMA
○ 32/16/8 bit quantization
● Cost-based optimization
● Fault tolerance
Distributed Training with TensorFlow
![Page 39: Machine Intelligence at Google Scale: Vision/Speech API](https://reader034.vdocuments.us/reader034/viewer/2022042800/58a1a2801a28abf84d8c4131/html5/thumbnails/39.jpg)
Data Parallelism = split data, share model
(but ordinary network is
1,000x slower than GPU and
doesn't scale)
![Page 40: Machine Intelligence at Google Scale: Vision/Speech API](https://reader034.vdocuments.us/reader034/viewer/2022042800/58a1a2801a28abf84d8c4131/html5/thumbnails/40.jpg)
Cloud ML demo video
![Page 41: Machine Intelligence at Google Scale: Vision/Speech API](https://reader034.vdocuments.us/reader034/viewer/2022042800/58a1a2801a28abf84d8c4131/html5/thumbnails/41.jpg)
Jeff Dean's keynote: YouTube video
Define a custom TensorFlow graph
Training at local: 8.3 hours w/ 1 node
Training at cloud: 32 min w/ 20 nodes (15x faster)
Prediction at cloud at 300 reqs / sec
Cloud ML demo
![Page 42: Machine Intelligence at Google Scale: Vision/Speech API](https://reader034.vdocuments.us/reader034/viewer/2022042800/58a1a2801a28abf84d8c4131/html5/thumbnails/42.jpg)
Summary
![Page 43: Machine Intelligence at Google Scale: Vision/Speech API](https://reader034.vdocuments.us/reader034/viewer/2022042800/58a1a2801a28abf84d8c4131/html5/thumbnails/43.jpg)
Ready to use Machine Learning models
Use your own data to train models
Cloud Vision API
Cloud Speech API
Cloud Translate API
Cloud Machine Learning
Develop - Model - Test
Google BigQuery
Stay Tuned….
Cloud Storage
Cloud Datalab
NEW
Alpha
GA BetaGA
AlphaBeta
GA
![Page 44: Machine Intelligence at Google Scale: Vision/Speech API](https://reader034.vdocuments.us/reader034/viewer/2022042800/58a1a2801a28abf84d8c4131/html5/thumbnails/44.jpg)
Links & Resources
Large Scale Distributed Systems for Training Neural Networks, Jeff Dean and Oriol Vinals
Cloud Vision API: cloud.google.com/vision
Cloud Speech API: cloud.google.com/speech
TensorFlow: tensorflow.org
Cloud Machine Learning: cloud.google.com/ml
Cloud Machine Learning: demo video
![Page 45: Machine Intelligence at Google Scale: Vision/Speech API](https://reader034.vdocuments.us/reader034/viewer/2022042800/58a1a2801a28abf84d8c4131/html5/thumbnails/45.jpg)
Thank you!