gatsby kaken-2017-pfn okanohara
TRANSCRIPT
AI in real worldAutomobile, Robotics,
Bio/Healthcare and Art Creation
Daisuke Okanohara
Preferred Networks
May. 11 2017@Gatsby-Kaken Joint Workshop
Preferred Networks (PFN)
“Make everything intelligent and collaborative”
Founded : March. 2014 (Founder:Toru Nishikawa (CEO), Daisuke Okanohara (EVP))
Office: Tokyo, San Mateo
Employees: ~80 (doubles every year)
Investors: FANUC, Toyota, NTT
2
Preferred Networks’ positioning in AI: Industrial IoT
3
Consumer Industrial
Cloud
Device
Infrastructure
Factory
Robot
Automotive
Healthcare
Smart City
Industry4.0
Industrial
Edge-side
Automobile
Robotics
Anomaly Detection
Example: FANUC Reducer Anomaly Detection[Presented at iREX 2015]
7
Anomaly detection using deep generative models
No anomaly
Found anomalies
Normal Anomaly
Actual sensor data from reducers
Can predict the failure much earlier thanthe existing methods
We heavily use deep generative models to detect anomalies
Deep learning based methods
異常スコア
Detect 40 days before the failure
Threshold
Existing methods
Elapsed time
Detect just before the failure
Robot failure
Robot failure
15日前
Life Science
The National Cancer Center in Japan and Preferred Networks start collaborative research in deep learning
Accuracy for Breast Cancer Diagnosis
90%
99%
80%Mammography
SOTA Liquid Biopsy
SOTA Liquid Biopsy
with Deep Learning
Art Creator
Random sampling of images using GAN [2015]
13
PaintsChainer (#PaintsChainer)
GAN training. U-Net + Super-resolution
Released Jan. 2017, and already painted about one
million line images
Much cooler newer version will be released soon
http://free-illustrations.gatag.net/2014/01/10/220000.html
PaintsChainer
Tweet from @munashihc
Technologies
Chainer : Flexible deep learning framework
https://github.com/pfnet/chainer
113 contributors
2,473 stars & 639 fork
8,804 commits
Active development & release
— v1.0.0 (June 2015) to v1.23.0 (May 2017)
17
Original developer
Seiya Tokui
ChainerRL: deep reinforcement learning library[2016]
Implements various SOTA deep RL algorithms
— User can quickly try Atari 2600 and openAI gym tasks
Yasuhisa Fujita
To process this huge amount of data, we need to apply parallel computing to deep learning
ChainerMNScalable Trainining of Deep Learning Model
ChainerMN
developer
Takuya Akiba
Scaling Result for CNTK, MXNet, TensorFlow and Chainer
Validation Accuracy against # of GPUs
23
Future AI needs 100Exa ~ 1Zeta flops
1E〜100E Flops1TB /car / day10~1000 cars, 100days
Life Science
Speech Rec. Robotics/Drone
10P〜 Flops
5000 hours of speech, 0.1 miliion of generated speech[Baidu 2015]
100P 〜 1E Flops10M SNPs per person. 100PF for 1million, 1EF for100 million.
10P(Image) 〜 10E(Video) Flops
100million images,
ImageVideo Rec.
1E〜100E Flops
1TB/device/year1million ~ 100 milliondevices
Autonomous Driving
10PF 100EF100PF 1EF 10EF
P:Peta E:ExaF:Flops
Machine generated data is much bigger than human generated data
These estimation is based on;
To finish training using 1GB within 1day require 1Tflops
Computing Infrastructure
Current PFN’s infrastructure
— >1000 GPUs, ~ 10PFlops, connected by InfiniBand in 2Q 2017
— Still not enough for current R&D demand
Unsupervised learning, learning from Video, RL
We are developing a new chip specialized for DL ops
— Super power-efficient chip enable ~1 Peta DL ops per 1Chip
— Plan to build a cluster capable of 1 Exa DL ops by 2019
Since brain has 1 Zeta Flops*1, we require more resource
— We expect to have such a cluster by 2034
— This is optimistic, but expect several new technology will emerge
24
*1 http://timdettmers.com/2015/07/27/brain-vs-deep-learning-singularity/
Semi-supervised LearningVirtually Adversarial Training [arxiv:1704.03976]
SoTA of semi-supervised learning on CIFAR-10, SVHNTakeru Miyato
* CIFAR-10, SVHNを含んだ実験結果は投稿準備中
IMSAT(VAT) [Hu and Miyato 17]
IMSAT: VAT + Information Maximization CriterionUnsup. Discrete Coding
SoTA on Unsup. Clusteirng and Hash Learning
Result during 2016 summer internship
Conclusion and Future Work
Recognition to planning, controlling, and creation
— Deep learning was first used in recognition tasks but now used for
many different tasks
Future Work
— Increase data and computing resources significantly (x1000) ?
Generate high-volume data in real world (use robotics?)
New hardware and networks achieving 1 Zeta flops
— Interpretability and controllability of AI systems in critical tasks
— A new way to accumulate these obtained knowledges
New language, and communication for machines (and human)
— We can learn a lot from brain research