gluon: new mxnet interface to accelerate research · tensorflow microsoft cntk xnet . 4549 lines...
TRANSCRIPT
![Page 1: Gluon: new MXNet interface to accelerate research · TensorFlow Microsoft CNTK xnet . 4549 lines (4698 sloc) 63.9 KB le 11 2e 21 name : input : " ResNet-1ê1 " "data" 1 3 224 224](https://reader030.vdocuments.us/reader030/viewer/2022040611/5ed6e9d8ff4a11075f7708bf/html5/thumbnails/1.jpg)
Gluon: new MXNet interface to accelerate research
Mu Li AWS Deep Learning
https://mli.github.io/cvpr17/
![Page 2: Gluon: new MXNet interface to accelerate research · TensorFlow Microsoft CNTK xnet . 4549 lines (4698 sloc) 63.9 KB le 11 2e 21 name : input : " ResNet-1ê1 " "data" 1 3 224 224](https://reader030.vdocuments.us/reader030/viewer/2022040611/5ed6e9d8ff4a11075f7708bf/html5/thumbnails/2.jpg)
2
2012before 2013 2014 2015 2016 2017
mxnetimperativesymbolic
![Page 3: Gluon: new MXNet interface to accelerate research · TensorFlow Microsoft CNTK xnet . 4549 lines (4698 sloc) 63.9 KB le 11 2e 21 name : input : " ResNet-1ê1 " "data" 1 3 224 224](https://reader030.vdocuments.us/reader030/viewer/2022040611/5ed6e9d8ff4a11075f7708bf/html5/thumbnails/3.jpg)
Caffe
3
ResNet-101-deploy.prototxt
✦ Protobuf as the interface ✦ Portable ❖ caffe binary + protobuf model
✦ Reading and writing protobuf are not straightforward
…. (4K lines of codes)
![Page 4: Gluon: new MXNet interface to accelerate research · TensorFlow Microsoft CNTK xnet . 4549 lines (4698 sloc) 63.9 KB le 11 2e 21 name : input : " ResNet-1ê1 " "data" 1 3 224 224](https://reader030.vdocuments.us/reader030/viewer/2022040611/5ed6e9d8ff4a11075f7708bf/html5/thumbnails/4.jpg)
Tensorflow
4
✦ A rich set of operators (~2000) ✦ The codes are not very easy to
read, e.g. not python-like
Implement Adam
> 300 lines of codes
![Page 5: Gluon: new MXNet interface to accelerate research · TensorFlow Microsoft CNTK xnet . 4549 lines (4698 sloc) 63.9 KB le 11 2e 21 name : input : " ResNet-1ê1 " "data" 1 3 224 224](https://reader030.vdocuments.us/reader030/viewer/2022040611/5ed6e9d8ff4a11075f7708bf/html5/thumbnails/5.jpg)
Keras
✦ Simple and easy to use ✦ Difficult to implement
sophisticated algorithms
5
![Page 6: Gluon: new MXNet interface to accelerate research · TensorFlow Microsoft CNTK xnet . 4549 lines (4698 sloc) 63.9 KB le 11 2e 21 name : input : " ResNet-1ê1 " "data" 1 3 224 224](https://reader030.vdocuments.us/reader030/viewer/2022040611/5ed6e9d8ff4a11075f7708bf/html5/thumbnails/6.jpg)
Pytorch & Chainer
✦ Flexible ✦ Complicate programs might
be slow to run
6
![Page 7: Gluon: new MXNet interface to accelerate research · TensorFlow Microsoft CNTK xnet . 4549 lines (4698 sloc) 63.9 KB le 11 2e 21 name : input : " ResNet-1ê1 " "data" 1 3 224 224](https://reader030.vdocuments.us/reader030/viewer/2022040611/5ed6e9d8ff4a11075f7708bf/html5/thumbnails/7.jpg)
MXNet
✦ Symbolic on network definition ✦ Imperative on tensor computation ✦ Huh.., not good enough
7
Implement Resnet
Implement Adam
![Page 8: Gluon: new MXNet interface to accelerate research · TensorFlow Microsoft CNTK xnet . 4549 lines (4698 sloc) 63.9 KB le 11 2e 21 name : input : " ResNet-1ê1 " "data" 1 3 224 224](https://reader030.vdocuments.us/reader030/viewer/2022040611/5ed6e9d8ff4a11075f7708bf/html5/thumbnails/8.jpg)
8
2012before 2013 2014 2015 2016 2017
mxnetimperative
symbolicgluon
![Page 9: Gluon: new MXNet interface to accelerate research · TensorFlow Microsoft CNTK xnet . 4549 lines (4698 sloc) 63.9 KB le 11 2e 21 name : input : " ResNet-1ê1 " "data" 1 3 224 224](https://reader030.vdocuments.us/reader030/viewer/2022040611/5ed6e9d8ff4a11075f7708bf/html5/thumbnails/9.jpg)
Gluon at a glance
9
net.hybridize() converts from
imperative to symbolic execution
![Page 10: Gluon: new MXNet interface to accelerate research · TensorFlow Microsoft CNTK xnet . 4549 lines (4698 sloc) 63.9 KB le 11 2e 21 name : input : " ResNet-1ê1 " "data" 1 3 224 224](https://reader030.vdocuments.us/reader030/viewer/2022040611/5ed6e9d8ff4a11075f7708bf/html5/thumbnails/10.jpg)
In summary✦ Symbolic ❖ efficient & portable ❖ but hard to use
10
✦ tesla
✦ Imperative ❖ flexible ❖ may be slow
✦ Gluon ❖ imperative for developing ❖ symbolic for deploying