a javascript library to run onnx models in browsers and node•a pure javascript implementation of...
TRANSCRIPT
![Page 1: A Javascript library to run ONNX models in browsers and Node•A pure JavaScript implementation of ONNX framework •Optimize ONNX model inference on both CPUs and GPUs •Support](https://reader033.vdocuments.us/reader033/viewer/2022060906/60a140ea954e6c027f1ea2c0/html5/thumbnails/1.jpg)
ONNX.jsA Javascript library to run ONNX models
in browsers and Node.js
Emma Ning| Senior product manager, Microsoft
![Page 2: A Javascript library to run ONNX models in browsers and Node•A pure JavaScript implementation of ONNX framework •Optimize ONNX model inference on both CPUs and GPUs •Support](https://reader033.vdocuments.us/reader033/viewer/2022060906/60a140ea954e6c027f1ea2c0/html5/thumbnails/2.jpg)
JS-based Client-side applications
• Website
• Electron APP
• A framework for building cross platform desktop apps with JavaScript, HTML, and CSS
• Compatible with Mac, Windows, and Linux
![Page 3: A Javascript library to run ONNX models in browsers and Node•A pure JavaScript implementation of ONNX framework •Optimize ONNX model inference on both CPUs and GPUs •Support](https://reader033.vdocuments.us/reader033/viewer/2022060906/60a140ea954e6c027f1ea2c0/html5/thumbnails/3.jpg)
JS-based Client-side Machine Learning
• Benefits of Client-side Machine Learning
• Privacy protection
• Real-time Analysis
• Offline Capability
• More with JS-based Client-side Machine Learning
• Cross-platform support
• GPU acceleration
• Easy Access and distribution
![Page 4: A Javascript library to run ONNX models in browsers and Node•A pure JavaScript implementation of ONNX framework •Optimize ONNX model inference on both CPUs and GPUs •Support](https://reader033.vdocuments.us/reader033/viewer/2022060906/60a140ea954e6c027f1ea2c0/html5/thumbnails/4.jpg)
ONNX.JS Background - ONNX
Open Neural Network Exchange
• Consisting of:
• common Intermediate Representation
• full operator spec
• Model = graph composed of computational nodes
• Supports both DNN and traditional ML
• Backward compatible with comprehensive versioning
An open standard for representing machine learning models
![Page 5: A Javascript library to run ONNX models in browsers and Node•A pure JavaScript implementation of ONNX framework •Optimize ONNX model inference on both CPUs and GPUs •Support](https://reader033.vdocuments.us/reader033/viewer/2022060906/60a140ea954e6c027f1ea2c0/html5/thumbnails/5.jpg)
ONNX converters for popular frameworks
• Native Support
• Pytorch
• CNTK
• Open sourced Converter Tools
• Tensorflow: onnx/tensorflow-onnx
• Keras: onnx/keras-onnx
• Scikit-learn: onnx/sklearn-onnx
• CoreML: onnx/onnxmltools
• LightGBM: onnx/onnxmltools
• LibSVM: onnx/onnxmltools
• XGBoost: onnx/onnxmltools
LightGB
M
![Page 6: A Javascript library to run ONNX models in browsers and Node•A pure JavaScript implementation of ONNX framework •Optimize ONNX model inference on both CPUs and GPUs •Support](https://reader033.vdocuments.us/reader033/viewer/2022060906/60a140ea954e6c027f1ea2c0/html5/thumbnails/6.jpg)
Community
![Page 7: A Javascript library to run ONNX models in browsers and Node•A pure JavaScript implementation of ONNX framework •Optimize ONNX model inference on both CPUs and GPUs •Support](https://reader033.vdocuments.us/reader033/viewer/2022060906/60a140ea954e6c027f1ea2c0/html5/thumbnails/7.jpg)
ONNX.JS
• A pure JavaScript implementation of ONNX framework
• Optimize ONNX model inference on both CPUs and GPUs
• Support a variety of browsers on major OSes
graph engine
Model file
Model DAG
Execution engine
CPU - JSGPU -
WebGL
backend profiler
logger
utils
Input data
Output data
CPU -WASM
![Page 8: A Javascript library to run ONNX models in browsers and Node•A pure JavaScript implementation of ONNX framework •Optimize ONNX model inference on both CPUs and GPUs •Support](https://reader033.vdocuments.us/reader033/viewer/2022060906/60a140ea954e6c027f1ea2c0/html5/thumbnails/8.jpg)
ONNX.js
For running on CPU
• Leverage WebAssembly to execute model at near-native speed
• Leverage Web Workers to provide a "multi-threaded" environment for
parallelization in operators
![Page 9: A Javascript library to run ONNX models in browsers and Node•A pure JavaScript implementation of ONNX framework •Optimize ONNX model inference on both CPUs and GPUs •Support](https://reader033.vdocuments.us/reader033/viewer/2022060906/60a140ea954e6c027f1ea2c0/html5/thumbnails/9.jpg)
ONNX.js
For running on GPUs
• Leverage WebGL, a popular standard for accessing GPU capabilities
• Apply novel optimizations to reduce GPU processing cycles and data transfer
between CPU and GPU
![Page 10: A Javascript library to run ONNX models in browsers and Node•A pure JavaScript implementation of ONNX framework •Optimize ONNX model inference on both CPUs and GPUs •Support](https://reader033.vdocuments.us/reader033/viewer/2022060906/60a140ea954e6c027f1ea2c0/html5/thumbnails/10.jpg)
Website
Electron APP
ONNX Model
Frameworks
Train models with various
frameworks or services
Convert into ONNX with
ONNX Converters
Model inference with
ONNX.JS
Model inference with ONNX.JS
![Page 11: A Javascript library to run ONNX models in browsers and Node•A pure JavaScript implementation of ONNX framework •Optimize ONNX model inference on both CPUs and GPUs •Support](https://reader033.vdocuments.us/reader033/viewer/2022060906/60a140ea954e6c027f1ea2c0/html5/thumbnails/11.jpg)
HTML exampleto use ONNX.js
![Page 12: A Javascript library to run ONNX models in browsers and Node•A pure JavaScript implementation of ONNX framework •Optimize ONNX model inference on both CPUs and GPUs •Support](https://reader033.vdocuments.us/reader033/viewer/2022060906/60a140ea954e6c027f1ea2c0/html5/thumbnails/12.jpg)
Using NPM and bundling tools to use ONNX.js
![Page 13: A Javascript library to run ONNX models in browsers and Node•A pure JavaScript implementation of ONNX framework •Optimize ONNX model inference on both CPUs and GPUs •Support](https://reader033.vdocuments.us/reader033/viewer/2022060906/60a140ea954e6c027f1ea2c0/html5/thumbnails/13.jpg)
ONNX.js Demo
• Models supported
• Resnet-50 - a deep convolutional networks for
image classification
• Squeezenet - a light-weight convolutional
networks for image classification
• Emotion_ferplus - a deep convolutional neural
network for emotion recognition in faces
• Mnist - a convolutional neural network to
predict handwritten digits
• Tiny_yolov2 - a real-time neural network for
object detection
ONNX.js demo website
![Page 14: A Javascript library to run ONNX models in browsers and Node•A pure JavaScript implementation of ONNX framework •Optimize ONNX model inference on both CPUs and GPUs •Support](https://reader033.vdocuments.us/reader033/viewer/2022060906/60a140ea954e6c027f1ea2c0/html5/thumbnails/14.jpg)
We’d love to embrace your contribution
Operator Coverage Improvement
Performance improvement
More demos with more models
![Page 15: A Javascript library to run ONNX models in browsers and Node•A pure JavaScript implementation of ONNX framework •Optimize ONNX model inference on both CPUs and GPUs •Support](https://reader033.vdocuments.us/reader033/viewer/2022060906/60a140ea954e6c027f1ea2c0/html5/thumbnails/15.jpg)
Thanks