accelerating ai and deep learning workflows with matlab and … · demo –deep learning with...
TRANSCRIPT
![Page 1: Accelerating AI and Deep Learning Workflows with MATLAB and … · DEMO –DEEP LEARNING WITH MATLAB CONTAINER ON AWS Interactive tools for designing complex networks Design AI Network](https://reader035.vdocuments.us/reader035/viewer/2022071513/61339e35dfd10f4dd73b3463/html5/thumbnails/1.jpg)
1
Accelerating AI Workflows with MATLAB and NVIDIA NGC
Akhil Docca - Senior Product Marketing Manager at NVIDIA
Andy Thé – Partner Manager at MathWorks
![Page 2: Accelerating AI and Deep Learning Workflows with MATLAB and … · DEMO –DEEP LEARNING WITH MATLAB CONTAINER ON AWS Interactive tools for designing complex networks Design AI Network](https://reader035.vdocuments.us/reader035/viewer/2022071513/61339e35dfd10f4dd73b3463/html5/thumbnails/2.jpg)
2
AGENDA
▪ NVIDIA and MathWorks Partnership
▪ What is the NVIDIA NGC catalog?
▪ How NVIDIA NGC helps with accelerating AI workflows?
▪ Demo: Deep Learning with MATLAB using NGC on AWS
▪ Conclusion and Call-to-Action
▪ Q&A
![Page 3: Accelerating AI and Deep Learning Workflows with MATLAB and … · DEMO –DEEP LEARNING WITH MATLAB CONTAINER ON AWS Interactive tools for designing complex networks Design AI Network](https://reader035.vdocuments.us/reader035/viewer/2022071513/61339e35dfd10f4dd73b3463/html5/thumbnails/3.jpg)
3
OPTIMIZED CUDA GENERATION
FROM MATLAB CODENVIDIA GPU SUPPORT IN 800+
FUNCTIONS:
• MATLAB
• Deep Learning Toolbox
• Image Processing Toolbox
• Signal Processing Toolbox
No need for CUDA expertise!
NVIDIA AND MATHWORKS COLLABORATIONINTEGRATE NVIDIA GPUS WITH MATLAB FROM EMBEDDED TO ENTERPRISE
EMBEDDED SYSTEMSGENERAL-PURPOSE
TECHNICAL COMPUTING
CUDA
GPU Coder
cuDNN, cuSolver,
cuBLAS TensorRT
MATLAB
MATLAB and NVIDIA GPU CLOUD
Instantly access on-premises and
cloud GPUs with MATLAB and NGC
ON-PREM AND CLOUD
![Page 4: Accelerating AI and Deep Learning Workflows with MATLAB and … · DEMO –DEEP LEARNING WITH MATLAB CONTAINER ON AWS Interactive tools for designing complex networks Design AI Network](https://reader035.vdocuments.us/reader035/viewer/2022071513/61339e35dfd10f4dd73b3463/html5/thumbnails/4.jpg)
4
AI HAS REACHED THE TIPPING POINT
Autonomous Mobility
Over 50 Percent of Organizations Are Using AI in Production
AI Adoption In The Enterprise 2020 O’Reilly Survey
Recommendation System
Logistics
Conversational AI Fraud Detection
Medical Imaging
![Page 5: Accelerating AI and Deep Learning Workflows with MATLAB and … · DEMO –DEEP LEARNING WITH MATLAB CONTAINER ON AWS Interactive tools for designing complex networks Design AI Network](https://reader035.vdocuments.us/reader035/viewer/2022071513/61339e35dfd10f4dd73b3463/html5/thumbnails/5.jpg)
5
POLL # 1
Have you ever heard about NVIDIA NGC Catalog?
a. Yes
b. No
![Page 6: Accelerating AI and Deep Learning Workflows with MATLAB and … · DEMO –DEEP LEARNING WITH MATLAB CONTAINER ON AWS Interactive tools for designing complex networks Design AI Network](https://reader035.vdocuments.us/reader035/viewer/2022071513/61339e35dfd10f4dd73b3463/html5/thumbnails/6.jpg)
6
AI WORKFLOWS ARE COMPLEX
Extract
Data ScientistData Engineer Developer DevOps
Setup Build Model Train Optimize Integrate Optimize Build App Deploy & MonitorTransform Load Feature Engineering
Timeline
![Page 7: Accelerating AI and Deep Learning Workflows with MATLAB and … · DEMO –DEEP LEARNING WITH MATLAB CONTAINER ON AWS Interactive tools for designing complex networks Design AI Network](https://reader035.vdocuments.us/reader035/viewer/2022071513/61339e35dfd10f4dd73b3463/html5/thumbnails/7.jpg)
7
AI WORKFLOWS ARE COMPLEX
Extract
Data ScientistData Engineer Developer DevOps
Setup Build Model Train Optimize Integrate Optimize Build App Deploy & MonitorTransform Load Feature Engineering
Timeline
Extract Setup Build Model Train Optimize Integrate Optimize Build App Deploy & MonitorTransform Load Feature Engineering
Containers Pre-Trained Models SDKs Helm Charts
SIMPLIFYING AI WORKFLOWS WITH THE NGC CATALOG
Accelerate ETL Simplify Training Build Faster Deploy Reliably
![Page 8: Accelerating AI and Deep Learning Workflows with MATLAB and … · DEMO –DEEP LEARNING WITH MATLAB CONTAINER ON AWS Interactive tools for designing complex networks Design AI Network](https://reader035.vdocuments.us/reader035/viewer/2022071513/61339e35dfd10f4dd73b3463/html5/thumbnails/8.jpg)
8
Poll # 2
Do you or your colleagues use containers today?
a. Yes
b. No
![Page 9: Accelerating AI and Deep Learning Workflows with MATLAB and … · DEMO –DEEP LEARNING WITH MATLAB CONTAINER ON AWS Interactive tools for designing complex networks Design AI Network](https://reader035.vdocuments.us/reader035/viewer/2022071513/61339e35dfd10f4dd73b3463/html5/thumbnails/9.jpg)
9
WHY CONTAINERS?
• Agnostic to the underlying OS
• Packages the application with its all
dependencies
• Users can run multiple versions of the same
application
• Easy to deploy, track and maintain
• Portability allows researchers to collaborate
and federate scientific work
![Page 10: Accelerating AI and Deep Learning Workflows with MATLAB and … · DEMO –DEEP LEARNING WITH MATLAB CONTAINER ON AWS Interactive tools for designing complex networks Design AI Network](https://reader035.vdocuments.us/reader035/viewer/2022071513/61339e35dfd10f4dd73b3463/html5/thumbnails/10.jpg)
1010
NGC CONTAINERS ENABLE YOU TO FOCUS ON BUILDING AI
PERFORMANCE OPTIMIZED DEPLOY ANYWHEREENTERPRISE READY SOFTWARE
Scanned for CVEs, malware, crypto
Tested for reliability
Backed by Enterprise support
Scalable
Updated monthly
Better performance on the same system
Docker | cri-o | containerd | Singularity
Bare metal, VMs, Kubernetes
Multi-cloud, on-prem, hybrid, edge
![Page 11: Accelerating AI and Deep Learning Workflows with MATLAB and … · DEMO –DEEP LEARNING WITH MATLAB CONTAINER ON AWS Interactive tools for designing complex networks Design AI Network](https://reader035.vdocuments.us/reader035/viewer/2022071513/61339e35dfd10f4dd73b3463/html5/thumbnails/11.jpg)
1111
DO WHAT YOU DO BEST, FASTER
PERFORMANCE OPTIMIZED DEPLOY ANYWHERE
Scalable
Updated Monthly
Better performance on the same system
Docker | cri-o | containerd | Singularity
Bare metal, VMs, Kubernetes
Multi-cloud, on-prem, hybrid, edge
ENTERPRISE READY SOFTWARE
Scanned for CVEs, malware, crypto
Tested for reliability
Backed by Enterprise support
0
0.2
0.4
0.6
0.8
1
1.2
1.4
1.6
1.8
1 GPU 2 GPUs 4 GPUs
Rela
tive S
peedup
Number of GPUs
R2020a R2020b
1.03X
1.26X
1.6X
Windows 10, Intel® Xeon® E5-2623 v4 @ 2.60 GHzNVIDIA® Titan V 12GB GPUs
![Page 12: Accelerating AI and Deep Learning Workflows with MATLAB and … · DEMO –DEEP LEARNING WITH MATLAB CONTAINER ON AWS Interactive tools for designing complex networks Design AI Network](https://reader035.vdocuments.us/reader035/viewer/2022071513/61339e35dfd10f4dd73b3463/html5/thumbnails/12.jpg)
1212
FIND THE RIGHT FIT WITH CREDENTIALS
Faster training
Higher accuracy
Transparency through credentials
PRE-TRAINED MODELS RESOURCES
Get started with code samples
Customize NGC models
Reproduce with recipes
ResNet-50, SSD, MobileNet, VGG16
WaveGlow, BERT, NeMo
Wide & Deep, DLRM & many more
WIDE RANGE OF USE CASES
![Page 13: Accelerating AI and Deep Learning Workflows with MATLAB and … · DEMO –DEEP LEARNING WITH MATLAB CONTAINER ON AWS Interactive tools for designing complex networks Design AI Network](https://reader035.vdocuments.us/reader035/viewer/2022071513/61339e35dfd10f4dd73b3463/html5/thumbnails/13.jpg)
13
POLL # 3
How long have you been working with AI?
a. < 1 year
b. < 2 years
c. 2+ years
d. 5+ years
![Page 14: Accelerating AI and Deep Learning Workflows with MATLAB and … · DEMO –DEEP LEARNING WITH MATLAB CONTAINER ON AWS Interactive tools for designing complex networks Design AI Network](https://reader035.vdocuments.us/reader035/viewer/2022071513/61339e35dfd10f4dd73b3463/html5/thumbnails/14.jpg)
1414
INDUSTRY APP FRAMEWORKS FOR END-TO-END AI WORKFLOWS
Domain adaptability
Significantly reduce development time
TRANSFER LEARNING TOOLKIT TENSORRT
Optimizes for low latency and high-throughput
Integrated with major frameworks
High performance inference
Supports multiple frameworks
TRITON
![Page 15: Accelerating AI and Deep Learning Workflows with MATLAB and … · DEMO –DEEP LEARNING WITH MATLAB CONTAINER ON AWS Interactive tools for designing complex networks Design AI Network](https://reader035.vdocuments.us/reader035/viewer/2022071513/61339e35dfd10f4dd73b3463/html5/thumbnails/15.jpg)
15
Poll # 4
What AI application focus:
a. NLP
b. Computer vision
c. Speech
d. Recommender systems
e. Other
![Page 16: Accelerating AI and Deep Learning Workflows with MATLAB and … · DEMO –DEEP LEARNING WITH MATLAB CONTAINER ON AWS Interactive tools for designing complex networks Design AI Network](https://reader035.vdocuments.us/reader035/viewer/2022071513/61339e35dfd10f4dd73b3463/html5/thumbnails/16.jpg)
16
NGC PRIVATE REGISTRYSecure, Cloud-hosted Platform to Collaborate, Build, Deploy Faster
Flexible role-based access control Automatic container scanning Share containers, models and helm
charts with your teams
MANAGE ACCESS SECURE SOFTWARE COLLABORATE
![Page 17: Accelerating AI and Deep Learning Workflows with MATLAB and … · DEMO –DEEP LEARNING WITH MATLAB CONTAINER ON AWS Interactive tools for designing complex networks Design AI Network](https://reader035.vdocuments.us/reader035/viewer/2022071513/61339e35dfd10f4dd73b3463/html5/thumbnails/17.jpg)
17
DEPLOY ANYWHEREUnified Experience Across On-Prem and Cloud
OEM Systems Certified by NVIDIA Public Cloud Providers
Workload Portability
PERFORMANCE OPTIMIZED | TESTED FOR RELIABILITY | SCALABLE | SECURE | PORTABLE
![Page 18: Accelerating AI and Deep Learning Workflows with MATLAB and … · DEMO –DEEP LEARNING WITH MATLAB CONTAINER ON AWS Interactive tools for designing complex networks Design AI Network](https://reader035.vdocuments.us/reader035/viewer/2022071513/61339e35dfd10f4dd73b3463/html5/thumbnails/18.jpg)
18
DEMO – DEEP LEARNING WITH MATLAB CONTAINER ON AWS
Interactive tools for
designing complex
networks
Design AI Network Deployment
Access and prepare
large volumes of
data
Data Preparation
Accelerating training
with on demand GPUs
AI Model Tuning
Deploy validated
models anywhere
![Page 19: Accelerating AI and Deep Learning Workflows with MATLAB and … · DEMO –DEEP LEARNING WITH MATLAB CONTAINER ON AWS Interactive tools for designing complex networks Design AI Network](https://reader035.vdocuments.us/reader035/viewer/2022071513/61339e35dfd10f4dd73b3463/html5/thumbnails/19.jpg)
19
STEPS TO RUN MATLAB NGC DOCKER CONTAINER ON AWS
1. Create EC2 instance on AWS
2. Use PuTTY to connect to the EC2 instance
3. Pull & run the MATLAB container from the NGC catalog
4. Login into your MathWorks.com account
5. Run MATLAB deep learning examples
Check out the blog for a detailed walkthrough:
– tinyurl.com/3urvtyp8 or
– https://blogs.mathworks.com/deep-learning/2021/05/03/ai-with-matlab-ngc
![Page 20: Accelerating AI and Deep Learning Workflows with MATLAB and … · DEMO –DEEP LEARNING WITH MATLAB CONTAINER ON AWS Interactive tools for designing complex networks Design AI Network](https://reader035.vdocuments.us/reader035/viewer/2022071513/61339e35dfd10f4dd73b3463/html5/thumbnails/20.jpg)
20
Demo Video: MATLAB+NGC+AWS
Placeholder as video will
be played separately
![Page 21: Accelerating AI and Deep Learning Workflows with MATLAB and … · DEMO –DEEP LEARNING WITH MATLAB CONTAINER ON AWS Interactive tools for designing complex networks Design AI Network](https://reader035.vdocuments.us/reader035/viewer/2022071513/61339e35dfd10f4dd73b3463/html5/thumbnails/21.jpg)
21
ACCELERATING TRAINING WITH MULTIPLE GPUS ON IMAGENET
▪ 150 GB
▪ >1.2M images
▪ 50k test images
▪ 1,000 categories
▪ ~3.5x faster / Epoch
– 4 vs 1 GPU - V100
162
85
45
0 20 40 60 80 100 120 140 160 180
1 GPU
2 GPUs
4 GPUs
ResNet-50 training on ImageNet data-setTraining time per Epoch (V100)
Minutes
~3.5x Faster w/
4 GPUs
![Page 22: Accelerating AI and Deep Learning Workflows with MATLAB and … · DEMO –DEEP LEARNING WITH MATLAB CONTAINER ON AWS Interactive tools for designing complex networks Design AI Network](https://reader035.vdocuments.us/reader035/viewer/2022071513/61339e35dfd10f4dd73b3463/html5/thumbnails/22.jpg)
22
START BUILDING AI FASTER WITH THE NGC CATALOG
Get started today with MATLAB and NGC: https://ngc.nvidia.com/catalog/containers/partners:matlab
Check out the blog for a detailed walkthrough:
http://tinyurl.com/3urvtyp8
Learn more at nvidia.com/ngc