![Page 1: Leveraging Microsoft Azure's GPU N-Series for Compute ... · Karan Batta, Program Manager, Microsoft Azure Alexey Kamenev, Software Engineer, Microsoft Research S6839 . Agenda Azure](https://reader034.vdocuments.us/reader034/viewer/2022042711/5f87b9f204860674b5163ccb/html5/thumbnails/1.jpg)
Leveraging Microsoft Azure’s GPU N-Series for Compute and Visualization
Karan Batta, Program Manager, Microsoft Azure
Alexey Kamenev, Software Engineer, Microsoft Research
S6839
![Page 2: Leveraging Microsoft Azure's GPU N-Series for Compute ... · Karan Batta, Program Manager, Microsoft Azure Alexey Kamenev, Software Engineer, Microsoft Research S6839 . Agenda Azure](https://reader034.vdocuments.us/reader034/viewer/2022042711/5f87b9f204860674b5163ccb/html5/thumbnails/2.jpg)
Agenda
Azure
HPC in the Cloud
Technology/Architecture
CNTK Overview & Demo
![Page 3: Leveraging Microsoft Azure's GPU N-Series for Compute ... · Karan Batta, Program Manager, Microsoft Azure Alexey Kamenev, Software Engineer, Microsoft Research S6839 . Agenda Azure](https://reader034.vdocuments.us/reader034/viewer/2022042711/5f87b9f204860674b5163ccb/html5/thumbnails/3.jpg)
![Page 4: Leveraging Microsoft Azure's GPU N-Series for Compute ... · Karan Batta, Program Manager, Microsoft Azure Alexey Kamenev, Software Engineer, Microsoft Research S6839 . Agenda Azure](https://reader034.vdocuments.us/reader034/viewer/2022042711/5f87b9f204860674b5163ccb/html5/thumbnails/4.jpg)
Vision
GPU based capabilities in cloud infrastructure
High end performance
Not “Swiss-army knife” approach
Deliver and empower developer scenarios
Achieve true “HPC in the Cloud”
Critical workloads
![Page 5: Leveraging Microsoft Azure's GPU N-Series for Compute ... · Karan Batta, Program Manager, Microsoft Azure Alexey Kamenev, Software Engineer, Microsoft Research S6839 . Agenda Azure](https://reader034.vdocuments.us/reader034/viewer/2022042711/5f87b9f204860674b5163ccb/html5/thumbnails/5.jpg)
HPC in the Cloud
APP
exe
exe
exe
exe
![Page 6: Leveraging Microsoft Azure's GPU N-Series for Compute ... · Karan Batta, Program Manager, Microsoft Azure Alexey Kamenev, Software Engineer, Microsoft Research S6839 . Agenda Azure](https://reader034.vdocuments.us/reader034/viewer/2022042711/5f87b9f204860674b5163ccb/html5/thumbnails/6.jpg)
Workflow
Rendering Algorithm
Executable
Azure
GPU VMs
Upload data
Submit job
Split job/ setup execution pipeline
Manage
GPU Visualization
Analytics Dynamic Modelling Virtual Desktops
{REST
AP
I}
Return results
Outputs
![Page 7: Leveraging Microsoft Azure's GPU N-Series for Compute ... · Karan Batta, Program Manager, Microsoft Azure Alexey Kamenev, Software Engineer, Microsoft Research S6839 . Agenda Azure](https://reader034.vdocuments.us/reader034/viewer/2022042711/5f87b9f204860674b5163ccb/html5/thumbnails/7.jpg)
Where?
Finance
• FX Options
• Risk Management
• Hedge Fund Management
Manufacturing & Oil/Gas
• Automotive design
• Reservoir modelling
• Manipulation of models & parts
Media
• Streaming games/video
• Transcoding
• Social media analysis
Rendering
• VFX/Ray-Tracing rendering
• CAD applications
• Simulations
![Page 8: Leveraging Microsoft Azure's GPU N-Series for Compute ... · Karan Batta, Program Manager, Microsoft Azure Alexey Kamenev, Software Engineer, Microsoft Research S6839 . Agenda Azure](https://reader034.vdocuments.us/reader034/viewer/2022042711/5f87b9f204860674b5163ccb/html5/thumbnails/8.jpg)
Technology
DDA (Discrete Device Assignment)
Introduced in Windows Server 2016
Pass-through PCIe devices
Allows for close to bare-metal performance
![Page 9: Leveraging Microsoft Azure's GPU N-Series for Compute ... · Karan Batta, Program Manager, Microsoft Azure Alexey Kamenev, Software Engineer, Microsoft Research S6839 . Agenda Azure](https://reader034.vdocuments.us/reader034/viewer/2022042711/5f87b9f204860674b5163ccb/html5/thumbnails/9.jpg)
![Page 10: Leveraging Microsoft Azure's GPU N-Series for Compute ... · Karan Batta, Program Manager, Microsoft Azure Alexey Kamenev, Software Engineer, Microsoft Research S6839 . Agenda Azure](https://reader034.vdocuments.us/reader034/viewer/2022042711/5f87b9f204860674b5163ccb/html5/thumbnails/10.jpg)
Architecture
Applications
GPU Provisioning
Host OS
Client OS
Hardware
• Custom Applications
• Data and Applications from the Azure Marketplace
• Bring your own Image
• Azure VM Marketplace Images
• Hyper-V
• DDA
• NVIDIA M60 GPU (Viz SKU)
• NVIDIA K80 GPU (Compute SKU)
![Page 11: Leveraging Microsoft Azure's GPU N-Series for Compute ... · Karan Batta, Program Manager, Microsoft Azure Alexey Kamenev, Software Engineer, Microsoft Research S6839 . Agenda Azure](https://reader034.vdocuments.us/reader034/viewer/2022042711/5f87b9f204860674b5163ccb/html5/thumbnails/11.jpg)
Visualization VMs
NV6 NV12 NV24
Cores 6
(E5-2690v3) 12
(E5-2690v3) 24
(E5-2690v3)
GPU 1 x M60 GPU (1/2
Physical Card) 2 x M60 GPU (1 Physical Card)
4 x M60 GPU (2 Physical Cards)
Memory 56 GB 112 GB 224 GB
Disk ~380 GB SSD ~680 GB SSD ~1.5 TB SSD
Network Azure Network Azure Network Azure Network
![Page 12: Leveraging Microsoft Azure's GPU N-Series for Compute ... · Karan Batta, Program Manager, Microsoft Azure Alexey Kamenev, Software Engineer, Microsoft Research S6839 . Agenda Azure](https://reader034.vdocuments.us/reader034/viewer/2022042711/5f87b9f204860674b5163ccb/html5/thumbnails/12.jpg)
Compute VMs
NC6 NC12 NC24 NC24r
Cores 6
(E5-2690v3) 12
(E5-2690v3) 24
(E5-2690v3) 24
(E5-2690v3)
GPU 1 x K80 GPU (1/2
Physical Card) 2 x K80 GPU (1 Physical Card)
4 x K80 GPU (2 Physical Cards)
4 x K80 GPU (2 Physical Cards)
Memory 56 GB 112 GB 224 GB 224 GB
Disk ~380 GB SSD ~680 GB SSD ~1.5 TB SSD ~1.5 TB SSD
Network Azure Network Azure Network Azure Network Azure Network +
RDMA (RoCE)
![Page 13: Leveraging Microsoft Azure's GPU N-Series for Compute ... · Karan Batta, Program Manager, Microsoft Azure Alexey Kamenev, Software Engineer, Microsoft Research S6839 . Agenda Azure](https://reader034.vdocuments.us/reader034/viewer/2022042711/5f87b9f204860674b5163ccb/html5/thumbnails/13.jpg)
CNTK Alexey Kamenev
Senior Software Engineer
Microsoft Research
![Page 14: Leveraging Microsoft Azure's GPU N-Series for Compute ... · Karan Batta, Program Manager, Microsoft Azure Alexey Kamenev, Software Engineer, Microsoft Research S6839 . Agenda Azure](https://reader034.vdocuments.us/reader034/viewer/2022042711/5f87b9f204860674b5163ccb/html5/thumbnails/14.jpg)
CNTK Overview
• A deep learning tool that balances • Efficiency: Can train production systems as fast as possible
• Performance: Can achieve state-of-the-art performance on benchmark tasks and production systems
• Flexibility: Can support various tasks such as speech, image, and text, and can try out new ideas quickly
• Inspiration: Legos • Each brick is very simple and performs a specific function
• Create arbitrary objects by combining many bricks
• CNTK enables the creation of existing and novel models by combining simple functions in arbitrary ways.
• Historical facts: • Created by Microsoft Speech researchers (Dong Yu et al.) 4 years ago
• Was quickly extended to handle other workloads (image/text)
• Open-sourced (CodePlex) in early 2015
• Moved to GitHub in Jan 2016
![Page 15: Leveraging Microsoft Azure's GPU N-Series for Compute ... · Karan Batta, Program Manager, Microsoft Azure Alexey Kamenev, Software Engineer, Microsoft Research S6839 . Agenda Azure](https://reader034.vdocuments.us/reader034/viewer/2022042711/5f87b9f204860674b5163ccb/html5/thumbnails/15.jpg)
Resources
• “Deep Learning in Microsoft with CNTK” – Alexey Kamenev, Microsoft – Hall 3 – 4.30pm
• CNTK (Deep-Learning toolkit) • https://github.com/Microsoft/CNTK
• DDA (Direct Device Assignment) • http://blogs.technet.com/b/virtualization/archive/2015/11/23/discrete-
device-assignment-gpus.aspx
• NVIDIA announcement • http://nvidianews.nvidia.com/news/nvidia-gpus-to-accelerate-microsoft-
azure
![Page 16: Leveraging Microsoft Azure's GPU N-Series for Compute ... · Karan Batta, Program Manager, Microsoft Azure Alexey Kamenev, Software Engineer, Microsoft Research S6839 . Agenda Azure](https://reader034.vdocuments.us/reader034/viewer/2022042711/5f87b9f204860674b5163ccb/html5/thumbnails/16.jpg)
Thank you!