![Page 1: with Open Data Hub Blueprint for AI on Red Hat OpenShift · 2020. 1. 10. · Blueprint for building an AI-as-a-service platform on OpenShift Meta-Operator Integrates the best open](https://reader035.vdocuments.us/reader035/viewer/2022071412/61088b9444ee317ebb611297/html5/thumbnails/1.jpg)
Blueprint for AI on Red Hat OpenShift with Open Data HubSherard Griffin, Senior Manager AI Center of Excellence
![Page 2: with Open Data Hub Blueprint for AI on Red Hat OpenShift · 2020. 1. 10. · Blueprint for building an AI-as-a-service platform on OpenShift Meta-Operator Integrates the best open](https://reader035.vdocuments.us/reader035/viewer/2022071412/61088b9444ee317ebb611297/html5/thumbnails/2.jpg)
RED HAT DALLAS EMERGING TECH SUMMIT - DEC 5, 2019
Introductions
2
Sherard GriffinSenior Manager, AI Center of ExcellenceRaleigh, NC
Responsible for OpenDataHub.io and Red Hat’s AI partner enablement
![Page 3: with Open Data Hub Blueprint for AI on Red Hat OpenShift · 2020. 1. 10. · Blueprint for building an AI-as-a-service platform on OpenShift Meta-Operator Integrates the best open](https://reader035.vdocuments.us/reader035/viewer/2022071412/61088b9444ee317ebb611297/html5/thumbnails/3.jpg)
RED HAT DALLAS EMERGING TECH SUMMIT - DEC 5, 2019
What do Data Scientists Want?
3
“Self-service cloud like” experience for my machine learning projects
Access to a rich set of modelling frameworks, data, and computational resources
Collaborate with colleagues
Deliver my work into production with speed, agility and repeatability to drive business value
Self service portal to select ML frameworks, data access
Perform ML Modelling
Inferencing w/ hardware acceleration
ML model deployment in app dev
![Page 4: with Open Data Hub Blueprint for AI on Red Hat OpenShift · 2020. 1. 10. · Blueprint for building an AI-as-a-service platform on OpenShift Meta-Operator Integrates the best open](https://reader035.vdocuments.us/reader035/viewer/2022071412/61088b9444ee317ebb611297/html5/thumbnails/4.jpg)
RED HAT DALLAS EMERGING TECH SUMMIT - DEC 5, 2019
Today’s Challenges with AI Infrastructure
4
Limited non-shared resourcesModeling takes a long time or can’t achieve desired accuracy without pooled resources
Difficult to share and collaborateManual, error-prone methods of sharing data science work with others
Team(s) of data scientists and developersLimited multi-user environments
Challenging production deploymentsComplicated methods for deploying models into production pipeline
DATA SCIENTIST DATA SCIENTIST DATA SCIENTIST
![Page 5: with Open Data Hub Blueprint for AI on Red Hat OpenShift · 2020. 1. 10. · Blueprint for building an AI-as-a-service platform on OpenShift Meta-Operator Integrates the best open](https://reader035.vdocuments.us/reader035/viewer/2022071412/61088b9444ee317ebb611297/html5/thumbnails/5.jpg)
RED HAT DALLAS EMERGING TECH SUMMIT - DEC 5, 2019
How Does OpenShift Help with AI/ML workloads?
5
EXISTING AUTOMATION
TOOLSETS
SCM(GIT)
CI/CD
SERVICE LAYER
PERSISTENTSTORAGE
REGISTRY
RHEL
NODE
c
RHEL
NODE
RHEL
NODE
RHEL
NODE
RHEL
NODE
RHEL
NODE
C
C
C C
C
C
C CC C
RED HATENTERPRISE LINUX
MASTER
API/AUTHENTICATION
DATA STORE
SCHEDULER
HEALTH/SCALING
PHYSICAL VIRTUAL PRIVATE PUBLIC HYBRID
DATA SCIENTIST
ML deployed across clouds, datacenter,
and edge
ML services, load-balanced,
and scaled
ML microservices scheduled and
orchestrated on shared resources
Software Development
Lifecycle
ML in Production
![Page 6: with Open Data Hub Blueprint for AI on Red Hat OpenShift · 2020. 1. 10. · Blueprint for building an AI-as-a-service platform on OpenShift Meta-Operator Integrates the best open](https://reader035.vdocuments.us/reader035/viewer/2022071412/61088b9444ee317ebb611297/html5/thumbnails/6.jpg)
CONFIDENTIAL designator
6
Connected Drive & Autonomous Driving
Back-end AI Platform
Data-driven diagnosis Data-driven diagnosis
Democratize data science for oil and gas exploration
Containerized Apache Spark
Healthcare and public sector Automotive Financial Oil and gas
Discover Financial Services
Jupyter notebooks as a serviceMinistry of Defence (Israel)
RBC BankPublic cloud alternative for research
AI/ML on OpenShift Momentum is Strong
![Page 7: with Open Data Hub Blueprint for AI on Red Hat OpenShift · 2020. 1. 10. · Blueprint for building an AI-as-a-service platform on OpenShift Meta-Operator Integrates the best open](https://reader035.vdocuments.us/reader035/viewer/2022071412/61088b9444ee317ebb611297/html5/thumbnails/7.jpg)
CONFIDENTIAL designator
Why Red Hat for your AI/ML lifecycle
7
A leader and trusted
provider in the open
source space.
Expertise helping top
brands operationalize
containers & Kubernetes
powered hybrid cloud for
AI/ML.
Proven track record Open source advocate
Established strategic
partnerships with key
AI/ML ISVs.
Powerful partnerships
Helps complete the AI/ML
architecture.
Comprehensive portfolio
![Page 8: with Open Data Hub Blueprint for AI on Red Hat OpenShift · 2020. 1. 10. · Blueprint for building an AI-as-a-service platform on OpenShift Meta-Operator Integrates the best open](https://reader035.vdocuments.us/reader035/viewer/2022071412/61088b9444ee317ebb611297/html5/thumbnails/8.jpg)
CONFIDENTIAL designator
Red Hat portfolio helps complete the ML architecture
8
Software-definedInfrastructure
Data Services - Databases (SQL, NoSQL, etc.), Data Lake, etc.
Compute Acceleration (GPU, FPGA, TPU)
Hybrid, Multi Cloud Platform with self service capabilities
Physical Virtual Private Public Hybrid
ML Software tools
Set goals
Gather and prepare data
Develop ML model
Deploy ML models in app dev process
Implement Apps and inference
ML models monitoring & management
![Page 9: with Open Data Hub Blueprint for AI on Red Hat OpenShift · 2020. 1. 10. · Blueprint for building an AI-as-a-service platform on OpenShift Meta-Operator Integrates the best open](https://reader035.vdocuments.us/reader035/viewer/2022071412/61088b9444ee317ebb611297/html5/thumbnails/9.jpg)
CONFIDENTIAL designator
9
Red Hat is the design partner for AI/ML solutions across industries
Connected Drive & Autonomous Driving
Data driven diagnosis Data driven diagnosis
Democratize data science for oil and gas exploration
Containerized Apache Spark
Healthcare and public sector Automotive Financial Oil and gas
Discover Financial Services
Jupyter notebooks as a serviceMinistry of Defence (Israel)
RBC Bank
![Page 10: with Open Data Hub Blueprint for AI on Red Hat OpenShift · 2020. 1. 10. · Blueprint for building an AI-as-a-service platform on OpenShift Meta-Operator Integrates the best open](https://reader035.vdocuments.us/reader035/viewer/2022071412/61088b9444ee317ebb611297/html5/thumbnails/10.jpg)
RED HAT DALLAS EMERGING TECH SUMMIT - DEC 5, 2019
What is Open Data Hub?
10
Reference Architecture Community ProjectBlueprint for building an AI-as-a-service platform on OpenShift
Meta-OperatorIntegrates the best open source AI/ML and data engineering community projects as an operator
Production-readyRunning in Red Hat’s datacenter and used by teams for over a year
OpenDataHub.io
![Page 11: with Open Data Hub Blueprint for AI on Red Hat OpenShift · 2020. 1. 10. · Blueprint for building an AI-as-a-service platform on OpenShift Meta-Operator Integrates the best open](https://reader035.vdocuments.us/reader035/viewer/2022071412/61088b9444ee317ebb611297/html5/thumbnails/11.jpg)
RED HAT DALLAS EMERGING TECH SUMMIT - DEC 5, 2019
End-to-End AI/ML Process on OpenShift
11
Data Acquisition & Preparation
ML Model Selection, Training, Testing
ML Model Deployment in App. Dev. Process
![Page 12: with Open Data Hub Blueprint for AI on Red Hat OpenShift · 2020. 1. 10. · Blueprint for building an AI-as-a-service platform on OpenShift Meta-Operator Integrates the best open](https://reader035.vdocuments.us/reader035/viewer/2022071412/61088b9444ee317ebb611297/html5/thumbnails/12.jpg)
RED HAT DALLAS EMERGING TECH SUMMIT - DEC 5, 2019
Interactive Notebooks
Data Explorationand Engineering
Open Data Hub v0.5 Operators
12
Object Storage
Data Ingestion
Workflows
Model Lifecycle
Metadata ManagementHive Metastore
Service Monitoring
![Page 13: with Open Data Hub Blueprint for AI on Red Hat OpenShift · 2020. 1. 10. · Blueprint for building an AI-as-a-service platform on OpenShift Meta-Operator Integrates the best open](https://reader035.vdocuments.us/reader035/viewer/2022071412/61088b9444ee317ebb611297/html5/thumbnails/13.jpg)
RED HAT DALLAS EMERGING TECH SUMMIT - DEC 5, 2019
Upstream Community Projects in the AI/ML Space
13
13
ML toolkit and lifecycle Kube
AI-as-a-service platform based on OpenShift, Ceph, Kafka, JupyterHub,
Spark, and more
Home for K8’s community to share operators for various apps/tools
NVIDIA NGC GPU optimized
and curated
Tensorflow
Others
![Page 14: with Open Data Hub Blueprint for AI on Red Hat OpenShift · 2020. 1. 10. · Blueprint for building an AI-as-a-service platform on OpenShift Meta-Operator Integrates the best open](https://reader035.vdocuments.us/reader035/viewer/2022071412/61088b9444ee317ebb611297/html5/thumbnails/14.jpg)
RED HAT DALLAS EMERGING TECH SUMMIT - DEC 5, 2019
Data Scientists and Engineers Can Share Resources on OpenShift
14
JupyterHub, Spark, Hue, Ceph, etc.DATA SCIENTIST
DEVELOPER
DATA ENGINEER
DATA SCIENTIST DATA SCIENTIST DATA SCIENTIST
![Page 15: with Open Data Hub Blueprint for AI on Red Hat OpenShift · 2020. 1. 10. · Blueprint for building an AI-as-a-service platform on OpenShift Meta-Operator Integrates the best open](https://reader035.vdocuments.us/reader035/viewer/2022071412/61088b9444ee317ebb611297/html5/thumbnails/15.jpg)
RED HAT DALLAS EMERGING TECH SUMMIT - DEC 5, 2019
Notebooks as a Service in Open Data Hub
15
Red HatOpenShift Container Platform
Project Foo Project Bar
DATA SCIENTIST DATA SCIENTIST DATA SCIENTIST DATA SCIENTIST DATA SCIENTIST DATA SCIENTIST
Resource isolation(cpu, mem, gpu, nvme)
Shared resource pools
Data and networkisolation
Role based accesscontrol, quotas
Self servicenotebooks
![Page 16: with Open Data Hub Blueprint for AI on Red Hat OpenShift · 2020. 1. 10. · Blueprint for building an AI-as-a-service platform on OpenShift Meta-Operator Integrates the best open](https://reader035.vdocuments.us/reader035/viewer/2022071412/61088b9444ee317ebb611297/html5/thumbnails/16.jpg)
RED HAT DALLAS EMERGING TECH SUMMIT - DEC 5, 2019
CI/CD to Production for Data Scientists
16
References
Source 2 Image for Jupyter Notebooks: https://blog.openshift.com/jupyter-on-openshift-part-3-creating-a-s2i-builder-image/
OCI Container
DATA SCIENTIST
Source-2-imageCheck-in to source repo
Deloy notebook container
Model test & iteration
Promote and Serve models into production as services
![Page 17: with Open Data Hub Blueprint for AI on Red Hat OpenShift · 2020. 1. 10. · Blueprint for building an AI-as-a-service platform on OpenShift Meta-Operator Integrates the best open](https://reader035.vdocuments.us/reader035/viewer/2022071412/61088b9444ee317ebb611297/html5/thumbnails/17.jpg)
RED HAT DALLAS EMERGING TECH SUMMIT - DEC 5, 2019
NVIDIA GPU Enablement for Containers
17
Modeling libs and frameworks
CUDA Toolkit
Universal Base Image
CRIO (+ GPU runtime plugin)
RHEL or RHEL CoreOS
CUDA driver for GPUs
NVIDIA NGC Containers
Enabled and supported in OCP 3.11 and OpenShift 4.1 (with BYO RHEL7 Workers)
![Page 18: with Open Data Hub Blueprint for AI on Red Hat OpenShift · 2020. 1. 10. · Blueprint for building an AI-as-a-service platform on OpenShift Meta-Operator Integrates the best open](https://reader035.vdocuments.us/reader035/viewer/2022071412/61088b9444ee317ebb611297/html5/thumbnails/18.jpg)
RED HAT DALLAS EMERGING TECH SUMMIT - DEC 5, 2019
Sharing GPUs in OpenShift for Model Training
18
Taint: GPU workloads only
K8s Job with Toleration for GPU workload only
Priority: P1
K8s Job with Toleration for GPU workload only
Priority: P99
Prio
rity
1
Prio
rity
99
GPU Admission Controller adds
the K8s Job annotations
Use Affinity and Anti-Affinity for softer rules (vs.
Taints/Tolerations)
Multitenancy with OpenShift Projects:
Each Project can have quotas for GPU usage
Priority Class can have quotas
Supported in OpenShift 3.11 (except Priority Class Quotas) and OpenShift 4.1 (RHEL7 only on GPU nodes)
![Page 19: with Open Data Hub Blueprint for AI on Red Hat OpenShift · 2020. 1. 10. · Blueprint for building an AI-as-a-service platform on OpenShift Meta-Operator Integrates the best open](https://reader035.vdocuments.us/reader035/viewer/2022071412/61088b9444ee317ebb611297/html5/thumbnails/19.jpg)
RED HAT DALLAS EMERGING TECH SUMMIT - DEC 5, 2019
Using GPUs in Open Data Hub
19
Select number of GPUs to use via UI
![Page 20: with Open Data Hub Blueprint for AI on Red Hat OpenShift · 2020. 1. 10. · Blueprint for building an AI-as-a-service platform on OpenShift Meta-Operator Integrates the best open](https://reader035.vdocuments.us/reader035/viewer/2022071412/61088b9444ee317ebb611297/html5/thumbnails/20.jpg)
20
Demo
![Page 21: with Open Data Hub Blueprint for AI on Red Hat OpenShift · 2020. 1. 10. · Blueprint for building an AI-as-a-service platform on OpenShift Meta-Operator Integrates the best open](https://reader035.vdocuments.us/reader035/viewer/2022071412/61088b9444ee317ebb611297/html5/thumbnails/21.jpg)
RED HAT DALLAS EMERGING TECH SUMMIT - DEC 5, 2019
21
Mission: With partners, make best-in-class AI/ML hybrid/multi-cloud ready for mainstream and enterprise markets
● Primary Focus on OpenShift - kubernetes-powered multi/hybrid-cloud AI/ML platform● Hardware Ecosystem for Performance - GPUs ● SW Ecosystem Enablement
○ Projects w/ Vendor Support - Certified vendors w/ commercial support○ Balance Platforms & Components - Provide choice
● Cloud-Like Experience - Operators, operatorhub.io, embedded operatorhub● Certifications with Red Hat Partner Connect (for Container and Operator certifications)
Via:● AI Reference Architecture - Open Data Hub - RH Ref Arch for AI for upstream/products/partners● AI Use Case Definitions - define core use cases - RH + Partners Enable these use cases
AI Partner Program
![Page 22: with Open Data Hub Blueprint for AI on Red Hat OpenShift · 2020. 1. 10. · Blueprint for building an AI-as-a-service platform on OpenShift Meta-Operator Integrates the best open](https://reader035.vdocuments.us/reader035/viewer/2022071412/61088b9444ee317ebb611297/html5/thumbnails/22.jpg)
RED HAT DALLAS EMERGING TECH SUMMIT - DEC 5, 2019
22
Who Are We Working With?
Just to name a few...
![Page 23: with Open Data Hub Blueprint for AI on Red Hat OpenShift · 2020. 1. 10. · Blueprint for building an AI-as-a-service platform on OpenShift Meta-Operator Integrates the best open](https://reader035.vdocuments.us/reader035/viewer/2022071412/61088b9444ee317ebb611297/html5/thumbnails/23.jpg)
RED HAT DALLAS EMERGING TECH SUMMIT - DEC 5, 2019
Find Out More!
23
Deploy in Minutes
https://opendatahub.io/docs.html
Join the Community
https://opendatahub.io/community.html
![Page 24: with Open Data Hub Blueprint for AI on Red Hat OpenShift · 2020. 1. 10. · Blueprint for building an AI-as-a-service platform on OpenShift Meta-Operator Integrates the best open](https://reader035.vdocuments.us/reader035/viewer/2022071412/61088b9444ee317ebb611297/html5/thumbnails/24.jpg)
CONFIDENTIAL Designator
linkedin.com/company/red-hat
youtube.com/user/RedHatVideos
facebook.com/redhatinc
twitter.com/RedHat
Red Hat is the world’s leading provider of
enterprise open source software solutions.
Award-winning support, training, and consulting
services make
Red Hat a trusted adviser to the Fortune 500.
Thank you
24
![Page 25: with Open Data Hub Blueprint for AI on Red Hat OpenShift · 2020. 1. 10. · Blueprint for building an AI-as-a-service platform on OpenShift Meta-Operator Integrates the best open](https://reader035.vdocuments.us/reader035/viewer/2022071412/61088b9444ee317ebb611297/html5/thumbnails/25.jpg)
25
RED HAT DALLAS EMERGIN
G TECH SUM
MIT - DEC 5, 2019
STAY ENGAGEDDevelopers. redhat.comYour access point for no-cost developer tools and product subscriptions, how-tos, and demos
Red Hat User GroupsMeetups for networking and tech deep diveswww.meetup.com/Dallas-Red-Hat-Users-Group/
DevNationVirtual and live eventsCatch replays at https://developers.redhat.com/devnation/
Next.redhat.comStay in touch with the Office of the CTO