supervessel: cognitive computing platform on the openstack...
TRANSCRIPT
Revolutionizing the Datacenter
Join the Conversation #OpenPOWERSummit
SuperVessel: Cognitive Computing Platformon the OpenStack based OpenPOWER Cloud
Yong Hua Lin
IBM Research China
Join the Conversation #OpenPOWERSummit
Cognitive Computing brings new challenges and opportunities
4/1/2016 2
Machine
Learning
Machine
Reasoning
Knowledge
Engineering
Multilingual NLP,
NLG & Speech
Machine
Vision
Human-Machine
Interaction
Data Mining
& Pattern Reco
Decision /
Optimization
Rule-based
Engines Data
Insights
Decision
MakingResearch & Discovery
Affective Dialog
Artifact Generation
Policy
Assurance
Radiologist Assistant
Personal Concierge
HR / Legal Assistant
Supply Chain
Assistant
Intelligence Analyst
Could we flexibly schedule parallel computation for very
computation intensive workload?
Could we satisfy the work for both training and deployment stage?
Easily program for CPU and GPU/FPGA ?
Data set management and model management?
Cognitive Computing on Cloud ?
How to do resource scheduling for
CPU/GPU/FPGA ?
Cognitive computing refers to systems that learn at scale, reason with purpose and interact with humans naturally.
• Cognitive Computing
• Big data service
• Cloud Data Service
• Accelerator service
• IoT application
development platform
• X86 to POWER porting
service
SuperVessel: The IBM OpenPOWER cloud for Researchers and Developers
3
SuperVessel Cloud
www.ptopenlab.comOpenPOWEROpenStack
SuperVessel Services
4
Computing service
Data store service
Network service
Big Data Service
Cloud Data Service
IoTDevelopmen
t Service
Super Marketplace (Accelerators, Images, Applications)
Infrastructure as Service
Platform layer service
SuperVessel provides multiple layers services.
Accelerator DevOps Service
Cognitive Computing Service
Accelerator service
OpenStack-based cloud infrastructure with FPGA and GPU
User account & authentication
manage
User dashboard
Admin dashboard
Virtual point management
Statistic and analysis
Platform
Management
System
FPGAIBM POWER servers
OpenPOWER server
Container pool for POWER7 LPAR
Distributed file system / shared file system
KVM pool for POWER8 LE/BE
KVM pool for x86Container pool for POWER8 LE/BE
Container pool for x86
Nova Neutron Cinder
LxC/ Docker
Horizon
OpenStack controller (HA)
Nova
NeutronGlance Cinder
HEAT Senlin
Ironic Swift
Keystone
Services layer
System maintenance
System monitoring
Resource usage metering
System analysis
Services for cloud administration
Baremetalmanagement
Image management
Nova Neutron Cinder
KVM
Nova Neutron Cinder
KVM
Nova Neutron Cinder
LxC/ Docker
Nova Neutron Cinder
LxC/ Docker
X86 server
GPU/FPGA scheduler
Auto Provision
GPU
Heterogeneous computing for cognitive cloud with GPU and FPGA
6
Train Data Set DNN Net File
Trained
model
Application Data from User
Training (development) Stage Recognition (deployment) Stage
Big data platform (Hadoop, Spark)
Deep Learning platform(caffe, Torch, Theano,
TensorFlow, etc.)
Model pool
Data Management
CPU + GPU cluster
Data Cleansing
Feature Engineering
Modeling
Deep Learning platformApplication servers, DB service, messaging, etc.
CPU + FPGA cluster
ApplicationRecognition, classification
To support both GPU and FPGA in the cloud infrastructure will be very important for Cognitive Cloud
7
SuperVessel 2.0 : Cognitive Computing System on OpenPOWER Cloud
Try it on : https://dashboard.ptopenlab.com/computing/
• Cognitive Infrastructure
Service
• Cognitive Computing service
• Cognitive Solution and Demo
service
8
GPU service and GPU accelerated Deep Learning
• SuperVessel provides the GPU sharing service by extending OpenStack and dockercapability. It is the first GPU sharing service in the public cloud.
• Users could apply the docker instance on SuperVessel
• Users could apply the deep learning development environment on SuperVessel, e.g. Caffe, Torch, Theano, and TensorFlow.
• All the DL environment will assign the GPU resource for acceleration automatically.
FPGA accelerator sharing service and FPGA maker zone
Xilinx Virtex 7 FPGA
Acceleration Hardware
Key technologies
Accelerator developers : Easily develop and deploy accelerator on cloud
• FPGA virtualization with POWER KVM
• CAPI virtualization with Docker
• OpenStack-based accelerator service
management
Application developers : Easily use accelerator for application
Accelerator Maker Zone
• Upload
accelerator
• Cloudify
Accelerator Service in Cloud
HEAT
orchestrator
POWER8/PowerKVM/Docker
Compute Network StorageFPGA/GPUaccelerator
• Apply VM
with
accelerator
1. Launched the accelerator service in Apr.2015
2. GCG announced SuperVessel to support ISV supporting planwith accelerator service (Jul.2015)
3. GCG announced the Accelerator App Store on SuperVessel on Sep.16 with partners
• SuperVessel developed the leading FPGA accelerator sharing service by extending OpenStack capability.
• IBM SuperVessel team and Xilinx collaboratively developed the first Accelerator DevOps Service on the cloud and it will be launched in the OpenPOWER summit 2016 (Apr.5~8, 2016)
• It leveraged the OpenStack HEAT and Nova-docker capability.
Accelerator DevOps Service on OpenPOWER cloud
• FPGA resource virtualization with Docker and KVM. (2015)
• Accelerator scheduling for FPGA resource, by enabling high level framework
e.g. Mesos, OpenStack, Kubenetes, etc. (2016)
• Debug and fault handling across FPGA+VM and IDE environment (2016)
• CAPI virtualization and sharing (2016)
Online Accelerator project management
Online developmentservice with
Cloud-based IDE
Publish to Accelerator App. Store and deployment for
application on cloud
Key Technologies
Test in VM/Dockerequipped with FPGA(for POWER8 & CAPI)
(Collaboration with Xilinx)
Easily Develop and Build!!!
AccDNN Service : Bridge of Deep Learning on FPGA Without Programming
Our Objectives:• Build up a seamless Bridge between deep learning and FPGA for data scientists without any FPGA experience.
• Needn’t involve any programming effort and keep the RTL level performance.
• Directly use net definition file and trained model parameters (weights) to generate FPGA implementation automatically.
Train Data Set DNN Net File Model Parameters(weights)
AccDNN
Training Process (GPU Accelerator)
Online application with Recognition Process (FPGA Accelerator)
Download
Supervessel user could generate DNN accelerator on FPGA
Automatically on OpenPOWER cloud
CPU + GPU CPU + FPGA
FPGA DNN accelerator
bit file
Cognitive application
Tbl.1 Performance comparison with CIFAR-10 (12 layers with ~384K weights)
Throughput(images/s)
Images / watts
Latency
POWER8 CPU 85
Xilinx xc7vx690tffg1
157-2
6671(2016/3/8)
223.4 150 us
The tested performance of AccDNN is 23x of CPU
Cognitive service (on BlueMix)
Model training (e.g. with Caffe)
Recognition with accelerator
SuperVessel : The Cognitive Computing platform on OpenPOWER Cloud
SuperVessel 2.0 is to build the cognitive computing platform on OpenPOWER cloud. Free for R&D activities on OpenPOWER.
SuperVessel demonstrated the OpenPOWER advantage for Cognitive Computing: • Massive Dockers Support – SMT capability
• High performance with optimized accelerator interface : CAPI and NVLink
• Easy for programming with CAPI
4/1/2016 12
SuperVessel cloud URL: www.ptopenlab.com
Backup
4/1/2016 13
We are entering a new period of computing history -- the cognitive computing era.
Nanotechnology
SupercomputingNeuroscience
Cognitive
Computing
Cognitive systems will require innovation breakthroughs
at every layer of information technology.
Nanotechnology
Programming
Machine Learning
Computer System Design
Information Management
Interface Man & Machine
The Tabulating Era
(1900s – 1940s)The Programming Era
(1950s – present)
The Cognitive Era
(2011 – )
Cognitive computing refers to systems that learn at scale, reason with purpose and interact with humans naturally.