1

2020, December, 03Solodov IgorBigData & AI Research TeamNovosibirsk Research Centre, Huawei Russia.

Huawei Cloud Big Data & AI

2

Telecom

IT

ITS / Mobility

Industry 4.0

E-/M-Health

Wireless Networks

Cloud and AI

Optical System

Terminals and IoTings

+ +New Materials

New Theories

New Components

New Devices

3

Better connection = more data!

4

5

Huawei Cloud Big Data & AI - product History

Hadoop Kernel Optimization And

Community Contributions

Performance-oriented and equipment-based

Reliable and secure self-management

Enterprise Private Cloud

Public Cloud (Enterprise Intelligence)

2007 2011 2013 2015 2017-2020

Big Data research

Cloud EI ServiceTelco Big Data Product

& Solution

• IDC：Top1 big data provider in China

• Patent: 190+

6

Open Ecosystem is the Foundation of science and technology development

7

Huawei DLI services – hight level architecture model

data ingest

Modeltraining

InferenceData

preprocess

CarbonData(Unified Data Management)

Dataanalysis

Data annotation

Monitor

ModelArtsDIS/CDM MRS/DLI

Data Governance

Object Storage

DWS

CSS

GES

KG

Data Source

Data Source

Data Source

DB、Text、Image、Video、Voice…

DB、Text、Image、Video、Voice…

DB、Text、Image、Video、Voice…

8

CarbonSQL

Stream, Batch ingest

ETL

AnalystData scientistData engineer

DB

JSON

XML

CSV

AVRO

JPG

WAV

TXT

CarbonData

OBS/HDFSMachine learning

Deep learning

BI，data mart

BI, adhoc, AI, update, unstructured

One data – multiple processing/analysis options

columnar

row index

Compression/Encoding

ACID

…

AutoTune

IDC Public Private compase

CarbonFabric

9

HBaseCarbonFile

One-stop big data platform — MRS

Hive

Batch processing

SparkSQL Presto

Interactive search

FlinkSQL

Stream query

MLlib

Data mining

PySpark

Flink StormSparkStreaming

Stream computing

ORCKafka

Flume

Loader

Data access

MapReduce TezSpark

Batch processing

IoT access

Third-party tools

Parquet TXT

HDFS

Enterprise-class one-stop big data platform, 100% compatible with open source ecosystem APIs Optimized for cloud storage: Indexing, caching, and object store optimization Professional O&M assurance

CarbonData

MRS: One-Stop Big Data Platform

10

BigData ETL

11

Datasource analysis of multiple data formats and SQL on AI intelligent analysis are supported.

12

Geographic BigData Analysis

13

Large-scale Log Analysis

14

HUAWEI CLOUD EI - variety of AI services and functions

MindSpore

GPU/X86

ModelArts

General APIs Advanced APIs

ASR TTS CBS

Moderation

Image

FaceOCR

ImageSearch VCM VCT

VGS VCC VCRIDS

Pre-Integrated Solutions

GES

NLP

AIS

Basic Platform

Services

City Internet Home Vehicle

MLSDLS Batch

Logistics Healthcare Campus Manufacturing

UPredict RLSExeMLData

Lake

Ascend Kunpeng

21Platform

services 22Vision

services 12Language

services

Decision-making

services4

52Platform

functions 99 APIs 8Pre-integrated

solutions

59

159 Functions

Services

As of March 31, 2019

Available on huaweicloud.com/ei/

15

ModelArts, One-Stop AI Development Platform

DataData

processing

Model training

Model management

Deployment

AI market

Data collection

Data filtering

Data labeling

Version management

Public dataset

Online coding

Common AI engine

Pre-integrated algorithm

Hyper-parameter selector

Distributed cluster

Model visualization

ExeML

Online services

Batch services

Edge services

AI Application 1

AI Application 2

Model Updating

Data Optimization

Model trading

API trading

Dataset trading

Model warehouse

Model traceability

Precision tracking

APP DeveloperCitizen Data Scientist

AI ExpertFor all AI developers: AI Ops

16

Video

DataPreprocess

Images

Intelligent Labelling

Powered by MindData AI Data Framework

Labelled Dataset

Checking Active Learning

Manual Labelling

High Confident cases

Difficult cases

Pre-labelled Dataset

Train Manage Deploy Data

Intelligent Labelling

17

ModelArts, Outstanding Performance

Inference

Performance

Training

Performance

https://dawn.cs.stanford.edu/benchmark/ImageNet/train.html

https://dawn.cs.stanford.edu/benchmark/ImageNet/inference.html

18 min.

4 min.

Fast.ai on AWS HUAWEI CLOUD EI

4.21 ms

2.45 ms

Alibaba Cloud HUAWEI CLOUD EI

Time taken to train an image classification system

(128 nodes, 93+% accuracy on ImageNet)

Latency required to classify 1 ImageNet image

using a model with 93+% accuracy

As of March 31, 2019

Train Manage Deploy Data

18

Model ManagementTrain Manage Deploy Data

Version

managementModel tracing Precision tracking

80% 90%

Model Management

Model

acquisition

Model

deployment

Model warehouse

19

Edge Model Optimization(Device, latency, and accuracy constraints)

Frictionless Intelligence Extension

API

Batch

AI models

Online service

Batch inference

Edge inference

Edge inference

High throughput, low latency, auto-scale

Inference optimization

Large batch data inference

High-efficiency distributed computing

Deeply integrated with IEF

Supports Huawei Ascend AI chip

Supports Huawei SDC, CloudLink, etc.

Network

Distillation

Model

Compression

Channel pruning

Quantization

Test Bed

Evaluation

Train Manage Deploy Data

20

• The Huawei's Connection Management Platform is an example of

building an industrial system based on a Cloud Native architecture

21

Thank You