“intelligent multimedia” · “intelligent multimedia” ... text audio image video interaction...
TRANSCRIPT
“Intelligent Multimedia”
WHAT’S MISSING?
WANG Yuntao（Mr.）Artificial Intelligence Department
Cloud Computing and Big Data Research InstituteChina Academy of Information and Communication Technology
No.11 South Yuetan Street, Beijing, P.R.ChinaMobile: +86-18611547086
Email: [email protected], [email protected]
Traditional Multimedia
Definitions
SG16
Multimedia is content that uses a combination of differentcontent forms such as text, audio, images, animations, videoand interactive content.
--Wikipedia
Text Audio Image Video interaction Multimedia Eq 1
Coding System Applications Multimedia Eq 2
Intelligent Multimedia
No Clear Descriptions yet, but
o Easy access to extensive, searchable archives of mixed text,
graphics, sounds, narrations, and video footage
o More Human-friendly interactions
o More than Just content consumers, deep mining of
multimedia data
o Not only human, but we want machines to understand
multimedia as well
……
Identifying Missing link…
We need more intelligent applications, new applications indicates a profound impact and even revolution to existing multimedia architectures
Coding System Applications Multimedia
Solid Technical Foundations
Major modern applications
Focused more on creation and transmission…
State-of-the-art applications are booming…
Identifying Missing link…
Text Audio Image Video interaction Multimedia
Natural Language Processing• Machine Translation• Automatic Abstracting• Automatic Generation……
Intelligent Speech• Speech recognition• Speech Synthesis• Question Answering……
Computer and Machine Vision• Face recognition• Object detection……
Computer and Machine Vision• Content Audition• Automatic Pilot……
Human-Machine Interface• Speech Interaction• Brain-computer interface……
We should we do?Figure out the framework
Applications are booming, we need to identify the common technical barriers behind all these applications, and figure out the Intelligence Enablers.
QoS
Representation
Computation
Data New requirements of data preparation
More mining and analyzing tasks
More Human-friendly requirements
New Intelligent QoS requirements
We should we do?Data: Data preparation
Multimedia Data Intelligent Multimedia Data
DATA LABELLING
As the gasoline of modern AI industry, data labelling has brought new requirements and challenges.
Data collection
Data labelling
Data Delivery
Data quality control
In-depth data mining and analyzing tasks brings new technical demands.Deep learning is transforming how we design computers -- Jeff Dean
Multimedia Architecture Intelligent Multimedia Architecture
System Point of View
RepresentationNetwork design
Algorithm Optimization
We should we do?Computation: System impact
Example:SVAC Surveillance video and audio codingDefines new data analysis descriptions: Rules for image analysis; Object detection; Feature analysis; Object/Behavior recognition; Statistics for objects counting
To facilitate intelligent data mining, new frame structures are proposed.
Multimedia Coding Intelligent Multimedia Coding
We should we do?Representation: Coding
How good is the video quality? How good is the compression ratio?……
Multimedia QoS Intelligent Multimedia QoS
New QoS metrics and assessment methodology are required to evaluate the intelligent part.
We should we do?QoS
How intelligent is the robot? How good is the speech recognition?……
101 Intelligent network car
102 Intelligent service robot
103 Intelligent UAV
104 Medical image auxiliary
diagnosis system
105 Intelligent identification
system
106 Intelligent speech
interactive system
107 Intelligent translation
system
108 Smart home products
··· ···
Intelligent product
201 Intelligent sensor
202 Neural network chip
203 Other basic hardware
204 Open source platform
205 Deep learning computing
platform
206 Other classes in core
foundation
··· ···
Core foundation
301 Key technical equipment
of intelligent manufacturing
302 Networked cooperative
manufacturing platform
303 Digital workshop
304 Intelligent factory
305 Other classes in
intelligent manufacturing
intelligent manufacturing
401 Industry training
resource repository
402 Intellectual property
service platform
403 others
support system
In order to further to promote the industrialization and integration application of new-generation artificial intelligence technology, AIIA has recruited artificial intelligencetechnologies and application cases for member companies and cooperative institutions. Thescope of solicitation involves the following areas:
What we have done…Collection of domestic AI tech and application cases
What we have done… Assessment and evaluation of AI related multimedia services & products
Topics Ongoing work under AIIA Enterprises
Smart Speaker Evaluation of level of intelligence of Smart Speakers
Baidu, Alibaba, Tencent, JD.com, Xiaomi, etc.
Intelligence Speech Assessment and evaluation of Intelligence Speech Service Platforms
Baidu, Tencent, iFlyteck, AISpeech, d-Ear, etc.
Computer Vision Assessment and requirement of deep-learning based face recognition and verification
Baidu, Alibaba, Tencent, YituTech, CloudWalk, Hikvision, DaHuaTech, etc.
Multimedia Datasets Standards of Datasets used for AI training and inference, including data collection, data labelling, data control and data delivery. Covering speech recognition, speech synthesis, etc.
SpeechOcean, iFlytech, Tsinghua University, datatang, etc.
Thank you for your support!
WANG Yuntao（Mr.）Artificial Intelligence DepartmentCloud Computing and Big Data Research InstituteChina Academy of Information and Communication TechnologyNo.11 South Yuetan Street, Beijing, P.R.ChinaMobile: +86-18611547086Email: [email protected], [email protected]