An Automatic Video Information Processing and Conversion System for Multimedia Messaging Service and Mobile Internet

Miss Chan Pik Wah, PatMiss Ngai Cheuk Han, Edith

Video over InternEt and Wireless Technologies Lab., CUHK

6th IT Excellence Awards (Post Secondary)

2004/06/09

VIEWLab, CSE, CUHK

Video over InternEt and Wireless Technologies Laboratory

VIEWLab is a research and application development laboratory Computer Science and Engineering, CUHK Prof. Michael R. Lyu

Focus on Video Information Processing Technologies Mobile and Wireless Application

Development

VIEWLab, CSE, CUHK

Presentation Outline

MotivationFunctionality & DesignDemonstrationInnovationProspect for future commercializationQ&A

VIEWLab, CSE, CUHK

Motivation

Needs of mobile video content is increasingVideo information processing technologies developedA prototype system is created to automate the video authoring processEnabling faster and cheaper mobile video entertainment

VIEWLab, CSE, CUHK

Functionality & Design

An automatic video information processing and conversion system for MMS and Mobile InternetTargets Fully automation Mechanical processing with pre-defined

knowledge Various video content type Short processing time Configurable for different needs

VIEWLab, CSE, CUHK

Overview of System

Advertisement Removal

Program Identification

Scene Analysis

MMS(Pictures +

Sound) /Mobile Video

Video Streaming / Progressive Download

MMS Component

Creation

Mobile InternetVideo Clip Conversion

TV Broadcast

VIEWLab, CSE, CUHK

Video Information Processing

These are the technologies to extract information from video using visual and audio channelsBy extracting/identifying the information, “effective” condensation/summary techniques can be applied through information fusion or recognition stepsWe define a type of information a “modality”

Transcript/Speech On-screen characters Face Visual features

VIEWLab, CSE, CUHK

Every Bit Counts

MMS size is limited to within hundred KB In order to delivery “useful” information to

viewer Condensation/summary techniques are required

to locate/prioritize the most important bits

Bandwidth cost is relatively higher than other channels the “value” of the content transmitted is curial

to successfully MMS business Utilize the “value” of the bandwidth use, only

transmitting the important bits

VIEWLab, CSE, CUHK

Main Features of Video2MMS

Advertisement removal TV program identification Video condensation/summaryMMS creation and adaptation for different mobile handsets

VIEWLab, CSE, CUHK

Advertisement Removal

Based on the logo detection techniques developed, TV program and non TV program segments can be separated automatically

Program

TV

Broadcast

Non Program Non Program

VIEWLab, CSE, CUHK

TV Program Identification

Based on the visual features and audio characteristics, the desire TV program can be identified

VIEWLab, CSE, CUHK

Video Condensation / Summary

Condense the long video into short slide show presentation Segment the video into different scene

Two pass minimum difference color histogram Choose the most representative shots

Video editing style Longest scene

Video Summary

VIEWLab, CSE, CUHK

MMS Creation and Adaptation

Different handsets have different MMS size and screen size Nokia 7650, 3650 - 100KB, 176 x 136 SonyEricsson T68i, T300 – 30KB, 101 x 80 Panasonic GD88 – 45KB, 176 x 136

Convert the pictures to correct screen size Utilize the MMS size for most information (number of pictures and duration)Configurable for new handsets

VIEWLab, CSE, CUHK

MMS Matrix

MMS Size = audio size + (picture size * n)Audio size = length of audio * 0.8k, (4.75 bit-

rate)Picture size = f(screen size, number of colors)

Handset MMS Size

Duration

Screen Size

Number of Color

AverageSize

Number of Pictures

Nokia 3650 100KB 60sec 174 x 136

128 10K 4 ~ 5

SonyEricsson T68i

30KB 30sec 101 x 80

32 3K 3 ~ 4

Panasonic GD88

45KB 45sec 176 x 136

64 5K 4 ~ 5

VIEWLab, CSE, CUHK

Mobile Internet Video Clips Conversion

Control third party modules to cut and convert video into short video clipsDifferent handsets have different MMS size and screen size 3GPP format RealVideo format Microsoft Windows Media Format

Clips size configurable for different usage 100KB for MMS delivery >100KB for video streaming

VIEWLab, CSE, CUHK

Demonstration Video (1)

(click to play)

ATV News Headline

AMR + Animated Gif

Nokia 3650

100KB

20 seconds

174 x 136

128 colors

Animated Gif (6 pic.)

AMR Audio

VIEWLab, CSE, CUHK

Demonstration Video (2)

(click to play)

ATV News Headline

3GP for Mobile Internet

Nokia 3650

195KB

20 seconds

174 x 136

256 colors

VIEWLab, CSE, CUHK

Demonstration Video (3)

(click to play)

ATV Weather Report

AMR + Animated Gif

Nokia 3650

100KB

60 seconds

174 x 136

128 colors

Animated Gif (5 pic.)

AMR Audio

VIEWLab, CSE, CUHK

Demonstration Video (4)

(click to play)

ATV Hourly News

AMR + Animated Gif

Nokia 3650

100KB

40 seconds

174 x 136

128 colors

Animated Gif (6 pic.)

AMR Audio

VIEWLab, CSE, CUHK

Demonstration Video (5)

(click to play)

Horse Racing

3GP in MMS

Nokia 3650

95KB

20 seconds

174 x 136

VIEWLab, CSE, CUHK

Demonstration Video (6)

(click to play)

Horse Racing

RealVideo in MMS

Nokia 3650

95KB

20 seconds

120 x 96

VIEWLab, CSE, CUHK

Demonstration Video (7)

(click to play)

Soccer Goal Shot

RealVideo in MMS

Nokia 3650

92KB

5 seconds

120 x 96

VIEWLab, CSE, CUHK

System Demonstration

ModulesEditor

Main Panel

Task Scheduler

TaskProfile

VIEWLab, CSE, CUHK

Innovation

We believed this is the first application applied Video Information Processing

technologies on automating the content

preparation processes for mobile handset in Hong Kong

VIEWLab, CSE, CUHK

Prospect for future commercialization

Base on this prototype, our laboratory technology commercialization team developed a production system called Video2MMS™Video2MMS™ was licensed to HKCSL in spring 2003 to prepare daily news and weather video clips and MMSPress release http://www.hkcsl.com/show.jsp?

pid=4&cnid1=407&type1=1&rhtID1=3&lid=2

VIEWLab, CSE, CUHK

Tested Handsets

MMS Nokia 7560, 3650 SonyEricssion T68i, T300 Panasonic GD88

VideoClips (Download) MS PocketPC, XDA

VIEWLab, CSE, CUHK

Recommended System Requirements

Intel P4 1.0G Hz 512MB RAM Microsoft Windows 2000 or above DirectX 8.1 TV Capture Card (MPEG-1) Optional third party software

Real Encoder Nokia 3GP Encoder Sony Ericsson AMR Convertor Microsoft Windows Media Encoder

VIEWLab, CSE, CUHK

Q&A