Content Based MPEG Video Traffic Content Based MPEG Video Traffic

ModelingModeling Ali M. Dawood and Mohammed Ghanbari, senior member, IEEEAli M. Dawood and Mohammed Ghanbari, senior member, IEEE

Presented by Presented by

Premchander Reddy & Lakshmi deepthi PasupuletiPremchander Reddy & Lakshmi deepthi Pasupuleti

To To

Donald AdjerohDonald Adjeroh

As a partial requirement for course CS558As a partial requirement for course CS558

What is video modeling?What is video modeling?

Video model is an aid for designing and testing future communication networks that will carry multiplexed video traffic. It is an essential tool in estimating many networking issues such as the delay arising from statistical multiplexing and the bandwidth required for carrying video

Survey…Survey…....Classic ModelingClassic Modeling

Non MPEG Non MPEG

Maglaris Maglaris

Sen Sen GrunenfelderGrunenfelder HeymanHeyman HughesHughes shim shim

MPEGMPEG

PanchaPancha HeymanHeyman WuWu KrunzKrunz NiNi

Classic ModelingClassic Modeling

• In the classical modeling the mean and In the classical modeling the mean and variance of real video are matched to an AR variance of real video are matched to an AR ( Auto regressive) model or any known ( Auto regressive) model or any known distribution function. The nature of the video distribution function. The nature of the video content and the length of the video is not content and the length of the video is not taken into consideration here.taken into consideration here.

• But modeling a video considering its nature But modeling a video considering its nature and content can obviously result in better and content can obviously result in better representation of video.representation of video.

Introduction: Content Based ModelingIntroduction: Content Based Modeling

• Decomposition of videoDecomposition of video

Video Clip : Such as a FilmVideo Clip : Such as a Film

Stories : Such as News sessionStories : Such as News session

Shots : Continuous action Shots : Continuous action

GOP : Group of PicturesGOP : Group of Pictures

Video Frames : I P B frames Video Frames : I P B frames

Shot ClassificationShot Classification

• Shot is a homogeneous Shot is a homogeneous videovideo

• Modeling of video should Modeling of video should start from modeling of shot.start from modeling of shot.

• Texture and Motion are used Texture and Motion are used to classify shots into groupsto classify shots into groups

• 3 levels of texture and 3 3 levels of texture and 3 levels of motion are chosen levels of motion are chosen

• The levels are namely LL LM The levels are namely LL LM LH ML MM MH HL HM HHLH ML MM MH HL HM HH

• L M H stand for Low Medium L M H stand for Low Medium and High respectivelyand High respectively

Measuring the Texture and MotionMeasuring the Texture and Motion

• Texture: The average magnitudes of the Texture: The average magnitudes of the DCT coefficients of luminance/block for DCT coefficients of luminance/block for each frame is calculated and then each frame is calculated and then averaged over the shot.averaged over the shot.

• Motion: The magnitude of motion Motion: The magnitude of motion vectors/macro block are extracted for vectors/macro block are extracted for each frame type and then averaged each frame type and then averaged over the shot.over the shot.

• The relation between average The relation between average DCT coefficient and bit rate is DCT coefficient and bit rate is distinct for the I-frame.distinct for the I-frame.

• Due to motion it is not so Due to motion it is not so distinct for P and B frames.distinct for P and B frames.

• So we take the texture So we take the texture information from the I-information from the I-frames.frames.

• The motion information Is The motion information Is taken from P and B frames taken from P and B frames since I frame is intra-frame since I frame is intra-frame coded.coded.

• I frames are combined with I frames are combined with those of P and B frames for a those of P and B frames for a reliable classification.reliable classification.

• For example the classification For example the classification of texture can be known from of texture can be known from I frames and motion-based I frames and motion-based classification is known from P classification is known from P and B frames.and B frames.

Characterization of Real Characterization of Real VideoVideo• The shot classification The shot classification

were applied to a 30 were applied to a 30 min BBC news bulletin.min BBC news bulletin.

• The frequencies of The frequencies of occurrence of each shot occurrence of each shot type was tabulated.type was tabulated.

• The transition The transition probability table was probability table was also tabulated.also tabulated.

• Transition probability Transition probability table gives us the table gives us the probability of a probability of a particular shot type particular shot type following the next type.following the next type.

Composition of Video ClipsComposition of Video Clips

• Mean bit rate is calculated Mean bit rate is calculated for each shot type and is for each shot type and is divided into I,P,B frames.divided into I,P,B frames.

• After classification of shots After classification of shots and determination of bit and determination of bit rate, proportion of I,P,B bit rate, proportion of I,P,B bit rates, the shot can be rates, the shot can be defined as a vector.defined as a vector.

SSkk(AR_I(AR_Iii, AR_P, AR_Pii, AR_B, AR_Bii, , ttkk) )

k=1,2..N is the kk=1,2..N is the kthth shot in a shot in a clip of N shots, i=1,2..9 is clip of N shots, i=1,2..9 is the ithe ithth shot type,t shot type,tkk is the is the duration of the kduration of the kthth shot. shot.

Summary of synthetic Summary of synthetic generation of CBMgeneration of CBM

• 1. Define the number of shots(N) in the video clip.1. Define the number of shots(N) in the video clip.

• 2. Specify the shot type and derive the mean bit 2. Specify the shot type and derive the mean bit rate of each shot type, and derive the mean bit rate of each shot type, and derive the mean bit rate of each shot from the overall mean bit rate.rate of each shot from the overall mean bit rate.

• 3. Specify the shot duration, according to the 3. Specify the shot duration, according to the statistics and Gamma function.statistics and Gamma function.

• 4. Using the mean and variance, calculate the 4. Using the mean and variance, calculate the auto-regressive (AR) model’s parameters for the auto-regressive (AR) model’s parameters for the kth shot[6].kth shot[6].

• 5. Go to step 3 for the kth + 1 shot.5. Go to step 3 for the kth + 1 shot.

Results from Simulation of Results from Simulation of Deterministic CBMDeterministic CBM• The performance of The performance of

the proposed model the proposed model was tested against a was tested against a real video clip.real video clip.

• A virtual video clip was A virtual video clip was edited from 11shots edited from 11shots and the proposed and the proposed model was applied.model was applied.

• It was observed that It was observed that the CBM traffic closely the CBM traffic closely follows the real non follows the real non homogenous MPEG homogenous MPEG traffic.traffic.

Realistic CBMRealistic CBM

• Since the deterministic CBM is Since the deterministic CBM is based on subjective based on subjective description of the video description of the video content, the shot classification content, the shot classification may vary from person to may vary from person to person.person.

• In order to derive a more In order to derive a more realistic content based video realistic content based video model the transition and the model the transition and the durations are made durations are made probabilistic, based on the probabilistic, based on the shot characteristics.shot characteristics.

• A new shot type transition A new shot type transition probability table is probability table is formulated.formulated.

• A nine-state model is used to A nine-state model is used to represent the probabilistic represent the probabilistic CBM.CBM.

Summary of Probabilistic Summary of Probabilistic CBMCBM• 1.Start from an initial state.1.Start from an initial state.• 2. Find the duration of the slot with a 2. Find the duration of the slot with a

gamma function of gamma function of αα=2 and =2 and ββ=70.=70.• 3. According to the type of the shot, use 3. According to the type of the shot, use

the table to calculate the auto-regressive the table to calculate the auto-regressive (AR) model’s parameters.(AR) model’s parameters.

• 4. Run AR model for the duration of the 4. Run AR model for the duration of the shot given in step 2.shot given in step 2.

• 5. Transit to the next state according to 5. Transit to the next state according to the new transition table.the new transition table.

• 6. Go to step 2.6. Go to step 2.

Comparison of ResultsComparison of Results

• A 2 min video clip was modeled with a classical AR A 2 min video clip was modeled with a classical AR method, deterministic CBM and probabilistic CBM.method, deterministic CBM and probabilistic CBM.

• The worst performance was observed for the The worst performance was observed for the classical method which do not consider video classical method which do not consider video content.content.

• The best performance was observed for The best performance was observed for deterministic CBM .deterministic CBM .

• The purely statistical probabilistic CBM had much The purely statistical probabilistic CBM had much better performance than the classical model.better performance than the classical model.

• The network performance with these traffics was The network performance with these traffics was also evaluated, where each model’s traffic has also evaluated, where each model’s traffic has been fed into an ATM multiplexer with network been fed into an ATM multiplexer with network loads of 70% and 90%.loads of 70% and 90%.

Network PerformanceNetwork Performance

Limitations Limitations

1.1. Image representations based on low-Image representations based on low-level visual primitives such as texture, level visual primitives such as texture, and motion.and motion.

2.2. The determination of shots is also a The determination of shots is also a complex task. complex task.

3.3. Different people have different visual Different people have different visual perceptions, so classification of shots perceptions, so classification of shots based on color, texture and motion based on color, texture and motion becomes a problem.becomes a problem.

Suggested ImprovementsSuggested Improvements

• The classification of shots based on The classification of shots based on contextual information such as contextual information such as appearance of an anchor in a video appearance of an anchor in a video can be useful.can be useful.

• This type of classification is easy as This type of classification is easy as the contextual information from the contextual information from which the classification is done is which the classification is done is viewed as the same by all the people.viewed as the same by all the people.