video coding 10110100 - universitetet i oslo · • container-format and mm framework – by apple...
TRANSCRIPT
![Page 1: Video Coding 10110100 - Universitetet i oslo · • Container-Format and MM Framework – By Apple Inc. – Originally developed for Macintosh • Start: Charles Wiltgen, 1992](https://reader034.vdocuments.us/reader034/viewer/2022050608/5faf855e0c275721d342a9fc/html5/thumbnails/1.jpg)
10110100
Vid
eo C
odin
gVideo Coding
Ifi, UiO Norsk RegnesentralVårsemester 2008Wolfgang Leister
Course INF5081Multimedia Coding and Applications
110100Vid
eo C
odin
g
Authors ...
• Wolfgang Leister• Peter Oel, • Clemens Knoerzer
![Page 2: Video Coding 10110100 - Universitetet i oslo · • Container-Format and MM Framework – By Apple Inc. – Originally developed for Macintosh • Start: Charles Wiltgen, 1992](https://reader034.vdocuments.us/reader034/viewer/2022050608/5faf855e0c275721d342a9fc/html5/thumbnails/2.jpg)
110100Vid
eo C
odin
gThe story so far ...
• Data compression– information theory– run length encoding– Huffman coding– Zif-Lempel(-Welch) algorithm– Arithmetic coding
• Colour coding and raster images
110100Vid
eo C
odin
g
Overview
• Video Formats– Container Formats: AVI,QuickTime– MJPEG– H.261– MPEG I, II– MPEG TS– H.263
![Page 3: Video Coding 10110100 - Universitetet i oslo · • Container-Format and MM Framework – By Apple Inc. – Originally developed for Macintosh • Start: Charles Wiltgen, 1992](https://reader034.vdocuments.us/reader034/viewer/2022050608/5faf855e0c275721d342a9fc/html5/thumbnails/3.jpg)
110100Vid
eo C
odin
gAVI
• Audio Video Interleaved, by Microsoft, 1992• specialisation of RIFF
– Resource Interchange File Format• Container-Format for Audio and Video• File consists of blocks (chunks)
– Each chunk characterised by 4 letters– format: ID (4Byte) length (4 Byte) data– Each chunk can contain sub-chunks.
• Outdated, but still in use
110100Vid
eo C
odin
g
QuickTime
• Container-Format and MM Framework– By Apple Inc.– Originally developed for Macintosh
• Start: Charles Wiltgen, 1992• Currently: Quicktime 7.x
![Page 4: Video Coding 10110100 - Universitetet i oslo · • Container-Format and MM Framework – By Apple Inc. – Originally developed for Macintosh • Start: Charles Wiltgen, 1992](https://reader034.vdocuments.us/reader034/viewer/2022050608/5faf855e0c275721d342a9fc/html5/thumbnails/4.jpg)
110100Vid
eo C
odin
gQuicktime Audio
• Apple Lossless• Audio Interchange (AIFF)• Digital Audio: Audio CD - 16-
bit (CDDA), 24-bit, 32-bit integer & floating point, and 64-bit floating point
• MIDI• MPEG-1 Layer 3 Audio (.mp3)• MPEG-4 AAC Audio (m4a,
.m4b, m4p)•
• QDesign Music• Qualcomm PureVoice
(QCELP)• Sun AU Audio• ULAW and ALAW Audio• Waveform Audio (WAV)
110100Vid
eo C
odin
g
Quicktime Video• 3GPP & 3GPP2 file formats• AVI file format• Bitmap (BMP) codec and file
format• DV file (DV NTSC/PAL and DVC
Pro NTSC/PAL codecs)• Flash & FlashPix files• GIF and Animated GIF files• H.261, H.263, and H.264 codecs• MPEG-1, MPEG-2, MPEG-4,
AVC
• JPEG, Photo JPEG, and JPEG-2000 • Quartz Composer Composition• QuickTime Movie (.mov) and QTVR
movies• Sorenson Video 2 and 3 codecs• Apple Video, Cinepak,• Component Video, Graphics, Planar
RGB• still image formats: PNG, TIFF, TGA• Cached information from streams: QTCH
![Page 5: Video Coding 10110100 - Universitetet i oslo · • Container-Format and MM Framework – By Apple Inc. – Originally developed for Macintosh • Start: Charles Wiltgen, 1992](https://reader034.vdocuments.us/reader034/viewer/2022050608/5faf855e0c275721d342a9fc/html5/thumbnails/5.jpg)
110100Vid
eo C
odin
gMJPEG
• Motion-JPEG• Sequence of JPEG-Frames• not a standard• many proprietary formats
– e.g., AVI and QuickTime• only Baseline-JPEG• audio track(s)
110100Vid
eo C
odin
g
MJPEG
• Compression as in JPEG• Temporal dependencies are not used.• suitable for video cutting software:
– cut is possible at each frame– no quality decrease when cutting
• Hardware support possible!
![Page 6: Video Coding 10110100 - Universitetet i oslo · • Container-Format and MM Framework – By Apple Inc. – Originally developed for Macintosh • Start: Charles Wiltgen, 1992](https://reader034.vdocuments.us/reader034/viewer/2022050608/5faf855e0c275721d342a9fc/html5/thumbnails/6.jpg)
110100Vid
eo C
odin
g MJPEG by Parallax... example
Header
Pointer to Index
Offset1 Offset2 ... Offsetn Offset1
Offset2
Offsetn
...Frame Frame ... Frame
FrameLo
ad V
ideo
EoF
ram
eVideo-data
Load
Vid
eo
Load
Aud
io
EoF
ram
eVideo-data
Audio-data
110100Vid
eo C
odin
g
MJPEG by Parallax
• Header contains:– Id / Version– Frames per second / number of frames– Width / Height– Bandwidth– Quantising factor– Number of Audio Tracks / Sampling Rate– Offset of frame index
![Page 7: Video Coding 10110100 - Universitetet i oslo · • Container-Format and MM Framework – By Apple Inc. – Originally developed for Macintosh • Start: Charles Wiltgen, 1992](https://reader034.vdocuments.us/reader034/viewer/2022050608/5faf855e0c275721d342a9fc/html5/thumbnails/7.jpg)
110100Vid
eo C
odin
gITU-T / CCITT
• Standardisation Organisation– CCITT (Commité Consultatif International de
Télécommunications et Télégraphique)– ITU-T (International Telecommunication Union)
110100Vid
eo C
odin
g
H.261
• Video Codec for Audiovisual Services– Image telephony / video conferences– Adapted for ISDN (p x 64 kbit/s)– Constant data rate by feedback
• 4:2:0 Sampling (Chrominance-channels with half the resolution of luminance channel (number of rows and columns))
• Resolutions:– CIF: 352 x 288 (Common Intermediate Format)– QCIF: 176 x 144 (Quarter CIF)
![Page 8: Video Coding 10110100 - Universitetet i oslo · • Container-Format and MM Framework – By Apple Inc. – Originally developed for Macintosh • Start: Charles Wiltgen, 1992](https://reader034.vdocuments.us/reader034/viewer/2022050608/5faf855e0c275721d342a9fc/html5/thumbnails/8.jpg)
110100Vid
eo C
odin
gSubsampling
• A:B:C Notion of CCIR-601• 4:2:2 horizontal 2:1 downsampling of
colour channels• 4:1:1 horizontal 4:1 downsampling of
colour channels• 4:2:0 horizontal and vertical 2:1
downsampling of colour channels
110100Vid
eo C
odin
g
ExampleY Cb, Cr
each
4:2:2 352 x 288 176 x 288
4:1:1 352 x 288 88 x 288
4:2:0 352 x 288 176 x 144
![Page 9: Video Coding 10110100 - Universitetet i oslo · • Container-Format and MM Framework – By Apple Inc. – Originally developed for Macintosh • Start: Charles Wiltgen, 1992](https://reader034.vdocuments.us/reader034/viewer/2022050608/5faf855e0c275721d342a9fc/html5/thumbnails/9.jpg)
110100Vid
eo C
odin
g Cr Cr Cr
Cr Cr Cr
Cr Cr Cr
Cr Cr Cr
Cb Cb Cb
Cb Cb Cb
Cb Cb Cb
Cb Cb Cb
Example 4:2:2
Y Y Y Y Y Y
Y Y Y Y Y Y
Y Y Y Y Y Y
Y Y Y Y Y Y
110100Vid
eo C
odin
g
Cr Cr Cr
Cr Cr Cr
Cb Cb Cb
Cb Cb Cb
Example 4:2:0
Y Y Y Y Y Y
Y Y Y Y Y Y
Y Y Y Y Y Y
Y Y Y Y Y Y
![Page 10: Video Coding 10110100 - Universitetet i oslo · • Container-Format and MM Framework – By Apple Inc. – Originally developed for Macintosh • Start: Charles Wiltgen, 1992](https://reader034.vdocuments.us/reader034/viewer/2022050608/5faf855e0c275721d342a9fc/html5/thumbnails/10.jpg)
110100Vid
eo C
odin
gH.261
Two frame types:• IntraFrames
– Very similar to JPEG-Image (DCT, Quantising, Coding)
• InterFrames– Code differences to previous frame.– Movements are compensated by motion-
estimation.
110100Vid
eo C
odin
g
Intra- / InterFramesIntraFrame
IntraFrame
InterFrames
InterFrames
![Page 11: Video Coding 10110100 - Universitetet i oslo · • Container-Format and MM Framework – By Apple Inc. – Originally developed for Macintosh • Start: Charles Wiltgen, 1992](https://reader034.vdocuments.us/reader034/viewer/2022050608/5faf855e0c275721d342a9fc/html5/thumbnails/11.jpg)
110100Vid
eo C
odin
gMotion Estimation
110100Vid
eo C
odin
g
Motion-Estimation
![Page 12: Video Coding 10110100 - Universitetet i oslo · • Container-Format and MM Framework – By Apple Inc. – Originally developed for Macintosh • Start: Charles Wiltgen, 1992](https://reader034.vdocuments.us/reader034/viewer/2022050608/5faf855e0c275721d342a9fc/html5/thumbnails/12.jpg)
110100Vid
eo C
odin
gH.261
• Motion-Vector max. ±15 Pixels• all DCT-Coefficients are quantised with
same value• Quantising controled by output stream
(feedback).• Quantising has dead zone
110100Vid
eo C
odin
g
H.261 data layout
![Page 13: Video Coding 10110100 - Universitetet i oslo · • Container-Format and MM Framework – By Apple Inc. – Originally developed for Macintosh • Start: Charles Wiltgen, 1992](https://reader034.vdocuments.us/reader034/viewer/2022050608/5faf855e0c275721d342a9fc/html5/thumbnails/13.jpg)
110100Vid
eo C
odin
gMPEG
• Motion Picture Expert Group• ISO/IEC 11172 (MPEG-1) • ISO/IEC 13818 (MPEG-2)• Video / Audio compression and coding
110100Vid
eo C
odin
g
MPEG History
• MPEG-1: video and audio (1992)• MPEG-2: Digital TV and DVD (1994)• MPEG-3: HDTV (withdrawn)• MPEG-4: Multimedia applications (1998)• MPEG-7: MM search and filtering (2001)• MPEG-21: Multimedia framework
![Page 14: Video Coding 10110100 - Universitetet i oslo · • Container-Format and MM Framework – By Apple Inc. – Originally developed for Macintosh • Start: Charles Wiltgen, 1992](https://reader034.vdocuments.us/reader034/viewer/2022050608/5faf855e0c275721d342a9fc/html5/thumbnails/14.jpg)
110100Vid
eo C
odin
gMPEG History
• MPEG-1: video and audio (1992)• MPEG-2: Digital TV and DVD (1994)• MPEG-3: HDTV (withdrawn)• MPEG-4: Multimedia applications (1998)• MPEG-7: MM search and filtering (2001)• MPEG-21: Multimedia framework
110100Vid
eo C
odin
g
MPEG-1 • ISO 11172
• Part 1: Systems• Part 2: Video• Part 3: Audio (Layer I, II, III)• Part 4: Conformance• Part 5: Software Simulation
![Page 15: Video Coding 10110100 - Universitetet i oslo · • Container-Format and MM Framework – By Apple Inc. – Originally developed for Macintosh • Start: Charles Wiltgen, 1992](https://reader034.vdocuments.us/reader034/viewer/2022050608/5faf855e0c275721d342a9fc/html5/thumbnails/15.jpg)
110100Vid
eo C
odin
gMPEG-2 • ISO 13818
• Parts 1-5 as in MPEG-1• Part 6: DSM-CC (Digital Storage Medium Command and Control)
• Part 7: NBC (Non-Backwards Compatible Audio) • Part 8: 10-bit video extension (withdrawn)
• Part 9: RTI (Real-time interface between set-top box and head end server)
110100Vid
eo C
odin
g
MPEG
Which parts of MPEG 2 are of interest for this course?• Part 2: Video• Part 3: Audio (Layer I, II, III)• Part 1: Systems• Part 6: DSM-CC
![Page 16: Video Coding 10110100 - Universitetet i oslo · • Container-Format and MM Framework – By Apple Inc. – Originally developed for Macintosh • Start: Charles Wiltgen, 1992](https://reader034.vdocuments.us/reader034/viewer/2022050608/5faf855e0c275721d342a9fc/html5/thumbnails/16.jpg)
110100Vid
eo C
odin
gMP3
= MPEG-1
Audio Layer 3!
110100Vid
eo C
odin
g
MPEG
• Differences to H.261– Motion-Vectors not limited to ±15 Pixels– Motion-Vektors not necessarily integer
numbers– 3 (4) Frame-Types (I-, P-, B-Frames)– Data stream not limited to p x 64 kbit/s.– Quantising of coefficients with matrix
![Page 17: Video Coding 10110100 - Universitetet i oslo · • Container-Format and MM Framework – By Apple Inc. – Originally developed for Macintosh • Start: Charles Wiltgen, 1992](https://reader034.vdocuments.us/reader034/viewer/2022050608/5faf855e0c275721d342a9fc/html5/thumbnails/17.jpg)
110100Vid
eo C
odin
gMacro blocks
• Adjacent 8x8-Blocks of channels are joined to macro blocks.
• Depending on sub sampling several 8x8 blocks are in one macro block
110100Vid
eo C
odin
g
MPEG• Frame-Types:
– I-Frame: Like IntraFrames of H.261 (ca. every 15. Frame)
– P-Frame: Like InterFrames of H.261 (Predicted Frame). Related to previous I or P-Frame
– B-Frame: (Bidirectional predicted Frame) No equivalent of H.261. Related to previous and successor I- or P-Frame
![Page 18: Video Coding 10110100 - Universitetet i oslo · • Container-Format and MM Framework – By Apple Inc. – Originally developed for Macintosh • Start: Charles Wiltgen, 1992](https://reader034.vdocuments.us/reader034/viewer/2022050608/5faf855e0c275721d342a9fc/html5/thumbnails/18.jpg)
110100Vid
eo C
odin
gI-, P- and B-Frames
I-FrameI-Frame
B-Frames
P-Frame
B-Frames
B-Frames
P-Frame
110100Vid
eo C
odin
g
I-, P- and B-FramesI
I
B-Frames
P
B-Frames
B-Frames
P
![Page 19: Video Coding 10110100 - Universitetet i oslo · • Container-Format and MM Framework – By Apple Inc. – Originally developed for Macintosh • Start: Charles Wiltgen, 1992](https://reader034.vdocuments.us/reader034/viewer/2022050608/5faf855e0c275721d342a9fc/html5/thumbnails/19.jpg)
110100Vid
eo C
odin
gI-, P- and B-Frames
I
I
B-Frames
P
B-Frames
B-Frames
P
110100Vid
eo C
odin
g
Motion
• Motion compensation– normative part of MPEG– Decoder’s point of view
• Motion estimation– NOT normative part of MPEG– Encoder’s point of view
![Page 20: Video Coding 10110100 - Universitetet i oslo · • Container-Format and MM Framework – By Apple Inc. – Originally developed for Macintosh • Start: Charles Wiltgen, 1992](https://reader034.vdocuments.us/reader034/viewer/2022050608/5faf855e0c275721d342a9fc/html5/thumbnails/20.jpg)
110100Vid
eo C
odin
gMotion Compensation
• 1 motion vector for each region• region = macro block∀⇒ 1 motion vector per macro block• Precision: 1 Pel, ½ Pel• motion vector coded differentially
(prediction derived from preceding macro block)
• Rules for resetting motion displacements
110100Vid
eo C
odin
g
Motion Compensation
• P-Pictures: – forward motion vectors
• B-Pictures: – forward motion vectors– backward motion vectors– if both used: average of pel values from forward and
backward motion-compensated reference picture
![Page 21: Video Coding 10110100 - Universitetet i oslo · • Container-Format and MM Framework – By Apple Inc. – Originally developed for Macintosh • Start: Charles Wiltgen, 1992](https://reader034.vdocuments.us/reader034/viewer/2022050608/5faf855e0c275721d342a9fc/html5/thumbnails/21.jpg)
110100Vid
eo C
odin
gMotion Compensation
• MPEG-2:– as in MPEG-1 (previous slide)– uses different names– Dual Prime Motion
• for interlaced video• averaging predictions from two adjacent fields
of opposite parity
110100Vid
eo C
odin
g
Motion-Estimation
I/P
I/P
B
![Page 22: Video Coding 10110100 - Universitetet i oslo · • Container-Format and MM Framework – By Apple Inc. – Originally developed for Macintosh • Start: Charles Wiltgen, 1992](https://reader034.vdocuments.us/reader034/viewer/2022050608/5faf855e0c275721d342a9fc/html5/thumbnails/22.jpg)
110100Vid
eo C
odin
gMotion-Estimation
• Criterion for block matching– Mean Square Error– Mean Absolute Distortion
∑∑= =
++++−++=15
0i
m
15
0j
n |i)dyyi,dx(xVi)yi,(xV|(1/256)y)MAD(x,
– Sum of Absolute Distortions (SAD)– Minimization of the bitstream
110100Vid
eo C
odin
g
Motion Estimation
• Correlation between motion vectors
![Page 23: Video Coding 10110100 - Universitetet i oslo · • Container-Format and MM Framework – By Apple Inc. – Originally developed for Macintosh • Start: Charles Wiltgen, 1992](https://reader034.vdocuments.us/reader034/viewer/2022050608/5faf855e0c275721d342a9fc/html5/thumbnails/23.jpg)
110100Vid
eo C
odin
gMotion Estimation
• Motion displacement search algorithms– pel-recursive
• iterative process• use intensity gradient and frame difference
– block matching• compute measure of distortion• select vector that minimizes distortion
110100Vid
eo C
odin
g
Motion Estimation
• Fast search algorithm - sparse sampling• Variable resolution search techniques• Statistically sparse searches• Spatial continuity• Telescopic search• 3D spatial / temporal estimation• Phase correlation• others ...
![Page 24: Video Coding 10110100 - Universitetet i oslo · • Container-Format and MM Framework – By Apple Inc. – Originally developed for Macintosh • Start: Charles Wiltgen, 1992](https://reader034.vdocuments.us/reader034/viewer/2022050608/5faf855e0c275721d342a9fc/html5/thumbnails/24.jpg)
110100Vid
eo C
odin
gMotion Estimation
Overview in:Mitchell, Pennebaker, Fogg, and LeGall:MPEG Video Compression Standard,Chapman&Hall, 1996p 301 ff.
110100Vid
eo C
odin
g
Frame Order
Display orderI0B1B2P3B4B5P6B7B8I9...
I0P3B1B2P6B4B5I9B7B8 …Coding order
![Page 25: Video Coding 10110100 - Universitetet i oslo · • Container-Format and MM Framework – By Apple Inc. – Originally developed for Macintosh • Start: Charles Wiltgen, 1992](https://reader034.vdocuments.us/reader034/viewer/2022050608/5faf855e0c275721d342a9fc/html5/thumbnails/25.jpg)
110100Vid
eo C
odin
gMPEG data layout
110100Vid
eo C
odin
g
MPEG Encoder
FrameRe-order Motion
Estimator- DCT Q VLC BufferMulti-
plex
Regulator
Q-1
DCT-1
+Framestore
andPredictor
Motion-Vectors
Modes
![Page 26: Video Coding 10110100 - Universitetet i oslo · • Container-Format and MM Framework – By Apple Inc. – Originally developed for Macintosh • Start: Charles Wiltgen, 1992](https://reader034.vdocuments.us/reader034/viewer/2022050608/5faf855e0c275721d342a9fc/html5/thumbnails/26.jpg)
110100Vid
eo C
odin
gMPEG Decoder
Buffer Q-1 DCT-1 +
Framestoreand
PredictorMotion-Vectors
MUX-1
Side Information
Quantizer Stepsize
110100Vid
eo C
odin
g
MPEG TS• Packetised Elementary Stream (PES)• MPEG-2 Programme Stream• MPEG-2 Transport Stream (TS)
– fixed size 188 bytes packets• DSM-CC: Digital Storage Medium Command and
Control• Service Information Tables
– Information on streams, programmes, networks, conditional access, textual description, rating, ...
MPEG-1 compatibility
![Page 27: Video Coding 10110100 - Universitetet i oslo · • Container-Format and MM Framework – By Apple Inc. – Originally developed for Macintosh • Start: Charles Wiltgen, 1992](https://reader034.vdocuments.us/reader034/viewer/2022050608/5faf855e0c275721d342a9fc/html5/thumbnails/27.jpg)
110100Vid
eo C
odin
gMPEG-2 TS
110100Vid
eo C
odin
g
MPEG-2 Multiplexer
![Page 28: Video Coding 10110100 - Universitetet i oslo · • Container-Format and MM Framework – By Apple Inc. – Originally developed for Macintosh • Start: Charles Wiltgen, 1992](https://reader034.vdocuments.us/reader034/viewer/2022050608/5faf855e0c275721d342a9fc/html5/thumbnails/28.jpg)
110100Vid
eo C
odin
g Presentation UnitAccess Unit
110100Vid
eo C
odin
g
PES
![Page 29: Video Coding 10110100 - Universitetet i oslo · • Container-Format and MM Framework – By Apple Inc. – Originally developed for Macintosh • Start: Charles Wiltgen, 1992](https://reader034.vdocuments.us/reader034/viewer/2022050608/5faf855e0c275721d342a9fc/html5/thumbnails/29.jpg)
110100Vid
eo C
odin
gProgramme Stream Multiplex
• Pack Header – contains system clock reference– must occur every 0.7 sec.
110100Vid
eo C
odin
g
MPEG-2 TS
![Page 30: Video Coding 10110100 - Universitetet i oslo · • Container-Format and MM Framework – By Apple Inc. – Originally developed for Macintosh • Start: Charles Wiltgen, 1992](https://reader034.vdocuments.us/reader034/viewer/2022050608/5faf855e0c275721d342a9fc/html5/thumbnails/30.jpg)
110100Vid
eo C
odin
gMPEG-2 Transport Packet
110100Vid
eo C
odin
g
MPEG-2 TS
![Page 31: Video Coding 10110100 - Universitetet i oslo · • Container-Format and MM Framework – By Apple Inc. – Originally developed for Macintosh • Start: Charles Wiltgen, 1992](https://reader034.vdocuments.us/reader034/viewer/2022050608/5faf855e0c275721d342a9fc/html5/thumbnails/31.jpg)
110100Vid
eo C
odin
gPSI
• PSI = Programme Specific Information• Programme Map Table (PMT)
– what belongs to a programme?–
110100Vid
eo C
odin
g
PSI
• PSI = Programme Specific Information• Programme Map Table (PMT)• Programme Association Table (PAT)
– which PMT are available?– PID=0
![Page 32: Video Coding 10110100 - Universitetet i oslo · • Container-Format and MM Framework – By Apple Inc. – Originally developed for Macintosh • Start: Charles Wiltgen, 1992](https://reader034.vdocuments.us/reader034/viewer/2022050608/5faf855e0c275721d342a9fc/html5/thumbnails/32.jpg)
110100Vid
eo C
odin
gPSI
PMT: PID=1127 PAT: PID=0 (always)
110100Vid
eo C
odin
g
PSI
• PSI = Programme Specific Information• Programme Map Table (PMT)• Programme Association Table (PAT)• Network Information Table (NIT)
– PMT=0– optional, content not defined by MPEG– channel frequencies– satellite transponder– modulation characteristics– service originator, service name,
![Page 33: Video Coding 10110100 - Universitetet i oslo · • Container-Format and MM Framework – By Apple Inc. – Originally developed for Macintosh • Start: Charles Wiltgen, 1992](https://reader034.vdocuments.us/reader034/viewer/2022050608/5faf855e0c275721d342a9fc/html5/thumbnails/33.jpg)
110100Vid
eo C
odin
gPSI
• PSI = Programme Specific Information• Programme Map Table (PMT)• Programme Association Table (PAT)• Network Information Table (NIT)• Conditional Access Table (CAT)
– scrambling system(s) in use– PID for conditional access management– PIC for entitlement information– not defined by MPEG
110100Vid
eo C
odin
g
• PSI = Programme Specific Information• Programme Map Table (PMT)• Programme Association Table (PAT)• Network Information Table (NIT)• Conditional Access Table (CAT)• Programme Stream Map
– for programme streams
PSI
![Page 34: Video Coding 10110100 - Universitetet i oslo · • Container-Format and MM Framework – By Apple Inc. – Originally developed for Macintosh • Start: Charles Wiltgen, 1992](https://reader034.vdocuments.us/reader034/viewer/2022050608/5faf855e0c275721d342a9fc/html5/thumbnails/34.jpg)
110100Vid
eo C
odin
gPSI
• Programme Map Table (PMT)• Programme Association Table (PAT)• Network Information Table (NIT)• Conditional Access Table (CAT)
110100Vid
eo C
odin
g
MPEG and IP• MPEG is both storage and transport medium• MPEG streams can be transported on IP• UDP packets can be transported on MPEG• Unidirectional application protocols• Broadcast• Multicast• Unicast
![Page 35: Video Coding 10110100 - Universitetet i oslo · • Container-Format and MM Framework – By Apple Inc. – Originally developed for Macintosh • Start: Charles Wiltgen, 1992](https://reader034.vdocuments.us/reader034/viewer/2022050608/5faf855e0c275721d342a9fc/html5/thumbnails/35.jpg)
110100Vid
eo C
odin
gTime
• Multiplexer / Decoder Clock measure time in units of 27 MHz, 42 bit
• Time Stamps expressed in units of 90 kHz, 33 bitProgramme Stream:• System Clock Reference (SCR)
– at least every 0.7 secTransport Stream:• Programme Clock Reference (PCR)
– at least every 0.1 sec
110100Vid
eo C
odin
g
Time
• Presentation Time Stamp (PTS)• Decoding Time Stamp (DTS)
– temporarily stored at DTS, not shown– I and P pictures (accessed for B pictures)– PTS ≤ DTS
![Page 36: Video Coding 10110100 - Universitetet i oslo · • Container-Format and MM Framework – By Apple Inc. – Originally developed for Macintosh • Start: Charles Wiltgen, 1992](https://reader034.vdocuments.us/reader034/viewer/2022050608/5faf855e0c275721d342a9fc/html5/thumbnails/36.jpg)
110100Vid
eo C
odin
gMPEG - I
• Resolution: 360 x 288 x 25 (CIF Eur.)352 x 240 x 30 (CIF US)
• Bitrate ≤ 1,862 Mbit/s• Macroblocks/Frame <= 396• Macroblocks/Second <= 9900
– (396 x 25, 330 x 30)• No Interlace• Audio: 2 Channels (Stereo)
110100Vid
eo C
odin
g
MPEG - II
• Resolutions– low: 360 x 288 (352 x 240)– main: 720 x 596 (704 x 480)– high1440: 1440 x 1152 (HDTV 4:3)– high: 1920 x 1080 (HDTV 16:9)
• Bitrate: 2 – 80 Mbit/s• Interlace Support• 5 Audio-Channels
![Page 37: Video Coding 10110100 - Universitetet i oslo · • Container-Format and MM Framework – By Apple Inc. – Originally developed for Macintosh • Start: Charles Wiltgen, 1992](https://reader034.vdocuments.us/reader034/viewer/2022050608/5faf855e0c275721d342a9fc/html5/thumbnails/37.jpg)
110100Vid
eo C
odin
gData rates
Frametyp
I P B ∅MPEG-1 SIF1.15 Mbit/s
150.000 50.000 20.000 38.000
MPEG-2 6014.00 Mbit/s
400.000 200.000 80.000 130.000
I-Distance : 15P-Distance: 3
110100Vid
eo C
odin
g
H.263
• ITU-T approved standard– “Video coding for low bit rate communication”– v1 1996– v2 1998 = H.263+; focus today– v3 ? H.263++/H.26L
• Unspecified, variable bit rate (<28.8 kbps)• Developed for low delay environments
![Page 38: Video Coding 10110100 - Universitetet i oslo · • Container-Format and MM Framework – By Apple Inc. – Originally developed for Macintosh • Start: Charles Wiltgen, 1992](https://reader034.vdocuments.us/reader034/viewer/2022050608/5faf855e0c275721d342a9fc/html5/thumbnails/38.jpg)
110100Vid
eo C
odin
gH.263 block diagram
CC
T Q
Q
T
P
p
t
qz
q
v
Videoin
Tovideomultiplexcoder
TQP
CCptqzqv
–
–
TransformQuantizerPicture Memory with motion compensated variable delay
Coding controlFlag for INTRA/INTERFlag for transmitted or notQuantizer indicationQuantizing index for transform coefficientsMotion vector
110100Vid
eo C
odin
g
H.263 sampling blocks• 4:2:0 sampling
– luminance Y to chrominance CB, CR
• Block: – 8 x 8 pixels
• Macroblock (MB): – 4 Y + CB + CR blocks
• Group of blocks (GOB): – One or more rows of MBs– In GOB header: resynchronization
Y Y
Y YC
![Page 39: Video Coding 10110100 - Universitetet i oslo · • Container-Format and MM Framework – By Apple Inc. – Originally developed for Macintosh • Start: Charles Wiltgen, 1992](https://reader034.vdocuments.us/reader034/viewer/2022050608/5faf855e0c275721d342a9fc/html5/thumbnails/39.jpg)
110100Vid
eo C
odin
gH.263 resolutions
• Five standardized resolutions– CIF: 352 x 288 (as in H.261)– QCIF: 176 x 144 (as in H.261)– Sub-QCIF: 128 x 96– 4CIF: 704 x 576 (little used?)– 16CIF: 1408 x 1152 (little used?)
• Custom resolutions negotiable– multiple of 4 in both directions
110100Vid
eo C
odin
g
H.263 frames
• Two (six) frame types:– I-frames: intra– P-frames: predictive (inter)– B-frames (optional): bidirectional predicted– PB-frames (optional): decoded B and P
frame as one unit– EI-frames (optional): enhanced I-frame– EP-frames (optional): enhanced P-frame
![Page 40: Video Coding 10110100 - Universitetet i oslo · • Container-Format and MM Framework – By Apple Inc. – Originally developed for Macintosh • Start: Charles Wiltgen, 1992](https://reader034.vdocuments.us/reader034/viewer/2022050608/5faf855e0c275721d342a9fc/html5/thumbnails/40.jpg)
110100Vid
eo C
odin
gH.263 coding
• Spatial redundancy (intra coding):– DCT– Quantisation– Variable length coding (Huffman)
• Temporal redundancy (inter coding):– Motion compensation
• Block-based comparison (MB or block)• Non-integral motion vector values (half-pixel)
– Motion estimation
110100Vid
eo C
odin
g
H.263 coding options (1)
• 16 modes, negotiable at session start:– Efficiency / improved picture quality (10)
• Unrestricted Motion Vector• Syntax-Based Arithmetic Coding• Advanced Prediction!
– Four Motion Vectors per Macroblock– Overlapped Block Motion Compensation
• PB Frame• Advanced Intra Coding• Alternate Inter VLC• Modified Quantization• Deblocking Filter• Improved PB Frame
![Page 41: Video Coding 10110100 - Universitetet i oslo · • Container-Format and MM Framework – By Apple Inc. – Originally developed for Macintosh • Start: Charles Wiltgen, 1992](https://reader034.vdocuments.us/reader034/viewer/2022050608/5faf855e0c275721d342a9fc/html5/thumbnails/41.jpg)
110100Vid
eo C
odin
gH.263 coding options (2)
• Error robustness (3) (lossy channel)– Slice-Structured– Reference Picture Selection– Independent Segment Decoding
• Scalability!– Temporal, signal-to-noise ratio (SNR),
spatial• Reference Picture Resampling• Reduced Resolution Update
110100Vid
eo C
odin
g
H.263 levels
• Preferred combination of supported options (profiles)– Level 1: advanced intra coding, deblocking filter,
full-frame freeze, modified quantization– Level 2: unrestricted motion vector, slice-
structured, reference picture resampling + level 1– Level 3: advanced prediction, improved PB
frames, independent segment decoding, alternate VLC + level 2
![Page 42: Video Coding 10110100 - Universitetet i oslo · • Container-Format and MM Framework – By Apple Inc. – Originally developed for Macintosh • Start: Charles Wiltgen, 1992](https://reader034.vdocuments.us/reader034/viewer/2022050608/5faf855e0c275721d342a9fc/html5/thumbnails/42.jpg)
110100Vid
eo C
odin
gLiterature
• Joan L. Mitchell, William B. Pennebaker, Chad E. Fogg, and Didier J. LeGall: ”MPEG Video Compression Standard”, Chapman&Hill, ISBN 0-412-08771-5, 1996
• Ming-Ting Sun, and Amy R. Reibman:”Compressed Video over Networks”, Marcel Dekker, ISBN 0-8247-9423-0, 2001, Kap. 2
• Wolfgang Leister, Svetlana Boudko, Ole Aamodt, Peter Holmes: ”Digital TV - a survey”, NR Report 988, 2002. http://publications.nr.no/digitv.pdf
110100Vid
eo C
odin
g
Literature
• MPEG System:– P.A. Sarginson: ”MPEG-2: Overview of the
System Layer”, 1996, http://www.bbc.co.uk/rd/pubs/reports/1996-02.pdf
![Page 43: Video Coding 10110100 - Universitetet i oslo · • Container-Format and MM Framework – By Apple Inc. – Originally developed for Macintosh • Start: Charles Wiltgen, 1992](https://reader034.vdocuments.us/reader034/viewer/2022050608/5faf855e0c275721d342a9fc/html5/thumbnails/43.jpg)
110100Vid
eo C
odin
gThe End of this Lecture