multimedia services: video
TRANSCRIPT
Multimedia Services:Video
Sep-2015
Dani Gutiérrez PorsetAssociate Professor
Communications Engineering
Eman ta zabal zazu
2 2Multimedia Services:
Video
Thanks, Licences and Tools
● Thanks to people and organizations who took or take part in free software and free knowledge projects, specially Wikimedia Foundation and KDE
● This presentation is licensed as CC BY-SA 3.0 EShttp://creativecommons.org/licenses/by-sa/3.0/es/
● This presentation has been made with KDE, LibreOffice, Inkscape, Gimp, Chromium, Firefox
3 3Multimedia Services:
Video
Sources and References
● Images from Wikimedia Foundation, if not referenced other source. Logos and trademarks belong to respective organizations
● Texts:
– Wikipedia pages and referenced articles and material– “Guide to Voice and Video over IP” - Sun, Mkwawa, Jammeh,
Ifeachor– “Video over IP” - Wes Simpson– “Computer Networking, a top-down approach” - Kurose, Ross
4 4Multimedia Services:
Video
Index
● Introduction● Codecs and Containers● MPEG● Wires and Connectors
5 5Multimedia Services:
Video
Parameters
● Image + Audio parameters:– Raster Image parameters: size in pixels, colors,...– Audio parameters: sampling rate, bitrate,...
● Frame Rate (fps): Constant or Variable (VFR)● Bitrate: CBR or VBR● Compression: spatial and/or time domains
Introduction
6 6Multimedia Services:
Video
Video scanning:Progressive vs Interlaced. Fields
● Progressive video:
– In all frames each line appears next to previous line
– Used in computer graphics. Easier for software processing.
– e.g. 720p
Introduction
● Interlaced:
– A frame is divided in two fields: odd field (for odd lines) and even field (for even lines). Each time one field is showed
– Used in broadcasting. Permits lower frame rate.
– E.g. 525i, 625i, 1080i
7 7Multimedia Services:
Video
Frame rates and Flicker effect
● Flicker effect: an image needs to be updated >= 50 times/sec. Solutions:– Progressive: repeat the same frame– Interlaced: use frame field
● Typical frame rates:– PAL 50i– NTSC 60i (exactly 29,97 fps interlaced)– Others: 24p for video adquisition, 48p for high quality,...
● i and p letters usually appear near the number of vertical lines, e.g. 720p
Introduction
8 8Multimedia Services:
Video
Raw video
● Also called Uncompressed video and Clean HDMI (no overlaid text or graphics)
● Useful for High quality edition● Very high bitrate and space
e.g. 24bit @ 1080i @ 60fps :24 x 1920x1080 x 60 / 2 = 1.49 Gbps
● If recording to computer, need of RTOS and fast hard disks (e.g. SSD)
Introduction
9 9Multimedia Services:
Video
Video compression
● Types of compression:– Intraframe– Interframe: near frames are correlated. Make a
combination of actual and predicted frame contents
● Techniques:– Spatial correlation– Motion estimation– Motion compensation
Introduction
10 10Multimedia Services:
Video
Motion compensation andMotion estimation
● Describe a picture in terms of the transformation of a reference picture to the current picture
● Supposition: the only difference between one frame and another is the result of either the camera moving or an object in the frame moving
● Aim: determine motion vectors that describe the transformation from a 2D image (whole picture or zones) to another one
Introduction
11 11Multimedia Services:
Video
Codecs and Containers
● Containers:
– Main streams: Video + Audio + Subtitles
– Others: Chapters, Captions, Menus,...● Classifications:
– Codecs: highest supported Bitrate & Resolution, Streaming support,...
– Containers: Video and Audio supported formats,...
http://en.wikipedia.org/wiki/Comparison_of_container_formats
http://en.wikipedia.org/wiki/Comparison_of_video_codecs
http://en.wikipedia.org/wiki/Comparison_of_video_encoders
http://en.wikipedia.org/wiki/Comparison_of_video_converters
http://en.wikipedia.org/wiki/Video_codec
Codecs and Containers
12 12Multimedia Services:
Video
Organizations for standarization
● ISO/IEC MPEG: Moving Picture Experts Group– MPEG* codecs and containers
● ITU VCEG: Video Coding Experts Group: H.26* codecs
● SMPTE: Society of Motion Picture and Television Engineers
● (ISO/IEC JPEG: Joint Photographic Experts Group, from image)
Codecs and Containers
13 13Multimedia Services:
Video
Examples of Standard codecs
● H.261 and H.263: for lower bit rates videoconference. Low resolutions (CIF, QCIF,...)
● MPEG* (see later)● Motion JPEG, Motion JPEG-2000: each video
frame encoded as separate image. Adequate for editing, but higher bitrates than MPEG-2, MPEG-4
Codecs and Containers
14 14Multimedia Services:
Video
Correspondences betweenMPEG and VCEG codecs
● H.261 <-----> MPEG-1 Part 2● H.262 <-----> MPEG-2 Part 2● H.263 <-----> MPEG-4 Part 2 = MPEG-4 Visual
(implemented in DivX, Xvid codecs)● H.264 <-----> MPEG-4 Part 10 = MPEG-4 AVC● H.265 <-----> MPEG-H Part 2
Codecs and Containers
15 15Multimedia Services:
Video
Other Codecs
● Open and free:– Theora, from Xiph.org foundation, initially proposed for HTML5
● Proprietary codecs:– Microsoft WMV codecs: WMV7, WMV8, WMV9 = standard
SMTPE VC1– Real Networks– Sorenson (used in Adobe Flash and in Apple Quicktime
multimedia framework)– On2 Technologies: VP5, VP6, VP8 (used in Adobe Flash), VP9.
Google purchased On2 in 2010
Codecs and Containers
16 16Multimedia Services:
Video
Examples of Standard Containers
● MPEG Program Stream (MPEG PS): MPEG-1 Part 1 and MPEG-2 Part 1– VCD (MPEG-1 PS), SVCD (MPEG-2 PS)– VOB container:
● Used as container format in DVD-Video● Contents: Video, Audio, Subtitles, Menus● Format: Subset of MPEG PS
● MPEG Transport Stream (MPEG TS): MPEG-2 Part 1● MP4: MPEG-4 Part 14
Codecs and Containers
17 17Multimedia Services:
Video
Other Containers
● Open, free:– Ogg from Xiph.org foundation, initially proposed in HTML5– Matroska: admits lot of contents– WebM, derived from Matroska, for use in HTML5
● Proprietary:– Real Media– Apple Quicktime (.mov)– Microsoft AVI, ASF– Adobe Flash FLV– (GIF animation)
Codecs and Containers
18 18Multimedia Services:
Video
DV format
● Specification (codecs, containers,...) to save video and audio, mainly over a FireWire cable (IEE 1394)
● Frequently used to store digital video, e.g. video camera recorders
● Video:– Interlaced, 720x576@50 4:1:1 or 720x480@60 4:2:0– Lossy compression with intraframe DCT
● Audio: no compression
Codecs and Containers
19 19Multimedia Services:
Video
MPEG
● Specification history:– 1990: MPEG-1. Legacy. Uses: VCD– 1996: MPEG-2. Uses: DVD Video, Digital TV, first
HDTVs and Blu-ray– (No MPEG-3)– 2000: MPEG-4– 2003: H.264 = MPEG-4 Part 10 = MPEG-4 AVC
● MPEG LA manages licence fees e.g. for devices
MPEG
20 20Multimedia Services:
Video
MPEG StructureMPEG
21 21Multimedia Services:
Video
Macroblocks, Blocks and Slices
● Macroblocks of 16px x 16px● Block size depending on chroma subsampling, e.g.
– 4:2:0: 4 for luma, 1 for Cr and 1 for Cb block. Used in DVDs– 4:2:2: 4 for luma, 2 for Cr and 2 for Cb
● Type of Blocks:– Transform blocks: to apply DCT calculations– Prediction blocks: to apply motion compensation
● In MPEG-4 AVC macroblocks can be grouped horizontally into slices
MPEG
22 22Multimedia Services:
Video
Picture or Frame types
● I-frames (Intra-coded): coded without references to other frames; same as a static full image. Least compressed. Equivalent to JPEG image
● P-frames (Predicted) or delta-frames: encoded frames with reference to the previous ones. Medium compressed
● B-frames (Bi-predictive): encoded with reference to previous and forward frames. Maximum compression
● D-frames: Old: in MPEG-1 low quality, not referenced by other frames. Used for fast previews
MPEG
23 23Multimedia Services:
Video
Group of Pictures
● A GOP is a Group of a single intraframe (I) and zero or more interframes (P and B). It always begin with an I frame and ends with the last frame before next I frame
● Types: Open GOP if it uses I frame of the next GOP, Closed if it's self-contained
● GOP structure referenced by two numbers:– N = GOP length: distance between two full images (I)– M: distance between two anchor frames (I or P)– Example: N=12, M=3: IBBPBBPBBPBB I
MPEG
24 24Multimedia Services:
Video
Comparison of GOP lengths
● Long, better to reduce bitrate and bandwidth● Short:
– Better Quality, mainly if fast image changes– Less delay– Easier editing
MPEG
25 25Multimedia Services:
Video
MPEG-2
● MPEG-2 = H.262● Commonly used in professional video recording and broadcasting.
Huge installed base of devices (DVD Video, Digital TV)● Video Features:
– Video: interlacing (PAL, NTSC) and HD video (720p, 1080i)– 12 combinations of 6 Profiles (compression techniques) and 4 Levels
(resolutions), that require distinct bandwidth and complexitye.g: DVDs: main level (720x576 in PAL or 720x480 in NTSC) and main profile (color 4:2:0)
● Audio features: 5.1 MPEG-1 and AAC
MPEG
26 26Multimedia Services:
Video
MPEG-4
● Improvements in video compression (specially for IP networks), ~ 50% bandwidth of MPEG-2
● Techniques:– New video objects, created from synthetic sources (raster or
vectorial graphics, text) that use less bandwidth and can be controlled by receptor user
– Variable Macroblock size: smaller for accuracy in some image areas, bigger for less bandwidth
– Fractal compression– Allows B frames based on other B frames
MPEG
27 27Multimedia Services:
Video
MPEG Audio compression
● Lossy compression, from 4:1 in MPEG-1 layer I to 32:1 (AAC+)
● Audio streams of 32 kHz, 44.1 kHz, 48 kHz● Main families:
– MPEG-1 audio Layers: 3 versions: I, II, III (“mp3”)– Advanced Audio Coding (AAC ), from MPEG-2. There
are 3 versions, e. g. AAC: up to 48 channels, lossless compression mode, audio synthetic objects (e. g. MIDI)
MPEG
28 28Multimedia Services:
Video
Video hardware:wires and connectors
● Each specification defines the protocols, signals, electrical interfaces and mechanical structure
● For video and/or audio● For analog and/or digital signals, e.g.
– Analog: VGA– Digital: HDMI, IEEE 1394=Firewire (similar to USB but faster)
● Each one supports a maximum spatial resolution and frequency, e.g.– VGA: QXGA (2048 × 1536) @ 85– HDMI: 4K (4096 × 2160) @60
Wires and Connectors
http://en.wikipedia.org/wiki/List_of_video_connectors
29 29Multimedia Services:
Video
Video hardware:Analog wires and connectors
● Composite Video: 1 conductor with all info● S-Video (Separate Video): 2 conductors separating luma
and chroma● Component Video: 3 conductors, for RGB or YUV/YPbPr.
For sync signal 3 choices: on green (Y), 1 conductor for both V and H sync, 2 conductors for separate V and H sync
● VGA: carries R, G, B and H, V syncs● SCART: audio (stereo) + video (composite and S-Video)
Wires and Connectors
30 30Multimedia Services:
Video
Video hardware:HDMI wire and connector
● High-Definition Multimedia Interface● Features:
– Digital– Uncompressed video in RGB or YCbCr– Digital Content Protection: before sending info verifies the remote device is
authorized, and encrypts the info– No subtitles support
● Distinct supported A/V capabilities depending on HDMI version:– For Video: resolution (up to 4K), fps, color spaces, chroma subsamplings, color
depth, aspect ratios, multi streaming (up to 2 streams)– For Audio: no. of channels (up to 32), sample sizes and rates, formats (Dolby
Digital, DTS,...), multi streaming (up to 4 streams)http://en.wikipedia.org/wiki/HDMI
Wires and Connectors