mpeg-4 avc/h.264
TRANSCRIPT
International Telecommunication UnionInternational Multimedia Telecommunications Consortium
Joint ITU-T Workshop and IMTC Forum 2006 “H.323, SIP: is H.325 next?“San Diego, 9-11 May 2006
ITU-T
MPEGMPEG--4 AVC/H.264 4 AVC/H.264 Digital Video Compression StandardDigital Video Compression Standard
Ajay LuthraAdv. Tech., Motorola [email protected]
Joint ITU-T Workshop and IMTC Forum 2006 “H.323, SIP: is H.325 next?“San Diego, 9-11 May 2006 2
ITU-T
MPEG-4 AVC/H.264
o Historyo Technologyo Adoptiono Next Steps
o Start drinking from the fire hose
Joint ITU-T Workshop and IMTC Forum 2006 “H.323, SIP: is H.325 next?“San Diego, 9-11 May 2006 3
ITU-T
MPEG-4 AVC / H.264
ISO (MPEG)
ITU(VCEG)
JVT(Common Standard)
MPEG-4 AVC(Part 10)
H.264 / H.26L
Joint Video Team(Dec 2001)
Joint ITU-T Workshop and IMTC Forum 2006 “H.323, SIP: is H.325 next?“San Diego, 9-11 May 2006 4
ITU-T
AVC/H.264
…… Half the bit rate of MPEGHalf the bit rate of MPEG--22Better picture Better picture ……
Joint ITU-T Workshop and IMTC Forum 2006 “H.323, SIP: is H.325 next?“San Diego, 9-11 May 2006 5
ITU-T
o Objective• PSNR vs Bit Rate
o Subjective
AVC/H.264 Video Quality Tests
Joint ITU-T Workshop and IMTC Forum 2006 “H.323, SIP: is H.325 next?“San Diego, 9-11 May 2006 6
ITU-T
2022242628303234363840
0 500 1000 1500 2000 2500 3000 3500 4000Bitrate (kb/s)
PSN
R (d
B)
MP4 AVC (N=15, M=3)
MP2 (N=15, M=3)
MP4 ASP (N=15, M=3)
Note: PSNR at a given bit rate is encoder dependent and will vary from one encoder to another
MPEG-2, MPEG-4 ASP, AVC/H.264 (MP)Mobile & Calendar (CIF)
Joint ITU-T Workshop and IMTC Forum 2006 “H.323, SIP: is H.325 next?“San Diego, 9-11 May 2006 7
ITU-T
24
26
28
30
32
34
36
38
40
0 500 1000 1500 2000 2500 3000Bitrate (kb/s)
PSN
R (d
B)
MP4 AVC (N=15, M=3)
MP2 (N=15, M=3)
MP4 ASP (N=15, M=3)
MPEG-2, MPEG-4 ASP, AVC/H.264 (MP)BUS (CIF)
Joint ITU-T Workshop and IMTC Forum 2006 “H.323, SIP: is H.325 next?“San Diego, 9-11 May 2006 8
ITU-T
22
24
26
28
30
32
34
36
0 1000 2000 3000 4000 5000 6000Bitrate (kb/s)
PSN
R (d
B)
AVC (N=15, M=3)MP2 (N=15, M=3)MP4 (N=15, M=3)
MPEG-2, MPEG-4 ASP, AVC/H.264 (MP)Mobile & Calendar (HHR - 352x480)
Joint ITU-T Workshop and IMTC Forum 2006 “H.323, SIP: is H.325 next?“San Diego, 9-11 May 2006 9
ITU-T
M obil and Calendar (720x480)
24
26
28
30
32
34
36
400 2400 4400 6400 8400
Bitrate (kb/s)
PSNR
(dB)
MPEG-4 AVCMPEG 2
AVC/H.264 (MP) vs MPEG-2 (MP)
Joint ITU-T Workshop and IMTC Forum 2006 “H.323, SIP: is H.325 next?“San Diego, 9-11 May 2006 10
ITU-T
Fl_HD (1920x1088)
28
29
30
31
32
33
34
35
5000 9000 13000 17000
Bitrate (kb/s)
PSN
R (d
B)
MPEG-4 AVCMPEG 2
AVC/H.264 (MP) vs MPEG-2 (MP)
Joint ITU-T Workshop and IMTC Forum 2006 “H.323, SIP: is H.325 next?“San Diego, 9-11 May 2006 11
ITU-T~ % Bit rates required for the same PSNR ( ~ 32dB)
(Normalization: MPEG-2 = 100%)
6275
37
68
5033
100 100 100
0
20
40
60
80
100
120
MPEG-2 MPEG-4 ASP AVC/H.264
% Mobil & Calendar (CIF)
Mobil & Calendar (HHR)Bus (CIF)
Note: PSNR at a given bit rate is encoder dependent and will vary from one encoder to another
MPEG-2, MPEG-4 ASP, AVC/H.264 (MP)
Joint ITU-T Workshop and IMTC Forum 2006 “H.323, SIP: is H.325 next?“San Diego, 9-11 May 2006 12
ITU-T
Subjective Testing
o MPEG Tests• AVC/H.264 (Main Profile), MPEG-2 (Main Profile)
o Blu-Ray Disc Founders (BDF)• AVC/H.264 (High Profile), MPEG-2 (Main Profile)
Joint ITU-T Workshop and IMTC Forum 2006 “H.323, SIP: is H.325 next?“San Diego, 9-11 May 2006 13
ITU-T
MPEG Verification Test
o ITU-R 500-11• Double Stimulus Continuous Quality Scale
(DSCQS) Method
Seq. B Seq. A Seq. B
A B A* B*
Seq. A
Vote 1
2 s 10 s 2 s 10 s 2 s 10 s 2 s 10 s 4 s 1 s
Seq. B Seq. A Seq. B
A B A* B*
Seq. A
Vote 2
2 s 10 s 2 s 10 s 2 s 10 s 2 s 10 s 4 s 1 s
High
20
Low
40
60
80
1A 1B 2A 2B
- Mean Opinion Square and 95% Confidence Interval
Joint ITU-T Workshop and IMTC Forum 2006 “H.323, SIP: is H.325 next?“San Diego, 9-11 May 2006 14
ITU-T
Mobile & Calendar (SD)
1
2
3
4
5
AVC_4M
AVC_6M
AVC_3M
AVC_2M25
AVC_1M5
HiQ_6
M
HiQ_4
M
TM5_6M
HiQ_2
M25
TM5_4M
HiQ_3
M
TM5_3M
TM5_2M
25MPEG Verification Test
Joint ITU-T Workshop and IMTC Forum 2006 “H.323, SIP: is H.325 next?“San Diego, 9-11 May 2006 15
ITU-THarbour 720(60p)
1
2
3
4
5
AVC_10M AVC_20M TM5_20M TM5_10M HiQ_20M AVC_6M HiQ_10M TM5_6M HiQ_6M
MPEG Verification Test
Joint ITU-T Workshop and IMTC Forum 2006 “H.323, SIP: is H.325 next?“San Diego, 9-11 May 2006 16
ITU-T
New Mobile & Calendar 1080(30i)
1
2
3
4
5
AVC_10M AVC_20M HiQ_20M TM5_20M TM5_10M HiQ_10M
MPEG Verification Test
Joint ITU-T Workshop and IMTC Forum 2006 “H.323, SIP: is H.325 next?“San Diego, 9-11 May 2006 17
ITU-T
Blu-Ray Disc Founders (BDF)
o HDTV Subjective Testing
Joint ITU-T Workshop and IMTC Forum 2006 “H.323, SIP: is H.325 next?“San Diego, 9-11 May 2006 18
ITU-T
o High at 8 Mbps nominally beats MPEG-2 at 24 Mbpso Nominally transparent on 1080p24 at 16 Mbps
H.264/AVC FRExt 8Mbps
Original MPEG-224 Mbps
Mean Score
2
2.5
3
3.5
4
4.5
5
H.264/AVC FRExt
12Mbps
H.264/AVC FRExt
16Mbps
H.264/AVC FRExt
20Mbps
3.65 3.714.00
4.03 3.90
3.59
Blu-Ray Disc FoundersJVT_L033 - High Profile Tests (HDTV)
Joint ITU-T Workshop and IMTC Forum 2006 “H.323, SIP: is H.325 next?“San Diego, 9-11 May 2006 19
ITU-T
AVC/H.264
…… Half the bit rate of MPEGHalf the bit rate of MPEG--22
Better picture Better picture ……
Joint ITU-T Workshop and IMTC Forum 2006 “H.323, SIP: is H.325 next?“San Diego, 9-11 May 2006 20
ITU-To I, P, B, Bs, SP, SIo Improved Spatial Prediction
• 4x4• 8x8• 16x16
o Improved Temporal Prediction• Multiple reference frames• Variable block size MC - 7 Block Patterns
— 16x16, 16x8, 8x16, 8x8, 8x4, 4x8, 4x4• Up to 16 MVs per MB• Quarter Pixel Interpolation• Loop filter• Skip• Direct• Weighted
o Interlaced Video Coding Tools• Frame or Field predictions
o Context-based Adaptive Binary Arithmetic Coding (CABAC)o 4x4 or 8x8 Transformso Other
AVC / H.264
Joint ITU-T Workshop and IMTC Forum 2006 “H.323, SIP: is H.325 next?“San Diego, 9-11 May 2006 21
ITU-T
AVC/H.264 Coding ToolsMultiple Reference Frames
Current Picture
Decoded Picturesas Reference
Joint ITU-T Workshop and IMTC Forum 2006 “H.323, SIP: is H.325 next?“San Diego, 9-11 May 2006 22
ITU-T
AVC/H.264 Coding Tools
o Variable macroblock size
16
16
16
16
8
8
4 4 4 4MPEG-4AVC/H.264
MPEG-2
Joint ITU-T Workshop and IMTC Forum 2006 “H.323, SIP: is H.325 next?“San Diego, 9-11 May 2006 23
ITU-T
o Variable macroblock size
AVC/H.264 Coding Tools
Joint ITU-T Workshop and IMTC Forum 2006 “H.323, SIP: is H.325 next?“San Diego, 9-11 May 2006 24
ITU-T
AVC/H.264 Coding Tools
Legend: QCIF; Black Boundary Line – Skip; White – Intra; Gray - Inter (P)
Joint ITU-T Workshop and IMTC Forum 2006 “H.323, SIP: is H.325 next?“San Diego, 9-11 May 2006 25
ITU-T
AVC/H.264
o Motion estimation accuracy• ¼ Pixel
Joint ITU-T Workshop and IMTC Forum 2006 “H.323, SIP: is H.325 next?“San Diego, 9-11 May 2006 26
ITU-T
Built-in (In Loop) Deblocking Filter
No Deblocking Deblocking
Joint ITU-T Workshop and IMTC Forum 2006 “H.323, SIP: is H.325 next?“San Diego, 9-11 May 2006 27
ITU-T
+FMO+ASO+Red. Slices
AVC/H.264 Profiles
BaselineMainExtended
+ CABAC
+ SP, SI+ Data
Partitioning
+ B+ MBAFF
• I , P• CAVLC
High+ 8x8
Joint ITU-T Workshop and IMTC Forum 2006 “H.323, SIP: is H.325 next?“San Diego, 9-11 May 2006 28
ITU-T
AVC/H.264 Profiles
o High Profile• Highest compression or video quality at a given bit rate• Suitable for good quality entertainment video distribution
o Baseline Profile• Least complexity• Error resilient • Suitable for telephony, conferencing application
Joint ITU-T Workshop and IMTC Forum 2006 “H.323, SIP: is H.325 next?“San Diego, 9-11 May 2006 29
ITU-TLevel 1.0: QCIF @ 15frames/secLevel 1.1: QCIF @ 30 frames/sec, CIF @7.5 frames/secLevel 1.2: CIF @ 15 frames/secLevel 2.0: CIF @ 30 frames/secLevel 2.1: HHR @ 25 or 30 frames/secLevel 2.2: SDTV @ 15 frames/secLevel 3.0: SDTV: 720x480x30i, 720x576x25i
10 Mbps (max.), up to 5 (max. resolution) reference framesLevel 3.1: HDTV - 1280x720x30p, SVGA (800x600) 50+p Level 3.2: HDTV - 1280x720x60pLevel 4.0: HDTV (all formats) - 1920x1080x30i, 1280x720x60p, 2kx1kx30p
20 Mbps (max.), up to 4 (max. resolution) reference framesLevel 4.1: HDTV - 1920x1080x30i, 1280x720x60p, 2kx1kx30p
50 Mbps, up to 4 (max. resolution) reference framesLevel 4.2: S-HDTV - 1920x1080x60pLevel 5.0: S-HDTV/D-Cinema – 2kx1kx72p Level 5.1: S-HDTV/D-Cinema – 2kx1kx120p, 4kx2kx24p, 4kx2kx30p
AVC/H.264 – Levels
< 64 kbps
< 250 Mbps
> 5000 x
Joint ITU-T Workshop and IMTC Forum 2006 “H.323, SIP: is H.325 next?“San Diego, 9-11 May 2006 30
ITU-T
Transport of AVC / H.264
o Transport of MPEG-4 AVC using MPEG-2 System: ISO/IEC 13818-1 • PDAM (Proposed Draft AMendment) in May 2002• FPDAM (Final Proposed Draft AMendment) in Dec 2002• FDAM in July 2003• Approved AMD
o IP delivery• MPEG-2 TS over UDP/IP, or• RTP over IP
Joint ITU-T Workshop and IMTC Forum 2006 “H.323, SIP: is H.325 next?“San Diego, 9-11 May 2006 31
ITU-T
AVC/H.264
o Applications• Wherever you need to save bandwidth and/or storage capacity
Joint ITU-T Workshop and IMTC Forum 2006 “H.323, SIP: is H.325 next?“San Diego, 9-11 May 2006 32
ITU-T
AVC/H.264 Adoption
o 3GPP o 3GPP2 o ARIB (Japan) o ATSC o Blu-ray Disc o DLNA o DMB (Korea) o DVB o DVD Forum (HD-DVD)o IETF AVT - RTP payload spec
approved as RFC 3984o ISMA o SCTEo US DoD MISB - Adopted as US
government preferred codec up to 1080p
Joint ITU-T Workshop and IMTC Forum 2006 “H.323, SIP: is H.325 next?“San Diego, 9-11 May 2006 33
ITU-T
o Direct Broadcasting Satellite• DirectTV, BSkyB, DISH, Premiere, Euro 1080
o Terrestrial broadcasting• HDTV pay DTV in France
o IPTV• KPN in Netherlands, Belgacom in Belgium, SBC/ATT …
o Streaming• Quicktime
o Video Conferencingo Mobile TVo Portable Media Players
• Video iPOD, Cell Phoneso HD-DVDo Blu-Rayo Video Gameso Digital Cable
AVC/H.264 Commercial Deployment
Joint ITU-T Workshop and IMTC Forum 2006 “H.323, SIP: is H.325 next?“San Diego, 9-11 May 2006 34
ITU-T
o Products• NAB 2006 : 125+ companies
— Services— Encoders— Decoders— STB— ICs— Cores— Software— …
AVC/H.264 Commercial Deployment
Joint ITU-T Workshop and IMTC Forum 2006 “H.323, SIP: is H.325 next?“San Diego, 9-11 May 2006 35
ITU-T
o Extensions• Higher resolution and quality
— Studios, D-cinema— Very High Definition to home?
AVC/H.264 Standard
Joint ITU-T Workshop and IMTC Forum 2006 “H.323, SIP: is H.325 next?“San Diego, 9-11 May 2006 36
ITU-T
AVC/H.264 Profiles
o Higher Resolution and Quality— 4:2:0/10bits – High 10 bit (Hi10P)— 4:2:2/10bits – High 4:2:2 (Hi422)
Joint ITU-T Workshop and IMTC Forum 2006 “H.323, SIP: is H.325 next?“San Diego, 9-11 May 2006 37
ITU-T
o Extensions• Higher resolution and quality
— Studios, D-cinema— Very High Definition to home?
• Scalable Video Coding (SVC)— Multiple layers— Potential alternative for seamless mobility
o Across networkso Across deviceso Across applications
AVC/H.264 Standard - Next Steps
Joint ITU-T Workshop and IMTC Forum 2006 “H.323, SIP: is H.325 next?“San Diego, 9-11 May 2006 38
ITU-T
Scalability
Single Layer Multiple Layers
Joint ITU-T Workshop and IMTC Forum 2006 “H.323, SIP: is H.325 next?“San Diego, 9-11 May 2006 39
ITU-T
Scalability
o Temporalo Spatialo SNRo Fine Granular (FGS)
Joint ITU-T Workshop and IMTC Forum 2006 “H.323, SIP: is H.325 next?“San Diego, 9-11 May 2006 40
ITU-T
o Extensions• Higher resolution and quality
— Studios, D-cinema— Very High Definition to home?
• Scalable Video Coding (SVC)— Multiple layers— Potential alternative for seamless mobility
o Across networkso Across deviceso Across applications
• Multi-View Coding
AVC/H.264 Standard - Next Steps
Joint ITU-T Workshop and IMTC Forum 2006 “H.323, SIP: is H.325 next?“San Diego, 9-11 May 2006 41
ITU-T
AVC/H.264 Standard - Next Steps
o Extensions• Higher resolution and quality
— Studios, D-cinema— Very High Definition to home?
• Scalable Video Coding (SVC)— Multiple layers— Potential alternative for seamless mobility
o Across networkso Across deviceso Across applications
• Multi-View Coding
o Next standard• Significant improvement over AVC/H.264 ?
— H.265?— MPEG-x ?
• Hunt is ON