h.256 presentation

WBU-ISOG FORUM 27 28 November 2012

HEVC: High-Efficiency Video CodingNext generation video compression

Jérôme VIERON Advanced Research Manager

H.264/AVC HEVC

Confidential 2Keep content looking great wherever it’s played

Jérôme Viéron - November 20121080p@25fps at 2.7 Mbits/s

Outline

1. HEVC in a nutshell2. HEVC status: standardization and market3. Technical description4. Compression performance5. HEVC and Interlaced coding6. ATEME & HEVC


Jérôme Viéron - November 2012

Outline




HEVC in a nutshell

● Context• HEVC: High Efficiency Video Coding

J i t t d d f ISO IEC/MPEG d ITU T/VCEG JCTVC • Joint standard of ISO-IEC/MPEG and ITU-T/VCEG: JCTVC • Successor of H.264/MPEG AVC• ITU- H.265 and ISO- MPEG H Part 2

● GoalsAchieve a compression gain of 50% over H 264/AVC • Achieve a compression gain of 50% over H.264/AVC

• x10 complexity max for encoder and x2/3 max for decoder

● Status• Gains are already obtained • Standard will be finalized January 2013 (First version)



• Standard will be finalized January 2013 (First version)

HEVC: Standardization status

• Strong industrial and academic interests • Up to1000 contributions per meeting (and still 400 at last meeting)

M i d t i l t i l d ( th H 264/AVC)• Many industrial actors involved (more than H.264/AVC)



HEVC: Standardization status

● Standardization Timeline• July 2012: DIS (Draft International Standard)

J 2013 FDIS (Fi l D ft I t ti l St d d)• January 2013: FDIS (Final Draft International Standard)• Extensions /amendments are considered for mid-2013 –mid 2014:

Scalable, 3D, 4:X:X, bit-depths >10 bits, color , (Interlace ?)…

● On-going standardization processProfiles definitions/refinement : Main 10 bits Still picture• Profiles definitions/refinement : Main, 10 bits, Still picture

• Standard draft text V9 (JCTVC-K1003-v9)

• HM 9: reference software (http://hevc.kw.bbc.co.uk/trac/browser)

• Further tools investigations • 3D activity in MPEG and Scalable activity in JCTVC



HEVC: Transport/System Layer Integration

● On going definitions (MPEG, IETF,…,DVB): benefit from H.264/AVC● MPEG Media Transport (MMT) ?



HEVC: Potential applications

What becomes possible with 50% video rate reduction?

● Existing applications and usage scenariosg pp g• IPTV over DSL : Large shift in IPTV eligibility• Facilitated deployment of OTT and multi-screen services

More customers on the same infrastructure: most IP traffic is video• More customers on the same infrastructure: most IP traffic is video• More archiving facilities

● Future services• 1080p60/50 with bitrates comparable to 1080i • Immersive viewing experience: Ultra-HD (4K 8K)• Immersive viewing experience: Ultra HD (4K, 8K)• Premium services (sports, live music, live events,…): home theater,

Bars venue, mobile HD 3DTV Full frame per view at today’s HD delivery rates



• HD 3DTV Full frame per view at today s HD delivery rates

HEVC: Market Timeline

● Tentative Timeline?

2010 2011 2012 2013IS early 2013

2014 2015

HEVC servicesHEVC

HEVC Extensions

Offline S/W enc.S/W dec.

Catchup TV - VOD

Live H/W enc.IPTV-Broadcast

Live S/W enc.H/W dec.WebTV -

IPTVHEVC Extensions(3D, Scalability, Professional)

Extensions services



Outline




HEVC: Yet Another Hybrid Codec?

● Main structure very similar to conventional hybrid codecs

Confidential 12

● But many differences in terms of flexibility: significant gain

Coding Structure: H.264/AVC vs HEVC

H 264/AVCH.264/AVC HEVC

16x16 Macroblock Coding Unit64x64 to 8x864x64 to 8x8

Quadtree coding structureBlock coding structure

3 Intra partitioning

4 Inter partitioning

PU

TU+4 sub-partitioning 8x8

2 Transform sizes: 4x4 8x8

Prediction Unit and Transform Unit partitioning

Multiples sizes/forms: 64x64 to 4x4

TU

4x4, 8x8

Up to 9 Intra prediction directions 35 Intra prediction directions

17 16 15 14 13 12 11 10 9 8 7 6 5 4 3 2



Efficient spatio-temporal mv prediction

Coding Structure: CU and PU

● Coding Unit (CU)• 8x8 to 64x64

● Prediction Unit (PU)• Intra:

Square: 2Nx2N, NxNSqua e: ,

Inter:• Inter: 2Nx2N, NxN, dNxN, NxdN.AMP (Asymmetric Motion Partitioning) AMP



Coding Structure: TU

● Transform Unit (TU) • DCT (Discrete Cosine Tranform) • Various sizes from 4x4 to 32x32• To adapt the transform to the signal frequency characteristics

TU can overlap PU• TU can overlap PU

• Intra- DCT+ DST (Discrete Sine Transform) on Luma intra 4x4

• InterDCT only- DCT only



On Quadtree structure representation and coding

● Quadtree• Clean and efficient structure for multi-partitioning codecs

Ch i li i• Cheap signalization• Fast decision algorithm well-known (pruning)

01 2 PU

3 4

5 69

0

1 2 3 47 810 1112 13

1415

TU



Intra prediction (1/2)

● Luma: 35 prediction directions (33 + Planar + DC)18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34

H.264/AVC HEVC

0 : Intra_Planar1 : Intra_DC35: Intra_FromLuma

● Planar predictor:

H.264/AVC HEVC

● Planar predictor:



Intra prediction (2/2)

● 3 Most Probable Modes (MPM): derived from neighboring

● Three coefficient scanning orders depending on the mode

Diagonal up-right Horizontal Vertical



Inter prediction (1/2)

● Adaptive Motion Vector Prediction: AMVP• Motion vector predictor competition (spatial and temporal)

3 MV candidates• 3 MV candidates

Fig. 1 AMVP

● Merge● « Motion copy » mode + Residual pixel coding● Correction of Quadtree redundancies● Correction of Quadtree redundancies• Extension of H.264/AVC Direct mode• 5 PU candidates (spatial and temporal)

Ski h id lFig. 2 M



• « Skip » when no residual Merge

Inter prediction (2/2)

● 7 or 8-tap interpolation filter for luma: ¼ Pel

4 t i t l ti filt f h P l● 4-tap interpolation filter for chroma: 1/8 Pel



Filtering

● Deblocking Filter● Along CU boundaries as in H.264/AVC

Parallelizable● Parallelizable

● Sample Adaptive Offset: SAO● After deblocking● Add offset to pixels depending on their categorization (band, edge)● Up to 6% bitrate savings● Up to 6% bitrate savings.



High level parallelism tools

● Slices● Wavefront Parallel processing: WPP

● 1 thread per CU line X

X 1% loss compared to a

X

X

Pixel and MV dependency Probabilties dependency

CABAC probabilitites KEY

1% loss compared to a non parallel bistream

● Tiles

1thread per Tile

Probabilties dependency

X Block(s) being encoded

1 2 3 10 11 16 17 182 7% loss compared to a

● 1thread per Tile

Example of Frame partitioning:

4 5 6 12

7

25

8

26

9

27

14

31

13 19 20 21

15

32

22

35

23

36

24

37

28 29 30 33 34 38 39 40

2,7% loss compared to a non parallel bistream



p p g

41 42 43 44 45 46 47 48

Other tools and features

● One unique entropy coder: CABAC with a more sophisticated context management scheme

• Improved compression efficiencyp p y• Improved throughput speed while reducing its context memory requirements

● High Level Syntax: (very close to H 264/AVC)● High Level Syntax: (very close to H.264/AVC)• SPS, PPS (Sequence/Picture Parameter Set) and new VPS (Video Parameter Set)• NAL (Network Abstraction Layer)

SEI (Supplemental Enhancement Information ) metadata• SEI (Supplemental Enhancement Information ) metadata

• VUI (Video Usability Information) metadata

● No interlace coding tool● Field coding only● Only SEI message « Picture timing » (pic_struct)



HEVC toolset versus H.264/AVC and MPEG-2MPEG 2 H 264/AVC HEVCMPEG-2 H.264/AVC HEVC

Partition size Macroblock 16x16 Macroblock 16x16 (Large) Coding Unit8x8 to 64x64

Partitioning Inter 16x8, Intra 8x8 Sub-block down to 4x4

Prediction UnitQuadtree down to 4x44x4 Quadtree down to 4x4Square, symmetric and

asymmetric(only square for intra)

Transform Floating DCT Integer DCT8x8 4x4

Transform Unit square IDCT from 32x32 to 4x48x8, 4x4 IDCT from 32x32 to 4x4+ DST Luma Intra 4x4

Intra prediction DC predictor Up to 9 predictors 35 predictors

Motion prediction Vector from one Spatial Median Advanced Motion Neighbor (3 blocks) Vector Prediction AMVP

(spatial + temporal)

« Motion-copy » mode

/ Direct mode Merge mode

Motion precision ½ Pixel bilinear ½ Pixel 6 tap ¼ Pixel 7or 8 tapMotion precision ½ Pixel bilinear ½ Pixel 6-tap,¼ Pixel bi-linear

¼ Pixel 7or 8-tap,1/8 Pixel 4-tap chroma

Entropy coding VLC CABAC, CAVLC CABAC

Filters /Deblocking Filter

Deblocking FilterSample Adaptive Offset



g p p

Main Profile: Levels and Tiers



Outline




HEVC compression performance

● Comparison with H.264/AVC● JM 18.3 and 18.4 (High Profile) vs HM 7 and 8 (Main Profile)

WVGA to Full HD video sequences● WVGA to Full HD video sequences

● Objective quality (PSNR-Bjontegaard)• Average bitrate savings of 35% for entertainment applications (RA)• Coding performance increases with resolutions: >39% for HD and beyond• Average bitrate savings of 40 % low delay applications (Low delay)• Confirmed with ATEME’s in-house encoders

● Subjective quality (Mean Opinion Score)● Subjective quality (Mean Opinion Score)• Average bitrate savings of 50% for equivalent perceived quality (RA) • From 30% up to 67% on certain 1080p content



HEVC compression performance for 4K broadcast

● Performance assessment on 4K native content• Both H.264/AVC and HEVC encoders (demonstrated at IBC 2012)

CBR same GOP structure and comparable configuration• CBR, same GOP structure and comparable configuration• 4:2:0 sampling and 8-bit material

4K t t● 4K content• Digital Cameras: Red EPIC, JVC, Sony F65 • Captures performed by FranceTelevisions and Orange: 4EVER consortium• Scanned films: SVT reference sequences

● 38% to 50% bitrate savings observed (Bjontegaard)38% 50% g ( j g )• Further results in SMPTE technical paper October 2012 • Observed gain would allow the broadcast of 4KTV (50/60p) at video

bitrates below 13Mbps (Confirmed by subjective testing)



bitrates below 13Mbps (Confirmed by subjective testing)

Outline




HEVC and Interlaced

● So far, interlaced only supported with SEI message ● Pic_struct (AVC like) => « Picture timing » SEI message

Assuming field only coding ● Assuming field only coding ● No specific coding tool (neither PAFF or MBAFF)

L i t l d id t t t till l t ● Legacy interlaced video content represent still a very large part: Low-Level Interlaced coding tools support was expressed

● Contributions from Broadcasters and service providers ● Contributions from National Bodies (US, Japan, France,…)● EBU Liaison to MPEG in last Shanghai meeting (October 2012)

● Contributions reported assessment of HEVC interlaced coding● Limited performance against H.264/AVC were highlighted

Major issue regarding Chroma misalignment



● Major issue regarding Chroma misalignment

HEVC and Interlaced

● MPEG AHG on Study of interlace coding● Created for next 6 months (http://lists.uni-klu.ac.at/mailman/listinfo/interlace)

● Mandates:● Define sequences test set● Highlight HEVC interlaced use casesg g● Evaluate current HM performance over H.264/AVC● Evaluate improvement (normative and non-normative)

Interlaced profile for 2014?Interlaced profile for 2014?

● Participants:ATEME ( h i ) C t C bl L b M t l Z ● ATEME (co-chair), Comcast, CableLabs, Motorola, Zenverge, Sony, Mitsubishi, Orange, Technicolor, Harmonic, NBC, DirecTv, …

U t d f id !



● Urgent need for video sequences!

Conclusion

● Overview of future HEVC video compression standard● Performance is more than promising: 50% bit-rate savings on HD and beyond

● First version will be delivered in January 2013 (up to 10 bits, 4:2:0)● HEVC extensions are already considered for next year : Scalable, 3D, professional

● Interlaced coding not fully supported (only SEI message)

● ATEME is working on its own HEVC encoding core technology targeting demos and field operation for H1 2013

● French research consortium 4EVER for Enhanced Video ExpeRience● French research consortium 4EVER – for Enhanced Video ExpeRience● 3 year collaborative project multi-million project• Research, develop and promote an Enhanced quality Television Experience.

HEVC technology development and codec prototyping



- HEVC technology development and codec prototyping- Ultra High Definition - 4KTV, format definition and qualification.

M i Th k YMerci, danke, gracias, grazie, xièxie, chokrane, Thank You,dankie, faleminderit, Barak Allahu fiik, chnorakaloutioun, çox sag olun,a ni kié, bedankt, waita, eskerrik, dhanyabaad, dziakuju, thint ko, kyayt ti t i kié t é é bl d i à i l t k htzu tin pa te, a ni kié, trugéré, blagodaria, gràcies, salamat, kam sahhamnida, grazie,mèsi, tak, dankon, akpé, kiitos, gracie,tapadh leat,go raibh maith agat, diolch, efharisto, aguyjé, meherbani, mahalo, toda,tau, dhanyavad, köszönöm, terima kasih, takk, arigatô, tanemirt, hvala,, y , , , , g , , ,dhanyavadagalu, akun, murakoze, sobodi, tenki, spas, mercé, khobchai, gratias ago, paldies, choukrane, aciu, blagodaram, terima kasih,misaotra, nizzik hajr, aabhari aahe, bayarlalaa, dank u wel, mercé,tack shukriya danki motashakkeram mamnun dzikuj obrigadotack, shukriya, danki, motashakkeram, mamnun, dzikuj, obrigado,dakujem, multumesc, spacibo, marci, faafetai lava, gratzias, ahsante,salamat po, mauruuru, nanedri, rahmat, dkuji, kop khun krap, yekeniele,tesekkur ederim, sagolun, diakuiu, cám Ön, grces, djiere dieuf,enkosi, a dank, ngiyabonga, siyabonga, ngiyabonga kakhulu

Jérôme VIERON Advanced Research [email protected] @

h.256 presentation

Documents

hevc status

st d d

playedjrme viron

d activity

d th h

mpeg avc itu

nutshell context hevc

avc hevcconfidential