MPEG offer for 3D Graphics Compression

Web3D Conference, June 2013@San Sebastien

Marius PREDA

Institut MINES Telecom, France

MPEG 3DG Chairman

Hard problem Data heterogeneity

Scene Geometry Attributes Animation

Different Representation Methods Different Application Requirements

3D Graphics Compression

A good compression method is agnostic to the signal but exploits the signal nature

3DG Comp: N dimensional problemImage and Video is easy, only color

MPEG 3DG Technologies

MPEG-1/2(AV content)

1992/4

VRML

1997

• Part 11 - BIFS: -Binarisation of VRML -Extensions for streaming -Extensions for server command -Extensions for 2D graphics - Real time augmentation with audio & video• Part 2 - Visual: - 3D Mesh compression - Face animation

1998

• Part 2 – Visual - Body animation

1999

MPEG-4 v.1

MPEG-4 v.2

First form of broadcast signal augmentation

MPEG 3DG Technologies

MPEG-4

2003

•AFX 2nd Edition: - Animation by morphing - Multi-texturing

2005

• AFX 3rd Edition - WSS for terrain and cities - Frame based animation

2007

MPEG-4

MPEG-4

• Part 16 - AFX: - A rich set of 3D graphics tools - Compression of geometry, appearance, animation

• AFX 4th Edition - Scalable complexity mesh coding

2011

MPEG-4A rich set of 3D Graphics representation and compression tools

MPEG 3DG Technologies

MPEG-4

2003

•AFX 2nd Edition: - Animation by morphing - Multi-texturing

2005

• AFX 3rd Edition - WSS for terrain and cities - Frame based animation

2007

MPEG-4

MPEG-4

• Part 16 - AFX: - A rich set of 3D graphics tools - Compression of geometry, appearance, animation

• AFX 4th Edition - Scalable complexity mesh coding

2011

MPEG-4

2009

• Part 25 - Compression of third-party XML (X3D, COLLADA)

MPEG-4

2004

• Part 16 - X3D Interactive Profile

MPEG-4

Today’s Topic : Royalty Free

Static Geometry Compression: TFAN

Animation and Animated Geometry Compression: BBA and FAMC

Web 3D Graphics Coding

Compression Benchmark Platform

Focus on 3D Object Compression

Decompose a triangle mesh into a set of triangle fansEncoder each triangle fan with a special code depending on the configuration of its neighborhood Arithmetically encode the TFAN codes

TFans: 1675, 62937, 2984

Low complexity: linear encoding/decoding complexity Handles directly (without stitching) arbitrary connectivity (manifold or not, oriented or not, with or without holes, any genus) Offers the possibility to preserve the Vertices Order (VO) and Triangles Order (TO)

Principle

Advantages

Triangle FAN

Connectivity encoding

Triangle FAN

From K. Mammou, Phd Thesis 2008

Bone-base Animation - BBA

Geometry Seamless mesh: shapes sharing the same vertices list

TextureImage Mapping on vertices sub-set

HierarchySkeleton layerMuscle layer

Animation Parameters1D controllers: bones and muscles

Courtesy of VRLab, EPFL

Courtesy of IMT

Widely supported by ATs

Able to build and animate any kind of skeleton

Courtesy of Televirtual

Realistic and high quality animation

Bone-base Animation - BBA

Frame #nUncompressed BBA

Binary file

Frame P

Prediction

Arithmetic coding

Quantization

Frame I

Arithmetic coding

Quantization

Segment #nUncompressed BBA

DCT

Binary file

Segment P

Prediction

DC Q

Segment I

Huffman coding

DC Q

DC Coeff. AC Coeff.

Huffman coding

AC Q

Bone-base Animation - BBA

Two compression methods: frame-based and DCT-based

5-30 kbps

Frame-based Mesh Compression - FAMC

What if no skeleton is defined, only animated meshes?

FAMC: cluster the vertices and encode cluster animationparameters plus corrections

Frame-based Mesh Compression - FAMC

LPCA : Linear PCAAWC : Anisotropic Wavelets Compression CPCA : Clustered PCATWC : Temporal Wavelet-based Compression

Compression benchmarking

more than 10 000 objects of various kinds distortion vs bitrate encoding and decoding time per object and global statistics

Compression benchmarking

API to integrate third party codecs

3D Studio Max

MPEG-4 File

Exp

ort

MP4toJS Export

MPEG Web3DG Format

JSONDescription

Texture(JPEG)

SC3DMC BBA

Web3DCoding Scripts

GraphicsCodec

WebGLEngine

BBA Decoder

JSON Parser

SC3DMC Decoder

glIndexedFaceSet

MPEG Web3DG

JSONDescription

Texture(JPEG)

SC3DMC

BBA

{"object" : { "name": "rabbit", "shapes":[ { "filename": "rabbit_0.s3d", "transform": [1.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 1.0], "materials":[ { "texture": "rabbit_0_0.jpg" } ] ,"animation":{ "anm_filename": "rabbit_0.anm", "bba_streams": [ { "name": "rabbit_0_0.bba", "fileName": "rabbit_0_0.bba" }]}}]}}

Web 3D Graphics Coding

Web3DCoding Scripts

GraphicsCodec

WebGLEngine

BBA Decoder

JSON Parser

SC3DMC Decoder

glIndexedFaceSet

MPEG Web3DG

JSONDescription

Texture(JPEG)

SC3DMC

BBA

{"object" : { "name": "rabbit", "shapes":[ { "filename": "rabbit_0.s3d", "transform": [1.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 1.0], "materials":[ { "texture": "rabbit_0_0.jpg" } ] ,"animation":{ "anm_filename": "rabbit_0.anm", "bba_streams": [ { "name": "rabbit_0_0.bba", "fileName": "rabbit_0_0.bba" }]}}]}}

AMD of MPEG-4 Part 16 JSON support for object graph light (textual) representation of bones hierarchy requirements for supporting GPU texture formats

Web 3D Graphics Coding

TFAN and BBA open source implementation 3DSMax to MPEG-4 (TFAN and BBA) open source exporter JS Implementation of TFAN and BBA decoders WebGL Implementation

Open Resources

www.MyMultimediaWorld.com : Software/OpenSource Tab

The outside world is waiting us with a UNIQUE compression solution for A 3D Object

Time to set the common base and work TOGETHER for a common profile

Many technologies are already available, time to do SELECTIONS

Conclusions