Towards Video on the Web with HTML5

Authors

François Daoust, Philipp Hoschka — W3C/ERCIM, Sophia-Antipolis, France

Charalampos Z. Patrikakis — School of Electrical and Computer Engineering, NTUA, Athens, Greece

Rui S. Cruz, Mário S. Nunes — IST/INESC-ID/INOV, Lisbon, Portugal

David Salama Osborne — Atos Research and Innovation, Madrid, Spain

Presenter

François Daoust <fd@w3.org>, World Wide Web Consortium

This presentation

http://www.w3.org/2010/Talks/1014-html5-video-fd/

14 October 2010, Barcelona, NEM Summit

The research leading to these results has received funding from the European Union's Seventh

Framework Programme (FP7/2007-2013) under grant agreement n°248687 - Open Media Web

(OMWeb)

and n°248474 Socially Aware, collaboRative, scAlable Coding mEdia distributioN (SARACEN)

Outline

Photo by Elaine Vallet

What is HTML5?

Let's make sure we're talking about the same thing…

Video on the Web

codecs, metadata, streaming, fragments

More TV-Relevant Features

CSS, SVG, Canvas

Web Applications

Device APIs, Widgets

W3C: Shaping the Web of the future

Web Standards

(X)HTML, CSS, XML, SVG, PNG, XSLT, WCAG, RDF, ...

Consortium

330 members, from industry and research

World-wide

Offices in many countries, including Brazil, China, India, Morocco, South

Africa, ...

One Web!

Founded and directed by inventor of the Web, Tim Berners-Lee

Global participation

32,000 people subscribed to mailing lists, 1,500+ participants in 60+ Groups

Part 1:

What is HTML5?

HTML5, what is it?

It's a specification developed by the W3C HTML Working Group:

Video on the Web

Web applications

Canvas

new APIs

…

And more importantly:

HTML on the move

Disruptive technology

The Next Open Web Platform

The HTML5 specification is a portion of the next Open Web platform:

HTML5

CSS 2.1

CSS 3 Selectors

CSS 3 Media Queries

CSS 3 Text

CSS 3 Backgrounds and Borders

CSS 3 Colors

CSS 3 2D Transformations

CSSOM View Module

CSS 3 Transitions

CSS 3 Animations

CSS 3 Multi-Columns

CSS Namespaces

SVG 1.1

WAI-ARIA 1.0

MathML 2.0

ECMAScript 5

2D Context

WebGL

Web Storage

Indexed Database

Web Workers

Web Sockets Protocol/API

Geolocation

Progress Events

Element Traversal

DOM Level 3 Events

Media Fragments

XMLHttpRequest

Selectors API

CSSOM View Module

File API

RDFa

Microdata

WOFF

HTTP 1.1 part 1 to part 7

TLS 1.2 (updated)

IRI (updated)

…

Legend:

spec: developed within W3C

spec: developed outside of W3C (IETF, Khronos Group, ECMA)

Quick background on HTML5

History

Effort started by Ian Hickson, while working at Opera

2004: Opera, Mozilla, and Apple creates the WHAT WG

2007: W3C restarts the HTML Working Group

2010: Apple, Google, Mozilla, Microsoft, and Opera (and others) are implementing, or are committed to, HTML5

The HTML working group in W3C

Chaired by Maciej Stachowiak (Apple), Paul Cotton (Microsoft), Sam Ruby (IBM)

40 W3C Member organizations, all committed to W3C Royalty-Free terms

439 group participants

259 Invited Experts (most of them public invited experts)

10 mailing lists

7553 emails on public-html… since January 2010

Roadmap

W3C Last Call in May 2011

Part 2:

Video on the Web

The <video> tag in HTML5

A regular HTML tag

A standard API

var vid = document.getElementById("vid");vid.play();vid.pause();vid.currentTime = 0;

Interaction with CSS

video { border-radius: 2em; translate(100px, -100px) skewY(30deg) scale(0.5,0.5); opacity: 0.5;}

<video src="myMovie" id="vid" />

Video and Canvas

The HTML5 <canvas> tag allows for direct manipulation of video data:

Dump video frame to a <canvas> tag

Analyse video frame as an array of pixels with JavaScript

React consequently

Muppet says: Open Media Web!

W3C Standard released in 2001 (SVG 1.2 released in 2009)

SVG is included in HTML5

Complete freedom to redesign user interfaces

Possibility to clip and filter video at will

[Clock ticking]

[tick, tick, tick, …]

CC00:05 04:23

Video and Vector Graphics

Tick!

Audio/Video: Codecs and formats

Lots of challenges and patents.

Video codec

H.264 (profiles?), VP8, Theora, Dirac 2.1, …

Audio codec

AAC, WMA, Vorbis, PCV, …

Captioning, video description

SAMI, SMIL, Hi-Caption, CMML, TTML, 3GPP TS 26.245, MPSub, …

Container format

.avi, .mp4, .mov, .ogg, .flv, .mkv, …

No default audio/video/container format in HTML5 for the time being.

Captioning format will be specified by HTML5.

Projected support in main Web browsers

Internet Explorer (>=9)

H.264 (profile?) in an MP4 container

Safari (desktop, iPhone, iPad)

H.264 Baseline in an MP4 container

Firefox

Theora in an OGG container

VP8 in a WebM container

Opera

Theora in an OGG container

VP8 in a WebM container

Google Chrome

H.264 Baseline in an MP4 container

Theora in an OGG container

VP8 in a WebM container

At a minimum, once browsers are out:

One version that uses VP8 in a WebM container (Vorbis for audio)

Another version that uses H.264 Baseline in an MP4 container (AAC low complexity for audio)

Metadata

Many standards

MPEG-7, CableLabs, TV-ANytime, EBU, XML, …

Ontology for Media Resource 1.0

Last Call working draft, EBU is among participants

API for Media Resource 1.0

Media fragments

Photo by Robert Freund

http://www.example.com/example.ogv#xywh=160,120,320,240http://www.example.com/example.ogv?t=10,20

Streaming Audio/Video

Progressive download — basic file transfer

HTTP streaming — same with byte range

Using other transport protocols — UDP, DCCP and the like

HTTP adaptive streaming

HTTP Live Streaming by Apple

IIS Smooth Streaming by Microsoft for Silverlight

Dynamic streaming in Adobe Flash Player 10.1

3GPP adaptive HTTP Streaming (AHS)

OpenIP TV Forum

MPEG Dynamic Adaptive Streaming over HTTP (DASH)

What about browsers that will support WebM?

More standardization needed both at protocol level (IETF?)

and HTML/API level (W3C?)

Web and TV Interest Group at W3C:

public-web-and-tv@w3.org

Peer-to-peer connections

Ongoing efforts that relate to P2P

WebSockets (API and protocol)

for text-based connections between a client and a server

HTML Device specification

extracted from the HTML5 spec

No P2P protocol/API in HTML5 so far.

Call for actions issued by the editor of the specification in July 2010. Main use cases

Main use cases that will push for P2P

Video conferencing

Real-time games

Peer-to-peer file transfer

Part 3:

More TV-relevant Features

CSS Transitions

CSS Transitions describe how CSS properties change smoothly from one value to another over a given duration.

Example (CSS transitions must be supported):

Home

About

Feedback

Help

Contact

CSS Media Queries

<link rel="stylesheet" type="text/css" href="base.css" /><style type="text/css" media="screen and (min-width: 481px)"> @import url("advanced.css");</style><link rel="stylesheet" type="text/css" href="base.css" media="handheld, only screen and (max-device-width: 480px)" />

Fonts: WOFF File Format

« Les représentants du peuple français, constitués en Assemblée nationale,considérant que l'ignorance, l'oubli ou le mépris des droits de l'homme sont lesseules causes des malheurs publics et de la corruption des gouvernements, ontrésolu d'exposer, dans une déclaration solennelle, les droits naturels, inaliénableset sacrés de l'homme. »

(Déclaration des droits de l'Homme et du citoyen de 1789)

Towards Web Applications

Web vs native

Integration with device quoted as one of the main limitations of Web apps

Device APIs

W3C Widgets

Making Web applications:

downloadable

signable

sellable

with dedicated user interface

Summary

HTML5 is the next Open Web platform

The <video> tag is almost there

No standard audio/video codec and container format in HTML5 for the time being

(H.264/MP4, VP8/WebM and Theora/Ogg as main choices in browsers)

More standardization needed around streaming

No more limits in Graphical User Interfaces

(SVG, Canvas and advanced CSS)

Powerful Web Applications possible on multiple devices (desktop, mobile, TV, …)

(Device APIs, Widgets)

W3C Web on TV Workshop

Spring 2011, in Europe

Thanks

François Daoust <fd@w3.org>

World Wide Web Consortium

Special thanks to Philippe le Hégaret and Dominique Hazaël-Massieux for material in these slides

http://www.w3.org/2010/Talks/1014-html5-video-fd/

14 October 2010, Barcelona, NEM Summit

Questions/Discussion welcome…

Follow the Open Media Web project:

Web site: http://openmediaweb.eu/

RSS feed: http://openmediaweb.eu/feed/

Twitter: @w3c_omweb

The research leading to these results has received funding from the European Union's Seventh

Framework Programme (FP7/2007-2013) under grant agreement n°248687 - Open Media Web

(OMWeb)

and n°248474 Socially Aware, collaboRative, scAlable Coding mEdia distributioN (SARACEN)