multimedia and html5 - think vitamin html5 online conference 01.11.2010
DESCRIPTION
Multimedia and HTML5 There is a wealth of new functionality coming to browsers under the umbrella term of HTML5, but certainly the most talked-about is the inclusion of native media playback and the visual possibilities offered by the new canvas element. In this session we'll cover the basics of audio, video and canvas in HTML5 - mixing context and theory with simple live demos and examples that showcase the power of these new tools for developers. http://thinkvitamin.com/online-conferences/html5-nov/ Links to demos used in the presentation: http://people.opera.com/patrickl/experiments/video/basic/ http://people.opera.com/patrickl/experiments/flash-overlap/ http://people.opera.com/patrickl/experiments/flash-overlap/fixed.html http://people.opera.com/patrickl/experiments/video/hover+transition/ http://www.youtube.com/html5 http://camendesign.com/code/video_for_everybody http://camendesign.com/code/video_for_everybody/test.html http://people.opera.com/patrickl/experiments/webm/basic-controls/ http://people.opera.com/patrickl/experiments/webm/fancy-controls/ http://people.opera.com/patrickl/experiments/webm/fancy-swap/ http://people.opera.com/patrickl/experiments/audio/wilhelm/controls.html http://people.opera.com/patrickl/experiments/audio/wilhelm/ http://people.opera.com/patrickl/experiments/canvas/particle/2/ http://mariuswatz.com/works/abstract01js/index.html http://www.splintered.co.uk/experiments/archives/paranoid_0.2/ http://alteredqualia.com/canvasmol/ http://www.splintered.co.uk/experiments/archives/canvas-ambilight/ http://people.opera.com/patrickl/experiments/canvas/image-edit/ http://github.com/mezzoblue/PaintbrushJS http://html5doctor.com/video-canvas-magic/ http://media.chikuyonok.ru/ambilight/ http://diveintohtml5.org/everything.html http://www.modernizr.com/ http://github.com/Modernizr/Modernizr/wiki/HTML5-Cross-browser-Polyfills http://sublimevideo.net/ http://videojs.com/ http://www.happyworm.com/jquery/jplayer/TRANSCRIPT
Multimedia and HTML5
Patrick H. Lauke / Think Vitamin HTML5 Online Conference / 1 November 2010
AUDIO, VIDEO AND CANVAS SUPPORT
Web Evangelist at Opera
HTML5<!DOCTYPE html>
“...extending the language to better support Web applications [...] This puts HTML in direct competition with other technologies[...] , in particular Flash and Silverlight.”
Ian Hickson, Editor of HTML5http://lists.w3.org/Archives/Public/public-html/2009Jan/0215.html
making your site Fonzie compliant...
<video>
Adobe Flash currently most commonvideo delivery mechanism
<object width="425" height="344"><param name="movie"
value="http://www.youtube.com/v/9sEI1AUFJKw&hl=en&fs=1&"></param>
<param name="allowFullScreen" value="true"></param>
<param name="allowscriptaccess" value="always"></param>
<embed src="http://www.youtube.com/v/9sEI1AUFJKw&hl=en&fs=1&" type="application/x-shockwave-flash" allowscriptaccess="always" allowfullscreen="true" width="425" height="344"></embed></object>
<video src="video.webm" controls Autoplay Loop preload="none" poster="poster.jpg" width="320" height="240"> <a href="video.webm">Download movie</a></video>
video as native object● “plays nice” with rest of the page● keyboard accessibility built-in
video formatsthe big debate
MP4 / H.264
ubiquitous, patent encumbered, licensing/royalties
Ogg Theora
“free and open”, no licensing/royaltiesnot many tools for it, not web optimised
WebM
open and royalty-free, web optimised
providing multiple sources<video controls autoplay poster="…" width="…" height="…">
<source src="movie.webm" type="video/webm" /><source src="movie.ogv" type="video/ogg" /><source src="movie.mp4" type="video/mp4" /><!-- fallback content -->
</video>
still include fallback for old browsershttp://camendesign.com/code/video_for_everybody
<video controls autoplay poster="…" width="…" height="…"><source src="movie.webm" type="video/webm" /><source src="movie.ogv" type="video/ogg" /><source src="movie.mp4" type="video/mp4" /><object width="…" height="…" type="application/x-
shockwave-flash" data="player.swf"><param name="movie" value="player.swf" /><param name="flashvars" value=" … file=movie.mp4" /><!-- fallback content -->
</object></video>
powerful (simple) APIto script your own controls
controlling a <video> element
var v = document.getElementById('player');
v.play();v.pause();v.volume = … ;v.currentTime = … ;…
events fired by <video> element
var v = document.getElementById('player');v.addEventListener('loadeddata', function() { … }, true)v.addEventListener('play', function() { … }, true)v.addEventListener('pause', function() { … }, true)v.addEventListener('timeupdate', function() { … }, true)v.addEventListener('ended', function() { … }, true)…
<audio>
audio exactly the same as video
<audio src=”music.mp3” controls autoplay></audio>[...]<audio controls autoplay>
<source src="music.mp3" type="audio/mpeg" /><source src="music.oga" type="audio/ogg" /><!-- fallback content -->
</audio>
same format debacle: MP3 vs Ogg Vorbis (vs WAV)
<canvas>
canvas = “scriptable images”
<canvas width="…" height="…"></canvas>
canvas has standard API methods for drawing
ctx = canvas.getContext("2d");ctx.fillRect(x, y, width, height);ctx.beginPath();ctx.moveTo(x, y);ctx.lineTo(x, y);ctx.bezierCurveTo(x1, y1, x2, y2, c1, c2);
canvas drawing ready-made images
ctx = canvas.getContext("2d");var logo = new Image();logo.src = 'logo.png';ctx.drawImage(logo,x1,y1,w1,h1,x2,y2,w2,h2);
or call in an existing image already on the page
canvas access to image data array
ctx = canvas.getContext("2d");canvasData = ctx.getImageData(x,y,w,h);[R,G,B,A,R,G,B,A,R,G,B,A,R,G,B,A, … ]
canvas accessibility concerns
canvas also works with video
ctx = canvas.getContext("2d");v = document.getElementById('player');ctx.drawImage(v,x1,y1,w1,h2,x2,y2,w2,h2);
grab currently displayed frame (update as appropriate)
video, audio and canvas on any devicewithout plugins
(Java / Flash / Silverlight not ubiquitous)
is it all safe to use, right now?
don't do browser sniffing
http://www.flickr.com/photos/timdorr/2096272747/
feature-detectionprogressive enhancement, graceful degradation
http://diveintohtml5.org/everything.html
feature-detection for audio/video
if (!!document.createElement('video').canPlayType;) { … }if (!!document.createElement('audio').canPlayType;) { … }
feature-detection for audio/video codecs
var v = document.createElement('video');if (!!(v.canPlayType)&& ((v.canPlayType('video/webm;codecs="vp8,vorbis"') == 'probably') || (v.canPlayType('video/webm;codecs="vp8, vorbis"') == 'maybe'))) { … }
patching older browsersgithub.com/Modernizr/Modernizr/wiki/HTML5-Cross-browser-Polyfills
ready-made HTML5 audio/video players(for the lazy)
HTML5 as Flashkiller?
not a question of HTML5 replacing Flash...
giving developers a choice!
www.opera.com/developerpeople.opera.com/patrickl/presentations/thinkvitamin-html5_01.11.2010/thinkvitamin-html5_01.11.2010.pdf