html5 multimedia - browser-native video, audio and canvas - meet.js summit / poznan / 14 january...
TRANSCRIPT
![Page 1: HTML5 multimedia - browser-native video, audio and canvas - meet.js Summit / Poznan / 14 January 2012](https://reader034.vdocuments.us/reader034/viewer/2022052619/5551512bb4c905c6268b55c3/html5/thumbnails/1.jpg)
HTML5 multimedia
Patrick H. Lauke / meet.js Summit / Poznań / 14 January 2011
BROWSER-NATIVE VIDEO, AUDIO AND CANVAS
![Page 2: HTML5 multimedia - browser-native video, audio and canvas - meet.js Summit / Poznan / 14 January 2012](https://reader034.vdocuments.us/reader034/viewer/2022052619/5551512bb4c905c6268b55c3/html5/thumbnails/2.jpg)
Web Evangelist at Opera
![Page 3: HTML5 multimedia - browser-native video, audio and canvas - meet.js Summit / Poznan / 14 January 2012](https://reader034.vdocuments.us/reader034/viewer/2022052619/5551512bb4c905c6268b55c3/html5/thumbnails/3.jpg)
making your site Fonzie compliant...
![Page 4: HTML5 multimedia - browser-native video, audio and canvas - meet.js Summit / Poznan / 14 January 2012](https://reader034.vdocuments.us/reader034/viewer/2022052619/5551512bb4c905c6268b55c3/html5/thumbnails/4.jpg)
<video>
![Page 5: HTML5 multimedia - browser-native video, audio and canvas - meet.js Summit / Poznan / 14 January 2012](https://reader034.vdocuments.us/reader034/viewer/2022052619/5551512bb4c905c6268b55c3/html5/thumbnails/5.jpg)
Adobe Flash currently most commonvideo delivery mechanism
![Page 6: HTML5 multimedia - browser-native video, audio and canvas - meet.js Summit / Poznan / 14 January 2012](https://reader034.vdocuments.us/reader034/viewer/2022052619/5551512bb4c905c6268b55c3/html5/thumbnails/6.jpg)
<object width="425" height="344"><param name="movie"
value="http://www.youtube.com/v/9sEI1AUFJKw&hl=en&fs=1&"></param>
<param name="allowFullScreen" value="true"></param>
<param name="allowscriptaccess" value="always"></param>
<embed src="http://www.youtube.com/v/9sEI1AUFJKw&hl=en&fs=1&" type="application/x-shockwave-flash" allowscriptaccess="always" allowfullscreen="true" width="425" height="344"></embed></object>
![Page 7: HTML5 multimedia - browser-native video, audio and canvas - meet.js Summit / Poznan / 14 January 2012](https://reader034.vdocuments.us/reader034/viewer/2022052619/5551512bb4c905c6268b55c3/html5/thumbnails/7.jpg)
<video src="video.webm"></video>
![Page 8: HTML5 multimedia - browser-native video, audio and canvas - meet.js Summit / Poznan / 14 January 2012](https://reader034.vdocuments.us/reader034/viewer/2022052619/5551512bb4c905c6268b55c3/html5/thumbnails/8.jpg)
<video src="video.webm" controls autoplay muted loop preload="none|metadata|auto" poster="poster.jpg" width="320" height="240"> <a href="video.webm">Download movie</a></video>
![Page 9: HTML5 multimedia - browser-native video, audio and canvas - meet.js Summit / Poznan / 14 January 2012](https://reader034.vdocuments.us/reader034/viewer/2022052619/5551512bb4c905c6268b55c3/html5/thumbnails/9.jpg)
video as native object
● behaves like any other HTML element● keyboard accessibility out-of-the-box
![Page 10: HTML5 multimedia - browser-native video, audio and canvas - meet.js Summit / Poznan / 14 January 2012](https://reader034.vdocuments.us/reader034/viewer/2022052619/5551512bb4c905c6268b55c3/html5/thumbnails/10.jpg)
people.opera.com/patrickl/experiments/webm/hover+transition
![Page 11: HTML5 multimedia - browser-native video, audio and canvas - meet.js Summit / Poznan / 14 January 2012](https://reader034.vdocuments.us/reader034/viewer/2022052619/5551512bb4c905c6268b55c3/html5/thumbnails/11.jpg)
![Page 12: HTML5 multimedia - browser-native video, audio and canvas - meet.js Summit / Poznan / 14 January 2012](https://reader034.vdocuments.us/reader034/viewer/2022052619/5551512bb4c905c6268b55c3/html5/thumbnails/12.jpg)
powerful (simple) API
![Page 13: HTML5 multimedia - browser-native video, audio and canvas - meet.js Summit / Poznan / 14 January 2012](https://reader034.vdocuments.us/reader034/viewer/2022052619/5551512bb4c905c6268b55c3/html5/thumbnails/13.jpg)
controlling <video> with JavaScript
var v = document.getElementById('player');
v.play();v.pause();v.volume = … ;v.currentTime = … ;…
![Page 14: HTML5 multimedia - browser-native video, audio and canvas - meet.js Summit / Poznan / 14 January 2012](https://reader034.vdocuments.us/reader034/viewer/2022052619/5551512bb4c905c6268b55c3/html5/thumbnails/14.jpg)
events fired by <video>
var v = document.getElementById('player');
v.addEventListener('loadeddata', function() { … }, true)v.addEventListener('play', function() { … }, true)v.addEventListener('pause', function() { … }, true)v.addEventListener('timeupdate', function() { … }, true)v.addEventListener('ended', function() { … }, true)…
![Page 16: HTML5 multimedia - browser-native video, audio and canvas - meet.js Summit / Poznan / 14 January 2012](https://reader034.vdocuments.us/reader034/viewer/2022052619/5551512bb4c905c6268b55c3/html5/thumbnails/16.jpg)
people.opera.com/patrickl/experiments/webm/basic-controls
![Page 17: HTML5 multimedia - browser-native video, audio and canvas - meet.js Summit / Poznan / 14 January 2012](https://reader034.vdocuments.us/reader034/viewer/2022052619/5551512bb4c905c6268b55c3/html5/thumbnails/17.jpg)
people.opera.com/patrickl/experiments/webm/fancy-controls
![Page 18: HTML5 multimedia - browser-native video, audio and canvas - meet.js Summit / Poznan / 14 January 2012](https://reader034.vdocuments.us/reader034/viewer/2022052619/5551512bb4c905c6268b55c3/html5/thumbnails/18.jpg)
people.opera.com/patrickl/experiments/webm/fancy-swap
![Page 19: HTML5 multimedia - browser-native video, audio and canvas - meet.js Summit / Poznan / 14 January 2012](https://reader034.vdocuments.us/reader034/viewer/2022052619/5551512bb4c905c6268b55c3/html5/thumbnails/19.jpg)
HTML5 means all your old dHTML is cool again!
![Page 20: HTML5 multimedia - browser-native video, audio and canvas - meet.js Summit / Poznan / 14 January 2012](https://reader034.vdocuments.us/reader034/viewer/2022052619/5551512bb4c905c6268b55c3/html5/thumbnails/20.jpg)
video formatsthe big debate?
![Page 21: HTML5 multimedia - browser-native video, audio and canvas - meet.js Summit / Poznan / 14 January 2012](https://reader034.vdocuments.us/reader034/viewer/2022052619/5551512bb4c905c6268b55c3/html5/thumbnails/21.jpg)
HTML5 does not specify video container/codec
(same as with images in HTML 4.01)
![Page 22: HTML5 multimedia - browser-native video, audio and canvas - meet.js Summit / Poznan / 14 January 2012](https://reader034.vdocuments.us/reader034/viewer/2022052619/5551512bb4c905c6268b55c3/html5/thumbnails/22.jpg)
MP4/H.264or
Ogg Theoraor
WebM/VP8
![Page 23: HTML5 multimedia - browser-native video, audio and canvas - meet.js Summit / Poznan / 14 January 2012](https://reader034.vdocuments.us/reader034/viewer/2022052619/5551512bb4c905c6268b55c3/html5/thumbnails/23.jpg)
MP4 / H.264
ubiquitous, patent encumbered, licensing/royalties
![Page 24: HTML5 multimedia - browser-native video, audio and canvas - meet.js Summit / Poznan / 14 January 2012](https://reader034.vdocuments.us/reader034/viewer/2022052619/5551512bb4c905c6268b55c3/html5/thumbnails/24.jpg)
Ogg Theora
free and open, no licensing feesnot many tools for it, not web optimised
![Page 25: HTML5 multimedia - browser-native video, audio and canvas - meet.js Summit / Poznan / 14 January 2012](https://reader034.vdocuments.us/reader034/viewer/2022052619/5551512bb4c905c6268b55c3/html5/thumbnails/25.jpg)
WebM / VP8
open and royalty-free, web-optimisedsupport by hardware and software vendors
![Page 26: HTML5 multimedia - browser-native video, audio and canvas - meet.js Summit / Poznan / 14 January 2012](https://reader034.vdocuments.us/reader034/viewer/2022052619/5551512bb4c905c6268b55c3/html5/thumbnails/26.jpg)
providing multiple sources<video controls width="…" height="…">
<source src="movie.mp4" type="video/mp4"><source src="movie.webm" type="video/webm"><source src="movie.ogv" type="video/ogg"><!-- fallback content -->
</video>
![Page 27: HTML5 multimedia - browser-native video, audio and canvas - meet.js Summit / Poznan / 14 January 2012](https://reader034.vdocuments.us/reader034/viewer/2022052619/5551512bb4c905c6268b55c3/html5/thumbnails/27.jpg)
specifying codecs<video controls width="…" height="…">
<source … type='video/mp4; codecs="avc1.42E01E,mp4a.40.2"'><source … type='video/webm; codecs="vorbis,vp8"'><source … type='video/ogg; codecs="theora,vorbis"'><!-- fallback content -->
</video>
![Page 28: HTML5 multimedia - browser-native video, audio and canvas - meet.js Summit / Poznan / 14 January 2012](https://reader034.vdocuments.us/reader034/viewer/2022052619/5551512bb4c905c6268b55c3/html5/thumbnails/28.jpg)
feature-detection for codecs
v.canPlayType('video/webm;codecs="vp8,vorbis"')
"probably" | "maybe" | ""
![Page 29: HTML5 multimedia - browser-native video, audio and canvas - meet.js Summit / Poznan / 14 January 2012](https://reader034.vdocuments.us/reader034/viewer/2022052619/5551512bb4c905c6268b55c3/html5/thumbnails/29.jpg)
![Page 32: HTML5 multimedia - browser-native video, audio and canvas - meet.js Summit / Poznan / 14 January 2012](https://reader034.vdocuments.us/reader034/viewer/2022052619/5551512bb4c905c6268b55c3/html5/thumbnails/32.jpg)
flash fallback for older browsershttp://camendesign.com/code/video_for_everybody
![Page 33: HTML5 multimedia - browser-native video, audio and canvas - meet.js Summit / Poznan / 14 January 2012](https://reader034.vdocuments.us/reader034/viewer/2022052619/5551512bb4c905c6268b55c3/html5/thumbnails/33.jpg)
<video controls width="…" height="…"><source src="movie.mp4" type="video/mp4" /><source src="movie.webm" type="video/webm" /><source src="movie.ogv" type="video/ogg" />
<object width="…" height="…" type="application/x-shockwave-flash" data="player.swf">
<param name="movie" value="player.swf" /><param name="flashvars" value=" … file=movie.mp4" /><!-- fallback content -->
</object>
</video>
![Page 34: HTML5 multimedia - browser-native video, audio and canvas - meet.js Summit / Poznan / 14 January 2012](https://reader034.vdocuments.us/reader034/viewer/2022052619/5551512bb4c905c6268b55c3/html5/thumbnails/34.jpg)
<audio>
![Page 35: HTML5 multimedia - browser-native video, audio and canvas - meet.js Summit / Poznan / 14 January 2012](https://reader034.vdocuments.us/reader034/viewer/2022052619/5551512bb4c905c6268b55c3/html5/thumbnails/35.jpg)
audio...exactly the same as video
<audio src=”music.mp3” controls autoplay … ></audio>
<audio controls autoplay><source src="music.mp3" type="audio/mpeg" /><source src="music.oga" type="audio/ogg" /><!-- fallback content -->
</audio>
formats: MP3 vs Ogg Vorbis (vs WAV)
![Page 36: HTML5 multimedia - browser-native video, audio and canvas - meet.js Summit / Poznan / 14 January 2012](https://reader034.vdocuments.us/reader034/viewer/2022052619/5551512bb4c905c6268b55c3/html5/thumbnails/36.jpg)
people.opera.com/patrickl/experiments/audio/wilhelm
![Page 37: HTML5 multimedia - browser-native video, audio and canvas - meet.js Summit / Poznan / 14 January 2012](https://reader034.vdocuments.us/reader034/viewer/2022052619/5551512bb4c905c6268b55c3/html5/thumbnails/37.jpg)
people.opera.com/patrickl/experiments/audio/windchime
![Page 38: HTML5 multimedia - browser-native video, audio and canvas - meet.js Summit / Poznan / 14 January 2012](https://reader034.vdocuments.us/reader034/viewer/2022052619/5551512bb4c905c6268b55c3/html5/thumbnails/38.jpg)
no need for <audio> in your document
var s = document.createElement('audio');s.src = 'aaargh.oga'; s.play();
(same for video as well, if you're crazy...)
![Page 39: HTML5 multimedia - browser-native video, audio and canvas - meet.js Summit / Poznan / 14 January 2012](https://reader034.vdocuments.us/reader034/viewer/2022052619/5551512bb4c905c6268b55c3/html5/thumbnails/39.jpg)
<canvas>
![Page 40: HTML5 multimedia - browser-native video, audio and canvas - meet.js Summit / Poznan / 14 January 2012](https://reader034.vdocuments.us/reader034/viewer/2022052619/5551512bb4c905c6268b55c3/html5/thumbnails/40.jpg)
canvas = “scriptable images”
<canvas width="…" height="…"></canvas>
![Page 41: HTML5 multimedia - browser-native video, audio and canvas - meet.js Summit / Poznan / 14 January 2012](https://reader034.vdocuments.us/reader034/viewer/2022052619/5551512bb4c905c6268b55c3/html5/thumbnails/41.jpg)
canvas has standard API methods for drawing
ctx = canvas.getContext("2d");ctx.fillRect(x, y, width, height);ctx.beginPath();ctx.moveTo(x, y);ctx.lineTo(x, y);ctx.bezierCurveTo(x1, y1, x2, y2, c1, c2);…
www.w3.org/TR/2dcontext
![Page 42: HTML5 multimedia - browser-native video, audio and canvas - meet.js Summit / Poznan / 14 January 2012](https://reader034.vdocuments.us/reader034/viewer/2022052619/5551512bb4c905c6268b55c3/html5/thumbnails/42.jpg)
people.opera.com/patrickl/experiments/canvas/particle/3
![Page 44: HTML5 multimedia - browser-native video, audio and canvas - meet.js Summit / Poznan / 14 January 2012](https://reader034.vdocuments.us/reader034/viewer/2022052619/5551512bb4c905c6268b55c3/html5/thumbnails/44.jpg)
canvas drawing ready-made images
ctx = canvas.getContext("2d");
var logo = new Image();logo.src = 'logo.png';
ctx.drawImage(logo,x1,y1,w1,h1,x2,y2,w2,h2);
or call in an existing image already on the page
![Page 45: HTML5 multimedia - browser-native video, audio and canvas - meet.js Summit / Poznan / 14 January 2012](https://reader034.vdocuments.us/reader034/viewer/2022052619/5551512bb4c905c6268b55c3/html5/thumbnails/45.jpg)
www.splintered.co.uk/experiments/archives/paranoid_0.4
![Page 46: HTML5 multimedia - browser-native video, audio and canvas - meet.js Summit / Poznan / 14 January 2012](https://reader034.vdocuments.us/reader034/viewer/2022052619/5551512bb4c905c6268b55c3/html5/thumbnails/46.jpg)
canvas access to image data array
ctx = canvas.getContext("2d");canvasData = ctx.getImageData(x,y,w,h);
[R,G,B,A,R,G,B,A,R,G,B,A,R,G,B,A, … ]
![Page 47: HTML5 multimedia - browser-native video, audio and canvas - meet.js Summit / Poznan / 14 January 2012](https://reader034.vdocuments.us/reader034/viewer/2022052619/5551512bb4c905c6268b55c3/html5/thumbnails/47.jpg)
www.splintered.co.uk/experiments/archives/canvas-ambilight
![Page 49: HTML5 multimedia - browser-native video, audio and canvas - meet.js Summit / Poznan / 14 January 2012](https://reader034.vdocuments.us/reader034/viewer/2022052619/5551512bb4c905c6268b55c3/html5/thumbnails/49.jpg)
canvas also works with video
ctx = canvas.getContext("2d");v = document.getElementById('player');
ctx.drawImage(v,x1,y1,w1,h2,x2,y2,w2,h2);
grab currently displayed frame (update as appropriate)
![Page 52: HTML5 multimedia - browser-native video, audio and canvas - meet.js Summit / Poznan / 14 January 2012](https://reader034.vdocuments.us/reader034/viewer/2022052619/5551512bb4c905c6268b55c3/html5/thumbnails/52.jpg)
is it all safe to use, right now?
![Page 54: HTML5 multimedia - browser-native video, audio and canvas - meet.js Summit / Poznan / 14 January 2012](https://reader034.vdocuments.us/reader034/viewer/2022052619/5551512bb4c905c6268b55c3/html5/thumbnails/54.jpg)
don't do browser sniffing
http://www.flickr.com/photos/timdorr/2096272747/
![Page 55: HTML5 multimedia - browser-native video, audio and canvas - meet.js Summit / Poznan / 14 January 2012](https://reader034.vdocuments.us/reader034/viewer/2022052619/5551512bb4c905c6268b55c3/html5/thumbnails/55.jpg)
![Page 56: HTML5 multimedia - browser-native video, audio and canvas - meet.js Summit / Poznan / 14 January 2012](https://reader034.vdocuments.us/reader034/viewer/2022052619/5551512bb4c905c6268b55c3/html5/thumbnails/56.jpg)
feature-detectionprogressive enhancement, graceful degradation
diveintohtml5.info/everything.html
![Page 57: HTML5 multimedia - browser-native video, audio and canvas - meet.js Summit / Poznan / 14 January 2012](https://reader034.vdocuments.us/reader034/viewer/2022052619/5551512bb4c905c6268b55c3/html5/thumbnails/57.jpg)
feature-detection for audio/video
if (!!document.createElement('video').canPlayType;) { … }
if (!!document.createElement('audio').canPlayType;) { … }
![Page 62: HTML5 multimedia - browser-native video, audio and canvas - meet.js Summit / Poznan / 14 January 2012](https://reader034.vdocuments.us/reader034/viewer/2022052619/5551512bb4c905c6268b55c3/html5/thumbnails/62.jpg)
dev.w3.org/2011/webrtc/editor/getusermedia.html
![Page 63: HTML5 multimedia - browser-native video, audio and canvas - meet.js Summit / Poznan / 14 January 2012](https://reader034.vdocuments.us/reader034/viewer/2022052619/5551512bb4c905c6268b55c3/html5/thumbnails/63.jpg)
camera accessnavigator.getUserMedia({video:true},success,error);
![Page 67: HTML5 multimedia - browser-native video, audio and canvas - meet.js Summit / Poznan / 14 January 2012](https://reader034.vdocuments.us/reader034/viewer/2022052619/5551512bb4c905c6268b55c3/html5/thumbnails/67.jpg)
HTML5 as Flashkiller?
![Page 68: HTML5 multimedia - browser-native video, audio and canvas - meet.js Summit / Poznan / 14 January 2012](https://reader034.vdocuments.us/reader034/viewer/2022052619/5551512bb4c905c6268b55c3/html5/thumbnails/68.jpg)
not a question ofHTML5 replacing Flash...
![Page 69: HTML5 multimedia - browser-native video, audio and canvas - meet.js Summit / Poznan / 14 January 2012](https://reader034.vdocuments.us/reader034/viewer/2022052619/5551512bb4c905c6268b55c3/html5/thumbnails/69.jpg)
giving developers a choice!