expo08nyc moving pictures
Post on 30-May-2018
215 Views
Preview:
TRANSCRIPT
-
8/14/2019 Expo08nyc Moving Pictures
1/70
Moving PicturesImplementing Video on Flickr
Cal Henderson
-
8/14/2019 Expo08nyc Moving Pictures
2/70
Web 2.0 Expo NYC, 17th September 2008 2
Hello
-
8/14/2019 Expo08nyc Moving Pictures
3/70
Web 2.0 Expo NYC, 17th September 2008 3
Flickr
Large scale kitten sharing website
Started 2003, launched 2004
5 years old this december
Almost 3 billion photos
-
8/14/2019 Expo08nyc Moving Pictures
4/70
Web 2.0 Expo NYC, 17th September 2008 4
Enter: Video
Video was added this year
Launched April 2008
Many hundreds of thousands of videosuploaded
Many millions of playbacks
-
8/14/2019 Expo08nyc Moving Pictures
5/70
Web 2.0 Expo NYC, 17th September 2008 5
-
8/14/2019 Expo08nyc Moving Pictures
6/70
Web 2.0 Expo NYC, 17th September 2008 6
-
8/14/2019 Expo08nyc Moving Pictures
7/70Web 2.0 Expo NYC, 17th September 2008 7
-
8/14/2019 Expo08nyc Moving Pictures
8/70Web 2.0 Expo NYC, 17th September 2008 8
-
8/14/2019 Expo08nyc Moving Pictures
9/70Web 2.0 Expo NYC, 17th September 2008 9
-
8/14/2019 Expo08nyc Moving Pictures
10/70Web 2.0 Expo NYC, 17th September 2008 10
Video? Thats justlike photos!
-Me, before Flickr Video
-
8/14/2019 Expo08nyc Moving Pictures
11/70
Web 2.0 Expo NYC, 17th September 2008 11
12 4 Steps
4 main tasks
Uploading
Transcoding
Storage
Serving & Playback
-
8/14/2019 Expo08nyc Moving Pictures
12/70
Web 2.0 Expo NYC, 17th September 2008 12
1.
Upload
-
8/14/2019 Expo08nyc Moving Pictures
13/70
Web 2.0 Expo NYC, 17th September 2008 13
Simple upload
Web forms Just like any other file
But files are large / huge
-
8/14/2019 Expo08nyc Moving Pictures
14/70
Web 2.0 Expo NYC, 17th September 2008 14
Issues
Two components for uploading:
Sending from the client
Receiving on the server
Both of these present problems
-
8/14/2019 Expo08nyc Moving Pictures
15/70
Web 2.0 Expo NYC, 17th September 2008 15
Sending from the client
Multiple options
Simple form
Flash
Desktop app
-
8/14/2019 Expo08nyc Moving Pictures
16/70
Web 2.0 Expo NYC, 17th September 2008 16
Simple Forms
Pros Very easy to implement
Works on every browser out of the box
Cons
Upload progress is harder
Slow
Select a single file at once
-
8/14/2019 Expo08nyc Moving Pictures
17/70
Web 2.0 Expo NYC, 17th September 2008 17
Flash
Pros Upload progress is easy
Fast
Multi select of files
Cons
Harder to implement
Flash isnt quite ubiquitous
-
8/14/2019 Expo08nyc Moving Pictures
18/70
-
8/14/2019 Expo08nyc Moving Pictures
19/70
Web 2.0 Expo NYC, 17th September 2008 19
Making Progress
Upload progress Not impossible with plain forms
Just need to be able to query the upload
progress via AJAX
Multiple machines
The VIP issue Enter Perlbal
-
8/14/2019 Expo08nyc Moving Pictures
20/70
Web 2.0 Expo NYC, 17th September 2008 20
Making Progress
Browser
Web 1
Web 2
1. Browser starts upload
2. Web server broadcastsprogress via UDP
3. Browser queries progressvia AJX call
Loadbalancer
-
8/14/2019 Expo08nyc Moving Pictures
21/70
-
8/14/2019 Expo08nyc Moving Pictures
22/70
-
8/14/2019 Expo08nyc Moving Pictures
23/70
-
8/14/2019 Expo08nyc Moving Pictures
24/70
Web 2.0 Expo NYC, 17th September 2008 24
2.
Transcode
-
8/14/2019 Expo08nyc Moving Pictures
25/70
-
8/14/2019 Expo08nyc Moving Pictures
26/70
Web 2.0 Expo NYC, 17th September 2008 26
So many formats
But very few of these formats can beplayed back cross platform
Without special software or hardware
Formats are designed to do one thing well They dont always manage even that
Transcoding puts all videos on an equalfooting
-
8/14/2019 Expo08nyc Moving Pictures
27/70
Web 2.0 Expo NYC, 17th September 2008 27
Video file basics
Most file types are really just containers MOV, FLV, AVI
The data inside can be in multiple formats We call these codecs (encoder + decoder)
Files contains multiple streams
Both audio and video
-
8/14/2019 Expo08nyc Moving Pictures
28/70
Web 2.0 Expo NYC, 17th September 2008 28
Interleave
Audio and video are often interleaved Hence AVI
A video file looks like this: Headers
Video chunk
Audio chunk
Video chunk
Audio chunk
Etc
-
8/14/2019 Expo08nyc Moving Pictures
29/70
Web 2.0 Expo NYC, 17th September 2008 29
Compress
Raw video files are huge A bitmap for every frame
Rarely used, even in post production
At 30 fps, that gets crazy pretty quickly
We dont need to store every frame Static backgrounds dont change (much)
between frames
-
8/14/2019 Expo08nyc Moving Pictures
30/70
Web 2.0 Expo NYC, 17th September 2008 30
Compresssss
Intraframe Treat each frame as a picture
Compress it (just like JPEG)
DCT (Discrete Cosine Transform)
Interframe
Store the differences between frames
Treat the pixels as a 3D array to becompressed
-
8/14/2019 Expo08nyc Moving Pictures
31/70
Web 2.0 Expo NYC, 17th September 2008 31
The IPB
Three frame types: I, P & B
Intra coded pictures
A full raw frame
Predicted pictures Based on a single reference frame
Bi-predictive pictures Based on two or more reference frames
-
8/14/2019 Expo08nyc Moving Pictures
32/70
Web 2.0 Expo NYC, 17th September 2008 32
IPBIPBIPBIPB
Reference frames may be I, P or B
P & B frames may contain a mix of image
data and motion vector displacements
I frames require the most bits
Then P frames
Then B frames
-
8/14/2019 Expo08nyc Moving Pictures
33/70
Web 2.0 Expo NYC, 17th September 2008 33
Bad terminology
We should really say picture (Not frame)
Because of interlacing
Really, we encode fields not frames Picture is the general term
And H.264 contains slices Sub-regions of the field
Macroblocks & Artifacts
-
8/14/2019 Expo08nyc Moving Pictures
34/70
-
8/14/2019 Expo08nyc Moving Pictures
35/70
-
8/14/2019 Expo08nyc Moving Pictures
36/70
Web 2.0 Expo NYC, 17th September 2008 36
Seekable
-
8/14/2019 Expo08nyc Moving Pictures
37/70
-
8/14/2019 Expo08nyc Moving Pictures
38/70
Web 2.0 Expo NYC, 17th September 2008 38
Flash! Woah-oh!
The big question:
Flash?
-
8/14/2019 Expo08nyc Moving Pictures
39/70
-
8/14/2019 Expo08nyc Moving Pictures
40/70
Web 2.0 Expo NYC, 17th September 2008 40
The Flash Player
Flash Player 6
March 2002
Video: Sorenson Spark (H.263)
Audio: MP3
Or ADPCM / Uncompressed
Or Nellymoser Asao
-
8/14/2019 Expo08nyc Moving Pictures
41/70
Web 2.0 Expo NYC, 17th September 2008 41
Second Generation
Flash Player 7
August 2005
Video: On2 TrueMotion VP6
Audio: MP3
-
8/14/2019 Expo08nyc Moving Pictures
42/70
Web 2.0 Expo NYC, 17th September 2008 42
The hot shit
Flash Player 9 (update 3) December 2007
Video: H.264 (MPEG-4 Part 10) w/ container formats from MPEG-4 Part 14
Audio: AAC (MPEG-4 Part 3)
Plus 3GPP Timed Text (MPEG-4 Part 17)
-
8/14/2019 Expo08nyc Moving Pictures
43/70
-
8/14/2019 Expo08nyc Moving Pictures
44/70
Web 2.0 Expo NYC, 17th September 2008 44
H.264
Not proprietary Good compression
MPEG Standard Open, but patented
Patent licenses from the MPEG LA
Unclear how this applies to (L)GPL code
But probably badly
YouTube using it for higher quality streams
iPhones and AppleTV
-
8/14/2019 Expo08nyc Moving Pictures
45/70
-
8/14/2019 Expo08nyc Moving Pictures
46/70
Web 2.0 Expo NYC, 17th September 2008 46
More software
MEncoder libmpcodecs uses libavcodec
VLC libvlc uses libavcodec
So basically the same Different muxing, same codecs
-
8/14/2019 Expo08nyc Moving Pictures
47/70
Web 2.0 Expo NYC, 17th September 2008 47
Free H.264?
Unfortunately, not really
x264 is the only usable one
Its pretty good
MEncoder can use it
Still limited in options at this point
Again, dubiously legal
-
8/14/2019 Expo08nyc Moving Pictures
48/70
Web 2.0 Expo NYC, 17th September 2008 48
Non-free tools
Flash encoder Not automatable
On2 FlixEngine
Creators of VP6 Windows or Linux Some support for H.264
Rhozet Carbon Coder The new hot shit Good H.264 support Windows
-
8/14/2019 Expo08nyc Moving Pictures
49/70
Web 2.0 Expo NYC, 17th September 2008 49
Choices
Video codec Resolution
Bitrate (VBR, CBR)
Keyframes
Audio codec
Channels
Bit depth
Sampling rate
-
8/14/2019 Expo08nyc Moving Pictures
50/70
Web 2.0 Expo NYC, 17th September 2008 50
Doing it at scale
Not really a problem Very easily parallelizable
Amazon EC2 is awesome here
Exactly what it was design for
Grow/shrink as needed
But, per-CPU software licensing
-
8/14/2019 Expo08nyc Moving Pictures
51/70
Web 2.0 Expo NYC, 17th September 2008 51
3.
Store
-
8/14/2019 Expo08nyc Moving Pictures
52/70
Web 2.0 Expo NYC, 17th September 2008 52
Easy!
Really, just like photos
But with bigger files
Same disk layout as any other serving
But the serving part is slower
-
8/14/2019 Expo08nyc Moving Pictures
53/70
Web 2.0 Expo NYC, 17th September 2008 53
But..
Remember the files are huge
Operations take time
More likely to fail halfway through
Checksums are your friend
Do it all asynchronously
-
8/14/2019 Expo08nyc Moving Pictures
54/70
Web 2.0 Expo NYC, 17th September 2008 54
4.
Serve & Playback
-
8/14/2019 Expo08nyc Moving Pictures
55/70
Web 2.0 Expo NYC, 17th September 2008 55
The choice
Streamingvs
Progressive
-
8/14/2019 Expo08nyc Moving Pictures
56/70
Web 2.0 Expo NYC, 17th September 2008 56
Streaming
Pros Easily seekable
Live feeds
Cons
Special server software
Slower to start Firewall troubles
-
8/14/2019 Expo08nyc Moving Pictures
57/70
Web 2.0 Expo NYC, 17th September 2008 57
Progressive download
Pros Just use a web server
Play offline
Firewall/proxy friendly
Cons
Harder to seek ahead (but not impossible)
-
8/14/2019 Expo08nyc Moving Pictures
58/70
Web 2.0 Expo NYC, 17th September 2008 58
Streaming tech
Non flash stuff Well ignore that
Youre using flash, right?
RTMP
Real-time Messaging Protocol
Proprietary (thanks Adobe!)
-
8/14/2019 Expo08nyc Moving Pictures
59/70
Web 2.0 Expo NYC, 17th September 2008 59
RTMP
RTMP - Raw TCP socket stuffs RTMPT RTMP tunneled over HHTP
For firewalls, etc
Flash Media Server
previously Flash Communication Server
Wowza Pro $1000/server
-
8/14/2019 Expo08nyc Moving Pictures
60/70
Web 2.0 Expo NYC, 17th September 2008 60
Open source
Its not all bad
Red5
Java implementation of RTMP server
Mostly feature complete
Beta quality, but usable in production
Facebook
-
8/14/2019 Expo08nyc Moving Pictures
61/70
Web 2.0 Expo NYC, 17th September 2008 61
Progressive
Used by the majority of large sites
Very simple!
Seekable with server support
-
8/14/2019 Expo08nyc Moving Pictures
62/70
Web 2.0 Expo NYC, 17th September 2008 62
Seeking
Serve the FLV starting at a different point
Just add a simple FLV preamble before
seeking into the file
Simple to do in PHP, Perl, etc
mod_flvx for Apache
mod_secdownload for lighttpd
-
8/14/2019 Expo08nyc Moving Pictures
63/70
Web 2.0 Expo NYC, 17th September 2008 63
5.Other considerations
-
8/14/2019 Expo08nyc Moving Pictures
64/70
Web 2.0 Expo NYC, 17th September 2008 64
Review
Videos are slow Expensive to review
Review grids
Doesnt cover audio
-
8/14/2019 Expo08nyc Moving Pictures
65/70
Web 2.0 Expo NYC, 17th September 2008 65
N t h?
-
8/14/2019 Expo08nyc Moving Pictures
66/70
Web 2.0 Expo NYC, 17th September 2008 66
Not enough?
Social tools are useful here
-
8/14/2019 Expo08nyc Moving Pictures
67/70
Web 2.0 Expo NYC, 17th September 2008 67
Summary
S i
-
8/14/2019 Expo08nyc Moving Pictures
68/70
Web 2.0 Expo NYC, 17th September 2008 68
Summing up
Flash makes sense
For uploading too
H.264 is probably your best bet today Transcoding software still costs money
Unless youre willing to take on the risk
Progressive download is basicallyawesome
Th d!
-
8/14/2019 Expo08nyc Moving Pictures
69/70
Web 2.0 Expo NYC, 17th September 2008 69
The end!
-
8/14/2019 Expo08nyc Moving Pictures
70/70
top related