expo08nyc moving pictures

8/14/2019 Expo08nyc Moving Pictures

1/70

Moving PicturesImplementing Video on Flickr

Cal Henderson


2/70

Web 2.0 Expo NYC, 17th September 2008 2

Hello


3/70


Flickr

Large scale kitten sharing website

Started 2003, launched 2004

5 years old this december

Almost 3 billion photos


4/70


Enter: Video

Video was added this year

Launched April 2008

Many hundreds of thousands of videosuploaded

Many millions of playbacks


5/70



6/70



7/70Web 2.0 Expo NYC, 17th September 2008 7



Video? Thats justlike photos!

-Me, before Flickr Video


11/70


12 4 Steps

4 main tasks

Uploading

Transcoding

Storage

Serving & Playback


12/70


1.

Upload


13/70


Simple upload

Web forms Just like any other file

But files are large / huge


14/70


Issues

Two components for uploading:

Sending from the client

Receiving on the server

Both of these present problems


15/70


Sending from the client

Multiple options

Simple form

Flash

Desktop app


16/70


Simple Forms

Pros Very easy to implement

Works on every browser out of the box

Cons

Upload progress is harder

Slow

Select a single file at once


17/70


Flash

Pros Upload progress is easy

Fast

Multi select of files

Cons

Harder to implement

Flash isnt quite ubiquitous


18/70


19/70


Making Progress

Upload progress Not impossible with plain forms

Just need to be able to query the upload

progress via AJAX

Multiple machines

The VIP issue Enter Perlbal


20/70


Making Progress

Browser

Web 1

Web 2

1. Browser starts upload

2. Web server broadcastsprogress via UDP

3. Browser queries progressvia AJX call

Loadbalancer


21/70


22/70


23/70


24/70


2.

Transcode


25/70


26/70


So many formats

But very few of these formats can beplayed back cross platform

Without special software or hardware

Formats are designed to do one thing well They dont always manage even that

Transcoding puts all videos on an equalfooting


27/70


Video file basics

Most file types are really just containers MOV, FLV, AVI

The data inside can be in multiple formats We call these codecs (encoder + decoder)

Files contains multiple streams

Both audio and video


28/70


Interleave

Audio and video are often interleaved Hence AVI

A video file looks like this: Headers

Video chunk

Audio chunk

Video chunk

Audio chunk

Etc


29/70


Compress

Raw video files are huge A bitmap for every frame

Rarely used, even in post production

At 30 fps, that gets crazy pretty quickly

We dont need to store every frame Static backgrounds dont change (much)

between frames


30/70


Compresssss

Intraframe Treat each frame as a picture

Compress it (just like JPEG)

DCT (Discrete Cosine Transform)

Interframe

Store the differences between frames

Treat the pixels as a 3D array to becompressed


31/70


The IPB

Three frame types: I, P & B

Intra coded pictures

A full raw frame

Predicted pictures Based on a single reference frame

Bi-predictive pictures Based on two or more reference frames


32/70


IPBIPBIPBIPB

Reference frames may be I, P or B

P & B frames may contain a mix of image

data and motion vector displacements

I frames require the most bits

Then P frames

Then B frames


33/70


Bad terminology

We should really say picture (Not frame)

Because of interlacing

Really, we encode fields not frames Picture is the general term

And H.264 contains slices Sub-regions of the field

Macroblocks & Artifacts


34/70


35/70


36/70


Seekable


37/70


38/70


Flash! Woah-oh!

The big question:

Flash?


39/70


40/70


The Flash Player

Flash Player 6

March 2002

Video: Sorenson Spark (H.263)

Audio: MP3

Or ADPCM / Uncompressed

Or Nellymoser Asao


41/70


Second Generation

Flash Player 7

August 2005

Video: On2 TrueMotion VP6

Audio: MP3


42/70


The hot shit

Flash Player 9 (update 3) December 2007

Video: H.264 (MPEG-4 Part 10) w/ container formats from MPEG-4 Part 14

Audio: AAC (MPEG-4 Part 3)

Plus 3GPP Timed Text (MPEG-4 Part 17)


43/70


44/70


H.264

Not proprietary Good compression

MPEG Standard Open, but patented

Patent licenses from the MPEG LA

Unclear how this applies to (L)GPL code

But probably badly

YouTube using it for higher quality streams

iPhones and AppleTV


45/70


46/70


More software

MEncoder libmpcodecs uses libavcodec

VLC libvlc uses libavcodec

So basically the same Different muxing, same codecs


47/70


Free H.264?

Unfortunately, not really

x264 is the only usable one

Its pretty good

MEncoder can use it

Still limited in options at this point

Again, dubiously legal


48/70


Non-free tools

Flash encoder Not automatable

On2 FlixEngine

Creators of VP6 Windows or Linux Some support for H.264

Rhozet Carbon Coder The new hot shit Good H.264 support Windows


49/70


Choices

Video codec Resolution

Bitrate (VBR, CBR)

Keyframes

Audio codec

Channels

Bit depth

Sampling rate


50/70


Doing it at scale

Not really a problem Very easily parallelizable

Amazon EC2 is awesome here

Exactly what it was design for

Grow/shrink as needed

But, per-CPU software licensing


51/70


3.

Store


52/70


Easy!

Really, just like photos

But with bigger files

Same disk layout as any other serving

But the serving part is slower


53/70


But..

Remember the files are huge

Operations take time

More likely to fail halfway through

Checksums are your friend

Do it all asynchronously


54/70


4.

Serve & Playback


55/70


The choice

Streamingvs

Progressive


56/70


Streaming

Pros Easily seekable

Live feeds

Cons

Special server software

Slower to start Firewall troubles


57/70


Progressive download

Pros Just use a web server

Play offline

Firewall/proxy friendly

Cons

Harder to seek ahead (but not impossible)


58/70


Streaming tech

Non flash stuff Well ignore that

Youre using flash, right?

RTMP

Real-time Messaging Protocol

Proprietary (thanks Adobe!)


59/70


RTMP

RTMP - Raw TCP socket stuffs RTMPT RTMP tunneled over HHTP

For firewalls, etc

Flash Media Server

previously Flash Communication Server

Wowza Pro $1000/server


60/70


Open source

Its not all bad

Red5

Java implementation of RTMP server

Mostly feature complete

Beta quality, but usable in production

Facebook


61/70


Progressive

Used by the majority of large sites

Very simple!

Seekable with server support


62/70


Seeking

Serve the FLV starting at a different point

Just add a simple FLV preamble before

seeking into the file

Simple to do in PHP, Perl, etc

mod_flvx for Apache

mod_secdownload for lighttpd


63/70


5.Other considerations


64/70


Review

Videos are slow Expensive to review

Review grids

Doesnt cover audio


65/70


N t h?


66/70


Not enough?

Social tools are useful here


67/70


Summary

S i


68/70


Summing up

Flash makes sense

For uploading too

H.264 is probably your best bet today Transcoding software still costs money

Unless youre willing to take on the risk

Progressive download is basicallyawesome

Th d!


69/70


The end!


70/70

expo08nyc moving pictures

Documents