aws re:invent 2016: high performance cinematic production in the cloud (mae304)

© 2016, Amazon Web Services, Inc. or its Affiliates. All rights reserved.

November 28, 2016

High Performance Cinematic Production in the Cloud

Richard Welsh, CEO, Sundog Media ToolkitKonstantin Wilms, Principal Solutions Architect, AWS

MAE 304

What to Expect from this Session

Production on AWS

• Macro Perspective

• Building Blocks for Post-Production Pipelines

Sundog Media Toolkit

• Platform, Workflows, Scale, Production Techniques

• Billy Lynn’s Long Halftime Walk

• Challenges & Benefits

noun | film·mak·ing | \-ˌmā-kiŋ\Filmmaking

The process of making a film incorporating five major stages:Development, Pre-Production, Production, Post-Production, & Distribution

Benefit from massive economies of scale

Adapt your media storage and compute needs using AWS

Stop guessing capacityHandle unpredictable & bursty media needs programmatically

Trade capital expense for variable expense

Pay for media you store and process, as you go

Go global in minutesGlobal availability instantly,

with no commit

Stop spending money on running & maintaining data centers

Focus your resources on your media needs

Increase speed and agilityShorten time to market,

test new approaches

Why AWS for Post Production?

Playout & Distribution

Visual Effects & Editing Analytics

DAM & Archive

Digital Supply Chain

Publishing

OTT

Acquisition

AWS Media Workloads

Media – ALL IN – to the Cloud

The New Normal

Media – ALL IN – to the Cloud

Where Do I Begin?

Buzzword?Specific scope

Interoperable services Allow for rapid innovation

De-Coupling Micro & Composable Services

Performance Optimization

Storage

Render FarmTranscodeMetadata

Pipeline and License Manager

Graphics Artist Workstations

• Content has gravity • Network Bandwidth• Hybrid/All-in Cloud• I/O Performance

• Ability to burst at a very short notice

• Cost? • Performance• Security

• License mobility/Elasticity• Dependency Management

(hybrid scenario)

• Interactivity• High-Performant Storage• Hardware Support

Post-Production Components

All-In Cloud Enabled Domino Effect

PIPELINESMapping Studio Workflows to AWS & Partner Offerings

Ingest Storage Editing Processing

PUSH OR PULLCONTENT & ARTIFACTS

ACCESS SOURCE FILESLOCATED IN AMAZON S3

CREATE, EDIT, & COLLABORATE IN THE CLOUD

SCALE OUT ON ELASTIC CAPACITY FOR PROCESSING

Content production and post-production companies are leveraging AWS to accelerate and streamline creative, editing, and rendering workloads with highly scalable cloud computing and storage.

Post-Production Pipeline

Ingest

PUSH OR PULLCONTENT & ARTIFACTS

AWS Services

• Amazon S3 Transfer Acceleration: Fast data transfer to S3

• AWS Direct Connect: Dedicated network connectivity

• AWS Snowball: Petabyte-scale data transfer

• Amazon CloudFront: Global content delivery network

• AWS Storage Gateway: On-prem-to-cloud storage connectivity

• Elemental Cloud Appliances: On-prem video processing for ingest

Partners


AWS Services

• Amazon S3: Highly scalable, durable object storage

• Amazon Glacier: Secure, durable, petabyte-scale data archival

• Amazon EFS: Simple, elastic, scalable file system

• Amazon Elastic Block Store: Persistent block storage

• AWS Marketplace: Partner solutions for storage

Partners

Storage

ACCESS SOURCE FILESLOCATED IN S3


AWS Services

• Amazon EC2: Scalable CPU/GPU computing capacity

• Amazon Workspaces: Virtual desktops in the cloud


• Amazon EBS: Persistent block storage


Partners

Editing

CREATE, EDIT, & COLLABORATE IN THE CLOUD


Processing

SCALE OUT ON ELASTIC CAPACITY FOR PROCESSING

AWS Services

• Amazon EC2: Scalable CPU/GPU/Spot computing capacity


• Amazon EBS: Persistent block storage


• Elemental Cloud: On-demand, scalable video processing

Partners


STORAGEContent storage options on AWS for post-production pipelines

1 PB raw storage

800 TB usable storage

600 TB allocated storage

400 TB application data

Cloud Storage

Storage Capacity

Storage Services

Amazon EBS

Block storage for use

with Amazon EC2

Amazon S3

Massively scalable

storage & front-end

11 9’s of durability

IA for infrequent access

Internet scale

storage via API

AWS Storage Gateway

S3,

Amazon

Glacier

Integrates on-premises

IT and AWS storage

Amazon Glacier

$0.004/GB/month

11 9’s of durability

Multiple copies across

different DCs

Storage for archiving and

backup

EC2

EBS

Amazon EFS

Share File storage for

use with Amazon EC2

EC2

EFS

Massively scalable

Pay for what you use

High Performance

1000’s of hosts

Up to 16TB/volume

Up to 20K PIOPS

SSD-backed

Cold & Throughput

Encryption

BLOCK FILE OBJECT OBJECT

VPCVPCVPC

Asset Lifecycle

Long-Term Archive

All tiers accessible through a single API

Oldest content trickles down to Amazon Glacier

automatically to save cost

Amazon S3

S3 Infrequent Access

Amazon Glacier

Life

Cyc

le P

olic

ies

Near-Line Storage

Frequently Accessed

Shared Storage Shared Storage

On-prem StorageAWS

Direct Connect

Storage Cache

Amazon S3

Lustre on EC2

Avere on EC2

EFS

AWS

Direct Connect

Hydrate workers

EC2 Spot

FXT on-prem

Access Models

Tightly vs. Loosely Coupled Storage

Post-Production tasksNLE/Review & Approve/VFX(Hybrid, on premises, or in Cloud)

Raw AssetsSecondary Copy

Amazon Glacier in another Region

Raw AssetsPrimary Copy

Amazon Glacier

Finished (Golden Copy) ArchivePrimary Copy

Amazon Glacier

Ass

et M

anag

emen

t (D

AM

/MA

M)

(Hyb

rid

, on

pre

mis

es, o

r in

Clo

ud

)

Active ArchiveAmazon S3(Standard)

Active ArchiveAmazon S3

(Infrequent Access)

Secondary & Tertiary CopiesAmazon Glacier in another Region

A secondary offsite copy of the active archive – usually coldUsed for DR purposes

Med

ia S

up

ply

Ch

ain

(Hyb

rid

, on

pre

mis

es o

r in

clo

ud

)

B2

B D

eliv

ery

ove

r D

X o

r In

tern

et Amazon Glacieringest

Affiliates/Third-Party Vendors

(Primary Copy) AWS Region 1 (Secondary & Tertiary Copies – optional) AWS Region 2 or on-premises

Production Archive

deliver

Object Store(Amazon S3)

Block Store(Amazon EBS)

Shared FS(Amazon EFS)

Performance

Per-Operation Latency Variable Lowest, Consistent Low, Consistent

Throughput Scale Multiple GBs per sec Single GBs per sec Multiple GBs per sec

DataAvailability/Durability

Across Multi-AZs Redundant Within an AZ Across Multi-AZs

Characteristics

Access 1-1000’s of Clients A Single EC2 instance 1-1000’s of Clients

Use Cases

Active Archive and Lifecycle to Cold ArchiveMedia Processing (Transcoding, Rendering, QC, Farms) Depending on Application Conformity

DatabasesBoot VolumesMedia Processing (Transcoding, Rendering, QC, Farms) SingletonStreaming

Data AnalyticsMedia Processing (Transcoding, Rendering, QC, Farms) DistributedContent ManagementWeb Serving/Publishing

Workload Specific Storage

EDITINGUsing AWS compute for content & media post-production

Editing in the Cloud

Amazon WorkSpaces

NVIDIA K520 GPU, 16GB RAM, 8 VCPUs

Heatmaps

H.264 vs. PCoIP

• PCoIP codecs were designed for interactive applications that are often static and are dominated by computer-generated content

• Video codecs (like H.264) are designed for passive streaming of constantly changing, natural-image camera-captured content

Designed for graphically intense cloud delivery

• The PCoIP protocol delivers fundamentally better image quality for text and 3D graphics

• The PCoIP protocol is optimized for cloud delivery:

• Consumes 60-80% less bandwidth for text and graphics

• Requires 50+% less server CPU,leaving more cycles for the applications

Cloud Editing Protocols

PROCESSINGUsing AWS compute for content & media post-production

Reserved

Make a low, one-time

payment and receive

a significant discount

on the hourly charge

For committed

utilization

On-Demand

Pay for compute

capacity by the hour

with no long-term

commitments

For spiky workloads,

or to define needs

Spot

Bid for unused capacity

at a Spot Price -

fluctuates based on

supply & demand

For time-insensitive or

transient workloads

Consumption Models

Compute IntensiveIntel ES-2666 v3 (Haswell) Optimized specifically for EC2

Memory IntensiveLowest price point per GiB of RAM

GPUs 40K CUDA cores192 GB of video memory

Enhanced NetworkingHigher PPS, lower network jitter, low latency

I/O IntensiveSSD Storage, EBS Optimized

High Storage24 x 2000 GiB per instance

Amazon S3

Amazon SQS Queues (orchestrated by Amazon SWF)

M4

M3

C4

C3 I3G2

R4

P2 D2R3

X1

Task Optimization

Accelerated Computing

Rendering, Compositing, Encoding, Non-Linear Editing, VR

*12-core E5-2697-v2 Intel Xeon

Why GPUs?

• Ubiquitous – available at high scale to

application developers worldwide

• Enabling a high degree of parallelism – each

GPU has thousands of cores

• Consistent, well-documented set of APIs (CUDA,

OpenACC, OpenCL)

• Supported by a wide variety of ISVs and open

source frameworks

• Deploy as Spot for compute workloads

• Single K80 10x faster than CPU on applications*

CPU vs. GPU RenderingScheduling• Multi-threaded, multi-process• Select instance based on thread #s• Or map across GPUs:

CUDA_0: Frame 1-25CUDA_1: Frame 26-50

• Increase in Scheduler Complexity

Dependencies• File loading - CPU/storage• Package, meshes, textures - CPU• 3D rendering - CPU/GPU• Compositing - CPU• Image export - CPU/storage

SUNDOG MEDIA TOOLKITAn On-Demand Cloud-Based Post-Production Platform

Asset Management

Workflows

Collaborative Review

Playlist Mastering

3D Visualization

Services Platform

• Full Stack Public

• Hybrid

Storage Architecture

Tool IdentityWorkflow DependenciesTool Handling / ReportingData I/OMetadata HandlingContent Handling

ProcessCore

Workflow Tool Shell

Encryption Points

Workflow Manager Worker (Primary)UI

Worker (Clone)

Worker (Clone)

Worker (Clone)

Worker (Clone)

Worker (Clone)

Worker (Clone)

Worker (Clone)

Worker (Clone)

Worker (Clone)

Worker (Clone)

Worker (Clone)

Worker (Primary)

Workflow Creation Workflow Analysis Worker SpawningAvailability Zone Spanning

(availability/outage)

Job Completion

Worker Flushing

Sta

tus/P

rogre

ss R

ep

ort

ing

Worker (Clone)

Worker (Clone)

Worker (Clone)

Worker (Clone)

Worker (Clone)

Worker (Clone)

Primary zone (a)

Zo

ne

cZ

on

e b

Scale Decisions:

Tool process

Dependencies (intra and inter-

tool)

Size- bit depth

- resolution

- layers

Frame count E

lastic r

eso

urc

e e

xp

an

sio

n

Workflow & Scale Decision Process

Iterate

Inspectpixels

Vicinity (spatial)Scene-BasedObject-BasedIncluding 3D

Colour (spectral)

Motion (temporal)

Analyse to sub-pixel level the statistical consistencyof that pixel with the expected value

Make a fine adjustment

Multi-Mode Analysis Super PixelReconstruction

Spectral Reconstruction

Temporal Reconstruction of Detail

Spatial Reconstruction

Source

Blurred

Sharpened

TrueImage Processed

Original = Baseline

Blurred = PSNR high, SSIM low

Sharpened = PSNR low, SSIM high

TrueImage = PSNR high, SSIM high

Compression Efficiency

• 120 Frames Per Second

• 3D (240 real FPS)

• 4K (4096x2160 – not UHD “4K”)

• High Dynamic Range (16-bit float)

Uncompressed Data Rate

= 40 x average cinema

= 55 x average HD

Billy Lynn’s Long Halftime Walk

Billy Lynn’s Long Halftime Walk

Iterations

Da

ta p

oin

ts p

er

fra

me

(4

K)

Data points

per pixel

Trillion

(x1012)

10.1

21.9

28.0

56.1

Challenges

• Scaling to 120fps 3D 4K uncompressed data processing

• Breaking traditional workflows (i.e., traditional thinking)

• Staying Safe

Conclusion

Customer Feedback

“Sundog cloud based workflows have enhanced our ability to deliver an improved product for our clients and are a massive step change in how we go about our business.”

“TrueImage exceeded all expectations and is the primary reason that the movie looks so good on the big screen … it actually had a powerful effect on the editorial and story telling”

Conclusion

Find Us!

sundogtools.com

@sundogmt

Thank you!

Remember to complete your evaluations!

Industry Pre-Day: Monday 11/28• MAE303 - Discovery Channel's Broadcast Workflows and Channel

Origination on AWS• MAE302 - Turner's Cloud Native Media Supply Chain for TNT, TBS, Adult

Swim, Cartoon Network, CNN • MAE301 - Accelerating the Transition to Broadcast and OTT Infrastructure

in the CloudRegular Conference: Tuesday – Thursday 11/29 – 12/1• ARC406 - Encoding Artifacts to Emmy Awards: Taking on Terabyte-Scale, 1-

Gbps, 4K Video Processing in the Cloud• CMP312 - Powering the Next Generation of Virtual Reality with Verizon• CTD305 - Media Delivery from the Cloud: Integrated AWS Solutions for

Premium Over the Top (OTT) Content• STG302 - Deep Dive on Amazon Glacier

Related Sessions

aws re:invent 2016: high performance cinematic production in the cloud (mae304)

Technology