s3479 - gpu-accelerated image processing and motion...

S3479 - GPU-accelerated Image Processing and Motion Analysis Tools for Modern Moving Images

Lance Maurer ( Founder and CEO, Cinnafilm Inc. )

Lance Maurer is the founder, president, and CEO of Cinnafilm, Inc. Prior to launching Cinnafilm in 2007, Maurer spent ten

years in the American space industry as a lead mechanical design engineer. His passion for film and his belief in technical

excellence drove him to create a technology dedicated to properly solving many of the most difficult problems faced by

moving images, with the aim of maximizing the video experience for the end user in all settings. Powered by NVIDIA GPU

computing, Cinnafilm technology has been used to restore some of the world's most important film and television

archives, and is also used to optimize modern digital cinema and broadcast media, delivering the highest quality result in

every format. Cinnafilm is based out of Albuquerque, NM, USA.

CEO and founder Lance Maurer will discuss Pixel Strings®, the scalable, real-time, GPU-accelerated motion vector engine CEO and founder Lance Maurer will discuss Pixel Strings®, the scalable, real-time, GPU-accelerated motion vector engine

that drives Cinnafilm's two flagship products: Dark Energy® and Tachyon™. Dark Energy is used to create high quality

theatrical and HD content by removing noise and film grain without harming image detail and by simulating new film grain

structure. Tachyon, a faster-than-real-time plugin for many third-party transcoding systems, is used to automatically

correct broken cadence and deliver optimum format and framerate conversions, ensuring higher quality output than with

standard transcode solutions alone. Tachyon received the HPA Engineering Excellence Award for 2012. Lance will also

discuss some of his industry-trend predictions for the ever-changing market of multi-media delivery in the years to come.

Session Level: All

Session Type: Talk

Tags: Media & Entertainment

Portions of this presentation may removed from the PDF or online version as certain images not be replicated without

expressed written content of the content’s owners.

GPU-Accelerated Image Processing

and Motion Analysis Tools for

Modern Moving Images

Lance Maurer,CEO and founder of Cinnafilm, Inc.

•Welcome

•Video Clip

•Company introduction

•A Brief history of time

•Cinnafilm technology

•Pixel Strings GPU powered Engine•Pixel Strings GPU powered Engine

•Dark Energy

•Tachyon

•Forward looking thoughts

•Closing

Video Clip

A Brief History of (Cinnafilm’s) TimeA Brief History of (Cinnafilm’s) Time

2007 20092010

Plugins & Plugins &

2011 - Present

TachyonTachyon

2003

“Digital “Digital

Alchemy”Alchemy”

FoundedFounded

2007

Cinnafilm Inc. Cinnafilm Inc.

FoundedFounded

2004

GPU ResearchGPU Research

2009

ARRI ARRI

RelativityRelativity

Plugins & Plugins &

SDK DesignSDK Design

Dark EnergyDark Energy

TachyonTachyon

Standards Standards

TranscodeTranscode

We work with the best in the biz…

Twentieth Century FOX

Warner Brothers

NBC Universal

FotoKem

Deluxe

Turner

ESPN

Sony Network Entertainment

Digital Film Central

Zoetrope Aubrey Productions

MTVN

ING

Music Choice

FixafilmESPN

BBC

WETA

Park Road Post

Cineric

Goldcrest

AMC

DI Factory

Chimney Pot

Fixafilm

ARRI TV

Pixion

JCA

Eclaire

NAC Image Tech

Prasad

The Mill

Bandito Brothers

• Cinnafilm Technology– The “Special Forces” of Image

Optimization

– Used to restore and repurpose priceless, classic films and television

– Used to match textures of various types, from various camera sources, for digital various types, from various camera sources, for digital cinema

– Used to create optimized imagery and compression for modern digital content - any format, any platform

– Provides the very best video experience on any device

Cinnafilm – High Quality Image Processing:• Dark Energy – denoise, texture and film simulation, and

automatic dustbusting; used to restore and optimize for HD

distribution, and used to optimize and texturize digital

cinema for theatrical release

• Tachyon – high-quality, standards and frame-rate converter,

8

• Tachyon – high-quality, standards and frame-rate converter,

cadence and intermixed format correction, high end scaling,

and format conversions; all automated and flexible.

• Pixel Strings – real time motion vector engine calculated on

the GPU – the backbone of Cinnafilm’s high end image

processing solutions.

In a nutshell…

CONVERTED /

OPTIMIZEDPOST

10

Top shelf

Noise/

Grain removal

No loss of image detailNo loss of image detailNo loss of image detailNo loss of image detail

“Absolutely the best grain “Absolutely the best grain

reduction process I have ever reduction process I have ever

seen. With this process I even seen. With this process I even

think you could shoot think you could shoot

cinemascope on super16mm”cinemascope on super16mm”

-- Denny ClairmontDenny Clairmont

Combined with realistic film

grain simulation = TEXTURE

Repurpose MaterialRepurpose Material

Anomaly

Detection & Removal

Automated ProcessAutomated Process

Texture for Digital CinemaCase Study: Act of Valor• Shot ~80% on Canon 5D, in 30 fps

• Partially shot with 35mm film and

Sony Camera

• Dark Energy was used to:• Upres Canon material to 2K

• Remove compression blocking

“Act of Valor”, with permission from Relativity Media

• Remove compression blocking

• Remove image noise

• Remove film grain from 35mm

• Simulate new 35mm film grain for

whole film end to end - straight to

DCP without any film out

• First major motion picture shot

primarily on a handheld camera, #1 at

box office on Oscar Weekend 2012

Tachyon is:• A 100% file-based image processing pipeline, combined with

a powerful transcoding toolset.

• The result of over 9 years of dedicated engineering research

and applied science in the fields of moving image motion

research and frame synthesis.

• A tool that accept any source format in and create any target

16

• A tool that accept any source format in and create any target

format out necessary, mobile to 4K, automatically and faster

than real time.

• A tool that can remove pulldown and intermixed (progressive

and interlaced content) automatically.

• Used for editorial prep (mezzanine normalization) and final

delivery (multi-device)

Why It Matters to Post…• Typical methods for high quality standards conversion

(+transcode) involves clunky and lossy workflows requiring

time consuming trips to/from SDI devices – not always with

the good imaging results. Transcoders themselves do not

provide high quality images for complex conversions provide high quality images for complex conversions

(stuttering, ghosting, blending, etc).

• Visual Media has become an anytime/anywhere delivery

system (in/out); post-production must grow to meet this

paradigm without losing the image high quality that matters.

• Tachyon provides a 100% digital workflow.

What Does Tachyon Do?

•Automated cadence correction

•Composited image error correction

•Mixed mode normalization

•Dark Energy noise reduction

•Dark Energy dustbusting•Dark Energy dustbusting

•Motion based frame rate conversions (any to any)

•Hardware quality upres/downres

•Complex corrections for animation

All in one toolset, and all in a SINGLE pass.

CASE STUDIES…

Case Study ATachyon provided ESPN the first, 100% file-based,

faster than real time solution that combined high

quality format conversion, image scaling, AND

transcoding (Multi 8-GPU server systems) –

European Cup 2012European Cup 2012

1080i25 1080i25 �������� 720p 59.94 (live)720p 59.94 (live)

Case Study B3:2 pulldown

with Progressive CGI Overlay

Tachyon is able to move 3:2 pulldown without

harming or “tearing” progressive overlays.

29.97 i29.97 i––> 23.976p Conversion> 23.976p Conversion

ResultsResults

Results

Results

Results

Results

Results

Results

Results

Results

Results

Results

Results

Results

Results

Case Study CBroken 3:2 Cadence Errors

(3:2 Pattern Not Carried (3:2 Pattern Not Carried

Through Scene Cuts)Through Scene Cuts)Through Scene Cuts)Through Scene Cuts)

Tachyon can detect and remove all 3:2 (and other Tachyon can detect and remove all 3:2 (and other

telecine) patterns, even when the pattern is telecine) patterns, even when the pattern is

inconsistent, broken, or randominconsistent, broken, or random

29.97i 29.97i ––> 23.976p Conversion> 23.976p Conversion

Interlaced Frame 1

Interlaced Frame 2

Progressive Frame 1

Progressive Frame 2

Progressive Frame 3

Interlaced Frame 1

Progressive Frame 1 IMPROPER CADENCE!

Interlaced Frame 1 CADENCE RESTARTED

Interlaced Frame 2

Progressive Frame 1

Progressive Frame 2

Progressive Frame 3

Interlaced Frame 1

Interlaced Frame 2

Progressive Frame 1

Progressive Frame 2

Progressive Frame 3

Results

Results

Results

Results

Results

ResultsOne Frame Before Cut

ResultsOne Frame After Cut

Results

Results

Results

Results

Results

Results

Results

Results

Results

Forward Looking Predictions*

1. Movies will be released in all formats simultaneously

on the same day

2. Product placement and banners will replace

commercial spots on television (soon to be just

“streaming”)

3. 3D will fail theatrically and at home, because 2D

85

3. 3D will fail theatrically and at home, because 2D

monitors will get magnitudes better (market driven)

4. HFR will succeed – but in various ways, and not as we

think. Capture method will harmonize on 4Kx120fps

and content will go straight to cloud for storage and

processing.

5. Dead people will star in movies again.

*TED who?

The best digital video experience, on

every device, always.

THANK YOU

QUESTIONS?QUESTIONS?

www.cinnafilm.com