s3479 - gpu-accelerated image processing and motion...
TRANSCRIPT
S3479 - GPU-accelerated Image Processing and Motion Analysis Tools for Modern Moving Images
Lance Maurer ( Founder and CEO, Cinnafilm Inc. )
Lance Maurer is the founder, president, and CEO of Cinnafilm, Inc. Prior to launching Cinnafilm in 2007, Maurer spent ten
years in the American space industry as a lead mechanical design engineer. His passion for film and his belief in technical
excellence drove him to create a technology dedicated to properly solving many of the most difficult problems faced by
moving images, with the aim of maximizing the video experience for the end user in all settings. Powered by NVIDIA GPU
computing, Cinnafilm technology has been used to restore some of the world's most important film and television
archives, and is also used to optimize modern digital cinema and broadcast media, delivering the highest quality result in
every format. Cinnafilm is based out of Albuquerque, NM, USA.
CEO and founder Lance Maurer will discuss Pixel Strings®, the scalable, real-time, GPU-accelerated motion vector engine CEO and founder Lance Maurer will discuss Pixel Strings®, the scalable, real-time, GPU-accelerated motion vector engine
that drives Cinnafilm's two flagship products: Dark Energy® and Tachyon™. Dark Energy is used to create high quality
theatrical and HD content by removing noise and film grain without harming image detail and by simulating new film grain
structure. Tachyon, a faster-than-real-time plugin for many third-party transcoding systems, is used to automatically
correct broken cadence and deliver optimum format and framerate conversions, ensuring higher quality output than with
standard transcode solutions alone. Tachyon received the HPA Engineering Excellence Award for 2012. Lance will also
discuss some of his industry-trend predictions for the ever-changing market of multi-media delivery in the years to come.
Session Level: All
Session Type: Talk
Tags: Media & Entertainment
Portions of this presentation may removed from the PDF or online version as certain images not be replicated without
expressed written content of the content’s owners.
GPU-Accelerated Image Processing
and Motion Analysis Tools for
Modern Moving Images
Lance Maurer,CEO and founder of Cinnafilm, Inc.
•Welcome
•Video Clip
•Company introduction
•A Brief history of time
•Cinnafilm technology
•Pixel Strings GPU powered Engine•Pixel Strings GPU powered Engine
•Dark Energy
•Tachyon
•Forward looking thoughts
•Closing
Video Clip
A Brief History of (Cinnafilm’s) TimeA Brief History of (Cinnafilm’s) Time
2007 20092010
Plugins & Plugins &
2011 - Present
TachyonTachyon
2003
“Digital “Digital
Alchemy”Alchemy”
FoundedFounded
2007
Cinnafilm Inc. Cinnafilm Inc.
FoundedFounded
2004
GPU ResearchGPU Research
2009
ARRI ARRI
RelativityRelativity
Plugins & Plugins &
SDK DesignSDK Design
Dark EnergyDark Energy
TachyonTachyon
Standards Standards
TranscodeTranscode
We work with the best in the biz…
Twentieth Century FOX
Warner Brothers
NBC Universal
FotoKem
Deluxe
Turner
ESPN
Sony Network Entertainment
Digital Film Central
Zoetrope Aubrey Productions
MTVN
ING
Music Choice
FixafilmESPN
BBC
WETA
Park Road Post
Cineric
Goldcrest
AMC
DI Factory
Chimney Pot
Fixafilm
ARRI TV
Pixion
JCA
Eclaire
NAC Image Tech
Prasad
The Mill
Bandito Brothers
• Cinnafilm Technology– The “Special Forces” of Image
Optimization
– Used to restore and repurpose priceless, classic films and television
– Used to match textures of various types, from various camera sources, for digital various types, from various camera sources, for digital cinema
– Used to create optimized imagery and compression for modern digital content - any format, any platform
– Provides the very best video experience on any device
Cinnafilm – High Quality Image Processing:• Dark Energy – denoise, texture and film simulation, and
automatic dustbusting; used to restore and optimize for HD
distribution, and used to optimize and texturize digital
cinema for theatrical release
• Tachyon – high-quality, standards and frame-rate converter,
8
• Tachyon – high-quality, standards and frame-rate converter,
cadence and intermixed format correction, high end scaling,
and format conversions; all automated and flexible.
• Pixel Strings – real time motion vector engine calculated on
the GPU – the backbone of Cinnafilm’s high end image
processing solutions.
In a nutshell…
CONVERTED /
OPTIMIZEDPOST
10
Top shelf
Noise/
Grain removal
No loss of image detailNo loss of image detailNo loss of image detailNo loss of image detail
“Absolutely the best grain “Absolutely the best grain
reduction process I have ever reduction process I have ever
seen. With this process I even seen. With this process I even
think you could shoot think you could shoot
cinemascope on super16mm”cinemascope on super16mm”
-- Denny ClairmontDenny Clairmont
Combined with realistic film
grain simulation = TEXTURE
Repurpose MaterialRepurpose Material
Anomaly
Detection & Removal
Automated ProcessAutomated Process
Texture for Digital CinemaCase Study: Act of Valor• Shot ~80% on Canon 5D, in 30 fps
• Partially shot with 35mm film and
Sony Camera
• Dark Energy was used to:• Upres Canon material to 2K
• Remove compression blocking
“Act of Valor”, with permission from Relativity Media
• Remove compression blocking
• Remove image noise
• Remove film grain from 35mm
• Simulate new 35mm film grain for
whole film end to end - straight to
DCP without any film out
• First major motion picture shot
primarily on a handheld camera, #1 at
box office on Oscar Weekend 2012
Tachyon is:• A 100% file-based image processing pipeline, combined with
a powerful transcoding toolset.
• The result of over 9 years of dedicated engineering research
and applied science in the fields of moving image motion
research and frame synthesis.
• A tool that accept any source format in and create any target
16
• A tool that accept any source format in and create any target
format out necessary, mobile to 4K, automatically and faster
than real time.
• A tool that can remove pulldown and intermixed (progressive
and interlaced content) automatically.
• Used for editorial prep (mezzanine normalization) and final
delivery (multi-device)
Why It Matters to Post…• Typical methods for high quality standards conversion
(+transcode) involves clunky and lossy workflows requiring
time consuming trips to/from SDI devices – not always with
the good imaging results. Transcoders themselves do not
provide high quality images for complex conversions provide high quality images for complex conversions
(stuttering, ghosting, blending, etc).
• Visual Media has become an anytime/anywhere delivery
system (in/out); post-production must grow to meet this
paradigm without losing the image high quality that matters.
• Tachyon provides a 100% digital workflow.
What Does Tachyon Do?
•Automated cadence correction
•Composited image error correction
•Mixed mode normalization
•Dark Energy noise reduction
•Dark Energy dustbusting•Dark Energy dustbusting
•Motion based frame rate conversions (any to any)
•Hardware quality upres/downres
•Complex corrections for animation
All in one toolset, and all in a SINGLE pass.
CASE STUDIES…
Case Study ATachyon provided ESPN the first, 100% file-based,
faster than real time solution that combined high
quality format conversion, image scaling, AND
transcoding (Multi 8-GPU server systems) –
European Cup 2012European Cup 2012
1080i25 1080i25 �������� 720p 59.94 (live)720p 59.94 (live)
Case Study B3:2 pulldown
with Progressive CGI Overlay
Tachyon is able to move 3:2 pulldown without
harming or “tearing” progressive overlays.
29.97 i29.97 i––> 23.976p Conversion> 23.976p Conversion
ResultsResults
Results
Results
Results
Results
Results
Results
Results
Results
Results
Results
Results
Results
Results
Case Study CBroken 3:2 Cadence Errors
(3:2 Pattern Not Carried (3:2 Pattern Not Carried
Through Scene Cuts)Through Scene Cuts)Through Scene Cuts)Through Scene Cuts)
Tachyon can detect and remove all 3:2 (and other Tachyon can detect and remove all 3:2 (and other
telecine) patterns, even when the pattern is telecine) patterns, even when the pattern is
inconsistent, broken, or randominconsistent, broken, or random
29.97i 29.97i ––> 23.976p Conversion> 23.976p Conversion
Interlaced Frame 1
Interlaced Frame 2
Progressive Frame 1
Progressive Frame 2
Progressive Frame 3
Interlaced Frame 1
Progressive Frame 1 IMPROPER CADENCE!
Interlaced Frame 1 CADENCE RESTARTED
Interlaced Frame 2
Progressive Frame 1
Progressive Frame 2
Progressive Frame 3
Interlaced Frame 1
Interlaced Frame 2
Progressive Frame 1
Progressive Frame 2
Progressive Frame 3
Results
Results
Results
Results
Results
ResultsOne Frame Before Cut
ResultsOne Frame After Cut
Results
Results
Results
Results
Results
Results
Results
Results
Results
Forward Looking Predictions*
1. Movies will be released in all formats simultaneously
on the same day
2. Product placement and banners will replace
commercial spots on television (soon to be just
“streaming”)
3. 3D will fail theatrically and at home, because 2D
85
3. 3D will fail theatrically and at home, because 2D
monitors will get magnitudes better (market driven)
4. HFR will succeed – but in various ways, and not as we
think. Capture method will harmonize on 4Kx120fps
and content will go straight to cloud for storage and
processing.
5. Dead people will star in movies again.
*TED who?
The best digital video experience, on
every device, always.
THANK YOU
QUESTIONS?QUESTIONS?
www.cinnafilm.com