delivering on the promise of the cloud for digital media, aspera on demand
TRANSCRIPT
Delivering on the Promise of the Cloud for Digital Media Aspera OnDemand
Bhavik Vyas | Director of Cloud Solutions & Partnerships | 14th June 2012
ASPERA SOFTWARE INC
Automate Collaborate Distribute
14 14
Creating next-generation transport technologies that move
the world’s digital assets at maximum speed, regardless of
file size, transfer distance and network conditions.
Transport
Aspera Corporate Overview
Headquarters: Emeryville, CA
Founded: 2004
Ownership: Privately Held
Employees: ~100
Patents: Issued and pending in over 32 countries
Customers: over 1,200 customers, over 10,000 licenses deployed
Markets Served: Media and Entertainment, Federal Government, Life Sciences, Healthcare, Cloud
Computing, Software and Gaming, Financial Services, Legal, eDiscovery, Engineering, Technology,
Telecommunications, Service Providers, Architecture and Design, Enterprise IT
Creators of the fasp™ protocol
• Innovative, patented, highly efficient bulk data transport technology
• Unique and core to Aspera’s high-performance file transfer software suite
• Outperforms software and hardware WAN acceleration solutions
• Ranked first in every WAN transfer throughput benchmark
THE CLOUD IS REAL
THE MEDIA CLOUD?... IT’S HERE.
Thursday, March 8, 2012, New York, NY
Aspera, Amazon Web Services and Zencoder teamed up for a special event at New York City’s Tribeca Grand Hotel on Thursday, March 8th, 2012. During this afternoon of presentations by industry experts,
attendees experienced what is now possible in the cloud for digital media companies.
Our unique, patented faspTM transport technologies provide unparalleled speed, efficiency, and bandwidth control over any size, distance, and network.
TRANSPORT
ASPERA PRODUCT & TECHNOLOGY PORTFOLIO
Complete portfolio of servers and clients for
high-speed data delivery and distribution.
DISTRIBUTE
Global person-to-person and project-based
exchange and collaboration of files
and directories.
COLLABORATE
Web-based application and SDK for creating and managing automated file-
based workflows.
AUTOMATE
ASPERA’S BIG-DATA MARKETS
VIDEO IS BIG-DATA…VERY BIG-DATA.
HOW BIG? BIG.
STANDARD FORMATS
CD DVD Blu-ray
.75 GB 4.7 GB 25 GB
COMPRESSED VIDEO
FORMAT SD 720p HD 1080p HD
Stream rate (Mbps) 1.5 5 8
1 hour of content .7 GB 2.3 GB 3.6 GB
UNCOMPRESSED VIDEO
FORMAT SD 720p HD 1080p HD
Stream rate (Mbps) 125 553 1290
1 hour of content 56 GB 249 GB 581 GB
UNCOMPRESSED VIDEO
FORMAT Digital Cinema 2K Digital Cinema 4K 3D Cinema Stream
Stream rate (Mbps) 1900 3800 5730
1 hour of content 855 GB 1710 GB 2579 GB
A single digital cinema production can be 800K–1M 2K/4K frames
FIRST MAJOR BOTTLENECKS: WAN TRANSFER
TCP PERFORMANCE: BOUND BY DESIGN
Throughput µ1 RTT * p( )RTT = round trip time
p = packet loss rate
TCP powers: • FTP • SCP • RSYNC • HTTP • CIFS • NFS
BIG-DATA and WAN TRANSFER WITH TCP
TCP WAS DESIGNED IN THE EARLY 80’S
• When data was small & bandwidth was limited
• Fantastic for reliable data delivery
• Not fast enough for big-data
TCP IS THE ENGINE THAT DRIVES
• FTP, HTTP & HTTPS
• RSYNC, SCP
• CIFS & NFS
TCP DOES NOT LIKE NETWORK LATENCY/ RTT
• Geographic distance increases latency
• Network congestion increases latency
TCP DOES NOT LIKE PACKET LOSS
• Loss is caused by congestion
• Different network capacity
• Wireless and satellite communications
ASPERA FASP: HIGH-PERFORMANCE DATA TRANSPORT
Maximum transfer speed
• Line-rate transfer performance
• Transfer performance scales with bandwidth
Uncompromising security and reliability
• Secure, user/endpoint authentication
• AES-128 cryptography in transit and at-rest
Congestion Avoidance and Policy Control
• Automatic, full utilization of available bandwidth
• On-the-fly prioritization and bandwidth allocation
Scalable management, monitoring and control
• Real-time progress, performance and bandwidth utilization
• Detailed transfer history, logging, and manifest
Low Overhead
• Optimal end-to-end throughput efficiency
• High performance with large files or large sets of small files
Enterprise-class File Transfer
• Transfers up to thousands of times faster than FTP
• Precise and predictable transfer times
SECOND MAJOR BOTTLENECKS: LOCAL HTTP I/O
2nd Bottleneck — Data Center
1st Bottleneck - WAN
GETTING BIG-DATA IN/OUT OF S3
SOLUTION: ASPERA ON-DEMAND DIRECT-TO-S3
client cargo downloader
mobile apps
connect plug-in
point-to-point
OVERCOMING BOTH BOTTLENECKS
#1 — TRANSFER DATA TO EC2 OVER WAN EFFECTIVE THROUGHPUT
• HTTP transfer over WAN (single stream)
• Typical internet conditions
• 50–250ms latency & 0.1–3% packet loss
• 15 parallel http streams
<10 Mbps
<10 to 100 Mbps
• Aspera fasp transfer over WAN to EC2 up to 1Gbps
(per EC2 Extra Large Instance)
#2 — TRANSFER DATA FROM EC2 TO S3 EFFECTIVE THROUGHPUT
• Single HTTP stream
• 15 parallel http streams • Must use S3 Multipart API
• Must chuck data on r/w
Up to 100 Mbps
Up to 1Gbps
• Aspera S3 Proxy
• With parallel I/O http streams Up to 1Gbps
(per EC2 Extra Large Instance)
#1 + #2 — WAN TRANSFER: ON-PREM TO/FROM TO
S3 EFFECTIVE THROUGHPUT
• Aspera On Demand for AWS
• With parallel I/O http streams Up to 1Gbps
(per EC2 Extra Large Instance)
MEDIA INGEST AND STORAGE ARCHITECTURE
18
BIG-DATA: ACCESSED & DELIVERED BY ASPERA
File transfer
ASPERA SOFTWARE PRODUCT & TECHNOLOGY PORTFOLIO
Transport
Distribute
Complete portfolio of servers and clients for high-
speed data delivery and distribution.
Enterprise and Connect Server
• Universal file transfer server and web-based
interface and directory listing
Client and Point-to-point
• Uni- and bi-directional transfer clients
Connect
• Web browser plug-in
Mobile
• High-speed transfer for mobile devices
Sync
• Highly scalable, multidirectional file replication
and synchronization
Collaborate
Global person-to-person and project-based
exchange and collaboration of files and directories.
faspex™ Server
• Secure digital delivery and collaborative file
transfers with remote users and partners
• Web, email, mobile client options
• Comprehensive administration, user
management & access control
faspex™ Multi-Server / HA
• Automated bi-directional relays between sites
• 3-tier architecture with support for clustering, HA
Cargo
• Automated package downloads
Automate
Web-based application and SDK for creating and
managing automated file-based workflows.
Orchestrator
• Intuitive graphical workflow designer
• File processing decision tree and flow
• Rich and flexible plug-in architecture for third-
party process integration
• Comprehensive library of plug-ins for
transcoding, A/V, QC, archive, notifications
• High volume processing
• Detailed dashboard, workflow, and step-level
progress reporting.
• Open development framework for designing
and integrating automation pipelines
Our unique, patented transport technologies provide unparalleled speed, efficiency, concurrency and bandwidth control over any size,
distance, and network
fasp™
Patented, file-based bulk data transport
fasp-AIR™
3G, LTE and Wi-Fi networks
fasp3™
Next-gen fasp for any block-data
fasp-MC™
fasp for delivery over multicast
Aspera On-Demand S3|Direct
High-speed transfer direct to cloud storage (S3)
Console transport management
web-based management, monitoring, and reporting
Transcoding
Video Quality Control
Watermarking
File Transfer
Encryption
Ad Insertion &
Advertising Planning
IT Management
Antivirus
OPEN PLATFORM 3RD PARTY LIBRARY
ASPERA -AWS CASE STUDIES
• Internationally disperse venues and broadcasters
• Contribution, processing and AWS S3 cloud archive
• Automated processing, routing, notifications and delivery
International Sports Tournament
High Volume Broadcast Operations
• Business critical, revenue generating system
• Thousands of content and metadata files per hour
• Lights-out, hands-off, 7x24x365
INTERNATIONAL SPORTS TOURNAMENT
International Sports Tournament
Media Centers at the Sports Venues
People’s Homes
Globally-DispersedBroadcasters
INTERNATIONAL SPORTS TOURNAMENT – ASPERA FASP™ TRANSFERS
International Sports Tournament
fasp
International Data/Broadcast Center
HD + Metadata
•HD •Metadata
fasp
Content Web Portal
•Low-Res Proxies •Audio •Metadata
fasp
Broadcasters
People’s
Homes
Globally-DispersedBroadcasters
Sports Venue Media Center
INTERNATIONAL SPORTS TOURNAMENT – WORKFLOW ORCHESTRATION
Content Web
Portal International Sports Tournament
International Data/Broadcast Center
1. Detect arrival 2. Notify Operations
Aspera Orchestrator: Controlled
Processing and Distribution Workflow
3. Stage for processing 4. Detect processed content 5. Transfer HD to AWS 6. Transfer Low-Res to Portal
On-going: • R/T monitoring • Notifications
On-going: • Cloud monitoring • HA management
Aspera Faspex API call for remote package creation and distribution
Sports Venue Media Center
fasp
People’s
Homes
Globally-DispersedBroadcasters
HD + Metadata
fasp
fasp
fasp
NETFLIX & THE CLOUD
Vendors Post
Houses Netflix
Staging
Netflix Cloud
Encoding CDN’s
Tens of TB arriving each month
Even bigger spikes before a new region launch
Files up to hundreds of GB in size
Multiple vendors and post houses delivering to us from across the world
Proprietary hardware is painful And doesn’t scale well And doesn’t scale quickly! Vendors claim delivery when they haven’t
delivered
more encoding as each axis grows
Co
nte
nt
Cat
alo
g
Device Ecosystem
2007
2012
2007 2012
At Netflix For better business agility
For unpredictable business growth
We want to use clouds, not build them ▪ Our raison d’etre is movies, not datacenters
In Encoding Operations For response to spikes of content
▪ Big launch days
▪ Entire region launches
▪ New device launches (Wii, PS3, iPad etc)
Vast unified pool of resources
No cloud quite like AWS at the moment
Literally thousands of machines can be transcoding at any one time
Amazon is also a great Netflix partner Open to multiple clouds when they can meet
our scale requirements
We could build something suitable
But is our time best spent here?
And how much time would we need to try and shift an industry in this area?
Buying makes more sense
Focus: We aren’t the only company with this need
Talent density: We spend money to license the technology, but we gain even more talent and bandwidth to solve Netflix specific problems
Industry recognized digital delivery technology
Excellent core product
Optimized for huge file transfers under adverse network conditions
Built in monitoring and controls
Extensive structured logging for diagnosing issues
Strong and responsive support team
Worrying about disk space Months of lag time to scale out Distracting IT with proprietary hardware Building for spikes instead of averages
We get to spend more time growing and less time worrying about the logistics of growth
