E-Guide

How the media and entertainment

industry is backing up ‘big data’

Big data is a term often used in the storage industry today. As

companies accumulate more and more data, they must find more

efficient ways to manage and store it. But the media and

entertainment industry has been managing this big data for years.

Read this SearchStorage.com E-Guide to see what technologies and

strategies they use to cope with their big data issues.

Sponsored By:

SearchStorage.com E-Guide

How the media and entertainment industry is backing up ‘big data’

Sponsored By: Page 2 of 9

E-Guide

How the media and entertainment

industry is backing up ‘big data’

Table of Contents

Lessons learned, applied in the media and entertainment data storage

industry

Resources from Atempo

SearchStorage.com E-Guide

How the media and entertainment industry is backing up ‘big data’

Sponsored By: Page 3 of 9

Lessons learned, applied in the media and entertainment data storage industry

By Terri McClure

Managing big data is all the talk across the IT landscape nowadays, but it’s a topic the

media and entertainment industry has been dealing with longer than just about anyone. The

last five years have seen a mass migration to digital media formats for both audio and video

capture, production and delivery—not just for new content but for older, celluloid content

that is at risk for deterioration and loss.

Digital file formats are the ―gift that keeps on giving‖ in the media and entertainment

vertical industry. Vendors need to support file formats that can be delivered via many

various endpoint devices. Take Pandora Radio for instance. Pandora has more than 75

million registered users in the United States, and the company’s personalized radio stations

play on more than 200 devices, including PCs, smart phones, iPads, and in-home connected

devices such as televisions, Blu-ray players, table-top radios and digital media players.

Delivering audio across so many endpoint devices means that Pandora needs to store data

in different formats tailored to play on these devices, drastically multiplying its storage

requirements.

And it’s not just the number of files and the multiplicative factor related to storing multiple

formats that create a challenge in media and entertainment–file formats are growing in size

and density. Just like the pixel density in digital photography increases the size of our

photos, supporting new high definition audio and video formats also increases the size of a

finished movie or song. For example, the shift from standard definition to high definition

video increased storage requirements by a factor of 6X. And when considering technologies

like 3D video, storage requirements can double because there are essentially two copies of

the movie made—one copy for each eye. Depending on video format and compression, a

finished two hour movie can range from 1.5 GB to 8 or 10 GB in size, and the raw footage

from multiple takes, special effects and CGI edits can easily consume a petabyte of capacity

if filmed in 3D. Additionally, sound tracks in multiple languages and the ―out take‖ and

commentary segments produced also add on to the overall storage requirements.

SearchStorage.com E-Guide

How the media and entertainment industry is backing up ‘big data’

Sponsored By: Page 4 of 9

The media and entertainment industry has benefitted from the digitization of media–once

data gets cataloged onto spinning disk it can be retrieved almost immediately in response to

world events. With the passing of a celebrity like Liz Taylor or Michael Jackson, the

television station that can produce the footage for the lifetime retrospective has a

competitive advantage when it comes to drawing advertisers and can charge more money

for air time.

Digital media is also playing a big role in professional sports. Most major league baseball

teams record and catalog every play–who was on the field, who is pitching, the pitch count,

batter count, who is on which base and what pitch is thrown. Later analysis can be used to

calculate statistical probabilities or just study videos for ―tells‖ that may indicate if a pitcher

might throw to first to hold a runner or make a pitch. The same goes for other professional

sports, such as football and basketball.

So it comes as no surprise that the media and entertainment market has come under so

much scrutiny by the big storage vendors. The major vendors are attacking the market from

their positions of strength, with disk-based clustered NAS solutions, while a whole slew of

tape vendors are seeing a second wind thanks to the streaming performance of tape and

the cost effectiveness of tape as an archive medium. Even cloud storage is playing a key

role; the media and entertainment industry is an early adopter of cloud storage technology

for enabling collaborative workflows and content distribution. We’ll tackle these technologies

one at a time, starting with clustered NAS.

Why clustered NAS?

Media and entertainment post-production work is largely done via file-based workflows so

that the editing team can share access to the raw footage. There are a number of clustered

file systems available that excel at the streaming performance required to support the

heavy sequential workload required to meet performance demands in these environments–

such as Quantum Corp.’s StorNext, FalconStor Software’s HyperFS and IBM GPFS. There are

also a number of integrated systems from vendors like BlueArc Corp., DataDirect, EMC

Isilon, NetApp Inc. and Panasas Inc. And solutions like the Avere Systems Inc. scale-out

NAS services platform can add scale-out manageability and performance, as well as edge

caching, in front of legacy scale-up NAS.

SearchStorage.com E-Guide

How the media and entertainment industry is backing up ‘big data’

Sponsored By: Page 5 of 9

These systems are seeing success in this market because most traditional scale-up NAS

solutions simply cannot meet the throughput or capacity demands of streaming and storing

very large files–they are designed to serve up a high number of small files and lots of file

IO, rather than a smaller number of large files that require high file streaming throughput.

Clustered NAS systems are designed to scale-out into a multi-node architecture by adding

processing power and network connections to increase throughput in line with additional

capacity. Many nodes can be deployed to work in parallel to stream large files over all of the

available processors and network connections, providing very high throughput rates. With

most systems, as nodes are added, they are used and managed as a single system,

absorbing new capacity as it is added and automatically load balancing across the cluster.

These systems can typically scale into multi-petabyte capacities to meet the burgeoning

storage demands of today’s media and entertainment industry.

Archiving takes center stage

In the media and entertainment industry, the term archive often applies to the working data

set used in post production. The term deep archives is used for long term finished products.

The active project often leverages clustered NAS for performance, but once a project is

completed, it is moved to a long term storage medium such as dense disk-based storage,

tape or cloud storage. It is not just the finished project that needs to be archived. Over

time, sequels or outtake reels are produced, or the anniversary of an iconic movie saga like

Star Wars comes along that requires easy access to raw footage that can be woven together

for a compelling story or peek behind the scenes. The value of all of the footage is

immeasurable–it simply cannot be recreated, but it can be leveraged for value time and

again.

The volume of raw footage that needs to be archived for a single project can be massive.

Multi-tier archive solutions and dense disk-based object storage archives are often used to

manage copies across the tiers of storage, but they don’t back them up—there are simply

not enough hours in the day to perform a traditional backup of these massive rich media

data repositories. When considering the graphic intensive productions such as animated

movies, many of these companies would rather backup the raw files, which tend to be

SearchStorage.com E-Guide

How the media and entertainment industry is backing up ‘big data’

Sponsored By: Page 6 of 9

smaller in size and are more suitable for deduplication versus the rendered files, or the final

product, because they know that in a worst-case scenario these could be recovered by

resubmitting the raw files to the rendering farms again. But if these rendered files were lost,

it would cause a disruption, which is why archiving the rendered files is a good solution for

those types of companies.

Finally, it’s almost impossible to recreate many of these images. If a nature documentary is

out shooting a hawk diving for its prey, the camera guys don’t necessarily get to say ―hey,

let’s retake that sequence.‖ The hawk just doesn’t take orders like that. And if that take is

lost, it takes time and money to go shoot it again.

From a functional perspective–and because of the file types used in media and

entertainment, end users find that newer disk-based solutions with deduplication technology

really don’t help them at all–so tape is still a big player for them because they can efficiently

stream the data due to the sheer size of the files. Plus, storing tape on a shelf is more cost

effective than a spinning disk.

The cloudification of media and entertainment

Since conventional data protection methods do not always apply because of the sheer

volume of data in media and entertainment, cloud storage is gaining some popularity. Cloud

storage solutions, like those offered from Nirvanix Inc. and Amazon, can offer cost-effective

multi-site replication to provide an added layer of data protection.

Cloud storage can also offer a cost-effective long term archive strategy that keeps the

massive amounts of raw footage available and accessible for easy reuse. While more

expensive than storing tapes, cloud storage carries the advantage of faster time to access

archive footage.

Cloud storage can also help on the content distribution front–both Amazon and Nirvanix

allow creation of customer buckets (Amazon) or child accounts (Nirvanix) that allow content

creators to upload files specifically for their customers to access and download. This is

priceless in collaborative workflows where a CGI house may need to work on a portion of a

film and can have near immediate access to the required files.

SearchStorage.com E-Guide

How the media and entertainment industry is backing up ‘big data’

Sponsored By: Page 7 of 9

The bigger truth

The next wave of storage wars will be over rich media content–not just because of the

massive advances in media and entertainment technology, but the entire wave of the

―consumerization of IT‖. There is more multimedia and user generated content making its

way into IT and delivered out via any number of endpoint access devices. And that

generates a ton of storage revenue opportunity.

But make no mistake–the media and entertainment market is not your typical IT shop. The

buyers are often a line of business managers concerned with cost and time to market–

meaning performance and efficiency are at the forefront. And traditional backup policies and

methods don’t apply–the working data sets are too big to be backed up in a typical backup

window–restore would be a costly nightmare if files needed to be rebuilt from incremental

backups.

PROTECTING AND PRESERVING CONTENT IN ALL STAGES OF DIGITAL MEDIA WORKFLOWS

Preserve with Atempo Digital ArchiveAtempo Digital Archive allows you to migrate content from primary storage to near-line and deep archival storage—disk, tape and cloud. With Atempo Digital Archive, you can maximize storage capacity today while preserving content long-term to monetize in the future.

Protect with Atempo Time NavigatorAtempo Time Navigator provides enterprise-class data protection for complex, heterogeneous environments, with high performance and proven scalability to the petabyte level.

www.atempo.com

SearchStorage.com E-Guide

How the media and entertainment industry is backing up ‘big data’

Sponsored By: Page 9 of 9

Resources from Atempo

White Paper: The Digital Archive's New Leading Role in Media Workflows

Customer Story: How REELZCHANNEL Delivers More Content with Less Effort

Customer Story: How the National Film Board of Canada Preserves 70 Years of

Film History

About Atempo

Atempo enables organizations to preserve and protect digital information simply and

effectively, across any infrastructure, on any platform, over long periods of time. Atempo's

archiving solutions deliver policy-based and workflow-driven management of rich media

files, e-mail and other digital assets to maximize the efficiency and performance of storage

systems and reduce long-term storage costs. Atempo's fully-integrated software portfolio

also includes backup and recovery of heterogeneous servers, workstations and laptops

throughout the enterprise — from the data center to remote offices. Atempo serves

thousands of customers around the world through a sales and support network of over 200

resellers and partners.