digital media archive - stp.tv€¦ · your broadcast & it convergence partner digital media...

YOUR BROADCAST & IT CONVERGENCE PARTNER

Digital MediaDigital MediaDigital MediaDigital Media

ArchiveArchiveArchiveArchive


Digital Media Archive Overview

2

ApplicationsApplicationsApplicationsApplicationsAutomationAutomationAutomationAutomation

Media Asset ManagementMedia Asset ManagementMedia Asset ManagementMedia Asset ManagementNewsroom Newsroom Newsroom Newsroom

Media NetworkMedia NetworkMedia NetworkMedia Network

Broadcast Devices : Broadcast Devices : Broadcast Devices : Broadcast Devices :

Transmission Transmission Transmission Transmission ServersServersServersServersNewsRoomNewsRoomNewsRoomNewsRoom SystemsSystemsSystemsSystems

NLE editing NLE editing NLE editing NLE editing

Storage Area NetworkStorage Area NetworkStorage Area NetworkStorage Area Network

Archive Archive Archive Archive LibraryLibraryLibraryLibraryTape Library

ExternalisationExternalisationExternalisationExternalisationNearline StorageNearline StorageNearline StorageNearline Storage

Archive Archive Archive Archive GUIGUIGUIGUI

Data MoverIntegratedIntegratedIntegratedIntegrated

TranscoderTranscoderTranscoderTranscoder

Ethernet NetworkEthernet NetworkEthernet NetworkEthernet Network

Control Network

API Archive API Archive API Archive API Archive

Archive Archive Archive Archive Manager Manager Manager Manager

Data Data Data Data MoverMoverMoverMover

Option Option Option Option TranscoderTranscoderTranscoderTranscoderAnalyserAnalyserAnalyserAnalyser

New Media New Media New Media New Media IPTV, VOD, 3GPIPTV, VOD, 3GPIPTV, VOD, 3GPIPTV, VOD, 3GP

BrowserBrowserBrowserBrowser

Tape Library

DisasterDisasterDisasterDisaster recoveryrecoveryrecoveryrecovery


We would like to present you

through

concrete existing projects an

overview

of

some topics of archive systems


Some Case Stories in order to speak about :

• Different Files Format – Why ? What to choose ?

• Different Storage – Why ? – What to choose ?

• Different Library – Why ? What to choose ?

• Different Drives – Why ? – What to choose ?

• Media HSM – IT HSM ?

• Different Applications ?

• Transcoding – Analyse – Monitoring – Browse : In-Out ?

• Disaster Recovery – Multiple Site – Multiple Robot

• Workflow


Central Play out system of RTL Group company, migration from a classic tape based play-out system to a fully digital file based automated system and very large centralized Archive System.

21 channels on air

Historical and Playout Archive System


• achieved its migration from a classic tape based play-out

system to a fully “ tapeless ” digital file based automated

system with centralized media archiving

• Linked with National Archive project and used as disaste r

recovery site

• Remote Acces and link from Brussels (RTL-TVI ) to the

Archiving System and

• Already 200.000 hours of Video



In the thematic Channels control room, 1 operator drives 10 channels of transmission with an automation system



The Storagetek PowderHorn Library can store up to 1.000.000 hours !



Up to 450 exchanges of cartridges per hour, the system isfocused on access time, throughput and reliability



Our methodology ...

• Customized proposalin close relation with customer requests

• Confidentiality

• Respect of Budget & Time Frame

• Testing Design & Performances

• Follow up & Training

• Insurance of investments in the long run

• Specific procedures in order to provide to the customer the foreseen functionalities


One of the First Fully File Based

National Broadcaster


Full Total File Base Operation

Full Complete Tapeless Integration Production - Postproduction – News – Archive – Play Out :

- Ingest on central media storage and local work cent er

- Integration with NRCS

- Low res search and edit

- Hi res conforming/editing

- Playout of Rundown

- Very large Archive System integrating also the IT Storag e


6 Avid Unity ISIS

GMII

Airspeed

Omneon

Arvid (lo res)

45 AvidCraft Editing

Option

Ardendo Browse and edit

SDI

Omneon

NRCSAvid iNews

SGT automation

or iNews Control Air

variant

SDI

Ardendo dual file ingest

MediaFile

Avid DMSTelestream FlipFactory

WoConline

Global Media Archive

HistoricalArchive

IT Archive Storage

IBM GPFS2 PETA BYTE DISK


Global Multipurpose Archive System

The Archiving System

is used for different purposes :

- Historical Media Assets

- Global Media Assets : News

PlayOut

Production -

Postproduction

- IT Storage



The Archiving System

is driven by different applications:

- DIVArchive from Front Porch Digital

for Media

- Tivoli from IBM

for IT



The Archive Robot

is virtualy divided in different partitions:

- News – Play Out

- Production - Postproduction

- Historical Media Assets

- IT



The Archive Robot

is handling different type of drives

- LTO3 : Historical Media Assets

- LTO 4 : IT Storage - Production

- Titanium : Play Out – News


Different Files FormatWhy ? What to choose ?

In that project what is used :

- Historical Assets : MPEG2 – Long GOP 15 Mb/s

- MPEG2 MXF OP1a @ 50 Mb/s

- MPEG2 MXF OP ATOM - For Editing not in Archive

- MPEG1 General Browse

- WM9 Browse Historical Assets

- 3G - Mobile TV

- Tomorrow HD …


Different Files Format

RTL uses :

- Historical Assets : MPEG2 – Long GOP 10 & 15 Mb/s

- MPEG2 MXF OP1a @ 50 Mb/s

- MPEG1 for browse

- Tomorrow HD ….

So : 1 application = 1 format….or more

Several applications = several format

Digital Archive is Format Agnostic !

Giving thus Protection of the future and of investment


Storage Tools

20

Media RobotMedia RobotMedia RobotMedia Robot

Tape Library

Hard DriveHard DriveHard DriveHard Drive

Tape Library

DisasterDisasterDisasterDisaster RecoveryRecoveryRecoveryRecovery –––– Media RobotMedia RobotMedia RobotMedia Robot

Off LineOff LineOff LineOff Line

On Line On Line On Line On Line ServersServersServersServers


Different StorageWhy ? What to choose ?

21

Media from the archive library can be ejected

and stored on shelves (or offsite) while still

tracked by the Media HSM

Large and expandable robotic tape or optical

archive media device controlled by the Media

HSM

Less expensive spinning disk controlled by

the CSM. Requires a “copy” operation to

ONLINE disk for content access

Spinning disk connected directly to broadcast

devices (video servers, editing platforms,

newsroom systems, etc)

Description Cost and AccessStorage

Tier

Storage Cost: Very Cheap

Access Speed: Slow

OFFLINE

Storage Cost: $$

Access Speed: Fast

ARCHIVE

Storage Cost: $$$

Access Speed: Very Fast

NEARLINE

Storage Cost: $$$$$

Access Speed: Immediate

ONLINE

Online

Storage Management



- Hard Drives Advantages - Disadvantages :

Faster

Capacity is growing

Price is decreasing but still higher and needs maintenance

Reliability weaker

- Tape Robot Advantages - Disadvantages :

Best reliability

Price still much lower

Much Larger Capacity

Capacity is growing ( but little less than Hard Drive )

Price is decreasing ( but less than Hard Drive )



VRT uses :

- Hard Drives : - Standard Raid Storage SUN 6140 - 20 TB

( Low res WM9 historical assets – on line )

- Extremely large and Redundant Storage

IBM GPFS : 2 PetaByte !

All Media Assets type near on line

- Digital Robot : - Very large Robot SL 8500 Storagetek

after migration from L700 ( all assets )

- ADIC : first IT Robot



Internal Users

Filing - 6 x 1 inch

3 xLTO 3Drive

Diva ActorManager 72GB

Content Checking

&Sub clipping

Low Res Server 20 TB WM9

PATRIMONIALARCHIVING PROJECT

Preview ingest station

Restore

Filing Server

Append

Archivists

TelestreamFlipFactory

(WM9)

SwitchSwitch

Restore(Preview+)

DBOS Loader & Media storageinterface

DBOS DeviceController

DBOS Server

VRT Supernet

12xLTO4

Drive for IT

Externalisation

Image Processing


6 Avid Unity ISIS

GMII

Airspeed

Omneon

Arvid (lo res)

45 AvidCraft Editing

Option

Ardendo Browse and edit

SDI

Omneon

NRCSAvid iNews

SGT automation

or iNews Control Air

variant

SDI

Ardendo dual file ingest

MediaFile

Avid DMSTelestream FlipFactory

WoConline

Global Media Archive

HistoricalArchive

IT Archive Storage

IBM GPFS2 PETA BYTE


RTL uses :

- Hard Drives : - Standard Raid Storage SUN - 40 TB

- Digital Robot : - 2 Very large Robot, located on 2 sites

- PowderHorn Storagetek

- SL 8500 Storagetek ( CNA )

Pay Tv Channel uses :

- Hard Drives : - Standard Raid Storage

- Digital Robot : - 2 Robot, located on 2 separate sites

- Sony Petasite

- SL 700 Storagetek


YOUR BROADCAST & IT CONVERGENCE PARTNER27

Hard Disk - NearLine Storage

Type of Disk Related to access time and reliability :

SATA or SCSI

Type of Raid technology : Raid 1– Raid 3 - Raid 5 – Raid 6

And from simple storage to quite complex :

- IBM GPFS

- Exanet

- Isilon

- MediaGrid

YOUR BROADCAST & IT CONVERGENCE PARTNER28

Stockage NearLine ( Disques ) - Exanet

Comparatif Storagetek


Different Librairies


So No single answer, choice is related to the requested

fuctionalities, reliability, and……price.

For Archive Robot choice one important topic is :

Ability to grow and to integrate different type

of drives and as far as possible ability to

integrate future drive with future technology



Which Media / Support to choose :

DVD – Holographic – Data Tape ?


Storage Performance at 50Mbps

37


VRT uses :

- LTO 3 Historical Assets ( MPE2 Long GOP )

Browse ( WM9 )

This migrated already from LTO2

- LTO4 General Media ( Play out – Production ….)

MPEG2 MXF - General Browse ( MPEG1 )

- LTO4 IT This migrated already from LTO3

Different DrivesWhy ? What to choose ?


RTL uses :

- 9840 Historical – Play Out

Short Assets only -( MPE2 Long GOP )

Why = very short access time ( but small capacity)

- 9940 Historical – Play Out

All other Assets -( MPE2 Long GOP – MPEG1 )

Why = large capacity –good access time

- Platinum : All items for new customer

CNA National Archive Institution



Mixed Drives

• For different type like fast access and small capacity and

lower access but high capacity

• Typically SUN/StorageTek T9840C and T9940B or IBM LTO3,

LTO4, Sun/StorageTek T10000

• For a migration due to new technology:

– LTO2 to LTO3, LTO4


So for Archive Robot actually Tape Data, are still

the best fit for the Broadcast applications

But new technologies are coming, like Holographic

and performance are increasing.



Media HSM versus IT HSM

• Support for the widest base of asset management, automation,

video server, storage and archive devices

• Dedicated applications Not an HSM solution, but rather a content

aware, intelligent storage management solution

• IT HSM are Archive Manager which can of course manage the

different components of the archive but without dedicated tool

for Media


Storage Resources Same for Media or IT HSM

Servers• Robot Tape Libraries

– Quantum/Adic

– IBM

– Qualstar

– Sony

– SUN/StorageTek

– SpectraLogic

• Tape drives

– LTO-2

– LTO-3

– LTO-4

– 9840C (STK)

– 9940B (STK)

– T10000 (STK)

– S-AIT1 (SONY)

Storage Resources• DVD jukeboxes

– Asaca

(Matsushita drives)

• Disk

– Any SAN storage

– Any NAS storage

– Omneon Mediagrid

– Seachange BML


Storage Resources

SERVERSwith IT HSM

Servers

Storage Resources• FTP Servers


Storage Resources

SERVERS with Media HSM

Servers • Avid

• EVS

• Incite

• Leitch

• Omneon

• Pinnacle

• Quantel

• SeaChange

• SGI

• Sony

• Thomson GV

• Vela

• Any FTP servers

Storage Resources


• Ardendo

• ……

Limited Existing Interface with Automation / MAM/ Traffic System

IT HSM Interfaced Applications


• Anywhere

• Ardendo

• Artesia

• Ascential

• Aveco

• Blue Order

• CIS

• Dalet

• Harris/Arkemedia

• Harris/Encoda

• Harris/Invenio

• Harris/Louth

• Ibis

• Incite

• Konan

• Marquis

• Mediagenix

• Netia

• Omnibus

• PebbleBeach

• Pharos

• Probel

• Proconsultant

• Publitronic

• S4M

• SGT

• Sobey

• Tedial

• TGV/ContentShare

• TGV/AuroraNews

• TMD

Many : Automation / MAM/ Traffic System

Media HSM Interfaced Applications


Broadcast EnvironmentDifferent Applications can be interfaced

Media ManagementMedia Management

PostPost--ProductionProduction

Ingest, Playout,Ingest, Playout,

Production ServersProduction Servers

AutomationAutomation

Mid and Long term Storage

Traffic SystemTraffic System

Media Archive ManageMedia Archive ManageMedia Archive ManageMedia Archive Manager r r r


Media HSM Features

Standard Features

• Near line disk and tape storage • Off-line storage (Externalization)• Tape de-fragmentation• Priorities management on requests• Queue Manager Analysis (QMA)• Estimated Time Completed (ETC)• Multiple Instances Object • Co-location of objects • Object copy• Tape Groups & Disk Array grouping• Library partitioning • Flow control management • Louth VACP (Harris)• SNMP agent• Metasource• Automatic spanning of content • TRU and FTU Utilities• Application Programming Interface

(API)• Graphical User Interface (GUI)

Other Features

• Application Filtering• Avid Unity support• Drop Folder Monitor (DFM)• Database Clustering• Export / Import • Mixed drives• Multiple robots • Object Transfer Utility (OTU)• Partial Restore • Multiple Parallel Restore • Storage Plan Manager (SPM)• Transcode Plug-in• Quality Analysis


ResourcesAssignment &Management

ContentLocalization

Storage/Archive Manager: The Functions

ContentDuplication

MigrationPolicies

Interface ToVideo

Equipments

FaultsIdentification& Recovery

ErrorsReporting

EventsLogging

ContentGrouping

Interface ToStorage

Technologies

DataMovement

ContentTranscoding

VideoPartial

Retrieval

PriorityManagement

LoadBalancing

Interface ToLegacy

Systems

Interface ToBroadcast

Applications

CacheManagement

TapeDe-Fragment

TapesSpanning


Partial Restore (1/2)

• Optimized access time for movie excerpts

• Parsing and selection direct from tape (no disk cache

required)

• Time-code offsets can be specified for video clips:

– Format or server type

– Start time-code

– End time-code

– New Name for the restored clip

• Uses frame index when available


Partial Restore (2/2)

– AVID/Pinnacle • MediaStream: MPEG-2 LGOP Program or System Stream)

– Final Cut Pro• QT Embedded

– Leitch• Nexio LXF: DV25, MPEG-2 I-Frame and LGOP

– Matrox based servers AVI : DV25/DV50, MPEG-2 50I and LGOP– Omneon Spectrum or Mediagrid:

• Quicktime: D10, MPEG-2 I-frame, DV25• MXF OP1a D10, DV25, DV50

– Quantel• Clipbox: D10• sQserver MXF OP1a IMX30, DV25, DV50

– Sony e-VTR MXF D10

– Thomson Grass Valley:

• Profile XP, PDR in GXF: DV25, DV50, MPEG2-50I, or MPEG-2 LGOP, MJPEG

• K2 in GXF: MPEG-2 I-Frame and MPEG-2 LGOP (SD & HD), DV25, DV50 and MJPEG in SD

• K2 in MXF OP1a: IMX/D10 (30 and 50), DV25 and DV50

– Generic MXF OP1a : D10 (IMX) and DV25, DV50– Standard MPEG2 TS format


Manager Queue Processing

• Since DIVArchive V5.10.6

• Concept of “priority tier” : value which can be added to a request

priority

• Manager regularly collects the tape-id mounted in drives

• Requests requiring a tape already mounted will see their priority

increased of the “priority tier”

• Only 1st tape is taken into account when object is spanned


Tape Partitionning

Other

…

Set

Set 1 Set 2

Channel A Channel B

…

news

movies

2 months

…

Low

Res

series

Group


• Back up

• Disaster Recovery

• Multiple Robots

• Multiple Sites

Redundancy Possibilities


Tape Back up

Tape Library Tape Library


Back up Robot



Disaster Recovery–Multiple Sites

x2

Fiber Channel

Switch

Application

Nearline Tape Storage

Main Robot

x2DIVArchive

Control GUI

Library Control

DIVArchive Manager

Network

Switch

x2

DIVArchive Actor2

Transcoder

DIVArchive Actor1

Backup Manager

Nearline Disk Storage

DIVArchive API

Tape Library

x

Nearline Tape Storage

Backup Robot

Ingest and playout Servers

Video ServerVideo ServerVideo Server

Tape Library

x

x2x2

IP network

Main Site Backup Site

Backup Manager

Actor 3

DIVArchive Actor4

Transcoder

Network

Switch



Disaster RecoveryFull Production Mirroring Multiple Sites


Transcoding or Analysis Plug-in

• Based on Transcoding or Analysis engines

• Automatic Transcoding initiated by Media HSM

• Rules can differ according to initial Media name (Group/Array) or object category

• Typical workflows: – Archive high-res material on disk array

– Transcode to MPEG4 or WM9 for browsing

– Make available low res on a browsing server

– Archive low-res material on tape

– Create another high res instance on tape

– Delete instance on disk


• Networking and storage systems must be designed to support advanced file based workflows

• Bandwidth bottlenecks can be easy introduced and cripple the system – or worse!

• Storing your content at excessive bitrates (ie. uncompressed) can be costly and perhaps unnecessary

• Ensure the system can incrementally scale in bandwidth, storage and technology for new business/revenue opportunities ( future HD for ex )

System Bandwidth must

fit the requested Workflow

61 YOUR BROADCAST & IT CONVERGENCE PARTNER


Our methodology ...

• Customized proposalin close relation with customer requests

• Confidentiality

• Respect of Budget & Time Frame

• Testing Design & Performances

• Follow up & Training

• Insurance of investments in the long run

• Specific procedures in order to provide to the customer the foreseen functionalities


• Independent from manufacturers

• In order to cope with different suppliers we have set up a

• Quality and Co-responsability Agreement

• Based on architecture and functinalitiesagreed with customer

Our Methodology


SpasiboThank You

digital media archive - stp.tv€¦ · your broadcast & it convergence partner digital media...

Documents