introduction of file based workflows 111004 vfinal
DESCRIPTION
How does File-based workflow, MXF, SOA, fits together and slowly bring you to Web 3.0TRANSCRIPT
Introduction for File-based Workflows
MJ Drouin, VSF meeting
October 4th 2011, Atlanta
Who Am I? • Director with 15 years of Media experience, I lead technical teams of up to 30 people and 3 managers to
deliver most aspects of the business - Operations, Customer Services, Information Technologies, Project Management, Engineering, and Procurement. I developed expertise in managing cable, wireless, satellite, transmission, engineering and software development. I worked at Bell, Teleglobe, Intelsat, CBC and as a expert consultant.
• Professional Engineer, Masters in Technological Management, ITIL foundation, PMP
• I have extensive experience in speaking since the 90's (ouf!), subjects were always a mix of technologies, commerce and management.
– Past engagements included: Cloud computing training for Content Providers (2011), My video is a file, now what? (2010), DVB-H: Video over IP goes wireless (2010), Service Oriented Architecture (SOA): Tutorial for the Broadcast Community (2010), Pitfalls of SW development: What every executive should know (2009), 8 weeks on Twitter: I love it! (2009), The Future of Broadcast Archive Systems (2008), DVB-H: Video over IP goes Wireless (2006), HDTV distribution (2005), Video over IP (2004) and, Several broadcast technical training (1994-1998): Video 101, Digital Video and Compression 201, Video testing techniques 301.
• http://www.linkedin.com/in/mjdrouin
• http://twitter.com/mjdrouin
2011-10-04 2 MJ Drouin© - VSF Meeting
2011-10-04 3 MJ Drouin© - VSF Meeting
Agenda • Traditional vs File-based Workflow • Implementation of File-based
Workflows – Roadblocks – Pains – Facilitators
• MXF: The Basics – Advantages – SMPTE MXF standards – Partition pack – Played Material vs Saved File – Operational Patterns – MXF on your HDD: AS-02 – Types of MXF files
• FIMS: Phase 1 objectives – SOA – Broadcast workflow in small services:
Cloud Computing?
• What’s Next? – Paper, hand-made, multi-interface
unrelated environment – Linking Open Data Project – The rest of the world is moving: Web
1.0, 2.0 and 3.0 – Linked data: Creating new things
2011-10-04 MJ Drouin© - VSF Meeting 4
What is this all about: Traditional broadcast workflow
5 MJ Drouin© - VSF Meeting 2011-10-04
Theoretical file-based broadcast Workflow
6
Archives
Low-res Proxi
2011-10-04 MJ Drouin© - VSF Meeting
Roadblocks for its Implementation
• Skills
• Split IT & Broadcast departments
• Requires – again – a brand new infrastructure
• Lack of standards, interoperability
• Mostly specialized broadcasters who really do it
• Metadata management not integrated, yet
2011-10-04 7 MJ Drouin© - VSF Meeting
File-based Broadcasting concerns
File-based Production
Metadata Vendor File
compatibility
Consumer vs Professional
Stds
Auto-fill
Persistence
Workflows Standards
Post & Edit
To air
8
Archiving
DRM File multiple format
Asset Mngmt
Q.A.
AMS interop
Content Re-use
2011-10-04 MJ Drouin© - VSF Meeting
Pains during its Implementation • Ingress • Egress • Equipment flexibility and
age • Not-really standard
standards such as AVI • Scheduling &
Automation: Yet again, other software needed to make it work…
• Video on computer vs on screen
• Quality of source not easily separated from encoding & file transfer
• OU Data circuits
• Software offering = only large systems – not yet as services
2011-10-04 9 MJ Drouin© - VSF Meeting
Facilitators • More than 1000 pages of standards (SMPTE/AMWA/FIBS/EBU) • Economics • Non-real-time transmissions • Bandwidth growth in Telecom and Public internet • Mobile tools: Intelligent phones & Tablets • Content creators pushing for ubiquity • MXF …
Hybrid environment is here to stay a little longer…
2011-10-04 MJ Drouin© - VSF Meeting 10
2011-10-04 MJ Drouin© - VSF Meeting 11
Advantages MXF • Shared essence
• Everybody that works on it builds metadata content
• Open standard – independent of platform, OS, vendor, etc
• Create a web of linked essence/data
• But…
• What’s real, today?
2011-10-04 12 MJ Drouin© - VSF Meeting
SMPTE & MXF Base documents
• SMPTE 377M: The MXF File Format Specification (the overall master document)
• SMPTE EG41: MXF Engineering Guide (A guide explaining how to use MXF)
• SMPTE EG42: MXF Descriptive Metadata (A guide explaining how to use descriptive metadata in MXF)
Generic containers
• SMPTE 379M: Generic Container (the way that essence is stored in MXF files)
• SMPTE 381M: GC-MPEG (how to store MPEG essence data in MXF using the Generic Container)
• SMPTE 383M: GC-DV (how to store DV essence data in MXF using the Generic Container)
• SMPTE 385M: GC-CP (how to store SDTI-CP essence data in MXF using the Generic Container)
• SMPTE 386M: GC-D10 (how to store SMPTE D10 essence data in MXF using the Generic Container)
• SMPTE 387M: GC-D11 (how to store SMPTE D11 essence data in MXF using the Generic Container)
• SMPTE 382M: GC-AESBWF (how to store AES/EBU and Broadcast Wave audio essence data in MXF using the Generic Container)
• SMPTE 384M: GC-UP (how to store Uncompressed Picture essence data in MXF using the Generic Container)
• SMPTE 388M: GC-AA (how to store A-law coded audio essence data in MXF using the Generic Container)
• SMPTE 389M: Generic Container Reverse Play System Element
• SMPTE 394M: System Item Scheme-1 for Generic Container
• SMPTE 405M: Elements and Individual Data Items for the GC SI Scheme 1
Operational patterns
• SMPTE 390M: OP-Atom (a very simple and highly constrained layout for simple MXF files)
• SMPTE 378M: OP-1a (the layout options for a minimal simple MXF file)
• SMPTE 391M: OP-1b
• SMPTE 392M: OP-2a
• SMPTE 393M: OP-2b
• SMPTE 407M: OP-3a, OP-3b
• SMPTE 408M: OP-1c, OP-2c, OP-3c
Metadata, dictionaries and registries
• SMPTE 380M: DMS1 (a standard set of descriptive metadata to use with MXF files)
• SMPTE 436M: MXF Mappings for VBI Lines and Ancillary Data Packets
• SMPTE RP210: SMPTE Metadata Dictionary (the latest version is available here: http://www.smpte-ra.org/mdd/index.html )
• SMPTE RP224: Registry of SMPTE Universal Labels
2011-10-04 13 MJ Drouin© - VSF Meeting
MXF – The basics: Partition Pack
MXF wrapper
Audio/video /metadata
Physically Divides File
Time to bytes offset
Identification Operational Pattern Storage Information Package & Track Structure - EDL structures and Timecode Essence Description - Codecs, channels, Descriptive Metadata 2011-10-04 14 MJ Drouin© - VSF Meeting
Played material/saved material/timecodes
Audio
Video
Time Code
Audio
Video
Time Code 1
Time Code 2
Metadata Audio
Metadata Video
Material Package
File Package (using SMPTE UMID)
Tracks
Position bits vs timecodes
Tracks
Metadata Content 2011-10-04 15 MJ Drouin© - VSF Meeting
Operational Patterns
• How to use and reconstruct
• MXF files
2011-10-04 16 MJ Drouin© - VSF Meeting
AS-02: MXF files on your HDD
alice_v0.mxf
-
AS02_clips
alice_a0.mxf
alice_a1.mxf
alice
-
alice.mxf
media
alice_v2.mxf
-
alice_vbi0.mxf
alice_vanc0.mxf
show_format.doc
thumbnails
certificate.xml
QC_report.xml
thumb0.jpg
thumb1.jpg extra 2011-10-04 17 MJ Drouin© - VSF Meeting
7 Types of MXF files … Already! • Op-1a – Single interleaved file (MPEG2 + some VANC/VBI + uncompressed
audio) • XD-CAM HD – Single interleaved file (MPEG2 @ specific bitrate + 2-8 mono
AES) • OP-Atom (AVID & AAF) – AAF synchronised component file (post) • OP-Atom (Panasonic P2) – XML synchronized component file + constrained
by format & codec (cameraedit) • dCinema – xml synchronized JPEG X’Y’Z component file (cinema
workflows) • AS-02 – MXF synchronized MXF component files with manifest and shim
for interop good for reading • AS-03 – MXF synchronized MXF interleaved files (very strict for playout
servers) 2011-10-04 18 MJ Drouin© - VSF Meeting
And still… pretty complex
2011-10-04 19 MJ Drouin© - VSF Meeting
2011-10-04 20 MJ Drouin© - VSF Meeting
FIMS Framework – Specification #1
2011-10-04 21 MJ Drouin© - VSF Meeting * From FIMS Media SOA Framework v1.0
Service Oriented Architecture
XML, Web Services, MXF
2011-10-04 22 MJ Drouin© - VSF Meeting
Now that you split the workflow in Services…
• You can outsource it and grow utilisation of 3rd party ressources … on demand?
• Cloud computing?
Could be another Operational Model for MXF?
May make it easier for Broadcasters to invest/or not in the required additional infrastructure…
2011-10-04 MJ Drouin© - VSF Meeting 23
Standards for Media Workflow = Enabler for Cloud Computing
Video Encoding,
Image processing
Video Storage,
Metrix, Elasticity Security, Access,
multi-screen
conversions
2011-10-04 24 MJ Drouin© - VSF Meeting 3rd party services – on-demand
But… What’s next?
2011-10-04 MJ Drouin© - VSF Meeting 25
This is still our world …
2011-10-04 MJ Drouin© - VSF Meeting 26
Possible to look more like that?
• Don’t you think it will look more like that than pure SOA with a super-centralized enterprise bus & Servers?
2011-10-04 MJ Drouin© - VSF Meeting 27
The rest of the world is… • Web 1.0 = content to all • Web 2.0 = bidi web, content from users as well as
producers (read / write) web • Beyond? Web 3.0
– web = services – Intelligent data = Structured info = Links and filters – Beyond PC: Mobile, IPTV, etc – Web of things: Machines sending info on the web
2011-10-04 28 MJ Drouin© - VSF Meeting
Isn’t it what a lot of people do at Broadcasters?
• Linked Data enables data to be opened up and connected so that people can build interesting new things from it. (via Tim Berners-Lee) Linked Data is Blooming; ReadWriteWeb, May 2009
2011-10-04 29 MJ Drouin© - VSF Meeting