© copyright khronos group, 2004 - page 1 nab 2004 briefing neil trevett senior vp market...
TRANSCRIPT
© Copyright Khronos Group, 2004 - Page 1
NAB 2004 BriefingNAB 2004 Briefing
Neil TrevettNeil TrevettSenior VP Market Development, 3DlabsSenior VP Market Development, 3Dlabs
President, Khronos President, Khronos
Jean-Luc DeryJean-Luc DeryProduct Development Manager, Discreet Product Development Manager, Discreet
Chairman, OpenML Working GroupChairman, OpenML Working Group
© Copyright Khronos Group, 2004 - Page 2
OpenML – Market Catalyst OpenML – Market Catalyst For Media AuthoringFor Media Authoring
OpportunityFalling costs of media
authoring hardware and software by moving from
custom designs to off-the-shelf components
ProblemNo open, cross platform
standards to enable interoperability
dmSDK
OpenGL
EvolutionOpen Source Programs,
Adopters Programs,Specification evolution
SolutionThe OpenML standard provides software interoperability between
Platforms, Operating systems, Hardware devices
OpenML 1.0 SDK announced here at
NAB 2004!
OpenML 1.0 brings key technologies into open
standardsDeployment
OpenML 1.0 SDK enables and encourages
widespread industry implementation
© Copyright Khronos Group, 2004 - Page 3
Open standards bodyCreating royalty-free API standards
Leveraging OpenGL-related technologiesBuilding markets for dynamic media
Small footprint 3D graphics API for
embedded systems
Media API for Dynamic Media
Authoring
MEDIAAUTHORING
MEDIADELIVERY
© Copyright Khronos Group, 2004 - Page 4
Khronos Member MomentumKhronos Member Momentum50 member companies and growing50 member companies and growing
Promoting Members
Contributing Members
© Copyright Khronos Group, 2004 - Page 5
Dynamic Media ProgrammingDynamic Media ProgrammingSynergistic integration of OpenML and OpenGL Synergistic integration of OpenML and OpenGL • What OpenGL is to GRAPHICS - 2D, 3D and image processing…
…OpenML is to MEDIA - video and audio processing• OpenML and OpenGL together create a unified programming environment
- With seamless integration of graphics and media
• Tightly integrated graphics, video and audio = DYNAMIC MEDIA
Workstation3D graphics
Professional video and audio authoring
OpenML enables interoperability of media authoring applications across: Platforms, Operating Systems, Hardware devices
© Copyright Khronos Group, 2004 - Page 6
?
The OpenML Digital Media SolutionThe OpenML Digital Media SolutionProgramming EnvironmentProgramming Environment
ApplicationsApplications
WindowWindow SystemSystem OpenGLOpenGL
®® MLdc ML
Synchronization
OpenGL Extensions
2D Graphics 3D Graphics Displays Video Audio
© Copyright Khronos Group, 2004 - Page 7
Technical Strategy for OpenML 1.0Technical Strategy for OpenML 1.0Move PROVEN technologies into Open StandardsMove PROVEN technologies into Open Standards
UST/MSCSynchronization
#4: Synchronization of diverse media streams (graphics, audio, video)
SGI’s dmSDKDigital Media Handling
#1: Configure and control the Input/Output and processing of video and digital audio ML
OpenGL ExtensionsIntegrating graphics and video
#3: Enable OpenGL hardware to add more value to the media authoring workflow
SGI’s Xdc extension to X11Display Control
#2: Configure and control display devices MLdc
Problem Solution
© Copyright Khronos Group, 2004 - Page 8
AdopterPackagesAdopter
Packages
Consortium Participation ModelConsortium Participation Model
PromotersPromoters
ContributorsContributors
AdoptersAdopters
RatifiedSpecifications
RatifiedSpecifications
Members
ImplementersImplementers
Participation and vote in Working Groups
Conformance Tests etc. Fee for access – fees waived for members
Anyone can download specifications and SDKs and implement royalty-free products
Conforming products can use trademark
Openly and publicly distributed – free of charge
Any organization can join. All work is done in WORKING GROUPS: WGs
Produce Final Specifications and other products. WGs open to ANY member. All work-in-progress is Confidential to the
membership
Contributor rights PLUS guaranteed seat on the Board. The Board decides on strategy: what Working Groups, how spend the budget, ratification of Final
Specifications
SDKsSDKs
Libraries, utilities and examples. Licensed for commercial use – free of charge
© Copyright Khronos Group, 2004 - Page 9
OpenML 1.0 SDK ReleaseOpenML 1.0 SDK ReleaseEnabling and encouraging OpenML AdoptionEnabling and encouraging OpenML Adoption
Executable ML and MLdc Executable ML and MLdc libraries and utilities to run libraries and utilities to run OpenML applications for OpenML applications for IRIX, Linux and WindowsIRIX, Linux and Windows
ML and MLdc library ML and MLdc library source and header files source and header files to enable new OpenML to enable new OpenML applications to be builtapplications to be built
Source code for example Source code for example audio and video device audio and video device
modules with full modules with full documentation documentation
Source code for example Source code for example OpenML applications OpenML applications
with full documentation with full documentation
SDK
Available today at Available today at www.khronos.orgwww.khronos.org
Free of charge for Free of charge for commercial use commercial use under Khronos under Khronos
licensing agreementlicensing agreement
Can be used as a Can be used as a starting point for starting point for
building commercial building commercial applicationsapplications
© Copyright Khronos Group, 2004 - Page 10
OpenML Future Development OpenML Future Development Encouraging widespread industry adoptionEncouraging widespread industry adoption
OpenML 1.0 SDK
OpenML Open Source Program
OpenML 1.0 Adopters Package
OpenML 2.0Specification
NAB 2004 SIGGRAPH 2004 IBC 2004 NAB 2005
Utilities, libraries and source code to enable rapid implementation
To enable the industry to co-develop support for many devices across all
platforms
Conformance Tests to enable conformant products to use the OpenML trademark
Next generationMedia LibrarySpecifications
© Copyright Khronos Group, 2004 - Page 11
Possible OpenGL/OpenML RoadmapPossible OpenGL/OpenML Roadmap
OpenGL 1.2Pbuffers for off-screen storage. Imaging Library for Filtering, Color correction, Blending, Image
enhancement, Color space conversion
OpenML 1.0OpenGL extensions for:
Synchronization, Interlace processing, Chroma resampling,
Flexible alpha processing.ML MLdc UST/MSC
OpenGL 2.0High-level shaders for
flexible and powerful image and video processing
OpenML 2.0Enhanced media synchronization, extended video format and color
space processing
OpenGL 2.1Advanced memory management and synchronization
© Copyright Khronos Group, 2004 - Page 12
OpenML Outreach and EngagementOpenML Outreach and EngagementIndustry Involvement will drive momentumIndustry Involvement will drive momentum• SDK Released - OpenML Forums at www.khronos.org
- Adopters can request, discuss programming and the evolution of OpenML and the SDK
• OpenML - for multimedia application vendors- Complete software environment for advanced media on wide selection of systems and devices
• OpenML – for hardware vendors- Gives more ISVs better access to hardware device capabilities
• OpenML - for end users- Better functionality across a wider range of platforms at cheaper prices
The Khronos Group encourages your participation to help grow the dynamic media market!
© Copyright Khronos Group, 2004 - Page 13
Any Questions?Any Questions?
OpenGL is a registered trademarks of Silicon Graphics, Inc., and OpenML is a trademark of Silicon Graphics, Inc., used with permission by the Khronos Group. All other product names, trademarks,
and/or company names are used solely for identification and belong to their respective owners.
www.khronos.org
© Copyright Khronos Group, 2004 - Page 14
OpenML 1.0 OpenML 1.0 Technical OverviewTechnical Overview
© Copyright Khronos Group, 2004 - Page 15
?
The OpenML Digital Media SolutionThe OpenML Digital Media SolutionProgramming EnvironmentProgramming Environment
ApplicationsApplications
WindowWindow SystemSystem OpenGLOpenGL
®® MLdc ML
Synchronization
OpenGL Extensions
2D Graphics 3D Graphics Displays Video Audio
© Copyright Khronos Group, 2004 - Page 16
OpenML’s “ML” API OpenML’s “ML” API Media device control interfaceMedia device control interface• API to configure/control video/audio device input, output and processing
- Low level - provides device independence without imposing policy
• Perfect environment for implementing higher-level software - Such as QuickTime
OpenInventorOpenInventor QuickTimeQuickTime
OpenGLOpenGL MLML
Low levelLow level(ioctl interface)(ioctl interface)
3D Media
© Copyright Khronos Group, 2004 - Page 17
OpenML’s “ML” API OpenML’s “ML” API Market-proven audio/video processingMarket-proven audio/video processing• ML enables abstracted query and control of media processing devices
- E.g. IEEE1394 cameras, video transcoders, OSS audio processors
• ML is used to setup processing pipelines between I/O and devices- With optimized buffering and synchronization between devices and the application- Asynchronous communication between application and media devices
ApplicationApplicationMedia Input Jacks Media Output Jacks
TranscoderTranscoder
Control over input/output parameters such as data format and quality levels
Control over transcoder operation
Buffered, synchronized message-based data communication
Synchronization and control
© Copyright Khronos Group, 2004 - Page 18
MLModulesMLModulesDevice Driver FrameworkDevice Driver Framework
MLML
ApplicationApplication
ML
MLmodules
user
kernel
h/w AudioAudio
ossoss
Ossaudio
Ossaudio
VideoVideo
V4lV4l
V4lvideo
V4lvideo
1394 camcorder1394 camcorder
13941394
S/WXcoders
S/WXcoders
Higher-level libraryHigher-level library
Creates device independent abstractions
© Copyright Khronos Group, 2004 - Page 19
OpenML’s “MLdc” APIOpenML’s “MLdc” APIAbstracted display control Abstracted display control • Allows an application to control what is displayed on monitors
- Sophisticated level of control beyond most windowing systems
• MLdc sets up channels to drive each display screen- Channels that are driven from sections of physical frame buffers - Number and type of supported channels depends on the display device
• Properties that can be controlled include:- The number of channels and where their display data is derived- Interrogating monitor information- Precise video positioning- Video format query and settings- Vertical retrace rate - Interlaced vs. progressive- Gamma ramp control- Video sync sources- Genlock state notification
Frame Buffers
Progressive ScanComputer MonitorProgressive ScanComputer Monitor
Progressive Scan HDTV Monitor
Progressive Scan HDTV Monitor
Interlaced Video Monitor
Interlaced Video Monitor
Channels
© Copyright Khronos Group, 2004 - Page 20
Media SynchronizationMedia SynchronizationA critical part of any authoring environmentA critical part of any authoring environment• Need to ensure that all media streams start at the same time
- And stay in synch
• Non-trivial problem to solve- Devices from various manufacturers - Device latencies are unpredictable- OS scheduling latencies
Audio Device AAudio Device A
Video Device AVideo Device A
Audio Device BAudio Device B
Gfx Device BGfx Device B
Gfx Device AGfx Device A
ApplicationApplication Do all the streams come out in sync?
© Copyright Khronos Group, 2004 - Page 21
The UST/MSC/SBC SolutionThe UST/MSC/SBC SolutionAccurate media synchronizationAccurate media synchronization• UST - Unadjusted System Time – time reference for software and devices
- 64-bit monotonic counter with nanosecond resolution, microsecond system-wide accuracy
• MSC - Media Stream Counter - incremented for each channel sample- i.e. Video - per frame, Graphics - per retrace, audio - per audio sample, e.g. 44.1KHz
• SBC – Swap Buffer Count - incremented on graphics buffer swap- Not necessarily in synch with retrace
UST
Graphics devices
Digital audio/video devicesSBC
OpenGLOpenGL MLML
MSC
Output graphicFrames/ samples
Video/AudioIO samples
© Copyright Khronos Group, 2004 - Page 22
Using UST/MSC/SBCUsing UST/MSC/SBCRobust Synchronization – even on non real-time OSRobust Synchronization – even on non real-time OS• Every media sample is time-stamped
- Timestamp can be accessed through OpenGL and ML APIs for temporal analysis and control
• Application can synchronize output events to UST- Enabling precise timing and synchronization in non-real-time operating systems
• Applications can detect dropped samples- And take corrective action
© Copyright Khronos Group, 2004 - Page 23
OpenML’s OpenML’s OpenGL ExtensionsOpenGL ExtensionsVideo imaging and 3D graphics integrationVideo imaging and 3D graphics integration• OpenML is built around OpenGL• OpenML and OpenGL: A tight and synergistic integration
© Copyright Khronos Group, 2004 - Page 24
OpenML and OpenGLOpenML and OpenGLTight and synergistic integrationTight and synergistic integration• Compliant OpenML assumes a robust OpenGL implementation
- OpenGL 1.2.1 with GLX 1.3 (Linux/UNIX) or WGL (Windows)
• OpenML extends OpenGL with video primitives and DCC rendering features- Defines a set of OpenGL extensions that OpenML compliant hardware must implement
• Pbuffers are the fundamental method of integrating graphics and video- Enables buffering and off-screen processing of video data
• OpenGL imaging extensions are used to process video streams- Filtering, color correction, blending, image enhancement, color space conversion
• Significant opportunity for OpenGL hardware vendors to add more value- Use OpenGL graphics hardware for 3D and video acceleration
© Copyright Khronos Group, 2004 - Page 25
OpenML’s OpenGL ExtensionsOpenML’s OpenGL ExtensionsBringing together the worlds of 3D and videoBringing together the worlds of 3D and video• Synchronization control
- Primitives to control UST/MSC synchronization from OpenGL- Functions added: WaitForMSC, SwapBuffersMSC, GetSyncValues
• Interlace support- Based on two previous extensions: SGIX_interlace, INGR_interlace_read
Enables reading and writing of the frame buffer while skipping every other line- Enables processing (e.g., convolution) on interlaced video streams
• Texture color mask management to enable advanced compositing- Enables independent loading of RGB images and Alpha masks in texture memory- Recently promoted from an SGI extension to an ARB extension- Works for textures like the OpenGL color mask works for the color buffer
• Chroma resampling to support pixel formats such as YCrCb- Upsampling and downsampling when converting between YCrCb and RGB- Sampling options are: Replicate, Zero fill, Average