or how to handle multimedia on linux
TRANSCRIPT
Solving the puzzle of multimedia on linux
or how to handle multimedia on linux
Muriel Moscardini
Fluendo SA
Agenda
1. Introduction 2. Multimedia 3. Linux as a solution 4. Stakes of multimedia on linux 5. What solutions do we have 6. In practice… 7. Market solutions 8. Conclusion
1. Introduction A talk for all, Distributors and Users, companies, IT directors, device
makers, software vendors, developers, product managers,…to understand how to cope with patented multimedia formats
Not a talk about legitimacy of patent on software but solutions to live
with it Here to inform, each being responsible for taking his own decisions
Share experience, give tips, explain, make it clearer
The talk is open to all the feedback you may have to share.
2 - Multimedia
2.1. Multimedia everywhere
+31%
2.2. Market Figures
Multimedia Consumption : A Vibrant Market
-11%
$8 billion by 2014*
70% watch online video*
*source Digital Media Post
VOD and Streaming 2010/2011
Digital Downloads +
On-line subscription
Among all online consumers,
*source PricewaterhouseCooper's *source Nielsen
DVD and BluRay rentals
2010/2011
2.3 -Multimedia in the enterprise
Multimedia content
IN HOUSE PRODUCTION Recording Training Recording events Marketing productions Corporate webcasts
CONSUMPTION E learning Business content All company divisions Social share
EXTERNAL CONTENT On line or not Streamed On demand or Live
Content Delivery Network Internal or external
Multimedia is everywhere we need to care. It is important for your
“customers” whether you sell them devices or software, or whether
they are your employees
3 – Linux as a solution
3.1 Why Linux is a good answer? For device makers and enterprises
Low cost
Ease of customization
Very large choice of projects available, high potential of differentiation
Stable kernel
Ported to numerous platforms
Innovation stimulated by the open community
Very large ecosystem of software development company providing support and project services
Allow a mix between free and proprietary solutions to take advantage of both
Linux has been chosen for Embedded solutions a long time ago and drove the innovation, we even didn’t know.
Final users acceptance has taken longer time, but thanks to new types of man-machine interfaces (smartphone, tablets) Linux based OS have become a solution for millions of people.
3.2 Solution recognized for Embedded and for final users
3.3 Challenges of Linux in the enterprise
Proprietary OS
- Pay by unit - All included - No or few custom - Used for years - Usage habits - Pay for upgrades - Ecosystem of proprietary solutions
Linux distribution
- Free, and “a la carte” set up of paying apps
- Large functional scope available - Free and Proprietary applications
available - Diverse - New - Paying support services available
Ensure users acceptation Maintain efficiency Maintain scope coverage
Multimedia may be a major gap
4 – Stakes of multimedia on linux
4.1 Generic model
Multimedia Application
Linux OS (Fedora, Suse, Mandriva, Ubuntu, customer OS,…)
Multimedia Framework open: Gstreamer, VLC, Phonon,
Patented Formats
Free or proprietary license
Free license
Free Formats A/V Codecs
4.2 Why it can become a constraint?
• Some countries recognize these patents • Ie. US, Germany recognize patent on software
• Use/Distribute patented format would require to
be patent licensee
• Many of the patent licenses are incompatible with most free software licenses so it may be incompatible to distribute both together.
• Ie. Software codec under a GPL license together with a patent license
4.3 Licenses incompatibility
• Some terms in patent licenses are incompatible with some terms in open licenses*
• Mainly the fact that open source (free software) licenses require to give the right to use, copy, modify, distribute
• Patent licenses often forbid copy, reverse engineering, release
source code and often bear per-copy running royalties and therefore request tracking
• The main impact may be to be obliged to get some
software with compatible license (restricted rights – not GPL)
*ref. Section 11 of the GNU LGPL (version 2)
4.4 Why we didn’t care before?
BEFORE • Use of proprietary OS • Pay for license • All in one • We didn’t know or look for what was included in
the price • Patent where included in the license price when
the format was supported • If not we were even not used to wonder why • Deal as a black box
4.5 Now what has changed?
NOW, what is changing with Linux
• We build and choose the modules of our solution, no more a black box
• we don’t pay upfront, nothing patented is included a priori
• If we decide to include an open and free project, there is no business model (specific for companies or resellers) allowing you to distribute these pieces of software (patented codecs) in an obvious way
4.6 Why should we care?
• Linux concerns millions of people
• Linux is growing and interest is also growing
• In the patent world, Patent holder claim infringment if not licensed
• Now patent holders care about linux
4.7 Why it impacts companies?
In countries where patents on software are recognized, infringing the patent can lead to negative legal consequences
• Companies distributing thousands of units may be
concerned about being sued
• Companies who have patents themselves may be concerned about respecting patent licenses
• Enterprises deploying thousand of workstations /desktops may not want to endorse responsibility
4.8 First level of impact When choosing a solution, implementing a sofware, distributing it Pay more attention and analyse what we use,
distribute, bundle on a licensing point of view
Find and accept new ways to build the solutions to comply licenses, accept to make compromises Ie. Proprietary mix with OS
Be Geography sensible
4.9 Second level of impact
The solution may not be completely free anymore
• Be patent license compliant often means - be licensee, sign agreements - may imply some royatly to be paid to the patent holder, - tracking units, tracking distribution/use - may imply choices of specific software to comply with the
patent license terms
Require new skills and help from experts in licensing
5 – What solutions do we have?
Safer way is to use the free formats available (open and free of patent and copyright)
When you can control the source of content, it is the best option, with no constraints
5.1 Use free formats
Examples of free formats
Audio codec Video codec
ALAC— lossless audio codec, previously a proprietary format of Apple Inc. CMML — timed metadata and subtitles DAISY Digital Talking Book— a talking book format FLAC — lossless audio codec Musepack — an audio codec ADPCM audio LPCM audio ILBC audio XSPF — a playlist format for multimedia Speex — speech codec WavPack — "Hybrid" (lossless/lossy) audio codec
Theora - video PNG — a raster image format standardized by ISO/IEC SMIL — a media playlisting format and multimedia integration language SVG — a vector image format standardized by W3C VRML/X3D — realtime 3D data formats standardized by ISO/IEC WebM — a video/audio format JPEG 2000— an image format standardized by ISO/IEC DASH demuxer, for adaptive streaming HLS demuxer for adaptive streaming MNG — moving pictures, based on PNG
Containers
Ogg — container for Vorbis, FLAC, Speex and Opux (audio formats) & Theora
Matroska (mkv) — container for all type of multimedia formats (audio, video, images, subtitles)
Bundle and resell: choose not to provide the support directly but rely upon the fact the final user can get it: integrate some “plug-in” management program, where the user will be directed to a source providing solutions to get them, or rely upon source of content providing their own (direct user dowload)
Company: block the content of any source that you don’t cover, and rely upon content that don’t need patent license to be read
5.2 Don’t provide directly the support for patented formats
• As it is not always possible to choose the formats to be supported depending on the source of content you want to support.
• In this case, the approach is to use proprietary implementations and and get the license for the patent.
Bundle with software under compatible licenses
5.3 Use implementation with patent license
5.3 Use implementation with patent license
= READY
TO SHIP/USE PRODUCT
•User satisfaction (internal/external) •Cope with laws in country of Distribution or use •Reduce patent infrigment risk and law pursuit
Codec pack and other multimedia application from
the market answering licensing
requirements Patent licenses...
+
Some of the patented formats
Format Known patent licensing entities/ patent holders
Windows Media Audio
Windows Media Video
Windows Media network protocol
Smooth Streaming demuxer
H264 /MPEG4part10/AVC) video
MPEG2 video
MPEG4part2 video
AAC/HEAAC audio
DD/AC3 audio
DD+/EAC3 audio
TrueHD audio
Dolby Headphones audio
Pulse audio
AMR NB WB audio
MP3/MPEG1LayerIII audio
H265 video
SILK audio
DTS audio
RMV video
RMA audio
Microsoft
Microsoft
Microsoft
Microsoft
MPEGLA
MPEGLA
MPEGLA
Via Licensing (Dolby)
Dolby
Dolby
Dolby
Dolby
Dolby
Voice Age
Thomson
Not yet defined
Skype
DTS
Real networks
Real networks
Containers
ASF
3GPP
Microsoft
3GPP
5.4 What if we deal with hardware codecs
•To cope with multimedia requirements, harware manufacturers more and more include the encoding(decoding directly in the platform
•They may be implementation licensee but will never be final products
•Ie. If you resell an APP considering you will use the hardware decoder of the platform, you are responsible
7 – In practice…
7.1 Generic matters when you start Define the need and scope:
- for device makers define the target and identify the multimedia consumption behavior of your target
- for companies identify the need for each category of worker and ensure where multimedia must be supported
- choose the OS/distro the most adapted to your business
Define the technical configuration
- define the multimedia framework to be used and list players available
- list the codecs needed to be supported
7.2 Define if you are eligible
Main patent licensors will require the final OEM or final user to be the licensee.
Some patent licensors will provide 2 levels of license
- One for implementation: when you provide the software (codec)
- One for distribution
You need to identify where you are in the chain and what you provide
7.3 How does that work? Step 1
1. You identify your needs and possible providers
- You define the codec list and identify possible vendors of implementation
- For each format you identify known licensors. Nobody will never
commit to an exhaustive list of patent holders, as some may come up and nobody knows it. Be sure to identify the ones that count for you.
7.3 How does that work? Step 2
2. You get the codec under the appropriate license
You buy the implementation from a codec vendor
You develop or make develop for you
- Some codecs may require to acquire a specific license for implementation different from the one for use or distribute
- Some may be developped following public specs even if it needs then a license to be distributed
You buy from codec licensors
- Some licensors provide their own implementation
7.3 How does that work? Step3
3. You get the appropriate patent license
Main patent licensors will require you first to define
- The device: hand held, Desktop like, embedded
- The target : internal use, bundle and resell, ODM, OEM, Software vendor,…
- For the entire system or for only one application?
Depending on this you may have different licenses. Be careful you may have to sign different licenses if you want to cover different use
7.4 Codec implementation Business model
It completely depends on the software vendors
It can be a fix fee or a per unit fee, or even a buy out
It will often come as a normal software license, with an offer on support services
It will mainly be binary
7.5 Patent license Business model
It is usually based upon a per quarter / semester / year royalty reporting per unit (reporting and payment after sales)
Some propose % of revenue generated on a service (for profesionnal application / streaming servers,…)
Some propose annual caps or volume discount (ie. MPEGLA) that can be paid upfront
7.4 Patent license costs
It depends on Formats
Sometimes it depends on the use Systeme wide or per application (ie: Via licensing)
It globally goes from $0,10 to $2 per unit
Price per unit is usually a price per machine or per application
Price may differ depending on the type of device (ie: Desktop like, hand held, ...)
7.4 Patent license costs
Some require up front fee, or minimum fee per year whatever the units sold
Some propose a one year fee for one application for PC software (ie: Via licensing)
Some propose free units for a certain volume
7.6 What process to follow? STEP 1 DEFINE THE SCOPE Device makers Enterprise
Target users Market target Employees
Users behaviour Mobile, home, work, perso,….
Work, which division, perso too?
HDW/OS/Framework/Player used
Cross platform? Hardare codecs?
Linux distro by default?
Source of content Depend on market May depend on division
Features supported Decode? Encode? Decode? Encode?
Formats needed User consumption depend on known sources
User consumption fixed by the company
Quantity
Distribution volume forecast
Number of users
Country Distribution Sites (if multinational)
7.6 What process to follow?
STEP 2 TESTING Device makers Enterprise
Evaluation Testing in target environment
Test or buy trial versions
STEP3 FORMALIZE
Product license
Reseller agreement to bundle and distribute Or pay for development
Site licensing agreement for use
Business model options Per unit royalty fee Annual fee Buy out
One time fee license + support agreement
8 – Market solutions
Solution providers
License a full proprietary solution (application): Corel, Cyberlink, Realnetworks,
Get proprietary codecs implementations: Fluendo, Corecodec, Rovi (main concept), Xvid, Packet video, Dolby, DTS, H264 licensing, Enthropy wave,…
+ Get patent licenses: Microsoft, MPEGLA, Via licensing, Dolby, DTS, VoicAge,Technicolor,…
Get a packaged product one-stop-shop on Gstreamer: Fluendo
Example: Gstreamer
Hardware platform
Operating system
Gstreamer
Media Players (A/V, DVD,…)
Streaming, Adaptive streaming, transcoding
Codecs, containers, muxers, demuxers
Specific elements HDW acceleration
Streaming server Video editors
Rendering Post processing, audio/video edition
Video conferencing
Stable, supported
Modular
Cross platform (Gstreamer SDK)
Wide coverage
Widely used
LGPL V2
Gstreamer.net and gstreamer.com
Conclusion Linux growth introduces specific consideration of multimedia
When talking about legal issues on linux, we often open Pandora’s box, but it should not be a big deal, and solutions can be found.
Now solutions are available meanwhile things on pure legal side may take time to evolve.
The most important may be as usual to know what we do, to rely upon experts who exist, and to be able to measure the risks we take.
To feel comfortable, work with lawyers and developers and believe the ecosystem we are part of, brings value, and helps also make things evolve enabling all types of needed configurations. So get in touch with the appropriate partner and you will find solutions.
Contact
Muriel Paumier-Moscardini
Skype: muriel.moscardini
Download slides on http://events.linuxfoundation.org/events/collaboration-summit/slides