Multi-Source Development: Enabling Faster, Lower Cost Innovation with Open Source Software
Black Duck Software
September 22, 2009
Copyright © 2008 Black Duck Software, Inc. All Rights Reserved.
Introduction to Black Duck Software
Mission
Accelerate time-to-market and reduce development costs by providing products and services for finding, managing and deploying open source software in a multi-source development process, at-scale.
Founded in 2002 and backed by top investors
Over 600 customers worldwide
Partnerships with global leaders
Copyright © 2008 Black Duck Software, Inc. All Rights Reserved.
Agenda
Market Dynamics
Development Challenges
Multi-Source Development
Meeting the Challenges: Best Practices
Case Studies
Summary
Copyright © 2008 Black Duck Software, Inc. All Rights Reserved.
Difficult Times Still Require Innovation
Economic slowdown = budget cuts– Global IT spending is shrinking– Between 1/09 and 4/09 Gartner
lowered their 2009 Global IT forecast by $270B
Still need to innovate– Differentiation to respond to increased competition– Operational efficiencies to continue to execute
Challenge: innovate more with less– How to lower the cost and risk of innovation, and
accelerate time-to-solution?
Copyright © 2008 Black Duck Software, Inc. All Rights Reserved.
Lowering the Cost of Innovation: the Compelling Economics of Open Source
Linux Example: Leverage of 14:1– Open source community contributes
$1.4 Billion– Red Hat spends $100 Million
Customer saves 88% of development – 19K lines of new code, 140K lines of
open source
– Savings of approx. $20,000 for every 1,000 lines of code of OSS used
“The fundamental economics of software development leads you to open-source software”
– David Rivas, Nokia VP for S60 Software
“The fundamental economics of software development leads you to open-source software”
– David Rivas, Nokia VP for S60 Software
Copyright © 2008 Black Duck Software, Inc. All Rights Reserved.
Potential of Open Source
Gartner estimates the impact of open source:
$37B in 2009– Infrastructure Software: $30B– Application Software: $ 7B
$77B by 2012:– Infrastructure software: $58 billion– Application software: $19 billion
Source: Gartner November 2008
Copyright © 2008 Black Duck Software, Inc. All Rights Reserved.
The Future of Software is Open
Software development has changed forever– Internet, community development &
OSS licensing– Componentization and re-use– Agile methods
OSS has gone mainstream– 85% of enterprises use OSS today– 45% of OSS use is Running Mission-critical applications– 70% of OSS contributors are corporate developers– Microsoft OSS code repository (CodePlex)
Large pool of proven, reusable software– Over 200,000 OSS projects– 5+ billion lines of code
Copyright © 2008 Black Duck Software, Inc. All Rights Reserved.
Top Programming Languages Used By Open Source Projects
(Share is calculated based on lines of code)
Source: Black Duck Software.Note: The table above illustrates the top languages used in open source projects. This data is updated daily. This snapshot was taken on September 1, 2009. Visit: http://www.blackducksoftware.com/oss/licenses#top20
• 80% of open source is C, C++, Java, Shell and JavaScript
• Of the top 5, only JavaScript is gaining in share – up over 2 points
• Overall static languages losing share to dynamic languages
Rank LanguageAll Projects
- Share (% )
Trailing 12-Month Share
(% )
Trailing 12-Month
Gain/ Loss (% )1 C 40.9 40.3 - 0.62 C++ 14.0 13.4 - 0.63 J ava 11.0 10.3 - 0.74 Shell 9.0 7.1 - 1.95 J avascript 5.6 7.6 2.16 PHP 4.9 5.2 0.37 Perl 3.2 2.4 - 0.88 Python 2.7 2.6 - 0.19 SQL 1.6 2.7 1.1
10 C# 1.2 1.3 0.111 Assembler 1.2 0.8 - 0.412 Pascal 0.9 0.7 - 0.213 Ruby 0.8 1.0 0.214 TCL 0.4 0.3 - 0.115 Ada 0.4 0.2 - 0.2
Copyright © 2008 Black Duck Software, Inc. All Rights Reserved.
Top 20 Most Commonly Used Licenses in Open Source Projects
Source: Black Duck SoftwareNote: The table above illustrates the top 20 licenses that are used in open source projects, according to the Black Duck Software KnowledgeBase. This data is updated daily. This snapshot was taken on September 1, 2009. Visit: http://www.blackducksoftware.com/oss/licenses#top20
• Top 10 licenses account or 93% of OSS projects
• Top 20 licenses account for 97%
• Rank by # of OSS projects using the license
Rank License 1 GNU General Public License (GPL) 2.02 GNU Lesser General Public License (LGPL) 2.13 Artistic License (Perl)4 BSD License 2.05 GNU General Public License (GPL) 3.06 Apache License 2.07 MIT License8 Code Project Open 1.02 License9 Mozilla Public License (MPL) 1.1
10 Microsoft Public License (Ms-PL)11 Common Public License (CPL)12 zlib/libpng License13 Eclipse Public License (EPL)14 Academic Free License15 GNU Lesser General Public License (LGPL) 3.016 Open Software License (OSL)17 Mozilla Public License (MPL) 1.018 Common Development and Distribution License (CDDL)19 PHP License Version 3.020 Ruby License
Copyright © 2008 Black Duck Software, Inc. All Rights Reserved.
Development Challenges: What We’re Hearing Goals for reuse/standardization of up to 80%; build / fix / fit
20%
Scale – ad hoc use of hundreds of OSS components has led to a management/tracking nightmare
Increase agility, velocity of development
Desire to take advantage of the benefits of open source but need to have oversight and control– Manual governance, compliance and approval processes
are cumbersome/burdensome to developers, prone to error, often ignored $7800/yr to manage OSS components (Source: Black Duck)
Copyright © 2008 Black Duck Software, Inc. All Rights Reserved.
Challenges of Using Open Source at Scale
Manual management methods are inadequate, prone to error…when open source usage proliferates– E.g., version proliferation raises complexity and likelihood of errors
When managed poorly, use of open source can introduce risks and challenges: – Legal exposure due to unmet license obligations– Security vulnerabilities– Regulatory violations– Unsupported open source– Version proliferation
Copyright © 2008 Black Duck Software, Inc. All Rights Reserved.Copyright © 2007 Black Duck Software, Inc. All Rights Reserved. Confidential and Proprietary.
The Story of Cisco’s Software Supply-Chain
Developers modified firmware turning a low-end ($60) device into a high-function router
The storycontinues...
embedded the code in one of its chipsets
used GPL code to customize Broadcom’s
standard Linux distribution
bought for $500M in 2003
adopted this technology into its WRT54G wireless broadband router
Source code made available by
FSF accused Ciscoof a license violation
Meeting the Challenges
Copyright © 2008 Black Duck Software, Inc. All Rights Reserved.
Multi-Source Development with Open Source is the “New Normal”
YOUR COMPANY
Software Application
Open Source Software
Internally Developed
Code
Outsourced Code Development
Commercial 3rd-Party Code
Individuals
Universities
Corporate Developers
Code
Obligations
Copyright © 2008 Black Duck Software, Inc. All Rights Reserved.
Meeting the Challenges: Best Practices
Best practices fall into three areas:
1.Standardization and reuse
2. Automated Collaboration
3.Compliance
Copyright © 2008 Black Duck Software, Inc. All Rights Reserved.
1. Standardization and Reuse
Typical Problems– “Don’t know what I’ve got” – difficult to leverage
knowledge across teams– Version proliferation– Unnecessary rework
Reinventing the wheel when code already exists Seeking approval for previously approved components
Best Practices– Create a catalog of approved components to promote/enforce
standardization and reuse across the development organization Approval process integrates company policy to increase
efficiency Enhance internal catalog with company specific
attributes/metadata– Make better decisions early in dev process
Automated code search– Automatically track “where used”
Improves maintainability Remediating security and quality issues
Copyright © 2008 Black Duck Software, Inc. All Rights Reserved.
2. Automated Collaboration Typical Problems: gap exists within development;
between development and other functions– Difficult for developers to be on the same page
Sharing information, components– Difficult to get legal and other roles on same
page with developers– Manual review/approval of OSS components
“Status” of OSS review is difficult to know Code approvals taking days/weeks
Best Practice: automate key interactions– Automate group interaction
Manage and automate complex review/approval processes across multiple roles/functions/groups
Capture communication between users during review/approval (Comments, questions, learnings)
– Notifications across functions Real time security vulnerability alerts Notification of approved/disapproved components
Copyright © 2008 Black Duck Software, Inc. All Rights Reserved.
3. Compliance Typical Problems
– Lack of controls on open source use Un-vetted code gets into code base Difficult to validate that approved code is what’s shipped
– Risk/exposure from unmet license obligations– Risk/exposure from export restrictions on crypto code
Best Practices– Automate component request/approval– Continuous Validation
Auto-scan code to identify OSS components and license obligations Integrate into build process to streamline development Integrate into issue tracking (remediation, unknown code,
defect/issue, etc.)– Automatic documentation and reporting
BoM Show met/unmet license obligation to guide legal/dev staff
Copyright © 2008 Black Duck Software, Inc. All Rights Reserved.
Best Practice #0. Creating and Implementing an Open Source Policy
Audit the company code base
Evaluate open source use profiles
Create open source policy
Educate employees
Monitor ongoing policy compliance– Trust, but verify
Source: Navica
Copyright © 2008 Black Duck Software, Inc. All Rights Reserved.
Samples Contents ofA Concise Open Source Software Policy
Copyright © 2008 Black Duck Software, Inc. All Rights Reserved.
Evaluating OSS Projects
Current offering (maturity)
Project governance
Community participation
License strategy
Ecosystem
Features, frequency and number of releases, bug fixes
Leadership, structure, charter, goals, strategy
Number of participants, activity level, frequency of commits
Commercially friendly, viral, dual/multilicense
Service, support, extensions, add-ons, training, consulting
Copyright © 2008 Black Duck Software, Inc. All Rights Reserved.
Case Studies
– Landmark Graphics– Reliant Security– Attivio– QNX
Copyright © 2008 Black Duck Software, Inc. All Rights Reserved.
Case Study 1: Landmark Graphics
Landmark Graphics supplies software to Oil and Gas industry across a broad variety of applications areas
OSS Steward monitors policy compliance
Prioritize standardization
Restructured release process– Uses Black Duck Suite to monitor compliance– PM assumes responsibility for OSS– Remediate if/as violations are found
Contributing back in limited cases
Result: Rapid adoption of the latest models and technologies, with accurate identification of OSS dependencies
Copyright © 2008 Black Duck Software, Inc. All Rights Reserved.
Case Study 2: Reliant Security
Reliant sells PCI compliant in-store systems that include many OSS subsystems.
Set a clear policy for OSS use
Tuned acquisition policies– OSS first mandate– Prioritized “ilities”– Loosely coupled design
Adjusted dev processes– OSS use identified at design– Developer on the hook for provenance
Result: Significant customer savings over commercial alternatives
Copyright © 2008 Black Duck Software, Inc. All Rights Reserved.
Case Study 3: Attivio
Attivio’s unified information access platform extends enterprise search capabilities across documents, data and media.
Result: Have been able to get to get to market faster and focus on true IP differentiators because of OSS.
• Simple OSS policy that is easy to understand
• OSS used for commodity architectural components
• Only using OSS components compatible with a commercial license
• Maintains a common folder of all approved OSS libraries
• Uses Black Duck Suite scan reports to prove active governance to sales prospects
Copyright © 2008 Black Duck Software, Inc. All Rights Reserved.
Case Study 4: QNX
QNX produces middleware, development tools, and real-time operating system software for the embedded market
Using OSS for over 15 years, in production products
Customers needed a license guide to manage product use
Categorize all code components with 3 levels of risk
Sensitize developers about use of OSS
Use Black Duck to automate creation of license guide and track OSS evolution
Publishing their own source for many components (but not as OSS)
Result: Have been able to get to get to market faster and take advantages of third party components to broaden portfolio
Copyright © 2008 Black Duck Software, Inc. All Rights Reserved.
Summary
The pressure to do more with less is driving development organizations to multi-source development
Using open source components at scale brings with it a variety of challenges
Companies embracing open source have evolved best practices to tackle the challenges and thereby enjoy the benefits
Copyright © 2008 Black Duck Software, Inc. All Rights Reserved.
Resources
ROI Calculator– www.blackducksoftware.com/open-source-roi-calculator
Search for open source code to reuse– www.koders.com
White Papers (ROI, Agile and OSS, Best Practices)– www.blackducksoftware.com/resources/whitepapers
Best Practices for Open Source Adoption with Jeff Hammond, Forrester Research– http://www.blackducksoftware.com/form/70160000000Hv06