open source development as a software engineering concept
TRANSCRIPT
Open source development as a software engineering concept
Daniel Le Berre
The University Of Newcastle
Overview of the lecture
• History of open Source
• Definition
• Major features of Open Source development
• Why to open source (summary)
• Making money with open source
• Case study: Netbeans/Forte4J (Sun)
Brief History of Open Source Software
• 1984: The GNU (GNU’s Not Unix) project is launched by Richard Mathew Stallman (RMS)Notion of “Free Software”
• 1991: Linus Torvalds announces the availability of his Unix kernel, Linux.
• 1997: Eric Raymond writes “The cathedral and the Bazaar”. (using “free software”).
• 1998: The term “open source” is used by ER after the announce of Netscape to release the sources of its browser.
Open-Source: Two religions
Richard Stallman (FSF) Eric Raymond (OSI)
Free Software Foundation
• Stallman is a saint in the Church of Emacs---Saint IGNUcius.
• Sainthood in the Church of Emacs requires living a life of purity--but in the Church of Emacs, this does not require celibacy (a sigh of relief is heard). Being holy in our church means installing a wholly free operating system--GNU/Linux is a good choice--and not putting any non-free software on your computer. Join the Church of Emacs, and you too can be a saint!
Open Source Initiative
• […] the real reason for the re-labeling is a marketing one. We're trying to pitch our concept to the corporate world now. We have a winning product, but our positioning, in the past, has been awful. The term "free software" has a load of fatal baggage; to a businessperson, it's too redolent of fanaticism and flakiness and strident anti-commercialism. Mainstream corporate CEOs and CTOs will never buy "free software," manifestos and clenched fists and all. But if we take the very same tradition, the same people, and the same free-software licenses and change the label to "open source" – that, they'll buy.
The meaning of “free” software
Quick Open Source definition (1)
• Open source promotes software reliability and quality by supporting independent peer review and rapid evolution of source code.
Quick Open Source Definition (2)
Open source promotes software reliability and quality by supporting independent peer review and rapid evolution of source code. To be OSI certified, the software must be distributed under a license that guarantees the right to read, redistribute, modify, and use the software freely.
The GNU Project
• A free Unix-like software system• It is not just an OS, but also
– Compiler– Debugger– Linker– Shell– Editor– …
Linux
• Initially a new kernel to extend Minix
• Then it develops by himself
• Linux is just a kernel, not a complete OS!
GNU Tools + Linux = GNU/Linux
(alternative to GNU/Linux exists: GNU/Hurd for instance)
The cathedral and The bazaar
• First worldwide claim of the importance of free software development
• Consequences– Netscape decides to release Mozilla– The Halloween documents written at Microsoft
Huge media cover!
Famous Open Source Projects
• Emacs /GCC/GNU Tools (FSF)• GNU/Linux, FreeBSD, OpenBSD
(Free Unix)• Apache (daemon http, 60% of the web
servers)• KDE /GNOME (desktops for Linux)• MySQL (database)…
Some [Commercial] Open Source projects
• Mozilla / Netscape: web browserwww.mozilla.org
• OpenOffice / StarOffice: MS Office clonewww.openoffice.org
• NetBeans / Forte for Java: Java IDEwww.netbeans.org
• Darwin / MacOs X : operating systemwww.opensource.apple.com
Open Source definition(from www.opensource.org)
• Open source doesn't just mean access to the source code!– Free Redistribution– Source Code– Derived Works– Integrity of The Author's Source Code– No Discrimination Against Persons or Groups– No Discrimination Against Fields of Endeavor– Distribution of License– License Must Not Be Specific to a Product– License Must Not Contaminate Other Software
Free Redistribution
The license shall not restrict any party from selling or giving away the software as a component of an aggregate software distribution containing programs from several different sources. The license shall not require a royalty or other fee for such sale.
Source Code
The program must include source code, and must allow distribution in source code as well as compiled form. Where some form of a product is not distributed with source code, there must be a well-publicized means of obtaining the source code for no more than a reasonable reproduction cost–preferably, downloading via the Internet without charge. The source code must be the preferred form in which a programmer would modify the program. Deliberately obfuscated source code is not allowed. Intermediate forms such as the output of a preprocessor or translator are not allowed.
Derived Works
The license must allow modifications and derived works, and must allow them to be distributed under the same terms as the license of the original software.
Integrity of The Author's Source Code
The license may restrict source-code from being distributed in modified form only if the license allows the distribution of "patch files" with the source code for the purpose of modifying the program at build time. The license must explicitly permit distribution of software built from modified source code. The license may require derived works to carry a different name or version number from the original software.
No Discrimination Against Persons or Groups
The license must not discriminate against any person or group of persons.
No Discrimination Against Fields of Endeavor
The license must not restrict anyone from making use of the program in a specific field of endeavor. For example, it may not restrict the program from being used in a business, or from being used for genetic research.
Distribution of License
The rights attached to the program must apply to all to whom the program is redistributed without the need for execution of an additional license by those parties.
License Must Not Be Specific to a Product
The rights attached to the program must not depend on the program's being part of a particular software distribution. If the program is extracted from that distribution and used or distributed within the terms of the program's license, all parties to whom the program is redistributed should have the same rights as those that are granted in conjunction with the original software distribution.
License Must Not Contaminate Other Software
The license must not place restrictions on other software that is distributed along with the licensed software. For example, the license must not insist that all other programs distributed on the same medium must be open-source software.
Key of Success
• Launching a project/participating is cheap:– Need only a computer and an internet access
• Internet = worldwide!• Evolutionary concept (genetic algorithm,
Darwinian process)– Several variants of a program can be created, the best
will survive.
• Reward:– Making something– Peer recognition
“Every good work of software starts by scratching a developer’s personal itch”
“To solve an interesting problem, start by finding a problem that is interesting to you.”
Usually, open source software is coming from someone specific need (writing a driver for its own graphic card, an emacs mode for its software, etc.)
People needs are different:- Efficiency- Features- GUI The software evolves in different ways in parallel!
“Good programmers know what to write. Great ones know what to rewrite (and reuse)”
“The next best thing to having good ideas is recognizing good ideas from your users. Sometimes the latter is better.”
Adapting software for its own need save time and money
Classical concept of OOP/software engineering
“Plan to throw one away; you will, anyhow.”
• “Often, the most striking and innovative solutions come from realizing that your concept of the problem was wrong.”
• “Perfection (in design) is achieved not when there is nothing more to add, but rather when there is nothing more to take away.”
“Treating your users as co-developers is your least-hassle route to rapid improvement and
effective debugging.” “If you treat your beta-testers as if they're
your most valuable resource, they will respond by becoming your most valuable resource.”
Beware: works well only if your users are also developers.
“Release early. Release Often. And Listen to your customers.”
• Releases can happen every day!
• Using a stable/development policy can help.(Linux kernel: 2.0.2 stable, 2.1.4 developer)
• Helps to keep users/developers interested
“Given a large enough beta-tester and co-developer base, almost every problem will be
characterized quickly”
• Internet is worldwide, so many (an unexpected) potential beta-tester available.
• You do not pay for the software, so you understand some bugs can occur.
• “Delphi” effect
• Against the law: “adding developers to a late project makes it later.”
Necessary Preconditions to Bazaar design
• Needs something runnable and testable to play with.
• Certain level of design and coding skills.
• A bazaar project coordinator or leader must have good people and communication skills.
Against the Bazaar
• Running unstable software (defect installation)
• “People are quick to volunteer to help, but slow to help.”
Open Source and Security
• Doesn't closed source help protect against crack attacks?
• Security holes can be fixed quickly with open source software.
• The Quake cheats:Problem of conception (tradeoff security/performances),
not of Open Source
Why Open Source ?
• Reliability/stability/scalability are critical• Correctness of design and implementation is not
readily verified by means other than independent peer review
• the software is critical to the user’s control of his/her business
• the software establishes or enables a common computing and communication infrastructure.
• Key methods are part of common engineering knowledge
Some licenses
• GNU General Public License (GNU GPL)All distributed software including GPL code must be
GPL.
• GNU Lesser GPL (GNU LGPL)Non-contaminating GPL (for libraries)
• The Artistic licenseTo keep an “artistic”control over the development of a
package (“standard version”).
Others Open Source Licenses• The BSD license • The MIT license • The Mozilla Public License v. 1.0 (MPL) • The Qt Public License (QPL) • The IBM Public License • The MITRE Collaborative Virtual Workspace License (CVW License) • The Ricoh Source Code Public License • The Python license • The zlib/libpng license • The Apache Software License • The Vovida Software License v. 1.0 • The Sun Internet Standards Source License (SISSL) • The Intel Open Source License • The Mozilla Public License 1.1 (MPL 1.1) • The Jabber Open Source License
Making money from Open Source?
• There are several strategies– Loss-Leader/Market-Positioner (Netscape)– Widget Frosting (Apple)– Give Away the recipe, Open a restaurant (Redhat)– Accessorizing (O’Reilly)– Free the future, sell the present (Alladin)– Free the software, sell the brand– Free the software, sell the content
(see The Magic Cauldron, Eric Raymond, for more details)
Loss-leader/Market Positionner
• To create or maintain a market position for proprietary software that generates a direct revenue stream. In the most common variant, open-source client software enables sales of server software, or subscription/advertising revenue associated with a portal site.
• Examples: Mozilla, OpenOffice
Widget Frosting
• For hardware manufacturers (including anything from peripheral boards all the way up to entire computer system)
• Advantage: nothing to loose
• Example: Apple with Darwin
Give away the recipe, open a restaurant
• Create a market position for services (instead of proprietary software in the Loss-Leader/Market-Positioner case)
• Examples: Cygnus Solution (translation of GNU tools) RedHat, Mandrake and all other Linux commercial distributions.
• Local example (Newcastle): Cybersite www.cybersite.com.au
Accessorizing
• To sell accessories for open source software. (from mugs, T-shirt up to books)
Example: O’Reilly
From closed to open source
• A software can be initially released as closed software and becoming open source
• Example: the game DOOM (Id Software)– Initially (late 93): the first one in his category!
(better to stay closed source)
– After a while, competitors appeared (Duke Nukem)
– Late 97: releasing source opened a second market for the game!
Case study: NetBeans/Forte4J
http://www.sun.com/forte/ffj/resources/articles/opensource.html
NetBeans started as a student project in the Czech Republic (originally called Xelfi), in 1996. The goal was to write a Delphi-like Java IDE in Java. A company was formed around this project, called NetBeans. There were two commercial versions of NetBeans, called Developer 2.0 and 2.1. Around May of 1999, NetBeans released a beta of what was to be Developer 3.0 - some months later, in October '99, NetBeans was acquired by Sun Microsystems. After some additional development time, Sun released the Forte for Java Community Edition IDE - the same IDE that had been in beta as NetBeans Developer 3.0.
There had always been interest in going Open Source at NetBeans. In June 2000, Sun open-sourced the NetBeans IDE …
Forte4J conclusions (1)
Feature SetThe developer community gets more of the features they want, with none of the
commercial clutter.
The community really has gotten more of the features they want. However, a few points should probably be deducted for some "commercial clutter" features that nevertheless help the product. This probably rates as an "A–".
EvolutionThe technology evolves faster as more developer effort can be applied to the features.
The technology has evolved very quickly, and the collaborative effects are notable. Several companies are helping push the technology forward, but there are few miracles and the laws of software development physics still apply. This probably rates a "B".
Forte4J conclusions (2)
Stability
Quality happens sooner as every beta tester can be a bug fixer.
This has proven true, particularly in the early days of the NetBeans project. If the population of users gets biased increasingly toward
commercial developers less inclined towards fixing tool bugs, this effect will diminish. On balance, however, the effort involved in the testing of bug fixes can be expected to increase as the code base grows, so the bug fixes themselves become a less significant benefit. This probably gets a "B" as well.
Forte4J conclusions (3)
CostDevelopment costs are lower as a result of harnessing talent pools from around
the world.
Development costs are not measurably lower, and there is a lot of conflicting data. Since it can't really be measured, this aspect will have to be given an "incomplete".
UsersUser populations are higher as the software is free in binary and source form.
User populations are indeed dramatically higher. In fact, the biggest 'competitor' to the Forte for Java IDE is Emacs– another free software product. This aspect deserves an "A".
Forte4J conclusions (4)
ISVs (independent software vendors)
ISVs are more likely to base products on open source, rather than proprietary code basis.
This also seems to have been borne out in reality. It is interesting to note that most ISVs have not yet been willing to put their own intellectual property into Open Source, but they are very attracted by Sun's doing so. This also earns an "A".
References
• www.opensource.orgEric Raymond parish “Open Source Initiative”
• www.fsf.orgRichard Stallman parish “Free Software Foundation”
• www.tuxedo.org/~esr/Eric Raymond writings
Open Source = gift culture?
• “Gift cultures are adaptations to abundance. Appears in populations that do not have significant material-scarcity problems with survival goods.”
• “The only available measure of competitive success is reputation among one’s peer”
Same thing for research culture/academia