open source systems 1users.ece.utexas.edu/~perry/education/382v-s08/l07.pdf · bazaar style"...
TRANSCRIPT
OutlineOpinions on Open Source
Empirical Data on Open SourceConclusion
Bibliography
Open Source Systems 1
Tony Petz and Robert Grant
2008-02-11
Tony Petz and Robert Grant Open Source Systems 1
OutlineOpinions on Open Source
Empirical Data on Open SourceConclusion
Bibliography
1 Opinions on Open SourceEric S. RaymondRichard P. Gabriel
2 Empirical Data on Open SourceHalloran and Schleris
3 Conclusion
4 Bibliography
Tony Petz and Robert Grant Open Source Systems 1
OutlineOpinions on Open Source
Empirical Data on Open SourceConclusion
Bibliography
Eric S. RaymondRichard P. Gabriel
Eric S. Raymond
Tony Petz and Robert Grant Open Source Systems 1
OutlineOpinions on Open Source
Empirical Data on Open SourceConclusion
Bibliography
Eric S. RaymondRichard P. Gabriel
Eric S. Raymond
Who is Eric S. Raymond?
Open source advocate and author
Author of
The Cathedral and the BazaarHomesteading the NoosphereThe Magic CauldronThe New Hacker’s Dictionary (formerly the Jargon File)The Art of UNIX Programming
Developer of
fetchmailemacs lisp library
Tony Petz and Robert Grant Open Source Systems 1
OutlineOpinions on Open Source
Empirical Data on Open SourceConclusion
Bibliography
Eric S. RaymondRichard P. Gabriel
The Cathedral and the Bazaar
Contrasts two styles of open source development
Cathedral – Contribution to the main product highly restricted
Examples include early gcc, closed sourceprojects
Bazaar – Contribution very open and encouraged
Examples include new gcc, fetchmail“release early and often, delegate everything youcan, be open to the point of promiscuity” –Linus TorvaldsSounds like the worse-is-better philosophy
Tony Petz and Robert Grant Open Source Systems 1
OutlineOpinions on Open Source
Empirical Data on Open SourceConclusion
Bibliography
Eric S. RaymondRichard P. Gabriel
fetchmail
A mail utility written by Raymond to fetch mail from remotemailboxes to a users local machine
Modified from an earlier utility (popclient by Carl Harris)
In emulation of Torvalds, Raymond set out to develop this ina bazaar style
Tony Petz and Robert Grant Open Source Systems 1
OutlineOpinions on Open Source
Empirical Data on Open SourceConclusion
Bibliography
Eric S. RaymondRichard P. Gabriel
fetchmail experience
Developed because of a personal need
Having users with access to source code was a great help
Release early, release often, and listen to your customers
more than once a day?“Given enough eyeballs, all bugs are shallow.” – Linus’ LawMost good ideas come from your users
Tony Petz and Robert Grant Open Source Systems 1
OutlineOpinions on Open Source
Empirical Data on Open SourceConclusion
Bibliography
Eric S. RaymondRichard P. Gabriel
The Delphi Effect
The averaged opinion of a mass of equally expert observers isquite a bit more reliable a predictor than the opinion of asingle randomly-chosen one of the observers
Linux amplifies this effect, since the observers are self selected
People who are interested enough to get involved
Tony Petz and Robert Grant Open Source Systems 1
OutlineOpinions on Open Source
Empirical Data on Open SourceConclusion
Bibliography
Eric S. RaymondRichard P. Gabriel
Debugging is Parallelizable
Raymondism
More users find more bugs
Especially users with access to the source code
Lots of random testers are more effective for complex bugsthat one highly skilled
Tony Petz and Robert Grant Open Source Systems 1
OutlineOpinions on Open Source
Empirical Data on Open SourceConclusion
Bibliography
Eric S. RaymondRichard P. Gabriel
Necessary Preconditions for the Bazaar Style
“It’s fairly clear that one cannot code from the ground up inbazaar style”
Even Linus didn’t try it
Developer community needs something runnable and testable
This doens’t have to work particularly wellBut it needs to (1) RUNand (2) convince developers that it can be evolved intosomething worthwhile
Needs to present a plausible promise
Tony Petz and Robert Grant Open Source Systems 1
OutlineOpinions on Open Source
Empirical Data on Open SourceConclusion
Bibliography
Eric S. RaymondRichard P. Gabriel
Necessary Preconditions (cont.)
Don’t launch projects you cannot follow through on
Raymond says this has worked well so farBut really- how many un-used and ill-conceived projects existin sourceforge?We’d say that our package repository management friends siftthrough the garbage for us
Project coordinators need good people and communicationskills
Tony Petz and Robert Grant Open Source Systems 1
OutlineOpinions on Open Source
Empirical Data on Open SourceConclusion
Bibliography
Eric S. RaymondRichard P. Gabriel
The Social Context of OSS
Raymondism
To solve an interesting problem, start by finding a problem that isinteresting to you.
The best hacks start out as solutions to the author’s ownproblems
When developers are not territorial about their code, goodthings happen
This is “egoless programming”Encourage others to look for bugsImprovements happen fasterExtreme programming is a good example of this concept
The bazaar method strongly mitigates Brook’s Law
Tony Petz and Robert Grant Open Source Systems 1
OutlineOpinions on Open Source
Empirical Data on Open SourceConclusion
Bibliography
Eric S. RaymondRichard P. Gabriel
Social Context (cont.)
What prevented this style of project before?
The internet was not yet bornGeographically compact communities that encouraged“egoless” programming thrived: Bell Labs, MIT, UC Berkeley
Linux was the first project to use the entire world as it’s talentpool
The Linux boom coincides with the WWW boom
Tony Petz and Robert Grant Open Source Systems 1
OutlineOpinions on Open Source
Empirical Data on Open SourceConclusion
Bibliography
Eric S. RaymondRichard P. Gabriel
Social Context (cont.)
The Internet was necessary but not sufficient
Linux needed a leadership style and a set of cooperativecustoms
Developers needed to attract more developersNeeded to get maxiumum benefit from the medium
So what is the appropriate leadership style for a bazaar styledeploment?
Tony Petz and Robert Grant Open Source Systems 1
OutlineOpinions on Open Source
Empirical Data on Open SourceConclusion
Bibliography
Eric S. RaymondRichard P. Gabriel
Social Context (cont).
It is acting on the principle of common understanding instead ofthe principle of command
Raymond quotes Pyotr Alexeyvich Kropotkin’s “principle ofunderstanding”
Linux behaves like a free market ecology
Selfish agents attempt to maximize utilityProduces self-correcting spontaneous orderUses a non-classic utility function based on reputation
Volunteer activity driven by “egoboo” or ego-boosting
Tony Petz and Robert Grant Open Source Systems 1
OutlineOpinions on Open Source
Empirical Data on Open SourceConclusion
Bibliography
Eric S. RaymondRichard P. Gabriel
Social Context (cont.)
This “egoboo” driven free market economy drives developersto virtuousness
Quality of codeConscientious and strangely complete documentation
Raymondism
Provided a communications medium at least as good as theInternet, and provided a leadership free of coercion, many headsare better than one.
Tony Petz and Robert Grant Open Source Systems 1
OutlineOpinions on Open Source
Empirical Data on Open SourceConclusion
Bibliography
Eric S. RaymondRichard P. Gabriel
On Management Principles
One could argue that future value, in terms of service andcontinued support, is actually what people are buying when theybuy software
However, this assumes OSS cannot deliver a sustaineddevelopment effort
GNU Emacs has been continually improved for 20 years
Linux has seen 12+ years of continued improvement, onrapidly changing hardware platforms
Tony Petz and Robert Grant Open Source Systems 1
OutlineOpinions on Open Source
Empirical Data on Open SourceConclusion
Bibliography
Eric S. RaymondRichard P. Gabriel
Management Principles (cont.)
What about buying legal liability, and the opportunity to sue ifthings go awry?
Most software licenses disclaim any such guarantees
Legal cases where money was recovered after softwarenonperformance are extremely rare
Tony Petz and Robert Grant Open Source Systems 1
OutlineOpinions on Open Source
Empirical Data on Open SourceConclusion
Bibliography
Eric S. RaymondRichard P. Gabriel
Management Principles (cont.)
So what do traditional software development managers do?
Define goals and keep everyone focused on themMonitor progress and ensure crucial details aren’t skippedMotivate people to do boring, but necessary, drudgeworkOrganize the deployment of peopleMarshal resources to bear on the problem
Let’s examine each of these goals and see how they stand upunder the bazaar model
Tony Petz and Robert Grant Open Source Systems 1
OutlineOpinions on Open Source
Empirical Data on Open SourceConclusion
Bibliography
Eric S. RaymondRichard P. Gabriel
Management Principles (cont.)
Marshal resources to bear on the problem
Traditionally a defensive tactic
Everyone brings their own resources to the table
No need to keep them out of the hands of other managers
Tony Petz and Robert Grant Open Source Systems 1
OutlineOpinions on Open Source
Empirical Data on Open SourceConclusion
Bibliography
Eric S. RaymondRichard P. Gabriel
Management Principles (cont.)
Organize the deployment of people
Open-source hackers self-select based on competence andenthusiasm
There is a belief that only about the top 5% of programmerstake part in open-source projects
Which might explain the quality of open-source codeCheaper and easier to recruit volunteers than to managepeople who would rather be doing something else
Tony Petz and Robert Grant Open Source Systems 1
OutlineOpinions on Open Source
Empirical Data on Open SourceConclusion
Bibliography
Eric S. RaymondRichard P. Gabriel
Management Principles (cont.)
Motivate people to do boring, but necessary, drudgework
Naysayers claim that the OSS community can only be reliedon to do work that is ‘sexy’ or technically ‘sweet’
Boring, but necessary, work “will only ever be done bymoney-motivated cubicle peons”
What if we accept these two statements to be true?
We create a sort of “boredom boundary line”It only exists as long as no one in the OSS community findsthe problem interestingThen they create a technically superior solution owing to theirinterest and curiousity in the problem
I think these are good arguments, but not the whole story
Tony Petz and Robert Grant Open Source Systems 1
OutlineOpinions on Open Source
Empirical Data on Open SourceConclusion
Bibliography
Eric S. RaymondRichard P. Gabriel
Management Principles (cont.)
Monitoring to ensure that critical details don’t get skipped
This is where open source really shines
Decentralized peer review is the best way to ensure thatdetails don’t get skipped
Tony Petz and Robert Grant Open Source Systems 1
OutlineOpinions on Open Source
Empirical Data on Open SourceConclusion
Bibliography
Eric S. RaymondRichard P. Gabriel
Management Principles (cont.)
Define goals and keep everyone focused on them
To believe that traditional models are better, we have tobelieve that corporate management committees are moresuccessful than the foresight of the grand pumbas of the OSSworld
Well known folk theorems state that 60%-75% of softwareprojects are never completed, or rejected by their intendedaudienceIs it possible that the OSS management model excels here too?
Tony Petz and Robert Grant Open Source Systems 1
OutlineOpinions on Open Source
Empirical Data on Open SourceConclusion
Bibliography
Eric S. RaymondRichard P. Gabriel
Conclusions
Open source is fun
OSS is racking up technical, market-, and mind-sharesuccesses at an astounding rate
OSS is proving that “joy is an asset”
Humans take pleasure in work that is challenging, but not sochallenging that it is too hard to achieve
Raymond calls this the “optimal-challenge zone”
OSS purposefully builds on this principle!
Tony Petz and Robert Grant Open Source Systems 1
OutlineOpinions on Open Source
Empirical Data on Open SourceConclusion
Bibliography
Eric S. RaymondRichard P. Gabriel
More Ideas of Eric S. Raymond
...And now for something not all that different
Pervasive themes in Raymond literature
Taken at random from two other papers:
“Homesteading the Noosphere” and“The Magic Cauldron”
Tony Petz and Robert Grant Open Source Systems 1
OutlineOpinions on Open Source
Empirical Data on Open SourceConclusion
Bibliography
Eric S. RaymondRichard P. Gabriel
Varieties of Hacker Ideology
“Free software is“Commercial
“Free software is my life!”
software is theft and hoarding”g
“Commercial“Open source is a
good thing”
Commercial software is ok, if it’s well‐made”it s well made
“C ft i“OSS is ok. I respect it.”
“Com. software is ok, I just like OSS
better”p
better”
Open source programmers run the gamut of motivations
Historically, the most visible and best-organized have been thevery zealous and very anticommercial
Tony Petz and Robert Grant Open Source Systems 1
OutlineOpinions on Open Source
Empirical Data on Open SourceConclusion
Bibliography
Eric S. RaymondRichard P. Gabriel
The Hacker Milieu as a Gift Culture
Not generally recognized by anthropologists, “gift cultures areadaptations not to scarcity but to abundance”
Social status determined by what you give away
This idea fails to explain the culture of malicious hackers
Tony Petz and Robert Grant Open Source Systems 1
OutlineOpinions on Open Source
Empirical Data on Open SourceConclusion
Bibliography
Eric S. RaymondRichard P. Gabriel
Lockean Theories of Ownership
Explains the nature of ownership in the hacking world
Hackers do not own ideas, they own projects
Analogous to Locke’s ideas of land ownership, you acquire aproject by:
Starting oneTaking over and improving a dead oneBeing handed the controls to one by its original owner
This fails to recognize the adversarial fork!
Tony Petz and Robert Grant Open Source Systems 1
OutlineOpinions on Open Source
Empirical Data on Open SourceConclusion
Bibliography
Eric S. RaymondRichard P. Gabriel
The Manufacturing Delusion
“Noticing that computer programs, like all other kinds of tools orcapital goods, have two distinct kinds of economic value.”
Sale Value and Use Value
The ‘factory model’ of software design is founded on twopremises:
(1) Most developer time is paid for by sale value(2) This is proportional to development cost and use value
These are both false
Most code is written ‘in-house’75% of what programmers are paid to do is actuallymaintenanceThe service industry model is more appropriate
Tony Petz and Robert Grant Open Source Systems 1
OutlineOpinions on Open Source
Empirical Data on Open SourceConclusion
Bibliography
Eric S. RaymondRichard P. Gabriel
The Inverse Commons
Garret Hardin’s “Tragedy of the Commons” looms over everyattempt to explain cooperative behavior
When applied to OSS, this seems to indicate that it isunstable and will have a short life
No allocation policy for programmer time on the Internet
Some people predict that the “commons” will break up, thegood bits will go closed-source, and the rest will stagnate
However, the trend is opposite this
In fact, using software increases its valueI.e. “the grass grows taller when it’s grazed upon”
Tony Petz and Robert Grant Open Source Systems 1
OutlineOpinions on Open Source
Empirical Data on Open SourceConclusion
Bibliography
Eric S. RaymondRichard P. Gabriel
Richard P. Gabriel
Tony Petz and Robert Grant Open Source Systems 1
OutlineOpinions on Open Source
Empirical Data on Open SourceConclusion
Bibliography
Eric S. RaymondRichard P. Gabriel
Worse is Better
Presented as part of a keynote at EuroPAL in 1989
Richard P. Gabriel
CEO of Lucid, Inc from 1984-1994, (a major Lisp Company)About the poor state of the Lisp industry compared to theC/C++ industryNot about open source per se, but Raymond cites this paperas anticipating Linux and the superiority of bazaar-styledevelopment
Presented two philosophies of design:
The Right ThingWorse is Better
Tony Petz and Robert Grant Open Source Systems 1
OutlineOpinions on Open Source
Empirical Data on Open SourceConclusion
Bibliography
Eric S. RaymondRichard P. Gabriel
“The Right Thing” Design Philosophy
The following things are important to a designer:
Simplicity of both implementation and interface. Simplicity ofinterface is more important than simplicity ofimplementation.
Correctness in all observable respects is required.
Consistency in all respects is required. The design may be morecomplex to avoid inconsistency.
Completeness as far as is practical. All reasonably expected casesmust be accounted for. More important thansimplicity.
Tony Petz and Robert Grant Open Source Systems 1
OutlineOpinions on Open Source
Empirical Data on Open SourceConclusion
Bibliography
Eric S. RaymondRichard P. Gabriel
The “Worse-is-Better” Design Philosophy
The following things are important to a designer:
Simplicity of both implementation and interface. Simplicity ofimplementation is more important than simplicity ofinterface.
Simplicity is the most important consideration of a design.
Correctness in all observable respects is required. It is slightlybetter to be simple than correct.
Tony Petz and Robert Grant Open Source Systems 1
OutlineOpinions on Open Source
Empirical Data on Open SourceConclusion
Bibliography
Eric S. RaymondRichard P. Gabriel
The “Worse-is-Better” Design Philosophy (cont.)
Consistency or at least not overmuch inconsistency. Can besacrificed for simplicity, but it’s better to just dropuncommonly used inconsistent or overly complexparts.
Completeness as far as is practical. Can be sacrificed in favor ofany other quality, and must be sacrificed whenimplementation simplicity is threatened. Especiallyworthless is consistency of interface.
Tony Petz and Robert Grant Open Source Systems 1
OutlineOpinions on Open Source
Empirical Data on Open SourceConclusion
Bibliography
Eric S. RaymondRichard P. Gabriel
Comparison
“The Right Thing” represents the MIT approach
Common Lisp and Scheme
“Worse-is-Better” is a caricature of what he calls “the NewJersey approach”
UNIX, C
Tony Petz and Robert Grant Open Source Systems 1
OutlineOpinions on Open Source
Empirical Data on Open SourceConclusion
Bibliography
Eric S. RaymondRichard P. Gabriel
Why Worse is Better
The worse-is-better has better survival characteristics thanthe-right-thing.
C is easy to write a compiler for—it requires the programmerto write code that is easy to interpret
Both early UNIX and C had simple structures, were easy toport, required few machine resources, and provided 50% to80% of what you want from an operating system andprogramming language
Tony Petz and Robert Grant Open Source Systems 1
OutlineOpinions on Open Source
Empirical Data on Open SourceConclusion
Bibliography
Eric S. RaymondRichard P. Gabriel
Why Worse is Better (cont.)
Half the computers that exist at any time are worse thanmedian, and UNIX and C run fine on them, and it is easy toport to them
C code is portable because it is written on top of a virus
Once the virus has spread, there is a lot of pressure to improveit, and will be improved to closer to 90% of the-right-thing
Tony Petz and Robert Grant Open Source Systems 1
OutlineOpinions on Open Source
Empirical Data on Open SourceConclusion
Bibliography
Eric S. RaymondRichard P. Gabriel
Why Worse is Better (cont.)
Therefore, worse-is-better
1 will gain acceptance
2 will condition its users to expect less
3 will be improved to a point where it’s almost the right thing
“release early and often, delegate everything you can, beopen to the point of promiscuity” – Linus Torvalds
Tony Petz and Robert Grant Open Source Systems 1
OutlineOpinions on Open Source
Empirical Data on Open SourceConclusion
Bibliography
Halloran and Schleris
High Quality and Open Source Practices—Halloranand Schleris
Tony Petz and Robert Grant Open Source Systems 1
OutlineOpinions on Open Source
Empirical Data on Open SourceConclusion
Bibliography
Halloran and Schleris
High Quality and Open Source Practices
Claim: the quality of open source software is roughlyequivalent to commercial and government-developed software
How can this quality be further improved?
What attributes will allow a practice to be adopted in an opensource project?
Tony Petz and Robert Grant Open Source Systems 1
OutlineOpinions on Open Source
Empirical Data on Open SourceConclusion
Bibliography
Halloran and Schleris
Data Sources
Conclusions are based on a survey of practices in “a numberof” open source projects from November 2001 to November2002
11 projects summarized, including Apache, GNOME, gcc,Jakarta Tomcat, KDE, the Linux kernel, Mozilla, NetBeans,Perl, Python, XFree86
Tony Petz and Robert Grant Open Source Systems 1
OutlineOpinions on Open Source
Empirical Data on Open SourceConclusion
Bibliography
Halloran and Schleris
Observed Quality Practice
All used people as their main form of Quality Assurance
committers must review and accept all patchessometimes more elaborate processes such as Mozilla’s reviewand super review and Netbeans’ high resistance
All used web portals to provide their collaboration tools (CVS,Bugzilla, Mailman, Majordomo, DejaGnu, Tinderbox)
All projects used nightly builds
Virtually all projects used a public bug and issue tracking tool
Tony Petz and Robert Grant Open Source Systems 1
OutlineOpinions on Open Source
Empirical Data on Open SourceConclusion
Bibliography
Halloran and Schleris
Quality Intervention and the Walled Project Server
In theory, anyone can copy, modify, and redistribute opensource projects
What do open source “project leaders” actually control?
The Walled Project Server
Project leaders control how developers and others interact withthe persisent state of the project through the project serverDesigned to maximize outgoing information but strictly limitand control incoming information
Behind the server wall, controlled processes take place
Examples include code commits, documentation management,configuration management, builds, regression tests
Tony Petz and Robert Grant Open Source Systems 1
OutlineOpinions on Open Source
Empirical Data on Open SourceConclusion
Bibliography
Halloran and Schleris
Communication
Developers are largely self-managing and self-selected
Software policies are generally enforced by tools
This is possible and necessary because developers are rarelycollocated
Tony Petz and Robert Grant Open Source Systems 1
OutlineOpinions on Open Source
Empirical Data on Open SourceConclusion
Bibliography
Halloran and Schleris
Bootstrapping
With one exception, all tools used were also open source
Exception: SourceCast by CollabNet, which is build aroundopen source toolsportion of project server is proprietary
Often rationalized based on ideology, but more than that
it lowers the barrier to entry (for open source developers)enables developers to easily shift from tool use to tooldevelopment and repair—developers as users principle
Tony Petz and Robert Grant Open Source Systems 1
OutlineOpinions on Open Source
Empirical Data on Open SourceConclusion
Bibliography
Halloran and Schleris
Incrementality and Gentle Slope
Common toolset allows project newcomers to experience a“gentle slope” learning curve
Is this true of developers new to open source?
Because of communication practices, developers cancontribute incrementally
Graceful engagement and disengagement
New participants encouraged to read, debug, and document
Tony Petz and Robert Grant Open Source Systems 1
OutlineOpinions on Open Source
Empirical Data on Open SourceConclusion
Bibliography
Halloran and Schleris
Evolutionary Focus
All surveyed projects are in maintentance and evolution phases
Hypothesis
Open Source projects are sensitive to initial architecture
Must accomodate division of labor and long-term incrementalgrowth
Tony Petz and Robert Grant Open Source Systems 1
OutlineOpinions on Open Source
Empirical Data on Open SourceConclusion
Bibliography
Halloran and Schleris
Criteria for Open Source Adoption
Criteria for a quality practice or technology to be adopted by anopen source project:
1 incremental model for quality investment and payoff
2 incremental adoptability of methods and tools within thewalled server and in the client-side toolset
3 a trusted server-side implementation that can acceptuntrusted client-side input
4 a tool interaction style adoptable by practicing open sourceprogrammers
Tony Petz and Robert Grant Open Source Systems 1
OutlineOpinions on Open Source
Empirical Data on Open SourceConclusion
Bibliography
Halloran and Schleris
Possibilities
Possible:
testing technology
code analysis
Difficult:
formal methods
advanced program analysis
Tony Petz and Robert Grant Open Source Systems 1
OutlineOpinions on Open Source
Empirical Data on Open SourceConclusion
Bibliography
Questions?
?/!
Tony Petz and Robert Grant Open Source Systems 1
OutlineOpinions on Open Source
Empirical Data on Open SourceConclusion
Bibliography
Bibliography
E. Raymond.The Cathedral and the Bazaar.Knowledge, Technology, and Policy, 12(3):23–49, 1999.
E.S. Raymond.Homesteading the Noosphere.First Monday, 3(10), 1998.
Tony Petz and Robert Grant Open Source Systems 1
OutlineOpinions on Open Source
Empirical Data on Open SourceConclusion
Bibliography
Bibliography (cont.)
E.S. Raymond.The Magic Cauldron.Accessed: June, 6:2004, 1999.
R.P. Gabriel.LISP: Good news, bad news, how to win big.AI EXPERT., 6(6):30–39, 1991.
Tony Petz and Robert Grant Open Source Systems 1