1

Discovering, Modeling, and Re-enacting Work Processes and Practices in Free/Open Source Software Development Projects

Walt Scacchi, Chris Jensen, John Noll and Margaret Elliott

Institute for Software Research

University of California, Irvine

2

Context• Discovering hidden processes within a

large-scale, global, loosely-coordinated open source software development (OSSD) project.– Thousands of project participants– Developing, managing, and evolving over one

million knowledge-intensive artifacts– Weakly coordinated by centralized authorities– All data are open source

3

Context• Discover, model, (re-)enact, and repair

processes• Discover process context, participant

roles, tools, resources, interdependencies within and across projects over the Web

• Why?– Enterprises don’t know their processes– Process improvement, optimization, redesign– Process interdiction w/ competitive advantage

4

Overview

– Process discovery

– Process modeling

– Process re-enactment

– Discussion

– Conclusions

5

Process discovery• Participant observation (online, Web-based)• Collection and annotation of participant

created/modified artifacts– Objects of interaction– How objects are situated in facilitating collaboration,

conflict, or conflict mitigation

• Tracking artifacts added or modified in response to intra-community or inter-community dynamics

• Automated process data mining, categorization, and composition

6

Annotated chat transcript• <CyrilB> Hello (Outsider Critique-1• <CyrilB> Several images on the website seem to

be made with non‑free Adobe software, I hope I'm wrong: it is quite shocking. Does anybody know more on the subject ?

• <CyrilB> We should avoid using non‑free software at all cost, am I wrong ? (Extreme belief in free software (BIFS)-1)

• <CyrilB> Anyone awake in here ? Outsider Critique-1)

7

Current challenges• Examining multiple OSSD processes

across multiple interrelated OSSD projects.– NetBeans.org, Mozilla.org, Apache.org,

BioBeans.org, Tigris.org, Java Tool Community, etc.

• Leadership and control sharing within and across individuals, work groups, and projects as sources of coordination and conflict.

8

NetBeans.org Community Ecosystem

JCP

OpenOffice

W3C

Conflict

Coordination

Coordination

Conflict

Coordination

Conflict

Coordination

Conflict

Mozilla

NetBeans

Apache

Conflict

Coordination

Coordination

Coordination

ConflictConflict

9

Boundary Objects of Interaction– Development artifacts (“software informalisms”)– Protocols

• HTTP, RPCs

– Shared data formats• HTML, XML, CGI

– Community infrastructure tools• Defect repositories (e.g. Bugzilla), Collaborative development

and communication tools (e.g. CVS, online chat, discussion forums)

– Product infrastructure• Plug-ins, Modules, Helper applications

– OSSD processes

10

Direct Interaction

Mozilla

NetBeans

Apache

Conflict

Coordination

Coordination

ConflictTomcat

11

Intra-community issues

• Collaboration– Guidelines and policies

• Development tasks; style guidelines; public “floggings”

– Separation of concerns: • architectural strategy (plug-ins) for collaborative success; • freedom of extension/expression through contributed

source code--reduces involvement with socio-political project issues

• Volunteer versus salaried developers--collaboration breakdowns lead to product failures

12

Intra-community issues

• Leadership and Control– Accountability and expectations based on precedent and

volunteerism– Transparency in decision-making

• Project “management” limited to coordinating roles

– Consent in decision-making• Many contributors assume consensus decision-making, and

breakdowns arise when Sun asserts prerogative

• Conflict Resolution– Not face-to-face– Generally done in “public” via discourse transactions on

discussion forums, else turned over to community governance board for resolution.

13

Indirect process interactions across projects

Changes in: HTTP, CCS, DOM, URI/URL, XML,

XHTML standards

Bugzilla, compliance with W3C standard protocols/data formats, compressed HTTP

module support, Javascript support

Browser-specific actions, browser-error workarounds,

Tomcat integration into NetBeans, compliance with W3C standards,

Apache Ant integration into NetBeans

NetBeans workarounds for Mozilla shortcuts

NetBeans and Mozilla developers collaborate on spell-checking module,

NetBeans adopts Mozilla super review process

Apache releases new version of Tomcat

14

Inter-community issues• Communication and collaboration

– Bug reports and feature requests – Patches submitted– Java.net, Java Tools Community, and Java

Community Process

• Leadership and control across projects– Sun NetBeans vs. IBM Eclipse

• Conflict resolution– Mailing lists; Slashdot; Developer blogs

15

Process modeling

• Rich pictures with hyperlinked Use Case scenarios

• Directed and attributed resource flow graph

• Process domain ontology

16

Sun Microsystems

The BoardRelease Manager

Maintainer Developers/ Contributors

Website

Mailing Lists

Users

Contribute to community

, meet time constraints for the release

Maintain a project/ module, manage a group of developers

Ensure that the netbeans community is being run in a fair and open manner

Start new release phase, propose schedule/plan

CVS

Funds, support, Promote Java/Open source

IssueZilla

QA Team

Produce Q- builds and ensure quality of the software

release proposal, release updates,

branch for current release, release post mortem,

review release candidates (2) & decide final release

download development builds and test

, release Q-builds

download new release

report bugs

select feature to develop

, bug to fix, download netbeans,

commit code

decide features for the project and merge patches/bug fixes

, create module web page

make decisions for the community, on high level

grant CVS commit privilege to developers

Link to all Use Cases Links to all Agents

CVS Manager

Configure and

maintain CVS

grant access

Site Administrator

Manage website

deploy builds SourceCast

Tools

Link to Tools

Download and use free software

Community Manager

Share knowledge and ensure

all community issues are addressed

respond to tech issues, unanswered

questions

Rich Picture

17

Figure 2. A hyperlink selection within a rich hypermedia presentation that reveals a corresponding use case.

Test Builds• The QA team tests the latest nightly

builds every Friday• QA team executes a set of manual

tests on the builds as well as somesanity checks

• Test results are categorized as– Bug Types

• User Constraint:– The tests depend on the manual

tests specification• System Constraint:

– Not all bugs may be identified

18

19

NetBeans

20

Process re-enactment

• Generating executable or re-enactable process specifications from ontology

• “Low-fidelity” process re-enactment support– We don’t try to model everything– Focus on resource flow patterns– Accommodate gaps and detect inconsistencies in

process enactment models

• Re-enactments are interactive, navigational, and grounded in artifacts, tools, roles, and resource dependencies resulting from discovery and modeling

21

Formal model of a Netbeans.org process coded in

PML (excerpt)• ...• sequence Test {• action Execute automatic test scripts { • requires { Test scripts, release binaries } • provides { Test results }• tool { Automated test suite (xtest, others) } • agent { Sun ONE Studio QA team }• script { /* Executed off-site */ } }• action Execute manual test scripts { • requires { Release binaries } • provides { Test results } • tool { NetBeans IDE } • agent { users, developers, Sun ONE Studio QA team, Sun ONE Studio developers } • script { /* Executed off-site */ } }• iteration Update Issuezilla {• action Report issues to Issuezilla { • requires { Test results } • provides { Issuezilla entry } • tool { Web browser } • agent { users, developers, Sun ONE Studio QA team, Sun ONE Studio developers } • script { • <br><a href="http://www.netbeans.org/issues/">Navigate to Issuezilla </a> • <br><a href="http://www.netbeans.org/issues/query.cgi">Query Issuezilla </a> • <br><a href="http://www.netbeans.org/issues/enter_bug.cgi">Enter issue </a> } }• …

22

PML validation analysis

Summary of analysis for netbeans_req_release.pmlModel size (source lines): 307Actions: 36Resources: 72Actions neither requiring nor providing resources: 1Resources required but not provided (potential inputs): 0Resources provided but not required (potential outputs): 0Miracles: 2Black holes: 6Transformations: 30

23

PML analysis detail (excerpt)

home/jnoll/projects/peos//src/models/netbeans_req_release.pml:4:action 'ReviewNetBeans' provides no resources/home/jnoll/projects/peos//src/models/netbeans_req_release.pml:15:action 'SetReleaseDate' provides but does not require ReleaseDate/home/jnoll/projects/peos//src/models/netbeans_req_release.pml:22:action 'ReviewNetBeansVisionStatmenet' provides no resources/home/jnoll/projects/peos//src/models/netbeans_req_release.pml:33:action 'ReviewUncompletedMilestonesFromPreviousRelease' provides but does notrequire ProspectiveFeaturesForUpcomingRelease/home/jnoll/projects/peos//src/models/netbeans_req_release.pml:42:

24

25

26

Discussion• Patterns of interaction about objects or artifacts• Discovering and modeling socio-technical and

cultural evolution processes• Validation strategies and tactics• Process discovery, modeling, and re-enactment

implications• MKIDS Integration

27

Patterns of interaction about boundary objects/artifacts

• Patterns can be detected and include:– Integration of a tool or support for a technology created by

another community that create, update, or manage shared objects– Defect detection and reduction

• Organizations contribute defect reports/patches detected in another organization's tool or technology implementation

– Infrastructure evolution planning• Research contributing to discussions of future/changes in tools and

technologies– Discovery, assessment of effects on one’s own community

• These interactions give rise to additional opportunities for coordination and conflict

28

Socio-technical and cultural evolution processes

• New processes under study– Joining and contributing to a project in progress– Role-task migration: from project periphery to center– Alliance formation and community development

• Independent and autonomous project communities can interlink via social networks that manipulate objects of interaction– Enables possible exponential growth of interacting

and interdependent community as socio-technical interaction network

29

30

Validation strategies and tactics• Multi-mode modeling

– Collection and annotation of artifacts– Rich pictures with hyperlinked Use Case scenarios– Directed and attributed resource flow graph– Process domain ontology construction

• Simulated process re-enactment– Process model language generated from ontology– PML compiled into re-enactment environment– Automated PML source validation – Simulated walkthrough of process

• Integration via ethnographic hypermedia• Open to independent validation and interactive

traceability

31

Discovery, modeling and re-enactment implications

• Discovering, modeling, and understanding “hidden” software processes in large OSSD projects – requires semi-automated process discovery techniques– must span multi-project ecosystem

• Discovered processes (still) need to be modeled as narrative, hypermedia, and formal computational models.

• Understanding large, aggregated Internet-based projects requires process discovery, modeling tools, re-enactment and validation techniques.

32

Where We’re Going

33

MKIDS Integration

• Integration with USC efforts examined and feasible– Text analysis, categorization, summarization– Process taxonomies

• Integration with OSU, Stanford, and CMU efforts appear feasible– Task analysis – Process simulation and modeling– Social network analysis

• Other MKIDS modeling and scheduling efforts might be possible to integrate

34

Conclusions• We are examining processes within and across multiple

projects spanning multiple loosely-coupled communities• Multiple project/organizational interaction may be

coordinative or conflictive• Interaction is driven by ongoing synchronization and

stabilization of objects of interaction across project communities

• Project interaction patterns are emerging, detectable, modeled, and suitable for simulated re-enactment

• Discovering, modeling, validating, and re-enacting hidden processes within and across multiple inter-dependent projects is challenging and important.

35

References

• D.C. Atkinson, D.C. Weeks, and J. Noll. The Design of Evolutionary Process Modeling Languages, Proc. 11th Asia-Pacific Software Engineering Conf., Dec. 2004.

• D.C. Atkinson and J. Noll. Automated Validation and Verification of Process Models, Proc. 7th Intern. IASTED Conf. Software Engineering and Applications, November 2003.

• C. Jensen and W. Scacchi, Discovering, Modeling, and Reenacting Open Source Software Development Processes, Institute for Software Research, Submitted for publication, March 2004.

• C. Jensen and W. Scacchi, Process Modeling the Web Information Infrastructure, Proc. 5th. Software Process Simulation and Modeling Workshop, Edinburgh, Scotland, May 2004.

36

References

• C. Jensen and W. Scacchi, Data Mining for Software Process Discovery in Open Source Software Development Communities, Proc. Workshop on Mining Software Repositories, 96-100, Edinburgh, Scotland, May 2004.

• C. Jensen and W. Scacchi, Collaboration, Leadership, Control, and Conflict Negotiation in the NetBeans.org Community, Proc. 4th. Workshop on Open Source Software Engineering, 48-52, Edinburgh, Scotland, May 2004.

• W. Scacchi, Socio-Technical Interaction Networks in Free/Open Source Software Development Processes, revised version to appear in S.T. Acuña and N. Juristo (eds.), Peopleware and the Software Process, World Scientific Press, 2004.

• W. Scacchi, C. Jensen, J. Noll and M. Elliott, Multi-Modal Modeling of Open Source Software Requirements Processes, submitted for publication, September 2004.

37

Acknowledgements

• Project collaborators: – Darren Atkinson, Santa Clara University– Margaret Ellliot, Chris Jensen, UCI-ISR– Mark Ackerman, UMichigan, Ann Arbor – Les Gasser, UIUC

• Funding support (no endorsement implied):– National Science Foundation #0083075,

#0205679, #0205724, and #0350754.