trust in open source software development author: alberto sillitti (free university of bolzano)...

Trust in Open Source Software Development

Author: Alberto Sillitti (Free University of Bolzano)Date: 2 June 2008Location: Barcelona, Spain

Outline

• Trust in the QualiPSo project

• Assumptions and challenges

02/06/2008 Alberto Sillitti (Free University of Bolzano)


Overview

Vision

Through the definition of a CMMI-like model for FLOSS to improve the

trust in FLOSS software

Allow companies to use FLOSS to build their main stream products and services

How?

Through the identification of the quality factors of FLOSS products


Overview

Aims

Identification of the factors that affect the trust in FLOSS development analyzing how FLOSS is developed in:

• Companies• FLOSS communities

Definition of a CMMI-like model for the development of FLOSS

Development of a methodology and a set of tools to support the adoption of the CMMI-like model


Research assumptions and challenges

• Companies are interested in exploiting and participating to the FLOSS phenomenon

• FLOSS helps in reducing costs and time-to-market allowing companies to focus on the added value

Assumptions

• Difficulty of collecting data from different and non-structured sources

• Number of stakeholders• Novelty of the field

Challenges



Identify trust and quality elements in the product and in the development process

Analyze the structure of FLOSS communities (e.g., roles of the developers) and how it affects the development process, the quality, and the trust

Identify product and processes elements that guarantee a required quality level



Define of a CMMI-like model for FLOSS development

Perform experiments to access the real impact of the model and the tools

Identify and develop tools to support the adoption of the model



Application of Social Network Analysis

techniques to identify the structures of

FLOSS projects andcommunities

Analysis of FLOSS processes in communities and incompanies through

surveys

Analyze FLOSS communities and companies using two approaches:



Metrics identificationand collection from the

source code repositories

Analysis of FLOSS in

companies throughsurveys

Analyze FLOSS products using two approaches:


Advancement over the State of the Art

• Identification of FLOSS trustworthiness factors for:– Process

– Products

in companies and FLOSS communities• Linking of the trust-related goals to the specific

quality factors• Definition of a CMMI-like process for OSS• Definition of common tests and benchmarks• Definition of a set of tools for the assessment


Results

• Analysis of the structure of 3 FLOSS groups of interest:– Companies (large and SMEs)– Public administrations– FLOSS communities

• About 100 questionnaires collected


Results

What are the factors considered in the adoption of OSS?

• functional requirements 8 fundamental• customer satisfaction 7 very high• customer interoperability issues 7 very high• maintainability 7 very high• reliability 7 very high• interoperability 7 very high• documentation 6 high• duration of user community 6 high• modularity 6 high• standard architecture 6 high• standard compliance 6 high


Results

• law compliance 5 fairly high• environment 5 fairly high• tools 5 fairly high• user community witnessing quality 5 fairly high• short-term support 5 fairly high• type of licenses 5 fairly high• ROI 5 fairly high• performance 5 fairly high• usability 5 fairly high• reputation of vendor 4 medium• portability 4 medium


Results

• customer-imposed standard 3 low• existence of best practices 3 low• exist. of benchmarks/test suites 3 low• language uniformity 3 low• TCO 3 low• localization 3 low• structural complexity 3 low• patterns 3 low• self-containedness 3 low• existence of training / guidelines 2 very low• existence of maintainer organization 2 very low• distribution channel 1 negligible• size 1 negligible


Results

What are the elements in the process that allow you to trust the quality of the final result?

Quality of documentation 100%

Popularity of the product (number of users…) 80%

Standards used 60%

Roadmap respected 60%

Quality of test plan 60%

Following the test plan 60%

Communication channels available 60%

The copyright used, possible patents problems free 60%

Use of tools cvs, bug tracking 60%

Description of process followed 50%

NOT care of the process 50%

Number of bug reports (misunderstood functionality) 40%

Number of commits (developers) 40%

Independent body that checks the process 40%

Maintainability 40%

People in the project 40%

Sponsored companies and industry 40%


Results

User satisfaction 75%

Standards used 75%

Testing and test suits (automatic or not) 75%

Documentation 50%

Use of metrics: bug reports 50%

Number of users - community size 50%

Certification of the software’s quality by a third party 50%

Process followed (ITIL, RUP) 50%

NOT CMMI (too heavy for development) 25%

Quality assurance process followed 25%

Quality of the code, stability, security and usability of the software, features included, the time frame 25%

Who tests the code and the process?

The developers inside the company 80%

The community 60%

Project owner, manager 40%

Selected users test the software 40%

A separate team 40%

Customers 20%

Quality assurance team 20%

Integrators team 20%

What are the aspects for verifying quality of the product you use/produce ?


Conclusions

• Main research results:– The two most important elements that guarantee the

quality of the FLOSS product are a high quality documentation and the size of the user base

– The most important motivation for choosing a FLOSS product in comparison with a proprietary one is the price of the licence while TCO is not.

– Interviewed companies mainly develop products for their customers only losing an important contribution that FLOSS could bring them


Conclusions

• Main research results (cont.):– The motivation for developing FLOSS products are

related to services that they can offer on top of them– In many cases, the development process is not an

important factor to assess the quality of a FLOSS product

– The development process and the assessment of the quality are very important in specific areas (e.g., where certifications are needed)

trust in open source software development author: alberto sillitti (free university of bolzano)...

Documents

floss phenomenon floss

floss software

floss groups

model slide

high tools5

high maintainability7

high environment5

high usability5