understanding software quality assurance in oss
TRANSCRIPT
-
8/12/2019 Understanding Software Quality Assurance in Oss
1/20
1
Understanding Software Quality Assurance InOSS
Behutiye Woubshet NemaAdeyemi Bolaji
Shikur Henok2013
-
8/12/2019 Understanding Software Quality Assurance in Oss
2/20
2
-
8/12/2019 Understanding Software Quality Assurance in Oss
3/20
3
Table of Contents
Abstract.................................................................................................................................... 4
1. Introduction ............................................................................................................................ 5
2. Overview of Software quality assurance ................................................................................ 7
3. Quality Assurance in Open Source Software .......................................................................... 9
3.1 Quality assurance involvements in OSS ..........................................................................12
3.1.1 Developer perspective ..............................................................................................12
3.1.2 User perspective .......................................................................................................12
3.1.3 Organization perspective .........................................................................................13
4.0 Discussion...........................................................................................................................14
5. Conclusion ............................................................................................................................17
References ...............................................................................................................................18
-
8/12/2019 Understanding Software Quality Assurance in Oss
4/20
4
Abstract
There are lots of challenges confronting the open source software development, and one of theproblems is quality assurance. And with regards to this fact, the paper investigates the
responsible actors in assuring quality in open source software development. We meticulously
explore, evaluate and interpret the existing literature as regards quality in Open Source Software
(OSS) context.
The principal findings of the study show that sustainable community participation, development
processes, governance, systems and tools used to support development affect the quality
assurance of open source software. The paper reflects the activities which principal stakeholdersof OSS community should execute in order to achieve quality. The need to investigate on how to
encourage developers adherence towards strict development processes in OSS was alsoproposed as future work. The rationale behind this is that, it will help address the discrepancy indevelopment process which is a key source of quality issues in OSS. The study shows the key
stakeholders that are responsible in ensuring quality in different phases of software development
in OSS and the respective activities.
-
8/12/2019 Understanding Software Quality Assurance in Oss
5/20
5
1. Introduction
Open source communities have been rapidly attracting attentions across different fields. Theunderlying reason for this unprecedented development is still not completely known, although,
many Researchers concluded that, the reason is due to low cost of software development, others
(De, 2009; Nakakoji, Yamamoto, Nishinaka, Kishida, & Ye, 2002) argued that, because it is free,so it attracts lot of users. However, there are lots of challenges confronting Open Source
Software(OSS) projects, in general. According to (A. Khanjani & Sulaiman, 2011a), identified
one of the challenges to be in the area of quality of the software, and they also mentioned that,
one of the biggest task of OSS is assuring quality, same conclusion is reflected in literature, forexample (Nakakoji et al., 2002). Subsequently, OSS is yet to attain its full potential, as lot of
critical projects do not consider using OSS mainly because there is no strict process in assuring
quality, in other words, it seems no one is responsible for assuring quality. Further, Hedberg etal. (2007) predicted that, with the increase in commercial use of OSS, there will more people
involved in OSS communities, particularly, the end-users with cravings for quality products
(Hedberg, Iivari, Rajanen, & Harjumaa, 2007).
According to ISO 8402 quality assurance is define as All those planned and systematic actions
necessary to provide adequate confidence that a product or services will satisfy given
requirement of quality (ISO 8402). With regards to theaforementioned definition, it is logical tosay that before quality can be attained there must be a strict process to be followed, and in the
context of software engineering this process is called software testing process. Regrettably,
Gacek and Arief in (2004) stated that OSS developers are less interested in both documentation
and testing (Gacek & Arief, 2004) processes vital in assuring quality. Herberg et al. (2007) alsosupported this claim using these words usually there is no testing plan, the testcoverage is not
addressed (Hedberg et al., 2007). Nonetheless, (Opensource, 2013) mentioned that OSS model
is peculiar both in organizational structure and the development process. The OSS developmentprocess is not meticulously planned at the beginning of the project, it evolves in response to its
users demands, as contrast to the proprietary software model. OSS software development
involves distributed contributions of codes (A. Khanjani & Sulaiman, 2011a; Nakakoji et al.,2002). Thus, the software product of OSS is highly prone to defects or bugs, because the
developers (contributors) may be following different development processes (modules).
Although, some Researchers over the years has concluded that open source development has a
greater chance of increasing the quality of product or reducing the bug, because the underlying
model in open source is that, the more people that have access to the codes, the more chances ofdetecting bugs, contrast to the traditional closed source.
In addition, OSS is unique in a distinctive way, because of its structure- developer, co-developer
(who are regarded as contributor) and the end-users, these are the stakeholders in any OSS
project and they cluster around an OSS project as a community. However, without the presence
of the aforementioned principal actors, there is high possibility that a given OSS project may fail(Nakakoji et al., 2002). In order to maintain a well-focused literature review, we have drafted
-
8/12/2019 Understanding Software Quality Assurance in Oss
6/20
6
research questions as a guide to our review. Thus, the current study aim to investigate assuring
quality in OSS by examining different stakeholders(participant) in the development stage, with a
very focused research question of- who is responsible for quality assurance in OSS projects?.We believe that, answering this question will help uncover the areas that need improvement in
terms of strictly following development processes (developers) or encouragement- for example
increase users participation.
We intend to conduct this study, solely by analyzing literatures with the anticipation of bringing
to light the gaps in literature as well as identifying, whose role it is to assure quality in the
different phases of software life cycle in OSS. In addition, we assumed that, the result of thecurrent literature review will give a broader and concise definition over who is responsible for
software quality assurance in open source software development. As result, inform both research
communities and practitioners.
The primary search process was done using online databases (the ones Oulu University
subscribed to), search engines, electronic journals, and conference proceedings. The common
online resources includes; Google scholars, Web of science, Scopus, ProQuest Databases, ACMDigital library, IEEE xplore, SpringerLink, EBSCO, ScienceDirect, etc.
Further, papers that discussed software quality, software defect management, and software
quality assurance are the studies taken as the primary studies for the review. Additionally, thestudy classified the papers based on the year of publication, the research data, research method,
and most importantly the findings. Furthermore, papers or scientific studies that are not free or in
the database research support of university of Oulu were excluded, regardless the year ofpublication or the content of the paper. The papers with the above selection criteria were written
in English language, this is identified as one of the potential setbacks of the current study.
The rest of this paper is organized as follows. Section 2 provides an overview of qualityassurance in software engineering in general, as well as from open source software community
perspective. Section 3 explains in detail the factors affecting quality in OSS as describe in the
literature. Section 4 presents the discussion of the study. And finally, Section 5 presents the
conclusion of the study.
-
8/12/2019 Understanding Software Quality Assurance in Oss
7/20
7
2. Overview of Software quality assurance
Many studies have been conducted in the context of assuring software quality (Gacek & Arief,
2004; Hedberg et al., 2007). Amidst these studies are definition of QA in software engineering,
the best practices in assuring quality, and the factors that directly impact quality- developmentprocess, process conformance, and improvement of processes as well as validating processes.
Quality assurance is a way to assess whether the product has reached or satisfied customer
expectation. In quality assurance process there are steps that should be followed such as: design,development or evaluation of the product by testing the result against the requirements.
Although, these processes might change depending on the project and specific companies
requirement. The whole theme of quality assurance is to provide good quality products and
services for customers. (A. Khanjani & Sulaiman, 2011b)(A. Khanjani & Sulaiman, 2011a)
Furthermore, quality assurance is a substantial part of software development as it can be related
to correctness, usability, completeness and maintainability of software which affect economicand technical decisions made during software development. Software quality impacts a number
of important factors in our daily lives such as economy, personal and national security, health
and safety (Naik & Tripathy, 2011). Moreover, advancements in technologies and complexity ofsoftware systems also make it necessary to understand and achieve quality assurance in our
products and development processes. This is also supported by (Otte, Moreton, & Knoell,
2008a), The execution of SQA activities requires the introduction of a quality model to obtain
process, transparency, customer satisfaction, repeatable processes and methods.
In proprietary software development process, quality assurance process includes different stepsperformed by developers and testers along the development process. During the process differentstandards have been applied to assure compliances with set of requirements. Performing quality
assurance has been a problem in open source software development. The method used in
traditional software development process is not pertinent to OSS development since OSSdevelopment life cycle is unstructured and disordered. (Ardagna, Banzi, Damiani, & Frati, 2009).
Moreover, Software QA needs an integral collaboration of different elements (stakeholders) in
the software development ecosystem and a follow up of systematic procedures to attain quality.
It is highly important to ensure that processes and tools are utilized in efficient manner to achievequality.
Tuteja and Dubey (2012), argue that quality cannot be improved by assessing an already
completed product and highlight the importance of quality assurance measures targeted toimprove software development procedures, standards and tools used in the development. Inaddition, arguing software testing techniques as measurements of software quality the authors
depict that applying software testing methods in different lifecycle stages of software
development contribute to ensuring software quality. (Tuteja & Dubey, 2012)
Further, to achieve quality product, the conformance to the development process must be strict
and followed also in distributed teams (Ardagna et al., 2009; Tuteja & Dubey, 2012). Sivert
-
8/12/2019 Understanding Software Quality Assurance in Oss
8/20
8
(1997) defines process conformance in the context of software development, informally as the
degree of agreement between the software development process that is really carried out and the
process that is believed to be carried out (Srumgrd, 1997). The study aimed to uncover theimportance of process conformance in software engineering field. As a result the study helped to
propose four key components to measure process conformance in software development. The
components are -model for definition, measurement based on the model, an alternativemeasurement and guidelines for modification of a process, so as to enable conformance
measurement (Srumgrd, 1997). Rigidly, conforming to the processes of development,
subsequently assuring quality of the software product. In addition, we can infer that, if software
products have low quality- the reason maybe that the process of development is not conformed toor the process of development requires some improvement. Bhandari et al. (1993) selected a
narrowed aspect of software process improvement, which is defect-based oriented. They were
motivated by the inconsistency in implementation of processes in the context of software
development. However, the study responded to a practical question by a developer; I know whatI must produce; I know the process I must follow; and I have started development. How do I
detect if things are going wrong, and what do I do to get back on track? Bhandari et al., (1993),
also to create process improvement metric during development, further, the research method is acase study of a developing team of 25 people and the main conclusion of the study was that
process improvement works (Bhandari et al., 1993). This implies that during development
process, developing team (developers) could continually improve their processes, thus assuringquality.
In terms of development processes, it is highly important to validate the process. A relevantstudy in this regard was conducted by Cook and Jonathan (1999), the study provides deeper
insight in the relationship between a formal process model and process execution. As taken from
the study; When process models and process executions diverge, something significant is
happening. They investigated amidst the metrics involved in the processes, the human orbehavioral aspects. Further, they proposed three metrics for process validation; Recognition
Metric (REC), Simple String Distance Metric (SSD) and Non-linear String Distance Metric.
These metrics were demonstrated using the Test Unit task from (Kellner et al., 1990). This is
unique in the sense that it uses the data in process validation activity (Cook & Wolf, 1994).
In addition, a recent study by Dimech et al. (2013) aimed on improving process conformance as
regards to the requirements in the context of design-driven development. They support the ideathat initial requirement can be traced from high-level design forms through the final product.
Furthermore, the authors proposed an insight of handling issues relating the problem space
among which is conformance using DiaSuite methodology (Dimech & Balasubramaniam, 2013)
-
8/12/2019 Understanding Software Quality Assurance in Oss
9/20
9
3. Quality Assurance in Open Source Software
Unlike proprietary software development model which bases on systematic and proceduraltesting, the open source software development highly utilizes post-delivery bug reports in order
to establish Quality assurance of software. Developing open source software products requires
specific methods and techniques for quality assurance process to control the quality of theproduct (A. Khanjani & Sulaiman, 2011a). In addition, quick evolution of the software
development process which is mainly driven by various actors including developers, users, firms
and other participants is also another unique characteristic of the OSS model contributing to
Quality assurance. As stated by Bonaccorsi and Rossi (2003), increased participation bringsproject momentum and frequent releases lead to bugs correction and generate feedback flows
that improve the code and the understanding of the underlying problem, a practice in contrast to
the conventional way of assuring quality in proprietary software (Bonaccorsi & Rossi, 2003).
A survey research on applied quality assurance methods of OSS (Otte, Moreton, & Knoell,
2008b) found out that most successful OSS projects apply structured and organized developmentprocesses. The study also showed that engaging experienced project management, skilled
developers and users in the OSS community is substantial for quality assurance. Moreover, it
was also shown that it is essential to implement guidelines, documentations, standards and
processes to improve quality. The findings support the indispensable inclusion of a wide andexperienced OSS community together with adoption of systematic tools and development
processes to achieve software quality assurance.(Otte, Moreton, & Knoell, 2008b)
Further, high Quality Open source software development depends on fundamental sector ofsustainable community, code modularity, project management and test management. In order to
attain software quality, OSS practitioners should fully understand the dependency between these
sectors. Many researchers concluded that having a large sustainable community should be themain objective in OSS. A research on OSS project by Mark Aberdour (2007) concluded that a
large contributor presence was one of the main factors for high OSS quality (Aberdour, 2007).
Also, Code modularity is still one of the main factors for OSS quality which programmers canextend the program by working on separate modules, without the need to know the full
understanding of the system. This minimizes the risk of new bugs being introduced in other
modules. In addition, peer review and people management play a big role for OSS quality. Peer
review mainly focuses on assessing whether the contribution quality compliance with the code-
base. Moreover, people management plays a great role on establishing an effective environmentand culture; it is seen as important as system design. (Aberdour, 2007)
Furthermore, QA in open source software depends on two processes: code review and data
testing. The accessibility of open source software allows anyone to analyze and make thorough
assessment and get more bugs. Thus, having large community for knowledge sharing, paid and
unpaid contributors, expert developers for controlling and managing bugs, etc. For example,open source software like GNU/Linux, Apache, Firefox, etc., receive more bugs due to the fact
-
8/12/2019 Understanding Software Quality Assurance in Oss
10/20
10
that different contributors share their idea that leads to the possibility of increased bugs to be
released. (A. Khanjani & Sulaiman, 2011a)
Referring to the study of Nakakoji, Kumiyo, et al. (2002), a full understanding of the evolution
of an OSS system cannot be complete without understanding the evolution of the OSS
community and its roles in driving the evolution of the system (Nakakoji et al., 2002). Hence,we aim to examine the role of each stakeholder as regard to quality of product in OSS project.
The perceptions covered in this study includes; developers- core and contributors, users and
organization. Also, this classification is informed by the popular onion model of OSS pattern.
Figure 1, adopted originally from the onion model, shows that sustainable community is
constructed from small number of core developers and increasing number of contributing
developers, bug reporters, users and organizations. Sustainable community is like multi layered
onion with users in the outer layer and move towards core through bug reporters and contributingdevelopers. A user increases their involvement on the basis of their knowledge through the
process. As the users move towards the core, user becomes bug reporters, contributing
developers, and core developers consecutively.
Stakeholder involvements in different stages of open source software development are explained
by drawing a metaphor based on the onion model as shown below. In order to achieve highquality Open Source Software, the OSS community should organize processes from various
perspectives: organization perspective, user perspective, developers perspective and take
measures to assure quality (Aberdour, 2007).
Figure 1. A metaphor based on the onion model (Aberdour, 2007), shows sustainable community
in quality assurance from different stakeholder perspectives.
-
8/12/2019 Understanding Software Quality Assurance in Oss
11/20
11
A number of models and methods have been introduced to ensure software quality. Some of therecommended practices Hedberg et. al (2007), suggests organization and companies should
follow stricter quality assurance methods, since the commercial aspect of OSS is increasing
abundantly (Iivari, Hedberg, Rajanen, & Harjumaa, )(Hedberg, Iivari, Rajanen, & Harjumaa,2007). In addition, it is stated that planning and documenting quality assurance process eases the
communication problem between the increasing developers and users in OSS. It is also suggested
that more emphasis should be given to testing and Test Driven Method (TDD) method should be
used in the development process. Others (Wahyudin, Schatten, Winkler, & Biffl, 2007), alsoconcluded that software testing and peer reviews help to reduce software defects hence software
quality is ensured.
User, developer and organizational participations are seen as the main stakeholders essential tomaintain QA of OSS. The user participation and feedback constitutes one of the backbones of the
open source model (Raymond, 2008) Different studies have showed that OSS projects that have
the support of firms and a large scale of user community benefit from participation of theseactors in assuring Software quality. Therefore, it is important to understand different aspects of
stakeholder involvements and how each stakeholders participation affects quality of the
software. Hence, with a focused research question- who is responsible for quality assurance inOSS?
Sunindyo et al. (2013) study the business process in connections to OSS and proposed aframework that could help OSS project managers to efficiently manage the business process.
Owing to the fact that, it is the responsibility of the OSS project managers to manage the
development process and delivering of product to the End-users. Although, this role could be
demanding, considering the collection and analysis of various data collected from differentstakeholders. Further, the proposed framework consists of four levels; data source, data
collection, data integration and the validation level. The result of the study shows that, Open
source Model framework (OSMF) can assist managers handle business process more efficiently,
as well as enhancing judgment on project quality. (Sunindyo & Ekaputra, 2013)
Abdou et al. (2012) studied popular OSS projects; Apache HTTP server, Mozilla Web browser,
and NetBeans IDE. Their focus was to analyse the test processes undertaken in those projects.The results revealed that, three activities were in compliance to the Test Process Standard
ISO/IEC, although, significant dissimilarities were noticed in regards to the task associated with
the test process practices. They proposed a framework for software test process in OSS called
(OSS-TPF), which was compared with ISO/IEC Test process to reveal the areas that requiresimprovement in achieving quality product from OSS software development. (Abdou, Grogono,
& Kamthan, 2012)
In Rashid et al. (2012), studied perception of quality of OSS product with the focus of
investigating the process practices and problems associated with the development process from
the perception of the developers. The result of the study revealed that, there is no coordinationamong the developers, because of the distributed nature of OSS developmental model. However,
they highlighted the relevance of coordination between developers, thus, encourage developers
-
8/12/2019 Understanding Software Quality Assurance in Oss
12/20
12
to coordinate, since it makes the release process systematic. As a result assist to obtain the
objective of software release process. (Rashid, 2012)
3.1 Quality assurance involvements in OSS
Different sectors of the open source software community play roles in quality assurance process.Along community participation, governance, methodologies (development processes), systems
and tools contribute to ensuring quality in the open source software platform. However, after
reviewing processes and participations in this literature review only three key stakeholder
involvements will be emphasized to describe quality assurance participation and roles.
3.1.1 Developer perspective
Roberts et al., (2006) studied the motivations behind developer communities of the Open Sourcesoftware platform and showed that there is a significant interest for ensuring quality besides
regular contributions. The authors argue that developers interest in reputation in a specific
knowledge domain as a driving force for contributions of quality work. In other words,developers intention to seek a status and reputation for specific skill can be related to achieving
Quality in OSS. (Roberts, Adviser-Hann, & Adviser-Slaughter, 2006)
Luyin and Sebastian (2002) studied OSS quality assurance techniques and perception of QA byOSS developers. The study shows that, in addition to high level of user participation, developers
contribution is indispensable element in ensuring quality assurance under the OSS development
model. Discussing practices of developers, it was also shown that often OSS developers act asusers to validate functionality of software rather than following formal procedures of traditional
software testing. (Zhao & Elbaum, 2003)
3.1.2 User perspective
User participation in OSS software development is seen as crucial element that is needed to
acquire quality and improved product. OSS projects follow models that encourage user
involvement in order to facilitate development processes and assure quality. This is also
supported by different literatures and theories. Raymond E. on the Cathedral and the bazar
argues the importance of having users, Treating your users as co-developers is your least hassleroute to rapid code improvement and effective debugging. (Raymond, 2008)
Furthermore, the level of participation of users can also be related to the success and failure of
OSS projects. High level of user participation is seen as a common pattern and driving force ofQA behind most OSS projects.
Tobias et al. (2008) proposed a quality assurance process framework for ensuring quality in OSS.The key factors emphasized in the design of the framework included, balanced interaction of
-
8/12/2019 Understanding Software Quality Assurance in Oss
13/20
13
human factors, management skills and operating environments of organization and development
measures etc. These factors are shown to contribute to quality assurance of OSS products.
Moreover, the authors argue that QA is highly influenced due to user testing and debugging,therefore projects need to concentrate on enabling tasks, such as structured organization, good
documentation and effective communication.
Users skill in a specific knowledge domain together with the scale of an open source project can
be used to determine the value of user participation in OSS software quality assurance.
(Jullien & Zimmermann, 2009) show how users contribution can be affecting the quality of
OSS by introducing the concept of dominant user skill. Emphasizing on user base perspective,users skill level are used to show how product quality can be improved with bug reports,
patches( fixes) and new features from experienced users.
3.1.3 Organization perspective
Currently, different organizations are on the process of changing their critical systems and
applications to open source application, although, this decision is still highly restricted by the
quality of software in OSS. However, the underlying motivation in the adoption of OSS can bejustified by reduced costs and improved quality of software that exploits experiences of bigger
community. Sarrab et al. 2013, demands for improved and additional features along maintaining
product quality is also additional reason for organizational participation in Open sourcesoftware.(Sarrab & Rehman, 2013)
Organizations involved in development and usage of complex and mission critical software
applications find security pitfalls of the OSS as the main problem in integrating (adopting) OSSinto their platforms. And as a result, companies and enterprises are interested in pragmatic
solutions that enable assurance of OSS software. Hence, organizations will be involved in OSS
projects by initiating and making contributions to assure quality aspects of a project. Claudio etal. (2009b) show how assurance plays a key part as requirements for telecom companies
adopting OSS for their critical systems. In addition, the authors state that The criticality and
complexity of Telco systems (e.g., for OSS/BSS systems) require the analysis and control of
software quality during the whole development process (ensured by the quality or assessmentlevels of the factory or of the partner companies or vendors)(Ardagna et al., 2009b). This shows
organizational participation and contribution have higher importance in ensuring quality of OSS
projects.
-
8/12/2019 Understanding Software Quality Assurance in Oss
14/20
14
4.0 Discussion
In this literature review, a research question for determining principal actors of quality assurance
behind open source software was established as a basis of the study. And hence, areas related toquality assurance, development processes and community interest of participation have been
covered in the study to determine roles and participation in ensuring quality in OSS.
The principal findings of the study show that sustainable community participation, development
processes, governance, systems and tools used to support development affect the quality
assurance of open source software. By emphasizing on interests behind participation and
processes, three stakeholder involvements have been used to determine OSS quality assuranceroles and responsibilities. However, the findings also showed that governance models, level of
user and developer participation and scale of project affect the quality assurance of open source
software as shown in Figure 2. These findings also align with Aberdour (2007), which arguethat High-quality OSS relies on having a large, sustainable community results in rapid codedevelopment, effective debugging, and new features (Aberdour, 2007).
-
8/12/2019 Understanding Software Quality Assurance in Oss
15/20
15
Figure2.Factors influencing software quality in OSS development
The participation of different communities: users, developers and organizations together with
governance, systems and tools used in OSS development play an important role in assuringquality process.
In order to support the idea of sustainable community participation behind quality assurance in
OSS, we can further analyze the relationship between different actors and activities involved inOSS projects. As it can be shown from Table 1, it is essential to have participation of
organizations and developers in development and post -release phases. In these stages, the actors
(quality assurers) can enhance quality of software by creating coordination, utilizing efficient
systems and tools (test processes, quality code, patches.), following governance models thatencourage contributions etc. User participation (end user) is also shown in activities related to
bug reports, new feature suggestions, detecting bugs which mainly occur in post release phases
of OSS projects. Moreover, the table depicts interdependence among the (quality assurers) actorsin ensuring quality of OSS.
QA Assurer Activities Involvement
Developers(core-developer, contributor)
Govern Code review Peer review Develop test process Strict development
process Coordination Bug report Patch fixes
Development&
Post -releasephase
Users(end users)
Detect defect Report bugs Suggests features
Post releasephase
Organizations Govern Coordination Develop process Test
Sponsor(Fund)
Development,post release
Table1. Quality assurance activities in OSS and stakeholder involvements
-
8/12/2019 Understanding Software Quality Assurance in Oss
16/20
16
Table1 above shows the recommended activities that each stakeholder should perform to ensure
software quality in OSS. The core developers and contributors have more similarities in terms of
activities recommended, however, the contributor is required to follow strictly the developmentprocess, as well report bug. Similarly, the End- user is expected to also report bugs and suggest
additional features. Organizations should also govern, coordinate and develop processes.
As reflected in Table 1. There is a need to investigate how to encourage developers to follow
development process strictly in OSS. This will address the discrepancy in development process
which is a key source of quality issues in OSS.
Even though the study findings show that mainly organizational, user participation and developer
roles of involvement contribute to quality assurance of OSS, it lacks to support a detailed
analysis of different community participations and organizational structures of OSS models. In
addition, categorizations of the OSS community (stakeholder) involvements made in the studyconcentrate on developer, user and organizational participation. However, this comes as a
drawback since there are OSS projects with defined QA teams, project management roles,
documentation roles, software testing team and support roles which are responsible to qualityassurance to different degrees.
-
8/12/2019 Understanding Software Quality Assurance in Oss
17/20
17
5. Conclusion
In this review, we aim to answer the question who is responsible for assuring quality in OSS
project? Toward that, we meticulously explore, evaluate and interpret the existing literature as
regards quality in OSS context. Consequently, the existing knowledge revealed key factors that
have direct influence in assuring quality in OSS, these factors include: developers, contributors,organization, and the End-users. It is equally important to emphasize that, OSS project is
different from traditional software development because it is situated within a community, and as
such a holistic approach should be employed when investigating OSS project
The major hindrance in assuring quality in OSS is the fact that there is no formal planning, along
with developers lack of conformance to development processes, consequently, making
assurance of quality uncertain. To this end, the existing literature suggested important activitiesthat should be performed by the aforementioned stakeholders in the preceding paragraph.
The findings are in accordance with the literatures explored in this review, that developersshould provide a development plan, as well as encourage the contributor to strictly adhere to the
process. Similarly, contributors in the other hand are advised to follow development process (if
any), when developing software, they should also report bugs, fix patches. Finally, the End-users
are expected to report defects and suggest features.
The contribution of this literature can be observed from two perspectives- first, for practitioners
this review enhances decision making process in terms of quality by identifying different
activities that should be performed by the key stakeholder, as well as it uncover areas forimprovement. Secondly, for academia, it provides deeper understanding of how quality
assurance is practiced in reality, and also it suggests future work.
Future research should focus on raising quality assurance awareness in the open source
community and also encouraging uniformity of development processes among developers.
-
8/12/2019 Understanding Software Quality Assurance in Oss
18/20
18
References
Abdou, T., Grogono, P., & Kamthan, P. (2012). A conceptual framework for open sourcesoftware test process. Computer Software and Applications Conference Workshops(COMPSACW), 2012 IEEE 36th Annual, 458-463.
Aberdour, M. (2007). Achieving quality in open-source software. Software, IEEE, 24(1), 58-
64.
Ardagna, C. A., Banzi, M., Damiani, E., & Frati, F. (2009). Assurance process for large
open source code bases. Computational Science and Engineering, 2009. CSE'09.
International Conference on, , 3 412-417.
Ardagna, C. A., Banzi, M., Damiani, E., Frati, F., & El Ioini, N. (2009). An assurance model
for OSS adoption in next-generation telco environments. Digital Ecosystems and
Technologies, 2009. DEST'09. 3rd IEEE International Conference on, 619-624.
Bhandari, I., Halliday, M., Tarver, E., Brown, D., Chaar, J., & Chillarege, R. (1993). A case
study of software process improvement during development. Software Engineering, IEEETransactions on, 19(12), 1157-1170.
Bonaccorsi, A., & Rossi, C. (2003). Why open source software can succeed. Research Policy,
32(7), 1243-1258.
Cook, J. E., & Wolf, A. L. (1994). Toward metrics for process validation. Software Process,
1994.'Applying the Software Process', Proceedings., Third International Conference on the,
33-44.
De, R. (2009). Economic impact of free and open source software: A study in india.
Dimech, C., & Balasubramaniam, D. (2013). Maintaining architectural conformance during
software development: A practical approach. Software architecture (pp. 208-223) Springer.
Gacek, C., & Arief, B. (2004). The many meanings of open source. Software, IEEE, 21(1),
34-40.
-
8/12/2019 Understanding Software Quality Assurance in Oss
19/20
19
Hedberg, H., Iivari, N., Rajanen, M., & Harjumaa, L. (2007). Assuring quality and usability
in open soruce software development. Emerging Trends in FLOSS Research and
Development, , 2-2.
Iivari, N., Hedberg, H., Rajanen, M., & Harjumaa, L.Assuring quality and usability in open
source software development.
Jullien, N., & Zimmermann, J. (2009). Firms' contribution to opensource software and the
dominant user's skill. European Management Review, 6(2), 130-139.
Kellner, M. I., Feiler, P. H., Finkelstein, A., Katayama, T., Osterweil, L. J., Penedo, M. H., &
Rombach, H. D. (1990). Software process modeling example problem. Software ProcessWorkshop, 1990.'Support for the Software Process'., Proceedings of the 6th International,
19-29.
Khanjani, A., & Sulaiman, R. (2011a). The process of quality assurance under open source
software development. Computers & Informatics (ISCI), 2011 IEEE Symposium on, 548-552.
Khanjani, A., & Sulaiman, R. (2011b). The process of quality assurance under open source
software development. ISCI 2011 - 2011 IEEE Symposium on Computers and Informatics,
548-552.
Naik, S., & Tripathy, P. (2011). Software testing and quality assurance: Theory and practiceJohn Wiley & Sons.
Nakakoji, K., Yamamoto, Y., Nishinaka, Y., Kishida, K., & Ye, Y. (2002). Evolution patterns
of open-source software systems and communities. Proceedings of the International
Workshop on Principles of Software Evolution, 76-85.
Opensource, i. (2013). Open source. Retrieved 15.10, 2013, from opensource.org
Otte, T., Moreton, R., & Knoell, H. D. (2008a). Applied quality assurance methods under theopen source development model. Computer Software and Applications, 2008. COMPSAC'08.
32nd Annual IEEE International, 1247-1252.
Otte, T., Moreton, R., & Knoell, H. D. (2008b). Development of a quality assuranceframework for the open source development model. Software Engineering Advances, 2008.
ICSEA'08. the Third International Conference on, 123-131.
Rashid, M. M. (2012). Release process on quality improvement in open source software
project management. Journal of Computer Science, 8
Roberts, J. A., Adviser-Hann, I., & Adviser-Slaughter, S. A. (2006). Open source software
participation: Developer motivations and performance.
-
8/12/2019 Understanding Software Quality Assurance in Oss
20/20
20
Sarrab, M., & Rehman, O. M. H. (2013). Selection criteria of open source software: First
stage for adoption. International Journal of Information Processing and Management, 4(4),
51-58.
Srumgrd, S. (1997). Verification of process conformance in empirical studies of software
development. Department of Computer and Information Science, the Norwegian Universityof Science and Technology,
Sunindyo, W. D., & Ekaputra, F. J. (2013). OSMF: A framework for OSS processmeasurement. Information and communicatiaon technology (pp. 71-80) Springer.
Tuteja, M., & Dubey, G. (2012). A research study on importance of testing and qualityassurance in software development life cycle (SDLC) models. International Journal of Soft
Computing, 2
Wahyudin, D., Schatten, A., Winkler, D., & Biffl, S. (2007). Aspects of software quality
assurance in open source software projects: Two case studies from apache project. SoftwareEngineering and Advanced Applications, 2007. 33rd EUROMICRO Conference on, 229-236.
Zhao, L., & Elbaum, S. (2003). Quality assurance under the open source development model.
Journal of Systems and Software, 66(1), 65-75.