an empirical study of learning by osmosis in global software engineering

An empirical study of learning by osmosis in globalsoftware engineering

Patricia Lago1, Henry Muccini2,*,† and Muhammad Ali Babar3

1Department of Computer Science, VU University Amsterdam, The Netherlands2Department of Computer Science, University of L’Aquila, Italy

3IT University of Copenhagen, Copenhagen, Denmark

SUMMARY

To teach global software engineering, we devised a complementary distributed module with a shared projectinvolving both local and international teams. In local teams, students are located at the same university andtrained in one of the two complementary topics. In international teams, students are located at two differentuniversities and trained in one of the two complementary topics. This study empirically investigates whetherthe students in the international teams can compensate the extra effort required to deal with communication,coordination, and collaboration issues that characterize global software engineering projects with learningby osmosis (i.e., by transferring knowledge among globally distributed teams trained on different topics).The results show that there was no statistically significant difference between the performance of localand international teams. We assert that the students in the international and local teams perform equally well,thanks to learning by osmosis. However, our analysis of the self-reported questionnaire data revealed thatmost of the participants (i.e., 70%) would like to work in local teams in real-life project, 74% of the parti-cipants thought international teams were less efficient, and 41% of the participants reported lack of trust intheir international team members compared with their local team members. Copyright © 2011 John Wiley &Sons, Ltd.

Received 31 July 2010; Revised 18 June 2011; Accepted 14 July 2011

KEY WORDS: global software engineering; learning by osmosis; software engineering education

1. INTRODUCTION

Recently, a large number of software development companies have started moving their softwaredevelopment undertakings/centers to countries where required skilled personnels are widelyavailable at relatively low cost. Organizations have distributed their software development acrossmultiple sites, multiple countries, and multiple cultures [1], thus involving them in global software en-gineering (GSE) [2]. Whereas providing several kinds of new opportunities of saving cost and growththrough process and product innovation, GSE has created a wide variety of new challenges for soft-ware engineering practitioners and researchers, and risks of not achieving the desired objectives mayinclude quality, cost, and schedule [3–5]. GSE projects are often large-scale, and globally distributedsoftware engineering leads to significantly increased complexity for project teams who may have toface several kinds of new technical and social challenges in the areas of coordination, collaboration,and communication [6]. It is becoming apparent that GSE projects are more likely to be unsuccessfulas compared with collocated projects, because physical, time, cultural, organizational, and stakeholderdistances can have negative impact on communication, coordination, collaboration, and knowledge

*Correspondence to: Henry Muccini, Department of Computer Science, University of L’Aquila, Italy.†E-mail: [email protected]

Copyright © 2011 John Wiley & Sons, Ltd.

JOURNAL OF SOFTWARE: EVOLUTION AND PROCESSJ. Softw. Evol. and Proc. 2012; 24:693–706Published online 24 October 2011 in Wiley Online Library (wileyonlinelibrary.com). DOI: 10.1002/smr.565

exchange between globally distributed software engineering teams [7]. GSE projects can have up to50% failure rate ([6]; [20]).

Whereas practitioners and researchers have been extensively studying challenges involved in GSEand proposing solutions to them [8, 1, 9], there is relatively little attention being paid to educating thefuture software engineers on how to master different aspects of working in GSE teams [10], We assertthat there is a vital need of getting software engineering and computer science students learn how towork effectively and efficiently in GSE projects during their formal education in academic institutes.Moreover, such efforts can also provide academic researchers an ample opportunity to study how peo-ple use existing or gain new technical and socio–technical knowledge and skills required for GSE pro-jects. One of the initiatives focused on educating and training future software engineers is the GlobalSoftware Engineering European Master (or GSEEM) program.1 This program aims at providingthe students with the knowledge, understanding, and skills required to become effective and efficientmembers of GSE teams and be able to deal with the technical and social challenges characterizingGSE projects.

The objective of this paper is to report on the design, logistics, and findings from an empirical study,which aimed at understanding the impact of an enhanced coordination means (named “learning byosmosis”) in the performance of international GSE teams when compared with local teams. Learningby osmosis consists of learning complementary topics by working together with those who areassumed to be experts. In the study we report on, local and international teams were asked to deliverthe same project: local teams were composed of students located at the same university and extensivelytrained in one of the two complementary topics; international teams were composed of students locatedat two different universities where students were trained in one of the two complementary topics.

Starting from the understanding that GSE projects require complementary skills possessed by teamslocated at different locations, this empirical study investigates whether or not learning by osmosis iseffective, that is, if international teams can compensate the extra effort required to deal withcommunication, coordination, and collaboration issues that characterize GSE projects when learningby osmosis. The study has been conducted in the context of the Global Software EngineeringEuropean Master program.

The rest of this paper is structured as follows. Section 2 briefly discusses the background of thereported study. Section 3 contextualizes the research design and logistics of this study. The resultsare presented and discussed in Section 4. We share our observations from offering thecomplementary modules in general and from this study in particular in Section 5. The validitythreats have been presented in Section 6 and 7, respectively. The paper concludes in Section 8.

2. BACKGROUND AND MOTIVATION

We have been conducting a set of studies within a software engineering project course carried out bytwo universities in Italy and the Netherlands [11]. The course aims at enabling the students to experi-ence global issues and learn complementary technical skills, and performing detailed design by main-taining traceability. One of the key challenges of this type of courses is to teach the students how tobalance teamwork and technical skills in GSE projects without compromising the quality of the deliv-ered results. The first instance of the project course was carried out in 2006 after merging two coursesto design a joint course [11]. This project course is part of an international master program jointly of-fered by three European Universities located in Italy, the Netherlands, and Sweden [12].

The work reported in this paper has focused on the complementary module delivered by the Uni-versity of L’Aquila (UDA) and the VU University Amsterdam (VUA). The overall idea of complemen-tary modules is to teach complementary software engineering concepts in parallel and through sharedprojects. The two institutions (globally distributed at site X and site Y) deliver modules (A and B, re-spectively) that teach the students about a range of topics, which are different and complementary.For example, a module might teach software documentation and modeling, whereas another comple-mentary module might teach scientific writing, which is complementary to writing effective

1www.gseem.eu

694 PATRICIA LAGO, HENRY MUCCINI AND MUHAMMAD ALI BABAR

Copyright © 2011 John Wiley & Sons, Ltd. J. Softw. Evol. and Proc. 2012; 24:693–706DOI: 10.1002/smr

documentation. In our case, module A teaches how to model specific design solutions for Web appli-cations, whereas module B teaches how to document design decisions and rationale (among other ad-vanced topics in software design).

The complementary modules model typical situations in global organizations in which differentsites possess different skills and develop different parts of a shared product. Companies try todevelop local specialized skills that can be combined, and to transfer across the companies thenecessary knowledge to ensure a consistent product. The education institutes develop local skills(i.e., focus on the topics in which the local institution has the required competency and which can fitinto its delivered curricula) and to increase the amount of knowledge transferred across theinstitutions so that the students can develop further competencies with some additional effort. Toenable the transfer of knowledge across the modules, students work in teams on a shared project spe-cifically designed to enable them to learn the complementary software engineering topics taught inboth modules A and B from their team members.

Local teams are composed of students collocated in the same site. They are expected to possess theknowledge about the topics taught at one institute, whereas they have to gain knowledge about thecomplementary topics while working in the shared project. International teams are composed ofstudents from both sites, working together in a globally distributed setting, in order to cooperativelyproduce the shared project deliverables. These international teams are expected to learncomplementary software engineering topics by osmosis, gaining new knowledge and skills by collab-orating with others. More precisely, learning by osmosis means “learning by transferring knowledgeamong globally distributed teams trained in different topics”, that is, students learn from peers some-thing on which they have not got any lecture. Accordingly, the students in international teams can learnthe complementary software engineering content from their team members, which results in the knowl-edge about the complementary content being transferred more effectively between the two studentteams, shortening the learning period.

In this study, the material about Web design taught in 16 hours at the UDA was also summarized asone of the specialized topics at the VUA (2 hours). The material about design documentation andrationale, taught in 12 h at the VUA, was also summarized to the UDA students in one lecture (2 h).Hence, the students in the international teams were expected to learn by osmosis about the subjectmatter taught to their international colleagues. By assigning students from the two institutions to ajoint project team, they were expected to bring different but complementary skills, job experience,and cultural characteristics to a project team.

3. DESIGN OF THE STUDY

In the following paragraphs, we describe our study design in terms of study objective, researchquestion, and the design of the complementary module and shared project. For each element, weexplain the study setup for module edition 2008/2009.

3.1. Objective

Our objective was to observe the extent to which the shared project actually helped realize learning byosmosis and, specifically, the effectiveness of shared projects in transferring technical knowledgeamong peers. To study this objective, we needed to assess if the students working on shared projectswith international peers (and therefore having the opportunity to benefit from transferringcomplementary knowledge by osmosis) do actually perform better than students working locally andwho must learn all the necessary technical knowledge by themselves.

Therefore, we built two types of teams: local teams composed of peers located at the local university(i.e., Italian and Dutch teams), and international teams composed of peers from both universities (i.e.,both Italian and Dutch students together). Figure 1 shows that the local teams TX and TY (local to uni-versities X and Y, respectively) are fully trained on modules A and B, respectively. The teams receivedonly one lecture on the complementary topics. The international teams were instead trained in bothtopics. Half of the team was from University X and was fully trained in module A, half was from

LEARNING BY OSMOSIS IN GLOBAL SOFTWARE ENGINEERING 695


University Y and was fully trained in module B. The so-called international teams were supposed to do“extra work” in order to address the challenges (e.g., caused by geographical, temporal, and culturaldistances), which characterize most of the GSE teams. Because all teams needed to submit exactlythe same deliverable, the local and international teams needed to complement their initial knowledgeon the non-local topic or invest extra time on coordinating their global and distributed tasks.

3.2. Research question

Our main research question was: Can learning by osmosis influence the performance of internationalteams when cooperatively working on a GSE project?

Performance can be defined as something “accomplished; the fulfilment of a claim, promise, orrequest”.2 In the context of this study, performance is defined by the quality of the produced deliver-ables and the quality of the team work achieved:

• The quality of the design document delivered by each team is measured quantitatively using thegrading scheme designed for this task. To ensure comparable grading across teams and countries,both teachers used the same grading scheme for the whole project.

• The quality of teamwork is measured by analyzing the self-reported quantitative and qualitativedata gathered with a questionnaire, to be filled by all students at the end of the project. Weintended to use the findings from the analysis of the data gathered using the questionnaire to pro-vide further insights and evidence in support of the findings from analyzing the quantitative databased on the grades gained by each team. To make sure that all students would fill the question-naire carefully, the questionnaire was part of the project deliverables.

Hence, we measure the influence of osmosis indirectly by measuring the performance of theinternational teams as compared to the performance of the local teams.

3.3. Study setup

The complementary module for this study was composed of two modules: “Modeling of WebApplications” (MWA) and “Advanced Topics in Software Design” (ATSD). The main objective ofthe module is to teach complementary software engineering concepts in parallel through a sharedproject and enable the students from two institutes to experience the issues that characterize GSEprojects.

3.4. The complementary module

The MWA is a six ECTS module that teaches how to use UML for MWA. Among others, it has threelectures (4 h each) allocated to teaching UML and five lectures (4 h each) on specific Web modelingmethods. In the 2008/2009 edition, the MWA course was delivered in 6weeks, with an average of10 h of teaching per week. The ATSD is a six ECTS module including three lectures providing thebasic knowledge about effective design documentation, design decision, and rationale for theselection of UML diagrams. This course also has additional lectures on specialized design topics,among which is Web design. In the 2008/2009 edition, the ATSD course has been delivered in16weeks between September 2008 and December 2008, with an average of 2 h of teaching per

Figure 1. Local and international teams.

2Merriam-Webster on-line dictionary.



week. The complementary module was focusing on combining the two complementary topics ofdesign decision and rationale, and detailed UML Web design. The UDA teacher provided a 2-h lectureto VUA students on modeling web applications with the UML, whereas the VUA teacher provided a2-h lecture to UDA students on design decisions and design rationale.

3.5. The shared project

The MWA and ATSD students worked in teams on a shared project especially designed to enable themto learn complementary software engineering topics from both MWA and ATSD modules. Weprovided a common case study, called IntelliHome, meant to offer a range of services to controlhome devices (e.g., video-recorder, LAN/ADSL, heating), both remotely and/or locally.

To answer our main research question and, specifically, to assess the performance of theinternational teams versus local teams based on project deliverables, the course was designed tohave the MWA and ATSD local teams as well as international teams (see Figure 2). The internationalteams were composed of globally distributed students, two from MWA and two from ATSD. TheMWA students were supposed to bring specialized knowledge of web design, whereas the ATSDstudents were supposed to have specialized knowledge of design decisions and rationale, andhow to document them. Four international teams participated in the study. The local teams werecomposed of students working in collocated format at the same university and composed of fourstudents. We had three local teams at UDA and four at VUA.

3.6. Project timetable

The entire project was run over 18weeks and composed of two main parts: project part 1 was focusedon requirements elicitation and the definition of design space of the problem, and project part 2 wasfocused on the detailed design of the Web application. As shown in Figure 3, the international team-work had been distributed in a manner to have the VUA international students working on part 1 fromweek 37 to week 44 after which the teams were supposed to submit a design space milestone at the endof week 44. The UDA students, instead, started from week 44 (caused by the different schedule of thetwo courses at two institutes).

The students at the UDA had to carefully study and understand the design space report prepared bythe VUA students and to discover inconsistencies or incompleteness, and work with the design toproduce an agreed design space. Then, the students from the UDA had another 7weeks forproducing the detailed Web design based on the input provided to them by the VUA colleagues.The VUA students had to carefully study and understand the detailed Web design report preparedby the UDA students and inform the teammate from the UDA about any errors or problems in theirdetailed design deliverable.

The local VUA students started project part 1 from week 37 and the project part 2 from week 44.They released the design space deliverable during week 44 and a detailed web design deliverableduring week 51. The local UDA students started project part 1 from week 44 and part 2 from week51. They finished their design space deliverable during week 51 and a detailed web designdeliverable during week 04. The variation in the timetable for the start of the project parts andcompletion of project deliverables was caused by different schedules of the two courses as stated.

Each team had to submit the following deliverables:

MWA

ATSD

Design reportUDA local teams

Design reportVUA local teams

UDA

VUA

Complementarymodules

Design reportinternational teams

Shared project

2008/2009 edition

Figure 2. Complementary modules on software design.



• A design deliverable, to describe the complete design of the IntelliHome system from the systemrequirements to the design solution developed by the team. The core parts were the systemrequirements and design space description (to be developed in project part 1) and the web designsolution (to be developed in project part 2). A standard design space template was given to thestudents at the start of the project.

• A questionnaire, to gather self-reported qualitative and quantitative data to capture the perceptionsand experiences of the participants on different aspects of working in local versus working in in-ternational teams. Each student was supposed to fill the questionnaire at the end of the project. Thequestions in the questionnaire focused on different project aspects such as the actual workload, theamount and quality of learning, and tool support.

Apart from the aforementioned two project deliverables, the teams had weekly progress meetingswith the teaching staff for discussing project-related issues.

3.7. Assigned roles

The complementary module has been organized by assigning explicit roles to the members of eachteam whether local or international. The management roles were the communication tracker and thealigner. These roles were introduced to facilitate the knowledge transfer and to make the team mem-bers reflect on the types and amount of interactions due to teamwork. The communication trackerwas supposed to collect the information about the communications occurring between two or moreteam members. The aligner was in-charge of ensuring consistency between the two core parts of thedesign deliverable (design space and design solution).The technical roles are the design space expertand the topic expert. The design space expert is responsible for the development of the design spaceand of its quality, whereas the topic expert is responsible for the development of the design solutionand its quality. In a local team, we need one aligner and one communication tracker per team. In inter-national teams instead, two aligners are needed (one for each site), as both the students at one site coverthe same technical role (either both design space experts or both Web design experts), they can judgeinconsistencies for their expertise and function as channels for communicating inconsistencies in the“remote” work.

3.8. Grading scheme

We have indicated that the complementary modules taught at two universities covered both topics thatwere module-specific (e.g., Web applications design for UDA and design decisions and rationale forVUA), and topics that were specific for the complementary module in the other University and thatwere learned via the shared project. To measure the effectiveness of such modules in achieving theirpedagogical goals, we needed to have a grading scheme to evaluate the deliverables from the twomodules in order to enable both lecturers to assess the results of the shared projects as well asmodule-specific deliverables. Both the lecturer came up with a common grading scheme for theproject deliverables and for the project teamwork. The score was given on a 1 to 10 scale. A local

Figure 3. Teamwork distribution for international and local students.



grading criterion has also been devised for the module-specific topics. The common grading criteriawere:

Grading criteria for the project deliverables: Correctness, originality, internal consistency andtraceability, documentation quality (e.g., UML correctness, essence, relevant decisions), and link tothe selected topic. The criteria focus on the quality of the technical deliverables from a team.

Grading criteria for project teamwork: Teamwork planning (e.g., schedule and allocate work, ar-range meetings, consensus on tool usage), team functioning, use of coordination and collaborationtools, role assignment, quality of the assessment of the peer–team work (local/international peers),quality of the support provided to the peers, and questionnaire. These criteria focus on the quality ofthe collaborative performance and the soft skills of a team’s members.

4. RESULTS FROM THE EMPIRICAL STUDY

In this section, we present the results from our study. We have already indicated that we used twomeans of gathering data for finding answers to our main research question: grading of thedeliverables and analysis of the self-reported data collected with questionnaires.

4.1. Assessment of technical skills by student grades

This study observed 44 students, 28 divided into seven local teams, and 16 divided into fourinternational teams of four members each. Each international team had two members from Italy andtwo members from the Netherlands. For each country, we compared the grades of the deliverables(obtained following the grading scheme described in Section 3 on page 9) of the students workingin the international teams with their colleagues working in local teams. Table I shows the descriptivestatistics based on the data from the Italian students working in the international and local teams. Theanalysis of the grades shows that the mean scores for the international and local teams were 9 and 9.5,respectively. The results of analyzing the data using a non-parametric Mann–Whitney [13] test showedthat there is no significant difference between the two teams with a P value of 0.135.

These results show that the performance of the participants working in the international teams wasnot significantly lower than the participants who worked in the local teams. One possible interpretationfor this finding can be the benefit of “learning by osmosis” because of which the Italian students wouldhave gained subject matter knowledge and new skills, which they used in their projects. Anotherpossible interpretation can be that the participants who worked in the local teams did not find theteam work as challenging as their colleagues working in the international teams. This situation maynot have motivated them to work hard enough to do significantly well on project work carried outby local teams. A third possible interpretation can be the contemporary mechanics of teams workingon a project. People prefer to have electronic communication and coordination even when they arelocated in the same city. Hence, there may not have much difference between local and internationalteams except language and cultural differences. That means the coordination and communicationoverload was as much on the local teams as it was on the international teams, another possiblereason for a lack of significant difference between the performance of international and local teams.

Table II presents the descriptive statistics of the data from the Dutch students working in the inter-national and local teams. There were eight students in the international teams and 16 students in thelocal teams. The analysis of the grades shows that the mean scores for the international and local teamswere 8.96 and 9.31, respectively. The results of analyzing the data using a non-parametricMann–Whitneytest showed that there is no significant difference between two teams with a P value of 0.153.

Table I. Summary statistics for the participants from Italy.

Type of team Number of participants Mean SD SE

International 8 9.0 0.25198 0.08909Italian 12 9.5 0.65649 0.18951



The results from the Dutch students’ performance revealed that the Dutch students working ininternational teams were not significantly affected because of the overhead of communication,coordination, and cultural issues. It also shows that our results are consistent for the Dutch andItalian students. This is an interesting finding because Dutch local teams were multi-culture ascompared with Italian local teams, which consisted of all Italian students. If we try to interpret theresults for the Dutch students working in international teams in the context of the results for Italianstudents, we can say that Dutch students had much more experience in working in multi-cultureteams outside this course. The Italian students did not have this opportunity. One possibleinterpretation can be that the Italian students working in the international teams may have learnedfrom their Dutch teammates how to work in multi-cultural teams without having any negative effecton the project outcomes, which would be a positive (and unexpected) additional contribution of“learning by osmosis”.

We were also interested in comparing the performance of the students from the two countries.Table I and II also highlight the differences of the grades obtained by the Italian and Dutch studentsworking in the local and international teams. Although the Italian students achieved slightly bettergrades than their Dutch counterparts irrespective of whether they worked in local or internationalteams, there was no statistically significant difference between the performances of the Italian or Dutchstudents working either in the local or international teams.

These results show that the students performed equally well in their respective project tasks. Thatmeans they knew their respective subject matter quite well; this finding supports our study designobjective where students were supposed to contribute complementary skills (VUA in designrationale and documentation, and UDA in Web modeling). We assert that these findings areencouraging as there is evidence that students paired with different skill sets can help each other toperform the tasks that need complementary knowledge and skills. This is an interesting resultshowing that complementary modules with shared projects can support “learning by osmosis”.

4.2. Analysis of questionnaire data

We used an extensive post-study questionnaire for gathering the self-reported quantitative andqualitative data in order to gain an in-depth understanding of the participants’ opinions, feelings, andexperiences of working in one particular setting compared with another setting (i.e., local teamsversus international teams). In the following, we present the results from the analysis of the datagathered from questions related to each of the aforementioned parts. From both countries, there werea total of 44 students who had worked in either a local or an international team during this course.The results are based on the questionnaires returned by 39 students (23 Dutch and 16 Italian). Itshould be noted that we have not always used percentages that may add up to 100% because someof the students did not reply to all the questions in the questionnaires.

Table III summarizes the results from the responses to the questions designed to elicit the students’perceptions about their individual as well as team performance in the local compared with the interna-tional teams and their preference for a particular kind of team to work on real-life projects. Theresponses to the questions 1 and 2 show that, a majority of the respondents believed that their individ-ual and team performance was better when they worked in the local teams. However, there were nota-ble numbers of respondents who believed that their individual (28%) and team (23%) performanceswere equally good in both kinds of teams (i.e., local and international). The responses to question 3show that a majority of the students considered that working in the international teams had negativeeffect on their team discussion. A large majority of the respondents also perceived that working inthe international teams was less efficient, which means that the students cloud only complete lessamount of work in a given time when they were working in the international teams compared with

Table II. Summary statistics for the participants from the Netherlands.

Type of team Number of participants Mean SD SE

International 8 8.96 0.948 0.335Dutch 16 9.31 0.615 0.154



the local teams. These findings show that the students working in international teams had to deal withseveral kinds of overheads, which may have affected their speed of work on project tasks. A large ma-jority of the participants responded that they would prefer working in local teams in real-life projects.

These findings are very interesting considering the fact that the analysis of the quantitative datashow that the performance of the students who had worked in the international teams was notsignificantly affected (as compared with the performance of those who had worked in local teams).However, the subjective opinions of the students overwhelmingly favor working in the local teamsif given the choice. That means that managers of GSE teams need to take appropriate measures toaddress the emotional and perception sides of international teams rather than the technical aspects ofproject teams. Our findings also show that students managed to balance the team work and technicalskills as they did not allow their performance to be affected drastically because of working in theinternational teams despite the fact that the participants of the international teams frequentlyindicated the increased effort required for communication and coordination in GSE projects.

Trust is considered an important aspect of globally distributed teams [14, 15]. We were interested inknowing whether or not the students working in international teams trusted their international teammembers as much as their local teams members. Table IV shows that a majority of the students repliedthat they equally trusted their international team members; but a large percentage (41%) of the studentsreplied that they did not trust their international team member. We observed that more Italian students

Table III. Questionnaire PI: Responses to post-study questionnaire (team format).

Question 1(performance individual)

Twenty-one (54%) of the participants believed they performed well whileworking in local team (14 of them had worked in local teams only, eventhough seven of those who had worked in international team were also pre-ferring local teams.); 11 (28%) of the participants felt that they performedequally well in both the international and local teams; 7 (18%) of themthought that they performed well while working in international teams.

Question 2(performance team)

Twenty-one (54%) of the participants thought that their team performedwell while working in a local team (14 of them had worked in local teamsonly); 9 (23%) of the participants believed that their team performedequally well while working in both local and international teams;9 (23%) participants felt that their team performed well while working inan international team.

Question 3(impacts on plenary discussions)

Seventeen (46%) of the participants believed that working in the interna-tional teams had a small negative effect on their team discussions;10 (28%) of the participants thought that working in the internationalteams had a small or large positive effect on their team discussion (wemerge the two points on the scale); 6 (16%) participants reported no effect;4 (10%) of them believed that working in the international teams had neg-ative effect on their team discussion.

Question 4 (workload) Twenty-nine (74%) of the participants believed that working in an inter-national team compared with a local team was less efficient; 7 (18%) par-ticipants thought working in an international team and local team wasequally efficient; 3 (8%) participants considered working in an interna-tional team compared with a local team more efficient. In this context, ef-ficiency was defined to be the amount of work completed in a local orinternational team in similar amount of time.

Question 5 (preferred team type) Twenty-eight (70%) of the participants replied that they would like towork in local teams in real-life project; 12 (30%) of participants were ofthe opinion that overall, they would like to work in international teams inreal-life projects.

Table IV. Questionnaire PII: Responses to post-study questionnaire (trust on team members).

Question 1(trust international colleagues

Twenty-three (59%) of the participants replied that they had trusted their in-ternational team members as much as their local team members; 16 (41%) ofthe participants said that they had not trusted their international team membersas much as their local team members.



reported lack of trust towards their international team members than those of the Dutch students. Thisfinding provides useful insight, which emphasizes the importance of establishing and maintaining trustin international teams as reported previously [15]. Table V summarizes the students’ feedback on theusefulness of the management roles. We can notice that 63% of the student found this role very usefulin ensuring internal consistency in the design deliverable. Many students commented, indeed, thatinconsistencies occurred regularly, and because of this explicit role, it was possible to maintain thequality level of the documentation. Only 22% of the students believed that the aligner was of littleuse, and 15% said that it was not useful.

One of our assertions was that an aligner could help in realizing learning by osmosis. In theinternational teams, the aligner also acts as a local expert of what has been developed by the teammembers abroad. We observed that 24% (10 out of 41 students) of the negative answers are fromthe members of the local teams, who perceived that this role was an unnecessary complication;actually, out of 15 negative answers to the usefulness of aligner role, 10 came from members of thelocal teams. The answer to question 2 (in Table VI) further confirmed the positive influence of rolealigner, and specifically its effect on documentation consistency. Only six students did not feel thatconsistency increased because of having a specific person playing this role. Responses to question 3show that the communication tracker was not perceived as useful (49%) in improving communicationin the team. Only 39% of the participants reported many improvements, or improvement. This soundsreasonable; the communication tracker role has been introduced to ensure that data about type andamount of inter-team communication is carefully reported and was never meant to directly improvecommunication. Indeed, many students commented this to be an additional task that does not necessar-ily influence the quality of the results.

Table VI summarizes the responses to the questions related to the role and importance of design ra-tionale. The findings show that there was an overwhelmingly positive support for documenting designrationale because majority of the students replied positively to all the questions. It is interesting to ob-serve that this result resembles and confirms the findings reported by Tang and colleagues [16] basedon an extensive survey of practitioners.

In summary, our analysis of the data gathered through a questionnaire has revealed that theparticipation in the international teams was perceived as effort consuming, which has also beenfound in other studies [17, 18]. Despite that, the performance of the international teams is equal to thatof the local teams. It can be said that the knowledge transfer mechanisms introduced in the complemen-tary modules worked successfully.

Table V. Questionnaire PIII: Responses to post-study questionnaire (assigned roles).

Question 1(usefulness role aligner)

Sixteen (39%) of participants thought the role ‘Aligner’ wasvery useful in achieving consistency in the design deliverable;10 (24%) of the participants also found this role useful in main-taining consistency in the design documents; 9 (22%) partici-pants found this role a little useful; 6 (15%) participantsbelieved that this role was not useful.

Question 2(impact on docum. consistency)

Ten (24%) of the participants thought that the design consis-tency and traceability increased a lot because of assigning therole ‘Aligner’ to a specific person resulted; 12 (29%) participantssaid the consistency and traceability increased; 13 (31%) partici-pants believed the consistency increased a little bit because of‘Aligner’; 6 (14%) did not feel that the consistency and traceabil-ity increased because of the role ‘Aligner’ assigned to a specificperson.

Question 3(usefulness role com-munication tracker)

Seven (17%) and 9 (22%) participants believed that the role‘communication tracker’ assigned to a specific person resultedin a lot of improvement, or improvement in general, in commu-nication in the team; 6 (15%) thought communication improveda little bit; 19 (46%) participants said that there was no improve-ment in team communication because of the role ‘communicationtracker.’



5. OBSERVATIONS AND DISCUSSION

In this section, we report some further observations and lessons learned from the reported study. InSection 6, we also discuss the findings with reference to related literature.

Project timetable: We have described in Section 3 and graphically depicted in Figure 3 that theshared project was organized in order to have project part 1 and project part 2 sequentially deliveredby the VUA and UDA students, respectively. The students who kick-started the project earlier gained(almost) complete ownership of the requirements and the design decisions. That means that the inter-national team members who entered in the project in part 2 did not have much influence on the refine-ment/modification of the work that had been done by that stage. This might have had a negative effecton team building, ownership of the results, and on the quality of the design deliverable.

Four UDA students commented that kick-starting the project would have made an importantpositive difference, and everybody else from UDA recognized that it would have made a positivedifference. Based on our observations and the students’ comments, we can conclude that all studentsin a team start the project at the same time. There were some unexpected problems as the workloadat the two universities had been organized differently. The UDA students attended classes mostlyfor the whole week and carried out the project work in the evenings and during the weekends; theVUA students had fewer classes and would dedicate part of the working weekdays to assignmentsand project work. In many cases, this made it difficult for the international teams to find suitabletime slots for synchronous online discussions, and organize internal deadlines.

Deliverables: All students reported positive experiences from having and using explicit documen-tation of design rationale. Whereas some students commented that recording rationale can be a bit ex-pensive in terms of required documentation effort, all agreed that design decisions are a very usefultool for storing and tracing the results of the design decision-making process, it improves reuse. Theyalso reported that documented design decisions and their rationale can facilitate the transfer of knowl-edge about key design decisions in the international teams.

Assigned roles: Both VUA and UDA students found the assignment of roles and roles rotation agood idea to better organize and distribute the teamwork. However, the use of so many roles has beenperceived as a limitation (or overload) in local teams, which had the possibility to meet at any time. Atypical comment we received was: “why having an aligner role, if we can easily meet in person anddiscuss together?” About role rotation, four teams reported that role rotation on a weekly basis wastoo fast, creating unnecessary overhead.

Table VI. Questionnaire PIV: Responses to post-study questionnaire (importance of rationale).

Question 1(impact on communication improvement)

Twenty-six (67%) participants were of the opinion thatexplicitly documenting design decisions and rationalewas very important; 12 (31%) of the participants thoughtit is important; 3 (7%) participants said that it was some-what important.

Question 2(effectiveness of design rationale availability)

Eleven (28%) participants said that the availability of de-sign rationale in performing detailed design tasks wasvery effective; 25 (63%) found the availability of ratio-nale effective; however, 4 (10%) reported that the avail-ability of rationale was ineffective for detailed designtasks.

Question 3(rationale availability facilitates design)

Thirty-one (80%) of the participants were either fullyconvinced or convinced that the time and effort spent forexplicitly describing design decisions and capturing ratio-nale was fully justified; 8 (20%) of them were somewhatconvinced that it was justified.

Question 4(rationale availability enables fastaccess to decisions)

Thirty-seven (95%) participants strongly agreed oragreed that the access to design rationale enable them toquickly understand the design decisions without contact-ing the original designers; 2 (5%) participants disagreedwith this statement.



Social issue: The participants in the international teams did not know each other. In some cases, thisturned out to be problematic. Interestingly enough, four students recognized this as an added value:they commented that “no physical meeting and no personal knowledge can improve productivity,because the meeting focus is more about the common work”.

Timing issue: The participants in the international teams could not arrange meetings on short noticeto discuss urgent matters. A relevant factor was that UDA students would live in a small town(L’Aquila) where face-to-face meetings could be organized relatively quickly. The VUA students,instead, would live in a bigger city (Amsterdam) where the organization of a physical meetingcould become more problematic and time consuming.

Tooling support: The international teams must be provided with a set of appropriate tooling supportfrom day 1 in order to avoid delays in starting the project communication and coordination activities.Some teams decided to use their own set of collaboration tools, and eventually ended up spending alot of time on just selecting and configuring the tools used for the teamwork. This resulted in delays,misunderstandings, and inconsistencies that could have negatively influenced the quality of the workand the feelings of belonging to a team.

6. LIMITATIONS AND VALIDITY THREATS

The validity of the findings from analyzing the self-reported quantitative and qualitative data gatheredthrough self-administered questionnaire may suffer as people may tell what they expect the researchers tohear rather than their true thoughts, reflections, and observations. Or the respondents may havemisinterpreted the questionnaire, especially when they were asked to assume to be working in a situationin which they had not actually worked. However, we provided clear guidelines and explanations for eachquestion. Moreover, the questionnaire consisted of the questions, which had been used and found reliablein our previously conducted studies with the similar objectives [17, 18, 11]. Another threat to the validityof our study can be the method used to assess the outcome of the deliverables submitted by the local andinternational teams. We tried to minimize any potential impact of the grading scheme by having twolecturers-in-charge from both universities involved in this study to design a common grading scheme andthen apply that grading scheme consistently. Any disagreements about the grades and criteria usedwere discussed and resolved before finalizing the grade for data analysis purposes.

A reader should also be aware that the participants’ performance might have been affected by factorssuch as experience, technical and soft skills, and amount of time dedicated to the project. Whereas wehad asked the students to record the time spent on the project, there was a large variation in the amountof time claimed to be spent on the project by different participants; because the data reported were notreliable, we decided not to take that data into consideration for any analysis. However, many studentswho had worked in the international teams reported extra work on communication and collaborationtasks as a result of working in the international teams.

Geographical location of the participants, mainly in Italy and the Netherlands, can be another concern thatmay have some specific aspects. The students from the Netherlands were from multi-cultural backgroundscompared with Italian students. The participant representativeness is an issue because the participants weregraduate students with limited industrial experience. Researchers have suggested that software engineeringstudents can be used instead of professionals for empirical studies [19]. The international teams in ourstudy reported the use of collaborative and communication applications (e.g., google docs, Skype,and emails), which are also commonly used by the industrial global teams to support their activities.Moreover, the project work assigned to the international teams was also close to industrial projectwork. We assert that some of the findings from this study can be applicable to industrial projects.

7. CONCLUSION

Based on this study, we have learned that in projects requiring complementary skills, students ininternational teams can benefit from learning by osmosis and can perform as well as the students inlocal teams despite the extra effort usually required for GSE projects. Our findings are based on the



analysis of the quantitative data gathered from grading the students’ project deliverables and the self-reported quantitative and qualitative data gathered using a post-study questionnaire. Our findingssuggest that “learning by osmosis” can help reduce the required amount of training in a particularsubject matter for global software engineers as they can compensate the lack of formal training andextra effort required in GSE projects through learning by osmosis.

Our analysis of the gathered data has also revealed that students in the international teams doeffectively exchange complementary knowledge, hence, performing equally well compared with thestudents who worked in the local teams despite the students working in the international teams wereexpected to put extra effort because of the communication, coordination, and collaboration issuesthat characterizes GSE projects. Back to our research question – “Can learning by osmosis influencethe performance of the international teams when cooperatively working on a GSE project?” – we cananswer that learning by osmosis does positively influence the performance of the international teams:international teams exchange complementary knowledge by osmosis, which allows them to bothperform equally well compared with local teams, and dedicate the spare time to manage the GSEchallenges of communication, coordination and communication. This finding has a positivepedagogic effect: setting up shared projects with the appropriate knowledge transfer mechanismsallow educators to teach less in the traditional way (in the classroom) and creates new opportunitiesto teach the globalization aspects of software engineering in a more structured way. To providefurther evidence in support of the findings from the quantitative data, we analyzed the self-reportedquantitative and qualitative data gathered about different aspects of the teamwork experiences andperceptions in different settings, that is, local and global. Whereas the results do confirm the findingsbased on the quantitative data, our analysis of questionnaire-based data revealed interesting andsurprising details: (i) working in the GSE teams is perceived as effort consuming, but at the sametime performance does not decrease. We believe that this might be an indication of the negativeperception that the students might have of the GSE projects. This finding represents a potential riskfor educators, and should be carefully considered in the design of GSE courses. (ii) Role aligner,which has been introduced to timely identify inconsistencies between the outputs of the studentsworking remotely, had the unforeseen positive effect on the teammates: by knowing that someonewas double checking their work, they were confident of achieving a high quality project result; and(iii) Unfortunately, the students in our study also confirmed a well-known side effect of GSE, namelythe lack of trust in the remote teammates. This suggests that software engineering practitioners needto introduce explicit mechanisms for building trust among members of GSE projects [15].

ACKNOWLEDGEMENT

Thanks to Antony Tang for the useful comments and to the participants of the study.

REFERENCES

1. Herbsleb JD, Moitra D. Global software development. Software, IEEE 2001; 18(2):16–20.2. Bondi AB, Ros JP. Experience with training a remotely located performance test team in a quasi-agile global

environment. In: Fourth IEEE International Conference on Global Software Engineering, 2009; 254–261.3. Damian D, Izquierdo L, Singer J, Kwan I. Awareness in the wild: why communication breakdowns occur. In: Second

IEEE International Conference on Global Software Engineering, 2007; 81–90.4. Conchúir E, Ågerfalk P, Holmström OH, Fitzgerald B. Global software development: where are the benefits?

Communication of ACM 2009; 52(8):127–131.5. Tervonen I, Mustonen T. Offshoring test automation: observations and lessons learned. In: Fourth IEEE International

Conference on Global Software Engineering, 2009; 226–235.6. Ebert C, De Neve P. Surviving global software development. Software, IEEE 2001; 18(2):62–69.7. Fabriek M, van den Brand MSB, Harmsen F, Helms R. Reasons for success and failure in offshore software

development projects. In: Proceedings of the 16th European Conference on Information Systems, 2008; 1–12.8. Damian D. Global software development: growing opportunities, ongoing challenges. Software Process:

Improvement and Practice 2003; 8(4):179–182.9. Komi-Sirviö S, Tihinen M. Lessons learned by participants of distributed software development. Knowledge and

Process Management 2005; 12(2):108–122.10. Damian D, Moitra D. Global software development: how far have we come? IEEE Software 2006; 23(5):17–19.11. Lago P, Muccini H, Ali Babar M. Developing a course on designing software in globally distributed teams. In: The

3rd International Conference on Global Software Engineering, 2008; 249–253.



12. Lago P, Muccini H, Beus-Dukic L, Crnkovic I, Punnekkat S. GSEEM: teaching software engineering in a Europeanand global setting. International Journal of Engineering Education 2008; 24(4):747–760.

13. Shadish WR, Cook TD, Campbell DT. Experimental and Quasi-Experimental Design for Generalized CausalInference. Houghton Mifflin: Boston, USA 2002; 623.

14. Al-Ani B, Redmiles D. Trust in distributed teams: support through continuous coordination. IEEE Software 2009; 26(6):35–40.

15. Ali Babar M, Verner J, Nguyen PT. Establishing and maintaining trust in software outsourcing relationships: an em-pirical investigation. Journal of Systems and Software 2007; 80(9):1438–1449.

16. Tang A, Ali Babar M, Gorton I, Han J. A survey of architecture design rationale. Journal of Systems and Software2006; 79(12):1792–1804.

17. Ali Babar M, Kitchenham B, Jeffery R. Comparing distributed and face-to-face meetings for software architectureevaluation: a controlled experiment. Empirical Software Engineering 2008; 13(1):39–62.

18. Ali Babar M, Kitchenham B, Zhu L, Gorton I, Jeffery R. An empirical study of groupware support for distributedsoftware architecture evaluation process. Journal of Systems and Software 2006; 79(7):912–925.

19. Host M, Regnell B, Wohlin C. Using students as subjects – a comparative study of students and professionals inlead-time impact assessment. Empirical Software Engineering 2000; 5:201–214.

20. McCue A. Global IT deals rocket up to $163bn. 2005. http://www.silicon.com/management/cio-insights/2005/01/18/global-it-outsourcing-deals-rocket-up-to-163bn-39127146 [18 August 2011].

AUTHORS’ BIOGRAPHIES:

Patricia Lago is Associate Professor at the VU University Amsterdam, the Netherlands.Her research interests are in software-oriented and service-oriented architecture, architec-tural knowledge management, green IT and sustainable software engineering. Lago has aPhD in Control and Computer Engineering from Politecnico di Torino. She co-edited var-ious special issues and co-organized workshops on topics related to service-oriented andsoftware-oriented architecture, architectural knowledge management, and green IT. Shehas been Program Chair of the IEEE/IFIP Working Conference on Software Architecture,and is member of IEEE and ACM, and of the International Federation for InformationProcessing Working Group 2.10 on Software Architecture. She is Area Editor ServiceOrientation for the Elsevier Journal of Systems and Software. She published over 100peer-reviewed articles in international journals, conference proceedings, and books. Sheis local coordinator of the Global Software Engineering European Master Programme.

Henry Muccini is Assistant Professor at the University of L’Aquila, Italy. Henry’s re-search interests are in the Software Engineering field, and specifically on software archi-tecture modelling and analysis, component-based systems, model-based analysis andtesting, fault tolerance, and global software engineering. He has published various confer-ence and journal articles on these topics, co-edited two books on “Software Engineering ofFault Tolerant Systems” and “Architecting Fault Tolerant Systems V”, and co-organizedvarious workshops on related topics. He serves as PC member and reviewer in many inter-national conferences and journals. He is local coordinator of the Global Software Engi-neering European Master Programme. More detailed information may be found at www.HenryMuccini.com.

Muhammad Ali Babar is an Associate Professor at IT University of Copenhagen,Denmark. Previously, he worked as a researcher for different research institutes in Australiaand Ireland. He has authored/co-authored more than 100 peer-reviewed research papersin journals, conferences, and workshops. He co-edited a book, Software ArchitectureKnowledge Management: Theory and Practice. Dr Ali Babar has been guest editors ofseveral special issues/sections of IEEE Software, JSS, ESEJ, SoSyM, IST, and REJ. Apartfrom being on the program committees of several international conferences such asWICSA/ECSA, ESEM, SPLC, ICGSE, and ICSSP for several years, Dr. Ali Babar is alsochairing the program committees of several conferences such as WICSA/ECSA 2012,ICGSE 2011, ECSA 2010, and PROFES 2010. He is a member of the steering committeesof WICSA, ECSA, and ICGSE. He has presented tutorials in the areas of software archi-tecture and empirical approaches at various well-known international conferences. Prior tojoining R&D field, he worked as a software engineer and an IT consultant for several yearsin Australia. His current research interests include software architecture, cloud computing,global software development, and evidence-based software engineering. He obtained aPhD in Computer Science and Engineering from University of New South Wales, Australia.



an empirical study of learning by osmosis in global software engineering

Documents