9th pre-icis international research workshop on ......9th pre-icis international research workshop...

eProceedings of the 9th International Research Workshop on Information Technology Project Management (IRWITPM)

Auckland, New Zealand, December 13th, 2014 1

9th Pre-ICIS International Research Workshop on Information Technology Project Management (IRWITPM 2014) Workshop Proceedings 12/13/2014

AIS Special Interest Group on Information Technology Project Management



Contents

Workshop Welcome ...................................................................................................................................................... 3

Workshop Committee .................................................................................................................................................... 3

SIG ITProjMgmt Officers.............................................................................................................................................. 3

Sponsor Thanks ............................................................................................................................................................. 4

Exploring Prior Work History within Software Project Teams by Stacie Petter, University of Nebraska at Omaha;

Michelle Carter, University of Washington ................................................................................................................... 5

Increasing Customer Satisfaction – How to Manage Expectations in the Process of Developing Information

Systems* by Georgios Stavrou, University of Cologne; Oleg Pankratz, University of Cologne; Dirk Basten,

University of Cologne ................................................................................................................................................. 14

A Complex Adaptive Systems Perspective on Self-Organization in IS Project Portfolios by Roger Sweetman,

National University of Ireland, Galway; Orla O’Dwyer, National University of Ireland, Galway; Kieran Conboy,

National University of Ireland, Galway ....................................................................................................................... 28

Measuring Coordination in Agile Software Development by Diane Strode, Whitireia Polytechnic ........................... 37

Software Development in Multiteam Systems: A Longitudinal Study on the Effects of Structural Incongruences on

Coordination Effectiveness by Saskia Bick, University of Mannheim; Alexander Scheerer, University of Mannheim;

Kai Spohrer, University of Mannheim; Thomas Kude, University of Mannheim; Armin Heinzl, University of

Mannheim .................................................................................................................................................................... 49

Note: * marks best paper award.



WORKSHOP WELCOME

Welcome to Auckland for the 9th International Research Workshop on IT Project Management (IRWITPM 2014)

sponsored by the AIS Special Interest Group for Information Technology Project Management. This year's

workshop includes two completed research papers, three research-in-progress papers, and one panel.

It is my sincere hope that this year’s workshop will continue to facilitate the exchange of ideas between IT project

management researchers, educators, and practitioners from around the world and provide an opportunity for us to

renew and extend our network of IT project management colleagues.

I would also like to take this opportunity to thank the workshop authors, reviewers, participants, organizers, and

sponsors (Project Management Institute and the University of Nebraska at Omaha). Without these individuals, our

ninth annual meeting would not have been possible. Thank you again for engaging with this AIS SIG, and I hope

you continue to participate in its activities.

Alanah Mitchell, Appalachian State University, SIGITProjMgmt President

WORKSHOP COMMITTEE

Alanah Mitchell, Appalachian State University (Workshop Program Co-Chair)

Stacie Petter, University of Nebraska at Omaha (Workshop Program Co-Chair)

SIG ITPROJMGMT OFFICERS

Alanah Mitchell, Appalachian State University (President)

Michael Cuellar, Georgia Southern University (Secretary)

Radu Vlas, University of Houston – Clear Lake (Treasurer)

John Tripp, Baylor University (Communications and Publicity Chair)

Cecil Chua, University of Auckland (Membership and Community Relations Chair)

Deepak Khazanchi, University of Nebraska at Omaha (Founder)



SPONSOR THANKS

Petter & Carter Exploring Prior Work History within Software Project Teams



Exploring Prior Work History within Software Project Teams

Stacie Petter

University of Nebraska at Omaha

[email protected]

Michelle Carter

University of Washington

[email protected]

ABSTRACT

Software project management is challenging not only due to the technical requirements associated with creating

software, but also in dealing with interpersonal issues that arise during the course of a project. One interpersonal

dynamic within software project teams that is rarely discussed is the interaction among the team members

themselves. Using social identity theory as a lens, this research explores how subgroups based on individuals’ prior

work history could impact the project team. These prior working relationships could be a benefit to the team, or

alternatively, could create favoritism among some members of the team. We explore the phenomenon of how prior

work history affects the project team’s dynamic in the context of a massive multiplayer online role playing game

(MMORPG). Using the results, we offer suggestions for future research and practice to consider the impact of

social identity within software project teams.

Keywords

Software project teams, team dynamics, social identity theory.

INTRODUCTION

Software project teams are comprised of group members that must work together to accomplish tasks for the sake of

the greater goal of the project. Challenges arise when conflict negatively impacts a team’s ability to complete tasks

effectively (Kankanhalli et al. 2006). When dysfunctional conflict occurs within teams, it becomes increasingly

difficult for project managers to manage projects effectively.

Dysfunctional conflict may occur as individuals categorize themselves into different groups. For example,

individuals may find an affinity for other individuals in the project team that perform similar types of work, have a

similar background, or with whom they have a prior history of working together. As individuals within a project

team develop a sense of who they are, based on their relationships with fellow team members, their perception of

themselves and the social groups within the team can negatively or positively impact the project team’s processes

(Adams and Anantatmula 2010). When individuals identify strongly with a group, this can create feelings of

favoritism towards others in the group and discrimination towards others that are not group members (Turner 1975).

Using the lens of social identity theory, the objective of this study is to explore how teams might be impacted when

some of the team members have history or working together on past projects. We explain how we explored this

issue in the context of a massively multiplayer online role playing game and how we can use these findings to

identify opportunities for future research in software project management.

THEORETICAL BACKGROUND

Social Identity Theory

Humans naturally classify and categorize the world around them, and this is also consistent with how individuals

view themselves. Individuals create a sense of social identity by defining oneself based on membership in a social

group (Tajfel and Tuner 1979). Social identity theory examines the reasons why individuals in one group (i.e., an

in-group) discriminate against members in another group (i.e., an out-group). Conflict typically occurs within a

group when there are scarce resources, which promote competition among team members (Campbell 1965);

however, individuals may feel a preference towards members of their own group without any competition, even

when there are no monetary rewards or no need to complete a task (Turner 1975).




Social identity theory assumes that a) individuals want to view themselves positively; b) individuals believe that

being a part of a social group can be perceived positively or negatively; and c) one examines if their social group is

contributing positively or negatively to their sense of self-esteem by comparing their current group to other groups

(Tajfel and Tuner 1979).

Social identity theory informs us that during intense times of conflicts, individuals will not engage as individuals,

per se, but will rather act as a member of a group (Turner 1975). If a group feels threatened or devalued, the group

responds defensively (van Knippenberg 1984). However, this group rivalry does not occur unless individuals begin

to compare themselves to another group (Tajfel and Tuner 1979).

While social identity theory explains how groups respond to conflict, there are also benefits to having a strong social

identity in organizations. When a group has a strong social identity, it improves the team members’ commitment for

the organization, creates a sense of group norms so the group behaves as one unit, and improves group cohesion,

cooperation, and perceptions of the group (Ashforth and Mael 1989). Therefore, social identity can be something

that can create conflict or can mitigate conflict, depending on how one’s social identity is formed.

Social Identity in Software Project Teams

Within a software project, it is possible for individual team members to categorize themselves into subgroups. The

categorization of subgroups may be based on the type of work that they do (programmers versus quality assurance),

status (managerial titles versus non-managerial titles), or department (Ashforth and Mael 1989). When groups are

formed within a project team, those that view themselves similarly (e.g., programmers) would consider themselves

as an “in-group” and all others on the team (e.g., non-programmers) would be perceived as an “out-group.”

Another categorization that could establish social identity is one’s prior work history with other individuals on the

team. Individuals that had a positive joint work experience or project outcome would be more likely to view selves

as a group, creating a sense of social identity. However, if the prior work history was not positive, then individuals

may distance themselves from the in-group with a shared work history (Jackall 1978).

When individuals are assigned to work on a software project team, it may not be immediately known who has a

prior history of working together on past projects. To any individual that does not have a prior work history with

others on the team, that individual may not perceive an in-group versus an out-group. Figure 1 demonstrates how

individuals may perceive the same software project team based on their work history. Panel 1 demonstrates how

individuals with a prior work history (e.g., Person A or B) would perceive an in-group and an out-group. Panel 2

demonstrates how all other individuals without a prior work history (e.g., Person C, D, or E) would likely the view

the same team as a unit with no subgroups.

i.

Figure 1: Varying Perceptions of Software Project Team based on Social Identity

Collective action, or joint actions as a group, can be explained and predicted based on one’s social identity (van

Zomeren et al. 2008) which can alter the software project team’s dynamic and how individuals work together within

the group. Within a project team, newcomers (Persons C, D, or E above) may seek to create a sense of self within

Out-Group No Group Identification In-Group

A

B

D

E

C

B

A C

E

D

Panel 1: Perception of team by individuals

with prior work history

Panel 2: Perception of team by individuals

with no prior work history




the group (Ashforth and Mael 1989). People want to compare their beliefs with others that are similar to them

(Festinger 1954), so it is normal for newcomers to take a position similar to the in-group (Ashforth and Mael 1989).

Within a project team, members of the in-group could set the norms for the group because they would be a “united

front,” so to speak. However, members of the out-group (without even knowing that there is an in-group or an out-

group) could feel like they should go along with the in-group in the interest of being a part of the larger group.

Those that have had a negative prior work history may choose to disassociate with the in-group (e.g., Person C in

Panel 1) or view themselves as an individual rather than a group (e.g., Panel 2). Given that social identity can

negatively or positively impact the team’s dynamic, there is the potential to identify how social identity theory may

help to provide different insights into project team dynamics.

RESEARCH STUDY

In this exploratory study, we took a field study approach to investigate project teams in action. A Massively

Multiplayer Online Role Playing Game (MMORPG) provided the context for our investigation. In MMORPGs,

players develop avatars to represent themselves as they play the game to acquire resources and to accomplish goals.

While it is possible to play MMORPGs alone, many individuals work in teams alongside both known and unknown

players.

In many MMORPGs, players can choose to participate in a league. A league allows individuals to choose to

associate themselves with a certain people that play the game. An individual may know their league members in

real life or only through the game. If a person chooses to play in a league, it is possible to complete quests within

the game a) alone with no members of their league; b) only with members of their league; or c) with some members

of their league and with other players that are not part of their league. Individuals that do not wish to participate in a

league are able to play alone and work in ad hoc groups to complete quests.

Field Study Setting

We examined team interaction within an MMORPG quest. A quest is completed by five individuals working

together to accomplish a series of tasks. In the quest, the team has a shared goal to complete the quest, but each

individual has their own desire to acquire resources for personal needs, improve player statistics, or gain experience

points to increase skill or standing within the game. Individuals can join a quest with fellow league members or may

choose to complete the quest with strangers within the game as an online ad hoc team.

In a quest, each player adopts one of the following roles:

Soldier – Three people within the team have the role of attacking and destroying computer-generated enemies

within the quest. The soldier’s job is to defeat these enemies (often appearing in large groups or mobs) to

complete the quest and acquire resources.

Medic – One person that restores fellow team members that received damage during enemy attacks by healing

them or resurrecting them should the team member’s avatar die during an enemy encounter.

Captain – One person that serves as the primary target for the adversaries. The computer-generated enemies try

to prevent the team from accomplishing tasks within the quest. The goal of the captain is to draw enemy attacks

to allow the soldiers to effectively attack the opponents. The captain tends to set the pace for the team members

to complete tasks.

Field Study Task

To collect data, we solicited an individual outside of the research team to play as a confederate using a specific

protocol. The confederate was an individual with a high level of expertise, having over three years experience and

having reached the highest skill levels in the game with multiple characters.1

To develop the protocol for data collection, one of the researchers observed the confederate play several quests.

There were multiple discussions between the researcher and the confederate as to how to best observe team

dynamics in this context. We tested a variety of manipulations to ensure we had a behavior that could impact the

norms of the team that was strong enough to potentially elicit a response from team members. Through these

1 Within this MMORPG, an individual may have more than one character.




discussions among the researcher and the confederate, to explore team dynamics which may be impacted by social

identity, the confederate did not play as a member of a league and would encourage the team to alter its pace in the

completion of tasks. This is a behavior which the confederate had observed in his prior gameplay and could be

viewed as either a negative or positive change to the group norms depending on each team member’s needs.

Therefore, this task explores how team members respond when a non-group member proposes to alter the team’s

interaction during the project.

The completion of all of the tasks within a quest typically requires 30-60 minutes to complete with several

milestones that must be completed within each quest. To provide additional rigor to the study design, more

consistent with a field experiment, the confederate played the quest as he normally would (when he played on his

own, not engaged as a confederate) until the completion of the first milestone within the quest. Immediately after

this milestone, the confederate announced to the group “have to move this along i got to run soon.” The confederate

then proceeded through the quest as quickly as possible to alter the pace of the team and would do his best to

encourage his fellow team members to progress through the quest quickly.2

This change in pace was an attempt to alter the team’s norms to elicit a response. If one or more team members

moves ahead too quickly through the quest, other team members may not have enough time to replenish their

resources to complete their tasks, particularly if they are less skilled in the game. However, there could be team

members that would prefer a quicker pace due to their own personal time constraints. An analogy in an

organizational context could be if an individual within a team wants to quicken the pace of a project irrespective

other team members’ needs or preferences. The request made by the confederate to quicken the pace of the quest

could elicit either a negative response or a positive response from fellow team members depending on each team

member’s personal goals.

Relationship between Task and Software Projects

To demonstrate how an MMORPG may serve as analog to a software development project, Table 1 compares the

setting of a MMORPG quest to an agile software project across multiple dimensions.

Characteristic Agile Software Project MMORPG Quest

Objective Team members work together to complete a series of tasks while simultaneously balancing

constraints on time and resources

Team Size Can vary dramatically, but often use smaller

teams with nine or less individuals

Five individuals

Role Assignment Self-organized teams in which individuals

select their role

Individuals identify one of three roles that

they would like to have during the quest

Length of Project Typically weeks or months Approximately an hour

Collective Goal Seek to collectively complete the project

based on the customer’s needs and available

resources

Seek to accomplish primary objective of the

quest

Personal Goals Acquire new skills, knowledge, or contacts Acquiring resources and/or new skills

Leadership Facilitating role, such as a scrum master Facilitating role, such as the captain to help

set the pace

Table 1: Comparison of Software Projects and Study Context

As noted in Table 1, there are several dimensions in which there are similarities between an agile software

development project and an MMORPG quest. Certain aspects of agile software projects are more alike than

different from an MMORPG quest, such as small team size, use of a facilitator as a leader rather than a manager,

and a mix of personal and collective goals within the project.

Although not a complete analog to the workplace (Schultze et al. 2008), as a research setting, the MMORPG

environment has the potential to provide insight into organizational and group-level phenomenon (Assmann et al.

2 As a soldier, the confederate had little to no influence to encourage the team to proceed through tasks more quickly. Therefore,

his primary mechanism to encourage a faster pace was repeat his request to quicken the pace.




2010). MMORPGs can serve as a venue for exploratory research to explore issues to be further studied in

organizational settings. Another benefit of using MMORPGs as a research setting is the ability to study a

phenomenon by embedding a confederate to study team dynamics. MMORPGs also contain objective data and

allow recording of chat conversations and events, which allowed for team members’ reactions and activities to be

examined in detail.

Data Collection

During each quest, we recorded the avatar names of the other team members as well as chat logs and event logs. We

visited a third-party website and identified each avatar’s progress within the game, including history of completing

quests, and league membership. The chat logs recorded all public conversations within the quest among the players.

The event log recorded major events, such as if a team member’s avatar died during the quest or if players left or

entered the quest during game play.

All chat and event logs were reviewed with the confederate within 24 hours of completing each quest, with most

logs being reviewed with the confederate within 4 hours of him completing each quest. This review allowed the

confederate to explain the context of the activities that were recorded in the chat log and event log. Further, the

confederate would take notes after each quest giving his opinion as a player of the game as to how well the team

worked together to complete the tasks within each quest.

The confederate played thirty quests in the role of a soldier as described above. Three people on the team play as a

soldier (including the confederate), making the position generic and easy to replace on the team. On a project team,

this role would be the equivalent of someone with a generic skill set that is easy to replace should the team member

leave the project.3

Data Analysis

We analyzed the data by carefully reading the chat logs and event logs. One researcher would also speak with the

confederate about quests to gain additional insight not provided by the chat and event logs.

Each event and chat log was coded to identify reactions to the confederate’s request to quicken the pace of the team

as well as if the reaction was positive or negative. The chat and event logs also enabled us to identify the severity of

the reaction as well as how quickly each individual reacted (if at all) to the confederate’s request to speed up the

pace of the game. This data was triangulated with the information provided by the confederate at the end of each

quest to ensure we had a complete understanding of the team dynamic.

For each team member in a particular quest, we used chat and event logs to record the number of times an individual

supported or discouraged the confederate’s request to quicken the pace, a code to represent the nature of support or

discouragement, and if the individual was part of a league or not. Further, since all activities recorded in the chat

and event logs have a timestamp, we could also identify the length of time that it took for a person to express

support or displeasure about the confederate’s request. We compared the reactions of non-league members with

league members. We also examined team dynamics among teams with no league members and those with two or

more league members. This allowed us to use both qualitative and quantitative data to examine how league

members and non-league members reacted to the confederate’s requests to alter the pace of the quest.

RESULTS

We were unable to control for whether or not the confederate would be able to play a quest with members of a

league. Once the quest was completed, it was possible to identity if the fellow players were part of a league or not.

In some quests, the confederate had a suspicion that some team members knew one another, but this was rarely

confirmed during the quest. This is analogous to software projects in that it is not always obvious at the onset if

individuals have had a prior working history.

3 The confederate also completed thirty quests as a captain to compare how team members would respond to someone in a

different role making the request; however, length constraints for this workshop prevent us from discussing those results in this

paper.




Eleven of the thirty teams had members in leagues, with nine groups having two league members, one group having

three league members, and one group having four league members.

We expected that it could be likely that individuals that play in leagues would be more skilled than individuals that

do not play in a league. Individuals in leagues have a strong desire to play the game and are part of a larger group,

suggesting that they may be more invested in playing the game more than others. However, individuals that played

in leagues had only a slightly higher skill level than those individuals that were not part of a league (no significant

difference in skill level for league versus non-league players using a t-test).

Individuals that were playing with league members were more likely to express a negative opinion of the

confederate’s urging to speed up the pace of the quest than non-league members. Further, league members

expressed their discontent with this strategy more quickly than individuals that were not part of a league. In one

scenario in which three of the team members were league members, only the confederate spoke publicly. The first

record of any conversation in the public chat log occurred when the confederate asked to quicken the pace. When

there was no response, the confederate pushed the team again to move along quickly. The only item in the chat log

from any team member other than the confederate was a league member, who responded negatively in an attempt to

silence the confederate.

13:24:20 [Confederate]: have to move this along i have to run soon

13:26:40 [Confederate]: lets gogogo

13:26:51 [Medic-League Member]: shut up dude

In another quest, when the confederate proposed altering the pace of the group, teams that had members from a

league were sometimes quite harsh in their response to the confederate. In a quest, members could vote to eliminate

someone from the group with a majority vote (i.e., 3 out of 5 agreeing to remove a player). In one quest, the

confederate kept pushing the group to move faster. In this quest, a captain and soldier entered the quest together.

When the confederate stated a need to quicken the pace of the quest, the response from a soldier was sarcastic.

10:28:08 [Confederate]: have to move this along i have to run soon

10:28:23 [Soldier – League Member]: ah yeah we'll cater to a [soldier]

In this same scenario, the confederate kept prodding the group to move quickly. However, after a fifth request to

quicken the pace by stating “last one lets gogogo,” the group expelled the confederate from the team just before the

final task. The confederate felt like he was being punished so he would not receive any rewards by completing the

quest with the team.

In another scenario, the confederate stated his need to move things quickly. Five minutes later, things had slowed to

their original pace after a soldier that was not part of a league made a mistake. After stating “lets go go go” with no

verbal response and following up two minutes later with “come on guys whats the hold up”, the confederate was

expelled by the group only halfway through the quest.

Occasionally, fellow team members would support the idea of speeding up the pace of the game. Those in leagues

were less likely to agree with the idea to quicken the pace of the group, but if someone in the league was positive

about the idea, then they responded very quickly to affirm their agreement with the idea (more quickly than non-

league players). In one quest, the confederate kept prompting the group to speed up the pace; the captain, part of a

league with another soldier, shared ways to avoid enemies to proceed through the quest more quickly.

DISCUSSION

This study explores team dynamics within a project team in which some of the team members have a prior history of

working together using the lens of social identity theory. Given that was an exploratory study, the goal was not to

prove or disprove social identity theory, but rather explore how team dynamics might be affected using the lens of

social identity theory when some team members had a prior work history.

For those with a prior work history, social identity theory predicts that members of a group would have a desire (and

often pressure) to respond as a group (van Zomeren et al. 2008). Further, social identity theory would state that

there would be a tendency to distrust outsiders, particularly if they try to disrupt the group (van Knippenberg 1984).




When the confederate expressed a desire to change the pace of the game, and in particular, when he did so multiple

times in an effort to effect change, there was a swift reaction by league members to suppress this behavior of the

confederate as compared to scenarios in which there were no league players. Meaning, those with a prior work

history (as league players) were quicker to respond negatively to the situation than non-league players.

When league members responded together as a force to verbally criticize the confederate or expel him from the

quest, non-league players may not have realized that a subgroup of league players existed within the larger team. As

soon as multiple team members seemed to be in agreement, social identity theory suggests there would be pressure

to conform to the group in the treatment of the confederate (Ashforth and Mael 1989).

Potential for Future Research

In most software projects, the interpersonal dynamics within the software project team have the potential to impact

the outcomes of the project. Yet, with the proliferation of agile software development methods, the interaction and

interdependence among team members increases. We have little research in the software project management

domain that explores dynamics within software project management teams and the impact of the team’s dynamic on

the project. Prior research has also suggested that rather than defining roles for each team members, individuals

should be allowed to self-organize (Rogers and Lea 2005). Given that agile software development methods tend to

rely on the principle of self-organizing teams, it would be interesting to examine how social identity in agile teams

varies from software project teams that use traditional or waterfall development approaches.

Social identity theory is rarely used in the information systems or software project management literature with the

exception of discussion of national or organizational culture (Gallivan and Srite 2005; Hwang 2005; Straub et al.

2002). National culture (Straub et al. 2002; Hwang 2005), organizational culture (Gallivan and Srite 2005), and

other demographic variables such as gender (Kankanhalli et al. 2006) can affect how one defines themselves within

a group. Yet, this study offers alternative reasons to establish social identity, such as prior working history with a

team member. Future research could explore in a software project context how individuals create their social

identity, which is usually based on membership of multiple social groups (Straub et al. 2002; Gallivan and Srite

2005).

Future research could also consider how social identity based on prior work history impacts groups based on their

stage of development. For example, using Tuckman’s (1965) stages of group development, forming, storming,

norming, and performing, researchers could consider the positive and negative impacts of social identity within the

project team based on the stage of team development. Tuckman’s research has been criticized for failing to consider

the impact of social identity (Lembke and Wilson 1998); however, the impact of social identity on team dynamics

could be useful in understanding software project team dynamics. The manner in which social identity is developed

among members of a team is likely to have its beginnings in the forming stage. Social identity theory has the

potential to explain storming within a team (i.e., conflicts) and explain the development of team norms (i.e., team

behaviors) (Hogg and Reid 2006).

Most studies that examine social identity theory examine this phenomenon based on face-to-face interaction;

however, social identity has been observed in distributed groups that leverage technology for interaction (Rogers

and Lea 2005). Future research should continue to explore the role of social identity in both traditional, face-to-face

software project teams and distributed project teams and could examine if social identity is developed differently

based on the interaction of the group.

Considerations for Practice

When staffing a software development team, some project managers may seek out individuals with a successful joint

work history with other members of the project team. Social identity theory suggests that individuals’ self-

categorizations into subgroups could impact the team’s dynamic. Individuals categorize themselves into groups for

nominal reasons. Project managers should consider how such self-categorization (based on prior work history,

among others) could impact the team. In staffing software project teams, it is important for managers to be aware of

how strong cultures, language, and national heritage can influence creation of subgroups, which can subsequently

lead to conflict (Kankanhalli et al. 2006). A further concern that can occur when individuals within a team develop

a social identity based on a subgroup within the team is that out-group team members feel pressured to engage with

the team in a certain way.




We are not recommending that project managers avoid staffing individuals with a prior work history on the same

project, but rather that managers should recognize that there could be both intended and unintended consequences to

this choice. Therefore, software project managers may want to seek out approaches to help the group create a social

identity as an entire project team, rather than as subgroups within the team. Shared social identity creates a stronger

level of commitment to the team which can reduce conflict (Mortensen and Hinds 2001); therefore, if all team

members in the project team view themselves as a single group, rather than subgroups, there is the potential to reap

the positive benefits of social identity, such as group cohesion, less conflict, and higher levels of esprit de corps

(Ashforth and Mael 1989).

CONCLUSION

This study was an exploratory effort to examine if prior working relationships has an impact on project team

dynamics. Given that prior research typically assumes that prior work history can be a positive attribute within a

team, this study offers an alternative perspective. Few studies examine social identity in the context of software

projects. Research has found that project workers have develop a sense of identity with their organization and their

profession (Dwivedula and Bredillet 2010); however, the growth of agile methods and interdependence among team

members makes this theoretical lens ripe for additional study. While we have research that examines interactions

across other project team stakeholders, such as project managers and project sponsors (Krane et al. 2012) or project

managers and executives (Keil et al. 2014), we have little research examining the interpersonal dynamics within

software project teams. By looking inward at the dynamics within the software project team, we have potential to

examine and address additional issues that can positively or negatively impact software project management.

ACKNOWLEDGEMENTS

We wish to thank Rick Petter for helping to collect data for this study by playing as the confederate in the

MMORPG. We also thank the anonymous reviewers of this paper for their constructive comments in improving this

manuscript.

REFERENCES

Adams, S. L., and Anantatmula, V. (2010) "Social and Behavioral Influences on Team Process," Project

Management Journal, 41, 4, pp. 89-98.

Ashforth, B. E., and Mael, F. (1989) "Social Identity Theory and the Organization," Academy of Management

Review, 14, 1, pp. 20-39.

Assmann, J. J., Drescher, M. A., Gallenkamp, J. V., Picot, A. O., Welpe, I. M., and Wigand, R. T. (2010) "Mmogs

as Emerging Opportunities for Research on Virtual Organizations and Teams," in: Americas Conference on

Information Systems. Lima, Peru.

Brown, J. S., and Thomas, D. (2006) "You Play World of Warcraft? You're Hired!," Wired Magazine. Campbell, D. T. (1965) "Ethnocentric and Other Altruistic Motives," Nebraska Symposium on Motivation, D.

Levine (ed.), Lincoln, NE: University of Nebraska Press, pp. 283-311.

Dwivedula, R., and Bredillet, C. N. (2010) "The Relationship between Organizational and Professional Commitment

in the Case of Project Workers: Implications for Project Management," Project Management Journal, 41,

4, pp. 79-88.

Festinger, L. (1954) "A Theory of Social Comparison Processes," Human Relations, 7, 2, pp. 117-140.

Gallivan, M. J., and Srite, M. (2005) "Information Technology and Culture: Identifying Fragmentary and Holistic

Perspectives of Culture," Information and Organization, 15, pp. 295-338.

Hogg, M. A., and Reid, S. A. (2006) "Social Identity , Self-Categorization, and the Communication of Group

Norms," Communication Theory, 16, 1, pp. 7-30.

Hwang, Y. (2005) "Investigating Enterprise Systems Adoption: Uncertainty Avoidance, Intrinsic Motivation, and

the Technology Acceptance Model," European Journal of Information Systems, 14, 2, pp. 150-161.

Jackall, R. (1978) Workers in a Labyrinth: Jobs and Survival in Bank Bureaucracy. New York, NY: Allanheld,

Osmun.

Kankanhalli, A., Tan, B. C. Y., and Wei, K.-K. (2006) "Conflict and Performance in Global Virtual Teams," Journal

of Management Information Systems, 23, 3, pp. 237-274.

Keil, M., Smith, H. J., Iacovou, C. L., and Thompson, R. L. (2014) "The Pitfalls of Project Status Reporting," MIT

Sloan Management Review, 55, 3, pp. 57-64.




Krane, H. P., Olsson, N. O. E., and Rolstadas, A. (2012) "How Project Manager-Project Owner Interaction Can

Work within and Influence Project Risk Management," Project Management Journal, 43, 2, pp. 54-67.

Lembke, S., and Wilson, M. G. (1998) "Putting the "Team" into Teamwork: Alternative Theoretical Contributions

for Contemporary Management Practice," Human Relations, 51, 7, pp. 927-944.

Mortensen, M., and Hinds, P. J. (2001) "Conflict and Shared Identity in Geographically Distributed Teams," The

International Journal of Conflict Management, 12, 3, pp. 212-238.

Rogers, P., and Lea, M. (2005) "Social Presence in Distributed Group Environments: The Role of Social Identity,"

Behaviour & Information Technology, 24, 2, pp. 151-158.

Schultze, U., Hiltz, S. R., Nardi, B., Rennecker, J., and Stucky, S. (2008) "Using Synthetic Worlds for Work and

Learning," Communications of the Association for Information Systems, 22, 19, pp. 351-370.

Straub, D., Loch, K., Evaristo, R., Karahanna, E., and Srite, M. (2002) "Toward a Theory-Based Measurement of

Culture," Journal of Global Information Management, 10, 1, pp. 13-23.

Tajfel, H., and Tuner, J. (1979) "An Integrative Theory of Intergroup Conflict," in Social Psychology of Intergroup

Relations, W.G. Austin and S. Worchel (eds.). Brooks Cole Publishing.

Tuckman, B. W. (1965) "Developmental Sequence in Small Groups," Psychological Bulletin, 1965, 63, p. 6.

Turner, J. C. (1975) "Social Comparison and Social Identity: Some Prospects for Intergroup Behavior," European

Journal of Social Psychology, 5, 1, pp. 1-34.

van Knippenberg, A. (1984) "Intergroup Differences in Group Perceptions," in The Social Dimension: European

Developments in Social Psychology. Cambridge University Press, pp. 560-578.

van Zomeren, M., Postmes, T., and Spears, R. (2008) "Toward an Integrative Social Identity Model of Collective

Action: A Quantitative Research Synthesis of Three Socio-Psychological Perspectives," Psychological

Bulletin, 134, 4, pp. 504-535.

Stavrou et al. Managing Process Expectations in IS Projects



Increasing Customer Satisfaction – How to Manage Expectations

in the Process of Developing Information Systems

Georgios Stavrou

University of Cologne

[email protected]

Oleg Pankratz


[email protected]

Dirk Basten


[email protected] ABSTRACT

Considering success of information system development (ISD) projects a matter of perspective, stakeholder

satisfaction is often seen as an important success criterion. When evaluating satisfaction, expectations are essential –

in case of ISD projects expectations concerning both process and product. While previous research focuses on the

management of expectations concerning the product, lack of research exists concerning the process of ISD projects.

To close this gap, we explore the approaches that can be applied to manage expectations and guide customer

satisfaction with the process in ISD projects. By means of qualitative expert interviews, we focus on both types of

situations – those in which the experts were successful and less successful in managing customer expectations

concerning the ISD process. Our results from twelve interviews yield both concrete customer expectations (e.g.,

being involved by the contractor) and approaches to manage those expectations (e.g., creating transparency).

Researchers can use our results to further investigate concrete expectations and expectations management

approaches. Practitioners are provided with means to manage customer expectations, thus increasing customer

satisfaction and the likelihood of project success.

Keywords

Information systems, project success, customer satisfaction, expectation management, project management.

INTRODUCTION

About one third of all information system development (ISD) projects are considered unsuccessful since they do not

keep their plans regarding budget, schedule, and scope or need to be canceled (El Emam and Koru, 2008). Planning-

related criteria are traditionally applied to assess project success. However, projects are sometimes deemed

unsuccessful despite keeping their plans and successful even though not meeting plans (Baker, Murphy and Fisher,

1988; Nelson, 2005). Nelson (2005) denotes such projects as failed successes and successful failures. Instead of

confining oneself to planning-related criteria and claiming projects unsuccessful once they slightly deviate from

plans, additional criteria must be considered when assessing project success (Nelson, 2005). In particular,

importance must be attached to subjective criteria since success is a matter of perspective (Myers, 1995).

Information system (IS) literature suggests taking satisfaction into account when judging success of ISD projects.

The underlying assumption is that satisfied stakeholders tend to view a project as successful and unsatisfied

stakeholders as unsuccessful (Nevo and Wade, 2007). Predominantly, previous studies focus on user satisfaction

with the project outcome. DeLone and McLean (1992) identify user satisfaction with an IS as central for ISD project

success. Since user satisfaction significantly depends on user expectations (Zeithaml, Berry and Parasuraman, 1993),

realistic user expectations have been identified as a success factor for ISD projects (Ginzberg, 1981). Unrealistic

expectations are accordingly considered a major risk for ISD projects (Baccarini, Salm and Love, 2004). Petter

(2008) shows that user expectations regarding a project's outcome can be formed by managing expectations to

improve the satisfaction with the outcome and therefore the likeliness of project success. This finding is in line with

research claiming the management of stakeholder expectations within ISD projects to be an essential task in general

(Baccarini, 1999; Bourque and Fairley, 2014).

However, little research exists on strategies to meet stakeholder expectations in ISD projects (Petter, 2008). An

important differentiation in this regard concerns the concept of ISD project success, which is typically divided into




process success and product success (Baccarini, 1999; Basten, Joosten and Mellis, 2012; Liu, Chen, Chen and Sheu,

2011; Saarinen, 1996; Saarinen and Sääksjärvi, 1992). While the former refers to success of the development

process, emphasizing project management pillars like budget and schedule, the latter is concerned with success of

the project outcome, that is, the developed IS. Considering this distinction, expectations towards an ISD project may

differ among different stakeholder groups (Baccarini, 1999; Baker, Murphy and Fisher, 2008; Basten, Joosten and

Mellis, 2011; Freeman and Beale, 1992; Nelson, 2005; Nevo and Wade, 2007; Saarinen, 1996). While end-users of

an IS might be primarily concerned with characteristics of the IS and their performance using it, customer managers

commissioning the project might be more interested in the development process and agreed process indices like

schedule, budget, and requirements. Our research focuses on a specific stakeholder group – customer managers in

charge of an ISD project – and their satisfaction with the development process of the project. To increase this

satisfaction and thus overall project success, managers of the contractor organization need strategies to manage the

expectations of their counterparts in the customer organization. Accordingly, we pose the following research

question:

How can customer satisfaction be increased by managing customer expectations towards the ISD process?

Due to the novel character of our research, exploratory interviews are chosen as research method. As this work

advances into a scarcely considered field of study, we do not aim to cover all aspects regarding the possibilities of

expectation management related to the ISD process. Our findings can be seen as basis for further research and

indication of areas where it is needed.

In the next section, we review the current state of research regarding satisfaction, expectations, and expectation

management. We then describe our research approach, that is, the design of qualitative expert interviews and their

analysis. Subsequently, we present our results, followed by a discussion including implications for researchers and

practitioners as well as our study’s limitations. The article ends with a short conclusion.

RELATED WORK

Satisfaction in ISD Projects

Satisfied stakeholders evaluate a project as successful (Nevo and Wade, 2007; Wit, 1988). Stakeholder satisfaction

does not only depend on the fulfillment of project plans (e.g., projects meeting their plans can be considered

unsuccessful if stakeholders are dissatisfied with the project process or outcome). In turn, projects with a satisfying

process or outcome can be seen as successes even though not fulfilling their plans (Nelson, 2005). Satisfaction with

ISD projects is the sum of all stakeholders’ satisfactions (Nevo and Wade, 2007) since IS users cannot be satisfied in

the same way as project managers or developers (Nelson, 2005).

DeLone and McLean (1992, 2003) show that user satisfaction is a key criterion to measure success in most

theoretical and empirical studies. They focus on the project outcome and related satisfaction of its users. The process

and other stakeholders' views are not taken into account. Considering the ongoing quest for a comprehensive list of

success criteria, subjective ones (e.g., stakeholder satisfaction) are continuously gaining relevance (Nelson, 2005).

Ferreira and Cohen (2008) view stakeholder satisfaction with an ISD project as the sum of satisfaction with the

outcome and the process. They show that satisfaction with the process leads to satisfaction with the outcome and

conclude that early dissatisfaction with the process can contribute to dissatisfaction with the outcome later.

Dissatisfaction with ISD projects usually does not result from technical deficiencies; rather, it is caused by too little

attention given to psychological and organizational issues during development, roll-out, and usage (Markus and

Keil, 1994). The assumption of more powerful IS leading to increased user satisfaction has been weakened by

previous studies (Goodhue, 1995). Additional factors influencing the satisfaction with ISD include agile methods

(Ferreira and Cohen, 2008), user involvement (McKeen, Guimaraes and Wetherbe, 1994), and cost effectiveness

(Boyd, 2001).

Early studies in applied and social psychology show satisfaction to depend on expectations (Campbell, Converse

and Rodgers, 1976; Locke, 1969; Locker and Dunt, 1978; Shrauger, 1975). Besides technical aspects, expectations

play a major role when evaluating satisfaction of ISD projects (Conrath and Mignen, 1990). The importance of

expectations increases with the difficulty and ambiguousness of satisfaction assessments (Anderson and Sullivan,

1993). In particular, expectations play an important role when considering ISD as a service for which detailed




information is required, whose outcome can have extensive consequences, and whose goal is to build long-term

customer relationships (Ojasalo, 2001).

Expectations

Expectations are essential when evaluating satisfaction (Zeithaml et al., 1993) and are standards to evaluate

experiences. Different stakeholders have different expectations regarding ISD projects, which can overlap,

influence, or even contradict each other (Nevo and Wade, 2007). As Parasuraman, Zeithaml and Berry (1988) note,

different research streams define expectations in diverging ways. The following two theories regarding expectations

and their impact on satisfaction will be considered in this study.

Expectation-Confirmation Theory

Following Expectation-Confirmation Theory (ECT) (Oliver, 1980; Santos and Boote, 2003), relations between

expectations and satisfaction have been shown for different domains, including IS (Bhattacherjee, 2001).

Expectations are individuals’ predictions prior to usage of a product (Oliver, 1980) and a point of reference when

evaluating a product. Expectations’ confirmation through experience results in satisfaction. If experience diverges

from the expected, it leads to disconfirmation. If experiences exceed expectations, it results in positive

disconfirmation, which in turn leads to satisfaction. Negative disconfirmation (unfulfilled expectations) leads to

dissatisfaction. Even though disconfirmation has mainly been addressed in the context of consumer expectations

regarding products, ECT is not limited to physical products and can be transferred to ISD as a service

(Bhattacherjee, 2001; Olson and Dover, 1979).

Service Quality

The quality of a service is not easy to evaluate by objective criteria due to intangible nature, dependence on

customer and supplier, and close link of service provision and usage (Parasuraman, Zeithaml and Berry, 1985).

Therefore, people heavily rely on expectations when evaluating a service. Parasuraman et al. (1985) define service

quality as the discrepancy between expectations regarding a service and the experienced quality. Expectations in

service quality describe wishes how a service should be (Parasuraman et al., 1988; Santos and Boote, 2003). Service

quality is evaluated by measuring the discrepancy between expected and experienced service (Parasuraman et al.,

1988). In IS research, service quality has been mainly considered as user support from service providers as well as

the quality of information or functions of an IS (Pitt, Watson and Kavan, 1995).

Both theories underline the relevance of expectations for satisfaction. ECT’s concept of (dis-)confirmation can help

to understand the effects of expectations. Service quality highlights specific expectations, which can be tested with

regard to the ISD process. The definition of expectations as predictions is problematic since, according to ECT,

users must be satisfied if a system does fulfill their negative expectations about its outcome (Santos and Boote,

2003). In the following, expectations are thus viewed as wishes in reference to service quality.

Expectations regarding the ISD Process

Miller (2000) suggests that expectations in the ISD context are mainly focused on interpersonal relationships and

less on technical perfection or performance of the IS. He describes technical know-how, problem-solving or

consulting skills, and professionalism as potential expectations. Potter (2003) stresses the adherence to plans

regarding time and budget. Boyd (2001) mentions feedback, customer involvement, and conflict solution as

additional expectations regarding the ISD process.

Expectation Management

Expectation management is the process of confronting and forming expectations in order to generate advantages for

principals and agents (Miller, 2000). For this purpose, expectations have to be continuously monitored, understood,

and formed (Schmidt, Lyytinen, Keil and Cule, 2001). Expectation management aligns the views of different

stakeholders, helping to minimize unrealistic expectations and increase the overall project success (Bakker, Boonstra

and Wortmann, 2012). However, the immaterial and complex nature of ISD projects makes expectation management

a complicated endeavor (Baccarini et al., 2004). Various factors influencing customer expectations can be found in

literature (cf. Table 1). Even though these causes have not been identified while focusing on customer expectations

regarding the ISD process, they can be seen as hints for sources of unrealistic expectations in our expert interviews.




Factor References

Experiences of the customer from prior projects Lyytinen (1988), Zeithaml et al. (1993)

Mutual understanding (between customer and contractor of the

abilities, difficulties, and issues of the other party)

Boehm (2000)

Word-of-mouth recommendations from personal contacts or

experts (e.g., consumer reports or consultants)

Zeithaml et al. (1993)

Personal requirements and needs essential to the physical,

psychological, and social well-being of the customer

Zeithaml et al. (1993)

Excessive enthusiasm by contractor’s developers and

managers

Boehm (2000)

Promises made by the contractor to the customer

(e.g., advertising, personal selling)

Boehm (2000), Jørgensen and Sjøberg (2004),

Pitt and Jeantrout (1994), Zeithaml et al. (1993)

Table 1. Factors Influencing Customer Expectations

Furthermore, Table 2 lists approaches found in literature to manage stakeholder expectations in general. We

continue research by exploring the approaches applied in the context of ISD projects to manage expectations of

customer managers pertaining to the development process, which has not been in the focus of research yet.

Approaches to

Expectation

Management

Description References

Objective references

and models

Using benchmarks and well-calibrated models for cost or

schedule estimation to frame customer expectations

Boehm and Ross (1989),

Sheth and Mittal (1996)

Trust and understanding Building a trustful relationship by being honest – sharing

good as well as bad news openly throughout the project –

as well as by providing specific times for deliverables

Petter (2008)

User involvement Working interactively with users. Includes letting users

make decisions, listening to users and asking questions,

and keeping users updated throughout the project

Petter (2008)

Empathy Clarity regarding the goals and constraints of the other

party

Boehm (2000), Boehm and

Ross (1989)

Planning Establishing a realistic plan considering objectives,

milestones, responsibilities, approaches, and resources

Boehm (2000), Sheth and

Mittal (1996)

Communication Regular and clear exchange of information between

stakeholders

Boehm (2000), Boehm and

Ross (1989), Moynihan

(2002), Petter (2008),


Customer selection,

training, and orientation

Targeting desirable groups of potential customers and

educating customers on what they can realistically expect


Realistic promises by

sales department

Prior to project initiation, keeping promises realistic

rather than overly optimistic

Peters (1988)

Leadership Strong project manager/champion, social norms and

enforcement mechanisms. Articulating a clear project

vision and motivating the project team

Petter (2008), Sheth and

Mittal (1996)

Referencing

experiences and

alternatives

Often used to lower expectations, experiences from

former projects can be referenced and alternatives

suggested

Boehm and Ross (1989),

Kopalle and Lehmann

(2001)

Table 2. Approaches for Managing Expectations

RESEARCH APPROACH

To identify approaches for managing customer expectations concerning the ISD process, we rely on a qualitative

approach. For data collection, we use semi-structured interviews, which are prominent in IS research (Myers and

Newman, 2007) and an effective means to uncover unobserved links (Rubin and Rubin, 2005). For data analysis, we

aggregate the findings across the interviewees and focus on extracting approaches to manage customer expectations

concerning the ISD process.




Data Collection

For the acquisition of participants, we randomly contacted software-developing companies listed in the Hoppenstedt

company database (www.hoppenstedt-hochschuldatenbank.de). Since we were interested in approaches to manage

customer expectations, our respondents were project managers of the contractor organizations. We selected

informants with extensive experience in managing ISD projects. The interviews were conducted via telephone

during three months in the first half of 2013. In total, we interviewed twelve managers from twelve different

software-developing companies. Table 3 lists the informants (pseudonyms used to ensure confidentiality) along with

their experience in project management, company size, and highest qualification.

Pseudonym Experience as project

manager (# years)

Experience

(# projects)

# Employees in company Qualification

Mark 17 65 30 Diploma

Paul 39 39 12 Diploma

Thomas 15 6 20 PhD

Robert 8 30 30 Diploma

Kathy 23 20 350 Diploma

James 10 8 240 Diploma

Emily 15 30 40 Practical Education

David 7 120 25 Practical Education

Patrick 17 350 46 Practical Education

John 9 20 150 Practical Education

Michael 13 30 30 PhD

Ben 12 70 70 Diploma

⌀ 15 ⌀ 66 ⌀ 87

Table 3. Interviewee and Company Characteristics

Our interviews were organized in three parts: background of the respondent, customer satisfaction concerning the

ISD process, and customer expectations and their management to influence customer satisfaction (cf. Appendix A).

We asked each participant to recall two types of situations – those in which they were more and less successful in

managing customer expectations concerning the ISD process (Petter, 2008). Subsequently, we asked the respondents

to think of further insights into managing customer expectations of the ISD process. The interviews lasted between

45 and 70 minutes. They were audio-recorded and subsequently transcribed by one author for data analysis. For the

design of the interviews, we followed the guidelines by Myers and Newman (2007) as explained in Table 4.

Guideline Description of our Interview Design

1. Situating the

researcher

The interviewees did not know the authors in person, that is, we used cold calls to randomly

selected companies. The interviewees’ openness towards the interviewer may thus depend on

the interviewees’ trust towards the authors’ institution and the confidentiality of disclosure (cf.

guideline 7 below). We explained that the study is part of a PhD project at our university and

that the interviewer has an IS background (MSc Information Systems).

2. Minimizing

social dissonance

To minimize social dissonance, we aimed to ensure the interviewees feel comfortable at any

time. While the first contact was a cold call to the companies, we subsequently sent an email

outlining the research project and the role of interviews in it. The interviewer himself contacted

the potential respondents to ensure that any questions on behalf of the respondents could be

directly solved. By emphasizing that the interviewees are the experts and that no right or wrong

answers existed in this context, we carefully sensitized the participants for our study. Moreover,

we assured confidentiality (cf. guideline 7 below) and gave the participants full control over the

audio-recording (i.e., the participants could decide to turn off the recording at any time).

3. Representing

variety of voices

All respondents are or have been working in the position of an IS project manager. Since we

did not aim to assess the management of expectations within an intra-organizational context, we

interviewed managers from a variety of organizations to enable subject triangulation. By

randomly contacting companies and respondents, we are confident to have avoided biases

related to the selection of interviewees.




4. Everyone is an

interpreter

In order to reduce subjectivity, two authors independently analyzed the interviews and

conjointly aggregated the results in a subsequent step. Diverging assessments were discussed

until agreement was reached. For readers of this paper, we provide several direct quotes from

the interviews to enable a better understanding of the respondents’ views.

5. Using

mirroring

Beginning with general questions, we stepwise asked more specific questions about the

interviewees’ experiences. By assuring that no right or wrong answers existed in the context of

the study, we encouraged the interviewees to be as open as possible. We mostly used open

questions in our interviews (cf. Appendix A) to avoid imposing our wording on the

interviewees’. By asking for concrete project situations, we aimed to focus on vivid stories that

were revisited in follow-up questions.

6. Flexibility While following the interview guide in general, the interviewer paid special attention to the

responses given by the interviewees. In any occurrence of potentially relevant answers, the

interviewer followed the emerging line of inquiry and adapted the structure of the interview

accordingly.

7. Confidentiality

of disclosures

We guaranteed participants confidentiality and access to the aggregated results. In the

beginning of the interviews, we explained the procedures taken to ensure confidentiality and

adequate handling of the interviews. The interview transcripts were anonymized, that is, names

related to individuals or companies were replaced by pseudonyms. Subsequently, the links

between the transcripts and the interviewees were removed and the audio files deleted.

Table 4. Consideration of the Guidelines for Qualitative Interviews by Myers and Newman (2007)

Data Analysis

The data analysis was performed in three steps. These steps and the respective outcomes are illustrated in Figure 1.

First, the twelve recorded interviews were transcribed, resulting in twelve transcripts in the wording of our

respondents. Second, we coded the individual transcripts by assigning text passages to thematic labels. These labels

were either derived from our interview questions (e.g., relating to the relevance of the customer satisfaction with the

development process) or, in case of open questions, derived from the answers of our respondents (e.g., relating to

specific expectation management approaches like transparency). We read the transcripts several times to establish a

comprehensive understanding of the experts’ elaborations. Information was captured about customer satisfaction

with the ISD process, the expectations concerning the ISD process, the situations in which such expectations needed

to be managed, the approaches that the project managers decided to use, and the outcome of taking the approaches,

that is, whether influencing expectations has been successful. Finally, the thematically structured individual

transcripts were integrated into one table with interviews as rows, thematic categories as columns, and respective

text passages in the cells. Categories were derived by consolidating the thematic labels of the individual transcripts.

In the process of this integrated coding (cf. Figure 1), redundancies were eliminated and wordings of different

respondents consolidated.

Figure 1. Data Analysis

RESULTS

The results are organized in three subsections. First, we describe the impact of customer satisfaction with the

development process on overall project success, substantiating the relevance of this kind of satisfaction. Second, we

present the customer expectations towards the development process. We place emphasis on the third subsection –

12 interviews

12 transcripts

Thematically structured individual transcripts

Tabular categorization of all transcripts

1. Transcription

2. Individual coding

3. Integrated coding




the actual approaches to manage customer expectations, which can be applied to increase customer satisfaction and

ultimately project success. Concluding, Figure 2 visualizes the results.

Customer Satisfaction

All respondents indicate a high relevance of customer satisfaction with the development process. This is justified

with the resulting higher motivation of customers to cooperate during the project. With higher process satisfaction,

communication in the project increases and a cooperative climate is created in which customer and contractor

collaborate to realize ideas. However, the relevance of this kind of satisfaction can be lower if the customer does not

want a high degree of involvement in the project but expects the contractor to implement and deliver the requested

system autonomously.

Furthermore, all respondents see a connection between process and product satisfaction of the customer for two

reasons. First, a satisfying process is said to lead to a product that meets customer expectations since close

collaboration in such process reduces the risk of failing to fulfill customer needs. Second, a customer satisfied with

the process enters a coalition with the contractor on a psychological level and is more willing to overlook product

shortcomings. Regarding the impact of process and product satisfaction on the overall project success, the

statements of our respondents differ. While some experts attach higher importance to product satisfaction, others

consider process and product satisfaction to be equally important. However, there is general agreement that process

satisfaction has an impact on product satisfaction and both kinds of satisfaction influence the overall project success.

Customer Expectations towards Development Process

All but one respondents state that customer expectations towards the development process have a high impact on

customer satisfaction with this process. One expert reports on customers not having expectations at all. Others,

however, doubt that no expectations are a meaningful scenario as a customer without expectations has no interest in

the project. Moreover, expectations are often hidden rather than explicitly formulated.

Customer expectations can take different forms, which affects the impact of these expectations on customer

satisfaction. First, expectations can be realistic, that is, they reflect what is actually feasible. All respondents

consider realistic expectations to be ideal to satisfy customers. Second, expectations can be too high. Such

expectations are considered a risk for satisfying the customer since meeting too high expectations is usually difficult

or impossible. Customers with too high expectations need special care from the beginning since it is particularly

difficult to satisfy a customer who became unsatisfied early in the project. Finally, too low expectations can occur.

According to our respondents, such expectations play a minor role since they are rather rare and can be easily met.

Concrete expectations that emerged from our interviews are listed in Table 5.

Expectation Description # Respon-

dents

Customer

involvement

Customers expect to be involved in project management activities, especially

when critical and unanticipated situations arise. Interestingly, five respondents

also pointed out that sometimes the opposite of an extensive customer

involvement is expected. Customers who do not wish to be involved in the

process but want the contractor to implement and deliver the requested system

autonomously rather expect a minimum of involvement.

12

Responsiveness

of the contractor

Contractor’s readiness to reply to questions from the customer as well as the

contractor’s willingness to accept change requests.

12

Transparency Implies how well the customer feels informed about the development process.

The customer wants to see intermediate results and to be informed about the

project progress, reaching milestones, and arising problems or critical events.

11

Reliability Contractor’s adherence to agreed plans. 10

Empathy Contractor’s ability to see customer’s requirements from the perspective of the

latter.

9




Expertise Contractor’s technical and functional competence. Customers expect contractors

to possess and convey expertise in order to be able to understand their

requirements and enable a climate of trust.

9

Communication Exchanging information about recent and upcoming activities, deviations from

project plans, as well as discussion of problems and conjoint decision of

solutions. Customers also expect that their expectations are discussed.

8

Consulting and

problem-solving

skills

Contractors should express their critical opinion regarding the requirements and,

ideally, suggest several alternatives for the customer to choose rather than just

accepting and implementing customer’s wishes, which possibly turn out to be

inappropriate in the end.

7

Process

efficiency

Reduction of the effort to the required minimum, leading to a quick execution of

the project. Refers, for instance, to efficiency of communication (e.g., talking

directly to developers, leaving out intermediate project managers) and of

meetings (e.g., a thorough preparation of meetings by communicating all relevant

information in advance).

4

Establishing a

personal

relationship with

the contractor

Customers expect to get to know the contractor during the development and,

possibly, show sympathy to each other. Accordingly, they expect a designated

project team, which ideally does not change during the project. The relevance of

the personal relationship expectation increases with a growing project size.

4

Contractor’s

professional

appearance

Image, eloquence, smart and well-groomed appearance of the contractor’s project

team.

2

Personal

benefits

Customer representative’s own progress in business and personal regard,

including making a good impression on internal colleagues.

1

Table 5. Identified Customer Expectations towards the ISD Process

Expectations Management

All respondents state that customer expectations can be managed, four even consider it to be an indispensable task at

early project stages. The goal is to obtain realistic expectations. In doing so, not only customer wishes but also

objectives of the contractor organization must be considered. All experts agree that expectation management can

increase customer satisfaction with the development process by aligning expectations and actual process

perceptions. Accordingly, considering the described influence of process satisfaction on project success, agreement

exists on the positive impact of expectation management on project success.

Several factors influencing customer expectations emerged in our interviews. All respondents mention experience

of the customer to be an important factor in this regard. Without or with little experience from former

collaborations, unrealistic expectations become more likely. As Robert points out, “The customer does not know

what is possible and what is not”. A similar influencing factor stressed by our experts is technical know-how of the

customer (8 respondents). This does not mean that customers must be able to implement the software; however,

they should understand what software is, how it is developed, and what technical terms mean. Lack of technical

know-how often leads to too high expectations: “Working with people from the IT department is less problematic,

they are relatively realistic. If you work with someone from marketing, they only want to bring a product to the

market fast. Their expectations are mostly too high and need to be brought down to earth first” (Mark). Further

factors, which were mentioned by less than 6 respondents, are promises made by sales department

(2 respondents), requirements due to customer’s internal processes (2 respondents), trust between customer

and contractor (1 respondent), degree of customer’s sympathy (1 respondent), and size of customer

organization (1 respondent).

Our interviews yielded various approaches to manage customer expectations. First of all, a detailed and early

planning of the development process with the customer was mentioned (9 respondents). This includes defining

milestones, work packages, responsibilities, communication channels, escalation ladders, and risks in cooperation

with the customer. Furthermore, contractor and customer should discuss the degree of customer involvement and the




level of contractor’s responsiveness which are desired and to be expected during the development process. As

Thomas describes, “We discuss the development process with the customers in advance. […] We explain how we

work, how the interaction takes place, that we want to involve them, and that we are willing to react to their wishes

immediately. Thus, we influence their expectations”. Customers familiar with the plans from the start will align their

expectations accordingly. Even if the process deviates from the plans, customers are more likely to be satisfied since

they co-determined and approved those plans in the beginning.

Next, transparency is not only a customer expectation as described above, but also considered to be an effective

approach to manage expectations (8 respondents). Contractor’s project managers should make the development

process transparent to customers by providing information about current progress, mistakes and plan deviations, as

well as effects of customer’s requirements, expectations, and change requests. Disclosure of internal workflows is

said to increase customer satisfaction by building understanding and trust. Emily explains, “Transparency enables

understanding, which leads to trust. The more transparent I am, the better can the customer understand why some

things take longer, do not work, or cost more money”. A specific example of creating transparency was granting

customers access to the quality assurance system of the contractor, including all work tasks, their progress, and bugs,

thus giving the customer a constant overview of the current state of the project.

Another customer expectation described earlier that is also seen as an approach to manage expectations is customer

involvement (7 respondents). Beginning as early as possible, involved parties should get to know each other in

regular meetings and align their expectations. System specifications should be discussed with the customer step-by-

step before implementation. Especially agile processes provide the opportunity for regular tests, preliminary results,

and feedback. David describes how customer involvement was realized by granting access to the quality assurance

system, leading to success in a former project: “The customer was positively surprised about the involvement via the

QA system as he was able to communicate to the developers directly and discuss details without an intermediate

project manager”. However, Emily raises the concern that such direct communication is prone to

misunderstandings: “The customer is hoping to move forward more quickly by discussing something with the

developer directly. However, I have come to experience that the customer does not understand the developer. They

talk at cross purposes. Their thinking is different and needs to be translated. Thus, customers wanting to talk to

developers can backfire”. Furthermore, the ideal degree of customer involvement depends on the customer and

should thus be chosen with care in specific situations. As Michael points out, “One should bear in mind, however,

that the customer is busy, too. It can prove negative if you want to communicate too much”.

While contributing to other approaches, communication itself is said to be an approach to manage expectations

(7 respondents): “If you don’t talk to the people, everything runs aground” (Mark). By communicating early in the

project, involved parties learn what they can expect from one another. Several respondents advocate forcing regular

communication in form of conference calls, e-mails, and personal meetings. Response time should not exceed one

business day. One means to enable effective communication are prototypes, which allow the customer to provide

feedback early in the development process, facilitating the communication of expectations.

Next approach mentioned by our experts is referring to experience and alternatives (6 respondents). If customers

formulate problematic requirements or are sceptical about certain ideas of the contractor, project managers can

adjust customer expectations by referring to former projects in which certain alternatives proved inadequate:

“Usually, customers first tell us what they expect us to do, verbally or in a specification document. We comment on

it in a written form […] and communicate clearly if we see risks, even before accepting the project. […] Of course,

many why-questions arise on the part of the customer. Then, we explain with empathy that we know this business

area, we had this issue many times, and every time it was a critical one. Let’s not fall into that trap again” (Patrick).

However, Kathy points out that it is also important to listen to customers in this regard: “You can ask the customer

about his preferred course of action, and sometimes he has better ideas”.

The following approaches to manage expectations were mentioned by less than 6 respondents. Building trust

(4 respondents) is enabled by customer’s impression that the contractor is interested in conducting the project

successfully for the customer and has the required competence. Empathy (3 respondents) of the contractor leads to

better understanding and implementation of customer’s needs, and can be increased, for instance, by visiting the

customer site and getting familiar with its customs and circumstances. Finally, realistic promises made by sales

department (2 respondents) address the management of expectation before project initialization, when the first

contact with the customer takes place. Since it is difficult to lower expectations once they have been raised, sales




employees should steer customer expectations in a realistic direction from the start by discussing their feasibility and

potentially required expenses.

Figure 2. Results Overview

SUMMARY AND DISCUSSION

Our study yields three major findings. First, our interviews reveal the importance of customer satisfaction with the

development process for overall ISD project success. Second, our findings corroborate the strong impact that

customer expectations towards the ISD process have on customer satisfaction. While realistic expectations can

typically be fulfilled, thus leading to satisfied customers, our interviewees reported that they are often confronted

with unrealistically high expectations. As a result of the disconfirmation of these expectations, customers are usually

dissatisfied. Finally, we show how project managers can successfully manage customer expectations towards the

ISD process. By managing expectations, project managers can raise the likeliness of customer satisfaction and

ultimately ISD project success. Our interviewees describe multiple approaches that can be used for managing

various expectations of customer managers concerning ISD process. In the following, we discuss the implications of

these findings for researchers and practitioners, while noting the limitations of our study.

Implications for Researchers

Based on the theoretical underpinnings of ECT and Service Quality, we continue previous research on stakeholder

satisfaction in ISD projects. Stakeholder satisfaction is deemed highly relevant for project success (Nelson, 2005),

considered the sum of satisfaction with outcome and process (Ferreira and Cohen, 2008), and based on stakeholder

expectations (Baccarini, 1999; Bourque and Fairley, 2014). We shift the focus from user expectations, which are

typically concerned with the product (i.e., the developed IS), to the expectations of customer managers concerning

the development process. Our interviewees see satisfaction with the development process to predominantly have an

indirect influence on project success (mediated by satisfaction towards the product). However, they confirm the

general importance of managing expectations towards the development process on behalf of customer managers,

which has been widely neglected so far.

Previous research foci on expectations and satisfaction of users led us to explore expectations and satisfaction of

managers. For most of our interviewees, the explicit dispute about the management of manager expectations

towards the process was a novel one. This was reflected in the long time it took respondents to think of explicit

situations concerning the management of process expectations. Often, they were tempted to think of expectations

related to users and the product, and needed to be continuously reminded of our study’s focus. Accordingly, future

research might pay increased attention to the phenomenon of neglecting the development process when thinking of

IS Project Success

Customer Satisfaction withthe Development Process

Customer Satisfaction withthe Product

Customer Expectationstowards Development Process

(Dis)Confirmation

Approaches to manage Customer Expectations

Customer involvement, responsiveness of the

contractor, etc.

Planning of thedevelopment process,

transparency, customerinvolvement, etc.




expectations and their management in ISD projects. One reason might be the intangible nature of the development

process compared to the developed product. Due to their explorative nature, our results are a first step only and need

to be complemented with further inquiries. Directly comparing expectations and approaches for managing

expectations might reveal causal relations. For instance, interesting insights might result from analyzing the

approaches with regard to their applicability to specific expectations.

While most approaches to manage expectations concerning the development process match approaches identified in

other contexts (except for transparency, each approach identified in our study has been addressed in at least one

previous publication), transparency can be seen as particularly important in our context. Transparency has been

defined as the extent to which “team members incl. project manager are informed about project plan, status and all

events important to them” (Pankratz and Loebbecke, 2011, p. 6). When considering the management of user

expectations (Petter, 2008), one of three main strategies is user involvement and includes “keeping users involved

and updated throughout the project” (p. 704). While pointing towards the same direction, keeping users updated

cannot be equated with the transparency of all relevant information. The relevance of transparency in our context

can be explained by the intangible character of the development process, which – in contrast to the product –

requires active communication on behalf of the contractor to manage expectations.

Implications for Practitioners

Our results show the criticality of managing ISD process expectations. Considering our interviews, project managers

tend to be primarily concerned with the management of user expectations concerning the product. This is surprising

since our interviewees had no difficulties in thinking of numerous expectations customer managers may have

concerning the development process. Consequently, we recommend project managers to explicitly think about

whether and how they have managed expectations towards the development and to what extent their approaches

have affected the success of ISD projects. They can use our study as a starting point to develop strategies for coping

with this important project management task.

Three of the most frequently mentioned approaches (i.e., transparency, customer involvement, and communication)

are closely related (i.e., they all concern direct customer contact) yet different from each other. For instance, while

ensuring transparency throughout an ISD project requires the contractor to communicate with the customer,

communication with the customer does not necessarily lead to transparency. Furthermore, involving the customer in

the development process, for example by letting the customer make suggestions, does not make the development

process transparent. When developing strategies to manage process expectations, it is thus important to consider

dependencies between the identified approaches for expectation management.

The identified approaches should be carefully considered before application in specific projects. While high

customer involvement is in general perceived as a success factor in ISD projects (McKeen et al., 1994; Petter, 2008),

customers may be reluctant to closely collaborate with the contractor, for instance due to daily work obligations.

Moreover, approaches might not be applicable in some cases at all. Promises by sales departments might be made

prior to initializing a project, that is, when it is rather difficult to judge whether promises are realistic. Once

promises are made, the customer might lose trust in the contractor when promises are adapted in the course of the

project. The approaches’ applicability and thus the likeliness to increase customer satisfaction and project success

might therefore be contingent on projects’ context.

Limitations

As with any empirical study, ours is not free of limitations. First, the generalizability is limited by the sample size of

twelve respondents. While we randomly contacted companies to avoid a selection bias and the results in general

converge to common themes, we cannot guarantee that interviewing further project managers would not lead to

further insights. Second, our experts work for small and medium-sized enterprises. Managing expectations in larger

companies may be subject to further factors, which influence expectations and approaches to manage these. Third,

the interviews have been conducted via phone. Consequently, we were unable to observe respondents’ non-verbal

communication. However, using telephone interviews we were able to convince more project managers to

participate in our study compared to conducting face-to-face interviews, which are typically more effort-intensive.

Finally, expectations presented in this study have been mentioned by project managers on behalf of contractors. For

more detailed insights, interviews with customer managers are required. In our context, the choice of respondents

was guided by our focus to identify approaches to manage expectations.




CONCLUSION

We show the relevance of customer satisfaction concerning the ISD process for the success of ISD projects. The

identified approaches of managing expectations towards the development process can help project managers to

increase the likeliness of customer satisfaction and thus project success. By revealing customer expectations towards

the development process, we illustrate the diversity of aspects project managers need to address in order to pave the

way for successful projects. Whereas we contribute to a deeper understanding of the role of managing expectations

in ISD projects, dependencies between expectations and approaches for managing expectations are to be addressed

by future research.

APPENDIX A

Questions related to interviewee’s experiences, current position, and tasks.

Part

One

What kind of training / education have you received?

What is your current position and range of duty?

What is your working experience in ISD?

Questions related to customer satisfaction with the development process in IS projects.

Part

Two

How relevant is customer satisfaction concerning the development process for the overall success of an IS

project?

How relevant is customer satisfaction concerning the development process compared to customer

satisfaction concerning the product?

Are there any dependencies between customer satisfaction concerning the development process and

customer satisfaction concerning the product?

To what extent does your company measure customer satisfaction (in general, concerning process or

product)?

Questions related to customer expectations towards the ISD process and their management.

Part

Three

What impact do customer expectations have on customer satisfaction?

What expectations do customers have concerning the ISD process?

To what extent can project managers specifically influence customer satisfaction?

Can you think of an IS project in which you positively influenced customer expectations concerning the

development process?

How would you characterize the project and its context?

What expectations did the customer have?

How did you respond to these expectations?

What impact did your response have?

Can you think of an IS project in which you negatively influenced customer expectations concerning the

development process?

How would you characterize the project and its context?

What expectations did the customer have?

How did you respond to these expectations?

What impact did your response have?

Can you think of other tactics that might have had a positive impact?

Do you have any further recommendations for dealing with customer expectations?

To which contexts do these recommendations apply?

Table 6. Extract from Interview Guide

REFERENCES

Anderson, E. W., and Sullivan, M. W. (1993) The Antecedents and Consequences of Customer Satisfaction for

Firms, Marketing Science, 12, 2, 125–143.

Baccarini, D. (1999) The Logical Framework Method for Defining Project Success, Project Management Journal,

30, 4, 25–32.

Baccarini, D., Salm, G., and Love, P. (2004) Management of Risks in Information Technology Projects, Industrial

Management & Data Systems, 104, 4, 286–295.




Baker, B. N., Murphy, D. C., and Fisher, D. (1988) Factors Affecting Project Success, in D. I. Cleland, and W. R.

King (eds.) Project Management Handbook, New York, John Wiley & Sons, 902–919.

Baker, B. N., Murphy, D. C., and Fisher, D. (2008) Factors Affecting Project Success, in D. I. Cleland, and W. R.

King (eds.) Project Management Handbook, John Wiley & Sons, Inc, 902–919.

Bakker, K. de, Boonstra, A., and Wortmann, H. (2012) Risk Managements’ Communicative Effects Influencing IT

Project Success, International Journal of Project Management, 30, 4, 444–457.

Basten, D., Joosten, D., and Mellis, W. (2011) Developing a Situational Model of Information System Project

Success, in AIS Special Interest Group for Information Technology Project Management (ed.) Proceedings

of the 6th pre-ICIS International Research Workshop on IT Project Management, 5–17.

Basten, D., Joosten, D., and Mellis, W. (2012) Managers’ Perceptions of Information System Project Success,

Journal of Computer Information Systems, 52, 2, 12–21.

Bhattacherjee, A. (2001) Understanding Information Systems Continuance: An Expectation-Confirmation Model,

MIS Quarterly, 25, 3, 351–370.

Boehm, B. (2000) The Art of Expectations Management, Computer, 33, 1, 122–124.

Boehm, B. W., and Ross, R. (1989) Theory-w Software Project Management Principles and Examples, IEEE

Transactions on Software Engineering, 15, 7, 902–916.

Bourque, P., and Fairley, R. E. (2014) Swebok. Guide to the Software Engineering Body of Knowledge, IEEE

Computer Society, Los Alamitos.

Boyd, A. (2001) The Five Maxims of Project Satisfaction, Aslib Proceedings, 53, 10, 423–430.

Campbell, A., Converse, P. E., and Rodgers, W. L. (1976) The Quality of American Life: Perceptions, Evaluations,

and Satisfactions, Russell Sage Foundation.

Conrath, D. W., and Mignen, O. P. (1990) What is Being Done to Measure User Satisfaction with EDP/MIS,

Information & Management, 19, 1, 7–19.

DeLone, W. H., and McLean, E. R. (1992) Information System Success: The Quest for the Dependent Variable,

Information Systems Research, 3, 1, 60–95.

DeLone, W. H., and McLean, E. R. (2003) The DeLone and McLean Model of Information Systems Success: A

Ten-Year Update, Journal of Management Information Systems, 19, 4, 9–30.

El Emam, K., and Koru, A. G. (2008) A Replicated Survey of IT Software Project Failures, IEEE Software, 25, 5,

84–90.

Ferreira, C., and Cohen, J. (2008) Agile Systems Development and Stakeholder Satisfaction: A South African

Empirical Study, in Proceedings of the 2008 Annual Research Conference of the South African Institute of

Computer Scientists and Information Technologists on IT Research in Developing Countries: Riding the

Wave of Technology, 48–55.

Freeman, M., and Beale, P. (1992) Measuring Project Success, Project Management Journal, 23, 1, 8–17.

Ginzberg, M. J. (1981) Early Diagnosis of MIS Implementation Failure: Promising Results and Unanswered

Questions, Management Science, 27, 4, 459–478.

Goodhue, D. L. (1995) Understanding User Evaluations of Information Systems, Management Science, 41, 12,

1827–1844.

Jørgensen, M., and Sjøberg, D. I. K. (2004) The Impact of Customer Expectation on Software Development Effort

Estimates, International Journal of Project Management, 22, 4, 317–325.

Kopalle, P. K., and Lehmann, D. R. (2001) Strategic Management of Expectations: The Role of Disconfirmation

Sensitivity and Perfectionism, Journal of Marketing Research, 38, 3, 386–394.

Liu, J. Y.-C., Chen, H.-G., Chen, C. C., and Sheu, T. S. (2011) Relationships among Interpersonal Conflict,

Requirements Uncertainty, and Software Project Performance, International Journal of Project

Management, 29, 5, 547–556.

Locke, E. A. (1969) What is Job Satisfaction?, Organizational Behavior and Human Performance, 4, 4, 309–336.

Locker, D., and Dunt, D. (1978) Theoretical and Methodological Issues in Sociological Studies of Consumer

Satisfaction with Medical Care, Social Science & Medicine. Part A: Medical Psychology & Medical

Sociology, 12, 283–292.

Lyytinen, K. (1988) Expectation Failure Concept and Systems Analysts’ View of Information System Failures:

Results of an Exploratory Study, Information & Management, 14, 1, 45–56.

Markus, M. L., and Keil, M. (1994) If We Build it, They will Come: Designing Information Systems that People

Want to Use, Sloan Management Review, 35, 11.

McKeen, J. D., Guimaraes, T., and Wetherbe, J. C. (1994) The Relationship Between User Participation and User

Satisfaction: An Investigation of Four Contingency Factors, MIS Quarterly, 18, 4, 427–451.

Miller, H. (2000) Managing Customer Expectations, Information Systems Management, 17, 2, 1–4.




Moynihan, T. (2002) Coping with Client-based ‘People-problems’: The Theories-of-action of Experienced

IS/software Project Managers, Information & Management, 39, 5, 377–390.

Myers, M. D. (1995) Dialectical Hermeneutics: A Theoretical Framework for the Implementation of Information

Systems, Information Systems Journal, 5, 1, 51–70.

Myers, M. D., and Newman, M. (2007) The Qualitative Interview in IS Research: Examining the Craft, Information

Organization, 17, 1, 2–26.

Nelson, R. (2005) Project Retrospectives: Evaluating Project Success, Failure, and Everything in between, MIS

Quarterly Executive, 4, 3, 361–372.

Nevo, D., and Wade, M. R. (2007) How to Avoid Disappointment by Design, Communications of the ACM, 50, 4,

43–48.

Ojasalo, J. (2001) Managing Customer Expectations in Professional Services, Managing Service Quality, 11, 3,

200–212.

Oliver, R. L. (1980) A Cognitive Model of the Antecedents and Consequences of Satisfaction Decisions, Journal of

Marketing Research, 17, 4, 460–469.

Olson, J. C., and Dover, P. A. (1979) Disconfirmation of Consumer Expectations Through Product Trial, Journal of

Applied Psychology, 64, 2, 179–189.

Pankratz, O., and Loebbecke, C. (2011) Project Managers’ Perception of IS Project Success Factors - A Repertory

Grid Investigation, in V. Tuunainen, J. Nandhakumar, M. Rossi, and W. Soliman (eds.) Proceedings of the

19th European Conference on Information Systems.

Parasuraman, A., Zeithaml, V. A., and Berry, L. L. (1985) A Conceptual Model of Service Quality and Its

Implications for Future Research, Journal of Marketing, 49, 4, 41–50.

Parasuraman, A., Zeithaml, V. A., and Berry, L. L. (1988) Servqual, Journal of retailing, 64, 1, 12–40.

Peters, T. J. (1988) Thriving on Chaos: Handbook for a Management Revolution, Alfred A. Knopf, New York.

Petter, S. (2008) Managing User Expectations on Software Projects: Lessons from the Trenches, International

Journal of Project Management, 26, 7, 700–712.

Pitt, L. F., and Jeantrout, B. (1994) Management of Customer Expectations in Service Firms: A Study and a

Checklist, The Service Industries Journal, 14, 2, 170–189.

Pitt, L. F., Watson, R. T., and Kavan, C. B. (1995) Service Quality: A Measure of Information Systems

Effectiveness, MIS Quarterly, 19, 2, 173–187.

Potter, R. E. (2003) How CIOs Manage Their Superior's Expectations, Communications of the ACM, 46, 8, 74–79.

Rubin, H. J., and Rubin, I. (2005) Qualitative Interviewing. The Art of Hearing Data, SAGE Publications, Thousand

Oaks.

Saarinen, T. (1996) An Expanded Instrument for Evaluating Information System Success, Information

& Management, 31, 2, 103–118.

Saarinen, T., and Sääksjärvi, M. (1992) Process and Product Success in Information Systems Development, Journal

of Strategic Information Systems, 1, 5, 266–275.

Santos, J., and Boote, J. (2003) A Theoretical Exploration and Model of Consumer Expectations, Post-purchase

Affective States and Affective Behaviour, Journal of Consumer Behaviour, 3, 2, 142–156.

Schmidt, R., Lyytinen, K., Keil, M., and Cule, P. (2001) Identifying Software Project Risks: An International Delphi

Study, Journal of Management Information Systems, 17, 4, 5–36.

Sheth, J. N., and Mittal, B. (1996) A Framework for Managing Customer Expectations, Journal of Market-Focused

Management, 1, 2, 137–158.

Shrauger, J. S. (1975) Responses to Evaluation as a Function of Initial Self-perceptions, Psychological Bulletin, 82,

4, 581.

Wit, A. de (1988) Measurement of project success, International Journal of Project Management, 6, 3, 164–170.

Zeithaml, V. A., Berry, L. L., and Parasuraman, A. (1993) The Nature and Determinants of Customer Expectations

of Service, Journal of the Academy of Marketing Science, 21, 1, 1–12.

Sweetman et al. IS Project Portfolios: A Complex Adaptive Systems Perspective



A Complex Adaptive Systems Perspective on Self-Organization in IS Project Portfolios

Roger Sweetman

Lero,

National University of Ireland, Galway

[email protected]

Orla O’Dwyer

Lero


[email protected]

Kieran Conboy

Lero,


[email protected]

ABSTRACT

Portfolio management practices and theory continue to remain focused on a centralized “command and control”

perspective. Even though many organizations promote and encourage self-organization, particularly within their

software development teams, little is known about how or if IS project portfolios self-organize. Previous studies

have explored self-organization at organizational, team, or project level, but do not explore self-organization at

portfolio level. Self-organization facilitates the acceptance of innovative ideas and enables autonomous teams to

respond to changes in requirements or in the environment without management intervention. This research-in-

progress paper aims to firstly contribute to research by using the theory of complex adaptive systems to explain how

one aspect of control, namely self-organization, can occur in portfolios of IS projects. Secondly, this study will,

through the use of exploratory case studies, contribute to practice by determining the implications and challenges for

managers of self-organizing IS portfolios.

Keywords

Project portfolio management, self-organization, complex adaptive systems, control.

INTRODUCTION

Information systems (IS) development projects have suffered from high rates of failure for over 50 years. Projects

are characterized by major cost overruns (Jorgensen and Molokken-Ostvold 2006, Conboy 2010), quality problems

(Parnas and Lawford 2003) and late delivery (Payne 1995, De Reyck et al. 2005, Jiang and Klein 1999). This

“software crisis” (Naur and Randell 1969) motivated McFarlan (1981) to propose a portfolio approach to

information systems. Project portfolio management (PPM) enables IS managers to focus on the management of

multiple projects by ensuring projects are systematically evaluated, selected and implemented in order to deliver

organizational strategy (Jeffery and Leliveld 2004). Effective management of an Information Systems (IS) portfolio

is critical to managing risk, achieving business value from IS and also in aligning IS projects to organizational

strategy (Kauffman and Sougstad 2008, Reich and Benbasat 2000, Hatzakis et al. 2007, De Reyck et al. 2005).

Where IS project portfolio management (PPM) has been practiced effectively, massive savings in cost and time have

been realized (LeFave et al. 2008). PPM also has the potential to both reduce the incidence of individual software

project failure (McFarlan 1981) and allow successful projects compensate for unsuccessful ones (Costa et al. 2007).

Unfortunately the benefits from adopting a portfolio perspective in IS has failed to match those achieved in other

fields, e.g. financial portfolio management (Fabozzi et al. 2002), research and development (Stummer and

Heidenberger 2003, Mikkola 2001) and new product portfolio development (Cooper et al. 2001) where portfolio

theory has been applied in a more mature and effective manner. IS PPM research has over-emphasized project

selection (Meskendahl 2010, De Reyck et al. 2005) and until recently there has been little on how IS portfolios are

governed post-project selection (Frey and Buxmann 2012). The limited number of contributions that explicitly




address the issue of IS portfolio management focus on the need for a centralized perspective on project management

(Frey and Buxmann 2011) ignoring the complex and adaptive nature of contemporary IS portfolios.

Complex adaptive systems theory (CAS) has proven effective in studying complex phenomena in a number of

disciplines and is revealing new insights from which project and portfolio management can learn (Cooke-Davies et

al. 2008). IS project portfolios can be considered complex adaptive systems. They are complex because portfolios

are made up of a large number of interdependent projects and teams and have goals that are ill defined, ambiguous,

or subject to change (Bardhan et al. 2004). IS portfolios are adaptive because they must change in order to maintain

alignment with a rapidly changing business and technological environment (Merali et al. 2012, Blichfeldt and

Eskerod 2008) where organizational strategy, customer demand and user needs change rapidly (Angelou and

Economides 2008, Bardhan et al. 2004, Martinsuo 2013). CAS has been applied to general project portfolio

management (Perry 2012) and to problems in IS, but has yet to be applied to IS PPM despite practitioner interest

(Gartner 2014). An important feature of CAS is the lack of a single point of control meaning that behaviors can be

unpredictable making direct control difficult with a subsequent reliance on informal controls such as self-

organization (Rouse 2000).

Self-organization has been described as the anchor point phenomenon of complex adaptive systems theory (Tan et

al. 2005, Chiles et al. 2004). It can be defined as the spontaneous coming together of groups to perform some

purpose. The group decides what, how and when to perform and their activity is not explicitly directed from outside

the group (Mitleton-Kelly 2003b). Self-organized teams are able to react to changes in requirements or in the

environment and respond without management intervention (McGrath 2001, Uhl-Bien et al. 2007). Little is known

about how self-organization occurs in IS portfolios even though contemporary IS projects are arranged around the

principles of flexibility and self-organization (Highsmith 2013). Arising from this, this study will seek to answer the

following research questions:

1. How does CAS explain the occurrence of self-organization in IS project portfolios?

2. What are the implications and challenges for practitioners of self-organizing IS portfolios?

The next section will present a brief summary of the IS PPM control literature followed by the theory of CAS.

Subsequently, we present the CAS framework which will be used as the basis for this study. Our proposed research

method is then described. Finally, we summarize the proposed contributions, outline the next steps and identify

avenues for future research.

BACKGROUND AND LITERATURE REVIEW

Project Portfolio Management Control

PPM is a key function within an organization (Blichfeldt and Eskerod 2008) that requires managers to use control to

influence people to make decisions consistent with the three main portfolio goals: maximizing portfolio value,

achieving the right mix of projects, and aligning the portfolio with business strategy (Cooper and Edgett 1997).

While many organizations seek means to control portfolios of projects, they admit that these means are not easy to

find, especially portfolios with complex projects or interdependencies between projects (Rungi 2010, Bardhan et al.

2004). IS research mostly addresses project control with little known about the control of project portfolios

(Korhonen et al. 2014, Hansen and Kræmmergaard 2013). The most notable exceptions are recent studies by Hansen

& Kræmmergaard (2013) who identify PPM controls used in one organization and Korhonen et al. (2014) and

Martinsuo et al. (2014) who examine how uncertainties in project portfolios can be controlled.

Control is exerted through formal (behaviour, outcome) and informal (clan, self) control modes. In a project

portfolio control is more than a scaled up version of single project control. Rather than focusing on detail, portfolio

managers should enable individual managers to act creatively and allow projects and teams to self-organize, while

maintaining a necessary level of accountability (Aritua et al. 2009). Self-organization, a form of self-control, is

where the system evolves, or emerges over time into a coherent form or pattern without any explicit management

(Benbya and McKelvey 2006b, Cilliers 2000). Self-organization is highly effective for portfolio management as it

allows control to be distributed through the system (Kauffman 1995) with self-organized structures both robust

(Cilliers 2000, Levin 2003) and capable of responding to dramatic changes in the environment (Uhl-Bien et al.

2007).




Complex Adaptive Systems

Complexity theories have arisen from attempts in the natural sciences to model natural phenomenon (Gleick 1997).

They are concerned with the emergence of higher level order in dynamic non-linear systems where the laws of cause

and effect do not appear to apply (Beeson and Davis 2000). Complexity theories differ from mechanistic theories in

that rather than assuming a centrally controlled governing structure, order emerges from the interaction between the

different entities that populate the system. The three main branches of complexity theory are chaos (Bechtold 1997),

dissipative structures theory (Prigogine et al. 1985) and complex adaptive systems (CAS) (Goodwin 1994). CAS is

the most appropriate of these theories to act as lens to study IS project portfolios. This is because it is the only one

that does not take a macro approach to modelling systems. Instead, it models the phenomena at the micro level using

the agents that make up the system. It does not attempt to formulate rules for the whole system rather; it formulates

rules of interaction for the individual agents in the system. The IS project portfolio can be studied by focusing on

individuals, teams and projects (all of which can be represented as agents) and the interactions between them.

While CAS is defined in a number of different ways, most definitions of CAS involve agents interacting in self-

organizing ways with each other and the environment (e.g. Nan 2011, Holland 1992). For example, Benbya and

McKelvey (2006a) define a complex adaptive system as a system poised between order and chaos, that not only self

organizes, but directs its activity towards its own optimization. Vessey and Ward (2013) define a complex adaptive

system as “any system featuring a large number of interacting components that exhibits self-organization and

emergence under a certain level of tension, and whose aggregate activity is non-linear” e.g. biological habitats, cities

and the internet. However, the definition used for this study is the most all-encompassing one where Holland (1992,

1995) defines a CAS as a system composed of interacting agents, which undergo constant change, both

autonomously and in interaction with their environment to produce complex and adaptive behaviors and patterns.

These patterns are aggregate behaviors and structures that are not predictable from an analysis of the component

parts of the system. Rather, self-organization emerges as agents interact through sometimes simple rules which can

change and adapt as experience accumulates and environmental conditions change (Anderson 1999, Holland 1995).

CAS is no longer considered a new theory in organization studies (Anderson 1999). However, its application to IS is

more recent. For example, CAS has been applied to IT enabled organizational learning (Kane and Alavi 2007), IT

supported team processes (Curşeu 2006), improving IS alignment (Benbya and McKelvey 2006b), the role of IT in

the development of bureaucracy (Boisot 2006), and the impact of IT on organizational culture (Canessa and Riolo

2006). Further examples of CAS in IS research include processes for technology use (Nan 2011), IS project

management (Xia and Lee 2004) and agile software development (Vidgen and Wang 2006, Jain and Meso 2004).

This increasing prevalence of CAS in information systems research led to Merali (2006) describing it as the

“emergent domain” in management research. Yet, there has been a greater emphasis on the theoretical aspects of

CAS rather than empirical studies. Vidgen and Wang (2009) attribute this to the difficulty in making the abstract

principles of CAS suitably concrete for case study research. Therefore, we draw on CAS theory to propose a CAS

framework that will explain how self-organization can occur in portfolios of IS projects.

CONCEPTUAL FRAMEWORK

No single framework exists in the literature that comprehensively explains CAS. Instead, there are attempts to

mathematically model complex systems (e.g. Kauffman 1993, Boisot and Child 1999); develop frameworks to

create adaptive or evolutionary organizations (e.g. Volberda and Lewin 2003, Vessey and Ward 2013, Vidgen and

Wang 2006); develop frameworks that provide insights into general management issues (e.g. Snowden and Boone

2007, Cilliers 2000); and finally CAS has been used to explain emergent phenomena in complex organizations (e.g.

Plowman et al. 2007). This study treats self-organization as an emergent phenomenon. Therefore, we draw on CAS

frameworks from Lewin (1999), Alaa and Fitzgerald (2013) and (Nan 2011) to create a single framework that

encompasses the core components of a complex adaptive system (Figure 1), namely, agents, interactions,

environment, and feedback loops, all of which combine to result in the emergence of self-organization.

An agent is a general term to describe the individual actors or basic entities of action of a complex adaptive system

(Benbya and McKelvey 2006a, Nan 2011). For a system to be complex it must consist of a large number of different

agents which need not be complex in their own right (Cilliers 2000). For example, in an IS portfolio agents may be

individuals, teams or projects with diversity and individuality of the agents making a complex adaptive system

stronger (Perry 2012, Holland 1995, Gell-Mann 1994). Agents are governed by behavioral rules or schema (Nan

2011, Gell-Mann 1994) such as norms, values and beliefs that allow them to interpret the environment and the




actions of other agents (Argyris and Schön 1997). They are intelligent and capable of learning and adapting (Fuller

and Moran 2001) through eliciting feedback from other agents and the environment (Holland 1995) with no single

agent controlling the behavior of the system as a whole (Benbya and McKelvey 2006a).

Interactions describe the mutually adaptive behavior of agents (Nan 2011). It is the nature of the interactions, not the

agents themselves that determines the behavior of the system. Interactions occur by agents exchanging information,

energy or resources (Cilliers 2000). For example, in an IS portfolio project teams are constantly interacting with

each other by exchanging resources and information about interdependencies (Angelou and Economides 2008,

Kundisch and Meier 2011). Further, interactions between agents may affect the interacting agents (Mitleton-Kelly

2003b) and these interactions often cause them to adapt their rules or co-evolve with each other (Stacey 2002,

Mitleton-Kelly 2003a). Even interactions between a few elements can have nonlinear effects that are propagated

throughout the system (Holland 1995, Cilliers 2000). While agents may act in a selfish manner, the interactions

between them can result in aggregates that improve the system as a whole (Levin 1998). Holland (1992) writes that

a crucial characteristic of the interactions between agents is their capacity to anticipate the response of other agents.

This anticipation can affect the behavior of the system as a whole even if the anticipated response does not

materialize. Finally, interactions can only occur between agents in a system if they are connected. Without

connectivity the aggregate behavior of agents would be random (Dooley and Van de Ven 1999).

Figure 1. The conceptual framework for this study

The environment is the constantly changing setting in which the agents operate and interact. According to CAS

theory, a system must be open to its environment. For example, a project portfolio is an open system which

exchanges technical information and resources such as skills and people with its environment. It is necessary for a

system to be open before self-organization can occur as the emergence of aggregate structures increase the order of a

system. This increase in order is not possible unless energy and information are imported into the system and

disorder is exported back into the environment (Capra 1996). IS portfolios exist in a rapidly changing business and

technological environment (Merali et al. 2012, Blichfeldt and Eskerod 2008) with customer demands and user needs

changing frequently (Angelou and Economides 2008, Bardhan et al. 2004). Because the environment is constantly

changing, the maintenance of alignment between the IS portfolio and its environment is a key challenge in IS PPM

(De Reyck et al. 2005). According to Benbya and McKelvey (2006a) changes in the environment create an adaptive




tension stimulating the creation of order within the system meaning any “fitness” is only temporary. Thus, complex

adaptive systems are considered to be sub-optimal or operating at a point “far from equilibrium” (Fuller and Moran

2001) with the emphasis on improvement rather than optimization (Holland 1992).

Complex adaptive systems are characterized by the existence of many direct and indirect feedback loops (Cilliers

2000). Feedback loops occur where components in the output stage can inform components in the input stage

(Andriani 2003). The American school of complexity science sees feedback as the driving force of complex systems

adaption (Benbya and McKelvey 2006a) where interrelated parts feed back to each other driving or damping change

(Mitleton-Kelly 2003a). Positive and negative feedback loops can co-exist simultaneously (Mitleton-Kelly 2003a).

Positive feedback reinforces or amplifies the initial change and can result in a radically altered system. Negative

feedback dampens or suppresses the initial change (Mitleton-Kelly 2003a). In a portfolio, desired behavior can be

encouraged with positive feedback and unwanted behavior reduced through negative feedback. While management

literature often emphasizes the importance of negative feedback for control, emergent behaviors can be better

managed with positive feedback that allows and supports autonomous action (Choi et al. 2001).

Emergence is a phenomenon whereby well-formulated aggregate behavior arises from localized individual behavior

(Miller and Page 2007). According to Holland (1992) complex adaptive systems exhibit aggregate behaviors that

emerge from the interactions between the agents. In emergence “the whole is greater than the sum of the parts”

(Mitleton-Kelly 2003a). In an IS project portfolio examples of emergence are higher level structures such as special

interest groups and behaviors such as culture. Further, emergence is unpredictable and is immune to reasonable

variations in the behavior of individual agents (Miller and Page 2007). From the perspective of a CAS self-

organization emerges from the interaction of agents with each other and with the environment.

The application of CAS in IS PPM is limited. This study proposes to extend existing research on CAS and IS project

portfolio management with a view to explaining how self-organization can occur in portfolios of IS projects and

providing new insights for managers on the implications and challenges for managers of self-organizing IS

portfolios.

RESEARCH METHOD

While modelling is often deemed an appropriate method for gaining insights into complex systems, Cilliers (2001)

warns that models, by necessity, reduce the complexity of a system making a model of complex system flawed by

default. Therefore, drawing on the pragmatic IS movement where artefacts are designed “to be flexible, ready for

surprise and suitable for improvisation” (Niccolini 2013), an interpretivist qualitative approach is proposed for this

study. We will use multiple case studies, which is appropriate when a research phenomenon is investigated in its

natural setting (Yin 2009). This will provide a rich insight into self-organization in IS project portfolios. The

intention is to conduct a number of case studies across a range of industry sectors using a purposeful case selection

strategy to select cases that are information–rich (Patton 1990). This will also provide the opportunity to conduct

within-case and cross-case analysis. Case study selection will be carefully considered based on the size, business

and structure of potential cases (Benbasat et al. 1987). Organizations must be of a sufficient size and have a business

model sufficiently supported by IS to ensure at least one complex IS portfolio exists. By selecting sites where

contradictory results are expected it is intended to cover different theoretical conditions (theoretical replication) (Yin

2009). Data collection will continue until saturation is reached (Corbin and Strauss 2008).

A traceable ‘audit trail’ of the research process will be followed to improve the reliability and repeatability of the

research. An interview protocol will be prepared based on the main concepts of CAS (agents, interactions,

environment, feedback and emergence). Interview questions will be primarily open-ended and designed to elicit

examples of self-organization and management challenges with self-organisation. In order to answer the first

question we will identify interactions between portfolio agents, interactions between agents and the environment,

feedback loops and how each of these leads to the emergence of self-organization in IS portfolios. To answer the

second research question we will identify the management challenges and implications of self-organization in IS

portfolios. A pilot study with one portfolio manager and one project manager is planned to test the interview

protocol. Subsequently, additional organizations and personnel will be identified to participate in primary data

collection.




CONCLUSIONS AND NEXT STEPS

This study has a number of limitations. Firstly, while is not possible to generalize the findings of qualitative

research, the selection of a broad range of case studies, in order to achieve theoretical replication, ensures the

findings are rigorous and provides a sound foundation for future research. Secondly, the use of interviews to collect

data is subject to bias by the researcher. We will attempt to avoid this by corroborating information from different

participants. Thirdly, participants may not be willing or able to answer questions. In such instances interview

questions may be rephrased to elicit a response. Further, all participants will be assured of the confidentiality of the

study. Finally, concepts such as self-organization and emergence are abstruse and difficult to identify. We will

interview multiple project and portfolio managers in each organization in an attempt to address this and also to

improve the reliability and validity of the findings.

The implications for control of the increasing complexity of IS portfolios in organizations allied to the dearth of

research examining self-organization in IS portfolios motivates this study. There is little research on IS PPM and

this is reflected in the IS field where IS PPM practices are often inadequate. Research has failed to address the gap

between the command and control philosophy of PPM and contemporary methods focused on self-organization.

From a research perspective, this study will advance our understanding of self-organization in IS portfolios by using

CAS as a lens to explain how it can be facilitated. This is a meaningful addition to the literature as little empirical

work has attempted to understand self-organization in IS project portfolios. The study will also contribute to practice

by identifying the implications of self-organization for portfolio managers who may struggle in complex

environments if restricted to using only formal controls.

The next steps for this study are as follows: Firstly, data collected during the pilot study will be analyzed to identify

examples of how self-organization emerges in IS portfolios and subsequent managerial challenges. Secondly, a

number of additional cases will be identified for the study. Thirdly, primary data will be collected and analyzed

using within case and cross-case analysis (Miles and Huberman 1996) to answer both research questions. In terms of

future research, there are a number of interesting possibilities. Firstly, the results of this study will make easier to

model IS project portfolios as complex adaptive systems using agent based modelling. Secondly, the

operationalization of CAS will enable future empirical research into areas such as ambidextrous portfolios, the

emergence of culture and the role of feedback in portfolios.

This work was supported, in part, by Science Foundation Ireland grant 10/CE/I1855 to Lero - the Irish Software

Engineering Research Centre (www.lero.ie).

REFERENCES

Alaa, G. & Fitzgerald, G. (2013). Re-conceptualizing agile information systems development using complex

adaptive systems theory. Emergence: Complexity & Organization, 15.

Anderson, P. (1999). Complexity theory and organization science. Organization Science, 10, 216-232.

Andriani, P. (2003). Evolutionary dynamics of industrial clusters.

Angelou, G. & Economides, A. (2008). A real options approach for prioritizing ICT business alternatives: a case

study from broadband technology business field. Journal of the Operational Research Society, 59, 1340-

1351.

Argyris, C. & Schön, D. A. (1997). Organizational learning: A theory of action perspective. Reis, 345-348.

Aritua, B., Smith, N. J. & Bower, D. (2009). Construction client multi-projects - A complex adaptive systems

perspective. International Journal of Project Management, 27, 72-79.

Bardhan, I., Bagchi, S. & Sougstad, R. (2004). Prioritizing a portfolio of information technology investment

projects. Journal of Management Information Systems, 21, 33-60.

Bechtold, B. L. (1997). Chaos theory as a model for strategy development. Empowerment in Organizations, 5, 193-

201.

Beeson, I. & Davis, C. (2000). Emergence and accomplishment in organizational change. Journal of Organizational

Change Management, 13, 178-189.

Benbasat, I., Goldstein, D. K. & Mead, M. (1987). The case research strategy in studies of information systems. MIS

quarterly, 369-386.

Benbya, H. & Mckelvey, B. (2006a). Toward a complexity theory of information systems development. Information

Technology & People, 19, 12-34.

http://www.lero.ie/




Benbya, H. & Mckelvey, B. (2006b). Using coevolutionary and complexity theories to improve IS alignment: a

multi-level approach. Journal of Information Technology, 21, 284-298.

Blichfeldt, B. S. & Eskerod, P. (2008). Project portfolio management – There’s more to it than what management

enacts. International Journal of Project Management, 26, 357-365.

Boisot, M. (2006). Moving to the edge of chaos: bureaucracy, IT and the challenge of complexity. Journal of

Information Technology, 21, 239-248.

Boisot, M. & Child, J. (1999). Organizations as adaptive systems in complex environments: The case of China.

Organization Science, 10, 237-252.

Canessa, E. & Riolo, R. L. (2006). An agent-based model of the impact of computer-mediated communication on

organizational culture and performance: an example of the application of complex systems analysis tools to

the study of CIS. Journal of Information Technology, 21, 272-283.

Capra, F. 1996. The web of life: A new scientific understanding of living systems, Random House LLC.

Chiles, T. H., Meyer, A. D. & Hench, T. J. (2004). Organizational emergence: The origin and transformation of

Branson, Missouri's musical theaters. Organization Science, 15, 499-519.

Choi, T. Y., Dooley, K. J. & Rungtusanatham, M. (2001). Supply networks and complex adaptive systems: control

versus emergence. Journal of operations management, 19, 351-366.

Cilliers, P. (2000). What can we learn from a theory of complexity? Emergence, 2, 23-33.

Cilliers, P. (2001). Boundaries, hierarchies and networks in complex systems. International Journal of Innovation

Management, 5, 135-147.

Conboy, K., ; Morgan, L. (2010). Future research in agile systems development: applying open innovation principles

within the agile organisation. In T. Dingsoyr, T. Dybå & N. Brede Moe (Eds.), Agile software

development: current research and future directions (pp. 223-234).

Cooke-Davies, T., Cicmil, S., Crawford, L. & Richardson, K. (2008). We're Not in Kansas Anymore, Toto:

Mapping the Strange Landscape of Complexity Theory, and Its Relationship to Project Mangement.

Engineering Management Review, IEEE, 36, 5-21.

Cooper, R. G., Edgett, S. & Kleinschmidt , E. 2001. Portfolio Management for New Products Perseus Books.

Cooper, R. G. & Edgett, S. J. (1997). Portfolio management in new product development: Lessons from the leaders-

-I. Research Technology Management, 40, 16.

Corbin, J. & Strauss, A. 2008. Basics of Qualitative Research, London, UK, Sage Publications.

Costa, H. R., Barros, M. D. O. & Travassos, G. H. (2007). Evaluating software project portfolio risks. Journal of

Systems & Software, 80, 16-31.

Curşeu, P. L. (2006). Emergent states in virtual teams: a complex adaptive systems perspective. Journal of

Information Technology, 21, 249-261.

De Reyck, B., Grushka-Cockayne, Y., Lockett, M., Calderini, S. R., Moura, M. & Sloper, A. (2005). The impact of

project portfolio management on information technology projects. International Journal of Project


Dooley, K. J. & Van De Ven, A. H. (1999). Explaining complex organizational dynamics. Organization Science, 10,

358-372.

Fabozzi, F. J., Gupta, F. & Markowitz, H. M. (2002). The legacy of modern portfolio theory. The Journal of

Investing, 11, 7-22.

Frey, T. & Buxmann, P. (2011). The Importance of Governance Structures in IT Project Portfolio Management.

Frey, T. & Buxmann, P. (2012). IT project portfolio management - a structured literature review. ECIS 2012.

Fuller, T. & Moran, P. (2001). Small enterprises as complex adaptive systems: a methodological question?

Entrepreneurship & Regional Development, 13, 47-63.

Gartner. 2014. Gartner PPM & IT Governance Summit [Online]. Available:

http://www.gartner.com/technology/summits/na/program-management/agenda.jsp [Accessed 10/04/2014

2014].

Gell-Mann, M. (1994). Complex adaptive systems. Complexity: Metaphors, models and reality, 17-45.

Gleick, J. 1997. Chaos: Making a new science, Random House.

Goodwin, B. C. 1994. How the leopard changed its spots: The evolution of complexity, Princeton University Press.

Hansen, L. K. & Kræmmergaard, P. (2013). Transforming local government by project portfolio management:

Identifying and overcoming control problems. Transforming Government: People, Process and Policy, 7,

50-75.

Hatzakis, T., Lycett, M. & Serrano, A. (2007). A programme management approach for ensuring curriculum

coherence in IS (higher) education. European Journal of Information Systems, 16, 643-657.

http://www.gartner.com/technology/summits/na/program-management/agenda.jsp




Highsmith, J. 2013. Adaptive software development: a collaborative approach to managing complex systems,

Addison-Wesley.

Holland, J. H. (1992). Complex adaptive systems. Daedalus, 17-30.

Holland, J. H. 1995. Hidden order: How adaptation builds complexity, Basic Books.

Jain, R. & Meso, P. (2004). Theory of complex adaptive systems and Agile software development.

Jeffery, M. & Leliveld, I. (2004). Best practices in IT portfolio management. Mit Sloan Management Review, 45, 41-

49.

Jiang, J. J. & Klein, G. (1999). Risks to different aspects of system success. Information & Management, 36, 263-

272.

Jorgensen, M. & Molokken-Ostvold, K. (2006). How large are software cost overruns? A review of the 1994

CHAOS report. Information and Software Technology, 48, 297-301.

Kane, G. C. & Alavi, M. (2007). Information technology and organizational learning: An investigation of

exploration and exploitation processes. Organization Science, 18, 796-812.

Kauffman, R. J. & Sougstad, R. (2008). Risk Management of Contract Portfolios in IT Services: The Profit-at-Risk

Approach. Journal of Management Information Systems, 25, 17-48.

Kauffman, S. 1995. At home in the universe: The search for the laws of self-organization and complexity, Oxford

University Press.

Kauffman, S. A. 1993. The origins of order: Self-organization and selection in evolution, Oxford university press.

Korhonen, T., Laine, T. & Martinsuo, M. (2014). Management Control of Project Portfolio Uncertainty: A

Managerial Role Perspective. Project Management Journal, 45, 21-37.

Kundisch, D. & Meier, C. (2011). A new perspective on resource interactions in it/is project portfolio selection.

ECIS 2011 Proceedings.

Lefave, R., Branch, B., Brown, C. V. & Wixom, B. (2008). How Sprint Nextel Reconfigured IT Resources for

Results. MIS Quarterly Executive, 7.

Levin, S. A. (1998). Ecosystems and the biosphere as complex adaptive systems. Ecosystems, 1, 431-436.

Levin, S. A. (2003). Complex adaptive systems: Exploring the known, the unknown and the unknowable. Bulletin of

the American Mathematical Society, 40, 3-19.

Lewin, R. 1999. Complexity: Life at the edge of chaos, University of Chicago Press.

Martinsuo, M. (2013). Project portfolio management in practice and in context. International Journal of Project


Martinsuo, M., Korhonen, T. & Laine, T. (2014). Identifying, framing and managing uncertainties in project

portfolios. International Journal of Project Management, 32, 732-746.

Mcfarlan, F. W. (1981). Portfolio approach to information systems. Harvard Business Review, 59, 142-150.

Mcgrath, R. G. (2001). Exploratory learning, innovative capacity, and managerial oversight. Academy of

Management Journal, 44, 118-131.

Merali, Y. (2006). Complexity and information systems: the emergent domain. Journal of Information Technology,

21, 216-228.

Merali, Y., Papadopoulos, T. & Nadkarni, T. (2012). Information systems strategy: Past, present, future? The

Journal of Strategic Information Systems, 21, 125-153.

Meskendahl, S. (2010). The influence of business strategy on project portfolio management and its success - A

conceptual framework. International Journal of Project Management, 28, 807-817.

Mikkola, J. H. (2001). Portfolio management of R&D projects: implications for innovation management.

Technovation, 21, 423-435.

Miles, M. & Huberman, A. 1996. Qualitative Data Analysis, London, Sage.

Miller, J. H. & Page, S. E. 2007. Complex Adaptive Systems: An Introduction to Computational Models of Social

Life: An Introduction to Computational Models of Social Life.

Mitleton-Kelly, E. 2003a. Complex systems and evolutionary perspectives on organisations: the application of

complexity theory to organisations, Elsevier Science Ltd.

Mitleton-Kelly, E. 2003b. Ten principles of complexity and enabling infrastructures, Elsevier.

Nan, N. (2011). Capturing bottom-up information technology use processes: A complex adaptive systems model.

MIS Quarterly, 35.

Naur, P. & Randell, B. (1969). Software Engineering: Report of a conference sponsored by the NATO Science

Committee, Garmisch, Germany, 7-11 Oct. 1968, Brussels, Scientific Affairs Division, NATO.

Niccolini, D. 2013. Practice, Theory, Work and Organisation: An Introduction, Oxford University Press.

Parnas, D. L. & Lawford, M. (2003). The role of inspection in software quality assurance. Ieee Transactions on

Software Engineering, 29, 674-676.




Patton, M. 1990. Qualitative evaluation and research methods, Beverly Hills, CA, Sage.

Payne, J. H. (1995). Management of multiple simultaneous projects: a state-of-the-art review. International Journal

of Project Management, 13, 163-168.

Perry, M. P. (2012). Business driven project portfolio management: Conquering the top 10 risks that threaten

success. Project Management Journal, 43, 83.

Plowman, D. A., Solansky, S., Beck, T. E., Baker, L., Kulkarni, M. & Travis, D. V. (2007). The role of leadership in

emergent, self-organization. Leadership Quarterly, 18, 341-356.

Prigogine, I., Stengers, I. & Pagels, H. R. (1985). Order out of Chaos. Physics Today, 38, 97.

Reich, B. H. & Benbasat, I. (2000). Factors that influence the social dimension of alignment between business and

information technology objectives. MIS quarterly, 81-113.

Rouse, W. B. (2000). Managing Complexity. Information, Knowledge, Systems Management, 2, 143-165.

Rungi, M. Interdependency management in project portfolio management: How to implement required procedures.

PICMET '10 - Portland International Center for Management of Engineering and Technology, Proceedings

- Technology Management for Global Economic Growth, July 18-22 2010 Phuket. 1551-1561.

Snowden, D. J. & Boone, M. E. (2007). A leader's framework for decision making. harvard business review, 85, 68.

Stacey, R. D. 2002. Complexity and management, Routledge.

Stummer, C. & Heidenberger, K. (2003). Interactive R&D portfolio analysis with project interdependencies and

time profiles of multiple objectives. Engineering Management, IEEE Transactions on, 50, 175-183.

Tan, J., Wen, H. J. & Awad, N. (2005). Health care and services delivery systems as complex adaptive systems.

Communications of the Acm, 48, 36-44.

Uhl-Bien, M., Marion, R. & Mckelvey, B. (2007). Complexity Leadership Theory: Shifting leadership from the

industrial age to the knowledge era. Leadership Quarterly, 18, 298-318.

Vessey, I. & Ward, K. (2013). The Dynamics of Sustainable IS Alignment: The Case for IS Adaptivity. Journal of

the Association for Information Systems, 14, 283-311.

Vidgen, R. & Wang, X. Organizing for agility: A complex adaptive systems perspective on agile software

development process. 14th European Conference on Information Systems, June 12-14, 2006. 2006

Goteborg.

Vidgen, R. & Wang, X. (2009). Coevolving systems and the organization of agile software development.

Information Systems Research, 20, 355-376.

Volberda, H. W. & Lewin, A. Y. (2003). Co‐evolutionary Dynamics Within and Between Firms: From Evolution to

Co‐evolution. Journal of management studies, 40, 2111-2136.

Xia, W. & Lee, G. (2004). Grasping the complexity of IS development projects. Communications of the ACM, 47,

68-74.

Yin, R. K. 2009. Case study research: Design and methods, Sage.

Strode Measuring coordination in agile software development



Measuring Coordination in Agile Software Development

Diane E. Strode

Whitireia Polytechnic

[email protected]

ABSTRACT

Coordination has long been recognized as contributing to successful IT projects. Agile software development

provides many practices for achieving project coordination in small co-located projects. Given the importance of

coordination to successful software development projects and the increasing popularity of agile software

development, investigating coordination in this context is timely and potentially useful. This paper takes an existing

theory of coordination in co-located agile software development projects developed from case study research and

proposes a field test of that theory. The question addressed is what is the effect of an agile coordination strategy on

coordination effectiveness in co-located software development projects? This paper describes the initial theory of

coordination and a research design for field-testing that theory.

Keywords

Agile methods, co-located software development, coordination effectiveness, coordination strategy, coordination

theory, explicit coordination, implicit coordination.

INTRODUCTION

Developing an information system is a group effort and effective group efforts require coordination. Coordination

has been defined in many domains (Malone and Crowston, 1994). For example, in teamwork “Coordination means

the spatial and temporal synchronization of overt behaviors of two or more people so that those actions fit together

into an intended spatial and temporal pattern” (Arrow, McGrath, and Berdahl, 2000, p. 42). Coordination has long

been recognized as contributing to successful large-scale IT projects (Curtis, Krasner, and Iscoe, 1988; Kraut and

Streeter, 1995; Nidumolu, 1995) and more recently to projects distributed across countries, continents, and time

zones (Cummings, Espinosa, and Pickering, 2009; Kotlarsky, van Fenema, and Willcocks, 2008).

Agile software development is an approach to information systems development that is particularly concerned with

group endeavor. Agile software development is an umbrella term for any agile method, such as Scrum, Extreme

Programming (XP), or any assemblage of agile practices (Conboy and Fitzgerald, 2007). Originally intended for

software projects involving co-located teams, the approach is increasingly adapted for distributed project

environments (Jalali and Wohlin, 2012; Sarker, Munson, Sarker, and Chakraborty, 2009). In the last 15 years the

world-wide level of adoption has risen to include about 50% of software projects (Stavru, 2014) and shows no sign

of abating. Given the importance of coordination to successful systems development and the increasing popularity of

agile software development, research to investigate coordination in this context is timely and potentially useful.

Research focusing specifically on coordination in agile software development context is scant (Dingsoyr, Nerur,

Balijepally, and Moe, 2012; Strode, Huff, Hope, and Link, 2012), although case studies (Chuang, Luor, and Lu,

2014) and ethnographies have identified coordination as an important element in agile projects (Mishra, Mishra, and

Ostrovska, 2012; Pries-Heje and Pries-Heje, 2011; Sharp and Robinson, 2010). This research provides in-depth

knowledge about how coordination occurs in selected agile projects but does not explain the relationship between

the use of agile practices and effective software project coordination.

There is one theory focusing exclusively on coordination in co-located agile software development projects. This

theory was developed inductively and systematically by Strode (2012) from four case studies of agile and non-agile

software development projects. A refinement of the theory based on three case studies of agile software

development was published by Strode, Huff, Hope and Link (2012). The theory proposes that the coordination

effectiveness of an agile software development project is affected by the coordination strategy of the project. This

theory, although carefully argued and supported with evidence from in-depth cases studies, has not yet been tested in




a field study. This paper describes a research design for operationalizing and testing the concepts and relationships

proposed in this theory. The broad research question this research addresses is what is the effect of an agile

coordination strategy on coordination effectiveness in co-located software development projects?

This paper first summarises the theory proposed by Strode et al. (2012) and then describes the proposed research

design for testing the theory in the field. The status of the research, potential contributions, and limitations of the

study are discussed, and the paper concludes.

A THEORY OF COORDINATION IN AGILE SOFTWARE DEVELOPMENT PROJECTS

The coordination theory in agile software development projects proposed by Strode et al. (2012) was developed

from independent cases of agile software development. Cases were selected because they showed a typical profile

for co-located agile projects. Projects had a team size of 5 to 10, and the agile methods used were either Scrum or

Scrum with XP practices since these are the most commonly adopted agile methods (Stavru, 2014). The theory has

two primary theoretical concepts named coordination strategy and coordination effectiveness.

Coordination Strategy

The coordination strategy concept is concerned with the typical assemblage of agile practices that occurred in the

cases. This concept has synchronisation, structure, and boundary spanning dimensions.

Synchronisation is a relation that exists when things occur at the same time, or are simultaneous (Allen, 1990, p.

1236). Synchronisation is achieved with synchronisation activities and synchronisation artefacts. A synchronisation

activity involves all team members and brings them together at the same time and place for some pre-arranged

purpose. A daily stand-up team meeting, a product demonstration, and a retrospective meeting are examples of

synchronisation activities (for detailed descriptions of the most common agile practices see Williams (2010)). In the

agile project context, synchronisation activities occur at different frequencies: once per project, once per iteration or

sprint, daily, and ad hoc (as and when needed). For example, a project team meeting is held at the start of the project

to discuss process, technical, and domain issues. At the start of each sprint, the agile project team holds a planning

meeting to discuss progress, update the Scrum wallboard, and to decompose stories into tasks. Meetings are held

daily where each project team member individually reports progress and problems. Ad hoc meetings, between the

whole team and between sub-groups of the team, are held for planning and discussing issues as they arise.

Synchronisation artefacts are produced and consumed (or used up) during synchronisation activities. Any physical

thing generated during a synchronisation activity that contains information used by all team members in

accomplishing their work is a synchronisation artefact. Typical synchronisation artefacts included story cards, task

cards, a Scrum wallboard, and an automated test suite.

Structure is a second dimension of coordination strategy. Structure is defined in its common sense as the

arrangement of and relations between the parts of something complex, and has sub-dimensions of proximity,

availability, and substitutability. The agile methods Scrum and XP (Beck, 2000; Schwaber and Beedle, 2002)

specify that close proximity of the project team, timely response to requests for help from within the team, and the

sharing of work tasks among the team, as opposed to intense specialisation, are important for success in agile

projects. Each of these sub-dimensions was identified in the agile cases on which the theory was based and they are

defined as follows. Proximity is concerned with the physical closeness of individual team members; with adjacent

desks providing the highest level of proximity. Availability occurs when team members are continually present and

able to respond to requests for assistance or information. Substitutability occurs when team members are able to

perform the work of another to maintain time schedules.

Boundary spanning is the third dimension of coordination strategy. Boundary spanning has three sub-dimensions

including boundary spanning activities, the production of boundary spanning artefacts, and a coordinator role. In the

theory, boundary-spanning activities can occur once per project, per iteration, and ad hoc. A boundary spanning

activity is an activity performed by the team or the individual to elicit assistance or information from some unit or

organisation external to the project to achieve project goals. A boundary-spanning artefact is produced to enable

coordination beyond the team and project boundaries. An example is an email to request a new server from the IT

support section. A coordinator role is taken by a project team member specifically to support interaction with people

who are not part of the project team but who provide resources or information to the project. The agile methods

Scrum and XP do not explicitly provide for boundary spanning (Beck, 2000; Schwaber and Beedle, 2002).




Nevertheless, in the cases on which the theory is based, interaction with other organisations and other business units

within the organisation where the project was carried out, were important coordinative activities, therefore boundary

spanning was included as a dimension of coordination strategy in agile software development projects.

Coordination Effectiveness

Coordination effectiveness is the outcome of a particular coordination strategy in Strode et al.’s (2012) theory.

Coordination effectiveness was found to have an explicit and an implicit dimension. Coordination literature defines

explicit coordination as that which occurs when two or more team members use overt mechanisms such as

schedules, plans, and procedures, and send communication messages to one another using formal or informal, oral

or written, transactions to integrate their work (Espinosa, Lerch, and Kraut, 2004; Nonaka, 1994; Rico, Sanchez-

Manzanares, Gil, and Gibson, 2008). Coordination effectiveness was found to have three explicit dimensions: right

thing, right place, and right time. That is, when a situation is well coordinated then the right things are in the right

place, at the right time.

Implicit coordination occurs when team members anticipate the actions and needs of their colleagues and adjust

their behavior accordingly without preplanning or direct communication (Nonaka, 1994; Rico et al., 2008). Five

implicit dimensions of coordination effectiveness were identified in the theory. All project members need to 1) know

why they are working on a task (or have a shared goal), 2) know what is going on and when, 3) know what to do and

when, and, 4) know who is doing what, and 5) know who knows what.

Supporting Research

Prior research supports the existence of each dimension of Strode et al.’s (2012) coordination theory. This research

has been carried out in a variety of contexts including agile software development projects. Table 1 shows examples

of literature supporting each dimension of the theory and the context in which the research was carried out.

Although research supports each dimension in the theory, there does not seem to be any extant theory or model that

uniquely combines these concepts and dimensions, and proposes how they are related as Strode et al. (2012) have

done in the agile software development context.

Dimension Sub-dimension Indicative literature Context

Synchronisation Synchronisation activity Schmidt and Simone

(1996)

Salas, Sims, and Burke

(2005)

Arrow et al. (2000)

Cooperative work

Teamwork

Small groups

Synchronisation artefact Ren, Kiesler, and Fussell

(2008))

Sharp and Robinson

(2008)

Hospital settings

Agile software development

Structure Proximity Hoegl and Proserpio

(2004)

Teasley, Covi, Krishnan,

and Olson (2002)

Software development teams

Software development

Availability Weick and Roberts (1993)

Matook and Kautz (2008)

Flight operations


Substitutability Salas et al. (2005) Teamwork

Boundary

spanning

Boundary spanning activity Levina and Vaast (2005)

Information systems (IS)

development

Boundary spanning artefact Levina and Vaast (2005) IS development

Coordinator role Hoda, Noble, and Marshall

(2010)


Implicit

coordination

Know why/shared goal Parolia, Goodman, Li, and

Jiang (2007)

Salas et al. (2005)

IS development

Teamwork

Know what is going on and Yang, Kang, and Mason Software development teams




when (2008)

Know what to do and when Salas et al. (2005) Teamwork

Know who is doing what Moe, Dingsoyr, and Dyba

(2010)


Know who knows what Faraj and Sproull (2000)

Lin, Hsu, Cheng, and Wu

(2012)

Software development teams

IS development teams

Explicit

coordination

Right thing

Right place

Right time

Crowston and Osborn

(2003)

Process mapping

Table 1 Supporting literature for the coordination concepts

The Relationship between Coordination Strategy and Coordination Effectiveness

A theory has theoretical concepts, associations between concepts, boundaries, and system states (Weber, 2012).

Strode et al.’s (2012) theory of coordination has two associated concepts, and proposes that the coordination strategy

of an agile project is associated with a level of coordination effectiveness. The most salient boundary of this

theoretical system is the co-located agile software development project with additional restrictions, based on the

cases informing the theory, as follows.

1. A software development project using practices from Scrum, or Scrum and Extreme Programming, with

iterations (sprints) of one or two weeks.

2. An identifiable project team of 5 to 10 people working concurrently and full-time on the project, and who

are located in close proximity within the same room in direct line of sight of one another.

3. A project with a clear business purpose that is either providing a software product for another business unit

within the organisation or for an external organisation.

4. A project with a distinguishable customer or proxy customer. This can be a single person, a group, or

groups of people. This customer can work within the team (physically sited with the team and involved in

their daily work) or be an external party who is available for consultation.

The coordination strategy concept was found to have two states determined by the relationship of the customer with

the team. The customer in an agile project is expected to be a knowledgeable and committed person closely and

continuously involved with the project team on a daily basis (i.e. embedded) (Beck, 2000; Schwaber and Beedle,

2002). When the customer is embedded in the project then synchronisation activities and artefacts, and proximity,

availability, and substitutability are sufficient to achieve coordination effectiveness. When the customer is external

to project team, which is often what occurs in practice (Lohan, Conboy, and Lang, 2011), a project also requires

boundary-spanning activities, artefacts, and roles to achieve effective coordination. This is because an external

customer needs to be consulted more formally via pre-arranged meetings, and may need to prepare or read project

documents to remain involved with and contribute to the project. In addition, a special role of coordinator might be

arranged by the project team to facilitate interaction with the customer or other involved parties contributing to the

project. These states lead to proposition 1.

Proposition 1. When the customer is embedded within the project team, a coordination strategy that includes

synchronisation and structure coordination improves project coordination effectiveness. Synchronisation activities

and associated artefacts are required at all frequencies (i.e. per project, per iteration, daily, and ad hoc). When the

customer is an external party to the project then boundary spanning coordination is also needed. Boundary spanning

activities and associated artefacts are required at all frequencies (i.e. per project, per iteration, and ad hoc). A

boundary spanning coordinator role is also necessary.

Proposition 1 treats coordination effectiveness as a unitary concept. Propositions 2, 3, and 4 elaborate on the

relationships between the three coordination strategy dimensions and the two coordination effectiveness dimensions.

First, synchronisation activities such as iteration zero planning meetings, product demonstrations at the end of each

sprint, daily standup meetings, and other meetings held at irregular intervals, serve to increase the project team’s

implicit coordination. In addition, artefacts produced during these activities such as story cards, task cards, and

wallboard displays increase the project team’s implicit coordination. That is these meetings and artefacts increase

the project team members knowledge of the reasons why they are working on a task, increases their knowledge

about what is going on and when, their knowledge about what to do and when, and their knowledge about who is




doing what on the project. These meetings also increase their knowledge about who knows what within the project

team. This leads to proposition two.

Proposition 2. Synchronisation activities at all frequencies – per project, per iteration, daily, and ad hoc, along with

their associated synchronisation artefacts, increase implicit coordination effectiveness.

Similarly, the structural coordination mechanisms of close proximity, high availability, and high substitutability also

increase each sub-dimension of implicit coordination. This leads to proposition three.

Proposition 3. Structural coordination mechanisms including close proximity, high availability, and high

substitutability increase implicit coordination effectiveness.

Boundary spanning increases explicit coordination effectiveness, which is when the right things are in the right place

at the right time. Boundary-spanning activities include activities such as holding a meeting with the customer or

their representative. Boundary spanning artefacts include documents such as plans, requirements documents,

specifications, and formal requests for resources. In an agile project, these activities occur once per project usually at

project initiation, once per iteration such as when a product demonstration is held with invited customers, and ad hoc

which means as and when needed. The leads to proposition 4.

Proposition 4. Boundary spanning coordination mechanisms including boundary-spanning activities at all

frequencies, i.e. per project, per iteration, and ad hoc, their associated boundary-spanning artefacts, and a

coordinator role, increases explicit coordination effectiveness.

RESEARCH DESIGN

To test the theory of coordination proposed by Strode et al. (2012) in the field using quantitative methods, each

complex proposition stated in the theory (proposition 1 to 4) was decomposed into a series of simple testable

hypotheses. Table 2 shows each proposition and its related hypotheses. The model for this system of hypotheses is

shown in Figure 2.

Propositions Hypotheses

1 When the customer is embedded within the project

team, a coordination strategy that includes

synchronisation and structure coordination improves

project coordination effectiveness. Synchronisation

activities and associated artefacts are required at all

frequencies (i.e. per project, per iteration, daily, and

ad hoc).

When the customer is an external party to the project

then boundary spanning coordination is also needed.

Boundary spanning activities and associated artefacts

are required at all frequencies (i.e. per project, per

iteration, and ad hoc). A boundary spanning

coordinator role is also necessary.

H1. Customer involvement is positively related to

implicit coordination.

H2. Customer involvement is positively related to

explicit coordination.

H3. Customer involvement is negatively related to

boundary spanning activities.

H4. Customer involvement is negatively related to

boundary spanning artefacts.

H5. Customer involvement is negatively related to the

boundary spanning coordinator role.

2 Synchronisation activities at all frequencies – per

project, per iteration, daily, and ad hoc, along with

their associated synchronisation artefacts, increase

implicit coordination effectiveness.

H6. The frequency of synchronisation activities is

positively related to implicit coordination.

H7. The frequency of production of synchronisation

artefacts is positively related to implicit

coordination.

3 Structural coordination mechanisms including close

proximity, high availability, and high substitutability

increase implicit coordination effectiveness.

H8. Proximity is positively related to implicit

coordination.

H9. Availability is positively related to implicit

coordination.

H10. Substitutability is positively related to implicit

coordination.




4 Boundary spanning coordination mechanisms

including boundary-spanning activities at all

frequencies, i.e. per project, per iteration, and ad hoc,

their associated boundary-spanning artefacts, and a

coordinator role, increases explicit coordination

effectiveness.

H11. Boundary spanning activities are positively

related to explicit coordination

H12. Boundary spanning artefacts are positively

related to explicit coordination

H13. The boundary spanning coordinator role is

positively related to explicit coordination.

Table 2 Mapping of propositions to hypotheses




Figure 2 Theoretical model

This proposed field research has a unit of analysis at the project level. Therefore, field data will be drawn from

projects selected to fit the project profile, as listed and described in the previous section. That is, each project will fit

the theoretical boundaries and restrictions defined in Strode et al.’s (2012) theory. This will allow for some control

over variation in team size and in the assemblage of practices used. This also means that sample selection is

purposive rather than random.

+ H8

+ H9

Explicit

coordination

Proximity

Synchronisation

artefact

Availability

Synchronisation

activity

Implicit

coordination

+ H7

+ H12

+ H11

+ H1

+ H10

+ H6

Boundary

spanning activity

Coordinator role

Boundary

spanning artefact

+ H2

- H3

+ H13

- H5

- H4

Customer

involvement

Substitutability




Projects will be selected and data collected from each one using three survey instruments. The first instrument will

be for project profiling and will be completed by a single knowledgeable person on the project. This instrument

captures data such as estimated project duration, length of sprint, and other information likely to remain unchanged

for the project duration. The second instrument is for collecting coordination strategy data, and the third instrument

is for collecting coordination effectiveness data. Coordination strategy and coordination effectiveness are measured

using 7-point scales, coordination strategy items range from “Not followed at all” to “Followed very strictly”, and

coordination effectiveness items range from “Strongly agree” to “Strongly disagree”.

Each project will be visited weekly for up to 10 visits on the same day of the week at an agreed time. At the first

visit, the team will be divided into two groups of the same size (or nearly equal for groups of uneven size). One

group will complete the second questionnaire on the coordination strategy they used in the previous weeks work.

The other group will complete the third questionnaire on the coordination effectiveness of the previous weeks work.

In following visits, each group will be offered the alternative questionnaire. Assuming a team size of 10, there will

be a maximum of 50 individual data points collected for one project.

The reason for splitting the team into groups with one group completing the strategy instrument and the other

completing the effectiveness instrument is to reduce potential common methods variance. We believe this bias

would be particularly likely in this context where the respondents would be fully aware of the expected outcomes of

the practices they use in their projects. The most problematic source of common methods variance occurs when “the

data for both the predictor and criterion variable are obtained from the same person in the same measurement

context using the same item context and similar item characteristics” (Chang, van Witteloostuijn, and Eden, 2010, p.

179). To reduce this bias the simplest tactic for the project-level research proposed in this paper is to have different

project team members assess different variables in each timeframe. Data analysis will involve aggregating data from

a single project. Exploratory factor analysis followed by confirmatory factory analysis is proposed. The theoretical

model contains formative constructs therefore PLS-SEM, which is appropriate for this type of model, is planned for

data analysis using the SmartPLS™ tool.

STATUS OF THE RESEARCH

The research project is at the stage where preliminary survey instruments are developed (see Appendix for

questionnaire items for coordination strategy and effectiveness). The project profile questionnaire has been used in

prior projects to assess face validity. Face validity for the coordination strategy and coordination effectiveness

instruments has been assessed by 10 people including agile software development professionals, IT students, and

non-IT people, and the instruments have been adjusted accordingly. A further assessment of content validity is

planned using an item-rating task with agile software development professionals and project managers (MacKenzie,

Podsakoff, and Podsakoff, 2011). Negotiations with project teams are underway to collect data for a pretest to assess

psychometric scale properties, convergent, discriminant, and nomological validities (Straub, Boudreau, and Gefen,

2004).

POTENTIAL CONTRIBUTIONS

Potentially, this research will provide a better understanding of which agile practices contribute to effective project

coordination. This information will be useful to practitioners because currently they select their agile method and

adopt individual agile practices based on hard-won experience, the advice of consultants, or they adopt practices

without prior understanding of their individual and combined effects.

This research has limitations. The first is that sampling is purposive rather than random. A second limitation is the

sample size imposed by surveying only typical agile project teams, which are small, optimally ranging from 4 to 10

developers. These restrictions mean that the range of statistical tests available to analyse the data is limited.

CONCLUSION

This paper describes a proposed field test of a theory developed from case study research. The initial theory focuses

on the relationship between coordination strategy, which are the behaviors performed by agile project teams in a co-

located projects, and coordination effectiveness. The field test proposes to test four propositions that have been

elaborated into hypotheses.




APPENDIX

Coordination Strategy Construct

Dimension Sub-dimension Items

Synchronisation Synchronisation activity

This week we used… Iteration or sprint planning meeting Story or task breakdown session Team members together selected stories to work on Reflective workshop or retrospective meeting Software release Informal unscheduled whole team meeting Story point prioritising session Product demonstration with whole team and customer Unscheduled meeting of 2 or more people Product backlog maintenance session Daily standup or daily team meeting Progress tracking with user stories Daily build of the complete system Self-assignment of stories Pair programming Cross-team talk Continuous integration and testing Broadcast email Acceptance testing Informal chat using SMS or similar technology

Synchronisation artefact

This week we used … Coding standards A product backlog A continuously updated design document) A working version of the software User stories A burn-down chart A done list A whiteboard sketching or recording ideas A wiki for storing information Avatars on task or work item A software tool to store the backlog of stories Layered architecture (n or 3-tier) A source code control tool A unit test suite Ground rules Work items Work in progress limits A bug tracking tool (e.g. JIRA™) A Scrum wallboard – real or virtual A Kanban wallboard – real or virtual

Structure Proximity Customer was co-located with the team Team worked in open work area and could see one another

Availability Team had a single assignment and worked full-time

Substitutability Team members performed other’s tasks (e.g. developer did some testing, BA did some development)

Boundary spanning

Boundary spanning activity

This week the team and an external party (client, customer, end-user, or group of end-users)… Shared domain knowledge Worked together to generate a backlog Prioritized user stories together Viewed a software demonstration together Consulted daily together Had an unscheduled formal meeting together (face-to-face or by distance) Had an unscheduled informal face-to-face meeting together

Boundary spanning artefact

This week the team … Prepared documents for external parties

Coordinator role This week the team … Had a team member act as customer liaison

Table 3 Construct definition: Coordination Strategy




Coordination Effectiveness Construct

Dimension Sub-dimension Items In the previous week …

Implicit coordination

Know why/shared goal To what extent do you feel you understood: The overall project goal How tasks you worked on helped to achieve the project goal

Know what is going on and when To what extent do you feel you understood: What tasks were underway? Which tasks needed to be performed?

Know what to do and when To what extent do you feel that you knew: What tasks you should be working on? When the tasks you worked on were required to be finished?

Know who is doing what To what extent do you feel that you knew: What tasks your project team members were working on?

Know who knows what To what extent do you feel that you knew: What knowledge your project team members had? What capabilities your project team members had?

Explicit coordination

Right thing Right place Right time

To what extent do you feel that people, resources, and information you depended on to complete your work: Were available at the right time Were available at the correct location Were fit for use

Additional question

Do you think the project work was well coordinated this week?

Table 4 Construct definition: Coordination Effectiveness

REFERENCES

Allen, R. E. (1990). The concise Oxford dictionary of current English (8 ed.), Clarendon Press, Oxford.

Arrow, H., McGrath, J. E., and Berdahl, J. L. (2000). Small groups as complex systems: Formation, coordination,

development, and adaptation, Sage, Thousand Oaks.

Beck, K. (2000). Extreme Programming Explained: Embrace Change, Addison-Wesley, Boston.

Chang, S., van Witteloostuijn, A., and Eden, L. (2010). From the editors: Common method variance in international

business research. Journal of International Business Studies, 41, 1, 178-184.

Chuang, S.-W., Luor, T., and Lu, H.-P. (2014). Assessment of institutions, scholars, and contributions on agile

software development (2001-2012). The Journal of Systems and Software. doi:

http://dx.doi.org/10.1016/j.jss.2014.03.006.

Conboy, K., and Fitzgerald, B. (2007). The views of experts on the current state of agile method tailoring, in T.

McMaster, D. Wastell, E. Ferneley and J. DeGross (Eds.), IFIP International Federation for Information

Processing (Vol. 235, pp. 217-234). Springer, Boston.

Crowston, K., and Osborn, C. S. (2003). A coordination theory approach to process description and redesign, in T.

W. Malone, K. Crowston and G. A. Herman (Eds.), Organizing Business Knowledge: The MIT Process

Handbook (pp. 335-370), The MIT Press, Cambridge, Massachusetts.

Cummings, J. N., Espinosa, A. J., and Pickering, C. K. (2009). Crossing spatial and temporal boundaries in globally

distributed projects: A relational model of coordination delay. Information Systems Research, 20, 3, 420-

439.

Curtis, B., Krasner, H., and Iscoe, N. (1988). A field study of the software design process for large systems,

Communications of the ACM, 31, 11, 1268-1287.

Dingsoyr, T., Nerur, S., Balijepally, V., and Moe, N. (2012). A decade of agile methodologies: Towards explaining

agile software development. The Journal of Systems and Software, 85, 6, 1213-1221.

Espinosa, A. J., Lerch, F. J., and Kraut, R. E. (2004). Explicit versus implicit coordination mechanisms and task

dependencies: One size does not fit all. In E. Salas and S. M. Fiore (Eds.), Team Cognition: Understanding

the Factors that Drive Process and Performance (pp. 107-129), American Psychological Association,

Washington DC.




Faraj, S., and Sproull, L. (2000). Coordinating expertise in software development teams, Management Science, 46,

12, 1554-1568.

Hoda, R., Noble, J., and Marshall, S. (2010). Organizing self-organizing teams, Proceedings of the IEEE/ACM

International Conference on Software Engineering, ICSE 2010, May 2-9, 2010 (pp. 285-294), ACM, New

York, NY.

Hoegl, M., and Proserpio, L. (2004). Team member proximity and teamwork in innovative projects, Research

Policy, 33, 8, 1153-1165.

Jalali, S., and Wohlin, C. (2012). Global software engineering and agile practices: A systematic review, Journal of

Software: Evolution and Process, 24, 643-659. doi: 10.1002/smr.561.

Kotlarsky, J., van Fenema, P. C., and Willcocks, L. P. (2008). Developing a knowledge-based perspective on

coordination: The case of global software projects, Information and Management, 45, 2, 96-108.

Kraut, R. E., and Streeter, L. A. (1995). Coordination in software development, Communications of the ACM, 38, 3,

69-81.

Levina, N., and Vaast, E. (2005). The emergence of boundary spanning competence in practice: Implications for

implementation and use of information systems, MIS Quarterly, 29, 5, 335-363.

Lin, T.-C., Hsu, J. S.-C., Cheng, K., and Wu, S. (2012). Understanding the role of behavioural integration in ISD

teams: An extension of transactive memory systems concept, Information Systems Journal, 22, 211-234.

Lohan, G., Conboy, K., and Lang, M. (2011). Examining customer focus in IT project management: Findings from

Irish and Norwegian case studies, Scandinavian Journal of Information Systems, 23, 2, 29-58.

MacKenzie, S. B., Podsakoff, P. M., and Podsakoff, N. P. (2011). Construct measurement and validation procedures

in MIS and behavioral research: Integrating new and exisiting techniques, MIS Quarterly, 32, 2, 293-334.

Malone, T. W., and Crowston, K. (1994). The interdisciplinary study of coordination. ACM Computing Surveys, 26,

1, 87-119.

Matook, S., and Kautz, K. (2008). Mindfulness and agile software development. In S. L. Huff (Ed.), Proceedings of

the 19th Australasian Conference on Information Systems, ACIS 2008 (pp. 638-646). Retrieved from

http://aisel.aisnet.org/acis2008/58.

Mishra, D., Mishra, A., and Ostrovska, S. (2012). Impact of physical ambiance on communication, collaboration and

coordination in agile software development: an empirical evaluation, Information and Software

Technology, 54, 10, 1067-1078.

Moe, N., Dingsoyr, T., and Dyba, T. (2010). A teamwork model for understanding an agile team: A case study of a

Scrum project, Information and Software Technology, 52, 5, 480-491.

Nidumolu, S. (1995). The effect of coordination and uncertainty on software project performance: Residual

performance risk as an intervening variable, Information Systems Research, 6, 3, 191-219.

Nonaka, I. (1994). A dynamic theory of organizational knowledge creation. Organization Science, 5(1), 14-37.

Parolia, N., Goodman, S., Li, Y., and Jiang, J. J. (2007). Mediators between coordination and IS project

performance, Information and Management, 44, 7, 635-645.

Pries-Heje, L., and Pries-Heje, J. (2011). Why Scrum works. Proceedings of the Agile Conference 2011, Salt Lake

City, UT (pp. 20-28): IEEE Xplore digital library. doi: 10.1109/AGILE.2011.34

Ren, Y., Kiesler, S., and Fussell, S. (2008). Multiple group coordination in complex and dynamic task environments:

Interruptions, coping mechanisms, and technology recommendations, Journal of Management Information

Systems, 25,1, 105-130.

Rico, R., Sanchez-Manzanares, M., Gil, F., and Gibson, C. (2008). Team implicit coordination processes: A team

knowledge-based approach, Academy of Management Review, 33, 1, 163-184.

Salas, E., Sims, D. E., and Burke, C. S. (2005). Is there a 'big five" in teamwork? Small Group Research, 36, 5, 555-

599.

Sarker, S., Munson, C. L., Sarker, S., and Chakraborty, S. (2009). Assessing the relative contribution of the facets of

agility to distributed systems development success: an analytic hierarchy process approach, European

Journal of Information Systems, 18, 1, 285-299.

Schmidt, K., and Simone, C. (1996). Coordination mechanisms: towards a conceptual foundation of CSCW systems

design. Computer Supported Cooperative Work: The Journal of Collaborative Computing, 5, 155-200.

Schwaber, K., and Beedle, M. (2002). Agile Software Development with Scrum, Prentice Hall, Upper Saddle River,

New Jersey.

Sharp, H., and Robinson, H. (2008). Collaboration and co-ordination in mature eXtreme programming teams.

International Journal of Human-Computer Studies, 66, 7, 506-518.




Sharp, H., and Robinson, H. (2010). Three 'C's of agile practice: collaboration, co-ordination and communication. In

T. Dingsoyr, T. Dyba and N. B. Moe (Eds.), Agile software development: current research and future

directions, Springer-Verlag, Heidelberg.

Stavru, S. (2014). A critical examination of recent industrial surveys on agile method usage, The Journal of Systems

and Software, In press. doi: http://dx.doi.org/10.1016/j.jss.2014.03.041.

Straub, D., Boudreau, M.-C., and Gefen, D. (2004). Validation guidelines for IS postivist research. Communications

of the Association for Information Systems, 13, Article 24, 380-427.

http://aisel.aisnet.org/cais/vol13/iss1/24.

Strode, D. E. (2012). A theory of coordination in agile software development projects (PhD), Victoria University of

Wellington, New Zealand. Retrieved from http://hdl.handle.net/10063/2505.

Strode, D. E., Huff, S. L., Hope, B., and Link, S. (2012). Coordination in co-located agile software development

projects. The Journal of Systems and Software, 85, 6, 1222-1238. doi: 10.1016/j.jss.2012.02.017.

Teasley, S. D., Covi, L. A., Krishnan, M. S., and Olson, J. S. (2002). Rapid software development through team

collocation, IEEE Transactions on Software Engineering, 28, 7, 671-683.

Weber, R. (2012). Evaluating and developing theories in the information systems discipline, Journal of the

Association for Information Systems, 13, 1, 1-30.

Weick, K. E., and Roberts, K. H. (1993). Collective mind in organizations: Heedful interrelating on flight decks.

Administrative Science Quarterly, 38, 3, 357-381.

Williams, L. (2010). Agile software development methodologies and practices. In M. V. Zelkowitz (Ed.), Advances

in Computers (Vol. 80, pp. 1-44), Elsevier, Amsterdam.

Yang, D.-H., Kang, H.-R., and Mason, R. M. (2008). An exploratory study on the meta skills in software

development teams: Antecedent cooperation skills and personality for shared mental models. European

Journal of Information Systems, 17, 47-61.

Bick et al. Structural Incongruences in Software Development



Software Development in Multiteam Systems: A Longitudinal Study on the Effects of Structural

Incongruences on Coordination Effectiveness

Saskia Bick

University of Mannheim

[email protected]

Alexander Scheerer


[email protected]

Kai Spohrer


[email protected]

Thomas Kude


[email protected]

Armin Heinzl


[email protected]

ABSTRACT

This study examines structural incongruences between organizational and product domains and their implications

for coordination effectiveness in large-scale software development. We use the ongoing shift from on-premise to

cloud-based software solutions to examine longitudinal effects of structural incongruences, i.e. the mismatch

between organizational structures, including knowledge and task dependencies, and product structures, including

technical dependencies, and how they resolve. We integrate extant literature in this field with literature on multiteam

systems (MTS) and team composition to guide our longitudinal case study of one particular MTS within a large

software organization. First insights from an initial case study of three development teams of different MTSs show

how high level structural incongruences emerge on a team-level, providing a foundation for our subsequent study.

By exploring the effects of structural incongruences over time, we expect to contribute to existing literature on

organizational and product structure alignment as well as on MTS coordination effectiveness research.

Keywords

Organizational structure, product structure, structural incongruence, multiteam systems, dependencies, coordination

effectiveness, longitudinal case study, process theory

INTRODUCTION

In recent years, the ever increasing focus on faster delivery cycles and quicker reaction to customer feedback led to

profound changes for software development organizations, such as the widespread change in the provisioning mode

of software solutions (Shetty, 2013; Stuckenberg, Kude and Heinzl, 2014). Many large software development

organizations changed their license model by shifting from developing and selling on-premise software solutions,

i.e. software which is hosted and run by the customer, to offering on-demand software solutions, i.e. software which

is hosted and run by the vendor. Such a fundamental strategic change implies new requirements and challenges for

the entire organization, which needs to realize a fast delivery of new features for its on-demand solutions (Dillon,

Wu and Chang, 2010).

The modern large-scale software development organization tries to cope with these changing requirements by

establishing a particular, team-oriented organizational structure. In most large-scale IT projects, several teams work

together to implement a software system and a team of teams, or multiteam, setup has become widely accepted

practice (Scheerer, Hildenbrand and Kude, 2014). Typically, the multiteam system (MTS) structure has been based

on the modularized components of the software architecture (Keith, Demirkan and Goul, 2013; Subramanyam,

Ramasubbu and Krishnan, 2012). In such a setting, each team owns the development and maintenance responsibility

for one component. Modularized by means of the software architecture stack, i.e. the technological layers of a

software product, this could mean that team A owns the responsibility for the user interface layer, whereas team B is

responsible for the database layer of the software. Urged by customers to provide more frequent functional updates,




companies are currently focusing on alternative ways of structuring their development organization. Instead of

keeping a long-lasting and specialized software component responsibility, development teams repeatedly take on

responsibility for small parts of functionality, i.e. features, which deliver visible business value across various

components (Larman and Vodde, 2010). In complex enterprise software, however, single features necessarily span

multiple modular components and different layers of the existing technology stack (see Figure 1).

As a consequence, responsibilities and dependencies, both within and between teams, are undergoing major shifts.

Due to the lack of a clear component ownership (represented through continuous lines in the figure above),

organizational ties between teams are subject to change, as each specific team is now working on features across

different components and is always peripherally interfering with the work of neighboring teams. Furthermore,

dependencies both between and within teams are changing as the distribution of specialized knowledge and

expertise is altered (Larman and Vodde, 2010). Henceforth, the coordination between teams in an MTS needs to be

managed more coherently as to avoid conflicts due to misalignment of work. While this new form of organizational

structure is expected to increase delivery frequency, flexibility, and customer-centricity, the above mentioned

changes in responsibilities and dependencies, and more importantly the resulting effects on the organizational and

product structure, are still widely unknown. This issue is of particular interest when taking into consideration the

fact that the product structure is characterized by existing legacy code (Coplien and Harrison, 2004). Yet, to

successfully deliver a product in scope and in time, these dependencies require all actors in the development

organization to coordinate and adapt (Strode, Hope, Huff and Link, 2011).

Previous research in organizational design suggests that organizational structures, such as the division of labor, e.g.

task assignments or organizational and communication links, influence the structure of the developed product, i.e.

the technical architecture and its dependencies between design components (Colfer and Baldwin, 2010;

MacCormack, Baldwin and Rusnak, 2008; Sosa, Eppinger and Rowles, 2004). The reported congruence between

these structures does not imply any specific directionality, and evidence exists for both directions of influence as

well as for a reciprocal influence. Furthermore, several studies have shown that this mirroring relationship between

organizational and product structure is desirable, however, typically difficult to achieve and maintain (cf. Colfer and

Baldwin, 2010). So far, this research domain has mostly perceived organizational structures as links between

individual developers, i.e. looking at the phenomenon from the perspective of an individual as unit of analysis.

The increasingly important MTS structure in large-scale software development organizations has this far been

disregarded in the organizational and product alignment literature. Yet, literature on MTS and MTS coordination

describes highly complex communication and coordination processes between multiple teams, suggesting a very

different structural complexity than in a single team setting (Scheerer et al., 2014; Strode et al., 2011). The new

ways of structuring multiteam systems described above appear to disrupt the mapping of organizational and product

structure. Lacking a clearly defined responsibility of teams within an MTS for specific components, the

development organization is now characterized by a structure of shared responsibilities among multiple actors.

Changing the original task assignments and communication links which existed between specialized experts for

individual components, and instead sharing responsibilities for the latter may affect the coordination processes

within the MTS, and also weaken a sound product modularity. A strongly modularized product structure, however,

has been shown to be the source for development and coordination time reduction (Gomes and Joglekar, 2008; Yoo,

Henfridsson and Lyytinen, 2010).

Figure 2: Component Responsibility




Beyond the limited view in the organizational structure domain focusing only on links between individuals, research

in this area has so far been conducted independent of time. Nonetheless, the alignment between organizational and

product structure is to be understood as a continuous evolutionary process of change, which motivates researchers to

call for longitudinal studies assessing the effects of this alignment process (Colfer and Baldwin, 2010; Sosa et al.,

2004).

In sum, there appear to be important but theoretically unresolved effects between the described structures. We view

the concept of structural (in)congruence as the (mis)match between organizational structures, including knowledge

and task dependencies, and product structures, including technical dependencies. Embedded in the overarching

research question of how organizational and product structure realign after a disruption of congruence, we want to

find out how and why the coordination effectiveness within multiteam systems in large-scale agile software

development organizations is affected by the structural incongruence between the organizational and product

domain. To do so, we integrate literature on organizational structure, particularly on MTS and team composition,

and literature on product structure. Furthermore, pursuing a process-theoretic approach, one objective is to collect

rich and insightful data to propose a process model representing the structural alignment between the organization,

respectively an MTS, and the according software product under development. We aim to extend the understanding

of time-variant dependencies within the organizational and product domain in large-scale development and the

resulting coordination effectiveness by investigating it as a multilevel phenomenon in a real-life case study setting.

We are engaged in a research project with a large international software vendor which provides us with access to

extensive data from multiteam systems, not only in terms of their organizational but also of their product structure.

Embedding our study in this industry context adds to the practical relevance and further offers promising new

theoretical insights.

THEORETICAL FOUNDATION

Structural Alignment

The mirroring hypothesis, or Conway’s Law (Conway, 1968) as it is known in computer science, predicts an

alignment between organizational and (software) product structure. According to Colfer and Baldwin (2010), it

describes the congruence between the technical architecture, the division of labor and the division of knowledge in a

complex system. The technical architecture depicts the decomposition into specific software components and the

technical dependencies between those (Baldwin and Clark, 2000). The division of labor consists of the task

allocation to people or teams as well as the organizational ties, e.g. communication links or work flows between

actors. This division of labor results in and can be illustrated as task dependencies. The division of knowledge is

represented by the allocation of information to teams and how it is shared between them, i.e. knowledge

dependencies (Strode and Huff, 2012). As mentioned earlier, previous research postulates that this alignment, or

mirroring relationship between the organizational and the product structure, which eventually leads to a state of

structural congruence, is desirable, however, typically difficult to achieve and maintain in heterogeneous

environments (cf. Colfer and Baldwin 2010).

Organizational Structure

The previously mentioned organizational ties are defined by the way in which the software development system is

set up. The large-scale context shows particular challenges as large groups of people need to be coordinated, which

usually results in a hierarchical team of teams setup (Larman and Vodde, 2008) where several teams have to work

closely together in order to release a single software product.

Multiteam Systems

The previously described organizational setup has been defined as a multiteam system (MTS) by Mathieu et al.

(2001, p. 290) who state that MTSs are “two or more teams that interface directly and interdependently in response

to environmental contingencies toward the accomplishment of collective goals”. The collective goal of such systems

can be broken down into a goal hierarchy and constitutes a key characteristic of any MTS. The goal hierarchy marks

the boundary of an MTS in that all teams within the system share at least a distal goal while the individual teams

pursue their more proximal goals. This structure of goals leads to teams displaying input, process and outcome

interdependence with at least one other team (Mathieu et al., 2001). Previous studies have viewed the teams in MTS




research as a black box. Delving into knowledge dependencies between teams, however, requires looking into those

specific teams more closely.

Team Composition

Just like a larger MTS, a single team can be organized in different ways. Being seen as one predictor of the

important outcome variable team performance (Levine and Moreland, 1990), team composition is a heavily

investigated research area – quite contrary to the rather new field of MTS research. As opposed to surface-level

differences such as demographic attributes, deep-level differences, e.g. team cognition, skill and knowledge

distribution, promote an increasing heterogeneity among a team (Harrison, Price and Bell, 1998; Mannix and Neale,

2005). In line with this, Bunderson and Sutcliffe (2002) found dominant function diversity, i.e., the extent to which

team members differ in individual functional areas of expertise, intra- and interpersonal functional diversity, to

influence communication as well as coordination. That is, team communication and coordination are strongly

influenced by individuals being narrow functional specialists or broad functional generalists.

Extant research on coordination, i.e. the management of dependencies between individuals or teams (Malone and

Crowston, 1994; Mintzberg, 1979), as well as on coordination effectiveness (Strode et al., 2011) differs not only in

the respective unit of analysis, but also in its level of maturity (Scheerer et al., 2014). Furthermore, the effects of

dependencies, be it knowledge or task dependencies (Malone and Crowston, 1994; Strode and Huff, 2012;

Thompson, 1967), may not be generalized from the team to the MTS level without reflection.

Product Structure

The product structure, in our case a software product, is depicted within the software architecture. Here, the software

system is decomposed into components and linkages between them. Several architectural styles and patterns have

been developed over the years (cf. Clements et al. 2010; Taylor et al. 2009). In general, one can differentiate

between integral and modular architectures. Integral architectures exhibit considerable overlap between functional

elements and physical components, and display a tight coupling between interfaces (Ulrich, 1995). This leads to the

state that a change in one part of the product affects other parts (Yoo et al., 2010). A defining property of modular

architectures are the standardized interfaces between components which allow a system to be decomposed into

individual components that can be recombined (Schilling, 2000). The technical dependencies between these

components signify the relationship in the form that a change in component 1 may lead to a change in component 2

(Baldwin and Clark, 2000).

Summary and Initial Framework

We integrate existing literature on organizational and product structure, as well as literature on their alignment with

the aim to carve out the importance of task, knowledge and technical dependencies. Figure 3 illustrates these

inherent dependencies within the organizational and product structure. By visualizing them, our initial framework

provides the basis for a subsequent process model, which will be one contribution of our longitudinal study. Such a

process model will include specific events, which we will extract from our collected case study data. These

triggering events will then cause an alignment process to happen, which will result in an outcome of differing states

of the previously explained structural congruence. This process outcome shall then be linked to the coordination

effectiveness of the respective MTS.




Figure 3: Organizational and Product Structure Dimensions

RESEARCH DESIGN

We follow a multi-stage approach conducting research at one large enterprise software vendor. The company is

currently changing its business model from selling licenses for on-premise software installations to providing cloud-

based solutions. Thus, several organizational units are changing their structure from a component-focus to a feature-

focus. The organization has implemented a large-scale agile development approach to handle the increasing

complexity and foster flexibility. This development approach builds upon agile project management frameworks,

such as the Scrum approach (Larman and Vodde, 2008; Schwaber and Beedle, 2001), which is practiced throughout

the company and suggests task management through prioritized backlogs.

We divided our data collection into two phases. First, two of the authors have been immersed in the company over a

period of more than two years with several on-site visits to the company's headquarters per week. Based on their

experience coming from unstructured interviews, informal conversations and a general exploration of the software

development activities, we conducted three case studies with single development teams (Miles and Huberman, 1994;

Yin, 2009). These teams were chosen according to the organizational structure of the particular MTS they were

embedded in. Here, we purposefully selected three different MTSs with clearly differing organizational setups

regarding their component-, respectively their feature-orientation, to analyze structural effects on the individual

development team. The goal of these team-level studies was to gain initial insight into effects of incongruences at

different levels of the organizational structure. The first team worked in a traditional component setup, being

responsible for one component, team 2 integrated and orchestrated functionality delivered by other teams and team 3

developed end-to-end features across different components in the technology stack. We followed the three teams to

retrospectively find out where incongruences led to inhibitions of their daily work as evidenced by so called

disruptive events (Kude, Bick, Schmidt and Heinzl, 2014; Strong and Volkoff, 2010). Data collection took place

during summer 2013 and included 25 interviews and eleven observations of team meetings. Semi-structured and

structured interviews were conducted with each team's product owner and scrum master as well as six developers

per team, each taking about 45 minutes. In addition, open interviews with twelve senior developers, team leaders

and software architects from other teams helped us better understand the three teams' product contexts. Based on our

data, we analyzed 57 events that we used to detect structural incongruences and resulting adaptive behavior which

we conceptualized as the structural realignment across teams and products.

Based on these explorative pre-study results, we were able to select the particular MTS of one of the studied teams

which promised most revelatory insights into a structural realignment process. MTS ALPHA consists of six

development teams with more than 50 individuals and has very recently started to move from a component- to a

feature-based organizational structure. We have the opportunity to closely accompany and analyze this MTS during

the next three years to conduct a longitudinal case study. Such a longitudinal design is in line with prior multilevel




research and helps capture not only events and effects that become apparent rapidly but also ones that emerge in the

course of action and which are not immediately observable (Lapointe and Rivard, 2005; Strong and Volkoff, 2010).

Taking a process-view, we aim to focus on the changes emerging from the division of labor and knowledge, as well

as the changes in software architecture to better understand their interrelations and thus observe alignment. In more

detail, we will be able to analyze data coming from various sources such as interviews, existing documentation,

observation of teams in ALPHA, and historic and current data from the MTS’s project management system. This

diverse body of data will allow us to gain a multi-faceted picture of the observable events that accompany the

process of realignment between organizational and product structure and will help us gain insight into effects on the

coordination effectiveness of the teams within the respective MTS.

INITIAL INSIGHTS

Our preliminary findings from the pre-study provide interesting insights into the effects of structural incongruences

on development teams. We found team 3 to face a much higher number of disruptive events that were also more

severe than in the other teams. These disruptions also spanned a much wider variety of technological aspects. They

ranged from breakdowns after an update of graphical user interface components to externally introduced bugs in

some important database components. As a consequence, this team was much more often unable to resolve such

issues internally than the two other observed teams. Thus, work came to a halt more often and team 3 had to fall

back to time-consuming escalation mechanisms on the MTS-level. Before analyzing these events in more detail, we

needed to closely examine the working context of team 3. From interviews with managers and the team members we

learned that the MTS had been restructured about one year before. After following the typical component-oriented

MTS structure for years, a management decision resulted in a restructuring of the organizational unit into feature-

oriented teams. Apart from breaking up existing teams and with it the established division of labor, i.e.

organizational ties and communication links, the division of knowledge within and between the teams was

redistributed entirely. Furthermore, the historic component ownership came to a halt and was replaced by shared

responsibilities across various components. Based on these insights, we came to understand that this induced

structural incongruence between the existing legacy code and the restructured MTS resulted in responsibility and

communication conflicts between the teams which further led to decreased coordination effectiveness within the

whole MTS.

Team 3 was required to develop features end-to-end across the technology stack while the existing software

architecture was traditionally organized along single components. As peer teams had to work on the same

components without knowing much about each other’s activities, they frequently produced conflicts in each other’s

work: they changed parts of components in accordance with their own feature but thereby disrupted functions that

other teams relied on. What is more, the dependency on a high number of different components prevented the team

members from developing deep and specialized expertise in single components but forced them to become

superficially knowledgeable about various parts of the technology stack. As a result, it became impossible to gain

sufficient expertise to resolve issues rooted in the depth of a single component internally.

By contrast, team 1 neither experienced as severe disruptive events caused by the product structure nor were its

reactions nearly as strong. Although this may well be attributed to various other circumstances, the organizational

context of team 1 – namely the MTS ALPHA – was of a much more stable character. From interviews with the

MTS’s management, however, we learned about the intention to restructure the MTS from a component- to a

feature-oriented setting. Giving us access to their entire MTS provides us with the unique opportunity to conduct a

longitudinal case study promising very rich data resulting from this structural realignment process.

EXPECTED CONTRIBUTION AND OUTLOOK

Adopting a multilevel perspective on organizational and product structure, our goal is to understand the resolution of

structural incongruences and the effects on coordination effectiveness of multiteam systems. Our preliminary results

show that there are highly relevant but theoretically disregarded effects caused by such structural incongruences.

While these initial and exploratory results provide useful insights into the team-level effects, our next step will be to

conduct a longitudinal case study with one particular MTS at our industry partner. This will allow us to examine the

intermediate outcome variable coordination effectiveness on a multiteam-level and look at entailing effects on

performance indicators such as the time to market. This enables us to better understand the effects of structural

incongruences and potential realignment over time through adaptation processes.




Doing so, we aim to contribute our insights from a longitudinal case study to the body of knowledge on

organizational and product structure alignment. Answering the call for longitudinal research on this topic, we expect

to find out more about the effects of intentionally caused structural incongruences on MTS coordination

effectiveness over the course of time. Accompanying one particular MTS in the setting of a large-scale software

development organization during the change from one organizational setup to another, we will collect rich data on

the shift of task, knowledge and technical dependencies. Whereas recent work in MTS research has so far

conceptualized single teams as black boxes, we seek to investigate the role of team composition within an MTS in

more detail by looking at the division of labor and the division of knowledge at the team level. Initial insights in this

multilevel research study indicate that a redistribution of knowledge and a breakup of organizational ties within

individual teams can have severe effects on the overall effectiveness of the entire MTS.

Further, we aim to develop a process model which depicts the alignment process of the organizational and the

product structure based on the shift of dependencies between and within teams and their respective product domain.

Here, the goal is to identify triggering events, such as for instance a responsibility conflict, and further carve out

potential outcome patterns of this alignment process.

BIBLIOGRAPHY

Baldwin, C. Y. and Clark, K. B. (2000). Design Rules: The Power of Modularity (1st ed.). The MIT Press.

Bunderson, J. S. and Sutcliffe, K. M. (2002). Comparing Alternative Conceptualizations of Functional Diversity in

Management Teams: Process and Performance Effects. Academy of Management Journal, 45, 5, 875-893.

Clements, P., Bachmann, F., Bass, L., Garlan, D., Ivers, J., Little, R., … Nord, R. L. (2010). Documenting Software

Architectures: Views and Beyond (2nd ed.). Addison Wesley.

Colfer, L. and Baldwin, C. Y. (2010). The Mirroring Hypothesis: Theory, Evidence and Exceptions (No. 10-058).

Harvard Business School, 1-39.

Conway, M. E. (1968). How do committees invent. Datamation, 14, 4, 28-31.

Coplien, J. O. and Harrison, N. B. (2004). Organizational Patterns of Agile Software Development. Prentice Hall.

Dillon, T., Wu, C. and Chang, E. (2010). Cloud Computing: Issues and Challenges. In IEEE International

Conference on Advanced Information Networking and Applications, 27-33.

Gomes, P. J. and Joglekar, N. R. (2008). Linking Modularity with Problem Solving and Coordination Efforts.

Managerial and Decision Economics, 29, 5, 443-457.

Harrison, D. A., Price, K. H. and Bell, M. P. (1998). Beyond Relational Demography: Time and the Effects of

Surface- and Deep-Level Diversity on Work Group Cohesion. Academy of Management Journal, 41, 1, 96-

107.

Keith, M., Demirkan, H. and Goul, M. (2013). Service-Oriented Methodology for Systems Development. Journal of

Management Information Systems, 30, 1, 227-260.

Kude, T., Bick, S., Schmidt, C. and Heinzl, A. (2014). Adaptation Patterns in Agile Information Systems

Development Teams. In 22nd European Conference on Information Systems (ECIS), Tel Aviv, 1-15.

Lapointe, L. and Rivard, S. (2005). A Multilevel Model of Resistance to Information Technology Implementation.

MIS Quarterly, 29, 3, 461-491.

Larman, C. and Vodde, B. (2008). Scaling Lean & Agile Development: Thinking and Organizational Tools for

Large-Scale Scrum (1st ed.). Addison-Wesley Professional.

Larman, C. and Vodde, B. (2010). Practices for Scaling Lean & Agile Development Large, Multisite, and Offshore

Product Development with Large-Scale Scrum (1st ed.). Upper Saddle River, NJ: Addison Wesley.

Levine, J. M. and Moreland, R. L. (1990). Progress in Small Group Research. Annual Review of Psychology, 41,

585-634.

MacCormack, A., Baldwin, C. and Rusnak, J. (2008). Exploring the duality between product and organizational

architectures: A test of the “mirroring” hypothesis. Research Policy, 41, 8, 1309-1324.

Malone, T. W. and Crowston, K. (1994). The Interdisciplinary Study of Coordination. ACM Computing Surveys, 26,

1, 87-119.

Mannix, E. and Neale, M. A. (2005). What Differences Make a Difference? The Promise and Reality of Diverse

Teams in Organizations. Psychological Science in the Public Interest, 6, 2, 31-55.

Mathieu, J. E., Marks, M. A. and Zaccaro, S. J. (2001). Multiteam Systems. In N. Anderson, D. S. Ones, H. K.

Sinangil and C. Viswesvaran (Eds.), Handbook of Industrial, Work and Organizational Psychology, Vol. 2,

London: Routledge, 289-313.




Miles, M. B. and Huberman, A. M. (1994). Qualitative Data Analysis: An Expanded Sourcebook. Thousand Oaks:

Sage Publications.

Mintzberg, H. (1979). The Structuring of Organizations. Englewood Cliffs. Prentice-Hall.

Scheerer, A., Hildenbrand, T. and Kude, T. (2014). Coordination in Large-Scale Agile Software Development: A

Multiteam Systems Perspective. In 47th Hawaii International Conference on System Sciences (HICSS),

Waikoloa, HI, 4780-4788.

Schilling, M. A. (2000). Toward a General Modular Systems Theory and Its Application to Interfirm Product

Modularity. Academy of Management Review, 25, 2, 312-334.

Schwaber, K. and Beedle, M. (2001). Agile Software Development with Scrum. Prentice Hall.

Shetty, S. (2013). Gartner Says Cloud Computing Will Become the Bulk of New IT Spend by 2016. Gartner Report.

Retrieved July 25, 2014, from http://www.gartner.com/newsroom/id/2613015

Sosa, M. E., Eppinger, S. D. and Rowles, C. M. (2004). The Misalignment of Product Architecture and

Organizational Structure in Complex Product Development. Management Science, 50, 12, 1674-1689.

Strode, D. E., Hope, B., Huff, S. L. and Link, S. (2011). Coordination Effectiveness In An Agile Software

Development Context. In PACIS Proceedings.

Strode, D. E. and Huff, S. L. (2012). A taxonomy of dependencies in agile software development. In 23rd

Australasian Conference on Information Systems (ACIS), Geelong, 1-10.

Strong, D. M. and Volkoff, O. (2010). Understanding Organization - Enterprise System Fit: A Path to Theorizing the

Information Technology Artifact. MIS Quarterly, 34, 4, 731-756.

Stuckenberg, S., Kude, T. and Heinzl, A. (2014). Understanding the role of organizational integration in developing

and operating Software-as-a-Service. Journal of Business Economics, February, 1–32.

Subramanyam, R., Ramasubbu, N. and Krishnan, M. S. (2012). In Search of Efficient Flexibility: Effects of

Software Component Granularity on Development Effort, Defects, and Customization Effort. Information

Systems Research, 23, 3, 787-803.

Taylor, R. N., Medvidovic, N. and Dashofy, E. (2009). Software Architecture: Foundations, Theory, and Practice

(1st ed.). John Wiley & Sons.

Thompson, J. D. (1967). Organizations in Action: Social Science Bases of Administrative Theory. McGraw-Hill.

Ulrich, K. (1995). The role of product architecture in the manufacturing firm. Research Policy, 24, 3, 419-440.

Yin, R. K. (2009). Case Study Research: Design and Methods (4th ed.). SAGE Publications Inc.

Yoo, Y., Henfridsson, O. and Lyytinen, K. (2010). Research Commentary - The New Organizing Logic of Digital

Innovation: An Agenda for Information Systems Research. Information Systems Research, 21, 4, 724-735.

9th pre-icis international research workshop on ......9th pre-icis international research workshop...

Documents