enterprise cots implementation: process, stakeholder, and ... · csf during implementation....
TRANSCRIPT
Enterprise COTS Implementation: Process, Stakeholder,
and Control Perspective
by
Zafor Ahmed, PMP
A thesis submitted to the Faculty of Graduate and Postdoctoral
Affairs in partial fulfillment of the requirements for the degree of
Doctor of Philosophy
in
Management
,
Carleton University
Ottawa, Ontario
©2018
Zafor Ahmed
Page 2 of 307
Abstract
Cutting the cost of information technology (IT) spending is an ongoing initiative not only in private
industry, but also in national governments in an effort to balance their budgets. One key dimension
of this cost reduction effort is in optimizing procurement policies where buying and reusing are
prioritized. In the case of IT systems, buying generally means purchasing and implementing
commercially available packaged solutions, commonly known as COTS solutions.
COTS implementation projects, often part of an organization’s strategic IT initiatives, require a
large investment of organizational resources, yet often fail to achieve desired benefits. Facing a
persistently high failure rate, identifying IS implementation-related critical factors appears to be
essential for both theory and practice. Although knowledge about critical success factors (CSF) is
quite mature, an absence of benefits realization suggests a lack of effective management of relevant
CSF during implementation. Applying stakeholder theory and control theory can help develop new
perspectives in CSF research as they center around the concepts of “whose interest matters” and
“how to govern them,” respectively. The current research investigates public sector enterprise
COTS implementation from multiple perspectives, such as implementation processes, engagement
of stakeholders, and control balancing, and how these factors relate to implementation-specific
CSF. This enhances our current understanding of implementation processes, stakeholder
engagement, and control balancing related to enterprise COTS implementations.
The current research draws from control theory, control balancing theory, stakeholder theory,
stakeholder agency theory, CSF literature, and IS-implementation literature to develop an
argument that asserts an integral relationship among processes, stakeholder engagement, control
balancing, and CSF. In addition to proposing a new and vital construct—stakeholder orientation—
micro-level and macro-level models for enterprise COTS implementation are identified. Control
theory and control balancing theory is then used to capture and validate the dynamic nature of
control configurations during IS implementation. Stakeholder theory and stakeholder agency
theory is further applied to capture the dynamic stakeholder orientation in a IS implementation
project. Finally, through analysis and synthesis, the current research establishes vital relationships
among processes, control configurations, stakeholder orientations, and CSF, where proper CSF
management leads to a successful COTS-implementation project outcome.
Page 3 of 307
ACKNOWLEGEMENTS The completion of this thesis is the culmination of many years of effort. While the work described
in this document is mine, particularly the shortcomings, I could not have accomplished any of it
without considerable support from a number of people and organizations.
I would like to take the opportunity to express my profound gratitude to Dr. Vinod Kumar and Dr.
Uma Kumar, my co-supervisors, for their continuous and dedicated support, advice, and help in
completing this research. Their insightful guidance and hospitality, throughout my PhD journey,
have been immense and immeasurable and I could not have imagined having a better mentor for
my PhD studies.
I also wish to thank all the other members of my Thesis Committee, in particular, Dr. Alan Cai
and Dr. Greg Sears for this major undertaking. I am very grateful and indebted to both for agreeing
to be part of the examining committee, and for providing valuable comments, advice, and
suggestions in significantly improving this research.
I wish to express my sincere gratitude to the Sprott School of Business, Carleton University, for
the benefits that I gained during my studies at the School. I also wish to thank the Administrative
Staff at the Sprott School of Business. In particular, I would like to express my sincere appreciation
to Melissa Doric for her dedicated support.
I am grateful to my parents for their support, love and encouragement. My parents always
reinforced the idea from the early years on that a solid education is something that can never be
taken away. One of the key motivator for me in pursuing a Ph. D. was my mother and for this, I
dedicate this thesis to my mother – Begum Selina Khandakar.
The responsibility for errors and omissions rests with the author
Zafor Ahmed
Page 4 of 307
Contents
1.0 Introduction .......................................................................................................................................... 8
1.1 Research motivation ........................................................................................................................ 9
1.2 Research objectives ........................................................................................................................ 15
1.3 Research questions ......................................................................................................................... 16
1.4 Case study approach ..................................................................................................................... 17
1.5 Organization of this thesis ............................................................................................................ 19
2.0 Literature review ................................................................................................................................ 21
2.1 Selection process for articles ......................................................................................................... 22
2.2 COTS and IS implementation ....................................................................................................... 23
2.2.1 An agile implementation........................................................................................................ 34
2.2.2 The agile manifesto and agile values ................................................................................... 35
2.3 Stakeholders and stakeholder engagement ................................................................................ 37
2.4 Control configuration and control balancing ............................................................................. 45
2.5 Critical success factors (CSF) ........................................................................................................ 49
3.0 COTS implementation framework .................................................................................................. 62
3.1 Enterprise COTS and its implementation ................................................................................... 63
3.2 Stakeholder engagement in COTS implementation .................................................................. 69
3.2.1 A Framework towards stakeholder engagement ............................................................... 71
3.2.2 Stakeholder sensitivity and stakeholder engagement ....................................................... 73
3.3 Control balancing in COTS implementation .............................................................................. 77
3.4 Mapping the research questions and objectives ........................................................................ 81
4.0 Research methodology ...................................................................................................................... 84
4.1 Overall approach ............................................................................................................................ 84
4.2 Research design .............................................................................................................................. 84
4.3 Research method ............................................................................................................................ 88
4.3.1 Review literature ..................................................................................................................... 89
4.3.2 Synthesize literature ............................................................................................................... 90
4.3.3 Map research questions .......................................................................................................... 91
4.3.4 Design interviews.................................................................................................................... 91
Page 5 of 307
4.3.5 Select sample ............................................................................................................................ 92
4.3.6 Conduct interviews ................................................................................................................. 96
4.3.7 Analyze data ............................................................................................................................ 97
4.4 Reliability and validity of the cases ........................................................................................... 104
4.4.1 Reliability ............................................................................................................................... 104
4.4.2 Validity ................................................................................................................................... 104
5.0 Case results and discussion ............................................................................................................ 106
5.1 Case A: Replacing an enterprise system for financial market operations ........................... 106
5.1.1 Implementation process for Case A ................................................................................... 109
5.1.2 Control configuration and balancing ................................................................................. 114
5.1.3 Stakeholder engagement and sensitivity ........................................................................... 120
5.2 Case B: Replacing a shared system for financial regulations ................................................ 125
5.2.1 Implementation process for Case B .................................................................................... 129
5.2.2 Control configuration and balancing ................................................................................. 135
5.2.3 Stakeholder engagement and sensitivity ........................................................................... 139
5.3 Case C: Financial billing systems replacement ........................................................................ 144
5.3.1 Implementation process for Case C .................................................................................... 147
5.3.2 Control configuration and balancing ................................................................................. 153
5.3.3 Stakeholder engagement and sensitivity ........................................................................... 158
5.4 Case D: Enterprise content management (ECM) ..................................................................... 164
5.4.1 Implementation process for Case D ................................................................................... 168
5.4.2 Control configuration and balancing ................................................................................. 175
5.4.3 Stakeholder engagement and sensitivity ........................................................................... 181
6.0 Cross-Case Analysis and Synthesis ............................................................................................... 186
6.1 Enterprise COTS Implementation Model ................................................................................. 186
6.2 Control configurations and control balancing ......................................................................... 211
6.3 Stakeholder orientations ............................................................................................................. 229
6.3.1 Responsibility dimension (normative orientation) .......................................................... 233
6.3.2 Paternalism dimension (normative orientation) ............................................................... 235
6.3.3 Neoclassical dimension (instrumental orientation) ......................................................... 237
6.3.4 Strategic dimension (instrumental orientation) ................................................................ 241
Page 6 of 307
6.4 Processes, control balancing, stakeholder orientation and CSF ............................................ 246
7.0 Conclusion ........................................................................................................................................ 259
7.1 Conclusion .................................................................................................................................... 259
7.2 Summary of answers to Research Questions ........................................................................... 260
7.2.1 What is an activity- and process-based model for enterprise COTS implementation?
.......................................................................................................................................................... 260
7.2.2 How can organizational processes and tools contribute to COTS implementation
success? ............................................................................................................................................ 262
7.2.3 What is the nature of control configuration in a multi-partner COTS implementation
and what are the factors responsible for the application of control balancing? ................... 264
7.2.4 How can CSF and other challenges be successfully managed through optimal control
balancing by the project? ............................................................................................................... 265
7.2.5 What is the nature of stakeholder orientation during a COTS implementation for a
public sector IS implementation? ................................................................................................. 266
7.2.6 How can CSFs and other challenges be successfully managed by the project through
optimal stakeholder engagement? ............................................................................................... 268
7.3 Implications for theory ................................................................................................................ 269
7.4 Implications for Practice.............................................................................................................. 272
7.5 Limitation of Current Research ............................................................................................ 275
7.5.1 Firms sampled and generalizability ............................................................................ 275
7.5.2 Interview method ........................................................................................................... 275
7.5.3 Granularity of the stakeholder orientation framework ............................................ 276
7.6 Future Research Directions ................................................................................................... 276
7.6.1 Agile implementation process of enterprise COTS implementation ...................... 276
7.6.2 Trigger factors for shifting stakeholder orientations for COTS implementation .. 277
7.6.3 Enacting Clan control and self control on the development team .......................... 277
7.6.4 Effectiveness of stakeholder orientation and balancing isomorphic pressure ...... 278
References ............................................................................................................................................... 280
Appendix A – Research Approval ....................................................................................................... 291
1. Ethics Committee Approval ..................................................................................................... 291
2. Organizational Approval .......................................................................................................... 292
Appendix B - Interview Guide ............................................................................................................. 292
1. Email to Potential Interview Candidates ................................................................................ 292
Page 7 of 307
2. Informed Consent ...................................................................................................................... 293
3. Interview Protocol ...................................................................................................................... 295
Appendix C – Informant List and Details .......................................................................................... 295
1. Case A Informant list: ................................................................................................................ 295
2. Case B Informant list: ................................................................................................................ 296
3. Case C Informant list: ................................................................................................................ 296
4. Case D Informant list: ................................................................................................................ 297
Appendix D – Strategic Investment Gating Process ......................................................................... 298
Appendix E ............................................................................................................................................. 302
Appendix F: Acronym Table ................................................................................................................ 302
Appendix G: Data Collection Instrument ........................................................................................... 303
Page 8 of 307
1.0 Introduction
Deployment of COTS (Commercial Off The Shelf) as enterprise IT (Information Technology)
have steadily increased over last two decades, ranging from multi-million dollar ERP
implementation to a stand-alone packaged software for individual users. This trend is apparent in
both the public and the private sectors where reduction of IT expenditure and speed of deployment
are two of the salient drivers of IT procurement decisions (Chidley, 2014). Cutting the cost of
information technology (IT) spending is an ongoing initiative not only in private industry but also
at the federal government level because governments in different countries are trying to cut costs
to balance their budget. For example, in the attempt to cut IT spending across various departments,
the Canadian government has implemented IT streamlining initiatives that include the
consolidation of data centers across departments and the reduction of various redundant systems
(Data Center Consolidation Initiatives, 2015). One of the key dimensions of this cost reduction
effort is optimizing the procurement policies where buying and reusing (as opposed to building)
are prioritized. In the case of computer software, buying generally means purchasing and
implementing commercially available packaged solutions, commonly known as COTS solutions.
Often these COTS implementation projects are part of an organization’s strategic IT initiatives
that require a large investment of organizational resources, yet they fail to achieve the desired
benefits. This should not be surprising as the failure rate of information systems (IS) development
and implementation projects are still disturbingly high. Depending on the source, this failure rate
can be found to be as high as 70% (Doherty, Ashurst, and Peppard, 2011). Although a substantial
amount of research has been conducted to identify technological and organizational factors that
should be managed properly to successfully complete the projects and deliver the expected
Page 9 of 307
benefits, most of this effort is focused on the domain of well-known COTS (Cecez-Kecmanovic,
Kautz, and Abrahall, 2014). Furthermore, given the persistence of such high failure rates, benefit-
realization from identifying these IS implementation related critical factors are largely absent in
practice (Cobb, 1996; Doherty, Ashurst, and Peppard, 2011). Since the literature and knowledge
on critical success factors (CSF) are quite mature, an absence of benefits realization can be
explained through a lack of effective management of those identified CSF. Although a significant
percentage of the CSF literature primarily attempts to identify specific CSF related to different IS
implementation or context, a closer look reveals a ‘people’ aspect related to most of the identified
CSF. Therefore, employing theoretical lenses of ‘stakeholder theory’ and ‘control theory’ can help
develop new perspectives in a CSF research as they center around the concepts of ‘whose interest
matters’ and ‘how to govern them’ respectively.
The purpose of this thesis is to investigate public sector enterprise COTS1 implementation from
multiple perspectives such as implementation processes, engagement of stakeholders, control
balancing, and how these factors relate to implementation specific critical success factors (CSF).
This will significantly contribute to enhancing our current understanding of implementation
processes, stakeholder engagement, and control balancing related to COTS implementations.
1.1 Research motivation
COTS implementation differs from traditional software development projects or enterprise
systems implementation projects in several key aspects, such as the required level of functional
1 Enterprise COTS serve the needs of an enterprise as opposed to a single department, user or specialized application
Page 10 of 307
and technical expertise available within an organization (Uçar and Bilgen, 2013), the level of
integration complexities, the degree of engagement by the vendor, and the degree of control over
the vendor and the project partners. Additionally, the actual implementation process is often found
to be different from the documented processes observed in the literature (Torchiano and Morisio,
2004; Li et al., 2009; Jadhav and Sonar, 2009). This gap between theory and practice has
effectively contributed to a higher risk level and uncertainty surrounding enterprise COTS
implementation ( Birkmeier and Overhage, 2009). Although a substantial body of IS literature has
focused on examining various aspects related to the well-known categories of COTS products,
such as ERP (Enterprise Resource Planning) and CRM (Customer Relationship Management), the
implementation of non-ERP COTS has received very little attention from researchers (Ayala et al.
2011) which is the focus of this research.
While analyzing different implementation models and comparing them to the generic definition of
COTS package implementation, it becomes apparent that there is a clear gap between the existing
models found in literature and the actual implementation process that is followed. For example,
legacy system replacement would require a critical task of meta-data conversion for the new
packaged solution (Lewis, and Wrage, 2004), which is not included in any of the models we
reviewed. Some of the models do point out the components of glueware2 or custom code; however,
meta-data conversion and data reconciliation can be much more complex compared to the
development of custom components required to support a COTS package. Additionally, some
implementation tasks may take place during a different stage than those shown in the reviewed
models from the literature. This variation can be attributed to the maturity level of a COTS product
2 Glueware is a software solution or platform that is designed to seamlessly integrate disparate and decentralized software solutions and systems with related resources (Techopedia).
Page 11 of 307
where more mature products like ERP and CRM demonstrate a consistent pattern, and less mature
products exhibit a higher variation. For example, the model derived by Morisio et al. (2002) for
non-ERP COTS implementation shows simultaneous activities of product identification,
evaluation, selection, and requirement analysis. Product selection and detailed requirement
analysis are two of the most critical activities of a COTS implementation project, and yet,
depending on the maturity level of the COTS product, they could be performed at different stages
of the project.
In light of above possibilities, an empirical investigation of enterprise COTS implementation
projects would significantly contribute to both theory and practice by helping to identify and
understand critical implementation phases, activities, processes, and tools that determines the
implementation success.
The identification of a micro-level or detailed enterprise COTS implementation model, capturing
tools, processes, and procedures, creates the opportunity for investigating several other related
aspects. For a multi-partner and multi-vendor implementation effort in the public sector, elements
of social responsibility and control can be of particular interest to both researchers and
practitioners. Generally, as the number of stakeholder increases in a project, control structure also
becomes complex. Accountability for a public-sector organization often includes the social
responsibility aspect. More specifically, the management of critical success factors (CSF) through
the adopting of appropriate control configurations and ensuring optimal stakeholder engagement
can be a significant contribution in relation to both communities. Even the research concentrating
on the individual aspects of Information Systems (IS) implementation – such as CSF identification,
implementation process, or application of various controls – is quite substantial, but largely non-
existent are investigations of CSF management by optimally engaging stakeholders and balancing
Page 12 of 307
control by applying different control configurations (Finney and Corbett, 2007). Enterprise COTS
implementation projects are often part of an organization’s strategic IT initiatives and require a
large investment of organizational resources, yet often fail due to the inadequate management of
CSF and other implementation-specific challenges. Regardless of the large body of existing
research concentrating on CSF, the sheer volume of unsuccessful implementation efforts clearly
indicates the need to adopt a different theoretical lens to research this issue. Therefore, utilizing
stakeholder theory and the theory of control balancing, to examine the implementation of
enterprise COTS and the management of CSF during the implementation process, holds much
promise.
Stakeholder engagement
A major aspect of the current thesis centers around the organizational stakeholder orientation
during an enterprise COTS or IS implementation. The concept of stakeholder engagement is
becoming increasingly popular among researchers and management practitioners of information
systems (IS) to explain the diverse outcomes associated with the implementation of new
technology. From an in-depth analysis of 400 strategic decisions, including technology
acquisitions and strategic reorganizations, Nutt (2002) reported an overwhelming 50 percent
failure ratio. He primarily classified “failure” as aborting an IS project, partial implementation, or
a failure to produce expected results. Interestingly, most of the failures were attributed to decision-
makers’ inability to integrate and engage relevant stakeholders. This finding is also in agreement
with the earlier literature stressing the significance of stakeholder interests (Bryson et al. 1990;
Bryson and Bromiley, 1993; Burby, 2003; Margerum, 2002). Bryson (2004) indicated that an
inability to synthesize information possessed by various stakeholders and the failure to address
their concerns while making decisions are major flaws in thinking and action that can lead to
Page 13 of 307
failure. On the other hand, evidence of positive outcomes through cautious stakeholder
engagement is also evident in the literature. Aside from supporting issue-legitimization and
facilitating closer alignment between organization and society, the positive impacts of stakeholder
opinions on decision-making and project implementation processes are well established in the
literature (Deelstra, Nooteboom, Kohlmann, Van den berg, and Innanen, 2003). Several other
studies have also identified stakeholder engagement as an effective organizational strategy and a
means to improve external stakeholder relations, often used in the public and private sectors (Kivit,
2011). Despite numerous studies indicating the strategic significance of stakeholder engagement,
research on enterprise COTS implementation appears to be largely missing this focus. Lorenzo
(2004), through a comprehensive review of the enterprise systems (ES) research, identified a high
risk of ES implementation failure at the project phase (Buckhout et. al. 1999; Scott 1999;
Davenport 1998), diffusion phase (Shepherd, 2001; James and Wolf, 2000; Gilbert, 1999) and the
benefit realization phase (Shepherd, 2001; Markus and Tanis, 2000; Davenport, 1998). In addition,
the categorization of existing ES implementation research indicates four salient trends: critical
success factors, measuring success, descriptive case studies, and long-term challenges (Lorenzo,
2004). Finney and Corbet (2007) have also raised similar concerns while examining ERP literature,
stating that much of it has focused on critical success factors with very limited or no regard to the
stakeholder perspective. An intimate understanding of various stakeholder groups would make it
possible to effectively address the challenges related to the project phase, the diffusion phase, and
the benefit realization phase, thus enhancing the overall probability of successful ES/IS
implementation (Finney and Corbet, 2007). Therefore, a necessity for understanding the nature of
stakeholder orientation and filling the existing knowledge gap in this area are two primary
motivations for adopting a stakeholder lens for the current thesis.
Page 14 of 307
Control balancing
Another salient aspect of the current thesis revolves around the concept of control balancing during
an enterprise COTS or IS implementation. Effectively controlling project and relationships is one
of the most critical factors for the success of an Information Systems (IS) implementation project
(Gregory, Beck, and Keil, 2013). Leimeister, Heusler, and Krcmar (2010) also voiced a similar
concern that the equilibrium of coordination and control mechanisms are absolutely essential for
an effective IS project governance. A review of the control literature indicates a dominance of
control types and their application in a vertical relationship structure. However, the concept of
control balancing is a much recent one, and its application in a horizontal relationship structure
has received very little attention (Gregory et al. 2013).
The concept of control involves a controller (the person exercising control) and a controllee (the
target of control). Numerous empirical studies have investigated control in an organizational or IS
project context (Kirsch, 1997, 2004). However, a majority of the studies related to control theory
has focused on the modes of control comprised of two broad categories: formal and informal
(Crisp, 2002). Formal control is typically documented and initiated by the management. Informal
control is usually comprised of undocumented measures initiated by the employee themselves
(Jaworski, 1988), and are commonly identified as self-control and clan control (Ouchi, 1980). A
less prevalent aspect of control theory is “control configuration” which is distinct from the concept
of control (Gregory et al. 2013). Gregory et al. (2013) defined control configuration as a
combination of control type, control degree, and control style. They have also conceptualized
control balancing as the act of making targeted adjustments to an existing control configuration.
Page 15 of 307
An enterprise COTS implementation project involving multiple clients and vendors possesses both
horizontal and vertical relationship structures. As the current thesis includes COTS
implementation cases possessing both horizontal and vertical relationship structures, findings from
our analysis of selected cases revealed valuable insight concerning the application of control
balancing in a multiple stakeholder context.
1.2 Research objectives
Enterprise COTS implementation is an inherently challenging task, and a public-sector context for
such an implementation can be a source of very rich information with much potential for
contribution to the understanding of the issue. Therefore, the objectives pertaining to the current
thesis are following:
1. To identify a micro-level or detailed model for enterprise COTS implementation by
focusing on implementation phases, processes, and tools. Additionally, a micro-level
analysis can also help reveal the valuable information flow patterns among the
implementation phases.
2. To identify a set of CSFs that are relevant for enterprise COTS implementation and
examines the management aspect of them through process, stakeholder engagement and
control balancing. Understanding and managing CSFs are critical aspects of any IS
implementation project. CSFs have been a dominant research theme in ERP
implementation research, and similar to ERP implementation, managing CSFs is equally
important for any non-ERP COTS implementation project.
Page 16 of 307
3. To develop a theoretically sound framework that can be used to identify an organization’s
stakeholder orientation during an enterprise COTS implementation project. Stakeholder
engagement has often been a blurred area in much package-software implementation
research (Finney and Corbett, 2007), despite the potential to explain many related
consequences. Therefore, developing a framework to identify an organization’s
stakeholder orientation during an enterprise COTS implementation would be a significant
contribution in this domain.
4. To validate the theory of control balancing (Gregory, Beck, and Keil, 2013) by examining
the dynamic relationships among the project partners from one phase to another phase of
the implementation process. The concept of control has been previously subject to
numerous empirical investigations in the context of IS implementation. However, control
balancing, a related concept, has not received sufficient attention. Understanding the nature
of control balancing, from the perspective of what triggers a targeted adjustment of control
portfolio, shows significant potential for contributing to the literature.
1.3 Research questions
Through this thesis, we intend to investigate multiple aspects of an enterprise COTS
implementation process. Key dimensions of the current research are related to the following
domains:
1. Implementation models, processes and tools related to COTS implementation projects;
2. The dynamic nature of control balancing and control configuration related to a COTS
implementation;
3. Identifying an organization’s stakeholder orientation during a COTS implementation; and
Page 17 of 307
4. Relationships among CSFs, control balancing and stakeholder engagement.
Our research questions are:
1. What is an activity-based and process-based model for enterprise COTS implementation?
2. How can organizational processes and tools contribute to COTS implementation
success?
3. What is the nature of control configuration in a multi-partner COTS
implementation and what are the factors responsible for the application of control
balancing?
4. How can CSFs and other challenges be successfully managed through optimal
control balancing by the project?
5. What is nature of stakeholder orientation during a COTS implementation for a public
sector IS implementation?
6. How can CSFs and other challenges be successfully managed through optimal
stakeholder engagement by the project?
1.4 Case study approach
The current thesis adopts a multiple-case-study research design to inquire about the
implementation of enterprise COTS packages. Usually, when the topic of research concerns
contemporary events and investigation requires no behavioural control, a case study method is
better suited for the investigation. According to Yin (1994), a case study is an empirical inquiry
that i) relies on multiple sources of evidence with data needing to converge, ii) benefits from the
Page 18 of 307
prior development of theoretical propositions to guide data collection and analysis, iii) investigates
contemporary phenomena in depth within its real-life context, and iv) shows no clear boundaries
between the phenomenon and the context.
The above definition reflects both the appropriateness and the urge of a case-study research for
investigating the proposed research questions concerning a contemporary phenomenon (enterprise
COTS implementation) where the investigation requires no behavioural control over the events.
In addition, a case-based research on IS implementation can produce a much richer description of
the events, and can then be used for further abstraction through other methods like grounded theory
research (Fernández and Lehmann, 2011).
Data for the current thesis have been collected through semi-structured interviews using a set of
open-ended questions that allowed the participants to describe the process of a COTS
implementation project in details, the nature of controls utilized during the project, the nature of
stakeholder engagement, and the critical success factors relevant to the project. A thorough
literature review has also been conducted to identify the state of the knowledge related to COTS
implementation. This served a twofold purpose: a) identification of knowledge gaps which helped
formulate the interview questions, and b) clarification regarding the current status of theory to
identify points of theoretical contribution or extension.
The unit of analysis for the current thesis is the implementation of an enterprise COTS, and the
total number of cases that will be analyzed through this research are four. We have interviewed 40
employees from various ranks involved with COTS implementation projects. In particular, the
projects we have selected for this research relate to the implementation of an enterprise COTS
Page 19 of 307
solution not belonging to the category of well known COTS (i.e. ERP or CRM). Additional
selection criteria included a public-sector focus, multiple clients, multiple vendors, failed
implementation, and successful implementation. The selected sample includes two cases of
successful implementation and two cases of failed COTS implementation. Although a failure is
difficult to define for a public-sector COTS implementation, we focused on the “post-
implementation client satisfaction” and “support cost” to create a distinction between successful
and failed projects. Although the use of the word “failed” may not be used at the organizational
level, primary intention of the current research is to establish that a lack of CSF management
through stakeholder engagement and control can lead to adverse outcome for a COTS
implementation.
1.5 Organization of this thesis
This thesis is divided into seven chapters.
The next chapter provides a review and synthesis of relevant literature. It starts with a review of
enterprise COTS implementation, enterprise systems (ES) implementation, and Information
Systems (IS) implementation literature. This includes both academic and practice-oriented
literature to identify the knowledge gaps with regards to the enterprise COTS implementation
process. Next, a review is undertaken of the critical success factors (CSF), stakeholder
engagement, and control literature. As much of the existing CSF research focus on identifying
context specific CSF for IS implementation, this review helps justify the need for a new theoretical
approach for studying CSFs, as well as facilitates the development of a stakeholder orientation
framework.
Page 20 of 307
Chapter 3 provides a synthesis of the preceding chapters, including the literature review, using
vital arguments to signify the existing knowledge gaps in the domain. It also elaborates the
concepts of control, stakeholder engagement, and stakeholder sensitivity. This elaboration and
synthesis pave the way to propose the current research objectives in a more tangible fashion.
Chapter 4 describes the utilized research methods in detail, including specific steps that are
followed. A discussion on the case reliability and case validity is also included in this chapter.
Chapter 5 presents the project overview and background for each COTS implementation case
selected for analysis. We also presented summarized primary data and results of initial coding in
this section.
Chapter 6 presents cross case analysis and a synthesis for all selected cases. It also presents the
results from our pattern coding of the primary data and discusses the theoretical integration of the
emerging themes that were identified through a grounded theory approach.
Chapter 7 provides a conclusion for the current research. In addition to presenting summarized
answers to each of our research questions, contributions to both theory and practice, limitations of
the current research, and future research directions are also discussed in this section.
Page 21 of 307
2.0 Literature review
This chapter reviews the relevant literature from enterprise COTS implementation, enterprise
systems (ES) implementation, and Information Systems (IS) implementation literature. In
addition, a review of critical success factors related to both ERP and non-ERP packaged software,
stakeholder engagement, and control literature has also been conducted.
Section 2.1 describes the article selection process for the current research. Specifically, this section
outlines the electronic sources and scholarly databases used for selecting peer-reviewed academic
articles. In addition, it presents the keywords used for locating articles.
Section 2.2 reviews the literature related to COTS and IS implementation. The primary goal of this
review is to identify the existing knowledge and models related to the implementation process and
the knowledge gaps when it comes to enterprise COTS. This also sets the ground for a micro-level
analysis and facilitates a connection to the concept of CSF by identifying key activities.
Section 2.3 reviews IS literature related to stakeholder theory. A guiding motivation for this review
is to examine the concept of “stakeholder orientation” and highlight the difference between
“consideration” and “engagement.” In addition, this section points out variations related to the
application of stakeholder theory, challenges related to the adoption of a stakeholder lens for COTS
implementation, and the possibility of multiple stakeholder orientations by the project team.
Section 2.4 delves into the concepts of control and control balancing associated with control
theory. The primary contribution of this review is to provide a justification for studying control
balancing in a multiparty COTS implementation context. This section also highlights the
distinction between hierarchical and vertical control in the absence of a direct controller-controllee
relationship.
Page 22 of 307
Section 2.5 reviews the literature on critical success factors (CSFs) related to IS implementation.
It also helps to narrow down the CSFs for COTS implementation by introducing categorization
schemes and the concept of relevance for the current study.
2.1 Selection process for articles
The subsequent sections of the chapter offer a review of the relevant literature. The first step
involves searches of electronic databases of journal articles using selected keywords and phrases
related to the current research topic. The most frequently cited academic papers from peer-
reviewed journals, as well as seminal articles related to IS implementation, CSF, stakeholder
theory, and control, were consulted. In addition, industry literature related to COTS
implementation was also explored to ensure that knowledge has been correctly identified from
academic journals.
The literature search utilized two primary sources of electronic databases – Business Source
Complete and Google Scholar. These two were chosen, as the goal of the current literature search
was to ensure article selection from leading business journals due to the managerial nature of the
research questions proposed.
Google Scholar is a database that “provides a way to broadly search for literature across many
disciplines and sources” (Cresswell, 2009: 31). More specifically, Google Scholar provides access
to a large selection of social science and engineering literature (Mingers, Marci and Petrovici,
2012). It is also considered to be a valuable complementary source because of its content as well
as the compilation of the contents (Pomerantz, 2006). As Chen states, “Google Scholar is able to
retrieve all scholarly publications from database and websites that are open to Google Scholar.
Subscription based abstracts and index that are not open to Google Scholar still have some unique
Page 23 of 307
contents, but these unique contents are primarily non-scholarly journal materials” (Chen, 2010:
226). Chen (2010) also emphasizes that Google Scholar is a reliable complement to traditional,
subscription-based database of scholarly publications. Quality of knowledge, reliability, accuracy
of citation counts, and wide coverage of the literature have also been reported by other researchers
(Mingers, Marci, and Petrovici, 2012; O’Leary, 2011).
Business Source Complete is a database that focuses on providing a comprehensive set of “peer-
reviewed, business related journals” (Business Source Complete, n.d.). Business Source Complete
provides “extremely comprehensive full-text access to certain key business and management
journals” (Bryman and Bell, 2003: 555).
The search of relevant literature was conducted through the use of keywords from two different
sets. The first set included keywords like IS implementation, COTS, COTS implementation,
Enterprise Systems etc., while the second set included keywords like implementation process,
phases, CSF, stakeholder, stakeholder engagement, stakeholder sensitivity, control configurations,
control balancing, control portfolio etc.
2.2 COTS and IS implementation
For the purpose of this research, two major categories of literature: peer-reviewed academic
journals and industry literature were considered. Innovation process models for information
systems, ERP implementation and COTS-specific implementation were examined to capture the
details of COTS-based implementation practices. In the following sections, we review some of the
significant models that can be considered valuable when it comes to implementing a COTS
solution.
Page 24 of 307
ERP literature was reviewed because most of the enterprise systems and COTS products share one
key characteristic: they both are packaged software. This close resemblance is apparent from some
of the adopted definitions of ERP system given by ERP researchers. For example, following is the
definition given by Peslak (2008):
An ERP system is an integrated commercial off-the-shelf (COTS) software package
that can perform all the major business functions of an organization. These functions
generally include all elements of the value chain from raw material purchases,
inventory management, production, goods shipments, invoicing, accounting, and
human resource management.
Non-ERP COTS often has additional custom components or glueware developed to make the
packaged components usable or functional for a certain organizational environment. Both
packaged software and custom-built software display great variations in terms of acquisition and
implementation methods. In parallel to the acquisition and implementation variations,
organizational IT competencies have also increased to accommodate such variations. This
acquisition process variation and enhanced organizational IT capabilities, over time, have been
stated by Daneshgar et al. (2013):
Over recent decades, the decision to select a software acquisition method has
been mainly relevant to large and medium size organisations with a large IT
investment. Given the wide-spread adoption of technology over that period, a
majority of large organisations are now more likely to possess sufficient levels
of IT competency to enable them to use a variety of methods for acquiring and/or
managing the acquisition process of their software systems including in-house
Page 25 of 307
development, outsourcing, and setting up offshore collaborators. Such
capabilities are normally related to various phases of the system development
life cycle, as well as support services.
Although in several other large-scale software development initiatives such as ‘game-
development’ heavily utilize COTS packages and ‘off-the-shelf’ components, the focus
of this research is purely from an enterprise perspective which limits our literature review
to systems that integrate various enterprise processes to give a holistic view of the
enterprise. The following sections offer a review of the most prominent models for
enterprise COTS implementation (including well-known COTS such as ERP, CRM),
their focus of analysis (micro versus macro), and their similarities and differences. In
addition, a more recent practice of IS implementation – an agile implementation – has
also been discussed in section 2.2.1 and 2.2.2
Umble et al. (2003) ERP Implementation Steps :
Umble et al. (2003), reviewing several earlier ERP implementation studies (Langenwalter, 2000,
Odenet al. 1993, Ptak, 1999, Ptak, 2000), compiled a list of 11 recommended steps for a successful
implementation. These include: 1) reviewing the pre-implementation process to date, 2) installing
and testing any new hardware, 3) installing the software and performing the computer room pilot,
4) attending system training, 5) training on the conference room pilot, 6) establishing security and
necessary permissions, 7) ensuring that all data bridges are sufficiently robust and the data are
sufficiently accurate, 8) documenting policies and procedures, 9) bringing the entire organization
on-line, either in a total cutover or in a phased approach, 10) celebrating, and 11) improving
Page 26 of 307
continuously. These steps identified by Umble et al. (2003) are indeed critical for a successful
packaged solution implementation, however, they consist of pre-implementation, during-
implementation and post-implementation activities. In addition to project activities, Umble et al.’s
(2003) recommended steps include policies and processes. Due to the broad focus, recommended
11 steps do not cover the implementation activities in sufficient detail.
Shin and Lee (1996) Non-ERP Implementation Process:
In the domain of non-ERP packaged software, Shin and Lee (1996) provided a more
comprehensive list of activities related to implementation processes. Reviewing earlier COTS
implementation research, Shin and Lee (1996) suggested 25 activities grouped into three phases
and seven sub-phases for packaged software acquisition and implementation.
Table 1: IS Project Phases
Phases Sub-Phases Activities
A. Project Formulation
phase
A-1. Project initiation
A-2. Requirement Analysis
1) Constitute Committee
2) Define Problems to Solve
3) Define the Scope and Goals of the Project
4) Planning Overall Process
5) Analyze Current Task Performing System
6) Define Users’ Information Requirement
7) Check Various Organizational Constraints
8) Integrate and Adjust Analysis Output
9) Information Source Selection and Searching
10) Make List of Alternative ASPs after Rough
Screening
B. ASP Selection Sub-
phase
And Acquisition Phase
B-1. Preparation
B-2. Selection
B-3. Acquisition
C. Installation and Post-
Implementation Use
phase
C-1. Installation
C-2. Post Implementation
Page 27 of 307
11) Develop Content and Format of RFP (Request
for Proposal)
12) Develop Evaluation Model
13) Collect and Analyze Vendor Proposal
14) In-depth Interview with Candidate Vendors
15) Benchmark Test and/or Arrange On-site
Demonstration
16) Make Final Decision and Authorization
17) Contractual Negotiation, Order and Take
Delivery
18) Develop Implementation Plan
19) ASP Modification
20) Internal Procedure Modification
21) Provide Training
22) Data Conversion and System Transition
23) Testing New System in the User’s Environment
24) System Operation and Maintenance
Implementation
25) Performance Evaluation
Shin and Lee’s (1996) proposed phases resemble the consolidated groups of actual phases that
most COTS implementation projects follow. Although the sub-phases align closely with most
commonly observed implementation phases, Shin and Lee (1996) did not indicate clear
demarcation points for the activities, which is essential for associating them to a phase and
identifying critical inter-phase relationships.
Markus and Tanis (2000) Enterprise System Model:
Page 28 of 307
Soh and Markus (1995) proposed an “IT investment to business value” model, where three linked
processes outline how IT investments result in organizational value. Based on this framework
Markus and Tanis (2000) proposed Enterprise System Experience Cycle, consisting of four phases:
project chartering, configuration and rollout, shakedown, and onward and upward, elaborated on
below.
The project chartering phase comprises decisions leading up to the funding of an enterprise system
or packaged software. Building a business case for adopting packaged software, selecting the
appropriate software package (ASP), identifying a project manager, and obtaining necessary
budget approval are typically the key activities performed during this phase. This phase includes
both “Project Formulation” and “ASP Selection,” as indicated by Shin and Lee (1996).
The configuration and rollout phase comprises activities intended to get the packaged solution
deployed for organizational units in a usable form. This corresponds to the Shin and Lee’s (1996)
phase C, “Installation and Post-Implementation use.” Key activities for this phase include
packaged software configuration, integration with existing environment, testing, data conversion
and validation, training, and rollout.
The shakedown phase aims to stabilize the system through eliminating any bugs. Key activities of
this phase include bug fixing, system performance tuning, retraining, and acquiring sufficient
support staff. Shin and Lee’s (1996) sub-phase “C-2” and activity 24 are similar to this phase
proposed by Markus and Tanis (2000).
The onward and upward phase concerns the maintenance of the packaged software, typically
though supporting users, ensuring optimal usage and facilitating upgrades. This phase starts from
Page 29 of 307
normal operation until the system is replaced with an upgrade or a different system. Although Shin
and Lee (1996) referred to this phase through activity 25, Markus and Tanis (2000) explicitly
associate this phase with benefit realization from organizational IT investments.
Both Shin and Lee (1996) and Markus and Tanis (2000) tried to capture the implementation and
post-implementation aspects of a COTS product from an enterprise perspective, but through
slightly different conceptualizations. Shin and Lee (1996) takes a top-down approach where major
implementation phases are hierarchically decomposed into lower level phases and tangible activity
items. Markus and Tanis (2000) also captured higher level implementation phases of an enterprise
system, however the focus of analysis is higher level benefit realization.
Morisio et al. (2002) model:
Morisio et al. (2002), through their empirical investigation, have identified four major process
phases for COTS-based implementation: requirements, design, coding, and integration. They have
also indicated a distinct implementation process for packaged software, which is quite different
from traditional IS implementation process. They reported that “COTS projects were obliged to
follow a process quite different from traditional projects, with more effort put into requirements,
test and integration, and less into design and code” (Morisio et al., 2002. This conclusion was
supported through the identification of three types of differences compared to the traditional
implementation process. Morisio et al. (2002) reported these differences under three categories: 1)
New activities, 2) Reduced activities, and 3) Modified activities. New activities include tasks like
product evaluations, product familiarization, and vendor interaction (both technical and non-
technical). These tasks also warrant new roles like a COTS evaluation team, and a team member
responsible for interacting with the vendor. Modified activities include the design and architecture
Page 30 of 307
phase, where the design focus shifted more on how to fit pieces together rather than analyzing the
internal workings of different modules. Reduced activities include tasks like coding, debugging,
unit testing, and code inspection.
Figure 1: COTS implementation phases and key activities (Source: Morisio et al.2002)
Whereas the derived model by Morisio et al. (2002) captures the core activities of a COTS
implementation at some level of detail, a comparison with other existing models indicates that
this model does not take into account some of the leading and trailing implementation activities,
many of which are critical for a successful COTS implementation. Additionally, this model can
potentially be enhanced by capturing the level of engagement by the different stakeholders for
various key COTS-specific activities.
In the next section, industry literature and knowledge, specific to COTS implementation, are
reviewed to identify COTS implementation models that are followed by practitioners.
Page 31 of 307
The industry literature indicates that COTS implementation models are developed for vendor-
specific products, which are based on implementation best practices and heavily dependent on the
features of the vendor-provided tools. Models based on EPIC (Evolutionary Process for Integrating
COTS-based systems) principles and RUP (Rational Unified Process) for COTS implementation
appears to be the most relevant model for the purpose of the present research. The EPIC
methodology, developed by Albert (2002), consists of four spheres of influence and iteratively
converging decisions over time through accumulating knowledge and increasing stakeholder buy-
in. Rational Unified Process (RUP) is a configuration based on EPIC principles, providing
guidance for COTS project implementation. Stages of the RUP model for a COTS implementation
are discussed below.
RUP Model:
RUP has four implementation stages for a COTS project: Inception, Elaboration, Construction and
Transition. The Inception phase corresponds to the project formulation (Shin and Lee, 1996) or
project chartering (Markus and Tanis 2000) phase in the most commonly found implementation
models in the academic literature. The primary task here is to establish feasibility through a
business case, identifying one or more candidate solutions and achieve concurrence among
affected stakeholders regarding the life-cycle objectives for the project.
Elaboration phase focuses on early implementation activities with an aim to achieve sufficient
stability of the architecture and requirements. The primary objectives of this phase are selecting
and acquiring components, mitigating risks, and developing a predictable cost and schedule for
implementation.
Page 32 of 307
Construction phase corresponds to the actual user acceptance testing or end-of-development stage
where a production-quality release ready for the user community is available. In earlier
implementation models reviewed, both “Installation and Post-Implementation use (Shin and Lee,
1996)” and “Configuration and Roll out (Markus and Tanis, 2000)” indicate these activities, which
essentially result in a production-ready version of the packaged software.
Transition phase of RUP is the final implementation stage of a COTS product where the objective
is to transition the solution to its users. This stage overlaps multiple stages of some of the existing
implementation models. For example, the Configuration and roll-out, and Shakedown phases
(Markus and Tanis, 2000) both jointly include activities indicated by the transition phase.
RUP phases for COTS implementation are product-specific, capturing many of the critical
implementation aspects of packaged software, but this may not be suitable for implementation
phases when it comes to public sector. This can be primarily attributed to the different business
model followed by government organizations compared to the business model of private industry.
A brief comparison of all the reviewed models are presented on Table 2:
Table 2: Implementation Model Comparison
Model Implementation
Target
Focus of Analysis Strengths Gaps
Umble et al.
(2003)
ERP
Implementation
A comprehensive
review of ERP
implementations and
identification of critical
activities
identification of critical
activities of
implementation
Missing significant
number of COTS
implementation specific
activities
Page 33 of 307
Simplified
categorization of
activities
Representative items
due to the
comprehensive nature of
the study
Identification of tools
Identification of leading
and trailing activities
Sequencing of activities
Information flow
Shin and
Lee (1996)
ERP
Implementation
takes a top-down
approach where major
implementation phases
are hierarchically
decomposed into lower
level phases and
tangible activity items.
Identification of
actionable items
Identification of
processes
Logical Process/phase
based grouping
Identification of tools
Identification of leading
and trailing activities
Sequencing of activities
Information flow
Markus and
Tanis
(2000)
Enterprise
Systems
Captures higher level
implementation phases
of an enterprise
system, however the
focus of analysis is
organizational benefit
realization
Higher level benefit
realization
Captures post
implementation aspect
of a COTS product
Multiple phases are
consolidated to keep the
model simple
Lack of focus on the
initiation and pre-
initiation activities
Sequencing of activities
Information flow
Page 34 of 307
RUP and
EPIC
Model
Enterprise
COTS
Focuses on the
significance of iterative
convergence of
decisions and obtaining
stakeholder buy-in.
Very specific to
enterprise COTS
implementation projects
Supported by software
tools to control and
implement the flow and
phases
Vendor specific model
Information flow
Visibility of processes to
be able to adjust them
Morisio et
al. (2002)
Enterprise
COTS
Model developed
purely by analyzing
enterprise COTS
implementation and
primarily focuses on
the implementation
aspect
Captures several unique
aspects of enterprise
COTS implementation
does not take into
account some of the
leading and trailing
activities
Identification of tools
2.2.1 An agile implementation
Considered from a flow perspective, all of the reviewed implementation models assume a waterfall
or sequential model for the activity flow. This is significantly different from the approach targeted
by the current thesis, which seeks to investigate an “agile” implementation model for enterprise
COTS.
Agile methods emerged as an alternative to plan-driven software development methods over a
decade ago (Dingsøyr et al. 2012; Agile Software Development, 2014). These are a group of
software development methods based on iterative and incremental development. Several
definitions of agile development are prevalent in literature. For example, Henderson-Sellers and
Serour (2005) defined agility as “the ability to adapt to different changes and to refine and fine-
tune development processes as needed.” Also, Lee and Xia (2010) define software development
Page 35 of 307
agility “as the software team’s capability to efficiently and effectively respond to and incorporate
user requirement changes during the project life cycle.” A more comprehensive definition of
agility is provided by Conboy (2009) as:
The continual readiness of an information systems development (ISD) method to
rapidly or inherently create change, proactively or reactively embrace change, and
learn from change while contributing to perceived customer value (economy,
quality, and simplicity), through its collective components and relationships with
its environment.
2.2.2 The agile manifesto and agile values
The articulation of the agile manifesto in 2001 has contributed to a significant change in the
software engineering field (Dingsøyr et al. 2012). This change is further observed through the
appearance of a host of methods adhering to the tenets of the manifesto, such as eXtreme
programming (XP), scrum, lean software development, feature-driven development (FDD), and
crystal methodologies.
The 12 principles of the agile manifesto (Agile Software Development, 2014) include: (1) Highest
priority to satisfy the customer through early and continuous delivery of valuable software, (2)
Welcome changing requirements, even late in development, (3) Deliver working software
frequently, from a couple of weeks to a couple of months, with a preference to the shorter
timescale, (4) Joint working group of business people and developers throughout the project, (5)
Build projects around motivated individuals (6) Utilize face-to-face conversation as the most
Page 36 of 307
efficient and effective method of conveying information to and within a development team, (7)
Working software is the primary measure of progress, (8) Agile processes promote sustainable
development, (9) Continuous attention to technical excellence and good design enhances agility,
(10) Simplicity – the art of maximizing the amount of work not done – is essential, (11) The best
architectures, requirements, and designs emerge from self-organizing teams, and (12) At regular
intervals, the team reflects on how to become more effective, then tunes and adjusts its behaviour
accordingly.
Besides these 12 principles outlined in the manifesto, the core values of agile development – a
focus on the individuals and their interactions, early delivery of working software, customer
collaboration and responsiveness to change (positively contributing to implementation time), and
considering the quality and cost of the software – are essential for today’s highly dynamic and
competitive business environments (Stavru, 2014). Due to the flexibility and adaptability offered
by agile methodology, both software development and IS implementation initiatives demonstrate
the application of agile methods. This is also reflected in the increasing volume of academic
literature in this domain. Dingsøyr et al. (2012) have identified a total of 1,551 research papers on
agile software development published between 2001 and 2010. Research in this domain has also
demonstrated substantial variations, ranging from simple adoption of agile methods (e.g., Boehm,
2002; Nerur et al. 2005) to aspects of team dynamics (Moe et al., 2009), the consequences of test-
driven development (Erdogmus et al. 2005; Janzen and Saiedian, 2005), adoption and post-
adoption issues (Cao et al. 2009; Mangalaraj et al. 2009), and the challenges of implementing agile
principles in distributed settings (Ramesh et al., 2006).
Page 37 of 307
Although the benefits and diverse effects of adopting an agile implementation approach are well
established in the literature (Misra, Kumar and Kumar, 2009; Misra, Kumar and Kumar, 2010a;
Misra, Kumar and Kumar, 2010b; Misra et al. 2012), they focus primarily on the in-house or
greenfield development of software. Adopting a similar approach for a packaged software
implementation is anticipated to introduce new challenges, processes and methodological changes
that have not been sufficiently identified by earlier studies examining COTS implementation.
Therefore, agile implementation of an enterprise COTS application holds much potential for
contributing to our understating of the implementation process in terms of activity flow,
information flow, and inter-phase relationships.
Most of the implementation activities, processes and tools employed in an enterprise COTS
implementation often demonstrate a visible connection with the people that are interested in or
involved with the implementation. Next section reviews IS literature related to stakeholder theory.
A guiding motivation for this review is to examine the concept of “stakeholder orientation” and
highlight the difference between “consideration” and “engagement.” A review of the stakeholder
and stakeholder engagement also helps clarify different aspects relating to ‘whose interest matters’
and ‘how does it matter’ in an enterprise COTS implementation.
2.3 Stakeholders and stakeholder engagement
Conceptual origin and representation
Originally rooted in strategic management literature, stakeholder theory has been receiving
increasing attention – both by the managers and academics – since the publication of Freeman’s
Page 38 of 307
(1984) landmark book Strategic Management: A Stakeholder Approach (Mitchell and Angel,
1997). Although Freeman intended to suggest that stakeholder relationship is a useful unit of
analysis when it comes to strategy or strategic management, he also attempted to clarify that the
concept of stakeholder emerged much earlier from a Stanford research, which defined it in 1963
as “those groups without whose support the organization would cease to exist” (Freeman, 2004).
Freeman later proposed a definition of stakeholder that is much broader than Stanford’s original
definition: “A stakeholder in an organization is (by definition) any group or individual who can
affect or is affected by the achievement of the organization’s objectives” (Freeman 1984).
The original stakeholder model presented by Freeman (1984) in relation to the organization was
quite simple. Placing the organization at the center and the all other significant stakeholders around
it, Freeman’s (1984) original model closely resembles a wheel. Figure 2 presents a version of the
Freeman’s (1984) original model, which shows seven stakeholders:
Figure 2: Stakeholder model presented by Freeman (1984)
Page 39 of 307
Besides the structural representation of the stakeholder model, another aspect related to the
stakeholder theory is the nature of application or utilization of the concept. Both of these aspects
are dominant in stakeholder literature and deserve sufficient consideration when adopting a
stakeholder lens.
A significant contribution towards the understanding of diverse applications or different features
of stakeholder theory is the work of Donaldson and Preston (1995), which aims to clarify the nature
and the purpose of stakeholder theory. Donaldson and Preston (1995) identifies three different uses
of stakeholder theory, each of which have a distinct value proposition: (1) descriptive, (2)
instrumental, and (3) normative.
Donaldson and Preston (1995) proposes a descriptive application of stakeholder theory as the
“theory is used to describe, and sometimes to explain, specific corporate characteristics and
behaviours.” Instrumental application refers to the usage of stakeholder theory “to identify the
connections, or lack of connections, between stakeholder management and the achievement of
traditional corporate objectives (e.g., profitability, growth)”. Finally, a normative view of
stakeholder theory was identified as intended to “interpret the function of the corporation,
including the identification of moral or philosophical guidelines for the operation and management
of corporations”. These three aspects of stakeholder theory are described as nested layers, with the
normative aspect at the core, instrumental in the middle, and the descriptive aspect on outside, as
shown in Figure 3:
Page 40 of 307
Figure 3: Three aspects of Stakeholder Theory (Donaldson and Preston, 1995)
Defining Stakeholders
Besides the scope variations naturally appearing from diverse definitions, Friedman and Miles
(2006) identified indiscriminate usage of the term “stakeholder” over the last two decades. It is
frequently used by private industries, public-sector organizations, businesses and the media, but
often without any clear definition or understanding of the term itself (Mainardes, Raposo, and
Alves, 2011). While confusion in non-academic circles is not surprising due to a lack of knowledge
concerning stakeholder theory, a consensus around the definition of the term also seems difficult
to find among academic researchers. The diversity surrounding the definition is made clear by
looking at the works of Bryson (2004), Buchholz and Rosenthal (2005), Pesqueux and Damak-
Ayadi (2005), Friedman and Miles (2006), and Beach (2008), which combined contain a total of
66 different concepts of the term “stakeholder” (Mainardes, Raposo, and Alves 2011).
This large array of stakeholder definitions ranges from a very wide scope to a narrow scope. Most
of the definitions observed in the literature drift between these two extremes. The existence of
considerable variations regarding the definition of a stakeholder is clearly depicted through the
Page 41 of 307
work of Mitchell et al. (1997), which compiles a list of different views regarding stakeholders
observed in the literature. This is presented in Table 3.
Table 3: Who Is a Stakeholder? A Chronology
Source Stake
Stanford memo, 1963 "those groups without whose support the organization
would cease to exist" (cited in Freeman & Reed, 1983,
and Freeman, 1984)
Rhenman, 1964: -- "are depending on the firm in order to achieve their
personal goals and on whom the firm is depending for
its existence" (cited in Nasi, 1995)
Ahlstedt and Jahnukainen, 1971 "driven by their own interests and goals are participants
in a firm, and thus depending on it and whom for its sake
the firm is depending"
Freeman and Reed, 1983: 91 Wide: "can affect the achievement of an organization's
objectives or who is affected by the achievement of an
organization's objectives" Narrow: "on which the
organization is dependent for its continued survival"
Freeman, 1984: 46 "can affect or is affected by the achievement of the
organization's objectives"
Freeman and Gilbert, 1987: 397 "can affect or is affected by a business"
Cornell and Shapiro, 1987: 5 "claimants" who have "contracts"
Evan and Freeman, 1988: 75-76 "have a stake in or claim on the firm"
Evan and Freeman, 1988: 79 "benefit from or are harmed by, and whose rights are
violated or respected by, corporate actions"
Bowie, 1988: 112, n. 2 "without whose support the organization would cease to
exist"
Alkhafaji, 1989: 36 "groups to whom the corporation is responsible"
Carroll, 1989: 57 "asserts to have one or more of these kinds of stakes"-
"ranging from an interest to a right (legal or moral) to
ownership or legal title to the company's assets or
property"
Freeman and Evan, 1990 contract holders
Thompson et al., in 1991: 209 in "relationship with an organization"
Page 42 of 307
Savage et al., 1991: 61 "have an interest in the actions of an organization and ...
the ability to influence it"
Hill and Jones, 1992: 133 "constituents who have a legitimate claim on the firm ...
established through the existence of an exchange
relationship" who supply "the firm with critical
resources (contributions) and in exchange each expects
its interests to be satisfied (by inducements)"
Brenner, 1993: 205 "having some legitimate, non-trivial relationship with an
organization [such as] exchange transactions, action
impacts, and moral responsibilities"
Carroll, 1993: 60 "asserts to have one or more of the kinds of stakes in
business"-may be affected or affect ...
Freeman, 1994: 415 participants in "the human process of joint value
creation"
Wicks et al., 1994: 483 "interact with and give meaning and definition to the
corporation"
Brenner, 1995: 76, n. 1 "are or which could impact or be impacted by the
firm/organization"
Langtry, 1994: 433 the firm is significantly responsible for their well-being,
or they hold a moral or legal claim on the firm
Starik, 1994: 90 'can and are making their actual stakes known"-"are or
might be influenced by, or are or potentially are
influencers of, some organization"
Clarkson, 1994: 5 "bear some form of risk as a result of having invested
some form of capital, human or financial, something of
value, in a firm" or "are placed at risk as a result of a
firm's activities"
Clarkson, 1995: 106 "have, or claim, ownership, rights, or interests in a
corporation and its activities"
Nasi, 1995: 19 "interact with the firm and thus make its operation
possible"
Donaldson and Preston, 1995: 85 "persons or groups with legitimate interests in
procedural and/or substantive aspects of corporate
activity"
(Adopted from Mitchel et al., 1997)
Page 43 of 307
Classifying Stakeholders
A direct consequence of the variations surrounding the definition of “stakeholder” is the
emergence of diverse methods and approaches for identifying relevant stakeholders. The most
frequently appearing stakeholder classification schemes focus on: primary or secondary
stakeholders; owners and non-owners of a firm; owners of capital or owners of less tangible assets;
actors or those acted upon; those existing in a voluntary or involuntary relationship with the firm;
rights-holders, contractors, or moral claimants; resource providers to or dependents of a firm; risk-
takers or influencers; and legal principals to whom agent-managers bear a fiduciary duty (Mitchell
et al., 1997).
Among the research inspired by these wide variations, and aiming to develop a comprehensive
stakeholder classification scheme, the work of Mitchell et al. (1997) is particularly noteworthy.
They have proposed a distinct and practical way to classify stakeholders by focusing on three
different attributes: power, legitimacy, and urgency. Power indicates the possession of an ability
to make someone do something that would not be possible without the possession of that ability.
The power of the stakeholder over the organization may be coercive (strength or threat), normative
(legislative, the media) or utilitarian (holding resources or information) (Mainards, Raposo, and
Alves, 2011). Legitimacy is a perception regarding the actions of an entity where actions are seen
to be acceptable according to the norms of the society or organization. Urgency indicates the
immediate need for an action. It is also defined by the Merriam-Webster Dictionary as “calling for
immediate attention” or “pressing.” Considering the presence or absence of these three factors,
Mitchell et al. (1997) have proposed eight different categories of stakeholders.
Page 44 of 307
Despite the close relationship with context of an organization, where the attributes of power,
legitimacy and urgency are constantly in play, Mitchell et al.’s (1997) classification scheme is
heavily focused on the nature or type of the stakeholder, rather than answering the question “who
is a stakeholder?” This leaves us with room for exploring other classification schemes and
frameworks for analyzing stakeholders, depending on the context.
Identifying Stakeholders
Regardless of the wide variations surrounding who a stakeholder is and their categorizations, the
critical role played by stakeholders in IS implementation remains uncontested in the literature. The
body of literature on stakeholder theory is quite extensive, and this theory has been used by the
researchers to investigate organizational ambiance, strategic management, ethical concerns,
business planning processes, e-government, project management, environment management, as
well as the successful implementation of information and communication technologies and large
information systems (Mishra and Mishra 2013). Besides a great amount of recent evidence, earlier
investigations by Lyytinen and Hirschheim (1988, 1987) also demonstrated that the failure of IS
implementation is contingent upon the level of satisfaction of different stakeholder groups.
Although these findings are excellent indications of the suitability of the stakeholder lens while
considering an enterprise COTS implementation, there is a lack of attention in the area of
organizational orientation towards stakeholder engagement and stakeholder sensitivity in the
context of public-sector COTS implementation.
Despite the wide applicability of stakeholder theory in different contexts, current review of the
literature indicates that the focus is primarily centered around stakeholder categorization and
stakeholder management in general. We believe that during a large-scale commercial off-the-shelf
Page 45 of 307
(COTS) software implementation, maintaining the appropriate level of stakeholder engagement
and stakeholder sensitivity is of outmost importance for achieving success. This is also made
obvious by relating back to Mitchell et al.’s (1997) categorization of stakeholders, as it is possible
for one stakeholder group to move to a different stakeholder group by attaining necessary
attributes. But such a transition could possibly jeopardize a large-scale project if the number of
dangerous stakeholder increases. Beside the necessity of maintaining proper sensitivity, the point
of engagement and the nature of engagement are still largely unexplored in this context. A joint-
effort COTS implementation involving multiple government organizations and multiple vendors
has a much broader scope in terms of affected parties and people, compared to an internal COTS
implementation by a single organization. Since this type of implementation crosses a single
organizational border, it can also mean a loose association among stakeholders (Boonstra, 2008),
which makes their proper identification and engagement during various implementation phases
absolutely critical for success. Therefore, an investigation of stakeholder orientations during COTS
implementation can not only can help us understand the aspects of sensitivity and engagement, but
also help us identify the manner in which a stakeholder approach is used during a project.
2.4 Control configuration and control balancing
Both the consideration and engagement of all key stakeholders are essential for a successful
enterprise COTS implementation. Although the key tenet of stakeholder theory emphasizes the
“consideration” aspect, ensuring proper “engagement” calls for a different approach. The requisite
for optimal stakeholder engagement is apparent considering the many characteristics of a multi-
partner COTS implementation. Aside from being a complex task and embedded with numerous
interdependencies, enterprise COTS implementation requires eliciting input from multiple
stakeholders – often with non-overlapping knowledge (Maruping et al., 2009). Along with the
Page 46 of 307
benefits of complementary knowledge and skills offered by various stakeholders, engagement also
brings the challenge of divergent priorities, goals, motivations, and power relations, which requires
proper control and management to ensure a positive outcome (Kirsch et al. 2002; Clark et al. 1997).
Strong support for the presence of these non-technical challenges is also offered through several
earlier studies where IS project derailment had little relationship with technical challenges or
technology (Markus and Benjamin, 1996; Hartwick and Barki 1994; Robey et al. 1989).
Kirsch (1996) defined managerial control for IS projects as “attempts to ensure that individuals
working on organizational projects act according to an agreed-upon strategy to achieve desired
objectives”. The dyadic nature of control – between a controller (the person exercising control)
and a controllee (the target of control) – has inspired numerous empirical studies relating control
to organizational or IS project contexts through diverse perspectives (Kirsch, 1997, 2004).
Additionally, literature related to control theory has focused on the modes of control comprised of
two broad categories: formal and informal (Crisp, 2002). Formal control is typically documented
and initiated by the management, commonly comprised of outcome control and behaviour control
(Ouchi, 1979; Eisenhardt, 1985). Informal control is usually comprised of undocumented measures
initiated by the employee themselves (Jaworski, 1988), and are commonly identified as self-
control and clan control (Ouchi, 1980). Although the simultaneous application of multiple control
types – belonging to both formal and informal control categories – has been observed in IS
development efforts, the choice of any control type is subject to the presence of certain antecedents,
such as outcome measurability, behaviour observability, knowledge of appropriate behaviour, task
characteristics, a desire to exercise self-control, etc. (Kirsch, 1997). In a more recent investigation,
focusing on formal controls alone, Heumann et al. (2014) have identified four antecedents for
Page 47 of 307
control choices: task complexity, legitimacy concerns, performance considerations, and
performance/efficiency concerns, which are in agreement with the findings of Kirsch (1997).
Whereas antecedents are indeed a critical factor in determining the choice of control mechanism,
the circumstances leading to a change of an existing control are equally important to maintaining
the effectiveness of the controls. In addition, the concept of control effectiveness is significant, as
an applied control does not always generate the expected behaviour change. This is evident in the
paradoxical pattern identified by Tiwana and Keil (2009), which leads to the distinction between
attempted control and realized control. An analogous distinction has also been observed by
Heumann et al. (2014), who suggest that control propagation across multiple levels of hierarchy
can have different effects. This brings forth the notion of “controls recognized” by the controllee.
Considering the exclusive focus on one specific controller-controllee dyad in most prior studies,
Heumann et al.’s (2014) examination of control propagation through multiple hierarchies indeed
offers a novel contribution to IS control literature. However, research related to the simultaneous
examination of both vertical and horizontal control relations among multiple project partners of an
IS implementation is still in its infancy. Additionally, much of the control theory literature
concentrates on a single organizational context, and does not include scenarios of applying control
among non-hierarchical project partners or in client-vendor relationships. This is significantly
different compared to a single organizational context with a hierarchical relationship, as some of
the traditional control modes like behavioural control, clan control or self-control are difficult to
enact by the controller. Acknowledging this challenge in the context of a large offshore IS
implementation project, Gregory et al. (2013) adopted a different conceptualization of the control
Page 48 of 307
type by defining what a “control configuration” is. They have also conceptualized control
balancing as the act of making targeted adjustments to the control configuration, which consists of
control type, control degree, and control style.
Enterprise COTS implementations, similar to most ISD projects, face a high degree of requirement
volatility and technical complexity. With a large group of stakeholders, ensuring goal congruence
and alignment with project objectives requires the continual adoption of an appropriate control
configuration for stakeholders. The very nature of the context calls for an effective application of
the control balancing concept. Aside from the identification of control relationships among key
stakeholder groups, identifying the conditions triggering a change in this configuration is essential
for understanding control balancing. In an earlier attempt to identify the reasons for control change
during project implementation, Mahring (2002) suggested a changing relationship between
vendors and clients as one possibility. Choudhury and Sabherwal (2003) identified vendor
performance as one such reason, which can also be considered as an antecedent for the condition
identified by Mahring (2002). Considering the evolving nature of IS implementation, Kirsh (2004)
categorized these conditions into three major categories: (1) project context, containing task
characteristics, task interdependencies and project performance; (2) stakeholder context,
containing changes in knowledge and skills, the nature of relationship, the lack of common goals,
role expectation changes; and (3) global context, comprising priority differences, geographic
differences, time zone differences, and cultural differences. Although the global context can be
considered to be a more stable antecedent, a majority of the conditions can be translated to
stakeholder perceptions or relationships among them. The control balancing triggers identified by
Gregory et al. (2013) also support such conceptualizations, including shared understanding and
Page 49 of 307
unfulfilled expectations. Support for these conditions also found in earlier research focusing on
control (for example, Narayanaswamy, 2014) defined control loss as the extent to which people,
processes, and resources do not progress as expected, which resonates with the unfulfilled
expectations identified by Gregory et al. (2013). Other factors that can be classified as “risk
perception” are also hinted at as possible conditions for adjusting control mechanisms for
mitigating exchange hazards and uncertainty between exchanging partners in scenarios of
opportunism and bounded rationality (Williamson, 1985; 1991). Considering different orientations
of control balancing, as well as examining such concept vis-à-vis stakeholder engagement and
CSF, can thus offer tremendous benefits for multi-partner COTS implementation.
2.5 Critical success factors (CSF)
In parallel to processes or implementation phases, COTS implementation should also address the
critical success factors (CSF) to ensure an overall implementation success (Boynton and Zmud,
1984). The CSF in IS research is a composite concept that represents both sides of an IS
implementation coin, where one side is implementation success and the other is implementation
failure. Both success and failure are two of the oldest research traditions in IS research, as the first
ICIS (International Conference on Information Systems) conference in 1980 highlighted the
following questions: What is IS success, and what determines IS success (Petter et al. 2013). In
the context of designing management information systems, Rockart (1979; Bullen and Rockart,
1981) elaborated that there are a few factors which are fundamental for the success of a company,
and their identification is a way to ensure proper consideration is given to them. The identification
of CSFs has been widely investigated in IS research. More specifically, CSF-centric research
related to ERP implementation and adoption is extensive (Kini and Basaviah, 2013; Xu et al. 2002;
Page 50 of 307
Soh et al. 2000; Ribbers and Schoo, 2002; Scheer and Habermann, 2000; Esteves-Sousa and
Pastor-Collado, 2000; Bingi et al. 1999; Al-Mashari et al. 2003; Hong and Kim, 2002; Somers and
Nelson, 2001; Umble et al. 2003).
An early attempt to systematically study the key enabling factors of a strategic information system
(SIS) implementation is the work of Runge (1985), who used a multiple-case-study approach to
study 35 systems in Britain. Runge (1985) identified five factors as key enablers for successful
SIS implementation: (1) the presence of a product champion, (2) a high level of customer
involvement in the development, (3) an existing system as a base to build upon, (4) a high degree
of attention to marketing, and (5) the circumvention of existing IS planning and prioritization
procedures. In light of our current understanding of IS implementation, some of the factors
identified here may seem questionable, for example “the circumvention of existing IS planning
and prioritization procedures.” However, the focus on “strategic” and “external utilization” of the
systems is probably responsible for the identified enabling factors to differ from other early studies
on IS implementation.
Using a mail-in survey of IS executives, King et al. (1989) investigated significant facilitators
behind an organization’s creation of strategic systems. They identified several positive forces,
including: technical support within the firm, existing IT leadership position, and pressure from
competition. On the opposite side, they identified a set of inhibitors, including: the importance of
other priorities, the difficulty of assessing tangible contribution, the lack of appropriate planning,
and the lack of top management support.
Page 51 of 307
Although King et al. (1989) focused on both failure and success, the factors identified through
their research are far from comprehensive in nature. A much wider scope and comprehensive
consideration of success factors related to IS implementation were demonstrated in the study
conducted by Rockart and DeLong (1988). One of their critical contributions was to pave the way
for IS implementation researchers to reach agreement regarding significant factors, or CSFs,
related to IS implementation. Observing 30 cases of EIS (Enterprise Information Systems)
implementation, Rockart and DeLong (1988) identified eight areas as the most important for IS
implementation success: (1) committed executive sponsor, (2) committed operating sponsor, (3)
clear link to business objectives, (4) management of organizational resistance, (5) management of
data, (6) appropriate technology, (7) appropriate information systems staff, and (8) management
of system evolution and speed. A later study conducted by Fitzgerald (1990) in the UK also
confirmed that most of these factors are relevant to IS implementation success; however, he also
reported that “a clear link of business objectives” was largely missing, and that “organizational
resistance” was insignificant in the UK context.
Although the areas of significance identified by Rockart and DeLong (1988) have been confirmed
by several other studies (Paller and Laska, 1990; Bird, 1991; Watson et al., 1991; Whymark, 1991;
Rainer and Watson, 1995; Turban, 1995; McBride, 1997) in addition to Fitzgerald (1990), two
noticeable features of the identified factors are their strong tie to “during-implementation” phases
and the broadness of scope. Besides the scope issue, earlier studies on IS implementation success
had another key characteristic in common: the lack of focus on studying a common system. This
is another possible source of variation responsible for the divergent findings from different
researchers. The seminal work by DeLone and McLean (1992) can be seen as a much-needed
Page 52 of 307
response to this challenge, which attempted to generalize the concept of IS implementation
success. Reviewing the research published during the period 1981–87, DeLone and McLean
(1992) represent IS success as a dependent variable relating to six independent variables: (1)
system quality, (2) information quality, (3) use, (4) user satisfaction, (5) individual impact, and (6)
organizational impact. DeLone and McLean (1992) thus proposed a framework to define IS
implementation success through these six interrelated variables. The original DeLone and McLean
model of IS implementation success, a one of the most cited frameworks in IS research (Lowry et
al., 2007), has helped a significant number of subsequent IS researchers to investigate, modify, or
extend the concept of IS implementation success (Dwivedi et al., 2015).
In understanding IS success, the DeLone and McLean model has proven to be a useful framework
(Petter et al., 2013). Its wide applicability resulted from the focus on the broader dimensions of
the IS implementation. Focusing on the broader dimensions of IS implementation, Leavitt (1965)
proposed four major categories (tasks, people, technology, and structure) to explain the
interrelationship between IS and other aspects of the working environment.
Both the DeLone and McLean (1992) and Leavitt (1965) models explaining IS implementation
success consist of different dimensions that include several distinct yet correlated set of variables
or factors, which collectively contribute to each dimension. Concentrating on this second aspect,
Petter et al. (2013) attempted to take a closer look at each dimension, identifying 43 determinants
impacting IS implementation success. The distinction between the research conducted by Petter et
al. (2013) and all other existing studies is the issue of “breadth versus depth.” Despite the focus on
depth of the issue, Petter et al. (2013) organized these factors into five broader categories to align
Page 53 of 307
with the organizational change model of Leavett (1965). In addition, they identified 15 success
factors that have consistently been found to influence IS success: enjoyment, trust, user
expectations, extrinsic motivation, IT infrastructure, task compatibility, task difficulty, attitudes
toward technology, organizational role, user involvement, relationship with developers, domain
expert knowledge, management support, management processes, and organizational competence.
While both aspects of the literature – focusing on the breadth and depth of implementation
activities – are essential for understanding the phenomenon of IS implementation success, a focus
on depth clearly helps to associate identified factors with organizations actors or driving forces.
Such association may provide a tremendous benefit for organizational leaders in adopting proper
policies to effectively manipulate those underlying variables and achieve the desired benefits.
A parallel stream of research to IS implementation success is IS implementation failure, and such
research has been prominent over the last four decades (Dwivedi et al., 2015). The primary reason
for such interest can be attributed to the alarmingly high rate of IS development and
implementation project failure, which according to some sources is in the area of 70 percent
(Doherty et al., 2011). IS failure is typically defined as the mirror image of IS success (Cecez,
2015). Lyytinen and Hirschheim (1987) classified four different types of failure: (1) an IS
correspondence failure, when a system does not meet the predefined set of objectives (a system is
either abandoned or scaled down); (2) an IS process failure that denotes unsatisfactory
development or performance, and does not deliver a functioning system or runs over time and over
budget; (3) an IS interaction failure, defined as user dissatisfaction with, or poor usage of, a
Page 54 of 307
system; and (4) an IS expectation failure, indicating the inability of the information system to
fulfill stakeholders’ needs, expectations, and interests.
Studying a failed IS implementation is clearly a different perspective from the analysis of a
successful implementation as the primary motivation behind such studies is often the desire to
learn from failure (Dwivedi et al., 2015). Contributions from this stream of research are also quite
substantial, as many different causes and consequences of IS failure have been identified by
various researchers (Avison and Wilson, 2002; Barker and Frolick, 2003; Beynon-Davies, 1995,
1999; Bussen and Myers, 1997; Fitzgerald and Russo, 2005; McGrath, 2002; Nelson, 2007; Pan
et al., 2008; Scott and Vessey, 2000). Although the factors identified through IS implementation
failure studies are extensive, similar to IS success research, studies focusing on IS failure can be
divided based on their focus on either breadth or depth. For example, Nelson (2007) analyzed 99
IS implementation projects and identified 36 causes of IS failure. These causes are then further
categorized under four major dimensions: process, people, product, and technology. Several other
researchers have taken a similar approach to identifying various causes of IS implementation
failure (Al-Ahmad et al., 2009; Barclay, 2008; Dwivedi et al., 2013b; Kappelman et al., 2006;
Schmidt et al., 2001; Wallace et al., 2004; Yeo, 2002). Indeed, the broader categories or higher-
level abstractions identified through failure research are often identical to those identified through
IS success research. However, despite the similarity between major dimensions, the perspectives
of both IS success and failure are equally important to IS literature, as certain conditions or
variables may appear as salient in one perspective but not on the other.
Page 55 of 307
Early IS implementation research focusing on success and failure demonstrates considerable
differences in terms of context when compared to their more recent counterparts. Earlier research
often studied diverse forms of IS implementations, which appears to be changing in recent research
where the focus is shifting towards the implementation of common types of strategic information
systems like ERP (Enterprise Resource Planning) and CRM (Customer Relationship
Management). Most of these systems also belong to the category of packaged software or COTS,
which requires additional implementation considerations compared to a greenfield development.
Regardless of the nature of IS implementation, many success and failure factors identified in earlier
research are still largely applicable to the more recent context of enterprise IS implementation
(Dwivedi et al., 2015).
Enterprise COTS implementation, which appears to have a close resemblance with ERP
implementation, is very likely to share a sizable number of CSF identified in existing ERP
literature. It may contain additional CSFs and challenges resulting from the lack of specialization
on all business functions by the chosen COTS product. CSF have also been used to address key
implementation issues through numerous ERP implementation studies (Holland and Light, 1999;
Reel, 1999; Teltumbde, 1999; Wu and Chang, 1999; Kumar, Maheshwari, and Kumar, 2002a;
Kumar, Maheshwari, and Kumar, 2002b; Kumar, Maheshwari, and Kumar, 2003; Gupta et al.
2016). Researchers have categorized them into broader categories to identify distinct conceptual
domains and facilitate policy formulation. Cantu (1999) has proposed five dimensions of CSFs,
with a total of 22 attributes. The CSF implementation framework identified by Cantu (1999) is
presented in Table 4:
Page 56 of 307
Table 4: CSF implementation framework
Critical success factor CSF attributes
Management/organization Commitment
Education
Involvement
Project team selection
Training
Roles and responsibility
Process Alignment
Documentation
Integration
Process redesign
Technology Hardware
Software
Systems management
Interface
Data Master files
Transactional files
Data structure
Maintenance and integrity
People Education
Training
Skills development
Knowledge management
(Source: Cantu, 1999)
Page 57 of 307
Using a different perspective, a policy-related categorization of CSFs has also been observed in
the literature. Finney and Corbett (2007), through their comprehensive compilation and analysis
of ERP implementations, identified 26 CSFs belonging to either a strategic or tactical group. This
categorization is presented in Table 5. In a more recent attempt, Tarhini et al. (2015) have
systemically reviewed 35 ERP implementation research studies focusing on CSFs, and identified
a group of 51 factors, which are in agreement with the findings of Finney and Corbett (2007).
However, Tarhini et al. (2015) presented a shorter list of CSFs, based on their salience in the
literature, consisting of: top management support and commitment, training and education, project
management, clear vision and objectives of the ERP system, careful change management, and
interdepartmental communication.
Table 5: Critical Success Factors
Strategic Tactical
Visioning and planning,
Build a business case,
Project champion,
Implementation strategy and
timeframe,
Vanilla ERP,
Project management,
Change management,
Managing cultural change
Balanced team,
Project team: the best and brightest,
Communication plan,
Empowered decision makers,
Team morale and motivation,
Project cost planning and
management,
BPR and software configuration,
Legacy system consideration,
IT infrastructure
Client consultation,
Selection of ERP,
Consultant selection and
relationship,
Training and job redesign,
Troubleshooting/crises
management,
Data conversion and integrity,
System testing,
Post-implementation evaluation
(Source: Finney and Corbett, 2007)
Page 58 of 307
A review of the ERP-focused CSF literature indicates the existence of different categorizations of
the CSF attributes (Cantu 1999; Finney and Corbett, 2007), depending on the purpose of
investigation; however, they often agree in terms of actual attributes. Regardless of the
considerable similarities between COTS and ERP implementations, there are some notable
differences when it comes to implementation approach, customization, migration, and the post go-
live support model. As a result, all the CSF identified through ERP implementation research may
not be applicable for a COTS implementation. In addition, managing the CSF is another key
consideration for any enterprise COTS or IS implementation. Despite a substantial body of IS
research focusing on CSF, our review of literature indicate that it is often not possible to manage
all pertinent CSFs equally well. This is attributed to two primary reasons: (1) each CSF addressed
during implementation (i.e. the amount of time spent on each CSF) has an associated cost that
contributes to the overall project cost, and (2) missing processes or capabilities to address each
CSF during implementation (Sun et al., 2005). A closer look at each CSF attribute reveals why
managing them individually leads to an increased project cost. However, this can be minimized by
taking a process- or policy-based perspective that every project can utilize without significant
additional expenses resulting from CSF management. However, adoption of such policy related
approach call for an understanding that most of CSF are part of one of the three broader categories:
people, process, and/or environment. Table 6 attempts to map 26 CSF identified by Finney and
Corbett (2007) into one or more of these broader categories.
Page 59 of 307
Table 6: CSF and category mapping
CSF People Process Environment
Visioning and planning X
Build a business case X
Project champion
X
Implementation strategy and timeframe
X X X
Vanilla ERP
X X
Project management
X X
Change management X X X
Managing cultural change
X X X
Balanced team X X
Project team: the best and brightest, X
Communication plan X
Empowered decision makers X
Team morale and motivation X
Project cost planning and management X X
BPR and software configuration X
Page 60 of 307
Legacy system consideration X
IT infrastructure X
Client consultation, X X
Selection of ERP X X
Consultant selection and relationship X X
Training and job redesign, X X
Troubleshooting/crises management, X X
Data conversion and integrity X X
System testing X X
Post-implementation evaluation
X X
Table 6 indicates that all of the CSF identified by Finney and Corbett (2007) are related to the
people, process, and/or environment categories. Leveraging the Stakeholder engagement could be
an effective approach to better manage CSF during enterprise COTS implementation, as most CSF
can be related to an individual, group or project team, both within the organization and outside of
the organization. Another complementary aspect that can facilitate the management of CSF is
balancing of control configurations. Control balancing has a direct impact on stakeholder
engagement. Similarly, environment specific CSF can be directly influenced by the
implementation processes.
Page 61 of 307
Research concentrating on individual aspects of ERP/Information Systems (IS) implementation,
such as CSF identification, implementation processes, and application of various controls, are quite
substantial, but the investigation of the CSF management through optimally engaging stakeholders
and balancing of control through applying different control configurations is largely non-existent
in the literature (Finney and Corbett, 2007). Therefore, examining the management of CSF through
the lens of stakeholder theory and control balancing will significantly enhance our understanding
of the enterprise COTS implementation.
Page 62 of 307
3.0 COTS implementation framework
COTS implementation differs from traditional software development or enterprise system
implementation projects on several key dimensions, such as the required level of functional and
technical expertise available within an organization, the level of integration complexity, the degree
of engagement by vendors, and client control over vendors and partners. For example, in an in-
house software development effort, integration might not appear as a significant challenge due to
explicit consideration of the organizational context and operating environment during the design
and architecture phase of the development. However, COTS products are developed with a generic
audience in mind. This often presents an integration challenge for the implementing organization
of a COTS product. Regardless of such substantial difference between a traditional software
development and a COTS implementation, the implementation process for non-ERP family COTS
has not been well examined in the literature. In addition to a host organization and a COTS vendor,
adding multiple partners and third-party consulting firms to the list of stakeholders makes the
implementation process even more complicated from a stakeholder engagement, and control-
balancing perspectives.
Enterprise COTS implementation projects are part of an organization’s strategic IT initiatives and
require large investments of organizational resources, yet often fail due to inadequate management
of the critical success factors (CSF) and implementation-specific challenges. The existing
knowledge gap surrounding the COTS implementation process, the opportunity for better
understanding the successful management of CSF, and balancing control at different project stages
– in the context of enterprise COTS implementation – collectively act as the primary motivational
forces behind the current investigation. In the following sections, we lay the foundation for the
current research by introducing and explaining related concepts, as well as constructs.
Page 63 of 307
3.1 Enterprise COTS and its implementation
There are several definitions of COTS products available in both the literature and industry. For
the purpose of this research, we have adopted the definition presented by Brownsword et al. (2000)
who define COTS as: “sold, leased, or licensed to the general public, offered by a vendor trying to
profit from it, supported and evolved by the vendor, who retains the intellectual property rights;
available in multiple, identical copies; and used without source code modification”. Key marketing
features of COTS are: cost savings, ease of integration and extension, reliability and capability
(Newcomb, 2007). However, COTS adoption and implementation to replace a legacy system or to
support existing business processes require a very different tactic compared to either custom-
developed in-house solution or outsourced software development, because the COTS vendor has
developed the software with a general audience in mind. Newcomb (2007) argues that the unique
business rules and logic required by most legacy systems are often beyond the functionalities
offered by packaged solutions readily available on the market, and a COTS purchase is a riskier
solution compared to an in-house development. COTS implementations could range from a very
simple out-of-the-box implementation with no project involvement to a very complex multi-
million-dollar project such as an enterprise document management system using IBM’s P83.
Nature of COTS implementation
A COTS package solution is the integrated assembly of one or more of the following (Albert and
Brownsword, 2002a): i) COTS packages or COTS package components, ii) legacy systems (piece
of the systems being replaced), iii) reuse libraries and other reuse sources (e.g., freeware,
shareware), iv) appropriate linkage to the broader organization’s architecture with which the
3 P8 is an enterprise electronic document management system sold by IBM
Page 64 of 307
solution must interface, v) changes to the end-user business processes necessary to match the
processes provided in the COTS packages, vi) any required custom code (including wrappers and
glueware).
There are numerous COTS or packaged solutions on the market for various purposes, and they all
come with different approaches to implementation. Even the simplest level (out-of-the-box) COTS
implementation may require organizational process changes and vendor relationship management
activities. For example, out-of-the-box solutions like Microsoft Office or VMware requires large
organizations to manage licences properly, and various Integrated Development Environments
(IDEs) or source control solutions (Visual Source Safe or VSS). In addition, they require the
organization to change the way source code is developed and managed, which may trigger internal
process changes.
Figure 4: COTS customization process
The customization of COTS products ranges from “out-of-the-box” to “extended,” based on the
degree of behavioural change necessary to make the product capable of supporting organizational
business needs. Figure 4 outlines how a vanilla COTS product is customized for an organization
Page 65 of 307
through configuration change, integration, and extension. “Out-of-the-box” COTS usually refers
to products that are stand-alone and can be deployed with the vendor-provided default
configuration. COTS products in this category are generally not very expensive and used for user
productivity. Some examples are Notepad++, Beyond Compare for file comparison, Log MX for
log parsing, etc.
For enterprise-level COTS products, the customization process generally involves both
configuration and integration changes. The configuration of COTS usually refers to the aspect of
the product that defines how the COTS package should behave in certain context. The ability to
configure COTS products gives the user an opportunity to incorporate some of the unique business
requirements of an organization. For example, COTS configuration could include how an
application should handle user authentication (LDAP vs. local database), how the application
should be accessed, the duration of an inactive session, various security parameters, the file system
location on the network etc. Values for these configuration parameters are usually different for
different organizations, and can be configured through a graphical user interface (GUI) or a
configuration file specific to that COTS product. The integration of COTS, on the other hand,
refers to product compatibility and communication with other existing systems within the
organization. For example, SQL connectivity for the backend database of Microsoft’s dynamic
client relationship management (CRM), or accessing CRM database information from an outlook
email client, will impose integration requirements on the COTS product.
Page 66 of 307
Implementation Time
Figure 5: Relationship of task complexity and required time based on COTS customization style.
The extension of COTS requires a significant amount of development tasks and can result in the
addition of a completely new module for the purchased software, or some smaller custom
components commonly referred as “glueware,” to facilitate interaction with down-stream
organizational systems. As we move towards this end of the COTS customization spectrum, the
level of complexity and risk introduced by configuration changes, and the integration and
extension requirements, call for full project management activities and a dedicated team of people.
Figure 5 depicts this relationship for different COTS customization styles.
Target Implementation Style
Given the different customization styles, an Enterprise COTS implementation often involves
configuration changes, integration with existing systems, and extensions to interface with legacy
systems. Considering the complexity of this implementation process, restricting the investigation
focus to a public-sector implementation following an agile development approach further enhances
Integrated&
Extended
Integrated
Configured
Out of the Box
Complexity
Page 67 of 307
the possibility of contribution towards existing knowledge in this context. The review of IS
implementation literature in chapter 2 indicates a general tendency towards phase-specific
investigations that follow a horizontal line of query. A few researchers, however, took a mixed
approach by combining both horizontal and vertical analysis in their investigations. Due to the
context and selected implementation methodology, the current research aims to focus on both
depth and breadth by identifying low-level activities and major phases of an enterprise COTS
implementation. To further clarify the contribution of the current research and present a basis for
moving the analysis forward, an initial assessment of the target organization’s existing COTS
implementation practices has been conducted. This helped identify a general model that is
followed for most COTS implementations. Using this as a base model, the analysis of the primary
data will be conducted to identify the actual implementation process followed by project teams for
all selected cases.
Historic Implementation Practices within the Target Organization
Target organization for the current research maintains a policy requiring all project-related
documents as well as knowledge documents be stored and tracked using the in-house electronic
document management system (IBM’s electronic document management systems, EDM P8). This
practice is strictly enforced within the organization, which makes the EDM P8 a valuable data
source to begin with. In order to identify the base model or default model of COTS implementation,
we analyzed the electronic document repository (EDM P8) for 22 projects, and identified a basic
IT implementation model that is followed within the organization. This gave us a starting point for
the present research. Tables 7 and 8 provide a summary of the projects that were studied to identify
this model:
Page 68 of 307
Table 7: Analyzed project summary
Project
Type
Budget Timeline Impact
Number of
Projects
Percentage of
Projects
Type I 100K Up to 3 months Work unit /
Department
6 27.3
Type II 100K to 500K 3 months to 1
year
Cross Dept. /
Functional
8 36.4
Type III 500K to $1M 1 year to 2 year Cross-function /
Org. wide
5 22.7
Type IV 1M or Above 2 years or more External 3 13.6
Table 8: A Sample of COTS based projects analyzed
COTS Product / Project COTS Description/ Usage
EDM P8 IBM’s electronic document management system used
for enterprise document management
Microsoft Dynamic CRM Customer Relationship Management
Vizor Used for Regulatory Reporting System
Dollar Universe Enterprise Job Scheduling
Sirsi Library catalogue management
Markit’s Cadis Used for Market Data Management
Planview Enterprise reporting of work hours and resource/work
allocation
A.K.A Museum Artifact management and tracking
Maitre D'assystance Enterprise Incident and Service request/ ticketing
system
Based on the repository analysis, a general IS implementation model has been identified for the
host organization. This model consists of seven phases: initiation, requirement and planning,
analysis and conceptual design, design and architecture, development, delivery, and close out. This
initial model is shown in Figure 6.
Figure 6 : COTS implementation phases
Page 69 of 307
A closer look at Figure 6 reveals several major discrepancies between the initially identified COTS
implementation model and our proposed research topic. The current thesis aims to investigate a
micro-level implementation model for an enterprise COTS following an agile development
practice. However, the identified model indicates a sequential or waterfall implementation model,
comprising of higher level project phases. Although the notion of major stakeholders has been
incorporated into Figure 6, due to a higher level abstraction it is not possible to capture the
stakeholder engagement aspect here. Additionally, the model is also incapable of incorporating the
rich information related to information flows, vital processes and CSF. The primary and secondary
data analysis will particularly focus on these aspects while enhancing this preliminary model of
COTS implementation. Furthermore, to fulfill one of the primary objectives of this research,
implementation related processes will be analyzed to determine their relationship with the CSFs.
3.2 Stakeholder engagement in COTS implementation
While the significance of stakeholder engagement is well established in the literature,
organizational stakeholder orientation is not. Stakeholder theory is dominated by its normative
core; as Jones and Wicks (1999) write, “the interests of all stakeholders have intrinsic value, and
no set of interests is assumed to dominate the others”. This is closely aligned with Freeman’s
(1984) original call for managerial attention to all stakeholder interests as a vital success factor,
and which bear responsibility for the implications of organizational actions. However, as
organizational or managerial decisions have implications for the stakeholders of an organization,
stakeholders likewise can affect the organization. This reciprocal relationship, which often forms
the core notion of strategic stakeholder management, has been largely neglected in existing
stakeholder theory literature (Fassin, 2012).
Page 70 of 307
In a public-sector COTS implementation, stakeholder taxonomy is often significantly different
from a private or profit-oriented organization. Besides primary stakeholder groups like employees,
customers, partners, vendors and government, there are also indirect stakeholders, like civic
society and pressure groups who defend the interest of specific stakeholder groups, which thus
gives rise of the notions of reciprocity, loyalty, and fairness towards the organization by
stakeholders. Besides the possibility of an instrumental approach adopted by organizations, the
ethical dimensions of stakeholder theory could very well be abandoned by pressure groups and
stake watchers when the strategy formulation is guided by a political resource perspective (Fassin,
2012).
Besides the complexity of stakeholder taxonomy, reciprocity concerns clearly highlight the need
for a systematic understanding of organization stakeholder orientation during COTS
implementation initiatives by public-sector organizations. Earlier research on stakeholder
orientations defines the concept in terms of allocated resources (Berman et al. 1999), such as time
dedicated to certain activities. During COTS or IS implementation, the allocated time for certain
tasks clearly has implications for all stakeholders, as a positive outcome for one group may not be
perceived as such by all other stakeholders. Time allocation is only one aspect of stakeholder
orientation, and other aspects of resource allocation – such as amount, basis, and target – should
also be considered as “attributes” of organizational stakeholder orientation, which together
represent the notion of “stakeholder sensitivity.” This forms a core dimension of an organization’s
stakeholder orientation.
A joint-effort COTS implementation involving multiple government organizations and multiple
vendors has a much broader scope in terms of affected parties and people, compared to an internal
IS implementation by a single organization. Since this type of implementation crosses
Page 71 of 307
organizational borders, it can also imply a loose association among stakeholders (Boonstra, 2008),
which makes the proper identification and engagement of stakeholders during various
implementation phases absolutely critical for success. Considering the scope of investigation and
the breadth of the possible stakeholders, the current research chose to focus only on the primary
stakeholders who were directly engaged or affected by the implementation process of the COTS
implementation project. This approach is also consistent with Boddy and Buschanan’s (1986)
definition of organizational information system stakeholders: “All those who have a practical
concern for the effective application of new technologies, and who are in a position to take or to
influence decisions about why and how they are used”.
3.2.1 A Framework towards stakeholder engagement
Although Donaldson and Preston (1995) proposed that the core of stakeholder theory lies in its
normative dimension, from a COTS or IS implementation success perspective, instrumental and
descriptive versions of stakeholder theory appear to be more dominant in the literature, due to the
nature of the context. For the current research, we have also aligned ourselves with the descriptive
approach (Donaldson & Preston, 1995) to identify the salient and key stakeholders, and identify
the engagement processes pertaining to a COTS/IS implementation.
As indicated in the previous section, the identification of legitimate stakeholders is a challenging
task in many cases because of the extensive variations among different stakeholder classification
schemes that exist in the literature. Because of temporal restrictions on our study’s scope and the
exclusive focus on the COTS implementation process, we found a question-based stakeholder
Page 72 of 307
identification scheme most suitable for the purpose. A question-based framework, such as those
presented by Pouloudi and Whitley (1997) and Cavaye (1995), usually asks relevant questions
concerning the IS and its nature to identify the key stakeholders. Table 9 shows the adopted
question-based framework for identifying major stakeholder groups for the selected COTS
implementation cases.
Table 9: Stakeholder identification
Relevant Question Identified Stakeholders* Category**
Who are the initiators of the
system?
Who are the sponsors of the
system?
Who have to adopt the system and
make it work?
Who are the intended users?
Who will receive the output of the
COTS/IS?
Who are the intended developers
and operators of the COTS/IS?
Who will be impacted and affected
by the system?
Who will win or lose by using the
COTS/IS?
Page 73 of 307
* Indicates the major stakeholder groups such as client, vendor, internal support unit, project sponsor, project team member and so on. ** Indicates classification for the identified stakeholder such as internal, external, primary or secondary stakeholders.
3.2.2 Stakeholder sensitivity and stakeholder engagement
Stakeholder management is one of the dominant themes in stakeholder research and yet
stakeholder engagement – an aspect of stakeholder management – is often taken for granted or
ignored. Achterkamp and Vos (2008), and Brown and Jones (1998) have found that project failure
cannot be always attributed to ineffective project management practices; rather, inappropriate
social interactions among project stakeholders often can cause projects to fail. Some authors have
oscillated between involvement and engagement by using the attribute of reciprocity or mutual
benefit. Macleod (2012) and others have pointed to care and commitment as the determinant of
engagement levels. In an attempt to clarify the term “engagement,” Pushor (2007) explains: “In
comparison to involvement, engagement comes from en, meaning ‘make,’ and gage, meaning
‘pledge’ – to make a pledge (Harper, 2002), to make a moral commitment”.
Stakeholder engagement can be depicted as the organizational effort to involve relevant
stakeholders in a positive manner through exchange and cooperative relationships. Stakeholder
literature from business ethics, social accounting, and human resource management indicates that
stakeholder engagement typically relates to themes such as responsibility, managerialism, and
social control and construction (Greenwood, 2007). All three are closely related to the power and
authority possessed by different groups. Moreover, Pushor and Ruitenberg (2005) have argued that
flattening the command structure of the organization by sharing power and authority among
involved parties can lead to higher engagement driven by mutual benefits. Missonier and Loufrani-
Page 74 of 307
Fedida (2014) have proposed a conceptual approach towards stakeholder engagement anchored in
Actor-Network Theory, which acknowledges the dynamic and emergent nature of the relationships
among various stakeholders in a project. In addition, Missonier and Loufrani-Fedida (2014)
identified three aspects that can effectively contribute towards a solid understanding of stakeholder
engagement: (1) problematization – framing the issues of interest and identifying how they affect
the actors, (2) interessement and enrolment – assigning roles to stakeholders and the acceptance
of the assigned role by the stakeholders themselves, and (3) mobilization – reaching agreement
among stakeholders in terms of actions. These findings, however, are in agreement with Pushor’s
(2007) argument, as all three processes combined elucidate the mutual benefits for all stakeholders,
which in turn drives engagement level. Complementary to stakeholder engagement is the concept
of stakeholder sensitivity, by which an organization endorses the normative core of stakeholder
theory. Mitchell et al.’s (1997) conceptualization of stakeholder salience – based on power,
urgency and legitimacy – is one of the most significant works contributing to and understanding
of sensitivity. By stakeholder sensitivity, we denote an organization’s attitude towards the
definition of stakeholder-scope in terms of size, which of course is not a static construct.
A micro-level examination of stakeholder engagement, by focusing on a COTS implementation
project, could reveal an instrumental application (Donaldson and Preston, 1995) of stakeholder
orientation, in addition to the more prevalent normative applications. For example, stakeholder
engagement in IS implementation projects is often used as an instrument for advancing project
goals rather than aligning stakeholder interests. Therefore, it is hoped that the development of such
a framework to identify proper stakeholder orientations for a COTS implementation, by analyzing
Page 75 of 307
successful projects, can provide a way to more accurately define the applications of a stakeholder
approach.
Considering typical phases and commonly observed activities belonging to each phase of an
enterprise COTS implementation, a distinct combination of engagement and sensitivity appears to
be a practical phenomenon. Inspired by this observation, we propose a four-quadrant model
ranging from a high engagement-sensitivity level to low engagement-sensitivity, as presented in
Figure 7.
B A C D
Figure 7: Stakeholder sensitivity and stakeholder engagement
However, it must be emphasized that the themes related to each quadrant of this model will be
determined through a grounded research approach, where the analysis of primary and secondary
case data is the source of all distinct themes, if any. This new model has been proposed specifically
High Stakeholder Sensitivity
Low Stakeholder Sensitivity
High Stakeholder
Engagement Low Stakeholder
Engagement
Page 76 of 307
to help plot the interactions between the key variables of “stakeholder engagement” and
“stakeholder sensitivity.”
Stakeholder engagement, as stated earlier, can be depicted as the organizational effort to involve
relevant stakeholders in a positive manner through exchange and cooperative relationships.
Stakeholder sensitivity, on the other hand, is manifested through an organization’s attitude towards
the stakeholder-scope in terms of size and the level of influence on organizational decisions.
The x-axis of the model is labelled as “stakeholder engagement.” As indicated above, engaging
stakeholder groups in an initiative or task may require a very different set of routines than simply
considering them. In an enterprise COTS implementation project, this engagement often calls for
task performance, processes of consultation, communication, dialogue, and exchange. The
intensity of the engagement can be measured using the frequency of activities related to these
categories.
The y-axis of the model is labelled stakeholder sensitivity. Stakeholder sensitivity can be seen as
a proxy for the responsible treatment of stakeholders, or stakeholder agency in Greenwood’s
(2007) model. This construct largely indicates the breadth of the various stakeholder groups that
are considered at any given stage of the COTS implementation process and their perceived
influence on the outcome of that stage. Similar to the level of engagement, stakeholder sensitivity
can also vary considerably from one stage to another of an COTS implementation project.
Maintaining an optimal level of sensitivity versus engagement is extremely critical for COTS
projects, to prevent a certain category of stakeholder from moving to a different category; for
example, a “dormant” stakeholder moving to the category of “dangerous” stakeholder and
jeopardizing the success of an IS project. This dimension reflects the organizational attitude
Page 77 of 307
influenced by ethical and moral values. A “low” sensitive view may include only contractually
related stakeholders, such as clients, vendors, owners, etc., while a “high” sensitive view might
consider all the stakeholders impacted by the system; this can even include the general population,
who may eventually be impacted by the COTS system in some distant future.
Inspired by the concept of “optimal trust” (Wicks et al., 1999), we also suggest the possibility of
an optimal level for both dimensions at each quadrant. However, this concept of “optimal level”
can be intensely dependent on the context of engagement. Particularly for COTS implementations,
a certain level of stakeholder sensitivity deemed optimal for one stage could act as a source of
confusion and chaos for another stage. Therefore, in the proposed engagement-sensitivity model,
the dotted lines dividing each for the four quadrants are indicative of a balanced combination.
Considering the presence of stakeholder engagement and stakeholder sensitivity, we argue that a
different combination level of these two constructs will lead to distinct stakeholder orientations
for each of the implementation phases of the project. A qualitative analysis of the primary data
will help identify these distinct stakeholder orientations for the current COTS implementation
cases and their relationship with the CSF.
3.3 Control balancing in COTS implementation
Control balancing, a lightly investigated aspect within control literature, is one of the key focus of
the current research. Control balancing in COTS or IS implementation implies a targeted
adjustment of the control configuration. The following sections further elaborate the underlying
constructs of a control configuration to facilitate the understanding of control balancing in a COTS
implementation.
Page 78 of 307
A micro-level analysis of enterprise COTS implementation is expected to reveal several key
activities for each implementation phase. The completion of these tasks within a specified time
and with an expected quality is a difficult endeavour that requires the application of well-planned
control configuration. The examination of the control configuration for a multi-party enterprise
COTS implementation is thus valuable because of its underlying relationship with stakeholder
groups. Since each of the activities at different phases of the implementation process is driven by
one or more individuals, various types of controls are applied by the project to align individual
behaviours with organizational objectives (Kirsch, 1996). As identified in the literature review
section, earlier control research related to IS implementation largely focused on control
antecedents (Kirsch et al., 2002; Kirsch, 1996, 1997), the effects of control (Nidumolu and
Subramani, 2003; Henderson and Lee, 1992; Maruping et al., 2009), and the dynamics related to
control choices (Kirsch, 2004; Mähring, 2002; Choudhury and Sabherwal, 2003).
The multiple clients and multiple vendors involved in a COTS implementation are expected to
bring significant changes to the control landscape of the project. This has been indicated by
Heumann et al. (2014), who state that:
existing studies almost exclusively focus on one specific control dyad (e.g.,
Henderson and Lee, 1992; Kirsch, 1996, 1997; Kirsch et al., 2002, 2010), and those
studies that look at multiple dyads seem to ignore the hierarchical position of
controllers (Soh et al., 2011). As a consequence of this lack of multi-level analyses,
we do not know whether control actions…
Page 79 of 307
This hierarchical structure is a vital theme for the current thesis, as the solution integrators are
often engaged in a contractual relationship with the COTS vendor. Additionally, a horizontal
collaboration structure among clients also existed at the same time. To build upon our earlier
argument related to the existence of distinct stakeholder orientations, we also like to investigate
the possibility of distinct control configurations. In order to facilitate such an examination, we rely
on the theory of control balancing (Gregory et al., 2013) for this context.
Gregory et al. (2013) define control configuration utilizing three distinct dimensions related to
control: control types, control degree, and control styles. Control types are typically related to
attaining certain goals and can be classified into three different categories: a) Procedural controls
that are applied to improve efficiency and effectiveness, such as tracking project milestones, daily
status reports, definition of roles and responsibilities etc.; b) Social controls that are applied to
develop a shared understanding among project partners through social and team-building
activities, informal information exchange, intercultural workshops, etc.; and c) Hybrid controls
that combine both procedural and social controls in order to achieve both goals simultaneously.
Hybrid controls may include common understanding workshops, site visits, and reflection-in-
action sessions.
The second dimension, control degree, indicates the measurement or intensity of the applied
control. Control degree can vary from tight to relaxed, where tight indicates a high volume or high
intensity of control, and relaxed indicates the opposite. The final dimension, control style, is related
to the concept of controller versus controllee roles and direction of the control application. Typical
control styles are either: a) Unilateral, where one party, typically the client, controls the other
party, typically the vendor: or b) Bilateral, where the client and the vendor use control mechanisms
jointly with an emphasis on mutual agreement of control selection and use.
Page 80 of 307
Based on the above definition of control configuration, Gregory et al. (2013) proposed three
distinct control configurations: a) authoritative, b) coordinated, and c) trust-based. The theory of
control balancing argues that a dynamic nature of control configuration for an IS implementation
project – which is driven by the level of shared understanding among the partners (Gregory et al.,
2013), and is the application of the control theory in our current context – demonstrates great
potential to validate and extend the concept of control balancing. As the current thesis proposes
the existence of dynamic stakeholder orientations during a COTS implementation project, it is also
possible to identify the existence of multiple control configurations through an examination of
different implementation phases.
We have utilized the control configurations developed by Gregory et al., 2013 (presented in Table
10) to facilitate the analysis of the primary data and identify the application of control balancing
in the current case.
Table 10: Control Configurations
Control
Configuration
Definition Meaning and Implication Typical Examples of
Control Mechanisms
Authoritative
control
• Procedural
control portfolio
• Tight control
degree
• Unilateral
control style
An approach to
control based on a
traditional
client–vendor
perspective
Client–vendor perspective:
• Means that exchange parties see
themselves as client and vendor with
clearly separated roles and
responsibilities
• Implies that the client specifies
requirements, the vendor delivers upon
them, and control selection and use are
driven by the client
• Results in situations in which the client
dominates the relationship from a
managerial perspective
• Status reviews
• Detailed examination of
deliverables
• Tracking of project
goals
• Definition of
client/vendor roles and
responsibilities
• Operational process
documents
Page 81 of 307
Coordinated
control
• Hybrid control
portfolio
• Tight control
degree
• Bilateral control
Style
An approach to
control based on a
coordination
perspective
Coordination perspective:
• Means that exchange parties see
themselves as partners that need to
closely coordinate activities
• Implies that client and vendor work
toward coordinated, shared goals, search
jointly for problem solutions, and select
and use control mechanisms accordingly
• Results in situations in which neither
the client nor the vendor dominates
• Joint parallel testing
approach
• Site visits
• Workshops
• Reflection-in-action
sessions
• Coaching of team
members
• Joint communication
plan
• Lessons learned
sessions
• Feedback mechanisms
Trust-based
control
• Social control
portfolio
• Relaxed control
degree
• Bilateral control
Style
An approach to
control based on a
trust-based
perspective
Trust-based perspective:
• Means that exchange parties see
themselves as part of the same team
based upon mutual trust and shared
understanding
• Implies that the vendor delivers without
the client getting deeply involved in the
process, and fewer resources are
dedicated to controlling the vendor
• Results in situations in which problems
are solved instantly, new ideas for
improvements are generated, and the
vendor takes over more responsibility in
the relationship
• Direct and pragmatic
coordination
• Brainstorming sessions
• Spontaneous
communication (e.g.,
facilitated through open
plan offices)
• Informal exchange of
ideas
3.4 Mapping the research questions and objectives
This research investigates an enterprise COTS implementation project by a government
organization from multiple perspectives. Both the review of the relevant literature and the
preliminary analysis of the organization practices indicate a significant knowledge gap and
Page 82 of 307
potential for contribution by identifying a micro-level COTS implementation model. Additionally,
a task-based analysis of implementation phases can help reveal critical information flows, inter-
phase relationships, processes and tools that can all be utilized to achieve different project
objectives.
Secondly, adding a CSF perspective to the current thesis will facilitate the study of multiple related
concepts, such as implementation processes, stakeholder orientation and control balancing. As the
review of existing CSF literature indicates, most CSF related to IS implementation can be mapped
to people, process or environment, and a micro-level implementation model will help identify the
process and tools that are directly connected to the management of various CSFs during the
implementation. Although CSFs (and their relationships to stakeholder engagement and control
balancing) are a major aspect of the current thesis, the identification of CSFs is not. Therefore, we
will be relying on the subset of CSFs identified through each project’s lessons learned activity as
selected candidates for examination through the current research. This list of CSFs, identified by
the implementing organization, will then be further enhanced by the CSF identified by existing
research in this domain.
Thirdly, CSFs belonging to the “people” category often appears to be a salient dimension in IS
implementation research. Our reliance on stakeholder theory and the theory of control balancing
draws attention to this particular aspect of existing IS implementation literature. More specifically,
the possibility of multiple stakeholder orientations and multiple control configurations can
potentially have a direct impact on various CSFs. This addresses the concern identified in our
fourth and sixth research questions, which focus on the management of CSFs through stakeholder
engagement and control balancing.
Page 83 of 307
Finally, a naturally related question to the aspect of control balancing – “what causes it?” – is
presented in our third research question. Gregory et al. (2013) identified the deterioration of
“shared understanding” as a possible trigger factor for the control configuration to change. For a
complex multi-case investigation like the present one, it is possible to have multiple trigger factors
responsible for the dynamic nature of control balancing. Therefore, investigating the possible
causes of control balancing can be seen as an extension of and contribution to the relevant theory.
Table 11: Situating Research Questions
Research Objectives Research Questions
1 To identify a micro-level model for
enterprise COTS implementation by
focusing on implementation phases,
processes, and tools.
What is an activity and process-based model
for enterprise COTS implementation?
2 To identify a set of CSFs that are relevant
for enterprise COTS implementation and
examines the management aspect of them
through process, stakeholder engagement
and control balancing.
How can organizational processes and tools contribute
to COTS implementation success?
How can CSFs and other challenges be successfully
managed through optimal stakeholder engagement by
the project?
How can CSFs and other challenges be
successfully managed through optimal
control balancing by the project?
3 To develop a theoretically sound framework
that can be used to identify an
organization’s stakeholder orientation
during an enterprise COTS implementation
project.
What is nature of stakeholder orientation during a
COTS implementation for a public sector IS
implementation?
4 To validate the theory of control
balancing (Gregory, Beck, & Keil, 2013) by
examining the dynamic relationships among
the project partners from one phase to
another phase of the implementation
process.
What is the nature of control configuration in
a multi-partner COTS implementation and
what are the factors responsible for the
application of control balancing?
Page 84 of 307
4.0 Research methodology
Chapter 4 describes the research approach, design, and method that have been used to investigate
the research questions stated at the beginning of this thesis.
4.1 Overall approach
Current research is a grounded theory research based on multiple empirical cases. The basic
approach adopted for determining an agile COTS implementation and its relevance to stakeholder
orientation as well as control balancing is qualitative. We used interviews as the primary source of
data. A qualitative approach is selected to understand the meaning that “individuals …ascribe to
social or human problem” (Cresswell, 2009:4). In this research, identification of a micro-level or
detailed COTS implementation model and whether the critical success factors can be effectively
managed through adjusting stakeholder orientation and control configuration are the investigated
problems related to the successful implementation of an enterprise COTS. The meaning ascribed
by individuals is relevant in this case because the concept of success may be subjectively perceived
as well as objectively measurable.
4.2 Research design
The design for the current research uses multiple case studies to inquire about the actual process
of COTS implementation followed by a Canadian government organization. We have analyzed a
total of four cases through current research which includes two successful and two failed
implementations of an enterprise COTS product. A failed or ‘not so successful’ project can be
defined by looking at multiple criteria such as a lower return on investment (ROI), higher support
cost, user satisfaction, consideration for possible replacement etc. The inquiry also investigates the
Page 85 of 307
nature of stakeholder engagement and control balancing, and how they influenced the related CSFs
for the selected projects.
Over the past decades, case studies have gained considerable acceptance as a business research
method, particularly where holistic examination of complex phenomena in real-life settings is
required (Benbasat et al. 1987; Yin 1994). Due to its flexibility and individual variation, case study
has been consistently gaining popularity in IS research (Cavaye, 1996). In a case study research,
it is not only possible to combine several qualitative data collection methods such as interviews,
documentation, and observations but also include quantitative data such as questionnaires and time
series to provide richness and flexibility to the research process.
A case study research design can vary according to the type of case study one undertakes. Firstly,
akin to other research methods, the case study research can fulfill several purposes, the most typical
being exploration, description, and explanation (Post and Andrew, 1982; Babbie, 2001; Yin,
2003). Each of these three purposes is distinct and is employed to study different phenomenon. A
descriptive case study could provide us with a detailed description of a novel phenomenon, whilst
an explanatory case study could offer an aspect of sense-making by explaining why an event
occurred (Post and Andrew, 1982; Yin, 2003). The third type of case studies, an exploratory study,
usually employed to better understand a previously unexplored phenomenon (Babbie, 2001).
Considering the scope and the research questions posed earlier, both descriptive and exploratory
aspects are salient throughout the investigation approach taken for the current research.
Page 86 of 307
A case study design is appropriate because the phenomenon of interest, agile COTS
implementation in public sector, is a contemporary one that must be observed “within its real-life
context” (Dube and Pare, 2003:598). A case study design is also appropriate for the current
investigation as the research questions include ‘how’ and ‘why’ questions (Yin, 1994).
Additionally, such a complex phenomenon of enterprise COTS implementation is not suitable for
experimental inquiry simply because of the employee would perform such activities only as a part
of their day to day jobs not under controlled conditions.
“A case study is an empirical inquiry that investigates a contemporary phenomenon
within its real-life context, especially when the boundaries between phenomenon
and context are not clearly evident” (Yin, 1994:13).
Several earlier researchers acknowledged that immersion in rich data from multiple sources is a
strength of the case study method that makes it a valuable research method for developing
emergent theory, or for sharpening existing theory (Post and Andrew, 1982; Babbie, 2001; Voss
et al. 2002; Eisenhardt and Graebner, 2007; Siggelkow, 2007; Weick, 2007). The theory building
strengths of a case study research provide further motivation to use this method for our descriptive-
exploratory research which aims to refine and elaborate existing theories.
To investigate a phenomenon, a case study research can adopt either a single case or multiple cases
approach. (Yin, 2009; Eisenhardt, 1989). Single cases are usually the most common and simpler
approach for case study research and are often chosen to investigate rare, unusual or critical
circumstances (Yin, 2009).
For the current research, a multiple case study design is adopted because multiple cases are
required to adequately examine a COTS implementation process and develop a theory elaboration
Page 87 of 307
or extension. A multiple case study approach provides us with a broader access to the empirical
observations and an opportunity to collect sufficient data. Both of these aspects can support the
development of a valid theoretical model. The value of a multiple case approach, in highlighting
repeatable phenomena, has been previously argued by Leonard-Barton (1990) and Zivkovic
(2012). While developing robust and reliable theoretical constructs, a multi-case approach is also
considered more suitable over a single case approach by the wider research community (Bourgeois
and Eisenhardt, 1988; Yin, 1994; Dube and Pare, 2003).
A multi-case research increases the internal validity of identified constructs and proposed relations
through recursive pattern matching between theory and multiple sets of empirical evidence. This
is achieved through examining the same phenomenon across different contexts or environments
(Campbell, 1975; Bourgeois and Eisenhardt, 1988; Pare, 2004; Gibbert, Ruigrok, and Wicki,
2008).
In addition, a multiple case study research is often used to maintain external validity of a study
and ensure the generalizability of research findings by examining the same phenomenon in
different organizational context (Yin, 1981; 1994; 2003; Gibbert, Ruigrok and Wicki, 2008). Yin
(2009) states:
“Multiple cases resemble multiple experiments [... ] that means each case must be carefully
selected so that it (a) predict similar (a literal replication) or (b) predicts contrasting results but for
anticipatable reasons (a theoretical replication).”
Yin (2009) also suggests that a few cases (two to four) design would be ideal for literal replications,
whereas a few additional cases (four to six) can be used to pursue two different patterns of
theoretical replications. Since the current research is aiming more towards the literal replication of
Page 88 of 307
the findings, a sample size of four has been selected for this research. The primary reasoning for
selecting both successful and failed implementations is to eliminate any environmental and/or
organizational biases relating to the CSF.
4.3 Research method
Primary data was gathered through face-to-face semi-structured interviews. Informants selection
process took necessary precautions to ensure a representative population as a sample. To support
this, we categorized the stakeholder population into different role groups. Typical role groups
included project manager, project sponsor, end user, developer, solution architect, vendor, business
analyst, technical/enterprise architect etc. Interview participants included one or more individuals
from each role groups involved with the implementation of a COTS product. The focus of
interviews revolved around the implementation process, activity details, stakeholder
considerations, CSF, and control mechanisms.
Although a survey as a research instrument would be much more economical and capable of
reaching a large number of informants in comparison to an interview method (Cresswell, 2009),
we chose to utilize interviews because it provides the ability to explore an issue in depth.
Interviews also help maintain a balance by allowing interviewees to express their opinions,
concerns, point of views in their own words and enabling the researcher to guide the line of
questioning (Cresswell, 2009).
An observational method is also judged inappropriate because an observation can only be applied
during the implementation process and not upon the completion of the project. Since our data is
collected after the project go-live stage, to ensure a proper selection of both successful and failed
COTS implementations, using an observational method is not possible. In addition, an observation
Page 89 of 307
based approach is unlikely to allow the researcher understand the motivation behind using different
tools, processes and their interconnections with stakeholder orientation and the management of
CSF.
The key steps followed by this research are summarized in Table 12.
Table 12: Research Steps
Step Activity Expected Outcome
1 Review
Literature
Critical review of relevant literature which include: IS implementation, Critical
Success Factors, Stakeholder theory, and Control balancing. Provides guiding
theoretical base (Eisenhardt & Graebner, 2007; Layder, 1993, Yin, 1994)
2 Synthesize
Literature
Deductively related the existing theories and develop a tentative, a priori
theoretical link (Layder, 1993). Constructs and other tools identified from the
literature to facilitate data collection and analysis (Christensen, 2006).
3 Pose Research
Questions
Identification of key research questions, justified by the research gap and
literature synthesis (Layder, 1993; Yin, 1994).
4 Design
Interviews
Checklist to develop semi-structured interview questions, tentative interview
schedule, other process and tools to assist proper execution of a qualitative
interview.
5 Select Sample Identification of employees, vendors, partners and management who are
associated with the COTS implementation process related to the selected cases.
Focus on cases “chosen for the likelihood that they will offer theoretical insight”
(Eisenhardt & Grabner, 2007:27).
6 Conduct Interviews Audio records from the interviews, memos, contact summary sheet, memos,
transcriptions etc. They all are collectively the qualitative data set resulting from
the interview process.
7 Analyze Data Identification of process and multilevel codes from the data, identification of
anomalies and insight from the data that are not identified or explained by the
linked theories from step 2 ( Bryman & Bell, 2003; Christensen, 2006).
8 Refine Theoretical
Link
Based on the new insight or anomalies, modify the linked theories from step 2
(Carlile & Christensen 2005).
4.3.1 Review literature
Page 90 of 307
This thesis started with a review of the literature and identified possibilities of theory elaboration
as well as theory extension in chapter 2. The literature review provides “a rich theoretical
framework” (Yin, 1994:13) as the foundation for the investigation where the “existing theories
play a major part in the formulation of the new theory” (Layder 1993: 25).
The literature review follows a deductive process using existing theories and identifies potential
relationships between theories and phenomena (Weick, 1989). A review of the related literature
also provides the overall objectives for the research (Eisenhardt and Grabner, 2007) and guide the
interview design in subsequent stages of the research (Cresswell, 2009, Yin, 1994).
4.3.2 Synthesize literature
A link among stakeholder theory, control balancing theory and critical success factors is proposed
in Chapter 3. This link is derived through a review of existing CSF literature. Our review indicates
that most of the CSF identified through IS implementation research are related to people, process
or environment (Section 2.5 and Section 3.4) of an organization. Leveraging both the stakeholder
theory and the theory of control balancing, we acknowledge the existence of distinct groups of
CSF related to a COTS implementation. Subsequently, we reviewed existing literature to explain
how an enterprise COTS implementation can have multiple stakeholder orientations which, in turn,
can require multiple control configurations to successfully manage relevant CSF.
Constructs, processes and other tools were developed from the review of the literature in order to
facilitate the data analysis process (Christensen, 2006). These include:
1. Process related to IS and COTS implementation that will be used to guide the current study
of implementation process and contribute both horizontally and vertically to the existing
models in literature
Page 91 of 307
2. Definition of stakeholder engagement to contrast it with stakeholder sensitivity; a question-
based framework for identifying stakeholders.
3. A systematic way to categorize and study the critical success factors for IS implementation
process and relating them to stakeholders.
4. A framework to analyze control configurations in order to examine the control balancing
efforts during a COTS implementation.
4.3.3 Map research questions
The key research questions that will be investigated through current thesis are posed in Chapter 3
of this document. Section 3.1 delves deeper into the enterprise COTS implementation process to
identify our existing knowledge concerning implementation models, processes, and tools in this
context (research questions 1 & 2). Section 3.2 examines the engagement of stakeholders during a
COTS implementation process and sets the stage for probing organizational orientation towards
the stakeholders in such context (research questions 5 & 6). This section also proposes a potential
relation between the stakeholder orientation and CSF. Section 3.3 presents the concept of control
balancing to manage CSF during a COTS implementation (research question 4). In addition, this
section highlights the significance of factors influencing control configurations and the possibility
of multiple control configurations (research question 3).
4.3.4 Design interviews
Interview for the current research focused on four key areas directly related to six research
questions posed in Chapter 1 (section 1.3) and Chapter 3 (section 3.4) of this document. They are:
a) COTS implementation process and activities, b) management of stakeholders, c) CSF for the
implementation, and d) maintaining control throughout the project.
Page 92 of 307
Enterprise COTS implementation is a “situated activity” in Layder’s terminology where the firm
provides the “settings”. Potential interview questions are shown in Table 13 based on Layder
(1993).
Table 13: Potential interview questions
Situated Activity Checklist Potential Questions
“Who is doing what, to whom in this episode or strip of
interaction?”
What phases of enterprise COTS implementation the
project document followed?
What are the tools used to monitor implementation
progress of the project?
“What are the recurrent features of the behaviour and
interaction?”
How often a given control measure was used?
“What social functions do these patterns of behaviours
and forms of interaction serves? Are they intended or
unintended by the participants?”
Does an approach towards stakeholder align or
supports certain organizational mandate?
Was a given control able to minimize certain
unexpected outcome or risk?
“What forms of communication are being used?” How information was shared among the stakeholders?
How decisions and escalations were handled?
“What aspects of the setting are pertinent to the
analysis of particular episodes of activity? How do they
influence the actions?”
How decisions were implemented?
4.3.5 Select sample
For the current research, we investigated four different cases of COTS implementation. The
following criteria were used to select the organization, cases, and interview participants.
Since the current research aimed to investigate enterprise COTS implementation for the public
sector, our target organization must be a Canadian government department. Additional factors
such as multiple clients, and multiple vendors were also considered as selection criteria for the
projects. An overall project budget between 10 and 20 million dollars and a project duration
Page 93 of 307
between one and three years were used as guidelines to avoid selection of very small projects.
Enterprise COTS implies an interconnected and packaged software which is very different
compared to a stand-alone COTS product. This aspect was also considered while selecting the
sample cases for this research. A balanced selection of both successfully completed and failed
projects was also ensured. This resulted in the selection of two successful and two failed COTS
implementation projects for analysis. A failure for a completed project is often an extremely
difficult concept to define. This is because of the fact that a failure is often not apparent
immediately after the project’s completion. It only becomes visible after a COTS product has been
used for a few months. Therefore, we focused on “client dissatisfaction”, “lower than anticipated
ROI”, and “high support cost” to identify failed projects instead of relying on project terminations.
In addition, categorization for each selected project was reviewed by a business user of that COTS
product. The purpose of this review by an end-user was to confirm the validity of such
categorization and avoid any bias introduced by the researcher.
Initial interview targeted a total of 39 primary informants. Each interview lasted approximately an
hour and 20 minutes. This included a minimum of nine participants from each selected Case. Our
selection process considered people from different roles to construct a representative sample.
Participation to this face to face interview was voluntary and conformed to all ethical research
practices. For each selected case, number of interview participants and their roles are presented in
Table 14 through Table 17 below.
Page 94 of 307
Table 14: Case A Informants Summary
Type of Primary Data Description of the Informants Roles
Face to Face Semi-
structured Interview
Duration for each
Interview: 80 minutes
Role of Interviewee # of Interviews
with Project team
# of interviews
with vendor and
solution integrator
# of interviews
for Client/
Business Units
Project Manager 1 1
Business Lead 1
Business Analyst 1 1
Enterprise Architect 1
Data Architect 1
Solution Architect 1
Project team
member
2 1
Total # of
interviews
11
Table 15: Case B Informants Summary
Type of Primary Data Description of the Informants Roles
Face to Face Semi-
structured Interview
Duration for each
Interview: 80 minutes
Role of Interviewee # of Interviews
with Project team
# of interviews
with vendor and
solution integrator
# of interviews
for Client/
Business Units
Project Manager 1 1
Business Lead 1
Business Analyst 1 1
Enterprise Architect 1
Data Architect 1
Solution Architect 1 1
Project team
member
1 2
Total # of
interviews
12
Page 95 of 307
Table 16: Case C Informants Summary
Type of Primary Data Description of the Informants Roles
Face to Face Semi-
structured Interview
Duration for each
Interview: 80 minutes
Role of Interviewee # of Interviews
with Project team
# of interviews
with vendor and
solution integrator
# of interviews
for Client/
Business Units
Project Manager 1
Business Lead 1
Business Analyst 1 1
Enterprise Architect
Data Architect 1
Solution Architect
Project team
member
3 1
Total # of
interviews
9
Table 17: Case D Informants Summary
Type of Primary Data Description of the Informants Roles
Face to Face Semi-
structured Interview
Duration for each
Interview: 80 minutes
Role of Interviewee # of Interviews
with Project team
# of interviews
with vendor and
solution integrator
# of interviews
for Client/
Business Units
Project Manager 1
Business Lead 1
Business Analyst 1
Enterprise Architect
Data Architect
Solution Architect 1
Project team
member
3 1 1
Total # of
interviews
9
Page 96 of 307
In addition to primary interview data, we have also collected secondary data related to each
selected project. This was used to triangulate the research findings and enhance the validity of the
current research (Yin, 1994). Secondary sources of data are indicated in Table 18.
Tabel 18: Secondary Data Used for Triangulation
Types of Documents Purpose of Analysis
Meeting Minutes
Design Decision Documents
Project Planning documents
Test Cases and Plans
Analyze to determine the activities that took place on
certain phases, boundaries of the phases, links to other
phases, stakeholder considered and stakeholder
engaged, control configuration
4.3.6 Conduct interviews
Organizational approval and the approval of Carleton’s ethics committee have been obtained prior
to conducting the interviews for this research (please refer to appendix for approvals). The
participants were interviewed through a semi-structured interview:
“In semi-structured interviews the interviewer has a list of topics or questions that
that he or she wants to cover, although this list will be flexibly adhered to according
to the emergent demands of the interview situation. Semi-structured interviews are
designed to let interviewees respond in an open-ended way…..This individual’s
own interpretations and meaning are allowed to surface in the interview data”
(Layder, 1993:41).
Page 97 of 307
Interviews were conducted primarily to understand why certain process was followed, links
between phases, tools and processes that were utilized. Additionally, motivations for considering
‘who is a stakeholder’ and ‘what is a process’ are important and have been investigated. These
aspects directly relate to the investigation of organizational stakeholder orientation.
Interviews were conducted in person and were tape-recorded, with approval from the interviewer.
Interview data were used without identifying the interviewee. The interviewer took notes
immediately which served as ‘field notes’ and were used for guiding subsequent analysis of the
interview data.
4.3.7 Analyze data
With an aim to investigate the nature of control configuration, stakeholder orientation,
implementation processes and their collective relationship with CSF and generate a descriptive
and explanatory theory in this regard, we have resorted to a grounded theory method (Glaser &
Strauss, 1967; Martin & Turner, 1986; Turner, 1983). A grounded theory approach has been
extensively and effectively used for IS research in organizational context (Staehr, Shanks, &
Seddon, 2012), however, our primary motivations for adopting this approach are three:
First, is our ontological belief inclined more towards interpretivist approach as opposed to a realist
approach believing in an objective or external reality. This persuaded us to consider the perspective
of ‘social construction’ of reality by human actors in an IS implementation scenario (Berger and
Luckman, 1989). This led to our underlying epistemological belief that knowledge can be best
achieved by getting inside the world of those generating it (Orlikowski & Baroudi, 1991). As a
result, generative approach appears to be well suited for this purpose where actions and perceptions
of human stakeholders are investigated in a changing context of IS implementation. Therefore, a
Page 98 of 307
grounded theory approach is an ideal choice for our interpretivist approach as it "is an inductive,
theory discovery methodology that allows the researcher to develop a theoretical account of the
general features of a topic while simultaneously grounding the account in empirical observations
or data" (Martin and Turner, 1986, p. 141).
Second, COTS implementations investigated through this research is situated within an
organizational context. In addition, conceptualizing project as a ‘temporary organization’ adds
further complexity to the context. Although the role of context is not a primary premise for this
research, the contextual complexities introduced by the organization and the project must be
incorporated into an understanding of the phenomenon, rather than be simplified or ignored, to
produce accurate and useful results. This is also a major premise in of grounded theory research
(Martin & Turner, 1986; Pettigrew, 1990). Enterprise COTS implementation is a complex
phenomenon when considered from a process, stakeholder and control perspective and much of
the complexity often related to the context. As an isolated investigation of this phenomenon is not
very useful, we favour the use of a grounded theory method for the current research considering
the ability to incorporates the complexities of the organisational context (Orlikowski, 1993; Martin
and Turner, 1986; Pettigrew, 1990).
Lastly, grounded theory method originated with the aim to generate empirically grounded theory
(Glaser and Strauss, 1967), thus it is intended ‘to discover what is going on, rather than assuming
what should go on’ (Glaser, 1978:159) based on the systematic exploration of a phenomenon.
Therefore, supported by a rigorous, systematic and comprehensive approach to data collection and
analysis of multiple enterprise COTS implementations, our goal to identify useful theoretical
Page 99 of 307
conceptualisations, that explains the relationship between process, control, stakeholder orientation
and CSF, closely aligns with methodological purpose of a grounded theory.
To facilitate a rigorous, systematic and comprehensive analysis process, we have transcribed all
interview recordings. The transcribed interviews as well as the field notes taken during the
interview have been analyzed inductively (Bryman & Bell, 2003). As indicated earlier, the current
research follows a ‘Grounded theory’ approach for analyzing the qualitative data. Overall
grounded theory process followed by the current research is outlined in Table 12. Motivation to
use a grounded theory approach rises considering several aspects of an enterprise COTS
implementation. Identifying organizational stakeholder orientations and distinct control
configurations may help identify new constructs that drives these orientations or configurations.
However, these new constructs can only be identified through proper theoretical coding and theory
integration process for those codes.
Although we intend to respect the overall principle that grounded theory must explain the
behaviour being analyzed and fit the data collected from the case (Glaser and Strauss, 1967), we
primarily adopted the theoretical coding method used by Gregory, Beck and Keil (2013). Gregory
et al.’s (2013) theoretical coding and saturated category identification method is guided by
Glaser’s (1978, 1982) approach to the grounded theory research. Immediately after the problem-
formulation and case-study design phases, we performed an “open coding,” to identify various
indicators of stakeholder engagement and stakeholder sensitivity. In the next stage, we delimited
the coding process. The final three stages of the research process – theoretical coding, scaling up
and theoretical integration –focused primarily on establishing a theoretical connection between the
identified categories or orientations and the existing theoretical framework of the domain. Table
Page 100 of 307
19 outlines the details of the process that was followed while analyzing data and mapping them
into the core categories. This is different from the previously presented research steps in Table 12
as Table 19 focuses on the grounded theory process that was followed by the current research.
Table 19: Grounded Theory Research Process: Steps, Tasks and Outcomes
Research
Steps
Tasks Outcome
Problem
Formulation
Establishing the phenomenon in terms of
its practical relevance as a prerequisite to
produce grounded theory that has ‘grab’
(Glaser and Strauss 1967).
State what the problem is from a practice
and theory perspective and why it is
important (Van de Ven 2007).
Screening prior research to identify gaps
in the literature (Urquhart 2007).
Identified the enterprise COTS
implementation projects as a practically
relevant problem that many organizations
and project teams often struggle with.
Problem identified as COTS
implementation process, and managing of
CSF is often not well understood and
sources of struggle for the project team.
Identified gaps in the literature on COTS
implementation process, and management
of CSF through process, control
configurations, and stakeholder orientations
for enterprise COTS/IS implementations.
Multiple
case
study design
• Establishing engaged relationship
withpractitioners and negotiating access to
data (Pan and Tan 2011; Van de Ven 2007).
• Selecting a multiple candidate studies and
motivating the rationale for conducting a
multiple case study, e.g., the main criterion for
revelatory cases is “when an investigator has
an opportunity to observe and analyze a
phenomenon previously inaccessible to
scientific investigation” (Yin 2003, p. 42).
Obtained approval from Carleton’s ethics
committee to conduct an empirical study
containing human subject
Obtained approval from the leadership team
and reached an agreement with a large,
reputable government organization to
conduct a multiple case study of four
different enterprise COTS implementation
projects; Approval included access to both
primary data collection through face to face
interviews and secondary data collection
from organization’s electronic repositories.
Page 101 of 307
Selected “revelatory cases”: four instances
of enterprise COTS implementation
targeting four distinct operations areas of
the organization, which has been
inaccessible to scientific investigation
before (Choudhury and Sabherwal 2003, p.
313; Dibbern et al. 2008, p. 359).
Open coding
&
Data
collection
• Gathering rich primary and secondary data,
including intensive interviewing (Charmaz
2006).
• Coding the data and understanding what it is
about by going through interview transcripts
line by line, assigning conceptual labels to
data
segments, and identifying core categories
(Glaser 1978).
• Adhering to the principle of emergence of
grounded theory: categories should emerge
from the data in the sense that they must “fit”
(they must be readily, not forcibly, applicable
to and indicated by the data under study) and
“work” (they must be meaningfully relevant to
and be able to explain the behaviour under
study) (Glaser and Strauss 1967).
• Triangulating and comparing different slices
of data to find similarities and differences
(Charmaz 2006).
Conducted multiple in-person interviews,
and obtained project documentation such as
charters, gating decisions, project
management plan, meeting minutes,
escalations, contract related documents,
project completions reports and post-
deployment incident/problem records.
Generated more than 240 initial codes and
more than 400 pages of notes and analytical
memos (including spreadsheet
categorization notes/analysis).
Identified categories related to
implementation phases for selected projects
without applying existing practices
identified in literature
Identified categories related to the existence
of multiple and changing control categories
without applying existing concepts
identified in control literature that are
relevant the understanding of control
balancing decisions in COTS
implementation projects.
Identified categories related to the existence
of multiple and changing stakeholder
orientation without applying existing
concepts identified in stakeholder literature
that are relevant the understanding of
Page 102 of 307
stakeholder management in COTS
implementation projects.
Compared multiple perspectives, including
project team, business users, partners,
vendors, internal support teams, and the
leadership team and compared multiple
sources of data.
Compared the use of multiple processes and
tool from project’s intent perspective
Selective
coding
&
Data
collection
• Delimiting further coding to only those
concepts and variables that relate to the
emerged categories (Glaser 1978).
• Making constant comparisons between
instances of data labelled as a particular
category and other instances of data in the
same category to substantiate categories
(Urquhart et al. 2010).
• Further data collection guided by the
principle of theoretical sampling, i.e., deciding
on analytic grounds where to sample from
next (Glaser and Strauss 1967, p. 45).
Delimited further coding to a set of
tentative core categories which evolved into
control orientations and stakeholder
orientations
Followed the constant comparisons
technique of grounded theory research,
focusing on the development of categories
and concepts by constantly comparing data
to data (e.g., primary interview data to
secondary data such as project
documentation from electronic repository)
Theoretical
coding &
data
collection
• Analysis and specification of theoretical
relationships between core concepts and
categories (Bryant and Charmaz 2007, p. 25).
This theoretical coding (Glaser 1978), also
referred to as iterative conceptualization
(Urquhart et al. 2010), is aimed at increasing
the level of abstraction, relating categories to
each other, and clarifying which categories
may be properties of others.
Constructed detailed case narratives to
capture sequences of events and their
relationship at different project phases.
Validated the application of three different
control configurations with four distinct
intent which forms the four distinct control
orientations, influenced by three salient
trigger factors
Page 103 of 307
Combining control configurations with the
choice of processes, we have identified four
distinct stakeholder orientations
Scaling up • Engaging with other theories for theory
building: To raise the level of
conceptualization and scale up the emerging
theory, existing theories or concepts should be
used for comparisons (Urquhart 2007).
Thereby, meta theories and theoretical
categories with limited empirical content and
general scope are particularly suitable as
heuristic or sensitizing devices (Kelle 2007).
• Grouping higher level categories into
broader
themes with the goal of increasing the
generalizability of the theory and being able to
relate the theory to the broader literature
(Urquhart et al. 2010).
Engaged with literature on control
balancing (Gregory et al. 2014,) in ISD
context
Engaged with literature on stakeholder
management and engagement (Donanldson
and Preston, 1995)
Conceptualized control orientation and
stakeholder orientation as higher level
category
Defined it as making targeting adjust in
control configuration and stakeholder
orientations in terms of engagement level
and sensitivity level allows the project team
to manage CSF for the project.
Conceptualized four distinct orientations of
control configurations, and four distinct
stakeholder orientations
Theoretical
Integration
• Relating the theory to other theories in the
same or similar field by comparing the
substantive theory generated with other,
previously developed theories (Glaser 1978;
Urquhart et al. 2010).
Compared our core category of control
orientations with control balancing theory
(Gregory et al. 2014) and notion of “a
portfolio of control modes” (Kirsch 1997),
the literature on control dynamics
(Choudhury and Sabherwal 2003; Kirsch
2004)
Compared our core category of stakeholder
orientations with the stakeholder agency
theory (Hill & Jones, 1992) and analysis of
the application stakeholder theory’s
orientation (i.e. descriptive, instrumental,
Page 104 of 307
and normative) by Donaldson and Preston,
1995.
(Adapted from Gregory et al. 2013)
4.4 Reliability and validity of the cases
Reliability and validity of the current case study research was ensured in the following
manner.
4.4.1 Reliability
Reliability of this research is based on the use of a well-documented methodology and case study
protocol (Yin, 1994). A case database was created where all case interviews and field notes were
stored. This was to support a clear and documented chain of evidence (Yin, 1994). This database
supported derivation of the conclusion from our initial research questions. Audio transcription
were validated by a second person to ensure accuracy. Themes, categories and patterns derived
through qualitative analysis were constantly checked during the analysis to ensure the validity of
identified themes, categories and patterns.
4.4.2 Validity
Cresswell (2009) and Yin (1994) recommended several methods to ensure validity of a qualitative
research. We used the following strategies to ensure validity of current thesis.
Page 105 of 307
4.4.2.1 Triangulation
In addition to the primary data collected through interviews, we also collected secondary data for
triangulation. Themes, patterns, and categories were validated using secondary data which
primarily comprises project documentations, meeting-minutes, social event records, email
communications, and project office records.
4.4.2.2 Member Checking
A member checking was conducted after primary interview and analysis but before the final report
preparation. A second round of interviews with the initial participants was arranged. During the
second interview, identified themes, patterns and categories were presented to the informants for
further comments. They were asked to provide feedback to ensure the accuracy and correctness of
the analysis. To minimize any concern relating to biases resulting from a single coder, we have
consulted a subset of the informants to review all derived codes and the supporting raw data. This
additional validation during the data analysis process significantly enhanced the validity of the
primary and secondary codes for this research.
Page 106 of 307
5.0 Case results and discussion
In this chapter, we present and discuss the evidence we found and how it relates to linked
theories as presented in Chapters 2 and 3.
In Sections 5.1 through 5.4, we present the evidence from each case and discuss the results of
each case individually. We also highlight what was unique or exemplary in each case.
5.1 Case A: Replacing an enterprise system for financial market
operations
The project selected under Case A involves replacement of a critical legacy system with a COTS-
based alternative. The host organization classified the original system as ‘critical’ as the legacy
system operated on financial data that are essential for the organization’s money market
operations. The legacy system was initially deployed for production use in mid-2007. This system
allowed administration and pricing of securities held as collateral in other financial contract
processing systems of the organization as well as various financial operations of the organization
such as Securities Lending, Term Repo4, Special Purchase and Resell Agreement (SPRA) etc.
The primary goal of this project was to expand existing functional capabilities and increase the
efficiency of the original system. Expanding functional capabilities included the implementation
of several enhancements that were identified before and after the completion of the legacy system’s
deployment, incorporation of new high priority business requirements, decommission of some
4Under a term repurchase agreement, a bank will agree to buy securities from a dealer and then resell them a short time later at a specified price. The difference between the purchase and sale prices represents the interest paid for the agreement. Term repurchase agreements are used as a short-term cash-investment alternative (Investopedia).
Page 107 of 307
legacy features of the solution, and prepare the solution for the future inclusion of foreign market
operations. Through extensive discussions with the business community and contemplation by IT
solution architects, the project team was able to establish a long-term “roadmap” for this new
project. The roadmap provided an incremental growth of scope for the application and a schedule
that supported the attainment of organizational business objectives while ensuring all project
related technical initiatives are progressed in a logical sequence.
Financial Market Operations (FMO), the project selected for Case A, was initiated towards the end
of 2013. This project comprised of four separate releases. Each release delivered one of the four
domain-specific functionalities of the application over the 2013-2015 timeframe. Project team
followed an incremental approach of delivery where each release was building on the
functionalities delivered by the previous release. With a team comprised of 22 members, this
project decided to adopt a COTS-based system called “Cadis” to replace the legacy system. Cadis
is owned and supported by a globally leading data analytics company “IHS-Markit”. In addition
to internal support staffs and developers, the project team also included consultants from vendor
side (IHS-Markit) to assist with the design and development activities of the project.
Table 20 below shows a summarized profile of the FMO project, selected as Case A:
Table 20: Summarized Profile for Case A
Project Purpose Stakeholder
Composition
Project Cost
(Completion)
Project
Duration
Post deployment Remarks
Replace a legacy
system, used for
financial market
operations, with a
COTS based
product
Core Project team,
Extended project team,
support personnel,
consultants, Vendor,
Business users
10.5 Million
(CAD)
24
months
Project achieved the promised
benefits and experienced minimal
post deployment issues.
Page 108 of 307
FMO project was under taken with the organizational vision to achieve several major benefits
which included : (1) reduction of operational risk for the organization, (2) reduction of domestic
external pricing data providers from four to one thus reduced costs related to multiple data
providers and a simplified pricing processes, (3) improvement and simplification of business
processes, (4) automation of business processes, (5) simplification of the system architecture, (6)
improvement of hardware and software supportability, (7) improvement of the end-user experience
and addressing COTS products defects thus increasing business and IT effectiveness. Whereas, all
these factors provide quantifiable measures of the project success, for this research, we also
collected post-deployment incident information and user satisfaction data to assess the overall
project success.
We interviewed a total of 11 members for this Case. Each informant was directly involved with
the implementation of the project. The workgroup interviewed for Case A includes original project
team members, members from the customer-facing services team, and members of the user
community or business users. The customer-facing service team or internal support team provides
direct support to the business users by responding to questions about the product’s operation as
well as providing trouble-shooting and repair services. Some members of this COTS support team
were part of the core project team in 2013. In addition, several interview participants, belonging
to the project team and user community, were directly involved with the implementation of the
project, thus, were able to provide accurate information of the implementation process, stakeholder
engagement and control process.
Case A provided an early lesson about the interview process. First few informants of Case A asked
for additional clarifications regarding the ‘control’ mechanism. The informants thought that the
Page 109 of 307
control involves with the activities performed by the project controller and the project manager in
tracking time and budget of the project. We, therefore, modified the introduction to the interview
for all subsequent interviews by including the following statement:
“By ‘Control’ the questions refer to various processes utilized to ensure proper involvement and
engagement by different project participants. Controls might be formal such as meetings, reviews,
reminders or informal such as joint status review or discussion at social gathering”
5.1.1 Implementation process for Case A
Purpose of this initiative was to develop a solution that would address new domestic business
requirements, decommission existing legacy solutions, and enable the business unit for future
inclusion of foreign market operations. Through extensive discussions with the business
community and the IT Solution architects, an application “roadmap” was established which
provided a direction for incremental growth of scope. From an implementation perspective, Case
A followed an agile-like methodology by delivering the scope and features in incremental manner.
Although the prevalent agile terminologies and practices like “sprints” were not used by the project
team, project implementation practices followed by the core project team were clearly not a
waterfall method.
Activities related to the implementation of FMO project appears to be segmented, at the minimum,
into seven distinct areas or logically grouped activities.
Page 110 of 307
The initial set of activities included developing the project charter, setting up project controls,
conducting risk workshops, hosting project kickoff meetings, and obtaining steering committee’s
approval of the project charter.
The second set of activities comprised of developing preliminary project management plan and
obtaining approval for the plan. In addition, a key project artifact “Business Requirement
Document (BRD)” was finalized and approved at this stage. Other dominant activities appeared to
be related to the fulfilment of project team’s physical and logical requirements such as
provisioning of workstations, granting access to servers, offices, common spaces, implementing
co-location plan, and communicating reporting structure changes.
The third set of activities showed emphasis on the analysis of implementation process and
configuration of the COTS solution from a conceptual standpoint. Documents such as Technical
Conceptual Design (TCD), System Requirements Specifications (SRS) and System Use Cases
(SUC) were developed during this stage. Although the project management plan was progressively
elaborated throughout the lifecycle of the project, project team obtained a formal approval for the
completed project management plan at this point.
Next set of activities looked very closely at the solution design, solution configuration, and all
related components of the new COTS product. The conceptual design, produced by the previous
set of activities, was transformed into a concrete implementation plan at this phase. Activities at
this stage enabled the project to debate about feature or function-specific alternative designs.
Detailed functional and technical requirements were finalized and approved at this point.
Production and approval of detailed solution design were also completed in parallel to the
Page 111 of 307
requirements finalization. Other key activities involved development of test plans and delivery
plans.
Following the development of a detailed design and a requirements document, project team’s
activities appeared to be more hands-on and tangible in nature. Provisioning of development
environment and COTS application licences were completed at this stage and the project team was
fully equipped to start the application related coding and configuration activities. Activities such
as developing backend database components, glue-ware modules, extended features for the COTS,
new graphic user interfaces (GUI) were dominant at this stage. Due to a segmented nature of the
delivery, activities performed during this phase appeared to be repeating over the duration of
project’s lifecycle. Other activities such as preparing the user guide were initiated at this point.
Test case preparation, deployment of deliverable to the QA environment, execution of developer’s
unit testing, and assurance of code quality were also performed at this stage.
The sixth set of activities focused on ensuring the accuracy and validity of the project’s
deliverables, transition plan, and decommissioning of the legacy system. Project team in
conjunction with business users, reviewed the results of User Acceptance Testing (UAT). This was
a recurring activity as every module, delivered by the project team, went through an UAT as soon
as the module was identified as ‘release-ready’. Transition plan related items such as production
support model, end-user training plan and application’s disaster recovery posture were finalized
by the project team. The project also formally solicited the client or business sign-off for all project
deliverables and prepare the project for completion.
A final set of activities was a non-repeating set of activities. Due to the critical and complex nature
of the application, the project team agreed to support the operational or COTS support team
Page 112 of 307
regarding any system related issues for several months since the project’s go-live date. This special
arrangement between the project and COTS support team was finalized at this stage of the project.
Significant activities towards this end of the project included: seeking steering committee’s
approval to formally close the project, and reviewing the support agreements or SLA with the
vendor for accuracy. The Project team conducted multiple knowledge transfer sessions with the
business users and the IT operations team to build awareness of new processes. The Project team
also collected feedbacks from all project participants to ensure that COTS implementation related
best practices are correctly identified and captured for future use in similar context. Table 21 below
presents all salient activities performed by the project team in Case A. Respondent codes used in
Table 21 and in the subsequent tables are formed by combining three distinct pieces of information.
This helped us to uniquely identify each participant without disclosing their identity. For example,
the first letter in the respondent code “A01PT” indicates the Case association (i.e. Case A), the
two-digit numeric portion indicates the informant number (i.e. first person interviewed or 01), and
the tailing two letter code indicates the group affiliation (i.e. PT for project team, VR for vendor,
BU for business unit, and IT for internal IT support).
Table 21: Project Activities – Major Phases and flows for Case A
Project Activities Respondents Potential
Category
Develop a “Road Map”
Obtain Strategic Investment approval
A01PT, A02PT, A06BU Pre-Initiation
Establish project controls and project administration
Develop the Preliminary Project Charter
Conduct project Kick-Off meeting/presentation
Obtain approval of the Preliminary Project Charter
A01PT, A02PT, A03PT,
A04PT, A05VR, A06VR,
A07BU, A08BU, A10IT,
A11IT
Initiation
Develop preliminary Project Management Plan (PMP), as
appropriate
Obtain Approval of the preliminary PMP
A01PT, A02PT, A03PT,
A04PT, A05VR, A06VR,
Page 113 of 307
Generate estimates for project schedule, activities and resource
requirements
Arrange Project Team Logistics
Produce Business Use Cases/Business Process Modelling
Documentation
Enable and Oversee Production of the Business requirements
Document (BRD) ensure due diligence is done in the following
areas: ATIP, Audit, COOP, Procurement, SFS, and K&IS
Obtain Approval for Business Requirements Document (BRD),
including Business Use Cases
A07BU, A08BU, A09BU,
A10IT, A11IT
Requirement
analysis
Finalize Project Management Plan (PMP)
Obtain Final Approval of Project Management Plan (PMP)
Determine Solution Alternatives
Obtain Approval of the recommended solution
Engage and Begin Managing Chosen and Approved
Vendor/Service Provider
Produce Technical Conceptual Design (Solution Architecture
Document (SAD)
Obtain Stakeholder Approval of Technical Conceptual Design
(SAD)
Establish a Requirements Traceability Matrix
If needed, Complete the System Requirements Specifications
(SRS) and Produce System Use Cases
A02PT, A03PT, A04PT,
A05VR, A07BU, A08BU,
A10IT, A11IT
Conceptual
Design
Oversee Production of Detailed Requirements
Obtain Approval of Detailed Requirements
Oversee Production of Design Documents
Obtain Approval of Detailed Design
Produce Test Plan
Develop Test Cases
A02PT, A03PT, A04PT,
A05VR, A07BU, A08BU,
A10IT, A11IT
Design and
Architecture
Develop Programming Components
Plan and Perform Integrated Test of Built Solution in the Dev
Environment
Obtain Approval to Transfer built solution into the QA
Environment; execute transfer
Plan and Perform QA/Functional Testing
A01PT, A05VR, A06VR,
A10IT, A11IT
Development
Organize and Facilitate User Acceptance Testing (UAT) -
Release Candidate
Obtain User Acceptance Test (UAT) Approval of Solution
Obtain Client Approval to Transfer Solution into Production
Finalize Transition Plan
Obtain Approval sign-off of Support Agreement(s)
A01PT, A02PT, A03PT,
A05VR, A06VR, A07BU,
A08BU, A09BU, A11IT
Delivery
Page 114 of 307
Coordinate Delivery of User Training
Deploy Solution to Production
Monitor System Stability
Oversee Project Stand-Down
Produce Project Completion Report
Obtain Approval to Close the Project
A01PT, A02PT, A03PT,
A04PT, A05VR, A06VR,
A07BU, A08BU, A09BU,
A10IT, A11IT
Closing
In summary, implementation-related activities for the FMO project appear to be segmented, at
minimum, into seven logically distinct groups. Although the “pre-initiation” activities determined
the roadmap for a potential technical solution, approval of a project charter and a formal kick-off
meeting indicated a formal initiation phase for this project. Besides the initiation, FMO
implementation activities also indicated the existence of other logical groups, such as requirement
analysis, conceptual design, design and architecture, development, delivery, and a non-repeating
“close-out” phase.
5.1.2 Control configuration and balancing
When the FMO project initially started in September 2013, the project team and the business unit
already established a sound relationship with each other. This harmonized relationship resulted
from the cooperative search for an optimal solution to replace the legacy system. However, the
relationship between the core project team and the vendor exhibited significant gaps in terms of
understanding and expectations. This could be attributed to an absence of shared work history
between two parties. The vendor’s business was located in Europe and was never awarded any
contract by the organization. Similarly, the relationship between the core project team and
extended IT teams was also lacking any notable history of working together. As a result, project
Page 115 of 307
team placed significant emphasis on formal documentation such as contracts, roles and
responsibilities outline, meetings and vendor demonstrations. However, in the absence of a signed
contract or an agreement, formal status reporting was not exercised during this phase of the project.
Business user’s participations in project activities were more noticeable compared to those of the
extended IT support team as business users were frequently invited to project’s status update
meetings to ensure the accuracy and completeness of project’s scope.
Once the project team finalized the contract with the COTS vendor, a more formal and procedural
control was adopted to ensure a common understating and common perception regarding project’s
work scope. Project team introduced a mix of behavioural and outcome based controls such as
formal communication plans, regularly schedule status meetings, steering committee meetings etc.
and included the vendor within the target of these controls. As the project team was finalizing
several key documents like project management plan and business requirement document, it was
critical to keep the vendor well-informed and to obtain timely vendor-response to ensure the
accuracy of these documents. Although the project team initially maintained a trust-based
relationship with the COTS vendor as well as with internal IT support teams, lack of previous work
history, unique project context and exiting organizational context influenced the project
management team to adopt a frequent and active monitoring approach for the project’s progress.
This proactive outcome control was indicated by respondent A03PT who states:
“We laid out a high level plan for all parties involved with the project…our feeling
about the vendor was bit of mixed and the reason for this is simple…the product we
selected is mainly used for a different purpose – different form our intended
Page 116 of 307
use….also we never used this vendor before….. at the same quarter [name of the
organization] under took several large projects which made us concerned about
the level of internal [IT support teams] support we will get. On time delivery of
those application servers were on our critical path and we simply couldn’t wait till
due date on the service request. What if it’s not ready by the due date?”
At the same time, control mechanisms were planned, coordinated, and implemented jointly by the
project team and the client. This resulted in a bilateral or coordinated control style which helped
foster a cooperative relationship between the project team and the client very early in the project’s
lifecycle.
After the project team finalized the BRD and project management plan, the activity focus shifted
towards conceptual design and solution design of the COTS product. In addition to status meetings,
the vendor and the project team jointly conducted design discussion sessions. Besides technical
meetings, the project team conducted workshops focusing on project management methodology,
various process templates, and organizational release management practices. Informants also
reported regular socialization activities during and outside of the working hours. A trust-based
and less formal control mechanism were also apparent from the utilization of brainstorming
session, information exchange of ideas, and spontaneous communications. At the same time,
ensuring the timely configuration of all required technical environments such as Development,
Quality Assurance, Pre-Production and Production was a matter of concern for the project. These
tasks were performed by the internal infrastructure support team of the organization. The Project
Page 117 of 307
maintained a frequent status review and activity tracking relationship with all parties responsible
for the timely delivery of the project’s environments.
Next, the project team and the vendor started developing the planned solution. Although the mutual
commitment between vendor and the client appeared quite high at this phase, the control
mechanisms were not significantly relaxed. Project team maintained regularly schedule status
review meeting, milestone tracking, defects review sessions, inspection of documentation and
reports, testing deliverables. Additional social controls such as workshops, feedback gathering and
communication, lessons learned for each iteration were also observed at this point. Although a mix
of social and authoritative elements found in the exercised control configuration, both vendor and
the project team were tracking each other at the status review, defect monitoring and deliverable
review meetings. Primary reasons for such bi-directional control are two: (1) actual development
work for each iteration was divided between vendor provided consultants and project’s COTS
developers, and (2) development for some glueware modules were taking much longer than what
was planned. Additionally, a few low-quality deliverables also introduced a certain degree of
uncertainty on both sides. At this point, a bi-directional control configuration was indicated by the
interview participants. According to the informant A01PT:
Deadlines were a bit tight for the development and testing of the new Markit plugin
that was required for Matlab integration. This plugin is developed by Markit.
Although Markit was very much involved and working hard on this and responding
quickly a lot of back and forth was required between us and Markit and it took a
while before we got a working plugin. Also, it was new players on the Markit side
and new players on the [organization name] side (for both FMO and Matlab) so
Page 118 of 307
nobody was sure what/how to do things. In addition, the Matlab plugin required
was a lot more complicated then the existing Matlab plugins.
As the project team was preparing test cases and developing a technical user guide, timely
cooperation from business users was still quite necessary. Clients were often engaged through
brainstorming sessions, informal exchange of ideas, email communications and telephone
conversations. However, the client’s involvement frequency with project activities was much
reduced compared to the early phases involvement frequency and the response timeline was also
very relaxed. These engagement approaches indicate a trusting relationship between the project
team and the client. However, a more formal and outcome-based control, through regular status
and progress tracking, was maintained for the extended support team.
As the technical team finished developing the core solution, project team forged a solid
understanding with the COTS vendor. Towards the end of the project, both the project team and
the vendor demonstrated a heavy reliance on the informal exchange of ideas and communications.
During the development phase, tasks performance and meeting the commitments did not show
much deviations from the expectations held by the project team. This helped the project team
maintain the status quo in terms of procedural control for the current set of activities as well. Both
the vendor and the project team worked diligently to resolve any issues or concerns resulted from
the client acceptance testing. However, project actively prescribed and tracked the knowledge
transfer sessions and internal training sessions targeted for the COTS support team. This was a
Page 119 of 307
unilateral control approach by the project team to ensure the support readiness for the operational
team of the host organization.
As the project team was wrapping up all the activities, the relationship between the project team
and the client seemed slightly different as both parties were actively tracking and ensuring that all
commitments were fulfilled. Control activities that dominated between the client and the project
were feedback gathering, lessons-leaned, and deliverables review.
Table 22: Observed Control Configurations in Case A
Phase Vendor IT Support
Team
Business
Clients
Salient Factors Relating to
Control Decisions
Pre-initiation Trust Trust Trust No signed contract or agreement,
lack of knowledge about the
capability of selected COTS,
Industry foot print of the vendor
Initiation Authoritative Authoritative Trust Absence of shared History, Project
context; Organizational context (i.e.
PMO prescribed compliance
requirements)
Requirement
Gathering
Authoritative Authoritative Coordinated Need to improve shared
understanding, Clear understanding
on the future state
Conceptual
Design
Authoritative Authoritative Coordinated N/A
Design and
Architecture
Authoritative Trust Coordinated Confirmation of resource
availability
Development Coordinated Authoritative Trust Development of shared
understanding, Delays in delivery
Delivery Trust Authoritative Coordinated Improved understanding, Missing
information
Page 120 of 307
Closing Trust Coordinated Coordinated Improved understanding, fear of
unfulfilled commitments
In summary, the control portfolio for Case A demonstrated multiple distinct configurations over
the duration of the project’s lifecycle. Initially, the control portfolio for Case A was dominated by
a more trust-based approach due to the absence of a formal contract or agreement. An authoritative
approach was used to replace the trust-based control configuration due to the absence of a shared
history among project participants and a need to improve shared understanding. Although the level
of shared understating developed at a varying pace among different project partners, closing phases
of the project indicated the return of a trust-based control configuration, along with a coordinated
control configuration.
5.1.3 Stakeholder engagement and sensitivity
Before the FMO project was initiated, the program overseeing the project proceeded through
organization’s standard gate 1 and gate 2 approval processes. The purpose of these gates is to
review the appropriateness of the undertaking and ensure that there is an understanding of what is
to be done and why, making certain that the investment has a good direction and is aligned with
the organization’s long term strategy. After the business sponsor gained approval at the decision
gate 1, this investment proposal proceed to the gate 2 review. At this phase, only a selective set of
stakeholders were engaged with the activities. Roles involved at this stage were business sponsor,
PMO, IT program manger. Although the activities of this phase were quite significant, the project
team did not have any control over the inputs provided by the stakeholders.
Page 121 of 307
Once the project proposal received gate 2 approval confirming that the strategic investment is
sufficiently compelling and the work should continue, both the project management team and all
enabling departments were engaged with various project related activities. Activities such as
creating the project charter, setting up project controls, conducting risk workshops, holding project
kickoff meetings – all required inputs from broader groups of stakeholders. Due to the early stage
of the project and the adoption of a completely new technology (the COTS product), most of the
inputs and recommendations received by the project management team were taken quite seriously
and in an open manner. This indicated a very high sensitivity towards the project’s stakeholders.
Following the initiation of the project, the project management team focused more on creating the
project management plan and finalizing the business requirement document (BRD). At this point,
a much larger group of people from diverse background were involved with the project’s
requirements gathering activities. Considering a completely new implementation of a COTS
product, the project team tried to ensure a much broader focus on the scope during requirements
collections. Most stakeholders were engaged through brainstorming sessions and facilitated
workshops to provide their inputs. Some of the groups engaged by the project team included
business clients, extended support team, release management, IT Security, IT infrastructure, the
COTS vendor. Since the artifacts developed at this point will server both as a guide for and
commitment of the project team, stakeholder groups exerted significant influence at this phase of
the project. A high engagement level, in an inclusive manner, helped develop a shared
understanding among the key stakeholder groups of the project as stated by participant A02PT:
“Having the Technical Team Lead participate in the business requirements
gathering and in all working sessions with the business helped in the preparation
Page 122 of 307
of the Business Proposed Solution. Although the Business Requirements Document
traditionally comes first and then the Business Proposed Solution, doing the two in
parallel has benefits also - it helped in identifying/confirming some business
requirements and it helped the business understanding right away how their
requirements will be addressed in [Name of COTS System]”
The third and fourth set of activities, which involved development of a conceptual design and a
concrete implementation plan for the COTS modules, demonstrated a reduced stakeholder
engagement. Dominant theme for the activities on this phase indicated a more prescriptive
approach through refinement of initially gathered requirements. Leading stakeholders for the
activities were enterprise architect, solution architect, solution integrators, vendor’s solution
engineers, selected developers and enterprise data modeler. As the detailed functional and
technical requirements were finalized at this point, technical project lead and subset of the project
management team were heavily involved with the decision-making process at this phase.
Once the project team started implementing the design and technical components for the COTS
solution, activities progressed in much focused and self-directed manner. This implied a reduced
decision making activities and following a pre-established solution decisions. This approach
required a significantly less involvement from the extended stakeholder groups of the project.
Groups of roles driving the activities of this phase mostly comprised of solution developer, vendor-
provided consultants, technical team lead, database administrator, and technical team lead.
Developers from IT Support team were engaged to facilitate the technical knowledge transfer. A
Page 123 of 307
visible difference between this phase and the earlier phases is very minimal opportunity to
influence the approved solution design or add any new requirements.
As the development of deliverables was completed by the technical team and progressed through
the quality control process, the project team focused on convincing a wider audience that the
promised value has been delivered. In addition, the project also needed to ensure both external
partners and the internal IT support teams are ready for upcoming changes to business and
technical processes. To ensure this state of readiness, project team needed to communicate with a
large group of stakeholders starting from business sponsors to external recipients of data - located
outside of the organization. Groups of people engaged at this phase included: steering committee,
infrastructure support, IT Security, IT continuity of operations, operational support including
COTS support, middle-tier support, and database support, PMO, business users, up-stream systems
support and down-stream systems support teams. Besides obtaining approvals from key
stakeholders of the project, creating awareness and a positive mindset to accept the changes were
also the key motivating forces for such a large engagement. Despite a large group of people
engaged in this phase, stakeholder’s opportunity to influence the deliverables were very little as
most of the engagement techniques involved push communications such as presentations, emails,
and announcements. An increased level of task dependency and complexity of activities motivated
the project team to increase the engagement level as indicated by respondent A09BU:
“Deployment of NHA-MBS [module name] was a complex deployment involving
many groups as there were FMO, $U and Matlab changes. The steps to be followed
were carefully documented by the Project Manager, reviewed by the various
players involved and exercised in previous environments. This was very important
for a successful deployment to production.”
Page 124 of 307
As the project approached its completion of all major activities and obtained acceptance from the
user community, project team’s focus was mostly on ensuring operational readiness and learning
from mistakes. Although the stakeholders engaged at this stage was moderate, the opportunity to
influence project deliverables were negligible. Project team sought inputs from all key project
participants such as core project team, support team, IT infrastructure team, COTS vendor, and
business users. An apparent goal of the activities was to avoid similar mistakes in future.
High
Stakeholder
Engagement
Low
Stakeholder
Engagement
High
Stakeholder
Sensitivity
Low
Stakeholder
Sensitivity
Delivery & Transition
Initiation
Requirement Analysis & Collection
Pre Initiation
Development
Design & Architecture
Conceptual Design
Closing
Page 125 of 307
Figure 8: Stakeholder Orientation for Case A
In summary, the FMO project under Case A indicated a high stakeholder engagement during the
early lifecycle of the project. The requirements analysis phase also demonstrated a high level of
stakeholder sensitivity, as most of the inputs and recommendations received by the project
management team were included in the requirement documents. Significantly reduced stakeholder
engagement and stakeholder influence were observed during the mid-lifecycle of the project. This
helped the project team maintain its focus in their activities and minimized activity costs. However,
the engagement level gradually increased towards the end of the project’s lifecycle, although
sensitivity did not increase proportionally. Figure 8 above captures this dynamic nature of
stakeholder orientation for the entire project lifecycle for Case A where each project phase is
plotted on a different quadrant of the stakeholder orientation grid depending on the combined
engagement-sensitivity level of the phase.
5.2 Case B: Replacing a shared system for financial regulations
Case B involves a COTS implementation by the same organization where an external facing
enterprise system was replaced. The legacy system was used to collect, validate and maintain
financial data and financial returns filed by federally regulated Deposit Taking Institutions (DTIs)
of the country. The existing system was jointly owned by three different institutions of the
Canadian government (referred as Tri-agency from hereon). Previously, the legacy system had
undergone several enhancements since its deployment in 1998 to keep pace with the requirements
of the partner agencies, but eventually it became incapable of adapting to the increasing quantity
Page 126 of 307
and complexity of data collection and management needs. The system was also failing to comply
with the existing security, hardware, software and performance standards of the host organization.
Maintaining the status quo became an obviously inadequate option in the face of evolving business
drivers and business objectives of all three partner organizations. In this respect, several
dimensions appeared to be driving forces to consider a replacement of the existing system such as
(1) technical limitations of the existing solution prevented implementing any required
enhancement to support the collection of additional information, (2) the security standards used by
the legacy system were no longer considered adequate thus, elevating the level of risk for all
partner agencies that might result from a security breach exposing confidential/sensitive financial
information, (3) increasing dissatisfaction expressed by the external users of the system due to a
poor response time, (4) existing hardware and application components were going out of vendor-
support thus a significant increase in the maintenance cost and diverges from the enterprise
architecture direction, and (5) inadequate and inconsistent validation rules preventing a common
rule engine for the legacy system.
Following an in-depth analysis of the legacy system and the software market, the project team
recommended engaging Deloitte & Touche LLP, a leading solution integrator, to implement a
commercial-off-the-shelf (COTS) software solution, based on Vizor Software, that will meet the
Tri-agencies’ business requirements. This recommendation was accepted for implementation after
comparing it to the status quo as it involved significant expenditures to maintain the existing
system with inadequate functionality in addition to carrying a significant risk of system failure.
This project was initiated in May 2011. Over all project scope comprised of four distinct yet
integrated streams: core modules implementation, Partner Secure Remote Access (PSRA)
Page 127 of 307
configuration, Enterprise Application Integration (EAI) configuration, and Organization Change
Management (OCM). The core modules implementation consisted of two phases: (1) deposit
taking institution implementation which was delivered in the fourth quarter of 2013, and (2)
pension plan/insurance company implementation which was delivered in the April of 2014. PSRA,
EAI and OCM streams were planned to run concurrently with core module implementation. Vizor
based implementation of the core modules followed a pure agile approach with multiple sprints
delivering different scope of the project.
With a team comprising of 35 people, budget at completion (BAC) for this project was $19.5
million (CAD). Funding for the project was equally divided among three partner organizations,
although each partner agency was responsible for any expenses falling outside the project budget.
Cost estimates included a 15% contingency to allow for change requests and work involved in
customizing the solution. Since Vizor, the COTS vendor, was based in Dublin, Ireland,
development activities for this project were divided between two geographically separated
locations - Ottawa and Dublin.
Table 23 below shows a summary profile of the FRS project:
Table 23: Summarized Profile for Case B
Project
Name
Project
Purpose
Stakeholder
Composition
Project
Cost
(Completi
on)
Project
Duration
Post deployment
Remarks
Financial
Regulatory
System
Replace a
legacy
system with
a COTS
Core Project team,
Extended project team,
support personnel,
consultants, Vendor,
Solution Integrator,
Business users, Partner
19.5
Million
(CAD)
26 months
Project was able to
deliver all required
features for the core
module at completion;
however, the end users
including the support
Page 128 of 307
based
product
organizations, External
users/partners
personnel experience
several defects and
incidents with the
system.
In order to assess the influence of control configuration and stakeholder engagement on project’s
critical success factors, we augmented the list of CSF reported in IS literature with the CSF
identified from Case A and Case B. Additionally, to assess the level of success for this project, we
have resorted to the post-deployment incident records and user experience for both business and
IT users.
Through the project charter, this project promised to realize the following major benefits: (1) a
major extension of the Tri-agency’s ability to collect information and support future regulatory
changes of financial market, such as Basel III requirements thus, reducing operational and
reputational risk for the organization, (2) enhanced corporate effectiveness by reducing efforts
required to introduce or modify returns, creating capabilities to handle large multi-dimensional
returns, integrate the collection and validation of DTI, insurance and pension plan regulatory
filings into a single system, (3) enable simplified administration of filer accounts and
communication, (4) free up required resources for metadata management by introducing a self-
service model which includes contact management, password maintenance, and client
communication, (5) maximize the usage and availability of collected data for partner agencies, and
(5) reduced time for acceptance, validation and distribution of large regulatory information.
We interviewed a total of 12 members for this Case who were directly involved with the
implementation of the project. The workgroup interviewed for Case B included original project
Page 129 of 307
team members, members of the customer-facing services team, and members of the user
community or the business users, Tri-Agency partners, and the solution integration team.
5.2.1 Implementation process for Case B
The purpose of this initiative was to develop a COTS based solution that would address existing
shortcomings of the legacy system, help the host organization to comply with the international
financial regulations such as BASIL III, modernize existing application infrastructure from
‘capability and security’ perspective, and decommission the existing solution. Guided by the
organizational policy, the group leading this business initiative considered ‘buy before build’
approach while searching a replacement for the existing system. Due to the unique nature of the
application and a niche market, it was extremely difficult for the project team to identify an ideal
candidate solution early in the project’s life cycle. It was already anticipated that the project will
take over a year regardless of a ‘COTS’ or ’custom built’ approach. Considering a wide scope and
unique business requirements of this project, the host organization decided to handle the
procurement as a separate project. Unfortunately, the initial attempt to secure a vendor failed,
costing the organization nearly two million dollars and six months of project time. This served as
a source of a major ‘lessons learned’ for the project team. As a result of this negative experience,
several procurement documents (RFI and RFP) as well as solution requirements document were
modified to overcome problems encountered in earlier effort. In addition, the host organization
consulted with its peers outside of the country and identified one COTS based product used in a
similar context. These findings, coupled with a more flexible procurement approach, enabled the
project team conduct a successful procurement on its second attempt. The full procurement cycle
Page 130 of 307
included issuance of a revised RFP, proposals evaluation, vendor demonstrations, contract
negotiation and a contract award.
From an implementation perspective, the project selected under Case B followed a completely
agile methodology by delivering scope and features in nine distinct sprints over two separate
phases. Each sprint lasted for three to four weeks which is slightly longer than typical sprint
durations recommended by the agile methodology. Activities related to the implementation of Case
B appeared to be segmented, at the minimum, into five distinct areas or logical groups. However,
due to considerable efforts and investment during the RFP process of this project, much of the
work related to planning, requirements analysis, and conceptual design were already accomplished
during the procurement process.
The core project team was directly engaged in September 2012. At that point, the project
management team possessed an approved version of the Business Requirement Document (BRD),
Project management plan, and secured most of the project’s logistics. The project team primarily
focused on the conceptual solution and design that were developed during the vendor
demonstration and were not finalized. Influenced by the agile methodology, the overall conceptual
design was divided in logically segmented chunks that are consistent with overall design of the
solution. The requirements specification document and business use cases were revisited, finalized
and segmented. These changes also triggered a change for the initial project management plan.
Key artifacts produced by the project at this point were a requirement traceability matrix and an
acceptance plan for the deliverables. As the agile nature of the project required multiple and
frequent deployment to different development environments such as quality assurance, pre-
production and production. To address this challenge, project team initiated multiple “Request for
Page 131 of 307
Change (RFC)” for all planned sprint deployments. Approvals for the revised project management
plan, business use cases and the acceptance plan were also obtained at this point. Other critical
activities involved finalizing risk management plan, change management plan, quality
management plan, and escalation process.
Next set of activities involved developing various work packages that were to be delivered by the
first sprint of the project. An approved and validated design document was provided to vendor’s
development team located in Dublin, Ireland. This work included enhancements and modifications
of the COTS product. Integration and data migration activities related to sprint’s requirement were
implemented by the consultants, the solution integratory and organization’s IT support team
located in Ottawa. As the implementation of the approved design was proceeding as planned, the
design and architecture team started to finalize the design for the next sprint. Therefore, both
design and development related activities were taking place in parallel. Other key activities at this
stage involved the development of test and delivery plans for the sprint currently under
development.
An agile approach also influenced post development activities of the project. Although each sprint
developed a functional version of the COTS product with a subset of the functionalities, a full
transition of the final product and the decommissioning of the legacy system took place towards
the end of the project’s lifecycle. In each sprint, all functionalities developed went through a rigour
unit testing conducted by the developer in Dublin. Once a deployment packaged was shipped to
the host organization, the project team conducted usability, user acceptance and integration testing
for the package. Project obtained informal sign-off from business sponsors after each sprint. While
the development activities were progressing as per project management plan, architecture, design,
and development work for the upcoming sprint was also progressing in parallel. Transition
Page 132 of 307
activities such as – preparation of support documents, system’s security assessment, support
team’s training and developing a disaster recovery approach were finalized by the project team
Similar to those observed under case A, the final set of activities for this project was a non-
repeating set. Salient activities at this phase of the project’s lifecycle were collecting feedback
from different stakeholder groups, executing an administrative phase-closure by obtaining client
signoff for deliverables, closing all open procurement related contracts and updating necessary
project documentations.
Table 24: Project Activities – Major Phases and flows
Project Activities Respondents Activity
Group
Opportunity exploration
Develop a “Road Map”
Obtain Strategic Investment approval
Procurement of the candidate solution
B01PT, B02PT,
B06BU, B07BU,
B08BU, B09BU
Pre-Initiation
Establish project controls and project administration
Establish Governance Reporting for the project
Hold Risk workshop(s)
Complete Project Risk/Type/Gating Assessment
Develop the Preliminary Project Charter
Conduct project Kick-Off meeting/presentation
Obtain approval of the Preliminary Project Charter
Obtain Approval to Gate into Ph1.
B01PT, B02PT,
B03PT, B04PT,
B05VR, B06VR,
B07VR, B09BU,
B10BU, B11IT, B12IT
Initiation
Develop preliminary Project Management Plan (PMP), as appropriate
Obtain Approval of the preliminary PMP
Generate estimates for project schedule, activities and resource
requirements
Arrange Project Team Logistics
Produce Business Use Cases/Business Process Modelling
Documentation
B01PT, B03PT,
B04PT, B05VR,
B06VR, B07VR,
B08VR, B09BU,
B10BU, B11IT,
Requirement
analysis
Page 133 of 307
Enable and Oversee Production of the Business requirements
Obtain Approval for Business Requirements Document (BRD),
including Business Use Cases
B12IT
Finalize Project Management Plan (PMP)
Obtain Final Approval of Project Management Plan (PMP)
Establish Project Design Guidelines and Modelling Standards
Determine Solution Alternatives
Obtain Approval of the recommended solution
Engage and Begin Managing Chosen and Approved Vendor/Service
Provider
Produce Technical Conceptual Design (Solution Architecture
Document (SAD)
Obtain Stakeholder Approval of Technical Conceptual Design (SAD)
Establish a Requirements Traceability Matrix
Acceptance Plan
Complete the System Requirements Specifications (SRS) and
Produce System Use Cases
Initiate Request for Change (RFC) for planned implementation of the
recommended solution into the Production environment
Devise the Project Delivery Strategy
Coordinate setup of the Development and Test Environments
B02PT, B03PT,
B06VR, B07VR,
B08VR, B10BU,
B11IT, B12IT
Conceptual
Design
Oversee Production of Detailed Requirements
Obtain Approval of Detailed Requirements
Oversee Production of Design Documents
Obtain Approval of Detailed Design
Organize and Lead Tactical Planning for Development based on
Approved Detailed Design
Write the "To Be" (future state) Business Procedures
Update the Support and Operations Guide
Produce Test Plan
Develop Test Cases
B02PT, B03PT,
B06VR, B07VR,
B08VR, B10BU,
B11IT, B12IT
Design and
Architecture
Develop Programming Components
Plan and Perform Integrated Test of Built Solution in the Dev
Environment
Prepare User Guide
Prepare for QA Testing
Obtain Approval to Transfer built solution into the QA Environment;
execute transfer
Plan and Perform QA/Functional Testing
Arrange for Pre-Prod and/or Prod Environment Set-up
Facilitate Client Preparation of UAT Test Cases
Define Ongoing Support Agreement(s)
B03PT, B06VR,
B07VR, B08VR,
B10BU, B11IT,
B12IT
Development
Page 134 of 307
Produce Training Material
Organize and Facilitate User Acceptance Testing (UAT) - Release
Candidate
Obtain User Acceptance Test (UAT) Approval of Solution
Obtain Client Approval to Transfer Solution into Production
Complete Release Management Plan and Build Form
Finalize Transition Plan
Coordinate Final Support and Operations Guide
Coordinate Delivery of User Training
Deploy Solution to Production
Complete Project Delivery Phase-end Activities
B01PT, B03PT,
B04PT, B05VR,
B07VR, B08VR,
B09BU, B10BU,
B12IT
Delivery
Monitor System Stability
Oversee Project Stand-Down
Produce Project Completion Report
Obtain Approval of Project Completion Report
Complete Project Close-out phase-end/project-end Activities
Obtain Approval sign-off of Support Agreement(s)
Do Formal GO LIVE (Final Release)
Obtain Approval to Close the Project
B01PT, B02PT,
B03PT, B04PT,
B05VR, B06VR,
B07VR, B08VR,
B09BU, B10BU,
B11IT, B12IT
Close out
In summary, implementation-related activities for the FRS project appeared to be segmented, at
minimum, into five logically distinct groups. Pre-initiation or pre-project activities for FRS
consisted of significant efforts related to requirements collection and procurements. As a result,
only conceptual design, design and architecture, development, delivery, and closing phases were
performed by the core project team after a technical roadmap was established. Initiation and
requirement analysis were two additional phases for this project, although in this instance they
were performed as a separate project. Furthermore, implementation activities for the FRS project
followed an explicitly agile approach where conceptual design, design and architecture,
development, and delivery related tasks were executed in an iterative fashion until the entire
project scope was completed.
Page 135 of 307
5.2.2 Control configuration and balancing
From a control perspective, Case B demonstrated a significantly complex scenario. This control
complexity was a true reflection of the project’s stakeholder landscape as the project team was
controlling two external partner organizations, two distinct vendor teams, internal IT support team
and the internal business client.
In May 2011, a subset of the core project team in conjunction with business authorities from all
three partners were working towards a successful procurement. Due to the existence of a shared
system (Legacy application), all three partners had a shared work-history and some degree of
common understanding regarding business needs. Therefore, the activities between project team
and the business users involved site visits by partners, common understanding workshops, formal
and informal meetings, exchange of ideas and formal status reports. Announcements and progress
reports were often distributed to all partners through a ‘joint communication’ approach. Internal
IT teams were engaged by the project on an ad-hoc basis without any direct reporting relationship.
At the same time, Informal exchange of ideas between the project and internal IT teams were
prevalent and the support staff were invited to attend brainstorming sessions organized by the
project. As the contract was not awarded at this point yet, both vendors were engaged through a
mix of formal and informal approaches. Procurement documents and vendor demonstrations were
formal but the vendors were not part of the regular status meeting. Other actions of the project
team also indicated a quite formal and unilateral control style such as RFP specifications requiring
the COTS vendor to partner with a mature and well established solution integrator, provide a
formal project management structure on the vendor side, allowing the host organization to screen
Page 136 of 307
resumes and interview process for vendor provided resources, maintaining physical presence of
vendor personnel on client premises.
Soon after initiating the second phase of project, a contract award was made and time allocation
was confirmed for both the business users and internal IT support teams. This provided the
opportunity for everyone involved with the project to charge time against the project’s cost center.
Due to the involvement of two different vendors, early phases of the project relationship between
the project team and the vendors indicated very formal and frequent status reporting activities.
Most of the reporting from the vendor side involved confirmation of client’s requirements in
respect to the scope of the project. Project team decided to use a fixed cost contract and reduce
risk for the organization and its partners. Project’s choice of contract type motivated the vendor to
exert certain degree of control over the project team as both vendors tried to ensure that client
provided requirements are not significantly different compared to the specification on RFP
documents. On the other hand, the project team was trying to ensure that business needs are
adequately addressed and in a comprehensive manner. As a result, frequent communications and
review meetings were heavily utilized by both parities. Agile approach introduced daily stand up
meetings which is an integral part of agile methodology. Although primary purpose of daily stand
up was to identify and remove road blocks, it also showed the up to date status of all project
activities. IT infrastructure and extended support teams were not part of the daily stand up meeting
but weekly status meetings were used to monitor activities performed by other support teams.
Design of this COTS application required multiple application and database severs for each
environment (Development, QA, PreProduction and Production). Provisioning of these technical
Page 137 of 307
environments was slower than the expected turnaround time which caused the project management
team to adopt a close monitoring approach of project’s progress through weekly meetings.
Project on the case B developed a detailed ‘requirement traceability matrix (RTM)’ to keep track
of agreed scope of the project. A dedicated process called ‘design authority’ was established to
review and approve decisions concerning conceptual and actual design for each COTS module.
Dominated by brainstorming and facilitated workshops, design authority process introduced a fast-
paced decision making. Design decisions were also regularly tracked and reviewed by the project
management team. To facilitate the tracking of status, project introduced RACIN (Responsible,
Accountable, Concur, Inform and Notify) chart for various critical activities. As the conceptual
designs were produced and architectural decisions were made, project team and clients were
monitoring each other – forming a control loop. Project wanted the client to follow a process while
presenting a design related requirement (behaviour) and the client wanted assurance that certain
requirements are included in design decisions (outcome).
As the activities moved into implementation, project team maintained the existing status review
and daily stand ups for information exchange and health check. Development for the first few
sprints progressed according to the scheduled plan. Although the customization and modifications
of the COTS application were performed by the vendor’s technical team in Dublin, site-visits by
some of the vendor’s technical team members and on premise presence of vendor’s director
significantly reduced the feeling of uncertainty for the project team. Regardless of the vendor’s
effort to build trust with the project team, an increased concern was resulted from some defects in
deliverables and delayed delivery time of 4th and 5th sprints. This schedule slippage induced the
Page 138 of 307
project team to become cognizant in tracking the work progress and task outcome for vendor’s
technical team in Dublin.
Once the development work was in progress and sprint deliverables started to complete, project
team developed a solid understanding with both vendors. Each sprint included a demonstration of
deliverables with client to secure a formal acceptance and a ‘lessons learned’ session to collect
client’s feedback. Towards the end of the project, both the project team and vendors demonstrated
a heavy reliance on informal exchange of ideas and informal communications. Joint
communication and joint demonstration sessions were commonly observed between vendors and
the project team. Organization’s COTS support team participated in technical and business
knowledge transfer sessions offered by the vendor and core project team. Although several of the
knowledge transfer sessions were voluntary, project ensured that members of the internal COTS
support team are certified on the COTS product they will be supporting in near future.
Table 25: Observed Control Configurations in Case B
Phase Vendor IT Support
Team
Business
Clients
Salient Factors Relating to
Control Decisions
Pre-initiation Trust Trust Trust No signed contract or agreement,
lack of knowledge about the
capability of selected COTS,
Industry foot print of the vendor
Initiation Authoritative Authoritative Trust Absence of shared History, Project
context, Vendor capability,
Organizational context (i.e. PMO
prescribed compliance
requirements)
Requirement
Gathering
Coordinated Authoritative Coordinated Mutual agreement on requirements,
shared understanding, Clear
understanding on the future state,
missed deadlines
Page 139 of 307
Conceptual
Design
Authoritative Authoritative Coordinated Confirmation of resource
availability, Development of shared
understanding, protecting price
agreements
Design and
Architecture
Authoritative Authoritative Coordinated N/A
Development Authoritative Authoritative Trust Delays in delivery, reduction in
code quality, lack of physical
presence
Delivery Trust Authoritative Coordinated Improved understanding,
improvement of code quality, site
visits by vendor, Missing
requirement information
Closing Trust Coordinated Coordinated Improved understanding
In summary, the control portfolio for Case B demonstrated multiple distinct configurations over
the project’s lifecycle. Similar to Case A, the initial control portfolio was dominated by a more
trust-based approach due to the absence of a formal contract or agreement. As activities moved
towards the implementation and execution of the project’s scope, lack of mutual agreement on
requirements, lack of shared understanding, disagreements about the future state, and missed
deadlines encouraged the project team to adopt an authoritative portfolio. Additionally, to ensure
a timely information flow and validate business expectations, a coordinated control configuration
was observed between the project team and clients. As the project approached completion, a high
level of shared understanding among project participants allowed restoration of a trust-based
control portfolio.
5.2.3 Stakeholder engagement and sensitivity
Initiation of this project was subject to the successful satisfaction of all requirements set forth by
organization’s strategic investment gates which includes Gate 1: Opportunity proposal, Gate 2:
Page 140 of 307
Proposed Business Solution, and Gate 3: Business Case. These requirements were satisfied
through Pre-initiation activities of the project. As this case involved an external facing IT system
used by all existing financial institutions of Canada, preparing RFP and other procurement
documentations demonstrated a wide consideration of several stakeholder groups. Most of the
project’s pre-initiation and initiation activities were performed during the first phase of the project.
Primary objective of these activities was to secure a reliable and qualified COTS vendor. Besides
the immediate business users or application owners, external partners, external users, IT Support
teams, procurement department of the client’s organization, executive sponsors from all partner
organizations, business sponsors, PMO, IT Program manger were engaged with these activities.
Most of the requirements for the target COTS solution were collected and validated at this point
through inputs gathering, co-authoring of various project documents and providing approvals and
decisions. A high-level of engagement was also observed from the utilization of brainstorming
sessions, focus groups and facilitated workshops to capture stakeholder inputs. Fixed cost contract
also motivated the project team to be highly sensitive to any client recommendations and potential
requirements. Reason for this approach was to avoid potential change requests during the
implementation phase that may lead to cost increase. Several key artifacts were produced at this
stage which included: detailed business requirements, high-level implementation plan, RTM,
BRD, contracts and memorandum of understanding.
As the second phase of this project had access to a detailed requirements and RTM document,
project team were able to start design and architecture activities of the implementation plan.
Conceptual design activities painted a high-level picture from an enterprise wide integration of the
target application. All sprints for the project was also clearly identified at this point. Activities at
this conceptual planning stage as well as the design phase seemed more focused. Team
Page 141 of 307
management efforts, specific task assignments and user of team wiki were observed. Due to some
technical limitations of the selected COTS product and other down-stream internal systems,
change and improvement recommendations were not as flexible as it was in the earlier phases of
this project. Leading stakeholders for the activities were business leads from each partner
organizations, enterprise architect, solution architect, solution integrators, vendor’s solution
engineers, IT Security specialists, and enterprise data modeler.
For Case B, stakeholder engagement was slightly higher when the project entered in development
and implementation phases. Groups involved with the activities primarily included vendor’s
offshore development team, client’s internal development team, infrastructure support team.
Frequent input from the business users was needed to resolve decision conflicts as the offshore
development team did not have access to client’s production environment and production data.
This introduced some challenges during earlier sprints, but the project team ensured a minimal
change of scope during the development of an on-going sprint. COTS support team was formally
invited to technical knowledge transfer sessions to facilitate pre-production and production
deployment of sprint deliverable for the project.
Once a sprint was completed by the offshore development team in Dublin, it was delivered to client
for integration and deployment into organization’s internal servers. Each sprint included some
tasks that resembles the traditional delivery activities from a waterfall software implementation
model. Deployment was assisted by project’s solution integrator and vendor provided technical
support team. Although each sprint included a small subset of working functionalities, data
migration support provided by organization’s internal IT support teams was necessary for every
sprint. Following the deliver of a sprint, business communities and clients were engaged through
Page 142 of 307
sprint demonstration. Purpose of this demonstration focused on newly delivered functionalities
and business process changes. Sprint demonstrations were scheduled on a regular basis with the
business communities and project leads. COTS support team was invited to these meetings but
participation was optional. Targeted alerts for clients, information sessions, and presentations
indicated a more passive engagement by several stakeholder groups. Regardless of a passive
engagement approach, participations by different stakeholder groups increased at this point and
the project also introduced processes to capture recommendations and feedbacks from the user
communities. This was accomplished through the use of sprint reviews and lessons-learned
sessions. Information captured was analyzed and utilized to improve poorly performing processes
and relationship management approach on the next sprint of the project. Groups of people engaged
during sprint-deliver included – business leads, infrastructure support, IT Security, IT continuity
of operations, operational support including COTS support and database support, PMO, business
users, upstream systems support and downstream systems support teams.
Closing activities of this project showed a significantly higher stakeholder involvement compared
to that in the development phase. Project management team closely engaged members of other
internal projects which were impacted by the delivery of this project through coordination and
communication. Organization’s standard feedback survey was conducted to collect leant lessons
from all participating stakeholder. Partner organizations conducted their own lessons learned
activities independent of the host organization. Collected inputs and identified concerns were used
to adjust organization’s support agreement with the solution integrator and the vendor. A close
coordination with COTS support team was also visible as effective and efficient incident response
from the support team was critical to protect organization’s external reputation.
Page 143 of 307
In summary, the FRS project in Case B showed a very similar stakeholder orientation compared
to stakeholder orientation of Case A. Case B indicated an increasing trend of stakeholder
sensitivity, ranging from moderate to high, during initiation and requirements collection phases of
the project. At the same time, stakeholder engagement remained significantly high, which helped
the project develop a comprehensive list of stakeholder needs and expectations.
.
Figure 9: Stakeholder Orientation for Case B
High
Stakeholder
Engagement
Low
Stakeholder
Engagement
High
Stakeholder
Sensitivity
Low
Stakeholder
Sensitivity
Delivery & Transition
Initiation
Requirement Analysis & Collection
Pre Initiation
Development
Design & Architecture
Conceptual Design
Closing
Page 144 of 307
During the execution phase of the project, which included conceptual design, design and
architecture, and development, only groups with specialized skills were involved. Furthermore,
opportunities to influence project activities were tightly controlled. This helped the project team
maintain its focus on all execution-related activities of the project. Towards the end, the FRS
project showed a moderate to high engagement of stakeholders, coupled with little opportunity to
influence the project’s deliverables. Figure 9 above captures this dynamic nature of stakeholder
orientation for the entire project lifecycle for Case B where each project phase is plotted on a
different quadrant of the stakeholder orientation grid depending on the combined engagement-
sensitivity level of the phase
5.3 Case C: Financial billing systems replacement
Case C involves the replacement of a critical legacy system of the same organization with a COTS
based product.
Supported through a legacy system “Bank Note Distribution System (BNDS)”, the host
organization distributed bank notes to Financial Institutions (FIs) to replenish their inventory of
bank notes, push new bank notes into circulation, retrieve notes from when FI’s inventories grow
too large, and retrieve worn bank notes to retire from circulation. FIs are charged a fee by the host
organization for each transaction. These billable fees, in addition to the full amount of the notes
being exchanged, are ultimately posted as journal entries into the SAP financial system of the host
organization. The legacy BNDS billing solution used a mix of database sources, custom SAP code,
and multiple flavors of technologies. Over time, the existing solution had experienced several
issues, including billing and reporting inaccuracies discovered by the FIs. Several of the business
users of this application are located in the Montreal and Toronto Agency Operations Centers
Page 145 of 307
(AOC) of the organization. With the existing system, AOC users were unable to make any
adjustments to billing transactions before the transactions were posted to organization’s SAP
general ledger. Such incorrect transaction could only be resolved and rectified by organization’s
finance department by manually entering data due to the absence of a proper electronic interface
for business users. As a result, often financial institutions were receiving invoices from the host
organization that did not match with the actual transactions for a given period.
For case B, the solution proposed by the project team was a COTS based replacement of the legacy
system. With the proposed solution, billing data will reside within the existing databases but
standard billing functions will be introduced, and bill correction features will be available to
authorized users. The new COTS solution will also introduce a ‘Billing-report’ quality assurance
functionality prior to billing confirmation. The summary invoice, adjustment and charges details,
and settlement invoices will all originate from the same database source and will be produced by
the department’s authorized personnel to ensure correctness of the reporting before billing
confirmation is complete and reports are delivered to the FIs.
The project under case C was initiated in July 2012. The project incrementally developed and
delivered all key functionality of the new application during the 2012-2014 time frame. With a
team comprised of 15 members, the project decided to adopt a COTS based system called “Cadis”
to replace the legacy system. Cadis is owned and supported by a leading data analytics company
“IHS-Markit”. In addition to internal support staff and developers, members of the project team
included consultants from vendor side (IHS-Markit) and external consultants to assist with design
and development processes. The core project team, including the consultants, was located at host
organization’s head office in Ottawa.
Page 146 of 307
Table 26 below shows a summary profile of the FBSR project:
Table 26: Summarized Profile for Case C
Project
Name
Project
Purpose
Stakeholder
Composition
Project Cost
(Completion)
Project
Duration
Team
size
(Core)
Post deployment
Remarks
Financial
Billing
Systems
Replacement
(FBSR)
Replace a
legacy
system with
a COTS
based
product
Core Project
team, Extended
project team,
support
personnel,
consultants,
Vendor,
Business users
7.5 Million
(CAD)
18 months
15
Although project
was able to deliver
most of the required
functionalities, the
support team and
clients experienced
significant post
deployment issues
In order to identify the critical success factors and assess the level of success for this project, we
have utilized project’s lessons learned, business case and project charter. As a part of a larger
business process redesign program, FBSR project was launched to address transaction-fee related
billing issues which had adverse effects on both the SAP and Cognos data warehouse environments
of the organization.The primary benefits of this project aimed to (1) lower operational risk for the
organization by improving user interface and overall system usability, leading to a reduction or
elimination of manual data entry errors, (2) increase FI confidence in the organization’s processes
by providing the Financial Institution partners with reliable billing and supporting reports that will
enable them to reconcile their deposits, (3) improve business flexibility by centralizing billing
processing rules, fee amounts and detailed transactional information under one system.
We interviewed a total of 9 members for Case C who were directly involved with the
implementation of the FBSR project. The workgroup interviewed for Case B included original
Page 147 of 307
project team members, members of the customer-facing services team, and members of the user
community or the business users, Agency office users, and extended project team.
5.3.1 Implementation process for Case C
The primary intent of this initiative was to replace multiple legacy systems that were jointly used
for satisfying a single business need. The organization used the legacy system to control
distributions of bank notes to other financial institutions of the country. A joint review of the
application data and behaviour, by IT solution specialists and business users, indicated the
possibility of a COTS replacement for existing system. Based on this analysis, project team
decided to use a COTS product that is primarily used for processing financial data and data
warehousing solutions.
Case C followed a hybrid methodology for its implementation. Although an incremental
development approach was used for the technical implementation, a fully functional system was
delivered at the final phase of the project.
Activities related to the implementation of Case C appear to be segmented, at the minimum, into
seven distinct areas.
Initial activities included the developing of a project charter, setting up project controls, conducting
risk workshops, hosting project’s kickoff meeting, and seeking approval for the preliminary project
charter. This project was significantly different compared to typical COTS implementation project
as there was very little activities related to procuring the COTS solution prior to the project
initiation.
Page 148 of 307
In parallel to the project initiation activities, project management team was involved in the
development of the project management plan and the business requirement document. Approval
for these documents was sought after the project charter was approved by the business sponsor and
project steering committee. Interestingly, some of the preliminary activities such as
“familiarization with organizational project management processes and tools”, “reviewing and
understanding the basic project assignment” etc. were repeated by the project management team
at this phase. This could be attributed to a change in the project manager position of this project.
Initially assigned project manager had to leave this project due to some other organizational
priorities and the project-controller was requested to manage this project. In parallel to review the
project assignment, the new project manager also engaged in securing the project logistics. Due
to a change in existing relationship, most of the focus of the new project manager was on
negotiating with functional managers to secure necessary project human resources. As the solution
alternative analysis was completed and a candidate solution was selected, the project team started
to establish project design guidelines and modelling standards. At this stage, project team was able
to finalize solution requirements and reach an agreement with the COTS vendor in terms of work
effort and support requirements.
The third set of activities executed by the project team was more focused on the architectural model
of the future state. As the legacy system was used by multiple, geographically distributed agency
centers of the organization, a solid conceptual design and a data model to reduce process-related
conflicts were a priority for the project team. During this stage, the project team conducted multiple
rounds of technical and capability gap analysis. Results from gap-analysis triggered multiple
revisions of the conceptual design document for this project. Finally, the project team obtained
approval from business users and the business sponsor for the finalized conceptual design
Page 149 of 307
document. From a technical environment perspective, setting up the development and test
environments were completed at this point.
The next set of activates, conducted by the project team involved analysis of the actual functional
module of the COTS system in relation to the business requirements provided by the client.
Surprisingly, a significant number of the technical and security requirements were not the priority
for the project at this point. Most of the project team’s efforts were directed towards developing
an accurate data model, designing the user interfaces, designing automated flows to sanitize
incoming data, and integrating the new COTS solution with the organization’s financial system
(SAP). Besides getting an approval for the detailed solution design, the project team developed
test plans and delivery plans at this stage. Once the approval for detailed design was received,
project team prepared to proceed to the next phase of the project which included developing the
technical solution from a holistic perspective. Activities for this phase included the development
of backend database components, job scheduling solutions for process automation, graphic user
interface (GUI) design and legacy system’s data conversion for the target COTS system. As the
developers were working on implementing technical components of the solution, project team
finalized test cases to facilitate the quality assurance process. The project facilitated user
acceptance testing in an environment that is identical to the production environment of the
organization. Since the project was not following a segmented delivery pattern, business users
were able to execute their test-cases on a fully configured and integrated COTS application. Dry
run for the data migration and finalizing the audit requirements were completed at this point.
The sixth set of activities focused on ensuring the accuracy and validity of deliverables, transition
plan for the new COTS and decommissioning plan for the legacy system. Project team in
conjunction with the business users, reviewed the results of the user acceptance testing (UAT).
Page 150 of 307
This was also a repeating activity as every module, delivered by the project team, went through a
UAT as soon as it was tagged as ‘release-ready’. Transition plan such as production operations
support, support team’s training and disaster recovery methods was finalized by the project team.
The project formally solicited the client or business sign-off for all the project deliverables to
prepare the project for completion. Once the business approval was received, the organizational
release management and change management activities were initiated to migrate the solution to
the production environment. Although a formal go-live for the new COTS-based system was used
by the project, a parallel operation of old and new system was maintained. This parallel operation
was jointly supported by the project and the COTS support team to ensure a smoother transition
and address any unexpected behaviour of the new product.
The final set of activities for this project was seeking steering committee approval and business
approval to close the project. A feedback survey was conducted by the project team to comply
with the PMO requirements. Project management team also produced a project completion report
and engaged the COTS support team to monitor the stability of the new system.
Table 27: Project Activities – Major Phases and flows
Project Activities Respondents Activity
Group
Opportunity exploration
Obtain Strategic Investment approval
C01PT, C02PT,
C06BU, C07BU,
C08BU, C09BU
Pre-Initiation
Establish project controls and project administration
Gain basic understanding of project assignment
Establish Governance Reporting for the project
Develop the Preliminary Project Charter
Conduct project Kick-Off meeting/presentation
Obtain approval of the Preliminary Project Charter
Obtain Approval to Gate into Ph1.
C01PT, C02PT,
04PT, C05VR,
C06VR, C07BU,
C08BU, C09IT
Initiation
Page 151 of 307
Develop preliminary Project Management Plan (PMP)
Obtain Approval of the preliminary PMP
Generate estimates for project schedule, activities and resource
requirements
Arrange Project Team Logistics
Produce Business Use Cases/Business Process Modelling
Documentation
Enable and Oversee Production of the Business requirements
Obtain Approval for Business Requirements Document (BRD),
including Business Use Cases
C01PT, C02PT,
C04PT, C05VR,
C07BU, C08BU
Requirement
analysis
Finalize Project Management Plan (PMP)
Obtain Final Approval of Project Management Plan (PMP)
Establish Project Design Guidelines and Modelling Standards
Determine Solution Alternatives
Obtain Approval of the recommended solution
Execute Procurement Process (for software, hardware, or services)
Engage and Begin Managing Chosen and Approved
Vendor/Service Provider
Produce Technical Conceptual Design (Solution Architecture
Document (SAD))
Obtain Stakeholder Approval of Technical Conceptual Design
(SAD)
Establish a Requirements Traceability Matrix
If needed, Complete the System Requirements Specifications
(SRS) and Produce System Use Cases
Coordinate setup of the Development and Test Environments
C01PT, C02PT,
C03PT, C05VR,
C06VR, C07BU,
C08BU, C09IT
Conceptual
Design
Ensure diligence is done on: ATIP, Audit, COOP, Procurement,
SFS, and K&IS
Oversee Production of Detailed Requirements
Obtain Approval of Detailed Requirements
Oversee Production of Design Documents
Obtain Approval of Detailed Design
Organize and Lead Tactical Planning for Development based on
Approved Detailed Design
Write the "To Be" (future state) Business Procedures
Update the Support and Operations Guide
Produce the Project Preliminary Transition Plan
Produce the Project Delivery Plan
Produce Test Plan
Develop Test Cases
C01PT, C02PT,
C03PT, C04PT,
C05VR, C06VR,
C07BU, C08BU,
C09IT
Design and
Architecture
Develop Programming Components
Plan and Perform Integrated Test of Built Solution in the Dev
Environment
Prepare User Guide
C01PT, C02PT, ,
C04PT, C05VR,
C06VR, C07BU,
C09IT
Development
Page 152 of 307
Prepare for QA Testing
Obtain Approval to Transfer built solution into the QA
Environment; execute transfer
Plan and Perform QA/Functional Testing
Arrange for Pre-Prod and/or Prod Environment Set-up
Facilitate Client Preparation of UAT Test Cases
Define Ongoing Support Agreement(s)
Organize and Facilitate User Acceptance Testing (UAT) - Release
Candidate
Obtain User Acceptance Test (UAT) Approval of Solution
Obtain Client Approval to Transfer Solution into Production
Complete Release Management Plan and Build Form
Finalize Transition Plan
Prepare for Solution Transition into Production
Obtain Approval sign-off for the Support Agreement(s)
Coordinate Delivery of User Training
Deploy Solution to Production
Do Formal GO LIVE (Final Release)
C01PT, C02PT,
C04PT, C05VR,
C07BU, C08BU,
C09IT
Delivery
Monitor System Stability
Produce Project Completion Report
Complete Project Close-out phase-end/project-end Activities
Obtain Approval to Close the Project
C01PT, C02PT,
C03PT, C04PT,
C05VR, C06VR,
C07BU, C08BU,
C09IT
Close out
In summary, implementation-related activities for the FBSR project appeared to be segmented, at
minimum, into seven logically distinct groups. Although FBSR was a COTS-based
implementation, procurement-related activities were almost non-existent. The primary reason for
such behaviour was the vendor’s choice. The COTS product selected for the FBSR project was
already in use within the host organization. This indicated an already existing work-relationship
with the vendor. Regardless, requirements analysis, conceptual design, design and architecture,
development, delivery, and closing related activities were distinctly identified and performed by
the project team.
Page 153 of 307
5.3.2 Control configuration and balancing
Replacement of existing business processes and a search for a suitable candidate solution for this
project started in mid-2012. Business analysis efforts for this project involved engaging multiple
agency offices and various internal teams of the organization to accurately capture existing
business processes and activities of the business unit. In the absence of a candidate solution and
fully staffed project team, early efforts of the project management team involved frequent face to
face meetings, over the phone conversations, direct communications, and brainstorming sessions
to acquire information, exchange ideas about potential solutions and explore possibilities of
modified business processes. Most of the efforts were directed towards communication with the
business users. Internal IT specialists were occasionally consulted to validate the feasibility of the
emerging future state in terms of a technical feasibility perspective. A potential vendor as well as
a candidate COTS solution were not identified at this stage. In mid-2013, the project appeared to
be more formal and organized in terms of administration and control. With an approved project
charter and a full-time project manager assigned, the project management team were able to decide
on a candidate COTS solution. The COTS product selected for this project was the same COTS
product that is used for the project discussed under Case A. At the initiation stage and in other
early phases of this project, the project team demonstrated a very relaxed control over the vendor.
Reason for such relaxed control over the vendor is attributed to an existing engagement of the
vendor with a higher priority project within the same organization. On the other side, an increase
in status review, prescriptive operational procedures, and planning meeting were also observed.
Both IT support team and clients were also involved in a less authoritative manner as this project
was sharing its technical resources with the project discussed under Case A.
Page 154 of 307
Soon after the initiation activities completed, project team started finalizing the requirements for
the selected COTS. Business and technical requirements were used to negotiate and finalize a fixed
priced contract with the vendor. Due to an existing relationship with the vendor, contract
negotiation was much simpler and faster than the organization’s typical contract negotiation
cycles. This also helped the project team to avoid a lot of procurement related meetings such as
bidder conference and vendor demonstration. As the project team consisted mostly of the members
of the organization’s functional teams and external consultants, project started to track the work
efforts and activity progress for internal IT support team and the vendor. This was accomplished
through weekly status-update meetings and status-update emails for assigned activities. As the
infrastructure and other project-logistics related requests were fulfilled by internal support teams,
organization’s standard SLA (Service level agreement) was leveraged to track progress and
completion of these activities. Despite the application of some forms of outcome-based controls,
project team’s overall relationship with both the business users and the internal support teams was
dominated by a positive outcome expectation.
Once the requirements were finalized, project team immersed into a high-level technical design
and low-level component designs of the COTS. Database schema design and data conversion
planning was also conducted along with the design activities. Most of the decisions made during
this phase were through design workshops and design decision meetings. Project controls for the
vendor turned into a more authoritative or unilateral at this phase. Regular status tracking and
other methodical approaches were observed. Project management team, at this point, relied heavily
on weekly status updates to assess how the project activities were progressing on the vendor side.
This change in control style also occurred soon after the team had a new project manager replacing
Page 155 of 307
the initial project manager. Primary data indicates a relationship between the adopted control style
and shift in leadership of the project, as respondent C03PT states:
“…as you re-call we had change of PM [Project Manager]… [name of new PM]
was very new to this and this was her first project to manage as a project manager.
So she relied a lot on the what we said at the weekly meeting [project status meeting]
but when we got to the QA[Quality Assurance], we realized the problem that $U
[glueware component for the solution] was not ready and several cadis modules
weren’t either…then we had no choice but to escalate and make some noise.”
Next, the project team started the development activities for technical components. A significant
portion of this development work was done by the project’s technical team with occasional
assistance from and design validation by the vendor. Interactions with the vendor during the
development were more balanced and bi-directional in nature as some of the unique needs of this
project required both the vendor and the project team to equally contribute to ensure a timely
completion of all activities. Assistance from the business users appeared more passive and
informal during the development phase. Any clarifications or change of requirements were
consulted with the client. Reason for such ad-hoc engagement by the client indicated a high degree
of confidence by the project team over the client in obtaining timely and reliable responses. The
configuration of the Production and the Pre-production environments was conducted by the
infrastructure support team of the host organization. Such environment configurations and other
internal support tasks were enforced through the SLA and IT service desk.
Page 156 of 307
As the development moved towards its completion, the project experienced several significant
setbacks. These negative outcomes included delayed development of technical components,
delayed delivery of glueware components and incomplete scheduling module to support process
automation. Participant C07BU provided a specific example of one of such setbacks:
“While the project team did a very good job with collecting the business requirements in
most areas, the requirements for the Cognos reports were left until the last minute and
were not finalized in time to allow the development and QA team to implement and test
them properly. It led to a lot of last-minute changes just before the deployment as well as
post-production.”
Negative outcomes and unfulfilled expectations during the development phase changed the
relationships between the project management team, COTS support team, and the vendor. Most of
these unfulfilled expectations resulted from lack of cooperation and engagement by the internal
support teams. This also resulted in multiple escalations by the project team to enforce cooperation
from the vendor and the support team. An increase in status tracking and meeting frequency was
observed in comparison to earlier phases of the project. As several product demonstrations and
joint parallel testing were conducted during this phase, project team maintained a close
coordination with the clients and agency offices. With a shared goal of timely delivery of a fully
functional COTS system, project team and the client demonstrated a high level of cooperation to
solve any identified problems and redesign all affected business processes.
The closing of the project showed a different relationship compared to the projects in Case A and
in Case B. Discovery of unexpected defects during integration and user acceptance testing
Page 157 of 307
motivated the project team to negotiate an extended support agreement with the vendor. All
deliverables were tracked for the completeness in an authoritative manner by the project team.
Cooperation between the project team and the business users remained unchanged during the
lessons learned activities where they saw themselves as equally responsible for the success of the
project.
Table 28: Observed Control Configurations in Case C
Phase Vendor IT Support
Team
Business
Clients
Salient Factors Related to
Control Decision
Pre-initiation N/A Trust Trust Absence of formal authority
Initiation Trust Trust Trust Existing contract with organization;
Project Context;
Organizational Context (PMO
prescribed compliance
requirements)
Requirement Trust Trust Trust Change in leadership
Conceptual &
Conceptual
Design
Authoritative Trust Trust Following Norm, Leadership
competencies
Design &
Architecture
Authoritative Trust Trust Following Norm, Leadership
competencies
Development Authoritative Authoritative Trust Missed Milestones, delays with
deliverables
Delivery Authoritative Authoritative Trust Disputes with vendor,
Disagreements, Competing internal
resources
Closing Authoritative Authoritative Trust Following Norm
Page 158 of 307
In summary, the control portfolio for Case C demonstrated multiple distinct control configurations
over the duration of the project’s lifecycle, which were significantly different compared to the
control configuration of projects in Case A and Case B. Although an existing work relationship
with the vendor and missing formal authority in the project management team led to a trust-based
control configuration, the project team chose to continue such a trusting relationship with the
vendor, internal support teams, and the client for subsequent project phases. This decision can be
attributed primarily to a lack of leadership strength and project management experience, as a junior
project manager replaced a more experienced project manager. However, the second half of the
project’s lifecycle was dominated by an authoritative control configuration, as the project
management team was unable to re-establish the desired level of shared understanding.
Additionally, the level of negative anticipation remained high among project team members.
5.3.3 Stakeholder engagement and sensitivity
Stakeholder engagement during the strategic initiative screening or project gating phases was
significantly lighter for this project compared to projects selected for Cases A and B. The initiative
justification provided by the business unit was quite convincing for the organization’s senior
leadership; however, a potential technical solution was not in sight. Pre-initiation activities of this
project demonstrated a heavy reliance on existing documentation and business processes to
develop an outline for the future state. Business lead, business analyst, and a temporary project
manager were primarily responsible for satisfying all requirements set forth by investment
proposal evaluation gates of the organization. Regardless of a little involvement by a wider group
of stakeholders, input collected by the smaller project team and several artifacts produced were
Page 159 of 307
quite significant for this project. Similar to previous two cases, organizational PMO played a
supportive role at this stage of the project by providing advice and guidance to the project team.
Once the project proposal received gate 2 approval confirming that the business investment is
sufficiently compelling for the organization and the project should be initiated, both the project
management team and a selective set of roles such as business analysis and technical solution
architect were engaged to support initiation activities. Involvement by various stakeholders
slightly increased at this point as the project manager started seeking inputs from various business
and technical teams to develop a comprehensive and feasible project management plan. Due to the
absence of a potential candidate solution, a large portion of the inputs collected was in the tentative
state. Once the project team was formed and a decision was made to adopt the COTS utilized by
the project under Case A, stakeholder engagement for this project increased further as the vendor
was included in planning discussions and in the decision-making process. In contrary to
organization’s usual project management practice, project team did not engage the COTS support
team and several IT support teams at this point. This decision had a significant negative impact on
the project’s work package identification, effort estimations and the work breakdown structure
development. Focus of this pre-design phase was largely on finalizing business requirements and
technical requirements which also included non-functional requirements such as scalability,
usability and availability of the technical solution. These activities were largely driven by the
solution architect and business analysts through a direct communication with end users of the
potential system and other technical teams. More commonly used engagement techniques such as
design workshops or intensive brainstorming sessions were not utilized by the project. This
resulted in a lower stakeholder engagement. Such reduced or low stakeholder engagement also
Page 160 of 307
had significant negative impact during the development and delivery phase of the project as
indicated by respondent C02PT:
“Testing that involves AIMS needs more than the time allotted during the project.
The project would benefit greatly from having appropriate testing resources
assigned to the team… It became clear right from the beginning of the testing cycle
that preparing and loading of the source transactions (mandatory prior to
execution any of the test cases) takes a significant period of time which was not
accounted for when the original estimates were done…if I recall correctly – the QA
team became involved soon before the development… As a result the QA process
was not as thorough as expected (especially related to the Cognos reports testing)
and took more time than it should. I also felt that the project could use an expertise
from the professional QA testers - the majority of the testing was done by the
Business Analysts which I believe is important but not sufficient.”
Immediately after the completion of requirements gathering and analysis activities, the project
succeeded in securing sufficient human resources to fully staff the core project team including the
project management team. As the project team started the execution of the third and the fourth set
of activities, which involved conceptual design of the COTS solution and a concrete
implementation plan of the COTS modules, stakeholder engagement remained roughly at the same
level with a slight increase. This can be attributed to a selective use of RACIN matrix to engage
the stakeholders. As the selection of a candidate solution was delayed during the initiation of this
project, the core project team was striving to fill-in a large amount of missing information at this
point. This required inputs from several internal and external stakeholder groups. Subsequently,
Page 161 of 307
these inputs and gathered information became part of some critical artifacts produced by the project
such as business requirement document, business user cases, design documents, and solution
architecture. Primary stakeholders for these activities included enterprise architect, solution
architect, solution integrators, vendor’s solution engineers, selected developers and enterprise data
modeler.
As the project completed design and architecting activities for the new system, three different
teams started hands-on development of the planned technical components which included vendor’s
technical team addressing product enhancements, project’s technical team developing user
interface, data models, and back-end solution components, and organization’s COTS support team
developing scheduling components and integration links for downstream systems. As the
development of the new solution progressed, each team identified several technical and logical
discrepancies. This required additional clarifications from the business users and occasional design
changes for the technical design. This resulted in a higher than usual engagement level by different
stakeholder groups at the development phase.
Multiple deadlines and milestones slippage occurred during the development cycle of this project.
This required the development window to extend beyond the planned duration. Once all technical
components were marked completed, the project started to migrate all developed components to
the production environment. Missed milestones and missed deadlines motivated the project
management team to take a firm stance towards any new recommendations and improvement
suggestions by the client or other stakeholder groups. To accommodate any ‘nice-to-have’ features
and non-critical requirements, project team proposed future enhancement releases for the system.
Deliverables and documentations of the project indicated a minimal use of RACIN matrix. Besides
Page 162 of 307
a low engagement level for most stakeholders, project also struggled to bring COTS support team
onboard for its knowledge transfer and product demonstration sessions. Regardless of any
substantial opportunity to influence the deliverables at this stage, project attempted to engage
several groups of stakeholders which included – steering committee members, infrastructure
support, IT Security, IT continuity of operations, operational support including COTS support and
database support, PMO, business users, users from the external agency offices and downstream
systems support teams.
Once the deliverables were reviewed by business users and by the project team, additional support
resources and support time were requested for this new application. This was primarily due to
some concerns from the client side on the stability of the new system. During the go-live and
project closing phase, multiple high priority incidents were reported by the client when some
accounting entries in the organization’s SAP general ledger did not balance. This also resulted in
several escalations as the COTS support team were unable to resolve the issues and assistance
from project’s technical team was necessary. This is indicated by several interview participants.
According to participant C06VR:
“Earlier involvement of the developers from [internal COTS support team’s name] team
would help with more smooth integration into the operational environment. Knowledge
transfer sessions are helpful but nothing can replace the real hands-on experience. I am
aware that this was part of the project plan from the very beginning however due to the
resource limitations in [portfolio name related to the project] it has not been done”.
Page 163 of 307
Closing of the project indicated a reduced engagement by different stakeholder groups. This is
evident from the project’s lessons learned and feedback sessions as well as the project closing
survey where participations were very low. The feedback provided by various stakeholder groups
for this project was significantly less compared to other similar sized projects of the organization.
Issues and any modification suggestions were added to backlog to avoid any further delay in the
project’s go-live.
Figure 10: Stakeholder orientation for Case C
High
Stakeholder
Engagement
Low
Stakeholder
Engagement
High
Stakeholder
Sensitivity
Low
Stakeholder
Sensitivity
Delivery & Transition
Initiation
Requirement Analysis & Collection
Pre Initiation
Development
Design & Architecture
Conceptual Design
Closing
Page 164 of 307
Figure 10 above captures this dynamic nature of stakeholder orientation for the entire project
lifecycle for Case C where each project phase is plotted on a different quadrant of the stakeholder
orientation grid depending on the combined engagement-sensitivity level of the phase.
In summary, the FBSR project in Case C showed a significantly different stakeholder orientation
when compared to stakeholder orientation of Case A and Case B. Whereas both Case A and Case
B indicated an increase in stakeholder engagement and stakeholder sensitivity for their early
lifecycle phases, Case C showed a decline in stakeholder engagement. This was evident from a
low stakeholder engagement during the “Requirement Analysis” phase of the project. Although
the execution-related activities of the project maintained a moderate to low stakeholder
engagement level. as well as a moderate stakeholder sensitivity level, the engagement gradually
increased toward the end of the project’s lifecycle. Additionally, sensitivity level also increased as
most of the client’s feedback appeared in the form of an enhancement or a change request.
5.4 Case D: Enterprise content management (ECM)
Case D involves a replacement effort by the organization of its existing “content management
solution” with a well-known COTS – SharePoint. The Enterprise Content Management (ECM)
program was under taken to enable the organization store, manage, share and find electronic
information with ease and efficiency. With a pressing need to create a single, trusted and reliable
source of content, seamless records management, and effective integration with the organization’s
existing productivity tools, ECM project was identified as a much-needed business investment.
ECM aimed to refine the information management, governance and security framework to support
organization’s effective management of information lifecycle, establishing a good understanding
of accountabilities and standardization of existing practices for all staff. This project, therefore,
Page 165 of 307
promised to provide greater efficiency and effectiveness in delivery of information management
services, reduce information risks related to legal compliance and organizational reputation,
remove risks associated with the support of the existing documents management system, and
deliver enhanced staff productivity.
A significantly differing aspect of this project, compared to earlier three projects, is that ECM was
built on the work of a previous ECM project, which ran from 2009 to 2012. The previous project
developed a solution based on the existing document management system of the organization, but
experienced significant challenges during implementation. As a result, the ECM project executed
between 2009 and 2012 was terminated before reaching its completion or closing phase.
Regardless of this failure, the motivation of the business owners of this application in replacing
the existing document management system was still very high. This high motivation can be
attributed to several factors such as the lack of ability by the legacy ECM solution to support
existing business needs, and assumption of a significant risk by the business by using an
unsupported technology. The existing document management system was built on IBM’s Filenet
3.5 environment. As a product, Filenet 3.5 was at end of its product lifecycle and was identified as
an unsupported product by the vendor- IBM. As a result, the organization did not have any suitable
alternatives to ensure its compliance with legal requirements for records retention and destruction
under the Library and Archives of Canada Act. In addition, the ability to find records on a reliable
basis that may be needed to respond to litigation, access and privacy requests, or public inquiries
was also compromised.
SharePoint based ECM project was initiated on February 2012 which followed an agile approach
of deployment. The core project activities lasted till December 2014. With a core project team
Page 166 of 307
comprising of 22 people, budget at completion (BAC) for this project was $11.5 million (CAD).
The project team consisted of organization’s internal ITS staff, external consultants and Microsoft
provided engineers working at the organization’s head office in Ottawa.
Table 29 below shows a summary profile of the ECM project:
Table 29: Summarized Profile for Case D
Project Name Project
Purpose
Stakeholder
Composition
Project Cost
(Completion)
Project
Duration
Post deployment Remarks
Enterprise
Content
Management
(ECM)
Replace a
legacy system
with a COTS
based product
Core Project team,
Extended project
team, support
personnel,
consultants,
Vendor, Business
users
11.5 Million
(CAD)
32 months
Project achieved some of the
promised benefits and
experienced moderate post
deployment issues;
However, due to significant
issues with integration and
handling changes- client
satisfaction was very poor.
After experiencing a significant financial and time loss from a failed attempt to upgrade the
existing ECM between 2009 to 2012, the organization decided to refine its objectives, and select
a new technical direction. The key business drivers and requirements of the project remained
largely unchanged. However, the new technical direction was expected to provide a broader
collaborative platform, which had emerged as an organizational need.
The ECM program was undertaken with a promise of providing the organization a more
disciplined approach towards managing electronic documents and email, and delivering a
foundational infrastructure (including people, processes and technology) for improving the
management of organization’s information assets. Specific benefits that were hoped to achieve
under this program included: (1) increased end user satisfaction with the content management tools
Page 167 of 307
as filing, finding, and sharing information would be more efficient, and searching for documents
and emails would be easy, fast, and consistent, (2) increased staff productivity resulting from a
faster access to reliable and trustworthy information, reduced time and effort spent on managing
paper-based documents, reduced need for physical space to store paper-based records, and
increased accuracy of responses to ATIP requests, (3) increased quality of content by providing a
clear understanding of what documents need to be stored, a proper guidance for disposition of
information that are no longer of any value, thus reducing the likelihood of losing vital information,
(4) reduced risk by enabling the organization to effectively meet its legislative, corporate and
contractual obligations, and (5) consolidated foundation for future evolution of the organization’s
collaborative tools and processes. The legacy ECM was missing significant features and
enhancements made within the industry over the past decade. Features such as wikis, blogs, and
shared calendars, and support for co-authoring were available by a majority of the software
vendors in ECM industry.
In addition to the project’s fulfilment of promised benefits, we also collected post deployment
incident statistics and user satisfaction data to assess the level of implementation success for this
project. Similar to preceding three cases, pre-established set of critical success factors that were
obtained from the IS literature is augmented with the factors identified through project’s lessons
learned activities. Project’s lessons learned identified several CSF for this project which included:
an increased focus on usability and initial priority on end-user’s concerns in order to gain strong
user adoption, implementing technology that is compatible with the organization’s current
technology landscape, staffing key positions using organization’s internal staff in order to ensure
accountability and control over the vision, scope and plan, and facilitating change by introducing
Page 168 of 307
new functionality in a gradual fashion through multiple “waves” of deployment over the entire
lifecycle of the project.
We interviewed a total of 9 members for Case D who were directly involved with the
implementation of the project. The workgroup interviewed for Case D included original project
team members, members of the customer-facing service team, and members of the user community
or the business users, and the COTS vendor providing the COTS software.
5.4.1 Implementation process for Case D
Case D involves an example of an organization-wide deployment of a COTS product. This
implementation effort of the organization was a second attempt to replace an existing COTS
product which was used for the same purpose.
Six months prior to the initiation of the ECM project, host organization started an evaluation
process through a Request for Expression of Interest (RFEOI). This RFEOI process engaged 10
different COTS vendors (HP, EMC, OpenText, IBM, Oracle, Microsoft, Autonomy, Hyland,
Minisis and Xerox). Project team used a range of criteria to assess each product, supported by the
information collected through the RFEOI industry sources (such as Forrester, AIM, Real Story
Group and Gartner), and advice from external procurement experts. This led to a shortlist of two
products, Microsoft’s SharePoint and OpenText. Both vendors were invited to participate in a
formal RFP process for this project. On the basis of technical demonstrations and written
submissions, it was found that both products offered very different strengths and presented
different challenges. As both products clearly satisfied most of the project’s requirements, price
Page 169 of 307
became a key differentiating factor between two vendors. Microsoft presented a lower costs, in
comparison with the estimates provided by OpenText, for the initial installation, software
acquisition and overall lifecycle costs including ongoing licensing and support costs of the COTS
product. In addition to the cost savings, Microsoft demonstrated a COTS-based product that
provided better collaboration, better integration with the organization’s existing office software
and enhanced usability, which overall were regarded as key elements in gaining user acceptance.
In addition to product’s capabilities, numerus examples of SharePoint-based ECM deployments,
within organizations that are similar to the host organization, was also an influencing factor.
Considering the product features, productivity factors, and existence of a rich knowledge-base
within the industry, Microsoft was chosen as the preferred vendor and awarded the contract for
this project.
ECM project followed an agile development and deployment approach. Reason for adopting a
multiple-delivery approach was the large scope of this project and identified project risks. Lessons
learned from an earlier failed attempt of the same initiative indicated ‘scope management’ as a
significant risk factor for the project. Each department of the organization owned a large volume
of corporate data in the form of various documents and communications records. Moreover, the
migration and conversion process for each department or functional group was not very different
compared to each other. This motivated the project team to follow a multi-phased delivery
approach where deployment of the new COTS product will be limited to a single functional unit
at a time.
Activities related to the implementation of Case D appears to be segmented, at the minimum, into
seven distinct areas.
Page 170 of 307
Although the project’s initiation phase was short and fast compared to previous three projects, a
significant amount of groundwork for the current project was completed prior to its initiation.
Failure of the initial replacement attempt motivated the project team and the program management
team to be extra vigilant in this second attempt. This resulted in an extensive focus on procurement
activities and vendor engagements.
Common project initiation activities such as setting up project controls, conducting risk
workshops, arranging a project kickoff meeting, and soliciting approval for the project charter
were observed at this phase. Preparation of several important project artifacts such as ‘project
charter’, ‘business requirement document’, and ‘project communication plan’ required
significantly less effort at this phase as these artifacts were previously developed during the earlier
attempt of the same strategic initiative. Having these critical artifacts in a ‘near-complete’ state
also enabled the project team to execute procurement related activities much earlier.
Once the project charter and the BRD were approved by the project’s steering committee, project
management team started finalizing a comprehensive project management plan. Processes required
to secure project’s required human resources were also executed at this time. As the new COTS
product, replacing organization’s existing ECM, will host data and documents that are considered
as corporate resources, project team ensured that due diligence is done in the areas of Access to
Information Privacy (ATIP), Audit, Continuity of Operations (COOP), Procurement, and
Knowledge and Information Services (K&IS). Unlike earlier three cases, project under Case D
was able to finalize most of the business and technical requirements at this stage of project.
However, several of the technical and functional requirements needed refinement and
Page 171 of 307
modifications as the second attempt of this project had selected a different COTS solution
compared to its predecessor.
Once the project team embarked upon the third set of logical activities, the project team actively
started to manage the selected vendor and the service provider for this project. Such an approach
by the project team was primarily due to a new technical direction resulting from the adoption of
a new COTS product. Additionally, whereas the legacy ECM application was a dedicated and
specialized tool for documents management, the new COTS (SharePoint) was not seen as a
specialized product in this context. SharePoint is commonly used for collaboration and sharing.
For this project, the team decided to modify existing business processes and customize the features
of a default SharePoint version to make it suitable as an enterprise content management system.
Although the delivery plan of this project consisted multiple phases, each phase focusing on one
functional area at a time, conceptual design was more encompassing and took a holistic view of
the entire solution. Technical gaps and functional capability gaps were identified by the design
team. Based on these gaps, appropriate mitigation strategies were also developed by the project
team. Other significant activities observed during this phase included finalizing the project
management plan, maintaining a close coordination with the infrastructure team to setup the
development and test environments for the project, and coordinating with the target business unit
for initial deployment to device a test strategy.
The fourth group of activities involved developing a detailed design and architecture for each
module of the COTS system. Low level or component level design activities performed during this
phase were not significantly different compared to the previous three Cases; however, focus of the
project team at this stage was a single functional group. Processes and activities under this logical
Page 172 of 307
group were repeated for each functional group as each business unit of the organization required
very similar content management features. Activities of the project team included developing test
cases and test plans, developing delivery plan and obtaining approval for the plan, redesigning
business processes, designing interface layouts for the end users, developing migration plan and
data conversion plan. This group of activities was partially repeated for each functional area under
the master delivery plan of the project.
Immediately after the approval of project’s technical design and the technical implementation
plan, project started implementing the design specifications developed on earlier phases. Major
segments of work packages at this stage included designing the interface of the new ECM system,
preparing existing data and meta data for migration to SharePoint’s SQL database, preserving the
data relationship integrity during the migration, and developing additional custom codes and
solution to facilitate data migration. Quality assurance and integration testing started soon after the
first build was deployed into the QA environment. At the same time, client was supported by the
project team to prepare for the UAT related activities for the recently completed build.
As the project moved into its sixth logical group of activities which involved transitioning the
validated product into production environment, the project team also started the conceptual design
activities for the next functional group. Once the first build was validated by the QA team, it was
migrated to an environment which is very similar to the production environment for further
validation by the business users. A few defects and unexpected system behaviour were identified
by the user community which were immediately addressed by the project team. Primary focus of
the project team at this point were on data migration, business process update, user education and
training, support team’s knowledge transfer, ensuring availability of records from the
Page 173 of 307
decommissioned system. Once the business approval was received for delivered modules, release
management and change management activities were initiated to migrate the solution to the
production environment.
As the new ECM became a system of record or a live system in the organization’s production
environment, project management team started to collect feedbacks and comments from project
team, support teams, and user community on their entire experience in respect to this project. Other
activities by the project team involved monitoring the new ECM’s stability, overseeing the
decommissioning of the legacy ECM, and preparation of the ‘project completion report’.
Table 30: Project Activities – Major Phases and flows
Project Activities Respondents Phase
Upgrade attempt for the existing system
Alternative analysis
D01PT, D02PT,
D06BU, D07BU,
D09IT
Pre-Initiation
Obtain relevant PMD Toolkit Material for the Project
Become familiar with the organization’s PM Processes and Tools
Establish project controls and project administration
Gain basic understanding of project assignment
Establish Governance Reporting for the project
Develop the Preliminary Project Charter
Conduct project Kick-Off meeting/presentation
Obtain approval of the Preliminary Project Charter
Obtain Approval to Gate into Ph1.
D01PT, D02PT,
D03PT, D04PT,
D05VR, D06VR,
D07BU, D08BU,
D09IT
Initiation
Develop preliminary Project Management Plan
Obtain Approval of the preliminary PMP
Generate estimates for project schedule, activities and resource
requirements
Based upon estimates generated, update Planview [time allocation
tool] accordingly.
D01PT, D03PT,
D04PT, D05VR, ,
D07BU, D08BU,
D09IT
Requirement
analysis
Page 174 of 307
Arrange Project Team Logistics
Produce Business Use Cases/Business Process Modelling
Documentation
Obtain Approval for Business Requirements Document (BRD),
including Business Use Cases
Finalize Project Management Plan (PMP)
Obtain Final Approval of Project Management Plan (PMP)
Establish Project Design Guidelines and Modelling Standards
Execute Procurement Process (for additional software, hardware, or
services)
Engage and Begin Managing Chosen and Approved Vendor/Service
Provider
Produce Technical Conceptual Design (Solution Architecture
Document (SAD)
Obtain Stakeholder Approval of Technical Conceptual Design (SAD)
Acceptance Plan
Complete the System Requirements Specifications (SRS) and
Produce System Use Cases
Initiate Request for Change (RFC) for planned implementation of the
recommended solution into the Production environment
Devise the Project Delivery Strategy
Coordinate setup of the Development and Test Environments
Develop Preliminary Support and Operations Guide
D01PT, D03PT,
D04PT, D05VR,
D06VR, D07BU,
D08BU, D09IT
Conceptual
Design
Ensure due diligence is done in the following areas:
ATIP, Audit, COOP, Procurement, SFS, and K&IS
Oversee Production of Detailed Requirements
Obtain Approval of Detailed Requirements
Oversee Production of Design Documents
Obtain Approval of Detailed Design
Organize and Lead Tactical Planning for Development based on
Approved Detailed Design
Write the "To Be" (future state) Business Procedures
Produce the Project Delivery Plan
Obtain Approval of the Project Delivery Plan
Produce Test Plan
Develop Test Cases
D01PT, D02PT,
D03PT, D05VR,
D06VR, D07BU,
D09IT
Design and
Architecture
Oversee the Solution Build
Develop Programming Components and data migration tools
Plan and Perform Integrated Test of Built Solution in the Dev
Environment
Prepare User Guide
Finalize Implementation Preparation Planning and Due Diligence
Prepare for QA Testing
Plan and Perform QA/Functional Testing
D01PT, D02PT,
D04PT, D05VR,
D06VR, D09IT
Development
Page 175 of 307
Arrange for Pre-Prod and/or Prod Environment Set-up
Facilitate Client Preparation of UAT Test Cases
Define Ongoing Support Agreement(s)
Finalize Delivery Preparation Material
Organize and Facilitate User Acceptance Testing (UAT) - Release
Candidate
Obtain User Acceptance Test (UAT) Approval of Solution
Obtain Client Approval to Transfer Solution into Production
Finalize Transition Plan
Prepare for Solution Transition into Production
Obtain Approval sign-off of Support Agreement(s)
Coordinate Delivery of User Training
Deploy Solution to Production
Do Formal GO LIVE (Final Release)
D01PT, D02PT,
D03PT, D04PT,
D05VR, D06VR,
D07BU, D08BU,
D09IT
Delivery
Provide alternative and support for missing documents
Oversee Project Stand-Down
Produce Project Completion Report
Obtain Approval of Project Completion Report
Obtain Approval to Close the Project
D01PT, D02PT,
D03PT, D04PT,
D05VR, D06VR,
D07BU, D08BU,
D09IT
Close out
In summary, implementation related activities for the ECM project appeared to be segmented, at
minimum, into seven logically distinct groups. Although the activities performed during initiation
and requirement analysis were much lighter in terms of workload compared to similar phases of
the other three Cases, a differentiating characteristic for this project was an earlier failed attempt
to upgrade the project. That earlier upgrade attempt of the ECM project performed a significant
requirement analysis that formed a major input for the conceptual design activities during the
project’s second attempt. Other distinct activity groups for this project included conceptual design,
design and architecture, development, delivery, and a non-repeating “close-out” phase.
5.4.2 Control configuration and balancing
During the initial upgrade attempt for the legacy ECM system, the initiative was primarily driven
by the vendor and the vendor provided technical consultants. With a long history of working with
Page 176 of 307
the same vendor and using the same product, most of the vendor provided recommendations and
suggestions were accepted in an uncontested manner by the organization. This was one of the
significant reasons that had tremendous adverse effect on the initial upgrade attempt that failed.
This failure of a strategic initiative and the significant cost resulting from it had significant
influence on the initial control configuration of this project. This influence is evident from a more
proactive control approach adopted by the program management as well as the project team
overseeing this project.
Six months prior to its initiation, the project engaged multiple well-known top tier COTS vendors
to determine a viable technical direction. During this pre-procurement phase, existing market
information collected by the project and technical proposals submitted by vendors were closely
scrutinized by the project team. Once a contract was awarded, the relationship between the project
team and the vendor remained at a very formal level which included frequent status tracking by
the project team as well as close monitoring of any vendor proposed change requests. This
indicated an approach taken by the project where the key interest lies on “what the vendor is doing”
as opposed to “how it is being done”. Relationship between the project team and the business users
and technical support teams was more mature and required less monitoring. Informant D07BU
stressed on a positive and open partnership between project team, procurement and the business
clients:
“There were many open, transparent, honest conversations about what to do. The
right people were at the table. This likely resulted from an openness and desire to
get this project right after the results from the previous ECM project.”
Page 177 of 307
Designing and architecting of the overall COTS solution and individual modules were performed
by the core project team. Besides SharePoint developer, this team also included organization’s
technical architect and subject matter experts from the vendor side. Project maintained the ‘status
quo’ in terms of control configuration for the early phases of the lifecycle. This continued trust
based relationship between the project team and the IT Support teams introduced some negative
outcomes for the project which included delayed delivery of project’s infrastructure components.
Project team member D01PT indicated this by describing the negative impact of such trusted
relationship:
“Too often project team members did not trust their own expertise and we relied
quite heavily on vendors to supplement with their SME’s [subject matter
expertise]…….I would say that before any external SME arrives on site that we
ensure that the appropriate software and environments are ready so they can
begin their work immediately. In our case because there were delays in signing
the SOW [statement of work] there was no time left between receiving the pre-
installation checklist to when the external SME arrived on site. Several days were
spent preparing the environments and securing the proper software.”
During the design phase, project - client interactions were significantly reduced; However, the
project team maintained a close coordination with the business to acquire any clarifications related
to its design decisions. Vendor was managed closely by the project team through conventional
status reports and encouraging them to follow the established design processes of the organization.
Page 178 of 307
Project team conducted its own review of the vendor provided design recommendations to ensure
accuracy and alignment of design components. Vendor also provided prescriptive guidelines for
the project team to follow for the data and meta data preparations. Despite an bi-lateral and tight
control style exercised by the project team, several instances of delayed design decisions were
observed at this phase. Primary data indicates a lack of understanding regarding responsibilities
and an absence of roles clarity were to blame for these delays.
Once the data preparation, data migration and COTS configuration activities begun, the project
team and the vendor moved into a relationship that reflected a higher level of mutual
understanding. Very frequent communications, informal discussion sessions and informal
exchange of ideas were used to coordinate the activities of this phase as opposed to outcome based
controls utilized in earlier phase. Advice provided by the vendor was promptly adopted to resolve
unexpected road blocks encountered by the project team. This behaviour of the project team was
primarily due to lack of expert knowledge regarding the new COTS product within the
organization and project team’s belief that vendor share the same urge to complete the project on
time with high accuracy of deliverables. Common practices like the manner of code repository
utilization and code migration methods were used by project and the vendor in a similar way. Both
the business users and IT Support teams were expected to aid the project as needed, however, no
additional control mechanisms were deployed to proactively control the cooperation.
As the project moved into the delivery stage, status review and validation activities related to
deliverables significantly increased. This increased level of scrutiny resulted from a few
unexpected defects that were discovered during the user acceptance testing. Project team revisited
Page 179 of 307
the contractual obligations by the vendor to ensure that contractual commitments are fully met and
pointed out the deliverable gaps in a more formal fashion by escalating the issue to the program
oversight committee of this project.
The client and the project team of this project exercised joint communications approach to update
the business users and project steering committee on the status of deliverables and the progress of
defect remediation planning. Besides, lessons learned and feedback collection, internal IT Support
team and COTS support team also utilized job shadowing with project team and informal coaching
activities to get a good grasp on the new COTS platform. A tight and unilateral control style on
the vendor was maintain by the project at this very end of the project’s lifecycle. This unilateral
control style was a direct result of the negative experience, related to the deliverable qualities, from
the preceding phase of the project.
Table 31: Observed Control Configurations in Case D
Phase Vendor IT Support
Team
Business
Clients
Salient Factors Related to
Control Decision
Pre-initiation Coordinated Trust Trust Absence of formal authority, strive
to reach an agreement
Initiation Authoritative Trust Trust Absence of previous contract,
Negative past experience, pre-
disposed trust condition, previous
cooperative settings,
Project Context,
Organizational Context (PMO
prescribed compliance
requirements)
Page 180 of 307
Requirement Trust Trust Trust Reduced ability to monitor
behaviour, lack of technical know
how. Positive assumption on the
validity of previous requirements
Conceptual
Design
Trust Authoritative Trust Missed delivery dates
Design and
Architecture
Coordinate Authoritative Trust Lack of clarity for roles and
responsibility; high task inter-
dependency
Development Trust Authoritative Trust Absence of technical expertise;
Perceived attitude of the other
actor
Delivery Authoritative Authoritative Authoritative Disputes with vendor,
Disagreements, Competing
internal resources
Closing Authoritative Authoritative Trust Following Norm
In summary, the control portfolio for Case D demonstrated multiple distinct control configurations
over the project’s lifecycle, which was significantly different compared to the control
configuration of the projects in Case A and Case B, but somewhat similar to Case C. Although
missing formal authority in the project management team led to a trust-based control configuration,
the project team chose to continue a trusting relationship with the vendor, internal support teams,
and the client in subsequent project phases. Such a decision can be attributed primarily to a lack
of leadership strength and project management experience, as a junior project manager replaced a
more experienced project manager. However, the second half of the project’s lifecycle was
dominated by an authoritative control configuration as the project management team was not able
to re-establish the desired level of shared understanding. Furthermore, the level of negative
anticipation remained high among the project team members regarding the future performance of
other project stakeholders.
Page 181 of 307
5.4.3 Stakeholder engagement and sensitivity
Initial attempt to update the existing ECM demonstrated significantly light stakeholder
engagement in general. Despite a large project budget, most communications were centered around
and project activities were performed by a small set of project team members under the direction
of the legacy system’s provider. The vendor of the existing ECM shared a long working
relationship with the host organization. This vendor also enjoyed the reputation of being one of
the top consulting firm globally. As a result, the project team relied heavily on the vendor’s
guidance and directions for the initial upgrade attempt of the ECM. This resulted in a much smaller
stakeholder footprint compared to other similar sized projects. Once the initial attempt to upgrade
the organization’s ECM failed, the project steering committee decided to replace the existing ECM
instead of upgrading it. This decision had noticeable impact on project’s stakeholder engagement
approach as the number of stakeholders needed to be involved with the new initiative significantly
increased.
During the Pre-initiation of the project, several vendors (10) and their COTS based products were
reviewed to determine a suitable alternative. Diverse groups of IT support team, business users
including the business sponsor of the project, procurement department, and strategic investment
oversight committee were also involved with the process. Regardless of the heavy involvement by
several stakeholder groups, most requirements for this initiative were derived from the work
performed under earlier upgrade effort and existing documentations related to the legacy ECM.
This resulted in a low engagement level by the stakeholder during the initiation phase of this
project. Processes and tools utilized by the project team in early stage of the project also limited
Page 182 of 307
stakeholder’s ability to provide effective inputs and project team’s ability to capture critical
feedbacks. Informant D08BU stated multiple such challenges during initial phase of the project:
“….while there was a good process for the group to share their ratings in the
consensus sessions, I think participants would have liked more opportunity to
discuss their comparison of functional RFP ratings within their own functional
team first…not to mention that RFP deliverable was too complex. It had a main
body and 12 appendices. It was too time-consuming to review for us and created
lots of potential for mistakes…..I think the demos were too long. It also resulted in
diminishing returns.”
This trend continued as requirements collection and validation were led by a small set of
individuals which included business analysts and vendor. Due to earlier efforts of requirements
gathering and existence of detailed documentations for legacy ECM, influence of stakeholders on
the project’s activities was also much lower compared to the stakeholder influence observed in
Case A and in Case B. Cost of such low engagement and low sensitivity were apparent from the
remarks of respondent D02PT:
“Our project definitely could have benefitted from a broader user representation
on the evaluation Team…..even one or two more people would have brought helpful
broader knowledge of the user needs to the table. As it was, it was a bit stretch for
the few individuals to represent the breadth of needs. As well, it would have been
useful to have more users participate in the demos”
Page 183 of 307
Once the requirements document and procurement process were finalized, the project activities
showed a similar trend of stakeholder engagement compared to the earlier cases. During the
conceptual design of this SharePoint based ECM and the implementation planning of the COTS
modules, was activities were mostly driven by the design and architecture group of the project.
Due to a completely new technical solution, data mapping considerations between legacy system
and the new COTS, and enterprise integration considerations for the new ECM triggered additional
changes to previously developed design documentations. Project team did not invite all groups of
stakeholder who were previously involved; however, due diligence was exercised by considering
the solution from a broader perspective such as enterprise integration and legislative requirements
(ATIP). Primary stakeholders for project activities at this stage included enterprise architect,
solution architect, solution integrators, vendor’s solution engineers, selected developers, and data
migration specialists.
Upon the completion of design and architecting activities for the new ECM, project team’s efforts
were centered around implementing the design, developing support modules, configuring the
COTS applications and preparing the existing data for the legacy system for migration. Activities
at this point were highly focused and performed by the project’s development team.
Responsibilities for performing these activities were limited to a small number of team members
which allowed very little or no room for deviation from the original implementation plan. A
reduced opportunity for feedback from other stakeholder groups such as different departments of
the organization indicated a low engagement by as well as sensitivity of stakeholders. During the
deployment of the customized COTS product, a similar pattern for the stakeholder orientation was
Page 184 of 307
observed. The number of participants slightly increased as the delivery of the configured system
required new activities such as client validation, joint workshop and walkthrough sessions. This
also required support from COTS support team, infrastructure support team and respective
business areas. Although an increased involvement from different stakeholder groups, the
opportunity to influence the delivered solution was minimal.
Figure 11: Stakeholder orientation for Case D
High
Stakeholder
Engagement
Low
Stakeholder
Engagement
High
Stakeholder
Sensitivity
Low
Stakeholder
Sensitivity
Delivery & Transition
Initiation
Requirement Analysis & Collection
Pre Initiation
Development
Design & Architecture
Conceptual Design
Closing
Page 185 of 307
Closing phase of this project contained multiple lessons learned sessions to collect accurate
feedback on project’s performance and management of challenges. Online questioners were
distributed to wider audience involved with various project activities at different stages of the
project. Although not everyone involved with the project provided the feedback, responses
received were quite substantial in terms of number and quality of information. Interestingly, the
problem areas identified through the ‘lessons learned’ activity seemed to be repeating from phase
to phase which indicated an inability of the project management team to incorporate or address
any identified concerns in a timely manner.
In summary, the ECM project in Case D showed a significantly different stakeholder orientation
compared to Case A and Case B. Whereas both Case A and Case B showed an increase in
stakeholder engagement and stakeholder sensitivity during their early lifecycle phases, Case D
showed a decline in stakeholder engagement. The early lifecycle of the ECM project showed a
moderate to high sensitivity level and a moderate to low engagement level. Although an earlier
effort of requirements gathering and existence of detailed documentations for the legacy ECM
were responsible for such a stakeholder orientation, the impact of this low stakeholder engagement
was quite detrimental for the ECM project. Similar to Case C, the stakeholder engagement level
gradually increased toward the end of this project’s lifecycle. Additionally, the sensitivity level
increased as most of the client’s feedback appeared in the form of an enhancement, defect repairs,
or change requests. Figure 11 above captures this dynamic nature of stakeholder orientation for
the entire project lifecycle for Case D where each project phase is plotted on a different quadrant
of the stakeholder orientation grid depending on the combined engagement-sensitivity level of the
phase.
Page 186 of 307
6.0 Cross-Case Analysis and Synthesis
In this chapter, we discuss the evidence from all four cases and how it relates to the linked theories
presented in Chapters 2 and 3. In Section 6.1, we discuss the results of the cases as a set and use
the cross-case results to answer research questions 1 and 2. In Section 6.2, we identify and discuss
control configurations and control balancing scenarios for each project phase. In Section 6.3, we
discuss the phase-specific stakeholder orientation for the projects. Finally, in Section 6.4 we
capture the relationships among control configurations, processes, stakeholder orientations, and
CSF for the project.
6.1 Enterprise COTS Implementation Model
In this section, we analyze all four cases from an implementation perspective of enterprise COTS.
In addition to the identification of a holistic COTS implementation model, we examined
implementation-related processes and tools that played significant roles in influencing one or more
critical success factors for our selected projects.
In analyzing COTS implementation for Case A, B, C, and D, we primarily focused on two
important aspects: (a) the end-to-end process followed by each project team in terms of high-level
project phases, and significant linkages between the phases; and (b) key project activities
and implementation-related tools and processes utilized during different phases.
Because our first two research questions align with the qualitative research type—“the discovery
of regularities” (Miles & Huberman 1994)—we adopted an event structure analysis approach to
identify critical elements and explore their relationships. Analyzing the primary case data for all
Page 187 of 307
four cases, we constructed an event matrix that arranged a series of concrete events in
chronological order. This event matrix was then further analyzed to group events into different
logical domains (presented in Figure 12), preserving the sequence and salience of events. This
analysis facilitated answering both research questions concerning COTS implementation
processes, as a process is essentially a string of coherently related events (Miles & Huberman
1994). Based on a temporal- and purpose-based categorization of all implementation activities for
all four projects, eight major categories appear as salient: (a) pre-initiation, (b) initiation, (c)
requirement and planning, (d) analysis and conceptual design, (e) design and architecture, (f)
development, (g) delivery, and (h) close-out.
In examining end-to-end implementation flow, we took a recommended project implementation
template from the organization’s electronic resource repository as an a priori model. This model
or general implementation guideline, sequential in nature, was then further refined based on our
developed-event matrix. This refined model, containing eight distinct groups of activities, is
presented as the new COTS implementation model (Figure 12).
Page 188 of 307
Figure 12. Agile COTS Implementation Model
The implementation flow, presented in Figure 12, captures the logical sequence followed by all
four projects with some minor variations. The host organization’s project management and
implementation practices strongly resembled a traditional waterfall style, and adoption of agile
methodology within the organization was still in its infancy. The nature of the COTS under
implementation, the project manager’s knowledge of agile practices, and the project management
office’s project implementation guidelines were also responsible variations in implementation
practices across all the cases. This is evident from the observations that a highly agile
implementation process was followed in Case B and near-waterfall implementation practices were
followed in Case C. However, after a synthesis of practices and sequences followed by all four
projects, the overall implementation model appears much closer to an agile than a waterfall model.
It is also evident that the developed COTS implementation model is not a pure agile
implementation model as requirement analysis and closing activities were often not repeated.
Whereas Figure 12 presents the implementation process from a macro level showing major phases
of the projects, Figure 13 presents our findings from a micro or activity level. This activity-level
analysis captures the inter-activity relationships and the stakeholder groups responsible for the
success of each key activity. Since the pre-initiation activities of a project are often not considered
part of the project, the remaining seven distinct project phases of the COTS implementation model
from Figure 12 are logically grouped into five broader categories. The resulting model, showing
stakeholder engagement at the activity level, is presented in Figure 13.
Page 189 of 307
Figure 13: COTS Implementation activities and stakeholder engagement
In the following sections, we describe the COTS implementation flow model presented in Figure
12. This discussion also focuses on COTS-specific key activities (Figure 13), significant processes,
tools, and procedures employed by different project teams to successfully overcome challenges
and manage CSF during various phases of the project.
Page 190 of 307
Pre-implementation Phase
The pre-implementation phase, although often not identified as part of a project’s formal lifecycle,
is the very early set of activities performed in preparation for a project. Our micro-level analysis
of each project’s activities indicates the existence of this phase where business users primarily
assume a leading role to turn an investment proposal into a COTS implementation project. An
organization’s management personnel usually initiate IT projects when they have an opportunity
to enhance business processes and increase operational efficiency (Ward & Daniel, 2002).
Additionally, compliance, public safety, and reputation are among the driving forces behind most
strategic IT initiatives in government organizations. Public organizations analyze these factors
when exploring business opportunities. For large projects, this phase comes before an IT project
begins. Informants from all four cases reported compliance, reputation, and efficiency gains as a
few of the key driving motivations behind each project. Activities performed during the pre-
implementation phase, such as business-case development and requests for proposals (RFP), play
an important role in COTS implementation success. Although the development of a BRD and RFP
are closely related to the core activities of a project, a large portion of our identified activities are
administrative in nature. For example, informants from each case reported to perform all or a
subset of the following tasks: contacting ITS portfolio management to confirm that the project is
in a portfolio schedule, obtain and review the business and IT proposed initiative summary sheet
(ISS) with its associated cost estimate spreadsheet, review the business case from the Investment
Governance Office (IGO), and prepare for project-gating requirements (refer to Appendix for gate
overview).
Page 191 of 307
Initiation
We have identified initiation as one of the earliest phases for each project’s lifecycle. Traditionally,
this phase is identified by the activities taking place immediately before or after the project
chartering and kick-off meeting. A similar trend was observed for all four selected cases, but
instances of overlapping with pre-initiation was observed in Case D. Regardless of the demarcation
points, the objective of this phase remained same for all cases. This included expressing and
understanding the general requirement of the technical initiative, type and goal of the project, size
and time horizon of the initiative, the project’s key stakeholders, general resource requirements,
organizational impacts, budget requirements, assumptions, risks, and risk mitigation strategy. For
our selected projects, overlapping of these activities with the pre-initiation activities resulted
primarily from an active role played by the program management who have already completed
some of these activities on behalf of the project (Case C).
Besides the activities performed during the business opportunity proposal and the development of
the business case, developing and managing the RFP process appeared as one of the most vital
activities for Cases A, B, and D. Because Case C decided to use the same COTS product and
vendor selected by Case A, this decision saved the project considerable time and effort with respect
to its RFP process.
A government RFP is usually significantly different compared to a private sector RFP and often
complicated due to policies and legal constraints (Elgazzaret al. 2005). However, the role of RFP
observed in the current study clearly emphasizes the importance of achieving the right level of
requirements, analysis of the market and product space, considering the challenges of a niche
market, and geographic restrictions. Although debates on the merit of RFP continue (Popp &
Page 192 of 307
Dallis, 2012), because government organizations are subject to principles of fairness and
transparency, they cannot avoid such process. This is primarily because an RFP process provides
an open and equal opportunity for every qualified vendor to respond to a project opportunity and
minimize any preferential treatment. Regardless, the Gartner Research Group has presented
research findings that indicate that an organization increases its chances of selecting the right
vendor and product by 40 percent if it executes the RFP process well (Karamouzis & Longwood,
2007). For example, Case B clearly affirmed the significance of the RFP. Its initial RFP was unable
to acquire a successful vendor and so the project team had to make a second attempt by modifying
the initial RFP requirements. This cost the project and the organization nearly 2 million dollars
(CAD). On the second attempt, the project team produced requirements to a greater detail and was
able to achieve three very important positive effects for the project: (a) selecting the right type of
vendor contract, (b) achieving accuracy in work scoping, and (c) feeding the requirement and
analysis phase with the bulk of the requirements. Informant B04PT, a senior business analyst, for
Case B made the following comment.
When you going out with an RFP … your RFP forms your requirement even though
their actual requirement and planning phase is happening at much later stage. …
Our RFP requirements were very detailed from a business perspective … what the
system needs to do … they were very detailed … extremely detailed. … From a RFP
perspective we didn’t have a bidder who didn’t know what we wanted to do and
[the winning vendor and solution integrator] had a really good idea about the
business processes.
Other significant process and tools identified during the initiation of an enterprise COTS
implementation belong to the following categories:
Page 193 of 307
RACI matrix/chart (Tool):
A RACI (Responsible, Accountable, Consulted, Informed) matrix—more formally known as a
linear responsibility matrix—lists the core deliverables vertically as row headers and the various
stakeholders responsible for those deliverables as column headers. RACI categories are:
1. Responsible: The person or role responsible for creating or developing the
product or performing the task.
2. Accountable: The person or role responsible for the quality of the product or task.
3. Consulted: This person or role consulted to provide information that is input into
the product or task.
4. Informed: Stakeholders are informed about the actions taken by the actors listed
under Responsible section.
The RACI matrix can be perceived as one of the simplest of stakeholder analysis and engagement
tools. In a complex IS implementation involving multiple partners and multiple vendors,
disagreement over requirements, refusal to take ownership of requirements, or missing
requirements are not unusual events. A common source of these issues is a lack of contributions
from all relevant parties. This is often caused by the assumption that merely inviting the key
stakeholders to meetings or asking questions in requirements sessions ensures proper
contributions. A major flaw in this assumption is that active engagement is not necessary for
stakeholder contributions. One of the ways that IT projects can ensure the active engagement of
all key stakeholders throughout the IS implementation is by defining the RACI matrix. A sample
RACI Matrix is presented in Table 32 to show the task distribution for relevant stakeholders.
Page 194 of 307
Table 32: RACI Matrix
Role Groups / Individuals
RESPONSIBLE
Responsible for producing the deliverable:
• Determining direction & approach with the Approver
• Seeking concurrence from those on the consulted list
• Obtaining documented approval from the Approver
ACCOUNTABLE
Accountable for:
• Providing early direction to author of the process about approach and objectives
of the deliverable
• Approving the deliverable with consideration of the comments provided by
individuals on the consulted list
CONSULTED
Accountable for:
• Commenting on the deliverable, seeking advice from others in their groups, if
appropriate
• Supporting and endorsing the final deliverable (once approved)
INFORMED
• Consulted in order to provide views, requirements and information that will end
up as content in the deliverable
Support for the effectiveness of a RACI matrix is evident in both Case A and Case B, where each
project team heavily utilized this tool for all major implementation activities, such as requirement
analysis, business-proposed solution development, design and architecture, build books, user
guides, disaster recovery, and business continuity planning. An improper or lack of RACI
utilization can have the opposite effects, as in Cases C and D, where both projects struggled to
maintain desired engagement level for their stakeholders, experienced delays in deliverables due
to role confusion and identified missed requirements at a later stage of the project. According to
informant A04PT, a business analyst in Case A:
RACI chart is great way to get the right people engaged for the right task. … There
will always be someone doing the actual work but C (Consulted) and I (Informed)
parts are very easy to miss in large projects. … This might have some serious
implications down the road. … For us, it was absolutely critical. … The informed
Page 195 of 307
and consult part helped avoid some major disagreement and this is simply because
engagement is difficult unless you have some skin in the game.
Requirement and Planning
A second logical set of activities for all Cases were related to collecting or finalizing requirements
and formulating an overall implementation plan. Activities performed during the initiation phase
appear to have had a direct and considerable influence on activities performed during this phase,
as high-level requirements and constraints were identified during the initiation of each project.
These high-level requirements and constraints served as foundational blocks for the activities of
this phase. The objective of this phase appears related to development and communication of the
project charter and project management plan to everyone involved to ensure a common
understanding of the project objectives, strategy, and approach. Planning aspects of this phase also
aimed to establish a clear understating regarding team involvement and project organization,
communication expectations, and general project milestones or target dates. This ensures that
efforts are focused toward project goals. Additionally, one objective of this phase is to express the
client's business requirements and ensure that related non-functional requirements for the COTS
solution are identified (e.g., security, facilities, or business continuity).
For all instances of the selected projects, the requirement and planning stage was a much deeper
dive into the initial business requirements gathered during the RFP process. In addition to
reviewing the functional requirements at a more detailed level, the project teams also refined and
identified the non-functional requirements specific to the organizational environment. Vendor(s)
and the solution integrator were directly involved with project teams for each Case. Most
Page 196 of 307
informants identified this interaction to be absolutely critical. The reason for such a high level of
emphasis is that often the vendor-related contract was not finalized at this point (e.g., Case B, Case
D), and this direct interaction with the vendor gave everyone involved with the project a final
opportunity to minimize any possibilities of future change requests. Additionally, business
requirements developed during the pre-IT implementation period were primarily focused on
functional requirements from a “present-day” organizational perspective. These requirements
contained a very rudimentary definition of non-functional requirements. Solidifying the non-
functional and technical requirements and identifying product enhancement needs were also
performed during this phase. According to informant B06VR, the vendor’s technical lead in Case
B:
There were lots of room for interpretations which either drives/increase the cost
coz we are going to take a higher risk premium as a vendor. As a supplier we are
gonna to say—if there is this much slugging here, we are gonna have to charge
more because it could go this way or that way. It also gives more room for dispute
and CRs.
From a vendor perspective, this phase carried a lot of significance for Cases A, B, and D, as the
project teams exposed their existing systems to the potential vendor to help solidify work scope
and finalize their RFP-related bids. The high-level requirements released during the RFP typically
help vendors formulate their own perspective on the requirements. For example, one specific
requirement could be delivered out of the box. A second might be the specific requirement could
be a minor configuration on the COTS product. And the third could be addressed through a minor
glueware, and so on. It was only during this phase that both the vendors and the solution integrators
had an opportunity to perform due diligence on the RFP requirements. More specifically, vendors
Page 197 of 307
were able to validate the requirements with project teams, get a good handle on the legacy system’s
data in terms of volume and cleanliness, and comprehend the structure and complexity around the
metadata of the legacy systems. All these parameters directly influenced the scope determination
and bid precision for the vendors. According to informant B05VR, a senior project manager of the
vendor’s team in Case B:
With a fixed priced bid, we put in a risk premium on that. We come in and go—
yeah … we think it’s gonna cost a million dollar[s] but these are the things that are
[a] little bit variable and if [we] have any problems, we are gonna bid a million
and a half. That’s to cover our risk. What that does, increase the price of our bid
and we may now lose the bid because we have gone too high if someone else didn’t
take that risk premium.
Requirement Traceability Matrix (Tool)
During the requirement and planning phase, one very important artifact that was produced by the
project teams in Cases A and B was the Requirement Traceability Matrix (RTM). An RTM’s
primary goals are to: (a) provide a record of each stated, committed, and agreed requirement; (b)
track progress and decisions during the project regarding how each requirement is being addressed;
and (c) provide auditable proof at the end of the project regarding how each requirement was
satisfied and how the scope of work was met and controlled.
Establishing an RTM was critical not only for determining scope but also for controlling the
change request process at later stages of the project. For example, in Case B, the project’s RTM
initially contained 183 requirements and was partially reviewed by the vendor. Since the project
Page 198 of 307
implementation followed an agile approach, the RTM established at this phase evolved further
during the subsequent phases of the project. However, deliverables were validated against the
requirements in the RTM to trace its origin and connect the deliverable that satisfied that
requirement. According to informant B07VR, a solution integrator in Case B:
From the vendor perspective and from the client perspective … everything that we
do for a project better trace back to a requirement otherwise we shouldn’t have
been doing it. There should never have been a user story that couldn’t be attached
to a requirement in the SOW [Statement of Work]. The only case [in which] that
could happen [is] if this is a new requirement which should have been a CR
[Change Request]. Theoretically, between the SOW and CRs, that should be the
scope of what was delivered because some of the CR might have removed the scope
from the SOW and you get credit.
DeepDive ™ Process
Analysis of early implementation phases of Cases A, B, and C indicates the existence of a key
process, DeepDive™. This process was initially developed by the IDEO group (a learning design
company) for rapid product development. However, due to the effectiveness and efficiency of this
process in problem solving and idea creation, DeepDive™ is now increasingly used for many types
of activities (RapidBI, 2007). These include complex requirements analysis and conceptualization
for information systems development.
Page 199 of 307
IDEO’s original process consisted of five steps: (a) understand and observe; (b) synthesize; (c)
visualize; (d) prototype, evaluate, and refine; and (e) implement. This process is shown in Figure
14:
Figure 14: IDEO’s original process for rapid product development (Moen, 2001).
Boynton and Fischer of the International Institute of Management Development (IMD) further
refined IDEO’s original process into a generalized innovative problem-solving method called
DeepDive™ which was acquired by Deloitte Consulting in 2006. According to Deloitte, “The
DeepDive™ is a combination of brainstorming, prototyping and feedback loops merged into an
approach that executives can use with teams to help develop solutions for specific business
challenges” (Deloitte, 2010).
In Case A and Case B, DeepDive™ was used in two stages of the projects: requirement and
planning, and analysis and conceptual design. Typical DeepDive™ sessions lasted from few hours
to a full day. This process appeared very effective in the areas of requirements clarification, scope
solidification, and component design clarification. According to informant A04PT, a business
analyst in Case A:
Page 200 of 307
So it was important from their [Vendor] perspective to nail down the specifications
what exactly we wanted contractually and it was important for us too … to make
sure that all our [client] requirements are captured and interpreted correctly. …
Way down in development during the sprints at times a lot of requirements we
would say No, we wanted it that way and they [Vendor] were very clear and able
to go back in terms of traceability and say, No, we did a deep dive back in 2012
and this was … this was the specification.
Although Case C and Case D did not specifically use DeepDive™, both projects used some
variation of this process in early phases of the project’s lifecycle. As this process was not identified
as one of the standard organizational processes promote by the PMO, its utilization in Case C and
Case D were on a limited scale. A lack of utilization of DeepDive™ can be directly linked to several
requirement-related disputes experienced by both projects.
Analysis and Conceptual Design
Soon after the completion of requirements gathering and finalization, the activities for each project
showed a similar trend, which involved looking at the overall COTS solution from a conceptual
standpoint within the organizational context. The primary objective of this phase is to determine
which options or alternatives best meet the requirements of the target business solution for the
existing organizational context. Therefore, if a project is decided to proceed, this phase continues
with the stakeholders' decision and agreement on the most appropriate option or alternative to
develop a conceptual design for the solution. For all selected cases, an analysis was performed on
Page 201 of 307
various options against specified criteria, and a recommendation was presented to stakeholders
with substantiating documentation.
For Cases A, B, C, and D, analysis and conceptual design focused on requirements elaboration and
clarification. For example, the client and vendor jointly discussed and decided how a particular
requirement would be handled by the future system. Additionally, activities in this phase were
significant from the perspective of maintaining scope and system’s functionality and requirement
traceability. According to informant A09BU, a business analyst for Case A:
What we had was high level requirements, it would have been broken down into
three or four more granular requirements, then they became user stories, i.e. how
to do something, and we had traceability in there [using RTM]. In the end, what we
reported back was: this requirement is met by the following user stories, and these
were the tests that we did to confirm that the user stories were valid, so by the sign
off of all those test against all those user stories confirms that we have delivered
those requirements.
Engaging the extended development and support team of the organization (including the COTS
support team and ITS operations support) was a key factor for a smoother development cycle and
transition to support phase. The dotted line from this phase, leading to the pre-closeout link (see
Figure 12), indicates this engagement and information flow. Informants from all four cases
affirmed the significance of this information flow by presenting both positive and negative
evidence. For organizations that are similar to the host organization—where information
technology services (ITS) has a dedicated team for each core function (e.g., network, server,
storage, database, security, applications) and follow a rigid change-management process—delayed
Page 202 of 307
initiation of this engagement is very likely to impact project deadlines and critical milestones. For
the selected cases under current analysis, Cases A and B were able to avoid all transition-related
roadblocks by engaging the support team very early in the project life cycle. However, negative
consequences, such as infrastructure unavailability (e.g., servers, firewall rules, network
connectivity) resulting from a delayed engagement, were reported by informants in Cases C and
D.
Design Authority (Process)
The design authority process was one of the most effective processes introduced by the project
under Case B. It was established soon after the discovery of some discrepancies between client
requirements and vendor implementation of those requirements. This process consisted of two key
elements: design authority group and a design approval process. The design authority group
primarily consisted of roles drawn from the business users, like solution architect, enterprise
architect, data architect, and subject matter experts. At the beginning of each sprint, this group
reviewed the design aspects of the requirements and discussed how they should be implemented.
As a result of this discussion, the vendor captured the design decisions and initiated an approval
process by sending the proposed design to the client. The design approval process was a very tight
document review process where the design document went through multiple feedback
modification cycles between client and vendor. In Case B, the design authority produced nearly
60 design decisions throughout the life cycle of the project. Design decisions were proven to be
valuable in three key areas: (a) detailing how the system should behave at a lower level, (b)
minimizing assumptions on the vendor’s side regarding components being developed, and (c)
Page 203 of 307
ensuring alignment between delivered functionalities and specifications to avoid unnecessary
change requests.
All of the project team members, from both the client side and vendor side, supported the
effectiveness and efficiency of the design authority process. They also expressed an intention to
employ this process in future projects. According to informant B01PT, a senior project manager
in Case B:
Most time this work falls on the development team almost with little bit of
involvement from the BAs [Business Analysts]. … I would say establishing a design
authority that includes the vendor, includes the architects and business is really
critical. … There were a number of times that we had issues, questions or things
that were not going as they should and we fell back on the document that was
produced as a design authority to make sure that we were delivering to the design.
Although the name Design Authority was not adopted formally, Case B’s project demonstrated
similar activities during this phase. This involved establishing formal design decisions and using
them as reference points. However, for Cases C and D, design decisions were much less formal,
which introduced some design changes and design conflicts at the late stages of a project’s
lifecycle. Looking across the Cases, the design authority process helped reduce the development
cost and time for both Case A and Case B, as well as helped foster an active engagement level by
both the IT and business stakeholders. Clarifying the design decisions ahead of every sprint
significantly reduced the volume of communications for the offshore team on the vendor side.
Reduction of requirements related to communications not only reduced the idle time on the
developer side, but also minimized deviation from the requirements. Design decisions produced
Page 204 of 307
by the design authority also acted as a safeguard against unexpected change requests.
Unanticipated change requests can be a substantial source of expense for a fixed-cost project. A
review of the disapproved change requests found in the projects change management log indicates
that both Case A and Case B were able to avoid several high-impact changes in many instances
throughout a project’s lifecycle.
All four projects selected for this study followed a variation of agile implementation, but Case A’s
and Case B’s processes closely resembled a traditional sprint-based agile approach. Adoption of
the agile philosophy also caused the analysis and conceptual design activities to repeat itself
throughout the project’s lifecycle. From this stage of the project, there were two possible choices
for the next phase, depending on the complexity of the required functionality: the design and
architecture phase followed by the development phase, or directly into the development phase. The
second route was usually followed for simple and standard requirements that did not require a
design approval by the project’s design authority.
Design and Architecture
A distinct set of activities related to technical design and architecting were observed for all four
projects. The objective of this phase is to determine the detailed design of the solution, including
how it will integrate with the existing environment and systems. This phase also includes the
procurement of additional hardware, software, equipment and other materials, and identifies any
new operating and maintenance needs for the target COTS system.
In the design and architecture phase, the primary task is to take the input from high-level
requirements and user stories, and convert them into a detailed design document. The design
Page 205 of 307
document outlines every detail of the target system behaviour related to the requirements.
Informants from Case B and C reported some examples of these detailed behaviour for the target
system, including how the user interface (UI) is going to work, what rules should trigger when a
certain button is clicked, how the web services should behave, or notification procedure for
scheduled job failure. Because almost all four projects divided development activities between the
in-house team (glueware development by the solution integrator) and the vendor-provided
technical team (COTS or product enhancement by the vendor), design decisions made during this
phase were critical for both the project team and the vendor.
For Cases A, B, and D, this phase appeared to be a repeated set of activity as each project team
considered the design of a single module at a time. Case C, on the other hand, attempted to design
the complete solution at this point, taking an encompassing approach. Regardless, periodic design
review and repeat of design workshops were reported by the informants in Case C for the duration
of the development cycle of the project.
Development
Following the design of an overall solution, the technical architecture group passed the approved
design decisions to the project’s development team, which included vendor-provided developers
and offshore development teams. The objective of this phase appeared to develop and test the
business solution, including the required environments. The project team covered several stages
of testing at this point. Salient categories of testing for Cases A, B, and D included unit testing,
system testing, functional, and integration testing. For Case C, testing was mostly limited to
developer’s unit testing of the COTS modules. All four projects also drafted a production
Page 206 of 307
implementation plan, which included production roll-out, user and technical training, raising
organizational awareness, and clarifying operating and maintenance responsibilities. User and/or
procedures guides were drafted and revised as required by the projects.
Both the informants and secondary data of Cases A, B, and D indicate that development was one
of the most critical and challenging phases of the project. First, this phase was the biggest work
stream for all four projects, and second, and more importantly, several critical activities progressed
in a parallel fashion. For example, metadata conversion and validation, product enhancement, and
data migration and reconciliation all happened in an iterative fashion.
Implementation practices in Cases A, B, and D indicate that all three projects followed an agile
approach of IS development that was significantly different from the traditional waterfall
approach. In addition, both Case A and Case B had a component of an offshore-development
model, as the vendor’s technical teams were located in Europe.
Agile methods emerged over a decade ago as an alternative to plan-driven software development
methods (Dingsøyr et al., 2012; Agile Software Development, 2014). It is a philosophy of software
development based on iterative and incremental development. Conboy (2009) formally defined
agile as “the continual readiness of an information systems development (ISD) method to rapidly
or inherently create change, proactively or reactively embrace change, and learn from change while
contributing to perceived customer value (economy, quality, and simplicity), through its collective
components and relationships with its environment.” Besides the twelve principles outlined in the
agile manifesto (Agile Alliance, 2015), the core values of agile development focus on individuals
and their interactions, early delivery of working software, customer collaboration, and
responsiveness to changes. These all contribute positively to implementation time, quality, and
Page 207 of 307
cost of software—all of which are essential for today’s highly dynamic and competitive business
environments (Stavru, 2014). Projects in Case A and Case B fully utilized the advantage of the
greater flexibility offered by the agile development and implementation method using result-
oriented planning. This led the project teams to develop with high accuracy the most important
and complex areas of functionality. This also provided the greatest business value and actively
engaged all stakeholders to help shape the new system. According to informant B06VR, the
technical lead of the COTS vendor in Case B:
Each sprint gives the vendor and the BA’s [Business Analysts] increasing ability to
understand how the business users think about the requirements. Therefore
experience gained from one sprint significantly increases the accuracy of
interpretation of the high level requirements at the detailed design phase and also
the correctness of the functionality delivered. … Each sprint also reduces the Risk
since we developed the most risky component early. …We can re-plan way earlier
and when it’s ok to re-plan; coz agile gives you the ability to react … scale down
the requirements or add more resources. If you do your detail design upfront, it’s
not worth the paper it’s written on because until you go to the development—you
don’t realize the problems with your design.
For Cases A, B and D, an agile implementation approach motivated the project team to split high-
level requirements into a fixed number of sprints during the requirement and planning phase. Each
sprint delivered a certain group of business functionalities for the project. During the sprints,
analysis and conceptual design, design and architecture, and development and delivery phases all
occurred in an iterative fashion.
Page 208 of 307
One of the more salient processes during the development phase, as well as the design and
architecture phase, was change control. Change control consists of a review and approval
committee, a set of guidelines, and escalation points. Strict enforcement of change control helped
Cases A and B avoid several changes that could have added significant expense to the project’s
fixed-cost budget, thus causing the project to exhaust all of its contingency funds.
Delivery
Due to the agile nature of the implementation process for all selected Cases, the delivery signifies
a different meaning compared to the delivery phase of a traditional waterfall model. The objective
of this phase is to ensure a smooth production roll-out of the solution, with a high state of readiness
in the user community. Furthermore, this phase was critical to the effective handover of the new
COTS system between the project team and the operations area(s). In Cases A and B, project teams
ensured that the implementation of the support model occurred prior to the solution deployment
activities related to delivery. For Cases A, B, and C, the project team remained intact for a settle-
in period to assist the user community and operations support teams as required. This was intended
to address any post deployment and go-live incidents with the new COTS product. Whereas the
delivery in a waterfall implementation model has a single time slot, that was not the case for the
selected projects under analysis. As a result, all four projects indicated post-deployment defect
remediation activities during the project’s lifecycle.
The regulatory system implemented under Case B and the financial market system implemented
under Case A both contained a delivery phase at the end of each sprint, when a certain set of
business functions were completely developed (as planned during the requirement and planning
Page 209 of 307
phase). However, the final data migration to the target production environment was performed as
a part of the “go-live” activities of the project. In Case C and Case D, a similar trend was observed,
but the scope of the deliverables was much larger than a typical sprint-based model. This made the
delivery phase for Case C and Case D resemble the delivery phase of a waterfall model of IS
implementation.
Business validation or audit was one of the key activities performed during the delivery phase of
each project. This included, primarily, user acceptance and integration testing. When testing in
delivery phase, there are usually a number of prevalent models that most COTS or IS
implementation projects utilize. Either the vendor tests the system and sends the results to the
client or the testing is done jointly in an integrated fashion by the vendor and client. For Case A,
B, and D, the second option was chosen. For all three projects, integrated testing teams comprised
of the vendor, project’s technical team, and the client, conducted testing, targeting both functional
and non-functional requirements. This approach is usually well suited for agile developments and
has proven to be very effective, including for the present cases. According to informant B03PT, a
senior COTS developer in Case B:
One good thing about this project was testing … there was unit testing that was
done by the vendor but the usability testing was done with business resources.
Solution integrator provided [the]test manager but not the actual testers. … We
said the business needs to give us the test resources for functional, regression and
system integration testing. … So we combined functional testing with the UAT
essentially. … They are testing the system as it going to be for them as opposed to
Page 210 of 307
we are testing it and saying it’s all good and they not accepting it. … This is a new
concept because typically you have testers and then you have business users after.
Close-Out
All four cases analyzed for the current research indicated a concluding set of activities. Activities
of this phase are quite similar across all cases, as “close-out” is not only mandated by the PMO’s
project management guideline but is also a standard best practice recommended by the Project
Management Institute (PMI). The primary objective of this group of tasks is to bring the project to
closure from an operational, administration, financial and human resources perspective. The
project close-out phase provides an opportunity to assess a project's success and results through
team and client satisfaction surveys and identification of lessons learned to be applied to future
projects. While recognition should be provided to individuals and the team throughout the project
life cycle, this is a good time to formally recognize the efforts of team members. Informants from
Cases A, B, and D indicated multiple close-out activities where feedback was collected to improve
next iteration, and recognition mechanisms were used to reward outstanding team performance.
In summary, implementation process-related activities for all four Cases demonstrated the
existence of seven logical groups: initiation, requirement and planning, analysis and conceptual
design, design and architecture, development, delivery, close-out. Although a pre-initiation group
was frequently observed for most of the Cases, this formed a major input for the initiation and
requirements phases of the projects. Furthermore, an agile aspect of the implementation was
supported by the repeating conceptual design, design and architecture, development, and delivery
Page 211 of 307
activities. Whereas a macro-level model (Figure 12) indicates the information flow between logical
project phases, a micro-level or activity-based model of COTS implementation (Figure 13)
captures the level of engagement and contribution by major project stakeholders. Activity analysis
of Cases A, B, C, and D also indicated existence of several implementation-related processes and
tools, such as deep-dive, design authority, RTM, and joint testing team. Projects utilizing these
tools and processes demonstrated a higher control and positive influence on their CSF compared
to those that did not utilize these processes and tools.
6.2 Control configurations and control balancing
From a control configuration perspective, our analysis of Cases A, B, C, and D identified several
distinct control configurations that are directly related to each project’s success or failure.
Furthermore, going above the simple functional purpose to a more abstract level of intended
purpose reveals a dynamic nature of control configurations and their relationship to the
management of CSF or challenges throughout the implementation of each project. Delving deeper
into the processes, tools, and control configurations, and adopting a categorization perspective, we
detected four discrete control orientations for each project: (a) strategic, (b) responsibility, (c)
harmony, and (d) persuasion. A summarized comparison of the four orientations is presented in
Table 33:
Table 33: Comparison of Control Orientations
Control
Orientations
Related Project Phases Control Configuration Objective
Strategic Pre-initiation
Initiation
Alignment of the project with organizational objectives
and strategic management of stakeholders
Responsibility
Requirement and planning
Analysis and Conceptual
Design
Aligning stakeholder’s expectations and project’s
objectives at a logical level
Page 212 of 307
Harmony
Design & Architecture
Development
Align project deliverables to organizational capability and
minimize negative impact on non-project stakeholders,
thus, Aligning stakeholder’s expectations and project’s
objectives at a physical level
Persuasion
Delivery
Close-out
Promoting project objectives related to user’s acceptance
of deliverables and reinforce the achievement of ‘strategic
values’ promised during the gating phase.
Our qualitative analysis of primary data from all four Cases indicates targeted adjustment of
control configurations at different phases of the project. This provides clear indications of control-
balancing decisions by the projects. An examination of Cases A, B, C, and D’s implementation
from a control balancing perspective indicates the influence of three distinct trigger factors: (a)
shared understanding among stakeholder groups, (b) negative anticipation by one party regarding
the future performance of the other, and (c) deviation of expectations or unfulfilled expectations.
These three factors can be further classified into two major categories: re-active control balancing
and pro-active control balancing.
Strategic
Strategic orientation was observed very early in each project’s lifecycle, during the opportunity
exploration and strategic initiative proposal phases of the project, which can also be considered
pre-project phases. At an early stage, engaging stakeholders selectively but optimally appeared to
be a rudimentary force that helped build project capabilities and much needed social capital to
tackle several CSF dimensions and other challenges throughout the project-lifecycle, such as top
management commitments, project champion, organization-wide project visibility, timely and
effective escalation, project management strength, and culture change management. Pre-defined
project implementation templates, the project “gating” process, and project management office
(PMO) oversight facilitated the establishment of a governance committee for each project.
Page 213 of 307
Although processes such as project gating and PMO oversight can be classified as organizational
procedural controls, collectively they enabled establishment of further control mechanisms such
as a memorandum of understanding (MOU). Specifically, in Cases A and B, formulation of a clear
project governance structure and MOU played critical facilitation roles during later stages of the
project, when escalation was necessary to handle a time-constraint situation or force project
partners into requirement compliance through a non-institutional process.
Table 34 Strategic Control Orientation
Control
Orienta-
tions
Project
Phases
Control Balancing Trigger Factors CSF and Challenges Saturated
Code/Consolidated
Code for trigger
Strategic
Pre-
initiation;
Initiation
No signed contract [A], lack of
knowledge about the capability of
selected COTS [A], Industry foot print
of the vendor [A, B]; Absence of
shared history [A, B], Project context
[A, B], Organizational context [A, B],
Absence of previous contract [D],
Absence of formal authority [C],
Existing contract with organization
[C], Negative past experience [D],
Pre-disposed trust condition [D],
Previous cooperative settings [D];
PMO prescribed compliance
requirements [A, B, C, D].
Visioning and
planning,
Building a persuasive
business case,
Project champion,
Client Consultation,
Leadership
commitment,
Culture and process
management
Contextual limitations
[leading to negative
anticipation];
Absence of past
performance and
history [ leading to low
shared understanding];
Need to strategic
resource, and
commitment, support [
Need for shared
understanding
development];
Pre-initiation is often executed as a separate project or preparatory activities for large budget and
complex projects, as was the case for the projects in Case B and Case D. Pre-initiation activities
directly lead to the initiation phase of the project where objectives include: expressing and
understanding the general requirement, type, and goal of the project, size and time horizon
determination, key stakeholders engagement, general resource requirements identification,
organizational impacts anticipation, budget, assumptions, risks, and risk-mitigation strategy
development.
Page 214 of 307
In the absence of explicit understanding of each other’s processes, partner’s capabilities, and
lower-level deliverables, a trust-based control configuration was allowed to dominate relationships
among stakeholders. A trust-based control enhanced dynamics for the project team and facilitated
augmentation of shared understanding among project participants during early phases, which, in
turn, aided due diligence on both the client and vendor sides. For all four Cases, application of
trust-based informal controls reflects the absence of a formal authority by one party over the other
through a binding contract or agreement. Whereas the ability to exert formal control or
administrative controls was often absent at these early stages of the project, the project team’s urge
to establish a common understanding or increase the level of shared expectations with all
stakeholders triggered a change in control configuration from a unidirectional to bi-directional
style. Such change of control configuration was observed in Case D. Alteration of control
configurations are also necessary when a negative anticipation regarding the performance of one
party is prevalent. Negative anticipations in the early life of a project could result from negative
past experiences, an absence of shared work history, pre-disposed trust conditions, project context,
and/or organizational context. These factors often require adopting a tighter and unidirectional or
authoritative control style to build necessary shared understanding and ensure proper synergy
among the stakeholders.
Regardless of the direction of the control configuration change, a common trait related to the
control portfolio shift was to address contextual challenges and strategic alignment of the project
with organizational priorities. Establishing project gates or strategic investment governance
mechanisms as critical control elements help ensure a project’s alignment with organizational
priorities. A four-stage gating control is imposed by the organization and the PMO-influenced
Page 215 of 307
informal and collaborative control choices of the projects. (Refer to Appendix D for a detailed
overview of the organization’s project gating process.) A collaborative and enabling control style,
such as trust-based control, was particularly important in building leadership commitments for
each project. This was indicated by the informant C01PT, a senior project manager in Case C:
Maintaining trust and reputation as central authorities were definitely the
driving forces, of course beside the increasing operational needs .… It [pre-
initiation and initiation] was vital for the success. Little knowledge of each
other’s internal processes [among the clients] or no history [between
clients and vendor], RFP [request for proposal], MOU [memorandum of
understanding] they all helped tremendously, especially getting the right
people at the table and close gaps [in knowledge and understanding].
Gating requirements had to be met to secure the funds but you also measure
the top level commitment here [Pre-initiation and initiation].
Since a project often executes in a dynamic organizational environment, contextual challenges
such as inter-project dependencies and competing organizational priorities, often demand
alteration of a project’s existing control configuration. Several informants in Cases C and D
reported such context-related challenges, where over-reliance on a trust-based control
configuration with the vendor and internal IT support teams resulted in several delays for multiple
deliverables. This eventually caused both Cases C and D to miss a few vital requirements and
project deadlines. Although a collaborative control configuration was initially optimal for all four
Page 216 of 307
projects, the project management teams in Cases A and B recognized the need to improve shared
understanding during the initiation phase of the project. Facing a similar organizational context
and a non-existent shared history with the vendor, projects in Cases C and D decided not to
introduce cognizant behavioural and outcome-based controls to enhance the level of shared
understanding among project participants. Therefore, divergent expectations, reduced sense of
urgency, and reduced comprehension of strategic significance were observed among various
stakeholder groups. This also impacted several strategic aspects of the projects in Cases C and D,
including maintaining executive leadership commitments and support, ensuring a fair share of
scarce resources (i.e., internal support), and process innovation and cultural manipulations to
integrate new business processes.
Responsibility
Responsibility orientation was more salient during the early implementation phases—e.g., project
kick-off, requirement and planning, and analysis and conceptual design—that immediately
followed the strategic pre-initiation phase of each project. A dominant theme emerged through
analysis of the performed activities and adopted processes of the project kick-off, requirement and
planning, and conceptual design phases. From a control perspective, this theme supports the notion
of “aligning expectations at a logical level.” Therefore, tools and processes employed at this stage
supported a more encompassing approach towards stakeholder considerations compared to the
actual engagement level, closely aligning to the normative aspect of stakeholder theory (Donaldson
& Preston, 1995), which suggests that all stakeholder interests have intrinsic value and should be
taken into consideration. Additionally, the control objective for the project teams reflected a need
Page 217 of 307
to map new COTS capabilities to the need of different stakeholder groups and existing business
processes. Such change often needs an introduction of behavioural control through the support
model.
Comparing the responsibility orientation across all four cases, there is a clear divergence between
Cases A and B and Cases C and D in terms of control configurations. In Cases A and B, the project
applied a unidirectional and frequent control on both the vendor and internal IT support teams.
Although the initial control configuration was very relaxed for both projects, conceptualization of
the future state and project team’s concern about a potential cost increase led to the adoption of a
cautious and proactive approach in terms of control. Controlling the vendor in an authoritative
manner helped both projects overcome several critical issues related to the procurement (i.e., RFP)
and off-shore development process. Support for such an authoritative approach is offered by earlier
control researchers like Choudhury and Sabherwal (2003). They have indicated informal controls
to be an inappropriate choice for an outsourced project and pointed out the difficulties in applying
a “clan control,” even for co-located internal teams.
Project teams were also able to exert a certain degree of control over the vendor-provided technical
team and the vendor’s project management practices. According to an informant from the vendor
side of Case B, the COTS technical lead, frequent use of outcome-based control has been
identified:
[Encompassing considerations for stakeholders] doesn’t mean RFP
requirements are invalid; rather it might the broader focus [by the client]
causing the slugging. … RTM [requirement traceability matrix] definitely
helped us solidify the cost estimates a lot, it also helped them [the project
Page 218 of 307
team] in regular progress tracking by simple putting a check mark or IP
[in-progress] beside the requirement.
Timely provisioning of project IT infrastructure components has been an ongoing challenge for
this organization. Because it was experienced and reported by earlier projects within the
organization, both projects in Cases A and Case B considered infrastructure provisioning-related
challenges very early in the project’s lifecycle. Consequently, both projects decided to adjust their
control style for internal IT support teams from an initial trust-based control to a more formal and
authoritative control style consisting of outcome-based and behavioural controls. This was
achieved by establishing a reporting protocol for completed deliverables and the infrastructure
support team following a project-prescribed provisioning sequence. Frequent status tracking for
IT support team’s activities helped ensure an on-time delivery of internal infrastructure
components. On the client side, both projects increased the control frequency. Interestingly,
business clients also monitored and inquired about several project parameters. Additionally,
business expected the project team to follow a prescribed protocol, Technical Proposed Solution
(TPS), to validate functional and non-functional requirements presented through Business
Proposed Solution (BPS). Production and validation of TPS required significant informal
interactions between the client and the project team, as well as with the vendor. However,
production of TPS required the project team to follow a process introduced by the client (i.e., a
form of behavioural control). For large TPS, the client also requested regular progress updates
from the project team. This shift from an informal control relationship to a coordinated or bi-
directional control, comprising of formal and social controls, was driven by the mutual willingness
of the project team and client to quickly reach an agreement in terms of functional requirements
Page 219 of 307
of the future system. The project team identified several gaps in understanding and diverging
opinions during brainstorming sessions and workshops with the business users. Furthermore,
formulation of requirements by the business users showed significant dependency on technical
experts from both the core project team and the vendor. These factors required both the project
team and clients to maintain close coordination in terms of cooperation and communications with
a common goal of formulating valid and consistent requirements for the target system.
In Cases A and B, the control configuration shifted from informal and relaxed to a more
authoritative and coordinated style, but control configurations for Cases C and D were quite
different during corresponding project phases. Both projects under Cases C and D displayed an
informal and relaxed control style throughout. Although an optimal level of shared understanding
between the project team and several other stakeholder groups was not present, no significant
milestones were missed by the project teams. A closer examination of this control attitude in Cases
C and D points to the influence of several contextual factors. These factors were responsible for
masking the presence of several trigger conditions related to control balancing decisions. For
instance, in Case C a new project manager with no significant project management experience was
assigned to the project during the requirement and planning phase. On the one hand, assessing
shared understanding among project participants requires the possession of emotional intelligence.
But on the other hand, emotional intelligence develops with the experience of the project manager.
A lack experience also encourages a project manager to follow a more traditional (i.e.,
authoritative) control style.
Other contextual factors, such as reduced ability to monitor an actor’s behaviour (e.g., the off-
shore development team), lack of technical know-how of the new COTS system, and incorrect
positive assumptions about the validity of previously collected requirements, motivated the project
Page 220 of 307
team in Case D to maintain a trust-based control configuration with the vendor, internal IT support
teams, and the client. Although a trust-based control portfolio, implemented through interactions
and collaborations, dominated both projects under Cases C and D, this choice of control
configuration was not motivated by legitimacy concerns or a desire to enable employees to deal
more effectively with inevitable contingencies (Adler & Borys, 1996). As a result, this enabling
control style failed to achieve its natural effects in Cases C and D. This was reflected in both Cases
by missed requirements and delayed deliverables at a later stage of the project’s lifecycle.
Additionally, informants from both projects indicated a lack of project management strength,
inadequate legacy systems considerations, inability of timely escalation of issues, and
infrastructure-related roadblocks—all of which introduced considerable delays for the project.
Table 35 presents a consolidated view of the control configurations for all four projects,
emphasizing the dominating control themes.
Table 35: Responsibility orientation
Project
Phases
Control Balancing Trigger Factors CSF and Challenges Saturated
Code/Consolidated
Code
Requirement
and planning,
Analysis and
Conceptual
Design
Need to improve shared understanding [A],
Clear understanding on the future state [A];
Agreement on requirements [B], shared
understanding [B], Clear understanding on the
future state [B], missed deadlines [B];
Confirmation of resource availability [B],
Development of shared understanding [B],
protecting price agreements [B], Following
Norm [C], Leadership competencies [C],
Reduced ability to monitor behaviour [D], lack
of technical know how [D], Positive assumption
on the validity of previous requirements [D],
Missed delivery dates [D].
Communication plan,
RFP Execution, Legacy
system considerations,
Project Management
Strength, Offshore
Development Model,
escalation process, IT
Infrastructure
Understanding
current state and
future state [need to
develop common
understanding]
Need to minimize
opportunistic
behaviour
[Negative
Anticipation]
Page 221 of 307
Harmony
As project implementation moved towards design and development activities, each project team
adjusted processes, tools, and control objectives to promote harmony among different stakeholder
groups and other internal processes. In terms of control configuration, the salient theme of this
phase appears to be the objective of aligning stakeholder needs with organizational and project
capabilities from a physical and logical resources perspective, and removing roadblocks for the
core project team.
Table 36: Harmony Stage
Project
Phases
Control Balancing Trigger Factors CSF and
Challenges
Saturated
Code/Consolidated Code
Design and
Architecture,
Development
Confirmation of resource availability [A],
Development of shared understanding [A, B],
Delays in delivery [A, B], Protecting price
agreements[B], Following Norm [C],
Leadership competencies [C], Missed
Milestones [C], delays with deliverables [C],
Lack of clarity for roles and responsibility
[D], High task inter-dependency [D], Absence
of technical expertise [D], Perceived attitude
of the other actor [D]
Low engagement with broader scope
Data conversion
and integration,
Training and job-
redesign,
Change
management,
Dedicated internal
support resource
Deviation of expectations
[Unfulfilled expectations]
Control configuration took a slightly different turn at this stage due to the offshore development
component and an agile delivery approach. All three projects in Cases A, B, and D demonstrated
a bi-directional or mutual control over the vendor compared to a more formal, unidirectional
control observed during earlier phases. Although some aspects of authoritative control were
present in the early harmony phase during design and architecture activities, this configuration
turned into a more coordinated style during the development stage. Reasons for such shift were
related to minimal negative anticipation (i.e., low unfulfilled expectations and higher level of
common understanding) by the project team on the vendor side. Other factors for such a control
Page 222 of 307
shift were directly related to the project and organizational contexts. All three cases had an offshore
component in their development cycles where a portion of the code or COTS-related product
enhancements were performed by the vendor’s technical team. On-site project teams were also
comprised of organizational employees and vendor provided consultants. To maintain the level of
shared understanding and overcome data conversion- and integration-related challenges, a less
authoritative approach was adopted by these projects. This helped maintain team dynamics,
individual motivations, empower team members, encourage frequent open communications, and
improve sense of belonging, as reported by the informants in Cases A and B. The immediate effects
of this approach could be observed in higher team cohesion and, occasionally, the vendor taking
on more responsibilities than anticipated. A combination of formal and social controls to maintain
shared understating is indicated by informant A10IT, a solution architect in Case A:
Discrepancies discovered [requirements provided by project partners] here [on
this phase] was a matter of worry [deviation of expectation]. Design authority did
a good job in resolving them. Daily status meetings and tracking meeting minutes
also helped accelerate the inputs [i.e., feedback from partners]. … [L]ots of things
were happening on the other side [COTS vendor in Europe], and we certainly
didn’t want to find any roadblocks at the last minute [i.e., negative expectations].
The dominant control configuration trend between all four projects and IT support teams appeared
more authoritative. Although an acceptable level of shared understanding was present between the
IT support team and the projects, past experience and multiple on-going projects competing for
the same resources compelled the project team to consider the possibilities of resource
Page 223 of 307
unavailability for their project. Dictated by the overarching goal in the harmony phase, at this point
unidirectional controls were applied on IT support teams to enforce desired behaviour. This
approach differentiated the harmony phase from the authoritative approach of the responsibility
phase as certain behavioural controls, targeted toward internal IT teams, helped develop norms for
the project. This eventually contributed to a high level of shared expectations and understanding
between the project teams and support teams. In Case C, the control relationship between the
project team and IT support teams shifted significantly from a trust orientation to an authoritative
style at the end of the harmony phase. This change in control configuration was influenced by the
unavailability of internal resources for the project, and so some of the critical scheduling
components for the project were not delivered as expected (i.e., unfulfilled expectations and
deviations of expectations). Although a deviation of expectations can certainly trigger a change of
control configuration, for Case C it was unable to achieve the desired effect of establishing mutual
understanding or expectation for future deliverables. This was primarily due to a late detection of
issues where the project was approaching the end of its lifecycle and not enough time was left for
the project to build the required shared understanding by switching to an authoritative control
configuration. Additionally, the project in Case C faced significant knowledge integration
challenges during the delivery and post-deployment stages of the project, as the project expected
the COTS support team to attend all knowledge transfer and technical sessions. However, the
COTS support team was occupied with other responsibilities and the knowledge transfer lagged
behind. Additionally, a mix of behaviour- and outcome-based control also helped reduce
uncertainties and negative anticipations related to the vendor’s offshore development teams. Both
outcome- and behaviour-based controls were applied to internal support teams due to some
unfulfilled expectations experienced by the projects. This significantly reduced the level of
Page 224 of 307
confidence of the project teams in obtaining internal support in a timely fashion, and therefore
increased negative anticipation on other party’s performance.
The control relationship between the project and the business units appeared much relaxed
compared to the responsibility-oriented control portfolio. A heavy reliance on interpersonal
processes by the project team and use of active communication channels between the project and
the client were prevalent in all four Cases. Although a similar trend of interactions between the
project and business units was observed during the strategic orientation of the projects, outcome
for this chosen control style was very different as the trust-based control provided a means for
integrating tacit knowledge during development, and placed little emphasis on codification. Such
informal control configuration also reflects the high level of shared understanding between the
project team and the client, resulting from a more-collaborative exchange of information.
Effectiveness of such enabling controls, with optimal shared understanding, can result in an
efficient exchange and cooperative relationship, thus minimizing both transaction costs and
required control cost between client and the project team. This was pointed out by informant
B12IT, an infrastructure support specialist in Case B:
Smoother sprints were possible here [in the development phase] only
because we engaged them [internal support teams, change management]
when we should have. MOU also helped get things moving on our end
[project level] as waiting for one email reply [decision /approval from
client] might slow down the entire development process. … This called for
much tighter co-operation between us [host] and them [solution integrator
and vendor].
Page 225 of 307
Persuasion
Persuasion, observed during the delivery and close-out phases, appears as the fourth and the final
salient category of control orientation for all four selected cases, when control configurations took
an instrumental turn (Donaldson & Preston, 1995). This suggests an application of control
configuration to promote project objectives, including facilitating the user’s acceptance of
deliverables, negotiating future enhancements and bug fixes, finalizing support agreements,
dispute resolutions and alternative dispute resolutions with vendors, and promoting process and
culture change within the organization. As it was critical for each project to reinforce the
achievement of strategic values promised during the phase, managing and validating business
expectations and establishing fulfilment of those expectations were the central theme for all
activities at this point. This change in the project team’s intention was reflected in a change of the
control configuration of each project.
Table 37: Control Configuration at Persuasion
Orientations Project
Phases
Control Balancing Trigger
Factors
CSF and Challenges Saturated
Code/Consolidated
Code
Page 226 of 307
Persuasion
Delivery,
Close-out
Improved understanding,
Fear of unfulfilled commitments;
Missing information
Disputes with vendor,
Disagreements, Competing
internal resources;
Following Norm
Managing culture
change,
Post-implementation
evaluation,
Troubleshooting and
crisis management
Expectation fulfillment
[unfulfillment];
Reduced [or increased]
negative anticipation;
Improved shared
understanding [or gaps
in shared
understanding];
Control configuration embraced a different but interesting look at this stage compared to earlier
stages, primarily due to the project team’s attempt to establish the values of delivered releases,
including the final customized COTS, to a diverse group of internal and external stakeholders over
a short timespan.
Control configuration between the project and vendor in Cases A and B versus Cases C and D
were at two opposing ends of the control spectrum. Cases A and B adopted a more informal, trust-
based control, whereas projects in Cases C and D embraced an authoritative or formal control
relationship in respect to vendors. This discrepancy between control choices was an apparent
manifestation of challenges confronted by each project during the delivery and close-out activities.
As the delivery activities largely involve the end-user experience for the new COTS and its
technical suitability from an organizational infrastructure and integration perspective, control
configurations targeting tactical versus strategic aspects may have significantly different outcomes
for the project. At this stage, both projects under Cases A and B shifted their control from
coordinated to a trust-based configuration. This was possibly due to the high level of shared
understanding between the project team and the vendor. The ease of integration of the new COTS
system with the organization’s infrastructure and delivery of most of the functional and non-
functional requirements according to the RTM also supported the notion of fulfilled expectations
Page 227 of 307
on both the project team and the vendor side. Furthermore, for both Cases A and B, COTS vendors
had an active and sustained on-site presence to facilitate client testing of the deliverables, thus
boosting the project team’s confidence for anticipated vendor support. A positive outlook for all
three control-balancing trigger factors persuaded the project team to adopt an enabling control
style to support the maintenance of shared understanding.
A completely opposite depiction was observed in Cases C and D, where the control between the
project and vendor shifted from a trust-based to an authoritative configuration. Whereas some
authoritative aspects of control were observed for all four cases at this stage, this was primarily
due to the efforts required to perform an administrative closure for the projects where the project
team conducted a contract audit for the vendors. Aside from executing an administrative closure,
informants from Cases C and D reported several instances of missed requirements, integration
conflicts, and deviation of design. These aspects led to a higher level of unfulfilled expectations,
negative anticipation regarding future performance of vendor, and reduced shared understanding
on different dimensions of the finished product. Consequently, both projects in Cases C and D
adopted a formal control style. Similarly, an authoritative approach, consisting of a status review,
deliverable inspections, and escalations, was employed with respect to the internal IT support
teams as both projects experienced delays related their technical environments, such as network
connectivity, implementing firewall rules, storage area network (SAN) provisioning, and
vulnerability assessment (VAS) by the IT security.
Project control configurations with respect to the business units also differed among the Cases at
this stage. Although Cases A and B demonstrated a high level of shared understanding between
the project and the client, a coordinated control was applied to strengthen the existing shared
understanding and affirm expectation fulfilment. An unswerving impact of such approach was
Page 228 of 307
observed from reduced resistance towards new business processes introduced by the redesigned
COTS product. According to informant B09BU, business lead from Case B:
Joint parallel testing did a lot of good for us [Tri-agency partners]. Not
only it helped us acknowledge the fulfillment of each other’s requirements,
it also helped save a lot of time by brining everyone on the same page [i.e.,
shared understanding].
On the contrary, an informal or trust-based control style between the project team and business
users resulted in significant negative implications for projects in Cases C and D. These included
multiple complaints from business users regarding redesigned business processes, incorrect
implementation of expected requirements, technical defects resulting in IT incidents, and post-go-
live product enhancement proposals. Although an early detection of unfulfilled expectations can
be mitigated through a mix of formal behavioural and outcome-based controls by improving shared
understanding among stakeholders, both projects in Cases C and D detected these control-
balancing indicators much later in the project’s life cycle, thus shifting their control configurations
at a later stage during user-acceptance testing. This significantly reduced the reaction time and
time required to re-establish an effective shared understanding between the project and the
business.
In summary, current analysis of Cases A, B, C, and D indicate the existence of several distinct
control configurations that are directly related to each project’s success or failure. We detected
four discrete control orientations for each project: (a) strategic, (b) responsibility, (c) harmony,
Page 229 of 307
and (d) persuasion. The control configuration objective of the strategic orientation strived to align
the project with organizational objectives and manage stakeholders from a strategic standpoint.
Responsibility orientation of a control configuration, observed during the requirement and
planning and the analysis and conceptual design phases, aimed to align stakeholder’s expectations
and project’s objectives at a logical level. The third salient control orientation, harmony, was
observed during the design and architecture and the development phases. Primary objectives of
this control orientation were to align project deliverables to organizational capability and minimize
the negative impact on non-project stakeholders, thus aligning stakeholder expectations and
project objectives at a physical level. The final category of the identified control orientations,
persuasion, was observed during the delivery and the close-out phases of the projects.
Characteristics of such control orientation attempted to promote the project’s objectives related to
the user’s acceptance of deliverables, and to reinforce the achievement of “strategic values”
promised during each project’s gating phase.
6.3 Stakeholder orientations
Our analysis of Cases A, B, C, and D from a stakeholder-orientation perspective identified several
distinct stakeholder sensitivity engagement combinations directly related to each project’s
performance, as well as post-deployment stability. However, moving beyond the more prevalent
descriptive approach of stakeholder theory to its instrumental and normative core reveals a
different depiction of project and organizational intent related to stakeholder engagement
(Donaldson & Preston, 1995). Additionally, a rather stimulating set of relationships between
stakeholder orientation and management of CSF or challenges emerged from our micro-level
Page 230 of 307
analysis of the implementation processes. Delving deeper into each project’s phase-specific
stakeholder orientation, by adopting a process and tools perspective, we detected four distinct
stakeholder orientations for each project: (a) responsibility, (b) paternalism, (c) neoclassical, and
(d) strategic. A high-level summary of the four orientations is presented in Table 38.
Table 38: Comparison of Stakeholder Orientations
Stakeholder
Orientation
Project
Phases
Engagement
Level
Sensitivity
Level
Project Objective of Orientation
Responsibility
Initiation;
Requirement
and Planning
High
High
(Normative Orientation)
Encompassing involvement and
consideration of all project stakeholders
Paternalism
Conceptual
Design
Design &
Architecture
Low to
Moderate
Moderate to
High
(Normative Orientation)
Encompassing consideration with reduced
involvement of project stakeholders
Neoclassical
Development
Low to
Moderate
Low to
Moderate
(Instrumental Orientation)
Reduced engagement and sensitivity to
protect project’s financial and scope
baselines
Strategic
Pre-initiation,
Initiation
Close-out
Moderate to
High
Low to
Moderate
(Instrumental Orientation)
Encompassing engagement and reduced
influence of stakeholders to promote
project’s strategic goals
Our qualitative analysis of primary data from all four Cases indicates the adoption of different
stakeholder orientations at different phases of the project. An examination of this aspect of Cases
A, B, C, and D’s implementations indicate the influence of four distinct motivational factors: (a)
performing due diligence for internal and external stakeholders, (b) aligning capabilities and
expectations, (c) minimizing cost, and (d) upholding organizational mandate. In addition to the
Table 38, showing a summary comparison of four stakeholder orientations, Table 39 below
presents the phase-specific indicators for each orientation category.
Page 231 of 307
Table 39: Phase specific orientation analysis and code derivation
Implementation
phases
Processes, Procedures
Tools, Policies and
Roles
Selected Codes/Indicators Saturated
Code / Core
Categories
Pre-Initiation Business Gating,
Medium term plans
Alignment with Medium term plan, Protecting
Reputation and reduce reputational risk, Building
trust, Reduce operational Risk, Increase
organizational effectiveness, Compliance with
Basel III and governmental requirements
Strategic
Initiation MOU, RFI/RFP,
Procurement, RTM,
External
Communication,
Demonstration, Vendor
management, Contract
negotiations, reference
Checking, escalation
channels
Protecting Canada’s economy, Protecting
financial institutions, Ensuring financial
wellbeing of the people of the country;
Government’s ability to ensure financial stability
Responsibility
Requirement
and Planning
DeepDive-I,
Prototyping,
RTM
Requirements clarification in terms of vendor
proposed solutions; Consider all parties the
system will directly impact; capture all concerns
and requirements
Responsibility
Analysis and
Conceptual
Design
Non-functional
Requirements,
DeepDive-II,
Mapping the requirements to the functionalities
of the new system; Determining the technical
capabilities and what they should be, Propose
refinements to original requirements captured
Paternalism
Design and
Architecture
Design Authority
process, Capacity
planning, Balanced
representation
Controlling the actual design process through a
design authority group and a design approval
process, Selected representation from business
and IT; Proposing capacity extensions not
captured by RTM
Paternalism
Development Change Control,
Escalation management,
Scope Control, product
enhancement
Monitor scope Creep and scope seep leading to a
change, Fixed cost off-shore development,
identify product enhancements to help deliver
RTM in full
Neoclassical
Delivery &
Transition
Integrated Testing,
Implementation co-
ordinator,
Joint test team to enhance client/partner’s buy-in
thus enhancing project deliver success
Strategic
Page 232 of 307
probability, maintaining internal integrity and
control structures
Close-out Organizational policy
integration, Knowledge
management,
Recognition, Enhancing
PMO Capabilities
Sense of achievement and appreciation leading to
employee dedication and partner relationships;
external communication for meeting deadline of
preannounced dates with commercial partners
Strategic
Based on the evidence from all four Cases, our proposed framework for stakeholder orientation
can be further enhanced to capture optimal engagement-sensitivity combinations for each phase of
an IS implementation project. Supported by empirical evidence and observed positive impacts of
each combination, each quadrant of the model in Figure 15 is further subdivided into two segments
to represent low-to-moderate and moderate-to-high aspects of engagement and sensitivity.
Figure 15: Phase-specific stakeholder orientation.
Page 233 of 307
Furthermore, the project phases are plotted to identify the most effective stakeholder orientation
for that phase. Supporting evidence for Figure 15 and the effectiveness of particular phase-specific
stakeholder orientations are discussed below in Sections 6.3.1 through 6.3.4.
6.3.1 Responsibility dimension (normative orientation)
The sub-segments labeled A and B in the first quadrant comprise the responsibility dimension of
the project’s stakeholder orientation, where the project maintains a strong engagement level as
well as a strong sensitivity level towards the stakeholders, which can be seen as a manifestation of
corporate responsibility. High sensitivity and engagement also allow the organization to place
stakeholder interests ahead of the organization’s own interests, reflecting the foundational core of
stakeholder theory (Evan & Freeman, 2004). Recognition by the organization that all stakeholder
interests have intrinsic worth is the primary driver for this orientation. Strong sensitivity may at
times appear to be valued above organizational interest, which needs adjustment through
stakeholder engagement to determine the optimal level.
For all four selected Cases, the initiation and requirement and planning phases demonstrated a
strong responsibility orientation, where processes and tools such as memorandum of understanding
(MOU), request for information/request for proposal (RFI/RFP), procurement, requirement
traceability matrix (RTM), external communications, vendor management, contract negotiations
(fixed versus variable rate), the buy-versus-build discussion, establishing escalation channels,
DeepDive™, prototyping, and RTM were all used. Although the tools and processes utilized by
Page 234 of 307
each project indicated this common theme of responsibility toward stakeholders, each project
demonstrated a differing level of engagement sensitivity at this stage. This discrepancy among the
projects resulted from the project’s context and team composition. For example, projects in Cases
A and B involved a completely new COTS product without any prior relationship with a vendor.
In contrast, Case C benefited from an existing vendor relationship and Case D was a second
attempt for the same initiative. Regardless, a transparent and comprehensive approach to identify
and communicate with those who have legitimate stakes in the business initiative was
demonstrated in each project. This closely aligns with the argument of moral and political
philosopher Jürgen Habermas, who argued that the moral legitimacy of stakeholder engagement
can only be ensured through an engagement process that is largely or entirely free of any strategic
motivation. Although such a high level of engagement, often driven by morality, required
considerable financial resources and time. A further manifestation of this moral dimension was
established through a high level of sensitivity by each project, which clearly opposed any ulterior
motives or deceitful manner. Such honest, open, and fair engagement with stakeholders led to a
stakeholder deliberation process (Legacy, 2010) in Cases A and B. As the deliberation process is
only achievable through a face-to-face dialogue between actors (Amin & Cohendet, 2004, p. 89;
Gertler, 2003, p. 86), substantial reliance on brainstorming sessions, focus groups, and facilitated
workshops with extended stakeholder groups helped alleviate several challenges, which was
consistent with the predictions by ethical
strategists (Noland & Phillips, 2010).
Although Cases C and D attempted to engage stakeholders through an inclusive approach (Legacy,
2010) to the planning process, they failed to achieve stakeholder deliberation. Consequences for
Page 235 of 307
such an inclusive approach without the necessary deliberation aspect are reflected through missed
system requirements, lack of organizational awareness of the new COTS and introduced processes,
resource unavailability from various support teams, and costly change requests from vendors who
were not included in the project scope.
A comparison and contrast in terms of stakeholder engagement and sensitivity level among all four
Cases is presented on Table 40.
Table 40: Responsibility Dimension
Cases Project
Phases
Engagement considerations and
sensitivity controls
Engageme
nt Level
Sensitivity
Level
CSF and Challenges
Case A
Initiation;
Requirement
and Analysis
A very broad set of stakeholders
were considered and a large portion
of them were engaged by the project
team to ensure proper
representation; Organizational
reputation and reputation of the
government was a driving force;
Although a large group of
stakeholders were considered,
organization maintained focus on
business objectives
High High Implementation strategy and
timeframe; Requirement
completeness; Project
management strength; Project
cost planning and management;
Balanced team; Optimal COTS
selection; Execution of RFP;
Team morale and motivation
Case B High High
Case C Moderate Moderate
Case D Low Moderate
6.3.2 Paternalism dimension (normative orientation)
The sub-segments labeled C and D of the second quadrant comprise the paternalism dimension of
the organization’s stakeholder orientation, which is manifested through the project’s strong
sensitivity and low-to-moderate engagement of stakeholders. This can be interpreted as the
organization working in the best interests of stakeholders (moderate-to-high sensitivity) without
necessarily engaging them (i.e., a paternalistic management approach). This is supported by the
information orientation versus communication orientation aspects found in general stakeholder
Page 236 of 307
theory (Deetz, 1995), indicating deliberate choices that managers can make (Eskerod, Huemann,
& Savage, 2015).
Although the organization’s ability to maintain stakeholder sensitivity without engaging them may
be questionable, this orientation still appears to be well aligned with the normative aspect of
stakeholder theory, resembling the traditional version of corporate social responsibility.
Furthermore, stakeholder agency theory (Hill & Jones, 1992), a modification of agency theory,
attempts to explain the suppression of strategic behaviour of the firm through incentive alignment
mechanisms and various structural forms that police the implicit and explicit contracts between
managers and stakeholders. Moving to a micro level, each project’s stakeholders are a part of the
nexus of implicit or explicit contract with the project team, where the project management team
assumes a central position in the nexus of such contracts. Such conceptualization also displays a
striking resemblance to agency theory, where principals appoint or hire agents to carry out
activities on their behalf. Additionally, both stakeholder-agent and principal-agent relationships
involve an implicit or explicit contract where reconciling divergent interests is the primary goal.
Regardless of such uncanny resemblances, project management team are never appointed by the
collective body of stakeholders, which sets the stakeholder-agent perspective apart from the
principal-agent relationship. Resorting to this theoretical lens of stakeholder agency, a project’s
paternalism orientation can be argued as being in alignment with the normative core of stakeholder
theory.
For Cases A, B, C, and D, the conceptual design and planning and design and architecture phases
demonstrated a strong paternalism orientation, as processes and tools like DeepDive™, design
Page 237 of 307
authority, capacity planning, balanced representation, and RTM dominated the implementation
flow.
Aspects of paternalism were demonstrated through various processes. One high visibility process
was design authority. Comprising of two key elements—design authority group and design
approval cycle—the design authority process was introduced early in the life cycle of the project.
In addition to minimizing discrepancies between client and vendor expectations, instrumental use
of this process by the client determined what gets implemented and how it gets implemented.
Table 41: Paternalism Dimension
Cases Project
Phases
Engagement considerations
and sensitivity controls
Engagement
Level
Sensitivity
Level
CSF and Challenges
Case A Conceptual
design &
planning;
Design &
architecture
Moderate stakeholder sensitivity
with a strictly controlled
engagement process guided by
the project team to maintain
focus; Design authority process
helped to materialize this
engagement process and
sensitivity control.
Moderate Moderate
Requirement Risk;
Requirement consistency;
Requirements stability;
Roles and responsibility
conflicts;
Case B Moderate Moderate
Case C Moderate Low
Case D Low Low
6.3.3 Neoclassical dimension (instrumental orientation)
The sub-segments labeled E and F of the third quadrant comprise the neoclassical dimension of an
organization’s stakeholder orientation, which is demonstrated through the project’s low sensitivity
and low-level engagement of the stakeholders. Maintaining a high stakeholder engagement and
sensitivity does not come without a cost in an IS implementation project, as valuable project time
and resources are required to address concerns that could significantly reduce the project’s
effectiveness, as well negatively impact the scope of the deliverables. To avoid disastrous
consequences, projects may adopt an neoclassical orientation, which is an economically based
Page 238 of 307
view where the firm treats the stakeholders as instrumental, minimizing any interest in building
stakeholder relationships. Although such an economic efficiency perspective aligns well with
Friedman’s theorem (Friedman, 1970) that the only social responsibility of the firm is to increase
its profits, Friedman’s assertions have been critiqued by those who argue business ethics should
not be a merely self-interested position (Chryssides & Kaler, 1993, pp. 231–33; Weiss, 1994, pp.
76–77; Mintzberg, 1995, pp. 205, 214–15; Hoffman, 2002, pp. 716, 718–19). However, to put
Friedman’s view on business ethics into perspective and minimize such diverging views, Wagner-
Tsukamoto (2005) introduced an economic revision of Friedman’s theorem by introducing
concepts of concepts of ethical capital and active moral agency (Wagner-Tsukamoto, 2007). In
light of Friedman’s theorem, Goodpaster (1991, p. 59) argues that an instrumental stakeholder
management is limited to fiduciary responsibility, supplemented by legal compliance, a similar
argument can be applied in a project context where the organization and PMO impose the
compliance requirements for the project. Regardless, Wagner-Tsukamoto’s (Wagner-Tsukamoto,
2007) introduction of ethical capital infuses an instrumental or neoclassical stakeholder
management approach with ethical concerns that go beyond a mere compliance approach. This
allows integration between the project’s stakeholder orientation, driven by performance or
efficiency gains and predominantly in a utilitarian, consequentialist tradition, with moral
judgment. Such ethical capital, although primarily residing at the organizational level, enables
projects to adopt a neoclassical approach of stakeholder orientation, where either cost savings or
cost overruns are incurred as a result of a project’s actions and are ultimately passed on to the
organization.
Page 239 of 307
All four projects analyzed under Cases A, B, C, and D demonstrated a neoclassical orientation
during the development phase of the project, when there were very few indications of building
relationships among stakeholders, or engaging the stakeholder in an encompassing manner.
Although Cases A, B, and C contained an offshore development aspect, the development phase
was the biggest work stream for all four, and so substantial in terms of project budget. A low
stakeholder engagement at this point not only helped projects attain much-needed cost efficiency
gains but also helped avoid design disruptions, often observed in high stakeholder engagement
situations. Therefore, besides the obvious financial gain or cost-efficiency motive of a project,
organizational ethical capital enabled projects to act ethically yet maintain low engagement and
sensitivity during the development phase. Such an economic interpretation of a project’s
stakeholder orientation is essential to justify an instrumental application of stakeholder theory
without sacrificing entirely the normative core of the theory. Both Cases A and B demonstrated a
very low engagement and sensitivity combination during the development cycle, which helped
manage several CSF and challenges, such as balancing out the early cost overruns, scope creep
resulting from unchecked customer requirements, and high utilization of project resources, thus
minimizing the negative effects of shared resources. This also supported a vital success factor for
an agile development process that involved empowerment of the project team in the decision-
making process.
Effective adoption of a neoclassical orientation needs to be supported through appropriate tools
and processes as ethical capital allows the project team to choose alternatives that might translate
into higher expenditures for the organization in the long run. Projects in Cases A, B, and C decided
to implement a COTS solution through an agile development approach and engage an offshore
vendor along with an experienced solution integrator, all reflecting the organization’s objective of
Page 240 of 307
economic efficiency and risk aversion. Although all four projects experienced both anticipated and
unanticipated changes during the development phase, effective use of MOU, project governance
structure, pre-established escalation channels, and change-control processes supported the
project’s neoclassical approach by keeping stakeholder engagement at a very low level.
The change-control process itself played a critical role for RRS implementation in Case B, as the
project was based on a fixed-price contract, and any broken links between user stories, statement
of work, and RTM requirements would have caused additional changes and likely increased the
overall project delivery cost for the organization. Similarly, in Case A and Case D, the escalation-
management process had an economic orientation with low stakeholder engagement. This process
appeared vital for the project’s implementation due to the nature of the host organization, which
had a strict IT environment and access-separation policies guided by the security policy of the
organization. Deploying any new or modified code and migrating data from one environment to
another was dependent on the availability of internal COTS support staff. A few times, escalations
were necessary to reduce standby costs of the project team. Regardless of a similar organizational
context for all four projects, Cases C and D portray a slightly different stakeholder orientation in
development phase compared to Cases A and B, where the former pair of projects maintained a
moderate level of stakeholder engagement. This can be interpreted as a ripple effect of maintaining
low stakeholder engagement and sensitivity during responsibility orientation. Both Cases C and D
sought business user and vendor consultation for missing requirements, requirement conflicts, and
design discrepancies during this phase, which compelled the projects not only to engage a larger
group of stakeholders, but also change initial design decisions (i.e., higher sensitivity).
Despite a justification for the necessity and incorporation of constructs such as ethical capital, a
neoclassical approach is still open to debate and criticism. The foundational principle for this group
Page 241 of 307
of critiques is supported by a belief that self-interested behaviour should not be viewed as ethical
behaviour and good must not be done for reasons of profit. Although active moral agency, coupled
with ethical capital, builds a convincing counter argument, this is further strengthened by Jones
and Wicks’ (1999) proposed “enacted environment” construct. The neoclassical orientation of a
project may cross the normative boundary of stakeholder theory, but Jones and Wicks’ (1999)
“convergent stakeholder theory” argues that an endangered management’s relationship with a firm
can be avoided through implementation of an enacted environment that allows corporate managers
to behave morally in a stakeholder context.
Table 42: Neoclassical Dimension
Cases Project
Phases
Engagement considerations
and sensitivity controls
Engagement
Level
Sensitivity
Level
CSF and Challenges
Case A
Development
Maintains a low engagement
by a small group of
stakeholders; often due to
economic reasons. Sensitivity
control was strictly enforced
with an intent to avoid
significant scope impact
resulting from changes
introduced during
development
Low Low Project cost performance;
Development of internal
technical capabilities;
Integration and internal
process considerations;
Empowering development
team;
Case B Low Low
Case C Moderate Moderate
Case D Moderate Low
6.3.4 Strategic dimension (instrumental orientation)
The sub-segments labeled G and H of the fourth quadrant comprise the strategic dimension of an
organization’s stakeholder orientation, where the project maintains a moderate-to-high
engagement level, coupled with a moderate-to-low sensitivity level, placing the project’s strategic
goals above stakeholder interests. As stakeholders are used for furthering the goals of the project
Page 242 of 307
under a strategic orientation, an instrumental approach to stakeholder theory is a more salient
explanation compared to a normative or descriptive one.
Since Freeman’s (1984) ground-breaking contribution to stakeholder theory, a heightened interest
in the concept of the stakeholder has propelled it to a central place in strategic management
discourse (see, e.g., Mitchell, Agle, & Wood, 1997; Freeman, Harrison, & Wicks, 2007; Freeman,
Harrison, Wicks, Parmar, & De Colle, 2010). The rationale articulated by Freeman emphasized
the significance of support generated by various stakeholder groups and thereby enhanced an
organization’s chances for survival (Freeman, 1984). As Turner (1990) suggested, a project is a
“vehicle (or agency) for organizing resources,” and a conceptualized project is “an agency
established by a parent organization (the principal) to achieve specific objectives” (p. 3). This
depiction of a project as a “temporary organization” (Turner, 2003) allows it to be examined
through several organizational-level theoretical perspectives. Therefore, Freeman’s (1984)
argument on stakeholder support for organizational survival is equally applicable. This view is
further augmented if infused with resource dependency theory (Pfeffer & Salancik, 1978) to argue
that a project is dependent on the resources offered by the stakeholders to fulfill its mission and
aims. So, stakeholder engagements are of strategic significance for a project due to the fact that a
project needs contributions from them that are both financial and non-financial in nature (Morris
& Hough, 1987; Aarseth, Rolstadås, & Andersen, 2011). This view is also supported by others
who hold that strategic engagement is considered to a means of establishing social networks
between organizations and their stakeholders, thus, generating to useful resources (e.g., Kim,
Brunner, & Fitch Hauser, 2005, 2006; Swanson, 2013).
Page 243 of 307
For all four cases, the pre-initiation, close-out, and delivery phases demonstrated a strong strategic
orientation, where processes, tools, and roles (e.g., business-gating process, integrated testing,
implementation coordinating, business walk-through of deliverables, establishing perceptual
congruence among stakeholders, administrative project closure, collaborative knowledge
management) were used.
The purpose of the gating process is to review the appropriateness of an IS implementation
initiative and ensure that there is an understanding of what is to be done and why, making certain
that the investment has a good direction and is aligned with an organization’s strategic objectives.
A common theme for Cases A, B, C, and D indicated: (a) reducing operational and reputational
risk; (b) increasing corporate efficiency; and (c) increasing corporate effectiveness, as some of the
key benefits that would be realized upon completion of the project. From a project perspective, the
business opportunity gating process enabled the project to clarify the value proposition and, more
importantly, secure key stakeholders’ commitments for the initiative that were vital for the
project’s survival and success.
Strategic engagement of stakeholders was also observed during the delivery and closing phases of
the projects. For Cases A, B, and D, an integrated testing approach utilized during the project’s
delivery phase helped obtain a business buy-in for the delivered solution by affirming the client’s
expectations and increasing their confidence. The role of implementation coordinator observed in
Case B, specifically during the project delivery and close-out phases, enabled the project to
synchronize with existing organization practices of fixed-window-based change management,
configuration and asset management, SLA-based incident management supporting IT resiliency,
and disaster-recovery management to ensure business continuity by engaging appropriate internal
and external stakeholder. The combined effects of these selective engagements can be tied to
Page 244 of 307
service level and IT resiliency enrichment, which are closely aligned with both the project’s and
the organization’s strategic objectives.
Absence of such strategic stakeholder orientation could have a direct and dire consequence for the
project. For example, in Cases C and D, both projects experienced several impediments from
internal change management and IT security teams during the delivery phases, which resulted in
unexpected delays and escalations for the project. Additionally, a low perceptual congruence
between the client and project team introduced unexpected debates and challenges concerning the
deliverable, which resulted in bloated backlog of future enhancements for the COTS solution. Such
detrimental consequences have already been predicted by resource dependency theory through
concepts such as stakeholder multiplicity and stakeholder inclusiveness. This is commonly
manifested in situations with a low degree of stakeholder inclusiveness, where non-considered
stakeholders may still be in a position to harm the project through a multiplicity effect or potential
negative influence of one group of stakeholders over the other (Eskerod et al., 2015).
Since the sensitivity level is moderate to low, and the engagement level in this quadrant is high, it
is possible that an organization might pursue its strategic objectives through a purely instrumental
behaviour at the expense of a normative aspect of the stakeholder theory. Regardless, a moderate-
to-high sensitivity for this orientation often indicates significant future commitments for projects.
Table 43: Strategic Dimension
Cases Project
Phases
Engagement considerations and
sensitivity controls
Engageme
nt Level
Sensitivity
Level
CSF and Challenges
Case A
Pre-Initiation;
A moderate sensitivity towards
stakeholders and high engagement of
internal stakeholders for strategic
High Low Visioning and planning;
Project champion; Project
Management Strength; Data Case B High Low
Page 245 of 307
Case C Delivery;
Close-out
reason; A balance between the
sensitivity and engagement is
maintained but an opportunity to act in
bad faith did exist.
Additionally, getting insight from
current implementation was essential
as it would be used for future strategic
decisions. Drive was to get a
comprehensive feedback from a small
group of legitimate stakeholders
Moderate Moderate conversion and integrity;
System integration and
acceptance testing; Training
and job redesign; Post-
implementation evaluation;
Troubleshooting /crises
management; Applying
lessons-learned for each
iteration
Case D Low Moderate
In summary, current analysis of Cases A, B, C, and D from a stakeholder-orientation perspective
identified several distinct stakeholder sensitivity-engagement combinations. Each of these
stakeholder orientations showed an integral relationship with each project’s performance. The
presence of these orientations during specific project phases generated a positive impact on various
CSF for Case A and Case B. On the contrary, an absence of these stakeholder orientations during
the same project phases generated a negative impact for Case C and Case D. Therefore, a rather
stimulating set of relationships between stakeholder orientation and management of CSF or
challenges emerged from a micro-level analysis of the implementation processes. Four distinct
stakeholder orientations were identified for each project: (a) responsibility, (b) paternalism, (c)
neoclassical, and (d) strategic.
Responsibility orientation was observed during early project phases during initiation and
requirements and planning. With high stakeholder engagement and high stakeholder sensitivity,
responsibility orientation aimed to ensure an encompassing involvement and consideration of all
project stakeholders. Paternalism orientation was observed immediately following the requirement
analysis phases of the projects, during the conceptual design and design and architecture phases.
With a low to moderate engagement level and a moderate to high sensitivity level, paternalism
Page 246 of 307
aimed to ensure an encompassing consideration of stakeholder needs with reduced involvement of
project stakeholders. Neoclassical orientation was observed during each project’s execution phase
when deliverables were produced or software modules were developed by the core project team.
With a low to moderate engagement level and a low to moderate sensitivity level, a neoclassical
orientation tried to reduced stakeholder engagement and sensitivity to protect the project’s
financial baseline and scope baselines. The fourth and final stakeholder orientation observed
during the very early lifecycle for each project was classified as “strategic orientation.” Mostly
observed during pre-initiation, initiation, and the close-out phase, a strategic stakeholder
orientation sought to ensure encompassing stakeholder engagement and reduced influence of
stakeholders to promote a project’s strategic goals.
6.4 Processes, control balancing, stakeholder orientation and CSF
In this section, we integrate theory and research by combining the processes, control
configurations, and stakeholder orientations with CSF, along with theoretical support for such
integration. Before moving on to a discussion of how the elements relate to each other and to
facilitate the theoretical argument for our model, we present phase-specific control configurations,
stakeholder orientations, and CSF in Table 44.
Table 44: Control Configurations, Stakeholder Orientations, and CSF
Project Phases Effective Control
Configuration
and Intent
Effective
Stakeholder orientation
Related CSF/CFF (Combined)
Pre-Initiation
Strategic (Trust based control to
support ‘good faith’
engagement
Strategic (High
Engagement and Moderate
to Low Sensitivity)
Visioning and planning,
Project champion,
Project Management Strength
Page 247 of 307
Initiation Strategic (Authoritative
Control) Responsibility
(High Engagement and
High Sensitivity)
Balanced team;
Optimal COTS selection;
Execution of RFP;
Empowered decision makers;
Team morale and motivation
Requirement and
Planning Responsibility
(Authoritative Control
and Coordinated Control)
Responsibility
(High Engagement and
High Sensitivity)
Requirement completeness;
Engaging internal IT support teams
(Both infrastructure and COTS
support)
Analysis and
Conceptual
Design
Responsibility
(Authoritative Control
and Coordinated Control)
Paternalism (Moderate Engagement
and Moderate Sensitivity)
Requirement consistency;
Requirements stability
Design and
Architecture Responsibility
(Authoritative Control
and Coordinated Control)
Paternalism (Moderate Engagement
and Moderate Sensitivity)
Roles and Responsibility
Clarification
Development Harmony
Neoclassical Low (Engagement and
Low Sensitivity)
Development of internal technical
capabilities
Integration and internal process
considerations (i.e. change
management, security clearance
etc.)
Delivery Persuasion
Strategic (High
Engagement and Moderate
to Low Sensitivity)
Data conversion and integrity,
System integration and acceptance
testing,
Training and job redesign,
Close-out Persuasion
Strategic (High
Engagement and Moderate
to Low Sensitivity)
Post-implementation evaluation;
Troubleshooting/crises
management,
Applying lessons-learned for each
iteration
Enterprise COTS implementation is a complex, intensive, and dynamic activity that requires close
coordination and cooperation among a diverse group of stakeholders. For large-scale
implementation, as in this study’s Cases, this often involves internal and external end-users,
Page 248 of 307
partners, public, government, sponsors, organizational IT leaders, external partners, vendors (in
the form of COTS providers and solution integrators), suppliers, internal IT support teams, and the
project team. Cognizant engagement and management of such a diverse group of stakeholders is
absolutely critical for any project’s success, for stakeholders not only exert influence on a project’s
operations, they also provide much-needed resources and support (Verbeke & Tung, 2013).
Whereas a resource-based view is useful for explaining the process of resource accumulation and
exploitation (Barney, 1986), the most effective and efficient access and usage of these resources
is demonstrated by a stakeholder approach through a deliberate stakeholder management (Verbeke
& Tung, 2013).
Although the significance of stakeholder engagement is well established in IS ethics and strategy
literature, the debate concerning criteria for morally acceptable engagement is still quite alive
(Swanson, 2013), as characterization of stakeholder engagement ultimately depends on
management style. Aside from the possibility of engaging stakeholders in a deceitful manner to
fulfill ulterior motives, stakeholder engagement is typically understood as “practices the
organization undertakes to involve stakeholders in a positive manner in organizational activities”
(Greenwood, 2007). Accepting this common understanding, the analysis here of stakeholder
orientations for different phases of each project indicates that the recommended method is an
integrative approach (Austin, 2003) that facilitates strategic collaboration between the parties. This
approach is also supported by previous researchers who also favoured an integrated and dynamic
framework to facilitate strategic stakeholder engagement (Ostrander, 2004; Schultz & Hatch,
2005).
Page 249 of 307
Such a strategic approach may appear in direct contradiction with the normative core of
stakeholder theory, as proponents of a normative orientation often argue that “good must not be
done for reasons of profits” (Friedman 1970/1993; Chryssides & Kaler, 1993, p. 231). Yet a
combination of both normative and instrumental approaches is necessary for enterprise IS
implementation projects, as demonstrated by Cases A, B, C, and D. Justification for such an
approach is also well founded in both IS and stakeholder literature. Although pressures from
stakeholders towards homogeneity via isomorphism is more relevant at the organizational level
(Verbeke & Tung, 2013), handling divergent interests of various stakeholder groups needs a
sensitive and cognizant management approach at the project level. Such divergent interests are
particularly problematic in the presence of power differentials, as agency theorists have
emphasized (Hill & Jones, 1992). Setting aside vendors’ obvious profit-maximizing motive, the
cornerstone of agency theory assumes a diverging interest between the principal and the agent that
requires the agent establish appropriate incentives and monitoring to limit opportunistic behaviour
(Demsetz, 1983; Fama, 1980; Fama & Jensen, 1983). Such divergence can best be explained
through a critical theory perspective of stakeholder management, where a pragmatic discourse
seeking effectiveness (i.e., vendor), an ethical-existential discourse seeking goodness (i.e., project
management team, users), and a moral discourse seeking correctness or justice (i.e., organizational
leadership) all coexist in a stakeholder-engagement scenario (Hill & Jones, 1992).
Regardless of general advice to assess stakeholder engagement from an opportunity-cost
perspective (Verbeke & Tung, 2013) and to introduce new constructs like ethical capital (Wagner-
Tsukamoto, 2007), both implicit and explicit contracts in a stakeholder-agent scenario require
governance structures and processes to enforce them (Hill & Jones, 1992). Furthermore, to
Page 250 of 307
effectively leverage resources embedded in a social network resulting from stakeholder
engagements, suitable control configurations are indispensable. In the context of ISD or ES
implementation, the critical role of control in integrating complementary stakeholder capabilities
has already been affirmed in control literature (Kirsh, 1997). The salience of this aspect is even
more evident in a COTS implementation involving an offshore team. As Clark et al. (1998) stated,
“the truly critical success factors associated with successful outsourcing are those associated with
vendor governance.” Therefore, as optimal stakeholder engagement is essential for the survival
and success of any project, keen sensitivity is also necessary to mitigate any threat from divergent
objectives and opportunistic stakeholder behaviours. This is evident from the cases analyzed for
this research, where distinct stakeholder orientation and control configurations were adopted to
achieve desired project outcomes.
As explained earlier, the application of stakeholder engagement in IS projects is often limited to
mere inclusion of stakeholders in the project’s initiation process, but without much consideration
of its implication. This often fails to achieve required stakeholder deliberation, inclusive dialogue,
and cross-examination of ideas during the planning process or subsequent stages of the project
(Cohen, 1989; Gutmann & Thompson, 2004; Smith, 2003; Young, 2000). Several processes
utilized in Cases A, B, and C, such as user stories, requirement workshops, process-design
workshops, brainstorming sessions, and focus groups, helped facilitate the stakeholder-
deliberation process. Moreover, the use of tools and project management processes, such as RTM-
specific requirements, risk ownership, response ownership, RACIN categorization of project
participants, communication-control plans, and escalation plans, enabled each project to maintain
a desired level of stakeholder engagement. Paralleling engagement management, Cases A and B
Page 251 of 307
used explicit processes to adjust sensitivity levels. A few such processes include design authority,
internal change-control meetings, and backlog prioritization process. These allowed the project
teams to determine activity focus and adjust the project scope to deliver maximum organizational
value with high efficiency. Validity of this claim is further supported by the challenges experienced
in Cases C and D, where incomplete and inconsistent requirements and client-project
disagreements resulted in significant user dissatisfaction. Primary data indicate that a low level of
stakeholder engagement and sensitivity were directly responsible for such undesired outcomes.
Several processes enabled the project teams to maintain high engagement with stakeholders and
still control sensitivity, but some processes adopted in Cases B and D also allowed the projects to,
at times, reduce engagement and achieve both cost and performance efficiency. For example, Case
D used a self-selection process to establish roles and responsibilities that significantly reduced
roles-and-responsibility-related challenges and empowered the team, thus improving team morale
and motivation. According to a project manager in Case D, the benefits of such a process were
evident:
Each member was asked if they wanted to be part of the team with an understanding of the
commitment that will be expected and the approach that will be used. … Each team member
understood they would be called upon to help each other out. … For example, when testing
was behind, respective team members from both the business and ITS jumped-in to keep
the project on track. Business resources were there for business knowledge, testing,
requirements, documentation, and training.
Page 252 of 307
Several other processes used in Cases A, B, and D, such as environment consolidation, were more
technical in nature and permitted projects to maintain low stakeholder engagement, yet still
achieved an efficiency gain. As informant D03PT, a senior developer from Case D states:
I think processes, procedures, plans all need to be worked and reworked throughout the
entire life of the project. In our [name of the project] project we started with a strategic
plan that produced numerous processes and procedures. We kept rebuilding, redesigning,
and reimagining these the entire life of the project. One repeatable process, that I recall,
was redesigned several times. In the beginning, it took two weeks to complete this process,
but by the end of the project it took 2 minutes! … with more accurate results, saving team
members well over 10 weeks of work as well as required support from five other teams to
do the same job.
Similar to processes, control configurations and targeted adjustments of control portfolios also
exerted a significant influence on a project’s stakeholder orientation and on CSF. Building upon
the earlier discussion of diverging stakeholder interests, the usability of a control perspective in
relation to stakeholder orientation is also supported by the primary data from our selected Cases.
Divergence of stakeholder interests is primarily driven by different utility functions of various
groups, where satisfying one group’s interests would reduce the resources available to satisfy other
groups’ interests, as postulated by stakeholder-agency theory (Hill & Jones, 1992). This was a
potential scenario in Cases A and B and an actual scenario in Cases C and D, where incorporating
late requirements from business users would have reduced the vendor profit and/or reduced the
project’s cost efficiency. Such inefficient resource allocation resulting from uncorrected
stakeholder divergence is also referred to as utility loss (Hill & Jones, 1992). Therefore, the
Page 253 of 307
significant role of incentive, monitoring, and enforcement aligning of diverging stakeholder
interests to minimize a project’s utility loss clearly indicates a close relationship between control
configuration, stakeholder orientation, and the challenges faced by the project team.
To substantiate our claim regarding the influence of appropriate control configuration on CSF and
stakeholder orientation, a user satisfaction and risk perspective of IS implementation success is
essential. A close correlation between these two dimensions is observed in DeLone and McLean’s
(1992) six categories of success measurements—system quality, information quality, use, user
satisfaction, individual impact, and organizational impact—and six dimensions of IS
implementations risks—team, organizational environment, requirements, planning and control,
user, and project complexity (Wallance, 1999).
Our analysis of CSF in all four Cases clearly aligns with earlier categorizations, where absence
could be perceived as risk and presence could be used as measurement criteria for implementation
success. Despite a striking alignment with existing knowledge base, a stakeholder perspective
introduces a direct or indirect people aspect behind most CSF, highlighting the implication of
control configurations. For example, in Cases A and B, a trust-based control followed by an
authoritative control during early phases of the projects helped establish projects and ground rules,
secure commitments from organizational leaders and users, clarify expectations from the project
team, and develop a shared understanding among partners. This, in turn, reduced several categories
of risk in both projects in the dimensions of team, user, organizational environment, and planning.
Although the effects of such authoritative control, containing mostly behavioural controls, aimed
to improve the engagement level and coordination among team members, the levels of shared
Page 254 of 307
understanding and expectations were still very low due to minimal opportunity of engagement
from the business users. Additionally, in the absence of a formal contract and time-allocation
process, a procedural control was not very effective during the project’s early phases. This
motivated the projects in Cases A and B to retain an authoritative approach by employing both
outcome-based and behavioural controls for the vendor and extended IT support team to ensure
required cooperation, commitment, and shared understanding. This had clear positive effects for
establishing a common understanding, forming project norms, and minimizing negative
anticipations regarding future performance. Yet, once the desired outcome was achieved, both
projects shifted to coordinated and trust-based control configuration toward the end of the project’s
life cycle, and negative changes in any of the control-balancing trigger factors led the project to
adopt an authoritative approach of control.
A trust-based control portfolio was observed in all four Cases, although not in a harmonized
fashion. Adoption decision for control styles was influenced by several different factors, both
generalizable and idiosyncratic to the project. Such control was reflected in the project’s reduced
communications, status meetings, progress updates, managerial reviews, audits, and other
administrative activities. Such a control was adopted in Cases A and B due to higher levels of
shared understanding, positive performance by the controlee (i.e., vendor and IT support teams),
and a matured level of self and clan controls. A trust-base control is particularly important in an
agile development environment with diffuse stakeholder groups (Hill & Jones, 1992), and where
clan formation is possible and empowerment of the team necessary for effective task executions.
In both Cases A and B, a trust-based control demonstrated a positive influence by motivating
project participants and reducing information asymmetry. This was reflected in behavioural
Page 255 of 307
changes such as the vendor assuming a higher level of responsibility than required and proactive
communications on the client side. According to informant A02PT, the technical lead for Case A:
Empowering people is something that always leads to surprising results. Once a team
realizes they are empowered and protected, creativity flourishes, tensions reduce, and an
amazing sense of ownership and pride occurs, usually resulting in some highly creative,
innovative solutions. Given the right people for the job, the right tools for the job and an
environment to be creative, some out of the box solutions were found that allowed us to
meet all of our client’s design requirements.
To the contrary, improper application of a trust-based control can severely harm the
implementation process and overall project objective. Projects in Cases C and D often applied
trust-based controls, while Cases A and B opted for an authoritative or coordinated configuration.
Although an inexperienced project manager and a lack of technical aptitude among the core project
team in respect to the new COTS product were the driving forces behind such decisions, a few of
the negative consequences of an incorrect control configuration included delayed delivery of work
packages, inconsistent and incomplete requirements, poor functional quality, and increased budget
due to unexpected changes. In Cases C and D, this further reduced shared understanding and
increased negative anticipation about future performance amongst the project’s stakeholders, thus
resulting in a high level of user dissatisfaction, unexpected process changes, and post-go-live
challenges and crisis management.
Besides two salient control configurations, a third category of control configuration, coordinated
control (Gregory et al., 2014), was also observed in all Cases. Although Henderson and Lee (1992)
Page 256 of 307
indicated that a joint application of both formal and informal control in IS implementation results
in higher performance, Kirsch (1997) advocated a portfolio of controls to address the challenge of
diffuse stakeholder groups with diverse knowledge and skills. Complementing Kirsch’s (1997)
argument for a control portfolio, Soh et al. (2011) introduced the concept of controllee-initiated
controls, as well as called for separate controls targeted at each stakeholder group to address
distinctive attributes of each element in a multi-stakeholder IS implementation. The coordinated
control configuration observed in Cases A, B, C, and D closely supports these earlier arguments.
This control primarily supported the concept of value co-production by stakeholders (Zeithaml,
1981; Parasuraman et al., 1985; Bettencourt et al., 2002), where poor performance by one
stakeholder group has negative consequences for others. While contingent upon a strong mutual
commitment (Gregory et al., 2014), a coordinated control style significantly reduced requirement
conflicts during early stages and disputes during audit and inspections in the late stages. Besides
informal meetings, Cases A, B, and C arranged site visits by offshore vendors that helped clarify
shared understating of technical issues and client work culture, and established a vendor-proposed
coordination process to overcome negative impacts of time zone differences between Canada and
Europe. A bi-directional control is also observed in Cases A and B, where the client initiated
regular discussion sessions with the project’s technical team and the vendor to redesign business
processes, validate proposed requirements, and understand the technical capabilities of the new
COTS solution. This eventually resulted in intense cooperation among the parties, as well as
enhanced sharing of knowledge among stakeholder groups. Besides the obvious impact of better
understanding and effective work coordination among stakeholders, a coordinated control
configuration significantly reduced both requirement risks and user risks, thus leading to better
Page 257 of 307
process design, effective training plans, and job redesign, as well as lower resistance in acceptable
deliverables by establishing perceptual congruence among stakeholders.
In summary, primary data and codes derived from the same suggest the direct influence of control
configuration and various processes on a project’s critical success factors. Furthermore, both
control configuration and various processes influenced stakeholder orientation within a project by
adjusting stakeholder engagement and sensitivity. A project’s stakeholder orientation, in turn, had
a direct moderating relationship with various CSF in the COTS implementation project. These
findings directly support our proposed relationships among control configurations, stakeholder
orientations, processes, and CSF, as presented in Figure 16.
Figure 16. Relationship between process, control configuration, stakeholder orientation and
CSF
Additionally, the effect of a control configuration on stakeholder orientation and CSF are largely
dependent on context-specific factors of the project. Such contextual factors also dictate the nature
of a control-configuration adjustment, leading to a dynamic control configuration scenario
throughout the project’s lifecycle. This aspect is depicted in Figure 17 below.
Processes
Stakeholder Orientation
Control Configuration
COTS Implementation
Outcome CSF
Page 258 of 307
Figure 17. The dynamic nature of a control configuration.
In summary, synthesizing earlier analysis of implementation processes, control configurations and
stakeholder orientations for selected Cases affirmed a close relationship among three vital
constructs (process, stakeholder orientation, and control configuration) of an enterprise COTS
implementation project and the project’s CSF. Although both implementation-related processes
and control configurations influenced the nature of stakeholder orientation for a COTS
implementation project, all three aspects of a project were found to exert considerable influence
on CSF for the project. Additionally, a dynamic nature of the control configuration indicated the
necessity for a control configuration shift based on the presence of three distinct trigger factors:
level of shared understanding, negative anticipation, and deviation of expectation.
Page 259 of 307
7.0 Conclusion
Chapter 7 contains five sections. In Section 7.1, we provide the study’s conclusions. In Section
7.2, we summarize the answers to our research questions. In 7.3 and 7.4, we discuss the theoretical
contributions and impacts for practice, respectively. Section 7.5 identifies various limitations of
this research. Finally, in Section 7.6, we provide suggestions for future research.
7.1 Conclusion
We set out to explore an existing IS phenomenon of enterprise COTS implementation through a
novel perspective of process, control, stakeholder engagement, and critical success factors. Our
objective was to identify the relationships among processes, control configuration, and stakeholder
engagement, and how they influence CSF for an enterprise COTS implementation.
Consequently, we drew from control theory, control-balancing theory, stakeholder theory,
stakeholder agency theory, IS-related CSF literature, and IS-implementation literature to develop
our argument. In doing so, we have proposed a new and vital construct, stakeholder orientation,
for COTS-implementation projects. Initially, we identified a micro-level model for enterprise
COTS implementation, which facilitated identification of key processes and their nature in an IS
project. Subsequently, we employed control theory and theory of control balancing to capture and
validate the dynamic nature of control configurations in an IS project. We also leveraged
stakeholder theory and stakeholder agency theory to capture the dynamic stakeholder orientation
in a IS implementation project. Finally, through analysis and synthesis, we established vital
relationships among processes, control configurations, stakeholder orientations, and CSF where
proper management of CSF leads to a successful COTS implementation project outcome.
Page 260 of 307
7.2 Summary of answers to Research Questions
In this section, we summarize the answers to the six research questions investigated through the
current research.
7.2.1 What is an activity- and process-based model for enterprise COTS
implementation?
Enterprise systems, in the form of commercial software packages that enable organization-wide
data- and business-process integration, has received considerable attention in IS literature. Even
early researchers like Markus and Tanis (2000) have affirmed the existence of wide range of
options for implementation and ongoing operations for such systems. A rich body of literature has
also investigated the traditional software development process. However, implementation
processes for COTS-based enterprise systems have been argued to be substantially different from
traditional software development (Morisio et al., 2002). Although intersecting with the research
on ERP implementation efforts, we have pointed out the possibility of considerable variations in
implementation processes for non-generic or well-known COTS. Additionally, most
implementation process research for well-known COTS (i.e., ERP, CRM) have primarily focused
on macro-level categorization of the process. Motivated by this, we have identified a macro-level
COTS implementation process (Figure 12) by analyzing four different enterprise COTS
implementations. Significance of this macro-level model is summarized by the Table 45 below.
Page 261 of 307
Table 45: Process-based (macro level) model for COTS implementation
Purpose Identified phases Characteristics of
the model
Significance or Contribution
Purpose of a macro level or
process level COTS
implementation model is to
capture the logical flow of
implementation activities,
logical process groups and
identify significant
processes influencing the
project CSF
Pre-initiation,
Initiation,
Requirement and
planning,
Analysis and
conceptual design,
Design and
architecture,
Development,
Delivery,
Close-out.
Logical Process
groups
Information flow
among the
process groups
Implementation
related processes
Well-defined Project
boundaries
Capturing the nature of the
implementation practices
Integration of processes and
tools with implementation
phases
Foundation for a activity
based or micro level
implementation model
Enabled by this macro-level model, we have further analyzed the responsible processes, tools and
information flow to develop a micro-level or activity based model for enterprise COTS
implementation (Figure 13). The significance of this micro-level model is summarized by the
Table 46 below.
Table 46: Activity-based (micro level) model for COTS implementation
Purpose Identified phases Characteristics of the
model
Significance or
Contribution
The purpose of a macro
level COTS
implementation model is
to identify the key
activities in relation to the
stakeholders involved;
This provides the
foundation for both
stakeholder orientation
and control configuration
decisions
Procurement &
RFI/RFP
Requirement & Planning
Design & Architecture
Development
Delivery & Transition
Logical activity
grouping for each
phase
Stakeholder
engagement needs
Stakeholder
sensitivity control
Well-defined logical
phases
Depiction of
stakeholder
engagement
requirements and
points
Control directions
Page 262 of 307
Each of the four selected COTS products were intended for very different purposes, such as
administration of money market instruments, regulation of country-wide financial transactions,
control and administration of Canada’s currency circulation, and organization-wide knowledge
management. In addition to the identification of a generic implementation process for enterprise
COTS, a micro-level model also enabled us to identify the critical activities and project-specific
processes necessary to substantiate their value. The value aspect of identified processes and tools
were established by outlining their influence and relationship with each project’s stakeholder
engagement (Figure 16) and CSF.
7.2.2 How can organizational processes and tools contribute to COTS implementation
success?
COTS implementation success, well situated with the domain of IS implementation success, is no
longer an emerging research phenomenon in IS literature. Although this specific category of IS
has not received significant attention from researchers, practitioner involvement with COTS
implementation and IT investments under this category, from a process and tools perspective, is
rapidly growing. Motivated by this knowledge gap and industry trend, we aimed to investigate the
relationship between processes and tools and COTS implementation success. In answering this
question, we augmented the IS implementation-specific CSF from the literature with experiences
reported by informants in our analyzed Cases. We have found that the organizational tools and
processes introduced by various stakeholder groups have a direct relationship with the reported
CSF for each project. This relationship is summarized in Table 47 below.
Page 263 of 307
Table 47: Processes and tools influencing CSF
Name Category Impact CSF
RFP/RFI Process High engagement of stakeholder, and high
sensitivity towards stakeholder; Strategic alignment
of the project investment; Foundational process for
requirement and analysis.
Visioning and planning;
Build a business case;
Selection of appropriate
COTS product.
RACIN Tool Balancing the stakeholder engagement and
stakeholder sensitivity throughout the project
lifecycle; Stakeholder-role clarity; Facilitate
authoritative and co-ordinated controls
Project champion;
Change management;
Project management
RTM Tool Controlling stakeholder engagement and sensitivity
during early lifecycle; Managing stakeholder
expectation throughout the project’s
implementation; Validating deliverable scope
during the close-out phase
Client consultation;
Project management and
expectation management
Design
Authority
Process Preventing design changes and design conflicts;
Promoting high stakeholder engagement with
moderate sensitivity
Implementation strategy
and timeframe;
Empowered decision
makers; Team morale
and motivation; IT
infrastructure; Data
conversion and integrity
Deep Dive Process Generalized innovative problem-solving during
early phases of the projects; protecting integrity of
the requirements; minimizing conflicts
Legacy system
consideration; Client
consultation,
Joint Parallel
Testing
Process Minimizing scope validation delays and scope
related disagreement among the stakeholders;
enhancing shared understanding
Client consultation;
System testing
A positive influence on CSF led to a smoother COTS implementation process and an overall
implementation success. In parallel, we also captured scenarios where an inadequate management
of CSF resulted in client dissatisfactions, poor COTS quality or deliverables, and poor usability of
the delivered system.
Page 264 of 307
7.2.3 What is the nature of control configuration in a multi-partner COTS
implementation and what are the factors responsible for the application of control
balancing?
A seminal view of control traces its origin to Weber (1947), who defined control as a process of
creating and monitoring rules through hierarchical authority (Henderson & Lee, 1992).
Subsequent, researchers further contributed to the control literature by categorizing control in two
broader dimensions, formal and informal (Crisp, 2002). Both categories conceptualize control in
a dyadic sense through a controller and controllee relationship (Kirsch, 2004). Recent
contributions in this domain introduced the concept of control portfolio (Gregory et al., 2014) and
controlee-enacted controls in multi-stakeholder scenarios. Building on this perspective, we
attempted to investigate the nature of control configurations—more specifically, the dynamic
nature of control configuration and the underlying drivers for targeted control adjustment.
Our analysis of four COTS implementation projects indicates the existence of control
adjustment during various phases of the project. Based on evidence reported by all
informants, we identified four distinct control orientations for each project: (a) strategic,
(b) responsibility, (c) harmony, and (d) persuasion. Furthermore, we also identified three
distinct trigger factors: (a) shared understanding, (b) negative anticipation, and (c)
deviation of expectations. These three factors can be classified into two major categories:
(a) re-active control balancing, and (b) pro-active control balancing.
Page 265 of 307
7.2.4 How can CSF and other challenges be successfully managed through optimal
control balancing by the project?
Empirical research in the IS domain has identified a large number of factors contributing to the
success or failure of enterprise systems implementations, including well-known COTS such as
ERP and CRM. This is evident from our review of CSF literature in Chapter 2. Although these
factors reside in a technical context, they are often unrelated to the technology itself (i.e., the
enterprise or COTS system being implemented). Rather, they often relate to politics, motivation,
and conflict resulting from multi-stakeholder interactions and relationships due to divergent
priorities and goals (Markus & Benjamin, 1996; Hartwick & Barki, 1994; Roby et al., 1989).
Despite the risk of introducing potential negative factors that may cause an IS implementation
project to fail, engaging all legitimate stakeholders in a meaningful way is indispensable if we
harness the complementary skills, tangible resources, and knowledge possessed by them.
Additionally, IS implementation success largely depends on the attitude and perception of the
primary stakeholder of the project. Therefore, a significant portion of the CSF and challenges in
IS implementation context are found to be related to people and processes involved with the
project.
Our findings from Cases A, B, C, and D, as elaborated at the end of Section 5.1.2, 5.2.2, 5.3.2,
5.4.2, and 6.2, indicate that changing control configurations of the project, depending on certain
existing factors, facilitate the development of shared understanding among stakeholders and
manage stakeholders’ perceptions. These, in turn, result in higher cooperation, diligent
contributions, reduced activity cost, innovative solutions, high quality and timely delivery, and
higher user acceptance. All these factors, in one way or another, were found to be related to the
salient CSF and challenges for the analyzed Cases.
Page 266 of 307
7.2.5 What is the nature of stakeholder orientation during a COTS implementation for a
public sector IS implementation?
As it is imperative that effective management of stakeholder engagement and relationship, so as
to elicit their cooperation and contribution, is critical for an IS project’s success and survival
(Kirsch et al., 2002; Clark et al., 1997), maintaining a static approach to stakeholder management
can also be detrimental. Reasons for such conjecture are largely driven by the fact that stakeholder
goals and priorities are often divergent (Markus & Benjamin, 1996; Hartwick & Barki, 1994; Roby
et al., 1989). Therefore, we proposed at the onset of this research the need for optimal stakeholder
engagement, coupled with a mechanism to regulate stakeholder influence on a project’s activities
and objectives. Driven by this motive, our inquiry on this aspect of stakeholder engagement found
a dynamic pattern where stakeholder engagement varied at different phases of the project. This
was supported by most informants in Cases A, B, C, and D. This pattern indicates significant
stakeholder engagement during early phases of a project, gradually subsiding during the design
and development phase, then increasing towards the end of the project’s life cycle. Pattern coding
of the primary data indicated several underlying factors responsible for such engagement
variations throughout the life cycle. In addition, processes and mechanisms were also observed to
regulate the effects of stakeholder engagement in a project’s objective. Combining the aspect of
stakeholder engagement with stakeholder sensitivity, we developed a new construct, stakeholder
orientation, to categorize the engagement variations for our project. This allowed us to identify
four distinct stakeholder orientation for each project: (a) strategic, (b) responsibility, (c)
paternalism, and (d) neoclassical. These four orientations were mapped in a grid, with each
occupying a dedicated quadrant. Furthermore, based on the stakeholder engagement framework
Page 267 of 307
developed by Greenwood (2007), we have indicated the possibility of sub-segments within each
quadrant of our model depending on the intensity of stakeholder engagement and sensitivity. This
is described in Table 48 below.
Table 48: Stakeholder Engagement and Stakeholder Sensitivity
Segments/
Quadrant
Title Stakeholder
Engagement
Stakeholder
Sensitivity
Relationship between
stakeholder engagement
and stakeholder
sensitivity
A
(Responsibility)
Ethical
altruism
Strong engagement of
stakeholders
Acts in the interest of
primary stakeholders;
Sensitivity is moderate
to high in the sense
that a few legitimate
stakeholders might be
left out
Increasing stakeholder
sensitivity with strong
engagement measure driven
by corporate social
responsibility; organization
should not totally sacrifice
own interests to help others'
interests
B
(Responsibility)
Anti-
capitalism
Moderate to high
engagement of
stakeholders as
determined by the
organization
Considers the interest
of all stakeholder
Including illegitimate
Participation of so many
(including illegitimate)
stakeholders that the
purpose of the firm may get
compromised
C
(Paternalism)
Limited
Paternalism
Low to moderate
stakeholder engagement
determined by the
company
Acts in the interest of
a broad group of
stakeholders
determined by the
organization
Acting in the perceived
interest of the stakeholders
with limited consultation
D
(Paternalism)
Strong
Paternalism
No or little stakeholder
Engagement
Acts in the interest of
Legitimate stakeholder
as determined by the
organization
Acting in the perceived
interest of the stakeholders
without consultation to the
point of interference and
reduction of liberty.
E
(Neoclassic)
Market Low to moderate
stakeholder engagement
due to economic
reasons
Does not act in the
interest of legitimate
stakeholder
Controlled and highly
focused engagement to
further the interests of the
owners. Organization and
stakeholders as economic
entities
F
(Neoclassic)
Illegal (outside
the boundary
of the law or
No stakeholder
engagement as
determined by agents in
control of the company
Does not act in the
interest of legitimate
stakeholder;
Organizations act in their or
principal’s interests either
illegally or outside moral
minimum norms. Could
Page 268 of 307
7.2.6 How can CSFs and other challenges be successfully managed by the project
through optimal stakeholder engagement?
As indicated in section 7.2.4, stakeholder engagement is essential for a project’s survival and
success. At the same time, the possibility of opportunistic stakeholder behaviour is a realistic threat
to a project’s objective. Additionally, traditional control mechanisms are often ineffective in a
complex and outsourced IS implementation. Therefore, a stakeholder orientation is necessary to
manage both the engagement and sensitivity of stakeholders. In addition to identification of four
distinct stakeholder orientations, we also provided theoretical justifications to support their
alignment with key tenets of stakeholder theory. Adoption of different stakeholder orientations at
different phases of the project demonstrates a profound impact on the CSF and implementation
challenges. For example, a responsibility approach during early project phases substantially
reduced requirement and user risks for Cases A and B. An opposing effect was observed for Cases
C and D, which did not demonstrate sufficient stakeholder engagement and sensitivity.
Appropriate stakeholder orientation also supported the concept of stakeholder integration (Plaza-
accepted
custom)
Treats stakeholders as
purely instrumental
include fraud, theft, and
abuse of human rights
G
(Strategic)
Reputation/
Legitimacy
Engaging small groups
of legitimate
stakeholders to further
shareholder interests.
Stakeholders are
selected primarily
guided by the strategic
objectives of the
organization with a
low to moderate
sensitivity
Narrow focus in terms of
stakeholder sensitivity and
moderate on engagement;
Engaging stakeholders
enhances strategic
alignment, reputation and
legitimacy with
stakeholders.
H
(Strategic)
Irresponsibility
(bad faith)
Excessive engagement
without accountability
or responsibility
towards stakeholders
Appears to act in the
interest of only
influential stakeholder
s
Engaging with stakeholders
under deceptive conditions,
acting ‘‘as if’’ the aim is to
meet stakeholders’
interests.
Page 269 of 307
U´beda, Burgos-Jime´nez, & Moreno, 2010) in all four Cases via integration of stakeholders’
demands, knowledge absorption, and improved performance. This, in turn, positively influenced
several other risk dimensions for the projects, such as complexity risk and planning and control
risk.
To summarize, our findings from Cases A, B, C, and D, as elaborated at the end of Section 5.1.3,
5.2.3, 5.3.3, 5.4.3, and 6.3, indicate that the choice of various tools and processes enables the
project team to control both stakeholder engagement and stakeholder sensitivity. Opposing
stakeholder orientation between Cases A and B, and Cases C and D can be correlated to the number
of challenges faced by each project. This gives rise to the concept of optimal stakeholder
engagement. Additionally, an optimal stakeholder engagement also helps ensure a positive
influence on pertinent CSF for the project as well as mitigate any challenges.
7.3 Implications for theory
The current research extends the knowledge of enterprise COTS and IS implementations in several
major ways.
First, the current research captures a successful implementation process for enterprise COTS, with
a micro-level or task-oriented focus. This task-oriented focus led to the discovery of several
beneficial processes, tools, and activities that can facilitate the further study of stakeholder
engagement and application of control theory in this context. A literature review related to this
context also indicates that a vertical analysis of an end-to-end COTS or IS implementation has not
received sufficient attention. Therefore, the implementation model identified through the current
research offers a significant contribution in the IS implementation domain.
Page 270 of 307
Second, the end goal of a public sector IS implementation is different compared to private sector
IS implementation. Due to divergent mission objectives and stakeholder groups, value
maximization often takes precedence over profit maximization for most government
organizations. The interpretation of value also widely varies between stakeholders (Flak, 2005).
Scholl (2001) has studied major e-government initiatives undertaken by public sector
organizations using stakeholder theory. However, the scope of Scholl’s (2001) research does not
allow for a detailed analysis of stakeholder theory in relation to the characteristics of public sector
organizations. Through a case-based grounded theory approach, the current research identified the
major stakeholder orientations in public sector IS implementation, including the manifestation of
three major aspects of stakeholder theory identified by Donaldson and Preston (1995): normative,
descriptive, and instrumental. Together, these can be considered theory extension efforts by the
current research.
Third, findings by Ballejos and Montagna (2011) clearly indicate the significance of the
stakeholder perspective when considering a multi-party joint IS implementation effort, which is
even more salient for government organizations due to a larger stakeholder group, as well as
different ethical considerations and value-maximization objectives (Flak, 2005). Ballejos and
Montagna (2011) have also found that engagement alone is not adequate for ensuring sufficient
stakeholder consideration in an IS design process. Rather, a level of consistency in stakeholder
representation is necessary to avoid underachievement of IS project goals due to poorly conceived
procedures. Yet, a majority of stakeholder-related research in the IS domain focuses on the concept
of “organizational responsibility towards the stakeholder,” which directly aligns with the
normative core of stakeholder theory. The current study complements this conventional approach
towards stakeholder research in a significant way, both by considering the reality that stakeholders
Page 271 of 307
can also impact the organization or project, and by focusing on the concept of stakeholder
orientation, which is manifested through a combination of stakeholder engagement and
stakeholder sensitivity. Furthermore, capturing the dynamic nature of stakeholder orientation in a
complex IS implementation project through a well-defined framework will facilitate understanding
related to the adoption of normative and instrumental aspects of stakeholder theory. This will also
help us understand Wicks’ (1999) convergent stakeholder theory, according to which the
instrumental usage of stakeholder theory does not require sacrificing the normative core. From this
perspective, the current research can be considered a significant theory elaboration effort.
Fourth, the current research analyzed the enterprise COTS implementation process through the
control-balancing theory proposed by Gregory, Beck, and Keil (2014), and identified the necessity
of control balancing for a multi-partner COTS implementation with a distributed team. Our
application of control-balancing theory is much more complex compared to that of Gregory et al.
(2014). This contributes to the generalizability of this relatively new theory. Furthermore, the
identification of underlying trigger factors that lead to control-balancing decisions can be
considered a substantial contribution to control-balancing theory.
Fifth, one of the major objectives of the current research aimed to identify stakeholder orientation,
which is a broader concept compared to stakeholder engagement. The development of a sound
theoretical framework to assess an organization’s stakeholder orientation can be considered a
significant contribution to stakeholder theory.
Finally, the current research offers a significant contribution to enterprise IS implementation by
linking implementation-specific processes, control configurations, and stakeholder orientations to
critical success factors for IS implementation project. Existing literature in this domain has
Page 272 of 307
examined control configurations, processes, stakeholder engagements, and CSF in isolation.
However, the effect of processes and control on stakeholder engagement and, in turn, on the CSF
has not yet been examined. Thus, our established links through the integration of all these
theoretical perspectives offers a holistic view into enterprise COTS implementation.
7.4 Implications for Practice
The knowledge and insights acquired through the analysis of multiple enterprise COTS
implementations will be appreciated by practitioners interested in similar implementations.
First, a task-and-activity-level analysis of the implementation phases facilitated the identification
of critical tasks, processes, and tools. Critical phases, information flow, tasks, tools, and processes
identified through our investigation can be utilized for most large-scale COTS implementation
projects to enhance their probability of success.
Second, the identification of CSF and their management are two important aspects of an enterprise
COTS and most IS implementations. Practitioners in all organizations are exceedingly interested
in both aspects, as most large-scale projects include a lessons-learnt exercise towards the end of a
project. Our examination of CSF management through stakeholder engagement and control
balancing will offer valuable insight for practitioners, enabling them to better manage CSFs during
a COTS implementation.
Third, the identification and conceptualization of stakeholder orientations related to COTS
implementation can help an organization, specifically project management teams, be cognizant of
Page 273 of 307
engagement versus sensitivity, which will help project management teams maintain an optimal
stakeholder orientation by selecting the right tools and processes.
Fourth, an appealing aspect of the current research for practitioners is the enumeration of control
dynamics for an enterprise COTS implementation, as controls are heavily utilized by most IS-
implementation projects. Insight into the factors that require the adjustment of control
configuration during implementation might be a source of further interest and benefit for
practitioners.
Fifth, the framework developed in this research can be of tremendous benefit for organizations in
evaluating their IS-implementation practices from an organizational stakeholder perspective. An
old but still accurate management adage—“You can’t manage what you don’t measure”—clearly
indicates the value of assessing stakeholder engagements in IS implementation using the
framework proposed in this paper, and validates the alignment of IT initiatives with organizational
stakeholder orientation. Through a grounded theory research approach, we identified four major
dimensions of organizational stakeholder orientation: (a) responsibility, (b) paternalism, (c)
neoclassical, and (d) strategic. These were further subdivided into two subsections each, resulting
in a total of eight different organizational orientations based on the level of stakeholder
engagement and sensitivity. The proposed framework shows the possibility of sacrificing the
normative aspect entirely in favour of an instrumental approach; however, due to the public-sector
context of the analyzed cases, each project demonstrated the characteristics of a convergent
stakeholder approach, whereby implementation phases have a well-defined normative core and
supporting instrumental processes to make them viable (Jones & Wick, 1999). Therefore, the
current research, in addition to answering the “how” aspect of stakeholder theory’s value
Page 274 of 307
proposition from a micro perspective, serves to elaborate on the convergent stakeholder theory
proposed by Wicks (1999).
Finally, the current research used an agile methodology to analyze IS implementation projects. In
modern times, agile and agility are recurring concepts that have engulfed, even baffled, both IT
professionals and organizational leaders. This is a consequence of never-ending efforts to find new
ways of combating ever-increasing threats from market volatility that generates turbulent business
environments, pressure for shorter delivery cycles, demand for greater productivity and efficiency,
and a rapidly shifting information technology landscape. Regardless of compelling motivations,
embarking on the agility bandwagon without a sound understanding of the concept may have dire
consequences for a company’s IT division and the organization as a whole. This is because of the
reciprocal relationship between IT and business, as established through strategic IT-business
alignment equations in the IT governance literature (Rahumi, Møller, & Hvam, 2016). As IT-
enabled processes and consumption of IT resources lead to business-value maximization through
strategic IT-business alignment (Peterson, 2004), adoption of innovative IT practices is often
influenced by organizational-level variables. Due to a virtually inseparable and often symmetric
relationship between IT and business models, successful agile transformation of IS implementation
practices is heavily influenced by an organization’s internal and external environment. Further
potential complications to agile transformation are tied to the traditional or prevalent practitioner’s
perspective that is unable to conceive of IT services and capabilities as concurrently stable and
dynamic. The COTS implementation process developed through the current research presents a
model for practitioners to capture how agile process can be integrated with traditional practices.
Page 275 of 307
7.5 Limitation of Current Research
7.5.1 Firms sampled and generalizability
We have used a single firm to collect data for this thesis which comprised of four distinct enterprise
COTS implementation cases. The firm selected for this research was of sufficient size and
complexity that it had designated IT organization with segregated support teams specializing on
different areas of IT such as COTS application support, network support, infrastructure support,
middle-tier support. This also indicated the existence of formalized practices (Weill & Ross, 2004).
Thus, this research may not be applicable to firms that do not have such a function or structure.
Moreover, this research selected an organization that belongs to the public sector. As the
stakeholder engagement approach is often different due to existing processes and the drive for
transparency, optimal stakeholder engagement levels identified through the current research may
not be fully supported by similar enterprise COTS implementation within a private organization.
For example, a private organization may demonstrate a higher level of instrumental approach
toward the stakeholder engagement influenced by the dominance of ‘stockholder’ theory.
7.5.2 Interview method
Akin to any other case study research, the current research also shows the possibility of suffering
from informant-bias due to the face to face interview method. Interview informants may have
sought to justify their actions and bias the data (Cresswell, 2009). Since the chief of ITS and the
senior leadership team have sanctioned the data collection, interview informants may have been
influenced (Kvale,1996). Additionally, project leadership team might have overstated the
Page 276 of 307
stakeholder engagement and project contributors might have overstated the level of management
control. We designed the interview protocol to minimize such risks through assurances of
anonymity and by conducting interviews with multiple individuals with different perspectives
within each project.
7.5.3 Granularity of the stakeholder orientation framework
A key contribution of the current research is the introduction of a stakeholder orientation
framework capturing ‘stakeholder engagement’ and ‘stakeholder sensitivity’. As the framework
models ‘engagement’ and ‘sensitivity’ using the X axis and the Y axis, resulting quadrants
represents an intersection between the engagement and sensitivity. Current research plots different
implementation phases at different quadrant of the framework based on the level of stakeholder
engagement and sensitivity combination. Although we have proposed the existence of an optimal
level for both construct within each quadrant, the current framework is not granular enough to
produce quantitative values corresponding to the indicated levels.
7.6 Future Research Directions
7.6.1 Agile implementation process of enterprise COTS implementation
The current research attempted to analyze control configuration and stakeholder orientation for
four large enterprise COTS implementation projects. All four selected projects were implemented
by the same organization. Although the selected projects tried to follow an agile implementation
approach, overall agile maturity level for the organization was still in the infancy. Therefore, all
Page 277 of 307
four projects under Case A, B, C, and D indicated the presence of waterfall or traditional project
management practices such as comprehensive upfront planning of the project’s scope. However,
an agile approach is evident during the execution phases of the selected Cases. Since the agile
methodology promotes a high level of team cohesion and frequent interactions among the team
members, the dynamic nature of stakeholder orientation and control configuration is an ideal topic
of investigation for future research in the context of a more mature agile environment.
7.6.2 Trigger factors for shifting stakeholder orientations for COTS implementation
The current research identified three salient factors as trigger conditions for changing control
configurations during a COTS implementation project. These factors are: level of shared
understanding, negative anticipation, and deviation of expectation. Current research found a
correlation between the increase or decrease of the level of these trigger factors and a change in
control configuration of the project. We anticipate the existence of additional trigger factors related
to a control configuration change during a COTS implementation. A longitudinal study would be
a more suitable approach to identify such trigger factors influencing control configurations that
were not captured by the current research.
7.6.3 Enacting Clan control and self control on the development team
The current research investigated the dynamic nature of control configuration during enterprise
COTS implementation. One of the three control configurations, the trust-based control
configuration, utilizes informal control mechanisms such as social controls. However, concepts
such as clan control or self-control were not explored or investigated through the current research.
This appears as an interesting dilemma for control researchers because the core project team in a
Page 278 of 307
COTS implementation project includes highly technical individuals who are typically
compensated for their unique skills and abilities (Ang and Slaughter 2001). Clearly, reward
structures that emphasize individual achievement represent an incentive misalignment from a team
perspective. Therefore, earlier researchers have argued that exercise of self-control can be
detrimental to the relationship between the use of agile methodology and project quality (Maruping,
Venkatesh, & Agarwal, 2009). Future research should investigate contingencies under which the use
of self control yields positive outcomes for COTS implementation project adopting an agile
methodology.
7.6.4 Effectiveness of stakeholder orientation and balancing isomorphic pressure
The stakeholder approach to strategic management examines the firm within a myriad of
relationships. Such diversity of relationships is also very much vivid in a large-scale enterprise
COTS implementation. Several researchers argued that devoting appropriate attention to all
legitimate stakeholders is important to achieve superior performance (Donaldson and Preston
1995; Freeman 1984). However, Verbeke and Tung (2013) argued that stakeholder engagement
leads to resource heterogeneity and competitive advantages in an early stage, but then at a later
stage contribute to inter-firm homogeneity through pressures favoring shared practices. Such
isomorphism or isomorphic pressure is contrary to project innovation. Future research should
investigate how optimal stakeholder orientation can help overcome such isomorphic pressure from
various stakeholder groups.
7.6.5 Validating the current research in the context of a shifting technological landscape
Page 279 of 307
IT infrastructures at present time are significantly shifting from an in-house model to a cloud-
based model. This shift is primarily driven by the rapid increase in the processing power of
computers, the speed of communication channels, and a constant decline of storage costs. In the
presence of these positive innovations, businesses are increasingly moving to cloud to stay
competitive and avoid IT infrastructure obsolesce. This is obvious by the recent adoption trend of
SAP HANA and other cloud-based ERP and enterprise COTS solutions. The cloud introduces
several factors in the domain of stakeholder engagement and control in addition to the differences
with the implementation process. Future research can further enhance the findings of this current
study by validating the stakeholder engagement framework as well as develop new control
dynamics suitable for such a context.
Page 280 of 307
References
Achterkamp, M. C. and Vos, J. F. J. (2008). Investigating the use of the stakeholder notion in project
management literature, a meta-analysis. International Journal of Project Management, 26, 749–757.
Agile software development (15 June 2014), In Wikipedia, the free encyclopedia, retrieved from
http://en.wikipedia.org/wiki/Agile_software_development on June 15th 2014
Albert, C. and Brownsword, L. (2001). Meeting the Challenges of Commercial-Off-The-Shelf (COTS)
Products: The Information Technology Solutions Evolution Process (ITSEP). Proceedings of the First
International Conference on COTS-Based Software Systems.
Albert, C. and Brownsword, L. (2002). Evolutionary Process for Integrating COTS-Based Systems
(EPIC): Building, Fielding, and Supporting Commercial-Off-the-Shelf (COTS): Based Solution,
Technical Report, CMU/SEI-2002-TR-005, ESC-TR-2002-005, November 2002.
Abts, C. (1997). COTS Software Integration Cost Modeling Study. University of Southern California,
Center for Software Engineering, June 1997
Ang, S., S. A. Slaughter. 2001. Work outcomes and job design for contract versus permanent information
systems professionals on software development teams. MIS Quarterly, 25(3) 321
Ayala,J. Ing, J., Perrault, E., Elliott, G., Letkemann, L., and Baynton, M. (2011). The Potential of Online
Learning in Addressing Challenges in Field Instructor Training, Scholarship in the Human Services. 13(1)
Birkmeier, D. Q. and Overhage, S. (2009). A Survey of service identification approaches: Classification
framework, state of the art and comparison. Enterprise Modelling and Information System Architecture.
4(2). 20-36
Beach, S. (2008). Sustainability of network governance: stakeholder influence. Proceedings of
Contemporary Issues in Public Management: The 12th Annual Conference of the International Research
Society for Public Management (IRSPM XII), Brisbane, 1-23.
Beaubouef, G. B. (2009). Maximize Your Investment: 10 Key Strategies for Effective Packaged Software
Implementations, Packt Publishing. Birmingham, United Kingdom
Berman, S. L., Wicks, A. C., Kotha, S., and Jones, T. M. (1999). Does stakeholder orientation matter?
The relationship between stakeholder management models and firm financial performance. Academy of
Management Journal, 42, 488-506.
Babbie, E. (2001). The Practice of Social Research, Belmont, California: Wadsworth Thompson
Learning.
Ballejos, L. C., and Montagna, J. M. (2011). Modeling stakeholders for information systems design
processes. Requirements Engineering, 16, 281–296
Brownsword, L. and Place, P. (2000). Lessons Learned Applying Commercial Off-the-Shelf Products.
Software Engineering Institute, Carnegie Mellon, retrieved June 5th 2014 from
http://www.sei.cmu.edu/reports/99tn015.pdf
Page 281 of 307
Brownsword, L. and Oberndorf, T. (2000). Developing new process for COTS based Systems. IEEE
Software, 17 (4), 48
Boddy, D., and Buchanan, D. A. (1986). Managing New Technology. Basil Blackwell, Oxford, UK.
Brown, A. D. and Jones, M. R. (1998). Doomed to failure: narratives of inevitability and conspiracy in a
failed IS project. Organizational Studies, 19, 73–88.
Bryson, J. M. (2004). What to do when Stakeholder matter, Public Management Review, 6(1), 21–53
Bryson, J., Bromiley, P. and Jung, Y. S. (1990). Influences on the Context and Process on Project
Planning Success. Journal of Planning Education and Research, 9(3), 183 –185.
Bryson, J. and Bromiley, P. (1993). Critical Factors Affecting the Planning and Implementation of Major
Projects. Strategic Management Journal, 14, 319 – 337.
Beaubouef, G. B. (2009). Maximize Your Investment: 10 Key Strategies for Effective Packaged Software
Implementations, Packt Publishing 2009
Benbasat, I., Goldstein, D. and Mead, M. (1987). The Case Research Strategy in Studies of Information
Systems. MIS Quarterly Vol. 11, 369-386.
Brown, A.D., Jones, M.R., 1998. Doomed to failure: narratives of inevitability and conspiracy in a failed
IS project. Organ. Stud. 19, 73–88.
Bingi, P., Sharma, M.K. and Godla, J. (1999), “Critical issues affecting an ERP implementation”,
Information Systems Management,16, 7.
Bullen, C. V. & Rockart, J. F. (1981). A primer on critical success factors. Cambridge, MA: Center
for Information Systems Research, MIT.
Boynton, A. C. & Zmud, R. W. (1984). An assessment of critical success factors. Sloan Management
Review, 25(3), 17-27.
Buchholz, R. and Rosenthal, S. (2005). Toward a contemporary conceptual framework for
stakeholder theory, Journal of Business Ethics, 58(1), 137-48.
Burby, R. (2003). Making Plans That Matter: Citizen Involvement and Government Action. Journal of
the American Planning Association, 69(1), 33 – 50.
Buckhout, S., Frey, E. and Nemec, J. (1999). Making ERP Succeed: Turning Fear into Promise,
Technology, 15.
Bryant, A. and Charmaz, K. (2007). The Sage Handbook of Grounded Theory, Thousand Oaks, CA: Sage
Publications
Bourgeois, L. J., and Eisenhardt, K. M. (1988) “Strategic Decision Process in High Velocity
Environments: Four Cases in the Microcomputer Industry”, Management Science (34)7, 816-835.
Page 282 of 307
Bryman, A., and Bell, E. (2003). Business Research Methods (1st ed., p. 608). Oxford: Oxford University
Press.
Campbell, D.T. (1975). Degrees of freedom and the case study. Comparative Political Studies, (8), 178-
193.
Cresswell, J. W. (2009). Research Design: Qualitative, Quantitative and Mixed Methods Approaches
(Third Edition, p. 260). Thousand Oaks, California: Sage.
Carlile, P. R., and Christensen, C. M. (2005). The Cycles of Theory Building in Management Research
the Cycles of Theory Building in Management Research. Harvard Business School Working Paper 05-
057, Retrieved from http://www.hbs.edu/research/pdf/05-057.pdf
Christensen, C. (2006). The ongoing process of building a theory of disruption. Journal o f Product
Innovation Management, (2004), 39-55. Retrieved from http://onlinelibrary.wiley.eom/doi/10.llll/j.1540-
5885.2005.00180.x/full
Charmaz, K. (2006). Constructing Grounded Theory: A Practical Guide through Qualitative Analysis,
Thousand Oaks, CA: Sage Publications.
Chemuturi, M. (2013). Mastering IT Project Management. Plantation, FL, J. Ross Publishing
Clegg, H. and Montgomery, S. (2006). How to write an RFP for information products. Information
Outlook, 10 (6), 23–31
Cecez-Kecmanovic, D., Kautz, K. and Abrahall, R. (2014). Reframing Success and Failure of Information
Systems: A Performative Perspective, MIS Quarterly, 38(2), 561-588
Chidley, A. (2014). Use COTS Parts to Cut Costs In Military And Aerospace Systems. Retrieved from:
http://electronicdesign.com/components/use-cots-parts-cut-costs-military-and-aerospace-systems
Cobb, M. (1996). Unfinished Voyages: A Follow-Up to the CHAOS Report, The Standish Group.
(http://www.umflint.edu/~weli/courses/bus381/assignment/vo.pdf)
Conboy, K. (2009). Agility from first principles: reconstructing the concept of agility in information
systems development, Information Systems Resource, 20, 329–354
Choudhury, V. and Sabherwal, R. (2003) Portfolios of Control in Outsourced Software Development
Projects. Information Systems Research. 14(3)
Data Center Consolidation Initiatives (2015, December 03) Retrieved from:
https://www.canada.ca/en/shared-services/corporate/data-centre-consolidation.html
Davis, B. (2013). Mastering Software Project Requirements. Plantation, FL, J. Ross Publishing
Davenport, T. (1998). Putting the enterprise into the enterprise system, Harvard Business Review, 76 (4),
121-131.
Page 283 of 307
Davenport, T. (2000). Mission Critical: Realizing the Promise of Enterprise Systems, Boston,
Massachusetts: Harvard Business School Press.
Deelstra, Y., Nooteboom, S. G., Kohlmann, H. R., van den Berg, J., and Innanen, S. (2003). Using
knowledge for decision-making purposes in the context of large projects in The Netherlands.
Environmental Impact Assessment Review, 23(5), 517–541.
Doherty, N. F., Ashurst, C., and Peppard, J. (2011). Factors Affecting the Successful Realization of
Benefits from System Development Projects: Findings from Three Case Studies, Journal of Information
Technology, 1-16.
Donaldson, T. and Preston, L. E. (1995). The Stakeholder Theory of the Corporation: Concepts,
Evidence, and Implications, The Academy of Management Review, 20 (1), 65-91
Deloitte (2010). What is DeepDive? Retrieved on June 17, 2014 from
http://www.deloitte.com/view/en_US/us/Services/consulting/a02b312c66702210VgnVCM100000ba42f0
0aRCRD.htm
DeLone, W.H., and McLean, E.R. (1992). Information Systems Success: The Quest for the Dependent
Variable, Information Systems Research 3 (1), 60-95.
Dingsøyr, T., Nerur, S., Balijepally, V., and Moe, N.B., (2012). A decade of agile methodologies: towards
explaining agile software development. Journal of Systems and Software 85 (6), 1213–1221,
Dube, L., and Pare, G. (2003). Rigor in Information Systems Positivist Case Research: Current Practices,
Trends, and Recommendations. MIS Quarterly, 27(4), 597- 635.
Dwivedi, Y. K., Wastell, D., Laumer, S., Henriksen, H. G., Myers, M, D., Bunker, D., Elbanna, A.,
Ravishankar, M., N., and Srivastava, S.C. (2015). Research on information systems failures and
successes: Status update and future directions. Information Systems Frontier, 17:143–157
Elgazzar, S., Kark, A., Putrycz, E. and Vigder, M. (2005). COTS Acquision: Getting a Good Contract,
4th international conference on COTS-Based SOftware Systems, Spain 2005
Eisenhardt, K. M. (1989). Building theories from case study research. Academy of Management Review,
16 (3), 620-627
Eisenhardt, K. M. (1991). Better stories and better constructs. Academy of Management Review, 14(4),
532-550.
Eisenhardt, K. M. and Graebner, M. E. (2007). Theory building from cases: Opportunities and challenges.
Academy of Management Review, 50(1) 25-32.
Fassin, Y. (2012). Stakeholder Management, Reciprocity and Stakeholder Responsibility, Journal of
Business Ethics, 109, 83–96
Finney, S. and Corbett, M. (2007). ERP implementation: a compilation and analysis of critical success
factors. Business Process Management Journal, 13 (3), 329 - 347
Freeman, R. E. (1984). Strategic Management: A Stakeholder Approach. Boston: Pitman.
Page 284 of 307
Friedman, A. and Miles, S. (2006). Stakeholders: Theory and Practice, Oxford University Press,
Oxford.
Flak, L. S. and Rose, J. (2005). Stakeholder Governance: Adapting stakeholder theory to e-government.
Communications of the Association for Information Systems, 16, 662-664
Greenwood, M. (2007). Stakeholder Engagement: Beyond the Myth of Corporate Responsibility. Journal
of Business Ethics, 74 (4)
Gibbert, M., Ruigrok, W. and Wicki, B. (2008). What passes as a rigorous case study? Strategic
Management Journal, 29, 1465-1474.
Glaser, B. G., and Strauss, A. (1967). The Discovery of Grounded Theory: Strategies for Qualitative
Research, Chicago: Aldine Publishing Company
Glaser, B. G. (1978). Theoretical Sensitivity, Mill Valley, CA: The Sociology Press.
Gregory, R. W., Beck, R., and Keil, M. (2013). Control balancing in information systems development
offshoring projects. MIS Quarterly, 37(4), 1211-1232
Harper, D. (2001). Engage. In Online etymology dictionary. Retrieved on October 3, 2005 from:
http://www.etymonline.com/index.php?search=engage&searchmode=none
Henderson, J.C. and Lee, S. (1992). Managing I/S Design Teams: A control theories perspective,
Management Science 38(6), 757–777.
James, D. and Wolf, M. (2000). A Second Wind for ERP, The McKinsey Quarterly, 2, 100-107.
Jones, T. M., and Wicks, A. C. (1999). Convergent Stakeholder Theory. The Academy of Management
Review, 24(2), 206-221
Jadhav, A. S. and Sonar, R. M. (2009). Evaluating and selecting software packages: A review.
Information and Software Technology, 51, 555-653.
Karamouzis, F., and Longwood, J. (2007). Guidelines of an RFP process for standardized IT service
provider selections. Gartner Research Group
Kelle, U. (2007). The Development of Categories: Different Approaches in Grounded Theory, in The
Sage Handbook of Grounded Theory, A. Bryant and K. Charmaz (eds.). London: Sage Publications, 191-
213.
Kivits, R. (2011). Three component stakeholder analysis. International Journal of Multiple Research
Approaches, 5(3), 318–333.
Gupta, S. Misra, S. C., Singh, A., Kumar, V., and Kumar, U. (2016). Identification of Challenges and
their Ranking in the Implementation of Cloud ERP: A Comparative Study for SMEs and Large
Organizations, International Journal of Quality and Reliability Management, Accepted in June 2016. (in
Press 2016)
King W. R., Grover V. and Hufnagel E. H. (1989) 'Using Information and Information Technology for
Sustainable Competitive Advantage: Some Empirical Evidence' Information & Management 17(2), 87-93
Page 285 of 307
Kirsch, L.J. (1996) The management of complex tasks in organizations: controlling the systems
development process. Organization Science, 7, 1–21.
Kirsch, L.J. (1997) Portfolios of control modes and IS project management. Information Systems
Research, 8, 215–239.
Kirsch, L.J. (2004) Deploying common systems globally: the dynamics of control. Information Systems
Research, 15, 374–395.
Kumar, V., Maheshwari, B. and Kumar, U. (2003). An Investigation of Critical management Issues in
ERP Implementation: Empirical Evidence from Canadian Organizations, Technovation - The
International Journal of Technological Innovation and Entrepreneurship, Vol. 23, 793-807.
Kumar, V., Maheshwari, B. and Kumar, U. (2002). Enterprise Resource Planning Systems Adoption
Process: A Survey of Canadian Organizations, International Journal of Production Research, 40 (3), 509-
523.
Kumar, V., Maheshwari, B. and Kumar, U. (2002). ERP Systems Implementation: Best Practices in
Canadian Government Organizations, Government Information Quarterly, 19 (1).
Kvale, S. (1996). Interviews - An Introduction to Qualitative Research Interviews (p. 326]. Thousand
Oaks, California: Sage.
Lorenzo, O. (2004). A Comprehensive Review of the Enterprise Systems Research, Working Paper,
retrieved April 14, 2014 from http://latienda.ie.edu/working_papers_economia/wp04-12.pdf
Letavec, C. (2014). Strategic Benefits Realization. Plantation, Florida: J. Ross Publishing
Lyytinen, K., and Hirschheim, R. (1987). Information Systems Failures - a Survey and Classification of
the Empirical Literature. In Oxford Surveys in Information Technology. Oxford: Oxford University
Press, 257-309.
Lyytinen, K. (1988). Stakeholders, IS failures and soft systems methodology: an assessment. Journal of
Applied Systems Analysis, 15, 61-81.
Leavitt, H.J. (1965) Applied organizational change in industry: structural, technological and
humanistic approaches. In J.G. March (ed.), Handbook of Organizations. Chicago: Rand-
McNally, 1965, pp. 1144–1170.
Layder, D. (1993). New Strategies in Social Research: An Introduction and Guide (First., p. 218). Oxford:
Polity Press.
Leonard-Barton, D. (1990). A Dual methodology for case studies: Synergistic use of a longitudinal single
site with replicated multiple sites, Organizational Science, 1 (3), 248-266.Margerum, R. (2002).
Collaborative Planning: Building Consensus and a Distinct Model of Practice. Journal of Planning
Education and Research, 21, 237 – 53.
Lewis, G. A. and Wrage, L. (2004). A Case Study in COTS Product Integration using XML. COTS-Based
Software Systems: Third International Conference, ICCBSS
Page 286 of 307
Maybury, M. T. (2006). MITRE TECHNICAL REPORT: Expert Finding Systems. Retrieved May 10th
2014 from http://infoautoclassification.org/public/articles/Maybury_MITRE-Technical-Report-Expert-
Finding-Systems.pdf
Markus, L. and Tanis, C. (2000) ‘The enterprise systems experience – from adoption to success’, In R.W.
Zmud (Ed.), Framing the Domains of IT Research: Glimpsing the Future Through the Past, Cincinnati,
OH: Pinnaflex Educational Resources, Inc.
Mainardes, E. W., Alves, H. and Raposo, M. (2011). Stakeholder theory: issues to resolve. Management
Decision, 49 (2), 226-252
Mahring, M. (2002). IT Project Governance, Pf. D. dissertation, Stockholm School of Economics,
Stockholm, Sweden
Mishra, A. and Mishra, D. (2013). Applications of Stakeholder Theory in Information Systems and
Technology. Inzinerine Ekonomika-Engineering Economics, 24(3), 254-266
Misra, S. C., Kumar, V., Kumar, U., Fantazy, K. and Akhter, M. (2012). Agile Software Development
Practices: Evolution, Principles, and Criticisms, International Journal of Quality and Reliability
Management (IJQRM), 29 (9), 972-980.
Misra, S.C., Kumar, V. and Kumar, U. (2010). Identifying Some Critical Changes Required in Adopting
Agile Practices in Traditional Software Development Projects, International Journal of Quality and
Reliability Management, 27(4)
Misra, S. C., Kumar, U. and Kumar, V. (2010). Modeling Critical Challenges Required for Adopting
Agile Software Development Practices in Projects Practicing Traditional Plan Driven Practices, Software
Quality Professional Journal, American Society for Quality Publication, 12(3), 20-32.
Misra, S. C., Kumar, V. and Kumar, U. (2009). Modeling Factors that will Influence Success of Projects
that want to Adopt Agile Software Development Practices, Journal of Systems and Software, 82 (9),
1869-90.
Mitchell, R. K., and Agle, B. R. (1997). Toward a Theory of Stakeholder Identification and Salience:
Defining the Principle of Who and What Really Counts. Academy of Management Review, 22(4), 853-886
Moen, R. (2001). A Review of the IDEO Process, Retrieved on June 17, 2014 from
http://www.rwjf.org/content/dam/web-assets/2001/10/a-review-of-the-ideo-process
Morisio, M., Seaman, C.B., Basili, V.R., Parra, A.T., Kraft, S.E. and Condon, S.E. (2002). COTS-based
software development: Processes and open issues. The Journal of Systems and Software, 61, 189–199
Maruping, L.M., Venkatesh, V. and Agarwal, R. (2009). A Control Theory Perspective on Agile
Methodology Use and Changing User Requirements, Information Systems Research 20(3), 377–399.
Narayanaswamy, R. Grover, V. and Henry, R. M. (2013). The Impact of Influence Tactics in Information
System Development Projects: A Control-Loss Perspective, Journal of Management Information Systems,
30(1), 191-226
Page 287 of 307
Newcomb, P. (2007). Think Outside the COTS. The Software Revolution, Inc., retrieved June 27th 2014
from http://www.tsri.com/files/Think%20Outside%20the%20COTS.pdf
Nidumolu, S.R. and Subramani, M.R. (2003). The Matrix of Control: Combining process and structure
approaches to managing software development, Journal of Management Information Systems 20(3), 159–
196.
Nutt, P. (2002). Why decisions fail: avoiding the blunders and traps that lead to debacles, Berrett-Koehler
Publishers
Oberndorf, P., Brownsword, L. and Sledge, C. (2000). An Activity Framework for COTS-Based Systems.
Pittsburgh, PA: Software Engineering Institute, Carnegie Mellon University, 2000.
Oberndorf, T., Brownsword, L., Morris, E. J. and Sledge, C. A (1997). Workshop on COTS-Based
Systems, Software Engineering Institute, Carnegie Mellon University, retrieved June 10, 2014 from
http://resources.sei.cmu.edu/library/asset-view.cfm?assetid=12791
Ouchi, W.G. (1978) The transmission of control through organizational hierarchy. Academy of
Management Journal, 21, 173–192.
Ouchi, W.G. (1979) A conceptual framework for the design of organizational control mechanisms.
Management Science, 25, 833–848
Pan, S. L., and Tan, B. (2011). Demystifying Case Research: A Structured-Pragmatic-Situational (SPS)
Approach to Conducting Case Studies, Information and Organization, 21(3), 161-176.
Petter, S., DeLone, W., & McLean, E. R. (2013). Information systems success: the quest for the
independent variables. Journal of Management Information Systems, 29(4), 7–62.
Péraire, C. and Pannone, R. (2005). The IBM Rational Unified Process for COTS-based projects: An
introduction. IBM, Retrieved May 20, 2014 from
https://www.ibm.com/developerworks/rational/library/aug05/peraire-pannone/
Popp, M. and Dallis, D. (2012). Planning and Implementing Resource Discovery Tools in Academic
Libraries, IGI Global
Pesqueux, Y. and Damak-Ayadi, S. (2005). Stakeholder theory in perspective, Corporate
Governance, 5 (2), 5-22.
Pouloudi, A., and Whitley, E. A. (1997). Stakeholder Identification in Interorganizational Systems:
Gaining Insights for Drug use Management Systems. European Journal of Information Systems, 6(1), 1-
14.
Post, J. E. and Andrew, P. N. (1982). Case research in corporation and society studies. In L. E. Preston
(Ed.) Research in Corporate Social Performance Policy: A Research Annual, (pp 1-34), Greenwich, CT:
Jai Press Inc.
Pushor, D. (2007). Parent Engagement: Creating a Shared World. Invited Research Paper-Ontario
Education Research Symposium
Page 288 of 307
RapidBI (2007). Deep dive brainstorming technique, an innovative process for organizational
development, Retrieved on June 17, 2014 from
http://rapidbi.com/deepdivebrainstormingorganizationaldevelopment/
Rockart, J. F. (1979). Chief executives define their own data needs. Harvard Business Review, (March-
April).
Rockart, J. and De Long, D. (1988) Executive Support Systems, Dow Jones-Irwin, Homewood, Illinois.
Robertson, S. and Robertson, J. (2012). Mastering the Requirements Process: Getting Requirements
Right, Third Edition, Addison-Wesley Professional.
Runge, D. (1985). Telecommunications for Competitive Advantage unpublished PhD thesis, Templeton
College, Oxford.
Scholl, H. J. (2001). Applying Stakeholder Theory to e-Government: Benefits and Limits. Proceedings of
the 1st IFIP Conference on E-Commerce, E-Business, and E-Government (I3E 2001), Zurich,
Switzerland.
Scott, J. and Kaindl, L. (2000). Enhancing Functionality in an Enterprise Software Package, Information
and management, 37, 111-122
Shepherd, J. (2001). Enterprise Business Systems Must Be Upgraded and Maintained, AMR Research -
Executive View, (http://www.amrresearch.com/ exv/default.asp?i=57
Sykes, J.B. (Ed.). (1976). The concise Oxford English dictionary (6th
ed.). London: Oxford
University Press.
Software Development Lifecycle Phases (n.d.), University of Maryland. Retrieved July 3rd 2014 from
http://doit.maryland.gov/SDLC/COTS/Pages/Phase01Single.aspx
Stavru, S. (2014). A critical examination of recent industrial surveys on agile method usage; The Journal
of Systems and Software, 94, 87–97
Scheer, A-W. and Habermann, F. (2000), “Making ERP a success”, Association for Computing
Machinery. Communications of the ACM, 43, 57.
Sia, S.K. & Neo, B.S. (1997) Reengineering effectiveness and the redesign of organizational control:
a case study of the inland revenue authority of Singapore. Journal of Management Information
Systems, 14, 69–92.
Soh, C., Kien, S.S. and Tay-Yap, J. (2000), “Cultural fits and misfits: is ERP a universal solution?”,
Association for Computing Machinery. Communications of the ACM, 43, 47.
Somers, T.M. and Nelson, K.G. (2004), “A taxonomy of players and activities across the ERP
project life cycle”, Information & Management, 41, 257-78.
Page 289 of 307
Sun, A., Y., T., Yazdani, A., and Overend, J. D. (2005). Achievement assessment for enterprise resource
planning (ERP) system implementations based on critical success factors (CSFs). Int. J. Production
Economics, 98, 189–203
Torchiano, M. and Morisio, M. (2004). Overlooked aspects of COTS-based development, IEEE software
21 (2), 88-93
T. Ellis (1995). COTS Integration in Software Solutions – A Cost Model. Systems Engineering in the
Global Marketplace, NCOSE Symposium, St. Louis, MO
Tiwana, A., and Keil, M. (2009). “Control in Internal and Outsourced Software Projects,” Journal of
Management Information Systems, 26(3), 9-44.
Tarhini, A., Ammar, H., Tarhini, T. and Masa’deh, R. (2015). Analysis of the Critical Success Factors for
Enterprise Resource Planning Implementation from Stakeholders’ Perspective: A Systematic Review,
International Business Research, 8 (4)
Umble, E.J., Haft, R.R. and Umble, M.M. (2003), “Enterprise resource planning: implementation
procedures and critical success factors”, European Journal of Operational Research, 146, 241-57.
Urquhart, C. (2007). The Evolving Nature of Grounded Theory Method: The Case of the Information
Systems Discipline, in The Sage Handbook of Grounded Theory, A. Bryant and K. Charmaz (eds.),
London: Sage Publications, pp. 339-359.
Urquhart, C., Lehmann, H., and Myers, M. (2010). Putting the ‘Theory’ Back into Grounded Theory:
Guidelines for Grounded Theory Studies in Information Systems, Information Systems Journal, 20 (4),
357-381
Uçar, E and Bilgen, S. (2013). A Case-Based Model for Assessing the Effectiveness of Information
Systems Outsourcing, Journal of Information Technology Case and Application Research, 15 (3).
Van de Ven, A. H. (2007). Engaged Scholarship: A Guide for Organizational and Social Research, New
York: Oxford University Press
Verbeke, A. & Tung, V. (2013). The Future of Stakeholder Management Theory: A Temporal
Perspective. J Bus Ethics, 112:529–543
Ward, J. and Daniel, E. (2002). Benefits Management: How to Increase the Business Value of Your IT
Projects, John Wiley & Sons.
Wiegers, Karl E. (1996). Creating a Software Engineering Culture. New York: Dorset House Publishing.
Wiegers, Karl E. (2007). Practical Project Initiation: A Handbook with Tools. Microsoft Press
Weill, P., & Ross, J. W. (2004). IT Governance: How Top Performers Manage IT Decision Rights for
Superior Results (p. 269). Boston, Massachusetts: Harvard Business School Press.
Yin, R.K. (1981). The Case Study Crisis: Some Answers, Administrative Science Quarterly, (26) 1, 58-
65.
Page 290 of 307
Yin, R.K. (1984). Case study research: Design and methods, Newbury Park, CA: Sage.
Yin, R. K. (1994). Case study research: Design and methods (2nd ed.), Newbury Park, CA: Sage
Publications.
Yin, R.K. (2003). Case study research: Design and methods (3rd ed.), Thousand Oaks CA: Sage.
Zivkovic, J. (2012). Strengths and Weaknesses of Business Research Methodologies: Two Disparate Case
Studies, Business Studies Journal, (4) 2, 91-99.
Page 291 of 307
Appendix A – Research Approval
1. Ethics Committee Approval
Office of Research Ethics and Compliance
5110 Human Computer Interaction Bldg | 1125 Colonel By Drive
| Ottawa, Ontario K1S 5B6
613-520-2600 Ext: 2517
CERTIFICATION OF INSTITUTIONAL ETHICS CLEARANCE
The Carleton University Research Ethics Board-A (CUREB-A) has granted ethics clearance for the research
project described below and research may now proceed. CUREB-A is constituted and operates in compliance
with the Tri-Council Policy Statement: Ethical Conduct for Research Involving Humans (TCPS2).
Ethics Protocol Clearance ID: Project # 107704
Project Team Members: Mr. Zafor Ahmed (Primary Investigator)
Vinod Kumar (Research Supervisor)
Project Title: Enterprise COTS Implementation: Process, Stakeholder, and Control Perspective [Zafor Ahmed]
Funding Source (If applicable):
Effective: November 13, 2017 Expires: November 30, 2018.
Restrictions:
This certification is subject to the following conditions:
1. Clearance is granted only for the research and purposes described in the application. 2. Any modification to the approved research must be submitted to CUREB-A via a Change to
Protocol Form. All changes must be cleared prior to the continuance of the research. 3. An Annual Status Report for the renewal of ethics clearance must be submitted and cleared by the
renewal date listed above. Failure to submit the Annual Status Report will result in the closure of the file.If funding is associated, funds will be frozen.
4. A closure request must be sent to CUREB-A when the research is complete or terminated. 5. Should any participant suffer adversely from their participation in the project you are required to
report the matter to CUREB-A. Failure to conduct the research in accordance with the principles of the Tri-Council Policy Statement: Ethical
Conduct for Research Involving Humans 2nd edition and the Carleton University Policies and Procedures for
the Ethical Conduct of Research may result in the suspension or termination of the research project.
Page 292 of 307
Upon reasonable request, it is the policy of CUREB, for cleared protocols, to release the name of the PI, the title of the project, and the date of clearance and any renewal(s).
Please contact the Research Compliance Coordinators, at [email protected], if you have any questions or
require a clearance certificate with a signature.
CLEARED BY: Date: November 13, 2017
Andy Adler, PhD, Chair, CUREB-A
Bernadette Campbell, PhD, Vice-Chair, CUREB-A
2. Organizational Approval
No included in order to protect anonymity
Appendix B - Interview Guide
1. Email to Potential Interview Candidates
Subject: Invitation to participate in a research project on (Enterprise COTS Implementation:
Process, Stakeholder, and Control Perspective)
Dear <Name>,
My name is Zafor Ahmed and I am a doctoral student in the Sprott School of Business at Carleton
University. I am working on a research project under the supervision of Prof. Vinod Kumar and
Prof. Uma Kumar.
I am writing to you today to invite you to participate in a study entitled “Enterprise COTS
Implementation: Process, Stakeholder, and Control Perspective”. This study aims to investigate
four major projects implemented by the Bank of Canada from a stakeholder engagement and
control perspective.
This study involves one 60 minute interview that will take place in a mutually convenient, safe
location. With your consent, interviews will be audio-recorded. Once the recording has been
transcribed, the audio-recording will be destroyed. Additionally, care will be taken to protect your
identity at all time. This will be done by keeping all responses anonymous and allowing you to
request that certain responses not be included in the final project.
You will have the right to end your participation in the study at any time, for any reason, up until
Page 293 of 307
December 31st 2017. If you choose to withdraw, all the information you have provided will be
destroyed.
All research data, including audio-recordings and any notes will be encrypted. Any hard copies of
data (including any handwritten notes or USB keys) will be kept in a locked cabinet at the Bank
of Canada. Research data will only be accessible by the researcher and the research supervisor.
The ethics protocol for this project was reviewed by the Carleton University Research Ethics
Board, which provided clearance to carry out the research. (Clearance No. 107704)
CUREB-A:
If you have any ethical concerns with the study, please contact Dr. Andy Adler, Chair, Carleton
University Research Ethics Board-A (by phone at 613-520-2600 ext. 2517 or via email at
If you would like to participate in this research project, or have any questions, please contact me
Sincerely,
Zafor Ahmed, PMP, Ph.D. Candidate
Sprott School of Business
Carleton University
2. Informed Consent
LETTER OF INFORMED CONSENT: In-depth Interview
Title of research project: Enterprise COTS Implementation: Process, Stakeholder, and Control
Perspective
Date of ethics clearance: TBD
Page 294 of 307
Ethics Clearance for the Collection of Data: Expires: TBD
I understand that I have been invited to participate in a research study being carried out by Zafor
Ahmed, PhD Student, under the supervision of Prof. Uma Kumar, Sprott School of Business,
Carleton University and Prof. Vinod Kumar, Sprott School of Business, Carleton University.
I understand that the primary objective of this research study is to investigate the factors and
characteristics related to a successful enterprise COTS implementation in public sector. Also,
while participating in this research may not benefit me directly, I understand that the study will
advance my knowledge about the enterprise COTS implementation, Stakeholder engagement and
Control during COTS implementation in the Public Sector in general.
I understand that I am free to not answer questions, and to conclude the interview at any time. I
also understand that the interview will be audio recorded and may last up to 60 minutes.
I understand that I may withdraw from the study until December 31st 2017. Should I decide to
withdraw, the information I have provided will be destroyed and not used in the analysis or report
writing.
I understand that there may be some professional risk if my responses are critical of my employer.
To mitigate this risk, the researcher will avoid asking any question that will make me speak
negatively about my employer. I will retain anonymity in the final research paper and none of my
responses will be attributed to me. There will be no personally identifying pieces of information
collected for the purposes of this research and my department or agency will not be identified by
name or specific description.
I understand that the audio recordings will be stored electronically on an encrypted USB in a locked
drawer. These recordings will be transcribed and stored under a password-protected folder on a
secure computer. Recordings and transcriptions will be kept under lock and key in researcher’s
office. It will be ensured that the name of the respondent or the department or agency is not in the
transcription. The data will be used in dissemination of various academic journal papers and
conferences that come out of this research project. The recordings will be destroyed after
transcription, and transcriptions will be destroyed within one year of the completion of all data
collection. The data will be used only for this study.
This research project has been reviewed and has received ethics clearance by the Carleton
University Research Ethics Board. If you have any ethical concerns with the study, please contact
Dr. Andy Adler, Chair, Carleton University Research Ethics Board-A (by phone at 613-520-2600
ext. 2517 or via email at [email protected])
I have read and understand the above, and I have had my questions and concerns about the research
addressed: Yes No.
I agree for the interview to be audio-recorded: Yes No.
Page 295 of 307
Signature of Participant
Date Signed
Dr. Uma Kumar, Professor
Sprott School of Business Carleton University 1125 Colonel By Drive Ottawa, Ontario, K1S 5B6 [email protected] (613) 520-6601
Dr. Vinod Kumar, Professor Sprott School of Business Carleton University 1125 Colonel By Drive Ottawa, Ontario, K1S 5B6 [email protected] (613) 520-2379
Zafor Ahmed PhD Student Sprott School of Business Carleton University 1125 Colonel By Drive Ottawa, Ontario, K1S 5B6 [email protected] (447-5852613)
3. Interview Protocol
Appendix C – Informant List and Details
1. Case A Informant list:
Table 49: Case A Informant List
No Informant ID Role/Title Affiliation Member
Check
1 A01PT Senior Project Manager Host Organization/ Core Yes
2 A02PT Project Technical Lead Host Organization/ Core No
3 A03PT COTS Developer Host Organization/ Core No
4 A04PT Business Analyst Host Organization/ Core No
5 A05VR Project Manager COTS Vendor Yes
6 A06VR COTS Specialist/Consultant COTS Vendor No
7 A07BU Business Lead Host Organization/Business No
Page 296 of 307
8 A08BU Data Architect Host Organization/Business No
9 A09BU Business Analyst Host Organization/Business Yes
10 A10IT Solution Architect Host Organization/ ITS No
11 A11IT Enterprise Architect Host Organization/ ITS No
Total Number of Informants: 11
2. Case B Informant list:
Table 50: Case B Informant List
No Informant ID Role/Title Affiliation Member
Check
1 B01PT Senior Project Manager Host Organization/ Core Yes
2 B02PT Project Technical Lead Host Organization/ Core No
3 B03PT COTS Developer Host Organization/ Core No
4 B04PT Business Analyst Host Organization/ Core No
5 B05VR Senior Project Manager COTS Vendor Yes
6 B06VR COTS Specialist/Consultant COTS Vendor No
7 B07VR Solution Architect COTS Vendor / Integrator No
8 B08VR COTS Developer COTS Vendor / Integrator No
9 B09BU Business Lead Host Organization/Business Yes
10 B10BU Data Architect Host Organization/Business No
11 B11IT Enterprise Architect Host Organization/ ITS No
12 B12IT Infrastructure Support/Server Host Organization/ ITS No
Total Number of Informants: 12
3. Case C Informant list:
Table 51: Case C Informant List
No Informant ID Role/Title Affiliation Member
Check
1 C01PT Senior Project Manager Host Organization/ Core Yes
2 C02PT Project Technical Lead Host Organization/ Core No
3 C03PT COTS Developer Host Organization/ Core No
4 C04PT Business Analyst Host Organization/ Core No
5 C05VR Data Migration Specialist COTS Vendor Yes
6 C06VR COTS Specialist/Consultant COTS Vendor No
7 C07BU Business Lead Host Organization/Business No
8 C08BU Business Analyst Host Organization/Business Yes
9 C09IT Solution Architect Host Organization/ ITS No
Total Number of Informants: 09
Page 297 of 307
4. Case D Informant list:
Table 52: Case D Informant List
No Informant ID Role/Title Affiliation Member
Check
1 D01PT Senior Project Manager Host Organization/ Core No
2 D02PT Project Technical Lead Host Organization/ Core Yes
3 D03PT Senior COTS Developer Host Organization/ Core No
4 D04PT Business Analyst Host Organization/ Core No
5 D05VR SharePoint Specialist COTS Vendor Yes
6 D06VR COTS Specialist/Consultant COTS Vendor No
7 D07BU Business Authority Host Organization/Business Yes
8 D08BU Business User Host Organization/Business Yes
9 D09IT Enterprise Solution Architect Host Organization/ ITS No
Total Number of Informants: 09
Page 298 of 307
Appendix D – Strategic Investment Gating Process Gate 1 Approval Process
Page 299 of 307
Gate 2 Approval Process
Page 300 of 307
Gate 3 Approval
Page 301 of 307
Gate 4 Approval Process
Page 302 of 307
Appendix E
Table 53: Informants Summary for All Cases
Type of Primary Data Description of the Informants Roles
Face to Face Semi-
structured Interview
Duration for each
Interview: 80 minutes
Role of Interviewee # of Interviews
with Project team
# of interviews
with vendor and
solution integrator
# of interviews
for Client/
Business Units
Project Manager 4 2
Business Lead 4
Business Analyst 4 1 2
Enterprise Architect 2
Data Architect 1 2
Solution Architect 2 2
Project team
member
9 5 1
21 11 9
Total # of
interviews
41
Appendix F: Acronym Table
Table 54: Acronyms
Acronyms Meaning
AOC Agency Operations Centers
BRD Business Requirement Document
COTS Commercial Off The Shelf
CSF Critical success factors
CRM Customer Relationship Management
ECM Enterprise Content Management
ERP Enterprise Resource Planning
ES Enterprise systems
EIS Enterprise Information Systems
FDD feature-driven development
FMO Financial Market Operations
FRS Financial Regulatory System
FI Financial Institutions
FBSR Financial Billing Systems Replacement
GUI Graphic user interfaces
Page 303 of 307
IS Information Systems
SLA Service level agreement
TCD Technical Conceptual Design
PMO Project Management Office
RUP Rational Unified Process
RTM Requirement Traceability Matrix
SRS System Requirements Specifications
SUC System Use Cases
UAT User Acceptance Testing
XP eXtreme programming
Appendix G: Data Collection Instrument
Project Name : Participant Code :
Participant association : Host / Partner/ Vendor / Solution Integrator
Date of interview :
Location :
1. Did you have this phase in this Project? Were you involved?
_____ Yes ______No
2. Which other phase does this phase relate to in terms of information flow (other than
immediate next phase)? i.e. Information generated on this phase results in immediate
actions on what other phases
3. What are the Key Activities performed during this Phase?
4. What are the key processes used during this phase?
Page 304 of 307
Not Significant =1 ….2…..3…..4…….5 = Very Significant
Initia
tion
Re
qu
irem
en
t &
Pla
nn
ing
An
aly
sis &
Co
nce
ptu
al
De
sign
D
esig
n &
A
rchite
cture
De
ve
lop
me
nt
De
live
ry
Clo
se-O
ut
Was this phase present in this project?
How signification this phase was in terms to activities?
Contribution level By:
VIZOR
OSFI/CDIC
Bank of Canada
DELOITTE
General Implementation Related Questions
5. What defined the success for this project? [Success factors]
Time, Budget, Scope, Quality, And Other Factors: ------------------------------------------
6. Which part was most critical for Success? And Why ?
Pre-development / Development / Delivery
7. Which part was most High Risk? And Why ?
Pre-development / Development / Delivery
8. Overall challenges for this project?
Page 305 of 307
9. What would you do differently if you do the same project again?
10.A. If this project were to fail, what are the reasons you can think of?
10.B. And how/what phase this should have been ideally detected to minimize the damage?
11. How did vendor's agile approach affect the PM process in the Bank? What is positive and
what is negative about it?
12. Something(s) that contributed to your personal experience and will be beneficial for future projects involvements, something that surprised you etc.
13. Some elements that are valuable for Tri-agency but not captured through standard project management templates (could be a process, could practices, standards or anything that you can think of)
14. Any generalized model that appears to be a good standard for future projects of this scale and
context for the Tri-Agency or Organizations of similar size/environment.
15. What do you recommend for an organization like us (the BANK) can do enhance our chances of
being successful with this challenge (mentioned on #18) thus enhancing the chance of overall
project success with other similar COTS products?
16. Do you see any other opportunity for the Tri-Agency to make things better in the future? like
creating more certain kind of roles / different project management style or changing any
internal process or different technical unit to handle COTS based projects like this ?
17. Project Cost was higher than average ERP implementation or $10 to $15 million, what are your
thoughts on that and is there any way to reduce it for similar projects?
18. What were the major in-efficiencies for this project?
19. General discussion around RFP, Requirement, Design and Development phases
20. General Discussion around Migration and Metadata Conversion activities
21. Project Governance discussion – How it was structure and why?
Page 306 of 307
Control Related
Low (L=1 to 3) Moderate (M=4 to 8) High (H=over 8)
1.
Initia
tion
2.
Re
qu
irem
en
t &
Pla
nn
ing
3.
An
aly
sis &
Co
nce
ptu
al
De
sign
4
. D
esig
n &
A
rchite
cture
5.
De
ve
lop
me
nt
6.
De
live
ry
7.
Clo
se-O
ut
What was the control configuration on this phase?
Was the control effective (Y/N)?
Control Trigger events
Level of meetings (per week)
Status requests and deadlines
Joint activities among partners
Level of missed deadlines or expected deliverables?
Level of escalations due to issues
Procedural Control = P Trust Based Control= T Coordinated control=C
22. Why control configuration was changed from phase 1 to phase 2?
23. Was this change produced desired results?
24. Why control configuration was changed from phase 2 to phase 3?
25. Was this change produced desired results?
26. Why control configuration was changed from phase 3 to phase 4?
27. Was this change produced desired results?
28. Why control configuration was changed from phase 4 to phase 5?
29. Was this change produced desired results?
30. Why control configuration was changed from phase 5 to phase 6?
31. Was this change produced desired results?
32. Why control configuration was changed from phase 6 to phase 7?
Page 307 of 307
33. Was this change produced desired results?
Stakeholder Related
Low (Less than 30% stakeholders were involved)
Moderate (50% stakeholders were involved)
High (Over 70% stakeholders were involved)
Initia
tion
Re
qu
irem
en
t &
Pla
nn
ing
An
aly
sis &
Co
nce
ptu
al
De
sign
D
esig
n &
A
rchite
cture
De
ve
lop
me
nt
De
live
ry
Clo
se-O
ut
What was the level of stakeholder involvement at this phase? (L/M/H)
Stakeholder Influence
Host
Partners
Vendor
Solution Integrators
External/public
Clients
34. General discussion about stakeholder orientation and reasons for such orientation