a study of building an information systems project risk checklist...

A Study of Building an Information Systems Project Risk Checklist by Analyzing Critical

Factors of Information Systems Failures

A study submitted in partial fulfillment of the requirements for the degree of Master of Science in Information Management

at

THE UNIVERSITY OF SHEFFIELD

By

Lihong Zhou

September 2006

- 1 -

Abstract

Information systems within organizations were rapidly developed and widely used in

diverse industries in recent decades. However, currently, the astonishing rate of

information system failures becomes a controversial issue and a pertinent research

topic. Although a large amount of studies have contributed to this topic, there is also

a large number of information systems are not as successful as expected. The main

purpose of this dissertation is to create an Information Systems Project Risk

Checklist as a convenient elementary risk identification device for information

systems developers in avoiding and mitigating potential project risks in order to keep

information systems project away from failure.

This dissertation research is conducted by using an inductive research methodology

with secondary data source and qualitative data analysis approach. Both conceptual

and empirical study are carried out in this study by literature review and case study.

After Introduction and Research Methodology chapters, the extensive literature

review, on Information System Implementation, Nature of Failure, Project

Management and Risk Management, buildup a research theory background. The

initial version of Information Systems Project Risk Checklist is generated based on

this theory foundation.

The formal version of Information Systems Project Risk Checklist is presented after

2 case studies (Integrated National Crime Information System Project (INCIS) and

London Ambulance Service (LAS)). The formal version risk checklist is significant

revised from the initial version by applying the risk checklist into real life

environments.

- 2 -

Acknowledgement

I would like to give my extreme appreciation to my supervisor Ms Ana Cristina

Vasconcelos for her encouragement, guidance and assistance throughout the entire

project. Without her support the dissertation could not have been completed.

- 3 -

Contents

Abstract....................................................................................................................... 1 Acknowledgement ...................................................................................................... 2 Contents ...................................................................................................................... 3 1 Introduction ............................................................................................................. 6

1.1 Current Study ....................................................................................................................... 7 1.2 Research Motivation and Objectives.................................................................................. 10 1.3 What is Checklist ?............................................................................................................. 11 1.4 Why Study Failures? .......................................................................................................... 11

2 Research Methodology.......................................................................................... 13 2.1 Literature Review............................................................................................................... 14 2.2 Case Studies ....................................................................................................................... 14 2.3 Research Approach............................................................................................................. 16 2.4 Data Collection and Analysis ............................................................................................. 18 2.5 Dissertation Outline and Structure Map............................................................................. 20

3 Information Systems Implementation................................................................. 22 3.1 Definition of Information System ...................................................................................... 22 3.2 Nature of Information Systems .......................................................................................... 23 3.3 Importance of Information Systems ................................................................................... 26 3.4 Information Systems in Organizations ............................................................................... 27 3.5 Information Systems Development Methodologies ........................................................... 30

4 Nature of Failure ................................................................................................... 33 4.1 Definition of Failure........................................................................................................... 33 4.2 Types of Failure.................................................................................................................. 34 4.3 Information Systems Failures............................................................................................. 34

5 Project Management............................................................................................. 37 5.1 Managing Information Systems Projects ........................................................................... 38 5.2 System planning and estimating......................................................................................... 40 5.3 System Design, Development and Implementation ........................................................... 43 5.4 System Implementation, Testing and Maintenance............................................................ 45

6 Risk Management ................................................................................................. 48 6.1 Risk Management Process ................................................................................................. 50 6.2 Risk Identification.............................................................................................................. 51 6.3 Risk Assessment and Management .................................................................................... 52

7 Information Systems Project Risk Checklist ...................................................... 55 7.1 Literature Review Based Checklist Building ..................................................................... 55 7.2 Pre-Project.......................................................................................................................... 57 7.3 Customer ............................................................................................................................ 58 7.4 Project Management........................................................................................................... 59 7.5 Project Technical Aspect .................................................................................................... 60 7.6 System Development ......................................................................................................... 61

- 4 -

8 Case Study 1: Integrated National Crime Information System (INCIS)......... 63 8.1 Case Background Introduction........................................................................................... 63 8.2 Problem Domains Description ........................................................................................... 64

8.2.1 Pre-project ................................................................................................................... 64 8.2.2 Customer ..................................................................................................................... 68 8.2.3 Project Management.................................................................................................... 70 8.2.4 Project Technical Aspect ............................................................................................. 72 8.2.5 Project Development ................................................................................................... 75

8.3 Conclusion........................................................................................................................... 76 9 Case Study 2: London Ambulance Service ......................................................... 77

9.1 Case Background Introduction........................................................................................... 77 9.2 Problem Domains Description ........................................................................................... 78

9.2.1 Pre-Project................................................................................................................... 78 9.2.2 Customer ..................................................................................................................... 79 9.2.3 Project Management.................................................................................................... 82 9.2.4 Project Technical Aspect ............................................................................................. 85 9.2.5 Project Development ................................................................................................... 87

9.3 Conclusion.......................................................................................................................... 89 10 Formal Information Systems Project Risk Checklist ...................................... 92

10.1 Pre-Project........................................................................................................................ 92 10.2 Customer .......................................................................................................................... 94 10.3 Project Management......................................................................................................... 96 10.4 Project Technical Aspect .................................................................................................. 98 10.5 Project Development ........................................................................................................ 99

11 Conclusion and Future Research..................................................................... 101 11.1 Research Objectives ....................................................................................................... 101 11.2 Research Processes......................................................................................................... 102 11.3 Research Methodology................................................................................................... 103 11.4 Research Result .............................................................................................................. 104 11.5 Research Contribution and Future Research .................................................................. 105

Reference................................................................................................................. 108 Appendix I .............................................................................................................. 117 Appendix II ............................................................................................................. 121

- 5 -

List of Figures

Figure 1: Sauer’s Triangle of Dependences ................................................................. 9

Figure 2: Research Processes ..................................................................................... 13

Figure 3: Deductive Reasoning.................................................................................. 16

Figure 4: Inductive Reasoning ................................................................................... 17

Figure 5: Quantitative VS Qualitative From (Saunders, 2000:380) .......................... 19

Figure 6:Dissertation Structure Map.......................................................................... 21

Figure 7: Information Systems (Developed from Robson, 1997).............................. 22

Figure 8: A model of system (Curtis and Cobham, 1995:15) ................................... 23

Figure 9: General of Information System (Robson, 1997:84) ................................... 24

Figure 10: Model of Computerized Information System (O’Brien, 2006) ................ 25

Figure 11: Duality of Structure (Beynon-Davies, 2002: 220).................................... 27

Figure 12: The elements and context of a computer-based information system........ 28

Figure 13: Failure Types (Fortune &Peters, 1995: 23) .............................................. 34

Figure 14: Determinants of Information systems Success/failure ............................. 35

Figure 15: An example of Gantt chart from University of Sheffield (2000),Page 2................ 42

Figure 16: The Business Excellence Model of EFQM (Yeates and Cadle, 1996:171) ............ 43

Figure 17: The Risk Management Process (Yeates and Cadle, 1996: 190) ............... 50

Figure 18: Risk assessment process ........................................................................... 50

Figure 19: Risk Impact Scale (Developed from Yeates & Cadle (1996)).................. 53

Figure 20: Risk Likelihood Scale (Developed from Yeates & Cadle (1996)) ........... 53

Figure 21: Diagram of revisions on Information Systems Project Risk Checklist .... 56

Figure 22: Relations of 5 components ....................................................................... 56

Figure 23: Compare INCIS and LAS......................................................................... 90

Figure 24: Diagram of revisions on Information Systems Project Risk Checklist .. 101

Figure 25: Dissertation Structure Map..................................................................... 102

Introduction Lihong Zhou

- 6 -

Chapter 1

Introduction

Ever since the emerging of computer technology and Internet, the world has been

changed substantially. As the continuing evolvement of Internet and the combination

of computer technology and computer network, computers are playing an extremely

important role in people’s everyday work and life. An estimation pointed out by

GobalReach research agency, states that there were upwards of 700 million Internet

users during the year 2004-2005 (Hurst, 2006). Currently, computers in business

world are no longer just information storage and calculation tools, but also powerful

strategic tools, for instance, in the fields of communication, decision support, and

high level management tools. Galliers (1992) indicates computer based information

systems can support all organizations function effectively. Flowers (1996) also claim

that information systems are at the heart of all modern organizations. As the vital

importance of an effective information system within organizations, organizations

pay great amount of money and efforts on the implementation of information

systems.

Since 1960s, a number of information systems have been developed in both private

sectors and public organizations. In the most recent decades, the number has been

dramatically increased. However, the high rate of information system failure is a

pertinent topic. Recent decades of studies on failures of information system

development haven’t substantially decreased the failure rate. Connolly, Begg, and

Strachan (1996) indicates that only in 1996, about 30%-40% of IS development

projects failed or totally abandoned. Figures from survey conducted by Robbins

Gioia in the year of 2001, which covers industries of government, information

technology, communications, financial, utilities and healthcare, pointed out that 51%

ERP systems were viewed as unsuccessful; 46% ERP system were not working


- 7 -

properly as they designed to be (Aiken, 2002). Forester and Morrison (1994) shows,

a study from U.S. government’s General Accounting Office (GAO), 47% of projects

were delivered but not used, 29% were paid for but not delivered, and 19% were

abandoned or reworked. Martin Cobb pointed out a paradox of “we know why

projects fail, we know how to prevent their failure, so why do they still fail?” (The

Standish Group, Unfinished Voyages, 2006).

A possible reason of failures on information system is unsuccessfully identify and

resolve underlying risks within the system development. A formal and complete risk

management plan and a delicate project management can significantly save the

information system project from the failure. Yeates and Cadle (1996) claims that

before actual system developing, if it is not possible to eliminate potential risks

altogether, it is necessary to recognize the existing risks and make out a plan of

method of dealing with these risks when they actually take place.

1.1 Current Study

Possible risks and causations of information system failure are listed by various

articles. Sumner (1999) lists 12 distinct kinds of failures of information system

implementation: “resource failure; requirement failure; goal failure; technique failure;

user contact failure; organizational failure; technology failure; size failure; people

management failure; methodology failure; planing and contact failure; personality

failure”.

Kasser (1998) exhibits 7 most priorities out of 33 items of risk indicators during

system developing and implementation: poor requirements; poor communications

with customer; lack of, or, poor plans; lack of process and standards; lack of

management support; failure to validate original specification and requirement;

political consideration outweigh technical factors.


- 8 -

Drori (1997) not only divides an information system implementing project into 5

stages, but also points out negative factors for each stage. Stages Negative factors

System analyzing and requirement defining

1, incomplete comprehension of the system and the current situation of the organization; 2, only focusing on the drawbacks of current system may cause ignoring and excluding new features of the new system; 3, Partially understand the user requirements; 4, User requirements are formed only by management level withough the final system users;

System designing 1, inadequate initial general planing before detailed and complex designing; 2, too accrate designing causes the system extremely heavy and unhealthy; 3, without a prototype agreed by the customer; 4, poor communication and understanding among the system analyst, designer and the programmer;

System developing and acceptance testing

1, attempting produce a complete and perfect system in one stage; 2, initiate system programming when the system designing is not completed; 3, initiate system programming when the data structure of the system is not accurately defined; 4, accept testing, which is finished only finished by the programming team; 5, accept test result only by the management level without considering the final user of the customer organization; 6, programming without a preselected and unified methodology; 7, programming withou necessary documentations; 8, lack of considering reuseable components from the current system; 9, programmer-oriented system designing instead of user-oriented;

System training 1, swich to the new system all around the organization in the mean time; 2, one time training before system cut over; 3, training in a few condensed days; 4, initiate training without complete testing, including the treatment of occationally exceptional problems; 5, lack of user support in the working environment;

System maintainning 1, lack of a thorough maintenance plan; 2, assign unqualified staff for maintenance;


- 9 -

Nonetheless, Drori’s study only focus on identifying risks in 5 main phases of system

development. These risks are vitally critical, but they are inadequate for a complete

information project.

Sauer (1993) developed an information system failure model, which is named

Triangle of Dependencies. The model is consisted of 3 roles in information system

project: system, supporters and project organization.

(Sauer, 1993: 29)

Figure 1: Sauer’s Triangle of Dependences

Although Sauer’s model makes a positive contribution to comprehend the

information failure, it is not sufficient in identifying a specific risk or a critical factor.

These studies outstandingly exhibit potential risks during the developing of an

information project. However, their surveys are failrly incomplete and partial since

they mainly focus on most significant risks of information systems projects failures.

However, an information system project failure is caused by a series of inseparable

risks but not caused by a single significant risk. It is believable that a comprehensive

risk evaluation device such as a risk checklist can directly shows possible risks in

information systems project, is more practical and convinence for project developers

in planning, designing and implementing information systems.


- 10 -

1.2 Research Motivation and Objectives

The purpose of the thesis is to build up a complete risk checklist, which can identify

most common risks underlying in information system project, by taking literature

review and assessing 2 case studies of information systems failure in large public

organizations (Integrated National Crime Information System Project and London

Ambulance), discuss the differentiations of these two cases; refine and collect crucial

factors possibly induced the system failure;

The objectives of the dissertation can be highlighted as:

1. Research and evaluate the relevant current literature to establish the nature of

information system, system failure, project management and risk management.

2. Identify critical factors of information system failure from previous publications.

3. Analysis 2 case studies of Integrated National Crime Information System Project

and London Ambulance and identify underlying risks.

4. Formulate an Information Systems Project Risk Checklist which could be an

elementary tool for information systems developers in avoiding potential risks.

5. Suggest future enhancement and development based on this research.


- 11 -

1.3 What is Checklist ?

As the main purpose of this thesis is to generate an information system project risk

checklist by research on areas related to information systems implementation and

failures, it is necessary to clarify what is a checklist.

According to Stufflebeam (2000), checklists are valuable evaluation devices when

carefully developed, validated, and applied. A sound evaluation checklist clarifies the

criteria that at least should be considered when evaluating something in particular

area; aids the evaluator not to forget important criteria; and enhance the assessment is

objectively, credibility and guiding its operation; and assessing its out comes. In the

evaluation vernacular, checklists are useful for both formative and summative

evaluations.

1.4 Why Study Failures?

Failures are the pillars of successes. The intention of studying failure is to avoid the

certain failure causations and highly improve the success rate in the future by

reassessing the strategic plan and project processes from the past.

Fortune & Peters (1995) illustrates the importance of learning from failures by

exhibiting a statement of Ackoff (1994):

“When one does something right, one only confirms what is already known: how to

do it. A mistake is an indicator of a gap in one’s knowledge. Learning takes place

when a mistake is identified, its producers are identified and it is corrected”

(Fortune & Peters, 1995: 5)


- 12 -

Lyytinen and Robey (1999) stress on the importance of learning from failures of

information systems development. They mentioned that in order to avoid similar

failure causations in the upcoming project, it is greatly important to for information

systems developers to learn reasons of failures both from internal and external cases

and to effect changes in their own actions.

To conclude, the study of information system failures from the past provides a rich

source of lessons which are worth of investigating, learning and understanding.

Research Methodology Lihong Zhou

- 13 -

Chapter 2

Research Methodology

This research project employs an inductive qualitative research methodology. Mainly,

there are 2 approaches for this study, literature review and case studies.

Figure 2: Research Processes

This dissertation study starts from profound literature review, in order to construct a

research theory background for further empirical studies. More importantly, the

initial version of information system project risk checklist is built base on the

literature review. The purpose of these 2 case studies is to implement the initial risk

checklist to real life cases, and attempt to identify potential risks by using the risk

checklist. The main deliverable, a revised version of information system project risk

checklist, is presented after cases analyzing. The original risk checklist is

significantly revised by risks assessments in the 2 cases.

This chapter aims to exclusively explain the research instruments, research sources

and the overall research plan. At the beginning, 2.1 and 2.2, methodologies of

literature review and case study are discusses. Section 2.3 presents a research


- 14 -

methodology framework, in which substantially discusses inductive and deductive

methodologies. The Secondary data sources are also explained in this section with

qualitative and quantitative data analysis approaches section 2.4. Finally, section 2.5

introduces the structure of the dissertation and a dissertation structure map is

illustrated.

2.1 Literature Review

The literature review is a vital part of this dissertation as previously introduced.

Literature review not just reviews current literatures, but also creates a fundamental

knowledge platform for further cases analysis and research result support. Literature

review goes through basic concepts of information system implementation, nature of

failure, project management and risk management. An information system project

risk checklist is created at the end of literature review.

Based on Yin (1994) “Budding investigators think that the purpose of a literature

review is to determine the answer about what is known on a topic; in contrast,

experienced investigators review previous research to develop sharper and most

insightful questions about the topic” (Yin, 1994 quoted in Liu, 2002).

2.2 Case Studies

Case studies of Integrated National Crime Information System Project (INCIS) and

London Ambulance Service (LAS) are selected for this research.

The case of INCIS was took place in New Zealand. It concerned by public because

of it’s over ambitious strategic plan, high risk project profile and high project cost

(the estimated total cost of the project is NZ$110,000,000). According to the final


- 15 -

report, Small (2000), of the project, although the INCIS project was considered as a

high risk project, the project has the capability to use conventional technology with

enough time, appropriate governance and management, adequate resource, skill and

experience. However, the project was almost certainly failed. It achieved some of its

objectives but failed many of its main objectives.

London Ambulance is a well-known case of IS system failure. Not just the fact of

high estimated financial cost of £1.1–£1.5 million, but more importantly 20 people

may have died as the reason of system failure. Beynon-Davis (1999) claims that this

case is the most frequently quoted UK examples of information system failure.

These two cases are both well-known information system disasters with large scale

of project scopes, high project costs and high technical complexity. And they can

entirely represent large percent of common risks of information system implementing.

Results of these 2 cases are dramatic, but on the other hand, which can increase

public interests. Both of these 2 case studies are from Government Public Sectors.

Although there is a high profile nature of information system failure in private

sectors and companies, there has been very little academic research focused on them,

because of private companies unwilling to disclose details about their failures, as this

kind of information may detrimental to companies’ reputation. However, these kinds

of behaviors increase the possibility of repeat the mistakes for other private sectors.

Galliers (1992) indicates that the methodology of case study is the most popular

strategy in information system failure studies. This book defines 2 groups of

information systems research strategies. The first group is empirical strategy which

includes case studies, field studies, field tests, laboratory studies. Another group is

non-empirical, which is including conceptual, tutorial, review, and others. From the

research results of citation frequency study and published information system articles,

the case study methodology is the most common approach. Galliers (1992) outlines


- 16 -

the both of the bright side and dark side of this strategy. As the strongpoint, case

study captures greater details from the environment of reality. On the other hand,

case study strict to a single project or organization; hard to generalize identified

problems; lack of control of variables; the case can be interpreted distinctly by

different researchers.

2.3 Research Approach

Jarvinen (2000) defines a research approach as “a set of research methods that can be

applied to similar research methods that can be applied to the similar research

objects and research questions.” In order to conduct research appropriately, there are

2 main research theories (inductive reasoning and deductive reasoning) can be

applied according to different research circumstances. These 2 research strategy can

be seen as seeking the truth from opposite directions (Walliman, 2001).

Deductive reasoning starts from more general to more specific (Trochim, 2006).

Trochim (2006) introduces that the deductive reasoning informally named a “top-

down” approach, which thinking up a theory as the first step. Then narrow down to

more believable hypothesis. Narrow down further until feedback observation to

address the hypothesis. The final step, confirmation, is testing the hypothesis with

specific data.

Figure 3: Deductive Reasoning

(Trochim, 2006, from http://www.socialresearchmethods.net/kb/dedind.htm)


- 17 -

In contrast, inductive reasoning starts from more specific to more general, called

“bottom-up”. “Inductive research methodology begins with specific observations

and measures, begins to detect patterns and regularities, formulate some tentative

hypotheses, and finally ends up developing some general conclusions.” (Trochim,

2006)

Figure 4: Inductive Reasoning

(Trochim, 2006, from http://www.socialresearchmethods.net/kb/dedind.htm)

Nickerson (1986) states “effective reasoning requires the ability to develop

arguments and assess their validity to generate and test hypotheses, to judge the

plausibility of assertions, to identify possible courses of action, and to think through

consequences of choosing a particular course” (Nickerson , 1986, quoted in Watters

and English, 1995).

Liu (2002)’s research mentions inductive reasoning is more appropriate for studying

critical factors in information system project. Inductive reasoning starts from

problem statement and then generate conclusion from existing data which approach

uses current data to determine the key concept to be discuss. The main objective for

this dissertation is to generate a risk checklist for information system projects by

employing to research methodologies of literature review and case studies. In this

case, for this dissertation, it is better to use inductive reasoning as the primary

research approach.


- 18 -

2.4 Data Collection and Analysis

There two kinds of data for sociological research according to McNeill (1990).

Primary data is captured by researchers at first hand mainly by surveys, interviews

and observations. Secondary data is researchers collected from some available

sources. Although, secondary data source might collected from primary data source

which is collected for its original specific purpose, secondary data can provide a

useful source to answer research questions and accomplish the research objectives

(Saunders, 2000). Data for this dissertation is only collected from secondary data

sources, mainly publications, academic journals, electronic source and official case

reports. No direct primary data collection activities are involved in this study.

There are several advantages of using secondary data, pointed out by Saunders et al.

(2000). The data is the enormous saving in resources, in particular researchers’ time

and money; the secondary data can be quickly collected, in addition, they might

higher-quality data than could be obtained by collecting by your own; secondary data

provide the only possibility of undertaking longitudinal studies for those research

projects with time constraints; re-analyzing secondary data can lead to unexpected

new discoveries; finally, secondary data provides a permanent and available source

can be easily accessed.

In contrast, Saunders et al. (2000) also point disadvantages for secondary data. Firstly,

secondary data might be collected not match the research purpose. Secondary data

employed in this dissertation is closely related to the topic of information systems

implementation or information systems failure. Secondly, Saunders et al. (2000)

indicates that secondary data may be difficult or difficult to access. Researcher for

this dissertation can conveniently access the data resource from university libraries,

university academic databases and journals， and Internet connections. Additionally,

aggregations and definitions may be unsuitable. This thesis strictly selected

secondary data, and criticized the suitability for this dissertation research. Finally, the


- 19 -

initial purpose may affect the presentation of secondary data, especially for internal

organizational documents and external documents such as published company

reports and newspaper reports. This research project largely collects widely

published books, academic journals and official case reports, quit a few internal

organizational documents and newspaper reports are involved in the thesis.

There are 2 kinds of data for data analysis, quantitative data and qualitative data.

“Quantitative data refers to all such data and can be a product of all research

strategies. It can range from simple counts such as the frequency of occurrences to

more complex data (Saunders et al., 2000:326)”.

“Qualitative data are associated with such concepts and are characterized by their

richness and fullness based on your opportunity to explore a subject in as real a

manner as is possible (Robson, 1993, quoted in Saunders et al., 2000)”.

Differences between quantitative and qualitative data can be shown in the table

below: Quantitative Data Qualitative Data

Based on meanings derived from numbers Based on meanings expressed through words

Collections results in numerical and

standardized data

Collection results in non-standardized data

requiring classification into categories

Analysis conducted through the use of

diagrams and statistics

Analysis conducted through the use of

conceptualization

Figure 5: Quantitative VS Qualitative From (Saunders, 2000:380)

Research results from this dissertation are using qualitative data analysis

methodology from literature review and case studies.


- 20 -

2.5 Dissertation Outline and Structure Map

After the introducing research background and methodology (Chapter 1 and Chapter

2), Chapter 3 stresses on information system implementation. It starts with defining

what is information system and clarifying the importance of information system

within organizations. This part describes how and why organzations need

information systems, and lists out how information systems can benefit organizations.

It ends with information systems implementation methodologies which provides

suggestions on how to actual develop an information system. Definitions and the

nature of failure are discussed in chapter 4, which gives an conception of information

failure by study the nature of failure. Chapter 5 introduces information project

management step by step. All major processes of information project management

with potential significant risks are discusses thoroughly. Chapter 6 dedicated to risk

management. This chapter not only illustrates nature of risk, but also identifies

features of risks in information system implementations. A major part of this chapter,

analyze how to use techniques of risk management to address and eliminate potential

risks.

Chapter 7 is an important component in this thesis, which chapter generates an

Information Systems Project Risk Checklist. The risk checklist is intened to record

all significant risks within information system projects based on literature reviews in

previous chapters. Chapter 8 and Chapter 9 implement the risk checklist onto INCIS

and LAS case studies respectively. 2 cases are experimentally evaluated in the form

of risk checklist, on the other hand, the risk checklist is significantly revised through

the case assessment. Chapter 10 is the main research result. The formal version of

Information System Project Risk Checklist is introduced with sufficient discussions.

All these Chapters can be shown in the structure map below:


- 21 -

Figure 6:Dissertation Structure Map

This diagram can clearly illustrate the overall structure and processes in the thesis.

The research foundation and background is consisted of Chapter 3,4,5 and 6. The

main theory, Chapter 6 information system project risk checklist is introduced based

on the research foundation. Then, the risk checklist used as an analysis tool for

INCIS and LAS case studies in Chapter 8 and Chapter 9 respectively. Finally, and the

most importantly, research result, a revised version of information system project risk

checklist is announced in Chapter 10.

Information Systems Implementation Lihong Zhou

- 22 -

Chapter 3

Information Systems Implementation

Nowadays, information systems in organizations are playing a vitally central role.

Avision & Fitzgerald (2003) portrays that an information system can help an

organization operate more effectively and efficiently by providing useful processes

and information to its members and clients.

3.1 Definition of Information System

Information system can be comprehended on a number of distinct aspects.

Information system is an integrated, computer-based, user-machine system that

provides information for supporting operations and decision making functions (Awad,

1988). Sauer (1993) describes the information systems, particularly to computer-

based systems, as systems basically input and output information in order to

coordinate the work of many different organizational functions. Wolstenholme et al.

(1993) emphasize that information system has to present information and assist

information flow. Avison & Shah (1997) mentions an information system should not

only focus on the organization, but also the organizational environment. Information

technology (IT), Management information system (MIS), Decision support system

(DSS), Executive information systems (EIS), Strategic management information

system (SMIS) are considered as major components or functions for an information

system based on Robson (1997).

Figure 7: Information Systems (Developed from Robson, 1997)


- 23 -

3.2 Nature of Information Systems

The organizational information system, both computer-base or non-computer-based,

is built from an infrastructure. Based on Galliers (1992), the infrastructure groups the

organizational structure; communication channels (mails, telephones, faxes);

facilities (telephones, terminals, and computers); apparatus (PCs and servers);

software and necessary training.

Information systems are consisting of various components. Galliers (1992) explains

that in the real world, information systems are comprise from object, people, rules,

norms and commands. All systems, explained by Curtis and Cobham (1995), are

integrated from interrelated components with some purpose, goal and objective, and

designed processes within the system. The whole system processes can be showed as

the diagram below.

Figure 8: A model of system (Curtis and Cobham, 1995:15)

Robson (1997) develops a general conceptual model of information system based on

the model of system. The conceptual model simply exhibits several basic

components of an information system and map of information flow inside the system.


- 24 -

Figure 9: General of Information System (Robson, 1997:84)

Computer-based information systems, according to Stair and Reynolds (1999), are

composed of hardware, software, databases, communication media, people, and

procedures. “Hardware” mainly refers to computer equipments to perform inputting,

processing and outputting. “Software” is computer programs that control the

operation of the computer. “Database” is considered as the most important part of an

information system. A database stores all required and history of information.

“Communication media” is the electronic signal transmissions enable organizations

to link computer systems into effective networks. Information systems are purposed

to serve people, and a perfectly designed information system also needs personnel

who manage, run and maintain the system. Procedures are strategies, policies,

methods for using the information system. All factors of information systems, which

are explained above, can be shown in the Figure 10, made by O’Brien (2006).


- 25 -

Figure 10: Model of Computerized Information System (O’Brien, 2006)

Information systems are classified by Beynon-Davis (2002) along a horizontal

dimension or vertical dimension. Horizontal systems are differentiated by the type of

organization (private organizations and public sectors); Vertical systems are

distinguished by the three levels of human activity and decision made by the system.


- 26 -

3.3 Importance of Information Systems

High rate of information system failures has been shown at the beginning of the

article, but those astonishing numbers can’t stop numerous enterprises keep

implementing information system and ignore the great importance of information

systems. Current organizations heavily rely on information system in order to operate

efficiently and effectively (Yasin & Quigley, 1994).

Avison and Fitzgerald (2003) suggests that information systems are significantly

effect on current business which are said to be facing increased competition, global

challenges, and market shifts and rapid technological evolution. Boddy et al. (2005)

points out 3 main reasons of people more and more rely on information systems. In

the first place, “Electronic coordination”, modern information systems enable all

kinds of communications where information is routine. Information systems

electronically coordinate organizational activities of input, transformation and output.

Moreover, “Globalization”, information systems affect globalization by providing the

rapid flow of information which is extremely important for international distributed

organizations. “Information-intense firms”, information systems are particularly

important for those companies business in capture, create and distribute information

and knowledge to their customers.

Stair and Reynolds (1999) claim that information systems have the potential and the

ability to apply the latest technologies to work can result in a successful personal

career, organizations that reach their goals, and a society with a better quality of life.

The involvement of managers and decision makes in all aspects of information is a

major factor for organizational success, including higher profits and lower costs

(Stair and Reynolds, 1999).


- 27 -

3.4 Information Systems in Organizations

It is a well known truth that the success of an organization depends on its information

systems (Beynon-Davies, 2002). Information systems are employed by business

organizations for several intentions: to restructure and revise the organization

composition to the form of achieving their goals; to improve customer service and

customer satisfaction (Stair & Reynolds, 1999). Stair & Reynolds (1999) portrays

that the information technology once has been used to the organization, the nature of

work and the shape of organization have been changed.

An organization is a formal collection of people and other resources established to

accomplish a set of goals, according to Stair & Reynolds (1999). Beynon-Davies

(2002) points out there are 2 perspectives bottom-up or top-down. The definition of

bottom-up structure indicates that an organization is built up by amount of human

beings and their actions. A top-down structured organization means the

organizational structure exists independently of the human belonging to it. The top-

down concept considers that a larger social structure direct or constrain human

actions. Another important theory the structure theory is created by Anthony Giddens

(Beynon-Davies, 2002). This theory presents a “duality of structure”, which claims

that the human actions produce and reproduce social structure, on the other hand, the

social structure limits human actions from using the structure as a resource in

translating their own and other people’s actions. The duality of structure can be

exhibited by the Figure 11 below:

Figure 11: Duality of Structure (Beynon-Davies, 2002: 220)


- 28 -

Organizations can also be classified as for-profit organizations and nonprofit

organizations, according to Stair & Reynolds (1999). The main purpose for for-profit

organization is to enlarge incomes by maximizing revenues and minimizing costs.

Nonprofit organizations don’t take profit as the primary goal, such as social groups,

religious communities, and universities. An organization is a system regularly using

money, human, materials, machines and equipments as resources (Stair & Reynolds,

1999).

Figure 12: The elements and context of a computer-based information system

(Boddy et al., 2005)

From another point of view, Boddy et al.(2005) describes that the organization is one

of three social elements (people; procedures; organization). People are the most

sophisticated elements. It includes staffs who input data and who analyze outputs of

information system, and managers who administrate full scale of activities;


- 29 -

procedures are the rules or restricted actions of how people interact with the

information system; the organizational context is consisted procedures and human

actions of interacting with the system. The results of an information system depend

on the interaction between people, system and context (Boddy et al., 2005). The

Figure 12 above illustrates how organizational context working on an information

system project.


- 30 -

3.5 Information Systems Development Methodologies

To make information system actually take place, various different processes and

activities are required (Flynn, 1992). Flynn (1992) claims that to develop a normal

and simplest system, a project team is the basic human resource and other

requirements as hardware systems, supporting software platforms and

communication networks; besides, users and computer specialists cooperatively list

out user requirements and a system specification, select and integrate computer

hardware, conduct system coding, test system and user training. The series of

designing activities and developing forms are generally known as systems develop

process.

Information systems development is a fatal organizational process for many

organizations (Beynon-Davis, 2002). It is essential for an information systems

development follow a confirmed methodology to ensure the systems are conceived,

analyzed, designed and implemented. The conception of information systems

development methodology is defined, by Avison & Fitzgerald (2003), as:

“A collection of procedures, techniques, tools, and documentation which will help

the systems developers implement a new information system. A methodology will

consist of phases and sub-phases. These sub-phases provide developers plenty of

opportunities of selecting the most appropriate techniques at each stage and

effectively supporting developers to plan, manage, control, and test the project.”

(Avison & Fitzgerald, 2003: 20)

Methods, techniques and tools are pre-requisite for project producers when they

undertaking the project process (Avison & Fitzgerald, 2003):


- 31 -

“Methods”: these build up the developing frameworks (Beynon-Davis, 2002).

Beynon-Davis (2002) identifies three major types of developing methods:

1. Structured methods, with a linear model of the development process and clear

identification of inputs and outputs for each phases. Certain techniques are equipped

within structure frameworks.

2. Rapid application development (RAD) methods.

“This kind of methods employs an iterative model of the development process and

generally specifies high level phases based around some form of prototyping.”

(Beynon-Davis, 2002: 323)

3. Object-oriented (OO) methods. Such methods use object attempt to simulate the

real world and build up the model by identifying objects and attributes, relationships

for these models. Object behaviors are also defined in OO methods.

“Techniques”: mainly relate to analyzing and designing phases of developing; they

are usually components of methods (Beynon-Davis, 2002). A technique is a solution

of a particular activity in system development process (Avison & Fitzgerald, 2003).

Beynon-Davis (2002) identifies 2 distinct groups of techniques, developer-centric

techniques and user-centric techniques.

Developer-centric techniques are especially designed for developers to understand,

document and communicate problems to other developers. These techniques include

data analysis techniques, process analysis techniques and object analysis techniques

(Beynon-Davis, 2002).

User-centric techniques support project developers understand some work

environment and the potential technologies can be used by the development. These

techniques include prototyping, scenarios and use cases (Beynon-Davis, 2002).


- 32 -

“Tools”: artifacts used in the system project, include hardware, software, data and

internal, or external, communication facilities (Avison & Fitzgerald, 2003), (Beynon-

Davis, 2002). These tools are graphical user interface tools, fourth generation

languages, transaction processing systems, database management systems, local area

and wide area communication tools.

Nature Of Failure Lihong Zhou

- 33 -

Chapter 4

Nature of Failure

Last chapter illustrates basic conceptions and nature about information system. As

the next step, to study Information systems failures, it is essential to understand the

definitions of failure and the nature of information systems failures.

4.1 Definition of Failure

All kinds of technological and organizational systems suffer from failures (Sauer,

1993). As the extremely fundamental conception, there is no commonly agreed

definition of the definition of failure.

Oxford Advanced Learner’s Dictionary of Current English (1991) defines the success

as “achievement of a desired end, or of fame, wealth or social position.” In the

contrast, the dictionary portrays the failure as “failure of success (for instance: failure

in a examination); state of being inadequate; not functioning as is expected or

required.”

Fortune and Peters (1995) claims that “failure is something that has gone wrong or

not lived up to expectations.” (Fortune and Peters, 1995:21)

Fortune and Peters (1995) argues that almost all judgement about failure are

subjective; those comments about failure are coloured by personal perception,

circumstances and expectations. However, in several situations, it is quite hard to

define a project or a system is successful or unsuccessful. And distinct people may

have different arguments and opinions about a particular project, which some might

think it is a total failure but some people think it is acceptable.


- 34 -

4.2 Types of Failure

Fortune &Peters(1995) suggests a 4-type of failure on the perspective of how well a

project satisfies predefined project objects and user requirements. The 4-type of

failure is shown in the Figure 13 below:

Figure 13: Failure Types (Fortune &Peters, 1995: 23)

Type 1 failure stands for objectives from designer, sponsors or users are not fully

met. Those projects met original objects but with inappropriate or undesirable

consequences and side effects are categorized into failure type 2. Some failures are

designed to fail at particular time or occasions, which kind of failures can be seen as

an important part of the integrated system as the reason of safety issues. This kind of

failures are failure Type 3. The final kinds of failure, type 4, is when the project

meets objectives and requirement without undesirable consequecies or side effect,

benefits and merits of the project is no longer exist. (Fortune &Peters, 1995)

4.3 Information Systems Failures

Sauer (1993) indicates that information systems are the product of a process which is

open to flaws. Beynon-Davies (1999) not just illustrates similar statements but also

distinguishes the concepts of flaw and failure. Compare to concepts of failures

introduced in last section, flaws can be described as perceptions of undesirable

situation faced by project teams and stakeholder which can be solved at a cost or

accepted at a cost.

Failure Types Type 1: Objectives not met Type 2: Undesirable side effects Type 3: Designed failures Type 4: Inappropriate objectives


- 35 -

There is a broad spectrum of failures, Sauer (1993) claims that each type of failure of

technological and organizational system can be studied separately within its own

field, so as information system disasters has its unique features. Information system

failures, are multi-causal particularly for those complex projects (Bignell & Fortune,

1984), usually involve a complex interaction of factors within the human and

technical issues set in various situations rather than directly failure of a single man

(Sauer, 1993). Sauer (1993) lists determinants of information systems success/failure

from previous studies

Russell Ackoff (1967) Designers don’t understand their

objectives; Dearden (1972) Incompetent or ineffective for system

developers; Morgan and Soden (1973) Senior Information System Executive as

the prime determinant of success or failure;

Argyris (1971) Executive resistance as the reason of lack of interpersonal communication skills;

Colton (1972) Lucas (1975) Boland and Hirschheim (1985)

Behavior and social oriented problems are hindering projects development.

Figure 14: Determinants of Information systems Success/failure

Quoted in (Sauer, 1993: 21-22)

Laudon and Laudon (1991) classify information systems development problems into

4 areas: Design, Cost, Operations, Data. Lyytinen and Hirschheim (1987) proposed 4

concepts of information system failure: Correspondence failure; Process failure;

Interaction failure, and Expectation failure. Sauer (1993) explain these 4 factors as

follow:

Correspondence failure stands for failures unable correspond to predefined

objectives and requirements. This is the most common kind of information system

failure. Beynon-Davies (1999) indicates that an evaluation is required to be

conducted during the project, if the mismatch of correpondence between original

objectives and the evaluation, the project is considered a failure.


- 36 -

There are two distinct kinds of Process failures: fail to produce a system at all, and

an information system is produced but fail to meet the predefined or reasonable

constraints of budget and time.

Interaction failure stresses on user satisfaction. It is very common that systems

successfully implemented but fail to satisfy its users. If a system is heavily used

which means the system is a success; in the contrast, if the system is barely used or

working with serious problem which means the system is a failure.

Expectation failure is a conclusion of 3 previous failures, which is defined as an

information system unable to achieve the expectations from a specific stakeholder

group.

Project Management Lihong Zhou

- 37 -

Chapter 5

Project Management

Project management provides people with a pool of powerful tools that improves

their ability to plan, implement, and achieve specific organizational objectives.

Moreover, project management is more than just a set of tools, it is also a

management style of results-oriented that can buildup collaborative working

relationships among various characters (Gray & Larson, 2006). Avison & Fitzgerald

(2003) indicate that many information system failures may reason from inadequate

project management.

“A project is a complex, non-routine, one-time effort limited by time, budget,

resources, and performance specifications designed to meet customer needs (Gray &

Larson, 2006:4).” All projects generally follow a project life cycle. Project

management is not just a management tool, but also a fundamental discipline of

doing business. Gray & Larson (2006) illustrate that the increase of the importance

and the role of the project management in contributing to the strategic direction of

organizations.

The project life cycle model presented by Gray & Larson (2006), which separate the

entire life span of a project into 4 sequential stages: defining, planning, executing and

delivering. On the defining stages, project mangers have to detailed plan the whole

project, such as define project scope and objectives, project team construction and

assigning major responsibilities to appropriate personnel. Planning stage determines

what the project will do, when it will be scheduled, whom it will benefit, how to

execute the quality control and initiate the budget planning. Executing stage occupies

the major portion of the project work. On this stage, factors of time, cost, and

specification are used for steering the project processes. As the final stage, the


- 38 -

delivering stage includes 2 tasks: delivering the project product to the customer and

redeploying project resources.

5.1 Managing Information Systems Projects

Project management techniques are also the safeguard of keeping information

systems projects from failure. Avison & Fitzgerald (2003) depict that project

management in information systems projects is a theme that is important in most

methodologies; it can efficiently and effectively managing the project development,

delivering the project on time, within budget, complete and with the highest quality.

The basic intention of managing information systems projects is to develop required

systems that are (University of Sheffield, 2000):

(a) On time

(b) Within budget

(c) To a quality standard

(d) Modifiable to the future improvement

(e) Within the limits of the resources

Project management in information systems development projects is regarded more

difficult than other projects, according to University of Sheffield (2000). Projects in

constructing industry have much clearer project objectives and milestones. In

contrast, objectives for information systems projects are usually more abstractive and

have more ambiguous milestones. Besides, an information system project team is

constructed from people with various backgrounds. University of Sheffield (2000)

indicates that project managers have to manage multi-disciplinary team of people

who may never worked together before and may have different opinion with each

other’s point of view. Additionally, more sophisticated software and hardware are

normally required which demand system developers constantly changing developing

tools and techniques, and also not easy for final users to accept the advanced

technology.


- 39 -

On the basis of several references (Yeates & Cadle, 1996); (Beynon-Davies, 2002);

(Avison & Fitzgerald, 2003); (University of Sheffield, 2000), the thesis categorizes

the complete information systems project management processes generally into 3

phases:

(1) System planning and estimating;

(2) System design, development and implementation;

(3) System testing and maintenance;

At the beginning stage, planning and estimating, project scope and objectives are

defined; feasibility study and strategic planning are also conducted in this phase of

project. Besides, it is essential for a project manager to buildup a powerful project

team and to choose an appropriate developing methodology. At the stage, after

braking down the entire project and main tasks into several iterative sub-tasks,

project team needs to estimate systems development resource including the duration

and size of the system, potential problems and risks, and costs.

In the system development, design and implementation stage, project team programs

and implements the information system, as planned from the last stage, with certain

quality assurance plans. As the last phase of the whole project, testing and

maintenance are also vitally important. Testing is a tedious and exhausted work, but

it is compulsory. After thoroughly system testing, some of the defects detected are

corrected. University of Sheffield (2000) describes that in reality, as the reason of

several changes are considered to be necessary, parts of the development process

might repeated in system maintenance.

All these 3 major stages of information systems project management are detailed

introduced and discussed in the following parts, as well as necessary project

management tools and techniques in each stage.


- 40 -

5.2 System planning and estimating

As the very starting stage, it is the foundation of the entire project, and this stage is

critically important of associating to the project’s success/failure. Yeates & Cadle

(1996) state that first stage is important as its where the foundation of the whole

project are laid; future works are build based upon this foundation, thus the

importance of the stage should not be underestimated.

Yeates & Cadle (1996) clams that there are 5 concepts that the project team has to

define:

What is be carried out?

Why is it being carried out?

Who is going to do it?

How is it to be carried out?

When is it to be carried out?

In the first place, project team has to perfectly define and understand user

requirement. On the basis of user requirement, project objectives, scope and potential

constraints needs to be clearly identified. Project team can be supported from other

source from feasibility study report, project brief, project terms of refer-source and

contract document (Yeates & Cadle, 1996), on-site observation and interviews.

Yeates & Cadle (1996) states that a starting point for a good plan is proper

understanding of the user requirement, and the project manager must ensure user

requirements are fully comprehended before planning begins. An important problem

pointed out by University of Sheffield (2000) which states that customers usually

miscomprehend between “what we need” and “What we want”. It is common that an

organization consider a system which could be completely different from the system

design from system developers. In this case, system developers have to redesign both

of the hardware and the software in order to achieve the overall objectives.


- 41 -

The next step is to estimate the overall costs of developing and maintaining the

system as well as how the organization will be benefited from the expected

information system. Underestimated project costs and over-expected project benefits

could be a reason of project failure and customers’ dissatisfaction. Yeates & Cadle

(1996) suggest that each project needs to have a Business Case which sets out the

main problems or opportunities which are to be addressed; business case also

contains costs budget and expected profits.

A complete and effective project team is another key of the project. University of

Sheffield (1996) points out project team members are carefully selected for their

competency and compatibility; the project manager is responsible for managing time,

cost, resource and accurately communicates to every team member. The project team

also needs to set up goals for productivity, and team members must be motivated to

achieve them. Additionally, Yeates & Cadle (1996) mention a particularly important

factor that the roles and responsibilities are clearly set out in the project.

The basic method of planning and estimating are breaking down the whole project

into various stages. The decisions will be made on who is going to in charge for what

tasks, how to finish each task and when it has to be finished. Work breakdown

structure and Gantt Chart are the basic tools of planning and controlling the project.

(University of Sheffield, 2000)

Work breakdown structure (WBS) is a traditional approach and has been widely used

in many industries for a long time (Yeates &Cadle, 1996). Yeates & Cadle (1996)

introduce the concept of work breakdown structure as:

“Work breakdown structure is to break the overall project down progressively into

smaller and smaller chunks until we end up with individual tasks, or work packages,

that we can estimate sensibly and assign to team members.” (Yeates & Cadle,

1996:74)


- 42 -

The WBS diagram of an information systems development example is shown in the

appendix II.

Gantt chart, according to University of Sheffield (2000), is named after its originator

Henry Gantt. Gantt chart is a visual tool for communicating schedules to the

organizations. On the left hand side of Gantt chart, there are tasks and durations for

each task. A bar represents the timing and duration on the right hand side of the chart.

It can be used to see which tasks run in parallel and to calculate the total duration. An

example of Gantt chart is shown as follow:

Figure 15: An example of Gantt chart from University of Sheffield (2000),Page 2

Yeates &Cadles (1996) point out that effective planning is essential to the successful

execution of an information systems project. In Sumner (2000) conclusion, risk

factors of formal information management projects and point out 8 out of 18 risk

factors are related to this stage of project initiation. Thus, project team is required

more attentions on this part of development.


- 43 -

5.3 System Design, Development and Implementation

This stage is all about system designing and implementation of the final system to

the organization. Obviously, the project must be finished within planned duration and

budget, but another important factor is to ensure the final system with certain

required quality.

The quality of an information system can be evaluated by numbers of defects, faults

or bugs, failures and errors. According to University of Sheffield (2000), a defect is

an omission, imperfection, ambiguity or fault with a product; a failure is a

breakdown of a project or productions with unsatisfactory results. University of

Sheffield (2000) indicates that Verification and Validation (V&V) usually used for

checking the quality of the project. Validation refers to checking that the system

conforms to its specifications; validation refers to checking system achieves

customers requirements (Sommerville, 1996).

Yeates and Cadle (1996) present a model provided by the European Foundation for

Quality Management (EFQM). The model provides a group of quantitative

evaluation of the key criteria for a total quality business.

Figure 16: The Business Excellence Model of EFQM (Yeates and Cadle, 1996:171)


- 44 -

The model illustrates relationships between people, processes and results. A

successful Business Results is come from People Satisfaction, Customer Satisfaction,

Impact on Society and Impact on Society. These 3 factors depend on the Leadership

of driving factors of People Management, Policy and Strategy, Resources and

Processes. The Enabler concerns with how the organization approaches each of these

factors. The results represent what the organization is achieving or has achieved. The

percentage for every factor is the relative value.

There are 7 approaches of main tasks and activities of assuring information system

quality, based on University of Sheffield (2000)’s conclusion from Pressman

(1987/1992):

1. Application of Technical Methods

2. Formal Technical Reviews

3. Software Testing

4. Enforcement of Standards and Procedures

5. Control of Change

6. Measurement

7. Record-keeping and reporting

System design is another critical issue in this part of project. University of Sheffield

(2000) illustrates unable of fulfilling the customer expectation and achieving basic

business requirement mainly are resulted of the poor system design. Additionally,

based on the research from Kasser (1998), requirements change from customer

during system designing or after is the most dangerous factor from 33 critical factors

of information system failure.


- 45 -

5.4 System Implementation, Testing and Maintenance

After detailed system design, system implementation comes next to the customer’s

agreement on the systems specification, which shows a complete system description

and other specific project related information. Curtis and Cobham (2002) identify

several related tasks that need to be finished in system implementation. These tasks

are writing and testing programs, acquiring and installing hardware, training final

system users, changeover data from old manual or computerized information system

to new database. It is important that they are all completed before actual changeover

to the new system, and some of these activities can be carried out simultaneously.

System implementation can be divided into software system implementation and

hardware system implementation. University of Sheffield (2000) illustrates new files

need to be ready before the new system installation, and the system changeover

requires no significant changes in the data used. Testing is an indispensable but tedious activity during the system implementation.

Testing is a basic approach of ensures system with certain quality and ensuring user

requirements are achieved. Testing work must also be detailed planned with suitable

testing strategies. University of Sheffield (2000) provides 3 testing strategies: top-

down, bottom-up and mixed.

According to Curtis and Cobham (2002), there are 4 types of system changeover.

University of Sheffield points out advantages and disadvantages for each system

conversion strategy:

1) Direct Changeover: New systems replace the old legacy systems entirely. The

advantage of this approach is being quick. However, this approach heavily relies

on liability of the system design, implementation and well trained staffs.

Advantage: the organization can get immediate benefits from the new system.

Disadvantage: High risk.


- 46 -

2) Parallel Changeover: This approach is commonly used if the new system and

old legacy systems are quite similar. This approach requires the old system and

new system run in parallel.

Advantage: this approach can ensure the organization processes naturally without

any pitfalls during changing over the old system to the new system.

Disadvantage: expensive because of both of old and new systems are required

operate simultaneously.

3) Modular Changeover: This approach is for those systems can be separate up to

several identical modules and replace them piece by piece in parallel.

Advantages: every module is fully tested; users can easily accept the new system

and familiar the new system.

Disadvantages: Special attentions must be paid on making modules work

cooperatively.

4) Phased Changeover: If a system is constructed from numbers of individual

modules, the old system can be slowly phase out, in the mean time the new

system gradually phase in.

Advantages: problems exposed at the implementing phase can be learned and

avoided for the next phase.

Disadvantages: requires a long period of time which can create both positive and

negative user problems.

5) Pilot Changeover: Before the final conversion, project team provides a prototype

to small group of users. Project team continuously revises the system from users’

feedbacks.

Advantages: the final system is fully tested and achieved user requirements are

ensured.

Disadvantages: need users’ participation and may give customers an impression

that the new system is unreliable.


- 47 -

Maintenance is taken as the last step of system developing and last phase of System

Life Cycle. Maintenance is a long non-stopping process which continuously modifies

and improves the organizational information system. Although all information

systems are examined by detailed testing, University of Sheffield (2000) claims that

not all defects can be detected by the testing and some of those bugs can only be

emerged after certain duration of system operating. System maintenance includes

hardware maintenance and software maintenance, based on Curtis and Cobham

(2002). Hardware maintenance mainly concerns on maintaining system equipments

such as PCs and network. Software maintenance is a complex and costly process

(Curtis and Cobham, 2002). University of Sheffield (2000) defines software

maintenance is substantially rewriting the original software program.

Risk Management Lihong Zhou

- 48 -

Chapter 6

Risk Management

In Frosdick’s survey, the definition of risk is portrayed as “taking account of both

gains and losses; the probability that a particular adverse event occurs during a

stated period of time, or result from a particular challenge (Royal Society, 1992); a

combination of the probability, or frequency, of occurrence of a defined hazard and

the magnitude of the consequences of the occurrence (British Standard Institution,

1991).” Wright and Wright (2001) suggest that many researchers proposed that risks

maybe the reason of ERP failures. Risk management, the basic purpose, is to

significantly improve project quality and project performance by using

methodologies of systematic identification, evaluate and manage potential project

risks (Chapman & Ward, 1997). Risk management has became a main part in

organizational activities and it also help all activities achieve their objectives directly

and efficiently (Tchankova, 2002). Besides, risk management provides decision

makers systematic approaches to dealing with risks and uncertainties (Williams et al.,

2006). Yeates and Cadle (1996) conclude that projects of information systems are

becoming increasingly complex, and more unpredictable risks are exposed in these

projects. Although it is difficult to avoid risks all together, risks can be identified and

dealt with their negative impact by avoiding or mitigating (Yeates and Cadle, 1996).

As the risks are regarded as the main reason of information system failure and the

high unsuccessful information system project rate, the great necessity of risk

management is obvious and inevitable. Bandyopadhyay et al. (1999) not just point

out risk management for information projects is one of the most important tasks for

project developers, but also list 3 purposes of risk management in information

systems projects:


- 49 -

(1) Protect IT assets such as data, hardware and software from external threats.

(2) Avoid or mitigate complete losses by selecting and implementing the best

combination of security measures.

(3) Protect project from internal threats such as technical failures, sabotage and

unauthorized access.

According to Williams et al. (2006), before developing the formal risk management

plan, it is fatal to comprehend potential organizational risks, basically there are 3

classifications of risks: risks must to be managed, such as risks may jeopardize

project quality, government requirement and environment related issues; internal and

external classic risks; the third group of risks are those have no mandatory

requirements to work on, which usually without external forcing compliance and no

clear self-preservation reason to manage them.

Chapman & Ward (1997) present that risk management involves evaluating risks and

opportunities; assess and develop a contingency plan supporting the base plan, which

exhibits objectives of the project and how to achieve them. There are 2 main

approaches of risk management. As the main approach of risk management,

proactive risk management concerns planning which proactively identify potential

future threats. University of Sheffield (2002) coins the proactive risk management is

the best practice in Information Systems industry of preventing unexpected

consequences. Reactive risk management plan acts as a complementary approach in

the real risk management. This approach focuses on strategic plans on how to react

after a crisis.


- 50 -

6.1 Risk Management Process

Three main risk management stages are identified by Williams et al. (2006): (1) risk

recognition and identification; (2) risk assessment and prioritization; (3) risk

management. A general risk management process is displayed by Yeates and Cadle

(1996):

Figure 17: The Risk Management Process (Yeates and Cadle, 1996: 190)

A similar but more thorough risk management process made by Standards Australia

and Standards New Zealand is shown by Williams et al. (2006), and the process

diagram is shown as follow.

Figure 18: Risk assessment process

(Standards Australia and Standards New Zealand 2004 a, b)

(Williams et al., 2006)


- 51 -

Establish the context and risk identifications are 2 main tasks of risk recognition, or

risk assessment. Context establishment defines what is at risk, and then risk

identification points out uncertain events potentially jeopardize or benefits within the

context. Possible consequences of these uncertainties are defined in the phase of risk

identification. Risk prioritization aims to comprehend the nature and level of risks

identified in the stage of risk recognition. The first step of risk prioritization is risk

analysis which defines levels of risks by likelihood (probability and frequency of risk

occurrence) and risk consequences. After analyzing risks, risks are evaluated and

ranked by a certain risk-acceptance criterion. The next stage is to manage risk by

various methodologies (Williams et al., 2006). The simplest way of dealing with

unacceptable risks, claimed by Williams et al. (2006), is “four Ts” presented by

European Foundation for Quality Management (2005). “Four Ts” are Terminate (stop

activities associated to the risk); Treat (develop a plan to deal the risk); Tolerate

(accept the risk with undesirable defects); Transfer (transfer negative impact of risk

to other entity or organizations).

6.2 Risk Identification

As the very first step of risk management, risk identification is required to examine

the entire project and identify potential risk area. Risk identification initiates by

attempting create a list of all possible risks that could affect the project (Gray &

Larson, 2006).

Frosdick (1997) claims the brainstorming is the main intuitive technique of risk

identifying, which involves a group generating ideas off the top of their heads. Gray

& Larson (2006) briefly introduces the method. Typically, in the planning phase, the

project manager organize a risk management team consists of core developers and

relevant stakeholders. Brainstorming participants are encouraged to keep an open

mind and generate as many possible risks as possible.


- 52 -

Besides, according to Frosdick (1997), inductive risk identification analysis includes

preliminary hazard analysis, checklists and human error analysis. Hazard and

operability studies (HAZOPS) and Failure modes and effects criticality analysis

(FMECA) are the most common techniques. The HAZOPS is structured

brainstorming exercises for a multi-disciplinary team of experts systematically

consider every item in the project. Possible deviations from the intentions are

detected by using guidewords such as “not”, “more”, and “less”. FMECA is based on

thoroughly understand the whole organization and project after investigation, and

this technique usually carried out individually. FMECA presents a step by step

process for examining design weakness and possible failure causations base on either

hardware orientation, focusing on potential hardware equipment failure, or event

orientation, emphasizing on unexpected outputs and the negative effects.

Each project is unique, and various risks are unpredictable. It is obvious that in some

particular area, risk could arise and it is difficult for a project manage ensure all

possible risks are pointed out (Yeates & Cadle, 1996). Yeates & Cadle (1996) suggest

the project manager to be strictly honest about the risks.

Kliem & Ludin (2000) highlight the purpose of risk identification are provides

project managers to select personnel who are qualified of risk management processes,

identify critical risks and highlight potential risk areas, advanced understand the

system and processes of the project and its risks or goals.

6.3 Risk Assessment and Management

Activities of risk assessment, impact consequences and likelihood, are carried out

next to identifying and describing risks. Risk assessment is a fundamental decision

making process resulting in the selection of appropriate safeguards for information

systems projects (Lichtenstein, 1996). Lichtenstein (1996) defines 2 steps of risk

assessment. For the first step, risk analysis processes define the scope of the risk


- 53 -

assessment; identify information resources and prioritizes risks. The second step

involves a risk management process makes decisions of unacceptable risks; risk

transferring processes, or ignore risks, or reduce impacts from risks.

It is impossible to eliminate all risks completely but a better way is to reduce risk

likelihood and risk impact. The risk impacts largely depend on proportions of the

programming work undertaking by project developers based on Yeates & Cadle

(1996). The risk impact will be less if only a small proportion of programming for

coding team. Another factor of risk evaluation, mentioned by Yeates & Cadle (1996),

is the likelihood or probability of the risk context. There is no certain mathematical

method to value risk impact and likelihood, but they can be measured by the scale

below.

Risk Impact:

Figure 19: Risk Impact Scale (Developed from Yeates & Cadle (1996))

Risk Likelihood:

Figure 20: Risk Likelihood Scale (Developed from Yeates & Cadle (1996))

On the basis of 2 scales, a risk analysis formula is presented by University of

Sheffield (2001):

Risk Exposure = P × L

P: risk likelihood; L: risk impact;


- 54 -

The definition of risk exposure, provided by Boehm & DeMarco (1991), is the

probability of the risk occurrence and the potential loss caused by the risk.

After risk identification and risk analysis, there are 2 basic approaches of dealing

with the risks provided by Yeates and Cadle (1996). The first approach is to attempt

to prevent the risk from occurring, called avoidance. Mitigation is the other approach,

which intends to reduce the impact after risks taking place. In practice, these 2

approaches are both considered as the avoidance approach may fail and not as

efficient as it is expected (Yeates and Cadle, 1996).

Risk management is an ongoing nonstop process and more unexpected risks might

expose as the project developing. Yeate & Cadle (1996) suggest that the project

manage team should documents a risk management plan, particularly for those large

projects with a large amount of complex risks. The risk management plan should

enclose a statement of the scope and intensity of the risk management; an

explanation of the risk management cycle about when and how will carry out

particular risk treatment; a human resource plan about risk management; regularly

report of risk management status to senior managers.

Information Systems Project Risk Checklist Lihong Zhou

- 55 -

Chapter 7

Information Systems Project Risk Checklist

Current studies about risk assessment in information systems development are

introduced in the Introduction chapter, which chapter also points out the significant

insufficiency of current researches. In former chapters, the dissertation reviews and

discusses background fundamental definitions and theories of nature of information

systems, nature of failure, project management methodologies and techniques, risk

management on the aspect of information systems projects. The main objective of

this chapter, as well the main purposes of this thesis, is to frame an information

systems project risk checklist. All possible major risks that have been considered as

main causations of information system disasters in previous publications are intended

to be recorded in this risk checklist and discussed afterward. The purpose of

constructing this checklist is to produce an elementary guideline for information

systems project developers in order to avoid potential risks and keep the project on

the right track.

7.1 Literature Review Based Checklist Building

The initial version of Information Systems Project Risk Checklist, in this chapter, is

generated from deeply literature review on the aspect of risk management on

information systems and covers most frequent risk factors in information systems

project failures from previous researches.

The second version, the formal version, is presented at the end of case studies, is a

revised edition by applying the checklist onto the case study of INCIS and London

Ambulance Service.


- 56 -

Figure 21: Diagram of revisions on Information Systems Project Risk Checklist

Generally, both checklists are consists of 5 components, namely Pre-project, Project

management, Customer, Project management, Project technical aspect, Project

development. Relations between these 5 parts can be presented as the picture below:

Figure 22: Relations of 5 components

from Information Systems Project Risk Checklist

All potential information systems project risks are categorized into 5 facets which are

illustrated on 5 separated lists. Various risks are simultaneously recorded on 2 or

more lists, consequently 5 lists are closely interconnected.


- 57 -

7.2 Pre-Project

This part of checklist concerns critical issues that project team needs to beware

before actual project designing. This part not only covers project team side of story

but also identifies some negative issues within customers’ organizations, and the

most importantly, some risks about contracts are also illustrated. Information Systems Project Risk Checklist Part 1of 5

Pre-Project 1 Complex and unclear relationship among partners, customers and suppliers 2 Disagreement between contract involved partners 3 Failure to consider existing relationships when replacing systems 4 Failure to consider existing systems when replacing systems 5 Lack of senior management support 6 Project scope, objective and requirement specifications are ill defined 7 Resources are not allocated well 8 The customer lack of formal experience 9 Unclear payment schedule without linked milestones 10 Unsuitable business plan and vision 11 Without backup plans for delay or under-performance

As introduced in Chapter 5.1, before the project, the customer needs to have a

general understanding of a sound project plan and appropriate user requirement

specifications. Any modifications, or even revision, after contract on project user

requirements are risky (University of Sheffield, 2000; Sumner, 2000; Kasser, 1998).

Yeates & Cadle (1996) claims ambiguous roles of, and vague relationships with

every project parties, in addition of internal political difficulties in the customer’s

organization are major risk factors in information system projects. Customer side of

management plays an important role in this phase. First of all, supports from

management are the basic requirement of the project. Nah et al. (2001) suggest that

top management support is needed throughout the implementation and top

management needs to publicly and explicitly identify the project as a top priority.

Based on Kasser (1998)’s study, lack of senior management support is one of top 10

risks. With management’s support, adequate resources (fund, staff, etc) can be

allocated to the project. Yeates and Cadle (1996) imply that the contract is the biggest


- 58 -

risk since the contract is a strategic tool for parties convey potential project risks,

particularly project scope is ill-defined or not agreed between parties. The contract

needs to be suitable for the specific project with clearly defined payment schedule

and backup plans in case of delay and other unexpected emergencies. A clear

business plan and vision to steer the direction of the project is needed throughout the

project life cycle (Buckhout et al., 1999, quote by Nah et al., 2001). A business plan

that outlines proposed strategic and tangible benefits, resources, costs, risks and

timeline is critical (Wee, 2000 quoted by Nah et al., 2001).

7.3 Customer

Customers might not familiar with information technology. They might unable to

fully understand certain specific risks involved in information systems projects. Here

are some risks and suggestions on the aspect of customer. Information Systems Project Risk Checklist Part 2of 5

Customer 1 Changing requirements, scope and objectives 2 Conflicts between user departments 3 Final Users are unfamiliar with the technology and require additional training 4 Internal political difficulties 5 Lack of customer support 6 Reluctance of changing environment to accept the new system 7 Resources are not well assigned to the project team 8 Unable of unifying customers perspective on the project

Information systems projects require fully cooperation from all involved parties.

Conflicts between user departments and internal political difficulties can bring

troubles to information systems implementation (Yeates and Cadle, 1996). With the

development of information system, organization structure and culture will be largely

changed which may lead to significant user resistance and cause a series of risks.

Lyytinen (1988) illustrates that a stakeholders expectation failure is a gap between

stakeholders’ expectation in some ideal or standard and actual system performance. It

is critical to unify involved parties’ perspective on the project.


- 59 -

7.4 Project Management

As illustrated in the Chapter 5 of Project Management, project management provides

efficient and effective tools to manage information projects, assists project team

eventually successfully implement the information system. However, there are also

specific rules about project management. Information Systems Project Risk Checklist Part 3of 5

Project Management 1 Lack of an effective methodology and poor estimation 2 Unrealistic schedules and budgets 3 Lack of effective risk controlling plan 4 Milestones and related deliverables are ill-defined 5 Inexperienced team member in the business or technology 6 Inappropriate structure of project team 7 Team members are not fully committed to the project 8 Too many junior staff or too many senior staff 9 Unfamiliar environment for system developers 10 Lack of process and standard 11 No single person is accountable/responsible for the project 12 Unrealistic deadlines 13 Inappropriate staffing, personnel shortfalls

Yeates and Cadle (1996) suggests that an initial project plan may not necessarily be

presented before the contract but a comprehensive and proper project management

plan needs to be initialed as soon as possible. Kasser (1998) introduces lack of or

poor project plan is a significant risk. Besides, an effective project team is consisting

of well trained project developers. Neither too much experienced developers nor too

much inexperienced developers are both inappropriate (Yeates and Cadle, 1996).

Familiar developing methodologies and environment for developers are preferable

(Yeates and Cadle, 1996). Efficient communication among project managers, project

team, customer managers and system final users is the basic insurance for successful

project management and the final victory of information systems implementation.

Nah et al. (2001) introduces that the project must be formally defined in terms of its

milestones (Holland et al., 1999, quoted by Nah et al., 2001). Timeliness of project

and the forcing of timely decisions should be managed (Rosario, 2000, quoted by


- 60 -

Nah et al., 2001). Deadlines should be met to help stay within the schedule and

budget and to maintain credibility (Wee, 2000, quoted by Nah et al., 2001). A project

need a high level executive sponsor who has the power to set goals and legitimize

change, and continually strive to resolve conflicts and manage resistance (Nah et al.,

2001).

7.5 Project Technical Aspect

This part of the checklist focuses on those technical issues which might turn into

significant accidents. These risks identified in the table are not only for technicians.

They also require attentions from top managers. Information Systems Project Risk Checklist Part 4of 5

Project Technical Aspect 1 New or unproven developing methodology 2 Unstable or incompatible software system platform 3 Unfamiliar programming language to the developers 4 Unfamiliar developing environment to the developers 5 New or unproven hardware composition 6 Coding from high level requirement without thoroughly design 7 Lack of adequate technology infrastructure

On the technical aspect, a detailed and fully considered technology infrastructure is

the top priority. Both stability and compatibility of hardware and software platform

are highly required. Any instability and incompatibility can bring catastrophic system

crash. Unproven or unfamiliar programming languages, development tools and

methods can lead to project delay, design flaws and project failure. Normally, an

information systems project consists of several similar iterations. In this case,

reusable programming modules can bring a lot of merits to the project. As the reason

of reusable programming codes are not only familiar by programmers, but also

considered as proven and fully tested.


- 61 -

7.6 System Development

The final part of checklist looks into specific potential risks involved in each steps of

system development.

Information Systems Project Risk Checklist Part 5of 5

Project Development Development Stages

ID Risk Factors

1 Incomplete comprehension of the current system and current situation of the organization

2 Partially understand the user requirement

System Analyzing and Requirement Defining

3 User requirement are formed only by management level without final system users

4 Inadequate initial general planning before detailed and complex designing

5 Too accurate designing causes the system extremely heavy and unhealthy

6 Without an agreed prototype by the customer

System Designing

7 Poor communication and understanding among the system analyst, designer and the programmer

8 Attempting produce a complete and perfect system in one stage9 Initiate system programming when the system designing is not

completed 10 Accept testing without final users involvement 11 Programming without a preselected and unified methodology 12 Programming without necessary documentations 13 Lack of considering reuseable components from the current

system

System Developing and Acceptance Testing

14 Programmer-oriented system designing instead of user-oriented15 Lack of new system cutover methodologies 16 Insufficient user training

System Training

17 Initiate training without complete testing 18 Lack of a thorough maintenance plan System

Maintaining 19 Assign unqualified staff for system maintenance

The first step for an information systems project is System Analyzing and

Requirement Designing. Preliminary investigation about organization background

and organizational cultural is compulsory. As stated in Chapter 4.2, University of

Sheffield (2000) claims that customers usually miscomprehend between “what we

need” and “What we want”. The investigation also provides a good opportunity to


- 62 -

project team to completely understand the user requirement by observation, interview

key people and randomly talk to final system users.

System designing, as the next step, is carried out on the basis of well comprehension

of organization background and project definitions (objectives, scope and

requirement specifications). Drori (1997) advises that the phase requires constant

communicating within the project team; and also between the project team and the

customer. Too detailed designing or too general designing can cause system illnesses.

In order to ensure the project team is producing what the customer want, it is a good

strategy to have customer’s agreement on a prototype system.

A complete system design is the foundation of system developing. The system

developing also needs a minute step-by-step plan. The project team should avoid

producing the system in one phase (Drori, 1997). Programming strategy is defined at

the beginning of the stage. System testing and system training are equally important.

Both of them are required detailed planning before deployment. Moreover, Drori

(1997) indicates that system programming should be initiated after the system

designing is finished; and the final users training should be began after the final

system is fully tested. After the information system development, the organization

needs to initiate a constant system maintenance plan and allocate qualified staff to

implement it.

To conclude, at the beginning of this chapter the concept of how this version of

Information Systems Project Risk Checklist is produced and how to improve this

checklist in the following chapters is explained. The major part of this chapter

dedicates to present and discuss the initial version of Information Systems Project

Risk Checklist. In next 2 chapters, the initial version of checklist is significantly

revised by empirical study of 2 case studies.

Integrated National Crime Information System (INCIS) Lihong Zhou

- 63 -

Chapter 8

Case Study 1: Integrated National Crime

Information System (INCIS)

8.1 Case Background Introduction

The project of Integrated National Crime Information System (INCIS) was designed

to the New Zealand Police with the purpose of supporting operational Policing by

providing improved information, investigation and analysis capabilities. This will

minimize the incidence and effects of crime on the community through the detection

and apprehension of offenders and by crime prevention strategies. The strategic goals

of INCIS are described as:

“The overall strategic goal of ‘reducing the incidence and effects of crime’

transcends traditional crime prevention and seeks to stem and reverse crime trends.

The basic requirements to achieve this ambitious goal are significantly changes in

policies, processes and attitudes within the New Zealand Police.” (Small, 2000:23)

The Executive Summary of Ministerial Inquiry defines the INCIS project is a high

risk project, however it could be achieved with conventional and proved technology

with appropriate time and management, sufficient resources, skills and experience.

The result of INCIS project is achieved some of its objectives but many of its main

objectives are unachievable. IBM, the project partner, didn’t effectively assess and

manage risk, subsequently unable to deliver the system within the time constraint.


- 64 -

8.2 Problem Domains Description

8.2.1 Pre-project

1. Changes in Project Scope, Objectives and Requirements

The concept and vision of INCIS project are considerably relevant to the police’s

strategies, but project scope and primary objective of the project are ill defined, and

which significantly changed for several times during the project development.

Modifications of project definitions, such scope and objectives, are considered

catastrophic.

The main concern of the project modified from police strategy to financial objectives

and finally to a technology project. These changes dramatically affected already

defined and approved project scope and requirements, further revisions are required

in this circumstance. Originally, the project plan was to deliver INCIS by Release

One and Two. Under the description of INCIS Liaison Update Newsletter January

1995, Release One was required to install PC networks in all police stations, replace

the legacy with new Suspect and offence Information System, Crime Trend Analysis,

Intelligence Analysis and Mail Facilities by 1997. Release 2 was to implement Case

and Investigation System early 1997. However, the final project was implemented by

3 steps of Increment, which was regarded more focus on software development.

Increment one provided police a system with basic functions, such as electronically

input crime details into an independent database. The core of INCIS system was

Increment two, which significantly improve the basic system from Increment One

into an automatic information system. The implementation of increment two was

intended completely replace the legacy system. Increment Three was planed to add

extra features onto increment two.

The original scope and requirements of INCIS were largely changed as the reason of

implementing Business Process Re-engineering (BPR) after the contract. Although


- 65 -

police considered BPR after contract is not harmful, but approximately 8 months of

project delay is indirectly resulted by BPR changes. The Small (2000) emphasizes

the “… the general BPR work needed to be integrated with the INCIS BPR work…. If

BPR had been completed or substantially completed prior to contract, a number of

problems would not have arisen.” (Small, 2000:46)

“…if the BPR is not done first, that the technology will drive the business

requirements rather than the other way around.” (Small, 2000:44)

Risk Identification:

1. Project scope and objectives are ill defined.

2. Significantly change project scope and object after contract.

3. Inappropriate business processes, and reforming them after the contract. 2. Contract

Yeates and Cadle (1996) claim Contract is the major risk in the stage of Pre-project.

All involved parties will be penalized by unsuitable contracting by serious

consequences. The contract for INCIS was signed on 23 September 1994 and with

various obvious pitfalls.

The first contract problem was exposed during Business Process Re-engineering

which enlarged the scope of INCIS. The increment was defined after contract and not

covered by the contract.

“BPR should have been completed or substantially completed before Contract and

before completion of the design of application.” (Small, 2000:44)

“…if BPR had been completed or substantially completed prior to contract, a

number of problems would not have arisen.” (Small, 2000:44)


- 66 -

The contract, signed by both parties of police and IBM, was under the circumstance

of a number of unresolved issues in relation to technology, architecture, governance,

management and risks that required to be urgently reviewed by both parties and

INCIS project team. The contract was considered with too many outstanding issues.

The scope and the technology of the project were not satisfactorily defined. The

contract was a fixed or capped priced contract in Release one and an inductive priced

contract in relation to Release two.

“Neither of the parties was ready to enter into contract or to deliver on time or on

budget.” (Small, 2000:151) In the first place, they lack of thorough understanding of

organization status. This factor seriously affected correctly defining project scope

and addressing user requirements. University of Sheffield (2001) suggests that

incorrect project scope and creeping user requirement lead to either building a wrong

system or developing a correct system with bad quality. Moreover, risk management

plan and main risk issues were not specifically addressed before the contract. Small

(2000) points out the contract should not be signed without well-understood, at an

acceptable level, capable of management and accord with detailed risk identification

under the current organization form. A fixed and capped price contract is also

inappropriate. Yeates and Cadle (1996) advise that payment should clearly schedule

and closely connected with sound project milestones and deliverables. A similar

statement was also provided by Small (2000), which claims the schedule of payment

should in accordance to incremental delivery of system modules. Finally, technical

issues and development methodologies were not clarified. This factor induced major

changes with significant impact to the system design and implementation.

Methodologies and technologies substitutions are specifically discuss in 7.2.4 Project

Technical Aspect.

“The Chief Executive should not proceed to contract, unless the risks, as negotiated

in the proposed contract, are well understood, at an acceptable level, capable of

management and in accordance with the risks identified in a business case. Moreover,


- 67 -

the greater uncertainty, the more the contractor will increase the price to allow for it.

But there is a second and very major risk; that either party does not understand the

risk uncertainty. If this is the case, then the contract may prove untenable and could

lead to termination.” (Small, 2000:120)

The contract included a backup plan, called Off-Ramp, which was a significant factor

in police side of risk management. The Off-Ramp enabled police to stop the contract

in particular terms and conditions. Police nearly gave up the right of Off-Ramp. The

contract limited that the Off-Ramp could be conducted during a period of 90 days

from a specified commencing date, which is changed for several times. Small (2000)

suggests that “police should have used Off-Ramp in that specific circumstance, or at

least attempted to negotiate a Lay-by.” (Small, 2000:47)

On December 1997, police and IBM initialed a Deed of Variation which specified the

requirements to be achieved by IBM and add traffic, firearms and the justice

interfaces to the project. The cost for the development work to satisfy the

requirement was assessed $20 million which subsequently was proven inadequate.

“The Deed of Variation was considered robust by industry standard but also presents

management challenges to police (Small, 2000:53)”, opinioned by Phillip Fox,

solicitors, on the request of Treasury and Police.

Mr. Carr suggests that “the Contract contained agreement in word but that there was

not agreement in mind and heart.” (Small, 2000:149)


1. Lack measurement system for assessing and controlling project risk before

contract.

2. Project scope and objects are ill-defined in the contract.

3. Changing user requirements.


- 68 -

4. Unclear payment schedule without linked milestones and deliverables.

5. Lack of, or ignore, backup plans for delay or under performance.

6. Lack of thoroughly comprehension on organizational nature before contract.

7. Technical framework and development methodologies are not firmly defined in the

contract.

8.2.2 Customer The original INCIS plan dated back to 1985. Police had done considerable work to

develop a long term INCIS vision and concept. The entire INCIS project was fully

supported by the New Zealand government and the police. Top management was

quite committed and cooperated to the project.

However, police without an adequate understanding and detailed plan, particularly

the relations among INCIS, current organization form and long term business

strategy. Police significantly change project scope and requirements for several times.

These factors enormously impacted the successful INCIS system delivering.

Additionally, the communication within police is defective. A number of major risks

were possible to be avoided if formal reports were placed or fully considered.

Firstly, Ernst & Young (Australia) was assigned both by police and Treasury to report

pertinent suggestions particularly on strategic considerations, the business case and

the project tender bid. In this report, Ernst & Young mentioned that the report was

remarkably accurate and prescient document, but it was prepared within two weeks;

more importantly, Ernst & Young stated that report is not reliable for

recommendation to proceed with INCIS or to enter into contract, and it is incapable

of playing any important role. Moreover the Executive summary misrepresented the

whole report. However, police generated a paper to recommend Cabinet the

development and implementation of INCIS based on Ernst & Young report.


- 69 -

Secondly, in February 1994, a presentation for Minister for Information Technology

from a senior architect of Microsoft stated that the major aspects of the technology

were not deliverable and project is suggested should not be proceeded in that form at

that time. However, police asserted that these problems could be resolved and

recommend the project proceed.

Thirdly, in mid-1994, the Sapphire report identified a number of main business and

technical risks and advised that in order to make sure successfully deliver the INCIS

project these risks need to be resolved and managed. However, police consider these

risks were noticed in the report are those can be managed with a long time strategy

through the life of the project.

Finally, the Deed of Variation is the last remedy of the project. The amendment of

contract required extra cost about $20 million, but the Deed of Variation is signed

without authority from the Cabinet for the increased expenditure.

A number of additional critical issues were summarized by Mr. Steward Watson, who

was appointed as INCIS Project Director in October 1998. “(1) The project,

especially in the police side of the project, had no clear plan that was understood by

all; (2) police had no implementation plan, no training and no strategies in place

when IBM was ready to deliver Increment One.” (Small, 2000:55)


1.Changing project requirement, scope and objectives after contract.

2.Final users are unfamiliar with the technology and lack of training.

3.Incorrect decisions made by the project team due to the inefficient communications

within the customer organizations.


- 70 -

8.2.3 Project Management

On the project management point of view, the INCIS project management was

notably flawed and didn’t effectively direct the high risk project. The problems of

project management of INCIS project can be classified as follows:

In the first place, project schedule and budget were inadequately defined before

contract, project estimation was poorly generated. University of Sheffield (2001)

suggests that project cost estimation, and quality control are built on the project

schedule plan. The INCIS project experienced several times of changing scope and

requirements, it is reasonable that project management plans must be delicately

revised after changing. The capped priced contract wasn’t made by thorough

deliberation on the detailed project schedule and associated expenditures. As the

reason of unclear defining project schedule and related budgets, project quality

assurance and risk management inevitably can be regarded as on a wrong direction or,

at least, inappropriate.

Besides, as a major part of project management, risk management in INCIS was

significantly insufficient. In the conclusion of Small (2000), “there was little risk

management applied to the INCIS Project until the later stages. Uncertainty was

neither understood nor addressed, and many of the changes and decisions made

during the course of the project added substantially to the risk. Lack of risk

management contributed substantially to the project’s difficulties” (Small, 2000:117)

It is reasonable to appoint a risk management specialist as a risk manager, especially

for the project like INCIS with high risks and high complexity. “Because of the high

risk involved in the INCIS Project, it would have been generally accepted practice to

appoint a Risk Manager, whose rile would have been to ensure the appropriateness

and integrity of the risk management regime.” (Small, 2000:117) The contract is a

tool for transferring defined risks between parties. Thus, it is vital to produce a

complete project risk management plan with specific risk identifications before the


- 71 -

contract. “A fixed price contract for the whole of the work of INCIS was

inappropriate and, in fact, increased the risk.” (Small, 2000:71)

Additionally, it is project managers’ obligation to supervise and endorse the project

team to select a fit system developing methodology and technology for the final

information system. The developing methodology is not only required to be

conventional, but also need to be familiar to system developers. A specific technical

infrastructure had been chosen and recorded in the contract including Object

Oriented Technology (OOT), decision support technologies, portability, a process

manager and client/server architecture. Small (2000) states that “although the project

was of high risk, in 1993-1994 the concept or vision of INCIS was sound and should

have been capable of being achieved through the use of conventional technology.”

(Small, 2000:30) However, those technologies that INCIS used were “mostly

unproven for a large complex application” (Small, 2000:70). But in police’s report,

they claimed they had chosen proven leading technology.

The human resource is another project management problem for INCIS. University

of Sheffield (2001) introduces that whilst implementing computerized information

systems, besides managing time and resources, project managers must also many

project personnel, who are selected due to their competency and compatibility.

“Inappropriate staffing, personnel shortfalls” is on Sumner (2000)’s major list of risk

factors in information systems development. During the period of INCIS

development, the project had been managed by 3 project managers in total.

Noticeably, Tony Crewdson, appointed as the project manager in 1994, had extensive

IT experience but he didn’t have experience in managing large IT projects. “The

appointment process needs to ensure the appointment of persons with skill and

experience in the management of large IT projects.” (Small, 2000:44) After the

reassignment of Tony Crewdson in June 1998, the position of project manager was

empty until October 1998. The police chose an emerging developing methodology

but project developers were lack of experience. Yeates and Cadle (1996) advise that


- 72 -

project members should be wisely selected according to the project plan and project

objectives, not too many experienced members (cost overruns) and not too many

inexperienced members (effort overruns).


1. Unrealistic schedules and budgets.

2. Lack of an effective methodology and poor estimation.

3. Lack of effective risk identifications and risk management plan before contract.

4. Inadequate managing risks after contract.

5. Inexperienced team members in business or technology.

8.2.4 Project Technical Aspect The INCIS project was unique at the time with high complexity and high risks.

“There was no similar project or software package anywhere in the world.” (Small,

2000:30) This factor indicated that INCIS would have to build as a world-first

system by using emerging technology in an unproven way (Small, 2000). Small

(2000) shows that the INCIS objectives were capable are achieved by employing a

conventional technology. Police proposed very specific technical requirements:

Object Oriented Technology (OOT), decision support technologies, portability and a

process manage, and these technologies are on the basis of distributed Object

Oriented Two Tier Client/Server architecture.

However, Small (2000) claims “the technology were mostly unproven and

inappropriate for INCIS project”. (Small, 2000:70) These technologies included:

1. Object Oriented Technology for application, design and development. It is

undeniable that OOT has several significant advantages, however the decision to use

OOT is considered was high risk due to OOT was a very new technology and

standards were immature. There is no previous project in the size and complexity of

INCIS used OOT, particularly in the distributed Object Oriented Two Tier


- 73 -

Client/Server architecture. “The decision to use OOT made the implementation of the

INCIS application high risk because it was very new technology and standards were

immature. The methodologies and support tools for designing and developing

business systems using and OOT were not readily available nor was there a pool of

experienced developers.” (Small, 2000:71)

2. The decision of using distributed OO two tier client/sever was brutal. The

technology is very new and without any prove that the technology could meet project

objectives. “In the 1994 to 1997 period, this was an emerging technology and

subsequently has not fully met the claims originally made for it. There is no

successful examples of distributed OO two tier client/server architecture of the scale

(3000 plus users) and geographical distribution when proposed initially in the INCIS

project.” (Small, 2000:72)

3. The process manager was a core INCIS software component which integrating all

other software module together. The process manager was also a new product.

Sapphire report (Small, 2000:279) illustrated that “IBM had no demonstration that

they had the capacity to build the application”. Moreover, the process manager was

planned to be delivered in Release two, but whilst the project implementation the

application was required in Release one. This significantly increased the risk. Finally,

the process manager was substituted by a new software named Flowmark.

4. The decision support system is one of the major objectives that project team was

required to achieve. However, this objective was “very ambitious at that time”

(Small, 2000:73). Police intended to build this system and distribute is to most police

divisions and facilitate most staffs. This consequently required large workload of

local customization and extensive specialized training for police. Finally, the system

was ceased as the contract termination. (Small, 2000)


- 74 -

5. Portability also stopped when the contract was terminated. Small (2000) argues

that the application was unable to be finished in the timeframe and the requirement

for portability across a number of operating system platforms.

Generally, infrastructure and application development are the 2 main part of INCIS

technical solution. The technical infrastructure was mainly required to support the

INCIS software applications. The technical infrastructure included computer

hardware (server), system software, System Management Center (SMC), Wide Area

Network (WAN), Local Area Networks (LANs), and computer desktops. The core

architecture for INCIS, represents those technologies used for design the INCIS

system technical architecture, is consisted of Host Operating System MVS, Station

Server Operating System (OS/2 & LAN Server), Desktop Operating System (OS/2 &

LAN Requestor), Host Database (DB2), Station Cabling (Token Ring) and Network

Protocol (SNA). However, police had no technology substitution in spite of “IBM

warned them before contract that it was impossible to achieve the technology

specified”(Small, 2000:71). Besides, developing technologies had remarkably been

changed through system development. These technical substitutions are: Wide Area

Network (WAN) was changed from IBM’s System Network Architecture (SNA) to

Transmission Contract Protocol/Internet Protocol (TCP/IP); Local Area Network

(LAN) was changed from Token Ring protocol to Ethernet; the operating system

OS/2 was substituted by Windows NT; system architecture was changed from

distributed Object Oriented two tier client/server to a three tier client/server

architecture.


1. Emerging or unproven, and inappropriate developing technologies and

methodologies.

2. Unfamiliar developing technologies or methodologies for system developers.

3. New or unproven hardware solutions.

4. Significantly change technical infrastructure after system design.


- 75 -

8.2.5 Project Development System Analyzing and Requirement Defining: Project scope and user requirements

were not sufficiently defined before contract. Both parties of police and IBM brutally

proceed into contract without selecting proper proven technology specification and

complete project plan estimations. Small (2000) notifies another factor that there was

no proof of concept which is designated to test the proposed approach on a limited

scale. Poof of concept tests the concept, size, and likelihood that the system will

implemented as expected. “There was no comprehensive proof of concept early in

the Project and this was a significant factor in the Project failing to fully achieve its

objectives.” (Small, 2000:139)

System Designing: As the reason of project scope and user requirements were

incorrectly defined before actual project implementation, significant scope and

requirement changes were made after contract. This required immediate project plan

changing. Project technical infrastructures and development methodologies were also

largely modified in the stage of system designing. In this case, project developer

were requested quick reaction in new developing platform and rapid understanding

on new user requirements.



methodologies.

2. Project scope and objectives are ill-defined in the contract.

3. Significantly change project requirement, scope and objectives after contract.


- 76 -

8.3 Conclusion

INCIS project was incomplete rather than it was a total failure. Small (2000)

illustrates that “INCIS did deliver a number of benefits to police. The project

delivered a technical infrastructure, including approximately 3,000 desktops. The

project also delivered certain aspects of the INCIS application (Increment One)

which was essentially a National Intelligence System (NIS) replacement.” (Small,

2000:57) Nevertheless, the project did not achieve its original objectives. The project

failed on both technical and management sense. The INCIS was a world-first

application using ambitious and emerging technology with great project size and

complexity. However, “although the project was of high risk, in 1993-94 the concept

or vision of INCIS was sound and should have been capable of being achieved.”

(Small, 2000:30) This chapter analysis INCIS project from 5 different perspectives in

association with 5 components of Information System Project Risk Checklist.

London Ambulance Service (LAS) Lihong Zhou

- 77 -

Chapter 9

Case Study 2: London Ambulance Service

9.1 Case Background Introduction

LAS, was started since 1930, generally consists of an Accident and Emergency

Service (A&E) and a non-emergency Patient Transport Service (PTS).

Geographically, LAS is the largest ambulance service all around the world covers

600 square miles and a resident population of over 6.8 million. LAS accept over

5000 patients daily, receives 2000 to 2500 calls everyday.

The system was considered as the first emergency service system all around the

world, especially in the system size and complexity. The LAS planed to use

computerized system completely replace the manual system. The system design was

intended to go as far as possible. The proposed system was an entirely automated

system. In this system most of calls to Central Ambulance Control could be

automatically assigned the most suitable ambulance. Human staffs were only

working on the most complex cases by manually identifying location and allocate the

best resource. It is obvious that LAS management would be largely optimized with

greater efficiency to the command and control emergency service processes.

If the INCIS project was a information system technical failure, then the London

Ambulance Service (LAS) system was a tragedy. The LAS information systems

collapse caused significant delays in ambulances reaching patients on 26 and 27

October 1992 may lead to 20 deaths. Randell (1992) shows the LAS breakdown was

the main reason of people’s death. Finkelstein (1993) concludes that LAS is not a

direct and a single reason of patients’ death based on the 26 cases considered by

coroners’ court.


- 78 -

The system was not fully tested to a satisfactory level of quality and resilience before

full implementation on 26 October 1992. The following 2 days, 26 and 27 October

1992, were not exceptionally busy days with large amount of additional emergent

incidents and patients. However, the increase incoming calls are unidentified

duplicated calls because of the ambulance delays. “The system failure was not fail in

a technical sense, since generally the system did what is had been designed to do.

However, there were certain fatal system design deficiencies which cumulatively

caused system breakdown.” (Finkelstein, 1993:5) According to the Inquiry Team’s

investigation from the report of Finkelstein (1993), “neither the Computer Aided

Despatch (CAD) system itself, nor the system users were ready for full

implementation on 26 October 1992. The CAD system was not complete, not

properly tuned and not fully tested. The resilience of the system hardware under a full

load had not been tested. The fall back option to the second file server had certainly

not been tested. There were outstanding problems with data transmission to and from

the mobile data terminals. Staff, both in Central Ambulance Control (CAC) and

ambulance personnel, had no confidence on the system and they were not well

trained on using the system. ” (Finkelstein, 1993:35) Moreover, the system users

were working in unfamiliar positions, with no paper backup. The system was unable

to provide precise and complete information.

9.2 Problem Domains Description

9.2.1 Pre-Project Unlike INCIS, pre-project work had fairly well done in LAS which could be

contributed from the first attempt of implementing information systems. LAS had 2

attempts of building computer aided dispatch systems. The first attempt started in the

early 1980s with the main aim was to supply a system without mobile data, but

including a new radio infrastructure. The project was aborted in the 1990 after load

test performance. However, the first project provided LAS a clear definition of what


- 79 -

kind of system did they want and accurately produce appropriate project scope,

objectives and user requirements.

However, the user requirement producing didn't involve the ambulance crews, which

are a large proportion of final system user. A team was assembled in order to prepare

the requirement specification for the new system. The team was led by the Director

of Support Service, and composed of a System Manager, a Contract Analyst and the

Control Room Service Manager. As the reason of problems at the time with the staff

consultation process, only union representatives were involved in the producing

requirement specification.


1. User requirement are formed only by management level without final system users.

9.2.2 Customer “LAS management had received little or no effective management training over the

years.” (Finkelstein, 1993:6) Therefore, it is believable that the management level

staff didn’t have sufficient IT knowledge related to the forth coming LAS project.

Finkelstein (1993) suggests that “new management structures will need to be

accompanied by cohesive, progressive and properly resourced management training

for all tiers of management.” (Finkelstein, 1993:52)

“Poor communications between staff and staff association and senior LAS managers

have created an atmosphere of mistrust.” (Finkelstein, 1993:52) Besides, there was

incomplete “ownership” of the LAS system by the majority of its users. Many

identified problems within the system implementation accelerated the distrust.

Before the project, LAS experienced major culture changes and management reforms.

However, “the result of the initiatives undertaken by management from 1990-92 did

not revitalize management and staff as intended, but actually worsened when was


- 80 -

already a climate of mistrust and obstructiveness.” (Finkelstein, 1993:3) Some staff

expected the project to fail rather than it to succeed. The proposed new system would

impact very significantly on the way in which staff carried on their jobs. Certain

resistance also appeared on the installation of Datatrack equipment in the ambulance

vehicles. The new system would change the operational method of how ambulance

crews used to work. It was greatly important to ensure ambulance crews’ full

collaborating. There were evidences that crews deliberately damage or misuse the

Datatrack equipment. The impact of this risk is outstanding, and it is greatly

important for managers to overcome this problem. Finkelstein (1993) claims “the

size of the programme and the speed and depth of change were simply too aggressive

for the circumstances. Management clearly underestimated the difficulties involved

in changing the deeply ingrained culture of LAS and misjudged the industrial

relations climate so that staff were alienated to the changes rather than brought on

board”. (Finkelstein, 1993:3) Hirschheim and Klein (1989) provide several

approaches in dealing with user resistance. These approaches rely on a series of

games and strategies make users eventually accept changes. Finkelstein (1993)

advices management must be willing to have regular and open consultation with staff

representatives, and persuade them to overcome their concerns on previous

management approaches, comprehend the need of change and finally accept new

ideas.

It is also questionable in selecting suppliers. In the first place, the selection is lack of

certain criteria. After advertisement, there were 35 companies expressed an interest

in providing all or part of the system. LAS generated a checklist protocol in selecting

appropriate suppliers. Each prospective supplier was evaluated and scored on a

consistent to the checklist. In order of importance, these weighting factors were:

ability to perform the tasks required; ability to handle throughput and response time;

ease of use by staff; resilience; flexibility; ability to meet time table; cost; additional

feature. However, there are evidences that LAS didn’t follow up the checklist to

choose the highest ranking company, but followed the instructions that choose the


- 81 -

supplier with the lowest price. Finally, the SO was the only company can met the

requirements from LAS, and had been selected as the prime supplier with the

optimum solution of a system consisting of Apricot, System Options (SO), and

Datatrak. In the beginning, SO was not optimistic about biding for this contract and

they were unsuccessful in biding for more basic system for the Combridgeshire

Ambulance Service. The SO price for developing LAS system was amazing cheaper

than other bidders, which was skeptical and questioned by many bidders. SO is a

well established, small software company with a good reputation amongst their many

prior customers in providing satisfied system technical quality. Finkelstein (1993)

indicates that “within the time constraints imposed on the project and the scope of

the requirement, no software house could have delivered a workable solution. ……

In taking on the LAS project, which was far larger than anything they had previously

handled, so rapidly found themselves in a situation where they became out of their

depth.” (Finkelstein, 1993:21)

External society gave constant pressure on LAS to improve performance. LAS

management expected the implementation of information systems can upgrade LAS

service level and release the pressure. On the one hand certain amount of pressures

can ensure the project be finished in time and with good quality, on the other hand

too much pressures can force the project team setup rigid project deadlines and select

unproven technologies with ambitious project objectives. “….faced with concerted

pressure from its managing RHA, MPs, the public, health service consumers and the

media over improving performance times, it is by no means certain that the Service

would have been allowed to adopt a more measured approach to introducing

changes, particularly with CAD.” (Finkelstein, 1993:6)


1. Management in the customer had received over the years little or no effective

management training.

2. Management doesn’t understand technical issues.


- 82 -

3. Mistrust between management and staff.

4. Lack of confidence on the project.

5. Inappropriately select the system supplier with ambiguous selecting criteria.

6. The supplier is lack of experience or unsuitable for the particular project.

7. Constant external pressures and unable to properly manage them.

9.2.3 Project Management

The LAS project management was quite inadequate. It was flawed in following ways:

1. The ambitious LAS project was built on an impossible time schedule and budget.

The project group meeting on 17 June 1991 identified the improper timetable, and

the meeting minutes states that the industry average project duration in terms of

similar project size and complexity as LAS was approximately 18 month, but the

LAS system was asked fully implemented by January 1992 which was no more than

6 months and non-negotiateable. The timetable didn’t allow any revision and rework

on the system, everything must be coming right in the first time. LAS management

and LAS project team had a proposed budget in mind, for the whole system, of

around £1,500,000. The time schedule and budget were fixed and unchallengeable.

They were not made by carefully estimations and thoroughly calculation. Finkelstein

(1993) warns that “it was dangerous of setting unrealistic timescale without

consultation, and commitment of, those involved; and the public and its

representatives must be prepared to allow the LAS breathing space to put its house in

order.” (Finkelstein, 1993:6)

2. LAS project team chose PRINCE Project Management Methodology. However,

neither LAS nor the suppliers had direct experience in using the methodology.

Although a course was carried on for main project members to learn how to

executive the PRINCE, it is clear that the project team didn’t have actual real project

management experience in using PRINCE. “Although certain elements of the


- 83 -

PRINCE methodology were used, at least in the initial stages, it was not used in a

properly structured way through the duration of the project.” (Finkelstein, 1993:21)

3. It is stated that the LAS system was the “World First” pioneering new system

developed by leading technologies and methodologies. It was unacceptable that there

was no a full-time committed project management team. The project team was lead

by the Director of Support Service, and involves LAS Contract Analyst, Control

Room Service Manager, Director of Operations, Training Manager, Public Affairs

Manager, CTS representative, Administrative Support, Supplier representatives. But

none of them are fully committed to the project management. Finkelstein (1993)

suggests “professional project management might have resolved disputes between

suppliers more quickly and would have identified the fact that suppliers were often

providing optimistic reports of their progress. It is probable also that a professional

project manager would have questioned and “go behind” some of the cosy

assurances on the project progress that were being given by the suppliers.”

(Finkelstein, 1993:22)

4. In LAS project, quality management largely was supervised by Project Issue

Report (PIR). All quality related problems with software or any other aspect of the

system were communicated to the suppliers by the PIR. The PIR was designed to

monitor quality control and identified quality problems. Generally, the PIR report

system was a fairly efficient way to ensure project quality. However, PIR was not a

complete quality control plan which only identified negative quality issues but

without certain solutions to associate with them. Besides, “certain procedures were

often circumvented by SO who would on occasions amend software to meet

individual user’s wishes without going through the PIR route.”(Finkelstein, 1993:33)

By the 26 October 1992, the total number, of PIR submitted development problems

and software quality issues, was 1,513. 81 of them were remaining outstanding on

that date. Among the 81 issues, 2 were considered would not function in the

operational environment until this is rectified. 44 were considered would degrade the


- 84 -

quality of service to patient. LAS required a more efficient way to manage project

quality.

5. Lack of a complete risk management strategy in LAS project. Based on University

of Sheffield (1996) risk management is extremely important to information systems

projects. Risk management provides a disciplined environment for proactive decision

making to: assess continuously what could go wrong (risks); determine which risks

are important to deal with; implement strategies to deal with those risks. In spite of

Senior management, the project team, and the lead supplier had fully committed to

the project and constantly contribute their best efforts, they insufficiently identified

the major risks that were eventually cause the project failure, which mainly as the

result of the no formal risk management.

6. Project management was difficult to be implemented due to the ambiguous

relationships between contractors. On the perspective of LAS, they prefer Apricot as

the lead contractor rather than SO. SO was a very small software house but Apricot,

was entirely owned by Mitsubishi, was a much stronger party. In SO’s mind, Apricot

was also more appropriate as the lead contractor. In this case, SO could take off some

of the pressure off them, particularly in terms of project management. However,

Apricot declared that it is corporate policy to take on such a lead position if it is in a

position to control the project itself. Finally, a decision was made that SO would be

the lead contractor whereas they were incapable to take the position. “…SO would

need to demonstrate their previous experience and their ability to project manage a

project of this size. This concern was not specifically followed up, but LAS gained

implicit confidence from the fact that Apricot themselves must have had that

confidence. In fact Apricot did no positive vetting of SO and their abilities to deliver

on the contract.” (Finkelstein, 1993:30)


1. Unrealistic and inflexible project timelines and budget.


- 85 -

2. Lack of, or choose an unfamiliar project management methodology.

3. Team members are not fully committed to the project.

4. Lack of effective risk management plan.

5. Lack of a complete quality assurance methodology.

6. Ambiguous relationships of contractors, and disagreement between involved

partners after contract.

9.2.4 Project Technical Aspect The Apricot consortium was selected as the final technical solution which included Apricot,

System Options (SO), and Datatrak. Mainly, the entire system was consisted of 3 components:

Computer Aided Despatch (CAD); Computer Map Display (CMD); Automatic Vehicle Location

System (AVLS). This system was designed interface with the existing SOLO Mobile Data

Terminals (MDTs) implemented as a part of the prior aborted system. The complete CAD system

had 7 parts, namely CAD Software, CAD Hardware; RIFS Communication Interface; Radio

System; Datatrak Sub System; GazeKeer and Mapping Software; Mobile Data Terminals.

The hardware systems were very powerful in LAS. The computer hardware is industry standard

“IBM compatible” work stations and file servers. Most of the processing was finished in

workstations and, comparatively, file servers were not playing the key roles. All workstations

were 486 based 25mhz (the most powerful chip in standard use). The workstations were using

Microsoft Windows 3.0 environment which was a powerful operation system in providing an

user friendly interface and simultaneous multi-threading function.

However, there were various fatal technical problems about the new system The

system was fully deployed on the day of kickoff with these 2 serious faults.

1. Occasionally, Mobile Data Terminals provided incorrect information.

2. The system was running in a slow speed of response. There are 3 inseparable

possible reasons according to Finkelstein (1993): “inefficient software design;

insufficiently powerful hardware; operating environment insufficiently powerful to

support the application design.”(Finkelstein, 1993:34) These 3 issues are


- 86 -

interconnected, but the inefficient software design was the major reason of system

delay. Finkelstein (1993) points out that the project development team

inappropriately selected Visual Basic as the main design tool. But Visual Basic is a

convenience tool for fast system development rather than the development of fast

systems. “SO were using an, at the time, unproven development tool designed

primarily for prototyping and the development of small, non mission critical

systems.” (Finkelstein, 1993:34)

3. Based on Finkelstein (1993)’s study, low response speed was not the only problem

about the Visual Basic. Early system failures might caused by the development

platform of a combination of Visual Basic and Windows 3.0. “Basic routines within

Windows 3.0 have resulted in some unexplained system crashes.” (Finkelstein,

1993:34)

4. The entire system was not fully installed or tested by the day of kickoff according

to Finkelstein (1993).

5. The Datatrak system has been considered to be occasionally unreliable during the

system development and implementation phases. Datatrak is arguably accurate, thus

the investment on Datatrak can be retained. “Work will need to be done on improving

the accuracy and reliability of the system, possibly involving a change to the method

used to calculate location information. This will require LAS to work in close

partnership with Datatrack.” (Finkelstein, 1993:46)



methodologies.

2. Systems are not fully tested before final implementation.

3. Unstable or incompatible software and hardware system platform.

4. Constant close partnership between the customer and the supplier.


- 87 -

9.2.5 Project Development System Analyzing and Requirement Defining:

LAS clearly knew what kind of system they want and what main functions they need.

Project scope and objectives were outstandingly defined prior the actual system

development. Basically, the proposed Computer Aided Despatch (CAD) system

included the primary command and control functions of: call taking (receiving and

verifying detailed incident information; resource identification (decide which

ambulance to send); resource mobilization (allocate the most appropriate ambulance

to the incident); ambulance resource management (managing the ambulance

equipment and crews). Project requirement specifications were also generally well

presented, though there was little involvement from ambulance crews in requirement

generating. The system requirement specification was written right after the

abandonment of the first attempt of CAD system. It is inevitable that the first project

provided an opportunity to thoroughly understand the fundamental organization

requirements and accurately define project scope.

System Designing:

A combination of Visual Basic and Windows 3.0 was chosen as the primary

developing environment. Visual Basic, at that time, was unproven development tool.

The developing platform was also without any confirmations about its precision and

stableness. Evidences proved that the platform of combination caused serious system

delay which was one of major risks of final system collapse (detail information is

available in 9.2.4). Finkelstein (1993) concludes the CAD system and its supporting

infrastructure were flowed in “a need for near perfect input information in an

imperfect world; poor interface between crews, MDTs and the system; unreliability,

slowness and operator interface.” (Finkelstein, 1993:39)

System Developing and Acceptance Testing:

The LAS project team was fully aware the leading edge nature and complexity of the

proposed system. The system was developed on an unstable and unproven platform.


- 88 -

The system was not completely installed and tested until the day of kickoff on the

26th October 1992. “The completeness and quality of system testing is in doubt,

owing to the eventual piecemeal delivery and implementation of the system. Over the

following months various system elements were tested, but never as a fully integrated

whole” (Finkelstein, 1993:23)

Both functional and quality testing was operated partially finished system in January

1992. At that time various key components were not finished. Although these key

parts were tested in the following month, the complete and integrated system test had

never thoroughly tested. Another problem about the system testing was some random

failures were difficult to be simulated and can only happen in the real time.

Finkelstein (1993) suggests LAS project team built up a consistent automatic testing

program with sufficient test data to deal with such un-manual testable problems.

System Training:

Staff training had unsatisfactorily addressed in LAS. Mainly, there are 2 main critical

risks:

1. Significant “skill decay” between the time of training and actual implementing the

system in use, though the training both for Ambulance Crew and Central Ambulance

Control was carried out well in advance of the planned implementation date. Drori

(1997) indicates that “final users training should be began after the final system is

fully tested”.

2. There were also doubts on the quality of the training delivered both from LAS’s

own work based trainers and OS. System users received different levels of training

depending on what specific part of the system they use. Besides, a key issue of LAS

failure on the CAD system was ambulance staff and CAC didn’t fully appreciate

each other’s role in providing the service to London. This problem was produced by

separate training of these 2 aspects. Certain parts of the system could be trained

jointly to both sides. Finkelstein (1993) claims “This problem was probably


- 89 -

exacerbated by totally separate training of these two functions. Greater “buy in” to

the system would have been achieved had at least certain elements of the CAD

training been given jointly to both sides. This would have enabled them each to

understand how successful operation of the system could only come about through

CAC and the crews operating in full partnership. It would also have enabled them

each to better understand the stresses that the other works under.” (Finkelstein,

1993:31)


1. User requirement are formed only by management level without final system users.

2. Unstable or incompatible software and hardware system platform.

3. Inadequately test the final integrated system before final implementation.

4. Initiate training without complete testing.

5. Lack of a user training plan and insufficient user training.

9.3 Conclusion

INCIS project might just was an outstanding system failure, but LAS was definitely a

tragedy. Not just because of high financial cost of £1.1–£1.5 million, but more

importantly 20 people may have died indirectly caused by system failure. From the

case analysis in this chapter, there were significant risks both on technical and

management aspect. However, Finkelstein (1993) claims in the final project report

“on 26 and 27 October 1992 the computer system itself did not fail in a technical

sense. Response times did on occasions become unacceptable, but overall the system

did what it had been designed to do. However, much of the design had fatal flaws

that would, and did, cumulatively lead to all of the symptoms of system failure.”

(Finkelstein, 1993:5)


- 90 -

These two case studies have a lot of similar perspectives, such as they were both the

world-first computer system, which can highlight most significant risks. However,

this research pays more attentions on the differentiations in these cases. These

differentiations greatly increase the coverage of the empirical study and provide

opportunities to evaluate the Systems Project Risk Checklist on more distinct facets.

The table below shows similarities and differentiations details of these 2 case studies: Area Topic INCIS LAS

Project Definition √ √ Pre-Project

Contract √

Management Level √ √

Internal Difficulties √

Communication Barrier √ √

Customer

System Supplier Selection

√

Project Management Methodology

√

Poor Estimation √ √ Risk Management and Quality Assurance

√ √

Project Team

Project Management

Human Resource √ √

Developing Technology and Methodology

√

Developing Platform √

Project Technical Aspect

Developing Environment

System Analysis and Requirement Defining

√ √

System Designing √


√

System Training √ √

Project Development

System Maintaining

Figure 23: Compare INCIS and LAS


- 91 -

This table clearly shows similarities and differentiations between INCIS and LAS

case studies. The table is divided into 5 main areas according to the Information

Systems Project Risk Checklist. However, this table is not intended to illustrate

details information as the risk checklist. Therefore these 5 areas are further sub-

divided into distinct topics.

In the Pre-Project area, Project Definition stand for risks of inappropriately define

project scope, objectives and requirement specification before the contract and

significantly change them after system designing. Both INCIS and LAS had risks in

Project Definition. Contract means any risks could relate to project contract. Only

INCIS project involved significant contract related risks. In the Customer area, both

INCIS and LAS management were quite support and committed to these project.

Both INCIS and LAS had serious internal and external communication barriers

which is an outstanding risk. LAS had Internal Difficulties about mistrust and lack of

confidence on the project. The system suppliers for LAS project was also considered

inappropriate. In the aspect of Project Management, apart from LAS selected

unfamiliar project management methodology PRICE, both LAS and INCIS had risks

in poor project estimation, risk management and quality assurance, and human

resource (assign inexperienced project managers). On Project Technical Aspect,

INCIS showed risk in Developing Technology and Methodology; LAS incorrectly

selected developing platform.

LAS presented a large scale of System Developing risks, however INCIS project

showed risks in System Analyzing and Requirement Defining and System Training.

As illustrated in the table, topics of Project Team in Project Management,

Development Environment in Project Technical Aspect and System Maintaining in

System Development are not covered by these 2 cases. However, these topics are

covered and compensated by literature review.

Formal Information Systems Project Risk Checklist Lihong Zhou

- 92 -

Chapter 10

Formal Information Systems Project Risk Checklist

The purpose of this chapter is to revise the initial Information Systems Project Risk

Checklist from the case studies of previous 2 chapters. In the mean time the present

the main research result, the formal Information Systems Project Risk Checklist, of

this dissertation.

New identified risks and modified risks from the case studies are marked with “*”.

10.1 Pre-Project


Pre-Project 1 Complex and unclear relationship among partners, customers and suppliers 2 Disagreement between contract involved partners 3 Failure to consider existing relationships when replacing systems 4 Failure to consider existing systems when replacing systems 5 Lack of senior management support 6 Project scope, objective and requirement specifications are ill defined 7 Resources are not allocated well 8 The customer lack of formal experience 9 Unclear payment schedule without linked milestones 10 Unsuitable business plan and vision 11 Without, or ignore, backup plans for delay or under-performance * 12 Significantly change project scope, objective and requirement specifications

after contract * 13 Inappropriate business processes, and reform them after the contract * 14 User requirements are formed only by management level without final system

users * 15 Lack measurement system for assessing and controlling project risk before

contract * 16 Lack of thoroughly comprehension on organizational nature before contract * 17 Technical framework and development methodologies are not firmly defined in

contract *


- 93 -

The project scope, objectives were experienced significant modifications which were

considered as major causations of INCIS project failure. Business Process Re-

engineering (BPR) after contract required changes in project scope, objectives and

user requirement specifications which indirectly result approximately 8 months of

project delay. In contrast, pre-project work had fairly well done in LAS which could

be contributed from the first attempt of implementing information systems. However,

the LAS’s requirement specifications were formed only by management without final

system users. In INCIS project contract, technical framework and development

methodologies were not firmly defined which could be the reason of major changes

during project development. In INCIS, the Off-Ramp enabled police to stop the

contract on particular terms and conditions, but police nearly gave up the right of

Off-Ramp.


- 94 -

10.2 Customer


Customer 1 Significantly change project scope, objectives and requirement specifications

after contract

2 Conflicts between user departments

3 Final Users are unfamiliar with the technology and require additional training

4 Internal political difficulties

5 Lack of customer support

6 Reluctance of changing environment to accept the new system

7 Resources are not well assigned to the project team

8 Unable of unifying customers perspective on the project

9 Inefficient communication between involved parties *

10 Management in the customer received over the years little or no effective

management training *

11 Management doesn’t understand technical issue *

12 Mistrust between management and staff *

13 Lack of confidence on the project *

14 Inappropriately select the system supplier with ambiguous select criteria *

15 The supplier is lack of experience or unsuitable for the particular project *

16 Constant external pressure and unable to properly manage them *

In INCIS case, various warning signs during the system development had been

proposed, but they didn’t pay enough attentions by police due to lack of efficient

communication within police. In LAS case, inefficient communication caused

several major risks: mistrust between management and staff; lack of confidence to

the project; system supplier optimistically report the system development status to

customer. Additionally, LAS inappropriately selected the system supplier, SO,

without clear selecting criteria. So also had been proven have no sufficient


- 95 -

experiences on the project in size and complexity as LAS system. External society

gave constant pressure on LAS to improve performance. LAS management expected

the implementation of information systems can upgrade LAS service level and

release the pressure. However, LAS unsuccessfully managed the pressure. The

unclear “ownership” of the LAS system by the majority of its users worsens the

relationships between departments within LAS. Significant user resistance also

existed in the LAS.


- 96 -

10.3 Project Management


Project Management 1 Lack of an effective methodology and poor estimation 2 Unrealistic and inflexible timeliness and budgets * 3 Lack of effective risk identifications and risk management plan before contract,

inadequate managing risks after contract. * 4 Milestones and related deliverables are ill-defined 5 Inexperienced team members in business or technology 6 Inappropriate structure of project team 7 Team members are not fully committed to the project 8 Too many junior staff or too many senior staff 9 Unfamiliar environment for system developers 10 Lack of process and standard 11 No single person is accountable/responsible for the project 12 Client and development staff fail to attend scheduled meeting 13 Inappropriate staffing, personnel shortfalls 14 Lack of, choose an unfamiliar project management methodology * 15 Lack of a complete quality assurance methodology * 16 Ambiguous relationships and disagreement between involved parties after

contract *

Both of INCIS and LAS project schedule and budget were inadequately defined

before contract, project estimation was poorly planned. INCIS’s project scope and

requirement significantly changed during implementation. The capped priced

contract wasn't made on associated project schedule and budget. LAS made an

unrealistic and inflexible timescale which largely increased the difficulty of the

project. There was little risk management applied to both of INCIS project and LAS

project. Project teams for both INCIS and LAS were inappropriately constructed.

INCIS didn't appoint experienced project managers in the project; LAS had no full-

time committed project management team.

Besides, for INCIS project, developing methodologies and technologies were

inappropriately selected in the first place, and subsequently experienced major

changes during the system implementation. For LAS project, project team selected

an unfamiliar project management methodology which was an outstanding risk; LAS


- 97 -

quality assurance control was ineffective and incomplete; unclear contractors’

relationships between contractors made the project management difficult to be

implemented.


- 98 -

10.4 Project Technical Aspect


Project Technical Aspect 1 Emerging or unproven, and inappropriate developing technologies and

methodologies * 2 Unstable or incompatible software and hardware system platform * 3 Unfamiliar developing technologies or methodologies for system developers * 4 Unfamiliar developing environment to the developers 5 Coding from high level requirement without thoroughly design 6 Constant close partnership between the customer and the supplier after the

project * 7 Significantly change technical infrastructure after system design

For INCIS’s technical infrastructure, Object Oriented Technology and Object

Oriented two tier client/server were emerging and unproven. Major requirements on

project functions of process manager, decision support system and portability were

difficult to be implemented with great complexity. It was also an outstanding risk to

change the technical infrastructure after system design. In LAS project, the system

supplier SO was using, at that time, an unproven development tool, a combination

software platform of Visual Basic and Windows 3.0, designed primarily for

prototyping and the development of small, non mission critical system. The final

system was neither fully installed nor tested by the day of system kickoff. The

system was running in a slow speed of response. Additionally, Mobile Data

Terminals provided additional incorrect information.


- 99 -

10.5 Project Development


Project Development Development Stages

ID Risk Factors

1 Incomplete comprehension of the current system and current situation of the organization

2 Emerging or unproven, and inappropriate developing methodologies and technologies *

3 Partially understand the user requirement 4 Significantly change project scope, objectives and requirement

specifications *

System Analyzing and Requirement Defining

5 User requirement are formed only by management level without final system users

6 Inadequate initial general planning before detailed and complex designing

7 Unstable or incompatible software and hardware system platform.*

8 Too accurate designing causes the system extremely heavy and unhealthy


System Designing

10 Poor communication and understanding among the system analyst, designer and the programmer

11 Attempting produce a complete and perfect system in one stage12 Initiate system programming when the system designing is not

completed 13 Accept testing without final users involvement 14 Inadequately test the final integrated system before final

implementation * 15 Programming without a preselected and unified methodology 16 Programming without necessary documentations 17 Lack of considering reuseable components from the current

system


18 Programmer-oriented system designing instead of user-oriented19 Lack of new system cutover methodologies 20 Lack of a user training plan and insufficient user training *

System Training

21 Initiate training without complete testing 22 Lack of a thorough maintenance plan System


In INCIS case, project scope and user requirements were not sufficiently defined

before contract. Both parties of police and IBM brutally proceed into contract

without selecting proper proven technology specification and complete project plan


- 100 -

estimations. As the reason of project scope and user requirements were incorrectly

defined before actual project implementation, significant scope and requirement

changes were made after contract. Although in LAS project the project scope,

objectives and requirement specification were well planned out before the contract,

in the system designing the project team chose an unstable developing platform. The

LAS system was tested piece by piece during the system implementation, but the

final integrated system was not fully tested. Both functional and quality testing was

operated partially finished system in January 1992. At that time various key

components were not finished. Staff training was poorly planned and implemented in

the LAS. Significant “skill decay” between the time of training and actual

implementing the system in use. The quality of the train was also doubtful.

Conclusion and Future Research Lihong Zhou

- 101 -

Chapter 11

Conclusion and Future Research

Researches on information systems failure in recent decades haven’t outstandingly

decreased the astonishing rates of failures. This research project also dedicates to

study on possible reasons and potential risks in information system failures.

11.1 Research Objectives

As noticed in the chapter 1 introduction, the main purpose of this research is to build

up a complete checklist, which can identify most common risks underlying in

information system project. The risk checklist could be used as an elementary device

for information systems developers in assessing and avoiding potential risks.

Figure 24: Diagram of revisions on Information Systems Project Risk Checklist

Two versions of information systems project risk checklist has generated from this

study. The initial version of risk checklist is constructed by risk factors identified

from recent publications on critical factors of information system failures, based on a

research foundation built by substantial literature review on information systems

implementation, nature of failure, project management and risk management. The

major part of literature review is contributed to theory and research foundation

building which provide a research platform for subsequent empirical study.


- 102 -

The formal, second, version is presented after the cases analysis by using the first

version of risk checklist. The purpose of case studies is not just attempt to apply the

checklist to real life cases, but more importantly, to significantly revise the first

version of risk checklist. Cases of Integrated National Crime Information System

Project (INCIS) and London Ambulance Service (LAS) are selected. These two cases

are both well-known information system disasters with large scale of project scopes,

high project costs and high technical complexity which provide great opportunities to

investigate into real information system failures.

The main purpose and major objectives of this research have been successfully

completed. An overview of this project is provided in the subsequent section in this

chapter.

11.2 Research Processes

The entire research thesis can be divided into two major parts as the main

methodologies of how they are studied (Literature review and Case study).

Figure 25: Dissertation Structure Map


- 103 -

As shown in the picture above, extensive review of literatures, on Information

System Implementation (Chapter 3); Nature of Failure (Chapter 4); Project

Management (Chapter 5) and Risk Management (Chapter 6), consisted the basic

research foundation.

After Chapter 1 Introduction and Chapter 2 Research Methodology which explain the

research intention and how the research is actually conducted, Chapter 3 Information

Systems Implementation introduces basic definitions and the importance of

information system. Nature of failure in Chapter 4 not only shows why study failure,

but also explains advantages of study information system implementation from its

failures. Project management and Risk management are analysis and discuss in

Chapter 5 and 6 respectively.

The Chapter 7 can be considered as the end of literature review, and the start of

empirical study. Chapter 7 presents an initial version of Information System Project

Risk Checklist from literature review. Then, the initial risk checklist significantly

revised in INCIS (Chapter 8) and LAS (Chapter 9) case studies by assessing risks

within these 2 real life environment. At then end of the chapter 9, these 2 cases are

compared. A formal Information System Project Risk Checklist, which is the main

research result of this dissertation, is presented in Chapter 10. This version of risk

checklist is significantly revised from the initial version. Detailed information and

discussions about how this checklist was revised is available in Chapter 10.

11.3 Research Methodology

This dissertation research is conducted by using an inductive methodology where

literature review and 2 cases studies based empirical study have been carried out. As

introduced and discussed in the section 2.3 Research Approach in Chapter 2,

inductive research approach is more appropriate than deductive research approach


- 104 -

for studying risk factors in information system project. Inductive reasoning starts

from problem statement and then generate conclusion from existing data which

approach uses current data to determine the key concept to be discuss. For this

dissertation, it is advantageous to use inductive reasoning as the primary research

approach.

This dissertation mainly uses secondary data source. The thesis not only employs

advantages of secondary data (efficient, easy access, high quality and permanently

available), but disadvantages are also effectively avoid. Qualitative data analyses

methodology is selected in this research.

Literature review and case study are basic research approaches of this research

project. An initial version Information Systems Project Risk Checklist has been built

by extensive literature reviews. Risks illustrated on the risk checklist were selected

from previous academic research publications. The subsequent cases studies

evaluated the risk checklist with the real life environment and provide further risks

which are not covered in literature review.

11.4 Research Result

As introduced at the beginning of the thesis, the main purpose is to produce a

thorough risk checklist as an elementary tool for information systems project

developers by extensive literature review and case studies.

The research result, the formal Information Systems Project Risk Checklist

(presented in Chapter 10), has 5 main components. “Pre-Project” shows potential

risks that project team needs to beware before the actual system implementation.

Customer fully cooperation is critically important to the project. The second part of

the checklist, “Customer”, points out risks within customers. As introduced in

Chapter 5, project management provides efficient and effective tools to manage


- 105 -

information projects, assists project team eventually successfully implement

information systems. “Project Management” part of the risk checklist lists possible

critical factors and mistakes could take place on project management aspect. “Project

Technical Aspect” concentrates on project technical issues and “Project

Development” explains project risks phase by phase.

Risks identified by the Information Systems Project Risk Checklist are from

literature review and analysis on INCIS and LAS case studies. These risks cover

nearly every aspect of information systems implementation. Empirical study

conducted in this thesis also proved that the risk checklist can efficiently and

sufficiently identify most of risks in information system projects. The risk checklist

can effectively provide elementary suggestions for information systems developers

and can be used as a risk avoiding tool.

An Information System Project Risk Checklist is also available in the Appendix I.

11.5 Research Contribution and Future Research

“After decades of experience and countless publications, most IT projects still fail to

reach a satisfactory conclusion.” (Hyde, 2002, quoted in Liu, 2002:66)

After nearly half a year of reading on information systems implementation, and three

months of investigation on information system failure, there is a large amount of

researches have been contributed to the area of information systems failure. However,

they are still insufficient to actually solve the problems of information systems

failures. These studies are critisized as incomplete and partial since they mainly

focus on most significant risks of information systems projects failures.


- 106 -

This research, instead of looking into some specific failures or deeply study several

particular risks, intends to create an Information System Project Risk Checklist. This

list highlights most of those outstanding risks which can possiblely causes

information system failures. The Information System Project Risk Checklist is

intended to be built as an handy and elementary information systems project risk

avoidance device.

The Information Systme Project Risk Checklist can be used:

a) to build project management, risk management and quality assurance strategy;

b) to directly remind system developers about significant potential risks;

c) to objectively assess information systems projects during system implementation;

d) as an effective guidance for project managers and system developers to avoid

outstanding risks or, at least, reduce negative impacts from those risks.

e) as an easy tool for developers and evaluators to evaluator current project status at

any stage and on any facet during system implementaton.

f) as a convinent device for information researchers to evaluate any cases in any

environment.

Futher research can be undertaken based on research result of this dissertation

It is necessary to continuously investigate not only cases of information systems

failures, but also implemented unsatisfactory information systems and even

successfully developments.

Risks need more than just be identified, they also require further studies to point out

approaches to deal with these risk, or in some occasions turn risks into advantages.

The Information Systems Project Risk Checklist needs to be constantly be revised.

More risks should be identified and evaluated in further researches.


- 107 -

The Information System Project Risk Checklist can be further researched in how to

coorperate with project strategic planning.

The definition of information systems is a broad conception for computer-based

information technologies. Information System Project Risk Checklist can be further

developped for some particular projects, such as information systems for hospitals,

hotels, schools, etc.

Reference Lihong Zhou

- 108 -

Reference

Ackoff, R.L. (1967)."Management misinformation systems". Management Science,14,4,

Dec.

Ackoff, R.L. (1994). "It's a mistake!" Systems Practice, 7, 3-7.

Aiken, P. (2002). "Enterprise Resource Planning (ERP) Considerations."

Argyris, C. (1971). "Management information systems: the challenge to rationality and emotionality". Management Science, 17, 6, Feb (B-275-292). Avison, D. & Fitzgerald, G. (2003). Information Systems Development: Methodologies, Techniques, and Tools. McGRAW-HILL PUBLISHING COMPANY.

Avison, D. & Shah, H. (1997). The Information Systems Development Life Cycle: A

First Course in IS. McGraw Hill.

Awad, E. (1988). Management Information Systems: Concepts, Structures &

Applications. Benjamin-Cummings Publishing Company.

Bandyopadhyay, K., Mykytyn, P,. Mykytyn, K. (1999). "A Framework for

Integrated Risk Management in Information Technology". Management Decision, 37

(5), 437-444.

Beynon-Davies, P. (1999). "Human error and information systems failure: the case

of the London ambulance service computer-aided despatch system project."

Interacting with Computers, 11, 699-720.

Beynon-Davies, P. (2002). Information System: An Introduction to Informatics in

Organizations. Palgrave.


- 109 -

Bignell, V. & Fortune, J. (1984). Understanding Systems Failures. Manchester:

Manchester University Press.

Boddy, D., Boonstra,A.& Kennedy, G. (2005). Managing Information System: An

Organisational Perspective. Second Edition. . Prentice Hall Financial Times.

Boehm, B. & DeMarco, T. (1991). "Software Risk Management". Los Alamitos.

IEEE Computer Society Press, Los Alamitos.

Boland, R. & Hirschheim, R.A. (1985). "Series foreword in Hirschheim (1985)".

British Standard Institution (1991). "Quality Vocabulary." BS4778 (Part 3 Section

3.2=IEC 1990 50(191), BSI, London.

Buckhout, S., Frey, E. and Nemec, J. Jr (1999). "Making ERP succeed: turning

fear into promise". IEEE Engineering Management Review.

Chapman, C. & Ward, S. (1997). Project Risk Management: Processes, Techniques

and Insights. John Wiley & Sons.

Colton, K.W. (1972). "Computers and police: patterns of success and failure". Sloan

Management Review, winter, 1972-73, 75-98.

Connolly, T.M., Begg, C.E. & Strchan, A.D. (1996). Database systems: A practical

approach to design, implementation and management. Addison-Wesley.

Curtis, G. & Cobham, D. (2002). Business Information Systems: Analysis, Design

and Practice. FINANCIAL TIMES PRENTICE HALL.

Dearden, J. (1972). "Mis is a mirage". Harvard Business Review, Jan-Feb, 90-99.


- 110 -

Doke, E. R. & Barrier, T. (1994). "An Assessment of Information Systems

Taxonomies: Time to Re-evaluate?" Journal of Information technology. 9, pp149-157.

Drori, O. (1997). "From Theory to Practice or How Not to Fail in Developing

Information Systems." ACM SIGSOFT. Software Engineering Notes, vol. 22, no 1.

European Foundation for Quality Management. (2005). "EFQM Framework for

Risk Management". European Foundation for Quality Management, Brussels.

Brussels.

Finkelstein, A. (1993). Report to the Inquiry Into The London Ambulance Service.

[Online]http://www.cs.ucl.ac.uk/staff/A.Finkelstein/las/lascase0.9.pdf#search=%22R

eport%20to%20the%20Inquiry%20Into%20The%20London%20Ambulance%20Ser

vice.%22[Accessed 21/6/2006]

Flowers, S. (1996). Software Failure: Management Failure-amazing stores and

cautionary tales. John Wiley&Sons.

Flynn, D. (1992). Information System Requirements: Determination and Analysis.

McGRAW-HILL BOOK COMPANY.

Forester, T. & Morrison, P. (1994). Computer Ethics 1990. Cambridge,

Massachesette: MIT Press.

Fortune, J. & Peters, G. (1995). Learning From Failure- The Systems Approach.

Johnwy Wiley & Sons.

Frosdick, S. (1997). "The Techniques of Risk Analysis are Insufficient in

Themselves." Disaster Prevention and Management. Volume 6, Number 3, 1997,

pp.165-177.


- 111 -

Galliers, R. (1992). INFORMATION SYSTEMS RESEARCH: Issues, Methods and

Practical Guidelines. ALFRED WALLER LTD.

Gibson, J., Ivancevich, J., Donnelly, J. (2000). Organizations: Behavior, Structure,

Processes. Tenth Edition. McGraw-Hill.

Goulielmos, M. (2005). "Applying the organizational failure diagnosis model to the

study of information systems failure." Disaster Prevention and Management. Volume

14, Number 3, 2005, pp.362-377.

Gray, C. & Larson, E. (2006). Project Management : The Managerial Process. The

McGraw Hill.

Hira, R. (2003). Risk Assessment of Individual Industrial Information System Projects: A case study. University of Sheffield.

Hirschheim, R.A. (1985). Office Automation: A Social and Organizational

Perspective. Chichester: John Wiley.

Hirschheim, R. and Klein, H. (1989). "Four Paradigms of Information Systems Development". Communications of the ACM, 32 (October 1989).

Holland, C. and Light B (1999). "A Critical Success Factors Model For ERP

Implementation". IEEE Software.

Holland, P., Light, B. and Gibson, N. (1999). "A critical success factors model for enterprise resource planning implementation". Proceedings of the 7th European Conference on Information Systems, 1, 273-97. Hurst, R. (2006). How Big Is It? [Online]. www.bcentral.co.uk/marking/ebusiness/how-many-people-use-the-internet-what-do-they-use-it-for.mspx [Accessed 21/6/2006]


- 112 -

Hyde, A. (2002). "Defining IT project success and failure" British Computer Society

[Online]. http://www.bcs.org.uk/review02/html/p164.htm [Accessed 20/8/2006]

Jarvinen, P.H. (2000). "Research Questions Guiding Selection of an Appropriate

Research Method". Proceedings of the 8th EuropeanConference on Information Systems

(ECIS).

Kasser, J. (1998). "What Do You Mean You Can't Tell Me if My Project is in

Trouble?" FESMA98, Antwerp, Belgium. Antwerp, Belgium.

Kliem, R. &.Ludin, I. (1997). Reducing Project Risk. Gower Publishing Limited.

Laudon, K.C & Laudon, J.P. (1991). Management Information Systems -

Managing the Digital Firm, 5th Edition. Prentice Hall.

Lichtenstein, S. (1996). "Factors in the selection of a risk assessment method".

Information Management & Computer Security, 4/4, 20-25.

Liu, F. (2002). Critical Analysis the Factors which Contribute to IS Projects Failure in

Project Management Perspective: Three Characteristic Case Studies in the UK Public

Sectors. University of Sheffield.

Lucas, H.C. Jr. (1975). Why Information Systems Fail. New York: Columbia

University Press.

Lyytinen, K. (1988). "Expectation Failure Concept and System Analysts' View of

Information System Failures: Results of an Exploratory Study". Information &

Management, 14, 45-56.


- 113 -

Lyytinen, K. & Hirschheim, R. (1987). "Information Systems failures: a survey and

classification of the empirical literature". Oxford Surveys in Information Technology,

Vol 4, 257-309.

Lyytinen, K & Robey, D. (1999). "Learning Failure in Information System

Development". Information System Journal, 9, 85-101.

McNeill, P. (1990). Research Methods. London : Routledge.

Morgan, H.L & Soden, J. V. (1973). "Understanding MIS failures". Data Base, 2,

65-93.

Nah, F., Lau,J., and Kuang, J. (2001). "Critical factors for successful implementation of enterprise systems". Business Process Management Journal, 7 (3), 285-296.

Nickerson, R. (1986). Reflections on reasoning. Hillsdale, NJ: Lawrence Erlbaum

Associates.

O'Brien, J. (2006). Introduction to Information Systems Eighth Edition [Online].

http://www.mhhe.com/business/mis/obrien/jointis8.htm#top [Accessed 25/6/2006]

Pressman, R.S. (1987). Software Engineering: A Practitioners Approach. London:

McGraw-Hill Book Company Europe.

Pressman, R.S. (1992). Software Engineering: A Practitioners Approach - European

Adaptation. Berkshire: McGraw-Hill Book Company Europe.

Randell, B. (1992). London Ambulance Service [Online].

http://catless.ncl.ac.uk/Risks/13.88.html [Accessed 12/8/2006]


- 114 -

Robson, C. (1993). Real world Research. Oxford: Blackwell.

Robson, W. (1997). Strategic Management and Information Systems: An integrated

approach. PEARSON EDUCATION LIMITED.

Rosario, J.G. (2000). "On the leading edge: Critical success factors in ERP

implementation projects". Business World.

Royal Society (1992). "Risk: Analysuis, Perception and Management, Report of a

Royal Society Study Group". The Royal Society, London.

Sauer, C. (1993). Why Information Systems Fail: A case study approach. Alfred

Waller Ltd.

Saunders, M., Lewis, P. and Thornhill, A. (2000). Research Methods for Business

Students. Second Edition. Pearson Education Limited..

Small, F. (2000). Ministerial Inquiry Into INCIS [Online]. http://www.justice.govt.nz/pubs/reports/2000/incis_rpt/INCIS%20inquiry.pdf [Accessed 1/8/2006]

Sommerville, I. (1996). Software Engineering 5th edition. Addison Wesley,

wokingham.

Stair, R. & Reynolds, G. (1999). Principles of Information Systems: A Management

Approach. Fourth Edition. . Course Technology.

Stufflebeam, D. (2000). Guidelines for Developing Evaluation Checklists: The Checklists Development Checklist (CDC) [Online]. http://www.wmich.edu/evalctr/checklists/guidelines_cdc.pdf [Accessed 25/08/2006]


- 115 -

Sumner, M (1999). "Critical Success Factors in Enterprise Wide Information

Management Systems Projects." SIGCPR’99. , New Orleans. LA. USA. New

Orleans. LA. USA.

Sumner, M (2000). "Risk Factors in Enterprise Wide Information Management

Systems Projects". SIGCPR 2000, Evanston Illinois USA. Evanston Illinois USA.

Tchankova, L (2002). "Risk identification-basic stage in risk management".

Environmental Management and Health, 12 (3).

The Standish Group (2006). Unfinished Voyages [Online]. The Standish Group

International.

http://www.standishgroup.com/sample_research/unfinished_voyages_1.php

[Accessed 25/6/2006]

Thietart, R. et al. (2001). Doing Management Research. SAGE Publication Inc.

Trochim, W. (2006). Deductive and Inductive Thinking [Online].

http://www.socialresearchmethods.net/kb/dedind.htm [Accessed 15/8/2006]

University of Sheffield. (2000). Project Management. M3F2000

Walliman, N. (2001). Your Research Project: A Step By Step Guide For The First

Time Researcher. London: SAGE Publications.

Watters, J. J. & English, L. S. (1995). "Children's application of simultaneous and

successive processing in inductive and deductive reasoning problems: Implications

for developing scientific reasoning." Journal of Research in Science Teaching, 32(7),

699-714.


- 116 -

Wee, S. (2000). "Juggling toward ERP success: keep key success factors high". ERP

News, February.

Williams, R., Bertsch, B., Dale, B,. Wiele, T. Iwaarden, J., Smith, M. and Visser,

R. (2006). "Quality and risk management: what is the key issues?" The TQM

Magazine, 18 (1).

Wolstenholme, E.F., Henderson,S. and Gavine, A. (1993). The Evaluation of

Management Information Systems: A Dynamic and Holistic Approach. Wiley.

Wright, S. and Wright, A.H. (2002). "Information system assurance for enterprise

resource planning systems: implementation and unique risk considerations". Journal

of Information technology, 16, 5-15.

Yasin, M. M. &Quigley, J. (1994). "The Utility of Information Systems: Views of

CEOs and Information System Executives." Industrial Management & Data Systems,

94, 5, pp. 25-29.

Yeates, D. and Cadle, J. (1996). Project Management for Information Systems, 2nd

edition. Pitman.

Yin, R.K. (1994). Case Study Research: Design and Methods. London: Sage

Publications

Appendix I Lihong Zhou

- 117 -

Appendix I

Information System Project Risk Checklist Information Systems Project Risk Checklist Part 1of 5

Pre-Project 1 Complex and unclear relationship among partners, customers and suppliers

2 Disagreement between contract involved partners

3 Failure to consider existing relationships when replacing systems

4 Failure to consider existing systems when replacing systems

5 Inappropriate business processes, and reform them after the contract

6 Lack measurement system for assessing and controlling project risk before

contract

7 Lack of senior management support

8 Lack of thoroughly comprehension on organizational nature before contract

9 Project scope, objective and requirement specifications are ill defined

10 Resources are not allocated well

11 Significantly change project scope, objective and requirement specifications

after contract

12 Technical framework and development methodologies are not firmly defined in

contract

13 The customer lack of formal experience

14 Unclear payment schedule without linked milestones

15 Unsuitable business plan and vision

16 User requirements are formed only by management level without final system

users

17 Without, or ignore, backup plans for delay or under-performance


- 118 -


Customer 1 Conflicts between user departments

2 Constant external pressure and unable to properly manage them

3 Final Users are unfamiliar with the technology and require additional training

4 Inappropriately select the system supplier with ambiguous select criteria

5 Inefficient communication between involved parties

6 Internal political difficulties

7 Lack of confidence on the project

8 Lack of customer support

9 Management doesn’t understand technical issue

10 Management in the customer received over the years little or no effective

management training

11 Mistrust between management and staff

12 Reluctance of changing environment to accept the new system

13 Resources are not well assigned to the project team

14 Significantly change project scope, objectives and requirement specifications

after contract

15 The supplier is lack of experience or unsuitable for the particular project

16 Unable of unifying customers perspective on the project


- 119 -


Project Management 1 Ambiguous relationships and disagreement between involved parties after

contract

2 Client and development staff fail to attend scheduled meeting

3 Inappropriate staffing, personnel shortfalls

4 Inappropriate structure of project team

5 Inexperienced team members in business or technology

6 Lack of a complete quality assurance methodology

7 Lack of an effective methodology and poor estimation

8 Lack of effective risk identifications and risk management plan before contract,

inadequate managing risks after contract.

9 Lack of process and standard

10 Lack of, choose an unfamiliar project management methodology

11 Milestones and related deliverables are ill-defined

12 No single person is accountable/responsible for the project

13 Team members are not fully committed to the project

14 Too many junior staff or too many senior staff

15 Unfamiliar environment for system developers

16 Unrealistic and inflexible timeliness and budgets

Information Systems Project Risk Checklist Part 4of 5 Project Technical Aspect

1 Coding from high level requirement without thoroughly design

2 Constant close partnership between the customer and the supplier after the

project

3 Emerging or unproven, and inappropriate developing technologies and

methodologies

4 Significantly change technical infrastructure after system design

5 Unfamiliar developing environment to the developers

6 Unfamiliar developing technologies or methodologies for system developers

7 Unstable or incompatible software and hardware system platform


- 120 -


Project Development Development

Stages

ID Risk Factors

1 Incomplete comprehension of the current system and current situation

of the organization

2 Emerging or unproven, and inappropriate developing methodologies

and technologies

3 Partially understand the user requirement

4 Significantly change project scope, objectives and requirement

specifications

System

Analyzing

and

Requirement

Defining

5 User requirement are formed only by management level without final

system users

6 Inadequate initial general planning before detailed and complex

designing

7 Unstable or incompatible software and hardware system platform.

8 Too accurate designing causes the system extremely heavy and

unhealthy


System

Designing

10 Poor communication and understanding among the system analyst,

designer and the programmer

11 Attempting produce a complete and perfect system in one stage

12 Initiate system programming when the system designing is not

completed

13 Accept testing without final users involvement

14 Inadequately test the final integrated system before final

implementation

15 Programming without a preselected and unified methodology

16 Programming without necessary documentations

17 Lack of considering reuseable components from the current system

System

Developing

and

Acceptance

Testing

18 Programmer-oriented system designing instead of user-oriented

19 Lack of new system cutover methodologies

20 Lack of a user training plan and insufficient user training

System

Training

21 Initiate training without complete testing

22 Lack of a thorough maintenance plan System


Appendix II

- 121 -

Appendix II

An Example of WBS diagram of an information systems development:

a study of building an information systems project risk checklist...

Documents