evaluating potential investments in new technologies: balancing assessments of potential benefits...

Critical Perspectives on Accounting 19 (2008) 1197–1218

Evaluating potential investments in newtechnologies: Balancing assessmentsof potential benefits with assessments

of potential disbenefits, reliabilityand utilization

Stephen Fox ∗

Technical Research Centre of Finland, P.O. Box 1000,FIN-02044 VTT, Finland

Received 29 September 2007; received in revised form 1 October 2007; accepted 11 November 2007

Abstract

In recent years, the assessment of intangible benefits has become an explicit requirement of invest-ment evaluation techniques. By contrast, assessments of three factors which can prevent the realizationof any benefits have not become an explicit requirement. Those three factors are disbenefits, reliabil-ity and utilization. The importance of these factors may sometimes be identified prior to investmentevaluation when exploratory methods such as contingency planning are used. However, evidencepresented in this paper suggests that these three factors are often overlooked. Further, the evidencepresented in this paper suggests that investment performance often suffers as a result. Accordingly,it is argued that investment evaluations need to be balanced by making assessments of disbenefits,reliability and utilization an explicit requirement. This argument is supported by reporting of experi-ences from action research. These experiences indicate that investment evaluations can be balancedby making assessments of disbenefits, reliability and utilization an explicit requirement.© 2007 Elsevier Ltd. All rights reserved.

Keywords: Investment evaluation techniques; Disbenefits; Reliability; Utilization

∗ Fax: +358 020722 6251.E-mail address: [email protected].

1045-2354/$ – see front matter © 2007 Elsevier Ltd. All rights reserved.doi:10.1016/j.cpa.2007.11.002

mailto:[email protected]

dx.doi.org/10.1016/j.cpa.2007.11.002

1198 S. Fox / Critical Perspectives on Accounting 19 (2008) 1197–1218

1. Introduction

In recent years, the assessment of intangible benefits has become an explicit requirementof investment evaluation techniques (Gerwin, 1988; Lillrank et al., 2001; Murphy andSimon, 2002; Slagmulder et al., 1995; Whiting et al., 1996). Previously, intangible benefitsmay sometimes have been identified as being important prior to investment evaluationwhen exploratory methods such as scenario analysis (Linneman and Kennell, 1977) wereused. Nonetheless, the assessment of intangible benefits is now an explicit requirement ofevaluation techniques.

By contrast, assessments of disbenefits, reliability and utilization have not become anexplicit requirement of evaluation techniques. Yet, disbenefits, reliability problems andunder utilization can prevent the realization of any benefits, whether tangible or intangible.For example, under utilization can thwart strategic initiatives which a new technology wasintended to enable. Moreover, if an investment in a new technology is not utilized, it is notlikely that it will yield a return on investment. In spite of their significance, assessments ofthese three critical performance factors are not an explicit requirement of financial evaluationtechniques. Indeed, the term, disbenefit, was not found to be present within any existingevaluation technique.

The importance of these three critical performance factors may sometimes be identifiedprior to the application of evaluation techniques when exploratory methods such as contin-gency planning (Swanson et al., 2002) are used. However, evidence presented in this papersuggests that disbenefit, reliability and utilization issues are often overlooked. Further, theevidence presented in this paper suggests that investment performance often suffers as aresult. Accordingly, it is argued that investment evaluations need to be balanced by makingassessments of disbenefits, reliability and utilization an explicit requirement. This argumentis supported by reporting of experiences from action research. These experiences indicatethat investment evaluations can be balanced by making assessments of disbenefits, reliabilityand utilization an explicit requirement.

Many alternative techniques are available to support the financial evaluation of poten-tial investments in new technologies. Existing evaluation techniques can be grouped intofour categories: traditional capital investment appraisal techniques (CIAT); adjusted CIATs;new investment evaluation techniques; mixed investment evaluation techniques (Milis andMercken, 2004). CIATs include Internal Rate of Return, Net Present Value, and ReturnOn Investment. It has been argued for some time that such techniques are not well-suitedto the evaluation of investments in technology (Dixit and Pindyck, 1994; Kumar, 1997).Adjusted CIATs include the incorporation of estimated intangible benefits from an invest-ment (Whiting et al., 1996). New techniques include Real Options Analysis (Copelandand Antikarov, 2001). Mixed techniques include adaptations of the balanced scorecard(Buglione et al., 1999; Kaplan and Norton, 1992).

Potential investments in new technologies can be difficult to evaluate (Bacon, 1992;Clemons and Weber, 1990; Keen and Digrius, 2002; Putterill et al., 1996; Shank, 1996;Thatcher and Oliver, 2001). Nonetheless, review of existing evaluation techniques suggeststhat they do little to support the assessment of certain critical performance factors. Thosebeing factors which can determine the performance of a new technology in use and, asa result, often determine returns from investments in that technology. In particular, exist-

S. Fox / Critical Perspectives on Accounting 19 (2008) 1197–1218 1199

ing techniques do little, if anything, to direct users to three critical performance factors:disbenefits, reliability, and utilization.

In this paper, potential methods for assessing critical performance factors during invest-ment evaluation are described. These potential methods were identified, developed andapplied during an action research study which was carried out with twenty companies overa period of 2.5 years. The remainder of the paper is structured in six sections. The purpose,scope and method of the research are described in Section 2. The significance of disbenefits,reliability and utilization are explained in Sections 3–5. The content of these sections pro-vides evidence which suggests that financial evaluations need to be balanced by includingassessment of these critical performance factors. In Section 6, potential assessment methodsand their application is described. In the final section, conclusions from this research anddirections for future research are presented.

2. Research purpose, scope and method

2.1. Research purpose

The purpose of action research is to influence or change some aspect of whatever is thefocus of the research (Argyris et al., 1985; Lewin, 1946; Wilkinson, 1996). The purpose ofthis action research was to change one particular aspect of investment in new technology:evaluation of potential investments.

At the outset of the research, the participating organizations had the opinion that existinginvestment evaluation techniques are not sufficient for the evaluation of potential invest-ments in new technologies. It was their opinion that prompted the action research reportedhere. It has been argued that underlying issues may not be addressed when the purpose ofresearch is influenced by participating organizations (Scheurich, 1997). In this research, forexample, the participating organizations dissatisfaction with existing investment evaluationtechniques could have arisen from their own short-comings rather than any short-comingsof the techniques. Such a possibility was considered. However, the participant organiza-tions employed professionally qualified accountants who have considerable experience ofdealing with the financial aspects of investments in new technologies.

The twenty participating organizations included building owners, building design con-sultants, building component producers, building contractors and software companies. Alltogether these organizations provided representatives from an entire industry cluster: thereal estate and construction cluster. Accordingly, they provided a valuable opportunity togain broad insights into investment evaluation. The software companies in this cluster areoften formed by people who have been working within the cluster in other types of compa-nies such as building contractors. They are formed in order to develop and offer softwaresolutions which can improve the performance of the cluster. Having considered the experi-ence and diversity of the participating organizations, it was recognized that their opinionsabout investment evaluation techniques could have some validity. Furthermore, review ofthe literature revealed that there is wider debate in the content and application of investmentevaluation techniques (Adner and Levinthal, 2004; Akalu, 2003; Chenhall, 2004; Ekstromand Bjornsson, 2003; Irani and Love, 2002; Thomas, 2001). In particular, debate about


the need for evaluation techniques to be more compatible with the reality of investmentdecision-making and investment decision consequences.

2.2. Research scope

The scope of the research overall was the evaluation of potential investments in newtechnologies. The scope of field research was the evaluation of one particular new technologyby the twenty organizations participating in the research. The research was carried outbetween October 2003 and April 2006. During this period, the participating organizationssought to evaluate potential investments in a number of different technologies. However, theyall sought to evaluate potential investments in one new technology: computer-interpretableinformation models. In particular, information models of buildings and built environments.

Such computer-interpretable models are referred to as Building Information Models orBIMs (Tse et al., 2005). A computer-interpretable model can be described as digital objectsand their relationships. An object being, a single “container” of computer code (Arms,2000) that combines data (properties) and behaviour (methods). The introduction of BIMsis intended to make it much easier to repeatedly create, simulate and analyse alternativesolutions for the design, construction and operation of buildings. Examples of BIM use areprovided in subsequent sections.

Computer-interpretable information models are not limited to buildings and built envi-ronments. They are being introduced in a variety of other sectors including healthcare andthey are a focus of research in those sectors (Boxwala et al., 2004). The overall scope ofthe research was extended by including investigation of other technologies in other sectorsduring literature reviews.

2.3. Research method

Action research can be carried out as a spiral process involving a number of stages(Bassey, 1998; Kemmis and Wilkinson, 1998). The action research involved visits to thepremises of the participating companies. The action research involved company personnelwith operational responsibilities and personnel with strategic responsibilities. The detailsreported in this paper are those which are permissible within the companies’ confidentialityagreements.

In this research, the action research involved eight stages. First, unstructured interviewswere carried out with representatives from the participant organizations. Second, a literaturereview was carried out in order to identify existing evaluation techniques which are avail-able to support the investment decision-making process. This review revealed considerabledebate about the content and application of investment evaluation techniques. By the endof this stage, the research question was confirmed as being, how can existing investmentevaluation techniques be improved upon for evaluation of potential investments in newtechnologies?

The third stage involved carrying out further interviews in conjunction with demonstra-tions of BIMs. During this stage, the significance of disbenefits, reliability and utilizationbecame apparent. The fourth stage involved literature review in order to investigate the sig-nificance of disbenefits, reliability and utilization for investments in different technologies


in different industries. Fifth, the potential applicability of existing techniques was consid-ered. This included review of literature which seeks to explain where, how and why existingtechniques can work best. During this stage, it was identified that existing techniques pro-vide little support for the assessment of future disbenefits, reliability and utilization. By theend of this stage, the research hypothesis was formulated as, existing investment evaluationtechniques can be improved upon by making assessment of potential disbenefits, relia-bility and utilization an explicit requirement during the financial evaluation of potentialinvestments in new technology.

Subsequently in the sixth stage, participants’ investment decision-making processes werechanged by including the assessment of future disbenefits, reliability and utilization. Sev-enth, participant organizations’ opinions about this change were sought. During the eighthand concluding stage, possibilities for further change to their investment decision-makingprocesses were considered.

Strategies for dealing with reactivity, respondent bias and researcher bias (Lincoln andGuba, 1985) were taken from Ahern (1999) and Padgett (1998). In particular, the impartial-ity of the research was stressed in an effort to minimize reactivity. This was considered tobe essential as the purpose of the research had been influenced by the participating organi-zations. Notably, one participant remarked at the end of an interview, “this was interesting,I thought that you were coming to persuade me to use this technology—not to listen towhat I had to say”. Further, in an effort to reduce the influence of respondent bias, partici-pants were drawn from both the strategic and the operational levels of organizations. Also,participants who had particularly strong points of view were asked to name a person whodisagreed with that point of view. Subsequently, that person was involved in the research.As a result, the range of participants was more balanced than it might otherwise have been.Reflexivity was used in an effort to identify areas of potential researcher bias. One identifiedsource of potential researcher bias was existing positive opinions about the benefits of newtechnologies.

Further it was recognized that the existing literature concerned with the evaluation of newtechnologies could be somewhat polarized with argument between advocates and opponents(Koskela and Kazi, 2003). Accordingly, it was determined that the objectivity of literatureshould be given careful consideration.

3. Potential disbenefits

In this section, findings from literature review and field study are reported. Subsequently,relevance to investment evaluation is discussed.

3.1. Literature review findings

The word, disbenefits, has a number of definitions. These include: a disadvantage (Soanesand Hawker, 2005); something disadvantageous or objectionable (Merriam-Webster, 2005);something that makes a situation disadvantageous or unfavourable (Encarta®, 2005); andundesirable effects of an investment (Bannister et al., 2001). The word, disbenefit, has beenused in connection with technology for some years (Mulvany, 1999).


An example of disbenefit is the adverse publicity which Benetton received as a resultof its investment in new Radio-Frequency Identification (RFID) technology. RFID is atechnology that enables the remote storage and retrieval of data using devices called RFIDtags/transponders. RFID tags are tiny and can be attached to, or incorporated into, manydifferent types of products. RFID tags contain antennas to enable them to receive and respondto radio-frequency queries from an RFID transceiver. RFID technology enables the trackingof moving objects (Shepard, 2005). Benetton is a manufacturer of casual clothing. Benettonembedded RFID tags in its clothing in order to improve its distribution management. Intheory, Benetton could have tracked people wearing its RFID tagged clothing. Followingadverse publicity, Benetton stated that RFID tags would not be embedded in its clothing(O’Hara, 2004). Many costs may have arisen from this disbenefit. For example, the costsof abandoning its investment in new RFID technology and the costs of repairing damagedone to the Benetton brand.

In other cases, costs may be easier to quantify. For example, in the 1980s, many carassemblers invested heavily in new automation technology. Their intention was to increaseproductivity and quality while reducing costs. However, the promised economies proved tobe false economies. Toyota, for example, found that, although new automation technologyreduced the number of assembly-line workers at its new factory, the number of maintenancepersonnel rose dramatically. Since they alone understood the automated equipment, therewas little scope for continuous improvement of productivity and quality. The disbenefitwas, in simple terms, that robots were not able figure out how to carry out work tasks morequickly to a higher quality standard. When Toyota reduced automation by over sixty percentin the mid-1990s, the disbenefit was reduced. Subsequently, productivity rose and qualitydefects fell (The Economist, 1995).

Wider consideration of more commonplace investments in new technology reveals theprevalence of disbenefits. For example, it has been argued that manufacturing resourceplanning systems (MRPII) hindered users’ efforts to develop world class standards of man-ufacturing (Maskell, 1993). Reference to news reports reveals that even a new technologyas simple as word processing has resulted in disbenefits (BBC, 2003a). In particular, orga-nizations claim that they no longer believe that they are in control of producing documents.This could seem to be a trivial disbenefit. Nonetheless, the significance of disbenefits canescalate. Consider, for example, an “Iraq intelligence” document produced by the Britishgovernment in 2003. This document was praised by the United States of America’s thensecretary of state, Colin Powell. It was then revealed that the document had been copiedand pasted from an internet article (White and Whitaker, 2003). Subsequently, the docu-ment was condemned as a sham by the world’s media. This caused some loss of credibilityto the British government. This case illustrates how what could seem to be a general andtrivial disbenefit can become a specific and significant disbenefit. Another such example isthe general use of antibiotics having yielded hardier strains of bacteria that do not respondto pharmaceutical treatment (Tenner, 1997). This disbenefit may not be significant to anindividual until that individual falls victim to a hardier strain of bacteria. Similarly, theconcept of disbenefit may not seem significant to an organization until an organization hasto abandon an investment as a result of a disbenefit. For example, it has been reportedthat the automotive industry has had a disappointing return from its investments in newautomation technology in the 1990s. Following Toyota’s lead, many automotive compa-


nies cut back automation—after they had paid huge sums of money for it (Agrawal et al.,2001).

Overall, literature review revealed that while disbenefits are widely reported in news-papers, etc., there is little discussion of their significance in scientific literature addressinginvestment appraisal.

3.2. Field study findings

During the research, participants spoke of disbenefits arising from initial investmentsin the new technology of BIMs. For example, one example of a disbenefit arose duringthe concept design of a large commercial building. The building’s architect created a BIM.The property developer instructed the architect to show the BIM to intended building ten-ants to get their feedback. When the architect met the future tenants he found that theywere happy at the prospect of moving to a new building. When the architect informedthem that they could have input to the design of the building, they became even happier.The architect showed them the BIM and they requested that some modifications weremade to the building’s design. The architect reported their requests back to the propertydeveloper. Next time the architect saw the tenants their happiness decreased rather thanincreased. Upon instruction from the property developer, the architect had to inform thetenants that the design modifications which they had requested could not be achievedwithin the development budget for the building. Accordingly, the design modificationscould not be carried out. There is research showing people responding much more stronglyto losses than to gains (Kahneman and Tversky, 1984). So it was in this case. The futuretenants were happier before they were shown the BIM. The disbenefit was the build-ing tenants’ disappointment. What costs, if any, arise from this disbenefit remain to beseen.

The property developer had instructed the architect to get input from future tenantsbecause the use of BIMs offers radical reductions to the time required to modify build-ing designs and to calculate the cost implications of modifications. However, during theresearch it became apparent that initially attractive time reductions can lead to disbenefits.For example, the use of BIMs to calculate building construction costs made the preparationof competitive estimates more difficult. This is because the use of BIMs eliminates the needfor estimators to “take-off” material quantities. Quantity take-off is an important part of themanagement of building construction because it provides the essential information for thecalculation of costs, ordering of materials, and valuations of work done. When traditionalmethods such as scale rulers are used, estimators can spend 2 or more weeks “taking off”building material quantities from two-dimensional drawings printed on paper. The timespent “taking off” quantities can appear to be a waste of time. However, it can be during thistime that estimators gain insights into the design of a building which enable them to makeinformed commercial judgments. Typically, experienced estimators will instigate designmodifications that improve construction productivity without compromising the architect’saesthetic intent. Also, estimators are able to pass on their insights to construction managerswhich can further increase on-site productivity and quality. Thus, the radical time reductionbrought about by BIMs can make it far more difficult to prepare commercially competitivecost estimates.


Fig. 1. Significance of disbenefits.

3.3. Relevance to investment evaluation

The main examples of disbenefits reported above are summarized in Fig. 1. The RFIDcase illustrates how disbenefits can lead to all of an investment in technological innovationbeing abandoned. The automation case illustrates how disbenefits can lead to the majorityof an investment in technological innovation being abandoned. The BIM cases illustratethat potential disbenefits can have much longer term consequences than potential benefits.All together, the cases described above illustrate that disbenefits can be a critical factor inthe performance of an investment new technology.

Consideration of findings from literature review and field study suggest that there maybe four reasons for incorporating assessment of disbenefits into the evaluation of potentialinvestments in new technology. As summarized in Fig. 2, these are impact, prevalence,balance and difficulty. As disbenefits have led investments to be abandoned, it can beargued that disbenefits can have a determining impact on investments. The occurrenceof disbenefits in many sectors indicates their prevalence. The incorporation of disbenefitswould introduce greater balance into investment evaluation techniques which currentlyfocus upon benefits and ignore disbenefits. The difficulty of identifying disbenefits is thefourth reason put forward. Yet, review of investment decision-making techniques indicatesthat they do little, if anything, to support the assessment of future disbenefits. Further, in theexperience of the research participants, existing techniques do not support the assessmentof future disbenefits.

In the literature, risk analysis is related to investment decision-making (Simons, 1999).Further, risk analysis techniques such as risk adjusted discount rate, certainty equivalent,sensitivity, simulation and Monte Carlo are described (Dayananda et al., 2002). However,review of such literature revealed no explicit use of the term disbenefit within such tech-niques. It could be argued that management should be expected to use wide initiative inthe evaluation of new technologies, and that management should consider disbenefits whenapplying risk analysis techniques. On the other hand, it can be argued that it is unrealistic toexpect people to apply their initiative to the identification of potential disbenefits without

Fig. 2. Disbenefits—reasons to incorporate into investment evaluation.


that being an explicit requirement of evaluation. For example, the overall success of orga-nizations such as Benetton and Toyota suggests that their management has at least averagelevels of initiative. Yet, as shown by the examples above, even these organizations havefailed to identify potential disbenefits from investments in new technologies. Similarly, theproperty developer which instructed the use of BIMs to get tenant feedback is a successfulorganization. A singular characteristic of disbenefit is that with hindsight they can appearto be obvious. Unfortunately, unlike benefits, disbenefits are not pre-defined in technologyvendors’ information or perhaps in any other information relating to a new technology.

Further, it could be argued that techniques cannot include every possible aspect of evalu-ation and that not every aspect can be explicitly stated within techniques. Yet, the assessmentof intangible benefits has been incorporated into capital investment appraisal techniques forsome years (Whiting et al., 1996). Also, new evaluation techniques have been developed toenable the assessment of intangible benefits (Lillrank et al., 2001). As disbenefits can pre-vent the realization of any benefits, whether tangible or intangible, the failure to explicitlyinclude disbenefits in such techniques suggests a lack of balance.

Perhaps it could be argued that disbenefits are not an automatic consequence of aninvestment in a new technology and therefore should not be included in evaluation. Onthe other hand, financial benefits are not an automatic consequence of investments innew technology (Hempell, 2002; Melnyk and Narasimhan, 1992; Ross and Weill, 2002;Simons and Cushing, 2003), and the assessment of potential financial benefits is a pri-mary consideration in their evaluation. Also, it could perhaps be argued that disbenefitsare not intrinsic to any new technology. Rather, disbenefits arise from management. Forexample, while Benetton has experienced disbenefits from its investment in new RFIDtechnology, other companies’ investments in RFID have been successful (Planting, 2004).However, it has been argued that financial benefits are not intrinsic to any new technology(Chesborough and Rosenbloom, 2001) and that financial benefits are often dependent uponmanagement actions (Arunachalam, 2004; Dedrick et al., 2003; OECD, 2003). Nonetheless,the assessment of potential financial benefits is a primary consideration in their evaluation.

Following consideration of all the factors described above, it became apparent thatexisting investment evaluation techniques could perhaps be improved upon by makingassessment of potential disbenefits an explicit requirement.

4. Potential reliability


The reliability problems of new technologies are widely publicized through televisionand newspapers (BBC, 2003b). One notable example is that of a man shooting his laptopcomputer in frustration after it kept crashing. Subsequently, he hung his “dead” laptop ona wall as if it were a hunting trophy (BBC, 2003c). Reliability problems are not restrictedto individuals or to SMEs operating within small budgets. For example, share trading wassuspended on the Tokyo Stock Exchange for more than 4 h on Tuesday 1st November 2005after an upgrade to its software system caused a breakdown (BBC, 2005a). A few weekslater, a malfunction in the Tokyo Stock Exchange system prevented a typing error from


being corrected. The cost of this malfunction was over $200 million (BBC, 2005b). Theseproblems at the Tokyo Stock Exchange illustrate that reliability can be a significant factorin the performance of an investment in new technology.

The need to address reliability problems is widely recognized by users of investments innew technologies. The topic of improving reliability continues to appear in general manage-ment publications (Levery, 2005). Methodologies which are intended to reduce reliabilityproblems have been widely adopted. These include condition-based maintenance (Rao,1996) and total productive maintenance (Willmott, 2000). In addition to these genericapproaches, companies offer specialist services for managing the reliability of invest-ments in new technology (Ubha, 2005). As illustrated by the references above, literaturereview revealed that while reliability problems are widely reported in newspapers, profes-sional magazines, etc., there is little discussion of their significance in scientific literatureaddressing investment appraisal.


Research participants had concerns about two types of reliability problems. These canbe described as reliability problems which are visible and reliability problems which arenot visible.

Reliability problems such as crashing computers can be described as visible. In partic-ular, participants expressed frustrations about the timing of visible reliability problems. Inthe opinion of several participants, technological innovations were unreliable at the worstpossible times. For example, when there was a pressing need to extract information from aBIM, the software would not do what it was supposed to do. Participants at an operationallevel and participants at a strategic level attributed different levels of significance to reliabil-ity problems. For example, one participant with hands on experience of trying to exchangeand share BIMs with other organizations stated, “you can do it ten times and it still doesn’twork”. By contrast, one participant leading the formulation of strategies involving BIMsstated, “technology is the easy part”.

Research participants also expressed concerned about reliability problems which canbe described as not being visible. For example, when BIMs are not used, many buildingdesign and construction processes may be unreliable. However, the parties involved can seewhere the problems are and who has to deal with them. Perhaps an architect may send anincomplete set of two-dimensional drawings. Alternatively, there may be inconsistenciesbetween printed drawings and printed schedules. By contrast, when BIMs are used noneof the parties involved may be sure what information is incomplete and/or inconsistent.Contractors are so concerned about such “invisible” reliability problems that they are actu-ally re-creating BIMs themselves. Even though this takes a considerable amount of timeand money, they find this a more reliable option than checking BIMs issued to them byarchitects.

Most striking, was the use of the same phrase by different types of research participants.Different participants said that they did not “trust” BIMs. A common concern was theallocation of financial responsibility for reliability problems during data exchange. Forexample, a structural design consultant expressed concerns that his organization wouldbe liable if information contained in a BIM was “corrupted” during transfer to another


organization. In particular, if structural steel manufacturers whose “robots” can read directlyfrom a BIM would produce many high cost steel components incorrectly.

During the period of the research, the capability of BIM enabled processes was limitedby a number of reliability problems (Cook, 2004). For example, the exchange and sharingof BIMs among organizations was hindered by interoperability problems. In particular,the location of objects within BIMs can move around when BIMs are exchanged betweenorganizations. Other limitations were that some software did not have the capability to dealwith, or even accept, original shape definitions such as curves or polygons with more thanfive sides. Also, much BIM software was unable deal with intricate building designs and/orsophisticated building components such as shading devices. Consideration of such factorsprovided an indication of what applications of BIM technology would lead to reliabilityproblems.


The cases of visible and “invisible” reliability problems described above are summarizedin Fig. 3. Together these cases illustrate that reliability problems can be a critical factor inthe performance of a new technology.

The prevalence and impact of reliability problems are widely recognized. Moreover, theconsequences of reliability problems can multiply when returns from technology invest-ments depend upon successful interoperation between organizations (Charette, 2005). Forexample, when returns from investments in BIMs depend upon successful exchange andsharing with other organizations. Increasing interoperability also makes the identifica-tion of future reliability problems more difficult. Yet, review of investment evaluationtechniques indicates that they do little to support the assessment of future reliability.Further, in the experience of the research participants, existing evaluation techniques donot support the assessment of future reliability. Techniques, such as certainty equiva-lents, could perhaps be used to model the technical aspects of reliability. Nonetheless,despite having experienced reliability problems, the research participants had not includedreliability in investment appraisals. This may have been because the term, reliability, isnot explicitly included within existing techniques. The failure to include assessment offuture reliability calls into question the balance of existing investment evaluation tech-niques. This is because existing techniques focus users’ attention on the calculationof benefits without consideration of reliability problems which can often reduce bene-fits.

Fig. 3. Significance of reliability problems.


Following consideration of all the factors described above, it became apparent that exist-ing investment evaluation techniques could be improved upon by including assessment ofpotential reliability as an explicit requirement.

5. Potential utilization


Utilization can be a significant factor in the performance of an investment in new technol-ogy. In very simple terms, if an investment in a new technology is not utilized, it is not likelythat it will yield a return on investment. Moreover, under utilization can thwart strategicinitiatives which a new technology was intended to enable. The importance of utilization ishighlighted in books (Balle and Balle, 2005) and articles (Dywer, 2005). Under utilizationis recognized as a major issue in a variety of sectors (Ellis, 2005). Overall, literature reviewrevealed that while utilization problems are widely reported in newspapers, professionalmagazines, etc., there is little, if any, discussion of their significance in scientific literatureaddressing investment appraisal.


During the research, participants spoke of their frustration that their investments in BIMswere under utilized. Participants at an operational level and participants at a strategic levelhad very different opinions about why BIMs were not being used as much as they couldbe. Differences of opinion within firms are commonplace and are recognized within estab-lished theory (Cyert and March, 1963). Nonetheless, these differences of opinion were quitestriking. Participants working at a strategic level attributed under utilization to resistance tochange. Resistance to change is a well-established management topic (Lawrence, 1969) thatis addressed by numerous books (Hultman, 1998), training materials (Larkin et al., 1998),and techniques (Kim and Mauborgne, 2003). Participants working at a strategic level sawresistance to change as being an obvious and sufficient explanation for under utilization:even when BIMs were under utilized by young technology graduates.

By contrast, participants working at an operational level had much more detailed andoriginal explanations for under utilization of BIMs. Specific reliability problems were givenas explanations for under utilization. Notably, the examples of reliability problems whichthey gave were observed on occasions when those participants were not present. Personnelworking at an operational level had a clear perspective: they would gladly use any newtechnology that made it easier for them to fulfil their operational responsibilities. Conversely,they would use any new technology that made it more difficult for them to fulfil theiroperational responsibilities—only if they had to. Operational responsibilities that werementioned included meeting deadlines and minimizing defects.

Assessment of potential disbenefits and assessment of potential reliability provided indi-cations of when use of BIMs might not make it easier for operational personnel to fulfiltheir operational responsibilities. For example, when unproductive work is necessary towork around technical limitations. Accordingly, it seemed probable that personnel might


avoid using BIMs for designs that comprised complex geometries and intricate featuresbecause of reliability problems. Furthermore, personnel such as experienced cost estima-tors might be likely to avoid using BIMs for buildings with even simple designs due totheir concerns about disbenefits. In addition to factors which had been identified during theassessment of disbenefits and reliability, other factors emerged during the assessment ofpotential utilization. For example, architects tend to define buildings as systems of spaces.By contrast, heating, ventilating, and air conditioning engineers tend to see buildings assystems of thermal zones. This type of inconsistency among industry conventions and cre-ative thought processes can lead to the repeated remodelling of BIMs. This, in turn, canresult in uncertainty about the content and status of BIMs. As a result, the use of BIMs canappear to be an unattractive alternative.

Also, a number of participants pointed out that under utilization could become worseover time. This is because knowledge of how to use BIMs will diminish if training in theuse of BIMs is not reinforced by continual practice in the use of BIMs. In the opinion ofparticipants, any assumptions that individuals’ utilization would increase from a startingpoint of low utilization were incorrect. In their opinion, potential users would forget howBIMs were supposed to be used in practice and, thus, potential users would become unableto use BIMs—whether they wanted to or not.


All together, the cases described above illustrate that utilization can be a critical factorin the performance of an investment in new technology. In very simple terms, a technologythat is not used cannot enable the realization of tangible benefits and intangible benefits.Yet, in spite of the importance of utilization problems, review of investment decision-making techniques indicates that they do little to support the assessment of future utilization.Further, in the experience of the research participants, existing techniques do not supportthe assessment of future utilization.

This may have been because the term, utilization, is not explicitly included within existingtechniques. Again, the failure to include assessment of future utilization calls into questionbalance of existing evaluation techniques. This is because existing evaluation techniquesfocus users’ attention of the calculation of benefits without consideration of utilizationproblems which can prevent the realization of net benefits.

Furthermore, it is not easy to establish reasons for not including future utilization inthe evaluation of new technologies. The prevalence and impact of utilization problems arewidely recognized. Also, the difficulty of assessing future utilization is increasing. In somecases, an investment in new technology may be fully utilized but for a shorter period oftime than was anticipated. For example, it has been reported in the semi-conductor industrythat production equipment replacement rarely occurs as a result of the end of productequipment natural life. Rather, replacement occurs as a result of the introduction of furthernew technologies or as a result of the development of new products (Peng, 2005). In othercases, the rate of utilization may fluctuate over time. For example, rather than making fulluse of BIMs, potential users could continue to use paper-based approaches and make onlylimited partial use of BIMs. The partial use of investments in new technologies was an issuethat had been raised by the research participants working at a strategic level. Further, there


are reports of the introduction of new technologies being followed by the continued, or evenincreased use, of existing technologies. For example, there is evidence that the introductionof Email has increased the use of fax machines and paper (Rubens, 2003; Sellen and Harper,2001). In theory, Email may appear to be a more attractive alternative than paper and faxfor written communication. Yet, in practice, the limitations of Email have led to fax andpaper being a better alternative in some situations.

6. Possible assessment methods

The hypothesis was formulated that existing investment evaluation techniques could beimproved upon by making assessment of potential disbenefits, reliability and utilization anexplicit requirement during the financial evaluation of potential investments in new tech-nology. Preliminary tests of this hypothesis were carried out by introducing assessments ofdisbenefits, reliability and utilization into participant organizations’ evaluation of potentialinvestments. In this section, potential assessment methods for disbenefits, reliability and uti-lization are described, and their application during the research is discussed. First, findingsfrom literature review are reported. Then, field study findings are reported. Subsequently,the relevance of different methods to investment evaluation is discussed.


Literature review was carried out in order to identify existing methods which could beused to support the assessment of critical performance factors. Established creative thinkingmethods such as negative brainstorming (West, 2003) and devil’s advocacy (Walker, 2004)were identified as being potentially useful in the assessment of disbenefits. These methodsare based on the principle of seeking out counter-arguments. Also, it was identified that well-established comparative analysis methods could be used during the assessment of potentialutilization. Many are available (Cross, 1994) including weighting, rating and ranking (Jones,1992).

Also, it was recognized that well-established process capability principles could be drawnupon during the assessment of reliability problems. Outputs from capable processes conformto requirements reliably and consistently. For example, one issue in the research was thecapability of BIM enabled steel manufacturing processes to meet accuracy requirementsreliably and consistently. If information is inaccurate, the steel fabricators “robots” wouldmanufacture structural steel columns and beams to the wrong sizes. This would lead toscrap, rework, etc. Assessment of process capability is a part of Total Quality Management(Oakland, 1993) that has its origins in statistical process control (Oakland and Followell,1990). The assessment of process capability is well-established in practice (Price, 1985).Often the assessment of process capability is made after the decision to invest in a newtechnology has been made (Pande et al., 2000). However, it has been argued that processcapability should be considered earlier (Creveling et al., 2002). The principles of FailureMode and Effects Analysis (FMEA) could also be drawn upon during the assessment offuture reliability. This is a method designed to identify potential failure modes for a productor a process (Stamatis, 2003).


It is important to distinguish between exploratory methods such as contingency planning,dependency modelling, root cause analysis and scenario analysis and focused techniquessuch as FMEA and assessment of process capability. Exploratory methods are open intheir scope and are well-suited to the types of open investigations that precede financialevaluations. By contrast, FMEA, for example, is a technique with focused scope that iswell-suited to very detailed evaluations of a particular option.


Field work involved introducing assessments of disbenefits, reliability and utilizationinto the evaluation of potential investments in new technology. Assessments were informedby the methods described in the previous sub-section. A simple formula (including theterms, disbenefits, reliability and utilization) was developed to make the assessment ofdisbenefits, reliability and utilization an explicit requirement of investment evaluation. Theformula that was developed and used is shown below. The formula is shown in order toprovide information about the action research. It is not shown to advocate its use by readersor organizations which they may work with.

Net Benefit from ICT investment = (Potential × Utilization × Reliability) − Costs

Potential = (Benefits × Potential Utilization) − (Disbenefits × Potential Utilization)

Reliability = Total Unproductive Time

Total Productive Time

In one case, assessments of disbenefits led to a recognition that the use of BIMs tocalculate construction costs could eventually lead to construction costs increasing. This isbecause, as described in Section 3.2, when BIMs are used, there is no need for estimatorsto “take-off” quantities. As a result, two or more person weeks can be saved per project.Unfortunately, inexperienced estimators are not likely to become commercially astute esti-mators just by clicking on BIM cost calculation icons. Thus, companies that invest heavilyin BIMs have the immediate benefit of saving estimators’ time doing quantity “take-offs”.However, companies could eventually find themselves having no commercially astute esti-mators. Hence, building construction costs could actually increase over time due to theintroduction of BIMs.

This example illustrates that when potential disbenefits are identified during assessment,they need to be prevented from becoming real disbenefits. This does not mean that a newtechnology should not be invested in. Rather, the potential disbenefit should be preventedby the selective and targeted use of the new technology. For example, in this case, theprior learning of commercial astute estimators can be captured within a knowledge basewhich can be used in conjunction with BIMs. Further, BIMs can be used for the rapid costcalculation of building design modifications after an initial estimate has been calculatedwith a combination of traditional and BIM procedures. Consideration of how to preventpotential disbenefits can lead to reductions in the assessment of benefit. For example, thedevelopment of a knowledge base is an additional investment which might not otherwisehave been necessary.


A number of new technology investment options were eliminated as a result of theassessments. For example, one principal contractor decided not to invest in the use of BIMtechnology for “bid and construct” projects. Those are building projects were principalcontractors receive design information from two or more external design consultants. Fol-lowing the elimination of investment options, the remaining investment options could thenbe considered using any of the existing techniques.

Making assessment of disbenefits, reliability and utilization an explicit requirement wassufficient to encourage participants to question their assumptions about potential effectsfrom technology investments. In particular, which applications of BIMs could cause thedesired effects. Participants came to realize that applications of BIMs which would likelybe thwarted by disbenefits, reliability problems and under utilization were not likely tocause the desired effects.

Assessments were not intended to produce numeric quantifications. Rather, they wereintended to facilitate identification of potential investment options that would be underminedby disbenefits, reliability problems and/or under utilization. Interestingly, the simplicity ofthe assessments was welcomed by participants from both large organizations and SMEs.It has been argued that decision-making behaviour in small firms is markedly different todecision-making behaviour in large firms. In particular, that people in small firms oftenadopt a “good enough” approach rather than attempt to achieve optimal decisions (Jarviset al., 1996). In this research, a “good enough” approach was equally valued by both well-established multinationals and newly established companies.


The assessment of disbenefits, reliability and utilization was considered to be relevantby the organizations participating in the research. However, what is relevant to these orga-nizations may not be relevant to other organizations. In particular, the research participantscan be categorized as early adopters of new technologies. Globally, only a minority of orga-nizations are included this category. Most organizations are included in the other adoptercategories: early majority, late majority, and laggards (Mahajan et al., 1990).

Further, all of the research was carried out in Finland. The World Economic Forum(2005) has ranked Finland as the world’s most competitive nation during the years 2003,2004 and 2005. Finland is ranked highly in terms Gross Domestic Expenditure on researchand development (OECD, 2004a,b). In terms of patent registrations per capita, Finland ishigh in the world’s top ten. The number of patents applications per million people in Finlandis about twice the European Union (EU) average. Finland is an EU leader in the registrationof patents in high tech sectors. During recent years, the participating organizations had beeninvolved in a number of national and international technology development projects (TheHigh Technology Finland Team, 2005). Thus, the organizations participating in the researchmay be unusually concerned about improving the evaluation of potential investments in newtechnology.

On the other hand, the assessment of disbenefits, reliability and utilization may be rele-vant beyond the evaluation of potential investments in new technology. For example, manycompanies have invested in outsourcing arrangements in order to reduce costs. Recently,it has been reported that in some cases of outsourcing, the disbenefit of reduced flexibility


has detracted from the benefit of reduced costs. This has led to some organizations bring-ing outsourced operations back in-house (Deloitte, 2005). Indeed, it could be argued thatdisbenefits, reliability and utilization are relevant to nearly all investment decisions.

Perhaps the most important issue to consider when judging relevance is the purposeof investment evaluation techniques. This is not straightforward because different evalua-tion techniques have different purposes. Nonetheless, recent adaptations and alternatives totraditional capital investment appraisal techniques have been developed to support assess-ment of some factors which are critical to the performance of an investment. For example,the alignment of investments with strategic objectives can be considered to be a criticalperformance factor (Poon and Wagner, 2001), and adaptations of the balanced scorecardsupport assessment of strategic alignment (Buglione et al., 1999). If investment evaluationtechniques should support assessment of factors which are critical to the performance of aninvestment, then assessment of disbenefit, reliability and utilization is relevant.

Further, the content of investment evaluation techniques can determine the questionswhich are asked during investment evaluation. For example, the term, intangible benefits, hasbeen incorporated in investment evaluation techniques to focus attention of the assessmentof intangible benefits (Gerwin, 1988; Murphy and Simon, 2002; Slagmulder et al., 1995).If the content of investment evaluation techniques can determine the questions which areasked during investment evaluation, then the terms disbenefit, reliability and utilizationshould be incorporated in investment evaluation techniques.

7. Conclusions and future research

7.1. Conclusions

Traditional capital investment appraisal techniques have been supplemented by a varietyof adaptations and additional techniques. These adaptations and additional techniques focuson some factors which can be critical to the performance of investments. Findings from theresearch reported here suggest that disbenefit, reliability and utilization can be critical tothe performance of investments in new technology. Examples of these factors have beenpresented from architecture, engineering and construction. Further examples have beenpresented from automotive, finance, government and retail. Also, examples of technologiesthat are used in many sectors, such as Email and fax, have been provided. All together, theseexamples indicate that disbenefits, reliability and utilization are prevalent, as well as criticalperformance factors. Yet, review of existing investment evaluation techniques suggests thatthey do little, if anything, to support the evaluation of disbenefits, reliability and utilization.For example, these terms are not present within existing techniques.

During the research, the introduction of a simple formula including these terms facilitatedidentification of potential investment options which would be undermined by disbenefits,reliability problems and/or under utilization. In recent years a number of techniques havebeen introduced which focus attention on specific factors such as intangible benefits. Theintroduction of such techniques could be balanced by including assessment of disbenefit,reliability and utilization in investment evaluation. This is necessary because these factorscan prevent the realization of any benefits. Further, the research reported here suggests that


there may be no need to develop complicated new evaluation techniques or complicatedadaptations of existing evaluation techniques. Rather, it may be sufficient to make assess-ment of disbenefits, reliability and utilization an explicit requirement of financial evaluationprior to the application of existing techniques.

It is likely that the identification and elimination of disbenefits, reliability problems andunder utilization will continue to be haphazard—if assessment of these critical performancefactors is not included as an explicit requirement of financial evaluation. This could haveincreasingly serious consequences for societies and indeed the entire World. Consider,for example, the on-going consequences that face the World from the disbenefits of onetechnology: the automobile. These disbenefits include: road rage, traffic deaths, damagedlandscapes, local pollution, and global warming.

7.2. Limitations

There are several potential limitations to this research. In particular, field study involvedonly twenty organizations in one country investing in one technology. Further, these orga-nizations can be categorized as early adoptors, and early adopters are only a minority oftechnology investors. Also, reporting is restricted by the participating organizations require-ments for confidentiality. To address these limitations, investments in other technologies inother countries were investigated.

A further limitation may be that the purpose of research was influenced by the partic-ipating organizations. This possibility was investigated at the outset of the research, andreview of the literature revealed that there is wider debate in the content and applicationof investment evaluation techniques. Reactivity, respondent bias and researcher bias werealso recognized as potential limitations at the outset of the research. The impartiality of theresearch was stressed in an effort to minimize reactivity. In an effort to reduce the influenceof respondent bias, participants were drawn from both the strategic and the operationallevels of organizations. Also, participants who had particularly strong points of view wereasked to name a person who disagreed with that point of view. Subsequently, that personwas involved in the research. As a result, the range of participants was more balanced thanit might otherwise have been. Reflexivity was used in an effort to identify areas of poten-tial researcher bias. One identified source of potential researcher bias was existing positiveopinions about the benefits of new technologies.

7.3. Future research

Further research could seek to establish the extent of relevance. For example, by investi-gating whether, or not, disbenefit, reliability and utilization are critical to the performanceof non-technology investments. Further research could also include wider application ofsimple formulae containing the terms, disbenefit, reliability and utilization. Applications ofsimple formulae could be supported by well-established methods such as devil’s advocacy.Applications could also be supported by consideration of underlying principles from qualitymanagement such as process capability.

More sophisticated procedures for assessment could be developed and tested using sys-tem dynamics modelling software packages. These could be useful in seeking to define,


simulate and analyse possible relationships between benefits, disbenefits, reliability, uti-lization and costs. Further research might then seek to determine the overall usefulness ofsimple formulae compared to more sophisticated software-enabled procedures during theearly stages of investment evaluation. Such research could provide some insight into themerit of developing more sophisticated procedures for concurrent assessment of benefit,disbenefit, reliability, utilization and cost.

References

Adner R, Levinthal D. What is not a real option: considering boundaries for the application of real options tobusiness strategy. Academy of Management Review 2004;29:74–85.

Ahern K. Ten tips for reflexive bracketing. Qualitative Health Research 1999;9:407–11.Agrawal M, Kumaresh TV, Mercer GA. The false promise of mass customization. McKinsey Quarterly 2001:3.Akalu M. The process of investment appraisal: the experience of 10 large British and Dutch companies. Interna-

tional Journal of Project Management 2003;21:335–62.Argyris C, Putnam R, MacLain-Smith D. Action science. San Francisco, CA: Jossey-Bass; 1985.Arms WY. Digital libraries. MIT Press; 2000.Arunachalam V. Electronic data exchange: an evaluation of alternative organizational forms. Accounting, Orga-

nizations and Society 2004;29:227–41.Bacon J. The use of decision criteria in selecting information systems/technology investments. MIS Quarterly

1992;16:335–53.Balle F, Balle M. The gold mine. Brookline, MA: Lean Enterprise Institute; 2005.Bannister F, McCabe P, Remenyi D. How much did we really pay for that? The awkward problem of information

technology costs. The Electronic Journal of Information Systems Evaluation 2001;5:1–11.Bassey M. Action research for improving educational practice. In: Halsall R, editor. Teacher research and school

improvement: opening doors from the inside. Buckingham: Open University Press; 1998.BBC, 2003a. Document deluge threatens firms. BBC News. November 13. Available on line: http://news.bbc.co.

uk/go/pr/fr/-/2/hi/technology/3265055.stm (accessed 16 December 2005).BBC, 2003b. Keep cool over computer hassles. BBC News. October 23. Available on line: http://news.bbc.co.uk/

2/hi/technology/3204719.stm (accessed 16 December 2005).BBC, 2003c. US man shoot laptop. BBC News. March 6. Available on line: http://news.bbc.co.uk/2/hi/world/

americas/2826587.stm (accessed 16 December 2005).BBC, 2005a. Computer glitch hits Tokyo market. BBC News. November 1. Available on line: http://news.

bbc.co.uk/go/pr/fr/-/1/hi/business/4395258.stm (accessed 22 December 2005).BBC, 2005b. Tokyo market admits system error. BBC News. December 11. Available on line: http://news.

bbc.co.uk/go/pr/fr/-/1/hi/business/4519292.stm (accessed 22 December 2005).Boxwala A, Peleg M, Tu S, Ogunyemi O, Zeng Q, Wang D, Patel V, Greenes R, Shortliffe E. GLIF3: a representation

format for sharable computer-interpretable clinical practice guidelines. Journal of Biomedical Informatics2004;37:147–61.

Buglione L, Quintano N, Reo D. Balanced IT scorecard and EFQM: a balanced approach to performance mea-surement for software intensive organizations. In: Proceedings of the 2nd European Software MeasurementConference; 1999.

Charette RN. Why software fails. New York: IEEE Spectrum; 2005.Chenhall R. Integrative strategic performance measurement systems, strategic alignment of manufacturing, learn-

ing and strategic outcomes: an exploratory study. Accounting, Organizations and Society 2004;30:395–422.

Chesborough H, Rosenbloom R. The dual edged role of the business model in leveraging corporate technologyinvestments. In: Branscomb L, Auerswald P, editors. Taking technical risks: how innovators, executives, andinvestors manage high-tech risks. Cambridge, MA: Massachusetts Institute of Technology; 2001.

Clemons E, Weber B. Strategic information technology investments: guidelines for decision making. Journal ofManagement Information Systems 1990;7(2):9–28.

http://news.bbc.co.uk/go/pr/fr/-/2/hi/technology/3265055.stm

http://news.bbc.co.uk/2/hi/technology/3204719.stm

http://news.bbc.co.uk/2/hi/world/americas/2826587.stm

http://news.bbc.co.uk/go/pr/fr/-/1/hi/business/4395258.stm

http://news.bbc.co.uk/go/pr/fr/-/1/hi/business/4519292.stm


Cook C. Scaling the building information mountain. CAD User AEC Magazine, 17, March/April; 2004.Copeland T, Antikarov V. Real options: a practitioner’s guide. London: Texere; 2001.Creveling C, Slutsky J, Antis D, Creveling M. Design for six sigma in technology and product development.

Indianapolis, IN: Prentice Hall; 2002.Cross N. Engineering design methods: strategies for product design. 2nd ed. Chichester, W Susx: John Wiley and

Sons; 1994.Cyert R, March J. A behavioural theory of the firm. New Jersey: Prentice-Hall; 1963.Dayananda D, Irons R, Harrison S, Herbohn J, Rowland P. Capital budgeting: financial appraisal of investment

projects. Port Melbourne, VIC: Cambridge University Press; 2002.Dedrick J, Gurbaxani V, Kraemer K. Information technology and economic performance: a critical review of the

empirical evidence. ACM Computing Surveys 2003;35:1–28.Dixit A, Pindyck R. Investment under uncertainty. Princeton, NJ: Princeton University Press; 1994.Deloitte. Calling a change in the outsourcing market—the realities for the world’s largest organizations. Deloitte

Consulting; 2005.Dywer J. Real jobs, imaginary numbers. Engineering Management 2005;15(5):22–5.Ekstrom M, Bjornsson H. Evaluating IT investments in construction: accounting for strategic flexibility. CIFE

Technical Report #136. Stanford, CA: Stanford University; 2003.Ellis M. GM plants vulnerable. Detroit Free Press. Detroit Newspaper Partnership: Detroit, MI, 12 November;

2005.Encarta®, 2005. Encarta® Dictionary. Microsoft Encarta®: Buffalo, NY. Available on line: http://encarta.msn.

com/dictionary /disbenefit.html (accessed 16 December 2005).Gerwin D. A theory of innovation processes for computer-aided manufacturing technology. IEEE Transactions on

Engineering Management 1988;35:90–100.Hempell T. What’s spurious, what’s real? Measuring the productivity impacts of ICT at the firm-level. Mannheim:

Centre for European Economic Research; 2002.Hultman K. Making change irresistible: overcoming resistance to change in your organization. Palo Alto, CA:

Davies-Black Pub; 1998.Irani Z, Love P. Developing a frame of reference for ex-ante IT/IS investment evaluation. European Journal of

Information System 2002;11:74–82.Jarvis R, Kitching J, Curran J, Lightfoot G. The Financial Management of Small Firms. ACCA Research Report

#49. Glasgow: Association of Chartered Certified Accountants; 1996.Jones J. Design methods. New York, NY: Van Nostrand Reinhold; 1992.Kahneman D, Tversky A. Choices, values and frames. American Psychologist 1984;39:341–50.Kaplan R, Norton D. The balanced scorecard: measures that drive performance. Harvard Business Review

1992(January–February).Keen J, Digrius B. Making technology Investments profitable: ROI roadmap to better business cases. Chichester,

W Susx: John Wiley and Sons; 2002.Kemmis S, Wilkinson M. Participatory action research and the study of practice. In: Atweh B, Kemmis S, Weeks

P, editors. Action research in practice: partnerships for social justice in education. London: Routledge; 1998.Kim W, Mauborgne R. Tipping point leadership. Harvard Business Review 2003(April).Koskela L, Kazi AS. Information technology in construction: how to realize the benefits. In: Clarke S, Coakes E,

Gordon Hunter M, Wenn A, editors. Socio-technical and human cognition elements of information systems.London: Information Science Publishing; 2003.

Kumar R. Understanding the value of information technology enabled responsiveness. The Electronic Journal ofInformation Systems 1997;13:1–8.

Larkin T, Larkin S, Strebel P. Overcoming resistance to change. Harvard Business Audio 1998(August).Lawrence P. How to deal with resistance to change. Harvard Business Review 1969(January).Lewin K. Action research and minority problems. Journal of Social Issues 1946;2:34–6.Levery M. Motivating maintenance craftsmen. Engineering Management 2005;15(3):16–9.Lillrank P, Holopainen S, Paavola T. Catching intangible IT benefits. The Electronic Journal of Information Systems

Evaluation 2001;4(1):1–11.Lincoln Y, Guba E. Naturalistic inquiry. Newbury Park, CA: Sage; 1985.Linneman RE, Kennell JD. Shirtsleeve approach to long-range plans 1977;March–April:141–50.

http://encarta.msn.com/dictionary_/disbenefit.html


Mahajan V, Muller E, Srivastava RK. Determination of adopter categories by using innovation diffusion. Journalof Marketing Research 1990;27:37–50.

Maskell B. Why MRPII has not created world class manufacturing and where do we go from here? APICS1993(September).

Melnyk S, Narasimhan R. Computer integrated manufacturing. Homewood, IL: Business One Irwin; 1992.Merriam-Webster. Merriam-Webster Online Dictionary. Springfield, MA: Merriam-Webster. Available on line:

http://www.m-w.com/dictionary/disbenefit (accessed 16 December 2005); 2005.Milis K, Mercken R. The use of the balanced scorecard for the evaluation of Information and communication

technology projects. International Journal of Project Management 2004;22:87–97.Mulvany P. Assessing biotechnology impacts. Closing speech, intermediate technology development group asso-

ciates’ day. Rugby, Warks: Intermediate Technology Development Group, 17 October; 1999.Murphy K, Simon S. Intangible benefits valuation in ERP projects. Information Systems Journal 2002;12:301–20.Oakland J. Total quality management. 2nd ed. Oxford, OXON: Butterworth-Heineman; 1993.Oakland J, Followell R. Statistical process control. 2nd ed. Oxford, OXON: Butterworth-Heineman; 1990.OECD. Seizing the benefits of ICT in a digital economy. Paris: Organization for Economic Co-operation and

Development; 2003. p. 9.O’Hara M. Chips with everything. The Guardian. Guardian Newspapers Limited: London, 24 January; 2004.OECD. OECD science, technology, and industry outlook. Paris: Organization for Economic Co-operation and

Development; 2004a.OECD. OECD information technology outlook. Paris: Organization for Economic Co-operation and Development;

2004b.Padgett D. Qualitative methods in social work research: challenges and rewards. Thousand Oaks, CA: Sage; 1998.Pande P, Neuman R, Cavanagh R. The six sigma way: how GE, Motorola, and other top companies are honing

their performance. New York, NY: McGraw-Hill; 2000.Peng K. A new era dawns. Manufacturing Engineer 2005;84(4):44–7.Planting S. Tag team poised for profits. Financial Mail. Johannesburg: The Business Media Company, 6 August;

2004.Poon P, Wagner C. Critical success factors revisited: success and failure cases of information systems for senior

executives. Decision Support Systems 2001;30:393–418.Price F. Right first time. London: Gower; 1985.Putterill M, Maguire W, Sohal A. Advanced manufacturing technology investment: criteria for organizational

choice and appraisal. Integrated Manufacturing Systems 1996;7(5):12–24.Rao B. Handbook of condition monitoring. New York, NY: Elsevier Advanced Technology; 1996.Ross JW, Weill P. Six IT decisions your IT people shouldn’t make. Harvard Business Review

2002(November):85–91.Rubens P. Fax—the technology that refuses to die. BBC News. December, 15. Available on line: http://news.bbc.co.

uk/1/hi/magazine/3320515.stm (accessed 16/12/2003); 2003.Scheurich J. Research method in the postmodern. London: Falmer; 1997.Sellen AJ, Harper RHR. The myth of the paperless office. Cambridge, MA: MIT Press; 2001.Shank J. Analysing technology investments: from NPV to strategic cost management. Management Accounting

Research 1996;7:185–97.Shepard S. RFID. New York, NY: McGraw-Hill Professional; 2005.Simons R. Performance measurement and control systems for implementing strategy. Upper Saddle River, NJ:

Prentice Hall; 1999.Simons M, Cushing K. Technology suppliers unite to try to prove the value of IT investments to business. Computer

Weekly, 29 April; 2003.Slagmulder R, Bruggeman W, Wassenhove L. An empirical study of capital budgetting practices for strate-

gic investments in CIM technologies. International Journal of Production Economics 1995;40:121–52.

Soanes C, Hawker S. Compact oxford English dictionary of current English. Oxford, Oxon: Oxford UniversityPress; 2005.

Stamatis D. Failure mode and effect analysis: FMEA from theory to execution. Milwaukee, WI: ASQ QualityPress; 2003.

http://www.m-w.com/dictionary/disbenefit

http://news.bbc.co.uk/1/hi/magazine/3320515.stm


Swanson M, Wohl A, Pope L, Grance T, Hash J, Thomas R. Contingency planning guide for information technologysystems. Washington, DC: National Institute of Standards and Technology; 2002.

Tenner E. Why things bite back. New York, NY: Vintage; 1997.The Economist. The kindergarten that will change the world. The Economist, March; 1995.Thatcher M, Oliver J. The impact of technology investments on a firm’s production efficiency, product quality and

productivity. Journal of Management Information Systems 2001;18(2):17–45.The High Technology Finland Team. High Technology Finland Report. Helsinki: The High Technology Finland

Team; 2005.Thomas R. Business value analysis: coping with unruly uncertainty. Strategy and Leadership 2001;29(2):16–23.Tse TK, Wong KA, Wong KF. The utilisation of building information models in nD modelling: a study of data

interfacing and adoption barriers. ITcon 2005;10:85–110.Ubha H. A stitch in time. Engineering Management 2005;15(3):3.Walker S. Socratic strategies and devil’s advocacy in synchronous CMC debate. Journal of Computer Assisted

Learning 2004;20:172–82.West M. Effective teamwork: practical lessons from organizational research. Oxford, Oxon: Blackwell Publishing;

2003.White M, Whitaker B. UK war dossier a sham, say experts. The Guardian. London: Guardian Newspapers Limited,

7 February; 2003.Whiting R, Davies J, Knul M. Investment appraisal for IT systems. In: Willcocks L, editor. Investing in information

systems: evaluation and management. New York, NY: Chapman and Hall; 1996.Wilkinson M. Action research for people and organizational change. Brisbane: Queensland University of Tech-

nology; 1996.Willmott P. Total productive maintenance: a route to world class performance. Terre Haute, IN: TWI Press; 2000.World Economic Forum. Global Competitiveness Report 2005–2006. Geneva: World Economic Forum; 2005.

evaluating potential investments in new technologies: balancing assessments of potential benefits...

Documents