forecasting success on large projects: developing reliable scales to predict multiple perspectives...

PA

PE

RS

Project Management Journal � DOI: 10.1002/pmj 1

INTRODUCTION �

Large projects have an impact that can go well beyond the immediatecompletion of the project (Eweje, Turner, & Müller, 2012; Xue, 2009).Success is perceived not just by the traditional view of completing thework to time, cost, and quality, but also by whether the project deliv-

ers the desired outcome—that is, to deliver to the parent organizationdesired new capabilities and business objectives (Office of GovernmentCommerce, 2007; Xue, 2009)—and whether it achieves the desired longer-term impacts (Xue, 2009), including delivering the parent organization’sstrategic objectives (Eweje et al., 2012), and desired future development ofthe business (Shenhar & Dvir, 2007). Large projects will have a wider range of stakeholders making judgments about whether the project and its output,outcome, and impact have achieved the desired objectives, and these stake-holders will make those judgments over the months, years, and evendecades following project completion.

The perception of success by a project’s stakeholders often has little to dowith whether the project was completed on time, at cost, and with thedesired quality. There are well-known cases of projects that were substan-tially late and overspent but were later perceived to be very successful. TheSydney Opera House and Thames Barrier (Morris & Hough, 1987) are twoexamples. Meanwhile, other projects have been completed on time and atcost but have left their investors dissatisfied because they have failed todeliver the desired benefits. The Sydney Cross-City Tunnel for road traffic isone example. What this illustrates is that the wretched golden triangle ofproject success (time, cost, and quality) is an inadequate indicator of projectsuccess, and that success is not just related to completion of the project’sscope of work, but also to the achievement of business objectives (i.e., theproject delivers the desired outputs, outcomes, and impacts, that differentstakeholders assess these different levels of project success, and that they doso over different time frames).

The aim of this research is to develop a set of leading performance indi-cators for large projects that can be measured during project delivery to pre-dict project success. Project success is measured not just by completion ofthe scope of work to time, cost, and quality, but also by performance of theproject’s outputs, outcomes, and impacts, and thereby the achievement ofthe desired business objectives, as assessed by different stakeholders over

Forecasting Success on Large Projects:Developing Reliable Scales to PredictMultiple Perspectives by MultipleStakeholders Over Multiple Time FramesRodney Turner, SKEMA Business School, Université Lille Nord de France, Lille, FranceRoxanne Zolin, School of Management, Queensland University of Technology, Brisbane,Australia

ABSTRACT �

Our aim is to develop a set of leading performance indica-tors to enable managers of large projects to forecast during project execution how various stakeholders willperceive success months or even years into the operationof the output. Large projects have many stakeholders whohave different objectives for the project, its output, andthe business objectives they will deliver. The output of alarge project may have a lifetime that lasts for years, oreven decades, and ultimate impacts that go beyond itsimmediate operation. How different stakeholders perceivesuccess can change with time, and so the project manag-er needs leading performance indicators that go beyondthe traditional triple constraint to forecast how key stake-holders will perceive success months or even years later.In this article, we develop a model for project success thatidentifies how project stakeholders might perceive suc-cess in the months and years following a project. We iden-tify success or failure factors that will facilitate or mitigateagainst achievement of those success criteria, and a setof potential leading performance indicators that forecasthow stakeholders will perceive success during the life ofthe project’s output. We conducted a scale developmentstudy with 152 managers of large projects and identifiedtwo project success factor scales and seven stakeholdersatisfaction scales that can be used by project managersto predict stakeholder satisfaction on projects and so maybe used by the managers of large projects for the basis ofproject control.

KEYWORDS: project success criteria; proj-ect success factors; project failure factors;leading performance indicators; stakeholders;project complexity

Project Management Journal

© 2012 by the Project Management Institute

Published online in Wiley Online Library

(wileyonlinelibrary.com). DOI: 10.1002/pmj.21289

2 Project Management Journal � DOI: 10.1002/pmj

Forecasting Success on Large ProjectsP

AP

ER

S

different timescales. The leading per-formance indicators may be (as sug-gested by Turner, 2002) measures of thesuccess criteria that can be monitoredduring project delivery, and also maybe related to the project’s success fac-tors (or failure factors) and be symp-toms that the project is on track orgoing off the rails. We expect that theremay be combinations of the leadingperformance indicators that should setthe alarm bells ringing. It may be that agiven failure factor on its own may notbe a cause for concern, but that failurefactor in combination with anothermay be a cause for concern.

We are undertaking this research toaddress the following research ques-tions:1. How can we assess the success of

large projects in a more comprehen-sive way that takes into account theviews of multiple stakeholders overmultiple time frames over the life ofthe product the project produces?

2. What measures of performance dur-ing project delivery (leading per-formance indicators) will provide avalid and reliable forecast of thisassessment during project delivery?

In order to address this, we haveundertaken several steps:

1. We developed a project success modelreflecting the perception of the per-formance of the project’s output, outcome, and impact by differentstakeholders over different timescales.

2. We identified a set of success and fail-ure factors and leading performanceindicators that can be measured bythe project manager and projectteam during project execution thatcan potentially forecast those differ-ent perceptions of performance.

3. We conducted a scale developmentstudy to evaluate the ability of thoseleading performance indicators to actas valid and reliable indicators of thefuture perception of project success.

In the next section, we review the lit-erature on project success and, based onthat, develop a new model that judgesthe success of a project by the assess-ment by various project stakeholdersabout the performance of the project’soutput, outcome, and impact in themonths and years following the project.We identify success or failure factors thatcan facilitate or mitigate against theachievement of the desired outcomes,and potential leading performance indi-cators to act as predictors of futurestakeholder perception. We describe thescale development study we conducted

and the results of that study. We identi-fied two project success factor scalesand seven stakeholder satisfactionscales that provide new measures thatcan be used to forecast the perception ofstakeholders about project success andmay be used by the managers of largeprojects as the basis of project control.

A New Model for Project SuccessIn this section, we review relevant liter-ature on project success and, based onthat, develop a more comprehensivemodel of project success that indicateshow different stakeholders perceive theperformance of the project’s output,outcome, and impact over differenttimescales by combining the work ofseveral authors.

Turner (2009, first edition 1993)identified that project success may beperceived differently by different stake-holders over different timescales (seeTable 1). He suggested that the projectparticipants, including the projectmanager, the project team, and suppli-ers, judge success on completion of theproject (at its end). The operators of the project’s output and the consumersof the product it produces judge successin the months following the end of theproject based on how well it achieves itsimmediate business objectives. The

Measure of Success Stakeholder TimescaleThe project increases the shareholder value of the parent organization Shareholders End plus years

The project generates a profit Board End plus years

The project provides the desired performance improvement Sponsor End plus years

The new asset produced by the project works as expected Owner End plus months

The new asset produces a product or provides a service that Consumers End plus monthsconsumers want to buy

The new asset is easy to operate Operators End plus months

The project is finished on time, to budget, and with the desired quality All End

The project team had a satisfactory experience working on the project Project team Endand it met their needs

The contractors made a profit Contractors End

Table 1: Different perceptions of success by different stakeholders over different timescales, after Turner (2009).


investors or financiers of the projectjudge success in the years following theend of project based on how well itachieves corporate strategy and deliv-ers desired business development.

Shenhar and Dvir (2007) extendTurner’s (2009) model (Table 2). Theyidentify five categories of project success:1. Efficiency,2. Impact on the team,

3. Impact on the customer,4. Business success, and5. Preparing for the future.

In this model, the users, consumers,and investor are in one sense swept intoone entity, called the customer. However,business success and preparing for thefuture are of interest to the investorrather than the other two. Shenhar and

Dvir (2007, like Turner, 2009) suggest thatthe criteria to the left are judged at theend of the project, those in the middle inthe months following the project, andthose to the right years later.

Westerveld (2003) and Westerveldand Gaya-Walters (2001) in their proj-ect excellence model also identifiedthat several stakeholders have an inter-est in project success (Figure 1). They

Preparation for the Efficiency Impact on Team Impact on Customer Business Success Future

Meeting schedule Team satisfaction Meeting requirements Sales New technology

Meeting cost Team morale Meeting specification Profits New market

Yield, performance, Skill Benefit to the customer Market share New product linefunctionality

Team member growth Extent of use ROI, ROE New core competency

Other defined efficiencies Team member retention Customer satisfaction Cash flow New organizational

No burnout Customer loyalty Service qualitycapability

Brand name recognitionCycle time

Organizational measures

Regulatory approval

Table 2: Model of project success, after Shenhar and Dvir (2007).

Leadership &Team

Projectmanagement

Project results

Project Organization Project Results

Feedback

Policy & Strategy Appreciation byclient

Appreciation byproject personnel

Appreciation byusers

Appreciation byindirect parties

Appreciation bycontracting partners

Environment

Resources

Contracting

Figure 1: Project Excellence Model, after Westerveld and Gaya-Walters (2002).



AP

ER

S

suggested that project success can bemeasured by:• appreciation by the client,• appreciation by the project team,• appreciation by the users,• appreciation by the contractors, and• appreciation by the other interested

parties.

While the model suggests that the dif-ferent stakeholders may appreciate suc-cess in different ways, it does not suggestthat the different stakeholders may makejudgments over different timescales.

We propose that these concepts canbe integrated into a single model show-ing how different stakeholders perceivesuccess in different ways over differenttimescales.

Different TimescalesThe Asian Development Bank (ADB)has developed a results-based monitor-ing and evaluation system for projectsit is sponsoring in China (Xue, 2009).Based on the W.K. Kellogg FoundationLogic Model Development Guide(2004), this system identifies three lev-els of results, or objectives, assessedover differing time frames (Figure 2):• Project output: The new asset deliv-

ered by the project, commissioned atthe end of the project. It is sometimes

called the project deliverable or proj-ect objective (Turner, 2009). Its suc-cessful achievement will be judged atthe end of the project.

• Project outcome: The new capabilitiesthat operation of the new asset givesto the investing organization (Office ofGovernment Commerce, 2007). Theseenable the parent organization to donew things, solve problems, or exploitopportunities to achieve its businessobjectives and generate benefit. Itssuccessful achievement can be judgedin the months after the project,although it is expected that it will pro-vide benefit for years.

• Impact: The long-term performanceimprovement that it is expected thenew capabilities will enable the parentorganization to achieve. This willenable the parent organization toattain its strategic goals (Eweje et al.,2012) and its longer-term develop-ment objectives (Shenhar & Dvir,2007). It will be judged years after theend of the project.

This model (Figure 2) focuses onproject and business objectives.Modern developments also incorporatesocial and environmental objectives. Forinstance, the triple bottom line associat-ed with sustainability (Henriques &

Richardson, 2004) includes economic,environmental, and social objectives.Many of the projects studied by Xue(2009) included social and environ-mental as well as economic objectivesin their desired outcomes and impacts.We will therefore continue to refer tothe outcome and impact and thedesired business objectives but assumethat these may include economic, envi-ronmental, and social objectives assuggested by the triple bottom line.

Different StakeholdersHenriques and Richardson (2004) andXue (2009) introduced several differenttypes of stakeholders. We propose thatbased on their work, we can identifyeight different types of stakeholders(see Table 3). We also propose that theseeight stakeholders will have differentinterests in the project’s outputs, out-comes, and impacts as shown in Table 3,and so have different ways of judgingproject success over different time-scales. Westerveld (2003) suggests onestakeholder is the client. Turner (2009)suggests that the client can be split intotwo—the sponsor and the owner—andthese we propose as the first two stake-holders:• The owner or investor: This is the per-

son or group who pays for the project.They effectively buy the project’s out-put (new asset), pay for its operationafter the project, and obtain the bene-fit to repay their investment. Turner(2009) suggests that this group is onlyinterested in the project’s impactyears after the project. Here we sug-gest that they will be interested that theproject’s output should be delivered totime and cost and with appropriate fea-tures and levels of performance so thatit repays their investment. Their inter-est in the outcome will be that theasset continues to perform, and theoperating costs and revenue will besuch that they can make a profit. Theywill also be interested in the reputa-tion of the asset (Khang & Moe, 2008)and customer loyalty so they continueto receive their revenue stream. Their

Improvedperformance

Benefit

Resources

Operation

Exploitation

Implementation

Project

Impact

Outcomes

Outputs

Figure 2: Three levels of project results, after Xue (2009).


interest in revenue, operating costs,and profit will extend over the years tothe whole life value the new asset pro-vides. Shenhar and Dvir (2007) alsosuggest that their interest in theimpact covers the new technology,competence, and capability the assetprovides. We also suggest the asset

may be the first of a new class of prod-uct. The owner may receive thefinance they use from a financier, whocould be considered a separate stake-holder. Table 1 identifies one specificfinancier: the shareholders of the par-ent organization. Another may be abank. For our purposes, we suggest

that the financier will share theowner’s interests. In PRINCE2 (Officeof Government Commerce, 2009), theowner is the senior managers whomandate the project.

• The project sponsor or project execu-tive: These are senior managers fromthe owner or user organization who,

Results Project Output Project Outcome ImpactTimescale End of Project Plus Months Plus Years

Investor or owner Time Performance Whole life valueCost Profit New technologyFeatures Reputation New capabilityPerformance Consumer loyalty New competence

New class

Project executive or project sponsor Features Performance Future projectsPerformance Benefits New technologyTime and cost Reputation New capability

Relationships New classInvestor loyalty

Consumers Time Benefit Competitive advantagePrice of benefit Price of product Price of product Features Features Features

Developments Developments

Operators/users Features Usability New technologyPerformance Convenience New capabilityDocumentation Availability New competenceTraining Reliability New class

Maintainability

Project manager and project team Time Reputation Job securityCost Relationships Future projectsPerformance Repeat business New technologyLearning New competenceCamaraderieRetention Well-being

Senior supplier (design and/or management) Completed work Performance Future businessTime and cost Reputation New technologyPerformance Relationships New competenceProfit from work Repeat businessSafety recordRisk recordClient appreciation

Other suppliers (goods, materials, works, or services) Time Reputation Future businessProfit Relationships New technologyClient appreciation Repeat business New competence

Public Environmental impact Environmental impact Whole life social Social costs cost-benefit ratioSocial benefits

Table 3: The new model of project success.



AP

ER

S

prior to the project, identify the needfor the new asset and the potentialbenefit it will bring. They persuade the investor to provide the finance for theproject and during the project willcontinue to sponsor and support theproject to win financial and politicalsupport for it. At the end of the proj-ect, they are concerned that the newasset should have the desired featuresand perform to solve the problem orexploit the opportunity identified.Their concern with time and cost willbe that the new asset should poten-tially provide the investor with a prof-it or benefit. Their concern over thecoming months will be that the newasset is performing to provide the pre-dicted benefits, and so the supportthey have given the project is justified,and they are maintaining their repu-tation (Khang & Moe, 2008) and rela-tionship with the investor (Turner &Müller, 2006). In the long term, theywill want to gain support for futureprojects, and be interested in the newtechnology and new capabilities thenew asset is providing the organiza-tion with (Shenhar & Dvir, 2007).PRINCE2 (Office of GovernmentCommerce, 2009) calls this role theproject executive.

The second stakeholder Westerveld(2003) identifies is the users. Turner(2009) differentiates between the oper-ators (the group that operates the pro-ject’s output on behalf of the owner)and the consumers (the people whobuy the product or use the service thatthe project output makes or provides,and so pays the money that provides areturn to the owner). Shenhar and Dvir(2007) include the consumers in theclient. Thus, we can identify two morestakeholders:• The consumers: These are the people

or group who buy the product the newasset produces. They effectivelyobtain the benefit from the project’soutcomes and pay for that benefit.This provides the investor with its rev-enue stream. The consumers’ interest

in the output is the time that theybegin to receive the product or bene-fit, and the price they pay for it. Theprice will reflect the cost of the projectand of operating the new asset. Theywill be buying the features the newasset provides. This interest will con-tinue throughout the life of the asset(Shenhar & Dvir, 2007; Turner, 2009).Over the years, they may also be inter-ested in whether or not the benefitprovided by the asset will providethem with a competitive advantage.

• The operators or users: These are thepeople or group who operate the asseton behalf of the owner. On comple-tion, they will be interested in the fea-tures and performance of the asset,and in the documentation and train-ing they are given. During early opera-tions, their interest will be in theusability and convenience of the asset,and its availability, reliability, andmaintainability (ARM). Over the years,they will be interested in the newtechnology, capability, competence,and class (Shenhar & Dvir, 2007).PRINCE2 (Office of GovernmentCommerce, 2009) only identifies“users” and is primarily referring tothis group of people.

Westerveld (2003) identifies theproject team as a stakeholder. Shenharand Dvir (2007) also identify the impacton the team. This introduces the fifthstakeholder group.• The project manager and project team:

At the end of the project, they are ofcourse concerned with achieving thetriple constraint, whether the workwas completed to time and cost andthe new asset performs. However, theywill also be concerned with theirlearning from the project and thecamaraderie from working on it, their future career moves, and their personal well-being (Reid, 2007;Turner, Huemann, & Keegan, 2008;Turner, Keegan, & Crawford, 2003). Inthe months following the project, theywill be concerned with the reputationof their work (Khang & Moe, 2008) and

the maintenance of relationships andwhether they get repeat business(Turner & Müller, 2006). Over theyears, they will also be concerned withtheir job security (Turner et al., 2008),their future projects, and the develop-ment of new technology and compe-tence (Shenhar & Dvir, 2007).

Project management is often writ-ten from the perspective of the clientonly (Shenhar & Dvir, 2007). However,as identified by Westerveld (2003) andTurner (2009), the contractor or suppli-er has an interest in project success. Weidentify two types of suppliers: themain contractor and subcontractors ormaterial suppliers.• The senior supplier: This group is

senior management in the lead con-tractor. They may be from within theengineering or information systemsdepartment of the owner organiza-tion; they may be the consultant inthe traditional (FIDIC, remeasure-ment) contract, or they may be a man-aging or prime contractor (Turner,1995, 2003). At the end of the project,they are concerned that the work ofthe project should be completed totime and cost and that they will havemade a profit from the work. They willalso be interested in the safety recordand risk record for the project. Duringoperation, they will be concerned thatthe asset will perform as expected, tomaintain their reputation as a primecontractor (Khang & Moe, 2008) andso they will maintain client or investorloyalty (Turner & Müller, 2006). In theyears following, they will be interested inthe new technology, competence, capa-bility, and class (Shenhar & Dvir, 2007)and whether the success of this projectincreases the chance of future proj-ects. PRINCE2 (Office of GovernmentCommerce, 2009) calls this role the sen-ior supplier, and Turner and Keegan(2001) and Turner (2009) call this rolethe steward.

• Other suppliers: These are people orgroups who provide goods, materials,works, or services. Immediately after


the project, they will be concernedwith whether the project finished ontime so that they get paid promptly,and whether they made a profit. Overthe coming months, their interest willbe in their reputation (Khang & Moe,2008) and repeat business (Turner &Müller, 2006). Over the years, they willcontinue to be interested in repeatbusiness (Turner & Müller, 2006) andthe development of new technology(Shenhar & Dvir, 2007)

The last group of stakeholders isidentified by Westerveld (2003) but notby Shenhar and Dvir (2007). Westerveld(2003) refers to “other interested par-ties.” Turner (2009) refers to bystandersbut does not include them in Table 1.We focus in particular on the public.• The public: The last stakeholder we

consider is the public. Their concernthroughout the life of the asset will bewith environmental and socialimpacts (Atkinson, 1999). If the proj-ect is publicly funded, they may alsobe concerned about whether it is rep-resenting value for money, so thatthey know that their taxes have beenwell spent.

Forecasting Project SuccessWe have proposed that the success ofthe project can only be fully evaluatedby the stakeholders in the months andyears following completion of the proj-ect (Table 3). However, the projectmanager and project team have tomake judgments during project execu-tion about whether the project will besuccessful, and inevitably they focus onthe immediate goals: whether the workwill be completed to time, cost, andquality. However, to truly forecastwhether the project will be successful,they need to be able to predict how thevarious stakeholders will judge successin the months and years following theproject, and so they need controlparameters (leading performance indi-cators) that will help them to make thatjudgment. Our aim is to develop lead-ing performance indicators, metrics

that can be used to predict project suc-cess as suggested by Table 3 during thelife of the project. There are at least tworeasons for doing this:1. So that the project team can take

control decisions during project exe-cution to direct the project towardachievement of project success asjudged by the various stakeholdersin the months and years followingthe project, and not (just) to com-plete the work to time, cost, andquality.

2. So that the project team can identifyas early as possible if it is unlikelythat they will be able to achieve theproject goals within the range ofresources the stakeholders are willingto commit. In other words, if the proj-ect goals cannot be achieved, the soon-er the project is cancelled, the fewerresources will be wasted.

Turner (2002) suggested that lead-ing performance indicators will bemeasures of the success criteria thatcan be monitored during project execu-tion. However, Andersen, Birchall,Jessen, and Money (2006) showed thatthere is a direct relationship betweenproject success (or failure) factors andthe achievement of project success.Similar work was done by Yu, Flett, andBowers (2005), Jacobson and Choi(2008), and Khang and Moe (2008).Based on the work of these authors andthe contents of Table 3, we have devel-oped a list of success or failure factorsthat will facilitate or mitigate againstthe achievement of the success criteriaas perceived by the different stakehold-ers, and a list of indicators of stakeholdersatisfaction that are premeasures of the stakeholders’ success criteriashown in Table 4. The elements of Table4 are the items that formed the basis ofour scale development study. The suc-cess and failure factors were the basis ofour project success factor scales, andthe leading performance indicatorswere the basis of our project managers’assessment of stakeholder satisfactionscales.

Scale Development StudyWe conducted a scale developmentstudy to determine whether the ele-ments of Table 4 could be used to devel-op reliable measures to monitor projectsuccess factors and the project man-agers’ perceptions of stakeholders’ sat-isfaction. Although the items in thescales had been previously tested byresearch, and research indicates thatproject managers understand stake-holders’ perceptions on importantissues (Zolin, Cheung, & Turner, 2012),there was no certainty that the projectmanagers’ assessment of the stake-holders’ satisfaction would form a reli-able scale. Therefore, we needed toretest the reliability of all scales, whichwould also provide a smaller set ofmeasures that can be used during theproject to predict project outputs basedupon stakeholder satisfaction.

Scale Development ProcessIn developing new scales and the ques-tions used to measure them, we fol-lowed DeVellis’s eight steps for scaledevelopment. The goal of the factoranalysis was to condense the largenumber of items into a smaller numberof reliable scales. Our aim was to devel-op two scales: a project success factorscale and a stakeholder satisfactionscale. The project success factor scalewas populated with items from the sec-ond column of Table 4, which was con-structed from the following nineinput scales containing 31 items. Foreach scale, we show the number ofitems and a sample question:1. Rich project information (six items):

Project participants have an openand efficient way of informing eachother as necessary.

2. Stakeholder endorsement of projectplans (four items): All the organiza-tions involved in the project efforthave agreed to provide the projectwith sufficient resources.

3. Well-structured and formal projectapproach (four items): The project hasits own management plan for control,which is used in an appropriate way.


AP

ER

S

Success and Failure Factors Stakeholder Satisfaction IndicatorsProject Stakeholders (Jacobson & Choi, 2008) (Yu et al., 2005)

Investor or owner Clear and accepted purpose Satisfaction with specificationsSpecific plan Relationship with prime contractorOpen communications Prototype performanceStakeholder endorsement Earned valueEarly stakeholder influence Net project execution costInterested owner(Andersen et al., 2006; Khang & Moe, 2008; Turner & Müller, 2004)

Project executive or project sponsor Open communications Stakeholder satisfactionPolitical support Efficiency and effectiveness

ProfitsStrategic goalsOrganizational learning

Consumers Clear specifications Satisfaction with specificationsOpen communications Relationship with sponsorAcceptance Prototype performance(Pinto & Slevin, 1988)

Operators/users Clear specifications Satisfaction with specificationsCommitment Prototype performanceOpen communications(Andersen et al., 2006)

Project manager and project team Clear and accepted purpose Pride in workSpecific plans Job satisfactionCommitment Recognition Open communications Personal growthRespect and trust Skill growthCollaboration ContactsPolitical support Reputation Expert advice and review Top management support Risk awareness RetentionClear roles and responsibilities MoraleLeadership style Stress, frustration, and time pressure(Müller & Turner, 2007; Pinto & (Bryde, 2005; Turner et al., 2008)Slevin, 1988; Turner, 2009)

Senior supplier (design and/or management) Open communications Managed riskRisk awareness Safety recordRespect and trust Stakeholder satisfactionCollaboration Efficiency and effectiveness

Contract complianceProfitsStrategic goalsOrganizational learningReduced waste(Atkinson, 1999; Shenhar & Dvir, 2007)

Other suppliers (goods, materials, works, Commitment Business goalsor services) Open communications Contract compliance

Respect and trust ProfitCollaboration (Atkinson, 1999; Shenhar & Dvir, 2007)

Public Transparency Opportunity costAccountability Social impactsCommunity outreach Environmental impactsPolitical support (Atkinson, 1999; Yu et al., 2005)

Table 4: Project success and failure factors and indicators of stakeholder satisfaction for the project stakeholders in Table 3.


4. Strong project commitment (fiveitems): The project executors con-form to the planned cost schedule forall activities.

5. Early stakeholder influence (threeitems): All key participants have beenengaged in producing the businessplan, or have had the opportunity toinfluence it.

6. Well-understood and accepted projectpurpose (three items): If the projectfulfills its goals, the results will be ofgreat value to the end users.

7. Clear project constraints (threeitems): The final date of project com-pletion is clearly defined for thisproject.

8. Project execution flexibility (twoitems): The project goal and its termscan be changed if conditions make itnecessary.

9. Influence over ongoing project process-es (two items): All participants caninfluence both decision making andresponsibility allocation in the project.

The stakeholder satisfaction scalewas populated with items from thethird column of Table 4, which wasconstructed from the following nineinput scales containing 60 items:1. Project manager (satisfaction, etc.)

(ten items): Has the project providedyou with pride in your work?

2. Project owner (six items): Has appro-priate specifications? (they are satis-fied with them)

3. Project customer (four items): Hasappropriate specifications? (they aresatisfied with them)

4. Project operator (four items): Hasdeveloped appropriate documenta-tion and training?

5. Project executives (ten items): Hasallowed the project executive/projectsponsor to profit?

6. Project contractor (nine items): Hasdemonstrated contract compliance?

7. Project supplier (three items): Helpsthe supplier achieve its appropriatebusiness goals?

8. Public stakeholders (three items): Iscreating good value for money?

9. Project participants (eleven items):The project participants have oppor-tunities to learn from this project.

The survey contained a larger num-ber of items that are relevant to the con-tent of interest so it can function as arich source from which scales canemerge. We also included some addi-tional questions in the questionnaire tohelp detect various response biases(such as negative affect) and determinethe validity of the final scale. In mostcases, a 5- or 7-point Likert scale wasemployed.

Pilot StudyFirst we conducted a pilot study, the pur-pose of which was to ascertain the read-ability and appropriateness of surveyquestions. The pilot study involved eightproject managers or program directorsfrom the Australian Defence MaterielOrganisation (DMO) and was conductedin a telephone interview format. Thequestionnaire was refined based on the comments received. The final ques-tionnaire was again checked by twoDMO staff at the senior managementlevel and management support levelprior to execution for the main study.

Main StudyAn online survey tool was used. Thestudy involved project managers orprogram directors nominated by theDMO. A total of 237 DMO project man-agers or program directors were invitedto participate in the questionnaire sur-vey. After two e-mail reminders weresent, we received 152 completed ques-tionnaires, so the overall response ratewas about 64%. The 152 returned ques-tionnaires consisted of 50 respondentsfrom air projects, 40 from joint projects,31 from land projects, and 31 from seaprojects.

Factor AnalysisExploratory factor analysis was used toreduce the number of items in the twoscales. Cronbach’s alpha was used to ascertain the construct validity andreliability of the two scales. Questions

that detract from the Cronbach’s alphameasure of the scale, or those that donot load optimally on the scale’s factor,were dropped from the scale. Our cut-off was an alpha of 0.70, but two veryclose items were retained in case theseitems might score higher in futureresearch. We conducted a rotatedVarimax factor analysis using all suc-cess factor items. We repeated theprocess using the perceptions of stake-holder success items. These analyseswere conducted using Eigenvaluesgreater than 1. This analysis convergedwithin 11 iterations. We then took eachfactor and tested to ensure they loadedon one factor and, if so, calculated theCronbach’s alpha. Where a scale loadedon more than one factor, we tested theloadings and Cronbach’s alpha for eachsubscale. The resulting project successfactor scales are reported in Table 5,and the stakeholder satisfaction scalesare reported in Table 6.

ResultsWe found two project success factorscales, which each loaded on only onefactor and had a Cronbach’s alpha high-er than 0.7. Based upon the items ineach scale, we chose appropriatenames for the scales. The scale’s alphais provided in parentheses:1. Project Planning (0.850)2. Stakeholder Engagement (0.822)

We also found seven stakeholdersatisfaction scales:1. Stakeholder satisfaction (0.914)2. Project executive satisfaction (0.907)3. Product satisfaction (0.887)4. Product efficiency (0.924)5. Satisfaction with specifications

(0.877)6. Project manager satisfaction (0.875)7. Contractor satisfaction (0.948)8. Supplier profitability (0.947)9. Public stakeholder satisfaction

(0.951)

DiscussionLarge, complex projects often fail tomeet project output objectives of time,



AP

ER

S

cost, and quality. In the past, this hasbeen considered project failure,although stakeholders may still reaphuge benefits over time, thus categoriz-ing the project as a success in the longrun. Likewise, other projects finish ontime and at cost, and so are said to be asuccess, but fail to deliver the desiredbenefits to the stakeholders. This indi-cates that the iron triangle of TCQ (time,cost, and quality) is an indicator of proj-ect performance but not the best meas-ure of project success. Thus, to fairly andaccurately evaluate project success, oneneeds to consider the views of multiplestakeholders over multiple time frames.

We developed a model of projectsuccess that addressed the views ofmultiple stakeholder groups from theend of the project (outputs) to shortlyafter the project (outcomes) to thelonger term (impacts). We conducted ascale development study to determinewhat project success factors and stake-holder satisfaction indicators thatcould be measured during the life of theproject could form reliable scales to

predict project outputs. We found onlytwo project success factor scales: projectplanning and stakeholder engagement.The project planning scale measures theexistence of well-established informa-tion and communication routines andtools, key project information gatheredand distributed efficiently, and appro-priate project control. Stakeholderengagement indicates that stakeholdershave been given the opportunity to airtheir views, influence the project plans,and know what has been decided.

In contrast, we found nine stakeholder satisfaction indicators. Stake-holder satisfaction addresses the majorgoals of the owner, executive, contrac-tor, and suppliers. A high score on thisscale would indicate that these centralstakeholders are in general satisfiedwith the success of the project. The proj-ect executive satisfaction scale indicatesthe project executive is satisfied withissues like stakeholder satisfaction, proj-ect performance, efficiency and effective-ness, and achievement of the executive’sgoals. Product satisfaction indicates the

customer and operator prototypeworks and the product performs well.Product efficiency indicates that theowner, contractor, and customer thinkthe prototype works and the productperforms efficiently. Satisfaction withspecifications indicates the owner andcustomer and operator are satisfiedwith the specifications for the product.Project manager satisfaction indicatesthat the project manager gets pride,satisfaction, recognition, growth, con-tacts, and top management support.Contractor satisfaction indicates thecontractor believes stakeholders will besatisfied, the project is effective, andthere has been compliance with thecontract. Supplier profitability indi-cates that the suppliers have beenallowed to make a profit. Public stake-holder satisfaction indicates that, fromthe public perspective, the project hasbalanced social costs and benefits andhas acceptable environmental impacts.

Longitudinal research is needed todetermine if these success factors and indicators predict project outputs,

Input Scale Scale Name • Items Alpha Loading

1. Success in Project Planning 0.850Rich Project Information • The project has well-established information and 0.778

communication routines.Rich Project Information • All key project information is gathered and distributed 0.764

efficiently.Well-Structured and Formal Project Approach • The project has its own management plan for control, 0.767

which is used in an appropriate way.Well-Structured and Formal Project Approach • Project control is executed by good managerial or 0.798

technical methods.Well-Structured and Formal Project Approach • Planning tools or similar aids are used in an effective 0.753

way in project planning.Clear Project Constraints • The project is well described and coordinated with 0.687

activities in other projects.

2. Key Participants Engaged 0.822Early Stakeholder Influence • All key participants have been engaged in producing 0.898

the business plan or have had the opportunity to influence it.

Early Stakeholder Influence • All participants have been given the opportunity to 0.898air their views on the project’s goal or mission.

Early Stakeholder Influence • All key people engaged in the project know who has 0.793decided its terms of references.

Table 5: Project success factor scales.


Stakeholder Group Scale Name • Items Alpha Loading

1. Stakeholder Satisfaction 0.914Owner • Has a good relationship with the prime contractor? 0.813Owner • Has good performance? 0.804Owner • Has appropriate earned value? 0.881Executive • Has allowed the project executive/project sponsor to profit? 0.870Contractor • Has achieved stakeholder satisfaction? 0.840Contractor • Has helped the senior supplier achieve their appropriate business goals? 0.940Supplier • Helps the supplier achieve their appropriate business goals? 0.908Contractor • Has a good safety record? 0.929

2. Project Executive Satisfaction 0.907Executive • Has achieved stakeholder satisfaction? 0.869Executive • Has achieved satisfactory performance efficiency? 0.907Executive • Has achieved satisfactory performance effectiveness? 0.840Executive • Helps the project executive/project owner achieve their appropriate business goals? 0.801Executive • Has stakeholder satisfaction? 0.872

3. Product Satisfaction 0.887Customer • Has a useful prototype? 0.951Operator • Has a useful prototype? 0.972Operator • Has good performance? 0.878

4. Product Efficiency 0.924Owner • Has a useful prototype? 0.943Contractor • Has achieved performance efficiency? 0.876Contractor • Has managed risk appropriately? 0.992Customer • Has good performance? 0.979

5. Satisfaction With Specifications 0.877Owner • Has appropriate specifications? (they are satisfied with them) 0.802Customer • Has appropriate specifications? (they are satisfied with them) 0.925Customer • Has a good relationship with the project owner? 0.815Operator • Has appropriate specifications? (they are satisfied with them) 0.881

6. Project Manager Satisfaction 0.875Project Manager • Pride in your work? 0.759Project Manager • Job satisfaction? 0.863Project Manager • Recognition? 0.768Project Manager • Skill growth? 0.790Project Manager • Contacts? 0.672Project Manager • High morale? 0.795Project Manager • Attracts top management support? 0.652

7. Contractor Satisfaction 0.948Contractor • Has achieved stakeholder satisfaction? 0.921Contractor • Has achieved performance effectiveness? 0.964Contractor • Has reduced waste? 0.977Supplier • Has demonstrated contract compliance? 0.981Contractor • Has demonstrated contract compliance? 0.970

8. Supplier Profitability 0.947Contractor • Has achieved performance efficiency? 0.911Contractor • Has allowed the supplier to profit? 0.976Supplier • Has allowed the supplier to profit? 0.972

9. Public Stakeholder Satisfaction 0.951Public • Has balanced social costs and benefits? 0.977Public • Has acceptable environmental impacts? 0.977

Table 6: Project managers’ perceptions of stakeholder satisfaction indicators.



AP

ER

S

outcomes, and impacts. There are anumber of practical and theoreticalimplications of this study.

Practical ImplicationsFor practitioners, the evaluation ofproject success will be more difficultdue to the necessity of considering theperspectives of different stakeholdergroups. But this cost is offset by the sec-ond implication: better managementdecisions and, more important, better“Go-No/Go-GoBack” decisions duringthe life of the project. After projectcloseout, using this richer method ofevaluating success will provide moreperspicacious post-action reviews andlessons learned. Finally, it is hoped thismore sophisticated method of evaluat-ing success will create greater apprecia-tion of actual project achievementsamong stakeholders and the generalpublic. There may also be practicalbenefits to policy development inimproving the way project success fac-tors are assessed by stakeholders.Evaluations of project success by stake-holders are inherently subjective andcannot be summarized naively into theiron triangle without under- or overes-timating project success at criticalpoints in the project life cycle. Thismodel of project success makes clearthe connections between factors thatcan be measured during the life of theproject and long-term impacts of inter-est to stakeholders.

Academic ImplicationsOur major contributions are the recog-nition of stakeholders as the evaluatorsof project success combined with ourmethod of evaluating project successacross a range of time frames. Thesecontributions are operationalized in ourmodel of project success (see Table 3).By acknowledging the central role ofthe various stakeholder groups indetermining project success, weincrease our understanding of theimportance of stakeholder manage-ment. The work of this article also furtherenhances our understanding of projectsuccess, and we hope will initiate

research into leading performance indi-cators that can be used by project man-agers to forecast the way stakeholderswill perceive the project in the monthsand years following project completion.It is important that project managers areable to make decisions during projectexecution that will truly guide the projecttoward project success. We believe wehave initiated the development of suchleading performance indicators, but theones we have developed still have a fairlynarrow time horizon. More research isneeded to test the ability of these scalesto predict long-term project success. �

ReferencesAndersen, E. S., Birchall, D. A., Jessen,S. A., & Money, A. H. (2006). Exploringproject success. Baltic Journal ofManagement, 1(2), 127–147.

Atkinson, R. (1999). Project manage-ment: Cost, time and quality. Two bestguesses and a phenomenon. It’s timeto accept other success criteria.International Journal of ProjectManagement, 17(6), 337–342.

Bryde, D. J. (2005). Methods for man-aging different perspectives of projectsuccess. British Journal ofManagement, 16(2), 119–131.

Eweje, J., Turner, J. R., & Müller, J. R.(2012). Maximizing strategic valuefrom megaprojects: The influence ofinformation-feed on decision-makingby the project manager. InternationalJournal of Project Management, 30(6),639–651.

Henriques, A., & Richardson, J. (Eds.).(2004). The triple bottom line: Does itall add up? London, UK: Earthscan.

Jacobson, C., & Choi, S. O. (2008).Success factors: Public works and pub-lic-private partnerships. InternationalJournal of Public Sector Management,21(6), 637–657.

Khang, D. B., & Moe, T. L. (2008).Success criteria and factors for interna-tional development projects: A life-cycle-based framework. ProjectManagement Journal, 39(1), 72–84.

Morris, P. W. G., & Hough, G. (1987).The anatomy of major projects: A studyof the reality of project management.Chichester, UK: Wiley.

Müller, R., & Turner, J.R. (2007).Matching the project manager’s lead-ership style to project type.International Journal of ProjectManagement, 25(1), 21–32.

Office of Government Commerce.(2007). Managing successful pro-grammes (2nd ed.). London, UK: TheStationery Office.

Office of Government Commerce.(2009). Managing successful projectswith PRINCE2 (5th ed.). London, UK:The Stationery Office.

Pinto, J. K., & Slevin, D. P. (1988).Critical success factors in effectiveproject implementation. In D. I.Cleland & W. R. King (Eds.), Projectmanagement handbook (2nd ed.). New York, NY: Van Nostrand Reinhold.

Reid, A. (2007). Managing teams. In J. R. Turner (Ed.), The Gower handbookof project management (4th ed., pp. 721–738). Aldershot, UK: Gower.

Shenhar, A. J., & Dvir, D. (2007).Reinventing project management: Thediamond approach to successful growthand innovation. Boston, MA: HarvardBusiness School Press.

Turner, J. R. (Ed.). (1995). The commer-cial project manager. London, UK:McGraw-Hill.

Turner, J. R. (2002). Project success cri-teria. In M. Stevens(Ed.), APM projectmanagement pathways (Chap. 20).High Wycombe, UK: Association forProject Management.

Turner, J. R. (Ed.) (2003). Contractingfor project management. Aldershot,UK: Gower.

Turner, J. R. (2009). The handbook ofproject-based management: Leadingstrategic change in organizations (3rd ed.). London, UK: McGraw-Hill.

Turner, J. R., Huemann, M., & Keegan,A. E. (2008). Human resource manage-ment in the project-oriented organization.


Newtown Square, PA: ProjectManagement Institute.

Turner, J. R., & Keegan, A. E. (2001).Mechanisms of governance in the proj-ect-based organization: The role of thebroker and steward. EuropeanManagement Journal, 19(3), 254–267.

Turner, J. R., Keegan, A. E., & Crawford,L. (2003). Delivering improved projectmanagement maturity through experi-ential learning. In J. R. Turner (Ed.),People in project management(pp. 45–64). Aldershot, UK: Gower.

Turner, J. R., & Müller, R. (2004).Communication and cooperation onprojects between the project owner asprincipal and the project manager asagent. European Management Journal,22(3), 327–336.

Turner, J. R., & Müller, R. (2006).Choosing appropriate project man-agers: Matching their leadership styleto the type of project. NewtownSquare, PA: Project ManagementInstitute.

Westerveld, E. (2003). The projectexcellence model: Lining success crite-ria and critical success factors.

International Journal of ProjectManagement, 21(6), 411–418.

Westerveld, E., & Gaya-Walters, D.(2001). Het Verbeteren van uwProjectorganisatie: Het ProjectExcellence Model in de Praktijk.Dementen, the Netherlands: Kluwer.

Kellogg Foundation, W. K. (2004). Logicmodel development guide. Battle Creek,MI: Author.

Xue, Y. (2009). A results-based monitor-ing and evaluation system for key infra-structure projects (Unpublished doctor-al thesis). Lille School of Management,Lille, France.

Yu, A. G., Flett, P. D., & Bowers, J. A.(2005). Developing a value-centeredproposal for assessing project success.International Journal of ProjectManagement, 23(5), 428–436.

Zolin, R., Cheung, Y. K. F., & Turner, J. R.(2012). Level of alignment betweenproject managers and stakeholders onperformance indicators. ProjectPerspectives: The Annual Publication ofInternational Project ManagementAssociation 2012. Retrieved fromhttp://www.ipma.ch/Documents/Perspectives_2012.pdf

Rodney Turner is a managing consultant atEuroProjex Ltd and academic director of the PhDin project and programme management depart-ment at SKEMA Business School in Lille, France.He is an adjunct professor at the University ofTechnology Sydney and the Kemmy BusinessSchool in Limerick, Ireland. He is the author oreditor of 16 books, and more than 30 peer-reviewed journal articles. He is editor of theInternational Journal of Project Management. Heis vice president, Honorary Fellow, and formerchairman of the United Kingdom’s Association forProject Management, and former president andchairman of the International ProjectManagement Association. His research interestsare leadership and human resource manage-ment in project-oriented organizations; the man-agement of projects in small and medium-sizedenterprises; the success of complex projects;and governance, ethics, and trust on projects.

Roxanne Zolin, PMP, is an associate professor inthe School of Management at the QueenslandUniversity of Technology. She helped developand teach their executive master’s program incomplex project management. She obtained aPhD in construction engineering managementfrom Stanford University, where she also holds amaster’s degree in sociology.