The University of Manchester Research

Can Serious Games Improve Project Management DecisionMaking Under Complexity?DOI:10.1177/8756972818808982

Document VersionAccepted author manuscript

Link to publication record in Manchester Research Explorer

Citation for published version (APA):Rumeser, D., & Emsley, M. (2018). Can Serious Games Improve Project Management Decision Making UnderComplexity? Project Management Journal. https://doi.org/10.1177/8756972818808982

Published in:Project Management Journal

Citing this paperPlease note that where the full-text provided on Manchester Research Explorer is the Author Accepted Manuscriptor Proof version this may differ from the final Published version. If citing, it is advised that you check and use thepublisher's definitive version.

General rightsCopyright and moral rights for the publications made accessible in the Research Explorer are retained by theauthors and/or other copyright owners and it is a condition of accessing publications that users recognise andabide by the legal requirements associated with these rights.

Takedown policyIf you believe that this document breaches copyright please refer to the University of Manchester’s TakedownProcedures [http://man.ac.uk/04Y6Bo] or contact uml.scholarlycommunications@manchester.ac.uk providingrelevant details, so we can investigate your claim.

Download date:24. Aug. 2020

Can serious games improve project management

decision-making under complexity?

Abstract:

The existing literature on the application of serious (or educational) games in project

management and decision-making tends to ignore the effect of project complexity levels on

decision-making performance. This research fills this gap by conducting an experiment

whereby two similar project management decision-making games with different complexity

levels are applied in teaching students. Our findings suggest that these games can improve

players’ decision-making skills both in the less complex and in the more complex project

scenarios. We also discover that the simulated project complexity levels do not affect players’

decision-making performance improvement.

Keywords: Project management, complexity, decision-making, skill acquisition, game-based

learning, serious games, computer-based games, education

Introduction

The importance of decision-making skills in managing complex projects

The Hudson Miracle aircraft emergency landing (Pasztor, 2010) teaches us about the

importance of experience in decision-making (DM), particularly in complex circumstances.

Captain Chesley “Sully” Sullenberger was flying US Airways Flight 1549 when his plane

was hit and damaged by birds. Instead of landing at the nearest airport, Capt. Sully decided to

land on the Hudson River. The decision saved the lives of hundreds on board. However, by

recreating the incident using flight simulators, the National Transportation Safety Board

(NTSB) found that four attempts to return to the closest La Guardia runway were all

successful. Three of nine additional attempts were also successful. Those who landed them

safely were pilots who immediately decided to head towards La Guardia after the simulated

engine problems. Unlike in reality, these pilots had already known the problem beforehand

and did not spend time deciding on the course of action. Mr. Sullenberger “could have made

a different call,” according to Kitty Higgins, a former NTSB member, “but his decision used

the best information he had . . . and was based on his experience and instincts” (Pasztor,

2010).

Like in the world of aviation, DM under complexity is an important subject in project

management (PM). Modern PM decisions are often characterized by the complexity and

subjectivity of decision makers (Attri & Grover, 2014). In fact, one major reason that

complex projects fail is bad DM (Flyvbjerg, 2014). Either in private or public projects,

effective and efficient PM practice requires DM in selecting projects, choosing project

managers, evaluating bids, selecting vendors and so forth (Mian & Dai, 1999). In most

projects, decisions which take account of a trade-off between time, cost, quality and other

intangible factors are common.

DM, amongst other soft skills needed in managing projects, is regarded as “the missing link,”

crucial to project success (Belzer, 2001). It is something a project manager does on a daily

basis as he/she manages schedule, quality, risks and resources. Few projects fail because of

an error in Gantt chart/PERT/CPM analysis, incorrect role mapping, or a mistake in creating

cost charts. More often projects are unsuccessful because of project managers’ failure in

communicating problems, working within the culture of the organization, motivating their

team, managing stakeholders, understanding strategic objectives, solving issues effectively

and making the right decisions. Belzer (2011) suggests that experience is required to improve

these skills. Aligning with this, Iansiti’s (2000) work finds that the accumulated project

experience is directly proportional to project performance. In DM in particular, it is widely

accepted that DM skills should be trained in an environment in which decision makers are

able to learn experientially (Caird-Daley, Harris, Bessell, & Lowe, 2007).

Despite the importance of DM skills in projects, and in complex projects in particular, very

little guidance on DM, let alone on improving project managers’ DM skills through

experiential learning, can be found in PM literature (Deguire, 2010). If anything, the

importance of DM seems to be accepted as the common knowledge needed by a project

manager. DM is only mentioned a couple of times and is not carefully explored in A Guide to

the Project Management Body of Knowledge (PMBOK® Guide) – Fifth Edition (PMI,

2013). Many project managers continue to struggle with the concept of DM (Deguire, 2010).

This may be one of the key factors which prevents project success.

Serious games application in project decision-making: Identifying gaps in current research

Wateridge (1997) recognizes experiential learning and its key contribution to PM education.

As an illustration (Rubin, Johnson, & Yourdon, 1994), a project manager is similar to a pilot

who plans the course, tracks progress continually, monitors his plane’s condition and reacts

to problems he encounters. Flight simulators are often used to train pilots to master these

skills, as they provide them with experience in various scenarios without exposing them to

the risk and cost of flying a real plane. Pilots could not learn aviation skills from a textbook

or instructor alone, since they need hands-on experience to do this. Similarly, project

managers need active and hands-on experience to develop their DM skills.

Serious games (SGs), or games for educational purposes (Connolly, Boyle, MacArthur,

Hainey, & Boyle, 2012), can act as “flight simulators,” as they provide students with an

experiential learning experience without exposure to the risks and costs associated with

managing actual projects (Dantas, de Oliveira Barros, & Werner, 2004). The games have “the

advantage of enabling participants to be put into complex, realistic project situations …” (Al-

Jibouri, 2005). This is needed in PM education because of its struggle in educating project

managers to deal with project complexity (Thomas & Mengel, 2008).

If appropriately conducted, SGs can also contribute to developing intangible PM skills

(Khenissi et al., 2016) such as DM, which is considerably difficult to teach with a traditional

(or lecture-based) method. First, unlike the traditional method, SGs provides opportunities for

learners to interact with each other in the DM process (Chen & Lin, 2010). Second, SGs are

interwoven with numerous DM opportunities for the learners (A. Turner & Martinek, 1995),

in which they can explore and develop various DM and problem-solving skills. Furthermore,

SGs can simulate the stress and pressure of making time-constrained PM decisions (Chow,

Woodford, & Maes, 2011). Finally, these games can be used to develop students’ critical

thinking, which is the reason-based reflective thinking needed in making decisions (Ennis,

1991).

a. Gaps in existing studies: Serious games application in simulating/training for decision-

making

There are many studies which propose the use of SGs to simulate/train for DM skills. For

instance, Linehan et al. (2009) designed a game to improve the group DM skills of people

whose role is to manage real-world emergencies such as volcano eruptions, floods, fires and

chemical spills. The game is designed to encourage players to engage in the kinds of

behaviors which are often associated with those of successful DM groups. Fenton-O'Creevy

et al. (2015) propose a financial game to improve DM abilities of traders in regulating

emotions during trading. Wolfe and Chacko (1983) investigate the effect of team size on

game performance and DM behavior. Other studies (Iwai & Morita, 2016; Morita, Horiuchi,

Iwai, Oshima, & Yu, 2014) analyze the effect of culture/origin on DM process and behavior.

Lamiraud and Vranceanu (2015) evaluate the effect of gender on DM performance in games.

Similar to the previous studies, Bragge et al. (2017) examine the characteristics of the DM

groups (e.g. gender, education, cultural orientation and group size) and their effect on group

performance in an SG.

An interesting pattern is that these studies focus on either result (i.e. DM performance) or

process. Furthermore, most of them evaluate specific independent variables which affect the

result or process of DM, such as gender, cultural orientation, emotion, team size and

education. Interestingly, none of the studies has assessed complexity level as a factor

affecting decisions.

Few DM games are conducted in a PM context. One of the few is a research conducted by

Urbaczewski and Kleitsch (2011) which proposes the application of Tetris as a tool for

improving PM and DM skills. However, their research relies on analogies and reflections and

therefore lacks PM context, which is an important factor that determines complexity level

(Marques, Gourc, & Lauras, 2011). Chen and Lin (2010), in contrast, present a role-playing

game in a PM context to train for DM. Their approach, however, measures perceived

learning, which tends to be subjective. Therefore, this needs to be complemented by a more

objective measurement (e.g. performance improvement). As suggested by Raia (1966):

“although impressions and opinions are admittedly of some value, there is a need to appraise

the effectiveness of games in terms of more objective criteria”. Furthermore, like in the other

contexts discussed previously, studies in the PM context have not considered complexity as a

factor that affect DM performance.

b. Gaps in existing studies: Serious games application in simulating/training for project

management

In their attempt to deliver a PM education that offers an experiential learning experience,

many studies propose the application of SGs. A project scheduling game (Vanhoucke,

Vereecke, & Gemmel, 2005), for instance, simulates scheduling complexity in projects. It

focuses on the time/cost relationship and on critical-path network problems. Similar to this,

Shtub (2005) proposes a scheduling game in both a single-project and a program (i.e.

multiple projects) scenario.

In the project implementation phase, Maratou et al. (2014) simulate team-working activities

in coping with risk events during project execution. The project-execution game, which was

developed by Ofer and Amnon (2007), is another game that simulates risk events in the

project-execution phase. Von Wangenheim et al. (2011) propose DELIVER!, an educational

game that trains students to monitor and control project performance by applying the earned

value management technique.

Although most of the games simulate DM activities, none of these games measures how

decisions are improved when playing the games. Furthermore, none of the research assesses

the impact of simulated project complexity levels on DM performance. Our review of PM

and DM literature indicates that there is a gap between what is true in the real world (i.e. the

importance of DM skill in managing complex projects) and what is true in the serious gaming

world (illustrated by Figure 1).

Figure 1 A gap between reality and the serious gaming world in addressing DM under

project complexity

Existing work which could offer insights into addressing this gap was conducted by Raia

(1966). His work suggests that learning experience and the degree of simulated-environment

complexity are not directly proportional. However, Raia’s (1966) work is conducted in a

general business (i.e. sales, production and finance) context and does not consider the

interaction between teams as a component of increased project complexity. In PM, this

ProjectComplexity

Level

Decision-MakingSkill

ProjectComplexity

Level

Decision-MakingSkill

Real-worldproblem

Currentresearchinseriousgames

assumption is unrealistic, as most projects (and teams who manage them) are interrelated in a

multi-project (or program) context (Aritua, Smith, & Bower, 2009; Payne, 1995).

Research questions

The gaps discussed in the previous section prompt us to investigate the effect of PM serious

games application on DM performance under different levels of project complexity. More

specifically, we aim to answer the following questions:

(a) Can DM performance in project management be improved by playing SGs?

(b) Does project complexity level affect players’ DM performance improvement when

playing SGs?

In order to answer the questions, we conduct an experiment using two similar project DM

games which have different complexity levels, namely project crashing game (PCG) and

program crashing game (PgCG).

The structure of this study is as follows. Theories on project complexity, key decisions in

project life-cycle phases, project crashing, games, SGs and simulation, and DM are examined

in the next section. Following this, details on the experimentation (i.e. data collection,

measurement and analysis) and results are presented. The study concludes by discussing

findings, implications (i.e. theoretical and practical) and limitations.

Theory

Project complexity

PM is “the application of knowledge, skills, tools and techniques to project activities to meet

the project requirements” (PMI, 2013, p. 4). A project is a transient and temporary

organization which is surrounded by intrinsic uncertainty (J. R. Turner & Müller, 2003). It is

“a unique process” (ISO, 2003) strongly affected by its environment (Zwikael, Shimizu, &

Globerson, 2005).

With regard to complexity, PM is about making something complex happen through other

people within the time, budget and quality constraints (Dh, 1987). In PM literature, project

complexity is a key theme, as projects have become more complex and the capability to cope

with complexity has become a need to avoid failure (Bakhshi, Ireland, & Gorod, 2016).

Complexity is an increasingly important point of reference to grasp modern projects’

managerial demands and actual problems which are often encountered in projects (Kähkönen,

2008).

There are several factors that characterize project complexity. First, complexity is strongly

correlated to the degree of the interaction between elements in a project (Aritua et al., 2009;

Bakhshi et al., 2016; Vidal & Marle, 2008). In a program (or multi-projects) context, the level

of complexity is higher, as it is affected by both the interaction between the projects in the

program and the interaction between the elements within the projects (Aritua et al., 2009).

Project complexity is also affected by the level of ambiguity and uncertainty in the project

(Cicmil & Marshall, 2005) in a way that it is determined by how clear the objectives and

methods are (J. R. Turner & Cochrane, 1993). It is also directly proportional to the number of

decisions which need to be made (Vidal & Marle, 2008).

Geraldi and Adlbrecht (2007) introduce two types of project complexity, namely “complexity

of faith” and “complexity of fact”. The first, on the one hand, underlines the effect of

uncertainty levels on project complexity (e.g. developing something unique or dealing with

new problems, where past data/information is limited). The second, on the other hand, is

affected by the huge amount of interdependent information that needs to be analyzed. Too

much information could have a detrimental effect on projects, as project managers only need

the relevant information at the right time (Strait & Dawson, 2006). The challenge here is not

to get lost in the details, but to maintain a holistic view of the problem.

Types of key decisions in the life cycle of a project

In most cases, a typical project life cycle consists of four phases/stages: conceptualization,

planning, execution, and termination (Adams & Barnd, 2008). These are referred to by PMI

(2013) as “process groups” instead of “phases,” since they define “phases” as project

subcomponents (e.g. concept development, design, build, test, etc.). However, the elements

within each “phase” (or “process group”) are similar. In this study, we use the term “phase”

due to the widespread use of the four-phase life cycle (Mian & Dai, 1999).

Mian and Dai (1999) study decisions which are often made in each project life-cycle phase.

The first phase is conceptualization, in which projects are evaluated and selected. This is a

critical phase, as the uncertainty level is at its peak as a result of the limited amount of

available information and the severe consequences of making wrong decisions (Williams &

Samset, 2010). The credibility of the DM process in this phase is often questioned, as

decision makers’ individual or political interests could degrade DM quality (Flyvbjerg, Holm,

& Buhl, 2002). The planning phase is conducted after the project has been selected (Samset,

2008). Decisions are mainly on team and organization form selection, bid evaluation and

determining budgets. In the execution phase, the role of the project manager is to monitor and

control the project. Examples of decisions in this stage would include selecting equipment

and PM software, and deciding on the crashing/accelerating/compressing options if the

project needs to be completed earlier or if it experiences anticipated delay and therefore

additional resources and/or working hours are needed (Pinto, 2009). The termination phase

concludes the project. Decisions on team reassignment are usually made in this stage. A

project can be terminated because its objectives have been achieved or because they cannot or

will not be met (PMI, 2013). DM failure in this phase is usually caused by decision makers’

inability to stop projects despite their underperformance (Meyer, 2014; Staw & Ross, 1987).

An interesting phenomenon is that most educational games in PM literature simulate DM in

project planning and execution phases (e.g. Vanhoucke, Vereecke, & Gemmel, 2005, Von

Wangenheim, Savi, & Borgatto, 2011, Ayk, 2012). None of these games, however,

specifically focuses on simulating crashing (i.e. acceleration) decisions despite the fact that it

is often applied in practice, as most projects are late (Gerk & Qassim, 2008).

Serious games: A combination of games and simulations

The literature provides an interesting discussion on the distinction between games and

simulations. The key determining factor is their main purpose (Lundy, 2003). Games are only

for entertainment, whilst simulations are developed to train or develop certain skills. To

students, games may sound more attractive than simulations, although perhaps in the absence

of learning (Jones, 1989; Lane, 1995). Simulations, in contrast, underline tasks which require

more practical or academic thinking (Jones, 1989). Aligning with this, simulations offer

participants with the opportunity of acting and reflecting (Callanhan, 1999). This is not

always inherent in a pure gaming environment. Furthermore, games focus on

winning/competition, whilst simulations center on complex situations and realistic objectives

that normally need to be dealt with on a daily basis (Callanhan, 1999). Games are mostly

people-oriented (Shubik, 1983), as they enable more human interactions compared to

simulations (Lane, 1995).

SGs, or educational games, combine the characteristics of a game and a simulation. They are

developed not only to entertain players (Hendrix, Al-Sherbaz, & Victoria, 2016) but also to

help them learn and/or change their behavior (Connolly et al., 2012). Specifically, by playing

these games, participants are expected to learn new skills, expand existing knowledge and

learn a specific topic (Dempsey, Lucassen, & Rasmussen, 1996). Like pure games, they can

not only stimulate interactions but also be played individually.

Decision-making definitions and types

DM is a broad subject which has been studied since the mid-eighteenth century (Edwards,

1954). Different definitions on the subject and what it involves abound in the literature and

spread across most, if not all, disciplines (Shull, Delbecq, & Cummings, 1970). DM can be

viewed in terms of the event, i.e. selecting between two or more options (Lehto & Nah, 2006)

using two or more criteria (Korhonen, Moskowitz, & Wallenius, 1992). It can also be seen in

terms of the process leading to committed action which aims to yield satisfying results (Salas

& Klein, 2001).

There are two types of DM: logical and judgmental/intuitive (Mian & Dai, 1999; Simon,

1987). In logical DM, on the one hand, objectives and options are made explicit, the effects

of selecting other alternatives are quantified, and these effects are appraised by how close

they are to the objectives. These structured decisions can be resolved relatively easily using a

mathematical method or formula, such as operations-research-based decisions on the best

production mix to achieve optimum profit. In judgmental/intuitive DM, on the other hand,

decisions are often unstructured and subjective (e.g. decisions on employee selection and

promotion). These types of decisions are usually difficult to quantify and are more common

in the real world. In making these decisions, decision makers need to rely on their

experiences and emotional inputs (Burke & Miller, 1999).

Although intuitive DM is seen as an increasingly viable approach in today’s business (Burke

& Miller, 1999), many decision analysts still question its credibility (Deguire, 2010).

Therefore, it is not surprising that PM literature is still dominated by the logical DM approach

(Powell & Buede, 2006). By interviewing 60 experienced professionals who have significant

positions across various industries, Burke and Miller (1999) find that a majority of these

professionals “felt that intuition led to better decisions.” However, their work also suggests

that, in some cases, DM by intuition can “get you into trouble” when accuracy is of

importance (e.g. when conducting quality control). Burke and Miller (1999) further argue that

intuitive DM should be applied when decisions are to be made unexpectedly or quickly, when

coping with uncertainty or novel situations and when dealing with situations that lack explicit

cues. However, if the time pressure and uncertainty is low, all information is available and

problems are relatively simple with few DM options, the logical or analytic DM approach

tends to be more effective (Pascual & Henderson, 1997).

As managers do “not have the luxury of choosing between analytic and intuitive approaches

to problems,” they need to have both DM skills and apply them appropriately (Simon, 1987).

This is also true in projects, as project managers “need to make decisions fast, yet accurately”

(Urbaczewski & Kleitsch, 2011). The other challenge is that, in most cases, project decision

makers are unable to make fully rational decisions, since projects are characterized by

uncertainties and complexities (Miller & Hobbs, 2009). Hence, such decision makers tend to

make decisions which are merely “satisficing” or “good enough” (Isenberg, 1991).

Experimentation

Data collection

Data were collected from 285 students who attended an MSc Management of Projects

seminar at The University of Manchester. In the seminar, students played two PM games

sequentially, namely a PCG and a PgCG. Before playing the games, players were randomly

assigned into teams (i.e. project teams and program teams).

The instrument used to collect the data was an automatic data collection system embedded in

the computer-based games. This system recorded teams’ decisions (i.e. crashed activities) and

resulting project performance (i.e. cost and time) after each DM attempt. Computer-based

games are applied in this study because they record the events and decisions taken, which are

useful for debriefing and analysis purposes (Martin, 2000). Moreover, under this method,

players were released from tedious calculations required in the paper-based method. Instead

of spending effort on drawing the network diagram and calculating activity attributes in each

iteration (e.g. earliest start, earliest finish, etc.), players focused on making and improving

their decisions.

Game settings

The settings of both games were designed on the basis of the literature review discussed in

the previous section. Both analytical and intuitive DM skills were needed to achieve the game

objective, as players were required to make structured decisions in a gaming environment

where time and competition pressure took effect. Specifically, they needed to review the

crashing options, calculate the effect of their decisions (i.e. the effect of crashing each project

activity on time and cost), make decisions (i.e. select activities to crash), and evaluate

performance (i.e. actual time and cost to deliver the project) against the goal (i.e. project

schedule and budget).

In the first game (i.e. the PCG), 102 project teams (i.e. 2 to 3 students per team) were given a

case from a product development project. They were asked to crash/accelerate the project

since their company was attempting to launch a product before a competitor did. Teams had

to select activities to crash in order to achieve the new project target (i.e. accelerated schedule

within a budget constraint). In order to provide experiential learning by which students could

decide, reflect, and make improved decisions, multiple attempts were allowed. Each project

team competed against other teams in order to be the first to achieve the objective. This game

was designed to improve students’ DM skills in the execution stage, particularly in crashing

activities which are on the critical path and are cheapest to accelerate (Pinto, 2009).

The second game (i.e. the PgCG) is similar to the first. However, unlike in the first game,

each team was required to work together with other project teams in a larger program group

in order to crash the overall program. There were 21 program teams in total. As they were

given limited time, they could not achieve the game objective without effective and efficient

communication and leadership. Some program groups decided to appoint a program manager

who managed each project team’s manager, whilst others communicated via an online chat

box embedded in the game. This game was more complex than the first (i.e. in terms of the

number of decisions and interactions needed). Like in real and complex projects (Jankovic,

Stal-Le Cardinal, & Bocquet, 2010), the decisions in the PgCG were made collaboratively

because none of the team had the necessary information to decide alone.

Data measurement and analysis method

Instead of measuring a subjective parameter (i.e. perceived learning), this study measures a

more objective parameter (i.e. project performance after each decision), as recommended by

Raia (1966). The relationship of project complexity level and improved DM is analyzed

quantitatively by applying the cross-tabulation (chi-squared test of independence) method

(Levine, Berenson, & Stephan, 1999). This method is used when investigating whether two

or more categorical variables are independent or related to each other (i.e. in this study,

whether the number of teams who made improved decisions are affected by the complexity

levels of the games they played). Alternative methods (i.e. Fisher’s and McNemar’s tests)

were not selected because the first is for small sample sizes and the latter for matched (or

paired) samples (Fisher, Marshall, & Mitchell, 2011).

Results

a. Game attempts and decision-making performance

Figure 2a Decision performance in the PCG by number of attempts

0

10

20

30

40

50

60

70

80

90

100

1st 2nd 3rd 4th 5th 6th 7th 8th 9th 10th 11th 12th 13th 14th 15th 16th 17th

Attempt

number of teams who achieved game objective (cumulative)

number of teams who did not achieve game objective (cumulative)

Figure 2b Decision performance in the PgCG by number of attempts

Figures 2a and 2b indicate DM performance improvement when playing the PCG and the

PgCG respectively. Another interesting similarity is, in both games, the number of teams who

achieved the objective tends to increase sharply after the first attempts, but this rate declines

and reaches a steady state on the last attempts. It should be noted that a relatively high

proportion of teams (i.e. 51% in PCG and 43% in PgCG) did not achieve the game objective

after multiple attempts. At the end, more teams (i.e. in proportion) achieved the goals in the

more complex game (PgCG) than in the simpler one (PCG). This is a potential indication that

the more complex the simulated project is, the more impact it has on DM improvement.

However, this needs to be further analyzed, as we have not considered those who did not

achieve the objectives but made improved decisions.

0

2

4

6

8

10

12

14

16

18

1st 2nd 3rd 4th 5th

Attempt

number of teams who achieved game objective (cumulative)

number of teams who did not achieve game objective (cumulative)

b. Improved decisions: Less complex versus more complex project management games

Figure 3a Decision performance breakdown in the PCG

Figure 3b Decision performance breakdown in the PgCG

Figures 3a and 3b further indicate that decisions were improved by playing the games. In

fact, the proportion of teams who made improved decisions in both games are strikingly

similar (i.e. 59%, or 60 project teams, made improved decisions in the less complex game,

i.e. PCG, and 57%, or 12 program teams, made improved decisions in the more complex

game, i.e. PgCG – see black pie chart area in Figures 3a and 3b). This could cancel our

initial premise on the effect of simulated project complexity levels on DM improvement

discussed in the previous section (i.e. the more complex the simulated project is, the more

impact it has on DM improvement). Hence, a hypothesis testing is needed:

• Null hypothesis (Ho) : the simulated project complexity level in games affects

DM performance improvement.

• Alternative hypothesis (Ha) : the simulated project complexity level in games does

not affect DM performance improvement.

Table 1 displays the variables (or categories) and their values (i.e. game complexity levels

and number of teams who made/did not make improved decisions).

Table 1. Decision performance improvement and game complexity levels

Game complexity level

Number of teams who made

improved decisions

Number of teams who did not

make improved decisions

Total

PCG (less complex) 60 30 90

PgCG (more complex) 12 5 17

Total 72 35 107

As discussed in the Data measurement and analysis method section, in order to test the

hypothesis the chi-squared test of independence is applied on variables displayed in Table 1.

Applying this method (i.e. using IBM SPSS Version 22), we find that the p-value for this test

is 0.752, hence at 95% level of confidence (or 0.05 significance level), we reject our null

hypothesis and conclude that the simulated project complexity level in PM games is not

related to DM performance improvement.

Discussion

Key findings

There are two key findings identified in this research:

• The application of PM SGs can improve players’ DM skills;

• The complexity levels of the simulated project in the games do not affect DM

performance improvement.

Our first finding is consistent with the work of Morewedge et al. (2015), which suggests that

a single training intervention (e.g. by playing games or watching videos) can improve DM.

Aligning with this, Wolf (1972) argues that computer games can be used to help players to

explore DM abilities and behavior. As more attempts are made, players (i.e. project decision

makers) gain knowledge from similar prior experiences which are gained from reflecting on

their previous DM attempts and their effect on project performance (Huff & Prybutok, 2008).

This helps them to make improved decisions.

The second finding suggests that both the less complex and the more complex games have

the same benefit on players’ DM performance improvement. In other words, game

complexity levels do not affect project DM improvement. This is consistent with two existing

studies which are conducted in a general management context:

• Raia’s (1966) work, which finds that the learning benefits of a simple game are

essentially the same as those provided by a relatively more complex game and the

complexity degree of the simulated environment is not directly proportional to

students’ learning experience.

• Wolfe’s (1978) work, which finds that although increased game complexity is

identical with greater DM comprehensiveness, increased challenge and lesser

monotony, they are not positively linear to learning effectiveness.

There are several possible explanations as to why different simulated project complexity

levels have no effect on improved DM performance. Although the PCG is simpler, in the

more complex game (i.e. the PgCG), players (project teams) are required to communicate and

be transparent with each other in order to achieve the objective. Transparency in DM is a key

success factor in managing a portfolio of projects (Patanakul, 2015). From another

perspective, collaboration amongst project decision makers may be beneficial for their

learning, as they learn not only from their experience but also from others’ experiences.

Supporting this notion, Krischner et al. (2011) finds that in more high-complexity cognitive

tasks (i.e. like in the PgCG), collaborative learning becomes more effective. This is because

in completing these complex tasks players need to share the task’s higher “cognitive” load

with other players.

Furthermore, as the more complex game (i.e. the PgCG) presents more challenges and bases

the overall program group performance on the performance of its weakest project team, it is

less prone to the “social loafing” effect compared to the simpler game (Kerr & Tindale,

2004). “Social loafing” is a phenomenon where people reduce their work output when they

work in a group environment (Steiner, 2007). As the risk of being evaluated is reduced,

players can be tempted to “free ride” on other members’ efforts. The DM process in the more

complex game requires more control and leadership, and thus may reduce the risk of “social

loafing.”

Additional finding: Non-linear relationship between game attempts and project decision-

making improvement

In addition to the two main findings, we also identify another interesting phenomenon

whereby in both the more complex and the less complex PM games the number of teams

making improved decisions increases sharply on the first attempts, but the rate slows down

and reaches a steady state on the last attempts (see Figures 2a and 2b in the previous section).

This phenomenon is consistent with the mixtures argument theory proposed by Newell &

Rosenbloom (1981). At first, the faster learners dominate the total system learning because, a

fortiori, they learn faster. Nevertheless, these learners soon make fewer contributions to total

learning performance and the system is then dominated by the slow learners. Hence, the rate

of improvement is slower than the initial improvement rate (Newell & Rosenbloom, 1981).

Conclusions and implications

This research suggests that PM SGs can improve DM performance in both more complex and

less complex project scenarios. It further investigates the effect of complexity on students’

DM performance. The results suggest that different complexity levels in a simulated project

do not affect DM performance improvement.

The impact of this research can be significant for academics and practitioners. For academics,

the findings offer a new perspective, as they combine three major dimensions: DM, project

complexity and SGs. This complements existing studies in PM literature which only analyze

two of the three dimensions. In the real-world, this research could help PM educators to

acknowledge the potential benefit of SGs application in teaching, particularly in developing

trainees’ DM skills to cope with project complexity. This is significant, as current PM

education struggles to aid learners in coping with the actual complexity in projects (Thomas

& Mengel, 2008).

Limitations and directions for further research

The complexity factors used to distinguish the games are limited to the number of players’

decisions and interactions. Other complexity factors (e.g. risks, uncertainties, scope changes)

are not inserted into the games. Furthermore, the criteria used in this study to measure DM

performance (i.e. game objectives) are limited to time and cost, whilst project success criteria

are broader (e.g. quality and sustainability). We encourage further research to incorporate

these and other complexity factors in its games, as it may add insights to our findings.

However, one must be careful not to overcomplicate the games, as this can reduce learning

effectiveness (Al‐Jibouri & Mawdesley, 2001; Baird & Flavell, 1981).

Another limitation of this research is that the respondents who participated in the experiments

were master’s degree students who mainly have little or no project-related work experience.

To complement this work, we are conducting separate research which examines additional

feedback from experienced program managers and PM trainers and consultants on how these

games can be improved and how they can be applied in PM practice. In addition, a major PM

training company has invited us to apply these games in one of their training sessions, which

presents us with an opportunity to replicate the experiment.

References

Adams, J. R., & Barnd, S. E. (2008). Behavioral implications of the project life cycle. Project

Management Handbook, Second Edition, 206-230.

Al-Jibouri, S. (2005). The use of a simulation model as a game for teaching management of

projects in construction. The International journal of engineering education., 21(6),

1195.

Al‐Jibouri, S. H., & Mawdesley, M. J. (2001). Design and experience with a computer game

for teaching construction project planning and control. Engineering, Construction and

Architectural Management, 8(5/6), 418-427. doi:10.1108/eb021201

Aritua, B., Smith, N. J., & Bower, D. (2009). Construction client multi-projects – A complex

adaptive systems perspective. International Journal of Project Management, 27(1),

72-79. doi:10.1016/j.ijproman.2008.02.005

Attri, R., & Grover, S. (2014). Decision making over the production system life cycle:

MOORA method. International Journal of System Assurance Engineering and

Management, 5(3), 320-328.

Ayk, C. (2012). An Exploratory Study of Game-based M-learning for Software Project

Management. Journal of universal computer science, 18(14), 1933.

Baird, A. N., & Flavell, R. (1981). A project management game. Computers and Education,

5(1), 1-18. doi:10.1016/0360-1315(81)90022-1

Bakhshi, J., Ireland, V., & Gorod, A. (2016). Clarifying the project complexity construct:

Past, present and future. International Journal of Project Management, 34(7), 1199-

1213. doi:https://doi.org/10.1016/j.ijproman.2016.06.002

Belzer, K. (2001, 2001). Project management: Still more art than science.

Bragge, J., Kallio, H., Seppälä, T., Lainema, T., & Malo, P. (2017). Decision-Making in a

Real-Time Business Simulation Game: Cultural and Demographic Aspects in Small

Group Dynamics. International Journal of Information Technology & Decision

Making, 16(03), 779-815.

Burke, L. A., & Miller, M. K. (1999). Taking the mystery out of intuitive decision making.

The Academy of Management Executive, 13(4), 91-99.

Caird-Daley, A., Harris, D., Bessell, K., & Lowe, M. (2007). Training decision making using

serious games. Human Factors Integration Defence Technology Centre. Report No:

HFIDTC/2/WP4, 6(1).

Callanhan, M. R. (1999). Simulation and role play. Alexandria, VA.: American Society for

Training & Development.

Chen, H. M., & Lin, Y. (2010). A video game-based training mode for decision making in

construction project control and management.

Chow, A. F., Woodford, K. C., & Maes, J. (2011). Deal or No Deal: using games to improve

student learning, retention and decision-making. International journal of

mathematical education in science and technology, 42(2), 259-264.

Cicmil, S., & Marshall, D. (2005). Insights into collaboration at the project level: complexity,

social interaction and procurement mechanisms. Building Research & Information,

33(6), 523-535. doi:10.1080/09613210500288886

Connolly, T. M., Boyle, E. A., MacArthur, E., Hainey, T., & Boyle, J. M. (2012). A

systematic literature review of empirical evidence on computer games and serious

games. Computers & Education, 59(2), 661-686.

Dantas, A. R., de Oliveira Barros, M., & Werner, C. M. L. (2004, 2004). A Simulation-Based

Game for Project Management Experiential Learning.

Deguire, M. (2010, 2010). Project versus program business case decisions.

Dempsey, J. V., Lucassen, B., & Rasmussen, K. (1996). The instructional gaming literature:

Implications and 99 sources: University of South Carolina, College of Education.

Dh, H. (1987). Management: Principles and practices.

Edwards, W. (1954). The theory of decision making. Psychological Bulletin, 51(4), 380.

Ennis, R. (1991). Critical thinking: A streamlined conception. Teaching philosophy, 14(1), 5-

24.

Fenton-O'Creevy, M., Adam, M., Clough, G., Conole, G., Gaved, M., Lins, J. T., . . . Smidts,

A. (2015). A game based approach to improve traders' decision-making.

Fisher, M. J., Marshall, A. P., & Mitchell, M. (2011). Testing differences in proportions.

Australian Critical Care, 24(2), 133-138.

Flyvbjerg, B., Holm, M. S., & Buhl, S. (2002). Underestimating Costs in Public Works

Projects: Error or Lie? Journal of the American Planning Association, 68(3), 279-295.

doi:10.1080/01944360208976273

Geraldi, J. (2008). Patterns of complexity: The thermometer of complexity. Project

Perspectives, 29, 4-9.

Geraldi, J. G., & Adlbrecht, G. (2007). On faith, fact, and interaction in projects.

Gerk, J. E. V., & Qassim, R. Y. (2008). Project Acceleration via Activity Crashing,

Overlapping, and Substitution. Engineering Management, IEEE Transactions on,

55(4), 590-601. doi:10.1109/TEM.2008.927786

Hendrix, M., Al-Sherbaz, A., & Victoria, B. (2016). Game based cyber security training: are

serious games suitable for cyber security training? International Journal of Serious

Games, 3(1), 53-61.

Huff, R. A., & Prybutok, V. R. (2008). Information systems project management decision

making: The influence of experience and risk propensity. Project Management

Journal, 39(2), 34-47.

Iansiti, M. (2000). How the incumbent can win: managing technological transitions in the

semiconductor industry. Management Science, 46(2), 169-185.

Isenberg, D. J. (1991). How senior managers think: Open University Press.

ISO. (2003). 10006: 2003: Quality Management systems-Guidelines for quality management

in projects. Ontario: International Organization for Standardization (ISO).

Iwai, C., & Morita, M. (2016). An Experiment: An International Comparison of the Decision-

Making Process Using a Business Game Simulation and Gaming in the Network

Society (pp. 215-231): Springer.

Jankovic, M., Stal-Le Cardinal, J., & Bocquet, J.-C. (2010). Collaborative Decision-making

in Design Project Management. A Particular Focus on Automotive Industry. Journal

of decision systems, 19(1), 93-116.

Jones, K. (1989). Running, or Stumbling Through, Simulations. Simulation/Games for

Learning, 19(4), 160-167.

Kähkönen, K. (2008). Level of complexity in projects and its impacts on managerial

solutions. International Project Management Association, XXIX, 3.

Kerr, N. L., & Tindale, R. S. (2004). Group performance and decision making. Annu. Rev.

Psychol., 55, 623-655.

Khenissi, M. A., Essalmi, F., Jemni, M., Kinshuk, Graf, S., & Chen, N.-S. (2016).

Relationship between learning styles and genres of games. Computers & Education,

101, 1-14. doi:http://dx.doi.org/10.1016/j.compedu.2016.05.005

Kirschner, F., Paas, F., & Kirschner, P. A. (2011). Task complexity as a driver for

collaborative learning efficiency: The collective working‐memory effect. Applied

Cognitive Psychology, 25(4), 615-624.

Korhonen, P., Moskowitz, H., & Wallenius, J. (1992). Multiple criteria decision support-A

review. EUROPEAN JOURNAL OF OPERATIONAL RESEARCH, 63(3), 361-375.

Lamiraud, K., & Vranceanu, R. (2015). Group Gender Composition and Economic Decision-

Making: Evidence from the Kallystée Business Game.

Lane, D. C. (1995). On a resurgence of management simulations and games. Journal of the

Operational Research Society, 46(5), 604-625.

Lehto, M. R., & Nah, F. (2006). Decision‐Making Models and Decision Support. Handbook

of Human Factors and Ergonomics, Third Edition, 191-242.

Levine, D. M., Berenson, M. L., & Stephan, D. (1999). Statistics for managers using

Microsoft Excel (Vol. 660): Prentice Hall Upper Saddle River, NJ.

Linehan, C., Lawson, S., Doughty, M., & Kirman, B. (2009, 2009). Thereʼs no ʻIʼ in

ʻEmergency Management Team: ʼ Designing and evaluating a serious game for

training emergency managers in group decision making skills.

Lundy, J. (2003). E-learning simulation: Putting knowledge to work. Paper presented at the

Gartner US Symposium.

Maratou, V., Chatzidaki, E., & Xenos, M. (2014). Enhance learning on software project

management through a role- play game in a virtual world. Interactive Learning

Environments, 1-19. doi:10.1080/10494820.2014.937345

Marques, G., Gourc, D., & Lauras, M. (2011). Multi-criteria performance analysis for

decision making in project management. International Journal of Project

Management, 29(8), 1057-1069.

Martin, A. (2000). A simulation engine for custom project management education.

International Journal of Project Management, 18(3), 201-213.

Meyer, W. G. (2014). The Effect of Optimism Bias on the Decision to Terminate Failing

Projects. Project Management Journal, 45(4), 7-20. doi:10.1002/pmj.21435

Mian, S. A., & Dai, C. X. (1999, 1999). Decision-making over the project life cycle: An

analytical hierarchy approach.

Miller, R., & Hobbs, B. (2009). The complexity of decision-making in large projects with

multiple partners: Be prepared to change. Making essential choices with scant

information, 375-389.

Morewedge, C. K., Yoon, H., Scopelliti, I., Symborski, C. W., Korris, J. H., & Kassam, K. S.

(2015). Debiasing decisions: Improved decision making with a single training

intervention. Policy Insights from the Behavioral and Brain Sciences, 2(1), 129-140.

Morita, M., Horiuchi, M., Iwai, C., Oshima, M., & Yu, Z. (2014). An Experiment on Group

Decision-Making Using a Business Game: An International Comparison of MBA

Students in Japan; China; and Russia. Developments in Business Simulation and

Experiential Learning, 37.

Newell, A., & Rosenbloom, P. S. (1981). Mechanisms of skill acquisition and the law of

practice. Cognitive skills and their acquisition, 1, 1-55.

Pascual, R., & Henderson, S. (1997). Evidence of naturalistic decision making in military

command and control. Naturalistic decision making, 217-226.

Pasztor, A. (2010). 'Hudson Miracle' Gets Closer Look. Retrieved from

https://www.wsj.com/articles/SB10001424052748703612804575222482042335978?

mg=prod/accounts-wsj

Patanakul, P. (2015). Key attributes of effectiveness in managing project portfolio.

International Journal of Project Management, 33(5), 1084-1097.

Payne, J. H. (1995). Management of multiple simultaneous projects: a state-of-the-art review.

International Journal of Project Management, 13(3), 163-168. doi:10.1016/0263-

7863(94)00019-9

Pinto, J. K. (2009). Project management : achieving competitive advantage (2nd ed.,

International ed. ed.). Upper Saddle River, N.J. ; Harlow: Upper Saddle River, N.J. ;

Harlow : Pearson Education.

PMI. (2013). A Guide to the Project Management Body of Knowledge (5th ed. ed.). Newton

Square, PA: Project Management Institute.

Powell, R. A., & Buede, D. M. (2006). Decision-making for successful product development.

Project Management Journal, 37(1), 22.

Rubin, H. A., Johnson, M., & Yourdon, E. (1994). With the SEI as My Copilot: Using

Software Process" Flight Simulation" to Predict the Impact of Improvements in

Process Maturity. American Programmer, 7, 50-50.

Salas, E., & Klein, G. A. (2001). Linking expertise and naturalistic decision making:

Psychology Press.

Samset, K. (2008). How to overcome major weaknesses in mega-projects: the Norwegian

approach. Decision-making on mega-projects: Cost-benefit analysis, planning and

innovation, 173-188.

Savi, R., Wangenheim, C. G. V., & Borgatto, A. F. (2011). A model for the evaluation of

educational games for teaching software engineering. Paper presented at the

Proceedings - 25th Brazilian Symposium on Software Engineering, SBES 2011.

Shtub, A. (2010). Project management simulation with PTB Project Team Builder. Paper

presented at the Proceedings - Winter Simulation Conference.

Shubik, M. (1983). Gaming: A state of the art survey. Operational gaming: An international

approach, 13-22.

Shull, F. A., Delbecq, A. L., & Cummings, L. L. (1970). Organizational decision making.

Simon, H. A. (1987). Making management decisions: The role of intuition and emotion. The

Academy of Management Executive (1987-1989), 57-64.

Staw, B. M., & Ross, J. (1987). Behavior in escalation situations: Antecedents, prototypes,

and solutions. Research in organizational behavior.

Steiner, I. D. (2007). Group Process and Productivity (Social Psychological Monograph).

Strait, C., & Dawson, R. (2006). Too much information (Vol. 20(10), pp. 64-68): PM

Network.

Thomas, J., & Mengel, T. (2008). Preparing project managers to deal with complexity -

Advanced project management education. International Journal of Project

Management, 26(3), 304-315. doi:10.1016/j.ijproman.2008.01.001

Turner, A., & Martinek, T. J. (1995). Teaching for understanding: A model for improving

decision making during game play. Quest, 47(1), 44-63.

Turner, J. R., & Cochrane, R. A. (1993). Goals-and-methods matrix: coping with projects

with ill defined goals and/or methods of achieving them. International Journal of

Project Management, 11(2), 93-102. doi:http://dx.doi.org/10.1016/0263-

7863(93)90017-H

Turner, J. R., & Müller, R. (2003). On the nature of the project as a temporary organization.

International Journal of Project Management, 21(1), 1-8.

Urbaczewski, L., & Kleitsch, S. (2011). Tetris as a Tool to Teach Project Management and

Decision-Making Skills.

Vanhoucke, M., Vereecke, A., & Gemmel, P. (2005). THE PROJECT SCHEDULING

GAME (PSG): SIMULATING TIME/COST TRADE-OFFS IN PROJECTS. Project

Management Journal, 36(1), 51-59.

Vidal, L.-A., & Marle, F. (2008). Understanding project complexity: implications on project

management. Kybernetes, 37(8), 1094-1110.

Von Wangenheim, C. G., Savi, R., & Borgatto, A. F. (2011). DELIVER! – An educational

game for teaching Earned Value Management in computing courses. Information And

Software Technology. doi:10.1016/j.infsof.2011.10.005

Wateridge, J. (1997). Training for IS/IT project managers: a way forward. International

Journal of Project Management, 15(5), 283-288.

Williams, T., & Samset, K. (2010). Issues in front‐end decision making on projects. Project

Management Journal, 41(2), 38-49.

Wolf, G. (1972). SOME RESEARCH AND TEACHING WITH AN ON‐LINE

OLIGOPOLY GAME USING AN ARTIFICIAL PLAYER. Decision Sciences, 3(3),

101-114.

Wolfe, J. (1978). The effects of game complexity on the acquisition of business policy

knowledge. Decision Sciences, 9(1), 143-155.

Wolfe, J., & Chacko, T. I. (1983). Team-size effects on business game performance and

decision-making behaviors. Decision Sciences, 14(1), 121-133.

Zwikael, O., Shimizu, K., & Globerson, S. (2005). Cultural differences in project

management capabilities: A field study. International Journal of Project

Management, 23(6), 454-462.