when to prototype decision variables used in industry 1995 information and software technology

8/10/2019 When to Prototype Decision Variables Used in Industry 1995 Information and Software Technology

1/6

Inforr?iQrion nnd so&W? Technology 1995 37 (2) 113-I 18

When to prototype: decision

variables used in industry

Bill C Hardgrave

Computer Information Systems and Quantitative Analysis, College of Business Administration, University of

Arkansas, Fayetteville, AR 72701, US

internet:[email protected]

Prototyping continues to gain acceptance as a way to improve the development of information

systems. Recent surveys indicate over 60 usage in industry. However, there exists widespread

disagreement concerning the proper usage of prototyping. Speci&ally, the question is: When

should prototyping be used

? This study is an attempt to answer this question by examining the

factors used by IS managers in their decision to use prototyping. Results of the study indicate

a set of variables used by practitioners. Thus, it appears that, based upon the results of this

study and similar studies, a set of factors is being used by IS managers in their decision to

prototype.

Keywords: prototyping, systems development, decision variables

During the early 1980s prototyping emerged as a potential

solution to the problems facing information systems

development. With the promise of improved systems, faster

development, and higher user satisfaction, prototyping

quickly gained acceptance. The use of prototyping has

continued to grow during the past several years. A 1984

industry survey by Langle

et al.,

indicated that 33 % of

the respondents use prototyping. In 1987, 46% usage was

reported*; 49% in 19883; and 61% in 19904.

However, if prototyping does indeed provide all the

purported advantages, then why is the reported adoption of

prototyping not closer to lOO%? The reason is simple: if

not used properly, prototyping can be counterproductive.

Potential disadvantages of prototyping include: poor

documentation,

uncontrollable projects and unrealistic

expectations from users, among others. Industry users

quickly realized that prototyping was not a remedy for all

problems associated with systems development.

Because prototyping is not a cure-all, IS managers must

make the decision whether or not to use prototyping for a

specific project. Although the IS literature identifies several

variables which can assist the manager in his/her decision,

most of the variables have not been empirically

investigated. Therefore, the purpose of this study is to

identify empirically those decision variables which

are

being used in industry.

This is accomplished by an industry

survey which asks IS managers to identify those variables

which are used to make the decision whether or not to

prototype.

Background

Although prototyping has been used for several years, there

appears to be no unique definition, nor an established set of

guidelines (decision variables) for its use. An investigation

of the literature reveals a wide variety of variables which

allegedly impact the decision to prototype.

In an effort to determine, among other things, the benefits

and disadvantages of prototyping, Carey and Currey asked

questionnaire respondents to indicate when they thought

prototyping was the best strategy. Although this is not a

direct indication of the decision variables used in the

prototyping decision, it does provide some of the reasons

prototyping was initially considered. Their findings

indicated that prototyping was the best strategy for: (a) all

on-line development; (b) all new development; (c) when

user expectations or requirements are not clear; and (d)

when the appropriate tools are available.

Doke et aL6, used a Delphi panel of industry experts

to indicate the factors used in their decision to use

prototyping. The top factors determined from the study

were:

communicate design widely;

on-line system;

reduce development time;

developers lack experience;

feasibility in question;

alternatives need evaluated;

large and complex system;

0950-5849/95/ 09.50 0 1995 Elsevier Science B.V. All rights

reserved

113


2/6


3/6

When o prototype B C Hardgrave

Table 2. De&on variabks by frequency

importance. Thus, frequency of response and the impor-

Frototypingde&ion

Frequency Sum Mean

Std Range

tance of that factor provide different views of the decision

VariaMW

DW

factors. A detailed discussion of the 12 factors is provided

Reouirements

in the next section.

dktermination 24

200

8.33

2.13 3-10

Project size 19 139 7.32 2.85 l-10Complexity 16 127 7.94

2.56 l-10

Discussion

Availability of tools 16

Mode

16

Project duration 15

Feasibility/testing 14

Innovativeness 14

User exuerience 12

123 7.69

2.23 3-10

128 8.00

2.18 3-10 The discussion in this section will follow the

order of the

110 7.33

2.55 2-10 Table 2.

115 8.21

2.14 4-10

decision variables as presented in

104 7.43

2.38 3-10

78 6.50

2.60 3-10 Requirements determination

User in;olvement

12

105 8.75

1.30 6-10

Impact (importance) 10

75 7.50

2.91

2-10

Developer experience 8

54 6.75

2.99 l-10

Table sorted by frequency (i.e., number of times listed by respondents)

Table 3. Decision variables by mean

Prototyping decision Frequency Sum

Mean

STD Range

variables

DW

Respondents indicated that prototyping should be used under

conditions of ambiguous requirements, requirements

expected to change, users dont know what they want,

expected user modifications and process not well defined.

Perhaps the greatest benefit provided by prototyping is the

ability to determine system requirements when requirements

are particularly ambiguous, the users do not know exactly

what they want, or the requirements are expected to change.

Prototvning captures an initial set of reauirements and.

User involvement

Requirements

determinationeasibility/testing

Mode

12 105

24 2004 115

16 128

8.75

1.30 6-10 through it&at&e discovery, builds the iystem

to meet

the users needs.: 38% of the respondents indicated

8.33

2.13 3-10

.21

2.14 4-10 requirements determination to be a factor in

the decision

8.00

2.18 3-10 to prototype.

Complexity 16 127 7.94

2.56 l-10

Availability of tools 16 123 7.69

2.23 3-10

Impact (importance) 10 75 7.50

2.91 2-10

Project size

Innovativeness 14 104 7.43

2.38 3-10 Project size refers to the man-hours and/or cost

necessary

Project duration 15 110 7.33

2.55 2-10

Project size

19

139

7.32

2.85

l-10

to produce the system. Project size does not include pro-

Developer experience

8

54

6.75

2.99

l-10

ject duration or number of users; 30% of the respondents

User experience 12 78 6.50

2.60 3-10 indicated that the size of the project helped

determine

Table sorted bv mean reswnse

whether or not prototyping should be used. The argument

for prototyping large systems is that, because of its size,

specifications will change during the development of the

system3X4. Prototyping readily accommodates change,

thus proving its usefulness.

he respondents. For example, system throughput was

identified by only one respondent. Because it is not possible

to provide a full discussion of all 48 factors in an article of

reasonable length, only the top variables will be discussed

further. Tables 2 and 3 provide a summary of the top 12

decision variables identified by the respondents. To be

included in the tables, at least 10% of the respondents must

have identified the factor (i.e., at least seven respondents).

Although the inclusion of only 12 factors for discussion is

arbitrary, it is not meant to diminish the importance of the

remaining 36 factors. The list of all 48 variables can be

found in the Appendix.

Complexity

Several respondents simply specified that complexity was

a factor in their choice to use prototyping. Because no

explanation of complexity was provided, it is difficult to

determine exactly what the respondent meant. However,

one would expect complexity to be determined by such

characteristics as: age of technology, number of users,

project size, volume of new information generated by

system, and complexity of producing new information9*5.

Burns and Dennis suggest that prototyping is not suitable

for projects of high complexity because prototyping lacks

the discipline and structure needed to manage and control

the project.

Table 2 lists the factors in order of the number of times

identifed by the respondents. For example, requirements

determination was identified by 24 respondents. The

frequency of response indicates the popularity or breadth of

use of the factor in making the prototyping decision, but it

does not indicate the importance of the factor.

Table 3 lists the factors in order by the mean response.

For example, the mean response for requirements

determination is 8.33 1 = low importance, 10 = high

importance). The ranking by mean response indicates the

importance of the factor in the prototyping decision. Both

tables are provided because the rankings are different

between Tables 2 and 3. For example, user involvement

is ranked tenth by frequency, but first when ordered by

Availabili ty of tools

Responses such as ability to convert prototype to final

product, ease of prototyping,

ease of use of tools, and

methodology/tool match indicate that the availability of

tools must be considered in the prototyping decision.

However, the type of tool needed is dependent upon the

type of prototyping used.

Expendable prototypes, which are discarded after they

are no longer needed, can be created with simple tools,

Information and S ware Technology 1995 Volume 37 Number 2

115


4/6

When o protot ype: B C Har dgrave

such as word processors or graphics packages (e.g.,

Microsofts Powerpoint)637. Evolutionary prototypes,

which evolve into the final operational system, require

more sophisticated tools, such as database management

systems, fourth generation languages, or special-purpose

prototyping tools. These tools are needed to provide fast

response to user requests and to allow the prototypes to

evolve into the final project9X9.

Respondents identifed tools such as EIS Toolkit,

Excelerator, IEF, and numerous others, as aids to facilitate

prototyping. It would appear that almost any company

possesses the tools needed for expendable prototyping

(which could be hand-drawn on paper, if necessary), but

may not have the necessary tools needed for evolutionary

prototyping.

M ode

System mode can be categorized as on-line, batch, or a

mixture (of on-line and batch activities). Respondents

indicate that systems which contained some elements of on-

line processing should employ a prototyping approach. On-

line systems are much more related to user needs and

expectations than batch systems (because of the user

interface) and a prototyping approach should be u~ed~,~.

Project duration

Project duration, the time between the start of a project and

the delivery of the final system to the user, is an important

consideration in the selection of a development strategy.

Long-running projects encounter many problems caused by

organizational and individual changes as a direct

consequence of the passage of time. The longer the duration

of a project, the greater the changes are likely to be.

Because of its iterative nature, prototyping can facilitate

these changes2*.

Feasibility/testing

Management is often unwilling to experiment with new

concepts, ideas, and procedures, due to the risk level of

such undertakings23. A good project may sometimes be

dismissed because management will not commit resources

before fully knowing the benefits. Prototyping can be used

in situations where there is a need for experimentation and

learning before commitment of resources for a full

project24,25.The prototype provides the opportunity to test,

modify, and visualize a real-life system without tbe

resources needed for the full project. Prototyping allows

management to evaluate the system and make an informed

decision. Some of the terms used by respondents to

describe this include:

test some of the concepts,

validation of concepts, find out what system can do, and

determine impact on database.

Innovativeness

Innovation is an indicator of tbe foundation of the project,

i.e., is it a new development (high innovation) or a

modification to an existing system (low innovation)? New

system development (high innovation) requires a higher

user/developer interaction than modifications to existing

systems (low innovation). Thus, it would seem appropriate

to use prototyping, which facilitates the interaction between

user and developer22. Respondents indicated that systems

considered new development, from scratch, or auto-

mation of manual process should use a prototyping

approach.

User experi ence

User experience

refers to the users exposure to

computers and computerized systems. Lack of automation

experience is seen as a risk induce?. In this case, proto-

typing is a way to decrease risp. Users unfamiliar with

automated systems can develop a better idea of their system

requirements by being exposed to a prototype3,2s. Sophis-

tication of user, computer literacy of user, and users

DP experience, were considered by this studys respondents

in the decision to use prototyping.

User inv olv ement

Almost all advocates of prototyping point to user

involvement as one of the major advantages of prototyping.

Prototyping, compared to the traditional systems develop-

ment life cycle, requires more communication and inter-

action between the user and developer. Respondents felt

that, when user involvement was necessary, prototyping

should be used to facilitate the involvement. For example,

some indicated that prototyping should be used to gain user

interest and increase user activity in development

process.

Impact importance)

Responses such as criticality of system, mission critical

status, effect on business, and potential harm of bad

system indicate that IS managers view the impact of the

system on the organization as a factor in making the proto-

typing decision. Critical systems-systems that operate,

manage, and control the daily business activities-have a

very broad and strong impact on the organization.

Prototyping should be used for critical systems22329. or

critical systems it is extremely important that the system

meet specifications-prototyping facilitates this important

requirement.

Develo per experi ence

A developers experience with similar applications and the

developers overall experience impact the amount of

required user interaction. Possibly, if a developer has

development experience with similar applications (i.e.,

high developer-task proficiency), then prototyping should

not be used3. The reason is that when the task is familiar

to the developer, less user interaction is required to

determine requirements. Unnecessary involvement of users

may increase development time3.

Ot her vari ables w ort h menti oning

Other variables that respondents indicated, but did not

meet the 10% cut-off for inclusion in the table, include:

developer experience with prototyping tool (six times);

management support (four times); history of user/developer

116

Information and Sojiware Technology 1995 Volume 37 Number 2


5/6

When o protot ype: B C Hardgrave

Table 4. Decision variables: a comparison of this studys findings to

previousstudtes

Decision

ariables

This study Doke et al. Carey and

1992)

C-Y

1989)

when the user lacks DP experience? Additional research is

needed to evaluate the impact of the variables and the

resulting decision to use prototyping on the eventual

success of the information system.

Unclear requirements X X X

Large systems

X X

Complex systems

X X

Availability of tools X

X

On-line systems

X X X

Project duration

X X

Feasibility/testing X

X

New development

X X

User lacks DP experience X

Need user involvement X

X

Critical system

X

Developers experience

X X

Need early results

X

Alternatives need evaluated

X

Communicate design widely

X

Acknowledgements

The author is indebted to the reviewers for their valuable

comments, and to the companies that participated in this

study.

References

communication (three times); and number of users (two

times). According to the responses received, prototyping

should be used when the developer has experience with the

prototyping tool, when the project has management support,

and when only a few users are involved. Additionally,

prototyping may facilitate better communication when a

history

of poor communication between users and developers

exists.

Langle, G B, Leitheiser, R L and Naumann, J D A survey of

applications systems prototyping in industry In& & Manage. Vol 7

(1984) pp 273-284

Necco, C R, Gordon, C L and Tsai, N W Systems analysis and

design: current practices, MIS

Quarterly

Vol 11 No 4 (December

1987) pp 461-475

Carey, J M and McLeod, R Use of system development methodology

and tools J. Syst. Manage. VoI 39 No 3 (March 1988) pp 30-35

Doke, E R An industry survey of emerging prototyping methodologies

In & Manage. Vol I8 (April 1990) pp 169-176

Carey, J M and Currey, J D The prototyping conundrum Datamation

Vol 35 No 11 (June 1989) pp 29-33

Doke, E R, Swanson, N E and Hardgrave, B C The decision to

prototype information systems: a pilot study

1992

Proc. of he National

Decision Sciences Institute, San Francisco, CA (November 1992) pp

917-919

Summary

Table 4 summarizes the previous discussion and compares

the results of this study to the empirical studies mentioned

in the second section. As seen in Table 4, all four of the

variables in Carey and Curreys study were also

uncovered in this study. Similarly, eight of the variables

from the Doke et aL6 study corresponded to variables

from this study. Of the top 12 variables discussed in this

paper, only two variables, user lacks DP experience and

critical system, were not included in either the Carey and

Currey or Doke

et al. 6

study. The close association among

the three studies suggests that a set of decision variables

does indeed exist and is being used in industry.

Hardgrave, B C and Wilson, R L A survey of guidelines for the

proper usage of information system prototyping 1993 Proc. of the

National Decision Sciences Institute, Washington, DC (November

1993) pp 830-832

Niederman, F, Brancheau, J C and Wetherbe, J C Information

systems management issues for the 1990s

M IS Quarterly

ol 15 No 4

(December 1991) pp 475-500

Bums, R N and Dennis, A R Selecting the appropriate application

development methodology Dorabase Vol 17 No 1 (Fall 198.5) pp 19-23

10 Davis, G B Strategies for information requirements determination

IBM Systems Journal

Vol 21 (1982) pp 4-3 I

I I Naumann, J D and Jenkins, A M Prototyping: the new paradigm for

systems development MIS Quarterly Vol6 No 3 (September 1982) pp

29-44

12 Saarinen, T System development methodology and project success

If. Manage. Vol 19 (1990) pp 183-193

I3 Deamley, P A and Mayhew, P J In favour of system prototypes and

their integration into the systems development cycle Computer J. Vol

26 No 1 (1983) pp 36-42

Conclusion

This study has attempted to determine a set of variables

used by IS managers in their decision to use a prototyping

approach to IS development. The top 12 variables which

were suggested by respondents include: unclear require-

ments; large systems; complex systems; availability of

tools; on-line systems; project duration; feasibility/testing;

new development; user lacks DP experience; need user

involvement; critical system; and developers experience.

These variables correspond closely with similar studies

from Carey and Currey5 and Doke et

aL6.

I4 Guimaraes, T Understanding implementation failure J. Syst. Manage.

Vol 32 No 3 (March 1981) pp 12-17

15 El Louadi, M, Pollalis, Y A and Teng, J T C Selecting a systems

development methodology: a contingency framework

In5 Res.

Manage. J.

Vol 4 No 1 (Winter 1991) pp ll--19

I6 Burch, J G Systems analysis, design, and implementation Boyd &

Fraser, Boston, MA (1992)

I7 Carey, T T and Mason, R E A Information system prototyping:

techniques, tools, and methodologies ZNFOR Vol 21 (August 1983)

pp 177-191

I8 Whitten, J L, Bentley, L D and Barlow, V M Systems analysis

design methods (3rd edn) IRWIN, Burr Ridge, IL (1994)

I9 Alavi, M An assessment of the prototyping approach to information

systems development Comm. ACM Vol 27 No 6 (June 1984) pp

556-563

It appears that, based upon the results of this study and

similar studies, a set of factors is being used by IS managers

in their decision to use prototyping. Future research must

now evaluate the quality of these variables. For example:

Is prototyping the best strategy to use when the require-

ments are unclear? Is prototyping the best strategy to use

20 Carey, J M Prototyping: alternative systems development method-

ology

If: and Soft. Technol.

Vol32 No 2 (March 1990) pp 119-126

21 Graham, D R Incremental development: review of nonmonolithic

life-cycle development models If: and Sof. Technol. Vol 31 No 1

(January/February 1989) pp 7-20

22 DOS Santos, B L MIS project mangement: a contingency approach

Management Decision

Vol 26 No 6 (1988) pp 22-28

23 Pressman, R S

Sojiware engineering: a practitioners approach

(3rd

edn) McGraw-Hill (1992)

24 Alavi, M The evolution of information systems development

approach: some field observations DATABASE Vol 15 No 3 (Spring

1984) pp 19-24

hformation and Soj?ware Technology I995 Volume 37 Number 2

117


6/6

When o protot ype: B C Hardgrave

25 Mahmood, M A System development methods-a comparative

investigation MIS Quarter ly Vol 11 No 3 (September 1987) pp

293-311

26 Ahituv, N, Hadass, M and Neumann, S A flexible approach to

information system development MIS Quarterly Vol 8 No 2 (June

1984) pp 69-78

27 Tate, G and Verner, J Case study of risk management, incremental

development, and evolutionary prototyping In5

and Soft. Technol.

Vol 32 No 3 (April 1990) pp 207-214

28 Gavurin, S L Where does prototyping fit into IS development? J.

Sysf. Manage. Vol 42 No 2 (February 1991) pp 13-17

29 Boar, B Application prototyping: a life cycle perspective J. Syst.

Manage.

Vol 37 No 2 (February 1986) pp 25-31

30 Kraushaar, J M and Shirland, L E A prototyping method for

applications development by end users and information systems

specialists MIS Quarterly Vol 9 No 3 (September 1985) pp 189-197

Appendix: Complete list of factors

(in order by frequency of response)

Requirements determination (24)

Project size (19)

Complexity (16)

Availability of tools (16)

Mode (16)

Project duration (15)

Feasibility/testing (14)

Innovativeness (14)

User experience (12)

User involvement ( 12)

Impact (importance) (10)

Developer experience (8)

Developer experience with tools (6)

Management support (4)

History of end-user/systems analyst communication (3)

Reusability (3)

Impact on other programs (3)

User satisfaction (2)

Application platform (2)

Training tool (2)

Number of users (2)

Manhours available (2)

Distributed or centralized (1)

Design definition (1)

Type of project (1)

Interference with user needs (1)

Long life requirements (1)

Project definition (1)

Provide sign-off document (1)

Identify data elements (1)

Relational DBMS design (1)

Information flow (1)

Need to manage expectations (1)

Impact on users (1)

Request (1)

Program design (1)

Short life requirements (1)

System throughput (1)

User availability (1)

Data structure (1)

Consistency (1)

Conference room pilot (1)

Broad brush specifications (1)

Number of team members (1)

Cross applications (1)

Prototyping on small scale (1)

System flow-keying information (1)

Human engineering (1)

Note: the number in parentheses indicates the frequency of response.

8

Inf ormat ion and Sofrw are Technolo gy 1995 Vo lum e 37 Num ber 2

when to prototype decision variables used in industry 1995 information and software technology

Documents