factors influencing risks and outcomes in end-user … · factors influencing risks and outcomes in...

Proceedings of the 29th Annual Hawaii International Conference on System Sciences - 1996

Factors Influencing Risks and Outcomes in End-User Development

Diane Janvrin Department of Accounting

University of Iowa Iowa City, IA 52242

Abstract

End-user developed applications introduce many control risks into organizations. Literature suggests that influencing factors include developer experience, design approach, application type, problem complexity, t ime pressure, and presence or absence of reviav procedures. This research explores the impacts of diffeTent design approaches through two field experiments evaluating the use versus non-use of a structured design methodology when developing complex spreadsheets. Our results indicated that subjects using the methodology showed a significant reduction in the number of “linking errors,” i.e., mistakes in creating links between values that must pow from one area of the spreadsheet to another or from one worksheet to another in a common workbook. We also observed that factors such as gender, application expertise, and Workgroup configuration influenced spreadsheet error rates as well.

1. Introduction

During the past decade, the use of end-user developed applications, especially microcomputer spreadsheets, has grown exponentially. However, when identifying and evaluating risks within organizations, end-user developed computer applications are often overlooked. Research has revealed that as many as 44% of all end-user spreadsheets contain at least one error [5]. Such errors can have catastrophic results. For example, a Florida based construction company lost $254,000 by relying on an incorrect spreadsheet analysis for project costing [34].

Previous research suggests that errors in end- user developed applications may be influenced by several factors. The purpose of this research is to investigate the impacts of different design approaches on spreadsheet error rates and

Joline Mom’son Department of MIS

University of Wisconsin, Eau Claire Eau Claire, WI 54702

development times. The structure of the paper is as follows: Section 2 describes related work that forms the basis for our initial research model. Section 3 describes our structured spreadsheet design methodology, and Section 4 describes our experimental studies. Section 5 presents conclusions, limitations, and future research directions.

2. Background and Model Development

Cotterman & Kumar [9] classify end-user computing risks according to three major dimensions: end-user development, end-user operation, and end-user control of information systems. To our knowledge, no work has specifically identified and attempted to empirically validate factors impacting outcomes in end-user developed applications. Literature suggests that potential factors include developer attributes, development approach, developer configuration, problem/process characteristics and application domain. The following sections explore these areas and identify measures for predicting outcomes.

Developer Attributes Cotterman and I&mar [9] define an end-user as

my 0% anizational unit or person who has an interaction with the information system as a producer or consumer of information. Our work focuses on end-users who are both producers and consumers of information. Several studies have indicated that end-users are diverse, and that this diversity leads to a need for differentiated education, training, support, software tools and risk assessment (e.g., [14,19, 301).

Harrison & Rainer [17] assert that specific attributes include age, gender, computer experience, computer confidence, math anxiety, and cognitive style. Curtis et. al.‘s field studies on

1060-3425/96 $5.00 0 1996 IEEE 346

Proceedings of the 1996 Hawaii International Conference on System Sciences (HICSS-29) 1060-3425/96 $10.00 © 1996 IEEE


software development (e.g., [lo]) repeatedly confirm the importance of understanding the application domain. Adelson & Soloway [l] emphasize the importance of expertise in software design. Frank, Shamir & Briggs [15] found that PC user knowledge - i.e., application expertise - positively impacted security-related behavior. Further, several researchers indicate that user training and relevant experience are important factors (e.g., [2, 331).

Development Approach Design activities for end-user developed

applications tend to be unstructured and often non- existent. Salchenberger [33] notes that endl-users tend to develop and implement models without first performing requirements analysis or developing a system design. Brown & Gould [5] observed that their participants spent little time Pl arming before launching into spreadsheet coding. Ronen, Palley and Lucas [31] suggest that s#everal attributes of spreadsheets (i.e., non-professional developers, shorter development life cycles, ease of modification) tend to preclude a formal sprt?adsheet analysis and design process.

Almost all published spreadsheet (design approaches agree on one principle: the importance of keeping spreadsheet work areas small in order to make errors easier to find, limit the damaging effects of errors, and make spreadsheets easier to understand and maintain. Researchers have noted that structured design methodologies increase the accuracy and reliability of information systems (e.g., [241). Salchenberger [33] proposes the use of structured design methodologies in end-user development and discusses their effectiveness in two situations. Cheney, Mann & Amoroso [17] also support the use of a structured design methodology; they found that the more structured the tasks being performed by the end-user, the more likely the success of the application. Ronen, Palley and Lucas [31] describe a structured spreadsheet design approach based on modifying conventional data Bow diagram symbols to create spreadsheet flow diagrams. Several practitioner-directed articles encourage the use of structured design approaches and planning for spreadsheet development.

Developer Configuration Par&o Rr Halverson [28] and Nardi & Miller [26]

both note that the accuracy of end-user developed applications may be increased by having multiple developers collaborate. However, Panko &

Halverson [28] note that the increased communication and coordination costs of working in groups may outweigh the benefits of reduced error rates.

Problem/Process Characteristics Several studies identify the difficulties

associated with complex versus non-complex programming problems (e.g., [13, 321). Studies on the effect of t ime pressure on programming activities indicate that due to time pressure, end- user developers may not spend sufficient time on problem definition and diagnosis [2, 61. These researchers also noted an absence of formal review and validation procedures within end-user developed applications.

Application Domain Research on end-user development generally

falls into one of two streams: general application development and spreadsheet development. (Notable exceptions include Frank, Shamir & Briggs [15] and Batra [3]). Par&o and Halverson [28] appropriately summarize the state of spreadsheet research as follows:

“ln the past, spreadsheeting has been the Rodney Dangerfield of computer applications. Although everyone agrees that it is extremely important, fm researchers have studied it in any depth. Given its potential for harm as well as for good, this situation needs to change. “

Psychologists Hendry and Green [20] assert that spreadsheets have the ability to simplify programming by suppressing the complexities associated with conceptually simple operations like adding a list of numbers or handling input and output. As a result, users tend to ignore spreadsheet weaknesses such as the potential for errors or the difficulties experienced when debugging or reusing spreadsheets.

Outcomes Salchenberger [33] suggests that factors to use in

judging the quality of end-user developed applications include reliability, ease of use, maintainability, and auditability. Reliability can bc defined as the absence of logical and mathematical errors in a model. One of the earliest studies on spreadsheet outcomes (conducted by Brown and Gould [5] in 1987), found that of 27 spreadsheets created by experienced spreadsheet users, 41% contained user-generated errors. In another study,

347



Davies & &in [ll] audited a sample of spreadsheets within an organization and discovered 4 of 19 worksheets reviewed contained major errors and 13 of 19 had inadequate documentation. rank0 & Halverson [29] asked students to develop a spreadsheet model working individually, in groups of two, and in groups of four, and observed total error rates ranging from 53% - 80%. They also noted that very little time (only a few minutes) was spent in design prior to implementation.

Panko and Halverson [29] note that their participants made several different types of errors. At the highest level, errors could be divided into two types: oversight errors and conceptual errors. They define oversight errors as “dumb” errors such as typographical errors and misreading numbers on the problem statement. In contrast, conceptual errors involve errors in domain knowledge or general modeling logic. Par&o and Halverson [28] use Lorge and Solomon’s [23] terminology to further divide conceptual errors into Eureka and non-Eureka errors. Once an Eureka error is identified, everyone can quickly agree that the error exists. However, non-Eureka errors are errors in which the incorrectness of the action cannot be easily demonstrated.

Generally, researchers evaluate the usability of end-user developed applications with user satisfaction survey tools 114, 221. Salchenberger [33]

Developer Attributes Application Type / -x I

/ *Age l Gender * Computer Confidence * Domam Expertise l Application Expertise * Development Expertise - Math Anxiety - Cognitive Style - Training

\ * Experience

i End User-Developed

Application Outcomes -i l Reliability

l Ease of Use . ProblcmiProcess _+sSS--- -+ l Maintainability characteristics I ’ ’

’ l-----l

l Auditability /

* Problem Complexity ’ ’ ,r l cost

- Existence of Kevlew Procedures

Development Approach

Figure 1. Initial Research Framework

defines maintainability as the ability to change or update the system as needs change, and auditability as the ability to determine the source of data values in the model. Few researchers have evaluated end- user developed applications for either maintainability or auditability. We propose the inclusion of cost (usually measured in terms of development time) as another important end-user development outcome.

Based on prior theoretical and empirical work in this area, we have developed an initial research framework (Figure 1) that summ arizes end-user development risk factors and their relationships to potential outcomes. We now focus on determining the impacts of development approach on spreadsheet reliability and cost. The particular structured design methodology used in our studies is now described.

3. Structured Spreadsheet Design

Modem spreadsheet applications enable users to easily modularize large spreadsheets by creating “worksheets” (individual spreadsheet work areas) within a larger “workbook” (set of related worksheets). Users can link values from one worksheet to another within the same workbook, or link sheets among multiple workbooks. This technique readily supports scenario analysis, For example, all of the schedules leading to a master budget can be prepared on separate worksheets within a workbook, and then linked together so that the impact of a change in one parameter can be quickly evaluated with a single keystroke. However, this feature also leads to the likelihood of “linking errors.” Data on a source worksheet may be linked to the wrong cell on the target worksheet, or a value on a target worksheet that should have been linked from a previous worksheet is “hard- coded:” an actual value from a previous scenario is directly typed in. When the value should change for subsequent scenarios, the values for omitted or incorrect links do not change as they should, and errors result. To support the task of designing linked worksheets, we build upon the work of Ronen, Palley and Lucas [31] to develop a design methodology based on data flow diagram symbols.

I External I

Figure 2. Basic Modeling Symbols

348



Homebuilders’

Market

pgi--pshaE

Income

\_,/ Statement

Figure 3. Example Design Model : Master Budget for Small Home Fixture Manufacturer

Data flow diagrams (DFDs) are documentation tools used by systems analysts to show organizational data sources, destinations, processes, and data inputs and outputs. Our basic modeling symbols (Figure 2) are similar to those us:ed in standard DFD methodologies (e.g., Gane & Sarson Ml)- These include an external data source (depicted by a rectangle), and data flows (shown by labeled directed arrows). Worksheets are analogous to processes, and are shown as labeled circles. Data flows depict data inputs from external sources to worksheets, and data that is llinked among related worksheets.

Figure 3 shows an example design model. Worksheet pages are created to represent each of the modules shown on the design model (e.g., Input Page, Sales Budget, Production Budget, Pro Forma income Statement). Inputs from external sources are entered on a centralized Input Page, and1 then flow into the Sales Budget, where they are transformed into data items needed for calculations on the other schedules. Using named variables, the specified links are created. Inputs are easily changed on the lnput Page for scenario analysis.

4. Evaluation Studies

The basic research model guiding both studlies is shown in Figure 4. (The numbers following the variable names indicate if the variable was considered in Study #l, Study #2, or both). We hypothesized that both developer attributes and development approach would influence erro’r rate and development time. A positive relationship

between developer confidence and improved outcomes was noted by Harrison & Rainer [17]. Prior research has also noted a positive relationship between domain and application expertise and improved outcomes (e.g., [2, 61).

Structured systems design approaches (e.g. Constantine and Yourdon [S]), have successfully been used in systems development for over twenty- five years. This approach can be applied to spreadsheet design by identifying different worksheets and their corresponding data inflows and outflows. Thus, each individual worksheet effectively becomes a “black box” that takes a defined set of inputs and translates it into a prescribed set of outputs. Since the worksheet links are explicitly defined in the design, the result should be a decreased error rate for linking errors.

Study #l Subjects, Treatment, and Measures. Sixty-one

upper- and masters-level accounting and business administration majors in an accounting information systems course at a large public university were assigned to the treatment (structured design) and control (ad hoc design) groups based upon course sections. Assignment of treatments to specific sections was done randomly. All subjects were given one hour of formal training in spreadsheet development, and then instructed to develop a single spreadsheet workbook containing a series of linked worksheets using Microsoft Excel. The assigned problem contained ten separate worksheets, with a total of fifty-one linked data values among the worksheets. A paper template of

349



ard Deviations fStudv #I 1

Gender (1 =Male, 2=Female) j I.441 0.50, 1.50 ( Age Group(l=20-25, 2=26-30, 3=31-40,4=40-50) / 1.38 / 0.78: 1.56 0.981 1.301 0.671 1.191 0.541 1.w U.Y41

Class Level I1 =Jr.. 2=Sr.. 3=Grad) I l.QO/ 0.68; 2.11, ( I.58 1.821 0.69 1.97 0.60 / 1.83 0.75 ' 'Pretest SS Expertise ; 2.491 1.07; 2.61 ~ 1.20 2.431 1.02 2.68 1.053 2.30 1 .OQ

'Pretest Excel Expertise 1 2.341 1.14~ 2.221 1.00 2.41 / 1.19 2.42 I.231 2.27, 1.05 07 0 98 -FiT Eraertise 1 2.161 0.99 1.89 0.831 2.301 1.021 2.261 I.001 2

*Pretest Design Confidence ._.--. -. - -_r- -.-~

1.89’ 4.02 1 0.63 1 3.841 0.781 4.031 0.67 t

0.73 0.94

0.89:

3.721 2.87 j 2.73 j 2.931

2_40

4.85 9.51%/ TxZy

~

'Mean Number of Linking ErrorsNVorkbook

' l=Novice, 5=Expert) 2 I =Very Unconfident, 5=Very Confident 3 Total Possible Links = 51

t

I

t i

N = 66 I I "p <= "1; **p< =.05

The MANOVA analysis results imply that for structured design methodology was weakened subjects working alone, the use of the design when the subjects were working in pairs. Two methodology had a significant impact. potential explanations emerge: (1) subjects working Unfortunately, the advantage of using the in pairs gained other advantages that offset the

350



l Domain Expertise (2)

Development Approach

r* Ad hoc design (I, 2)

l Saucwred design (I. 2)

Outcomes l Reliability

-Link Errors (I, 2) -Conceptual

Enors (2) -Oversight

Errors (2) l cost

-Development Time (1, 2)

* Dev. Approach Usability (2)

Figure 4. Model for Experimental Studies.

the worksheets with one complete data set (from [25]) was given to all subjects to provide check figures and mitigate non-linking errors. (only numerical values were specified; the subjects still had to develop and correctly implement the underlying design in order to successfully perform scenario analysis). AU subjects were required to maintain logs recording time spent on design and development activities. The treatment group was required to submit their spreadsheet design prior to being assigned a password to enable computer data entry.

A pre-test questionnaire was administered to identify gender, age, and class level, as well as self- reported pre-test application expertise (in terms of both spreadsheets in general as well as the specific spreadsheet application, Microsoft Excel), DFD development expertise, and design confidence. Development time was determined by combining two measures: (1) self-reported design logs covering two- or three-day periods during the experiment; and (2) system-recorded measures of actual implementation times. The subjects were given sixteen days to complete the assignment. At the end of that period, questionnaires were administered to ascertain post-test spreadsheet and Excel expertise and design and coding confidence. Error rates were determined by examining the spreadsheet data files.

Analysis and Results. Due to an unexpected software constraint, we were forced to allow subjects in Study #l to work either individually or in self-selected pairs within treatment groups. (In Study #2 all subjects worked alone). Means and standard deviations for the independent and dependent variables for all subjects as well as

results sorted by configuration and treatment group are shown in Table 1. Analysis was done on a per- subject basis, with scores for time and error rate replicated for each subject in a given dyad.

We used a least squares multiple regression to test the research model components and proposed linkages. Examination of the correlations among the independent variables revealed no evidence of multicollinearity: that is, r’s <= .8 [4]. Table 2 shows that gender, age group, class level, and pretest Excel expertise had no impact on any of the outcomes, while configuration (singles versus pairs), treatment group (structured versus ad hoc design), and pre-test spreadsheet expertise all displayed significant correlations with one or more of the dependent variables.

Following Heise [18], the independent variables that did not explain a significant variation in the outcomes were excluded from further analysis. Table 3 shows that the variations in the dependent variables are explained to some degree by the independent variables in the reduced model. MANOVA analysis was used to detect potential interactions between the configuration and treatment variables on error rate. This revealed a weak significant difference (p = .lO) between the control to the treatment group, as well as a weak significant interaction (p = .lO) among the configurations and the treatment.

Summary and Discussion. Overall, subjects using the structured design technique exhibited significantly lower error rates: of the fifty-one possible links in the spreadsheet, the structured design subjects had an average of 3.57 (7%) linking errors, while the ad hoc design subjects averaged 4.1 (10%) linking errors. We believe this was because the design methodology forced the developers to explicitly identify the links between worksheets. This explicit identification was then carried over into the implementation. Virtually all of the linking errors were due to omitted links (where a value had been “hard coded” instead of linked) rather than incorrect links. This may have been because subjects were relying heavily on the check figures provided for the first scenario instead of examining the reasonableness of their answers for the subsequent cases.

This improvement in linking errors was mitigated by subject configuration: subjects working alone within the ad hoc design group averaged 7.44 linking errors per spreadsheet (14%), while subjects working alone in the design group and both sets of paired subjects all averaged approximately 3.8 linking errors per spreadsheet (7%).

351



impact of the design approach; or (2.) another external factor (or factors) induced some subjects to elect to work in pairs, and this factor also enabled them to create spreadsheets with fewer linking errors.

Our results also indicated that subjects working in pairs reported higher post-test spreadsheet expertise but lower design confidence. The latter result seemed inconsistent with literature suggesting that collaboration usually leads to greater confidence. The significantly lower overall t ime investment for the structured design subjects was unexpected but encouraging. (This result does not consider the time spent learning the design methodology initially, however). Finally, as expected, pre-test spreadsheet expertise was a significant predictor of post-test spreadsheet expertise, reduced error rates, and reduced time investments. The results of this study suggested that developer attributes, configuration and the development approach are all potential factors impacting end-user development outcomes. However, the unexpected results for time and post- test confidence as well as the unexpected configuration variable induced us to perform a second study with revised procedures and measures.

Study #2 The problem used in Study #2 was slightly more

complex than the one used in Study #l,. with ten different worksheets and a total of sixty-six linked cells. All subjects were required to work individually so as to increase sample size and allow a more isolated investigation of the impacts of the design methodology and developer attributes. Error types observed were expanded to include conceptual and oversight errors as well as linking errors; a blank template (from Hilton [2I], p. 439- 443) with only a single check figure was given. Explicit post-test measures were introduced to assess design confidence, coding confidence, and confidence that the spreadsheet was error-free. Additionally, a measure was included to explore the impact of potential domain expertise: subjects who were accounting majors had additional experience and coursework in preparing financial reports and other master budgeting topics than the non-accounting majors. Subjects in the treatment group were also asked to rate their post.-treatment DFD development confidence, and to rate their satisfaction with the development approach. Specifically, they were asked the degree to which

they agreed or disagreed with statements regarding: (1) whether the DFD design methodology helped them develop a better spreadsheet; (2) if using the methodology made the development effort take longer; and (3) whether they would use the methodology the next time they developed a complex spreadsheet.

Analysis and Results. Table 4 reports overall means and standard deviations for all measures. Correlation analysis indicated a high degree of correlation between pretest Excel expertise and overall spreadsheet expertise; since the Excel expertise measure had the highest correlation with the dependent variables, it was included in the regression analysis, and the pretest spreadsheet expertise measure was omitted. Table 5 shows the initial regression analysis for all variables. Significant results were attained for gender, pretest Excel expertise, DFD expertise, and design confidence. A reduced model analysis performed bY omitting the non-significant independent variables yielded essentially the same results. Table 6 shows the regression analysis results for the design methodology attitudinal measures obtained from the treatment group subjects.

Summary and Discussion. Table 5 suggests that our independent variables accounted for significant differences in all of the dependent measures except number of conceptual errors and overall t ime spent on design and coding. An interesting gender difference surfaced: females reported significantly (p<=.O5) less perceived posttest spreadsheet expertise and confidence that their spreadsheets were error-free. This concurs with Harrison & Rainer’s [17] finding that males tend to have more experience with computers and higher confidence levels. Interestingly, the females also had significantly (p<=.O5) fewer oversight errors. We postulated that this may have been a result of their lower confidence levels: perhaps the females reviewed their spreadsheets more and caught their oversight errors.

Subjects in higher class levels (i.e., seniors and masters students) also displayed less confidence that their spreadsheets were error free. There were no notable correlations between class level and any of the other independent variables; in fact, subjects in the higher levels seemed in general to have less pre-test application expertise. We attributed this result to the inexperience (and often misplaced?) overconfidence of youth.

352



Table 5. Regression Analysis Results (Study #2)

lomein Experience (l=Other, 2=Acct 0.57’ 0.38 1.02” 0.24 o:ss 2.72 0.17 0.08 2.52 -3.66 reatment Group (l=Control. 2=Treat) -0.12 -0.19 0.29 0.58 -0.‘15 -5.75” 0.06 0.00 -5.63”’ 1.03

Pre. Excel Expertise 0.30” 0.29” 0.30’ 0.43” 0.06 -3.18 -0.07” -0.15 -3.22’ -1 .ll

Pre. DFD Ex~ertiie 0.14 0.16 0.27” 0.16 0.04 -0.34 0.47 0.02 -0.23 -0.64

Domain experience was a positive predictor of post-test spreadsheet expertise (p-+.05) and posttest design confidence (p<=.Ol). Neither of these findings were particularly surprising. Domain experience was somewhat correlated to pre-test spreadsheet expertise (r=.19) as well, so apparently the accounting majors had more prior expertise with spreadsheets than the non- accounting majors. The problem was fairly complex, with ten different worksheets and a total of sixty-six links, so it is reasonable to expect that domain experience would contribute to d.esign confidence.

Using the structured design methodology significantly (p<=.Ol) reduced the number of link errors, but had no impact on the number of oversight or conceptual errors. Interestingly, link errors were by far the most common error made. Each spreadsheet (for the combined control and treatment groups) had an average of 9.07 total errors; of these, 84% were linking errors. It is important to note that using the design methodology had no significant impact on total t ime spent on the project (aside from the time taken to learn the methodology initially).

353



Table 6. Regression Analysis Results for Treatment Attitudinal Measures (Study #2)

1

;;<=.05; ‘“pc=.Ol ; ‘“‘pc=.OO1 1 ‘1 =Strongly Disagree; 5=Strongly Agree ?=Jr., 2=Sr., J=Grad. j ‘1 =Other, 2=Acctg.

Pretest Excel expertise was the most significant predictor, positively impacting post-test spreadsheet expertise (p<=.Ol), post-test Excel expertise (p<=.Ol), post-test design confidence (p<=.O5), and post-test coding confidence (p<=.Ol). Subjects with higher pre-test Excel expertise also made fewer conceptual errors (p<=.Ol) (and fewer overall errors (pc.01). These findings concur with the results of many studies emphas’izing the importance of application experience and resulting expertise (e.g.,[l, 271).

The attitudinal responses summ arized in Table 6 indicated that females were significantly less confident of their post-test DFD development ability. Pre-test design confidence was positively correlated to the belief that creating the DFD was helpful and would be used in the future.. Subjects with a higher degree of pretest spreadsheet expertise did not think that developing the DFD caused the development effort to take longer. Overall, from an attitudinal standpoint, the subjects were ambivalent to the costs and benefits of the structured design approach.

5. Conclusions, Limitations, and Future Research

Our experimental studies confirnned that developer attributes, specifically gender and domain and application expertise, significantly impacted error rates. Additionally, we have shown that using the structured design methodology significantly improved reliability for two different linked spreadsheet applications. Study #I also

inadvertently suggested that developer configuration is an important factor and has potential interactions with the development approach.

As mentioned previously, our initial study was somewhat confounded because the pairs were self- selected. Our future studies will address the impacts of collaborating in non-self-selected groups. Additionally, our research has focused on absolute error rates. A limitation of this approach is the question of how error rates are correlated to ultimate outcomes. i.e., the bottom line or poor decisions resulting from spreadsheet models. Also, the impact of the structured development approach on problems of differing complexity needs to be explored. Another area that needs to be addressed in the future involves the outcomes associated with the use of end-user developed spreadsheets by, other end users in terms of usability, maintainability, and audibility. Additionally, it will be interesting to explore the results when end-user developers know that their spreadsheets will be reviewed or audited. Finally, we have only considered spreadsheets; we need to determine if our overall causal model is applicable to other end- user development domains as well.

As end-user development increases in organizations the identification of strategies to minimize risks and improve outcomes is becoming increasingly important. This research has made an important contribution both by identifying end-user development risk factors and outcomes, and by developing and empirically evaluating the impacts of a structured design approach for spreadsheet development.

References

Adelson, 13. and Soloway, E., “The role of domain experience in software design,” IEEE Trans. on Swftware Eng., SE-11 (Nov. 1985), 1357- 1360. Alavi, M. & Weiss, I., “Managing the risks associated Twith end-user computing, “J. MIS 2,3 (Winter 1986), 413426. Batra, D., “A framework for studying human error behavior in conceptual database modeling,” Info. & Management 25 (Sept. 1993), 121-131. Billings, R.S. and Wroten, S.P., “Use of path analysis in industrial/organizational psychology: Criticisms and suggestions,” 1. Applied Psych. 63 (June 1978), 677-688.

354



5.

6.

Brown, P.S. & Gould, J. D., “An expenmental study of people creating spreadsheets,“ACM Trans. on ofice Irzfo. Systems 5 (July 1987), 258- 272. Cale, E., “Quality issues for end-user developed software,” I. Sysfems Management (Jan. 1994, 36- 39. Cheney, I?., Mann, R. and Amoroso, D., “Organizational factors affecting the success of end-user computing,” 7. MIS 3,Z (Summer 1986), 65-80. Constantine, L.L. and Yourdon, E. Structured Design. Prentice-Hall, Englewood Cliffs, New Jersey, 1979. Cotterman, W. & Kumar, K., “User cube: a taxonomy of end users, C. ACM 32, 11. (Nov. 1989), 1313-1320.

10. Curtis, B., Krasner, H. and Iscoe, N., “A field study of the software design process for large systems. C. ACM 31, 11 (Nov. 1988), 1268-1287.

11. Davies, N. & Ikin, C., “Auditing spreadsheets,” Australian Accountant @ec. 1987), 54-56.

12. Davis, F., “Perceived usefulness, perceived ease of use, and user acceptance of information technology,” MIS Quarterly 13,3 (Sept. 1989) 318339.

13. DeRemer, F. & Kron, H., “Programmin g-in-the- large versus programming-in-the-small,” IEEE Transactions on Software Engineering SE-2, (June 1976), 80-86.

14. Doll, W. & Torkzadeh, G., “A discrepancy model of end-user computing involvement,” Management Sci. 35, 10 (October 1989), 1151- 1171.

15. Frank, J., Shamir, B. and Briggs, W., “Security- related behavior of PC users in organizations,” Info. 6 Mgrnt. 21 (1991), 127-135.

16. Gane, C. and Sarson, T. Structured Systems Analysis. Prentice-Hall, Englewood Cliffs, N.J., 1979.

17. Harrison, A. & Rainer, R.K. Jr., “The influence of individual differences on skill in end-user computing,” 1. MIS 9,1, (Summer 1992), 93-111.

18. Heise, D.R., “Problems in path analysis and causal inference,” in E.F. Borgatta (Ed.), Sociological methodology, (38-73), Jossey-Bass, San Francisco, 1969.

19. Henderson, J. Pr Treaty, M., “Managing end- user computing for competitive advantage,” Sloan Mgmt. Review (Winter 1986) 3-74.

20. Hendry, D.G. Rr Green, T.R.G., “Creating and

comprehending and explaining spreadsheets: a cognitive interpretation of what discretionary users think of the spreadsheet model,” Int. 1. of Human-Computer Studies 40 (199-l), 1033-1065.

21. Hilton, R. Managerial Accounting (2nd Ed.). McGraw Hill, New York, 1994,439440.

22. Igbaria, M. and Parasuraman, S., “A Path Analytic Study of Individual Characteristics, Computer Anxiety and Attitudes toward Microcomputers,“ 1. of Mgmt 13,3 (1989), 373- 387.

23. Lorge, 1. & Solomon, H., “Two models of group behavior in the solution of Eureka - type problems,” Psychometrika, 20(2), 1955 (cited in VW.

24. Markus, L. &L Keil, M., “If we build it, they will come: designing information systems that people want to use,” Sloan Mgmt. Review (Summer 1994), 11-25.

25. Moriarity, S. Rr Allen, C.P. Cost Accounting (3rd Edition). John Wiley Rr Sons, New York, 1991, 285-291.

26. Nardi, B. &I Miller, J., “Twinkling lights and nested loops: distributed problem solving and spreadsheet development,“ lnt. 1. Man-Machine Studies 24 (1991), 161-184.

27. Ngwenyama, O., “Developing end-users’ systems development competence,” Info. & Management 25 (Dec. I993), 291-302.

28. Par&o, R.R. & Halverson, R. P., “Individual and group spreadsheet design: Patterns of Errors,” in Proc. of the 27th HICSS, Vol. IV, 4-10, IEEE Computer Society Press, 1994.

29. Panko, RX. & Halverson, R.P., “Patterns of errors in spreadsheet development,” unpublished working paper, University of Hawaii, 1995.

30. Rockart, J. & Flannery, L., “The management of end user computing” C. ACM 26,lO (Oct. 1983), 776-784.

31. Ronen, 8.; Palley, M.A. & Lucas, H.C., “Spreadsheet analysis and design,” C. ACM 32, 1 (Jan. 1989), 84-93.

32. Ross, D. & Schoman, K., “Structured analysis for requirements definition,” IEEE Trans. Sofhvare Eng. SE-3,l (Jan. 1977), 6-15.

33. Salchenberger, L., “Structured development techniques for user-developed systems,” Info. & Mgmt. 24 (Jan. 1993), 41-50.

34. Simkin, M. G., “How to validate spreadsherts,” J. of Accountancy 164,5 (Nov. 1987), 130-138.

355


factors influencing risks and outcomes in end-user … · factors influencing risks and outcomes in...

Documents