69

Research

A User Evaluation of Information Characteristics related to Demand Deposit Systems: An Empirical Analysis

Avi Rushinek Department of Accounting, Uniuersity of Miami, P.O. Box

248031, Coral Gables, FL 33124, USA; (305) 284- 5492

and

Sara F. Rushinek Department of Management Science and Computer Information

Systems, Unioersity of Miami, Coral Gables, FL 33124, USA;

(305) 284 - 6595

In a recent issue of Information and Management, Neu-

mann and Segev developed a correlation analysis of user

evaluation of information characteristics (IC) for system im-

provement, The purpose of this note is to offer a modest

extension to their theory which is otherwise viewed as an

excellent attempt at modeling realistic aspects of the user IC

evaluation. This empirical study factor analyzes the four IC

variables. These variables (Content, Accuracy, Frequency, and

Recency) are also ranked according to their relative dominance.

The differentiation level among these IC variables is found to

be low. Thus, the four variables have been reduced to one

factor. Moreover, a formula for an index of IC has been set.

Thus, it is possible to condense these four variables into one

factor.

Keywords: User Evaluation, Information Characteristics, De- mand Deposit Systems, Banking, Information Sys-

tems, Factor Analysis, Content, Accuracy,

Frequency, Recency, Computer Systems, Electronic

Data Processing (EDP), Interactive. Online Sys- tems.

North-Holland Information & Management 7 (1984) 69-72

1. Introduction and Objectives

User evaluations of computer systems have been prominent in recent research. However, one of the major sources of frustration in studying these is the large number of information characteristic (IC) variables that are related to it and the intricacies of these relationships.

In the literature, researchers have identified various system attributes. Adcock et al. [2] and subsequently Davis [4] considered the following system attributes: purpose, mode and format, re- dundancy/efficiency, rate, frequency, determinis- tic/ probabilistic, cost, value, reliability and valid- ity. Gorry and Scott Morton [6] selected source, scope, aggregation, time-horizon, currency, accu- racy and frequency as information characteristics. Adams and Schroeder [I] studied the accuracy,

_-_____--_ -~~-.

Sara Rushinek is currently an assistant professor of Management Information Systems in the Department of Mana- gement Science and Computer Infor- mation Systems at the University of Miami. She received her Ph. D. from the University of Texas at Austin. Her current interests are in the areas of user involvement in computerized management information systems, mini/micro computers, and expert systems.

Avi Rushinek is an assistant professor of Accounting and Information Sys- tems at the University of Miami. He holds a Ph. D. from the University of Texa at Austin. His interests include accounting information systems, mini and micro computers, computer-audit- ing, cost and managerial accounting, financial planning and modeling.

037%7206/84/$3.00 0 1984, Elsevier Science Publishers B.V. (North-Holland)

70 Reseurch

age, repetitiveness, summarization, descriptive content and source. This multitude of variables highlights the need for identification of dominant factors. Neumann and Segev [12] have contributed greatly to identifying systems IC. This study * is based upon the attributes chosen by Ein-Dor and Segev [5] and Neumann and Segev [12] which were the following: 1. Accuracy ~ percentage of errors in the data. 2. Content - information in the system.

3. Frequency - elapsed time between informa- tion disclosure to users.

4. Recency - delay between the real occur- rence and its presentation to

users. Neumann and Segev [12] have concluded the

following, based on their correlation analysis:

“correlations are positive, indicating that the evaluations of the various characteristics are not one at the expense of the other, but complemen- tary. The high positive correlations indicate that the four studied characteristics of information are perceived as strongly complementary by the users.

We found that content is a dominant information characteristic and the assessment of this character- istic affects other evaluations. That is, there is a spill-over effect in which evaluation of content causes evaluations of other characteristics to be similar.

. . . [Sampled users] do not differentiate among the various characteristics of the system. Content is the dominant information characteristic and there is a spill-over effect from users evaluations of the content of information to their evaluation of the accuracy, frequency and recency.”

2. Hypotheses

The aforementioned theories were viewed as null hypotheses for the present study in the follow- ing way: A. The lack of differentiation hypothesis suggests

that the f&r IC variables are somewhat redun- dant and can be reduced to one factor with little information loss.

B. The dominance hypothesis suggests that one of the four IC variables is dominant, while others

* The authors wish to acknowledge the helpful comments of E.H. Sibley and the anonymous reviewers of this paper.

are subordinate. The IC variable “Content” is

expected to be the most dominant, based on the

aforementioned literature.

According to this theory, four IC variables can be expected to collapse into one factor. Moreover, the authors expect h priori that the IC variable named “Content” will have the largest factor load- ing. Similarly, the other IC variables will also be ranked, according to the loadings.

Various other IC variables correlation have been discussed in the literature. Possible dominance or high correlation have been suggested by Ring and Cleland [7], Mintzberg [S], Mock [9], Munro and Davis [lo], and Neuschel [ll]. Although the terms used by the present study, differentiation and dominance, have been mentioned or inferred in all of these studies, they have not been analyzed methodically and systematically in the literature.

3. Data Collection and Sample Selection

A sample of 81 bank branch managers of a major (Fortune’s 500) bank were interviewed by Neumann and Segev [12]. Eighteen branch managers were excluded since their branches had operated less than three years. Other managers were excluded from the sample for one or more of the following reasons: (a) absence from regional meetings, (b) absence from the scheduled inter- view, and (c) failure to complete and return the questionnaire on time.

4. Defined Factors

Principal-component analysis asks what the best linear combination of IC variables would be. The best combination is such that would account for more of the variance in the data (original four IC variables) than any other linear combination of variables. For this analysis, no particular assump- tion about the structure of the IC variables is required [13]. As expected, the data in this case is rather redundant. Thus, only one factor exists. This factor is presented in Table 1, entitled: IC Factor Matrix Using Principal Factor with Itera- tions.

This shows the factor loadings of the IC varia- bles. As reflected by the alternative hypothesis, H(a), the single IC factor is less than the number

A. Rush&k and S. Rush&k / User Eoaluatton of Information Characteristrcs 11

Table 1 IC Factor Matrix Using Principal Factor with Iterations: Fac-

tor Loadings

IC Variable Name * IC Factor 1

Content

1. Accuracy 0.96182 ’ 2. Content 0.97264 ’ 3. Frequency 0.91775 3

4. Recency 0.91635 4

IC Variables are defined and explained by Neumann and

Segev [12]. Largest factor loading which indicates the dominant IC

variable [3]. Second largest, most dominant IC variable.

Third largest, most dominant IC variable. Smallest factor loading, least dominant IC variable.

of original IC variables (four). Similarly, the IC factor loading is significantly greater than zero. Accordingly, the null hypothesis, H(O), is rejected in favor of the alternate hypothesis, H(a). Conse- quently, one may conclude that these IC variables manifest substantial lack of differentiation. Fur- thermore, the IC variable “Content” appears to be dominant. Thus, one may conclude that there is lack of differentiation as well as a “spill over” effect.

5. Conclusions and Implications

In conclusion, the claims of Neumann and Segev [12] have been confirmed. Indeed, the four IC variables were similar and not substantially differ- ent from each other. This was supported by factor analysis which resulted in one IC factor. In addi- tion, the IC variable “Information Content” was found to be dominant and to spill over onto the other IC variables.

The implications of this study are that the present methodology of factor analysis may help the research to detect “holistic perception,” “lack of differentiation,” and “spill over effect” prob- lems early in the IC system analysis process. A factor analysis can be performed on a small pre- experimental (test) sample. Furthermore, this fac- tor analysis cannot only indicate lack of differ- entiation, but also quantify it, according to the number of redundant variables (three in this case). Quantifying the lack of differentiation can be par- ticularly useful for rank-ordering alternative

potential samples and selecting samples with acceptable differentiation. Accordingly, remedial action can be taken in order to alleviate these problems. Examples of such remedial action as suggested by Neumann and Segev [12] may in-

clude: (1) Sampling non-branch managers who may

manifest greater IC variable differentiation; (2) Using more independent IC variables and re-

ducing the “spill over effect;” (3) Using additional tools for system evaluation to

augment the IC users’ evaluation. In addition, the formula for computing a com-

posite IC variable or an estimated IC factor score for information content was constructed. This composite variable can be used as an index of measurement. One can use this single score to compare the performance of different systems at a given point in time. Alternatively, it can compare a given system across different points in time, in order to evaluate IC impact.

References

1’1

121

[31

[41

151

161

181

181

191

1101

C.R. Adams and R.G. Schroeder, Managers and MIS: “They Get What They Want,” Business Horizons XVI, 6

(1973) 63-68.

W.J. Adcock, W.A. Letzler. J.A. Terry, and M.W. Terry,

“A Framework of Evaluation of Management Information

Systems,” A Research Study for D.P. Group, Information

Systems Planning, International Business Machines Corp.,

Sloan School of Management, M.I.T. (Jan., 1968).

R.F. Cattell, “Factor Analysis: An Introduction to the

Essentials.” (I) The Purpose and Underlying Models. (II)

The Role of Factor Analysis in Research, Biometrics, 21:

pp. 190-215, 4055435, 1965.

G.B. Davis, Management Information Systems: Conceptual

Foundations, Structure and Deoelopment (McGraw-Hill, 1974).

P. Ein-Dor, and E. Segev, “Organizational Context and

the Success of Management Information Systems,” Management Science XXIV, 10 (1978) 1064-1077.

G.A. Gerry and M.S. Scott Morton, “A Framework for

Management Information Systems,” Sloan Management

Review XIII, 1 (1971) 55-70. W.R. Ring and D.I. Cleland, “The Design of Manage-

ment Information Systems: Information Analysis Ap-

proach,” Management Science XXII, 3 (1975) 286-297.

H. Mintzberg, Impedients to the Use of Management Infor-

mation (National Association of Accountants, New York,

1975).

T.F. Mock, “A Longitudinal Study of Some Information

Structure Alternatives,” Data Base V, 2, 3, 4 (1973) 40-49.

M.C. Munro and G.B. Davis, *‘Determining Management

Information Needs: A Comparison of Methods,” MIS

Quarterly I, 2 (1977) 55-67.

72 Kewurch

[ 1 l] R.F. Neuschel. “Improving Management lnformatlon Sys-

terns,” The McKinsey Quurtw!v (1976) 46-57.

[12] S. Neumann and E. Segev. “A Case Study of User Evalua-

tion of Information Characteristics for Systems Improve-

merits.“” Informurion md Management, 2 (1979), pp.

271-278, North-Holland Publishing Company.

[13] H.N. Nie, C.H. Hull, J.F. Jenkins. K. Steinbrenner and

D.H. Bentl. Stcrtisrazl Puckage for the Sociul Screnca,

McGraw-Hill, New York, pp. 468-514, 1975.