mis2009_2_2

8/12/2019 MIS2009_2_2

1/7

8/12/2019 MIS2009_2_2

2/7

Gordana Platia , Neo Balaban

12 Management Information Systems2/2009

information, (d) users level of training for work with Information System, (e) quality of outputinformation, (f) quality of available documentationand (g) users level of dependence on InformationSystem.

2. Information system functionalityperformances evaluation

During last decades, a large number of authorshave addressed the issue of Information Systemperformance evaluation. Most of complexes andclassifications of criteria for evaluation ofInformation System functionality andsuccessfulness in organization are defined. In thisfield of interest, several studies stand out, such asDeLone and McLean, proposed by Bailey andPearson, referred by Avison and Fitzgerald, Burchand Grudnitski. Evaluation criteria of InformationSystem functionality performances referring to theusers satisfaction with Information System in theframe of TQM concept (Total QualityManagement) are considered as InformationSystem top validation.

Bailey and Pearson (1983, pp. 519-529) havedefined 39 important factors which were used forassessment of users satisfaction with InformationSystem. Moreover, for each factor they set acriterion for its measuring. According to these

authors, factors that influence users satisfaction with Information System are: response time,accessibility, characteristics of used computerlanguage, realization of users demands, correctionof mistakes, model and data safety, systemdocumentations and procedures, system flexibilityand system compatibility; accuracy of output,promptness of output, output precision, outputliability, outputs due date, completeness andoutput format, output capacity, top managementparticipation, payment method for expenses ofservices, users trust in the system and userparticipation, users expectations from support ofcomputer based system, business effects from thesupport of computer based system and observationbenefits, technical abilities of CBIS employees(Computer-Based Information System), position ofCBIS employees, schedule of CBIS products andservices, necessary time for information systemdepartment to fulfill demands, processing periodfor demands of system alteration, support providedby the salesman, methods and means input/output

with CBIS center, users understanding of the

system and provision of user training, compatibilitybetween CBIS and other departments, priorities inresources distribution in CBIS, relations between

users and CBIS employees, communicationbetween users and CBIS employees, personnelcontrol over CBIS and organizational position ofthe CBIS unit.

De Lone and Mc Lean (1992) attempted anevaluation of Information System successfulness,

and proposed 180 traits of Information System, which they classified in six main categories: System quality: measurements of

Information System itself; Information quality: measurement of

Information System output; Information manipulation: recipients

handling of Information System output; User Satisfaction: recipients response to

handling the Information System output; Individual influence: effects of information

on recipients behavior; Organizational influence: effects ofinformation on organizational performances.

However, it has been noticed that these sixcategories refer only to systematic aspects ofInformation system successfulness, whileoverlooking the human aspects. This oversight canbe solved with factors proposed by Bailey andPearson, which cover both of these aspects. Theycombine the first five dimensions suggested byDeLone and McLean, leaving out organizational

influence. Additionally, they include human aspectin Information System successfulness, such asInformation System quality of service and conflictresolution between the users and Information

Technology professionals. Information Systemquality of service includes the following: improvingusers system knowledge, role and competencies ofInformation Technology professionals andefficiency of services. Resolution of conflictincludes: competition between the users andInformation Technology professionals for theorganizations resources, assignment ofinformation resources to users, communicationand relation between users and Information

Technology professionals, control of Information Technology professionals, and the organizationalposition of the Information TechnologyDepartment.

Li (Li & Cheung, 1987, pp. 15-28) argues thatBailey and Pearsons list of factors presenting 39criteria for Information System quality, are in line

with identified dimensions of DeLone and McLean(1992). The flaw is that they have left out thedimension of organizational influence, which isdue to a fact that users of all levels are more

8/12/2019 MIS2009_2_2

3/7

Methodological Approaches to Evaluation of Information Systems Functionality Performances and Importance ofSuccessfulness Factors Analysis

Management Information Systems2/2009 13

concerned with the influence of InformationSystem on personal performances, rather than tothe performances of the organization as a whole.

Therefore, they propose to add performances thatshould correct this overlook to the InformationSystem performances influencing the quality of

organizational functionality. Re-done methodlooks as follows:

1. Information System Performances: responsetime, accessibility, characteristics of usedcomputer language, realization of usersdemands, correction of mistakes, model anddata safety, system documentation andprocedures, system flexibility and systemcompatibility;

2. Information performances: accuracy ofoutput, promptness of output, precision of

output, reliability of output, arrival ofoutput, output completions and outputformat;

3. Information manipulation: output capacity;4. User satisfaction: top management

participation, payment method for expensesof service, users trust in the system and userparticipation;

5. Individual influence: users expectationsfrom support of computer based system,business effects from the support ofcomputer based system observation ofbenefits;

6. Performances of service: technical abilitiesof CBIS (Computer-Based InformationSystem) employees, position of CBISemployees, schedule of CBIS products andservices, necessary time of informationsystem department for fulfillment ofdemands, processing time for alteration ofsystem demands, support provided by thesalesman, methods and means ofinput/output with CBIS center, users

understanding of the system and providingtraining for users;7. Conflict resolution: competencies between

CBIS and other departments, priorities inresources distribution of CBIS, relationsbetween users and CBIS employees,communication between users and CBISemployees, personnel control over CBIS andorganizational position of the CBIS unit.

Therefore, seven more factors for evaluatingInformation System successfulness have been

added to the 39 factors defined by Bailey andPearson. These seven factors are: users attitudetoward usage of CBIS (follows into group of

conflict resolution), clarity of output,transparency of output (goes into group ofinformation quality), productivity of toolssupporting organizational structures (follows intogroup of user satisfaction), improvement ofproductivity thanks to CBIS, efficiency and

effectiveness of system (these factors go under thegroup organizational influences). The number offactors for measuring successfulness ofinformation system has increased to 46, whichcover all eight categories.

Avison and Fitzgerald (1995) state the followingcriteria for the assessment of information systemquality: acceptability, availability, cohesiveness,compatibility, documentation, learning simplicity,economization, efficiency, development speed,flexibility, functionality, implementation ability, lowjuncture, maintaining easiness, portability,reliability, sizeable, safety, simplicity,confidentiality, promptness and visibility.

Finally, we would like to point out one moremethodological solution. Starting from previousapproaches and criterion classification forinformation system functionality performancesevaluation, a group of domestic authors havesuggested their own method and classification,presented in table 1 (Balaban, Risti, urkovi,

Trnini, & Tumbas, 2006, p. 469) . This methodmaneuvers with 22 criteria (traits) of the

information system which are measured with afive-degree numeric scale of Likert type.

Table 1 Criteria for evaluating the performance functionalityof information system

8/12/2019 MIS2009_2_2

4/7



3. Importance of information systemsuccessfulness factors

Regardless of the number of criteria stated, allauthors agreed that requests for maximization of allnamed criterions would be without ground andthat the relative meaning of individual criteriadepends on specific situation, in the sense ofidentifying relative meaning of information systemsuccessfulness factors, in any described method.Pearson (1977) has conducted research in practiceand determined that, for groups of interest in theorganization the most important factors are thosepresented in table 2.

Ta ble 2 The most important factors for performances(successfulness) of IS

With detailed result analysis of this empiricalresearch, it can be noticed that five most importantfactors of information system successfulnessmentioned by the information system managers aremostly consistent with those mentioned by usermanager. However, there are two exceptions:Factors of relations and communication betweenusers and information technology specialists havebeen rated as more important than the factor ofunderstanding of users needs, while the usermanagers have rated the opposite. Factor

providing training for users has been pointed bythe user managers as less important than the factorof output size and the factor of business effectsfrom computer support, while information systemmanagers are pointing out the opposite. Attentionshould especially be devoted to the fact that thefactor of relation between user and informationsystem specialist has been rated as the third mostimportant factor of successfulness by theinformation system specialists, and as irrelevant bythe staff of information systems and users, whichpoints to the possibility of a problem is in theassessment process of information systemsuccessfulness between the information system

manager and other users connected to theinformation system.

Analysis of the factor of dicrepancies in theinformation system successfulness between ITspecialists and users, in sense mentioned above,leads toward several conclusions:

First of all, it has been shown that sevennew factors of information systemsuccessfulness are very important andshould be included in the instrument formeasuring the level of information systemsuccess. Those factors, together with 39factors listed by Bailey and Pearson, coverall 8 dimensions of information systemperformances.

Second, there are no important differencesbetween grading importance by informationsystem managers and informationtechnologies specialists. The sameconclusion can be drawn from gradingimportance by user managers and users.

Also, it has been shown that there aresignificant discrepancies betweeninformation system manager and usermanager, but there are significant differencesin importance ratings of informationtechnology specialists and informationsystem users.

Third, five most important factors of

information system successfulness pointedout by information system managers arealmost identical with the ranking of the mostimportant factors of information systemsuccessfulness according to informationtechnologies specialists, users and usermanagers. The main difference lays in thefactor of capabilities of the top management.Users see it as the most important factor forinformation system successfulness, whilemanagers think of it as moderatelyimportant. Consequently, top managementmust show enthusiasm and support toinformation technologies specialists, howthey would use possibilities of informationsystem for their managerial process. Onlytrough that effort can the user beencouraged to use more functions ofinformation system. This could, in return,increase the application of informationsystem in their everyday work and improvepossibilities for success of informationsystem.

Fourth, both user managers and informationsystem managers think that chargeback

8/12/2019 MIS2009_2_2

5/7



method and competition betweeninformation technologies specialists andusers are two least important factors.

Although it is obvious that these two factorsare irrelevant for the level of informationsystem success, they are irreplaceable

because excessive chargeback caninfluence users satisfaction with informationsystem and turn away users frominformation system services. Moreover,unfair competition can influenceorganizational equality in resourcesdistribution and hence jeopardize relationsand communication between users andinformation technology specialists.

Fifth, users and information technologiesspecialists as groups have different opinionsabout ratings on several factors ofinformation system successfulness. On theone hand, users and their managers assignmore importance to output results (services)of information system such as: noticedusefulness, form of output, business effectsof computer support, than informationtechnology specialists. On the other hand, ithas been shown that managers andspecialists for information technologies aremore concerned with the environment ofinformation system services (such as :

relations between users and informationtechnologies specialists and processing ofsystem alterations demands) than are users.In order for information system to besuccessful, the management of informationsystem should understand and manage thesedifferences. Also, they should pay moreattention to ends of services rather than themiddle of the information system during theplanning of resources and formulation ofstrategies for development and successfulfunctioning of information system.

Ratings of noticed importance individual factorsfor information system successfulness vary fromperson to person because everyone has a differentlevel of measurable standards and level of personalimportance. The process of ranking is a way toeliminate such individual differences. It generatesorder which presents relative levels of importance.Information system management should use onlythe order by importance of successfulness factorsof information system to identify relative meaningof one factor among others. It is shown that theorder of importance gathered from users isdifferent from the one gathered from information

technologies specialists. Such differencesemphasize that undertaken process for informationsystem evaluation, partially differentiates betweeninformation technology specialists (especiallyinformation system managers) and users.Information system management should

periodically conduct a survey of the importance ofinformation system successfulness factors noticedby information technologies specialists and users,in order to overcome the differences in importanceorder between these two groups.

The process of rating information systemsuccessfulness in an organization means gatheringgrades with questionnaires on the importance andsatisfaction of 46 information systemsuccessfulness factors from each and all functionalareas, from users as well as from informationtechnologies specialists. Questionnaires shouldconsist of two separate parts: one for grades ofimportance and other for grades of satisfaction.

The first one gives the opportunity to informationsystem management to assign priority correctiveactions and to distribute resources of informationsystem. For example, if one assumes that there aretwo unsatisfied factors of information systemsuccessfulness, successfulness IS-1 andsuccessfulness IS-2, and that successfulness of IS-1is ranked as more important than successfulnessIS-2 (under the assumption that other factors are

equal), first first it requires undertaking action anddirecting resources of the information systemtowards increasing the satisfaction level ofsuccessfulness factor IS-1, and then successfulnessIS-2. Similarly, if we have two factors ofinformation system successfulness with the sameimportance and negative satisfaction, the factor

with less satisfaction must be prioritized andreceive more resources. The second gives toevaluator of information system successfulness theopportunity to identify individual unsatisfiedfactors of information system successfulness andto inform information system management.

In order to receive truthful answers, appraisalsshould be conducted by an independent evaluatorand the participants must stay anonymous, exceptfor the identity of their functional areas. Knowingthe functional area of every participant,information system management can easily evaluatethe level of dissatisfaction within any individualfunctional information system and in line with thattake direct action toward solution. Moreover,participants should know as little as possible in

which manner information system managementuses assessments of information system

8/12/2019 MIS2009_2_2

6/7



successfulness in order to manage their projectsand resources, otherwise participants could start tomanipulate the evaluation of information systemsuccessfulness in order to withdraw indirectattention of information system management tothemselves. In order to avoid such events, they

should determine a detail check and verification ofunsatisfactory factors of information systemsuccessfulness.

Assessments of information systemsuccessfulness should be gathered periodicallyfrom each and all functional areas in organization.In order to get final grade on information systemfunctionality performances in organization,gathered results must be processed statistically. Forthe analysis of gathered data, the followingstatistical procedures are commonly used:arithmetic mean, standard deviation for each point,Interco relation of points, multiple regression ofpoints, liable scale, variance analysis, factor analysis,cluster analysis of variables (Word method),components analysis, method of multidimensionalscaling (squared Euclid discrepancies ), and others.Such information would provide the informationsystem management with the opportunity tooversee the total information system quality (acrossthe whole organization), so as to compare qualitiesbetween the different functional areas ofinformation system and to review improvement

process of functional or integral informationsystem. Furthermore, the managent of informationsystem is provided with a/the pattern of constantchanges for understanding information systemsuccess inside each functional area. Suchinformation is vital for information systemmanagement in shaping the future of informationsystem in organization.

Additionally, the application of instruments formeasuring successfulness and functionality ofinformation system in an organization has severalimplications for managerial process anddevelopment of information system. Every newinformation system manager must be trained tointerpret the assessments of information systemsuccessfulness, which he receives trough periodicalsurvey of all users, and also know how to developstrategies for solving problems revealed by users.Information system management must be includedin those marks, to the level on which every one ofthem can use grades of user satisfaction foridentifying areas of some problems, as well as toanalyze grades of importance in order to assign

priorities to activities of development ormaintenance of information system.

Finally, performances that have influence onthe success or failure of information system areoften argued in literature. They can be classified ineight different dimensions so they include not just

the system aspect but also the human aspect ofinformation system successfulness. Opinionanalysis of users, information technology specialistsand managers reveals that there are no significantdifferences in order and importance of informationsystem successfulness factors between informationtechnology specialists and information systemmanagers, as well as between users and usersmanagers. However, the order of factors byimportance is significantly different betweeninformation technology specialists and users.

Therefore, the most important information systemsuccessfulness factors are identified.

References

Avison, D. E., & Fitzgerald, G. (1995).Information Systems Development:Methodologies, Techniques and Tools. London: McGraw-Hill.Bailey, J. E., & Pearson, S. W. (1983). Development Of A Tool ForMeasuring And Analyzing Computer User Satisfaction.ManagementScience , 29 (5), 530545.Balaban, N., & Risti, . (2006).Business intelligence. Subotica: Universityof Economics Subotica.Balaban, N., Risti, ., urkovi, J., Trnini, J., & Tumbas, P. (2006).Information technologies and Information Systems. Subotica: University ofEconomics Subotica.Burtch, J., & Grudnitski, G. (1989).Information Systems: Theory andPractice. John Wiley & Sons: New York. De Lone, W. H., & McLean, E. R. (1992). Information Systems Success,The Quest for the Dependent Variable.Information Systems Research , 3 (1), 60-95.Harrison, W., Magel, K., Kluczny, R., & DeKock, A. (1982). ApplyingSoftware Complexity Metrics to Program Maintenance.Computer , 15 (9),65-79.Li, E. Y. (1997). Perceived Importance of Information Systems SuccessFactors: A Meta Analysis of Group Differences.Information andManagement , 32 (1), 15-28.Li, H. F., & Cheung, W. K. (1987). An Empirical Study of Software Metrics.

IEEE Transactions on Software. Engineering , 13 (6), 697-708.Parker, C., &Case, T. (1993). Management Information Systems, Strategyand Action. New York: Mitchell McGraw-Hill.Pearson, S. W. (1977). Measurement of computer user satisfaction.Unpublished Dissertation, Arizona State University, Tempe. Platia, G., & Tumbas, P. (2000). Some aspects of managing control ofsoftware products. Annals of the Faculty of Economics Subotica , 5 , 671-677.

8/12/2019 MIS2009_2_2

7/7



Gordana PlatiaUniversity of Novi SadFaculty of Economics SuboticaSegedinski put 9-11

24000 SuboticaSerbiaEmail: [email protected]

Neo Balaban University of Novi SadFaculty of Economics SuboticaSegedinski put 9-11

24000 SuboticaSerbiaEmail: [email protected]

mis2009_2_2

Documents