Journal of Research of the National Bureau of StandaidsVolume 90, Number 4, July-August 1985

Metrics and Techniquesto Measure

Microcomputer Productivity

Wilma M. Osborne and Lynne Rosenthal

National Bureau of Standards, Gaithersburg, MD 20899

Accepted: July 9, 1985

While it is generally assumed that the use of microcomputers helps to improve productivity in an officeenvironment, quantitative measures in this area are lacking. This paper addresses the measurement of theeffect on productivity in an end user, office environment as a result of the introduction of micromputer-basedtechnology. It is concerned with defining how productivity can be measured in such an environment and withcurrent efforts to measure changes in productivity. It identifies and assesses the various techniques andmeasures used to describe the magnitude of productivity improvements that result from the use of micro-computers in the workplace, and makes recommendations for ways in which changes in productivity, may bemeasured.

Key words: added-value; automation; efficiency, measure; methodology; metrics; microcomputers; officeenviornment; productivity; qualitative measurement; quantitative measurement.

Introduction

The role of information processing has changed withthe introduction of the microcomputer into the work-place. The microcomputer has become a "tool" thatoften enables the user to directly control informationprocessing needs without the assistance of the profes-sional ADP staff. The relative low cost of micro-computers, their adaptability to various applications,and the availability of software which is useful to non-computer professionals has resulted in the proliferationand use of microcomputers in the office environment.

While there is general agreement that micro-computers do increase office productivity, there is littlehard, quantifiable data to actually support this claim.

Many companies have published reports of substantialimprovements to productivity, but few have attemptedto actually measure the changes, and fewer still haveachieved any reliable measurements. By and large, thepublished reports have drawn the conclusion of pro -ductivity gains based on the perception of the managersand workers in the affected office environments.

The use of microcomputers by professional and cler-ical staffs overlap but are functionally different. Theclerical staff primarily utilizes word processing whichcan be measured at the individual level (e.g., number ofwords in within a specific period). It appears, however,that the primary benefits to the professional staff are notin speeding up the information flow, but in improvingthe depth of analysis and understanding of the availableinformation. Therefore, professional staff productivityshould be measured in terms of the information handledand processed. Through spread sheet analysis and ac-counting and financial systems, the professional can bet-ter understand the significance of the information and

305

About the Authors: Wilma M. Osborne andLynne Rosenthal are with NBS' Institute for Com-puter Sciences and Technology.

thus, can make more informed decisions which result inimproved performance of the entire organization.

This paper addresses the measurement of the effect onproductivity in an end user, office environment as aresult of the introduction of microcomputer-based tech-nology. It analyzes and assesses the various mea-surement techniques presented and makes recommen-dations on how to measure productivity gains in thefunctional workforce. Measuring productivity changesin an office environment primarily involves the assess-ment of the impact of new technology on qualitativefactors which cannot be measured directly. Thus, theinability to assign a numerical value to a factor such as"efficiency" limits the measurement and makes it highlysubjective. In general, if the product or service is com-pleted faster; if the quality and performance is im-proved; if there are noticeable, desirable differences inthe process; then there are productivity gains for theorganization.

Productivity is often equated with efficiency (reduc-ing unit costs, improving output per hour, reducing er-rors, etc.) or quantity of work performed. Quality ofoutput may, in fact, be far more important than quantitywhen measuring and assessing traditional white collarworker performance since accurate communication ofinformation is the primary function of such workers.The usefulness and effectiveness of this output is deter-mined by such qualitative characteristics as "effi-ciency," "completeness," and "effectiveness." Al-though these attributes are discernible to thoseinvolved, they are not directly measurable and thus, anassessment of changes in productivity is much moredifficult to make. Productivity measurement can befacilitated if the input properties and the output proper-ties are clearly defined. This is more difficult to do in awhite collar environment since most managerial andprofessional work does not have well-defined, mea-surable inputs or well-defined outputs. Thus, the use-fulness of any productivity measurement is dependentupon the accuracy of the perceptions of the qualitativefactors which are used.

The introduction and use of microcomputers haveincreased the productivity of users, but measuring theseincreases has not been easy. Much of the difficulty is dueto the fact that techniques for measuring productivityare neither well-known nor well-defined. Further, thebenefits do not always appear as single, discernible enti-ties, rather they are often small gains across many tasks.As a result, few organizations have initiated productiv-ity measurements citing the lack of money, time, knowl-edge, or an inclination to perform a comprehensive,scientific study. The situation is further complicated bythe lack of information needed to measure productivity.

One of the perennial problems is the difficulty in mea-suring the productivity of value-added activities. In-evi tably when a manual system is automated, many newfunctions and products are generated, usually withoutadditional resources and at negligible cost. These intan-

gible byproducts of automation,which did not exist pre-viously, can enhance a manager's decision making abil-ity. Another problem in measuring productivity gains isthe omission of some of the cost of converting frommanual to automated methods, and a comparison ofthese costs against the benefits. Some of the costs typi-cally omitted are site preparation, training, and ongoingsoftware maintenance. Quantifying professional or man-agerial productivity has proven to be difficult, particu-larly for such activities as: preparation for and attendingmeetings, reading and writing reports, responding totelephone calls, etc.

Prior to implementing a productivity measurementprogram, there are a number of factors to be considered.While the methodology (presented in table 5) is recom-mended for measuring changes in productivity, it maynot be feasible depending on organizational constraints.These constraints include:

* the unavailability of measurement data/information. It is difficult to measure changes in pro-ductivity in the absence of baseline data. If the informa-tion needed to conduct such a study is incomplete orunavailable it may be better to rely on the judgement ofthe users of the microcomputers.

* the lack of well-defined productivity measurementtechniques. Measures of productivity that are appropri-ate for the different levels and functions in the or-ganization must be well-defined.

* the cost, time, and effort to effectively measureproductivity.

* the lack of a means to capture much of what isdone in an office environment.

* the size of the organization or project, and thetype of applications. If the environment is small, the costof such a study could outweigh the benefits.

* the lack of (active) participation in the pro-ductivity measurement program by all impacted by theintroduction of microcomputer.

Once the decision has been made to measure changesin productivity, it is often desirable to implement a pro -ductivity improvement program to maximize pro-ductivity gains resulting from microcomputer use.

Quite simply, organizations are as concerned withwhat a productivity program is going to cost as withwhat it is going to return. Knowing how to spend fundswhere the most benefit can be realized, however, issometimes difficult. Before productivity can be effec-

306

tively measured, there must be a thorough under-standing of what is being done, the activities performed,for whom, and why. While the old maxim, "you can'timprove what you can't measure" is not entirely true, itis important to develop a strategy for defining what tomeasure, when to measure, and how to use the mea-surement data.

As the activities performed in the functional work-place become more complex and technical in nature,and ready access to reliable information becomes morecritical, it is essential to make better use of each individ-ual's time. The effectiveness of any productivity mea-sure is how accurately it reflects what is taking placewith respect to the ability of the individual, and ulti-mately the organization, to perform tasks easier, faster,and better.

Measuring Changes in Productivity

Techniques for measuring changes in productivityrely on two types of measurements: quantitative andqualitative. Quantitative techniques generally measurequantities of work over some unit of time such as "piecesper hour," "person-hours per completed product," or"defects per unit of time." These types of measurementsare fairly easy to quantify and are typical of mea-surements made when investigating productivitychanges in a production, blue-collar environment. Qua-litative techniques are those which address less tangibleattributes such as "quality," "effectiveness," and "effi-ciency." Measurement of such attributes is very difficultbecause it is highly subjective.

Assessing productivity and changes resulting fromthe introduction of microcomputers primarily requiresthe measuring or estimating of qualitative attributesrather than quantitative factors. While it is not possibleto obtain highly definitive qualitative measurements, it ispossible to assess the relative changes in productivitythrough a careful and consistent assessment of selectedqualitative attributes before and after the introduction ofthe new technology.

Measuring changes in productivity can be done at theglobal (organization), local (functional unit), or individ-ual level. Measurement of quantitative factors can besuccessfully performed at the lowest individual levelssince exact counts can be obtained for specific factors(number of letters typed, hours worked, number offorms processed, etc.). Assessment of qualitative attri-butes, however, is more reliable at a higher level withinan organization since the individual variances and in-consistencies will tend to balance out.

Based on the literature search which was conductedas part of this task, there is a great deal of interest inproductivity measurement in general and the effects ofmicrocomputers on productivity in particular. This lite-rature search included more than 200 sources, of which47 appear at the end of this paper as a selected bibli-ography. Many organizations have initiated efforts todetermine the effects of microcomputers on individualand organizational productivity. However, there is adearth of information on any actual quantifiablechanges. These reports consistently discuss the diffi-culties in performing such measurement and usuallyconclude that there has been improvement in pro-ductivity on a global level as a result of the introductionof microcomputers.

Some of the techniques discussed later in this sectionhave been proposed to provide quantifiable mea-surements. Organizations have been reluctant, however,to embark on the effort required to make such mea-surements citing the cost, lack of time, lack of informa-tion, and lack of proof that these techniques will actuallyprovide useable results. Thus, the consensus is that mi-crocomputers have improved productivity, but there islittle actual documentation to support that conclusion.

What Should be Measured

Productivity measurement, particularly of qualitativeitems, can be a costly endeavor. In a small organizationwith limited resources, it may be unreasonable to under-take a comprehensive study either to determine pro -ductivity gains or to determine how achieve to them.Regardless of the size of the organization, it may bedetermined after a careful evaluation of organizationgoals, objectives, and requirements that the difficulty ofimplementing a productivity measurement program justwould not be cost effective. Another instance in whicha program to measure changes in productivity may beunnecessary is when it is obvious that the use of micro-computers has resulted in improved productivity.

The attributes and factors to be measured must beselected and carefully defined. Since each organizationis unique, the attributes and factors selected for mea-surement will differ. After the technology has been in-troduced, sufficient time should transpire before con-ducting the second productivity measurement. This isimportant because there may be a short-term decline inproductivity which occurs while the users are learningto use the new technology properly. The baseline andnew productivity levels should then be carefully evalu-ated and an assessment made on the relative pro-ductivity changes which have been realized.

307

Factors that complicate measurement. Several factorsfrequently complicate the measurements:

* Value-added activities may not be adequatelymeasured. When a manual system is automated, manynew functions and products may be generated withoutrequiring additional resources or cost. These activities,while difficult to measure, should be taken into consid-eration.

* A number of cost factors are frequently omittedwhen measuring productivity gains. Included are thosecosts associated with the conversion from manual toautomated procedures such as: site preparation, training,and maintenance of both the hardware and the softwaresystems

* The introduction of new technology frequentlyresults in substantial changes to the office environment.Personnel become responsible for different or additionalparts of the process, duties shift, and some work may beeliminated while new work is created. Thus, attempts tomeasure individual productivity changes is often a caseof comparing apples and oranges. For this reason, theassessment should be made at the organizational levelrather than the individual level.

Factors and attributes for measuring productivity. Thefollowing three tables list some of the more commonlyused factors and attributes used in evaluating pro-ductivity.Table I identifies tangible, quantitative factorswhich can be measured directly. Table 2 and table 3

Table 1. Factors that can be measured and quantified to detennineproductivity gains.

- workload- schedules- cost/budget- end products- training cost- size of staff- methods/techniques- response/turnaround time- time to perfonm a specific task- number of new requests/alternatives examined- outputs before and after using microcomputers- amount of data handled, sorted, and calculated

Table 2. Attnbutes which can be measured but not easily quamtifiedto assess productivity gains.

- accuracy- efficiency- reliability- completeness- user acceptance- data accessibility- value added capabilities- improved analysis (budget, trends, etc.)- timeliness of reports/tickler files/information

identify less tangible, more subjective, qualitative attri-butes which should be assessed in attempting to evaluatethe productivity of an organization. Section three pro-vides a discussion of several case studies which make useof many of the factors and attributes identified in thetables below.

The productivity improvements most frequentlymentioned as a result of using microcomputers are in-creased workload, new or more work accomplished in ashorter time, and cost saving. Improved accuracy, effi-ciency, quality, attitude and morale are also cited asbenefits of microcomputer use. Not all of factors andattributes identified may be appropriate to a specificsituation, while others not found in these tables maybe critical in specific environments.

Criteria to measure desired outcomes. The ultimateobjective of introducing microcomputers is to increasethe productivity of an organization. This may be accom-plished by reducing costs, avoiding increases in costs,increasing value added activities/products, increasingemployee satisfaction or becoming more competitive.Our findings indicate that the identification of desiredoutcomes and the selection of appropriate measurementcriteria are essential to the success of any productivitymeasurement program. The desired outcome, more thanany other factor, influences the choice of criteria formeasuring the outcome and determining if goals havebeen achieved. Table 4 identifies some possible criteriafor different objectives and goals.

Measurement Techniques

Measurable improvements in productivity generallycan be attributed to a combination of human resourceand technological factors. Therefore, any effort to de-termine changes in productivity as a result of the intro-duction of microcomputers should consider both thework environment (equipment and tools) and the em-ployee effectiveness (training, education, attitude).

There are few effective measurement techniquesavailable for measuring productivity in the office envi-

Table3. Attributes which are not easily measured nor quantifiable todetermine productivity gains.

- control- flexibility- communication- attitude and morale- quality of decisions- new insights and learning- better understanding of business- effectiveness (of team work, etc.)- quality of presentations (graphic displays, etc)

308

Table 4. Criteria to measure desired outcomes for OA projects.

Desired Outcome Possible Criteria

To increase organizational Total output in number of unitsproductivity produced as a function of

labor, investment, etc. measuredin dollars

To reduce or avoid costs Cost of labor, materials, andoverhead

To increase value-added Contribution to profits fromwith products/services improved products/services

To increase managerial Time required to complete tasksproductivity and level of individual, unit, and

organization productivity

To increase timelines Average and variance of time toof information prepare/distribute information

To increase quality of Quality, accuracy. and corninformation pleteness of information

used to generate products

To provide more job Turnover or absenteeismsatisfaction

ronment. The qualitative measurements which are mostuseful in assessing changes in the office productivity aremost difficult to obtain. The quantitative measuringtechniques can be characterized as a comparison ofINPUT/OUTPUT before and after microcomputersare used. Although the INPUT is generally well -de-

fined, the difficult aspect in performing this type of mea-surement in the functional workplace is quantifying theOUTPUT.

The most commonly used techniques for measuringchanges in productivity are Questionnaires and Empir-ical Analysis (see table 5). The primary difference be-tween these two approaches is that the latter relies moreon intuitive knowledge and information gained throughexperience and less on a systematic, structured meth-odology. Both of these techniques make use of beforeand after information concerning the qualitative andquantitative aspects of the process and the products;both are relatively easy to employ; both can be admnnis-

Table 5. Productivity measurement techniques.

Questionnaire/Survey Empirical An] ysis

- Before and after' - Before and aftermeasurement measurement

- Assessment of need - Intuitive- Quantitative metrics - Quantitative metrics- Qualitative measures - Qualitative measures- Methodology/formula

IfTe establishment of a baseline level of productivity prior to the intro-duction of the new tecJnology is essential to the success of a "before and aftr&

sment.

tered formally or informally; and both can be used foralmost any size and type of environment/organization.

Questionnaire technique. The questionnaire/surveymethod of assessing changes in productivity resultingfrom the introduction of microcomputers appears to beone of the most useful. One of the advantages providedby the technique is that it can readily be adapted tomeasure global, as well as localized changes in pro -

ductivity. The questionnaire canbe used to gather infor-mation about characteristics and functions of the or-ganization; projected requirements and desired features;work process, equipment, and products; and profile dataon individual performance. It can also be used to isolateproblem areas, determine employee attitudes, and solicitsuggestions.

Another advantage provided by the questionnaire/survey method is that it can be used to obtain informa-tion from multiple levels of the organization. This ap-proach makes it possible to obtain information aboutperceived qualitative, as well as quantitative changes inproductivity. This is essential since the perceptions ofthe changes may differ. In fact, everyone who eitheruses microcomputers, or is impacted by changes in theenvironment as a result of their use, should be surveyed.Properly employed, it is one of the most effective tech-niques for obtaining, comparing, evaluating, and mea-suring changes in productivity within the organization.

Empirical analysis measuring technique. Empiricalanalysis is employed to measure productivity before andafter the introduction of microcomputers in the work-place. Empirical analysis relies primarily on experienceand observations about a particular area or environ-ment, and does not generally make use of systematicmethods or methodologies. It may be performed byweighing and measuring the applicable, quantitative ortangible factors such as those identified in table 1. Theweights are generally assigned to the variables to bemeasured based on the value and function of that vari-able within the specific environment. Qualitative factorsmay also be taken into consideration. Using empiricalanalysis, a baseline is established against which allchanges are measured.

Recommendations on measurement techniques. When us-mg either the questionnaire or the empirical analysistechnique to measure productivity, it is important tofocus on the extent of productivity changes at the globallevel, as opposed to the short term or incremental in-creases at the local level. The introduction of any newtechnology involves a learning period during whichthere may actually be a decline in productivity. Con-sequently, sufficient time must be allowed after micro-computers are introduced into the organization to per-

309

mit a stable state of operation to be established beforethe productivity impact is assessed. The usefulness ofthese techniques is dependent upon the accuracy andcompleteness of the information collected, and the skilland knowledge of the surveyor or evaluator.

Table 6 contains a set of recommended steps whichshould be followed in measuring productivity changesin an office environment.

Methodology for Measuring Productivity

There are a number of approaches for determining amethodology for measuring productivity changes. Oneapproach is to evaluate attempts by other organizationsto measure productivity, while another is to determinethe estimated cost of such a program, and then comparethis cost with the expected gains in productivity.Whether or not either of these methods are employed, itis essential to determine the goals or desired outcomes ofa productivity measurement program for the specificoffice environment. See table 4 for some desired out-come measurement criteria.

Preparing to measure change. The first step in pre-paring to measure changes in productivity after definingthe goals is to determine the feasibility of such a pro-gram. This requires that those involved have a thoroughunderstanding of what is meant by productivity, of theactivities to be measured, and of the areas likely to ben-efit most from the application of productivity measures.Since productivity has different meanings in differentenvironments, it is essential to establish a definition thatis suitable for the environment that is to be measured.

Each environment is unique and the attributes andfactors useful in one may not be appropriate for another.This is especially true in the case of qualitative, subjec-tive attributes such as reliability, accessibility, and effi-ciency. Therefore, it is necessary to define not only a setof factors and attributes (productivity indicators), but a

method for quantifying the qualitative attributes. It isalso essential that changes in productivity be measuredat the global or organizational level, as well as the localor individual level to ensure the least amount of bias. Ifthere is little or no assurance that a program to measurechanges in productivity can be justified from a budget-ary standpoint, the costs are likely to outweigh thegains. Finally, if it is determined that the implementationof such a program is not feasible, then planning shouldcease.

Performing Measurements. Once the decision has beenmade to proceed, it is essential to establish and measurethe baselined activities before and after microcomputersare introduced. After a sufficient time has elapsed, thereshould be a careful assessment and evaluation of anychanges in productivity. This process should be re-peated over a period of time at unscheduled intervals atsufficiently high levels within the organization to pre-clude constraints and otherwise, unrepresentative con-clusions. Table 6 describes a methodology for both pre-paring for and performing productivity measurements.

Case Studies

As part of our study, we surveyed a number of or-ganizations to determine their experiences in measuringproductivity. We found that:

* Virtually every organization reported substantialgains in productivity.

* Most organizations and studies which reportedgains in productivity from the use of microcomputersbased that claim on a perceived improvement and thesubjective judgment of management

* Actual, quantitative measurement studies were ei-ther not conducted or did not yield quantifiable results.

Table 6. Steps in measuring changes in productivity.

A. Planning and Preparing to Measure ChangeI. Determine the feasibility of measuring productivity changes in the specific environment.2. Develop a definition of productivity for the specific organizational enviromment.3. Identify the attributes to be used in measuring productivity and define a method of quantifying the subjective qualitative attributes.4. Develop and implement productivity measures at the global or organizational level rather than at an individual level to ensure a

reliable assessment

B. Perfruming MeasurementsL. Perforn a measurement to establish the baseline productivity level.2. Introduce the new technology.3. Permit a sufficient period of time to elapse to allow any short term decreases in productivity to be eliminated.4. Re-measure the productivity levels.5. Carefully evaluate the results to ensure that the results are interpreted within the context of the organizational environment. The

evaluation should be performed at the highest, global level possible to avoid local aberrations and biases.

310

This section presents the findings of eight case studies.Five of these are based on reports which are listed at thebeginning of each case study, and three on informationobtained directly from the organization. Table 7 pro-vides a matrix which identities the factors and attributesreferenced in each of the following case studies.

While these eight case studies provide some insight onthe different approaches used to determine changes inproductivity, they demonstrate the difficulty in identi-fying and measuring subjective attributes. Each of thecase studies reference increased workload, new or morework accomplished in shorter time, and cost saving as aresult of using microcomputers. In many environments,microcomputers are credited with increasing pro-ductivity if schedules are met in a timely maimer, if theworkload is handled without additional staff, and if theproducts are acceptable and of high quality without thebenefit of any "formal" measurements. Table 7 lists fac-tors or indicators identified in the selected case studies:

As indicated, very little quantitative data is availableand most of the conclusions of improved productivity

Table 7. Case Study Matrix.

Firm Table I Factors Table 2/3 AttributesMea ured Assessed

L. NSRDC workloadstafftimeetc.

2. GAO workloadmanpowercostnurber hoursdata handle:.

3. GSA manpowercostworkloadet.

4. USS costsworkloadetc.

5. American Productivity CenterBethlehem Steel

Polaroid

6. BaitkingData Ir'ocmisngand Research

tUmeS. Brokerage Finn

costsworklksdcostworkods

cost

timelinessefficiencyquality

improved analysis

efficiency

qualicative benefitstimeliness, etc.

users/mgrs. best judgeof how much productivityis achieved (no detail)

improved moraletimelinessCoirnimo ationreliability(errors reduced)quality of lifeteamwork

valueadded capabilities

user acceptance

are based on intuitive belief, not on firm, scientific mea-surement,

1. Navl Ship Research And Development Center

Van Eseltine, R.T., "The Scientific/EngineeringWorkstation Experiment: Plans and Progress," Pro-ceedings of the 22nd Annual Technical Symposium ofthe Washington, DC, Chapter of the ACM, Co-sponsored by the National Bureau ofStandards,June 1983, (Questionnaire and Empirical Analysis)

As part of the Technical Office Automation andCommunication (TOFACS) project, the David W. Tay-lor Naval Ship Research and Development Center hasundertaken a study of the effects of individual scientific/engineering workstations (SEWs) on the productivity ofscientists and engineers. A prototype network of SEWswas developed to assess the changes in productivitywhich could result from the introduction of SEA's. Ini-tial results of this research indicate that workstations areviable tools which aid productivity in a scientific andengineering environment.

The technique used in this study was to have thesubjects being studied perform the evaluation and as-sessment of their before and after productivity levelsfor a brief description of the methodology employed. Arather complex formula was used which basically in-volved applying values to attributes before and after theintroduction of the workstations, and then calculatingthe change in the ratio of the Output to Input totals.Quantitative data was gathered by having the technicalstaff assess the changes in how long it took to completea typical task with and without the scientific and en-gineering workstation. Most of the other data was moresubjective (qualitative) in nature and less easy lo quan-tify. The use of the subjects to evaluate their changes inproductivity resulted in more consistent assessments onan individual basis, but may have also introduced biaseswhich could affect the findings. In general, this methodappeared to work satisfactorily in this R&D laboratoryenvironment and the general methodology could be ap-plicable to other similar measurement attempts.

2. General Accounting Office

"Workstation Project Report To information Pol-icy Committee," Directed by Kenneth Pollock, As-sociate Director of Information Management Sys-tems, General Accounting Office, Internal Study,1982. (Questionnaire)

The GAO initiated an electronic workstation projectto determine if the installation of workstations "could be

311

cost effective at GAO in performing various auditorfunctions." The auditing (workload) functions are de-scribed and activities are divided into categories for thepurpose of measurement. Benefits and problems are dis-cussed. A matrix of automatable and non-automatableactivities was defined and provided the basis for deter-mining how best to utilize the workstations.

This report discusses the study and the methods em-ployed and concludes thatan approximate25% increasein the capacity to perform audit functions was realizedas a result of the introduction of the electronic work-stations. The basic measurement unit utilized was thenumber of staff hours actually needed versus the esti-mated number of hours which would have been re-quired if the electronic workstation had not been intro-duced. The report contains little information on theactual collection and analysis of data.

3. General Services Administration

"Final Report on the GSA End User ComputerPilot Project," Prepared by The General ServiceAdministration's End User Computer SupportStaff (KGS-t), September 28, 1983. (EmpiricalAnalysis)

The General Services Administration conducted a pi-lot project to study the effects of the introduction ofmicrocomputers within GSA. The project involved 500GSA employees using 53 microcomputers and consistedof the automation of 175 applications. The report pro-vides information on the experiences during the projectand identifies the actions completed or initiated to facil-itate end user computing. The report summarizes boththe qualitative and quantitative benefits encountered bythe end users. Most improvements were a result of auto-mating manual operations. Specific examples of pro -ductivity gains are given in terms of cost savings, man-hours, and increased workloads/tasks.

In discussing the findings, the report notes that insome cases productivity increases were measured "interms of staff hours, or dollars," but in others they werenot quantifiable because "the microcomputers satisfiedrequirements that were previously postponed due tostaffing shortages." Direct changes in productivity weremeasured in terms of changes in staff hours or dollarswhenever possible.

However, most of the conclusions drawn in this re-port are based on qualitative estimates of perceived im-provement to the process. Nevertheless, there is a verystrong indication that "the use of microcomputers canpay for themselves in less than one year" and can "helpprovide better, more timely products and more in-depthanalysis."

4. United States Senate

"The Pilot Test of Office Automation Equipmentin the Offices of United States Senators," Commit-tee on Rules and Administration United States Sen-ate, S-Prt 98-120, November 1983. (Questionnaire,Empirical Analysis, and Before/After Measure-ment)

This pilot study focused on office automation in theU.S. Senate offices. The method used to measure pro -ductivity gains was a before and after analysis of thefunctional requirements and the day-to-day office work-load. Participants were asked to complete forms whichidentified the areas that could be improved most byautomation. They were also asked to rank each of theareas in terms of importance to the performance of theirresponsibilities. Guidelines were developed to ensurethat everyone recorded the same types of information inassessing changes as a result of automation. The keyaspect of the productivity measurement program wasthe requirement that productivity goals and cost justifi-cation be established for each workstation to be in-stalled.

The test demonstrated that the staff could quicklylearn to use the equipment and put it to productive useThe report does not present any detailed information onmeasuring productivity changes but does state that "inthe final analysis, the actual users and office managersare the best judges of how much improvement has beenachieved."

5. American Productivity Center

Steel, "White-Collar Productivity: The NationalChallenge and Case Studies," sponsored by Steel-case Inc., Grand Rapids, Michigan, 1982. (Ques-tionnaire and Empirical Analysis)

Steelcase Inc. commissioned the American Pro-ductivity Center, a nationally recognized expert on pro -ductivity issues, to conduct a study on productivity in theworkplace. During this six month study, the Pro-ductivity Center sent survey questionnaires to 600 U.S.firms and received 140 responses. The study is based onthose responses and includes 25 case studies selected onthe basis of the techniques used to assess and measureproductivity gains resulting from the introduction ofoffice automation. While this study does not identify anyunique measurement techniques, it does suggest that al-most any productivity improvement program, no matterhow unstructured, can result in increased productivity.

312

Word processing was found to be the most effectivefactor in improving office productivity. Other factorsconsistently cited were team building and the work en-vironment. The productivity measurement programs oftwo of the most representative case studies are describedand evaluated below:

a) Bethlehem Steel (Methodology/Intuitive). Beth-lehem Steel initiated a Productivity through Office Sys-tems (PROS) effort to improve productivity of the400 person sales force and their support personnel. Thethrust of PROS, which was aimed at the secretarialforce, involved the introduction of both an office auto-mation system and a "Quality of Work Life" (QWL)methodology. The QWL methodology program en-courages greater employee participation, and providesfor training in all aspects of an office that contribute tothe "quality of life" in the office. As part of this effort,monthly PROS meetings were held to address teambuilding, problem solving, and other issues which canstrengthen employee capabilities. No formal techniquesor measurements were undertaken either before or afterthe office automation system was installed. There is astrong perception, however, that there were substantialgains in productivity which could be attributed to bothoffice automation and to the QWL methodology pro-gram. The reported subjective estimates of the pro-ductivity improvements were:

* increased output* more timely delivery* credibility of offices* morale improved* task difficulty reduced* communication improved* space more effectively used* response time reduced* errors reduced* quality of service enhanced

20%80%20%20%20%50%25%80%

5%50%

b) Polaroid (Needs Assessment). Polaroid, spurredby a reduction of 4000 employees between 1978 and1982, established an Office Technology Council to de-termine how to "better manage and utilize emergingoffice technologies, reduce cost and enhance the effec-tiveness and productivity of personnel." As a first task,the Council developed and implemented a seven-step"Needs Assessment Methodology" to justify the acqui-sition of new technology. An examination was made ofthe personnel, workload, and tools needed within theorganization to accomplish its mission; data was gath-ered and evaluated; and a summary of the qualitativebenefits expected from the application of electronictechnology was made. Polaroid considers this type ofassessment helpful in identifying methods for improving

productivity prior to the introduction of new tech-nology. The seven steps of the methodology are:

a) Orientation (Overall mission, functions,equipment, costs)

b) Professional activity profilec) Administrative profiled) Administrative reportinge) Detailed workloadf) Word processing benefits summaryg) Financial Analysis worksheet

needs,

6. Banking (Empirical Analysis)

A large northeastern bank is currently using 75 micro-computers for such varied applications as: budget andfinancial analysis, gas and oil studies, balance sheet re-porting, and custom tailored accounts. Although a for-mal study has not been conducted, this firm believes thatsignificant gains in productivity have been achieved andthat microcomputers have proven to be very cost effec-tive.

7. Data Processing and Research

(Empirical Analysis)

A large data processing firm provides service for the50 major U.S. banks in the country. Some of the applica-tions handled by the firm include: a significant amountof file transfer-from micro to mainframe; extensiveword processing; budget analysis; and value added pro-cessing. A much greater workload (volume of trans-actions) is now being handled in a shorter period oftime due to the use of microcomputers. No formal studyhas been conducted, but it is accepted by the firm thatthe use of microcomputers has resulted in increasedproductivity.

8. Brokerage Firm (Empirical Analysis)

A large New York brokerage firm makes extensiveuse of microcomputers to handle customer accountsaround the country. While no formal studies have beenundertaken, customers have indicated that the use ofmicrocomputers generally results in productivity gainsand that the replication of successful microcomputerapplications would increase both productivity and costeffectiveness. Overall, this firm has been successful inthe introduction of microcomputers. However, con-cerns were expressed that there may not always be suf-ficient control and coordination of this process.

313

As stated in the introduction to this section, two dis-tinct conclusions can be drawn from the available infor-mation on the measurement of the effect on productivityfrom the introduction of microcomputer-based tech-nology into the office environment.

* Nearly everyone claims to have obtained signifi-cant improvements in productivity.

* Virtually no one has successfully measured andquantified those changes.

There is strong, anecdotal, circumstantial evidencethat most of the claims of increased productivity arecorrect. There is, however, little reason to accept thespecific percentages which are cited. Nevertheless, thestrong perception of improved productivity has beensufficient to justify the further acquisition of micro-computers in many organizations.

Improving Productivity: Management

Considerations

Once the meaning of productivity is understood, thenext step is to ensure that the environment is conduciveto the performance of high quality work, and that theresponsibilities for a high level of productivity are well-defined. Individuals should have a clear understandingof theirjob responsibilities and should be held account-able for results. They must be made to feel that theymake a difference. If productivity is a major or-ganizational concern, then the individuals within theorganization must also be a major concern. This impliesthat there must be a feeling engendered among theworkers that they are: valued, trusted, challenged, mak-ing a contribution, and involved in decisions affectingthem.

Increases in productivity and profitability, however,cannot be achieved simply with acquisition of new andbetter technology. Such acquisitions should be accom-panied by a management commitment to implement acostjustified, strategic, system integration approachwhich addresses the human or social aspects of auto-mation on both the individual and the organization. Toooften, productivity has little or no meaning for the indi-vidual since it is viewed as an attempt by management toimpose more procedures and controls, and ultimately,more work with little additional remuneration. There islikely to be little incentive to use new tools and tech-niques or to improve performance, if a feeling exists thatproductivity measures are only being taken to increasethe organization's image or profits at the expense of theworker.

Improvements in productivity require upper manage-ment to play an active role in the productivity im-provement program and also requires that the affectedindividuals realize benefits from the changes. The intro-duction and use of microcomputer and related hardwareand software must be accompanied by proper and ade-quate training and regard for the individual's workingenvironment.

In order to achieve significant productivity gains,there must be an integrated, cost-justified, program de-signed to achieve improvements in predefined areaswhich would benefit most by the improved pro-

ductivity. The areas most frequently cdited as being inneed of improvement are: management, incentives forindividuals, work environments, tools, training, soft-ware quality and software maintenance. Therefore, theinitiation of a comprehensive productivity improvementprogram can be both a costly and long term effort whichrequires careful planning, coordination, and the cooper-ation of all concerned.

It is essential to plan for some recognizable gainwithin the first two years, substantially more within thenext two or three, and depending upon the extent andcost of the effort to achieve an increase in productivity,there should be still more gain within the next five years.A well-planned productivity program should have asubstantial payoff within three to ten years.

Implementing a productivity program, however, isnot without some risks. If it is not well-planned, There islikely to be excessive optimism and overestimation ofpotential productivity gains. If the efforts are not well-coordinated, there may be only spotty success, resultingin little overall benefit to the organization. If it is viewednegatively, the workers may decide to thwart its imple-mentation, causing it to fail. And finally, unless there iscooperation between each level of management, profes-sionals, clericals, end users, and others for whom theprogram is intended, there is little likelihood for success.

Table 8 outlines the basic steps which should be takento establish a Productivity Improvement Programwithin an organization. Adherence to this program willhelp ensure that the measurement of changes in pro -ductivity, using the metrics discussed earlier in this paperwill result in definite gains in productivity from theintroduction of current technology into an office envi-ronment.

Summary and Conclusions

While no definitive techniques for measuring pro-ductivity changes in an functional workplace were identi-fied, this study did find overwhelming support for theidea that the introduction of microcomputers to the

314

functional workforce will result in significant im-provements in productivity. The "measurement" tech-niques employed have usually been highly subjectiveand sensitive to biasing factors which can make the citedstatistics highly suspect. The most that can be concludedis that microcomputers appear to increase productivityand that this perception is very widely held.

A few organizations have identified a set of measuresunique to their environments. Some of these, as in thecase of NSRDC, may be conceptually transferred toother environments. However, with the exception oftraditional method of measuring inputs and outputs (be-fore and after), there are virtually no universally ac-cepted productivity measures for use in an office envi-ronment.

Summary

Our findings indicate that there are several key fac-tors which should be clearly understood when at-tempting to evaluate and measure the productivitywithin an organization. These findings are summarizedbelow:

* Few Effective Measures

The measurement of changes in productivity in a pro-duction environment is a well-understood process: how-ever, the measurement of such changes in an office envi-ronment is much more difficult to quantify. Whilevarious methods have been proposed for measuringchanges in productivity resulting from the introductionof microcomputers, very few are effective.

The useful measures are those which primarily ad-dress qualitative aspects of the environment and thework produced. These measures are highly subjective

and thus, must also be evaluated in light of the methodsand techniques used to produce the measurements.

* Qualitative Measures Should Be Global

Too often, an attempt is made to measure productivityat the atomic or detailed level. The introduction of mi-crocomputers may affect how and what individuals inan organization do to varying degrees. Indeed, the ac-tual work (throughput) of some individuals may appearto decrease. While this would seem to contraindicate theuse of microcomputers in this case, an examination at ahigher, macro level, may show that there has been aresultant overall improvement in productivity.

The key to a successful productivity improvementprogram is to define what is expected in terms of pro.-ductivity gains, specifically, what and how to measure,and then proceed. The best measurements of pro-ductivity changes in an office environment are qual-itative. These measures are the most accurate when em-ployed at an organizational level. It should be clearlyunderstood that the goal of any productivity program isto improve the overall productivity of the organization.Thus, the measurements should be made at the globallevel and indications of increases or decreases in pro-ductivity at individual levels should be evaluated withinthe context of the productivity changes of the entireorganization.

* Measurements Must Be Made Against an Estab-lished Baseline

Results of measurement techniques can be highly sus-pect since the items being measured or counted are oftensubject to various conflicting interpretations. Regard-less of the method or technique used, it is essential tomeasure changes in productivity against an establishedbaseline. The questionnaire method appears to be themost useful in obtaining, comparing, evaluating, and

Table B. Establishing a productivity improvement program.

1. Determine how the tasks/jobs are currently done.2. Determine where performance cam be improved.3. Define the level of productivity gain expected.4. Perform a cost-benefit analysis to determine if this is a feasible expectation.5. Understand that introducing automation requires significant up front costs.6. Take future inflation into account.7. Determine the amount to be committed to achieve productivity gains.8. Determine for whom equipment will be used. In white collar domains, there are clearly two sets of users: professional and

clerical.9. Determine for which applications equipment will be used.

10. Determine what kind of equipment will be used.11. Evaluate large, difficult to maintain programs for possible replacement with off the shelf packages.12. Evaluate activities that require substantial resources, i.e., OR, simulation, job scheduling, for possible replacement by more ef-

ficient ones.13. Run a pilot to substantiate the expected improvements.14. If the pilot is successful, go forward with the whole program.

315

measuring relative changes in productivity against abaseline. This method provides for gathering informa-tion on qualitative, as well as quantitative changes inproductivity.

* Allow Sufficient Time Before Measuring ProductivityChanges

The introduction of any new technology involves alearning period during which there may actually be adecline in productivity. On the other hand, the ex-citement of being involved in an "experiment," such asthe introduction of microcomputers m an organization,may also lead to an increase in productivity. The effect,however, may be more pronounced immediately afterthe introduction of microcomputers than later. Con-sequently, sufficient time must be allowed after micro-computers are introduced into the organization to per-mit a stable state of operation to be established beforethe productivity impact is assessed.

* Measurements Should Be Carefully Evaluated

Improper or undisciplined use of microcomputers canbe counterproductive due to the wrong work beingdone or to new work being created which is not needed.Thus, the measurement of the effects of microcomputeruse should be carefully evaluated. Care should be takento evaluate the impact on the entire organization, notjust the directly affected individual. In addition, mea-surements must be made over a long enough time periodto balance any short term drops or rises in productivity.

Microcomputer use within the organization decen-tralizes the computing resources and enables users(ADP professionals as well as those without previousADP experience) to be in direct control of their infor-mation processing activities. The user has a wide rangeof information on which to base decisions and can makethose decisions quickly and accurately. Consequently,the user can perform his job more efficiently and effec-tively. While microcomputers are frequently used asstand alone, general purpose tools, they are rapidly be-coming a means of accessing the large scale systems andother microcomputers. The link between the micro-computer and mainframe provides the user with broaderrange of capabilities and greater potential for pro-ductivity gains. The result is the work being done faster,the quality of reports and other documents is improved,and work is performed which was previously not possi-ble.

Conclusions

The primary purpose for measuring changes in pro -ductivity resulting from the use of microcomputers is toprovide evidence of their cost-effectiveness and impact

on the organization. This can be accomplished in a num-ber of ways ranging from a cursory examination of ac-tivities and products before and after the introduction ofmicrocomputers, to a plan which encompasses the rec-ommended methodology outlined in table 6. Westrongly recommend the use of this methodology todetermine first, whether to initiate a measurement pro-gram; and secondly, how to go about performing themeasurements.

A significant number of Federal Government and pri-vate sector organizations have introduced micro-computers and subsequently, have been able to identifygains in overall productivity. Although many of theseorganizations introduced microcomputers without thebenefit of a prior productivity measurement study or pro-gram, evidence of their cost effectiveness and impact onthe organization has come from both managers and end-users. The management of these organizations is con-vinced that improvements in products, services, effi-ciency, morale, and numerous other areas occurred as aresult of using microcomputers.

Prior to initiating a study or program to measurechanges in productivity, some consideration should begiven to the feasibility and cost benefits of such a pro-gram. (see step I of table 6). If it is determined that aproductivity measurement program is not feasible, or ifthere is little or no assurance that such a program couldbe justified from a budgetary standpoint, then it may bebetter to rely on the judgement of the managers andusers of the microcomputers.

Therefore, while a study or program to measurechanges in productivity as a result of using micro-computers may be a worthwhile endeavor, it is not nec-essary or appropriate for every environment.

Selected Bibliography

Abrams, Marshal D., "Using the Desktop Computer for Project Man-agement," Proceedings of the 22nd Annual Technical Symposium ofthe Washington, DC, Chapter of the ACM, Co-sponsored by theNational Bureau of Standards, June 1983.

Bartino, Jim, "Study Takes Exception To Belief That Firns Don'tControl Micros," Computerworld, Volume 17, Number 51, De-cember 19, 1983, pp 1,6.

Basili, V., and M. Zelkowitz, "Analyzing Medium Scale SoftwareDevelopment," Proceedings of the 3rd International Conference onSoftware Engineering, IEEE, 1978, 00. pp 116.

Boczany, William J., "Justifying Office Automation," Journal of Sys-tems Management, July 1983 pp 15-19.

Boehm, Barry W., "The TRW Software Productivity System," Sep-tember 1983.

Brown, Bruce R., "Productivity Measurement in Software En-gineering," prepared by Social Security Administration (SSA),Office of System integration, Software Technology and En-gineering Center Staff, June 1983.

Brown, Gary D., and Donald H. Sefton, "The Micros vs The Applica-tions Logjam," Datarnation, January 1984, 96-104.

316

Clucas, Richard, "Are Your Computers Paying OfM?," Personal Com-puting, December 1983, pp 118-122,231,232. Office AutomationIssue, June 15, 1983, Volume 17, Number 24A, pp 47-52.

Cochran, Henry, "Fourth-Generation Languages," Computerworld,Office Automation Issue, June 15, 1983, Volume 17, Number24A, pp 47-52.

COMP83a, "Corporate Moves With Micros," Computerworld, Office

Automation Issue, Volume 17, Number 41A, October 12, 1983,pp 13-15.

COMP83b, "DP Managers Say DSS Needed To Tie Together Cor-porate Micros," Computerworld, Volume 17, Number 42, Octo-ber 12, 1983, pp 32.

COMP83a, "Corporate Moves With Micros," Computerworld, OfficeAutomation Issue, Volume 17, Number 41A, October 12, 1983,pp 13-15.

COMP83b, "DP Managers Say DSS Needed To Tie Together Cor-porate Micros," Computerworld, Volume 17, Number 42, Octo-ber 12, 1983, pp 32.

COMP83c, "Workstation Rules Out Paperwork for Court," Comput-erworld, Volume 17, Number 46, November 14, 1983, pp 50.

COMP83d, "Most Senior Accountants Found Using Micro," Comput-erworld, Volume 17, Number 46, November 14, 1983, pp 26.

DATAS3, "Micros at Big Firms: A Survey," conducted by DataDecisions, Datamation, November 1983, pp 161-174.

EDP83b, "Future Effects of The End User Computing," EDP Ana-lyzer, Volume 21, Number 11, November, 1983, pp 1-12.

Feezor, Betty, "Microcomputers: A Delicate Balance," Computer-world, Office Automation Issue, Volume 17, Number 32A, Au-gust 17, 1983, pp 9-10.

Ferris, David, "The Micro-Mainframe Connection," Datamation, No-vember 83, pp 127-138.

Fleming, Maureen, and Jeffrey Silverstein, "Microcomputens andProductivity:An Analysis of MicrocomputerHardware and Soft-ware Usage in Business," Knowledge Industry Publications,White Plains, New York, 1984.

Fried, Lousi, "Nine Principles for Ergonomic Software," Datama-tion, November 1983, pp 163-166.

GAO82, "Strong Central Management of Office Automation WillBoost Productivity," Comptroller General, General AccountingOffice report AFMD 82-54, September 1982.

GAO3, "Federal Productivity Suffers Because Word Processing IsNot Well Managed," General Accounting Office, FGMSD-79-17, Report to Congress of the United States, April 6, 1979.

Gillin, Paul, "One Unified Strategy," Computerworld, Office Auto-mation Issue, Volume 17, Number 42, October 17, 1983, pp 22.

Goldfield, Randy J., "Achieving Greater White-Collar Productivityin the New Office," BYTE, May 1983, pp 154-172.

Grabow, Paul C., William Noble, and Cheng-Chi Huang, "ReusableSoftware Implementation Technology Reviews," Prepared byHughes Aircraft Company, Ground Systems Group,N66001-83-D-0095, FR84-17-660, October 1984.

Horwitt, Elisabeth, "Creating Your Own Solutions," Business Com-puter Systems, June 1983, pp 130-141.

IDC84, "Strategies for Microcomputers and Office Systems, Cost

Justification of Office Systems," Prepared by IDC CorporateHeadquarters for Contious Information Service Clients, IDCNo. 2533, Framingham, MA. 01701, July 1984.

Keen, Peter G. W., "Decision Support Systems and Managerial Prod-uctivity Analysis," Sloan School of Management, MassachusettsInstitute of Technology, September 1980.

Lambert, G. N., "A Comparative Study of System Response Time OnProgram Developer Productivity," IBM Systems Journal, Vol.23, Number 1, 1984, pp 36-43.

Lochovshy, Fred, "Improving Office Productivity: A TechnologyPerspective," Proceedings of the IEEE, Volume 71, Number 4,April 1983, pp 512-518.

Lyons, Gordon, "Microcomputers and the Writing of Programs,"Proceedings of Trends and Applications, sponsored by IEEE Com-puter Society and the National Bureau of Standards, May 1982,pp 65-68.

Marcus, M. Lynne, "The New Office: More Than You BargainedFor," Computerworld, Office Automation Issue, February 23,1983, Volume 17, Number 8A, pp 37-44.

Martin, James, Application Development Without Programmers,Prentice-Hall, Inc., New Jersey, 1982, pp 161-177.

Osborne, Wilma, and Lynn Rosenthal, Metrics and MicrocomputerProductivity Bibliography, DOD/ICST Information TechnologyProductivity Study, National Bureau of Standards, Institute forComputer Sciences and Technology, 1984.

Plasket, Richard and Paula Wilneff, "Productivity and DP Manage-ment: Losing Control?," Journal of Systems Management, Octo-ber 1983, pp 30-33.

Powers, Dick, "Conquering Microphobia," Computermorld, OfficeAutomation Issue, Volume 17, Number 32A, August 17, 1983, pp49-50.

Pressman, Roger S., Software Engineering, A Practitioner's Approach,McGraw-Hill Book Company, New York, NY, 1982, pp 66-69,164, 173, 329.

Regan, Harry J., "Executive Workstations: Efficiencies and Oppor-tunities for Management," Proceeding of the 22nd Annual Tech-nical Symposium of the Washington DC Chapter of the ACM, co-sponsored by NBS, June 1983.

Rubin, Charles, "Computing In High Places," Personal Computing,November 1983, pp 77-85.

Ryan, Hugh, "End-User Game Plan," Datamation, December 1983,pp 241-244.

Scharer, Laura, "User Training: Less is More," Datamation, July1983, pp 175-182.

Teger, Sandra L., "Factors Impacting the Evolution of Office Aunto-mation," Proceedings of the IEEE, Volume 71, Number 4, April1983, pp 503-511.

Thadhani, A. J., "Factors Affecting Programmer Productivity Dur-ing Application Development," IBM Systems Journal, Vol 23,Number 1, 1984, pp 19-35.

Tharrington, James M., "How Microcomputers Can Aid in Applica-tions," Computerworld, November 14, 1983, pp 80.

Walton, William B., "New Support for the End User," Computer-wrld, Office Automation Issue, Volume 17, Number 32A, Au-gust 17, 1983, pp 27-32.

Young, Arthur, "The Impact of Low Cost Computing Technology onthe Department of Defense," report by Arthur Young and Co.,February 8, 1982.

Zack, Robert, and Steven Guthrie, "The Micro-to-Mainframe Link,"Computerworld, Office Automation Issue, Volume 17, Number48A, November 30, 1983, pp 11-15.

317