validation model

7/30/2019 Validation Model

1/21

Cronshaw, S. F., Best, R., Zugec, L., Warner, M. A., Hysong, S. J., & Pugh, J. A. (2007). A fivfunctional job analysis as used in job redesign. Ergometrika, 4, 12-31.

A FIVE-COMPONENT VALIDATION M

FUNCTIONAL JOB ANALYSIS AS USREDESIGN

STEVEN F.CRONSHAWUniversity of Northern British Columbia

LYNDA ZUGECMercer Human Resources Consulting

SYLVIA J.HYSONGHouston Center for Quality of Care and Utilization

Studies & Baylor College of Medicine

RLockheed Martin

MELISSUnive

JACQUVeterans Evidence-B

Implementatio

ABSTRACT


2/21

CRONSHAW, BEST, ZUGEC, WARNER, HYSONG & PUGH1991), that rely primarily on the analysis and use ofjob analysis ratings, may find some of the validationprinciples proposed here to be relevant to their workbut, in the main, other more empirically-basedvalidation approaches will be better suited to theirresearch and application needs.

A BRIEF DESCRIPTION OF FUNCTIONALJOB ANALYSIS (FJA)

As the starting point to this paper, we will describeFJA as a job analysis system. FJA was developed

by Sidney A. Fine as a continuation of work startedin the Functional Occupational Classification Project(FOCP). The FOCP was conceived and initiated byFine in 1948 and funded by the U.S. Department ofthe Air Force. FJA had a major impact on thedevelopment of the third and subsequent versions ofthe Dictionary of Occupational Titles or DOTpublished by the United States Employment Service

FJA is carried out by ajob incumbents (the opguided by a specially tfocus group meets ovethe guidance of the trafacilitator collects inforthe job incumbents areskills, and abilities req(3) Tasks performed inAdditional time may beperformance standardor the job as a whole.

group session is takenstatements that collectthe job being analyzedthe job analyst after ththen is sent back to thetheir editing recommenrevalidation (i.e., A sigthe task bank cover at


3/21

ERGOMETRIKA, v. 4FIVE STRATEGIES FOR VALIDATING FJATASK BANKS

We will begin our development of a model for jobanalysis validation with the definition of validity fromthe Standards for Educational and PsychologicalTesting (American Educational ResearchAssociation, et al., 1999), modified to address thevalidation problem for job analysis: Validity refers tothe degree to which evidence, as informed bytheory, supports the inferences made from task bankdata that are required to justify and support the

proposed human resource management (HRM)uses of this information. This definition of jobanalysis validity entails certain assumptions that wenow summarize.

An FJA task bank cannot be said to be valid for allpurposes. As a result, the purpose to which the jobanalysis will be put must be explicated before

Many applications of Finferences be validatecomprehensiveness, aWebsters Dictionary, winferences to correspothese terms. Job anaratings) are accurate tofrom mistakes; that theThe data are comprehare inclusive, they incldomain. They are usabeing used; if they are

three inferences are gTable 1 and then crossthe five validation stratdiscussed throughout introduction to this papillustrated in detail by FFJA can support, and validated for a broad r


4/21

CRONSHAW, BEST, ZUGEC, WARNER, HYSONG & PUGHas required by FJA theory and methodology?.Careful control over job language is a fundamentalFJA principle (Fine & Cronshaw, 1999). The joblanguage question in turn can be broken down intothree aspects: Semantics, syntax, and pragmatics(Baghramian, 1998). Semantics deals with themeaning and meaningfulness of terms andsentences in the FJA task bank. Syntax isconcerned with rules for linking together terms(words) into correctly constructed task statements.Pragmatics involves the use of language by fleshand blood speakers (Baghramian, 2002) to achieve

their goals when developing and using the FJA taskbank. All three aspects the semantics, syntax,and pragmatics of job descriptive language areaddressed systematically, and in a theory-drivenfashion, through application of the FJA model andmethodology.

Two linkages in Figurelinguistic validation strathe intended FJA applcorrespondence betweFJA (its underlying thestatements that comprlogical connection is reFigure 1. An indispenthe FJA task statemenFJA principles that undlanguage. These semprinciples have evolve

and process and are fadetail here. Interestedand Cronshaw (1999) controlled use of job la

Figure 1


5/21

ERGOMETRIKA, v. 4A second connection in Figure 1 (i.e., linkage 2)must be attended to if FJA task data are to be validfor the intended purpose. The task statements inthe FJA task bank must be written with sufficientprecision (i.e., freedom from mistakes in semantics,syntactics, and pragmatics) that they can be ratedon the FJA rating scales with a high degree ofinterrater agreement. The detailed definitions for theFJA scales that are used for that purpose are givenin Fine and Getkate (1995). Research byCronshaw, Chung-Yan, and Schat (2006) showsthat generally very good interrater agreement on the

FJA scales can be expected if task statements arewell written and the FJA scales are properly used.The rating of task statements using the FJA scales(whether or not estimates of interrater agreementare obtained) contributes to linguistic validationbecause it forces additional discipline on the carefulwriting of the task statements and also serves as aquality control check on the accuracy of the written

experiences of job holto job analysis data anjob analysis data and ton the job is likely to cinferences based on Fwhere workers are invinterventions which dra

The linkages in Figurevalidation are 3, 4, andsummarize and descriincumbents, as reporte

focus group under the who is well versed in F(linkage 4). As indicatstatements, again basthe focus group membconformance with the the task statement (asCronshaw 1999 p 75


6/21

CRONSHAW, BEST, ZUGEC, WARNER, HYSONG & PUGHthe organization that spans several locations, or amultinational company across many countries). Inthis study, we will consider ecological validation thatspans six occupations in the VA primary carefunction in hospitals across the U.S.. Ecologicalvalidation as we have conceptualized it here is in theBrunswickian tradition (Brunswik, 1956) and sofundamentally differs from test validation in that itrequires the examination of, and accounting for, thesampling adequacy of situations, rather than ofsubjects. The sampling of situations (in this case,situations as tasks), rather than people, moves

ecological validation outside the domain of the usualnomothetically-based measurement of individual(person) differences that has dominated psychology(Danziger, 1990). We believe that this shift from afocus on people as the unit of analysis to a focus onsituations is entirely consistent with, and in factnecessary to achieve, the proper validation of jobanalysis However it will require a readjustment of

Social-organizational vthe user community reinformation in the FJA uses. This FJA validasocial validation that hin the field of applied b(social validation was literature by Wolf, 197evaluates the acceptaprogrammed intervent(Schwartz & Baer, 199organizational validatio

when the job analysis excitement in the userwith the statements of that job analysis must boredom, on the part opositive sign if organizbecome genuinely inteinformation and comm


7/21

ERGOMETRIKA, v. 4Table 2

Questions Addressed by Five Validation Strategies

Validation Strategy Question Addressed

LinguisticIs the job language in the tacontrolled as required by FJmethodology?

ExperientialDoes the task bank describeexperience on the job?

Ecological

Do the tasks contained in thwork performed by job incumrelevant context(s) (e.g., witdepartment, function, or org

job analysis results and appgeneralize?


8/21

CRONSHAW, BEST, ZUGEC, WARNER, HYSONG & PUGHTable 3

Results of Task Clarity and Task Verification Ratings By VA Occupation

Number of participants

Primary care job

Data collectionsites In focus groups

Completingrevalidation

Meanrating

ta

1. Physician 3, 4, 6 14 7 8

2. NursePractitioners/Physicians

Assistants

1, 3, 6 18 7 8

3. RegisteredNurses

1, 2 11 0

4. LicensedVocational Nurses

1, 4, 5 13 8 8

5. HealthTechnicians

4, 6 12 11 5


9/21


An Illustrative FJA Task Statement with Scale Ratings

Call patient from waiting room, direct patient to office or exam room, explain nexpatient (e.g. vital signs), open patient information in CPRS, verifying accuracy opatient identity, SSN, DOB), In order to: prepare patient for measurement of vita

FJA Scale Ratings

Things Function: 1A Data Function: 1 People FunctiWorker Instructions: 2 Reasoning Development: 2

Mathematical Development: 1 Language Development: 3

Sociotechnical Demand Scale Ratings

Work Technology: 2 Work Interaction: 2

Note. In the subsequent path analysis, the constructs measured by the above scales are: DaPeople Function = People functional skill ; Worker Instructions = Decisional discretion; Work T


10/21

CRONSHAW, BEST, ZUGEC, WARN

Figure 2

FJA Worker Function Chart

4a. Precision Workingb. Setting Upc. Operating-Controlling II

3a. Manipulatingb. Operating-Controlling Ic. Driving-Controllingd. Starting Up

High

Medium

6. Synth

5a. Innob. Coo

4. Analy

3a. Comb. Com

Things Dat


11/21

ERGOMETRIKA, v. 4The Job Analysis Inferences Validated in this Study

Three specific inferences are to be validated before

the task dictionary is used as a resource in taskreallocation across the VA primary care function inU.S. hospitals operated by the VA:

i. the tasks in the task dictionary accuratelydescribe the tasks performed by the presentjob incumbents;

ii. the dictionary is comprehensive in itscoverage of tasks presently performed acrossVA primary care settings in the U.S.; and

iii. the dictionary is usable for task reallocation asa part of job redesign in the VA primary carefunction.

RESULTS

The results of the FJA validation are presentedbelow under each of the five validation strategies

guidelines for thetask statements.

The focus group

bank for revalidaproject personnerecommended bwere discussed incorporated into

The focus group particjob were asked to ratestatements as an emp

validation. The results reported in Table 3. TVA primary care jobs wexception of the HealthHealth Technician occmore heterogeneous sfive occupations, accoclarity rating in Table 3


12/21

CRONSHAW, BEST, ZUGEC, WARNER, HYSONG & PUGHfor a given task statement, the task statementwas taken back to a job incumbent (not in theoriginal focus group) for rewording to achieve

better accuracy.

As an empirical means of experiential validation thefocus group participants for each primary care jobwere asked to verify whether they performed thetasks in the revalidation task bank for theiroccupation. The results of these analyses arereported in the far right column of Table 3. The taskverification percentages over the six VA primary

care jobs were high, again with the exception of theHealth Technician occupation, the greaterheterogeneity of the Health Technician occupationprobably accounting for its lower endorsement ratecompared to that of the other five primary careoccupations.

Ecological Validation Evidence for the

on the basis of thidentified by our cause systemati

collected in the Vcharacteristics wbecause larger fstaff, including ssmaller facilities perform healthcaPresence of a mbecause the mixmedical resident

and (3) Differencand communicatdegree of serviceexposure to adva(e.g., to reduce phospitals (see Yaand Rubenstein,

A roll out proce


13/21

ERGOMETRIKA, v. 4FJA theory uses the task as the basic unit of work tounderstand how worker skills can be best broughtinto alignment with the content of the work being

done and the context of the work-doing system thatsurrounds and envelopes the worker.Sociotechnical theory takes a task-based approachto conceptualization and analysis, and revolvesaround the proposition that organizationalobjectives are best metby the joint optimization ofthe technical and social system (Cherns, 1978, p.63). In both the FJA and sociotechnical systemstheories, workers as whole persons must be

integrated into the technical and organizationalarrangements of the workplace. Strong support forthe usability of the FJA task dictionary in jobredesign would be obtained if we could show thatcross-over propositions can be generated from anintegration of the FJA and sociotechnical theories.Recent research has provided support for the ideathat an integration of the FJA and sociotechnical

2001). The usability othis purpose would be for hypothetical-criteria

provided) if the predict(2003) were replicateddictionary. We used Lproposed by those autsociotechnical demandtask dictionary. The reanalysis using ratings VA task dictionary is pintercorrelation matrix

analysis is presented ibetween decisional disskill (the only path diffeAlfieri results) producemodel to the observedFit Index = .90). The when the VA data are the form of clinical pra


14/21

CRONSHAW, BEST, ZUGEC, WARN

Figure 3

Path diagram demonstrating mediating impact of worker instructions on relationfunctional skills

Decisiondiscretio

.45 (.35)

Complexity oftechnological

demand

.20 (.52)

.08 (.33)


15/21


Cross-tabulation of Levels of Social Demand with Levels of General Educational De

Care Tasks

Range of GED Rating Needed to Work at Giv

Level of Work Interaction(Social-organizationaldemand)

Reasoning Development Mathematics Development

1 2 3 4 5 1 2 3

1. Solitary work 6 14 14 9 2 36 8 1 2. Unilateral assistance 19 26 12 16 3 60 14 2

3. Bilateral assistance 20 29 20 27 14 86 20 4

4. Behavioral coordination 2 3 2 2 3 8 3 1


16/21

CRONSHAW, BEST, ZUGEC, WARNER, HYSONG & PUGHanalysis evidence, as informed by theory, supportthe interpretations of those data required for givenHRM uses. In this case, are the FJA task dictionary

and its component 243 task statements sufficientlyaccurate, comprehensive, and usable for theintended use for task reallocation in the VA? Basedon our validation results, we conclude that the VAtask dictionary is valid for the intended job redesignpurpose.

The relevance of the validation study is betterappreciated if the reader has a better understanding

of the approach and purpose of the proposed VA jobredesign. The VA approach to the job redesignrelies heavily on task reallocation, a common andwell-established practice in the job redesign field(Older, Waterson, & Clegg, 1997). Older et al.(1997) review the literature on task reallocationmethods and present a list of 20 requirements fortask reallocation (e g first the method must be

Table 7

Task Reallocation Crit

Redesign in the VA

1. Minimize task

primary care f

2. Ensure worke

with social-org

3. Maximize task

arrangements4. Match worker

arrangements

supervision) w

required to do

5. Maximize cost

6 Ensure human


17/21

ERGOMETRIKA, v. 4Figure 4

FJA-based interventions needed to rebalance the work-doing system concurrent wit

using VA task dictionary

Worker

Revise / rewrite HR policyand procedures; provide

required information,technology / tools and

equipment

Efficiency/Effectiveness

Productivity


18/21

CRONSHAW, BEST, ZUGEC, WARNER, HYSONG & PUGHredesigned primary care function. In turn, anaccurate and comprehensive task dictionary overthe primary care function and one that is validatedas such will assist in the exact wording of new HRpolicy and procedure (because the data arelinguistically validated) and help ensure thatrewritten policies and procedures documentationapplies across all VA facilities (because the data areecologically validated).

The larger corpus of job analysis literature has givensome attention to problem of validating job analysisdata. Most of these researchers have focused theirattention on validating job analysis ratings, rather

than job analysis inferences investigated in thisstudy. Tasks, jobs, and occupations can be rated ona number of scale metrics, including importance,criticality-of-error, criticality, and/or time spent(Harvey, 1991) and, more recently, improvability ofcareer-relevant skills (Maurer, Wrenn, Pierce, Tross,& Collins, 2003). Sometimes, the rating data from

meaningful metric andvalidation evidence neIn our validation modeSanchez and Levine isinference of job analysaddressed by both hyporganizational validatiocriterion proposed by Hsubsumed under the inaccuracy and is addreexperiential validation.to both ratings and naanalysis, the validationenriched enough to sh

validity (usability) and addressed, at least asThese two types of vamutually exclusive as Sanchez-Levine vs. Hboth sources are impoeach other in supportin


19/21

ERGOMETRIKA, v. 4Author Note

The research reported here was supported by theDepartment of Veterans Affairs, Veterans Health

Administration, Health Services Research and

Development Service (IIR #01-185 & HFP #98-002/REAP #05-129). The five junior authors salarieswere supported in part by the Department of Veterans

Affairs during the conduct of the research reported here.The views expressed in this article are those of theauthors and do not necessarily reflect the position orpolicy of the Department of Veterans Affairs or thepresent employers of the five junior authors. An earlierversion of this paper was presented at the 2006 AnnualMeeting of the Academy of Management in Atlanta, GA.

Correspondence and requests for reprints can beaddressed to Steven F. Cronshaw, School of Business,College of Science and Management, University ofNorthern British Columbia, Prince George, BC, CanadaV2N 4Z9.

REFERENCES

Cronshaw, S. F., & Alfof sociotechnical task discretion and function1107-1130.

Cronshaw, S. F., ChunH. (2006). ComparativAnalysis scale ratings rating and consensus manuscript.

Cronshaw, S. F., & Finevaluation of job redes

Edwards, J.C. Scott, &301-321), The human handbook. Thousand O

Cummings, L. L. & WoDevelopment and Cha


20/21

CRONSHAW, BEST, ZUGEC, WARNER, HYSONG & PUGHHarvey, R. J., Friedman, L., Hakel, M. D., &Cornelius, E. T. (1988). Dimensionality of the JobElement Inventory, a simplified worker-oriented jobanalysis questionnaire. Journal of AppliedPsychology, 73, 639-646.

Harvey, R. J., & Wilson, M. A. (2000). Yes Virginia,there is an objective reality in job analysis. Journal ofOrganizational Behavior, 21, 829-854.

Husserl, E. (1973). Experience and judgment:Investigations in a genealogy of logic. Evanston, IL:Northwestern University Press.

Klein, J. A. (1991). A re-examination of autonomy inlight of new manufacturing practices. HumanRelations, 44, 21-38.

Lewin, K. (1951). Field theory in social science;selected theoretical papers. In D. Cartwright (ed.).New York: Harper & Row.

Nunnally, J. C., & BernPsychometric theory (3Hill.

Older, M. T., Waterson(1997). A critical reassmethods and their app151-171.

Prien, E. P., Prien, K. OPerspectives on noncomethodologies. JournaPsychology, 18, 337-3

Rousseau, D. M. (198working with data, peoOccupational Psycholo

Ryan, A. M., & Sacketbetween graduate traiand individual psychol


21/21

Copyright 2007, by The Mitchell Project, LLC.

Permission to make digital or hard copies of all or part of this work for persgranted without fee provided that copies are not made or distributed for profitand that copies bear this notice and full citation on the first page. Copyrigwork owned by others than The Mitchell Project, LLC must be honored. permitted. To copy otherwise, to republish, to post on servers for commercialists requires prior specific permission and/or fee.

Request permission to publish from: Managing Editor, The Mitchell Project,

San Antonio, TX 78247.

validation model

Documents