Validity of Observational Validity of Observational Job Analysis MethodsJob Analysis Methods
Brian D. Lowe, Ph.D., CPEBrian D. Lowe, Ph.D., CPENational Institute for Occupational Safety and HealthNational Institute for Occupational Safety and Health
Cincinnati, OHCincinnati, OH
August 12, 2003August 12, 2003
presentation outline
Physical risk factors for WMSDs and job analysis methods for their characterization
NIOSH study of observational job analysis methods
Methods Results Conclusions
Validity considerations in job analysis
methods for assessing WMSD risk factorsmethods for assessing WMSD risk factors
Job Titles/SIC codeJob Titles/SIC code
Worker Self ReportWorker Self Report
Systematic ObservationSystematic Observation
Direct Measurement Direct Measurement (Instrumentation)(Instrumentation)
increasingreliability &precision
increasingconvenience
goals for exposure characterization(Kilbom, 1994)
• External Validity - identify exposures associated with increased risk for WMSDs
epidemiology
• Internal Validity - exposure is classified accurately relative to a known standard
biomechanics
Exposure Response
ObjectiveObjective
Group methods of scaling risk factors used in Group methods of scaling risk factors used in observational-based job analysesobservational-based job analyses
Compare observational estimates of risk factors Compare observational estimates of risk factors with instrumentation-based measureswith instrumentation-based measures electrogoniometer – wrist/forearm posture/kinematicselectrogoniometer – wrist/forearm posture/kinematics
optical motion capture – shoulder posture/kinematicsoptical motion capture – shoulder posture/kinematics
electromyography – force of exertionelectromyography – force of exertion
explore the likelihood and nature of errors in explore the likelihood and nature of errors in exposure characterizationexposure characterization
jobs simulated in the laboratoryjobs simulated in the laboratory
Job A~ 13 s
Job B ~ 8 s
Job C~ 56 s
Job D~ 46 s
electrogoniometerelectrogoniometer
-120
-80
-40
0
40
80
120
0 10 20 30 40 50 60
flexion/extension (α) supination/pronationJob C - cycle 3
angl
e (d
eg)
α
optical motion captureoptical motion capture
A: 4 – 2 HCSxB: 4 – 3C: A x B HCSyD: A x C HCSzE: 7 – 5 TCSZF: 7 – 6G: F x E TCSYH: G x E TCSX
ZT
-YT
-XT
-zH
-yH
xH
5
6
7
4
3
2
A
E
F
B
trunk
humerus4
2
3anatomicalpod
A: 4 – 2 HCSxB: 4 – 3C: A x B HCSyD: A x C HCSzE: 7 – 5 TCSZF: 7 – 6G: F x E TCSYH: G x E TCSX
ZT
-YT
-XT
-zH
-yH
xH
5
6
7
4
3
2
A
E
F
B
trunk
humerus4
2
3
4
2
3
2
3anatomicalpod
motion capture – shoulder kinematicsmotion capture – shoulder kinematics = cos-1 (X · x) = cos-1 [(Y · x)/sin()] = cos-1 [ -(X · y)/sin()]
x – z’ – x” Euler angle sequence: Rotation about x : Rotation about z’: Rotation about x”
- shoulder elevation - plane of shoulder elevation
0
0
30
60
90
120
0 10 20 30 40 50
elbow
sh elev
pl sh elev
-90
-60
-30
0
30
60
90
120
0 10 20 30 40 50
flex/ext
pro/sup
14.8000 s 14.8333 s14.7667 s
join
t a
ngl
e (d
egr
ees
)jo
int
an
gle
(de
gre
es)
time (s)
cycle 1 cycle 2 cycle 3 cycle 4
0
30
60
90
120
0 10 20 30 40 50
elbow
sh elev
pl sh elev
-90
-60
-30
0
30
60
90
120
0 10 20 30 40 50
flex/ext
pro/sup
14.8000 s 14.8333 s14.7667 s
join
t a
ngl
e (d
egr
ees
)jo
int
an
gle
(de
gre
es)
time (s)
cycle 1 cycle 2 cycle 3 cycle 4
video and instrumentation synchronizationvideo and instrumentation synchronization
participants and procedureparticipants and procedure
ParticipantsParticipants
28 professional ergonomists28 professional ergonomists
14 from academia,14 from industry/consulting14 from academia,14 from industry/consulting
12 - Ph.D./M.D., 13 - M.S., 3 - B.S.12 - Ph.D./M.D., 13 - M.S., 3 - B.S.
Years experience in ergonomics (1 – 30 yrs.)Years experience in ergonomics (1 – 30 yrs.)
ProcedureProcedure
Assigned one method for posture analysisAssigned one method for posture analysis
Estimated posture from video recording of jobs Estimated posture from video recording of jobs
Analyses were unguidedAnalyses were unguided
posture scalingposture scalingmethod 1 – 3 categoriesmethod 1 – 3 categories
11 22 33elbow flex elbow flex (deg)(deg) <40<40°° 4040°°-80-80°° >80>80°°
shoulder elev shoulder elev (deg)(deg) 00°°-40-40°° 4040°°-80-80°° >80>80°°
plane of sh elev plane of sh elev (deg)(deg) <30<30°° 3030°°-90-90°° >90>90°°
11 22 33
modemodepeakpeak wrist flex wrist flex (deg)(deg) >20>20°° 2020°°-0-0°°
peakpeak wrist ext wrist ext (deg)(deg) 00°°-20-20°° >20>20°°
modemodepeakpeak forearm sup forearm sup (deg)(deg) >40>40°° 4040°°-0-0°°
peakpeak forearm pro forearm pro (deg)(deg) 00°°-40-40°° >40>40°°
posture scalingposture scalingmethod 2 – 6 categoriesmethod 2 – 6 categories
11 22 33 44 55 66wrist flexwrist flex >45>45 45-20 45-20 20 -020 -0
wrist extwrist ext 0-200-20 20-4520-45 >45>45
forearm supforearm sup >60>60 60-3060-30 30-030-0
forearm proforearm pro 0-300-30 30-6030-60 >60>60
elbow flexelbow flex <20<20 20-4020-40 40-6040-60 60-8060-80 80-10080-100 >100>100
shoulder elevshoulder elev <20<20 20-4020-40 40-6040-60 60-8060-80 80-10080-100 >100>100
plane of sh elevplane of sh elev <0<0 0-300-30 30-6030-60 60-9060-90 90-12090-120 >120>120
posture scalingposture scalingmethod 3 - visual analog scale (VAS)method 3 - visual analog scale (VAS)
0neutral
10extreme
2 4 6 8
wrist flexionwrist extension
forearm supinationforearm pronation
elbow flexionshoulder elevation
plane of shoulder elevation
0°0°0°0° 0°0°0°
95°85°
145°135°150°180°150°
ResultsResultswrist/forearm – 3 categories (method 1)wrist/forearm – 3 categories (method 1)
0%
20%
40%
60%
80%
100%
-2 -1 0 1 2
error (# of categories)
prob
abilit
y
mode flex/ext
peak flex
peak ext
0%
20%
40%
60%
80%
100%
-2 -1 0 1 2error (# of categories)
pro
ba
bili
ty
mode sup/pro
peak sup
peak pro
error = estimated - measured
elbow/shoulder – 3 categories elbow/shoulder – 3 categories (method 1)(method 1)
mode
0%
20%
40%
60%
80%
100%
-2 -1 0 1 2error (# of categories)
prob
abili
ty
elbow flex
sh elev
pl sh elev
peak
0%
20%
40%
60%
80%
100%
-2 -1 0 1 2error (# of categories)
prob
abili
ty
elbow flex
sh elev
pl sh elev
wrist/forearm – 6 categories wrist/forearm – 6 categories (method 2)(method 2)
0%
10%
20%
30%
40%
50%
60%
-5 -4 -3 -2 -1 0 1 2 3 4 5
error (# of categories)
pro
ba
bili
ty
mode flex/ext
peak flex
peak ext
0%
10%
20%
30%
40%
50%
60%
-5 -4 -3 -2 -1 0 1 2 3 4 5error (#of categories)
pro
ba
bili
ty
mode sup/propeak suppeak pro
elbow/shoulder – 6 categories elbow/shoulder – 6 categories (method 2)(method 2)mode
0%
20%
40%
60%
-4 -3 -2 -1 0 1 2 3 4error (# of categories)
prob
abili
ty
elbow flexsh elevpl sh elev
peak
0%
20%
40%
60%
-4 -3 -2 -1 0 1 2 3 4
error (# of categories)
prob
abili
ty
elbow flex
sh elevpl sh elev
VAS – flexion/extension VAS – flexion/extension (method 3)(method 3)
peak average
wrist flexion wrist extension
r2 = 0.31*r2 = 0.28*
r2 = 0.02 r2 = 0.00
0%
20%
40%
60%
80%
100%
0% 20% 40% 60% 80% 100%
measured
est
ima
ted
0%
20%
40%
60%
80%
100%
0% 20% 40% 60% 80% 100%
measured
est
ima
ted
VAS – supination/pronation VAS – supination/pronation (method 3)(method 3)
peak average
forearm supination forearm pronation
r2 = 0.02r2 = 0.03
r2 = 0.02 r2 = 0.09
0%
20%
40%
60%
80%
100%
0% 20% 40% 60% 80% 100%
measured
est
ima
ted
0%
20%
40%
60%
80%
100%
0% 20% 40% 60% 80% 100%
measurede
stim
ate
d
VAS – shoulder and elbow VAS – shoulder and elbow (method 3)(method 3)
0%
20%
40%
60%
80%
100%
0% 20% 40% 60% 80% 100%
measured
estim
ated
0%
20%
40%
60%
80%
100%
0% 20% 40% 60% 80% 100%
measured
estim
ated
0%
20%
40%
60%
80%
100%
0% 20% 40% 60% 80% 100%
measured
estim
ated peak average
elbow flexion shoulder elevation plane of shoulder elev +
r2 = 0.47*r2 = 0.49*
r2 = 0.66*r2 = 0.46*
r2 = 0.03r2 = 0.18*
temporal distribution of posture temporal distribution of posture (wrist/forearm – 3 category)(wrist/forearm – 3 category)
0%
20%
40%
60%
80%
>20º f lex neutral(±20º)
>20º ext >40º sup neutral(±40º)
>40º pro
w rist f lex/ext forearm sup/pro
estimatedmeasured
per
cent
of w
ork
cyc
le
N N
N = neutral posture
temporal distribution of posturetemporal distribution of posture(wrist/forearm – 6 category)(wrist/forearm – 6 category)
0%
20%
40%
60%>
45°
f
45°
- 20
° f
20°
- 0°
f
0° -
20°
e
20°
- 45
° e
> 45
° e
> 60
° s
60°
- 30
° s
30°
- 0°
s
0° -
30°
p
30°
- 60
° p
> 60
° p
w rist f lex/ext forearm sup/pro
estimatedmeasured
per
cent
of w
ork
cyc
le
N N
temporal distribution of posturetemporal distribution of posture(elbow/shoulder – 3 category)(elbow/shoulder – 3 category)
0%
20%
40%
60%
80%
100%
0º - 40º 40º - 80º > 80º 0º - 40º 40º - 80º > 80º < 30º 30º - 90º > 90º
elbow flex/ext shoulder elev shoulder plane of elev
estimatedmeasured
per
cent
of w
ork
cyc
le
N N N
temporal distribution of posturetemporal distribution of posture (elbow/shoulder – 6 category)(elbow/shoulder – 6 category)
0%
20%
40%
60%0º
-20º
20º-
40º
40º-
60º
60º-
80º
80º-
100º
>100
º
0º-2
0º
20º-
40º
40º-
60º
60º-
80º
80º-
100º
>100
º
<0º
0º-3
0º
30º-
60º
60º-
90º
90º-
120º
>120
º
elbow flex/ext shoulder elev shoulder plane elev
estimatedmeasured
per
cent
of w
ork
cyc
le
N N N
DiscussionDiscussion
Performance does not necessarily reflect Performance does not necessarily reflect bestbest casecaseLimitations of the StudyLimitations of the Study Single video viewSingle video view Simulated job tasks (laboratory study)Simulated job tasks (laboratory study) Analysts had no familiarity with jobsAnalysts had no familiarity with jobs Methods may not have been familiar to analystsMethods may not have been familiar to analysts Little information regarding the strategy analysts usedLittle information regarding the strategy analysts used
Intended to reflect performance in the Intended to reflect performance in the typicaltypical case case
summary of findingssummary of findings
Posture classification accuracy related to the Posture classification accuracy related to the size of the joint/limb segments (Genaidy size of the joint/limb segments (Genaidy et alet al, , 1993; Baluyut 1993; Baluyut et alet al, 1995), 1995)
Posture classification accuracy related to the Posture classification accuracy related to the number of scale categoriesnumber of scale categories
p(correct classification) = 73% for most frequent p(correct classification) = 73% for most frequent shoulder/elbow posture w/3 categoriesshoulder/elbow posture w/3 categories
p(correct classification) = 30% for most frequent p(correct classification) = 30% for most frequent wrist/forearm posture w/6 categorieswrist/forearm posture w/6 categories
validity considerations in job analysisvalidity considerations in job analysis
Misclassification of working posture occurred Misclassification of working posture occurred in job analyses even when using a small in job analyses even when using a small number of posture categoriesnumber of posture categories
Posture misclassifications with higher Posture misclassifications with higher precision scale were more frequent, but their precision scale were more frequent, but their effect is lesseffect is less
Duration severity of posture tended to be Duration severity of posture tended to be underestimatedunderestimated
Disclaimer
Mention of any company name or product, or inclusion of any reference, does not constitute endorsement by the National Institute for Occupational Safety and Health.
Acknowledgment
The contributions of Dan Habes, NIOSH, Ed Krieg, NIOSH, and Ahmed Khalil, University of Cincinnati are greatly appreciated.
risk factors in physical workrisk factors in physical work
risk factors for work related musculoskeletal risk factors for work related musculoskeletal disorders (WMSDs)disorders (WMSDs)
postureposture
forceforce
repetitionrepetition
vibrationvibration
Ergonomic Exposure Assessment – Ergonomic Exposure Assessment – Observational AccuracyObservational Accuracy
temporal scalingmag
nitu
de s
cali
ng
time
post
ure
accuracy
lab simulation video recording presented to ergonomists
Motion AnalysisGoniometer
observation
job analysis methods for the systematic job analysis methods for the systematic observation of postureobservation of posture
peak (peak posture)
average (average posture)
mode (category with most frequently
occurring posture)
work sampling (periodic
observations)
near continuous (posture category vs.
time recording)
low (2 - 3 categories)
medium (4 - 6 categories)
high (> 6 categories, VAS
method or continuous
scale)
increasing difficulty
RULA
STRAIN INDEX
Keyserling (1986)
Armstrong et al (1982)
OCRA
Latko (1997)
OWAS
Drury (1987)
Temporal
Sp
atia
l
work cycle analysiswork cycle analysis
shoulder elevation – Job C
0
40
80
120
0 20 40 60 80 100 120 140 160 180 200 220
time (s)
angu
lar
posi
tion
(deg
)
cycle 1 cycle 2 cycle 3 cycle 4
upper limb postures evaluatedupper limb postures evaluatedelectrogoniometer
optical motion
capture
summary of other findingssummary of other findings
Time to completion of the analysis was not Time to completion of the analysis was not related to the resulting accuracyrelated to the resulting accuracy
No relationship between years experience and No relationship between years experience and accuracy of observational estimatesaccuracy of observational estimates
No relationship between work cycle variability No relationship between work cycle variability and accuracy of observational estimatesand accuracy of observational estimates
radial/ulnar deviationradial/ulnar deviation
Inter-rater agreement statisticsInter-rater agreement statistics
Intraclass correlation coefficient among raters Intraclass correlation coefficient among raters (ergonomists) less than for flex/ext, sup/pro(ergonomists) less than for flex/ext, sup/pro
3-category3-category 6-category6-category
flex/extflex/ext 0.2290.229 0.3420.342
pro/suppro/sup 0.2150.215 0.3080.308
rad/ulnrad/uln 0.2170.217 0.1230.123
Juul-Kristensen et al. (1997)Juul-Kristensen et al. (1997)
0%
20%
40%
60%
80%
flexion mild flexion neutral mild extension extension
pe
rce
nt o
f wo
rk c
ycle
analysts - present study
measured - present study
analyst - Juul Kristensen et al.
measured - Juul Kristensen et al.
Electrogoniometer Calibration
RR22 maximum errormaximum error
flex/extflex/ext 0.990.99 0.50.5°° @ 45 @ 45°° flex flex
sup/prosup/pro 0.940.94 2.52.5°° @ 45 @ 45°° pro pro
rad/ulnrad/uln 0.800.80 1010°° @ 30 @ 30°° uln uln
choice of ROM as VAS anchorchoice of ROM as VAS anchor
0° 100°
0° 80°
true magnitude
75%
60%
60°
Observation vs. ChanceObservation vs. Chance
estimated
> 60 sup 30 - 60 sup 0 - 30 sup 0 - 30 pro 30 - 60 pro > 60 pro marginal
> 60 sup 0.000 0.000 0.000 0.000 0.000 0.000 0.000
30 - 60 sup 0.000 0.000 0.000 0.000 0.000 0.000 0.000
measured 0 - 30 sup 0.000 0.000 0.025 0.075 0.000 0.000 0.100
0 - 30 pro 0.000 0.000 0.025 0.150 0.075 0.050 0.300
30 - 60 pro 0.000 0.000 0.000 0.050 0.000 0.025 0.075
> 60 pro 0.000 0.075 0.075 0.175 0.000 0.200 0.525
marginal 0.000 0.075 0.125 0.450 0.075 0.275 1.000
> 60 sup 30 - 60 sup 0 - 30 sup 0 - 30 pro 30 - 60 pro > 60 pro marginal
> 60 sup 0.000 0.000 0.000 0.000 0.000 0.000 0.000
30 - 60 sup 0.000 0.000 0.000 0.000 0.000 0.000 0.000
measured 0 - 30 sup 0.017 0.017 0.017 0.017 0.017 0.017 0.100
0 - 30 pro 0.050 0.050 0.050 0.050 0.050 0.050 0.300
30 - 60 pro 0.013 0.013 0.013 0.013 0.013 0.013 0.075
> 60 pro 0.088 0.088 0.088 0.088 0.088 0.088 0.525
marginal 0.167 0.167 0.167 0.167 0.167 0.167 1.000
ergonomists’observation
chance