maintaining physical standards using physical ability tests are your incumbents fit for the job
DESCRIPTION
Guidenace on implementing a physical ability test for your incumbent work force in a job-related and defensible manner.TRANSCRIPT
Fire & Police Selection, Inc. (FPSI) © 2011
WWW.FPSI.COM
Maintaining Physical Standards Using Physical Ability Tests: Are Your Incumbents Fit for the Job?
September 21, 2011
Stacy L. Bell, MSDan Biddle, PhD
Fire & Police Selection, Inc. (FPSI) © 2011
About the Speakers
• Stacy L. Bell, MS—Executive Vice President of Fire & Police Selection, Inc.
• Dan Biddle, PhD—President/CEO of Fire & Police Selection, Inc. (FPSI) and Biddle Consulting Group
Fire & Police Selection, Inc. (FPSI) © 2011
Presentation Overview
• Review of critical firefighter physical abilities • Comparison of fitness testing vs. work-sample testing• National survey results regarding the need for incumbent
testing• Legal pitfalls associated with arbitrary fitness tests and the
risks associated with employing inappropriate standards at the incumbent level
• Appropriate techniques to use for setting PAT cutoff scores for new hires and for incumbents
• Importance of “maintenance/wellness” testing of incumbents
Fire & Police Selection, Inc. (FPSI) © 2011
Critical Firefighter Physical Abilities
• 1994-1996 Firefighter Physical Ability Validation Study◦ 331 Fire Suppression Personnel from 41 fire departments identified
the critical physical duties performed by fire suppression personnel:
– Hose drags (dry and charged)
– Ladder raises, removal, and carries
– Walking and operating on ladders
– Searching for fire extension
– Removing conscious/unconscious victims from fire scene
– Ventilation techniques
– Climbing stairs while carrying tools/equipment
– Hoisting operations
Fire & Police Selection, Inc. (FPSI) © 2011
Comparing Fitness Tests to Work-Sample Tests
• Examples of Fitness Tests◦ 1.5 mile run
◦ Push-ups/Pull-ups
◦ Sit-ups
◦ Aerobic Capacity – Sub Maximal Stress Test
◦ Muscular Endurance & Strength (Arms & Legs)
◦ Flexibility Measurements (Shoulder, Trunk, & Legs)
◦ Body Fat Composition and BMI
• Samples of Work Sample Tests◦ Hose drags: http://fpsi.com/fire/video/fsi1.mpg
http://fpsi.com/fire/video/fsi2.mpg
◦ Walking/working on ladders: http://fpsi.com/fire/video/fsi4.mpg
◦ Removal of unconscious/conscious victim: http://fpsi.com/fire/video/fsi7.mpg
◦ Climbing stairs with tools/equipment: http://fpsi.com/fire/video/fsi11.mpg
Fire & Police Selection, Inc. (FPSI) © 2011
What’s the Big Difference Between a Fitness Test and a Work Sample Test?
• Fitness Tests measure a person’s fitness (in general) and then make the inference that “if they have fitness level X they should be able to perform the job of Y”
• Some fitness tests (e.g., VO2 Max/Cardio fitness) require factoring age and gender into the equation
◦ This can be tricky with various employment discrimination laws and federal enforcement agencies
• Work Sample tests directly measure whether applicants/incumbents can perform the job task.
◦ They answer the question: “Irrespective of this person’s age, gender, race, or disability, can they do the job?”
Fire & Police Selection, Inc. (FPSI) © 2011
NFPA 1583: Standard on Health-Related Fitness Programs for Firefighters
• In August of 2000, the NFPA published the current standard on health-related fitness programs for firefighters. The standard requires:
◦ Fire departments shall require structured participation of the health-related fitness program
◦ All members shall participate annually (at least) in a fitness assessment which measures:
– Aerobic capacity
– Body composition
– Muscular strength
– Muscular endurance
– Flexibility
Fire & Police Selection, Inc. (FPSI) © 2011
The Reality of Firefighter Fitness Programs
Source: United States Fire Administration, Survey of the Needs of the U.S. Fire Service – 2002Note: Based on 8,267 departments reporting. Numbers may not add to totals due to rounding.
10026,35480.221,14919.85,205TOTAL
10013,4408711,80112.21,639Under 2,500
1004,57282.53,77417.57982,500 to 4,999
1003,62975.92,75624.18735,000 to 9,999
1002,84366.21,88133.896210,000 to 24,999
1001,05352.455247.650125,000 to 49,999
10048750.124449.924350,000 to 99,999
10021546.510053.5115100,000 to 249,999
1006440.62659.438250,000 to 499,999
1003828.91171.127500,000 to 999,999
1001338.5561.581,000,000 or more
PercentNumber Depts.
PercentNumber Depts.
PercentNumber Depts.
TOTALNOYESPopulation of Community
Fire & Police Selection, Inc. (FPSI) © 2011
National Survey Results 2011
• FPSI surveyed over 2,000 fire departments across the U.S. to determine current practices of employing fitness/maintenance tests on incumbents and the need for such programs.
• Survey questions included:◦ Does your department use a PAT as part of its "new hire" recruitment process?
◦ Does your department use a PAT as part of an annual/maintenance standard for your incumbents?
◦ Do you believe that ACTIVE FIRE SUPPRESSION PERSONNEL should be tested annually to ensure that they possess the minimum physical abilities necessary to successfully perform the job?
◦ Who should be REQUIRED to pass an annual maintenance/wellness PAT?
◦ Which of the following consequences do you feel are acceptable for ACTIVE FIRE SUPPRESSION who cannot pass a maintenance/wellness PAT?
Fire & Police Selection, Inc. (FPSI) © 2011
Use of a PAT for New Hires
87%
13%
Fire & Police Selection, Inc. (FPSI) © 2011
Use of a PAT for Incumbents
72%
28%
Fire & Police Selection, Inc. (FPSI) © 2011
Need for a PAT for Incumbents
94%
6%
Fire & Police Selection, Inc. (FPSI) © 2011
Which Incumbents Should be Tested
99%
90%85%
84%
65% 64%
37%33%
40%40%
Fire & Police Selection, Inc. (FPSI) © 2011
Appropriate Consequences for Not Passing the PAT
98%
32%
18%26%
CONDITIONING PROGRAM—THE INCUMBENT IS PLACED ON A PROGRAM THAT INCLUDES DIETARY MODIFICATION AND PHYSICAL TRAINING
LEAVE OF ABSENCE—THE DEPARTMENT MAY ELECT TO PLACE THE INCUMBENT ON A LEAVE OF ABSENCE UNTIL WHICH TIME THE INCUMBENT IS ABLE TO PASS THE TEST
DISABILITY LEAVE—THE DEPARTMENT MAY ELECT TO PLACE THE INCUMBENT ON DISABILITY LEAVE UNTIL WHICH TIME THE INCUMBENT IS ABLE TO PASS THE TEST
RETIREMENT WITH PENSION—THE DEPARTMENT MAY ELECT TO TERMINATE EMPLOYMENT WITH THE INCUMBENT FOLLOWING CONTINUED ATTEMPTS TO IMPROVE TEST PERFORMANCE WITHOUT SUCCESS
98%
32%
18%
26%
Fire & Police Selection, Inc. (FPSI) © 2011
The Legal Landscape—In Plain English
• A Fire Department cannot discriminate on the basis of race, gender, age, religion, national origin, disability, etc.
• Intent is not a required element of a discrimination claim.
• A testing program applied equally to all may be discriminatory if it screens out “too” many members of a protected class (has an adverse or disparate impact).
Fire & Police Selection, Inc. (FPSI) © 201116
Disparate or Adverse Impact
• An employer uses a test that is facially neutral, but has an unjustified adverse impact on members of a protected class by screening them out. ◦ Everyone is given the same test, but members of one or more
protected class do not fare as well as others.
• What are the triggers?◦ Men v. Women Pass v. Fail
◦ Statistical Significance
Fire & Police Selection, Inc. (FPSI) © 2011
Title VII of the Civil Rights Act
• Prohibits discrimination on the basis of gender in all aspects of the employment relationship.
– Uniform Guidelines on Employee Selection provide that physical testing of incumbents must be based on requirements in Section 14C or 14B
• Must show test is “job-related for the position in question and consistent with business necessity.”
Fire & Police Selection, Inc. (FPSI) © 2011
Adverse/Disparate Impact: Legal OverviewDISPARATE IMPACT = An unlawful employment practice based on
disparate impact is established only if:
and
the respondent fails to demonstrate that the challenged practice is job-related for the position in question and consistent with business necessityor
the complaining party makes the demonstration described above with respect to an alternate employment practice, and the respondent refuses to adopt such alternative employment practice.
A complaining party demonstrates that a respondent uses a particular employment practice that causes an adverse impact
18
Fire & Police Selection, Inc. (FPSI) © 2011
Current Legal Context: Adverse Impact Discrimination Flowchart
“or”Diff. in Rates?
YES NO
Is the PPTValid?
YES NO
Alternative Employment
Practice?
NODefendant Prevails
YESPlaintiff Prevails
END
Plaintiff Prevails
Practice,Procedure,Test (PPT)
PlaintiffBurden
DefenseBurden
PlaintiffBurden
How selection processes are challenged . . .
Fire & Police Selection, Inc. (FPSI) © 2011
The Typical Testing Case
• Establish adverse impact◦ Parties agree, or◦ Plaintiffs present evidence
• FD shows job-related and consistent with business necessity◦ Both sides offer expert testimony
• Plaintiffs show less restrictive alternatives available◦ Both sides offer expert testimony
Fire & Police Selection, Inc. (FPSI) © 2011
A Typical Testing Case
• Female firefighters prove disparate impact concerning use of physical exams◦ Tests mostly for anaerobic traits (men excel) and not for
aerobic traits (women excel)• FD justifies use of test
◦ Validated, as each portion of exam designed to test a representative firefighting task
• Females unable to show a less restrictive alternative
Fire & Police Selection, Inc. (FPSI) © 2011
Physical Ability Testing Standards forNew Hires
• The Uniform Guidelines on Employee Selection Procedures (1978) state:
◦ “Where cutoff scores are used, they should normally be set so as to be reasonable and consistent with normal expectations of acceptable proficiency within the work force. Where applicants are ranked on the basis of properly validated selection procedures and those applicants scoring below a higher cutoff score than appropriate in light of such expectations have little or no chance of being selected for employment, the higher cutoff score may be appropriate, but the degree of adverse impact should be considered.” (Section 5H)
Fire & Police Selection, Inc. (FPSI) © 2011
Physical Ability Testing Standards forNew Hires
• FPSI’s standard approach is to utilize the “Modified Angoff Method.”
◦ The Modified Angoff method is the application of the modified method that received acceptance before the United States Supreme Court in U.S. v. South Carolina (1971).
◦ The Angoff method produces an average estimate of minimum competency using opinions from several SMEs.
◦ The modification followed in U.S. v. South Carolina lowered the Angoff average estimate by one, two, or three standard errors of measurement (SEMs) after consideration of several statistical and human factors.
Fire & Police Selection, Inc. (FPSI) © 2011
Physical Ability Testing Standards forNew Hires
• A sample of incumbent fire suppression personnel take the physical ability test.
• Incumbents identify what a minimally qualified time should be for the physical ability test (i.e., “normal expectations of acceptable proficiency”).
• Average the mean opinion time and add one SEM to set the final cutoff score.
Fire & Police Selection, Inc. (FPSI) © 2011
Physical Ability Testing Standards for Incumbents (Firefighters and Fire Captains)
• Incumbent opinions are not used to set the cutoff score for maintenance standards due to potential bias:◦ FFs may tend to stretch the time limit when they know it will be used to
monitor their job performance (i.e., they tend to overestimate the time)
• Incumbent maintenance standard cutoffs are based upon the mean of the incumbent time in the “norming” process:◦ Outliers trimmed using a 1.645 SD
◦ Mean plus 1 Standard Error of Measurement + 1.645 Standard Errors of Difference
• The outcome is the “normal expectations of acceptable proficiency in the workforce”
Fire & Police Selection, Inc. (FPSI) © 2011
Sample Maintenance PAT Results
Step 1: Compute Mean
Mean (Average)Step 2: Trim “outliers”
(slowest 5% / fastest 5%) & compute new Mean
Step 3: Add a value to the Mean that adjusts for test
unreliability (SEM)
Step 4: Identify the slowest score that is still in the “Normally Acceptable”
range (using SED X 1.645)
RESULT: Typically 10% - 15% of incumbents will need
to “remediate” and retest after 10-16 weeks of
training
Fire & Police Selection, Inc. (FPSI) © 2011
Remediate Status of Incumbents
• A total of 6 officers clearly fell outside of the minimum proficiency levels relating to physical ability expectations (i.e., 95% confidence interval that these scores are reliably different from the average) and should be required to improve their abilities through possible dietary changes, weight-loss programs, and/or physical fitness programs.
• These incumbents fall into a “remediate” category and are asked to retake the test after 10-16 week training program. The 10-16 week training program should consist of both cardio-vascular and strength training in the specific, fire suppression-related work behaviors that are measured by the test.
• Departments can choose whether they want the training program to be self directed or conducted by a department-designated exercise specialist.
Fire & Police Selection, Inc. (FPSI) © 2011
Remediate Status of Incumbents
• The 10-16 week remediate process may result in any of the four possible consequences:
◦ Conditioning program—The incumbent is placed on a program that include dietary modification and physical training.
◦ Leave of absence—The department may elect to place the incumbent on a leave of absence until which time the incumbent is able to pass the test.
◦ Disability leave—The department may elect to place the incumbent on disability leave until which time the incumbent is able to pass the test.
◦ Retirement with pension—The department may elect to terminate employment with the incumbent following continued attempts to improve test performance without success.
Fire & Police Selection, Inc. (FPSI) © 2011
The Effect of Age on Test Performance
• Data from a study consisting of 256 incumbent fire suppression personnel (mean age = 34.83 years) resulted in a correlation of .397*** when the age of the incumbent was correlated to test time.
• Data from a study consisting of 710 firefighter applicants (mean age = 29.47 years) resulted in a correlation of .149*** when the age of the firefighter applicant was correlated to test time.
• While age was statistically correlated with physical ability test performance in both the incumbent and the applicant study, there are a number of other factors likely contributing to the correlation:
– The motivation level of the applicants vs. the incumbents
– The physical fitness level of the applicants vs. the incumbents
Fire & Police Selection, Inc. (FPSI) © 2011
Importance of “Maintenance/Wellness” Testing of Incumbents
• Ensuring Safety of the Crew ◦ Nearly 50% of all injuries to civilian firefighters in 2002 were a result
of sprains, strains, and muscular pain—whereby overexertion is considered the primary causative factor. (NFPA)
◦ Nearly 50% of firefighter fatalities are heart attacks and about half of those who died had heart-related issues. (NFPA)
◦ “Overweight, out-of-shape fire fighters are an accident waiting to happen.” (NFPA, 2000)
• Ensuring Safety of the Public
• Reduce Worker Compensation Claims◦ The estimated total annual cost of firefighter injuries is between $2.8 - $7.8
billion. (NIST, 2005)
Fire & Police Selection, Inc. (FPSI) © 2011
Questions???
• Contact us
◦ Dan Biddle, PhD., CEO/President Fire & Police Selection, [email protected] Phone: 916.294.4250 x. 113
◦ Stacy L. Bell, M.S., Executive Vice President Fire & Police Selection, [email protected] Phone: 916.294.4242 x. 245