what mathematics do people really use in the … · 1 what mathematics do people really use in the...
TRANSCRIPT
1
WHAT MATHEMATICS DO PEOPLE REALLY USE IN THE WORKPLACE?Merrilea J. Mayo, Lead Strategist
New Options, New Mexico
June 3, 2012
A W.K. Kellogg Foundation funded projectNewOptions NEW MEXICO
2
There Are Tests for Workplace Mathematics, Some Quite Good.
ACT has profiled over 16,000 jobs and designed this test to match the math needed in over 90% of those jobs.
3
Cognitive Skills Tests in General Are Much Better at Predicting Who Will Be “Successful on the Job” Than a Degree Does
Ability Composite = measurement of 3 cognitive skills, strongest predictor of on-the-job performance
WorkKeys Math test = 1 cognitive skill, little less strong predictor
NewOptions NEW MEXICO MEASURING WORK READINESS
4
Another Way of Looking at the Numbers
VARIATION IN INDIVIDUALS’ ON-THE-JOB PERFORMANCE
IS SUCCESSFULLY EXPLAINED BY:
These numbers are the prior validity numbers, squared.
NewOptions NEW MEXICO MEASURING WORK READINESS
Unknown 99%
Level of Education 1%
Unknown75%
Combined (3) Cognitive Ability
Tests, 25%
Math Skill Test Alone, 16%
5
Skills (Ability Composite) Tests Predict Performance on Complex Jobs Better than Simple Jobs
JOB COMPLEXITY
(as determined from DOL’s “data” dimension in its Dictionary
of Occupational Titles)
HOW WELL SKILLS TESTING PREDICTS PERFORMANCE
(RAW VALIDITY)
1 = perfect prediction of performance0 = zero ability to predict performance
High Complexity .58
Medium Complexity .51
Low Complexity .40
Unskilled .23
References: J.E. Hunter and R.F. Hunter, “Validity and Utility of Alternate Predictors of Job Performance,” Psych Bull 96, 72-981984
NewOptions NEW MEXICO MEASURING WORK READINESS
6
Literally over 1,000 studies, over 40 years and many job types.
Literature Supporting Validity of .3 to .6 for Cognitive Skill Tests’ Prediction of Job Performance
• M.M. Olea and M.J. Ree, J. Appl. Psych 79, 845-851 (1994).
• M.H. Ree, J.A. Earles and M. Teachout, J. Appl. Psych 79, 518-524 (1994).
• K. Brown, H. Le and F.L. Schmidt, Intl. J. Selection and Assessment 14, 87-100
(2006).
• Schmidt, Ones, and Hunter, Ann. Review Psych 43, 627-70 (1992).
• J.E. Hunter and R.F. Hunter, Psych Bull 96, 72-98 (1984) Also: a full decade of
papers by Hunter from 1980-1990, covering an enormous number of datasets.
• K. Pearlman, F.L. Schmidt and J.E. Hunter, J. Appl. Psych 65, 373-406 (1980).
• F.L. Schmidt, J.E. Hunter, and J.R. Caplan, J. Appl. Psych 66, 261-273 (1981).
• L.C. Northrup, Validity Generalization Results for Apprentice and Helper-Trainer
Positions. Washington, DC: U.S. Office of Personnel Management, Office of
Staffing Policy (1986).
• J.A. Hartigan and A.K. Wigdor, Fairness in Employment Testing: Validity
Generalization, Minority Issues, and the General Aptitude Test Battery.
Washington, DC: National Academy of Sciences (1989).
• E.E. Ghiselli, The Validity of Occupational Aptitude Tests. New York: Wiley, 1966.
(1966), Also: E.E. Ghiselli, Personnel Psych. 26 461-477. (1973).
• J.F. Salgado, H, Anderson, S. Moscoso, C. Berta and F. De Fruyt, Personnel Psych
56, 573-605 (2003).
• H.R. Hirsch, L.C. Norhtrup, and F.L. Schmidt, Personnel Psych 39, 399-420 (1986).
• M.H. Trattner, The Vlidity of Aptitude and Ability Tests for Semiprofessional
Occupations Using the Schmidt-Hnter Interactive Validity Generalization
Procedures. Washington, DC: U.S. Office of Personnel Management, Office of
Staffing Policy (1988).
NewOptions NEW MEXICO MEASURING WORK READINESS
7
EmployerPositions Filled Using WorkKeys
Outcomes(compared to prior practices)
Subaru of Indiana Automotive (Lafayette,
IN)
Automotive assembly workers
25% reduction in turnover.
Covidian Health Care (Macon, GA)
Manufacturing production
Per-person hiring cost reduced from $2,300 to $600 Time-to-hire reduced from 45 days to 17 days Time-to-full-employee efficiency reduced from 2 years to 6 months.
Steelscape(Calama, WA; Richmond & Rancho Cucamonga, CA)
Self-directed manufacturing teams
<4% turnover overall and a “Best Practice” award from the American Psychological Association for its hiring process.
Bradner Village, a retirement community (Marion, IN)
Nurses, nursing assistants, dietary staff, etc.
37% reduction in turnover.
Inova Health System(Falls Church, VA)
Customer assistants & clinical technicians
73% reduction in turnover; $1.8M cost savings.
911 call center(Lexington county, SC)
Call center operators50% reduction in turnover; skills-related turnover went to 0.
Real-World Results, When Employers Hire by Cognitive Skills Tests
NewOptions NEW MEXICO MEASURING WORK READINESS
8
So We Have A Test of Workplace Math that Accurately Predicts Workplace Performance.
How Is It Different from “School Math”?
NewOptions NEW MEXICO MEASURING WORK READINESS
9
Content is Different.
NOT INCLUDED
• Trigonometry, Pre-Calculus, Calculus. Algebra is not explicitly included
but is useful for problems involving more complex order-of-operations.
INCLUDED
• Addition, subtraction, multiplication, division (these operations for
negative numbers as well), fractions, decimals, percentages, mixed
numbers, unit conversions, averages & other simple statistics, ratios,
rates, order of operations, perimeters, areas, volumes
All Problems Defined Against Workplace Standards: Math through 6th grade only.
NewOptions NEW MEXICO MEASURING WORK READINESS
10
Context is Different.
• All problems are “word problems”
• All problems are highly plausible “on-the-job” situations
• None of the problems has explicit math
– No “find the common denominator.”
– No written formulas you are told to rearrange or solve.
– YOU must interpret the English in terms of math and YOU
must choose the correct math tools to solve the problem.
NewOptions NEW MEXICO MEASURING WORK READINESS
11
Scoring Scheme is Different
• School Math View: Problem A is harder than Problem B because A is
trigonometry (taught in 9th grade) and B is arithmetic (taught in
2nd grade)
• Workplace Math View: Problem A and Problem B are the same.
Both require simple memorization of a fact. There is no way to
solve either problem using reasoning or deduction. Thus, both
questions measure math skill at a sub-literate level.
30°
A. What is the sin of 30°?
2+2=?
B. What is the sum of 2+2?
12
• More and more extraneous information to sort through.
• More and more rearranging of information required to
get to the answer.
• More and more chained steps. Sequencing is important.
In sum: Workplace Math is “critical thinking,” as applied to math.
NewOptions NEW MEXICO MEASURING WORK READINESS
In Tests of Workplace Math Skill, Higher Level Questions Have the Following Features:
13
Applied Mathematics Level 3(Qualified for Entry Level Job)
NewOptions NEW MEXICO MEASURING WORK READINESS
14
Applied Mathematics Level 5(Qualified for Skilled Labor or College)
NewOptions NEW MEXICO MEASURING WORK READINESS
15
Applied Mathematics Level 7(Qualified for Professional Level Work)
NewOptions NEW MEXICO MEASURING WORK READINESS
16
If We Measure the U.S. Population Against U.S. Jobs in Terms of Workplace Math Requirements, What Do We Find?
NewOptions NEW MEXICO MEASURING WORK READINESS
17
WorkKeys APPLIED MATH
0
20
40
60
80
100
0
20
40
60
80
100
3 4 5 6 7
% U.S. Population W
ho
Can Do the Job
(is a
t re
quired s
kill le
vel or
hig
her)
% U.S. Jobs
Skill Level
(N=798,589)(N=4,298)
(8% job-illiterate math)
Data from ACT, http://act.org/workkeys/charts/reading.html, http://act.org/workkeys/charts/math.html
NewOptions NEW MEXICO MEASURING WORK READINESS
}
(Un)fortunately for the US, most US jobs don’t require much math.
18
However, some important jobs do. (list of occupations requiring AM Level 7)
NewOptions NEW MEXICO MEASURING WORK READINESS
19
But Why Can Only 6% of the US Population Score a Level 7 in 6th Grade Math?
• The ability to code a “real situation” into math, i.e., to
apply math, is almost never taught and completely
missing, not only in K-12 but also the graduate level. This
may also explain the US’ poor PISA results.
NewOptions NEW MEXICO Month Day, Year
20
y = 0.8826x + 65.719R² = 0.1049
60
65
70
75
80
85
90
95
7 8 9 10 11 12 13 14 15 16 17
WorkKeysScale Scores: (65-90)
Years of Education
The ability to Apply Math Increases Agonizingly Slowly Through School
Years of EducationN
Size
7th Grade 10
8th Grade 33
9th Grade 72
10th Grade 122
11th Grade 252
12th Grade/HS Graduate/GED 974
13 1st year Postsecondary 92
14 2nd year Postsecondary 128
15 3rd year Postsecondary 51
16 4th year Postsecondary 41
17 5th year or higher 34
TOTAL 1,809
>3
3
4
5
6
7
WorkKeys Skill Levels
Correlation= .32
Data from NMDWS; analysis by Tom Kilijanek
NewOptions NEW MEXICO MEASURING WORK READINESS
21
Targeted curricula can do in one year, what it takes school 6-10 years to accomplish.
Applied MathGain
Locating InfoGain
Reading for InfoGain
Required Keytrain(California data,Kilijanek analysis
5.8/yr(N=132; p<0.001)
5.54/yr(N=218;p<0.001)
8.2/yr(N=63;p<0.001)
Optional Keytrain(Dona Ana Community College data)
5.6/yr(N=38; p<0.1)
3.6/yr(N=38; not sig)
2.8/yr(N=38; not sig)
School (N≈1800, New Mexico)
0.88/yr 0.74/yr 0.85/yr
22
0
20
40
60
80
100
0
20
40
60
80
100
3 4 5 6
% U.S. Population W
ho Can
Do the Job
(is a
t re
quired s
kill le
vel or
hig
her)
% U.S. Jobs
Skill Level: Locating Information
(N=660,044)(N=4,910)
8% illiterate in this skill
Understanding Charts & DiagramsIs Actually a Bigger Problem than Applied Math
Data from ACT, http://act.org/workkeys/charts/locate.html
NewOptions NEW MEXICO MEASURING WORK READINESS
There is not enough “excess” high skill population for employers to demand this skill at high levels PLUS any other criterion (e.g., individual lives in their state, has a particular degree, is a U.S. citizen, has prior experience).
23
National Deficiency in Locating Information is a Major Contributor to the “STEM crisis.”
NewOptions NEW MEXICO MEASURING WORK READINESS
Occupations Requiring Locating Information Level 6.
24
The “Workplace Math” View of the World is So Different
WORKPLACE SKILLS VIEW
• You need to be able to apply math,
but only at the 6th grade level.* *some exceptions apply
• Pre- and post-testing of even
college algebra class show zero
gains in applied math skill.
• School degrees predict 1-10% of
work performance variation.
CONVENTIONAL WISDOM
• You need a lot of math to get a
good job.
• Taking advanced math courses
gives you the ability to apply
math.
• School degrees are a good first
cut to predict who will perform
on the job.
NewOptions NEW MEXICO MEASURING WORK READINESS
26
Proof: Employers are Desperate to
Hire Quants
Junior Quantitative Analyst, PHD Machine Learning or Signal Processing – Investment Bank NYC $250k
PHD from a well-regarded university, in Physics, Computer Science, Machine Learning, Signal Processing, Computer Vision, Statistics, Econometrics, Operations Research
PhD Quant Analyst (High performance Computing) – Fixed Income Research NYC $250k
PhD in a Quantitative/Scientific subject. PhD from a top tier University.
Quant Developer/ Algo Developer- Algorithmic Trading Firm- Chicago $175-
250K To qualify for this position, you will have a Bachelors, Masters or PhD in Computer
Science etc from a red brick University with a high GPA.
From one week’s job postings in Quantjobs.com
28
How Can This Be?
“All over the world, it has become fashionable for Universities and Colleges to offer Masters degree programs in quantitative finance or financial engineering (FE), a code word meaning the solution of the Black-Scholes option pricing differential equation in as many ways as possible.
“Unless the [academic] field re-invents itself pronto and starts becoming relevant to what people actually do out there, graduates with newly minted financial engineering degrees hoping to see a decent return on their own or their parents’ sizable investment will continue to be sorely disappointed by their actual career prospects, and will keep wondering where in God’s name they went wrong.”
-Sylvain Raynes, Founding Principal of R&R Consulting, a structured credit metrics consultancy
29
We need to teach students to see the
world in terms of math, or they will
be unemployable.
NewOptions NEW MEXICO MEASURING WORK READINESS
30
In New Mexico, “degree-based hiring” is being replaced by “skill-based hiring.”
Your score on the applied math test determines whether you get a job.
NewOptions NEW MEXICOMEASURING WORK
READINESS
I need someone who is a Level 5 in Listening,a Level 4 in Applied Math
That’s me!!! I have those scores!
33
NM Employer Benefits Similar to Those Seen Nationally
• Turnover goes down to <2%.
–Buckman Regional Water Treatment
Plant: 0% after 1 year.
• Time to full employee proficiency is
typically halved (less training time).
• Time and Cost to Hire typically halved
– Farmington Public Library: Hiring
costs went from $2,500 to $750,
5 days to 2.
NewOptions NEW MEXICO MEASURING WORK READINESS
34
We Took Jobless, High-Poverty, At-Risk Youth in New Mexico, Selected Only the Ones with High Skill Scores, and Gave them Directly to Employers
“He’s fantastic! We are so pleased with his work,
and we would not have had him in our employment
if it wasn’t for you and New Options.”
--Smart Motors
“We were unsure whether New Options NM
would be able to send us the right person amazing
pleasure for us to work with. We have trained her
on both Quickbooks and as a telephone receptionist.
Madai did not like math, but now understands that
math can actually be fun if it is in using Quickbooks.
We think this is a good opportunity for our company
to mentor a youth, and to understand there is a
different talent pool that exists to tap into.”
--Santa Fe Title Company
“Candance has just done a
wonderful job. We are very pleased
and would love to have her as a
full-time employee if a position
existed. Dina and she are working
on accessing positions that are
available in our district and we will
help her apply. She is well
prepared, knowledgeable and
through her training with you,
it is obvious that she is well
prepared to enter the work force.”
--Santa Fe Public SchoolsTraining? We didn’t train anyone! They were already this good.
NewOptions NEW MEXICO MEASURING WORK READINESS
35
More Employer Quotes…
“We enjoy her very much. She was a great hire
for the summer. We actually offered her a
permanent job, but she decided she was going
to attend UNM this fall. I do like WorkKeys
because it helped determine who was eligible
for the job and who could handle the work, so
that is a big plus for WorkKeys. To be honest,
we would’ve found someone else through our
ads, but I’m glad we used WorkKeys.”
--Vogue College of Cosmetology
“Danielle is very qualified for the job, is kind,
sweet and does what she is told in a timely
manner. We found her through the New
options NM WorkKeys project. She has
performed at all tasks given to her and we
would very much like a way to keep her on
when she goes back to high school in the fall.”
--First Tee of Santa Fe
“Without the program we would not
have had any contact with our seven
candidates for the job, and Marissa
was one of them. Marissa was the only
one who came in, asked questions,
and seemed generally interested in
the greenhouse and what we do. She
was the only one out of the seven that
was engaged during the interview
process. She has done everything she
has been asked to do and keeps the
greenhouse looking good. She shows
initiative in her work; getting started
and working with-out needing help or
supervision. She is very good and who
you hope to get in the workplace.”
--New Earth Orchid
NewOptions NEW MEXICO MEASURING WORK READINESS