questmatch - using natural language processing to improve speed and quality of employee recruiting

A Seedlink Tech White Paper

in Collaboration with International Top Talent

February 2014

www.seedlinktech.com

www.RCXUE.com

Using Natural Language Processing to Improve Speed

and Quality of Employee Recruiting

QuestMatch:

Table of Contents

Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Key China Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Key Research Findings. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Questions Are Key to Job Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Less Screening, More Interacting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

� $OORZ�4XDOLĆHG�&DQGLGDWHV�WR�6WDQG�2XW�. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Use Technology to do More with Less . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

QuestMatch . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

QuestMatch Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

1DWXUDO�/DQJXDJH�3URFHVVLQJ��1/3��WR�,QFUHDVH�6SHHG�DQG�(IĆFLHQF\�. . . . . . . . . . . . . . . . . . .

Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

3

4

5

6

6

7

7

7

8

9

10

11

12

Introduction

5HFUXLWLQJ�TXDOLĆHG�SHRSOH�TXLFNO\�LV�D�KXJH�FKDOOHQJH�HYHU\ZKHUH��but it is especially labor-intensive in emerging markets. With a

workforce of nearly 800 million, screening résumés in China is

extremely tedious and low-value. McKinsey reports that only 10%

RI�&KLQHVH�ZRUNHUV�DUH�ĆW�IRU�UROHV�DW�D�PXOWLQDWLRQDO�FRUSRUDWLRQ�(MNC). Seedlink case studies indicate

that sourcing just one suitable Chinese

candidate requires reviewing between

50 – 100 résumés or CVs. The

challenging recruitment environment

in China can offset the vast market

opportunities.

This paper covers three main topics. First, we will outline key

UHVHDUFK�ĆQGLQJV�IURP�D�ZLGH�UHYLHZ�RI�WKH�PRVW�SHUWLQHQW�recruiting literature. Then, we suggest a four-step process for

LPSURYLQJ�WKH�HIĆFDF\�DQG�GHSWK�RI�SUH�LQWHUYLHZ�DVVHVVPHQWV��Finally, we introduce the concept of Natural Language Processing

(NLP) and QuestMatch to improve speed, while managing the

entire pre-interview process.

With a combination of continued

process improvement and cutting-

edge technology, companies can

vastly improve the quality of recruits

while reducing overall cost. The

improvements can impact company

ĆQDQFLDOV�LQ�WZR�ZD\V��)LUVW��JUHDWHU�HIĆFLHQF\�UHGXFHV�RYHUDOO�UHFUXLWPHQW�FRVW�SHU�DFTXLVLWLRQ��Second, better matches reduce overall turnover rate and in turn,

off-boarding and replacement costs. RCXUE with QuestMatch

provides an intuitive interface to bring NLP to the most demanding

companies’ recruitment practices.

McKinsey reports that only 10% RI�&KLQHVH�ZRUNHUV�DUH�ĆW�IRU�UROHV�

at a multinational corporation

Better matches reduce overall turnover rate and thus off-

boarding and replacement costs

3

QuestMatch: Using Natural Language Processing to Improve Speed and Quality of Employee Recruiting

Key China Data

Figure 1:

ï� 790 million people in active labor force 1

ï� 7 million college graduates in 2013 2

ï� Nearly 200 million total college graduates by 2020 3

ï� ��RI�FDQGLGDWHV�DUH�XQĆW�IRU�ZRUN�DW�01&V�4

ï� 20% of CVs and résumés�KDYH�VLJQLĆFDQW�LQDFFXUDFLHV�5

ï� ��RI�HPSOR\HHV�VWD\�DW�D�ĆUP�IRU�OHVV�WKDQ��\HDUV 6

ï� 80% of turnover is due to bad hiring decisions 7

1 ê/DERU�)RUFH��7RWDO�ë�7KH�:RUOG�%DQN��ODVW�PRGLĆHG��KWWS��OLEJXLGHV�OLE�PVX�HGX�FLWHGDWD� 2 “College Graduation Data”, China Ministry of Education, 2013. 3 ibid 4 Diana Farrell and Andrew Grant, “Addressing China’s Looming Talent Shortage,” McKinsey Global Insights Research,

2FWREHU�� 5 ê%DFNJURXQG�6FUHHQLQJ��0DNLQJ�6RXQG�5HFUXLWPHQW�&KRLFHV�ë�.UROO�$VLD�6WXG\��KWWS��ZZZ�NUROOEDFNJURXQGVFUHHQ-

LQJ�FRP�QHZV�URRP�QHZV�DUWLFOHV�DSDF�DUWLFOH�EDFNJURXQG�VFUHHQLQJ�� 6 “Retention: Is It Getting Enough Attention,” Hays Research, 2012. 7 ê0D[LPL]LQJ�<RXU�5HWXUQ�2Q�3HRSOH�ë�+DUYDUG�%XVLQHVV�5HYLHZ��0DUFK��KWWS��KEU�RUJ��PD[LPL]LQJ�\RXU�UHWXUQ�RQ�SHRSOH�DU��

4


8 Fritzsche and Brannick, “The Importance of Representative Design in Judgment Tasks: The Case of Resume Screening,”

Journal of Occupational and Organizational Psychology 75, no.2 (June 2002): 163.9 Cole et al., “Recruiters’ Perceptions and Use of Applicant Resume Information: Screening the Recent Graduate,” Applied Psychology: An International Review 56, no.2 (April 2007): 319 – 343.10 James M. Tyler and Jennifer Dane McCullough, “Violating Prescriptive Stereotypes on Job Resumes: A Self-

Presentational Perspective,” Management Communication Quarterly 23, no. 2 (November 2009): 272-287.11 %\UQH��'RQQ��ê$Q�2YHUYLHZ��DQG�8QGHUYLHZ��RI�5HVHDUFK�DQG�7KHRU\�ZLWKLQ�WKH�$WWUDFWLRQ�3DUDGLJP�ë�Journal of Social and Personal Relationships 14, no.3 (June 1997): 417-431.12 $QDW��5DIDHOL��ê3UH�(PSOR\PHQW�6FUHHQLQJ�DQG�$SSOLFDQWVè�$WWLWXGHV�7RZDUG�DQ�(PSOR\PHQW�2SSRUWXQLW\�ë�Journal of Social Psychology 139, no. 6 (December 1999): 700-712.13 Richard D. Arvey et al., “Interview Validity for Selecting Sales Clerks,” Personnel Psychology 40 (March 1987): 1-12.14 2USHQ��&KULVWRSKHU��ê3DWWHUQHG�%HKDYLRU�'HVFULSWLRQ�,QWHUYLHZV�9HUVXV�8QVWUXFWXUHG�,QWHUYLHZV��$�&RPSDUDWLYH�Validity Study,” Journal of Applied Psychology 70, no.4 (November 1985): 774-776.15 *DU\�&��2OLSKDQW��.DWKDULQH�+DQVHQ�DQG�%HFN\�-��2OLSKDQW��ê3UHGLFWLYH�9DOLGLW\�RI�D�%HKDYLRUDO�,QWHUYLHZ�7HFKQLTXH�ë�Marketing Management Journal 18, no. 2 (Fall 2008): 93-105.

Key Research Findings

The Bad News

The Good News

ï� /RZ�DJUHHPHQW�LQ�&9�résumé screening criteria among

professional recruiters 8

ï� Experienced recruiters’ inference of hard skills and personality

WUDLWV�IURP�&9�résumés is not statistically valid 9

ï� Recruiters are subject to identity-image biases and recommend

FDQGLGDWHV�ZKR�UHćHFW�WKHLU�RZQ�êVHOI�LPDJHë�FULWHULD��UDWKHU�than company criteria 10, 11

ï� Candidates subjected to more thorough and longer screening

hold more favorable attitudes toward job openings 12

ï� Interviews based on job analysis are statistically valid predictors

of performance 13, 14

ï� Non-traditional interview techniques offer similar results to

traditional (face-to-face) interviews 15


5

Discussion

Research suggests that CV and résumé screening for prospective

employees is inaccurate and subject to bias. Applicants commonly

augment the truth; while up to 20% of résumés have gross

discrepancies.16 Stretched truths can become outright lies. In

��D�SURPLQHQW�6LOLFRQ�9DOOH\�WHFKQRORJ\�&(2�ZDV�UHPRYHG�due to CV inaccuracies. The problem is pervasive; particularly in

&KLQD�ZKHUH�SRRU�SXEOLF�UHFRUGV�DQG�KLJK�WXUQRYHU�OHDGV�WR�GLIĆFXOW�background checks.

Furthermore, people conducting

screenings are subject to biases based

on personal opinions. Studies have

shown that mood biases, gender biases,

and self-identity biases all contribute

WR�ĆQDO�UHFRPPHQGDWLRQV�17, 18 In fact,

experienced recruiters even disagree

about what criteria are essential to job

performance.19��*LYHQ�WKH�ORZ�HIĆFDF\�RI�UDQGRP�&9�VFUHHQLQJ��how does a large company know that it is getting the best workers

or just more of the same unreliable workers?

This white paper examines a four-part model for how candidate

screening can be vastly improved.

1. Questions are key to job analysis

Focusing job analysis on key issues employees face,

rather than a checklist of requirements, provides

WKH�PRVW�ćH[LELOLW\��2XU�EHVW�UHVXOWV�DUH�DFKLHYHG�through creating simple open-ended questions to

describe job functions. Instead of creating survey-

based assessments and wordy job descriptions, hiring

managers and recruiters focus on understanding

VSHFLĆF�VNLOOV�QHHGHG��7KH�UHVXOW�LV�PRUH�QXDQFHG�GDWD�that can be analyzed and assessed.

How does a large company know that it is getting the best

workers or just more of the same unreliable workers?

16 Kroll Asia Study, 2008. 17 Tyler and McCullough, 2009. 18 Byrne, 1997. 19 Fritzsche and Brannick, 2002.


6

2. Spend less time screening and more time interacting with candidates

Searching and screening candidates is slow and

passive. Instead, assess candidates from the outset

with questions created from job analysis. Sort

candidates into groups and actively engage groups

into a dialogue. With QuestMatch, recruiters can pose

questions to candidates digitally—before committing

time and effort to phone or face-to-face interviews.

��$OORZ�TXDOLĆHG�FDQGLGDWHV�WR�VWDQGRXW

CVs and résumés are static. People with ideas, skills

and thoughts are reduced to a paper representation.

&DQGLGDWHVè�PDLQ�FRPSODLQW�LV�WKH�GLIĆFXOW\�LQ�VWDQGLQJ�out amongst large numbers of applicants.20 Dynamic

DVVHVVPHQW�YDVWO\�LPSURYHV�D�ĆUPèV�XQGHUVWDQGLQJ�of recruits, while uncovering high-potential workers

whose credentials underrepresent their abilities.

RCXUE with QuestMatch automates this entire

process.

4. Use technology to do more with less

Interviews conducted via non-traditional methods

deliver results similar to face-to-face interviews. Web

applications, phone, email, and text chat are all viable

alternatives to assess potential employees. The next

era of recruitment technology will increase pre-

assessment interaction with candidates and automate

ranking.

20 International Top Talent (ITT) Research, Fall 2012.

7


QuestMatch

QuestMatch is a dynamic assessment software that allows

recruiters to interview groups of candidates digitally. Assessment

is based on open-ended questions rather than survey-based

assessment. This format allows greater variance of answers and

deeper granularity of results. Using cutting-edge Natural Language

Processing (NLP), the answers are aggregated for comparative

analysis, and candidates are automatically ranked.

Figure 2:

8


Figure 3:

QuestMatch Data

ï� Variance in ranking by QuestMatch is statistically similar to the

variance recorded between individual recruiters.

ï� Case studies reported greater than 70% correlation between

recruiters’ rankings and QuestMatch rankings.

ï� Results were even more accurate for best and worst answers

(up to 90% accuracy).

ï� 3URFHVV�LQFUHDVHG�VSHHG�RI�DVVHVVPHQWV�E\�XS�WR�ĆYH�WLPHV�compared to traditional HR practices.

9


Natural Language Processing (NLP) to Increase Speed

DQG�(IĆFLHQF\

1/3�LV�D�ĆHOG�WKDW�FRPELQHV�FRPSXWHU�VFLHQFH��OLQJXLVWLFV�DQG�DUWLĆFLDO�LQWHOOLJHQFH��2QH�RI�WKH�FHQWUDO�REMHFWLYHV�RI�WKLV�ĆHOG�LV�HQDEOLQJ�FRPSXWHUV�WR�XQGHUVWDQG�DQG�GHULYH�meaning from human language input. Progress over the last

decade allows modern algorithms to assess the sentiment

and feelings of people. Going forward there is opportunity

to utilize NLP to automate even more

WDVNV��0RVW�UHFHQWO\�(G;��D�QRQ�SURĆW�sponsored by Harvard and MIT, is using

computer programs to grade student

papers. The variance of computer-

generated results is nearly identical to

variance in human readers.21

QuestMatch allows recruiters to reduce time spent on

creating and reviewing assessments through automation

and answer abstraction. The result is more engagement

ZLWK�OHVV�WHGLXP��6WDIĆQJ�SURIHVVLRQDOV�DUH�DEOH�WR�IRFXV�RQ�strategy, interaction and coordination rather than searching,

sorting and screening.

As non-traditional interview results

are similar to face-to-face interviews,22

in-depth questions can be posed to

prospective candidates as a screening

procedure. QuestMatch then automates

the entire assessment process, saving

organizations time and money.

The variance of computer-generated results is nearly

identical to variance in human readers

Non-traditional interview results are similar to face-to-face

interviews

21 0DUNRII��-RKQ��ê(VVD\�*UDGLQJ�6RIWZDUH�2IIHUV�3URIHVVRUV�D�%UHDN�ë�The New York Times, April 4, 2013.

22 *DU\�&��2OLSKDQW��.DWKDULQH�+DQVHQ�DQG�%HFN\�-��2OLSKDQW��)DOO��

10


23 Cole et al., April 2007. 24 Fritzsche and Brannick, June 2002.

Conclusion

Résumé screening is an outdated recruitment paradigm. Prospective

candidates are reduced to words on paper, while recruiters’

inferences of skills from résumés are not statistically valid.23

Experienced recruiters even disagree on which criteria to judge

resumes.24 Furthermore, screening is tedious. The process is both

slow and inaccurate.

,QQRYDWLYH�ĆUPV�KDYH�RSSRUWXQLW\�WR�YDVWO\�LPSURYH�WKH�VSHHG�and results of candidate screening through process improvement

DQG�QHZ�WHFKQRORJ\�DGRSWLRQ��:H�SURSRVH�5&;8(�&20�ZLWK�QuestMatch as the new paradigm in candidate screening.

Recruiters can now digitally interview groups of candidates and

automatically assess responses with NLP (Natural Language

Processing) and machine learning. The result is savings in time,

money, and effort.

11


References

Anat, Rafaeli, “Pre-Employment Screening and Applicants’ Attitudes Toward an Employment

� 2SSRUWXQLW\�ë�Journal of Social Psychology 139, no. 6 (December 1999): 700-712.

Richard D. Arvey et al., “Interview Validity for Selecting Sales Clerks,” Personnel Psychology 40

(March 1987): 1-12.

ê%DFNJURXQG�6FUHHQLQJ��0DNLQJ�6RXQG�5HFUXLWPHQW�&KRLFHV�ë�.UROO�$VLD�6WXG\��KWWS��ZZZ�� NUROOEDFNJURXQGVFUHHQLQJ�FRP�QHZV�URRP�QHZV�DUWLFOHV�DSDF�DUWLFOH�EDFNJURXQG�� VFUHHQLQJ��

%\UQH��'RQQ��ê$Q�2YHUYLHZ��DQG�8QGHUYLHZ��RI�5HVHDUFK�DQG�7KHRU\�ZLWKLQ�WKH�$WWUDFWLRQ�� Paradigm,” Journal of Social and Personal Relationships 14, no.3 (June 1997): 417-431.

Cole et al., “Recruiters’ Perceptions and Use of Applicant Resume Information: Screening the

Recent Graduate,” Applied Psychology: An International Review 56, no.2 (April 2007): 319 –

343.

“College Graduation Data”, China Ministry of Education, 2013.

Diana Farrell and Andrew Grant, “Addressing China’s Looming Talent Shortage,” McKinsey Global

� ,QVLJKWV�5HVHDUFK��2FWREHU��

Fritzsche and Brannick, “The Importance of Representative Design in Judgment Tasks: The Case

of Resume Screening,” Journal of Occupational and Organizational Psychology 75, no.2 (June

2002): 163.

International Top Talent (ITT) Research, Fall 2012.

ê/DERU�)RUFH��7RWDO�ë�7KH�:RUOG�%DQN��ODVW�PRGLĆHG��KWWS��OLEJXLGHV�OLE�PVX�HGX�FLWHGDWD�

0DUNRII��-RKQ��ê(VVD\�*UDGLQJ�6RIWZDUH�2IIHUV�3URIHVVRUV�D�%UHDN�ë�The New York Times, April 4,

2013.

ê0D[LPL]LQJ�<RXU�5HWXUQ�2Q�3HRSOH�ë�+DUYDUG�%XVLQHVV�5HYLHZ��0DUFK��KWWS��KEU�� RUJ��PD[LPL]LQJ�\RXU�UHWXUQ�RQ�SHRSOH�DU��

*DU\�&��2OLSKDQW��.DWKDULQH�+DQVHQ�DQG�%HFN\�-��2OLSKDQW��ê3UHGLFWLYH�9DOLGLW\�RI�D�%HKDYLRUDO�� Interview Technique,” Marketing Management Journal 18, no. 2 (Fall 2008): 93-105.

2USHQ��&KULVWRSKHU��ê3DWWHUQHG�%HKDYLRU�'HVFULSWLRQ�,QWHUYLHZV�9HUVXV�8QVWUXFWXUHG�,QWHUYLHZV�� A Comparative Validity Study,” Journal of Applied Psychology 70, no.4 (November 1985):

774-776.

“Retention: Is It Getting Enough Attention,” Hays Research, 2012.

James M. Tyler and Jennifer Dane McCullough, “Violating Prescriptive Stereotypes on Job

Resumes: A Self-Presentational Perspective,” Management Communication Quarterly 23,

no. 2 (November 2009): 272-287.

12


&RS\ULJKW�k��6HHGOLQN�7HFKQRORJ\�+ROGLQJV��/WG��DQG�RU�LWV�DIĆOLDWHV��$OO�rights reserved. This document is provided for information purposes only and

the contents hereof are subject to change without notice. This document is not

warranted to be error-free, nor subject to any other warranties or conditions,

whether expressed orally or implied in law, including implied warranties and

FRQGLWLRQV�RI�PHUFKDQWDELOLW\�RU�ĆWQHVV�IRU�D�SDUWLFXODU�SXUSRVH��:H�VSHFLĆFDOO\�disclaim any liability with respect to this document and no contractual obligations

are formed either directly or indirectly by this document. This document may

not be reproduced or transmitted in any form or by any means, electronic or

mechanical, for any purpose, without our prior written permission.

Seedlink, QuestMatch, and RCXUE are trademarks of Seedlink Technology

+ROGLQJV��/WG��DQG�RU�LWV�DIĆOLDWHV��2WKHU�QDPHV�PD\�EH�WUDGHPDUNV�RI�WKHLU�respective owners.

QuestMatch: Using Natural

Language Processing to Improve

Speed and Quality of Employee

Recruiting

Februrary 2014

Seedlink Technology Holdings, Ltd.

700 Changping Road

Shanghai, China

200060

www.seedlinktech.com

www.RCXUE.com

questmatch - using natural language processing to improve speed and quality of employee recruiting

Documents