the development of a motivational distortion scale

THE D E V E L O P M E N T OF A MOTIVATIONAL DISTORTION SCALE FOR USE WITH THE COUNTERPRODUCTIVITY SCALE OF THE

CALIFORNIA PSYCHOLOGICAL INVENTORY

by

EE-LING N G

B. A. (Hons.), The University of Nottingham, 1997

A THESIS SUBMITTED IN PARTIAL F U L F I L M E N T OF THE REQUIREMENTS FOR THE DEGREE OF

M A S T E R OF ARTS

in

THE F A C U L T Y OF G R A D U A T E STUDIES

(Department of Psychology, Psychometrics Programme)

We accept this thesis as conforming to the required standard

THE UNIVERSITY OF BRITISH C O L U M B I A

August 2002

© Ee-Ling Ng, 2002

In presenting this thesis in partial fulfilment of the requirements

for an advanced degree at the University of British Columbia, I

agree that the Library shall make it freely available for reference

and study. I further agree that permission for extensive copying of

this thesis for scholarly purposes may be granted by the head of my

department or by his or her representatives. It is understood that

copying or publication of this thesis for financial gain shall not

be allowed without my written permission.

Department of Psychology

The University of British Columbia

Vancouver, Canada

Date XI / ^ ^ t C ^ -2-002

ii

Abstract

A sample of 250 undergraduates was administered the California Psychological Inventory (CPI),

Form 434, on two separate occasions, under two separate conditions—(a) in which they were to

respond honestly, and (b) where they were to respond as if being considered for a job they

valued. Four different indices were used to assess a subject's change in responses, between the

two conditions, on a CPI-based counterproductivity (CPI-Cp) scale. Using a composite of these

four indices as a criterion measure of motivational distortion, a 56-item CPI-based motivational

distortion scale (CPI-MD) was developed on the basis of item-total correlations with this

criterion. Internal consistency and test-retest reliability estimates of the CPI-MD scale ranged

between .72 and .86. Correlations with NEO PI-R domain and facet scale provided evidence of

construct validity. The CPI-MD scale was used to develop an adjustment equation to correct

CPI-Cp scale scores for motivational distortion. Adjusted CPI-Cp scores manifested somewhat

different correlations with other personality variables than did scores on the original CPI-Cp

scale, and an assessment of criterion-related validity for various job-related criteria in two

separate workplace samples showed significant increases, for the adjusted scale, in validity in

one sample.

iii

Table of Contents

Abstract i i

Table of Contents i i i

List of Tables v

Study 1 : Development of a Motivational Distortion Criterion Method . 7 Results 12

Study 2: Development of the Motivational Distortion Predictor Scale Samples 13 Method 14 Results 17 Summary of Results 22

Study 3: Development of the Motivational Distortion Predictor Scale Samples 22 Method 25 Results 28 Summary of Results 33

Overall Conclusions and Discussion 34

References 36

Appendix A Analyses to Determine the Effectiveness of the Experimental Procedure in Study 1 53

Appendix B Complete Administration Instructions for the Two Experimental Conditions in Study 1 57

Appendix C Additional Information about the Phi Correlation and Variance of the Differences Measures of Change (Based on Lautenschlager, 1986) 58

Appendix D Estimation of Reliability for the Four Change Measures and the Composite Motivational Distortion Criterion 60

Appendix E Correlations between the Motivational Distortion Criterion and the 20 Standard CPI Scales 62

Appendix F Two Alternative Methods of Deriving Adjustment Equations to Correct for Motivational Distortion in CPI-Cp Scores 63

Appendix G Correlations of Adjusted and Unadjusted CPI-Cp Scale Scores with Scores on the NEO PI-R Domain and Facet Scales 66

iv

Appendix H Criterion-Related Validity Analyses for Adjusted CPI-Cp Scores with Adjustments Made on the Basis of the CPI Fake-Good Scale 68

List of Tables

V

Table 1 Means, Standard Deviations, and Intercorrelation Matrix of the Four Measures of Change 42

Table 2 Factor Pattern and Factor Score Coefficient Vectors Associated with the First

Principal Component 43

Table 3 The 56 CPI Items Selected for the CPI-MD Scale 44

Table 4 Normative Data for the CPI-MD Scale 46

Table 5 Internal-Consistency Reliability Estimates and Standard Errors of Measurement for the CPI-MD Scale in Three Separate-Gender Samples 47

Table 6 Correlations of Scores on the CPI-MD Scale with Scores on the Five NEO PI-R Domain Scales and the NEO PI-R Neuroticism, Extraversion, and Conscientiousness Facet Scales 48

Table 7 Descriptive Statistics and Correlation of the CPI-MD Scale and M D Criterion Scores 1 49

Table 8 Comparison of Means, Standard Deviations, and Correlations between Honest-Condition/Unadjusted and Adjusted CPI-Cp Scores in Three Separate-Gender Samples 1 50

Table 9 Correlations of Adjusted and Unadjusted CPI-Cp Scale Scores with NEO PI-R Domain and Conscientiousness Facet Scale Scores 51

Table 10 Correlations between the CPI-Cp Scale Scores (With and Without Adjustments for Motivational-Distortion) and Counterproductive-Behavior Criteria in Two Non-Management Level Samples 52

Table A l Means and Standard Deviations of the CPI-Cp Scores in the Employment and Honest Conditions of the Experimental Sample 55

Table A2 Descriptive Statistics of the CPI-Cp Scores in the Experimental Sample and a Normative Undergraduate Student Sample 56

Table F l Means and Standard Deviations of the Employment-Condition CPI-Cp and CPI-M D Scale Scores 65

Table HI Correlations between the CPI-Cp Scale Scores (With and Without Two Different Adjustments for Motivational-Distortion) and Counterproductive Behavior Criteria in Two Non-Management Level Samples 70

1

Introduction

With growing evidence that certain relevant personality constructs have substantial

predictive validity for various job-related criteria, there has been an increasing interest in the use

of personality inventories for personnel selection purposes (Barrick & Mount, 1991; Hough,

Eaton, Dunnette, Kamp, & McCloy, 1990; Hough & Schneider, 1996; Ones, Viswesvaran &

Schmidt, 1993). However, this increased use has also been met by concerns about the

susceptibility of personality measures to motivational distortion. Motivational distortion (MD)

can generally be defined as an altering of one's responses on a test because of the circumstances

under which he/she is taking it. In a job application context, motivational distortion can be

considered faking-good or presenting an artificially favorable impression of oneself in order to

increase one's chances of getting the job.

The purpose of this study was to develop a measure of motivational distortion for the

California Psychological Inventory (CPI), which could be used as a basis for adjusting

personality scale scores in order to obtain more honest score estimates for hiring purposes. The

relevance of this research is predicated on the negative impact of M D on personality testing for

personnel selection. Hence, a summary of the research evidence pertaining to the occurrence and

impact of M D in personnel selection will be presented. This is followed by a brief review on how

the validity scales included within existing personality inventories can be used to adjust content

scale scores for M D . Finally, as this study focuses on making M D corrections for a CPI-based

measure of counterproductivity (CPI-Cp scale; Hakstian, Farrell, & Tweed, 2002), a brief

description of the CPI-Cp scale will be provided.

Does Motivational Distortion Exist?

Research findings have clearly established that subjects can distort their responses on

non-cognitive measures in the desired direction, when instructed to do so (e.g., Alliger &

Dwight, 2000; Douglas, McDaniel, & Snell, 1996; Hough et al., 1990; McFarland, 2000;

2

McFarland & Ryan, 2000; Ones & Viswesvaran, 1998a; Rosse, Stecher, Miller, & Levin, 1998).

Viswesvaran and Ones (1999) performed a meta-analysis to quantify the magnitude of distortion

and found that on both personality and integrity tests, subjects who had been instructed to fake-

good increased their scores by about half a standard deviation on average.

In general, the main criticism that has been leveled against research studies on

motivational distortion in personnel testing is that while such studies have demonstrated that

applicants can "fake" if they choose to do so, they do not provide evidence that applicants do

"fake" in reality. However, a survey of the research literature yielded several recent studies that

provide evidence of distortion in real applicant subjects (e.g., Barrick & Mount, 1996; Becker &

Colquitt, 1992; Ellingson, Sackett, & Hough, 1999; Rosse et al., 1998). The applicant-based

motivational distortion studies cited here involved either personality test measures or biodata

inventories. It appears that more research using applicant subjects is needed on integrity tests, as

existing research studies have tended to be laboratory-based studies in which employment

conditions were experimentally simulated (e.g., Cunningham, Wong, & Barbee, 1994; Lobello &

Sims, 1990; Ryan & Sackett, 1987).

The Impact of Motivational Distortion on Validity in Personnel Selection

Research studies on the impact of motivational distortion in a personnel selection context

have led to conflicting conclusions. In general, studies support the fact that distortion results in

an over-representation of "fakers" in the top end of the score distribution, who are therefore more

likely to be hired in a top-down selection process (Christiansen, Goffin, Johnston, & Rothstein,

1994; Hough, 1998; Rosse et al., 1998). However, while M D can affect selection decisions, the

preponderance of research evidence seems to indicate that it does not degrade criterion-related

validity (Christiansen et al., 1994; Hough et al., 1990; Ones & Viswesvaran, 1998a, 1998b;

Ones, Viswesvaran, & Schmidt, 1993; Ones, Viswesvaran, & Reiss, 1996; Zickar, Ross, &

Levin, 1996). In contrast, only a few individual studies have provided evidence which

3

empirically supports the argument that faking degrades criterion-related validities, and these

were laboratory studies where faking was computer-simulated or experimentally induced (e.g.,

Douglas, McDaniel, & Snell, 1996; McDaniel, Douglas, & Snell, 1997; Zickar, 1997).

There have been attempts to provide some explanation for the discrepant findings,

within the research literature, regarding the impact of M D on criterion-related validity. These

explanations include (a) differences in the extent to which faking can be assumed to occur across

the different samples utilized in various studies—e.g., student samples, job incumbents, and job

applicants—(Hough, 1998); (b) different methods of operationalizing and conceptualizing faking

(e.g., Kluger & Collela, 1993; McFarland, 2000; McFarland & Ryan, 2000; Paulhus, 1991); and

(c) methodological limitations of using correlational analyses to represent the effects of faking

(e.g., Alliger & Dwight, 2000; Rosse et al., 1998).

Research findings on the effects of motivational distortion on the construct validity of

non-cognitive test measures have also been equivocal. On the one hand, studies have found

significant correlations between these test measures and social desirability scores (Cunningham,

Wong, & Barbee, 1994; Guastello & Rieke, 1991; Ones, Viswesvaran, & Reiss, 1996; Rosse et

al., 1998), and factor analytic studies have provided evidence that motivational distortion

damages the structural integrity of personality measures in applicant samples (Barrick & Mount,

1996; Cellar, Miller, Doverspike, & Klawsky,1996; Douglas, McDaniel, & Snell, 1996; Livneh

& Livneh, 1989; Schmidt & Ryan, 1993; Zickar & Robie, 1999). On the other hand, other

studies have demonstrated factorial invariance in faking samples (Collins & Glese, 1998;

Ellingson, Smith, & Sackett, 2001; Smith, Hanges, & Dickson, 2001; Ones & Viswesvaran,

1998a).

Overall, it would appear that Ones, Viswesvaran, and Schmidt's (1993) dismissal of

social desirability as a "red herring" is premature. Despite existing evidence of the contrary,

there is also sufficient evidence to suggest that faking can have a negative impact on selection

4 decisions, and criterion-related and construct validity, and therefore remains a legitimate concern

that needs to be addressed with further research.

Correcting Personality Scale Scores for Motivational Distortion

Because o f concerns about motivational distortion as a contaminant, many established

personality inventories such as the M M P I (Dahlstrom, Welsh, & Dahlstrom, 1972, 1975),

Sixteen Personality Factor Questionnaire (Winder, O'Dell, & Karson, 1975), Personality

Research Form (Jackson, 1967), Hogan's Personality Inventory (Hogan, 1986; Hogan & Hogan,

1972), and California Psychological Inventory (CPI; Gough, 1957, 1987; Gough & Bradley,

1996), include validity scales to detect various aspects o f motivational distortion. These validity

or M D scales are generally either special-purpose scales or are linear combinations o f personality

content scales.

One way in which these M D scales have been used to address motivational distortion in

personality testing is to use these scale scores as a basis for correcting personality content scale

scores for the effects o f distortion. This correction is effected by regressing the personality scale

scores on the M D scale score, with the residual representing the personality scale score adjusted

for distortion plus error.

As is typical with much o f the research on faking, there is also disagreement regarding the

use o f score corrections to address the influence o f M D . The main argument against the use o f

score corrections stem from meta-analytic evidence demonstrating that M D (broadly

conceptualized as social desirability) does not degrade criterion-related validity. Others have

suggested that social desirability can be job-related and therefore score corrections could

inadvertently partial out substantive or content-related variance (McCrae & Costa, 1983;

Nicholson & Hogan, 1990; Zerbe & Paulhus, 1987). Moreover, studies have shown that

corrected scores do not approximate honest scores (e.g., Ellingson, Sackett, & Hough, 1999;

Weiner & Gibson, 2000).

5

Despite the work cited above, some studies have demonstrated that using corrected

personality scale scores resulted in changes to the rank ordering of candidates which would

consequently affect top-down hiring decisions (Christiansen et al., 1994; Hough, 1998; Rosse et

al., 1998). Other researchers who support the use of corrected scores have put forward the

reasoning that M D is a confounding variable, and, in the interest of construct validity, controlling

the variance of M D in personality scale scores would lead to a more precise and valid

measurement of the intended construct of interest (e.g., Helmes, 2000).

The California Psychological Inventory (CPI) and its Validity Scales

The California Psychological has been widely used and researched in applied contexts,

including that of personnel selection, and has an established ability to predict externally-

measured job-related or organizational criteria. The current version of the CPI consists of 434

items measuring 20 standard scales and 13 special-purposes scales. Of the 20 standard scales,

three scales were originally constructed to measure motivational distortion and detect invalid

protocols. The Wb (Well-being) scale assessed tendencies to fake-bad; the Gi (Good Impression)

scale assessed tendencies to fake-good, and the Cm (Communality) scale assessed random

responding.

It was found that by optimally combining these three validity scales, along with other

standard CPI scales, prediction equations could be constructed to provide a more precise

classification of invalid protocols (Lanning, 1989). In the current edition of the CPI manual

(Gough & Bradley, 1996), the fake-good protocol validity equation for CPI Form 434 is a linear

combination of five separate CPI standard scales: Dominance, Empathy, Good Impression, Well-

being, and Flexibility.

The CPI-Counterproductivity (CPI-Cp) Scale

Hakstian et al. (2002) developed a CPI-Counterproductivity scale (CPI-Cp) designed to

measure tendencies to engage in a broad range of counterproductive behaviors, that is, behaviors

6

that were detrimental to the objectives of the organization such as property theft, disruptiveness,

absenteeism, tardiness, and withholding effort. On the basis of four student samples and three

workplace samples, the CPI-Cp scale was shown to possess high reliability and validity. The

decision to focus on developing motivational-distortion-adjustments for the CPI-Cp scale in this

study was based on the fact that this scale was likely to be the most susceptible to motivational

distortion in a personnel selection context.

Purposes of the Present Research

The twofold purpose of the present research was (a) to develop a measure of

motivational distortion tendencies, by means of items from the CPI, (referred to, in what follows,

as the CPI-MD scale), and (b) to use this scale as a basis for developing a means of correcting

CPI-Cp scale scores for motivational distortion. Unlike the existing CPI fake-good scale

described in the CPI manual (which will be referred to as CPI-FG in what follows) that measures

motivational distortion tendencies on the basis of five standard CPI scales, the proposed CPI-MD

scale avoids introducing extraneous variance by measuring M D on the basis of only those CPI

items that are empirically related to motivational distortion. In this manner, such a scale is

expected to be an improvement upon the existing fake-good scale, as it would offer a more

effective and precise measurement of a respondent's tendency to distort his/her responses on the

CPI when the inventory is taken under personnel selection conditions.

The research was conducted in three separate but related studies: (a) development of the

motivational distortion criterion (Study 1); (b) development of the motivational distortion

predictor scale (Study 2); and (c) development of the adjustment equation for CPI-Cp scores and

an evaluation of the psychometric impact of using adjusted scores on the CPI-Cp scale (Study 3).

7

Study 1: Development of a Motivational Distortion Criterion

Research Aims

The aim of this study was to develop a reliable measure of motivational distortion from

four separate measures of change that had previously been used to index motivational distortion

(Gordon & Gross, 1987; Lautenschlager, 1986). The purpose of this measure was for use as a

criterion for the development of the motivational distortion predictor scale in the next study.

Method

Sample

A total of 250 subjects, consisting of 55 male and 195 female undergraduate students at

the University of British Columbia (UBC), took the CPI Form 434 on two separate occasions

(one in an employment condition and one in an honest condition), with a time interval of about

5-7 days between administrations. Subjects were administered the CPI in small-group settings of

about 3-5, in a testing room or classroom at the university. These student subjects participated in

the experiment for course credit, and were also provided with feedback on their personality

profile.

Administration Procedures

A repeated measures experimental design was used in which all subjects were

administered the CPI under the employment condition first, followed by the honest condition, a

week later. Although previous research had indicated an order effect with repeated measures

designs (e.g., Ellingson, Sackett, & Hough, 1999; Hough et al., 1990), standardizing the

experimental order for all subjects was preferred because it allowed the employment condition to

more closely simulate the usual applicant setting where applicants would not have had prior

exposure to the personality inventory. The concerns with regard to the contamination of honest-

condition data as a result of a practice effect were addressed in some separate additional

8

analyses. The results suggested that this was not the case, as the honest condition scores were

representative of normative data for undergraduate students (see Appendix A).

The significant part of the honest- and employment-condition instructions that were read

out to subjects is as follows (the complete set of instructions can be found in Appendix B):

Honest: Please respond to the statements as you honestly see yourself to be.

Employment: Imagine that you are applying for a job that you would like to have.

You have been requested to complete this personality inventory as part of the

selection process. Your responses on the inventory will provide valuable

information about yourself that will be used as part of the decision-making

process. Please respond to the items as you would i f you were applying for this

job.

To prevent subjects from lapsing into honest responding during the employment

condition, subjects were reminded of the instructions (i.e., to complete the inventory as though

they were applying for a job), and visual reminders of the instructions (in an abridged form) were

also placed on every page of the inventory. Honest responding in the normal condition was

encouraged both with guarantees of confidentiality and with an incentive that subjects would be

provided feedback on their personality profile based on their responses in this condition.

Previous studies on motivational distortion have tended to instruct participants to

deliberately create a favorable impression of themselves when completing the test measures.

While such instructions successfully induce faking, they poorly simulate real-applicant settings

because the natural deterrents to motivational distortion (e.g., fear of detection) that occur in

such settings are eliminated and real individual differences in motivational distortion are

suppressed. As a result, observable motivational-distortion variance is reduced and the ability of

such empirical studies to detect the effects of distortion is likewise minimized (see Douglas,

McDaniel, & Snell, 1999; McFarland, 2000; Zickar, 1997). In contrast, the instructions

9

suggested here for the employment condition were believed to be a closer simulation of a

personnel selection context and would consequently be more conducive to capturing the

distribution of motivational distortion scores that can be assumed to exist in an employment

setting. However, concerns with regard to the potency of the present employment-condition

instructions in inducing motivational distortion were addressed in an adjunct analysis (see

Appendix A).

Operationalization of the Motivational Distortion Construct

Motivational distortion was operationalized as a within-subject measure of change

between a condition that provided motivation for distortion (the employment condition) and one

that did not (the honest condition). As a within-subject measure, each subject acted as his/her

own control and therefore any systematic changes in responses between one condition and the

next, could be construed as intentional distortion.

Four separate measures of change were used to assess the degree to which subjects'

responses to the CPI -Cp scale varied from one condition to the other. Taken together, these

measures were able to assess both overall and item-level shifts in responses to the CPI -Cp scale

(Gordon & Gross, 1978; Lautenschlager, 1986). For measures 1, 2 and 4 below, larger values

represent more distortion, whereas for measure 3, a larger value indicated less distortion. As

measures 3 and 4 are conceptually more complicated, additional information is provided in

Appendix C for further explanations on the computation of these two measures. A brief

description of the four measures is provided here:

(1) Raw-score difference: The difference in the CPI -Cp scale scores obtained in the employment

condition (CPI-CpemPioyment) and the honest condition (CPI-Cphonest)- The difference was

obtained by: CPI-Cphonest scores - CPI-CpemPioyment scores.

(2) Residual change scores: The residual about the regression line from regressing the C P I -

Cphonest on CPI-Cpempioyment scale. That is, the residual change score was obtained by: C P I -

10

Cphonest - CPI-Cphonest, where CPI-Cphonest is the predicted CPI-Cphonest score from the C P I -

Cpemployment SCOre.

(3) Phi correlation: The within-subject correlation coefficient which represented, for each

subject, the consistency of true-false responses on the 80 CPI-Cp items between the

employment and honest conditions.

(4) Variance of the differences : The within-subject variance of the differences in the responses

to each of the 80 CPI -Cp items between the two experimental conditions. (The data were

coded '1 ' for true and '2' for false.)

As noted, more detail on measures 3 and 4 appear in Appendix C.

Measurement and Analysis of the Motivational Distortion Criterion

Gender differences were tested for all four measures of change and significant gender

mean differences were found on the Raw-score Difference measure with females obtaining

higher mean scores than males, <248) = 233, p < .025, A = .352, .95 C.I. for A = (.054, .651).

For the other three measures, gender differences were not significant (p > .25). Therefore, scores

on the Raw-score Difference measure were mean-deviated within gender before being pooled for

the correlational analyses to follow. The separate-gender descriptive statistics and the pooled-

gender intercorrelation of these four measures appear in Table 1.

Insert Table 1 about here

The four measures were highly and uniformly correlated and this suggested a common

underlying construct. To maximize reliability, a composite of the four measures was used as the

final motivational distortion criterion (MD criterion) and the weighting of variables for this

composite was determined on the basis of first principal component scores.

The weight vector for obtaining the first principal component was derived from the

principal component analysis of the 4 x 4 intercorrelation matrix found in Table 1. As noted

11

earlier, this intercorrelation matrix was based on scores that had been mean-deviated, within

gender, on the Raw-score Difference measure. However, to retain existing gender differences in

the resulting criterion scores, the weight vector obtained from the principal component analysis

was applied to the non-mean-deviated z-scores of all four measures and then standardized to the

metric of the Raw-score Difference measure, to yield the appropriate scores for adjustment later.

Reliability Estimates of the Four Measures of Change and the MD Criterion

Reliability estimates were obtained for each of the four change measures. The reliability

estimates for both the Raw-score Difference and Residual Change Scores were computed using

equations developed by Williams, Zimmerman and Magazzati (1987). Deriving reliability

estimates for the Phi Correlation and Variance of the Differences measures presented some

complications because these were single-value, computationally-derived measures. It was

decided that by splitting the 80-item CPI-Cp scale into two equal subscales comprising the first

40 items and the last 40 items, two separate estimates (one for each subscale) of both the Phi

Correlation and Variance of the Differences measures could be obtained for each subject. The

correlation between these two estimates across all subjects would therefore represent a form of

split-half reliability. Finally, using standard equations for the reliability of a differentially-

weighted composite (see, e.g., Horst, 1966, pp.280-281), a reliability estimate for the M D

criterion (the principal component scores) was also obtained. Further elaboration on the

computation of these reliability estimates appear in Appendix D.

Additional Analysis of the Motivational Distortion Criterion Measure

The motivational distortion criterion measure was correlated with the 20 standard CPI

scales. This analysis was undertaken to determine, as far as possible, the conceptual meaning of

the M D criterion measure. Because it is not central to the purposes of the present study, results of

this analysis appear in Appendix E.

12

Results

As mentioned earlier, the high level of intercorrelation amongst the four measures of

change indicated one single underlying factor (see Table 1). Principal component analysis of the

4 x 4 R matrix revealed one eigenvalue greater than 1.0 and this factor accounted for 89% of the

variance. The factor pattern vector and factor score coefficient vector (the weight vector for

obtaining the standardized first principal component) appear in Table 2.


From the reliability analyses, it was found that the average reliability estimate of the four

separate measures of change was .76. (The specific' reliability estimates for each of the four

measures appear along the principal diagonal of the intercorrelation matrix in Table 1.) The

overall internal consistency reliability estimate of the M D criterion was .93.

Overall, the results show that the four measures of change were individually reliable and

highly correlated, and that a composite consisting of the first principal component of the four

measures would provide a more reliable measure of motivational distortion. This then constituted

the final M D criterion.

Study 2: Development of the Motivational Distortion Predictor Scale

Research Aims

In the previous study, a motivational distortion measure was developed under

experimental conditions in which both distorted and honest responses were obtained from the

same subject. Consequently, this measure of motivational distortion cannot be used in applied

settings because it is dependent on the availability of both distorted and honest data from the

same individual. In a hiring context, only one set of responses would be available and these

responses would likely have been contaminated by motivational distortion. Therefore, the aim of

13

this study was to develop a CPI-based motivational distortion predictor scale (CPI-MD scale)

that could be used to estimate the amount of distortion within a set of CPI scores obtained under

employment conditions.

Samples

A total of six distinct samples were employed in this study. Besides the UBC Sample

from Study 1 (named U B C Sample 4 in what follows), five additional samples from the Hakstian

et al. (2002) study were used. The total number of subjects for each of these additional samples

is presented below, along with the analyses for which the sample was used in this study. The

exact ns for each of the analyses to follow are provided later with the results.

1. UBC Sample 1—A total of 1,083 subjects consisting of 350 male and 733 female

U B C undergraduate students who were administered the CPI during the years 1988-1995. This

sample was used to develop norms, obtain internal-consistency reliability estimates, and assess

the construct validity of the CPI-MD scale.

2. UBC Sample 2—A total of 229 subjects, consisting of 71 male and 158 female U B C

undergraduate students who were administered the CPI during the period 1998-1999. These

subjects completed the CPI at two separate occasions with a time interval of 2-3 weeks. This

sample was used to derive test-retest reliability estimates of the predictor scale (CPI-MD scale)

and the CPI-FG scale.

3. UBC Sample 3—A total of 320 subjects consisting of 98 male and 222 female U B C

undergraduate students. Subjects in this sample were administered both the CPI and NEO PI-R

(Costa & McCrae, 1992) during the period 1998-2000. The construct validity of the CPI-MD

scale was assessed with this sample, using the Five Factor Model framework represented by the

NEO PI-R.

4. UBC Sample 4—A total of 250 subjects consisting of 55 male and 195 female UBC

undergraduate students. This sample was first used in Study 1 for the development of the M D

14

criterion measure, and in this study, only the employment condition data were used for the

development of the CPI-MD scale. Various other correlational analyses, conducted to compare

the CPI-MD scale with the CPI-FG scale, were also based on this sample. (The sample

characteristics can be found in the methods section of Study 1.)

In the above four U B C samples, the age range was narrow, with a median age of around

20. Subjects were of mixed race, with slightly more than half Caucasian and the rest mainly

Asian and Indo-Canadian. These student subjects were all administered the CPI in small group

settings of about 3 or 4 to 45-50, in a classroom or testing room at UBC.

5. Management Employee Sample—A total of 457 subjects consisting of 245 male and

212 female management employees of two large Canadian telecommunications companies.

Subjects ranged in ages of between 22 and 61 with a median of around 40 for males and 37 for

females. They were mainly first- and second-level management personnel (from all company

departments) and were administered the CPI, between the mid- to late-1990's, as part of more

comprehensive, on-going assessment-center research studies. This sample was used to provide

norms and to estimate internal consistency reliability for the CPI-MD scale.

6. Non-Management Job-Applicant Sample—A total of 528 subjects consisting of 193

male and 335 female applicants for a variety of non-management positions in a call center in

Vancouver, British Columbia. These subjects were administered the CPI, between 1995-1999, as

part of a longer test battery that included cognitive-ability tests and other assessment methods.

This sample was used to provide norms and to estimate internal consistency reliability for the

CPI-MD scale.

Method

Selection of Items

Using what can be considered a purely empirical methodology, the CPI-MD scale items

were selected from the CPI (Form 434) on the basis of their correlations with the M D criterion

15

measure developed in Study 1. For this item-selection process, the employment-condition CPI

data were correlated with the M D criterion using UBC Sample 4.

As there were no significant gender differences on the M D criterion scores (p > . 10), the

correlational analyses were performed with a pooled-gender data sample. Given the potential for

inflation due to capitalization on chance in these analyses, only those items that correlated with

the M D criterion scores at or beyond the .01 level of significance (nondirectional) were

considered. This standard guaranteed that, under ideal conditions, not more than 4 or 5 items

would be selected because of Type I error.

By this process, 56 items were selected for the CPI-MD scale; these are presented in

Table 3. The scoring procedure is to assign one point for each item under the "True" heading

responded to with a "true" response and one point for each "False" item responded to with a

"false". Thus, CPI-MD scores can potentially range from 0 to 56 with higher scores reflecting

greater tendencies for motivational distortion.


The distribution of CPI-MD scale scores was examined under the two conditions—

employment and honest—for the experimental sample (UBC Sample 4). In the honest condition,

the distribution was unimodal and very close to symmetric (summary statistics for the CPI-MD

scale under the honest condition appear in Appendix F, Table F l ) . The distribution of the CPI-

M D scale scores under the employment condition, however, was unimodal but slightly

negatively skewed (sample estimate of the index of skewness was -.20).

Establishment of Norms for the CPI-MD Scale and Analysis of Gender Differences

Three different kinds of normative data for the CPI-MD scale were derived in the

present study: U B C Sample 1 provided the normative distribution for an undergraduate student

population, whilst the management employee and non-management job-applicant sample

16

(Samples 5 and 6 above, respectively) provided two separate sets of industrial norms. In all three

samples, separate-gender norms were provided and analyses of gender mean differences on the

CPI-MD scale were performed. These were statistically powerful analyses due to the large

sample sizes of the gender-groups.

Estimation of Internal-Consistency and Test-Retest Reliability

The three separate-gender normative data samples above were also used to estimate the

internal-consistency reliability (Cronbach's alpha) of the CPI-MD scale. In addition, using

subjects from U B C Sample 2, separate-gender estimates of the short-term stability of the CPI-

M D scale were also obtained using test-retest correlations between scale scores obtained at two

time points (with a 2-3 week interval between administrations).

Estimation of Construct Validity

The Five Factor Model was used as a framework for examining the psychological

underpinnings of the motivational distortion construct measured by the CPI-MD scale. Using

U B C Sample 3, separate-gender correlations were obtained between CPI-MD scores and the five

NEO PI-R domain scales (reflecting the Big Five personality factors), that is, Neuroticism,

Extraversion, Openness, Agreeableness, and Conscientiousness. In addition, where these

correlations were found to be statistically significant, correlations between CPI-MD scores and

the facet scales associated with those particular domain scales were also obtained.

Correlation between the CPI-MD and CPI-Cp scales

To provide an estimate of the extent to which the CPI-Cp scale scores were associated

with CPI-MD scores, these were correlated, using the employment-condition data from U B C

Sample 4. Because of significant gender differences on these two scales (p < .01 for both), scale

scores were mean-deviated by gender before being pooled for the correlational analysis.

17

Estimation of Criterion-Related Validity

As an estimate of the criterion-related validity of the CPI-MD scale, scores on this 56-

item scale were correlated with the motivational distortion criterion measure (MD criterion)

developed in Study 1, using data from U B C Sample 4. As noted earlier, because of significant

gender differences on the CPI-MD scale, scores on this scale were mean-deviated by gender

before correlation with the M D criterion. (No gender differences were found on the M D criterion

as reported earlier.) Given the purely empirical item-selection process, the obtained criterion-

related validity estimate would be inflated to some extent because of capitalization on chance.

Comparisons between the CPI-MD Scale and the CPI Manual's Fake-Good Scale

The current motivational distortion predictor scale was intended to be an improvement

on the existing CPI Fake-good validity scale (CPI-FG) from the CPI manual (Gough and

Bradley, 1996) by offering a more precise measurement of the motivational distortion construct.

To evaluate the relative effectiveness of the CPI-MD and CPI-FG scale as measures of

motivational distortion, a few additional analyses were conducted to compare the psychometric

properties of these two scales.

Using the employment-condition data from U B C Sample 4, correlations of the CPI-FG

scale with the M D criterion and the CPI-MD scale were obtained. Because of significant gender

differences on both the CPI-MD and CPI-FG scale (p < .01 for both), the correlations were

obtained using scores that had first been mean-deviated within the separate-gender samples (to

scale out gender mean differences). Using U B C Sample 2, stability estimates of the CPI-FG

scale were also obtained to compare its test-retest reliability with that of the CPI-MD scale.

Results

Norms for the CPI-MD Scale and Analyses of Gender Differences

Table 4 presents the means and standard deviations of the CPI-MD scale for three

different groups of North American subjects, separately by gender. These norms provide a useful

18

source of reference for future use, although the development of local norms is strongly advised

where possible.


From Table 4, it can be seen that in samples where the test scores had the greatest

consequences, the CPI-MD mean scores were correspondingly larger: the undergraduate student

norm group showed the lowest mean scores, and the non-management job-applicants norm group

showed the highest mean scores. The analyses of gender differences for the three norm groups

revealed no significant differences in all three samples (p > .25).

Evidence of Reliability for the CPI-MD Scale

The internal consistency reliability estimates (Cronbach's alpha) for the separate-gender

samples of undergraduate students, management employees, and non-management job-

applicants, appear in Table 5.


It is encouraging to note that among the three samples, the non-management job-

applicants sample yielded the highest internal consistency reliability estimates, as this was the

population for which the CPI-MD scale was targeted for applied use. The comparatively higher

internal consistency reliability estimates obtained for this sample could be a result of the fact that

the CPI-MD scale was specifically developed to capture the construct of motivational distortion

as it occurred within a hiring context. Also, of course, as can be seen from Table 4, this sample

demonstrated the greatest variability on the CPI-MD scale.

The short-term stability estimates (test-retest correlations with 2-3 week interval) of the

CPI-MD scale, computed separately by gender, were: males, .803 (n = 71); females, .856 (n =

19

158). These separate-gender stability coefficients were not significantly different (p> .10), and

were, therefore, pooled to yield a single estimate of stability of .841 (n = 229).

Evidence of Construct Validity of the CPI-MD Scale

Scores on the CPI-MD scale were correlated with the five domain scales of the NEO PI-

R, separately by gender. In both gender groups, scores on three domain scales were significantly

correlated with CPI-MD scale scores (p < .001, non-directional)—Neuroticism, Extraversion,

and Conscientiousness—and the CPI-MD scale was correlated with the facet scales of these

three domains. The correlations appear in Table 6.


The pattern of correlations of the CPI-MD scale with the five NEO PI-R domain scales

was similar across both genders: Extraversion showed the highest correlation followed by

Conscientiousness and Neuroticism (ordered by magnitude of correlation coefficient obtained).

The correlations between the CPI-MD scale with the facet scales of Neuroticism, Extraversion,

and Conscientiousness also revealed a relatively similar profile between the genders. For the

Extraversion domain, Assertiveness emerged as the dominant component in both genders

(disattenuated r = .84). For the Conscientiousness domain, the key facets were Achievement

Striving and Self-Discipline for males, and Achievement Striving and Competence for females.

Lastly, for the Neuroticism domain, the two highest correlations (both negatively correlated with

CPI-MD) were Self-Consciousness and Vulnerability in both genders, although the level of

correlations was higher for females. Overall, the pattern of correlations suggest that for both

males and females, motivational distortion in an employment context is associated with higher

levels of Assertiveness and Achievement-Striving, and lower levels of Self-consciousness and

Vulnerability.

20

There are two possible explanations for the observed pattern of correlations. One

interpretation is that motivational distortion within an employment context is related to real

individual differences on these personality dimensions, or alternatively, that measurement of

these personality dimensions is particularly susceptible to distortion in an employment context.

The present experimental design does not allow for a more definitive interpretation to be made.

The reader is referred to other recent empirical studies (e.g., McFarland & Ryan, 2000; Ones,

Viswesvaran, & Reiss, 1996; Rosse et al., 1998) that may cast some light on this issue, even

though their findings are somewhat disparate.

The above results differ from the meta-analytic research findings in Ones, Viswesvaran

and Reiss's (1996) study, which showed that, among the Big Five personality factors (using self-

report measures), social desirability scales correlated the highest with Emotional Stability (the

inverse construct to Neuroticism), followed by Conscientiousness and Agreeableness. In

contrast, as reported above, the results from this study suggest that motivational distortion is

associated most highly with Extraversion, followed by Conscientiousness, and Emotional

Stability, in that order. One explanation for the differences in results obtained is that whereas the

Ones, Viswesvaran, and Reiss (1996) study used a broad conceptualization of the social

desirability construct, the present study specifically focused on the motivational distortion that

occurs within an employment context. If this is true, the difference in the correlational results

obtained between the two studies might be an indication of a need to refine the concept of social

desirability in future research on motivational distortion within employment settings. Indeed,

some researchers have pointed out that, within employment settings, job desirability might be a

more relevant construct than social desirability (e.g., Kluger & Collela, 1993).

Correlation between the CPI-MD and CPI-Cp Scale

The pooled-gender correlation between CPI-MD and CPI-Cp scale scores (both from

employment-condition data) was -.587 (n = 250, p < .0001). Although the obtained correlation

2 1

was inflated as a result of overlapping item content between the two scales (14 common items in

total), it provides an indication that CPI-Cp scores (employment-condition) contain variance

related to motivational distortion. (Note: The correlation between the CPI-Cp scores and the M D

criterion was found to be -.204, p < .001.)

Estimation of Criterion-Related Validity

The mean correlation (over the two gender samples) between the CPI-MD scale and the

M D criterion was .543 (n = 250). This correlation is clearly inflated because of capitalization on

chance (the CPI-MD scale items were selected on the basis of item-criterion correlations with the

M D criterion). Because of sample size (n = 250), it was not practical to attempt to establish

empirical cross-validation in this study.

Comparisons between the CPI-MD Scale and the CPI Manual's Fake-Good Scale

The pooled-gender correlation between the CPI-MD scale and the CPI-FG scale was

.843 (n = 250). The high correlation is to be expected, considering the fact that both scales are

tapping into the same global construct of motivational distortion.

The stability estimates of the CPI-FG scale were: for men, r=638 (n = 71); for women,

r=.818 = 158). These estimates were both significantly lower than those of the CPI-MD scale

(p < .05). The criterion-related validity of the CPI-FG scale (for the M D criterion) was .365 (over

both gender samples), and although the value is statistically significant (p < .0005, directional), it

is significantly lower (p < .001) than the corresponding correlation for the CPI-MD scale (.543).

Thus, even though some reduction in the criterion-related validity of the CPI-MD scale (for the

M D criterion) can be expected on cross-validation, the present results seem to suggest that the

CPI-MD scale offers a relatively more reliable and valid measure of motivational distortion than

does the CPI-FG scale.

22

Summary of Results

A 56-item CPI-based measure of motivational distortion (the CPI-MD scale) was

developed on the basis of item-criterion correlations with the motivational distortion criterion

measure developed in Study 1. Separate-gender norms for the CPI-MD scale were provided for

three different populations—undergraduate students, management employees, and non-

management job applicants. Internal consistency and stability reliability estimates of the scale

ranged between .72 and .86 across these same three populations. Employing the Five Factor

Model, as measured by the NEO PI-R, as a framework for construct validation, employment-

centered motivational distortion was found to be associated with Extraversion,

Conscientiousness, and Emotional Stability. A more fine-grained approach using the NEO PI-R

facet scales further revealed that the dominant components associated with motivational

distortion were Assertiveness, Achievement-Striving, low Self-consciousness, and

Invulnerability. Finally, when compared with the CPI Manual's Fake-good scale, the present

CPI-MD scale demonstrated relatively better reliability and criterion-related validity.

Study 3: Development of an Adjustment Equation to Correct CPI-Cp Scale Scores for

Motivational Distortion

Research Aims

Using the previously developed CPI-MD scale as a predictor measure of motivational

distortion, this study aimed to develop a correction equation for adjusting CPI-Cp scale scores

for the effects of motivational distortion so as to derive more honest score estimates for use in

personnel decision-making.

Samples

A total of six samples were used in this study. With the exception of one sample

(Sample 6 below), all of the other five samples in this study were previously used and described

23

in Study 2. The six data samples employed in this study are listed below, along with the sample

sizes and analyses for which the sample was used. Additional information is provided here for

Samples 5 and 6 as this information was not previously given.

1. UBC Sample 4—250 subjects (55 males, 195 females)—This sample was used in the

development of the adjustment equations, and for some additional correlational analyses that

were conducted to assess the effectiveness of the developed score adjustments in

approximating honest scores.

2. UBC Sample 2—229 subjects (71 males, 158 females)—This sample was used to obtain

stability estimates of the adjusted CPI-Cp scale scores.

3. UBC Sample 3— 320 subjects (98 males, 222 females)—This sample was used to assess the

construct validity of adjusted CPI-Cp scale scores.

4. Management Employee Sample—457 subjects (245 males, 212 females)—This sample was

used to provide norms of the adjusted CPI-Cp scale scores.

5. Non-Management Job Applicant Sample—528 subjects (193 males, 335 females)—The full

sample was used to provide norms of the adjusted CPI-Cp scale scores, and a subset of this

sample was used for assessing the criterion-related validity of adjusted CPI-Cp scores. This

subsample consisted of 79 applicants for a telephone service representative's job in a large

Canadian telecommunications company, with a ratio of about two females to every male job

applicant. These subjects were assessed with an experimental assessment battery (including

the CPI) after a positive hiring decision had been made (on the basis of an earlier selection

procedure), but before commencement on the job. A performance-appraisal inventory was

completed on them by their supervisors about 3-4 months into their jobs, and performance

ratings on this inventory was used as the job-criterion measure in the criterion-related

validity analyses.

24

6. Non-Management Employee Sample—This sample comprised a total of 91 subjects

consisting of 80 male and 11 female employees of an armored-car company in Vancouver,

British Columbia. The vast majority of the subjects were Caucasians, with an age range of

early 20's to mid-50's, and a median age of 32. During the early-to mid-1990's, these

subjects were administered the CPI (Form 462) prior to being hired although these data were

not available for decision-making at the time of hiring. In 1996, their personnel files were

accessed as a source of criterion data for the Hakstian et al. (2002) study. At that time, the

subjects had been employed for anywhere from a few months to nearly five years, with a

mean tenure of about two years. This sample, though small, was valuable as it provided an

estimate of long-term predictive validity coefficients of the adjusted CPI-Cp scores.

Job-Performance Criterion Measures Used in Criterion-Related Validity Analyses

The two employment samples (Sample 5 and 6 above) used for the assessment of

criterion-related validity of the adjusted CPI-Cp scale scores were originally used in the Hakstian

et al. (2002) study. A description of the criterion variables used in these two employment

samples is provided here:

1. Non-management sample of telephone service representatives (Subset of Sample 5).

For this sample, the criterion variables were measured by a performance-appraisal instrument

comprising 44 behavioral observation scale items assessing various work-related dimensions. Of

these, three work dimensions were conceptually related to the Conscientiousness construct—(a)

Availability for Work, (b) Desire to Improve, and (c) Use of Time. In addition, a sum of the 44

BOS items provided an Overall Performance measure.

2. Sample of armored-car company employees (Sample 6). The counterproductivity job

behavior criterion in this sample was assessed from anecdotal file commentary, reprimands,

disciplinary notations, etc., contained within the personnel file data of the employees. Hakstian et

al. (2002) devised a coding system whereby file data were coded for frequency and seriousness

25

of various on-the-job counterproductive behaviors. The resulting criterion scores fell on a 6-point

scale ranging from 0 (no counterproductive behaviors noted) to 5 (several and serious

incidents—including discharge). They reported that these scores were extremely positively

skewed: 42% of the sample received a zero score, the overall mean was 1.22 (on the 0-5 scale)

and standard deviation was 1.44.

Method

Development of the Adjustment Equation

Conceptually, the strategy for developing this adjustment equation involved predicting

the amount of motivational distortion contained within a set of CPI data obtained under

employment conditions, and then increasing the CPI-Cp scale scores by this amount of predicted

distortion. (Note: Counterproductivity is a negative construct and subjects would aim to depress

their CPI-Cp scores under employment conditions.)

To predict the amount of motivational distortion, a regression equation was constructed

to predict M D criterion scores from CPI-MD scale scores using employment-condition data from

U B C Sample 4 (Sample 1 in this study). The dependent variable used in the regression was the

M D criterion developed in Study 1 as these criterion scores indicated the extent to which

subjects had distorted their CPI-Cp scores in the employment condition.

To obtain the regression equation, the appropriate means, standard deviations, and

correlation coefficient were substituted into the following raw-score regression equation formula:

Y = r x y ^ X + ( Y - r x y ^ X ) , (1)

where Y represents the M D criterion, X represents the CPI-MD scale, and r x y represents the

correlation between the CPI-MD scale and the M D criterion. The means, standard deviations,

and correlation coefficient were based on pooled-gender data so as to obtain a more stable and

generic equation that could be applied for both genders. Even though significant gender

26

differences were reported for the CPI-MD scale (in Study 2), separate-gender equations were not

preferable because of the small sample size for males.

Two other alternative methods of developing the adjustment equation were considered,

but as the three sets of adjusted CPI-Cp scores were highly correlated (average intercorrelation =

.99), the procedure described above was favored as it offered the most conceptually

parsimonious method of adjustment. Detailed descriptions of the alternative methods of

adjustment, along with the equations for obtaining adjusted CPI-Cp scores by these methods, can

be found in Appendix F.

Correlations between Honest/Unadjusted CPI-Cp Scores and Adjusted CPI-Cp Scores

To provide an estimate of the extent to which the developed motivational-distortion

score adjustments to the CPI-Cp scale approximated honest scores, adjusted CPI-Cp scores

(based on CPI scores from the employment condition) were correlated with CPI-Cp scores from

the honest condition, using data from U B C Sample 4.

In addition, correlations between unadjusted and adjusted CPI-Cp scores were obtained

for the two employment samples—Samples 4 and 5. The relationship between these two sets of

scores would provide an indication of the extent to which motivational distortion contaminated

CPI-Cp scores obtained under employment conditions.

Norms for the CPI-Cp Scale Using Adjusted Scores

Separate-gender norms for the adjusted CPI-Cp scale were developed using the two

workplace samples: (a) the management-level sample (Sample 4 above), and (b) the non-

management-level sample (Sample 5 above). Analyses of gender differences on the adjusted

CPI-Cp scale scores were conducted in both samples. In addition, inferential tests of mean

differences were also conducted to compare unadjusted and adjusted CPI-Cp scores, separately

by gender.

27

Reliability of the Adjusted CPI-Cp Scale

Using U B C Sample 2 (Sample 2 in this study), the stability estimates of the adjusted

CPI-Cp scores was assessed, separately by gender, using test-retest correlations between adjusted

scores from two time points, with a 2-3 week time interval between administrations.

Construct Validity of the Adjusted CPI-Cp Scale

Using the Five Factor Model as a framework to assess the impact on the construct

validity of the CPI-Cp scale when scores were corrected for motivational distortion, adjusted

CPI-Cp scale scores were correlated with scores on the five NEO PI-R (Costa & McCrae, 1992)

domain scales and the six NEO PI-R Conscientiousness facet scales, separately by gender. Using

U B C Sample 3, this set of correlations was compared with that obtained from unadjusted scale

scores. (The Conscientiousness facet scales were emphasized in this study because of their

specific relevance to the counterproductivity construct. The comprehensive set of correlational

analyses, with all NEO PI-R facet scales included, appear in Appendix G.)

Assessment of the Criterion-Related Validity of Adjusted CPI-Cp Scores

CPI-Cp data from the two employment samples (Subsample 5 and Sample 6) were

corrected for the effects of motivational distortion and the resulting adjusted CPI-Cp scores were

correlated with criterion scores. Due to sample size constraints, these correlations were based on

pooled-gender samples. As pointed out in the Hakstian et al. (2002) study, given that there were

no significant gender mean differences on the criterion measures in both samples, the resulting

criterion correlations should be relatively free of between-groups correlational distortion.

The criterion correlations were also partially disattenuated for criterion unreliability for

the purposes of estimating true validity. In addition, for the armored-car company employees

sample, job tenure was an obvious contaminating variable in the criterion scores (correlating .39

with the criterion), as the number of counterproductivity work incidents recorded in the

employee personnel files would be a function of tenure. Therefore semi-partial correlations were

28

computed for this sample, with the length of tenure partialed out of the criterion scores in the

CPI-Cp-criterion correlation.

Results

Adjustment Equation for Correcting CPI-Cp Scale Scores for Motivational Distortion

As outlined above, the first step in obtaining the adjustment equation involved

constructing a regression equation for predicting M D criterion scores (Y) from CPI-MD scale

scores (X). The means, standard deviations, and correlation coefficients of these two variables in

the regression equation appear in Table 7.


Substituting these values into the raw-score regression equation formula in Eqn. (1), we obtain

the predicted raw score on motivational distortion as: Y = .637X - 10.869. Therefore, adding the

predicted amount of motivational distortion to the original CPI-Cp scale scores, the adjustment

equation for correcting CPI-Cp scale scores for the effects of motivational distortion is:

Adjusted CPI-Cp = Original CPI-Cp + .637(CPI-MD) - 10.869.

Correlations between Adjusted/Honest CPI-Cp Scores and Unadjusted CPI-Cp Scores

The correlations between employment-condition adjusted CPI-Cp scores and honest-

condition CPI-Cp scores in the undergraduate student experimental sample (UBC Sample 4), and

the correlations between unadjusted and adjusted CPI-Cp scores in two workplace samples,

together with the associated descriptive statistics, appear in Table 8.


Inferential tests revealed that the three pairs of separate-gender correlations were not

significantly different at the .05 level and therefore they were pooled across gender, within each

sample, to yield a weighted combined-gender correlation. The resulting pooled-gender

29

correlations are: (1) .654 (n = 250) for the adjusted-honest correlation in the undergraduate

student sample; (2) .843 (n = 457) for the adjusted-unadjusted correlation in the management-

level employees; and (3) .861 (n = 528) for the adjusted-unadjusted correlation in the non-

management job-applicant sample.

Although the correlation between the honest-condition and adjusted employment-

condition CPI-Cp scores (.654, pooled-gender) from the experimental undergraduate student

sample which the CPI-MD scale was developed, can be considered smaller than expected, this is

significantly greater (p < .01) than the correlation obtained between honest-condition and

unadjusted employment-condition scores (r=.517). Therefore, the results provide evidence to

suggest that the proposed MD-adjustments to the CPI-Cp scale scores brought the employment-

condition scores in closer alignment with honest-condition scores.

Norms for the CPI-Cp Scale Using Adjusted Scores

Separate-gender workplace-sample norms of the adjusted CPI-Cp scale scores are

provided for management-level employees and non-management job-applicants. The normative

data appear in Table 8 as descriptive statistics for the second and third sample-groups.

The results from the analysis of gender mean differences (males - females) for the

adjusted CPI-Cp scores was not significant for the management sample (p > .50) but significant

for the non-management job applicants sample, ^(526) = 3.93, p < .001, A = .351, .95 C.I. for A

= (.175, .526). Comparing the means of adjusted and unadjusted CPI-Cp scores within each

gender, results showed a significant increase (at the/? < .001 level) in CPI-Cp mean scores when

adjustments were made, in both the management and non-management level data samples, for

both genders. The increase in mean scores ranged from about .85 to 1.20 standard deviation units

between the two samples. These effect sizes are larger than those obtained by Alliger and Dwight

(2000) in their meta-analytic study on the susceptibility of integrity tests to distortion. They

found effect sizes of about Vi to Vi standard deviation for personality-based tests.

30

It is interesting to note as well that when adjustments for motivational distortion were

made to the CPI-Cp scale scores from the employment samples, the resulting adjusted scale

means ended up within .10 standard deviation units of the separate-gender means in the CPI

Basic Norm Sample given in Hakstian et al. (2002).

Reliability of the Adjusted CPI-Cp Scale

The test-retest correlations (with a 2-3 week interval between administrations),

computed separately by gender, were .883 (n = 71) for males and .869 (n = 158) for females. As

these two stability coefficients for adjusted CPI-CP scale scores were not significantly different,

they were pooled to yield a weighted estimate of .873 (n = 229) across the genders. The stability

estimate of the adjusted CPI-Cp scale scores was comparable to that of the original (unadjusted)

CPI-Cp scale reported by Hakstian et al. (2002) i.e. r = .893. This indicates that the adjustments

for motivational distortion did not substantially reduce the stability of this scale.

Construct Validity of the Adjusted CPI-Cp Scale

The correlations of CPI-Cp scale scores (both unadjusted and adjusted) with the five

NEO PI-R domain scales and the six NEO PI-R Conscientiousness facet scales appear in Table

9, separately by gender.


In both the male and female samples, the obtained pattern of correlations with NEO PI-

R domain scales differed between adjusted and unadjusted CPI-Cp scores. For males, whereas

unadjusted CPI-Cp scores suggested that self-reported counterproductivity was associated with

Extraversion, Agreeableness and Conscientiousness, adjusted scores suggested an association

only with Extraversion and Agreeableness. In addition, when compared with unadjusted scores,

the level of correlations obtained between adjusted CPI-Cp scores and NEO PI-R domain scores

was significantly higher for Extraversion, and significantly lower for Neuroticism and

31

Conscientiousness (all at p < .001, nondirectional). A similar pattern was observed amongst the

females: when compared with unadjusted CPI-Cp scores, adjusted scores showed a significantly

stronger relationship between self-reported counterproductivity and Extraversion, and a

significantly weaker relationship with Neuroticism and Conscientiousness (p < .001,

nondirectional).

At a global level, the pattern of correlations between CPI-Cp scores and NEO PI-R

Conscientiousness facet scales was similar for both adjusted and unadjusted scores, in both

gender samples. Among the six facet scales, Deliberation, Dutifulness and Self-discipline

showed the strongest association with self-reported counterproductivity for both adjusted and

unadjusted scores, although the level of correlation was reduced with adjusted scores.

Looking at gender differences for the five NEO PI-R domains and six

Conscientiousness facet scales, we see identical results for both unadjusted and adjusted CPI-Cp

scores. Among the five domain scales, significant gender differences were found for Neuroticism

(p < .05, nondirectional), with self-reported counterproductivity showing a Neuroticism

component among women but not among men. Among the six Conscientiousness facet scales,

significantly stronger correlations were obtained for women on both the Competence (p < .01,

nondirectional) and Self-discipline (p < .05, nondirectional) scales. These results further

emphasize the underlying differences in the psychological underpinnings of the

counterproductivity construct between the two genders, that were first identified by Hakstian et

al. (2002).

At this point, it is important to remind the reader that these correlations were obtained in

a sample (Sample 3 in this study) in which no motivational distortion was expected, as this was

an undergraduate student sample to which both the CPI and NEO PI-R were administered under

normal conditions. Thus, in these analyses, we cannot assert that the adjusted CPI-Cp scores are,

in fact, more honest than the unadjusted scores. In fact, the CPI-MD mean scores obtained by

32

this sample suggested a low incidence of motivational distortion overall (for males, 21.47, n =

98; for females, 23.01, n = 222), even though inferential tests revealed significant mean

differences (at p < .001) between the unadjusted and adjusted CPI-Cp scale scores, for both

genders. Therefore, it is necessary to view the above results with this caveat in mind, and

additional research is required to verify these findings.

Assessment of the Criterion-Related Validity of Adjusted CPI-Cp Scores

The correlations between the adjusted CPI-Cp scale scores and related job-performance

criteria in two workplace samples appear in Table 10, along with the criterion correlations

obtained using unadjusted CPI-CP scores. Both bivariate correlations and true validity

coefficients, in which criterion correlations have been partially disattenuated for criterion

unreliability, are presented. For the armored-car employee sample, semi-partial correlations, with

the length of tenure partialed out of the criterion in the predictor-criterion correlations, are given

as well. For partial disattenuation, inter-rater reliability estimates were needed, but not available

in the present data sets. Thus, the meta-analytically-derived values of Viswesvaran, Ones, and

Schmidt (1996) were used (as Hakstian et al., 2002, had done).


For the sample of telephone service representatives, the pattern of criterion correlations

across the four job-performance criteria revealed that criterion-related validity tended to increase

when adjusted CPI-Cp scale scores were used. Inferential test results demonstrated a significant

increase in validity coefficients for the Desire to Improve criterion [/(76) = 1.73,/? < .05] and the

Overall Performance criterion [7(76) = 2.27,/? < .025].

For the armored-car employee sample, the bivariate correlations showed a non

significant increase in the bivariate criterion-related validity coefficient (from .25 to .28) with the

use of adjusted CPI-CP scores. However, the semi-partial correlations showed a bridging of the

3 3

difference in criterion-related validity coefficients between using adjusted or unadjusted CPI-Cp

scores. Because of a higher correlation between the adjusted CPI-Cp scores and length of tenure

( r = .13, p > .10), than was obtained for the unadjusted CPI-Cp scores and length of tenure,

partialling out the effects of tenure from the criterion correlations led to a decrease in the validity

coefficients for the adjusted scores. (In contrast, the correlation between unadjusted CPI-Cp

scores and tenure was .03.)

For comparative purposes, similar criterion-related analyses were conducted with CPI-

Cp scores that were adjusted for motivational distortion on the basis of the CPI Manual's Fake-

good scale (CPI-FG scale). Using an identical developmental procedure to that described above,

a similar adjustment equation was obtained by building a regression equation to predict M D

criterion scores from the CPI-FG scale scores and then adding this predicted amount of distortion

to original CPI-Cp scale scores. The results obtained by using this new adjustment equation

indicated similar improvements in criterion-related validity although the magnitude of increase

was relatively less than that obtained by adjustments made using the CPI-MD scale developed in

this study (See Appendix H).

Summary of Results

On the basis of a pooled-gender sample of 250 undergraduate student subjects, an

adjustment equation was developed to correct CPI-Cp scores for the effects of motivational

distortion. The effect of applying this equation was to increase CPI-Cp scores by the amount of

distortion predicted on the basis of CPI-MD scale scores. Adjustments to the CPI-Cp scale scores

brought the scores in closer alignment with honest scores as evidenced by a higher correlation

obtained between adjusted and honest-condition scores as compared to unadjusted and honest-

condition scores. Using adjusted scores, a new set of CPI-Cp scale norms were provided for two

workplace samples.

34

Adjusted CPI-Cp scores maintained the high stability estimates of the original CPI-Cp

scale but showed some changes in construct validity. When scores were adjusted for

motivational distortion, correlations between the CPI-Cp scale and the Big Five factors of

personality suggested that self-reported counterproductivity was related to Extraversion and

Agreeableness for men, and predominantly to Extraversion, Agreeableness, and

Conscientiousness for women.

Finally, the study also demonstrated that when CPI-Cp scores were adjusted for

motivational distortion, criterion-related validity for certain job-performance criteria in one

employee sample increased.

Overall Conclusions and Discussion

The purpose of the present research was to develop a measure of motivational distortion

that could be used as a basis for making adjustments to CPI-Cp scale scores for motivational

distortion as it occurs under employment conditions. The CPI-MD scale provided such a

measure, and as a self-report personality-based measure, demonstrated fully adequate reliability.

The correlation of CPI-MD scale scores with the Big Five personality factors (using the NEO P i

ll.) revealed that motivational distortion was positively associated with Extraversion,

Conscientiousness, and Emotional Stability. This pattern of correlations differed from other

research involving various social desirability measures and is an indication that different

psychological constructs might be needed to explain the concept of motivational distortion

within organizational settings.

There are several advantages in the developed CPI-MD scale. As noted earlier, when

compared to the Fake-good scale from the CPI manual, the CPI-MD scale offers a more precise

measurement of motivational distortion, as it is based only on those CPI items that are construct-

relevant. Preliminary results also suggest that motivational distortion adjustments made using the

CPI-MD scale were relatively more effective in increasing criterion-related validity of the CPI-

35

Cp scale, as compared to adjustments made using the CPI-FG scale. In addition, when compared

with other available social desirability or response distortion/validity scales, the present CPI-MD

scale is especially adapted for use in employment settings. Not only was it designed to measure

motivational distortion as it specifically occurs under employment conditions (as a result of its

developmental methodology), but as a personality-based measure, it offers a covert assessment

of distortion which reduces somewhat the likelihood of contamination by motivational distortion

on the very scale itself.

At this juncture, it is important to point out that the 56 items of the CPI-MD scale were

identified using a student sample with simulated employment conditions and as such, the item-

composition might reflect to some extent this experimental approach. Further research in applied

settings is required to externally validate this scale. It should be noted as well that the present

CPI-MD scale was developed with the specific intention of using this scale as a basis for

adjusting scores on the CPI-Cp scale for motivational distortion. The use of the CPI-MD scale

for alternative purposes, including the derivation of adjustment equations for other personality

scales has not been tested. However, the results from this study indicate the viability of the scale

construction methodology used, and with this methodology, similar motivational distortion

measures and adjustment equations could be obtained for other personality inventories/scales.

Finally, this study provides evidence that adjusting personality scale scores for

motivational distortion can enhance the psychometric properties of the scale being measured. In

this case, the results showed that score-adjustments preserved test-retest reliability, altered the

construct validity of the scale being measured, and led to an increase in criterion-related validity

in one workplace sample. Although this is only a single study in the face of many other studies

that have found evidence of the contrary, these findings contribute to ongoing research that

supports the argument for controlling the effects of motivational distortion in personnel testing.

36

References

Alliger, G. M . , & Dwight, S. A. (2000). A meta-analytic investigation of the susceptibility of

integrity tests to faking and coaching. Educational and Psychological Measurement, 60(1),

59-72.

Barrick, M . R., & Mount, M . K. (1991). The big-five personality dimensions and job

performance. Personnel Psychology, 44, 1-26.

Barrick, M . R., & Mount, M . K. (1996). Effects of impression management and self-deception

on the predictive validity of personality constructs. Journal of Applied Psychology, 81, 262-

272.

Becker, T. E., & Colquitt, A. L. (1992). Potential versus actual faking on a biodata form: An

analysis along several dimensions of item type. Personnel Psychology, 45, 389-406.

Cellar, D. F., Miller, M . L., Doverspike, D. D., & Klawsky, J. D. (1996). Comparison of factor

structures and criterion-related validity coefficients for two measures of personality based on

the five factor model. Journal of Applied Psychology, 81(6), 694-704.

Christiansen, N . D., Goffin, R. D., Johnston, N . G , 8c Rothstein, M G . (1994). Correcting the

16PF for faking: Effects on criterion-related validity and individual hiring decisions.

Personnel Psychology, 47, 847-860.

Collins, J. M . , & Glese, D. H. (1998). Race, job applicants and the Five Factor Model, Journal of

Applied Psychology, 83, 531-544.

Costa, P. T., Jr., & McCrae, R. R. (1992). The Revised NEO Personality Inventory (NEO PI-R)

and NEO Five Factor Inventory (NEO-FFI) professional manual. Odessa, FL: Psychological

Assessment Resources.

Cunningham, M . R., Wong, D. T., & Barbee, A. P. (1994). Self-presentation dynamics on overt

integrity tests: Experimental studies of the Reid Report. Journal of Applied Psychology,

79(5), 643-658.

37

Dahlstrom, W. G., Welsh, G. S., & Dahlstrom, L. E. (Eds.) (1972). An MMPI Handbook (Vol. 1).

Minneapolis: University of Minnesota Press.

Dahlstrom, W. G., Welsh, G. S., & Dahlstrom, L. E. (Eds.) (1975). An MMP I Handbook (Vol. 2).

Minneapolis: University of Minnesota Press.

Douglas, E. F., McDaniel, M . A., & Snell, A. F. (1996, August). The validity of non-cognitive

measures decays when applicants fake. Paper presented at the annual meeting of the

Academy of Management, Cincinnati, OH.

Ellingson, J. E., Sackett, P. R., & Hough, L. M . (1999). Social desirability corrections in

personality measurement: Issues of applicant comparison and construct validity. Journal of

Applied Psychology, 84(2), 155-166.

Ellingson, J. E., Smith, D. B., & Sackett, P. R. (2001). Investigating the influence of social

desirability on personality factor structure. Journal of Applied Psychology, 86(1), 122-133.

Gordon, M . E., & Gross, R. H. (1978). A critique of methods for operationalizing the concept of

fakeability. Educational and Psychological Measurement, 38, 771-782.

Gough, H . G. (1957). Manual for the California Psychological Inventory. Palo Alto, CA:

Consulting Psychologists Press.

Gough, H. G. (1987). The California Psychological Inventory administrator's guide. Palo Alto,

CA: Consulting Psychologists Press.

Gough, H . G., & Bradley, P. (1996). The California Psychological Inventory manual (3rd ed.).

Palo Alto, CA: Consulting Psychologists Press.

Guastello, S. J., & Rieke, M . L. (1991). A review and critique of honest test research. Behavioral

Sciences and the Law, 9, 501-523.

Hakstian, A. R., Farrell, S., & Tweed, R. T. (2002). The assessment of counterproductive

tendencies by means of the California Psychological Inventory. International Journal of

Selection and Assessment, 10(1/2), 58-86.

38

Helmes, E. (2000). The role of social desirability in the assessment of personality constructs. In

R. D. Goffin & E. Helmes (Eds.), Problems and solutions in human assessment: Honoring

Douglas N. Jackson at seventy, (pp. 21-40). Norwell, M A : Kluwer Academic Publishers.

Hogan, R. (1986). Hogan Personality Inventory. Minneapolis, M N : National Computer Systems.

Hogan, J., & Hogan, R. (1972). Hogan Personality Inventory manual. Tulsa, OK: Hogan

Assessment Systems.

Horst, P. (1966). Psychological Measurement and Prediction. Belmont, CA: Wadsworth

Publishing.

Hough, L . M . (1998). Effects of intentional distortion in personality measurement and evaluation

of suggested palliatives. Human Performance, 5, 139-155.

Hough, L . M . Eaton, N . L. , Dunnette, M . D., Kamp, J. D., & McCloy, R. A. (1990). Criterion-

related validities of personality constructs and the effect of response distortion on those

validities [Monograph]. Journal of Applied Psychology, 75, 581-595.

Hough, L. M . , & Schneider, R. J. (1996). The frontiers of I/O personality research. In K.R.

Murphy (Ed.,), Individual differences and behavior in organizations (pp. 31-88). San

Francisco: Jossey-Bass.

Jackson, D. N . (1967). Personality Research Form manual (3 r d ed.). Port Huron, MI: Research

Psychologists Press.

Kluger, A. N . , & Colella, A. (1993). Beyond the mean bias: The effect of warning against faking

on biodata item variances. Personnel Psychology, 46, 763-780.

Lanning, K. (1989). Detection of invalid response patterns on the California Psychological

Inventory. Applied Psychological Measurement, 13, 45-56.

Lautenschlager, G. J. (1986). Within-subject measures for the assessment of individual

differences in faking. Educational and Psychological Measurement, 46, 309-316.

39 Livneh, H. , & Livneh, C. (1989). The five-factor model of personality: Is evidence of its cross-

measure validity premature? Personality and Individual Differences, 10(1), 75-80.

LoBello, S. G., & Sims, B. N . (1990). Fakability of a commercially produced pre-employment

test. Journal of Business and Psychology, 8, 265-273.

McCrae, R. R., & Costa, P. T., Jr. (1983). Social desirability scales: More substance than style.

Journal of Consulting and Clinical Psychology, 51, 882-888.

McDaniel, M . A. , Douglas, E. F., & Snell, A. F. (1997). A survey of deception among job

seekers. Paper presented at the 12 th annual meeting of the Society for Industrial and

Organizational Psychology.

McFarland, L . A. (2000). Toward an integrated model of applicant faking. Unpublished

manuscript, Michigan State University.

McFarland, L. A., & Ryan, A. M . (2000). Variance in faking across noncognitive measures.

Journal of Applied Psychology, 85(5), 812-821.

Murphy, K. R., & Davidshofer, C. O. (1998). Psychological testing: Principles and applications.

Upper Saddle River, NJ: Prentice-Hall.

Nicholson, R. A., & Hogan, R. (1990). The construct validity of social desirability. American

Psychologist, 45, 290-292.

Ones, D. S., & Viswesvaran, C. (1998a). The effects of social desirability and faking on

personality and integrity assessment for personnel selection. Human Performance, 11(2/3),

245-269.

Ones, D. S., & Viswesvaran, C. (1998b). Integrity testing in organizations. In R. W. Griffin, A.

O'Leary-Kelly & J. M . Collins (Eds.), Dysfunctional behavior in organizations: Vol 2. Non

violent behaviors in organizations. Greenwich, CT: JAI.

40

Ones, D. S., Viswesvaran, C , & Reiss, A. D. (1996). Role of social desirability in personality

testing for personnel selection: The red herring. Journal of Applied Psychology, 81(6), 660-

679.

Ones, D. S., Viswesvaran, C , & Schmidt, F. L. (1993). Comprehensive meta-analysis of

integrity test validities: Findings and implications for personnel selection and theories of job

performance [Monograph]. Journal of Applied Psychology, 78, 619-103.

Paulhus, D. L . (1991). Measurement and control of response bias. In J. P. Robinson, P. R.

Shaver, & L. S. Wrightsman (Eds.), Measures of personality and social psychological

attitudes (pp. 17-59). San Diego, CA: Academic Press.

Rosse, J. G., Stecher, M . D., Miller, J. L. , & Levin, R. A. (1998). The impact of response

distortion on preemployment personality testing and hiring decisions. Journal of Applied

Psychology, 83(4), 634-644.

Ryan, A. M . , & Sackett, P. R. (1987). Pre-employment honesty testing: Fakability, reactions of

test takers, and company image. Journal of Business and Psychology, 1(3), 248-256.

Schmidt, M . J., & Ryan, A. M . (1993). The Big Five in personnel selection: Factor structure in

applicant and nonapplicant populations. Journal of Applied Psychology, 78, 966-974.

Smith, D. B. , Hanges, P. J., & Dickson, M . W. (2001). Personnel selection and the Five-Factor

Model: Reexamining the effects of applicants' frame of reference. Journal of Applied

Psychology, 86(2), 304-315.

Viswesvaran, C , & Ones, D. S. (1999). Meta analyses of fakability estimates: Implications for

personality measurement. Educational and Psychological Measurement, 59(2), 197-210.

Viswesvaran, C , Ones, D. S., & Schmidt, F. L . (1996). Comparative analysis of the reliability of

job performance ratings. Journal of Applied Psychology, 81, 557-574.

41

Weiner, J. A. , & Gibson, W. M . (April 2000). Practical effects offaking on job applicant attitude

test scores. Paper presented at the 15 t h annual meeting of the Society of Industrial and

Organizational Psychology, New Orleans.

Williams, R. H. , Zimmerman, D. W., & Mazzagatti, R. D. (1987). Large sample estimates of the

reliability of the simple, residualized and base-free measures gain scores. Journal of

Experimental Education, 55, 116-118.

Winder, P., O'Dell, J. W., & Karson, S. (1975). New motivational distortion scales for the 16PF.

Journal of Personality Assessment, 39, 532-537.

Zerbe, W. J., & Paulhus, D. L. (1987). Socially desirable responding in organizational behavior:

A reconception. Academy of Management, 12(2), 250-264.

Zickar, M . , Ross, J., & Levin, R. (1996). Modeling the effects of faking on personality scales.

Paper presented at the Eleventh Annual Conference of the Society for Industrial and

Organizational Psychology. San Diego, CA.

Zickar, M . J. (1997). Computer simulation of faking on a personality test. Paper presented at the

annual meeting of the Society for Industrial and Organizational Psychology, St. Louis, MO.

Zickar, M . J., & Robie, C. (1999). Modeling faking good on personality items: An item-level

analysis. Journal of Applied Psychology, 84(4), 551-563.

42

Table 1

Means, Standard Deviations, and Intercorrelation Matrix of the Four Measures of Change Measure of Change3 Males Females Intercorrelation Matrix

(n = 55) (n = 195) (combined gender, n = 250)

Mean SD Mean SD 1 2 3 4

1. Raw score differenceb 8.96 11.72 12.79 10.50 (.78)

2. Residual change (standardized) -.13 1.12 .04 .96 .91 (.82)

3. Phi correlation .51 .26 .49 .22 -.81 -.79 (.70)

4. Variance of the differences .23 .13 .25 .11 .81 .80 -.99 (.74)

Note: The values in parentheses along the principal diagonal o f the intercorrelation matrix represent reliability estimates. aSee the text for a description of the four measures of change. bScores on the raw-score difference variable were mean-deviated by gender (to scale out gender differences) before pooling for the correlational analysis.

43

Table 2

Factor Pattern and Factor Score Coefficient Vectors Associated with the First Principal Component.

Pattern Vector Factor Score Coefficient Vector Variables3 (Weight Vector)

1. Raw score difference .936 .263

2. Standardized residual change .927 .260

3. Phi correlation -.954 -.268

4. Variance of the differences .956 .269

Variance accounted for by first principal component: 89%. aSee the text for a description of the four measures of change.

44

Table 3

The 56 CPI Items Selected for the CPI-MD Scale (Item numbers given are for CPI Form 434) Item No. Item Statement

True:

5 Our thinking would be a lot better off if we would just forget about words like "probably," approximately," and "perhaps."

14 I always follow the rule: business before pleasure

52 I usually take an active part in the entertainment at parties.

65 I think I would like the work of a clerk in a large department store.

69 I would disapprove of anyone's drinking to the point of intoxication at a party.

167 I would like to belong to several clubs or lodges.

179 When I work on a committee I like to take charge of things.

195 The most important things to me are my duties to my job and to other people.

202 If given the chance, I would make a good leader of people.

218 I love to go to dances.

222 I would like to belong to a discussion and study club.

223 I keep out of trouble at all costs.

226 People seem naturally to turn to me when decisions have to be made.

239 I like to talk before groups of people.

254 I have never deliberately told a lie.

255 Only a fool would try to change our American way of life.

278 If I get too much change in a store, I always give it back.

292 I used to like it very much when one of my papers was read to the class in school.

305 I often wish people would be more definite about things.

319 In a group, I usually take the responsibility for getting people introduced.

345 I go out of my way to meet trouble rather than try to escape it.

354 A strong person will be able to make up his or her mind even on the most difficult questions.

355 I have strong political opinions.

359 I think I am usually a leader in my group.

361 I like to have a place for everything and everything in its place.

370 Disobedience to any government is never justified.

371 I can't really enjoy a rest or vacation unless I have earned it by some hard work.

376 I enjoy planning things and deciding what each person should do.

382 Success is a matter of will power.

392 I daydream very little.

403 I have a natural talent for influencing people.

408 I always see to it that my work is carefully planned and organized.

424 The one to whom I was most attached and whom I most admired as a child was a woman (mother, sister, aunt, or other woman).

(table continues)

45

Table 3 (Continued)

The 56 CPI Items Selected for the CPI-MD Scale (Item numbers given are for CPI Form 434) Item No. Item Statement

False:

30 I gossip a little at times.

38 It is hard for me to start a conversation with strangers.

55 Some of my family have quick tempers.

66 Sometimes I feel like swearing.

91 Sometimes I think of things too bad to talk about.

124 I am likely not to speak to people until they speak to me.

139 It is all right to get around the law if you don't actually break it.

151 I'm not the type to be a political leader.

156 I hardly ever get excited or thrilled.

170 I often act on the spur of the moment without stopping to think.

191 I can remember "playing sick" to get out of something.

203 When things go wrong, I sometimes blame the other person.

238 Sometimes I just can't seem to get going.

258 In school I found it very hard to talk before the class.

262 There have been a few times when I have been very mean to another person.

268 At times I have been very anxious to get away from my family.

275 Sometimes I rather enjoy going against the rules and doing things I'm not supposed to do.

313 I dislike to have to talk in front of a group of people.

334 I get nervous when I have to ask someone for a job.

379 I would rather not have very much responsibility for other people.

414 I do not read every editorial in the newspaper every day.

416 I don't think I'm quite as happy as others seem to be.

418 I am embarrassed with people I do not know well.

426 I get tired more easily than other people seem to.

Note. Modified and reproduced by special permission of the Publisher, Consulting Psychologists Press, Inc., Palo Alto, CA 94303 from California Psychological Inventory™, Form 434 by Harrison G. Gough. Copyright 1987 by Consulting Psychologists Press, Inc. All rights reserved. Further reproduction is prohibited without the Publisher's written consent.

46

Table 4

Normative Data for the CPI-MD Scale Norm Group3 n M s

Undergraduate University Students

Males 350 23.47 6.83

Females 739 23.07 6.20

Management-Level Employees

Males 245 29.00 7.62

Females 212 28.32 7.05

Non-Management-Level Job Applicants

Males 193 35.11 8.05

Females 335 34.19 7.58

Note: The tabled means and standard deviations are for the 56-item CPI-MD scale described in the text, composed of the items listed in Table 3. aThese norm samples are obtained from the Hakstian et al. (2002) study. See the text for demographic details about each norm group listed above.

47

Table 5

Internal-Consistency (Alpha) Reliability Estimates, rtt, and Standard Errors of Measurement, se, for the CPI-MD Scale in Three Separate-Gender Samples

Sample n a sa r t t se

Undergraduate Students

Males

Females

Management-Level Employees

Males

Females

Non-Management-Level Job Applicants

Males 165 8.11 .86 3.03

Females 286 7.52 .83 3.10 aSample sizes and standard deviations differ slightly in some cases from those in Table 4 because only those subjects who had responded to every one of the 56 items of the CPI-MD scale were included in the analyses of internal-consistency reliability.

339 6.87 .77 <> 3.29

691 6.21 .72 3.29

184 7.72 .83 3.18

180 7.19 .81 3.13

48

Table 6

Correlations of Scores on the CPI-MD Scale with Scores on the Five NEO PI-R Domain Scales and the NEO PI-R Neuroticism, Extraversion and Conscientiousness Facet Scales.

Males (n = 98) Females (n = 222) Scale Correlated with CPI-MD Bivariate r Disattenuated" r Bivariate r Disattenuated" r NEO PI-R Domain Scales

Neuroticism -.32* -.41* -.40* -.49* Extraversion .51* .66* .49* .61* Openness .10 .13 .07 .09 Agreeableness -.06 -.08 -.02 -.03 Conscientiousness .36* .46* .45* .54*

Neuroticism Facet Scales Anxiety -.17 -.24 -.24* -.32* Angry Hostility -.16 -.24 -.17 -.22 Depression -.20 -.34 -.35* -.44* Self-Consciousness -.34* -.49* -.43* -.62* Impulsiveness -.16 -.24 -.25* -.38* Vulnerability -.34* -.48* -.41* -.55*

Extraversion Facet Scales Warmth .35* .47* .36* .48* Gregariousness .37* .51* .27* .37* Assertiveness .61* .84* .65* .84* Activity .34* .50* .36* .66* Excitement-Seeking .12 .20 .02 .04 Positive Emotions .34* .47* .35* .46*

Conscientiousness Facet Scales Competence .24 .38 .39* .54* Order .15 .22 .36* .49* Dutifulness .20 .34 .31* .47* Achievement Striving .38* .53* .41* .55* Self-Discipline .37* .52* .35* .45* Deliberation .09 .14 .22* .29*

"These correlations are fully disattenuated on the basis of the specific internal-consistency reliability estimates obtained in these samples. *p < .01 (non-directional).

49

Table 7

Descriptive Statistics and Correlation of the CPI-MD Scale andMD Criterion Scores. Mean SD

Measure (Variable in Regression) (n = 250) (w = 250)

Motivational distortion criterion (Y) 11.95 10.87

CPI-MD scale (X) 35.82 9.26

Correlation, r^, = .543

50

Table 8

Comparison of Means, Standard Deviations, and Correlations between Honest-Condition/Unadjusted and Adjusted CPI-Cp scores in Three Separate-Gender Samples

1. Undergraduate Student Sample3

Males Females (n = 55) (n = 195)

Type of CPI-Cp Score Mean SD Mean SD 1. Honest Condition 35.53 13.67 34.08 10.98 2. Adjusted 36.53 10.08 33.80 7.67 Correlation (r^) .70 .64

2. Management Level Employees Sample


Type of CPI-Cp Score Mean SD Mean SD 1. Unadjusted 28.39 9.19 28.45 8.07 2. Adjusted 35.99 8.67 35.62 7.57

Correlation (ri 2) .85 .83

3. Non-Management Level Job-Applicants Sample


Type of CPI-Cp Score Mean SD Mean SD 1. Unadjusted 23.94 9.98 21.62 9.63 2. Adjusted 35.43 7.64 32.53 8.34

Correlation (ri 2) .86 .86

'This is the sample in which the CPI-MD scale was developed.

51

Table 9

Correlations of Adjusted and Unadjusted CPI-Cp Conscientiousness Facet Scale Scores

Scale Scores with NEO PI-R Domain and

Males (n = 98) Females (n = 222) NEO PI-R Scale Correlated with Unadjusted Adjusted Unadjusted Adjusted CPI-Cp CPI-Cp CPI-Cp CPI-Cp CPI-Cp Domain Scales

Neuroticism .17 .05 .45* .30* Extraversion .39* .58* .20* .39* Openness .19 .23 .16 * .19* Agreeableness -.33* -.35* -.44* -.45* Conscientiousness -.38* -.24 -.60* -.44*

Conscientiousness Facet Scales Competence -.02 .07 -.43* -.28* Order -.18 -.12 -.31* -.17* Dutifulness -.37* -.29* -.52* -.41* Achievement Striving -.15 -.01 -.37* -.22* Self-Discipline -.29* -.15 -.53* -.41* Deliberation -.51* -.47* -.59* -.52*

*p < . 01 (nondirectional).

52

Table 10

Correlations between the CPI-Cp Scale Scores (with and without adjustments for motivational-distortion) and Counterproductive-Behavior Criteria in Two Non-Management Level Samples

1. Telephone service representatives (n = 79, combined genders)

Counterproductive-Behavior Criterion Form of Criterion Correlation Availability Desire to Use of Time Overall

for Work Improve Performance Unadjusted Bivariate r -.23* _ 2 7 * * -.26* -.23* Unadjusted True Validity 3 -.31* -.36** -.35* -.32* Adjusted Bivariate r -.26* _ 2 7 * * * _ 29** -.36*** Adjusted True Validity 3 -.35* _ 4 9 * * * _ 3 9 * * -.50***

2. Armored-car company employees (n = 91, combined genders)

Counterproductive-Behavior Criterion Form of Criterion Correlation Bivariate correlation Semi-partial correlationb

Unadjusted Bivariate/Semi-partial rh .25** .26* Unadjusted True Validity 3 .33** .35*

Adjusted Bivariate/Semi-partial r° .28** .25** Adjusted True Validity 3 .37** .33**

Note: *p < .05; **p < .01; ***p < .001 (all directional). Criterion correlations for the unadjusted CPI-Cp scores are taken from Hakstian et al. (2002).

3These true validities are the bivariate (or semi-partial) correlations corrected for attenuation in the criterion variable only by means of the inter-rater reliability estimates discussed in the text.

bLength of tenure was partialed out of the criterion variable in the CPI-Cp (adjusted and unadjusted) vs. criterion correlation since scores on this criterion were a function of the number of counterproductive work incidents recorded in employees' files over the tenure of their employment (see text for more detail).

53

Appendix A - Analyses to Determine the Effectiveness of the Experimental Procedure in Study 1

As identified within the body of the thesis, there were two main concerns with regard to

the effectiveness of the experimental procedure in Study 1. First, there were concerns that the

instructions given to simulate an employment context might not have been sufficiently potent to

have elicited motivational distortion for a sample consisting of undergraduate students for whom

the personality test results had effectively no consequence.

The second concern was with regard to the lack of randomization in the administration

of experimental conditions to the subjects. Instead, all subjects took the CPI under the

employment condition followed by the honest condition. As indicated in the body of the thesis,

the intention was to closely simulate employment conditions in which prior viewing of the

personality test would not have taken place. However, without controlling for the effects of

experimental order, there was concern that the honest condition might not truly elicit honest

responding, but was in some way eliciting contaminated responses, by having been preceded by

the employment condition.

The above two concerns can be summarized in the following two research questions:

1. Were the CPI-Counterproductivity scale scores obtained in the employment

condition significantly lower than those obtained in the honest condition? (The

employment condition represented the condition for motivational distortion and it

would be expected that subjects would tend to lower their counterproductivity

scores.)

2. Were the CPI-Counterproductivity scale scores obtained in the honest condition

comparable to a normative score distribution for undergraduate students?

If the research provided affirmative support for the above two questions, there would be

sufficient evidence to suggest that (1) the employment-condition instructions were able to elicit

54

motivational distortion, and (2) responses to the CPI in the honest condition were not

significantly contaminated by prior exposure to the inventory under a different test-taking

condition.

Method and Results

For research question 1, using the U B C sample from Study 1, separate-gender paired

samples 7-tests were conducted to determine whether CPI-Cpempioyment mean scores were lower

than CPI-Cphonest mean scores. The separate-gender means and standard deviations for the CPI-

Cp scores in the employment and honest conditions appear in Table A l . Inferential tests showed

significant differences for both genders: for males, 7(54) = -5.67, p < .0001, A = -.771, .95 C.I.

for A = (-1.044, -.499); for females, 7(194) = -17.02, p< . 0001, A = -1.311, .95 C.I. for A =

(-1.463, -1.159). Therefore, there is evidence to suggest that the experimental manipulation used

in Study 1 successfully elicited motivational distortion.

Insert Table A l about here

For research question 2, CPI-Cp mean scores obtained from the honest condition in Study

1 were compared with a large undergraduate student sample (UBC Sample 1 from Study 2—350

males, 739 females), separately by gender. The incidence of motivational distortion in this

normative sample can be assumed to be minimal as the student subjects had been asked to

complete the CPI honestly, for research purposes. The descriptive statistics appear in Table A2.

Insert Table A2 about here

Inferential test results showed non-significant mean differences for both males (p > .10) and

females (p > .25). Therefore, there is insufficient evidence to suggest that preceding the honest

condition with the employment condition affected the integrity of the CPI data obtained in the

honest condition.

55

Table A l

Means and Standard Deviations of the CPI-Cp Scores in the Employment and Honest conditions of the Experimental Sample

Males Females ( « = 55) (n = 195)

Experimental Condition Mean SD Mean SD

Employment 26.56 11.63 21.29 9.76

Honest 35.53 13.67 34.08 10.98

56

Table A2

Descriptive Statistics of the CPI-Cp scores in the Experimental Sample and a Normative Undergraduate Student Sample.

Males Females

Sample N Mean SD N Mean SD

Experimental 55 35.53 13.67 195 34.08 10.98

Normative 350 38.03 11.32 739 34.96 10.29

57

Appendix B - Administration Instructions for the Two Experimental Conditions in Study 1

The full instructions for the employment condition are as follows:

I will be describing to you a scenario and would like you to put yourself into that scenario when

you are responding to the personality test. Imagine that you are applying for a job that you would

like to have. This could be a career that you are pursuing or a job that you are currently applying

for. After sending in your application form, you are informed by the company that part of their

hiring procedure requires that you complete some paper-and-pencil tests. During the test session

at their office, you are requested to complete this personality inventory (hand the CPI to the

subject at this point). The HR representative informs you that your responses on the inventory

will provide valuable information about yourself that will be used as part of the decision-making

process. The personality inventory consists of 434 true/false statements. It will take you about

lhr to complete this and you will need to respond to every item. If you have no questions, please

begin. Remember to respond to the items as you would if you were applying for this job.

The full instructions for the honest condition are as follows:

You will notice that this is the same personality inventory that you completed last week. In that

session, you were asked to complete the inventory as you would if you were applying for a job

that you really want. It is normal for people who are applying for a job to respond to personality

questionnaires in a different way from how they would normally. This might or might not have

been true for you. Regardless, this time round, I would like you to respond to the statements as

you truly see yourself to be. You are assured once again that the information we gather from you

in this experiment will be kept confidential and only used for research purposes. In addition, the

feedback that you will receive on your personality profile will be based on your responses to the

inventory in today's session. Therefore, in order for this feedback to be as accurate as possible,

you are encouraged to be respond to the statements in the inventory as honestly as you can.

58

Appendix C - Additional Information about the Phi Correlation and Variance of the Differences

Measures of Change (based on Lautenschlager, 1986)

Phi Correlation Measure

As the CPI comprises true-false items, a 2 x 2 contingency table for each subject can be

set up, with the response choices for the employment condition represented by the rows and the

honest condition by the columns. As an example, one cell, say the "True-True" cell would

contain the number of items on which this subject answered True on both administrations. A phi

correlation coefficient can therefore be obtained for each subject where large positive values

indicate greater consistency of responses or more specifically, less distortion of responses in the

employment condition. As the phi coefficient tends towards zero or becomes negative, this

would indicate greater amounts of response shifts or motivational distortion. Therefore, this

within-subject Phi Correlation Measure provides an indication of the consistency of a subject's

responses to the CPI-Cp scale items between the employment and honest conditions.

Variance of the Differences Measure

The Variance of the Differences measure represents the within-subject variance of the

differences in the responses to each of the 80 CPI-Cp items taken in the two response

conditions—employment and honest. In the present study, CPI items were coded 1 for true and 2

for false. A difference in "scores" (honest - employment) was computed for each of the 80 CPI-

Cp items. A difference of "0" is obtained if responses on that item remained consistently true or

false between the two conditions, a " - 1 " i f the responses changed from true to false, and a "+1"

if the responses changed from false to true. Each subject would obtain a total of 80 separate item-

level difference scores (corresponding to the 80 CPI-Cp items) and the variance of these

differences was termed the within-subject Variance of the Differences measure. Large values

59

would indicate more shifts in responses across the 80 items and hence a greater amount of

motivational distortion having taken place in the employment condition. Smaller values would

indicate consistency in responding or less distortion.

For a more detailed review of the use of these two measures as within-subject measures

of individual differences in motivational distortion, the reader is referred to the original source,

Lautenschlager (1986).

6 0

Appendix D: Estimation of Reliability for the Four Change Measures and the Composite

Motivational Distortion Criterion

From Williams, Zimmerman, and Mazzagatti (1987), the reliability of simple difference

scores (rdd) and residualized difference scores (r r e s) can be calculated using the following

equations:

Si 2 ~t~ S2 2 — 2ri2SiS2 '

_ T22 + Ti2 2 T n — 2f i2 2

^es — 1 2 > I — In

where si and s2 represent the standard deviation of the CPI-Cp scale scores obtained in the honest

and employment condition, respectively, rn and r 2 2 represent the internal consistency reliability

estimates of the CPI-Cp scale scores obtained in the honest and employment condition,

respectively, and r i 2 represents the correlation between honest-condition and employment-

condition CPI-Cp scores.

For the Phi Correlation Measure, split-half reliability estimates were used as indicated in

the body of the thesis. The 80 CPI-Cp items were separated into two groups of 40 items. Taking

the first 40 items, a phi coefficient was computed for each subject that indicated his/her

response-consistency to these items between the two test conditions—employment and honest. A

second phi coefficient was similarly computed using the last 40 items. Therefore, each subject

would receive two separate phi coefficient values and the correlation between these two sets of

coefficients (across all subjects) represented a measure of split-half reliability.

For the Variance of the Differences Measure, a similar procedure of obtaining split-half

reliability estimates was used. For each subject, two separate variances were obtained from the

two different groups of 40 CPI-Cp items. The correlation of these two variances, across all

subjects, constituted the split-half reliability estimate.

61

Finally, the reliability estimate, r?c, of the aggregated M D criterion measure was

calculated using the standard equation for estimating the reliability of a differentially-weighted

linear combination of measures, each of whose reliabilities are known (see, e.g., Horst, 1966,

pp.280-281). As the M D criterion scores were the first principal component scores of the four

change measures, the weight vector was simply the weight vector for obtaining the first principal

component from standard scores on the constituent variables. The formula for r P C is as follows:

a'D,„2Dua rPC = 1 - - ; ,

a DaxRxxD„xa

where Du = I - D r i i , D r i i is a diagonal matrix of the reliabilities of the measures, a is the first

principal component weight vector, D 0 x is the diagonal matrix of the standard deviations of the

measures, and R x x is the matrix of intercorrelations of the four measures. In the present case, D C x

was simply the identity matrix as the first principal component was a (0, 1) variable prior to

rescaling into the metric of the raw-change scores.

62

Appendix E: Correlations between the Motivational Distortion Criterion and the 20 Standard CPI

Scales

Correlation with M D Criterion U B C Undergraduates

Males Females CPI (Form 434) Standard Scale (w = 55) (w = 195)

Dominance .34* .30*

Capacity for Status .15 .19*

Sociability .26 .25*

Social Presence .03 -.01

Self-Acceptance .07 .16*

Independence .05 .09

Empathy .10 .05

Responsibility .07 .19*

Socialization -.01 .11

Self-Control -.00 .17*

Good Impression .18 .31*

Communality -.06 -.04

Well-Being .00 .16*

Tolerance -.01 -.00

Achievement via Conformance .16 .24*

Achievement via Independence -.09 -.09

Intellectual Efficiency -.00 .06

Psychological Mindedness .01 -.04

Flexibility -.23 -.24*

Femininity/Masculinity .09 .03 *p < .05 (nondirectional).

63

Appendix F: Two Alternative Methods of Deriving Adjustment Equations to Correct for

Motivational Distortion in CPI-Cp Scores

Two alternative methods of developing an adjustment equation to correct CPI-Cp scores

for motivational distortion, were considered. As mentioned in the thesis body, scores obtained

from these two methods correlated almost perfectly (average r - .99) with scores obtained with

the method of adjustment presented in the thesis body. The high correlation amongst the three

methods is not surprising as the three separate methods of adjustments are, in effect, simply

different linear combinations of the CPI-MD and CPI-Cp scale scores. A l l three methods

involved a prediction of the amount of motivational distortion contained within a set of CPI data,

and adding this predicted amount of distortion to original CPI-Cp scores. The difference lay in

the means with which the prediction of motivational distortion was derived.

The concept behind these two alternative methods are presented here along with the

resulting adjustment equations. In both cases, the equations were based on data from the

experimental sample of Study 1.

Alternative Method 1

First, a regression equation for predicting employment-condition CPI-Cp scale scores

(CPI-Cpempioyment) from CPI-MD scale scores was obtained, separately by gender (due to gender

differences on the latter scale). Substituting the appropriate means, standard deviations and

correlation coefficient for the CPI-MD scale and CPI-Cpempioyment scores from Table F l into

Equation 1 in Study 3, the separate-gender raw-score regression equations are:

For males: Y = -.739X + 50.732;

For females : Y = -.626X + 44.252,

where Y represents the predicted CPI-CPempi0yment score and X , the CPI-MD scale score.

64

Insert Table F l about here

The residuals about this regression line (Y - Y) represent the portion of CPI-Cpempioyment scores

that cannot be predicted by CPI-MD scale scores and therefore, can be construed as a more

honest approximation of CPI-Cp scores. Correspondingly, the separate-gender adjustment

equations are simply the equations for obtaining these residual scores:

For males: Adjusted CPI-Cp = Original CPI-Cp + .739 (CPI-MD) - 50.732;

For females: Adjusted CPI-Cp = Original CPI-Cp + .626 (CPI-MD) - 44.252.

It should be noted that this method of adjustment results in an overall mean of zero for

the adjusted CPI-Cp scale scores. To obtain mean scale scores that are more reflective of honest

scores, the results from the adjustment equations can be rescaled to a different metric.

Alternative Method 2

In this method, a generic adjustment equation that could be applied to both genders, was

developed. As before, a regression equation was constructed for predicting employment-

condition CPI-Cp scale scores (CPI-Cpempioyment) from CPI-MD scale scores. However, in this

method, the means and standard deviation for the Y variable in the regression equation were

based on that of the raw-score difference metric (Y = 11.95 and Sy = 10.87). (Note: the raw-

score difference is one of the four change measures used in Study 1). Substituting these Y -

values, together with the total-group means, and standard deviations for the CPI-MD scale, and

the correlation of-.587 between the CPI-CPempioyment and CPI-MD scores from Table E l , the raw

score regression equation is: Y = -.690X + 12.77, where Y and X represent the predicted CPI-

CPempioyment score and CPI-MD scale score, respectively.

The generic adjustment equation would therefore be:

Adjusted CPI-Cp = Original CPI-Cp + .690 (CPI-MD scale score) - 12.77.

65

Table F l

Means and Standard Deviations of the Employment-Condition CPI-Cp and CPI-MD Scale Scores

Males Females Total (n = 55) (« = 195) in = 250)

CPI-scale Mean SD Mean SD Mean SD

CPI-Cp (Y) 26.56 11.63 21.29 9.76 22.45 10.41 CPI-MD (X) 32.71 9.24 36.70 9.16 35.82 9.26 Correlation, r^ (combined genders)8 = -.587 CPI-MD (Honest condition) 25.20 7.92 25.41 7.00 25.36 7.20 (for comparative purposes)

a The pooled-gender correlation was obtained between the CPI-Cp and CPI-MD scale scores with scores on both scales mean-deviated by gender, to scale out mean differences. A pooled-gender correlation provided a more stable estimate and was preferred over separate-gender correlations.

66

Appendix G: Correlations of Adjusted and Unadjusted CPI-Cp Scale Scores with Scores on the

NEO PI-R Domain and Facet Scales

Males (n = 98) Females (n = 222) NEO PI-R Scale Correlated with Unadjusted Adjusted Unadjusted Adjusted CPI-Cp CPI-Cp CPI-Cp CPI-Cp CPI-Cp Domain Scales

Neuroticism .17 .05 .45* .30* Extraversion .39* .58* .20* .39* Openness .19 .23 .16 .19* Agreeableness -.33* -.35* -.44* -.45* Conscientiousness -.38* -.24 -.60* -.44*

Neuroticism Facet Scales Anxiety .11 .05 .21* .12 Angry Hostility .28* .22 .27* .42* Depression .05 -.03 .34* .21* Self-Consciousness -.06 -.18 .15 -.01 Impulsiveness .37* .30* .43* .34* Vulnerability .01 -.11 .41* .26*

Extraversion Facet Scales Warmth .18 .30* -.04 .10 Gregariousness .37* .49* .20* .31* Assertiveness .27* .48* .09 .34* Activity .24 .36* .05 .20* Excitement-Seeking .44* .48* .43* .45* Positive Emotions .19 .31* .08 .22*

Openness Facet Scales Fantasy .32* .30* .32* .28* Aesthetics .11 .15 .09 .14 Feelings .26* .32* .18* .23* Actions .13 .13 .11 .14 Ideas .04 .09 -.09 -.05 Values -.01 -.03 .02 -.01

Agreeableness Facet Scales Trust -.04 -.05 -.24* -.23* Modesty -.51* -.53* -.45* -.45* Compliance -.00 .06 -.31* -.23* Altruism -.33* -.36* -.40* -.43* Straightforwardness -.25 -.32* -.14 -.24* Tender-mindedness -.14 -.18 -.15* .-16

(table continues)

67

Appendix G (Continued)

Males (n = 98) Females (n = 222) NEO PI-R Scale Correlated with Unadjusted Adjusted Unadjusted Adjusted CPI-Cp CPI-Cp CPI-Cp CPI-Cp CPI-Cp Conscientiousness Facet Scales

Competence -.02 .07 -.43* -.28* Order -.18 -.12 -.31* -.17* Dutifiilness -.37* -.29* -.52* -.41* Achievement Striving -.15 -.01 -.37* -.22* Self-Discipline -.29* -.15 -.53* -.41* Deliberation -.51* -.47* -.59* -.52*

*p < .01 (non-directional).

68

Appendix H - Criterion-Related Validity Analyses for Adjusted CPI-Cp Scores with

Adjustments Made on the Basis of the CPI Fake-Good Scale

A motivational-distortion adjustment equation for CPI-Cp scale scores was developed

on the basis of the CPI-FG scale scores. Using this set of adjusted CPI-Cp scale scores, the

criterion-related validity of the CPI-Cp scale was re-assessed and compared with the results

presented in the body of the thesis in which adjustments were based on the CPI-MD scale. This

additional analysis provides a comparison of the two motivational distortion scales in terms of

their relative effectiveness in improving criterion-related validity.

Development of the Adjustment Equation

Using an identical developmental procedure to that described in Study 3, a regression

equation was constructed for predicting M D criterion scores from the CPI-FG scale scores. On

the basis of pooled-gender data from U B C Sample 4 (« = 250), the mean and standard deviation

for the CPI-FG scale were 56.88 and 4.18 respectively; the mean and standard deviation for the

M D criterion were 11.95 and 10.87; the correlation between the M D criterion and CPI-FG scale

scores was .365. With these values, the raw score regression equation for predicting the M D

criterion from CPI-FG scale scores is: Y = .949X - 42.024, where Y and X represent the

predicted M D criterion score and the CPI-FG scale score, respectively. As Y is an indication of

the amount of motivational distortion within a set of CPI data obtained under employment-

conditions, adding these predicted scores to the original CPI-Cp scale scores would render a

more honest estimate of the latter. Therefore, the adjustment equation is:

Adjusted CPI-Cp = Original CPI-Cp + .949 (CPI-FG scale score) - 42.024.

Criterion-Related Validity Analyses

Using the adjustment equation above, CPI-Cp scores were corrected for motivational

distortion and using these adjusted scores (FG-adjusted CPI-Cp), the criterion-related validity of

69

the CPI-Cp scale was reassessed in the two employee samples identified in the main paper. The

correlations between the FG-adjusted CPI-Cp scores and related job-performance criteria appear

in Table HI , along with the bivariate criterion correlations obtained using unadjusted CPI-CP

scores and the previous set of adjusted CPI-Cp scores (CPI-Cp scale scores adjusted for

motivational distortion using the CPI-MD scale; MD-adjusted CPI-Cp).

Insert Table HI about here

For the sample of telephone service representatives, the FG-adjusted CPI-Cp scores

showed a significant increase in criterion correlations for the Desire to Improve [7(76) = 1.804, p

< .05] and Overall Performance criteria [t(16) = 2.275, p < .025], when compared to unadjusted

CPI-Cp scores. These results are similar to that obtained using MD-adjusted CPI-Cp scores. For

the armored-car employees sample, FG-adjusted CPI-Cp scores did not show any change in the

bivariate criterion correlations when compared with unadjusted scores. This contrasts with M D -

adjusted scores which showed a slight, albeit non-significant, increase in criterion correlations.

When semi-partial correlations are compared, no differences between unadjusted, MD-adjusted

and FG-adjusted CPI-Cp scores were observed.

Overall, the pattern of criterion correlations suggest that criterion-related validity was

increased when CPI-Cp scores were adjusted for motivational distortion and that the magnitude

of increase tended to be relatively larger for the MD-adjusted scores when compared to the FG-

adjusted scores, although the differential improvements between the two methods of adjustments

were not statistically significant at the .05 level. More research is needed, using larger data

samples, to provide more conclusive results.

70

Table H I

Correlations between the CPI-Cp Scale Scores (With and Without Two Different Adjustments for Motivational-Distortion) and Counterproductive-Behavior Criteria in Two Non-Management Level Samples

1. Telephone service representatives (« = 79, combined genders)

Counterproductive-Behavior Criterion

Predictor Scale Score Availability Desire to Use of Time Overall for Work Improve Performance

Original CPI-Cp -.23* -.27** -.26* -.23*

MD-Adjusted CPI-Cp a _ 26* — 37*** _ 29** — 36***

FG-Adjusted CPI-Cp b -.25* -.36*** -.26* -.34**

2. Armored-car company employees (n = 91, combined genders)

Counterproductive-Behavior Criterion

Predictor Scale Score Bivariate correlation Semi-partial correlation0

Original CPI-Cp .25* .26*

MD-Adjusted CPI-Cp a .28** .25**

FG-Adjusted CPI-Cp b .25* .25*

Note: *p < .05; **p < .01; ***p < .001 (all directional). Criterion correlations for the unadjusted CPI-Cp scores are taken from Hakstian et al. (2002).

aThe MD-adjusted CPI-Cp scale refers to CPI-Cp scale scores that have been adjusted for motivational distortion using the adjustment equation obtained on the basis of the CPI-MD scale.

bThe FG-adjusted CPI-Cp scale refers to CPI-Cp scale scores that have been adjusted for motivational distortion using the adjustment equation obtained on the basis of the CPI manual's fake-good scale.

°Length of tenure was partialed out of the criterion scores in the CPI-Cp (adjusted and unadjusted) vs. criterion correlation since scores on this criterion were a function of the number of counterproductive work incidents recorded in employee's files over the tenure of their employment (see text for more detail).

the development of a motivational distortion scale

Documents