Comparability of Personality Self‐Ratings Comparability of Personality Self‐Ratings AcrossAcross CulturesCultures

Two happy-ending storiesTwo happy-ending stories

René Mõttus, Jüri Allik, Anu RealoRené Mõttus, Jüri Allik, Anu Realo

University of TartuUniversity of Tartu

The problem

When subjective ratings are used for cross-cultural comparisons, researchers sometimes do not like the results

It's not only in psychology:

Chinese have more political freedom than Mexicans?

People live longer where they describe themselves being more ill?

The personality example: Conscientiousness

Conscientiousness ~ tendency of being organized, dependable, purposeful, controlled

The Distribution of Conscientiousness across the World:

– The highest scoring countries – Senegal, Benin

– The lowest scoring countries – Japan, South-Korea

High Conscientiousness predicts low GDP and poor health

“I am a reliable person” “I am often late”

The same responses to personality test items may refer to different trait levels in different cultures (Heine et al., 2008)

The same trait-related information is “translated” into ratings in different ways

Culture-sepcific standards for the behaviours that characterize conscientiousness:

– Being “on time” means a 25min time-window in US but 39min in Morocco (White, Valk & Dialmy, in press)

As a result, in some cultures, self-ratings tend to be low simply because everyone is rated according to harsh standards, and vice versa

Is there anything we can do?

We should make the differences in standards measurable

Having measured the standards, we could correct self-ratings accordingly

Haven't been done too often in cross-cultural personality

studies

A simple solution, and cheap as chips

The technique of anchoring vignettes (King et al., 2004)

We have to ask everyone to rate something (relevant) that is the same for everyone:

Short descriptions of hypothetical persons displaying various levels of the trait (called anchoring vignettes)

The vignettes are the same for everyone, thus any variance in their ratings can only reflect biases and error

If the ratings differ systematically across cultures, this probably shows culture-specific standards for the trait

Moreover, vignettes can do magic

Self-ratings can be “fixed” by anchoring them to vignette-ratings:

Vignettes are the common norm/baseline for everyone

Self-ratings can be anhored to the common norm

Anchored self-ratings are free from culture-specific standards, being thus more easily comparable

Recoded self-fatings y ~ vign

1 y < vign1

2 y = vign1

3 vign1 < y < vign2

4 y = vign2

5 vign2 < y < vign3

6 y = vign3

7 vign3 < y < vign4

8 y = vign4

9 vign4 < y < vign5

10 y = vign5

11 y > vign5

The first study ...

… with

Helle Pullmann, Jerome Rossier, Gregory Zecca, Jennifer Ah-Kion, Dénis Amoussou-Yéyé, Martin Bäckström, Rasa Barkauskiene,

Oumar Barry, Uma Bhowon, Fredrik Björklund, Aleksandra Bochaver, Konstantin Bochaver, Deon de Bruin, Helena Cabrera, Sylvia

Xiaohua Chen, A. Timothy Church, Dougoumalé Cissé, Daouda Donatien Dahourou, Xiaohang Feng, Yanjun Guan, Hyi-Sung Hwang, Fazilah Idris, Marcia S. Katigbak, Peter Kuppens, Anna Kwiatkowska,

Alfredas Laurinavicius, Khairul Anwar Mastor, David Matsumoto, Rainer Riemann, Joanna Schug, Brian Simpson, Caroline Ng Tseung

Country Language N

Australia English 463

Benin French 107

BurkinaFaso French 96

China (Changchun) Chinese 110

China (Beijing) Chinese 150

Estonia Estonian 110

Germany German 70

Hong-Kong Chinese 158

Japan Japanese 107

Lihtuania Lihtuanian 125

Malaysia Malay 211

Mali French 93

Mauritius French 100

Philippines Filipino 133

Poland Polish 100

Russia Russian 100

Senegal French 115

South-Africa English 109

South-Korea Korean 142

Sweden Swedish 100

Switzerland French 101

USA English 165

21 cultures,22 samples2,965 respondents

Capable, efficient, competent __ __ __ __ __ Inept, unprepared

Disorganized, sloppy __ __ __ __ __ Organized, neat, methodical

Dutiful, scrupulous __ __ __ __ __ Unreliable, undependable

Lazy, unambitious, aimless __ __ __ __ __ Ambitious, workaholic

Disciplined, persistent, strong-willed __ __ __ __ __ Procrastinating, quitting, weak

Spontaneous, careless, thoughtless __ __ __ __ __ Cautious, reflective, careful

I am ...

The person is ...

[Marc] often feels incapable of deciding and finding solutions to his problems. He always turns to his relatives and acquaintances for help and sometimes they indeed help him. However, at times the opinions of other people disagree, which makes it even more difficult for [Marc] to work out what he should do.

[30 vignettes displaying various levels of conscientiousness]

We recoded self-ratings in relation to vignette-ratings

– recoding expected to correct for the effect os potentially differing standards

Country rankings on self-rated Conscientiousness before and after correcting

– rank-order correlations from 0.78 to 0.92 (median 0.86)

Before and after correcting self-ratings

Rankings of cultures on Dutifulness

Before correcting After correcting1 Benin Benin2 Burkina Faso Burkina Faso3 Senegal Senegal4 China (Changchun) China (Changchun)5 China (Beijing) Mali6 Mali China (Beijing)7 South-Africa Poland8 Poland Estonia9 Malaysia South-Africa

10 Mauritius Malaysia11 Sweden Switzerland12 Philippines Philippines13 Germany USA14 Switzerland Mauritius15 USA Sweden16 South-Korea Lithuania17 Hong Kong Australia18 Estonia Russia19 Australia Germany20 Lithuania Hong Kong21 Russia Japan22 Japan South-Korea

The troubled children

Life-expec tancy

Com

pete

nce

Ord

er

Dut

ifuln

ess

Ach

ieve

men

t Stri

ving

Sel

f-Dis

cipl

ine

Cau

tious

ness

-0 ,8

-0 ,4

0,0

U n co rre cte d s e lf-ra tin g s C o rre cte d s e lf-ra tin g s

GD P

Com

pete

nce

Ord

er

Dut

ifuln

ess

Ach

ieve

men

t Stri

ving

Sel

f-Dis

cipl

ine

Cau

tious

ness

-0,8

-0,4

0,0

U n co rre cte d s e lf-ra tin g s C o rre cte d s e lf-ra tin g s

Something else must be twisting the rankings of cultures on Conscientiousness (different self-enhancement)

Or rankings of cultures are accurate, but our expectations about them are wrong*

Not much evidence for the effect of differing standards

* If our data does not support our hypotheses, though highly unlikely, it is sometimes possible that we started with incorrect hypothesis

A closer look on the predictive validity of culture-level mean Conscientiousness scores:

– Is the situation that bad afterall?

The second study

Conscientiousness has only been seen as a monolothic construct

– While it also has facets

The selection of criteria has not been rigorous enough:

– Theoretically unclear why high GDP should be related to high Conscientiousness

– Criteria not representative of populations

Previous studies haven't been good enough

Used facets in addition to domain scores

Used a larger sample of criteria:

– If we are not sure about the relationships of single criteria to aggregate Conscientiousness, let's try multiple criteria and hope to see at least some meaningful relationships

– Criteria from different categories:

• Indicative of behaviours of individuals (CVD prevalence)

• Indicative of societies in more general and abstract ways (economic freedom)

What we did (better, of course)

Clear hypotheses: predicted correlations

Mõttus, Allik, & Realo, 2010

Mõttus, Allik, & Realo, 2010

Nearly 30% of the correlations significant at p < 0.01

Different facets had different patterns of correlations to the criteria*

Mõttus, Allik, & Realo, 2010* Goodbye, Conscientiousness ...

In some facets, the observed correlations matched the predicted correlations to moderate degree

In some facets, the observed correlations contradicted the predicted correlations

Mõttus, Allik, & Realo, 2010

A closer look at the relationship of culture-level mean Conscientiousness scores to supposedly relevant criteria showed that sometimes personality scores predict criteria in an expected manner

Given that we bearly have any idea how the relationships should go (e.g. does culture influence personality or vice-versa), the amount of signal that we detected in the noise may be seen as a fairly good result

It's not that bad afterall

Sure there are loads of problems related to self-reports, especially when we try to compare them across cultures (e.g. differential self-enhancement)

– But there is no evidence for the particular DIF resulting from differential standards, at least for Conscientiousness and according to the vignettes-techique

– Like a detective story: the folk that everyone has been suspecting has an alibi afterall

• But who is the murder then?

Culture-level mean Conscientiousness scores do not perfectly predict things that researchers guess might be related to it:

– But why should they predict the criteria in the first place?

– The criteria and personality are both products of about a billion of things – expecting clear relationships with good effect sizes might be a naïve idea to start with

As we all know, most of the societal indicators are predicted by IQ differences between populations

– IQ predicts nearly everything with effect sizes well bove 0.70

– OK, a few historical-political things also matter

If this is true, there is not much variance in the societal indicators to be explained by personality

– If we assume personality to be causal thing

An ironic (or maybe not?) remark