understanding and using the implicit association test

8/7/2019 Understanding and Using the Implicit Association Test

1/22

Understanding and Using the Implicit

Association Test: I. An Improved Scoring

Algorithm

A. Greenwald, B. Nosek, M. Banaji

Presented By: Steven Entezari

Info-I 563 The Psychology of Human Computer Interaction

October 26th, 2010


2/22

Review

What is an Implicit Association Test

Everything we just talked about


3/22

General Method

Yale IAT Website Intended to function as an Internet equivalent of an interactive exhibit at a

science museum

Allow web visitors to experience what many lab subjects have: Inability to control the manifestations of automatic associations that are elicited by

the IAT method

Approximately 1.2 Million tests completed between October 1998 andMay 2002

Recruitment Via media coverage, links, and word of mouth

Characteristics of Respondents Anonymity was promised, however, some respondents answered

demographic questions: 61% Female

60% Below 24, 36% Between 24 and 50

0.7% Native American, 6.4% Asian, 5% Black, 3.8% Hispanic, 76% White, 1% Bi-Racial (Black-White),3.3% Multiracial, 4% Other

47% Some College, 21% Bachelors, 14% Post baccalaureate

80% From US, 10% Canada, Australia, Britain (evenly distributed)


4/22

General Method (cont)

Materials

Java Applet and Common Gateway Interface (CGI) technology

After Test, respondent received results showing Strong, Medium, Slight, orLittle/No strength of association on monitor

Self Report Measures

Before each IAT, respondents received optional surveys as well as demographic

questionnaires All Optional

Nine IAT Measures The one mainly focused on in this paper is the Election 2000 implicit candidate

preference

Others are cited when significance is appearent

Sequence of Tasks

Preliminary Information ofW

hat they may experience Chose IAT available on site

Self-Reported Inventories

Demographic Responses

Instructions

IAT

Results


5/22


6/22

Sequence of Trial Blocks Example


7/22

Analysis andReporting Strategy

Study 1

Usefulness of Practice Trials and Error Trials

Study 2

Comparing Five Transformations of Latencies

Study 3

Possible Respondent-Exclusion Criteria

Study 4

Treatment of Trials with Error Responses

Study 5

Treatments of Trials with Extreme (Fast/Slow) Latencies

Study 6

Additional Performance Criteria and Additional Data Sets

Studies 1 5 focused on:

Magnitude of implicit-explicit correlation of IAT scores with self-report

Resistance to correlated variation of the IAT measure with latency differencesamong respondents

Study 6 examined combinations of best from 1 5


8/22

S1 Usefulness of Practice and Error Trials

Conventional

Discards first two trials of each test block because of typically

lengthened latencies

Treats as Practice the two combined-task blocks that precede the

test blocks for combined-task.

Retains latencies from trials on which errors occurred

Study 1

Preliminary Exclusions of Very Long Latencies

Latencies above 10,000ms were excluded (not recoded)


9/22

S1 Usefulness of Practice and Error Trials

Regarding first two trials of combined-task blocks:

Self-Report measures and IAT held higher correlations

IAT extremity with respondents average latencies slightly lower

Indicates the first two trials should be included

Regarding data from practice blocks Higher correlations with self-report

To make use of data from practice, new IAT measures computed asequal-weight averages.

The Discovery that practice blocks provided a good IAT measure wasconfirmed by the data

Regarding Error Latencies

Inclusion of error latencies should enhance IAT effects

Should occur because errors were

Slower than correct responses

More frequent when task required giving same response to non-associatedtarget-attribute pairs.


10/22

S2 Comparing Five Transformations of Latencies

Study 2

Incorporated Changes from Study 1

Using First Two Trials of Combined Task Blocks

Using Practice Blocks

Using Trials on which Errors Occurred

Transformations of Latencies

Results

D measure performed best


11/22

S3 PossibleRespondent-Exclusion Criteria

Results

Average percentage of fast responses was only dimension for

which a relatively small exclusion of respondents achieved a

clearly useful result

Reciprocal measuredemonstrated dramatic

improvement as more fast

responses were excluded

Indicates that its

performance was mostimpaired by presence of

fast responses

All are with latency


12/22

S4 Treatment of Trials with ErrorResponses

Study 4

Considers procedures more elaborate than simply retaining ordiscarding error latencies e.g. replacing error latencies with values that functioned as error

penalties.

Incorporated findings from study 3 D measure was superior to other forms of transformations

Five types of Error Treatments No treatment (treated the same as correct responses)

Deletion of error trials from dataset

Replacement of errors with the block mean of correct responses +penalty 200, 400, 600, 800, 1000 ms

Replacement of errors with the block mean of correct responsesmultiplied by constant 1.0, 1.5, 2.0, 2.5, 3.0

Replacement of errors with the block mean of correct responses plusvalue computed as the block mean multiplies by .2, .4, .6, .8, or 1


13/22

S4 Treatment of Trials with ErrorResponses

Results

Three conclusions

1. Discarding error trialswas an inferior strategy

Confirms Study 1

2. Most successfulstrategy was usingunaltered errorlatencies

3. Most successfulformulas providedpenalties that averagevalue were close to

average approximate500ms penalty fromproceduralrequirement toprovide correctresponse after makingan error


14/22

S5 Treatments of Trials with Extreme Latencies (Slow/Fast)

Study 5

Three substudies

1. Deletion and recoding to boundary for lower tail of distribution

Boundaries: 300, 350, 400, 450, 500, 550 ms

2. Deletion and recoding to boundary for upper tail of distribution Boundaries: 6000, 4000, 3000, 2500, 2000 ms

3. Combinations of 1 & 2


15/22

S5 Treatments of Trials with Extreme Latencies (Slow/Fast)

Results

Lower Tail Treatments

Small increases in implicit-explicit correlation up to 450ms

Virtually unaltered

Upper Tail Treatments Modestly improved implicit-explicit correlations at all upper bound

values

All Increased contamination by average response latencies

Combined Lower and Upper Tail Treatments

Highest value of implicit-explicit correlation (r = 0.789)

Deletion below 400ms and recoding above 2500ms to 2500ms

Reminder, Upper Tail Treatments all increased contamination

Ultimately inconclusive and depends on results from Study 6

Due to small increases accompanied by undesired increases in thecorrelation of IAT scores with average latency


16/22

S6 Additional Performance Criteria and Additional Data Sets

Goal

Evaluate all scoring startegies that appeared promising in S1-S5

Performance Criteria

Implicit-Explicit Correlation

Resistance to Contamination related to speed of responding

Internal Consistency

Resistance to Order Effect Associations appear stronger when they are tested in Blocks 3/4 rather

than in Blocks 6/7

Resistance to reduction in IAT scores (Prior Experience)

Sensitivity to modal response tendencies

Like the AGE IAT typically showing considerably stronger associations ofyoung than old with pleasant

Magnitude of the standardized coefficient for path between latentimplicit and explicit variables in CFA CFA Confirmatory Factor Analysis


17/22


Implicit-Explicit Correlation

The 6 D measures outperformed all of the conventional measures

in every analysis

Combined deletion of values below 400ms with 600ms error

penalty slightly outperformed other error penalty formulas The 2 measures that used unaltered error latencies outperformed

the other measures that used error penalties

Resistance to Contamination Related to Speed of Responding

The 6 D measures were uniformly superior to all of the

conventional measures

Two of the measures that incorporated error penalties provided

superior performance


18/22


Internal Consistency

The two measures that did not use error penalties produced higher

internal consistency

Indicated that measures with relatively poor performance on the major

(first two) criteria had superior internal consistency Surprising

Did not indicate an increase in validity

Because of this, it may be appropriate to treate internal consistency as an uncertain

guide to construct validity

Order Effect

On average, order effects were similar in magnitude for conventionalmeasures and tested measures

However, for Gender-Science and Age IATs, the measures including

computed error penalties showed loarger order effects than the built in

error penalties


19/22


Resistance to the Effect of Prior IAT Experience

It was known that prior experience with IAT was associated with

reduction in IAT scores for those who recorded one or more prior uses

Little or no further reduction in IAT scores occurred for two or more

previous uses Sensitivity to Modal Response Tendencies

Measures that involved only the built in error penalties were slightly

superior to the computed error penalties

Magnitude of Implicit-Explicit path in CFA

Identified a Latent Explicit Factor and Latent Implicit Factor

Very little difference amongst tested measures


20/22

Improved Algorithm

Three Substantial Changes from Conventional Procedure

Use of Practice-Block Data

Blocks 4 & 7

Use of Error Penalties

Using Blocks 3/6 and 4/7

Use of Individual-Respondent Standard Deviations

Using Block 3/6 and 4/7


21/22

Improved Algorithm

Practice Block Data

Use of Error Penalties

Use of Individual-Respondent

Standard Deviations


22/22

Questions?

understanding and using the implicit association test

Documents