C

Cs

MS

a

ARRAA

KCR

Atbtbwio

tcpIpctsbiwoowdd

p

0d

Drug and Alcohol Dependence 124 (2012) 189– 190

Contents lists available at SciVerse ScienceDirect

Drug and Alcohol Dependence

jo u rn al hom epage: www.elsev ier .com/ locate /drugalcdep

ommentary

ommentary on Ataya et al. (2012), ‘Internal reliability of measures ofubstance-related cognitive bias’

att Field ∗, Paul Christiansenchool of Psychology, University of Liverpool, United Kingdom

r t i c l e i n f o

rticle history:eceived 4 October 2011eceived in revised form 19 January 2012

a b s t r a c t

A recent paper published in Drug and Alcohol Dependence (Ataya et al., 2012) is the first systematicattempt to establish the internal reliability of measures of substance-related attentional bias, and it

ccepted 7 February 2012vailable online 2 March 2012

eywords:ognitive biaseliability

makes an important and long overdue contribution to this topic. The authors reported that the reliabilityof attentional bias indices derived from the visual probe task was very poor, and while the attentionalbias index from the Stroop was a little better, it fell some way short of being acceptable, as ˛ > 0.70 inonly two of six studies in which the task was used. In this commentary, we speculate on some of thefactors that might have contributed to the poor reliability of the tasks, and how task reliability might beimproved in future studies.

In a recent paper published in Drug and Alcohol Dependence,taya et al. (2012) report the first systematic attempt to establish

he internal reliability of measures of substance-related attentionalias. The paper makes an important and long overdue contributiono this topic. The authors reported that the reliability of attentionalias indices derived from the visual probe task was very poor, andhile the attentional bias index from the Stroop was a little better,

t fell some way short of being acceptable, as > 0.70 in only twof six studies in which the task was used.

This is certainly not good news for researchers who use theseasks. However, we can speculate on some factors that mightontribute to the poor reliability of these measures, and suggestossible solutions. The first point concerns the stimuli that are used.

n general, studies use a wide range of substance-related words orictures. For example, an alcohol-related visual probe study mightonsist of pictures of beer, wine, and spirits. But there is no reasono believe that a given individual would respond to each of thosetimuli in the same manner. Consider an individual who drinks onlyeer, and prefers to drink a certain brand (e.g., Brand X) whenever

t is available. If that individual completes a visual probe task, weould expect him or her to be maximally responsive to pictures

f Brand X, less so to pictures of other beer brands (or picturesf generic beer in glasses), and not responsive at all to pictures of

ine or spirits. To take this example to an extreme, if the individualoes not drink spirits due to a previous bad experience with thoserinks, they may even perceive such cues as aversive and shift their

∗ Corresponding author at: School of Psychology, University of Liverpool, Liver-ool L69 7ZA, United Kingdom. Tel.: +44 (0) 151 7941137; fax: +44 (0) 151 7942945.

E-mail addresses: mfield@liv.ac.uk, mfield@liverpool.ac.uk (M. Field).

376-8716/$ – see front matter © 2012 Elsevier Ireland Ltd. All rights reserved.oi:10.1016/j.drugalcdep.2012.02.009

© 2012 Elsevier Ireland Ltd. All rights reserved.

attention away from them. Depending on the proportion of beerpictures in the picture set, the overall index of attentional bias inthis individual might be very small and also unreliable, given thatit is computed by taking the average reactivity to all of the stimulithat were presented. This explanation can be tested in future stud-ies in which individualised stimuli are presented (cf. Fadardi et al.,2006), or studies which compute separate attentional bias indices(and their reliability) for preferred and non-preferred drinks (whichwould differ between participants). We feel that this interpretationis supported by the results from the stratified analyses reported byAtaya et al., as these revealed that attentional bias as assessed withthe visual probe task was more reliable for smoking-related cuesthan for alcohol-related cues. Although different smokers smokedifferent brands, the majority of pictures used in our own setof smoking-related images (e.g., Field et al., 2004) depict close-ups of individuals smoking or handling cigarettes. Importantly, allcigarettes look much the same when taken out of their packaging, sowe speculate that smokers should react in a uniform way to picturesthat depict cigarettes, perhaps explaining the increased reliabilityof attentional bias for these types of cues. Again, this possibility canbe tested in future research.

Ataya et al. noted that the poor reliability of the measures mightbe partially attributable to the use of reaction time – a ‘noisy’ mea-sure – to infer attentional bias. A secondary re-analysis of two of ourown earlier studies supports this conclusion and also suggests thateye movement measures of attentional bias may have superior reli-ability. We computed Cronbach’s for two measures of attentional

bias obtained from a visual probe task in which cannabis-relatedand matched neutral pictures were presented for 2000 ms, to agroup of cannabis users and a control group of non-users (Fieldet al., 2006). Attentional bias was inferred from probe reaction

1 Alcoho

twtwtt(ierbdttstbci

rtwfatas(oscSmer

emimopbfn

90 M. Field, P. Christiansen / Drug and

ime and from eye movement ‘dwell time’ bias, i.e., the extent tohich participants maintained their gaze on cannabis-related pic-

ures for longer than on the matched neutral pictures. Across thehole sample, the eye movement index was more reliable than

he reaction time index ( = 0.71 vs. = 0.53, respectively). Fur-hermore, both measures were more reliable in cannabis users

= 0.70 and = 0.58 for the eye movement and reaction timendices, respectively) than in non-users ( = 0.64 and = 0.38 for theye movement and reaction time indices, respectively). In a moreecent study, we computed the same indices of attentional bias,ut this time for alcohol-related cues in a sample of heavy socialrinkers (Christiansen et al., in press). In this study, the reliability ofhe eye movement index was only slightly superior to the reactionime index ( = 0.59 and = 0.56, respectively). A more comprehen-ive comparison of the reliability of eye movement versus reactionime measures is required, but we speculate that the attentionalias for alcohol-related cues might be less reliable than the bias forannabis-related or smoking-related cues for the reasons outlinedn the preceding paragraph.

Regarding the Stroop task, Ataya et al. noted that acceptableeliability was only seen for the blocked version of the task, but nothe unblocked version. With the blocked Stroop, substance-relatedords are presented in one sub-block (and neutral words in a dif-

erent sub-block), whereas substance-related and neutral wordsre randomly intermixed in the unblocked version. One explana-ion for the superior reliability of the blocked version is that, as wergued above, participants may only respond to a minority of theubstance-related words that are presented, but carryover effectssee Waters et al., 2005) would cause colour-naming interferencen subsequent trials, even if participants were not actually respon-ive to the words that were presented on those trials. Therefore,arryover effects might increase the magnitude and reliability oftroop interference on the blocked version of the task, but theyight decrease the magnitude and reliability of Stroop interfer-

nce on the unblocked version, particularly if participants were notesponding in an equivalent fashion to all of the stimuli.

Finally, given the superior reliability of the blocked Stroop, Atayat al. recommend that future researchers favour this method ofeasuring attentional bias over the visual probe task. While this

s a sensible recommendation, we urge caution here because theodified Stroop is likely to be measuring variance in a number

f factors, such as inhibitory control and even attempts to sup-

ress involuntary word-reading, in addition to attentional capturey substance-related words (see Field and Cox, 2008). Indeed, theact that Stroop interference is likely to have multiple determi-ants probably explains why effects from the task are so robust

l Dependence 124 (2012) 189– 190

and why it has superior predictive validity, compared to the visualprobe task. However, if researchers are specifically interested inmeasuring biases in visuo-spatial attention, the visual probe taskand other measures should be preferred. But we agree that resultsfrom the visual probe task should be interpreted with caution untilthe reliability of the task can be improved.

In conclusion, the paper by Ataya and colleagues is importantreading for researchers who use these tasks in the study of addic-tion, or in other domains (e.g., anxiety). We hope that this is notthe beginning of the end for research on this topic, but instead weencourage researchers to investigate how the reliability of existingmeasures might be improved, for example by exploring the impactof using individualised stimuli, and measuring attention using eyemovement monitoring rather than inferring attention from reac-tion time measures.

Role of funding source

None declared.

Contributors

MF drafted the manuscript, PC re-analysed the existing datasetsand commented on the manuscript. Both authors contributed toand have approved the final manuscript.

Conflict of interest

All authors declare that they have no conflicts of interest.

References

Ataya, A.F., Adams, S., Mullings, E., Cooper, R.M., Attwood, A.S., Munafo, M.R., 2012.Internal reliability of measures of substance-related cognitive bias. Drug AlcoholDepend. 121, 148–151, doi:10.1016/j.drugalcdep.2011.08.023.

Christiansen, P., Cole, J.C., Field, M. Ego depletion increases ad-lib alcoholconsumption: investigating cognitive mediators and moderators. Exp. Clin. Psy-chopharmacol., doi:10.1037/a0026623, in press.

Fadardi, J.S., Cox, W.M., Klinger, E., 2006. Individualized versus general measures ofaddiction-related implicit cognitions. In: Wiers, R., Stacy, A.W. (Eds.), Handbookof Implicit Cognition and Addiction. Sage, Thousand Oaks, CA, pp. 121–133.

Field, M., Cox, W.M., 2008. Attentional bias in addictive behaviors: a review of itsdevelopment, causes, and consequences. Drug Alcohol Depend. 97, 1–20.

Field, M., Eastwood, B., Bradley, B.P., Mogg, K., 2006. Selective processing of cannabis

cues in regular cannabis users. Drug Alcohol Depend. 85, 75–82.

Field, M., Mogg, K., Bradley, B.P., 2004. Eye movements to smoking-related cues:effects of nicotine deprivation. Psychopharmacology 173, 116–123.

Waters, A.J., Sayette, M.A., Franken, I.H.A., 2005. Generalizability of carry-over effectsin the emotional Stroop task. Behav. Res. Ther. 43, 715–732.