assessment of unilateral spatial neglect_scoring star

Assessment of Unilateral Spatial Neglect: Scoring Star CancellationPerformance From Video Recordings—Method, Reliability, Benefits,

and Normative Data

Tom Manly, Anja Dove, Suzy Blows,Melanie George, and MaryAnn P. Noonan

Medical Research Council Cognition and Brain Sciences Unit

Thomas W. TeasdaleUniversity of Copenhagen

Christopher M. Dodds and Jessica FishMedical Research Council Cognition and Brain Sciences Unit

Elizabeth WarburtonUniversity of Cambridge

Unilateral spatial neglect, a debilitating condition affecting awareness of one side of space, is commonlyassessed using cancellation tasks in which patients cross out targets distributed over a sheet. Standardscores emphasize the left–right distribution of omissions. Here, the additional value of extractingtemporal as well as spatial aspects of performance from video recordings was examined. Videos from 18patients with left neglect and 19 healthy age-matched control participants were obtained. Interraterreliability of the video analysis was high. In addition to overall differences in target detection and bias,patients significantly differed from the control group in terms of the location of first cancellation, overallslowness, greater variability in speed, systematic slowing with time on task and as a function of targetlocation, less coherent search organization, and a sharply increased tendency to recancel targets.Considering a subset of patients whose scores, by standard criteria, were at or near the normal rangeindicated that these additional variables indeed increased the sensitivity of the task as well as allowed thesimultaneous assessment of spatial and nonspatial aspects of the disorder.

Keywords: unilateral neglect, assessment, cancellation, motor perseveration

Unilateral spatial neglect is a difficulty in detecting or acting oninformation on one side of space that cannot be fully explained bybasic sensory or motor loss. In its persistent form, which ismarkedly more common following right than left hemispherelesions and which predominantly affects left space (Stone, Halli-gan, & Greenwood, 1993), it impedes many daily activities and isassociated with poor outcome, including in motor recovery(Paolucci, Antonucci, Grasso, & Pizzamiglio, 2001). Taken to-gether with the range of rehabilitative interventions with the po-tential to reduce neglect (see Manly, 2002), detecting the disorder,even in possibly subtle forms, is important.

Cancellation tasks, in which patients are asked to cross out all ora subset of stimuli distributed over a page, are widely used in theassessment of neglect. The question addressed here was whether

video recording and retrospective scoring of performance increasethe sensitivity and yield of clinically relevant information fromthese measures. Cancellation tests are conventionally scored ret-rospectively in terms of number of targets detected and the relativefrequency of omissions on the left and right sides of the sheet. Ahypothetical patient who spent, say, 90% of the time searching theright of the sheet but nevertheless eventually found all of thetargets could therefore be classed as “recovered” by standardcriteria. For this reason, much can be gained from observationduring performance, including whether patients start on the right orleft, whether they appear “drawn” back to the right after a briefforay to the left, and whether they remark already cancelled targets(Bottini & Toraldo, 2003; Gainotti & Tiacci, 1970; Manly, Woldt,Watson, & Warburton, 2002; Na, Adair, Choi, Seo, & Kang, 2000;Rusconi, Maravita, Bottini, & Vallar, 2002). Keeping track of andrecording all of these features online is extremely difficult. Retro-spective scoring from video offers the prospect of reliable mea-surement and development of a normative base against which tojudge an individual’s performance. In particular, given increasingevidence linking neglect with a range of nonspatial impairments(Husain et al., 2001; Robertson, Manly, et al., 1997; Samuelsson,Hjelmquist, Jensen, Ekholm, & Blomstrand, 1998), knowingwhere and when actions occurred may be clinically informative.

We are not the first to explore this issue. Mark, Woods, Ball,Roth, and Mennemeier (2004) video-recorded 18 patients’ perfor-mance of a variant of the Star Cancellation test (see Measure).Their video analysis provided a number of measures of searchorganization not available via conventional scoring, including themean distance of each cancellation from a subsequent cancellationand, by analyzing the path of successive cancellations, the number

Anja Dove, Suzy Blows, Melanie George, MaryAnn P. Noonan, Chris-topher M. Dodds, and Jessica Fish, Medical Research Council Cognitionand Brain Sciences Unit, Cambridge, England; Thomas W. Teasdale,Department of Psychology, University of Copenhagen, Denmark; Eliza-beth Warburton, Department of Clinical Neurosciences, University ofCambridge, England.

We are grateful to the United Kingdom Medical Research Council for theirexclusive financial support for this work (MRC u.1055.01.003.00001.01) andparticularly to Roger Lucke and Gary Chandler for their design of thebespoke apparatus and to the participants for their assistance with thisstudy.

Correspondence concerning this article should be addressed to TomManly, Medical Research Council Cognition and Brain Sciences Unit,15 Chaucer Road, Cambridge CB2 7EF, United Kingdom. E-mail:[email protected]

Neuropsychology © 2009 American Psychological Association2009, Vol. 23, No. 4, 519–528 0894-4105/09/$12.00 DOI: 10.1037/a0015413

519

of intersections (a higher number being more indicative of a lessorganized search pattern). Although the patients differed from acontrol group across all parameters, this analysis revealed inter-esting dissociations. Poor search organization and a tendency torecancel were not, for example, significantly correlated with ne-glect severity.

Here we sought to extend the benefits demonstrated by Market al. (2004) in a number of ways, most notably by taking intoaccount timing as well as spatial dimensions of performance.Second, Mark et al.’s variant of the Star Cancellation testreplaced the conventional large star distractors with triangles.Although there were good reasons for this in terms of reducingfalse-positive cancellations in that study, it clearly makes directcomparison for clinicians using the standard version of the taskmore difficult. In the current study, we used parallel versions ofthe standard task. Third, in a pilot study we found substantialinterrater discrepancies even when using frame-by-frame videoanalysis. We therefore sought to develop a procedure that wouldminimize the need for subjective judgment and errors. In thecurrent study, two raters independently scored the video record-ings, and we assessed the reliability of the method. Fourth, weaimed to develop a method that could be applied to any can-cellation task by examining detection rates within regions of thesheet rather than specific target locations. Finally, the mainpurpose was to investigate whether the time required to performthe analysis paid dividends in terms of additional informationand enhanced task sensitivity.

To these ends, video recordings of 18 patients with leftneglect performing the Star Cancellation test were subjected toanalysis. In addition to examining the interrater reliability of themethod, the results from the patients were compared with thosefrom 19 age-matched, neurologically healthy control partici-pants. To examine the potential sensitivity of the analysis tomild forms of neglect, we then focused on a subgroup of thepatients who were close to or indistinguishable from the controlgroup in terms of their conventional Star Cancellation scores.

Method

Patient Participants

Following approval from the Local Research Ethics Commit-tee, 18 patients (7 men and 11 women, mean age � 68 years,SD � 13, range � 37–91 years) with left spatial neglect (asdiagnosed by the treating clinician) gave written informedconsent for their participation. Ten of the patients were in anearly stage following their stroke (�3 months), and the meantime post-cerebrovascular accident was 3.17 months (SD � 3.55).The patients were largely seen in a specialist stroke unit,although some had been discharged home or to other caresettings. The patients were referred to the research team on thebasis of having suspected or confirmed left unilateral neglect asa result of clinical observation, line bisection, or cancellationtask performance. Characteristics of the patient group are givenin Table 1.

Control Participants

Nineteen neurologically healthy volunteers (11 men and 8women), broadly age-matched with the patient group, were

recruited from the Medical Research Council Cognition andBrain Sciences Unit volunteer panel. The mean age was 67.79years (SD � 5.52, range � 56 –79 years), not significantlydifferent from the patient group, F(1, 36) � 0.005, p � .94. Allgave written informed consent to participate.

Measure

The Star Cancellation subtest of the Behavioural InattentionTest (Wilson, Cockburn, & Halligan, 1988) shows 56 smallblack stars distributed across a sheet of white paper, amiddistractors (larger black stars, letters, and words). The partici-pants completed either the original test or 1 of 11 parallelversions. In the parallel versions, the targets and distractorswere redistributed evenly within eight equally spaced regions ofthe sheet (see Figure 1).

Apparatus

To facilitate good quality video recordings, a bespoke can-cellation sheet holder/video stand was constructed. This con-sisted of a lightweight aluminum tray (420 mm � 30 mm) witha central, rubber-coated depression (29.6 mm � 21 mm), whichwas of a size and depth to hold a standard piece of A4 paper inthe center of the stand, flush with the surrounding surface. Thisallowed the task to be conducted against a visually uniformbackground, removed the risk of accidental or strategic move-ments of the sheet, and maintained the whole sheet within theviewing angle of the camera. The camera was mounted to theleft of the sheet to afford a good view of the acting hand forright-handed participants (a right mounting was available wereleft- handed participants to be seen). A relatively inexpensivecommercially available digital video camera was used, record-ing at 30 frames/s.

Procedure

Assessment. In the majority of cases we used the tray/videocamera stand placed centrally in front of the participant. Whenthe stand was not available, two recordings were made using atripod and six with a handheld camera. As per the standardclinical instructions, participants were asked to cross out all ofthe small star targets and to report when the task was complete.No time limit was imposed, and there was no other cue to eithercontinue or stop searching. Attempts to move the sheet to adifferent position were discouraged or corrected, as was anytendency to lean considerably to one side or other to gain adifferent view.

Video scoring. The video recordings were imported into com-mercial video-editing software that allowed frame-by-frame ad-vance and rewind. In addition to the video, the scorer had anunmarked paper copy of the relevant Star Cancellation sheet ontowhich the scoring grid (see Figure 1) had been imposed. For eachcancellation, the scorer recorded the time of occurrence from thevideo editor (to 0.04 seconds, the resolution of a frame) and foundand marked that particular star on the paper copy. From the copysheet, the scorer recorded in which of the eight boxes the cancel-lation occurred and whether it was the first cancellation or a

520 MANLY ET AL.

recancellation of that target.1 The time at which each cancellationoccurred was defined by the first visible contact of the pen at theoutset of that cancellation.

Derived scores. From the list showing the time and location ofcancellations, the following variables were extracted:

1. The location of the first and subsequent cancellations (ac-cording to the eight-box reference grid—see Figure 1).

2. The total number of targets cancelled.3. Whether a cancellation was a first or recancellation of a

target. (Here, to count as a recancellation, the participantmust have moved from that target and made at least onecancellation of another target before returning. This was toavoid misclassifying multiple lines through a target thatwere simply part of an emphatic cancellation style).

4. Time taken (from the first to last cancellation). Here andbelow, unless otherwise specified, cancellation refers tooriginal and recancellations.

5. Time per target (total time/total first cancellations, i.e.,excluding recancellations).

6. Time per action (total time/total cancellations, i.e., in-cluding recancellations).

1 The reference sheet was included to facilitate scoring recancellations.In a piloting exercise, two raters independently scored 15 video recordingswithout using a copy sheet, and agreement regarding total targets markedwas only 60%. It emerged that all discrepancies related to scoring ofrecancellations, it being very difficult to distinguish initial cancellationsfrom recancellations without scrolling back through the recording.

Table 1Age, Time Since Injury, Lesion Information, and Other Characteristics of the Patient Group

Patient

Age atassessment

(years)

Time sinceinjury

(months) Available lesion information Motor functionHemianopia/visual

extinction

1 51 5 Large right middle cerebral artery infarct involvingfrontal, temporal, and parietal cortex andcaudate/insula; evidence of small vessel disease

Left hemiparesis No/Yes

2 80 8 Right basal ganglia infarct, paraventricular whitematter ischaemia

Left hemiparesis Yes/NA

3 71 13 Subinsula hemorrhage (external capsule) andright paraventricular white matter lesion

Left hemiparesis Yes/NA

4 76 3 Infarct involving right anterior temporal andinferior frontal cortex, insula, head of caudate,globus pallidus, and internal capsule


5 66 1 Large right middle cerebral artery infarctinvolving frontal and anterior temporal cortex,insula, basal ganglia, and internal capsule


6 74 1 Large right middle cerebral artery infarctinvolving frontal and temporal cortex


7 70 2 Hematoma in the right basal ganglia andparaventricular regio

Left hemiparesis No/No

8 70 0 Infarct involving right posterior parietal lobe;evidence of atrophy of right temporal lobe

No difficulties Yes/NA

9 57 3 No scan available Left hemiparesis No/Yes10 49 9 Right hemorrhagic infarct involving basal

ganglia and paraventricular regionsLeft hemiparesis Yes/NA

11 84 4 Large right infarct extending from inferiorparietal to posterior temporal cortex

No left hemiparesis,lack of sensationon the left

Yes/NA

12 63 3 Lacunar-type right frontal infarct; lesion deepseated in white matter of the frontal lobe

No difficulties Unknown

13 76 0 Right middle cerebral artery infarct involvingventral frontal cortex

Left hemiparesis No/No

14 74 2 Subarachnoid hemorrhage affecting multiplebrain areas, including the right basal ganglia


15 67 1 Large right hemorrhage involving right posteriortemporal and parietal (including inferiorparietal) cortex

No difficulties Yes/NA

16 91 0 No scan available No difficulties Yes/NA17 60 1 Trauma leading to ischaemia, chronic subdural

collections, and damage to the right head ofthe caudate, basal ganglia, perisylvian, insula,right frontal cortex; additional small leftputamen infarct


18 37 1 Probable thrombotic stroke leading toattenuation in right basal ganglia (adjacent tofrontal horn) and internal capsule with somemass effect, as evidenced by compression ofthe frontal horn of the lateral ventricle

Some left-sidedweakness in theleg

No/Yes

521VIDEO SCORING OF CANCELLATION TESTS

7. Speed variability (for each participant, the standard de-viation of intercancellation intervals divided by themean intercancellation interval—the coefficient of vari-ation; Howell, 1992). This allows an estimate of vari-ability that is less influenced by an individual’s overallspeed.

8. Time required to complete each epoch of 10 successivecancellations. This was generated to examine any time-on-task slowing (e.g., whether the 11th–20th cancella-tions took significantly longer than the 1st–10th).

9. Estimated time allocated to each half of the sheet.Where two successive cancellations occurred on thesame side of the sheet, the intercancellation interval wasattributed to that side of the sheet. These intervals werethen summed and used to generate a time bias estimate(time attributed to the right side/time attributed to theright and left sides). A participant who cancelled targetsonly on the right would therefore get a bias score of 1.Where two successive cancellations occurred on oppo-site sides of the sheet, the interval was not attributed toeither side.

10. Search organization: An estimate of search efficiencywas derived by considering in which of the eight virtualboxes each cancellation occurred with reference to thepreceding cancellation. In general, any type of relativelyorganized search would mean that a given cancellationwas more likely to be in the same box or box adjacentto the previous cancellation rather than in a box that wastwo or more sections away. Accordingly, each cancel-lation was categorized with respect to the precedingcancellation as being near (in the same or an adjacentbox) or far (being in neither the same nor adjacent box).It is worth noting that this categorization could favorsome patients with relatively severe neglect; if cancel-lations occur only at the very right side of the sheet, theywill all be “near” to each other.

Interrater reliability. Ceiling effects and absence of recancel-lations in the performance of healthy participants were consideredlikely to return overly high estimates of interrater reliability. Thisanalysis was therefore conducted only on patient videos. To in-crease the available data, we included recordings of subsequentStar Cancellation performances from two patients. Two ratersindependently scored the 20 patient videos using the methodoutlined above. The analysis was conducted by a third researcher.

Results

Reliability of the Scoring Method

Total number of targets cancelled. The scores of the two ratersfor the total number of targets found for each of the 20 videos werecompared using intraclass correlation reliability analysis (two-waymixed model) implemented in SPSS v.12. This statistic takes intoaccount not just the relationship between the raters’ scores but alsothe absolute agreement in values. The correlation (r � .99, p �.001) indicates that variability between raters was negligible com-pared with variability between the patients’ performances. Eighty-five percent (17/20) of totals were in agreement to within onetarget, one was discrepant by two targets, and two were discrepantby four targets. The majority of discrepancies were due to differ-ences about whether a particular action was a cancellation, arecancellation, or a “mimed” cancellation in which no contact wasmade with the paper. Even when some videos were reviewed,disambiguating some of these actions proved impossible becauseof the camera angle or shadows on the sheet.

Location of cancelled targets. Levels of agreement regardingin which box each action had occurred, to within one target, wereat or in excess of 90%. Intraclass correlations ranged between .86and 1.00 (see Figure 1 for boxes; Box 1: r � 1.0; Box 2, r � .94;Box 3: r � .92; Box 4: r � .86; Box 5: r � 1.0; Box 6: r � .98;Box 7: r � .93 Box 8: r � 1.0).

Occurrence and location of recancellations. Recancellationswere noted in 17 of the 20 recordings (85%). The two raters agreedto within one recancellation in 90% of cases (intraclass correla-tion � .99). There were five discrepancies, with the raters dis-agreeing on whether or not contact with the paper had occurred.

Timing. Interrater agreement about the time by which 10,20, 30, and 40 cancellations (initial and recancellations) hadoccurred were considered. All patients made at least 20 cancel-lations (7 made fewer than 30, 3 fewer than 40 actions).Reliability was good: 10 actions, r(20) � .98, rating discrep-ancy M � 0.53 s, SD � 2.24; 20 actions, r(20) � .99, ratingdiscrepancy M � 1.70 s, SD � 4.00; 30 actions, r(14) � .97,rating discrepancy M � 2.77 s, SD � 6.34; 40 actions,r(10) � 1.0, rating discrepancy M � 1.80 s, SD � 2.97.

Comparing Performance in the Patient andControl Groups

Variables from the 18 initial patient video recordings werecompared with those from the 19 neurologically healthy partici-pants. In the few instances where discrepancies between raters hadoccurred, the mean of the two values was used.

Number of targets detected and spatial bias. The mean num-ber of targets cancelled by the control group was 55.47/56

Figure 1. One of the parallel versions of the Star Cancellation test usedin this study. Equal numbers of targets and distractors were distributedacross eight virtual boxes (grid and digits superimposed here but absentfrom the stimuli). Columns A–D, discussed in the Results section, are alsoshown.

522 MANLY ET AL.

(SD � 1.02, range � 52–56 targets). Thirteen participants can-celled all targets, 4 missed a single target, 1 missed 2 targets, and 1missed 4 targets. There was no statistically significant relationshipbetween accuracy and age (Pearson’s r � �.28, p � .25). Therewas no consistent bias in target detection; repeated measuresanalysis of variance (ANOVA) with the dependent variable oftargets detected and the factor of left/right side of the sheet for thehealthy volunteers, F(1, 18) � 0.32, p � .58. Of the 10 targetsmissed over the whole group, 6 were on the left of the sheet and 4were on the right.

A repeated measures ANOVA of target detection rates with thebetween-subjects factor of group (patient/control) and within-sub-jects factor of side (left/right) revealed statistically significant maineffects of group, F(1, 35) � 47.54, p � .001, and side, F(1,35) � 24.26, p � .001, and an interaction between group and side,F(1, 35) � 23.32, p � .001. As would be expected, patients foundfewer targets than controls overall and disproportionately fewer onthe left side of the sheet.

Relative rates of target detection within Columns A–D (seeFigure 1) were compared in a further repeated measures ANOVAwith the within-subject factor of column and the between-subjectsfactor of group (patient/control). As expected, there was a statis-tically significant main effect of group, F(1, 35) � 47.54, p �.001, column, F(3, 105) � 19.00, p � .001, and a significantinteraction, F(3, 105) � 17.51, p � .001. As shown in Figure 2,control group detection rates did not vary by column (beingessentially at ceiling), whereas patient group detection rates wereconsistent with a gradient from the right to the left of the sheet.Examining the patient results in isolation, post hoc comparisons ofthe differences between detection rates in each column revealed asignificant overall effect of column, F(3, 51) � 17.57, p � .001,and the Least Significant Difference test confirmed that detectionrates in each column differed from every other column for thepatients at at least the p � .05 level.

Sensitivity of the overall target detection measure. The authorsof the Star Cancellation test defined a clinical cutoff score as aperformance that was worse than the lowest scoring control par-ticipant. Applying that approach here would define a clinical cutoffas 5 or more omissions. This would by definition classify allcontrol participants correctly. Two patients (Patients 6 and 13)

would be above cutoff with scores of 55, and two patients (Pa-tients 15 and 17) may be considered marginal with scores of 49and 51, respectively.

Location of first cancellation. As shown in Figure 3, 78.95%of the healthy participants cancelled their first target in the upperleft quadrant of the sheet, with only 3 people beginning on theright. In contrast, for 66.66% of the patients, the first cancellationwas on the extreme right of the sheet, and for all but 1 person, itoccurred somewhere on the right of the sheet. Taking a simpleleft–right start point criterion would therefore correctly classifymore patients than the traditional overall target detection measure,although in this case it would misclassify 3 healthy participants.

Recancellations. Sixteen of the healthy participants did notrecancel any targets. Two participants made two recancellationseach, and a third made a single recancellation. Fifteen of the 18patients showed some recancellation behavior, with the meannumber of occurrences for this subgroup being 9.80 (SD � 12.65).The mean value for all 18 patients was 8.17 (SD � 12.08), farmore than in the healthy participants: one-way ANOVA, F(1,36) � 8.12, p � .007. The variance in the control group was toolow to generate meaningful percentile bands. Taking a cutoff ofproducing more recancellations than the control participant withthe greatest number would classify 10 of 18 patients as abnormalin this respect.

Speed of Performance

Overall speed. Control participants took a mean of 51.45 s(SD � 17.0) between their first and last cancellation, significantlyfaster than the mean time for the patients (176.59 s; SD � 182.48),F(1, 36) � 8.87, p � .005. The control participants’ mean time pertarget was 0.93 s (SD � 0.32), substantially faster than that ofpatients (5.29 s per target, SD � 6.49), F(1, 36) � 8.57, p � .006.

The relationship between time per target and age did not reachstatistical significance in the control group (Pearson’s r � .34, p �.15) or patient group (Pearson’s r � .45, p � .06), althougharguably there was a trend toward slower performances in theolder participants. For those wishing to compare individual patientdata with the control sample reported here, percentile levels for thetime-per-target score and other derived variables are presented inTable 2.

Variability in cancellation speed. To examine variability inperformance, we entered the coefficient of variation of the meantime per action into a univariate ANOVA with the factor of group.This revealed that despite the correction for differences in overallspeed, patients showed significantly more variability (percentagevariability � 98.2%, SD � 48.3) than the control group (45.9%,SD � 0.22), F(1, 35) � 18.23, p � .001. The percentiles forpercentage coefficient of variation in the control group are given inTable 2.

Systematic variation in cancellation speed: Slowing during thetask. The mean time-per-cancellation scores for the first two10-cancellation epochs were compared between the patient andcontrol groups in a repeated measures ANOVA. This confirmed astatistically significant main effect of group, F(1, 34) � 145.36,p � .001, with patients being slower than the control group, and asignificant main effect of epoch, F(1, 34) � 18.52, p � .001, withintercancellation intervals increasing as the task progressed. Therewas also a statistically significant interaction, F(1, 34) � 10.69,

Figure 2. Target detection rates for patients with left neglect and healthyparticipants in each of four evenly spaced regions of the sheet going fromleft (Column A) to right (Column D)—with standard error bars.


p � .002, that is, although both groups slowed in the second epoch,this was greater in the patient group (see Figure 4).

To facilitate future comparisons with the patient and controlgroups, we developed a percentage change measure—the increasein mean time per target from the first to the second 10 cancella-tions. This is presented in Table 2.

Systematic variation in cancellation speed: Slowing as a func-tion of spatial location. To examine whether patients becameslower as they moved from the right to the left of the page, weconducted a repeated measures ANOVA on the dependent variableof intercancellation interval, with the between-subjects factor ofgroup and the within-subjects factor of column (Columns C and Donly—some patients made very few cancellations in A and B).This showed a significant effect of group, F(1, 35) � 69.00, p �.001, the patients being slower overall than the controls, and asignificant effect of column, F(1, 35) � 13.91, p � .001, withintercancellation intervals being greater in Column C than D. The

interaction between group and column was also significant, F(1,35) � 13.95, p � .001—the patients were significantly slower indetecting targets that, although on the right of the sheet, wererelatively to the left. As the control group made few omissions,their performance over all four columns could be examined. Noslowing across the four columns was evident, F(3, 54) � 1.31, p �.28. Percentile bands from the healthy participants for the relativerate of slowing (increased intercancellation intervals) in ColumnsC relative to Column D are presented in Table 2. Caution shouldbe used, however, when there are only a few cancellations in thecolumn of interest.

Relative Amounts of Time Spent on the Right and LeftSide of the Sheet

The control group’s mean time bias score of 0.49 (SD � 0.04)did not significantly differ from the no-bias (0.5) point: single-sample t(18) � �1.19, p � .25. The patients’ mean rightward biasscore was 0.70 (SD � 0.23), which significantly differed from thecontrol group, F(1, 36) � 16.15, p � .001, and from the 0.5no-bias point: single-sample t(17) � 3.76, p � .002.

Figure 3. The frequency with which participants from the patient andcontrol groups made their first cancellation in each of eight equally spacedoctants of the cancellation sheet (e.g., 16 of the 19 control participantsmade their first cancellation in the upper left octant of the sheet).

Figure 4. Mean cancellation rate (seconds per cancellation) during the1st–10th and 11th–21st cancellations for the patient and control groups(standard error bars).

Table 2Percentile Levels, Means, and Standard Deviations From the Control Participants for Key Variables From StarCancellation Test Performance

PercentileCancellation

efficiencyVariability in

speedTime-on-task

slowing Number of recancellations Spatial slowingSearch

organization

10th 1.59 or higher 0.52 or higher 42.28 or higher Normal: 0–2 recancellations;Borderline: 3 recancellations;Abnormal: �3 recancellations

�20.53 or lower 15.09 or more25th 1.05 0.47 27.03 �13.53 3.6450th 0.81 0.43 9.90 �3.60 1.8275th 0.70 0.36 0.63 13.45 0.0090th 0.63 or lower 0.29 or lower �17.98 or lower 42.71 or higher 0.00M 0.93 0.46 14.49 1.93 3.10SD 0.32 0.22 27.06 22.17 5.00

Note. All variables are scaled such that the 10th percentile level reflects less accurate, slower, or more variable levels of performance. Cancellationefficiency � total time/total cancellations, not including recancellations; Variability in speed � mean adjusted coefficient of variation in detection speed;Time-on-task slowing � percentage increase in time per target from 1st–10th cancellation to 11th–20th cancellation, including recancellations; Spatialslowing � percentage increase in time per target from Column D to Column C; Search organization � percentage of cancellations that were “far” fromthe preceding cancellation.

524 MANLY ET AL.

Patterns in Cancellation Performance—Search Organization

A one-way ANOVA revealed that, despite the potential con-found of patients with severe neglect searching only a limited areaof the sheet, the patient group nevertheless made proportionatelyalmost twice as many cancellations “far” (see Method) from theprevious cancellation as the control group (patient group: 6.55%,SD � 4.23; control group: 3.10%, SD � 5.00), F(1, 36) � 4.90,p � .03. As with a previous report using a more elaborate method(Mark et al., 2004), this measure of organization was not correlatedwith overall target detection rates in the patient group (Pearson’sr � .16, p � .52). Percentiles for the percentage of “far” cancel-lations in the control group are presented in Table 2.

Correlations Between Derived Variables

The value of extracting additional variables from video analysiswould be reduced if their magnitude could be predicted with adegree of reliability from variables that are readily available fromstandardized scoring. To examine this, we performed correlationsbetween each of the key variables outlined here for the patientgroup and control group separately (see Tables 3 and 4).

For the patient group, the conventional score of the number oftargets detected did not significantly correlate with any of thederived variables with the exception of the bias toward spendingmore time on the right. The absence of other relationships suggeststhat, in principle, these factors dissociate and that a patient whowould be classified as normal in terms of target detection may bestatistically abnormal on other variables when compared with thecontrol group. One cluster of potential interest—and one thatwould survive appropriate p value correction for 21 compari-sons—is that patients who made a relatively high number ofrecancellations tended to be slower overall and to show greatervariability in their cancellation speed.

Statistically significant correlations, even at uncorrected p val-ues, were also infrequent in the control group (correlations be-

tween the total number of targets detected and the number ofrecancellations were not considered because of very low variance).There were two exceptions. Time per cancellation was signifi-cantly correlated with search organization (not surprisingly, thosewith a more erratic search pattern took longer to find targets).There was a strong and unexpected correlation between variabilityin cancellation speed and bias toward spending more time search-ing the right of the sheet. This correlation was, however, largelydue to a single individual who omitted two stars and who pausedfor a long period between two cancellations. With the exclusion ofthis participant from the analysis, the correlation was negligible(r � .09).

Relationship Between Scores and Time Since Injury

The time between the patients’ injury and the current assessmentranged from less than a month to 13 months. Examining thecorrelations between time since injury and key variables from theStar Cancellation measure revealed, as might be expected, thatthose in the more acute phase were more likely to miss moretargets (Pearson’s r � �.59, p � .03) and to show more rightwardbias (r � .54, p � .21). Correlations with search speed, speedvariability, time-on-task slowing, recancellation frequency, andsearch organization were all, however, statistically nonsignificant.

Relative Sensitivity of Key Scored Variables to MildForms of Neglect

As discussed, a particular interest in timing and other variablesnot traditionally scored from the Star Cancellation test was in theirpotential enhanced sensitivity to residual or mild spatial biases. Toexamine this, we formed a subgroup of four patients whose per-formance in terms of targets detected was above or at the marginsof the clinical cutoff level (Patients 6, 13, 15, and 17, whodetected 55, 55, 49 and 51 targets, respectively). Their scores onother variables were then compared with those of the control group

Table 3Intercorrelations Between Key Variables in the Patient Group

Variable

Number ofrecancellations

Time percancellation

Meanadjusted

coefficientof variationin detection

speed

Percentageslowing from1st–10th to11th–20th

cancellations

Percentageslowing fromcolumn D to

column C

Bias towardtime spent on

the rightSearch

organization

r p r p r p r p r p r p r p

Total stars detected �.05 .85 �.23 .30 .19 .46 �.03 .92 .25 .32 �.83 �.001 .16 .52Number of recancellations .69 .001 .660 .003 �.131 .62 .040 .87 .22 .37 �.070 .78Time per cancellation .49 .03 �.02 .95 .29 .24 .30 .23 .06 .83Mean adjusted coefficient of

variation in detectionspeed .50 .04 .12 .64 .03 .92 .41 .09

Percentage slowing from1st–10th to 11th–20thcancellations �.08 .75 .02 .93 .57 .02

Percentage slowing fromColumn D to Column C �.032 .20 .29 .25

Bias toward time spent onthe right �.29 .25


in a series of univariate ANOVAs. Although the overall targetdetection rates of these patients were close to those of the controlgroup, to maximally equate the two groups in terms of this stan-dard clinical score, we entered targets detected as a covariatewithin these analyses. As shown in Table 5, the patients divergedfrom the control group in terms of overall target detection time, thenumber of recancellations, and the degree of slowing associatedwith the detection of targets to the left. For the degree of slowingassociated with time on task and the general variability in inter-target intervals, despite the lack of a statistically significant effect,three of the patients fell below the 10th percentile level defined bythe control group on each measure. There were no striking differ-ences between these high-performing patients and the controlgroup in terms of their bias toward spending more time on one sideof the sheet or the other or the index or search organization.

Discussion

In this study, we examined the reliability and potential benefitsof retrospective scoring of Star Cancellation performance from

video recordings. The method used here produced a satisfactoryconcordance between two independent raters.

The conventional scoring of the Star Cancellation test yields anoverall target detection score and a lateralized bias score. Thequestion here was whether other aspects of patients’ perfor-mance, which are not preserved on the sheet, could be infor-mative. In particular, in addition to the definitive bias, persistentneglect has been associated with a range of impairments, in-cluding sustained attention (Robertson, Manly, et al., 1997;Samuelsson et al., 1998), maintenance of alertness (Robertson,Mattingley, Rorden, & Rorden, 1998; Sturm, Thimm, Kuest,Karbe, & Fink, 2006), and motor perseveration (Bottini &Toraldo, 2003; Gainotti & Tiacci, 1970; Manly et al., 2002; Naet al., 2000; Rusconi et al., 2002). Given evidence that some ofthese capacities may be important determinants of outcome(Corbetta, Kincade, Lewis, Snyder, & Sapir, 2005; Robertson,Ridgeway, Greenfield, & Parr, 1997), examining the timecourse and organization of cancellation responses has the po-tential to increase the sensitivity and utility of the measure.

Table 4Intercorrelations Between Key Variables in the Control Group

Variable

Mean adjustedcoefficient ofvariation in

detection speed

Percentageslowing from1st–10th to11th–20th

cancellations

Percentageslowing fromcolumn D to

column CBias toward timespent on the right

Searchorganization

r p r p r p r p r p

Time per cancellation .03 .91 .05 .85 �.1 .7 �.07 .77 .69 �.001Mean adjusted coefficient of

variation in detection speed �.24 .33 �.26 .29 �.75 �.001 .08 .75Percentage slowing from 1st–10th

to 11th–20th cancellations .38 .11 .35 .14 �.05 .83Percentage slowing from Column D

to Column C .08 .75 .06 .8Bias toward time spent on the right �.17 .49

Table 5The Sensitivity of Video-Derived Scores: Comparison of Healthy Participants With a Subset of Patients Who Were at or Near NormalLevels of Performance on Standard (Nonvideo) Scores

Variable

ControlHigh-scoring

patient

Difference univariateANOVA with totaltargets detected as

covariate Level of the controlpercentile of the 4

high scoring patientsM SD M SD F(1, 20) p

Overall time per target 0.93 0.32 2.48 1.03 12.16 .002 3 � 10th, 1 � 50thMean adjusted variance in the average

intercancellation interval 0.46 0.22 1.05 0.41 12.31 .04 3 � 10th, 1 � 75thSlowing of the second 10 cancellations relative

to the first 10 14.49 27.06 55.64 41.68 1.86 .187 3 � 10th, 1 � 50thNumber of recancellations 0.263 0.653 4.00 2.45 42.20 �.001Slowing in Column C relative to Column D 1.02 0.22 1.98 0.38 21.89 �.001 4 � 10thPercentage of cancellations “far” from the

preceding cancellation 3.1 5 5.54 3.84 0.061 .808 3 � 25th, 1 75th–90thRightward bias in relative time spent on the right

and the left side of the sheet 0.49 0.04 0.49 0.03 0.005 .947 2 � 25th, 2 � 90th

Note. Differences between the groups are illustrated with univariate ANOVA results and the percentile level, defined by the healthy participants, at whichthe mean patient performance would fall.

526 MANLY ET AL.

The results here showed that the patients were both slower andshowed considerably more variability in their cancellation pace(even taking into account their general slowness) than the controlgroup. They demonstrated significantly greater time-on-task slow-ing and slowing associated with cancelling more leftward targets.Although the performance of the control group indicates, for thefirst time, that one or two recancellations should not be consideredabnormal, half of the patient group showed a strikingly increasedtendency to return to and recancel targets, in one case up to 46times. Despite their search being relatively confined to the rightside of the sheet, the patients also demonstrated significantly moredisorganized search patterns than those of the control group, mark-ing a higher percentage of targets that were “far” from the pre-ceding cancellation.

Examining the correlations between the variables in the patientgroup demonstrated that the standardized measures of number oftargets detected did not correlate with any of the above measuresof speed, speed variability, time on task and leftward slowing,recancellation frequency, or search organization. In itself, thisindicates that video analysis provides information that could not besimply predicted from standardized scoring. The results, however,may appear in contradiction to previous work linking nonspatialimpairments with neglect: If, say, sustained attention or alertnessdeficits are associated with neglect and speed variability or time-on-task slowing is sensitive to sustained attention or alertness, whywere the relationships so weak here? It is important, in this respect,to distinguish between neglect severity and persistence. Althoughestimates vary, somewhere between 50% and 84% of patients withright-hemisphere damage show left neglect in the acute stage(Bowen, McKenna, & Tallis, 1999; Farne et al., 2004; Stone et al.,1993). Many, despite very pronounced initial spatial biases, showconsiderable spontaneous recovery within the first few days andweeks of convalescence (Bowen et al., 1999). The patients in thisstudy ranged from being 13 months to less than a month from theirinjury, and those in the more acute stages indeed showed thegreatest spatial bias. This factor decouples neglect severity per sefrom other nonspatial impairments and, although it would requirea prospective study to demonstrate it, the clear prediction fromprevious work is that those patients in the acute stage who are slowand variable in their search will be the ones most likely to continueto show bias in the longer term.

Aside from speed, a similar point can be made about themeasures of perseveration and poor search organization. As withprevious research (Mark et al., 2004; Na et al., 1999; Rusconi etal., 2002), these too were not significantly correlated with neglectseverity. Although there is to date little evidence linking thesemore “dysexecutive” characteristics to the persistence specificallyof neglect (although there are many studies linking these problemswith poor outcome in general (e.g., Hanks, Rapport, Millis, &Deshpande, 1999), there are at least two mechanisms that wouldsupport such a link. Executive systems have been argued to beengaged in error detection and flexible adaptation in light offeedback and changed context (Shallice & Burgess, 1996). It is notdifficult to see how such mechanisms could be a considerableadvantage in detecting and increasingly correcting for anomaliescaused by one’s neglect (patients often lack a direct subjectivesense of information being missing). Results such as those ofBartolomeo (2000) showing the reemergence of neglect in appar-

ently recovered patients when their executive resources are di-rected elsewhere would be consistent with their role in adaptivecompensation.

A second link between executive function and neglect may bethat which is mediated by changes in alertness. Studies in thehealthy population suggest that a drowsy or sleep-deprived stateis associated with an increased frequency of dysexecutive er-rors, including perseveration (Horne, 1993; Manly, Lewis, Rob-ertson, Watson, & Datta, 2001). It seems plausible that, werepatients’ levels of alertness to drop during task performance,they would become more vulnerable to perseverative recancel-lation and a less coherent search strategy. It is perhaps notablein this respect that recancellation frequency significantly cor-related (r values in the 0.6 – 0.7 range) with slowness andvariability in cancellation speed, and that search (dis)organiza-tion was significantly correlated with greater variability inspeed and time-on-task slowing. Given that they may formbarriers to spontaneous recovery, it is therefore possible thatthese nonspatial aspects of cancellation performance may formthe better longer term predictors of outcome than the spatialbias.

A particular focus in the study was on whether video-derivedscores discriminate controls from patients who, in terms of theirstandard clinical Star Cancellation scores, were close to or atnormal levels. On almost every measure that took into accounttemporal as well as spatial aspects of performance, the majorityof these patients diverged, sometimes sharply, from the normallevels. They were slower and more variable in response speedand showed systematic slowing in cancellation as they movedfrom the right to the left and with the passage of time. Inaddition, they produced significantly higher levels of recancel-lations.

Scoring cancellation tests from video recordings is timeconsuming in comparison with counting marked targets from asheet. An important question is whether this additional time isworth it. This crucially depends on the questions being asked. Ifone simply wanted to detect patients with relatively severeneglect at that time, the standard scoring is sufficient; indeed,the results here suggest that simply looking at where the firstcancellation is made tells you much of what you need to know.The results suggest, however, that video analysis both enhancesthe sensitivity of the measure to more subtle spatial biases andprovides potentially valuable information about associated im-pairments. Time is an issue for patients as well as therapists.Particularly in acute settings, there may be limits to the durationand variety of tests that patients can tolerate. By video record-ing cancellation performance, one can capture informationpotentially relevant to sustained attention, variability, organi-zation, and perseveration in parallel with the spatial bias infor-mation. The data from 19 healthy participants presented hereform some basis for making a normative comparison. In thelonger term, the results suggest that presenting cancellationtasks on a touch sensitive screen and developing software thatcan automatically calculate time, recancellation, and organiza-tion measures could be a significant advance in the clinicalassessment of neglect—saving time for both the patient andtherapist.


References

Bartolomeo, P. (2000). Inhibitory processes and spatial bias after righthemisphere damage. Neuropsychological Rehabilitation, 10, 511–526.

Bottini, G., & Toraldo, A. (2003). The influence of contralesional targetson the cancellation of ipsilesional targets in unilateral neglect. Brain andCognition, 53, 117–120.

Bowen, A., McKenna, K., & Tallis, R. C. (1999). Reasons for variabilityin the reported rate of occurrence of unilateral spatial neglect afterstroke. Stroke, 30, 1196–1202.

Corbetta, M., Kincade, M. J., Lewis, C., Snyder, A. Z., & Sapir, A. (2005).Neural basis and recovery of spatial attention deficits in spatial neglect.Nature Neuroscience, 8, 1603–1610.

Farne, A., Buxbaum, L. J., Ferraro, M., Frassinetti, F., Whyte, J., Ver-amonti, T., et al. (2004). Patterns of spontaneous recovery of neglect andassociated disorders in acute right brain-damaged patients. Journal ofNeurology, Neurosurgery & Psychiatry, 75, 1401–1410.

Gainotti, G., & Tiacci, C. (1970). Patterns of drawing disability in right andleft hemispheric patients. Neuropsychologia, 8, 379–384.

Hanks, R. A., Rapport, L. J., Millis, S. R., & Deshpande, S. A. (1999).Measures of executive functioning as predictors of functional ability andsocial integration in a rehabilitation sample. Archives of Physical Med-icine and Rehabilitation, 80, 1030–1037.

Horne, J. A. (1993). Human sleep, sleep deprivation and behaviour: Im-plications for the prefrontal cortex and psychiatric disorder. BritishJournal of Psychiatry, 162, 413–419.

Howell, D. C. (1992). Statistical methods for psychology. Belmont, CA:Duxbury Press.

Husain, M., Mannan, S., Hidgson, T., Wojciulik, E., Driver, J., & Kennard,C. (2001). Impaired spatial working memory across saccades contributesto abnormal search in parietal neglect. Brain, 124, 941–952.

Manly, T. (2002). Cognitive rehabilitation for unilateral neglect: Review.Neuropsychological Rehabilitation, 12, 289–310.

Manly, T., Lewis, G. H., Robertson, I. H., Watson, P. C., & Datta, A. K.(2001). Coffee in the cornflakes: Time-of-day, routine response controland subjective sleepiness. Neuropsychologia, 40, 1–6.

Manly, T., Woldt, K., Watson, P., & Warburton, E. (2002). Is motorperseveration in unilateral neglect “driven” by the presence of neglectedleft-sided stimuli? Neuropsychologia, 40, 1794–1803.

Mark, V. W., Woods, A. J., Ball, K. K., Roth, D. L., & Mennemeier, M.(2004). Disorganized search on cancellation is not a consequence ofneglect. Neurology, 63, 78–84.

Na, D. L., Adair, J. C., Choi, S. H., Seo, D. W., & Kang, Y. (2000).Ipsilesional versus contralesional neglect depends on attentional de-mands. Cortex, 36, 455–467.

Na, D. L., Adair, M. D., Kang, Y., Chung, C. S., Lee, K. H., & Heilman,K. M. (1999). Motor perseverative behaviour on a line cancellation task.Neurology, 52, 1569–1576.

Paolucci, S., Antonucci, G., Grasso, M., & Pizzamiglio, L. (2001). The roleof unilateral spatial neglect in rehabilitation of right brain-damagedischemic stroke patients: A matched comparison. Archives of PhysicalMedicine and Rehabilitation, 82, 743–749.

Robertson, I. H., Manly, T., Beschin, N., Haeske-Dewick, H., Homberg,V., Jehkonen, M., et al. (1997). Auditory sustained attention is a markerof unilateral spatial neglect. Neuropsychologia, 35, 1527–1532.

Robertson, I., Mattingley, J., Rorden, C., & Rorden, J. (1998, September10). Phasic alerting of neglect patients overcomes their spatial deficit invisual awareness. Nature, 395, 169–172.

Robertson, I. H., Ridgeway, V., Greenfield, E., & Parr, A. (1997). Motorrecovery after stroke depends on intact sustained attention: A 2-yearfollow-up study. Neuropsychology, 11, 290–295.

Rusconi, M. L., Maravita, A., Bottini, G., & Vallar, G. (2002). Is the intactside really intact? Perseverative responses in patients with unilateralneglect: A productive manifestation. Neuropsychologia, 40, 594–604.

Samuelsson, H., Hjelmquist, E., Jensen, C., Ekholm, S., & Blomstrand, C.(1998). Nonlateralized attentional deficits: An important componentbehind persisting visuospatial neglect? Journal of Clinical and Experi-mental Neuropsychology, 20, 73–88.

Shallice, T., & Burgess, P. (1996). The domain of supervisory processesand temporal organisation of behaviour. Philosophical Transactions ofthe Royal Society of London, 351, 1405–1412.

Stone, S. P., Halligan, P. W., & Greenwood, R. J. (1993). The incidence ofneglect phenomena and related disorders in patients with an acute right-or left-hemisphere stroke. Age and Ageing, 22, 46–52.

Sturm, W., Thimm, A., Kuest, J., Karbe, H., & Fink, G. R. (2006).Alertness-training in neglect: Behavioral and imaging results. Restor-ative Neurology and Neuroscience, 24, 371–384.

Wilson, B. A., Cockburn, J., & Halligan, P. (1988). Behavioural Inatten-tion Test. Bury St. Edmunds, England: Thames Valley Test Company.

Received April 10, 2008Revision received December 18, 2008

Accepted December 22, 2008 �

528 MANLY ET AL.

assessment of unilateral spatial neglect_scoring star

Documents