affordances - web...

Affordances

the object, bias the affordance appropriate to the task at hand. The original FARSmodel suggested that the bias was applied by PFC to F5; subsequent neuroanato-mical data (as analysed by Rizzolatti & Luppino, 2003) suggest that PFC and ITmay modulate action selection at the level of parietal cortex rather than premotorcortex. Figure 1 gives a partial view of ‘‘FARS Modificato’’, the FARS modelupdated to show this modified pathway. AIP may represent several affordancesinitially, but only one of these is selected to influence F5. This affordance thenactivates the F5 neurons to command the appropriate grip once it receives a ‘‘gosignal’’ from another region, F6 (pre-SMA), of prefrontal cortex. F5 also acceptssignals from areas 46 (dorsolateral prefrontal cortex), and F2 (dorsal premotorcortex)—all in prefrontal cortex (PFC)—to respond to working memory, andinstruction stimuli, respectively, in choosing among the available affordances. Notethat this same pathway could be implicated in tool use, bringing in semanticknowledge as well as perceptual attributes to guide the dorsal system (Johnson-Frey,2003; Johnson-Frey, Funnell, Gerry, & Gazzaniga, 2005).

It is worth briefly relating this model to data on patients DF (Goodale, Milner,Jakobson, & Carey, 1991) and AT (Jeannerod, Decety, Michel, 1994) which showeda dissociation between the praxic use of size information (parietal) and the‘‘declaration’’ of that information either verbally or through pantomime (infer-otemporal). We may talk about ‘‘how/parameterisation of action’’ versus ‘‘what/

Figure 1. ‘‘FARS modificato’’. The original FARS diagram (Fagg & Arbib, 1998) is here modified toshow PFC acting on AIP rather than F5. The idea is that AIP does not ‘‘know’’ the identity of the object,but can only extract affordances (opportunities for grasping for the object consider as an unidentifiedsolid); prefrontal cortex uses the IT identification of the object, in concert with task analysis and workingmemory, to help AIP select the appropriate action from its ‘‘menu’’.

EVOLUTION OF THE LANGUAGE-READY BRAIN 1129





Tuesday, October 25, 2011

Sunday, November 27, 2011

EVOCATION OF HAND ACTION REPRESENTATIONS 17

Figure 5. Mean response time in Experiment 3 as a function of action type (functional = filled symbols, volumetric = open symbols), prime relatedness, and cue presentation position. Error bars are 95% within-subjects confidence intervals appropriate for comparing means in related and unrelated conditions. the noun, can been seen in Figure 6. This double dissociation was statistically tested by comparing model that assumed a interaction between grasp type and sentence structure (i.e., Experiment 1 vs. Experiment 3) to a model that assumed no interaction. A Bayesian analysis provided positive evidence in favor of the interaction model, pBIC = .904, indicating that by shifting the surface structure of the context sentences, the advantage in priming of V-grasps relative to F-grasps seen in Experiment 1 was reversed in Experiment 3.

Discussion Context-specific resonance effects depend on the order of proximal and distal goal states described in a sentence. When the proximal goal occurs before the distal goal (John carried out action X on object Y to accomplish goal Z), Experiments 1 and 2 demonstrate striking contextual modulation of the grasp representations evoked by the noun (object Y). Merely changing the surface form of the sentence in Experiment 3 so that the distal goal of the intended action precedes the proximal goal abolished this effect. The unexpected and informative nature of this latter outcome is worth emphasizing. The word cell-phone in a V-context such as To clear the shelf, John lifted the cell-phone, triggers an F-grasp more strongly than a V-grasp, despite the fact that the meaning of the sentence unambiguously implies lifting rather than using an object! From an intuitive standpoint, this outcome appears outlandish. It is surely more reasonable to expect, tout court, that the mental simulation of actions referred to in a sentence would conform to the actions we ourselves carry out in a similar context. This assumption is often either implicitly or explicitly made in the burgeoning neuroimaging literature on the functional relationship between motor cortical activity and language. Speer, Reynolds, Swallow and Zacks (2009), for example, infer on the basis of patterns of

Figure 6. Mean priming of functional and volumetric actions as a function of the surface structure of the context sentence (Experiment 1: noun in middle of sentence; Experiment 3: noun at end of sentence). Error bars are 95% confidence intervals appropriate for comparing the mean priming effect to zero.

DYNAMICS OF ACTION REPRESENTATIONS 8

Figure 4. Mean priming effect in Experiment 1 as a function of grasp type and cue presentation position. Error bars are 95% confidence intervals appropriate for comparing the mean priming effect to zero. cued action and the spoken object name. Inspection of that figure reveals that subjects generally responded faster when the cue was placed at a later position relative to the word prime and that V-grasps were made faster than F-grasps. Our primary interest, however, lies in the pattern of priming effects, defined as the difference between unrelated and related conditions, shown in Figure 4. It is apparent that no priming was found when the hand cue was presented 150 ms in advance of the prime's onset. Beginning at the prime onset, however, a clear priming effect was present for F-grasps. For V-grasps, in contrast, a reverse priming effect was observed when the hand cue coincided with the onset of the prime, whereas a positive priming effect was found when the cue occurred at the midpoint of the prime's duration. Finally, the priming effect for V-grasps was no longer present when the cue appeared at the offset of the prime. These inferences about the pattern of priming effects are supported by the confidence intervals shown in Figure 4. Additional inferential analyses were conducted using the Bayesian analysis procedure proposed by Wagenmakers (2007; see also Masson, 2011). This procedure entails estimating the Bayesian posterior probability that one model rather than another is valid, given the observed data. Models being tested may correspond to the standard forms of null and alternative hypotheses used in null-hypothesis significance testing. For example, the null model, which assumes that no real effect is present (null hypothesis), may be compared to a model that assumes a real effect of an unspecified size is present (alternative hypothesis). Wagenmakers showed that when it is assumed that errors of measurement are normally distributed, as is

assumed in the standard analysis of variance (ANOVA), the posterior odds may be estimated using sums of squares from the ANOVA. These odds may readily be converted to conditional probabilities, which we designate as pBIC, that quantify the support in favor of either the null (no effect is present) or the alternative hypothesis (an effect is present), given the obtained data. Further, the conditional probabilities for the two hypotheses are complementary in that they sum to 1.0. Raftery (1995) provided verbal labels to characterize the strength of evidence associated with ranges of values of these probabilities (.50-.75: weak; .75-.95: positive; .95-.99: strong; > .99: very strong) and we adopt that terminology here. For those who wish to have familiar benchmarks, priming effects reported here as being supported by the data were significant at least at the .05 level and more typically at the .01 level, when standard hypothesis testing methods were applied. The relative size of the priming effect for F- and V-grasps was compared separately for each of the three later cue positions, given that that factor was partially nested within the two different groups of subjects and that no priming effect was apparent at the -150 ms position. The Bayesian analysis indicated that priming was clearly larger for F-grasps than for V-grasps at each of the three later cue positions. The posterior probabilities favoring the alternative hypothesis over the null hypothesis were substantial for all three positions: onset, pBIC = .974; middle, pBIC = .888; and end, pBIC = .929. In addition, for V-grasps, a model assuming different priming effects at the onset and middle positions (where reverse and positive priming effects, respectively, were found) was strongly preferred to a model that assumed no difference in priming at those two cue positions, pBIC = .959. Discussion Priming effects can reliably be observed when the cue is presented as soon as word onset. An examination of lift-off time (one of the two components of the total response times that we report here) indicated that grasp actions were launched on average about 600 ms following onset of the response cue. The F-grasp elicited by the word cue must evolve within this time window to exert an effect on lift-off time. It follows that an F-grasp must be generated quite rapidly after word onset, given that priming effects can be seen for cues time-locked to the initial segment of a word. We will later discuss hidden evidence that further confirms the relatively fast evocation of the F-grasp; its influence can be detected on reach-and-grasp actions cued 150 ms prior to word onset. The results also establish that an F-grasp is sustained over the word once it has been evoked. Priming of an action is clearly apparent when the cue is

V

F

To clear the shelf John lifted the cellphone Cellphone

The distal goal is prepotent and the noun at the end of the sentence simply evokes the action representation (F-grasp) typically seen in isolation. There is no representation of the proximal motor action.#

A

a) To clear the shelf John lifted the cellphoneb) John lifted the cellphone to clear the shelf


Figure 2. Mean priming effect in Experiment 1 as a function of sentence context, action type (functional = filled symbols, volumetric = open symbols), and cue presentation position. Error bars are 95% confidence intervals appropriate for comparing the mean priming effect to zero. assumed no interaction, pBIC > .999. Follow-up analyses evaluated a model that included a priming effect against a null effect model for each combination of sentence context and action type. Clear evidence favoring the priming effect model was found only when sentence context and action type matched (pBIC > .999 for functional sentences/actions, and pBIC = .992 for volumetric sentences/actions). When the context and action were mismatched, the null effect model was moderately favored over the priming effect model (pBIC = .802 for volumetric actions and functional sentences, and pBIC = .788 for functional actions and volumetric sentences). Next, we examined the pattern of priming across cue presentation locations. Here, our primary interest was in (a) whether priming was sustained or faded as

subjects listened to the final clause of the sentence and (b) whether context-specific effects on priming were sustained across the course of the sentence. Figure 2 shows that priming for F- and V-grasps within their matched sentence contexts peaked at some point during the presentation of the object noun. For F-grasps tested in functional sentence contexts, a model in which priming effects were assumed to decrease linearly across cue presentation locations was favored over a model that assumed stable priming across time, pBIC = .866. In addition, context specificity was examined by considering priming effects for F- and V-grasps when actions were cued during the presentation of the object noun versus during the presentation of the final clause of the sentence. When subjects were cued to act while listening to the noun, a model that assumed different amounts of priming for F- and V-grasps was very strongly preferred over a model that assumed equal amounts of priming, pBIC = .992. When subjects were cued during the final clause, however, a model assuming no differences between grasp types was preferred, pBIC = .750. For V-grasps primed in the context of volumetric sentences, the null model was favored over both a linear (pBIC = .913) and a quadratic model (pBIC = .854) of changes in priming effects across cue locations. The relatively small amount of priming seen with V-grasps may have prevented the emergence of clear evidence for dissipation of this priming over the course of the sentence. Another way of assessing the time course of contextual influences on priming of V-grasps is to examine where during the sentence V-grasps showed more priming than F-grasps (context specificity). Figure 2 clearly indicates that the advantage in priming of V-grasps over F-grasps was confined to just two cue locations, the end of the object noun and the middle of the final clause. Considering just these two locations, a model assuming a difference in priming between the two grasp types was favored over a model that assumed no such difference, pBIC = .890. Thus, V-grasps were contextually favored over F-grasps by volumetric sentence contexts by the time the end of the object noun was reached, but by the end of the sentence, this advantage was lost (for the end-of-sentence location, a null effect model was favored over a model that assumed a difference between grasp types, pBIC = .788). An additional question with respect to the time course of priming effects was whether F- and V-grasp action representations might initially compete with one another during the early stages of processing the name of a manipulable object, even when presented in a volumetric context. This possibility was tested by examining priming for F- and V-grasps in volumetric sentences at the first two cue locations (onset and middle of the object noun). Figure 2 indicates that priming was small but very similar for both types of

F

V




V

John lifted the cellphone to clear the shelf Cellphone

F

a) To clear the shelf John lifted the cellphoneb) John lifted the cellphone to clear the shelf

B




F

V




V


F

ACB

At which point is there evidence that the F-grasp has been inhibited?

D

1) A 2) B 3) C 4) B and C

B




F

V




V


F

ACB

Is the following statement True or False? The sentence has altered the time course of the V-grasp relative to the same motor representation evoked by the word in isolation. a) True b) False

D

B




F

V




V


F

ACB

At which point is there evidence that the inhibition of the F-grasp has dissipated?

D

1) A 2) B 3) C 4) D









Tuesday, October 25, 2011

Sunday, November 27, 2011

Visual Stimulus

Hidden Layer

Motor goal

C

Visual Stimulus

Hidden Layer

Motor goal C

CONTROL OF AUTOMATIC PROCESSES 339

Table 1 Training Stimuli

Task demand Color input Word input Output

Color red - - "red" Color green - - "green" Word - - RED "red" Word - - GREEN "green"

Note. Dashes indicate there was no input.

in a flatter region of the activation function. In the current model, we implemented a simpler version of this general scheme. All intermediate units were assumed to have a negative bias, so that they were relatively insensitive at rest. Task demand units provided an amount of activation to intermediate units in the corresponding pathway that offset this negative bias, driving their net input to zero. Thus, task demand units had the effect of sensitizing units in the corresponding pathway, and units in the inappropriate pathway remained in a relatively insensitive state.

Finally, we note that the connections between each task demand unit and all of the intermediate units within a given path-

way are assumed to be uniform in strength, so that activation of a task demand unit does not, by itself, provide any information to a given pathway. Its effect is strictly modulatory.

S imula t ions

We implemented the mechanisms described in the previous section in a specific model of the Stroop task. In the following sections, we describe how the model was used to simulate human performance in this task. We start by describing some of the general methods used in the simulations. We then describe four simulations that provide an explicit account of the attributes of automaticity and how they relate to practice. These are followed by two simulations that address issues concerning the relationship between attention and automaticity.

Simulation Methods

All simulations involved two phases, a training phase and a test phase.

Training Phase

The network was trained to produce the correct response when information was presented in each of the two processing pathways. Training patterns were made up of a task specification and input to the corre-

RESPONSE "red . . . . green"

i Color Word

INK COLOR Naming Reedlng WORD

TASK DEMAND Figure 3. Diagram of the network showing the connection strengths after training on the word-reading and color-naming tasks. (Strengths are shown next to connections; biases on the intermediate units are shown inside the units. Attention strengths---from task demand units to intermediate units--were fixed, as were biases for the intermediate units. The values were chosen so that when the task demand unit was on, the base input for units in the corresponding pathway was 0.0, whereas the base input to units in the other pathway was in the range of -4.0 to -4.9, depending on the experiment.)

Two different views of the relationship between perception and action.

Visual Stimulus

Hidden Layer

Motor goal

C

Visual Stimulus

Hidden Layer

Motor goal C

Why is choosing between these alternatives important?

and what is a “motor goal”?

Distal intentions Deals with the “why” of an action (Rational Control)

Proximal intentions

The intention to act now

Motor intentions More specific levels of control, like the type of grip chosen, and the

choice of left/right hand

Distal intentions

Clear the table

Proximal intentions

Pick up the cell phone

Motor intentions

Power grasp with the palm facing down

What task conditions automatically trigger M-intentions?

Tucker and Ellis

….actions afforded by a visual object are intrinsic to its representation. According to this position, representing visual information involves representing information about possible actions and thereby potentiating them. One consequence of this is that intended actions are formed from, and informed by, already existing visuomotor representations.

Utilization behaviour is often taken as support for the claim that actions are triggered without intentions.

Indeed, in humans, when frontal cortical control is lost through damage, "utilization behavior" can arise in which visual stimuli automatically elicit motor responses such as reaching and grasping (Lhermitte, 1983). This suggests that motor acts are controlled, in part, by the active suppression and subsequent selection and tuning of already existing sensorimotor connections

Tucker and Ellis:

M-intentions are an obligatory part of the representation of visual objects.

Indeed, in humans, when frontal cortical control is lost through damage, "utilization behavior" can arise in which visual stimuli automatically elicit motor responses such as reaching and grasping (Lhermitte, 1983).

Visual Properties of an Object

Action Features evoked by an Object

Response Production ResponseInhibition

Tucker and Ellis

Utilization behaviour. Alien Hand Syndrome.

Keypress responses to objects:Look ma, no hands….or at least, no intention to produce a grasp action

834 TUCKER AND ELLIS

to affect the speed with which the response was selected and executed. It is important to point out that hand dominance may override the effect of horizontal object orientation in many instances of everyday prehension. Thus one may often reach for and grasp an object with the dominant hand even though its orientation is not maximally compatible with a grasp made by that hand. This, however, does not affect the conclusions that can be drawn from the present study. Even though in instances of everyday prehension, hand selection will rarely be exclusively determined by object orientation, nonetheless, given a particular hand used, the horizontal orientation makes it more or less compatible with that hand. In Experiment 1 the horizontal orientation of the object could be said to be more or less compatible with the cued hand (whether or not the cued hand would have been used to grasp the object in real life). Thus, under the hypothesis put forward about action potentiation, compatibility effects would be expected from the relation between the left-right orientation of the object and the hand used to make the response, the latter being cued by object inversion.

Figure 1. Examples of the stimuli used in the experiments. Experiments 1 and 2: right orientation, upright (frying pan); left orientation, inverted (teapot). Experiment 3: anticlockwise wrist rotation compatibility, inverted (knife); clockwise wrist rotation compatibility, upright ( aerosol can).

Exper imen t 1

Method

Participants. Thirty students took part in the experiment. All were enrolled at the University of Plymouth and received course credit for their participation. All participants had normal or corrected-to-normai vision and were naive as to the purpose of the experiment. All except 2 participants reported that they were right-handed.

Apparatus and materials. Black and white transparencies of 22 graspable household objects made up the stimulus set (see Appen- dix A for a fist of objects used). All the objects were capable of being grasped and manipulated by one hand and were photographed in two horizontal orientations (one compatible with a right-hand grasp, the other with a left-hand grasp) and two vertical orientations (upright and inverted). There were thus 22 × 2 × 2 = 88 slides that were back-projected onto a translucent screen (46 × 46 cm) from two Kodak carousel random access projectors, modified to allow millisecond shutter control. Examples of the stimuli are shown in Figure 1. The participant was seated with his or her head 45 cm in front of the screen and with the index finger of each hand resting on two response buttons 30 cm apart and 15 cm in front of the screen. The objects were photographed so as to appear as if they were resting on the table at the position of the screen, at approximately their actual size, at a distance of 50 cm. They subtended visual angles of between 11" and 18".

Design and procedure. The experiment consisted of two blocks of 176 trials in which each object appeared twice in each horizontal and vertical orientation. Participants were instructed to make push-button responses with the left or right hand depending on whether the object was upright or inverted. The actual mapping of response hand to object inversion was blocked and pseudoran- domized so that an equal number of participants received each mapping in the first block. For most objects, whether the object was upright or inverted needed no definition. In the case of objects such as a knife or saw, participants were told that upside down or upright was defined with regard to the object's normal use. Such objects were thus photographed with the blade at right angles to the resting surface, rather than lying flat, and were upside down when the blade or teeth were pointing up rather than down. Participants experienced no difficulty in understanding this definition of inver-

sion. The left-right horizontal orientation of the object was irrelevant to the response. Participants were instructed to respond as fast as possible whilst maintaining accuracy. Slide order was randomized for each participant, and the experiment was run, and response latencies recorded, on an Acorn Archimedes computer. Each participant received 20 practice trials before each block. A trial began with the appearance of an object on the screen and ended when a response had been made or 3 s had elapsed. The objects remained in view until a response was made. There was a 4-s delay between the end of one trial and the beginning of the next. Participants were not given feedback on response latencies, but errors were immediately followed by a short tone from the computer.

Results

Response times. Two participants were removed from the analysis because their error rates exceeded 10%. Error trials and reaction times more than 2 SDs from the condition means were excluded from the analysis. The means for each object in each of the eight conditions were computed for each participant. For the participants analysis, condition means were obtained by averaging across objects, and for the materials analysis they were obtained by averaging across participants. An analysis of variance (ANOVA) was conducted on the participant data with the independent variables o f mapping (right-hand-upright/left-hand-inverted or left-hand-upright/right-hand-inverted [RH-UP and LH-UP, respectively]), response (left hand or right hand), and object orientation (left or right). There was a significant main effect of response mapping. Responses in the R H - U P mapping (M = 616.68 ms) were faster than responses in the L H - U P mapping (M = 650.35 ms), F(1, 27) = 8.61, p < .01. The only other significant effects were the two-way interactions between response mapping and hand of response and between hand of response and left-fight object orientation. The interaction between mapping and hand of response is easily interpretable as an effect of object inversion. Right-hand responses in the R H - U P

Task: Upright versus inverted. Respond with a speeded keypress.

L RUpright Inverted

Tucker and Ellis (1998)

15 with self-paced rest breaks between blocks. The order of trials in a block was randomized

for each subject.

Figure 2.2. Left: Response box used in experiments and hand position used in

Experiments I and III. Right: Laboratory setup for all experiments. Procedure: Subjects were seated about 50 cm away from a computer monitor in a

quiet room. Subjects sat comfortably with their index fingers resting on a response box

which was placed directly in front of them (see Figure 2.2 left and right). Each trial began

with a fixation cross that remained on the screen for 250 ms. The cross was then replaced

by either an image of an object and hand appearing in unison (0-ms SOA) or separated a

delay (250-ms SOA; see Figure 2.3). The image remained on the screen until the subject

made a laterality judgment with a key-press response.

Although on any given trial the hand and object prime could match with respect to

the handle alignment and/or orientation congruency, these dimensions were irrelevant to

the laterality judgment. These judgments were made by pressing the right or left key on

the button box for right- or left-handed images of hands, respectively. On 25% of the

trials, subjects were instructed to verbally report what the object prime was on that trial.

These responses were scored by the experimenter using a Macintosh computer keyboard.

Upright Inverted

….representing visual information involves representing information about possible actions and thereby potentiating them. One consequence of this is that intended actions are formed from, and informed by, already existing visuomotor representations. Actual actions are produced by the selection and elaboration of such representations.

Possible action?

OBJECTS AND ACTION POTENTIATION 835

640-

i

620

°**~.,O 7 •

e~°* i °°°° t~ 4,

| Left

| Right

Object Orientation 1--.o--, Left responses Right responses ] I -

Left Right Object Orientation

Figure 2. Mean reaction times (RTs) and error rates for Experiment 1 as a function of left-right object orientation and response (left or right hand).

mapping (M = 607.7 ms) tended to be faster than left-hand responses (M = 625.6 ms), whereas in the LH-UP mapping, left-hand responses (M = 642.3 ms) tended to be faster than right-hand responses (M = 658.4 ms), F(1, 27) = 16.8, p < .001. Because object inversion can be derived from the combination of hand of response and mapping rule it can easily be seen that the above results reflect the fact that responses to upright objects were, on average, 17 ms faster than responses to inverted objects. This result is to be expected because to determine whether an object is upright or inverted it must be recognized, and this will be faster for a canonical orientation.

The two-way interaction between object orientation and hand of response is the most interesting result. This interaction is displayed in Figure 2. Right-hand responses were faster when the irrelevant orientation of the object was also to the right (M = 627.3 ms) rather than to the left (M = 638.8 ms). Similarly, left-hand responses were faster when the orientation of the object was also to the left (M = 628.2 ms) rather than to the right (M = 639.8 ms), F(1, 27) = 11.85, p < .005. Palrwise comparisons (Newman-Keuls) showed both of these differences to be significant. For right-hand responses, q(2, 27) = 3.42, p < .05, and for left-hand responses q(2, 27) = 3.45, p < .05, MSE = 634.0.

Errors. Analysis of percentage error rates revealed a pattern of results similar to that for response times (see Figure 2), although the effect of mapping and the mapping by response interaction were not significant. The interaction between response and horizontal object orientation was significant, F(1, 27) = 13.51, p < .005. In addition, there was a small but significant effect of object orientation, with objects oriented to the left (left-hand grasp compatibility) producing fewer errors (M = 5.05) than objects oriented to the fight (M = 5.70), F(1, 27) = 4.76, p < .05. The pattern of errors indicated the absence of any speed-accuracy trade-offs.

Materials analysis. A materials analysis on response times with objects as a random factor and condition means averaged over participants yielded the same pattern of results as the participants analysis. The RH-UP mapping

produced faster responses (M = 618.2 ms) than the LH-UP mapping (M = 649.3 ms), F(1, 21) = 64.63, p < .001. Right-hand responses in the RH-UP mapping (M = 609.9 ms) were faster than left-hand responses (M = 626.5 ms), whereas in the LH-UP mapping, left-hand responses (M = 640.8 ms) were faster than right-hand responses (M = 657.7 ms), F(1, 21) = 6.39, p < .05. Again, the two-way interaction between object orientation and hand of response was significant, with right-hand responses being executed faster when the object was oriented to the right (M = 627.5 ms) than when it was oriented to the left (M = 640.1 ms), whereas left-hand responses were faster when the object was oriented to the left (M = 629.3 ms) than when it was oriented to the right (M = 638.0 ms), F(1, 21) = 22.79,p < .001.

The stimuli used in this experiment constitute only one sample of the population of graspable objects whose horizontal orientation can affect the ease with which they are grasped by a particular hand. They were thus treated as a random factor. In order to provide a test of the ability of the interaction between response hand and object orientation to generalize to a new sample of participants and objects simultaneously, we computed Min F '~ (see Clark, 1973). The result obtained, Min F'(1, 46) = 7.79, p < .01, was highly significant, which suggests that this effect is unlikely to be restricted to the particular objects used in the experiment.

Discussion

The first experiment showed that the left-right orientation of common graspable objects had a significant effect on the speed with which a particular hand made a simple push- button response, even though the horizontal object orientation was irrelevant to response determination. The orientations of the objects were chosen so as to make them

~Min F' provides a conservative test of the ability of an effect to generalize simultaneously to a new sample of participants and objects. Exact F ratios cannot be obtained with participants and objects as random factors in a single analysis. The formula for Min F' is given in Appendix B.

Evidence that the effect is genuinely due to the triggering of a grasp action

UprightLeft finger Inverted

Right finger

OBmCTS AND ACTION POTENTIATION 837

620,

g:d 610 -

600

6

o 5 J. 3-

Left Rl'ght Object Orientation

I ---O--. l~ftresponses I . . . . . - o - - Risht rcsponse.s

i L~ft Right

Object Orientation

Figure 3. Mean reaction times (RTs) and error rates for Experiment 2 as a function of left-right object orientation and response (left or right finger).

Results

Response times. The data from 3 subjects were not analyzed because their error rates exceeded 10%. Error trials and reaction times more than 2 SDs from the condition means were excluded from the analysis. For the rest, mean response times for each object in each condition were computed. As in Experiment 1, condition means for the participants analysis were computed by averaging over objects, whilst for the materials analysis they were computed by averaging over participants. An ANOVA was performed on the participant data with mapping, object orientation, and response (left or right finger) as independent variables. There was a significant main effect of object orientation. Responses to objects oriented to the left were faster (M = 612.9 ms) than responses to objects oriented to the right (M = 621.3 ms), F(1, 26) = 4.73, p < .05. As in Experiment 1, there was a significant two-way interaction between mapping and response, best understood as a speed advantage for upright objects. In the RF-UP (right-finger- upright), mapping, right (middle) finger responses (M = 612.1 ms) were faster than left (index) finger responses (M = 621.6 ms), whereas in the LF-UP (left-finger-uprigh0 mapping, left-finger responses (M = 607.1 ms) were faster than right-finger responses (M = 627.6 ms), F(1, 26) = 13.61, p = .001. This two-way interaction simply reflects the fact that responses to upright objects were on average 15 ms faster than responses to inverted objects, as observed in Experiment 1. Of most interest, however, is the two-way interaction between response and horizontal object orientation. The pattern of means was quite different from that observed in Experiment 1 (see Figure 3). Right responses to objects oriented to the left (M = 617.1 ms) were actually slightly faster than the same responses to objects oriented to the right (/14 = 622.6 ms). For left responses, objects oriented to the left tended to be responded to faster (M = 608.7 ms) than objects oriented to the right (M = 620.0 ms). This two-way interaction was not significant, 2 F(1, 26) = 1.17, p = .29.

Error rates. The pattern of error rates was similar to that for response times (see Figure 3). An ANOVA on percentage

error rates found a single significant main effect of left-right object orientation. Participants made fewer errors to objects oriented to the left (M = 3.52) than to objects oriented to the right (M = 4.49), F(1, 26) = 8.82,p < .01.

Materials analysis. An ANOVA with objects as a random factor and condition means obtained by averaging over participants showed the same pattern of results as that observed in the participants analysis. The effect of object orientation was significant, with left-oriented objects (M = 620.9 ms) being responded to faster than right- oriented objects (M = 612.2 ms), F(1, 19) = 4.38, p = .05. The interaction between response and mapping was significant and identical in form to that observed in the participants analysis, F(1, 19) = 5.94, p < .05. Again, the interaction of most interest, that between response and left-right object orientation, was not significant, F(1, 19) = 0.53, p = .476.

As expected from the results of both the participants and materials analyses, computation of Min F ' gave an insignificant result: Min F'(1, 35) = 0.36, critical value at c~ (.05) = 4.13.

Experiments I and 2 compared. The main purpose of Experiment 2 was to help determine the extent to which the compatibility effect observed in Experiment 1 could be attributed to graspability rather than to the abstract coding of object orientation into left and right codes congruent or incongruent with the left-right response locations. The critical comparison is between the Response × Object Orientation interactions in the two experiments. This can be seen from a comparison of Figures 2 and 3. In order to test whether these interactions differed significantly, we computed a further ANOVA on the data from both experiments

2In an analysis using medians rather than means we did in fact find a significant result for this effect. Its significance was much smaller than in Experiment 1, and it was not present for the materials analysis. Use of untrimmed means and a straight 1,000-ms cutoff produced the same results as those reported in the text. (For a discussion of the problems of finding the right measure for reducing the effect of outliers in reaction time data, see Ratcliff, 1993.)

First sign Inconsistencies

For most objects, whether the object was upright or inverted needed no definition. In the case of objects such as a knife or saw, participants were told that upside down or upright was defined with regard to the object's normal use. 834 TUCKER AND ELLIS

to affect the speed with which the response was selected and executed. It is important to point out that hand dominance may override the effect of horizontal object orientation in many instances of everyday prehension. Thus one may often reach for and grasp an object with the dominant hand even though its orientation is not maximally compatible with a grasp made by that hand. This, however, does not affect the conclusions that can be drawn from the present study. Even though in instances of everyday prehension, hand selection will rarely be exclusively determined by object orientation, nonetheless, given a particular hand used, the horizontal orientation makes it more or less compatible with that hand. In Experiment 1 the horizontal orientation of the object could be said to be more or less compatible with the cued hand (whether or not the cued hand would have been used to grasp the object in real life). Thus, under the hypothesis put forward about action potentiation, compatibility effects would be expected from the relation between the left-right orientation of the object and the hand used to make the response, the latter being cued by object inversion.

Figure 1. Examples of the stimuli used in the experiments. Experiments 1 and 2: right orientation, upright (frying pan); left orientation, inverted (teapot). Experiment 3: anticlockwise wrist rotation compatibility, inverted (knife); clockwise wrist rotation compatibility, upright ( aerosol can).

Exper imen t 1

Method

Participants. Thirty students took part in the experiment. All were enrolled at the University of Plymouth and received course credit for their participation. All participants had normal or corrected-to-normai vision and were naive as to the purpose of the experiment. All except 2 participants reported that they were right-handed.

Apparatus and materials. Black and white transparencies of 22 graspable household objects made up the stimulus set (see Appen- dix A for a fist of objects used). All the objects were capable of being grasped and manipulated by one hand and were photographed in two horizontal orientations (one compatible with a right-hand grasp, the other with a left-hand grasp) and two vertical orientations (upright and inverted). There were thus 22 × 2 × 2 = 88 slides that were back-projected onto a translucent screen (46 × 46 cm) from two Kodak carousel random access projectors, modified to allow millisecond shutter control. Examples of the stimuli are shown in Figure 1. The participant was seated with his or her head 45 cm in front of the screen and with the index finger of each hand resting on two response buttons 30 cm apart and 15 cm in front of the screen. The objects were photographed so as to appear as if they were resting on the table at the position of the screen, at approximately their actual size, at a distance of 50 cm. They subtended visual angles of between 11" and 18".

Design and procedure. The experiment consisted of two blocks of 176 trials in which each object appeared twice in each horizontal and vertical orientation. Participants were instructed to make push-button responses with the left or right hand depending on whether the object was upright or inverted. The actual mapping of response hand to object inversion was blocked and pseudoran- domized so that an equal number of participants received each mapping in the first block. For most objects, whether the object was upright or inverted needed no definition. In the case of objects such as a knife or saw, participants were told that upside down or upright was defined with regard to the object's normal use. Such objects were thus photographed with the blade at right angles to the resting surface, rather than lying flat, and were upside down when the blade or teeth were pointing up rather than down. Participants experienced no difficulty in understanding this definition of inver-

sion. The left-right horizontal orientation of the object was irrelevant to the response. Participants were instructed to respond as fast as possible whilst maintaining accuracy. Slide order was randomized for each participant, and the experiment was run, and response latencies recorded, on an Acorn Archimedes computer. Each participant received 20 practice trials before each block. A trial began with the appearance of an object on the screen and ended when a response had been made or 3 s had elapsed. The objects remained in view until a response was made. There was a 4-s delay between the end of one trial and the beginning of the next. Participants were not given feedback on response latencies, but errors were immediately followed by a short tone from the computer.

Results

Response times. Two participants were removed from the analysis because their error rates exceeded 10%. Error trials and reaction times more than 2 SDs from the condition means were excluded from the analysis. The means for each object in each of the eight conditions were computed for each participant. For the participants analysis, condition means were obtained by averaging across objects, and for the materials analysis they were obtained by averaging across participants. An analysis of variance (ANOVA) was conducted on the participant data with the independent variables o f mapping (right-hand-upright/left-hand-inverted or left-hand-upright/right-hand-inverted [RH-UP and LH-UP, respectively]), response (left hand or right hand), and object orientation (left or right). There was a significant main effect of response mapping. Responses in the R H - U P mapping (M = 616.68 ms) were faster than responses in the L H - U P mapping (M = 650.35 ms), F(1, 27) = 8.61, p < .01. The only other significant effects were the two-way interactions between response mapping and hand of response and between hand of response and left-fight object orientation. The interaction between mapping and hand of response is easily interpretable as an effect of object inversion. Right-hand responses in the R H - U P

Normal use?

Covert task demands

A motor intention can be automatically triggered when we pay attention to the functional properties of an object


for each subject.















Do you notice anything?

The participant was seated with his or her head 45 cm in front of the screen and with the index finger of each hand resting on two response buttons 30 cm apart and 15 cm in front of the scree

The two response buttons were 2.5 cm apart and positioned centrally 15 cm in front of the viewing screen

The findings …. are consistent with a long line of research, started by Tucker and Ellis (1998), demonstrating that people automatically activate motor programs when viewing manipulable objects.


for each subject.















X

Alignment effects on keypresses but no test of the possibility that these effects are confined to responses with the left or right hand.

Oguro, Ward, Bracewel, Hindle, Rafal (2009) - - Object was a frying pan. Pellicano, Iani, Borghi, Rubichi, Nicoletti (2010) - - Object was a flashlight. Tipper, Paul and Hayes (2006) - - Object was a door handle. Loach, Frischen, Bruce and Tsotsos (2008) - - Object was a door handle. Sevos, Grosselin, Pellet, Massoubre and Brouillet (2013) - - Pictures of 22 handled objects (mug, frying pan, etc). Murphy, van Velzen and de Fockert (2012) - - Pictures of 5 handled objects. Kostove and Janyan (2012) - - 12 images of common graspable objects. Galpin, Tipper, Dick and Poliakoff (2011) - - Object was a door handle. Poliakoff, Galpin, Dick, Moore and Tipper (2007) - - Object was a door handle. Anderson, Yamagishi and Karavia (2002) - - a variety of familiar and unfamiliar objects.

Limits on Action Priming by Pictures of Objects

Alfred B. YuWashington University in St. Louis and Army ResearchLaboratory, Aberdeen Proving Ground, Maryland

Richard A. Abrams and Jeffrey M. ZacksWashington University in St. Louis

When does looking at an object prime actions associated with using it, and what aspects of those actionsare primed? We examined whether viewing manmade objects with handles would selectively facilitateresponses for the hand closest to the handle, attempting to replicate a study reported by Tucker and Ellis(1998). We also examined whether the hypothesized action priming effects depended upon the responsehand’s proximity to an object. In 7 experiments, participants made judgments about whether picturedobjects were manmade or natural or whether the objects were upright or inverted. They responded bypressing buttons located either on the same or opposite side as the objects’ handles, at variable distances.Action priming was observed only when participants were explicitly instructed to imagine picking up thepictured objects while making their judgments; the data provide no evidence for task-general automaticpriming of lateralized responses by object handles. These data indicate that visually encoding an objectactivates spatially localized action representations only under special circumstances.

Keywords: action priming, affordance, stimulus-response compatibility

Does viewing a graspable object inevitably prime the motorsystem for action? This intuitive proposal has received consider-able recent attention—and considerable support. Viewing manip-ulable objects has been found to facilitate actions that are typicallyassociated with using them (e.g., Tucker & Ellis, 1998; Bub,Masson, & Cree, 2008; Borghi, Bonfiglioli, Ricciardelli, Rubichi,& Nicoletti, 2007). In particular, reaching and grasping responsesare sometimes facilitated when they are compatible with an ob-ject’s afforded actions and inhibited when they are incompatible.Such action priming effects have been reported for a number oftasks involving left- and right-handed button presses (Tucker &Ellis, 1998; Tipper, Paul, & Hayes, 2006; Iani, Baroni, Pellicano,& Nicoletti, 2011) and grasping responses (Bub & Masson, 2006;Tucker & Ellis, 2004).However, there are still important open questions about when

viewing an object primes an action related to that object and whichaspects of that action are primed. A number of researchers haveargued that objects automatically prime actions (Tucker & Ellis,1998; Grèzes & Decety, 2002; Borghi, 2005; Makris, Hadar, &Yarrow, 2011), which is consistent with some embodied theories

of cognition where action plans are intrinsic to the conceptualrepresentation of objects (e.g., Barsalou, 1999; Derbyshire, Ellis,& Tucker, 2006). According to those views, under typical circum-stances, viewing an object should automatically activate someaspect of an action plan relevant to using that object. Presumably,automatic priming implies that affordances should prime actions inthe absence of an explicit intention to pick up or use the objects.We examine this proposition in the present study. Our resultsindicate that action plans are not automatically activated uponviewing objects, although they can be activated through an effort-ful process involving motoric simulation.When one reaches out to pick up an object, the action represen-

tation must specify where in space the reach will be directed andhow one’s grip will be formed. Ellis and Tucker (2000) referred tothese distinct components of potential actions as microaffordances.There is good reason to think that these aspects are controlleddifferently. Grip form is controlled by the distal musculature,whereas location is controlled by the proximal musculature, andthese are associated with different cortical circuits (Haaxma &Kuypers, 1974; Berlucchi, Aglioti, & Tassinari, 1994). Grip formis likely to be consistently associated with an object type—grasp-ing a pencil will almost always entail a precision grip, whereasgrasping a coconut will nearly always entail a power grip. Loca-tion, on the other hand, is less likely to be consistently associatedwith an object type. Pencils and coconuts can both appear at manydifferent locations relative to the body. Therefore, different aspectsof the motor control system may be recruited, depending onwhether a movement requires fine control over location or gripparameters. These two aspects of motor control may be sensitive todifferent features of visually presented stimuli. This raises animportant question about the role of object representations inaction control: Do semantic representations of objects contributedifferently to the control of movement location and fine motorparameters? Two previous studies have addressed this question

This article was published Online First July 21, 2014.Alfred B. Yu, Department of Psychology, Washington University in St.

Louis, and Human Research and Engineering Directorate, Army ResearchLaboratory, Aberdeen Proving Ground, Maryland; Richard A. Abrams andJeffrey M. Zacks, Department of Psychology, Washington University in St.Louis.This work was supported by a United States Department of Defense

Science, Mathematics, and Research for Transformation (SMART) schol-arship to the first author, and NIMH Grant RO1MH70674 and NIA GrantR01AG031150 to the third author.Correspondence concerning this article should be addressed to Alfred B.

Yu, United States Army Research Laboratory, RDRL-HRS-C, AberdeenProving Ground, MD 21005. E-mail: [email protected]

Journal of Experimental Psychology:Human Perception and Performance

In the public domain

2014, Vol. 40, No. 5, 1861–1873http://dx.doi.org/10.1037/a0037397

1861

People stuck to the idea and resorted to “Butsee”

The findings …. are consistent with a long line of research, started by Tucker and Ellis (1998), demonstrating that people automatically activate motor programs when viewing manipulable objects.

CHAPTER SIX

Does the Concept of AffordanceAdd Anything to Explanations ofStimulus–Response CompatibilityEffects?Robert W. Proctor*,1, James D. Miles†*Department of Psychological Sciences, Purdue University, West Lafayette, Indiana, USA†Department of Psychology, California State University Long Beach, Long Beach, California, USA1Corresponding author: e-mail address: [email protected]

Contents

1. Introduction 2282. Information Processing and SRC 2293. Ecological Approach to Perception 2304. Affordance Accounts of SRC Effects 2325. Ecological Affordance Accounts of SRC Effects 233

5.1 Catching Affordance 2335.2 State of the Action System 2395.3 Summary 241

6. Representational Affordance Accounts of SRC Effects 2426.1 Grasping Affordance with Keypress Responses 2426.2 Grasping Affordance with Grasp Responses 2536.3 TRoPICALS: A Representational Affordance Model 256

7. Conclusion 260References 261

Abstract

The concept of affordance has been increasingly applied to stimulus–response compat-ibility effects over the past 25 years, for which most explanations have been from aninformation-processing perspective. We consider affordance accounts offered fromthe ecological perception approach associated with J. J. Gibson and from theinformation-processing approach (which we call representational affordance accounts).With regard to the latter, we discuss whether any value is gained by incorporating aconcept from one worldview (ecological psychology) into explanations within anotherworldview (information processing). We discuss shortcomings of the representationalaffordance approach in general, including lack of clear justification and definition forthe concept of affordance representation, and critically evaluate several lines of researchthat have been interpreted as support for specific affordances. We conclude that there is

Psychology of Learning and Motivation, Volume 60 # 2014 Elsevier Inc.ISSN 0079-7421 All rights reserved.http://dx.doi.org/10.1016/B978-0-12-800090-8.00006-8

227

CHAPTER SIX

Does the Concept of AffordanceAdd Anything to Explanations ofStimulus–Response CompatibilityEffects?Robert W. Proctor*,1, James D. Miles†*Department of Psychological Sciences, Purdue University, West Lafayette, Indiana, USA†Department of Psychology, California State University Long Beach, Long Beach, California, USA1Corresponding author: e-mail address: [email protected]

Contents

1. Introduction 2282. Information Processing and SRC 2293. Ecological Approach to Perception 2304. Affordance Accounts of SRC Effects 2325. Ecological Affordance Accounts of SRC Effects 233

5.1 Catching Affordance 2335.2 State of the Action System 2395.3 Summary 241

6. Representational Affordance Accounts of SRC Effects 2426.1 Grasping Affordance with Keypress Responses 2426.2 Grasping Affordance with Grasp Responses 2536.3 TRoPICALS: A Representational Affordance Model 256

7. Conclusion 260References 261

Abstract

The concept of affordance has been increasingly applied to stimulus–response compat-ibility effects over the past 25 years, for which most explanations have been from aninformation-processing perspective. We consider affordance accounts offered fromthe ecological perception approach associated with J. J. Gibson and from theinformation-processing approach (which we call representational affordance accounts).With regard to the latter, we discuss whether any value is gained by incorporating aconcept from one worldview (ecological psychology) into explanations within anotherworldview (information processing). We discuss shortcomings of the representationalaffordance approach in general, including lack of clear justification and definition forthe concept of affordance representation, and critically evaluate several lines of researchthat have been interpreted as support for specific affordances. We conclude that there is

Psychology of Learning and Motivation, Volume 60 # 2014 Elsevier Inc.ISSN 0079-7421 All rights reserved.http://dx.doi.org/10.1016/B978-0-12-800090-8.00006-8

227

Does the Concept of Affordance Add Anything to Explanations of Stimulus–Response Compatibility Effects? Robert W. Proctor*,1, James D. Miles†*Department of Psychological Sciences, Purdue University, West Lafayette, Indiana, USA

†Department of Psychology, California State University Long Beach, Long Beach, California, USA 1Corresponding author: e-mail address: [email protected]

Song, Chen, and Proctor (2013) pursued the issue of whether the cor-

respondence effect obtained by Pellicano et al. (2010) with orientation judg-

ments could indeed be attributed to a grasping affordance. Song et al.

showed in a first experiment that the pattern of results reported by

Pellicano et al. (2010) for orientation judgments could be replicated, finding

an 18 ms correspondence effect for stimuli in the active state and no such

effect for stimuli in the passive state. Their experiments 2 and 3 sought to

separate effects due to asymmetrical visual properties of the stimuli from

those related to grasping. When the graspable handle was removed from

the flashlight in experiment 2, the correspondence effect increased in size,

Figure 6.8 Depiction of Pellicano et al.'s flashlight stimuli for the orientation-judgmentexperiment. The top two flashlights are in the active state, whereas the bottom two arein the passive state. The upper member of each pair is in inverted orientation, with thehandle to the right, and the lower member is in upright orientation, with the handle tothe left. The vertical lines are to illustrate the asymmetry of the strips, which was morepronounced with the flashlight in the active than passive state. From Cho and Proctor(2013), reprinted with permission.

250 Robert W. Proctor and James D. Miles

Thus, the results argue against a grasping-affordance account and are explainable in terms of left–right asymmetries of visual features.

The Simon Effect

Left Right

Task: Right key for blue, left key for green

+

Spatially Incompatible

The Simon Effect

Left Right

Task: Right key for blue, left key for green

+

Spatially CompatibleSimon Effect:RT incompatible - RT compatible

Vertically oriented stimulus and response sets

Increasing functions According to Wascher et al.’s (2001)account, when stimuli and responses vary along the verticaldimension (up and down locations), the Simon effect is notdue to automatic activation (because there is no directcorrespondence of up and down stimulus locations with leftand right hands), and therefore, the vertical Simon taskshould show a flat or an increasing effect function. Stürmeret al. (2002, Experiment 3) obtained results consistent withthis prediction in an experiment that used vertical S–Rarrangements as a control procedure for measuring LRPs.Participants responded to a square or a diamond stimulus,displayed in an upper or lower location on the screen, bypressing an upper or lower key on a keypad, which was in anormal position on the table. The Simon effect was largefor trials following corresponding trials and absent for thosefollowing noncorresponding trials, a result that has beenreplicated several times (e.g., Hommel, Proctor, & Vu,2004). Of importance, the effect following correspondingtrials increased from about 40 ms at the shortest of 20 RTbins to more than 60 ms at the last few bins, whereas theeffect following noncorresponding trials drifted from 0 toslightly negative (<10 ms) at the last couple of RT bins. Thecombined Simon effect function has a slightly positiveslope, and not the negative slope customarily observed forleft and right locations.

Wiegand and Wascher (2005, Experiment 1) replicatedthe increasing Simon effect function for the verticaldimension and the typical decreasing effect function forthe horizontal dimension in a single experiment (Fig. 5, leftpanel). As in Wascher et al. (2001), participants respondedto the letter A or B, this time presented in one of five boxes,one at fixation and the others to the left, right, top, andbottom of the fixated box (with the remaining boxes filledwith neutral filler lines). The response keys were arrayedvertically, and half of the participants responded with theright hand on the top key and the left on the bottom, andvice versa for the other half. The Simon effect for verticalstimulus location with key location increased from 10 ms atthe shortest of nine RT bins to 23 ms at the longest. Incontrast, the Simon effect for left and right stimuluslocations with left and right hands showed the typicaldecreasing function (beginning at 30–35 ms and decreasingto about 10 ms). This result pattern is consistent with thedistinction between the translation mechanism and visuo-motor activation, because the above–below stimuli did nothave an anatomical correspondence with the left and righthands, whereas the left–right stimuli did, which would leadto increasing and decreasing functions, respectively.

Proctor, Vu, and Nicoletti (2003) examined the Simontask for a two-dimensional version of the Simon task inwhich a single stimulus can correspond or not with theresponse on both the vertical and horizontal dimensions. In

Fig. 5 The Simon effect magnitude (in milliseconds) for each bin asshown in Wiegand and Wascher’s (2005) Experiment 1 (left panel)and Proctor et al.’s (2003) Experiment 1 (right panel), respectively. Adecreasing function for horizontal stimuli (plotted by the bold line inboth figures) and an increasing time course for vertically aligned ones,were obtained in both experiments. Left panel from Wiegand andWascher (2005), Dynamic aspects of stimulus-response correspon-

dence. Evidence for two mechanisms involved in the Simon effect.Journal of Experimental Psychology: Human Perception and Perfor-mance, 31, 453–464. Reprinted with permission of the AmericanPsychological Association. Right panel from Proctor et al. (2003),Does right–left prevalence occur for the Simon effect? Perception &Psychophysics, 65, 1318–1329. Reprinted with permission of thePsychonomic Society, Inc

Psychon Bull Rev (2011) 18:242–266 253

+

Object Centered

Handle+Bowl

Body CenteredBowl only

activation of the corresponding response, followed by dissipationof this activation (e.g., De Jong et al., 1994). The opposite findingof an increase in the Simon effect across the RT distribution for thefrying pan stimuli indicates greater variability for the noncorre-sponding trials than for the corresponding trials (Zhang & Korn-blum, 1997). This greater variability could be taken to suggest thatthe coding of relative position of the object handle does not occurimmediately and thus exerts more influence on those trials forwhich the participant is taking longer to respond. The pattern ofincreasing effects is in agreement with the distributional analysesfor power and precision grasp responses reported by Tucker andEllis (2001) and Derbyshire et al. (2006) and for keypress re-sponses reported by Riggio et al. (2008). It also provides evidencethat converges with the results of Phillips and Ward’s (2002)precuing study with keypress responses, which showed that theeffect of an object precue increased as the SOA between it and asuperimposed imperative stimulus lengthened.

Experiment 2

Experiment 1 differed from that of Tucker and Ellis (1998) inusing color as the relevant stimulus dimension for the object-basedSimon task, rather than up or down orientation of the graspableobject part. This difference in color versus orientation judgmentscould be the cause of the discrepant results. Symes, Ellis, andTucker (2005) found that the Simon effect for orientation of agraspable object part was evident when people had to classifyobjects by keypresses according to whether they were used in the

kitchen or garage (Experiment 1) but not when they had to classifythem as red or green color (Experiment 2). Similarly, Tipper, Paul,and Hayes (2006) found a Simon effect of door-handle directionwhen responses were based on the object’s shape but not whenthey were based on its color, leading Tipper et al. to conclude thataffordance-based motor activation did not occur for color discrim-inations.If it is necessary to attend to an object’s shape for a grasping

affordance to be activated, then Tucker and Ellis’s (1998) findingof no within-hand Simon effect for object properties should bereplicated when the required judgments relate to an object’s shaperather than its color. Therefore, Experiment 2 was conductedsimilarly to the object-based condition of Experiment 1, but par-ticipants had to judge whether the pan handle was angled towardthe top or bottom (i.e., whether the pan orientation was upright orinverted), as in Tucker and Ellis’s study.Two additional stimulus sets were tested that were similar to the

frying pans but without a grasping affordance. For one set, only thehandle tip and pan were visible. This disembodied handle-tipcondition is similar to one from Tipper et al.’s (2006) study, inwhich they found that the affordance-based effects they obtainedwere eliminated when the hinge/door attachment component wasremoved from door handles, making them not appear to be han-dles. For the remaining stimulus set, the stimuli were dashed linesat the same angles as the pan handles, presented without the pan.If a grasping affordance contributes to performance with the fryingpans when people are judging orientation, then the within-hand

Figure 2. Within- and between-hand Simon effects (in milliseconds) plotted as a function of the mean RT foreach quartile in the different task conditions of Experiments 1–3.

857OBJECT-BASED SIMON EFFECT

This

doc

umen

t is c

opyr

ight

ed b

y th

e A

mer

ican

Psy

chol

ogic

al A

ssoc

iatio

n or

one

of i

ts a

llied

pub

lishe

rs.

This

arti

cle

is in

tend

ed so

lely

for t

he p

erso

nal u

se o

f the

indi

vidu

al u

ser a

nd is

not

to b

e di

ssem

inat

ed b

road

ly.

activation of the corresponding response, followed by dissipationof this activation (e.g., De Jong et al., 1994). The opposite findingof an increase in the Simon effect across the RT distribution for thefrying pan stimuli indicates greater variability for the noncorre-sponding trials than for the corresponding trials (Zhang & Korn-blum, 1997). This greater variability could be taken to suggest thatthe coding of relative position of the object handle does not occurimmediately and thus exerts more influence on those trials forwhich the participant is taking longer to respond. The pattern ofincreasing effects is in agreement with the distributional analysesfor power and precision grasp responses reported by Tucker andEllis (2001) and Derbyshire et al. (2006) and for keypress re-sponses reported by Riggio et al. (2008). It also provides evidencethat converges with the results of Phillips and Ward’s (2002)precuing study with keypress responses, which showed that theeffect of an object precue increased as the SOA between it and asuperimposed imperative stimulus lengthened.

Experiment 2

Experiment 1 differed from that of Tucker and Ellis (1998) inusing color as the relevant stimulus dimension for the object-basedSimon task, rather than up or down orientation of the graspableobject part. This difference in color versus orientation judgmentscould be the cause of the discrepant results. Symes, Ellis, andTucker (2005) found that the Simon effect for orientation of agraspable object part was evident when people had to classifyobjects by keypresses according to whether they were used in the

kitchen or garage (Experiment 1) but not when they had to classifythem as red or green color (Experiment 2). Similarly, Tipper, Paul,and Hayes (2006) found a Simon effect of door-handle directionwhen responses were based on the object’s shape but not whenthey were based on its color, leading Tipper et al. to conclude thataffordance-based motor activation did not occur for color discrim-inations.If it is necessary to attend to an object’s shape for a grasping

affordance to be activated, then Tucker and Ellis’s (1998) findingof no within-hand Simon effect for object properties should bereplicated when the required judgments relate to an object’s shaperather than its color. Therefore, Experiment 2 was conductedsimilarly to the object-based condition of Experiment 1, but par-ticipants had to judge whether the pan handle was angled towardthe top or bottom (i.e., whether the pan orientation was upright orinverted), as in Tucker and Ellis’s study.Two additional stimulus sets were tested that were similar to the

frying pans but without a grasping affordance. For one set, only thehandle tip and pan were visible. This disembodied handle-tipcondition is similar to one from Tipper et al.’s (2006) study, inwhich they found that the affordance-based effects they obtainedwere eliminated when the hinge/door attachment component wasremoved from door handles, making them not appear to be han-dles. For the remaining stimulus set, the stimuli were dashed linesat the same angles as the pan handles, presented without the pan.If a grasping affordance contributes to performance with the fryingpans when people are judging orientation, then the within-hand

Figure 2. Within- and between-hand Simon effects (in milliseconds) plotted as a function of the mean RT foreach quartile in the different task conditions of Experiments 1–3.

857OBJECT-BASED SIMON EFFECT

This

doc

umen

t is c

opyr

ight

ed b

y th

e A

mer

ican

Psy

chol

ogic

al A

ssoc

iatio

n or

one

of i

ts a

llied

pub

lishe

rs.

This

arti

cle

is in

tend

ed so

lely

for t

he p

erso

nal u

se o

f the

indi

vidu

al u

ser a

nd is

not

to b

e di

ssem

inat

ed b

road

ly.

Simon effect

Object-based Simon effect

TIME COURSE OF MOTOR AFFORDANCES 1

Running head: TIME COURSE OF MOTOR AFFORDANCES

Time Course of Motor Affordances Evoked by Pictured Objects

Daniel N. Bub, Michael E. J. Masson, and Ragav Kumar

University of Victoria

Correspondence to: Michael Masson Department of Psychology University of Victoria P.O. Box 1700 STN CSC Victoria BC V8W 2Y2 Canada tel: 250-721-7536 fax: 250-721-8929 email: [email protected]

Word count: 12,154

+

Object Centered

Handle+Bowl

Body CenteredBowl only

Given that a handled object can be used with either the left orright hand depending on its orientation, consideration of the ob-ject’s affordance may generate a preference to respond with thealigned hand that interacts with the selection of the hand requiredfor a key-press response. In support of this argument, we note thatTipper et al. (2006) presented short video clips of a hand reachingtoward, grasping, and pushing down a door handle, so as toenhance the very minimal effects of alignment obtained in pilotexperiments that omitted any demonstration of the object beingused. Furthermore, alignment effects were greater if the doorhandle was displayed slanting downward to the left or right asthough in the act of being turned, and more modest (even afterseeing the video clip) if subjects viewed the handle in a horizontalposition. Clearly, much was done in this study to encourageobservers to consider the dynamic properties of door handles.Without this background context, no alignment effects occurredwhen observers merely pressed a response button to signal theirperceptual judgments. Tucker and Ellis (1998) similarly instructedsubjects to judge the upright or inverted orientation of objects likea knife or saw by considering their use. Subjects may have appliedthis instruction quite generally in deciding on the orientation ofhandled objects, implicitly attending to objects with respect to theactions needed to use them.

Evoking Automatic Handle Alignment Effects

The points argued above are intended to cast some doubt on theview that object identification alone triggers hand action represen-tations that influence lef̃t–right button presses. Constraints on thealignment effects discussed so far imply that observers must alsobe biased by intentional set to consider how they would hold or usethe manipulable objects they identify. We distinguish betweenthese effects—which appear to be induced by explicit consider-ation of an object’s affordance—and the automatic activation ofaction representations evoked by a manipulable object in tasks thatdo not require observers to attend to the object’s affordances.

We will use the term standard mapping to refer to automaticallyevoked action representations that are independent of an observ-er’s intentions. The actions pertinent to the observer’s intentionswe will call contextual mapping (cf. Murray, Bussey, & Wise,2000). For example, a beer mug oriented with its handle to theright would have a standard mapping to a right-handed clenchedgrasp. The contextual mapping will be congruent with this stan-dard mapping if the observer has the intention that the right handpick up the object by its handle. Of course, contextual mapping isflexible in that intended hand actions on the beer mug may conflictwith the standard mapping, as when the observer decides to use theleft hand to grasp a right facing handle.

We have developed a behavioral visuomotor task that pits thestandard mapping of a reach and grasp action, driven by thestimulus properties of a handled object like a beer mug or fryingpan, against a contextual mapping between an arbitrary cue, likecolor, and an action. Subjects are trained to make speeded reachand grasp actions in response to a color patch, using a singleresponse element placed directly in front of them (see Figure 1A).One color signals that the action is to be carried out with the lefthand; another color indicates the same action with the right hand.

Under what circumstances might the image of a handled objectthat is irrelevant to the formal requirements of the contextual

mapping task yield motor representations that interact with theaction determined by color? We have already emphasized thecrucial, albeit tacit role played by intentional set in generatingalignment effects in previous research using speeded responses toincidental properties (orientation or shape) of handled objects (e.g.,Tipper et al., 2006; Tucker & Ellis, 1998). With respect to ourensemble of color and object, it is color, not the handled object,that forms the basis for decision and action. The nature of themotor response in the contextual mapping task (mapping color toaction), however, may determine whether the standard object–action mapping exerts an influence.

Neuropsychological studies offer direct support for the claim thatthe nature of the intended motor response can play a fundamental rolein modulating the influence of the standard mapping between objectsand actions. Riddoch et al. (1998) have demonstrated in neurologicalcases that the kind of intended action applied to an object modulatesinterference effects caused by involuntary evocation of the standardmapping. For example, in the contextual mapping task, the requiredaction was to pick up a cup on the left or right side of the table usingthe hand on the same side as the object, regardless of which way thehandle of the cup was oriented (e.g., a left-hand grasp was required fora cup on the left, irrespective of whether the cup handle faced left orright). Despite being able to articulate the rule, these patients—evincing a form of utilization behavior—often produced the wrongresponse (e.g., grasping a cup on the left with the right hand) when thehandle of the cup was aligned with the responding hand. Crucially,these inadvertent responses driven by the handle were not observedwhen the task was simply to point with the left or right hand depend-ing on the location of the cup. Thus, intrusion of standard mapping ona contextual mapping task can be modulated by the intended action(see also Hommel, 2000; Humphreys & Riddoch, 2007; Linnell,Humphreys, McIntyre, Laitinen, & Wing, 2005).

We define intentional set operationally, therefore, just in termsof the hand action used by subjects to indicate their perceptualdecisions about the color of an object. The shape of the responseelement subjects grasp in response to color can be chosen to afforda particular hand posture; for example, a horizontal cylinder re-quires a (wrist) inverted, closed grasp, whereas for a verticallyoriented C-shaped response element (see Figure 1A), the handposture required is (wrist) vertical, clenched grasp. In this way, itis possible to vary the parameters of the final goal state associatedwith the color in relation to the standard mapping afforded by the

Figure 1. The response elements (A) and grayscale versions of the objectphotographs (B) used in Experiment 1.

343DYNAMICS OF HANDLE ALIGNMENT EFFECTS

Given that a handled object can be used with either the left orright hand depending on its orientation, consideration of the ob-ject’s affordance may generate a preference to respond with thealigned hand that interacts with the selection of the hand requiredfor a key-press response. In support of this argument, we note thatTipper et al. (2006) presented short video clips of a hand reachingtoward, grasping, and pushing down a door handle, so as toenhance the very minimal effects of alignment obtained in pilotexperiments that omitted any demonstration of the object beingused. Furthermore, alignment effects were greater if the doorhandle was displayed slanting downward to the left or right asthough in the act of being turned, and more modest (even afterseeing the video clip) if subjects viewed the handle in a horizontalposition. Clearly, much was done in this study to encourageobservers to consider the dynamic properties of door handles.Without this background context, no alignment effects occurredwhen observers merely pressed a response button to signal theirperceptual judgments. Tucker and Ellis (1998) similarly instructedsubjects to judge the upright or inverted orientation of objects likea knife or saw by considering their use. Subjects may have appliedthis instruction quite generally in deciding on the orientation ofhandled objects, implicitly attending to objects with respect to theactions needed to use them.

Evoking Automatic Handle Alignment Effects

The points argued above are intended to cast some doubt on theview that object identification alone triggers hand action represen-tations that influence lef̃t–right button presses. Constraints on thealignment effects discussed so far imply that observers must alsobe biased by intentional set to consider how they would hold or usethe manipulable objects they identify. We distinguish betweenthese effects—which appear to be induced by explicit consider-ation of an object’s affordance—and the automatic activation ofaction representations evoked by a manipulable object in tasks thatdo not require observers to attend to the object’s affordances.

We will use the term standard mapping to refer to automaticallyevoked action representations that are independent of an observ-er’s intentions. The actions pertinent to the observer’s intentionswe will call contextual mapping (cf. Murray, Bussey, & Wise,2000). For example, a beer mug oriented with its handle to theright would have a standard mapping to a right-handed clenchedgrasp. The contextual mapping will be congruent with this stan-dard mapping if the observer has the intention that the right handpick up the object by its handle. Of course, contextual mapping isflexible in that intended hand actions on the beer mug may conflictwith the standard mapping, as when the observer decides to use theleft hand to grasp a right facing handle.

We have developed a behavioral visuomotor task that pits thestandard mapping of a reach and grasp action, driven by thestimulus properties of a handled object like a beer mug or fryingpan, against a contextual mapping between an arbitrary cue, likecolor, and an action. Subjects are trained to make speeded reachand grasp actions in response to a color patch, using a singleresponse element placed directly in front of them (see Figure 1A).One color signals that the action is to be carried out with the lefthand; another color indicates the same action with the right hand.

Under what circumstances might the image of a handled objectthat is irrelevant to the formal requirements of the contextual

mapping task yield motor representations that interact with theaction determined by color? We have already emphasized thecrucial, albeit tacit role played by intentional set in generatingalignment effects in previous research using speeded responses toincidental properties (orientation or shape) of handled objects (e.g.,Tipper et al., 2006; Tucker & Ellis, 1998). With respect to ourensemble of color and object, it is color, not the handled object,that forms the basis for decision and action. The nature of themotor response in the contextual mapping task (mapping color toaction), however, may determine whether the standard object–action mapping exerts an influence.

Neuropsychological studies offer direct support for the claim thatthe nature of the intended motor response can play a fundamental rolein modulating the influence of the standard mapping between objectsand actions. Riddoch et al. (1998) have demonstrated in neurologicalcases that the kind of intended action applied to an object modulatesinterference effects caused by involuntary evocation of the standardmapping. For example, in the contextual mapping task, the requiredaction was to pick up a cup on the left or right side of the table usingthe hand on the same side as the object, regardless of which way thehandle of the cup was oriented (e.g., a left-hand grasp was required fora cup on the left, irrespective of whether the cup handle faced left orright). Despite being able to articulate the rule, these patients—evincing a form of utilization behavior—often produced the wrongresponse (e.g., grasping a cup on the left with the right hand) when thehandle of the cup was aligned with the responding hand. Crucially,these inadvertent responses driven by the handle were not observedwhen the task was simply to point with the left or right hand depend-ing on the location of the cup. Thus, intrusion of standard mapping ona contextual mapping task can be modulated by the intended action(see also Hommel, 2000; Humphreys & Riddoch, 2007; Linnell,Humphreys, McIntyre, Laitinen, & Wing, 2005).

We define intentional set operationally, therefore, just in termsof the hand action used by subjects to indicate their perceptualdecisions about the color of an object. The shape of the responseelement subjects grasp in response to color can be chosen to afforda particular hand posture; for example, a horizontal cylinder re-quires a (wrist) inverted, closed grasp, whereas for a verticallyoriented C-shaped response element (see Figure 1A), the handposture required is (wrist) vertical, clenched grasp. In this way, itis possible to vary the parameters of the final goal state associatedwith the color in relation to the standard mapping afforded by the

Figure 1. The response elements (A) and grayscale versions of the objectphotographs (B) used in Experiment 1.

343DYNAMICS OF HANDLE ALIGNMENT EFFECTS

Object turns green:

Make a speeded reach- and-grasp action with your right hand.

Micro-affordances: Right-hand vertical palm power grasp.

Object turns blue:

Make a speeded reach- and-grasp action with your left hand.

mapping on the production of a reach and grasp response. A veryearly influence would be reflected in effects appearing across theentire response time distribution, whereas a late influence wouldlead to effects being found only among longer latency trials.

Liftoff time. Mean correct liftoff times are shown in the toppart of Figure 2 as a function of handle alignment, congruencybetween object and grasp response, and cue delay. These data weresubmitted to an analysis of variance (ANOVA) with alignment andcue delay as repeated-measures factors and congruency as abetween-subjects factor. Type I error rate was set at .05 for this andall analyses reported here. The ANOVA revealed significant ef-fects of handle alignment, F(1, 94) ! 19.78, mean square error(MSE) ! 214, "p

2 ! .17, and cue delay, F(1, 94) ! 86.62, MSE !343, "p

2 ! .48. The interaction between alignment and cue delaywas also significant, F(1, 94) ! 11.19, MSE ! 87, "p

2 ! .11. Inaddition, however, the three-way interaction was significant, F(1,94) ! 6.76, MSE ! 87, "p

2 ! .07. The pattern of means in Figure 2indicates that the three-way interaction emerged because the han-dle alignment effect was present for congruent grasps (12 ms) onlywhen the color cue was delayed (immediate onset: F # 1; delayedonset: F(1, 47) ! 16.57, MSE ! 207, "p

2 ! .26), whereas a small

alignment effect (7 ms) was present for incongruent grasps bothwith immediate color onset and with delayed onset (immediateonset: F(1, 47) ! 9.04, MSE ! 104, "p

2 ! .16; delayed onset: F(1,47) ! 9.73, MSE ! 145, "p

2 ! .17).We further examined the handle alignment effect by plotting the

cumulative response time distribution for each congruency bycue-delay condition, as shown in Figure 2. These distributions arebased on the mean liftoff time for equal-sized quintiles (i.e., liftofftimes for each subject were rank ordered and the first quintileconsisted of the first [shortest] 20% of the observations; the secondquintile consisted of the next 20%, and so on). ANOVAs wereconducted to determine whether alignment effects varied acrossquintiles. The main effect of quintile was, of course, alwayssignificant so we do not discuss that point further. For the con-gruent grasp condition with immediate color onset, the interactionbetween alignment and quintile only approached significance oncethe Greenhouse-Geisser correction for violation of sphericity wasapplied, F(4, 188) ! 3.29, MSE ! 170, p # .06, "p

2 ! .07. AsFigure 2 indicates, there was a small alignment effect (9 ms)emerging in the longest quintile. For incongruent grasps withimmediate color onset, the interaction between alignment andquintile was significant, F(4, 188) ! 24.27, MSE ! 140, "p

2 ! .34,with alignment effects apparent only in the two longest quintiles (8ms and 26 ms). With delayed color onset, both grasp types pro-duced alignment effects that did not significantly vary acrossquintiles, F # 1 for congruent grasps, F(4, 188) ! 2.21, MSE !101, p $ .10, for incongruent grasps.

Clear alignment effects in the congruent condition emerged onlywhen the standard mapping evoked by the handled object wasgiven a head start by delaying the onset of the color cue. In theincongruent condition, alignment effects were observed in the0-delay condition, but only in the two longest response-time quin-tiles. This pattern of results suggests that the evocation of thestandard mapping is slightly delayed for congruent actions. Wepropose that in the congruent condition, presentation of the objectinitially evokes a representation of hand shape congruent with theobject. We assume that this representation is evoked for bothhands, creating a competition that must eventually be resolved infavor of one or the other hand. This view is consistent withneurophysiological evidence indicating that patterns of activity inpremotor cortex, especially during preparatory stages of process-ing, can be relatively independent of the choice of response hand(Cisek, Crammond, & Kalaska, 2003). Bilateral preparation givesway to an effector-dependent representation as the movementunfolds. Consequently, it will take some time for the ensuingcompetition between the two hands to be tipped in favor of the sidealigned with the handle, leading to a delay in the alignment effect.

For the incongruent condition, the action cued by the colorconflicts with (rather than supports) the action evoked by theobject. This conflict will weaken the degree to which the handshape associated with the object is evoked during the preparatorystage of the movement. As a result, the competition between thetwo hands is reduced and the influence of the handle’s position canhave an earlier impact on hand selection.

Movement time. Mean time required to move the responsehand after liftoff to make contact with the response element isshown in Figure 3. These data and the analyses we report are basedonly on 43 subjects in the congruent condition because movementtime data for the remaining five subjects were lost due to an

Figure 2. Mean liftoff time (upper panels) and associated cumulativeresponse time distributions (middle and lower panels) in Experiment 1.Error bars indicate the within-subjects 95% confidence interval based onthe MSE for the comparison between aligned and not-aligned conditions ateach cue delay (Loftus & Masson, 1994; Masson & Loftus, 2003). Thecumulative response time distributions show the effect of handle alignmentseparately for congruent (left panels) and incongruent (right panels) graspsin each cue delay condition. A response-time advantage is indicated in thecumulative distributions when a function reaches a given probability at anearlier response time value (i.e., is displaced to the left).

346 BUB AND MASSON

Okay!

But the effects are slow to accrue over time

SOA

Bub and Masson also found that the influence of object handle alignment developed slowly—alignment effects were larger as reaction times increased. This increasing effect of handle alignment was attributed by them to low dimensional overlap between the object and response type, as prescribed by Kornblum et al. (1990) (see also Lu & Proctor, 2001).

Bub and Masson’s (2010) view is again more consistent with a spatial coding account than a true affordance one previously mentioned, the dimensional overlap model is fundamentally representational in nature.

Second, as previously suggested by Phillips and Ward (2002), this slow development of the alignment effect is not consistent with a functionally automatic process such as an affordance, which brings about rapid interaction with the environment. Third, similar increases in the Simon effect are also observed for task stimuli that present spatial information in a more abstract format (and therefore cannot be considered affordances) such as words and arrows, indicating a longer processing time for the spatial coding of complex semantic information (Proctor, Miles, & Baroni, 2011). Accordingly, the time course of alignment effects for objects with handles may reflect the activation of complex semantic spatial codes that in turn activate associated response codes rather than object affordances directly activating the associated actions.


Figure 1. Illustration of the sequence of events on a trial and the response elements from

Experiment 1. In this case, the stimulus onset asynchrony was 250 ms. The object prime and

hand cue shown here are from Experiment 1B and the actual hand cue was flesh colored in that

experiment. The response elements are shown in the lower left of the figure.

and moved it to the target element. Contact with the element closed an electrical circuit that

signaled a computer, allowing us to measure the response time. In the second training phase,

subjects named each of the prime objects, presented in their upright orientation, two times. Some

latitude was allowed in the names generated by the subjects (e.g., glass instead of beer mug) as

long as they were used consistently.

In the test phase, each trial began with a fixation cross presented at the center of the

computer monitor, which was the cue for the subject to ensure that the index of each hand was

resting on one of the response box buttons. Once their fingers were in place, the fixation cross


Figure 1. Illustration of the sequence of events on a trial and the response elements from Experiment 1. The stimulus onset asynchrony of 250 ms is illustrated here for each condition of the experiment. After viewing the prime for 250 ms, the hand cue was superimposed on the object prime. Relative to the vertical/horizontal orientation and the right/left position of the object's handle, the cued hand action was congruent or incongruent and aligned or not aligned with the prime, as indicated here. The relevant response element for a particular cued response is shown in each example. The object primes and hand cues shown here are from Experiment 1B and the actual hand cue was flesh colored (as shown in the online version of this article) in that experiment.


Figure 2. Mean response time and mean alignment and congruency effect in Experiment 1 as a

function of prime orientation and stimulus onset asynchrony (SOA). Means for response times

are jittered on the horizontal axis for clarity. Aligned/NotAlign = response hand aligned or not

aligned with the position of prime object's handle; Cong/Incon = wrist orientation of required

action congruent or incongruent with the depicted view of the prime object's handle. Error bars

for mean response times are 95% within-subject confidence intervals suitable for comparing all

four means within a given (SOA) condition (Loftus & Masson, 1994; Masson & Loftus, 2003).

Error bars for effect sizes are 95% highest density intervals.


Figure 3. Mean alignment and congruency effect at each response time quintile in Experiment 1.

Means are positioned horizontally according to the mean response time within a given quintile,

averaged across both conditions being compared to produce the indicated effect. Error bars

indicate the 95% highest density intervals.

When rotated primes were presented, there was modest evidence for a congruency effect at

the 0-ms SOA, but this effect favored the canonical orientation of the primes, rather than the

orientation actually in view. We take this outcome as evidence for the convergent impact of two,






When rotated primes were presented, there was modest evidence for a congruency effect at

the 0-ms SOA, but this effect favored the canonical orientation of the primes, rather than the

orientation actually in view. We take this outcome as evidence for the convergent impact of two,


0.8% for 0-ms and 250-ms SOA, respectively. No effects were present in the analysis of movement time. When rotated primes were used, there was no indication of an alignment effect (BF = 7.1 favoring the null hypothesis) and there was only weak evidence for an interaction between alignment and SOA (BF = 2.0 favoring the presence of an interaction). Figure 2 shows, however, that there is evidence for an alignment effect at 250-ms SOA (the 95% highest posterior density interval for plausible effect sizes excludes zero). We note that the lack of a reliable interaction between alignment and SOA indicates that we cannot claim that the alignment effect is reliably larger at 250-ms than at 0-ms SOA. Although the overall effect of congruency was not supported (BF = 5.8 favoring the null hypothesis), there was strong evidence for an interaction between congruency and SOA (BF > 1,000). As indicated in Figure 2, a modest negative effect of congruency was apparent at 0-ms SOA, consistent with an influence of the canonical orientation of the prime, but a robust positive congruency effect occurred at 250-ms SOA (in both cases, the 95% highest posterior density interval excludes zero). The overall error rate in the rotated prime condition was 1.5%, and a Bayesian analysis indicated that none of the effects in the design were supported (BF > 2.6 favoring the null hypothesis in all cases). This outcome indicates that the response time effects are not compromised by speed/accuracy trade-offs. The analysis of movement time found only an effect of SOA (BF = 47.5), with shorter movement time when the SOA was 250 ms. To obtain a more detailed view of the speed with which alignment and congruency effects accrue in the generation of reach-and-grasp responses, we examined these effects across the full distribution of response times. Doing so allows us to ask whether these effects are apparent even among the fastest responses that subjects produce. The analysis involved assigning each subject's rank-ordered response times within a condition into successive, equal-size quintiles, with the 20% shortest response times assigned to the first quintile, the next 20% to the second quintile, and so on (see Dittrich, Kellen, & Stahl, 2014; Ridderinkhof, 2002; and Yap, Balota, & Tan, 2013, for other applications of this method). We next computed each subject's mean response time within each quintile for each condition. For the alignment effect, conditions were defined as aligned and not-aligned, collapsing across congruency. For the congruency effect, congruent and incongruent condition means were obtained by averaging across the alignment variable. We computed the relevant effect at each quintile by taking the difference between the aligned and not-aligned, or the congruent and incongruent conditions, to determine the size of the relevant effect at each quintile for each subject.

Figure 3. Mean alignment and congruency effect at each response time quintile in Experiment 1. Means are positioned horizontally according to the mean response time within a given quintile, averaged across both conditions being compared to produce the indicated effect. Error bars indicate the 95% highest density intervals. The mean alignment and congruency effects across quintiles are shown in Figure 3. These functions are referred to as delta plots, as they express the size of the difference between conditions as a function of response-time quantile (Ridderinkhof, 2002). In all cases, there is no indication that effect size increases with longer response times. If anything, there is a general tendency for effects to diminish at longer response times. The relatively flat functions indicate that the effect in question persists across the full response-time distribution, suggesting that the effect has shifted the entire distribution rather than, for example, influencing only the trials with the longest response times (Yap et al., 2013). For upright primes, there is evidence for both alignment and congruency effects in the first quintile, corresponding to the fastest group of responses, and this occurs at both the 0-ms and 250-ms SOA. For rotated primes, only at the 250-ms SOA are both alignment and congruency effects apparent in the shortest quintiles. The results of Experiment 1 indicate that for upright primes alignment and, to some extent at least, congruency effects emerge very early during the processing of a pictured object. Although the alignment effect for upright primes increased with longer SOA, the effect was clearly present even among the fastest responses with a 0-ms SOA. Further, there was no tendency for this alignment effect to increase with longer response times, contrary to what might be expected if the source of the effect




The outcome ofthe action to the object is automaticallyaccessed.


Figure 1. Illustration of the sequence of events on a trial and the response elements from

Experiment 1. In this case, the stimulus onset asynchrony was 250 ms. The object prime and

hand cue shown here are from Experiment 1B and the actual hand cue was flesh colored in that

experiment. The response elements are shown in the lower left of the figure.

and moved it to the target element. Contact with the element closed an electrical circuit that

signaled a computer, allowing us to measure the response time. In the second training phase,

subjects named each of the prime objects, presented in their upright orientation, two times. Some

latitude was allowed in the names generated by the subjects (e.g., glass instead of beer mug) as

long as they were used consistently.

In the test phase, each trial began with a fixation cross presented at the center of the

computer monitor, which was the cue for the subject to ensure that the index of each hand was

resting on one of the response box buttons. Once their fingers were in place, the fixation cross

Beer Mug 300 ms


Figure 6. Mean response time, mean congruency and hand-dominance effect, and mean

congruency effect at each response time quintile in Experiment 3. Means for response time

quintiles are positioned horizontally according to the mean response time within a given quintile,

averaged across congruent and incongruent conditions. Error bars for mean response times are

95% within-subject confidence intervals suitable for comparing all four means, and error bars for

plots of effects indicate the 95% highest density intervals.






Consistent with what we observed in Experiment 1, alignment and congruency effects

clearly emerge with a 0-ms SOA between prime object and action cue, and are present even

among the fastest responses that subjects produce. These findings are not consistent with the

suggestion (e.g., Proctor & Miles, 2014) that priming of reach-and-grasp responses by pictures of

handled objects is a product of abstract spatial coding of object features. Rather, they are

compatible with the proposition that object primes directly evoke their associated action

representations, which in turn allow for more efficient programming and execution of compatible

reach-and-grasp movements.

Experiment 3

As a test of the validity of our interpretation of the results of Experiments 1 and 2, we

Beer Mug

Research by a number of teams is converging to suggest that (grasp) actions activate regions of lateral occipital cortex that also respond to images of hands (Astafiev et al., 2004; Peelen and Downing, 2005; Orlov et al., 2010; Bracci et al., 2012). Moreover, a previous paper from Gallivan and colleagues reported that upcoming hand actions (grasping versus reaching with the fingers) can be decoded in regions of ventral and lateral temporal-occipital cortex that were independently defined as showing differential BOLD responses for different categories of objects (e.g., objects, scenes, body parts; Gallivan et al., 2013a). Furthermore, the regions of lateral occipital cortex that respond specifically to images of hands are directly adjacent to those that respond specifically to images of tools, and also exhibit strong functional connectivity with areas of somatomotor cortex (Bracci et al., 2012).

elife.elifesciences.org

Mahon. eLife 2013;2:e00866. DOI: 10.7554/eLife.00866 1 of 3

In our daily lives, we interact with a vast array of objects—from pens and cups to hammers and cars. Whenever we recognize and use an object,

our brain automatically accesses a wealth of back-ground knowledge about the object’s structure, properties and functions, and about the move-ments associated with its use. We are also con-stantly observing our own actions as we engage with objects, as well as those of others. A key ques-tion is: how are these distinct types of information, which are distributed across different regions of the brain, integrated in the service of everyday behavior? Addressing this question involves specifying the internal organizational structure of the representations of each type of information, as well as the way in which information is exchanged or combined across different regions. Now, in eLife, Jody Culham at the University of Western Ontario (UWO) and co-workers report a significant advance in our understanding of these ‘big picture’ issues by showing how a specific type of information about object-directed actions is coded across the brain (Gallivan et al., 2013b).

Copyright Mahon. This article is distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use and redistribution provided that the original author and source are credited.

INSIGHT

NEUROSCIENCE

Watching the brain in actionFunctional magnetic resonance imaging has been used to identify the different networks in the brain that underpin the use of tools by humans.

BRADFORD Z MAHON

Related research article Gallivan JP,

McLean DA, Valyear KF, Culham JC. 2013.

Decoding the neural mechanisms of

human tool use. eLife 2:e00425.

doi: 10.7554/eLife.00425

Image Certain brain regions respond

preferentially to viewing tools and planning

tool actions (red) or to viewing the human

body and planning hand actions (green)

A great deal is known about which brain regions represent and process different types of knowledge about objects and actions (Martin, 2007). For instance, visual information about the structure and form of objects, and of body parts, is represented in ventral and lateral temporal occipital regions (Goodale and Milner, 1992). Visuomotor processing in support of object-directed action, such as reaching and grasping, is represented in dorsal occipital and posterior parietal regions (Culham et al., 2003). Knowledge about how to manipulate objects according to their function is represented in inferior-lateral parietal cortex, and in premotor regions of the frontal lobe (Culham et al., 2003; Johnson-Frey, 2004).

Culham and colleagues—who are based at the UWO, Queen’s University and the University of Missouri, and include Jason Gallivan as first author—focus their investigation on the neural substrates that underlie our ability to grasp objects. They used functional magnetic resonance imaging (fMRI) to scan the brains of subjects performing a task in which they had to alternate between using their hands or a set of pliers to reach towards or grasp an object. Ingeniously, the pliers were reverse pliers—constructed so that the business end opens when you close your fingers, and closes when your fingers open. This made it possible to dissociate the goal of each action (e.g., ‘grasp’) from the movements involved in its execution (since in the case of the pliers, ‘grasping’ is accomplished by opening the hand).

Gallivan et al. used multivariate analyses to test whether the pattern of responses elicited across a set of voxels (or points in the brain)

16

Figure 2.3: Trial procedure; 0-ms SOA condition (top), 250-ms SOA condition (bottom).

2.1.2 Results and Discussion

On average, accuracy on the hand classification task was 97%. The mean accuracy

of object identification was 76%. Data were analysed using the R statistical language (R

Core team, 2016). Response time was measured from the onset of the hand cue to the

completion of a key-press response. Response times shorter than 100 ms were excluded

as probable anticipatory responses. Response times longer than 1,400 ms were excluded

as outliers. This upper bound was set so that no more than 0.5% of correct responses were

excluded, in keeping with a recommendation by Ulrich and Miller (1994).

16










17 The mean response times as a function of SOA, alignment, and congruency are

presented in Figure 2.4.An analysis of variance (ANOVA) was computed with SOA,

alignment, and congruency as repeated-measures factors. This analysis indicated that

responses on aligned trials were faster than on misaligned trials (F [1, 36] =100.90,

MSE=1,451, p<0.0001). Responses were also faster on congruent trials than on

incongruent trials (F [1, 36] =8.89, MSE=825, p <0.01).

Responses in the 250-ms SOA condition were faster than in the 0-ms SOA

condition (F [1, 36] =208.30, MSE=5,148, p<0.0001). An interaction between SOA and

alignment was found (F [1, 36] =7.30, MSE=1,016, p=0.01) indicating that the alignment

effect was bigger at the longer SOA. No other interactions were significant Fs< 1.74.

Figure 2.4: Mean response time in Experiment I as a function of prime alignment

and prime congruency across 0-ms and 250-ms SOA. Error Bars represent 95% within-subjects confidence intervals appropriate for evaluating the alignment effect and the congruency effect within each SOA condition (Loftus & Masson, 1994; Masson & Loftus, 2003).

46

Appendix B Alignment effect size across objects Experiments I, II and III

Figure b1: Alignment effect size collapsed across objects collapsed across congruency and SOA conditions in Experiment I. Error Bars represent 95% within-subjects confidence intervals appropriate for evaluating the alignment effect for each object individually.

19 An additional object analysis was conducted to evaluate the alignment effect size

across each object (see Appendix B).

The effect of alignment found in Experiment I was present at the shortest response

times and remained relatively constant across the response time distribution. Object-

based Simon effects typically grow across the response distribution, showing little or no

effect in among the shortest response times (Proctor et al., 2011).A departure from this

pattern suggests that the underlying mechanism for the alignment effect may involve

action representations associated with the object primes. Importantly, these results were

produced by making a key press on a response box rather than a reach-and-grasp action

on a grasping element. Therefore, the presence of grasp elements could not be

responsible for the alignment and congruency effects observed here. The image of an

object itself evoked representations of relevant actions.

Figure 2.5: Delta plot of effect size across quintile in Experiment I, collapsed across

SOA and congruency.



What the is the differencebetween the followingtwo tasks in regard to thetriggering of grasp affordances?

a) Task 1: Classify the image ofthe hand as left versus rightwith a left or right handed keypress.

b) Task 2: Respond the image ofa left versus right hand with a reach-and-grasp action.


Nevertheless, for the sake of completeness we also mention the outcome of analyses of movement time as a dependent measure. Response times longer than 2,900 ms in Experiment 1A (0.48%) and 2,000 ms in Experiment 1B (0.50%) were treated as outliers and were excluded from analyses. These cutoffs were determined so that no more than 0.5% of correct responses were excluded from either experiment (Ulrich & Miller, 1994). A response was classified as an error if the subject made contact with the wrong response element, responded with the incorrect hand, or applied an incorrect grasp to an element. Although we did not record error types, casual reports from the experimenters indicated that most errors were of the former two types. Errors were excluded from the response time analyses. Data were analyzed by computing the Bayes factor favoring either a model that included the effect of interest or a model that excluded that effect. In addition, mean effects are plotted with 95% highest posterior density intervals, which show the 95% most plausible estimated values for effect size (Morey, Hoekstra, Rouder, Lee, & Wagenmakers, 2016). Analyses were computed using the BayesFactor package in R with default settings for parameters (see Rouder, Morey, Speckman, & Province, 2012, for the theoretical foundation of the methods used in this package, and Rouder, Morey, Verhagen, Swagman, & Wagenmakers, in press, for a practical explanation of its use). A Bayes factor greater than 3.0 is generally deemed to be at least "positive" evidence for the favored model over the less likely model (Raftery, 1995). In reporting our results, we report the Bayes factor (BF) and indicate whether it favors the presence or absence of an effect. Whenever we report that the Bayesian analysis indicates the presence of an effect, that effect would also be significant in a standard null-hypothesis significance test with p < .05. Analyses of the –250-ms SOA condition in Experiment 1A revealed no evidence for priming effects for either upright or rotated primes in either response times or error rates (BF > 2.4 favoring the null hypothesis in all cases). Mean response time for that SOA condition was 1,727 ms and the mean percent error was 0.6%. Given that this SOA condition produced no priming effects, we combined the data from the 0-ms and 250-ms SOA conditions across Experiments 1A and 1B for the analyses that we report here. Mean response time for upright and for rotated prime conditions are shown in Figure 2. Our primary interest was in testing the presence of congruency and alignment effects at the 0-ms and 250-ms SOA conditions, and so we also present the mean congruency and alignment effects in Figure 2. The alignment effect was defined as the difference in response time between trials where the handle was

Figure 2. Mean response time and mean alignment and congruency effect in Experiment 1 as a function of prime orientation and stimulus onset asynchrony (SOA). Means for response times are jittered on the horizontal axis for clarity. Aligned/NotAlign = response hand aligned or not aligned with the position of prime object's handle; Cong/Incon = wrist orientation of required action congruent or incongruent with the depicted view of the prime object's handle. Error bars for mean response times are 95% within-subject confidence intervals suitable for comparing all four means within a given (SOA) condition (Loftus & Masson, 1994; Masson & Loftus, 2003). Error bars for effect sizes are 95% highest density intervals. positioned on the same side of midline as the cued response hand (aligned) and trials where the handle and the response hand were on opposite sides of the midline (not aligned). The congruency effect was defined as the difference between response times when the depicted orientation of the object's handle was congruent versus incongruent with respect to the orientation of the cued hand action. Effect sizes are plotted with 95% highest posterior density intervals, representing the most plausible values for effect size. Considering first the priming effects for upright objects, a Bayesian analysis with alignment, congruency, and SOA as factors indicated that subjects responded faster with a longer SOA and that both congruency and alignment effects were observed (BF > 1,000 in all cases). In addition, the alignment effect increased with SOA (BF = 324.7), whereas there was no clear evidence for such an increase for the congruency effect (BF = 1.2 favoring the null hypothesis). An analysis of error rates, however, indicated that the congruency effect was stronger with the longer rather than the shorter SOA (BF = 5.7). The overall error rate when upright primes were used was 1.6%, and the congruency effect went from -0.2% to


Nevertheless, for the sake of completeness we also mention the outcome of analyses of movement time as a dependent measure. Response times longer than 2,900 ms in Experiment 1A (0.48%) and 2,000 ms in Experiment 1B (0.50%) were treated as outliers and were excluded from analyses. These cutoffs were determined so that no more than 0.5% of correct responses were excluded from either experiment (Ulrich & Miller, 1994). A response was classified as an error if the subject made contact with the wrong response element, responded with the incorrect hand, or applied an incorrect grasp to an element. Although we did not record error types, casual reports from the experimenters indicated that most errors were of the former two types. Errors were excluded from the response time analyses. Data were analyzed by computing the Bayes factor favoring either a model that included the effect of interest or a model that excluded that effect. In addition, mean effects are plotted with 95% highest posterior density intervals, which show the 95% most plausible estimated values for effect size (Morey, Hoekstra, Rouder, Lee, & Wagenmakers, 2016). Analyses were computed using the BayesFactor package in R with default settings for parameters (see Rouder, Morey, Speckman, & Province, 2012, for the theoretical foundation of the methods used in this package, and Rouder, Morey, Verhagen, Swagman, & Wagenmakers, in press, for a practical explanation of its use). A Bayes factor greater than 3.0 is generally deemed to be at least "positive" evidence for the favored model over the less likely model (Raftery, 1995). In reporting our results, we report the Bayes factor (BF) and indicate whether it favors the presence or absence of an effect. Whenever we report that the Bayesian analysis indicates the presence of an effect, that effect would also be significant in a standard null-hypothesis significance test with p < .05. Analyses of the –250-ms SOA condition in Experiment 1A revealed no evidence for priming effects for either upright or rotated primes in either response times or error rates (BF > 2.4 favoring the null hypothesis in all cases). Mean response time for that SOA condition was 1,727 ms and the mean percent error was 0.6%. Given that this SOA condition produced no priming effects, we combined the data from the 0-ms and 250-ms SOA conditions across Experiments 1A and 1B for the analyses that we report here. Mean response time for upright and for rotated prime conditions are shown in Figure 2. Our primary interest was in testing the presence of congruency and alignment effects at the 0-ms and 250-ms SOA conditions, and so we also present the mean congruency and alignment effects in Figure 2. The alignment effect was defined as the difference in response time between trials where the handle was

Figure 2. Mean response time and mean alignment and congruency effect in Experiment 1 as a function of prime orientation and stimulus onset asynchrony (SOA). Means for response times are jittered on the horizontal axis for clarity. Aligned/NotAlign = response hand aligned or not aligned with the position of prime object's handle; Cong/Incon = wrist orientation of required action congruent or incongruent with the depicted view of the prime object's handle. Error bars for mean response times are 95% within-subject confidence intervals suitable for comparing all four means within a given (SOA) condition (Loftus & Masson, 1994; Masson & Loftus, 2003). Error bars for effect sizes are 95% highest density intervals. positioned on the same side of midline as the cued response hand (aligned) and trials where the handle and the response hand were on opposite sides of the midline (not aligned). The congruency effect was defined as the difference between response times when the depicted orientation of the object's handle was congruent versus incongruent with respect to the orientation of the cued hand action. Effect sizes are plotted with 95% highest posterior density intervals, representing the most plausible values for effect size. Considering first the priming effects for upright objects, a Bayesian analysis with alignment, congruency, and SOA as factors indicated that subjects responded faster with a longer SOA and that both congruency and alignment effects were observed (BF > 1,000 in all cases). In addition, the alignment effect increased with SOA (BF = 324.7), whereas there was no clear evidence for such an increase for the congruency effect (BF = 1.2 favoring the null hypothesis). An analysis of error rates, however, indicated that the congruency effect was stronger with the longer rather than the shorter SOA (BF = 5.7). The overall error rate when upright primes were used was 1.6%, and the congruency effect went from -0.2% to







The outcome ofthe action to the object is automaticallyaccessed.

A M CON 486.3 526.4 INC 502.2 533.0

A M CON 520.9 520.0 INC 526.7 533.1

Upright Rotated

Classifying the image of hand as left versus rightwith a keypress triggers a grasp action that doesnot take into account the outcome of an action.

16
















Planning a reach and grasp action to the image of a hand triggers a motor representation from a task-irrelevant depicted object that does include the outcome of the grasp action.

affordances - web...

Documents