What made you respond face (or word)?

• Something in your brain made you decide face or word.

• Can we determine where this decision is made?

• Related domain: Motion Direction Discrimination and area MT

What determines the percept and the response?

• Observe a correlation between motion direction judgements and activity of cells in area MT. (Britten et al. 1996, Gold & Shadlen, 2000)

• If neurons that responded to leftward motion were highly active, the monkey chose 'left' as the decision.

From Schall, 1999

Evoked Response Potential (ERP) and Face Stimuli

• N170: negative-going potential at 170 ms

• Largest over the right parietal lobe, also on the left parietal lobe.

From Tanaka and Curran (2001)

N170 Properties:

• Faces produce the largest amplitude.

• Strong evidence of expertise: Bird experts have larger N170's to pictures of birds than pictures of dogs. Dog experts show the reverse. (Tanaka & Curran 2001).

• Mainly perceptually based: prior exposure of a face does not produce large changes in the N170 for subsequent presentations (Rik Henson, AIC 2003).

• Scalp distribution and latency suggest that the N170 component reflects the perceptual processing of complex visual stimuli.

A Thought Question:

• What was going on in your perceptual regions when you thought you saw a face or a word?

• Could we capture the current state (at least indirectly) with the N170 component?

• Would the N170 be larger when you thought you saw a face?

Central Question:

• Can we relate the size of the N170 to the response in the noise-alone condition?

• Will it be larger when subjects think they see a face?

An Experiment• Show Faces and Words Embedded in

Noise:

High Contrast

Faces

High Contrast Words

Low ContrastWords

Low Contrast

Faces

Noise Alone

Methods

• Face, word and noise-alone trials were presented in random order.

• Ten naïve participants– 120 trials per condition per subject

• On each trial: Did you see a face or a word?– Subjects were told that a stimulus appeared on each trial, and that they

should guess if they were unsure.

EEG Recording Sites

Methods continued

• Analyze the data according to the subject’s responses on the noise-alone trials.

• Incredibly important point: The noise was the same across ALL trials and stimuli. Physically the same. Not just identically distributed, but identical. There was only one noise field for the entire experiment.

Together, these procedures hold the physical stimulus constant on noise-alone trials.

-6

-4

-2

0

2

4

0 100 200 300 400 500 600

Right Temporal Lobe (T6)

time (ms)

High Contrast FaceHigh Contrast Word

Am

plit

ud

e (

V)

Central Question:

Will we see a larger N170 to the noise-alone stimulus when subjects think

they see a face as opposed to a word?

-6

-4

-2

0

2

4

0 100 200 300 400 500 600

Right Temporal Lobe (T6)

time (ms)

Responded 'Face' To Noise-Alone TrialResponded 'Word' To Noise-Alone Trial

High Contrast Face

Am

plit

ud

e (

V)

* t(9) = 2.74, p = .023, two-tailed

*

Main Result:• On noise-alone trials: Larger N170 when observers report

seeing a face than when report seeing a word.• Occurs in 9 of the 10 subjects.• No other differences in any other channel at the P100, N170

or P300 components.• Unlikely to just reflect activity for an already-made decision.• Relates activity in the perceptual processing areas to the

behavioral response. Greater activity in the N170 neurons is associated with ‘face’ responses to the noise-alone stimulus.

One interpretation: Greater activity in the face processing region biases the response towards a ‘face’ response.

Alternative Explanations for this Greater Activity

• Attention to different spatial frequencies or face-like features in the noise– Unlikely to see rapid changes in spatial frequency tuning in a

mixed design.– No P100 differences that might be associated with changes in

activity in different spatial frequency channels.

• One obvious exception: Prior trial priming– Seeing a face on the previous trial may leave residual activity in

the face neurons or make subjects look for face-like features in the noise.

Prior Trial Priming

Am

pli

tud

e (

V)

-4

-2

0

2

4

0 100 200 300

time (ms)

Noise-Alone Trials Conditioned on Prior Trial and Response

Responded'Word'

Responded'Face'

Face on Prior Trial

Word onPrior Trial

Third Possibility:Stochastic Activity

• In the domain of binocular rivalry, Blake and Logothetis introduced the idea of a process that involved stochastic activity in perceptual regions, which could bias the response toward one percept or the other at different points in time. (Blake & Logothetis 2002).

• A similar process could be at work in the face processing neurons:

When activity is high in the N170 neurons due to random stochastic fluctuations, the observer may be biased to

respond ‘face’ on that trial.

Implications for Internal Noise• Internal noise does not just limit performance or decrease

calculation efficiency, but also operates in feature space to bias the response toward one alternative or another.

• Manipulations such as varying the power of the noise or the stimulus pairs being compared may help constrain models of internal noise.

• Supports trial-by-trial variability in parameters:– Starting point and drift-rate variability in Ratcliff’s Diffusion

model.– These sources of variability take on a perceptually-based

interpretation. Link with perceptual brain areas.

The Important Stuff:

High Contrast

Faces

High Contrast Words

Low ContrastWords

Low Contrast

Faces

Noise Alone

-4

-2

0

2

4

0 100 200 300 400 500 600

Right Temporal Lobe (T6)

time (ms)

*

Responded 'Face' To Noise-Alone TrialResponded 'Word' To Noise-Alone Trial

For Noise-Alone Trials:

Am

plit

ude

(V

)

* t(9) = 2.74, p = .023

ERP and Faces

• Intracranial recordings reveal N200 at sites in IT/fusiform gyrus

From Allison, Puce, Spencer, and McCarthy (1999)

Which Stimuli Evoke an N170/N200?

Any face or face-like

visual stimulus

N170 Properties: Top-down Influences

• Could be just a face-detection system (Bentin, et. al. 1996)

• No effect of task demands such as selective attention to faces vs. objects. (Caquil, Edmonds, & Taylor, 2000)

• No effect of familiarity of the face (Bentin & Deouell, 2000)

• fMRI - IT active anytime there’s a face. (Gauthier et al. 1999)

• However, IT also active while imagining a face. (O’Craven & Kanwisher 2000)

Contextual Influences On N170

Bentin et al (2002)• Block 1:

– Stimulus Set A

• Block 2:– Set B (experimental)– Set C (control)– Set D (all; targets)

• Block 3:– Set A again (all)

Contextual Influences On N170Bentin et al (2002)

• Face context elicits N170 to schematic eyes

• Once the stimulus has been interpreted as containing face-like features, a stronger N170 is produced.

N170 Summary• Represents activity of face-selective neurons most

likely in area IT.• Magnitude varies with the degree to which

physiognomic information is perceived in the image.

• Is the N170 related to the response in the noise-alone condition?

• If so, tie the activity in face-selective cells to the percept and the response.

Would we get an N170 to the noise-alone trials at all?

• Previous work showed only a weak N170 to random noise stimuli.

• Pilot study: Blocked presentations of faces and words.• Result: when looking for faces, get a larger N170 to noise-

alone trials than when looking for words.• Replicates Bentin et al., extends to images that contain no

facial features. We see top-down effects on the N170 even with no face-like features.

Face Condition : Male or female?

Word condition:Honesty or Trust?

Experiment 1 Results - T5

-100 0 100 200 300 400 500 600-12

-10

-8

-6

-4

-2

0

2

4

6

8

Grand Average for T5

Time (ms)

Vo

ltage

(m

icro

volts

)

High-Contrast FaceLow-Contrast FaceZero-Contrast FaceHigh-Contrast WordLow-Contrast WordZero-Contrast Word

Experiment 1 Results - T5

*

-100 0 100 200 300 400 500 600-3

-2

-1

0

1

2

3

4

5

6

Grand Average for T5

Time (ms)

Vo

ltage

(m

icro

volts

)

*t(8) = 2.62, p = .031

Zero-Contrast WordZero-Contrast Face

Neural Processing of Face Stimuli

• Cells in inferotemporal cortex (IT) are known to respond selectively to faces– (single cell recording, fMRI)

• Slight right hemisphere dominance

Experiment 1 Results - T6

*-100 0 100 200 300 400 500 600-3

-2

-1

0

1

2

3

4

5

Grand Average for T6

Time (ms)

Vo

ltage

(m

icro

volts

)

* t(8) = 2.35, p = .047

Zero-Contrast WordZero-Contrast Face

Experiment 1 Conclusion

When an observer expects a face (versus a word)

there is a greater N170