Chapter 4: The Visual Cortex and Beyond

Overview of Questions

• How can brain damage affect a person’s perception?

• Are there separate brain areas that determine our perception of different qualities?

• How has the operation of our visual system been shaped by evolution and by our day-to-day experiences?

Pathway from Retina to Cortex

• Signals from the retina travel through the optic nerve to the

– Lateral geniculate nucleus (LGN)

– Primary visual receiving area in the occipital lobe (the striate cortex)

– And then through two pathways to the temporal lobe and the parietal lobe

– Finally arriving at the frontal lobe

Figure 4.1 (a) Side view of the visual system, showing the three major sites along the primary visual pathway where processing takes place: the eye, the lateral geniculate nucleus, and the visual receiving area of the cortex. (b) Visual system seen from underneath the brain showing how some of the nerve fibers from the retina cross over to the opposite side of the brain at the optic chiasm.

Processing in the LGN

• LGN cells have center-surround receptive fields.

• Major function of LGN is to regulate neural information from the retina to the visual cortex.

– Signals are received from the retina, the cortex, the brain stem, and the thalamus.

– Signals are organized by eye, receptor type, and type of environmental information.

Figure 4.2 (a) Inputs and outputs of an LGN neuron. The neuron receives signals from the retina and also receives signals from the cortex, from elsewhere in the thalamus (T), from other LGN neurons (L), and from the brain stem. Excitatory synapses are indicated by Y’s and inhibitory ones by T’s. (b) Information flow into and out of the LGN. The sizes of the arrows indicate the sizes of the signals. (Part a adapted from Kaplan, Mukherjee, & Shapley, 1993.)

Organization in LGN

• Each LGN has six layers

– Each layer receives signals from only one eye

• Layers 2, 3, and 5 receive signals from the ipsilateral eye

• Layers 1, 4, and 6 receive signals from the contralateral eye

• Thus, each eye sends signals to both LGNs and the information for each eye is kept separated.

Figure 4.3 Cross section of the LGN showing layers. Red layers receive signals from the ipsilateral (same side of the body) eye. Blue layers receive signals from the contralateral (opposite side) eye.

Maps: Representing Spatial Layout

• Retinotopic map - each place on the retina corresponds to a place on the LGN

• Determining retinotopic maps - record from neurons with an electrode that penetrates the LGN obliquely

– Stimulating receptive fields on the retina shows the location of the corresponding neuron in the LGN

Figure 4.4 Points A, B, and C on the cup create images at A, B, and C on the retina and cause activation at points A, B, and C on the lateral geniculate nucleus (LGN). The correspondence between points on the LGN and retina indicates that there is a retinotopic map on the LGN.

Figure 4.5 Retinotopic mapping of neurons in the LGN. The neurons at A, B, and C in layer 6 of the LGN have receptive fields located at positions A’, B’, and C’ on the retina. This mapping can be determined by recording from neurons encountered along an oblique electrode track. Also, neurons along a perpendicular electrode track all have their receptive fields on about the same place on the retina.

The Map on the Striate Cortex

• Cortex shows retinotopic map, too.

– Electrodes that recorded activation from a cat’s visual cortex show:

• Receptive fields on the retina that overlap also overlap in the cortex.

• This pattern is seen using an oblique penetration of the cortex.

Figure 4.13 Retinotopic mapping of neurons in the cortex. When the electrode penetrates the cortex obliquely, the receptive fields of the neurons recorded from the numbered positions along the track are displaced, as indicated by the numbered receptive fields; neurons near each other in the cortex have receptive fields near each other on the retina.

Neurons in Striate Cortex

• Simple cortical cells – Side-by-side receptive fields– Respond to spots of light– Respond best to bar of light oriented along

the length of the receptive field• Orientation tuning curves

– Shows response of simple cortical cell for orientations of stimuli

Figure 4.6 (a) The receptive field of a simple cortical cell. (b) This cell responds best to a vertical bar of light that covers the excitatory area of the receptive field. The response decreases as the bar is tilted so that it also covers the inhibitory area. (c) Orientation tuning curve of a simple cortical cell for a neuron that responds best to a vertical bar (orientation = 0). (From Hubel & Wiesel, 1959.)

Neurons in Striate Cortex - continued

• Complex cells– Like simple cells

• Respond to bars of light of a particular orientation

– Unlike simple cells• Respond to movement of bars of light in

specific direction

Figure 4.8 (a) Response of a complex cell recorded from the visual cortex of a cat. The stimulus bar is moved back and forth across the receptive field. The cell fires best when the bar is positioned with a specific orientation and is moved in a specific direction (*). (From Hubel and Wiesel, 1959.) (b) Response of an end-stopped cell recorded from the visual cortex of the cat. The stimulus is indicated by the light area on the left. This cell responds best to a medium-sized corner that is moving up (*). (From “Receptive fields and functional architecture in two non-striate visual areas (18 and 19) of the cat,” by D. H. Hubel and T. N. Wiesel, 1965, Journal of Neurophysiology, 28, 229-289.)

Neurons in Striate Cortex - continued

• End-stopped cells– Respond to:

• Moving lines of specific length• Moving corners or angles

– No response to:• Stimuli that are too long

Feature Detectors

• Neurons that fire to specific features of a stimulus

• Pathway away from retina shows neurons that fire to more complex stimuli

• Cells that are feature detectors:– Simple cortical cell– Complex cortical cell– End-stopped cortical cell

Table 4.1 Properties of neurons in the optic nerve, LGN and cortical neurons

Selective Adaptation

• Neurons tuned to specific stimuli fatigue when exposure is long.

• Fatigue or adaptation to stimulus causes– Neural firing rate to decrease– Neuron to fire less when stimulus

immediately presented again• Selective means that only those neurons that

respond to the specific stimulus adapt.

Method for Selective Adaptation

• Measure sensitivity to range of one stimulus characteristic

• Adapt to that characteristic by extended exposure

• Re-measure the sensitivity to range of the stimulus characteristic

Stimulus Characteristics for Selective Adaptation

• Gratings are used as stimuli– Made of alternating light and dark bars– Angle relative to vertical can be changed to

test for sensitivity to orientation– Difference in intensity can be changed to

test for sensitivity to contrast

Figure 4.9 (a) Gratings that vary in orientation. (b) A vertical grating. The contrast is high for the gratings on the left, low for the ones on the right.

Method for Contrast Sensitivity

• Measure contrast threshold by decreasing intensity of grating until person can just see it.

• Calculate the contrast sensitivity by taking 1/threshold.

• If threshold is low, person has high contrast sensitivity.

Figure 4.10 Procedure for a selective adaptation experiment. See text for details.

Method for Orientation Sensitivity

• Use a high contrast grating• Measure sensitivity to different orientations• Adapt person to one orientation • Re-measure sensitivity to all orientations• Psychophysical curve should show selective

adaptation for specific orientation if neurons are tuned to this characteristic.

Figure 4.11 (a) Results of a psychophysical selective adaptation experiment. This graph shows that the participant’s adaptation to the vertical grating causes a large decrease in her ability to detect the vertical grating when it is presented again, but less effect on gratings that are tilted to either side of the vertical. (b) Orientation tuning curve of the simple cortical cell from Figure 4.6.

Selective Rearing Experiments

• Animals are reared in environments that contain only certain types of stimuli

– Neurons that respond to these stimuli will become more predominate due to neural plasticity.

– Blakemore and Cooper (1970) showed this by rearing kittens in tubes with either horizontal for vertical lines.

– Both behavioral and neural responses showed the development of neurons for the environmental stimuli.

Figure 4.12 (a) Striped tube used in Blakemore and Cooper’s (1970) selective rearing experiments. (b) Distribution of optimal orientations for 52 cells from a cat reared in an environment of horizontal stripes, on the left, and 72 cells from a cat reared in an environment of vertical stripes, on the right (From Blakemore & Cooper, 1970.)

Brain Imaging Techniques

• Positron emission tomography (PET)

– Person is injected with a harmless radioactive tracer

– Tracer moves through bloodstream

– Monitoring the radioactivity measures blood flow

– Changes in blood flow show changes in brain activity

Brain Imaging Techniques - continued

• PET - subtraction method

– Brain activity is determined by:

• Measuring activity in a control state

• Measuring activity in a stimulation state

• Subtracting the control activity from the stimulation activity

Figure 4.16 The subtraction technique that is used to interpret the results of brain imaging experiments. See text for explanation.

Brain Imaging Techniques - continued

• Functional magnetic resonance imaging (fMRI)

– Hemoglobin carries oxygen and contains a ferrous molecule that is magnetic

– Brain activity takes up oxygen, which makes the hemoglobin more magnetic

– fMRI determines activity of areas of the brain by detecting changes in magnetic response of hemoglobin

• Subtraction technique is used like in PET

Figure 4.17 (a) Red and blue areas show the extent of stimuli that were presented while a person was in an fMRI scanner. (b) Red and blue indicates areas of the brain activated by the stimulation in (a). (From Dougherty et al., 2003.)

Maps and Columns in the Striate Cortex

• Cortical magnification factor

– Fovea has more cortical space than expected

• Fovea accounts for .01% of retina

• Signals from fovea account for 8% to 10% of the visual cortex

• This provides extra processing for high-acuity tasks.

Figure 4.14 The magnification factor in the visual system: The small area of the fovea is represented by a large area on the visual cortex.

Organization in Columns - continued

• Visual cortex shows:

– Location columns

• Receptive fields at the same location on the retina are within a column

– Orientation columns

• Neurons within columns fire maximally to the same orientation of stimuli

• Adjacent columns change preference in an orderly fashion

• 1 millimeter across the cortex represents entire range of orientation

Figure 4.19 When an electrode penetrates the cortex perpendicularly, the receptive fields of the neurons encountered along this track overlap. The receptive field recorded at each numbered position along the electrode track is indicated by a correspondingly numbered square.

Figure 4.20 All of the cortical neurons encountered along track A respond best to horizontal bars (indicated by the red lines cutting across the electrode track.) All of the neurons along track B respond best to bars oriented at 45 degrees.

Figure 4.21 If an electrode is inserted obliquely into the cortex, it crosses a sequence of orientation columns. The preferred orientation of neurons in each column, indicated by the bars, changes in an orderly way as the electrode crosses the columns. The distance the electrode is advanced is exaggerated in this picture.

Organization in Columns - continued

• Visual cortex shows (cont.)

– Ocular dominance columns

• Neurons in the cortex respond preferentially to one eye.

• Neurons with the same preference are organized into columns.

• The columns alternate in a left-right pattern every .25 to .50 mm across the cortex.

Organization in Columns - continued

• Visual cortex shows (cont.)

– Hypercolumns contain:

• A single location column

• Left and right dominance columns

• A complete set of orientation columns (0 to 180 degrees)

• This is called the “ice-cube” model.

Figure 4.22 Schematic diagram of a hypercolumn as pictured in Hubel and Wiesel’s ice-cube model. The light area on the left is one hypercolumn, and the darkened area on the right is another hypercolumn. The darkened area is labeled to show that it consists of one location column, right and left ocular dominance columns, and a complete set of orientation columns.

Figure 4.24 How a tree creates an image on the retina and a pattern of activation on the cortex. See text for details.

Figure 4.25 How the trunk of the tree pictured in Figure 4.24 would activate a number of different orientation columns in the cortex.

Lesioning or Ablation Experiments

• First, an animal is trained to indicate perceptual capacities.

• Second, a specific part of the brain is removed or destroyed.

• Third, the animal is retrained to determine which perceptual abilities remain.

• The results reveal which portions of the brain are responsible for specific behaviors.

What and Where Pathways

• Ungerleider and Mishkin experiment

– Object discrimination problem

• Monkey is shown an object

• Then presented with two choice task

• Reward given for detecting the target object

– Landmark discrimination problem

• Monkey is trained to pick the food well next to a cylinder

What and Where Pathways - continued

• Ungerleider and Mishkin (cont.) - Using ablation, part of the parietal lobe was removed from half the monkeys and part of the temporal lobe was removed from the other half.

– Retesting the monkeys showed that:

• Removal of temporal lobe tissue resulted in problems with the landmark discrimination task - What pathway

• Removal of parietal lobe tissue resulted in problems with the object discrimination task - Where pathway

Figure 4.26 The two types of discrimination tasks used by Ungerleider and Mishkin. (a) Object discrimination: Pick the correct shape. Lesioning the temporal lobe (shaded area) makes this task difficult. (b) Landmark discrimination: Pick the food well closer to the cylinder. Lesioning the parietal lobe makes this task difficult. (From Mishkin, Ungerleider, & Macko, 1983.)

Figure 4.27 The monkey cortex, showing the what, or ventral pathway from the occipital lobe to the temporal lobe, and the where, or dorsal pathway from the occipital lobe to the parietal lobe. The where pathway is also called the how pathway. (From Mishkin, Ungerleider, & Macko, 1983.)

What and Where Pathways - continued

• What pathway also called doral pathway

• Where pathway also called ventral pathway

• Both pathways:

– originate in retina and continue through two types of ganglion cells in the LGN.

– have some interconnections.

– receive feedback from higher brain areas.

What and Where Pathways - continued

• Where pathway may actually be “How” pathway

– Dorsal stream shows function for both location and for action.

– Evidence from neuropsychology

• Single dissociations: two functions involve different mechanisms

• Double dissociations: two functions involve different mechanisms and operate independently

Table 4.2 A double dissociation

What and How Pathways - Neuropsycholgical Evidence

• Behavior of patient D.F.

– Damage to ventral pathway due to gas leak

– Not able to match orientation of card with slot

– But was able to match orientation if she was placing card in a slot

– Other patients show opposite effects

– Evidence shows double dissociation between ventral and dorsal pathways

Figure 4.29 Performance of D.F. and a person without brain damage for two tasks: (a) judging the orientation of a slot; and (b) placing a card through the slot. See text for details. (From Milner & Goodale, 1995.)

What and How Pathways - Further Evidence

• Rod and frame illusion

– Observers perform two tasks: matching and grasping

• Matching task involves ventral (what) pathway

• Grasping task involves dorsal (how) pathway

– Results show that the frame orientation affects the matching task but not the grasping task.

Figure 4.30 (a) Rod and frame illusion. Both small lines are oriented vertically. (b) Matching task and results. (c) Grasping task and results. See text for details.

Modularity: Structures for Faces, Places, and Bodies

• Module - a brain structure that processes information about specific stimuli

– Inferotemporal (IT) cortex in monkeys

• Responds best to faces with little response to non-face stimuli

– Temporal lobe damage in humans results in prosopagnosia.

Figure 4.32 (a) Monkey brain showing location of the inferotemporal (IT) cortex. (b) Human brain showing location of the fusiform face area (FFA), which is located under the temporal lobe.

Figure 4.33 Size of response of a neuron in the monkey’s IT cortex that responds to face stimuli but not to nonface stimuli. (Based on data from Rolls & Tovee, 1995.)

Modularity: Structures for Faces, Places, and Bodies - continued

• Evidence from humans using fMRI and the subtraction technique show:

– Fusiform face area (FFA) responds best to faces.

– Parahippocampal place area (PPA) responds best to spatial layout.

– Extrastriate body area (EBA) responds best to pictures of full bodies and body parts.

Figure 4.35 fMRI responses of the human brain to various types of stimuli: (a) areas that were most strongly activated by houses, faces, and chairs; (b) all areas activated by each type of stimulus. (From Alumit Ishai, Leslie G. Ungerleider, Alex Martin, & James V.Haxby,”The representation of objects in the human occipital and temporal cortex,” Journal of Cognitive Neuroscience, 12:2 (2000), 35-51.) © 2000 by the Massachusetts Institute of Technology.)

Evolution and Plasticity: Neural Specialization

• Evolution is partially responsible for shaping sensory responses:

– Newborn monkeys respond to direction of movement and depth of objects

– Babies prefer looking at pictures of assembled parts of faces

– Thus “hardwiring” of neurons plays a part in sensory systems

Evolution and Plasticity: Neural Specialization - continued

– Experience-dependent plasticity in humans

• Brain imaging experiments show areas that respond best to letters and words.

• fMRI experiments show that training results in areas of the FFA responding best to:

–Greeble stimuli

–Cars and birds for experts in these areas

Figure 4.36 (a) Greeble stimuli used by Gauthier. Participants were trained to name each different Greeble. (b) Brain responses to Greebles and faces before and after Greeble training. (Reprinted by permission from Macmillan Publishers Ltd: Nature Neuroscience, 2 568-573. From Figure 1a, p. 569, from Gauthier, I., Tarr, M. J., Anderson, A. W., Skudlarski, P. L., & Gore, J. C., Activation of the middle fusiform “face area” increases with experience in recognizing novel objects.” 1999.)