neural mechanisms for evaluating environmental variability ... 2015 - neural mechanisms for... ·...

Article

Neural Mechanisms for Ev
aluating EnvironmentalVariability in Caenorhabditis elegans
Highlights

d C. elegans learn variability in their food environment and

modify future behavior

d The authors describe the minimal circuit that evaluates

environmental variability

d Dopamine levels store information about variability to

regulate behavior

d The amount of CREB determines the rate of learning

variability

Calhoun et al., 2015, Neuron 86, 1–14April 22, 2015 ª2015 Elsevier Inc.http://dx.doi.org/10.1016/j.neuron.2015.03.026

Authors

Adam J. Calhoun, Ada Tong, ...,

Tatyana O. Sharpee,

Sreekanth H. Chalasani

[email protected]

In Brief

How does an animal evaluate variability in

its environment? Calhoun et al. describe

how C. elegans sensory circuits learn

about food variability and use that

information to drive a long-lasting

behavior in a dopamine-dependent

manner.

mailto:[email protected]

http://dx.doi.org/10.1016/j.neuron.2015.03.026

Neuron

Article

Neural Mechanisms for Evaluating EnvironmentalVariability in Caenorhabditis elegansAdam J. Calhoun,1,2,3,6 Ada Tong,2 Navin Pokala,4 James A.J. Fitzpatrick,5 Tatyana O. Sharpee,1,3

and Sreekanth H. Chalasani1,2,*1Neurosciences Graduate Program, University of California, San Diego, La Jolla, CA 92093, USA2Molecular Neurobiology Laboratory, The Salk Institute for Biological Studies, La Jolla, CA 92037, USA3Computational Neurobiology Laboratory, The Salk Institute for Biological Studies, La Jolla, CA 92037, USA4HowardHughesMedical Institute, Lulu and AnthonyWang Laboratory of Neural Circuits and Behavior, The Rockefeller University, NewYork,

NY 10065, USA5Waitt Advanced Biophotonics Center, The Salk Institute for Biological Studies, La Jolla, CA 92037, USA6Present address: Princeton Neuroscience Institute, Princeton University, Princeton, NJ 08544, USA

*Correspondence: [email protected]


SUMMARY

The ability to evaluate variability in the environment isvital for making optimal behavioral decisions. Herewe show that Caenorhabditis elegans evaluatesvariability in its food environment and modifies itsfuture behavior accordingly. We derive a behavioralmodel that reveals a critical period over which infor-mation about the food environment is acquired andpredicts future search behavior. We also identify apair of high-threshold sensory neurons that encodevariability in food concentration and the downstreamdopamine-dependent circuit that generates appro-priate search behavior upon removal from food.Further, we show that CREB is required in a subsetof interneurons and determines the timescale overwhich the variability is integrated. Interestingly, thevariability circuit is a subset of a larger circuit drivingsearch behavior, showing that learning directly mod-ifies the very same neurons driving behavior. Ourstudy reveals how a neural circuit decodes environ-mental variability to generate contextually appro-priate decisions.

INTRODUCTION

Neural circuits respond to changing sensory environments and

generate appropriate behaviors by integrating prior experience

with new information. In particular, the variability of the environ-

ment plays a crucial role in shaping an animal’s behavioral strat-

egy, as unpredictable environments lead to unpredictable

reward. When presented with a choice, animals prefer a behav-

ioral strategy that will generate stable reward rather than uncer-

tain reward (MacLean et al., 2012; Platt and Huettel, 2008).

Selecting an uncertain rewardmay lead to dangerous outcomes.

For example, in the case of food, no reward (absence of food)

represents possible starvation (Watson, 2008). Thus, animals

devote considerable resources to determining reliability in their

environment (Escobar et al., 2007; MacLean et al., 2012). In

general, a successful behavioral strategy requires knowledge

about the variability in the environment (Tishby and Polani,

2011). Evaluating variability can be particularly challenging, as

the underlying neural circuit must learn and remember the vari-

ability in reward over different timescales.

In the visual system, the rate and statistics of action potential

firing encode information about rapidly occurring variations in

stimuli. Adaptation to variability in visual stimuli usually occurs

within seconds (Ozuysal and Baccus, 2012). Moreover, the

speed of resolving ambiguities approaches the physical limits

imposed by the sampling rate and noise of individual neurons

(DeWeese and Zador, 1998; Fairhall et al., 2001; Wark et al.,

2009). However, little is known about how the neural circuit eval-

uates reliability over minutes to generate complex behaviors

that last many minutes (Monosov and Hikosaka, 2013). We

approach this problem by studying how environmental variability

affects decision making in Caenorhabditis elegans. Its relatively

small nervous system, which consists of only 302 neurons with

identified synaptic connections, is ideally suited for identifying

neural mechanisms at the resolution of single cells and circuits

(Chalasani et al., 2007; Chalfie et al., 1985; de Bono and Maricq,

2005; White et al., 1986). Moreover, specific food-related

behaviors and their associated neuronal circuits are well-

characterized. For example, C. elegans removed from bacterial

food execute an initial local search of a restricted area for about

15 min by interrupting forward movements with seemingly

random reorientations (turns). After 15 min, animals disperse

by suppressing the reorientations and execute a global search

of a larger area (Gray et al., 2005; Hills et al., 2004; Wakabayashi

et al., 2004). The local followed by global search strategy is

similar to one adopted by ants returning to their nest (Lanan,

2014; Merkle and Wehner, 2008; Wehner et al., 2006). Given

the similarity in behavioral search strategies, we suggest that

the underlying neural mechanisms identified in C. elegans are

relevant across many species.

Cell ablation experiments identified a 48-neuron circuit in

C. elegans consisting of 6 sensory neurons, 16 interneurons,

and 26 motor neurons that regulates local search behavior

Neuron 86, 1–14, April 22, 2015 ª2015 Elsevier Inc. 1

Please cite this article in press as: Calhoun et al., Neural Mechanisms for Evaluating Environmental Variability in Caenorhabditis elegans, Neuron(2015), http://dx.doi.org/10.1016/j.neuron.2015.03.026

mailto:[email protected]


Flu

ores

cenc

e

A

Reo

rient

atio

ns/m

in

0

2

4 Large patchSmall patch

0

2

4

Local Global

***

n.s.

0 3015

local global

C

E

B

D

time

time

sim

ulat

ed c

hang

e in

bac

teria

Reo

rient

atio

ns/m

in

time (min)

on food

off food)

10 mm

start end

start end

1mm

1mm

small patch

large patch

0 0.5 1−4000

−2000

0

Normalized distance from edge

Der

ivat

ive

of m

ean

inte

nsity

7.4 mm (small)12 mm19 mm (large)23 mm

bacteria patch size

Pro

babi

lity

of lo

catio

n

edge center0

0.5

1

Large patchSmall patch

**

**

F G

edge center

sim

ulat

ed c

hang

e in

bac

teria

Distance from edge

Sm

all p

atch

Larg

e pa

tch

Figure 1. Worms Exploring Differently Sized Food Patches Experience Different Sensory Environments

(A) Cross-section of a bacterial patch expressing green fluorescent protein (GFP) shows higher concentrations on the edge compared to the center.

(B) The spatial derivative from the edge to the center of the patch shows a steep edge gradient and constant center across a range of patch sizes (n = 5 for each

size). Lines represent population mean, and shaded areas are SEM.

(C) Worms on food show a lower residence time on the edge of the patch when taken from small patches (small patch, n = 28; large patch, n = 32; t test **p < 0.01).

Bars are population means, and error bars are SEM.

(D) A Markov model was run to simulate worm movement across a bacterial patch (see also Supplemental Experimental Procedures). The observed bacterial

concentration was more variable for small patches (top right) compared to those on large patches (bottom right).

(legend continued on next page)

2 Neuron 86, 1–14, April 22, 2015 ª2015 Elsevier Inc.


(Gray et al., 2005). Since this search circuit is characterized, we

focused on whether environmental variability modulates local

search behavior and its underlying neural circuit. We found

that only a subset of the circuit defined above is involved in

evaluating variability. The smaller circuit, which consists of

4 sensory neurons and 12 interneurons, evaluates food vari-

ability and modifies local search behavior. Further, dopamine

signaling is required to encode food variability, influencing

both sensory neurons and interneurons. We used a combina-

tion of genetics, behavioral analysis, calcium imaging, and

theoretical methods to investigate how a neural circuit evalu-

ates environmental variability to determine contextually appro-

priate behavior.

RESULTS

Defining a Variable Sensory ExperienceC. elegans feeds on bacteria and is typically cultured in the

lab on plates with E. coli patches (Brenner, 1974). We hypoth-

esized that animals experience changing environments as they

traverse a bacterial patch. To gain insight into an animal’s sen-

sory experience, we analyzed the gradients of fluorescence

across a bacterial patch expressing green fluorescence pro-

tein (Labrousse et al., 2000) (Figure 1A). Using fluorescence in-

tensity as a proxy for bacterial concentrations, we found that

patches of different sizes have similarly steep gradients at

the edge and are flat in the center of the patch (Figures 1B,

S1A, and S1B). We then analyzed the behavior of animals

exploring patches of different sizes. We found that animals

on small patches (7.4 mm diameter) spend more time at the

edge than animals on large patches (19 mm) (Figures 1C,

S1C, and S1D).

We hypothesized that animals on small patches aremore likely

to be at the edge because the relative edge size (ratio of circum-

ference to surface area) of a lawn is greater for small patches

compared to large patches. We verified the effect quantitatively

with a simple Markov model, using empirically obtained data

(speed and size of animal) to simulate random movement on

food patches of different sizes. The location probabilities pre-

dicted by the simulation matched observed positions of animals

exploring patches (compare Figures 1C and S1E). Our simulation

also confirmed that animals exploring small patches are more

likely to encounter edges compared to those exploring large

patches. Consequently, animals exploring a small patch

encounter large changes in bacterial variability more frequently

than those exploring a large patch (Figures 1D and S1). These

data suggest that animals exploring differently sized food

patches experience differing levels of food variability. Small

patch exposure results in a more frequently changing food expe-

rience, while large patch exposure results in a more constant

experience.

Variable Sensory Experience Modifies Local SearchBehaviorTo test whether the differences in ‘‘on-food’’ sensory experience

modify subsequent ‘‘off-food’’ behavior, we analyzed the

response of animals that explored differently sized bacterial

patches in the food search assay (Figure 1E) (Gray et al., 2005;

Wakabayashi et al., 2004). We observed that local search was

modified by prior on-food experience. Specifically, the fre-

quency of reorientations an animal executes during local search

was directly proportional to the size of the patch from which it

was removed. For example, experiencing a small patch results

in about 25% fewer turns (Figures 1F, 1G, and S2A). This expe-

rience specifically modified the number of large reorientations in

locomotory paths caused by large-angled turns, but not the

number of non-reorienting reversals (Figures S2B–S2D). These

data further indicate that patch experience only affects behavior

relevant to local search, but not overall locomotion (Figure S2E).

Moreover, the patch size only influences local search behavior

(2–15 min off food) but not global search (20–30 min off food)

(Figures 1F–1G, Table S1). In particular, we found that animals

continued to exhibit patch-size-regulated change in local search

(plasticity) even on patches of dead bacteria (see Experimental

Procedures, Figure S2H, Table S1). The patch-size-dependent

plastic behavior was not specific to E. coli patches, as animals

also displayed similar behavioral responses after exploring

patches of Comamonas sp. or Pseudomonas fluorescens (Fig-

ure S2G, Table S1).

To test whether this patch-size-dependent behavior occurs in

the wild, we analyzed the behavior of wild isolates. We found that

CB4856 (Hawaiian), AB1 (Australian), and RC301 (German) iso-

lates all displayed patch-size-dependent search, suggesting

that this might be an ecologically relevant behavior (Figure S3A,

Table S1). We also found that RC301 animals showed signifi-

cantly more patch-size-dependent plasticity when compared

to other wild isolates (Figure S3A, Table S1). The data suggest

that various C. elegans strains use information about food expe-

rience to modify local search as a general strategy for exploring

their environment. Collectively, these results indicate that ani-

mals exploring different patch sizes experience different levels

of variability in their sensory environment and use that informa-

tion to modify local search behavior.

The On-Food Sensory Experience May Be Influenced byEating and Oxygen-Sensing BehaviorsDifferences in bacterial density between the edge and center of a

bacterial patch (Figures 1A and 1B) may modify off-food search

in several ways. To test the possibility that an animal consumes

different amounts of food at the edge and at the center, we

analyzed the behavior of eating-deficient mutants, eat-2 (McKay

et al., 2004), in our behavioral paradigm. We found that eat-2

mutants were unable to distinguish between small and large

(E) Animals were removed from small and large patches of bacteria, and their local search behavior was analyzed. Example trajectories of animals removed from

small and large lawns during the first 2–5 min of their local search (right).

(F) Animals removed from a large patch (red) execute more turns during local search (2–15 min) when compared to animals from a small patch (black,

Kolmogorov-Smirnov test, ***p < 10�4). Global search (20–28 min) is not affected by patch size (n R 62).

(G) Bars represent a population mean, and error bars and shaded areas are SEM. In this figure and Figures 7, S2, S4, and S7, search behavior is shown as

‘‘Reorientations/min.’’



patches, suggesting that eating behavior affects local search

(Figure S3B, Table S1). To directly assess eating behavior, we

measured the number of pharyngeal pumps that an animal

made on the edge as opposed to the center of a patch. An ani-

mal’s pumping rate is directly proportional to the amount of

food it consumes (Avery and You, 2012). We found that animals

pumped significantly more on the edge (average = 270.9 pumps/

min; SEM = 9.19) compared to those on the center (average =

203.2 pumps/min; SEM = 11.49). Thus, an animal is likely to

consumemore bacteria on the edge when compared to the cen-

ter, and this difference might affect local search.

We also tested whether differences in oxygen or carbon diox-

ide concentrations between the edge and the center of a bacte-

rial patch influence local search. Animals with mutations in

soluble guanylate cyclase gcy-35 are unable to sense oxygen

gradients (Gray et al., 2004). gcy-35 mutants were unable to

distinguish between small and large patches and performed

similar local search after removal from both patches (Figure S3C,

Table S1). This result suggests that oxygen sensing is also

required to generate patch-size-dependent local search. By

contrast, receptor guanylate cyclase gcy-9 mutants that cannot

sense carbon dioxide gradients (Hallem et al., 2011) can distin-

guish between small and large patches (Figure S3C, Table S1).

Collectively these results indicate that differences in bacterial

consumption and oxygen sensing contribute to the on-food

experience and alter local search behavior.

A Critical Period of Patch Experience Predicts LocalSearchWe hypothesized that animals use the sensory experience ac-

quired during patch exploration to modify their future local

search behavior. To identify features of patch sensory experi-

ence that best predict the number of reorientations executed

during local search, we monitored the on- and off-food behavior

of animals. We tracked the tip of an animal’s nose as it explored

the food environment and then later analyzed local search

behavior. We found that the frequency of movement between

the edge of a bacterial patch and the center, i.e., movement be-

tween regions of high and low bacterial concentrations, ac-

counts for a large fraction of the difference in local search

behavior (Figures 2A, 2C, and S4). We also tested other possible

predictors of behavior, including time spent off the food patch or

movements on the center of a patch, and found that none of

them could predict reorientations during local search (Figure S4).

These results suggest that animals may evaluate a patch by their

locomotory paths between the edge and center and use this in-

formation to modify local search.

Next, we attempted to identify a critical time period of the on-

food patch experience that modifies off-food local search. We

used maximum noise entropy (MNE) (Fitzgerald et al., 2011) to

extract a filter that identified on-food time points relevant to

modifying off-food search. In particular, the filter maximizes un-

certainty across trials in order to be consistent with the known

input/output relationships but makes no other assumptions

(see also Experimental Procedures). The filter reveals a critical

on-food period that predicts off-food local search behavior—

the period from 5 to 25 min preceding removal from food

accounts for a large degree of the difference in the number of re-

orientations each animal will make after removal from food (Fig-

ure 2A). By contrast, neither the immediate 5 min preceding

removal from food nor any time prior to 30 min before removal

predicts future behavior. Notably, using the position of the

nose gave a significantly better prediction than using the center

of mass of the body (data not shown). Other variables, such as

time spent off food and change in position while not on the

edge (the change in the set of all the positions excluding time

on edge), showed no relevance to behavior (Figures S4B–S4D).

Our model based on the filter above predicted that an animal

transitioning more frequently between the edge and the center

of the patch would execute fewer reorientations during local

search. Given that animals exploring small patches transition be-

tween the edge and center more often than animals on large

patches (Figure 1C), the model accounted for the observation

that animals removed from small patches reorient less than ani-

mals removed from large patches. Themodel also predicted that

individual animals on similarly sized food patches would reorient

a different number of times if they have different numbers of edge

encounters. We modeled the behavior of animals with a differing

number of edge encounters and found that those that transi-

tioned more frequently between the edge and the center of the

lawn (Figure 2Biii) made fewer turns after removal from food

during local search (Figure 2Biv). Thus, our model can broadly

account for differences in the number of turns performed by in-

dividual animals based on their on-food experience (Figure 2C).

We designed an additional assay to confirm that the variability

that an animal experiences on food affects its local search

behavior. We increased the variability of the patch, but not total

amount of bacteria in the patch, by adding bumps (see Experi-

mental Procedures, Figure 2D, Table S1). We found that when

animals explored small bumpy patches with increased vari-

ability, they executed 20% fewer turns compared to regular

small patches (Figure 2D, Table S1). Interestingly, increasing

the variability in a large patch did not strongly affect the local

search (Figure 2D, Table S1). These results show that animals

integrate the variability of their sensory environment 5–25 min

prior to food removal and then use that information to drive local

search behavior.

A Subset of Sensory Neurons, ASI and ASK AreSpecialized for Evaluating Patch VariabilityPrevious studies identified three pairs of sensory neurons (ASI,

ASK, and AWC) that drive local search behavior (Gray et al.,

2005;Wakabayashi et al., 2004). All of these neurons extend their

dendrites to the nose of the animal, where they detect environ-

mental changes and control behavior (Gray et al., 2005; White

et al., 1986) (Figure 3A). We examined the role of ASI, ASK,

and AWCneurons in learning the variability of the bacterial patch.

We compared the reorientations/min executed by wild-type

animals during local search when they are removed from small

or large patches. We found that animals removed from large

patches have a 20% increase in their reorientations/min

compared to those removed from small patches (Figure 3B,

Table S1). Interestingly, blocking neurotransmitter release by ex-

pressing tetanus toxin light chain fragment (TeTx) (Schiavo et al.,

1992) in ASI and ASK, but not AWC, sensory neurons disrupted

patch-size-dependent search. Moreover, AWC-ablated animals



alsomaintained their ability to distinguish between the two patch

sizes, confirming that the AWC sensory neurons are not required

for integrating patch size into local search behavior (Figure 3B,

Table S1).

Glutamate is the major excitatory neurotransmitter in the

C. elegans nervous system and has been shown to play a crucial

role in regulating local search behavior (Chalasani et al., 2007).

To test whether glutamate release from sensory neurons was

required to achieve plasticity in local search, we analyzed the

behavior of signaling mutants. EAT-4 is the C. elegans homolog

of the vesicular glutamate transporter, and its loss results in

severe defects in multiple behaviors including food search

Observed reorientations

Mod

el p

redi

ctio

n of

reo

rient

atio

nsB

C

Minutes prior to start of search

Con

trib

utio

n of

edg

e re

side

nce

time

disp

ersi

on (

a.u.

)

0

30

60

0

30

60

i

iii

ii

iv

Reo

rient

atio

nsR

eorie

ntat

ions

r=0.80, p<1x10-4

pred

icte

d

obse

rved

pred

icte

d

obse

rved

30 45 60 75

*30

45

60

75

start end

start end

A

.

*

.

D

% in

crea

se in

reo

rient

atio

ns

from

sm

all l

awns

n.s. ** *

Large Large &Bumpy

Small & Bumpy

40

0

-30

Larg

eLa

rge

&B

umpy

Sm

all &

B

umpy

Sm

all

5 mm

0-20-40-60

0

Figure 2. A Filter Predicts the Number of Turns during Local Search

(A) A filter representing the contribution of the dispersion in movement orthogonal to the edge predicts that this movement is important between 5 to 25 min prior

to removal from the patch for determining turns during local search. Lines represent mean of four jack-knives (see also Experimental Procedures), and shaded

area represents SEM.

(B) (i, iii) Example tracks of animals exploring similarly sized patches. The animal exploring the top patch (red *) generates more turns during local compared to the

animal exploring the bottom patch (red ,). (ii, iv) Comparing predictions from the filter to observed total number of turns.

(C)Model prediction provides a good fit of actual off-food search behavior for individual animals (r = 0.8, p < 10�4). Data from the example tracks are indicatedwith

a red * and ,.

(D) Animals were allowed to explore small and large patches as well as patches where denser food was placed in the center of the patch. Animals exploring small

patches with large bumps performed significantly fewer turns than small patches without bumps (n R 30). Bars represent population means, and error

bars represent SEM. * indicates significantly different population means compared to behavior of animals exploring small patches (Kolmogorov-Smirnov test,

p < 0.05). Significance was also corrected for a false discovery rate of 0.05.



(Chalasani et al., 2007; Lee et al., 1999). Restoring EAT-4 in indi-

vidual neurons did not rescue patch-size-driven plasticity but

restoring EAT-4 to both ASI and ASK neurons rescued the ability

to distinguish patch size (Figure 3C, Table S1). However, we

found that restoring function of the EAT-4 transporter to any

one of AWC, ASI, or ASK individually rescued the mutant’s

inability to transition from a local to a global search (Figure 3D).

Interestingly, we found that rescuing eat-4 in both ASI and ASK

neurons increased the number of reorientations throughout the

entire assay (Table S1), suggesting that glutamate release from

ASI and ASK influences the duration of local search at the

expense of transition to global search (Figure 3D).We also tested

various glutamate receptormutants and found that AMPA recep-

tor glr-1, glutamate-gated chloride channels glc-1, glc-2, glc-3,

and glc-4, and metabotropic receptors mgl-1, but not mgl-2,

were required for the patch-size-driven plasticity (Table S1).

Our mutant analyses indicate a complex role for glutamate

signaling in driving local search behavior. These results show

that glutamate release from any of the three neurons, ASI,

ASK, or AWC, can drive local search behavior but that release

from both ASI and ASK is necessary and sufficient for patch-

size-driven local search. Our results also highlight a key differ-

ence between the circuit that generates local search and the

circuit controlling its behavioral plasticity.

To further investigate how ASI and ASK neurons detect

variability of food stimuli, we used calcium imaging to analyze

sensory neuron activity in response to different concentra-

tions of bacterial stimuli in transgenic animals expressing

the GCaMP calcium sensor (Tian et al., 2009). We used a

custom-designed microfluidics device that allowed us to trap

animals, deliver precisely timed stimuli to the nose, and record

neural activity (Chalasani et al., 2007; Chronis et al., 2007). An-

imals exploring a food patch on a plate experience a 10-fold

change in bacterial stimuli when moving between the center

and edge of the bacterial patch (Figure S1B). Thus, for these

imaging experiments, we presented stimuli that represented

a similar fold change and recorded neuronal responses (see

Experimental Procedures). Consistent with previous results

(Chalasani et al., 2007), we found that AWC neurons responded

to the removal of food stimuli in a dose-dependent manner

(Figures 4A and S5A). In contrast, ASK sensory neurons re-

sponded to the removal of large, but not small, concentrations

of food stimuli (Figures 4B and S5B). ASI neurons responded

to the addition of large, but not small, changes in stimuli (Fig-

ures 4C and S5C). Next, we tested whether the neurons

encode absolute or relative changes in food stimuli. We found

that both ASI and ASK neurons responded to a 10-fold, but

not a 2-fold, change in food stimuli, suggesting that they

signal relatively large changes in stimuli (Figures 4D, 4E, S5D,

and S5E).

We have shown that animals experience large changes in

their food environment when they encounter the edge of a bac-

terial patch (Figure 1). To probe how ASI and ASK neural activity

could encode edge encounters, we analyzed their responses to

temporal changes in food stimuli. We found that ASK neurons

responded similarly to the removal after 1 or 5 min exposure

to bacterial stimuli (Figures 4F and S5F). Similarly, ASI re-

sponses to the addition of food were not affected by time inter-

val between two stimulations (Figures 4G and S5G). These

results indicate that ASI and ASK can reliably report stimuli

that are separated by at least a minute. Given the observed ac-

tivity patterns, we suggest that ASI and ASK neurons provide

information about the large fluctuations seen on the edge of a

food patch. AWC neurons, by contrast, provide signals that

do not influence patch-size-driven learning. Thus, ASI and

ASK neurons together detect variability in the food environment

A B

WT AWC- AWC::TeTx

ASI::TeTx

ASK::TeTx

eat-4

AWC ASI ASK ASI+ASK

% in

crea

se in

reo

rient

atio

ns

% in

crea

se in

reo

rient

atio

ns

C D

eat-4cDNA -

Food/odor

AS

K

AW

CA

SI

AnteriorPosterior

*** **

*%

incr

ease

in r

eorie

ntat

ions

dur

ing

loca

l vs

glob

al s

earc

h

- AWC ASI ASK ASI+ASK

-20

0

20

40

-20

0

20

40

-40

0

eat-4

********

-

WT

**

-

WT

20

40

60

80

100

eat-4cDNA

***

-20

Figure 3. Specialized Roles of Sensory

Neurons

(A) Schematic showing three pairs of amphid

sensory neurons (ASI, ASK, and AWC) that extend

their dendrites to the nose of the animal and drive

local search behavior.

(B) Blocking neurotransmission by expressing

tetanus toxin light chain fragment (TeTx) in ASI or

ASK, but not in AWC, is sufficient to block patch-

size-driven plasticity in local search (n R 30) as

illustrated by percent increase in reorientations.

(C) eat-4 animals are unable to distinguish be-

tween patch sizes. Simultaneous rescue of eat-4 in

ASI and ASK is sufficient to restore patch-size-

driven plasticity (n R 24).

(D) Rescue of eat-4 in any of the three sensory

neurons is sufficient to rescue local to global

search transition, but not for worms to distinguish

between patch sizes. All bars are population

means, and error bars represent SEM (Kolmo-

gorov-Smirnov test, *p < 0.05, **p < 0.01, ***p <

10�3). In Figures 3, 5, 7, S6, and S7, search data

are presented as percent increase in the number of

reorientations when animals are removed from

large compared with those removed from small

patches.



responding to large, but not small, changes in bacterial

concentrations.

Dopamine Acting via D1-like Receptors Modulates anInterneuron NetworkASI and ASK sensory neurons synapse onto a common set of

interneuron targets that includes AIY, AIA, AIB, and AIZ (White

et al., 1986) (Figure 5A). To test whether AIY, AIA, AIB, and

AIZ are involved in patch-size-dependent behavior, we blocked

neurotransmitter release from these neurons by expressing

tetanus toxin expression under cell-selective promoters

(Schiavo et al., 1992). We found that blocking neurotransmitter

release from any of the interneurons prevented patch-size-

dependent local search (Figure 5B, Table S1), suggesting that

the plasticity is a property of multiple interneurons and does

not occur at a single neuron or synapse.

To understand how the interneuron network is regulated, we

tested whether dopamine modifies behavioral plasticity in local

search. Dopamine has been shown to influence learning in

other model organisms (Dayan and Balleine, 2002; Hills

et al., 2004; Waddell, 2010; Wise, 2004). We found that

mutants in dopamine signaling do not show patch-size-

dependent plasticity (Figure 5C, Table S1). Specifically, we

found that worms with excessive dopamine, dat-1 dopamine

transporter mutants (Chase and Koelle, 2007), behaved simi-

larly to animals removed from small patches (Figure S6A),

while worms lacking dopamine, cat-2 tyrosine hydroxylase

mutants (Chase and Koelle, 2007), behaved similarly to ani-

mals removed from large patches, irrespective of prior food

experience. Given that animals lacking dopamine behave as

if they experienced a low variability, large patch environment

and perform a local search accordingly, we suggest that this

form of local search may be the default behavior. We further

hypothesized that increasing dopamine signaling would modify

local search behavior to reflect a small patch experience. To

test the hypothesis, we mapped the dopamine circuit and

0 10 20 30 40 50 60−100

0

100

200

300

400

500

0 10 20 30 40 50 60−100

0

100

200

300

400

0 10 20 30 40 50 60−100

0

100

200

300

400

0 10 20 30 40 50 60−100

0

100

200

300

400

AWC ASK ASI

ASK ASI

0 10 20 30 40 50 60−100

0

100

200

300

400

0 10 20 30 40 50 60−100

0

100

200

300

400

0 10 20 30 40 50 60−100

0

100

200

300

400

A B C

D E

F

5x107 cfu/ml5x108 cfu/ml5x109 cfu/ml

5x106 cfu/ml

1 minute5 minutes

5x108 to 2.5x108 cfu/ml 5x108 to 5x107 cfu/ml

time (s) time (s) time (s)

time (s) time (s)

time (s) time (s)

ΔF

/F0

(%)

ΔF

/F0

(%)

ΔF

/F0

(%)

ΔF

/F0

(%)

ΔF

/F0

(%)

-20

0

40

80

120

-20

0

40

80

120

-20

0

40

80

120

-20

0

40

80

120

-20

0

40

80

120

-20

0

40

80

120

-20

0

40

80

120

**

*

*

*

*

n.s.

*

*

n.s.

1 minute5 minutes

5x107 cfu/ml5x108 cfu/ml5x109 cfu/ml

5x106 cfu/ml 5x107 cfu/ml5x108 cfu/ml5x109 cfu/ml

5x106 cfu/ml

2.5x108 to 5x108 cfu/ml 5x107 to 5x108 cfu/ml

GASK ASI

Food Food Food

More Food

Food Food

LessFood

Less Food

More Food

Figure 4. Neural Responses to Varying Food

(A–C) AWC (A) and ASK (B) sensory neuron responses to removal and ASI (C) responses to the addition of bacterial stimuli.

(D and E) ASK (D) and ASI (E) respond to large, but not small, changes in bacterial stimuli.

(F) ASK shows similar response to removal of food stimulus after 1 or 5 min of food presentation.

(G) ASI neurons also respond similarly to the addition of food stimuli after its absence for 1 or 5min. Average data in each panel is shown using solid lines, while the

SEM is shown by the lightly colored region around each line. Bar graphs show the average change in fluorescence during the entire 50 s window after the addition

or removal of stimulus, shown for all conditions. Presence of stimulus is represented by gray boxes; * indicates significant difference between population means

(t test, p < 0.05, n R 10). Multiple comparisons were corrected for a false discovery rate of 0.05.



analyzed the effects of manipulating dopamine levels on local

search behavior.

In order to identify the source of dopamine that influences

patch-size-dependent behavioral plasticity, we performed

cell-specific rescue experiments in dopamine mutants. In the

C. elegans hermaphrodite, dopamine is released from CEP,

ADE, and PDE neurons (Chase and Koelle, 2007). We were

able to restore patch-size-dependent search in cat-2 mutants

by expressing the wild-type gene specifically in CEP, but not

ADE and PDE dopaminergic neurons (Figure 5C, Table S1).

CEP neurons are postsynaptic to the ASK sensory neurons

(White et al., 1986), which we have shown above to be part

of the variability detection module. However, CEP neurons

also directly detect bacteria and drive locomotory behaviors

(Sawin et al., 2000). Therefore, we tested whether the sensory

function of CEP neurons is required for patch size-dependent

plasticity. We found that cat-6 mutants, which lack CEP sen-

sory cilia (Perkins et al., 1986), were still able to learn to distin-

guish between small and large patches, which suggests that

CEP sensory functions are not required for the plasticity in local

search (Figure S6B). Together, these results suggest that CEP

neurons, which receive synaptic connections from ASK, are

sufficient to release the dopamine required for local search

plasticity.

Next, we investigated the receptors that detect CEP-released

dopamine. TheC. elegans genome encodes several homologs of

the mammalian D1- or D2-like receptors (Chase and Koelle,

2007). We found that mutants in two D1-like receptors, dop-1

and dop-4, and the D2-like receptor, dop-3, were defective in

patch-size-dependent search (Figures 5D, 5E, and S6C, Table

S1). However, mutants in the D2-like receptors dop-2 and

dop-6 were able to distinguish the two patch sizes (Figure S6C,

B CA

AIZ AIB

AIAAIY

ASI ASK

CEP

% in

crea

se in

reo

rient

atio

ns

% in

crea

se in

reo

rient

atio

ns

-20

0

20

40

-20

0

20

40

WT dop-1

AWC ASK ASI AIB/AIZ

AIYdop-1cDNA

- -

WT dop-4

AWCAIB/AIZ

AIB AIA AIYdop-4cDNA

- -

-20

0

20

40

-20

0

20

40

WT AIY::TeTx

AIA::TeTx

AIB/AIZ::TeTx

AIB::TeTx

WT dat-1cat-2cDNA

CEP

*** * **

** ** ** **

% in

crea

se in

reo

rient

atio

ns

D E

ADE/PDE

cat-2

- - -

% in

crea

se in

reo

rient

atio

ns

Figure 5. Interneuron Network for Learning Patch Size

(A) Schematic showing the proposed post-synaptic targets of the ASI and ASK sensory neurons required for patch-size-driven plasticity. CEP is a dopaminergic

neuron releasing dopamine (red circles).

(B) Expressing TeTx in any of the four interneurons blocks patch-size-driven plasticity (n R 34).

(C) Plasticity is also abolished in cat-2 and dat-1 mutants. cat-2 is specifically required in CEP neurons, but not in ADE or PDE neurons (n R 33).

(D) Mutants in the D1-like dopamine receptor, dop-1, are defective in patch-size-driven plasticity, and this receptor is specifically required in the ASI sensory

neurons (n R 30).

(E) dop-4 acts in AIB and AIZ, interneurons that are downstream of the ASI and ASK sensory neurons (nR 24). All bars represent populationmeans, and error bars

represent SEM; * indicates significantly different population means (Kolmogorov-Smirnov test, *p < 0.05, **p < 0.01, ***p < 10�3). Multiple comparisons were

corrected for a false discovery rate of 0.05.



Table S1). Previous studies suggest that DOP-1 and DOP-3 play

mutually antagonistic roles, often in the same neurons. More-

over, mutants in these genes often display a phenotype that is

restored when both genes are defective (Chase et al., 2004).

Consistent with these results, we found that dop-1;dop-3 double

mutants were normal in their patch-size-driven local search (Fig-

ure S6C, Table S1). Thus, D1-like dopamine signaling is required

for patch-size-dependent plasticity. Using cell-specific rescues,

we found that DOP-1 receptors in ASI sensory neurons were suf-

ficient to restore plasticity (Figure 5D, Table S1). Similarly,

rescuing DOP-4 receptors in both AIB and AIZ interneurons,

which are postsynaptic to ASI and ASK sensory neurons (White

et al., 1986), is sufficient to restore plasticity (Figures 5A and 5E,

Table S1). To test whether dopamine was acting specifically in

the circuit driving plasticity of local search and not on dopamine

1 m

M5

mM

10 m

M0

15

20

Reo

rient

atio

ns

Buffer

+Dopamine (DA)

AS

I res

pons

e to

food

ad

ditio

n Δ

F/F

0 (%

)

AS

K r

espo

nse

to fo

od

rem

oval

ΔF

/F0

(%)

+10 mM DANo DA

+10 mM DANo DA

-20

0

40

80

-20

0

40

80

B

*

*

Training plate Assay plate

-20

0

20

40

60

% r

eorie

ntat

ions

buf

fer

vs

10m

M D

A

***

A

DC

Training plate(on food)

Assay plate(off food)i

ii

n.s. **

High B

acte

ria

(5x1

08 cf

u/m

l)

Low B

acte

ria

(5x1

06 cf

u/m

l)

High B

acte

ria

(5x1

08 cf

u/m

l)

Low B

acte

ria

(5x1

06 cf

u/m

l)

n.s.

n.s.

Figure 6. Dopamine Levels Modify Plastic

Local Search and Neuronal Activity

(A) (i) Schematic showing the training and testing

phases of our learning assay. (ii) Addition of

dopamine changes in search strategy when added

during on-food experience, but not during search

(n R 30).

(B) Dose-dependent effect of dopamine added

during training suppresses turns during off-food

local search (n R 30). Significance is relative to

5 mM DA condition with multiple comparisons

corrected for a false discovery rate of 0.05. Buffer

is also significantly different from 5 mM DA and

10mMDA conditions. Bars in (A) and (B) represent

population means, and error bars represent SEM.

(C and D) Bar graphs showing calcium responses

of ASI and ASK neurons (n R 10) stimulated with

changes in food and averaged over 30 s after

stimulus change. Exogenous dopamine modifies

neuronal activity of ASI neurons (C), but not ASK

neurons (D). Bars represent population means,

and error bars represent SEM (Kolmogorov-Smir-

nov test, *p < 0.05, ***p < 10�3). Multiple compar-

isons were corrected for a false discovery rate

of 0.05.

receptors in other circuits, we examined

the dop-1, dop-2, dop-3, dop-4 (dop#)

quadruple mutant (Gaglia and Kenyon,

2009). The quadruple mutant was unable

to discriminate patch size, but local

search plasticity was restored when

DOP-4 receptor function was rescued in

AIB and AIZ interneurons alone (Fig-

ure S6D, Table S1). We suggest that while

dopamine signaling has widespread ef-

fects onC. elegans neural circuitry (Chase

and Koelle, 2007), dopamine receptors

are only needed on the ASI, AIB, and

AIZ neurons in the patch-size-dependent

behavior circuit. Collectively, our results

suggest that CEP neurons release dopa-

mine that can be sensed by D1-like re-

ceptors on ASI sensory neurons and the

downstream AIB and AIZ interneurons to regulate local search

plasticity.

Manipulating Dopamine Levels Modifies Plasticity andNeuronal ActivityTo further confirm a role for dopamine, we exogenously manipu-

lated its levels and analyzed its effects on patch-size-dependent

search behavior. We found that adding dopamine to the training

plate (see Experimental Procedures) during learning was suffi-

cient to modify subsequent local search behavior (Figure 6A,

TableS1). Bycontrast, addingdopamine to the local searchassay

test plate had no effect on this behavior (Figure 6A, Table S1).

Moreover, the dopamine effect was dose dependent, suggesting

that the dopamine present during on-food exploration plays a

crucial role in modulating local search behavior (Figure 6B). Our



data show that increased dopamine suppresses the number of

turns during local search and suggests that dopamine levels are

instructive to the plasticity of local search behavior.

Next, using calcium imaging we tested whether dopamine

directly modifies the activity of target ASI neurons. We found

that ASI responses to the addition of large concentrations of

food were greatly increased in the presence of dopamine (Fig-

ure 6C). We also found that dopamine reduced ASI basal activity

upon stimulation with low food concentrations (Figure 6C), sug-

gesting that dopamine amplifies ASI responses to food. Consis-

tent with our genetic analysis of dopamine receptor mutants, we

found that dopamine did not modify ASK responses (Figure 6D),

indicating that dopamine acts on ASI, but not ASK neurons.

CREB Influences Acquisition TimeWe next tested whether patch-size-dependent search plasticity

requires protein synthesis and CREB signaling. Cycloheximide

has been used in C. elegans to block protein synthesis in a

dose-dependent manner (Kauffman et al., 2010; Szewczyk

et al., 2002). We designed a new behavioral assay in which we

transferred animals exploring a large patch to a small patch for

a 1 hr time period and then analyzed their local search after

removal from the small patch (Figure 7Ai). We predicted that

animals would learn the small patch size and switch their local

search behavior to reflect removal from a small lawn. Consistent

with our prediction, we found that wild-type animals switched

their behavior to small-patch-driven behavior (Figure 7Aii, Table

S1). We observed that blocking protein synthesis with cyclohex-

imide during small-patch exploration prevented learning the new

patch size; drug-treated animals behaved as if they had been

removed from the original large patch (Figure 7Aii, Table S1).

Moreover, cycloheximide did not have a general effect on

behavior. Drug-treated animals moved between similarly sized

patches did not exhibit changes in reorientations (Figure S7B,

Table S1). These data confirm that the animals require protein

synthesis during patch exploration to learn a new patch size.

CREB signaling is required for long-term memory in a number

of organisms including C. elegans (Kauffman et al., 2010; Silva

et al., 1998). Not surprisingly, we found that crh-1 (C. elegans

CREB homolog) mutants are unable to learn in our paradigm.

This defect was rescued when CREB function was specifically

restored to AIB and AIZ interneurons (Figure 7B, Table S1), sug-

gesting that CREB functions in AIB and AIZ interneurons to regu-

late learning of patch size. We hypothesized that changes in

CREB protein levels in interneurons might also influence local

search behavior. To test the hypothesis, we analyzed the

behavior of wild-type animals overexpressing crh-1 (CREB pro-

tein) in AIB and AIZ interneurons. We found that these transgenic

animals acted similarly to wild-type in their ability to learn patch

size and perform local search (Table S1). We modified the trans-

fer assay described above to test whether changing the level of

CREB protein in these AIB and AIZ interneurons would alter the

rate of learning patch size. In this modified assay, animals were

housed on large patches overnight and then transferred to small

patches for differing periods before we assessed their off-food

search behavior (Figure 7Ci). Consistent with the prediction of

the sensory filter (Figures 2A and 2C), we found that wild-type

animals took 30 min to learn the new environment and to switch

their behavior (Figures 7C and S7D, Table S1). We also observed

a correlation between the amount of CREB (increases in trans-

gene leads to increases in protein expressed; Mello et al.,

1991) and the time required to learn the small patch and switch

AB transfer off-food

i

ii

WT crh-1

AIB/AIZ

AIB AIY AIA

0

40

20

% in

crea

se in

reo

rient

atio

ns

crh-1cDNA

- -Large patch Small patch

0

3

1.5

0.1

mM

1 m

M*

*

Reo

rient

atio

ns/m

in

C+CHX

+CHX

Large

Small Assay

Large Assay

+ CHX- CHX

i

ii

****

WT

10 n

g/µl

25 n

g/µl

50 n

g/µl

crh-1

0

30

60

Max

imum

tim

e to

lear

nne

w p

atch

siz

e (m

inut

es)

AIB/AIZ::crh-1(OE)

Figure 7. Molecular Mechanisms of Learning

(A) (i) Schematic of transfer assay showing animals moved from large to small patches with or without cycloheximide (CHX). (ii) Animals learned the distribution of

food on the small patch and switched their behavior, except when protein synthesis was inhibited by CHX; concentrations are indicated in each bar (n R 28).

(B) Learning requires crh-1 in the AIB and AIZ interneurons (n R 27).

(C) (i) Schematic of transfer assay indicates that (ii) animals overexpressing crh-1 in AIB and AIZ neurons learn more quickly than wild-type animals. Graph shows

length of time an animal must spend on a new, small patch before turning rate after removal is no longer statistically different from animals that explored small

patches overnight (Figure S7D). Learning rate is dependent on the amount of crh-1 transgene expressed in the interneurons. Animals lacking crh-1 do not show a

difference in turning rate between small and large patches. All bars represent population means, error bars represent SEM, and * indicates significance (Kol-

mogorov-Smirnov test, p < 0.05). Multiple comparisons corrected for using a false discovery rate of 0.05.



behavior (Figures 7C and S7D, Table S1). Next, we tested

whether CREB acted downstream of the sensory neurons to

learn patch size by blocking neurotransmission from ASK. We

found that loss of ASK transmission slowed the rate of learning

compared to animals overexpressing crh-1 (Figure S7E, Table

S1), suggesting that this faster rate of learning also requires sen-

sory activity. Thus, we suggest that the amount of CREB protein

in AIB and AIZ interneurons regulates the time required to ac-

quire new information, providing a potential role for how CREB

may influence learning.

DISCUSSION

Our results show that C. elegans estimate environmental vari-

ability and use that information to alter local search accordingly.

These results are consistent with previous reports showing that

animals modify their locomotory patterns and egg laying based

on prior exposure to different foods (Shtonda and Avery, 2006;

Waggoner et al., 1998). Given our observation that local search

strategy used after removal from large patches (low variability

environments) is the default search behavior, we suggest that

C. elegans assume consistency in their food environment. This

likely reflects the statistics of signals in the natural environment

where low temporal and spatial frequencies dominate (Ruder-

man and Bialek, 1994; Simoncelli and Olshausen, 2001).

We found that when animals explore bacterial patches, ASI

and ASK neurons evaluate environmental variability. Encoding

environmental variability requires two signals: one representing

decrements in bacterial concentration, which is signaled

through ASK, and another representing increases in bacteria,

signaled through ASI. Our results suggest that low activity

in ASI and ASK neurons keeps this circuit in a default

configuration (typical large patch, low variability experience)

(Figure 8A). However, when ASK and ASI sensory neurons

receive inputs that indicate a high variability environment

(typical small patch experience), their activity is greatly

enhanced. We suggest that increased sensory activity

leads to increased dopamine release from the CEP neurons,

which are post-synaptic to ASK neurons. The increased dopa-

mine in turn acts on D1-like receptors on ASI sensory neurons

and the downstream AIB and AIZ interneurons in order to

modify search behavior in the new food-free environment

(Figure 8B).

Behavioral Model Predicts Future Local SearchBehaviorOur behavioral filter enabled us to predict certain characteristics

of future actions (frequency of reorientations) based on prior

experience. This was surprising, as previous studies have shown

that worm search behavior is stochastic (Gray et al., 2005;

Pierce-Shimomura et al., 1999). Previous attempts at behavioral

prediction in other animals have focused on estimating the prob-

ability of an animal performing a particular behavior (Coen et al.,

2014; Tai et al., 2012). By contrast, we were able to predict spe-

cific behaviors, such as the number of turns that an animal would

execute during a future local search, based on the history of their

prior experiences. Additionally, our prediction is for a behavior

that lasts many minutes after the initial stimulus (Figures 1F

and 2A–2C). Moreover, we show that dimensionality reduction

techniques designed for predicting properties of individual neu-

rons (Fitzgerald et al., 2011) can also be used to successfully pre-

dict whole animal behavior.

Dopamine and CREB Signaling Interpret SensoryInformationOur studies show that ASK and ASI sensory neurons detect vari-

ance in the food environment and modify local search behavior.

Combining these data with our behavioral filter, we suggest that

the variance detected between 5min and 25min prior to removal

from food is critical to generate local search. We also show that

dopamine released from the CEP dopaminergic neurons plays a

crucial role in modifying local search behavior. We suggest that

the sensory-neuron-detected envrionmental variability is stored

as the amount of dopamine in the circuit. Further, we speculate

that dopamine levels are integrated over the 25 min time interval

and interpreted by downstream neuronal circuitry to modify local

search behavior. Interestingly, dopamine levels are also inte-

grated over multiple timescales ranging from seconds to tens

of minutes to generate behaviors in vertebrate circuits (Schultz,

2007a, 2007b). Our results are consistent with those observed

in vertebrate models in which dopamine plays a crucial role in

motivation, reward, and risk-associated behaviors (Schultz,

BA Figure 8. A Sub-Circuit Nested within a

Larger Search Circuit Encodes Sensory

Variability to Generate Contextually Appro-

priate Local Search

(A and B) Schematic showing the learning circuit

when animals explore large (A) or small (B)

patches. On large patches, ASI and ASK neurons

are not active, and default behavior with more

turns is observed. On small patches, ASI and ASK

neurons respond to large fluctuation in bacterial

concentrations (seen at the edges), leading to

dopamine release from CEP neurons and fewer

turns. The dopamine is sensed by D1-like re-

ceptors on ASI and their post-synaptic partners

AIB and AIZ neurons. CREB acts parallel to

dopamine signaling in the same AIB and AIZ in-

terneurons. Arrows represent direction of infor-

mation flow from CEP neurons.



2002; Schultz et al., 1997). In particular, increased dopamine has

been associated with unpredictable environments and an in-

crease in exploration, while reduced dopamine favors energy

conservation and less exploration (Beeler, 2012; Beeler et al.,

2012). These results confirm a conserved role for dopamine

signaling in regulating risk-associated behaviors from worms to

mammals.

Our results also suggest a novel role for CREB signaling, a

pathway that has previously been shown to play a crucial

role in regulating learning and memory (Frank and Greenberg,

1994; Silva et al., 1998). Different aspects of learning, such

as the value to be learned, learning rate, and timescale

of learning, have been shown to be under the control of

distinct neuromodulatory systems (Doya, 2002). We speculate

that learning during local search might also have indepen-

dent aspects, including acquisition value (information about

food variability) and acquisition rate (time needed to acquire

information about food variability. Dopamine signaling may

represent acquisition value, which is indicated as number

of turns during local search. While the amount of CREB pro-

tein influences the acquisition rate (time). We are unable to

find a direct link between dopamine and CREB signaling in

the local search circuit but find that exogenous dopamine

can modify the behavior of crh-1 mutants (Figure S7C). Taken

together, our results indicate that CREB and dopamine might

act in two parallel pathways to represent information in the

circuit.

Two Circuit Modules Estimate VariabilityWe suggest that two circuit modules within the local search

network combine to estimate variability over many minutes.

The first module consists of identified sensory neurons that

only respond to large changes in the sensory environment,

similar to ON and OFF cells in the retina (Wassle, 2004). Just

as ON and OFF cells act as edge detectors responding to large

increases or decreases in light intensity (Schiller et al., 1986),

ASI and ASK respond to large increases and decreases in bac-

terial concentration. These results suggest that diverse sensory

circuits use parallel ON and OFF pathways to evaluate vari-

ability as a general mechanism. The second module includes

interneurons, which are reminiscent of networks proposed for

reinforcement learning of average value (Schultz, 1997). This

is typically proposed as an error signal sent through dopamine

(Doya, 2002; Schultz, 1997; Schultz et al., 1997), similar to

the response of ASK, which could signal through CEP. This

suggests that the same circuit module may be used to learn

different types of value, such as average reward or reward

variability.

We propose that combining circuit modules may be

an evolutionarily efficient method for producing desired

behaviors. If the structure of neural circuits has evolved to

optimally extract relevant information, the local search circuits

described here and its computations are likely to be repre-

sented in other nervous systems. Further studies of nested

circuits should generate a better understanding of how neural

circuits estimate and use uncertain information to drive plastic

behaviors, crucial components of risk strategy and decision

making.

EXPERIMENTAL PROCEDURES

Please see Supplemental Experimental Procedures for details on strains,

molecular biology, calcium imaging, and hidden Markov model.

Imaging Bacteria Patches

Bacteria expressing GFP (Labrousse et al., 2000) were plated identically to

bacteria used for learning assays. Five plates, each containing patches of

10 ml, 50 ml, 100 ml, and 200 ml bacteria, were imaged on a Zeiss Stereo micro-

scope using a Zeiss MRM camera. As the highest fluorescence was at the

edge of the patch, peaks were fit to an ellipse in order to extract fluorescence

profiles and the bacterial center (Figure 1A). For each individual patch, a fluo-

rescence intensity profile was extracted every 36� and averaged. Normalized

profiles were found by dividing the distance from the edge by the radius of the

patch, so that the distance at the edge was 0 and at the center was 1 (Fig-

ure 1B). Profiles are well fit to a 1/R distribution from the edge to the center

of the patch. We designated the edge of the patch as the 20% of the patch

closest to where the bacterial lawn ends on a small patch and the remaining

80% the center. The edge of larger patches was chosen to represent the

same absolute distance from the transition between bacteria and no bacteria.

Behavioral Assay

Specific volumes (10 ml [small, 7.4 mm diameter], 50 ml, 100 ml [large, 19 mm

diameter], or 200 ml) of a bacterial culture (OD600 = 0.4) were seeded on

NGM agar plates (3g NaCl, 17 g agar, 2.5 g peptone, 1 ml of 1M CaCl2, 1 ml

of 5 mg/ml cholesterol, 1 ml of 1 M MgCl2, and 25 ml of 1 M KH2PO4 buffer

in 1 l of water) to obtain food patches of varying sizes. A dilution series was

used to estimate the number of colony-forming units in a bacterial culture

with OD600 = 0.4 and found it to be 5.18 3 108 cfu/ml. To characterize off-

food search behavior after exposure to different patch sizes, five to six L4

stage animals were first allowed to explore either small or large patches over-

night. Then, for off-food testing, worms were removed from these training

plates to assay plates lacking food, where they were corralled by a filter paper

soaked in 200 mM Cu(II)SO4 solution. Local search behavior on the assay

plates was recorded for 30 min using a Pixelink camera and analyzed by

custom software. Data presented were collected from at least six plates tested

on at least three different days.

Learning assays were performed similarly, except after overnight explora-

tion on large patches, animals were transferred to another plate containing

either a small or large patch of bacteria for 15, 30, 45, and 60 min (Figures

7C and S7D). For assays utilizing exogenous chemicals, 10 mM dopamine

or cycloheximide (0.1 mM–1 mM) was spread on agar plates and allowed to

absorb into NGM agar plates for 90 min. Either 10 or 100 ml of bacterial culture

was then seeded on these treated plates. The plates were allowed to dry for

60 min before animals were allowed to explore the patch for 1 hr. To obtain

dead bacteria patches, bacteria were killed in a water bath at 65�C for 3 hr

before plating. Diluted bacterial patches were obtained by diluting live or

dead bacteria cultures 10-fold in culture media. We obtained ‘‘bumpy’’ bacte-

rial patches by initially plating 1/10 of the volume of the culture onto a plate and

then adding the rest of the 9/10 onto that dried patch.

Prediction and Maximum Noise Entropy

In order to identify the on-food variables driving patch-size-dependent search,

we performed behavior assays as described above and analyzed resulting

data with a behavioral filter. Filters were extracted via the method of maximum

noise entropy (MNE) (Fitzgerald et al., 2011). Other methods such as reverse

correlation find the input that best correlates with the output. By contrast,

MNE finds the filter that maximizes uncertainty across trials in order to be

consistent with the known input-output relationships but makes no other as-

sumptions. Here, the parameters were binned into 3 min time periods to find

the time bins that best predict the behavioral output—the number of turns

upon removal from food. The prediction for the number of turns was obtained

by fitting a MNE feature as C½1+ expða+ f!, s!Þ��1, where C and a are con-

stants, the vector f!

represents time points forming the MNE feature, and vec-

tor s! represents a particular time profile describing the worm location at the

edge of the patch. The model makes different predictions for the number of

turns depending on the history of sensory experience described by s!. MNE



filters were extracted using four subsets of the data (jack-knives), which were

then averaged. Final turning was predicted using the stimulus energy from the

MNE filter, and a subset of the data was then fitted with nonlinearity in accor-

dance with the linear-nonlinear model (Fitzgerald et al., 2011).

SUPPLEMENTAL INFORMATION

Supplemental Information includes Supplemental Experimental Procedures,

seven figures, and two tables and can be found with this article online at

http://dx.doi.org/10.1016/j.neuron.2015.03.026.

AUTHOR CONTRIBUTIONS

A.J.C. conceived and conducted the experiments, interpreted the data, and

co-wrote the paper. A.T. generated strains and conducted behavioral assays.

N.P. developed the software for tracking animals. J.A.J.F. took high-resolution

images of bacteria patches. T.O.S. interpreted the data, provided guidance for

generating the dimensionality reduction filter, and co-wrote the paper. S.H.C.

conceived the experiments, interpreted the data, and co-wrote the paper.

ACKNOWLEDGMENTS

We thank the Caenorhabditis Genetics Center (CGC), the National Brain

Research Project (NBRP, Japan), C. Bargmann, and R. Komuniecki for strains

and constructs; O. Hobert and N. Flames for cell-specific promoters; C. Ste-

vens for helpful discussions; and J. Simon for illustrations. We are also grateful

to L. Hale, B. Tong, C. Yeh, S. Leinwand, K. Quach, and H. Lau for helpful com-

ments on the manuscript and other members of the Chalasani laboratory for

critical help, advice, and insights. This work was funded by grants from The

G. Harold and Leila Y. Mathers Foundation to C. Bargmann and by The Rita

Allen Foundation and an NIH R01MH096881-03 to S.H.C. A.J.C was initially

supported by a graduate research fellowship from the NSF and later by a

training grant from NIMH. J.A.J.F. gratefully acknowledges financial support

from the Waitt Foundation, NCI P30 CA014195-40, and NINDS P30

NS072031-03. T.O.S. was supported by NIH Grant R01EY019493 and

P30EY019005 and National Science Foundation (NSF) CAREER Award

1254123.

Received: July 15, 2014

Revised: January 18, 2015

Accepted: February 20, 2015

Published: April 9, 2015

REFERENCES

Avery, L., and You, Y.J. (2012). C. elegans feeding. WormBook, 1–23.

Beeler, J.A. (2012). Thorndike’s Law 2.0: Dopamine and the Regulation of

Thrift. Front Neurosci 6, 116.

Beeler, J.A., Frazier, C.R., and Zhuang, X. (2012). Dopaminergic enhancement

of local food-seeking is under global homeostatic control. Eur. J. Neurosci. 35,

146–159.

Brenner, S. (1974). The genetics of Caenorhabditis elegans. Genetics 77,

71–94.

Chalasani, S.H., Chronis, N., Tsunozaki, M., Gray, J.M., Ramot, D., Goodman,

M.B., and Bargmann, C.I. (2007). Dissecting a circuit for olfactory behaviour in

Caenorhabditis elegans. Nature 450, 63–70.

Chalfie, M., Sulston, J.E., White, J.G., Southgate, E., Thomson, J.N., and

Brenner, S. (1985). The neural circuit for touch sensitivity in Caenorhabditis

elegans. J. Neurosci. 5, 956–964.

Chase, D.L., and Koelle, M.R. (2007). Biogenic amine neurotransmitters in C.

elegans. WormBook, 1–15.

Chase, D.L., Pepper, J.S., and Koelle, M.R. (2004). Mechanism of extrasynap-

tic dopamine signaling in Caenorhabditis elegans. Nat. Neurosci. 7, 1096–

1103.

Chronis, N., Zimmer, M., and Bargmann, C.I. (2007). Microfluidics for in vivo

imaging of neuronal and behavioral activity in Caenorhabditis elegans. Nat.

Methods 4, 727–731.

Coen, P., Clemens, J., Weinstein, A.J., Pacheco, D.A., Deng, Y., and Murthy,

M. (2014). Dynamic sensory cues shape song structure in Drosophila. Nature

507, 233–237.

Dayan, P., and Balleine, B.W. (2002). Reward, motivation, and reinforcement

learning. Neuron 36, 285–298.

de Bono, M., and Maricq, A.V. (2005). Neuronal substrates of complex behav-

iors in C. elegans. Annu. Rev. Neurosci. 28, 451–501.

DeWeese, M., and Zador, A. (1998). Asymmetric Dynamics in Optimal

Variance Adaptation. Neural Computation 10, 1179–1202.

Doya, K. (2002). Metalearning and neuromodulation. Neural Netw. 15,

495–506.

Escobar, C., Martınez-Merlos, M.T., Angeles-Castellanos, M., del Carmen

Minana, M., and Buijs, R.M. (2007). Unpredictable feeding schedules unmask

a system for daily resetting of behavioural and metabolic food entrainment.

Eur. J. Neurosci. 26, 2804–2814.

Fairhall, A.L., Lewen, G.D., Bialek, W., and de Ruyter Van Steveninck, R.R.

(2001). Efficiency and ambiguity in an adaptive neural code. Nature 412,

787–792.

Fitzgerald, J.D., Rowekamp, R.J., Sincich, L.C., and Sharpee, T.O. (2011).

Second order dimensionality reduction using minimum and maximum mutual

information models. PLoS Comput. Biol. 7, e1002249.

Frank, D.A., andGreenberg, M.E. (1994). CREB: amediator of long-termmem-

ory from mollusks to mammals. Cell 79, 5–8.

Gaglia, M.M., and Kenyon, C. (2009). Stimulation of movement in a quiescent,

hibernation-like form of Caenorhabditis elegans by dopamine signaling.

J. Neurosci. 29, 7302–7314.

Gray, J.M., Karow, D.S., Lu, H., Chang, A.J., Chang, J.S., Ellis, R.E., Marletta,

M.A., and Bargmann, C.I. (2004). Oxygen sensation and social feeding medi-

ated by a C. elegans guanylate cyclase homologue. Nature 430, 317–322.

Gray, J.M., Hill, J.J., and Bargmann, C.I. (2005). A circuit for navigation in

Caenorhabditis elegans. Proc. Natl. Acad. Sci. USA 102, 3184–3191.

Hallem, E.A., Spencer, W.C., McWhirter, R.D., Zeller, G., Henz, S.R., Ratsch,

G., Miller, D.M., 3rd, Horvitz, H.R., Sternberg, P.W., and Ringstad, N. (2011).

Receptor-type guanylate cyclase is required for carbon dioxide sensation by

Caenorhabditis elegans. Proc. Natl. Acad. Sci. USA 108, 254–259.

Hills, T., Brockie, P.J., and Maricq, A.V. (2004). Dopamine and glutamate con-

trol area-restricted search behavior in Caenorhabditis elegans. J. Neurosci.

24, 1217–1225.

Kauffman, A.L., Ashraf, J.M., Corces-Zimmerman, M.R., Landis, J.N., and

Murphy, C.T. (2010). Insulin signaling and dietary restriction differentially influ-

ence the decline of learning and memory with age. PLoS Biol. 8, e1000372.

Labrousse, A., Chauvet, S., Couillault, C., Kurz, C.L., and Ewbank, J.J. (2000).

Caenorhabditis elegans is amodel host for Salmonella typhimurium. Curr. Biol.

10, 1543–1545.

Lanan,M. (2014). Spatiotemporal resource distribution and foraging strategies

of ants (Hymenoptera: Formicidae). Myrmecol News 20, 53–70.

Lee, R.Y., Sawin, E.R., Chalfie, M., Horvitz, H.R., and Avery, L. (1999). EAT-4, a

homolog of a mammalian sodium-dependent inorganic phosphate cotrans-

porter, is necessary for glutamatergic neurotransmission in caenorhabditis

elegans. J. Neurosci. 19, 159–167.

MacLean, E.L., Mandalaywala, T.M., and Brannon, E.M. (2012). Variance-sen-

sitive choice in lemurs: constancy trumps quantity. Anim. Cogn. 15, 15–25.

McKay, J.P., Raizen, D.M., Gottschalk, A., Schafer, W.R., and Avery, L. (2004).

eat-2 and eat-18 are required for nicotinic neurotransmission in the

Caenorhabditis elegans pharynx. Genetics 166, 161–169.

Mello, C.C., Kramer, J.M., Stinchcomb, D., and Ambros, V. (1991). Efficient

gene transfer in C.elegans: extrachromosomal maintenance and integration

of transforming sequences. EMBO J. 10, 3959–3970.




Merkle, T., and Wehner, R. (2008). Landmark guidance and vector navigation

in outbound desert ants. J. Exp. Biol. 211, 3370–3377.

Monosov, I.E., andHikosaka, O. (2013). Selective and graded coding of reward

uncertainty by neurons in the primate anterodorsal septal region. Nat.

Neurosci. 16, 756–762.

Ozuysal, Y., and Baccus, S.A. (2012). Linking the computational structure of

variance adaptation to biophysical mechanisms. Neuron 73, 1002–1015.

Perkins, L.A., Hedgecock, E.M., Thomson, J.N., and Culotti, J.G. (1986).

Mutant sensory cilia in the nematode Caenorhabditis elegans. Dev. Biol.

117, 456–487.

Pierce-Shimomura, J.T., Morse, T.M., and Lockery, S.R. (1999). The funda-

mental role of pirouettes in Caenorhabditis elegans chemotaxis. J. Neurosci.

19, 9557–9569.

Platt, M.L., and Huettel, S.A. (2008). Risky business: the neuroeconomics of

decision making under uncertainty. Nat. Neurosci. 11, 398–403.

Ruderman, D.L., and Bialek, W. (1994). Statistics of natural images: Scaling in

the woods. Phys. Rev. Lett. 73, 814–817.

Sawin, E.R., Ranganathan, R., and Horvitz, H.R. (2000). C. elegans locomotory

rate is modulated by the environment through a dopaminergic pathway and by

experience through a serotonergic pathway. Neuron 26, 619–631.

Schiavo, G., Benfenati, F., Poulain, B., Rossetto, O., Polverino de Laureto, P.,

DasGupta, B.R., and Montecucco, C. (1992). Tetanus and botulinum-B neuro-

toxins block neurotransmitter release by proteolytic cleavage of synaptobre-

vin. Nature 359, 832–835.

Schiller, P.H., Sandell, J.H., andMaunsell, J.H. (1986). Functions of theON and

OFF channels of the visual system. Nature 322, 824–825.

Schultz, W. (1997). Dopamine neurons and their role in reward mechanisms.

Curr. Opin. Neurobiol. 7, 191–197.

Schultz, W. (2002). Getting formal with dopamine and reward. Neuron 36,

241–263.

Schultz, W. (2007a). Behavioral dopamine signals. Trends Neurosci. 30,

203–210.

Schultz, W. (2007b). Multiple dopamine functions at different time courses.

Annu. Rev. Neurosci. 30, 259–288.

Schultz, W., Dayan, P., and Montague, P.R. (1997). A neural substrate of pre-

diction and reward. Science 275, 1593–1599.

Shtonda, B.B., and Avery, L. (2006). Dietary choice behavior in Caenorhabditis

elegans. J. Exp. Biol. 209, 89–102.

Silva, A.J., Kogan, J.H., Frankland, P.W., and Kida, S. (1998). CREB and mem-

ory. Annu. Rev. Neurosci. 21, 127–148.

Simoncelli, E.P., and Olshausen, B.A. (2001). Natural image statistics and neu-

ral representation. Annu. Rev. Neurosci. 24, 1193–1216.

Szewczyk, N.J., Peterson, B.K., and Jacobson, L.A. (2002). Activation of Ras

and the mitogen-activated protein kinase pathway promotes protein degrada-

tion in muscle cells of Caenorhabditis elegans. Mol. Cell. Biol. 22, 4181–4188.

Tai, L.H., Lee, A.M., Benavidez, N., Bonci, A., and Wilbrecht, L. (2012).

Transient stimulation of distinct subpopulations of striatal neurons mimics

changes in action value. Nat. Neurosci. 15, 1281–1289.

Tian, L., Hires, S.A., Mao, T., Huber, D., Chiappe, M.E., Chalasani, S.H.,

Petreanu, L., Akerboom, J., McKinney, S.A., Schreiter, E.R., et al. (2009).

Imaging neural activity in worms, flies andmicewith improvedGCaMP calcium

indicators. Nat. Methods 6, 875–881.

Tishby, N., and Polani, D. (2011). Information theory of decisions and actions.

In Perception-Action Cycle (Springer), pp. 601–636.

Waddell, S. (2010). Dopamine reveals neural circuit mechanisms of fly mem-

ory. Trends Neurosci. 33, 457–464.

Waggoner, L.E., Zhou, G.T., Schafer, R.W., and Schafer, W.R. (1998). Control

of alternative behavioral states by serotonin in Caenorhabditis elegans.

Neuron 21, 203–214.

Wakabayashi, T., Kitagawa, I., and Shingai, R. (2004). Neurons regulating the

duration of forward locomotion in Caenorhabditis elegans. Neurosci. Res. 50,

103–111.

Wark, B., Fairhall, A., and Rieke, F. (2009). Timescales of inference in visual

adaptation. Neuron 61, 750–761.

Wassle, H. (2004). Parallel processing in the mammalian retina. Nat. Rev.

Neurosci. 5, 747–757.

Watson, K.K. (2008). Evolution, risk, and neural representation. Ann. N Y Acad.

Sci. 1128, 8–12.

Wehner, R., Boyer, M., Loertscher, F., Sommer, S., and Menzi, U. (2006). Ant

navigation: one-way routes rather than maps. Curr. Biol. 16, 75–79.

White, J.G., Southgate, E., Thomson, J.N., and Brenner, S. (1986). The struc-

ture of the nervous system of the nematode Caenorhabditis elegans. Philos.

Trans. R. Soc. Lond. B Biol. Sci. 314, 1–340.

Wise, R.A. (2004). Dopamine, learning and motivation. Nat. Rev. Neurosci. 5,

483–494.



neural mechanisms for evaluating environmental variability ... 2015 - neural mechanisms for... ·...

Documents