Task used to study instrumental learning. A: layout of the behavioral devices. There were 2 response ports with photobeams to detect entries into the response port. The response ports flanked a central fluid dispenser that delivered 20% sucrose solution into the reward port (here a spout; photobeam not shown). Using operant methods, animals learned to make responses in the active response port to receive sucrose in the reward port. Responses in the inactive port had no consequence. The availability of sucrose was signaled by an acoustic noise stimulus. In physiological experiments, a speaker placed directly above the glass spout delivered an acoustic noise stimulus when sucrose solution was available. In behavioral devaluation experiments, a standard fluid dipper made an acoustic stimulus when activated. B and C: rats were trained over a series of sessions, using a progression of operant schedules of reinforcement, to make repetitive entries into the active response port. Rats first experienced an autoshaping session, in which sucrose was delivered into the reward port only in the presence of the acoustic stimulus, which occurred approximately every 60 s. There was no requirement for responding in the response port during the autoshaping sessions. Next, rats were required to enter the response port to earn rewards (FR1, “Fixed Ratio of 1”). Finally, rats could earn rewards only when they entered the response port after the end of a pseudorandomly chosen interval (with mean of 20 or 40 s, chosen from an exponential distribution). Sessions were run daily until the rats earned 50 (Autoshaping, FR1) or 100 sucrose rewards (RI sessions). All rats that were implanted with arrays of electrodes (n = 5) were trained using the full series of procedures for a total of 700 earned rewards, as shown in C. Rats that were trained for devaluation (n = 32) were divided into 2 groups, with half of the rats trained until they received 150 rewards (B) and the other half trained to 700 rewards (C).

Localization of recording sites in the dorsal striatum. Electrode tips are depicted as dots. Dark dots represent electrodes in the lateral part of dorsal striatum. Light dots represent electrodes in the medial part of dorsal striatum. Coronal and horizontal sections are shown on the left and right, respectively, at the level of the striatum that was implanted for this study. Recordings of neuronal activity were made across the anterior to posterior extent of the lateral and medial striatum.

Acquisition of the instrumental task. A: the mean rate of responding per minute in the response port is plotted as a function of the number of sucrose rewards earned. Bands (dashed lines) represent the 95% confidence interval around the means (estimated using bootstrap methods). The types of operant schedule used for each block of rewards are indicated within the plot. The rate of responding in the response port increased with training, reaching an asymptotic level after about 500 rewards were earned (3rd session with RI40 schedule of reinforcement). B: the probabilities of entry into the reward port in the presence and absence of the acoustic noise stimulus are plotted as a function of the number of rewards earned. As in A, the bands indicate the 95% confidence intervals. Rats were increasingly more likely to enter the reward port following the noise stimulus (black) compared with control responses in the absence of the stimulus (gray). C: response times (RTs) to the noise stimulus were measured as the time between the onset of the stimulus and withdrawal from the response port. The mean (left) and SD (right) of the RTs decreased over training, especially during the initial sessions (FR1, RI20, RI40; lines depict medians and interquartile ranges across subjects). D: movement time (MT) to the spout, from response port withdrawal to reward port entry, also decreased during training. There was a very large decrease in the variability of these movements between the 1st (FR1) and 2nd (RI20) sessions. E and F: response and MTs are shown for a single rat. Raster plots show the timing of response time from onset of the stimulus to withdrawal from the response port (top panel) and MT from withdrawal from the response port to entry into the reward port (i.e., contact with the spout, bottom panel). Data from the first 50 rewarded trials from the first session (FR1) are shown as gray tick marks. Data from the last 50 rewarded trials from the last session (RI40) are shown as black tick marks. Rewarded trials are ordered from early to late in each session, with the early part of each session shown at the top of each raster.

Effects of devaluation on instrumental behavior. A: devalued (white) and control (gray) animals made an equal number of head entries in the response port during extinction testing. Data are expressed as the proportion of entries into the response port relative to the last day of training (means ± SE). B: entries into the reward port during extinction testing were significantly reduced in the devalued animals (white) compared with control animals (gray). Data are depicted as the raw number of entries into reward ports during the 5-min extinction session (means ± SE). The effect of devaluation was similar when the data were expressed as a proportion of the last training day. C: sucrose consumption after the extinction session decreased in devalued rats (white) compared with controls (gray).

Examples of learning-related changes in striatal activity. A: spike activity, recorded from a single electrode, is shown for the series of training sessions (columns) with random-interval procedures. Firing rates changed progressively, becoming increasingly modulated in each subsequent training session. The 1st row in each column contains raster plots. The 2nd row shows the average perievent histograms. Perievent histograms were generated for display purposes by convolving the spike trains (1-ms bins) with Gaussian windows (20 ms SD). The 3rd row shows the spike waveforms. The 4th row shows the interspike intervals (1-ms bins). B: changes in striatal firing rates were also apparent over ensembles of simultaneously recorded neurons in the medial and lateral striatum. Here, we show activity from all neurons recorded in one rat in the first and last sessions of random interval training. Neuronal activity is more visibly modulated at the end of training, particularly sharply around the stimulus in the lateral striatal neurons.

Learning-related changes in population averages from the striatum. A: population averages of normalized firing rates are shown for all striatal neurons (lateral and medial) during the initial training sessions (VI60, FR1, RI20, RI40). The timing of withdrawals from the response port are shown above each population average as boxplots, except for the autoshaping session. Perievent histograms were generated by convolving the spike trains (±0.5 s, 1-ms bins) with Gaussian windows (25 ms SD) and decimating by a factor of 25. B: absolute changes in population activity are shown for the neurons in A. Here, firing rates were normalized to have a mean of zero and were squared prior to averaging. This analysis was done to account for changes in firing rates independent of whether the change was an increase or decrease in firing rate. Together, these plots show that consistent modulation of population activity occurred exclusively during the withdrawal from the response port.

Task-related modulations of firing rate. A: neuronal activity is shown for a neuron with significant modulation of firing rate around the reward-predictive stimulus and increased firing rate during movement from the response port to the reward port. B: neuronal activity is shown for another neuron with significant modulation around the reward-predictive stimulus. This neuron fired during entries into the response port and did not fire during movement from the response port to the reward port. In both panels, the top plots are raster plots and show the timing of spikes around the onset of the noise stimulus. The periods when the rats were in the response port are depicted as horizontal gray bars (some of which are from previous responses and occurred before the time of the stimulus). The timings of subsequent entries into the reward port are depicted as gray triangles (starting ∼0.25 s after the stimulus). The 2nd and 3rd plots from the top show the average firing rate (measured in 25-ms bins) and the spike probability (measured in 1-ms bins), respectively. The bottom plots show the results of the empirical fluctuation analysis, with the statistic y_n(t) (see ) plotted on the ordinate axis and time around the stimulus plotted on the abscissa. The black lines depict deviations from the expected value of the firing rate and the gray lines depict the boundaries of the fluctuations that would be expected based on a random diffusion process (Brownian motion).

Neuronal correlates of task acquisition. A: the fractions of neurons with task-related modulations of firing rate are plotted as a function of the number of rewards earned. Neurons in both the medial (gray) and lateral (black) striatum showed increases in task-related modulations over the period of learning and more neurons in the lateral striatum were modulated during learning. B: the fractions of neurons with modulations of firing rate during sucrose consumption are plotted as a function of the number of rewards earned. Neurons in both the medial (gray) and lateral (black) striatum showed learning-related increases in neuronal modulation and, as above, more neurons in the lateral striatum were modulated during learning. C: the fractions of neurons with modulations of firing rate during a narrow time window around the onset of the acoustic noise stimulus (−30 to +20 ms compared with +20 to +70 ms) are plotted as a function of the number of rewards earned. Significantly more lateral neurons fired in response to the acoustic noise stimulus compared with medial neurons and the fractions of stimulus-related activations in the lateral striatum increased with training. D: the fractions of neurons with modulations of firing rate during the withdrawal from the response port (±50 ms) are plotted as a function of the number of rewards earned. Significantly more lateral neurons fired during this behavior compared with medial neurons and the fractions of response-related neurons did not increase over the period of training.

Stimulus-dependent modulations of response-related activity in the dorsal striatum. A: lateral striatal neurons fired differently during entries into the response port that produced or did not produce the acoustic noise stimulus (rank-sum test, P < 0.05). Each column is a different neuron recorded in a different session. Neuronal activity is aligned to the stimulus (dark gray) or to the time when the stimulus would have occurred (absence of stimulus, light gray). Withdrawals from the response port are indicated in the raster plots by black circles. Perievent histograms were generated for display purposes by convolving the spike trains (1-ms bins) with Gaussian windows (20 ms SD). B: neuronal activity is plotted around the time of entry into the reward port for the same cells shown in A. The cells fired immediately after the stimulus and during movement to the reward port and were not further modulated during the acquisition or consumption of sucrose.

Learning-related changes in local field potentials (LFPs). A: mean LFPs recorded from lateral electrodes at the time of the stimulus. Four blocks of 50 trials are depicted (autoshaping: lightest color, FR1, first block of RI40 training; last block of RI40 training: darkest color). Numbers in parentheses indicate the rewards accumulated in the block. The recorded potentials are characterized by an initial series of positive fluctuations followed by a wider negative fluctuation. Shaded areas depict the SE. B: mean local field potentials recorded from medial striatum at the time of the stimulus. Conventions are as in A. C: the range of LFP fluctuations (maximum–minimum voltage, 0–250 ms after the stimulus) increased with training, but did not differ by area. D: power spectra of the LFP signals following the stimulus (0–512 ms) are depicted across all blocks of response training. Power was noted in 3 bands: a low-frequency band (<5 Hz), a theta-frequency band (5–8 Hz), and a gamma-frequency band (30–45 Hz). E: power in the theta frequency was higher in lateral than that in medial striatum and varied significantly with training. F: significant spike/LFP coherence was found only at low frequencies (<5 Hz; data not shown). The proportion of significant spike/LFP pairs was greater in the lateral than that in medial striatum and increased with training.