^{1}Zanvyl Krieger Mind/Brain Institute and Solomon H. Snyder Department of Neuroscience, Johns Hopkins University, Baltimore, MD 21218, USA. yau@jhu.edu

Abstract

We have previously analyzed shape processingdynamics in macaque monkey posterior inferotemporal cortex (PIT). We described how early PIT responses to individual contour fragments evolve into tuning for multifragment shape configurations. Here, we analyzed curvatureprocessingdynamics in areaV4, which provides feedforward inputs to PIT. We contrasted 2 hypotheses: 1) that V4curvature tuning evolves from tuning for simpler elements, analogous to PIT shape synthesis and 2) that V4curvature tuning emerges immediately, based on purely feedforward mechanisms. Our results clearly supported the first hypothesis. Early V4 responses carried information about individual contour orientations. Tuning for multiorientation (curved) contours developed gradually over ∼50 ms. Together, the current and previous results suggest a partial sequence for shape synthesis in ventral pathway cortex. We propose that early orientation signals are synthesized into curved contour fragment representations in V4 and that these signals are transmitted to PIT, where they are then synthesized into multifragment shape representations. The observed dynamics might additionally or alternatively reflect influences from earlier (V1, V2) and later (central and anterior IT) processing stages in the ventral pathway. In either case, the dynamics of contour information in V4 and PIT appear to reflect a sequential hierarchical process of shape synthesis.

Curvature tuning hypotheses. Circuit diagrams portray possible network mechanisms underlying V4curvature tuning. Neurons in V1/V2 project to V4 neurons. Gray scale in the circuit diagrams indicates signal strength. Response curves below the diagrams represent hypothetical temporal profiles for V4 orientation tuning (blue and cyan) and curvature tuning (red). Vertical line indicates stimulus onset. Shaded regions indicate the response intervals depicted in the corresponding network diagrams. (A) Responses to individual orientations under the curvature synthesis model. Orientation input is sufficient to drive V4 orientation responses but fails to initiate recurrent network processes for curvature synthesis. (B) Responses to multiorientation (curved) contours under the curvature synthesis model. Dashed black trace indicates summed V4 response levels. Early V4 responses reflect tuning for orientation components. Simultaneous multiorientation inputs are sufficient to initiate recurrent network processes. Late V4 responses show selectivity for curvature based on this recurrent network processing. (C) Lack of response to individual orientations under the threshold nonlinearity model. (D) Simultaneous multiorientation inputs sum to produce immediate curvature tuning under the threshold nonlinearity model.

Example neural responses and model. (A) Left: contour fragment stimuli (shown here as white icons) were flashed in the cell's RF while the monkey performed a fixation task (see Materials and Methods). Background gray level (see scale bar) indicates response to each stimulus averaged across the 500-ms presentation period. Middle: linear/nonlinear models were based on Gaussian tuning functions for 2 component orientations (θ_{1} and θ_{2}, blue and cyan bars spanning a range of 1 standard deviation around peak) and their angular offset (θ_{rp}, white arrow). Responses were modeled as a weighted sum of linear orientation and nonlinear orientation configuration terms (equation). Right: relationship between observed and predicted response rates (r^{2} = 0.55). (B) Response profiles of linear (blue and cyan histograms) and nonlinear (red histogram) response components in the temporal model. Solid vertical line indicates stimulus onset (0 ms). Dashed vertical lines indicate times of interest (Fig. 3).

Population-level dynamics based on contour tuning models. (A) Peak linear and nonlinear response components for the 62 neurons in our analysis sample. Each cell is represented by adjacent rows corresponding to linear (blue) and nonlinear (red) response components exceeding a 70% peak-to-baseline threshold (see Materials and Methods). The blue curves indicate each neuron's total linear response summed over both linear weight functions (e.g., sum of blue and cyan curves in Fig. 2C). Brightness is mapped to the 70–100% range. Neurons are plotted from top to bottom in order of descending nonlinearity index values. (B) Three example neurons showing nonlinear, transitional, and linear tuning patterns (top to bottom). In each case, blue, cyan, and red histograms depict the 2 linear and 1 nonlinear temporal weighting functions, respectively. (C) Top: average temporal profiles of the linear (blue) and nonlinear (red) estimated response components across 62 neurons. Vertical lines mark the first time point above 90% of baseline-to-peak range (see Materials and Methods). Bottom: average linear response component peaked early for both primarily linear cells (dashed blue) and mixed linear/nonlinear cells (solid blue). Average nonlinear response component was delayed for both primarily nonlinear cells (dashed red) and mixed linear/nonlinear cells (solid red).

Observed and predicted temporal response patterns. Observed and predicted stimulus response patterns at 5 selected time points. Conventions as in Figure 2A. First row: responses predicted by first linear model component (μ_{1} = 120°). Second row: responses predicted by second linear response component (μ_{2} = 70°). Third row: responses predicted by nonlinear model component. Fourth row: complete model predictions. Fifth row: observed response patterns.

Relationship between the transition index and the nonlinearity index. Positive transition values (horizontal scale) signify transition from linear to nonlinear tuning. The average transition value (dashed vertical line) across the analysis sample was 0.52 ± 0.18 (mean ± SE). Nonlinearity index values (vertical scale) based on responses averaged across the entire 500-ms response period range from 0 (purely linear) to 1 (purely nonlinear).

Linear and nonlinear peak time distributions. (A) Histogram of linear (blue) and nonlinear (red) 90% threshold peak times for 62 neurons in the analysis sample. (B) Cumulative distribution plots of linear (blue) and nonlinear (red) peak times.

Population-level dynamics based on ANOVA and vector strength. Solid vertical line indicates stimulus onset time. Dashed vertical lines indicate peak times for each curve. Gray curve: average response variance explained across entire V4 population (n = 127) by ANOVA with component orientation main effects, as a function of time. V4 responses are maximally modulated by component orientation information 115 ms after stimulus onset. Average response variance explained by ANOVA at each time point was limited due to the size of the time bins. ANOVA explained substantially more response variance in rates averaged across the entire response period (Supplementary Fig. S1D). Black curve: average vector strength values across entire V4 population (n = 127) as a function of time represent evolution of curvature selectivity. Larger values signify greater selectivity. Filled markers indicate statistically significant vector strength values. Vector strength values peak 170 ms after stimulus onset.

Dynamic shape synthesis in V4 and PIT. Temporal profile of linear and nonlinear response components across 62 V4 neurons and 89 PIT neurons. Curves show time courses of average weight values following stimulus onset (0 ms). Vertical lines indicate peak times for each curve. Linear V4 responses (solid blue) are signals for individual orientations. Nonlinear V4 responses (solid red) represent orientation combinations, i.e., contour fragments characterized by orientation change (curvature). Initial responses in PIT (PIT linear, dashed blue) are signals for individual contour fragments, presumably derived from V4 inputs. Later PIT responses (PIT nonlinear, dashed red) represent contour fragment combinations. Idealized stimulus diagrams at top exemplify how contour elements in V4 and PIT RFs (dashed circles) could be synthesized across time.

## PubMed Commons