Toy simulation illustrating the potential benefits of analyzing for information in fine-grained activity patterns. (a) A region of interest (ROI) consisting of 113 voxels. (b) The fMRI data matrix (time by voxel) for the ROI. The voxels are lined up in arbitrary order along the space axis. (c) In a standard activation-based analysis of the ROI, all 113 time courses are first averaged across voxels, yielding an average time course (black line). This shows that there is little difference in the overall activation of the region during the two experimental conditions indicated by the rectangular reference functions (red for condition 1, blue for condition 2). (d) Modeling each voxel time course with a separate hemodynamic predictor for each trial suggests that each condition (red, blue) is associated with a replicable and distinct spatial activity pattern. (e) The average spatial patterns associated with the two conditions (red, blue) are plotted with standard-error bars. (Only 30 of the 113 voxels are shown.) For all voxels, the error bars overlap – indicating that the univariate effects are not significant in any voxel, even without correction for multiple comparisons. A mapping analysis would not mark any voxel in this toy simulation. However, a multivariate test performed on all voxels (multivariate analysis of variance here) demonstrates that the two activity patterns (red, blue) are in fact significantly different (p<0.05). For a similar example based on actual fMRI data, see Fig. 5.

In this figure photographs serve as a stand-in for activity patterns. Even if the neuronal activity pattern is corrupted beyond recognition in the fMRI pattern, information distinguishing the experimental conditions will still be present, as long as the fMRI patterns are replicable and distinct for each condition. This would not be the case in a brain region displaying the same neuronal (and thus the same fMRI) activity pattern in both conditions. Information-based analysis detects differences between the spatiotemporal activity patterns present in a region in different experimental conditions. It can, thus, reveal fine-grained neuronal information, even when the underlying neuronal patterns cannot be accurately visualized with fMRI.

In order to map a neuroimaging volume for information in fine-grained activity patterns, a multivariate searchlight can be moved throughout brain space to obtain a continuous map of local pattern information (Kriegeskorte et al. 2006). (a) The searchlight, here, is a sphere of 4-mm radius illuminating 33 voxels (isotropic, 2-mm width). (b) The searchlight is centered on each voxel in turn (red arrows spaced more widely in the figure for visual simplicity) to select a spherical voxel cluster to be subjected to joint multivariate analysis. (c) The resulting map reveals, not activation, but information present in local activity patterns. Here the map shows regions, whose response pattern distinguishes two categories of visual objects. The information-based map has been thresholded at a false-discovery rate (FDR) of q<0.05 to select the voxels to be highlighted. The voxel color encodes the univariate t value, revealing the actual contrast pattern. Voxels below the information-based threshold but within the searchlight scope of a suprathreshold voxel are marked transparently. In this example, there are more suprathreshold voxels in the information-based map (nontransparently highlighted) than in the activation-based map, i.e. the t map from data smoothed by convolution with the a sphere of the same radius (not shown). (d) The t map from unsmoothed data (texture map in (c)), thresholded to highlight the same number of voxels as the information-based map, reveals salt-and-pepper patterns, whose significance is difficult to assess. When correctly thresholded (not shown), the unsmoothed-data t map highlights very few voxels, failing to detect the widespread pattern information evident in (c).

(a) Comparison of fMRI (standard and hi-res) and invasive electrophysiology in terms of spatial precision and number of channels, with which brain activity is simultaneously measured. The gray shading indicates the precision range of hi-res fMRI (≤ 2 mm voxel width). (b) The FCNR drops rapidly when fMRI resolution is increased (challenge (2), green). The curve depends on many parameters, but retains its sigmoid shape for realistic values. The curve shown (for a 3T scanner) is based on the physiological-noise model of Krüger et al. (2001) and on theory and measurements of Bodurka et al. (2007): we assume physiological effects with a standard deviation of 1.25% signal change (measured value for gray matter during resting state). Here we further assume that half this physiological variance is of interest (e.g. related to some experimental task, thus defining the functional contrast) and the other half is physiological noise. We consider gray-matter voxels reflecting physiological effects with no partial-volume sampling. We assume a spatial signal-to-noise ratio (SNR) of 100 for 2-mm-wide isotropic voxels. This SNR is intermediate between the values determined for a birdcage coil and a 16-channel receive-only array coil used with a 3T MRI scanner (Bodurka et al. 2007). The blue curve shows the rapid growth of the number of voxels needed to cover the brain as voxels become smaller (challenge (3), blue). (c) In the hi-res range (gray shading), fMRI may give an inaccurate (blurred, distorted, displaced) image of the neuronal activity pattern (challenge (1), red). Moreover, a voxel-to-voxel intersubject correspondency is very difficult to define and, in fact, may not exist (challenge (4), magenta). Standard fMRI, by contrast, operates at the spatial scale of cortical areas. At this coarser scale, hemodynamic blurring, distortion and displacement of neuronal activity patterns is less problematic and intersubject correspondency is roughly determined by the Talairach common space, and more precisely by cortex-based intersubject alignment or a functional localizer experiment.

Results of Cheng et al. (2001) show that fMRI patterns elicited in V1 by left- and right-eye stimulation (a and b, respectively) are replicable and distinct, but corrupted by vascular artefacts obscuring the ocular-dominance pattern present in the neuronal responses. Because ocular-dominance domains are spatially complementary by definition, they could be visualized by subtracting the fMRI patterns (c). In general, we cannot rely on this assumption and hi-res fMRI patterns will misrepresent the fine structure of neuronal patterns (as seen in a and b). Cheng et al. performed gradient-echo echoplanar imaging at 4T using (0.47mm)² × 3mm voxels.

(a) Visual gratings of different orientations elicit very similar responses in single voxels (isotropic, 3-mm wide) within early visual areas V1 and V2 (left). However, when the responses of many voxels are linearly combined with appropriate positive and negative weights (determined using independent data), the resulting combined response can reveal that the region contains significant information about the orientation (right). Error bars indicate the standard deviation. Figure adapted from Kamitani & Tong (2005). (b) A linear combination of the voxel responses can discriminate conditions, even when both the best-discriminating single voxel and the average across all voxels (the regional activation level) do not show significant differences. Here 100 voxels in V1 were used to discriminate two stimulus orientations that were invisible to subjects due to masking. Error bars indicate the standard error of the mean. Figure adapted from Haynes & Rees (2005).

Corresponding functional regions (green ROIs) are defined (1) by a functional localizer experiment analyzed separately for each subject or (2) by selecting a region in Talairach space and projecting it back into the native space of each subject’s data or (3) by cortex-based intersubject alignment. Blue double arrows indicate the coarse correspondency defined by one of these three methods. ROIs will not in general have equal numbers of voxels. A precise voxel-to-voxel correspondency (gray double arrows) may not exist and is not needed. Each subject’s ROI undergoes individual analysis (red arrows) that estimates the information in its local activity pattern (e.g. the mutual information between stimulus and multivariate response). The information effect need not be quantified information-theoretically, but can be measured by any multivariate effect estimate (e.g. a t value obtained for independent data in a linear-discriminant subspace, or, similarly, the time course data projected onto that linear discriminant, as described in the text). Combination across subjects can then proceed at the level of the individual information effects in a fixed- or random-effects framework for group analysis.