|
|
J Am Med Inform Assoc. 2001 Sep–Oct; 8(5): 443–459. | PMCID: PMC131042 |
Copyright © 2001, American Medical Informatics Association An Integrated Software Suite for Surface-based Analyses of Cerebral Cortex David C. Van Essen, PhD, Heather A. Drury, MS, James Dickson, MS, John Harwell, MS, Donna Hanlon, MS, and Charles H. Anderson, PhD Affiliation of the authors: Washington University School of Medicine, St. Louis, Missouri. Received March 16, 2001; Accepted May 8, 2001. The authors describe and illustrate an integrated trio of software programs for carrying out surface-based analyses of cerebral cortex. The first component of this trio, SureFit (Surface Reconstruction by Filtering and Intensity Transformations), is used primarily for cortical segmentation, volume visualization, surface generation, and the mapping of functional neuroimaging data onto surfaces. The second component, Caret (Computerized Anatomical Reconstruction and Editing Tool Kit), provides a wide range of surface visualization and analysis options as well as capabilities for surface flattening, surface-based deformation, and other surface manipulations. The third component, SuMS (Surface Management System), is a database and associated user interface for surface-related data. It provides for efficient insertion, searching, and extraction of surface and volume data from the database. The cerebral cortex is the dominant structure of the mammalian brain and is responsible for an impressively diverse range of sensory, motor, and cognitive functions. Anatomically, the cortex is a sheet-like structure whose surface area greatly exceeds the surface area of a smooth, solid shape containing it. In large-brained mammals, including humans, the cortex is extensively folded, and the pattern of convolutions varies considerably from one individual to the next. The irregularity and variability of these convolutions pose major challenges in analyzing and visualizing cortical structure, function, and development. These problems are particularly acute in view of the explosion of high-resolution data derived from many different experimental methods, particularly non-invasive neuroimaging methods such as structural and functional magnetic resonance imaging (MRI) applied to human beings and nonhuman primates. Computational cortical cartography represents a powerful general approach to dealing with these problems by using surface-based visualization and analysis methods. The essence of the approach is to represent the cortex by explicit surface reconstructions onto which various types of experimental information are mapped. The advantages of surface reconstructions can be grouped into four main categories: - Visualization using multiple configurations. Once generated, surface reconstructions can be manipulated in shape to improve visualization. Commonly used configurations, besides the initial (fiducial) three-dimensional shape of the cortex, include inflated (extensively smoothed) surfaces, spherical surfaces (used for surface-based coordinates, as discussed below), and flat maps, which allow the entire hemisphere to be viewed with only modest distortions, albeit at the price of artificial cuts (akin to those used in maps of the earth's surface).
- Localization using surface-based coordinates. It is often important to be precise about exact locations in the cortex. Surface-based coordinates (latitude and longitude on a sphere) provide a concise, precise, and objective metric that respects the topology of the cortical surface. In this respect, they offer important advantages over standard stereotaxic coordinates or geographically referenced descriptions.
- Compensation for individual variability. Comparison of results obtained from different individuals is inherently challenging because of the combination of geographic differences (in the pattern of folding) and functional differences in the size of each cortical area (which can vary two-fold or more) and in their shape and location. Surface-based warping provides a natural and powerful way to compensate for these differences while respecting the topology of the cortical sheet.
- Consolidation onto surface-based atlases. With the explosion of information about cortical structure and function, surface-based atlases will become increasingly important, just as atlases of the earth's surface are invaluable repositories of information about countless aspects of geography. Information from many different individuals can be brought into a common reference frame.
The many steps involved in surface-based cortical analysis can be subdivided into four main stages, each dealing with distinct types of data and having different analysis objectives. Historically, the methods used to carry out each of these processing stages initially involved manual or other noncomputerized methods that have only recently been supplanted by automated or semi-automated methods of computerized neuroanatomy. - Acquisition of structural data. Surface-based analysis starts with a source of structural information that can be used to infer the shape of the cortical sheet. Until the 1990s, surface reconstructions were generally based on postmortem histology, using photographs of histologic sections. The situation has changed dramatically with the advent of noninvasive imaging methods (particularly structural MRI) that can routinely be used to image the structure of cerebral cortex in human beings as well as laboratory animals.
- Segmentation and surface reconstruction. Generating accurate surface reconstructions of cerebral cortex is a challenging problem in general, because the cortex is highly complex in shape and because the image data on which reconstructions are based are typically noisy or contain artifactual irregularities. The earliest explicit surface representations were generated by physical models.1,2 A variety of methods are now available that allow cortical segmentation and surface reconstruction with reasonable accuracy and robustness (see below).3–6
- Surface reconfiguration and flattening. The earliest approaches to reconfiguring cortical shape involved generating cortical flat maps by manual methods, such as tracing contours with a pencil and tracing paper.7,8 A variety of methods are now available for inflating the cortex and for making computer-generated flat maps using various metrics for minimizing distortions.4,9–13
- Mapping to a surface-based atlas. Early surface-based atlases were based on manually generated flat maps to which data were transposed manually, on the basis of geographic landmarks and approximate distances.8,14 The emergence of surface-based deformation algorithms15–17 has allowed an objective approach to bringing data into register with an atlas while respecting the topology of the cortical surface.16,18–19
Although the use of surface reconstructions has increased rapidly in recent years, enormous potential remains untapped in terms of ongoing neuroimaging and systems neuroscience studies that do not yet capitalize on the power of surface-based analyses. This is largely because key elements of the enabling technology have only recently become available. The contribution of our laboratory to this effort involves the development of an integrated software suite for surface-based analyses of cerebral cortex. Our objective has been to provide software that is freely available to the neuroscience community, runs on multiple hardware platforms, and can be used to carry out the major stages of surface reconstruction and analysis efficiently and in as automated a manner as possible. The integrated system has three main software components. The first component, SureFit (Surface Reconstruction by Filtering and Intensity Transformations), is used primarily for cortical segmentation, volume visualization, and initial surface generation. The second component, Caret (Computerized Anatomical Reconstruction and Editing Tool Kit), provides a wide range of surface visualization and analysis options as well as capabilities for surface flattening, surface-based deformation, and a number of other surface manipulations. The third component, SuMS (Surface Management System), is a database and associated user interface for dealing efficiently with the increasing number of complex data sets associated with surface-based analyses. It provides an organized framework for efficiently inserting, searching, and extracting surface and volume data from a database. Here we provide a general introduction to and overview of the surface-based analyses that can be carried out using SureFit, Caret, and SuMS. Key algorithmic principles underlying the main processing steps are described, and selected examples are used to illustrate the methods and the utility of the overall approach. Additional information is available in the user's guide to SureFit and Caret ( http://stp.wustl.edu/resources/cortcart.html), in the on help menu for Caret ( http://stp.wustl.edu/caret/html4.3/reference_manual_toc.html), and in related publications. 11,20–22Overview of Processing Stages in Surface-based Analysis Surface Reconstruction Some of the key stages of surface-based analyses are shown in a generic flow chart (Figure 1) and in an illustrative structural and functional MRI (fMRI) data set processed in SureFit and Caret (Figure 2). For each experimental hemisphere under investigation, a key initial objective is to obtain a fiducial surface reconstruction—one that represents the shape of the cortex as accurately as possible. The strategy for achieving this objective depends on the nature of the primary structural data. If structural MRI or cryosection data are available, the preferred strategy is to use a volume-based method such as SureFit to carry out automated segmentation and surface reconstruction (Figure 1, left side) . The segmentation process, applied to structural MRI image data of the type shown in Figure 2A, results in a segmentation (a binary volume) whose boundary represents the shape of the cortex, as in the slice shown in Figure 2B. This segmentation (or “reconstruction substrate”) is used to automatically generate an explicit surface representation, i.e., a wire-frame tessellation whose nodes lie on the boundary of the segmentation (typically 50,000 to100,000 nodes for a human hemisphere) and whose surface topology is defined by the links between nodes. The initial (raw) surface has a blocky appearance associated with the cubical voxels (typically 1 mm3) of the segmentation. Hence, it is slightly smoothed to generate a satisfactory fiducial surface (Figure 2C). If only histologic sections are available, surfaces can be reconstructed by an alternative contour-based reconstruction strategy, using options available in Caret (Figure 1, right side). This entails drawing contours along a particular cortical layer (e.g., layer 4) in individual sections, bringing contours from adjacent sections into register, and generating an explicit surface reconstruction using an automated tessellation method. Although the method for surface reconstruction is different, the outcome of segmentation-based vs. contour-based processes is fundamentally the same, that is, an explicit surface that represents the shape of the convoluted cortex. Visualizing Experimental Data Surface reconstructions are invaluable for visualizing many types of experimental data, ranging from minimally processed representations of primary data to highly abstracted representations. One particularly important class of experimental data involves functional activation patterns obtained using fMRI. For example, Figure 2 D shows an fMRI activation focus in a coronal slice through the intraparietal sulcus (overlaid on the structural MRI slice), generated in a behavioral paradigm involving saccadic eye movements. 23 These data were then mapped from the volume onto a surface representation using a SureFit algorithm that takes local surface orientation into account when mapping volumetric fMRI data onto the surface. Surface Reconfiguration Once a fiducial surface has been generated, its shape can be modified into several alternative configurations that are widely used in surface-based analysis. Standard surface configurations include inflated maps (Figure 2F), spherical maps (Figure 2G), and flat maps (Figure 2H). These exemplar surfaces are painted with a combination of functional and geographic data, in which darker shading indicates buried cortex; fMRI data are represented in the figure by white or light for surface nodes above a threshold activation level. Each alternative configuration has advantages for visualization and analysis. The inflated configuration looks like a lissencephalic hemisphere and gives an intuitive sense of geographic location. The spherical configuration (Figure 2G) allows for determination of spherical coordinates (latitude and longitude), which constitute a precise and objective localization metric that respects surface topology. The flat map shows the entire pattern in a single view (Figure 2H), with distortions reduced by artificial cuts like those used on flat maps of the earth's surface. Notice that the polar coordinates, while generated on the sphere, can be readily viewed and interpreted after projection to the flat map. All the surface maps, but particularly the flat map, reveal that the behavioral task (saccadic eye movements) was associated with numerous activation foci that are more efficiently and accurately visualized on a single surface than in a series of volume slices. Surface-based Atlases The final stage of surface-based analysis illustrated in Figures 1 and 2 involves mapping data from an individual hemisphere to a surface-based atlas. To ensure a mapping that respects the topology of the cortical surface, it is desirable to use a surface-based deformation algorithm applied to spherical maps, which are unaffected by cuts in the surface. In Figure 2, the source hemisphere (G) was deformed to the target hemisphere (I) using geographically defined landmarks in the source and target maps and a differeomorphic deformation algorithm developed by Bakircioglu et al. 17 In Figure 2 J, the results of the deformation are shown by projecting the data onto a flat map of the surface-based atlas. This allows the data from a single subject to be viewed in relation to any other data registered with the atlas, such the boundaries of Brodmann's architectonic subdivisions shown on the flat map. 20Database Entry and Retrieval Each of the display formats and the ancillary data shown in Figure 2 are useful for specific visualization and analysis options. Consequently, it is desirable to be able to access any or all of the data in an efficient, organized way, even though the data comprise more than a dozen files that represent a variety of volume data, surface geometry data, and ancillary experimental data. This access was achieved by storing the data in the SuMS database, which provides several options for searching and retrieving data. Data entry and retrieval from SuMS as well as data visualization in SureFit and Caret are greatly facilitated by grouping common families of files into organized sets that are listed in “specification files.” By selecting the appropriate volume or surface specification file (or files) of interest, a family of related files can be downloaded as a group. An especially easy and direct way to do this is by direct hyperlink connections to particular specification files in SuMS, as exemplified by the hyperlinks specified in the legend to Figure 2. We now describe SureFit, Caret, and SuMS each in greater detail, to illustrate their functionality and the algorithms on which they are based. Visualization SureFit includes general-purpose capabilities for volume visualization (viewing one or two volumes concurrently) and surface visualization (viewing up to three surfaces concurrently). For example, Figure 3A shows the main SureFit window, with a coronal slice through a cortical segmentation overlaid on the structural MRI volume. The overlay option allows the relationship between the segmentation (solid shading; red in the actual view) and the intensity data to be compared directly. Two surface viewing windows are shown, one with the fiducial surface (Figure 3B) and the other with an inflated surface (Figure 3C). The spatial relationships among different volume and surface representations can be visualized using a “pick” option, which highlights points in each of the surface views (arrows) that correspond to the cross-hairs in the slice viewer. Segmentation and Surface Reconstruction The main functionality of SureFit is to generate segmentations and surface reconstructions of cerebral cortex (like those in Figure 3) from structural MRI or other scale image data. This involves an automated sequence of image-processing operations and other functions, with only two stages (volume preparation and interactive error correction) that entail significant user interaction. Volume Preparation Prior to launching the automated segmentation process, several preparatory steps are carried out. These include: - Tagging each data set with appropriate general information (e.g., case name, region, and specific comments)
- Orienting the volume and cropping it to the desired spatial extent
- Identifying the anterior commissure (AC) as a key geographic landmark
- Setting two parameters (the matter and white matter peaks in the intensity histogram) that are needed for automatic segmentation.
Automated Segmentation Automated segmentation in SureFit is based on several structural characteristics of cerebral cortex that are schematized in Figure 4. The cortex is a sheet-like tissue, approximately constant in thickness, adjoined on its inner side by subcortical white matter. On its outer side, the pial surface adjoins cerebrospinal fluid in gyral regions, but in sulcal regions the cerebrospinal fluid gap between apposed pial surfaces may be very narrow. SureFit aims for a segmentation whose boundary runs approximately midway through the cortical thickness (i.e., cortical layer 4), because this represents the associated cortical volume more accurately than do segmentations running along either the inner or outer boundary. 4 To this end, SureFit generates probabilistic representations for both the inner boundary and the outer boundary, using cues related to image intensity, intensity gradients, and the spatial relationships between the inner and outer boundaries. One general strategy is to postpone deterministic (yes/no) decisions that create binary volumes as long as possible, until various probabilistic representations have been generated and appropriately combined. Key stages in the SureFit segmentation process are illustrated in Figure 5 for a slice through lateral temporal cortex of a structural MRI volume (Figure 5A). This case was chosen because the image data are relatively noisy and low in contrast, making automated segmentation an inherently challenging process. The intensity histogram (Figure 5B) for this volume shows a characteristic pair of intensity peaks, one associated with white matter and the other with matter. Along the inner (–white) boundary, the image intensity should be approximately midway between the peaks for gray matter and white matter. This intensity-based cue is made explicit using a Gaussian intensity transformation, whose peak represents the most likely inner boundary intensity (inner boundary arrow in Figure 5B) and whose standard deviation reflects the noisiness in the image data (see legend for details). The resultant inner boundary intensity transformation has local maxima running approximately along the inner boundary, but there are many irregularities owing to fluctuations in the image data. Other cues for the inner boundary are related to the intensity gradient, whose magnitude is shown in Figure 5D. The gradient along the inner boundary is steeper than in neighboring gray matter and white matter, but it is more shallow than in other regions (e.g., along parts of the outer boundary). This information, along with cues related to the direction of the intensity gradient, is combined with the intensity-based map in Figure 5C to generate a composite inner boundary map (Figure 5E) that is more reliable than the individual contributing maps (see legend to Figure 5 for details). For the outer (pial) boundary, one cue is based on image intensity (Figure 5F), using a Gaussian intensity transformation centered at the outer boundary arrow in Figure 5B. This intensity-based map has prominent local maxima along gyral boundaries, where matter adjoins cerebrospinal fluid. In sulcal regions the local maxima are less pronounced, because a prominent cerebrospinal fluid gap is generally lacking. Instead, evidence for the outer boundary in sulcal regions is collected using a customized filtering process whose output is high along regions midway between two oppositely oriented inner boundaries (Figure 5G). The composite outer boundary map (Figure 5H) reflects a combination of the cues in Figures 5F and 5G and gradient-based cues analogous those used for the inner boundary. In addition to the probabilistic inner and outer boundary maps, a segmented representation of cerebral white matter is generated (Figure 5I). This entails thresholding the intensity volume at a level that fills most of the white matter, disconnecting noncerebral structures (brainstem, cerebellum, skull, and eye), transecting mid structures, and removing the extraneous regions by a flood-filling process. The brainstem is disconnected using a diagonal transection applied near the pons (based on its location relative to the anterior commissure). The eye and skull are disconnected by automatically identifying and removing the high-intensity (fatty) region behind the eyeball. In Figure 5 J, the maps of inner and outer boundaries are combined to generate a smooth map of position along the radial axis (i.e., the axis deated by arrows in Figure 4). This involves taking the difference between blurred versions of the inner and outer boundary maps (and also assigning maximal intensity to regions near the core of cerebral white matter). The resultant radial position map is white in the interior, black on the exterior, and smoothly graded in intensity across the cortical thickness. Thresholding the radial position map at an intermediate intensity level generates a segmented volume (Figure 5 K) whose boundary tends to run midway through the cortical thickness, as can best be appreciated by viewing the segmentation superimposed over the structural MRI intensity volume (Figure 5 L). The initial segmentation is used as the substrate for generating an explicit surface reconstruction. 24 Error Detection and Correction Topological errors (“handles”) in the initial segmentation are typically attributable to noise, large blood vessels, or regional inhomogeneities in the structural MRI volume, or a combination of these. Errors are localized by inflating the initial surface reconstruction to a highly smoothed ellipsoidal shape and using the orientation of surface normals to identify “crossover” regions where the surface is folded over on itself. Clusters of surface nodes associated with cross-overs are mapped from the fiducial surface into corresponding voxel clusters in the volume. If the error is a bridge across opposite banks of a sulcus (an “exohandle”), correction entails removing voxels from the bridge. If, instead, the error is a hole in the white matter between two sulci, an “endohandle,” correction entails adding voxels to the hole. The automated error correction process tests for both exohandles and endohandles in the vicinity of each location determined to contain an error. The localized patches used for these tests conform to the shape of temporary segmentations that are based on different threshold levels for the radial position map. If the trial patch reduces the number of topological handles in the segmentation, as determined from an Euler count applied to the volume, 25 it is accepted as a permanent correction and the process moves on to the next error patch. The automated error correction process sometimes fails, especially for handles that are notably large or irregular. Residual errors can be removed by interactive editing, which allows voxels to be added or removed one at a time or in small clusters using dilation or erosion steps within small masked regions. The processing time needed for automated segmentation in SureFit is less than an hour for the initial segmentation plus several hours for automated error correction when run on an SGI Octane. Processing times can be significantly faster on PC systems running Linux. Mapping Geographic and Functional Data Once a satisfactory segmentation is achieved, regions of buried cortex are automatically identified. This entails a combination of dilation, erosion, and volume subtraction operations to generate a volume that includes sulcal but not gyral cortex. Nodes of the reconstructed surface that intersect this sulcus-only volume are identified as buried cortex and are shaded to provide a representation of sulcal versus gyral geography (Figure 2E to J). The process of mapping fMRI data onto surface reconstructions takes advantage of information about local surface orientation. For each surface node, the fMRI image volume is convolved with an ellipsoidal filter centered on that node and oriented parallel to the local surface tangent. The output values are converted to RGB color values according to a predesignated color look-up table and are stored in an “RGB_paint” file format for viewing in SureFit or Caret. In addition, the outputs are stored as scalar values in a “metric file” format for subsequent visualization and analysis in Caret. Volume and Surface Specification Files SureFit generates volume and surface specification files at the end of the automated segmentation process. The volume specification file lists the structural MRI volume, the segmentation files, and several key intermediate files. The surface specification file includes both VTK file format (Visualization Tool Kit 26) and the Caret geometry file format (in which node coordinates and node topology are stored in separate files) as well as paint files that represent cortical geography. This facilitates data entry into SuMS as well as visualization and surface-based analyses in Caret. Surface Visualization Caret includes numerous options for surface visualization, manipulation, and other surface-based analyses. We illustrate some of these visualization capabilities using a surface-based atlas for the macaque monkey as an exemplar data set. The main Caret screen (Figure 6A) includes a surface visualization window as well as menu options along the top and a status display row along the bottom. The initial step in loading data is to select an appropriate surface specification file (using the “Open Specification File” option in the File submenu shown in Figure 6A). This brings up a dialog box (Figure 6B) for choosing the subset of data files to be loaded. Surface geometry is determined by selecting one topology file and two coordinate files—one as the reference (REF) configuration and another as an auxiliary (AUX) configuration. Other available file types are grouped according to the type of experimental data. The specification file in Figure 6B lists four classes of ancillary data—latitude-longitude files, paint files, border files, and atlas files—that are described more fully below. To facilitate the selection process, topology files are identified as CLOSED, OPEN, or CUT, and coordinate files are identified by their configuration (FIDUCIAL, INFLATED, SPHERICAL, or FLAT). Additional information can be obtained by pressing the query (“?”) button, which brings up information in the file header, and by pressing the “List” button for paint and border files, which shows the different labels or categories contained in the file. Once the selected data files are loaded, numerous menu options are available for changing the surface configuration, the viewing perspective, and the information displayed on the surface or in relation to it. In Figure 6A, the flat map configuration has been selected (using the “AUX” toggle button), surface nodes were painted to show cortical geography using the “G” (geography) toggle button, surface tiles were filled using the “Tile” toggle button, and latitude and longitude isocontours were displayed using the “B” (borders) toggle button. The ancillary data shown in Figure 6 A were generated using several of the data analysis options available in Caret. The latitude and longitude isocontours were generated by a process that requires a spherical map to be loaded in the AUX configuration. The contour points were then converted from the initial configuration-specific three-dimensional coordinates to a “border projection” file format. This allows contours to be viewed in relation to any surface configuration (e.g., the flat map in Figure 6 A) by expressing contour point locations in relation to the nearest tile (i.e., in “barycentric” coordinates 27). To generate the map of cortical geography shown in Figure 6A, the sulcal pattern was initially represented by a map of folding (mean curvature) computed for the fiducial surface but displayed on the flat map. Border contours were drawn around the perimeter of each sulcus as visualized on a flat map. Nodes enclosed by one or another closed sulcal boundary were automatically identified, and the assignments were stored in a paint file that contains up to five separate categorical types of information for each node. Figure 7 illustrates several additional Caret visualization options that facilitate comparisons between different surface configurations and different partitioning schemes. On the left (Figure 7 A and B) is the Felleman and Van Essen 14 areal partitioning scheme for visual areas; on the right (Figure 7 C and D) is a probabilistic atlas based on the Lewis and Van Essen areal partitioning scheme, 18,28 with a few specific areas identified (V2, V4, TE1–3). A blending option allows the visual areas and the geography map to be viewed concurrently. The probabilistic atlas was generated from architectonic maps of visual areas in five macaque hemispheres by deforming them to the macaque atlas (discussed below). In this scale figure, brightness reflects the fraction of cases that have the same areal identity at a given location; on the Caret screen, different hues are used to discriminate the individual areas. An option for identifying nodes using the mouse cursor (enabled using the “IDN” toggle button) provides a useful way to assess relationships between different configurations and to extract textual information about each node. Selecting a node in one configuration (e.g., the flat map) highlights that node when the view is switched to other configurations or to a different node coloring scheme (white squares and arrows in Figure 7A to D). In addition, the pop-up Status Message window (Figure 7E) shows many types of information about the selected node. This includes the node identity plus its spatial location in three different coordinate systems—three-dimensional stereotaxic coordinates (if the fiducial configuration is loaded in the REF configuration), Cartesian map coordinates (if a standard flat map is loaded in the AUX configuration), and latitude and longitude (if a latitude-longitude file is loaded). Additional information includes node attributes contained in the currently loaded paint file (in this example, assignments of cortical geography, lobe, functional area, and modality) and atlas file (in this case, the areal identity for each hemisphere contributing to the probabilistic atlas). Caret includes several data formats for visualizing and analyzing fMRI and other spatially complex patterns of neuroimaging or neuroanatomic data (cf. Figure 2). Data types that can be assigned to nodes and used to color the surface include RGB_paint and metric files, which are generated by SureFit when it maps fMRI data from an initial volume representation onto a surface. A file type that is particularly useful for surface-based deformations is the “activation file,” which is generated by applying an adjustable threshold to the data in a metric file. Cell files represent data related to the spatial distribution of labeled neurons or other discrete data. Cell files can be converted to representations of cell density to facilitate quantitative analyses of connectivity patterns. Foci files represent an alternative type of data related to the centers of PET or other neuroimaging data. Data points in foci files can be represented with associated uncertainty limits reflecting individual variability and spatial uncertainty in mapping data based on stereotaxic coordinates. 29Caret also contains options for generating, visualizing, and saving various types of information about surface geometry. These include representations of mean curvature (folding), intrinsic (Gaussian) curvature, distortion (AUX surface compared with REF surface), and geometric cross-overs (creases or topological handles in the surface). Modifying Surface Geometry Caret includes numerous options for modifying surface configuration (shape) or topology, or both, many of which were used in generating the data shown in Figures 2, 5, 6, and 7. These are grouped into six general classes—smoothing, geometric projections or transformations, contour-based reconstruction, automated flattening and distortion reduction, interactive surface editing, and surface-based deformation. - Smoothing. Smoothing operations include conventional smoothing (iteratively moving each node to the average of its neighbors), targeted smoothing (preferentially smoothing nodes that are highly distorted relative to the reference surface), and surface inflation (smoothing coupled with preferential expansion of nodes close to the center of gravity, to accelerate the smoothing of highly convoluted surfaces).
- Geometric transformations and projections. Options for geometric projection or transformation of surfaces include translation, scaling, rotation, and mirror-reflection; projection of a surface radially outward to a sphere or ellipsoid; and projection of a sphere to a plane (by a topology-preserving transformation if the sphere has a hole oriented along the positive z direction).
- Surface reconstruction. Contour-based surface reconstruction is used when the input data are a series of section contours rather than pre-existing surfaces generated by SureFit or other automated volume segmentation and surface reconstruction methods. Caret includes options for aligning individual section contours and generating surface reconstructions from the stack of aligned contours, using the Nuages algorithm.30
- Flattening. Automated flattening and distortion reduction involves two main stages of processing. In the first stage, the fiducial surface is reconfigured by smoothing or geometric projection steps to attain a spherical or flattened shape, depending on the desired geometry. Multi-resolution morphing is then applied to generate a minimally distorted flat map or spherical map. In both cases, the underlying algorithm10,11 involves resampling, downsampling, and morphing (iterative application of forces to each node to reduce distortions relative to the fiducial configuration). Surface resampling represents the initial reference and auxiliary configurations with a hexagonal array of nodes that are uniformly spaced in the auxiliary configuration. Once resampled, a surface can be downsampled (by deleting alternate nodes) to generate progressively coarser representations of the original shape. Multi-resolution morphing starts at the coarsest level of resampling, and is then reapplied after successive stages of upsampling to a finer resolution. The cycle is repeated until the overall distortions have reached an acceptably low level. When Caret is run on an SGI Octane, this takes several hours for a complete human hemisphere.
- Interactive editing. Interactive editing of surface geometry includes a number of options—drawing and applying cuts to the surface, deleting regions that have been disconnected by cuts, repositioning individual nodes, connecting or disconnecting selected node pairs, and processing the modified topology file to identify edge nodes and establish a consistent orientation of surface normals.
- Surface-based deformation. Options for surface-based deformation in Caret allow the map (the source) to be deformed to another (the target) while constrained by explicitly designated landmarks. The algorithm for deforming flat maps was developed by Joshi and Miller.15,21 Landmarks are drawn along what the user judges to be corresponding locations on the source and target maps. The landmark contours are resampled to establish corresponding numbers of landmark points on each source and target landmark contour. The landmarks are then used as constraints for a two-dimensional differeomorphic deformation algorithm. This algorithm computes solutions to a transport equation in which the landmark matching is constructed to minimize a running smoothness energy on the velocity field. Additional data (as a coordinate file, border file, or activation file) are carried passively with the deformation.
Spherical maps can be deformed using an algorithm developed by Bakircioglu et al. 17 The basic strategy is similar to that for flat deformations, but it entails using Laplacian differential operators constrained to the tangent space of the sphere and basis functions that are expressed as spherical harmonics. We have recently developed an improved method that is based on landmark-constrained smoothing and morphing of coordinates. As a practical matter, landmarks can be drawn on flat maps of the source and target hemisphere, where visualization is easiest, then projected to the corresponding spherical maps. Conversely, once deformations have been applied to the spherical maps, it is generally preferable to view the results on flat maps (cf. Figure 2 I and J). The architecture of SuMS (Surface Management System) comprises four main components: - A Java-based database structure designed specifically for cortical surface representations
- A high-capacity, secure data cache capable of storing large amounts of data
- A database manager (Sybase SQL Server) for coordinating the file cache and database access
- Search and retrieval capabilities using either a web browser (WebSuMS) or the downloadable SuMS Client software (a downloadable graphical user interface that runs on Java-compatible platforms).
For data entry it is currently necessary to use the SuMS Client. For most purposes WebSuMS is the easier to use, but it does not yet include all the capabilities of the SuMS Client. Data Entry Data entry in SuMS is a simple process in which the names of the volume or surface specification files intended for insertion are entered into the appropriate dialog box in the SuMS Client. For successful data entry, all files must be present in the directory locations listed in the specification file and the requisite metadata must be included in the appropriate file headers. Files generated using SureFit and Caret (v4.3 or higher) automatically include this information in the individual files or in the specification files. At the outset of the data entry process, the metadata for all files are checked for completeness, and prompts are given if required information is missing. A cross-checking process avoids duplicate insertion of files and ensures that only new data sets are added to SuMS. Data Search and Retrieval WebSuMS provides several convenient ways to identify and download files of interest, starting from the SuMS search page ( http://stp.wustl.edu/sums/sums.cgi), shown in Figure 8 A). One option is to enter the name of a particular volume specification file or surface specification file into the appropriate dialog box, if the name (or portion of the name) is known. Another is to search for particular files using one or more search criteria shown in the bottom half of the window. A third option is to select View Atlases near the top of the window. This brings up a list of four standard specification files for the macaque and human atlases (Figure 8B). Selection of the MACAQUE_ATLAS. spec brings up the list of files shown in Figure 8C. These can be downloaded as a group, then viewed in Caret after the files are uncompressed. An even simpler and faster way to retrieve files is to make a direct hyperlink connection to a particular specification file in the database, starting from a separate application such as an on journal article, PDF file, or e-mail message. For example, the data for the individual hemisphere illustrated in Figure 2 can be immediately accessed by a hyperlink connection to the appropriate specification file ( http://stp.wustl.edu/sums/sums.cgi?specfile=Demo.L.full.jamia.Fig2.spec). This directly links to a specification file containing the relevant data files. Selecting the “Download All” option downloads these files to the desired directory in a single step. The search process in the SuMS Client is similar, but is more flexible because it includes an intermediate repository. The first stage is to choose a combination of search criteria for identifying files of potential interest and to initiate the search process. The second stage is to view the results of the initial search and to select any or all of these files for placement in a search repository. The search repository is a listing of files provisionally targeted for downloading. The contents of the repository (akin to a “shopping cart”) can be expanded by repeating the search process using a different set of criteria. The third stage is to select the final set of files from the search repository and then to initiate the download process. Access Control SuMS uses a security model based on the designation of owners, readers, and writers for each file in the database. This ensures that private data are protected, public data are freely available, and any data can be made available to some investigators (e.g., collaborators) but not others. This security scheme is flexible enough for any owner to allow access to restricted data sets on an individual-user or group basis. Surface-based analyses have tremendous potential for enhancing progress in understanding cortical structure, function, and development in health and disease. If widely adopted, the software tools described here for computational cortical cartography, along with related tools developed in other laboratories, will aid in capitalizing on this potential. The ultimate measures of progress will, of course, be based on the actual scientific and medical discoveries that benefit from surface-based analyses. In addition, several more proximate measures will be of interest to monitor over the next several years. These include the pace with which the neuroscience community adopts cortical surface reconstructions as a standard way to analyze and display experimental results from individual subjects; adopts surface-based atlases and probabilistic representations as general strategies for bringing experimental data into register on a common substrate; adopts surface-based coordinates as a concise and objective primary metric for describing locations on the cortical surface, to complement the use of Talairach stereotaxic coordinates; and uses surface visualization software and databases of surface representations as a supplementary option for assessing and visualizing experimental data associated with published studies. SureFit, Caret, and SuMS have been under development in our laboratory for a number of years. However, they have only recently reached a state of maturity that allows semi-automated execution of a complete processing sequence, proceeding from primary structural and functional data to data sets that are represented on cortical flat maps, transformed to surface-based atlases, and stored in a database. Some of the functionality of the SureFit/Caret/SuMS suite is currently unique, but most of its capabilities are shared by one or more other brain-mapping software packages. These include FreeSurfer, 4,13 Brain Voyager, 5,31 and mrGray. 6 It is useful to discuss briefly the key issues involved in evaluating any of these packages, although detailed analysis and comparison are outside the scope of this discussion. Segmentation and Surface Reconstruction Two primary sets of issues are involved in evaluating different surface reconstruction methods: - How robust, easy to use, and accessible is the software for generating surfaces and mapping data onto surfaces?
- How accurate are the surface representations and how accurately are experimental data mapped onto the surfaces?
Structural image data obtained in neuroimaging and neuroanatomic studies vary widely in overall quality. This is attributable to differences in image contrast, noise, and regional inhomogeneities arising from the particular methods or devices used for image acquisition and from the individual subjects under study. Accordingly, each method of segmentation should be tested for its robustness and accuracy, using data sets that span a wide range in quality. This has yet to be done systematically, because of the newness of the methods, the inherent complexity of the data, and the lack of generally accepted standards for evaluation. One important metric is whether the segmentation/surface is topologically correct. Topological errors (handles) are particularly deleterious for flattening and subsequent processing stages, unless they are small and outside the main region of experimental interest. Another important metric is spatial accuracy, i.e., how close the surface runs to the desired target boundary. This is especially important if the objectives of the experimental analysis are sensitive to modest but systematic biases or to less frequent but larger deviations from the desired trajectory. However, assessment of spatial accuracy is inherently problematic, because a “ground truth” or “gold standard” that precisely represents the target layer is generally not available for structural MRI or cryosectional image data. A reasonable alternative strategy is for one or more expert anatomists to draw contours along the target layer in a number of selected regions in which the trajectory can be confidently estimated. The distance from each point on the target contour to the nearest point on a fiducial surface reconstruction can then be determined and displayed as a histogram, and the distribution of errors can be expressed by measures such as the error and standard deviation. If those who draw the target contours are unaware of the results of any particular segmentation, this approach can serve as an objective basis for evaluating and comparing the performance of different segmentation methods. Surface Manipulation: Flattening and Deformations A number of methods are currently available for generating cortical flat maps and spherical maps from fiducial surface representations. Given that the fiducial surface contains a complex pattern of intrinsic (Gaussian) curvature, 29 significant areal distortions are inevitable on any flat map or spherical map representation. Nonetheless, it is desirable to minimize these distortions and to quantify the magnitude and distribution of residual distortions. Residual distortions can affect surface-based analyses in two major ways. First, they affect visual impressions about the relative sizes of different regions and intracortical distances between regions. This is analogous to how distortions on earth maps affect impressions about the relative sizes of different continents (e.g., Greenland vs. South America). Fortunately, compensation for such perceptual biases can be largely made by analyzing and reporting surface areas and geodesic distances determined on the fiducial reconstruction. Another problem is that surface-based warping from one hemisphere to another can be affected by distortions, especially if the pattern of distortions differs on the source and target maps. This puts a premium on minimizing distortions on whichever surface configuration (sphere or flat map) is used as the substrate for surface-based deformations. Histograms of areal distortion for different surface nodes provide an objective basis for comparing different flattening algorithms. An inherent advantage of surface-based deformations is that the process respects the topology of the cortical sheet in compensating for individual variability. For this reason, surface-based deformations should in principle achieve substantially better registration than volume-based deformations, even if the volumetric approach uses a very high-dimensional parameter space to constrain the deformation. This prediction is supported by initial comparisons of surface-based vs. volume-based approaches to the registration problem. 16,20 However, the issue is of sufficient general importance that more extensive quantitative comparisons are strongly warranted. As progressively more data are mapped to surface-based atlases, it will be feasible to generate an increasingly diverse and rich set of probabilistic representations. One type of representation, relatively close to the primary data, involves probabilistic maps of fMRI activation patterns from multiple subjects carrying out similar or identical behavioral tasks 16 (see Figure 2 above). Given the enormous diversity of behavioral tasks that have been or will be used on multiple subjects in fMRI studies, the number of such probabilistic maps is effectively open-ended. Another type of representation, involving greater abstraction from the data, involves probabilistic maps of cortical areas whose boundaries are estimated from functional, architectonic, or other types of experimental data (cf. Van Essen et al. 18). To the extent that most or all cerebral cortex is in fact divisible into genuine, well-defined subdivisions, it might be hoped that probabilistic maps of cortical areas would converge toward a single, consensual representation. However, a more likely scenario is that numerous alternative partitioning schemes will remain in widespread use. This will lead to a multiplicity of probabilistic maps based on different schemes, different criteria used to deform individual data sets to a surface-based atlas, or different substrates used as the target surface-based atlas. These considerations emphasize the pressing need for continued progress in several aspects of computerized cortical cartography. One need is to develop improved methods for evaluating the quality of registration of individual data sets to an atlas, on the basis of objective measures that can be applied to a diversity of data types. Another need is to improve the methods for surface-based deformation. For example, it is desirable to have hybrid methods that combine aspects of the landmark-based approach described here with approaches based on a continuous valued representation as used by Fischl et al. 16 A third need is to enhance interoperability, so that surface-based analyses for any given data set can easily migrate from one software suite to another. This will facilitate comparisons among data sets obtained in different laboratories and will allow a more extensive sets of analyses to be carried out on data sets of particularly broad interest. A fourth need is for ongoing enhancements in the database infrastructure, for coping with the flood of surface-related data. We believe that the SuMS database described here has considerable promise as an initial effort in this direction. If the concept is indeed successful, however, it will need to be scaled up by several orders of magnitude to cope with the estimated 100,000 cortical surface reconstructions per year that may emerge in this decade from brain-mapping efforts in basic and clinical laboratories around the world. 22 The advantages of bringing these data under the umbrella of one or more databases will become increasingly evident, just as it has in such other scientific arenas as genomics and protein structure. 32–34 This will impose substantial pressures for increased data capacity, network speeds, and a richer array of search capabilities. The authors thank the many beta testers at Washington University and elsewhere whose bug reports and constructive suggestions have greatly helped to improve Caret, SureFit, and SuMS. 1. Daniel PM, Whitteridge D. The representation of the visual field on the cerebral cortex in monkeys. J Physiol (Paris). 1961;159:203–21. 2. Gattass R, Gross CG. Visual topography of striate projection zone (MT) in posterior superior temporal sulcus of the macaque. J Neurophysiol. 1981;46:621–38. [PubMed] 3. Van Essen DC, Drury HA, Anderson CH. An automated method for reconstructing complex surfaces, including the cerebral cortex. Soc Neurosci Abstr. 1999;25:1929. 4. Dale AM, Fischl B, Sereno MI. Cortical surface-based analysis, part I: Segmentation and surface reconstruction. NeuroImage. 1999;9:179–94. [PubMed] 5. Goebel R, Khorram-Sefat D, Muckli L, Hacker H, Singer W. Functional imaging of mirror and inverse reading reveals separate coactivated networks for oculomotion and spatial transformations. NeuroReport. 1998;9:713–9. [PubMed] 6. Wandell BA, Chial S, Backus BT. Visualization and measurement of the cortical surface. J Cogn Neurosci. 2000;12:739–52. [PubMed] 7. Van Essen DC, Zeki SM. The topographic organization of rhesus monkey prestriate cortex. J Physiol. 1978;277:193–226. [PubMed] 8. Van Essen DC, Maunsell JHR. Two-dimensional maps of the cerebral cortex. J Comp Neuro.l 1980;191:255–81. 9. Schwartz EL, Shaw A, Wolfson E. A numerical solution to the generalized mapmaker's problem: flattening nonconvex polyhedral surfaces. IEEE Trans Pattern Anal Machine Intell. 1989;11:1005–8. 10. Carman GJ, Drury HA, Van Essen DC. Computational methods for reconstructing and unfolding the cerebral cortex. Cerebl Cortex. 1995;5:506–17. 11. Drury HA, Van Essen DC, Anderson CH, Lee CW, Coogan TA, Lewis JW. Computerized mappings of the cerebral cortex: a multiresolution flattening method and a surface-based coordinate system. J Cogn Neurosci. 1996;8:1–28. [PubMed] 12. Teo PC, Sapiro G, Wandell BA. Creating connected representations of cortical matter for functional MRI visualization. IEEE Trans Med Imag. 1997;16:852–63. 13. Fischl B, Sereno MI, Dale AM. Cortical surface-based analysis, part II: Inflation, flattening, a surface-based coordinate system. NeuroImage. 1999;9:195–207. [PubMed] 14. Felleman DJ, Van Essen DC. Distributed hierarchical processing in primate cerebral cortex. Cereb Cortex. 1991;1:1–47. [PubMed] 15. Joshi S. Large deformation diffeomorphisms and Gaussian random fields for statistical characterization of brain submanifolds [doctoral thesis]. St. Louis, Mo.: Department of Electrical Engineering, Sever Institute of Technology, Washington University, 1998. 16. Fischl B, Sereno MI, Tootell RB, Dale AM. High-resolution intersubject averaging and a coordinate system for the cortical surface. Hum Brain Mapp. 1999;8:272–84. [PubMed] 17. Bakircioglu M, Joshi S, Miller MI. Landmark matching on the sphere via large deformation diffeomorphisms. Proc SPIE Med Imaging Image Processing. 1999;3661:710–5. 18. Van Essen DC, Lewis JW, Drury HA, et al. Mapping visual cortex in monkeys and humans using surface-based atlases. Vision Res. 2001;41:1359–78. [PubMed] 19. Drury HA, Van Essen DC, Joshi SC, Miller MI. Analysis and comparison of areal partitioning schemes using two-dimensional fluid deformations. NeuroImage. 1996;3:S130. 20. Drury HA, Van Essen DC, Corbetta M, Snyder AZ. (1999) Surface-based analyses of the human cerebral cortex. In: Toga A (ed). Brain Warping. San Diego, Calif.: Academic Press, 1999:337–63. 21. Van Essen DC, Drury HA, Joshi S, Miller MI. Functional and structural mapping of human cerebral cortex: solutions are in the surfaces. Proc Natl Acad Sci USA. 1998;95:788–95. [PubMed] 22. Dickson J, Drury H, Van Essen DC. Surface management system (SuMS): a surface-based database to aid cortical surface reconstruction, visualization and analysis. Phil Trans Royal Soc Ser B. 2001, in press. 23. Corbetta M, Akbudak E, Conturo TE, et al. A common network of functional areas for attention and eye movements. Neuron. 1998;21:761–73. [PubMed] 24. Lorensen W, C H. Marching cubes: a high-resolution 3-D surface reconstruciton algorithm. Comput Graph. 1987;21:163–9. 25. Lee T-C, Kashyap R, Chu C-N. Building skeleton models via 3-D medial surface/axis thinning algorithms. Graph Models Image Processing. 1994;56:462–78. 26. Schroeder W, Martin K, Lorensen B. The Visualization Tookit: An Object-oriented Approach to 3D Graphics. 2nd ed. Saddle River, NJ: Prentice Hall, 1997. 27. Farin G, Hanford D. The Geometry Toolbox for Graphics and Modeling. Natick, Mass.: AK Peters, 1998. 28. Lewis JW, Van Essen DC. Architectonic parcellation of parieto-occipital cortex and interconnected cortical regions in the Macaque monkey. J Comp Neurol. 2000;428:79–111. [PubMed] 29. Van Essen DC, Drury HA. Structural and functional analyses of human cerebral cortex using a surface-based atlas. J Neurosci.1997;17:7079–102. [PubMed] 30. Geiger B. Three-dimensional Modeling of Human Organs and Its Application to Diagnosis and Surgical Planning. Institute National de Recherche Informatique et Automatique, 1993. Technical Report 2105. 31. Goebel R. A fast, automated method for flattening cortical surfaces. NeuroImage. 2000;11:680. 32. Berman HM, Westbrook J, Feng Z, et al. The Protein Data Bank. Nucl Acids Res. 2000;28:235–42. [PubMed] 33. Marshall E. Rival genome sequencers celebrate a milestone together. Science. 2000;228:2294–5. 34. Pennisi E. Finally, the Book of Life and Instructions for Navigating it. Science. 2000;228:2304–7.
|
This article has been 
based Analysis
![[triangle]](/corehtml/pmc/pmcents/rtrif.gif)
