an integrated software suite for surface-based analyses of...

The cerebral cortex is the dominant structure of themammalian brain and is responsible for an impres-sively diverse range of sensory, motor, and cognitivefunctions. Anatomically, the cortex is a sheet-likestructure whose surface area greatly exceeds the sur-face area of a smooth, solid shape containing it. Inlarge-brained mammals, including humans, the cor-tex is extensively folded, and the pattern of convolu-tions varies considerably from one individual to thenext. The irregularity and variability of these convo-lutions pose major challenges in analyzing and visu-alizing cortical structure, function, and development.

These problems are particularly acute in view of theexplosion of high-resolution data derived from manydifferent experimental methods, particularly non-invasive neuroimaging methods such as structuraland functional magnetic resonance imaging (MRI)applied to human beings and nonhuman primates.

Computational cortical cartography represents a pow-erful general approach to dealing with these problemsby using surface-based visualization and analysismethods. The essence of the approach is to representthe cortex by explicit surface reconstructions ontowhich various types of experimental information aremapped. The advantages of surface reconstructionscan be grouped into four main categories:

■ Visualization using multiple configurations. Once gen-erated, surface reconstructions can be manipulatedin shape to improve visualization. Commonly usedconfigurations, besides the initial (fiducial) three-dimensional shape of the cortex, include inflated(extensively smoothed) surfaces, spherical surfaces(used for surface-based coordinates, as discussedbelow), and flat maps, which allow the entire hemi-sphere to be viewed with only modest distortions,albeit at the price of artificial cuts (akin to thoseused in maps of the earth’s surface).

Affiliation of the authors: Washington University School ofMedicine, St. Louis, Missouri.This work was supported by Human Brain Project grant R01MH60974-06, funded jointly by the National Institute of MentalHealth, National Science Foundation, National Cancer Institute,National Library of Medicine, and the National Aeronautics andSpace Administration; by grant EY02091 from National EyeInstitute, and by The National Partnership for AdvancedComputational Infrastructure.Correspondence and reprints: David C. Van Essen, PhD,Department of Anatomy and Neurobiology, WashingtonUniversity School of Medicine, 660 Euclid Avenue, St. Louis, MO63110; e-mail: <[email protected]>.Received for publication: 3/16/01; accepted for publication:5/8/01.

443

Application of Information Technology ■

An Integrated Software Suitefor Surface-based Analyses of Cerebral Cortex

A b s t r a c t The authors describe and illustrate an integrated trio of software programs forcarrying out surface-based analyses of cerebral cortex. The first component of this trio, SureFit(Surface Reconstruction by Filtering and Intensity Transformations), is used primarily for corticalsegmentation, volume visualization, surface generation, and the mapping of functional neuroimaging data onto surfaces. The second component, Caret (Computerized AnatomicalReconstruction and Editing Tool Kit), provides a wide range of surface visualization and analysisoptions as well as capabilities for surface flattening, surface-based deformation, and other surfacemanipulations. The third component, SuMS (Surface Management System), is a database and associated user interface for surface-related data. It provides for efficient insertion, searching, and extraction of surface and volume data from the database.

■ J Am Med Inform Assoc. 2001;8:443–459.

Journal of the American Medical Informatics Association Volume 8 Number 5 Sep / Oct 2001

DAVID C. VAN ESSEN, PHD, HEATHER A. DRURY, MS, JAMES DICKSON, MS, JOHN HARWELL, MS, DONNA HANLON, MS, CHARLES H. ANDERSON, PHD

■ Localization using surface-based coordinates. It is oftenimportant to be precise about exact locations in thecortex. Surface-based coordinates (latitude and lon-gitude on a sphere) provide a concise, precise, andobjective metric that respects the topology of thecortical surface. In this respect, they offer importantadvantages over standard stereotaxic coordinatesor geographically referenced descriptions.

■ Compensation for individual variability. Comparisonof results obtained from different individuals isinherently challenging because of the combinationof geographic differences (in the pattern of fold-ing) and functional differences in the size of eachcortical area (which can vary two-fold or more)and in their shape and location. Surface-basedwarping provides a natural and powerful way tocompensate for these differences while respectingthe topology of the cortical sheet.

■ Consolidation onto surface-based atlases. With theexplosion of information about cortical structureand function, surface-based atlases will becomeincreasingly important, just as atlases of theearth’s surface are invaluable repositories of infor-mation about countless aspects of geography.Information from many different individuals canbe brought into a common reference frame.

The many steps involved in surface-based corticalanalysis can be subdivided into four main stages,each dealing with distinct types of data and havingdifferent analysis objectives. Historically, the meth-ods used to carry out each of these processing stagesinitially involved manual or other noncomputerizedmethods that have only recently been supplanted byautomated or semi-automated methods of computer-ized neuroanatomy.

■ Acquisition of structural data. Surface-based analy-sis starts with a source of structural informationthat can be used to infer the shape of the corticalsheet. Until the 1990s, surface reconstructionswere generally based on postmortem histology,using photographs of histologic sections. The situ-ation has changed dramatically with the advent ofnoninvasive imaging methods (particularly struc-tural MRI) that can routinely be used to image thestructure of cerebral cortex in human beings aswell as laboratory animals.

■ Segmentation and surface reconstruction. Generatingaccurate surface reconstructions of cerebral cortexis a challenging problem in general, because thecortex is highly complex in shape and because theimage data on which reconstructions are based aretypically noisy or contain artifactual irregularities.

The earliest explicit surface representations weregenerated by physical models.1,2 A variety ofmethods are now available that allow cortical seg-mentation and surface reconstruction with reason-able accuracy and robustness (see below).3–6

■ Surface reconfiguration and flattening. The earliestapproaches to reconfiguring cortical shape in-volved generating cortical flat maps by manualmethods, such as tracing contours with a penciland tracing paper.7,8 A variety of methods are nowavailable for inflating the cortex and for makingcomputer-generated flat maps using various met-rics for minimizing distortions.4,9–13

■ Mapping to a surface-based atlas. Early surface-basedatlases were based on manually generated flatmaps to which data were transposed manually, onthe basis of geographic landmarks and approxi-mate distances.8,14 The emergence of surface-baseddeformation algorithms15–17 has allowed an objec-tive approach to bringing data into register withan atlas while respecting the topology of the corti-cal surface.16,18–19

Although the use of surface reconstructions hasincreased rapidly in recent years, enormous potentialremains untapped in terms of ongoing neuroimagingand systems neuroscience studies that do not yet cap-italize on the power of surface-based analyses. This islargely because key elements of the enabling technol-ogy have only recently become available.

The contribution of our laboratory to this effortinvolves the development of an integrated softwaresuite for surface-based analyses of cerebral cortex.Our objective has been to provide software that isfreely available to the neuroscience community, runson multiple hardware platforms, and can be used tocarry out the major stages of surface reconstructionand analysis efficiently and in as automated a man-ner as possible.

The integrated system has three main software com-ponents. The first component, SureFit (SurfaceReconstruction by Filtering and Intensity Trans-formations), is used primarily for cortical segmenta-tion, volume visualization, and initial surface genera-tion. The second component, Caret (ComputerizedAnatomical Reconstruction and Editing Tool Kit), pro-vides a wide range of surface visualization and analy-sis options as well as capabilities for surface flattening,surface-based deformation, and a number of other sur-face manipulations. The third component, SuMS(Surface Management System), is a database and asso-ciated user interface for dealing efficiently with theincreasing number of complex data sets associated

VAN ESSEN ET AL., Cortical Surface-based Analysis Software444

with surface-based analyses. It provides an organizedframework for efficiently inserting, searching, andextracting surface and volume data from a database.

Here we provide a general introduction to andoverview of the surface-based analyses that can becarried out using SureFit, Caret, and SuMS. Keyalgorithmic principles underlying the main process-ing steps are described, and selected examples areused to illustrate the methods and the utility of theoverall approach. Additional information is avail-able in the user’s guide to SureFit and Caret (http://stp.wustl.edu/resources/cortcart.html), in the on-line help menu for Caret (http://stp.wustl.edu/caret/html4.3/reference_manual_toc.html), and inrelated publications.11,20–22

Overview of Processing Stages in Surface-based Analysis

Surface Reconstruction

Some of the key stages of surface-based analyses areshown in a generic flow chart (Figure 1) and in anillustrative structural and functional MRI (fMRI) dataset processed in SureFit and Caret (Figure 2). Foreach experimental hemisphere under investigation, akey initial objective is to obtain a fiducial surfacereconstruction—one that represents the shape of thecortex as accurately as possible.

The strategy for achieving this objective depends onthe nature of the primary structural data. If structur-al MRI or cryosection data are available, the pre-ferred strategy is to use a volume-based method suchas SureFit to carry out automated segmentation andsurface reconstruction (Figure 1, left side) . The seg-mentation process, applied to structural MRI imagedata of the type shown in Figure 2A, results in a seg-mentation (a binary volume) whose boundary repre-sents the shape of the cortex, as in the slice shown inFigure 2B. This segmentation (or “reconstructionsubstrate”) is used to automatically generate anexplicit surface representation, i.e., a wire-frame tes-sellation whose nodes lie on the boundary of the seg-mentation (typically 50,000 to100,000 nodes for ahuman hemisphere) and whose surface topology isdefined by the links between nodes. The initial (raw)surface has a blocky appearance associated with thecubical voxels (typically 1 mm3) of the segmentation.Hence, it is slightly smoothed to generate a satisfac-tory fiducial surface (Figure 2C).

If only histologic sections are available, surfaces canbe reconstructed by an alternative contour-based

reconstruction strategy, using options available inCaret (Figure 1, right side). This entails drawing con-tours along a particular cortical layer (e.g., layer 4) inindividual sections, bringing contours from adjacentsections into register, and generating an explicit sur-face reconstruction using an automated tessellationmethod. Although the method for surface recon-struction is different, the outcome of segmentation-based vs. contour-based processes is fundamentallythe same, that is, an explicit surface that representsthe shape of the convoluted cortex.

Visualizing Experimental Data

Surface reconstructions are invaluable for visualizingmany types of experimental data, ranging from min-imally processed representations of primary data tohighly abstracted representations. One particularlyimportant class of experimental data involves func-tional activation patterns obtained using fMRI. Forexample, Figure 2D shows an fMRI activation focusin a coronal slice through the intraparietal sulcus

445Journal of the American Medical Informatics Association Volume 8 Number 5 Sep / Oct 2001

F i g u r e 1 Processing stages in surface-based analysis.The upper portion shows the primary stages involved ingenerating an initial (“raw”) surface reconstruction from aprimary source of structural data. Entries on the upper leftindicate stages associated with extracting a cortical seg-mentation from volumetric structural data. Entries on theupper right indicate analogous stages associated withextracting and aligning section contours to represent corti-cal shape. The lower half shows transformations from theraw cortical surface to various alternative configurationsof the same surface as well as deformations to a surface-based atlas.

(overlaid on the structural MRI slice), generated in abehavioral paradigm involving saccadic eye move-ments.23 These data were then mapped from the vol-ume onto a surface representation using a SureFitalgorithm that takes local surface orientation intoaccount when mapping volumetric fMRI data ontothe surface.

Surface Reconfiguration

Once a fiducial surface has been generated, its shapecan be modified into several alternative configura-tions that are widely used in surface-based analysis.Standard surface configurations include inflatedmaps (Figure 2F), spherical maps (Figure 2G), andflat maps (Figure 2H). These exemplar surfaces arepainted with a combination of functional and geo-graphic data, in which darker shading indicatesburied cortex; fMRI data are represented in the fig-ure by white or light gray for surface nodes above athreshold activation level.

Each alternative configuration has advantages forvisualization and analysis. The inflated configurationlooks like a lissencephalic hemisphere and gives an

intuitive sense of geographic location. The sphericalconfiguration (Figure 2G) allows for determination ofspherical coordinates (latitude and longitude), whichconstitute a precise and objective localization metricthat respects surface topology. The flat map shows theentire pattern in a single view (Figure 2H), with dis-tortions reduced by artificial cuts like those used onflat maps of the earth’s surface.

Notice that the polar coordinates, while generated onthe sphere, can be readily viewed and interpreted afterprojection to the flat map. All the surface maps, butparticularly the flat map, reveal that the behavioraltask (saccadic eye movements) was associated withnumerous activation foci that are more efficiently andaccurately visualized on a single surface than in aseries of volume slices.

Surface-based Atlases

The final stage of surface-based analysis illustrated inFigures 1 and 2 involves mapping data from an indi-vidual hemisphere to a surface-based atlas. To ensurea mapping that respects the topology of the corticalsurface, it is desirable to use a surface-based defor-


F i g u r e 2 Illustrative data set takenthrough the processing sequence illus-trated in Figure 1. A, Coronal slicethrough a structural MRI volume ofhuman left hemisphere. B, Cortical seg-mentation through the same coronalslice, generated using the SureFitmethod. C, The fiducial surface generat-ed from this segmentation. D, Fun-ctional MRI (fMRI) data overlaid on thesame structural MRI data shown in A,generated in a behavioral paradigminvolving eye movements.23 E, ThefMRI data painted on the cortical sur-face and displayed in white and lightgray shades. F, The same fMRI data dis-played on an inflated surface, with cor-tical geography (sulcal regions) shownin darker shades. G, Spherical map thatshows fMRI data, cortical geography,and latitude-longitude isocontours. H,The same data shown on a cortical flatmap. I, Spherical map of the VisibleMan atlas, with fMRI data deformed tothe atlas. J,.Flat map of the Visible Manatlas that includes cortical geography,deformed fMRI data projected from thesphere to the flat map, and boundariesof Brodmann’s architectonic areas asmapped by Drury et al.20 The data forthe individual case can be downloadedfrom SuMS via a hyperlink connectionto http://stp.wustl.edu/sums/sums.cgi?specfile=Demo.L.full.jamia.Fig2.spec.

mation algorithm applied to spherical maps, whichare unaffected by cuts in the surface. In Figure 2, thesource hemisphere (G) was deformed to the targethemisphere (I) using geographically defined land-marks in the source and target maps and a differeo-morphic deformation algorithm developed byBakircioglu et al.17 In Figure 2J, the results of thedeformation are shown by projecting the data onto aflat map of the surface-based atlas. This allows thedata from a single subject to be viewed in relation toany other data registered with the atlas, such theboundaries of Brodmann’s architectonic subdivisionsshown on the flat map.20

Database Entry and Retrieval

Each of the display formats and the ancillary datashown in Figure 2 are useful for specific visualizationand analysis options. Consequently, it is desirable tobe able to access any or all of the data in an efficient,organized way, even though the data comprise morethan a dozen files that represent a variety of volumedata, surface geometry data, and ancillary experi-mental data. This access was achieved by storing thedata in the SuMS database, which provides severaloptions for searching and retrieving data. Data entryand retrieval from SuMS as well as data visualizationin SureFit and Caret are greatly facilitated by group-

ing common families of files into organized sets thatare listed in “specification files.” By selecting theappropriate volume or surface specification file (orfiles) of interest, a family of related files can be down-loaded as a group. An especially easy and direct wayto do this is by direct hyperlink connections to par-ticular specification files in SuMS, as exemplified bythe hyperlinks specified in the legend to Figure 2.

We now describe SureFit, Caret, and SuMS each ingreater detail, to illustrate their functionality and thealgorithms on which they are based.

SureFit

Visualization

SureFit includes general-purpose capabilities for vol-ume visualization (viewing one or two volumes con-currently) and surface visualization (viewing up tothree surfaces concurrently). For example, Figure 3Ashows the main SureFit window, with a coronal slicethrough a cortical segmentation overlaid on thestructural MRI volume. The overlay option allowsthe relationship between the segmentation (solidshading; red in the actual view) and the intensitydata to be compared directly. Two surface viewingwindows are shown, one with the fiducial surface


F i g u r e 3 Volume and surfacevisualization in SureFit. A, Themain SureFit window, which canbe used as a general volumetricslice viewer and in connectionwith the SureFit segmentationprocess. B, Fiducial surface re-construction displayed in theSureFit surface viewer. C, In-flated cortical surface displayedsimultaneously in a second sur-face viewer.

(Figure 3B) and the other with an inflated surface(Figure 3C). The spatial relationships among differ-ent volume and surface representations can be visu-alized using a “pick” option, which highlights pointsin each of the surface views (arrows) that correspondto the cross-hairs in the slice viewer.

Segmentation and Surface Reconstruction

The main functionality of SureFit is to generate seg-mentations and surface reconstructions of cerebralcortex (like those in Figure 3) from structural MRI orother grayscale image data. This involves an auto-mated sequence of image-processing operations andother functions, with only two stages (volume prepa-ration and interactive error correction) that entail sig-nificant user interaction.

Volume Preparation

Prior to launching the automated segmentationprocess, several preparatory steps are carried out.These include:

■ Tagging each data set with appropriate generalinformation (e.g., case name, region, and specificcomments)

■ Orienting the volume and cropping it to thedesired spatial extent

■ Identifying the anterior commissure (AC) as a keygeographic landmark

■ Setting two parameters (the gray matter and whitematter peaks in the intensity histogram) that areneeded for automatic segmentation.

Automated Segmentation

Automated segmentation in SureFit is based on sev-eral structural characteristics of cerebral cortex thatare schematized in Figure 4. The cortex is a sheet-liketissue, approximately constant in thickness, adjoinedon its inner side by subcortical white matter. On itsouter side, the pial surface adjoins cerebrospinal fluidin gyral regions, but in sulcal regions the cere-brospinal fluid gap between apposed pial surfacesmay be very narrow. SureFit aims for a segmentationwhose boundary runs approximately midwaythrough the cortical thickness (i.e., cortical layer 4),because this represents the associated cortical vol-ume more accurately than do segmentations runningalong either the inner or outer boundary.4 To thisend, SureFit generates probabilistic representationsfor both the inner boundary and the outer boundary,using cues related to image intensity, intensity gradi-ents, and the spatial relationships between the innerand outer boundaries. One general strategy is topostpone deterministic (yes/no) decisions that createbinary volumes as long as possible, until variousprobabilistic representations have been generatedand appropriately combined.

Key stages in the SureFit segmentation process areillustrated in Figure 5 for a slice through lateral tem-poral cortex of a structural MRI volume (Figure 5A).This case was chosen because the image data are rela-tively noisy and low in contrast, making automatedsegmentation an inherently challenging process. Theintensity histogram (Figure 5B) for this volume showsa characteristic pair of intensity peaks, one associatedwith white matter and the other with gray matter.

Along the inner (gray–white) boundary, the imageintensity should be approximately midway betweenthe peaks for gray matter and white matter. This inten-sity-based cue is made explicit using a Gaussian inten-sity transformation, whose peak represents the mostlikely inner boundary intensity (inner boundary arrowin Figure 5B) and whose standard deviation reflectsthe noisiness in the image data (see legend for details).The resultant inner boundary intensity transformationhas local maxima running approximately along theinner boundary, but there are many irregularitiesowing to fluctuations in the image data. Other cues forthe inner boundary are related to the intensity gradi-ent, whose magnitude is shown in Figure 5D. The gra-dient along the inner boundary is steeper than in


F i g u r e 4 Schematic representation of key structuralfeatures of cerebral cortex that are relevant to the SureFitsegmentation algorithm. The cortical sheet is approximate-ly constant in thickness and adjoins white matter on itsinner boundary. Its outer (pial) boundary adjoins cere-brospinal fluid in gyral regions and oppositely orientedcortical sheet in sulcal regions.

neighboring gray matter and white matter, but it ismore shallow than in other regions (e.g., along parts ofthe outer boundary). This information, along withcues related to the direction of the intensity gradient,is combined with the intensity-based map in Figure 5Cto generate a composite inner boundary map (Figure5E) that is more reliable than the individual contribut-ing maps (see legend to Figure 5 for details).

For the outer (pial) boundary, one cue is based onimage intensity (Figure 5F), using a Gaussian intensi-ty transformation centered at the outer boundaryarrow in Figure 5B. This intensity-based map hasprominent local maxima along gyral boundaries,where gray matter adjoins cerebrospinal fluid. In sul-cal regions the local maxima are less pronounced,because a prominent cerebrospinal fluid gap is gen-erally lacking. Instead, evidence for the outer bound-ary in sulcal regions is collected using a customizedfiltering process whose output is high along regionsmidway between two oppositely oriented innerboundaries (Figure 5G). The composite outer bound-ary map (Figure 5H) reflects a combination of thecues in Figures 5F and 5G and gradient-based cuesanalogous those used for the inner boundary.

In addition to the probabilistic inner and outer bound-ary maps, a segmented representation of cerebralwhite matter is generated (Figure 5I). This entailsthresholding the intensity volume at a level that fillsmost of the white matter, disconnecting noncerebralstructures (brainstem, cerebellum, skull, and eye),transecting midline structures, and removing theextraneous regions by a flood-filling process. Thebrainstem is disconnected using a diagonal transectionapplied near the pons (based on its location relative tothe anterior commissure). The eye and skull are dis-connected by automatically identifying and removingthe high-intensity (fatty) region behind the eyeball.

In Figure 5J, the maps of inner and outer boundariesare combined to generate a smooth map of positionalong the radial axis (i.e., the axis delineated byarrows in Figure 4). This involves taking the differ-ence between blurred versions of the inner and outerboundary maps (and also assigning maximal intensi-ty to regions near the core of cerebral white matter).The resultant radial position map is white in the inte-rior, black on the exterior, and smoothly graded inintensity across the cortical thickness. Thresholdingthe radial position map at an intermediate intensitylevel generates a segmented volume (Figure 5K)whose boundary tends to run midway through thecortical thickness, as can best be appreciated by view-ing the segmentation superimposed over the struc-

tural MRI intensity volume (Figure 5L). The initialsegmentation is used as the substrate for generatingan explicit surface reconstruction.24

Error Detection and Correction

Topological errors (“handles”) in the initial segmen-tation are typically attributable to noise, large bloodvessels, or regional inhomogeneities in the structuralMRI volume, or a combination of these. Errors arelocalized by inflating the initial surface reconstruc-tion to a highly smoothed ellipsoidal shape and usingthe orientation of surface normals to identify “cross-over” regions where the surface is folded over onitself. Clusters of surface nodes associated with cross-overs are mapped from the fiducial surface into cor-responding voxel clusters in the volume. If the erroris a bridge across opposite banks of a sulcus (an “exo-handle”), correction entails removing voxels from thebridge. If, instead, the error is a hole in the white mat-ter between two sulci, an “endohandle,” correctionentails adding voxels to the hole. The automatederror correction process tests for both exohandles andendohandles in the vicinity of each location deter-mined to contain an error.

The localized patches used for these tests conform tothe shape of temporary segmentations that are basedon different threshold levels for the radial positionmap. If the trial patch reduces the number of topo-logical handles in the segmentation, as determinedfrom an Euler count applied to the volume,25 it isaccepted as a permanent correction and the processmoves on to the next error patch. The automatederror correction process sometimes fails, especiallyfor handles that are notably large or irregular.Residual errors can be removed by interactive edit-ing, which allows voxels to be added or removed oneat a time or in small clusters using dilation or erosionsteps within small masked regions.

The processing time needed for automated segmen-tation in SureFit is less than an hour for the initialsegmentation plus several hours for automated errorcorrection when run on an SGI Octane. Processingtimes can be significantly faster on PC systems run-ning Linux.

Mapping Geographic and Functional Data

Once a satisfactory segmentation is achieved, regionsof buried cortex are automatically identified. Thisentails a combination of dilation, erosion, and vol-ume subtraction operations to generate a volume thatincludes sulcal but not gyral cortex. Nodes of thereconstructed surface that intersect this sulcus-only


volume are identified as buried cortex and are shad-ed to provide a representation of sulcal versus gyralgeography (Figure 2E to J).

The process of mapping fMRI data onto surfacereconstructions takes advantage of information aboutlocal surface orientation. For each surface node, thefMRI image volume is convolved with an ellipsoidalfilter centered on that node and oriented parallel tothe local surface tangent. The output values are con-verted to RGB color values according to a predesig-nated color look-up table and are stored in an“RGB_paint” file format for viewing in SureFit orCaret. In addition, the outputs are stored as scalarvalues in a “metric file” format for subsequent visu-alization and analysis in Caret.

Volume and Surface Specification Files

SureFit generates volume and surface specificationfiles at the end of the automated segmentationprocess. The volume specification file lists the struc-tural MRI volume, the segmentation files, and sever-al key intermediate files. The surface specification fileincludes both VTK file format (Visualization ToolKit26) and the Caret geometry file format (in whichnode coordinates and node topology are stored inseparate files) as well as paint files that represent cor-tical geography. This facilitates data entry into SuMSas well as visualization and surface-based analyses inCaret.

Caret

Surface Visualization

Caret includes numerous options for surface visuali-zation, manipulation, and other surface-based analy-ses. We illustrate some of these visualization capabil-ities using a surface-based atlas for the macaquemonkey as an exemplar data set.

The main Caret screen (Figure 6A) includes a surfacevisualization window as well as menu options alongthe top and a status display row along the bottom. Theinitial step in loading data is to select an appropriatesurface specification file (using the “OpenSpecification File” option in the File submenu shownin Figure 6A). This brings up a dialog box (Figure 6B)for choosing the subset of data files to be loaded.Surface geometry is determined by selecting onetopology file and two coordinate files—one as the ref-erence (REF) configuration and another as an auxiliary(AUX) configuration. Other available file types aregrouped according to the type of experimental data.

The specification file in Figure 6B lists four classes ofancillary data—latitude-longitude files, paint files,border files, and atlas files—that are described morefully below. To facilitate the selection process, topolo-gy files are identified as CLOSED, OPEN, or CUT,and coordinate files are identified by their configura-tion (FIDUCIAL, INFLATED, SPHERICAL, or FLAT).


F i g u r e 5 (Opposite) Stages of cortical segmentation in SureFit. A, Coronal slice through occipital cortex in a human struc-tural MRI volume. B, Intensity histogram for the image volume shown in A. Arrows indicate values for parameters used togenerate intensity-based probabilistic maps of cortical regions or boundaries. C, An intensity-based probabilistic map of theinner boundary. This is based on a Gaussian intensity transformation:

G(n) ∝ e�(I(n) �Ipeak)2/�

2low for I(n) < Ipeak and G(n) ∝ e

�(I(n) �Ipeak)2/�2high for I(n) > Ipeak

where I(n) is the intensity value for the nth voxel, Ipeak is an estimate of the most likely intensity value for the tissue type orboundary, and the standard deviations �low and �high are related to the noisiness of the image data. D, Map of the magnitudeof the intensity gradient. E, The composite inner boundary map. One component of this composite map is the intensity-basedmap of the inner boundary (C). Another component is derived by determining where the intensity gradient is intermediatein magnitude (made explicitly by a Gaussian intensity transformation analogous to that in C) and pointed opposite to thegradient of a probabilistic map of gray matter (not shown; also based on a Gaussian intensity transformation). A third com-ponent selectively emphasizes regions near the crowns of gyri (where the underlying white matter is notably thin) by test-ing for regions containing two gradient-based inner-boundary domains that are in close proximity but pointed in oppositedirections. F, Intensity-based map of the outer boundary. G, Map of the outer boundary in sulcal regions, based on evidencefor two inner boundaries that are pointed in opposite directions and are each displaced about one cortical thickness (3 mmfor human cortex) from the voxel being tested. H, The composite outer boundary map, derived from the intensity-basedouter boundary map (F), the sulcal outer boundary (G), and gradient-based cues analogous to those used for the innerboundary. I, Binary map of cerebral white matter, based on thresholding the intensity volume and removing various non-cerebral structures. J, A map of positions along the radial axis, generated by blurring both the inner and outer boundarymaps, normalizing the output by dividing the difference by the sum at each voxel, and assigning a maximal value to con-tained in the interior of cerebral white matter (i.e., in an eroded version of the image shown in I). K, The initial segmentedvolume obtained by thresholding the radial position map. L, The initial segmented volume superimposed on the originalintensity volume.

Additional information can be obtained by pressingthe query (“?”) button, which brings up informationin the file header, and by pressing the “List” buttonfor paint and border files, which shows the differentlabels or categories contained in the file.

Once the selected data files are loaded, numerousmenu options are available for changing the surfaceconfiguration, the viewing perspective, and the infor-mation displayed on the surface or in relation to it. InFigure 6A, the flat map configuration has been select-ed (using the “AUX” toggle button), surface nodeswere painted to show cortical geography using the“G” (geography) toggle button, surface tiles werefilled using the “Tile” toggle button, and latitude andlongitude isocontours were displayed using the “B”(borders) toggle button.

The ancillary data shown in Figure 6A were generat-ed using several of the data analysis options availablein Caret. The latitude and longitude isocontours weregenerated by a process that requires a spherical mapto be loaded in the AUX configuration. The contour

points were then converted from the initial configu-ration-specific three-dimensional coordinates to a“border projection” file format. This allows contoursto be viewed in relation to any surface configuration(e.g., the flat map in Figure 6A) by expressing con-tour point locations in relation to the nearest tile (i.e.,in “barycentric” coordinates27).

To generate the map of cortical geography shown inFigure 6A, the sulcal pattern was initially represent-ed by a map of folding (mean curvature) computedfor the fiducial surface but displayed on the flat map.Border contours were drawn around the perimeter ofeach sulcus as visualized on a flat map. Nodesenclosed by one or another closed sulcal boundarywere automatically identified, and the assignmentswere stored in a paint file that contains up to five sep-arate categorical types of information for each node.

Figure 7 illustrates several additional Caret visualiza-tion options that facilitate comparisons between dif-ferent surface configurations and different partition-ing schemes. On the left (Figure 7,A and B) is the


F i g u r e 6 A, The main Caret screen, showing a cortical flat map from the macaque surface-based atlas. The tear-off menuon the upper left shows options available in the File menu. B, The File Selection dialog after a specification file for the macaquesurface-based atlas has been selected. Files listed in the specification file are displayed in appropriate categories. Default filesections in each category are indicated by depressed buttons on the left. The currently loaded border file or border projectionfile can be toggled on and off using the “B” (Borders) toggle button. Selection of the geography (“G”) toggle button colorsnodes according to the node identities and the color assignments contained in the currently loaded “area color file.”

Felleman and Van Essen14 areal partitioning schemefor visual areas; on the right (Figure 7C and D) is aprobabilistic atlas based on the Lewis and Van Essenareal partitioning scheme,18,28 with a few specificareas identified (V2, V4, TE1–3). A blending optionallows the visual areas and the geography map to beviewed concurrently. The probabilistic atlas was gen-erated from architectonic maps of visual areas in fivemacaque hemispheres by deforming them to themacaque atlas (discussed below). In this grayscalefigure, brightness reflects the fraction of cases thathave the same areal identity at a given location; onthe Caret screen, different hues are used to discrimi-nate the individual areas.

An option for identifying nodes using the mouse cur-sor (enabled using the “IDN” toggle button) providesa useful way to assess relationships between differentconfigurations and to extract textual informationabout each node. Selecting a node in one configura-tion (e.g., the flat map) highlights that node when theview is switched to other configurations or to a dif-ferent node coloring scheme (white squares andarrows in Figure 7A to D). In addition, the pop-upStatus Message window (Figure 7E) shows manytypes of information about the selected node. Thisincludes the node identity plus its spatial location inthree different coordinate systems—three-dimen-sional stereotaxic coordinates (if the fiducial configu-ration is loaded in the REF configuration), Cartesianmap coordinates (if a standard flat map is loaded inthe AUX configuration), and latitude and longitude(if a latitude-longitude file is loaded). Additionalinformation includes node attributes contained in thecurrently loaded paint file (in this example, assign-ments of cortical geography, lobe, functional area,and modality) and atlas file (in this case, the arealidentity for each hemisphere contributing to theprobabilistic atlas).

Caret includes several data formats for visualizingand analyzing fMRI and other spatially complex pat-terns of neuroimaging or neuroanatomic data (cf.Figure 2). Data types that can be assigned to nodesand used to color the surface include RGB_paint andmetric files, which are generated by SureFit when itmaps fMRI data from an initial volume representa-tion onto a surface. A file type that is particularly use-ful for surface-based deformations is the “activationfile,” which is generated by applying an adjustablethreshold to the data in a metric file. Cell files repre-sent data related to the spatial distribution of labeledneurons or other discrete data. Cell files can be con-verted to representations of cell density to facilitatequantitative analyses of connectivity patterns. Foci


F i g u r e 7 Visualization and identification of multipledata types simultaneously loaded into Caret. A, The fidu-cial configuration of the macaque surface-based atlas,showing visual areas in the Felleman and Van Essen14 par-titioning scheme. B, Flat map showing the same set ofvisual areas. C, Probabilistic map of visual areas in theLewis and Van Essen28 partitioning scheme, shown on thefiducial configuration. Brighter shading indicates a higherfraction of cases associated with the same visual area(which appear in color on the Caret screen). D, Flat mapshowing the Lewis and Van Essen areas. In A through D,a node in one of the temporal lobe areas was highlighted(white square, which appears green on the Caret screen). E, Thepop-up Caret status message window, displaying infor-mation about the highlighted node. The three-dimension-al position represents the coordinates of the nodes in theREF configuration (here, the fiducial surface). The two-dimensional position represents the coordinates in theAUX configuration (here, the flat map in Cartesian stan-dard coordinates with the origin at the ventral tip of thecentral sulcus). The latitude (�20.5 ) and longitude(�101.9 ) represent values in the currently loaded latitude-longitude file, which was previously generated using aspherical map with areal distortions minimized and ori-ented to the spherical standard coordinate frame (ventraltip of the central sulcus at the lateral pole). The fourth rowdisplays information from the currently loaded paint file(LOBE.TEMP signifying the temporal lobe; “???” signify-ing no entry for that column of the five-column paint file).The last row displays information from the currentlyloaded atlas file. This contains five entries, only three ofwhich are shown here, indicating that the selected node isan architectonic area TE1-3 in some but not all cases fromthe atlas file data set.28 The data sets illustrated here can bedownloaded for viewing in Caret from http://stp.w u s t l . e d u / s u m s / s u m s . c g i ? s p e c f i l e = 2 0 0 1 - 0 3 -02.79O.R.LEWIS_VE_ON_ATLAS.spec.

files represent an alternative type of data related tothe centers of PET or other neuroimaging data. Datapoints in foci files can be represented with associateduncertainty limits reflecting individual variabilityand spatial uncertainty in mapping data based onstereotaxic coordinates.29

Caret also contains options for generating, visualiz-ing, and saving various types of information aboutsurface geometry. These include representations ofmean curvature (folding), intrinsic (Gaussian) curva-ture, distortion (AUX surface compared with REFsurface), and geometric cross-overs (creases or topo-logical handles in the surface).

Modifying Surface Geometry

Caret includes numerous options for modifying sur-face configuration (shape) or topology, or both, manyof which were used in generating the data shown inFigures 2, 5, 6, and 7. These are grouped into six gen-eral classes—smoothing, geometric projections ortransformations, contour-based reconstruction, auto-mated flattening and distortion reduction, interactivesurface editing, and surface-based deformation.

■ Smoothing. Smoothing operations include conven-tional smoothing (iteratively moving each node tothe average of its neighbors), targeted smoothing(preferentially smoothing nodes that are highlydistorted relative to the reference surface), andsurface inflation (smoothing coupled with prefer-ential expansion of nodes close to the center ofgravity, to accelerate the smoothing of highly con-voluted surfaces).

■ Geometric transformations and projections. Optionsfor geometric projection or transformation of sur-faces include translation, scaling, rotation, andmirror-reflection; projection of a surface radiallyoutward to a sphere or ellipsoid; and projection ofa sphere to a plane (by a topology-preservingtransformation if the sphere has a hole orientedalong the positive z direction).

■ Surface reconstruction. Contour-based surfacereconstruction is used when the input data are aseries of section contours rather than pre-existingsurfaces generated by SureFit or other automatedvolume segmentation and surface reconstructionmethods. Caret includes options for aligning indi-vidual section contours and generating surfacereconstructions from the stack of aligned contours,using the Nuages algorithm.30

■ Flattening. Automated flattening and distortionreduction involves two main stages of processing.

In the first stage, the fiducial surface is reconfig-ured by smoothing or geometric projection stepsto attain a spherical or flattened shape, dependingon the desired geometry. Multi-resolution morph-ing is then applied to generate a minimally dis-torted flat map or spherical map. In both cases, theunderlying algorithm10,11 involves resampling,downsampling, and morphing (iterative applica-tion of forces to each node to reduce distortionsrelative to the fiducial configuration). Surfaceresampling represents the initial reference andauxiliary configurations with a hexagonal array ofnodes that are uniformly spaced in the auxiliaryconfiguration. Once resampled, a surface can bedownsampled (by deleting alternate nodes) togenerate progressively coarser representations ofthe original shape. Multi-resolution morphingstarts at the coarsest level of resampling, and isthen reapplied after successive stages of upsam-pling to a finer resolution. The cycle is repeateduntil the overall distortions have reached anacceptably low level. When Caret is run on an SGIOctane, this takes several hours for a completehuman hemisphere.

■ Interactive editing. Interactive editing of surfacegeometry includes a number of options—drawingand applying cuts to the surface, deleting regionsthat have been disconnected by cuts, repositioningindividual nodes, connecting or disconnectingselected node pairs, and processing the modifiedtopology file to identify edge nodes and establisha consistent orientation of surface normals.

■ Surface-based deformation. Options for surface-baseddeformation in Caret allow the map (the source) tobe deformed to another (the target) while con-strained by explicitly designated landmarks. Thealgorithm for deforming flat maps was developedby Joshi and Miller.15,21 Landmarks are drawnalong what the user judges to be correspondinglocations on the source and target maps. The land-mark contours are resampled to establish corre-sponding numbers of landmark points on eachsource and target landmark contour. The land-marks are then used as constraints for a two-dimen-sional differeomorphic deformation algorithm. Thisalgorithm computes solutions to a transport equa-tion in which the landmark matching is constructedto minimize a running smoothness energy on thevelocity field. Additional data (as a coordinate file,border file, or activation file) are carried passivelywith the deformation.

Spherical maps can be deformed using an algo-rithm developed by Bakircioglu et al.17 The basic


strategy is similar to that for flat deformations, butit entails using Laplacian differential operatorsconstrained to the tangent space of the sphere andbasis functions that are expressed as spherical har-monics. We have recently developed an improvedmethod that is based on landmark-constrainedsmoothing and morphing of coordinates. As apractical matter, landmarks can be drawn on flatmaps of the source and target hemisphere, wherevisualization is easiest, then projected to the corre-sponding spherical maps. Conversely, once defor-mations have been applied to the spherical maps, itis generally preferable to view the results on flatmaps (cf. Figure 2I and J).

SuMS

The architecture of SuMS (Surface ManagementSystem) comprises four main components:

■ A Java-based database structure designed specifi-cally for cortical surface representations

■ A high-capacity, secure data cache capable of stor-ing large amounts of data

■ A database manager (Sybase SQL Server) for coor-dinating the file cache and database access

■ Search and retrieval capabilities using either a webbrowser (WebSuMS) or the downloadable SuMSClient software (a downloadable graphical userinterface that runs on Java-compatible platforms).

For data entry it is currently necessary to use theSuMS Client. For most purposes WebSuMS is the eas-ier to use, but it does not yet include all the capabili-ties of the SuMS Client.

Data Entry

Data entry in SuMS is a simple process in which thenames of the volume or surface specification filesintended for insertion are entered into the appropri-ate dialog box in the SuMS Client. For successful dataentry, all files must be present in the directory loca-tions listed in the specification file and the requisitemetadata must be included in the appropriate fileheaders. Files generated using SureFit and Caret(v4.3 or higher) automatically include this informa-tion in the individual files or in the specification files.At the outset of the data entry process, the metadatafor all files are checked for completeness, andprompts are given if required information is missing.A cross-checking process avoids duplicate insertionof files and ensures that only new data sets are addedto SuMS.

Data Search and Retrieval

WebSuMS provides several convenient ways to iden-tify and download files of interest, starting from theSuMS search page (http://stp.wustl.edu/sums/sums.cgi), shown in Figure 8A). One option is toenter the name of a particular volume specificationfile or surface specification file into the appropriatedialog box, if the name (or portion of the name) isknown. Another is to search for particular files usingone or more search criteria shown in the bottom halfof the window.

A third option is to select View Atlases near the topof the window. This brings up a list of four standardspecification files for the macaque and human atlases(Figure 8B). Selection of the MACAQUE_ATLAS.spec brings up the list of files shown in Figure 8C.These can be downloaded as a group, then viewed inCaret after the files are uncompressed.

An even simpler and faster way to retrieve files is tomake a direct hyperlink connection to a particularspecification file in the database, starting from a sep-arate application such as an online journal article,PDF file, or e-mail message. For example, the data forthe individual hemisphere illustrated in Figure 2 canbe immediately accessed by a hyperlink connectionto the appropriate specification file (http://stp.wustl.edu/sums/sums.cgi?specfile=Demo.L.full.jamia.Fig2.spec). This directly links to a specification file con-taining the relevant data files. Selecting the“Download All” option downloads these files to thedesired directory in a single step.

The search process in the SuMS Client is similar, butis more flexible because it includes an intermediaterepository. The first stage is to choose a combinationof search criteria for identifying files of potentialinterest and to initiate the search process. The secondstage is to view the results of the initial search and toselect any or all of these files for placement in asearch repository. The search repository is a listing offiles provisionally targeted for downloading. Thecontents of the repository (akin to a “shopping cart”)can be expanded by repeating the search processusing a different set of criteria. The third stage is toselect the final set of files from the search repositoryand then to initiate the download process.

Access Control

SuMS uses a security model based on the designationof owners, readers, and writers for each file in thedatabase. This ensures that private data are protect-ed, public data are freely available, and any data can


be made available to some investigators (e.g., collab-orators) but not others. This security scheme is flexi-ble enough for any owner to allow access to restrict-ed data sets on an individual-user or group basis.

Discussion

Surface-based analyses have tremendous potentialfor enhancing progress in understanding corticalstructure, function, and development in health anddisease. If widely adopted, the software toolsdescribed here for computational cortical cartogra-

phy, along with related tools developed in other lab-oratories, will aid in capitalizing on this potential.The ultimate measures of progress will, of course, bebased on the actual scientific and medical discoveriesthat benefit from surface-based analyses.

In addition, several more proximate measures will beof interest to monitor over the next several years.These include the pace with which the neurosciencecommunity adopts cortical surface reconstructions asa standard way to analyze and display experimentalresults from individual subjects; adopts surface-based atlases and probabilistic representations as


F i g u r e 8 Screen displays using WebSuMS to access the SuMS database. A, The WebSuMS search page (http://stp.wustl.edu/sums/sums.cgi) can be used to identify specification files in the SuMS database that meet whatever search crite-ria are entered by the user. This can be a partial or complete name of a particular volume or surface specification file (toprows) or a particular case from the currently selected species (macaque in this example) plus the option of adding criteria suchas key words in the comment section of any data file or in the data fields of paint files, border files, and cell files containedin SuMS. B, The current listing of surface-based atlas specification files obtained by selecting the “View Atlases” menu but-ton. C, The specification file obtained by selecting one of the macaque atlases (LEWIS_VE_ON_ATLAS.spec). The entire setof files can be downloaded as a group, then viewed in Caret after being uncompressed. For WebSuMS, the user interfaceswith the SuMS Web Server via HTTP (HyperText Transfer Protocol) using standard HTML forms and scripts based on CGI(Common Gateway Interface). For the SuMS Client, Remote Method Invocation (developed by Sun Microsystems) is used asthe interface with a JDBC (Java Database Connectivity) driver. One Java application or applet (the SuMS client in this con-text) can call the methods of another Java application (the SuMS server) running on a different host machine.

general strategies for bringing experimental data intoregister on a common substrate; adopts surface-based coordinates as a concise and objective primarymetric for describing locations on the cortical surface,to complement the use of Talairach stereotaxic coor-dinates; and uses surface visualization software anddatabases of surface representations as a supplemen-tary option for assessing and visualizing experimen-tal data associated with published studies.

SureFit, Caret, and SuMS have been under develop-ment in our laboratory for a number of years. How-ever, they have only recently reached a state of matu-rity that allows semi-automated execution of a com-plete processing sequence, proceeding from primarystructural and functional data to data sets that are rep-resented on cortical flat maps, transformed to surface-based atlases, and stored in a database. Some of thefunctionality of the SureFit/Caret/SuMS suite is cur-rently unique, but most of its capabilities are shared byone or more other brain-mapping software packages.These include FreeSurfer,4,13 Brain Voyager,5,31 andmrGray.6 It is useful to discuss briefly the key issuesinvolved in evaluating any of these packages,although detailed analysis and comparison are outsidethe scope of this discussion.

Segmentation and Surface Reconstruction

Two primary sets of issues are involved in evaluatingdifferent surface reconstruction methods:

■ How robust, easy to use, and accessible is the soft-ware for generating surfaces and mapping dataonto surfaces?

■ How accurate are the surface representations andhow accurately are experimental data mappedonto the surfaces?

Structural image data obtained in neuroimaging andneuroanatomic studies vary widely in overall quality.This is attributable to differences in image contrast,noise, and regional inhomogeneities arising from theparticular methods or devices used for image acquisi-tion and from the individual subjects under study.Accordingly, each method of segmentation should betested for its robustness and accuracy, using data setsthat span a wide range in quality. This has yet to bedone systematically, because of the newness of themethods, the inherent complexity of the data, and thelack of generally accepted standards for evaluation.

One important metric is whether the segmenta-tion/surface is topologically correct. Topologicalerrors (handles) are particularly deleterious for flat-tening and subsequent processing stages, unless they

are small and outside the main region of experimen-tal interest. Another important metric is spatial accu-racy, i.e., how close the surface runs to the desiredtarget boundary. This is especially important if theobjectives of the experimental analysis are sensitiveto modest but systematic biases or to less frequentbut larger deviations from the desired trajectory.However, assessment of spatial accuracy is inherent-ly problematic, because a “ground truth” or “goldstandard” that precisely represents the target layer isgenerally not available for structural MRI or cryo-sectional image data.

A reasonable alternative strategy is for one or moreexpert anatomists to draw contours along the targetlayer in a number of selected regions in which the tra-jectory can be confidently estimated. The distancefrom each point on the target contour to the nearestpoint on a fiducial surface reconstruction can then bedetermined and displayed as a histogram, and the dis-tribution of errors can be expressed by measures suchas the error and standard deviation. If those who drawthe target contours are unaware of the results of anyparticular segmentation, this approach can serve as anobjective basis for evaluating and comparing the per-formance of different segmentation methods.

Surface Manipulation: Flattening and Deformations

A number of methods are currently available forgenerating cortical flat maps and spherical mapsfrom fiducial surface representations. Given that thefiducial surface contains a complex pattern of intrin-sic (Gaussian) curvature,29 significant areal distor-tions are inevitable on any flat map or spherical maprepresentation. Nonetheless, it is desirable to mini-mize these distortions and to quantify the magnitudeand distribution of residual distortions.

Residual distortions can affect surface-based analy-ses in two major ways. First, they affect visualimpressions about the relative sizes of differentregions and intracortical distances between regions.This is analogous to how distortions on earth mapsaffect impressions about the relative sizes of differentcontinents (e.g., Greenland vs. South America).Fortunately, compensation for such perceptual biasescan be largely made by analyzing and reporting sur-face areas and geodesic distances determined on thefiducial reconstruction.

Another problem is that surface-based warping fromone hemisphere to another can be affected by distor-tions, especially if the pattern of distortions differs onthe source and target maps. This puts a premium on


minimizing distortions on whichever surface configu-ration (sphere or flat map) is used as the substrate forsurface-based deformations. Histograms of areal dis-tortion for different surface nodes provide an objectivebasis for comparing different flattening algorithms.

An inherent advantage of surface-based deforma-tions is that the process respects the topology of thecortical sheet in compensating for individual vari-ability. For this reason, surface-based deformationsshould in principle achieve substantially better regis-tration than volume-based deformations, even if thevolumetric approach uses a very high-dimensionalparameter space to constrain the deformation. Thisprediction is supported by initial comparisons of sur-face-based vs. volume-based approaches to the regis-tration problem.16,20 However, the issue is of suffi-cient general importance that more extensive quanti-tative comparisons are strongly warranted.

As progressively more data are mapped to surface-based atlases, it will be feasible to generate anincreasingly diverse and rich set of probabilistic rep-resentations. One type of representation, relativelyclose to the primary data, involves probabilistic mapsof fMRI activation patterns from multiple subjectscarrying out similar or identical behavioral tasks16

(see Figure 2 above). Given the enormous diversity ofbehavioral tasks that have been or will be used onmultiple subjects in fMRI studies, the number of suchprobabilistic maps is effectively open-ended.

Another type of representation, involving greaterabstraction from the data, involves probabilistic mapsof cortical areas whose boundaries are estimated fromfunctional, architectonic, or other types of experimentaldata (cf. Van Essen et al.18). To the extent that most orall cerebral cortex is in fact divisible into genuine, well-defined subdivisions, it might be hoped that proba-bilistic maps of cortical areas would converge toward asingle, consensual representation. However, a morelikely scenario is that numerous alternative partition-ing schemes will remain in widespread use. This willlead to a multiplicity of probabilistic maps based ondifferent schemes, different criteria used to deformindividual data sets to a surface-based atlas, or differ-ent substrates used as the target surface-based atlas.

Conclusion

These considerations emphasize the pressing needfor continued progress in several aspects of comput-erized cortical cartography. One need is to developimproved methods for evaluating the quality of reg-

istration of individual data sets to an atlas, on thebasis of objective measures that can be applied to adiversity of data types. Another need is to improvethe methods for surface-based deformation. Forexample, it is desirable to have hybrid methods thatcombine aspects of the landmark-based approachdescribed here with approaches based on a continu-ous valued representation as used by Fischl et al.16 Athird need is to enhance interoperability, so that sur-face-based analyses for any given data set can easilymigrate from one software suite to another. This willfacilitate comparisons among data sets obtained indifferent laboratories and will allow a more extensivesets of analyses to be carried out on data sets of par-ticularly broad interest.

A fourth need is for ongoing enhancements in thedatabase infrastructure, for coping with the flood ofsurface-related data. We believe that the SuMS data-base described here has considerable promise as aninitial effort in this direction. If the concept is indeedsuccessful, however, it will need to be scaled up byseveral orders of magnitude to cope with the esti-mated 100,000 cortical surface reconstructions peryear that may emerge in this decade from brain-map-ping efforts in basic and clinical laboratories aroundthe world.22 The advantages of bringing these dataunder the umbrella of one or more databases willbecome increasingly evident, just as it has in suchother scientific arenas as genomics and protein struc-ture.32–34 This will impose substantial pressures forincreased data capacity, network speeds, and a rich-er array of search capabilities.

The authors thank the many beta testers at Washington Universityand elsewhere whose bug reports and constructive suggestionshave greatly helped to improve Caret, SureFit, and SuMS.

References ■

1. Daniel PM, Whitteridge D. The representation of the visualfield on the cerebral cortex in monkeys. J Physiol (Paris).1961;159:203–21.

2. Gattass R, Gross CG. Visual topography of striate projectionzone (MT) in posterior superior temporal sulcus of themacaque. J Neurophysiol. 1981;46:621–38.

3. Van Essen DC, Drury HA, Anderson CH. An automatedmethod for reconstructing complex surfaces, including thecerebral cortex. Soc Neurosci Abstr. 1999;25:1929.

4. Dale AM, Fischl B, Sereno MI. Cortical surface-based analysis,part I: Segmentation and surface reconstruction. NeuroImage.1999;9:179–94.

5. Goebel R, Khorram-Sefat D, Muckli L, Hacker H, Singer W.Functional imaging of mirror and inverse reading reveals sep-arate coactivated networks for oculomotion and spatial trans-formations. NeuroReport. 1998;9:713–9.


6. Wandell BA, Chial S, Backus BT. Visualization and measure-ment of the cortical surface. J Cogn Neurosci. 2000;12:739–52.

7. Van Essen DC, Zeki SM. The topographic organization of rhe-sus monkey prestriate cortex. J Physiol. 1978;277:193–226.

8. Van Essen DC, Maunsell JHR. Two-dimensional maps of thecerebral cortex. J Comp Neuro.l 1980;191:255–81.

9. Schwartz EL, Shaw A, Wolfson E. A numerical solution to thegeneralized mapmaker’s problem: flattening nonconvex poly-hedral surfaces. IEEE Trans Pattern Anal Machine Intell.1989;11:1005–8.

10. Carman GJ, Drury HA, Van Essen DC. Computational meth-ods for reconstructing and unfolding the cerebral cortex.Cerebl Cortex. 1995;5:506–17.

11. Drury HA, Van Essen DC, Anderson CH, Lee CW, CooganTA, Lewis JW. Computerized mappings of the cerebral cortex:a multiresolution flattening method and a surface-based coor-dinate system. J Cogn Neurosci. 1996;8:1–28.

12. Teo PC, Sapiro G, Wandell BA. Creating connected represen-tations of cortical gray matter for functional MRI visualization.IEEE Trans Med Imag. 1997;16:852–63.

13. Fischl B, Sereno MI, Dale AM. Cortical surface-based analysis,part II: Inflation, flattening, a surface-based coordinate system.NeuroImage. 1999;9:195–207.

14. Felleman DJ, Van Essen DC. Distributed hierarchical process-ing in primate cerebral cortex. Cereb Cortex. 1991;1:1–47.

15. Joshi S. Large deformation diffeomorphisms and Gaussian ran-dom fields for statistical characterization of brain submanifolds[doctoral thesis]. St. Louis, Mo.: Department of Electrical En-gineering, Sever Institute of Technology, WashingtonUniversity, 1998.

16. Fischl B, Sereno MI, Tootell RB, Dale AM. High-resolutionintersubject averaging and a coordinate system for the corticalsurface. Hum Brain Mapp. 1999;8:272–84.

17. Bakircioglu M, Joshi S, Miller MI. Landmark matching on thesphere via large deformation diffeomorphisms. Proc SPIEMed Imaging Image Processing. 1999;3661:710–5.

18. Van Essen DC, Lewis JW, Drury HA, et al. Mapping visualcortex in monkeys and humans using surface-based atlases.Vision Res. 2001;41:1359–78.

19. Drury HA, Van Essen DC, Joshi SC, Miller MI. Analysis andcomparison of areal partitioning schemes using two-dimen-sional fluid deformations. NeuroImage. 1996;3:S130.

20. Drury HA, Van Essen DC, Corbetta M, Snyder AZ. (1999)Surface-based analyses of the human cerebral cortex. In: TogaA (ed). Brain Warping. San Diego, Calif.: Academic Press,1999:337–63.

21. Van Essen DC, Drury HA, Joshi S, Miller MI. Functional andstructural mapping of human cerebral cortex: solutions are inthe surfaces. Proc Natl Acad Sci USA. 1998;95:788–95.

22. Dickson J, Drury H, Van Essen DC. Surface management sys-tem (SuMS): a surface-based database to aid cortical surfacereconstruction, visualization and analysis. Phil Trans RoyalSoc Ser B. 2001, in press.

23. Corbetta M, Akbudak E, Conturo TE, et al. A common net-work of functional areas for attention and eye movements.Neuron. 1998;21:761–73.

24. Lorensen W, Cline H. Marching cubes: a high-resolution 3-D sur-face reconstruciton algorithm. Comput Graph. 1987;21:163–9.

25. Lee T-C, Kashyap R, Chu C-N. Building skeleton models via 3-D medial surface/axis thinning algorithms. Graph ModelsImage Processing. 1994;56:462–78.

26. Schroeder W, Martin K, Lorensen B. The Visualization Tookit:An Object-oriented Approach to 3D Graphics. 2nd ed. SaddleRiver, NJ: Prentice Hall, 1997.

27. Farin G, Hanford D. The Geometry Toolbox for Graphics andModeling. Natick, Mass.: AK Peters, 1998.

28. Lewis JW, Van Essen DC. Architectonic parcellation of pari-eto-occipital cortex and interconnected cortical regions in theMacaque monkey. J Comp Neurol. 2000;428:79–111.

29. Van Essen DC, Drury HA. Structural and functional analysesof human cerebral cortex using a surface-based atlas. JNeurosci.1997;17:7079–102.

30. Geiger B. Three-dimensional Modeling of Human Organs andIts Application to Diagnosis and Surgical Planning. InstituteNational de Recherche Informatique et Automatique, 1993.Technical Report 2105.

31. Goebel R. A fast, automated method for flattening cortical sur-faces. NeuroImage. 2000;11:680.

32. Berman HM, Westbrook J, Feng Z, et al. The Protein DataBank. Nucl Acids Res. 2000;28:235–42.

33. Marshall E. Rival genome sequencers celebrate a milestonetogether. Science. 2000;228:2294–5.

34. Pennisi E. Finally, the Book of Life and Instructions forNavigating it. Science. 2000;228:2304–7.


an integrated software suite for surface-based analyses of...

Documents