Fast Algorithms for the Reconstruction and

Analysis of Basic Circuits in the Mouse

Cortical MapsMinisymposium on High-throughtput Microscopy and Analysis

November 18, 2008

Yoonsuck Choe

Texas A&M University

Collaborators: Bruce H. McCormick, Louise C. Abbott, John Keyser, David

Mayerich, Jaerock Kwon, Zeki Melek, Donghyeop Han, Pei-San Huang, Stephen

J. Smith, Kristina Micheva.

Sponsors: NIH/NINDS (#1R01-NS54252), NSF (#0079874), Texas HECB

(ATP#000512-0146-2001)

1

In Memory of Bruce H. McCormick

Bruce H. McCormick (1928–2007)

• Designer of the Knife-Edge Scanning Microscope (Mayerich et al.

2008)

2

Long-Term Goal

http://mouseatlas.org http://nervenet.org

• Image the whole mouse brain in submicrometer

resolution.

• Extract the connectome, the complete structural

description of the connection matrix.

• Analyze the basic circuits.

3

Plan of Attack

Shepherd (2003)

• Imaging: high-throughput, whole-brain

microimaging (KESM, ATLUM, Array Tomo.)

• Connectome: from raw image stacks to a complete

structural description of brain connectivity (tracing).

• Basic circuits: from geometric structure to

functional modules of the brain network (mining).

4

Challenges

Shepherd (2003)

• Imaging: staining/labeling, speed, volume limit

• Connectome: amount of data (TB to PB), density of

objects, missing info, noise in data, validation.

• Basic circuits: combinatorial explosion in search,

limited theoretical insight on circuit organization.

5

Overview

1. Background: Geometric reconstruction approaches

2. Fast tracing in 2D

3. Fast tracing in 3D

4. Fast interactive visualization and filtering

5. Discussion: validation, basic circuit mining

6

Background

Approaches in geometric tracking and reconstruction:

• Segment then connect

• Supervised learning

• Vector tracing

7

Background: Seg. then Conn.

Segment Reconstruct

Est. StructureImage Stack Segmented Image Stack

• Register images, segment, and then connect

(Chklovskii 2008).

• Most straight-forward and popular approach (e.g.,

ITK).

• In many cases done manually (Brown et al. 2007;

Fiala 2005) (Ascoli lab, Harris lab)

8

Background: Supervised Learning

Input:Data volume

Output:Restoration

Supervised learning with 3D convolutional neural

networks (Jain et al. 2007) (Seung lab):

• Train weights with manually segmented data set as

the target.

• Test with unlabeled raw data set.

9

Background: Vector Tracing

(Al-Kofahi et al. 2002) (Roysam lab)

• Trace along the natural flow of the fibrous object.

• Use steerable templates for computational efficiency.

• Used in commercial tracing software.

10

Overview of Our Methods

1. Fast tracing in 2D

2. Fast tracing in 3D

3. Fast interactive visualization and filtering

11

Fast Tracing in 2D

*

inte

nsity

position

• Examine along border of

moving window.

• Find cross section (black

part) and interpolate.

• Check the interpolation

against data pixels.

12

Fast Tracing in 2D

ci

ci+1 ci+1

ci

ci+2

step i step i+1

ci+21

2

Han and Choe

• Move window in the direction of fiber (dendrite, axon,

vasculature) direction.

• Can effectively handle branches.13

Tracing Results

Seed Can et al. (1999)

Haris et al. (1999) Our method14

Robustness

�: Ours; ♦ Can et al.; �: Harris et al.

Accuracy tested based on synthetic data (by varying

fiber width).

• Ours much more robust to varying fiber width,

compared to competing approaches such as Can

et al. (1999) or Haris et al. (1999).

15

Performance

0

1000

2000

3000

4000

5000

6000

7000

8000

1 2 3 4 5 6

Tota

l tra

ce le

ngth

Data set

Can et al.Harris et al.Our method

0 10 20 30 40 50 60 70 80 90

100 110

1 2 3 4 5 6

Tim

e/Un

it Le

ngth

Tra

ced

Data set

Can et al.Harris et al.Our method

(a) Total trace length (b) time/unit trace length

Performance measured on vascular data set:

• Compared to Can et al. and Haris et al., our method

(green) traces (a) longer distances, (b) faster.

• Accuracy needs to be checked.

16

Fast Tracing in 3Dpredict

correct

Predictor–Corrector

Mayerich and Keyser (2008), Busse et al. (2006)

• Predictor–Corrector approach.

17

Fast Tracing in 3D

Match!

t = 3

t = 2

t = 1

Template matching Tangential slices Templates

• Template matching.

• Use graphics hardware (GPU) for fast matrix

operations.

18

Tracing Results

• Spinal cord vasculature (mouse).

19

Tracing Results

• Neurons (Array Tomography, zebrafish tectum)

• Tracing (left) and cleaned data based on trace

(right).

20

Performance

0.1

1

10

100

1000

10000

1 10 100 1000 10000

Tim

e (m

s)

Number of Samples

Single Core 2.0GHzQuad Core 2.0GHz

CPU with GPU SamplingFull GPU GeForce 7300

0

5

10

15

20

25

1 10 100 1000 10000

Spee

dup

Fact

or

Number of Samples

GPU (Sampling Only)Single Core 2.0GHz

Run time Speedup (Full GPU)

• Use of GPU gives an order-of-magnitude reduction

in computation time.

21

Fast Interactive Visualization/Filtering

A

B

C

For fast, interactive visualization (Melek et al. 2006):

• Use self-orienting surfaces.

• Use GPU acceleration.

22

Fast Interactive Visualization/Filtering

(a) Wire (b) SOS (c) Ori. filtered

• Fast interactive rendering in realtime.

• Interactive filtering for geometric properties.

• Important for interactive editing.23

Discussion• Uses: constrain models (Blue brain, Neuroconstruct, NeuGen,

Netmorph, L-neuron); provide statistics; simulation (NEURON,

GENESIS); etc.

• Reconstruction: validation is a huge issue (Warfield et al. 2004)

(digital phantom: Koene); fast editing support (proof-reading);

high-performance computing support (Rao)

• Connectivity estimation: need to estimate/infer connectivity

(Kalisman et al. 2003); array tomography (Micheva/Smith)

• Basic circuit mining: NP-complete problem

• Other issues: connection strength, delay, sign (inh/exc), degree

of structure/function coupling

24

Acknowledgments

• People:

– Texas A&M: Y. Choe, B. McCormick, J. Keyser, L. C. Abbott, D.

Mayerich, D. Han, J. Kwon, Y. H. Bai, D. C.-Y. Eng, H.-F. Yang, G.

Kazama, K. Manavi, W. Koh, Z. Melek, J. S. Guntupalli, P.-S. Huang, A.

Aluri, H. S. Muddana

– Stanford: S. J. Smith, K. Micheva, J. Buchanan, B. Busse

– UCLA: A. Toga

– Special thanks: T. Huffman (Arizona State U), R. Koene (Boston U),

Bernard Mesa (Micro Star Technologies)

• Funded by: NIH/NINDS (#1R01-NS54252); NSF (MRI #0079874 and ITR

#CCR-0220047), Texas Higher Education Coordinating Board (ATP

#000512-0146-2001), the Department of Computer Science, and the Office

of the Vice President for Research at Texas A&M University.

25

Concluding Remarks

KESMATLUMArray Tomo.SBF−SEM

EditingGeometric Desc.

Data stack

NEURON, GENESIS, NeuGen, Blue Brain, Neuroconstruct

Image processing

Digital Phantom

Reconstruction Validation

NetmorphNeuGenNeuroconst.

HPC

26

Open Discussion

• How can the resulting structural data be used?

– What kind of scientific questions can be addressed?

– How can it be used to answer medical/clinical questions?

– How to bridge structure and function?

• How much detail is needed?

• How can we validate the results?

• What are the technological obstacles?

• How can we utilize power of high-performance computing?

27

ReferencesAl-Kofahi, K. A., Lasek, S., Szarowski, D. H., Pace, C. J., Nagy, G., Turner, J. N., and Roysam, B. (2002). Rapid automated

three-dimensional tracing of neurons from confocal image stacks. IEEE Transactions on Information Technologyin Biomedicine, 6:171–187.

Brown, K. M., Donohue, D. E., D’Alessandro, G., and Ascoli, G. A. (2007). A cross-platform freeware tool for digitalreconstruction of neuronal arborizations from image stacks. Neuroinformatics, 3:343–359.

Busse, B., Smith, S. J., Taylor, C. A., and Arakaki, R. Y. (2006). Development of a semi-automated method for three-dimensional neural structure segmentation. In Society for Neuroscience Abstracts. Washington, DC: Society forNeuroscience. Program No. 834.13. Online.

Can, A., Shen, H., Turner, J. N., Tanenbaum, H. L., and Roysam, B. (1999). Rapid automated tracing and featureextraction from retinal fundus images using direct exploratory algorithms. IEEE Transactions on InformationTechnology in Biomedicine, 3:125–138.

Chklovskii, D. (2008). From neuronal circuit reconstructions to principles of brain design. In Proceedings of the 5thComputational and Systems Neuroscience Meeting (COSYNE 2008 Abstracts), 331.

Fiala, J. C. (2005). Reconstruct: A free editor for serial section microscopy. Jounral of Microscopy, 218:52–61.

Haris, K., Efstratiadis, S., Maglaveras, N., Pappas, C., Gourassas, J., and Louridas, G. (1999). Model-based morphologi-cal segmentation and labeling of coronary angiograms. IEEE Trans. Med. Imag., 18:1003–1015.

27-1

Jain, V., Murray, J. F., Roth, F., Seung, H. S., Turaga, S., Briggman, K., Denk, W., and Helmstaedter, M. (2007). Usingmachine learning to automate volume reconstruction of neuronal shapes from nanoscale images. In Society forNeuroscience Abstracts. Washington, DC: Society for Neuroscience. Program No. 534.7. Online.

Kalisman, N., Silberberg, G., and Markram, H. (2003). Deriving physical connectivity from neuronal morphology. Biologi-cal Cybernetics, 88:210–218.

Mayerich, D., Abbott, L. C., and McCormick, B. H. (2008). Knife-edge scanning microscopy for imaging and reconstructionof three-dimensional anatomical structures of the mouse brain. Journal of Microscopy, 231:134–143.

Mayerich, D., and Keyser, J. (2008). Filament tracking and encoding for complex biological networks. In Proceedings ofACM Symposium on Solid and Physical Modeling, 353–358.

Melek, Z., Mayerich, D., Yuksel, C., and Keyser, J. (2006). Visualization of fibrous and thread-like data. IEEE Transactionson Visualization and Computer Graphics, 12(5):1165–1172.

Warfield, S. K., Zou, K. H., and Wells, W. M. (2004). Simultaneous truth and performance level estimation (STAPLE): Analgorithm for the validation of image segmentation. IEEE Transactions on Medical Imaging, 23:903–921.

27-2