Dissecting, Imaging, and Modeling

Brain Networks

KOCSEA 2008

October 26, 2008

Yoonsuck ChoeBrain Networks Laboratory

Department of Computer Science

Texas A&M University

Joint work with: Bruce McCormick, Louise Abbott, John Keyser, David Mayerich,

Jaerock Kwon, Donghyeop Han, and Pei-San Huang,

1

Introduction

Main research questions:

1. How does the brain work?

2. How can we use the knowledge to build intelligent artifacts?

Approach:

1. Computational neuroanatomy

Image source: http://www.nervenet.org/papers/Cerebellum2000.html

2

Overview

• Connectomics

• Knife-Edge Scanning Microscope

• Structural reconstruction algorithms

3

Connectomics

4

Connectomics

• Connectome: Complete structural description of the connection

matrix of the brain (see e.g. Sporns et al. 2005).

• Connectomics: Acquisition and mining of the connectome.

• The only available connectome: that of the C. elegans (White

et al. 1986).

Image source: http://www.mouseatlas.org/data/mouse/stages/t47/view

http://www.nervenet.org/papers/Cerebellum2000.html

5

Why Connectomics Research?

• Structure of the nervous system as a foundation of

its function.

– Dynamical properties can be estimated from

static structure.

• Intensive study of single neurons and their molecular

properties must be complemented by a system-level,

architectural perspective.

• Discover modules that make up the brain (motifs,

basic circuits).

• To understand how the brain works!6

Goal of the Project

Obtain and reconstruct the full mouse connectome at

a sub-micrometer resolution.

• 77-78d weight 26–30g

• 13 mm (A-P) × 9.5 mm (M-L) × 6 mm (D-V)

• 75 million neurons (Williams 2000)

7

Knife-Edge Scanning Microscope

8

Knife-Edge Scanning Microscope

• Designed by Bruce H. McCormick.

• Diamond microtome, LM optics, high-speed linescan camera,

precision 3-axis stage [Movie]9

Operational Principles of the KESM

• Aerotech precision stage moves resin-embedded brain tissue across knife

(x/y 20 nm, z 25 nm encoder resolution).

• Back-illumination through diamond knife.

• Nikon CF1 Flour 10X or 40X objectives (NA 0.3/0.8, water imm.).

• Dalsa CT-F3 high-speed line scan camera images the tip of the knife at

44KHz. [Movie]10

KESM Imaging

Line−scan Camera

Microscope objective Diamond knife

Light source

Specimen

Brain specimen is embedded in plastic block.

11

KESM Imaging

Line−scan Camera

Microscope objective Diamond knife

Light source

Specimen

Plastic block is moved toward the knife.

12

KESM Imaging

Line−scan Camera

Microscope objective Diamond knife

Light source

Specimen

Thin tissue slides over knife and gets imaged.

13

KESM Imaging

Line−scan Camera

Microscope objective Diamond knife

Light source

Specimen

Successive line scan constructs a long image.

14

KESM Imaging

Line−scan Camera

Microscope objective Diamond knife

Light source

Specimen

One sweep results in a∼ 4, 000× 20, 000 image (∼ 80MB).

15

KESM Imaging

Line−scan Camera

Microscope objective Diamond knife

Light source

Specimen

One brain results in∼ 25, 000 images.

16

Stair-Step Cutting

Kwon et al. (2008)

• Width of the knife and the field of view of the

objective are not wide enough to cut the entire top

facet of the tissue block.

17

Automated Sectioning/Imaging S/W

• Automated stage controller and image acquisition system

developed in-house.

• Fully automated operation without human intervention: 8 hours a

day, 5 days a week.

18

KESM Data: Golgi Stain

• Mouse cortex (sagittal section).

19

KESM Data: India Ink

• Mouse spinal cord vasculature. [Movie]20

KESM Results: Volume Visualization

Nissl (Cortex) India ink (Spinal cord) Golgi (Pyramidal cell)

Golgi (Cortex) Golgi (Cerebellum) Golgi (Purkinje cell) [Movie]

21

Structural Reconstruction

Algorithms

22

Reconstruction Approaches

Raw data or volume visualization is not enough:

Structural reconstruction is needed.

• Segment-then-connect: the most common approach

• 3D convolutional network: Jain et al. (2007)

• Template-matching-based vector tracing: Al-Kofahi

et al. (2002)

23

Reconstruction: Tracing in 2D

*

inte

nsity

position

ci

ci+1 ci+1

ci

ci+2

step i step i+1

ci+21

2

Choe et al. (2008)

• Moving window with cubic tangential trace spline method.

• Investigates pixels only on the moving window border and on the

interpolated splines for fast processing.

24

Tracing Results

Seed Can et al. (1999)

Haris et al. (1999) Our method25

Robustness Comparison

0

10

20

30

40

50

20 30 40 50

Width

Err

or

0

20

40

60

80

100

120

20 30 40 50

Width

Err

or

Open diamonds: Harris et al.; Closed diamonds: Can et al.; Closed boxes: Our approach.

• Accuracy tested based on synthetic data (by varying

fiber width): Linear (left), curvy (right).

• Much more accurate compared to competing

approaches such as Can et al. (1999); Haris et al.

(1999). 26

Reconstruction: Tracing in 3DMatch!

t = 3

t = 2

t = 1

Template matching Tangential slices Templates

(Mayerich and Keyser 2008; Mayerich et al. 2008)

• Use a moving sphere and trace along points on the

surface of the sphere.

• Use graphics hardware (GPU) for fast matrix

operations during template matching.

27

Tracing Results

Spinal cord vasculature (KESM)

Neuron (Array Tomography, tectum) Vasculature (KESM, cerebellum)28

Speeding Up Tracing Using GPU

0.1

1

10

100

1000

10000

1 10 100 1000 10000

Tim

e (m

s)

Number of Samples

Single Core 2.0GHzQuad Core 2.0GHz

CPU with GPU SamplingFull GPU GeForce 7300

0

5

10

15

20

25

1 10 100 1000 10000

Fact

or

Number of Samples

Single Core 2.0GHzGPU (Sampling Only)

Run time Speedup

• Performance figures demonstrate the speedup

obtained by using GPU computation.

• Speedup achieved by using the full capacity of

GPUs show an almost 20-fold speedup compared to

single-core CPU-based runs.

29

Preliminary Branching Statistics

(vasculature)

Sample Statistics from Reconstructed KESM Brain Vasculature Data (1 mm3 volume)

Region Segments Length Branches Surface Volume Volume

5 5 (mm) (mm2 ) (mm3 ) (% of total)

Neocortex 11459.7 758.5 9100.0 10.40 0.0140 1.4%

Cerebellum 34911.3 1676.4 19034.4 20.0 0.0252 2.5%

Spinal Cord 36791.7 1927.6 26449.1 22.2 0.0236 2.4%

• Geometric structures extracted using the automated

reconstruction algorithms allow us to conduct

quantitative investigation of the structural properties

of brain microstructures.

30

Wrap-Up

31

Discussion and Future Work

• Main contribution: novel imaging method plus

computational algorithms for automated structural

analysis.

• Future work:

– Full-brain reconstruction and validation

– Estimating connectivity from sparsely stained

data (cf. Kalisman et al. 2003)

– Linking structure to function

32

Conclusion

• Understanding brain function requires a system-level

investigation at a microscopic resolution.

• Innovative microscopy technologies are enabling a

data-driven investigation linking the microstructure to

the system.

• The massive data can only be effectively understood

through automated computational algorithms.

33

Acknowledgments

• People:

– Texas A&M: B. McCormick, J. Keyser, L. C. Abbott, D. Mayerich, D.

Han, J. Kwon, Y. H. Bai, D. C.-Y. Eng, H.-F. Yang, G. Kazama, K.

Manavi, W. Koh, Z. Melek, J. S. Guntupalli, P.-S. Huang, A. Aluri, H. S.

Muddana

– Stanford: S. J. Smith, K. Micheva, J. Buchanan, B. Busse

– UCLA: A. Toga

– Others: T. Huffman (Arizona State U), R. Koene (Boston U), Bernard

Mesa (Micro Star Technologies)

• Funded by: NIH/NINDS (#1R01-NS54252); NSF (MRI #0079874 and ITR

#CCR-0220047), Texas Higher Education Coordinating Board (ATP

#000512-0146-2001), and the Department of Computer Science, and the

Office of the Vice President for Research at Texas A&M University.

34

In Memory of Bruce H. McCormick

Bruce H. McCormick (1928–2007)

• Designer of the Knife-Edge Scanning Microscope

35

ReferencesAl-Kofahi, K. A., Lasek, S., Szarowski, D. H., Pace, C. J., Nagy, G., Turner, J. N., and Roysam, B. (2002). Rapid automated

three-dimensional tracing of neurons from confocal image stacks. IEEE Transactions on Information Technologyin Biomedicine, 6:171–187.

Can, A., Shen, H., Turner, J. N., Tanenbaum, H. L., and Roysam, B. (1999). Rapid automated tracing and featureextraction from retinal fundus images using direct exploratory algorithms. IEEE Transactions on InformationTechnology in Biomedicine, 3:125–138.

Haris, K., Efstratiadis, S., Maglaveras, N., Pappas, C., Gourassas, J., and Louridas, G. (1999). Model-based morphologi-cal segmentation and labeling of coronary angiograms. IEEE Trans. Med. Imag., 18:1003–1015.

Jain, V., Murray, J. F., Roth, F., Seung, H. S., Turaga, S., Briggman, K., Denk, W., and Helmstaedter, M. (2007). Usingmachine learning to automate volume reconstruction of neuronal shapes from nanoscale images. In Society forNeuroscience Abstracts. Washington, DC: Society for Neuroscience. Program No. 534.7. Online.

Kalisman, N., Silberberg, G., and Markram, H. (2003). Deriving physical connectivity from neuronal morphology. 88:210–218.

Kwon, J., Mayerich, D., Choe, Y., and McCormick, B. H. (2008). Lateral sectioning for knife-edge scanning microscopy. InProceedings of the IEEE International Symposium on Biomedical Imaging. In press.

Mayerich, D., Abbott, L. C., and Keyser, J. (2008). Visualization of cellular and microvessel relationship. IEEE Transactionson Visualization and Computer Graphics (Proceedings of IEEE Visualization). In press.

35-1

Mayerich, D., and Keyser, J. (2008). Filament tracking and encoding for complex biological networks. In Proceedings ofACM Symposium on Solid and Physical Modeling, 353–358.

Sporns, O., Tononi, G., and Kotter, R. (2005). The human connectome: A structural description of the human brain. PLoSComputational Biology, 1:e42.

White, J. G., Southgate, E., Thomson, J. N., and Brenner, S. (1986). The structure of the nervous system of the nematodecaenorhabditis elegans. Philosophical Transactions of the Royal Society of London B, 314:1–340.

Williams, R. W. (2000). Mapping genes that modulate mouse brain development: a quantitative genetic approach. InGoffinet, A. F., and Rakic, P., editors, Mouse Brain Development, 2149. New York: Springer.

35-2