[1] Fan, J. 1998. Notes on Poisson distribution-based minimum error thresholding. Pattern Recogn. Lett. 19(5): 425-431

[2] Boykov, Y. and FUNKA-LEA, G. 2006. Graph Cuts and Efficient N-D Image Segmentation. International Journal of

Computer Vision, 70(2): 109:131

[3] Xiongwu Wu, Yidong Chen, Bernard R. Brooks, and Yan A. Su. 2004. The Local Maximum Clustering Method and

Its Application in Microarray Gene Expression Data Analysis. EURASIP Journal on Applied Signal Processing, 2004(1):

53-63

[4] Boykov, Y., Veksler, O., and Zabih, R. 2001. Fast approximate energy minimization via graph cuts. IEEE

Transactions on Pattern Analysis and Machine Intelligence, 23(11):1222–1239.

• The initial binarization is refined using Graph-Cuts [2].

• Given an observation field of the pixels intensities, find a labeling configuration

that minimizes the following energy function:

where

• The goal then is to find the threshold that minimizes the error criterion function:

FARSIGHT: A Framework for Automated Quantification of 2D and 3D Multi-Parameter Images of Biological Tissues

Yousef Al-Kofahi1, Christopher Bjornsson2, Arunachalam Narayanaswamy1, Badrinath Roysam1, William Shain3

1 Electrical, Computer, and Systems Engineering Department, Rensselaer Polytechnic Institute, Troy, NY 12180-3590

2 Center for Biotechnology and Interdisciplinary Studies, Rensselaer Polytechnic Institute, Troy, NY 12180-3590

3 Laboratory of Nervous System Disorders, Wadsworth Center, Albany, NY 12201-0509.

Abstract

Associative Image Analysis Framework

Experimental Results and Validation

R1

R2FundamentalScienceFundamentalScience

ValidatingTestBEDsValidatingTestBEDs

L1L1

L2L2

L3L3

R3

S1 S4 S5S3S2

Bio-Med Enviro-Civil

We present a systematic ‘divide and conquer’ methodology for analyzing

three-dimensional (3D) multi-parameter images of brain tissue to delineate

and classify key structures, and compute quantitative associations among

them. Automated 3D segmentation and tracing algorithms were used to

delineate cell nuclei, vasculature, and cell processes. The cells are then

classified into four major classes (Neurons, Microglia, Astrocytes and

Endothelials) by using a two-step semi-supervised classifier

� The image is decomposed into 5 channels

� Each channel is segmented/analyzed independently

Segmentation and Classification Output: A Composite 3D rendering

Color Code:

Astrocytes

Microglia

Neurons

Endothelials

Blood Vessels

• Classification ResultsCell classification results using all the cells

501955144Endothelials (245 found)

318310023

833772

282260326

Tota

l Erro

rs

Endoth

elials

Neu

rons

Micro

glia

Astro

cyte

s

Neurons (341 found)

Microglia (85 found)

Astrocytes (348 found)

Tota

l Erro

rs

Endoth

elials

Neu

rons

Micro

glia

Astro

cyte

s

Human

% o

f Erro

rs

SUM 117

Machine

Classification

8.1%

9.4%

9.1%

20.0%

11.5%

601959249Endothelials (245 found)

3810310028

833772

332580326

Tota

l Erro

rs

Endoth

elials

Neu

rons

Micro

glia

Astro

cyte

s

Neurons (341 found)

Microglia (85 found)

Astrocytes (348 found)

Tota

l Erro

rs

Endoth

elials

Neu

rons

Micro

glia

Astro

cyte

s

Human

% o

f Erro

rs

SUM 139

Machine

Classification

9.5%

9.4%

11.1%

24.5%

13.6%

Cell classification results after 5 manual corrections to the training set

Nuclei Feature Vectors

Initial Fuzzy c-means clustering

Class 1:

Neurons

Training Set Extraction

Class 2:

Microglia

(Optional Step): Manual Correction of Training Set Errors

Class 3:

Astrocytes

Class 4:

Endothelials

SVM Training / SVM Classification

Class 1:

Neurons

Class 2:

Microglia

Class 3:

Astrocytes

Class 4:

Endothelials

Association and Feature Extraction

• Associative Features:� Computed for each nucleus to quantify relationships with other channels.

� Examples: A) Neuronal nuclei are surrounded by Nissl signal

B) Microglial nuclei are surrounded by IBA-1 signal

C) Endothelial cells are adjacent vesselsAAA CCB DDC

Initial Binarization

G-C Binarization Refinement

Distance Map Generation

Select A Connected

Component (CC)

Region Adjacency

Graph Coloring

Graph Building/Learning

Graph Cuts

Alpha-expansions

Last

CC?

NO Yes

Input Image

Binarization & Seeds Detection Cells Separation

Segmentation Output

Local Maximum

Initial Clustering

Multi-scale LoG filtering

Seeds Detection

Details of the Nuclear Segmentation Algorithm Cell Classification

References

Spectral Un-mixing

NisslChannel

Iba-1Channel

TraceProcesses

GFAPChannel

TraceProcesses

EBAChannel

Surface

Segmentation

DAPIChannel

NuclearSegmentation

5-channel 3-D image of 100

um thick rat hippocampal

slice:blue: Nuclei

purple: Nissl

green: Vasculature

red: astrocytes processes

yellow: microglia.

Association And Feature Extraction

Cell Classification

• Intrinsic Features:� A set of 23 morphometric, topological, and intensity- based features for each nucleus.

• Image Binarization

• The image is initially binariezed using the Poisson distribution-based minimum

error thresholding [1]

• The normalized image histogram is modeled by a mixture of Poisson distributions:2 2

( )

1 1

1( ) ( ) ( | ) ( ) ( ( )) , {0,1}

!

j t t

j j j

j j

p t P t p t j P t e t jt

µµ

−

= =

= = =∑ ∑

O

*t

*L

, ( )

( ) ( ; ) ( , )x x x y

x x y N x

Energy L D L O V L L∈

= +∑ ∑

( ) ( )0 0 0 0 1 1 1 1( ) ( ) ln ( ) ( ) ln ( ) ( ) ln ( ) ( ) ln ( )J t P t P t t t P t P t t tµ µ µ µ µ= − + − +

( { , }; ) ln ( | {0,1})x x x

D L Source Sink O p t j= = − =

( , ) ( , ) ( , )x y x y

V L L B x y L Lδ= ⋅2

2

1, if( ) ( )where ( , ) , and ( , ) exp

0, if( ) 2

x y x y

x y

x y

L L t tL L B x y

L Lδ

σ

≠ −= = − =

• Seeds Detection and Initial Segmentation

• Cells are detected using a scale-normalized Laplacian of Gaussian (LoG) filter

• The filter is used on different scales:2 2

2

min max2 2

( , ; ) ( , ; )( , ; ) , [ ,..., ]

norm

G x y G x yLoG x y

x y

σ σσ σ σ σ σ

∂ ∂= + =

∂ ∂ • Assuming the distance from a point to the background is

• The response at that point in the resulting LoG-response image is given by:

{ }min maxwhere max , min{ , ( )}

MAXDmap xσ σ σ=

x ( )Dmap x

( )R

( )min[ , ]

( ) arg max ( ; ) ( )MAX

normR x LoG x I xσ σ σ

σ∈

= ∗

R

• Graph Coloring and Segmentation Refinement

• Initial segmentation is refined using a Graph-Cuts based approach ( ) [4].

• The number of alpha-expansions required is equal to the number of labels (cells)

�Problem: Impractical when the number of labels in a connected component is large

�Solution: Reduce the number of labels using Graph Coloring:

1. Build a Region Adjacency Graph (RAG) from the initial segmentation

2. Use a sequential graph coloring algorithm to assign different colors to adjacent cells

expansionα −

{ }( | ) max ( ; , ) | ( ) x i ip x L j G x color i jµ= = ∑ =

x R∈thj

and( ; ) ln ( | ) ( , ) ( , ) ( , )x x x x y x yD L j O p x L j V L L B x y L Lδ= = − = = ⋅

( )1, if( )

where ( , ) , and ( , ) exp | |0, if( )

x y

x y x y

x y

L LL L B x y t t

L Lδ

≠= = − −

=

• The energy function terms are given as follows:

This work was supported in part by Gordon-CenSSIS, The Bernard M. Gordon Center for Subsurface

Sensing and Imaging Systems, under the Engineering Research Centers Program of the National

Science Foundation (Award Number EEC-9986821).

Output

Overview of the Nuclear Segmentation Algorithm

• gives a topographical surface, where a cell is

represented by a Gaussian blob

• Each blob (nucleus) has one local maximum

point called “seed point”

• Local maximum clustering [3] is used to

assign each foreground point to one seed

• A Gaussian model is used to represent each cell

• The probability for a point to be assigned the color is:

Initial Segmentation with 513 nuclei Coloring output with 10 colors