[1] Fan, J. 1998. Notes on Poisson distribution-based minimum error thresholding. Pattern Recogn. Lett. 19(5): 425-431
[2] Boykov, Y. and FUNKA-LEA, G. 2006. Graph Cuts and Efficient N-D Image Segmentation. International Journal of
Computer Vision, 70(2): 109:131
[3] Xiongwu Wu, Yidong Chen, Bernard R. Brooks, and Yan A. Su. 2004. The Local Maximum Clustering Method and
Its Application in Microarray Gene Expression Data Analysis. EURASIP Journal on Applied Signal Processing, 2004(1):
53-63
[4] Boykov, Y., Veksler, O., and Zabih, R. 2001. Fast approximate energy minimization via graph cuts. IEEE
Transactions on Pattern Analysis and Machine Intelligence, 23(11):1222–1239.
• The initial binarization is refined using Graph-Cuts [2].
• Given an observation field of the pixels intensities, find a labeling configuration
that minimizes the following energy function:
where
• The goal then is to find the threshold that minimizes the error criterion function:
FARSIGHT: A Framework for Automated Quantification of 2D and 3D Multi-Parameter Images of Biological Tissues
Yousef Al-Kofahi1, Christopher Bjornsson2, Arunachalam Narayanaswamy1, Badrinath Roysam1, William Shain3
1 Electrical, Computer, and Systems Engineering Department, Rensselaer Polytechnic Institute, Troy, NY 12180-3590
2 Center for Biotechnology and Interdisciplinary Studies, Rensselaer Polytechnic Institute, Troy, NY 12180-3590
3 Laboratory of Nervous System Disorders, Wadsworth Center, Albany, NY 12201-0509.
Abstract
Associative Image Analysis Framework
Experimental Results and Validation
R1
R2FundamentalScienceFundamentalScience
ValidatingTestBEDsValidatingTestBEDs
L1L1
L2L2
L3L3
R3
S1 S4 S5S3S2
Bio-Med Enviro-Civil
We present a systematic ‘divide and conquer’ methodology for analyzing
three-dimensional (3D) multi-parameter images of brain tissue to delineate
and classify key structures, and compute quantitative associations among
them. Automated 3D segmentation and tracing algorithms were used to
delineate cell nuclei, vasculature, and cell processes. The cells are then
classified into four major classes (Neurons, Microglia, Astrocytes and
Endothelials) by using a two-step semi-supervised classifier
� The image is decomposed into 5 channels
� Each channel is segmented/analyzed independently
Segmentation and Classification Output: A Composite 3D rendering
Color Code:
Astrocytes
Microglia
Neurons
Endothelials
Blood Vessels
• Classification ResultsCell classification results using all the cells
501955144Endothelials (245 found)
318310023
833772
282260326
Tota
l Erro
rs
Endoth
elials
Neu
rons
Micro
glia
Astro
cyte
s
Neurons (341 found)
Microglia (85 found)
Astrocytes (348 found)
Tota
l Erro
rs
Endoth
elials
Neu
rons
Micro
glia
Astro
cyte
s
Human
% o
f Erro
rs
SUM 117
Machine
Classification
8.1%
9.4%
9.1%
20.0%
11.5%
601959249Endothelials (245 found)
3810310028
833772
332580326
Tota
l Erro
rs
Endoth
elials
Neu
rons
Micro
glia
Astro
cyte
s
Neurons (341 found)
Microglia (85 found)
Astrocytes (348 found)
Tota
l Erro
rs
Endoth
elials
Neu
rons
Micro
glia
Astro
cyte
s
Human
% o
f Erro
rs
SUM 139
Machine
Classification
9.5%
9.4%
11.1%
24.5%
13.6%
Cell classification results after 5 manual corrections to the training set
Nuclei Feature Vectors
Initial Fuzzy c-means clustering
Class 1:
Neurons
Training Set Extraction
Class 2:
Microglia
(Optional Step): Manual Correction of Training Set Errors
Class 3:
Astrocytes
Class 4:
Endothelials
SVM Training / SVM Classification
Class 1:
Neurons
Class 2:
Microglia
Class 3:
Astrocytes
Class 4:
Endothelials
Association and Feature Extraction
• Associative Features:� Computed for each nucleus to quantify relationships with other channels.
� Examples: A) Neuronal nuclei are surrounded by Nissl signal
B) Microglial nuclei are surrounded by IBA-1 signal
C) Endothelial cells are adjacent vesselsAAA CCB DDC
Initial Binarization
G-C Binarization Refinement
Distance Map Generation
Select A Connected
Component (CC)
Region Adjacency
Graph Coloring
Graph Building/Learning
Graph Cuts
Alpha-expansions
Last
CC?
NO Yes
Input Image
Binarization & Seeds Detection Cells Separation
Segmentation Output
Local Maximum
Initial Clustering
Multi-scale LoG filtering
Seeds Detection
Details of the Nuclear Segmentation Algorithm Cell Classification
References
Spectral Un-mixing
NisslChannel
Iba-1Channel
TraceProcesses
GFAPChannel
TraceProcesses
EBAChannel
Surface
Segmentation
DAPIChannel
NuclearSegmentation
5-channel 3-D image of 100
um thick rat hippocampal
slice:blue: Nuclei
purple: Nissl
green: Vasculature
red: astrocytes processes
yellow: microglia.
Association And Feature Extraction
Cell Classification
• Intrinsic Features:� A set of 23 morphometric, topological, and intensity- based features for each nucleus.
• Image Binarization
• The image is initially binariezed using the Poisson distribution-based minimum
error thresholding [1]
• The normalized image histogram is modeled by a mixture of Poisson distributions:2 2
( )
1 1
1( ) ( ) ( | ) ( ) ( ( )) , {0,1}
!
j t t
j j j
j j
p t P t p t j P t e t jt
µµ
−
= =
= = =∑ ∑
O
*t
*L
, ( )
( ) ( ; ) ( , )x x x y
x x y N x
Energy L D L O V L L∈
= +∑ ∑
( ) ( )0 0 0 0 1 1 1 1( ) ( ) ln ( ) ( ) ln ( ) ( ) ln ( ) ( ) ln ( )J t P t P t t t P t P t t tµ µ µ µ µ= − + − +
( { , }; ) ln ( | {0,1})x x x
D L Source Sink O p t j= = − =
( , ) ( , ) ( , )x y x y
V L L B x y L Lδ= ⋅2
2
1, if( ) ( )where ( , ) , and ( , ) exp
0, if( ) 2
x y x y
x y
x y
L L t tL L B x y
L Lδ
σ
≠ −= = − =
• Seeds Detection and Initial Segmentation
• Cells are detected using a scale-normalized Laplacian of Gaussian (LoG) filter
• The filter is used on different scales:2 2
2
min max2 2
( , ; ) ( , ; )( , ; ) , [ ,..., ]
norm
G x y G x yLoG x y
x y
σ σσ σ σ σ σ
∂ ∂= + =
∂ ∂ • Assuming the distance from a point to the background is
• The response at that point in the resulting LoG-response image is given by:
{ }min maxwhere max , min{ , ( )}
MAXDmap xσ σ σ=
x ( )Dmap x
( )R
( )min[ , ]
( ) arg max ( ; ) ( )MAX
normR x LoG x I xσ σ σ
σ∈
= ∗
R
• Graph Coloring and Segmentation Refinement
• Initial segmentation is refined using a Graph-Cuts based approach ( ) [4].
• The number of alpha-expansions required is equal to the number of labels (cells)
�Problem: Impractical when the number of labels in a connected component is large
�Solution: Reduce the number of labels using Graph Coloring:
1. Build a Region Adjacency Graph (RAG) from the initial segmentation
2. Use a sequential graph coloring algorithm to assign different colors to adjacent cells
expansionα −
{ }( | ) max ( ; , ) | ( ) x i ip x L j G x color i jµ= = ∑ =
x R∈thj
and( ; ) ln ( | ) ( , ) ( , ) ( , )x x x x y x yD L j O p x L j V L L B x y L Lδ= = − = = ⋅
( )1, if( )
where ( , ) , and ( , ) exp | |0, if( )
x y
x y x y
x y
L LL L B x y t t
L Lδ
≠= = − −
=
• The energy function terms are given as follows:
This work was supported in part by Gordon-CenSSIS, The Bernard M. Gordon Center for Subsurface
Sensing and Imaging Systems, under the Engineering Research Centers Program of the National
Science Foundation (Award Number EEC-9986821).
Output
Overview of the Nuclear Segmentation Algorithm
• gives a topographical surface, where a cell is
represented by a Gaussian blob
• Each blob (nucleus) has one local maximum
point called “seed point”
• Local maximum clustering [3] is used to
assign each foreground point to one seed
• A Gaussian model is used to represent each cell
• The probability for a point to be assigned the color is:
Initial Segmentation with 513 nuclei Coloring output with 10 colors