methods in image analysis – lecture 2 local operators and global transforms
DESCRIPTION
Methods in Image Analysis – Lecture 2 Local Operators and Global Transforms. George Stetten, M.D., Ph.D. CMU Robotics Institute 16-725 U. Pitt Bioengineering 2630 Spring Term, 2004. Preface. Some things work in n dimensions, some don’t. It is often easier to present a concept in 2D. - PowerPoint PPT PresentationTRANSCRIPT
1
Methods in Image Analysis – Lecture 2Local Operators and Global Transforms
CMU Robotics Institute 16-725
U. Pitt Bioengineering 2630
Spring Term, 2004
George Stetten, M.D., Ph.D.
2
Preface
• Some things work in n dimensions, some don’t.• It is often easier to present a concept in 2D.• I will use the word “pixel” for n dimensions.
3
Point Operators
• f is usually monotonic, and shift invariant.• Inverse may not exist.• Brightness/contrast, “windowing”.• Thresholding.• Color Maps.• f may vary with pixel location, eg., correcting for
inhomogeneity of RF field strength in MRI.
( ) ( )[ ]yxIfyxI ,,' =
4
Histogram Equalization
• A pixel-wise intensity mapping is found that produces a uniform density of pixel intensity across the dynamic range.
5
Adaptive Thresholding from Histogram
• Assumes bimodal distribution.• Trough represents boundary points between
homogenous areas.
6
Algebraic Operators
• Assumes registration.• Averaging multiple acquisitions for noise reduction.• Subtracting sequential images for motion detection, or
other changes (eg. Digital Subtractive Angiography).• Masking.
( ) ( ) ( )[ ]yxIyxIgyxI ,,,,' 21=
7
Re-Sampling on a New Lattice
• Can result in denser or sparser pixels.• Two general approaches:
– Forward Mapping (Splatting)
– Backward Mapping (Interpolation)
• Nearest Neighbor
• Bilinear
• Cubic
• 2D and 3D texture mapping hardware acceleration.
8
Neighborhood Operators
• Kernels• Cliques• Markov Random Fields• Must limit the relationships to be practical.
9
Convolution and Correlation
• Template matching uses correlation, the primordial form of image analysis.
• Kernels are mostly used for “convolution” although with symmetrical kernels equivalent to correlation.
• Convolution flips the kernel and does not normalize.• Correlation subtracts the mean and generally does
normalize.
10
Neighborhood PDE Operators
• For discrete images, always requires a specific scale.
• “Inner scale” is the original pixel grid.• Size of the kernel determines scale.• Concept of Scale Space, Course-to-Fine.
11
• Vector• Direction of maximum change of scalar intensity I.• Normal to the boundary.• Nicely n-dimensional.
Intensity Gradient
zyxzyx ˆˆˆˆˆˆ zyx IIIdz
dI
dy
dI
dx
dII ++=++=∇
12
• Scalar• Maximum at the boundary• Orientation-invariant.
Intensity Gradient Magnitude
€
∇I = Ix2 + Iy
2 + Iz2
( ) 1
2 =∇I ⋅ ˆ n
13
IxI
yI I∇
14
Classic Edge Detection Kernel (Sobel)
xI⇒⎥⎥⎥
⎦
⎤
⎢⎢⎢
⎣
⎡
−−−
101202101
yI⇒⎥⎥⎥
⎦
⎤
⎢⎢⎢
⎣
⎡
−−− 121000121
15
Isosurface, Marching Cubes (Lorensen)
• 100% opaque watertight surface• Fast, 28 = 256 combinations, pre-computed
16
• Marching cubes works well with raw CT data.
• Hounsfield units (attenuation).• Threshold calcium density.
17
Direct shading from gradient (Levoy, Drebin)
• Voxels are blended (translucent).• Opacity proportional to gradient magnitude.• Rendering uses gradient direction as “surface” normal.
18
• Ixy = Iyx = curvature• Orientation-invariant.• What about in 3D?
Jacobian of the Intensity Gradient
⎥⎦
⎤⎢⎣
⎡=
⎥⎥⎥⎥
⎦
⎤
⎢⎢⎢⎢
⎣
⎡
yyyx
xyxx
II
II
dy
Id
dxdy
Iddydx
Id
dx
Id
2
22
2
2
2
19
Viewing the intensity as “height”• Differential geometry: the surface and tangent plane • Example: cylindrical surface, curvature = 0.• Move in the y direction, no roll: Ixy = 0• Move in the x direction, no pitch: Iyx = 0
= Ix
= Iy
20
Laplacian of the Intensity
• Divergence of the Gradient.• Zero at the inflection point of the
intensity curve.
222
2
2
22
2
22
2
22
zzyyxx IIIdz
Id
dy
Id
dx
IdI ++=⎟⎟
⎠
⎞⎜⎜⎝
⎛+⎟⎟
⎠
⎞⎜⎜⎝
⎛+⎟⎟
⎠
⎞⎜⎜⎝
⎛=∇
I
Ix
Ixx
⎥⎥⎥
⎦
⎤
⎢⎢⎢
⎣
⎡
−−−−−−−−
111
181
111
21
Difference of Gaussian Operators (DOG)
• Conventionally, 2 concentric Gaussians of different scale.• Acts like a Laplacian,
22
Binomial Kernel
• Repeated averaging of neighbors => Gaussian by Central Limit Theorem.
1 1 1 1 2 1 1 3 3 1 1 4 6 4 1
1 1 2 4 2 3 9 9 3 4 16 24 16 4
1 2 1 3 9 9 3 6 24 3624 6
1 3 3 1 4 16 24 16 4
1 4 6 4 1
23
Binomial Difference of Offset Gaussian (DooG)
• Not the conventional concentric DOG• Subtracting pixels displaced along the x axis after repeated
blurring with binomial kernel yields Ix
-1 0 1 -1 -2 0 2 1 -1 -4 -6 -4 0 4 6 4 1
-2 -4 0 4 2 -4 -16 -24 -16 0 16 24 16 4
-1 -2 0 2 1 -6 -24 -36 -24 0 24 36 24 6
-4 -16 -24 -16 0 16 24 16 4
-1 -4 -6 -4 0 4 6 4 1
24
Boundary Profiles (Tamburo)
• Splatting in an ellipsoid along the gradient.
25
Boundary Profiles
• Fitting the cumulative Gaussian
distance along gradient
μ
σ σ
d d
p1
p2
sampled region ofprofile
μ
1 2intensity
I1
I2
26
Boundary profiles reduce error in location due to sampling.
27
Other basis sets for boundary kernel.
• Partial derivatives of the Gaussian• Wavelets• Statistical texture analysis.
28
Texture Boundaries
• Two regions with the same intensity but differentiated by texture are easily discriminated by the human visual system.
29
Ridges
• Attempt to tie the image structure together in a locally continual manner.
• Along some manifold in less than n dimensions.• Local maximum along the normal to the ridge.• Canny edge detector is a boundariness ridge.• A “core” is a medialness ridge.• Medialness within an object is the property of
being equidistant from two boundaries.
30
Global Transforms in n dimensions
• Geometric (rigid body)
– n translations and rotations.
• Similarity– Add 1 scale (isometric).
• Affine– Add n scales (combined with rotation => skew).
– Parallel lines remain parallel.
• Projection
⎟⎟⎠
⎞⎜⎜⎝
⎛2
n
31
Orthographic Transform Matrix
• Capable of geometric, similarity, or affine.• Homogeneous coordinates.• Multiply in reverse order to combine• SGI “graphics engine” 1982, now standard.
⎥⎥⎥
⎦
⎤
⎢⎢⎢
⎣
⎡
⎥⎥⎥
⎦
⎤
⎢⎢⎢
⎣
⎡=
⎥⎥⎥
⎦
⎤
⎢⎢⎢
⎣
⎡′′
1
10013,22,21,2
3,12,11,1
y
x
aaa
aaa
y
x
32
Translation by (tx , ty)
⎥⎥⎥
⎦
⎤
⎢⎢⎢
⎣
⎡
⎥⎥⎥
⎦
⎤
⎢⎢⎢
⎣
⎡=
⎥⎥⎥
⎦
⎤
⎢⎢⎢
⎣
⎡′′
1
100
10
01
1
y
x
t
t
y
x
y
x
Scale x by sx and y by sy
⎥⎥⎥
⎦
⎤
⎢⎢⎢
⎣
⎡
⎥⎥⎥
⎦
⎤
⎢⎢⎢
⎣
⎡=
⎥⎥⎥
⎦
⎤
⎢⎢⎢
⎣
⎡′′
1
100
00
00
1
y
x
s
s
y
x
y
x
33
• 2 x 2 rotation portion is orthogonal (orthonormal vectors).• Therefore only 1 degree of freedom, .
Rotation in 2D
⎥⎥⎥
⎦
⎤
⎢⎢⎢
⎣
⎡
⎥⎥⎥
⎦
⎤
⎢⎢⎢
⎣
⎡ −=
⎥⎥⎥
⎦
⎤
⎢⎢⎢
⎣
⎡′′
1
100
0cossin
0sincos
1
y
x
y
x
θθ
θθ
θ
34
• 3 x 3 rotation portion is orthogonal (orthonormal vectors).• 3 degree of freedom (dotted circled), , as expected.
Rotation in 3D
⎥⎥⎥⎥
⎦
⎤
⎢⎢⎢⎢
⎣
⎡
⎥⎥⎥⎥
⎦
⎤
⎢⎢⎢⎢
⎣
⎡
=
⎥⎥⎥⎥
⎦
⎤
⎢⎢⎢⎢
⎣
⎡
′′′
1
1000
0
0
0
13,32,31,3
3,22,21,2
2,12,11,1
z
y
x
aaa
aaa
aaa
z
y
x
⎟⎟⎠
⎞⎜⎜⎝
⎛2
n
35
• For X-ray or direct vision, projects onto the (x,y) plane.• Rescales x and y for “perspective” by changing the “1” in
the homogeneous coordinates, as a function of z.
Non-Orthographic Projection in 3D
⎥⎥⎥⎥
⎦
⎤
⎢⎢⎢⎢
⎣
⎡
⎥⎥⎥⎥
⎦
⎤
⎢⎢⎢⎢
⎣
⎡
=
⎥⎥⎥⎥
⎦
⎤
⎢⎢⎢⎢
⎣
⎡
′′′
1
100
0100
0010
0001
1
z
y
x
k
z
y
x
36
Anisotropic Scaling of Vectors (ITK Software Guide 4.25, 4.26)
dx1
dI
dx2
dI
itkCovarientVector objects are used to represent properties such as the gradient, since stretching the image in the x dimension lowers the x gradient component (slope).
€
dx1 > dx2 ⇒dI
dx1
<dI
dx2
itkVector
itkCovarientVector
dx1 dx2
dy
itkVector objects are used to represent distances between locations, velocities, etc.
dx1 dx2
dy