multi-dimensional image analysis - petra christian university
TRANSCRIPT
![Page 1: Multi-dimensional Image Analysis - Petra Christian University](https://reader031.vdocuments.us/reader031/viewer/2022012219/61e00c710643d75b1c33c4b0/html5/thumbnails/1.jpg)
Center for Image Processing
Multi-dimensional Image Analysis
Lucas J. van Vlietwww.ph.tn.tudelft.nl/~lucas
![Page 2: Multi-dimensional Image Analysis - Petra Christian University](https://reader031.vdocuments.us/reader031/viewer/2022012219/61e00c710643d75b1c33c4b0/html5/thumbnails/2.jpg)
Center for Image ProcessingCenter for Image Processing
CBP course: Multi-Dimensional Image Analysis 2
Image Analysis Paradigm
Texture filtering
analysis
segmentation
Imagerestoration
Imageenhancement
sensorImageformation scene
pre-processing
classification
Measurements:Point: edge location, isophote curvaturesGlobal: size and shape descriptorsLocal: texture attributes anisotropy, orientation, scale
![Page 3: Multi-dimensional Image Analysis - Petra Christian University](https://reader031.vdocuments.us/reader031/viewer/2022012219/61e00c710643d75b1c33c4b0/html5/thumbnails/3.jpg)
Center for Image ProcessingCenter for Image Processing
CBP course: Multi-Dimensional Image Analysis 3
Multi-dimensional analysis
Goal: Use sampling-error free measurements for analog properties in digitized images
� Point measurements for object identification� Exact boundary location� (Principal) curvatures
� Global measurement for object description� Integrated object intensity (gray-volume)� Size in 2D: area and perimeter� Size in 3D: volume, surface area, length� Shape: Bending energy, Euler numbers
� Local texture analysis� Anisotropy, orientation, scale
Use gray-scale rather than binary operations to obtain high accuracy and precision
![Page 4: Multi-dimensional Image Analysis - Petra Christian University](https://reader031.vdocuments.us/reader031/viewer/2022012219/61e00c710643d75b1c33c4b0/html5/thumbnails/4.jpg)
Center for Image ProcessingCenter for Image Processing
CBP course: Multi-Dimensional Image Analysis 4
Better geometric measurements
� State-of-the-art cameras offer:� Large images of 1000 x 1000 pixels or more� 12 – 16 bit photometric information
� A binary image is disturbed by aliasing� Thresholding corrupts data irreversible
Faithful representation Binary representation
![Page 5: Multi-dimensional Image Analysis - Petra Christian University](https://reader031.vdocuments.us/reader031/viewer/2022012219/61e00c710643d75b1c33c4b0/html5/thumbnails/5.jpg)
Center for Image ProcessingCenter for Image Processing
CBP course: Multi-Dimensional Image Analysis 5
Location of curved edges
� Zero-crossing of second derivatives applied to blurred curved edges are biased.� Laplace: Dxx + Dyy = Dcc + Dgg outwards� in grad direction: Dgg inwards� PLUS 2Dgg + Dcc on edge !
� Results on an ellipse: object with slowly varying curvature.
PLUS
SDGDLaplace
PLUS
x
y
g
cg
cg
c g = gradientc = contour
![Page 6: Multi-dimensional Image Analysis - Petra Christian University](https://reader031.vdocuments.us/reader031/viewer/2022012219/61e00c710643d75b1c33c4b0/html5/thumbnails/6.jpg)
Center for Image ProcessingCenter for Image Processing
CBP course: Multi-Dimensional Image Analysis 6
Isophote curvature in 2D
An isophote is a curve of constant gray-value (level curve)� Curvature is the change of contour direction per unit length� Curvature (κ=1/R) with R the radius of the osculating circle
cc
g
DD
κ
−
=object contour
g =(Dx,Dy)
c =(-Dy,Dx) Usage: corner detection,dominant points, bending energy
![Page 7: Multi-dimensional Image Analysis - Petra Christian University](https://reader031.vdocuments.us/reader031/viewer/2022012219/61e00c710643d75b1c33c4b0/html5/thumbnails/7.jpg)
Center for Image ProcessingCenter for Image Processing
CBP course: Multi-Dimensional Image Analysis 7
Isophote curvature in 3D
� 2D isophote surface patch in 3D image space� Principal curvatures κ1 and κ2 along c1 and c2: 1 2c c g⊥ ⊥
1 11
c c
g
DD
κ
−
=2 2
2c c
g
DD
κ
−
=
1 2 0κ κ> > 2 1 0κ κ< < 1 20, 0κ κ> =
1 20, 0κ κ> < 1 2 0κ κ= =
Usage for local shape: elliptic (convex, concave), cylinder,saddle, flat
![Page 8: Multi-dimensional Image Analysis - Petra Christian University](https://reader031.vdocuments.us/reader031/viewer/2022012219/61e00c710643d75b1c33c4b0/html5/thumbnails/8.jpg)
Center for Image ProcessingCenter for Image Processing
CBP course: Multi-Dimensional Image Analysis 8
Sampling-error free measures
� A sampling-error free measurement is a digital measurement performed on a sampled image that is exactly equal to its analog counterpart
The “sum” of all samples is measured without thresholding and does not introduce a sampling error
( ) ( ) ( ) ( )2
12
sum , 2 0,0
iff :x y x y
sampling Nyquist
b b i j B
f f
π= ∆ ∆ ∆ ∆ =
>
∑
Digital track
=
Analog track
reconstruction
operation in analog space
samplingoperation in digital space
![Page 9: Multi-dimensional Image Analysis - Petra Christian University](https://reader031.vdocuments.us/reader031/viewer/2022012219/61e00c710643d75b1c33c4b0/html5/thumbnails/9.jpg)
Center for Image ProcessingCenter for Image Processing
CBP course: Multi-Dimensional Image Analysis 9
Sum() as measure for …
� Sum() is a sampling-error free measure
� Recipe for object measurements in gray-scale images:1. Transform the input image with the object into an output image
whose sum() is directly proportional to the feature to be measured
2. The transformation must consist of sampling-error free operations
3. Proper sampling is required to avoid aliasing
4. Bias correction terms can be extracted from the mathematical framework! (not empirically)
![Page 10: Multi-dimensional Image Analysis - Petra Christian University](https://reader031.vdocuments.us/reader031/viewer/2022012219/61e00c710643d75b1c33c4b0/html5/thumbnails/10.jpg)
Center for Image ProcessingCenter for Image Processing
CBP course: Multi-Dimensional Image Analysis 10
2D area & 3D volume
� Clip image a to produce a flat object on a flat background� Use “soft-clipping” to avoid aliasing
output after:
thresholding
hard-clipping
erf-clipping( )
( )
erf
2
3
clip
sumD
D
b a
Ab b
V
=
= =∑
linear input slope
![Page 11: Multi-dimensional Image Analysis - Petra Christian University](https://reader031.vdocuments.us/reader031/viewer/2022012219/61e00c710643d75b1c33c4b0/html5/thumbnails/11.jpg)
Center for Image ProcessingCenter for Image Processing
CBP course: Multi-Dimensional Image Analysis 11
2D perimeter, 3D surface area
� Transform the “flat”, bandlimited object into a contour
Analytical dilation / erosion by Taylor series around a smooth edge (with height = 1) over a distance δ =½
( ) ( )
( ) ( )
12
212
212
:
:
:
g gg
g gg
g
dilation b r b r b b
erosion b r b r b b
contour bδ
δ δ δ
δ δ δ
=
+ ≈ + +
− ≈ − + −
≈
a
b
|bg |
![Page 12: Multi-dimensional Image Analysis - Petra Christian University](https://reader031.vdocuments.us/reader031/viewer/2022012219/61e00c710643d75b1c33c4b0/html5/thumbnails/12.jpg)
Center for Image ProcessingCenter for Image Processing
CBP course: Multi-Dimensional Image Analysis 12
Shape: 2D bending energy
� 2D bending energy proportional to the bending energy of a deformed circular rod (Young ’74).
� The simply-connected, closed contour with the minimum bending energy is the circle (2π/R, not scale invariant).
b |bg| κ κ2
Differential geometry Image analysis
( )2 2be be gE s ds E bκ κ= =∑∫�
![Page 13: Multi-dimensional Image Analysis - Petra Christian University](https://reader031.vdocuments.us/reader031/viewer/2022012219/61e00c710643d75b1c33c4b0/html5/thumbnails/13.jpg)
Center for Image ProcessingCenter for Image Processing
CBP course: Multi-Dimensional Image Analysis 13
Shape: 3D bending energy
� Elastic rods (SCC space curves)� κ1 ⇒ cross-section of rod� κ2 ⇒ trajectory of rod
� circle has minimal bending energy
� Torque forces are neglected
� Deflected thin plates (SCC surfaces)� principal curvatures κ1 and κ2
� Poisson’s ratio p [0 ,½]. (let p = 0)� sphere has minimal energy (8π). � Dimensionless and therefore scaling
invariant.
( )22rodE s dsκ= ∫� ( )2 2
1 28plateE p dSπ κ κ= + +∫∫�
22rod gE bκ=∑ ( )2 2
1 2plate gE bκ κ= +∑
Differential geometry
Image analysis
![Page 14: Multi-dimensional Image Analysis - Petra Christian University](https://reader031.vdocuments.us/reader031/viewer/2022012219/61e00c710643d75b1c33c4b0/html5/thumbnails/14.jpg)
Center for Image ProcessingCenter for Image Processing
CBP course: Multi-Dimensional Image Analysis 14
Length in 3D
Resume: An erf-clipped object of unit intensity, bvolume: V = Σ bsurface area: A = Σ |bg|length: L = ??
� image b contains a cylinder (length L,radius R)Σ b = πR2 L = volumeΣ bg = 2πRL = areaΣ bgg = 2πL = length
� Length of spaghetti
� In the plane perpendicular to the string: g,cΣ bgg + bcc = 0 all g • c = 0Σ bgg = –Σ bcc = 2π (see Euler)
13 2D ggL b
π
= ∑
b
bcc
bgg
![Page 15: Multi-dimensional Image Analysis - Petra Christian University](https://reader031.vdocuments.us/reader031/viewer/2022012219/61e00c710643d75b1c33c4b0/html5/thumbnails/15.jpg)
Center for Image ProcessingCenter for Image Processing
CBP course: Multi-Dimensional Image Analysis 15
Shape: Euler numbers
� Euler numbers characterize the topology2D: number of objects – number of holes3D: number of objects – number of handles
(donut & coffee cup have Euler number = 0)
Differential geometry Image analysis
Hopf:
Gauss-Bonnet:
( ) 12 2
11 2 3 1 24
2
4
D g
D g
s ds N b
dS N b
π
π
κ π κ
κ κ π κ κ
= =
= =
∑∫
∑∫∫
�
�
b
bg
κ
![Page 16: Multi-dimensional Image Analysis - Petra Christian University](https://reader031.vdocuments.us/reader031/viewer/2022012219/61e00c710643d75b1c33c4b0/html5/thumbnails/16.jpg)
Center for Image ProcessingCenter for Image Processing
CBP course: Multi-Dimensional Image Analysis 16
Curvilinear structures
10 km 10 cm
1 cm
10 cm
∆z = 10 µm
� Anisotropy� Orientation� Scale� Curvature
![Page 17: Multi-dimensional Image Analysis - Petra Christian University](https://reader031.vdocuments.us/reader031/viewer/2022012219/61e00c710643d75b1c33c4b0/html5/thumbnails/17.jpg)
Center for Image ProcessingCenter for Image Processing
CBP course: Multi-Dimensional Image Analysis 17
Domains vs Scale
� Texture attributes are domain properties
![Page 18: Multi-dimensional Image Analysis - Petra Christian University](https://reader031.vdocuments.us/reader031/viewer/2022012219/61e00c710643d75b1c33c4b0/html5/thumbnails/18.jpg)
Center for Image ProcessingCenter for Image Processing
CBP course: Multi-Dimensional Image Analysis 18
Orientation and Scale
� Rotational invariant chirp image
Orientation mapby Gradient Structure Tensor
Scale mapby Gaussian Scale Space
![Page 19: Multi-dimensional Image Analysis - Petra Christian University](https://reader031.vdocuments.us/reader031/viewer/2022012219/61e00c710643d75b1c33c4b0/html5/thumbnails/19.jpg)
Center for Image ProcessingCenter for Image Processing
CBP course: Multi-Dimensional Image Analysis 19
Gradient Structure Tensor
� How to combine vectors?
ϕ
λ 1
λ 2
= ⋅ =
2
2
x x yt
x y y
f f f
f f fG g g
xf
yf
![Page 20: Multi-dimensional Image Analysis - Petra Christian University](https://reader031.vdocuments.us/reader031/viewer/2022012219/61e00c710643d75b1c33c4b0/html5/thumbnails/20.jpg)
Center for Image ProcessingCenter for Image Processing
CBP course: Multi-Dimensional Image Analysis 20
GST: anisotropy, orientation
� Closed-form solutions for:
12 2
2tan x y
x y
f f
f fϕ
−
= −
1 2
1 2anisotropy: A λ λ
λ λ
−=
+
( ) ( )
( ) ( )
2 22 2 2 211 2
2 22 2 2 212 2
4
4
x y x y x y
x y x y x y
f f f f f f
f f f f f f
λ
λ
= + + − +
= + − − +
ϕ
λ1 λ2
A
![Page 21: Multi-dimensional Image Analysis - Petra Christian University](https://reader031.vdocuments.us/reader031/viewer/2022012219/61e00c710643d75b1c33c4b0/html5/thumbnails/21.jpg)
Center for Image ProcessingCenter for Image Processing
CBP course: Multi-Dimensional Image Analysis 21
PVC particle roughness
� Contour information (shape features) failed to rank batches according to “quality”
� Lobes show up as lines rather than points� Measure the local anisotropy (ellipses)
![Page 22: Multi-dimensional Image Analysis - Petra Christian University](https://reader031.vdocuments.us/reader031/viewer/2022012219/61e00c710643d75b1c33c4b0/html5/thumbnails/22.jpg)
Center for Image ProcessingCenter for Image Processing
CBP course: Multi-Dimensional Image Analysis 22
Roughness = Integrated anisotropy
� Three scales: gradient, window, particle
λ
λ= − =
∑∑
2
11 0.44R
λ
λ= − =
∑∑
2
11 0.61R
![Page 23: Multi-dimensional Image Analysis - Petra Christian University](https://reader031.vdocuments.us/reader031/viewer/2022012219/61e00c710643d75b1c33c4b0/html5/thumbnails/23.jpg)
Center for Image ProcessingCenter for Image Processing
CBP course: Multi-Dimensional Image Analysis 23
Roughness distributions
� (Cumulative) distributions for N=30 particles can be used as roughness measure
� Roughness measure correlates well with product quality
0.39 0.42 0.45 0.48 0.51 0.54 0.57 0.
6
0.63 0.66 0.69 0.72
smoothmiddle
rough
0
2
4
6
8
10
12
coun
t
roughness
particle roughness histograms
smoothmiddlerough
0.39 0.42 0.45 0.48 0.51 0.54 0.57 0.60 0.63 0.66 0.69 0.72
smoothmiddle
rough0%
25%
50%
75%
100%
roughness
cumulative particle roughness histograms
![Page 24: Multi-dimensional Image Analysis - Petra Christian University](https://reader031.vdocuments.us/reader031/viewer/2022012219/61e00c710643d75b1c33c4b0/html5/thumbnails/24.jpg)
Center for Image ProcessingCenter for Image Processing
CBP course: Multi-Dimensional Image Analysis 24
Orientation-driven analysis
� Dominant orientation of Gradient Structure Tensor� Strongest peak in orientation space
Dominant orientationof GST yields the gradient2-weightedorientation map.
Strongest orientationin ϕ-space allows veryabrupt changes inorientation map.
![Page 25: Multi-dimensional Image Analysis - Petra Christian University](https://reader031.vdocuments.us/reader031/viewer/2022012219/61e00c710643d75b1c33c4b0/html5/thumbnails/25.jpg)
Center for Image ProcessingCenter for Image Processing
CBP course: Multi-Dimensional Image Analysis 25
Orientation space
� Decompose the image into narrow orientation bands
� Apply a nonlinear operator in orientation space:� Labeling yields segmentation� Peak selection for detection
ΦΦΦΦ ΦΦΦΦ−−−−1111
ΦΦΦΦ
selectivity
![Page 26: Multi-dimensional Image Analysis - Petra Christian University](https://reader031.vdocuments.us/reader031/viewer/2022012219/61e00c710643d75b1c33c4b0/html5/thumbnails/26.jpg)
Center for Image ProcessingCenter for Image Processing
CBP course: Multi-Dimensional Image Analysis 26
Circle in image-space …
� A circle in 2D image space becomes a double-helix in orientation space.
� Note that orientation-axis is periodic with π
![Page 27: Multi-dimensional Image Analysis - Petra Christian University](https://reader031.vdocuments.us/reader031/viewer/2022012219/61e00c710643d75b1c33c4b0/html5/thumbnails/27.jpg)
Center for Image ProcessingCenter for Image Processing
CBP course: Multi-Dimensional Image Analysis 27
Overlapping circles
� Two overlapping circles in 2D image space compose a single object.
� Since the circles cross at different orientations, they become separated in orientation space.
![Page 28: Multi-dimensional Image Analysis - Petra Christian University](https://reader031.vdocuments.us/reader031/viewer/2022012219/61e00c710643d75b1c33c4b0/html5/thumbnails/28.jpg)
Center for Image ProcessingCenter for Image Processing
CBP course: Multi-Dimensional Image Analysis 28
Orientation selectivity
� Green = first peak, Red = second peak, Blue = remainder
N=8 N=16 N=32
orientation
N=8
N=16
![Page 29: Multi-dimensional Image Analysis - Petra Christian University](https://reader031.vdocuments.us/reader031/viewer/2022012219/61e00c710643d75b1c33c4b0/html5/thumbnails/29.jpg)
Center for Image ProcessingCenter for Image Processing
CBP course: Multi-Dimensional Image Analysis 29
Multi-Scale
Series of images filtered of decreasing scale: Scale-space� Sample the scales logarithmically using filters of size = basescale
yields n scales per octave
Input imagescale 0
scale 1
scale 2
scale 3
scale 4
scale 5
var (scale 1)
var (scale 2)
var (scale 3)
var (scale 4)
var (scale 5)
Local variance between scales n and n-1.
{ }11 13212 ,2 ,2 ,...,2 nbase ∈
Scale difference ≈ scale derivative
Scal
e sp
ace
Scal
e sp
ace
![Page 30: Multi-dimensional Image Analysis - Petra Christian University](https://reader031.vdocuments.us/reader031/viewer/2022012219/61e00c710643d75b1c33c4b0/html5/thumbnails/30.jpg)
Center for Image ProcessingCenter for Image Processing
CBP course: Multi-Dimensional Image Analysis 30
Chirp exampleSc
ale-
spac
e
Scale derivative Spatial variance
Scal
e-sp
ace
![Page 31: Multi-dimensional Image Analysis - Petra Christian University](https://reader031.vdocuments.us/reader031/viewer/2022012219/61e00c710643d75b1c33c4b0/html5/thumbnails/31.jpg)
Center for Image ProcessingCenter for Image Processing
CBP course: Multi-Dimensional Image Analysis 31
Gaussian scale space
� Chirp of varying contrast� Low-pass filters of increasing scale:
color code (fine to course)� Normalization: sum per pixel is constant
![Page 32: Multi-dimensional Image Analysis - Petra Christian University](https://reader031.vdocuments.us/reader031/viewer/2022012219/61e00c710643d75b1c33c4b0/html5/thumbnails/32.jpg)
Center for Image ProcessingCenter for Image Processing
CBP course: Multi-Dimensional Image Analysis 32
Scale analysis
� Scale information reveals geological structures in seismic data
Absolute Normalized
![Page 33: Multi-dimensional Image Analysis - Petra Christian University](https://reader031.vdocuments.us/reader031/viewer/2022012219/61e00c710643d75b1c33c4b0/html5/thumbnails/33.jpg)
Center for Image ProcessingCenter for Image Processing
CBP course: Multi-Dimensional Image Analysis 33
Morphological scale space
� Measure the hole-size distributionThe image acts like a sieve
� Subtract closings of increasing scaleThe hole-size is the scale that closes the gap
2 4 8 16 32 64 128 256
Scale256 x 256
![Page 34: Multi-dimensional Image Analysis - Petra Christian University](https://reader031.vdocuments.us/reader031/viewer/2022012219/61e00c710643d75b1c33c4b0/html5/thumbnails/34.jpg)
Center for Image ProcessingCenter for Image Processing
CBP course: Multi-Dimensional Image Analysis 34
Labeling of holes by size
2 4 8 16 32 64 128 256 2 4 8 16 32 64 128 256
![Page 35: Multi-dimensional Image Analysis - Petra Christian University](https://reader031.vdocuments.us/reader031/viewer/2022012219/61e00c710643d75b1c33c4b0/html5/thumbnails/35.jpg)
Center for Image ProcessingCenter for Image Processing
CBP course: Multi-Dimensional Image Analysis 35
Pore-size distribution
2 4 8 16 32 64 128 256
Milk + substrateMilk + substrate + enzime C
Milk
Milk (blue) + substate (red) + enzyme (green)
Average pore-size distributions imagesN=32
![Page 36: Multi-dimensional Image Analysis - Petra Christian University](https://reader031.vdocuments.us/reader031/viewer/2022012219/61e00c710643d75b1c33c4b0/html5/thumbnails/36.jpg)
Center for Image ProcessingCenter for Image Processing
CBP course: Multi-Dimensional Image Analysis 36
Literature
pdf files available at: www.ph.tn.tudelft.nl/~lucas
� L.J. van Vliet and P.W. Verbeek, Better geometric measurements based on photometric information, Proc. IEEE Instrumentation and Measurement Technology Conf. IMTC94 (Hamamatsu, Japan, May 10-12), 1994, 1357-1360.
� G.M.P. van Kempen et al., The application of a local dimensionality estimator to the analysis of 3D microscopic network structures, in: SCIA'99, Proc. 11th Scandinavian Conference on Image Analysis (Kangerlussuaq, Greenland, June 7-11), 1999, 447-455.
� M. van Ginkel et al., Improved Orientation Selectivity for Orientation Estimation, in: SCIA'97, Proc. 10th Scandinavian Conference on Image Analysis (Lappeenranta, Finland, June 9-11), 1997, 533-537.