![Page 1: Image Processing and Computer Vision · Visual Computing: JoachimM.Buhmann — Computer Vision and Image Processing 27/274 •fˆ(u) is also called the continuous spectrum of f(x)](https://reader034.vdocuments.us/reader034/viewer/2022051814/6034ddef7b286f70f036e6b5/html5/thumbnails/1.jpg)
Image Processing andComputer Vision
Visual Computing: Joachim M. Buhmann — Computer Vision and Image Processing 1/274
![Page 2: Image Processing and Computer Vision · Visual Computing: JoachimM.Buhmann — Computer Vision and Image Processing 27/274 •fˆ(u) is also called the continuous spectrum of f(x)](https://reader034.vdocuments.us/reader034/viewer/2022051814/6034ddef7b286f70f036e6b5/html5/thumbnails/2.jpg)
Computer Vision
What is computer vision? Interpreting images!
10011101001010100000001000001010101101110100110110001000101111010001000010100110101111111010010001...
=
The computer sees 1001110100101010000000001110101...
Visual Computing: Joachim M. Buhmann — Computer Vision and Image Processing 3/274
![Page 3: Image Processing and Computer Vision · Visual Computing: JoachimM.Buhmann — Computer Vision and Image Processing 27/274 •fˆ(u) is also called the continuous spectrum of f(x)](https://reader034.vdocuments.us/reader034/viewer/2022051814/6034ddef7b286f70f036e6b5/html5/thumbnails/3.jpg)
Image Processing
What is image processing? Restoring images without ex-traction of semantic information!filtering of images
blurring + noise
optimal linearfiltering
Visual Computing: Joachim M. Buhmann — Computer Vision and Image Processing 4/274
![Page 4: Image Processing and Computer Vision · Visual Computing: JoachimM.Buhmann — Computer Vision and Image Processing 27/274 •fˆ(u) is also called the continuous spectrum of f(x)](https://reader034.vdocuments.us/reader034/viewer/2022051814/6034ddef7b286f70f036e6b5/html5/thumbnails/4.jpg)
Some Topics of Computer Vision: Quantization
Image Quantization in space and intensity/color.Sampling the image intensity at discrete positions is called spatial quantization whichrequires to limit the high frequency content of images to avoid aliasing.Intensity quantization is achieved by scalar quantization of grey values or vector quan-tization (clustering) of color spaces.Below is an example for spatial color quantization which combines spatial quantizationand dithering with error diffusion.
a) b) c)
Visual Computing: Joachim M. Buhmann — Computer Vision and Image Processing 5/274
![Page 5: Image Processing and Computer Vision · Visual Computing: JoachimM.Buhmann — Computer Vision and Image Processing 27/274 •fˆ(u) is also called the continuous spectrum of f(x)](https://reader034.vdocuments.us/reader034/viewer/2022051814/6034ddef7b286f70f036e6b5/html5/thumbnails/5.jpg)
Edge Detection
Goal: reduce the image content to semantically informativeparts -¿ edges.
Bildverarbeitung und Computersehen WS'01
Kantendetektion
Visual Computing: Joachim M. Buhmann — Computer Vision and Image Processing 6/274
![Page 6: Image Processing and Computer Vision · Visual Computing: JoachimM.Buhmann — Computer Vision and Image Processing 27/274 •fˆ(u) is also called the continuous spectrum of f(x)](https://reader034.vdocuments.us/reader034/viewer/2022051814/6034ddef7b286f70f036e6b5/html5/thumbnails/6.jpg)
Image Formats: Gauss-Laplace Pyramid
Wavelets or Gauss-Laplace pyramids recode the image to de-correlate pixels by exploiting the self-similar nature of naturalimages. Thereby, we achieve significantly higher compressi-on rates.
Bildverarbeitung und Computersehen WS'01
Hierarchische BildformateWavelets, Gauß-Laplace
Visual Computing: Joachim M. Buhmann — Computer Vision and Image Processing 7/274
![Page 7: Image Processing and Computer Vision · Visual Computing: JoachimM.Buhmann — Computer Vision and Image Processing 27/274 •fˆ(u) is also called the continuous spectrum of f(x)](https://reader034.vdocuments.us/reader034/viewer/2022051814/6034ddef7b286f70f036e6b5/html5/thumbnails/7.jpg)
Stereo vision: Extract depth in-formation from pairs of came-ra images.
Motion estimation: Estimatethe movement of objects inthe image by estimating theirapparent motion from opticalflow.
Shape from shading: Estimatethe shape of objects fromtheir appearence and theirshading.
Shape from motion: Use moti-on parallax to estimate depth.
Shape from texture: Exploittexture variations as changesof surface normal w.r.t. line ofsight.
Bildverarbeitung und Computersehen WS'01
3d InformationdurchShape from{shading, texture}und durchObjektwissen
Visual Computing: Joachim M. Buhmann — Computer Vision and Image Processing 8/274
![Page 8: Image Processing and Computer Vision · Visual Computing: JoachimM.Buhmann — Computer Vision and Image Processing 27/274 •fˆ(u) is also called the continuous spectrum of f(x)](https://reader034.vdocuments.us/reader034/viewer/2022051814/6034ddef7b286f70f036e6b5/html5/thumbnails/8.jpg)
Image Understanding
Perceptual Grouping
Figure-ground segmentation
Bildverarbeitung und Computersehen WS'01
Figur-Hintergrund-Trennung
Bildverarbeitung und Computersehen WS'01
Figur-Hintergrund-Trennung
Visual Computing: Joachim M. Buhmann — Computer Vision and Image Processing 9/274
![Page 9: Image Processing and Computer Vision · Visual Computing: JoachimM.Buhmann — Computer Vision and Image Processing 27/274 •fˆ(u) is also called the continuous spectrum of f(x)](https://reader034.vdocuments.us/reader034/viewer/2022051814/6034ddef7b286f70f036e6b5/html5/thumbnails/9.jpg)
The Image Formation Process
optical axislense
focal planeimageplane
incoming light
g(x1, x2)
f(x1, x2)
Visual Computing: Joachim M. Buhmann — Computer Vision and Image Processing 10/274
![Page 10: Image Processing and Computer Vision · Visual Computing: JoachimM.Buhmann — Computer Vision and Image Processing 27/274 •fˆ(u) is also called the continuous spectrum of f(x)](https://reader034.vdocuments.us/reader034/viewer/2022051814/6034ddef7b286f70f036e6b5/html5/thumbnails/10.jpg)
Mathematical Modelling of Image Processing
Def.: An image is a continuous, two-dimensional function of thelight intensity or color (x = (x1, x2))
f : R2 → R+ or R3+
x 7→ f(x) or (f1, f2, f3)T (x)
Question: How can we compensate an image deformation,e.g., defocussing?
Goal: reconstruct f(x) from g(x) in the presence of noise!
Model assumption:
1) When f(x) is shifted then g(x) is shifted as well.
2) Doubling the incoming light intensity will double the bright-ness g(x).
Visual Computing: Joachim M. Buhmann — Computer Vision and Image Processing 11/274
![Page 11: Image Processing and Computer Vision · Visual Computing: JoachimM.Buhmann — Computer Vision and Image Processing 27/274 •fˆ(u) is also called the continuous spectrum of f(x)](https://reader034.vdocuments.us/reader034/viewer/2022051814/6034ddef7b286f70f036e6b5/html5/thumbnails/11.jpg)
Linear Shift-Invariant Systems
Strategy for restauration: invert the transformation T whichmaps the original image f(x) to the defocussed image g(x).
Linearity: (assumption) restrict T to linear operators!
f1 −→ transform T −→ g1
f2 −→ transform T −→ g2
αf1 + βf2 −→ transform T −→ αg1 + βg2 ∀α, β ∈ R
• Linearity is typically only in the low intensity range fulfilled since physical systems tendto saturate.
• fi, gi are intensities ≡ power per area with fi, gi ≥ 0 in the full domain.• Often we experience non-linear imaging errors!
Visual Computing: Joachim M. Buhmann — Computer Vision and Image Processing 12/274
![Page 12: Image Processing and Computer Vision · Visual Computing: JoachimM.Buhmann — Computer Vision and Image Processing 27/274 •fˆ(u) is also called the continuous spectrum of f(x)](https://reader034.vdocuments.us/reader034/viewer/2022051814/6034ddef7b286f70f036e6b5/html5/thumbnails/12.jpg)
Shift invariance: (assumption)
f(x1, x2) −→ transform T −→ g(x1, x2)
f(x1 − a, x2 − b) −→ transform T −→ g(x1 − a, x2 − b)
• Shift invariance holds only in a limited range since images are finite objects.
Remarks: The assumption of linearity is a significant limitati-on but it gives the advantage that the linear filter theory iscompletely developed.
• An analogous one-dimensional theory applies to passiveelectrical circuits, although there time is the essential dimen-sion and causality constraints the signal.
Visual Computing: Joachim M. Buhmann — Computer Vision and Image Processing 13/274
![Page 13: Image Processing and Computer Vision · Visual Computing: JoachimM.Buhmann — Computer Vision and Image Processing 27/274 •fˆ(u) is also called the continuous spectrum of f(x)](https://reader034.vdocuments.us/reader034/viewer/2022051814/6034ddef7b286f70f036e6b5/html5/thumbnails/13.jpg)
How Can We Identify a Transformation?
Dirac’s δ-function (1D):∞∫−∞
δ(x− a)f(x)dx = f(a)
• Integration with the δ-function “samples” the function f(x) atthe position x0 = a.• The δ-function is a “generalized function”.
• Regularization of the Dirac “function”:
δ(x) = limε→0
{1ε |x| ≤ ε
2
0 elseor
δ(x) = limε→0
1√2πε
exp(− x2
2ε2)
Visual Computing: Joachim M. Buhmann — Computer Vision and Image Processing 14/274
![Page 14: Image Processing and Computer Vision · Visual Computing: JoachimM.Buhmann — Computer Vision and Image Processing 27/274 •fˆ(u) is also called the continuous spectrum of f(x)](https://reader034.vdocuments.us/reader034/viewer/2022051814/6034ddef7b286f70f036e6b5/html5/thumbnails/14.jpg)
Convolution and the Point Spread Function
Assumption: δ(x1, x2) −→ T −→ h(x1, x2)
With linearity and shift invariance it holds:
g(x1, x2) = T f(x1, x2)
= T∫ ∞
−∞
∫ ∞
−∞f(ξ, η)δ(x1 − ξ, x2 − η)dξdη
linearity=
∫ ∞
−∞
∫ ∞
−∞f(ξ, η) [T δ(x1 − ξ, x2 − η)]︸ ︷︷ ︸
h(x1−ξ,x2−η)[shift inv.]
dξdη
= (f ∗ h)(x1, x2)
Linear, shift invariant systems can be written as convolutions!
Visual Computing: Joachim M. Buhmann — Computer Vision and Image Processing 15/274
![Page 15: Image Processing and Computer Vision · Visual Computing: JoachimM.Buhmann — Computer Vision and Image Processing 27/274 •fˆ(u) is also called the continuous spectrum of f(x)](https://reader034.vdocuments.us/reader034/viewer/2022051814/6034ddef7b286f70f036e6b5/html5/thumbnails/15.jpg)
Identification of the Kernel
Let f(x1, x2) = δ(x1, x2), i.e., the image is a white dot with“infinite” intensity. Then the measured image g(x1, x2) is gi-ven by
g(x1, x2) = (δ ∗ h)(x1, x2)
=∫ ∞
−∞
∫ ∞
−∞δ(ξ, η)h(x1 − ξ, x2 − η)dξdη
= h(x1, x2)
⇒ T δ(x1, x2) = h(x1, x2)
⇒ testing the linear shift-invariant system with a δ-peak will re-veal the convolution kernel h(x1, x2) of the system.
Visual Computing: Joachim M. Buhmann — Computer Vision and Image Processing 16/274
![Page 16: Image Processing and Computer Vision · Visual Computing: JoachimM.Buhmann — Computer Vision and Image Processing 27/274 •fˆ(u) is also called the continuous spectrum of f(x)](https://reader034.vdocuments.us/reader034/viewer/2022051814/6034ddef7b286f70f036e6b5/html5/thumbnails/16.jpg)
Schematic View of a Convolution
h(x)
L
out−of−focus plane focal planeg(x) f(x)
x1
1
x
• g(x1) depends on f(x) for all x ∈ [x1 − L2 , x1 + L
2 ].• convolution kernel hx1(x) describes the influence of f(x) onto g(x1).• shift invariance of hx1(x) results in cumulative influence:
g(x1) =∫ L/2
−L/2
f(x)h(x1 − x)dx =∫ L/2
−L/2
f(x1 − x)h(x)dx
≈ f(0)h(x1)∆ + f(∆)h(x1 −∆)∆ + f(2∆)h(x1 − 2∆)∆ + . . .
+f(−∆)h(x1 + ∆)∆ + f(−2∆)h(x1 + 2∆)∆ + . . .
Visual Computing: Joachim M. Buhmann — Computer Vision and Image Processing 17/274
![Page 17: Image Processing and Computer Vision · Visual Computing: JoachimM.Buhmann — Computer Vision and Image Processing 27/274 •fˆ(u) is also called the continuous spectrum of f(x)](https://reader034.vdocuments.us/reader034/viewer/2022051814/6034ddef7b286f70f036e6b5/html5/thumbnails/17.jpg)
Convolution: 1D-Example
y(t) = (x ∗ h)(t) =∫
x(τ)h(t− τ)dτ
Visual Computing: Joachim M. Buhmann — Computer Vision and Image Processing 18/274
![Page 18: Image Processing and Computer Vision · Visual Computing: JoachimM.Buhmann — Computer Vision and Image Processing 27/274 •fˆ(u) is also called the continuous spectrum of f(x)](https://reader034.vdocuments.us/reader034/viewer/2022051814/6034ddef7b286f70f036e6b5/html5/thumbnails/18.jpg)
Convolution: 1D-Example (cont’d)
y(t) = (x ∗ h)(t) =∫
x(τ)h(t− τ)dτ
Visual Computing: Joachim M. Buhmann — Computer Vision and Image Processing 19/274
![Page 19: Image Processing and Computer Vision · Visual Computing: JoachimM.Buhmann — Computer Vision and Image Processing 27/274 •fˆ(u) is also called the continuous spectrum of f(x)](https://reader034.vdocuments.us/reader034/viewer/2022051814/6034ddef7b286f70f036e6b5/html5/thumbnails/19.jpg)
Facts about Convolution
• Linear shift-invariant (LSI) systems can be written as convo-lutions.• The convolution kernel h characterizes the LSI system uni-
quely.• Cascades of LSI systems: the convolution is commutative
and associative:g ∗ h = h ∗ g
(f ∗ g) ∗ h = f ∗ (g ∗ h)
f1 −→ T1 : h1 −→ T2 : h2︸ ︷︷ ︸h1 ∗ h2
−→ g1
⇒ one of the most important operations in signal processing
Visual Computing: Joachim M. Buhmann — Computer Vision and Image Processing 20/274
![Page 20: Image Processing and Computer Vision · Visual Computing: JoachimM.Buhmann — Computer Vision and Image Processing 27/274 •fˆ(u) is also called the continuous spectrum of f(x)](https://reader034.vdocuments.us/reader034/viewer/2022051814/6034ddef7b286f70f036e6b5/html5/thumbnails/20.jpg)
Convolution Kernel for Image Defocussing
Defocussing an image amounts to convolving it with a ‘pillbox’:
h(x1, x2) =
{1
πR2 x21 + x2
2 ≤ R2
0 otherwise
Note: this convolution kernel is normalized:∫ ∫
h(x1, x2)dx1dx2 = 1
Visual Computing: Joachim M. Buhmann — Computer Vision and Image Processing 21/274
![Page 21: Image Processing and Computer Vision · Visual Computing: JoachimM.Buhmann — Computer Vision and Image Processing 27/274 •fˆ(u) is also called the continuous spectrum of f(x)](https://reader034.vdocuments.us/reader034/viewer/2022051814/6034ddef7b286f70f036e6b5/html5/thumbnails/21.jpg)
Convolution Kernel for Image Defocussing
original image convolved with pillbox kernel
Visual Computing: Joachim M. Buhmann — Computer Vision and Image Processing 22/274
![Page 22: Image Processing and Computer Vision · Visual Computing: JoachimM.Buhmann — Computer Vision and Image Processing 27/274 •fˆ(u) is also called the continuous spectrum of f(x)](https://reader034.vdocuments.us/reader034/viewer/2022051814/6034ddef7b286f70f036e6b5/html5/thumbnails/22.jpg)
A Motion Kernel
Each light dot is transformed into a short line along the x1-axis:
h(x1, x2) =12l
[θ(x1 + l)− θ(x1 − l)] δ(x2)
Visual Computing: Joachim M. Buhmann — Computer Vision and Image Processing 23/274
![Page 23: Image Processing and Computer Vision · Visual Computing: JoachimM.Buhmann — Computer Vision and Image Processing 27/274 •fˆ(u) is also called the continuous spectrum of f(x)](https://reader034.vdocuments.us/reader034/viewer/2022051814/6034ddef7b286f70f036e6b5/html5/thumbnails/23.jpg)
Denoising Time Series
22. Oktober 2001Bildverarbeitung und
Computersehen WS'01
Faltung eines verrauschten Signals
50 55 60 65 70 75 80 85 90 95 100-10
0
10
20
30
40
50
60Original Data
50 55 60 65 70 75 80 85 90 95-10
0
10
20
30
40
50
60
-2 -1.5 -1 -0.5 0 0.5 1 1.5 20
0.02
0.04
0.06
0.08
0.1
0.12
Convolution Kernel
Visual Computing: Joachim M. Buhmann — Computer Vision and Image Processing 24/274
![Page 24: Image Processing and Computer Vision · Visual Computing: JoachimM.Buhmann — Computer Vision and Image Processing 27/274 •fˆ(u) is also called the continuous spectrum of f(x)](https://reader034.vdocuments.us/reader034/viewer/2022051814/6034ddef7b286f70f036e6b5/html5/thumbnails/24.jpg)
Lena with Gaussian Blurring and Noise
Lena: original / noisy(Gaussian noise with σ=30)
PSNR=18.4482
Gaussian blurring kernel:
h(x1, x2) =1
2πσ2exp(−x2
1 + x22
2σ2)
Visual Computing: Joachim M. Buhmann — Computer Vision and Image Processing 25/274
![Page 25: Image Processing and Computer Vision · Visual Computing: JoachimM.Buhmann — Computer Vision and Image Processing 27/274 •fˆ(u) is also called the continuous spectrum of f(x)](https://reader034.vdocuments.us/reader034/viewer/2022051814/6034ddef7b286f70f036e6b5/html5/thumbnails/25.jpg)
Lena Convolved with a Laplacian Filter
12. November 2001Bildverarbeitung und
Computersehen WS'01
Faltung von Lena mit Laplace-Kern
Laplacian filter:
h(x1, x2) = ∇2 exp(−x21 + x2
2
2σ2) = − 1
σ2
(1− r2
σ2
)exp(− r2
2σ2) r2 := x2
1 + x22
Note: here the normalization is∫ ∫
h(x1, x2)dx1dx2 = 0.
Visual Computing: Joachim M. Buhmann — Computer Vision and Image Processing 26/274
![Page 26: Image Processing and Computer Vision · Visual Computing: JoachimM.Buhmann — Computer Vision and Image Processing 27/274 •fˆ(u) is also called the continuous spectrum of f(x)](https://reader034.vdocuments.us/reader034/viewer/2022051814/6034ddef7b286f70f036e6b5/html5/thumbnails/26.jpg)
The Fourier Transformation: Basic Facts
Def.: Let f be an absolutely integrable function over R. TheFourier transformation of f is defined as
f(u) ≡ F [f(x)] =∫ +∞
−∞f(x) exp(−ı2πux)dx.
The inverse Fourier transformation is given by the formula
f(x) ≡ F−1[f(u)] =∫ +∞
−∞f(u) exp(ı2πux)du.
Note: while f(x) is always real, f(u) is typically complex.
Visual Computing: Joachim M. Buhmann — Computer Vision and Image Processing 27/274
![Page 27: Image Processing and Computer Vision · Visual Computing: JoachimM.Buhmann — Computer Vision and Image Processing 27/274 •fˆ(u) is also called the continuous spectrum of f(x)](https://reader034.vdocuments.us/reader034/viewer/2022051814/6034ddef7b286f70f036e6b5/html5/thumbnails/27.jpg)
• f(u) is also called the continuous spectrum of f(x).• If x is a space coordinate, then u is called the spatial fre-
quency .
Inversion formula: f(x) is represented as a continuous super-position of waves with amplitude f(u).
Example of an odd function approximated by sinus waves(Remember: exp(ıx) = cos(x) + ı sin(x)):
f(x) ≈ f(u0) sin(2πu0x) + f(u1) sin(2πu1x) + f(u2) sin(2πu2x) + . . .
Visual Computing: Joachim M. Buhmann — Computer Vision and Image Processing 28/274
![Page 28: Image Processing and Computer Vision · Visual Computing: JoachimM.Buhmann — Computer Vision and Image Processing 27/274 •fˆ(u) is also called the continuous spectrum of f(x)](https://reader034.vdocuments.us/reader034/viewer/2022051814/6034ddef7b286f70f036e6b5/html5/thumbnails/28.jpg)
Fourier Transformation: Example 1 (box)
Given the box function
f(x) =12l
(θ(x + l)− θ(x− l)) =
{12l if|x| ≤ l
0 otherwise
the Fourier transform is
f(u) ≡ F [f(x)] =∫ +∞
−∞f(x) exp(−ı2πux)dx
=∫ l
−l
12l· (cos(2πux)− ı sin(2πux)︸ ︷︷ ︸
∫→0
)dx
=sin(2πul)
2πul≡ sinc(2πul)
Visual Computing: Joachim M. Buhmann — Computer Vision and Image Processing 29/274
![Page 29: Image Processing and Computer Vision · Visual Computing: JoachimM.Buhmann — Computer Vision and Image Processing 27/274 •fˆ(u) is also called the continuous spectrum of f(x)](https://reader034.vdocuments.us/reader034/viewer/2022051814/6034ddef7b286f70f036e6b5/html5/thumbnails/29.jpg)
Fourier Transformation: Example 1 (box)
Graphs of box and sinc-function for l = 12:
Visual Computing: Joachim M. Buhmann — Computer Vision and Image Processing 30/274
![Page 30: Image Processing and Computer Vision · Visual Computing: JoachimM.Buhmann — Computer Vision and Image Processing 27/274 •fˆ(u) is also called the continuous spectrum of f(x)](https://reader034.vdocuments.us/reader034/viewer/2022051814/6034ddef7b286f70f036e6b5/html5/thumbnails/30.jpg)
Fourier Transformation: Example 2 (Gauss)Given the function
f(x) =1√
2πσx
exp(− x2
2σ2x
)
the Fourier transform is
f(u) ≡ F [f(x)] =∫ +∞
−∞f(x) exp(−ı2πux)dx
=1√
2πσx
∫ ∞
−∞exp(− x2
2σ2x
) · (cos(2πux)− ı sin(2πux)︸ ︷︷ ︸∫→0
dx
=† exp(− u2
2σ2u
) where σu =1
2πσx
† [Abramowitz, Stegun: Handbook of Mathematical Functions, 1972]
⇒ the Fourier transform of a Gaussian is a (unnormalized) Gaussian!
The larger the variance σ2x, the smaller the variance σ2
u: σx · σu = 12π
Visual Computing: Joachim M. Buhmann — Computer Vision and Image Processing 31/274
![Page 31: Image Processing and Computer Vision · Visual Computing: JoachimM.Buhmann — Computer Vision and Image Processing 27/274 •fˆ(u) is also called the continuous spectrum of f(x)](https://reader034.vdocuments.us/reader034/viewer/2022051814/6034ddef7b286f70f036e6b5/html5/thumbnails/31.jpg)
Fourier Transformation: Example 3 (Dirac’s δ)
The Fourier transform of Dirac’s δ-function is
δ(u) ≡ F [δ(x)] =∫ +∞
−∞δ(x) exp(−ı2πux)dx
= exp(−ı2πu · 0)
= 1
⇒ the Fourier transform of the δ-function equals 1 for all fre-quencies u.
Visual Computing: Joachim M. Buhmann — Computer Vision and Image Processing 32/274
![Page 32: Image Processing and Computer Vision · Visual Computing: JoachimM.Buhmann — Computer Vision and Image Processing 27/274 •fˆ(u) is also called the continuous spectrum of f(x)](https://reader034.vdocuments.us/reader034/viewer/2022051814/6034ddef7b286f70f036e6b5/html5/thumbnails/32.jpg)
Properties of the Fourier Transformation
Linearity: If F [f(x)] = f(u) and F [g(x)] = g(u) then it holdsfor all complex numbers a, b ∈ C
F [af(x) + bg(x)] = af(u) + bg(u)
Shift: If F [f(x)] = f(u) then it holds for c ∈ R
F [f(x− c)] = f(u) exp(−ı2πcu)
Modulation: If F [f(x)] = f(u) then it holds for c ∈ R
F [f(x) exp(ı2πcx)] = f(u− c)
Visual Computing: Joachim M. Buhmann — Computer Vision and Image Processing 33/274
![Page 33: Image Processing and Computer Vision · Visual Computing: JoachimM.Buhmann — Computer Vision and Image Processing 27/274 •fˆ(u) is also called the continuous spectrum of f(x)](https://reader034.vdocuments.us/reader034/viewer/2022051814/6034ddef7b286f70f036e6b5/html5/thumbnails/33.jpg)
Scaling: If F [f(x)] = f(u) and c > 0
F [f(cx)] =1cf(
u
c)
Differentiation: Let f be piecewise continuous and absolutelyintegrable. If the function xf(x) is absolutely integrable thenthe Fourier transform f is continuous and differentiable. Itholds
F [xf(x)] =ı
2π
d
duf(u)
F [d
dxf(x)] = ı2πuf(u)
Visual Computing: Joachim M. Buhmann — Computer Vision and Image Processing 34/274
![Page 34: Image Processing and Computer Vision · Visual Computing: JoachimM.Buhmann — Computer Vision and Image Processing 27/274 •fˆ(u) is also called the continuous spectrum of f(x)](https://reader034.vdocuments.us/reader034/viewer/2022051814/6034ddef7b286f70f036e6b5/html5/thumbnails/34.jpg)
Parseval’s Equality: Let f be piecewise continuous and abso-lutely integrable. Then the Fourier transform f(u) = F [f(x)]satisfies: ∫ ∞
−∞|f(x)|2dx =
∫ ∞
−∞|f(u)|2du
Power Spectrum: Considering the auto-correlation functionΦff(x) of a complex function f for x ∈ R,
Φff(x) =∫ ∞
−∞f(ξ − x)f(ξ)dξ .
The Fourier transform is given by
Φff(u) ≡ F [Φff(x)] = |f(u)|2.
(f(x) is the conjugate complex function of f(x))
Visual Computing: Joachim M. Buhmann — Computer Vision and Image Processing 35/274
![Page 35: Image Processing and Computer Vision · Visual Computing: JoachimM.Buhmann — Computer Vision and Image Processing 27/274 •fˆ(u) is also called the continuous spectrum of f(x)](https://reader034.vdocuments.us/reader034/viewer/2022051814/6034ddef7b286f70f036e6b5/html5/thumbnails/35.jpg)
Fourier Transform of Convolution
Given: convolution g(x) = (f ∗ h)(x) =∫
f(ξ)h(x− ξ)dξ
Calculate Fourier transform of g:
g(u) ≡ F [g(x)] =∫ +∞
−∞
[∫ +∞
−∞f(ξ)h(x− ξ)dξ
]exp(−ı2πux)dx
=∫ +∞
−∞f(ξ)
[∫ +∞
−∞h(x− ξ) exp(−ı2πux)dx
]dξ
=∫ +∞
−∞h(u)f(ξ) exp(−ı2πuξ)dξ
= h(u)f(u)
⇒ Convolution in spatial domain becomes multiplication inFourier space.
Visual Computing: Joachim M. Buhmann — Computer Vision and Image Processing 36/274
![Page 36: Image Processing and Computer Vision · Visual Computing: JoachimM.Buhmann — Computer Vision and Image Processing 27/274 •fˆ(u) is also called the continuous spectrum of f(x)](https://reader034.vdocuments.us/reader034/viewer/2022051814/6034ddef7b286f70f036e6b5/html5/thumbnails/36.jpg)
Modulation Transfer Function
System Behavior in Fourier Space: How is a harmonic oscil-lation transformed by convolution kernel h?⇒ amplitude modulation A(u):
exp(ı2πux) −→ kernel h(x) −→ A(u) exp(ı2πux)
Eigenfunction of the convolution with eigenvalue A(u) is theoscillation f(x) = exp(ı2πux).
Output g(x) = (f ∗ h)(x) =∫
exp(ı2πuξ)h(x− ξ)dξ
= exp(ı2πux)∫
exp(−ı2πuξ)h(ξ)dξ = h(u) exp(ı2πux)
Note: the eigenvalue A(u) equals h(u) = F [h](u).
Visual Computing: Joachim M. Buhmann — Computer Vision and Image Processing 37/274
![Page 37: Image Processing and Computer Vision · Visual Computing: JoachimM.Buhmann — Computer Vision and Image Processing 27/274 •fˆ(u) is also called the continuous spectrum of f(x)](https://reader034.vdocuments.us/reader034/viewer/2022051814/6034ddef7b286f70f036e6b5/html5/thumbnails/37.jpg)
Image Filtering in the Frequency Domain
2D Fourier transformation of an image f(x), x := (x1, x2):
f(u, v) ≡ F [f(x)] =
+∞∫−∞
+∞∫−∞
f(x1, x2) exp(−ı2π(ux1 + vx2))dx
High-pass filtering: remove low frequencies, for examplechoose maximum value B:
fhp(u, v) =
{f(u, v) if u2 + v2 > B2
0 otherwise
Inverse Fourier transformation yields high-pass-filtered imagefhp(x) = F−1[fhp(u, v)]
Visual Computing: Joachim M. Buhmann — Computer Vision and Image Processing 38/274
![Page 38: Image Processing and Computer Vision · Visual Computing: JoachimM.Buhmann — Computer Vision and Image Processing 27/274 •fˆ(u) is also called the continuous spectrum of f(x)](https://reader034.vdocuments.us/reader034/viewer/2022051814/6034ddef7b286f70f036e6b5/html5/thumbnails/38.jpg)
Example of Image Filtering
original image high-pass-filtered
⇒ edge detection
Visual Computing: Joachim M. Buhmann — Computer Vision and Image Processing 39/274
![Page 39: Image Processing and Computer Vision · Visual Computing: JoachimM.Buhmann — Computer Vision and Image Processing 27/274 •fˆ(u) is also called the continuous spectrum of f(x)](https://reader034.vdocuments.us/reader034/viewer/2022051814/6034ddef7b286f70f036e6b5/html5/thumbnails/39.jpg)
Low-pass filtering: analogous to high-pass filter, but removehigh frequencies
Example:
original image low-pass-filtered
⇒ removing noise
Visual Computing: Joachim M. Buhmann — Computer Vision and Image Processing 40/274
![Page 40: Image Processing and Computer Vision · Visual Computing: JoachimM.Buhmann — Computer Vision and Image Processing 27/274 •fˆ(u) is also called the continuous spectrum of f(x)](https://reader034.vdocuments.us/reader034/viewer/2022051814/6034ddef7b286f70f036e6b5/html5/thumbnails/40.jpg)
The Image Restoration Problem
f(x) −→ h(x) −→ g(x) −→ h(x) −→ f(x)
The ‘inverse’ kernel h(x) should compensate the effect of theimage degradation h(x), i.e.,
(h ∗ h)(x) = δ(x)
h may be determined more easily in Fourier space:
F [h](u, v) · F [h](u, v) = 1
To determine F [h] we need to estimate
1. the distortion model h(x) (point spread function) or F [h](u, v) (modu-lation transfer function)
2. the parameters of h(x), e.g. r for defocussing.
Visual Computing: Joachim M. Buhmann — Computer Vision and Image Processing 41/274
![Page 41: Image Processing and Computer Vision · Visual Computing: JoachimM.Buhmann — Computer Vision and Image Processing 27/274 •fˆ(u) is also called the continuous spectrum of f(x)](https://reader034.vdocuments.us/reader034/viewer/2022051814/6034ddef7b286f70f036e6b5/html5/thumbnails/41.jpg)
Image Restoration: Motion Blur
Kernel for motion blur h(x) = 12l
(θ(x1 + l)− θ(x1 − l)
)δ(x2)
(a light dot is transformed into a small line in x1 direction).
Fourier transformation:
F [h](u, v) =12l
+l∫−l
exp(−ı2πux1)
+∞∫−∞
δ(x2) exp(−ı2πvx2)dx2︸ ︷︷ ︸=1
dx1
=sin(2πul)
2πul=: sinc(2πul)
Visual Computing: Joachim M. Buhmann — Computer Vision and Image Processing 42/274
![Page 42: Image Processing and Computer Vision · Visual Computing: JoachimM.Buhmann — Computer Vision and Image Processing 27/274 •fˆ(u) is also called the continuous spectrum of f(x)](https://reader034.vdocuments.us/reader034/viewer/2022051814/6034ddef7b286f70f036e6b5/html5/thumbnails/42.jpg)
sinc(u) = sin(u)u
1/sinc(u)
h(u) = F [h](u) = sinc(2πul) F [h](u) = 1/h(u)
Problems:• Convolution with the kernel h completely cancels the frequen-
cies ν2l for ν ∈ Z. Vanishing frequencies cannot be recovered!
• Noise amplification for F [h](u, v)� 1.
Visual Computing: Joachim M. Buhmann — Computer Vision and Image Processing 43/274
![Page 43: Image Processing and Computer Vision · Visual Computing: JoachimM.Buhmann — Computer Vision and Image Processing 27/274 •fˆ(u) is also called the continuous spectrum of f(x)](https://reader034.vdocuments.us/reader034/viewer/2022051814/6034ddef7b286f70f036e6b5/html5/thumbnails/43.jpg)
Avoiding Noise Amplification
Regularizedreconstruction filter:
F [h](u, v) =F [h]
|F [h]|2 + ε
Singularities are avoidedby the regularization ε.
The size of ε implicitly determines an estimate of the noise level in the image, since wediscard signals which are dampened below the size ε.
Visual Computing: Joachim M. Buhmann — Computer Vision and Image Processing 44/274