image processing - fakultät informatik — tu · pdf fileimage processing class 2 ......

© 2007 Thomas Brox 1

Image Processing

Image Processing

Class 2Image acquisition and representation


Image Processing The human eye

From Birnbaumer-Schmidt 1999


Image Processing The first stages of human vision

• Simple cells: Mexican hat operator– Radial symmetric– Exhibitory or inhibitory in the center (two types)– Vice-versa in the outer region– D. Marr: Correspond to second derivative

filters (Laplacian, Laplacian of Gaussian)

• Simple cells: orientation selective cells– Selective response to stripes of certain

orientation and scale– Correspond to Gabor filters– Responsible for texture and motion – Found by Hubel-Wiesel 1959 (cat, primate)

• Complex cells: integration of responses in a nonlinear fashion


Image Processing Pinhole camera

• Objects points are projected to image points by

• Most simple camera model

• Practical problem: sharpness vs. light intensity

• Ratio steered by aperture (size of the hole)


Image Processing Optical camera

• Light focusing (solves problem of pinhole camera)

• Large aperture and sharpness at a certain depth possible

• Thin lenses:

• Wide lenses: focal length depends on orientation and color

• Application in computer vision: shape from defocus(Favaro-Soatto 2005)

• Usually approximation with pinhole camera


Image Processing Fisheye effect


Image Processing Gray value images

• Continuous 3D world projects lightintensities into a 2D plane

• Image is a continuous function

• is called image domain, it is usually rectangular

Continuous image Discrete, sampled image


Image Processing Sampling

• In digital images, intensities are only given on a pixel grid (e.g. grid of a CCD chip)

• Discretization of the image domain

• Grid points are called pixels (picture elements)

• Grid is usually a rectangular point grid with equal spacing

• Grid size defines the spacing of pixels

• Often same spacing in all directions:

• True spacing not known Grid size of input image set to


Image Processing Continuous vs. discrete

• In this course we will usually treat the image as a continuous functionand discretize this model only for implementation

• Advantages of continuous models– The scene is continuous– The continuous model is the limiting case for successively finer grids– Consistent discretization of the continuous model ensures certain properties

(e.g. rotational invariance, exact length measurement)– Mathematics provides lots of tools to deal with continuous models

• Reasons for discrete models– The image data is discrete– Model is closer to its implementation– There exist some efficient discrete optimization techniques


Image Processing Image resolution, downsampling, upsampling

• Given a certain image of a scene, the number of grid points to represent the discrete image is called the image resolution

• Reducing the number of grid points is called downsampling

• Increasing the number of grid points is called upsampling

Fixed image size

Fixed grid size


Image Processing Aliasing and Moiré effect

• A function can be represented by its frequency components (Fourier transformation)

• A discrete signal can only represent frequencies up to a certain limit Nyquist frequency

• Ignorance of the Nyquist frequency leads to aliasing artifacts (e.g. straight lines become stepped)

• Sampling of periodic signals is a frequency modulation (multiplication of two periodic signals)

• It can lead to Moiré effects

• Videos: temporal aliasing Moiré effect, Source: Wikipedia


Image Processing Moiré effect

Top left: Rastered source image.

Top right: Downsampled rasteredimage (1% reduction in

size).

Bottom left: Source downsampled,

then rastered.

Bottom right: Downsampled rasteredimage (20% reduction

in size).

Source: Wikipedia


Image Processing

• Two different frequencies may lead to the same discrete signal

• An input signal can be reconstructed from the sampled signal in a unique way if the sampling rate is two times the bandwidth of the input signal

• Reduction of bandwidth can be achieved by smoothing the signal

• Consequence: Smooth your input image sufficiently before downsampling

Nyquist-Shannon theorem


Image Processing Downsampling operators

• Decanting operator: ensures minimum smoothing necessary to avoidaliasing

• Pixels from high resolution image spill their intensity to the pixels of the low resolution image

• In case of overlap, intensity distribution to both cells according to the overlap ratio

• Normalization of intensity value in the low resolution image by the downsampling factor

• Operator is separable: can be applied sequentially along all axes

High resolution source

Low resolution target


Image Processing Downsampling operators

• Bilinear interpolation: weighted average of neighboring pixels

• Project coarse grid point tofiner grid

• Compute weighted averagealong x-axis

• Compute weighted average along y-axis

• Sufficient smoothing only for downsampling factors

• Otherwise, additional smoothing necessary


Image Processing Upsampling

• Decanting operator can also be used for upsampling

• Bilinear upsampling works the same way as bilinear downsampling

• Bilinear upsampling recommended

High resolution target

Low resolution source


Image Processing Quantization

• Discretization of the co-domain

• Saves disk space

• Usually 256 gray scales 8 bit per pixel (bpp)

• Humans can distinguish only 40 gray scales (but several thousand colors)

• Optimal quantization by clustering (e.g. k-means)

• We will usually assume


Image Processing Quantization

256 scales 16 scales

4 scales 2 scales (binary image)


Image Processing Types of images

• Generalization of images with respect of– Dimensionality of the image domain– Dimensionality of the co-domain

• “Standard” images: image domain is two-dimensional, co-domain one-dimensional (gray scale)

• Other dimensionalities of the image domain:– 1D signals– 3D images (volumetric images, image sequences)– 4D images (sequence of volumetric images)

• Other co-domains– Vectors (e.g. color images)– Matrices


Image Processing Volumetric 3D data

Rendering of a 3D ultrasound image of a human fetus in its 10th week. From Weickert-Zuiderveld-ter Haar Romeny-Niessen 1997


Image Processing Image sequences

Flowergarden sequence


Image Processing Color images

Color image Red channel

Green channel Blue channel


Image Processing Matrix-valued images

• Diffusion tensor MRI: flow preferences of water molecules

• Each voxel (volume element) comprises a 3x3 matrix

Matrix channels of DT-MRIWeinstein-Kindlmann-Lundberg 1999

3D visualization via ellipsoidsData: Anna Villanova, BMT, Eindhoven

Visualization: MIA group, Saarland University


Image Processing Summary

• Images are projections of the real, continuous world and therefore continuous functions

• Digital images are discrete approximations of these functions

• Downsampling of image requires smoothing/averaging in order to avoid aliasing artifacts

• There are various generalizations of the typical grayscale image

• Particularly color images and image sequences play an important role in image processing


Image Processing Programming assignment - downsampling

• Unzip ImageProcessingEx01.zip and look at the C program Ex01.cpp

• Compile this program via g++ Ex01.cpp –I. –lm –o Ex01

• Run the program

• Look at the result image via xv leopardSmall.pgmAre you satisfied with the result? If not, why?

• Implement downsampling by the decanting operator

• Implement downsampling by bilinear interpolation

• Compare the three results

• Extra assignment: implement also the corresponding upsampling operators

image processing - fakultät informatik — tu · pdf fileimage processing class 2 ......

Documents