introduction to image processing with scipy and numpy · pdf fileintroduction some theory...

IntroductionSome Theory

Doing the Stuff in PythonDemo(s)Q and A

Introduction to Image Processing with SciPyand NumPy

Anil C Rcranil@ee.iisc.ernet.in

Department of Electrical EngineeringIndian Institute of Science

September 19, 2010

Anil C R Image Processing

IntroductionSome Theory

Doing the Stuff in PythonDemo(s)Q and A

Outline

1 IntroductionImage ProcessingWhat are SciPy and NumPy?

2 Some TheoryFiltersThe Fourier Transform

3 Doing the Stuff in Python

4 Demo(s)

Anil C R Image Processing

IntroductionSome Theory

Doing the Stuff in PythonDemo(s)Q and A

Image ProcessingSciPy and NumPy

Outline

1 IntroductionImage ProcessingWhat are SciPy and NumPy?

2 Some TheoryFiltersThe Fourier Transform

3 Doing the Stuff in Python

4 Demo(s)

Anil C R Image Processing

IntroductionSome Theory

Doing the Stuff in PythonDemo(s)Q and A

Image ProcessingSciPy and NumPy

What are Images?

Continious domain, Continious range

f : R2 → R(R3)

Discrete domain, Continious range

f : Z2 → R(R3)

Discrete domain, Discrete range

f : Z2 → Z(R3)

Finite domain, Continious range

f : Zm × Zn → R(R3)

Anil C R Image Processing

IntroductionSome Theory

Doing the Stuff in PythonDemo(s)Q and A

Image ProcessingSciPy and NumPy

What are Images?

Continious domain, Continious range

f : R2 → R(R3)

Discrete domain, Continious range

f : Z2 → R(R3)

Discrete domain, Discrete range

f : Z2 → Z(R3)

Finite domain, Continious range

f : Zm × Zn → R(R3)

Anil C R Image Processing

IntroductionSome Theory

Doing the Stuff in PythonDemo(s)Q and A

Image ProcessingSciPy and NumPy

Using Matrices to Represent Images

f as an element of Rm×n(Rm×n×k )

⇒ Linear Algebra⇒ LAPACK, BLAS, etc⇒ FORTRAN, C, etc⇒ Super Hard⇒ MATLAB⇒ Super Expensive⇒ SciPy + NumPy, GNU Octave, Scilab, etcPyCon 2010⇒ SciPy + NumPy

Anil C R Image Processing

IntroductionSome Theory

Doing the Stuff in PythonDemo(s)Q and A

Image ProcessingSciPy and NumPy

Outline

1 IntroductionImage ProcessingWhat are SciPy and NumPy?

2 Some TheoryFiltersThe Fourier Transform

3 Doing the Stuff in Python

4 Demo(s)

Anil C R Image Processing

IntroductionSome Theory

Doing the Stuff in PythonDemo(s)Q and A

Image ProcessingSciPy and NumPy

NumPy

Numerical ProcessingStarted off as numecric written in 1995 by Jim Huguni et al.Numeric was slow for large arrays and was rewritten forlarge arrays as NumarrayTravis Oliphant, in 2005 merged them both into NumPy

Anil C R Image Processing

IntroductionSome Theory

Doing the Stuff in PythonDemo(s)Q and A

Image ProcessingSciPy and NumPy

SciPy

Libraries for scientific computingLinear AlgebraStatisticsSignal and Image processingOptimizationODE Solvers

Anil C R Image Processing

IntroductionSome Theory

Doing the Stuff in PythonDemo(s)Q and A

FiltersThe Fourier Transform

Outline

1 IntroductionImage ProcessingWhat are SciPy and NumPy?

2 Some TheoryFiltersThe Fourier Transform

3 Doing the Stuff in Python

4 Demo(s)

Anil C R Image Processing

IntroductionSome Theory

Doing the Stuff in PythonDemo(s)Q and A

FiltersThe Fourier Transform

Filters

Keep what you want and throw away the restStudying filters is the most important part in ImageProcessingClassified into linear and non-linear filters

Anil C R Image Processing

IntroductionSome Theory

Doing the Stuff in PythonDemo(s)Q and A

FiltersThe Fourier Transform

Linear Filters

Given images f1, f2, . . . , fn a filter H is called linear ifH(α1f1 + α2f2 + · · ·+ αnfn) =

α1H(f1) + α2H(f2) + · · ·+ αnH(fn)

Linearity can be useful in fast computation

Anil C R Image Processing

IntroductionSome Theory

Doing the Stuff in PythonDemo(s)Q and A

FiltersThe Fourier Transform

Outline

1 IntroductionImage ProcessingWhat are SciPy and NumPy?

2 Some TheoryFiltersThe Fourier Transform

3 Doing the Stuff in Python

4 Demo(s)

Anil C R Image Processing

IntroductionSome Theory

Doing the Stuff in PythonDemo(s)Q and A

FiltersThe Fourier Transform

Time and Frequency Domains

⇓

Anil C R Image Processing

IntroductionSome Theory

Doing the Stuff in PythonDemo(s)Q and A

FiltersThe Fourier Transform

Fourier Transform

Continious FTF (ωxx , ωyy) =

∫∫R2 f (x , y)exp(−iωxx − iωyy)dxdy

Discrete FT

F (ωxx , ωyy) =∑M

0∑N

0 f (x , y)exp(−i{2πωxM x − 2πωy

N y})

Notation: F is the FT of f , also F = F{f}

Anil C R Image Processing

IntroductionSome Theory

Doing the Stuff in PythonDemo(s)Q and A

FiltersThe Fourier Transform

Convolution

Continious(f ~ g)(x , y) =

∫∫R2 f (x ′, y ′)g(x − x ′, y − y ′)dxdy

Discrete(f ~ g)(x , y) =

∑∑Z2 f (x , y)g(x − x ′, y − y ′)

Theorem(a) F{f ~ g} = FG(b) F{fg} = F ~ G

Any linear filter can be written as a convolution.

Anil C R Image Processing

IntroductionSome Theory

Doing the Stuff in PythonDemo(s)Q and A

FiltersThe Fourier Transform

Fast Fourier Transform (FFT)

Computing the Discrete Fourier Transform takes O(n2m2)for an m × n imageFFT Computes the same in O(n log nm log m)

Anil C R Image Processing

IntroductionSome Theory

Doing the Stuff in PythonDemo(s)Q and A

Interactive Python

Install NumPyInstall SciPyInstall MatplotlibInstall IPython

Running IPython$ ipython -pylab

Anil C R Image Processing

IntroductionSome Theory

Doing the Stuff in PythonDemo(s)Q and A

Fast Fourier Transform (FFT)

FFT in NumPyIn[1]: from scipy import lenaIn[2]: f = lena()In[3]: from numpy.fft import fft2 #unnecessary if you invoke ipython with -pylabIn[4]: F = fft2(f)In[5]: imshow(real(F))

Anil C R Image Processing

IntroductionSome Theory

Doing the Stuff in PythonDemo(s)Q and A

Fast Fourier Transform (FFT)

FFT in NumPyIn[1]: from scipy import lenaIn[2]: f = lena()In[3]: from numpy.fft import fft2 #unnecessary if you invoke ipython with -pylabIn[4]: F = fft2(f)In[5]: imshow(real(F))

Anil C R Image Processing

IntroductionSome Theory

Doing the Stuff in PythonDemo(s)Q and A

Fast Fourier Transform (FFT)

FFT in NumPyIn[1]: from scipy import lenaIn[2]: f = lena()In[3]: from numpy.fft import fft2 #unnecessary if you invoke ipython with -pylabIn[4]: F = fft2(f)In[5]: imshow(real(F))

Anil C R Image Processing

IntroductionSome Theory

Doing the Stuff in PythonDemo(s)Q and A

Fast Fourier Transform (FFT)

FFT in NumPyIn[1]: from scipy import lenaIn[2]: f = lena()In[3]: from numpy.fft import fft2 #unnecessary if you invoke ipython with -pylabIn[4]: F = fft2(f)In[5]: imshow(real(F))

Anil C R Image Processing

IntroductionSome Theory

Doing the Stuff in PythonDemo(s)Q and A

Fast Fourier Transform (FFT)

FFT in NumPyIn[1]: from scipy import lenaIn[2]: f = lena()In[3]: from numpy.fft import fft2 #unnecessary if you invoke ipython with -pylabIn[4]: F = fft2(f)In[5]: imshow(real(F))

Anil C R Image Processing

IntroductionSome Theory

Doing the Stuff in PythonDemo(s)Q and A

Demo: Cells

Input ImageAnil C R Image Processing

IntroductionSome Theory

Doing the Stuff in PythonDemo(s)Q and A

Demo: Cells

OutputAnil C R Image Processing

IntroductionSome Theory

Doing the Stuff in PythonDemo(s)Q and A

Demo: CellsProcedure

Consider the image:

Anil C R Image Processing

IntroductionSome Theory

Doing the Stuff in PythonDemo(s)Q and A

Demo: CellsProcedure Cont’d

Find the variance of the neighborhood of each pixel, store themas a 2D array

Anil C R Image Processing

IntroductionSome Theory

Doing the Stuff in PythonDemo(s)Q and A

Demo: CellsProcedure Cont’d

Find the variance of the neighborhood of each pixel, store themas a 2D array

Anil C R Image Processing

IntroductionSome Theory

Doing the Stuff in PythonDemo(s)Q and A

Demo: CellsProcedure Cont’d

Find the variance of the neighborhood of each pixel, store themas a 2D array

Anil C R Image Processing

IntroductionSome Theory

Doing the Stuff in PythonDemo(s)Q and A

Demo: CellsProcedure Cont’d

Variance map (V )V > E{V}

Anil C R Image Processing

IntroductionSome Theory

Doing the Stuff in PythonDemo(s)Q and A

Demo: CellsProcedure Cont’d

Variance map (V )V > E{V}

Anil C R Image Processing

IntroductionSome Theory

Doing the Stuff in PythonDemo(s)Q and A

Demo: CellsProcedure Cont’d

Algorithm on the cell image:

Anil C R Image Processing

IntroductionSome Theory

Doing the Stuff in PythonDemo(s)Q and A

Q and A

Anil C R Image Processing