1 asu mat 591: opportunities in industry! asu mat 591 image processing science and robotic vision...
Post on 26-Dec-2015
216 Views
Preview:
TRANSCRIPT
1
ASU MAT 591: Opportunities in Industry!
ASU MAT 591ASU MAT 591
Image Processing ScienceImage Processing Scienceand Robotic Visionand Robotic Vision
Rod PickensRod PickensPrincipal Research EngineerPrincipal Research Engineer
Lockheed Martin, IncorporatedLockheed Martin, Incorporated
2
ASU MAT 591: Opportunities in Industry!
Signals and Processing
Signals– Analog and discrete signals– Dimensionality of signals
1-D signals Sounds (temporal), echocardiogram, seismic signal
2-D signals (this presentation) Images (spatial)
3-D signals Video sequences of images (spatial and temporal)
Signal processing– Synthesize and analyze signals– Filter signals using low-pass, band-pass, and high-pass filter– Modify signals such as warp, delay, stretch, rotate, shrink, …– Restore and enhance signals– Recognize patterns and detect signals
3
ASU MAT 591: Opportunities in Industry!
Signal Processing: Now
Animal
Touch
Vision
Hearing
Smell
Taste
Robotic
Touch
Vision
Hearing
Smell
Taste
5
ASU MAT 591: Opportunities in Industry!
Analysis and Synthesis of Light
)(tf )(tf
White Light Out
)(tf )(tf
)(wF
dtetfwF jwt)()(
dtetfwF jwt)()(
Fourier Analysis
dwewFtf jwt)()(
dwewFtf jwt)()(
Fourier SynthesisWhite Light In
Inverse Functions
6
ASU MAT 591: Opportunities in Industry!
Fourier Transforms are Inverse Functions
)((
))),(((
Functions Inverse
*),()(
Synthesize
*),()),((
Analyze
1
1
)(1
)(
),ωF(ωFTFT),ωF(ω
yxfFTFTf(x,y)
dxdyeF),ωF(ωFTf(x,y)
dxdyeyxfyxfFT),ωF(ω
yxyx
yxjyxyx
yxjyx
yx
yx
11 xx
7
ASU MAT 591: Opportunities in Industry!
Inverse Functions
DerivativeInv Fourier TransInv Radon TransWarp Correction
IntegralFourier TransformRadon Transform
Warp Data
()1f
()f
)(1 xffx )(xfy
)(1 yffy )(1 yfx 11 xx
8
ASU MAT 591: Opportunities in Industry!
Filtering
)(tf )(tf
)(tf f )(tf f
)(wF
White Light In
Filtering removes all but red colorsFiltering removes all but red colorsRed Light Out
9
ASU MAT 591: Opportunities in Industry!
Television
)(tf )(tf
)(tf f )(tf f
)(wF
Channel 6Filtering removes all but Channel 6Filtering removes all but Channel 6
Television Stations 3, 5, 6, 13, 15, …
Television
10
ASU MAT 591: Opportunities in Industry!
Television
)(tf )(tf
)(tf f )(tf f
)(wF
Television Stations 3, 5, 6, 13, 15, …
Channel 15Filtering removes all but Channel 15Filtering removes all but Channel 15
Television
11
ASU MAT 591: Opportunities in Industry!
Radio
)(tf )(tf
)(tf f )(tf f
)(wF
Radio Stations
Station 100.7Filtering removes all but Station 100.7Filtering removes all but Station 100.7
Radio Stations 91.5, 96.9, 100.7
Radio
12
ASU MAT 591: Opportunities in Industry!
Radio
)(tf )(tf
)(tf f )(tf f
)(wF
Radio Stations
Station 96.9Filtering removes all but Station 96.9Filtering removes all but Station 96.9
Radio Stations 91.5, 96.9, 100.7
Radio
13
ASU MAT 591: Opportunities in Industry!
Vision
)(tf )(tf
)(tf f )(tf f
)(wF
BookFiltering removes all but a bookFiltering removes all but a book
Scene of a Room: walls, books, desks, chairs,
windows,…
Robot vision
14
ASU MAT 591: Opportunities in Industry!
Vision
)(tf )(tf
)(tf f )(tf f
)(wF
Scene of a Room
TableFiltering removes all but a tableFiltering removes all but a table
Scene of a Room: walls, books, desks, chairs,
windows,…
Robot vision
15
ASU MAT 591: Opportunities in Industry!
Graphics to build a scene
dwewFwDtf jwtd )()()(
dwewFwDtf jwtd )()()(
Synthesis
)(wF
)(tfd )(tfd
Scene of a RoomDescriptor of scene is D(w)
All Room Contents
16
ASU MAT 591: Opportunities in Industry!
Data compression
)(tf )(tf
)(wF
Signal
Filter that eliminates less important data.
)(~tf )(
~tf
Approximation of Signal
17
ASU MAT 591: Opportunities in Industry!
Data compression goal
)(tf )(tf
)(~tf )(
~tf
)(wF
Signal
Filter that eliminates less important data.
Approximation of Signal
)()(~
tftf )()(~
tftf
19
ASU MAT 591: Opportunities in Industry!
FormatCorrect Errors
Preprocess Restore
Analyze Recognize
The Example Architecture
Format
Descriptions
DataCorrect Errors
Communications
Preprocess
NormalizeRemove NoiseRemove Distortions
Restore
Remove Sensor Effects
Analyze
Decompose Signals
Recognize
Label Signals
Will Discuss in more detail!
20
ASU MAT 591: Opportunities in Industry!
FormatCorrect Errors
Preprocess Restore
Analyze Recognize
Preprocess
Preprocess
Descriptions
Data
NormalizeRemove NoiseRemove Distortions
21
ASU MAT 591: Opportunities in Industry!
Noisy Input Image
Fourier Based Noise Filtering
From Jason Plumb at http://noisybox.net/weblog/
Clearer Output Image
Mostly Noise so is Zeroed
Mostly Signal
Fourier Synthesis
Fourier Analysis
Fourier Transform and Filter the Noise
22
ASU MAT 591: Opportunities in Industry!
Filtering and Enhancing Data
From Mathworks homepage at http://www.mathworks.com/
Math to follow
23
ASU MAT 591: Opportunities in Industry!
Filtering: Analysis
)(tf )(tf
)(wF
Image
dxdyeyxf),ωF(ω yxjyx
yx )(*),(
Analyze
Analysis
24
ASU MAT 591: Opportunities in Industry!
Filtering: Removing Noise
)(tf )(tf
)(wF
Filtering: removes noiseFiltering: removes noise
Image
otherwise 0
if
Filter22
yx
yxyxf
),ωF(ω),ω(ωF
25
ASU MAT 591: Opportunities in Industry!
Filtering: Synthesis
Enhanced
)(tf )(tf
)(tf f )(tf f
)(wF
Image
dxdyeF(x,y)f yxjyxff
yx )(*),(
Synthesize
Synthesis
26
ASU MAT 591: Opportunities in Industry!
Filtering
Enhanced
)(tf )(tf
)(tf )(tf
)(wF
Filtering: removes noiseFiltering: removes noise
Image
dxdyeFf(x,y)
),ωF(ω),ω(ωF
dxdyeyxf),ωF(ω
yxjyx
yxyx
yxf
yxjyx
yx
yx
)(
22
)(
*),(
Synthesize
otherwise 0
if
Filter
*),(
Analyze
Analysis
Synthesis
27
ASU MAT 591: Opportunities in Industry!
Enhancing the Data: Linear map
I=Intensity
I1
p(I1)
Input ImageIntensity Histogram
I2
p(I2)
Output ImageIntensity Histogram
(more contrast)
I1
I2
I2 = m* I1
Enhance (stretch) Using Linear Mapping
28
ASU MAT 591: Opportunities in Industry!
Warping data
From Mathworks homepage at http://www.mathworks.com/
Suppose we have unwanted camera motion.
29
ASU MAT 591: Opportunities in Industry!
Warping data
From Mathworks homepage at http://www.mathworks.com/
We can correct motion errors if we know motion model.
30
ASU MAT 591: Opportunities in Industry!
Warping data
From Mathworks homepage at http://www.mathworks.com/
31
ASU MAT 591: Opportunities in Industry!
Warping Correction is an Inverse Function
)(1 xffx )(1 yffy
WarpingCorrection
Warping
()1f
()f
)(xfy
)(1 yfx
33
ASU MAT 591: Opportunities in Industry!
Linear Algebra to Flip
x1
y1
x1
x2
x2=- x1
y1
y2
y2=y1
1
1
2
2
10
01
y
x
y
x
112
112
)(
)(
yygy
xxfx
x2
y2
34
ASU MAT 591: Opportunities in Industry!
Linear Algebra to Flip
x1
y1
x1
x2
x2=- x1
y1
y2
y2=y1
1
1
2
2
10
01
y
x
y
x
112
112
)(
)(
yygy
xxfx
I(x1,y1)
x2
y2
x2
y2
35
ASU MAT 591: Opportunities in Industry!
Linear Algebra to Flip
x1
y1
x1
x2
x2=- x1
y1
y2
y2=y1
1
1
2
2
10
01
y
x
y
x
112
112
)(
)(
yygy
xxfx
I(x1,y1)
I(x2,y2)
x2
y2
x2
y2
x2
y2
37
ASU MAT 591: Opportunities in Industry!
Linear Algebra to Flip
x1
y1
x1
x2
x2=- x1
x2
y2
y1
y2
y2=y1
1
1
2
2
10
01
y
x
y
x
112
112
)(
)(
yygy
xxfx
I(x1,y1)
I(x2,y2)=I(f(x1),g(y1))
x2
y2
x2
y2
x2
y2
38
ASU MAT 591: Opportunities in Industry!
Linear Algebra to Flip
x1
y1
x2
x1
x1=- x2
x2
y2
y2
y1
y1=y2
2
2
1
1
10
01
y
x
y
x
1221
1221
)(
)(
yyyg
xxxf
I(x2,y2)
39
ASU MAT 591: Opportunities in Industry!
Linear Algebra to Flip
x1
y1
x2
x1
x1=- x2
x2
y2
y2
y1
y1=y2
2
2
1
1
10
01
y
x
y
x
1221
1221
)(
)(
yyyg
xxxf
I(f-1(x2), g-1(y2))
I(x2,y2)
40
ASU MAT 591: Opportunities in Industry!
Linear Algebra to Flip
x1
y1
x1
x2
x2=- x1
x2
y2
y1
y2
y2=y1
1
1
2
2
10
01
y
x
y
x
12
12
yy
xx
I (x1,y1)=I(f-1(x2), g-1(y2))
I (x2,y2)
42
ASU MAT 591: Opportunities in Industry!
Linear Algebra to Flip and Shrink
x1
y1
x1
x2
x2
y2
y1
y2
y2 = -0.5 * y1
x2 = 0.5 * x1
12
12
*5.0
*5.0
yy
xx
1
1
2
2
5.00
05.0
y
x
y
x
43
ASU MAT 591: Opportunities in Industry!
Correcting warped data (camera motion)
From Mathworks homepage at http://www.mathworks.com/
))(),(( 21
121
1 ygyxfxI
))(),(( 1212 ygyxfxI ),( 11 yxI
),( 22 yxI
If we can determine f(), g(), f-1(), and g-1(), then we can correct camera motion!
11 xx
44
ASU MAT 591: Opportunities in Industry!
FormatCorrect Errors
Preprocess Restore
Analyze Recognize
Restoration
Restore
Descriptions
Data
Remove Sensor Effects
45
ASU MAT 591: Opportunities in Industry!
Restoring data for smear, optics,…
From Mathworks homepage at http://www.mathworks.com/
UsesLinear Systems
Theory
Next
Smear and optics can be viewed as filters that can degrade an image!
46
ASU MAT 591: Opportunities in Industry!
Restoring data for smear, optics,…
From Mathworks homepage at http://www.mathworks.com/
UsesLinear Systems
Theory
Next
Restoration
47
ASU MAT 591: Opportunities in Industry!
Restoration: Analysis
),( yxf ),( yxf
),( yx wwF
Image
dxdyeyxf),ωF(ω yxjyx
yx )(*),(
Analyze
Analysis
48
ASU MAT 591: Opportunities in Industry!
Filtering: Removing Smear
Smr-1(wx,wy) is a filter that removes smear or restores the original object.
Smr-1(wx,wy) is a filter that removes smear or restores the original object.
Image
)(
Filter Restoring1 ),ωF(ω,ωωSmr),ω(ωF yxyxyxf
),( yxf ),( yxf
),( yx wwF
50
ASU MAT 591: Opportunities in Industry!
Filtering
),( yxf ),( yxf
),( yx wwF
Image Restored to best look like original Object
Image Restored to best look like original Object
Image
dxdyeF(x,y)f
),ω)F(ω,ω(ωSmr),ω(ωF
dxdyeyxf),ωF(ω
yxjyxff
yxyxyxf
yxjyx
yx
yx
)(
1
)(
*),(
Synthesize
Restore
*),(
Analyze
Object
),( yxf f ),( yxf f
Smear inverted as a filterSmear inverted as a filter
51
ASU MAT 591: Opportunities in Industry!
Restoring data for smear, optics,…
From Mathworks homepage at http://www.mathworks.com/
UsesLinear Systems
Theory
Image(wx,wy) Next
52
ASU MAT 591: Opportunities in Industry!
Restoring data for smear, optics,…
From Mathworks homepage at http://www.mathworks.com/
UsesLinear Systems
Theory
Image(wx,wy)
Smr(wx,wy)*Image(wx,wy)
Next
53
ASU MAT 591: Opportunities in Industry!
Restoring data for smear, optics,…
From Mathworks homepage at http://www.mathworks.com/
UsesLinear Systems
Theory
Image(wx,wy)
Smr(wx,wy)*Image(wx,wy)
Image(wx,wy) *Smr-1(wx,wy)* Smr(wx,wy)
Image(wx,wy)= Image(wx,wy) *1(wx,wy )11 xx
54
ASU MAT 591: Opportunities in Industry!
FormatCorrect Errors
Preprocess Restore
Analyze Recognize
Synthesis and Analysis
Descriptions
Data
Decompose / Compose Signals - Transforms: Fourier, SVD, Wavelets - Statistical Analysis: parametric and non-parametric
Synthesize
Analyze
55
ASU MAT 591: Opportunities in Industry!
Fourier Transform
)(tf )(tf
White Light Out
)(tf )(tf
)(wF
dtetfwF jwt)()(
dtetfwF jwt)()(
Fourier Analysis
dwewFtf jwt)()(
dwewFtf jwt)()(
Fourier SynthesisWhite Light In
56
ASU MAT 591: Opportunities in Industry!
Fourier Transform
Magnitude Phase
From Wolfram homepage at http://documents.wolfram.com
57
ASU MAT 591: Opportunities in Industry!
Radon Transform
From Mathworks homepage at http://www.mathworks.com/
58
ASU MAT 591: Opportunities in Industry!
Wavelet Transform
From Wolfram homepage at http://documents.wolfram.com
59
ASU MAT 591: Opportunities in Industry!
Common Transforms
Fourier Discrete fourier Cosine Sine Hough Hadamard Slant Karhunen-Loeve Fast KL SVD Sinusoidal
60
ASU MAT 591: Opportunities in Industry!
Statistics
From Mathworks homepage at http://www.mathworks.com/
61
ASU MAT 591: Opportunities in Industry!
FormatCorrect Errors
Preprocess Restore
Analyze Recognize
Recognition
Recognize Descriptions
Data
Label Signals - Signal Detection - Pattern Recognition - Artificial Intelligence
62
ASU MAT 591: Opportunities in Industry!
Fea
tur e
1Feature 2 *
Fea
ture
1
Feature 2
Class 1(daisy)
Class 2(rose)
Class 3(sun flower)
* Features are mathematical measurements
Pattern Recognition
Classification
BayesianNeural netsNearest neighborsLinear
Transforms: Fourier, Wavelet, …Statistics: mean, st. dev, …Shape: Fourier, Hough, MomentsTexture: Cooccurrence, Eigen Filters, …
Analysis Tools Features
Feature 1: Hough measureFeature 2: 3rd Eigen Filter
Analysis
63
ASU MAT 591: Opportunities in Industry!
Mathematical Decisions
z
o
Class 1 is z
Class 2 is o
o
o
o
o
o
o
oo
oo
o
o
z
z
z
z
z
z
z
zz
z
zz
z
f1
f2
z
How do we separate the
classes?
o
oo
o
o
64
ASU MAT 591: Opportunities in Industry!
Mathematical Decisions
z
o
Class 1 is z
Class 2 is o
o
o
o
o
o
o
oo
oo
o
o
z
z
z
z
z
z
z
zz
z
zz
z
f1
f2
z
Linear decision
o
oo
o
o
65
ASU MAT 591: Opportunities in Industry!
Mathematical Decision
z
o
Class 1 is z
Class 2 is o
o
o
o
o
o
o
oo
oo
o
o
z
z
z
z
z
z
z
zz
z
zz
z
f1
f2
z
Linear decision
o
oo
o
o
66
ASU MAT 591: Opportunities in Industry!
Mathematical Decision
z
o
Class 1 is z
Class 2 is o
o
o
o
o
o
o
oo
oo
o
o
z
z
z
z
z
z
z
zz
z
zz
z
f1
f2
z
Quadratic decision
1or 1
BoundaryDecision Reasonable2
122
12 ffff
o
oo
o
o
67
ASU MAT 591: Opportunities in Industry!
Mathematical Decision
zClass 1 is z
z
z
z
z
z
z
z
zz
z
zz
z
f1
f2
zone class isobject then 12
12 ff
68
ASU MAT 591: Opportunities in Industry!
Mathematical Decision
o
twoclass isobject then 1212 ff
Class 2 is o
o
o
o
o
o
oo
oo
o
of1
f2
o
oo
o
o
69
ASU MAT 591: Opportunities in Industry!
Mathematical Decision
z
o
Class 1 is z
Class 2 is o
o
o
o
o
o
o
oo
oo
o
o
z
z
z
z
z
z
z
zz
z
zz
z
f1
f2
z
-1
3
one class isobject so 113)1(3 2 o
oo
o
o
twoclass isobject then 1
one class isobject then 12
12
212
ff
ff
70
ASU MAT 591: Opportunities in Industry!
Isolate Object: Segmentation
From Mathworks homepage at http://www.mathworks.com/
Analysis Synthesis
71
ASU MAT 591: Opportunities in Industry!
Analyze Object: Features
- Length- Width- Contour- Orientation
- Edges- Skeleton- Texture Details- Intensity
From Mathworks homepage at http://www.mathworks.com/
72
ASU MAT 591: Opportunities in Industry!
Matched Filtering (registration)
From Mathworks homepage at http://www.mathworks.com/
Input Image or Iin(x,y)
73
ASU MAT 591: Opportunities in Industry!
Matched Filtering (registration)
From Mathworks homepage at http://www.mathworks.com/
Exemplar (reference) or Iref(x,y)
Input Image or Iin(x,y)
74
ASU MAT 591: Opportunities in Industry!
Matched Filtering (registration)
From Mathworks homepage at http://www.mathworks.com/
Exemplar (reference) or Iref(x,y)
Input Image or Iin(x,y)
2112222 )),(),((min(, yxIyxIyx refin
x2
error
x
75
ASU MAT 591: Opportunities in Industry!
Matched Filtering (registration)
From Mathworks homepage at http://www.mathworks.com/
Exemplar (reference) or Iref(x,y)
Input Image or Iin(x,y)
2112222 )),(),((min(, yxIyxIyx refin
x2
error
x
Actually search form min of x,y simultaneously!
76
ASU MAT 591: Opportunities in Industry!
FormatCorrect Errors
Preprocess Restore
Analyze Recognize
Image Processing: Summary
Format
Descriptions
DataCorrect Errors
Communications
Preprocess
NormalizeRemove NoiseRemove Distortions
Restore
Remove Sensor Effects
Analyze
Decompose Signals
Recognize
Label Signals
77
ASU MAT 591: Opportunities in Industry!
References
Fundamentals of Image Processing by Jain
Digital Image Analysis by Gonzalez and Wintz
Pattern Recognition by Fukunaga
Pattern Recognition Principles Tou and Gonzalez
Detection, Estimation, and Modulation Theory by Van Trees
Pattern Classification by Duda and Hart
Robot by Hans Moravec (graphics from www.amazon.com)
78
ASU MAT 591: Opportunities in Industry!
Touch
Vision
Hearing
Smell
Taste
Signal Processing: 50 years from now
Touch
Vision
Hearing
Smell
Taste
Robotic Evolved
Hmmm.
Vision
79
ASU MAT 591: Opportunities in Industry!
Touch
Vision
Hearing
Smell
Taste
Signal Processing: 50 years from now
Touch
Vision
Hearing
Smell
Taste
Robotic Evolved
Wow!
Vision
top related