signals & systemsce.sharif.edu/courses/90-91/2/ce242-3/resources/root/... · 2020. 9. 7. ·...
TRANSCRIPT
Lecture 16
Signals & SystemsIntroduction to Compressed Sensing
Adapted from:• M. Davenport, M. F. Duarte, Y. C. Eldar, G. Kutyniok, “Introduction to Compressed Sensing”, 2011• J. Romberg, “Imaging via Compressive Sampling”, IEEE Signal Processing Magazine, 2008• M. Davenport, “Compressed Sensing: Theory and Practice”
Ali Soltani-Farani
Abbas HosseiniAbbas Hosseini
Spring 2012
Lecture 16
Digital Revolutiong
If we sample a band-limited signal at twice its highest
frequency, then we can recover it exactly
Whittaker-Nyquist-Kotelnikov-Shannon
Sharif University of Technology, Department of Computer Engineering, signals & systems2
Lecture 16
Sensor Explosionp
Sharif University of Technology, Department of Computer Engineering, signals & systems3
Lecture 16
Data Delugeg
By 2011, ½ of digital universe will have no home
[The Economist – March 2010]
Sharif University of Technology, Department of Computer Engineering, signals & systems4
Lecture 16
Motivations
sample NC K Storesample Compress K Store
K
decompressN
Sharif University of Technology, Department of Computer Engineering, signals & systems5
Lecture 16
Motivations
Original Picture
Nonlinear ReconstructionUsing 10% of CoefficientsCoefficients
WaveletR i
Histogram of C ffi iRepresentation Coefficients
Sharif University of Technology, Department of Computer Engineering, signals & systems6
Lecture 16
Motivations
Why go to so much effort to acquire all the data when most f h t t ill b th ?of what we get will be thrown away?
Reducing number of SensorsReducing number of Sensors
Reducing measurement time
Very important in MRI
Reducing sampling ratesReducing sampling rates
Sharif University of Technology, Department of Computer Engineering, signals & systems7
Lecture 16
Compressed Sensingp g
Compressed Sensing is a method for:p g
Sampling Sparse signals with a rate much lower than
iproposed by Nyquist
Reconstructing signal using samples with quality
comparable to compressed signals
Sharif University of Technology, Department of Computer Engineering, signals & systems8
Lecture 16
Sparsity & k-Sparsityp y p y
5-Sparse Approximately Sparse
Sharif University of Technology, Department of Computer Engineering, signals & systems9
Lecture 16
What DO Compressing Algorithms DO?p g g
Transforming the signal to an orthonormal basis that most of the desired signals are sparse in thatdesired signals are sparse in that.
Taking K largest coefficients in that basis.
Sharif University of Technology, Department of Computer Engineering, signals & systems10
Lecture 16
Generalized Notion of Samplingp g
In common image sampling we measure values of each pixel. We can
look at this as:
Sharif University of Technology, Department of Computer Engineering, signals & systems11
Lecture 16
Generalized Notion of Samplingp g
Instead of a single pixel, take any linear function:
1 1 2 2
1 1
= , , = , , , = , m m
m m n n
y x y x y xY X
Sharif University of Technology, Department of Computer Engineering, signals & systems12
Lecture 16
Compressive Sensing [Donoho; Candes, Romberg, Tao - 2004]p g [ g ]
Sharif University of Technology, Department of Computer Engineering, signals & systems13
Lecture 16
Sparsity Through Historyp y g y
Willi f O G d Ri h (1795) Constantin CarathéodoryWilliam of Occam (1288-1348 AD)
“Entities must not bel i li d
Gaspard Riche (1795)
algorithm for estimating the parameters of a few
Constantin Carathéodory1907
Given a sum of K sinusoids we can recover from 2K+1multiplied
unnecessarily”the parameters of a few complex exponentials
( )( ) i i
kj t
ix t e
we can recover from 2K+1 random samples
1
( ) i
kj t
ii
x t e
Sharif University of Technology, Department of Computer Engineering, signals & systems14
1i 1i
Lecture 16
Sparsity Through Historyp y g y
Arne Beurling (1938)
Given a sum of K impulses we can recover from only a
Ben Tex (1965)Given a signal with
bandlimit B, we can corrupt i t l f l th 2 /Bpiece of the Fourier Transform an interval of length 2π/B
and still recover perfectly
1
( ) ( )k
i ii
x t t t
Sharif University of Technology, Department of Computer Engineering, signals & systems15
Lecture 16
Sparsityp y
N
1
j jj
x
ampl
es
K N
N S
a
Large Coefficients
Sharif University of Technology, Department of Computer Engineering, signals & systems16
Lecture 16
How can we exploit this prior knowledge of sparsity?p p g p y
Key Questions:y Q
How to design the sensing matrix, with minimum rows, while preser ing the str ct re of the original signal?preserving the structure of the original signal?
How to recover the original signal from the measurements?
Sharif University of Technology, Department of Computer Engineering, signals & systems17
Lecture 16
Matrix Designg
Restricted Isometry Property (RIP)
For any pair of k-sparse signals and 1x 2x
21 2 21 1
x x
2
1 2 2
1 1x x
Sharif University of Technology, Department of Computer Engineering, signals & systems18
Lecture 16
Random Measurements
Choose a random matrix:
Fill out the entries of with i.i.d samples from a sub-Gaussian
distribution
( log( ))M O k N k
Stable: Information preserving, robust to noise
Democratic: Each measurement has “equal weight” Democratic: Each measurement has “equal weight”
Universal: Will work with any fixed orthonormal basis
Sharif University of Technology, Department of Computer Engineering, signals & systems19
Lecture 16
Signal Recoveryg y
Given y x e
Find x
Ill-posed inverse problem
Sharif University of Technology, Department of Computer Engineering, signals & systems20
Lecture 16
Signal Recovery: g y
Sharif University of Technology, Department of Computer Engineering, signals & systems21
Lecture 16
Signal Recovery in noiseg y
Optimization based methods
1 2ˆ arg min s.t
Nx x y x
Greedy/Iterative Algorithms
Nx
OMP, StOMP, ROMP, CoSaMP, Thresh, SP, IHT
kx x1
0 12 2ˆ kx xx x C e C
k
Sharif University of Technology, Department of Computer Engineering, signals & systems22
Lecture 16
Compressive Sensing in Practicep g
• Tomography in medical imaging – each projection gives you a set of Fourier coefficients
– fewer measurements mean
� more patientsp
� sharper images
� less radiation exposure
• Wideband signal acquisition – framework for acquiring sparse, wideband signals
– ideal for some surveillance applications
• “Single-pixel” camera
Sharif University of Technology, Department of Computer Engineering, signals & systems23
Lecture 16
Single Pixel Camerag
Sharif University of Technology, Department of Computer Engineering, signals & systems24
Lecture 16
Image Acquisitiong q
Sharif University of Technology, Department of Computer Engineering, signals & systems25