compressed sensing in measurement how does it work, why … · what \signal processing" means. is...
TRANSCRIPT
-
Compressed Sensing in MeasurementHow does it work, why – and when?
Claudio Narduzzi
Department of Information EngineeringUniversity of Padua – Italy
Scuola “Italo Gorini” 2019
Napoli, Italy - September 4, 2019
-
Presentation Outline
1 Compressed IntroWhat is compressed sensing?Where’s the problem, really?Beginnings – and some application examples
2 CS Example: Frequency Domain MeasurementMeasuring a sinewave
3 Sensing or Sampling?
4 Solution of CS problemsSparse recovery and the `1-normSensing Scheme DesignRandom sensing matrix – Signal Processing analysis
5 A Look at CS ApplicationsBiomedical ImagingSensor monitoring application
6 Conclusions
-
Compressed IntroCS Example: Frequency Domain Measurement
Sensing or Sampling?Solution of CS problems
A Look at CS ApplicationsConclusions
What is compressed sensing?Where’s the problem, really?Beginnings – and some application examples
Compressed Intro
C. Narduzzi Compressed Sensing in Measurement
-
Compressed IntroCS Example: Frequency Domain Measurement
Sensing or Sampling?Solution of CS problems
A Look at CS ApplicationsConclusions
What is compressed sensing?Where’s the problem, really?Beginnings – and some application examples
What is compressed sensing?
Let x be a (large) vector of N signal values. Consider matrix Φ,having size M × N with M � N.
The product y = Φ · x yields a (much) smaller, compressed vectorof data:
y = Φ · xCall Φ the sensing matrix, and y the vector of measured values.
C. Narduzzi Compressed Sensing in Measurement
-
Compressed IntroCS Example: Frequency Domain Measurement
Sensing or Sampling?Solution of CS problems
A Look at CS ApplicationsConclusions
What is compressed sensing?Where’s the problem, really?Beginnings – and some application examples
Questions
Do the measured values y represent in compressed form thesame information as x, or is anything lost in the process?
Conditions on x?
Conditions on Φ?
Answers – Sparsity
Primary assumption: signal x is sparse “in some domain”
Sensing matrix Φ is “reasonably well-behaved”
If needed, x can be recovered from y “almost surely”
Characterisation and analysis of uncertainty are needed
C. Narduzzi Compressed Sensing in Measurement
-
Compressed IntroCS Example: Frequency Domain Measurement
Sensing or Sampling?Solution of CS problems
A Look at CS ApplicationsConclusions
What is compressed sensing?Where’s the problem, really?Beginnings – and some application examples
A look past the “WOW” effect
An engineer typically finds the idea quite cool:
“measure little, learn a lot”
A kind of game of “hide and seek” . . .
. . . but, as soon as one gets into details, the mathematics mayget somewhat scary
or, theoretical details that are important to the understandingof CS may be missed, with disappointing results.
Well, surely that’s what researchers are for, right?
C. Narduzzi Compressed Sensing in Measurement
-
Compressed IntroCS Example: Frequency Domain Measurement
Sensing or Sampling?Solution of CS problems
A Look at CS ApplicationsConclusions
What is compressed sensing?Where’s the problem, really?Beginnings – and some application examples
What’s this lecture about
The purpose of this lecture is:
go through CS, and survive;
get somewhat closer to an engineering viewpoint of CS.
We can get there. . .
. . . if you are not afraid of matrices and have at least an idea ofwhat “signal processing” means.
Is CS useful in measurement?
C. Narduzzi Compressed Sensing in Measurement
-
Compressed IntroCS Example: Frequency Domain Measurement
Sensing or Sampling?Solution of CS problems
A Look at CS ApplicationsConclusions
What is compressed sensing?Where’s the problem, really?Beginnings – and some application examples
What is a “sparse” signal ?
x = Ψ · aA linear transformation allows to map x into a vector a that issparse, i.e., it has few non-zero elements.
C. Narduzzi Compressed Sensing in Measurement
-
Compressed IntroCS Example: Frequency Domain Measurement
Sensing or Sampling?Solution of CS problems
A Look at CS ApplicationsConclusions
What is compressed sensing?Where’s the problem, really?Beginnings – and some application examples
Signal model: the “sparsifying” matrix
x = Ψ · a
In CS parlance, Ψ is called a sparsifying matrix, because it mapsthe signal into a domain where its representation is sparse.
the equation gives a linear relationship between x and a;
Ψ may be a transform matrix (e.g., a discrete Fouriertransform), but needs not be, nor even be orthogonal.
Equation x = Ψ · a can be considered a model of the measurand.
C. Narduzzi Compressed Sensing in Measurement
-
Compressed IntroCS Example: Frequency Domain Measurement
Sensing or Sampling?Solution of CS problems
A Look at CS ApplicationsConclusions
What is compressed sensing?Where’s the problem, really?Beginnings – and some application examples
Putting it all together . . .
y = Φ · Ψ · agiven M “compressive” measurements in vector y
with known sensing matrix Φ and sparsifying matrix Ψ
find the solution a of: y = Φ ·Ψ · a = A · a
But:dim[y]=M, dim[a]=N with M�N: underdetermined ?
C. Narduzzi Compressed Sensing in Measurement
-
Compressed IntroCS Example: Frequency Domain Measurement
Sensing or Sampling?Solution of CS problems
A Look at CS ApplicationsConclusions
What is compressed sensing?Where’s the problem, really?Beginnings – and some application examples
Where’s the problem, really?
it is assumed that a has only K non-zero elements (K -sparse)
if K < M the equation is actually overdetermined, BUT
we do not know where non-zero elements of a are!
Sparsity constraint: find a as the maximally sparse solution
C. Narduzzi Compressed Sensing in Measurement
-
Compressed IntroCS Example: Frequency Domain Measurement
Sensing or Sampling?Solution of CS problems
A Look at CS ApplicationsConclusions
What is compressed sensing?Where’s the problem, really?Beginnings – and some application examples
Formalising the approach
The problem can be formally expressed in mathematical terms:
the count of non-zero elements is a (pseudo)-norm (`0);
recast the problem as constrained optimisation:
mina ‖a‖0 subject to: y = ΦΨa
All nice and clean, but:
this problem has combinatorial complexity;
if a is assumed to have K non-zero elements, there are(NK
)possible combinations
C. Narduzzi Compressed Sensing in Measurement
-
Compressed IntroCS Example: Frequency Domain Measurement
Sensing or Sampling?Solution of CS problems
A Look at CS ApplicationsConclusions
What is compressed sensing?Where’s the problem, really?Beginnings – and some application examples
When did it start?
Compressed Sensing (CS) has been developed for over ten years asan attractive and sophisticated mathematical theory
Candés, 2006
[VOLUME 25 NUMBER 2 MARCH 2008]
SP Magazine, 2008
C. Narduzzi Compressed Sensing in Measurement
-
Compressed IntroCS Example: Frequency Domain Measurement
Sensing or Sampling?Solution of CS problems
A Look at CS ApplicationsConclusions
What is compressed sensing?Where’s the problem, really?Beginnings – and some application examples
Solution of CS problems
Convex optimisation approach
replace `0 norm by `1 norm and solve:
mina ‖a‖1 subject to: y = ΦΨa
or, using Lagrange multipliers:
mina[‖a‖1 + λ‖y −ΦΨa‖2
]interest in compressed sensing sparked by significant progressin `1–`2 optimisation
“seminal papers” written by mathematicians and algorithmists
C. Narduzzi Compressed Sensing in Measurement
-
Compressed IntroCS Example: Frequency Domain Measurement
Sensing or Sampling?Solution of CS problems
A Look at CS ApplicationsConclusions
What is compressed sensing?Where’s the problem, really?Beginnings – and some application examples
Spreading interest . . .
and the measurement community took notice:
Introduzione al campionamento compressoSparsità e l’equazione Ax = y
Emanuele Grossi
DAEIMI, Università degli Studi di Cassinoe-mail: [email protected]
Gorini 2010, Pistoia
C. Narduzzi Compressed Sensing in Measurement
-
Compressed IntroCS Example: Frequency Domain Measurement
Sensing or Sampling?Solution of CS problems
A Look at CS ApplicationsConclusions
What is compressed sensing?Where’s the problem, really?Beginnings – and some application examples
Measurement application 1: Single-Pixel Camera
incident light (image) modulated by a digital micromirrordevice (DMD);
digital micromirror device (DMD) DMD structure
each micromirror changes tilt individually, sending light eithertowards or away from detector;
modulated image sent to single pixel detector.
C. Narduzzi Compressed Sensing in Measurement
-
Compressed IntroCS Example: Frequency Domain Measurement
Sensing or Sampling?Solution of CS problems
A Look at CS ApplicationsConclusions
What is compressed sensing?Where’s the problem, really?Beginnings – and some application examples
Single-pixel camera - recovery
acquisition of detector output with M different randommodulation patterns (different mirror tilt angles);
M digitised values from one high-resolution ADC;total number of pixels = DMD elements = N � M;
Digital Signal Processing Group, Rice University
C. Narduzzi Compressed Sensing in Measurement
-
Compressed IntroCS Example: Frequency Domain Measurement
Sensing or Sampling?Solution of CS problems
A Look at CS ApplicationsConclusions
What is compressed sensing?Where’s the problem, really?Beginnings – and some application examples
Variation on a theme – single-pixel hyperspectral imaging
Single detector makes complex sensing feasible:
high-sensitivity detectors, optical spectrum analyser (OSA).
F. Magalhes, M. Abolbashari, F.M. Arajo, M.V. Correia, F. Farahi, “High-resolution hyperspectral single-pixelimaging system based on compressive sensing”, Opt. Eng. 51(7), (May 25, 2012)
C. Narduzzi Compressed Sensing in Measurement
-
Compressed IntroCS Example: Frequency Domain Measurement
Sensing or Sampling?Solution of CS problems
A Look at CS ApplicationsConclusions
What is compressed sensing?Where’s the problem, really?Beginnings – and some application examples
Measurement application 2: Monitoring RF Spectral Bands
Y.C. Eldar, M. Mishali et al. several papers, see:http://webee.technion.ac.il/people/YoninaEldar/index.php
a modulated wideband converter can be realised by a RFhardware circuit implementation (shown);
MWC enables multi-band detection of RF signals.
C. Narduzzi Compressed Sensing in Measurement
-
Compressed IntroCS Example: Frequency Domain Measurement
Sensing or Sampling?Solution of CS problems
A Look at CS ApplicationsConclusions
What is compressed sensing?Where’s the problem, really?Beginnings – and some application examples
Modulated wideband converter (MWC)
Y.C. Eldar, M. Mishali et al. several papers, see:http://webee.technion.ac.il/people/YoninaEldar/index.php
Note how signals widely spaced in frequency are aliased intonarrower bands, that require lower sampling rates.
C. Narduzzi Compressed Sensing in Measurement
-
Compressed IntroCS Example: Frequency Domain Measurement
Sensing or Sampling?Solution of CS problems
A Look at CS ApplicationsConclusions
Measuring a sinewave
CS Example: FrequencyDomain Measurement
C. Narduzzi Compressed Sensing in Measurement
-
Compressed IntroCS Example: Frequency Domain Measurement
Sensing or Sampling?Solution of CS problems
A Look at CS ApplicationsConclusions
Measuring a sinewave
A simple case: measuring a sinewave
How many samples do we need to represent a sinewave?
let the sinewave frequency be f0
(Shannon) a signal x(t), whose frequency band isupper-limited to fMAX , is completely determined from asequence of samples taken at uniformly spaced intervals:
T ≤ 12fMAX
in principle, the number of samples is infinite
x [n] with: −∞ < n < +∞.
C. Narduzzi Compressed Sensing in Measurement
-
Compressed IntroCS Example: Frequency Domain Measurement
Sensing or Sampling?Solution of CS problems
A Look at CS ApplicationsConclusions
Measuring a sinewave
“Degrees of freedom” – do we need infinite samples?
A sampled sinewave can be simply described by the equation:
x [n] = A0 cos(2πf0T · n + φ0)that contains just three unknowns: A0, f0 and φ0.
Other than by infinite observation, there seems to be no wayto distinguish in the time domain a sinewave with frequencyf0 from a sinewave of frequency: f
′0 = f0 + δ for any value of δ.
C. Narduzzi Compressed Sensing in Measurement
-
Compressed IntroCS Example: Frequency Domain Measurement
Sensing or Sampling?Solution of CS problems
A Look at CS ApplicationsConclusions
Measuring a sinewave
Linear/non-linear relationships
Using for x [n] = A0 cos(2πf0T · n + φ0) the alternative expression:
x [n] = A0ejφ0 · e j2πf0T ·n + A0e−jφ0 · e−j2πf0T ·n
one may note that samples x [n] are dependent:
linearly, on a complex parameter depending, in turn, on A0and φ0
through a non linear relationship, on frequency f0
If frequency f0 is known, samples x [n] depend linearly on just twoparameters – A0, φ0. Sinewave is sparse in the frequency domain.
C. Narduzzi Compressed Sensing in Measurement
-
Compressed IntroCS Example: Frequency Domain Measurement
Sensing or Sampling?Solution of CS problems
A Look at CS ApplicationsConclusions
Measuring a sinewave
Uniform sampling, known frequency – linear equation
T = 12f0 – under-determined system (rank-1 matrix)[x [0]
x [1]
]=
[12
12
−12 −12
][A0e
jφ0
A0e−jφ0
]
coherent sampling, T = 13f0 (exactly three samples perperiod) – A0 and φ0 can be determined (rank-2 matrix) x [0]x [1]
x [2]
=
12
12
12e
j 2π3
12e−j 2π
3
12e
j 4π3
12e−j 4π
3
[ A0e jφ0A0e
−jφ0
]
Required number of samples is finite and very small
C. Narduzzi Compressed Sensing in Measurement
-
Compressed IntroCS Example: Frequency Domain Measurement
Sensing or Sampling?Solution of CS problems
A Look at CS ApplicationsConclusions
Measuring a sinewave
Introducing a finite frequency grid
So far, so good:
either f0 unknown, acquire an infinite number of samples;
or, f0 known perfectly, acquire just three samples;
anything in between?
Let’s make life a bit easier:
define a discrete grid with “reasonably fine” step ∆F ;
frequency value can be given as f0 = k0 ·∆F for some k0 ∈ Zassume a finite frequency range: |f | ≤ fMAXand choose N so that fMAX∆F =
N2
the finer the grid, the larger N needs to be
C. Narduzzi Compressed Sensing in Measurement
-
Compressed IntroCS Example: Frequency Domain Measurement
Sensing or Sampling?Solution of CS problems
A Look at CS ApplicationsConclusions
Measuring a sinewave
Inverse DFT as “sparsifying matrix”
If T = 1N·∆F , the mathematical expression of sinewave samples
when f = k ·∆F is: x [n] = A02 ejφ0e j2π
k0nN + A02 e
−jφ0e−j2πk0nN
N time-domainsamples
inverse Discrete Fouriertransform (IDFT) matrix
N Fourier (DFT)coefficients
x = Ψ · aC. Narduzzi Compressed Sensing in Measurement
-
Compressed IntroCS Example: Frequency Domain Measurement
Sensing or Sampling?Solution of CS problems
A Look at CS ApplicationsConclusions
Measuring a sinewave
Sparsity and spectral leakage – everybody knows. . .
If f0 6= k0 ·∆F , all DFT coefficients may be non-zero, but:for each frequency component having frequency fx ∼= kx∆F ,DFT coefficient magnitude decreases as |k − kx | gets largerif the signal is compressible, that is:
|ak | ≤ C · |k − kx |−q with:q > 0
it can likewise be recovered from compressive measurements
C. Narduzzi Compressed Sensing in Measurement
-
Compressed IntroCS Example: Frequency Domain Measurement
Sensing or Sampling?Solution of CS problems
A Look at CS ApplicationsConclusions
Measuring a sinewave
“Sensing” the sinewave
y = Φ · x
Rather than acquire a (large) vector x of N signal samples, we usethe M × N sensing matrix Φ to acquire M � N measurements:
Write: Φ =
φT
1
φT2...
φTM
Then:
y1
y2...
yM
=φT
1x
φT2
x...
φTM
x
Each measurement is a linear combination of the signal samples.
C. Narduzzi Compressed Sensing in Measurement
-
Compressed IntroCS Example: Frequency Domain Measurement
Sensing or Sampling?Solution of CS problems
A Look at CS ApplicationsConclusions
Sensing or Sampling?
C. Narduzzi Compressed Sensing in Measurement
-
Compressed IntroCS Example: Frequency Domain Measurement
Sensing or Sampling?Solution of CS problems
A Look at CS ApplicationsConclusions
Compressed Sensing or Compressive Sampling?
CS is variously understood to be a short form of either:“compressed sensing”, or: “compressive sampling”
what’s the difference – if any at all?
let the generic signal – or “measurand” x(t) be “sensed”through a linear device, represented by math operator φ(t):
y(t) =
∫ +∞−∞
x(τ)φ(t − τ)dτ
let y(t) be uniformly sampled at suitable instants ti :
y(ti ) =
∫ +∞−∞
x(τ)φ(ti − τ)dτ
C. Narduzzi Compressed Sensing in Measurement
-
Compressed IntroCS Example: Frequency Domain Measurement
Sensing or Sampling?Solution of CS problems
A Look at CS ApplicationsConclusions
Comparison with the sensing matrixConsider a sequence of measurement vectors y
i, y
i+1, . . . and
compare, for ti = iNT :
CS measurement vector:
yi
y1[i ]
y2[i ]...
yM [i ]
=φT
1xi
φT2
xi...
φTM
xi
“linearly sensed” measurands:y1[i ]
y2[i ]...
yM [i ]
=
∫ +∞−∞ x(τ)φ1(iNT − τ)dτ∫ +∞−∞ x(τ)φ2(iNT − τ)dτ
...∫ +∞−∞ x(τ)φM(iNT − τ)dτ
Each element in the vector sequence y
i, y
i+1, . . . is obtained from
the original “signal” through filtering and (down)sampling.
C. Narduzzi Compressed Sensing in Measurement
-
Compressed IntroCS Example: Frequency Domain Measurement
Sensing or Sampling?Solution of CS problems
A Look at CS ApplicationsConclusions
Can’t we always see it this way?
The sensing matrix Φ has been interpreted as the mathematicaldescription of a bank of M linear, finite impulse response (FIR)filters that process input vectors x in parallel, followed by adownsampler.
Is that all it takes?
Actually, no.
multi-dimensional data (e.g., image processing), are alsoarranged into vectors – structure harder to understand/exploit.
CS conditions are usually discussed with regards to Φ and Ψ
examples where x is indeed a one-dimensional signal vectorare still useful and help get a better grasp on CS.
C. Narduzzi Compressed Sensing in Measurement
-
Compressed IntroCS Example: Frequency Domain Measurement
Sensing or Sampling?Solution of CS problems
A Look at CS ApplicationsConclusions
Relation to Sampling Theory
“Shannon theorem” (Cauchy, 1843): If a function f (t) contains nofrequencies higher than W cps, it is completely determined by giving its
ordinates at a series of points spaced 1/2W seconds apart
Can be seen as:
bandlimiting by a linear filter:
y(t) =
∫ +∞−∞
x(τ)φ(t − τ)dτ with: φ(t) = sin 2πWt2πWt
sampling the filtered version of x(t) at instants ti =i
2W :
y(ti ) =
∫ +∞−∞
x(τ)sinπ(i − 2W τ)π(i − 2W τ)
dτ
C. Narduzzi Compressed Sensing in Measurement
-
Compressed IntroCS Example: Frequency Domain Measurement
Sensing or Sampling?Solution of CS problems
A Look at CS ApplicationsConclusions
Sparse recovery and the `1-normSensing Scheme DesignRandom sensing matrix – Signal Processing analysis
Solution of CS problems
C. Narduzzi Compressed Sensing in Measurement
-
Compressed IntroCS Example: Frequency Domain Measurement
Sensing or Sampling?Solution of CS problems
A Look at CS ApplicationsConclusions
Sparse recovery and the `1-normSensing Scheme DesignRandom sensing matrix – Signal Processing analysis
“Forcing” sparsity
constrain the solution to be sparse:vector a should have few non-zero elements (`0 norm);it should satisfy the measurement equation: y = Φ ·Ψ · a
replace `0 norm by `1 norm and solve constrainedoptimisation problem:
mina‖a‖1 subject to: y = ΦΨa
Greedy algorithms
iterative, sub-optimal solution
good computational efficiency
C. Narduzzi Compressed Sensing in Measurement
-
Compressed IntroCS Example: Frequency Domain Measurement
Sensing or Sampling?Solution of CS problems
A Look at CS ApplicationsConclusions
Sparse recovery and the `1-normSensing Scheme DesignRandom sensing matrix – Signal Processing analysis
Where are the non-zeroes?
If we already knew where non-zero elements are, i.e., if vector ahad known support, solving the problem would be fairly simple.
x = ΨS · a
C. Narduzzi Compressed Sensing in Measurement
-
Compressed IntroCS Example: Frequency Domain Measurement
Sensing or Sampling?Solution of CS problems
A Look at CS ApplicationsConclusions
Sparse recovery and the `1-normSensing Scheme DesignRandom sensing matrix – Signal Processing analysis
Known-support reconstruction
Locations of non-zero columns in matrix ΨS coincide withthe support of vector a.Only the reduced vector aS contains non-zero elements.
x = ΨS · aSC. Narduzzi Compressed Sensing in Measurement
-
Compressed IntroCS Example: Frequency Domain Measurement
Sensing or Sampling?Solution of CS problems
A Look at CS ApplicationsConclusions
Sparse recovery and the `1-normSensing Scheme DesignRandom sensing matrix – Signal Processing analysis
Pseudo-inverse solution
Given a measurement vector y with M elements, and K < Mnon-zero elements of a, there are more equations than unknowns:
y = Φ ·ΨS · aS
Values of non-zero elements of a are determined by computing the(Moore-Penrose) pseudo-inverse:
aS =[(ΦΨS)
H (ΦΨS)]−1
(ΦΨS)H y
C. Narduzzi Compressed Sensing in Measurement
-
Compressed IntroCS Example: Frequency Domain Measurement
Sensing or Sampling?Solution of CS problems
A Look at CS ApplicationsConclusions
Sparse recovery and the `1-normSensing Scheme DesignRandom sensing matrix – Signal Processing analysis
Sparse recovery and the `1-norm
Different shape of `1 and `2 “balls” in R2
C. Narduzzi Compressed Sensing in Measurement
-
Compressed IntroCS Example: Frequency Domain Measurement
Sensing or Sampling?Solution of CS problems
A Look at CS ApplicationsConclusions
Sparse recovery and the `1-normSensing Scheme DesignRandom sensing matrix – Signal Processing analysis
Sparse recovery and the `1-norm, plus uncertainty
`1 norm minimises the number of “large elements”
C. Narduzzi Compressed Sensing in Measurement
-
Compressed IntroCS Example: Frequency Domain Measurement
Sensing or Sampling?Solution of CS problems
A Look at CS ApplicationsConclusions
Sparse recovery and the `1-normSensing Scheme DesignRandom sensing matrix – Signal Processing analysis
What makes a good CS scheme?
Sensing matrix design:
what are the requirements of a good sensing matrix?
are there specific design criteria?
since the measurement equation is:
y = ΦΨa
does design of Φ depend on matrix Ψ?
C. Narduzzi Compressed Sensing in Measurement
-
Compressed IntroCS Example: Frequency Domain Measurement
Sensing or Sampling?Solution of CS problems
A Look at CS ApplicationsConclusions
Sparse recovery and the `1-normSensing Scheme DesignRandom sensing matrix – Signal Processing analysis
How to set-up a CS problem
Possibly, most difficult part of the job. It requires:
analysis of the signal at hand to:consider a suitable linear signal model
define the sparsifying matrix
analysis/design of a sensing schemefind out corresponding sensing matrix
assess its properties w.r.t. the sparsifying matrix
C. Narduzzi Compressed Sensing in Measurement
-
Compressed IntroCS Example: Frequency Domain Measurement
Sensing or Sampling?Solution of CS problems
A Look at CS ApplicationsConclusions
Sparse recovery and the `1-normSensing Scheme DesignRandom sensing matrix – Signal Processing analysis
Recovery and unique solution
Remember the original problem is:
mina ‖a‖0 subject to y = ΦΨa
but we do not want to solve this one.
We require a K -sparse solution to be unique:
assume two solutions exist, a1 and a2, then:
ΦΨa1 6= ΦΨa1 → ΦΨ(a1 − a2) 6= 0for this to hold, any subset of 2K columns of Ω = ΦΨ mustbe linearly independent
C. Narduzzi Compressed Sensing in Measurement
-
Compressed IntroCS Example: Frequency Domain Measurement
Sensing or Sampling?Solution of CS problems
A Look at CS ApplicationsConclusions
Sparse recovery and the `1-normSensing Scheme DesignRandom sensing matrix – Signal Processing analysis
Maximal incoherence and random sampling
According to CS theory, Φ should be incoherent with Ψ.Coherence is a measure of similarity between Φ and Ψ, definedas:
µ(Φ,Ψ) = maxi ,j ‖〈φi , ψj〉‖ i = 1, 2, . . .M j = 1, 2, . . .N
where:incoherence 1 ≤ µ ≤
√N maximal coherence
Assume matrix Φ is built from a set of vectors forming anorthonormal random basis: it can be shown that:
µ(Φ,Ψ) =√
2 logN with “overwhelming probability”
(Note: for N = 100, one has:√N = 10, whereas
√2 loge N
∼= 3. Not thelower bound, but a reasonably good value.)
C. Narduzzi Compressed Sensing in Measurement
-
Compressed IntroCS Example: Frequency Domain Measurement
Sensing or Sampling?Solution of CS problems
A Look at CS ApplicationsConclusions
Sparse recovery and the `1-normSensing Scheme DesignRandom sensing matrix – Signal Processing analysis
Design of the sensing matrix
Random sensing matrices
this topic is among the most extensively discussed in CS
a well-known claim is that a random sensing matrix Φ allowsto satisfy key CS conditions “with overwhelming probability”
this is helped by the fact that sparsifying matrices are oftenorthonormal or, at least, “very close” to preserving vectornorms
design of deterministic Φ possible, met more limited interest
C. Narduzzi Compressed Sensing in Measurement
-
Compressed IntroCS Example: Frequency Domain Measurement
Sensing or Sampling?Solution of CS problems
A Look at CS ApplicationsConclusions
Sparse recovery and the `1-normSensing Scheme DesignRandom sensing matrix – Signal Processing analysis
Random sampling in a CS framework
Consider M � N values extracted from x by random sampling.This is equivalent to premultiplying ΦS (or Φ) by a (0, 1) randommatrix, with a single non-zero element per row, to obtain vector y:
y = Φ · x = Φ ·ΨS · aSC. Narduzzi Compressed Sensing in Measurement
-
Compressed IntroCS Example: Frequency Domain Measurement
Sensing or Sampling?Solution of CS problems
A Look at CS ApplicationsConclusions
Sparse recovery and the `1-normSensing Scheme DesignRandom sensing matrix – Signal Processing analysis
Random Sensing Matrix
simplest: random sampling (0− 1 matrix)other common possibilities:
* random matrix elements drawn from Gaussian probabilitydensity function
* Bernoulli matrix: elements assuming two values, [−1, 1] withequal probability – see: single-pixel camera, modulatedwide-band converter
Why “overwhelming probability”?
some signal vectors x may not always be recoverable
position and magnitude of non-zero elements in a is a prioriunknown, motivating analysis based on success probability
relating such probabilities to uncertainty is desirable
C. Narduzzi Compressed Sensing in Measurement
-
Compressed IntroCS Example: Frequency Domain Measurement
Sensing or Sampling?Solution of CS problems
A Look at CS ApplicationsConclusions
Sparse recovery and the `1-normSensing Scheme DesignRandom sensing matrix – Signal Processing analysis
Noise, stability and the Restricted Isometry Property (RIP)
In practice, attention has to be paid to the effects of noise anduncertainty. Consider:
y = ΦΨa + w = Ωa + w
where w is assumed to be additive white Gaussian noise.
Given a positive integer K , the isometry constant δK of Ω is thesmallest number such that, for any K -sparse vector a:
(1− δK )‖a‖22 ≤ ‖Ωa‖22 ≤ (1 + δK )‖a‖22 with δK < 1
δK is a sort of measure of the numerical conditioning of Ω. If Ψ isorthonormal, RIP can be referred to Φ only, becoming one of thesensing matrix design parameters.
C. Narduzzi Compressed Sensing in Measurement
-
Compressed IntroCS Example: Frequency Domain Measurement
Sensing or Sampling?Solution of CS problems
A Look at CS ApplicationsConclusions
Sparse recovery and the `1-normSensing Scheme DesignRandom sensing matrix – Signal Processing analysis
Noise effect on CS solutions
The K -sparse vector aK is an approximate solution to the “noisy”equation y = Ωa + w, bounded by:
‖aK − a‖2 < CN · ε+ CK · σK
where:
ε upper bounds sensing noise (on vector y);
σK bounds approximation error (on vector a);
CN , CK are constants
since CN > 1, sensing noise is amplified: employ low-noisesensors.
C. Narduzzi Compressed Sensing in Measurement
-
Compressed IntroCS Example: Frequency Domain Measurement
Sensing or Sampling?Solution of CS problems
A Look at CS ApplicationsConclusions
Sparse recovery and the `1-normSensing Scheme DesignRandom sensing matrix – Signal Processing analysis
RIP and its consequences
If Ω satisfies RIP of order 2K it is possible to find a K -sparsesolution aK , such that:
in the noiseless case (i.e., when w ∼= 0) it is always possible torecover a unique K -sparse solution x̂
if δ2K ≤√
2− 1, there exist two positive constants C0, C1that bound the distance of the recovered vector x̂ from theoriginal vector x:
‖x̂− x‖2 ≤ C0‖xK − x‖1√
K+ C1ε.
C. Narduzzi Compressed Sensing in Measurement
-
Compressed IntroCS Example: Frequency Domain Measurement
Sensing or Sampling?Solution of CS problems
A Look at CS ApplicationsConclusions
Sparse recovery and the `1-normSensing Scheme DesignRandom sensing matrix – Signal Processing analysis
A “greedy” alternative – orthogonal matching pursuit
define residue r. Given an estimate â, one has: r = y −Ωâcompute the N-size vector ΩHri (where
H denotes conjugate
transposition – the subscript is the iteration index)
the set of indices of the largest non-zero elements of a, calledthe support S , is found iteratively (initially, S0 = ∅):
Si = Si−1 ∪{
arg maxn
∥∥∥[ΩHri]n
∥∥∥2
}where n is the index for elements in ΩHri , as well as in a.
the pseudo-inverse solution then follows:
âSi =[(ΦΨSi )
H (ΦΨSi )]−1
(ΦΨSi )H y
C. Narduzzi Compressed Sensing in Measurement
-
Compressed IntroCS Example: Frequency Domain Measurement
Sensing or Sampling?Solution of CS problems
A Look at CS ApplicationsConclusions
Sparse recovery and the `1-normSensing Scheme DesignRandom sensing matrix – Signal Processing analysis
Orthogonal matching pursuit (OMP) – how does it work?
since S0 = ∅, initially âS0 = 0 and r1 = yconsider then the first iteration and look for the index of thelargest non-zero element of ΩHy:
arg maxn
∥∥∥[ΩHri]n
∥∥∥2
∼= arg maxn
∥∥∥[ΩHΩa]n
∥∥∥2
is it also the largest non-zero element of a?
“reasonably” it is, because of the restricted isometry property
then, OMP works like “peeling off” the most significantcontributions to y, one by one, in descending order
C. Narduzzi Compressed Sensing in Measurement
-
Compressed IntroCS Example: Frequency Domain Measurement
Sensing or Sampling?Solution of CS problems
A Look at CS ApplicationsConclusions
Sparse recovery and the `1-normSensing Scheme DesignRandom sensing matrix – Signal Processing analysis
How many measurements?
Generally, the number of measurements required to ensure the CSproblem has a unique K -sparse solution is of the order of:
K log
(N
K
)which holds for both `1 − `2 and greedy approaches.
(Note: for N = 100 and K = 2 (e.g., single sinewave), K logeNK∼= 8: less than
M = 10 random measurements may suffice, instead of N = 100 samples. The
number drops to 3 with coherent sampling – quite close in performance.)
If RIP of order 2K holds, there exists a positive constant C suchthat:
M ≥ C · K log(N
K
)C. Narduzzi Compressed Sensing in Measurement
-
Compressed IntroCS Example: Frequency Domain Measurement
Sensing or Sampling?Solution of CS problems
A Look at CS ApplicationsConclusions
Sparse recovery and the `1-normSensing Scheme DesignRandom sensing matrix – Signal Processing analysis
What all this means in practice...
According to CS theory, the acquisition stage:
is linear and can be modelled by a matrix equation;
is totally blind, i.e. no prior information is needed;
can be random, extracted by some probability distribution;
is required to satisfy RIP of order 2K ;
can provide compression, to a rate of nearly one order ofmagnitude.
What changes take place going from x to y?
C. Narduzzi Compressed Sensing in Measurement
-
Compressed IntroCS Example: Frequency Domain Measurement
Sensing or Sampling?Solution of CS problems
A Look at CS ApplicationsConclusions
Sparse recovery and the `1-normSensing Scheme DesignRandom sensing matrix – Signal Processing analysis
Random sensing matrix – Signal Processing analysis
Recall the sensing matrix Φ can be interpreted as a bank of M FIRfilters in parallel. Each sensing matrix row contains the coefficientsone FIR filter:
ym[i ] = φTm
xn =N−1∑n=0
x(iNT − nT )φm(nT )
where:
φm(nT ) ∈ {+1,−1}let: ∆F =
1NT
three filter frequencyresponses shown in plot
C. Narduzzi Compressed Sensing in Measurement
-
Compressed IntroCS Example: Frequency Domain Measurement
Sensing or Sampling?Solution of CS problems
A Look at CS ApplicationsConclusions
Sparse recovery and the `1-normSensing Scheme DesignRandom sensing matrix – Signal Processing analysis
Random sensing matrix as a filter bank
Each filter randomly weights signal frequencies up to 12T
Binary sequences are uncorrelated, average PSD being uniformbetween 0 and 12T (see plot on the left).
C. Narduzzi Compressed Sensing in Measurement
-
Compressed IntroCS Example: Frequency Domain Measurement
Sensing or Sampling?Solution of CS problems
A Look at CS ApplicationsConclusions
Sparse recovery and the `1-normSensing Scheme DesignRandom sensing matrix – Signal Processing analysis
Down sampling → random aliasing
continuously sampled signal, CSoperates on N-lengthconsecutive blocks
each measurement is takenfrom filter outputs at the rate ofone sample every NT seconds
multiple randomly aliasedsamples
any signal “randomly aliased” into frequency band [0, 12NT ]
recovery possible if, collectively, a “de-aliased” signal can bereconstructed from compressed measurements
C. Narduzzi Compressed Sensing in Measurement
-
Compressed IntroCS Example: Frequency Domain Measurement
Sensing or Sampling?Solution of CS problems
A Look at CS ApplicationsConclusions
Biomedical ImagingSensor monitoring application
A Look atCS Applications
C. Narduzzi Compressed Sensing in Measurement
-
Compressed IntroCS Example: Frequency Domain Measurement
Sensing or Sampling?Solution of CS problems
A Look at CS ApplicationsConclusions
Biomedical ImagingSensor monitoring application
Tomography - problem analysis
Main features of a tomographic problem:
medical imaging scanner expected output image:g(n1∆, n2∆), where ∆ is pixel size;
sensors acquire cross-section information from a fewdifferent angular positions only;
acquired signals contain spatially encoded frequencycomponents at a few frequencies.
C. Narduzzi Compressed Sensing in Measurement
-
Compressed IntroCS Example: Frequency Domain Measurement
Sensing or Sampling?Solution of CS problems
A Look at CS ApplicationsConclusions
Biomedical ImagingSensor monitoring application
Tomography case study
Example Test Image (Logan-Shepp Phantom)
acquisition along radials (b)
spatial Fourier transform
finite number of frequencies
E. J. Candés, J. Romberg, T. Tao, Robust UncertaintyPrinciples: Exact Signal Reconstruction from Highly
Incomplete Frequency Information , IEEE Trans. Inform.Theory, 52(2): 489 - 509.
C. Narduzzi Compressed Sensing in Measurement
-
Compressed IntroCS Example: Frequency Domain Measurement
Sensing or Sampling?Solution of CS problems
A Look at CS ApplicationsConclusions
Biomedical ImagingSensor monitoring application
What about noise?
Tomographic imaging requires solving equation:
y = Rg
where linear operator R represents sensing by the scanner (oftengiven as Radon transform), to obtain vector g (that containsimage pixels)
for noisy measurements, y = Rg + n
then, a minimal energy solution would be:
ĝ = ming‖g‖2 s.t. ‖y − Rg‖2 < ε
This produces artifacts - (see image).
C. Narduzzi Compressed Sensing in Measurement
-
Compressed IntroCS Example: Frequency Domain Measurement
Sensing or Sampling?Solution of CS problems
A Look at CS ApplicationsConclusions
Biomedical ImagingSensor monitoring application
Tomography - convex optimization
Artifacts in energy-constrained backpropagation are allowed by thesparsity-indifferent constraint
min total variation (TV): minimise energy of first-orderderivatives along x1, x2 axes:
‖g‖TV =∑n1,n2
√|D1 [g(n1, n2)] |2 + |D2 [g(n1, n2)] |2
ĝ = ming‖g‖TV s.t. ‖y − Rg‖2 < ε
' `1-norm bound on image wavelet coefficients.
C. Narduzzi Compressed Sensing in Measurement
-
Compressed IntroCS Example: Frequency Domain Measurement
Sensing or Sampling?Solution of CS problems
A Look at CS ApplicationsConclusions
Biomedical ImagingSensor monitoring application
Compressive Sensing MRI
MRI scans may employ different patterns;
acquiring enough scans to obtain diagnostic quality images isoften a lengthy process;
motion artifacts;
patient well-being can be affected (long spells of immobility,claustrophobia, difficult breathing);
reduce number of scans, preserve information;
fast MRI.
C. Narduzzi Compressed Sensing in Measurement
-
Compressed IntroCS Example: Frequency Domain Measurement
Sensing or Sampling?Solution of CS problems
A Look at CS ApplicationsConclusions
Biomedical ImagingSensor monitoring application
Sensor monitoring – Electrocardiogram (ECG)
long-term monitoring of ECG plotsis needed for several heart-relateddiseases
high sampling rate (250/360 Hztypical)
high-resolution ADC (16-bittypical) to account for signaldynamics and allow for baselinefluctuations
C. Narduzzi Compressed Sensing in Measurement
-
Compressed IntroCS Example: Frequency Domain Measurement
Sensing or Sampling?Solution of CS problems
A Look at CS ApplicationsConclusions
Biomedical ImagingSensor monitoring application
Random sensing matrix
Bernoulli matrix (randomly distributed ’+1’, ’−1’ values)obtained from pseudo-random binary sequence (PRBS)
fixed-length N-sample blocks
baseline removal
although noise level may increase,compressed measurementshave narrower dynamics (seeplot) – 8-bit digitization suffices
C. Narduzzi Compressed Sensing in Measurement
-
Compressed IntroCS Example: Frequency Domain Measurement
Sensing or Sampling?Solution of CS problems
A Look at CS ApplicationsConclusions
Biomedical ImagingSensor monitoring application
Working with compressed measurements – detection
A. Galli, C. Narduzzi, G. Giorgi, “ECG Monitoring and Anomaly Detection Based on Compressed Measurements”,2018 3rd International Conference on Biomedical Imaging, Signal Processing, Bari, Italy, 11–13 October 2018.
when a specific individual is monitored, basic shapes of ECGQRS-wave, T-wave and P-wave components are knowable
templates can be built in the compressed domain
detection of ECG wave components by simple matching
C. Narduzzi Compressed Sensing in Measurement
-
Compressed IntroCS Example: Frequency Domain Measurement
Sensing or Sampling?Solution of CS problems
A Look at CS ApplicationsConclusions
Ten years later
SP Magazine, 3/2018 SP Magazine, 4/2018
Community: https://nuit-blanche.blogspot.com/ (I. Carron)
C. Narduzzi Compressed Sensing in Measurement
-
Conclusions
analysis and exploitation of structure within data remains akey focus of research
advanced measurement applications are relying on increasinglysophisticated signal processing methods
Compressive Sensing is primarily a way to reduce theamount of sensor data;
sensing and reconstruction can be entirely separated, bothconceptually and, possibly, from the physical viewpoint;
open field of research for efficient reconstructionalgorithms;
enhance data acquisition (faster, less data) withoutsacrificing accuracy.
-
Thank You
M. Fornasier, H. Rauhut, Compressive sensing, ...
CS-based reconstruction of a fresco in Cappella Ovetari (Eremitani Church, Padua, Italy) from remaining fragments(in colour) and black and white photographs. Frescoes in Cappella Ovetari were hit and almost completely
destroyed in Allied air bombing during World War II. Arguably, an extreme approach to sparsity.
Compressed IntroWhat is compressed sensing?Where's the problem, really?Beginnings – and some application examples
CS Example: Frequency Domain MeasurementMeasuring a sinewave
Sensing or Sampling?Solution of CS problemsSparse recovery and the 1-normSensing Scheme DesignRandom sensing matrix – Signal Processing analysis
A Look at CS ApplicationsBiomedical ImagingSensor monitoring application
Conclusions