1 sampling and reconstruction of stream of diracs — beyond...
TRANSCRIPT
1
Sampling and Reconstruction of Stream ofDiracs — Beyond Shannon Theorem
EECS 551 Project Final Report
QINGSI WANG, ZHAOSHI MENG and HONG LI
Abstract
In this report, we present the completion of milestones thatwe set in the previous writeup. Inaddition to implementing existing algorithms in literature, including the annihilating-filter-based and thesubspace-based methods, we analyze the theoretical and numerical performance of the algorithm withrespect to different kernels (in filter-based implementation) and various controllable parameters (orderof annihilating filter (M ), number of samples (N ), bandwidth of the filter (B) andetc.). Based on theabove analysis, we also propose practical methods to obtainstable and robust reconstruction. We finallyconclude our research by extensive empirical studies.
I. INTRODUCTION
In this section, we first briefly review our learning objectives of this project, and then providean overview of related work in literature. At the end of this section, the completion of the desiredmilestones is reported.
A. Review of learning objectives
In this project, we have studied the following signal sampling and reconstruction problem.The classic Shannon sampling theorem basically states thatfor any bandlimited signals, theperfect reconstruction is theoretically possible provided that the sampling frequency is at least theNyquist rate. However, this result did not provide the solution to the sampling and reconstructionproblem of non-bandlimited signals, for instance the stream of Diracs of which the supportin frequency domain is of an infinite length. To close this theoretic gap, Vetterliet al. [1]proposed the concept ofrate of innovation, or number of degrees of freedom per unit of time,and they further presented that sampling and reconstruction of classes of signals with finite rateof innovation (e.g., the stream of Diracs), can be solved with an adequate sampling kernel anda sampling rate greater than or equal to the rate of innovation.
Through this project, we aimed to understand this state-of-the-art algorithm and its extension.To meet this end, we started from analyzing the simple peak-finding algorithm. In our lastwriteup, after investigating the strength, weakness and the fundamental limits of the simplepeak-finding algorithm, we proposed one possible frameworkof the new algorithm and thisframework is repeated in Figure 1. We observed that existingalgorithms for reconstructingsignals with finite rate of innovation are essentially basedon the same framework. Hence, itmotivated us to first identify the conceptual components of the existing algorithms and conductperformance evaluation before we progress with the study ondeveloping improved algorithms,which can be the first step of thereproducible research[2].
2
h(t)estimate the
parameters of
y[n]
s(t) y(t) t 1 t 2sample
y(t)
y[n]
Fig. 1. The modified system block diagram proposed in writeupone.
B. Related work
Extensive studies have been conducted in the sampling and reconstruction of signals withfinite rate of innovation. In [1], Vetterliet al. proposed that these signals could be uniformlysampled at rate of innovation using appropriate kernels, and then be perfectly reconstructed.Specifically, this claim was justified by the authors for concrete examples like infinite periodicand finite-length streams of Diracs, nonuniform splines, and piecewise polynomials, and couldbe naturally extended to other cases. In this sampling and reconstruction algorithm, annihilatingfilter method is a pivotal component. For infinite periodic streams of Diracs, the annihilatingfilter equation is constructed and solved in the frequency domain, while for finite-length case,the above operation is directly performed in the time domain. Similar approaches can be appliedfor splines and polynomials.
However, in [1] only deterministic and noiseless signals were considered. Although the authorssuggested that oversampling and singular value decompostion (SVD) could be used to tacklethe presence of noise, no theoretical justification was provided to guarantee the performance ofthe algorithm under noise. In [3], the original method was extended to the noisy case. In thiswork, the total least-squares approach was utilized to solve the annihilating filter equation inmoderate noisy cases, and Cadzow’s Iterative Denoising wasproposed to alleviate aggravatednoise. In addition, Bluet al. in [3] suggesed that one should be aware of the tradeoff betweenthe algorithm performance and the computational complexity. To elaborate this point, the authorsprovided analysis on the Cramar-Rao Bound of the One-Diraccase. In later work [4], the authorspresented a more robust approach to achieve numerically improved performance. This methodwas constructed in the framework of the subspace approach, and a modified version of algorithmthat could improve the resolution of the algorithm was proposed as well.
C. Completion of milestones in writeup one
Our completion of the milestones set in the previous writeupis reported as follows.• Understand the pivot concepts and principle of the algorithm1 in [1];
We are currently with a relatively comprehensive understanding of the annihilating-filter andthe subspace methodologies, which are two main approaches to realize the sampling andreconstruction of FRI signals. We also identified the functionalities of convolution kernelsin this algorithm. This part will be reported in Section 2.
• Implement this algorithm in MATLAB and test its performance;
1We would like to refer to this algorithm and its extensions asFRI algorithms in the rest of this report
3
We implemented the algorithm based on the annihilating-filter method and the subspacemethod, respectively, and we also designed appropriate test-benches to demonstrate thefundamental transition behavior of this algorithm. In addition, we evaluated the effects ofdifferent parameters on the algorithm performance throughnumerical studies. This partwill be reported in Section 2 and Section 3.
• Try to derive one or more extensions with better performancefor some specific applicationscenarios.
Basically, we did not obtain conceptually innovative extensions for this algorithm, but weproposed and implemented some techniques to improve the stability of the simulation basedon practical considerations. This part will be reported in Section 3 and Appendix.
II. THE FRAMEWORK OF FRI ALGORITHMS
Before we rigorously investigate the FRI algorithms proposed in literature, we would like todemonstrate an intuitive argument which in first place helped us convinced with the sufficiency ofsampling rate greater than the rate of innovation. This argument was partially reported in writeupone, and we review it here to facilitate the flow of understanding in this section. Consider thefollowing argument based on the framework in Figure 1: for a stream ofK Diracs s(t) = aT dwherea =
[
a1 a2 · · · aK
]Tandd =
[
δ(t − t1) δ(t − t1) · · · δ(t − tK)]T
, we then have
y(t) = h(t) ∗ s(t)
= h(t) ∗
(
[
a1 a2 · · · aK
]
δ(t − t1)δ(t − t1)
· · ·δ(t − tK)
)
,
⇓
Y (jω) = H(jω)
(
[
a1 a2 · · · aK
]
ejωt1
ejωt2
· · ·ejωtK
)
,
⇓
Yd(ejω) = Hd(e
jω)
(
[
a1 a2 · · · aK
]
ejωt1
ejωt2
· · ·ejωtK
)
.
Thus, for any fixedw =[
ω1 ω2 · · · ω2K
]
, we have
y = aT FH,
where
y =[
Yd(ejω1) Yd(e
jω2) · · · Yd(ejω2K)
]T,
F =
ejω1t1 ejω2t1 · · · ejω2K t2
ejω1t2 ejω2t2 · · · ejω2K t1
......
ejω1tK ejω2tK · · · ejω2KtK
,
4
and
H =
Hd(ejω1)
Hd(ejω2)
.. .Hd(e
jω2K )
.
The above system of equations consists of2K equations with2K unknown, i.e.K time instantsandK amplitudes of Diracs. it is then promising that we can solve this equation system, thusreconstructing the stream of Diracs with only2K samples. This insight provides us an overviewon the breakthrough from the bandwidth limit of the classic Shannon sampling theorem.
A. Annihilating-filter Method
In general, if the samples of signal could be expressed as
S[n] =
K−1∑
k=0
ckunk , ∀n ∈ Z, (1)
then the annihilating-filter method can be applied to find thevalues ofck anduk. This methodis elaborated in [1] and we provide a sketch of it as follows.
Step 1. Finding the annihilating filter. The filter coefficients are expected to satisfy that
K∑
l=0
A[l]S[n − l] = 0, ∀n ∈ Z,
or equivalentlyS · A = 0,
where
S =
...... · · ·
...S[0] S[−1] · · · S[−K]S[1] S[0] · · · S[−(K − 1)
......
. . ....
S[k] S[k − 1] · · · S[0]...
... · · ·...
,
and
A =
A[0]A[1]
...A[k]
.
Provided the number of samplesN ≥ 2K + 1, this system of equations will admit asolution whenrank(S) = K.
Step 2. Finding theuk. With the obtained filter coefficientsA[l], we can construct the filter asA(z) =
∑Kl=0 A[l]z−l, and then the valuesuk are simply the roots ofA(z).
5
Step 3. Finding theck. With the obtained values ofuk, it suffices to takeK equations in (1) andsolve this system forck, i.e.,
1 1 · · · 1u0 u1 · · · uK−1...
... · · ·...
u(k−1)0 u
(k−1)1 · · · u
(k−1)K−1
·
c0
c1...
cK−1
=
S[0]S[1]
...S[K − 1]
,
which has a unique solution whenuk 6= ul, ∀k 6= l
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1−3
−2
−1
0
1
2
3
4
5
6Original Diracs and Filtered Signal
Time
Am
plitu
de
diracfiltered signalnoisyless samples
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1−3
−2
−1
0
1
2
3
4
5
6Sampled Signal (noisy), # of samples: 5 SNR= 20
TimeA
mpl
itude
filtered signalnoisy samples
−15 −10 −5 0 5 10 15−0.5
0
0.5
1
1.5Frequency Response of the Kernel
Frequency
Am
plitu
de
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1−3
−2
−1
0
1
2
3
4
5
6Reconstructed Diracs and Signal
Time
Am
plitu
de
estimated diracsfiltered signalestimated filtered signal
Fig. 2. The intermediate outputs of the annihilating-filter-based FRI algorithm: an example of the sinc kernel.
B. Sampling and reconstruction of streams of Diracs based onannihilating filter method
In this part, we briefly introduce the application of annihilating filter method in the recon-struction of stream of Diracs [1] [3].
1) Reconstruction of Periodic Continuous-time FRI Signalswith Bandlimit Kernel: In thispart, we assume the stream of Diracs is periodic continuous-time, and we use a bandlimit kernelto filter. Since non-periodic signal could be seen as a truncated version of periodic signal, thissignal model is more general. Specifically, the signalx(t) has the following form:
x(t) =∑
n∈
K−1∑
k=0
ckδ(t − tk − nτ)
Periodic continuous-time FRI signals could be generally expressed through Fourier series
x(t) =∑
m∈Z
X[m]ej(2πmt/τ).
6
The Fourier series coefficients are
X[m] =1
τ
K−1∑
k=0
cke−j(2πmtk/τ), m ∈ Z. (2)
Then the samplesyn could be expressed as
yn =∑
m
X[m]〈hB(t − nT ), ej(2πmt/τ)〉
=∑
m
X[m]HB(2πm
τ)ej(2πmnT/τ)
=M∑
m=−M
X[m]ej(2πmnT/τ).
HenceX[m] could be calculated from samples of the signal via discrete-time Fourier transform(DTFT). Now with obtainedX[m], (2) is essentially in the same form as (1), and thereforetk’sand ck’s could be solved through annihilating filter method.
2) Reconstruction of Finite-length FRI Signals with Gaussian Kernel: Consider a continuous-time signal with a finite number of weighted Diracs:
x(t) =
K−1∑
k=0
ckδ(t − tk).
If we use Gaussian kernel to filter this signal, then the samples obtained could be expressedas:
yn =〈hσ(t − nT ), x(t)〉
=K−1∑
k=0
cke−(tk−nT )2/2σ2
=
K−1∑
k=0
(cke−t2
k/2σ2
) · entkT/σ2
· e−n2T 2/2σ2
(3)
Let S[n] = en2T 2/2σ2
yn, ak = cke−t2
k/2σ2
, anduk = etkT/σ2
, then (3) is equivalent to
S[n] =
K−1∑
k=0
ckunk , ∀n = 0, ..., N − 1.
Thus we could use annihilating filter method to solvetk’s and ck’s, even without Fouriertransformation.
7
3) Solving the annihilating filter equation by Total Least Square (application of SVD):Solvingthe annihilating filter equation:
S · A = 0 (4)
is straightforward in the noiseless case. However, in the presence of noise, equation (4) is notsatisfied exactly, thus posing a total least square(TLS) problem [5]. The optimal solution ofAis the eigenvector associated with the smallest eigenvalueof S
TS. We adopt this TLS approach
in our implementation.Note that the estimation of amplitudes of Diracs, i.e.ck’s, is also implemented using total
least square, since the inaccurate rootsuk is equivalent to add noise to the Vandermonde matrix.
C. Sampling and reconstruction of stream of Diracs based on Subspace method
Although great efforts have been made to improve the robustness of the annihilating filtermethod [1] [3], it is still not sufficient for good overall numerical performance, due to the ill-conditioning of the problem, and the additional computational complexity of the aforementionedrefinements.
In [4], the well-known subspace method, or state-space method [6], is used in reconstructionof the Dirac signal in the presence of noise. Due to limited space, we only briefly introduce themethodology of subspace-based algorithms [4], and we have implemented it as a comparison tothe annihilating filter method.
Step 1 Given a set of the Fourier series coefficientsX[m], construct anM × N Hankel matrixas follows:
X =
X[0] X[1] · · · X[N − 1]X[1] X[2] · · · X[N ]
......
X[M − 1] X[M ] · · · X[M + N − 2]
,
whereM, N > K.Step 2 Compute the singular value decomposition ofX, i.e., X = USV H . Find the principal
left and right singular vectors,US andVS, as the singular vectors corresponding to theKlargest singular values ofX.
Step 3 Estimate the signal poleszk = e−jω0tk by computing the eigenvalues of a matrixZ, definedas
Z = U †S · US.
where (·) and (·) denote the operations of omitting the first and the last row of(·)respectively.
Step 4 Find the coefficientsck as a least-squares solution to the Vandermonde system as in theannihilating filter method.
III. A LGORITHM PERFORMANCE ANALYSIS
In this section, we present our understanding and analysis of the performance of implementedalgorithms through simulation studies. In the first part of the evaluation, we only consider thecase ofK = 2, i.e. only 2 Diracs, and we fix the amplitudes of two Diracs to be one. Let∆t bethe difference of time between two Diracs, i.e, the time resolution of Diracs. We first examine the
8
robustness of the algorithm with one specific kernel (we present our results for thesinc kernel)in the presence of noise by evaluating the performance undervarious values of the SNR and wethen repeat this experiment under different settings of∆t for the same kernel. The measure thatwe use for evaluation is the mean-squared-error(MSE) between the reconstructed Diracs and theoriginal ones, for both time locations and amplitudes. By conducting a similar experiment, wealso compare the performance of the annihilating-filter-based algorithm with differen kernels andthat of the subspace-based algorithm. At the end of the first part, we demonstrate the fundamentaltransitional behavior of the FRI algorithms.
In the second part, we consider the performance of the algorithm under different combinationsof parameters, wherein we setK = 3 for the sake of a larger range of each controllable parameter.
A. Time resolution, kernel impacts and transitional behavior
−10 0 10 20 30 40 50 60−90
−80
−70
−60
−50
−40
−30
−20
−10
0
SNR (dB)
MS
Et (
dB)
MSEt vs SNR
∆ t=0.05
∆ t=0.1
∆ t=0.25
B
A
(a) MSE of reconstructed locations(sinc filter)
−10 0 10 20 30 40 50 60−70
−60
−50
−40
−30
−20
−10
0
10
SNR (dB)
MS
EA (
dB)
MSEA vs SNR
∆ t=0.05
∆ t=0.1
∆ t=0.25
(b) MSE of reconstructed amplitudes(sinc filter)
−10 0 10 20 30 40 50 60−70
−60
−50
−40
−30
−20
−10
0
10
SNR (dB)
MS
E o
f Am
plitu
de
MSE of Amplitude vs SNR
∆T=0.05∆T=0.10∆T=0.25
(c) MSE of reconstructed locations(sinc square filter)
−10 0 10 20 30 40 50 60−90
−80
−70
−60
−50
−40
−30
−20
−10
0
SNR (dB)
MS
E o
f Loc
atio
n
MSE of Location vs SNR
∆T=0.05∆T=0.10∆T=0.25
(d) MSE of reconstructed amplitudes(sinc square filter)
Fig. 3. The algorithm performance withsinc under different time resolutions.
1) The comparison of different∆T : Applying the sinc kernel, the algorithm performancefor location reconstruction under different time resolutions is shown in the Figure 3(a). Foreach individual curve, it can be observed that there exists atransition point (B) below which thealgorithm works well for all the SNR levels. By regression, the slope is found to be approximately-1. Beyond point (B), there is a sharp edge from (B) to (A). Theeffectiveness of the algorithmfalls promptly. Under the SNR greater than (A), the algorithm fails and the estimated location
9
becomes a random result. Since at point (A), one Dirac is thought to be perfectly estimated whilethe other one is uniformly located in the interval [0,1]. Thus, the variance of the location is 1/12or -10.8dB, as suggested by the simulation results. For different time resolutions, the Diracsignal with larger time difference is more robust to noise. Similarly, as shown in Figure 3(b),for different SNR levels, the MSE of amplitude follows almost the same behavior as that oflocation.
−10 0 10 20 30 40 50 60−90
−80
−70
−60
−50
−40
−30
−20
−10
0
SNR (dB)
MS
Et (
dB)
MSEt vs SNR (∆t = 0.25)
subspacesincsinc
square
Gaussian σ=0.2
(a) ∆t = 0.25
−10 0 10 20 30 40 50 60−80
−70
−60
−50
−40
−30
−20
−10
0
SNR (dB)M
SE
t (dB
)
MSEt vs SNR (∆t = 0.10)
subspacesincsinc
square
Gaussian σ=0.2
(b) ∆t = 0.10
−10 0 10 20 30 40 50 60−70
−60
−50
−40
−30
−20
−10
0
SNR (dB)
MS
Et (
dB)
MSEt vs SNR (∆t = 0.05)
subspacesincsinc
square
Gaussian σ=0.2
A
B
(c) ∆t = 0.05
Fig. 4. Comparison among different kernels in location estimation.
2) The comparison of different kernels and the subspace method: The performance compari-son of applying different kernels is shown in the Figure 4. Itis observed that whatever kernel isapplied, the curve is always characterized by the two transition points (A) and (B) as describedin the previous part and the MSE converges to -10.8dB as the noise power goes beyond thethreshold point (A). But thesinc kernel constantly outperforms over the other two kernels whilethe Gaussian kernel is relatively the worst.
The reason that thesinckernel outperformssinc squarekernel is due to the better conditioningof the Fourier matrix. Recall that in the first step of annihilating filter method we calculate thefrequency coefficient of the signalX[m]’s by solving a discrete Fourier transform equation. Forsinc kernel, this could be done by simply performing inverse transform. However in the case
10
of sinc squarewe need to take the frequency coefficients of the filter into consideration (thesecoefficients act as weights on theX[m]’s). Therefore the numerical property of the matrix in thiscase is not as good as Fourier matrix, as in the case ofsinc. This could be one of the reasons.
In the case of Gaussian kernel, due to the different application of annihilating filter method(not in frequency domain), the conditioning property of theToeplitz matrix in the presence ofnoise is severely deteriorated. Thus the performance is worse. Another limitation of Gaussiankernel method is that when the sample number is large, the algorithm meets overflow problem.
We also present the results of subspace-based method as comparison. It can be seen thatsubspace-based method behaves differently to annihilating filter method, above certain thresh-old SNR it is not as good as annihilating filter method, however below that critical point, itoutperforms. Interestingly, its MSE also converges approximately to−10dB.
3) Fundamental transition and general asymptotic behaviors: We conclude our the discussionin this part by plotting the MSE of reconstructed Diracs on the plane of(SNR, ∆T ), as shownin Figure 5. In the first three figures for the annihilating filter method, there is clearly a transitionfrom “failure of reconstruction” to “tolerably successfulreconstruction”. We also note that theboundary depends on both SNR and∆T . In the last figure we present the result of the sameexperiment for the subspace-based method, and the boundarydoes not largely depend on∆T ,which means that this algorithm is more robust in the case where Diracs are closely spaced.
B. Effects of controllable parameters:N, M, B
In this part, we consider the periodic continuous-time caseto illustrate our analysis andunderstanding of the algorithm in both theoretical and numerical ways. Other signal modelscan be analyzed in a similar way.
We note that the aforementioned reconstruction algorithm has the following general steps:• Obtain the appropriate samples(see Appendix for details inimplementation of sampling)
that fit the model of annihilating filter method. In the case ofperiodic continuous-time FRIsignals, we obtainX[m] by performing Discrete Time Fourier Transform(DTFT);
• Solve the annihilating filter equation to obtain the annihilating filter coefficients;• Factoring the z-transform of filter coefficients and find the zeros;• Recover the time locations of Diracs;• Construct a Vandermonde matrix and recover the amplitudes of Diracs by solving a least
squares problemIn these steps, there are three adjustable parameters that are very critical to the performance
of the algorithm:N, M , andB. In the following part, we carefully analyze how to determinethese parameters to strengthen the robustness and the performance of the FRI algorithms.
1) The Effect of Number of SamplesN : In the first step we solve a linear equationY = FX,whereF is aN ×(2M +1) Fourier matrix andY is the matrix of (noisy) samples. It is clear thatN should be no less than2M +1. This limit provides the minimum value of parameterN whenM is fixed. Hence, an interesting question rises:how many samples are actually “useful”, or“necessary” in solving2M +1 unknownX[m]’s. To answer this open question, we observe that,assuming the bandwidth of the filter isBandwidth, all the Fourier coefficients that lie outsidethe bandwidth are zeros, and thus they provide no information in recoveringX[m]. The actualnumber of non-zerosfrequencysamples(the corresponding Fourier coefficients of time-domainsamples) are
Neffective ≈ N ×Bandwidth
fs= N ×
Bandwidth
N/τ= Bandwidth · τ (5)
11
0 0.05 0.1 0.15 0.2 0.25−10
0
10
20
30
40
50
60
MSEt versus (SNR,∆T)
∆T
SN
R (
dB)
0.050.1
0.150.2
020
4060
−100
−80
−60
−40
−20
0
SN
R (dB
)
MSEt versus (SNR,∆T)
∆T
(a) MSE of reconstructed locations(sinc filter)
0 0.05 0.1 0.15 0.2 0.25−10
0
10
20
30
40
50
60
∆T
SN
R (
dB)
MSEa versus (SNR,∆T)
0.050.10.15
0.2
0 20 40 60
−80
−60
−40
−20
0
20
SNR (dB)
MSEa versus (SNR,∆T)
∆T
MS
Ea (
dB)
(b) MSE of reconstructed amplitudes(sinc filter)
0 0.05 0.1 0.15 0.2 0.25−10
0
10
20
30
40
50
60
∆T
SN
R (
dB)
MSEt versus (SNR,∆T)
0.050.10.15
0.20 20 40 60
−100
−80
−60
−40
−20
0
SN
R (dB
)
MSEt versus (SNR,∆T)
∆T
(c) MSE of reconstructed locations(sinc square(triangular) filter)
0 0.05 0.1 0.15 0.2 0.25−10
0
10
20
30
40
50
60
∆T
SN
R (
dB)
MSEt versus (SNR,∆T)
0.050.1
0.150.2
020
4060
−16
−14
−12
−10
−8
−6
∆T
MSEt versus (SNR,∆T)
SNR (dB)
MS
E (
dB)
(d) MSE of reconstructed locations(subspace method)
Fig. 5. Fundamental transition behavior of the FRI algorithm
12
Bandwidth
F_sample
F_sample*2
Bandwidth
N_e ective=3
N_e ective=3
Increasing sampling frequency does not change N_e ective.
Fig. 6. An example of spectrum of signal and frequency samples
which does not depend onN . Thus, increasingN in the time-domain only increases the numberof zero-valued samples in frequency domain, while the actual information contained doesn’tincrease and we should still obtain similarX[m]’s in the first step.
Moreover, the matrixF is deterministic, independent of samples, and then the numericalproperty of the operation doesn’t depend onN either.Therefore, we conclude that given fixedM and B, number of samplesN doesn’t affect the performance of the algorithm.
The simulation results shown in Figure 7 assert our statement on the effects ofN . GivenK = 3 andM = 4, whenN > 2M + 1, the performance is not affected byN .
2) The Effect of Number of Annihilating Filter CoefficientsM : In the second step of recon-struction, we need to solve an M-order annihilating equation to find filter coefficients. It is clearthat M should be greater than or equal toK, the number of Diracs. ThusK is the minimumpossible value forM . LargerM results in more filter coefficients, and then more zeros(or roots)of the corresponding z-transform in the third step. In noiseless case, the zeros(or roots) of the z-transform of filter coefficients should beuk = e−j(2πtk/τ), thus lying on the unit circle. However,due to the effect of noise, the actual position of zeros may beviolated, thus not be on the unitcircle. In fact, the condition number of the Toeplitz matrixis very large in the presence of noise,this will result in a numerical instability.
Another reason that may cause poor performance of the algorithm is the ill-conditioning of theroot-finding algorithm. In Matlab, the root-finding commandroots is performed by calculating
13
−10 0 10 20 30 40 50 60−80
−70
−60
−50
−40
−30
−20
−10
SNR (dB)
MS
E o
f Loc
atio
n
MSE of Location vs SNR
N=9N=11N=13N=15
(a) MSE of reconstructed locations for different N
−10 0 10 20 30 40 50 60−70
−60
−50
−40
−30
−20
−10
0
10
SNR (dB)
MS
E o
f Am
plitu
de
MSE of Amplitude vs SNR
N=9N=11N=13N=15
(b) MSE of reconstructed amplitudes for different N
Fig. 7. The algorithm performance under different settingsof N (K = 3, M = 4).
the eigenvalues of the companion matrix2:
F =
−X[1] −X[2] · −X[m − 1] −X[m]1 0 · · · 0 00 1 · · · 0 0...
...0 0 · · · 1 0
Therefore it is clear that the noisy samplesyn’s will result in inaccurate estimatedX[m]’s, andthe eigenvalue problem could be ill-conditioned. Hence, the roots of the z-transform of filtercoefficients could be inaccurate as well.
To overcome the above two difficulties, we propose to choose parameterM larger thanK.Thus we will obtain more thanK zeros by solving the equation. And then we only select theKzeros that areclosest to the unit circleto calculate the time locations of the Diracs, and abandonthe spurious roots. This manipulation actually “pushes” the noise to the noise subspace(whichwe will illustrate more clearly later), thus separated the signal and noise. Experiments show thatthe ratio of the largest singular values to the K-th largest smallest singular value of the Toeplitzmatrix, which we call “the modified condition number”, is much smaller than the original one.Therefore the numerical stability is improved. This methodalso effectively reduces the conditionnumber of the companion matrix in the root-finding algorithm, thus improving the accuracy ofthe roots.
In applying this strategy, one must be aware that increasingM means increasing the numberof non-zero samples in frequency domain. As a result, we musthave enough bandwidth of thefilter. Therefore2M + 1 should be less or equal toNeffective calculated in (5). So increasingMactually is essentiallyoversampling.
However,M should not be too large, which will result in over-modeling [4], and even thebestK roots that are closest to unit circle are not accurate in thiscase either.
The results of empirical experiment are shown in Figure 9. Given K = 3, when M increasefrom 3 to 4, the performance is improved. However, further increase of M induces instability
2Please refer to the MATLAB help manual
14
−1 −0.5 0 0.5 1 1.5 2
−1
−0.5
0
0.5
1
Real Part
Imag
inar
y P
art
Example of four−zero system
Fig. 8. An example of multiple zero system. We only keep the K zeros close to the unit circle.
problem as expected.Above analysis and insights provide us with an approach to estimate the threshold SNR value
for this annihilating filter method to work. Assuming the SVDof Toeplitz matrixS(M ×M) isgiven by:
S = USV T
in the noiseless case, where
S =
σ1
σ2
. . .σM
contains all the singular values.In the noise case, the matrix becomesS = S + E, whereE is the noise matrix. In terms
of SVD, the signal power lies in the signal subspace, which corresponds to the firstK largestsingular values of the Toeplitz matrix. And the additional noise could be approximately seen ascomponents in the orthogonal subspace of signal subspace, which corresponds to the(K + 1)th
to Mth singular values. (This assumption is often easy to be satisfied, since the noise is oftenorthogonal to signals.)Therefore the threshold noise power should be the minimum value whosecorresponding singular value exceeds the smallest singular value of signal, i.e.σK . Abovethis critical SNR, the firstK singular values of the matrixX should not changed significantly,no matter how much the SNR is. We present a numerical experiment result to support ourobservation in Figure 10.
15
−10 0 10 20 30 40 50 60−100
−90
−80
−70
−60
−50
−40
−30
−20
−10
SNR (dB)
MS
E o
f Loc
atio
nMSE of Location vs SNR
M=3M=4M=5M=6
(a) MSE of reconstructed locations for different M
−10 0 10 20 30 40 50 60−70
−60
−50
−40
−30
−20
−10
0
10
SNR (dB)
MS
E o
f Am
plitu
de
MSE of Amplitude vs SNR
M=3M=4M=5M=6
(b) MSE of reconstructed amplitudes for different M
Fig. 9. The algorithm performance under different settingsof M (K = 3) .
−10 0 10 20 30 40 50 600
0.5
1
1.5
2
2.5
3
3.5
4
4.5
5
SNR (dB)
Val
ue o
f sin
gula
r va
lue
σ 2
The second singular value of Toeplitz matrix (K=2,M=3)
∆T=0.05∆T=0.10∆T=0.20
ThresholdSNR
Fig. 10. TheKth Singular Value of Toeplitz Matrix in different SNR
We could see that above the threshold SNR, theKth singular value(and along with all thesingular values larger than it) does not changed significantly. But when the SNR is below thatvalue, theKth singular value changed rapidly, which means the noise has destroyed the signalsubspace, and the algorithm should not working. A comparison between this figure and thefigures before could confirm our assertion. This method couldbe used to estimate the thresholdSNR for this annihilating filter method.
3) The Effect of Bandwidth ofsinc kernel B: Following the argument in Part (1) and (2),given N , M fixed andN > 2M + 1, we are aware of the fact thatincreasingB is equivalentto increase non-zero samples in the frequency domain, i.e.Neffective. However, as long asNeffective is greater than or equal to2M +1, denoted byNcritical, increasingB cannot improvethe performance further.The statement is proved by the simulation result below in Figure 11.Given K = 3 and M = 4, when the number of effective samples is smaller than9 · Ncritical),increasing B from 3 to 4 or equivalently increasing the bandwidth from 7 to 9, the performanceis improved. However, further increasingB making effective samples larger than2M + 1 = 9,the performance remains the same.
16
−10 0 10 20 30 40 50 60−90
−80
−70
−60
−50
−40
−30
−20
−10
SNR (dB)
MS
E o
f Loc
atio
n
MSE of Location vs SNR
B=3B=4B=6B=10
(a) MSE of reconstructed locations for different B
−10 0 10 20 30 40 50 60−70
−60
−50
−40
−30
−20
−10
0
10
SNR (dB)
MS
E o
f Am
plitu
de
MSE of Amplitude vs SNR
B=3B=4B=6B=10
(b) MSE of reconstructed amplitudes for different B
Fig. 11. The algorithm performance under different settings of B (K = 3, M = 4).
IV. CONCLUSION
The sampling theorems for classes of signals with finite rateof innovation can be regarded asthe generalization of the classic “bandlimited and sinc kernel” case. As the core of this project,the principle that goes beyond the Shannon sampling theoremand the pivotal concepts in thealgorithm first proposed in [1], are extensively studied. Through simulation studies, we identifiedthe fundamental transition behavior of this algorithm, andwe also explored the effects of severalcontrollable parameters on the performance of algorithm. As the practice of the first step inthe reproducible research, we completed the milestones setin the previous proposal and forthe future work, we may consider the effect of the underlyinguniform sampling method in thealgorithm, and discuss the potentials of other sampling strategies.
V. ACKNOWLEDGEMENT
We would like to thank Prof. Nadakuditi for his suggestions and insights during the project.
REFERENCES
[1] Martin Vetterli, Pina Marziliano, and Thierry Blu. Sampling signals with finite rate of innovation.IEEE Transactions onSignal Processing, 50(6):1417–1428, 2002.
[2] Pina Marziliano. Reproducible research: A case study ofsampling signals with finite rate of innovation.Proc. ICASSP2007, 4:IV–1265–IV–1268, April 2007.
[3] T. Blu, P. L. Dragotti, M. Vetterli, P. Marziliano, and L.Coulot. Sparse sampling of signal innovations.Signal ProcessingMagazine, IEEE, 25(2):31–40, March 2008.
[4] Irena Maravic and Martin Vetterli. Sampling and reconstruction of signals with finite rate of innovation in the presence ofnoise. IEEE Transactions on Signal Processing, 53:2788–2805, 2004.
[5] Wynn C. Stirling Todd K. Moon.Mathematical Methods and Algorithms for Signal Processing. Prentice Hall, 1999.[6] S. Y. Kung, K. S. Arun, and D. V. Bhaskar Rao. State-space and singular-value decomposition-based approximation methods
for the harmonic retrieval problem.J. Opt. Soc. Am., 73(12):1799–1811, 1983.
VI. A PPENDIX
Practical Considerations in Implementation of Sampling: time domain versus frequencydomain
17
Sampling the generated stream of Diracs is the first step in implementing existing FRIalgorithms. In practice, this step can be completed by hardware operations which are basicallyassumed to perform on the analog input signals. Given the assumption that the original Diracsignal is periodic, the filtered signal is a superposition ofshiftedsinc functions. However, sincethe sinc function has an infinite length of support in the time domain,the exact amplitude ofthe filtered signal at any time instant is determined by adding the spread ofsinc functions fromall the periods, which cannot be implemented in discrete simulation.
To solve this problem, one direct approach is the approximation by considering only a finitenumber of periods; however, this approach leads to stability problem in reconstruction. One canalleviate the instability by taking more periods into consideration with the price of increasedtime complexity of the algorithm, but perfect reconstruction is still rarely achievable. Anotherapproach is to sample in the frequency domain. Note that the frequency support of thesinckernel is finite (which is the well-known ideal rectangle low-pass filter) and consequently theFourier transform of the filtered signal is finitely supported within the bandwidth of thesinckernel. Thus, we can uniformly sample the supported frequency spectrum and then applyingdiscrete inverse Fourier transform to obtain the corresponding samples in the time domain whichare exact theoretically. In addition, this approach can be naturally extended to any bandlimitedkernel. Although some a-priori knowledge is used by the simulator in the latter approach, itdoes not change the structure of the algorithm and do help us to assess the effectiveness of thealgorithm.