a wavelet-based multilevel approach for blind deconvolution problems

Copyright © by SIAM. Unauthorized reproduction of this article is prohibited.

SIAM J. SCI. COMPUT. c© 2014 Society for Industrial and Applied MathematicsVol. 36, No. 4, pp. A1432–A1450

A WAVELET-BASED MULTILEVEL APPROACH FOR BLINDDECONVOLUTION PROBLEMS∗

MALENA I. ESPANOL† AND MISHA E. KILMER‡

Abstract. We develop a multilevel approach to solving a blind deconvolution problem, withthe ultimate intent of recovering signals which are known to have edges. First, we discuss howto generate a hierarchy of blind deconvolution problems by means of the Haar wavelet transform,and we give a modified regularized total least norm approach for solving the resulting coarse-gridproblems. Use of the Haar wavelet transform for intergrid manipulation is motivated by the factthat they can preserve desirable properties of the blurring matrix when restricted to the coarse grid,and because their orthonormality helps with the interpretability of the noise and subproblems inthe hierarchy. Recognizing that in blind deconvolution problems the blurring matrices are oftenassumed to be structured, we subsequently discuss treatment of the case when both the known linearoperator and the unknown perturbation to the operator are banded Toeplitz matrices. For this case,since the matrix structure is inherited at coarser levels, a modified regularized structured total leastnorm approach is introduced, and a quasi-Newton method is employed to solve the coarse-grid andresidual correction problems. Numerical examples show the potential of our multilevel method torecover both signals with edges and blurring operators.

Key words. blind deconvolution, multilevel, regularization, Haar wavelets, total least normproblem

AMS subject classifications. 65F10, 65F22, 65F50

DOI. 10.1137/130928716

1. Introduction. We consider the signal deblurring problem [16]

(1.1) Axtrue = btrue,

where the blurring matrix, A, and the blurred signal represented by the vector btrue

are given, and the true signal represented by the vector xtrue is sought. We areparticularly interested in recovering signals that contain “edges.” The matrix A willhave a specific structure depending on the boundary conditions imposed on the signal.When Dirichlet conditions are used, A will be Toeplitz; that is, A will be constantalong diagonals.

Unfortunately, in practice, only a noise corrupted version of btrue, namely, b =btrue+e, is available due to measurement noise and/or approximation error, and littleis known about the additive noise e. Furthermore, in many applications, A can beestimated but is not known exactly. Thus, the discrete model of the blurring to beconsidered is

(1.2) (A+ Etrue)x = b = btrue + e,

where btrue = (A+ Etrue)xtrue is unavailable and Etrue is an unknown perturbationof A. Here, A and b are assumed known. Even if A + Etrue were known, however,exact recovery of xtrue would not be possible due to the presence of the unknown

∗Submitted to the journal’s Methods and Algorithms for Scientific Computing section July 11,2013; accepted for publication (in revised form) April 18, 2014; published electronically July 8, 2014.

http://www.siam.org/journals/sisc/36-4/92871.html†Department of Mathematics, The University of Akron, Akron, OH 44325 (mespanol@uakron.

edu).‡Department of Mathematics, Tufts University, Medford, MA 02155 ([email protected]).

A1432

Dow

nloa

ded

11/1

5/14

to 1

30.2

07.5

0.37

. Red

istr

ibut

ion

subj

ect t

o SI

AM

lice

nse

or c

opyr

ight

; see

http

://w

ww

.sia

m.o

rg/jo

urna

ls/o

jsa.

php

http://www.siam.org/journals/sisc/36-4/92871.html

mailto:[email protected]




MULTILEVEL METHOD FOR BLIND DECONVOLUTION PROBLEMS A1433

additive noise e. Regularization is needed for recovery of a good approximation toxtrue, whether or not Etrue is known. The sense in which xtrue is deemed a goodapproximation, for our purposes, means that edge information is well recovered. Sincethe underlying model is a convolution, the problem of recovering a good approximationto xtrue is sometimes referred to as blind deconvolution [22, 23].

Moreover, as (1.2) represents the discretization of a Fredholm integral equationof the first kind [16], it is known that the singular values of the blurring matrix willdecay rapidly toward zero, with no break in the singular value spectrum. In thispaper, we restrict ourselves to problems satisfying the following assumptions:

1. A ∈ Rm×m with m = 2k for some integer k. This is a common assump-

tion when dealing with signals and ensures efficiency in computing wavelettransforms. However, our approach can be extended to rectangular matricesA by adjusting the dimensions of the wavelet transforms on each side of Aaccordingly.

2. ‖Etruextrue‖2 ≥ ‖e‖2. If this assumption is violated, errors due to the pres-ence of Etrue can be neglected and simpler methods should be used.

3. The elements of e are uncorrelated, distributed as N (0, s21), but the value ofs1 is unknown.

4. In this work we make no statistical assumptions on the entries of Etrue. Infact, we shall present numerical results with a perturbation matrix that isnot random which is motivated by a blind deconvolution problem where theblurring kernel is misspecified.

A well-known approach [24] to compute an approximate solution to the blinddeconvolution problem as modeled by (1.2) is to solve a regularized total least norm(RTLN) problem

(1.3) minE,x

c‖E‖2F + ‖(A+ E)x− b‖22 + λp‖Lx‖pp,

where 1 ≤ p ≤ 2 , and c and λ are scalar parameters. The regularization operator L istypically taken to be the identity or a discrete derivative operator. This approach aimsto minimize the right-hand-side error e and the perturbation matrix E simultaneously.In the special case when p = 2, this problem is called the regularized total least squares(RTLS) problem,1 and when λ = 0, this is simply the classical total least squares(TLS) problem. As we are interested in recovering edge information, we typically usep ≈ 1, as is typical in the literature when the goal is to recover edges in the contextof Tikhonov regularization.

A number of papers address the RTLS problem [2, 4, 14, 19, 24, 28], and the TLSproblem is a well-studied problem [15, 17, 20]. Some methods developed previouslyfor solving the (R)TLS and RTLN problems considered the particular cases whenthe perturbation operator has a special structure (e.g., Toeplitz structure, sparsity,displacement structure) [1, 3, 5, 10, 24, 27]. As such, they are often referred to as(regularized) structured TLS or TLN problems. Here, we present a multilevel ap-proach with the goal of pushing the computationally intensive optimization necessaryfor solving these p-norm-regularized problems to coarser grids. In [11], the authorsconsider a multilevel approach to the signal restoration problem when the blurringoperator A is known (Etrue = 0). Their method makes use of Haar wavelets to

1Regularization can be achieved by means of other methods, as in [7, 12], but since wefocus strictly on Tikhonov regularization in this paper, we use the acronym to refer to Tikhonovregularization only for simplicity.

Dow

nloa

ded

11/1

5/14

to 1

30.2

07.5

0.37

. Red

istr

ibut

ion

subj

ect t

o SI

AM

lice

nse

or c

opyr

ight

; see

http

://w

ww

.sia

m.o

rg/jo

urna

ls/o

jsa.

php


A1434 MALENA I. ESPANOL AND MISHA E. KILMER

define intergrid operators and coarse-grid problems. We take a similar approach inthe present paper, which allows us to define a set of coarse-grid modified RTLN(MRTLN) problems. If, additionally, Toeplitz structure and a certain level of band-edness on E on the fine grid are assumed, we show that the total number of unknownson each grid scales with the problem size.

Recently, there have been a number of multilevel algorithms for signal and imagedeblurring [8, 9, 18, 21, 25, 26, 29]. However, multilevel approaches for the blinddeconvolution problem seem not to have been investigated to date. To our knowledge,the only published article which is closely related is [30]. In this article, the authorsapply a multilevel approach to the unregularized TLS problem. Furthermore, whilethe authors do make use of wavelets, their approach is notably different from ours: itdoes not include a pre-smoothing step, its prolongation operator is made by paddingzeros, and it assumes all high-frequency components to be zero. Our approach is basedon the classical multigrid approach and includes an RTLN solver as a smoother. Ourpresentation also differs from theirs in that we give an analysis of the coarse-gridproblems which links them to the fine-grid problem in a natural way.

One thing that distinguishes multilevel algorithms for solving deblurring or otherinverse problems from their use in solving partial differential equations is that the goalson each level are different. Rather than smoothing noise, in deblurring we want to becareful not to mix noise into signal, and for this we need regularization and a carefulaccounting of the sources of noise from level to level, which we will show is operatorand problem dependent. But this also means choosing regularization parameters ateach level. There are many different regularization parameter selection techniques inthe literature, and we do not address this here but rather present results for a widerange of parameters. The need for more or less regularization at each level influencesthe amount of work at each level in unpredictable ways as well. Hence, in this paper,as in several of the aforementioned papers on multilevel methods for inverse problems,we do not address the concept of work-units as proof of the efficiency of a multilevelapproach but reserve this for future research.

This paper is organized as follows. In the beginning of section 2, we give mo-tivation for regularization. In preparation for a multilevel approach in which initialestimates of the error matrix and solution would be known (projected from a previousgrid level), we give motivation for a modification of the original RTLN problem toaccount for the different effects of noise on the solution. This MRTLN problem is initself a nice contribution to the literature on blind deconvolution, independent of amultilevel implementation, since one could imagine other situations in which previ-ous estimates could be used as starting guesses. In section 3, we introduce the Haarwavelet transform, and we use it to identify appropriate coarse-grid problems. Thissection also contains analysis of the properties of the resulting coarse-grid systemsthat are crucial to our multilevel method. In section 4 we consider the case thatthe blurring matrix is a banded Toeplitz matrix, and based on that assumption, wedevise a computationally efficient multilevel algorithm by taking advantage of thestructure. Numerical results are presented in section 5 that illustrate that the mul-tilevel approach proposed is a promising technique for blind deconvolution problems.Conclusions and future work are the subject of section 6.

2. The RTLN problem. Our goal is to find a regularized solution (i.e., ap-proximation to xtrue) to the system

(2.1) (A+ E)x = b,

Dow

nloa

ded

11/1

5/14

to 1

30.2

07.5

0.37

. Red

istr

ibut

ion

subj

ect t

o SI

AM

lice

nse

or c

opyr

ight

; see

http

://w

ww

.sia

m.o

rg/jo

urna

ls/o

jsa.

php



20 40 60 80 100 120

10−4

10−3

10−2

10−1

100

101

0 20 40 60 80 100 12010

−4

10−2

100

102

104

106

108

1010

1012

Fig. 1. Left: singular values of A (solid line) versus singular values of A + E (dashed line).

Right: comparison of coefficients |uTi e

σi| (dashed line), |u

Ti btrue

σi| (solid line), and |u

Ti Extrue

σi| (dotted

line).

given that E = Etrue is unknown. In fact, we would like to recover E in the process.The need for regularization is justified by the following argument. Since A is a blurringmatrix, the singular values of A decay toward 0 with no significant gap in the singularvalue spectrum. Assume that the perturbation E verifies that A + E is still ill-conditioned. Figure 1 shows the singular values of A and A + E on our running testcase (see Problem 1 in section 5) for comparison. Thus, even if E were known, solving(A+E)x = b exactly would still give a contaminated solution. Therefore, we need toincorporate a regularization term.

We will analyze the influence of E in the system and derive an important re-striction on the actual matrix perturbation E = Etrue that justifies the approach wepresent in this article. Since (A + E)xtrue = btrue for E = Etrue, we rearrange (2.1)to write

Ax = b− Ex = btrue + e− Extrue.

Viewed in this way, the right-hand side consists of the noise-free data, btrue, additivewhite noise, e, and signal-correlated noise, Extrue. Although neither e nor Extrue isknown, we wish to analyze the effect of each term on the exact solution of this system.

Suppose A is m×m invertible for simplicity. The SVD of A is A =∑m

i=1 σiuivTi .

Thus, we can write

x =

m∑i=1

(uTi b

true

σi+

uTi e

σi+

uTi Extrue

σi

)vi.

For small values of i, we expect |uTi btrue

σi| to be dominant. However, for a certain index

j and i > j, the data and noise get mixed (see Figure 1). If |uTi Extrue| < |uT

i e|, thenone might as well consider the entire term e − Extrue as unknown noise, set E = 0in the model, and try instead to find a regularized solution to Ax = b, which is acomputationally easier problem. We therefore assume E to be such that ‖Extrue‖2 >‖e‖2 and solve the RTLN problem (1.3).

We now wish to make a subtle point. In the case where A is invertible, e = 0,and λ = 0, but Etrue �= 0, the solution to (1.3) is x = A−1b and E = 0. In otherwords, it is not possible to recover xtrue and Etrue from this formulation. However,according to the discussion above, even if e = 0, since A is ill-conditioned, x = A−1b

Dow

nloa

ded

11/1

5/14

to 1

30.2

07.5

0.37

. Red

istr

ibut

ion

subj

ect t

o SI

AM

lice

nse

or c

opyr

ight

; see

http

://w

ww

.sia

m.o

rg/jo

urna

ls/o

jsa.

php



is worthless due to the signal-correlated noise term Etruextrue in the right-hand side.Indeed, it is the inclusion of the regularization term on x that, in the case that Ais invertible, enables recovery of a decent approximation of xtrue and, hopefully, ofEtrue. Moreover, in this context, the balance between the regularization on x and theconstraint ‖E‖F is especially important as determined by λ and c, respectively, lestthe former dominate the qualitative behavior of the solution. Thus, in the context ofthis paper, we regard the inclusion of c‖E‖F in the cost function as “regularization”on the unknown E, with c a regularization parameter. This differs from some of theother literature on the subject of TLN and TLS, where the presence is due to purelystatistical considerations, but is consistent with our choice of Etrue in the numericalresults.

2.1. The MRTLN problem. In this section, we assume that we have initialguesses E0 and x0. This is motivated by the fact that in our multilevel approach wewill have such information coming from another level. We wish to determine how tomodify the optimization problem to account for this additional information.

Consider (1.2), and assume that estimates E0, x0 for Etrue and xtrue are known.Next, we look for corrections Ec and xc such that E = E0 + Ec and x = x0 + xc.Substitution into the forward model gives

(A+ E0 + Ec)(x0 + xc) = b,

which is equivalent to

((A+ E0)︸︷︷︸known

+Ec)xc + Ec x0︸︷︷︸known

= b− (A+ E)x0︸︷︷︸known

= r.

Since A + E0, r, and x0 are known, we make a change of variables, A ← A + E0,x← xc, E ← Ec, to obtain what we refer to as the residual equation,

(2.2) (A+ E)x+ Ex0 = r,

in which E (formerly called Ec) and x (formerly called xc) need to be computed.This equation differs from (1.2) only by the presence of Ex0 and the fact that nowthe right-hand side is referred to as r instead of b. We note that the right-hand sidestill contains noise, hence the need for regularization; however, the regularization termin (1.3) is a constraint on the signal (enforcing a priori information about edges on thesignal estimate), and we need to maintain that throughout. Thus, in accordance withthe RTLN approach (see (1.3)) and the constraint observation, this suggests we oughtto solve the following variant of the RTLN problem, that is, the MRTLN problem:

(2.3) minE,x

{c‖E‖2F + ‖(A+ E)x+ Ex0 − r‖22 + λp‖L(x+ x0)‖pp

}with

xtrue ≈ x+ x0 and Etrue ≈ E + E0.

In section 4 we will introduce the structured version of MRTLN that is specific forthe banded Toeplitz case.

Dow

nloa

ded

11/1

5/14

to 1

30.2

07.5

0.37

. Red

istr

ibut

ion

subj

ect t

o SI

AM

lice

nse

or c

opyr

ight

; see

http

://w

ww

.sia

m.o

rg/jo

urna

ls/o

jsa.

php



3. Multilevel algorithm.

3.1. Basic multilevel framework. The idea of our multilevel method for theblind deconvolution problem is to define a sequence of system of equations of the form

(3.1) (Ai + Ei)xi + Eixi0 = ri, 0 ≤ i ≤ n,

that decrease in size. The superscript i denotes the ith level we have processed. Inparticular, i = 0 corresponds to the so-called finest level and i = n to the coarsestlevel. We call these systems (for i > 0) coarse-grid equations.

A basic 2-level approach would consist of computing a regularized solution to thefine-scale system (this is the so-called pre-smoothing step), computing the residual,solving the corresponding coarse-grid equation, plugging that estimate (via interpola-tion, if necessary) into the fine-scale equation, and correcting the result. If the coarse-grid problem itself is of the same flavor as the fine-grid problem, then this process isrepeated recursively, leading to a multilevel approach.

3.2. Haar decomposition. In this section, we identify the coarse-grid correc-tion problem and the residual correction problem through the use of the Haar wavelettransform. To derive the appropriate equations, let us transform the forward model(2.2) via the Haar wavelet transform [13] and analyze the results.

Let W denote the matrix associated with the one-dimensional Haar matrixtransform,

(3.2) WT =1√2

⎛⎜⎜⎜⎜⎜⎜⎜⎜⎜⎜⎜⎜⎝

1 1 0 0 . . . . . . 0 00 0 1 1 . . . . . . 0 0...

......

.... . .

. . ....

...0 0 0 0 . . . . . . 1 11 −1 0 0 . . . . . . 0 00 0 1 −1 . . . . . . 0 0...

......

.... . .

. . ....

...0 0 0 0 . . . . . . 1 −1

⎞⎟⎟⎟⎟⎟⎟⎟⎟⎟⎟⎟⎟⎠

=

[WT

1

WT2

].

Then, our forward model can be written in the wavelet domain as

(A+ E)x+ Ex0 = r,

where A = WTAW , E = WTEW , x = WTx, x0 = WTx0, and r = WT r. We parti-tion the transformed problem into blocks of size 2k−1 to obtain the following problem:

(3.3)

[A11 + E11 A12 + E12

A21 + E21 A22 + E22

] [x1

x2

]+

[E11 E12

E21 E22

] [x01

x02

]=

[r1r2

].

In the wavelet literature [6], the subvector x1 is referred to as the scaling coeffi-cients for x, while x2 is the vector of wavelet coefficients for x. Similarly, x01 containsthe scaling coefficients for x0 and x02 the wavelet coefficients for x02.

3.3. Coarse-grid problem. We begin by using the transformed equation toformulate the appropriate coarse-grid problem. From the first block equation of (3.3),we have

(3.4) (A11 + E11)x1 + E11x01 = r1 − (A12 + E12)x2 − E12x02.

We now wish to argue that (3.4) determines the coarse-grid problem. The followingobservations help establish this fact:

Dow

nloa

ded

11/1

5/14

to 1

30.2

07.5

0.37

. Red

istr

ibut

ion

subj

ect t

o SI

AM

lice

nse

or c

opyr

ight

; see

http

://w

ww

.sia

m.o

rg/jo

urna

ls/o

jsa.

php



10 20 30 40 50 6010

−6

10−5

10−4

10−3

10−2

10−1

100

101

0 10 20 30 40 50 6010

−5

10−4

10−3

10−2

10−1

100

101

102

103

Fig. 2. Left: singular values of A11 (solid line) versus singular values of A11 + E11 (dashed

line). Right: comparison of coefficients |uTi E11x1/σi| (dashed line), |uT

i rtrue1 /σi| (solid line), and

|uTi ((A12 + E12)x2 − E12x02)/σi| (dotted line).

• x1 is a representation of x on a coarse grid,• A11 + E11 is still ill-conditioned (see [11]), and• the term (A12+E12)x2−E12x02 can be treated as noise (see discussion below).

The first two remarks are related to important properties of the fine-grid problemthat are carried over to the coarse-grid problem. Our last bullet point is relatedto the fact that we know neither x2 nor E12 and therefore we cannot compute thecorresponding terms on the right of (3.4). On the other hand, if those terms arenot dominant, we might be able to mitigate the effects by solving for approximatesolutions using the same MRTLN approach we are using on the fine grid. To do so,we need to argue that those terms can be treated as noise for which the regularizationis effective.

To make this argument, we use the SVD of A11 and compare the spectral coor-dinates of (A12 + E12)x2 − E12x02 with those of rtrue1 and E11x1. Figure 2 showsthat, for our running example, the spectral coordinates of (A12+ E12)x2− E12x02 aresmaller than those of E11x1 in the subspace related to smooth modes.

The preceding argument suggests that

(3.5) (A11 + E11)x1 + E11x01 = r1

is a coarse-scale version of the same forward model (2.2) we are using on the fine scale.Therefore, a regularized solution of it can be obtained by solving

(3.6) minx1,E11

{c‖E11‖2F + ‖(A11 + E11)x1 + E11x01 − r1‖22 + λp‖L(x1 + x01)‖pp

},

where L is a first-derivative operator, and 1 < p ≤ 2. We call this problem the coarse-grid correction problem. Furthermore, since (3.5) looks exactly like (2.2) we can applythis coarsening process again to (3.5). By doing so several times we create a sequenceof coarse-grid problems. When considering more than two levels we have more thanone coarse-grid problem. Then, the pair solution of the coarse-grid correction problemcorresponding to i �= n will be denoted (xpre, Epre) (pre-smoothing).

3.4. Residual correction. We also need to recover the wavelet coefficients x2

as well as the other three blocks of E: E12, E21, and E22. Assuming that we haveapproximations x∗

1 of x1 and E∗11 of E11 from the coarse-grid correction, then (3.3)

can be rewritten as

Dow

nloa

ded

11/1

5/14

to 1

30.2

07.5

0.37

. Red

istr

ibut

ion

subj

ect t

o SI

AM

lice

nse

or c

opyr

ight

; see

http

://w

ww

.sia

m.o

rg/jo

urna

ls/o

jsa.

php



(3.7)

[A12 + E12

A22 + E22

]x2 +

[E12

E22

]x02 +

[0

E21

](x01 + x∗

1) = rnew

with

rnew =

[r1r2

]−[A11 + E∗

11

A21

]x∗1 −

[E∗

11

0

]x01.

To solve for the unknowns, we will again formulate an MRTLN problem:

minx2,E12,E21,E22

c‖E12‖2F + c‖E21‖2F + c‖E22‖2F(3.8)

+

∥∥∥∥[A12 + E12

A22 + E22

]x2 +

[E12

E22

]x02 +

[0

E21

](x01 + x∗

1)− rnew

∥∥∥∥2

2

+ λp

∥∥∥∥L(x0 +W

[x1

x2

])∥∥∥∥pp

,

where L is a first-derivative operator, and 1 < p ≤ 2 as in (3.6). This problem will bereferred to as the residual correction problem. Note that unlike traditional multigrid,this method does not use interpolation to the fine grid to define the correction. Indeed,it is very important to note that x2 lives on a coarse grid, but the regularization isapplied at a “finer” scale than the unknown vector x2 for which we are optimizing.

Now that we have defined both the coarse-grid and the residual correction prob-lems, we are in a position to summarize our multilevel algorithm.

3.5. The general multilevel method. Our multilevel algorithm is as follows.Algorithm 1. Multilevel V-cycle: MGM.

Input: Ai, ri, xi0

Output: xi, Ei

1. If i = n2. [xi, Ei] = Solve (3.6)3. Else4. If i �= 05. [xi

pre, Eipre] = Solve (3.6)

6. Else7. xi

pre = 0, Eipre = 0

8. End If9. ri = ri − (Ai + Ei

pre)xipre − Ei

prexi0

10. ri+1 = WT1 ri

11. Ai = Ai + Eipre

12. Ai+1 = WT1 AiW1

13. xi+10 = WT

1 (xi0 + xi

pre)

14. [xi+11 , Ei+1

11 ] = MGM(Ai+1, ri+1, xi+10 )

15. rinew = WT ri −WTAiW1xi+11 −

[Ei+1

11

0

](xi+1

1 + xi+10 )

16. [xi+12 , Ei+1

12 , Ei+121 , Ei+1

22 ] = Solve (3.8)

17. Eic = W

[Ei+1

11 Ei+112

Ei+121 Ei+1

22

]WT

18. xi = xipre +W1x

i+11 +W2x

i+12

19. Ei = Eipre + Ei

c

20. End If

Dow

nloa

ded

11/1

5/14

to 1

30.2

07.5

0.37

. Red

istr

ibut

ion

subj

ect t

o SI

AM

lice

nse

or c

opyr

ight

; see

http

://w

ww

.sia

m.o

rg/jo

urna

ls/o

jsa.

php



Recall that the superscript i is used to indicate the current level. The coarsestlevel is denoted by i = n, and the finest level corresponds to i = 0.

The next section presents details that make the algorithm even more efficient bymeans of exploiting matrix structure.

4. The banded Toeplitz case. The nonconvex nonlinear optimization prob-lem (1.3) is computationally very difficult since we are minimizing with respect to Eand x, that is, we have a total of m2 +m unknowns, and only m data components.When Dirichlet conditions are imposed to the signal deblurring problem, the matrixA is Toeplitz, and then we can assume that E is Toeplitz as well. Many articles havebeen written on solving (1.3) for p = 2 subject to the constraint that E is Toeplitz orotherwise structured [24, 27]. Consider the following Toeplitz matrix:

E =

⎡⎢⎢⎢⎢⎢⎣

t0 t−1 t−2 . . . t−(m−1)

t1 t0 t−1 . . . t−(m−2)

t2 t1 t0 . . . t−(m−3)

......

.... . .

...tm−1 tm−2 tm−3 . . . t0

⎤⎥⎥⎥⎥⎥⎦ .

The vector t = (t−(m−1), . . . , t−1, t0, t1, . . . , tm−1) is called the Toeplitz vector thatgenerates E. Thus, the minimization problem (1.3) can be written as

(4.1) mint,x

{c‖E(t)‖2F + ‖(A+ E(t))x − b‖22 + λp‖Lx‖pp

}.

This problem is the so-called regularized, structured total least norm (RSTLN) prob-lem. In constraining E this way, the number of unknowns is reduced to 3m− 1.

If we have initial guesses E0 and x0, we can solve themodified RSTLN (MRSTLN)problem,

(4.2) mint,x

{c‖E(t)‖2F + ‖(A+ E(t))x+ E(t)x0 − r‖22 + λp‖L(x+ x0)‖pp

},

where r = b − (A + E0)x0 and A = A + E0, and then the solutions are x = x0 + xand E = E + E0. We shall solve MRSTLN problems at each level in the multilevelalgorithm. Therefore, we need to study the structure of the matrices at coarser levels.

As shown in [11], when we use Haar wavelets to derive the coarse-scale and cor-rection equations, the Toeplitz structure of the coarse-scale matrices, Aij (likewise

Eij) for i, j = 1, 2, is inherited from the Toeplitz structure of the fine-grid matrix, A(E). Since we want to construct our algorithms for solving the optimization problemsto exploit this fact, we need to summarize the substance of the results in [11] as theyapply to E.

Corollary 4.1. Let E be an m×m matrix with Toeplitz structure, and m = 2k.Then, the 2k−1 × 2k−1 submatrices E11, E12, E21, and E22 defined in (3.3) also haveToeplitz structure. In particular, if E has upper bandwidth ku ≤ m/2 and lowerbandwidth kl ≤ m/2, Eij are m/2 × m/2 banded Toeplitz matrices with bandwidthsku/2� and kl/2�. Moreover, their corresponding Toeplitz vectors eij have lengthku/2�+ kl/2�+ 1 and satisfy

e12 = −e21 and e22 = e11 + T e12,

where T =

( 2 4 ... 40 2 ... 4.........

...0 0 ... 2

).

Dow

nloa

ded

11/1

5/14

to 1

30.2

07.5

0.37

. Red

istr

ibut

ion

subj

ect t

o SI

AM

lice

nse

or c

opyr

ight

; see

http

://w

ww

.sia

m.o

rg/jo

urna

ls/o

jsa.

php



From now on, we assume E to be a banded Toeplitz matrix. In this case, Corol-lary 4.1 allows us to reduce the total number of unknowns for both the coarse-grid andresidual correction problems. In other words, the number of unknowns in the opti-mization problem scales appropriately from one grid to the next. Therefore, problems(3.6) and (3.8) can both be written as modified RSTLN equivalents, as we now discuss.

4.1. Coarse-grid problem revisited. Instead of solving problem (3.6), wesolve its structured version, the MRSTLN problem, which given the notation in theprevious section takes the form

(4.3) minx1,e11

{c‖De11‖22 + ‖A11x1 + E11(x1 + x01)− r1‖22 + λp‖L(x1 + x01)‖pp

},

or its equivalent formulation

(4.4) minx1,e11

{c‖De11‖22 + ‖A11x1 + F e11 − r1‖22 + λp‖L(x1 + x01)‖pp

},

where the vector e11 is the Toeplitz vector corresponding to E11, D is a diagonalweighting matrix [27], and F is a matrix such that F e11 = E11(x1 + x01). Note thatF is also a banded Toeplitz matrix, but not necessarily square. To see this, considerthe following small example:⎡

⎢⎢⎣x2 x1 0 0x3 x2 x1 0x4 x3 x2 x1

0 x4 x3 x2

⎤⎥⎥⎦⎡⎢⎢⎣t−1

t0t1t2

⎤⎥⎥⎦ =

⎡⎢⎢⎣t0 t−1 0 0t1 t0 t−1 0t2 t1 t0 t−1

0 t2 t1 t0

⎤⎥⎥⎦⎡⎢⎢⎣x1

x2

x3

x4

⎤⎥⎥⎦ .

Then, in the general case where we want to define F such that F e = Ex with Ebeing a banded Toeplitz matrix with upper bandwidth ku, and lower bandwidth kl,the Toeplitz vector that generates F is given by f = (0kl

, x, 0ku), where 0s denotes azero vector of size s.

In order to solve (4.4), we apply the following modified version of the RSTLNalgorithm introduced in [24].

Algorithm 2. Coarse-grid solver.Input: A11, r1, x01

Output: x1, E11

1. Set E11 = 0 and e11 = 0.

2. Compute x1 by minimizing ‖A11x1 − r1‖2.3. Construct F from x1 + x01.

4. Compute r = r1 − A11x1 and Dp = p(p− 1)diag(|λL(x1 + x01)|p−2).5. For k = 1, 2, . . . until |Δx1|, |Δe11| ≤ ε repeat

5.1. Solve

minΔe11Δx1

∥∥∥∥∥[ √

2F√2(A11+E11)√

2cD 0

0 λD1/2p L

] [Δe11Δx1

]−[ √

2r

−√2cDe11

− λp−1D

1/2p L(x1+x01)

]∥∥∥∥∥2

2

.

5.2 Set x1 = x1 +Δx1 and e11 = e11 +Δe11.

5.3 Construct E11 from e11, and F from x1 + x01.

5.4 Compute r = r1 − (A11 + E11)x1 − E11x01, andDp = p(p− 1)diag(|λL(x1 + x01)|p−2).

Dow

nloa

ded

11/1

5/14

to 1

30.2

07.5

0.37

. Red

istr

ibut

ion

subj

ect t

o SI

AM

lice

nse

or c

opyr

ight

; see

http

://w

ww

.sia

m.o

rg/jo

urna

ls/o

jsa.

php



4.2. Residual correction revisited. Next we want to derive the structure-constrained version of (3.8). From previous discussion, we know that e21 = −e22.Thus, we want to solve

minx2,e12,e22

2c‖De12‖22 + c‖De22‖22(4.5)

+

∥∥∥∥[A12

A22

]x2 +

[E12

E22

](x2 + x02) +

[0

−E12

](x∗

1 + x01)− rnew

∥∥∥∥2

2

+

∥∥∥∥λL(x0 +WT

[x1

x2

])∥∥∥∥pp

,

or equivalently

minx2,e12,e22

2c‖De12‖22 + c‖De22‖22 +∥∥∥∥[A12

A22

]x2 +

[Fa 0Fb Fc

] [e12e22

]− rnew

∥∥∥∥2

2

(4.6)

+ λp

∥∥∥∥L(x0 +WT

[x1

x2

])∥∥∥∥pp

,

where Fae12 = E12(x2 + x02) and Fbe12 = E12(−x∗1 − x01). Note that Fc is equal to

Fa since Fae22 = E12(x2 + x02). This is because in order to generate Fc we only needthe structure of E22 (the structure is equivalent to that of E12), and its entries whichare x2 + x02.

We also know that e22 = e11 + T e12. Thus, we can rewrite the minimizationproblem (4.6) as

minx2,e12

2c‖De12‖22 + c‖D(e11 + T e12)‖22 +∥∥∥∥[A12

A22

]x2 +

[Fa 0Fb Fa

] [e12

e11 + T e12

]− rnew

∥∥∥∥2

2

(4.7)

+ λp

∥∥∥∥L(x0 +WT

[x1

x2

])∥∥∥∥pp

.

By algebraic manipulation, we have that[Fa 0Fb Fa

] [e12

e11 + T e12

]=

[Fa

Fb

]e12 +

[0Fa

](e11 + T e12) =

[Fa

Fb + FaT

]e12 +

[0Fa

]e11.

We can use this result to rewrite problem (4.7) as

minx2,e12

2c‖De12‖22 + c‖D(e11 + T e12)‖22

(4.8)

+

∥∥∥∥[A12

A22

]x2 +

[Fa

Fb + FaT

]e12 +

[0Fa

]e11 − rnew

∥∥∥∥2

2

+ λp

∥∥∥∥L(x0 +WT

[x1

x2

])∥∥∥∥pp

,

or, for simplicity in future reference,

(4.9) minx2,e12

{2c‖r1‖22 + c‖r2‖22 + c‖r3‖22 + ‖r4‖pp

}.

Adapting Algorithm 2 to solve (4.8), we get the following algorithm.

Dow

nloa

ded

11/1

5/14

to 1

30.2

07.5

0.37

. Red

istr

ibut

ion

subj

ect t

o SI

AM

lice

nse

or c

opyr

ight

; see

http

://w

ww

.sia

m.o

rg/jo

urna

ls/o

jsa.

php



Algorithm 3. Residual correction solver.Input: A12, A22, rnew , x1, x0

Output: x2, E12

1. Set E12 = 0, E22 = 0 and e12 = 0.

2. Compute x2 by minimizing

∥∥∥∥[A12

A22

]x2 − rnew

∥∥∥∥2

.

3. Construct Fa and Fb.4. Compute r1, r2, r3, r4 of (4.9) and Dp = p(p− 1)diag(|r4|q−2).5. For k = 1, 2, . . . until |Δx2|, |Δe12| ≤ ε repeat

5.1. Solve

minΔx2,Δe12

∥∥∥∥∥∥∥∥∥∥

⎡⎢⎢⎢⎢⎣√2

[Fa

Fb + FaT

] √2

[A12 + E12

A22 + E22

]2√cD 0√

2cDT 0

0 λD1/2p LW2

⎤⎥⎥⎥⎥⎦[Δe12Δx2

]+

⎡⎢⎢⎣√2r3

2√cr1√2cr2

λp−1D

1/2p r4

⎤⎥⎥⎦∥∥∥∥∥∥∥∥∥∥

2

2

.

5.2 Set x2 = x2 +Δx2 and e12 = e12 +Δe12.

5.3 Construct E12, E22, Fa, and Fb.5.4 Compute r1, r2, r3, r4 of (4.9) and Dp = p(p− 1)diag(|r4|p−2).

These algorithms are used at every level of the multilevel algorithm. Consequently,the percentage of unknowns relative to the problem dimension for the residual cor-rection problem (and coarse-grid problems) remains the same as for the fine-gridproblem.

4.3. Reconstructing E. Once we have the four matrices Eij , we want to getthe m×m Toeplitz matrix E. Using MATLAB notation, define the following vectors:

c1 = 0.5(E11(:,m/2)− E12(:,m/2)),

c2 = 0.5(E11(:,m/2) + E12(:,m/2)),

c3 = 0.5(E11(2 : m/2, 1)− E12(2 : m/2, 1)),

c4 = 0.5(E11(2 : m/2, 1) + E12(2 : m/2, 1)).

Then, define

y(1 : 2 : m) = c1,y(2 : 2 : m) = c2,y(m+ 1 : 2 : 2m− 2) = c3,y(m+ 2 : 2 : 2m− 2) = c4,

y(2m− 1) = (E11(1,m/2) + 2E12(1,m/2)− E22(1,m/2))/4.

From here, we have that the Toeplitz vector of E is t = 2(I +Zm)−1y, where Zm

is the downshift matrix (i.e., a Toeplitz matrix where its Toeplitz vector t is a vectorof all zeros except t−1 = 1). Since the matrix I+Zm is lower bidiagonal, this is easilycomputed.

Most importantly, in the banded case, the last relation to reconstruct E,

y(2m− 1) =1

4(E11(1,m/2)− 2E21(1,m/2)− E22(1,m/2)),

is equal to zero since the entries of the block matrices needed are all zeros. Thus,our estimate of E can be recovered exclusively from the two vectors e11, e12 that were

Dow

nloa

ded

11/1

5/14

to 1

30.2

07.5

0.37

. Red

istr

ibut

ion

subj

ect t

o SI

AM

lice

nse

or c

opyr

ight

; see

http

://w

ww

.sia

m.o

rg/jo

urna

ls/o

jsa.

php



obtained from the coarse-grid and residual correction solves, respectively. Each ofthese two vectors is half the length of the Toeplitz vector for E.

5. Numerical results. In this section we present some results we have obtainedusing our multilevel approach and other standard methods for comparison. All compu-tations were performed in MATLAB. In the following examples, our goal is to recoverthe original signal from a blurred, noisy signal as well as the blurring operator.

Problem 1. The blurring operator is defined by an m×m nonsymmetric bandedToeplitz matrix A, where its first row and first column are defined by the vectors

zu =1

2πσ2[exp(−([0 : ku − 1].ˆ2)/(2σ2)); zeros(1,m− ku)]

and

zl =1

2πσ2[exp(−([0 : kl − 1].ˆ2)/(2σ2)); zeros(1,m− kl)],

where ku and kl specify the upper and lower bandwidths of the matrix A, and theparameter σ corresponds to the variance of the Gaussian function

h(x) =1

2πσ2exp

(− x2

2σ2

).

We use m = 128, and the parameters σ = 3, ku = 19, and kl = 20. The matrix A isnormalized so that ‖A‖2 = 1. The condition number of A is κ(A) = 2.8× 1013.

The perturbation matrix Etrue is defined by the difference between A and the trueblurring operator Atrue which is a 128 × 128 nonsymmetric banded Toeplitz matrixwith parameters σ = 2, ku = 14, and kl = 13. Thus, we have Atrue = A + Etrue,and therefore the perturbation matrix Etrue is also a nonsymmetric banded Toeplitzmatrix with the same upper and lower bandwidths as A.

The exact solution xtrue is the vector of length 128 shown in Figure 3. The noise-free blurred signal btrue is computed as btrue = (A + Etrue)xtrue. Thus, we have

a signal-correlated noise measured by ‖Etruextrue‖2

‖btrue‖2= 0.12 and we can say that the

signal-correlated noise level is 12%. The elements of the noise vector e are normally

distributed with zero mean and standard deviation chosen such that ‖e‖2

‖btrue‖2= 0.01.

Then, we say that the level of noise of the right-hand side is 1%. The noisy right-handside of our system is defined by b = btrue + e (see Figure 3).

20 40 60 80 100 120

−4

−3

−2

−1

0

1

2

3

4

20 40 60 80 100 120

−4

−3

−2

−1

0

1

2

3

4

Fig. 3. Problem 1. Left: true solution xtrue. Right: blurred, noisy right-hand side b.

Dow

nloa

ded

11/1

5/14

to 1

30.2

07.5

0.37

. Red

istr

ibut

ion

subj

ect t

o SI

AM

lice

nse

or c

opyr

ight

; see

http

://w

ww

.sia

m.o

rg/jo

urna

ls/o

jsa.

php



To evaluate different algorithms, we take the approximate signal x and the ap-proximate E and compute relative errors

err(x) =‖x− xtrue‖1‖xtrue‖1 and err(E) =

‖E − Etrue‖F‖Etrue‖F .

In all regularization methods, we use p = 1.5 and the regularization operator

L =

⎛⎝−1 1

−1 1

. . .. . .−1 1

⎞⎠ ,

which is a scaled discrete approximation to the first derivative operator, with noassumptions on boundary conditions. We define 10 logarithmically spaced values ofλ from 10−3 to 10. We loop for all values of λ and define the optimal value of λ, λopt,as the one that makes err(x) minimum.

For comparison, we compute Tikhonov solution xTik, where we ignored E bysolving

(5.1) minx‖Ax− b‖22 + λp‖Lx‖pp,

and the approximate solution xEtrue

obtained by solving

(5.2) minx‖(A+ Etrue)x− b‖22 + λp‖Lx‖pp,

where we assume complete knowledge of E. We also compute the solutions to theMRSTLN problem

(5.3) mint,x

{c‖E(t)‖2F + ‖(A+ E(t))x+ E(t)x0 − r‖22 + λp‖L(x+ x0)‖pp

},

for c = 1 and c = 0.1, and initial guesses x0 = xTik and E0 = 0. Table 1 showsoptimal parameters and relative errors, and Figure 4 shows the reconstructed signals.By looking at the solution xEtrue

(Figure 4), we can see that knowing E exactly makesa huge impact in recovering edges. Moreover, since the signal-correlated noise levelin this case is higher than the level of noise on the right-hand side, the MRSTLNsolution is better than the Tikhonov solution, as expected.

We test the method for different number of levels, that is, we compute the solutionof a 1-level, 2-level, and 3-level MGMs. In Table 2, we report the relative errors ofx and E, the number of iterations of solvers applied at each level, and the optimalvalues of λ for each level. To obtain the 1-level (finest level) solution, 13 iterations ofAlgorithm 2 are needed to converge. The 2-level solution is obtained by 15 iterationsof Algorithm 2 and 3 iterations of Algorithm 3. Recall that in the 2-level approachboth the coarse-grid and the residual correction problems are half the size of the finest

Table 1

Problem 1. Relative errors corresponding to the solutions in Figure 4. The notation λ(i) meansthat from the 10 values of λ that were used, the ith one made err(x) minimum.

xTik xEtrueMRSTLN (c = 1) MRSTLN (c = 0.1)

err(x) 0.2101 0.1100 0.1746 0.1510err(E) 1 0 0.6671 0.4407λopt λ(8) λ(4) λ(6) λ(5)

Dow

nloa

ded

11/1

5/14

to 1

30.2

07.5

0.37

. Red

istr

ibut

ion

subj

ect t

o SI

AM

lice

nse

or c

opyr

ight

; see

http

://w

ww

.sia

m.o

rg/jo

urna

ls/o

jsa.

php



20 40 60 80 100 120

−4

−3

−2

−1

0

1

2

3

4

20 40 60 80 100 120

−4

−3

−2

−1

0

1

2

3

4

20 40 60 80 100 120

−4

−3

−2

−1

0

1

2

3

4

20 40 60 80 100 120

−4

−3

−2

−1

0

1

2

3

4

Fig. 4. Problem 1. Top left: xTik obtained for “optimal” lambda but taking E = 0. Top right:

xEtrueobtained by solving (5.2) assuming Etrue is known. Bottom left: MRSTLN (5.3) for optimal

lambda with c = 1. Bottom right: solution to MRSTLN (5.3) for optimal lambda but c = 0.1

Table 2

Problem 1. Relative errors obtained by applying different number of levels. They correspondto the solutions shown in Figure 5. The notation λ(i, j, k, l) means that λ(i), λ(j), λ(k), and λ(l)combined made err(x) minimum.

1 level 2 levels 3 levelserr(x) 0.1746 0.1493 0.1735err(E) 0.6671 0.6959 0.6502iter 13 15,2 15,10,2,2λopt λ(6) λ(6, 8) λ(6, 5, 8, 5)

problem. For the 3-level solution, we need basically the same work as the 2-level inaddition to the work in the extra level (10 iterations of Algorithm 2 and 2 iterations ofAlgorithm 3). The notation λ(i, j, k, l) means, for the 3-level case, that for Algorithm2 in level 2 we need λ(i), for Algorithm 2 for level 3 we need λ(j), for Algorithm 3in level 2 we need λ(k), and for Algorithm 3 in level 1, we need λ(l). We loop forall values of λ at each level and choose the combination of λ values that make therelative error minimum. We kept the value of c = 1 for all levels.

Problem 2. The following example is set up similarly to Problem 1 and, unlessnoted otherwise, definitions, methods, and parameters used are the same. In thisexample we consider the blurring operator to be a 512×512 symmetric banded Toeplitzmatrix A with parameters σ = 5, ku = kl = 16. In this case, A is not normalized.Its condition number is κ(A) = 7.8 × 1013. The true blurring operator Atrue isalso a 512 × 512 symmetric banded Toeplitz matrix, but with parameters σ = 2,ku = kl = 20. The exact solution xtrue and the noise-free blurred signal btrue are

Dow

nloa

ded

11/1

5/14

to 1

30.2

07.5

0.37

. Red

istr

ibut

ion

subj

ect t

o SI

AM

lice

nse

or c

opyr

ight

; see

http

://w

ww

.sia

m.o

rg/jo

urna

ls/o

jsa.

php



20 40 60 80 100 120

−4

−3

−2

−1

0

1

2

3

4

0 20 40 60 80 100 120−5

−4

−3

−2

−1

0

1

2

3

4

5

0 20 40 60 80 100 120−5

−4

−3

−2

−1

0

1

2

3

4

5

Fig. 5. Problem 1. Solutions obtained by multilevel method with different levels: 1 level (left),2 levels (middle), and 3 levels (right).

0 100 200 300 400 500−1.5

−1

−0.5

0

0.5

1

1.5

2

2.5

0 100 200 300 400 500−1.5

−1

−0.5

0

0.5

1

1.5

2

2.5

Fig. 6. Problem 2. Left: true solution xtrue. Right: blurred, noisy right-hand side b (right).

shown in Figure 6. We have a signal-correlated noise level of 16% and a level of noisefor the right-hand side of 1%. In this case, we use the p-norm with p = 1.1 and allmethods have initial guess x0 = 0. For the multilevel methods, we consider c = 0.1 forall the corresponding minimization problems. Although this example is symmetric,our algorithm does not consider its symmetry. A symmetric version of this algorithmcould be easily developed which will improve accuracy and reduce the computationcost further. For comparison Tables 3 and 4 show relative errors, optimal parameters,and number of iterations, and Figures 7 and 8 show the reconstructed signals.

Table 3

Problem 2. Relative errors corresponding to the solutions in Figure 7. The notation λ(i) meansthat from the 10 values of λ that were used, the ith one made err(x) minimum.

xTik xEtrueMRSTLN (c = 1) MRSTLN (c = 0.1)

err(x) 0.1191 0.0281 0.0802 0.0542err(E) 1 0 0.8710 0.3686λopt λ(6) λ(3) λ(6) λ(5)

Table 4

Problem 2. Relative errors obtained by applying different number of levels. They correspondto the solutions shown in Figure 8. The notation λ(i, j, k, l) means that λ(i), λ(j), λ(k), and λ(l)combined made err(x) minimum.

1 level 2 levels 3 levelserr(x) 0.0542 0.0652 0.0262err(E) 0.3686 0.3871 0.2230iter 20 19,2 19,43,1,2λopt λ(6) λ(4, 3) λ(4, 3, 7, 6)

Dow

nloa

ded

11/1

5/14

to 1

30.2

07.5

0.37

. Red

istr

ibut

ion

subj

ect t

o SI

AM

lice

nse

or c

opyr

ight

; see

http

://w

ww

.sia

m.o

rg/jo

urna

ls/o

jsa.

php



0 100 200 300 400 500−1

−0.5

0

0.5

1

1.5

0 100 200 300 400 500−1

−0.5

0

0.5

1

1.5

0 100 200 300 400 500−1

−0.5

0

0.5

1

1.5

0 100 200 300 400 500−1

−0.5

0

0.5

1

1.5

Fig. 7. Problem 2. Top left: xTik obtained for optimal lambda but taking E = 0. Top right:

xEtrueobtained by solving (5.2) assuming Etrue is known. Bottom left: MRSTLN (5.3) for optimal

lambda with c = 1. Bottom right: solution to MRSTLN (5.3) for optimal lambda but c = 0.1

0 100 200 300 400 500−1

−0.5

0

0.5

1

1.5

0 100 200 300 400 500−1

−0.5

0

0.5

1

1.5

0 100 200 300 400 500−1

−0.5

0

0.5

1

1.5

Fig. 8. Problem 2. Optimal solutions with respect to err(x) obtained by multilevel method withdifferent levels: 1 level (left), 2 levels (middle), and 3 levels (right).

6. Conclusions. We developed an edge-preserving wavelet-based multilevelmethod for a blind deconvolution problem. We began with an analysis of the for-ward model and noise contributions that illustrated when inclusion of the operatorerror term, E, was warranted in the context of deblurring. Subsequent analysis of themodel after transformation to the wavelet domain using Haar wavelets led us to for-mulate a multilevel algorithm consisting of two modified RTLN (MRTLN) problemson each level: the coarse-grid and the residual correction problems. The formulationof this MRTLN problem and in particular the inclusion of the regularization terms areexplained in terms of the different errors inside the system. For the case when boththe known linear operator and the unknown perturbation to the operator are bandedToeplitz matrices, we gave quasi-Newton algorithms to solve the MSRTLN problems

Dow

nloa

ded

11/1

5/14

to 1

30.2

07.5

0.37

. Red

istr

ibut

ion

subj

ect t

o SI

AM

lice

nse

or c

opyr

ight

; see

http

://w

ww

.sia

m.o

rg/jo

urna

ls/o

jsa.

php



on each grid. Numerical experiments illustrate that this is a promising technique forblind deconvolution problems worthy of future study. Of particular interest is thestudy of appropriate parameter selection techniques for each level, as well as a studyof how the need for possibly varying regularization at each level can be used to definean appropriate metric of work for a V-cycle. Additional future work includes theextension of the multilevel approach to two-dimensional image blind deconvolution,and a statistical study of MRSTLN, under further assumptions on E, as a maximumlikelihood estimator to deepen the understanding of its properties.

Acknowledgments. The authors thank Dianne O’Leary for helpful comments.

REFERENCES

[1] A. Beck and A. Ben-Tal, A global solution for the structured total least squares problem withblock circulant matrices, SIAM J. Matrix Anal. Appl., 27 (2005), pp. 238–255.

[2] A. Beck and A. Ben-Tal, On the solution of the Tikhonov regularization of the total leastsquares problem, SIAM J. Optim., 17 (2006), pp. 98–118.

[3] A. Beck, A. Ben-Tal, and C. Kanzow, A fast method for finding the global solution of theregularized structured total least squares problem for image deblurring, SIAM J. MatrixAnal. Appl., 30 (2008), pp. 419–443.

[4] A. Beck, A. Ben-Tal, and M. Teboulle, Finding a global optimal solution for a quadraticallyconstrained fractional quadratic problem with applications to the regularized total leastsquares, SIAM J. Matrix Anal. Appl., 28 (2006), pp. 425–445.

[5] A. Beck and Y. C. Eldar, Structured total maximum likelihood: An alternative to structuredtotal least squares, SIAM J. Matrix Anal. Appl., 31 (2010), pp. 2623–2649.

[6] A. Boggess and F. J. Narcowich, First Course in Wavelets with Fourier Analysis, PrenticeHall, Englewood Cliffs, NJ, 2001.

[7] T. F. Chan and C. K. Wong, Total variation blind deconvolution, IEEE Trans. Image Process.,7 (1998), pp. 370–375.

[8] M. Donatelli, A multigrid for image deblurring with Tikhonov regularization, Numer. LinearAlgebra Appl., 12 (2005), pp. 715–729.

[9] M. Donatelli, An iterative multigrid regularization method for Toeplitz discrete ill-posed prob-lems, Numer. Math. Theory Methods Appl., 5 (2012), pp. 43–61.

[10] M. Donatelli and N. Mastronardi, Fast deconvolution with approximated PSF by RSTLSwith antireflective boundary conditions, J. Comput. Appl. Math., 236 (2012), pp. 3992–4005.

[11] M. I. Espanol and M. E. Kilmer, Multilevel approach for signal restoration problems withToeplitz matrices, SIAM J. Sci. Comput., 32 (2010), pp. 299–319.

[12] R. D. Fierro, G. H. Golub, P. C. Hansen, and D. P. O’Leary, Regularization by truncatedtotal least squares, SIAM J. Sci. Comput., 18 (1997), pp. 1223–1241.

[13] P. J. Van Fleet, Discrete Wavelet Transformation: An Elementary Approach with Applica-tions, Wiley, New York, 2008.

[14] G. H. Golub, P. C. Hansen, and D. P. O’Leary, Tikhonov regularization and total leastsquares, SIAM J. Matrix Anal. Appl., 21 (1999), pp. 185–194.

[15] G. H. Golub and C. F. Van Loan, An analysis of the total least squares problem, SIAM J.Numer. Anal., 17 (1980), pp. 883–893.

[16] P. C. Hansen, Rank-Deficient and Discrete Ill-Posed Problems, SIAM, Philadelphia, 1998.[17] S. Van Huffel and J. Vandewalle, The Total Least Squares Problem: Computational Aspects

and Analysis, SIAM, Philadelphia, 1991.[18] E. Klann, R. Ramlau, and L. Reichel, Wavelet-based multilevel methods for linear ill-posed

problems, BIT, 51 (2011), pp. 669–694.[19] G. Lee, H. Fu, and J. L. Barlow, Fast high-resolution image reconstruction using Tikhonov

regularization based total least squares, SIAM J. Sci. Comput., 35 (2013), pp. B275–B290.[20] I. Markovsky and S. Van Huffel, Overview of total least squares methods, Signal Proc., 391

(2004), pp. 75–98.[21] S. Morigi, L. Reichel, F. Sgallari, and A. Shyshkov, Cascadic Multiresolution Methods

for Image Deblurring, SIAM J. Imaging Sci., 1 (2008), pp. 51–74.[22] M. Ng and B. Plemmons, Blind deconvolution and structured matrix computations with

applications to array imaging, in Blind Deconvolution: Theory and Applications, CRCPress, Boca Raton, FL, 2007, pp. 377–418.

Dow

nloa

ded

11/1

5/14

to 1

30.2

07.5

0.37

. Red

istr

ibut

ion

subj

ect t

o SI

AM

lice

nse

or c

opyr

ight

; see

http

://w

ww

.sia

m.o

rg/jo

urna

ls/o

jsa.

php



[23] D. P. O’Leary, Scientific Computing With Case Studies, SIAM, Philadelphia, 2009.[24] A. Pruessner and D. P. O’Leary, Blind deconvolution using a regularized structured total

least norm algorithm, SIAM J. Matrix Anal. Appl., 24 (2003), pp. 1018–1037.[25] L. Reichel and A. Shyshkov, Cascadic multilevel methods for ill-posed problems, J. Comput.

Appl. Math., 233 (2010), pp. 1314–1325.[26] A. Rieder, A wavelet multilevel method for ill-posed problems stabilized by Tikhonov regular-

ization, Numer. Math., 75 (1997), pp. 501–522.[27] J. B. Rosen, H. Park, and J. Glick, Total least norm formulation and solution for structured

problems, SIAM J. Matrix Anal. Appl., 17 (1996), pp. 110–126.[28] D. M. Sima, S. Van Huffel, and G. H. Golub, Regularized total least squares based on

quadratic eigenvalue problem solvers, BIT, 44 (2004), pp. 793–812.[29] C. Vonesch and M. Unser, A fast multilevel algorithm for wavelet-regularized image restora-

tion, IEEE Trans. Image Process., 18 (2009), pp. 509–523.[30] W. Zhu, Y. Wang, and J. Zhang, Total least-square reconstruction with wavelets for optical

tomography, J. Opt. Soc. Amer., 15 (1998), pp. 2639–2650.

Dow

nloa

ded

11/1

5/14

to 1

30.2

07.5

0.37

. Red

istr

ibut

ion

subj

ect t

o SI

AM

lice

nse

or c

opyr

ight

; see

http

://w

ww

.sia

m.o

rg/jo

urna

ls/o

jsa.

php

a wavelet-based multilevel approach for blind deconvolution problems

Documents