J Math Imaging Vis (2015) 52:345–355DOI 10.1007/s10851-015-0568-x

Retinex by Higher Order Total Variation L1 Decomposition

Jingwei Liang · Xiaoqun Zhang

Received: 30 June 2014 / Accepted: 3 February 2015 / Published online: 10 March 2015© Springer Science+Business Media New York 2015

Abstract In this paper, we propose a reflectance andillumination decomposition model for the Retinex prob-lem via high-order total variation and L1 decomposition.Based on the observation that illumination varies smootherthan reflectance, we propose a convex variational modelwhich can effectively decompose the gradient field of imagesinto salient edges and relatively smoother illumination fieldthrough the first- and second-order total variation regular-izations. The proposed model can be efficiently solved bya primal–dual splitting method. Numerical experiments onboth grayscale and color images show the strength of the pro-posed model with applications to Retinex illusions, medicalimage bias field removal, and color image shadow correction.

Keywords Retinex · Image decomposition · High-ordertotal variation · Shadow correction

1 Introduction

In the past decades, the study of the Retinex problemhas inspired a wide range of applications and discussions[3,14,21]. TheRetinex theory is originally proposed by LandandMcCann [14] as amodel of color perceptionof the human

J. LiangGREYC, CNRS UMR 6072, ENSICAEN, Université de Caen,Caen, Francee-mail: jingwei.liang@ensicaen.fr

X. Zhang (B)Department of Mathematics, MOE-LSC, and Institute of NaturalSciences, Shanghai Jiao Tong University, Shanghai, Chinae-mail: xqzhang@sjtu.edu.cn

visual system (HVS), whose idea is that HVS can ascer-tain reflectance of a field in which both illumination andreflectance are unknown. Our visual system tends to see thesame color of a given scene regardless of different illumi-nation conditions. In other words, it ensures that the colorof the objects remains relatively constant under varying illu-minations. One of the most well-known Retinex illusions isthe Adelson’s checkerboard shadow illusion1 shown in Fig.1. Visually, region A of Fig. 1a seems darker than regionB, while digitally these two regions have exactly the sameintensity value. This phenomena is causedbydifferent illumi-nation conditions, and the perceived intensity of the objectsis the combination of reflectance and illumination. By tak-ing into account the surroundings of the object (shadow ofthe cylinder, periodic pattern of the checkerboard), our HVSdiscounts the illumination and perceive the reflectance auto-matically.

The primary goal of Retinex theory is to decompose agiven image I into two components, reflectance R and illu-mination L such that

I (x) = R(x) × L(x), (1)

for x ∈ Ω where Ω ⊂ R2 is the domain of the image. To

simplify (1), one can apply the logarithm assuming that bothR and L are positive which leads to

i(x) = r(x) + l(x). (2)

Most of the decomposition methods are based on this addi-tive model. The first Retinex algorithm proposed in [14] isbased on path following, and further studied in [3,20]. Later

1 http://web.mit.edu/persci/people/adelson/checkershadow_illusion.html

123

346 J Math Imaging Vis (2015) 52:345–355

Fig. 1 Checkerboard shadow illusion. a original checkerboard image,b illustration of illusion free image

on, the idea of path following is formulated into variationaland PDE-based models. In [21], Morel et al. show that if thelight paths are assumed to be symmetric random walks, thenthe Retinex solution satisfies a discrete Poisson equation andcan be efficiently solved by using only two FFTs. Under vari-ational framework, the followed total variation (TV) model[19] aims to extract piecewise constant reflectance r withdata term in gradient field

r̂ = argminr

∫Ω

(t |∇r | + 1

2|∇r − ∇i |2

)dx, (3)

where t > 0 is the weight parameter. A simpler L1-basedmodel is proposed in [18], which reads

r̂ = argminr

∫Ω

|∇r − δt (∇i)|dx, (4)

where δt (∇i) is the thresholded gradient field with respect toparameter t . This model appears to be efficient on suppress-ing the lighting effect of testing images, however, the lossof reflectance details and contrast can also be observed dueto penalization on the magnitude of image gradient. More-over, a rigorous analysis of the proposed model is missing.Recently, Zosso et al. [30] extended the TV-based models toa unified non-local formulation.

Decomposition models by penalizing both r and l simul-taneously are also studied. For instance Kimmel et al. in [13]propose the followed TV + H1 decomposition model

r̂ = argminr

∫Ω

(|∇r | + α(r − i)2

+β|∇(r − i)|2)dx, (5)

where illumination l = i − r is implicitly assumed to besmooth and penalized with H1 norm. A similar model isfurther investigated by Ng and Wang in [22] with more con-straints enforced. More specifically, the following minimiza-tion problem over both r and l is considered

minr≤0,l≥i

∫Ω

(|∇r | + α

2|∇l|2

+ β

2(i − r − l)2 + μ

2l2

)dx, (6)

where the l ≥ i constraint is based on the assumption that0 < R ≤ 1.

In this paper, we also consider a decomposition approachto recover both r and l simultaneously. In particular, we focuson extracting illumination l using high-order regularization.Our proposed model is closely related to recently developedhigh-order total variation regularization. Therefore, in thefollowing, we present some related models and notations.

1.1 High-Order Total Variation Methods

It is well known that total variation regularization [24] suf-fers the so-called staircase artifact. In order to suppress thiseffect, many higher order functionals have been studied sincethe pioneering infimal convolution model proposed in [5] oncombining the first- and second-order total variation, whichis defined as,(φ�ψ

)(u) = inf

u=v+wφ(v) + ψ(w),

where φ and ψ are two functionals.The infimal convolution of thefirst- and second-order vari-

ations proposed in [5] with weight takes the form

Jβ(u) = infv+w=u

∫Ω

|∇v|dx + β

∫Ω

|∇2w|dx, (7)

where∫Ω

|∇2w| denotes the total variation of the Hessian ofw for w ∈ W 2,1(Ω). Thus, image u can now contain bothpiecewise constant and piecewise linear components. Thisformulation is restudied in [25,26] in the discrete setting forimage restoration.

In a more general dual formulation, the infimal convolu-tion of the first- and second-order total variation is definedas

ICTVβ(u) = supp∈I1q∈I2

infw

∫Ω

((u − w)div(p)

+ βwdiv2(q))dx,

(8)

where I1 = {p ∈ C1c (Ω;R2), ||p||∞ ≤ 1} and I2 =

{p ∈ C2c (Ω;R2×2), ||p||∞ ≤ 1}. Note that if the sym-

metric Hessian is considered, we can also define I2 ={p ∈ C2

c

(Ω;Sym2(R2)

), ||p||∞ ≤ 1} where Sym2(R2)

denotes the space of symmetric matrices. Here, we abusethe notation of the infinity norm || · ||∞ in C1

c (Ω;R2)

and C2c

(Ω;Sym2(R2)

). A similar, yet different formulation,

known as total generalized variation (TGV), is proposed andrigorously studied in [4]. In particular, the second-order TGVis defined as,

123

J Math Imaging Vis (2015) 52:345–355 347

TGV2β(u)= sup

v

{ ∫Ω

udiv2vdx | v∈C2c(Ω,Sym2

(R2))

,

||v||∞ ≤ β, ||divv||∞ ≤ 1

}. (9)

In a simplified form, the alternative primal form of (9) canbe written as

TGV2β(u) = inf

w

{∫Ω

|∇u − w|dx + β

∫Ω

|∇w|dx},

(10)

where∇w ∈ R2×2 is the (symmetrized) gradient of the defor-

mation field w ∈ R2. This formulation can be also viewed

as replacing the decomposition u = v + w in (7) with thedecomposition of the gradient field∇u = ∇v+w. More dis-cussions on theoretical properties of the connections betweenthe two functionals can be found in [1,4].

Another way of using high-order TV is the direct combi-nation, not infimal convolution, of the first-order and higherorder regularizations, such as [2,16,17,23]. For example [23]proposes a regularization model with direct combination ofthe first- and second-order TV for image restoration

minu

{1

2

∫Ω

(Tu − u0)2dx

+ α

∫Ω

|∇u|dx + β

∫Ω

|D2u|dx},

where T is a bounded linear operator and D is the gradientin distribute sense. This model has been studied in a moregeneral and theoretical setting under the space of boundedHessian. Finally, nonlinear high-order regularization [8,27],combining with curvature line (e.g. Euler’s elastic), is alsovery popular, especially for image inpainting. In this paper,we focus more on the first- and second-order total variation.

1.2 Our Contributions

The infimal convolution of the first- and second-order totalvariation (in both ICTV and TGV) is designed to balancethe first and high-order singularities presented in images. Byinvolving higher order derivatives, these functionals can cap-ture higher order edges instead of only piecewise constantcomponents. It has been shown that these methods can sup-press staircase artifact significantly.

From another point of view, formulation (10) [and (7)]decomposes ∇u field into two parts, L1 norm of the resid-ual ∇u − w and TV semi-norm of w. It is well known thatTV + L1 decomposition has interesting geometrical prop-erties. As studied in [7,28], TV + L1 decomposition modelallows a scale-dependent decomposition of geometry fea-tures, which is invariant to image contrast. In Retinex the-ory, as considered in previous path following based work,the illumination varies relatively slower, hence can be con-

sidered having bigger scale in the gradient field. Further-more, the nature of illumination often follows certain paths,thus piecewise linear approximation can model this behaviorappropriately.

This motivates us to consider a decomposition model sep-arating the higher order piecewise smooth components fromthe edges of relatively smaller scale in the gradient field. Inparticular, we propose to decompose the image i into theillumination l and reflectance r , and set the regularizationsas

J (r) =∫

Ω

|∇r |dx,

Jβ(l) = β

∫Ω

|∇2l|dx(11)

We call this proposed approach higher order total varia-tion L1 (HoTVL1) illumination and reflectance decomposi-tion model. Close connections to the previous infimal con-volution models (7) and (10) are discussed, where we aim toextract higher order singularities as the smoother illumina-tion component. To the best of our knowledge, this is also thefirst time that higher order infimal convolution model is usedfor the purpose of image decomposition. Furthermore, thismodel is different from the TV + H1 decomposition modelconsidered in [13,22], where the H1 norm is penalized forillumination. In the section of numerical experiments, weshow that the proposed model can better separate differentscales of smoothness and singularities by TV + L1 decom-position of the gradient field. Moreover compared to the L1-based model [18], the proposed method can preserve betterthe edges with small magnitude without smearing out fea-tures of image with low intensities.

The rest of the paper is organized as following. In Sect.2, we present the proposed model with constraints and dis-cuss the connections with previous higher order regulariza-tion models. Furthermore, the existence and uniqueness ofthe solution for the proposedmethod are rigorously discussedin an extended functional space, i.e. , the product space of thebounded variation and bounded Hessian. In Sect. 3, a splitinexact Uzawa based primal–dual splitting algorithm [29]is applied to solve the proposed model. Finally, we presentnumerical experiments on synthetic gray scale images, visualillusion images, medical images with bias filed and real colorimage examples, and compare the performance with some ofthe aforementioned existing variational-based methods.

2 High-Order TV + L1 Decomposition Model

2.1 Proposed Model

Here, we present the first higher order TV + L1 variationalmodel for reflectance and illumination decomposition, whichreads

123

348 J Math Imaging Vis (2015) 52:345–355

minr,l

{Eα,β(r, l) = 1

2

∫Ω

(i − r − l)2dx

+α( ∫

Ω

|∇r |dx + β

∫Ω

|∇2l|dx)}

, (12)

where l denotes illumination and r the reflectance, α >

0, β > 0 are regularization parameters.Roughly speaking, we extract relatively smoother piece-

wise linear component as l, and texture part as r in gradientfield for Retinex decomposition. The proposed model can beinterpreted as infimal convolution, where the connections areas following,

– Let u = r + l, then (12) is equivalent to

minu,l

{1

2

∫Ω

(u − i)2dx

+α

∫Ω

(|∇(u − l)| + β|∇2l|)dx},

⇐⇒ minu

{1

2

∫Ω

(u − i)2dx + αICTVβ(u)

}. (13)

– If we further replace ∇l by v =[v1v2

]in (12), then we

obtain the following model

minu,v

{1

2

∫Ω

(u − i)2dx

+α

∫Ω

(|∇u − v| + β|∇v|)dx},

⇐⇒ minu

{1

2

∫Ω

(u − i)2dx + αTGV2β(u)

}. (14)

Note that for both infimal convolutionmodels, l is not explic-itly given, while it can be extracted from numerical schemesor solved via the Poisson equation ∇l = v with boundaryconditions.

Model (12) and the infimal convolution (13), (14) havesome drawbacks if we are interested in the solutions of r andl. It is easy to see that any solution pair (r̂ + c, l̂ − c) with cbeing a constant is still a solution, and this non-uniquenessmay greatly affect the quality of the solution. Furthermore,to show the existence of solutions, coercivity of the energyis also needed for the theoretical proof.

Therefore, we consider an extended version ofmodel (12),which imposes box constraints on both r and l components

minr∈Br ,l∈Bl

{Eα,β,τ (r, l) = 1

2

∫Ω

(i − r − l)2dx

+α(∫

Ω

|∇r |dx + β

∫Ω

|∇2l|dx)

+ τ

2

∫Ω

l2dx

}, (15)

where τ is a small positive number to ensure the boundednessof l, Br andBl are the box constraints for r and l respectively.For instance, similar to the constraints considered in [22],one can consider Br =] − ∞, 0] and Bl = [i,+∞[ underthe assumptions that R ∈ (0, 1].

2.2 Existence and Uniqueness of Solution

The functional Eα,β,τ (r, l) in (15) is defined onW 1,1×W 2,1.To establish the existence for such regularization, we usuallyneed to discuss a larger Banach space for r and l. More pre-cisely, we consider r in the bounded variation space BV(Ω)

and l in the bounded Hessian space BH(Ω), and show theexistence of the solution pair (r, l) of the following functionalin product space,

minr∈BV(Ω)

l∈BH(Ω)

{Eα,β,τ (r, l) = 1

2||i − r − l||2

+α(||Dr ||1+β||D2l||1

)+ τ

2||l||2 + ιB(r, l)

}, (16)

where || · || denotes the norm in L2(Ω),B denotes the boxconstraint

B = Br × Bl ,

ιB(·) = ιBr (r) + ιBl (l) is the corresponding indicator func-tion, ||Dr ||1 and ||D2l||1 denote the total variation of the first-and second-order derivatives in distribution sense, which isdefined later.

We first present some preliminaries for bounded Hessianspace proposed in [9] and in [2,23] in the context of imagerestoration. Let Ω be an open subset of Rn with Lipschitzboundary, recall that the Sobolev space W 1,1(Ω) is definedas

W 1,1(Ω) = {u ∈ L1(Ω) | ∇u ∈ L1(Ω)

}.

The standard total variation semi-norm in distributionsense is defined as

||Du||1 =∫

Ω

|Du|dx = supp∈C1

c (Ω)n

||p||∞≤1

∫Ω

udivpdx, (17)

where divp = ∑ni=1

∂pi∂xi

(x).Following Demengel [9] and [23], we consider the

space of bounded Hessian functions BH(Ω) (also called asBV2(Ω) in [2]) on extending the notion of total variation(17). Define

||D2u||1 =∫

Ω

|D2u|dx

= supξ∈C2

c (Ω;Rn×n)

||ξ ||∞≤1

∫Ω

〈∇u, div(ξ)〉dx, (18)

123

J Math Imaging Vis (2015) 52:345–355 349

where div(ξ) = (divξ1, · · · , divξn) with

∀i, ξi ={ξ

(1)i , · · · , ξ

(n)i

}∈ R

n, divξi =n∑

k=1

∂ξ(k)i

∂xk,

and ||ξ ||∞ = supx∈Ω

√∑ni, j=1 |ξ ( j)

i (x)|2. The space BH(Ω)

consists of all functions u ∈ W 1,1(Ω) whose distributionalHessian is a finite Radon measure, i.e.

BH(Ω) = {u ∈ W 1,1(Ω) | ||D2u||1 < ∞}

.

It is immediate to see that W 2,1(Ω) ⊂ BH(Ω) andBH(Ω) is a Banach space equipped with norm ||u||BH(Ω) =||u||1 + ||∇u||1 + ||D2u||1, where || · ||1 denotes the L1 normin the corresponding space. In the following, we summarizesome definitions and main properties in BH(Ω), which canbe found in [2,9,23]:

– (A weak∗ topology) Let {uk}k∈N, u belong to BH(Ω).Sequence {uk} converges to u weakly∗ in BH(Ω) if

||uk − u||1 → 0, ||∇uk − ∇u||1 → 0,∫Ω

〈∇uk, div(ξ)〉dx

−→∫

Ω

〈∇u, div(ξ)〉dx,∀ξ ∈ C2c (Ω;Rn×n). (19)

– (Lower semi-continuity) The semi-norm ||D2u||1 is lowersemi-continuous endowed with strong topology ofW 1,1(Ω). More precisely, if ||uk − u||1 → 0 and ||∇uk −∇u||1 → 0, then

||D2u||1 ≤ lim infk→∞ ||D2uk ||1.

In particular, for {uk}k∈N ∈ W 1,1(Ω), if

lim infk→∞ ||D2uk ||1 < ∞,

then u ∈ BH(Ω).– (Compactness in BH(Ω)) Suppose that {uk}k∈N is

bounded in BH(Ω), then there exists a subsequence{uk j } j∈N and u ∈ BH(Ω) such that {uk j } j∈N weakly∗converges to u.

– (Embedding) If Ω has a Lipschitz boundary and it isconnected, then it can be shown that there exists positiveconstants C1, C2 such that

||∇u||1 ≤ C1||D2u||1 + C2||u||1, (20)

and BH(Ω) is continuously embedded in L2(Ω) whenn = 2.

In the following, we use the above-mentioned propertiesof BH(Ω) together with similar ones of BV(Ω) to establishthe existence and uniqueness of solution for problem (16).

Theorem 1 Suppose i ∈ L2(Ω) and α, β, τ > 0, then prob-lem (16) admits a unique solution pair (r∗, l∗) ∈ BV(Ω) ×BH(Ω).

Proof Let {(rk, lk)}k∈N be a minimizing sequence of (16),and M > 0 be the upper bound of the sequence such that

||rk + lk − i ||2 < M, ||lk ||2 < M, (21)

||Drk ||1 < M, ||D2lk ||1 < M, (22)

rk ∈ Br , lk ∈ Bl , (23)

for every k ∈ N. Since {lk}k∈N is uniformly bounded inL2(Ω), it is easy to see that {rk}k∈N is also uniformlyboundedin L2(Ω) combining with (21) and i ∈ L2(Ω). Moreover, bythe boundedness of Ω , sequence {rk} is bounded in L1(Ω)

and moreover bounded in BV(Ω) since ||Drk ||1 < M .For {lk}, we can derive similarly that {lk} is bounded in

L1(Ω). By virtue of the embedding inequality (20), we alsohave

||∇lk ||1 < C1||D2lk ||1 + C2||lk ||1 < M ′,

where M ′ is a constant number for all k ∈ N. Thus {lk} isuniformly bounded in BH(Ω).

From the compactness theorem in bothBVandBHspaces,we can obtain a subsequence (rk j , lk j ) converging weakly∗to (r∗, l∗) in BV(Ω) × BH(Ω). It is easy to see that thefunctional is proper since any constant function l and r hasfinite energy. Furthermore, the overall functional Eα,β,τ isconvex and l.s.c under weak topology, the constraint sets Br

andBl are closed, thereforewe can derive that theminima canbe attained at (r∗, l∗) by the theory of calculus of variation. Itis also straightforward to see that the solution is unique sincethe functional is strongly convex with respect to (r, l). ��

3 Primal–Dual Splitting Algorithm

In this section, we present the algorithm for solving the dis-crete version of (15), which has been well studied in the lit-erature. For instance the widely used split Bregman method[12] and primal–dual splittingmethods [6,10] based on oper-ator splitting techniques. Here, we adopt the split inexactUzawa (SIU) method developed in [29] which yields a sim-ple iteration scheme for our problem.

Note also that the two unknowns r, l are coupled in (15),therefore, an inner alternating scheme is applied in orderto fully decouple the sub-problems for r and l. However, itshould be mentioned that such alternating scheme does notguarantee the whole sequence converge to the minimizer ofthe original problem.

123

350 J Math Imaging Vis (2015) 52:345–355

We first define some variables and notations to simplify(15). Denote

x =[rl

], A = [

Id, Id], B = [

0, Id],

and the auxiliary ones

u = ∇r, v = ∇2l,

y =[uv

], and L =

[∇, 00, ∇2

].

Here, we use forward difference and Neumann bound-ary conditions for the discrete gradient and symmetrizedHessian. In R2, for u ∈ W 1,1(Ω), define

D(∇u) = 1

2

(D + DT )

(∇u) =[ξ11, ξ12ξ21, ξ22

],

as the symmetrized Hessian, where

ξ11 = ∂v1

∂x1, ξ22 = ∂v2

∂x2,

ξ12 = ξ21 = 1

2

(∂v1

∂x2+ ∂v2

∂x1

),

(24)

for v =[v1v2

]= ∇u and the divergence can be defined

accordingly.We further define the following functionals,

H(x) = 1

2||i − Ax ||2,

J (x, y) = α||y||1,β + ιB(x) + τ

2||Bx ||2,

where ||y||1,β = ||u||1 + β||v||1. As a result, (15) can be for-mulated into the following form

minx,y

H(x) + J (x, y) s.t. Lx = y, (25)

whose augmented Lagrangian formula is

maxp

minx,y

{L(p; x, y) = H(x) + J (x, y)

+〈p, Lx − y〉 + ν

2||Lx − y||2

}, (26)

where p =[prpl

]is the Lagrangian multiplier.

The SIU method is an inexact variant of alternating direc-tionmethodofmultipliers (ADMM) [11] applied to the aboveproblem with an extra proximal term for the update of xk+1,which reads⎧⎪⎪⎪⎪⎪⎨⎪⎪⎪⎪⎪⎩

xk+1 = argminx

L(pk; x, yk

)+ 1

2||x − xk ||Mν

,

yk+1 = argminy

L(pk; xk+1, y

),

pk+1 = pk + ν(Lxk+1 − yk+1

),

(27)

where Mν is a positive definite matrix. To obtain an easyiterative scheme, we choose Mν = Id − νLT L , where 0 <

ν < 1/||LT L ||.In the following, we present a brief derivation for the

update of xk+1 and yk+1 respectively.

Update of xk+1: Let Mν = Id − νLT L , then

xk+1 = argminx

L(pk; x, yk

)+ 1

2||x − xk ||2Mν

= argminx

ιB(x) + 1

2||i − Ax ||2

+ τ

2||Bx ||2 + 1

2||x − wk ||2,

(28)

where wk = xk − LT (νLxk + pk − νyk) =[wkr

wkl

]. The

corresponding first order optimality condition of (28) is

0 ∈ NB(x) + AT (Ax − i) + τ BT Bx + (x − wk)

∈[NBr (r)NBl (l)

]+

[r + l − ir + l − i

]+ τ

[0l

]+

[r − wk

rl − wk

l

].

Let w̃r = wkr +i, w̃l = wk

l +i , andPC the projection oper-ator onto a nonempty closed convex set C. In the following,we discuss the updating formula under different scenarios ofthe box constraints B.

– If Bl = R, which implies that there is no constraint on l,we have

{NBr (r) + 2r + l = w̃r ,

r + (τ + 2)l = w̃l .

Substituting the second equation in the first one, weobtain

NBr (r) + 2τ + 3

τ + 2r = w̃r − w̃l

τ + 2.

Thus, the update formula for rk+1 and lk+1 reads

⎧⎪⎪⎨⎪⎪⎩rk+1 = PBr

((τ + 2)w̃r − w̃l

2τ + 3

),

lk+1 = w̃l − rk+1

τ + 2.

– If Br = R, similarly we get

{2r + l = w̃r ,

2NBl (l) + (2τ + 3)l = 2w̃l − w̃r .

123

J Math Imaging Vis (2015) 52:345–355 351

And the update formula for rk+1 and lk+1 are

⎧⎪⎪⎨⎪⎪⎩

lk+1 = PBl

(2w̃l − w̃r

2τ + 3

),

rk+1 = w̃r − lk+1

2.

– If both Br and Bl are not the whole space, then the sub-problem is coupled on r and l, and an alternating projec-tion method is applied, which reads

⎧⎪⎪⎨⎪⎪⎩r ← PBr

(w̃r − l

2

),

l ← PBl

(w̃l − r

2 + τ

).

(29)

Note that in theory many iterations are needed for thisstep to get accurate rk+1, lk+1.

Update of yk+1: The update of yk+1 is rather simple. As||y||1,β = ||u||1 + β||v||1 is separable, we haveyk+1 = argmin

yL(pk; xk+1, y)

= argminy

α||y||1,β + ν

2||y − Lxk+1 − pk/ν||2

= argminu,v

α||u||1 + ν

2||u − ∇rk+1 − pkr /ν||2

+ αβ||v||1 + ν

2||v − ∇2lk+1 − pkl /ν||2,

which leads to the following two simple threshold steps{uk+1 = Tα/ν

(∇rk+1 + pk+1r /τ

),

vk+1 = Tαβ/ν

(∇2lk+1 + pk+1l /τ

),

(30)

where Tγ (a) = max(||a|| − γ, 0) a||a|| is the isotropic soft-

threshold operator.Finally, the update for the dual variable pk+1 is straight-

forward. To this end, combining (29), (30) and the update ofpk+1 (27) we obtain the iteration scheme for solving (15).

4 Numerical Tests

To demonstrate the performance of our proposed method, inthis section, we consider several image decomposition prob-lems, including synthetic examples, Retinex illusion exam-ples, medical image bias field removal and color imageshadow correction. We compare our proposed method (15)(HoTVL1) with two recent variational methods (4) (MMO)and (6) (NW) proposed in [18] and [22] respectively.

For all the tests, the recovered r of MMO and our methodare the direct outputs from the model, so is NW method forthe synthetic example and Retinex problem. While for the

Fig. 2 Twosynthetic image examples.a synthetic image r1,b syntheticshadow l1, c composed image i1 = r1 + l1; d synthetic image r2,e synthetic shadow l2, f composed image i2 = r2 + l2

Fig. 3 Decomposition comparison of synthetic example 1. From top tobottom and from left to right: recovered reflectance r and illuminationl by MMO, NW, and the proposed method

bias field removal and color image shadow correction, theoutput r of NW method is obtained via i − l.

4.1 Synthetic Images

We start with synthetic examples with two different cases. Asshown in Fig. 2, two piecewise constant images r with differ-ent geometry properties, so are the corresponding shadows lwith piecewise smooth structures, and the simulated imageis the sum of them, namely i = r + l.

Decomposition results are shown in Figs. 3 and 4. Aswe can observe, MMO and HoTVL1 produce better visual

123

352 J Math Imaging Vis (2015) 52:345–355

Fig. 4 Decomposition comparison of synthetic example 2. From top tobottom and from left to right: recovered reflectance r and illuminationl by MMO, NW, and the proposed method

Table 1 Recovered intensity values of regions A and B of the twoimages

Image Original MMO NW HoTVL1

Checker-board A 120 80 90 85

B 120 145 125 174

Cube A 140 26 85 10

B 140 191 110 250

results than NW method on these examples. Intuitively, NWmethod penalizes the �2-norm of the gradient of the illu-mination which is not contrast invariant, hence the recov-ered illumination l contains more information of the edgesof reflectance. On the contrast, only very weak signal of r iscontained in the recovered illumination l by MMO methodand our proposed method. This will be further demonstratedin Retinex illusion and color image examples.

For this example, we set β = 10 for both examples, α = 2for the first synthetic image, and α = 4 for the second one.Both of the box constraints are [0, 255].

4.2 Retinex Illusion

The aforementioned checkerboard shadow image and theLogvinenko’s cube shadow illusion image [15], as shownin Fig. 5, are tested for Retinex illusion. For both images,though visually region B is brighter than region A, they areof the same intensity value.

Table 1 shows the comparison of the recovered intensityvalues of the two regions of the three methods. Our proposedmethod recovers the best contrast compared to the other twoon both images.

Fig. 5 Two test images for Retinex illusion. aAdelson’s checkerboardshadow image, the intensity value of the marked area is 120. b Logvi-nenko’s cube shadow image, the intensity value of the marked area is140

Fig. 6 Decomposition comparison of the checkerboard example. Fromtop to bottom and from left to right: recovered reflectance r and illumi-nation l by MMO, NW, and the proposed model

Figure 6 shows the visual comparison on the checker-board image. Similar to the synthetic example, MMO andthe proposedmethod obtain visually preferable outputs, how-ever, the recovered illumination in our proposedmethod con-tains less reflectance information compared toMMO.Similarconclusions can be drawn from the comparison of the cubeimage, see Fig. 7.

It can be noticed that the brightness of MMO’s result andours is very different, especially for Fig. 7. This is mainlydue to the fact that for MMO method, the intensity value ofthe top left pixel is used to solve the Poisson equation, alsothe right column and bottom row of the output is brighterthan the other parts of the image due to boundary condi-tion.

In this test, β = 10 for both examples, and we set α = 4for the checkerboard image, and α = 10 the cube image. Thebox constraints are Br = [0, 255] and Bl = [−255, 0] forboth images.

123

J Math Imaging Vis (2015) 52:345–355 353

Fig. 7 Decomposition comparison of the cube example. From top tobottom and from left to right: recovered reflectance r and illuminationl by MMO, NW, and the proposed method

Fig. 8 MRI image bias field removal. a, d, g, j image view 1 and theresult of the three methods; b, e, h, k image view 2 and the result of thethree methods; c, f, i, l image view 3 and the result of the three methods

Fig. 9 Comparison of recovered reflectance r on Text image

4.3 Medical image bias field correction

In medical imaging, the obtained images may be corruptedby bias fields due to non-uniform illuminations, parallelMRIimaging for example. The correction of the bias field is sim-ilar to Retinex problem where we need to remove the lighteffect caused by illumination. Here, we adopt the settingsdiscussed in [18], and Fig. 8 shows the comparison of themethods. As observed, all methods can provide visual prefer-able results compared to the original ones, especially for thebottom part of the images.

For this example, we set α = 1/10 and β = 20. Thebox constraints are Br = [−20, 0] and Bl = [−20, 0] aftertaking the logarithm, and I is pre-scaled to (0, 1].

4.4 Color Image Shadow Correction

The final illustrative example is the correction of shadowedcolor image. For the two color images given in Figs. 9a and10a, we choose the HSV (hue, saturation, value) color spaceand only process theV channel, assuming that shadowaffectsonly the brightness of the image. Figure 9 shows the compari-son on the text image. Note that NWmethod also adopts one-step γ -correction as post-processing, which may improvethe contrast of under-exposed images. We can see that theestimated shadow of our method is visually more accurateand the shadow effect can be partially removed in the recov-

123

354 J Math Imaging Vis (2015) 52:345–355

Fig. 10 Comparison of recovered reflectance r on Wall image

Fig. 11 Comparison of recovered illumination l on Text image

Fig. 12 Comparison of recovered illumination l on Wall image

ered reflectance. Similar observation can be obtained fromFigs. 11 and 12.

For this test, we set α = 1/12 for text image and 1/10 forthe wall image, β = 10 is fixed. The box constraints are thesame as the bias field correction’s.

4.5 Computational Time

To conclude this part, we present the computational timecomparison of the three methods. Since we have box con-straints for both r and l, inner iteration is needed for theupdate of xk+1, and the number of iterations is set as 1 for alltests. Among the methods, MMO method is the fastest dueto its simplicity, NWmethod is the second, and our proposedmethod is more time consuming mainly due to the second-order TV regularization. For example in the synthetic exam-ple with image size 128 × 128, it takes less than 1 s for theMMOmethod, around 1 s for the NW, while around 15 s forour method. However, the computational time of our methodcan be improved by choosing more efficient algorithms.

5 Conclusions

In this paper, we have presented a novel reflectance and illu-mination decomposition model based on high-order TV +L1 regularization, which is closely related to the infi-mal convolution of first- and second-order TV regulariza-tions. The proposed model can nicely separate the rela-tively smoother piecewise linear component, which is mod-eled as the global illumination, from the relatively detailedreflectance edges. Numerical experiments on inhomoge-neous background removal and color shadow correction haveshown that our proposed model extracts HVS preferableillumination, and detail preserved reflectance, comparing toother decomposition models [18,22].

Acknowledgments We would like to thank the authors of [18] and[22] for sharing their source codes, and the reviewers of this manuscriptfor their helpful comments and suggestions. The work of X. Zhangwas supported by NSFC11101277, NSFC91330102 and 973 program(#2015CB856000).

References

1. Benning, M., Brune, C., Burger, M., Müller, J.: Higher-orderTV methods-enhancement via Bregman iteration. J. Sci. Comput.54(2–3), 269–310 (2013). doi:10.1007/s10915-012-9650-3

2. Bergounioux, M., Piffet, L.: A second-order model for imagedenoising. Set. ValuedVar. Anal. 18(3–4), 277–306 (2010). doi:10.1007/s11228-010-0156-6

3. Bertalmo, M., Caselles, V., Provenzi, E.: Issues about Retinex the-ory and contrast enhancement. Int. J. Comput. Vis. 83(1), 101–119(2009). doi:10.1007/s11263-009-0221-5

4. Bredies, K., Kunisch, K., Pock, T.: Total generalized varia-tion. SIAM J. Imaging Sci. 3(3), 492–526 (2010). doi:10.1137/090769521

5. Chambolle, A., Lions, P.L.: Image recovery via total variationminimization and related problems. Numer. Math. 76(2), 167–188(1997). doi:10.1007/s002110050258

6. Chambolle, A., Pock, T.: A first-order primal-dual algorithm forconvex problems with applications to imaging. J. Math. ImagingVis. 40(1), 120–145 (2011). doi:10.1007/s10851-010-0251-1

123

J Math Imaging Vis (2015) 52:345–355 355

7. Chan, T.F., Esedolu, S.: Aspects of total variation regularized �1function approximation. SIAM J. Appl. Math. 65(5), 1817–1837(2005). http://www.jstor.org/stable/4096154

8. Chan,T.F.,Kang, S.H.: Euler’s elastica and curvature based inpaint-ings. SIAM J. Appl. Math. 63, 564–592 (2002). doi:10.1137/S0036139901390088

9. Demengel, F.: Fonctions hessien born. Annales de l’institut Fourier34(2), 155–190 (1984). http://eudml.org/doc/74627

10. Esser, E., Zhang, X., Chan, T.F.: A general framework for a class offirst order primal-dual algorithms for convex optimization in imag-ing science. SIAM J. Imaging Sci. 3, 1015–1046 (2010). doi:10.1007/s10915-010-9408-8

11. Gabay, D.: Applications of the method of multipliers to variationalinequalities, Augmented Lagrangian methods: applications to thesolution of boundary-value problems, 15, chap. IX, pp. 299–331(1983)

12. Goldstein, T., Osher, S.: The split Bregman method for �1-regularized problems. SIAM J. Imaging Sci. 2(2), 323–343 (2009).doi:10.1137/080725891

13. Kimmel, R., Elad, M., Shaked, D., Keshet, R., Sobel, I.: A vari-ational framework for Retinex. Int. J. Comput. Vis. 52(1), 7–23(2003). doi:10.1023/A:1022314423998

14. Land, E.H., McCANN, J.J.: Lightness and Retinex theory. J.Opt. Soc. Am. 61(1), 1–11 (1971). doi:10.1364/JOSA.61.000001.http://www.opticsinfobase.org/abstract.cfm?URI=josa-61-1-1

15. Logvinenko, A.D.: Lightness induction revisited. PERCEPTION-LONDON-28, 803–816 (1999). doi:10.1068/p2801

16. Lysaker, M., Lundervold, A., Tai, X.C.: Noise removal usingfourth-order partial differential equation with applications to med-ical magnetic resonance images in space and time. IEEE TransImage Process 12(12), 1579–1590 (2003). doi:10.1109/TIP.2003.819229

17. Lysaker, M., Tai, X.C.: Iterative image restoration combining totalvariation minimization and a second-order functional. Int. J. Com-put. Vis. 66(1), 5–18 (2006). doi:10.1007/s11263-005-3219-7

18. Ma, W., Morel, J.M., Osher, S., Chien, A.: An �1-based variationalmodel for retinex theory and its application to medical images. In:CVPR, pp. 153–160. IEEE (2011). doi:10.1137/100806588

19. Ma, W., Osher, S.: A TV bregman iterative model of Retinex the-ory. Inverse Probl. Imaging 6(4), 697–708 (2012). doi:10.3934/ipi.2012.6.697

20. Marini,D., Rizzi,A.:A computational approach to color adaptationeffects. Image Vis. Comput. 18(13), 1005–1014 (2000). doi:10.1016/s0262-8856(00)00037

21. Morel, J., Petro, A., Sbert, C.: A PDE formalization of Retinextheory. IEEE Trans. Image Process. 19(11), 2825–2837 (2010).doi:10.1109/TIP.2010.2049239

22. Ng, M.K., Wang, W.: A total variation model for Retinex. SIAMJ. Imaging Sci. 4(1), 345–365 (2011). doi:10.1137/100806588

23. Papafitsoros, K., Schönlieb, C.: A combined first and second ordervariational approach for image reconstruction. J. Math. ImagingVis. 48(2), 308–338 (2014). doi:10.1007/s10851-013-0445-4

24. Rudin, L.I., Osher, S., Fatemi, E.: Nonlinear total variation basednoise removal algorithms. Phys. D 60(1), 259–268 (1992). doi:10.1016/0167-2789(92)90242-f

25. Setzer, S., Steidl, G.: Variational methods with higher order deriv-atives in image processing. In: N. Press (ed.) Approximation XII,pp. 360–386. Brentwood (2008)

26. Setzer, S., Steidl,G., Teuber, T.: Infimal convolution regularizationswith discrete �1-type functionals. Comm.Math. Sci. 9(3), 797–872(2011). doi:10.4310/cms.2011.v9.n3.a7

27. Tai, X.C., Hahn, J., Chung, G.J.: A fast algorithm for Euler’s elas-tica model using augmented Lagrangian method. SIAM J. ImagingSci. 4(1), 313–344 (2011). doi:10.1137/100803730

28. Yin, W., Goldfarb, D., Osher, S.: The total variation regularized L1

model for multiscale decomposition. Multiscale Model. Simul. 6(2007). doi:10.1137/060663027

29. Zhang, X., Burger, M., Osher, S.: A unified primal-dual algorithmframework based on Bregman iteration. J. Sci. Comput. 46(1), 20–46 (2011). doi:10.1137/09076934X

30. Zosso, D., Tran, G., Osher, S.: A unifying retinex model based onnon-local differential operators. In: IS&T/SPIE Electronic Imag-ing, pp. 865,702–865,702. International Society for Optics andPhotonics (2013). doi:10.1117/12.2008839

Jingwei Liang is now a Ph.D.candidate in Signal and Imageprocessing at GREYC, ENSI-CAEN and Université de CaenBase Normandie. Before that,he received his master degreein applied mathematics in 2013fromShanghai Jiao TongUniver-sity, China, and bachelor degreein Electronic and InformationEngineering from Nanjing Uni-versity of Posts and Telecommu-nications. His current researchinterests mainly focus on convexoptimization and image process-ing.

Xiaoqun Zhang is an associatedprofessor at department of math-ematics and institute of naturalsciences in Shanghai Jiao TongUniversity, Shanghai, China. Shereceived her bachelor and mas-ter degrees from Wuhan Univer-sity, China in 2000 and 2003.She obtained her Ph.D. degree inapplied mathematics from Uni-versity of South Brittany, Francein 2006. From July 2007 to July2010, she worked in Universityof California, Los Angeles asa CAM assistant professor. Her

research interests include mathematical image processing, computervision, inverse problems and optimization.

123