exponential inequalities in calibration problems with gaussians errors

This article was downloaded by: [Stony Brook University]On: 28 October 2014, At: 14:13Publisher: Taylor & FrancisInforma Ltd Registered in England and Wales Registered Number: 1072954 Registered office: Mortimer House,37-41 Mortimer Street, London W1T 3JH, UK

Communications in Statistics - Theory and MethodsPublication details, including instructions for authors and subscription information:http://www.tandfonline.com/loi/lsta20

Exponential Inequalities in Calibration Problems withGaussians ErrorsAbdelnasser Dahmani a & Karima Belaide aa Laboratory of Applied Mathematics, Faculty of Sciences Exactes , University A. MIRABejaia , AlgeriaPublished online: 20 Aug 2013.

To cite this article: Abdelnasser Dahmani & Karima Belaide (2013) Exponential Inequalities in Calibration Problems withGaussians Errors, Communications in Statistics - Theory and Methods, 42:19, 3596-3607, DOI: 10.1080/03610926.2011.635257

To link to this article: http://dx.doi.org/10.1080/03610926.2011.635257

PLEASE SCROLL DOWN FOR ARTICLE

Taylor & Francis makes every effort to ensure the accuracy of all the information (the “Content”) containedin the publications on our platform. However, Taylor & Francis, our agents, and our licensors make norepresentations or warranties whatsoever as to the accuracy, completeness, or suitability for any purpose of theContent. Any opinions and views expressed in this publication are the opinions and views of the authors, andare not the views of or endorsed by Taylor & Francis. The accuracy of the Content should not be relied upon andshould be independently verified with primary sources of information. Taylor and Francis shall not be liable forany losses, actions, claims, proceedings, demands, costs, expenses, damages, and other liabilities whatsoeveror howsoever caused arising directly or indirectly in connection with, in relation to or arising out of the use ofthe Content.

This article may be used for research, teaching, and private study purposes. Any substantial or systematicreproduction, redistribution, reselling, loan, sub-licensing, systematic supply, or distribution in anyform to anyone is expressly forbidden. Terms & Conditions of access and use can be found at http://www.tandfonline.com/page/terms-and-conditions

http://www.tandfonline.com/loi/lsta20

http://www.tandfonline.com/action/showCitFormats?doi=10.1080/03610926.2011.635257

http://dx.doi.org/10.1080/03610926.2011.635257

http://www.tandfonline.com/page/terms-and-conditions

http://www.tandfonline.com/page/terms-and-conditions

Communications in Statistics—Theory and Methods, 42: 3596–3607, 2013Copyright © Taylor & Francis Group, LLCISSN: 0361-0926 print/1532-415X onlineDOI: 10.1080/03610926.2011.635257

Exponential Inequalities in CalibrationProblems with Gaussians Errors

ABDELNASSER DAHMANI AND KARIMA BELAIDE

Laboratory of Applied Mathematics, Faculty of Sciences Exactes,University A. MIRA Bejaia, Algeria

This article establishes exponential inequalities for the probability of the distancebetween the approximated solution and the exact one for a calibration problem.We consider an operator equation taking the following form Ax = u, where A is acompact operator. We assume that the right-hand side u is not known exactly, andwe have only its approximation by using the process of gaussian error.

Keywords Calibration; Cramer condition; Linear operator.

Mathematics Subject Classification 65N21; 53C38.

1. Introduction

Calibration, also called inverse regression, is a classical problem that appears oftenin a regression setup under fixed design. In simple linear regression, it is assumedthat two variables x and y are linearly related with unknown intercept and slopeparameters. In particular, the exogenous variable x is assumed to be preciselymeasurable and the endogenous variable y is assumed to be a random variabledepending on x via a linear function. The linear calibration problem bears aresemblance to simple linear regression since it is assumed that the variables x andy are linearly related. However, in calibration, interest centers on estimating anunknown value of x which corresponds to an observed value of y. A review ofstatistical calibration is given by Osborne (1991). Statistical calibration involves datacollected in two stages. In the first stage, several values of an endogenous variableare observed, each corresponding to a known value of an exogenous variable. In thesecond stage, one or more values of the endogenous variable are observed whichcorrespond to an unknown value of the exogenous variable.

A large variety of problems arising from different areas of applied sciencescan be often regarded mathematically as an equation with an operator taking the

Received June 8, 2011; Accepted October 19, 2011Address correspondence to Karima Belaide, Laboratory of Applied Mathematics,

Faculty of Sciences Exactes, University A. MIRA Bejaia, Algeria; E-mail: [email protected]

3596

Dow

nloa

ded

by [

Ston

y B

rook

Uni

vers

ity]

at 1

4:13

28

Oct

ober

201

4

Processes with Gaussian Error 3597

following form:

Ax = u� (1)

where x is unknown solution in Hilbert space �. u ∈ � and A is a compact operatorin ��.

In this article, we establish an exponential inequalities for the probabilityof the distance between the approximated solution and the exact solution of alinear operator equation with an approximated second member. These inequalitiesyields the almost complete convergence and the convergence rate of approximatesolution.

1.1. Notation and Hypothesis

Let �� be a probabilistic space, A � � → � a linear compact operator, andits inverse A−1 is defined on the set Im�A�. Let us denote by ue the unknown exactvalue of the right hand side of equation (1), and xe ∈ � the unique solution suchthat Axe = ue.

The right-hand side ue of Eq. (1) can be approximated by using the sampleu1� � � � � un. This class of problems is called ill-posed problems (see Alber andRyazantseva, 2006; Bauer and Munk, 2007; Engl et al., 1996; Kress, 1999; Ramm,2005a,b; Tautenhahn, 2002; Tikhonov and Arsenin, 1977). Then the goal is to findan approximate and stable solution for (1). First, we consider an estimator of u byusing a process of gaussian error.

2. Setting of the Problem

Consider Eq. (1), and assume that the exact solution xe of Eq. (1) is an elementof a compact set DR = �x ∈ � � �x� ≤ R�R > 0�. In practice, the available data arenot known exactly, and we have only its approximation (Bissantz et al., 2004, 2007;Cardot, 2002; Dahmani and Bouhmila, 2006; Dahmani et al., 2009; Kaipio andSomersalo, 2004; Tarantola, 2005).

To simulate the process of the error measurments on ue, we set

ui = ue + i� i = 1� 2� � � � � n (2)

where �i�i≥1 is a sequence of independent Gaussian, zero mean, and identicallydistributed random element defined on �� into the Hilbert space �, satisfying��i�2 = 2 < +�. Note that the Gaussian process satisfies the following Cramercondition:

� �i�m ≤ m!2� �1�2 Lm−2� m ∈ �� m ≥ 2� (3)

where L is a positive constant and � is the mean operator.To estimate the right-hand side ue� we use a sample u1� � � � � un and the strong

law of large numbers gives the empirical mean un = 1n

∑ni=1 ui as natural a exhaustive

Dow

nloa

ded

by [

Ston

y B

rook

Uni

vers

ity]

at 1

4:13

28

Oct

ober

201

4

3598 Dahmani and Belaide

estimate of ue. Therefore, the problem becomes, solving the following equation:

Ax = un (4)

Denote by A∗ the adjoint operator of A and consider the equation

A∗Ax = A∗un� (5)

Let us denote by x�n the unique regularised solution

x�n = �A∗A+ �I�−1A∗un�

Proposition 2.1. Let > 0. For all � > 0, if

�A∗�un − ue�� ≤

then

�x�n − xe� ≤ ��

Proof. It suffies to fulfill the following inclusion

�� ∈ � � �A∗�un ��− ue �� ≤ � ⊂ �� ∈ � � �x�n ��− xe �� ≤ ��

Denote by w� �DR� the modulus of continuity of operator �A∗A�−1, x�e is such us�A∗A+ �I�x�e = A∗ue. For the sake of symplicity, let us ignore the argument �.

�A∗A+ �I�x�n = A∗un

�A∗A+ �I�x�e = A∗ue

then,

�A∗A+ �I� �x�n − x�e� = A∗ �un − ue�

On the other hand,

�A∗ �un − ue�� ≥ � �x�n − x�e� − �A∗A �x�n − x�e��≥ � �x�n − xe� − � �

3− w� �DR��

This implies that

�x�n − xe� ≤

� + �′ + w� �DR�

� �

For a suitable choice of and R, say = � �3 and R = � �

6 , then w� �DR� ≤ � �3 .

Consequently,

�x�n − xe� ≤ ��

Dow

nloa

ded

by [

Ston

y B

rook

Uni

vers

ity]

at 1

4:13

28

Oct

ober

201

4


In the following theorem, we show that the considered estimator x�n convergesin probability to the exact solution xe of Eq. (1).

Theorem 2.1. Let A be a linear compact operator and xe be the unique exact solutionof Eq. (1) in the compact set DR. Let u1� u2� � � � � un be a sequence of i.i.d. randomelements with mean ue and satisfying

ui = ue + i� i = 1� 2� � � � � n

where �i�i≥1 is a sequence of independent Gaussian, zero mean, and identicallydistributed random element defined on �� , then ∀� > 0

� ��x�n − xe� > �� ≤ exp

− n�2�2

182�A�2[1−

√SpA

∗A��3

√n

]2 (6)

where SpA∗A is a spectrum of A∗A.

Proof.

��x�n − xe� > �� ≤ ��A∗�un − ue�� > �

≤ �

{⟨1√nA∗

n∑i=1

i�1√nA∗

n∑i=1

i

⟩> n 2

}

Put

2 = r2

n�

then,

� ��x�n − xe� > �� ≤ �

{⟨1√nA∗

n∑i=1

i�1√nA∗

n∑i=1

i

⟩> r2

}

Using Chernoff inequality, for t ∈ �+ we obtain

� ��x�n − xe� > �� ≤ exp(−tr2

)�

{exp

(t

⟨1√nA∗

n∑i=1

i�1√nA∗

n∑i=1

i

⟩)}

≤ exp(−tr2

)�

{exp

(12

⟨√2tnA∗

n∑i=1

i�

√2tnA∗

n∑i=1

i

⟩)}�

By using the caracteristic function of gaussian random variables on � Hilbert space,it is easy to show the following representation:

exp(12 A∗Af� f�

)=

∫�exp � f� h�� dh� f ∈ ��

Dow

nloa

ded

by [

Ston

y B

rook

Uni

vers

ity]

at 1

4:13

28

Oct

ober

201

4


where � is a Gaussian distribution in � and A∗A is the correlation operator

� ��x�n − xe� > �� ≤ exp(−tr2

)�

(∫�exp

⟨√2tn

n∑k=1

k� h

⟩� �dh�

)

≤ exp(−tr2

) (∫�

n∏k=1

[1+ ∑

m≥2

1m!

(√2tn

)m

� �k�m �h�m]� �dh�

)�

The sequence of Gaussian random variables �i� sytisfying the Cramer condition:

� ��x�n − xe� > ��

≤ exp(−tr2

) ∫�

n∏k=1

(1+ ∑

m≥2

1m!

(√2tn

)m

�h�m m!2� �1�2 Lm−2

)� �dh�

≤ exp(−tr2

) ∫�

n∏k=1

1+ 1

2

(√2tn

)2

�h�2 � �1�2∑m≥0

(√2tn�h�L

)m � �dh�

� ��x�n − xe� > �� ≤ exp(−tr2

) ∫�

n∏k=1

1+ t� �1�2 �h�2

n

1

1−√

2tnL �h�

� �dh� �

For a suitable choice of t, such as the right side of the above inequality, is a termof convergent geometric sequence; let

0 < t <

(1− 1

2�h�)2n(√

2�h�L)2 �

We obtain 1+ t� �k�2 �h�2

n

1

1−√

2tnL �h�

≤ 1+ 2t� �k�2 �h�

n

≤ exp

(2t� �1�2 �h�

n

)�

The precedente inequality becomes

� ��x�n − xe� > �� ≤ exp(−tr2

) ∫�exp

(2t� �1�2 �h�

)� �dh� �

The Gaussian variable 1 can be written using the hilbertian basis �ek� as follows:

1 =∑k≥1

1� ek� ek =∑k≥1

√�k 1� ek�√

�kek�

Dow

nloa

ded

by [

Ston

y B

rook

Uni

vers

ity]

at 1

4:13

28

Oct

ober

201

4


�k ∈ �+ is such as the sequence of Gaussian random variables(1k

) = ( 1�ek�√

�k

)are

centered and reduced, then

1 =+�∑k=1

√�k

1kek

and

�1 = Ae1� Ae1� = �Ae1�2 = �A�2�

Hence, we have

� ��x�n − xe� > �� ≤ exp(−tr2

) ∫�exp

(2t �h�

+�∑k=1

�k�∣∣1k∣∣2

)� �dh�

≤ exp(−tr2

) (1+ 2t� �1�

∑k≥1

�k�∣∣1k∣∣2

+ 12

[2t

∑k≥1

�k�∣∣1k∣∣2

]2

� �1�2∑m≥0

[2t

∑k≥1

�k�∣∣1k∣∣2 L

]m �

For a suitable choice of t, such as the right side of the above inequality, is a termof convergent geometric sequences:

� ��x�n − xe� > �� ≤ exp(−tr2

) (1+ 2tE �1�

∑k≥1

�kE∣∣1k∣∣2

+ 12

[2t

∑k≥1

�kE∣∣1k∣∣2

]2

E ��1�21

1− 2tL∑

k≥1 �kE∣∣1k∣∣2

≤ exp(−tr2

) 1+ 2tE �1�

1−√2tL

∑k≥1�kE

∣∣1k∣∣2∑k≥1

�kE∣∣1k∣∣2

+ 12

2t

∑k≥1 �kE

∣∣1k∣∣21−

√2tL

∑k≥1�kE

∣∣1k∣∣2

2

� �1�2 �

Let

0 < t <�� 1� − 2�2

2L �� 1��2∑

k≥1 �k�∣∣1k∣∣2

� ��x�n − xe� > �� ≤ exp(−tr2

) (1+ t �� 1��2

∑k≥1

�k�∣∣1k∣∣2

Dow

nloa

ded

by [

Ston

y B

rook

Uni

vers

ity]

at 1

4:13

28

Oct

ober

201

4


+ 12

[t� �1�

∑k≥1

�k�∣∣1k∣∣2

]2

� �1�2

≤ exp(−tr2

)�

[exp

(t �1�2

∑k≥1

�k�∣∣1k∣∣2

)]�

We put � �1�2 =∑

k≥1 �k�∣∣1k∣∣2 = 2, and we know

�(exp

[t2 �1�2

])≤ ∏

k≥1

1√1− 2t2�1

≤ exp

(t2

1− 2t2�1

∑k≥1

�k

)�

Consequently,

� ��x�n − xe� > �� ≤ exp(−tr2

)exp

(t2

1− 2t2�1

∑k≥1

�k

)

≤ exp(−tr2

)exp

(t2

1− 2t2�A�2∑k≥1

�k

)�

Choice t maximizes the function f �t� = exp(−tr2 + t2

1−2t2�A�2∑

k≥1 �k

). The right

side of the last inequality is maximized by

t∗ = 122�A�2

[1−

√SpA

∗A

r

]�

where∑k≥1

�k = SpA∗A� and r2 = n�2�2

9 .

Then,

� ��x�n − xe� > �� ≤ exp

− n�2�2

182�A�2[1−

√SpA

∗A��3

√n

]2

�

Corollary 2.1. Under the assumptions of Theorem 2.1, the sequence �x�n� convergesalmost completely (a.co.) to the exact solution xe of Eq. (1).

It is easy to show that there exists a positive constant K such that, for asufficiently small �, the inequality (6) can be rewritten as follows:

� ��x�n − xe� > �� ≤ e−Kn�2 � (7)

Since the numerical series of general term e−Kn�2 is convergent, we then obtain thefollowing result for all positive �:

�∑n=1

� ��x�n − xe� > �� < +�� (8)

Dow

nloa

ded

by [

Ston

y B

rook

Uni

vers

ity]

at 1

4:13

28

Oct

ober

201

4


Corollary 2.2. Under the assumptions of Theorem 2.1, we have

x�n − xe = O

(√log nn

)p�co� (9)

By setting, � = �0√n−1 log n in (7), we obtain the following inequality for all �0:

�{�x�n − xe� > �0

√n−1 log n

}≤ n−K�0 � (10)

For a suitable choice of �0, say �0 = 1+dk2

� d > 0, the right side of the above inequalityis a term of a convergent series. This yields (9).

Corollary 2.3. Under the assumptions of Theorem 2.1, for a given significancethreshold �, it exists an integer n� for which

�{∥∥∥x�n� − xe

∥∥∥ ≤ �}≥ 1− �� (11)

i.e., the exact solution xe of Eq. (1) belongs to the closed ball with center x�n� and radius� with probability greater than or equal to 1− �.

Indeed, we have

limn→+� exp

(−Kn�2) = 0� (12)

i.e., it exists an integer n� for which we have

n ≥ n� �⇒ exp(−Kn�2

) ≤ �� (13)

Then, (11) follows from (6), (7), and (13).

3. Simulation

We have conducted a simulation study to show that the estimtor x�n is consistent,and the ampirical mean u is a good choice of the right side hand of Eq. (1).

The details of the study simulation are as follows.The Hilbert space � is L2 ��0� 1�� space. u ∈ �, a linear compact operator

A ∈ � �� is given by the Fredholm equation of first espece (Kress, 1999) is givenby, for � ∈ �

A� �s� =1∫

0

K �s� t� � �t� dt�

K is Kernel function given by

K �s� t� ={�1− s� t� 0 ≤ t ≤ s�

s �1− t� � s ≤ t ≤ 1�

Dow

nloa

ded

by [

Ston

y B

rook

Uni

vers

ity]

at 1

4:13

28

Oct

ober

201

4


Table 1Mean squar error between x�n and xex = �

n\� 0,1 0,01 0,001 0,0001

10 16,3940 15,7235 13,9438 6,9254100 1,6385 1,5719 1,3944 0,69251000 0,1638 0,1572 0,1394 0,0693

We propose comparing the exact solution � �s� of

A� �s� = sin3 ��s�

to the considered estimator

x�n = �A∗A+ �I�−1 �A∗u� �

where

ui �s� = sin3 ��s�+ �i

�1� � � � �n is a sequence of random, independent, zero mean, and identicallydistributed N �0� 1� variables.

The inverse of operator �A∗A+ �I� is given by Krasnov et al. (1976), such usthe image of both eiger element �i by �A∗A+ �I� is given by,

�A∗A+ �I�−1 ��i� =1

�+ �i�i�

Figure 1. Mean square error is 13.9438. (color figure available online)

Dow

nloa

ded

by [

Ston

y B

rook

Uni

vers

ity]

at 1

4:13

28

Oct

ober

201

4



�i is eigervalue of operator A∗A associated to eigervector �i. � is sufficiently smallsuch as the operator �A∗A+ �I� is invertible. For differents sample sizes n andparameter �, we give the mean square error between x�n and �. The simulation resultsare reported in Table 1.

Figures 1, 2, 3, and 4 present plots of the exact solution and its approximationfor n = 10 and 100. These plots once again underscore the optimality of x�n. Asmight be expected, the results are much better when sample size n is large enough,for example, for n = 1� 000 the mean square error is 0�1638 for larger � = 0� 1. While


Dow

nloa

ded

by [

Ston

y B

rook

Uni

vers

ity]

at 1

4:13

28

Oct

ober

201

4



a small size n = 10 and very smal � = 0�0001 the mean squar error is 6�9254. Theconvergence of approximate solution is quite satisfactory.

References

Alber, Y., Ryazantseva, I. (2006). Nonlinear Ill-Posed Problem of Monotone Type. Dordrecht,the Netherlands: Springer.

Bauer, F., Munk, A. (2007). Optimal regularization for ill-posed problems in metric space.J. Inverse Ill-Posed Probl. 15:37–148.

Bissantz, N., Hohage, T., Munk, A. (2004). Consistency and rate of convergence of nonlinear Tikhonov regularization with random noise. Inverse Probl. 20:1773–1789.

Bissantz, N., Hohage, T., Munk, A. (2007). Convergence rates of general regularizationmethodes for statistical inverse problems and applications. SIAM J. Num. Anal.45:2610–2636.

Cardot, H. (2002). Spatially adaptive splines for statistical linear inverse problems.J. Multivariate Anal. 81:100–119.

Dahmani, A., Bouhmila, F. (2006). Consistancy of landweber algorithm in ill-posed problemdata. Competes Rendus de l’Academie des Sciences. Serie I (Mahtématiques) 343:87–491.

Dahmani, A., Ait Saidi, A., Bouhmila, F., Aissani, M. (2009). Consistancy of Tikhonov’sregularization in ill-posed problem with random data. Statist. Probab. Lett. 79:722–727.

Engl, H. W., Hanke, M., Neubauer, A. (1996). Regularization of Inverse Problems. Dorrecht:Kluwer.

Kaipio, J., Somersalo, E. (2004). Computational and Statistical Methods for Inverse Problems.New York: Springer Verlag.

Krasnov, M., Kisselev, A., Makarenko, G. (1976). Equations Intégrales. Problèmes etexercices. [Edition Mir Moscow].

Kress, R. (1999). Linear Integral Equation. 2nd ed. Applied Mathematical Sciences. Vol. 82.Springer-Verlag.

Osborne, C. (1991). Statistical calibration: a review. Int. Statist. Rev. 59:309–336.Ramm, A. G. (2005a). Dynamical systems method for ill-posed problem equations with

monotone operators. Communicationin Nonlinear Science and Numerecal Simulation10:35–940.

Dow

nloa

ded

by [

Ston

y B

rook

Uni

vers

ity]

at 1

4:13

28

Oct

ober

201

4


Ramm, A. G. (2005b). Inverse Problems. Mathematical and Analytical Techniques WithApplications to Engineering. Boston: Springer.

Tarantola, A. (2005). Inverse Problem Theory and Methods for Model Parameter Estimation.Philadelphia: SIAM.

Tautenhahn, U. (2002). On the method of Lavrentiev regularisation for nolinear ill-posedproblems. Inverse Probl. 18(1):191–207.

Tikhonov, A. N., Arsenin, V. A. (1977). Solutions of Ill-Posed Problems. Washington:Winston & Sons.

Dow

nloa

ded

by [

Ston

y B

rook

Uni

vers

ity]

at 1

4:13

28

Oct

ober

201

4

exponential inequalities in calibration problems with gaussians errors

Documents