approximation problems for digital image processing and …€¦ · a techniques of digital image...

Approximation Problems for DigitalImage Processing and Applications

Danilo Costarelli, Marco Seracini, and Gianluca Vinti(B)

Department of Mathematics and Computer Science, University of Perugia,1, Via Vanvitelli, 06123 Perugia, Italy

[email protected], [email protected],

[email protected]

Abstract. In this note, some approximation problems are discussedwith applications to reconstruction and to digital image processing. Wewill also show some applications to concrete problems in the medicaland engineering fields. Regarding the first, a procedure will be presented,based on approaches of approximation theory and on algorithms of digitalimage processing for the diagnosis of aneurysmal diseases; in particularwe discuss the extraction of the pervious lumen of the artery startingfrom CT image without contrast medium. As concerns the engineeringfield, thermographic images are analyzed for the study of thermal bridgesand for the structural and dynamic analysis of buildings, working there-fore in the field of energy analysis and seismic vulnerability of buildings,respectively.

1 Introduction

In the diagnosis of vascular pathologies, such as stenosis of main vessels oraneurysms, CT (computer tomography) images play a central role (see e.g.,[19]). In particular, in order to diagnose aneurysm of the aorta artery (see e.g.,[23]) it is necessary to identify inside the artery, the pervious lumen of the vessel,i.e., the zone in which the blood flows, and to quantify the rate of the possibleocclusion. In CT images, is not possible to distinguish the contours of the lumenfrom the rest of the vessel.

In general, to solve the above problem the vascular surgeons and the radiol-ogists resort to CT image with contrast medium, which makes the blood radio-opaque, and therefore recognizable with respect to other anatomical structures.However, for patients with severe kidney’s diseases or allergic problems, the intro-duction of contrast medium is not possible. For this reason, becomes crucial tohave techniques for the automatic segmentation of the lumen of the vessels in CTimages without contrast medium, since the gold-standard procedure to diagnoseaneurysms of the aorta artery is the CT.

We develop a procedure to accomplish the above task (see [12]); startingfrom CT images without contrast medium, of size n × n, we process the ROIusing the sampling Kantorovich (SK) algorithm. The latter algorithm provides

c© Springer International Publishing AG, part of Springer Nature 2018O. Gervasi et al. (Eds.): ICCSA 2018, LNCS 10960, pp. 19–31, 2018.https://doi.org/10.1007/978-3-319-95162-1_2

http://crossmark.crossref.org/dialog/?doi=10.1007/978-3-319-95162-1_2&domain=pdf

20 D. Costarelli et al.

a techniques of digital image processing which allows to reconstruct a givenimage with an increased resolution of nR × nR pixels, where R is a suitableinteger scaling factor. The above algorithm can be deduced from the theory ofsampling Kantorovich operators Sw [6,11,13–15,18], and their approximationresults in various setting; for other approximation results by means of otherkind of operators, see e.g., [1,7,16,17]. Then the above procedure is based onthe application of suitable algorithms of digital image processing, such as waveletdecomposition, normalization, equalization, and thresholding.

In the last years, the SK algorithm has been successfully applied in seismicand energetic engineering. For what concerns applications in seismic engineering,some models have been developed for studying the behaviors of buildings underseismic action starting from thermographic images. While, for what concernsenergetic engineering, the SK algorithm has been applied in order to derive anautomatic procedure for the detection of thermal bridges from thermographicimages and to study energy performance of buildings.

2 Multivariate Sampling Kantorovich Operators

The multivariate sampling Kantorovich operators are defined as follows:

(Sχwf)(x) :=

∑

k∈Zn

χ(wx − tk) ·[wn

∫

Rwk

f(u) du

](x ∈ R

n, w > 0),

where f : Rn → R is a locally integrable function, such that the above series isconvergent for every x ∈ R

n, see [14], and

Rwk :=

[k1w

,k1 + 1

w

]×

[k2w

,k2 + 1

w

]× . . . ×

[kn

w,kn + 1

w

],

k := (k1, . . . , kn) ∈ Zn, are the sets in which we consider the mean values of the

signal f . The SK algorithm for image reconstruction and enhancement consistsin an optimized implementation of the above sampling operators, with kernelsχ : Rn → R satisfying the following assumptions [13]:

(χ1) χ is summable on Rn, and bounded in a ball containing the origin of Rn;

(χ2) For every x ∈ Rn: ∑

k∈Zn

χ(x − k) = 1;

(χ3) For some β > 0, we assume that the discrete absolute moment of order βis finite, i.e.,

mβ(χ) := supu∈R

∑

k∈Zn

|χ(u − k)| · ‖u − k‖β < +∞.

Approximation Problems for Digital Image Processing and Applications 21

Fig. 1. The bivariate Fejer kernel.

where ‖ · ‖ denotes the Euclidean norm.We immediately provide some typical examples of kernels χ that can be used

according to the required assumptions. The most used method to construct mul-tivariate kernels is to consider the product of n kernels of one variable. Indeed,for instance, the definition of the multivariate Fejer kernel can be formulated asfollows:

Fn(x) =n∏

i=1

F (xi), x = (x1, . . . , xn) ∈ Rn, (1)

where F (x), x ∈ R, denotes the univariate Fejer kernel, which is defined by:

F (x) :=12

sinc2(x

2

), x ∈ R, (2)

where the well-known sinc-function is that defined as sin(πx)/πx, if x �= 0, and1 if x = 0 (see Fig. 1).

By the sinc-function it is possible to define another class of kernels, whichis widely used, i.e., the Jackson-type kernels. The multivariate expression of theJackson-type kernels, is the following:

J nk (x) :=

n∏

i=1

Jk(xi), x = (x1, . . . , xn) ∈ Rn, (3)

where Jk(x), x ∈ R are defined by:

Jk(x) := ck sinc2k( x

2kπα

), x ∈ R, (4)

with k ∈ N, α ≥ 1, and ck is a non-zero normalization coefficient, given by:

ck :=[∫

R

sinc2k( u

2kπα

)du

]−1

.

Figure 2 shows an example of the bivariate Jackson type kernel of first orderwith α = 1.


Fig. 2. The bivariate Jackson type kernel of first order with α = 1.

Since the sinc-function has unbounded support, we usually say that F (x)and J n

k (x) are not duration limited kernels. To know the duration of a kernelis important in order to implement the numerical evaluation of the operatorsstudied in this section. In fact, operators based upon kernels with unboundedduration, need to be truncated for the evaluation. For the latter reason, we alsoprovide examples of duration limited kernels. For instance, we can consider thewell-know central B-spline of order s, defined by:

Ms(x) :=1

(s − 1)!

s∑

i=0

(−1)i

(si

)(s

2+ x − i

)s−1

+, (5)

where the function (x)+ := max {x, 0} denotes the positive part of x ∈ R. Thecorresponding multivariate spline kernels are then defined by:

Mns (x) :=

n∏

i=1

Ms(xi), x = (x1, . . . , xn) ∈ Rn. (6)

Figure 3 shows an example of the bivariate B-spline type kernel of order 3.For the sampling Kantorovich operators, with kernels e.g., as above, the

following approximation results hold.

Theorem 2.1 ([13]). Let f : Rn → R be a given bounded signal. Then:

limw→+∞(Sχ

wf)(x) = f(x),


Fig. 3. The bivariate B-spline kernel of order 3.

at any point of continuity of f . Moreover, if f is uniformly continuous on Rn,

it turns out that:

limw→+∞ ‖Sχ

wf − f‖∞ = limw→+∞ sup

x∈Rn

|(Sχwf)(x) − f(x)| = 0.

Finally, if the signal f belongs to Lp(Rn), 1 ≤ p < +∞, we have:

limw→+∞ ‖Sχ

wf − f‖p = limw→+∞

(∫

Rn

|(Sχwf)(x) − f(x)|p dx

)1/p

= 0.

Increasing the sampling rate and choosing an appropriate kernel χ, it ispossible to enhance the images/signals f under consideration. For more detailsconcerning the SK algorithm, see e.g., [4,5,13].

3 Digital Image Processing by Sampling KantorovichOperators

Multivariate sampling Kantorovich operators, with kernels as above, are suitableto be used in order to process digital images, see [4,5,14,20].

A bi-dimensional digital gray scale image A (matrix) can be represented byusing a step function I belonging to Lp(R2), 1 ≤ p < +∞. I is defined by:

I(x, y) :=m∑

i=1

m∑

j=1

aij · 1ij(x, y) ((x, y) ∈ IR2),

where 1ij(x, y), i, j = 1, 2, . . . ,m, are the characteristics functions of the sets(i − 1, i] × (j − 1, j] (i.e., 1ij(x, y) = 1, for (x, y) ∈ (i − 1, i] × (j − 1, j]and 1ij(x, y) = 0 otherwise). The above function I(x, y) is defined in such a waythat, to every pixel (i, j) the corresponding gray level aij is associated. Now thefamily of bivariate sampling Kantorovich operators applied to the function I,(SwI)w>0 (for some kernel χ) approximates I pointwise at the continuity pointsand in Lp-sense, so it is possible to use it for the reconstruction and enhancementof the original image. To achieve a new image (matrix), SwI (for some w > 0)


is sampled with a fixed sampling rate. In particular, the reconstruction ofthe approximating images (matrices) is available considering different samplingrates; this is possible since we know SwI analytically in all its domain.

If the sampling rate is chosen higher than the original one, a new imagewith an increased resolution with respect to the original, is obtained. The aboveprocedure has been implemented by using MATLAB, in order to achieve analgorithm based on the multivariate sampling Kantorovich theory.

For the sake of completeness, the pseudo-code of the above algorithm isreported (see Table 1).

Practical reconstruction and enhancement of some biomedical and engineer-ing images that lead to interesting results from the “diagnostic” point of vieware presented in the following section.

4 Applications to Biomedical Images

Thanks to relatively recent developments in the field of medical imaging, a bigamount of data is nowday available for the diagnosis of different pathologies.

It is auspicable to apply the SK algorithms, together with other Digital ImageProcessing (D.I.P.) techniques, to support doctors in the diagnostic process. Inthis direction, a particular version of the SK algorithm has been specificallydeveloped for the segmentation of the pervious lumen of the aorta artery in CT(computer tomography) images without contrast medium.

For patients with severe kidney’s diseases or allergic problems the introduc-tion of contrast medium must be avoided. For this reason, the availability oftechniques for the automatic segmentation of the lumen of the vessels in CTimages without contrast medium becomes crucial, since the goal standard pro-cedure to diagnose aneurysms of the aorta artery is the CT.

The numerical procedure for the detection of the pervious area of the lumenof the aorta artery has the following crucial steps:

• enhancement of the original CT image without contrast medium;• application of a wavelet decomposition method in 5 levels and computation

of the residual component of the image;• application of normalization and equalization;• classification of the pixel’s histogram associated to each processed image and

computation of the threshold value for the segmentation of the pervious lumenof the aorta artery.

The above procedure, together with some numerical results, have beenschematically depicted in Fig. 4.

For the validation of the method, specific indexes of performance can beintroduced so that a comparison with a reference set of images is possible. Prac-tically, the reference consists of a corresponding acquisition performed in thesame patient after the introduction of the contrast medium. Due to the factthat the acquisitions are performed in different times and that the patient couldchange position during the exam (think for example to the natural expansion of


Table 1. Pseudo-code of the SK algorithm for image reconstruction and enhancement.

I

χ

I n × n

w > 0 R

•χ

•n · R × n · R

•I

•χ

k

χ(wx − k) ·[w2 ∫

Rwk

I(u) du]

x

n · R × n · R

the chest during normal breathing), it is difficult to individuate a strictly univo-cal reference. The image registration procedure, i.e., the necessity to superimposeimages coming from different CT sets, influences the estimation of the numericalresults.

With the aim to take into account of these problems in terms of quantitativeevaluation (see e.g., [21]), multiple measurements can be performed using:


Fig. 4. The schematic plot of the procedure for the segmentation of the pervious lumenof the aorta artery.

• the number of the pixels wrongly classified;• the number of pixels wrongly classified compared to the number of pixels

included in the Region Of Interest (R.O.I.);• the circularity of the extracted zone (see [22]);• the ratio of the circularity between the extracted zones;• the area of the extracted zone compared with the contrast medium reference;• the Hausdorff distances between the contours of the extracted zone compared

with the contrast medium reference;• the Hausdorff distance between the full sets of the extracted zone compared

with the contrast medium reference.

The Hausdorff distance measures the mismatch level between two sets ofpoints, A and B, considering the maximum value of the distance of A from Band viceversa. Let A = {a1, a2, . . . , an} and B = {b1, b2, . . . , bm} be two non-empty discrete subsets of a metric space (M, d); the Hausdorff distance dH isdefined as:

dH = max{d(A,B), d(B,A)}where:

d(A,B) = maxi

minj

|ai − bj |

d(B,A) = maxj

mini

|ai − bj |

with i ∈ [1, n], j ∈ [1,m].Figure 5 shows an example of Hausdorff distance between two discrete sets

of points.The main advantages of this approach relies in the potential possibility of

performing diagnosis concerning vascular pathologies even for those patients whoexhibit severe kidney’s diseases or allergic problems, for which CT images withcontrast medium cannot be used.


Fig. 5. Hausdorff distance dH between two discrete sets of points, A and B: on the left,in green, dH between the full sets. The white zone represents the intersection betweenthe two sets of pixels, the red zone contains the points belonging only to A, the bluezone the ones belonging only to B. The length of the green line is dH = 7.616. On theright dH between the borders (dH = 26.24). Both images are of size 111 × 111 pixels.(Color figure online)

5 Applications to Engineering Problems

SK algorithms can be successfully applied in civil engineering, from one hand toimprove the thermal bridges quantitative assessment by infrared thermography,from the other hand to support non invasive and non destructive structuralanalysis.

In the first contest, the intervention on the existing building envelope ther-mal insulation is the main and effective solution in order to achieve a significantreduction of the building stock energy needs. The infrared technique [8] is themethodology of the energy diagnosis aimed to identify qualitatively the princi-pal causes of energy losses: the presence of thermal bridges. Those weak parts ofthe building envelope in terms of heat transfer result not easy to treat with anenergy efficiency intervention, while they are gaining importance in the build-ings total energy dispersion, as the level of insulation of opaque and transpar-ent materials is continuously increasing. It is generally possible to evaluate theenergy dispersions through these zones with a deep knowledge of the materialsand the geometry using a numerical method. The analysis of surface temper-atures of the undisturbed wall and of the zone with thermal bridge, allows todefine the Incidence Factor of the thermal Bridge (Itb), see [3]. This parameteris strongly affected by the thermographic image accuracy, therefore, SK algo-rithm enhances the image resolution and the consequent accuracy of the energylosses assessment. An experimental campaign in a controlled environment (hotbox apparatus) has been conducted on three typologies of thermal bridge, firstlyperforming the thermographic survey and then applying the enhancement SK


Fig. 6. On the left, a thermographic image depicting a beam-pillar-joint 2D thermalbridge, while on the right we have in red the profile of the thermal bridge extracted bythe above method after the application of the SK algorithm. (Color figure online)

algorithm to the infrared images in order to compare the Itb and the linear ther-mal transmittance ψ values. Results show that the proposed methodology couldbring to an accuracy improvement of the total buildings envelope energy lossesevaluated by quantitative infrared thermography.

SK algorithm allows the implementation of a further process applicable tothe images, in order to extract the physical boundaries of the hidden materialscausing the thermal bridge, so revealing itself as a useful tool to identify exactlythe suitable points of intervention for the thermal bridge correction. The appli-cation of the imaging process on the quantitative infrared thermography is aninnovative approach that makes more accurate the evaluation of the actual heatloss of highly insulating buildings and reaching a higher detail on the detectionand treating of thermal bridges.

Concerning the application of the SK algorithm to the energetic engineering,in [4,5] a segmentation method has been developed and applied in order to detectthe shape of thermal bridges of the building envelope from thermographic images(see Fig. 6). Generally speaking, a thermal bridge is characterized by a significanttemperature gradient compared with the average value of the surrounding area(undisturbed zones). The temperature gradient is high when a change of materialdue to the geometrical contours of thermal bridge structure exists (e.g. contour ofa pillar or a beam). Analysing the histogram associated to the infrared thermalbridge image, which can be interpreted as the distribution of probability oftemperature occurring on the thermal map, two peaks (P1 and P2) representingthe homogeneous temperature areas can be identified: one of the undisturbedarea of the wall and the other one of the thermal bridge. Between these two peaksit is possible to find a minimum value which, in view of the above probabilisticinterpretation of the data, can be associated to the minimum error due to thewrong classification of pixels located inside the thermal wall but classified asexternal, and viceversa. The temperature Tm, which corresponds to the valuethat minimizes the above misclassification error, identifies the suitable thresholdvalue to segment the thermal bridge shape from the background. (see Fig. 7).


Fig. 7. Generic distribution of probability for the temperature in a thermal bridge.

First of all, we enhance the original thermographic image by the SK algorithmand then we determine the threshold value by means of a sort of probabilisticmethod (see [4,5] again) based on the analysis of the pixel’s histogram of thethermographic data. Moreover, it also allows to determine the heat losses of thebuildings, using a suitable incidence factor of thermal bridge, introduced in [2,3].It has been proved that the application of the SK algorithm is crucial in orderto have an accurate estimate of the heat losses of the buildings.

Finally, we recall that in [9,10] the SK algorithm has been used in order toenhance the thermographic images of walls, to improve the texture procedure,which allows the automatic separation of mortar and bricks in the masonries(see Fig. 8), to extract the elastic parameters and finally, to study the dynamicbehavior of buildings under seismic action. Also here, the application of SKalgorithm revealed to be crucial.

Fig. 8. On the left, the texture of a wall starting from a thermographic image, while onthe right we have the texture after the application of the SK algorithm to the originalthermographic image.


Acknowledgments. The authors are members of the Gruppo Nazionale per l’AnalisiMatematica, la Probabilita e le loro Applicazioni (GNAMPA) of the Istituto Nazionaledi Alta Matematica (INdAM).

The authors are partially supported by the “Department of Mathematics and Com-puter Science” of the University of Perugia (Italy). Finally, the first two authors ofthe paper have been partially supported within the 2017 GNAMPA-INdAM Project“Approssimazione con operatori discreti e problemi di minimo per funzionali del calcolodelle variazioni con applicazioni all’imaging”.

The research related to this paper is part of the project: “Metodi di Approssi-mazione e Applicazioni” funded by the basic research fund, 2015 of the University ofPerugia.

References

1. Angeloni, L., Vinti, G.: Approximation with respect to Goffman-Serrin variationby means of non-convolution type integral operators. Numer. Funct. Anal. Optim.31, 519–548 (2010)

2. Asdrubali, F., Baldinelli, G.: Thermal transmittance measurements with the hotbox method: calibration, experimental procedures, and uncertainty analyses ofthree different approaches. Energy Build. 43, 1618–1626 (2011)

3. Asdrubali, F., Baldinelli, G., Bianchi, F.: A quantitative methodology to evaluatethermal bridges in buildings. Appl. Energy 97, 365–373 (2012)

4. Asdrubali, F., Baldinelli, G., Bianchi, F., Costarelli, D., Evangelisti, L., Rotili,A., Seracini, M., Vinti, G.: A model for the improvement of thermal bridgesquantitative assessment by infrared thermography. Appl. Energy 211, 854–864(2018)

5. Asdrubali, F., Baldinelli, G., Bianchi, F., Costarelli, D., Rotili, A., Seracini, M.,Vinti, G.: Detection of thermal bridges from thermographic images by means ofimage processing approximation algorithms. Appl. Math. Comput. 317, 160–171(2018)

6. Bardaro, C., Butzer, P.L., Stens, R.L., Vinti, G.: Kantorovich-type generalizedsampling series in the setting of Orlicz spaces. Sampl. Theory Signal Image Process.6(1), 29–52 (2007)

7. Bardaro, C., Karsli, H., Vinti, G.: Nonlinear integral operators with homogeneouskernels: pointwise approximation theorems. Appl. Anal. 90(3–4), 463–474 (2011)

8. Bianchi, F., Pisello, A., Baldinelli, G., Asdrubali, F.: Infrared thermography assess-ment of thermal bridges in building envelope: experimental validation in a testroom setup. Sustainability 6(10), 7107–7120 (2014)

9. Cluni, F., Costarelli, D., Minotti, A.M., Vinti, G.: Enhancement of thermographicimages as tool for structural analysis in earthquake engineering. NDT E Int. 70,60–72 (2015)

10. Cluni, F., Costarelli, D., Minotti, A.M., Vinti, G.: Applications of sampling Kan-torovich operators to thermographic images for seismic engineering. J. Comput.Anal. Appl. 19(4), 602–617 (2015)

11. Costarelli, D., Minotti, A.M., Vinti, G.: Approximation of discontinuous signals bysampling Kantorovich series. J. Math. Anal. Appl. 450(2), 1083–1103 (2017)

12. Costarelli, D., Seracini, M., Vinti, G.: A segmentation procedure of the aorta arteryfrom CT images without contrast medium (2018, submitted)


13. Costarelli, D., Vinti, G.: Approximation by multivariate generalized sampling Kan-torovich operators in the setting of Orlicz spaces. Boll. U.M.I. 9(IV), 445–468(2011). Special volume dedicated to Prof. Giovanni Prodi

14. Costarelli, D., Vinti, G.: Approximation by nonlinear multivariate sampling Kan-torovich type operators and applications to image processing. Numer. Funct. Anal.Optim. 34(8), 819–844 (2013)

15. Costarelli, D., Vinti, G.: Degree of approximation for nonlinear multivariate sam-pling Kantorovich operators on some functions spaces. Numer. Funct. Anal. Optim.36(8), 964–990 (2015)

16. Costarelli, D., Vinti, G.: Approximation by max-product neural network operatorsof Kantorovich type. Results Math. 69(3), 505–519 (2016)

17. Costarelli, D., Vinti, G.: Max-product neural network and quasi-interpolation oper-ators activated by sigmoidal functions. J. Approx. Theory 209, 1–22 (2016)

18. D. Costarelli, G. Vinti.: An inverse theorem of approximation by sampling Kan-torovich series. In: Proceedings of the Edinburgh Mathematical Society (2018, inPrint). arXiv:1801.08695

19. Deserno, T.M.: Biomedical Image Processing. Biological and Medical Physics,Biomedical Engineering. Springer, Heidelberg (2011). https://doi.org/10.1007/978-3-642-15816-2

20. Kaijser, T.: Computing the Kantorovich distance for images. J. Math. Imag. Vis.9, 173–191 (1998)

21. Meijering, E.H.W., Niessen, W.J., Viergever, M.A.: Quantitative evaluation ofconvolution-based methods for medical image interpolation. Med. Image Anal. 5,111–126 (2001)

22. Olson, E.: Particle shape factors and their use in image analysis part 1: theory. J.GXP Compliance 15(3), 85–96 (2011)

23. Takayama, T., Yamanouchi, D.: Aneurysmal disease: the abdominal aorta. Surg.Clin. 93(4), 877–891 (2013)

http://arxiv.org/abs/1801.08695

https://doi.org/10.1007/978-3-642-15816-2

https://doi.org/10.1007/978-3-642-15816-2

approximation problems for digital image processing and …€¦ · a techniques of digital image...

Documents