aging prediction of ch samples based on surface microgeometry · i.m. ciortan, g. marchioro, c....

EUROGRAPHICS Workshop on Graphics and Cultural Heritage (2018)R Sablatnig and M Wimmer (Editors)

Aging Prediction of Cultural Heritage Samples Based on SurfaceMicrogeometry

I M Ciortan1 G Marchioro1 C Daffara1 R Pintus2 E Gobbetti2 and A Giachetti1

1Computer Science Department University of Verona Italy2Visual Computing Group Center for Advanced Studies Research and Development in Sardinia Italy

AbstractA critical and challenging aspect for the study of Cultural Heritage (CH) assets is related to the characterization of the materialsthat compose them and to the variation of these materials with time In this paper we exploit a realistic dataset of artificiallyaged metallic samples treated with different coatings commonly used for artworksrsquo protection in order to evaluate differentapproaches to extract material features from high-resolution depth maps In particular we estimated on microprofilometricsurface acquisitions of the samples performed at different aging steps standard roughness descriptors used in materials scienceas well as classical and recent image texture descriptors We analyzed the ability of the features to discriminate different agingsteps and performed supervised classification tests showing the feasibility of a texture-based aging analysis and the effectivenessof coatings in reducing the surfacesrsquo change with time

CCS ConceptsbullComputing methodologies rarr Machine learning approaches Neural networks bullApplied computing rarr Arts and humani-ties bullGeneral and reference rarr Metrics

1 Introduction

An important visual computing task related to Cultural Heritage(CH) objectsrsquo study and conservation is the characterization of sur-faces at microscopic level and the analysis of how their shapeschange over time

Microprofilometric probes [SB00] can reconstruct surfaces witha micrometric resolution and the characteristics of the measuredsurface patterns can give important information about the physicalchemical properties and appearance of the material Surface metrol-ogy provides well established tools to quantitatively analyze rough-ness patterns by using methodologies and parameters describedin ISO standards related to profile analysis and areal characteri-zation [Bla13]

These measurements have been seldom evaluated in comparativeworks with the purpose of classifying specific materials of interestThis task however is particularly interesting in the CH domainwhere the analysis of material properties and their evolution dueto both time and different environmental conditions is particularlyuseful for conservators

The problem is non-trivial as many factors influence the texturepattern of the surface depth at different scales

In this paper we present results of tests aimed at using surfacemetrology tools as well as different image processing techniques

to characterize and classify according to aging samples of bronzeand silver alloys typically used in artworks

Our testbed consists of Cultural Heritage material samples arti-ficially aged provided by an Italian art conservation and restorationlaboratory

2 Related Work

Several descriptors are already used in material science to charac-terize surface roughness Profile parameters are usually exploitedin material science to characterize roughness from linear measure-ments To deal with surface capture aerial parameters are describedin the ISO25178 standard to characterize materials [Bla13] or toanalyze the relevance of parameters during surface-changing pro-cesses [DKB14 VGBEMlowast10]

In the image processing domain there are many other methodscommonly used to characterize statistical properties of a signal reg-ularly sampled in 2D ie the texture pattern [TJlowast93] In principleall those descriptors can be applied as well for depth map descrip-tion A survey of specific issues of texture analysis methods for ma-terial science is presented in [Bun13] Examples of different texturecharacterization methods not included in surface metrology proto-cols are second order statistics like co-occurrence matrices [Har79]describing joint variations of gray-scale at selected distances Localbinary patterns [GZZ10] are other popular visual descriptors mea-suring robust statistics of pixel neighborhoods Another classical

ccopy 2018 The Author(s)Eurographics Proceedings ccopy 2018 The Eurographics Association

DOI 102312gch20181352 httpsdiglibegorghttpswwwegorg

IM Ciortan G Marchioro C Daffara R Pintus E Gobbettiamp A Giachetti Aging Prediction of CH Samples Based on Surface Microgeometry

texture analysis method not included in surface roughness analysisprotocols is the characterization of patch properties using statisticsof filter banksrsquo output [VZ02]

Recent advances in texture analysis revealed that other meth-ods based on learned representations of local patterns can out-perform the classical ones for classification using joint distribu-tion of neighboring pixels [VZ03] advanced key-point descriptorslike SIFT coupled with encoders like Fisher Vectors or feature ex-traction methods based on Convolutional Neural Networks (CNN)training [CMKlowast14 CMKV16]

Feature extraction refers to using a pre-trained deep neural net-work and activate it in order to compute features for a new datasetat a certain layer in the architecture of the network The choiceof the activation layer is mainly a choice of design preferenceeven though there is a certain dependency between the similarityof the target data and the data that was used to train the pre-trainednetwork the more they are similar deeper layers in the architec-ture can be chosen This is mainly due to the fact that early layerslearn low-level features (edges blobs color) while last layers learntask-specific high-level features Two notable works that imple-mented the reuse of pretrained CNNs for texture analysis are pre-sented in [CMKlowast14] and [CMKV16] In the former one they loadthe AlexNet model while in latter they use the VGG models TheAlexNet network (8 layer architecture) was trained on a subset ofthe ImageNet database (more than a million images) for the Large-Scale Visual Recognition Challenge [RDSlowast15] (ILSVRC 2012)and can classify images into 1000 object categories Hence themodel has learned rich feature representations for a wide range ofimages Meanwhile the VGG models have won the ILSVRC 2014and are designed according to a deeper architecture (16 and 19 lay-ers)

To our knowledge there is no previous work in the literature thattested both ISO descriptors and texture analysis methods togetherand compared their ability to discriminate the roughness of differ-ent materials employed in CH objects

It would be therefore extremely useful to understand how spe-cific standard parameters and texture analysis methods can be use-ful for this specific task

For this reason we performed a study aimed at the evaluationof different roughness characterization methods on artificially agedmetallic objects

3 Study design

This research features metallic samples artificially degraded in anaging chamber The samples were created by a conservation labo-ratory involved in a vast research project aimed at the characteriza-tion of artworks The samples are made of silver alloys and bronzealloys Each dataset has a reference plate left uncoated and the resttreated with distinct coatings (one different coating per plate) typi-cally used in CH to prevent or at least slow down the aging processThe appearance of the uncoated samples between the different ag-ing steps is shown in Figure 1 It is possible to see that evidentchanges appear in the material at a macroscopic level however weare interested in using the microsurface analysis to find if some

Figure 1 The uncoated silver sample (top) at aging steps t0 t1 t2t3 and the uncoated bronze sample (bottom) at t0 t1 t2

consistent changes in the surface depth properties can be measuredwith time evolution

The silver plates have the dimensions of approximately7times25 cm with a thickness of 01 cm and they were cut from asheet of sterling silver (alloy of silver 925 and copper 75) un-dergoing additional polishing and brushing steps The coatings thathave been applied to the silver plates for preventing the fast agingeffects are those widely used by the conservatorrsquos community andare based on acrylic resin and wax

The bronze samples were created as an alloy of 90 copper and10 tin because this is the most representative mix employed inarts dating as far as the ancient times and common as well forthe Renaissance period Each bronze coupon has a dimension of8times5 cm and a thickness of 04 cm The coatings applied to thebronze samples as simple blended or layered combinations arethose most encountered in the conservation community acrylicresin (incral44) and soter wax The coating should protect againstcorrosion that is the principal aging danger to which the bronze al-loys kept in outdoor environments are susceptible to Consideringthis the artificial aging of the bronzes was performed by simulat-ing the outdoor conditions in a test chamber Surface acquisition forour study has been carried out with a microprofilometer based onconoscopic holography [DGMlowast17 GMD17] The spatial samplingof the original acquisitions was 0025 mm for the silver couponsand respectively 005 mm for the bronze coupons

Our study is aimed at verifying the possibility of characteriz-ing modifications in the material roughness at a small scale usingboth standard roughness descriptors and other texture characteri-zation methods used in the image processing domain To test thisidea we processed depth maps cropping small patches (with realsizes 05 cm for both silver and bronze samples) and after localbackground surface subtraction (planar surface) we independentlyestimated on each patch ISO roughness descriptors and texture fea-tures The procedure has been repeated for uncoated and coated


148


samples at the different time stepsTo ensure a fair comparison weperformed an accurate image registration using manually checkedcontrol points to align the depth images of the different time stepsin the same reference grid

The outcome of the procedure was for each material a set ofdepth texture patches labeled with a time step as displayed in Fig-ure 2 On these patches we computed ISO areal roughness pa-rameters and evaluated their capacity to distinguish between agingsteps and we performed time classification experiments with ISOdescriptors classical image texture descriptors and CNN-based de-scriptors Our pipeline as presented in Figure 3 concludes withfeeding the various descriptors into multiple classifiers and evalu-ating the accuracy of the aging classification for each of the studiedmaterials

31 ISO Roughness descriptors

We estimated ISO surface descriptors on patches using Moun-tainsMap httpwwwdigitalsurfcom a professionalsoftware tool widely applied in the surface metrology domainAreal roughness parameter features from the ISO25178 standard[Bla13] are divided in groups (see Table 1) related to height statis-tics (eg Sq root mean square height of the surface Ssk Sku skew-ness and kurtosis of height distribution) to spatial periodicity (egSal fastest decay auto-correlation rate Str texture aspect ratio ofthe surface Std texture direction of the surface) to the spatial shapeof the data (Sdq root mean square gradient of the surface Sdr de-veloped area ratio) to the shape of the regions resulting from awatershed segmentation (S10z ten point height) to the features ofthe material ratio (Abbott-Firestone) bearing curve (Spk reducedpeak height Sk core roughness depth Smr1 upper bearing area Smr2lower bearing area) finally

To understand which are the roughness descriptors most suitableto characterize material aging we used the approach proposed in[DKB14] eg we performed the analysis of variances (ANOVA)From the 3 groups of patches acquired at different time steps weestimated the F-score - the ratio between the variance of the meansand the average of the sample variances for each group which isan estimate of the overall population variance (assuming all groupshave equal variances) Higher values of this score correspond tolarger significance of the parameter variations across time steps

32 Classical texture descriptors

The set of ISO descriptors is quite rich but actually does not in-clude the most widely classical texture descriptors There is a vari-ety of 2D texture pattern characterization methods in the literatureand it is quite hard to tell which is the best for the characterizationof a specific classes of patterns as there are actually many differenttypes of texture characterization with different invariance proper-ties [LCFlowast18]

For fine-grained texture characterization popular choices not in-cluded in the ISO sets are second order statistics like gray co-occurrence matrix (COOM) specialized filter banks eg MR8[VZ03] local binary pattern (LBP) [OPM02]

Gray Level Co-occurrence Matrices sum the number of times

Table 1 3D Roughness descriptors according to the ISO 25178standard computed in this paper

3D Roughness descriptors defined by ISO 25178Height ParametersSq microm Root-mean-square heightSsk - SkewnessSku - KurtosisSp microm Maximum peak heightSv microm Maximum pit heightSz microm Maximum heightSa microm Arithmetic mean heightFunctional ParametersSmr Areal material ratioSmc microm Inverse areal material ratioSxp microm Extreme peak heightSpatial ParametersSal mm Autocorrelation lengthStr - Texture-aspect ratioStd Texture directionHybrid ParametersSdq - Root-mean-square gradientSdr Developed interfacial area ratioFunctional Parameters (Volume)Vm mm3mm2 Material volumeVv mm3mm2 Void volumeVmp mm3mm2 Peak material volumeVmc mm3mm2 Core material volumeVvc mm3mm2 Core void volumeVvv mm3mm2 Pit void volumeFeature ParametersSpd 1mm2 Density of peaksSpc 1mm Arithmetic mean peak curvatureS10z microm Ten point heightS5p microm Five point peak heightS5v microm Five point pit heightSda mm2 Mean dale areaSha mm2 Mean hill areaSdv mm3 Mean dale volumeShv mm3 Mean hill volumeFunctional Parameters (Stratified surfaces)Sk microm Core roughness depthSpk microm Reduced summit heightSvk microm Reduced valley depthSmr1 Upper bearing areaSmr2 Lower bearing area


149


Figure 2 Depth map patch from the uncoated silver sample displayed at different aging labels (in order from left to right) t0 t1 t2 t3 Theroughness changes from visible scratches at t0 to the apparition of new high-depth elements at t1 and finally shifts to the formation of patinaas it approaches t3

Figure 3 Our workflow for aging classification of cultural heritage materials based on multiple roughness and texture descriptors

that a given pixel pair is found adjacent in the original gray-scaleimage Based on this characterization of the spatial relationship be-tween pixels several statistics can be computed In our applicationwe mapped the depth range of the whole set of patches within 32gray levels assuming isotropic texture and averaging x and y (hor-izontal and vertical) displacement vectors of length 1 and 3 pixelsestimating at the two scales contrast correlation and energy fea-turesThese three features are estimated (contrast correlation en-ergy) building a 6 component descriptor vector

Local Binary Patterns encode the order relationship between apixel and a given neighborhood For each pixel a binary array witha number of components equal to the size of the neighbourhood iscomputed then histograms characterize the frequency of these val-ues and are finally concatenated into a feature vector for the entireimage For our data LBP features were extracted with rotationalinvariance [OPM02] with different values of radius (13) and circlesampling (816 points)

A simple and effective texture description may be obtained withthe response of filter banks The Maximum Response filter bank(MR8) described in [VZ02] characterizes patches with average andstandard deviation of the maximal output of oriented filters as wellas average and standard deviations of isotropically filtered values

33 Texture descriptors based on pre-trained CNN

We evaluated state of the art CNN-based approaches by usingfeatures obtained from pre-trained deep networks as suggested in[CMKlowast14 CMKV16] We tried different CNN-based texture la-belling methods proposed in the cited papers the penultimate fullyconnected layer of the pre-trained AlexNet architecture (FC-CNN)

and respectively the convolutional layer from the pre-trained net-work VGG19 pooled into a Fisher vector (FV-CNN) representa-tion Due to the high dimensionality of the latter amounting toan array size of 65k and computational limitations the LDC andNEURC classifiers could not be tested for FV-CNN Moreover thenumber of patches per sample sums up to an average of approxi-mately 100 which impeded a CNN fine-tuning approach

34 Supervised labelling methods

Using the different kind of patch descriptors we trained and testeddifferent supervised classification methods The classifiers coverdifferent approaches including generative discriminative paramet-ric and non parametric methods exploiting the implementationsprovided in the PRTools 5 Matlab package [dRTLlowast17] and in theLIBSVM library [CL11]

In our tests we compare a simple linear naive Bayesian classifier(ldc) linear support vector machine (libsvc) k-nearest neighbor(knnc) nonlinear Parzen classifier (parzenc) and non-linear feed-forward neural network with one hidden layer (neurc)

35 Patch classification tests

Using the extracted roughness descriptors on the patchesrsquo collec-tions we performed for the different materials a 10-fold cross-validation test to estimate accuracy in time label estimation withthe different classifiers previously cited In detail we estimated theaverage classification error of 10 tests where we took 90 of thepatches of each class as training set and the rest as test set It isworth mentioning that for the classification based on the classi-cal image texture descriptors and CNN features data augmentation


150


(flip rotation 90 degrees rotation 180 degrees) was used to increasethe collection of patches On the contrary since the ISO descriptorsare rotation-invariant there was no benefit in implementing dataaugmentation in this case

36 Effects of coatings on age characterization

Another use of the microprofilometric measurements can be as wellto evaluate how coatings influence the roughness Coated surfaceshave been captured and similarly characterized in small patcheswith ISO areal parameters and texture descriptors differences withuncoated samples and evolution of parameters with time has beenanalyzed

Figure 4 Bar charts showing the ranking of the first 10 rough-ness descriptors based on the F-score (x-axis) computed with theANOVA analysis Top Ranking for the uncoated silver sample Bot-tom Ranking for the uncoated bronze sample

4 Results

41 Best ISO descriptors for aging discrimination

To begin with the analysis of ISO descriptors reveals that both typeof surfaces have the most relevant roughness parameter (accord-ing to the F-score) belonging to the Functional parameters categoryrecommended for stratifies surfaces as can be noticed in 4 Hencethe changes for both uncoated silver and uncoated bronze seem tobe reasonably quantified by the segmentation of local differencesProbably this is because the material bearing ratios characterizewell stratified surfaces Thus the reduced peaks are the areas thatare removed by initial abrasion in a surface and Spk represents theaverage height of the reduced peaks In addition Smr1 describes the

areal material ratio that divides the reduced peaks from the core sur-face while Smr2 characterizes the areal material ratio that dividesthe reduced valleys from the core surface To analyze the signif-icance of the parameters to discriminate between different agingsteps (t0 t1 t2 t3) we plotted the distribution of the most relevantdescriptor for the uncoated silver and uncoated bronze samples aswell as for their coated variations with box-and-whiskers plots inFigure 5 The medians of the patches corresponding to the threetime steps are represented by the red line These plots point outinteresting behaviors

First of all as expected on the uncoated silver samples we mea-sure significant differences for the most relevant roughness valuesat different times that can actually be used to distinguish betweenthe small patches at different time steps

This is evident looking at the top left plot in Figure 5 Variationsof Spk are clearly statistically significant especially comparing t0and t1 (this is visible from the lack of overlap between the interquar-tile ranges represented by the blue rectangles) After t1 differencesare then less relevant with time even if a decreasing trend is clearAlso the inter-sample variability of the roughness parameters is de-creasing for the uncoated sample (see quartile boxes) suggestingthat formation of patina tends to even the roughness and flatten thesample

Looking at the evolution of the same parameter in the coatedsilver samples however it is possible to see that the behavior isnot perpetuated and there are also large variations of the roughnessdescriptors For C1 and C2 the medians seem to stay constant withaging as opposed to C3 where there is a dramatic change betweent0 and t3 almost as in the case of the uncoated silver sample Thisbehavior suggests that according to the Spk ISO descriptor C1 andC2 are more protective against aging than C3 for the silver materialThe behavior of the variances is however quite strange and thismay be due to the irregular distribution of the coating creating localdifferences that may then differently evolve with aging

The differences are less discernible in the case of bronze sam-ples Also in this case the most aging-discriminant feature (Smr1changes significantly after the first time step for the uncoated sam-ple but then it seems constant Differently from the silver it ap-pears quite difficult here to characterize age-related changes inroughness from ISO descriptors trends

It can be therefore interesting to search for other possible texturedescriptors that may characterize aging at a smaller scale To inves-tigate this we compared several texture classification methods onthe task of predicting correct time label from examples (extractedon the same sample)

42 Classification outcomes

Table 2 shows the classification accuracy obtained in the crossval-idation tests where we predict time labels of a subset of patchesgiven the rest of the set as examples As expected from the fea-tures behaviors previously described all ISO descriptors providequite poor results even if it is possible to see that the accuracy isclearly higher on the uncoated materials as expected This shouldmean that the classification results are not due to over-fitting butmeasure a real effect


151


Figure 5 Box plots showing the time evolution of the ISO feature with the highest F-score for time discrimination Top The Spk feature forsilver patches with different coatings uncoated one layer of acrylic two layers of acrylic one layer of acrylic + one layer of wax BottomThe Smr1 feature for the bronze patches with different coatings uncoated incral44 incral44 + wax wax + incral44 + wax

ISOImage texture

descriptorsCNN-based

texture descriptorssymbol coating classifier all iso coom lbp mr8 fv-cnn fc-cnn

LDC 053 034 018 014 - 075KNNC 038 037 041 025 018 009

LIBSVC 039 049 039 013 013 00652 patches NEURC 042 019 012 016 - 059

UC uncoated

PARZENC 039 039 069 023 075 075LDC 066 049 017 032 - 075

KNNC 055 051 043 046 037 036LIBSVC 071 061 051 035 026 031

48 patches NEURC 060 040 035 037 - 062C1 1c acrylic

PARZENC 058 051 065 064 075 075LDC 075 059 027 051 - 075

KNNC 073 055 060 051 046 058LIBSVC 067 061 059 057 034 047


PARZENC 071 057 069 056 075 075LDC 062 067 041 041 - 075

KNNC 059 051 052 049 045 045LIBSVC 064 065 055 042 034 033

44 patches NEURC 059 056 055 047 - 070C3 1c acrylic + wax overlay

PARZENC 061 052 059 051 075 075

Table 2 Error of cross-validated (10-folds) classifiers (from top to bottom naive Bayes k-nearest neighbour support vector machineneural network and parzen classifier) for aging predictionperformed on silver samples with different coatings and based on classical texturedescriptors neural net features and all the roughness parameters


152


ISOImage texture



LDC 047 027 033 029 - 067KNNC 048 036 053 046 033 038


UC uncoated

PARZENC 049 040 059 046 067 067LDC 059 051 051 052 - 067

KNNC 054 053 055 050 048 050LIBSVC 059 057 051 052 039 048

116 patches NEURC 056 048 053 048 - 060C1 incral44

PARZENC 058 055 066 051 067 067LDC 060 031 029 033 - 067

KNNC 060 043 055 045 035 036LIBSVC 059 052 037 032 027 028

109 patches NEURC 055 035 031 035 - 053C2 incral44 +wax

PARZENC 065 045 065 045 067 067LDC 046 043 038 041 - 067

KNNC 048 058 051 052 045 049LIBSVC 045 066 043 040 037 043

102 patches NEURC 044 041 040 038 - 057C3 wax + incral44 + wax

PARZENC 046 057 060 062 067 067

Table 3 Error of cross-validated (10-folds) classifiers (from top to bottom naive Bayes k-nearest neighbour support vector machineneural network and parzen classifier) for aging predictionperformed on bronze samples with different coatings and based on classicaltexture descriptors neural net features and all the roughness parameters

This behavior is maintained also by classifying with classical orCNN based texture descriptors In this case however the classifi-cation of uncoated silver samples is quite accurate with accuracyup to 94 using CNN based methods and close to 90 with lo-cal binary patterns (LBP) This means that it is possible to see thatthere are measurable effects of aging on high resolution texture

For the bronze samples we again have better results for the un-coated sample (see Table 3) meaning that there are measurablechanges reduced by the use of coatings even if the results arepoorer

It is clear that the results are not indicating a way to easily char-acterize the age of a surface with texture analysis as each sam-pleartworks has different background texture due to the specificmaterial treatment (brushing polishing etc) that makes differentobjects not comparable

But for example the classification outcomes provide interest-ing information about coatingsrsquo effects The silver resulting in thehighest overall classification error is C3 (one layer of acrylic plusone layer of wax) followed by C2 (two layers of acrylic) and lastlyby C1 (one layer of acrylic) This suggests that for silver the multi-layered coatings protect more than the single-layered coatings

The magnitude of difference in surface variation with aging be-tween variously coated samples is lower for the bronze dataset thanit is for the silver dataset Nonetheless the most stable bronze coat-ing over different aging steps is C1 (incral44) according to all de-scriptors On the other hand it seems that the coating C2 with abase layer of C1 (incrall44) and a top layer of wax has a higher ag-ing discriminatory power However given the complex behaviourand interplay between the different protective films it is difficult

to assign the influence of each separate component in multi-coatedcases

5 Discussion

The analysis of the degradation of artworksrsquo surfaces is a quite chal-lenging task In this paper we have shown that selected standardroughness descriptors used in material science applied to surfacegeometry captured at a microscopic scale can reveal changes in thesurface of metals across simulated aging steps However even if an-alyzed on a single metallic surface only few roughness measure-ments seems to be uniform in the sample and to characterize thedifferences appearing in the artificial aging process Furthermorethe amount of descriptorsrsquo changes decrease with time

The differences in roughness seem to be better captured by clas-sical or CNN based texture descriptors This has been demonstratedwith supervised labelling tests The combinations of local texturedescriptors and classifiers allow a reasonably successful classifica-tion of time steps given example patches from the same surface atthe different time steps This is not a pure overfitting effect as itcan be shown that when a coating is applied on the metallic sam-ples the ability of the method to discriminate time steps is reducedor it disappears

This means that the roughness analysis can be also exploited toassess the effects of different coatings used to reduce aging effectsCoatings actually makes aging differences in microgeometry lessevident and are more stable with time variation

The fact that texture analysis can capture aging related featuressuggests to further investigate the possibility of finding ways to


153


model aging effects with image processing applied to micropro-filometry data A possible idea could be to learn the associationtexture variations of patches with the related time interval

Acknowledgements Work partially supported by the Scan4Reco project(EU Horizon 2020 Framework Programme for Research and Innovationgrant agreement no 665091) the DSURF project funded by the Italian Min-istry of University and Research and Sardinian Regional Authorities underprojects VIGEC and VisampVideoLab

References

[Bla13] BLATEYRON F The areal field parameters In Characterisationof areal surface texture Springer 2013 pp 15ndash43 1 3

[Bun13] BUNGE H-J Texture analysis in materials science mathemat-ical methods Elsevier 2013 1

[CL11] CHANG C-C LIN C-J LIBSVM A library for support vec-tor machines ACM Transactions on Intelligent Systems and Technology2 (2011) 271ndash2727 Software available at httphttpwwwcsientuedutw~cjlinlibsvm 4

[CMKlowast14] CIMPOI M MAJI S KOKKINOS I MOHAMED SVEDALDI A Describing textures in the wild In Proceedings of theIEEE Conference on Computer Vision and Pattern Recognition (2014)pp 3606ndash3613 2 4

[CMKV16] CIMPOI M MAJI S KOKKINOS I VEDALDI A Deepfilter banks for texture recognition description and segmentation Inter-national Journal of Computer Vision 118 1 (2016) 65ndash94 2 4

[DGMlowast17] DAFFARA C GABURRO N MARCHIORO G ROMEOA BASILISSI G CAGNINI A GALEOTTI M Surface micro-profilometry for the assessment of the effects of traditional and inno-vative cleaning treatments of silver In Lasers in the Conservation ofArtworks XI (Proceedings of International Conference LACONA XI)(2017) Nicolaus Copernicus University Scientific Publishing Housedoi10127753875-409 2

[DKB14] DELTOMBE R KUBIAK K BIGERELLE M How to selectthe most relevant 3d roughness parameters of a surface Scanning 36 1(2014) 150ndash160 1 3

[dRTLlowast17] DE RIDDER D TAX D M LEI B XU G FENG M ZOUY VAN DER HEIJDEN F Prtools introduction Classification Param-eter Estimation and State Estimation An Engineering Approach UsingMATLAB 2nd Edition (2017) 17ndash42 4

[GMD17] GABURRO N MARCHIORO G DAFFARA C A versatileoptical profilometer based on conoscopic holography sensors for acqui-sition of specular and diffusive surfaces in artworks In Optics for ArtsArchitecture and Archaeology VI (2017) vol 10331 International So-ciety for Optics and Photonics p 103310A 2

[GZZ10] GUO Z ZHANG L ZHANG D A completed modeling of lo-cal binary pattern operator for texture classification IEEE Transactionson Image Processing 19 6 (2010) 1657ndash1663 1

[Har79] HARALICK R M Statistical and structural approaches to tex-ture Proceedings of the IEEE 67 5 (1979) 786ndash804 1

[LCFlowast18] LIU L CHEN J FIEGUTH P ZHAO G CHELLAPPA RPIETIKAINEN M A survey of recent advances in texture representationarXiv preprint arXiv180110324 (2018) 3

[OPM02] OJALA T PIETIKAINEN M MAENPAA T Multiresolutiongray-scale and rotation invariant texture classification with local binarypatterns IEEE Transactions on pattern analysis and machine intelli-gence 24 7 (2002) 971ndash987 3 4

[RDSlowast15] RUSSAKOVSKY O DENG J SU H KRAUSE J SATHEESHS MA S HUANG Z KARPATHY A KHOSLA A BERNSTEIN MET AL Imagenet large scale visual recognition challenge InternationalJournal of Computer Vision 115 3 (2015) 211ndash252 2

[SB00] STOUT K J BLUNT L Three dimensional surface topographyElsevier 2000 1

[TJlowast93] TUCERYAN M JAIN A K ET AL Texture analysis Hand-book of pattern recognition and computer vision 2 (1993) 235ndash276 1

[VGBEMlowast10] VAN GORP A BIGERELLE M EL MANSORI M GHI-DOSSI P IOST A Effects of working parameters on the surface rough-ness in belt grinding process the size-scale estimation influence In-ternational Journal of Materials and Product Technology 38 1 (2010)16ndash34 1

[VZ02] VARMA M ZISSERMAN A Classifying images of materi-als Achieving viewpoint and illumination independence Computer Vi-sionacircATECCV 2002 (2002) 255ndash271 2 4

[VZ03] VARMA M ZISSERMAN A Texture classification Are filterbanks necessary In Computer vision and pattern recognition 2003Proceedings 2003 IEEE computer society conference on (2003) vol 2IEEE pp IIndash691 2 3


154


texture analysis method not included in surface roughness analysisprotocols is the characterization of patch properties using statisticsof filter banksrsquo output [VZ02]

Recent advances in texture analysis revealed that other meth-ods based on learned representations of local patterns can out-perform the classical ones for classification using joint distribu-tion of neighboring pixels [VZ03] advanced key-point descriptorslike SIFT coupled with encoders like Fisher Vectors or feature ex-traction methods based on Convolutional Neural Networks (CNN)training [CMKlowast14 CMKV16]

Feature extraction refers to using a pre-trained deep neural net-work and activate it in order to compute features for a new datasetat a certain layer in the architecture of the network The choiceof the activation layer is mainly a choice of design preferenceeven though there is a certain dependency between the similarityof the target data and the data that was used to train the pre-trainednetwork the more they are similar deeper layers in the architec-ture can be chosen This is mainly due to the fact that early layerslearn low-level features (edges blobs color) while last layers learntask-specific high-level features Two notable works that imple-mented the reuse of pretrained CNNs for texture analysis are pre-sented in [CMKlowast14] and [CMKV16] In the former one they loadthe AlexNet model while in latter they use the VGG models TheAlexNet network (8 layer architecture) was trained on a subset ofthe ImageNet database (more than a million images) for the Large-Scale Visual Recognition Challenge [RDSlowast15] (ILSVRC 2012)and can classify images into 1000 object categories Hence themodel has learned rich feature representations for a wide range ofimages Meanwhile the VGG models have won the ILSVRC 2014and are designed according to a deeper architecture (16 and 19 lay-ers)

To our knowledge there is no previous work in the literature thattested both ISO descriptors and texture analysis methods togetherand compared their ability to discriminate the roughness of differ-ent materials employed in CH objects

It would be therefore extremely useful to understand how spe-cific standard parameters and texture analysis methods can be use-ful for this specific task

For this reason we performed a study aimed at the evaluationof different roughness characterization methods on artificially agedmetallic objects

3 Study design

This research features metallic samples artificially degraded in anaging chamber The samples were created by a conservation labo-ratory involved in a vast research project aimed at the characteriza-tion of artworks The samples are made of silver alloys and bronzealloys Each dataset has a reference plate left uncoated and the resttreated with distinct coatings (one different coating per plate) typi-cally used in CH to prevent or at least slow down the aging processThe appearance of the uncoated samples between the different ag-ing steps is shown in Figure 1 It is possible to see that evidentchanges appear in the material at a macroscopic level however weare interested in using the microsurface analysis to find if some

Figure 1 The uncoated silver sample (top) at aging steps t0 t1 t2t3 and the uncoated bronze sample (bottom) at t0 t1 t2

consistent changes in the surface depth properties can be measuredwith time evolution

The silver plates have the dimensions of approximately7times25 cm with a thickness of 01 cm and they were cut from asheet of sterling silver (alloy of silver 925 and copper 75) un-dergoing additional polishing and brushing steps The coatings thathave been applied to the silver plates for preventing the fast agingeffects are those widely used by the conservatorrsquos community andare based on acrylic resin and wax

The bronze samples were created as an alloy of 90 copper and10 tin because this is the most representative mix employed inarts dating as far as the ancient times and common as well forthe Renaissance period Each bronze coupon has a dimension of8times5 cm and a thickness of 04 cm The coatings applied to thebronze samples as simple blended or layered combinations arethose most encountered in the conservation community acrylicresin (incral44) and soter wax The coating should protect againstcorrosion that is the principal aging danger to which the bronze al-loys kept in outdoor environments are susceptible to Consideringthis the artificial aging of the bronzes was performed by simulat-ing the outdoor conditions in a test chamber Surface acquisition forour study has been carried out with a microprofilometer based onconoscopic holography [DGMlowast17 GMD17] The spatial samplingof the original acquisitions was 0025 mm for the silver couponsand respectively 005 mm for the bronze coupons

Our study is aimed at verifying the possibility of characteriz-ing modifications in the material roughness at a small scale usingboth standard roughness descriptors and other texture characteri-zation methods used in the image processing domain To test thisidea we processed depth maps cropping small patches (with realsizes 05 cm for both silver and bronze samples) and after localbackground surface subtraction (planar surface) we independentlyestimated on each patch ISO roughness descriptors and texture fea-tures The procedure has been repeated for uncoated and coated


148














149
















150






4 Results












151



ISOImage texture



LDC 053 034 018 014 - 075KNNC 038 037 041 025 018 009


UC uncoated

PARZENC 039 039 069 023 075 075LDC 066 049 017 032 - 075

KNNC 055 051 043 046 037 036LIBSVC 071 061 051 035 026 031


PARZENC 058 051 065 064 075 075LDC 075 059 027 051 - 075

KNNC 073 055 060 051 046 058LIBSVC 067 061 059 057 034 047


PARZENC 071 057 069 056 075 075LDC 062 067 041 041 - 075

KNNC 059 051 052 049 045 045LIBSVC 064 065 055 042 034 033


PARZENC 061 052 059 051 075 075



152


ISOImage texture



LDC 047 027 033 029 - 067KNNC 048 036 053 046 033 038


UC uncoated

PARZENC 049 040 059 046 067 067LDC 059 051 051 052 - 067

KNNC 054 053 055 050 048 050LIBSVC 059 057 051 052 039 048


PARZENC 058 055 066 051 067 067LDC 060 031 029 033 - 067

KNNC 060 043 055 045 035 036LIBSVC 059 052 037 032 027 028


PARZENC 065 045 065 045 067 067LDC 046 043 038 041 - 067

KNNC 048 058 051 052 045 049LIBSVC 045 066 043 040 037 043


PARZENC 046 057 060 062 067 067








5 Discussion






153




References





















154














149
















150






4 Results












151



ISOImage texture



LDC 053 034 018 014 - 075KNNC 038 037 041 025 018 009


UC uncoated

PARZENC 039 039 069 023 075 075LDC 066 049 017 032 - 075

KNNC 055 051 043 046 037 036LIBSVC 071 061 051 035 026 031


PARZENC 058 051 065 064 075 075LDC 075 059 027 051 - 075

KNNC 073 055 060 051 046 058LIBSVC 067 061 059 057 034 047


PARZENC 071 057 069 056 075 075LDC 062 067 041 041 - 075

KNNC 059 051 052 049 045 045LIBSVC 064 065 055 042 034 033


PARZENC 061 052 059 051 075 075



152


ISOImage texture



LDC 047 027 033 029 - 067KNNC 048 036 053 046 033 038


UC uncoated

PARZENC 049 040 059 046 067 067LDC 059 051 051 052 - 067

KNNC 054 053 055 050 048 050LIBSVC 059 057 051 052 039 048


PARZENC 058 055 066 051 067 067LDC 060 031 029 033 - 067

KNNC 060 043 055 045 035 036LIBSVC 059 052 037 032 027 028


PARZENC 065 045 065 045 067 067LDC 046 043 038 041 - 067

KNNC 048 058 051 052 045 049LIBSVC 045 066 043 040 037 043


PARZENC 046 057 060 062 067 067








5 Discussion






153




References





















154
















150






4 Results












151



ISOImage texture



LDC 053 034 018 014 - 075KNNC 038 037 041 025 018 009


UC uncoated

PARZENC 039 039 069 023 075 075LDC 066 049 017 032 - 075

KNNC 055 051 043 046 037 036LIBSVC 071 061 051 035 026 031


PARZENC 058 051 065 064 075 075LDC 075 059 027 051 - 075

KNNC 073 055 060 051 046 058LIBSVC 067 061 059 057 034 047


PARZENC 071 057 069 056 075 075LDC 062 067 041 041 - 075

KNNC 059 051 052 049 045 045LIBSVC 064 065 055 042 034 033


PARZENC 061 052 059 051 075 075



152


ISOImage texture



LDC 047 027 033 029 - 067KNNC 048 036 053 046 033 038


UC uncoated

PARZENC 049 040 059 046 067 067LDC 059 051 051 052 - 067

KNNC 054 053 055 050 048 050LIBSVC 059 057 051 052 039 048


PARZENC 058 055 066 051 067 067LDC 060 031 029 033 - 067

KNNC 060 043 055 045 035 036LIBSVC 059 052 037 032 027 028


PARZENC 065 045 065 045 067 067LDC 046 043 038 041 - 067

KNNC 048 058 051 052 045 049LIBSVC 045 066 043 040 037 043


PARZENC 046 057 060 062 067 067








5 Discussion






153




References





















154






4 Results












151



ISOImage texture



LDC 053 034 018 014 - 075KNNC 038 037 041 025 018 009


UC uncoated

PARZENC 039 039 069 023 075 075LDC 066 049 017 032 - 075

KNNC 055 051 043 046 037 036LIBSVC 071 061 051 035 026 031


PARZENC 058 051 065 064 075 075LDC 075 059 027 051 - 075

KNNC 073 055 060 051 046 058LIBSVC 067 061 059 057 034 047


PARZENC 071 057 069 056 075 075LDC 062 067 041 041 - 075

KNNC 059 051 052 049 045 045LIBSVC 064 065 055 042 034 033


PARZENC 061 052 059 051 075 075



152


ISOImage texture



LDC 047 027 033 029 - 067KNNC 048 036 053 046 033 038


UC uncoated

PARZENC 049 040 059 046 067 067LDC 059 051 051 052 - 067

KNNC 054 053 055 050 048 050LIBSVC 059 057 051 052 039 048


PARZENC 058 055 066 051 067 067LDC 060 031 029 033 - 067

KNNC 060 043 055 045 035 036LIBSVC 059 052 037 032 027 028


PARZENC 065 045 065 045 067 067LDC 046 043 038 041 - 067

KNNC 048 058 051 052 045 049LIBSVC 045 066 043 040 037 043


PARZENC 046 057 060 062 067 067








5 Discussion






153




References





















154



ISOImage texture



LDC 053 034 018 014 - 075KNNC 038 037 041 025 018 009


UC uncoated

PARZENC 039 039 069 023 075 075LDC 066 049 017 032 - 075

KNNC 055 051 043 046 037 036LIBSVC 071 061 051 035 026 031


PARZENC 058 051 065 064 075 075LDC 075 059 027 051 - 075

KNNC 073 055 060 051 046 058LIBSVC 067 061 059 057 034 047


PARZENC 071 057 069 056 075 075LDC 062 067 041 041 - 075

KNNC 059 051 052 049 045 045LIBSVC 064 065 055 042 034 033


PARZENC 061 052 059 051 075 075



152


ISOImage texture



LDC 047 027 033 029 - 067KNNC 048 036 053 046 033 038


UC uncoated

PARZENC 049 040 059 046 067 067LDC 059 051 051 052 - 067

KNNC 054 053 055 050 048 050LIBSVC 059 057 051 052 039 048


PARZENC 058 055 066 051 067 067LDC 060 031 029 033 - 067

KNNC 060 043 055 045 035 036LIBSVC 059 052 037 032 027 028


PARZENC 065 045 065 045 067 067LDC 046 043 038 041 - 067

KNNC 048 058 051 052 045 049LIBSVC 045 066 043 040 037 043


PARZENC 046 057 060 062 067 067








5 Discussion






153




References





















154


ISOImage texture



LDC 047 027 033 029 - 067KNNC 048 036 053 046 033 038


UC uncoated

PARZENC 049 040 059 046 067 067LDC 059 051 051 052 - 067

KNNC 054 053 055 050 048 050LIBSVC 059 057 051 052 039 048


PARZENC 058 055 066 051 067 067LDC 060 031 029 033 - 067

KNNC 060 043 055 045 035 036LIBSVC 059 052 037 032 027 028


PARZENC 065 045 065 045 067 067LDC 046 043 038 041 - 067

KNNC 048 058 051 052 045 049LIBSVC 045 066 043 040 037 043


PARZENC 046 057 060 062 067 067








5 Discussion






153




References





















154




References





















154

aging prediction of ch samples based on surface microgeometry · i.m. ciortan, g. marchioro, c....

Documents