Modelling of Suspended Matter by Hybrid

RBF-IGNG Network *Parid ALILAT, Saliha LOUMI

Faculty of Electrical Engineering and Computer Science, Institute of Electronics. Laboratory of Image Processing and Radiation.

BP 32, EI-Alia Bab - Ezzouar, 16111, Algiers, Algeria

faralilat@yahoo.fr, salihaloumi@yahoo.fr

Abstract- The aim of this paper is to look for efficient algorithms allowing to deal with taking into account the modelization and the cartography of the sea components. Through the analysis carried on the family of Growing Neural Gas with an evolutive (scalable) architecture and with non supervised competitive learning, some modifications and improvements (modified IGNG) have been brought in order to associate them with the neural network of modelization; this modification is essentially focused on the automation of parameters. So as to improve the results, we propose in our technique to widen the sphere of influence of the RBF by multiplying the Gaussian widths by a factor which is automatically sought for so as to minimize the error of modelization on the learning basis. A study on the type of Gaussian widths of the REF and their shapes has been carried. The developed methodology, the implemented procedures and the proposed networks all yielded satisfying results.

Keywords- Neural network, Remote sensing , RBF, lGNG,

Classification, Suspended matter.

I. INTRODUCTION

The seas and the oceans play an important role in the life cycle of the earth (food chain and climatic equilibrium). Their observation is therefore fundamental and must be general and frequent with the objective to estimate their wealth and the impact of the global or local modifications on their health [1]. The spatial observation alone can nowadays allow a frequent and global apprehension. Indeed, satellite imagery has made it possible to measure many parameters (wind, water temperature, swell, tides, atmospheric parameters, chlorophyll concentration, suspended matter. .. ) which were up to a recent past quite difficult to know because they required in situ measurements. One must note that some of these measures are possible only once an analysis of the colour of the water[2] has been made because the optical properties of many of its components such as alga, dissolved organic matters, suspended non organic particles (sand, mud, clay ... ) may affect its colour [3].

This paper is devoted to the development of a technique allowing the modelization, by this analysis of the suspended particulate matter (SPM) in the Algiers bay (south west of the

978-1-4799-0299-6/13/$31.00 ©2013 I EEE

Mediterranean Sea)[4]. The networks developed and used for the modelization are the radial basis function (REF) network, associating the Incremental Growing Neural Gas (IGNG) neuron-classifier network with a dynamical self organized map which is modified so as to integrate it in this macro structure of modelization. The data used for the modelization is on the one hand, the multispectral images of the micro satellite (ALSA T -1) and on the other hand a learning basis made up of the SPM values and the associated radiometric values in the bands of this sensor will be the subject of Section 2.The proposed methodology and the description of the used networks are presented in section 3 since section 4 is devoted to obtained results.

II. DATA AND SENSOR

The site under study is the Algiers bay with geographical

coordinates 36° 39' 00 N to 36° 51' 00 N and 3° 00' 30 E to

3° 16' 20 E. The utilized data represent multi spectral images

acquired on March, 10lh 2003 by the Algerian micro satellite

ALSAT-I [5] wich is the first of a series of 05 satellites

launched in the frame of the DMC program (Disaster

Monitoring constellation) for the period from 2002 to 2005. It

is equipped with high capacities of altitude command and orbit

as well as a high downloading rate. Table I presents the main

characteristics.

To eliminate the contribution of the atmosphere, we have

used the A TCOR package for the atmospheric correction. This

tool requires parameters depending from the photography

conditions and the climatic data. The calculation of the solar

angles (zenith and azimuth) is carried with the help of a

software of the ATCOR package, called "SUNNY". It requires

the introduction of the date and time of passage of the satellite

over the zone corresponding to the image. The atmospheric

correction is taken in charge by of the ATCOR2 module which

does not integrate the technical specificities of the ALSA T I

sensor. Since the spectral characteristics of the ALSA Tl

images are very close to those of the Landsat ETM+ [5] we

have substituted the ALSATI by the Landsat-7 ETM+. This

module requires the spatial resolution, the calibration file, the

atmospheric correction model, the visibility and the mean

elevation in km.

T ABLE I. CHARACTERISTICS OF THE MICRO SATELLITE ALSAT-1

Launching November 28, 2002 by COSMOS-3M

Altitude 680km - 98° tilt

Dimensions 600cm x 60cm x 60cm

Weight 90 kgf

Imaging mode broom brush

Multispectral CCD imager Two overlapping bands @ 5%

Imaging system Optical 150mmfocal distance .

Resolution / Swath width 32m/600km

Pixel number

Storage

Spectral bands

Scene

Cycle

So as to let the positions of the images data coincide with

the reality on the ground (in situ measures), a georeferencing

has been done. This p was carried with the help of six control

points by using the UTM projection, a transformation of

polynomial type with degree 1, and interpolation of the closest

neighbour.

III. METHODOLOGY FOR MODELLING SPM

The methodology for the determination of the SPM is neuronal model. It consists (figure 1) to present the three bands of the ALSA T -1 image to the network, then the multiplication of the result by a mask necessary to the isolation of the zone of interest which is the water.

Network Modelling SPM

Masque

..

dr

Fig 1. Diagram of modelling the suspended particulate matter

A. Creation o{the mask

The mask allowing to consider only the water part of the image is done by the [SODATA algorithm. The cleaning of the

10000 pixels

1 Go byte

Green 0. 523 - 0. 605j.lm

Red 0. 629 - 0. 69011111

Near Infrared O. 774 - 0. 900j.lm

600km x 560km (max. )

5 days

mask is a necessary operation in the view of its improvement. It is realized with the help of the tools of the mathematical morphology [6][7]. We have used on the one hand the basic operations such as the opening and closure and on the other hand operations of reconstruction by conditional dilatation. Through this process, we have eliminated the small ruggednesses without however eliminating Reghaia' s lake, Keddara' s dam and El Hamiz's dam [3]. Figure 2 illustrates the isodata mask and its cleaning.

B. Neuronal Modelization

The network involved in this modelization a neuronal macro structure associating the RBF network to the modified dynamic self organised map IGNG network (figure 3).

Input

;-------------------, ,,'

, .---��--� .---�----�

Modified IGN

Fig 3. hybrid RBF/IGNG modified network

- a- - b-

----==t=_.L louds

Mask imperfections �-t---,.,

Fig 2. ISODATA classification of the ALSAT-I image. a : Brut mask, b: Cleaned mask

1) Network with radial Basis Functions.

The REF network is a supervised neuronal network. It is a "specialisation" of the multilayer perceptron (MLP) and consists of 3 layers: an input layer, a radial basis layer and a linear output layer. Each layer is totally connected to the next. Figure 4 shows a radial basis network in its matricial representation.

/ p a2

Rxl S2xl

� \�----�v�----�/ \�----�v�----�/ Input Layer

Radial basis layer linear layer

Fig 4 .. RBF Network

The universal property of approximator of these networks has been proved for radial Gaussians [9] and more generally for the RBF [10]. The general expression of the output for the RBF network is:

(1)

Where f1i is the vector (possibly a scalar) of the centres, a} a scalar, P the input vector (possibly a scalar) of the RBF neurone, Wi the weight linking the RBF neurone i to the output

neurone If! and S' the number of RBF. The network output is

simply a linear combination of the outputs of the REF neurones multiplied by the weights of their respective connexions. Starting from the fact that the RBF and output layers realize distinct tasks, the learning of the network can be achieved in two distinct steps:

• Determination of the positions of the centres and estimation of the variances of the radial neurones.

• Optimization of the weights of the output linear layer.

b The determination of the positions of the centres

may be realized in different ways. In the present work, we aim to realize this task with the help of the modified IGNG (neurone network of the dynamical SOM) . The centers thus determined will be taken as equal to the reference vectors of the latter (WD. The basis functions which are tested in this work are mono and multi variance Gaussians of the form

¢(x) = exp[_ IIX -,LIf 1 2(j

2 (2)

Three (3) widths of hyper spherical and an other hyper elliptical width of distance have been implemented. These are:

• Statistical width implemented for multi variable and multi variance Gaussian functions

Nj 2

Illpi - ,LIjll 2 (.) i=l (j stat ] = -'-=''--

-N-.--

) (3)

Where Nj is the number of instances pi of the Cj class with

center J1j and 11.11 is the 12 norm j= 1 . . . ,SI and Sl is the

number of nodes of the REF layer equal to the number of classes.

• Statistical width which consists to comput for each class,

the Euclidean distance of the furthest instance of the

class. For this technique, each REF neurone has its width.

(4)

where Pi is an instance of the class Cj with center J1J.

• The greatest distance between the centers and the associated furthest instances s

• In the hyper elliptical distance case, the width of a REF is a distance vector between the centroid and the furthest sample in the class. Each component of the vector is the maximal distance along an axis of the input space and it is determined independently of the others; this allows defining the samples of the same class. This width is determined by the following equation:

m runs the dimension of the input, i runs the samples of the class C/, J1m,j is the component m of the center of the jth RBF and Pi,m the mth component of the ith stimulus belonging to the class Ct. The activation of this RBF in that case is given by equation 7.

¢j (�) = exp[- � f [pm, i ,- f.Lm, j ]2] (7)

2 m=1 (J'm,j

dbeing the dimension of the input space.

The fixation of the widths of the Gaussians must be done so as they be neither too large (difficult convergence) nor too narrow (mediocre generalization). In view of an improvement of the results, an automatic spreading technique of these widths has been developed and it will be the object of section 4.

Once the centers are positioned and the widths of the basis functions are fixed, the second step of the learning process consists in the estimation of the weights of the linear output layer. This latter is realized by the utilization of the least square method and the inverse or pseudo-inverse matrix. This method which is used for the learning of the static models or non loop predictors, whose output is linear relative to the unknown parameters, is totally adequate in the case of our network. For

the nth instance from the N of the learning basis, equation 8

allows us to write the /' output of the network with the following expression:

Sl y(n) = I 0iXi (n) , n = l,oo.,N

i=1

xi (n) = ¢JPn) et 0i = W],i (8)

The minimization of the expression of the mean quadratic error between the obtained and desired outputs, allow us to determine the connexion weight vector between the REF layer and the considered output node with the equation:

(9)

Yp is the desired output vector and X is the activation matrix of the REF layer, opposed to the instances of the learning basis.

The solution ° of equation 9 exists under the condition that the matrix XiX be invertible. This condition is generally satisfied when N is very large compared to S 1. When this matrix is singular, we use the pseudo-inverse .

2) Incremental Growing Neural Gas Network (IGNG)

This network has dynamical self organizing maps [I I ]. It is able to learn without considering previous learned data. it has two types of neurones: mature neurones and embryo neurones[12][13]. Further, each neurone has an old and a reference vector and each connexion has an old [14][15]. In our application, a modification of the IGNG algorithm has been proposed to automate the radius of the hypershere choice.

The aim is to get the adequate parameter (J'which would yield the number of classes (mature prototype neurones) hoped at the end of the classification. The proposed technique requires only the choice of an initial parameter: the initial radius

(J'o=(J'(O), although in practice, we adopt an initial value(J'o which is equal to the smallest hyper sphere encompassing the space filled by the normalized data multiplied by the factor I ISI. At each training, this parameter evolves in an iterative way. The stopping criterion being the convergence to the network having the desired number Sl of mature neurones (of

s' course, some forbearance is accepted) (J'(k + 1) = -1 (J'(k)

S

Where S lis the desired number of mature neurones, s' is the number of mature neurones in the k

th training step. The IGNG algorithm, thus modified is presented as follows:

Modified IGNG algorithm

1. Initialisation o{the parameters Set tml}X. '1g. '1v, amax, amalUreoo (J = (Jo, the number of desired classes (mature neurones) for S and the tolerance s on this number, and the zero tolerance indicator.

2. Time's initialisation: t= 1

3. Repeat as long as t:S tmax : ( all steps under 3 are identical to the classicallGNG)

3. 1. Presentation of an input data

3.2. Locating of the winners

3. 3. Ageing of the connexions of the winner

3. 4. Update of the reference vector

3. 5. Rejuvenation of the connexion of the winners

3. 6. Death of the ageing connexions

3. 7. Ageing of the neurones

3. 8. Maturation of the embryo neurones

3. 9. Increment t.

4. Count the number s' of mature neurones (with age more than alnuture)

5. If s '= S go to step (10).

, s

6. Comput (J": (J"new = -1 (J"old S

7. If the difference between the numbers of matures and S is less than s, increment the tolerance indicator.

8. If the tolerance indicator is more than a fixed threshold, go to step (10)

9. Return to step (2)

1 O. End of learning..

IV. RESULTS AND DISCUSSIONS

For the training of this modified hybrid network RBF/IGNG of the SPM modelization, we have used a learning basis of 2500 non repetitive samples and a control basis of 1000 other non repetitive samples. A study of the influence of the number of nodes over the REF layer has been carried by considering a REF network with (03) nodes in the first layer and one node at the output layer.

The criteria needed for the evaluation of the different developed techniques and their implementation are on the one side, the root mean square error (RMS) at the learning and control; this RMS is compared with the output average value (the average of SPM value) by determining the relative RMS in percentage, and on the other side the measure of the maximum gap between the obtained output and the reference SPM at the control step.

The study which we have carried with modified hybrid REF/IGNG networks has shown that, for any used hyper spherical width, a spreading of this latter was necessary for the improvement of the modelization results. We propose in our technique to widen the influence radii of the REF by multiplying the width of the Gaussian by a factor A. The

technique consists in letting j.} vary with a step (0.1 in our application), in realizing the learning for each case, and in determining the RMS. The stopping criterion is a RMS to reach in the learning phase or a stop before the fluctuations appear. Thus, this factor is automatically sought so as to minimize the RMS. As an illustration, we represent in figure 5 the case of a RBF/IGNG network modified with 20 neurons. By using this technique, an experiment taking into account the

number of RBF nodes and the type of hyper spherical widths, has shown that globally, the RMS is by far improved (it has been divided by 15).

(j)

5

4.5

4

3.5 �

I I - - - - - - - - - - - - - 1 - ---1-- - - - - - --

I I I - -1- - - - - - - - - - - - - - I - - - - I - - - - - - - - -

3� � �I� � � � � � � � � � � � � � I

____ L ________ _

il2 2.5

2

1.5

0.5

5 10 15 20 ')...2

25 30 35

Fig 5. RMS as function ofA2 for a modified RBF-IGNG with 20 nodes

Figure 6 illustrates the improvement results of this technique. For a better estimation of the results, we present in table II, the RMS and relative RMS (RMSrelutive) at the learning and at the control and M (the absolute largest error over the samples of the control basis). The minimum and maximum of the desired output are respectively 93.95mg/l and 248.32mg/1. The means of the MES are: 21l.l6 mg/l in the learning basis and 21O.46mg/l in the control basis.

T ABLE II PERFORMANCES FOR EACH MODIFIED RBF-IGNG AS A FUNCTJON OF "A

Learning Control

Nodes A. RMS RMS,e/an,l%) RMS RMS,elan,l%) M

20 6.29 0.61 0287 0.530 0.250 3.08 50 3.77 0.098 0.046 0.100 0.049 0.86 80 3 02 0.051 0.024 0.061 0.029 0.57 100 3.00 0.031 0.015 0.036 0.017 032 120 2.86 0.045 0.021 0.053 0025 0.52 150 2.77 0.027 0.013 0.041 0.019 0.62 200 2.61 0.017 0.008 0.030 0.014 0.48 250 226 0.032 0.015 0.050 0.024 0.58

One must note that the RMS in the learning and control phases are very weak (less than 0.1 for a number of nodes more or equal to 50). The same goes for the absolute maximum error at control (M) compared to the extreme values of the MES. Note that the control RMS is superior to the learning RMS, which is explained by the fact that the control basis contains no sample of the learning basis and denotes the strong ability this network has to generalize in its test phase.

- a -260

240

220

�2OO ::J U1 �100 ...., ::J IE;] 0.. ...., ::JUO

0 120

100

eooo 100 120 140 l6IJ If[] 200 Desired output

260

240

220

�2OO ::J U1 �180 ...., ::J IE;] 0.. ...., ::J 140

0 120

100

220 240 26IJ 80 80

- b -

/ /

100 120 140 IE;] If[] 200 220 240 260 Desired Output

Fig 6. Performances of a modified RBFIIGNG with 20 RBF nodes.

a: wide (JrJ[ST without sprawl RMS=9.06, b:wide (JrJ[ST with sprawl RMS =0.61

The results obtained by this technique for a network at 80 REF nodes are:

• At learning RMS=0.025 and RMSrelative=0.012% • and at control: RMSrelative=0.0126% and M=0.30.

The error has diminished relatively to the spherical Gaussians in a rate more than 2 and these results are equivalent to those with a number of RBF more than 150 in the case of hyper spherical width Gaussians .This method of computation of the widths allows thus to diminish the number of RBF of the hidden layer, while still improving the network performances.

The generalization of the modified REF/IGNG

trained by the technique of search of the optimal factor A is presented in figure 7. It is about the multi spectral ALSAT-1 satellite image of the Algiers bay, geo referenced and atmospherically corrected.

Fig. 7. SPM Image obtained by modified REF-IGNG with 80 nodes

V. CONCLUSION

Through the analysis carried over the neuronal classifier with progressive architecture, dynamical non supervised competitive learning with self organised map of the Growing Neural Gas family and most precisely the Incremental, modifications and improvements were necessary in view of its integration in the hybrid neuronal network of modelization. The modification is essentially on the automation of its parameter of choice of the radius of the insertion hyper sphere. The association of the REF with the modified IGNG in this modelization is not fortuitous insofar as we search an incremental learning. In order to improve the results, we have widened the radii of influence of the RBF by multiplying the widths of the Gaussians by an automatically sought factor, so as to minimize the error of modelization on the learning basis. A study on the type of widths of the RBF, their shape and their spreading has been realized.. By this technique, we have sought for a global multiplier of the width (the same for all the REF and all the components of the hyper ellipsoid); we think a better result can be obtained if the factor is locally sought for (for each REF and each component of the width). However this technique is very greedy in computation time. The developed methodology, the implemented procedures and the proposed network has given full satisfaction as for the modelization of the suspended matter.

REFERENCES

[1] 1. M. Froidefond, D. Doxaran, 'Teledetection Optique Appliquee a l'etude de eaux c6tieres," Teledetection, vol. 4, no. 2, pp. 157-174, 2004.

[2] C. L. PARKINSON, "Aqua: An Earth-Observing Satellite Mission to Examine Water and Other Climate Variables," IEEE Trans. Geosci. Remote Sensing, vol. 41, no. 2, pp. 173-183, Feb. 2003.

[3] Merrad H. , "Caracterisation des Eaux C6tieres a partir d'Images Multi spectrales de MODIS et ALSAT-I : Application au littoral Algerien " These de Magister Universite des Sciences et de la Technologie Houari Boumediene a Alger. 2005.

[4] Alilat F. , Loumi S. , Merrad H. & Sausal B. , " Nouvelle approche du reseau ARTMAP Flou Application 11 la classification multispectrale des images SPOT XS de la baie d'Alger ", Revue Frauyaise de Photogrametrie et de Teledetection SFPT, (2005-1),.no.I77, pp 17-24, 2005.

[5] "Actes des Journees Techniques ALSA T 11 Utilisateurs," 14 et 15 Juillet 2003 - Alger.

[6] Coster M. & Cherman 1. L., Precis d'analyse d'images. Editions du CNRS. 1985.

[7] Serra.l., Image Analysis and Mathematical Morphology. Academic Press, London. 1982.

[8] Hartmau E. 1. & Kowalski 1. M., Layered Neural Networks with Gaussian Hidden Units as Universal Approximations. Neural Computation, Vol. 2, pp.210-215. 1990.

[9] Powell M. 1. D.,. Radial basis functions for multivariable interpolation : A review. In 1. C. Mason aud M. G. Cox, editors, IMA Conf. Algorithm for Approximation of Functions and Data, Oxford, UK, Oxford Univ. Press, pp.143-167. 1987

[10] Tschirhart F. , Lioret A. , "Reseaux de neurones formels appliques 11 I'lntelligence Artificielle et au jeu memoire de master de recherche en multimedia et animation numerique", Paris, 121 p. 2009

[11] Fritzke B. , Growing cell structures - A self:orgauizing network for unsupervised and supervised learning. Neural Networks, Vol. 7, No 9, pp. 1441-1460. 1994.

[12] Fritzke B. , "Growing grid - a self organizing network with constant neighbourhood rauge aud adaptation strength". Neural Processing letter, (2), 9-13. 1995a.

[13] Fritzke B. , A Growing Neural Gas Network Learns Topologies". In G. Tesauro, D.S. Touretzky & T.K. Leen (editors), Advances in Neural Infonnation Processing Systems 7, MIT Press, Cambridge MA, pp.625-632, 1995.

[14] Martinetz T. M. , Berkovich S. G. , & Schulten K. 1., "Neural-gas network for vector quantization and its application to time-series prediction". IEEE Traus. Neural Networks, Vol. 4, No. 4, pp. 558-569. 1993.

[15] Dreyfus G. , Martinez .l.M.,Samuelides M., "Apprentissage statistique : reseaux de neurones, cartes topologiques, machines 11 vecteurs supports", Eyrolles . 2008.