barycentre method 09 -...

Computational Geosciences - 2015 - PART I METHODS

9 Barycentre Method9- 1 Introduction

One applicable condition of the Newton method is that the Jacobi matrix isn’t deficient in rank. To deal with thisproblem one may use Extended Newton-Raphson Method, however if your system is special, namely consists ofdistance equations, you can employ Barycentere Method. This method has global convergence and avoids thecalculations of the Jacobi matrix.

The general form of the distance equations is

(xi - x0)2 + (yi - y0)2 + (zi - z0)2 - Yi = 0, i = 1, ..., n

where Xi (xi, yi, zi) is the coordinates of ith known point,Yi is measured distance of the ith point and X0 (x0, y0, z0) is coordinates of the unknown point,which to computed , n is the number of equations .

Xue at al (2013) suggested the following iterative formula, which is a fixed point method practically, to solve suchproblem,

X0k+1 =

1

ni=1

nSiX0

k

where the upper index referring to the kth iteration,

Si(X0 ) = xi + Xi-X0Xi - X0

Yi, i = 1,...,n

The implementation of this algorithm is quite simple. Let us define a function as g

X0k+1 = gX0

k

Then this fixed point problem can be solved with iteration with an initial guess value.

9- 2 Implementation in Mathematica

The implementation of this algorithm in Mathematica is very straight forward. Let this g(X0) function as

g[x_] := TotalMapThread #1 +x - #1

Norm[#1 - x]#2 &, {X, Y} Length[Y]

Considering a test example, let the input data as

X = {{10, 20, 6}, {8, 30, 0}, {10, -20, 6}, {8, -30, 0},

{-10, -20, -6}, {-8, -30, 0}, {-10, 20, -6}, {-8, 30, 0}};

Y = {22.18, 30.958, 23.1749, 30.88, 23.07, 30.44, 22.70, 30.72};

p1 = ListPointPlot3D[X, PlotStyle PointSize[0.02], BoxRatios {1, 1, 0.8}]

-10

-5

0

5

10

-20

0

20

-5

0

5

Fig. 9.1 Test example for solving distance equations

The distance equations

e = (xi - x0)2 + (yi - y0)2 + (zi - z0)2 - si;

datan = {x1 10, y1 20, z1 6, s1 22.18,

x2 8, y2 30, z2 0, s2 30.958,

x3 10, y3 -20, z3 6, s3 23.1749,

x4 8, y4 -30, z4 0, s4 30.88,

x5 -10, y5 -20, z5 -6, s5 23.07,

x6 -8, y6 -30, z6 0, s6 30.44,

x7 -10, y7 20, z7 -6, s7 22.70, x8 -8, y8 30, z8 0, s8 30.72

};

n = 8;

F = Table[e /. datan, {i, 1, n}]

-22.18+ (10 - x0)2 + (20 - y0)2 + (6 - z0)2 , -30.958+ (8 - x0)2 + (30 - y0)2 + z02 ,

-23.1749+ (10 - x0)2 + (-20 - y0)2 + (6 - z0)2 , -30.88+ (8 - x0)2 + (-30 - y0)2 + z02 ,-23.07+ (-10 - x0)2 + (-20 - y0)2 + (-6 - z0)2 , -30.44+ (-8 - x0)2 + (-30 - y0)2 + z02 ,

-22.7 + (-10 - x0)2 + (20 - y0)2 + (-6 - z0)2 , -30.72+ (-8 - x0)2 + (30 - y0)2 + z02

9- 3 Comparing different methods9- 3- 1 Extended Newton - Raphson Method

First we employ the Extended Newton method using pseudoinverse.

NewtonExtended[f_List, x_List, x0_List, eps_: 10^-12, n_: 100] :=

With[{jac = N[Outer[D, f, x]]},

FixedPointList[(# + PseudoInverse[jac /. Thread[x N[#]]].(-f /. Thread[x N[#]])) &,

N[x0], n, SameTest (Sqrt[Abs[(#1 - #2).(#1 - #2)]] < eps &)]]

As we have seen this method applies pseudoinverse J+ in order to solve over-determined nonlinear system:

2 Barycentre_Method_09.nb

As we have seen this method applies pseudoinverse in order to solve over-determined nonlinear system:

xi+1 = xi - J+ (xi) f (xi)

It needs initial guess values, now we employ the initial values {0.3,0.3,0.3} suggested by the Xue et al (2013).

solNE = NewtonExtended[F, {x0, y0, z0}, {0.3, 0.3, 0.3}]; // AbsoluteTiming

{0.0650993, Null}

The result is

Last[solNE]

{0.278119, 0.123038, -0.499993}

which is incorrect since the method is not converging. For example, the first coordinate is oscillating

S = Transpose[solNE];

ListPlot[S[[1]], Joined True]

20 40 60 80 100

-1.0

-0.8

-0.6

-0.4

-0.2

0.2

Fig. 9.2 Oscillating solution by Extended Newton method

9- 3- 2 Extended Newton - Raphson Method with relaxation

To improve this result, standard relaxation method can be employed, namely

xi+1 = (xi + J+ (xi) f (xi)) λ + (1 - λ) xi = xi + λ J+ (xi) f (xi)

where 0 < λ ≤ 1. Let us consider λ = 0.5,

λ = 0.5;

NewtonExtendedλ[f_List, x_List, x0_List, eps_: 10^-12, n_: 100] :=

With[{jac = N[Outer[D, f, x]]}, FixedPointList[

(# + λ PseudoInverse[jac /. Thread[x N[#]]].(-f /. Thread[x N[#]])) &,

N[x0], n, SameTest (Sqrt[Abs[(#1 - #2).(#1 - #2)]] < eps &)]]

solNE = NewtonExtendedλ[F, {x0, y0, z0}, {0.3, 0.3, 0.3}]; // AbsoluteTiming

{0.0393074, Null}

Now the result is

Last[solNE]

{-0.214093, 0.123774, 0.549276}

which is converging. For example, the first coordinate is

S = Transpose[solNE];

Barycentre_Method_09.nb 3

ListPlot[S[[1]], Joined True, Frame True]

0 10 20 30

-0.214

-0.212

-0.210

-0.208

-0.206

Fig. 9.3 Convergence of the solution of the Extended Newton method with relaxation

The number of iterations insuring error - the difference between the two subsequent result - is less then 10-12

Length[solNE]

399- 3- 3 Local minimization

The objective to be minimized is

obj = Total[Table[e2, {i, 1, n}]]

-s1 + (-x0 + x1)2 + (-y0 + y1)2 + (-z0 + z1)22+

-s2 + (-x0 + x2)2 + (-y0 + y2)2 + (-z0 + z2)22+

-s3 + (-x0 + x3)2 + (-y0 + y3)2 + (-z0 + z3)22+

-s4 + (-x0 + x4)2 + (-y0 + y4)2 + (-z0 + z4)22+

-s5 + (-x0 + x5)2 + (-y0 + y5)2 + (-z0 + z5)22+

-s6 + (-x0 + x6)2 + (-y0 + y6)2 + (-z0 + z6)22+

-s7 + (-x0 + x7)2 + (-y0 + y7)2 + (-z0 + z7)22+

-s8 + (-x0 + x8)2 + (-y0 + y8)2 + (-z0 + z8)22

FindMinimum[obj /. datan, {{x0, 0.3}, {y0, 0.3}, {z0, 0.3}},

Method "PrincipalAxis"] // AbsoluteTiming

{0.00577685, {1.50821, {x0 -0.214096, y0 0.123773, z0 0.54928}}}

Surprising running time! But do not forget that this function is a compiled version.9- 3- 4 Global minimization

Without initial guess

solN = NMinimize[obj /. datan, {x0, y0, z0},

Method "DifferentialEvolution"] // AbsoluteTiming

{0.223882, {1.50821, {x0 -0.214093, y0 0.123774, z0 0.549276}}}9- 3- 5 Levenberg Marquardt method


The problem can be considered as a parameter estimating problem, namely we are looking for the parameters ofthe model {a, b, c}

s = (x - a)2 + (y - b)2 + (z - c)2

where we have measured values for {x, y, z} and s. Since this is a quadratic problem let us employ the Levenberg-Marquardt method. Preparing the data

data = MapThread[Flatten[{#1, #2}] &, {X, Y}]

{{10, 20, 6, 22.18}, {8, 30, 0, 30.958}, {10, -20, 6, 23.1749}, {8, -30, 0, 30.88},{-10, -20, -6, 23.07}, {-8, -30, 0, 30.44}, {-10, 20, -6, 22.7}, {-8, 30, 0, 30.72}}

Applying built in function of Mathematica

AbsoluteTiming{nlm, n} = Block{m = 0},

NonlinearModelFitdata, (x - a)2 + (y - b)2 + (z - c)2 , {a, b, c},

{x, y, z}, StepMonitor m++, Method "LevenbergMarquardt", m;

{0.0440701, Null}

Then the model with the estimated parameters

Normal[nlm]

(0.214093 + x)2 + (-0.123774+ y)2 + (-0.549276+ z)2

The number of the iteration step

n

15

The statistics of the estimation

nlm["ParameterTable"]

Estimate Standard Error t- Statistic P- Valuea -0.214093 1.03821 -0.206213 0.844758b 0.123774 0.211925 0.584048 0.584544c 0.549276 2.00329 0.274186 0.794908and

nlm["ParameterConfidenceIntervalTable"]

Estimate Standard Error Confidence Intervala -0.214093 1.03821 {-2.8829, 2.45472}

b 0.123774 0.211925 {-0.420995, 0.668544}

c 0.549276 2.00329 {-4.60035, 5.6989}

The method does not need initial guess value.9- 3- 6 Barycentre method

Employing fixed point method,

solB = FixedPointList[g[#] &, {0.3, 0.3, 0.3},

2000, SameTest (Norm[#1 - #2] < 10^-12 &)]; // AbsoluteTiming

{0.117267, Null}

The result is


solB // Last

{-0.214093, 0.123774, 0.549276}

The number of iterations is

Length[solB]

1096

In Table 9.1 we summarize these resultsTable 9 .1

Method Computing time (s) Number of iterationsExtended Newton (λ = 0) not converging -

Extended Newton (λ = 0.5) 0.0393 39Local minimization

(PrincipalAxis )

0.0057 -

Global minimization *

(DifferentialEvolution )

0.2240 -

Levenberg - Marquardt* 0.0440 15Barycentre 0.1170 1096

*Remark: initial guess was not required

One can see that Barycentre method is not the fastest - consider that the error limit is very low ϵ ≤ 10-12 - but farmore simple to implement it than the other methods.

9- 4 Study global convergence

Let us study the robustness of the Barycentre method. Now we create 8000 initial conditions laying in a cuboid of[-100 000, 100 000]3.

init = RandomReal[{-100000, 100000}, {8000, 3}];

Show[{ListPointPlot3D[init, BoxRatios {1, 1, 1}],

ListPointPlot3D[{solB // Last}, PlotStyle Directive[Red, PointSize[0.02]]]}]

-100 000-50 000

050 000

100 000

-100 000

-50 000

0

50 000100 000

-100 000

-50 000

0

50 000

100 000


Fig. 9.4 Random initial values and the solution (red point)

Let us employ parallel computation

LaunchKernels2 $ProcessorCount // Quiet;

DistributeDefinitions[g, init];

AbsoluteTiming[solinit = Map[Last[#] &, ParallelMap[

FixedPointList[g[#] &, #, 2000, SameTest (Norm[#1 - #2] < 10^-12 &)] &, init]];]

{218.673, Null}ListPlot[{Transpose[solinit][[1]], Transpose[solinit][[2]], Transpose[solinit][[3]]},

Joined True, Frame True, Axes None]

0 2000 4000 6000 8000

-0.2

0.0

0.2

0.4

Fig. 9.5 Convergence of the solution of the Barycentre method

Fig. 9.5 shows that the Barycentre method has been converged with all of the 8000 initial guess

9- 5 Barycentre Method with relaxation

It seems that employing relaxation the Barycentre method can run faster :

xk+1 = (1 + λ) g xk - λ xk

λ = 0.5;

DistributeDefinitions[g, init, λ];

AbsoluteTiming[

solinit = Map[Last[#] &, ParallelMap[FixedPointList[( (1 + λ ) g[#] - λ #) &, #,

2000, SameTest (Norm[#1 - #2] < 10^-12 &)] &, init]];]

{152.295, Null}


ListPlot[{Transpose[solinit][[1]], Transpose[solinit][[2]], Transpose[solinit][[3]]},

Joined True, Frame True]

0 200 400 600 800 1000

-0.2

0.0

0.2

0.4

Fig. 9.6 Convergence of the solution of theBarycentre method with relaxation

Indeed with relaxation the running time may be reduced by roughly 25 %.

9- 6 Example9- 6- 1 Problem definition

Let us see a more realistic problem! We consider the Stuttgart Central Test Network.

Haussman str. (1324.238 m)

K1

Schlossplatz (566.864 m)

Dach FH (269.231 m)

Dach LVM (400.584 m)

Liederhalle (430.529 m)

Linden museum (364.980 m)Eduardpfeifer (542.261 m)

Fig. 9.7 Stuttgart Central Test Network

Let us see a more realistic problem! We consider the Stuttgart Central Test Network.The prototype of the equations,

e = (xi - x0)2 + (yi - y0)2 + (zi - z0)2 - si2;

The GPS coordinates (Xi, Yi, Zi) in the global reference frame F○ (X, Y, Z), i = 1,...,7. are in Table 9.2


Table 9 .2 The GPS coordinates of the stations

Stations X (m) Y (m) Z (m)K1 (reference point) 4 157 066.1116 671 429.6655 4 774 879.3704

1 4 157 246.5346 671 877.0281 4 774 581.63142 4 156 749.5977 672 711.4554 4 774 981.54593 4 156 748.6829 671 171.9385 4 775 235.54834 4 157 066.8851 671 064.9381 4 774 865.82385 4 157 266.6181 671 099.1577 4 774 689.85366 4 157 307.5147 671 171.7006 4 774 690.56917 4 157 244.9515 671 338.5915 4 774 699.9070

Then the input data

datan = {x1 -> 4157246.5346, y1 -> 671877.0281, z1 -> 4774581.6314, s1 -> 566.8635,

x2 -> 4156749.5977, y2 -> 672711.4554, z2 -> 4774981.5459, s2 -> 1324.2380,

x3 -> 4156748.6829, y3 -> 671171.9385, z3 -> 4775235.5483, s3 -> 542.2609,

x4 -> 4157066.8851, y4 -> 671064.9381, z4 -> 4774865.8238, s4 -> 364.9797,

x5 -> 4157266.6181, y5 -> 671099.1577, z5 -> 4774689.8536, s5 -> 430.5286,

x6 -> 4157307.5147, y6 -> 671171.7006, z6 -> 4774690.5691, s6 -> 400.5837,

x7 -> 4157244.9515, y7 -> 671338.5915, z7 -> 4774699.9070, s7 -> 269.2309

};

In our case m = 7.

n = 3; m = 7;

Now, we consider the direct numerical solution of the overdetermined system,

X = Table[{xi, yi, zi}, {i, 1, m}] /. datan

4.15725× 106, 671877., 4.77458× 106,4.15675× 106, 672711., 4.77498× 106, 4.15675× 106, 671172., 4.77524× 106,4.15707× 106, 671065., 4.77487× 106, 4.15727× 106, 671099., 4.77469× 106,4.15731× 106, 671172., 4.77469× 106, 4.15724× 106, 671339., 4.7747× 106

Y = Table[si, {i, 1, m}] /. datan

{566.864, 1324.24, 542.261, 364.98, 430.529, 400.584, 269.231}9- 6- 2 Direct Least Square via Global Minimization

obj = Total[Table[e2, {i, 1, m}]]

-s12 + (-x0 + x1)2 + (-y0 + y1)2 + (-z0 + z1)22 +-s22 + (-x0 + x2)2 + (-y0 + y2)2 + (-z0 + z2)22 + -s32 + (-x0 + x3)2 + (-y0 + y3)2 + (-z0 + z3)22 +-s42 + (-x0 + x4)2 + (-y0 + y4)2 + (-z0 + z4)22 + -s52 + (-x0 + x5)2 + (-y0 + y5)2 + (-z0 + z5)22 +-s62 + (-x0 + x6)2 + (-y0 + y6)2 + (-z0 + z6)22 + -s72 + (-x0 + x7)2 + (-y0 + y7)2 + (-z0 + z7)22

AbsoluteTiming[

solN = NMinimize[obj /. datan, {x0, y0, z0}, Method "DifferentialEvolution"];]

{0.204901, Null}solN

0.00316074, x0 4.15707× 106, y0 671430., z0 4.77488× 106NumberForm[solN, 12]

0.00316074276896, x0 4.15706611153× 106, y0 671429.665479, z0 4.77487937031× 106

Initial guess is not needed!


9- 6- 2 Levenberg Marquardt method

data = MapThread[Flatten[{#1, #2}] &, {X, Y}]

4.15725× 106, 671877., 4.77458× 106, 566.864,4.15675× 106, 672711., 4.77498× 106, 1324.24,4.15675× 106, 671172., 4.77524× 106, 542.261,4.15707× 106, 671065., 4.77487× 106, 364.98,4.15727× 106, 671099., 4.77469× 106, 430.529,4.15731× 106, 671172., 4.77469× 106, 400.584,4.15724× 106, 671339., 4.7747× 106, 269.231

Applying built in function of Mathematica

dataP = SetPrecision[data, 20];

AbsoluteTiming{nlm, n} = Block{m = 0},

NonlinearModelFitdataP, (x - a)2 + (y - b)2 + (z - c)2 , {a, b, c}, {x, y, z},

StepMonitor m++, Method "LevenbergMarquardt", WorkingPrecision 20, m;

{0.0348059, Null}

Then the model with the estimated parameters

Normal[nlm][[1]]

-4.1570661114736358615× 106 + x2 +(-671429.66548915455588+ y)2 + -4.7748793702615594357× 106 + z2

NumberForm[%, 12]

-4.15706611147× 106 + x2 + (-671429.665489+ y)2 + -4.77487937026× 106 + z2

The number of the iteration step

n

40

The statistics of the estimation

NumberForm[nlm["ParameterTable"], 10]

Estimate Standard Error t- Statistic P- Valuea 4.157066111 × 106 0.000117317386 3.5434357 × 1010 3.8058620 × 10-42

b 671429.6655 0.0000178972897 3.7515718 × 1010 3.0289927 × 10-42

c 4.774879370 × 106 0.000111421357 4.2854256 × 1010 1.7789978 × 10-42

and

NumberForm[nlm["ParameterConfidenceIntervalTable"], 10]

Estimate Standard Error Confidence Intervala 4.157066111 × 106 0.000117317386 4.157066111 × 106, 4.157066112 × 106

b 671429.6655 0.0000178972897 {671429.6654, 671429.6655}

c 4.774879370 × 106 0.000111421357 4.774879370 × 106, 4.774879371 × 106

The method does not need initial guess value.


sol = {x0 -Normal[nlm][[1]][[1]][[1]][[1]],

y0 -Normal[nlm][[1]][[2]][[1]][[1]], z0 -Normal[nlm][[1]][[3]][[1]][[1]]}

x0 4.1570661114736358615× 106,y0 671429.66548915455588, z0 4.7748793702615594357× 106

obj /. SetPrecision[datan, 20] /. sol

0.00654989459- 6- 3 Barycentre method

Employing fixed point method,

X1 = SetPrecision[X, 20]; Y1 = SetPrecision[Y, 20];

g[x_] := TotalMapThread #1 +x - #1

Norm[#1 - x]#2 &, {X, Y} Length[Y]

solB = FixedPointList[g[#] &, {1, 1, 1}, 10000, SameTest (Norm[#1 - #2] < 10^-12 &)]; //

AbsoluteTiming

{0.490247, Null}

The result is

soli = solB // Last

4.15707× 106, 671430., 4.77488× 106NumberForm[%, 12]

4.15706611147× 106, 671429.665489, 4.77487937026× 106


Length[solB]

6666sol = {x0 soli[[1]], y0 soli[[2]], z0 soli[[3]]}

x0 4.15707× 106, y0 671430., z0 4.77488× 106obj /. datan /. sol

0.00655531

The solution is very precise, although the running time long. Let us test the robustness by employing a very badinitial guess

solB = FixedPointList[g[#] &, {-1000, -1000, -1000},

10000, SameTest (Norm[#1 - #2] < 10^-12 &)]; // AbsoluteTiming

{0.488261, Null}

The result is

soli = solB // Last

4.15707× 106, 671430., 4.77488× 106NumberForm[%, 12]

4.15706611147× 106, 671429.665489, 4.77487937026× 106


Length[solB]

6666


sol = {x0 soli[[1]], y0 soli[[2]], z0 soli[[3]]}

x0 4.15707× 106, y0 671430., z0 4.77488× 106obj /. datan /. sol

0.00655531

Indeed, the Barycenter method is very robust, insensitive on the initial guess values, although it needs relativelylong computation time.


barycentre method 09 -...

Documents