DIFFICULTY IN OPTIMIZATION FUNCTIONS OF MATLAB AND HOW TO ANALYZE

H.A Parhusip Center of Applied Science and Mathematics

Universitas Kristen Satya WacanaJl. Dipenegoro 52-60, Salatiga, 50711, Central Java

Telephone number : 0062-298-321212Email address : hannaariniparhusip@ yahoo.co.id

Abstract

This paper presents an example of using optimization function in MATLAB. The used data is considered as a smooth function of 3 variables. Unlike in a literature, one needs to design a smooth function in order to start with an optimization. After the parameters are obtained the optimization is employed to obtain the minimizer. The obtained objective function is noncoercive. Therefore the minimizer is obtained by solving the nonlinear system of the Lagrangian function which is constructed as a perturbed objective function. The optimizer can not be considered as the best solution since the Hessian is not positive definite.

Keywords : noncoercive function, Jacobian, least square, Hessian, singularity

Introduction

Understanding output of MATLAB’s program is not an easy task. After using any functions in MATLAB correctly, one needs to relate with an expected result. Therefore theoretical background that may be used in the functions must be known.

MATLAB is also used for undergraduate students for many subjects in particular computation mathematics, numerical methods, data analysis, linear algebra, nonlinear optimization and mathematical modeling which is based on differential equations. Having very much time in theories in those subjects, students are very lack on applications. On the other hand, giving practical applications by too much in programming, students have very low theoretical background for those

subjects and students could not also describe formally mathematical reasons in the given output by MATLAB.

There are many authors addressed on developing the MATLAB code for a particular purposes. For data single directional (azimuthal) in geosciences for instance, Jones [1] proposed a MATLAB script since for this special purpose one may not have the related function from the standard MATLAB Toolbox. The lsqnonlin (one of optimization functions in MATLAB) can not be used for the directional azimuthal data. The given data is shown in Table 1. The goal of this measurement is to identify the maximum protein that can be obtained from various observations.

Table 1The used data for the optimization study (Observed by. Y.Martono at Lab. Chemistry, SWCU,2011)

(B)Percentage of

yeast(Y)

ProteinDay

1

ProteinDay

2

ProteinDay

3

ProteinDay

41 5 0.714 0.75 1.76 3.221 7.5 1.33 1.88 0.21 1.33

1 12.5 0.88 1.38 0.93 2.38

Reseach Method

Some authors refer to linear regression to model percentage change over time (Suissa,et.all,1989). In the case of molecular protein, this approachs can not be applied since one needs modeling three-dimensional detail by homology (Sander and Scheider,1991). Since the number of the given data is small, The first approach in this paper is to present the percentage protein as a smooth function and then we introduce the optimization procedures. Let us introduce dimensionless variables,i.e t : time ; B : percentage of beans ;Y: percentage of yeast, P : percentage of protein. We assume that the given data will be a continuous function as

, unknown parameters (1)

Note that we have made data into dimensionless to proceed futher computations. Using the standard least square means that we have to

minimize the residual function . Following

the standard optimization to find the critical means that we have to solve the system obtained by . In more details,

this leads to solve and simultaneously,i.e

; (2a)

:= =0. (2b)

Any algorithm with an iterative procedure requires an initial

guesses of solutions. There are several well-known algorithms such as

Newton-method, Broyden method, trust-region and using evolutionary

algorithm [2]. The last algorithm transforms the system of equations into

a multi-objective optimization problem. The ill-posed nonlinear

equations may also occur. Buong and Dung [3] have provided a

theoretical study on this particular problem by doing a regularization on

the problem which based on Tikhonov regularization method. However

the problem was designed in the variational form which was too much

beyond from our study in this paper. Thus, this paper will use the

standard Newton method. The Newton method solves a general system

with a formula

(3)

which provides a nonsingularity of matrix Jacobian on each iteration step. The formula (3) is employed to Eq. (2a)-(2b) by deriving their derivatives manually each term on these equations. Therefore one needs

(4)

with = ,

= ,

,

= .

If we consider carefully, the Jacobian matrix (4) is also the

Hessian matrix of R. Therefore the observation of singularity of (4) leads

to the observation of positive definiteness of the Hessian matrix R. Thus

it inspires us to use the condition that Hessian matrix must be positive

definite in order to work with Eq.(3). In a short expression means that we

have to satisfy and - > 0. Observe that the

second condition is the determinant of which is not allowed to be

zero since it guarantees the existence of the invers of on each

interation, i.e

(5)

Therefore we only use the Eq.(5) to have the existence of Eq.(4). Thus in

order to use Newton method, we need to include - > 0 for

each iteration. Since we seek the solution of , then the

iteration stops as .

Result and Discussion

Using the lsqnonlin,m from MATLAB, the function (1) is obtained as

, (8)

with the error 0.9008 %. This result is considerable good. Since the data

are not too large, we may make a list of each value of the approximation

compared to the observation (shown in Table 1) and it is depicted in

Figure 1. Now,one may proceed the optimization goal ,i.e subject

to the constraints and t > 0.

Unfortunately, the standard procedure in literature for an optimization is

a minimization case as shown by Peressini,et.all [4] and the function in

MATLAB is usually proposed a minimization solver. Therefore one may

set up the optimization model as :

MATLAB provides fmincon.m function to solve this minimization problem.

Up to this step, the program can not give a reasonable solution. One reason is

due to the property of the objective function which is not coercive (i.e

) If the objective function is coercive then it is guaranteed at

least there exists one minimizer (Peressini,et.all,1988). How do we prove ?.

Table 1 Comparisan measurement (first column) and approximation (second column) of protein (dimensionless)

0.2217 0.11450.4130 0.13640.2733 0.17010.2329 0.27750.5839 0.33070.4286 0.41250.5466 0.46580.0652 0.55510.2888 0.69241.0000 0.67280.4130 0.80180.7391 1.0000

Figure 1. The illustration of

Note that which tends to

minus infinity. Thus it is not coercive. What can we do ?. We can not change

the objective function since it has been obtained from the previous

optimization process. Thank to Peressini(et all,1988), that it offers the idea to

perturb the objective function. More specifically, for each define

. (7)

Unfortunately, we need also property that is convex

(Peressini,et.all,1988,page 52) i.e

, in a convex set subset of . (8)Let us try to study this condition for all values in the given domain and we

write into more general form,i.e = . As a consecuence, one needs

to compute

Using the values, in the given domain we obtain that the

condition (8) is satisfied for . This is shown on Figure 2. Thus, the

property (8) is satisfied and hence the function = is

convex. Clearly, the is a quadratic function which is known to be

convex. Therefore the perturbed function defined Eq.( 7) is convex. We

are done to show that is convex. The proof that is coercive is

not shown here for simplicity, which is mentioned into detail in Peressini

(page 229). The optimization problem becomes

Figure 2. Illustration of obtained by each value of >from the given domain.Penalty Method with a noncoercive objective function

We need to construct the Lagrangian for as follows

, m = the number of constraints.

This Lagrangian becomes an unconstraint objective function with additional 2

unknown parameters.We can use the function fminunc.m in MATLAB by

defining the Lagrangian function as the objective function. We have tried 5 sets

of initial guesses and the observation shows unreasonable answers. Analytic

observation will explain why the function fminunc.m and fmincon.m do not

give unique solutions.

One needs to construct analytically and solve it by

solver of nonlinear system equations. We have

; (9a)

; (9b)

(9c)

; (9d)

(9e)

We write the system as . One obtains

directly from Eq.(9e) and Eq.(9d) that =0;

respectively. Substitute these result to Eq,(9c), one yields .

As a result, to satisfy Eq.(9b). On the other hand since it will

make Eq. (9b) is undefined if . Therefore and it is free

variable. Thus the system has infinitely many solutions in the form

with . Moreover this conclusion is

practically useless. However this explains us that the Newton method

will not converge if this method is used blindly. By forcing tends

to zero (as shown in the Figure) , the solution is not a real vector after 747

iterations though the method works well by showing that tends to zero

as depicted by Figure 3. Thus the Newton method fails which agrees with

analytic result.

Figure 3. Illustration for each iteration

Conclusion

This paper has presented an example of using the optimization solver

provided by MATLAB. The used data is considered as a smooth function of 3

variables. After the parameters are obtained the optimization is employed to

obtain the minimizer of the noncoercive objective function.

The minimizer is obtained by solving the nonlinear system of the

Lagrangian function which is constructed as a perturbed objective function. The

optimizer can not be considered as the best solution since the Hessian is not

positive definite. One may avoid the computation of gradient by using ant

colony algorithm, particle swarm algorithm (Rao,2009).

AknowledgementThe author gratefully acknowledge to Yohanes Martono for supporting his data to this numerical work with MATLAB .

ReferencesBuong,N and Dung. D, (2009).Regularization for a Common Solution of a System of Nonlinear Ill-Posed Equations, Int. Journal of Math. Analysis,3(34), 1693-1699. Grosan, C and Abraham, A,(2008). Multiple Solutions for a System of Nonlinear Equations, International Journal of Innovative Computing, Information and Cotrol, x( x), ISSN 1349-4198.Jones, T.A, (2006). MATLAB functions to analyze directional (azimuthal) data-I:Single-sample inference, Computers & Geosciences 32 (2006) 166-175. Peressini, A.L, Sullivan, F.E, Uhl,J, (1988). The Mathematics of Nonlinear Programming, Springer Verlag, New York, Inc. Rao, S.S. (2009). Engineering Optimization, Theory and Practice, John Wiley & Sons, Inc., Hoboken, New Jersey.Sander C, and Scheider,R, (1991). Database of homology-derived protein structures and the structural meaning of sequence alignment, Protein,

9(1),56-68.Suissa S, Levinton C, Esdaile JM. (1989). Modeling percentage change: a potential linear mirage,J.Clin. Epidemiol, 42(9),843-848.