test problem generation for quadratic-linear pessimistic bilevel optimization

ISSN 1995-4239, Numerical Analysis and Applications, 2014, Vol. 7, No. 3, pp. 204–214. c© Pleiades Publishing, Ltd., 2014.Original Russian Text c© A.V. Orlov, A.V. Malyshev, 2014, published in Sibirskii Zhurnal Vychislitel’noi Matematiki, 2014, Vol. 17, No. 3, pp. 245–257.

Test Problem Generation for Quadratic-LinearPessimistic Bilevel Optimization

A. V. Orlov1* and A. V. Malyshev2**

1Institute of System Dynamics and Control Theory, Siberian Branch,Russian Academy of Sciences, ul. Lermontova 134, Irkutsk, 664033 Russia

2Luxand, Inc., 901 N. Pitt str. Suite 325 Alexandria, VA 22314 USAReceived May 15, 2013

Abstract—A generation method of quadratic-linear bilevel optimization test problems in a pes-simistic formulation is proposed and justified. Propositions about the exact form and the number oflocal and global pessimistic solutions in generated problems are proved.

DOI: 10.1134/S1995423914030033

Keywords: test problem generation, bilevel optimization, guaranteed (pessimistic) solution,kernel problems.

1. INTRODUCTION

Experimental verification of workability and effectiveness of new methods of solving computationalmathematics problems (in particular, optimization problems) and their comparisons with existingapproaches requires a sufficiently large range of test problems of different complexity and dimensions. Itis also desirable to have information about the structure and properties of test problems to estimate thelimits of applicability of examined methods. Following the classical monograph of Polyak [1], good testproblems should have a certain set of properties: they should be unified and commonly accepted; theyshould model typical difficulties for a given class of problems; they should be sufficiently compact; theyshould not have specific features that give advantages to this or that method; finally, the solution to thetest problem should be known.

The simplest method to construct test problems is (pseudo)random generation of problem data.However, this method is not always possible, and one cannot argue that thus-generated problems ensurethe above-mentioned properties. The use of this approach is justified only if the optimization problem hasa solution for all initial data and, say, the value of the objective function in this solution is known. Forexample, these conditions are satisfied by a nonconvex bilinear problem equivalent to the problem ofsearching for the Nash equilibrium in a bimatrix game [2].

Another method of choosing the field of test examples is the use of existing test collections andlibraries. For instance, the DIMACS library for discrete optimization problems [3], the Schittkowskiand Hock collection of problems of nonconvex (global) optimization [4, 5], the Floudas and Pardaloscollection [6], and the more recent Leyffer collection [7] including nonlinear mixed-integer problems,problems with equilibrium constraints, and vector optimization problems are well known and widelyused. However, it is not for all classes of problems that such libraries are available. Moreover, thismethod, in particular, for nonconvex optimization problems, is complicated by the fact that only the“best known solution” is available for most problems, whereas there is no information about the globalsolution.

An approach based on the development of special methods of generation (construction) of testproblems is devoid of the last drawback. In this case, the global solution to the test problem is known byconstruction, and it is often possible to control the properties of the constructed problem by changing the

*E-mail: [email protected]**E-mail: [email protected]

204

TEST PROBLEM GENERATION 205

generation parameters. In this direction, some interesting results were obtained by the team headed byMoshirvaziri [8–11], who proposed methods for construction concave minimization problems, reverse-convex optimization problems, and linear problems of bilevel programming, and also by the team headedby Vicente and Calamai [12–15], who studied several classes of problems of quadratic and bileveloptimization. An original method of generation of classes of test functions for global optimizationmethods, which allows construction of sets of nondifferentiable, continuously differentiable, and twicecontinuously differentiable functions of different complexity and dimensions, is also worth mentioning[16]. Certainly, this list of publications on the topic is far from being complete.

It should be noted that all works dealing with generation of bilevel optimization test problems ofvarious classes that could be found in the literature [10, 13, 14] consider only problems in an optimisticformulation [17]. In this case, it is assumed that the objective of the decision maker (DM) at the upperlevel can be correlated with the actions of the DM at the lower level. Nevertheless, such a situation israrely encountered in practical problems with a hierarchic structure; a more adequate modeling tool isconsidering problems in a guaranteed (pessimistic) formulation, where the upper level has to act on thebasis of the generalized principle of the guaranteed result [18]. Problems in this formulation are muchmore complicated from the viewpoint of finding their numerical solution and are not yet very popular,though there are some recent publication in this field [19–21].

In this connection, we believe that the development of new methods of construction of bilevel opti-mization test problems with a pessimistic solution is an urgent problem of operations research. It seemsreasonable to follow the way of the above-mentioned teams who worked with gradual complication of theconsidered problems and to start the study from one of the simplest classes of bilevel problems, namely,quadratic-linear problems (with a quadratic function at the upper level and a linear function at the lowerlevel).

For this purpose, the method proposed by Calamai and Vicente [13, 14] is developed to be appliedto problems of this class with a pessimistic solution. In our opinion, this method is the most attractiveone: for instance, in contrast to [10], implementation of this method does not require any complicatedoperations and solving auxiliary problems, except for elementary actions with matrices and vectors.

2. FORMULATION OF THE PROBLEM AND IDEAOF THE GENERATION METHOD

Let us consider a quadratic-linear bilevel problem with a pessimistic solution in the followingformulation:

W (x)�= sup

y

{F (x, y) | y ∈ Y∗(x)

}↓ min

x,

x ∈ X, Y∗(x)�= Argmin

y

{G(y) | y ∈ Y (x)

},

⎫⎪⎪⎬

⎪⎪⎭(BP)

where F (x, y)�= 1

2 〈x,Cx〉 + 〈c, x〉 − 12〈y,C1y〉 + 〈c1, y〉, G(y) = 〈d, y〉, X

�=

{x ∈ R

m | Ax ≤ a, x ≥0}

, Y (x)�=

{y ∈ R

n | A1x + B1y ≤ b, y ≥ 0}

, A ∈ Rp×m, A1 ∈ R

q×m, B1 ∈ Rq×n, c ∈ R

m, c1, d ∈R

n, a ∈ Rp, b ∈ R

q; C = C� ≥ 0 and C1 = C�1 ≥ 0 are non-negatively defined matrices; W (x) is the

estimation of the efficiency of the strategies of the upper-level player [18]. Problems of this kind werestudied from the viewpoint of constructing methods of their solution, in particular, in [19, 20].

It should be noted that of interest from the viewpoint of searching for the pessimistic solution are onlythose bilevel problems in which the solution to the lower-level problem is nonunique for certain feasiblevalues of x ∈ X. Indeed, in the opposite case, we have

∀x ∈ X Y∗(x) ={y(x)

}, y(x) : G(y(x)) < G(y) ∀ y ∈ Y (x), y = y(x),

hence, the minimum and maximum of the function F (x, ·) on the set Y∗(x) consisting of one point y(x)coincide:

W (x)�= sup

y

{F (x, y) | y ∈ Y∗(x)

}= F (x, y(x)) = inf

y

{F (x, y) | y ∈ Y∗(x)

}. (1)

NUMERICAL ANALYSIS AND APPLICATIONS Vol. 7 No. 3 2014

206 ORLOV, MALYSHEV

Based on the chain of equalities (1), we can conclude that, instead of minimizing the estimation of theefficiency W (x) of the upper-level player strategy, it is possible to minimize its criterion of efficiencyF (x, y) over the set of the variables (x, y), i.e., to search for the optimistic solution to the consideredproblem [17].

The proposed method of test problem generation of the form (BP) consists of three stages. The firststage of generation is the construction of the so-called bilevel kernel problems of small dimensions(with a nonunique solution at the lower level) and analytical search for all local and global pessimisticsolutions to these problems. At the second stage, a separable bilevel problem of a needed dimension isconstructed by uniting a finite number of kernel problems with different properties. Finally, at the thirdstage of the generation method, a special replacement of variables is applied to eliminate the separationproperty of the constructed problem. Each of these stages is described and justified below.

3. KERNEL PROBLEMS AND THEIR ANALYTICAL SOLUTIONS

Let us consider kernel problems within the analyzed class of bilevel problems of the following form:

W (x) = supy

{x2 − 8x + pky1 − 2y2

2 | y ∈ Y∗(x)}↓ min

x,

x ∈ [0; 6], Y∗(x)�= Argmin

y

{− y1 | y1 + y2 ≤ x, 0 ≤ y1 ≤ 3, y2 ≥ 0

},

⎫⎪⎪⎬

⎪⎪⎭(KPk)

where x ∈ R, y = (y1, y2) ∈ R2, and pk is a certain real parameter. In contrast to kernel problems from

[13, 14], where the variables are scalar at both levels, in problems of the form (KPk) we have to use twovariables at the lower level for the solution to the corresponding optimization problem to be nonunique.

To search for the solution to the problem (KPk), we first present the set Y∗(x) of the lower-levelproblem solutions for different values of x ∈ [0; 6] in an explicit form. Let us consider the lower-levelproblem included into the problem (KPk) separately:

y1 ↑ maxy

, y1 + y2 ≤ x, 0 ≤ y1 ≤ 3, y2 ≥ 0. (FKP(x))

The problem (FKP(x)) is a problem of linear programming (LP) in the space R2. Therefore, its solution

can be obtained graphically [22].

In Figs. 1 and 2, the feasible set of the problem (FKP(x)) is denoted by Y (x), and the vector of thegradient of the objective function of the problem (FKP(x)) is denoted by ∇. As can be easily seen, theset Y∗(x) of solutions to the problem (FKP(x)) for x ∈ [0; 3] consists of one point (y1, y2)=(x, 0); if

Fig. 1. Y∗(x) for x ≤ 3. Fig. 2. Y∗(x) for x > 3.



x ∈ [3; 6], this set is a segment{(3, y2) | 0 ≤ y2 ≤ x−3

}. If x = 3, this segment degenerates into the

point (x, 0). Thus, the set of solutions to the problem (FKP(x)) has the following form:

Y∗(x) =

⎧⎪⎨

⎪⎩

{(x, 0)

}if x ∈ [0; 3],

{(3, y2) | 0 ≤ y2 ≤ x − 3

}if x ∈ [3; 6].

(2)

Let us return to the solution to the problem (KPk). Using representation (2) of the set Y∗(x), we canwrite the expression for the objective function of the problem (KPk) as

W (x) =

⎧⎪⎨

⎪⎩

x2 − 8x + pkx if x ∈ [0; 3],

supy

{x2 − 8x + 3pk − 2y2

2 | 0 ≤ y2 ≤ x − 3}

if x ∈ [3; 6].(3)

Note that the function under the sign sup in (3) with x ∈ [3; 6] decreases with respect to y2 ≥ 0.Therefore, its least upper bound is reached at the point y2 = 0:

supy

{x2 − 8x + 3pk − 2y2

2 | 0 ≤ y2 ≤ x − 3}

= x2 − 8x + 3pk. (4)

Uniting (3) and (4), we obtain the explicit expression for the objective function W (x) of the problem(KPk):

W (x) =

⎧⎪⎨

⎪⎩

x2 − 8x + pkx if x ∈ [0; 3],

x2 − 8x + 3pk if x ∈ [3; 6].

Thus, W (x) is a continuous piecewise-quadratic function. One can demonstrate that the problemhas two stationary points at pk ∈ [2; 8]: x = 4 − 0.5pk ∈ [0; 3] and x = 4 ∈ [3; 6]; for other values of pk,it has one stationary point x = 4. In what follows, we consider only three fixed values of the parameterpk = 3, 4, 6 (k = 1, 2, 3), which lie within the segment [2; 8].

Figures 3, 4, and 5 show the functions W (x) corresponding to three different values of the parameterpk : p1 = 3, p2 = 4, and p3 = 6. We can see that the problem (KP1) (the problem (KPk) for pk = p1)has a local solution at the point x = 2.5 and a global solution at the point x = 4; the problem (KP2)has two global solutions at the points x = 2 and x = 4 and has no local solutions that are not globalsolutions; the problem (KP3) has a global solution at the point x = 1 and a local solution at the pointx = 4.

Fig. 3. W (x) for pk = p1 = 3.


208 ORLOV, MALYSHEV

Fig. 4. W (x) for pk = p2 = 4. Fig. 5. W (x) for pk = p3 = 6.

It should be noted that, performing actions similar to those described above, it is also possible to findoptimistic solutions in the problems (KPk), but this study is outside the scope of the present paper andis not described here.

4. COMBINATION OF KERNEL PROBLEMS AND CONSTRUCTIONOF A SEPARABLE BILEVEL PROBLEM

Let us consider a bilevel problem obtained by uniting an arbitrary number r of kernel problems (KPk),k ∈ {1, 2, 3}, which belongs to the class of bilevel problems (BP):

supy

{ r∑

i=1

F (xi, y2i−1, y2i; pi) | y ∈ Y∗(x)}

↓ minx

, x ∈ Π[0; 6],

Y∗(x)�= Argmin

y

{ r∑

i=1

G(y2i−1) | (y2i−1, y2i) ∈ Y (xi), i = 1, . . . , r}

,

⎫⎪⎪⎪⎪⎬

⎪⎪⎪⎪⎭

(SP)

where F (xi, y2i−1, y2i; pi)�= x2

i − 8x + piy2i−1 − 2y22i, pi ∈

{p1, p2, p3

}, G(y2i−1)

�= −y2i−1, Π[0; 6]

�=

{x = (x1, . . . , xr) | xi ∈ [0; 6], i = 1, . . . , r

}, Y (xi)

�=

{(y1, y2) | y1 + y2 ≤ xi, y1 ≤ 3, y1 ≥ 0, y2 ≥

0}

, i = 1, . . . , r, x ∈ Rr, y ∈ R

2r.

It should be noted that the structure of the problem (SP) is such that the number of variables atthe lower level is twice the number of variables at the upper level. This property does not limit thevariety of generated test problems because this variety is ensured by different values of the parameter pi.Moreover, such a situation (where the number of variables at the lower level is greater than the numberof variables at the upper level) is natural for practical hierarchical problems where several lower-layerplayers depending on the upper level are often modeled.

It should be further noted that the lower-level problem of the bilevel problem (SP),r∑

i=1

G(y2i−1) ↓ miny

, (y2i−1, y2i) ∈ Y (xi), i = 1, . . . , r, (FSP(x))

is a separable optimization problem. Its solution can be found in the following way (see also [23]).

Lemma 1. The vector y = (y1, y2, . . . , y2r) is a solution to the problem (FSP(x)) if and only if itscomponents (y2i−1, y2i) are solutions to the problems (FKP(xi)), i = 1, . . . , r.

Proof. Necessity. Let y ∈ Sol(FSP(x)), then the following inequality is valid:r∑

i=1

G(y2i−1) ≤r∑

i=1

G(y2i−1) ∀ y : (y2i−1, y2i) ∈ Y (xi), i = 1, . . . , r. (5)



For an arbitrary index j ∈ {1, . . . , r}, by virtue of separateness of constraints, vectors of the formy(y2j−1, y2j) := (y1, . . . , y2j−2, y2j−1, y2j , y2j+1, . . . , y2r) :(y2j−1, y2j) ∈ Y (xj) are feasible in the prob-lem (FSP(x)). Therefore, inequality (5) remains valid at y = y(y2j−1, y2j):

r∑

i=1

G(y2i−1) ≤ G(y2j−1) +∑

i�=j

G(y2i−1) ∀ (y2j−1, y2j) ∈ Y (xj). (6)

Simplifying (6), we obtain an inequality equivalent to the inclusion

(y2j−1, y2j) ∈ Sol(FKP(xj)).

Sufficiency. Let the following inclusions be valid: (y2i−1, y2i) ∈ Sol(FKP(xi)) ∀ i = 1, . . . , r. Then thepoint y is feasible in the problem (FSP(x)), and the following inequalities are valid:

G(y2i−1) ≤ G(y2i−1) ∀ (y2i−1, y2i) ∈ Y (xi), i = 1, . . . , r. (7)

Summing up inequalities (7) with respect to i, we obtain inequality (5), which, by virtue of feasibility ofthe point y in the problem (FSP(x)), is equivalent to the required inclusion y ∈ Sol(FSP(x)).

To describe the structure of local solutions to the problem (SP), we need the following auxiliarystatement.

Lemma 2. For all values of x ∈ Π[0; 6], the following equalities are satisfied:

supy

{ r∑

i=1

F (xi, y2i−1, y2i; pi) | y ∈ Y∗(x)}

= maxy

{ r∑

i=1

F (xi, y2i−1, y2i; pi) | y ∈ Y∗(x)}

=r∑

i=1

max(y2i−1,y2i)

{F (xi, y2i−1, y2i; pi) | (y2i−1, y2i) ∈ Y∗(xi)

}. (8)

Proof. Let us consider an auxiliary problem for x ∈ Π[0; 6]:r∑

i=1

F (xi, y2i−1, y2i; pi) ↑ maxy

, y ∈ Y∗(x). (P1(x))

By virtue of Lemma 1, the inclusion y ∈ Y∗(x) is equivalent to the inclusions (y2i−1, y2i) ∈ Y∗(xi)∀ i = 1, . . . , r; therefore, the problem (P1(x)) can be written in the form of the following equivalentseparable optimization problem:

r∑

i=1

F (xi, y2i−1, y2i; pi) ↑ maxy

, (y2i−1, y2i) ∈ Y∗(xi), i = 1, . . . , r. (P1(x))

For this problem, we can prove a statement similar to Lemma 1: y ∈ Sol(P1(x)) if and only if(y2i−1, y2i) ∈ Argmax(y2i−1,y2i)

{F (xi, y2i−1, y2i; pi) | (y2i−1, y2i) ∈ Y∗(xi)

}∀ i = 1, . . . , r.

Owing to the latter inclusion, the second equality in (8) is satisfied.The first equality in (8) is valid by virtue of the structure of the set Y∗(xi) of solutions to the problems

(FKP(xi)), i = 1, . . . , r, which, depending on the choice of xi ∈ [0; 6], is either a point or a segment(nonempty compact set); therefore, the exact upper edge in (8) is reached.

Remark. By virtue of the first equality in (8), the problem (SP) can be written as

maxy

{ r∑

i=1

F (xi, y2i−1, y2i; pi) | y ∈ Y∗(x)}

↓ minx

,

x ∈ Π[0; 6], Y∗(x)�= Argmin

y

{ r∑

i=1

G(y2i−1) | (y2i−1, y2i) ∈ Y (xi), i = 1, . . . , r}

.

⎫⎪⎪⎪⎪⎬

⎪⎪⎪⎪⎭

(SP)


210 ORLOV, MALYSHEV

Proposition 1. The point x ∈ Rr is a local solution to the problem (SP) (x ∈ LocSol(SP)) if and

only if its components xi are local solutions to the problems (KPk) corresponding to the valuespk = pi, i = 1, . . . , r, used in generating the problem (SP).

Proof. Necessity. Let x ∈ Π[0; 6] and ∃ ε > 0 such that the following inequality is valid for all x ∈Π[0; 6] ∩ U(x, ε), where U(x, ε)

�=

{x | ‖x − x‖ ≤ ε

}:

maxy

{ r∑

i=1

F (xi, y2i−1, y2i; pi) | y ∈ Y∗(x)}

≤ maxy

{ r∑

i=1

F (xi, y2i−1, y2i; pi) | y ∈ Y∗(x)}

. (9)

Using the second equality in (8) (see Lemma 2), we can write inequality (9) in a component-by-component form by placing the “max” signs under the sign of summation. As this inequality is valid∀x ∈ Π[0; 6]: ‖x − x‖ ≤ ε, we assume that xi = xi, i = j, where j ∈ {1, . . . , r} is a certain fixed index.As a result, we obtain

∀xj ∈ [0; 6] : |xj − xj| ≤ ε

max(y2j−1,y2j)

{F (xj , y2j−1, y2j; pj) | (y2j−1, y2j) ∈ Y∗(xj)

}

≤ max(y2j−1,y2j)


}, (10)

which, in view of the arbitrary choice of j ∈ {1, . . . , r}, is equivalent to the validity of the inclusionsxi ∈ LocSol(KPk), pk = pi, i = 1, . . . , r.Sufficiency. Let now xi ∈ LocSol(KPk), k : pk = pi, i = 1, . . . , r, or, which is the same, xj ∈ [0; 6] andinequality (10) is satisfied ∀j ∈

{1, . . . , r

}for ε = εj . Assuming that ε = min

{ε1, . . . , εr

}and summing

up inequalities (10) with respect to j, we obtain

∀x ∈ Π[0; 6] : |xj − xj | ≤ ε, j = 1, . . . , r,r∑

j=1max

(y2j−1,y2j)


}

≤r∑

j=1max

(y2j−1,y2j)

{F (xj , y2j−1, y2j ; pj) | (y2j−1, y2j) ∈ Y∗(xj)

}.

Hence, taking into account Lemma 2, we conclude that inequality (9) is satisfied ∀x ∈ Π[0; 6] :|xi − xi| ≤ ε, i = 1, . . . , r. In view of the definition of the Euclidean norm, we can easily see that

U(x, ε) ⊂{x | |xi − xi| ≤ ε, i = 1, . . . , r

}.

Therefore, inequality (9) is also satisfied ∀x ∈ Π[0; 6] ∩ U(x, ε). In other words, the inclusion x ∈LocSol(SP) is valid.

For global solutions to the problem (SP), there exists a proposition similar to Proposition 1.

Proposition 2. The point x ∈ Rr is a (global) solution to the problem (SP) if and only if its

components xi are solutions to the problems (KPk) corresponding to the values of the parameterspk = pi, i = 1, . . . , r used in generating the problem (SP).

To prove this proposition, it is sufficient to set ε = +∞ in the proof of Proposition 1 and to assumethat U(x, ε) = R

r. �The next proposition describes the dependence of the number of local and global solutions to the

problem (SP) on the set of problems (KPk) used for its generation.

Proposition 3. The problem (SP) that includes r1 kernel problems of the form (KP1), r2 kernelproblems of the form (KP2), and r3 kernel problems of the form (KP3), so that r1 + r2 + r3 = r,has 2r local solutions among which 2r2 solutions are global solutions to this problem.



Proof. By virtue of Proposition 1, each local solution to the problem (SP) can be presented as x =(x1, . . . , xr), where xi is a certain local solution to the problem (KPk) with pk = pi, i = 1, . . . , r. As thecomponents of the vector x are independent of each other and each kernel problem (KPk) has exactlytwo local solutions, the direct product rule (see, e.g., [24]) says that the problem (SP) has exactly 2r

local solutions.Similarly, by virtue of Proposition 2, each global solution to the problem (SP) can be presented as

x∗ = (x∗1, . . . , x

∗r). In this case, r1 + r3 components of the vector x∗ are fixed because the kernel problems

(KP1) and (KP3) have one global solution each. Each of the remaining r2 components of the vector x∗can take two different values. Using the direct product rule, as above, we conclude that the problem(SP) has exactly 2r2 global solutions.

5. TRANSFORMATION OF THE SEPARABLE PROBLEM

At the final stage of the generation method, it is necessary to eliminate separateness to expandthe variety of the constructed problems of the form (BP). For this purpose, we use the substitutionx = MxM−1

x x, y = MyM−1y y, where Mx ∈ R

m×m : det Mx = 0 and My ∈ Rn×n : det My = 0 (for the

problem (SP), m = r and n = 2r) and replacement of variables z = M−1x x and u = M−1

y y. Applyingthe above-described transformation to an arbitrary problem of the class (BP), we obtain the problem

supu

{F ′(z, u) | u ∈ Y ′

∗(z)}↓min

z, z ∈ X ′, Y ′

∗(z)�= Argmin

u

{⟨M�

y d, u⟩| u ∈ Y ′(z)

}, (BP ′)

where F ′(z, u)�= 1

2

⟨z,M�

x CMxz⟩

+⟨M�

x c, z⟩− 1

2

⟨u,M�

y C1Myu⟩

+⟨M�

y c1, u⟩

, X ′ �=

{z ∈ R

m |AMxz ≤ a,Mxz ≥ 0

}, Y ′(z)

�=

{y ∈ R

n | A1Mxz + B1Myu ≤ b, Myu ≥ 0}

.

Using the standard definitions of the linear algebra, one can easily show that the matrices C andC1 remain non-negatively determined after the transformations M�

x CMx and M�y C1My . Therefore, the

problem (BP ′) obtained after replacement of variables remains in the class of quadratic-linear bilevelproblems (BP). To find the relationship between the solutions (local and global ones) to the originalproblem (BP) and to the transformed problem (BP ′), we need the following auxiliary statements.

Lemma 3. The following relations are valid:

(i) x ∈ X ⇔ M−1x x ∈ X ′,

(ii) y ∈ Y (x) ⇔ M−1y y ∈ Y ′(M−1

x x),

(iii) y ∈ Y∗(x) ⇔ M−1y y ∈ Y ′

∗(M−1x x).

Proof. (i) Using the definition of the set X and the obvious equality E = MxM−1x , we can write the

inclusion x ∈ X in the form x ∈{x | AMxM−1

x x ≤ a, MxM−1x x ≥ 0

}, which, by definition of the set

X ′, is equivalent to the inclusion M−1x x ∈ X ′.

(ii) Is proved in a similar manner.

(iii) Let y ∈ Y∗(x). Then⟨d,MyM

−1y y

⟩≤

⟨d,MyM

−1y y

⟩∀ y ∈ Y (x). By virtue of the property of

the scalar product and item (ii) of this Lemma, the last inequality yields⟨M�

y d,M−1y y

⟩≤

⟨Myd,

M−1y y

⟩∀ (M−1

y y) ∈ Y (M−1x x), which is equivalent to the inclusion M−1

y y ∈ Y ′∗(M

−1x x).

Lemma 4. In the case of consistency of the matrix and vector norms and an arbitrary choice ofε > 0, we have

(i)(M−1

x x)∈ U

(M−1

x x, ε/‖Mx‖)⇒ x ∈ U(x, ε),

(ii) x ∈ U(x, ε/

∥∥M−1

x

∥∥)

⇒(M−1

x x)∈ U

(M−1

x x, ε)

.

In other words, the point x lies in the neighborhood of the point x if and only if the point M−1x x

belongs to a certain neighborhood of the point M−1x x.


212 ORLOV, MALYSHEV

Proof. (i) Let(M−1

x x)∈ U

(M−1

x x, ε/‖Mx‖). Then

∥∥M−1

x x − M−1x x

∥∥ ≤ ε/‖Mx‖. In view of consis-

tency of the matrix and vector norms, the last inequality yields∥∥Mx

(M−1

x x − M−1x x

)∥∥ ≤ ‖Mx‖∥∥M−1

x x − M−1x x

∥∥ ≤ ‖Mx‖ε

‖Mx‖. (11)

Chain (11) yields the inequality ‖x − x‖ ≤ ε, which is equivalent to the inclusion x ∈ U(x, ε).(ii) Is proved in a similar manner.

The next proposition justifies the transition from the problem (SP) (which belongs to the class ofbilevel problems (BP)) to the problem (BP ′), which allows us to eliminate separateness.

Proposition 4. The point x is a local (global) solution to the problem (BP) if and only if the pointM−1

x x is a local (global) solution to the problem (BP ′).

Proof. Necessity. Let x ∈ LocSol(BP), then x ∈ X and, moreover, ∃ ε > 0:

supy

{F (x, y) | y ∈ Y∗(x)

}≤ sup

y

{F (x, y) | y ∈ Y∗(x)

}∀x ∈ X ∩ U(x, ε). (12)

By virtue of item (i) of Lemma 3, the point M−1x x is a feasible point in the problem (BP ′).

Further, using the obvious equality F ′(M−1x x,M−1

y y) �

= F (x, y) and Lemma 3, we rewrite (12) inthe following form:

∀x ∈ U(x, ε) : M−1x x ∈ X ′

supy

{F ′(M−1

x x,M−1y y

)| M−1

y y ∈ Y ′∗(M−1

x x)}

≤ supy

{F ′(M−1

x x,M−1y y

)| M−1

y y ∈ Y ′∗(M−1

x x)}

. (13)

Using item (i) of Lemma 4, we obtain the following expression from (13):

∀(M−1

x x)∈ U

(M−1

x x, ε‖Mx‖

): M−1

x x ∈ X ′

supy

{F ′(M−1

x x,M−1y y

)| M−1

y y ∈ Y ′∗(M−1

x x)}

≤ supy

{F ′(M−1

x x,M−1y y

)| M−1

y y ∈ Y ′∗(M−1

x x)}

. (14)

Replacing the variables z = M−1x x and u = M−1

y y in (14), we obtain

∀ z ∈ U

(M−1

x x, ε‖Mx‖

)∩ X ′

supu

{F ′(M−1

x x, u)| u ∈ Y ′

∗(M−1

x x)}

≤ supu

{F ′(z, u) | u ∈ Y ′

∗(z)}.

Therefore, M−1x x ∈ LocSol(BP ′).

Sufficiency. Let us now assume that M−1x x ∈ LocSol(BP ′), then M−1

x x ∈ X ′ and ∃ ε > 0:

∀ z ∈ U(M−1

x x, ε)∩ X ′

supu

{F ′(M−1

x x, u)| u ∈ Y ′

∗(M−1

x x)}

≤ supu

{F ′(z, u) | u ∈ Y ′

∗(z)}. (15)

In accordance with Lemma 3, it follows from the inclusion M−1x x ∈ X ′ that the point x is feasible in

the problem (BP). Further, replacing the variables z = M−1x x and u = M−1

y y and using Lemma 3 anditem (ii) of Lemma 4, as in proving the necessity, we obtain the following expression from (15):

supy

{F (x, y) | y ∈ Y∗(x)

}≤ sup

y

{F (x, y) | y ∈ Y∗(x)

}∀x ∈ U

(x,

ε∥∥M−1

x

∥∥

)∩ X.



Thus, x ∈ LocSol(BP).Finally, it should be noted that it is sufficient to set ε = +∞ and to assume that U(x, ε) = R

m toprove the relationship between the global solutions to the problems (BP) and (BP ′).

Corollary. The number of local (global) solutions of the problem (BP ′) coincides with the number oflocal (global) solutions to the problem (BP).

Let us describe one of the methods of constructing the matrices Mx and My, which was proposed in[13, 14]. Let Mx := HxDxHx and My := HyDyHy, where Dx and Dy are arbitrary positively determined

diagonal matrices, and Hx�= E − 2vxv�x and Hy

�= E − 2vyv

�y are the Householder matrices (reflection

matrices) [25]. Here vx ∈ Rm and vy ∈ R

n are vectors such that ‖vx‖ = ‖vy‖ = 1.

By virtue of the equality v�x vx = ‖vx‖2 = 1, we have the following chain:

HxHx =(E − 2vxv�x

)(E − 2vxv�x

)= E − 4vxv�x + 4vxv�x vxv�x = E.

Similar equalities are also valid for vy. Therefore, inverse matrices with respect to Mx and My can becalculated in this case by the formulas M−1

x = HxD−1x Hx and M−1

y = HyD−1y Hy.

Thus, all necessary elements of the generation method are described. Choosing the values of r1,r2, and r3 to specify the number of kernel problems of each class in the “large” problem and acting inaccordance with the above-described stages, it is possible to generate a comparatively simple problem(e.g., for r1 = r3 = 0, each local solution is simultaneously its global solution by virtue of Proposition 3and Corollary) or an extremely complicated problem (the number of its local solutions that are not itsglobal solutions is 2r − 2r2 in accordance with Proposition 3 and the corollary, i.e., it exponentiallydepends on r). It is possible to construct an arbitrary number of different problems of a desired degree ofcomplexity by choosing the diagonal matrices Dx and Dy and the vectors vx and vy with the help of anygenerator of pseudorandom numbers.

6. CONCLUSIONS

To conclude, it should be noted that the method proposed in this paper can be simply implementedbecause it includes only elementary operations with matrices and vectors. The generated test problemscan be presented in a compact form. At the same time, as the method allows one to construct bilevelproblems of an arbitrary dimension with known guaranteed (global) solutions and with a very largenumber of local solutions that are not global solutions, we can say that typical difficulties of problemsof the examined class are modeled. Therefore, we can conclude that test problems generated by thedeveloped methods will satisfy the basic requirements to test problems [1].

ACKNOWLEDGMENTS

This work was supported by the Russian Foundation for Basic Research (project no. 11-01-00270-a).

REFERENCES1. Polyak, B.T., Vvedenie v optimizatsiyu (Introduction to Optimization), Moscow: Nauka 1983.2. Strekalovskii, A.S. and Orlov, A.V., Bimatrichnye igry i bilineynoe programmirovanie (Bimatrix Games

and Bilinear Programming), Moscow: Fizmatlit, 2007.3. DIMACS Implementation Challenges, URL: http://dimacs.rutgers.edu/Challenges/.4. Hock, W. and Schittkowski, K., Test Examples for Nonlinear Programming Codes, Berlin: Springer-

Verlag, 1981.5. Schittkowski, K., More Test Examples for Nonlinear Programming Codes, Berlin: Springer-Verlag, 1987.6. Floudas, C.A. and Pardalos, P.M., A Collection of Test Problems for Constrained Global Optimization

Algorithms, Berlin: Springer-Verlag, 1990.7. Sven Leyffer’s Test Problems, URL: http://wiki.mcs.anl.gov/leyffer/index.php/Sven_Leyffer’s_Test_

Problems.8. Moshirvaziri, K., Construction of Test Problem for a Class of Reverse Convex Program, J. Optim. Th. Appl.,

1994, vol. 81, no. 2, pp. 343–354.


214 ORLOV, MALYSHEV

9. Moshirvaziri, K., Generalization of the Construction of Test Problems for Nonconvex Optimization, J. Glob.Optim., 1994, vol. 5, no. 1, pp. 21–34.

10. Moshirvaziri, K., Amouzegar, M.A., and Jacobsen, S.E., Test Problem Construction for Linear BilevelProgramming Problem, J. Glob. Optim., 1996, vol. 8, pp. 235–243.

11. Moshirvaziri, K., Construction of Test Problems for Concave Minimization under Linear and NonlinearConstraints, J. Optim. Th. Appl., 1998, vol. 98, no. 1, pp. 83–108.

12. Vicente, L., Calamai, P., and Judice, J., Generation of Disjointly Constrained Bilinear Programming TestProblems, Comput. Optim. Appl., 1992, vol. 1, no. 3, pp. 299–306.

13. Calamai, P. and Vicente, L., Generating Linear and Linear-Quadratic Bilevel Programming Problems, SIAMJ. Sci. Comput., 1993, vol. 14, no. 4, pp. 770–782 (Archive).

14. Calamai, P. and Vicente, L., Generating Quadratic Bilevel Programming Test Problems, ACM Trans. Math.Soft., 1994, vol. 20, pp. 103–119.

15. Vicente, L., Calamai, P., and Judice, J., A New Technique for Generating Quadratic Test Problems, Math.Progr., 1993, vol. 61, pp. 215–231.

16. Gaviano, M., Kvasov, D.E., Lera, D., and Sergeyev, Y.D., Algorithm 829: Software for Generation of Classesof Test Functions with Known Local and Global Minima for Global Optimization, ACM Trans. Math. Soft.,2003, vol. 29, no. 4, pp. 469–480.

17. Dempe, S., Foundations of Bilevel Programming, Dordrecht, The Netherlands: Kluwer, 2002.18. Germeyer, Yu.B., Igry s neprotivopolozhnymi interesami (Games with Non-Opposite Interests) Moscow:

Nauka, 1976.19. Malyshev, A.V. and Strekalovskii, A.S., Relationship between Some Problems of Bilevel and Nonlinear

Optimization, Izv. Vuzov, Mat., 2011, no. 4, pp. 99–103.20. Malyshev, A.V. and Strekalovskii, A.S., Global Search for Pessimistic Solutions in Quadratic-Linear

Problems of Bilevel Optimization, Izv. Irkut. Gos. Univ., 2011, vol. 4, no. 1, pp. 73–82.21. Wiesemann, W., Tsoukalas, A., Kleniati, P.-M., and Rustem, B., Pessimistic Bilevel Optimization, SIAM J.

Optim., 2013, vol. 23, no. 1, pp. 353–380.22. Vasil’ev, F.P. and Ivanitskii, A.Yu., Lineynoe programmirovanie (Linear Programming), Moscow: Faktorial,

1998.23. Bazaraa, M. and Shetty, C., Nonlinear Programming: Theory and Algorithms [Russian translation],

Moscow: Mir, 1982.24. Ivanov, B.N., Diskretnaya matematika. Algoritmy i programmy: Ucheb. posobie (Discrete Mathematics.

Algorithms and Codes: Tutorial), Moscow: Laboratoriya Bazovykh Znanii, 2002.25. Kalitkin, N.N., Chislennye metody (Numerical Methods), Moscow: Nauka, 1978.


test problem generation for quadratic-linear pessimistic bilevel optimization

Documents