algorithms for vector optimization problems · 2015. 6. 25. · algorithms for vector optimization...
TRANSCRIPT
-
Algorithms for Vector Optimization
Problems
Firdevs Ulus
A Dissertation
Presented to the Faculty
of Princeton University
in Candidacy for the Degree
of Doctor of Philosophy
Recommended for Acceptance
by the Department of
Operations Research & Financial Engineering
Adviser: Professor Birgit Rudloff
June 2015
-
c© Copyright by Firdevs Ulus, 2015.
All rights reserved.
-
Abstract
This dissertation studies algorithms to solve linear and convex vector (multi-objective) op-
timization problems. A parametric simplex algorithm for solving linear vector optimization
problems (LVOPs) and two approximation algorithms for solving convex vector optimiza-
tion problems (CVOPs) are provided. The algorithms work for any number of objectives
and for general polyhedral ordering cones.
The parametric simplex algorithm can be seen as a variant of the multi-objective simp-
lex (Evans-Steuer) algorithm [17]. Different from it, this algorithm works in the parameter
space and does not aim to find the set of all efficient solutions. Instead, it finds a subset
of efficient solutions that is enough to generate the whole frontier and corresponds to
a solution concept introduced in [34]. In that sense, this algorithm can also be seen as a
generalization of the parametric self-dual simplex algorithm, which originally is designed for
solving single objective linear optimization problems, and is modified to solve two objective
bounded LVOPs with the positive orthant as the ordering cone in [43] by Ruszczyǹski
and Vanderbei. The algorithm proposed here works for any dimension, any solid pointed
polyhedral ordering cone and for bounded as well as unbounded problems.
Numerical results are provided to compare the proposed algorithm with an objective
space based LVOP algorithm (Benson’s algorithm in [22]), which also provides a solution in
the sense of [34], and with the Evans-Steuer algorithm [17]. The results show that for non-
degenerate problems the proposed algorithm outperforms Benson’s algorithm and is on par
with Evan-Steuer algorithm. For highly degenerate problems Benson’s algorithm [22] excels
the simplex-type algorithms as expected by being an objective space algorithm; however,
the parametric simplex algorithm performs much better than Evans-Steuer algorithm.
For solving convex vector optimization problems (CVOPs), two approximation algo-
rithms are provided. Both algorithms solve the CVOP and its geometric dual problem
simultaneously. The first algorithm is an extension of Benson’s outer approximation algo-
iii
-
rithm, and the second one is a dual variant of it. Both algorithms provide an inner as well
as an outer approximation of the (upper respectively lower) image. Only one scalar convex
program has to be solved in each iteration. We allow objective and constraint functions
that are not necessarily differentiable, allow solid pointed polyhedral ordering cones, and
relate the approximations to an appropriate �-solution concept.
Some illustrative examples and also numerical results are provided for the approxima-
tion algorithms.
iv
-
Acknowledgements
My doctoral studies have come to this point thanks to many people that I have had the
opportunity to interact.
First of all, I am definitely indebted to my advisor, Prof. Birgit Rudloff for her invalu-
able advice, her patience and encouragement throughout my Ph.D. studies at Princeton.
She always had the time for our discussions that made this dissertation possible. The ques-
tions she raised during those discussions not only shaped the research that is presented in
here but also taught me how to think thoroughly and to be precise. Thank you Birgit, also
for your kindness, for being an amazing role model and a friend!
I would like to express my special gratitude to Prof. Erhan Çınlar for many reasons.
Taking the amazing courses that he offered and also being a teaching assistant to his
famous probability class have taught me how to think probabilistically. Besides, I had the
privilege both to rejoice in and to observe his ’art of teaching’. My hope is to digest his
principles while I will be the one who is teaching. Dear Prof. Çınlar, I have always felt
your support and courage for all my endeavors since the very beginning of my doctoral
studies at Princeton, thank you!
I would like to thank Prof. Robert Vanderbei for his collaborations, his ceaseless support
and kindness. I am thankful to Prof. Andreas Löhne for his time and collaborations. His
generous and invaluable reviews were extremely helpful to complete this dissertation. I
would like to thank Prof. Matthias Ehrgott for his support, Prof. Mengdi Wang to accept
reviewing the material presented here and Prof. Amir Ali Ahmadi to be a part of my
dissertation committee.
Many thanks go to my friends Kobby (Nana) Aboagye, Buse Aktaş, Çağın Ararat,
Emre Barut, Mehmet Emin Başbuğ, Sema Berkiten, Enis Dinç, Bianca Dumitrascu, Zach
Feinstein, Georgina Hall, Canan Kaşıkara, Chintan Mehta, Özlem & Onur Özyeşil, Patrick
v
-
Rebeschini, Tardu Sepin, Özlem & Yavuz Yetim and many others for their invaluable
friendship and for the enjoyable memories at Princeton.
’Small World Coffee’ definitely needs to be mentioned here as the provider of the best
coffee in town as well as a great environment to work. Most parts of this thesis have been
written there. Thanks!
Special thanks go to my parents, Ziya Ulus and Vasfiye Ulus, as well as my brothers
Sadık Zeki Ulus and Özgür Ulus for their love and endless support. Teşekkürler sevgili
ailem!
Finally, to the one within: Thanks for reminding me all the time, ”Paradise is
portable!”
vi
-
To Ziya, Vasfiye, Sadık and Özgür.
vii
-
Contents
Abstract . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . iii
Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . v
List of Tables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . x
List of Figures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xi
1 Introduction 1
1.1 Linear Vector Optimization . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.2 Convex Vector Optimization . . . . . . . . . . . . . . . . . . . . . . . . . . 5
1.3 Outline . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
2 Linear Vector Optimization Problems 9
2.1 Preliminaries . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
2.2 Problem Setting and Solution Concepts . . . . . . . . . . . . . . . . . . . . 11
3 Parametric Simplex Algorithm for LVOPs 16
3.1 Parametric Simplex Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . 17
3.1.1 The Parameter set Λ . . . . . . . . . . . . . . . . . . . . . . . . . . 18
3.1.2 Segmentation of Λ: Dictionaries and their optimality region . . . . 19
3.1.3 The set JD of entering variables . . . . . . . . . . . . . . . . . . . . 21
3.1.4 Pivoting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
3.1.5 Detecting unbounded problems and constructing the set X̄ h . . . . 24viii
-
3.1.6 Partition of Λ: putting it all together . . . . . . . . . . . . . . . . . 26
3.1.7 The algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28
3.1.8 Initialization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32
3.2 Illustrative Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37
3.3 Comparison of Different Simplex Algorithms for LVOP . . . . . . . . . . . 48
3.4 Numerical Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53
4 Convex Vector Optimization Problems 58
4.1 Problem Setting and Solution Concepts . . . . . . . . . . . . . . . . . . . . 59
4.2 Geometric Duality . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62
5 Primal and Dual Approximation Algorithms for CVOPs 68
5.1 Assumptions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68
5.2 Primal Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70
5.3 Dual Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79
5.4 Remarks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83
5.5 Examples and numerical results . . . . . . . . . . . . . . . . . . . . . . . . 85
Bibliography 97
ix
-
List of Tables
3.1 Statistics of results for randomly generated problems . . . . . . . . . . . . 54
3.2 Numerical results for randomly generated problems . . . . . . . . . . . . . 55
3.3 Statistics of results for randomly generated degenerate problems where q =
3. For the first row n = 5,m = 15; for the second row n = m = 10; and for
the last row n = 15,m = 5. . . . . . . . . . . . . . . . . . . . . . . . . . . . 56
3.4 Numerical results for degenerate problems where q = 3. For the first set of
problems n = 5,m = 15; for the second set of problems n = m = 10; and
for the last set of problems n = 15,m = 5. . . . . . . . . . . . . . . . . . . 56
3.5 Statistics of results for randomly generated degenerate problems. . . . . . . 57
5.1 Computational results for Example 5.5.1 . . . . . . . . . . . . . . . . . . . 90
5.2 Computational results for Example 5.5.2 . . . . . . . . . . . . . . . . . . . 91
5.3 Computational results for Example 5.5.3 . . . . . . . . . . . . . . . . . . . 94
5.4 Computational results for Example 5.5.4 . . . . . . . . . . . . . . . . . . . 96
x
-
List of Figures
3.1 Optimality regions after the first iteration of Example 3.2.1. Green corre-
sponds to visited and yellow corresponds to boundary dictionaries. . . . . . 40
3.2 Optimality regions after the second iteration of Example 3.2.1. . . . . . . . 41
3.3 Optimality regions after the third and fourth iterations of Example 3.2.1. . 42
3.4 Optimality regions after the fifth, sixth and seventh iterations of Exam-
ple 3.2.1. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43
3.5 Lower image P of Example 3.2.1. . . . . . . . . . . . . . . . . . . . . . . . 44
3.6 Parameter set after the first iteration of Example 3.2.5. Green corresponds
to the optimality region of the visited dictionary D0 and gray corresponds
to the region of unboundedness. . . . . . . . . . . . . . . . . . . . . . . . . 46
3.7 The lower image P of Example 3.2.5. . . . . . . . . . . . . . . . . . . . . . 47
3.8 Lower image P of Example 3.2.6. . . . . . . . . . . . . . . . . . . . . . . . 48
3.9 Lower image P of Example 3.3.2. . . . . . . . . . . . . . . . . . . . . . . . 52
5.1 Upper image (left) of the problem given in Example 5.5.1 and the lower
image (right) of its geometric dual. . . . . . . . . . . . . . . . . . . . . . . 86
5.2 The inner and outer approximations for the upper image after the first three
iterations of Algorithm 2 with ‘break’ in line 18. . . . . . . . . . . . . . . . 87
5.3 The inner and outer approximations of the upper image provided by Algo-
rithm 2 (left) and its alternative (right). . . . . . . . . . . . . . . . . . . . 88
xi
-
5.4 The inner and outer approximations of the lower image of geometric dual
problem after the first four iterations of Algorithm 3 with ‘break’ in line 21. 89
5.5 The inner and outer approximations of the lower image D provided by Al-
gorithm 3 (left) and its alternative (right). . . . . . . . . . . . . . . . . . . 90
5.6 The outer approximation of the upper image (left) and the inner approxima-
tion of the lower image (right) by Algorithm 2 for � = 0.01 for Example 5.5.2. 91
5.7 The outer approximation of the upper image P̃ for Example 5.5.3 by Algo-
rithm 2; displayed for p ∈ P̃ with p1 + p2 + p3 ≤ 36. Top left stands for
the variant with‘break’ and with � = 0.1; top right for the variant with‘no
break’ and with � = 0.1; bottom left for the variant with ‘break’ and with
� = 0.05; bottom right for the variant with ‘no break’ and with � = 0.05. . 93
xii
-
Chapter 1
Introduction
For a vector (multi-objective) optimization problem, several objective functions have to be
optimized over a feasible set determined by constraint functions. In general, the objective
functions are conflicting, hence a unique feasible solution optimizing all the criteria does
not exist. Vector optimization problems (VOPs) have been studied for decades and many
methods have been developed to solve or approximately solve them. As in scalar opti-
mization, only special problem classes are tractable. Linear vector optimization problems
(LVOPs) and convex vector optimization problems (CVOPs) can be listed as two of the
most studied classes of VOPs. In this thesis both type of problems and some solution con-
cepts are discussed. An algorithm to solve LVOPs and two algorithms to approximately
solve CVOPs are presented as proposed and discussed in [42, 37].
1.1 Linear Vector Optimization
There are many methods to solve LVOPs in the literature. First, there are algorithms which
are extensions of the simplex method and working in the variable space. In 1973, Evans
and Steuer [17] developed a simplex algorithm that finds the set of all ’efficient extreme
1
-
solutions’ and the set of all ’unbounded efficient edges’ in the variable space. Later, some
variants of this algorithm have been developed, see for instance [28, 11, 54]. More recently,
Ehrgott, Puerto and Rodriguez-Ch́ıa [13] developed a primal-dual simplex method that
works in the parameter space. This algorithm does not guarantee to find the set of all
efficient solutions, instead it provides a subset of efficient solutions that are enough to
generate the whole efficient frontier in case the problem is ’bounded’. As stated in [13], it
is expected to work better for LVOPs with a network structure. All of these simplex-type
algorithms are designed to solve LVOPs with any number of objective functions where the
ordering is component-wise. Among these, Evans-Steuer algorithm [17] is implemented as a
software called ADBASE [51]. In [43], Ruszczyǹski and Vanderbei developed an algorithm
to solve LVOPs with two objectives and the efficiency of this algorithm is equivalent to
solving a single scalar linear program by the parametric simplex algorithm. Indeed, the
algorithm is a modification of the parametric simplex method and it produces a subset of
efficient solutions that generate the whole efficient frontier in case the problem is bounded.
For more simplex-type methods one can see e.g. the survey paper by Ehrgott and Wiecek
[16] and the references therein.
Apart from the algorithms that work in the variable or parameter space, there are al-
gorithms working in the objective space. In [9], Dauer provides an analysis of the objective
space for LVOPs. Later, Dauer and Liu [10] proposed a procedure to determine the non-
dominated extreme points and edges of the image of the feasible region. In 1998, Benson [3]
proposed an outer approximation algorithm that also works in the objective space. These
methods are motivated by observing that typically the dimension of the objective space
is much smaller than the dimension of the variable space, decision makers tend to choose
a solution based on objective values rather than variable values, and often many efficient
solutions are mapped to a single efficient point in the objective space.
2
-
Löhne [34] introduced a solution concept for LVOPs that takes into account these ideas.
Accordingly, a solution consists of a set of ’point maximizers (efficient solutions)’ and a set
of ’direction maximizers (unbounded efficient edges)’, which altogether generate the whole
efficient frontier. If a problem is ’unbounded’, then a solution needs to have a nonempty
set of direction maximizers. Recently, several variants of Benson’s algorithm, which give a
solution in the sense described in [34] are developed. While some of these algorithms are
designed to solve bounded problems only, some of them can solve unbounded problems as
long as the efficient frontier has at least one vertex. However, this can be done only by
using an additional Phase 1 algorithm, see for instance [34, Section 5.4]. The algorithms
provided in [47, 48, 34, 12] solve in each iteration at least two LPs that are in the same
size as the original problem. An improved variant where only one LP has to be solved in
each iteration have been proposed independently in [7] and [22]. In addition to solving
(at least) one LP, these algorithms solve a vertex enumeration problem in each iteration.
As it can be seen in [14, 7, 38], it is also possible to employ an online vertex enumeration
method. In this case, instead of solving a vertex enumeration problem from the scratch in
each iteration, the vertices are updated after an addition of a new inequality.
In this thesis, a parametric simplex algorithm to solve LVOPs in any size and with any
solid pointed polyhedral ordering cone is presented, as it is proposed in [42]. Although
the structure of the algorithm is similar to Evans-Steuer algorithm, it is different since
the algorithm proposed here works on the parameter space and it finds a solution in the
sense that Löhne proposed in [34]. In other words, instead of generating the set of all
extreme points and direction maximizers, it only finds a subset of them that already allows
to generate the whole efficient frontier. More specifically, the difference can be seen at two
points. First, at each iteration, we employ a different and a more efficient method to find
the ’entering (efficient nonbasic) variables’. Secondly, for each entering variable, instead of
performing all possible pivots for all ’leaving (efficient basic) variables’, we perform a single
3
-
pivot by picking only one of the leaving variables. In this sense, the algorithm provided
here can also be seen as a generalization of the algorithm proposed by Ruszczyǹski and
Vanderbei [43] to unbounded LVOPs with more than two objectives and with more general
ordering cones.
In each iteration the parametric simplex algorithm provides a set of parameters which
make the current vertex optimal. This parameter set is given by a set of inequalities among
which the redundant ones are eliminated. This is an easier procedure than the vertex
enumeration problem, which is required in some objective space algorithms. Different
from the objective space algorithms, the algorithm provided here doesn’t require to solve
an additional LP in each iteration. Moreover, the parametric simplex algorithm works also
for unbounded problems even if the efficient frontier has no vertex and generates direction
maximizers at no additional cost.
As in the scaler case, the efficiency of simplex-type algorithms are expected to be better
whenever the problem is non-degenerate. In vector optimization problems, one may observe
different types of redundancies if the problem is degenerate. The first one corresponds to
the ’primal degeneracy’ concept in scalar problems. In this case, a simplex-type algorithm
may find the same maximizer for many iterations. That is to say, one remains at the
same vertex of the feasible region for multiple iterations. The second type of redundancy
corresponds to the ’dual degeneracy’ concept in scaler problems. Accordingly, the algorithm
may find different maximizers which yield the same objective values. In other words, one
remains at the same vertex of the image of the feasible region. Different from these, a
simplex-type algorithm may find maximizers which yield objective values that are not
vertices of the image of the feasible region. Since these points are on a non-vertex face of
the image set, they are not necessary to generate the whole efficient frontier. Thus, one
can consider these solutions also as redundant.
4
-
The parametric simplex algorithm provided here may also find redundant solutions.
However, it will be shown that the algorithm terminates at a finite time, that is, there is
no risk of cycling. Moreover, compared to the simplex-type algorithms in the literature,
the parametric simplex algorithm finds much less redundant solutions in general.
For the parametric simplex algorithm, we provide different initialization methods. One
of the methods requires to solve two LPs while a second method can be seen as a Phase 1
algorithm. Both of these methods work for any LVOP. Depending on the structure of the
problem, it is also possible to initialize the algorithm without solving an LP or performing
a Phase 1 algorithm.
1.2 Convex Vector Optimization
Convex vector optimization problems are more difficult to solve than LVOPs as expected.
Most of the methods are dealing with the specific subclasses of CVOPs. There are many
algorithms which work for bi-criteria problems. Within this subclass, there are algorithms
which can solve problems where the two objective functions are required to be linear while
the constraints are allowed to be convex, see for instance [6, 19, 53]. In addition, there are
some algorithms that can solve general bi-criteria convex problems, see [24, 33].
Some of the CVOP algorithms that can solve problems with possibly more than two
objectives are designed to solve specific type of problems. For instance, Smirnov [49] pro-
posed an algorithm to solve problems with linear objectives and convex constraints, Fliege
and Heseler [18] proposed an algorithm to solve problems with convex quadratic objec-
tive functions and linear constraints, and Chernyk [5] proposed an inner approximation
algorithm for CVOPs where the image of the feasible set is required to be convex.
A polyhedral inner approximation algorithm to solve more general CVOPs is provided
by Shandl, Klamroth and Wiecek in [45]. Later, this method has been extended in [32] such
5
-
that it provides both inner and outer approximations to the efficient frontier in the objective
space. For solving CVOPs with possibly more than two objective functions, Nefëdov [40]
also proposed an algorithm. Different from the others, this algorithm approximates the
’efficient set’ in the variable space.
In the literature, the accuracy of the approximations of the efficient frontier are mea-
sured in different ways. Many of these accuracy measures depend on some feedback from
the decision maker, see for instance [45, 32]. In some of the bi-objective CVOP algorithms,
as in [19, 53], the Hausdorff distance between the approximation and the efficient frontier
is used as a measure. Among the algorithms for more general CVOPs, Nefëdov [40] used
the Hausdorff distance to measure the accuracy of his approximation algorithm. For more
detailed classification and comparison of approximation methods for solving CVOPs, see
the survey paper by Ruzika and Wiecek [44].
Recently, Ehrgott, Shao, and Schöbel [15] developed a Benson type approximation
algorithm for solving bounded CVOPs. They extended Benson’s algorithm to approximate
the set of all efficient values in the objective space from LVOPs to CVOPs and provide
both an inner and an outer approximation to the efficient frontier. For the accuracy of the
approximation, a measurement which does not depend on any feedback from the decision
maker is employed. As this algorithm works in the objective space, it is motivated by the
same arguments as given in Section 1.1 for objective space algorithms to solve LVOPs.
In this thesis, we present a generalized and simplified version of the algorithm in [15],
as it is proposed and studied in [37]. There are main differences between the algorithm
presented in [15] and the one presented in here. First, at each iteration after solving a
convex problem to find a point on the efficient frontier, we use the dual optimal solution
of this problem to construct the next approximation, instead of solving one more convex
program and computing the gradients of the objective and constraint functions at certain
points depending on the optimal value of this additional convex program. This allows us
6
-
to solve problems where objective and constraint functions are not differentiable, as long
as the scaler problems can be solved. At the same time, we reduce the overall cost of the
algorithm since we solve one convex program instead of two in each iteration.
The second main difference between the two algorithms can be seen in measuring the
approximation error. For the algorithm presented here, we employ the measure as used
in [22]. As the one used in [15], this measure also does not depend on any feedback from
the decision maker. Moreover, using this one allows the algorithms to be applicable to
a larger class of problems. More specifically, we show that it is possible to solve some
examples where the feasible region is not compact.
In addition to these differences, we obtain a finer approximation by including suitable
points in the primal and dual inner approximations throughout the algorithm. We relate
the approximation to an �-solution concept involving infimum attainment. Accordingly, an
�-solution yields an inner and outer approximation to the efficient frontier and the Hausdorff
distance between the inner and the outer approximations is bounded by a constant multiple
of �.
For the algorithms provided here we allow more general ordering cones. Indeed, any
solid pointed polyhedral cone can be used as the ordering cone. We motivate the necessity
of using an ordering cone different from the positive orthant by an example for which a
’set valued risk measure’ is computed.
Finally, in addition to the simplified variant of the algorithm presented in [15], we
present a dual variant of it as proposed in [37]. The dual algorithm provides an alternative
approximation and �-solution.
7
-
1.3 Outline
In Chapter 2, the notations used through this thesis and some well known results from
convex analysis are presented, the linear vector optimization problem and the solution
concepts are discussed. The parametric simplex algorithm for solving linear vector opti-
mization problems is explained in Chapter 3. We also provide some illustrative examples
as well as numerical results regarding the algorithm in this chapter.
The convex vector optimization problems and solution concepts are provided in
Chapter 4. In addition, the geometric duality theory for CVOPs, which is used in the
construction of the algorithms to solve CVOPs, is discussed. In Chapter 5 we explain the
primal and dual approximation algorithms for CVOPs. Some examples to illustrate the
algorithms and some numerical results are provided.
8
-
Chapter 2
Linear Vector Optimization Problems
In this chapter, we review basic definitions and main results about the linear vector opti-
mization problems. Section 2.1 introduces the notations and review some basic concepts
from convex analysis that are used through this thesis. In Section 2.2, the problem setting
and solution concepts of linear vector optimization problems are discussed. Some re-
sults that are necessary to explain the parametric simplex algorithm are also presented in
Section 2.2.
2.1 Preliminaries
The complement, interior, relative interior, closure, boundary, convex hull, and conic hull
of a set A ∈ Rq is denoted by AC , intA, riA, clA, bdA, convA, coneA, respectively. We
define cone ∅ = {0}. If A ⊆ Rq is a polyhedral convex set, it can be represented as the
intersection of finitely many half spaces, that is,
A =r⋂i=1
{y ∈ Rq| (wi)Ty ≥ γi} (2.1.1)
9
-
for some r ∈ N, w1, . . . , wr ∈ Rq \ {0}, and γ1, . . . , γr ∈ R. Every non-empty polyhedral
convex set A can also be written as
A = conv {x1, . . . , xs}+ cone {k1, . . . , kt}, (2.1.2)
where s ∈ N \ {0}, t ∈ N, each xi ∈ Rq is a point, and each kj ∈ Rq \ {0} is a direction of
A. Note that k ∈ Rq \ {0} is called a direction of A if A+ {αk ∈ Rq| α > 0} ⊆ A. The set
A∞ := cone {k1, . . . , kt} is the recession cone of A. The set of points {x1, . . . , xs} together
with the set of directions {k1, . . . , kt} are called the generators of the polyhedral convex set
A. We say ({x1, . . . , xs}, {k1, . . . , kt}) is a V-representation of A whenever (2.1.2) holds,
whereas representation (2.1.1) of A by half-spaces is called H-representation.
A convex cone C is said to be solid, if it has a non-empty interior; pointed if it does
not contain any line; and non-trivial if {0} ( C ( Rq. A non-trivial convex pointed cone
C defines a partial ordering ≤C on Rq: v ≤C w if and only if w − v ∈ C.
Let C ⊆ Rq be a non-trivial convex pointed cone. A point y ∈ A is called C-minimal
element of A if ({y} − C \ {0}) ∩ A = ∅. If the cone C is solid, then a point y ∈ A
is called weakly C-minimal element if ({y} − intC) ∩ A = ∅. The set of all (weakly)
C-minimal elements of A is denoted by (w)MinC (A). Symmetrically, a point y ∈ A is a
called C-maximal element of A if ({y}+ C \ {0})∩A = ∅ and weakly C-maximal element
of A if ({y}+ intC) ∩ A = ∅, whenever C is solid. The set of all (weakly) C-maximal
elements of A is denoted by (w)MaxC (A). Clearly, (w)MaxC (A) = (w)Min−C (A).
The (positive) dual cone of C is the set C+ :={z ∈ Rq| ∀y ∈ C : zTy ≥ 0
}.
A subset F of a convex set A is called an exposed face of A if there exists a supporting
hyperplane H to A, with F = A ∩H.
10
-
2.2 Problem Setting and Solution Concepts
We consider a linear vector optimization problem (LVOP) in the following form
maximize P Tx (with respect to ≤C) (P)
subject to Ax ≤ b,
x ≥ 0,
where P ∈ Rn×q, A ∈ Rm×n, b ∈ Rm, and C ⊆ Rq is a solid pointed polyhedral ordering
cone. We denote the feasible set by X := {x ∈ Rn| Ax ≤ b, x ≥ 0}. Throughout,
we assume that (P) is feasible, i.e., X 6= ∅. The image of the feasible set is defined as
P T [X ] := {P Tx ∈ Rq| x ∈ X}.
We consider the solution concept for LVOPs introduced by Löhne in [34]. Note that this
is an adaptation of a solution concept for complete-lattice-valued optimization problems
which was introduced in [27], see also Remark 4.1.2 in Chapter 4. In [34], it is defined
for minimization problems and here we adapt it to maximization problems. Accordingly,
a point x̄ ∈ X is said to be a (weak) maximizer for (P) if P T x̄ is (weakly) C-maximal in
P [X ]. The set of (weak) maximizers of (P) is denoted by (w)Max(P). The homogeneous
problem of (P) is given by
maximize P Tx (with respect to ≤C) (Ph)
subject to Ax ≤ 0,
x ≥ 0.
The feasible region of (Ph), namely X h := {x ∈ Rn| Ax ≤ 0, x ≥ 0}, satisfies X h = X∞,
that is, the non-zero points in X h are exactly the directions of X . A direction k ∈ Rn \{0}
11
-
of X is called a (weak) maximizer for (P) if the corresponding point k ∈ X h \ {0} is a
(weak) maximizer of the homogeneous problem (Ph).
Definition 2.2.1 ([34, 22]). A set X̄ ⊆ X is called a set of feasible points for (P) and a
set X̄ h ⊆ X h \ {0} is called a set of feasible directions for (P).
A pair of sets(X̄ , X̄ h
)is called a finite supremizer for (P) if X̄ is a non-empty finite
set of feasible points for (P), X̄ h is a (not necessarily non-empty) finite set of feasible
directions for (P), and
convP T [X̄ ] + coneP T [X̄ h]− C = P T [X ]− C. (2.2.3)
A finite supremizer (X̄ , X̄ h) of (P) is called a solution to (P) if it consists of only
maximizers.
The set P := P T [X ] − C is called the lower image of (P). Let {y1, . . . , yI} be the
collection of the generating vectors of the ordering cone C. Then, ({0} , {y1, . . . , yI}) is
a V-representation of the cone C, that is, C = cone {y1, . . . , yI}. Clearly, if (X̄ , X̄ h) is
a finite supremizer, then (P T [X̄ ], P T [X̄ h] ∪ {−y1, . . . ,−yI}) is a V-representation of the
lower image P .
Definition 2.2.2. (P) is said to be bounded if there exists p ∈ Rq such that P ⊆ {p}−C.
Remark 2.2.3. Note that the recession cone of the lower image, P∞, is equal to the lower
image of the homogeneous problem, that is, P∞ = P T [X h] − C, see [34, Lemma 4.61].
Clearly, P∞ ⊇ −C, which also implies P+∞ ⊆ −C+. In particular, if (P) is bounded, then
we have P∞ = −C and X̄ h = ∅.
12
-
The weighted sum scalarized problem for a parameter vector w ∈ C+ is
maximize wTP Tx (P1(w))
subject to Ax ≤ b,
x ≥ 0,
and the following well known proposition holds.
Proposition 2.2.4 ([39, Theorem 2.5]). A point x̄ ∈ X is a maximizer (weak maxi-
mizer) of (P) if and only if it is an optimal solution to (P1(w)) for some w ∈ intC+
(w ∈ C+ \ {0}).
Proposition 2.2.4 suggests that if one could generate optimal solutions, whenever they
exist, to the problems (P1(w)) for w ∈ intC+, then this set of optimal solutions X̄ would
be a set of (point) maximizers of (P). Indeed, it will be enough to solve problem (P1(w))
for w ∈ riW , where
W := {w ∈ C+ : wT c = 1}, (2.2.4)
for some fixed c ∈ intC. Note that (P1(w)) is not necessarily bounded for all w ∈ riW .
Denote the set of all w ∈ riW such that (P1(w)) has an optimal solution by Wb. If one can
find a finite partition (W ib )di=0 of Wb such that for each i ∈ {0, . . . , d} there exists xi ∈ X
which is an optimal solution to (P1(w)) for all w ∈ W ib , then, clearly, X̄ = {x0, . . . , xd} will
satisfy (2.2.3) provided one can also generate a finite set of (direction) maximizers X̄ h.
Trivially, if problem (P) is bounded, then (P1(w)) can be solved optimally for all
w ∈ C+, X̄ h = ∅, and (X̄ , ∅) satisfies (2.2.3). If problem (P) is unbounded, we will construct
in Section 3.1 a set X̄ h by adding certain directions to it whenever one encounters a set of
weight vectors w ∈ C+ for which (P1(w)) cannot be solved optimally.
13
-
The following proposition will be used to prove that this set X̄ h, together with
X̄ = {x0, . . . , xd} will indeed satisfy (2.2.3). It provides a characterization of the recession
cone of the lower image in terms of the weighted sum scalarized problems. More precisely,
the negative of the dual of the recession cone of the lower image can be shown to consist
of those w ∈ C+ for which (P1(w)) can be optimally solved.
Proposition 2.2.5. The recession cone P∞ of the lower image satisfies
−P+∞ = {w ∈ C+| (P1(w)) is bounded}.
Proof. By Remark 2.2.3, we have P∞ = P T [X h] − C. By the definition of the dual cone
and using 0 ∈ X h and 0 ∈ C, we obtain
−P+∞ = {w ∈ Rq| ∀xh ∈ X h, ∀c ∈ C : wT (P Txh − c) ≤ 0}
= {w ∈ C+| ∀xh ∈ X h : wTP Txh ≤ 0}. (2.2.5)
Let w ∈ −P+∞, and consider the weighted sum scalarized problem of (Ph) given by
maximize wTP Tx (Ph1(w))
subject to Ax ≤ 0,
x ≥ 0.
By (2.2.5), xh = 0 is an optimal solution, which implies by strong duality of the linear
program that there exist y∗ ∈ Rm with ATy∗ ≥ Pw and y∗ ≥ 0. Then, y∗ is also dual
feasible for (P1(w)). By the weak duality theorem, (P1(w)) can not be unbounded.
For the reverse inclusion, let w ∈ C+ be such that (P1(w)) is bounded, or equivalently,
an optimal solution exists for (P1(w)) as we assume X 6= ∅. By strong duality, the dual
14
-
problem of (P1(w)) has an optimal solution y∗, which is also dual feasible for (Ph1(w)).
By weak duality, (Ph1(w)) is bounded and has an optimal solution x̃h. Then, w ∈ −P+∞
holds. Indeed, if we assume the contrary, there exists x̄h ∈ X h with wTP T x̄h > 0. This is a
contradiction to the optimality of x̃h since x̃h+x̄h ∈ X h and wTP T (x̃h+x̄h) > wTP T x̃h.
15
-
Chapter 3
Parametric Simplex Algorithm for
LVOPs
As briefly mentioned in Chapter 1, Evans and Steuer [17] proposed a simplex algorithm to
solve linear multi-objective problems. The algorithm moves from one vertex of the feasible
region to another until it finds the set of all (point and direction) maximizers. In this
chapter we provide a parametric simplex algorithm to solve LVOPs where the structure of
the algorithm is similar to the Evans-Steuer algorithm. Different from it, the parametric
simplex algorithm provides a solution in the sense of Definition 2.2.1, that is, it finds a
subset of point maximizers and a subset of direction maximizers that generate the lower
image. This allows the algorithm to deal with the degenerate problems more efficiently
than the Evans-Steuer algorithm. More detailed comparison of the two algorithms can be
seen in Section 3.3.
On the other hand, in [43], Ruszczyǹski and Vanderbei generalize the parametric self
dual method, which originally is designed to solve scalar LPs [52], to solve bi-objective
bounded linear vector optimization problems. This is done by treating the second objective
function as the auxiliary function of the parametric self dual algorithm. In order to solve
16
-
the bi-objective LVOP, that is, to find a solution in the sense of Definition 2.2.1, one needs
to find the optimal basic solutions corresponding to all ’meaningful’ choices of parameters.
In this sense, the algorithm provided here can be seen as a generalization of the parametric
simplex algorithm from bi-objective bounded LVOPs to n-objective LVOPs (n ≥ 2) that
are not necessarily bounded where we also allow for an arbitrary solid pointed polyhedral
ordering cone C.
For the particular examples considered in [43], the initialization is done by considering
the structures of the problems. In [17] however, a general initialization step that would
work for any problem is provided. Here, in addition to the existing initialization methods,
we provide a third method which can be seen as a Phase 1 algorithm.
This chapter is organized as follows. In Section 3.1, we describe the details of the para-
metric simplex algorithm and prove that it finds a solution in the sense of Definition 2.2.1
to any LVOP that has a solution. In Section 3.2, we provide some basic examples to illus-
trate the parametric simplex algorithm. In particular, we show the different initialization
methods by an example, we solve a bounded and an unbounded problem , and we solve a
LVOP for which the lower image has no vertex. In Section 3.3, we compare the parametric
simplex algorithm with the existing simplex type algorithms. We illustrate that the algo-
rithm provided here has the potential to deal with degenerate problems better than the
existing simplex-type algorithms. Finally, we provide numerical results in Section 3.4.
3.1 Parametric Simplex Algorithm
In this section we explain the algorithm steps in detail. We assume that the problems that
we consider are known to have a solution. One can keep in mind that this assumption is
not restrictive as the methods of initialization proposed in Section 3.1.8 will verify if the
problem has a solution or not.
17
-
Assumption 3.1.1. There exists a solution to problem (P).
This assumption is equivalent to having a nontrivial lower image P , that is, ∅ 6= P ( Rq.
Clearly P 6= ∅ implies X 6= ∅, which is equivalent to our standing assumption as given in
Section 2.2. Moreover, by Definition 2.2.1 and Proposition 2.2.4, Assumption 3.1.1 implies
that there exist a maximizer which guarantees that there exists some w0 ∈ intC+ such
that problem (P1(w0)) has an optimal solution. In Section 3.1.8, we will propose methods
to find such a w0. It will be seen that the algorithm provided here finds a solution if there
exists one.
3.1.1 The Parameter set Λ
Throughout the algorithm we consider the scalarized problem (P1(w)) for w ∈ W where W
is given by (2.2.4) for some fixed c ∈ intC. As W is q − 1 dimensional, we will transform
the parameter set W into a set Λ ⊆ Rq−1. Assume without loss of generality that cq = 1.
Indeed, since C was assumed to be a solid cone, there exists some c ∈ intC such that
either cq = 1 or cq = −1. For cq = −1, one can consider problem (P) where C and P are
replaced by −C and −P .
Let c̃ = (c1, . . . , cq−1)T ∈ Rq−1 and define the function w(λ) : Rq−1 → Rq and the set
Λ ⊆ Rq−1 as follows:
w(λ) := (λ1, . . . , λq−1, 1− c̃Tλ)T ,
Λ := {λ ∈ Rq−1 : w(λ) ∈ C+}.
As we assume cq = 1, cTw(λ) = 1 holds for all λ ∈ Λ. Then, w(λ) ∈ W for all λ ∈ Λ
and for any w ∈ W , (w1, . . . , wq−1)T ∈ Λ. Moreover, if λ ∈ int Λ, then w(λ) ∈ riW and
if w ∈ riW , then (w1, . . . , wq−1)T ∈ int Λ. Throughout the algorithm, we consider the
18
-
parametrized problem
(Pλ) := (P1(w(λ)))
for some generic λ ∈ Rq−1.
3.1.2 Segmentation of Λ: Dictionaries and their optimality re-
gion
Throughout this thesis we will use the terminology for the simplex algorithm as it is used in
[52]. First, we introduce slack variables [xn+1, . . . , xn+m]T to obtain x ∈ Rn+m and rewrite
(Pλ) as follows
maximize w(λ)T [P T 0]x (Pλ)
subject to [A In]x = b,
x ≥ 0,
where In is the identity matrix and 0 is the zero matrix in the correct size. We consider
a partition of the variable indices {1, 2, . . . , n + m} into two sets B and N . Variables
xj, j ∈ B, are called basic variables and xj, j ∈ N , are called nonbasic variables. We write
x =[xTB x
TN]T
and permute the columns of [A In] to obtain[B N
]satisfying [A In]x = BxB+NxN , where
B ∈ Rm×m and N ∈ Rm×n. Similarly, we form matrices PB ∈ Rm×q, and PN ∈ Rn×q such
that [P T 0]x = P TB xB + PTNxN . In order to keep the notation simple, instead of writing
[P T 0]x we will occasionally write P Tx, where x stands then for the original decision
variable in Rn without the slack variables.
19
-
Whenever B is nonsingular, xB can be written in terms of the nonbasic variables as
xB = B−1b−B−1NxN .
Then, the objective function of (Pλ) is
w(λ)T [P T 0]x = w(λ)T ξ(λ)− w(λ)TZTNxN ,
where ξ(λ) = P TB B−1b and ZN = (B
−1N)TPB − PN .
We say that each choice of basic and nonbasic variables defines a unique dictionary.
Denote the dictionary defined by B and N by D. The basic solution that corresponds to D
is obtained by setting xN = 0. In this case, the values of the basic variables become B−1b.
Both the dictionary and the basic solution corresponding to this dictionary are said to be
primal feasible if B−1b ≥ 0. Moreover, if w(λ)TZTN ≥ 0, then we say that the dictionary
D and the corresponding basic solution are dual feasible. We call a dictionary and the
corresponding basic solution optimal if they are both primal and dual feasible.
For j ∈ N , introduce the halfspace
IDj := {λ ∈ Rq−1 : w(λ)TZTN ej ≥ 0},
where ej ∈ Rn denotes the unit column vector with the entry corresponding to the variable
xj being 1. Note that if D is known to be primal feasible, then D is optimal for λ ∈ ΛD,
where
ΛD :=⋂j∈N
IDj .
That is, the basic solution corresponding to D is an optimal solution for (Pλ) for all λ ∈ ΛD.
The set ΛD ∩ Λ is said to be the optimality region of dictionary D.
20
-
Proposition 2.2.4 already shows that a basic solution corresponding to a dictionary
yields a (weak) maximizer of (P). Throughout the algorithm we will move from dictionary
to dictionary and collect their basic solutions into a set X̄ . We will later show that
this set will be part of the solution (X̄ , X̄ h) of (P). The algorithm will yield a partition
of the parameter set Λ into optimality regions of dictionaries and regions where (Pλ) is
unbounded. The next subsections explain how to move from one dictionary to another and
how to detect and deal with unbounded problems.
3.1.3 The set JD of entering variables
We call (IDj )j∈JD the defining (non-redundant) collection of half-spaces of the optimality
region ΛD ∩ Λ if JD ⊆ N satisfies
ΛD ∩ Λ =⋂j∈JD
IDj ∩ Λ and
ΛD ∩ Λ (⋂j∈J
IDj ∩ Λ, for any J ( JD.(3.1.1)
For a dictionary D, any nonbasic variable xj, j ∈ JD, is a candidate entering variable. Let
us call the set JD the index set of entering variables for dictionary D.
For each dictionary throughout the algorithm, the index set of entering variables is
found. This can be done by different methods. First, one can employ a modified Fourier-
Motzkin elimination algorithm as described in [2]. Secondly, using the duality of polytopes,
the problem of finding defining inequalities can be transformed to the problem of finding
a convex hull of given points. Then, the algorithms developed for this matter can be
employed. Finally, in order to check if j ∈ N corresponds to a defining or a redundant
21
-
inequality one can consider the following linear program in λ ∈ Rq−1
maximize w(λ)TZTN ej
subject to w(λ)TZTN ej̄ ≥ 0, for all j̄ ∈ N \ ({j} ∪ J redun),
w(λ)TY ≥ 0,
where J redun is the index set of redundant inequalities that have been already found and
Y = [y1, . . . , yI ] is the matrix where y1, . . . , yI are the generating vectors of the ordering
cone C. The inequality corresponding to the nonbasic variable xj is redundant if and only
if the optimal solution to this problem yields w(λ∗)TZTN ej ≤ 0. In this case, we add j to
the set J redun. Otherwise, it is a defining inequality for the region ΛD ∩ Λ and we add j
to the set JD. The set JD is obtained by solving this linear program successively for each
untested inequality against the remaining.
Remark 3.1.2. For the numerical examples provided in Section 3.2, the last method is
employed. Note that the number of variables for each linear program is q − 1, which is
much smaller than the number of variables n of the original problem in general. Therefore,
each linear program can be solved accurately and fast. Thus, this is a reliable and sufficiently
efficient method to find JD.
Note that different methods may yield a different collection of indices as the set JD
might not be uniquely defined. For example, whenever IDj = IDk for j 6= k ∈ N , at most
one of j and k can be included in the set JD. Assuming that the corresponding inequality
is nonredundant, there are at least two different collection of defining inequalities: the one
including j but not k and the one including k but not j. Note that even if JD is not
uniquely defined, the proposed algorithm works with any choice of JD.
22
-
3.1.4 Pivoting
In order to initialize the algorithm one needs to find a dictionary D0 for the parametrized
problem (Pλ) such that the optimality region of D0 satisfies ΛD
0 ∩ int Λ 6= ∅. Note that
the existence of D0 is guaranteed by Assumption 3.1.1 and by Proposition 2.2.4. There are
different methods to find an initial dictionary and these will be discussed in Section 3.1.8.
For now, assume that D0 is given. By Proposition 2.2.4, the basic solution x0 corresponding
to D0 is a maximizer to (P). As a part of the initialization, we find the index set of entering
variables JD0
as defined by (3.1.1).
Throughout the algorithm, for each dictionary D with given basic variables B,
optimality region ΛD ∩ Λ, and index set of entering variables JD, we select an entering
variable xj, j ∈ JD, and pick analog to the standard simplex method a leaving variable xi
satisfying
i ∈ arg mini ∈ B
(B−1N)ij > 0
(B−1b)i(B−1N)ij
, (3.1.2)
whenever there exists some i with (B−1N)ij > 0. Here, indices i, j are written on be-
half of the entries that correspond to the basic variable xi and the nonbasic variable xj,
respectively. Note that this rule of picking leaving variables, together with the initializa-
tion of the algorithm with a primal feasible dictionary D0, guarantees that each dictionary
throughout the algorithm is primal feasible.
If there exists a basic variable xi with (B−1N)ij > 0 satisfying (3.1.2), we perform
the pivot xj ↔ xi to form the dictionary D̄ with basic variables B̄ = (B ∪ {j}) \ {i}
and nonbasic variables N̄ = (N ∪ {i}) \ {j}. For dictionary D̄, the optimality condition
λ ∈ ID̄i with respect to the nonbasic variable xi is equivalent to λ ∈ cl (IDj )C = {λ ∈ Rq−1 :
w(λ)TZTN ej ≤ 0}. In other words, we have ID̄i = cl (IDj )C .
23
-
Assume dictionary D̄ is considered at some point in the algorithm and i ∈ J D̄. If xi
is picked as the entering variable and xj is the corresponding leaving variable, it is known
that the resulting dictionary will be the dictionary D considered above. Thus, we call
(i, j) an explored pivot (or direction) for D̄. We denote the set of all explored pivots of
dictionary D̄ by ED̄.
3.1.5 Detecting unbounded problems and constructing the set
X̄ h
Now, consider the case where there is no candidate leaving variable for an entering variable
xj, j ∈ JD, of dictionary D, that is, (B−1Nej) ≤ 0. As one can not perform a pivot, it
is not possible to go beyond the halfspace IDj . Indeed, the parametrized problem (Pλ) is
unbounded for λ /∈ IDj . The following proposition shows that in that case, a direction
of the recession cone of the lower image can be found from the current dictionary D, see
Remark 2.2.3.
Proposition 3.1.3. Let D be a dictionary with basic and nonbasic variables B and N ,
ΛD ∩ Λ be its optimality region satisfying ΛD ∩ int Λ 6= ∅, and JD be the index set of
entering variables. If for some j ∈ JD, (B−1Nej) ≤ 0, then the direction xh defined by
setting xhB = −B−1Nej and xhN = ej is a maximizer to (P) and P Txh = −ZTN ej.
Proof. Assume (B−1Nej) ≤ 0 for j ∈ JD and define xh by setting xhB = −B−1Nej and
xhN = ej. By definition, the direction xh would be a maximizer for (P) if and only if it is a
(point) maximizer for the homogeneous problem (Ph), see Chapter 2. It holds
[A In]xh = BxhB +Nx
hN
= B(−B−1Nej) +Nej
= 0.
24
-
Moreover, xhN = ej ≥ 0 and xhB = −B−1Nej ≥ 0 by assumption. Thus, xh is primal feasible
for problem (Ph) and also for problem (Ph1(w(λ))) for all λ ∈ Λ, that is, xh ∈ X h \ {0}.
Let λ ∈ ΛD ∩ int Λ, which implies w(λ) ∈ riW ⊆ intC+. Note that by definition of the
optimality region, it is true that w(λ)TZTN ≥ 0. Thus, xh is also dual feasible for (Ph1(w(λ))),
it is an optimal solution for the parametrized homogeneous problem for λ ∈ ΛD ∩ int Λ.
By Proposition 2.2.4 (applied to (Ph) and (Ph1(w(λ)))), xh is a maximizer of (Ph). The
value of the objective function of (Ph) at xh is given by
[P T 0]xh = P TB xhB + P
TNx
hN
= −P TB B−1Nej + P TN ej
= −(P TB B
−1N − P TN)ej
= −ZTN ej.
Remark 3.1.4. If for an entering variable xj, j ∈ JD, of dictionary D, there is no can-
didate leaving variable, we conclude that the problem (P) is unbounded in the sense of
Definition 2.2.2. Then, in addition to the set of point maximizers X̄ one also needs to find
the set of (direction) maximizers X̄ h of (P), which by Proposition 3.1.3 can be obtained
by collecting directions xh defined by xhB = −B−1Nej and xhN = ej for every j ∈ JD with
B−1Nej ≤ 0 for all dictionaries visited throughout the algorithm. For each dictionary D,
we denote the set of indices of entering variables with no candidate leaving variable for
dictionary D by JDb := {j ∈ JD| B−1Nej ≤ 0}. In other words, JDb ⊆ JD is such that for
any j ∈ JDb , (Pλ) is unbounded for λ /∈ IDj .
25
-
3.1.6 Partition of Λ: putting it all together
We have seen in the last subsections that basic solutions of dictionaries yield (weak) point
maximizers of (P) and partitions Λ into optimality regions for bounded problems (Pλ),
while encountering an entering variable with no leaving variable in a dictionary yields
direction maximizers of (P) as well as regions of Λ corresponding to unbounded prob-
lems (Pλ). This will be the basic idea to construct the two sets X̄ and X̄ h and to obtain
a partition of the parameter set Λ. In order to show that (X̄ , X̄ h) produces a solution to
(P), one still needs to ensure finiteness of the procedure, that the whole set Λ is covered,
and that the basic solutions of dictionaries visited yield not only weak point maximizers
of (P), but point maximizers.
Let us denote the set of all primal feasible dictionaries D satisfying ΛD ∩ Λ 6= ∅ by D.
Note that there are at most finitely many dictionaries, so D is a finite collection. Let the
set of parameters λ ∈ Λ yielding bounded scalar problems (Pλ) be Λb. Then it is true that
Λb := {λ ∈ Λ : (Pλ) has an optimal solution} (3.1.3)
=⋃D∈D
(ΛD ∩ Λ).
This can be shown easily. First, let λ ∈ Λ be such that (Pλ) has an optimal solution with
an optimal dictionary Dλ. Clearly, Dλ ∈ D and λ ∈ ΛDλ ∩Λ. For the reverse inclusion, let
λ ∈ ΛD ∩Λ for some D ∈ D. Then the basic solution for D is optimal for (Pλ) and clearly,
λ ∈ Λb.
Observe that whenever xj, j ∈ JD, is the entering variable for dictionary D ∈ D and
there exists a leaving variable xi, the optimality region ΛD̄ for dictionary D̄ after the pivot
is guaranteed to be non-empty. Indeed, it is easy to show that
∅ ( ΛD̄ ∩ ΛD ⊆ {λ ∈ Rq−1 : w(λ)TZN ej = 0},
26
-
where N is the nonbasic variables of dictionary D. Moreover, the basic solutions read
from dictionaries D and D̄ are both optimal solutions to the parametrized problem (Pλ)
for λ ∈ ΛD̄ ∩ ΛD. Note that the common optimality region of the two dictionaries has no
interior. In some cases it is possible that ΛD̄ itself has no interior and it is a subset of the
neighbouring optimality regions corresponding to some other dictionaries. Clearly, these
dictionaries do not provide a new region within Λb.
Moreover, in case there are multiple leaving variables for the same entering variable,
instead of performing all possible pivots, it is enough to pick one of the possible leaving
variables and continue with this choice. Indeed, choosing different leaving variables lead
to different partitions of the same subregion within Λb. In this way, one finds different
optimal solutions to (Pλ) for the same set of λ’s, which is not necessary in order to find a
solution to (P) in the sense of Definition 2.2.1.
By these two observations, it is clear that there is a (finite) collection of dictionaries
D̄ ⊆ D which defines a partition of Λb in the following sense
⋃D∈D̄
(ΛD ∩ Λ) = Λb. (3.1.4)
Assume for now that the set Λb is connected. Indeed, this holds and it will be stated
shortly in Remark 3.1.7. Now, if there is at least one dictionary D ∈ D̄ with JDb 6= ∅, then
by Remark 3.1.4, (P) is unbounded and since Λb is connected, the following holds as well
⋂D∈D̄, j∈JDb
(IDj ∩ Λ) = Λb. (3.1.5)
Remark 3.1.5. Even though it is possible to come across dictionaries with opti-
mality regions having empty interior, for any dictionary D found during the algorithm
ΛD ∩ int Λ 6= ∅ holds. This is guaranteed by starting with a dictionary D0 satisfying
27
-
ΛD0 ∩ int Λ 6= ∅ together with the rule of selecting the entering variables, see (3.1.1). More
specifically, through the algorithm, whenever IDj corresponds to the boundary of Λ it is
guaranteed that j /∈ JD. By this observation and by Proposition 2.2.4, it is clear that the
basic solution corresponding to the dictionary D is not only a weak maximizer but it is a
maximizer.
3.1.7 The algorithm
The aim of the parametrized simplex algorithm is to visit a set of dictionaries D̄
satisfying (3.1.4).
In order to explain the algorithm we introduce the following definition.
Definition 3.1.6. D ∈ D is said to be a boundary dictionary if ΛD and the index set
of entering variables JD is known. A boundary dictionary is said to be visited if the
resulting dictionaries of all possible pivots from D are boundary and the index set JDb (see
Remark 3.1.4) is known.
The motivation behind this definition is to treat the maximizers as nodes of a graph.
Note that more than one dictionary may correspond to the same maximizer and we treat
the set of all such dictionaries as a single node. The possible pivots between dictionaries
corresponding to different maximizers are considered as the edges of this graph.
Remark 3.1.7. The graph described above is connected. The proof for the case C = Rq+
is given in [46], but it can be generalized easily to any polyhedral ordering cone. Note that
this implies that the set Λb is connected.
The idea behind the algorithm is to visit a sufficient subset of ’nodes’ to cover the set
Λb. This can be seen as a special online traveling salesman problem. Indeed, we employ
the terminology used in [31]. The set of all ’currently’ boundary and visited dictionaries
through the algorithm are denoted by BD and V S, respectively.
28
-
The algorithm starts with BD = {D0} and V S = ∅, where D0 is the initial dictionary
with index set of entering variables JD0. We initialize X̄ h as the empty set and X̄ as {x0},
where x0 is the basic solution corresponding to D0. Also, as there is no explored directions
for D0 we set ED0
= ∅.
For a boundary dictionary D, we consider each j ∈ JD and check the leaving variable
corresponding to xj. If there is no leaving variable, we add xh defined by xhB = −B−1Nej,
and xhN = ej to the set X̄ h, see Proposition 3.1.3. Otherwise, a corresponding leaving
variable xi is found. If (j, i) /∈ ED, we perform the pivot xj ↔ xi as it has not been not
explored before. We check if the resulting dictionary D̄ is marked as visited, or boundary.
If D̄ ∈ V S, there is no need to consider D̄ further. If D̄ ∈ BD, we know that (i, j) is
an explored direction for D̄, so we update ED̄ accordingly. In both cases, we continue by
checking some other entering variable of D. If D̄ is neither visited nor boundary but a new
dictionary, then the corresponding basic solution x̄ is added to the set X̄ , the index set
of entering variables J D̄ is computed, (i, j) is added to the set of explored directions ED̄,
and D̄ itself is added to the set of boundary dictionaries. Whenever all j ∈ JD have been
considered, D becomes visited. Thus, D is deleted from the set BD and added to the set
V S. The algorithm stops when there are no more boundary dictionaries.
Theorem 3.1.8. Algorithm 1 returns a solution (X̄ , X̄ h) to (P).
Proof. Algorithm 1 terminates in a finite number of iterations as the overall number of
dictionaries is finite and during the algorithm only a subset of those are visited. Also, by
construction, the algorithm never performs ’already explored pivots’, see line 13. Thus,
there is no risk of cycling and it returns (X̄ , X̄ h) after finitely many iterations.
Note that X̄ , X̄ h are finite sets of feasible points and directions, respectively, for (P), and
they consist of only maximizers by Propositions 2.2.4 and 3.1.3 together with Remark 3.1.5.
Hence, it is enough to show that (X̄ , X̄ h) satisfies (2.2.3).
29
-
Algorithm 1 Parametric Simplex Algorithm for LVOP
1: Find D0 and the index set of entering variables JD0;
2: Initialize
{BD = {D0}, X̄ = {x0};V S, X̄ h, ED0 , R = ∅;
3: while BD 6= ∅ do4: Let D ∈ BD with nonbasic variables N and index set of entering variables JD;5: for j ∈ JD do6: Let xj be the entering variable;7: if B−1Nej ≤ 0 then8: Let xh be such that xhB = −B−1Nej and xhN = ej;9: X̄ h ← X̄ h ∪ {xh};10: P T [X̄ h]← P T [X̄ h] ∪ {−ZTN ej}11: else12: Pick i ∈ arg mini∈B, (B−1N)ij>0
(B−1b)i(B−1N)ij
;
13: if (j, i) /∈ ED then14: Perform the pivot with entering variable xj and leaving variable xi;15: Call the new dictionary D̄ with nonbasic variables N̄ = N ∪ {i} \ {j};16: if D̄ /∈ V S then17: if D̄ ∈ BD then18: ED̄ ← ED̄ ∪ {(i, j)};19: else20: Let x̄ be the basic solution for D̄;21: X̄ ← X̄ ∪ {x̄};22: P T [X̄ ]← P T [X̄ ] ∪ {P T x̄};23: Compute the index set of entering variables J D̄ of D̄;24: Let ED̄ = {(i, j)};25: BD ← BD ∪ {D̄};26: end if27: end if28: end if29: end if30: end for31: V S ← V S ∪ {D}, BD ← BD \ {D};32: end while
33: return
{(X̄ , X̄ h) : A finite solution of (P);(P T [X̄ ], P T [X̄ h] ∪ {y1, . . . , yI}) : V representation of P .
30
-
Observe that by construction, the set of all visited dictionaries D̄ := V S at termination
needs to satisfy (3.1.4). Indeed, there are finitely many dictionaries and Λb is a connected
set, see Remark 3.1.7.
It is guaranteed by (3.1.4) that for any w ∈ C+, for which (P1(w)) is bounded, there
exists an optimal solution x̄ ∈ X̄ of (P1(w)). Then, it is clear that (X̄ , X̄ h) satisfies (2.2.3)
as long as R := coneP T [X̄ h] − C is the recession cone P∞ of the lower image. If for all
D ∈ D̄ the set JDb = ∅, then (P) is bounded, X̄ h = ∅, and trivially R = −C = P∞. If there
is at least one dictionary D ∈ D̄ with JDb 6= ∅, then by Remarks 3.1.4 and 3.1.7, (P) is
unbounded, X̄ h 6= ∅ and D̄ also satisfies (3.1.5). By definition of IDj , we can write (3.1.5)
as
Λb =⋂
D∈V S, j∈JDb
{λ ∈ Λ : w(λ)TZTNDej ≥ 0}, (3.1.6)
where ND is the set of nonbasic variables corresponding to dictionary D.
By construction and by Proposition 3.1.3, we have
R = cone {−ZTNDej : j ∈
⋃D∈V S
JDb } − C
= cone ({−ZTNDej : j ∈
⋃D∈V S
JDb } ∪ {−y1, . . . ,−yI}),
where {y1, . . . , yI} is the set of generating vectors for the ordering cone C. The dual cone
can be written as
R+ =⋂
D∈V S, j∈JDb
{w ∈ Rq : wTZTNDej ≤ 0} ∩
I⋂i=1
{w ∈ Rq : wTyi ≤ 0}. (3.1.7)
The proof will be completed by showing that −P+∞ = −R+ as it implies P∞ =
coneP T [X̄ h] − C. Let w ∈ −P+∞. Then, by proposition 2.2.5, (P1(w)) has an op-
timal solution. As cTw > 0, also(P1(
wcTw
))
= (Pλ̄) has an optimal solution, where
31
-
λ̄ := 1cTw
(w1, . . . , wq−1)T and thus w(λ̄) = w
cTw. By the definition of Λb given by (3.1.3),
λ̄ ∈ Λb. Then, by (3.1.6), λ̄ ∈ Λ and w(λ)TZTNDej ≥ 0 for all j ∈ JDb , D ∈ V S. This holds
if and only if w ∈ −R+ by definition of Λ and by (3.1.7). The other inclusion can be shown
symmetrically.
Remark 3.1.9. In general, the simplex-type algorithms are known to work better if the
problem is not degenerate. If the problem is degenerate, Algorithm 1 may find redundant
maximizers. The effects of degeneracy will be provided in more detail in Section 3.4. For
now, let us mention that it is possible to eliminate the redundancies by additional steps in
Algorithm 1. There are two type of redundant maximizers.
a. Algorithm 1 may find multiple point (direction) maximizers that are mapped to the same
point (direction) in the image space. In order to find a solution that is free of these type
of redundant maximizers, one may change line 21 (9) of the algorithm such that the
current maximizer x (xh) is added to the set X̄ (X̄ h) only if its image is not in the
current set P T [X̄ ] (P T [X̄ h]).
b. Algorithm 1 may also find maximizers whose image is not a vertex on the lower image.
One can eliminate these maximizers from the set X̄ by performing a vertex elimination
at the end.
3.1.8 Initialization
There are different ways to initialize Algorithm 1. We provide two methods, both of which
also determine if the problem has no solution. Note that (P) has no solution if X = ∅ or
if the lower image is equal to Rq, that is, if (P1(w)) is unbounded for all w ∈ intC+. We
assumed X is nonempty. Moreover, for the purpose of this section, we assume without
loss of generality that b ≥ 0. Indeed, if b � 0, one can find a primal feasible dictionary by
applying any ’Phase 1’ algorithm that are available for the usual simplex method, see [52].
32
-
The first initialization method finds a weight vector w0 ∈ intC+ such that (P1(w0))
has an optimal solution. Then the optimal dictionary for (P1(w0)) is used to construct
the initial dictionary D0. There are different ways to choose the weight vector w0. The
second method of initialization can be thought of as a Phase 1 algorithm. It finds an initial
dictionary as long as there exists one.
Finding w0 and Constructing D0
The first way to initialize the algorithm requires finding some w0 ∈ intC+ such
that (P1(w0)) has an optimal solution. It is clear that if the problem is known to
be bounded, then any w0 ∈ intC+ works. However, it is a nontrivial procedure in general.
In the following we give two different methods to find such w0. The first method can also
determine if the problem has no solution.
a. The first approach is to extend the idea presented in [17] to any solid pointed polyhedral
ordering cone C. Accordingly, finding w0 involves solving the following linear program:
minimize bTu (P0)
subject to ATu− Pw ≥ 0,
Y T (w − c) ≥ 0,
u ≥ 0,
where c ∈ intC+, and the columns of Y are the generating vectors of C.
Proposition 3.1.10. Under the assumption b ≥ 0, (P) has a maximizer if and only
if (P0) has an optimal solution.
Proof. (P) has a maximizer if and only if there exists w ∈ intC+ such that (P1(w))
has an optimal solution by Proposition 2.2.4. Using the strong duality results for linear
33
-
programs this holds if and only if
minimize bTu (D1(w))
subject to ATu− Pw ≥ 0,
u ≥ 0,
has optimal solution. We will show that (P0) has optimal solution if and only if there
exists w ∈ intC+ such that (D1(w)) has an optimal solution. First, let (u∗, w∗) be an
optimal solution for (P0). Note that w∗ ∈ intC+ and u∗ is feasible for (D1(w∗)). Since
(D1(w∗)) is also bounded by weak duality and the assumption b ≥ 0, there exists an
optimal solution to (D1(w∗)).
On the other hand, let w∗ ∈ intC+ and u∗ be optimal for (D1(w∗)). If w∗ satisfies
Y T (w∗ − c) ≥ 0, then (u∗, w∗) is feasible for (P0). If Y T (w∗ − c) � 0, then there exist
γ > 0 such that Y T (γw∗ − c) ≥ 0 and γu∗ is feasible for D1(γw∗). Note that (P0) is
bounded which can easily be seen by observing that the feasibility of the dual program.
Then, there exists an optimal solution to (P0).
It is clear by the proof of Proposition 3.1.10 that (P0) is bounded. Moreover, if (P0) is
infeasible, we can conclude that the lower image has no vertex and (P) has no solution.
In case it has an optimal solution (u∗, w∗), then one can take w0 = w∗
cTw∗∈ intC+, and
solve (P1(w0)) optimally. For the randomly generated examples of Section 3.2 we have
used this method.
b. Using the idea provided in [43], it might be possible to initialize the algorithm without
even solving a linear program. By the structure of a particular problem, one may start
with a dictionary which is trivially optimal for some weight w0. In this case, one can
34
-
start with this choice of w0, and get the initial dictionary D0 even without solving an
LP. An example is provided in Section 3.2, see Remark 3.2.3.
In order to initialize the algorithm, w0 can be used to construct the initial dictionary
D0. Without loss of generality assume that cTw0 = 1, indeed one can always normalize
since c ∈ intC implies cTw0 > 0. Then, clearly w0 ∈ riW . Let B0 and N 0 be the set of
basic and nonbasic variables corresponding to the optimal dictionary D∗ of (P1(w0)). If
one considers the dictionary D0 for (Pλ) with the basic variables B0 and nonbasic variables
N 0, the objective function of D0 will be different from D∗ as it depends on the parameter
λ. However, the matrices B0, N0, and hence the corresponding basic solution x0, are the
same in both dictionaries. We consider D0 as the initial dictionary for the parametrized
problem (Pλ). Note that B0 is a nonsingular matrix as it corresponds to dictionary D∗.
Moreover, since D∗ is an optimal dictionary for (P1(w0)), x0 is clearly primal feasible
for (Pλ) for any λ ∈ Rq−1. Furthermore, the optimality region of D0 satisfies ΛD0∩int Λ 6= ∅
as λ0 := [w01, . . . , w0q−1] ∈ ΛD
0 ∩ int Λ. Thus, x0 is also dual feasible for (Pλ) for λ ∈ ΛD0,
and x0 is a maximizer to (P).
Perturbation Method
The second method of initialization works similar to the idea presented for Algorithm 1
itself. Assuming that b ≥ 0, problem (Pλ) is perturbed by an additional parameter µ ∈ R
as follows:
maximize (w(λ)TP T − µ1T )x (Pλ,µ)
subject to Ax ≤ b,
x ≥ 0,
35
-
where 1 is the vector of ones. After introducing the slack variables, consider the dictionary
with basic variables xn+1, . . . , xm+n and with nonbasic variables x1, . . . , xn. This dictionary
is primal feasible as b ≥ 0. Moreover, it is dual feasible if Pw(λ)− µ1 ≤ 0. We introduce
the optimality region of this dictionary as
M0 := {(λ, µ) ∈ Λ× R+ | Pw(λ)− µ1 ≤ 0}.
Note that M0 is not empty as µ can take sufficiently large values.
The aim of the perturbation method is to find an optimality region M such that
M ∩ ri (Λ× {0}) 6= ∅, (3.1.8)
where Λ×{0} := {(λ, 0) | λ ∈ Λ}. If the current dictionary satisfies (3.1.8), then it can be
taken as an initial dictionary D0 for Algorithm 1 after deleting the parameter µ. Otherwise,
the defining inequalities of the optimality region are found. Clearly, they correspond to the
entering variables of the current dictionary. The search for an initial dictionary continues
similar to the original algorithm. Note that if there does not exist a leaving variable for an
entering variable, (Pλ,µ) is found to be unbounded for some set of parameters. The algo-
rithm continues until we obtain a dictionary for which the optimality region satisfies (3.1.8)
or until we cover the the parameter set Λ×R+ by the optimality regions and by the regions
that are known to yield unbounded problems. At termination, if there exist no dictionary
that satisfies (3.1.8), then we conclude that there is no solution to problem (P). Otherwise,
we initialize the algorithm with D0. See Exercise 3.2.1, Remark 3.2.4.
36
-
3.2 Illustrative Examples
We provide some examples and numerical results in this section. The first example illus-
trates how the different methods of initialization and the algorithm work. It is taken
from [50] and was used in [13] to illustrate the algorithm provided there. The next example,
taken from [34], is also illustrative and is known to be unbounded. The third example shows
that the Algorithm 1 can find a solution even though the lower image does not have any
vertices.
Example 3.2.1. Consider the following problem
maximize (x1, x2, x3)T with respect to ≤R3+
subject to x1 + x2 + x3 ≤ 5
x1 + 3x2 + x3 ≤ 9
3x1 + 4x2 ≤ 16
x1, x2, x3 ≥ 0.
Let c = (1, 1, 1)T ∈ intR3+. Clearly, we have Λ = {λ ∈ R2 : λ1 + λ2 ≤ 1, λi ≥ 0, i = 1, 2}.
Let us illustrate the different initialization methods.
Remark 3.2.2. (Initializing by solving (P0), see Section 3.1.8 a.) A solution of (P0)
is found as w∗ = (1, 1, 1)T . Then, we take w0 = (13, 1
3, 1
3)T as the initial weight vector.
x0 = (5, 0, 0)T is an optimal solution found for P1(w0). The indices of the basic variables
of the corresponding optimal dictionary are B0 = {1, 5, 6}. We form dictionary D0 of
37
-
problem (Pλ) with basic variables B0:
ξ = 5λ1 −λ1x4 −(λ1 − λ2)x2 −(2λ1 + λ2 − 1)x3
x1 = 5 −x4 −x2 −x3
x5 = 4 +x4 −2x2
x6 = 1 +3x4 −x2 +3x3.
Remark 3.2.3. (Initializing using the structure of the problem, see Section 3.1.8
b.) The structure of Example 3.2.1 allows us to initialize without solving a linear program.
Consider w0 = (1, 0, 0)T . As the objective of P1(w0) is to maximize x1 and all variables need
to be nonnegative, x1 = min{ b1a11 ,b2a21, b3a31} = min{5, 9, 16
3} = 5, x2 = x3 = 0 is an optimal
solution of P1(w0). The corresponding slack variables are x4 = 0, x5 = 4, and x6 = 1.
Note that this corresponds to the dictionary with basic variables B = {1, 5, 6} and nonbasic
variables N = {2, 3, 4}, which yields the same initial dictionary D0 as above. Similarly,
one could have chosen the initial weight vector w0 to be (0, 1, 0)T or (0, 0, 1)T . Different
choices might yield different initial dictionaries. At this point, one needs to be careful as
w0 /∈ intC+ but w0 ∈ bdC+. In order to ensure that the corresponding initial solutions
are maximizers and not only weak maximizers, one needs to check the optimality regions
of these possible initial dictionaries. If the optimality region has a nonempty intersection
with int Λ, which is the case for D0 found via w0 = (1, 0, 0)T , then the corresponding basic
solution is a maximizer. In general, if one can find w ∈ intC+ such that (P1(w)) has a
trivial optimal solution, then the last step is clearly unnecessary.
38
-
Remark 3.2.4. (Initializing using perturbation method, see Section 3.1.8.) The
starting dictionary for (Pλ,µ) of the perturbation method is given as
ξ = (λ1 − µ)x1 +(λ2 − µ)x2 +(1− λ1 − λ2 − µ)x3
x4 = 5 −x1 −x2 −x3
x5 = 9 −x1 −3x2 −x3
x6 = 16 −3x1 −4x2.
This dictionary is optimal for M0 = {(λ, µ) ∈ Λ × R : µ ≥ λ1, µ ≥ λ2, µ ≥ 1 − λ1 − λ2}.
Clearly, M0 does not satisfy (3.1.8). The defining inequalities of this region correspond
to x1, x2 and x3. If x1 enters, then the leaving variable is x4 and the next dictionary has
the optimality region M1 = {(λ, µ) ∈ Λ × R : λ1 ≥ µ, λ1 ≥ λ2, λ1 + 2λ2 ≥ 1} which
satisfies (3.1.8). Then, by deleting µ the initial dictionary is found to be D0 as above.
Different choices of entering variables in the first iteration might yield different initial
dictionaries.
Consider the initial dictionary D0. Clearly, ID0
4 = {λ ∈ R2 : λ1 ≥ 0}, ID0
2 = {λ ∈ R2 :
λ1 − λ2 ≥ 0}, and ID0
3 = {λ ∈ R2 : 2λ1 + λ2 ≥ 1}. The defining halfspaces for ΛD0 ∩ Λ
correspond to the nonbasic variables x2, x3, thus we have JD0 = {2, 3}.
The iteration starts with the only boundary dictionary D0. If x2 is the entering vari-
able, x6 is picked as the leaving variable. The next dictionary, D1, has basic variables
B1 = {1, 2, 5}, the basic solution x1 = (4, 1, 0)T , and a parameter region ΛD1 = {λ ∈ R2 :
4λ1 − 3λ2 ≥ 0,−λ1 + λ2 ≥ 0, 5λ1 − 2λ2 − 1 ≥ 0}. The halfspaces corresponding to the
nonbasic variables xj, j ∈ JD1
= {6, 3} are defining for the optimality region ΛD1 ∩ Λ.
Moreover, ED1
= {(6, 2)} is an explored pivot for D1.
From dictionary D0, if x3 enters, then x1 is the leaving variable. Dictionary D2 af-
ter this pivot has basic variables B2 = {3, 5, 6}, the basic solution x2 = (0, 0, 5)T , and
39
-
ΛD2
= {λ ∈ R2 : λ1 + λ2 ≤ 1, 2λ1 + λ2 ≤ 1, λ1 + 2λ2 ≤ 1}. The halfspaces corresponding to
the nonbasic variables xj, j ∈ JD2
= {2, 1} are defining for the optimality region ΛD2 ∩ Λ.
Moreover, ED2
= {(1, 3)} is an explored direction for D2.
After the first iteration, we have V S = {D0}, and BD = {D1, D2}. Figure 3.1 shows
the optimality regions of the dictionaries D0, D1, D2 after the first iteration. The color
green indicates that the corresponding dictionary is visited, yellow stands for boundary
dictionaries.
Figure 3.1: Optimality regions after the first iteration of Example 3.2.1. Green correspondsto visited and yellow corresponds to boundary dictionaries.
For the second iteration, we consider D1 ∈ BD. There are two possible pivots with
entering variables xj, j ∈ JD1
= {6, 3}. For x6, x2 is found as the leaving variable. As
(6, 2) ∈ ED1, the pivot is already explored and not necessary. For x3, the leaving variable is
found as x5. The resulting dictionary D3 has basic variables B3 = {1, 3, 2}, basic solution
x3 = (83, 2, 1
3)T , the optimality region ΛD
3 ∩ Λ = {λ ∈ R2+ : λ1 + 8λ2 ≤ 5,−2λ1 − λ2 ≤
−1, 5λ1 − 2λ2 ≤ 1}, and the indices of the entering variables JD3
= {4, 6, 5}. Moreover,
40
-
we write ED3
= {(5, 3)}. At the end of the second iteration we have V S = {D0, D1}, and
BD = {D2, D3}. Figure 3.2 shows the optimality regions after the second iteration.
Figure 3.2: Optimality regions after the second iteration of Example 3.2.1.
Consider D2 ∈ BD with JD2 = {2, 1} for the third iteration. For x2, the leaving variable
is x5. This pivot yields a new dictionary D4 with basic variables B4 = {3, 2, 6} and basic
solution x4 = (0, 2, 3)T . The indices of the entering variables are found as JD4
= {1, 4, 5}.
Also, we have ED4
= {(5, 2)}. For x1, the leaving variable is x3. The corresponding pivot
is already explored for D2. At the end of iteration 3, we thus have V S = {D0, D1, D2} and
BD = {D3, D4}.
For the next iteration, D3 with JD3
= {4, 5, 6} is considered. The pivot for x4 yields
dictionary D5 with basic variables B5 = {1, 4, 2}, basic solution x5 = (2.4, 2.2, 0)T , and
JD5
= {3, 6}. Note that (3, 4) is now an already explored pivot for D5, ED5 = {(3, 4)}.
The leaving variable for x5 is x3, which is an already explored direction for D3. The pivot
for j = 6 yields one of the currently boundary dictionaries, D4. Thus, we add (1, 6) to the
set of already explored directions for D4, ED4
= {(5, 2), (1, 6)}.
41
-
The optimality regions after the third and the fourth iterations can be seen from
Figure 3.3.
Figure 3.3: Optimality regions after the third and fourth iterations of Example 3.2.1.
For the fifth iteration we consider D4 with JD4
= {1, 4, 5}. If x1 enters x6 is the
leaving variable. Note that (1, 6) ∈ ED4 and the pivot is not necessary. Similarly, for
x5 the leaving variable is x2 and (5, 2) ∈ ED4. If x4 enters, the leaving variable is found
as x3. This pivot yields a new dictionary D6 with basic variables B6 = {4, 2, 6}, basic
solution x6 = (0, 3, 0)T , and entering variables xj, j ∈ JD6
= {3, 1}. Moreover, we have
ED6
= {(3, 4)}. BD = {D5, D6}, and V S = {D0, D1, D2, D3, D4} at the end of iteration
5.
Next, we consider D5 ∈ BD with JD5 = {3, 6}. If x3 is the entering variable, x4 is
found as the leaving variable. This is an already explored pivot for D5. If x6 enters, then
x1 is the leaving variable. This pivot yields dictionary D6 ∈ BD. We update the explored
pivots for D6 as ED6
= {(3, 4), (1, 6)}. Clearly BD = {D6}, and we add D5 to the set V S.
In the last iteration, we consider D6 with JD6
= {3, 1}. The pivots corresponds to this
dictionary is x4 ↔ x3 and x1 ↔ x6, which are already explored for D6. Thus, after seventh
42
-
iteration we have BD = ∅. Figure 3.4 shows the corresponding optimality regions after
each iteration.
Figure 3.4: Optimality regions after the fifth, sixth and seventh iterations of Example 3.2.1.
Note that the problem is bounded, as for each entering variable there exist a leaving
variable. The solution to the problem is then (X̄ , ∅) with
X̄ ={
5
0
0
,
4
1
0
,
0
0
5
,
8/3
2
1/3
,
0
2
3
,
12/5
11/5
0
,
0
3
0
}.
The lower image can be seen in Figure 3.5.
Example 3.2.5. Consider the following problem
maximize P Tx with respect to ≤R3+
subject to x1 − x2 ≤ −1
x2 − x3 ≤ −2
x3 − x4 ≤ −3
x4 − x5 ≤ −4
x1, x2, x3, x4, x5 ≥ 0,
43
-
Figure 3.5: Lower image P of Example 3.2.1.
where
P T =
4 0 0 4 −4
−4 4 4 −4 0
0 0 −4 0 0
.
Let c = (1, 1, 1)T ∈ intR3+. Clearly, we have Λ = {λ ∈ R2 : λ1 + λ2 ≤ 1, λi ≥ 0, i = 1, 2}.
As b � 0, we first perform a Phase 1 simplex algorithm to get a primal feasible initial
dictionary. Then, we follow the initialization method explained in Section 3.1.8 a. to find
w0 = (13, 1
3, 1
3) as the initial scalarization parameter. Then, x0 = (0, 1, 3, 6, 10)T is an
optimal solution to P1(w0). The basic variables of the corresponding optimal dictionary is
44
-
B0 = {2, 3, 4, 5}. We form dictionary D0 to problem (Pλ) with basic variables B0:
ξ = ξ(λ) +4(2λ1 + λ2 − 1)x1 +4(λ1 + 2λ2 − 1)x6 +4(λ1 + λ2 − 1)x7 −4λ2x8 −4λ1x9
x2 = 1 +x1 +x6
x3 = 3 +x1 +x6 +x7
x4 = 6 +x1 +x6 +x7 +x8
x5 = 10 +x1 +x6 +x7 +x8 +x9.
The defining inequalities of the optimality region ΛD0 ∩ Λ correspond to the nonbasic
variables x1, x6, thus we have JD0 = {1, 6}.
For the first iteration, we consider the only boundary dictionary D0. If x1 is the entering
variable, there is no leaving variable according to the minimum ratio rule (3.1.2). We
conclude that problem (Pλ) is unbounded for λ ∈ {R2 : 2λ1 + λ2 > 1}. Note that
B−1N =
−1 −1 0 0 0
−1 −1 −1 0 0
−1 −1 −1 −1 0
−1 −1 −1 −1 −1
,
and the first column corresponds to the entering variable x1. Thus, xhB0 = (x
h2 , x
h3 , x
h4 , x
h5)T =
(1, 1, 1, 1)T and xhN0 = (xh1 , x
h6 , x
h7 , x
h8 , x
h9)T = e1 = (1, 0, 0, 0, 0)T . Thus, we add
xh = (xh1 , . . . , xh5) = (1, 1, 1, 1, 1)
T to the set X̄ h, see Algorithm 1, line 12. Also, by
Proposition 3.1.3, P Txh = (4, 0,−4)T is an extreme direction of the lower image P.
Whenever x6 enters, there is no leaving variable either. Thus, (Pλ) is also unbounded for
λ ∈ {R2 : λ1 + 2λ2 > 1}, (0, 1, 1, 1, 1)T is a maximizer to homogeneous problem (Ph) and
45
-
(0, 4,−4)T is an extreme direction for P. See the optimality region after the first iteration
in Figure 3.6.
Figure 3.6: Parameter set after the first iteration of Example 3.2.5. Green corresponds tothe optimality region of the visited dictionary D0 and gray corresponds to the region ofunboundedness.
At the end of first iteration we have BD = ∅ and V S = {D0}. The algo-
rithm terminates with the solution (X̄ , X̄ h), with X̄ = {(0, 1, 3, 6, 10)T} and X̄ h =
{(1, 1, 1, 1, 1)T , (0, 1, 1, 1, 1)T}. The lower image can be seen in Figure 3.7.
Example 3.2.6. Consider the following example.
maximize (x1 − x2, x3 − x4)T with respect to ≤R2+
subject to x1 − x2 + x3 − x4 ≤ 1
x1, x2, x3, x4 ≥ 0.
Let c = (1, 1)T ∈ intR2+. Clearly, we have Λ = [0, 1] ⊆ R. Using the method described
in Section 3.1.8 a., we find w0 = (12, 1
2) as the initial scalarization parameter. Then,
46
-
Figure 3.7: The lower image P of Example 3.2.5.
x0 = (1, 0, 0, 0)T is an optimal solution to P1(w0). The basic variable of the corresponding
optimal dictionary is B0 = {1}. We form dictionary D0 to problem (Pλ) :
ξ = λ −λx5 +(1− 2λ)x3 +(2λ− 1)x4
x1 = 1 −x5 +x2 −x3 +x4.
The defining inequalities of the optimality region ΛD0 ∩Λ correspond to the nonbasic vari-
ables x3, x4, thus we have JD0 = {3, 4}.
For the first iteration, we consider D