A Long Step Cutting Plane Algorithm That Uses the Volumetric Barrier 1

Srinivasan Ramaswamyramass@rpi.edu

Dept. of Decision Sciences & Engg. SystemsRensselaer Polytechnic Institute

Troy, NY 12180

John E. Mitchell 2

mitchj@rpi.eduDepartment of Mathematical Sciences

Rensselaer Polytechnic InstituteTroy, NY 12180

DSES Technical Report

June 20, 1995

Abstract

A cutting plane method for linear/convex programming is described. It is based on the vol-umetric barrier, introduced by Vaidya. The algorithm is a long step one, and has a complexityof O(n1.5L) Newton steps. This is better than the O(n

√mL) complexity of non-cutting plane

long step methods based on the volumetric barrier, but it is however worse than Vaidya’s originalO(nL) result (which is not a long step algorithm). Major features of our algorithm are that whenadding cuts we add them right through the current point, and when seeking progress in the objec-tive, the duality gap is reduced by half (not provably true for Vaidya’s original algorithm). Fur-ther, we generate primal as well as dual iterates, making this applicable in the column generationcontext as well. Vaidya’s algorithm has been used as a subroutine to obtain the best complexityfor several combinatorial optimization problems – e.g, the Held-Karp lower bound for the Trav-eling Salesperson Problem. While our complexity result is weaker, this long step cutting planealgorithm is likely to be computationally more promising on such combinatorial optimizationproblems with an exponential number of constraints. We also discuss a multiple cuts version —where upto p ≤ n ‘selectively orthonormalized’ cuts are added through the current point. Thishas a complexity of O(n1.5Lp log p) quasi Newton steps.

Key words: Cutting plane, volumetric center, long step, Selective Orthonormalization, linearprogramming, convex programming.

1AMS 1991 subject classification. Primary 90C05. Secondary 90C25, 65K05.2Research partially supported by ONR Grant number N00014–94–1–0391.

1 Introduction

The problem of interest is to solve(P ) min cTx

S.T Ax ≥ b

where x ∈ <n, A ∈ <m×n, and m n.Its dual is :

(D) max bTy

S.T AT y = c

y ≥ 0

where AT ∈ <n×m , b, y ∈ <m, c ∈ <n and m n.Recently, there has been some work on cutting plane methods, especially in the context of the

convex feasibility problem. Most of these recent methods are “central cutting” plane methods, wherethe test point is in the center of the polytope that approximates the convex set at the current iteration.The most commonly used test point is the analytic center of the polytope (see eg., [5, 23]), sincethis is fairly easy to compute. Other test points that have been used include the center of the largestinscribed sphere [10] or ellipsoid [19] and the volumetric center [20].

Since m n , it would be preferable to use a cutting plane approach to solve (P), whereinonly a promising set of constraints or hyperplanes are kept, with hyperplanes being added/deleted ateach iteration. Such an approach has been provided by Vaidya [20] (based on the volumetric barrier),and Atkinson & Vaidya [5](based on the logarithmic barrier). These authors discuss a cutting planealgorithm for finding a feasible point in a convex set. At each iteration, they maintain a polytope thatis guaranteed to contain the convex set, and use the analytic center of this polytope as the test point. Ifthis point is not contained in the convex set, an oracle returns a hyperplane that separates the test pointand the convex set. This hyperplane is used to update the polytope. Further, hyperplanes currentlyin the polytope that are deemed ‘unimportant’ according to some criteria are dropped. Clearly, suchan algorithm can be very useful in solving problems like (P ). Much like the ellipsoid algorithm(see [12]), these algorithms can be applied to linear programming problems by cutting on a violatedconstraint when the test point is infeasible, and cutting on the objective function when the test pointis feasible but not optimal. However, from a computational point of view, this is an unwise strategy,and is likely to perform much like its worst case bound all the time (like with the ellipsoid algorithm).This is particularly so if the cuts (be it violated constraints or objective function cuts) are ‘backed-off’instead of adding it through the current point, as is the case in Vaidya [20] and Atkinson & Vaidya [5].In both these cases, the cuts are backed off and added so that the next iterate is guaranteed to be in asmall ellipsoid around the current point, suggesting that objective improvement cannot be too much(or improvement towards feasibility, if we add a violated constraint). Yet another disadvantage is thatin order to seek progress in the objective, we need to add a cut, increasing the number of constraints.

In [14], Mitchell and Ramaswamy give a long step cutting plane algorithm based on the loga-rithmic barrier. This may be viewed as casting the cutting plane algorithm of [5], which was basedon the analytic center (see [18]), in a long step framework. In stages where the objective is sought tobe improved, the duality gap is cut by half, which is better than in [5]. Also, seeking progress in theobjective amounts to reducing the numerical value of the barrier parameter, instead of adding a con-straint. The long step cutting plane algorithm had a complexity ofO(nL2), same as that of Atkinsonand Vaidya.

1

Here, our goal is to do the same with the cutting plane algorithmof Vaidya [20], which is based onthe volumetric barrier. This algorithm, while theoretically the best known to date, has many com-putationally undesirable features. Indeed, at first sight, the algorithm and the constants associatedwith it seem complicated and impractical, which may explain the lack of any computational studyof Vaidya’s algorithm in the literature. The recent work of Anstreicher [2] achieves great improve-ments in the constants used by Vaidya, making the algorithm seem viable. We believe that casting itin a long step framework is a further step in this direction, even at the expense of a slightly weakercomplexity result. Indeed, long step algorithms typically always have a slightly poorer global com-plexity, but tend to perform much better in practice.

It is important to fully appreciate the importance of Vaidya’s result - recently, Bertsimas and Or-lin [7] have given complexity for several combinatorial optimization subproblems using Vaidya’salgorithm as a subroutine. These complexity results improve on the previously best known for sev-eral problems, including for example computing the Held-Karp lower bound for Traveling Salesper-son Problems among others. We believe that our long step version, while theoretically weaker, isdesigned specifically to solve the kind of large linear problems encountered in [7], and is likely toshow better performance.

Thus, our interest is in a long step cutting plane algorithm that uses the volumetric barrier func-tion explicitly, partly because of the successful implementation of such algorithms (which were basedon the logarithmic barrier) for linear programming. Another point is that by making use of the longstep framework, we avoid having to increase the number of constraintswhenever we need to drive theobjective function value down - instead of cutting on the objective function, we just reduce the valueof a scalar parameter. Also, we add violated cuts right through the current point, because it wouldnot improve our complexity theoretically if we were to back off the cuts as in Vaidya [20]. We alsodiscuss the addition of multiple cuts, which is an important aspect of any successful implementationof a cutting plane algorithm. Finally, since in such an algorithm we would maintain primal and dualvariables, it allows for early termination when the sub-optimality is deemed to be within allowablelimits. In our algorithm, when an iterate is infeasible in the ‘overall problem’, constraints are addeduntil feasibility is regained. Additionally, constraints are dropped whenever possible according tosome criterion. We prove a complexity of O(n1.5L) iterations for this algorithm.

2 Preliminaries

The original problem of interest is :

(P ) min cTx

Ax ≥ b , x ∈ <n

where A ∈ <m×n.Its dual is :

(D) max bTy

ATy = c

y ≥ 0, y ∈ <m

where AT ∈ <n×m and m n.

2

We point out here that we do not actually require m to be finite — we will never work withthe formulations (P ) or (D) directly. We will just approximate them by a finite number of linearconstraints. We also assume that there exists a problem dependent constant L such that(i) The set of optimal solutions to (P ) is guaranteed to be contained in the n-dimensional hypercubeof half-width 2L given by x ∈ <n : |xi| < 2L, and(ii) The set of optimal solutions to (P ) contains a full dimensional ball of radius 2−L.We may interpret (ii) as saying that it suffices to find a solution to (P ) to within an accuracy 2−L.

At any given iteration, we operate with a relaxation of (P ). The algorithm is initialized with therelaxation

(P0) min cTx

cTx ≥ −‖c‖12L

Ix ≥ (−2L)en−Ix ≥ (−2L)en

where F (P0) ie., the feasible set of this relaxation contains X , which denotes the set of optimalsolutions to (P ). The vector of dimension nwhose components are all equal to one is denoted by en,or simply e. Indeed, e is used to denote the vector of all ones with the dimension being understoodfrom the context. It should be noted that n is always the same as n, and we may thus drop the hats.This is also the case with c.

At iteration k, we would have something like :

(Pk) min cTx

Akx ≥ bk

with Ak ∈ <mk×n.Throughout most of the discussion, we may omit the subscripts k since that causes no ambiguity.

Also, we sometimes use P to refer to the problem as well as the polytope defined by its constraints.Again, we do this since it is clear from the context which we are referring to.

We need to define the so-called logarithmic barrier function :

F (x) ≡ −m∑i=1

ln si (1)

where si is equal to aTi x − bi , and ai is the ith row of A. We use si without explicitly noting itsfunctional dependence on x.

It is easily seen that the gradient of the barrier function is given by

∇F (x) = −m∑i=1

aisi

= −ATS−1e (2)

and the Hessian by

H(x) ≡ ∇2F (x) =m∑i=1

aiaTi

s2i= ATS−2A (3)

Now, define the so-called Volumetric Barrier function:

V (x) ≡ 12

log det(∇2F (x)) (4)

3

Since we are interested in a long-step algorithm based on this barrier, we will instead directlywork with the function

ψ(x, π, l)≡ cTx− l

π+ V (x) (5)

where π is of course the barrier parameter, and l is some lower bound to the optimum objective func-tion value of (P ).

For a given value of π , x(π) denotes the unique minimizer of this barrier function. This mini-mizer exists and is unique if and only if the polytopeP is bounded — see, e.g., Anstreicher [2]. Werefer to this unique point when we use the term (exact) π-center.

In our analysis towards the end, we refer to the polytope P ∗. This polytope is to be understoodas follows — if we have the current point x, then

P ∗ ≡ P ∩ y ∈ <n : cTy ≤ β (6)

where β is equal to max(cTx, u), and u is an upper bound on the optimal value of cTx. Thus, thispolytope is certain to contain the optimal set to the problem of our interest and the current point.This property is important, and enables us to decide termination criteria based on the volume of thispolytope. It is important to note that the polytopeP ∗ is used only in the theoretical analysis - we donot explicitly add upper-bound constraints that restrict subsequent iterates to be in the polytopeP ∗,although we could.

For convenience, we sometimes use the notation

‖v‖B ≡√vTB−1v

for any v ∈ <n where B is a symmetric positive definite matrix.The outline of this paper is as follows. In the rest of this section, we discuss the properties of

the volumetric barrier function, including analyzing the convergence of the quasi Newton steps (i.e.,Newton like steps based onQ instead of ∇2ψ). The reader interested in the main thrust of this papermay want to skip this, referring back only as necessary. In section 3, we present the algorithm. In thesection after that, we analyze the complexity of the algorithm, considering cut addition, cut deletionand reducing the barrier parameter in separate subsections. In section 5 we consider this algorithmwith the addition of multiple cuts in case 2.1. Finally, in section 6, we state our conclusions anddiscuss other algorithms from the literature.

2.1 Properties of the Volumetric Barrier

Here, we state some results that we need about the volumetric barrier function. When it causes noconfusion to do so, we suppress the dependence of ψ(·) on π and l.

Proposition 1 Let ψ(x) be defined as in equation (5). Define

P (s) = S−1A(ATS−2A)−1ATS−1

and let P = P P where denotes the Schur or Hadamard product (i.e., Pij = P 2ij). Define

σi ≡ Pii, and Σ ≡ diagσi. We then have

g(x) ≡ ∇ψ(x) =c

π−

m∑i=1

σiaisi

=c

π− ATΣS−1e (7)

∇2ψ(x) = ATS−1(3Σ − 2P )S−1A = ∇2V (x) (8)

4

Proof :See Anstreicher [2].

It is worth noting that

σi(x) =aTi H(x)−1ai

si(x)2

As in Vaidya [20], we will want to deal with an approximation to ∇2V rather than ∇2V itself.We work towards that end.

Lemma 1 For any x ∈ P , we have(i) σi ≤ 1 ∀i.(ii)

∑mi=1 σi = n.

Proof :See Vaidya [20] or Anstreicher [2].

Now, let us define

Q(x) ≡m∑i=1

σiaia

Ti

s2i= ATS−2ΣA (9)

We want to claim that this matrix is a good approximation to the Hessian of V (x).

Lemma 2 LetQ(x) be defined as in equation (9), and let ∇2V (x) be defined as in equation (8). Wethen have, ∀x ∈ P ,

Q(x) ∇2V (x) 3Q(x)

where denotes the partial ordering induced by the cone of symmetric positive semidefinite matrices— for any two symmetric psd matricesA and B, A B if and only if B −A is symmetric positivesemidefinite.

Proof :See Anstreicher [2].

Now that we have established that the matrixQ(x) can serve as a reasonable approximation to theHessian of V (x), it is natural to want to take Newton-like steps based onQ(x) rather than ∇2V (x).Formally speaking, we need to define the notion of proximity to the exact π-center, and show thatwhen we are close to the exact center (measured by the proximity measure) the Newton-like stepsgive us good convergence. But before we can state that, we need some technical lemmas and propo-sitions.

We first state a very well known proposition — a proof may be found in any standard text onmatrices [13]. This is a result that we make heavy use of later in our proofs.

5

Proposition 2 Let A and B be n × n symmetric positive definite matrices, and let t be a positivescalar. Then,

A tB

implies thatA−1 (1/t)B−1

Definition 1 For any x ∈ P, r ∈ <, r ≥ 0, define Σ(x, r) as y ∈ <n : |aTi (y−x)|aT

i x−bi≤ r ∀i.

We note that Σ(x, r) ⊆ P for r ≤ 1.The set Σ(x, r) was originally introduced by Vaidya [20] (see also [4, 6]), and may be thought of

as a polyhedral neighbourhood, and a generalization of a small ball around the current point. Now,we can develop approximations to various quantities at points in this generalized neighborhood.

Proposition 3 Let x ∈ P . Let y ∈ Σ(x, r) for some r < 1. Then, for all ξ ∈ <n,

(1 − r)2

(1 + r)4ξTQ(x)ξ ≤ ξTQ(y)ξ ≤ (1 + r)2

(1− r)4ξTQ(x)ξ

1(1 + r)2

ξTH(x)ξ ≤ ξTH(y)ξ ≤ 1(1− r)2

ξTH(x)ξ

Proof :See Vaidya [20].

It will turn out that the relationship between the matrices H (the Hessian of the logarithmic bar-rier) and Q will be important to us. Towards quantifying that relationship, define

µ(x) ≡ infξ∈<n

ξTQ(x)ξξTH(x)ξ

(10)

It is worth noting that µ ≤ 1 and µ ≥ mini=1 to m σi. Having defined µ, the following result isquite straightforward.

Proposition 4

µH Q ≤ H

Proof :Follows from the definition of µ, and the fact that µ ≤ 1.

We will also want to approximate the quantity µ(y) for y ∈ Σ(x, r).

Proposition 5 Let x ∈ P . Let y ∈ Σ(x, r), r < 1. We then have

(1 − r)4

(1 + r)4µ(x) ≤ µ(y) ≤ (1 + r)4

(1− r)4µ(x)

6

Proof :See Vaidya [20].

We now state another technical lemma that we need for future use.

Lemma 3 Let x ∈ <n be any point in int(P ). Then, ∀i from 1 to m, we have

aTi Q(x)−1aisi(x)2

≤ 1√µ(x)

∀x ∈ P

Proof :See Vaidya [20].

Lemma 4

E(Q(x0), x0, (µ(x0))14 r) ⊆ Σ(x0, r)

Proof :See Vaidya [20].

We are now ready to proceed further. Our final goal is to establish that the quantityg(z)TQ(z)−1g(z)defines a measure of proximity of the point z to theπ-center. Roughly speaking, we want to establishthat if this measure is small, then ψ(z) − ψ(z∗) is small, and vice versa, where z∗ denotes the un-known minimizer ofψ(x). We will do this as in [20], by integrating some quantity along a trajectoryfrom z∗ to z. To formalize this, we must first establish the existence of the trajectory.

Theorem 1 (Vaidya) Let v ∈ <n be fixed. Define Φ(x, t) ≡ g(x) − tv, where t is a scalar andx ∈ P . Then the equation Φ(x, t) = 0 implicitly defines x as a function of t, and we may writex = x(t) along this trajectory. Moreover, x(t) is an analytic function of t, and

dx(t)dt

= ∇2ψ(x(t))−1v

Further, if x(t1) and x(t2) are two points on this trajectory, we can write

ψ(x(t2))− ψ(x(t1)) =∫ t2

t1

tvT∇2ψ(x(t))−1vdt

Proof :This theorem, as well as the underlying idea of using this trajectory are due to Vaidya [20]. A proofof this may be found there.

7

It may by now be clear how we plan to boundψ(z)−ψ(z∗) — by usingv = ∇ψ(z) in theorem 1. Weknow we may approximate the quantity vT∇2V (x(t))−1v inside the integrand by vTQ(x(t))−1v.But in order to boundQ(x(t)) independently of t, so that we may pull it outside the integral, we needto know that the entire trajectory lies inside Σ(z, r) for some small r. We need some more technicallemmas before we can do this.

Lemma 5 Let v ∈ <n be fixed. Let x be a fixed point inP , and suppose that∇ψ(x) = tv. Considerthe trajectory defined by∇ψ(x(t)) = tv. For the portion of this trajectory that lies in Σ(x, r) (wherer < 1), we have for every i = 1 to m,

| aTidx(t)dt

aTi x(t) − bi| ≤ (1 + r)3

(1− r)2(vTQ(x)−1v)1/2

(µ(x))1/4

Proof :Note that dx(t)dt = ∇2V (x(t))−1v. Thus,

|aTidx(t)dt

| = |aTi ∇2V (x(t))−1v|≤ (aTi ∇2V (x(t))−1ai)1/2(vT∇2V (x(t))−1v)1/2

Therefore,

|aTi dx(t)dt |si(x(t))

≤ (aTi ∇2V (x(t))−1ai)1/2

si(x(t))(vT∇2V (x(t))−1v)1/2

≤ (aTi Q(x(t))−1ai)1/2

si(x(t))(vTQ(x(t))−1v)1/2

≤ (vTQ(x(t))−1v)1/2

(µ(x(t)))1/4

where the last step follows from lemma 3. Now, all we need is to approximateQ(x(t)) and µ(x(t))for all x(t) ∈ Σ(x, r). Doing this using propositions 3 and 5, we get the desired result.

Now, we use the above lemma to show how far one must go along the trajectory (measured interms of the difference in t values) before one leaves the set Σ(x, r). This proof is exactly along thelines of a similar proof for the weighted analytic center that may be found in [4].

Lemma 6 Let v ∈ <n be fixed. Let x be a fixed point inP , and suppose that∇ψ(x) = tv. Considerthe trajectory defined by ∇ψ(x(t)) = tv. Let t be such that x(t) is not in Σ(x, r), for some r < 1.Then, we have

|t− t| ≥ (r− 0.5r2)(1 − r)2(µ(x))1/4

(1 + r)3(vTQ(x(t))−1v)1/2

Proof :WLOG, let x(t) be the first point that lies outside Σ(x, r). Then, we must have, for some k,

aTk x(t) − bk

aTk x(t) − bk= 1 + r

8

oraTk x(t) − bk

aTk x(t) − bk= 1 − r

In either case, we have

| ln(aTk x(t) − bk

aTk x(t) − bk)| ≥ r − r2

2

But

| ln(aTk x(t) − bk

aTk x(t) − bk)| = | ln(sk(x(t)))− ln(sk(x(t)))|

= |∫ t

t

aTkdx(t)dt

sk(x(t))dt|

≤∫ t

t| a

Tkdx(t)dt

sk(x(t))||dt|

≤ |t− t| (1 + r)3

(1− r)2(vTQ(x)−1v)1/2

(µ(x))1/4

where the last inequality follows from lemma 5. The final result of this lemma follows by rearrangingterms.

2.2 Convergence of Quasi-Newton Steps

Now, we are finally ready to establish that Ψ(z) ≡ g(z)TQ(z)−1g(z) indeed is a reasonable measureof proximity to the exact π-center.

Lemma 7 Let δ = 10−4. Let z ∈ P , and suppose that g(z)TQ(z)−1g(z) ≤ δ√µ(z). Then,

(i) z∗ ∈ Σ(z, 1.1√δ), where z∗ denotes the unknown minimizer of ψ(z).

(ii) µ(z) ≤ 1.1µ(z∗).(iii) ψ(z)− ψ(z∗) ≤ 0.55g(z)TQ(z)−1g(z).(iv) ψ(z)− ψ(z∗) ≥ 0.156g(z)TQ(z)−1g(z).

Proof :(i) Consider the trajectory defined by ∇ψ(x) = t∇ψ(z). Clearly, x(1) = z and x(0) = z∗. Now, letus look at Σ(z, r) for r = 1.1

√δ. Let t be such that x(t) 6∈ Σ(z, r). Applying the result of lemma

6, we have

|t− 1| ≥ (r − 0.5r2)(1− r)2[µ(z)]1/4

(1 + r)3[g(z)TQ(z)−1g(z)]1/2

≥ 1.03

for r = 1.1√δ, under the hypothesis of this lemma. As a consequence, we conclude that the trajec-

tory x(t) is contained in Σ(z, 1.1√δ) for t ∈ [0, 1]. Thus, x(0) = z∗ lies in Σ(z, 1.1

√δ).

(ii) Noting that z∗ ∈ Σ(z, r) for r = 1.1√δ, and applying proposition 5, this part follows directly.

(iii) We will integrate along the trajectory defined by g(x(t)) = tg(z), which we know from (i) is

9

contained in Σ(z, r) for r = 1.1√δ. Using theorem 1 and lemma 2 (in the third step) and proposition

3, we can write

ψ(z)− ψ(z∗) =∫ 1

0

dψ

dtdt

=∫ 1

0∇ψ(x(t))T

dx

dtdt

=∫ 1

0tg(z)T∇2V (x(t))−1g(z)dt

≤∫ 1

0tg(z)TQ(x(t))−1g(z)dt

≤ (1 + r)4

(1 − r)2g(z)TQ(z)−1g(z)

∫ 1

0tdt

≤ 0.55g(z)TQ(z)−1g(z)

where the last two steps use r = 1.1√δ.

(iv) As above, we can write

ψ(z)− ψ(z∗) =∫ 1

0

dψ

dtdt

=∫ 1

0tg(z)T∇2V (x(t))−1g(z)dt

≥ 13

∫ 1

0tg(z)TQ(x(t))−1g(z)dt

≥ 13

(1− r)4

(1 + r)2g(z)TQ(z)−1g(z)

∫ 1

0tdt

≥ 0.156g(z)TQ(z)−1g(z)

where the last two steps use r = 1.1√δ.

Thus, we have shown that if the measure is small, we are indeed in the proximity of the unknown

minimizer. Now we address the issue of convergence to this minimizer using quasi Newton stepsbased on Q. As before, we first define a trajectory - here it will simply be a straight line. We willshow that this trajectory is in the neighbourhoodof the current point, and then we will integrate alongit.

Lemma 8 Let z be a point satisfying gTQ(z)−1g < δ√µ(z). Define η = −Q(z)−1g. Let z(t) =

z + tη, for t ∈ [0, λ], λ < 1. Then we have

z(t) ∈ Σ(z, t√δ)

Proof :Note that z(t) − z = tη. Therefore,

(z(t) − z)TQ(z)(z(t) − z) = t2ηTQ(z)η= t2gTQ(z)−1g

≤ t2δ√µ(z)

= (t√δ(µ(z))

14 )2

10

Thus, we have z(t) ∈ E(Q(z), z, t√δ(µ(z))

14 ), which, by lemma 4, implies that z(t) ∈ Σ(z, t

√δ).

This tells us that once we are ‘close’ to the weighted volumetric center, we can even take a full stepin this quasi-Newton direction without violating feasibility. However, as we will soon see, we willbe restricted to less than a full step in order to guarantee a certain decrease in the volumetric potentialfunction. We will do this by integrating along the line we defined in lemma 8. But first we need somemore technical lemmas.

Lemma 9 Define ψ(t) := ψ(z(t)), where z(t) is defined in lemma 8. Choose λ = 0.2. Under thehypothesis of lemma 8, we can say that

d2ψ(t)dt2

≤ 3.037Ψ(z) ∀t ∈ [0, λ]

Proof :We have z(t) = z + tη. Therefore,

dψ(t)dt

= g(z(t))Tη

andd2ψ(t)dt2

= ηT∇2V (z(t))η ∀t ∈ [0, λ]

From lemma 2, we have ∇2V (z(t)) 3Q(z(t)). Thus,

d2ψ(t)dt2

≤ 3ηTQ(z(t))η ∀t ∈ [0, λ]

Also, lemma 8 and proposition 3 imply that

d2ψ(t)dt2

≤ 3ηTQ(z(t))η

≤ 3(1 + λ

√δ)2

(1− λ√δ)4

ηTQ(z)η

≤ 3.037Ψ(z)

where the last step uses λ = 0.2 and δ = 10−4.

Now we will use the above lemma to bound the first derivative dψdt in the interval t ∈ [0, λ], so

that we may use that bound to guarantee a certain decrease in the volumetric potential function.

Lemma 10 Let z satisfy Ψ(z) < δ√µ(z). Let η and λ be defined as before. Let z(t) = z + tη,

t ∈ [0, λ]. Thendψ(t)dt

≤ (−1 + 3.037t)Ψ(z)

11

Proof :

dψ(t)dt

=dψ(0)dt

+∫ t

0

d2ψ(s)ds2

ds

= g(z)Tη +∫ t

0

d2ψ(s)ds2

ds

≤ −Ψ(z) + 3.037tΨ(z)

We are now in a position to analyze the convergence of the quasi-Newton method.

Lemma 11 Let z satisfy Ψ(z) < δ√µ(z). Let η and λ be defined as before. Let z = z + λη,

λ = 0.2. Let z∗ denote the (unknown) minimizer of ψ(·, π). Then,

ψ(z) − ψ(z∗) ≤ 0.75(ψ(z)− ψ(z∗))

Proof :We note that Ψ(z) < δ

√µ(z) implies that ψ(z) − ψ(z∗) ≤ 0.55Ψ(z) by lemma 7, or equivalently

that Ψ(z) ≥ 1.8(ψ(z)− ψ(z∗)). Also,

ψ(z) − ψ(z) =∫ λ

0

dψ(t)dt

dt

≤ Ψ(z)∫ 0.2

0(−1 + 3.037t)dt

= −0.1392Ψ(z)≤ (1.8)(−0.1392)(ψ(z)− ψ(z∗))≤ −0.25(ψ(z)− ψ(z∗))

Thus, we can say that

ψ(z) − ψ(z∗) = ψ(z) − ψ(z) + ψ(z)− ψ(z∗)≤ 0.75(ψ(z)− ψ(z∗))

Thus, we have shown that if a point z is reasonably close to the exact π-center, as measured byΨ(z), then taking quasi-Newton steps results in linear convergence. However, it is also useful toanalyze the behaviour of the quasi-Newton steps starting from a point that is not an approximate(weighted) volumetric center. We do that now.

Lemma 12 Let z be any point in int(P ) that does not satisfy Ψ(z) < δ√µ(z). Let η := Q(z)−1g,

λ = 0.2, λ = λ√δ(µ(z))

14√

Ψ(z), and let z(t) ≡ z − tη for t ∈ [0, λ]. Then z(t) ∈ Σ(z, λ

√δ).

12

Proof :As before, note that z(t) − z = tη. Therefore,

(z(t) − z)TQ(z)(z(t) − z) = t2ηTQ(z)η= t2gTQ(z)−1g

= t2Ψ(z)≤ λ2Ψ(z)

= λ2δ√µ(z)

Thus, we have z(t) ∈ E(Q(z), z, λ√δ(µ(z))

14 ), which, by lemma 4, implies that z(t) ∈ Σ(z, λ

√δ).

Once again, as before, in order to analyze the quasi Newton step, we will need to bound the

second derivative d2ψ(t)dt2

, and then the first derivative.

Lemma 13 Under the hypothesis of lemma 12, we have

d2ψ(t)dt2

≤ 3.037Ψ(z) ∀t ∈ [0, λ]

Proof :This proof is identical to the proof of lemma 9.

Now we are ready to bound the first derivative along the entire portion of the step.

Lemma 14 Under the hypothesis of lemma 12, we can say that

dψ(t)dt

≤ (−1 + 3.037t)Ψ(z) ∀ t ∈ [0, λ]

Proof :Again, this proof is identical to that of lemma 10.

We can now put together the results of the previous lemmas to show that the quasi-Newton stepsstill converge to the π-center even when started from a point that is not an approximate center.

Lemma 15 Let z be a point in the interior of the polytopeP that does not satisfy Ψ(z) < δ√µ(z).

Let η, λ and λ be defined as in lemma 12. Let z := z − λη. Then,

ψ(z) − ψ(z) ≤ −0.1392δ√µ(z)

13

Proof :We have

ψ(z) − ψ(z) =∫ λ

0

dψ(t)dt

dt

≤ Ψ(z)∫ λ

0(−1 + 3.037t)dt

= Ψ(z)((12)(3.037)λ2 − λ)

≤ −0.1392δ√µ(z)

where the last step uses the definition of λ and the fact that Ψ(z) > δ√µ(z).

Thus, it is clear that at least some decrease is obtainable through these quasi Newton steps. Inorder to bound the magnitude of this decrease below, we need to place a lower bound on µ(z) forarbitrary z ∈ P .

Lemma 16 Let z be any point in P := x ∈ <n : Ax ≥ b, where A ∈ <m×n, and b ∈ <m. Letthe quantity µ(z) be defined as in equation (10). Then we can say that

µ(z) ≥ 1m

Proof :See Anstreicher [1].

This allows us to say that the quasi Newton steps will give us at least a Ω(1/√m) decrease in

ψ(·, π), even when started from a point that is not ‘close’ to the π-center. This result will be usefulto us later on.

3 The Algorithm

For the purpose of initializing the algorithm, we assume that the origin is feasible in problem (P ).This is reasonable since if we know any feasible point in (P ), we can modify the problem to makethe origin feasible.

The algorithm is shown in figure 1. It should be mentioned that where a lower case letter denotesa vector, the corresponding upper case letter denotes a diagonal matrix with this vector along thediagonal.

The basic idea of the algorithm is this : at any iteration, if a constraint has become unimportant(i.e., it has a small σj), then that hyperplane is dropped. If the constraints are deemed important, andthe current point is infeasible in (P ), then there is a violated constraint, and the polytope is updatedwith this constraint. (We assume that there is an ‘oracle’ that can find this violated constraint.) If thecurrent point is feasible in (P ), then the lower bound is updated and the barrier parameter is reduced.

14

Figure 1: The AlgorithmStep 0 : Initialization

π = 2L k = 0 A =

cT

In−In

b =

−2L‖c‖1

−2Len−2Len

x = [0 · · ·0]T ; s = Ax− b ;

Step k : The Iterative StageHave point x = xk, which is an approximate π-center.Calculate Γ(x) = mini=2 to m σi(x).

Case 1 : Γ< εThen, for some j we have σj(x) = min2 to m σi(x) < ε

Drop the hyperplane aj , resulting in polytope Pd.Take O(

√n+ log n) Quasi-Newton steps to find next approximate π-center xd.

Get dual solution y = πS−1Σ(e− S−1Ad), where d = Q−1∇ψ(xk, π, l).If satisfactory, STOP.

Case 2 : Γ ≥ ε

Subcase 2.1 : x is not feasible in (P )

Then ∃ j such that aTj x < bj .Add aTj x ≥ aTj x

k to the polytope, getting new polytope Pa. Set m = m+ 1.Find new approximate π-center xa in O(

√n+ logn) quasi Newton steps.

Subcase 2.2 : x is feasible in (P )Test x for optimality. If optimal, get dual solution y and STOP.Else,Set l = cTx− 2nπ. Let lprev denote previous lower bound.If lprev ≥ l, the lower bound cannot be improved upon.If lprev ≤ l, remove existing constraint cTx ≥ lprev, add cTx ≥ l.Set π ≡ πρwhere ρ ∈ (0.5, 1) is a constant independent of the problem. TakeO(n1.5+log n) quasiNewton steps with linesearch to find new approximate π-center.

15

Our convergence proof is as follows : first, we show that our dropping criterion implies that therecannot be more than O(n) number of constraints at any stage of the algorithm. Then, we use this toshow that if ψ(x, π) > O(nL), then the volume of some ellipsoid that contains the polytope musthave become too small, thus enabling termination. Finally, we show that this must happen in no morethan O(n1.5L) number of Newton steps.

In much of the analysis of this algorithm, we have only assumed that the current iterate is anapproximate π-center. However, for some of the proofs, we assume that we have an exact π-center.This helps in focussing attention on the key issue without having to keep track of details that wouldarise from having only an approximate π-center. It goes without saying that the algorithm itself usesonly approximate centers.

We now formally define what we mean by an approximate center.

Definition 2 A point z ∈ int(P ) is called an approximate π-center if it satisfies

∇ψ(z, π, l)TQ(z)−1∇ψ(z, π, l)< min(δ√µ(z), 1/

√m)

The motivation for part of this definition must be clear - the other part will be considered in sec-tion 4.3

4 Complexity Analysis

In this section, we attempt to show that every stage of the algorithm may be accomplished in a rea-sonable number of Newton steps, and we also establish the global complexity of the algorithm. Inseparate subsections, we analyze the necessary lemmas pertaining to adding a cut, dropping a cut andreducing the barrier parameter. An important difference between our cutting plane algorithm and thatof Vaidya [20] or Anstreicher [2] is that we never drop the lower bounding constraint cTx ≥ l. Weneed this constraint to be always present in the current system of inequalities for our complexity anal-ysis. Therefore, no matter what the value of σ1(x) = cT (∇2F (x))−1c/(cTx − l)2, we do not dropthis constraint. This weakens our local convergence results - we prove a bound ofO(

√n) steps after

adding a cut in order to find the next π-center. However, this does not affect the overall complexityof the algorithm, which would anyway be O(n1.5L). We do make up in some measure, because thisallows us to add the new cut directly through the current point.

4.1 Adding a Cut

4.1.1 Effect of Adding a Cut on the Volumetric Potential

Here, we analyze the effect of adding a cut on the volumetric potential function ψ(·) - in particular,if z denotes the current π-center, z denotes the new π-center, and ψ(·) denotes the potential functionwith the new cut included as well, then we want to show that there is some constant ηa such that

ψ(z, π) ≥ ψ(z, π)+ ηa

We note that since this is an occurrence of case 2.1, we must have min2 to m σj > ε. However,it is possible to have σ1 < ε at this point in the algorithm. An increase in the potential function iseasy to show under the case when the lower bounding constraint satisfies σ1(z) > ε — indeed, thelemmas of Vaidya [20] and Anstreicher [2] apply directly. When this is not the case, we are forced

16

to use the property that all the other sigmas are bigger than ε. This complicates matters significantly.Since this is the harder case, we only prove our results for this case.

First we need some technical lemmas.We will begin by considering how far the new minimizer must be from the old one. Let all quan-

tities with tildes refer to the system of inequalities where the lower bounding constraint is presentbut ignored. What we mean is that it is not dropped (the algorithm requires that we keep it in), butwe ignore it. Thus,

Q =m∑2

σi/s2i aia

Ti (11)

H =m∑2

1/s2i aiaTi (12)

µ ≡ infξ∈<n

ξT Qξ

ξT Hξ> ε (13)

Define r ≡ ‖Q(z)12 (z−z)‖

µ(z)14

. It follows that

z ∈ E(Q(z), z, µ(z)14 r)

Thus, r is a good measure of the ‘distance’ of z from z.Since we will now be working with Q instead of withQ, we will need to relate the two.

Lemma 17 Let x ∈ int(P ). Let Q(x) and H(x) be defined as before. Suppose that σ1(x) < ε.Define

Q(x) ≡ Q(x)− σ1(x)s1(x)2

ccT

and

H(x) ≡ H(x)− ccT

s1(x)2

(i.e., the lower bounding constraint is ignored). Then, for any ξ ∈ <n, we have

‖ξ‖Q ≤ ‖ξ‖Q/√

1− ε

and‖ξ‖H ≤ ‖ξ‖H/

√1 − ε

Proof :Consider the first part.

‖ξ‖2Q

≡ ξT Q−1ξ

= ξT (Q− σ1ccT/s21)

−1ξ

≤ ξTQ−1ξ/(1 − σ1cTQ−1c

s21)

≤ ξTQ−1ξ/(1 − σ1cTH−1c

µs21)

17

= ξTQ−1ξ/(1 − σ1σ1

µ)

≤ ξTQ−1ξ/(1 − σ1)= ξTQ−1ξ/(1 − ε)

where in the third step, we used the fact that the smallest eigenvalue of the matrix (I−σ1Q−1ccT/s21)

is equal to 1 − (σ1cTQ−1c/s21). In the penultimate step, we used the fact that µ ≥ σmin = σ1 (see

[20]), and the fact that σ1 < ε was used in the last step. Similarly, we have

‖ξ‖2H

≡ ξT H−1ξ

= ξT (H − ccT/s21)−1ξ

≤ ξTH−1ξ/(1− cTH−1c/s21)= ξTH−1ξ/(1− σ1)≤ ξTH−1ξ/(1− ε)

The result follows.

We need one other technical lemma before we can say that z is in some polyhedral neighbourhood

of z.

Lemma 18 Let x ∈ int(P ). LetQ(x) and Q(x) be defined as before. Let H(x) and µ(x) be definedas in equations (12) and (13). We then have

aTi Q(x)−1ais2i

≤ 1/√µ(x)(1− ε) ∀ i = 2 to m

Proof :Let A denote the current constraint matrix A with the first row removed. Let Σ and S denote thebottom-right (m− 1)× (m− 1) submatrices of Σ and S respectively. Then consider the projectionmatrix

M ≡ Σ0.5S−1A(AT S−2ΣA)−1AT S−1Σ0.5

We have

Mii = σi+1aTi+1Q(x)−1ai+1

s2i+1

≤ 1 ∀ i = 1 to m− 1

Therefore,aTi Q(x)−1ai

s2i≤ 1/σi ∀ i = 2 to m (14)

But

σi =aTi H(x)−1ai

s2i

=aTi (H + ccT/s21)

−1ais2i

≥ aTi H−1ais2i

(1− σ1)

≥ aTi Q−1ais2i

µ(1 − σ1) (15)

18

Substituting (15) in (14) and using the fact that σ1 < ε, we get the desired result.

Lemma 19

z ∈ E(Q(z), z, µ(z)14 r) ⊆ Σ(z, r/

√1 − ε)

Proof :From the very definition of r it is obvious that z ∈ E(Q(z), z, µ(z)

14 r). Only the second part needs

to be shown. From lemma 18, we know that

aTi Q−1ais2i

≤ 1/√µ(1− ε)

for i = 2 to m - i.e., all but the lower bounding constraint. Then, for i = 2 to m,

|aTi (z − z)|si(z)

≤ µ14 r

√aTi Q

−1ai/si

≤ r/√

1 − ε

Now consider the first constraint.

|cT (z − z)|s1(z)

≤ µ14 r

√cT Q−1c

s1(z)

≤ µ−14 r

√cT H−1c

s1(z)

≤ µ−14 r

√σ1/(1 − σ1)

≤ µ−14 r

√ε/(1− ε)

≤ rε14/√

1− ε

≤ r/√

1 − ε

where the inequality in the last-but-one step follows from µ > ε.The result follows.

Henceforth, we denote r ≡ r/√

1 − ε.First, we show that if r > 1

4 , then there is a constant ηa such that ψ(z) ≥ ψ(z) + ηa.

Lemma 20 Let r be defined as above. If r > 14 , then

ψ(z) ≥ ψ(z) + ηa

where ηa = 7.1× 10−7.

19

Proof :Consider a straight line joining z and z. Let z be a point on this line such that ‖Q(z)

12 (z − z)‖ =

0.25µ14 . Let r ≡ 0.25/

√1 − ε. Define z(t) ≡ z + t(z − z). Clearly we have z(t) ∈ Σ(z, r). Then

we have

dψ(z(t))dt

= ∇ψ(z(t))T (z − z)

d2ψ(z(t))dt2

= (z − z)T∇2ψ(z(t))(z − z)

Also,

d2ψ(z(t))dt2

= (z − z)T∇2ψ(z(t))(z − z)

≥ (z − z)TQ(z(t))(z − z)

≥ (1− r)2

(1 + r)4(z − z)TQ(z)(z − z)

≥ (1− r)2

(1 + r)4(z − z)T Q(z)(z − z)

=(1− r)2

(1 + r)4õ/16

≥ (1− r)2

(1 + r)4√ε/16

Therefore,

dψ(z(t))dt

=∫ t

0

d2ψ

dt2dt

≥ (1− r)2

(1 + r)4t√ε/16

Thus,

ψ(z) − ψ(z) ≥ ψ(z)− ψ(z)≥ ψ(z)− ψ(z)

=∫ 1

0

dψ

dtdt

≥∫ 1

0

(1 − r)2

(1 + r)4t(√ε/16)dt

=(1− r)2

(1 + r)4√ε/32

≥ 7.1× 10−7

Now, we will establish that we can guarantee a certain increase in the potential function even ifr ≤ 1

4 . But first, we need to prove a another technical lemma.

20

Lemma 21 Let z be the current exact π-center. Let the cut vTx ≥ vT z be added, and let z denotethe new exact π-center. Let r be as defined earlier. Define

q(z) ≡ vTH(z)−1v

sv(z)2

where sv(·) denotes the slack corresponding to this new constraint. Under the hypothesis that r < 14 ,

we can say that

q(z) ≥ 78√ε(1 − ε)

1r2

Proof :First, we note that z ∈ E(Q(z), z, µ(z)

14 r). Thus,

sv(z) ≡ vT (z − z)

≤ µ(z)14 r

√vT Q(z)−1v

Now,

q(z) =vTH(z)−1v

sv(z)2

≥ (1 − r)2vTH(z)−1v

sv(z)2

≥ (1 − r)2vTH(z)−1v

µ(z)12 r2vT Q(z)−1v

≥ (1 − r)2(1 − ε)vT H(z)−1v

µ(z)12 r2vT Q(z)−1v

≥ (1 − ε)(1 − r2)√µ/r2

≥ (1 − ε)(1 − r2)√ε/r2

≥ 78(1− ε)

√ε/r2

where r ≡ r/√

1 − ε, the fourth step uses lemma 17, and the last step uses r < 14 and ε < 1

2 .

Now we may use this lemma to state the result we have been working towards.

Lemma 22 Let z denote the current π-center, and let the cut vTx ≥ vT z be added. Let z denotethe new π-center. Let r be defined as before. Under the hypothesis that r < 1

4 , we can say that

ψ(z) ≥ ψ(z) + 6.99× 10−4

where ε is chosen to be 10−8.

21

Proof :

ψ(z) =12

log det(H(z) +vvT

sv(z)2) +

cT z − l

π

=12

log[(det H(z))(1 +vTH(z)−1v

sv(z)2)] +

cT z − l

π

= ψ(z) +12

log(1 + q(z))

≥ ψ(z) +12

log(1 + q(z))

where the last step follows because z is the minimizer of ψ(·). Also, we have

q(z) ≥ 78(1− ε)

√ε/r2

≥ 78(1− ε)

√ε/0.252

≥ 1.399× 10−3

The result then follows.

At this point, we have shown that adding a cut always increases the potential function by at leasta constant amount. We have simplified the analysis by considering the difference only between thecurrent (exact) π-center and the (exact) π-center of the new polytope. This has been done only tominimize the additional book-keeping and focus on the main idea - similar results may easily beproven using approximate centers, by simply considering the difference in potential function be-tween an approximate center and an exact one. Further, we have performed the analysis for the casewhere σ1 < ε, which is indeed the harder case. It is straightforward to repeat the above analysis forthe case where this is not so. Indeed, this would be simpler since we need not consider the Qmatrix,and can do a more direct analysis. Furthermore, the lemmas in [20, 2] would be directly applicableif we had σ1 > ε, and we would not need to prove any results.

Now we will consider the affine step that we will take after adding the new cut through the currentpoint. Again, for simplicity, we will assume that we have an exact π-center.

4.1.2 The Affine Step

Here we study the mechanics of adding a new cut right through the current (exact, for simplicity)π-center and finding the next approximate π-center. The idea of adding cuts right through the currentinterior point and using an affine step to regain an interior point may be found, for example, in [15,11, 16]. In particular, if x is the point after taking an affine step, then we will show that ψ(x)−ψ(z)is bounded above by a constant, which will then imply that the new π-center z may be approximatedin O(

√m) quasi-Newton steps. Again, as before, things are complicated by the fact that σ1 may be

less than ε, and again we need to use the fact that all other sigmas are greater than ε. Here we do thenecessary analysis only for the case where σ1(z) < ε – it is straightforward to see that the other casecan only further strengthen our results.

22

Let z ∈ int(P ) be the current π-center. Let it be the case that σ1 < ε. Define daff ≡ βr Q(z)−1v,

where vTx ≥ vT z is the new cut, β is a constant, and r ≡ ‖v‖Q(z). Let x ≡ z + daff . Our goal is

to show that ψ(x) − ψ(z) is bounded by a constant.

Lemma 23

x ∈ Σ(z,β√

(1− ε)µ(z))

Proof :

s = s(z) + ds

= s(z) +β

rAQ−1v

= S(z)(e+β

rS−1AQ−1v)

Within this proof alone, denote the vector βr S−1AQ−1v by p. Then,

‖p‖2 = (β

r)2vT Q−1ATS−2AQ−1v

= (β

r)2vT Q−1H(z)Q−1v

≤ (β

r)2vT Q−1H(z)Q−1v

11 − ε

≤ 1µ(z)

(β

r)2vT Q−1QQ−1v

11 − ε

=β2

µ(1 − ε)

where we again used lemma 17 in the third step.Thus, ‖p‖ ≤ β√

µ(z)(1−ε) , which implies the result.

This immediately gives us a step length for the affine step - we can safely chooseβ to be 0.5√µ(z) ≥

0.5×10−4. The point xwill then be in the interior of the new polytope, and it will also be far enoughaway for us to bound ψ(x) − ψ(z) above by a constant.

Lemma 24 Let z denote the current π-center, and let the cut vTx ≥ vT z be added. Suppose thatσ1 < ε = 10−8. Let x ≡ z + β

r Q(z)−1v, where r ≡ ‖v‖Q(z). Let quantities with hats refer to the

augmented system of inequalities. Let τ ≡ β/√

(1 − ε)µ(z). Then, we can say that

ψ(x) ≤ ψ(z) + α1

for some constant α1.

23

Proof :First, we observe that lemma 23 and the definition of τ imply that x ∈ Σ(z, τ). Consider the lineconnecting z and x. Define z(t) ≡ z + t(x− z) ∀t ∈ [0, 1]. We then have

dψ(z(t))dt

= ∇ψ(z(t))T (x− z)

Also,

d2ψ(z(t))dt2

= (x− z)T∇2ψ(z(t))(x− z)

≤ 3(x− z)TQ(z(t))(x− z)

≤ 3(1 + τ)2

(1− τ)4(x− z)TQ(z)(x− z)

≤ 3(1 + τ)2

(1− ε)(1− τ)4(x− z)T Q(z)(x− z)

= 3(1 + τ)2

(1− ε)(1− τ)4β2

Thus, we have

dψ(z(t))dt

≤ 3(1 + τ)2

(1− ε)(1− τ)4β2t

and

ψ(x)− ψ(z) ≤ 1.5(1 + τ)2

(1− ε)(1 − τ)4β2 (16)

Also, we have

q(x) ≡ vTH(x)−1v/sv(x)2

=vTH(x)−1v

β2r2

≤ (1 + τ)2vTH(z)−1v

β2r2

≤ (1 + τ)2vT H(z)−1v

β2r2

= (1 + τ)2vT H(z)−1v

β2vT Q(z)−1v

≤ (1 + τ)2/β2

Thus,

ψ(x) − ψ(z) = ψ(x)− ψ(z) +12

log(1 + q(x))

≤ 1.5(1 + τ)2

(1− ε)(1− τ)4β2 +

12

log(1 +(1 + τ)2

β2)

Finally, choosing β = 0.5× √µ(z), we have

0.5× 10−4 ≤ β ≤ 0.5

24

and τ ≤ 0.51. Using these, it follows that ψ(x) − ψ(z) is bounded above by a constant. Thus, theconvergence lemmas 11 and 15 imply that the next approximate π-center may be found in O(

√m)

Newton steps.

4.2 Dropping A Cut

In this section, we analyze what happens when case 1 of the algorithm has occurred. It is thereforethe case that σj = σmin < ε for some j 6= 1, and the minimum is only over 2 to m. Without lossof generality, we may assume that σm = minj=2 to m σj . Thus, at this stage, the mth constraint isbeing dropped. This leads to a decrease in the potential functionψ(·). However, our goal is to showthat this decrease is not too much. Specifically, we have to show that the maximum possible decreasewhen we drop a constraint is definitely less than the least possible increase when we add a cut.

As in the analysis of constraint addition, we run into trouble because of the fact that it is possibleto have σ1 < σm, and thus the results of Vaidya [20] or Anstreicher [2] do not apply here. Therefore,as before, we consider only the harder case of when σ1 < σm = σmin. We prove our results for thiscase. It is very straightforward that the results generalize to the case where σ1 ≥ σm.

Lemma 25 Let P = x|aTi x ≥ bi, i = 1 to m. (Recall that the first constraint is the lowerbounding constraint. Thus, a1 = c.) Let it be the case that σ1 < σm = mini=2 to m σi < ε. Letthe new polytope be defined by dropping themth constraint. Let z be the new π-center, and let ψ(·)denote the potential function corresponding to the new polytope. Define

Q(z) =m∑i=2

σi(z)aiaTi /si(z)2

H(z) =m∑i=2

aiaTi /si(z)

2

µ(z) = infξ∈<n

ξT Q(z)ξξT H(z)ξ

≥ σm

Also define r ≡ ‖Q(z)12 (z−z)‖

µ(z)14

. Then r < 14 .

Proof :Suppose that r > 1

4 . Let x be a point on the line joining z and z such that

‖Q(z)12 (x− z)‖µ(z)

14

=14

Within this proof alone, define r ≡ 0.25/√

1− ε. Exactly as in the proof of lemma 19, we can showthat

x ∈ E(Q(z), z, 0.25µ14 ) ⊂ Σ(z, 0.25/

√1 − σm) ⊆ Σ(z, 0.25/

√1− ε)

Let z(t) ≡ z + t(x− z). We then have, as before,

d2ψ(z(t))dt2

= (x− z)T∇2ψ(z(t))(x− z)

25

≥ (x− z)TQ(z(t))(x− z)≥ (x− z)T Q(z(t))(x− z)

≥ (1 − r)2

(1 + r)4(x− z)T Q(z)(x− z)

=(1 − r)2

(1 + r)4õ(z)/16

Therefore, we can say thatdψ(z(t))

dt≥ (1 − r)2

(1 + r)4tõ(z)/16

which then leads to the result that ψ(x) − ψ(z) ≥ (1 − r)2(1 + r)−4√µ(z)/32. Using r =

0.25/√

1 − ε and ε = 10−8, we get ψ(x) − ψ(z) ≥ 0.2332

√µ(z) ≥ 0.23

32

√σm(z). Also,

σm(x) =aTmH(x)−1am

sm(x)2

≤ (1 + r)2

(1 − r)2σm(z)

≤ 2.78σm(z)

Thus, we can write

ψ(x) = ψ(x) +12

log(1− σm(x))

≥ ψ(x) +12

log(1− 2.78σm(z))

≥ ψ(z) +0.2332

√σm(z) +

12

log(1 − 2.78σm(z))

≥ ψ(z) +0.2332

√σm − 1.39σm

(1 − 2.78σm)

We also know that σm < ε = 10−8. It may be verified that the function 0.2332

√x − 1.39x

1−2.78x isnonnegative and increasing in x for x ∈ [0, 10−8]. Thus,

ψ(x) ≥ ψ(z) ≥ ψ(z)

But strict convexity of ψ(·) implies that it should be strictly decreasing along the line joining zand z, and thus we must have ψ(x) < ψ(z). Thus we have a contradiction. It therefore has to be thecase that r < 1

4 .

Now, we are able to state the result we require.

Lemma 26 Under the conditions of lemma 25, we have

ψ(z) ≥ ψ(z)− ηd

where ηd = 2.781× 10−8. Thus, we also have ηd < ηa.

26

Proof :Using techniques used in the previous lemma, we can write

ψ(z) ≥ ψ(z) +0.232

õ(z)r2

andσm(z) ≤ k(r)σm(z)

where k(r) ≡ (1+r)2

(1−r)2 , and r = r/√

1 − ε. Thus,

ψ(z) ≥ ψ(z) +0.232

õ(z)r2 +

12

log(1 − σm(z))

≥ ψ(z) +0.232

√σm(z)r2 +

12

log(1− k(r)σm(z))

≥ ψ(z) +12

log(1− k(r)σm(z))

≥ ψ(z)− 12

k(r)σm(z)(1− k(r)σm(z))

≥ ψ(z)− 12

k(0.25)σm(z)(1− k(0.25)σm(z))

≥ ψ(z)− 12

k(0.25)ε(1− k(0.25)ε)

≥ ψ(z)− 2.781× 10−8

proving the result.

Corollary 1 When case 1 of the algorithmoccurs and a constraint is dropped, no more thanO(√m+

logm) quasi-Newton steps are needed to find the next approximate π-center.

Thus, we have established that adding a constraint through the current point and dropping a con-straint as per the algorithm are not too expensive regarding the time needed to find the next approx-imate center. Now, we need to analyze subcase 2.2 of the algorithm, when we drop the barrier pa-rameter.

4.3 Reducing the Barrier Parameter

In this subsection, we analyze subcase 2.2 of the algorithm, where we seek progress in the objectivefunction by reducing the barrier by a constant factor ρ ∈ [0.5, 1).

First, we have to justify the lower bound we use.

Lemma 27 Let x ∈ int(P ) satisfy∇ψ(x, π)TQ(x)−1∇ψ(x, π) ≤ 1/√m. Then cTx− z∗ ≤ 2nπ,

where z∗ denotes the optimal value of cTx over the current polytope. Also, y ≡ πS−1Σ(e−S−1Ad),where d ≡ Q−1∇ψ(x, π, l) is feasible in the dual with a duality gap of smaller than 2nπ.

Proof :See Anstreicher [3], lemma 3.2.

27

This implies that in our algorithm, whenever subcase 2.2 occurs, we are justified in setting the

lower bound to cTx − 2nπ after ensuring that x satisfies the required proximity criterion. (This istrue because the current polytope at that stage is guaranteed to contain x∗, the optimal solution toour original problem of interest).

Now we need to address the issue of how many quasi Newton steps may be required to find thenext approximateπ-center after reducing the barrier parameter. We need some technical lemmas firstbefore we can answer this.

Here, as before, we will assume that the current point is an exact π-center for the sake of sim-plicity. It is not hard to extend these results for the general case.

Lemma 28 Suppose that we have a point z ∈ int(P ), where P denotes the current polytope. Sup-pose that z is an exact π-center. Define H(z) ≡ ∑m

i=2 aiaTi /si(z)

2 as before. Then we have

cT H(z)−1c

n2π2≤ 1/(1− σ1(z))

Proof :Since z is an exact π-center, we have

c/π = ATΣS−1e

Therefore,

cTH(z)−1c

π2= eTΣS−1A(ATS−2A)−1ATS−1Σe

= σS−1A(ATS−2A)−1ATS−1σ

≤ ‖σ‖2

≤ ‖σ‖21

= n2

where σ ≡ [σ1 σ2 ...σm]T . Now, using the Sherman-Morrison-Woodbury formula,

cT H(z)−1c

π2=

cTH(z)−1c

π2(1− σ1)≤ n2/(1− σ1)

which proves the result.

Now we will bound the decrease inψ(·) that results when the barrier parameter is dropped. Here,

we need to be much more careful with our notation, since it must be emphasized that changing thelower bound changes the first constraint, and therefore the polytope. This is made explicit by writingψ(x, π, l).

Lemma 29 Assume we have a point x1, which is an exact π-center corresponding to a barrier pa-rameter value of π1. Case 2.2 of the algorithm occurs, and we reduce π1 to π2 = ρπ1, whereρ ∈ [0.5, 1). Let x2 denote the new exact π-center. Let l1 and l2 denote the lower bounds that areactive at the beginning and end of this occurrence of case 2.2.

We then haveψ(x2, π2, l2) ≥ ψ(x1, π1, l1) − 8n

28

Proof :Let x0 denote the point which corresponds to the previous occurrence of case 2.2. Thus, x0 is anapproximate center for value π = π0 = π1/ρ. At this (previous) stage, the candidate for a newlower bound would have been cTx0 − 2nπ0. Thus, we can say

l1 ≥ cTx0 − 2nπ0

Therefore, we have cTx1 − l1 ≤ cTx1 − cTx0 + 2nπ0. But since x0 was feasible in the overallproblem, cTx0 ≥ l2 ≥ cTx1 − 2nπ1. Thus,

(cTx1 − l1)/π1 ≤ 2nπ1 + 2nπ0 = 2nπ1(1 + 1/ρ) ≤ 6n (17)

where we use the fact that ρ ∈ [0.5, 1).Now,

ψ(x2, π2, l2) − ψ(x1, π1, l1) =cTx2 − l2

π2− cTx1 − l1

π1+ V (x2, l2) − V (x1, l1)

≥ −cTx1 − l1π1

+ V (x2, l2)− V (x1, l1)

≥ −6n + V (x2, l2) − V (x1, l1)≥ −6n + V (x2, l1) − V (x1, l1) + V (x2, l2) − V (x2, l1)

Since l2 ≥ l1, for all the points in the new polytope we must have V (x, l2) ≥ V (x, l1). Therefore,

ψ(x2, π2, l2) − ψ(x1, π1, l1) ≥ −6n + V (x2, l1)− V (x1, l1)

Now we need to bound the difference V (x2, l1) − V (x1, l1). Since x1 minimizes ψ(·, π1, l1), wehave

cTx1

π1+ V (x1, l1) ≤ cTx2

π1+ V (x2, l1)

or

V (x2, l1)− V (x1, l1) ≥ cTx1 − cTx2

π1

Unfortunately, we can no longer say that cTx2 ≤ cTx1, since by possibly changing the lower boundwe may have changed the polytope. However, since x1 is feasible, and because cTx2 − 2nπ2 isa valid lower bound on the optimal objective function value (irrespective of whether or not x2 isfeasible), we can say that cTx2 − 2nπ2 ≤ cTx1, or

cTx1 − cTx2 ≥ −2nπ2

Therefore,(cTx1 − cTx2)/π1 ≥ −2nπ2/π1 ≥ −2n

Thus, we have

ψ(x2, π2, l2) − ψ(x1, π1, l1) ≥ −6n − 2n = −8n

29

Lemma 30 Assume that we have a point x1 that is an exact π-center corresponding to π1, and case2.2 of the algorithm occurs. Therefore, the lower bound may possibly have been reset from l1 to l2,and π2 = ρπ1. We can then say

ψ(x1, π2, l2) ≤ ψ(x1, π1, l1) + 9n

Proof :Let l ≡ cTx1 − 2nπ1. If l > l1, then l2 = l, and otherwise l2 = l1. We have

ψ(x1, π2, l2) − ψ(x1, π1, l1) =cTx1 − l2

π2− cTx1 − l1

π1+ V (x1, l2) − V (x1, l1)

≤ cTx1 − l2π2

+ V (x1, l2) − V (x1, l1)

Now, l2 ≥ cTx1 − 2nπ1. Therefore,

ψ(x1, π2, l2) − ψ(x1, π1, l1) ≤ (2nπ1/π2) + V (x1, l2)− V (x1, l1)≤ 4n+ V (x1, l2) − V (x1, l1)

Case 1 : l2 = l1In this case we are done, since V (x1, l2) = V (x1, l1). Therefore,

ψ(x1, π2, l2) − ψ(x1, π1, l1) ≤ 4n

Case 2 : l2 = cTx1 − 2nπ1

Here we need to bound V (x1, l2) − V (x1, l1).

V (x1, l2) − V (x1, l1) =12

log det(H(x1)) +12

log(1 +cT H−1c

(cTx1 − l2)2)

− 12

log det(H(x1))− 12

log(1 +cT H−1c

(cTx1 − l1)2)

≤ 12

log(1 +cT H−1c

(cTx1 − l2)2)

=12

log(1 +cT H−1c

4n2π21

)

But by lemma 28, we have

cT H(x1)−1c

4n2π21

≤ 1/(4(1− σ1)) ≤ 1/(4ε)

where the last inequality assumes thatm ≥ n ≥ 2, and since case 2.2 has occurred, all other sigmasare greater than ε. Therefore,

V (x1, l2) − V (x1, l1) ≤ 12

log(1 + 0.25ε−1) ≤ 8.52

using ε = 10−8. Thus, we have

ψ(x1, π2, l2) − ψ(x1, π1, l1) ≤ 4n+ 8.52≤ 9n

where the last inequality again uses n ≥ 2. This proves the result.

30

Thus, to find the next π-center after dropping the barrier parameter, a reduction in ψ(·) of nomore than 17n is required. It follows that no more than O(n

√m + logm) quasi Newton steps are

necessary to find the next π-center.

Corollary 2 An occurrence of case 2.2 of the algorithm will need a total of no more thanO(n√m+

logm) quasi Newton steps.

4.4 Overall Complexity

Finally we are in a position to analyze the overall complexity of the algorithm.

Theorem 2 This algorithm will terminate in at most O(n√nL) quasi Newton steps.

Proof :At any stage in the algorithm, suppose that z is the current point that we have. Again, for simplicity,assume that z is an exact π-center, and is therefore the minimizer of ψ(·, π, l). Let P denote thecurrent polytope, which is defined by m constraints. Consider the polytope

P ∗ ≡ P ∩ x|cTx ≤ u

where u ≡ max(cT z, cTxprev). Thus, P ∗ is guaranteed to contain the current point as well as theoptimal solution. (This is because xprev is the point at which subcase 2.2 occurred most recently,which implies that it was feasible. Therefore, the optimal solution must have smaller objective func-tion value). We are interested in bounding the volume of P ∗ above.

For the analysis alone, imagine that the cut cTx ≤ u has been added. Thus, the polytope P ∗ isnow defined by m + 1 constraints. Let quantities with stars refer to this augmented polytope. Letxa and xv denote its analytic and volumetric centers respectively (of the augmented polytope). It isknown (see, e.g., [5]) that

P ∗ ⊆ E(H∗(xa), xa, m+ 1)

Therefore

vol(P ∗) ≤ (2(m+ 1))n(det(H∗(xa)))−12

≤ (2(m+ 1))n(det(H∗(xv)))−12

≤ (2(m+ 1))n(det(H(xv)))−12

= (2(m+ 1))ne−V (xv)

≤ (2(n

ε+ 3))ne−V (xv) (18)

where the last step follows because our dropping criterion implies thatm ≤ n/ε+2. (This is becauseall the sigmas have to sum to n).

Now, since z is the minimizer of ψ(·, π, l), we have

cTxv − l

π+ V (xv) ≥ cT z − l

π+ V (z)

31

or

V (xv) ≥ V (z) − cTxv − cT z

π

If it is the case that cTxv ≤ cT z, then V (xv) ≥ V (z). Otherwise, it must be the case that u =cTxprev . We can then say

cT z ≥ l

≥ cTxprev − 2nπprev= cTxprev − 2nπ/ρ≥ cTxprev − 4nπ

where the second step follows because the lower bound must have been set at the previous occurrenceof case 2.2, or it must have been even better than the candidate. Also, since xv ∈ P ∗, we havecTxv ≤ u = cTxprev. Therefore,

cTxv − cT z ≤ 4nπ

Thus, V (xv) ≥ V (z) − 4n. Finally, V (z) = ψ(z, π, l)− (cT z − l)/π. From an analysis exactlylike the one that preceded equation (17), we can say that

cT z − l

π≤ 6n

Therefore, V (z) ≥ ψ(z, π, l)− 6n. Putting it all together, we have

V (xv) ≥ ψ(z, π, l)− 10n (19)

Now it is clear from equations (19) and (18) that if ψ(·) > O(nL), then the volume of the polytopeP ∗ would have fallen below 2−nL, which is too small to contain a ball of radius 2−L. Thus thealgorithm must terminate if ψ(·) > O(nL).

Let us now look at the stages that occur in our algorithm. Each occurrence of case 2.2 can setthe value of ψ(·) back by upto O(n) — and there can be no more than O(L) of these. Thus, thetotal setback to ψ(·) because of the case 2.2 stages is O(nL). Also, since the number of constraintdeletions cannot be more than the number of constraint additions, and since ηa > ηd, it followsthat inO(nL) non-case 2.2 stages, the value of ψ(·) will be sufficiently high. Thus, there can be nomore than O(nL) non case 2.2 stages, and O(L) case 2.2 stages. Also, each occurrence of case 2.2requires no more than O(n

√n + log n) quasi Newton steps (since m = O(n)), and each non case

2.2 stage requires no more than O(√n+log n) quasi Newton steps. Thus, the total number of quasi

Newton steps required by the algorithm is O(n√nL+ nL log n) = O(n

√nL).

5 Adding Multiple Cuts

In this section we discuss a version of our long step algorithm where multiple cuts are added in case2.1. The algorithm is exactly the same as before, except that we may be adding p ≤ n cuts in case2.1. For other work in the literature, pertaining to adding multiple cutting planes, see Ye [24] andRamaswamy and Mitchell [16].

Suppose that case 2.1 has occurred, and we have an exact π-center (for simplicity) z. Recall thatagain it is possible to have σ1(z) < ε, although all the other sigmas are greater than ε. As before,

32

this is the harder case, and we will assume this to be the case in our analysis here. Suppose that pcuts are to be added, and these are given by vTi x ≥ vTi z, for i = 1 to p. We will assume that thevectors vi are ortho positive and normal with respective to Q — i.e., they have the property

< vi, vj >≥ 0

and

‖vi‖2Q(z)

≡< vi, vi >= 1

where < x, y >≡ xT Q(z)−1y for any x, y ∈ <n.For a motivation of this, see Ramaswamy and Mitchell [16]. For completeness, we include a

brief description of the Selective Orthonormalization procedure by which the vi’s may be generatedfrom p arbitrary cuts. For further details, see [16]. Let the p arbitrary cuts be given by uTi x ≥ uTi z.

Selective Orthonormalization ProcedureStep 0 : Define v1 ≡ u1

‖u1‖Q. Set k = 2.

Step k : Define J(k) = i ∈ 1, 2, ..., k− 1| < uk, vi > < 0.Set gk ≡ uk − ∑

i∈J(k) < uk, vi > vi.Set vk ≡ gk

‖gk‖Q

Step Check : If k = p, STOP. Else set k = k + 1, goto step k.It is shown in [16] that the vis define valid cuts, in addition to having other nice properties. Most

importantly, a feasible direction for the affine step is now easy.Define

daff ≡ β

rQ(z)−1(

p∑i=1

vi)

where β is a constant we will soon choose, and

r ≡ ‖p∑i=1

vi‖Q ≤ p

Lemma 31 Let x denote the point after an affine step — i.e., x ≡ z + daff . Then,

x ∈ Σ(z,β√

µ(z)(1− ε))

Proof :Exactly like the proof of lemma 23.

As before, we may safely choose β = 0.5√µ(z) ≥ 0.5×10−4. We wish to bound ψ(x)−ψ(z)

above by a constant. But first, we state another lemma.It will be convenient to denote U ≡ [v1 v2 ... vp]. Then we can write

p∑i=1

vi = Ue

33

Lemma 32 Let z denote the current exact π-center. Let the p cuts defined by the vis as above beadded, and let x denote the point after an affine step. Let Q, H and µ be defined as in equations(11), (12) and (13) respectively. Let τ ≡ β√

µ(z)(1−ε) . Define

qi(x) ≡ vTi H(x)−1vi/(vTi x− vTi z)2

for i = 1 to p. Then,qi(x) ≤ (1 + τ)2p2/β2

Proof :

qi(x) =vTi H(x)−1vi

(vTi x− vTi z)2

=vTi H(x)−1vi

(βr )2 < vi, Ue >2

≤ (1 + τ)2r2

β2

vTi H(z)−1vi< vi, Ue >2

≤ (1 + τ)2r2

β2

vTi H(z)−1vi< vi, Ue >2

≤ (1 + τ)2r2

β2

vTi Q(z)−1vi< vi, Ue >2

=(1 + τ)2r2

β2

< vi, vi >

< vi, Ue >2

≤ (1 + τ)2p2

β2

where the last step follows because r ≤ p, < vi, vi >= 1 and < vi, Ue >≥< vi, vi >= 1.

Lemma 33 Under the hypothesis of lemma 32, we have

ψ(x) ≤ ψ(z) + 1.5β2 (1 + τ)2

(1− ε)(1 − τ)4

Proof :Exactly like the analysis preceding equation (16).

It is a direct consequence of lemmas 32 and 33 that

ψ(x) − ψ(z) ≤ O(p logp)

(This is easily seen from the definition of ψ(·) here).Finally, if z denotes the new π-center after the addition of multiple cuts, we can say that

ψ(z)− ψ(z) ≥ ηa

34

We do not have to prove this result — the lemmas of section 4.1.1 apply here directly since addingmultiple cuts can only strengthen our result.

Thus, it follows that the next approximate π-center may be found inO(√np log p+logn) quasi

Newton steps. This leads then to an overall complexity ofO(n1.5Lp log p+nL log n) quasi Newtonsteps.

6 Conclusions

We have used Vaidya’s algorithm as a framework to create a long step cutting plane algorithm basedon the volumetric barrier function. Since it is a long step method, our algorithm has a slightly weakercomplexity of O(n1.5L) Newton steps. Recently, many path following algorithms have been dis-cussed in the literature - some are based on the logarithmic barrier function (see [17, 8, 21]), someon the volumetric barrier function (see [22, 20, 1, 3]), and some on the hybrid volumetric-logarithmicbarrier function (see [22, 3]). In particular, the best known long step complexity for linear program-ming (without using cutting planes) is O((nm)0.5L), obtained using the hybrid potential function(see [22, 3]). (The same hybrid barrier yields the best known short step complexity ofO((nm)0.25)for an algorithm without cutting planes). However, all these methods have complexity depending onm, and thus when m is large enough (or infinite, as for arbitrary convex programs) these algorithmsare not satisfactory. This paper is the only analysis of a long step, cutting plane algorithm based onthe volumetric barrier that we are aware of.

There are two algorithms that we can compare ours with. First, Vaidya’s original algorithm,which has an O(nL) complexity. However, this backs off the cut instead of adding it through thecurrent point. It is true that a version of Vaidya’s algorithm which adds cuts right through the cur-rent volumetric center can be designed. An affine step followed by centering steps can then be takento find the next volumetric center. Such an algorithm would have a complexity of O(n1.5L), sameas ours. However, our algorithm has two advantages over this - first, we do not need to add a cut toseek progress in the objective, and secondly, we are able to guarantee cutting the duality gap by halfin such iterations. Vaidya’s algorithm (even with cuts added right through) does not do that. Anotheralgorithm we could compare our algorithm with is the long step cutting plane method based on thelogarithmic center, which we discuss in [14]. This algorithm has a complexity ofO(nL2). However,for theoretical reasons,L is required to beO(nφ), whereφ is the (maximum) number of bits requiredto store any row of the constraint matrix A. Thus,L = O(n), which implies that our algorithm hereimproves theoretically on the algorithm in [14]. However, from a computational standpoint, meth-ods based on the volumetric center have yet to be tested in the literature. It would be interesting tosee how our algorithm here compares with, say, that of [14] in terms of performance.

We also note that our algorithm is applicable to arbitrary convex programs, provided the sepa-ration problem is solvable (see [12]). We also assume that the objective function is linear, but anyconvex optimization problem can be reformulated to have a linear objective (see [9]).

References

[1] Anstreicher, K. M. “Large Step Volumetric Potential Reduction Algorithms for Linear Pro-gramming”. Department of Management Sciences, University of Iowa, Iowa City. May 5,1994.

35

[2] Anstreicher, K. M. “On Vaidya’s Volumetric Cutting Plane Method for Convex Program-ming”. Department of Management Sciences, University of Iowa, Iowa City. Sep 27, 1994.

[3] Anstreicher, K. M. “Volumetric Path Following Algorithms for Linear Programming”. De-partment of Management Sciences, University of Iowa, Iowa City. September 27, 1994.

[4] Atkinson, D. S. “Scaling and Interior Point Methods in Optimization”. Ph.D Dissertation, Co-ordinated Science Laboratory, College of Engineering, University of Illinois at Urbana Cham-paign, USA, 1992.

[5] Atkinson, D.S and Vaidya, P.M. “An Analytic Center Based Cutting Plane Algorithm for Con-vex Programming”. Technical report, Dept. of Mathematics, University of Illinois at UrbanaChampaign, June 1992.

[6] Atkinson, D.S and Vaidya, P.M. “A Scaling Technique For Finding The Weighted AnalyticCenter Of A Polytope”. Mathematical Programming, 1992 (NOV 2) Vol. 57 pp 163 - 192.

[7] Bertsimas, D. and Orlin, J. B. “A Technique for Speeding Up the Solution of the LagrangeanDual”. Mathematical Programming, vol 63, no. 1. 1994. pp23-46.

[8] Den Hertog, D. PhD Thesis. “Interior Point Approach to Linear, Quadratic and Convex Pro-gramming, Algorithms and Complexity”. Faculty of Mathematics and Informatics, TU Delft,the Netherlands, Sep 1992.

[9] Den Hertog, D., Kaliski, J., Roos, C. and Terlaky, T. “A Logarithmic Barrier Cutting PlaneMethod for Convex Programming”. Report 93-43, Delft University of Technology, 1993.

[10] Elzinga, J and Moore, T. G. “A Central Cutting Plane Algorithm for the Convex ProgrammingProblem”. Mathematical Programming, 8, 134-145, 1975.

[11] Goffin, J. L., Luo, Z. Q. and Ye, Y. “Further Complexity Analysis of a Primal-Dual ColumnGeneration Algorithm for Convex or Quasiconvex Feasibility Problems”. Faculty of Manage-ment, McGill University, Montreal, Canada. August 1993.

[12] Grotschel, M , Lovasz, L and Schrijver, A. “Geometric Algorithms and Combinatorial Opti-mization”. Springer Verlag, 1988.

[13] Horn, R. A. and Johnson, C. R. “Matrix Analysis”. Cambridge University Press, 1985.

[14] Mitchell, J. E., and Ramaswamy, S. “A Long Step Cutting Plane Algorithm for Linear andConvex Programming”. Technical Report number 37-93-387, Dept. of DSES, R.P.I, Troy, NY12180. 1993.

[15] Mitchell, J. E. and Todd, M. J. “Solving Combinatorial Optimization Problems Using Kar-markar Algorithm”. Mathematical Programming, 1992 (OCT 5) Vol. 56 pp 245 - 284.

[16] Ramaswamy, S. and Mitchell, J. E. “On Updating the Analytic Center After the Addition ofMultiple Cuts”. DSES Technical Report 37-94-423, Rensselaer Polytechnic Institute, Troy,NY 12180. October 27, 1994.

[17] Renegar, J. “A Polynomial-Time Algorithm, Based on Newton’s Method, for Linear Program-ming”. Mathematical Programming, 40, pp59-93, 1988.

36

[18] Sonnevend, Gy. “An ‘Analytic Center’ for Polyhedrons and New Classes of Global Algo-rithms for Linear (Smooth, Convex) Programming”. Lecture Notes in Control and Informa-tion Sciences, Springer Verlag, New York, 84, 866-876, 1985.

[19] Tarasov, S. P , Khachiyan, L. G. and Erlich, I. I. “The Method of Inscribed Ellipsoids”. SovietMath Dokl., 37, 1988.

[20] Vaidya, P. M. “A New Algorithm for Minimizing Convex Functions over Convex Sets”. Toappear in Math. Prog.

[21] Vaidya, P. M. “An Algorithm for Linear Programming Which RequiresO(((m+n)n2+(m+n)1.5n)L) Arithmetic Operations”. Mathematical Programming, 47, 1990. pp175-201.

[22] Vaidya, P. M. and Atkinson, D. S. “A Technique for Bounding the Number of Iterations inPath FollowingAlgorithms”, in Complexity in Numerical Optimization, P. M. Pardalos, editor,World Scientific, Singapore, 1993. pp462-489.

[23] Ye, Y. “A Potential Reduction Algorithm Allowing Column Generation”. SIAM Journal onOptimization, 2, pp7-20, 1992.

[24] Ye, Y. “Complexity Analysis of the Analytic Center Cutting Plane Method That Uses Multi-ple Cuts”. Dept. of Management Sciences, The University of Iowa, Iowa City, Iowa 52242.September 1994.

37