tutorial on ellipsoid method

000001002003004005006007008009010011012013014015016017018019020021022023024025026027028029030031032033034035036037038039040041042043044045046047048049050051052053

054055056057058059060061062063064065066067068069070071072073074075076077078079080081082083084085086087088089090091092093094095096097098099100101102103104105106107

Optimization Methods Term Paper

CS481 Optimization Methods 2012

Topic 0: Ellipsoid Method

Instructor: Prof. C. V. Jawahar Team: Sumit Sidana, Amit Rai, Mohit Mundhra, Deepti Singhal, Avinash S

1. Introduction

The Ellipsoid Method, proposed by Khachiyan [4, 3], proves that the Linear Programming (LP) can be solved in polyno-mial time. The most prominent algorithm discussed so far, the Simplex Method, is not polynomial time and has an exponential

running time in the worst case. Ellipsoid method is theoretically better than simplex method, but very slow practically and not

competitive with Simplex. Nevertheless, it is a very important theoretical tool for developing polynomial time algorithms for

a large class of convex optimization problems, which are much more general than linear programming. The ellipsoid method

is an iterative algorithm used for minimizing convex optimization. The basic idea of the ellipsoid method is to convert the

optimization problem into the feasibility problem, and solve the feasibility problem by generating a sequence of ellipsoids

whose volume uniformly decreases at every step. Before going into the details of method, let us see the challenges in other

existing solutions.

1.1. Challenges in other methods

1.1.1 Simplex method is non-polynomial time algorithm

The simplex algorithm moves from one basic feasible solution to an adjacent one, each time improving the value of the objec-tive function. However, it can take an exponentially large number of steps to terminate. To prove that the simplex algorithm is

not polynomial time, we need to come up with a class of instances which are unfortunate and exhibit an exponential sequence

of pivots for which the value of objective function is decreasing for minimization problem and increasing for maximizationproblem.

Klee and Minty demonstrated that simplex algorithm has poor worst-case performance when initialized at one corner of

Klee-Minty cube [5]. The Klee-Minty cube (named after Victor Klee and George J. Minty) is a unit cube whose corners havebeen slightly perturbed. In general, ddimensional cube can be represented as:

0 xj 1, j = 1, 2, ..., d. (1)

1

108109110111112113114115116117118119120121122123124125126127128129130131132133134135136137138139140141142143144145146147148149150151152153154155156157158159160161

162163164165166167168169170171172173174175176177178179180181182183184185186187188189190191192193194195196197198199200201202203204205206207208209210211212213214215


This d dimensional cube has 2d faces, one for each inequality, and 2d vertices, one for setting each subset of x1, x2, , xd to

one and rest to zero. The perturbed Klee-Minty cube is defined by inequalities, for some 0 < < 1/2:

1 x1 1 xj1 xj xj1, j = 2, 3, , d.

(2)

Let us take the example of 2-dimensional perturbed Klee-Minty cube, as shown in figure 1. It can be verified that the

(1, 1e)

2

(e, 1 e )2

AB

CD

x1

x2

(1, e)

(e,e )

Figure 1. Example for exponential time simplex algorithm

cost function increases strictly with each move along the path. Let the objective is to maximize x2, note that x2 increasesalong the route A B C D. So if the pivoting rule is always to move to the adjacent basic feasible solution for whichthe entering variable has the least index (Blands rule), then the simplex algorithm will require 2d 1 pivoting steps beforeterminating. With this pivoting rule the simplex algorithm has exponential worst-case time complexity.

Now let us see how it can be formalized. In order to put 2 in standard form, we add d slack and d surplus variables. The

LP problem is defined asmax xd

x1 r1 = x1 + s1 = 1

xj xj1 rj = 0, j = 2, 3, , dxj + xj1 + sj = 1, j = 2, 3, , d

xj , rj , sj 0, j = 1, , d

(3)

To prove the complexity of simplex method we use the following Lemmas for the proofs of the lemmas refer [6, 2].

Lemma 1.1 Every set of feasible bases of problem defined above will have all the xjs present in it and only one of rj or sj .

Lemma 1.2 If S and S are two subsets of {1, 2, , d} such that d S and d / S, then xSd > xS

d . Furthermore, if S = Sd,then xSd = 1 xSd .

2

216217218219220221222223224225226227228229230231232233234235236237238239240241242243244245246247248249250251252253254255256257258259260261262263264265266267268269

270271272273274275276277278279280281282283284285286287288289290291292293294295296297298299300301302303304305306307308309310311312313314315316317318319320321322323


Here xS denote the bases feasible set where rjs are non zero. The value of xj in xS will be denoted by xSj . This lemma is

valid when S is non empty.

Lemma 1.3 Let the subset of {1, 2, , d} be enumerated in such a way that xS1d xs2d ... xS2d

d . Then the inequalities are

strict, and the bases feasible sets xSj and xSj+1 are adjacent for j = 1, 2, , 2d 1.

With this we have exhibited an exponential sequence of adjacent vertices for which the values of objective function is con-stantly increasing.

Theorem 1.4 For every d > 1 there is an LP with 2d equations, 3d variables, and inter coefficient with absolute valuebounded by 4, such that simplex may take 2d 1 iterations to find the optimum solution.

Proof Take = 1/4 and multiply all equations of 3 by 4, so that all coefficients are integers. Since the objective is to max-imize xd, the exponentially long chain of adjacent bases feasible sets whose existence is established by 1.3 has decreasingcost. Thus the theorem follows.

Thus the simplex method is not polynomial time algorithms.

1.1.2 Cutting plane method

In cutting plane method, it can be difficult to compute appropriate next query point and localization polyhedron grows in

complexity as algorithm progresses. These issues can be addressed in ellipsoid method.

2. Ellipsoid Method

Let us discuss some of the definitions related to the method, before discussing the actual algorithm:

Definition An n n symmetric matrix D is called positive definite if xTDx > 0 fall nonzero vectors x Rn.

Definition A set E of vectors in Rn of the form E = E(z,D) = {x Rn|(x z)TD1(x z) 1}, where D in an n npositive definite symmetric matrix, is called an ellipsoid with center z Rn.

Definition A polyhedron P is full-dimensional if it has positive volume as shown in figure 2.

First we will cover the ground for ellipsoid method. The ellipsoid method can be used to decide whether a polyhedron

P = {x Rn|Ax b} is empty or not. And then we extend this concept to solve the LP problem (LPP). Following are theassumptions made to simplify the understanding of ellipsoid method:

The polyhedron P is bounded.

P is either empty or full-dimensional.

3

324325326327328329330331332333334335336337338339340341342343344345346347348349350351352353354355356357358359360361362363364365366367368369370371372373374375376377

378379380381382383384385386387388389390391392393394395396397398399400401402403404405406407408409410411412413414415416417418419420421422423424425426427428429430431


P

Figure 2. Bounded and full-dimensional polyhedron contains a ball and is contained in a ball

We can make our calculations in infinite precision (i.e. the square roots can be computed exactly in unit time).

The first assumption implies that there exists a ballE0 = E(x0, r2I), with volume V , that containsP . The second assumption

requires that either P is empty, or P has positive volume (i.e. V ol(P ) > v for some v > 0). Initially we assume that theellipsoid E0, v, V are a priori known.

2.1. The Algorithm

This subsection discusses the algorithm for ellipsoid method. Fist we will describe intuitively how the method works: The

method generates a sequence Et of ellipsoids with centers xt, such that P is contained in Et. If xt P , then P is nonemptyand the method terminates. If xt / P , then there exists a constraint which is violated (i.e. xt satisfies axt < b, where a isone of the rows of A, and b is the corresponding entry of b). In this case P is contained in the intersection of the ellipsoidEt and a half-space that passes through the center of ellipsoid as shown in figure 3. The new ellipsoid Et+1 is generated

which covers the half ellipsoid and whose volume is only a fraction of the volume of the previous ellipsoid Et. Repeating

this process, we either find a point in P , or we conclude that the volume of P is very small and, therefore P is empty.

Figure 3. An Iteration of Ellipsoid Method [1]

The formal algorithm for the ellipsoid method is as follows:

4

432433434435436437438439440441442443444445446447448449450451452453454455456457458459460461462463464465466467468469470471472473474475476477478479480481482483484485

486487488489490491492493494495496497498499500501502503504505506507508509510511512513514515516517518519520521522523524525526527528529530531532533534535536537538539


Algorithm 1: Ellipsoid Method:

Input:

(a) A matrix A and a vector b that define the polyhedron P = {x Rn|aix bi, i = 1, ...,m}.

(b) A number v such that either P is empty or V ol(P ) > v.

(c) A ball E0 = E(x0, r2I) with volume at most V , such that P E0.

Output:

A feasible point x P if P is nonempty, or a statement that P is empty.

Algorithm:

1. (Initialization)Let t = 2(n+ 1) log(V/v);E0 = E(x0, r2I);D0 = r2I; t = 0

2. (Main iteration)

(a) If t = t stop; P is empty.

(b) If xt P stop; P is nonempty.

(c) If xt / P find a violated constraint, that is, find an i such that aixi < bi

(d) Let Ht = {x Rn|aix aixt} . Find an ellipsoid Et+1 = E(xt+1, Dt+1 with

xt+1 = xt +1

n+ 1

DtaiaiDtai

, (4)

Dt+1 =n2

n2 1(Dt 2

n+ 1

Dtaia

iDtaiDtai

). (5)

(e) t = t+ 1

The volume of the new ellipsoid formed is actually less than the volume of the previous ellipsoid. This fact is used by the

equations 4 and 5 while actually calculating the new ellipsoid. This proof has been covered in section 3.

(Note: the fractions 1n+1 , 2n+1 and n2

n21 are called step, dilation and expansion terms respectively. They are denoted as

= 1n+1 , =2

n+1 , and =n2

n21 in rest of the document.)

5

540541542543544545546547548549550551552553554555556557558559560561562563564565566567568569570571572573574575576577578579580581582583584585586587588589590591592593

594595596597598599600601602603604605606607608609610611612613614615616617618619620621622623624625626627628629630631632633634635636637638639640641642643644645646647


2.2. Bisection Method Analogy

This section shows that in dimension n = 1, the ellipsoid method closely resembles binary search, a technique to decide

if several intervals in the real time have a non-empty intersection. In one dimension, ellipsoids can be seen as intervals.

Consider the polyhedron:

P = {x R | x 0 , x 1 , x 2 , x 3} (6)

Let E0 be the interval [0, 5], centered at x0 = 2.5. Since x0 / P the algorithm chooses the violated inequality x 2 andconstructs the ellipsoid E1 that contains the interval E0 {x|x 2.5} = [0, 2.5]. The ellipsoid E1 is in the interval [0, 2.5]with center x1 = 1.25. This ellipsoid E1 contained in P as shown in figure 4. Notice that this is same as the binary search

algorithm, that it always reduce the search interval to half.

0

0 1 2 3 4 5

P

E

E1

Figure 4. Ellipsoid Method in one dimension

2.3. Ellipsoid Method for LPP

This subsection describes how the ellipsoid method is used to solve the Linear Programming Problems. In general, the LP

problem and its dual is defined as:

P : min cTx s.t. Ax bD : max bT y s.t. AT y = c, y 0.

(7)

By strong duality of linear programming cTx = bT y. Thus the optimal solution of LP and its dual exists if the solution

satisfy following inequalities:

cTx = bT y

Ax bAT y = c

y 0.

(8)

6

648649650651652653654655656657658659660661662663664665666667668669670671672673674675676677678679680681682683684685686687688689690691692693694695696697698699700701

702703704705706707708709710711712713714715716717718719720721722723724725726727728729730731732733734735736737738739740741742743744745746747748749750751752753754755


Thus an LP problem can be formulated as an feasibility problem of equation 8, and can be solved using ellipsoid method for

deciding whether the polyhedron P = {x Rn : Ax b} is non-empty.

2.4. Variations of Ellipsoid Method

2.4.1 Sliding Objective Ellipsoid Method

Suppose we wish to the solve an optimization problem {cTx : Ax > b, x > 0}. Now we can apply the ellipsoid methodagain (with a strict inequality) to the new polyhedron given by P {x Rn : cTx < cTx0}. If this is empty, then x0 isoptimal. Otherwise, we have a new solution x1 P , say, with strict smaller objective function than cTx0. Now we canreapply the ellipsoid method to the new polyhedron.

Figure 5. Sliding Objective Ellipsoid Method

At each iteration we add a new constraint in the direction of the vector c. All the constraints cTx < cTxt are parallel to

one another. One can show that by this procedure we can reach the optimum in polynomial running time.

2.4.2 Deepest Cut Ellipsoid Method

Although ellipsoid is a very robust method which guarantees convergence with any random starting criterion, it is quite slow

in going past each of its iterations considering the reasonable amount of computation that requires to be done. One way to

speed it up is by being greedy and cutting the volume of the ellipsoid by more than half in each iteration. The most efficient

cut(deepest) would be along the that hyperplane ax 6 b of the Polyhedron P whose condition is violated by the center of thegiven ellipsoid. Such a method is called the Deepest Cut Ellipsoid Method. This is facilitated by the choosing the appropriate

7

756757758759760761762763764765766767768769770771772773774775776777778779780781782783784785786787788789790791792793794795796797798799800801802803804805806807808809

810811812813814815816817818819820821822823824825826827828829830831832833834835836837838839840841842843844845846847848849850851852853854855856857858859860861862863


step, dilation and expansion parameters.

:=1 + n

n+ 1, (9)

:=2 + 2n

(n+ 1)(1 + ), (10)

:=n2

(n2 1)(1 2) , (11)

:=a baDa

(12)

3. Mathematical Base of Ellipsoid Method

This section discusses all the mathematical arguments and proofs required to support ellipsoid method.

3.1. Starting Ellipsoid:

As a starting ellipsoid, we can use a ball centered at the vector ( 12 ,12 , ....,

12 ) and of radius

12

n (which goes through all

{0, 1}n vectors). This ball has a volume:V ol(E0) =

1

2n(n)nV ol(Bn), (13)

where Bn denotes the ball of unit radius in n-space. Therefore, the volume of Bn is:

V ol(Bn) =

n2

(n2 + 1)(14)

This can be approximated to n2 (or even 2n). Thus

log(V ol(E0)) = O(nlogn) (15)

3.2. Termination Criterion:

We will now argue that if P is non-empty, then its volume may be determined. Assume that P is non-empty, say v0 P {0, 1}n. Our assumption that P is full dimensional implies that there exists v1, v2, ..., vn P {0, 1}n = S such thatthe simplex v1, v2, ..., vn is full dimensional. The vis may not be in P . Instead, define wi for i = 1, ..., n by:

wi =

vi ifcT vi 6 x

+ 12

v0 + (vi v0) otherwise(16)

8

864865866867868869870871872873874875876877878879880881882883884885886887888889890891892893894895896897898899900901902903904905906907908909910911912913914915916917

918919920921922923924925926927928929930931932933934935936937938939940941942943944945946947948949950951952953954955956957958959960961962963964965966967968969970971


where = 12ncm .This implies that wi P as

cTwi = cT v0 + c

T (vi v0) 6 x + 12ncm

= x +1

2(17)

We have that P contains the convex set C = conv({v0, w1, w2, ..., wn}) and V ol(C) is 1n! times the volume of the paral-lelopiped spanned by wi v0 = i(vi v0) (with i {, 1} for i = 1, ..., n. This parallelopiped has the volume equal tothe product of the i (which is at least n) times the volume of a parallelopiped with integer vertices, which is atleast 1.Thus,

V ol(P ) > V ol(C) =1

n!

(1

2ncm

)n. (18)

Taking logs, we see that the number of iterations of the ellipsoid algorithm before either discovering that P is empty or that

a feasible point exists is at most log(V ol(E0)) log(V ol(P )) = O(nlogn+ nlogcm). This is indeed polynomial. We thusascertain that Ellipsoid Method does indeed converge in polynomial-time.

3.3. Proof that E would encapsulate the Half Ellipsoid E H

Theorem 3.1 Let E = E(z,D) be an ellipsoid in Rn, and let a be a nonzero vector. Consider the halfspace H = {x Rn|ax az} and let

z = z +1

n+ 1

DaaDa

D =n2

n2 1(D 2

n+ 1

DaaD

aDa

)

The matrix D is symmetric and positive definite and thus E = E(z, D) is an ellipsoid. Moreover, EH E.Proof First, consider the case where z = 0,D = I and a = e1 = (1, 0, ..., 0)T . So,E0 = {x Rn : xTx 6 1} andH0 = {x Rn : x1 > 1} as shown in figure 6. Hence,

E = E(e1

n+ 1,

n2

n2 1(I 2

n+ 1e1e

T1 )) (19)

Re-writing this, we shall have

E0 =

{x Rn :

(n+ 1

n

)2(x1 1

n+ 1

)2+n2 1n2

ni=2

x2i 6 1

}(20)

=

{x Rn : n

2 1n2

ni=1

x2i +2(n+ 1)

n2x21 +

(n+ 1

n

)2(2x1n+ 1

+1

(n+ 1)2

)6 1

}(21)

9

97297397497597697797897998098198298398498598698798898999099199299399499599699799899910001001100210031004100510061007100810091010101110121013101410151016101710181019102010211022102310241025

102610271028102910301031103210331034103510361037103810391040104110421043104410451046104710481049105010511052105310541055105610571058105910601061106210631064106510661067106810691070107110721073107410751076107710781079


Figure 6. Case where E0 = {x Rn : xTx 6 1}

Therefore,

E0 =

{x Rn : n

2 1n2

ni=1

x2i +1

n2+

2(n+ 1)

n2x1(x1 1) 6 1

}(22)

Now suppose x E0H0.Then 0 6 x1 6 1, and so x1(x1 1) 6 0. Also,

ni=1 x

21 6 1. Hence

n2 1n2

ni=1

x21 +1

n2+

2(n+ 1)

n2x1(x1 1) 6 n

2 1n2

+1

n2= 1 (23)

which verifies that x E0, proving that E0 H0 E0.Now, consider the general case and construct an affine transformation T (.) such that T (E) = E0, T (H) = H0 and

T (E) = E0. The result then follows that as affine transformation preserve set inclusion, i.e. if A B Rn and T (.) is anaffine transformation, then T (A) T (B).Given E = E(Z,D), introduce the affine transformation

T (x) = RD12 (x z) (24)

where R is a rotation matrix which rotates the unit ball so that D 12 a is aligned with the unit vector e1 = (1, 0, ..., 0)T i.e.

RTR = I (25)

RD12 a = D 12 e1 (26)

10

108010811082108310841085108610871088108910901091109210931094109510961097109810991100110111021103110411051106110711081109111011111112111311141115111611171118111911201121112211231124112511261127112811291130113111321133

113411351136113711381139114011411142114311441145114611471148114911501151115211531154115511561157115811591160116111621163116411651166116711681169117011711172117311741175117611771178117911801181118211831184118511861187


So now, T (E) = E0 since

x E (x z)TD1(x z) 6 1 (27)

(x z)D12 RTD12 (x z) 6 1 (28)

RD12

(x z) E0 (29)

T (x) E0. (30)

Similarly, T (H) = H0 since

x E aT (x z) > 0 (31)

aTD 12RTRD12 (x z) > 0 (32)

eT1 T (x) > 0 (33)

T (x) H0 (34)

Similarly, one can show T (E) = E0. Above, we proved that E0H0 E0, which is equivalent to T (E)T (H) T (E),which implies E H E.

3.4. Proof that Vol(E) < e 1(2(n+1)) Vol(E)

Theorem 3.2 Let E = E(z,D) be an ellipsoid in Rn, and let a be a nonzero vector. Consider the halfspace H = {x Rn|ax az} and let

z = z +1

n+ 1

DaaDa

D =n2

n2 1(D 2

n+ 1

DaaD

aDa

)

The matrix D is symmetric and positive definite and thusE = E(z, D) is an ellipsoid. Moreover, V ol(E) < e1/(2(n+1))V ol(E).

Proof We have thatV ol(E)

V ol(E)=

V ol(T (E))

V ol(T (E))=

V ol(E0)

V ol(E0)(35)

Now,

E0 = E

(e1

n+ 1,

n2

n2 1(I 2

n+ 1e1e

T1

))(36)

So, introduce the affine transformation

F (x) =e1

n+ 1+

(n2

n2 1(I 2

n+ 1e1e

T1

)) 12

x. (37)

11

118811891190119111921193119411951196119711981199120012011202120312041205120612071208120912101211121212131214121512161217121812191220122112221223122412251226122712281229123012311232123312341235123612371238123912401241

124212431244124512461247124812491250125112521253125412551256125712581259126012611262126312641265126612671268126912701271127212731274127512761277127812791280128112821283128412851286128712881289129012911292129312941295


It can be verified that E0 = F (E0). So

V ol(E0) =

det

(n2

n2 1(I 2

n+ 1e1eT1

))V ol(E0) (38)

Hence,

V ol(E0)

E0=

(n2

n2 1)n

2(1 2

n+ 1

) 12

(39)

=n

n+ 1

(n2

n2 1) (n1)

2

(40)

< e1

(n+1)

(e

1(n21)

) (n1)2 (41)

= e1

(2(n+1)) (42)

We have used the inequality 1 + a < ea(a 6= 0). Therefore,

V ol(E)

V ol(E)< e

1(2(n+1)) (43)

Thus, we have successfully proven that the volume of E will always be considerably lower than E and E will cover the

half ellipsoid formed by E H .

3.5. Complexity in Ellipsoid Method

This section shows that ellipsoidal method solves the linear programming feasibility problem with integer data in polyno-

mial time. Consider now a polyhedron P = {x Rn|Ax b}, where A, b have integer entries with magnitude bounded bysome U , and assume that rows of A span Rn. In a further section on proofs, we show that if the polyhedron P is bounded and

either empty or full dimensional, then the ellipsoid method correctly decides whether P is empty or not in O(nlog(V/v))

iterations. We also show in subsequent section on proofs that v and V can be chosen in terms of n and U as follows:

v = nn(nU)n2(n+1), V = (2n)n(nU)n

2

. (44)

These estimates lead to an upper bound on the number of iterations of the ellipsoid method, which is O(n4log(nU)). If

we choose an arbitrary polyhedron P , we have assumed that such a polyhedron P is bounded and is either empty or full

dimensional. But, in practice Polyhedron can be unbounded or it also may not be full dimensional. But it is possible to

modify the inputs of the ellipsoid method, if the polyhedron P is unbounded or not full dimensional.

Let A be an m n integer matrix and let b be a vector in Rm. Let U be the largest absolute value of the entries in A and b.

12

129612971298129913001301130213031304130513061307130813091310131113121313131413151316131713181319132013211322132313241325132613271328132913301331133213331334133513361337133813391340134113421343134413451346134713481349

135013511352135313541355135613571358135913601361136213631364136513661367136813691370137113721373137413751376137713781379138013811382138313841385138613871388138913901391139213931394139513961397139813991400140114021403


(a) Every extreme point of the polyhedron P = {x Rn|Ax b} satisfies

(nU)n xj (nU)n, j = 1, 2, ..., n. (45)

(b) Every extreme point of the standard form polyhedron P = {x Rn|Ax = b, x 0} satisfies

(mU)m xj (mU)m, j = 1, 2, ...,m. (46)

Let P = {x Rn|Ax b}. We assume that A and b have integer entries, which are bounded in absolute value by U .Let = 1/2n + 1((n + 1)u)(n+1) and P = {x Rn|Ax b e}, where e = (1, 1, ..., 1). Here if P is empty, then Pis empty and if P is non empty, then P is full dimensional. If the polyhedron P is arbitrary,then we first form the bounded

polyhedron PB , where PB is defined as :

PB = x P | p xj p (nU)n, j = 1, ..., n. (47)

We can then perturb PB to form a new polyhedron PB,. As already noted, P is non empty if and only if it has an extreme

point, in which case PB is non empty. PB is nonempty if and only if PB, is nonempty. We can therefore apply the

ellipsoid algorithm to PB,, and decide whether P is empty or not. It is not hard to check that the number of iterations

is O(n6log(nU)). We also need to ensure the fact that the number of arithmetic operations per iteration is polynomially

bounded in n and logU . There are two difficulties, however. First,the computation of the new ellipsoid involves taking a

square root. Although this might seem easy, taking square roots in a computer cannot be done exactly (the square root of aninteger can be an irrational number). Therefore, we need that if we only perform calculations in finite precision, the errorwe can make at each step of the computation will not lead to large inaccuracies in later stages of computation. Second, we

need to show that the numbers we generate at each step of computation have polynomial size. A potential difficulty is that as

the numbers get multiplied, we might create numbers as large as 2U , which is exponential in logU . We can overcome these

difficulties. It has been shown that if we only use O(n3logU) binary digits of precision, the numbers computed during the

algorithm have polynomially bounded size and the algorithm still correctly decides whether P is empty in O(n6log(nU))

iterations. However, we do not cover the proofs of these claims here.

13

140414051406140714081409141014111412141314141415141614171418141914201421142214231424142514261427142814291430143114321433143414351436143714381439144014411442144314441445144614471448144914501451145214531454145514561457

145814591460146114621463146414651466146714681469147014711472147314741475147614771478147914801481148214831484148514861487148814891490149114921493149414951496149714981499150015011502150315041505150615071508150915101511


4. Problems in Ellipsoid Method

4.1. Performance of the ellipsoid method in practice

The ellipsoid method solves linear programming problems in O(n6log(nU)) iterations. Since the number of arithmetic

operations per iteration is polynomial function of n and log U , this result in a polynomial number of arithmetic operations.

This running time compares favorably with the worst case running time of simplex method,which is exponential. However,the

ellipsoid method has not been practically successful, because it needs a small number of iterations on most practical linear

programming problems.

The behaviour of the ellipsoid method emphasizes the pitfalls in identifying polynomial time algorithms with efficient

algorithms. The difficulty arises because we insist on a universal polynomial bound for all instances of the problem. The

ellipsoid method achieves a polynomial bound for all instances. However, it typically exhibits slow convergence. There have

been several proposed improvements to accelerate the convergence of the ellipsoid method, such as the idea of deep cuts.

However it does not seem that these modifications can fundamentally affect the speed of convergence.

Rather than revolutionizing linear programming, the ellipsoid method has shown that linear programming is efficiently

solvable from a theoretical point of view. In this sense, the ellipsoid method can be seen as a tool for classifying the

complexity of linear programming problems. This is important, because a theoretically efficient algorithm is usually followed

by the development of practical methods.

4.2. Exponential number of constraints

For discussing the problems consider the linear programming problem of minimizing cx subject to constraints Ax b.We assume that A is m n matrix and that the entries of A and b are integer. As we know that the number of iterations inthe ellipsoid method is polynomial in n and log U , where n is the dimension of x, and U is a bound on the magnitude of the

entries of A and b. What is remarkable about this result is that the number of iterations is independent of the number m of

constraints. This suggests that we may be able to solve, in time polynomial in n and log U , problems in which the number

m of constraints is very large, e.g., exponential in n. If m is, for example, equal to 2n, we need (2n) time just to inputproblem in n would then appear to be impossible.

5. Conclusion

We conclude our discussion on Ellipsoid Method with the remark that this method is not very practical but theoretically

shows that LP is a polynomial time algorithm, which was not possible to prove before this method. This material covers the

following details about ellipsoid method:

The intuitive idea and the algorithm,

14

151215131514151515161517151815191520152115221523152415251526152715281529153015311532153315341535153615371538153915401541154215431544154515461547154815491550155115521553155415551556155715581559156015611562156315641565

156615671568156915701571157215731574157515761577157815791580158115821583158415851586158715881589159015911592159315941595159615971598159916001601160216031604160516061607160816091610161116121613161416151616161716181619


The key geometric result behind the ellipsoid method,

Mathematical proves required to support the method,

Complexity Analysis,

The ellipsoid method for the feasibility problem,

The ellipsoid method for optimization (LP) problems.

References

[1] Mit lecture notes for introduction to mathematical programming fall 2009. http://ocw.mit.edu/courses/electrical-engineering-and-4

[2] Vedio lecture of course linear programming and extensions by prof. prabha sharma, iit kanpur.http://www.nptel.iitm.ac.in/courses/111104027/39. 2

[3] B. Aspvall and R. E. Stone. Khachiyans linear programming algorithm. Journal of Algorithms, 1(1):1 13, 1980. 1[4] L. G. Khachiyan. A Polynomial Algorithm in Linear Programming. Soviet Mathematics Doklady, 20:191194, 1979. 1

[5] V. Klee and G. J. Minty. How good is the simplex algorithm? In O. Shisha, editor, Inequalities, volume III, pages 159175. AcademicPress, New York, 1972. 1

[6] C. H. Papadimitriou and K. Steiglitz. Combinatorial optimization: algorithms and complexity. Prentice-Hall, Inc., Upper SaddleRiver, NJ, USA, 1982. 2

15

tutorial on ellipsoid method

Documents