€¦ · 1 intr oduction in the moder n inter ior-point theor y, all difficult constr aints are...

95
Algorithms for Cone Programming (Part I) Levent Tunc ¸el Dept. of Combinatorics and Optimization, Faculty of Mathematics, University of Waterloo, Canada. [email protected] May 12, 2004 1

Upload: others

Post on 06-Aug-2020

0 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: €¦ · 1 INTR ODUCTION In the moder n inter ior-point theor y, all difficult constr aints are pushed into the con v e x set constr aints and/or con v e x cone constr aints. Then,

Algorithm

sfor

Cone

Program

ming

(PartI)

Leven

tTu

ncel

Dep

t.o

fC

om

bin

atorics

and

Op

timizatio

n,

Faculty

of

Math

ematics,

Un

iversityo

fW

aterloo

,

Can

ada.

[email protected]

May

12,2004

1

Page 2: €¦ · 1 INTR ODUCTION In the moder n inter ior-point theor y, all difficult constr aints are pushed into the con v e x set constr aints and/or con v e x cone constr aints. Then,

1IN

TR

OD

UC

TIO

N

1In

trod

uctio

n

This

partofthecourse

isalm

ostcompletely

focusedon

Interior-PointM

ethods

anda

biton

The

Ellipsoid

Method

2

Page 3: €¦ · 1 INTR ODUCTION In the moder n inter ior-point theor y, all difficult constr aints are pushed into the con v e x set constr aints and/or con v e x cone constr aints. Then,

1IN

TR

OD

UC

TIO

N

Inthe

modern

interior-pointtheory,alldifficultconstraints

arepushed

into

theconvex

setconstraintsand/or

convexcone

constraints.T

hen,eachof

theseconvex

inclusionconstraints

aretreated

viaa

strictlyconvex

barrier

functionw

ithvery

specialproperties.

We

willpresentm

ostofourresults

inthe

fullgeneralityofan

arbitrary

convexset�

in���

oran

arbitraryconvex

cone�

in� �

.H

owever,

specialattentionw

illbepaid

toS

emidefinite

Program

ming

(SD

P).

Recall,

����

denotesthe

convexcone

of �

symm

etric,positive

semidefinite

matrices

overthe

reals.

����

isthe

interiorof� �

;i.e.,theconvex

coneof �

symm

etric,

positivedefinite

matrices

overthe

reals.

3

Page 4: €¦ · 1 INTR ODUCTION In the moder n inter ior-point theor y, all difficult constr aints are pushed into the con v e x set constr aints and/or con v e x cone constr aints. Then,

1IN

TR

OD

UC

TIO

N

Second-order

cone:

��� ����

���

� � � �

� � � �

����

SD

Pstands

forsem

idefiniteprogram

ming

where �

ism

adeup

fromdirect

sums

ofvarious� ��

(possiblyunder

some

linearisom

orphisms).

SO

CP

standsfor

second-ordercone

programm

ingw

here �

ism

adeup

fromdirectsum

sofvarious

��� �

(possiblyunder

some

linear

isomorphism

s).

4

Page 5: €¦ · 1 INTR ODUCTION In the moder n inter ior-point theor y, all difficult constr aints are pushed into the con v e x set constr aints and/or con v e x cone constr aints. Then,

1IN

TR

OD

UC

TIO

N

We

considerthe

semidefinite

programm

ing(S

DP

)problem

sin

the

following

primal� �

anddual���

forms.

� ��

inf

� ����

� ��

��

� �

����

sup����

��� ������

��

�� �

where

isa

linearoperator

from

� �

to ���

,sothat�

� � �

and �

denotesthe

adjointof

.

5

Page 6: €¦ · 1 INTR ODUCTION In the moder n inter ior-point theor y, all difficult constr aints are pushed into the con v e x set constr aints and/or con v e x cone constr aints. Then,

1IN

TR

OD

UC

TIO

N

Withoutloss

ofgenerality,we

assume

that

issurjective.

Ifnot,we

can

considerthe

representation� �����

����������� ������� ,w

here

�� � � �

forevery�

.

beingsurjective

isequivalentto

��� ��� ���� ��

beinglinearly

independent.T

helatter

canbe

assumed

withoutloss

of

generality,sinceifthey

arelinearly

dependent,theneither

thesystem

� ����� ����������� ���� ��

hasno

solution,orthere

aresom

e

redundantequationsw

hichcan

beelim

inated.In

thefirstcase,� �

is

infeasible.In

thesecond

case,allredundantequations,andcorresponding

��� ��

canbe

eliminated,to

arriveatan

equivalentproblemsatisfying

the

assumption.

6

Page 7: €¦ · 1 INTR ODUCTION In the moder n inter ior-point theor y, all difficult constr aints are pushed into the con v e x set constr aints and/or con v e x cone constr aints. Then,

1IN

TR

OD

UC

TIO

N

Under

thisassum

ption,forany

solution,� � � �� ,oftheequation

�� ��� � �� ���

,the �

partofthesolution

uniquelyidentifies

the

corresponding�.

Som

etimes,in

interior-pointalgorithms,itis

convenient

torefer

onlyto �

when

onem

entionsa

feasiblesolution

of���� .

The

abovesetting

oftheprim

al-dualSD

Ppair

canbe

embedded

inthe

following

more

generalsettingofconic

convexoptim

izationproblem

s:

� � ��

inf����

� �

��

���

where

isa

surjectivelinear

map

and�

isa

pointed,closed,convex

conew

ithnon-em

ptyinterior.

7

Page 8: €¦ · 1 INTR ODUCTION In the moder n inter ior-point theor y, all difficult constr aints are pushed into the con v e x set constr aints and/or con v e x cone constr aints. Then,

1IN

TR

OD

UC

TIO

N

We

definethe

dualof� � ��

as

� � ��

sup

� ��� �

�� ���

��

��

����

where �

isthe

dualofcone �w

ithrespectto���� � ,i.e.

� �

��� ��� �� ������

We

willrefer

tothis

settingas

theconic

convexoptim

izationsetting.

8

Page 9: €¦ · 1 INTR ODUCTION In the moder n inter ior-point theor y, all difficult constr aints are pushed into the con v e x set constr aints and/or con v e x cone constr aints. Then,

1IN

TR

OD

UC

TIO

N

SD

Pproblem

fitsinto

thisgeneralsetting

byletting

�� �� �� �

(thatis,�� �� ������

),

��� �� �

,

�� � �

� �� �� � .

9

Page 10: €¦ · 1 INTR ODUCTION In the moder n inter ior-point theor y, all difficult constr aints are pushed into the con v e x set constr aints and/or con v e x cone constr aints. Then,

1IN

TR

OD

UC

TIO

N

Under

thesedefinitions

we

have���

.I.e.,the

coneofsym

metric

positivesem

idefinitem

atricesis

self-dualunder

thetrace

innerproduct.

In

additionto

beingself-dual,the

cone� �

enjoysanother

symm

etry

property,inthatitis

homogeneous.

Thatis,the

setofnonsingularlinear

transformations

keeping� �the

same

(theautom

orphismgroup

of� �

)is

richenough

tocontain

lineartransform

ationsw

hichm

apany

fixedinterior

pointtoany

otherfixed

interiorpointof� �

.C

onvexcones

with

both

properties,i.e.hom

ogeneousself-dualcones,are

alsocalled

symm

etric.

10

Page 11: €¦ · 1 INTR ODUCTION In the moder n inter ior-point theor y, all difficult constr aints are pushed into the con v e x set constr aints and/or con v e x cone constr aints. Then,

1IN

TR

OD

UC

TIO

N

��

isself-dualifthere

existsan

inner-productunderw

hich� ���

.

��

ishom

ogeneousifA

ut� ��

actstransitively

onint� �

� .

��

issym

metric

if �is

homogeneous

andself-dual.

So,w

ehave

theunderlying

optimization

problems:

Sym

metric

Cone

Program

ming

(Sym

CP

)

Hom

ogeneousC

oneP

rogramm

ing(H

omC

P)

11

Page 12: €¦ · 1 INTR ODUCTION In the moder n inter ior-point theor y, all difficult constr aints are pushed into the con v e x set constr aints and/or con v e x cone constr aints. Then,

1IN

TR

OD

UC

TIO

N

�A

homogeneous

polynomial�

�� ����

ishyperbolic

inthe

direction��� �

,iftheunivariate

polynomial(in�

��

)

�� ����

hasonly

realrootsfor

every�� �

.

Aconvex

cone�

ishyperbolic

ifitis

�� ���� �

��� � � ������

fora

polynomial�

which

ishyperbolic

inthe

direction��� �

.

Hom

ogeneouscones

make

upa

propersubsetofhyperbolic

cones.

Hyperbolic

Cone

Program

ming

(HypC

P)

12

Page 13: €¦ · 1 INTR ODUCTION In the moder n inter ior-point theor y, all difficult constr aints are pushed into the con v e x set constr aints and/or con v e x cone constr aints. Then,

1IN

TR

OD

UC

TIO

N

Strictly

speakingw

ehave,

������ �

�� ��

��� �� �

��

��� �

��� � � �

�� ��

How

ever,insom

esense,

������ �

�� ���

�� �� ��

���� �

��� � � �

�� ��

Yetinan

anothersense,

���

��� �

�� ���

�� �� ��

���� �

��� � � �

�� ��

13

Page 14: €¦ · 1 INTR ODUCTION In the moder n inter ior-point theor y, all difficult constr aints are pushed into the con v e x set constr aints and/or con v e x cone constr aints. Then,

1IN

TR

OD

UC

TIO

N

Recallthe

weak

dualityrelation:

Pro

po

sition

1.1Let

��

befeasible

in� �� ,and�

�� ����

befeasible

in� �� .

Then

� ���

� ���

��

���

�� �

Sim

ilarlyfor

theconic

convexoptim

izationsetting...

Therefore,ifw

estart

with �

� �

and �� �

bothfeasible

intheir

respectiveproblem

s,then

decreasing� �� �

willgetus

closerto

optimality!14

Page 15: €¦ · 1 INTR ODUCTION In the moder n inter ior-point theor y, all difficult constr aints are pushed into the con v e x set constr aints and/or con v e x cone constr aints. Then,

1IN

TR

OD

UC

TIO

N

Since

thelinear

operator

issurjective,w

ecan

always

find

��� � �

suchthat

��� �� �

For

thedual,because

oftheform

we

chose,we

canalw

aysfind

���� �

,

��� � �

suchthat

��

�������

���

Denoting

���� �� � ��� �� � � �

we

claimthat� �

and����

areequivalentto

thefollow

ingpair.

15

Page 16: €¦ · 1 INTR ODUCTION In the moder n inter ior-point theor y, all difficult constr aints are pushed into the con v e x set constr aints and/or con v e x cone constr aints. Then,

1IN

TR

OD

UC

TIO

N

��

��

inf

������

��

���

�� �

� �

��

��

inf�

��� ��

��

���

��� �

�� �

here,

��

denotesthe

orthogonalcomplem

entof�

.16

Page 17: €¦ · 1 INTR ODUCTION In the moder n inter ior-point theor y, all difficult constr aints are pushed into the con v e x set constr aints and/or con v e x cone constr aints. Then,

1IN

TR

OD

UC

TIO

N

We

have

� ��

��

inf

����

���

���

��

and

� ��

��

inf� ��

��

�����

inthe

generalconicconvex

optimization

setting.To

establishthe

equivalence,firstnotethatthe

feasibleregions

arepreserved

(in� ��

��

we

onlyrefer

to

� ).

17

Page 18: €¦ · 1 INTR ODUCTION In the moder n inter ior-point theor y, all difficult constr aints are pushed into the con v e x set constr aints and/or con v e x cone constr aints. Then,

1IN

TR

OD

UC

TIO

N

Recallthe

proofofthew

eakduality

relation(P

roposition1.1).

For

every�� �

satisfying� � �� ,and

forevery� � � ��

satisfying

��� ����� ��

we

have

� ��

� ��� ��

� � � �

Ifwe

fix��� ����

suchthat

��

������� �

thenfor

all�� �

satisfying� � ��

we

have

� ��

�� �

�����������

where

theconstantis� ��

�� �

.T

herefore,minim

izing

� ��

subjecttoany

setofconstraints,containingthe

constraint� � ��

isequivalentto

minim

izing

� ���

subjecttothe

same

setofconstraints.

18

Page 19: €¦ · 1 INTR ODUCTION In the moder n inter ior-point theor y, all difficult constr aints are pushed into the con v e x set constr aints and/or con v e x cone constr aints. Then,

1IN

TR

OD

UC

TIO

N

Sim

ilarly,we

canestablish

theequivalence

ofthedualproblem

s.W

efix

suchthat

��� �

�then

forall� � � ��

satisfying �� �� �� ��

,we

have�� ��� ��

��� ���

�������

where

theconstantis

�����

� .T

herefore,maxim

izing� ��� �

subjectto

anysetofconstraints

containingthe

constraint �� �� �� ��

is

equivalenttom

inimizing�

�� �

subjecttothe

same

setofconstraints.

19

Page 20: €¦ · 1 INTR ODUCTION In the moder n inter ior-point theor y, all difficult constr aints are pushed into the con v e x set constr aints and/or con v e x cone constr aints. Then,

2E

LLIPS

OID

ME

TH

OD

2E

llipso

idM

etho

d�

�� �

isan

ellipsoidifthere

exist�� � �

(determining

thecenter)

and

�� � �

(determining

thesize

andthe

shape)such

that

������ �� �� � �

�� ���

��� ����

��

��� ��

We

canalternatively

expressthe

ellipsoidas

theim

ageofthe

unitballin

� �

(denotedby��

� � �� )

underan

affinem

appingas

follows:

�� �� �� �� �� ���� � �� ���

The

volume

oftheellipsoid

isproportionalto

thesquare-rootofthe

determinantofthe

positivedefinite

matrix

determining

itsshape:

vol� �� �� ��� �

���� �� vol� ��� � ��� �

20

Page 21: €¦ · 1 INTR ODUCTION In the moder n inter ior-point theor y, all difficult constr aints are pushed into the con v e x set constr aints and/or con v e x cone constr aints. Then,

2E

LLIPS

OID

ME

TH

OD

The

volume

ofthe

-dimensionalunitballis

vol� ��� � ��� �

� �� �

�� �� ��� �

where

�� ����� �� ���� ��

�� ,for .

We

firststudythe

ellipsoidm

ethodas

analgorithm

which

computes

apoint

inan

implicitly

describedconvex

set.In

thisbasic

setting,itiseasy

tosee

thattheellipsoid

method

isa

beautifulandtheoretically

verypow

erful

generalizationofthe

bisectionm

ethodfrom

�to� �

,foran

arbitrary

.

21

Page 22: €¦ · 1 INTR ODUCTION In the moder n inter ior-point theor y, all difficult constr aints are pushed into the con v e x set constr aints and/or con v e x cone constr aints. Then,

2E

LLIPS

OID

ME

TH

OD

2.1Ingredients:

Separation

Oracles,Inscribed

andC

ircumscribed

Ellipsoids

2.1In

gred

ients:

Sep

aration

Oracles,In

scribed

and

Circu

mscrib

edE

llipso

ids

Toappreciate

thefulltheoreticalpow

erofthe

Ellipsoid

Method,w

ew

ill

move

away

fromthe

explicitdescription

oftheconvex

optimization

problem

athand.Instead,w

ew

illassume

thatwe

aregiven

accessto

aseparation

oracle.Let�

�� �

bethe

convexsetw

eare

interestedin

optimizing

over

orless

ambitiously

justfindinga

pointinsideit(the

set�

).W

edefine

� -relaxationof�

asfollow

s:

relax� ��

�����

�� ���

�� ��

forsom

e

���

22

Page 23: €¦ · 1 INTR ODUCTION In the moder n inter ior-point theor y, all difficult constr aints are pushed into the con v e x set constr aints and/or con v e x cone constr aints. Then,

2E

LLIPS

OID

ME

TH

OD

2.1Ingredients:

Separation

Oracles,Inscribed

andC

ircumscribed

Ellipsoids

Aw

eakseparation

oraclefor�

takesas

input

��

� �

,

��

.It

eitheroutputs

��

relax� ��

�� ”or�

�� �

suchthat� �

� ���

and

� �

��

� �

� �

�� ��

relax� ��

�� �

Th

eorem

2.1F

orevery

compact,convex

setin���

with

nonempty

interior,

thereexists

aunique

minim

alvolume

ellipsoidcontaining

thatset.

Moreover,shrinking

thatellipsoid(around

itscenter)

bya

factorofatm

ost

givesan

ellipsoidcontained

inthe

convexset.

The

uniqueellipsoid

describedin

theabove

theoremis

usuallycalled

the

Lowner-John

ellipsoid.

The

factor

inthe

abovetheorem

isthe

best

possible.(T

he

-dimensionalsim

plexproves

theclaim

forevery

.)

23

Page 24: €¦ · 1 INTR ODUCTION In the moder n inter ior-point theor y, all difficult constr aints are pushed into the con v e x set constr aints and/or con v e x cone constr aints. Then,

2E

LLIPS

OID

ME

TH

OD

2.1Ingredients:

Separation

Oracles,Inscribed

andC

ircumscribed

Ellipsoids

Let

����

�� ���

��� �� �

��

��� ��

and

�����

� ���� ��� ��

� �

forsom

e�

� � ��

� � .W

ew

illassume

� �

.W

ew

ouldlike

to

constructthesm

allestvolume

ellipsoid �

containingthe

half-ellipsoid

��

.

Let�� � �

and

�� � �

denotethe

centerand

thepositive

definite

matrix

determining�

.T

hen

���

� ����

� ��

��

� �

� �� �

��� �

� ���� � ��

��� ��

�24

Page 25: €¦ · 1 INTR ODUCTION In the moder n inter ior-point theor y, all difficult constr aints are pushed into the con v e x set constr aints and/or con v e x cone constr aints. Then,

2E

LLIPS

OID

ME

TH

OD

2.1Ingredients:

Separation

Oracles,Inscribed

andC

ircumscribed

Ellipsoids

We

canexplicitly

compute

thevolum

eof�

interm

softhe

volume

of�

,

since�

isa

rank-1update

of�

.W

henw

etake

theim

ageof���

under

them

apping� �

�� �

,ourellipsoid�

becomes

theunitball.

Under

this

mapping

ourupdate

formula

becomes

��

� �� �

��� �

� ����

�� �

��

��

�� �

� �

where

��

� �� �� �

.T

heeigenvalues

of

� ��

��� �

��������

��

�� �

are � ��

� �

and�

(with

multiplicity� �

��� ).

Therefore,

���� �� �

����

� �

��

��� ��

� �� �

��� ��

� �� ���� �

25

Page 26: €¦ · 1 INTR ODUCTION In the moder n inter ior-point theor y, all difficult constr aints are pushed into the con v e x set constr aints and/or con v e x cone constr aints. Then,

2E

LLIPS

OID

ME

TH

OD

2.1Ingredients:

Separation

Oracles,Inscribed

andC

ircumscribed

Ellipsoids

Hence

thevolum

eof �

canbe

relatedto

thevolum

eof �

asfollow

s:

vol� �� �

�� �

��� ��

����

vol� �� �

Th

eorem

2.2W

ehave

��

��

and��

vol� ��

vol� ���

���

� �26

Page 27: €¦ · 1 INTR ODUCTION In the moder n inter ior-point theor y, all difficult constr aints are pushed into the con v e x set constr aints and/or con v e x cone constr aints. Then,

2E

LLIPS

OID

ME

TH

OD

2.2C

omplexity

Analysis

forthe

Ellipsoid

Method

2.2C

om

plexity

An

alysisfo

rth

eE

llipso

idM

etho

d

Suppose

vol� �� �

andw

ew

ant ��

suchthatvol� �

�� �� .

Then

by

thelasttheorem

,

��

vol� ���

vol� ���

��� �

and�

� ���

� �� ��

iterationssuffice.

Sim

ilarly,if�

isthe

radiusofthe

initialball(whose

image

underthe

mapping

��� �

�� �

containsthe

set)and

we

wantthe

stoppingcriterion

tobe

thattheradius

ofthecurrent

ball(whose

image

underthe

mapping

��� �

�� �

contains�

)is

at

most� ,then

using

vol� ���

vol� ��

��

��

��

�27

Page 28: €¦ · 1 INTR ODUCTION In the moder n inter ior-point theor y, all difficult constr aints are pushed into the con v e x set constr aints and/or con v e x cone constr aints. Then,

2E

LLIPS

OID

ME

TH

OD

2.2C

omplexity

Analysis

forthe

Ellipsoid

Method

we

findthat�

� � ���

� �� ��

iterationssuffice.

Th

eorem

2.3Let�

�� �

bea

convexsetsuch

that����� � �� .

Suppose

we

arealso

given

��

.Ifw

ehave

accessto

aw

eak

separationoracle

for�,then

inpolynom

ialtime

(polynomialin

� ����

� �� ��� ),w

ecan

compute

��

relax� �� ��

orprove

that

vol� �� �� .

These

resultscan

beextended

tooptim

izinga

convexfunction

over�

.W

e

needone

more

ingredienttodealw

iththe

objectivefunction.

Let

�� ����

bea

convexfunction.

28

Page 29: €¦ · 1 INTR ODUCTION In the moder n inter ior-point theor y, all difficult constr aints are pushed into the con v e x set constr aints and/or con v e x cone constr aints. Then,

2E

LLIPS

OID

ME

TH

OD

2.2C

omplexity

Analysis

forthe

Ellipsoid

Method

Defi

nitio

n2.1

Asubgradientoracle

for

takesas

input

��� �

and

returnsin

polynomialtim

e(polynom

ialin

andfor

aproper

definition,

size��� )

����

and��� �

suchthat

�� �

������

� ��

�� � �� � ��

Suppose

we

areinterested

insolving

theconvex

optimization

problem

��

��� ����

� �w

here

�� ����

isa

convexfunction.

29

Page 30: €¦ · 1 INTR ODUCTION In the moder n inter ior-point theor y, all difficult constr aints are pushed into the con v e x set constr aints and/or con v e x cone constr aints. Then,

2E

LLIPS

OID

ME

TH

OD

2.2C

omplexity

Analysis

forthe

Ellipsoid

Method

Th

eorem

2.4Let�

�� �

bea

convexsetand

�� �� �

begiven

suchthat

���

�� ����

���� � �� �

forsom

e

��� �

(here,

isn

ot

given).Let�

��

bealso

given.S

upposethata

subgradientoraclefor

�and

aw

eakseparation

oraclefor�

areavailable.

Then

after

� ��

��

���

��

�� �

��

iterations,theellipsoid

method

returnsa

feasiblesolution

���

suchthat

���� �

��

��� ����

� �� �

Inthe

above,� ���

����

��� �

��� ��

��

��� �

��� �� .

30

Page 31: €¦ · 1 INTR ODUCTION In the moder n inter ior-point theor y, all difficult constr aints are pushed into the con v e x set constr aints and/or con v e x cone constr aints. Then,

2E

LLIPS

OID

ME

TH

OD

2.3B

ibliographicalNotes

2.3B

iblio

grap

hicalN

otes

Until1979

veryfew

mathem

aticiansin

theW

estknewaboutthe

ellipsoid

method.

Khachiyan,in

1979proved

thattheE

llipsoidM

ethodcan

be

adaptedto

solvelinear

programm

ingproblem

sin

polynomialtim

e(hence

settlinga

longoutstanding

problem).

This

announcementcaused

an

unprecedentedreaction

fromthe

media.

The

Ellipsoid

Method

was

originallyproposed

byIudin

andN

emirovskiin

1976also

some

relatedw

orkis

dueto

Shor

in1977.

This

originalmethod

was

designedto

dealwith

essentiallyany

convexoptim

izationproblem

thatcanbe

posedin

afinite

dimensionalspace

bythe

potentialusageof

oracles(an

importantpointw

hichshould

beem

phasizedis

thatthe

functionsinvolved

indefining

thefeasible

solutionset,the

objective

31

Page 32: €¦ · 1 INTR ODUCTION In the moder n inter ior-point theor y, all difficult constr aints are pushed into the con v e x set constr aints and/or con v e x cone constr aints. Then,

2E

LLIPS

OID

ME

TH

OD

2.3B

ibliographicalNotes

functionneed

no

tbe

differentiable).F

ora

niceexposure

toE

llipsoid

Method

forlinear

programm

ingproblem

s,seethe

surveypaper

byB

land,

Goldfarb

andTodd,O

perationsR

esearch29

(1981)1039–1091.

Shortly

afterK

hachiyan’sresult,itw

asestablished

thatthism

ethodis

a

verypow

erfultoolindeterm

iningthe

computationalcom

plexitystatus

(hencethe

degreeofdifficulty)

ofvariouscom

binatorialoptimization

problems.

Agood

referenceis

Geom

etricalgorithm

sand

combinatorial

optimization

byM

.Grotschel,L.Lovasz

andA

.Schrijver.

32

Page 33: €¦ · 1 INTR ODUCTION In the moder n inter ior-point theor y, all difficult constr aints are pushed into the con v e x set constr aints and/or con v e x cone constr aints. Then,

2E

LLIPS

OID

ME

TH

OD

2.3B

ibliographicalNotes

Inthe

late1980’s,E

llipsoidM

ethodw

asapplied

tosom

eproblem

sin

System

andC

ontrolTheory

(seethe

bookLinear

Matrix

Inequalitiesin

System

andC

ontrolTheory

byB

oydetal.).

These

problems

were

small

butrelativelydifficultconvex

optimization

problems.

Since

late1980’s

and

early1990’s

interior-pointmethods

consistentlytook

overthe

solution

process.T

heseare

them

ethodsw

ediscuss

andanalyze

next.

33

Page 34: €¦ · 1 INTR ODUCTION In the moder n inter ior-point theor y, all difficult constr aints are pushed into the con v e x set constr aints and/or con v e x cone constr aints. Then,

3C

EN

TR

AL

PATH

3C

entralP

ath

One

ofthem

ostimportantconcepts

ininterior-pointm

ethodsis

thecentral

path.W

earrive

atthisconceptvia

anothercentalconcept:

thebarrier

forthe

difficultconstraints.

Let

�� ����

bea

logarithmically

homogeneous

self-concordant

barrierfor�

.

34

Page 35: €¦ · 1 INTR ODUCTION In the moder n inter ior-point theor y, all difficult constr aints are pushed into the con v e x set constr aints and/or con v e x cone constr aints. Then,

3C

EN

TR

AL

PATH

Defi

nitio

n3.1

(LHS

CB

)Let

�int� �� ��

bea���

-smooth

convex

functionsuch

that

isa

barrierfor �

(i.e.

�� � ��

as�

int� ��

approaches� �)

andthere

exists� �

suchthatfor

each� ,

�� ��

��� �

���

� �� �

and

� ��� �� �� �� �����

� ��

�� �� �� ��� �� �

forall

int� ��

andfor

all���

.T

hen�

iscalled

a�

-LHS

Cbarrier

for�

.

35

Page 36: €¦ · 1 INTR ODUCTION In the moder n inter ior-point theor y, all difficult constr aints are pushed into the con v e x set constr aints and/or con v e x cone constr aints. Then,

3C

EN

TR

AL

PATH

��� ��

� ��

��

�� int� �

� � � �� ��� ��

Legendre-FenchelC

onjugate

��

alsohas

theabove

mentioned

propertiesfor�

forthe

same

barrier

parameter�

.

isa

veryim

portantparameter

ofthesebarriers.

Currently,one

ofthe

bestiterationbounds

forinterior-pointm

ethodsfor

conicconvex

optimization

is

���

��

tocom

putean

� -optimalsolution.

36

Page 37: €¦ · 1 INTR ODUCTION In the moder n inter ior-point theor y, all difficult constr aints are pushed into the con v e x set constr aints and/or con v e x cone constr aints. Then,

3C

EN

TR

AL

PATH

Let� .

Consider

� � �� �

inf

� �� �

��� �

� �

��

��� �

and

� � �� �

inf

�� ��� ��

���� ��

��� ���

���

� �

��� �

37

Page 38: €¦ · 1 INTR ODUCTION In the moder n inter ior-point theor y, all difficult constr aints are pushed into the con v e x set constr aints and/or con v e x cone constr aints. Then,

3C

EN

TR

AL

PATH

Itisw

ell-known

that

Th

eorem

3.1S

uppose� � �� �� ��

int� ��

int� ��

feasiblein

� � ��

and� � ��

exists.T

hen� � �� �

and� � �� �

havea

unique

optimalsolution

pair� �� ,� �� �� � �� ��� ,foreach

� .

Defi

nitio

n3.2

� � � �� ��� �� � �� ��� �� �

iscalled

theprim

al-dual

centralpathfor

thepair� � �� �� � �

� .

Som

etimes

we

referonly

to� � �� � �� ��� .

38

Page 39: €¦ · 1 INTR ODUCTION In the moder n inter ior-point theor y, all difficult constr aints are pushed into the con v e x set constr aints and/or con v e x cone constr aints. Then,

3C

EN

TR

AL

PATH

3.1C

entralPath

forS

DP

3.1C

entralP

athfo

rS

DP

Let’sfocus

onS

DP

first.H

ere,�� ��

����

��

� ���� ��� �

��� �� �

��

� ���� ���

��� ��

�39

Page 40: €¦ · 1 INTR ODUCTION In the moder n inter ior-point theor y, all difficult constr aints are pushed into the con v e x set constr aints and/or con v e x cone constr aints. Then,

3C

EN

TR

AL

PATH

3.1C

entralPath

forS

DP

The

centralpathis

equivalentlycharacterized

asthe

uniquesolution

(via

necessaryand

sufficientconditionsfor

optimality)

oftheconvex

optimization

problem

� �� � :

� �� ����

� �� ��

�� �����

�� �

Let’sdo

thesubstitutions�

� ���

and ����

� � ��

.W

eobtain

the

system

� ��

�� ��

�� ������

��

��

� � ��

40

Page 41: €¦ · 1 INTR ODUCTION In the moder n inter ior-point theor y, all difficult constr aints are pushed into the con v e x set constr aints and/or con v e x cone constr aints. Then,

3C

EN

TR

AL

PATH

3.1C

entralPath

forS

DP

For

each

� ,the

uniquesolution

oftheabove

system� �� �� ��� �� � �� ���

definesthe

primal-dualcentralpath.

Path-F

ollowing

algorithms

“closely”or

“loosely”follow

thispath

tothe

setof

optimalsolutions.

(Note

thatunderthe

assumption

oftheexistence

of

strictlyfeasible

pointsfor

bothprim

alandthe

dual,bothproblem

sdo

have

optimalsolutions

andthere

isno

dualitygap.)

Potential-R

eductionalgorithm

sreduce

theproblem

tothatofm

inimizing

(ordriving

to

��

)a

combination

oftheobjective

function:

�����

�� ��

�� �� �

anda

barrier(e.g.,

��

� ���� ���

ora

measure

ofcentrality.

41

Page 42: €¦ · 1 INTR ODUCTION In the moder n inter ior-point theor y, all difficult constr aints are pushed into the con v e x set constr aints and/or con v e x cone constr aints. Then,

3C

EN

TR

AL

PATH

3.2N

eighbourhoodsofthe

CentralP

ath

3.2N

eigh

bo

urh

oo

ds

of

the

Cen

tralPath

Again,w

efirstfocus

onS

DP.To

followthe

centralpath,orto

understand

thepotential-reduction

algorithms

ina

unifyingw

ay,itisw

orthstudying

the

neighbourhoodsofthe

centralpath.In

theprim

al-dualsetting,thisis

quite

elegantandeffective.

Given

strictlyfeasible

points�and�

,define�� �� �� �

Let

denotethe

setofallstrictlyfeasible

solutionpairs� �

� ��

( �

satisfiesallthe

primalconstraints

andis

positivedefinite,�

satisfiesallthe

dualconstraintsand

ispositive

definite).T

henw

ecan

expressm

any

neighbourhoods.

42

Page 43: €¦ · 1 INTR ODUCTION In the moder n inter ior-point theor y, all difficult constr aints are pushed into the con v e x set constr aints and/or con v e x cone constr aints. Then,

3C

EN

TR

AL

PATH

3.3N

eighbourhoodsB

asedon

theA

lgebraicD

escriptionofthe

CentralP

ath

3.3N

eigh

bo

urh

oo

ds

Based

on

the

Alg

ebraic

Descrip

tion

of

the

Cen

tralPath

Let��� � ��

bean

absoluteconstant.

We

define

(i)so-called

wide

neighborhoods

��� ��

� �

� �� ���

������� � �� ���

�� �

� � �

�� ��

(ii)infinity-norm

neighborhoods

�� ��

���

� �� ���

��

��� �� � �� ���

�� �

��

�������

��

(ii) �orequivalently,

�� ��� �

� �� ���

��

����� ��� ���

�� �

� ������ ��

43

Page 44: €¦ · 1 INTR ODUCTION In the moder n inter ior-point theor y, all difficult constr aints are pushed into the con v e x set constr aints and/or con v e x cone constr aints. Then,

3C

EN

TR

AL

PATH

3.3N

eighbourhoodsB

asedon

theA

lgebraicD

escriptionofthe

CentralP

ath

(iii)so-called

tight(ornarrow

)neighborhoods

�� ��

���

� �� ���

��

����� ��� ���

�� �

� ������ �

Note

that(asis

well-know

n),

CentralP

ath

�� ��

�� ��

��� ��

��

�44

Page 45: €¦ · 1 INTR ODUCTION In the moder n inter ior-point theor y, all difficult constr aints are pushed into the con v e x set constr aints and/or con v e x cone constr aints. Then,

3C

EN

TR

AL

PATH

3.4N

eighbourhoodsB

asedon

theA

nalyticD

escriptionsofthe

CentralP

ath

3.4N

eigh

bo

urh

oo

ds

Based

on

the

An

alyticD

escriptio

ns

of

the

Cen

tralPath

We

definea

measure

ofcentralitybased

onthe

barriervalues:

�� �� ��� �

��

� �� �

��

� ���� ���

��

� ���� ��� �

Th

eorem

3.2F

orevery� �

� ��� � �

� �

,�� �� �� �

Moreover,the

equalityholds

aboveiff �

�� � �

forsom

e

� .

This

theoremgeneralizes

theA

rithmetic-G

eometric

Mean

Inequalityand

thecorresponding

characterizationfor

equality.

45

Page 46: €¦ · 1 INTR ODUCTION In the moder n inter ior-point theor y, all difficult constr aints are pushed into the con v e x set constr aints and/or con v e x cone constr aints. Then,

3C

EN

TR

AL

PATH

3.4N

eighbourhoodsB

asedon

theA

nalyticD

escriptionsofthe

CentralP

ath

We

canalso

definea

proximity

measure

basedon

thegradients

ofthe

barrier�

��

� ���� ��� ,

��

� ���� ��� :

Let

������ � �

�� � �

We

have

Th

eorem

3.3F

orevery�

,��

,�

�� ��

Equality

holdsabove

iff ��

� � ��

.

The

abovetheorem

generalizesthe

Arithm

etic-Harm

onicM

eanInequality

andthe

correspondingcharacterization

forequality.46

Page 47: €¦ · 1 INTR ODUCTION In the moder n inter ior-point theor y, all difficult constr aints are pushed into the con v e x set constr aints and/or con v e x cone constr aints. Then,

3C

EN

TR

AL

PATH

3.4N

eighbourhoodsB

asedon

theA

nalyticD

escriptionsofthe

CentralP

ath

Inthe

general,conicconvex

optimization

setting,thecentralpath

equation

� ��

��

�� �

replaces��

�� � �

�.

The

firstproximity

measure

isgeneralized

to:

�� � ��� ��

��

� � ��

��� � �

��� �� �� �

The

nexttheoremshow

sthatfor

everypair

ofinteriorsolutions� � �� ,the

proximity

measure

isnonnegative

anditis

equaltozero

ifandonly

ifthe

point� � ��

lieson

thecentralpath.

47

Page 48: €¦ · 1 INTR ODUCTION In the moder n inter ior-point theor y, all difficult constr aints are pushed into the con v e x set constr aints and/or con v e x cone constr aints. Then,

3C

EN

TR

AL

PATH

3.4N

eighbourhoodsB

asedon

theA

nalyticD

escriptionsofthe

CentralP

ath

Th

eorem

3.4Let

bea

LHS

CB

for �

with

parameter�

.T

hen�� � �� �

forall

int� �� ���

int� �� �

Moreover,the

inequalityabove

holdsas

equalityiff

� ���

��� � �

forsom

e� �

48

Page 49: €¦ · 1 INTR ODUCTION In the moder n inter ior-point theor y, all difficult constr aints are pushed into the con v e x set constr aints and/or con v e x cone constr aints. Then,

3C

EN

TR

AL

PATH

3.4N

eighbourhoodsB

asedon

theA

nalyticD

escriptionsofthe

CentralP

ath

For

convenience,we

write

����

��

�� ��

and

�����

��

�� � .O

necan

thinkof

and��

asthe

shadowiterates,as

��

int� ��

and

���

int� ��

andif� � ��

isa

feasiblepair,then

�� �

iff��� �

iff� � ��

lieson

thecentralpath.

We

alsodenote

��� ��

����� � �

Th

eorem

3.5F

orevery� � ��

�int� �

��

int� �� ,

���

��

Equality

holdsabove

iff ��

��

�� ��

(andhence

� ��

��

�� � ).

49

Page 50: €¦ · 1 INTR ODUCTION In the moder n inter ior-point theor y, all difficult constr aints are pushed into the con v e x set constr aints and/or con v e x cone constr aints. Then,

3C

EN

TR

AL

PATH

3.5P

ath-Follow

ing

3.5P

ath-F

ollo

win

g

While

mostelegantpotential-reduction

algorithms

mightnotm

akeany

referenceto

thecentralpath

(evenincluding

thetheoreticalanalysis),

path-following

andpotential-reduction

algorithms

arevery

closely

connected.

Currently,m

ostofthepracticalim

plementations

ofIPM

sboth

forLP

and

Sym

CP

arebased

onm

odificationsofpath-follow

ingalgorithm

s.

50

Page 51: €¦ · 1 INTR ODUCTION In the moder n inter ior-point theor y, all difficult constr aints are pushed into the con v e x set constr aints and/or con v e x cone constr aints. Then,

3C

EN

TR

AL

PATH

3.5P

ath-Follow

ing

There

arem

anystrategies

availableto

usfor

following

thecentralpath.

Our

algorithms

generatesearch

directions

��

and ��

andstep

sizes

��

and

��

andupdate

����

�� ���

and

��

���

� �� �

Intheory,itis

veryconvenientto

take

�� �

��

��

� �

51

Page 52: €¦ · 1 INTR ODUCTION In the moder n inter ior-point theor y, all difficult constr aints are pushed into the con v e x set constr aints and/or con v e x cone constr aints. Then,

3C

EN

TR

AL

PATH

3.5P

ath-Follow

ing

How

ever,thereare

possibleadvantages

inpractise

toallow

themto

take

differentvalues.

52

Page 53: €¦ · 1 INTR ODUCTION In the moder n inter ior-point theor y, all difficult constr aints are pushed into the con v e x set constr aints and/or con v e x cone constr aints. Then,

3C

EN

TR

AL

PATH

3.5P

ath-Follow

ing

Whatproperties

dow

eask

forin

thesearch

directions?

Improve

thecurrentduality

gap� �� �

Getcloser

tothe

centralpathw

ithoutincreasing� �� �

(verym

uch)

Asuitable

combination

ofthefirsttw

oabove!

We

canalso

mix

theseproperties

“externally”in

apredictor-corrector

methods.

53

Page 54: €¦ · 1 INTR ODUCTION In the moder n inter ior-point theor y, all difficult constr aints are pushed into the con v e x set constr aints and/or con v e x cone constr aints. Then,

3C

EN

TR

AL

PATH

3.5P

ath-Follow

ing

Let’sdefine

�� ��

� ���

� ���

and

�� ��

�����

� �� �

Given

�� � ��

(acentrality

parameter

definingsom

eofthe

propertiesof

thesearch

direction),thereare

many

searchdirections

achieving

� �� �� � �� �� �� �

�� ��

��� � �� � �

andfor

largeenough

(e.g.,

�� �� ,

�� �� � ,

�� ��

� ,...),

� �� �� � �� ���

staysin

asuitable

neighbourhoodofthe

centralpath.

More

onsearch

directionsfor

SD

Patthe

endofthis

lectureand

atthe

beginningofthe

next...

54

Page 55: €¦ · 1 INTR ODUCTION In the moder n inter ior-point theor y, all difficult constr aints are pushed into the con v e x set constr aints and/or con v e x cone constr aints. Then,

4P

RIM

AL-D

UA

LP

OT

EN

TIA

LF

UN

CT

ION

4P

rimal-D

ualP

oten

tialFu

nctio

n

Given�

��

���and�

���

���

apair

ofprimal-dualfeasible

andinterior

pairs,

we

would

liketo

havea

simple

way

ofcomparing

them.

We

havetw

o

criteria:

smaller

theduality

gap� �� �

isthe

better,

smaller

thedistance

tothe

centralpath(thatis,the

valueof

�� �� �� )

isthe

better.

55

Page 56: €¦ · 1 INTR ODUCTION In the moder n inter ior-point theor y, all difficult constr aints are pushed into the con v e x set constr aints and/or con v e x cone constr aints. Then,

4P

RIM

AL-D

UA

LP

OT

EN

TIA

LF

UN

CT

ION

The

nextfunction,calledthe

primal-dualpotentialfunction,serves

sucha

purposeand

allows

usto

designand

performthe

complexity

analysis

directlyon

it.

�� � �� ��� �

���

�� �� �� �

�� �� �� �

where

(we

willtake

���

in

ouranalysis).

56

Page 57: €¦ · 1 INTR ODUCTION In the moder n inter ior-point theor y, all difficult constr aints are pushed into the con v e x set constr aints and/or con v e x cone constr aints. Then,

4P

RIM

AL-D

UA

LP

OT

EN

TIA

LF

UN

CT

ION

Th

eorem

4.1S

upposew

ehave

� �� �� �� ��

feasiblefor� �

and����

suchthat

�� �� �� �� �� �

��

��

forsom

e

��� � �� �

Ifwe

generate� �� ��� �� ���

feasiblein� �

and����

suchthat

�� �� �

� ��� �� ��� �

�� �� �

� � ���� �� � �����

��

forevery

� ��

forsom

e

anabsolute

constant,thenfor

some

�� ���

��

� �� ��� ,w

ehave

� �� ��� �� �� �

�� �� �� �� � �

forevery

�� �57

Page 58: €¦ · 1 INTR ODUCTION In the moder n inter ior-point theor y, all difficult constr aints are pushed into the con v e x set constr aints and/or con v e x cone constr aints. Then,

5A

LGO

RIT

HM

AN

DC

OM

PU

TATIO

NA

LC

OM

PLE

XIT

YA

NA

LYS

IS

5A

lgo

rithm

and

Co

mp

utatio

nalC

om

plexity

An

alysis

Based

onthe

lasttheorem,our

problemofproducing

anapproxim

ately

optimalsolution

pairis

reducedto

decreasingthe

potentialfunctionvalue

bya

constant,inevery

iteration.W

ew

anttoupdate� �

� ��� �� ���

to

� �� � ��� �� � ���

suchthat� �� �� , �� ������

, ��

,

��

areallm

aintained;moreover,� �

� �

isdecreased

and

�� �� ��

isnotincreased

alot(in

comparison

tothe

dualitygap).

We

canexpress

theupdate

from

� �� ��� �� ���

to� �� � ��� �� � ���

bya

pairofsearch

directions ��

, ��� � �

respectivelyand

astep

size

�� �

.W

e

dropthe

iteratenum

bersfor

thispartofour

studyand

define

�� ��� ���

� ���

and �� ��� ���

� �� �

58

Page 59: €¦ · 1 INTR ODUCTION In the moder n inter ior-point theor y, all difficult constr aints are pushed into the con v e x set constr aints and/or con v e x cone constr aints. Then,

5A

LGO

RIT

HM

AN

DC

OM

PU

TATIO

NA

LC

OM

PLE

XIT

YA

NA

LYS

IS

Tom

aintainthe

feasibilityofthe

iterates,thesearch

directionsm

ustsatisfy

���

�� �

and �� �

� ����

� � �

forsom

e

��

����

;i.e., ��

mustbe

inthe

nullspaceof

����

and ��

mustbe

inthe

rangeof �

���� .

We

willderive

analgorithm

thatissym

metric

between

theprim

alandthe

dual(primal-dualsym

metry

),we

would

alsolike

tohave

ouralgorithm

invariantunderthe

symm

etriesofthe

coneconstrains

(scale-invariance).

Tobe

more

specific,insteadofform

alizingthese

vaguegoals,w

ew

ill

describean

approachw

hichattains

thesegoals.

59

Page 60: €¦ · 1 INTR ODUCTION In the moder n inter ior-point theor y, all difficult constr aints are pushed into the con v e x set constr aints and/or con v e x cone constr aints. Then,

5A

LGO

RIT

HM

AN

DC

OM

PU

TATIO

NA

LC

OM

PLE

XIT

YA

NA

LYS

IS

For

every� �� ��� � �

� �

we

would

liketo

havea

self-adjoint,

positivedefinite

lineartransform

ation

�� ���� �

suchthat

��

Aut� � � � �

��� �� �

� ��

� �� �

���

��� � �

�� �

� ��

� � ��

� �� �

.

Ifwe

canfind

suchtransform

ation�

,thenw

ecan

map

ourprim

al-space

with

them

apping

� ��

andthe

dual-spacew

ith

.T

hism

odificationdoes

notchangeanything

significantly,exceptthatourcurrentprim

al-dual

iterateis

mapped

onto� ���� .

Let’selaborate:

60

Page 61: €¦ · 1 INTR ODUCTION In the moder n inter ior-point theor y, all difficult constr aints are pushed into the con v e x set constr aints and/or con v e x cone constr aints. Then,

5A

LGO

RIT

HM

AN

DC

OM

PU

TATIO

NA

LC

OM

PLE

XIT

YA

NA

LYS

IS

Since

�(and

therefore

� ��

;because,

isself-adjoint)

isan

automorphism

ofthecone

ofpositivesem

idefinitem

atrices,them

ost

importantpartofthe

problem(for

thecurrentinterior-pointm

ethod

approach)is

unchanged.W

edefine

�����

���� ������

��� ��� ��

��

����� �

����

�� �

��

�� �����

�� �

61

Page 62: €¦ · 1 INTR ODUCTION In the moder n inter ior-point theor y, all difficult constr aints are pushed into the con v e x set constr aints and/or con v e x cone constr aints. Then,

5A

LGO

RIT

HM

AN

DC

OM

PU

TATIO

NA

LC

OM

PLE

XIT

YA

NA

LYS

IS

Now

,� ��

and����

become

���

�inf

���

���

�� ��

��

�� �

�� � � � �� � �

��

��

sup� ��

���� �� ���

���

��

�� � � � �� ��

Inthese

scaledspaces,the

searchdirections

arestillorthogonal:

��

mustlie

inthe

nullspaceof

�����

and

��

mustlie

inthe

rangeof

� ����� .

62

Page 63: €¦ · 1 INTR ODUCTION In the moder n inter ior-point theor y, all difficult constr aints are pushed into the con v e x set constr aints and/or con v e x cone constr aints. Then,

5A

LGO

RIT

HM

AN

DC

OM

PU

TATIO

NA

LC

OM

PLE

XIT

YA

NA

LYS

IS

Let’sanalyze

theduality

gap.W

ehave

� �� �� � �� ��

� �� � �

�� ����

�� �

��

�� �

Therefore,ifw

etake

as

��

and

��

theorthogonalprojection

of

ontothe

nullspaceof

�����and

rangeof

� �����

respectively,thenw

ew

ill

havethe

bestsearchdirection

toreduce

theduality

gapin

thissetting.

Now

,let’sturn

tothe

centralitypart

ofthepotentialfunction.

Whatkind

of

searchdirection

would

improve

thebarrier

functionvalues

inthis

setting?

We

utilizethe

following

technicallemm

aw

hichsum

marizes

many

ofthe

niceproperties

ofthebarrier

function

.

63

Page 64: €¦ · 1 INTR ODUCTION In the moder n inter ior-point theor y, all difficult constr aints are pushed into the con v e x set constr aints and/or con v e x cone constr aints. Then,

5A

LGO

RIT

HM

AN

DC

OM

PU

TATIO

NA

LC

OM

PLE

XIT

YA

NA

LYS

IS

Lem

ma

5.1Let�

� � �

.S

uppose�� � �

satisfies

� �� �� ������� �

��� �

� �� ��

��

Then

�� �� �� �

�� �� � � �

�� ���� �

�� ���� �

�� �� �� �

�� �� � � �

� �� ��

�� �

� �� �� �

64

Page 65: €¦ · 1 INTR ODUCTION In the moder n inter ior-point theor y, all difficult constr aints are pushed into the con v e x set constr aints and/or con v e x cone constr aints. Then,

5A

LGO

RIT

HM

AN

DC

OM

PU

TATIO

NA

LC

OM

PLE

XIT

YA

NA

LYS

IS

Rem

ark5.1

The

condition

� �� ���

(� �

)im

plies� ���� �

(

).T

hisis

clearfrom

thestatem

entofthelem

ma.

Butitcan

alsobe

directlyobserved

asfollow

s:

� � �� �

�� � �

�� ��� �

�� �

��

Therefore,� � �

�� ��� �

�� �

� ���

.B

utthisis

equivalentto

� �� �

�� ��� �

�� �

� .

Ifwe

applythe

automorphism�

�� ����� �

of

� �

toboth

sides,we

obtainequivalently �

��� .

65

Page 66: €¦ · 1 INTR ODUCTION In the moder n inter ior-point theor y, all difficult constr aints are pushed into the con v e x set constr aints and/or con v e x cone constr aints. Then,

5A

LGO

RIT

HM

AN

DC

OM

PU

TATIO

NA

LC

OM

PLE

XIT

YA

NA

LYS

IS

Let’sfocus

onthe

firstorderupper

estimate

on

�� �� �����

�� �� ���

givenby

theabove

lemm

a.W

eobtain

� � ��� �

� � �� �� �

�� �� � ���

�� � ��

��

�� �

��

� �

Thus,as

inour

analysisofthe

dualitygap,ifw

etake

as

��

and

��

the

orthogonalprojectionof

� ��

ontothe

nullspaceof

�����

andrange

of

�������

respectively,thenw

ew

illhavethe

bestsearchdirection

toreduce

thefirstorder

termin

theupper

boundon

thevalue

ofthebarrier

terms

�� �� �

�� �� ,in

thissetting.

Therefore,to

reducethe

valueofthe

potentialfunction,itseems

desirable

tochoose

am

atrixw

hichis

anonnegative

linearcom

binationof

��

and

� ��

andthen

define

��

and

��

asthe

orthogonalprojectionsofthis

matrix

ontothe

nullspaceof

�� ��

andthe

rangeof

� �����

respectively.

66

Page 67: €¦ · 1 INTR ODUCTION In the moder n inter ior-point theor y, all difficult constr aints are pushed into the con v e x set constr aints and/or con v e x cone constr aints. Then,

5A

LGO

RIT

HM

AN

DC

OM

PU

TATIO

NA

LC

OM

PLE

XIT

YA

NA

LYS

IS

This

isprecisely

whatw

edo

next.Let

��

� ��

� �

� �� �

��� �

��

Note

that���

��

iff

� ��

���

� ��

� �� �

��

.B

utthelatter

leadsto

a

contradiction(that �

)upon

takingthe

innerproductw

ith

of

bothsides.

Therefore,�

���

andw

edefine

� �

���

���

�In

fact,���

��

isconnected

toa

measure

ofcentrality.R

ecall

��� �� � �

�� � �

�� � �

��

� ��

�67

Page 68: €¦ · 1 INTR ODUCTION In the moder n inter ior-point theor y, all difficult constr aints are pushed into the con v e x set constr aints and/or con v e x cone constr aints. Then,

5A

LGO

RIT

HM

AN

DC

OM

PU

TATIO

NA

LC

OM

PLE

XIT

YA

NA

LYS

IS

Then

��

� ��

� �

� �

� �� �

��� �

��

��

���

� � ���

������� �

Note

that� ���

���

is ���

times

thesquared

normofthe

errorin

the

equation��

� � ��

,where

thenorm

isw

ithrespectto

thelocalm

etric

inducedby

� �

.W

ehave

Co

rollary

5.1F

orevery�

� �� � �

,we

have

���

� ��

�� �

� �

The

equalityholds

aboveiff�

�� � �

.

68

Page 69: €¦ · 1 INTR ODUCTION In the moder n inter ior-point theor y, all difficult constr aints are pushed into the con v e x set constr aints and/or con v e x cone constr aints. Then,

5A

LGO

RIT

HM

AN

DC

OM

PU

TATIO

NA

LC

OM

PLE

XIT

YA

NA

LYS

IS

Then

��

,

�� ,

��

make

upthe

uniquesolution

ofthesystem

:

��

��

���

� �� �

� ����

��

��

� ��

��

��

By

definition,

��

��� �� ��

��

�� ��

��

�� ��

���

Therefore,w

eim

mediately

concludethat

��

���

���

and�

��

��

����

Now

,we

analyze

� ��� �

and

� ��� �

.W

ehave

� ��� ��

������� �

��� � �

69

Page 70: €¦ · 1 INTR ODUCTION In the moder n inter ior-point theor y, all difficult constr aints are pushed into the con v e x set constr aints and/or con v e x cone constr aints. Then,

5A

LGO

RIT

HM

AN

DC

OM

PU

TATIO

NA

LC

OM

PLE

XIT

YA

NA

LYS

IS

� �

��

����� �

��

��

�� � �

��

���� � � �

���

��

�� � � �

���

��� �

��

� �����

��

��� ��

��� �

��

� �����

�� ��� ��� �

Note

thatinthe

abovederivation,w

eencountered

thelinear

operator�� �

�� ���

�����

which

happensto

coincide(in

thiscase)

with

thelinear

operator

��� �

�� ��� �

�� �

�� �����70

Page 71: €¦ · 1 INTR ODUCTION In the moder n inter ior-point theor y, all difficult constr aints are pushed into the con v e x set constr aints and/or con v e x cone constr aints. Then,

5A

LGO

RIT

HM

AN

DC

OM

PU

TATIO

NA

LC

OM

PLE

XIT

YA

NA

LYS

IS

Sim

ilarly,� �

�� ���

��� �

��

� �����

��

��� �

��� �

��

� �����

�� ��� ��� �

71

Page 72: €¦ · 1 INTR ODUCTION In the moder n inter ior-point theor y, all difficult constr aints are pushed into the con v e x set constr aints and/or con v e x cone constr aints. Then,

5A

LGO

RIT

HM

AN

DC

OM

PU

TATIO

NA

LC

OM

PLE

XIT

YA

NA

LYS

IS

We

have,�� ��

�� � �

� �

���

� �� �� � �� ��

� �� �

����� �

�� �

� ��� ��

�� �

�� �

�� �� � �

� ��� ��

�� �

�� �

�� �� �

�� �

� �� �

����

����� �

��

� �

� �� ��� ��� �

�� �

�� �� ��� ���� �

��

���

�� �

� �

� �� ��� ��� �

� ��

�� �� ��� ���� �

���� ��

��� �

� �� ��� ��� �

� ��

�� �� ��� ���� �

72

Page 73: €¦ · 1 INTR ODUCTION In the moder n inter ior-point theor y, all difficult constr aints are pushed into the con v e x set constr aints and/or con v e x cone constr aints. Then,

5A

LGO

RIT

HM

AN

DC

OM

PU

TATIO

NA

LC

OM

PLE

XIT

YA

NA

LYS

IS

�� �

�� �

�������� �

�� �

���

����

73

Page 74: €¦ · 1 INTR ODUCTION In the moder n inter ior-point theor y, all difficult constr aints are pushed into the con v e x set constr aints and/or con v e x cone constr aints. Then,

5A

LGO

RIT

HM

AN

DC

OM

PU

TATIO

NA

LC

OM

PLE

XIT

YA

NA

LYS

IS

ALG

OR

ITH

M

Given� �

� �� �� ��

feasiblein� �

and����

suchthat�

� �

,�

� �

.A

lsogiven

is

��� � ��

suchthat

�� �� �� �� �� �

��

� �� �� .

���� .

While� �

� ��� �� ��

�� �� �� �� � �

� �� �� �

� ��� �

�� �

� � �� ��� �� ��

� ��

� �� ��� �� �

� �� �

� �� ��� �

�� �

�����

���� ��

��

��

�� � �

��� �

� ���

� ���

��

����

� �� �

� ��� �� ���

��� �

���

���

��� �

74

Page 75: €¦ · 1 INTR ODUCTION In the moder n inter ior-point theor y, all difficult constr aints are pushed into the con v e x set constr aints and/or con v e x cone constr aints. Then,

5A

LGO

RIT

HM

AN

DC

OM

PU

TATIO

NA

LC

OM

PLE

XIT

YA

NA

LYS

IS

Solve

thesystem

��

��

���

� �� �

� ����

��

��

� ��

��

��

Com

pute

�����

���

�� �� �

� ���

��

��

��� �

� ���

�� �

��

��

� ��

��

.

Let�� � ��� ��

� ���

���

��

���

�� � ��� ��

� ���

��� �

��

��

� ��

����

���

.

end� While�

75

Page 76: €¦ · 1 INTR ODUCTION In the moder n inter ior-point theor y, all difficult constr aints are pushed into the con v e x set constr aints and/or con v e x cone constr aints. Then,

5A

LGO

RIT

HM

AN

DC

OM

PU

TATIO

NA

LC

OM

PLE

XIT

YA

NA

LYS

IS

The

aboveis

whatis

typicallycalled

apotentialreduction

algorithm.

We

provedthe

following

theorem.

Th

eorem

5.1T

heabove

algorithmterm

inatesin

atmost

��

��

� �� ��

iterationsw

ithfeasible �

� ��� �� ��

suchthat

� �� ��� �� �� �

�� �� �� �� � �

Even

thoughthe

algorithmrequires

theiterates

tolie

inthe

interiorofthe

underlyingcone

constraints,we

canrelax

theinitialfeasibility

assumption

byusing

auxiliaryoptim

izationproblem

s.

76

Page 77: €¦ · 1 INTR ODUCTION In the moder n inter ior-point theor y, all difficult constr aints are pushed into the con v e x set constr aints and/or con v e x cone constr aints. Then,

6IN

FE

AS

IBLE

-STA

RT

ALG

OR

ITH

MS

6In

feasible-S

tartAlg

orith

ms

Another

approachis

tow

orkin

thefram

ework

ofthealgorithm

thatwe

discussed(or

some

otherprim

al-dualinterior-pointalgorithm)

butmodify

thesearch

directionsso

thatthesearch

directionsalso

tryelim

inatethe

errorin

thelinear

equationsdefining

theprim

alanddualfeasible

regions.

Insteadofhaving

oursearch

directions��

and��

lyingin

thenullspace

of����

andrange

of ����� ,w

eask

thattheysatisfy

thefollow

ingsystem

ofequations:

���

�� ��

� �

� ���

and �� �

� � ��

� ��

�� �

� ���

�� ���

77

Page 78: €¦ · 1 INTR ODUCTION In the moder n inter ior-point theor y, all difficult constr aints are pushed into the con v e x set constr aints and/or con v e x cone constr aints. Then,

6IN

FE

AS

IBLE

-STA

RT

ALG

OR

ITH

MS

The

analysisbecom

esm

orecom

plicated,however,this

isone

ofthe

popularw

aysto

solveS

DP

problems

inpractise.

The

algorithms

needto

carefullym

onitorthe

progressin

attainingfeasibility,reducing� �

� �

as

wellas

theproxim

ityto

the“centralsurface.”

(Since

we

allowinfeasible

iterates,we

willbe

concernedw

iththe

distanceto

the“centralsurface”

ratherthan

thecentralpath.)

For

instance,thealgorithm

shouldnotallow

thefastreduction

of� �� �

unlessthe

iteratesare

gettingto

benear

feasibleatleastas

fast.

78

Page 79: €¦ · 1 INTR ODUCTION In the moder n inter ior-point theor y, all difficult constr aints are pushed into the con v e x set constr aints and/or con v e x cone constr aints. Then,

7O

TH

ER

INT

ER

IOR

-PO

INT

ALG

OR

ITH

MS

,GE

NE

RA

LR

EM

AR

KS

7O

ther

Interio

r-Po

int

Alg

orith

ms,G

eneralR

emarks

The

searchdirections

thatwe

discussedare

known

asthe

NT

direction(for

Nesterov-Todd).

These

algorithms

havebeen

generalizedto

convex

optimization

problems

overarbitrary

convexcones.

Other

primal-dualalgorithm

sthatare

usefulandpopular

relyon

search

directionsproposed

Helm

berg-Rendl-V

anderbei-Wolkow

icz/Kojim

a-Shindoh-H

ara/Monteiro

(HK

Mdirection)

andA

lizadeh-Haeberly-O

verton(A

HO

direction).A

llthese

directionscan

bedefined

andtreated

ina

unifiedw

ay(due

toY.Z

hang,

some

otherrelated

work

isdue

toM

onteiro-Y.Zhang):

79

Page 80: €¦ · 1 INTR ODUCTION In the moder n inter ior-point theor y, all difficult constr aints are pushed into the con v e x set constr aints and/or con v e x cone constr aints. Then,

7O

TH

ER

INT

ER

IOR

-PO

INT

ALG

OR

ITH

MS

,GE

NE

RA

LR

EM

AR

KS

Let�� � ���

.D

efine

���� ���

��� �

asfollow

s

��� ��

� ���� �

��� ��� �

� ��

This

������is

calledthe

symm

etrizedsim

ilaritytransform

ation.To

compute

thesearch

direction,we

solvethe

system

� �

���

��

� �

� ��� �

�� �

� � ��

��

�� �

� ���

�� ���

��� �

� ���

� ��

� �� ���

��

���

��� �

� ���� ���

where

�� � ��

aparam

eterfixed

bythe

user/algorithmand

� �� �

� ��� �� ���

asbefore.

80

Page 81: €¦ · 1 INTR ODUCTION In the moder n inter ior-point theor y, all difficult constr aints are pushed into the con v e x set constr aints and/or con v e x cone constr aints. Then,

7O

TH

ER

INT

ER

IOR

-PO

INT

ALG

OR

ITH

MS

,GE

NE

RA

LR

EM

AR

KS

Choosing�

� ��

givesthe

AH

Odirection,�

���� �

� ��� �� �

yieldsthe

HK

Mdirection,choosing

any �� � ���

suchthat

� ���

� �� ��� ��� �

� � �� ��� �� ��

� ��� �� ��� �� �

� �� �

� �� ��� ��� �

(forinstance,�

� �� � �

� ��� �� ��

� ��� �� ��� �� �

� ��

�� �� ��� ��� �

)gives

theN

Tdirection.

The

nextlecturestarts

with

adiscussion

ofthecom

putationalissues

relatedto

thesearch

directionsfor

SD

Pw

hichties

innicely

with

thebundle

methods .

81

Page 82: €¦ · 1 INTR ODUCTION In the moder n inter ior-point theor y, all difficult constr aints are pushed into the con v e x set constr aints and/or con v e x cone constr aints. Then,

7O

TH

ER

INT

ER

IOR

-PO

INT

ALG

OR

ITH

MS

,GE

NE

RA

LR

EM

AR

KS

We

canalso

designa

wide

rangeofprim

al-dualalgorithms

withoutthe

conicstructure

orlogarithm

ichom

ogeneity:

Polynom

ialtime

IPM

s

WIT

HO

RW

ITH

OU

T

theC

onicS

tructureand

Logarithmically

Hom

ogeneous

Barriers!

(Froma

recentpaperby

Nem

irovskiandT.)

82

Page 83: €¦ · 1 INTR ODUCTION In the moder n inter ior-point theor y, all difficult constr aints are pushed into the con v e x set constr aints and/or con v e x cone constr aints. Then,

7O

TH

ER

INT

ER

IOR

-PO

INT

ALG

OR

ITH

MS

,GE

NE

RA

LR

EM

AR

KS

We

aregiven

a�

-SC

B�

with

adom

ain

��

andthe

Legendre-Fenchelconjugate

��

(with

aslightdifference

fromthe

previousdefn.)

of

;thedom

ainof

��

isdenoted

��

.�

� is

acone:

��

��

���

�� ��

alinear

embedding

��� �

with

thenullspace

�� �

andthe

image

intersecting

��

;

avector�� � .

83

Page 84: €¦ · 1 INTR ODUCTION In the moder n inter ior-point theor y, all difficult constr aints are pushed into the con v e x set constr aints and/or con v e x cone constr aints. Then,

7O

TH

ER

INT

ER

IOR

-PO

INT

ALG

OR

ITH

MS

,GE

NE

RA

LR

EM

AR

KS

theoptim

izationproblem

��

�� ���� ���

� ���

� �� ��

��� �

we

areinterested

insolving;

thefunction

�� � �

�� � ��

which

isa�

-SC

Bfor

cl� �� .

84

Page 85: €¦ · 1 INTR ODUCTION In the moder n inter ior-point theor y, all difficult constr aints are pushed into the con v e x set constr aints and/or con v e x cone constr aints. Then,

7O

TH

ER

INT

ER

IOR

-PO

INT

ALG

OR

ITH

MS

,GE

NE

RA

LR

EM

AR

KS

Ashifted

centralpath

Lem

ma

7.1F

or� ,the

“primal-dualpair”� ��� �

� �� �� �� �� ���

is

uniquelydefined

bythe

relations� ��

��

�� ���

� ��

�� �� �

���

� ���

��� �� �� �� �

� ��

�� � ��� �

Moreover,

� �� �� �

argmin

� ���� ��

� �� �� �

���� �

85

Page 86: €¦ · 1 INTR ODUCTION In the moder n inter ior-point theor y, all difficult constr aints are pushed into the con v e x set constr aints and/or con v e x cone constr aints. Then,

7O

TH

ER

INT

ER

IOR

-PO

INT

ALG

OR

ITH

MS

,GE

NE

RA

LR

EM

AR

KS

7.1P

roximity

measure

7.1P

roximity

measu

re

Letusdefine

theproxim

itym

easureas

thefunction

�� ��� �

�� � �� �

��� ��

� � � � �

���

�� ��

(Legendre-Fenchelgap

between

and

��

).N

oticethatfor

every��

andevery�

���

,we

have�� ���

andfor

sucha

pair� ���

we

have

�� ��� �

iff� ��

�� � �� .

Using

thissetup

many

path-following

andpotential-reduction

algorithms

canbe

derivedand

analyzed.

86

Page 87: €¦ · 1 INTR ODUCTION In the moder n inter ior-point theor y, all difficult constr aints are pushed into the con v e x set constr aints and/or con v e x cone constr aints. Then,

RE

FE

RE

NC

ES

RE

FE

RE

NC

ES

Referen

ces

[1]F.A

lizadeh,“Interior

pointmethods

insem

idefiniteprogram

ming

with

applicationsto

combinatorialoptim

ization,”S

IAM

Journalon

Optim

ization5

(1995)13–51.

[2]F.A

lizadeh,J-P.A.H

aeberly,andM

.L.Overton,

Prim

al-dual

interior-pointmethods

forsem

idefiniteprogram

ming:

Convergence

rates,stabilityand

numericalresults,

SIA

MJ.O

ptim.,8:746–768,

1998.

[3]A

.Ben-Taland

A.N

emirovski,Lectures

onM

odernC

onvex

Optim

ization.Analysis,A

lgorithms,and

Engineering

Applications,

MP

S/S

IAM

Series

onO

ptimization,

Philadelphia,PA

,US

A,2001.

87

Page 88: €¦ · 1 INTR ODUCTION In the moder n inter ior-point theor y, all difficult constr aints are pushed into the con v e x set constr aints and/or con v e x cone constr aints. Then,

RE

FE

RE

NC

ES

RE

FE

RE

NC

ES

[4]C

.B.C

hua,Relating

homogeneous

conesand

positivedefinite

cones

via�

-algebras,SIA

MJ.O

pt.14(2004)

500-506.

[5]J.Farautand

A.K

oranyi,Analysis

onS

ymm

etricC

ones,Oxford

University

Press,N

Y,US

A,1994.

[6]M

.Grotschel,L.Lovasz

andA

.Schrijver,

Geom

etricalgorithm

sand

combinatorialoptim

ization(S

pringer,New

York,1988).

[7]O

.Guler,B

arrierfunctions

ininterior

pointmethods,M

ath.ofOper.

Res.21

(1996),pp.860–885.

[8]O

.Guler,H

yperbolicP

olynomials

andInterior

PointM

ethodsfor

Convex

Program

ming,M

ath.Oper.R

es.22(1997)

350–377.

88

Page 89: €¦ · 1 INTR ODUCTION In the moder n inter ior-point theor y, all difficult constr aints are pushed into the con v e x set constr aints and/or con v e x cone constr aints. Then,

RE

FE

RE

NC

ES

RE

FE

RE

NC

ES

[9]C

.Helm

berg,F.Rendl,R

.Vanderbeiand

H.W

olkowicz,A

n

interior-pointmethod

forsem

idefiniteprogram

ming.S

IAM

J.Optim

.6

(1996)342–361.

[10]R

.A.H

ornand

C.R

.Johnson,Topics

inm

atrixanalysis,

Cam

bridge

University

Press,C

ambridge,1994.

Corrected

reprintofthe1991

original.

[11]Jarre,F.Interior-P

ointMethods

viaS

elf-Concordance

orR

elative

LipschitzC

ondition,Habilitationsschrift,

University

ofWurzburg,July

1994.

[12]N

.Karm

arkar,A

newpolynom

ialtime

algorithmfor

linear

programm

ing.C

ombinatorica

4(1984)

373–395.

89

Page 90: €¦ · 1 INTR ODUCTION In the moder n inter ior-point theor y, all difficult constr aints are pushed into the con v e x set constr aints and/or con v e x cone constr aints. Then,

RE

FE

RE

NC

ES

RE

FE

RE

NC

ES

[13]M

.Kojim

a,M.S

hidaand

S.S

hindoh,S

earchdirections

inthe

SD

P

andthe

monotone

SD

LCP

:generalizationand

inexactcomputation.

Math.P

rogramm

ing,85:51–80,1999.

[14]M

.Kojim

a,S.S

hindohand

S.H

ara,Interior-pointmethods

forthe

monotone

linearcom

plementarity

problemin

symm

etricm

atrices.

SIA

MJ.O

ptim.7

(1997)86–125.

[15]R

.D.C

.Monteiro,F

irstandsecond

orderm

ethodsfor

semidefinite

programm

ing,Math.P

rog.B97

(2003)209–244.

[16]R

.D.C

.Monteiro,P

rimal-dualpath

following

algorithms

for

semidefinite

programm

ing.SIA

MJ.O

ptim.7

(1997)663–678.

90

Page 91: €¦ · 1 INTR ODUCTION In the moder n inter ior-point theor y, all difficult constr aints are pushed into the con v e x set constr aints and/or con v e x cone constr aints. Then,

RE

FE

RE

NC

ES

RE

FE

RE

NC

ES

[17]R

.D.C

.Monteiro

andT.T

suchiya,Polynom

ialconvergenceofa

new

family

ofprimal-dualalgorithm

sfor

semidefinite

programm

ing,SIA

M

J.Optim

.9(1999)

551–577.

[18]A

.Nem

irovskiandL.Tuncel,“C

one-free”prim

al-dualpath-following

andpotentialreduction

polynomialtim

einterior-pointm

ethods,

Research

Report

CO

RR

2002-32,Dept.ofC

ombinatorics

and

Optim

ization,FacultyofM

athematics,U

niversityofW

aterloo,

Canada,O

ctober2002.

[19]Yu.E

.Nesterov

andA

.S.N

emirovskii,

Interior-PointP

olynomial

Algorithm

sin

Convex

Program

ming,

SIA

M,P

hiladelphia,PA,U

SA

,

1994.

91

Page 92: €¦ · 1 INTR ODUCTION In the moder n inter ior-point theor y, all difficult constr aints are pushed into the con v e x set constr aints and/or con v e x cone constr aints. Then,

RE

FE

RE

NC

ES

RE

FE

RE

NC

ES

[20]N

esterov,Yu.E.and

M.J.Todd

(1997).Self-scaled

barriersand

interior-pointmethods

forconvex

programm

ing,Math.O

per.Res.22

1–46.

[21]Yu.E

.Nesterov

andM

.J.Todd,Prim

al-dualinterior-pointmethods

for

self-scaledcones,S

IAM

J.Optim

ization8

(1998)324–364.

[22]J.R

enegar,AM

athematicalV

iewofInterior-P

ointMethods

inC

onvex

Optim

ization,MP

S/S

IAM

Series

onO

ptimization,P

hiladelphia,PA,

US

A,2001.

[23]R

ockafellar,R.T.(1970).C

onvexA

nalysis,Princeton

University

Press,P

rinceton,New

Jersey.

[24]A

.Schrijver,

Theory

ofLinearand

IntegerP

rogramm

ing.John

Wiley

andS

ons,1986.

92

Page 93: €¦ · 1 INTR ODUCTION In the moder n inter ior-point theor y, all difficult constr aints are pushed into the con v e x set constr aints and/or con v e x cone constr aints. Then,

RE

FE

RE

NC

ES

RE

FE

RE

NC

ES

[25]A

.Shapiro,F

irstandsecond

orderanalysis

ofnonlinearsem

idefinite

programs,M

ath.Program

ming

B77

(1997)301–320.

[26]M

.J.Todd,On

searchdirections

ininterior-pointm

ethodsfor

semidefinite

programm

ing,Optim

izationM

ethodsand

Softw

are11

(1999)1–46.

[27]M

.J.Todd,Potential-reduction

methods

inm

athematical

programm

ing,Mathem

aticalProgram

ming

76(1997)

3–45.

[28]L.Tuncel,G

eneralizationofprim

al-dualinterior-pointmethods

to

convexoptim

izationproblem

sin

conicform

,Foundations

of

Com

putationalMathem

atics1

(2001)229–254.93

Page 94: €¦ · 1 INTR ODUCTION In the moder n inter ior-point theor y, all difficult constr aints are pushed into the con v e x set constr aints and/or con v e x cone constr aints. Then,

RE

FE

RE

NC

ES

RE

FE

RE

NC

ES

[29]L.Tuncel,P

otentialreductionand

primal-dualm

ethods,Handbook

of

Sem

idefiniteP

rogramm

ing:T

heory,Algorithm

sand

Applications,H

.

Wolkow

icz,R.S

aigalandL.V

andenberghe(eds.),K

luwer

Academ

ic

Publishers,B

oston,MA

,US

A,2000,pp.235–265.

[30]L.Tunceland

H.W

olkowicz,S

trengthenedexistence

anduniqueness

conditionsfor

searchdirections

insem

idefiniteprogram

ming,

Research

Report

CO

RR

2003-20,Dept.ofC

ombinatorics

and

Optim

ization,FacultyofM

athematics,U

niversityofW

aterloo,Ontario,

Canada,July

2003.

[31]L.V

andenbergheand

S.B

oyd,Aprim

al-dualpotentialreduction

method

forproblem

sinvolving

matrix

inequalities,Mathem

atical

Program

ming

69(1995)

205–236.

94

Page 95: €¦ · 1 INTR ODUCTION In the moder n inter ior-point theor y, all difficult constr aints are pushed into the con v e x set constr aints and/or con v e x cone constr aints. Then,

RE

FE

RE

NC

ES

RE

FE

RE

NC

ES

[32]L.V

andenbergheand

S.B

oyd,“S

emidefinite

Program

ming,”

SIA

M

Review

38(1996)

49–95.

[33]H

.Wolkow

icz,S

ome

applicationsofoptim

izationin

matrix

theory.

LinearA

lgebraand

itsA

pplications,40:101–118,1981.

[34]H

.Wolkow

icz,R.S

aigalandL.V

andenberghe,editors.H

andbookof

Sem

idefiniteP

rogramm

ing:T

heory,Algorithm

s,andA

pplications,

Kluw

erA

cademic

Publishers,B

oston,MA

,2000.xxvi+

654pages.

[35]Y.Ye,A

pproximating

quadraticprogram

ming

with

boundand

quadraticconstraints,M

ath.Program

ming

84(1999)

219–226.

[36]Y.Z

hang,O

nextending

some

primal-dualinterior-pointalgorithm

s

fromlinear

programm

ingto

semidefinite

programm

ing.S

IAM

J.

Optim

.,8:365–386,1998.

95