convex sets - university of michiganmepelman/teaching/ioe611/handouts/sets.pdfane set: contains the...

Convex sets

I a�ne and convex sets

I some important examples

I operations that preserve convexity

I separating and supporting hyperplanes

I generalized inequalities

I dual cones and generalized inequalities

IOE 611: Nonlinear Programming, Fall 2017 2. Convex sets Page 2–1

A�ne set

a�ne combination of x1 and x2: any point x of the form

x = ✓x1 + (1� ✓)x2 where ✓ 2 R

Affine set

line through x1, x2: all points

x = θx1 + (1 − θ)x2 (θ ∈ R)PSfrag replacements

x1

x2

θ = 1.2θ = 1

θ = 0.6

θ = 0θ = −0.2

affine set: contains the line through any two distinct points in the set

example: solution set of linear equations {x | Ax = b}

(conversely, every affine set can be expressed as solution set of system oflinear equations)

Convex sets 2–2

line through x1, x2: all a�ne combinations of x1 and x2

a�ne set: contains the line through any two distinct points in theset (i.e., a set that’s closed under a�ne combinations)

example: solution set of linear equations {x | Ax = b}(conversely, every a�ne set can be expressed as solution set ofsystem of linear equations)


Convex setconvex combination of x1 and x2: any point x of the form

x = ✓x1 + (1� ✓)x2 where 0 ✓ 1

line segment between x1 and x2: all points

x = ✓x1 + (1� ✓)x2

with 0 ✓ 1convex set: contains line segment between any two points in theset

x1, x2 2 C , 0 ✓ 1 =) ✓x1 + (1� ✓)x2 2 C

examples (one convex, two nonconvex sets)

Convex set

line segment between x1 and x2: all points

x = θx1 + (1 − θ)x2

with 0 ≤ θ ≤ 1

convex set: contains line segment between any two points in the set

x1, x2 ∈ C, 0 ≤ θ ≤ 1 =⇒ θx1 + (1 − θ)x2 ∈ C

examples (one convex, two nonconvex sets)

Convex sets 2–3

IOE 611: Nonlinear Programming, Fall 2017 2. Convex sets –3

Convex combination and convex hull

convex combination of x1,. . . , xk

: any point x of the form

x = ✓1x1 + ✓2x2 + · · ·+ ✓k

x

k

with ✓1 + · · ·+ ✓k

= 1, ✓i

� 0

convex hull conv S : set of all convex combinations of points in S

Convex combination and convex hull

convex combination of x1,. . . , xk: any point x of the form

x = θ1x1 + θ2x2 + · · · + θkxk

with θ1 + · · · + θk = 1, θi ≥ 0

convex hull conv S: set of all convex combinations of points in S

Convex sets 2–4IOE 611: Nonlinear Programming, Fall 2017 2. Convex sets Page 2–4

Convex conecone: a set C such that for any x 2 C and ✓ � 0, ✓x 2 C

conic (nonnegative) combination of x1 and x2: any point of theform

x = ✓1x1 + ✓2x2 where ✓1, ✓2 � 0

Convex cone

conic (nonnegative) combination of x1 and x2: any point of the form

x = θ1x1 + θ2x2

with θ1 ≥ 0, θ2 ≥ 0

PSfrag replacements

0

x1

x2

convex cone: set that contains all conic combinations of points in the set

Convex sets 2–5

convex cone: a cone that is convex, i.e., set that contains allconic combinations

{✓1x1 + ✓2x2 + · · ·+ ✓k

x

k

| ✓i

� 0}

of points in the setIOE 611: Nonlinear Programming, Fall 2017 2. Convex sets Page 2–5

Examples of a�ne/convex sets and convex cones

Before we get to interesting examples, some trivial ones:

I An empty set ;, a singleton {x0} ⇢ R

n, the whole space R

n

are a�ne and convex sets in R

n

I Any line is a�ne. If it passes through 0, it is also a convexcone

I A line segment is convex, but not a�ne (unless a singleton)

I A ray (set {x0 + ✓v | ✓ � 0}) is convex but not a�ne; ifx0 = 0, it is a convex cone

I A subspace (a subset of Rn closed under sums and scalarmultiplications) is a�ne, convex, and is a convex cone


Hyperplanes and halfspaceshyperplane: set of the form {x | aT x = b} (a 6= 0)

Hyperplanes and halfspaces

hyperplane: set of the form {x | aTx = b} (a ̸= 0)

PSfrag replacements a

x

aTx = b

x0

halfspace: set of the form {x | aTx ≤ b} (a ̸= 0)

PSfrag replacements

a

aTx ≥ b

aTx ≤ b

x0

• a is the normal vector

• hyperplanes are affine and convex; halfspaces are convex

Convex sets 2–6

halfspace: set of the form {x | aT x b} (a 6= 0)

Hyperplanes and halfspaces

hyperplane: set of the form {x | aTx = b} (a ̸= 0)

PSfrag replacements a

x

aTx = b

x0

halfspace: set of the form {x | aTx ≤ b} (a ̸= 0)

PSfrag replacements

a

aTx ≥ b

aTx ≤ b

x0

• a is the normal vector

• hyperplanes are affine and convex; halfspaces are convex

Convex sets 2–6

Ia is the normal vector of the hyperplane/halfspace

I hyperplanes are a�ne and convex; halfspaces are convex


Polyhedrasolution set of finitely many linear inequalities and equalities

Ax � b, Cx = d

(A 2 R

m⇥n, C 2 R

p⇥n, � is componentwise inequality)

Polyhedra

solution set of finitely many linear inequalities and equalities

Ax ≼ b, Cx = d

(A ∈ Rm×n, C ∈ Rp×n, ≼ is componentwise inequality)

PSfrag replacements

a1 a2

a3

a4

a5

P

polyhedron is intersection of finite number of halfspaces and hyperplanes

Convex sets 2–9

polyhedron is intersection of finite number of halfspaces andhyperplanes


Positive semidefinite cone

notation:

IS

n is set of symmetric n ⇥ n matrices

IS

n

+ = {X 2 S

n | X ⌫ 0}: positive semidefinite n ⇥ n matrices

X 2 S

n

+ () z

T

Xz � 0 for all z

IS

n

++ = {X 2 S

n | X � 0}: positive definite n ⇥ n matrices

S

n

+ is a convex cone

example:

x y

y z

�2 S

2+

Positive semidefinite cone

notation:

• Sn is set of symmetric n × n matrices

• Sn+ = {X ∈ Sn | X ≽ 0}: positive semidefinite n × n matrices

X ∈ Sn+ ⇐⇒ zTXz ≥ 0 for all z

Sn+ is a convex cone

• Sn++ = {X ∈ Sn | X ≻ 0}: positive definite n × n matrices

example:

!x yy z

"∈ S2

+

PSfrag replacements

xyz

0

0.5

1

−1

0

10

0.5

1


Euclidean balls and ellipsoids(Euclidean) ball with center x

c

and radius r :

B(xc

, r) = {x | kx � x

c

k2 r} = {xc

+ ru | kuk2 1}

ellipsoid: set of the form

{x | (x � x

c

)TP�1(x � x

c

) 1}

with P 2 S

n

++ (i.e., P symmetric positive definite)

Euclidean balls and ellipsoids

(Euclidean) ball with center xc and radius r:

B(xc, r) = {x | ∥x − xc∥2 ≤ r} = {xc + ru | ∥u∥2 ≤ 1}

ellipsoid: set of the form

{x | (x − xc)TP−1(x − xc) ≤ 1}

with P ∈ Sn++ (i.e., P symmetric positive definite)

PSfrag replacementsxc

other representation: {xc + Au | ∥u∥2 ≤ 1} with A square and nonsingular

Convex sets 2–7

other representation: {xc

+ Au | kuk2 1} with A square andnonsingular


Norm balls and norm cones

norm: a function k · k that satisfies

I kxk � 0; kxk = 0 if and only if x = 0

I ktxk = |t| · kxk for t 2 R

I kx + yk kxk+ kyknotation: k · k is a general (unspecified) norm; k · ksymb is aparticular normnorm ball with center x

c

and radius r : {x | kx � x

c

k r}

norm cone:

{(x , t) | kxk t} ⇢ R

n+1

Euclidean norm cone is calledsecond-order cone

Norm balls and norm cones

norm: a function ∥ · ∥ that satisfies

• ∥x∥ ≥ 0; ∥x∥ = 0 if and only if x = 0

• ∥tx∥ = |t| ∥x∥ for t ∈ R

• ∥x + y∥ ≤ ∥x∥ + ∥y∥

notation: ∥ · ∥ is general (unspecified) norm; ∥ · ∥symb is particular norm

norm ball with center xc and radius r: {x | ∥x − xc∥ ≤ r}

norm cone: {(x, t) | ∥x∥ ≤ t}

Euclidean norm cone is called second-order cone

PSfrag replacements

x1x2

t

−10

1

−1

0

10

0.5

1

norm balls and cones are convex

Convex sets 2–8

norm balls and cones are convex


Operations that preserve convexity

practical methods for establishing convexity of a set C

1. apply definition

x1, x2 2 C , 0 ✓ 1 =) ✓x1 + (1� ✓)x2 2 C

2. show that C is obtained from simple convex sets(hyperplanes, halfspaces, norm balls, . . . ) by operations thatpreserve convexity

I Cartesian product (S1 ⇥ S2 = {(x1, x2) | x1 2 S1, x2 2 S2}; ifS1 and S2 are convex, so is S1 ⇥ S2)

I intersectionI a�ne functionsI perspective functionI linear-fractional functions


Intersectionthe intersection of (any number of) convex sets is convex

example:

S = {x 2 R

m | |px

(t)| 1 for |t| ⇡/3}

where p

x

(t) = x1 cos t + x2 cos 2t + · · ·+ x

m

cosmt

for m = 2:

Intersection

the intersection of (any number of) convex sets is convex

example:S = {x ∈ Rm | |p(t)| ≤ 1 for |t| ≤ π/3}

where p(t) = x1 cos t + x2 cos 2t + · · · + xm cosmt

for m = 2:

PSfrag replacements

0 π/3 2π/3 π

−1

0

1

t

p(t

)

PSfrag replacements

x1

x2 S

−2 −1 0 1 2−2

−1

0

1

2

Convex sets 2–12S

t

= {x 2 R

m | �1 (cos t, . . . , cosmt)T x 1}; S = \|t|⇡/3St .


A�ne functionsuppose f : Rn ! R

m is a�ne (f (x) = Ax + b with A 2 R

m⇥n,b 2 R

m)I the image of a convex set under f is convex

S ✓ R

n convex =) f (S) = {f (x) | x 2 S} ✓ R

m convex

Iexamples: scaling, translation, projection, sum of sets

I the inverse image f

�1(C ):

f

�1(C ) = {x 2 R

n | f (x) 2 C}

the inverse image of a convex set under f is convex

C ✓ R

m convex =) f

�1(C ) convex

Iexample: solution set of linear matrix inequality{x | x1A1 + · · ·+ x

m

A

m

� B} (with A

i

,B 2 S

p)I

example: hyperbolic cone {x | xTPx (cT x)2, cT x � 0}(with P 2 S

n

+)IOE 611: Nonlinear Programming, Fall 2017 2. Convex sets Page 2–14

Perspective and linear-fractional function

perspective function P : Rn+1 ! R

n:

P(x , t) = x/t, domP = {(x , t) | t > 0}

images and inverse images of convex sets under perspective areconvex

linear-fractional function f : Rn ! R

m:

f (x) =Ax + b

c

T

x + d

, dom f = {x | cT x + d > 0}

images and inverse images of convex sets under linear-fractionalfunctions are convex


example of a linear-fractional function

f (x) =1

x1 + x2 + 1x : R2 ! R

2

example of a linear-fractional function

f(x) =1

x1 + x2 + 1x

PSfrag replacements

x1

x2 C

−1 0 1−1

0

1

PSfrag replacements

x1

x2

f(C)

−1 0 1−1

0

1

Convex sets 2–15


Separating hyperplane theorem

if C and D are disjoint convex sets, then there exists a 6= 0, b suchthat

a

T

x b for x 2 C , a

T

x � b for x 2 D

Separating hyperplane theorem

if C and D are disjoint convex sets, then there exists a ̸= 0, b such that

aTx ≤ b for x ∈ C, aTx ≥ b for x ∈ D

PSfrag replacementsD

C

a

aTx ≥ b aTx ≤ b

the hyperplane {x | aTx = b} separates C and D

strict separation requires additional assumptions (e.g., C is closed, D is asingleton)

Convex sets 2–19

the hyperplane {x | aT x = b} separates C and D

strict separation (aT x < b for x 2 C , a

T

x > b for x 2 D)requires additional assumptions (e.g., C is closed, D is a singleton;see example 2.20)


Supporting hyperplane theorem

supporting hyperplane to set C at boundary point x0:

{x | aT x = a

T

x0}

where a 6= 0 and a

T

x a

T

x0 for all x 2 C

Supporting hyperplane theorem

supporting hyperplane to set C at boundary point x0:

{x | aTx = aTx0}

where a ̸= 0 and aTx ≤ aTx0 for all x ∈ C

PSfrag replacements C

a

x0

supporting hyperplane theorem: if C is convex, then there exists asupporting hyperplane at every boundary point of C

Convex sets 2–20

supporting hyperplane theorem: if C is convex, then thereexists a supporting hyperplane at every boundary point of C


A theorem of alternative for linear systems

Theorem Given A 2 R

m⇥n and b 2 R

m, exactly one of

(i) Ax < b and (ii) AT� = 0, b

T� 0, � � 0, � 6= 0

has a solution.Proof

I Easy to see (i) and (ii) can’t both have solutions.

I If (i) does not have a solution, C = {b � Ax | x 2 R

n} andD = {y 2 R

m

++} are convex disjoint sets and can be separated.


Generalized inequalities

a convex cone K ✓ R

n is a proper cone if

IK is closed (contains its boundary)

IK is solid (has nonempty interior)

IK is pointed (contains no line)

examples

I nonnegative orthantK = R

n

+ = {x 2 R

n | xi

� 0, i = 1, . . . , n}I positive semidefinite cone K = S

n

+

I nonnegative polynomials of degree n � 1 on [0, 1]:

K = {x 2 R

n | x1+x2t+x3t2+ · · ·+x

n

t

n�1 � 0 for t 2 [0, 1]}


Generalized inequalitiesa proper cone K can be used to define a generalized inequality:

x �K

y () y�x 2 K , x �K

y () y�x 2 intK

examples

I componentwise inequality (K = R

n

+)

x �R

n

+y () x

i

y

i

, i = 1, . . . , n

I matrix inequality (K = S

n

+)

X �S

n

+Y () Y � X positive semidefinite

these two types are so common that we drop the subscript in �K

properties: many properties of �K

are similar to on R, e.g.,

x �K

y , u �K

v =) x + u �K

y + v


Minimum and minimal elements�

K

is not in general a linear ordering: we can have x 6�K

y andy 6�

K

x

x 2 S is the minimum element of S with respect to �K

if

y 2 S =) x �K

y (equivalently, S ✓ x + K ).

x 2 S is a minimal element of S with respect to �K

if

y 2 S , y �K

x =) y = x (equivalently, (x�K )\S = {x}).

example (K = R

2+)

x1 is the minimum element of S1x2 is a minimal element of S2

Minimum and minimal elements

≼K is not in general a linear ordering : we can have x ̸≼K y and y ̸≼K x

x ∈ S is the minimum element of S with respect to ≼K if

y ∈ S =⇒ x ≼K y

x ∈ S is a minimal element of S with respect to ≼K if

y ∈ S, y ≼K x =⇒ y = x

example (K = R2+)

x1 is the minimum element of S1

x2 is a minimal element of S2

PSfrag replacements

x1

x2S1

S2


optimal production frontier

I di↵erent production methods use di↵erent amounts ofresources x 2 R

n

I production set P : resource vectors x for all possibleproduction methods

I e�cient (Pareto optimal) methods correspond to resourcevectors x that are minimal w.r.t. Rn

+

example (n = 2)x1, x2, x3 are e�cient; x4, x5 are not

optimal production frontier

• different production methods use different amounts of resources x ∈ Rn

• production set P : resource vectors x for all possible production methods

• efficient (Pareto optimal) methods correspond to resource vectors xthat are minimal w.r.t. Rn

+

example (n = 2)

x1, x2, x3 are efficient; x4, x5 are not

PSfrag replacements

x4x2

x1

x5

x3λ

P

labor

fuel


Dual cones and generalized inequalities

dual cone of a cone K :

K

⇤ = {y | hx , yi � 0 for all x 2 K}

examples

IK = R

n

+: K⇤ = R

n

+ (for x , y 2 R

n, hx , yi = y

T

x)

IK = S

n

+: K⇤ = S

n

+ (for X ,Y 2 S

n,hX ,Y i =

Pi ,j Xij

Y

ij

= tr(XY ))

IK = {(x , t) | kxk2 t}: K ⇤ = {(y , s) | kyk2 s}

IK = {(x , t) | kxk1 t}: K ⇤ = {(y , s) | kyk1 s}

first three examples are self-dual cones

dual cones of proper cones are proper, hence define generalizedinequalities:

y ⌫K

⇤ 0 () y

T

x � 0 for all x ⌫K

0


Minimum and minimal elements via dual inequalities

minimum element w.r.t. �K

x is minimum element of S i↵ forall � �

K

⇤ 0, x is the uniqueminimizer of �T

z over S


minimum element w.r.t. ≼K

x is minimum element of S iff for allλ ≻K∗ 0, x is the unique minimizerof λTz over S

PSfrag replacements x

S

minimal element w.r.t. ≽K

• if x minimizes λTz over S for some λ ≻K∗ 0, then x is minimal

PSfrag replacements

Sx1

x2

λ1

λ2

• if x is a minimal element of a convex set S, then there exists a nonzeroλ ≽K∗ 0 such that x minimizes λTz over S

Convex sets 2–22

minimal element w.r.t. �K

I if x minimizes �T

z over S for some � �K

⇤ 0, then x isminimal


minimum element w.r.t. ≼K

x is minimum element of S iff for allλ ≻K∗ 0, x is the unique minimizerof λTz over S

PSfrag replacements x

S

minimal element w.r.t. ≽K

• if x minimizes λTz over S for some λ ≻K∗ 0, then x is minimal

PSfrag replacements

Sx1

x2

λ1

λ2

• if x is a minimal element of a convex set S, then there exists a nonzeroλ ≽K∗ 0 such that x minimizes λTz over S

Convex sets 2–22

I if x is a minimal element of a convex set S , then there existsa nonzero � ⌫

K

⇤ 0 such that x minimizes �T

z over SIOE 611: Nonlinear Programming, Fall 2017 2. Convex sets Page 2–25

convex sets - university of michiganmepelman/teaching/ioe611/handouts/sets.pdfane set: contains the...

Documents