lecture 2: convex sets - university of illinoisangelia/l2_sets.pdf · lecture 2 outline • review...

Lecture 2: Convex sets

August 28, 2008

Lecture 2

Outline

• Review basic topology in Rn

• Open Set and Interior

• Closed Set and Closure

• Dual Cone

• Convex set

• Cones

• Affine sets

• Half-Spaces, Hyperplanes, Polyhedra

• Ellipsoids and Norm Cones

• Convex, Conical, and Affine Hulls

• Simplex

• Verifying Convexity

Convex Optimization 1

Lecture 2

Topology Review

Let {xk} be a sequence of vectors in Rn

Def. The sequence {xk} ⊆ Rn converges to a vector x̂ ∈ Rn when

‖xk − x̂‖ tends to 0 as k →∞

• Notation: When {xk} converges to a vector x̂, we write xk → x̂

• The sequence {xk} converges x̂ ∈ Rn if and only if for each component

i: the i-th components of xk converge to the i-th component of x̂

|xik − x̂i| tends to 0 as k →∞


Lecture 2

Open Set and Interior

Let X ⊆ Rn be a nonempty set

Def. The set X is open if for every x ∈ X there is an open ball B(x, r) that

entirely lies in the set X, i.e.,

for each x ∈ X there is r > 0 s.th. for all z with ‖z − x‖ < r, we have z ∈ X

Def. A vector x0 is an interior point of the set X, if there is a ball B(x0, r)

contained entirely in the set X

Def. The interior of the set X is the set of all interior points of X, denoted

by∫(X)

• How is∫(X) related to X?

• Example X = {x ∈ R2 | x1 ≥ 0, x2 > 0}∫(X) = {x ∈ R2 | x1 > 0, x2 > 0}∫

(S) of a probability simplex S = {x ∈ Rn | x � 0, e′x = 1}

Th. For a convex set X, the interior∫(X) is also convex


Lecture 2

Closed Set

Def. The complement of a given set X ⊆ Rn is the set of all vectors that do

not belong to X:

the complement of X = {x ∈ Rn | x /∈ X} = Rn \X

Def. The set X is closed if its complement Rn \X is open

• Examples: Rn and ∅ (both are open and closed)

{x ∈ R2 | x1 ≥ 0, x2 > 0} is open or closed?

hyperplane, half-space, simplex, polyhedral sets?

• The intersection of any family of closed set is closed

• The union of a finite family of closed set is closed

• The sum of two closed sets is not necessarily closed• Example: C1 = {(x1, x2) | x1 = 0, x2 ∈ R}

C2 = {(x1, x2) | x1x2 ≥ 1, x1 ≥ 0}C1 + C2 is not closed!

• Fact: The sum of a compact set and a closed set is closed


Lecture 2

Closure


Def. A vector x̂ is a closure point of a set X if there exists a sequence

{xk} ⊆ X such that xk → x̂

Closure points of X = {(−1)n/n | n = 1,2, . . .}, X̂ = {1− x | x ∈ X}?

• Notation: The set of closure points of X is denoted by cl(X)

• What is relation between X and cl(X)?

Th. A set is closed if and only if it contains its closure points, i.e.,

X is closed iff cl(X) ⊂ X

Th. For a convex set, the closure cl(X) is convex


Lecture 2

Boundary


Def. The boundary of the set X is the set of closure points for both the set

X and its complement Rn \X, i.e.,

bd(X) = cl(X) ∩ cl(Rn \X)

• Example X = {x ∈ Rn | g1(x) ≤ 0, . . . , gm(x) ≤ 0}. Is X closed?

What constitutes the boundary of X?


Lecture 2

Dual Cone

Let K be a nonempty cone in Rn

Def. The dual cone of K is the set K∗ defined by

K∗ = {z | z′x ≥ 0 for all x ∈ K}

• The dual cone K∗ is a closed convex cone even when K is neither closed

nor convex

• Let S be a subspace. Then, S∗ = S⊥.

• Let C be a closed convex cone. Then, (C∗)∗ = C.

• For an arbitrary cone K, we have (K∗)∗ = cl(conv(K)).


Lecture 2

Convex set

• A line segment defined by vectors x and y is

the set of points of the form αx + (1− α)y for α ∈ [0,1]

• A set C ⊂ Rn is convex when, with any two vectors x and y that belong

to the set C, the line segment connecting x and y also belongs to C


Lecture 2

Examples

Which of the following sets are convex?

• The space Rn

• A line through two given vectors x and y

l(x, y) = {z | z = x + t(y − x), t ∈ R}• A ray defined by a vector x

{z | z = λx, λ ≥ 0}

• The positive orthant {x ∈ Rn | x � 0} (� componentwise inequality)• The set {x ∈ R2 | x1 > 0, x2 ≥ 0}• The set {x ∈ R2 | x1x2 = 0}


Lecture 2

Cone

A set C ⊂ Rn is a cone when, with every vector x ∈ C, the ray {λx | λ ≥ 0}belongs to the set C

• A cone may or may not be convex• Examples: {x ∈ Rn | x � 0} {x ∈ R2 | x1x2 ≥ 0}

For a two sets C and S, the sum C + S is defined by

C + S = {z | z = x + y, x ∈ C, y ∈ S} (the order does nor matter)

Convex Cone Lemma: A cone C is convex if and only if C + C ⊆ C

Proof: Pick any x and y in C, and any α ∈ [0,1]. Then, αx and (1−α)y

belong to C because... . Using C +C ⊆ C, it follows that ... Reverse: Let

C be convex cone, and pick any x, y ∈ C. Consider 1/2(x + y)...


Lecture 2

Affine Set

A set C ⊂ Rn is a affine when, with every two distinct vectors x, y ∈ C,

the line {x + t(y − x) | t ∈ R} belongs to the set C

• An affine set is always convex

• A subspace is an affine set

A set C is affine if and only if C is a translated subspace, i.e.,

C = S + x0 for some subspace S and some x0 ∈ C

Dimension of an affine set C is the dimension of the subspace S


Lecture 2

Hyperplanes and Half-spaces

Hyperplane is a set of the form {x | a′x = b} for a nonzero vector a

Half-space is a set of the form {x | a′x ≤ b} with a nonzero vector a

The vector a is referred to as the normal vector

• A hyperplane in Rn divides the space into two half-spaces

{x | a′x ≤ b} and {x | a′x ≥ b}• Half-spaces are convex

• Hyperplanes are convex and affine


Lecture 2

Polyhedral Sets

A polyhedral set is given by finitely many linear inequalities

C = {x | Ax � b} where A is an m× n matrix

• Every polyhedral set is convex• Linear Problem

minimize c′x

subject to Bx ≤ b, Dx = d

The constraint set {x | Bx ≤ b, Dx = d} is polyhedral.


Lecture 2

Ellipsoids

Let A be a square (n× n) matrix.

• A is positive semidefinite when x′Ax ≥ 0 for all x ∈ Rn

• A is positive definite when x′Ax > 0 for all x ∈ Rn, x 6= 0

An ellipsoid is a set of the form

E = {x | (x− x0)′P−1(x− x0) ≤ 1}where P is symmetric and positive definite

• x0 is the center of the ellipsoid E

• A ball {x | ‖x− x0‖ ≤ r} is a special case of the ellipsoid (P = r2I)

• Ellipsoids are convex


Lecture 2

Norm Cones

A norm cone is the set of the form

C = {(x, t) ∈ Rn × R | ‖x‖ ≤ t}

• The norm ‖ · ‖ can be any norm in Rn

• The norm cone for Euclidean norm is also known as ice-cream cone

• Any norm cone is convex


Lecture 2

Convex and Conical HullsA convex combination of vectors x1, . . . , xm is a vector of the form

α1x1 + . . . + αmxm αi ≥ 0 for all i and∑m

i=1 αi = 1

The convex hull of a set X is the set of all convex combinations of the

vectors in X, denoted conv(X)

A conical combination of vectors x1, . . . , xm is a vector of the form

λ1x1 + . . . + λmxm with λi ≥ 0 for all i

The conical hull of a set X is the set of all conical combinations of the

vectors in X, denoted by cone(X)


Lecture 2

Affine Hull

An affine combination of vectors x1, . . . , xm is a vector of the form

t1x1 + . . . + tmxm with∑m

i=1 ti = 1, ti ∈ R for all i

The affine hull of a set X is the set of all affine combinations of the vectors

in X, denoted aff(X)

The dimension of a set X is the dimension of the affine hull of X

dim(X) = dim(aff(X))


Lecture 2

SimplexA simplex is a set given as a convex combination of a finite collection of

vectors v0, v1, . . . , vm:

C = conv{v0, v1 . . . , vm}The dimension of the simplex C is equal to the maximum number of linearly

independent vectors among v1 − v0, . . . , vm − v0.

Examples

• Unit simplex {x ∈ Rn | x � 0, e′x ≤ 1}, e = (1, . . . ,1), dim -?

• Probability simplex {x ∈ Rn | x � 0, e′x = 1}, dim -?


Lecture 2

Practical Methods for Establishing Convexity of a Set

Establish the convexity of a given set X

• The set is one of the “recognizable” (simple) convex sets such as

polyhedral, simplex, norm cone, etc

• Prove the convexity by directly applying the definition

For every x, y ∈ X and α ∈ (0,1), show that αx + (1− α)y is also in X

• Show that the set is obtained from one of the simple convex sets through

an operation that preserves convexity


Lecture 2

Operations Preserving Convexity

Let C ⊆ Rn, C1 ⊆ Rn, C2 ⊆ Rn, and K ⊆ Rm be convex sets. Then, the

following sets are also convex:

• The intersection C1 ∩ C2 = {x | x ∈ C1 and x ∈ C2}

• The sum C1 + C2 of two convex sets

• The translated set C + a

• The scaled set tC = {tx | x ∈ C} for any t ∈ R• The Cartesian product C1 × C2 = {(x1, x2) | x1 ∈ C1, x2 ∈ C2}

• The coordinate projection {x1 | (x1, x2) ∈ C for some x2}

• The image AC under a linear transformation A : Rn 7→ Rm:

AC = {y ∈ Rm | y = Ax for some x ∈ C}

• The inverse image A−1K under a linear transformation A : Rn 7→ Rm:

A−1K = {x ∈ Rn | Ax ∈ K}


lecture 2: convex sets - university of illinoisangelia/l2_sets.pdf · lecture 2 outline • review...

Documents