convexity - relate.cs.illinois.edu
Post on 25-May-2022
13 Views
Preview:
TRANSCRIPT
Convexity
S ⊆ Rn is called convex if for all x , y ∈ S and all 0 ⩽ α ⩽ 1
f : S → R is called convex on S ⊆ Rn if for \ x , y ∈ S and all 0 ⩽ α ⩽ 1
Q: Give an example of a convex, but not strictly convex function.
179
Convexity: Consequences
If f is convex, . . .
If f is strictly convex, . . .
180
Optimality ConditionsIf we have found a candidate x
∗ for a minimum, how do we know it
actually is one? Assume f is smooth, i.e. has all needed derivatives.
181
Optimization: Observations
Q: Come up with a hypothetical approach for finding minima.
Q: Is the Hessian symmetric?
Q: How can we practically test for positive definiteness?
182
If ex O V Y
Yes CSchwartz th m Ey t Fax
Cholesky A L E
In-Class Activity: Optimization Theory
In-class activity: Optimization Theory
183
Sensitivity and Conditioning (1D)How does optimization react to a slight perturbation of the minimum?
184
I f xx fix a tot X is the true min
fifth Ighth't Hot
loft f ext h't stol
EX Ihle FIEF8 half as many
16
digits Fff
Sensitivity and Conditioning (nD)
How does optimization react to a slight perturbation of the minimum?
185
f x't th s fix it hxfex s
direction É s HAI S the11511 1
IN E Eminentconditioning depends on Hf
Unimodality
Would like a method like bisection, but for optimization.
In general: No invariant that can be preserved.
Need extra assumption.
186
Golden Section Search
Suppose we have an interval with f unimodal:
Would like to maintain unimodality.
187
Golden Section Search: Efficiency
Where to put x1, x2?
Convergence rate?
Demo: Golden Section Proportions [cleared]
188
Newton’s Method
Reuse the Taylor approximation idea, but for optimization.
Demo: Newton’s Method in 1D [cleared]
189
fix th a fix t f ath tf ex Ich
f approx f with f at ticmin I ch to get X.tl
I ch o f Xia f Xia h
h e fifty solving f ex o with Newton
quadratic
In-Class Activity: Optimization Methods
In-class activity: Optimization Methods
190
Steepest DescentGiven a scalar function f : R
n→ R at a point x , which way is down?
Demo: Steepest Descent [cleared]191
Direction of steepest descent ofline search e.g
Golden section
1 Xo init guess
2 Sk Pf Xk3 min f Xia task4 X reel Xk t ASK
Steepest Descent: ConvergenceConsider quadratic model problem:
f (x) =1
2xTAx + c
Tx
where A is SPD. (A good model of f near a minimum.)
192
If A Xtc
en Xk Xt then
Hertilla Fitted ÉiIleLinear convergence
I
top related