ties483 nonlinear optimization - jyväskylän...
TRANSCRIPT
General information
Master (/PhD) level course in mathematical information technology, 5 credits
Mandatory for master students in computational sciences
https://korppi.jyu.fi/kotka/r.jsp?course=148416
Homepage: http://users.jyu.fi/~jhaka/opt/
On Mondays at 14.15 and on Wednesdays at 12.15, January 13th – March 5th, 2014
Room: AgC 231
Mailing list: [email protected]
Contents
Introduction to nonlinear optimization
Optimization for a single variable (line search)
Unconstrained optimization: optimality
conditions and methods
Optimization with constraints: optimality
conditions and methods
Optimization software
Introduction to multiobjective optimization
Solving optimization problems in practice
Learning outcomes
Ability to identify different types of optimization problems
Understand basic concepts in solving nonlinear optimization problems
Understand optimality conditions for unconstrained and constrained optimization problems and be able to apply them in verifying the optimality of a solution
Understand basics of choosing and implementing optimization methods
Know how to find and apply software for solving nonlinear optimization problems
Understand differences in solving convex and non-convex optimization problems
Recognize the basics of solving multiobjective optimization problems
Passing the course
Course consists of – Lectures
– Quizzes (at the end of Wednesday lectures)
– Programming assignment (in pairs)
– Demos (6): need to reserve a weekly time for feedback
Grading – Demos: 40%
– Programming assignment: 40%
– Quizzes: 20%
How demanding the course is?
5 credits → 5*26 = 130 hours of work
– Lectures: 16*2 = 32 hours
– Programming assignment: 50/2 = 25 hours
– Demos: 6*3 = 18 hours
– Self study: 130 - 32 – 25 - 18 = 55 hours
Study practices in the course
I will inform you about the topics of the coming
week’s lectures in advance
You should study the topics before coming to
the lecture because
– There are discussion about the topics in smaller
groups during the lectures (peer-support)
– Helps you in asking clarifying questions
– Help at initiating discussion during the lectures
Let’s introduce ourselves
Who are you and what are you studying?
Master or PhD student?
Previous experiences of optimization?
How is this course related to your
studies/research?
What are your expectations about this course?
Basic mathematical concepts
Vectors: 𝑥 = (𝑥1, … , 𝑥𝑛) ∈ 𝑅𝑛, components are
denoted by subscripts
– Superscripts denote different vectors: 𝑥1, 𝑥2 ∈ 𝑅𝑛
(Euclidean) norm: 𝑥 = ( 𝑥𝑖2𝑛
𝑖=1 )1/2
Distance between vectors: 𝑑 𝑥, 𝑦 = 𝑥 − 𝑦
Basic mathematical concepts (cont.)
Convex sets: a set 𝑆 is convex if for all 𝑥, 𝑦 ∈ 𝑆
and 𝜆 ∈ 0,1 → 𝜆𝑥 + 1 − 𝜆 𝑦 ∈ 𝑆
Convex combinations of vectors: vector 𝑥 ∈ 𝑅𝑛
is a convex combination of vectors 𝑥1, … , 𝑥𝑝 ∈𝑅𝑛 if there exist multipliers 𝜆1, … , 𝜆𝑝 ≥ 0 s.t.
𝜆𝑖𝑝𝑖=1 = 1 and x = 𝜆𝑖𝑥
𝑖𝑝𝑖=1
– If 𝜆𝑖 ∈ 𝑅 then 𝑥 is a linear combination
Basic mathematical concepts (cont.)
Linearly independent vectors: a set of vectors
is linearly independent if none of the vectors
can be represented as a linear combination of
the others
– If the set of vectors is not linearly independent, then
it is linearly dependent (some vector is a linear
combination of the others)
For example, the basis of an Euclidean space
𝑅𝑛 is linearly independent
Basic mathematical concepts (cont.)
Unimodal functions: function 𝑓: 𝑅 → 𝑅 is unimodal in 𝑎, 𝑏 if for
some 𝑥∗ ∈ 𝑎, 𝑏 it is true that 𝑓(𝑥) is strictly decreasing in [𝑎, 𝑥∗) and strictly increasing in (𝑥∗, 𝑏].
Convex functions: Function 𝑓: 𝑆 → 𝑅 is convex if for all 𝑥, 𝑦 ∈ 𝑆
and 𝜆 ∈ [0,1]: 𝑓 𝜆𝑥 + 1 − 𝜆 𝑦 ≤ 𝜆𝑓 𝑥 + 1 − 𝜆 𝑓(𝑦)
𝑥 𝑦
convex convex non-convex
What is optimization?
”Scientific approach to decision making” –
Prof. Saul I. Gass
Searching for the best solution with respect to
given constraints
Enables systematic search of the best solution
(cf. trial and error)
Examples of practical optimization
Process design and optimization
Optimal shape design
Portfolio optimization
Route optimization in logistics
Supply chain management
etc.
Optimization problem
Objective function (cost function)
= measure for the goodness of the solution
Variables (decision, design, ...)
= values change the solution
Constraints (equality, inequality)
= define feasible solutions
Feasible region = all the constraints are satisfied
Parameters = values don’t change during
optimization (cf. variables)
Mathematical formulation
• Feasible region
• Optimality: find 𝑥∗ ∈ 𝑆 such that 𝑓 𝑥∗ ≤ 𝑓 𝑥 ∀ 𝑥 ∈ 𝑆
• Note: solutions of the optimization problems max 𝑓(𝑥) and min−𝑓(𝑥) are the same
Example1: mixing problem
Refinery produces 3 types of gasoline by mixing 3 different grude oil. Each grude oil can be purchased maximum of 5000 barrels per day. Let us assume that octane values and lead concentrations behave linearly in mixing. Refining costs are 4$ per barrel and the capacity of the refinery is 14000 barrels per day. Demand of gasoline can be increased by advertizing (demand grows 10 barrels per day for each $ used for advertizing).
Determine the production quantities of each type of gasoline, mixing ratios of different grude oil and the advertizing budget so that the daily profit is maximized.
Mixing problem
Gasoline1 Gasoline2 Gasoline3
Sale price 70 60 50
Lower limit for octane 10 8 6
Upper limit for lead 0.01 0.02 0.01
Demand 3000 2000 1000
Refining costs 4 4 4
Grude oil 1 Grude oil 2 Grude oil3
Purchase price 45 35 25
Octane value 12 6 8
Lead concentration 0.005 0.02 0.03
Availability 5000 5000 5000
Mixing problem
Variables: – 𝑥𝑖𝑗 = amount of grude oil 𝑖 used for producing gasoline 𝑗
– 𝑦𝑗 = the amount of money used for advertizing gasoline 𝑗
Net income:
– 𝑥11: 70 − 45 − 4 = 21
– 𝑥12: 60 − 45 − 4 = 11
– 𝑥13: 50 − 45 − 4 = 1
– 𝑥21: 70 − 35 − 4 = 31
– 𝑥22: 60 − 35 − 4 = 21
– 𝑥23: 50 − 35 − 4 = 11
– 𝑥31: 70 − 25 − 4 = 41
– 𝑥32: 60 − 25 − 4 = 31
– 𝑥33: 50 − 25 − 4 = 21
Mixing problem
Objective function:
– max 21𝑥11 + 11 𝑥12 + 𝑥13 + 31𝑥21 + 21𝑥22 +11𝑥23 + 41𝑥31 + 31𝑥32 + 21𝑥33 − 𝑦1 − 𝑦2 − 𝑦3
Nonnegativity:
– 𝑥𝑖𝑗 ≥ 0 ∀ 𝑖, 𝑗 𝑎𝑛𝑑 𝑦𝑗 ≥ 0 ∀ 𝑗
Capacity:
– 𝑥𝑖𝑗3𝑗=1
3𝑖=1 ≤ 14000
Mixing problem
Demands:
– Gasoline 1: 𝑥11 + 𝑥21 + 𝑥31 = 3000 + 10𝑦1
– Gasoline 2: 𝑥12 + 𝑥22 + 𝑥32 = 2000 + 10𝑦2
– Gasoline 3: 𝑥13 + 𝑥23 + 𝑥33 = 1000 + 10𝑦3
Availabilities:
– Grude oil 1: 𝑥11 + 𝑥21 + 𝑥31 ≤ 5000
– Grude oil 2: 𝑥12 + 𝑥22 + 𝑥32 ≤ 5000
– Grude oil 3: 𝑥13 + 𝑥23 + 𝑥33 ≤ 5000
Mixing problem
Octane values:
– Gasoline 1: 12𝑥11+ 6𝑥21+8𝑥31𝑥11+ 𝑥21+𝑥31
≥ 10
– Gasoline 2: 12𝑥12+ 6𝑥22+8𝑥32𝑥12+ 𝑥22+𝑥32
≥ 8
– Gasoline 3: 12𝑥13+ 6𝑥23+8𝑥33𝑥13+ 𝑥23+𝑥33
≥ 6
Lead concentrations:
– Gasoline 1: 0.005𝑥11+ 0.02𝑥21+0.03𝑥31
𝑥11+ 𝑥21+𝑥31 ≤ 0.01
– Gasoline 2: 0.005𝑥12+ 0.02𝑥22+0.03𝑥32
𝑥12+ 𝑥22+𝑥32 ≤ 0.02
– Gasoline 3: 0.005𝑥13+ 0.02𝑥23+0.03𝑥33
𝑥13+ 𝑥23+𝑥33 ≤ 0.01
Example 2: Water allocation
Water allocation
Papermaking process consumes lots of water
Water can be circulated and reused in different parts of the process as long as it remains fresh enough
Fresh water costs
The aim is to minimize the amount of fresh water required by the process
How to formulate the optimization problem?
Water allocation
Objective function: minimize the amount of
fresh water used
Constraints:
– water used should be fresh enough
– Energy and mass balances between the
different unit processes (requires a process
model)
Can not (usually) be formulated explicitly
but requires e.g. the use of a process
simulation software
Different types of optimization
problems
Linear = all functions are linear
Nonlinear = at least one function is nonlinear
Continuous = variables real-valued
Discrete = only finite (or countable) number of
possible values for the variables
Stochastic = problem contains uncertainties
Multiobjective = multiple objective functions
Different types of optimization
problems
Unconstrained = all values of the variables are
feasible
Box constraints = variables have upper and
lower bounds
Linear constraints = feasible region is convex
polyhedron
Nonlinear constraints = feasible region can be
anything
Local vs. global optima
-5 -4 -3 -2 -1 0 1 2
-1.5
-1
-0.5
0
0.5
1
1.5
2
minimize sin(x2+x)+cos(3x) -5≤x≤2
global minimum
local minima
Local vs. global optima
-4
-2
0
2
4
-4
-2
0
2
-1
0
1
2
-4
-2
0
2
4
Only two variables… → curse of dimensionality
Material for nonlinear optimization
I have a bunch of books that can be loaned
Lecture material of other (famous) teachers
around the world (WWW)
Other types of material from WWW
Examples of optimization literature
P.E. Gill et al., Practical Optimization, 1981
M.S. Bazaraa et al., Nonlinear Programming: Theory and Algorithms, 1993
D.P. Bertsekas, Nonlinear Programming, 1995
S.S. Rao, Engineering Optimization: Theory and Practice, 1996
J. Nocedal, Numerical Optimization, 1999
A.R. Conn et al., Introduction to Derivative-Free Optimization, 2009
M. Hinze et al., Optimization with PDE Constraints, 2009
L.T. Biegler, Nonlinear Programming – Concepts, Algorithms, and Applications to Chemical Processes, 2010
Journals in optimization Applied Mathematics and Optimization
Computational Optimization and Applications
European Journal of Operational Research
Decision Support Systems
Journal of Global Optimization
Journal of Multi-Criteria Decision Analysis
Journal of Optimization Theory and Applications
Mathematical Programming
Omega
Operations Research
Optimization Letters
Optimization Methods and Software
Optimization
SIAM Journal on Control and Optimization
SIAM Journal on Optimization
Structural and Multidisciplinary Optimization
…
Examples of journals in application
areas
AIChE Journal
American Institute of Aeronautics and Astronautics
Applied Thermal Engineering
Computers & Chemical Engineering
Engineering Optimization
Engineering with Computers
Environmental Modelling & Software
Industrial & Engineering Chemistry Research
Journal of Environmental Engineering and Science
Optimization and Engineering
Water Science and Technology
…
Topic of the lecture on January 15th
Optimization for a single variable (line search)
Study this before the lecture!