introduction to optimization - cermics – centre d...
TRANSCRIPT
![Page 1: Introduction to Optimization - CERMICS – Centre d ...cermics.enpc.fr/~delara/MABIES/pgajardo_optimization_IHP2013.pdf · Introduction Optimality conditions Introduction to Optimization](https://reader031.vdocuments.us/reader031/viewer/2022021903/5b9da90609d3f2de128ca4f5/html5/thumbnails/1.jpg)
Introduction
Optimalityconditions
Introduction to Optimization
Pedro Gajardo1 and Eladio Ocana2
1Universidad Tecnica Federico Santa Marıa - Valparaıso -Chile
2IMCA-FC. Universidad Nacional de Ingenierıa - Lima - Peru
Mathematics of Bio-Economics (MABIES) - IHP
January 29, 2013 - Paris
![Page 2: Introduction to Optimization - CERMICS – Centre d ...cermics.enpc.fr/~delara/MABIES/pgajardo_optimization_IHP2013.pdf · Introduction Optimality conditions Introduction to Optimization](https://reader031.vdocuments.us/reader031/viewer/2022021903/5b9da90609d3f2de128ca4f5/html5/thumbnails/2.jpg)
Introduction
Optimalityconditions
Contents
Introduction
Optimality conditions
![Page 3: Introduction to Optimization - CERMICS – Centre d ...cermics.enpc.fr/~delara/MABIES/pgajardo_optimization_IHP2013.pdf · Introduction Optimality conditions Introduction to Optimization](https://reader031.vdocuments.us/reader031/viewer/2022021903/5b9da90609d3f2de128ca4f5/html5/thumbnails/3.jpg)
Introduction
Optimalityconditions
Optimization problems
In order to formulate an optimization problem, the followingconcepts must be very clear:
decision variables
restrictions
objective function
![Page 4: Introduction to Optimization - CERMICS – Centre d ...cermics.enpc.fr/~delara/MABIES/pgajardo_optimization_IHP2013.pdf · Introduction Optimality conditions Introduction to Optimization](https://reader031.vdocuments.us/reader031/viewer/2022021903/5b9da90609d3f2de128ca4f5/html5/thumbnails/4.jpg)
Introduction
Optimalityconditions
Optimization problems
In order to formulate an optimization problem, the followingconcepts must be very clear:
decision variables
restrictions
objective function
![Page 5: Introduction to Optimization - CERMICS – Centre d ...cermics.enpc.fr/~delara/MABIES/pgajardo_optimization_IHP2013.pdf · Introduction Optimality conditions Introduction to Optimization](https://reader031.vdocuments.us/reader031/viewer/2022021903/5b9da90609d3f2de128ca4f5/html5/thumbnails/5.jpg)
Introduction
Optimalityconditions
Optimization problems
In order to formulate an optimization problem, the followingconcepts must be very clear:
decision variables
restrictions
objective function
![Page 6: Introduction to Optimization - CERMICS – Centre d ...cermics.enpc.fr/~delara/MABIES/pgajardo_optimization_IHP2013.pdf · Introduction Optimality conditions Introduction to Optimization](https://reader031.vdocuments.us/reader031/viewer/2022021903/5b9da90609d3f2de128ca4f5/html5/thumbnails/6.jpg)
Introduction
Optimalityconditions
Optimization problemsDecision variables
One or morevariableson whichwe can decide(harvestingrate or effort, level of investment, distribution of tasks,parameters)
Objective: to find thebestvalue for the decision variable
We denote byx the decision variable
x can be a number, a vector, a sequence, a function, ..........
In a general framework, we denote byX the set where thedecision variable is (x ∈ X)
![Page 7: Introduction to Optimization - CERMICS – Centre d ...cermics.enpc.fr/~delara/MABIES/pgajardo_optimization_IHP2013.pdf · Introduction Optimality conditions Introduction to Optimization](https://reader031.vdocuments.us/reader031/viewer/2022021903/5b9da90609d3f2de128ca4f5/html5/thumbnails/7.jpg)
Introduction
Optimalityconditions
Optimization problemsDecision variables
One or more variables on which we can decide (harvestingrate or effort, level of investment, distribution of tasks,parameters)
Objective:to find thebestvalue for the decision variable
We denote byx the decision variable
x can be a number, a vector, a sequence, a function, ..........
In a general framework, we denote byX the set where thedecision variable is (x ∈ X)
![Page 8: Introduction to Optimization - CERMICS – Centre d ...cermics.enpc.fr/~delara/MABIES/pgajardo_optimization_IHP2013.pdf · Introduction Optimality conditions Introduction to Optimization](https://reader031.vdocuments.us/reader031/viewer/2022021903/5b9da90609d3f2de128ca4f5/html5/thumbnails/8.jpg)
Introduction
Optimalityconditions
Optimization problemsDecision variables
One or more variables on which we can decide (harvestingrate or effort, level of investment, distribution of tasks,parameters)
Objective: to find thebestvalue for the decision variable
Wedenote byx the decision variable
x can be a number, a vector, a sequence, a function, ..........
In a general framework, we denote byX the set where thedecision variable is (x ∈ X)
![Page 9: Introduction to Optimization - CERMICS – Centre d ...cermics.enpc.fr/~delara/MABIES/pgajardo_optimization_IHP2013.pdf · Introduction Optimality conditions Introduction to Optimization](https://reader031.vdocuments.us/reader031/viewer/2022021903/5b9da90609d3f2de128ca4f5/html5/thumbnails/9.jpg)
Introduction
Optimalityconditions
Optimization problemsDecision variables
One or more variables on which we can decide (harvestingrate or effort, level of investment, distribution of tasks,parameters)
Objective: to find thebestvalue for the decision variable
We denote byx the decision variable
x can be a number, a vector, a sequence, a function, ..........
In a general framework, we denote byX the set where thedecision variable is (x ∈ X)
![Page 10: Introduction to Optimization - CERMICS – Centre d ...cermics.enpc.fr/~delara/MABIES/pgajardo_optimization_IHP2013.pdf · Introduction Optimality conditions Introduction to Optimization](https://reader031.vdocuments.us/reader031/viewer/2022021903/5b9da90609d3f2de128ca4f5/html5/thumbnails/10.jpg)
Introduction
Optimalityconditions
Optimization problemsDecision variables
One or more variables on which we can decide (harvestingrate or effort, level of investment, distribution of tasks,parameters)
Objective: to find thebestvalue for the decision variable
We denote byx the decision variable
x can be a number, a vector, a sequence, a function, ..........
In a general framework,we denote byX the set where thedecision variable is (x ∈ X)
![Page 11: Introduction to Optimization - CERMICS – Centre d ...cermics.enpc.fr/~delara/MABIES/pgajardo_optimization_IHP2013.pdf · Introduction Optimality conditions Introduction to Optimization](https://reader031.vdocuments.us/reader031/viewer/2022021903/5b9da90609d3f2de128ca4f5/html5/thumbnails/11.jpg)
Introduction
Optimalityconditions
Warning
This tutorial is onContinuous Optimization
That means that decision variables are continuous variables
What that means?
Firstly, the setX where the decision variable belongs, is a(linear) vector space
![Page 12: Introduction to Optimization - CERMICS – Centre d ...cermics.enpc.fr/~delara/MABIES/pgajardo_optimization_IHP2013.pdf · Introduction Optimality conditions Introduction to Optimization](https://reader031.vdocuments.us/reader031/viewer/2022021903/5b9da90609d3f2de128ca4f5/html5/thumbnails/12.jpg)
Introduction
Optimalityconditions
Warning
This tutorial is onContinuous Optimization
That means that decision variables are continuous variables
What that means?
Firstly, the setX where the decision variable belongs, is a(linear) vector space
![Page 13: Introduction to Optimization - CERMICS – Centre d ...cermics.enpc.fr/~delara/MABIES/pgajardo_optimization_IHP2013.pdf · Introduction Optimality conditions Introduction to Optimization](https://reader031.vdocuments.us/reader031/viewer/2022021903/5b9da90609d3f2de128ca4f5/html5/thumbnails/13.jpg)
Introduction
Optimalityconditions
Warning
This tutorial is onContinuous Optimization
That means that decision variables are continuous variables
What that means?
Firstly, the setX where the decision variable belongs, is a(linear) vector space
![Page 14: Introduction to Optimization - CERMICS – Centre d ...cermics.enpc.fr/~delara/MABIES/pgajardo_optimization_IHP2013.pdf · Introduction Optimality conditions Introduction to Optimization](https://reader031.vdocuments.us/reader031/viewer/2022021903/5b9da90609d3f2de128ca4f5/html5/thumbnails/14.jpg)
Introduction
Optimalityconditions
Warning
This tutorial is onContinuous Optimization
That means that decision variables are continuous variables
What that means?
Firstly, the setX where the decision variable belongs, is a(linear) vector space
![Page 15: Introduction to Optimization - CERMICS – Centre d ...cermics.enpc.fr/~delara/MABIES/pgajardo_optimization_IHP2013.pdf · Introduction Optimality conditions Introduction to Optimization](https://reader031.vdocuments.us/reader031/viewer/2022021903/5b9da90609d3f2de128ca4f5/html5/thumbnails/15.jpg)
Introduction
Optimalityconditions
Optimization problemsRestrictions
Constraints that the decision variable has to satisfy
If for a certain value of the decision variable therestrictions are satisfied, we say that it is a feasiblesolution
In a general framework, we denote byC ⊆ X the set of allfeasible solutions
If we have two decision variables,x1 andx2 and they haveto satisfy the following constraints:x1 ≥ 0, x2 ≥ 0,2 x1 + 3 x2 ≤ 5, we denote
C = {(x1, x2) ∈ R2 | x1 ≥ 0, x2 ≥ 0, 2 x1 + 3x2 ≤ 5}
![Page 16: Introduction to Optimization - CERMICS – Centre d ...cermics.enpc.fr/~delara/MABIES/pgajardo_optimization_IHP2013.pdf · Introduction Optimality conditions Introduction to Optimization](https://reader031.vdocuments.us/reader031/viewer/2022021903/5b9da90609d3f2de128ca4f5/html5/thumbnails/16.jpg)
Introduction
Optimalityconditions
Optimization problemsRestrictions
Constraints that the decision variable has to satisfy
If for a certain value of thedecision variable therestrictions are satisfied, we say that it is a feasiblesolution
In a general framework, we denote byC ⊆ X the set of allfeasible solutions
If we have two decision variables,x1 andx2 and they haveto satisfy the following constraints:x1 ≥ 0, x2 ≥ 0,2 x1 + 3 x2 ≤ 5, we denote
C = {(x1, x2) ∈ R2 | x1 ≥ 0, x2 ≥ 0, 2 x1 + 3x2 ≤ 5}
![Page 17: Introduction to Optimization - CERMICS – Centre d ...cermics.enpc.fr/~delara/MABIES/pgajardo_optimization_IHP2013.pdf · Introduction Optimality conditions Introduction to Optimization](https://reader031.vdocuments.us/reader031/viewer/2022021903/5b9da90609d3f2de128ca4f5/html5/thumbnails/17.jpg)
Introduction
Optimalityconditions
Optimization problemsRestrictions
Constraints that the decision variable has to satisfy
If for a certain value of the decision variable therestrictions are satisfied, we say that it is a feasiblesolution
In a general framework,we denote byC ⊆ X the set of allfeasible solutions
If we have two decision variables,x1 andx2 and they haveto satisfy the following constraints:x1 ≥ 0, x2 ≥ 0,2 x1 + 3 x2 ≤ 5, we denote
C = {(x1, x2) ∈ R2 | x1 ≥ 0, x2 ≥ 0, 2 x1 + 3x2 ≤ 5}
![Page 18: Introduction to Optimization - CERMICS – Centre d ...cermics.enpc.fr/~delara/MABIES/pgajardo_optimization_IHP2013.pdf · Introduction Optimality conditions Introduction to Optimization](https://reader031.vdocuments.us/reader031/viewer/2022021903/5b9da90609d3f2de128ca4f5/html5/thumbnails/18.jpg)
Introduction
Optimalityconditions
Optimization problemsRestrictions
Constraints that the decision variable has to satisfy
If for a certain value of the decision variable therestrictions are satisfied, we say that it is a feasiblesolution
In a general framework, we denote byC ⊆ X the set of allfeasible solutions
If we have two decision variables,x1 andx2 and they haveto satisfy the following constraints:x1 ≥ 0, x2 ≥ 0,2 x1 + 3 x2 ≤ 5, we denote
C = {(x1, x2) ∈ R2 | x1 ≥ 0, x2 ≥ 0, 2 x1 + 3x2 ≤ 5}
![Page 19: Introduction to Optimization - CERMICS – Centre d ...cermics.enpc.fr/~delara/MABIES/pgajardo_optimization_IHP2013.pdf · Introduction Optimality conditions Introduction to Optimization](https://reader031.vdocuments.us/reader031/viewer/2022021903/5b9da90609d3f2de128ca4f5/html5/thumbnails/19.jpg)
Introduction
Optimalityconditions
Warning
This tutorial is onContinuous Optimization
That means that decision variables are continuous variables
What that means?
Secondly, the setC of feasible solutions is not a discrete set (setof isolated points)
![Page 20: Introduction to Optimization - CERMICS – Centre d ...cermics.enpc.fr/~delara/MABIES/pgajardo_optimization_IHP2013.pdf · Introduction Optimality conditions Introduction to Optimization](https://reader031.vdocuments.us/reader031/viewer/2022021903/5b9da90609d3f2de128ca4f5/html5/thumbnails/20.jpg)
Introduction
Optimalityconditions
Warning
This tutorial is onContinuous Optimization
That means that decision variables are continuous variables
What that means?
Secondly, the setC of feasible solutions is not a discrete set (setof isolated points)
![Page 21: Introduction to Optimization - CERMICS – Centre d ...cermics.enpc.fr/~delara/MABIES/pgajardo_optimization_IHP2013.pdf · Introduction Optimality conditions Introduction to Optimization](https://reader031.vdocuments.us/reader031/viewer/2022021903/5b9da90609d3f2de128ca4f5/html5/thumbnails/21.jpg)
Introduction
Optimalityconditions
Optimization problemsObjective function
It is the mathematical representation formeasuring thegoodnessof values for the decision variable
This representation is done through a function
f : X −→ R
![Page 22: Introduction to Optimization - CERMICS – Centre d ...cermics.enpc.fr/~delara/MABIES/pgajardo_optimization_IHP2013.pdf · Introduction Optimality conditions Introduction to Optimization](https://reader031.vdocuments.us/reader031/viewer/2022021903/5b9da90609d3f2de128ca4f5/html5/thumbnails/22.jpg)
Introduction
Optimalityconditions
Optimization problemsObjective function
It is the mathematical representation for measuring thegoodnessof values for the decision variable
This representation is done through a function
f : X −→ R
![Page 23: Introduction to Optimization - CERMICS – Centre d ...cermics.enpc.fr/~delara/MABIES/pgajardo_optimization_IHP2013.pdf · Introduction Optimality conditions Introduction to Optimization](https://reader031.vdocuments.us/reader031/viewer/2022021903/5b9da90609d3f2de128ca4f5/html5/thumbnails/23.jpg)
Introduction
Optimalityconditions
Optimization problems
To find thebestvalue for the decision variable from all feasiblesolutions
To find x ∈ C ⊆ X such that
f (x) ≤ f (x) (or f (x) ≥ f (x)) for all x ∈ C
minx∈X
f (x)
(
or maxx∈X
f (x)
)
subject to
x ∈ C
![Page 24: Introduction to Optimization - CERMICS – Centre d ...cermics.enpc.fr/~delara/MABIES/pgajardo_optimization_IHP2013.pdf · Introduction Optimality conditions Introduction to Optimization](https://reader031.vdocuments.us/reader031/viewer/2022021903/5b9da90609d3f2de128ca4f5/html5/thumbnails/24.jpg)
Introduction
Optimalityconditions
Optimization problems
To find thebestvalue for the decision variable from all feasiblesolutions
To find x ∈ C ⊆ X such that
f (x) ≤ f (x) (or f (x) ≥ f (x)) for all x ∈ C
minx∈X
f (x)
(
or maxx∈X
f (x)
)
subject to
x ∈ C
![Page 25: Introduction to Optimization - CERMICS – Centre d ...cermics.enpc.fr/~delara/MABIES/pgajardo_optimization_IHP2013.pdf · Introduction Optimality conditions Introduction to Optimization](https://reader031.vdocuments.us/reader031/viewer/2022021903/5b9da90609d3f2de128ca4f5/html5/thumbnails/25.jpg)
Introduction
Optimalityconditions
Optimization problems
To find thebestvalue for the decision variable from all feasiblesolutions
To find x ∈ C ⊆ X such that
f (x) ≤ f (x) (or f (x) ≥ f (x)) for all x ∈ C
minx∈X
f (x)
(
or maxx∈X
f (x)
)
subject to
x ∈ C
![Page 26: Introduction to Optimization - CERMICS – Centre d ...cermics.enpc.fr/~delara/MABIES/pgajardo_optimization_IHP2013.pdf · Introduction Optimality conditions Introduction to Optimization](https://reader031.vdocuments.us/reader031/viewer/2022021903/5b9da90609d3f2de128ca4f5/html5/thumbnails/26.jpg)
Introduction
Optimalityconditions
Optimization problemsExample: maximizing a rectangular surface
With a given fence, to enclose the largest rectangular area
Objective: maximize the rectangular surface
Decision variables: lengths of the rectangular figure
Constraints:
positive lengths
length of the fence
![Page 27: Introduction to Optimization - CERMICS – Centre d ...cermics.enpc.fr/~delara/MABIES/pgajardo_optimization_IHP2013.pdf · Introduction Optimality conditions Introduction to Optimization](https://reader031.vdocuments.us/reader031/viewer/2022021903/5b9da90609d3f2de128ca4f5/html5/thumbnails/27.jpg)
Introduction
Optimalityconditions
Optimization problemsExample: maximizing a rectangular surface
With a given fence, to enclose the largest rectangular area
Objective: maximize the rectangular surface
Decision variables: lengths of the rectangular figure
Constraints:
positive lengths
length of the fence
![Page 28: Introduction to Optimization - CERMICS – Centre d ...cermics.enpc.fr/~delara/MABIES/pgajardo_optimization_IHP2013.pdf · Introduction Optimality conditions Introduction to Optimization](https://reader031.vdocuments.us/reader031/viewer/2022021903/5b9da90609d3f2de128ca4f5/html5/thumbnails/28.jpg)
Introduction
Optimalityconditions
Optimization problemsExample: maximizing a rectangular surface
With a given fence, to enclose the largest rectangular area
Objective: maximize the rectangular surface
Decision variables: lengths of the rectangular figure
Constraints:
positive lengths
length of the fence
![Page 29: Introduction to Optimization - CERMICS – Centre d ...cermics.enpc.fr/~delara/MABIES/pgajardo_optimization_IHP2013.pdf · Introduction Optimality conditions Introduction to Optimization](https://reader031.vdocuments.us/reader031/viewer/2022021903/5b9da90609d3f2de128ca4f5/html5/thumbnails/29.jpg)
Introduction
Optimalityconditions
Optimization problemsExample: maximizing a rectangular surface
With a given fence, to enclose the largest rectangular area
Objective: maximize the rectangular surface
Decision variables: lengths of the rectangular figure
Constraints:
positive lengths
length of the fence
![Page 30: Introduction to Optimization - CERMICS – Centre d ...cermics.enpc.fr/~delara/MABIES/pgajardo_optimization_IHP2013.pdf · Introduction Optimality conditions Introduction to Optimization](https://reader031.vdocuments.us/reader031/viewer/2022021903/5b9da90609d3f2de128ca4f5/html5/thumbnails/30.jpg)
Introduction
Optimalityconditions
Optimization problemsExample: maximizing a rectangular surface
With a given fence, to enclose the largest rectangular area
Objective: maximize the rectangular surface
Decision variables: lengths of the rectangular figure
Constraints:
positive lengths
length of the fence
![Page 31: Introduction to Optimization - CERMICS – Centre d ...cermics.enpc.fr/~delara/MABIES/pgajardo_optimization_IHP2013.pdf · Introduction Optimality conditions Introduction to Optimization](https://reader031.vdocuments.us/reader031/viewer/2022021903/5b9da90609d3f2de128ca4f5/html5/thumbnails/31.jpg)
Introduction
Optimalityconditions
Optimization problemsExample: maximizing a rectangular surface
Decision variables: a (height) andb (width)
Objective: maximizes(a,b) = a b
Constraints:positive lengths:a > 0, b > 0
length of fence (L > 0): 2(a+ b) = L
b = L2 − a
New objective functionf (a) = a(
L2 − a
)
Rewrite restrictions:a > 0, a < L2
![Page 32: Introduction to Optimization - CERMICS – Centre d ...cermics.enpc.fr/~delara/MABIES/pgajardo_optimization_IHP2013.pdf · Introduction Optimality conditions Introduction to Optimization](https://reader031.vdocuments.us/reader031/viewer/2022021903/5b9da90609d3f2de128ca4f5/html5/thumbnails/32.jpg)
Introduction
Optimalityconditions
Optimization problemsExample: maximizing a rectangular surface
Decision variables:a (height) andb (width)
Objective: maximizes(a,b) = a b
Constraints:positive lengths:a > 0, b > 0
length of fence (L > 0): 2(a+ b) = L
b = L2 − a
New objective functionf (a) = a(
L2 − a
)
Rewrite restrictions:a > 0, a < L2
![Page 33: Introduction to Optimization - CERMICS – Centre d ...cermics.enpc.fr/~delara/MABIES/pgajardo_optimization_IHP2013.pdf · Introduction Optimality conditions Introduction to Optimization](https://reader031.vdocuments.us/reader031/viewer/2022021903/5b9da90609d3f2de128ca4f5/html5/thumbnails/33.jpg)
Introduction
Optimalityconditions
Optimization problemsExample: maximizing a rectangular surface
Decision variables:a (height) andb (width)
Objective: maximizes(a,b) = a b
Constraints:positive lengths:a > 0, b > 0
length of fence (L > 0): 2(a+ b) = L
b = L2 − a
New objective functionf (a) = a(
L2 − a
)
Rewrite restrictions:a > 0, a < L2
![Page 34: Introduction to Optimization - CERMICS – Centre d ...cermics.enpc.fr/~delara/MABIES/pgajardo_optimization_IHP2013.pdf · Introduction Optimality conditions Introduction to Optimization](https://reader031.vdocuments.us/reader031/viewer/2022021903/5b9da90609d3f2de128ca4f5/html5/thumbnails/34.jpg)
Introduction
Optimalityconditions
Optimization problemsExample: maximizing a rectangular surface
Decision variables:a (height) andb (width)
Objective: maximizes(a,b) = a b
Constraints:positive lengths:a > 0, b > 0
length of fence (L > 0): 2(a+ b) = L
b = L2 − a
New objective functionf (a) = a(
L2 − a
)
Rewrite restrictions:a > 0, a < L2
![Page 35: Introduction to Optimization - CERMICS – Centre d ...cermics.enpc.fr/~delara/MABIES/pgajardo_optimization_IHP2013.pdf · Introduction Optimality conditions Introduction to Optimization](https://reader031.vdocuments.us/reader031/viewer/2022021903/5b9da90609d3f2de128ca4f5/html5/thumbnails/35.jpg)
Introduction
Optimalityconditions
Optimization problemsExample: maximizing a rectangular surface
Decision variables:a (height) andb (width)
Objective: maximizes(a,b) = a b
Constraints:positive lengths:a > 0, b > 0
length of fence (L > 0): 2(a+ b) = L
b = L2 − a
New objective functionf (a) = a(
L2 − a
)
Rewrite restrictions:a > 0, a < L2
![Page 36: Introduction to Optimization - CERMICS – Centre d ...cermics.enpc.fr/~delara/MABIES/pgajardo_optimization_IHP2013.pdf · Introduction Optimality conditions Introduction to Optimization](https://reader031.vdocuments.us/reader031/viewer/2022021903/5b9da90609d3f2de128ca4f5/html5/thumbnails/36.jpg)
Introduction
Optimalityconditions
Optimization problemsExample: maximizing a rectangular surface
Decision variables:a (height) andb (width)
Objective: maximizes(a,b) = a b
Constraints:positive lengths:a > 0, b > 0
length of fence (L > 0): 2(a+ b) = L
b = L2 − a
New objective functionf (a) = a(
L2 − a
)
Rewrite restrictions:a > 0, a < L2
![Page 37: Introduction to Optimization - CERMICS – Centre d ...cermics.enpc.fr/~delara/MABIES/pgajardo_optimization_IHP2013.pdf · Introduction Optimality conditions Introduction to Optimization](https://reader031.vdocuments.us/reader031/viewer/2022021903/5b9da90609d3f2de128ca4f5/html5/thumbnails/37.jpg)
Introduction
Optimalityconditions
Optimization problemsExample: maximizing a rectangular surface
Decision variables:a (height) andb (width)
Objective: maximizes(a,b) = a b
Constraints:positive lengths:a > 0, b > 0
length of fence (L > 0): 2(a+ b) = L
b = L2 − a
Newobjective functionf (a) = a(
L2 − a
)
Rewrite restrictions:a > 0, a < L2
![Page 38: Introduction to Optimization - CERMICS – Centre d ...cermics.enpc.fr/~delara/MABIES/pgajardo_optimization_IHP2013.pdf · Introduction Optimality conditions Introduction to Optimization](https://reader031.vdocuments.us/reader031/viewer/2022021903/5b9da90609d3f2de128ca4f5/html5/thumbnails/38.jpg)
Introduction
Optimalityconditions
Optimization problemsExample: maximizing a rectangular surface
Decision variables:a (height) andb (width)
Objective: maximizes(a,b) = a b
Constraints:positive lengths:a > 0, b > 0
length of fence (L > 0): 2(a+ b) = L
b = L2 − a
New objective functionf (a) = a(
L2 − a
)
Rewriterestrictions:a > 0, a < L2
![Page 39: Introduction to Optimization - CERMICS – Centre d ...cermics.enpc.fr/~delara/MABIES/pgajardo_optimization_IHP2013.pdf · Introduction Optimality conditions Introduction to Optimization](https://reader031.vdocuments.us/reader031/viewer/2022021903/5b9da90609d3f2de128ca4f5/html5/thumbnails/39.jpg)
Introduction
Optimalityconditions
Optimization problemsExample: maximizing a rectangular surface
maxa∈R
f (a) = a(
L2 − a
)
subject to
a > 0
a < L2
![Page 40: Introduction to Optimization - CERMICS – Centre d ...cermics.enpc.fr/~delara/MABIES/pgajardo_optimization_IHP2013.pdf · Introduction Optimality conditions Introduction to Optimization](https://reader031.vdocuments.us/reader031/viewer/2022021903/5b9da90609d3f2de128ca4f5/html5/thumbnails/40.jpg)
Introduction
Optimalityconditions
Optimization problemsExample: Optimizing a portafolio
A firm wishes to maximize the utilities of its portafolio
Objective: maximize profits
Decision variables: amount to invest in each fund
Constraints:
nonnegative investments
budget restrictions
minimal or maximal bounds for investments
![Page 41: Introduction to Optimization - CERMICS – Centre d ...cermics.enpc.fr/~delara/MABIES/pgajardo_optimization_IHP2013.pdf · Introduction Optimality conditions Introduction to Optimization](https://reader031.vdocuments.us/reader031/viewer/2022021903/5b9da90609d3f2de128ca4f5/html5/thumbnails/41.jpg)
Introduction
Optimalityconditions
Optimization problemsExample: Optimizing a portafolio
A firm wishes to maximize the utilities of its portafolio
Objective: maximize profits
Decision variables: amount to invest in each fund
Constraints:
nonnegative investments
budget restrictions
minimal or maximal bounds for investments
![Page 42: Introduction to Optimization - CERMICS – Centre d ...cermics.enpc.fr/~delara/MABIES/pgajardo_optimization_IHP2013.pdf · Introduction Optimality conditions Introduction to Optimization](https://reader031.vdocuments.us/reader031/viewer/2022021903/5b9da90609d3f2de128ca4f5/html5/thumbnails/42.jpg)
Introduction
Optimalityconditions
Optimization problemsExample: Optimizing a portafolio
A firm wishes to maximize the utilities of its portafolio
Objective: maximize profits
Decision variables: amount to invest in each fund
Constraints:
nonnegative investments
budget restrictions
minimal or maximal bounds for investments
![Page 43: Introduction to Optimization - CERMICS – Centre d ...cermics.enpc.fr/~delara/MABIES/pgajardo_optimization_IHP2013.pdf · Introduction Optimality conditions Introduction to Optimization](https://reader031.vdocuments.us/reader031/viewer/2022021903/5b9da90609d3f2de128ca4f5/html5/thumbnails/43.jpg)
Introduction
Optimalityconditions
Optimization problemsExample: Optimizing a portafolio
A firm wishes to maximize the utilities of its portafolio
Objective: maximize profits
Decision variables: amount to invest in each fund
Constraints:
nonnegative investments
budget restrictions
minimal or maximal bounds for investments
![Page 44: Introduction to Optimization - CERMICS – Centre d ...cermics.enpc.fr/~delara/MABIES/pgajardo_optimization_IHP2013.pdf · Introduction Optimality conditions Introduction to Optimization](https://reader031.vdocuments.us/reader031/viewer/2022021903/5b9da90609d3f2de128ca4f5/html5/thumbnails/44.jpg)
Introduction
Optimalityconditions
Optimization problemsExample: Optimizing a portafolio
A firm wishes to maximize the utilities of its portafolio
Objective: maximize profits
Decision variables: amount to invest in each fund
Constraints:
nonnegative investments
budget restrictions
minimal or maximal bounds for investments
![Page 45: Introduction to Optimization - CERMICS – Centre d ...cermics.enpc.fr/~delara/MABIES/pgajardo_optimization_IHP2013.pdf · Introduction Optimality conditions Introduction to Optimization](https://reader031.vdocuments.us/reader031/viewer/2022021903/5b9da90609d3f2de128ca4f5/html5/thumbnails/45.jpg)
Introduction
Optimalityconditions
Optimization problemsExample: Optimizing a portafolio
A firm wishes to maximize the utilities of its portafolio
Objective: maximize profits
Decision variables: amount to invest in each fund
Constraints:
nonnegative investments
budget restrictions
minimal or maximal bounds for investments
![Page 46: Introduction to Optimization - CERMICS – Centre d ...cermics.enpc.fr/~delara/MABIES/pgajardo_optimization_IHP2013.pdf · Introduction Optimality conditions Introduction to Optimization](https://reader031.vdocuments.us/reader031/viewer/2022021903/5b9da90609d3f2de128ca4f5/html5/thumbnails/46.jpg)
Introduction
Optimalityconditions
Optimization problemsExample: Optimizing a portafolio
Decision variables: x1, x2, . . . , xn, wherexj the quantity toinvest in the fundj
Objective: maximize
(1+ r1)x1 + (1+ r2)x2 + . . .+ (1+ rn)xn
wherer j is the rentability of the fundj
Restrictions:
Non-negative investments:xj ≥ 0 j = 1, 2, . . . , n
Budget restriction:
x1 + x2 + . . .+ xn ≤ B
Minimal/maximal bounds for investments:
aj ≤ xj ≤ bj for j = 1, 2, . . . , n
![Page 47: Introduction to Optimization - CERMICS – Centre d ...cermics.enpc.fr/~delara/MABIES/pgajardo_optimization_IHP2013.pdf · Introduction Optimality conditions Introduction to Optimization](https://reader031.vdocuments.us/reader031/viewer/2022021903/5b9da90609d3f2de128ca4f5/html5/thumbnails/47.jpg)
Introduction
Optimalityconditions
Optimization problemsExample: Optimizing a portafolio
Decision variables:x1, x2, . . . , xn, wherexj the quantity toinvest in the fundj
Objective: maximize
(1+ r1)x1 + (1+ r2)x2 + . . .+ (1+ rn)xn
wherer j is the rentability of the fundj
Restrictions:
Non-negative investments:xj ≥ 0 j = 1, 2, . . . , n
Budget restriction:
x1 + x2 + . . .+ xn ≤ B
Minimal/maximal bounds for investments:
aj ≤ xj ≤ bj for j = 1, 2, . . . , n
![Page 48: Introduction to Optimization - CERMICS – Centre d ...cermics.enpc.fr/~delara/MABIES/pgajardo_optimization_IHP2013.pdf · Introduction Optimality conditions Introduction to Optimization](https://reader031.vdocuments.us/reader031/viewer/2022021903/5b9da90609d3f2de128ca4f5/html5/thumbnails/48.jpg)
Introduction
Optimalityconditions
Optimization problemsExample: Optimizing a portafolio
Decision variables:x1, x2, . . . , xn, wherexj the quantity toinvest in the fundj
Objective: maximize
(1+ r1)x1 + (1+ r2)x2 + . . .+ (1+ rn)xn
wherer j is the rentability of the fundj
Restrictions:
Non-negative investments:xj ≥ 0 j = 1, 2, . . . , n
Budget restriction:
x1 + x2 + . . .+ xn ≤ B
Minimal/maximal bounds for investments:
aj ≤ xj ≤ bj for j = 1, 2, . . . , n
![Page 49: Introduction to Optimization - CERMICS – Centre d ...cermics.enpc.fr/~delara/MABIES/pgajardo_optimization_IHP2013.pdf · Introduction Optimality conditions Introduction to Optimization](https://reader031.vdocuments.us/reader031/viewer/2022021903/5b9da90609d3f2de128ca4f5/html5/thumbnails/49.jpg)
Introduction
Optimalityconditions
Optimization problemsExample: Optimizing a portafolio
Decision variables:x1, x2, . . . , xn, wherexj the quantity toinvest in the fundj
Objective: maximize
(1+ r1)x1 + (1+ r2)x2 + . . .+ (1+ rn)xn
wherer j is the rentability of the fundj
Restrictions:
Non-negative investments: xj ≥ 0 j = 1, 2, . . . , n
Budget restriction:
x1 + x2 + . . .+ xn ≤ B
Minimal/maximal bounds for investments:
aj ≤ xj ≤ bj for j = 1, 2, . . . , n
![Page 50: Introduction to Optimization - CERMICS – Centre d ...cermics.enpc.fr/~delara/MABIES/pgajardo_optimization_IHP2013.pdf · Introduction Optimality conditions Introduction to Optimization](https://reader031.vdocuments.us/reader031/viewer/2022021903/5b9da90609d3f2de128ca4f5/html5/thumbnails/50.jpg)
Introduction
Optimalityconditions
Optimization problemsExample: Optimizing a portafolio
Decision variables:x1, x2, . . . , xn, wherexj the quantity toinvest in the fundj
Objective: maximize
(1+ r1)x1 + (1+ r2)x2 + . . .+ (1+ rn)xn
wherer j is the rentability of the fundj
Restrictions:
Non-negative investments:xj ≥ 0 j = 1, 2, . . . , n
Budget restriction:
x1 + x2 + . . .+ xn ≤ B
Minimal/maximal bounds for investments:
aj ≤ xj ≤ bj for j = 1, 2, . . . , n
![Page 51: Introduction to Optimization - CERMICS – Centre d ...cermics.enpc.fr/~delara/MABIES/pgajardo_optimization_IHP2013.pdf · Introduction Optimality conditions Introduction to Optimization](https://reader031.vdocuments.us/reader031/viewer/2022021903/5b9da90609d3f2de128ca4f5/html5/thumbnails/51.jpg)
Introduction
Optimalityconditions
Optimization problemsExample: Optimizing a portafolio
Decision variables:x1, x2, . . . , xn, wherexj the quantity toinvest in the fundj
Objective: maximize
(1+ r1)x1 + (1+ r2)x2 + . . .+ (1+ rn)xn
wherer j is the rentability of the fundj
Restrictions:
Non-negative investments:xj ≥ 0 j = 1, 2, . . . , n
Budget restriction:
x1 + x2 + . . .+ xn ≤ B
Minimal/maximal bounds for investments:
aj ≤ xj ≤ bj for j = 1, 2, . . . , n
![Page 52: Introduction to Optimization - CERMICS – Centre d ...cermics.enpc.fr/~delara/MABIES/pgajardo_optimization_IHP2013.pdf · Introduction Optimality conditions Introduction to Optimization](https://reader031.vdocuments.us/reader031/viewer/2022021903/5b9da90609d3f2de128ca4f5/html5/thumbnails/52.jpg)
Introduction
Optimalityconditions
Optimization problemsExample: Optimizing a portafolio
maxx1,x2,...,xn∈R
(1+ r1)x1 + (1+ r2)x2 + . . .+ (1+ rn)xn
subject to
xj ≥ 0 j = 1,2, . . . ,n
x1 + x2 + . . .+ xn ≤ B
aj ≤ xj ≤ bj j = 1,2, . . . ,n
![Page 53: Introduction to Optimization - CERMICS – Centre d ...cermics.enpc.fr/~delara/MABIES/pgajardo_optimization_IHP2013.pdf · Introduction Optimality conditions Introduction to Optimization](https://reader031.vdocuments.us/reader031/viewer/2022021903/5b9da90609d3f2de128ca4f5/html5/thumbnails/53.jpg)
Introduction
Optimalityconditions
Optimization problemsExample: Least square problem
Given points on the plane, to find theline of best fitthroughthese points
Objective: To minimize the (squared) error between a lineand the points
Decision variables: the parameters of a line
![Page 54: Introduction to Optimization - CERMICS – Centre d ...cermics.enpc.fr/~delara/MABIES/pgajardo_optimization_IHP2013.pdf · Introduction Optimality conditions Introduction to Optimization](https://reader031.vdocuments.us/reader031/viewer/2022021903/5b9da90609d3f2de128ca4f5/html5/thumbnails/54.jpg)
Introduction
Optimalityconditions
Optimization problemsExample: Least square problem
Given points on the plane, to find theline of best fitthroughthese points
Objective: To minimize the (squared) error between a lineand the points
Decision variables: the parameters of a line
![Page 55: Introduction to Optimization - CERMICS – Centre d ...cermics.enpc.fr/~delara/MABIES/pgajardo_optimization_IHP2013.pdf · Introduction Optimality conditions Introduction to Optimization](https://reader031.vdocuments.us/reader031/viewer/2022021903/5b9da90609d3f2de128ca4f5/html5/thumbnails/55.jpg)
Introduction
Optimalityconditions
Optimization problemsExample: Least square problem
Given points
(x1, y1); (x2, y2); . . . ; (xn, yn)
to find theline of best fitthrough these points
Decision variables (the parameters of a line): mandn,where the expression of a line in the plane is
y = mx+ n
Objective: To minimize
n∑
k=1
(mxk + n− yk)2
![Page 56: Introduction to Optimization - CERMICS – Centre d ...cermics.enpc.fr/~delara/MABIES/pgajardo_optimization_IHP2013.pdf · Introduction Optimality conditions Introduction to Optimization](https://reader031.vdocuments.us/reader031/viewer/2022021903/5b9da90609d3f2de128ca4f5/html5/thumbnails/56.jpg)
Introduction
Optimalityconditions
Optimization problemsExample: Least square problem
Given points
(x1, y1); (x2, y2); . . . ; (xn, yn)
to find theline of best fitthrough these points
Decision variables (the parameters of a line):mandn,where the expression of a line in the plane is
y = mx+ n
Objective: To minimize
n∑
k=1
(mxk + n− yk)2
![Page 57: Introduction to Optimization - CERMICS – Centre d ...cermics.enpc.fr/~delara/MABIES/pgajardo_optimization_IHP2013.pdf · Introduction Optimality conditions Introduction to Optimization](https://reader031.vdocuments.us/reader031/viewer/2022021903/5b9da90609d3f2de128ca4f5/html5/thumbnails/57.jpg)
Introduction
Optimalityconditions
Optimization problemsExample: Least square problem
maxm, n ∈ R
n∑
k=1(mxk + n− yk)
2
![Page 58: Introduction to Optimization - CERMICS – Centre d ...cermics.enpc.fr/~delara/MABIES/pgajardo_optimization_IHP2013.pdf · Introduction Optimality conditions Introduction to Optimization](https://reader031.vdocuments.us/reader031/viewer/2022021903/5b9da90609d3f2de128ca4f5/html5/thumbnails/58.jpg)
Introduction
Optimalityconditions
Optimization problemsExample: Harvesting of a renewable resource
To maximize the benefit from the harvesting of a naturalresource
Objective: maximize present utilities
Decision variables: harvesting levels (or effort) at eachperiod
Constraints:
nonnegative harvesting
biology
relation between harvesting and the amount of the resource
environmental constraints
![Page 59: Introduction to Optimization - CERMICS – Centre d ...cermics.enpc.fr/~delara/MABIES/pgajardo_optimization_IHP2013.pdf · Introduction Optimality conditions Introduction to Optimization](https://reader031.vdocuments.us/reader031/viewer/2022021903/5b9da90609d3f2de128ca4f5/html5/thumbnails/59.jpg)
Introduction
Optimalityconditions
Optimization problemsExample: Harvesting of a renewable resource
To maximize the benefit from the harvesting of a naturalresource
Objective: maximize present utilities
Decision variables: harvesting levels (or effort) at eachperiod
Constraints:
nonnegative harvesting
biology
relation between harvesting and the amount of the resource
environmental constraints
![Page 60: Introduction to Optimization - CERMICS – Centre d ...cermics.enpc.fr/~delara/MABIES/pgajardo_optimization_IHP2013.pdf · Introduction Optimality conditions Introduction to Optimization](https://reader031.vdocuments.us/reader031/viewer/2022021903/5b9da90609d3f2de128ca4f5/html5/thumbnails/60.jpg)
Introduction
Optimalityconditions
Optimization problemsExample: Harvesting of a renewable resource
To maximize the benefit from the harvesting of a naturalresource
Objective: maximize present utilities
Decision variables: harvesting levels (or effort) at eachperiod
Constraints:
nonnegative harvesting
biology
relation between harvesting and the amount of the resource
environmental constraints
![Page 61: Introduction to Optimization - CERMICS – Centre d ...cermics.enpc.fr/~delara/MABIES/pgajardo_optimization_IHP2013.pdf · Introduction Optimality conditions Introduction to Optimization](https://reader031.vdocuments.us/reader031/viewer/2022021903/5b9da90609d3f2de128ca4f5/html5/thumbnails/61.jpg)
Introduction
Optimalityconditions
Optimization problemsExample: Harvesting of a renewable resource
To maximize the benefit from the harvesting of a naturalresource
Objective: maximize present utilities
Decision variables: harvesting levels (or effort) at eachperiod
Constraints:
nonnegative harvesting
biology
relation between harvesting and the amount of the resource
environmental constraints
![Page 62: Introduction to Optimization - CERMICS – Centre d ...cermics.enpc.fr/~delara/MABIES/pgajardo_optimization_IHP2013.pdf · Introduction Optimality conditions Introduction to Optimization](https://reader031.vdocuments.us/reader031/viewer/2022021903/5b9da90609d3f2de128ca4f5/html5/thumbnails/62.jpg)
Introduction
Optimalityconditions
Optimization problemsExample: Harvesting of a renewable resource
To maximize the benefit from the harvesting of a naturalresource
Objective: maximize present utilities
Decision variables: harvesting levels (or effort) at eachperiod
Constraints:
nonnegative harvesting
biology
relation between harvesting and the amount of the resource
environmental constraints
![Page 63: Introduction to Optimization - CERMICS – Centre d ...cermics.enpc.fr/~delara/MABIES/pgajardo_optimization_IHP2013.pdf · Introduction Optimality conditions Introduction to Optimization](https://reader031.vdocuments.us/reader031/viewer/2022021903/5b9da90609d3f2de128ca4f5/html5/thumbnails/63.jpg)
Introduction
Optimalityconditions
Optimization problemsExample: Harvesting of a renewable resource
To maximize the benefit from the harvesting of a naturalresource
Objective: maximize present utilities
Decision variables: harvesting levels (or effort) at eachperiod
Constraints:
nonnegative harvesting
biology
relation between harvesting and the amount of the resource
environmental constraints
![Page 64: Introduction to Optimization - CERMICS – Centre d ...cermics.enpc.fr/~delara/MABIES/pgajardo_optimization_IHP2013.pdf · Introduction Optimality conditions Introduction to Optimization](https://reader031.vdocuments.us/reader031/viewer/2022021903/5b9da90609d3f2de128ca4f5/html5/thumbnails/64.jpg)
Introduction
Optimalityconditions
Optimization problemsExample: Harvesting of a renewable resource
To maximize the benefit from the harvesting of a naturalresource
Objective: maximize present utilities
Decision variables: harvesting levels (or effort) at eachperiod
Constraints:
nonnegative harvesting
biology
relation between harvesting and the amount of the resource
environmental constraints
![Page 65: Introduction to Optimization - CERMICS – Centre d ...cermics.enpc.fr/~delara/MABIES/pgajardo_optimization_IHP2013.pdf · Introduction Optimality conditions Introduction to Optimization](https://reader031.vdocuments.us/reader031/viewer/2022021903/5b9da90609d3f2de128ca4f5/html5/thumbnails/65.jpg)
Introduction
Optimalityconditions
Optimization problemsExample: Harvesting of a renewable resource
x(t + 1) = F(x(t)) − h(t) t = t0, t0 + 1, . . . ,T − 1
x(t0) = x0 given (e.g. the current state of the resource)
where
x(t) ≥ 0 is the level of the resource at periodt
h(t) ≥ 0 is the harvesting at periodt
F : R −→ R is the biological growth function of theresource
![Page 66: Introduction to Optimization - CERMICS – Centre d ...cermics.enpc.fr/~delara/MABIES/pgajardo_optimization_IHP2013.pdf · Introduction Optimality conditions Introduction to Optimization](https://reader031.vdocuments.us/reader031/viewer/2022021903/5b9da90609d3f2de128ca4f5/html5/thumbnails/66.jpg)
Introduction
Optimalityconditions
Optimization problemsExample: Harvesting of a renewable resource
x(t + 1) = F(x(t)) − h(t) t = t0, t0 + 1, . . . ,T − 1
x(t0) = x0 given (e.g. the current state of the resource)
where
x(t) ≥ 0 is the level of the resource at periodt
h(t) ≥ 0 is the harvesting at periodt
F : R −→ R is the biological growth function of theresource
![Page 67: Introduction to Optimization - CERMICS – Centre d ...cermics.enpc.fr/~delara/MABIES/pgajardo_optimization_IHP2013.pdf · Introduction Optimality conditions Introduction to Optimization](https://reader031.vdocuments.us/reader031/viewer/2022021903/5b9da90609d3f2de128ca4f5/html5/thumbnails/67.jpg)
Introduction
Optimalityconditions
Optimization problemsExample: Harvesting of a renewable resource
x(t + 1) = F(x(t)) − h(t) t = t0, t0 + 1, . . . ,T − 1
x(t0) = x0 given (e.g. the current state of the resource)
where
x(t) ≥ 0 is the level of the resource at periodt
h(t) ≥ 0 is the harvesting at periodt
F : R −→ R is the biological growth function of theresource
![Page 68: Introduction to Optimization - CERMICS – Centre d ...cermics.enpc.fr/~delara/MABIES/pgajardo_optimization_IHP2013.pdf · Introduction Optimality conditions Introduction to Optimization](https://reader031.vdocuments.us/reader031/viewer/2022021903/5b9da90609d3f2de128ca4f5/html5/thumbnails/68.jpg)
Introduction
Optimalityconditions
Optimization problemsExample: Harvesting of a renewable resource
The total benefits can be represented by
B =T−1∑
t=t0
ρ(t−t0)U(x(t),h(t))
where
U(x,h) is the instantaneous profit if we havex and weharvesth
0 ≤ ρ ≤ 1 is a descount factor
![Page 69: Introduction to Optimization - CERMICS – Centre d ...cermics.enpc.fr/~delara/MABIES/pgajardo_optimization_IHP2013.pdf · Introduction Optimality conditions Introduction to Optimization](https://reader031.vdocuments.us/reader031/viewer/2022021903/5b9da90609d3f2de128ca4f5/html5/thumbnails/69.jpg)
Introduction
Optimalityconditions
Optimization problemsExample: Harvesting of a renewable resource
The total benefits can be represented by
B =T−1∑
t=t0
ρ(t−t0)U(x(t),h(t))
where
U(x,h) is the instantaneous profit if we havex and weharvesth
0 ≤ ρ ≤ 1 is a descount factor
![Page 70: Introduction to Optimization - CERMICS – Centre d ...cermics.enpc.fr/~delara/MABIES/pgajardo_optimization_IHP2013.pdf · Introduction Optimality conditions Introduction to Optimization](https://reader031.vdocuments.us/reader031/viewer/2022021903/5b9da90609d3f2de128ca4f5/html5/thumbnails/70.jpg)
Introduction
Optimalityconditions
Optimization problemsExample: Harvesting of a renewable resource
The total benefits can be represented by
B =T−1∑
t=t0
ρ(t−t0)U(x(t),h(t))
where
U(x,h) is the instantaneous profit if we havex and weharvesth
0 ≤ ρ ≤ 1 is a descount factor
![Page 71: Introduction to Optimization - CERMICS – Centre d ...cermics.enpc.fr/~delara/MABIES/pgajardo_optimization_IHP2013.pdf · Introduction Optimality conditions Introduction to Optimization](https://reader031.vdocuments.us/reader031/viewer/2022021903/5b9da90609d3f2de128ca4f5/html5/thumbnails/71.jpg)
Introduction
Optimalityconditions
Optimization problemsExample: Harvesting of a renewable resource
maxh(t0),h(t0+1),...,h(T−1)
T−1∑
t=t0ρ(t−t0)U(x(t),h(t))
subject to
x(t + 1) = F(x(t)) − h(t) t = t0, t0 + 1, . . . ,T − 1
x(t0) = x0
x(t) ≥ 0 t = t0 + 1, t0 + 2, . . . ,T
0 ≤ h(t) ≤ F(x(t)) t = t0, t0 + 1, . . . ,T − 1
![Page 72: Introduction to Optimization - CERMICS – Centre d ...cermics.enpc.fr/~delara/MABIES/pgajardo_optimization_IHP2013.pdf · Introduction Optimality conditions Introduction to Optimization](https://reader031.vdocuments.us/reader031/viewer/2022021903/5b9da90609d3f2de128ca4f5/html5/thumbnails/72.jpg)
Introduction
Optimalityconditions
Optimization problemsImportant remark: min is equivalent to max
We can restrict our study only to minimization problems
To find x ∈ C ⊆ X such that
f (x) ≤ f (x) for all x ∈ C
minx∈X
f (x)
subject to
x ∈ C
![Page 73: Introduction to Optimization - CERMICS – Centre d ...cermics.enpc.fr/~delara/MABIES/pgajardo_optimization_IHP2013.pdf · Introduction Optimality conditions Introduction to Optimization](https://reader031.vdocuments.us/reader031/viewer/2022021903/5b9da90609d3f2de128ca4f5/html5/thumbnails/73.jpg)
Introduction
Optimalityconditions
Optimization problemsImportant remark: min is equivalent to max
We can restrict our study only to minimization problems
To find x ∈ C ⊆ X such that
f (x) ≤ f (x) for all x ∈ C
minx∈X
f (x)
subject to
x ∈ C
![Page 74: Introduction to Optimization - CERMICS – Centre d ...cermics.enpc.fr/~delara/MABIES/pgajardo_optimization_IHP2013.pdf · Introduction Optimality conditions Introduction to Optimization](https://reader031.vdocuments.us/reader031/viewer/2022021903/5b9da90609d3f2de128ca4f5/html5/thumbnails/74.jpg)
Introduction
Optimalityconditions
Optimization problemsImportant remark: min is equivalent to max
To find x ∈ C ⊆ X such that
f (x) ≥ f (x) for all x ∈ C
maxx∈X
f (x)
subject to
x ∈ C
![Page 75: Introduction to Optimization - CERMICS – Centre d ...cermics.enpc.fr/~delara/MABIES/pgajardo_optimization_IHP2013.pdf · Introduction Optimality conditions Introduction to Optimization](https://reader031.vdocuments.us/reader031/viewer/2022021903/5b9da90609d3f2de128ca4f5/html5/thumbnails/75.jpg)
Introduction
Optimalityconditions
Optimization problemsImportant remark: min is equivalent to max
To find x ∈ C ⊆ X such that
f (x) ≥ f (x) for all x ∈ C
maxx∈X
f (x)
subject to
x ∈ C
![Page 76: Introduction to Optimization - CERMICS – Centre d ...cermics.enpc.fr/~delara/MABIES/pgajardo_optimization_IHP2013.pdf · Introduction Optimality conditions Introduction to Optimization](https://reader031.vdocuments.us/reader031/viewer/2022021903/5b9da90609d3f2de128ca4f5/html5/thumbnails/76.jpg)
Introduction
Optimalityconditions
Optimization problemsImportant remark: min is equivalent to max
(Pmax)
maxx∈X
f (x)
subject to
x ∈ C
is equivalent to problem
(Pmin)
minx∈X
−f (x)
subject to
x ∈ C
In the following sense:
x is solution of(Pmax) if and only if it is solution of(Pmin)
The optimal value of(Pmax) is the negative of the optimalvalue of(Pmin)
![Page 77: Introduction to Optimization - CERMICS – Centre d ...cermics.enpc.fr/~delara/MABIES/pgajardo_optimization_IHP2013.pdf · Introduction Optimality conditions Introduction to Optimization](https://reader031.vdocuments.us/reader031/viewer/2022021903/5b9da90609d3f2de128ca4f5/html5/thumbnails/77.jpg)
Introduction
Optimalityconditions
Optimization problemsImportant remark: min is equivalent to max
(Pmax)
maxx∈X
f (x)
subject to
x ∈ C
is equivalent to problem
(Pmin)
minx∈X
−f (x)
subject to
x ∈ C
In the following sense:
x is solution of(Pmax) if and only if it is solution of(Pmin)
The optimal value of(Pmax) is the negative of the optimalvalue of(Pmin)
![Page 78: Introduction to Optimization - CERMICS – Centre d ...cermics.enpc.fr/~delara/MABIES/pgajardo_optimization_IHP2013.pdf · Introduction Optimality conditions Introduction to Optimization](https://reader031.vdocuments.us/reader031/viewer/2022021903/5b9da90609d3f2de128ca4f5/html5/thumbnails/78.jpg)
Introduction
Optimalityconditions
Optimization problemsImportant remark: min is equivalent to max
(Pmax)
maxx∈X
f (x)
subject to
x ∈ C
is equivalent to problem
(Pmin)
minx∈X
−f (x)
subject to
x ∈ C
In the following sense:
x is solution of(Pmax) if and only if it is solution of(Pmin)
The optimal value of(Pmax) is the negative of the optimalvalue of(Pmin)
![Page 79: Introduction to Optimization - CERMICS – Centre d ...cermics.enpc.fr/~delara/MABIES/pgajardo_optimization_IHP2013.pdf · Introduction Optimality conditions Introduction to Optimization](https://reader031.vdocuments.us/reader031/viewer/2022021903/5b9da90609d3f2de128ca4f5/html5/thumbnails/79.jpg)
Introduction
Optimalityconditions
Optimization problemsExistence of solutions
(P)
minx∈X
f (x)
subject to
x ∈ C
When(P) has solution?
If [a,b] = C ⊆ X = R
WhenX = Rn, there exists at least one solution ifC is
closed and bounded
![Page 80: Introduction to Optimization - CERMICS – Centre d ...cermics.enpc.fr/~delara/MABIES/pgajardo_optimization_IHP2013.pdf · Introduction Optimality conditions Introduction to Optimization](https://reader031.vdocuments.us/reader031/viewer/2022021903/5b9da90609d3f2de128ca4f5/html5/thumbnails/80.jpg)
Introduction
Optimalityconditions
Optimization problemsExistence of solutions
(P)
minx∈X
f (x)
subject to
x ∈ C
When(P) has solution?
If [a,b] = C ⊆ X = R
WhenX = Rn, there exists at least one solution ifC is
closed and bounded
![Page 81: Introduction to Optimization - CERMICS – Centre d ...cermics.enpc.fr/~delara/MABIES/pgajardo_optimization_IHP2013.pdf · Introduction Optimality conditions Introduction to Optimization](https://reader031.vdocuments.us/reader031/viewer/2022021903/5b9da90609d3f2de128ca4f5/html5/thumbnails/81.jpg)
Introduction
Optimalityconditions
Contents
Introduction
Optimality conditions
![Page 82: Introduction to Optimization - CERMICS – Centre d ...cermics.enpc.fr/~delara/MABIES/pgajardo_optimization_IHP2013.pdf · Introduction Optimality conditions Introduction to Optimization](https://reader031.vdocuments.us/reader031/viewer/2022021903/5b9da90609d3f2de128ca4f5/html5/thumbnails/82.jpg)
Introduction
Optimalityconditions
Optimality conditions
(P)
minx∈X
f (x)
subject to
x ∈ C
Optimality conditions are mathematical expressions: equationsor system of equations, system of inequalities, ordinarydifferential equations, partial differential equations
Solutions of the mathematical problems obtained fromoptimality conditions are related with the solutions of theoptimization problem
![Page 83: Introduction to Optimization - CERMICS – Centre d ...cermics.enpc.fr/~delara/MABIES/pgajardo_optimization_IHP2013.pdf · Introduction Optimality conditions Introduction to Optimization](https://reader031.vdocuments.us/reader031/viewer/2022021903/5b9da90609d3f2de128ca4f5/html5/thumbnails/83.jpg)
Introduction
Optimalityconditions
Optimality conditions
(P)
minx∈X
f (x)
subject to
x ∈ C
Optimality conditions are mathematical expressions: equationsor system of equations, system of inequalities, ordinarydifferential equations, partial differential equations
Solutions of the mathematical problems obtained fromoptimality conditions are related with the solutions of theoptimization problem
![Page 84: Introduction to Optimization - CERMICS – Centre d ...cermics.enpc.fr/~delara/MABIES/pgajardo_optimization_IHP2013.pdf · Introduction Optimality conditions Introduction to Optimization](https://reader031.vdocuments.us/reader031/viewer/2022021903/5b9da90609d3f2de128ca4f5/html5/thumbnails/84.jpg)
Introduction
Optimalityconditions
Optimality conditions
(P)
minx∈X
f (x)
subject to
x ∈ C
Optimality conditions are mathematical expressions: equationsor system of equations, system of inequalities, ordinarydifferential equations, partial differential equations
Solutions of the mathematical problems obtained fromoptimality conditions are related with the solutions of theoptimization problem
![Page 85: Introduction to Optimization - CERMICS – Centre d ...cermics.enpc.fr/~delara/MABIES/pgajardo_optimization_IHP2013.pdf · Introduction Optimality conditions Introduction to Optimization](https://reader031.vdocuments.us/reader031/viewer/2022021903/5b9da90609d3f2de128ca4f5/html5/thumbnails/85.jpg)
Introduction
Optimalityconditions
Optimality conditions
(P)
minx∈X
f (x)
subject to
x ∈ C
The idea is to obtain optimality conditions and then to solvetheassociated mathematical problems (analytically or numerically)
![Page 86: Introduction to Optimization - CERMICS – Centre d ...cermics.enpc.fr/~delara/MABIES/pgajardo_optimization_IHP2013.pdf · Introduction Optimality conditions Introduction to Optimization](https://reader031.vdocuments.us/reader031/viewer/2022021903/5b9da90609d3f2de128ca4f5/html5/thumbnails/86.jpg)
Introduction
Optimalityconditions
Optimality conditions
(P)
minx∈X
f (x)
subject to
x ∈ C
The idea isto obtain optimality conditionsand thento solve theassociated mathematical problems (analytically or numerically)
![Page 87: Introduction to Optimization - CERMICS – Centre d ...cermics.enpc.fr/~delara/MABIES/pgajardo_optimization_IHP2013.pdf · Introduction Optimality conditions Introduction to Optimization](https://reader031.vdocuments.us/reader031/viewer/2022021903/5b9da90609d3f2de128ca4f5/html5/thumbnails/87.jpg)
Introduction
Optimalityconditions
Optimality conditionsNecessary conditions
(P)
minx∈X
f (x)
subject to
x ∈ C
Suppose thatoptimality conditionsof this problem arerepresented by a system of equations(S)
We say that(S) is a necessary condition if a solution of(P) is a solution of(S)
The idea is to find solutions of(S) in order to have a list ofcandidates for the solutions of(P)
If (S) is solved only for one point, then it is solution of(P)or (P) does not have solution
Warning: A solution of(S) may be not a solution of(P)
![Page 88: Introduction to Optimization - CERMICS – Centre d ...cermics.enpc.fr/~delara/MABIES/pgajardo_optimization_IHP2013.pdf · Introduction Optimality conditions Introduction to Optimization](https://reader031.vdocuments.us/reader031/viewer/2022021903/5b9da90609d3f2de128ca4f5/html5/thumbnails/88.jpg)
Introduction
Optimalityconditions
Optimality conditionsNecessary conditions
(P)
minx∈X
f (x)
subject to
x ∈ C
Suppose that optimality conditions of this problem arerepresented by a system of equations(S)
We say that(S) is a necessary condition if a solution of(P) is a solution of(S)
The idea is to find solutions of(S) in order to have a list ofcandidates for the solutions of(P)
If (S) is solved only for one point, then it is solution of(P)or (P) does not have solution
Warning: A solution of(S) may be not a solution of(P)
![Page 89: Introduction to Optimization - CERMICS – Centre d ...cermics.enpc.fr/~delara/MABIES/pgajardo_optimization_IHP2013.pdf · Introduction Optimality conditions Introduction to Optimization](https://reader031.vdocuments.us/reader031/viewer/2022021903/5b9da90609d3f2de128ca4f5/html5/thumbnails/89.jpg)
Introduction
Optimalityconditions
Optimality conditionsNecessary conditions
(P)
minx∈X
f (x)
subject to
x ∈ C
Suppose that optimality conditions of this problem arerepresented by a system of equations(S)
We say that(S) is a necessary condition if a solution of(P) is a solution of(S)
The idea isto find solutions of(S) in orderto have a list ofcandidates for the solutions of(P)
If (S) is solved only for one point, then it is solution of(P)or (P) does not have solution
Warning: A solution of(S) may be not a solution of(P)
![Page 90: Introduction to Optimization - CERMICS – Centre d ...cermics.enpc.fr/~delara/MABIES/pgajardo_optimization_IHP2013.pdf · Introduction Optimality conditions Introduction to Optimization](https://reader031.vdocuments.us/reader031/viewer/2022021903/5b9da90609d3f2de128ca4f5/html5/thumbnails/90.jpg)
Introduction
Optimalityconditions
Optimality conditionsNecessary conditions
(P)
minx∈X
f (x)
subject to
x ∈ C
Suppose that optimality conditions of this problem arerepresented by a system of equations(S)
We say that(S) is a necessary condition if a solution of(P) is a solution of(S)
The idea is to find solutions of(S) in order to have a list ofcandidates for the solutions of(P)
If (S) is solved only for one point, then it is solution of(P)or (P) does not have solution
Warning: A solution of(S) may be not a solution of(P)
![Page 91: Introduction to Optimization - CERMICS – Centre d ...cermics.enpc.fr/~delara/MABIES/pgajardo_optimization_IHP2013.pdf · Introduction Optimality conditions Introduction to Optimization](https://reader031.vdocuments.us/reader031/viewer/2022021903/5b9da90609d3f2de128ca4f5/html5/thumbnails/91.jpg)
Introduction
Optimalityconditions
Optimality conditionsNecessary conditions
(P)
minx∈X
f (x)
subject to
x ∈ C
Suppose that optimality conditions of this problem arerepresented by a system of equations(S)
We say that(S) is a necessary condition if a solution of(P) is a solution of(S)
The idea is to find solutions of(S) in order to have a list ofcandidates for the solutions of(P)
If (S) is solved only for one point, then it is solution of(P)or (P) does not have solution
Warning: A solution of(S) may be not a solution of(P)
![Page 92: Introduction to Optimization - CERMICS – Centre d ...cermics.enpc.fr/~delara/MABIES/pgajardo_optimization_IHP2013.pdf · Introduction Optimality conditions Introduction to Optimization](https://reader031.vdocuments.us/reader031/viewer/2022021903/5b9da90609d3f2de128ca4f5/html5/thumbnails/92.jpg)
Introduction
Optimalityconditions
Optimality conditionsNecessary condition: unrestricted case
(P){
minx∈R
f (x)
where f : R −→ R
A necessary condition isf ′(x) = 0 where
f ′(x) = limt→0
f (x+ t)− f (x)t
That means: Ifx is a solution of(P) thenf (x) = 0
If we know the expression off ′ we can solve the equationf ′(x) = 0 in order to have candidates of solutions
Warning!!!
![Page 93: Introduction to Optimization - CERMICS – Centre d ...cermics.enpc.fr/~delara/MABIES/pgajardo_optimization_IHP2013.pdf · Introduction Optimality conditions Introduction to Optimization](https://reader031.vdocuments.us/reader031/viewer/2022021903/5b9da90609d3f2de128ca4f5/html5/thumbnails/93.jpg)
Introduction
Optimalityconditions
Optimality conditionsNecessary condition: unrestricted case
(P){
minx∈R
f (x)
where f : R −→ R
A necessary condition isf ′(x) = 0 where
f ′(x) = limt→0
f (x+ t)− f (x)t
That means:If x is a solution of(P) thenf (x) = 0
If we know the expression off ′ we can solve the equationf ′(x) = 0 in order to have candidates of solutions
Warning!!!
![Page 94: Introduction to Optimization - CERMICS – Centre d ...cermics.enpc.fr/~delara/MABIES/pgajardo_optimization_IHP2013.pdf · Introduction Optimality conditions Introduction to Optimization](https://reader031.vdocuments.us/reader031/viewer/2022021903/5b9da90609d3f2de128ca4f5/html5/thumbnails/94.jpg)
Introduction
Optimalityconditions
Optimality conditionsNecessary condition: unrestricted case
(P){
minx∈R
f (x)
where f : R −→ R
A necessary condition isf ′(x) = 0 where
f ′(x) = limt→0
f (x+ t)− f (x)t
That means: Ifx is a solution of(P) thenf (x) = 0
If we know the expression off ′ we can solve the equationf ′(x) = 0 in order to have candidates of solutions
Warning!!!
![Page 95: Introduction to Optimization - CERMICS – Centre d ...cermics.enpc.fr/~delara/MABIES/pgajardo_optimization_IHP2013.pdf · Introduction Optimality conditions Introduction to Optimization](https://reader031.vdocuments.us/reader031/viewer/2022021903/5b9da90609d3f2de128ca4f5/html5/thumbnails/95.jpg)
Introduction
Optimalityconditions
Optimality conditionsNecessary condition: unrestricted case
(P){
minx∈R
f (x)
where f : R −→ R
A necessary condition isf ′(x) = 0 where
f ′(x) = limt→0
f (x+ t)− f (x)t
That means: Ifx is a solution of(P) thenf (x) = 0
If we know the expression off ′ we can solve the equationf ′(x) = 0 in order to have candidates of solutions
Warning!!!
![Page 96: Introduction to Optimization - CERMICS – Centre d ...cermics.enpc.fr/~delara/MABIES/pgajardo_optimization_IHP2013.pdf · Introduction Optimality conditions Introduction to Optimization](https://reader031.vdocuments.us/reader031/viewer/2022021903/5b9da90609d3f2de128ca4f5/html5/thumbnails/96.jpg)
Introduction
Optimalityconditions
Optimality conditionsNecessary conditions: restrictions
(P)
minx∈R
f (x)
a ≤ x ≤ b
where f : R −→ R
At a pointx observe thatf increase in the sense of (thesign of) f ′(x)
There are three posibilities:
an interior pointa < x < b is a solution. Thenf ′(x) = 0
x = a is a solution. Thenf ′(x) = f ′(a) ≥ 0
x = b is a solution. Thenf ′(x) = f ′(b) ≤ 0
Observe that iff ′(a) > 0 andf ′(b) < 0, then there existsa < x < b such thatf ′(x) = 0
![Page 97: Introduction to Optimization - CERMICS – Centre d ...cermics.enpc.fr/~delara/MABIES/pgajardo_optimization_IHP2013.pdf · Introduction Optimality conditions Introduction to Optimization](https://reader031.vdocuments.us/reader031/viewer/2022021903/5b9da90609d3f2de128ca4f5/html5/thumbnails/97.jpg)
Introduction
Optimalityconditions
Optimality conditionsNecessary conditions: restrictions
(P)
minx∈R
f (x)
a ≤ x ≤ b
where f : R −→ R
At a pointx observe thatf increase in the sense of (thesign of) f ′(x)
There are three posibilities:
an interior pointa < x < b is a solution. Thenf ′(x) = 0
x = a is a solution. Thenf ′(x) = f ′(a) ≥ 0
x = b is a solution. Thenf ′(x) = f ′(b) ≤ 0
Observe that iff ′(a) > 0 andf ′(b) < 0, then there existsa < x < b such thatf ′(x) = 0
![Page 98: Introduction to Optimization - CERMICS – Centre d ...cermics.enpc.fr/~delara/MABIES/pgajardo_optimization_IHP2013.pdf · Introduction Optimality conditions Introduction to Optimization](https://reader031.vdocuments.us/reader031/viewer/2022021903/5b9da90609d3f2de128ca4f5/html5/thumbnails/98.jpg)
Introduction
Optimalityconditions
Optimality conditionsNecessary conditions: restrictions
(P)
minx∈R
f (x)
a ≤ x ≤ b
where f : R −→ R
At a pointx observe thatf increase in the sense of (thesign of) f ′(x)
There are three posibilities:
an interior pointa < x < b is a solution. Thenf ′(x) = 0
x = a is a solution. Thenf ′(x) = f ′(a) ≥ 0
x = b is a solution. Thenf ′(x) = f ′(b) ≤ 0
Observe that iff ′(a) > 0 andf ′(b) < 0, then there existsa < x < b such thatf ′(x) = 0
![Page 99: Introduction to Optimization - CERMICS – Centre d ...cermics.enpc.fr/~delara/MABIES/pgajardo_optimization_IHP2013.pdf · Introduction Optimality conditions Introduction to Optimization](https://reader031.vdocuments.us/reader031/viewer/2022021903/5b9da90609d3f2de128ca4f5/html5/thumbnails/99.jpg)
Introduction
Optimalityconditions
Optimality conditionsNecessary conditions: restrictions
(P)
minx∈R
f (x)
a ≤ x ≤ b
where f : R −→ R
At a pointx observe thatf increase in the sense of (thesign of) f ′(x)
There are three posibilities:
an interior pointa < x < b is a solution. Thenf ′(x) = 0
x = a is a solution. Thenf ′(x) = f ′(a) ≥ 0
x = b is a solution. Thenf ′(x) = f ′(b) ≤ 0
Observe that iff ′(a) > 0 andf ′(b) < 0, then there existsa < x < b such thatf ′(x) = 0
![Page 100: Introduction to Optimization - CERMICS – Centre d ...cermics.enpc.fr/~delara/MABIES/pgajardo_optimization_IHP2013.pdf · Introduction Optimality conditions Introduction to Optimization](https://reader031.vdocuments.us/reader031/viewer/2022021903/5b9da90609d3f2de128ca4f5/html5/thumbnails/100.jpg)
Introduction
Optimalityconditions
Optimality conditionsNecessary conditions: restrictions
(P)
minx∈R
f (x)
a ≤ x ≤ b
where f : R −→ R
At a pointx observe thatf increase in the sense of (thesign of) f ′(x)
There are three posibilities:
an interior pointa < x < b is a solution. Thenf ′(x) = 0
x = a is a solution. Thenf ′(x) = f ′(a) ≥ 0
x = b is a solution. Thenf ′(x) = f ′(b) ≤ 0
Observe that iff ′(a) > 0 andf ′(b) < 0, then there existsa < x < b such thatf ′(x) = 0
![Page 101: Introduction to Optimization - CERMICS – Centre d ...cermics.enpc.fr/~delara/MABIES/pgajardo_optimization_IHP2013.pdf · Introduction Optimality conditions Introduction to Optimization](https://reader031.vdocuments.us/reader031/viewer/2022021903/5b9da90609d3f2de128ca4f5/html5/thumbnails/101.jpg)
Introduction
Optimalityconditions
Optimality conditionsNecessary conditions: restrictions
(P)
minx∈R
f (x)
a ≤ x ≤ b
where f : R −→ R
At a pointx observe thatf increase in the sense of (thesign of) f ′(x)
There are three posibilities:
an interior pointa < x < b is a solution. Thenf ′(x) = 0
x = a is a solution. Thenf ′(x) = f ′(a) ≥ 0
x = b is a solution. Thenf ′(x) = f ′(b) ≤ 0
Observe thatif f ′(a) > 0 andf ′(b) < 0, thenthere existsa < x < b such thatf ′(x) = 0
![Page 102: Introduction to Optimization - CERMICS – Centre d ...cermics.enpc.fr/~delara/MABIES/pgajardo_optimization_IHP2013.pdf · Introduction Optimality conditions Introduction to Optimization](https://reader031.vdocuments.us/reader031/viewer/2022021903/5b9da90609d3f2de128ca4f5/html5/thumbnails/102.jpg)
Introduction
Optimalityconditions
Optimality conditionsNecessary conditions: restrictions
A necessary condition of the problem
(P)
minx∈R
f (x)
a ≤ x ≤ b
is
(S)
Find x, α, β such that
f ′(x) − α+ β = 0
x ∈ [a,b]
α, β ≥ 0
α(x− a) = 0
β(x− b) = 0
![Page 103: Introduction to Optimization - CERMICS – Centre d ...cermics.enpc.fr/~delara/MABIES/pgajardo_optimization_IHP2013.pdf · Introduction Optimality conditions Introduction to Optimization](https://reader031.vdocuments.us/reader031/viewer/2022021903/5b9da90609d3f2de128ca4f5/html5/thumbnails/103.jpg)
Introduction
Optimalityconditions
Optimality conditionsNecessary conditions: restrictions
A necessary condition of the problem
(P)
minx∈R
f (x)
a ≤ x ≤ b
is
(S)
Find x, α, β such that
f ′(x) − α+ β = 0
x ∈ [a,b]
α, β ≥ 0
α(x− a) = 0
β(x− b) = 0
![Page 104: Introduction to Optimization - CERMICS – Centre d ...cermics.enpc.fr/~delara/MABIES/pgajardo_optimization_IHP2013.pdf · Introduction Optimality conditions Introduction to Optimization](https://reader031.vdocuments.us/reader031/viewer/2022021903/5b9da90609d3f2de128ca4f5/html5/thumbnails/104.jpg)
Introduction
Optimalityconditions
Optimality conditionsGlobal vs local minimum
(P)
minx∈X
f (x)
subject to
x ∈ C
We say thatx ∈ C is a global minimum, if it is a solutionof (P)
We say thatx ∈ C is a local minimum, if for everyx ∈ Cclose enough, one hasf (x) ≤ f (x)
A global optimum is a local optimum
![Page 105: Introduction to Optimization - CERMICS – Centre d ...cermics.enpc.fr/~delara/MABIES/pgajardo_optimization_IHP2013.pdf · Introduction Optimality conditions Introduction to Optimization](https://reader031.vdocuments.us/reader031/viewer/2022021903/5b9da90609d3f2de128ca4f5/html5/thumbnails/105.jpg)
Introduction
Optimalityconditions
Optimality conditionsGlobal vs local minimum
(P)
minx∈X
f (x)
subject to
x ∈ C
We say thatx ∈ C is a global minimum, if it is a solutionof (P)
We say thatx ∈ C is a local minimum, if for everyx ∈ Cclose enough, one hasf (x) ≤ f (x)
A global optimum is a local optimum
![Page 106: Introduction to Optimization - CERMICS – Centre d ...cermics.enpc.fr/~delara/MABIES/pgajardo_optimization_IHP2013.pdf · Introduction Optimality conditions Introduction to Optimization](https://reader031.vdocuments.us/reader031/viewer/2022021903/5b9da90609d3f2de128ca4f5/html5/thumbnails/106.jpg)
Introduction
Optimalityconditions
Optimality conditionsGlobal vs local minimum
(P)
minx∈X
f (x)
subject to
x ∈ C
We say thatx ∈ C is a global minimum, if it is a solutionof (P)
We say thatx ∈ C is a local minimum, if for everyx ∈ Cclose enough, one hasf (x) ≤ f (x)
A global optimum is a local optimum
![Page 107: Introduction to Optimization - CERMICS – Centre d ...cermics.enpc.fr/~delara/MABIES/pgajardo_optimization_IHP2013.pdf · Introduction Optimality conditions Introduction to Optimization](https://reader031.vdocuments.us/reader031/viewer/2022021903/5b9da90609d3f2de128ca4f5/html5/thumbnails/107.jpg)
Introduction
Optimalityconditions
Optimality conditionsSufficient condition: unrestricted case
(P){
minx∈R
f (x)
We say that(S) is a sufficient condition if a solution of(S)is a solution of(P)
The idea is to find solutions of(S) in order to havesolutions of(P)
In the general case, this is very difficult
It is much realistic to have that solutions of(S) are localsolutions of(P)
A (local) sufficient condition to(P) is f ′(x) = 0 andf ′′(x) > 0
![Page 108: Introduction to Optimization - CERMICS – Centre d ...cermics.enpc.fr/~delara/MABIES/pgajardo_optimization_IHP2013.pdf · Introduction Optimality conditions Introduction to Optimization](https://reader031.vdocuments.us/reader031/viewer/2022021903/5b9da90609d3f2de128ca4f5/html5/thumbnails/108.jpg)
Introduction
Optimalityconditions
Optimality conditionsSufficient condition: unrestricted case
(P){
minx∈R
f (x)
We say that(S) is a sufficient condition if a solution of(S)is a solution of(P)
The idea isto find solutions of(S) in order to havesolutions of(P)
In the general case, this is very difficult
It is much realistic to have that solutions of(S) are localsolutions of(P)
A (local) sufficient condition to(P) is f ′(x) = 0 andf ′′(x) > 0
![Page 109: Introduction to Optimization - CERMICS – Centre d ...cermics.enpc.fr/~delara/MABIES/pgajardo_optimization_IHP2013.pdf · Introduction Optimality conditions Introduction to Optimization](https://reader031.vdocuments.us/reader031/viewer/2022021903/5b9da90609d3f2de128ca4f5/html5/thumbnails/109.jpg)
Introduction
Optimalityconditions
Optimality conditionsSufficient condition: unrestricted case
(P){
minx∈R
f (x)
We say that(S) is a sufficient condition if a solution of(S)is a solution of(P)
The idea is to find solutions of(S) in order to havesolutions of(P)
In the general case, this isvery difficult
It is much realistic to have that solutions of(S) are localsolutions of(P)
A (local) sufficient condition to(P) is f ′(x) = 0 andf ′′(x) > 0
![Page 110: Introduction to Optimization - CERMICS – Centre d ...cermics.enpc.fr/~delara/MABIES/pgajardo_optimization_IHP2013.pdf · Introduction Optimality conditions Introduction to Optimization](https://reader031.vdocuments.us/reader031/viewer/2022021903/5b9da90609d3f2de128ca4f5/html5/thumbnails/110.jpg)
Introduction
Optimalityconditions
Optimality conditionsSufficient condition: unrestricted case
(P){
minx∈R
f (x)
We say that(S) is a sufficient condition if a solution of(S)is a solution of(P)
The idea is to find solutions of(S) in order to havesolutions of(P)
In the general case, this is very difficult
It is much realistic to have thatsolutions of(S) are localsolutions of(P)
A (local) sufficient condition to(P) is f ′(x) = 0 andf ′′(x) > 0
![Page 111: Introduction to Optimization - CERMICS – Centre d ...cermics.enpc.fr/~delara/MABIES/pgajardo_optimization_IHP2013.pdf · Introduction Optimality conditions Introduction to Optimization](https://reader031.vdocuments.us/reader031/viewer/2022021903/5b9da90609d3f2de128ca4f5/html5/thumbnails/111.jpg)
Introduction
Optimalityconditions
Optimality conditionsSufficient condition: unrestricted case
(P){
minx∈R
f (x)
We say that(S) is a sufficient condition if a solution of(S)is a solution of(P)
The idea is to find solutions of(S) in order to havesolutions of(P)
In the general case, this is very difficult
It is much realistic to have that solutions of(S) are localsolutions of(P)
A (local) sufficient condition to(P) is f ′(x) = 0 andf ′′(x) > 0
![Page 112: Introduction to Optimization - CERMICS – Centre d ...cermics.enpc.fr/~delara/MABIES/pgajardo_optimization_IHP2013.pdf · Introduction Optimality conditions Introduction to Optimization](https://reader031.vdocuments.us/reader031/viewer/2022021903/5b9da90609d3f2de128ca4f5/html5/thumbnails/112.jpg)
Introduction
Optimalityconditions
Optimality conditionsSufficient condition
If we find x ∈ R such that
(S)
{
f ′(x) = 0
f ′′(x) > 0
Thenx is a local minimum of the problem
(P){
minx∈R
f (x)
![Page 113: Introduction to Optimization - CERMICS – Centre d ...cermics.enpc.fr/~delara/MABIES/pgajardo_optimization_IHP2013.pdf · Introduction Optimality conditions Introduction to Optimization](https://reader031.vdocuments.us/reader031/viewer/2022021903/5b9da90609d3f2de128ca4f5/html5/thumbnails/113.jpg)
Introduction
Optimalityconditions
Optimality conditionsSufficient condition
If we find x ∈ R such that
(S)
{
f ′(x) = 0
f ′′(x) > 0
Thenx is a local minimum of the problem
(P){
minx∈R
f (x)
![Page 114: Introduction to Optimization - CERMICS – Centre d ...cermics.enpc.fr/~delara/MABIES/pgajardo_optimization_IHP2013.pdf · Introduction Optimality conditions Introduction to Optimization](https://reader031.vdocuments.us/reader031/viewer/2022021903/5b9da90609d3f2de128ca4f5/html5/thumbnails/114.jpg)
Introduction
Optimalityconditions
Conclusions
(P)
minx∈X
f (x)
subject to
x ∈ C
To formulate an optimization problem,the followingconcepts must be clear:
Decision variable(s):x ∈ X
Objective function:f : X −→ R
Restrictions:C ⊂ X
A maximization problem is equivalent to a minimizationproblem (deal with−f (x) instead off (x))
![Page 115: Introduction to Optimization - CERMICS – Centre d ...cermics.enpc.fr/~delara/MABIES/pgajardo_optimization_IHP2013.pdf · Introduction Optimality conditions Introduction to Optimization](https://reader031.vdocuments.us/reader031/viewer/2022021903/5b9da90609d3f2de128ca4f5/html5/thumbnails/115.jpg)
Introduction
Optimalityconditions
Conclusions
(P)
minx∈X
f (x)
subject to
x ∈ C
To formulate an optimization problem,the followingconcepts must be clear:
Decision variable(s): x ∈ X
Objective function:f : X −→ R
Restrictions:C ⊂ X
A maximization problem is equivalent to a minimizationproblem (deal with−f (x) instead off (x))
![Page 116: Introduction to Optimization - CERMICS – Centre d ...cermics.enpc.fr/~delara/MABIES/pgajardo_optimization_IHP2013.pdf · Introduction Optimality conditions Introduction to Optimization](https://reader031.vdocuments.us/reader031/viewer/2022021903/5b9da90609d3f2de128ca4f5/html5/thumbnails/116.jpg)
Introduction
Optimalityconditions
Conclusions
(P)
minx∈X
f (x)
subject to
x ∈ C
To formulate an optimization problem,the followingconcepts must be clear:
Decision variable(s):x ∈ X
Objective function: f : X −→ R
Restrictions:C ⊂ X
A maximization problem is equivalent to a minimizationproblem (deal with−f (x) instead off (x))
![Page 117: Introduction to Optimization - CERMICS – Centre d ...cermics.enpc.fr/~delara/MABIES/pgajardo_optimization_IHP2013.pdf · Introduction Optimality conditions Introduction to Optimization](https://reader031.vdocuments.us/reader031/viewer/2022021903/5b9da90609d3f2de128ca4f5/html5/thumbnails/117.jpg)
Introduction
Optimalityconditions
Conclusions
(P)
minx∈X
f (x)
subject to
x ∈ C
To formulate an optimization problem,the followingconcepts must be clear:
Decision variable(s):x ∈ X
Objective function:f : X −→ R
Restrictions: C ⊂ X
A maximization problem is equivalent to a minimizationproblem (deal with−f (x) instead off (x))
![Page 118: Introduction to Optimization - CERMICS – Centre d ...cermics.enpc.fr/~delara/MABIES/pgajardo_optimization_IHP2013.pdf · Introduction Optimality conditions Introduction to Optimization](https://reader031.vdocuments.us/reader031/viewer/2022021903/5b9da90609d3f2de128ca4f5/html5/thumbnails/118.jpg)
Introduction
Optimalityconditions
Conclusions
(P)
minx∈X
f (x)
subject to
x ∈ C
To formulate an optimization problem, the followingconcepts must be clear:
Decision variable(s):x ∈ X
Objective function:f : X −→ R
Restrictions:C ⊂ X
A maximization problem is equivalent to a minimizationproblem(deal with−f (x) instead off (x))
![Page 119: Introduction to Optimization - CERMICS – Centre d ...cermics.enpc.fr/~delara/MABIES/pgajardo_optimization_IHP2013.pdf · Introduction Optimality conditions Introduction to Optimization](https://reader031.vdocuments.us/reader031/viewer/2022021903/5b9da90609d3f2de128ca4f5/html5/thumbnails/119.jpg)
Introduction
Optimalityconditions
Conclusions
(P)
minx∈X
f (x)
subject to
x ∈ C
Optimality conditions are mathematical systems
The idea is to solve these systems in order to:
have candidates of solutions (if the system is a necessarycondition)
obtain a local solution (if the system is a sufficientcondition)
![Page 120: Introduction to Optimization - CERMICS – Centre d ...cermics.enpc.fr/~delara/MABIES/pgajardo_optimization_IHP2013.pdf · Introduction Optimality conditions Introduction to Optimization](https://reader031.vdocuments.us/reader031/viewer/2022021903/5b9da90609d3f2de128ca4f5/html5/thumbnails/120.jpg)
Introduction
Optimalityconditions
Conclusions
(P)
minx∈X
f (x)
subject to
x ∈ C
Optimality conditions are mathematical systems
The idea isto solve these systemsin order to:
have candidates of solutions (if the system is a necessarycondition)
obtain a local solution (if the system is a sufficientcondition)
![Page 121: Introduction to Optimization - CERMICS – Centre d ...cermics.enpc.fr/~delara/MABIES/pgajardo_optimization_IHP2013.pdf · Introduction Optimality conditions Introduction to Optimization](https://reader031.vdocuments.us/reader031/viewer/2022021903/5b9da90609d3f2de128ca4f5/html5/thumbnails/121.jpg)
Introduction
Optimalityconditions
Conclusions
(P)
minx∈X
f (x)
subject to
x ∈ C
Optimality conditions are mathematical systems
The idea isto solve these systemsin order to:
have candidates of solutions(if the system is anecessarycondition)
obtain a local solution (if the system is a sufficientcondition)
![Page 122: Introduction to Optimization - CERMICS – Centre d ...cermics.enpc.fr/~delara/MABIES/pgajardo_optimization_IHP2013.pdf · Introduction Optimality conditions Introduction to Optimization](https://reader031.vdocuments.us/reader031/viewer/2022021903/5b9da90609d3f2de128ca4f5/html5/thumbnails/122.jpg)
Introduction
Optimalityconditions
Conclusions
(P)
minx∈X
f (x)
subject to
x ∈ C
Optimality conditions are mathematical systems
The idea isto solve these systemsin order to:
have candidates of solutions (if the system is a necessarycondition)
obtain a local solution(if the system is asufficientcondition)
![Page 123: Introduction to Optimization - CERMICS – Centre d ...cermics.enpc.fr/~delara/MABIES/pgajardo_optimization_IHP2013.pdf · Introduction Optimality conditions Introduction to Optimization](https://reader031.vdocuments.us/reader031/viewer/2022021903/5b9da90609d3f2de128ca4f5/html5/thumbnails/123.jpg)
Introduction
Optimalityconditions
Next part
(P)
minx∈X
f (x)
subject to
x ∈ C
Several decision variablesx1, x2, . . . , xn. That is
x =
x1
x2...
xn
∈ Rn = X
Necessary and sufficient conditions
Why the convexity (or concavity) is relevant?
![Page 124: Introduction to Optimization - CERMICS – Centre d ...cermics.enpc.fr/~delara/MABIES/pgajardo_optimization_IHP2013.pdf · Introduction Optimality conditions Introduction to Optimization](https://reader031.vdocuments.us/reader031/viewer/2022021903/5b9da90609d3f2de128ca4f5/html5/thumbnails/124.jpg)
Introduction
Optimalityconditions
Next part
(P)
minx∈X
f (x)
subject to
x ∈ C
Several decision variablesx1, x2, . . . , xn. That is
x =
x1
x2...
xn
∈ Rn = X
Necessary and sufficient conditions
Why the convexity (or concavity) is relevant?
![Page 125: Introduction to Optimization - CERMICS – Centre d ...cermics.enpc.fr/~delara/MABIES/pgajardo_optimization_IHP2013.pdf · Introduction Optimality conditions Introduction to Optimization](https://reader031.vdocuments.us/reader031/viewer/2022021903/5b9da90609d3f2de128ca4f5/html5/thumbnails/125.jpg)
Introduction
Optimalityconditions
Next part
(P)
minx∈X
f (x)
subject to
x ∈ C
Several decision variablesx1, x2, . . . , xn. That is
x =
x1
x2...
xn
∈ Rn = X
Necessary and sufficient conditions
Why the convexity (or concavity) is relevant?