lecture 1 optimal control ws 2019/2020ifat · •uncertainty description and quantification focal...
Post on 27-Jul-2020
1 Views
Preview:
TRANSCRIPT
Opt
imal
Con
trol
201
9/20
20
Lecture 1
Optimal Control WS 2019/2020
Prof. Dr.-Ing. Rolf Findeisen
Laboratory for Systems Theory and Automatic Control
Opt
imal
Con
trol
201
9/20
20
Team
Lecturer
Prof. Rolf Findeisen
Dr. Navid Noroozi
Assistants
Hoang Hai Nguyen
Hannes Rewald
Mohamed Ibrahim
Process Control and ModelingAchim Kienle
Institute for Automation Otto-von-Guericke-University Magdeburg
3
• 65 employees, approx. 45 Ph.D. students• Close connection to• 7 research groups/chaired positions (at the University)
Autonomous Automation Systems Steffi Knorn
Integrated AutomationChristian Diedrich
Process AutomationUlrich Jumar
Measurement TechnologyUlrike Steinmann
Control of Distributed Parameter SystemsStefan Palis
Systems Theory and ControlRolf Findeisen
Research Activities• 6 Postdocs/scientific associates• 21 Phd students• Acad. coop. : MIT, EPFL, ETH, UC Berkley, Imperial Coll., DLR, ...• Industrial coop. : Airbus, Baker Hughes, Bosch, IAV, Siemens, Volkswagen,...
Fields of Applications• Autonomous Systems, Autonomous Driving• Energy Systems (Smart Grids, Batteries, Wind Energy…)• Robotics, Aerospace, UAV• Chemical Processes, Biotechnologies, Biopharmaceuticals
Theoretical Basis• Optimal and predictive control (MPC)• Machine Learning and Artificial Intelligence• Uncertainty Description and Quantification
Focal Research Areas• Fusing Machine Learning and Control with Guarantees• Distributed Systems, Modularization and Scalability, Cooperative Systems• Cyber Physical Systems, Network Controlled Systems• Fusion of Planning and Control• Identification, Verification, Validation• Embedded optimization
Goal: Development of theoretical sound control methods for save, flexible cooperative autonomous systems
Opt
imal
Con
trol
201
9/20
20
Objectives of the lecture
Objectives• Introduce main concepts and theory behind optimal control• Overview of numerical solution approaches• Overview of embedded optimization/model pred. control
Optimal control:Find an input/feedback for a dynamical system such that a performance measure is optimized satisfying constraints
?
?
Lecture is method and theory oriented!
Opt
imal
Con
trol
201
9/20
20
Organization
Format
• 2 SWS lecture, 1 SWS exercise• lectures and exercises are given in English
Lectures and exercises are handled flexible
• XXX XXX XXX• Wednesday, 17:00-19:00 G03-315• Thursday, 11:00-13:00 G02-109
Announcements/downloadshttp://ifatwww.et.uni-magdeburg.de/syst/education/courses/oc/
Exercises
• 4 exercises (classroom and computer) + 2 general Q&A sessions • exercise sheets are available approx. 1 week before exercise
Consultation hours
• make an appointment via emailNext forseen lectures/exercises
30.10, 17:00-18:00, 25.10, 1.11, 8.11
Opt
imal
Con
trol
201
9/20
20
Recommended literature
Optimal Control[1] R. Bellman. Dynamic Programming. Princeton University Press, Princeton, New Jersey, 1957. [2] L.D. Berkovitz. Optimal Control Theory. Springer-Verlag, New York, 1974. [3] D.P. Bertsekas. Dynamic Programming and Optimal Control. Athena Scientific Press. 2nd edition, 2000. [4] L.M. Hocking. Optimal Control. An Introduction to the Theory with Applications. Oxford Applied Mathematics and Computing Science Series. Oxford University Press, Oxford, 1991. [5] J.L. Troutmann. Variational Calculus and Optimal Control. Undergraduate Texts in Mathematics. Springer, 1991.
Optimization[6] S. Boyd, L. Vandenberghe. Convex Optimization. Cambridge University Press, 2004. [7] J. Nocedal, S. Wright. Numerical Optimization. Springer, 2006.
Model Predictive Control[8] J.B. Rawlings, D.Q. Mayne. Model Predictive Control: Theory and Design, 2009. [9] E.F. Camacho, C. Bordons. Model Predictive Control, Springer, 1995.
Opt
imal
Con
trol
201
9/20
20
Structure
1. IntroductionWhat is optimal control?Examples
2. Static Optimization
3. Basic Setup of optimal control problemsCost function, constraintsExistence of solutions
4. Analytic approaches to optimal controlDynamic ProgrammingProntryagin minimum principle
5. Numerical approaches to optimal controlDirect and indirect methodsConvex optimizationModel predictive control
6. Embedded model predictive controlEmbedded optimizationSolution of QPs for MPC on embedded platformsCode generation and implementation aspects
0 0 0 0 0 0
1 1 1 1
-1 -1 -1 -1
Stage 0 Stage 1 Stage 2 Stage 3 Stage 4 Stage 5
“Time“
“S tate“Initial-node
Terminal-node
5 4 1
5 1 5
2 3 41 3
44 1
11
2
2 2
2
233
3
3
CostSource: NASA
Opt
imal
Con
trol
201
9/20
20
Introduction to optimal control
•Optimal control?•Examples•Mathematical setup•Remarks on existence of solutions
Opt
imal
Con
trol
201
9/20
20
Optimal Control
Goal: „To optimize the operation of a dynamical system.“
„optimize“ could be minimization or maximization• Minimize cost/energy• Maximize profit• Minimize tracking error• …
Opt
imal
Con
trol
201
9/20
20
Optimal Control
Given a dynamical system , determine open-loop input or feedback , such that we are• minimizing performance objective• satisfying constraints: input constraints
state constraints
Key components: 1. model of the system2. constraints3. performance objective/functional
?
?
Opt
imal
Con
trol
201
9/20
20
Example (rocket car)
Objective: minimize fuel expense while bringing the car from to in a finite time ► there can be many solutions, or none, depending on constraints.
Mathematical:
s.t.:
no friction (designed with optimization methods)
Remarks: • is a function, not a single value; • is a functional of (in general of )
►we have to look at the differential equation (it is a problem!)
Opt
imal
Con
trol
201
9/20
20
Example:
Difference to static optimization
Static optimization:
s.t.
• No dynamical system• Finite dimensional
• u is not a function/no trajectory• F is not a functional
Opt
imal
Con
trol
201
9/20
20
More examples
Example (rocket car 2): bring the car in minimum time to the origin
Example (rocket car 3): minimize least square cost for fixed
is free
► Linear Quadratic Regulator (LQR)
Opt
imal
Con
trol
201
9/20
20
More examples: discrete time
Example (managing spending/savings): maximize consumption over n years
income per yearconsumed money
System model:
Objective:
Constraint:
given
Remarks:• discrete problem• end time fixed• is free
System model
Constraints
Objective
Opt
imal
Con
trol
201
9/20
20
More examples: traveling salesman (1800s, 1930s)
Traveling salesman problem: visit all cities/shortest path
► Shortest path problem• Very classical problem• Multi-stage optimization problem ► Bellman, Dynamic Programming• Finite dimensional• Still a lot of ongoing research
0 0 0 0 0 0
1 1 1 1
-1 -1 -1 -1
Stage 0 Stage 1 Stage 2 Stage 3 Stage 4 Stage 5
“Time“
“S tate“Initial-node
Terminal-node
5 4 1
5 1 5
2 3 41 3
44 1
11
2
2 2
2
233
3
3
Cost
Opt
imal
Con
trol
201
9/20
20
More examples: Goddard‘s Rocket Problem (1919)
How to use the throttle to send the rocket as high as as possible?
System model
Constraints Objective
Height (h) Mass (h)
Velocity (v) Thrust (u)
www.mcs.anl.gov
Opt
imal
Con
trol
201
9/20
20
More examples: penicillin production
System model
Constraints Objective
where
How to use the input to a reactor to produce as much penicillin as possible?
Opt
imal
Con
trol
201
9/20
20
Control in the era of communication
Technology advancements in many fields1. cheap communication
• high speed and reliability• affordable wireless communication
2. affordable computation & memory• computational power even in the smallest device• possibility to store and process (large) amounts of data
3. new sensors and actuators4. reliable batteries & improved energy efficiency
• How to control large networks of interconnected systems?• Smart energy networks• Smart buildings• Smart factories
• Optimal control and embedded optimization will play a key role
Opt
imal
Con
trol
201
9/20
20
Internet of things - Industry 4.0: The future?
Internet of things:• Even the smallest device will have the possibility
of computing control commands and communicating information
ECU
Embedded system
Si
Technology advancements open new possibilities for the future of computing
Industry 4.0:The fourth industrial revolution1. First revolution: mechanization steam engine (1800s)2. Second revolution: electrification electricity (1900s)3. Third revolution: digitalization computer (1960s)4. Fourth revolotion: computarization sens. Inf. & comp. (Today?)
Six design principles of Industry 4.0:1. Interoperatibility2. Virtualization3. Decentralization4. Real-time Capability5. Service Orientation6. Modularity
Opt
imal
Con
trol
201
9/20
20
Advantages and challenges optimal control
Advantages
• Systematic way to design controller/find optimal input► objective, model, constraints
• Consider constraints• Nonlinear systems• Nature behaves “optimal” (►Fermat‘s principle)
Drawbacks/challenges• Finding a solution is challenging
• Often not possible in an analytic way (only linear case trivial)• Numerical solution is necessary
• Existence? uniqueness?
Fields of applications:economics, aeronautics, mechanical engineering, chemical engineering, biology, medicine, electrical engineering, information technology, …
Opt
imal
Con
trol
201
9/20
20
Static Optimization
Opt
imal
Con
trol
201
9/20
20
Why static optimization for optimal control?
Direct approaches transform optimal control problems into static optimization problemsHow to solve static optimization problems?
Necessary and sufficient conditions for optimality• Unconstrained optimization• Constrained optimization
• Equality constraints• Inequality constraints
Usually iterative algorithms are based on derivative information
Opt
imal
Con
trol
201
9/20
20
Definition (Optimality):A feasible optimal point is optimal if
Optimal points are denoted by:
Static optimization
Static optimization: Find min (max) of a scalar objective function subject to constraints.
When satisfies (1) and (2), it is called a feasible point.
maximum?
Opt
imal
Con
trol
201
9/20
20
Example: ball and spring
Goal: find point of rest ►minimize potential energy of spring and ball
Opt
imal
Con
trol
201
9/20
20
Local / global minima: convexity
Finding a global minima or maxima is in general hard.
Opt
imal
Con
trol
201
9/20
20
Convexity of functions and sets
Definition (convex function): A function is called convex, if
Definition (convex set): A set is called convex, if for all
Opt
imal
Con
trol
201
9/20
20
Local / global minima: convexity
Theorem (convex problem):If is convex and if the feasible set is convex , then the optimization problem is convex and any local minimizer of is a global one
The feasible set is convex if:• Inequality constraints are convex• Equality constraints are affine
Opt
imal
Con
trol
201
9/20
20
• The intersection of an arbitrary number of convex sets is a convex set:
• The empty set is convex because it satisfies the definition of convexity.• The sub-level sets of a convex function are convex
• If are convex functions, then is a convex function for all
• A quadratic function is convex if and only if is positive semi definite.
• A quadratic function is strictly convex if and only if is positive definite.
Operations preserving convexity
Details, see e.g.Boyd, S. P. & Vandenberghe, L. Convex Optimization, University Press, Cambridge, 2004
Opt
imal
Con
trol
201
9/20
20
Example: constrained LQR
constrained LQR (linear quadratic regulator)
• constraints are affine• cost function convex iff
►usually convex problem
Opt
imal
Con
trol
201
9/20
20
We consider the problem
is twice continuously differentiable• is the gradient of • is the Hessian of
Unconstrained (multidimensional) static optimization
Theorem (necessary conditions for a local minimum)If is a local minimizer of then:• First order condition
►If is convex, this condition is also sufficient for a (global) minimum• Second order condition
Theorem (sufficient condition for a local minimum)If the following conditions are satisfied:
Then is a local minimizer of ( is locally convex around )
Opt
imal
Con
trol
201
9/20
20
Example: unconstrained ball and spring
Note: is convex.
Opt
imal
Con
trol
201
9/20
20
For difficult problems, finding an analytic solution is not possibleGeneral idea: find a sequence , which converges to the optimal vector
Iterative/numerical solution methods
Most methods are based on the following algorithm:• is a search direction and is the step length
1. Find a descent direction (the cost is decreasing)2. Pick step length (typically: line search)3. Set
Direct search: Search by evaluating Indirect search: Search by using derivative information
Opt
imal
Con
trol
201
9/20
20
Steepest descent approach/ gradient method
Based on linear approximation:Choose steepest descent direction:
Linear convergence:
Remarks:• Simple to calculate• Gradient required• Only linear convergence
Opt
imal
Con
trol
201
9/20
20
Quadratic convergence:
Newton-Raphson-method
Based on quadratic approximation:
Minimize with respect to
Remarks: • Gradient and Hessian required • Need to calculate the inverse• Much faster
Opt
imal
Con
trol
201
9/20
20
Constrained static optimization: Lagrangian
Problem setup:
Definition (Generalized Lagrangian)
Opt
imal
Con
trol
201
9/20
20
Duality
Primal Problem Dual Problem
Dual variables: Dual function:
Properties of the dual problem:• The optimal value of the dual problem is always a lower bound of the optimal value of the primal problem• The dual problem is always concave, even when the primal is not convex
Opt
imal
Con
trol
201
9/20
20
Duality gap
The difference between the optimal value of the primal problem and the optimal value of the dual problem is called duality gap:
For a convex problem the duality gap is 0In that case it is said that strong duality holds.• (Constraint qualification should also hold)
Duality gap
Opt
imal
Con
trol
201
9/20
20
Remark• It is often assumed that the constraints satisfy certain regularity conditions
at the optimum. These conditions are called constraint qualification (CQ)• The most common CQ condition is the linear independence constraint
qualification (LICQ)• The constraint gradients are linearly independent
Equality constrained static optimization
problem setup:
Does not satisfy LICQ
Opt
imal
Con
trol
201
9/20
20
Necessary conditions of optimality
Definition (Lagrangian):
Theorem (necessary conditions using the Lagrangian)If is a local minimum of subject to and the gradientsare linearly independent, then there exists a Lagrange multiplier vector s.t.• first order condition
• second order condition
Remark:If: , then the first and second order conditions are also sufficient
Opt
imal
Con
trol
201
9/20
20
Example: ball and springCooking recipe:
1. Build Lagrangian2. Calculate3. Solve for4. If necessary, check second
order conditions
Opt
imal
Con
trol
201
9/20
20
Inequality constrained problems
Problem setup:
Definition (Generalized Lagrangian)
Opt
imal
Con
trol
201
9/20
20
First order necessary conditions: KKT conditionsTheorem (Karush Kuhn Tucker conditions)Let be a regular point (i.e. and are linearly
independent respectively) and a local minimum. Then there exists and s.t.
The conditions are also sufficient for convex problems with strong duality
Opt
imal
Con
trol
201
9/20
20
Example: ball and spring
KKT conditions yield the equilibrium of forces
two inequalities
Opt
imal
Con
trol
201
9/20
20
Lagrange multiplier/shadow price
What happens if one relaxes a constraint?
more general way:
Therefore, Lagrange multipliers are known as shadow prices.
Opt
imal
Con
trol
201
9/20
20
Iterative solution methods for constrained optimization
Constrained optimization problem• Transform the problem directly to an unconstrained problem
► Solve unconstrained problem• Penalty method• Barrier method• Method of augmented Lagrangian
• Apply approximation methods to KKT conditions• SQP method (Newton‘s method applied to KKT conditions) .
Opt
imal
Con
trol
201
9/20
20
Idea: Approximate constrained problem
with an additional quadratic penalizing term
Quadratic penalty method
Example:
yields
Opt
imal
Con
trol
201
9/20
20
Quadratic penalty method (continued)
Algorithm:1. Choose an initial penalty factor (small) and an initial guess for 2. Solve
3. Increase and use as initial guess for and go back to 2.
Remarks:• Use algorithms to solve unconstrained problems• For equality constrained problems is often as differentiable as • In the case of inequality constaints the Hessian is not differentiable• Ill conditioned for big • For finite the method may yield infeasible solutions
Opt
imal
Con
trol
201
9/20
20
Logarithmic barrierIdea: approximate constrained problem using an extended cost function
Remarks:• Intermediate minimizers are feasible• The extended cost is at least as
smooth as the constraint function• A feasible inital guess is required• Optima at borders are hard to find• Problem of ill conditioning with growing
iterations steps • Add quadratic penalty for equality
constraints
Algorithm: solve for decreasing values of
Opt
imal
Con
trol
201
9/20
20
Idea: apply Newton‘s method to the KKT conditions of
the KKT conditions are
use Newton‘s method to find a solution
SQP methods (sequential quadratic programming)
First two terms of Taylor series of the KKT conditions around
solve for in each iteration step
Opt
imal
Con
trol
201
9/20
20
SQP methods (continued)
Alternative point of viewsolve at all iterates a quadratic approximation of the original problem with linearized equality constraints
Possible to extended to inequality constraints.
Remarks:• If inequality constraints, need to select active and inactive constraints• additional line search step is possible • need for efficient QP solvers• huge storage demand for• SQP is widely used for nonlinear problems
Opt
imal
Con
trol
201
9/20
20
Summary
• Field of static optimization well developed• Convexity plays key role• Lagrange multipliers to deal with constraints• Many good numerical/iterative solution approaches
• Transform to unconstrained problem• Penalty methods• Barrier methods
• SQP (sequential quadratic programming)• Finding a global maxima/minima challenging
top related