optimal control design techniques/ppt/padhi... · d. e. kirk, optimal control theory: an...
TRANSCRIPT
1
Optimal Control DesignOptimal Control DesignOptimal Control DesignOptimal Control Design
Prof. Radhakant PadhiProf. Radhakant PadhiProf. Radhakant PadhiProf. Radhakant Padhi
Dept. of Aerospace Engineering
Indian Institute of Science - Bangalore
Prof. Radhakant Padhi, IISc-Bangalore2
Acknowledgement:
Indian Institute of Science
� Founded in 1909 – more than 100 years old…
� Founded by J. N. Tata (in consultation with Swami Vivekananda) – land was donated by Mysore king.
� Deemed University in 1958
� More than 40 departments
� Ranked No.1 in India for higher education
� Only institute in India among best 100 in global ranking
For further information,
please visit www.iisc.ernet.in
20 September 2016
2
Collaboration & Research Funding
� Defense R&D Organisation (DRDO)
• Missile Complex (ASL, RCI, DRDL, ANURAG)
• ARDE
• CAIR
� Indian Space Research Organisation (ISRO)
• VSSC
• ISAC
� Air Force Research Lab (AFRL), USA
� Private Aerospace Companies
• Coral Digital Technologies
• Team Indus (Axiom Research Lab)
Prof. Radhakant Padhi, IISc-Bangalore 320 September 2016
Integrated Control Guidance and Estimation Lab (ICGEL)
& Aerospace Systems Lab (ASL)
Research Areas in ICGEL: • Nonlinear, Optimal & Adaptive Control
• Dynamic Inversion & Neuro-Adaptive Designs
• Single Network Adaptive Critic (SNAC)
Guidance and Control
of Missiles
Guidance and control
of UAVs
Feedback Control for
Customized Automatic
Drug Delivery
Dept. of Aerospace Engineering
Indian Institute of Science, Bangalore
Contact:
Prof. Radhakant Padhi
E. mail: [email protected]
• Drug is delivered as per
patient’s condition (not in
open loop) - Fast recovery
& Reduced side effects
• Demonstrated for blood
cancer, diabetes regulation
& Milk-fever of cows
• Guidance and Control for
automatic landing.
• Stereo Vision based
reactive collision avoidance
using ultra low-cost
cameras
• Nonlinear differential
geometric guidance for
collision avoidance
• Model Predictive Static Programming (MPSP)
• Online Modified (OM) Design for Enhanced Robustness
• State Estimation for Feedback Guidance & Control
Nonlinear & Neuro-
Adaptive Control of
High-Perf. Aircrafts
A new robust nonlinear
approach is developed
for better control of high
performance (large L/D)
aircrafts, which are
unstable in nature.
• Robust Formation flying
of satellites using online
modified real-time
optimal control
• Robust large attitude
maneuvers of satellites in
presence of significant
modelling errors
• MPSP and it variants are
used to develop optimal
guidance algorithm for
better performance .
Examples:
• Impact Angle Constrained
Guidance of Tactical
Missiles
• Integrated Guidance and
Control for Missiles for
Ballistic Missile Defence
Formation Flying and
Attitude Control of
Satellites
Current Team (2016)
13 Ph.D. Students, 1 Master Student
2 Project Associates, 2 Project Assistants
(many more in the past)
Optimal Process Control
3
Acknowledgement:
Graduated Students & Other Co-workers
� Mangal Kothari (Faculty in IIT-Kanpur)
� Arnab Maity (Faculty in IIT-Bombay)
� Sk. Faruque Ali (Faculty in IIT-Madras)
� Gurunath Gurala (Faculty in IISc-Bangalore)
� Harshal Oza (Faculty in Ahmedabad Univ., Ahmedabad)
� Prasiddha Nath Dwivedi (Scientist in DRDO, Hyderabad)
� Prem Kumar (Scientist in DRDO, Hyderabad)
� Girish Joshi (Former scientist in ISRO, doing his Ph.D. in USA)
� Kapil Sachan (currently a Ph.D. student)
� Avijit Banerjee (currently a Ph.D. student)
� Omkar Halbe (Working in EADS)
� Charu Chawla (Working in a Pvt. Company)…and many more!
20 September 2016 Prof. Radhakant Padhi, AE Dept., IISc-Bangalore 5
Outline
� Lecture – 1
• Generic Overview of Optimal Control Theory
� Lecture – 2
• Real-time Optimal Control using MPSP
� Lecture – 3
• Solution of Challenging Practical Problems
using MPSP
Prof. Radhakant Padhi, AE Dept., IISc-Bangalore 6
4
Lecture Lecture Lecture Lecture –––– 1111
An Overview of Optimal Control DesignAn Overview of Optimal Control DesignAn Overview of Optimal Control DesignAn Overview of Optimal Control Design
Prof. Radhakant PadhiProf. Radhakant PadhiProf. Radhakant PadhiProf. Radhakant Padhi
Dept. of Aerospace Engineering
Indian Institute of Science - Bangalore
8
Why Optimal Control?
Summary of Benefits
� A variety of difficult real-life problems can be formulated in the framework of optimal control.
� State and control bounds can be incorporated in the control design process explicitly.
� Incorporation of optimal issues lead to a variety of advantages, like minimum cost, maximum efficiency, non-conservative design etc.
� Trajectory planning issues can be incorporated into the guidance and control design.
Prof. Radhakant Padhi, AE Dept., IISc-Bangalore
5
Prof. Radhakant Padhi, AE Dept., IISc-Bangalore 9
Role of Optimal Control
Question: What is R(s)? How to design it??Unfortunately, books remain completely silent on this!
Optimization(Optimal Control)
Optimization(Optimal Control)
Mission Objectives
Prof. Radhakant Padhi, AE Dept., IISc-Bangalore 10
A Tribute to
Pioneers of Optimal Control
� 1700s
• Bernoulli, Newton
• Euler (Student of Bernoulli)
• Lagrange
....200 years later....
� 1900s
• Pontryagin
• Bellman
• Kalman
Bernoulli
Euler
Lagrange
Pontryagin
BellmanKalman
Newton
6
An Interesting Observation
� Euler (1726) - Lagrange - Fourier - Dirichlet -
Lipschitz - Klein [1A]
� Euler (1726) - Lagrange - Poisson - Dirichlet -
Lipschitz - Klein [1B]
� Gauss (1799) - Gerling - Pluecker - Klein [2]
>> Klein - Lindeman - Hilb - Baer - Liepman -
Bryson - Speyer - Bala - Padhi [3]
� Gauss (1799) - Bessel - Scherk - Kummer - Prym -
Rost - Baer - Liepman - Bryson - Speyer - Bala -
Padhi [4]
Prof. Radhakant Padhi, AE Dept., IISc-Bangalore
11
12
Optimal control formulation:
Key components
An optimal control formulation consists of:
• Performance index that needs to be optimized
• Appropriate boundary (initial & final) conditions
• Hard constraints
• Soft constraints
• Path constraints
• System dynamics constraint (nonlinear in general)
• State constraints
• Control constraints
Prof. Radhakant Padhi, AE Dept., IISc-Bangalore
7
Prof. Radhakant Padhi, AE Dept., IISc-Bangalore 13
Optimal Control Problem
Prof. Radhakant Padhi, AE Dept., IISc-Bangalore 14
Meaningful Performance Index
8
Prof. Radhakant Padhi, AE Dept., IISc-Bangalore 15
Meaningful Performance Index
Prof. Radhakant Padhi, AE Dept., IISc-Bangalore 16
Optimum of a Functional
9
Prof. Radhakant Padhi, AE Dept., IISc-Bangalore 17
Fundamental Theorem of
Calculus of Variations
Prof. Radhakant Padhi, AE Dept., IISc-Bangalore 18
Fundamental Lemma
10
Prof. Radhakant Padhi, AE Dept., IISc-Bangalore 19
Optimal Control Problem
� Performance Index (to minimize / maximize):
� Path Constraint:
� Boundary Conditions:
Prof. Radhakant Padhi, AE Dept., IISc-Bangalore 20
Necessary Conditions of
Optimality
� Augmented PI
� Hamiltonian
� First Variation
11
Prof. Radhakant Padhi, AE Dept., IISc-Bangalore 21
Necessary Conditions of
Optimality
� First Variation
� Individual terms
Prof. Radhakant Padhi, AE Dept., IISc-Bangalore 22
Necessary Conditions of
Optimality
0
12
Prof. Radhakant Padhi, AE Dept., IISc-Bangalore 23
Necessary Conditions of
Optimality
� First Variation
Prof. Radhakant Padhi, AE Dept., IISc-Bangalore 24
Necessary Conditions of
Optimality
� First Variation
13
Prof. Radhakant Padhi, AE Dept., IISc-Bangalore 25
Necessary Conditions of
Optimality: Summary
� State Equation
� Costate Equation
� Optimal Control Equation
� Boundary Condition
Prof. Radhakant Padhi, AE Dept., IISc-Bangalore 26
Necessary Conditions of
Optimality: Some Comments
� State and Costate equations are dynamic equations. If one is stable, the other turns out to be unstable!
� Optimal control equation is a stationary equation
� Boundary conditions are split: it leads to Two-Point-Boundary-Value Problem (TPBVP)
� State equation develops forward whereas Costate equation develops backwards.
� It is known as “Curse of Complexity” in optimal control
� Traditionally, TPBVPs demand computationally-intensive iterative numerical procedures, which lead to “open-loop” control structure.
14
Prof. Radhakant Padhi, AE Dept., IISc-Bangalore 27
General
Boundary/Transversality Condition
General condition:
Special Cases:
Example Example Example Example –––– 1: A Toy Problem1: A Toy Problem1: A Toy Problem1: A Toy Problem
Prof. Radhakant PadhiProf. Radhakant PadhiProf. Radhakant PadhiProf. Radhakant Padhi
Dept. of Aerospace Engineering
Indian Institute of Science - Bangalore
15
Prof. Radhakant Padhi, AE Dept., IISc-Bangalore 29
Example
Problem:
Solution:
Costate Eq.
Optimal control Eq.
Prof. Radhakant Padhi, AE Dept., IISc-Bangalore 30
Example
Boundary Conditions
Define
Solution
16
Prof. Radhakant Padhi, AE Dept., IISc-Bangalore 31
Example
Use the boundary condition at
Use the boundary condition at
Prof. Radhakant Padhi, AE Dept., IISc-Bangalore 32
Example
Four equations and four unknowns:
17
Prof. Radhakant Padhi, AE Dept., IISc-Bangalore 33
Example
� Solution for State and Costate
� Solution for Optimal Control
Example Example Example Example –––– 2: Orbit Transfer Problem2: Orbit Transfer Problem2: Orbit Transfer Problem2: Orbit Transfer Problem
Prof. Radhakant PadhiProf. Radhakant PadhiProf. Radhakant PadhiProf. Radhakant Padhi
Dept. of Aerospace Engineering
Indian Institute of Science - Bangalore
18
Prof. Radhakant Padhi, AE Dept., IISc-Bangalore 35
Example (Maximum Radius Orbit
Transfer at a Given Time)
Prof. Radhakant Padhi, AE Dept., IISc-Bangalore 36
Example (Maximum Radius Orbit
Transfer at a Given Time)
19
Prof. Radhakant Padhi, AE Dept., IISc-Bangalore 37
System Dynamics and B.C.
System dynamics Boundary conditions
Prof. Radhakant Padhi, AE Dept., IISc-Bangalore 38
Performance index
20
Prof. Radhakant Padhi, AE Dept., IISc-Bangalore 39
Necessary Condition
Prof. Radhakant Padhi, AE Dept., IISc-Bangalore 40
Necessary Condition
21
A Classical Numerical Approach for Solving A Classical Numerical Approach for Solving A Classical Numerical Approach for Solving A Classical Numerical Approach for Solving
Optimal Control Problems: Gradient MethodOptimal Control Problems: Gradient MethodOptimal Control Problems: Gradient MethodOptimal Control Problems: Gradient Method
Prof. Radhakant PadhiProf. Radhakant PadhiProf. Radhakant PadhiProf. Radhakant Padhi
Dept. of Aerospace Engineering
Indian Institute of Science - Bangalore
Prof. Radhakant Padhi, AE Dept., IISc-Bangalore 42
Gradient Method
� Assumptions:
• State equation satisfied
• Costate equation satisfied
• Boundary conditions satisfied
� Strategy:
• Satisfy the optimal control equation
22
Prof. Radhakant Padhi, AE Dept., IISc-Bangalore 43
Gradient Method
Prof. Radhakant Padhi, AE Dept., IISc-Bangalore 44
Gradient Method
� After satisfying the state & costate equations and boundary conditions, we have
� Select
� This leads to
23
Prof. Radhakant Padhi, AE Dept., IISc-Bangalore 45
Gradient Method
� We select
� This lead to
� Note:
� Eventually,
Prof. Radhakant Padhi, AE Dept., IISc-Bangalore 46
Gradient Method: Procedure
� Assume a control history (not a trivial task)
� Integrate the state equation forward
� Integrate the costate equation backward
� Update the control solution
• This can either be done at each step while integrating the costate equation backward or after the integration of the costate equation is complete
� Repeat the procedure until convergence
24
Prof. Radhakant Padhi, AE Dept., IISc-Bangalore 47
Gradient Method: Selection of
� Select so that it leads to a certain
percentage reduction of
� Let the percentage be
� Then
� This leads to
Dynamic Programming and Dynamic Programming and Dynamic Programming and Dynamic Programming and
HamiltonHamiltonHamiltonHamilton––––JacobiJacobiJacobiJacobi––––Bellman (HJB) TheoryBellman (HJB) TheoryBellman (HJB) TheoryBellman (HJB) Theory
Prof. Radhakant PadhiProf. Radhakant PadhiProf. Radhakant PadhiProf. Radhakant Padhi
Dept. of Aerospace Engineering
Indian Institute of Science - Bangalore
25
Prof. Radhakant Padhi, AE Dept., IISc-Bangalore 49
Fundamental Philosophy
Fundamental Theorem
Any part of an optimal trajectory is an optimal trajectory!
Motivation / Objective
To obtain a “state feedback” optimal control solution
A
C
B
Non-optimal path
Optimal path
Prof. Radhakant Padhi, AE Dept., IISc-Bangalore 50
Optimal Control Problem
26
Prof. Radhakant Padhi, AE Dept., IISc-Bangalore 51
Hamilton–Jacobi–Bellman (HJB)
Equation
Prof. Radhakant Padhi, AE Dept., IISc-Bangalore 52
Hamilton–Jacobi–Bellman (HJB)
Equation…contd.
27
Prof. Radhakant Padhi, AE Dept., IISc-Bangalore 53
Hamilton–Jacobi–Bellman (HJB)
Equation…contd.
Prof. Radhakant Padhi, AE Dept., IISc-Bangalore 54
Hamilton–Jacobi–Bellman (HJB)
Equation…contd.
28
Prof. Radhakant Padhi, AE Dept., IISc-Bangalore 55
Hamilton–Jacobi–Bellman (HJB)
Equation…contd.
Prof. Radhakant Padhi, AE Dept., IISc-Bangalore 56
Hamilton–Jacobi–Bellman (HJB)
Equation…contd.
29
Prof. Radhakant Padhi, AE Dept., IISc-Bangalore 57
Hamilton–Jacobi–Bellman (HJB)
Equation…contd.
Prof. Radhakant Padhi, AE Dept., IISc-Bangalore 58
Summary of HJB Equation
� Define optimized cost
function V as:
� Then V(t) must satisfy:
HJB equation
30
Prof. Radhakant Padhi, AE Dept., IISc-Bangalore 59
Dynamic Programming:
Some Relevant Results
Prof. Radhakant Padhi, AE Dept., IISc-Bangalore 60
Dynamic Programming:
Some Relevant Results
31
Example: A Benchmark Toy ProblemExample: A Benchmark Toy ProblemExample: A Benchmark Toy ProblemExample: A Benchmark Toy Problem
Prof. Radhakant PadhiProf. Radhakant PadhiProf. Radhakant PadhiProf. Radhakant Padhi
Dept. of Aerospace Engineering
Indian Institute of Science - Bangalore
Prof. Radhakant Padhi, AE Dept., IISc-Bangalore 62
Example
32
Prof. Radhakant Padhi, AE Dept., IISc-Bangalore 63
Example
Prof. Radhakant Padhi, AE Dept., IISc-Bangalore 64
Example
33
Prof. Radhakant Padhi, AE Dept., IISc-Bangalore 65
Example-2
Prof. Radhakant Padhi, AE Dept., IISc-Bangalore 66
Dynamic Programming:
Some Important Facts
� Dynamic programming is a powerful technique in the sense that if the HJB equation is solved, it leads to a “state feedback form” of optimal control solution.
� HJB equation is both necessary and sufficient for the optimal cost function.
� At least one of the control solutions that results from the solution of the HJB equation is
guaranteed to be stabilizing.
34
Prof. Radhakant Padhi, AE Dept., IISc-Bangalore 67
Dynamic Programming:
Some Important Facts
� The resulting PDE of the HJB equation is extremely difficult to solve in general.
� Dynamic Programming runs into a “huge” Computational and storage requirements for reasonably higher dimensional problems. This is a severe restriction of dynamic programming technique, which Bellman termed as “curse of dimensionality”.
Prof. Radhakant Padhi, AE Dept., IISc-Bangalore 68
Books on Optimal Control Design
� R. Padhi, Applied Optimal Control, Wiley, Manuscript
Under Preparation (expected in 2018).
� D. S. Naidu, Optimal Control Systems, CRC Press,
2002.
� D. E. Kirk, Optimal Control Theory: An Introduction,
Prentice Hall, 1970.
� A. E. Bryson and Y-C Ho, Applied Optimal Control,
Taylor and Francis, 1975.
� A. P. Sage and C. C. White III, Optimum Systems
Control (2nd Ed.), Prentice Hall, 1977.
35
Prof. Radhakant Padhi, AE Dept., IISc-Bangalore 69
Survey Papers on Classical
Methods for Optimal Control Design
� H. J. Pesch (1994), “A Practical Guide to the Solution of Real-Life Optimal Control Problems”, Control and Cybernetics, Vol.23, No.1/2, 1994, pp.7-60.
� R. E. Larson (1967), “A Survey of Dynamic Programming Computational Procedures”, IEEE Transactions on Automatic Control, December, pp. 767-774.
� M. Athans (1966), “The Status of Optimal Control Theory and Applications for Deterministic Systems”, IEEE Trans. on Automatic Control, Vol. AC-11, July 1966, pp.580-596.
Prof. Radhakant Padhi, AE Dept., IISc-Bangalore 70
Thanks for the Attention….!!