![Page 1: Francesco Borrelli Email: fborrelli@berkeley.edu ... · Berkeley Autonomous 1/10 Race Car Project Complete Open Source Ubuntu, RoS, OpenCV, Julia, IPOPT ... One task execution referred](https://reader034.vdocuments.us/reader034/viewer/2022050204/5f57a5ba1437da07145253ba/html5/thumbnails/1.jpg)
Borrelli (UC Berkeley) Iterative Learning MPC 2018 CDC– Slide 1
Iterative Learning Model Predictive Control
Francesco BorrelliEmail: [email protected]
University of CaliforniaBerkeley, USA
www.mpc.berkeley.edu
![Page 2: Francesco Borrelli Email: fborrelli@berkeley.edu ... · Berkeley Autonomous 1/10 Race Car Project Complete Open Source Ubuntu, RoS, OpenCV, Julia, IPOPT ... One task execution referred](https://reader034.vdocuments.us/reader034/viewer/2022050204/5f57a5ba1437da07145253ba/html5/thumbnails/2.jpg)
Borrelli (UC Berkeley) Iterative Learning MPC 2018 CDC– Slide 2
Acknowledgements
Learning MPC: Ugo Rosolia
Adaptive MPC: Monimoy Bujarbaruah, George Xiaojing Zhang
Autonomous Drift: Edo Jelavick, George Xiaojing Zhang
Analog Optimization: Sergey Vicky
Autonomous Drift: Edo Jelavick, Yuri Glauthier
Analog Optimization: Sergey Vicky
Connected Cars: Jacopo Guanetti, Jongsang Suh, Roya Firoozi, Yeojun Kim , Eric Choi
BARC: Jon Gonzales, Tony Zeng, Charlott Vallon
Research Sponsors:
Hyundai Corporation
Ford Research Labs, Siemens, Mobis, Komatzu
National Science Foundation
Office of Naval Research
![Page 3: Francesco Borrelli Email: fborrelli@berkeley.edu ... · Berkeley Autonomous 1/10 Race Car Project Complete Open Source Ubuntu, RoS, OpenCV, Julia, IPOPT ... One task execution referred](https://reader034.vdocuments.us/reader034/viewer/2022050204/5f57a5ba1437da07145253ba/html5/thumbnails/3.jpg)
Borrelli (UC Berkeley) Iterative Learning MPC 2018 CDC– Slide 3
Iterative Learning Model Predictive Control
![Page 4: Francesco Borrelli Email: fborrelli@berkeley.edu ... · Berkeley Autonomous 1/10 Race Car Project Complete Open Source Ubuntu, RoS, OpenCV, Julia, IPOPT ... One task execution referred](https://reader034.vdocuments.us/reader034/viewer/2022050204/5f57a5ba1437da07145253ba/html5/thumbnails/4.jpg)
Borrelli (UC Berkeley) Iterative Learning MPC 2018 CDC– Slide 4
Iterative Learning Model Predictive Control
![Page 5: Francesco Borrelli Email: fborrelli@berkeley.edu ... · Berkeley Autonomous 1/10 Race Car Project Complete Open Source Ubuntu, RoS, OpenCV, Julia, IPOPT ... One task execution referred](https://reader034.vdocuments.us/reader034/viewer/2022050204/5f57a5ba1437da07145253ba/html5/thumbnails/5.jpg)
Borrelli (UC Berkeley) Iterative Learning MPC 2018 CDC– Slide 5
Now Available on Amazon
![Page 6: Francesco Borrelli Email: fborrelli@berkeley.edu ... · Berkeley Autonomous 1/10 Race Car Project Complete Open Source Ubuntu, RoS, OpenCV, Julia, IPOPT ... One task execution referred](https://reader034.vdocuments.us/reader034/viewer/2022050204/5f57a5ba1437da07145253ba/html5/thumbnails/6.jpg)
Borrelli (UC Berkeley) Iterative Learning MPC 2018 CDC– Slide 6
Constrained Infinite-Time Optimal Control
“Solved” as..
![Page 7: Francesco Borrelli Email: fborrelli@berkeley.edu ... · Berkeley Autonomous 1/10 Race Car Project Complete Open Source Ubuntu, RoS, OpenCV, Julia, IPOPT ... One task execution referred](https://reader034.vdocuments.us/reader034/viewer/2022050204/5f57a5ba1437da07145253ba/html5/thumbnails/7.jpg)
Borrelli (UC Berkeley) Iterative Learning MPC 2018 CDC– Slide 7
Repeated Solution of Constrained Finite Time Optimal Control
Predictive Controller:
![Page 8: Francesco Borrelli Email: fborrelli@berkeley.edu ... · Berkeley Autonomous 1/10 Race Car Project Complete Open Source Ubuntu, RoS, OpenCV, Julia, IPOPT ... One task execution referred](https://reader034.vdocuments.us/reader034/viewer/2022050204/5f57a5ba1437da07145253ba/html5/thumbnails/8.jpg)
Borrelli (UC Berkeley) Iterative Learning MPC 2018 CDC– Slide 8
Repeated Solution of Constrained Finite Time Optimal Control
Approximates the `tail' of the cost Approximates the `tail' of the constraints
N constrained by computation and forecast uncertaintyRobust and stochastic versions subject of current research
![Page 9: Francesco Borrelli Email: fborrelli@berkeley.edu ... · Berkeley Autonomous 1/10 Race Car Project Complete Open Source Ubuntu, RoS, OpenCV, Julia, IPOPT ... One task execution referred](https://reader034.vdocuments.us/reader034/viewer/2022050204/5f57a5ba1437da07145253ba/html5/thumbnails/9.jpg)
Borrelli (UC Berkeley) Iterative Learning MPC 2018 CDC– Slide 9
Repeated Solution of Constrained Finite Time Optimal Control
Predictive Controller:
Predictive Control: Theory & Computation
![Page 10: Francesco Borrelli Email: fborrelli@berkeley.edu ... · Berkeley Autonomous 1/10 Race Car Project Complete Open Source Ubuntu, RoS, OpenCV, Julia, IPOPT ... One task execution referred](https://reader034.vdocuments.us/reader034/viewer/2022050204/5f57a5ba1437da07145253ba/html5/thumbnails/10.jpg)
Borrelli (UC Berkeley) Iterative Learning MPC 2018 CDC– Slide 10
Repeated Solution of Constrained Finite Time Optimal Control
Predictive Controller:
Predictive Control Classical Theory
![Page 11: Francesco Borrelli Email: fborrelli@berkeley.edu ... · Berkeley Autonomous 1/10 Race Car Project Complete Open Source Ubuntu, RoS, OpenCV, Julia, IPOPT ... One task execution referred](https://reader034.vdocuments.us/reader034/viewer/2022050204/5f57a5ba1437da07145253ba/html5/thumbnails/11.jpg)
Borrelli (UC Berkeley) Iterative Learning MPC 2018 CDC– Slide 11
Terminal cost: Control Lyapunov function
Terminal constraint set: Control Invariant set
Predictive Control Theory: Sufficient conditions to guarantee
Convergence to the desired equilibrium point/regionConstraint satisfaction at all times
Control Invariant Set
Control Lyapunov Function
![Page 12: Francesco Borrelli Email: fborrelli@berkeley.edu ... · Berkeley Autonomous 1/10 Race Car Project Complete Open Source Ubuntu, RoS, OpenCV, Julia, IPOPT ... One task execution referred](https://reader034.vdocuments.us/reader034/viewer/2022050204/5f57a5ba1437da07145253ba/html5/thumbnails/12.jpg)
Borrelli (UC Berkeley) Iterative Learning MPC 2018 CDC– Slide 12
Repeated Solution of Constrained Finite Time Optimal Control
Predictive Controller:
Predictive Control Computation
![Page 13: Francesco Borrelli Email: fborrelli@berkeley.edu ... · Berkeley Autonomous 1/10 Race Car Project Complete Open Source Ubuntu, RoS, OpenCV, Julia, IPOPT ... One task execution referred](https://reader034.vdocuments.us/reader034/viewer/2022050204/5f57a5ba1437da07145253ba/html5/thumbnails/13.jpg)
Borrelli (UC Berkeley) Iterative Learning MPC 2018 CDC– Slide 13
Offline π(∙) and Online π(x) Computation
Option 1 (Offline Based): “Complex” Offline, “Simple” Online
π0(∙) often piecewise constant or affine disturbance feedback
Dynamic Programing is one choice
Sampling model reduction/aggregation required for n>5
Option 2 (Online Based): “Simple” Offline, “Complex” Online
Compute on-line π 0(x(t)) with a “sophisticated” algorithm
Interior point method solver is one choice
Convexification required for real-time embedded control
![Page 14: Francesco Borrelli Email: fborrelli@berkeley.edu ... · Berkeley Autonomous 1/10 Race Car Project Complete Open Source Ubuntu, RoS, OpenCV, Julia, IPOPT ... One task execution referred](https://reader034.vdocuments.us/reader034/viewer/2022050204/5f57a5ba1437da07145253ba/html5/thumbnails/14.jpg)
Borrelli (UC Berkeley) Iterative Learning MPC 2018 CDC– Slide 14
Online Based
Excellent, (non-) convex open-source solvers
Tailored solvers for embedded linear and nonlinear MPC
Offline Based
For linear and piecewise linear systems: explicit MPC
Mixing pre-computation and online-optimization
Suboptimal MPC
Fast Online Implementation on embedded FPGA, GPU
Analog MPC: microsecond sampling time
Major effort over the past 20 years for enlarging MPC application domain A very biased story
![Page 15: Francesco Borrelli Email: fborrelli@berkeley.edu ... · Berkeley Autonomous 1/10 Race Car Project Complete Open Source Ubuntu, RoS, OpenCV, Julia, IPOPT ... One task execution referred](https://reader034.vdocuments.us/reader034/viewer/2022050204/5f57a5ba1437da07145253ba/html5/thumbnails/15.jpg)
Borrelli (UC Berkeley) Iterative Learning MPC 2018 CDC– Slide 15
Iterative Learning Model Predictive Control
![Page 16: Francesco Borrelli Email: fborrelli@berkeley.edu ... · Berkeley Autonomous 1/10 Race Car Project Complete Open Source Ubuntu, RoS, OpenCV, Julia, IPOPT ... One task execution referred](https://reader034.vdocuments.us/reader034/viewer/2022050204/5f57a5ba1437da07145253ba/html5/thumbnails/16.jpg)
Borrelli (UC Berkeley) Iterative Learning MPC 2018 CDC– Slide 16
Three Forms of Learning1 - Skill acquisition
![Page 17: Francesco Borrelli Email: fborrelli@berkeley.edu ... · Berkeley Autonomous 1/10 Race Car Project Complete Open Source Ubuntu, RoS, OpenCV, Julia, IPOPT ... One task execution referred](https://reader034.vdocuments.us/reader034/viewer/2022050204/5f57a5ba1437da07145253ba/html5/thumbnails/17.jpg)
Borrelli (UC Berkeley) Iterative Learning MPC 2018 CDC– Slide 17
Three Forms of Learning2 - Performance Improvement
![Page 18: Francesco Borrelli Email: fborrelli@berkeley.edu ... · Berkeley Autonomous 1/10 Race Car Project Complete Open Source Ubuntu, RoS, OpenCV, Julia, IPOPT ... One task execution referred](https://reader034.vdocuments.us/reader034/viewer/2022050204/5f57a5ba1437da07145253ba/html5/thumbnails/18.jpg)
Borrelli (UC Berkeley) Iterative Learning MPC 2018 CDC– Slide 18
Three Forms of Learning3 - Computation Load Reduction
![Page 19: Francesco Borrelli Email: fborrelli@berkeley.edu ... · Berkeley Autonomous 1/10 Race Car Project Complete Open Source Ubuntu, RoS, OpenCV, Julia, IPOPT ... One task execution referred](https://reader034.vdocuments.us/reader034/viewer/2022050204/5f57a5ba1437da07145253ba/html5/thumbnails/19.jpg)
Borrelli (UC Berkeley) Iterative Learning MPC 2018 CDC– Slide 19
Three Forms of Learning.Practice in order to:
Learning from demonstrationTransfer learning Learning from simulations
Iterative Learning Computational load reduction of control policy
Acquire a Skill Improve Performance Reduce Computational Load
![Page 20: Francesco Borrelli Email: fborrelli@berkeley.edu ... · Berkeley Autonomous 1/10 Race Car Project Complete Open Source Ubuntu, RoS, OpenCV, Julia, IPOPT ... One task execution referred](https://reader034.vdocuments.us/reader034/viewer/2022050204/5f57a5ba1437da07145253ba/html5/thumbnails/20.jpg)
Borrelli (UC Berkeley) Iterative Learning MPC 2018 CDC– Slide 20
Three Forms of Learning.Practice in order to:
Learning from demonstrationTransfer learning Learning from simulations
Iterative Learning Computational reduction of control policy
Acquire a Skill Improve Performance Reduce Computational Load
![Page 21: Francesco Borrelli Email: fborrelli@berkeley.edu ... · Berkeley Autonomous 1/10 Race Car Project Complete Open Source Ubuntu, RoS, OpenCV, Julia, IPOPT ... One task execution referred](https://reader034.vdocuments.us/reader034/viewer/2022050204/5f57a5ba1437da07145253ba/html5/thumbnails/21.jpg)
Borrelli (UC Berkeley) Iterative Learning MPC 2018 CDC– Slide 21
Learning MPC Applied to Robo-Cars (instead of robo-soccer players..)
![Page 22: Francesco Borrelli Email: fborrelli@berkeley.edu ... · Berkeley Autonomous 1/10 Race Car Project Complete Open Source Ubuntu, RoS, OpenCV, Julia, IPOPT ... One task execution referred](https://reader034.vdocuments.us/reader034/viewer/2022050204/5f57a5ba1437da07145253ba/html5/thumbnails/22.jpg)
Borrelli (UC Berkeley) Iterative Learning MPC 2018 CDC– Slide 22
Autonomous Cars @MPC Lab
![Page 23: Francesco Borrelli Email: fborrelli@berkeley.edu ... · Berkeley Autonomous 1/10 Race Car Project Complete Open Source Ubuntu, RoS, OpenCV, Julia, IPOPT ... One task execution referred](https://reader034.vdocuments.us/reader034/viewer/2022050204/5f57a5ba1437da07145253ba/html5/thumbnails/23.jpg)
Borrelli (UC Berkeley) Iterative Learning MPC 2018 CDC– Slide 23
Autonomous Vehicles- Motion Control Through:
Acceleration, Braking, SteeringAlso:
4 braking torquesGear RatioEngine torque + front and rear distribution4 dampers for active suspensions
Hyundai Genesis G70
![Page 24: Francesco Borrelli Email: fborrelli@berkeley.edu ... · Berkeley Autonomous 1/10 Race Car Project Complete Open Source Ubuntu, RoS, OpenCV, Julia, IPOPT ... One task execution referred](https://reader034.vdocuments.us/reader034/viewer/2022050204/5f57a5ba1437da07145253ba/html5/thumbnails/24.jpg)
Borrelli (UC Berkeley) Iterative Learning MPC 2018 CDC– Slide 24
Static Nonlinearities: Tires
Inequality Constraints: Safety region
Uncertain Tire Model, Road Friction, Obstacles
Nonlinear Dynamical System
Useful Model Abstraction
and
![Page 25: Francesco Borrelli Email: fborrelli@berkeley.edu ... · Berkeley Autonomous 1/10 Race Car Project Complete Open Source Ubuntu, RoS, OpenCV, Julia, IPOPT ... One task execution referred](https://reader034.vdocuments.us/reader034/viewer/2022050204/5f57a5ba1437da07145253ba/html5/thumbnails/25.jpg)
Borrelli (UC Berkeley) Iterative Learning MPC 2018 CDC– Slide 25
Tires and RoadSimplified Nonlinear Model
Slip Angle
Lateral ForceDry Asphalt
Snow
Ice
![Page 26: Francesco Borrelli Email: fborrelli@berkeley.edu ... · Berkeley Autonomous 1/10 Race Car Project Complete Open Source Ubuntu, RoS, OpenCV, Julia, IPOPT ... One task execution referred](https://reader034.vdocuments.us/reader034/viewer/2022050204/5f57a5ba1437da07145253ba/html5/thumbnails/26.jpg)
Borrelli (UC Berkeley) Iterative Learning MPC 2018 CDC– Slide 28
Berkeley Autonomous 1/10 Race Car Projectwww.barc-project.com
Complete Open SourceUbuntu, RoS, OpenCV, Julia, IPOPTCamera, IMU, Ultrasounds, LIDARCloud-Based
![Page 27: Francesco Borrelli Email: fborrelli@berkeley.edu ... · Berkeley Autonomous 1/10 Race Car Project Complete Open Source Ubuntu, RoS, OpenCV, Julia, IPOPT ... One task execution referred](https://reader034.vdocuments.us/reader034/viewer/2022050204/5f57a5ba1437da07145253ba/html5/thumbnails/27.jpg)
Borrelli (UC Berkeley) Iterative Learning MPC 2018 CDC– Slide 30
Three Forms of Learning1 - Skill acquisition
![Page 28: Francesco Borrelli Email: fborrelli@berkeley.edu ... · Berkeley Autonomous 1/10 Race Car Project Complete Open Source Ubuntu, RoS, OpenCV, Julia, IPOPT ... One task execution referred](https://reader034.vdocuments.us/reader034/viewer/2022050204/5f57a5ba1437da07145253ba/html5/thumbnails/28.jpg)
Borrelli (UC Berkeley) Iterative Learning MPC 2018 CDC– Slide 31
Three Forms of Learning2 - Performance Improvement
![Page 29: Francesco Borrelli Email: fborrelli@berkeley.edu ... · Berkeley Autonomous 1/10 Race Car Project Complete Open Source Ubuntu, RoS, OpenCV, Julia, IPOPT ... One task execution referred](https://reader034.vdocuments.us/reader034/viewer/2022050204/5f57a5ba1437da07145253ba/html5/thumbnails/29.jpg)
Borrelli (UC Berkeley) Iterative Learning MPC 2018 CDC– Slide 32
Three Forms of Learning3 - Computation Load Reduction
Average CPU Load at each iteration
Lap Time at each iteration
![Page 30: Francesco Borrelli Email: fborrelli@berkeley.edu ... · Berkeley Autonomous 1/10 Race Car Project Complete Open Source Ubuntu, RoS, OpenCV, Julia, IPOPT ... One task execution referred](https://reader034.vdocuments.us/reader034/viewer/2022050204/5f57a5ba1437da07145253ba/html5/thumbnails/30.jpg)
Borrelli (UC Berkeley) Iterative Learning MPC 2018 CDC– Slide 34
Three Forms of Learning
How we do this?
Model Predictive ControlA Simple Idea (which exploits the iterative nature of the tasks)
Important Design Steps
Acquire a Skill Improve Performance Reduce Computational Load
![Page 31: Francesco Borrelli Email: fborrelli@berkeley.edu ... · Berkeley Autonomous 1/10 Race Car Project Complete Open Source Ubuntu, RoS, OpenCV, Julia, IPOPT ... One task execution referred](https://reader034.vdocuments.us/reader034/viewer/2022050204/5f57a5ba1437da07145253ba/html5/thumbnails/31.jpg)
Borrelli (UC Berkeley) Iterative Learning MPC 2018 CDC– Slide 35
Iterative Learning Model Predictive Control
![Page 32: Francesco Borrelli Email: fborrelli@berkeley.edu ... · Berkeley Autonomous 1/10 Race Car Project Complete Open Source Ubuntu, RoS, OpenCV, Julia, IPOPT ... One task execution referred](https://reader034.vdocuments.us/reader034/viewer/2022050204/5f57a5ba1437da07145253ba/html5/thumbnails/32.jpg)
Borrelli (UC Berkeley) Iterative Learning MPC 2018 CDC– Slide 36
One task execution referred to as “iteration” or “episode”Same initial and terminal state at each iterationNotation:
Iterative Tasks - Problem Setup
![Page 33: Francesco Borrelli Email: fborrelli@berkeley.edu ... · Berkeley Autonomous 1/10 Race Car Project Complete Open Source Ubuntu, RoS, OpenCV, Julia, IPOPT ... One task execution referred](https://reader034.vdocuments.us/reader034/viewer/2022050204/5f57a5ba1437da07145253ba/html5/thumbnails/33.jpg)
Borrelli (UC Berkeley) Iterative Learning MPC 2018 CDC– Slide 37
Iterative Tasks - Problem Setup
One task execution referred to as “iteration” or “episode”Same initial and terminal state at each iterationNotation:
![Page 34: Francesco Borrelli Email: fborrelli@berkeley.edu ... · Berkeley Autonomous 1/10 Race Car Project Complete Open Source Ubuntu, RoS, OpenCV, Julia, IPOPT ... One task execution referred](https://reader034.vdocuments.us/reader034/viewer/2022050204/5f57a5ba1437da07145253ba/html5/thumbnails/34.jpg)
Borrelli (UC Berkeley) Iterative Learning MPC 2018 CDC– Slide 38
Iterative Tasks - Problem Setup
One task execution referred to as “iteration” or “episode”Same initial and terminal state at each iterationNotation:
![Page 35: Francesco Borrelli Email: fborrelli@berkeley.edu ... · Berkeley Autonomous 1/10 Race Car Project Complete Open Source Ubuntu, RoS, OpenCV, Julia, IPOPT ... One task execution referred](https://reader034.vdocuments.us/reader034/viewer/2022050204/5f57a5ba1437da07145253ba/html5/thumbnails/35.jpg)
Borrelli (UC Berkeley) Iterative Learning MPC 2018 CDC– Slide 39
Goal
Safety guarantees:
Constraint satisfaction at iteration j → satisfaction at iteration j+1
Performance improvement guarantees:
Closed loop cost at iteration j+1 ≤cost at iteration j
Learned fromdata
Iterative Learning MPCIncorporating data in advanced model based controller
![Page 36: Francesco Borrelli Email: fborrelli@berkeley.edu ... · Berkeley Autonomous 1/10 Race Car Project Complete Open Source Ubuntu, RoS, OpenCV, Julia, IPOPT ... One task execution referred](https://reader034.vdocuments.us/reader034/viewer/2022050204/5f57a5ba1437da07145253ba/html5/thumbnails/36.jpg)
Borrelli (UC Berkeley) Iterative Learning MPC 2018 CDC– Slide 40
Learned fromdata
Learning MPCIncorporating data in advance model based controller
Simplification (general case later)
Known/nominal model
Infinite Horizon Task
Uncertainty and model adaptation later (and at this conference)
![Page 37: Francesco Borrelli Email: fborrelli@berkeley.edu ... · Berkeley Autonomous 1/10 Race Car Project Complete Open Source Ubuntu, RoS, OpenCV, Julia, IPOPT ... One task execution referred](https://reader034.vdocuments.us/reader034/viewer/2022050204/5f57a5ba1437da07145253ba/html5/thumbnails/37.jpg)
Borrelli (UC Berkeley) Iterative Learning MPC 2018 CDC– Slide 43
Learning Model Predictive Control (LMPC)
• Recursive feasibility• Iterative feasibility
![Page 38: Francesco Borrelli Email: fborrelli@berkeley.edu ... · Berkeley Autonomous 1/10 Race Car Project Complete Open Source Ubuntu, RoS, OpenCV, Julia, IPOPT ... One task execution referred](https://reader034.vdocuments.us/reader034/viewer/2022050204/5f57a5ba1437da07145253ba/html5/thumbnails/38.jpg)
Borrelli (UC Berkeley) Iterative Learning MPC 2018 CDC– Slide 44
Iteration 0
Assume at iteration 0 the closed-loop trajectory is feasible
![Page 39: Francesco Borrelli Email: fborrelli@berkeley.edu ... · Berkeley Autonomous 1/10 Race Car Project Complete Open Source Ubuntu, RoS, OpenCV, Julia, IPOPT ... One task execution referred](https://reader034.vdocuments.us/reader034/viewer/2022050204/5f57a5ba1437da07145253ba/html5/thumbnails/39.jpg)
Borrelli (UC Berkeley) Iterative Learning MPC 2018 CDC– Slide 45
Iteration 0
Assume at iteration 0 the closed-loop trajectory is feasible
Fact
is a control invariant
![Page 40: Francesco Borrelli Email: fborrelli@berkeley.edu ... · Berkeley Autonomous 1/10 Race Car Project Complete Open Source Ubuntu, RoS, OpenCV, Julia, IPOPT ... One task execution referred](https://reader034.vdocuments.us/reader034/viewer/2022050204/5f57a5ba1437da07145253ba/html5/thumbnails/40.jpg)
Borrelli (UC Berkeley) Iterative Learning MPC 2018 CDC– Slide 46
Iteration 1, Step 0
Use SS0 as terminal set at Iteration 1
![Page 41: Francesco Borrelli Email: fborrelli@berkeley.edu ... · Berkeley Autonomous 1/10 Race Car Project Complete Open Source Ubuntu, RoS, OpenCV, Julia, IPOPT ... One task execution referred](https://reader034.vdocuments.us/reader034/viewer/2022050204/5f57a5ba1437da07145253ba/html5/thumbnails/41.jpg)
Borrelli (UC Berkeley) Iterative Learning MPC 2018 CDC– Slide 47
Iteration 1, Step 0
Use SS0 as terminal set at Iteration 1
![Page 42: Francesco Borrelli Email: fborrelli@berkeley.edu ... · Berkeley Autonomous 1/10 Race Car Project Complete Open Source Ubuntu, RoS, OpenCV, Julia, IPOPT ... One task execution referred](https://reader034.vdocuments.us/reader034/viewer/2022050204/5f57a5ba1437da07145253ba/html5/thumbnails/42.jpg)
Borrelli (UC Berkeley) Iterative Learning MPC 2018 CDC– Slide 48
Iteration 1, Step 1
Use SS0 as terminal set at Iteration 1
![Page 43: Francesco Borrelli Email: fborrelli@berkeley.edu ... · Berkeley Autonomous 1/10 Race Car Project Complete Open Source Ubuntu, RoS, OpenCV, Julia, IPOPT ... One task execution referred](https://reader034.vdocuments.us/reader034/viewer/2022050204/5f57a5ba1437da07145253ba/html5/thumbnails/43.jpg)
Borrelli (UC Berkeley) Iterative Learning MPC 2018 CDC– Slide 49
Iteration 1, Step 1
Use SS0 as terminal set at Iteration 1
![Page 44: Francesco Borrelli Email: fborrelli@berkeley.edu ... · Berkeley Autonomous 1/10 Race Car Project Complete Open Source Ubuntu, RoS, OpenCV, Julia, IPOPT ... One task execution referred](https://reader034.vdocuments.us/reader034/viewer/2022050204/5f57a5ba1437da07145253ba/html5/thumbnails/44.jpg)
Borrelli (UC Berkeley) Iterative Learning MPC 2018 CDC– Slide 50
Iteration 1, Step 2
Use SS0 as terminal set at Iteration 1
![Page 45: Francesco Borrelli Email: fborrelli@berkeley.edu ... · Berkeley Autonomous 1/10 Race Car Project Complete Open Source Ubuntu, RoS, OpenCV, Julia, IPOPT ... One task execution referred](https://reader034.vdocuments.us/reader034/viewer/2022050204/5f57a5ba1437da07145253ba/html5/thumbnails/45.jpg)
Borrelli (UC Berkeley) Iterative Learning MPC 2018 CDC– Slide 51
Iteration 1, Step 2
Use SS0 as terminal set at Iteration 1
![Page 46: Francesco Borrelli Email: fborrelli@berkeley.edu ... · Berkeley Autonomous 1/10 Race Car Project Complete Open Source Ubuntu, RoS, OpenCV, Julia, IPOPT ... One task execution referred](https://reader034.vdocuments.us/reader034/viewer/2022050204/5f57a5ba1437da07145253ba/html5/thumbnails/46.jpg)
Borrelli (UC Berkeley) Iterative Learning MPC 2018 CDC– Slide 52
Iteration 1, Step 3
Use SS0 as terminal set at Iteration 1
![Page 47: Francesco Borrelli Email: fborrelli@berkeley.edu ... · Berkeley Autonomous 1/10 Race Car Project Complete Open Source Ubuntu, RoS, OpenCV, Julia, IPOPT ... One task execution referred](https://reader034.vdocuments.us/reader034/viewer/2022050204/5f57a5ba1437da07145253ba/html5/thumbnails/47.jpg)
Borrelli (UC Berkeley) Iterative Learning MPC 2018 CDC– Slide 53
Iteration 1, Step 4
Use SS0 as terminal set at Iteration 1
![Page 48: Francesco Borrelli Email: fborrelli@berkeley.edu ... · Berkeley Autonomous 1/10 Race Car Project Complete Open Source Ubuntu, RoS, OpenCV, Julia, IPOPT ... One task execution referred](https://reader034.vdocuments.us/reader034/viewer/2022050204/5f57a5ba1437da07145253ba/html5/thumbnails/48.jpg)
Borrelli (UC Berkeley) Iterative Learning MPC 2018 CDC– Slide 54
Iteration 2 Safe Set
![Page 49: Francesco Borrelli Email: fborrelli@berkeley.edu ... · Berkeley Autonomous 1/10 Race Car Project Complete Open Source Ubuntu, RoS, OpenCV, Julia, IPOPT ... One task execution referred](https://reader034.vdocuments.us/reader034/viewer/2022050204/5f57a5ba1437da07145253ba/html5/thumbnails/49.jpg)
Borrelli (UC Berkeley) Iterative Learning MPC 2018 CDC– Slide 55
Constructing the terminal set
![Page 50: Francesco Borrelli Email: fborrelli@berkeley.edu ... · Berkeley Autonomous 1/10 Race Car Project Complete Open Source Ubuntu, RoS, OpenCV, Julia, IPOPT ... One task execution referred](https://reader034.vdocuments.us/reader034/viewer/2022050204/5f57a5ba1437da07145253ba/html5/thumbnails/50.jpg)
Borrelli (UC Berkeley) Iterative Learning MPC 2018 CDC– Slide 56
Terminal Set : Convex all of Sample Safe Set
for Constrained Linear Dynamical Systems is a Control Invariant Set
![Page 51: Francesco Borrelli Email: fborrelli@berkeley.edu ... · Berkeley Autonomous 1/10 Race Car Project Complete Open Source Ubuntu, RoS, OpenCV, Julia, IPOPT ... One task execution referred](https://reader034.vdocuments.us/reader034/viewer/2022050204/5f57a5ba1437da07145253ba/html5/thumbnails/51.jpg)
Borrelli (UC Berkeley) Iterative Learning MPC 2018 CDC– Slide 57
Learning Model Predictive Control (LMPC)
• Convergence• Performance
improvement• Local optimality
![Page 52: Francesco Borrelli Email: fborrelli@berkeley.edu ... · Berkeley Autonomous 1/10 Race Car Project Complete Open Source Ubuntu, RoS, OpenCV, Julia, IPOPT ... One task execution referred](https://reader034.vdocuments.us/reader034/viewer/2022050204/5f57a5ba1437da07145253ba/html5/thumbnails/52.jpg)
Borrelli (UC Berkeley) Iterative Learning MPC 2018 CDC– Slide 58
Terminal Cost at Iteration 0
![Page 53: Francesco Borrelli Email: fborrelli@berkeley.edu ... · Berkeley Autonomous 1/10 Race Car Project Complete Open Source Ubuntu, RoS, OpenCV, Julia, IPOPT ... One task execution referred](https://reader034.vdocuments.us/reader034/viewer/2022050204/5f57a5ba1437da07145253ba/html5/thumbnails/53.jpg)
Borrelli (UC Berkeley) Iterative Learning MPC 2018 CDC– Slide 59
Terminal Cost at Iteration 0
A control Lyapunov “function”
![Page 54: Francesco Borrelli Email: fborrelli@berkeley.edu ... · Berkeley Autonomous 1/10 Race Car Project Complete Open Source Ubuntu, RoS, OpenCV, Julia, IPOPT ... One task execution referred](https://reader034.vdocuments.us/reader034/viewer/2022050204/5f57a5ba1437da07145253ba/html5/thumbnails/54.jpg)
Borrelli (UC Berkeley) Iterative Learning MPC 2018 CDC– Slide 60
Terminal Cost at the j-th iteration
≡
Define
Compute terminal cost as
![Page 55: Francesco Borrelli Email: fborrelli@berkeley.edu ... · Berkeley Autonomous 1/10 Race Car Project Complete Open Source Ubuntu, RoS, OpenCV, Julia, IPOPT ... One task execution referred](https://reader034.vdocuments.us/reader034/viewer/2022050204/5f57a5ba1437da07145253ba/html5/thumbnails/55.jpg)
Borrelli (UC Berkeley) Iterative Learning MPC 2018 CDC– Slide 61
Terminal Cost: Barycentric Approximation of Q()
Control Lyapunov Function
(for Constrained Linear Dynamical Systems)
![Page 56: Francesco Borrelli Email: fborrelli@berkeley.edu ... · Berkeley Autonomous 1/10 Race Car Project Complete Open Source Ubuntu, RoS, OpenCV, Julia, IPOPT ... One task execution referred](https://reader034.vdocuments.us/reader034/viewer/2022050204/5f57a5ba1437da07145253ba/html5/thumbnails/56.jpg)
Borrelli (UC Berkeley) Iterative Learning MPC 2018 CDC– Slide 62
ILMPC Summary
MPC strategy:
Optimize over inputs and lambdasFor constrained linear systems
Safety guarantees:- Constraint satisfaction at iteration j=> satisfaction at iteration j+1
Performance improvement guarantees:- Closed loop cost at iteration j >= cost at iteration j+1
Convergence to global optimal solutionConstraint qualification conditions required for cost decrease
![Page 57: Francesco Borrelli Email: fborrelli@berkeley.edu ... · Berkeley Autonomous 1/10 Race Car Project Complete Open Source Ubuntu, RoS, OpenCV, Julia, IPOPT ... One task execution referred](https://reader034.vdocuments.us/reader034/viewer/2022050204/5f57a5ba1437da07145253ba/html5/thumbnails/57.jpg)
Borrelli (UC Berkeley) Iterative Learning MPC 2018 CDC– Slide 63
Performance Improvement Proof
Conjecture
Notation
Closed-loop state and input trajectory at iteration j
![Page 58: Francesco Borrelli Email: fborrelli@berkeley.edu ... · Berkeley Autonomous 1/10 Race Car Project Complete Open Source Ubuntu, RoS, OpenCV, Julia, IPOPT ... One task execution referred](https://reader034.vdocuments.us/reader034/viewer/2022050204/5f57a5ba1437da07145253ba/html5/thumbnails/58.jpg)
Borrelli (UC Berkeley) Iterative Learning MPC 2018 CDC– Slide 64
Step 1:
Performance Improvement Proof
![Page 59: Francesco Borrelli Email: fborrelli@berkeley.edu ... · Berkeley Autonomous 1/10 Race Car Project Complete Open Source Ubuntu, RoS, OpenCV, Julia, IPOPT ... One task execution referred](https://reader034.vdocuments.us/reader034/viewer/2022050204/5f57a5ba1437da07145253ba/html5/thumbnails/59.jpg)
Borrelli (UC Berkeley) Iterative Learning MPC 2018 CDC– Slide 65
Performance Improvement Proof
Step 1:
![Page 60: Francesco Borrelli Email: fborrelli@berkeley.edu ... · Berkeley Autonomous 1/10 Race Car Project Complete Open Source Ubuntu, RoS, OpenCV, Julia, IPOPT ... One task execution referred](https://reader034.vdocuments.us/reader034/viewer/2022050204/5f57a5ba1437da07145253ba/html5/thumbnails/60.jpg)
Borrelli (UC Berkeley) Iterative Learning MPC 2018 CDC– Slide 66
Performance Improvement Proof
Step 1:
![Page 61: Francesco Borrelli Email: fborrelli@berkeley.edu ... · Berkeley Autonomous 1/10 Race Car Project Complete Open Source Ubuntu, RoS, OpenCV, Julia, IPOPT ... One task execution referred](https://reader034.vdocuments.us/reader034/viewer/2022050204/5f57a5ba1437da07145253ba/html5/thumbnails/61.jpg)
Borrelli (UC Berkeley) Iterative Learning MPC 2018 CDC– Slide 67
Performance Improvement Proof
Step 1:
![Page 62: Francesco Borrelli Email: fborrelli@berkeley.edu ... · Berkeley Autonomous 1/10 Race Car Project Complete Open Source Ubuntu, RoS, OpenCV, Julia, IPOPT ... One task execution referred](https://reader034.vdocuments.us/reader034/viewer/2022050204/5f57a5ba1437da07145253ba/html5/thumbnails/62.jpg)
Borrelli (UC Berkeley) Iterative Learning MPC 2018 CDC– Slide 68
Performance Improvement Proof
Step 1:
Step 2:
![Page 63: Francesco Borrelli Email: fborrelli@berkeley.edu ... · Berkeley Autonomous 1/10 Race Car Project Complete Open Source Ubuntu, RoS, OpenCV, Julia, IPOPT ... One task execution referred](https://reader034.vdocuments.us/reader034/viewer/2022050204/5f57a5ba1437da07145253ba/html5/thumbnails/63.jpg)
Borrelli (UC Berkeley) Iterative Learning MPC 2018 CDC– Slide 69
Performance Improvement Proof
Step 1:
Step 2:
![Page 64: Francesco Borrelli Email: fborrelli@berkeley.edu ... · Berkeley Autonomous 1/10 Race Car Project Complete Open Source Ubuntu, RoS, OpenCV, Julia, IPOPT ... One task execution referred](https://reader034.vdocuments.us/reader034/viewer/2022050204/5f57a5ba1437da07145253ba/html5/thumbnails/64.jpg)
Borrelli (UC Berkeley) Iterative Learning MPC 2018 CDC– Slide 70
Performance Improvement Proof
Step 1:
Step 2:
![Page 65: Francesco Borrelli Email: fborrelli@berkeley.edu ... · Berkeley Autonomous 1/10 Race Car Project Complete Open Source Ubuntu, RoS, OpenCV, Julia, IPOPT ... One task execution referred](https://reader034.vdocuments.us/reader034/viewer/2022050204/5f57a5ba1437da07145253ba/html5/thumbnails/65.jpg)
Borrelli (UC Berkeley) Iterative Learning MPC 2018 CDC– Slide 71
Performance Improvement Proof
![Page 66: Francesco Borrelli Email: fborrelli@berkeley.edu ... · Berkeley Autonomous 1/10 Race Car Project Complete Open Source Ubuntu, RoS, OpenCV, Julia, IPOPT ... One task execution referred](https://reader034.vdocuments.us/reader034/viewer/2022050204/5f57a5ba1437da07145253ba/html5/thumbnails/66.jpg)
Borrelli (UC Berkeley) Iterative Learning MPC 2018 CDC– Slide 72
Iterative Learning MPC
Optimize over inputs and lambdas
Simple proofs
For constrained linear systems
Safety and Performance improvement guarantees
Convergence to global optimal solution (for linear
Constraint qualification conditions required for cost decrease
If full column rank, improvement cannot be obtained
![Page 67: Francesco Borrelli Email: fborrelli@berkeley.edu ... · Berkeley Autonomous 1/10 Race Car Project Complete Open Source Ubuntu, RoS, OpenCV, Julia, IPOPT ... One task execution referred](https://reader034.vdocuments.us/reader034/viewer/2022050204/5f57a5ba1437da07145253ba/html5/thumbnails/67.jpg)
Borrelli (UC Berkeley) Iterative Learning MPC 2018 CDC– Slide 73
Control objective
System dynamicsSystem constraints
Starting Position
Constrained LQR Example
![Page 68: Francesco Borrelli Email: fborrelli@berkeley.edu ... · Berkeley Autonomous 1/10 Race Car Project Complete Open Source Ubuntu, RoS, OpenCV, Julia, IPOPT ... One task execution referred](https://reader034.vdocuments.us/reader034/viewer/2022050204/5f57a5ba1437da07145253ba/html5/thumbnails/68.jpg)
Borrelli (UC Berkeley) Iterative Learning MPC 2018 CDC– Slide 74
Control objective
System dynamicsSystem constraints
Terminal Constraint
Initial Condition
Iterative LMPC with horizon N=2
Will not work!
Will work if one sets N=3
![Page 69: Francesco Borrelli Email: fborrelli@berkeley.edu ... · Berkeley Autonomous 1/10 Race Car Project Complete Open Source Ubuntu, RoS, OpenCV, Julia, IPOPT ... One task execution referred](https://reader034.vdocuments.us/reader034/viewer/2022050204/5f57a5ba1437da07145253ba/html5/thumbnails/69.jpg)
Borrelli (UC Berkeley) Iterative Learning MPC 2018 CDC– Slide 75
Comparison with R.L.??
RL term too broadTwo good references:
Bertsekas paper connecting MPC and ADP*Lewis and Vrabile survey on CSM**Recht survey (section 6): https://arxiv.org/abs/1806.09460
ILMPC highlightsContinuous state formulationConstraints satisfaction and Sampled Safe SetsQ-function constructed (learned) locally based on cost/model
driven exploration and past trailsQ-function at stored state is “exact” and lowerbounds property
at intermediate points (for convex problems)*Dynamic Programming and Suboptimal Control: A Survey from ADP to MPC**Reinforcement Learning and Adaptive Dynamic Programming for Feedback Control
![Page 70: Francesco Borrelli Email: fborrelli@berkeley.edu ... · Berkeley Autonomous 1/10 Race Car Project Complete Open Source Ubuntu, RoS, OpenCV, Julia, IPOPT ... One task execution referred](https://reader034.vdocuments.us/reader034/viewer/2022050204/5f57a5ba1437da07145253ba/html5/thumbnails/70.jpg)
Borrelli (UC Berkeley) Iterative Learning MPC 2018 CDC– Slide 76
About Model Learning in Racing
![Page 71: Francesco Borrelli Email: fborrelli@berkeley.edu ... · Berkeley Autonomous 1/10 Race Car Project Complete Open Source Ubuntu, RoS, OpenCV, Julia, IPOPT ... One task execution referred](https://reader034.vdocuments.us/reader034/viewer/2022050204/5f57a5ba1437da07145253ba/html5/thumbnails/71.jpg)
Borrelli (UC Berkeley) Iterative Learning MPC 2018 CDC– Slide 77
Autonomous Racing Control Problem
Start & end position
System dynamicsSystem constraints
Obstacle avoidance
Control objective
![Page 72: Francesco Borrelli Email: fborrelli@berkeley.edu ... · Berkeley Autonomous 1/10 Race Car Project Complete Open Source Ubuntu, RoS, OpenCV, Julia, IPOPT ... One task execution referred](https://reader034.vdocuments.us/reader034/viewer/2022050204/5f57a5ba1437da07145253ba/html5/thumbnails/72.jpg)
Borrelli (UC Berkeley) Iterative Learning MPC 2018 CDC– Slide 78
Learning Model Predictive Control (LMPC)
![Page 73: Francesco Borrelli Email: fborrelli@berkeley.edu ... · Berkeley Autonomous 1/10 Race Car Project Complete Open Source Ubuntu, RoS, OpenCV, Julia, IPOPT ... One task execution referred](https://reader034.vdocuments.us/reader034/viewer/2022050204/5f57a5ba1437da07145253ba/html5/thumbnails/73.jpg)
Borrelli (UC Berkeley) Iterative Learning MPC 2018 CDC– Slide 80
Learning Process
The lap time decreases until the LMPC converges to a set of trajectories
![Page 74: Francesco Borrelli Email: fborrelli@berkeley.edu ... · Berkeley Autonomous 1/10 Race Car Project Complete Open Source Ubuntu, RoS, OpenCV, Julia, IPOPT ... One task execution referred](https://reader034.vdocuments.us/reader034/viewer/2022050204/5f57a5ba1437da07145253ba/html5/thumbnails/74.jpg)
Borrelli (UC Berkeley) Iterative Learning MPC 2018 CDC– Slide 82
Learning Model Predictive Control (LMPC)
![Page 75: Francesco Borrelli Email: fborrelli@berkeley.edu ... · Berkeley Autonomous 1/10 Race Car Project Complete Open Source Ubuntu, RoS, OpenCV, Julia, IPOPT ... One task execution referred](https://reader034.vdocuments.us/reader034/viewer/2022050204/5f57a5ba1437da07145253ba/html5/thumbnails/75.jpg)
Borrelli (UC Berkeley) Iterative Learning MPC 2018 CDC– Slide 83
Nonlinear Dynamical System
Useful Vehicle Model Abstraction
![Page 76: Francesco Borrelli Email: fborrelli@berkeley.edu ... · Berkeley Autonomous 1/10 Race Car Project Complete Open Source Ubuntu, RoS, OpenCV, Julia, IPOPT ... One task execution referred](https://reader034.vdocuments.us/reader034/viewer/2022050204/5f57a5ba1437da07145253ba/html5/thumbnails/76.jpg)
Borrelli (UC Berkeley) Iterative Learning MPC 2018 CDC– Slide 84
Nonlinear Dynamical System
Useful Vehicle Model Abstraction
Kinematic Equations
![Page 77: Francesco Borrelli Email: fborrelli@berkeley.edu ... · Berkeley Autonomous 1/10 Race Car Project Complete Open Source Ubuntu, RoS, OpenCV, Julia, IPOPT ... One task execution referred](https://reader034.vdocuments.us/reader034/viewer/2022050204/5f57a5ba1437da07145253ba/html5/thumbnails/77.jpg)
Borrelli (UC Berkeley) Iterative Learning MPC 2018 CDC– Slide 85
Nonlinear Dynamical System
Useful Vehicle Model Abstraction
Kinematic Equations
Identifying the Dynamical System
Linearization around predicted trajectory
![Page 78: Francesco Borrelli Email: fborrelli@berkeley.edu ... · Berkeley Autonomous 1/10 Race Car Project Complete Open Source Ubuntu, RoS, OpenCV, Julia, IPOPT ... One task execution referred](https://reader034.vdocuments.us/reader034/viewer/2022050204/5f57a5ba1437da07145253ba/html5/thumbnails/78.jpg)
Borrelli (UC Berkeley) Iterative Learning MPC 2018 CDC– Slide 86
Nonlinear Dynamical System
Useful Vehicle Model Abstraction
Kinematic Equations
Dynamic Equations
Identifying the Dynamical SystemLocal Linear Regression
Linearization around predicted trajectory
![Page 79: Francesco Borrelli Email: fborrelli@berkeley.edu ... · Berkeley Autonomous 1/10 Race Car Project Complete Open Source Ubuntu, RoS, OpenCV, Julia, IPOPT ... One task execution referred](https://reader034.vdocuments.us/reader034/viewer/2022050204/5f57a5ba1437da07145253ba/html5/thumbnails/79.jpg)
Borrelli (UC Berkeley) Iterative Learning MPC 2018 CDC– Slide 87
Useful Vehicle Model Abstraction
Identifying the Dynamical System
Important Design Steps1. Compute trajectory to linearize around uses previous optimal inputs and
inputs in the safe set2. Enforce model-based sparsity in local linear regression
Local Linear Regression
Linearization around predicted trajectory
![Page 80: Francesco Borrelli Email: fborrelli@berkeley.edu ... · Berkeley Autonomous 1/10 Race Car Project Complete Open Source Ubuntu, RoS, OpenCV, Julia, IPOPT ... One task execution referred](https://reader034.vdocuments.us/reader034/viewer/2022050204/5f57a5ba1437da07145253ba/html5/thumbnails/80.jpg)
Borrelli (UC Berkeley) Iterative Learning MPC 2018 CDC– Slide 88
Nonlinear Dynamical System
Useful Vehicle Model Abstraction
The velocity update is not affected by Position and Acceleration command
![Page 81: Francesco Borrelli Email: fborrelli@berkeley.edu ... · Berkeley Autonomous 1/10 Race Car Project Complete Open Source Ubuntu, RoS, OpenCV, Julia, IPOPT ... One task execution referred](https://reader034.vdocuments.us/reader034/viewer/2022050204/5f57a5ba1437da07145253ba/html5/thumbnails/81.jpg)
Borrelli (UC Berkeley) Iterative Learning MPC 2018 CDC– Slide 89
Useful Vehicle Model Abstraction
Identifying the Dynamical System
Important Design Steps1. Compute trajectory to linearize aroundpusing previous optimal inputs
and inputs in the safe set2. Enforce model-based sparsity in local linear regression3. Use data close to current state trajectory for parameter ID4. Use kernel K() to weight differently data as a function of distance to
linearized trajectory
Local Linear Regression
Linearization around predicted trajectory
![Page 82: Francesco Borrelli Email: fborrelli@berkeley.edu ... · Berkeley Autonomous 1/10 Race Car Project Complete Open Source Ubuntu, RoS, OpenCV, Julia, IPOPT ... One task execution referred](https://reader034.vdocuments.us/reader034/viewer/2022050204/5f57a5ba1437da07145253ba/html5/thumbnails/82.jpg)
Borrelli (UC Berkeley) Iterative Learning MPC 2018 CDC– Slide 90
Accelerations
![Page 83: Francesco Borrelli Email: fborrelli@berkeley.edu ... · Berkeley Autonomous 1/10 Race Car Project Complete Open Source Ubuntu, RoS, OpenCV, Julia, IPOPT ... One task execution referred](https://reader034.vdocuments.us/reader034/viewer/2022050204/5f57a5ba1437da07145253ba/html5/thumbnails/83.jpg)
Borrelli (UC Berkeley) Iterative Learning MPC 2018 CDC– Slide 91
Results
Gain from steering to lateral velocity
![Page 84: Francesco Borrelli Email: fborrelli@berkeley.edu ... · Berkeley Autonomous 1/10 Race Car Project Complete Open Source Ubuntu, RoS, OpenCV, Julia, IPOPT ... One task execution referred](https://reader034.vdocuments.us/reader034/viewer/2022050204/5f57a5ba1437da07145253ba/html5/thumbnails/84.jpg)
Borrelli (UC Berkeley) Iterative Learning MPC 2018 CDC– Slide 95
About Model Learning Ball in Cup
![Page 85: Francesco Borrelli Email: fborrelli@berkeley.edu ... · Berkeley Autonomous 1/10 Race Car Project Complete Open Source Ubuntu, RoS, OpenCV, Julia, IPOPT ... One task execution referred](https://reader034.vdocuments.us/reader034/viewer/2022050204/5f57a5ba1437da07145253ba/html5/thumbnails/85.jpg)
Borrelli (UC Berkeley) Iterative Learning MPC 2018 CDC– Slide 96
Ball in a Cup System with MuJoCo
![Page 86: Francesco Borrelli Email: fborrelli@berkeley.edu ... · Berkeley Autonomous 1/10 Race Car Project Complete Open Source Ubuntu, RoS, OpenCV, Julia, IPOPT ... One task execution referred](https://reader034.vdocuments.us/reader034/viewer/2022050204/5f57a5ba1437da07145253ba/html5/thumbnails/86.jpg)
Borrelli (UC Berkeley) Iterative Learning MPC 2018 CDC– Slide 97
Ball in a Cup Control Problem
Start & end position
System dynamicsSystem constraints
Obstacle avoidance
Control objective
![Page 87: Francesco Borrelli Email: fborrelli@berkeley.edu ... · Berkeley Autonomous 1/10 Race Car Project Complete Open Source Ubuntu, RoS, OpenCV, Julia, IPOPT ... One task execution referred](https://reader034.vdocuments.us/reader034/viewer/2022050204/5f57a5ba1437da07145253ba/html5/thumbnails/87.jpg)
Borrelli (UC Berkeley) Iterative Learning MPC 2018 CDC– Slide 98
Learning Model Predictive Control (LMPC)
![Page 88: Francesco Borrelli Email: fborrelli@berkeley.edu ... · Berkeley Autonomous 1/10 Race Car Project Complete Open Source Ubuntu, RoS, OpenCV, Julia, IPOPT ... One task execution referred](https://reader034.vdocuments.us/reader034/viewer/2022050204/5f57a5ba1437da07145253ba/html5/thumbnails/88.jpg)
Borrelli (UC Berkeley) Iterative Learning MPC 2018 CDC– Slide 99
Useful Mujoco Model Abstraction
Identifying the Dynamical SystemLocal Linear Regression
Linearization around predicted trajectory
Important Design Steps1. Compute trajectory to linearize aroundpusing previous optimal inputs
and inputs in the safe set2. Enforce model-based sparsity in local linear regression3. Use data close to current state trajectory for parameter ID4. Use kernel K() to weight differently data as a function of distance to
linearized trajectory
![Page 89: Francesco Borrelli Email: fborrelli@berkeley.edu ... · Berkeley Autonomous 1/10 Race Car Project Complete Open Source Ubuntu, RoS, OpenCV, Julia, IPOPT ... One task execution referred](https://reader034.vdocuments.us/reader034/viewer/2022050204/5f57a5ba1437da07145253ba/html5/thumbnails/89.jpg)
Borrelli (UC Berkeley) Iterative Learning MPC 2018 CDC– Slide 100
Ball in a Cup System
At iteration 0 find a sequence by sampling parametrized inputs profiles (takes 5mins)
Use ILMPC: At iteration 1, time reduced of 10%, cup height movement reduced of 35%
![Page 90: Francesco Borrelli Email: fborrelli@berkeley.edu ... · Berkeley Autonomous 1/10 Race Car Project Complete Open Source Ubuntu, RoS, OpenCV, Julia, IPOPT ... One task execution referred](https://reader034.vdocuments.us/reader034/viewer/2022050204/5f57a5ba1437da07145253ba/html5/thumbnails/90.jpg)
Borrelli (UC Berkeley) Iterative Learning MPC 2018 CDC– Slide 101
Back to our main chart..
![Page 91: Francesco Borrelli Email: fborrelli@berkeley.edu ... · Berkeley Autonomous 1/10 Race Car Project Complete Open Source Ubuntu, RoS, OpenCV, Julia, IPOPT ... One task execution referred](https://reader034.vdocuments.us/reader034/viewer/2022050204/5f57a5ba1437da07145253ba/html5/thumbnails/91.jpg)
Borrelli (UC Berkeley) Iterative Learning MPC 2018 CDC– Slide 102
Three Forms of Learning
Skill acquisition Performance improvement
How we do this?
Model Predictive Control +A Simple Idea +Good Practices
Reduce load forRoutine Execution
![Page 92: Francesco Borrelli Email: fborrelli@berkeley.edu ... · Berkeley Autonomous 1/10 Race Car Project Complete Open Source Ubuntu, RoS, OpenCV, Julia, IPOPT ... One task execution referred](https://reader034.vdocuments.us/reader034/viewer/2022050204/5f57a5ba1437da07145253ba/html5/thumbnails/92.jpg)
Borrelli (UC Berkeley) Iterative Learning MPC 2018 CDC– Slide 103
Offline π(∙) and Online π(x) Computation
Option 1 (Offline Based): “Complex” Offline, “Simple” Online
π(∙) often Piecewise Constant (except special classes)
Dynamic Programing is one choice
Basic rule: n>5 impossible
Option 2 (Online Based): “Simple” Offline, “Complex” Online
Compute on-line π(x) with a “sophisticated” algorithm
Interior point method solver is one choice
Basic Rule: avoid use `home-made’ solvers
![Page 93: Francesco Borrelli Email: fborrelli@berkeley.edu ... · Berkeley Autonomous 1/10 Race Car Project Complete Open Source Ubuntu, RoS, OpenCV, Julia, IPOPT ... One task execution referred](https://reader034.vdocuments.us/reader034/viewer/2022050204/5f57a5ba1437da07145253ba/html5/thumbnails/93.jpg)
Borrelli (UC Berkeley) Iterative Learning MPC 2018 CDC– Slide 104
One Simple Way: Data-Based Policy for π(∙)
![Page 94: Francesco Borrelli Email: fborrelli@berkeley.edu ... · Berkeley Autonomous 1/10 Race Car Project Complete Open Source Ubuntu, RoS, OpenCV, Julia, IPOPT ... One task execution referred](https://reader034.vdocuments.us/reader034/viewer/2022050204/5f57a5ba1437da07145253ba/html5/thumbnails/94.jpg)
Borrelli (UC Berkeley) Iterative Learning MPC 2018 CDC– Slide 105
One Simple Way: Data-Based Policy for π(∙)
Historical dataof converged iterations
![Page 95: Francesco Borrelli Email: fborrelli@berkeley.edu ... · Berkeley Autonomous 1/10 Race Car Project Complete Open Source Ubuntu, RoS, OpenCV, Julia, IPOPT ... One task execution referred](https://reader034.vdocuments.us/reader034/viewer/2022050204/5f57a5ba1437da07145253ba/html5/thumbnails/95.jpg)
Borrelli (UC Berkeley) Iterative Learning MPC 2018 CDC– Slide 107
Three Forms of Learning3 - Computation Load Reduction
Average CPU Load at each iteration
Lap Time at each iteration
![Page 96: Francesco Borrelli Email: fborrelli@berkeley.edu ... · Berkeley Autonomous 1/10 Race Car Project Complete Open Source Ubuntu, RoS, OpenCV, Julia, IPOPT ... One task execution referred](https://reader034.vdocuments.us/reader034/viewer/2022050204/5f57a5ba1437da07145253ba/html5/thumbnails/96.jpg)
Borrelli (UC Berkeley) Iterative Learning MPC 2018 CDC– Slide 108
Experimental Results
Factor of 10
![Page 97: Francesco Borrelli Email: fborrelli@berkeley.edu ... · Berkeley Autonomous 1/10 Race Car Project Complete Open Source Ubuntu, RoS, OpenCV, Julia, IPOPT ... One task execution referred](https://reader034.vdocuments.us/reader034/viewer/2022050204/5f57a5ba1437da07145253ba/html5/thumbnails/97.jpg)
Borrelli (UC Berkeley) Iterative Learning MPC 2018 CDC– Slide 109
Data Based Policy: Alternatives
Nearest NeighborTrain ReLU Neural NetworkLocal Explicit MPC
All Continuous Piecewise Affine Policies
![Page 98: Francesco Borrelli Email: fborrelli@berkeley.edu ... · Berkeley Autonomous 1/10 Race Car Project Complete Open Source Ubuntu, RoS, OpenCV, Julia, IPOPT ... One task execution referred](https://reader034.vdocuments.us/reader034/viewer/2022050204/5f57a5ba1437da07145253ba/html5/thumbnails/98.jpg)
Borrelli (UC Berkeley) Iterative Learning MPC 2018 CDC– Slide 110
Learned fromdata
Learning MPCIncorporating data in advance model based controller
In Practice
Noise and model uncertainty: Robust case
What about noise and model uncertainty?
![Page 99: Francesco Borrelli Email: fborrelli@berkeley.edu ... · Berkeley Autonomous 1/10 Race Car Project Complete Open Source Ubuntu, RoS, OpenCV, Julia, IPOPT ... One task execution referred](https://reader034.vdocuments.us/reader034/viewer/2022050204/5f57a5ba1437da07145253ba/html5/thumbnails/99.jpg)
Borrelli (UC Berkeley) Iterative Learning MPC 2018 CDC– Slide 111
ILMPC – Robust and Adaptive design
![Page 100: Francesco Borrelli Email: fborrelli@berkeley.edu ... · Berkeley Autonomous 1/10 Race Car Project Complete Open Source Ubuntu, RoS, OpenCV, Julia, IPOPT ... One task execution referred](https://reader034.vdocuments.us/reader034/viewer/2022050204/5f57a5ba1437da07145253ba/html5/thumbnails/100.jpg)
Borrelli (UC Berkeley) Iterative Learning MPC 2018 CDC– Slide 112
At Iteration 0
Linear System
Terminal Goal Set
Successful Iteration
![Page 101: Francesco Borrelli Email: fborrelli@berkeley.edu ... · Berkeley Autonomous 1/10 Race Car Project Complete Open Source Ubuntu, RoS, OpenCV, Julia, IPOPT ... One task execution referred](https://reader034.vdocuments.us/reader034/viewer/2022050204/5f57a5ba1437da07145253ba/html5/thumbnails/101.jpg)
Borrelli (UC Berkeley) Iterative Learning MPC 2018 CDC– Slide 113
At Iteration 1
CVX hull is not a robust invariant!
![Page 102: Francesco Borrelli Email: fborrelli@berkeley.edu ... · Berkeley Autonomous 1/10 Race Car Project Complete Open Source Ubuntu, RoS, OpenCV, Julia, IPOPT ... One task execution referred](https://reader034.vdocuments.us/reader034/viewer/2022050204/5f57a5ba1437da07145253ba/html5/thumbnails/102.jpg)
Borrelli (UC Berkeley) Iterative Learning MPC 2018 CDC– Slide 114
ILMPC – Robust and Adaptive design
Robust invariants
“Robustify” Q-function (and dualize for computational efficiency)
Chance constraints
See my group papers at this conference if interested..
![Page 103: Francesco Borrelli Email: fborrelli@berkeley.edu ... · Berkeley Autonomous 1/10 Race Car Project Complete Open Source Ubuntu, RoS, OpenCV, Julia, IPOPT ... One task execution referred](https://reader034.vdocuments.us/reader034/viewer/2022050204/5f57a5ba1437da07145253ba/html5/thumbnails/103.jpg)
Borrelli (UC Berkeley) Iterative Learning MPC 2018 CDC– Slide 115
For Iterative tasks I discussed
By using Iterative learning MPC, i.e.
Model Predictive ControlA Simple Idea (which exploits the iterative nature of the tasks)
A Few Important Design Steps
How to obtain performance improvement and reduced computational load while satisfying constraints
![Page 104: Francesco Borrelli Email: fborrelli@berkeley.edu ... · Berkeley Autonomous 1/10 Race Car Project Complete Open Source Ubuntu, RoS, OpenCV, Julia, IPOPT ... One task execution referred](https://reader034.vdocuments.us/reader034/viewer/2022050204/5f57a5ba1437da07145253ba/html5/thumbnails/104.jpg)
Borrelli (UC Berkeley) Iterative Learning MPC 2018 CDC– Slide 116
The End