stochastic model predictive...
TRANSCRIPT
Stochastic Model Predictive Control
Pantelis Sopasakis
IMT Institute for Advanced Studies Lucca
February 10, 2016
Outline
1. Intro: stochastic optimal control
2. Classification of SMPC approaches
3. Scenario based SMPC
4. Affine disturbance feedback
1 / 94
I. Introduction
X Stochastic optimal control
X Control policies
X Dynamic programming
2 / 94
Stochastic optimal control
Stochastic optimal control lies at the core of every stochastic MPCformulation.
3 / 94
Stochastic optimal control
Uncertain dynamical system:
xk+1 = f(xk, uk, wk),
where wk lives in a probability space (Ωk,Fk,Pk)1.
In stochastic optimal control, we get take our decision uk+j|k at futuretime k + j taking into account the available information up to that time.
1The probability distribution function of wk may be a function of xk and uk, thatis P = P(dwk | xk, uk). See Bertsekas and Shreve, 1978.
4 / 94
Stochastic OC + Causality = ♥
At k = j we observe xj and uj and we decide the control action using
I The initial information x0 (and w0),
I The current observation, that is xj (and wj)
I The history of control actions
Overall...uj = µj(x0, w0, u0, . . . , uj−1, xj , wj).
We thus construct the space ΠN = (µ0, . . . , µN−1) of (causal) controlpolicies. In some cases it suffices to assume2
uj = µj(xj).
2These are called Markov policies.
5 / 94
Stochastic OC + Causality = ♥
Assume we can observe wk at time k:
I k = 0 Observe x0, w0
I k = 0 Decide u0 = µ0(x0, w0)
I k = 1 System response x1 = f(x0, w0, u0)
I k = 1 Observe x1, w1
I k = 1 Decide u1 = µ1(x0, w0, u0, x1)
I k = 2 System response x2 = f(x1, u1, w1)
I k = 2 Observe x2, w2 ...
6 / 94
Stochastic OC + Causality = ♥
Assume we can observe wk at time k:
I k = 0 Observe x0, w0
I k = 0 Decide u0 = µ0(x0, w0)
I k = 1 System response x1 = f(x0, w0, u0)
I k = 1 Observe x1, w1
I k = 1 Decide u1 = µ1(x0, w0, u0, x1)
I k = 2 System response x2 = f(x1, u1, w1)
I k = 2 Observe x2, w2 ...
6 / 94
Stochastic OC + Causality = ♥
Assume we can observe wk at time k:
I k = 0 Observe x0, w0
I k = 0 Decide u0 = µ0(x0, w0)
I k = 1 System response x1 = f(x0, w0, u0)
I k = 1 Observe x1, w1
I k = 1 Decide u1 = µ1(x0, w0, u0, x1)
I k = 2 System response x2 = f(x1, u1, w1)
I k = 2 Observe x2, w2 ...
6 / 94
Stochastic OC + Causality = ♥
Assume we can observe wk at time k:
I k = 0 Observe x0, w0
I k = 0 Decide u0 = µ0(x0, w0)
I k = 1 System response x1 = f(x0, w0, u0)
I k = 1 Observe x1, w1
I k = 1 Decide u1 = µ1(x0, w0, u0, x1)
I k = 2 System response x2 = f(x1, u1, w1)
I k = 2 Observe x2, w2 ...
6 / 94
Stochastic OC + Causality = ♥
Assume we can observe wk at time k:
I k = 0 Observe x0, w0
I k = 0 Decide u0 = µ0(x0, w0)
I k = 1 System response x1 = f(x0, w0, u0)
I k = 1 Observe x1, w1
I k = 1 Decide u1 = µ1(x0, w0, u0, x1)
I k = 2 System response x2 = f(x1, u1, w1)
I k = 2 Observe x2, w2 ...
6 / 94
Stochastic OC + Causality = ♥
Assume we can observe wk at time k:
I k = 0 Observe x0, w0
I k = 0 Decide u0 = µ0(x0, w0)
I k = 1 System response x1 = f(x0, w0, u0)
I k = 1 Observe x1, w1
I k = 1 Decide u1 = µ1(x0, w0, u0, x1)
I k = 2 System response x2 = f(x1, u1, w1)
I k = 2 Observe x2, w2 ...
6 / 94
Stochastic OC + Causality = ♥
Assume we can observe wk at time k:
I k = 0 Observe x0, w0
I k = 0 Decide u0 = µ0(x0, w0)
I k = 1 System response x1 = f(x0, w0, u0)
I k = 1 Observe x1, w1
I k = 1 Decide u1 = µ1(x0, w0, u0, x1)
I k = 2 System response x2 = f(x1, u1, w1)
I k = 2 Observe x2, w2 ...
6 / 94
Stochastic optimal control
Hereafter we assume uk = µ(xk)3.
Three equivalent formulations:
1. In nested form
2. Over a product probability space
3. As a dynamic programming recursion
3This is an essential assumption to formulate the stochastic OCP as a DP recur-sion. This way, uk is computed at time k without using historical information ofthe process, i.e., any of w0, w1, . . . , wk−1.
7 / 94
Nested formulation
Formulation as a nested problem: The total cost function is (whereπ = (µ0, µ1, . . . , µN−1) with ui = µi(xi))4
VN (x0, π) = Ew0
[`0(x0, µ0(x0), w0) + Ew1
[`1(x1, µ1(x1), w1)
+ Ew2 [· · ·+ EwN−1 [`N−1(xN−1, µN−1(xN−1), wN−1)
| xN−1, µN−1(xN−1)]
| · · · ]]| x0, µ0(x0)
],
where the states xk satisfy
xk+1 = f(xk, µk(xk), wk)
for k ∈ N[0,N−2].
4It’s easy to wedge in a terminal cost function of the form `N (xN , uN , wN ) =Vf (xN , wN ).
8 / 94
Product space formulation
We can use the following result to rearrange the terms in VN . For everymeasure space (Ω,F ,P) and measurable h : Ω→ IR and λ ∈ (−∞,∞] wehave
λ+
∫hd P =
∫(λ+ h)dP.
And recall that the expectation of a random variable h on a probabilityspace (Ω,F ,P) is given by the Lebesgue integral
E[h] =
∫hdP.
9 / 94
Product space formulation
Assume `k > −∞. Then, it is
VN (x0, π) = Ew0
[Ew1
[Ew2
[. . .EwN−1 [
N−1∑k=0
`k(xk, µk(xk), wk)
| xN−1, µN−1(xN−1)] | · · ·]| x0, µ0(x0)
],
10 / 94
* The product probability space
To define a product probability space, we need to introduce the notion of ap-system on Ω5 which is a collection of sets A so that
A1, A2 ∈ A ⇒ A1 ∩A2 ∈ A.
Example: On IR, the class A = (−∞, b], b ∈ IR is a p-system.
5p for product; aka π-system or pi-system.
11 / 94
* The product probability space
If two (probability) measures coincide on a p-system, they coincide every-where, thus, it suffices to define a measure on a p-system.
Recall that the cartesian product of two set A, B is
A×B 3 (a, b); a ∈ A, b ∈ B.
Let (A,FA) and (B,FB) be measurable spaces; then let
A = SA × SB;SA ∈ FA, SB ∈ FB
To define a (prob.) measure on A×B it suffices to define it on A.
12 / 94
* The product probability space
13 / 94
Product space formulation
If the conditions of Fubini’s Theorem are satisfied6, then
VN (x0, π) = E
[N−1∑k=0
`k(xk, µk(xk), wk),
]
where E is the expectation operator in the product measure space of(Ωk,Fk,Pk) for k ∈ N[0,N−1] and the states x1, x2, . . . , xN−1 arefunctions of x0 and w0, w1, . . . , wN−1 satisfying the system dynamics.
6What are these conditions? See: R. Ash, Real analysis and probability, Academicpress, 1972.
14 / 94
DP recursion
It follows from the nested formulation of VN that the DP recursion is
V ?0 (x) = 0,
and for j = 0, . . . , N − 1,
V ?j+1(x) = inf
u∈UN−j(x)EwN−j
[`N−j(x, u, wN−j) + V ?
j (f(x, u, wN−j))].
Notice that tacitly we have assumed uk = µk(xk); this is a condicio sinequa non7 for DP.
7In Latin, condicio sine qua non refers to an indispensable and essential action,condition, or ingredient. We will study later the case of scenario trees where wecan aberrate from this rule.
15 / 94
Stochastic programming vs DP
1. In DP we are bound to assume uk = µk(xk)8; in stochastic
programming we can have uk = µk(x0, w0, . . . , wk−1, xk)
2. In DP we assume that the underlying random processw0, w1, . . . , wN−1 is stagewise independent.
3. There are cases where we can use apply DP without assumingstagewise independence; e.g., scenario trees (later).
8Such policies are known as Markovian. Whether a non-Markovian policy can bebetter than a Markovian one is a nontrivial question which is treated in Bertsekas& Shreve, 1978.
16 / 94
Remarks
Stochastic programming problems are very difficult to solve even for(ostensibly) simple cases such as unconstrained linear systems.
We usually have to resort to simplifying assumptions, such as:
1. Assume that the underlying process is
1.1 iid1.2 iid and normal
2. Discretisation of probability distributions (scenario trees)
3. Optimise over Markovian policies only, i.e., uk = µk(xk)
4. Optimise over semi-Markovian policies only, i.e., uk = µk(x0, xk)
5. Parametrisation of inputs, e.g., uk =∑k−1
i=0 Hiwi + hi
17 / 94
A little exercise
Assume `k(·, ·, w) are convex for all w ∈ Ωk and the system dynamics islinear,
xk+1 = A(wk)xk +B(wk)uk + d(wk).
We impose the constraints uk ∈ U where U is a nonempty convex closedset. Assume that wk are stagewise independent and uk = µk(xk).
Show that V ?N (x) is a convex function.
18 / 94
Exercise
Assume `k(x, u, w) = x′Qkx+ u′Rku, with Qk ∈ Sn+, Rk ∈ Sn++ and thesystem dynamics is given by
xk+1 = Akxk +Buk + vk
where Ak ∼MN n×n(Ak, Uk, Vk)9 and vk ∼ Nn(dk,Σk); Ak and vk are
independent and neither of those is known at time k. Determine V ?2 (x)
using DP.
9A(wk) is a random matrix and it follows the matrix normal distribution whosedefinition and many useful properties can be found in: A.K. Gupta and D.K.Nagar, Matrix variate distributions, Chapman & Hall, 2000.
19 / 94
Further reading
1. D.P. Bertsekas and S.E. Shreve, Stochastic optimal control: thediscrete time case, Academic press, 1978.
2. A. Shapiro, D. Dentcheva and A.Ruszczynski, Lectures on stochasticprogramming – modeling and theory, MPS-SIAM series onoptimization, 2009.
20 / 94
II. SMPC taxonomy
21 / 94
System dynamics
X Linearxk+1 = A(wk)xk +B(wk)uk + d(wk)
X Nonlinearxk+1 = f(xk, uk, wk)
22 / 94
Type of uncertainty
X Additivexk+1 = f(xk, uk) + wk
X Parametric (linear case)
xk+1 = A(wk)xk +B(wk)uk
X Both
23 / 94
Uncertainty over time #1
X Time-varyingxk+1 = f(xk, uk, wk),
X Time-invariantxk+1 = f(xk, uk, w),
24 / 94
Uncertainty over time #2
X IID – all wk have the same probability distribution & they areindependent,
X Markovian – the probability distribution of wk+1 is conditioned bywk.
25 / 94
Control policy parametrisation
X Affine policy parametrization10
uk+j|k = µj(wk, wk+1|k, . . . , wk+j−1|k)
= Hjwk+j−1|k + bj
X Blocking affine policy parametrization
X Prestabilising feedback control as in stochastic tube MPC11
X Open-loop control actions12
10Kouvaritakis, Cannon and Munoz-Carpintero 2013; Oldewurtel et al. 2008; Ko-rda, Gondhalekar, Cigler, Oldewurtel 2011.
11Cannon, Kouvaritakis, Ng 2009; Cannon et al. 2011.12Kim and Braatz 2013; Bernardini and Bemporad, 2009, 2012.
26 / 94
Type of constraints
X Hard constraints(xk, uk) ∈ Z
X Probabilistic constraints
X IndividualP[G(i)x(t) ≤ g(i)] ≥ 1− αi∀i
X JointP[G(i)x(t) ≤ g(i),∀i] ≥ 1− α
X Expectation13
X Saturation of inputs14
13Hokayem, Cinquemani et al. 2012.14Hokayem, Chatterjee and Lygeros 2009.
27 / 94
Uncertainty propagation
X Stochastic tube
xk = zk + ek
uk = Kxk + ck
X Scenario-based
X Gaussian mixture
X Other
28 / 94
Availability of feedback information
X State
X Full state feedbackX Output feedback
X Disturbance
X Measured disturbanceX Not measured
29 / 94
Further reading
1. A. Mesbah, “Stochastic Model Predictive Control: A Review,” IEEE ControlSystems Magazine, 2016.
2. M. Kamgarpour, P. Hokayem, D. Chatterjee, M. Prandini, S. Garatti and A. Abate,“Final report on model predictive control for stochastic hybrid systems,” report ofproject “Moves”: http://www.movesproject.eu/deliverables/WP3/D3.2.pdf
30 / 94
III. Scenario trees
X The scenario tree structure
X Causality
X DP on a scenario tree
31 / 94
Motivation
1. Useful for numerical computations
2. Can be constructed from observations (data-driven)
3. They model non-iid processes
4. They provide a model for uncertainty propagation
5. Assumption: Ωk are finite
32 / 94
Applications
I Micro-grids [Hans et al. ’15]
I Drinking water networks [Sampathirao et al. ’15]
I HVAC [Long et al. ’13, Zhang et al. ’13, Parisio et al. ’13]
I Financial systems [Patrinos et al. ’11, Bemporad et al., ’14]
I Chemical process [Lucia et al. ’13]
I Distillation column [Garrido and Steinbach, ’11]
33 / 94
Scenario tree structure
34 / 94
Definitions
I Let N − 1 be the last stage of the tree
I At stage k we have µ(k) nodes and µ(0) = 1
I The nodes at stage N − 1 are called leaf nodes
I The node at stage 0 is the root node
I A scenario is an admissible path from the root node to a leaf node
I The tree counts µ(N − 1) scenarios
I The value of ω at stage k, node i is denoted by ωikI Each node i ∈ N[1,µ(k)] at stage k ∈ N[0,N−2] has a set of children
child(k, i) ⊆ N[1,µ(k+1)]
I Each node i ∈ N[1,µ(k)] at stage k ∈ N[0,N−2] has a unique ancestorwhich is a node j ∈ N[1,µ(k−1)] at stage k − 1
35 / 94
Definitions
I Let N − 1 be the last stage of the tree
I At stage k we have µ(k) nodes and µ(0) = 1
I The nodes at stage N − 1 are called leaf nodes
I The node at stage 0 is the root node
I A scenario is an admissible path from the root node to a leaf node
I The tree counts µ(N − 1) scenarios
I The value of ω at stage k, node i is denoted by ωikI Each node i ∈ N[1,µ(k)] at stage k ∈ N[0,N−2] has a set of children
child(k, i) ⊆ N[1,µ(k+1)]
I Each node i ∈ N[1,µ(k)] at stage k ∈ N[0,N−2] has a unique ancestorwhich is a node j ∈ N[1,µ(k−1)] at stage k − 1
35 / 94
Definitions
I Let N − 1 be the last stage of the tree
I At stage k we have µ(k) nodes and µ(0) = 1
I The nodes at stage N − 1 are called leaf nodes
I The node at stage 0 is the root node
I A scenario is an admissible path from the root node to a leaf node
I The tree counts µ(N − 1) scenarios
I The value of ω at stage k, node i is denoted by ωikI Each node i ∈ N[1,µ(k)] at stage k ∈ N[0,N−2] has a set of children
child(k, i) ⊆ N[1,µ(k+1)]
I Each node i ∈ N[1,µ(k)] at stage k ∈ N[0,N−2] has a unique ancestorwhich is a node j ∈ N[1,µ(k−1)] at stage k − 1
35 / 94
Definitions
I Let N − 1 be the last stage of the tree
I At stage k we have µ(k) nodes and µ(0) = 1
I The nodes at stage N − 1 are called leaf nodes
I The node at stage 0 is the root node
I A scenario is an admissible path from the root node to a leaf node
I The tree counts µ(N − 1) scenarios
I The value of ω at stage k, node i is denoted by ωikI Each node i ∈ N[1,µ(k)] at stage k ∈ N[0,N−2] has a set of children
child(k, i) ⊆ N[1,µ(k+1)]
I Each node i ∈ N[1,µ(k)] at stage k ∈ N[0,N−2] has a unique ancestorwhich is a node j ∈ N[1,µ(k−1)] at stage k − 1
35 / 94
Definitions
I Let N − 1 be the last stage of the tree
I At stage k we have µ(k) nodes and µ(0) = 1
I The nodes at stage N − 1 are called leaf nodes
I The node at stage 0 is the root node
I A scenario is an admissible path from the root node to a leaf node
I The tree counts µ(N − 1) scenarios
I The value of ω at stage k, node i is denoted by ωikI Each node i ∈ N[1,µ(k)] at stage k ∈ N[0,N−2] has a set of children
child(k, i) ⊆ N[1,µ(k+1)]
I Each node i ∈ N[1,µ(k)] at stage k ∈ N[0,N−2] has a unique ancestorwhich is a node j ∈ N[1,µ(k−1)] at stage k − 1
35 / 94
Definitions
I Let N − 1 be the last stage of the tree
I At stage k we have µ(k) nodes and µ(0) = 1
I The nodes at stage N − 1 are called leaf nodes
I The node at stage 0 is the root node
I A scenario is an admissible path from the root node to a leaf node
I The tree counts µ(N − 1) scenarios
I The value of ω at stage k, node i is denoted by ωikI Each node i ∈ N[1,µ(k)] at stage k ∈ N[0,N−2] has a set of children
child(k, i) ⊆ N[1,µ(k+1)]
I Each node i ∈ N[1,µ(k)] at stage k ∈ N[0,N−2] has a unique ancestorwhich is a node j ∈ N[1,µ(k−1)] at stage k − 1
35 / 94
Definitions
I Let N − 1 be the last stage of the tree
I At stage k we have µ(k) nodes and µ(0) = 1
I The nodes at stage N − 1 are called leaf nodes
I The node at stage 0 is the root node
I A scenario is an admissible path from the root node to a leaf node
I The tree counts µ(N − 1) scenarios
I The value of ω at stage k, node i is denoted by ωik
I Each node i ∈ N[1,µ(k)] at stage k ∈ N[0,N−2] has a set of childrenchild(k, i) ⊆ N[1,µ(k+1)]
I Each node i ∈ N[1,µ(k)] at stage k ∈ N[0,N−2] has a unique ancestorwhich is a node j ∈ N[1,µ(k−1)] at stage k − 1
35 / 94
Definitions
I Let N − 1 be the last stage of the tree
I At stage k we have µ(k) nodes and µ(0) = 1
I The nodes at stage N − 1 are called leaf nodes
I The node at stage 0 is the root node
I A scenario is an admissible path from the root node to a leaf node
I The tree counts µ(N − 1) scenarios
I The value of ω at stage k, node i is denoted by ωikI Each node i ∈ N[1,µ(k)] at stage k ∈ N[0,N−2] has a set of children
child(k, i) ⊆ N[1,µ(k+1)]
I Each node i ∈ N[1,µ(k)] at stage k ∈ N[0,N−2] has a unique ancestorwhich is a node j ∈ N[1,µ(k−1)] at stage k − 1
35 / 94
Definitions
I Let N − 1 be the last stage of the tree
I At stage k we have µ(k) nodes and µ(0) = 1
I The nodes at stage N − 1 are called leaf nodes
I The node at stage 0 is the root node
I A scenario is an admissible path from the root node to a leaf node
I The tree counts µ(N − 1) scenarios
I The value of ω at stage k, node i is denoted by ωikI Each node i ∈ N[1,µ(k)] at stage k ∈ N[0,N−2] has a set of children
child(k, i) ⊆ N[1,µ(k+1)]
I Each node i ∈ N[1,µ(k)] at stage k ∈ N[0,N−2] has a unique ancestorwhich is a node j ∈ N[1,µ(k−1)] at stage k − 1
35 / 94
Definitions
Conditional probability:
pi,jk = P[ωk+1 = ωjk+1 | ωk = ωik]
We have ∑j∈child(k,i)
pi,jk = 1, for all k ∈ N[0,N−2], i ∈ N[1,µ(k)]
36 / 94
* Alternative definitions
1
2
3
4
5
6
7
8
9
10
37 / 94
* Alternative definitions
I We enumerate all nodes of the tree by an index α ∈ N[1,A]
I The value of ω at node α is denoted by ωαI The set of nodes at stage k is denoted by Ωk
I Each α ∈ Ωk, k ∈ N[0,N−1] defines the set of children nodeschild(α) ⊆ Ωk+1
I Each α ∈ Ωk, k ∈ N[1,N ] defines a unique ancestor anc(α) ∈ Ωk−1
I We define the probability vectors pα for α ∈ Ωk so that
X pα ∈ IR| child(α)|
X∑β∈child(α) p
α(β) = 1
X pα(β) = P[ωk+1 = β | ωk = α]
38 / 94
* Alternative definitions
I We enumerate all nodes of the tree by an index α ∈ N[1,A]
I The value of ω at node α is denoted by ωα
I The set of nodes at stage k is denoted by Ωk
I Each α ∈ Ωk, k ∈ N[0,N−1] defines the set of children nodeschild(α) ⊆ Ωk+1
I Each α ∈ Ωk, k ∈ N[1,N ] defines a unique ancestor anc(α) ∈ Ωk−1
I We define the probability vectors pα for α ∈ Ωk so that
X pα ∈ IR| child(α)|
X∑β∈child(α) p
α(β) = 1
X pα(β) = P[ωk+1 = β | ωk = α]
38 / 94
* Alternative definitions
I We enumerate all nodes of the tree by an index α ∈ N[1,A]
I The value of ω at node α is denoted by ωαI The set of nodes at stage k is denoted by Ωk
I Each α ∈ Ωk, k ∈ N[0,N−1] defines the set of children nodeschild(α) ⊆ Ωk+1
I Each α ∈ Ωk, k ∈ N[1,N ] defines a unique ancestor anc(α) ∈ Ωk−1
I We define the probability vectors pα for α ∈ Ωk so that
X pα ∈ IR| child(α)|
X∑β∈child(α) p
α(β) = 1
X pα(β) = P[ωk+1 = β | ωk = α]
38 / 94
* Alternative definitions
I We enumerate all nodes of the tree by an index α ∈ N[1,A]
I The value of ω at node α is denoted by ωαI The set of nodes at stage k is denoted by Ωk
I Each α ∈ Ωk, k ∈ N[0,N−1] defines the set of children nodeschild(α) ⊆ Ωk+1
I Each α ∈ Ωk, k ∈ N[1,N ] defines a unique ancestor anc(α) ∈ Ωk−1
I We define the probability vectors pα for α ∈ Ωk so that
X pα ∈ IR| child(α)|
X∑β∈child(α) p
α(β) = 1
X pα(β) = P[ωk+1 = β | ωk = α]
38 / 94
* Alternative definitions
I We enumerate all nodes of the tree by an index α ∈ N[1,A]
I The value of ω at node α is denoted by ωαI The set of nodes at stage k is denoted by Ωk
I Each α ∈ Ωk, k ∈ N[0,N−1] defines the set of children nodeschild(α) ⊆ Ωk+1
I Each α ∈ Ωk, k ∈ N[1,N ] defines a unique ancestor anc(α) ∈ Ωk−1
I We define the probability vectors pα for α ∈ Ωk so that
X pα ∈ IR| child(α)|
X∑β∈child(α) p
α(β) = 1
X pα(β) = P[ωk+1 = β | ωk = α]
38 / 94
* Alternative definitions
I We enumerate all nodes of the tree by an index α ∈ N[1,A]
I The value of ω at node α is denoted by ωαI The set of nodes at stage k is denoted by Ωk
I Each α ∈ Ωk, k ∈ N[0,N−1] defines the set of children nodeschild(α) ⊆ Ωk+1
I Each α ∈ Ωk, k ∈ N[1,N ] defines a unique ancestor anc(α) ∈ Ωk−1
I We define the probability vectors pα for α ∈ Ωk so that
X pα ∈ IR| child(α)|
X∑β∈child(α) p
α(β) = 1
X pα(β) = P[ωk+1 = β | ωk = α]
38 / 94
* Alternative definitions
I We enumerate all nodes of the tree by an index α ∈ N[1,A]
I The value of ω at node α is denoted by ωαI The set of nodes at stage k is denoted by Ωk
I Each α ∈ Ωk, k ∈ N[0,N−1] defines the set of children nodeschild(α) ⊆ Ωk+1
I Each α ∈ Ωk, k ∈ N[1,N ] defines a unique ancestor anc(α) ∈ Ωk−1
I We define the probability vectors pα for α ∈ Ωk so that
X pα ∈ IR| child(α)|
X∑β∈child(α) p
α(β) = 1
X pα(β) = P[ωk+1 = β | ωk = α]
38 / 94
* Alternative definitions
I We enumerate all nodes of the tree by an index α ∈ N[1,A]
I The value of ω at node α is denoted by ωαI The set of nodes at stage k is denoted by Ωk
I Each α ∈ Ωk, k ∈ N[0,N−1] defines the set of children nodeschild(α) ⊆ Ωk+1
I Each α ∈ Ωk, k ∈ N[1,N ] defines a unique ancestor anc(α) ∈ Ωk−1
I We define the probability vectors pα for α ∈ Ωk so that
X pα ∈ IR| child(α)|
X∑β∈child(α) p
α(β) = 1
X pα(β) = P[ωk+1 = β | ωk = α]
38 / 94
* Alternative definitions
I We enumerate all nodes of the tree by an index α ∈ N[1,A]
I The value of ω at node α is denoted by ωαI The set of nodes at stage k is denoted by Ωk
I Each α ∈ Ωk, k ∈ N[0,N−1] defines the set of children nodeschild(α) ⊆ Ωk+1
I Each α ∈ Ωk, k ∈ N[1,N ] defines a unique ancestor anc(α) ∈ Ωk−1
I We define the probability vectors pα for α ∈ Ωk so that
X pα ∈ IR| child(α)|
X∑β∈child(α) p
α(β) = 1
X pα(β) = P[ωk+1 = β | ωk = α]
38 / 94
A few properties
For every k ∈ N[0,N−2] we have
Ωk+1 =⋃
i∈N[1,µ(k)]
child(k, i)
and, for fixed k, sets child(k, i)i are disjoint.
39 / 94
A few properties
For α ∈ Ωk the family of sets child(α)α∈Ωk defines a partition in Ωk+1,that is
Ωk+1 =⋃α∈Ωk
child(α),
andα1 6= α2 ⇒ child(α1) ∩ child(α2) = ∅.
40 / 94
A few properties
The probability of a scenario15, identified by a leaf node α ∈ ΩN , isdefined as
πα = P[ωN = α | ω1 = ω10],
and is given by
πα =
N−1∏i=1
panci(α),
where anc1(α) = anc(α) and
anck+1(α) = anc(anck(α)).
15Detail: This probability is defined in the product space (∏k Ωk,⊗kFk).
41 / 94
Notes
I We allow wik = wjk for i 6= j – it is not the value that identifies thenode16!
I We have as many scenarios as leaf nodes, that is µ(N − 1),
I Scenarios are sequences (wikk )k where ik−1 = anc(k, ik),
I Every node wik of the tree can be identified by their stage k an ascenario running through it or
I by a sequence (wijj )j leading to wik,
I The probability of a scenario (wikk )k (on the space of scenarios),which is identified by a leaf node i ∈ N[1,µ(N−1)] is given by theproduct of conditional probabilities that connect its nodes.
16it is, instead, its history
42 / 94
Notes
I We allow wik = wjk for i 6= j – it is not the value that identifies thenode16!
I We have as many scenarios as leaf nodes, that is µ(N − 1),
I Scenarios are sequences (wikk )k where ik−1 = anc(k, ik),
I Every node wik of the tree can be identified by their stage k an ascenario running through it or
I by a sequence (wijj )j leading to wik,
I The probability of a scenario (wikk )k (on the space of scenarios),which is identified by a leaf node i ∈ N[1,µ(N−1)] is given by theproduct of conditional probabilities that connect its nodes.
16it is, instead, its history
42 / 94
Notes
I We allow wik = wjk for i 6= j – it is not the value that identifies thenode16!
I We have as many scenarios as leaf nodes, that is µ(N − 1),
I Scenarios are sequences (wikk )k where ik−1 = anc(k, ik),
I Every node wik of the tree can be identified by their stage k an ascenario running through it or
I by a sequence (wijj )j leading to wik,
I The probability of a scenario (wikk )k (on the space of scenarios),which is identified by a leaf node i ∈ N[1,µ(N−1)] is given by theproduct of conditional probabilities that connect its nodes.
16it is, instead, its history
42 / 94
Notes
I We allow wik = wjk for i 6= j – it is not the value that identifies thenode16!
I We have as many scenarios as leaf nodes, that is µ(N − 1),
I Scenarios are sequences (wikk )k where ik−1 = anc(k, ik),
I Every node wik of the tree can be identified by their stage k an ascenario running through it or
I by a sequence (wijj )j leading to wik,
I The probability of a scenario (wikk )k (on the space of scenarios),which is identified by a leaf node i ∈ N[1,µ(N−1)] is given by theproduct of conditional probabilities that connect its nodes.
16it is, instead, its history
42 / 94
Notes
I We allow wik = wjk for i 6= j – it is not the value that identifies thenode16!
I We have as many scenarios as leaf nodes, that is µ(N − 1),
I Scenarios are sequences (wikk )k where ik−1 = anc(k, ik),
I Every node wik of the tree can be identified by their stage k an ascenario running through it or
I by a sequence (wijj )j leading to wik,
I The probability of a scenario (wikk )k (on the space of scenarios),which is identified by a leaf node i ∈ N[1,µ(N−1)] is given by theproduct of conditional probabilities that connect its nodes.
16it is, instead, its history
42 / 94
Notes
I We allow wik = wjk for i 6= j – it is not the value that identifies thenode16!
I We have as many scenarios as leaf nodes, that is µ(N − 1),
I Scenarios are sequences (wikk )k where ik−1 = anc(k, ik),
I Every node wik of the tree can be identified by their stage k an ascenario running through it or
I by a sequence (wijj )j leading to wik,
I The probability of a scenario (wikk )k (on the space of scenarios17),which is identified by a leaf node i ∈ N[1,µ(N−1)] is given by theproduct of conditional probabilities that connect its nodes.
16it is, instead, its history17We are going to give a formal definition of this space later.
42 / 94
IID processes
43 / 94
Markov chains
E
A
E
E
E
A
A
44 / 94
Filtration
A filtration is an increasing sequence of σ-algebras which, here, we mayconstruct as follows
I FN−1 = 2ΩN−1 : take all subsets of ΩN−1
I Let FN−2 ⊆ FN−1 be the smallest σ-algebra containing allchild(N − 2, i) for all i.
I Recursively, construct FN−j ⊆ FN−j+1
I Eventually, F0 = ∅,ΩN−1 and recall that w0 is deterministic.
45 / 94
Filtration
A filtration is an increasing sequence of σ-algebras which, here, we mayconstruct as follows
I FN−1 = 2ΩN−1 : take all subsets of ΩN−1
I Let FN−2 ⊆ FN−1 be the smallest σ-algebra containing allchild(N − 2, i) for all i.
I Recursively, construct FN−j ⊆ FN−j+1
I Eventually, F0 = ∅,ΩN−1 and recall that w0 is deterministic.
45 / 94
Filtration
A filtration is an increasing sequence of σ-algebras which, here, we mayconstruct as follows
I FN−1 = 2ΩN−1 : take all subsets of ΩN−1
I Let FN−2 ⊆ FN−1 be the smallest σ-algebra containing allchild(N − 2, i) for all i.
I Recursively, construct FN−j ⊆ FN−j+1
I Eventually, F0 = ∅,ΩN−1 and recall that w0 is deterministic.
45 / 94
Filtration
A filtration is an increasing sequence of σ-algebras which, here, we mayconstruct as follows
I FN−1 = 2ΩN−1 : take all subsets of ΩN−1
I Let FN−2 ⊆ FN−1 be the smallest σ-algebra containing allchild(N − 2, i) for all i.
I Recursively, construct FN−j ⊆ FN−j+1
I Eventually, F0 = ∅,ΩN−1 and recall that w0 is deterministic.
45 / 94
Filtration
A filtration is an increasing sequence of σ-algebras which, here, we mayconstruct as follows
I FN−1 = 2ΩN−1 : take all subsets of ΩN−1
I Let FN−2 ⊆ FN−1 be the smallest σ-algebra containing allchild(N − 2, i) for all i.
I Recursively, construct FN−j ⊆ FN−j+1
I Eventually, F0 = ∅,ΩN−1 and recall that w0 is deterministic.
45 / 94
Filtration
46 / 94
Filtration
47 / 94
Filtration
48 / 94
Filtration
Some remarks:
I Every node i ∈ N[1,µ(k)] at a stage k corresponds to an event in Fk.
I The cardinality of Fk is 2|Ωk| (why?)
I The product space (Ω,F ,P) =∏k(Ωk,Fk,Pk), equipped with the
filtration Fkk becomes a filtered probability space.
49 / 94
State sequence
The state dynamics is given by xk+1 = f(xk, uk, wk), so, starting at k = 0from a state x0 and knowing w0 the predicted state sequence is18
x1 = f(x0, µ0(x0, w0), w0)
= f(x0, u0, w0)
and for all j ∈ N[0,N−2]
xj+1 = f(xj , µj(x0,wj , xj), wj)
where wj = (w0, w1, . . . , wj).
18Warning: abuse of notation! Here x0 is an observation whereas xk is an esti-mate of a future state (which is a random variable). Thus, x2 is not the statemeasurement at time k = 2; A more proper notation would be xk+2|k.
50 / 94
State sequence
51 / 94
Decision making across the tree nodes
I At k = 0 we know x0 and w0, so we decide u0
I At k = 1 the state will be x1 = f(x0, u0, w0) and we observe w1 ∈ Ω1
I For each i ∈ N[1,µ(1)] we decide a ui1 and apply it to the system
I The next state will be xi2 = f(x1, ui1, w
i1), i ∈ N[1,µ(1)]
I At stage k we decide the input uk according to the informationavailable so far, so
uk = µk(x0,wj , xj)
I or, equivalenty, we choose uik at each node of the tree at stage k.
52 / 94
State sequence
The state sequence becomes
xik+1 = f(xjk, uik, w
ik),
for all k ∈ N[0,N−2] and i ∈ N[1,µ(k+1)], where j = anc(k, i).
53 / 94
Adaptation to Fk
By construction ukk is an Fk-adapted random process (why?) and
E [uk | Fk] = uk.
Sequence xkk is a predictable process wrt Fkk, i.e., xk+1 isFk-adapted, that is
E [xk+1 | Fk] = xk+1.
Indeed, recall that
xk+1 = f(xk, µk(x0,wk, xk), wk).
54 / 94
DP on scenario trees
Assuming xk and wk are observable at time k; we decide uik at every edgeof the tree, i.e., one input for every node of the w-tree.
55 / 94
DP on scenario trees
Exercise.Solve the DP problem assuming xk and wk are observable at time k.Assume linear dynamics of the form19
xik+1 = A(wik)xjk +B(wik)u
ik + wik,
for all k ∈ N[0,N−2] and i ∈ N[1,µ(k+1)], where j = anc(k, i).
19For convenience we may denote Aik = A(wik) and Bik = B(wik). Since wk isobservable at time k, we also observe Ak and Bk.
56 / 94
DP on scenario trees
Assuming only xk is observable at time k; we decide uik at every edge ofthe x-tree, i.e., as in the following figure:
57 / 94
Scenario-based SMPC
We’ll see how to design a MS-stabilising SMPC using scenario treerepresentations of uncertainty.
58 / 94
Definition of LV
Consider the linear autonomous system
xk+1 = f(xk, wk).
Assume wk is not observable at time k and let V be a function whichmaps a x ∈ IRn to a IR+-valued random variable20 for which we define therandom variable
LV (xk) := E[V (xk+1)− V (xk) | Fk].
20We’ll avoid the details to keep the notation reasonably simple.
59 / 94
A useful property of LV
A useful property of LV (the discrete version of Dynkin’s formula.): For0 ≤ k1 ≤ k2:
E [V (xk2)− V (xk1) | Fk1 ] = E
k2∑j=k1
LV (xk)
∣∣∣∣∣ Fk1
Note: To prove this we only need to use the tower property:
H1 ⊆ H2 ⇒ E [E [X | H2] | H1] = E[X | H1],
where X is a r.v. on (Ω,F ,P) and Hi are sub-σ-algebras of F .
60 / 94
A useful property of LV
Proof. Take 0 ≤ k1 ≤ k2 and
V (xk2)− V (xk1) =
k2∑j=k1
V (xj+1)− V (xj)
⇒ E [V (xk2)− V (xk1) | Fk1 ] = E
k2∑j=k1
V (xj+1)− V (xj) | Fk1
= E
k2∑j=k1
E[V (xj+1)− V (xj) | Fj ] | Fk1
= E
k2∑j=k1
LV (xk)
∣∣∣∣∣ Fk1 .
61 / 94
Lyapunov theorem (MSS)
Assume that for all x ∈ IRn we have
LV (x) ≤ −γ‖x‖2,
for some γ > 0, then xk+1 = f(xk, wk) is MSS21.
Proof (Exercise). Use the property of LV with k1 = 0 and k2 = k.
21We further have that E[‖xk‖2 | F0
]k is an `2 sequence.
62 / 94
Lyapunov theorem (MSS)
If for all x ∈ IRn we have
LV (x) ≤ −α(‖x‖2),
for some convex K-class function α, then xk+1 = f(xk, wk) is MSS.
Proof. The proof is an exercise. Show with an example that the convexityrequirement cannot be omitted. Show also that the we can alternativelyuse the condition LV (x) ≤ −x′Lx, where L = L′ 0.
63 / 94
Lyapunov theorem (MSES)
If for all x ∈ IRn we have
LV (x)≤ −γ‖x‖2,α‖x‖2 ≤ V (x) ≤ β‖x‖2,
for some α, β, γ > 0, then xk+1 = f(xk, wk) is MSES.
Proof. Easy exercise.
64 / 94
MS-stabilising stochastic MPC
Assume that wk over (Ω,F ,P) is IID. We formulate the following SMPCproblem (unconstrained case) [Bernardini and Bemporad, 2012]:
V ?(xk) = minπ=µiN−1
i=0
EVN (xk, π)
xk|k = xk,
xk+i+1|k = A(wk+i|k)xk+i|k +B(wk+i|k)µi(xk+i|k), ∀i ∈ N[0,N−1]
LV (xk|k) ≤ −x′k|kLxk|k,
for some L = L′ 0, where V (x) = x′Px and
LV (xk|k) = E[V (xk+1|k)− V (xk) | xk|k]
65 / 94
MS-stabilising stochastic MPC
Because of the constraint
LV (xk|k) ≤ −x′k|kLxk|k,
the control lawuk = µ0(xk),
leads to a MSES closed-loop system (this SMPC problem is recursivelyfeasible).
66 / 94
MS-stabilising stochastic MPC
NOTE:We can choose any cost function VN (xk, π)!
67 / 94
What about those trees?
Hold on... we’ll get there.
68 / 94
Inexact knowledge of P(dwk)
We previously assumed that:
I wk is an IID process on a probability space (Ω,F ,P)
I The probability measure P is exactly known (and time invariant)
I In case Ω is finite and F = 2Ω if suffices to know p = (P(wi))si=1 forall wi ∈ Ω,
I When Ω is finite we can identify probability measures with vectors
I LetD =
p ∈ IRs : p ≥ 0, 1′sp = 1
,
thus any probability measure p is an element of D.
I A set P ⊆ D is a set of probability measures.
69 / 94
Inexact knowledge of P(dwk)
We previously assumed that:
I wk is an IID process on a probability space (Ω,F ,P)
I The probability measure P is exactly known (and time invariant)
I In case Ω is finite and F = 2Ω if suffices to know p = (P(wi))si=1 forall wi ∈ Ω,
I When Ω is finite we can identify probability measures with vectors
I LetD =
p ∈ IRs : p ≥ 0, 1′sp = 1
,
thus any probability measure p is an element of D.
I A set P ⊆ D is a set of probability measures.
69 / 94
Inexact knowledge of P(dwk)
We previously assumed that:
I wk is an IID process on a probability space (Ω,F ,P)
I The probability measure P is exactly known (and time invariant)
I In case Ω is finite and F = 2Ω if suffices to know p = (P(wi))si=1 forall wi ∈ Ω,
I When Ω is finite we can identify probability measures with vectors
I LetD =
p ∈ IRs : p ≥ 0, 1′sp = 1
,
thus any probability measure p is an element of D.
I A set P ⊆ D is a set of probability measures.
69 / 94
Inexact knowledge of P(dwk)
We previously assumed that:
I wk is an IID process on a probability space (Ω,F ,P)
I The probability measure P is exactly known (and time invariant)
I In case Ω is finite and F = 2Ω if suffices to know p = (P(wi))si=1 forall wi ∈ Ω,
I When Ω is finite we can identify probability measures with vectors
I LetD =
p ∈ IRs : p ≥ 0, 1′sp = 1
,
thus any probability measure p is an element of D.
I A set P ⊆ D is a set of probability measures.
69 / 94
Inexact knowledge of P(dwk)
We previously assumed that:
I wk is an IID process on a probability space (Ω,F ,P)
I The probability measure P is exactly known (and time invariant)
I In case Ω is finite and F = 2Ω if suffices to know p = (P(wi))si=1 forall wi ∈ Ω,
I When Ω is finite we can identify probability measures with vectors
I LetD =
p ∈ IRs : p ≥ 0, 1′sp = 1
,
thus any probability measure p is an element of D.
I A set P ⊆ D is a set of probability measures.
69 / 94
Inexact knowledge of P(dwk)
We previously assumed that:
I wk is an IID process on a probability space (Ω,F ,P)
I The probability measure P is exactly known (and time invariant)
I In case Ω is finite and F = 2Ω if suffices to know p = (P(wi))si=1 forall wi ∈ Ω,
I When Ω is finite we can identify probability measures with vectors
I LetD =
p ∈ IRs : p ≥ 0, 1′sp = 1
,
thus any probability measure p is an element of D.
I A set P ⊆ D is a set of probability measures.
69 / 94
Inexact knowledge of P(dwk)
We will now drop the IID assumption and assume that wk is a randomvariable over (Ω,F ,Pk) where Ω = wisi=1 is finite.
We define for P ∈ D
LV (x, µ,P) := EP[V (xk+1|k)− V (xk|k) | xk|k = x]
=
∫ΩV (xk+1|k)− V (xk) P(dw | xk|k = x)
=
s∑i=1
P[wk = wi]︸ ︷︷ ︸pi
V (A(wi)x+B(wi)µ(x))− V (x)
70 / 94
Inexact knowledge of P(dwk)
In that case, the MS-stabilising constraint becomes:
LV (x, µ0,P) ≤ −x′Lx,∀P ∈ D,
for some µ0.
No probabilistic information
Exact knowledge of the prob. measure
This is reminiscent of the worst-case distribution approach22.
22See A. Shapiro, “Worst-case distribution analysis of stochastic programs,” Math.Prog. 107(1), pp. 91-96, 2005.
71 / 94
MS-stabilising scenario-based SMPC
MS-stabilising SMPC formulation:
V ?(xk) = minπ=µiN−1
i=0
EVN (xk, π)
xk|k = xk,
xlk+i+1|k = A(wlk+i|k)xιk+i|k +B(wlk+i|k)u
lk+i|k,
∀i ∈ N[0,N−1], ι = anc(i, l), l ∈ N[1,µ(i+1)]
LV (xk+1|k, uk|k,P) ≤ −x′kLxk,∀P ∈ P
If P ⊆ D is a polytope with vertices PκKκ=1, we need to impose theabove stabilising constraint only for those vertices, i.e.,
LV (x1, u0,Pκ) ≤ −x′0Lx0,∀κ ∈ N[1,K]
72 / 94
The constrained case
I The same approach applies to state/input-constrained system so longas the formulation is recursively feasible
I For details see: D. Bernardini and A. Bemporad, “Stabilizing ModelPredictive Control of Stochastic Constrained Linear Systems,” IEEETAC 57(6), pp. 1468–1480, 2012.
73 / 94
IV. Affine disturbance feedback
74 / 94
Problem statement – dynamics
We’ll be studying a very simple case. The system dynamics is given by
xk+1 = Axk +Buk +Gwk,
where wk is an iid process with wk ∼ N (0, I). The disturbance wk is notobservable at time k.
75 / 94
Problem statement – constraints
The system is subject to hard input constraints
Huuk ≤ Ku,
and probabilistic state constraints (stage-wise):
P[H lx · xk ≤ K l
x] ≥ 1− αl, l ∈ N[1,s]
where H lx are vectors and K l
x are scalars and αl ∈ [0, 1].
76 / 94
Problem statement – policies
Along the horizon the inputs uk+j|k are determined by causal control lawsof the form
uk+j|k = ψk+j|k(xk|k, xk+1|k, . . . , xk+j|k)
= µk+j|k(xk|k, wk|k, . . . , wk+j−1|k),
so, along the horizon, we need to determine
µµµk =(uk|k, µk+1|k, . . . µk+N−1|k
)which is a sequence of functions. Functions are infinite dimensionalobjects, so the optimisation problem becomes intractable.
77 / 94
The stochastic MPC problem
We formulate the following optimisation problem:
V ?N (xk) = min
µµµkEVN (xk,µµµk),
subject to
xk+j+1|k = Axk+j|k +Buk+j|k +Gwk+j|k,∀j ∈ N[0,N−1]
uk+j|k = µk+j|k(xk|k, wk|k, . . . , wk+j−1|k), ∀j ∈ N[0,N−1]
Huuk+j|k ≤ Ku,∀j ∈ N[0,N−1]
P[hlx · xk+j|k ≤ klx] ≥ 1− αl, ∀l ∈ N[1,s], ∀j ∈ N[1,N ]
wk+j|k ∼ N (0, I), ∀j ∈ N[0,N−1]
xk|k = xk
78 / 94
Affine disturbance feedback
For convenience define
wk = (wk|k, wk+1|k, . . . , wk+N−1|k).
We restrict ourselves to causal policies whose functions have the simpleform
µk+j|k(wk) = mj +
j−1∑i=0
Mj,iwk+j|k
We then need to determine Mj,i and mj , so the opt. problem becomesfinite dimensional.
79 / 94
Affine disturbance feedback
The affine disturbance feedback policy can be concisely written as
µµµk(wk) = Mwk + m,
where
M =
0 . . . . . . 0 0
M1,0 0 . . . 0 0...
MN−1,0 MN−1,1 . . . MN−1,N−2 0
, m =
m0
m1...
mN−1
80 / 94
Finite dimensional problem
Define also
xk = (xk, xk+1|k,...,xk+N|k),
uk = (uk|k, uk+1|k,...,uk+N−1|k).
The state evolution of the system, xk given a sequence of inputs uk and asequence of disturbances wk is given by23
xk = Axk + Buk + Gwk.
23Exercise: Determine A, B and G.
81 / 94
Probabilistic constraints
... and substituting uk = µµµk(wk):
xk = Axk + Buk + Gwk
= Axk + BMwk + Gwk + Bm
= Axk + (BM + G)wk + Bm
Then, the probabilistic constraints are written as
P[Hlx(Axk + (BM + G)wk + Bm) ≤ Kl
x] ≥ 1− αl
⇔ P[Hlx(BM + G)wk + Hl
x(Axk + Bm) ≤ Klk] ≥ 1− αl
82 / 94
Distribution function
Let X be a random value over a probability space (IR,BIR,P). Thedistribution of X is a measure µ : BIR → [0, 1] with
µ(B) = P(X ∈ B)
= P(ω | X(ω) ∈ B).
The distribution of X is identified by the following function (why?) knownas distribution function of X, c : IR→ [0, 1]
c(x) = µ((−∞, x])
= P(X < x)
= P(ω ∈ Ω | X(ω) < x).
83 / 94
Quantile function
We define the quantile function of X, Q : [0, 1]→ IR as
Q(p) = infx ∈ IR : p ≤ c(x)= infx ∈ IR : P[X ≤ x] ≥ p
This is a type of inverse of the distribution function.
84 / 94
Normal distribution
When X is normally distributed, X ∼ N (µ, σ), Q(p) is given by anexplicit formula, that is
Q(p) = µ+ 2√σ erf−1(2p− 1).
As a result
P[X ≤ x] ≥ 1− α⇔ x ≥ Q(1− α).
85 / 94
Multivariate Normal distribution
When X is normally distributed, X ∼ N (x,Σ) and Z := AX + b, then
Z ∼ N (Ax+ b, AΣA′).
86 / 94
A little observation
Let y1 ∼ N (0n, In) and y2 ∼ N (0n, In) and y1, y2 are independentrandom variables. Then
y := (y1, y2) ∼ N (02n, I2n).
In our casewk ∼ N (0, I).
87 / 94
Probabilistic constraints revisited
The probabilistic constraints are written as
P[Hlx(BM + G)︸ ︷︷ ︸row-vector
wk︸︷︷︸N (0,I)
≤ gl] ≥ 1− αl
where gl = Klk −Hl
x(Axk + Bm). This becomes
Q(1− αl)‖Hlx(BM + G)‖2 ≤ gl,
where Q is the quantile function of N (0, 1). This leads to the formulationof a second-order cone problem.
88 / 94
Simplification
Consider the following constraints with parameter βl > 0Hlx(BM + G)wl
k + Hlx(Axk + Bm) ≤ Kl
x
‖wlk‖∞ ≤ βl
Then, a posteriori we can determine the probability
P[Hlxxk ≤ Kl
x | ‖wlk‖∞ ≤ βl],
given that wlk ∼ N (0, I)
89 / 94
Simplification
But we know that
P[Hlxxk > Kl
k | ‖wlk‖ ≤ βl] ≤
√eβle−
(βl)2
2 ,
so for the probabilistic constraints to be satisfied it suffices to choose βl sothat
√eβle−
(βl)2
2 ≤ αl
90 / 94
Discussion
X Affine disturbance feedback and affine state feedback are equivalent[Goulart et al., ’05]
X it is a suboptimal choice in the space of measurable functions
X But it is a computationally tractable approach
X The problem size explodes as the prediction horizon increases
X But there are approximating techniques such as the blocking affineparametrization.
91 / 94
Discussion
X Affine disturbance feedback and affine state feedback are equivalent[Goulart et al., ’05]
X it is a suboptimal choice in the space of measurable functions
X But it is a computationally tractable approach
X The problem size explodes as the prediction horizon increases
X But there are approximating techniques such as the blocking affineparametrization.
91 / 94
Discussion
X Affine disturbance feedback and affine state feedback are equivalent[Goulart et al., ’05]
X it is a suboptimal choice in the space of measurable functions
X But it is a computationally tractable approach
X The problem size explodes as the prediction horizon increases
X But there are approximating techniques such as the blocking affineparametrization.
91 / 94
Discussion
X Affine disturbance feedback and affine state feedback are equivalent[Goulart et al., ’05]
X it is a suboptimal choice in the space of measurable functions
X But it is a computationally tractable approach
X The problem size explodes as the prediction horizon increases
X But there are approximating techniques such as the blocking affineparametrization.
91 / 94
Discussion
X Affine disturbance feedback and affine state feedback are equivalent[Goulart et al., ’05]
X it is a suboptimal choice in the space of measurable functions
X But it is a computationally tractable approach
X The problem size explodes as the prediction horizon increases
X But there are approximating techniques such as the blocking affineparametrization.
91 / 94
Discussion (cont’d)
X Recursive feasibility is impossible to guarantee when the disturbanceis unbounded
X Recursively feasible formulations are — as expected — tooconservative [M. Korda et al. ’11]
X We can however design a probabilistically resolvable control scheme[M. Ono, ’12]
92 / 94
Discussion (cont’d)
X Recursive feasibility is impossible to guarantee when the disturbanceis unbounded
X Recursively feasible formulations are — as expected — tooconservative [M. Korda et al. ’11]
X We can however design a probabilistically resolvable control scheme[M. Ono, ’12]
92 / 94
Discussion (cont’d)
X Recursive feasibility is impossible to guarantee when the disturbanceis unbounded
X Recursively feasible formulations are — as expected — tooconservative [M. Korda et al. ’11]
X We can however design a probabilistically resolvable control scheme[M. Ono, ’12]
92 / 94
Further reading
1. M. Lorenzen, F. Dabbene, R. Tempo and F. Allgower, Constraint-Tightening andStability in Stochastic Model Predictive Control, arXiv: 1511.03488v1, 2015.
2. M. Oldewurtel, C.N. Jones, M. Morari, A Tractable Approximation of ChanceConstrained Stochastic MPC based on Affine Disturbance Feedback, 2008.
3. M. Ono, Closed-Loop Chance-Constrained MPC with Probabilistic Resolvability,51st IEEE CDC, Maui, Hawaii, USA, 2012.
4. M. Cannon, B. Kouvaritakis, D. Ng, Probabilistic tubes in linear stochastic modelpredictive control, Systems & Control Letters 58, pp. 747–753, 2009.
5. D. Chatterjee, P. Hokayem, J. Lygeros, Stochastic receding horizon control withbounded control inputs: a vector space approach, arXiv:0903.5444v3, 2009.
6. M. Korda, R. Gondhalekar, J. Cigler, and F. Oldewurtel, Strongly FeasibleStochastic Model Predictive Control, IEEE CDC, 2011.
7. M.Cannon, P. Couchman and B. Kouvaritakis, MPC for stochastic systems,Chapter in Assessment and Future Directions of Nonlinear Model PredictiveControl, Springer, 2007.
93 / 94
Further reading
8. M. Cannon, Stochastic Model Predictive Control: State space methods, Lecturenotes available online at http://users.ox.ac.uk/∼engs0169/pdf/cannon ifac08c.pdf
9. H. Kushner, Introduction to stochastic control, Holt, Rinehart and Winston Inc.,New York, 1971.
10. M. Agarwal, E. Cinquemani, D. Chatterjee and J. Lygeros, On convexity ofstochastic optimization problems with constraints, European Control Conference(ECC), Budapest, Hungary, 2009.
94 / 94