Monitoring the Evolution of Cumulus Cloudswith a Fleet of UAVs
Alessandro Renzaglia, Christophe Reymann and Simon Lacroix
LAAS-CNRS, Toulouse
9 Mar. 2016 / Montpellier
A. Renzaglia (LAAS-CNRS) SkyScanner@LAAS 9 Mar. 2016 1 / 32
Introduction: the SkyScanner project
Outline
1 Introduction: the SkyScanner project
2 Modeling the environmentProblem StatementGaussian processes: an introductionLinks between models and planning
3 Trajectory generationProblem FormulationOptimization Method
4 Experiments & ResultsMeso-NH simulations: a realistic environmentPath planning: preliminary results
5 Summary and prospects
A. Renzaglia (LAAS-CNRS) SkyScanner@LAAS 9 Mar. 2016 2 / 32
Introduction: the SkyScanner project
Mapping the micro-physical properties of cumulusclouds
Spatio-temporal evolution of µ-physical properties of a cumulus ?Issues :
Size, at the base: ≈100m,height: ≈1kmShort lifespan ≈ 20min to 1h
Solution:A plane ?A drone (UAV) ?⇒ A fleet of UAVs
A. Renzaglia (LAAS-CNRS) SkyScanner@LAAS 9 Mar. 2016 3 / 32
Introduction: the SkyScanner project
The SkyScanner project
Project financed by the STAE foundation
CNRM: Model of thecloud µ-physicsISAE: Conception ofan automaticallyoptimized vehicleONERA: Optimizedcontrol architectureENAC: 3D Windestimation, thePaparazzi autopilotLAAS: Path planningand mapping
A. Renzaglia (LAAS-CNRS) SkyScanner@LAAS 9 Mar. 2016 4 / 32
Introduction: the SkyScanner project
Objectives of a mission
A fleet (>2) of UAVs has to collect data inside a cumulus cloudObjectives:
(Maximum) duration: 1h
Identify the different areas and characteristic variables of thecloud:altitude of the base, height, strength of the ascending currents,...⇒ parametric, conceptual model
Map the evolution of some µ-physical parameters in predefinedareas⇒ (dense) statistical model
A. Renzaglia (LAAS-CNRS) SkyScanner@LAAS 9 Mar. 2016 5 / 32
Introduction: the SkyScanner project
A hierarchy of models
parametric modelEx: updraft = f (diam, height)
stochastic regression model
A. Renzaglia (LAAS-CNRS) SkyScanner@LAAS 9 Mar. 2016 6 / 32
Introduction: the SkyScanner project
Two-stage planning approach
Task planning
∆T ≈ 1min
Path planning
∆T ≈ 10sec
A. Renzaglia (LAAS-CNRS) SkyScanner@LAAS 9 Mar. 2016 7 / 32
Modeling the environment
Outline
1 Introduction: the SkyScanner project
2 Modeling the environmentProblem StatementGaussian processes: an introductionLinks between models and planning
3 Trajectory generationProblem FormulationOptimization Method
4 Experiments & ResultsMeso-NH simulations: a realistic environmentPath planning: preliminary results
5 Summary and prospects
A. Renzaglia (LAAS-CNRS) SkyScanner@LAAS 9 Mar. 2016 8 / 32
Modeling the environment Problem Statement
Modeling the environment
Issues:
Reconstructing a 3D+Time map from punctual and sparsemeasurements
Size of a cumulus
We use Gaussian Processes to solve this regression problem
«A Gaussian Process is a collection of random variables, any finitenumber of which have a joint Gaussian distribution.», CE Rasmussenet CKI Williams, Gaussian Processes for Machine Learning (2006)).
Gaussian ⇒ entirely defining by its mean and covariance
A. Renzaglia (LAAS-CNRS) SkyScanner@LAAS 9 Mar. 2016 9 / 32
Modeling the environment Problem Statement
Gaussian Process Regression (GPR)
+ Abundance in literature (although more statistics - big data)+ Continuous world, prediction of the error
+/− Cost of inference: O(n3) building model, O(n2) each inference− Choosing kernels, slow hyper-parameters optimization
Noiseless GPR GPR with noise
A. Renzaglia (LAAS-CNRS) SkyScanner@LAAS 9 Mar. 2016 10 / 32
Modeling the environment Gaussian processes: an introduction
Problem statement
Given some samples, we wish to reconstruct the underlying process f
Definition
f : Rnfeatures −→ RX �−→ y = f (X )
In our case, nfeatures = 4
→ the features are the space-time locations of the samples
A. Renzaglia (LAAS-CNRS) SkyScanner@LAAS 9 Mar. 2016 11 / 32
Modeling the environment Gaussian processes: an introduction
Definitions
Definitionf ≈ GP(m(X), k(X,X))m(x): expectation, zero in most casesk(x, x�): covariance or kernel (function)
DefinitionLet (X,Y ) be an ensemble of n samples in Rk × R, then:
m(x) := 0 (zero mean)
Σ := k(X,X) + σ2noiseI
Σ is the n × n covariance matrix accounting for a Gaussian noise ofvariance σ2
noise
A. Renzaglia (LAAS-CNRS) SkyScanner@LAAS 9 Mar. 2016 12 / 32
Modeling the environment Gaussian processes: an introduction
Kernel families
How to choose the covariance function (kernel) ?
Squared Exponential kernel family
Choose a kernel family→ sets a prior (stationarity,periodicity...)
Set hyper-parameters→ optimization
A. Renzaglia (LAAS-CNRS) SkyScanner@LAAS 9 Mar. 2016 13 / 32
Modeling the environment Gaussian processes: an introduction
Inference
TheoremInference: for a given sample x� in RK
y� = k(x�,X)Σ−1y
V(y�) = k(x�, x�)− k(x�,X)Σ−1k(x�,X)�
y� is the mean, V(y�) the variance at point x� of the functionsrepresented by the GP conditioned by the X previous samples
Cost of inference:O(n3) for inverting Σ
O(n2) for subsequent predictions
A. Renzaglia (LAAS-CNRS) SkyScanner@LAAS 9 Mar. 2016 14 / 32
Modeling the environment Gaussian processes: an introduction
Illustration of the mapping process
2.10 2.15 2.20 2.25 2.30 2.35 2.40
x coordinate (km)
2.10
2.15
2.20
2.25
2.30
2.35
2.40
yco
ordi
nate
(km
)
Ground truth
−0.5
0.0
0.5
1.0
1.5
2.0
2.5
3.0
3.5
2.10 2.15 2.20 2.25 2.30 2.35 2.40
x coordinate (km)
2.10
2.15
2.20
2.25
2.30
2.35
2.40
yco
ordi
nate
(km
)
Predicted mean y�
−0.5
0.0
0.5
1.0
1.5
2.0
2.5
3.0
3.5
2.10 2.15 2.20 2.25 2.30 2.35 2.40
x coordinate (km)
2.10
2.15
2.20
2.25
2.30
2.35
2.40
yco
ordi
nate
(km
)
Predicted variance V [y�]
0.25
0.50
0.75
1.00
1.25
1.50
1.75
2.00
2.10 2.15 2.20 2.25 2.30 2.35 2.40
x coordinate (km)
2.10
2.15
2.20
2.25
2.30
2.35
2.40
yco
ordi
nate
(km
)
Ground truth
−0.5
0.0
0.5
1.0
1.5
2.0
2.5
3.0
3.5
2.10 2.15 2.20 2.25 2.30 2.35 2.40
x coordinate (km)
2.10
2.15
2.20
2.25
2.30
2.35
2.40
yco
ordi
nate
(km
)
Predicted mean y�
−0.5
0.0
0.5
1.0
1.5
2.0
2.5
3.0
3.5
2.10 2.15 2.20 2.25 2.30 2.35 2.40
x coordinate (km)
2.10
2.15
2.20
2.25
2.30
2.35
2.40
yco
ordi
nate
(km
)
Predicted variance V [y�]
0.25
0.50
0.75
1.00
1.25
1.50
1.75
2.00
2.10 2.15 2.20 2.25 2.30 2.35 2.40
x coordinate (km)
2.10
2.15
2.20
2.25
2.30
2.35
2.40
yco
ordi
nate
(km
)
Ground truth
−0.6
0.0
0.6
1.2
1.8
2.4
3.0
3.6
2.10 2.15 2.20 2.25 2.30 2.35 2.40
x coordinate (km)
2.10
2.15
2.20
2.25
2.30
2.35
2.40
yco
ordi
nate
(km
)
Predicted mean y�
−0.6
0.0
0.6
1.2
1.8
2.4
3.0
3.6
2.10 2.15 2.20 2.25 2.30 2.35 2.40
x coordinate (km)
2.10
2.15
2.20
2.25
2.30
2.35
2.40
yco
ordi
nate
(km
)
Predicted variance V [y�]
0.25
0.50
0.75
1.00
1.25
1.50
1.75
2.00
A. Renzaglia (LAAS-CNRS) SkyScanner@LAAS 9 Mar. 2016 15 / 32
Modeling the environment Links between models and planning
Information gathering
Is a planned path interesting in terms of gathered information ?
Goal: maximize information given a criterion on the model
minimizing the covariance between samples:D: maximize differential entropy (log-det of Σ−1)T: maximize trace of Σ−1
Let m new samples Xnew and ΣX the n × n covariance matrix betweenthe n previous samples X :
ΣXnew |X = k(Xnew ,Xnew )− k(Xnew ,X)Σ−1X k(Xnew ,X)�
We then compute directly D and T from the conditional covariance ΣXnew |X
A. Renzaglia (LAAS-CNRS) SkyScanner@LAAS 9 Mar. 2016 16 / 32
Modeling the environment Links between models and planning
Information gathering
Is a planned path interesting in terms of gathered information ?
Goal: maximize information given a criterion on the model
minimizing the covariance between samples:D: maximize differential entropy (log-det of Σ−1)T: maximize trace of Σ−1
Let m new samples Xnew and ΣX the n × n covariance matrix betweenthe n previous samples X :
ΣXnew |X = k(Xnew ,Xnew )− k(Xnew ,X)Σ−1X k(Xnew ,X)�
We then compute directly D and T from the conditional covariance ΣXnew |X
A. Renzaglia (LAAS-CNRS) SkyScanner@LAAS 9 Mar. 2016 16 / 32
Trajectory generation
Outline
1 Introduction: the SkyScanner project
2 Modeling the environmentProblem StatementGaussian processes: an introductionLinks between models and planning
3 Trajectory generationProblem FormulationOptimization Method
4 Experiments & ResultsMeso-NH simulations: a realistic environmentPath planning: preliminary results
5 Summary and prospects
A. Renzaglia (LAAS-CNRS) SkyScanner@LAAS 9 Mar. 2016 17 / 32
Trajectory generation Problem Formulation
Problem Formulation
Task to achieveMaximize the information gain within an area of interest whileminimizing the energy consumption
Centralized approach: ground station and no communication issues
Aircraft dynamics:
Constant airspeedConstrained control inputs: turn radius R and power input Pin
A. Renzaglia (LAAS-CNRS) SkyScanner@LAAS 9 Mar. 2016 18 / 32
Trajectory generation Problem Formulation
Problem Formulation
Task to achieveMaximize the information gain within an area of interest whileminimizing the energy consumption
Centralized approach: ground station and no communication issues
Aircraft dynamics:
Constant airspeedConstrained control inputs: turn radius R and power input Pin
A. Renzaglia (LAAS-CNRS) SkyScanner@LAAS 9 Mar. 2016 18 / 32
Trajectory generation Problem Formulation
From an objective to an optimization function
Three criteria to optimize during the mission:Energy:
U(j)E (t0,∆T ) = 1 − 1
Pmaxin ∆T
t0+∆T�
t=t0
P(j)in (t)dt
Information Gain:
UI(v) = max�
0, min�
1,v + vTb − 2vTp
2(vTb − vTp)
��
Region of Interest:
UjG(t0,∆T ) =
db(X jt0+∆T )− db(X j
t0)
Vzmax∆T
Total Utility Function:
Utot = wEUE + wIUI + wGUG
A. Renzaglia (LAAS-CNRS) SkyScanner@LAAS 9 Mar. 2016 19 / 32
Trajectory generation Optimization Method
Approach
Finite (short) horizon ∆T ∼ 20sModel reliabilityComputational complexity
Planning in control spaceCurrents strongly affect navigation (unfeasible movements,unreachable areas, etc.)
The trajectories are discretized and defined as a sequence ofcontrol inputs {u0,udt , ...,u∆T−dt}
A. Renzaglia (LAAS-CNRS) SkyScanner@LAAS 9 Mar. 2016 20 / 32
Trajectory generation Optimization Method
Approach
Two-step optimization scheme:
Blind random sampling fortrajectories initializations
Constrained stochastic gradientascent algorithm (SPSA) withlocal convergence guarantee
A. Renzaglia (LAAS-CNRS) SkyScanner@LAAS 9 Mar. 2016 21 / 32
Trajectory generation Optimization Method
Stochastic Gradient Approximation
Stochastic Gradient Approximation by Simultaneous PerturbationStochastic Approximation (SPSA) algorithm:
uk+1 = Π(uk + ak g(uk ))
gk (uk ) =
U(uk+ck∆k )−U(uk−ck∆k )2ck∆k1
...U(uk+ck∆k )−U(uk )−ck∆k
2ck∆kN
ak > 0, ck > 0, ak → 0, ck → 0,∞�
k=0
ak = ∞,∞�
k=0
a2k
c2k< ∞
A. Renzaglia (LAAS-CNRS) SkyScanner@LAAS 9 Mar. 2016 22 / 32
Trajectory generation Optimization Method
An illustrative example
Artificial 2D wind field
Fictitious utility function
The goal is to maximize theutility collected along thepath
A. Renzaglia (LAAS-CNRS) SkyScanner@LAAS 9 Mar. 2016 23 / 32
Experiments & Results
Outline
1 Introduction: the SkyScanner project
2 Modeling the environmentProblem StatementGaussian processes: an introductionLinks between models and planning
3 Trajectory generationProblem FormulationOptimization Method
4 Experiments & ResultsMeso-NH simulations: a realistic environmentPath planning: preliminary results
5 Summary and prospects
A. Renzaglia (LAAS-CNRS) SkyScanner@LAAS 9 Mar. 2016 24 / 32
Experiments & Results Meso-NH simulations: a realistic environment
Meso-NH simulation: example
Still frame from a simulation. In shades of grey the liquid water
content, in orange the +0.5 m.s−1 upwind isometric curves
A. Renzaglia (LAAS-CNRS) SkyScanner@LAAS 9 Mar. 2016 25 / 32
Experiments & Results Meso-NH simulations: a realistic environment
Meso-NH simulations
Meso-NH: Large scale simulations, model created and validated bymeteorologists. Simulations provided by the CNRM.
Scenario = cumulus cloud field arising from daily convectionSimulates all micro-physical properties (incl. wind)...700 MB per frameat one frame per secondOne hour → 2.5 TB...weeks of computing on the meteo-france cluster
→ Statistical study of the environmental model (pending)
A. Renzaglia (LAAS-CNRS) SkyScanner@LAAS 9 Mar. 2016 26 / 32
Experiments & Results Path planning: preliminary results
One agent in a static wind field
UAV trajectoryAltitude of the UAV
during the flight
A. Renzaglia, C. Reymann, and S. Lacroix, “Monitoring the evolution of clouds with UAVs“, ICRA 2016
A. Renzaglia (LAAS-CNRS) SkyScanner@LAAS 9 Mar. 2016 27 / 32
Experiments & Results Path planning: preliminary results
Three agents in a dynamic wind field
x(km
)
2.0
2.1
2.2
2.3
2.4
2.5 y (km)2.0
2.12.2
2.32.4
2.5z
(km
)0.80
0.85
0.90
0.95
1.00
x(km
)
2.0
2.1
2.2
2.3
2.4
2.5 y (km)2.0
2.12.2
2.32.4
2.5
z(k
m)
0.80
0.85
0.90
0.95
1.00
x(km
)2.0
2.1
2.2
2.3
2.4
2.5 y (km)2.0
2.12.2
2.32.4
2.5
z(k
m)
0.80
0.85
0.90
0.95
1.00
−1.6 −0.8 0.0 0.8 1.6 2.4 3.2 4.0
Vertical wind velocity in m.s−1
0 50 100 150 200 250 300
Time (seconds)
800
850
900
950
1000
1050
Alti
tude
(met
ers)
AllNo informationNo energy
All No information No energy74
76
78
80
82
84
86
88
90
Bat
tery
leve
l(%
)
Trajectories, altitude profilesand battery levels for threeUAVs flying simultaneously
C. Reymann, A. Renzaglia, F. Lamraoui, M. Bronz and S. Lacroix, ”Adaptive Sampling of Cumulus Clouds with a Fleet of UAVs“,
Autonomous Robots, under review
A. Renzaglia (LAAS-CNRS) SkyScanner@LAAS 9 Mar. 2016 28 / 32
Summary and prospects
Outline
1 Introduction: the SkyScanner project
2 Modeling the environmentProblem StatementGaussian processes: an introductionLinks between models and planning
3 Trajectory generationProblem FormulationOptimization Method
4 Experiments & ResultsMeso-NH simulations: a realistic environmentPath planning: preliminary results
5 Summary and prospects
A. Renzaglia (LAAS-CNRS) SkyScanner@LAAS 9 Mar. 2016 29 / 32
Summary and prospects
Summary
SkyScanner@LAAS, today:
A realistic meteorological simulationA stochastic environmental model using GPRA simple utility function integrating energy and IG criteriaA stochastic path planning algorithm
⇒ The first iteration of a complete simulation environment
A. Renzaglia (LAAS-CNRS) SkyScanner@LAAS 9 Mar. 2016 30 / 32
Summary and prospects
Prospects
Environment model:better handling of the time dimensiondeveloping a conceptual cloud modelembedding prior knowledge into the GP model
Planning:proper multi-criteria utility functionusing Paparazzi which integrates a realistic FDMtask planning
Real experiments
A. Renzaglia (LAAS-CNRS) SkyScanner@LAAS 9 Mar. 2016 31 / 32
Summary and prospects
Questions ?
Thank you for your attention
More information about the SkyScanner project at:https://www.laas.fr/projects/skyscanner/
A. Renzaglia (LAAS-CNRS) SkyScanner@LAAS 9 Mar. 2016 32 / 32