adjoint based inversion in geodynamic modelling: a guide · ada lovelace workshop 2019 georg reuber...
TRANSCRIPT
Georg Reuber,
Boris Kaus, Anton Popov
Adjoint based inversion in geodynamic modelling:A guide
Motivation
229/8/2019Ada Lovelace Workshop 2019
Georg Reuber
Geodynamic modelling
Motivation
329/8/2019Ada Lovelace Workshop 2019
Georg Reuber
Geodynamic modelling
Idealized test
Pusok (2015)
• Parameterize real scenario→ Easy to test/quantify effects
Motivation
429/8/2019Ada Lovelace Workshop 2019
Georg Reuber
Geodynamic modelling
Idealized test Inversion
Pusok (2015)
• Parameterize real scenario→ Easy to test/quantify effects
• Fit data→ Ultimately represents nature
Baumann (2015)
uobs
ρ,η
Motivation
529/8/2019Ada Lovelace Workshop 2019
Georg Reuber
Measurement device
Physical description
Inversion
Motivation
629/8/2019Ada Lovelace Workshop 2019
Georg Reuber
Measurement device
Physical description Computer simulation: u
(Field) measurement: uobs
Inversion
Motivation
729/8/2019Ada Lovelace Workshop 2019
Georg Reuber
Measurement device
Physical description Computer simulation: u
(Field) measurement: uobs
Cost function:F = 0.5(u-uobs)
T(u-uobs)
Inversion
Motivation
829/8/2019Ada Lovelace Workshop 2019
Georg Reuber
Measurement device
Physical description Computer simulation: u
(Field) measurement: uobs
Cost function:F = 0.5(u-uobs)
T(u-uobs)
Inversion
Update computer
Simulation until F = 0
Motivation
929/8/2019Ada Lovelace Workshop 2019
Georg Reuber
E.g: uobs
Inversion
ρ,η
Motivation
1029/8/2019Ada Lovelace Workshop 2019
Georg Reuber
ρ η
F (u, uobs)
Inversion
uobs
ρ,η
Motivation
1129/8/2019Ada Lovelace Workshop 2019
Georg Reuber
Simulation → Find minimum of F
Inversion
uobs
ρ,η
ρ η
F (u, uobs)
Motivation
1229/8/2019Ada Lovelace Workshop 2019
Georg Reuber
Grid search
Inversion
uobs
ρ,η
→ Find minimum of F
ρ η
F (u, uobs)
Motivation
1329/8/2019Ada Lovelace Workshop 2019
Georg Reuber
Statistical
• (Randomized) statistical search
ρ
η → Find minimum of F
F (u, uobs)
Inversion
Motivation
1429/8/2019Ada Lovelace Workshop 2019
Georg Reuber
Statistical
• Grid Search Sampling
ρ
η
F (u, uobs)
Inversion
Motivation
1529/8/2019Ada Lovelace Workshop 2019
Georg Reuber
Statistical
• Grid Search Sampling→Many simulations→ Broad knowledge of cost function
ρ
η
F (u, uobs)
Inversion
Motivation
1629/8/2019Ada Lovelace Workshop 2019
Georg Reuber
Inversion
Statistical
• Monte-Carlo Sampling
ρ
η
Motivation
1729/8/2019Ada Lovelace Workshop 2019
Georg Reuber
Inversion
Statistical
• Monte-Carlo Sampling
ρ
η
Motivation
1829/8/2019Ada Lovelace Workshop 2019
Georg Reuber
Inversion
Statistical
• Monte-Carlo Sampling→Many simulations→ Broad knowledge of cost function
ρ
η
Motivation
1929/8/2019Ada Lovelace Workshop 2019
Georg Reuber
Inversion
Statistical
• MCMC Sampling
ρ
η
Motivation
2029/8/2019Ada Lovelace Workshop 2019
Georg Reuber
Inversion
Statistical
• MCMC Sampling
ρ
η
Motivation
2129/8/2019Ada Lovelace Workshop 2019
Georg Reuber
Inversion
Statistical
• MCMC Sampling→ Few simulations→ Requires fine-tuning
ρ
η
Motivation
2229/8/2019Ada Lovelace Workshop 2019
Georg Reuber
ρ
η
Inversion
Statistical
• Bayes Theorem→ Prior knowledge goes in sampling
Motivation
2329/8/2019Ada Lovelace Workshop 2019
Georg Reuber
ρ
η
Inversion
Statistical
… many more methods
• Bayes Theorem→ Prior knowledge goes in sampling→ Compute posterior→ Uncertainty in parameters
Motivation
2429/8/2019Ada Lovelace Workshop 2019
Georg Reuber
Deterministic
∇F
Inversion
ρ
η
Gradient based:
Motivation
2529/8/2019Ada Lovelace Workshop 2019
Georg Reuber
• Few ‚simulations‘• Sensitive to local minima
Inversion
Deterministic
Gradient based:
ρ
η
Motivation
2629/8/2019Ada Lovelace Workshop 2019
Georg Reuber
Gradient based [gradient descent]:
Inversion
Deterministic
ρ
η
Motivation
2729/8/2019Ada Lovelace Workshop 2019
Georg Reuber
Gradient based [(Quasi)-Newton]:
… more methods
Hessian matrix→ Relates to the
covariance matrixin Bayesian context
Inversion
Deterministic
ρ
η
Motivation
2829/8/2019Ada Lovelace Workshop 2019
Georg Reuber
• Few ‚simulations‘• Sensitive to local minima
• How to evaluate gradient?
Inversion
Deterministic
Gradient based:
ρ
η
Motivation
2929/8/2019Ada Lovelace Workshop 2019
Georg Reuber
• Few ‚simulations‘• Sensitive to local minima
• How to evaluate gradient?
→ Here simple: analytical
→ Or FD:
Inversion
Deterministic
Gradient based:
ρ
η
Motivation
3029/8/2019Ada Lovelace Workshop 2019
Georg Reuber
… n dimensions
• 5 rock phases in thermo-elasto-visco-plastic model = 40 parameters (=dimensions)• Seismology: wavespeed at every node is a free parameter = # nodes parameters (billions)
→ Every viscosity free parameter = # nodes parameters
η η+ρ η+ρ+Kη+ρ+K+…
nD-Inversion
Motivation
3129/8/2019Ada Lovelace Workshop 2019
Georg Reuber
… n dimensions
• Statistical: 40 parameters + 10 samples per parameter = 10^40 simulations• Deterministic: Two solves per parameter with FD = 80 simulations (per descent iteration)
η η+ρ η+ρ+Kη+ρ+K+…
nD-Inversion
Motivation
3229/8/2019Ada Lovelace Workshop 2019
Georg Reuber
… n dimensions
• Statistical: 40 parameters + 10 samples per parameter = 10^40 simulations• Deterministic: Two solves per parameter with FD = 80 simulations (per descent iteration)
→More efficient gradient computation available?
η η+ρ η+ρ+Kη+ρ+K+…
nD-Inversion
Motivation
3329/8/2019Ada Lovelace Workshop 2019
Georg Reuber
• Adjoint method:• Independent of # parameters (very efficient)• Requires forward (nonlinear) + adjoint solve (linear) and gradient computation→ Requires formulation of adjoint equation
… n dimensions
η η+ρ η+ρ+Kη+ρ+K+…
nD-Inversion
3429/8/2019Ada Lovelace Workshop 2019
Georg Reuber
Method
Constrained optimization:• Appears a lot:
• E.g. Economics: Maximize profit in terms of labor hours and raw material with limited budget
• Logistics• Thermodynamics• …
Inversion
Deterministic
3529/8/2019Ada Lovelace Workshop 2019
Georg Reuber
Method
Inversion
Deterministic
3629/8/2019Ada Lovelace Workshop 2019
Georg Reuber
Method
Inversion
Deterministic
3729/8/2019Ada Lovelace Workshop 2019
Georg Reuber
Method
Inversion
Deterministic
→ Find lowest point of Hike
3829/8/2019Ada Lovelace Workshop 2019
Georg Reuber
Method
Inversion
Deterministic
I) National Park (with Bears)
3929/8/2019Ada Lovelace Workshop 2019
Georg Reuber
Method
Inversion
Deterministic
I) National Park (with Bears)II) Foggy day
4029/8/2019Ada Lovelace Workshop 2019
Georg Reuber
Method
Inversion
Deterministic
I) National ParkII) Foggy day
4129/8/2019Ada Lovelace Workshop 2019
Georg Reuber
Method
Inversion
Deterministic
I) National ParkII) Foggy day
→ Find lowest point of Hike
4229/8/2019Ada Lovelace Workshop 2019
Georg Reuber
Method
Constrained optimization:• In our example:
1. Function represents mountain belt
2. Find lowest point of hike
Hiking trailValleys
Inversion
Deterministic
x
y
4329/8/2019Ada Lovelace Workshop 2019
Georg Reuber
Method
Constrained optimization:• Function F(x,y) - topography• Constraint g(x,y) – hike
F(x,y)
g(x,y)
Inversion
Deterministic
x
y
4429/8/2019Ada Lovelace Workshop 2019
Georg Reuber
Method
Constrained optimization:• Function F(x,y) - topography• Constraint g(x,y) – hike
F(x,y)
g(x,y)
Inversion
Deterministic
x
y
4529/8/2019Ada Lovelace Workshop 2019
Georg Reuber
Method
Constrained optimization:• Function F(x,y) - topography• Constraint g(x,y) – hike
F(x,y)
g(x,y)
Inversion
Deterministic
x
y
4629/8/2019Ada Lovelace Workshop 2019
Georg Reuber
Method
Constrained optimization:• Function F(x,y) - topography• Constraint g(x,y) – hike
F(x,y)
g(x,y)
Inversion
Deterministic
x
y
4729/8/2019Ada Lovelace Workshop 2019
Georg Reuber
Method
Constrained optimization:• Function F(x,y) - topography• Constraint g(x,y) – hike
F(x,y)
g(x,y)
Inversion
Deterministic
x
y
4829/8/2019Ada Lovelace Workshop 2019
Georg Reuber
Method
Constrained optimization:• Function F(x,y) - topography• Constraint g(x,y) – hike
→ Find common gradient along constraint
F(x,y)
g(x,y)
Inversion
Deterministic
x
y
4929/8/2019Ada Lovelace Workshop 2019
Georg Reuber
Method
Constrained optimization:
Inversion
Deterministic
F(x,y)
g(x,y)
x
y
→ Solve for x,y & λ→ x & y = coordinates of minimum→ [P(x,y) = lowest point→ λ = d(P)/d(Radius)]
5029/8/2019Ada Lovelace Workshop 2019
Georg Reuber
Adjoint method(PDE – constrained optimization)
I) Field inversion
II) Vector (p) inversion
Field
51
(1)
Objective function (no regularization):
(2)
(3)
(4)
(5)
(6)
29/8/2019Ada Lovelace Workshop 2019
Georg Reuber
Field
52
(1)
Objective function (no regularization):
(2)
Stokes (inversion constraint, linear, no advection):
(3)
(4)
(5)
(6)
29/8/2019Ada Lovelace Workshop 2019
Georg Reuber
Field
53
(1)
Objective function (no regularization):
(2)
Stokes (inversion constraint, linear, no advection):
(3)
(4)
Lagrangian (constrained optimization):
(5)
(6)
29/8/2019Ada Lovelace Workshop 2019
Georg Reuber
→ Find critical points of this function
Field
54
(1)
Objective function (no regularization):
(2)
Stokes (inversion constraint, linear, no advection):
(3)
(4)
Lagrangian (constraint optimization):
(5)
(6)
29/8/2019Ada Lovelace Workshop 2019
Georg Reuber
→ Find critical points of this function
I. FP multiplied by some Lagrange multipliers (shape functions) is the weak form (FEM) of the FP
II. FP linear – (often) self- adjointIII. If FP nonlinear – additional terms appear (e.g. viscosity
derivative) – like deriving Jacobian
IV. Gradient of parameter at every node – ‚field‘ inversionV. Independent of discretization
VI. Can be derived for any variable that occurs in the equation, e.g. principal stress directions:
Reuber et al. (in prep)
Vector
55
Approach implemented in LaMEM (equal result):
(1)
1) Objective function (no regularization):
(2)
(3)
(4)
5529/8/2019Ada Lovelace Workshop 2019
Georg Reuber
Vector
56
Approach implemented in LaMEM (equal result):
(1)
1) Objective function (no regularization):
(2)
2) Objective function derivative (final gradient sought):
(3)
(4)
First term = 0 (!= 0 if regularization)
5629/8/2019Ada Lovelace Workshop 2019
Georg Reuber
Vector
57
Approach implemented in LaMEM (equal result):
(1)
1) Objective function (no regularization):
(2)
2) Objective function derivative (final gradient sought):
(3)
2) Residual derivative (R = 0):
(4)
First term = 0 (!= 0 if regularization)
5729/8/2019Ada Lovelace Workshop 2019
Georg Reuber
Jacobian
58
3) Substitute:
(5)
(6)
(7)
(8)
Vector
5829/8/2019Ada Lovelace Workshop 2019
Georg Reuber
59
3) Substitute:
(5)
(6)
4) Final:
(7)
(8)
Vector
5929/8/2019Ada Lovelace Workshop 2019
Georg Reuber
60
3) Substitute:
(5)
(6)
4) Final:
(7)
(8)
Vector
6029/8/2019Ada Lovelace Workshop 2019
Georg Reuber
Note: Two options to compute gradients with the adjoint method:I) Converge forward problem – compute missing partial derivatives – reuse FP Jacobian - DtOII) Write two codes – one for FP and one for AP – combine solution to compute Gradient - OtD
6129/8/2019Ada Lovelace Workshop 2019
Georg Reuber
What are these gradients goodfor?
I) Obviously for gradient basedinversion
II) Sensitivity kernels in geodynamics –resolution proxy
III) Automatic derivation of scaling laws
Sensitivity
62
Tromp et al. (2005)
29/8/2019Ada Lovelace Workshop 2019
Georg Reuber
Sensitivity
63
Tromp et al. (2005)
29/8/2019Ada Lovelace Workshop 2019
Georg Reuber
Physical resolution test
Why not try in geodynamics as well?
→ What are the velocities at the surface most sensitive to?→ Compute gradient as usual but use:→ Sensitivity of solution to ρ + η at every node→ E.g. resolution test in seismology
Geodynamic sensitivity kernels
6429/8/2019Ada Lovelace Workshop 2019
Georg Reuber
Vx Vz
Sensitivity
6529/8/2019Ada Lovelace Workshop 2019
Georg Reuber
→ What are the velocities at the surface most sensitive to?→ Compute gradient as usual but use:→ Sensitivity of solution to ρ + η at every node→ E.g. resolution test in seismology
Sensitivity
6629/8/2019Ada Lovelace Workshop 2019
Georg Reuber
→ What are the velocities at the surface most sensitive to?→ Compute gradient as usual but use:→ Sensitivity of solution to ρ + η at every node→ E.g. resolution test in seismology
Sensitivity
6729/8/2019Ada Lovelace Workshop 2019
Georg Reuber
Timestep 1
Absolute sensitivity: Density)
Sensitivity
6829/8/2019Ada Lovelace Workshop 2019
Georg Reuber
Timestep 1
Timestep 50
Absolute sensitivity: Density)
Absolute sensitivity: Density)
Cheap! Can be done every timestep
6929/8/2019Ada Lovelace Workshop 2019
Georg Reuber
Automatic derivation of scalinglaws
Scaling law
70
• Nondimensional 2 layer Rayleigh-Taylor instability• Random perturbation at interface
Reuber et al., 2017
29/8/2019Ada Lovelace Workshop 2019
Georg Reuber
Scaling law
71
• Nondimensional 2 layer Rayleigh-Taylor instability• Random perturbation at interface
Reuber et al., 2017
29/8/2019Ada Lovelace Workshop 2019
Georg Reuber
No cost function (as before)
Scaling law
72
Reuber et al., 2017
29/8/2019Ada Lovelace Workshop 2019
Georg Reuber
General (multiplicative) scaling law
No cost function!
Scaling law
73
Reuber et al., 2017
29/8/2019Ada Lovelace Workshop 2019
Georg Reuber
General (multiplicative) scaling law
Exponent
Prefactor
No cost function!
Scaling law
74
Reuber et al., 2017
29/8/2019Ada Lovelace Workshop 2019
Georg Reuber
Scaling law
75
→ Exact reproduction of analytical solution
29/8/2019Ada Lovelace Workshop 2019
Georg Reuber
Scaling law
76
Several ‚domes‘ evolve
Reuber et al., 2017
29/8/2019Ada Lovelace Workshop 2019
Georg Reuber
Scaling law
77
Several ‚domes‘ evolve
Reuber et al., 2017
29/8/2019Ada Lovelace Workshop 2019
Georg Reuber
Conclusions
7829/8/2019Ada Lovelace Workshop 2019
Georg Reuber
• Adjoint method very efficient:• independent of number of parameters (‚field inversion‘)• Can be derived for any parameter for any PDE
Conclusions
7929/8/2019Ada Lovelace Workshop 2019
Georg Reuber
• Adjoint method very efficient:• independent of number of parameters (‚field inversion‘)• Can be derived for any parameter for any PDE
• Statistic methods:• Full knowledge (topology) of cost function• (Almost) No additional implementation required• Numerical cost scales with number of parameters
Conclusions
8029/8/2019Ada Lovelace Workshop 2019
Georg Reuber
• Adjoint method very efficient:• independent of number of parameters (‚field inversion‘)• Can be derived for any parameter for any PDE
• Statistic methods:• Full knowledge (topology) of cost function• (Almost) No additional implementation required• Numerical cost scales with number of parameters
• Deterministic methods:• Gradient information
→ Reveals sensitivity of e.g. velocity on density – resolution test→ Scaling law computable→ Time integration→ Hessian – link to statistic methods