Analytic Methods

Bjorn Engquist

University of Texas at Austin

Tutorial on analytic multi-scale techniques in the IPAM program: Bridging Time and Length Scales in

Materials Science and Bio-Physics, September 15-16

Outline

1. Overview

1.1 Multi-scale models

1.2 Computational complexity

1.3 Analytic model reduction

2. Classical analytic techniques

2.1 singular perturbation

2.2 stiff dynamical systems

2.3 homogenization

2.4 geometrical optics

3. Analytically based numerical methods

3.1 numerical model reduction

3.2 heterogeneous multi-scale methods

1. Overview

We are interested in analyzing and simplifying mathematical models of multi-scale processes. A goal is to reduce a model with a broad range of scales to one with a narrow range. Such simpler models or effective equations are typically easier to analyze and better as basis for numerical simulation. It is naturally of interest to study the difference in the solution of the original multi-scale problem and the reduced model with fewer scales.

The derivation of effective equations is an important component in most theoretical sciences. Physics is a classical example. We will focus on techniques developed within applied mathematics.

1.1 Multi-scale models

It is often natural to define scales in a physical process.

Examples of space scales are the size of the airplane, the turbulent eddies and the distance between the atoms. The time sales vary from the time of flight to the vibrations of the electrons. Different models are typically derived independently for the different scales. The derivation could be based on (1) physical principles, (2) mathematical derivation or (3) identification in a predetermined model.

Time (s)

Space (m)

Quantum mechanics“Schrödinger”

Molecular dynamics“Newton’s equations”

Kinetic theory“Boltzmann”

Continuum theory“Navier-Stokes”

1 Å 1

1

10-15

The scales could be given by the geometry or in the differential equations. In the Navier-Stokes equation the Reynolds number guides the nonlinear generation of a wide spectrum of scales. The Maxwell’s equation is the same on atomistic and galactic scales and multi-scale effects comes from boundary conditions.

In our mathematical analysis we will define the scales more explicitly, for example, by a scaling law.

The function f(x) = f(x,x/), where f(x,y) is 1-periodic in y, or where f(x,y) F(x), x>0, y>0, as y , are said to contain the scales 1 and .

The scales are also naturally described by a scale-based transform of a function as, for example, Fourier or wavelet transforms.

Let us formally write the original multi-scale differential equation as,

where F represents the differential equations with initial and boundary conditions. We are interested in finding and such that,

and to study the properties of the limiting process. The topology for the weak or strong convergence will be different for different cases.

The computational cost for directly numerically approximating is often prohibitingly high.

€

Fε (uε ) = 0

€

limε →0

uε = u

F (u ) = 0€

F

€

u

€

uε

Computational strategies

€

Fε (uε ) = 0

€

F (u ) = 0

€

F Δ (u Δ ) = 0 €

Fε ,Δ (uε ,Δ ) = 0

€

˜ F ε ,Δ ( ˜ u ε ,Δ ) = 0

1.2 Computational complexity

• A major reason for deriving effective equations with a narrow range of scales is the high computational cost of directly solving multi-scale problems.

• With the size of the computational domain = 1 in each dimention and the smallest wavelength= the typical number of operations in the solution of a multi-scale differential equation in d dimensions is,

€

flop = O((N(ε)ε−1)dr )

N(): number of unknowns per wavelength to achieve a given accuracy (N()2 from Shannon sampling theorem,

N() O(-1/2) for standard second order finite difference methods).

: the shortest wavelength to be approximated

d: number of dimensions

r: exponent for number of flops per unknown in the numerical method (r=1 for explicit methods and r=3 for Gaussian elimination of dense matrices)

If r=1 and N() is bounded by a constant we have,

€

flop = O((N(ε)ε−1)dr )

€

flop = O(ε−d )

Even with the best numerical methods , and this prohibits numerical simulation based on direct atomistic models over typical system sizes.

The upper limit for a teraflop computer is thus practically =10-4 with 10000 degrees of freedom in each dimension, R3+1.

New approximate effective equations must be derived or the computation must be reduced to a small sample of the original domain.

€

flop = O(ε−d )

1.3 Analytic model reduction

• These techniques are often found in the physics, mechanics or in the classical applied mathematics literature.

• They are commonly seen as part of the applied science rather than “just” a mathematical technique.

• They have been very useful for understanding multi-scale problems and for deriving effective equations but many have no rigorous mathematical justification.

• Many times the different models for different ranges of scales are derived independently and the connection between the models developed later.

• The purpose of the reduction may be for computational purposes or for easier analysis.

Examples of analytic techniques

Applied mathematics and mechanics related techniques

• Singular perturbations ()• Stiff dynamical systems ()• Homogenization methods ()• Geometrical optics and geometrical theory of diffraction ()• Boundary layer theory ()

Examples from theoretical physics

• Renormalization group methods• Semi classical representation, path integral techniques, Wigner

distributions• Density function theory

Examples of results from multi-scale approximations

• Born-Oppenheimer approximation• Cauchy-Born rule

Analytically based numerical methods

• Multigrid• Fast multipole method• Car-Parinello method• Quasi-continuum method• Heterogeneous multi-scale method ()

2. Classical analytic techniques

We discussed briefly a number of mathematical techniques for deriving effective equations in 1.3. We will consider four of those in more detail and they are chosen to give representative examples of a variety of analytic techniques.

2.1 Singular perturbations of differential equations

2.2 Stiff ordinary differential equations

2.3 Homogenization of elliptic differential equations

2.4 Geometrical optics

2.1 Singular perturbations

We will consider examples where the the micro scales are localized. The goal is the derivation of the limiting effective equations and the study of the limiting process.

€

− d2uε

dx 2+ a

duε

dx+ buε = f (x) 0 < x <1

uε (0) = uL , uε (1) = uR

ε > 0, a > 0, b > 0

The formal limit of this differential equation is of first order and only requires one boundary condition. In this case we can solve the original problem to see which boundary condition should be kept

The inhomogeneous part of the solution uih is smooth as 0. The homogenous part uh matches the boundary conditions resulting from uih with z1 and z2 the roots of the characteristic equation,

€

uε = uih + uh

uih (x) = exp(−b /a(x −ξ ) f (ξ )dξ + O(ε)0

x

∫

uh (x) = A1 exp(z1x) + A2 exp(z2x)

€

−z2 + az + b = 0

z1 = a /(2ε) + ((a /(2ε))2 − b), z2 = a /(2ε) − ((a /(2ε))2 − b),

Recall the form of the homogeneous part,

The coefficients A1 and A2 are determined to match the boundary conditions

Thus uih is exponentially small in away from a boundary layer close to x=1.

€

uh (x) = A1 exp(z1x) + A2 exp(z2x)

z1 = a /ε + O(ε), z2 = ε /a + O(ε 3)

€

A1 + A2 = −uih (0)

A1 exp(z1) + A2 exp(z2) = −uih (1)

A1 ≈ −uih (1)exp(−z1), A2 = −A1

The effective equation is

and u converges to u point wise in any domain 0xr<1, with the error O().

The inner solution and the boundary layer solution can be matched together to form an approximation for the full interval. This type of approximation goes under the name of matched asymptotics. One such example is the tipple deck method in fluid mechanics. Three approximating layers are matched.

€

adu

dx+ bu = f (x), 0 < x <1

u(0) = uL

€

∂u

∂t+ u

∂u

∂x+ v

∂u

∂y+

∂p

∂x=

1

R(∂ 2u

∂x 2+

∂ 2u

∂y 2)

∂v

∂t+ u

∂v

∂x+ v

∂v

∂y+

∂p

∂y=

1

R(∂ 2v

∂x 2+

∂ 2v

∂y 2)

∂u

∂x+

∂u

∂y= 0, y > 0, − ∞ < x < ∞, t > 0,

u(x,y,0), v(x, y,0) given initial values,

u = v = 0, y = 0, − ∞ < x < ∞,

Prandtl boundary layer equations

One classical example of an effective boundary layer equation is the Prandtl equation as a limit of high Reynolds number Navier-Stokes

equations,

The Prandtl assumption is that the inertia terms are balanced by the viscous terms in the a boundary layer of thickness (0<y< ). Rescaling the independent variables y/ and using the divergence free condition,

implies the scaling u=O(1), v=O(). Following the tradition we will use y for the new variable and study the scaling of the terms in the original equations.

€

∂∂y

→δ−1 ∂

∂η,

∂u

∂x+

∂v

∂y= 0

Balancing inertia and viscous terms implies R=O(-2).

€

∂u

∂t+ u

∂u

∂x+ v

∂u

∂y+

∂p

∂x=

1

R(∂ 2u

∂x 2+

∂ 2u

∂y 2)

1 1 δ δ−1 1 1 δ−2

€

∂v

∂t+ u

∂v

∂x+ v

∂v

∂y+

∂p

∂y=

1

R(∂ 2v

∂x 2+

∂ 2v

∂y 2)

δ δ δ δ−1 δ 2 δ 2 1

Leading orders of in the second equation gives ,

We then get the Prandtl boundary layer equation from the first equation, €

∂p

∂y= 0 ⇒ p = P(x)

€

∂u

∂t+ u

∂u

∂x+ v

∂u

∂y+ Px =

∂ 2u

∂y 2

v = −∂u

∂xdς ,

0

y

∫

u(x, y,0) given initial values

u(x,0, t) = 0, u(x,1, t) = U(x, t)

• This effective equation does not contain the small parameter 1/R. It has been used for analysis and numerical simulations. There is no rigorous derivation as the limit of the Navier-Stokes equations.

• There is a well established existence and uniqueness theory for the case that uy > 0 initially.

• Other well known effective equations for the high Reynolds number limit are the various turbulence models.

2.2 Stiff dynamical systems

Analysis of certain types of stiff dynamical systems resembles that of singular perturbations above. A system of ordinary differential equations is said to be stiff if the eigenvalues of the matrix A below are of strongly different magnitude or if the magnitude of the eigenvalues are large compared to the length of interval of the independent variable,

€

du

dt= Au + f (t), t > 0, u : R1 → Rd

u(0) = u0

The following nonlinear system is stiff for 0 < << 1,

If the conditions below are valid it has resemblance to the singular perturbation case,

€

duε

dt= f (uε ,vε ),

dvε

dt= ε−1g(uε ,vε ), t > 0

uε (0) = u0, vε (0) = v0

€

Re(σ (∂g

∂vε

)) ≤ λ < 0, det(∂g

∂uε

) ≠ 0

From

We have the differential algebraic equations (DAE),

The original functions have an exponential transient of order O(1) right after t=0 before converging to (u,v). The reduced system represents the slow manifold of the solutions of the original system.

Compare the Born-Oppenheimer approximation and the Car-Parinello method.

€

duε

dt= f (uε ,vε ),

dvε

dt= ε−1g(uε ,vε )

€

uε (t) → u(t), vε (t) → v(t), t ≥ t > 0, as ε → 0,

∂u

∂t= f (u,v), u(0) = u0

g(u,v) = 0, defines v

Oscillatory solutions

• A typical example is finite temperature molecular dynamics.• For nonlinear problems simple averaging does not work

<f(u)> f(<x>).• Averaging must take resonance into account. This is possible

for polynomial f.• KAM-theory analyses effect of perturbations.• Simple oscillatory example,

€

duε

dt= ε−1vε ,

dvε

dt= −ε−1uε ,

dwε

dt= uε

2 + vε2,

uε (0) = 0, vε (0) =1, wε (0) = 0

⇒ uε (t) = sin(t /ε), vε (t) = cos(t /ε), wε (t) = w(t) = t

2.3 HomogenizationHomogenization is an analytic technique that applies to a wide class of multi-scale differential equations. It is used for analysis and for derivation of effective equations.

Let us start with the example of a simple two-point boundary value problem where a may represent a particular property in a composite material.

Any high frequencies in a interact with those in to create low frequencies.

€

d

dx(aε (x)

duε

dx) = f (x), 0 < x <1,

uε (0) = uε (1) = 0

aε (x) = a(x /ε) > 0

€

duε

dx

If we assume a(y) to be 1-periodic then a(x/) is highly oscillatory with wave length . The oscillations in a will create oscillations in the solution

u. The oscillations in a and u interact to create low frequencies from these high frequencies. The effective equations can not simply be derived by taking the arithmetic average of a.

This example can be analyzed by explicitly deriving the solution. After integration of the differential equation we have

€

aε (x)duε

dx= f (

0

x

∫ ξ )dξ + C

uε (x) = (0

x

∫ aε (ξ )−1( f (η )dη0

ξ

∫ + C)dξ

The constant C is determined by the boundary conditions,

In this explicit form of the solution it is possible to take the limit as 0. €

0 = (0

1

∫ aε (ξ )−1( f (η )dη0

ξ

∫ + C))dξ

C = − (0

1

∫ aε (ξ )−1 f (η )dη0

ξ

∫ )dξ / (0

1

∫ aε (ξ )−1ξdξ

€

limε →0

(0

x

∫ a(ξ /ε)−1F(ξ )dξ = a(y)dy F(ξ )dξ , F ∈ C[0,1]0

x

∫0

1

∫

The limit solution is thus,

where A is the harmonic average. Differentiations yield the effective or homogenized equation,€

uε (x) → u (x) = A−1 ( f (η )dη + C)dξ , as ε → 0,0

ξ

∫0

x

∫

A = ( a(y)−1dy0

1

∫ )−1

€

Ad2u

dx 2= f (x), 0 < x < 0,

u (0) = u (1) = 0.

Elliptic homogenization problems

For most problems there is no closed form solution and the procedure in our simple example cannot be followed.

The typical approach in to assume an expansion of the solution in terms of a small parameter , insert the expansion into the differential equation and then to find some closure process to achieve the convergence result and the effective equation.

Assume the matrix a(x,y) to be positive definite and 1-periodic in y, The function a0(x,y) is also assumed to be positive and 1-periodic in y. The asymptotic assumption on u is as follows,

€

−∇⋅(a(x,x /ε)∇uε (x)) + a0(x,x /ε)uε (x) = f (x), x ∈ Ω

uε (x) = 0, x ∈ ∂Ω boundary of Ω ⊂Rd

€

uε (x) = u0(x, x /ε) + εu1(x, x /ε) + ε 2u2(x, x /ε) + ...

u j (x, y), 1− periodic in y, j =1,2,...

Introduce the variable y=x/ and equate the different orders of . The equation for the -2 terms is

with periodic boundary conditions in y. This implies

The equation for the -1 terms gives a representation of u1 in terms of u. The terms of order O(1), O(), etc. couple the unknown terms in the expansion of u but the closure assumption that u2(x,y) is 1-periodic in y generates the effective equation as conditions on u for existence of u2.

€

−∇ya(x, y)∇ yu0(x, y) = 0

€

u0(x,y) = u(x)

The effective or homogenized equations take the form,

The function is a solution of the cell problem,

€

−∇A(x)∇u(x) + a (x)u(x) = f (x)

A : aij (x) =1

T(aij

T

∫ (x, y) + aik

∂κ j (x,y)

∂yk

)dy

a (x) =1

Ta0

T

∫ (x, y)dy

€

−(∂

∂y i

aik (x,y))∂κ j

∂yk

) = −∂aij (x, y)

∂y i

κ periodic bc in y

General homogenizations

• The same technique also applies to many hyperbolic and parabolic equations and can, for example, be used to derive the Darcy law from the Stokes equations..

• By homogenizing scale by scale several different scales can be handled ( a=a(x,x/1,x/2,…), 10, 2/ 10,…)

• The assumption of periodicity can be replaced by stochastic dependence.

• Compensated compactness and the theories for -, G-, and H-convergence are powerful non-constructive analytic technique for analyzing the limit process.

2.4 Geometrical optics

Geometrical optics equations are effective equations for high frequency wave propagation. Instead of directly approximating highly oscillatory functions geometrical optics gives the phase (x,t) and amplitude A(x,t).

In this case the effective formulation were known long before the wave equation form.

Note that new variables are introduced different from the strong or weak limit of the original dependent variables.

Scalar wave equation

The velocity is denoted by c and the initial values are assumed to be highly oscillatory such that the following form is appropriate,

€

∂u2(x.t)

∂t 2= c(x)2Δu(x, t)

u(x,0) = u0(x),∂u(x,0)

∂t= u1(x)

€

u(x, t) = exp(iωϕ (x, t)) A j

ω= 0

∞

∑ (x, t)ω− j , ω >>1

Insert the expansion into the wave equation and equate the different orders of (=-1). The leading equations give the eikonal and transport equations that do not contain ,

The traditional ray tracing can be seen as the method of characteristics applied to the eikonal equation,€

∂∂t

+ c(x)∇ϕ = 0, ( ⋅ = Euclidean norm)

∂A0

∂t+ c(x)

∇ϕ ⋅∇A0

∇ϕ+

c(x)2Δϕ −∂ 2ϕ

∂t 2

2c(x)∇ϕA0 = 0

€

dx

dt=∇ pH(x, p),

dp

dt=∇ xH(x.p)

H(x.p) = c(x) p

Generalizations

• The analysis discussion above fails at boundaries. The geometrical theory of diffraction (GTD) adds correction terms for diffraction at corners and introduces the presence of creeping waves of the shadow zone.

• The approach extends to frequency domain formulations and other differential equations, for example, linear elasticity and Maxwell’s equations.

• Compare WKB and the Schrödinger equation.• The nonlinear eikonal equation is of Hamilton Jacoby type and

follows the viscosity solution theory.• If c(x) = c(x,x/) is oscillatory, homogenization ( << -1),

geometrical optics ( >> -1) or special expansions ( -1) apply.

3. Analytically based numerical methods

• These techniques are used when appropriate effective equations cannot be derived by analytical or physical methods.

• Fast methods resolving all scales (complexity O(-d))– High order methods reducing number of unknowns– Traditional multi -scale methods: multi-grid, fast multi-pole

(using special features in operator)• Numerical model reduction methods starting with all scales

resolved ()– Multi-scale finite element methods (MSFEM)– Wavelet based model reduction

• Fast methods not resolving all scales (using special features in solution, i.e. scale separation) ()

3.1 Numerical model reduction

We will briefly consider two classes of methodologies:

• Standard model reduction of input-output systems as in control theory

• Model reduction using compression and special basis functions

Remark. The computational cost of using these methods is at least as large as the solution of the original full system. The gain comes from the potential of using the same reduced system for a large set of inputs.

Standard model reduction

Consider the input-output system

The matrix A may be the result of a spatial discretization and the dimension n is assumed to be much larger than m and p.

Transient and filtered modes are eliminated to produce an approximation with lower dimensional A. SVD of A is a possible technique. Different methods are found in the control literature.

€

dx(t)

dt= Ax(t) + Bu(t), x ∈ Rn, u∈ Rm

y(t) = Cx(t) + Du(t), y ∈ R p

Special basis functions

We will briefly mention two examples: the multi-scale finite element method (MSFEM) and wavelet based homogenization.

In MSFEM the basis functions that are used in the finite element method are chosen to satisfy the homogeneous form of the original multi-scale problem, [Hou].

The wavelets in wavelet based homogenization are used to keep the reduced operators sparse during the computation. A discretized differential equation equation in a wavelet basis is reduced by Schur complement [E., Runborg]

€

A11 A12

A21 A22

⎛

⎝ ⎜

⎞

⎠ ⎟uH

uL

⎛

⎝ ⎜

⎞

⎠ ⎟=

f H

fL

⎛

⎝ ⎜

⎞

⎠ ⎟ → (A22 − A21A11

−1A12)uL = f l − A21A11−1 f H

3.2 Heterogeneous multi-scale methods

The heterogeneous multi-scale method (HMM) is a framework for developing and analyzing computational multi-scale models. A macro-scale method is coupled to a micro-scale method. The micro-scale technique is only applied in part of the computational domain and it is used to supply missing information to the macro-scale algorithm.

The coupling is based on related theory for analysis of effective equations. The gain in efficiency over applying the micro-scale method everywhere is the restricted use of the computational expensive micro-scale technique.

Compare the “quasi continuum” and the “equation free” methods.

The HMM framework

• Design macro-scale scheme for the desired variables. The scheme may not be valid in all of the computational domain (type A) or components of the scheme may not be known in full domain (type B).

• Use micro-scale numerical simulations to supply missing data in macro-scale model

Remarks (1) Macro-scale model may, for example, need boundary data locally (type A) or data for constitutive laws all over computational domain (type B)

(2) Macro- and micro-scale variables can be in different spaces and be governed by different equations

(3) HMM theory as design principle

• Type A: Macro-scale model is accurate enough in most of computational domain 1. Microscale model used in the complement 2. Compare singular perturbation.

• Type B: A Macro-scale model is not fully known throughout computational domain. Compare homogenization.

1

2

Example: homogenization of elliptic equation (type B),

Assume there exists a homogenized equation (not known),

Approximate U, for example by p1-functions on a coarse grid and apply Galerkin with numerical quadrature. Evaluate the stiffness matrix using the micro-scale equation above at a small domain around the quadrature points.

€

−∇⋅(aε (x)∇uε (x)) = f (x), x ∈ D⊂Rd

uε (x) = 0, x ∈ ∂D, a1 ≥ aε ≥ a0 > 0

€

−∇⋅(A(x)∇U(x)) = f (x), x ∈ D⊂Rd

U(x) = 0, x ∈ ∂D, A ≥ A > 0

uε →U, ε → 0

v and w satisfy the original differential operator with boundary conditions matching .

€

(∇Φ j ⋅A∇Φ k )(xl ) ≈1

δ d∇vl

ε

Iδ (xl )

∫ (x)⋅a ε(x)∇wlε (x)dx

Convergence results

• FVM for hyperbolic and parabolic equations and FEM for elliptic equations when applied to standard linear homogenization problems.

• FVM approximating the diffusion equation as limit of Brownian motion.

• FDM for selected dynamical systems. Both dissipative and oscillatory solutions are analyzed but only those with limited resonance.

€

U0 −UHMM ≤ C(H p + e(HMM)),

Examples of HMM simulations

• Fluid simulations: [E, Ren], contact line on multiphase fluid solid interaction. Type A example: Continuum model valid but for contact line where MD is applied.

• Solid simulation: [E, Li], thermal expansion.Type B example: Micro-scale MD model needed in full domain of elasticity continuum model.

• Combustion fronts: [Sun, E], micro-scale simulation with chemistry to evaluate macro scale properties at front.

• Stiff dynamical systems: [Sharp, E, Tsai]. Intervals with short time steps to evaluate the effective force for macro time steps.

QuickTime™ and aTIFF (LZW) decompressor

are needed to see this picture.

QuickTime™ and aTIFF (LZW) decompressor

are needed to see this picture.

QuickTime™ and aTIFF (LZW) decompressor

are needed to see this picture.QuickTime™ and aTIFF (LZW) decompressor

are needed to see this picture.

References

Arnold, V., Mathematical methods of classical mechanics, Springer-Verlag, 1980.

Arnold, V., Ordinary Differential Equations, Springer-Verlag, 1992

Bensoussan, A.,J.-L. Lions, G. Papanicolaou, Asymptotic analysis of periodic structures, Noth-Holland, 1978.

E, E., B. Engquist, Multicale modeling and computation, AMS Notices, 2003.

Verhulst, F., Methods and Applications of Singular Perturbations, Springer-Verlag, 2005.