an efficient algorithm for hamilton–jacobi equations in high dimension

7/23/2019 An efficient algorithm for HamiltonJacobi equations in high dimension

1/15

Digital Object Identifier (DOI) 10.1007/s00791-004-0124-5

Comput Visual Sci 7: 1529 (2004) Computing andVisualization inScience

Regular article

An efficient algorithm for HamiltonJacobi equations in high dimension

Elisabetta Carlini1, Maurizio Falcone2, Roberto Ferretti3

1 Dipartimento di Metodi e Modelli Matematici, Universit di Roma La Sapienza, via Scarpa, 16, 00161 Roma, Italy(e-mail: [email protected])

2 Dipartimento di Matematica, Universit di Roma La Sapienza, P. Aldo Moro 2, 00185 Roma, Italy(e-mail: [email protected])

3 Dipartimento di Matematica, Universit di Roma Tre, L.go S. L. Murialdo, 1, 00146 Roma, Italy(e-mail: [email protected])

Received: 13 December 2002 / Accepted: 16 July 2003Published online: 7 April 2004 Springer-Verlag 2004

Communicated by: K. Mikula

Abstract.In this paper we develop a new version of the semi-Lagrangian algorithm for first order HamiltonJacobi equa-tions. This implementation is well suited to deal with prob-lems in high dimension, i.e. in Rm with m 3, which typ-ically arise in the study of control problems and differentialgames. Our model problem is the evolutive HamiltonJacobiequation related to the optimal control finite horizon problem.We will give a step-by-step description of the algorithm fo-cusing our attention on two critical routines: the interpolationin high dimension and the search for the global minimum.We present some numerical results on test problems whichrange from m = 3 tom = 5 and deal with applications to frontpropagation, aerospace engineering, ecomomy and biology.

1 Introduction

In this paper we report on some recent developments in theconstruction of semi-Lagrangian schemes for the solution ofHamiltonJacobi equations. As it is well known, these equa-tions arise in the solution of optimal control problems anddifferential games. The value function is typically character-ized as the unique viscosity solution of the corresponding Dy-namic Programming equation, the so-called Bellman equa-tion for control problems and Isaacs equation for differential

games. Although the theory of viscosity solutions and the an-alysis of approximation schemes are usually developed inRm ,from a practical point of view there are severe limitationsto the implementation of the schemes for m 3. This is thedrawback of Dynamic Programming with respect to the open-loop approach based on the Pontryagin Maximum Principle.The latter in fact requires only the solution of a TPBVP (Two

This work has been supported by funds from M.U.R.S.T. (projectMetodologie Numeriche Avanzate per il Calcolo Scientifico) and fromthe TMR Network Viscosity solutions and applications (contract FMRX-CT98-0234). We gratefully acknowledge the technical support given byCASPUR to the development of this research.

Point Boundary Value Problem) for a system of ordinary dif-ferential equations. It must be noted that the TPBVP givesonly a partial characterization of the optimal trajectory andof the optimal open-loop control since its equations are onlynecessary conditions of optimality. On the other hand, pro-vided the value functionv could be computed, the DynamicProgramming approach would give a full description of theproblem since the knowledge ofvallows to compute also ap-proximate feedback optimal controls (cfr. [2] and referencestherein). This is the main motivation to develop efficient algo-rithms in high dimension. What is a feasible program takinginto account the limitations of modern workstations? Manyoptimal control problems coming from mechanical engineer-

ing, robotics and finance which involve a state space withhundreds of variables seem to be far beyond the range ofcomputation. Nonetheless, it is often possible to reduce themto a smaller number of significant new variables. Our goalis to deal with problems with about ten state variables. Inthis respect, it is very important to note that we do not havereal limitations on the nonlinearities appearing in the dynam-ics. In fact, the theory of viscosity solutions allows to dealwith nonlinear controlled vectorfields under very mild as-sumptions, i.e. we assume that the vectorfield is Lipschitzcontinous and locally bounded. It is worth to note that for dif-ferential games the need for high-dimensional algorithms iseven stronger since the value function of a game set in Rm de-

pends on 2m variables, i.e. the Isaacs equation is set inR

2m

(see [2, 26]).There are, of course, several ingredients which can be

used to solve a problem with many state variables. The firstone, as we mentioned, is state space reduction, whereas an-other possibility is to develop parallel algorithms based on thedomain decomposition approach. Parallel algorithms basedon domain decomposition for first-order HamiltonJacobiequations have been introduced and analysed in [7, 16, 17].In any case, at the very end one should be able to deal witha problem in Rm, m= 3, . . . , 10, in every computing node(or workstation if the computation is distributed on a network,e.g. using MPI). This is our goal here. The coupling between


2/15

16 E. Carlini et al.

all the above ingredients will depend on the specific applica-tion and is beyond the scope of this paper.

Although the contribution of this paper is essentially atthe algorithmic level it is important to give some back-ground on the theoretical results which support these newdevelopments. The serial algorithm which is the basis forthis high-dimensional version was introduced in [14] (seealso [8, 9] for the stationary version). A series of conver-gence results for continuous as well as discontinuous value

functions have been developed in the framework of viscos-ity solutions. The interested reader can look at [13] and atthe survey papers [4, 10] for the description of the main re-sults and for a more complete list of references includingalso different approximation schemes (e.g. finite differences).A first order semi-Lagrangian scheme on unstructured gridshas been implemented in [33]. High-order extensions of theabove semi-lagrangian schemes have been obtained and ana-lysed in [1113] and [20]. In order to set the paper intoperspective we would also like to mention other techniquesthat have been adopted to compute the solution of HamiltonJacobi equations up to dimension 3. Finite differences areprobably the most popular method for such equations. In the

pioneering paper [29] one can find a finite difference schemefor the mean curvature flow of an interface. That is a sim-ple example of what can be obtained by the level-set methodwhich has been used to solve problems from a variety of ap-plications as shown in [35] and [30]. Recent developmentsinclude the so-called fast marching method which is aneffective method to speed-up the algorithm by means of a lo-calization of the computation in a narrow band centered atthe front (cfr. [36, 37]), the analysis and implementation ofadaptive grid algorithms (e.g. in [22, 24, 25] and [33]).The outline of the paper is as follows. Section 2 is devoted toset up the model problem and to recall some basic facts aboutits semi-Lagrangian approximation. In Sect. 5 we describestep-by-step the high-dimensional algorithm and its routines,

including the minimization procedure. Sections 3 and 4 arerespectively devoted to the interpolation and to the minimiza-tion routines. Finally, Sect. 6 presents some numerical resultson test problems coming from various fields of applications.

2 The model problem and its approximation

Let us consider the following evolutive HamiltonJacobiequation

vt(x, t) +H(x, t, v, v(x, t)) = 0 , (1)

with(x, t) Rm [0, T], coupled with the initial condition

v(x, 0) = v0(x), for x Rm . (2)

We choose the Hamiltonian Has

H(x, t, v(x, t)) = v(x, t)

+ supaA

[v(x, t) b(x, t, a) g(x, t, a)].

(3)

The solution of (1), (2) is the value function of a finitehorizon optimal control problem (see e.g. [2] and [5] for de-tails) so that it can be represented as

v(x, t) = infA

t

0

g(y(), t ,(t )) d+ v0(y(t))

(4)where x Rm , t [0, T], A { : [0, +[A , measur-able}, A is a possibly compact subset ofRl and y() is thesolution trajectory of the following (controlled) system of or-dinary differential equation :

y() = b(y(), t ,(t )), [0, t]y(0) =x.

(5)

In order to obtain a numerical scheme we can first con-sider the evolution betweent andt+tand perform a timediscretization using the rectangle rule for the integral in (4)and the Euler scheme in (5). This leads to the following time-discrete version of (4):

vt(x, t+t) = minaA

t g(x, t, a)

+ vt(x+tb(x, t, a), t)

. (6)

The typical SL-scheme is obtained projecting (6) on a grid.Working for the moment in one space dimension, introducingthe standard notations

xi= ix, tn= nt, (7)

and further replacingvt(x, nt)in (6) with a numerical re-construction vn (x) based on the values vni = v

n(xi ), we canwrite the first order scheme in R (see [14]) as

vn+1i = minaA

[tg(xi, nt, a)

+ vn(xi +tb(xi, tn, a))]

v0i = v0(xi ) .

(8)

A class of high-order semi-Lagrangian schemes can beobtained combining a high-order method for the Cauchyproblem (5) with a more accurate quadrature formula for theintegral and with a high-order reconstruction forvn. The gen-eral form of such schemes in R is

vn+1

i = min(a1,...,ap)A

p

pi=1

ti g(xi , nt, ai )

+ vn(xi + h(xi , t, a1, . . . , ap))

v0i = v0(xi )

where xi = (i ,x, a1, . . . , ap) are intermediate points usedin the quadrature formula, thei are its weights and is theHenrici function of the method used for the Cauchy problem(see [11, 13]).

Turning to the case of dimension m , we give the schemein the whole space assuming Rm so that we can avoid(for the moment) boundary conditions. Let us define the lat-tice L(x1, . . . , xm ,t) by L {(x

p, tn) Rm [0, T] :

xp (p1x1,p2x2, . . . , pmxm), for pi Z} where xi ,i= 1, . . . , m are the space steps, and again we will use thestandard notations tn= nt, v

np to denote the numerical ap-

proximation at (xp, tn) (p Zm, n N) and vn (x) to denote


3/15

An efficient algorithm for HamiltonJacobi equations in high dimension 17

its extension to all x Rm. Using first-order time approxima-tions as in (8), we obtain

vn+1p = minaA

vnxp +t

xp, tn , a

+tg(xp, tn, a)

. (9)

In general, the analysis of high-order versions of (9), ob-tained by a discretization of order q in time andr in space,yields the consistency error (see [1113]):

Lx,t(x) C

tq +

xr+1

t

. (10)

Also, convergence of the scheme can be proved under suitableassumptions on the relationship between the discretizationsteps (see [20]).

In this framework, we may wish to estimate the increasein efficiency due to high-order implementations. To this endwe can take, as a rough measure of complexity, the number of nodes in the space-time grid, that is

1

xmt

1

xm+ ,

where we have again denoted by m the dimension of thephysical space, and assumed that t x, with > 0. Onthe other hand, assuming that the consistency estimate (10)could also work as an error estimate for the solution (this isconfirmed by numerical experiments), we also have for theerror:

xq +xr+1 xmin(q,r+1)

so that at last, expressing the complexity with respect to therequired error, we obtain

m+

min(q,r+1) .

For example, if we work at constant Courant number (sincethe Courant number is t

x, this means = 1), we get

m+1

min(q,r)

which shows that the improvement in efficiency due to thehigh-order implementation is even greater in higher dimen-sion.

Note that the SL-scheme (9) has the same structure of

the representation formula of the exact solution written forv0= vn andt= t. Nonetheless, there are two crucial steps

in this algorithm. First we have to compute the value ofv onthe right-hand side by an interpolation procedure based onthe values at the nodes of the lattice L becausez p(a) xp +tb(xp, tn, a)in general is not a grid point. This must be donefor every a A if A is a finite set. The second crucial stepis the minimization on A which cannot be done by simplecomparison ifAis not finite.

The interpolation procedure is standard as far as the di-mension of the problem and the degree of the polynomial aresmall. It becomes more difficult in high dimension becauseone has to construct a recursive procedure which computes

the value performingm one-dimensional interpolations. Notethat, in principle, any type of procedure to recover the valueofvat the foot of characteristics by the values on the nodes ofthe lattice can be used. Of course, locating the foot of charac-teristics requires some additional work with respect to finitedifferences schemes but this step is not expensive on struc-tured grids when one simply divides the coordinatez

pi (a)by

the space discretization step xi in order to determine thecell containing the foot. We will always work in rectangu-

lar domains with structured grids to simplify the problem.However, the scheme does not really require structured gridsand it can be implemented also on triangular grids where anadditional cost is required in order to determine the trianglecontaining the foot of the characteristic (such a scheme hasbeen introduced and implemented in [33]).As we mentioned, the second difficulty is minimization. Tothis end one can adopt several strategies: direct search on a fi-nite number of points in A, penalization methods, stochasticminimization techniques. The first solution is less expensivebut introduces an additional error because it leads to a dis-cretization of the control set. The choice of a stochastic al-gorithm for the global minimization is too expensive since

in the SL-scheme this step must be repeated at every gridpoint. We have used (see Sect. 3) a deterministic Powell-typealgorithm.

3 High-dimensional interpolation

The purpose of this section is to describe an algorithm tointerpolate a function f : D R, D Rm , D = [a1, b1] . . . [am , bm ],independently ofm . This means that the algo-rithm can be used in principle for any dimension.

The algorithm is based on the product of one-dimensionalinterpolations, i.e. the point at which we approximate f isprojected onto spaces of lower and lower dimension, until

a one-dimensional interpolation is performed. The key idea isto use a tree structure for storing the points of the reconstruc-tion stencil, the interpolation being computed during the visitof the tree. The tree structure has been adopted by Grne [22]to store the informations of the adaptive algorithm on struc-tured grid where it is necessary to keep the values at the ver-tices of the subcells generated by the iterative procedure. Thefirst step is to build a multidimensional grid. This is obtainedpartitioning each real interval [ai, bi ] with space stepk(i)andn(i)nodes (i= 1, . . . , m). Let us define xi(1) ,. . . ,xi (n(i))asthe i-th coordinates of the nodes, for i = 1, . . . , m. Let us con-sider the Lagrange basis functions of orderq, {j (x)}0jq inone space dimension and the corresponding polynomial inter-

polation Pq

u(x) =qj=0u(xk(j))j (x).We can now describe the general formulation for the multi-mensional interpolation. Letz Rm, z = (z1, . . . ,zm) be thepoint where we need to compute the function f, known only

at the grid nodes. Let fbe the approximation of f and fm itsvalue computed atz.

Let 0 i1, . . . , im q. We have:

fm = f(z1, . . . ,zm )

fm1im = f(z1, . . . ,zm1,xm(im))

...


4/15


fmsims+1,...,im = f(z1, . . . ,xms+1(ims+1) ,. . . ,xm(im ))

...

f1i2 ,...,im = f(z,x2(i2) ,. . . ,xm (im ))

f0i1 ,...,im = f(x1(i1) ,. . . ,xm (im)).

where

fm =q

j=0

fm1j j (zm )

fm1im =

qj=0

fm2j,im j (zm1)

...

fmsims+1,...,im =

qj=0

fms1j,ims1,...,im j (zms)

...

f1i2 ,...,im =

qj=0

f0j,i2,...,im j (z1). (11)

We observe that f0i1,... ,im are the values of fat grid nodes. Inthe next subsections we analyse in detail the main steps ofthe algorithm: storage of the initial data, location of the gridcell containing z, set-up of the tree structure and interpola-tion along the tree. Describing the algorithm, arrays will bedenoted in bold face characters.

3.1 Storage of initial data

The vectork Rm contains the space steps. In order to avoidany dependence on the dimension of the domain, the nodecoordinates (xi (1), . . . ,xi (n(i)), (i= 1, . . . , m), are storedin a unique vector x Rn tot, where n tot =

mj=1 n(j) ; thus

x = (x1(1), . . . ,x1(n(1)),x2(1), . . . ,xm (n(m))).For an easy determination of the components of this vec-

tor, we define the integer vector last Nm , where last(j) =j1i=0 n (i), so that the j -th components on the i-th dimension,

xi (j), corresponds to x(j + last(i)).In the same spirit, we store in a unique vector v Ri n(i)

the values of the function on the nodes.In order to determine easily the components where

we store the values of f at the nodes (x1(i1), . . . ,xj (ij ),

. . . ,xm (im)), we use a function (named Ind in the program)that, given as input the index vector I= (i1, i2, . . . , im),where 0 ik n(k) (k= 1, . . . , m), returns an integer whichis the position in v. Therefore, the value f(x1(i1), . . . ,xm (im)is stored in the componentv(Ind(I)).

3.2 Grid cell determination

For each z where we interpolate f, we define the integervector s Nm to represent the cell containing z , so that thej-th component z(j) will be between the s(j)-th and the(s(j) + 1)-th node. Using a cycle of calls to the subroutine

interval, which determines the j-th left component of thenode for everyz(j) R, we constructs.

3.3 Tree structure

The nodes which are really needed for interpolation, arestored in a vector with tree structure. If we use linear interpo-lation this will be a binary tree, with quadratic interpolation

a ternary tree and with cubic interpolation a quaternary tree.The number of points appearing in the interpolation formulain every cell will be denoted by q.In the case of linear interpolation, the first step is to projectzonto two spaces (q = 2):

V ar1m1 = {y Rm such that y1 = x1(s(1))},

V ar2m1 = {y Rm such that y1 = x1(s(1) + 1)}.

In the quadratic case we will consider three spaces (q = 3):one corresponding to the first component of the left nodeof z(1), one corresponding to the first componenent of theright node and one corresponding to the first componenentof the middle point of the subinterval containing z(1) in[a1, b1]. For cubic approximation we need four spaces (q =4):V ar1m1, V ar

2m1as for the linear case and two more, re-

lated to the first adjacent nodes.In the next steps, for every projection z, we continue toproject onto the spaces related to the adjacent nodes of thenextz components.

Let us describe in detail the linear case. Letz be the rootof the tree. Its sons are P11 (x1(s(1)),z(2), . . . , z(m))andP12 = (x1(s(1) + 1),z(2), . . . ,z(m)). The sons ofP

11 will be

P21 = (x1(s(1)),x2(s(2)),z(3) ,. . . ,z(m))

and

P22 = (x1(s(1)),x2(s(2) + 1),z(3) ,. . . ,z(m))

and the sons ofP12 will be

P23 = (x1(s(1) + 1),x2(s(1)),z(3) ,. . . ,z(m))

and

P24 = (x1(s(1) + 1),x2(s(2) + 1),z(3) ,. . . ,z(m)).

At the level(m 1) of the tree the projections are related tothe(m 1)-th coordinate, the first and last points of this levelbeing

Pm11 = (x1(s(1)),...,xm1(s(m 1)),z(m))

and

Pm12m1

= (x1(s(1) + 1) ,. . . ,xm1(s(m 1) + 1),z(m)).

The sons of these points are

Pm1 = (x1(s(1)),...,xm(s(m)),

Pm2 = (x1(s(1) + 1) ,. . . ,xm1(s(m 1)),,xm(s(m) + 1))

and

Pm2m 1 = (x1(s(1) + 1) ,. . . ,xm1(s(m 1) + 1),xm(s(m)),

Pm2m = (x1(s(1) + 1) ,. . . ,xm(s(m) + 1)).


5/15


Fig. 1. The a-tree structurefor linear interpolation

Fig. 2. The b-tree for linearinterpolation

Figure 1 shows the structure of the a-tree.The subroutine Treefill, given as input z and the vec-

tors, returns the tree array as an integer vectora Nalbtot. Thelenght of this array isalbtot=

mi=0q

i .Note that we do not store in the vectoraall the components ofthe tree points. Only the index of the components which differfrom the father components are stored. The first componentrelated toz is defined 0.

a = [0, s(1), s(1) + 1 sons of z

,

sons of s(1)

s(2), s(2) + 1, s(2), s(2) + 1 sons of(s(1)+1)

, . . .].

At thek-th step, we stay at the k-th level of the tree, con-tainingq k vertices; level 0 is the root level. Once at the m -thlevel, we can start to interpolate, since all the informations atthe values of fat the nodes are finally available.

3.4 Interpolation

We proceed building another tree, the b-tree, whose verticescontain the approximate values of f computed on the vertices

ofa: the generic vertexb(i)contains the approximation f onthe point corresponding at thea(i)vertex.These approximations are obtained from interpolation of the

values of frelated to the sons ofa(i)(computed at the previ-ous step of the algorithm). The tree b has one level less thana, because we use the last level ofa to build the (m 1)-thlevel ofb. The root ofb contains the required approximation

fm = f(z).Recalling the definition of the points P11 , . . . ,P

m2m in the pre-

vious section, Fig. 2 gives a sketch of the b-tree.

4 Minimization routines

In this section, we describe the minimization step in the algo-rithm. According to (9), the function to be minimized is, forany grid pointxp and any timetn,

Fnp (a) := vnxp +tb

xp, tn , a

+tg

xp, tn , a

whose computation requires in particular a reconstruction ofvn at the pointxp +tb(xp, tn, a).


6/15


Therefore, at any node and for any time step we need tocarry out theglobalminimization of a function which:

has a high computational cost to be computed; does not allow a simple way of computing even first order

derivatives they depend on the local expression of the re-construction operator and are in general discontinuous atthe interface between two space grid cells;

may be nonconvex (even ifg was convex, the global con-vexity also depends onvn ).

However, there are features which allow to overcomesome of these difficulties:

ifH is strictly convex, then the analytic solution is semi-concave (see [2]) and if in addition g is also smooth, thisimplies that, at the minimum point,Fnp would be differen-tiable. Although this is not exactly true for the numericalsolution, experiments show that in practice this is the case;

in smooth regions and with high-order reconstructions,the jump of derivatives across grid cells is negligible;

global convexity of Fnp is not a major concern unless forvery large time steps the range in which the minimum issearched is typically of radiusO(t).

Although many minimization algorithms may work re-placing exact expression of derivatives by incremental ratios,we use for the minimization step a Powell-like algorithmwhich is genuinely derivative-free, and which is widely con-sidered as a good compromise between convergence speedand a low number of function evaluations. More precisely, theprogram uses Brents routine PRAXIS from NETLIB (see [6]for details) which also works on the principle of finding con-

jugate directions by means of interpolated line searches. Thisroutine has proven to be robust enough to carry out the mini-mization of an unsmooth function, while retaining superlinearconvergence whenever the function is smooth.

Of course, the need for a global minimum requires a tabu-lation phase to be performed prior to PRAXIS. The tabulation

is done as a product of one dimensional tabulations with thesame number of samples on each dimension. The number ofsamples and step size for this operation depend on the numer-ical domain of dependence, and ultimately on t. The largert is, the wider the numerical domain of dependence, themore accurate the tabulation phase should be in order to avoidlocal minima.

5 The algorithm for high-dimensional problems

Equation (1) is approximated on an m-rectangular domain Rm, obtained as a product ofm bounded intervals. The

general form of a SL-scheme for (1) can be obtained couplingdifferent methods for the approximation of the characteristicswith various quadrature formulae as in (2) and local recon-structions. In fact, the dynamical system can be approximatedby respectively Euler, Trapezoidal or 4th order Runge Kuttaschemes, whereas the integral in the right-hand side of (5) canbe approximated by a quadrature formula of the same order(e.g. rectangle, trapezoid,. . . ) in order to obtain accurate re-sults in the overall approximation. Moreover, the values at allthe up-wind points z = x+ h(x,A) (where x is a node) isreconstructed by P1, P2 or cubic interpolation, which havebeen described in detail in the previous section. Coordinatesof the nodes are stored in the vector x, whereas the vector

v Rmi=1 n(i) contains the values of approximate solution on

the nodes. We briefly sketch the main procedures used in theprogram.

subroutine INV-IND

Each index j , such that 1 j nnode, selects only one nodeon the m -dimensional grid. The subroutine returns the indexvector of itsm components.

subroutine FIRST ESTIMATE CTRLThis sub-routine computes an initial guess for the control vec-tor where the numerical Hamiltonian attains its minimum.Since the dimension of the space is variable, this guess iscomputed as a product of one dimensional tabulations (see theprevious section).

subroutine INTER-m

It interpolates the function v(z, n ht) on z Rm, knowningthe vector v of function values on the grid nodes. The subrou-tine has been described in detail in Sect. 3.

functionHJ

This function computes the righ-hand side of (9) and calls theexternal function PRAXIS for the computation of the mini-mum. A cycle for j = 1, . . . , nnodeis performed.At the j-th step:

j [subroutine INV-IND ] node Rm,

where the vectornode returns the coordinates of the node atwhich we are approximating the solution.Up-wind points are computed as:

z = node htSPOST(node, ctrl, ht),

where z Rm is the endpoint of the trajectory starting atnode, subject to the dynamical system (5), and SPOSTis thefunction used to discretize this ODE.Ifz / , then we have to change htin order to have z .This procedure is equivalent to imposing a Dirichlet boundarycondition for equation (1).In the call

call INTER-m(z, vn1, interp),

approximate solution at up-wind points is interpolated and theresult is stored in the variable interp. In the next step we fi-nally compute the numerical Hamiltonian:

H J= interp + htFORMAQUAD(node, ctrl, ht).

Where FORMAQUAD is the function corresponding to thequadrature rule for the continuous cost function.The cycle ends for j= nnode.

functionHJtab

This function computes the numerical solution of (1), ap-proximating the minimum by a control tabulation on anm-dimensional sphere of unit radius, centered in the origin.This function is used in the case of front propagation, wherethe continuous hamiltonian is

H(x, t, v(x, t)) = v(x, t).


7/15


(It is known that the control vector where the minimum is at-tained belongs to such sphere).The tabulation is performed projecting on the sphere thepoints of an m -dimensional grid, centered in the origin. Thesubroutine avoids the use of PRAXIS and therefore the ap-proximation of the cost function which vanishes in the frontpropagation model.

Remark: Computational cost

It is quite important to quantify the computational cost ofthe algorithm and to show how it depends on the order of thereconstruction,q, and on the dimensionm of the problem.

To this end we will assume to work on a theoretical calcu-lator with infinite memory, neglecting the cost of all memoryallocations. We also assume that the basic floating-point op-erations, i.e. sum and multiplication, have unity cost in ourcalculations.

We will denote by Cdivision the cost of a division op-eration, by Cfun the cost of a call of a function, and byCInter(q) the cost of a one-dimension polynomial inter-polation of order q. We will assume all these values to beknown quantities (which depend on the hardware and on the

compiler) and we will write the numerical cost as a functionof these variables.

Let us consider the algorithm based on a first ordermethod for (5) and the cost integral. Similar considerationswill apply to higher order schemes, which will clearly bemore expensive. We limit the analysis to the case where theminimization is performed by tabulation since the iterativemethod implemented in PRAXISis difficult to analyse fromthe point of view of complexity. However, as we will see later,the scheme based on PRAXIS minimization is faster than theother. Let us consider the scheme:

vn+1

i = minan A

[t+ g(xi, nt, an)

+Pqvn(xi +tb(xi, tn, an)) . (12)

Let p be the number of subintervals in each dimension, so pm

will be the total number of nodes. We will denote by niterthe number of time iterations and by CA the cardinality of thediscrete set of controls A .

The global cost for the algorithm is given by:

niter[pm(CA CInterM(m, q) +Cfun)]. (13)

where CInterM(m, q) is the computational cost of the

m-dimensional interpolation. Now we will write this cost inan explicit form.Essentially what we do for each m -dimensional interpo-

lation is to project m-times the up-wind point zi (a) xi +tb(xi, tn, an) to a lower dimension space. Each projectionhas the cost of one division (see the a-tree in Fig. 1). Once wehave reached the one dimensional space we should perform2m1 one dimensional interpolations (see the b-tree in Fig. 2).So we have

CInterM(m, q) mCdivision+ 2m1CInter(q).(14)

In conclusion, the numerical cost of the scheme can be ex-pressed as:

niter{pm[CA (m Cdivision+ (15)

2m1CInter(q)) +Cfun]}.

Remark: the algorithm for minmax Hamiltonians

It is interesting to note that the approach described in this

section can also be applied to some HamiltonJacobi equa-tions with non convex hamiltonians. A typical example is theIsaacs equation related to pursuit-evasion games (see [26]).

We briefly sketch the essential features of this numericalproblem, whereas the interested reader will find the details,for example, in [3, 4].

A closed set T Rm is given and the game is the follow-ing: the first player a1 wants to minimize the arrival timeand the second player a2 wants to maximize the same cost.A typical example is to choose the target T as

T = { (y1,y2) Rm : |y1 y2| }

for some 0. The Isaacs equation corresponding to the

value of the game (provided it exists) is

w + mina2A2

maxa1A1

{b(x, a1, a2) Dw} 1 = 0 in Rm \T,

coupled with the homogeneous Dirichlet boundary conditionon the target

w = 0, inT .

The dynamic programming fully discrete scheme is con-structed on a grid and is written in the form of the fixed pointiteration

Vn+1

=F(Vn

)

with the map F : Rnnode Rnnode defined componentwise asfollows:

Fi (V)

max

a2min

a1Pi (a1, a2, V) + 1 forx

i / T,

0 for xi T

where eh and Pi is the polinomial reconstruction of thevalue at the point zi (a1, a2) x

i + hb(xi , a1, a2). It can beproved that Fhas a unique fixed pointV, that fixed point isour approximate solution at the nodes of the grid.

The minmax operator can be computed comparing a num-

ber of different values for every node and every fixed point it-eration. An alternative in this situation is to use a saddle-pointroutine instead of the minimization routine in Sect. 4. All theother steps of the algorithm, including the high-dimensionalinterpolation step, coincide with the steps described in thissection.

6 Numerical tests

The following tests show the performancesof the algorithm interms of the errors in L, L1 and L2 . The errors of the ap-proximations of a function v, evaluated at the barycentre of


8/15


cells, on a grid with a fixed number of nodes are computedaccording to the norms

1 =

nnode

i=1

(|ei |vol(K))(vol()

1(16)

and

2 =nnodei=1

(|ei |vol(K))2))

12 (vol()1 . (17)

Where ei are the errors at the cell barycenters, nnode isthe total number of nodes in the domain and Kis the ele-mentary grid cell. Note that such norms do not coincide withthe classical discrete definitions, but the variable dimension ofthe problem makes it necessary to renormalize by the volumeof the domain. In all the numerical test is the cartesianproduct of bounded interval ofR andnnode =Nm , where Nis the number of nodes in each one-dimensional interval.

6.1 Test 1 Linear dynamics, smooth case

Let us consider problem (1)(4) for

b(x, t, ) = , g(x, t, ) =1

22, A = Rm .

The initial condition reads

v0(x) =

1 x2 ifx2


9/15


Errors in Tables 46 are computed at time 0.5 with timestep h t= 0.1 on the domain[2, 2]m. The minimum of thehamiltonian with respect to the control is obtained by tabula-tion over the boundary of a sphere with unity radius. This isobtained by tabulating the cube [1, 1]m and projecting onlythe boundary nodes on the sphere boundary; we have chosen6 points on each side so the total number of boundary nodesis 6m 4m.

6.3 Test 3 Front propagation

Here we compute the solution of the level set equation (1) foran initial configuration of the front corresponding to a couple

Fig. 3. Front propagation

of spheres in R3, i.e. for

v0(x) = min(x 1, x 2) , (18)

where 1= (1, 1, 0), 2= (1, 1, 0) and where the frontevolves by superposition of a motion at constant normal speedand a rotation around the origin on thex1 x2 plane.

The numerical solution has been computed by an Euler/P2scheme with 50 nodes per side of the domain = [3, 3]3,

and the level surface v(x, t) = 0.3 plotted in Fig. 3 fromt= 0.1 to t= 1.1. Note that the propagation of the front ap-pears to be remarkably isotropic and the singularity generatedby the collision of the fronts is well resolved (basically, withina single space step).


10/15


6.4 Test 4 The moon landing problem

This is a classical control problem which was solved by Miele[27, 28] (see also the presentation in [21]). A spacecraft isattempting to land smoothly on the moon using a minimumamount of fuel. The gravity acceleration g is assumed to beconstant near the moon. The equations of motion are

h = v (19)

v = g + m1 (20)

m = k (21)

where h and v are the height and the vertical velocity ofthe spacecraft, the control is the thrust of the spacecraftsengine and m is the mass of the spacecraft. The control isnaturally bounded:

0 . (22)The initial time ist0 = 0 and the final time tfis the first time

the spacecraft touches the moon. The end conditions for thisproblem are

h(0) = h0, v(0) = v0, m(0) = M+ F (23)

h(tf) = 0 , v(tf) = 0 (24)

where h0 and v0 are the initial height and velocity of thespacecraft, M and Fare mass of the spacecraft without fueland the initial mass of the fuel.

The problem of minimizing the fuel is equivalent to mini-mize m(t1)over the class U, where U = {piecewise contin-uous function(t)defined on [t0, t1] s.t. 0 (t)

}.

Let (x1,x2,x3) = (h, v, m). Treating the final condition

by penalization, the problem we have to solve iswt(x, t) = inf

A{b(x, ) w(x, t)}

w(x, T) = (x) = x3 +x2

1+x2

2

1.

The solution solves a problem of optimal control

w(x, t) = inf()A

J(x, t, ()) = infA

(y(T)) ,

so we apply our scheme to the forward problem

ut(x, t) = infA{b(x, ) u(x, t)}u(x, 0) = (x)and we computewby w(x, t) = u(x, T t).

For each initial point on the numerical domain we cancompute the optimal trajectory, using the optimal control(t) = argmin

Aw(y(t), t).

Figure 4 shows the optimal trajectory in the phase space(velocity, space). The trajectory corresponds to the initial con-dition (x1,x2,x3) = (0.3, 0, 18), the test is performed withthe Heun scheme to approximate the dynamics and a P2 re-construction in the interpolation routine. We used space stepsof(0.03, 0.15, 0.26)and a time step oft= 0.03.

The spacecraft results to land attf = 0.99. The end pointx1,x2 = (0, 0)is reached with some numerical errors, this isdue to the penalization strategy.

As we are using a Heun scheme to approximate the trajec-tories, the discrete controls belong to a domain ofR2. In fact

Fig. 4.Moon Landing, optimal trajectories

Fig. 5.Moon Landing, optimal control 1

Fig. 6.Moon Landing, optimal control 2


11/15


the Heun scheme in this case is

y1 =xj +t1

2[b(xj, tn , 1)

+ b(xj +tb(xj, tn, 1), tn+1, 2)],

so we need a bidimensional discrete control vector to recon-struct the optimal trajectory. Figures 5, 6 show the discreteoptimal controls, respectively 1 and 2 (we note that pass-

ing from the discrete to the continuous control, 1 and 2may be interpreted in terms of successive moments of thecontinuous control, see [19]). All the figures shown that theoptimal trajectory is obtained falling down by moon grav-ity until a switch time t, and then braking until the landingwith maximum thrust. The jump in the control from zero(no thrust) to (maximum thrust) is obtained at the switchtimet. This is in perfect agreement with theory.

6.5 Test 5 A Ramsey model for a multi-sector economy

Let us consider an economy which has three sectors Si ,i= 1, . . . , 3 (the model can be easily extended to many sec-

tors) and let us denote by Yi , Ki , L i the output rate, thecapital and the labour of sector Si . The capital and labourare connected to the production through a production functionYi = Fi (Ki ,L i ). A classical example is the CobbDouglasfunction

F(K,L) = etKL1 , (0 1) (25)

whereis a coefficient connected to the technical innovation(see [1]). For every sector, we can divide by L i and define

yi = YiL i

, ki = KiLi

, i = 1, . . . , 3 obtaining the pro-capite pro-duction functions

yi= ei tk

ii , (0 i 1). (26)

A typical choice is i = m(1 i ) (see [1]), where m is theconstant rate of technical innovation. The rate of consumptionc(t)is a proportionu (the control) of a functiong(k(t))of thecapital, i.e. for every sector the consumption rate is given by

ci = u(t)gi(ki ), (umin u 1). (27)

Then, the rate of change of the capital in every sector (assum-ing that the sectors are separated) is

ki= f(ki ) u(t)g(ki) (umin u 1). (28)

Let Hbe a utility function which represents the utility of

the economy of consuming at ratec = c1 + c2 + c3. This func-tion H is typically monotone increasing and bounded. Thegoal will be to maximize the functional

tft0

H(c(t))dt, (29)

with respect to the choice of the control u () under the con-straintski(t0) = k

0i,ki(tf) = k

1i.

The following figures describe the optimal control u and theevolution in time of the variables ci , ki ,i = 1, . . . , 3 for the

following choice of the parameters 1= 0.2, 2= 0.5 and3 = 0.8,m = 1,

g(k) = k kmin forc>0, (30)

H(c) = Mc2

1 + c2 forc>0. (31)

In our numerical experiment we have set kmin= 0.33k0

and M= 10. The initial condition for the three componentsof the capital isk0(6, 4, 2)and the final objective fortf = 1.8is k(1.8) = (8, 6, 4). As in Test 4 we reverse time in orderto solve a Cauchy problem for the evolutive HJ equation,the penalization on the final position of the trajectory is thentranslated into the initial condition

u0(k) = |k k(1.8)|2

.

We have used in the computation 50 different controlsin [0.1,1], t= 0.05, = 0.0001 and 30 nodes in spacefor every dimension. As one can see looking at Figs. 810the capital always increase up to a time t which is close to

1.2, then the first two component of the capital decrease.The control switches from 0.1 to 1 approximately at the

Fig.7. Ramsey model, optimal controls

Fig.8. Ramsey model, optimal trajectory k1


12/15




Fig.11. Ramsey model, optimal trajectory c1

same time (see Fig. 7). The final position of the trajectory is(7.91, 7.41, 5.68). The final position is different from the tar-get, this is probably due to the penalization term coupled withthe fact that we used a 1 dimensional control for a three di-mensional system. The consumption rate is approximately 0.1and then jumps up after the control switching.

Fig. 12.Ramsey model, optimal trajectory c2

Fig. 13.Ramsey model, optimal trajectory c3

6.6 Test 6 Control of the HIV-1 dynamics

In the paper [31] Perelson and Nelson have recently describedthe dynamics of the HIV-1 virus infection (the virus thatcauses AIDS). We follow their model just adding a controlterm in order to describe the drug therapy effect.The model considers three populations: the uninfected cells C,the productively infected cellsIand the virusV. All the pop-ulations are described by their density per unit of volume. Thedynamics is

C= s + pC1 CCmax

dCC (1 u)kVC, (32)I= (1 u)kVC dII, (33)

V=NdII dVV, (34)

where s represents the rate at which new Ccells are createdfrom sources within the body, p is the maximum proliferationrate for the Ccells,Cmax is the the cell population density atwhich the proliferation shuts off,dC ,dI anddV are (respec-tively) the death rates for the C, I andV populations, NdI isthe average rate of virion production,u is the control driven


13/15


by the drug therapy. The term k VCtells that the variation ofthe population of uninfected and infected cell is proportionalto VC (at least one virion should enter a cell to infect it).A usual constraint is dCCmax> s. A typical drug therapy isa treatment based on RT (reverse transcriptase) or protease in-hibitors. The coefficientu, the control of our model, measurethe effectiveness of the inhibitor: for u = 1 the inhibition is100% effective, wheras for u = 0 there is no inhibition. Typ-ically, 0 u umax


14/15


Table 7.P1 Interpolation

m nnode CPU time

2 202


15/15


30. Osher, S., Fedkiw, R.P.: Level-set methods: an overview and somerecent results. CAM report, Department of Mathematics, UCLA,2000

31. Perelson, A.S., Nelson, P.W.: Mathematical analysis of HIV-1 dynam-ics in vivo. SIAM Review 41, 344 (1999)

32. Perelson, A.S., Kirschner, D.E., de Boer, R.: Dynamics of HIV infec-tion of CD4+ cells. Math. Biosci. 114, 81125 (1993)

33. Sagona, M., Seghini, A.: An adaptive scheme on unstructured grids forthe ShapefromShading problem. In [18].

34. Santos, M.S., Vigo-Aguiar, J.: Analysis of a numerical dynamic pro-

gramming algorithm applied to economic models. Econometrica 66,409426 (1998)

35. Sethian, J.A.: Level Set Method. Evolving interfaces in geometry,fluid mechanics, computer vision, and materials science CambridgeMonographs on Applied and Computational Mathematics, Vol. 3,Cambridge: Cambridge University Press 1996

36. Sethian, J.A.: Fast marching methods. SIAM Rev. 41, 199235(1999)

37. Sethian, J.A., Vladimirsky, A.: Ordered upwind methods for staticHamiltonJacobi equations: theory and algorithms. University of Cal-ifornia at Berkeley, preprint, June 2001

38. Sethian, J.A., Vladimirsky, A.: Fast methods for the Eikonal and re-

lated HamiltonJacobi equations on unstructured meshes. Proc. Natl.Acad. Sci. USA 97, 56995703 (2000)

an efficient algorithm for hamilton–jacobi equations in high dimension

Documents