A Primer on the Calculus of Variations and Optimal Control TheoryMike Mesterton-Gibbons

STUDENT MATHEMAT ICAL L IBRARYVolume 50

A Primer on the Calculus of Variations and Optimal Control Theory

http://dx.doi.org/10.1090/stml/050

A Primer on the Calculus of Variations and Optimal Control Theory

Mike Mesterton-Gibbons

STUDENT MATHEMAT ICAL L IBRARYVolume 50

American Mathematical SocietyProvidence, Rhode Island

Editorial Board

Gerald B. FollandRobin Forman

Brad G. Osgood (Chair)Michael Starbird

Cover art by Karen Mesterton-Gibbons.

2000 Mathematics Subject Classification. Primary 49–01, 49K15, 49N05,49N35; Secondary 92–02, 92D50.

For additional information and updates on this book, visitwww.ams.org/bookpages/stml-50

Library of Congress Cataloging-in-Publication Data

Mesterton-Gibbons, Mike.A primer on the calculus of variations and optimal control theory / Mike

Mesterton-Gibbons.p. cm. — (Student mathematical library ; v. 50)

Includes bibliographical references and index.ISBN 978-0-8218-4772-5 (alk. paper)1. Calculus of variations. 2. Control theory. 3. Mathematical optimization.

I. Title.

QA315.M46 2009515′.64—dc22 2009007701

Copying and reprinting. Individual readers of this publication, and nonprofitlibraries acting for them, are permitted to make fair use of the material, such as tocopy a chapter for use in teaching or research. Permission is granted to quote briefpassages from this publication in reviews, provided the customary acknowledgment ofthe source is given.

Republication, systematic copying, or multiple reproduction of any material in thispublication is permitted only under license from the American Mathematical Society.Requests for such permission should be addressed to the Acquisitions Department,American Mathematical Society, 201 Charles Street, Providence, Rhode Island 02904-2294 USA. Requests can also be made by e-mail to reprint-permission@ams.org.

c© 2009 by the American Mathematical Society. All rights reserved.Reprinted with corrections by the American Mathematical Society, 2014.

The American Mathematical Society retains all rightsexcept those granted to the United States Government.

Printed in the United States of America.

©∞ The paper used in this book is acid-free and falls within the guidelinesestablished to ensure permanence and durability.

Visit the AMS home page at http://www.ams.org/

10 9 8 7 6 5 4 3 2 19 18 17 16 15 14

To the memory of my brother

Johnwho inspired one to keep things simple

Contents

Foreword ix

Acknowledgments xiii

Lecture 1. The Brachistochrone 1

Lecture 2. The Fundamental Problem. Extremals 7

Appendix 2: The Fundamental Lemma 15

Lecture 3. The Insufficiency of Extremality 19

Appendix 3: The Principle of Least Action 26

Lecture 4. Important First Integrals 29

Lecture 5. The du Bois-Reymond Equation 35

Appendix 5: Another Fundamental Lemma 40

Lecture 6. The Corner Conditions 41

Lecture 7. Legendre’s Necessary Condition 51

Appendix 7: Yet Another Lemma 55

Lecture 8. Jacobi’s Necessary Condition 57

Appendix 8: On Solving Jacobi’s Equation 65

vii

viii Contents

Lecture 9. Weak Versus Strong Variations 67

Lecture 10. Weierstrass’s Necessary Condition 73

Lecture 11. The Transversality Conditions 81

Lecture 12. Hilbert’s Invariant Integral 91

Lecture 13. The Fundamental Sufficient Condition 101

Appendix 13: The Equations of an Envelope 108

Lecture 14. Jacobi’s Condition Revisited 111

Lecture 15. Isoperimetrical Problems 119

Appendix 15: Constrained Optimization 124

Lecture 16. Optimal Control Problems 127

Lecture 17. Necessary Conditions for Optimality 135

Appendix 17: The Calculus of Variations Revisited 146

Lecture 18. Time-Optimal Control 149

Lecture 19. A Singular Control Problem 159

Lecture 20. A Biological Control Problem 163

Lecture 21. Optimal Control to a General Target 167

Appendix 21: The Invariance of the Hamiltonian 180

Lecture 22. Navigational Control Problems 183

Lecture 23. State Variable Restrictions 195

Lecture 24. Optimal Harvesting 203

Afterword 219

Solutions or Hints for Selected Exercises 221

Bibliography 245

Index 249

Foreword

This set of lectures forms a gentle introduction to both the classical

theory of the calculus of variations and the more modern develop-

ments of optimal control theory from the perspective of an applied

mathematician. It focuses on understanding concepts and how to ap-

ply them, as opposed to rigorous proofs of existence and uniqueness

theorems; and so it serves as a prelude to more advanced texts in

much the same way that calculus serves as a prelude to real anal-

ysis. The prerequisites are correspondingly modest: the standard

calculus sequence, a first course on ordinary differential equations,

some facility with a mathematical software package, such as Maple,

Mathematica� (which I used to draw all of the figures in this book)

or MATLAB—nowadays, almost invariably implied by the first two

prerequisites—and that intangible quantity, a degree of mathemati-

cal maturity. Here at Florida State University, the senior-level course

from which this book emerged requires either a first course on par-

tial differential equations—through which most students qualify—or

a course on analysis or advanced calculus, and either counts as suffi-

cient evidence of mathematical maturity. These few prerequisites are

an adequate basis on which to build a sound working knowledge of

the subject. To be sure, there ultimately arise issues that cannot be

addressed without the tools of functional analysis; but these are in-

tentionally beyond the scope of this book, though touched on briefly

ix

x Foreword

towards the end. Thus, on the one hand, it is by no means necessary

for a reader of this book to have been exposed to real analysis; and

yet, on the other hand, such prior exposure cannot help but increase

the book’s accessibility.

Students taking a first course on this topic typically have diverse

backgrounds among engineering, mathematics and the natural or so-

cial sciences. The range of potential applications is correspondingly

broad: the calculus of variations and optimal control theory have

been widely used in numerous ways in, e.g., biology [27, 35, 58],1

criminology [18], economics [10, 26], engineering [3, 49], finance [9],

management science [12, 57], and physics [45, 63] from a variety

of perspectives, so that the needs of students are too extensive to

be universally accommodated. Yet one can still identify a solid core

of material to serve as a foundation for future graduate studies, re-

gardless of academic discipline, or whether those studies are applied

or theoretical. It is this core of material that I seek to expound

as lucidly as possible, and in such a way that the book is suitable

not only as an undergraduate text, but also for self-study. In other

words, this book is primarily a mathematics text, albeit one aimed

across disciplines. Nevertheless, I incorporate applications—cancer

chemotherapy in Lecture 20, navigational control in Lecture 22 and

renewable resource harvesting in Lecture 24—to round out the themes

developed in the earlier lectures.

Arnold Arthurs introduced me to the calculus of variations in

1973-74, and these lectures are based on numerous sources consulted

at various times over the 35 years that have since elapsed; sometimes

with regard to teaching at FSU; sometimes with regard to my own re-

search contributions to the literature on optimal control theory; and

only recently with regard to this book. It is hard now to judge the

relative extents to which I have relied on various authors. Neverthe-

less, I have relied most heavily on—in alphabetical order—Akhiezer

[1], Bryson & Ho [8], Clark [10], Clegg [11], Gelfand & Fomin [16],

Hadley & Kemp [19], Hestenes [20], Hocking [22], Lee & Markus

[33], Leitmann [34], Pars [47], Pinch [50] and Pontryagin et al. [51];

and other authors are cited in the bibliography. I am grateful to all

of them, and to each in a measure proportional to my indebtedness.

1Bold numbers in square brackets denote references in the bibliography (p. 245).

Foreword xi

To most of the lectures I have added exercises. Quite a few are

mine; but most are drawn or adapted from the cited references for

their aptitude to reinforce the topic of the lecture. At this level of

discourse, various canonical exercises are pervasive in the pedagogi-

cal literature and can be found in multiple sources with only minor

variation: if these old standards are not de rigueur, then they are at

least very hard to improve on. I have therefore included a significant

number of them, and further problems, if necessary, may be found

among the references (as indicated by endnotes). Only solutions or

hints for selected exercises appear at the end of the book, but more

complete solutions are available from the author.2

Finally, a word or two about notation. Just as modelling compels

tradeoffs among generality, precision and realism [36], so pedagogy

compels tradeoffs among generality, rigor and transparency; and cen-

tral to those tradeoffs is use of notation. At least two issues arise. The

first is the subjectivity of signal-to-noise ratio. One person’s oasis of

terminological correctness may be another person’s sea of impenetra-

ble clutter; and in any event, strict adherence to unimpeachably cor-

rect notation entails encumbrances that often merely obscure. The

second, and related, issue is that diversity is intrinsically valuable

[46]. In the vast ecosystem of mathematical and scientific literature,

polymorphisms of notation survive and prosper because, as in nature,

each variant has its advantages and disadvantages, and none is uni-

versally adaptive. Aware of both issues, I use a mix of notation that

suppresses assumed information when its relevance is not immediate,

thus striving at all times to emphasize clarity over rigor.

2To any instructor or bona fide independent user. For contact details, seehttp://www.ams.org/bookpages/stml-50/index.html.

Acknowledgments

I am very grateful to the two anonymous reviewers of the original

manuscript for their valuable comments and suggestions; to Ed Dunne

for his advice and encouragement throughout; to Barbara Beeton for

expert technical support; to Jennifer Wright Sharp for expert edito-

rial assistance; to Karen and Sarah Mesterton-Gibbons for carefully

proofing the manuscript; to Alex Mesterton-Gibbons for assistance

with artwork; to Karen Mesterton-Gibbons for so imaginatively in-

terpreting six of my figures to produce such captivating cover art (now

affectionately known as Graphman); to Peter Sykes for enhancing its

appeal with his cover design; and, above all, to Karen, Sarah and

Alex, all three, for their constant moral support. They are the reason

that I have a wonderful life.

xiii

Bibliography

1. N. I. Akhiezer, The Calculus of Variations, Blaisdell Publishing Com-pany, New York, 1962.

2. A. M. Arthurs, Calculus of Variations, Routledge & Kegan Paul, Lon-don, 1975.

3. M. Athans and P. L. Falb, Optimal Control, McGraw-Hill, New York,1966.

4. D. J. Bell and D. H. Jacobson, Singular Optimal Control Problems,Academic Press, London, 1975.

5. G. A. Bliss, Calculus of Variations, Open Court Publishing Company,La Salle, Illinois, 1925.

6. O. Bolza, Lectures on the Calculus of Variations, 3rd ed., Chelsea Pub-lishing Company, New York, 1973, edited reprint of 1904 original.

7. L. Brand, Differential and Difference Equations, Wiley, New York,1966.

8. A. E. Bryson and Y.-C. Ho, Applied Optimal Control, Hemisphere Pub-lishing, Washington, D.C., 1975.

9. P. Chen and S. M. N. Islam, Optimal Control Models in Finance,Springer, New York, 2005.

10. C. W. Clark, Mathematical Bioeconomics: The Optimal Managementof Renewable Resources, 2nd ed., John Wiley, New York, 1990.

11. J. C. Clegg, Calculus of Variations, Oliver & Boyd, Edinburgh, 1968.

12. M. M. Connors and D. Teichroew, Optimal Control of Dynamic Op-erations Research Models, International Textbook Company, Scranton,Pennysylvania, 1967.

245

246 Bibliography

13. L. E. Elsgolc, Calculus of Variations, Addison-Wesley, Reading, Mas-sachusetts, 1962.

14. G. M. Ewing, Calculus of Variations with Applications, Norton, NewYork, 1969.

15. J. Franks, A (Terse) Introduction to Lebesgue Integration, AmericanMathematical Society, Providence, Rhode Island, 2009.

16. I. M. Gelfand and S. V. Fomin, Calculus of Variations, Prentice Hall,Englewood Cliffs, New Jersey, 1963.

17. H. H. Goldstine, A History of the Calculus of Variations from the 17ththrough the 19th Century, Springer, New York, 1980.

18. D. Grass, J. P. Caulkins, G. Feichtinger, G. Tragler, and D. A. Behrens,Optimal Control of Nonlinear Processes, Springer, Berlin, 2008.

19. G. Hadley and M. C. Kemp, Variational Methods in Economics, NorthHolland, Amsterdam, 1971.

20. M. R. Hestenes, Calculus of Variations and Optimal Control Theory,John Wiley, New York, 1966.

21. S. Hildebrandt and A. Tromba, The Parsimonious Universe: Shapeand Form in the Natural World, Springer, New York, 1996.

22. L. M. Hocking, Optimal Control: An Introduction to the Theory withApplications, Clarendon Press, Oxford, 1991.

23. G. H. Howison and J. Ray, A Treatise on Analytic Geometry, Wilson,Hinkle & Co., Cincinnati, Ohio, 1869.

24. R. G. Huffaker, M. G. Bhat, and S. M. Lenhart, Optimal trappingstrategies for diffusing nuisance-beaver populations, Natural ResourceModeling 6 (1992), 71–97.

25. E. L. Ince, Ordinary Differential Equations, Dover, New York, 1956,reprint of 1926 original.

26. M. I. Kamien and N. L. Schwartz, Dynamic Optimization: The Calcu-lus of Variations and Optimal Control in Economics and Management,North Holland, New York, 1981.

27. M. Kimmel and A. Swierniak, Control theory approach to cancerchemotherapy: benefiting from phase dependence and overcoming drugresistance, Tutorials in Mathematical Biosciences III: Cell Cycle, Prolif-eration, and Cancer (A. Friedman, ed.), Lecture Notes in Mathematics,vol. 1872, Springer, 2006, pp. 185–221.

28. G. Knowles, An Introduction to Applied Optimal Control, AcademicPress, New York, 1981.

29. A. N. Kolmogorov and S. V. Fomin, Elements of the Theory of Func-tions and Functional Analysis, Dover, Mineola, New York, 1999, reprintof 1957 (Volume 1) and 1961 (Volume 2) originals.

Bibliography 247

30. E. Kreyszig, Introductory Functional Analysis with Applications, Wiley,New York, 1977.

31. P. D. Lax, Functional Analysis, Wiley, New York, 2002.

32. L. P. Lebedev and M. J. Cloud, The Calculus of Variations and Func-tional Analysis: With Optimal Control and Applications in Mechanics,World Scientific, London, 2003.

33. E. B. Lee and L. Markus, Foundations of Optimal Control Theory, JohnWiley, New York, 1967.

34. G. Leitmann, The Calculus of Variations and Optimal Control, PlenumPress, New York, 1981.

35. S. Lenhart and J. T. Workman, Optimal Control Applied to BiologicalModels, Chapman & Hall/CRC, Boca Raton, Florida, 2007.

36. R. Levins, The strategy of model building in population biology, Amer-ican Scientist 54 (1966), 421–431.

37. R. Martin and K. L. Teo, Optimal Control of Drug Administration inCancer Chemotherapy, World Scientific, Singapore, 1994.

38. E. A. Maxwell, An Analytical Calculus, vol. 2, Cambridge UniversityPress, Cambridge, 1954.

39. I. McCausland, Introduction to Optimal Control, John Wiley, NewYork, 1969.

40. J. P. McDanell and W. F. Powers, Necessary conditions for joiningoptimal singular and nonsingular subarcs, SIAM Journal on Control 9(1971), 161–173.

41. M. Mesterton-Gibbons, On the optimal policy for combined harvestingof independent species, Natural Resource Modeling 2 (1987), 109–134.

42. , On the optimal policy for combining harvesting of predator andprey, Natural Resource Modeling 3 (1988), 63–90.

43. , A technique for finding optimal two-species harvesting policies,Ecological Modelling 92 (1996), 235–244.

44. , A Concrete Approach to Mathematical Modelling, John Wiley,New York, 2007, corrected reprint of 1989 original.

45. R. K. Nesbet, Variational Principles and Methods in TheoreticalPhysics and Chemistry, Cambridge University Press, Cambridge, 2003.

46. S. E. Page, The Difference: How the Power of Diversity Creates Bet-ter Groups, Firms, Schools, and Societies, Princeton University Press,Princeton, New Jersey, 2007.

47. L. A. Pars, An Introduction to the Calculus of Variations, John Wiley,New York, 1962.

48. , A Treatise on Analytical Dynamics, John Wiley, New York,1965.

248 Bibliography

49. I. P. Petrov, Variational Methods in Optimum Control Theory, Aca-demic Press, New York, 1968.

50. E. R. Pinch, Optimal Control and the Calculus of Variations, OxfordUniversity Press, Oxford, 1993.

51. L. S. Pontryagin, V. G. Boltyanskii, R. V. Gamkrelidze, and E. F.Mishchenko, The Mathematical Theory of Optimal Processes, L. S. Pon-tryagin Selected Works (R. V. Gamkrelidze, ed.), vol. 4, Gordon andBreach, New York, 1986, revised translation of 1961 Russian original.

52. D. L. Ragozin and G. Brown, Harvest policies and nonmarket valuationin a predator-prey system, Journal of Environmental Economics andManagement 12 (1985), 155–168.

53. F. P. J. Rimrott and W. M. Szczygielski, Analogies between the Chap-lygin problem and the Kepler problem, Technische Mechanik 15 (1995),no. 4, 315–324.

54. R. T. Rockafellar, Convex Analysis, Princeton University Press, Prince-ton, New Jersey, 1970.

55. W. Rudin, Functional Analysis, 2nd ed., McGraw-Hill, New York, 1991.

56. H. Ruskeepaa, Mathematica� Navigator, 2nd ed., Academic Press, Am-sterdam, 2004.

57. S. P. Sethi and G. L. Thompson, Optimal Control Theory: Applicationsto Management Science and Economics, 2nd ed., Springer, New York,2000.

58. G. W. Swan, Applications of Optimal Control Theory in Biomedicine,Marcel Dekker, New York, 1984.

59. G. W. Swan and T. L. Vincent, Optimal control analysis in thechemotherapy of IgG multiple myeloma, Bulletin of Mathematical Bi-ology 39 (1977), 317–337.

60. J. L. Troutman, Variational Calculus with Elementary Convexity,Springer, New York, 1983.

61. T. L. Vincent and W. J. Grantham, Nonlinear and Optimal ControlSystems, John Wiley, New York, 1997.

62. F. Y. M. Wan, Introduction to the Calculus of Variations with Appli-cations, Chapman & Hall, New York, 1995.

63. R. Weinstock, Calculus of Variations: With Applications to Physicsand Engineering, Dover, New York, 1974, reprint of 1952 original.

64. J. Wilen and G. Brown, Optimal recovery paths for perturbations oftrophic level bioeconomic systems, Journal of Environmental Economicsand Management 13 (1986), 225–234.

65. L. C. Young, Lectures on the Calculus of Variations and Optimal Con-trol Theory, 2nd ed., Chelsea Publishing Company, New York, 1980.

Index

C1, 8

C2, 8

D1, 8

abnormal case, 172, 177

acceleration

due to gravity, 2, 27, 34

accessory equation, 59

action integral, 26

admissible, 8, 127, 136

admissible control, see control

admissible extremal, see extremal

admissible variation, see variation

augmented state, 142, 168–176

bang-bang control, see control

beaver population, 218

Bernoulli, John, 1

biotechnical productivity, 206, 211,

216

brachistochrone problem, 1, 25,30–32, 56, 109

branch cut, 105

broken extremal, see extremal

calculus of variations, 1, 9, 127,131, 134, 146

cancer chemotherapy, 163–166

carrying capacity, 129, 204

catchability, 129, 204

catenary, 221

Chaplygin’s problem, 190–193

circle, 87, 123

co-state

equations, 140, 142, 160

variable, 140

vector, 142

adjusted, 176

modified, 175

cone of attainability, 172

conjugate point, 59–65, 115

constrained optimization, 124–126

control

admissible, 136, 146, 154, 164

bang-bang, 131, 144, 155

feedback, 146, 152, 166, 187, 189,209

open-loop, 146

optimal, 136–142, 146, 152, 154

existence of, 216, 219

synthesis of, 146, 151–157

piecewise-continuous, 167

singular, 144, 159–162, 208–215

time-optimal, 149–151, 183–189

controllability, 154–155

convex

combination, 167

subset, 167

corner, 8, 23, 36, 41

condition

first, 38, 45, 47, 147

249

250 Index

second, 41–47, 79, 147

Weierstrass-Erdmann, 47, 72,147

cost

axis, 142, 168

functional, 136

general, 174–176

terminal, 176

cycloid, 5, 6, 32

Dido’s problem, 123

differentiability, 8

dimensionless variables andparameters, 127–130, 163–164,188

direct method, 20, 27, 28, 132

direction field, 91

discontinuous control, 134, 167

discount factor or rate, 129, 205

du Bois-Reymond equation, 37

ecosystem

two-species, 203

eigenvalue, 150

complex, 155

of Jacobian matrix, 210

real, 152, 153, 155

ellipse, 39, 112

polar equation of, 192

embedded extremal, see extremal

endpoint minimum, 42, 46

energy, 2, 27

envelope, 107, 113–114

equations of, 108

equilibrium point, 151, 204

node

stable, 152, 154, 155

unstable, 153–155, 211

saddle point, 154–155, 211

Euler’s rule, 123, 148

Euler-Lagrange equation, 13, 26,29, 37, 59, 72, 85, 123, 147

excess function

Weierstrass’s, 78, 102, 106

existence of optimal control, seecontrol

extinction, 206, 216

extremal, 13

admissible, 15, 17–18, 20, 24,31–34nonunique, 17, 49, 62, 111unique, 25

broken, 23, 38, 48–50simple, 48–50

embedded, 99, 102–108regular, 58

straight-line, 21extremization, 13

feedback control, see controlfield

of extremals, 91, 101–107

of semi-extremals, 107first integral of the Euler-Lagrange

equation, 30, 32first variation, see variation

fishery, 129, 203free

endpoint, 86terminal time, 136

frictionless bead, 1, 34functional, 3functional analysis, ix, 220fundamental lemma, 15

fundamental problem, 7–17fundamental sufficient condition,

see sufficient condition

globalmaximum, 134

minimum, 68, 116Green’s theorem, 94, 133growth rate

per capita

maximum, 129, 204

Hamiltonian, 29, 45, 140, 142, 147,

160, 164invariance of, 180–181modified

for state variable restrictions,197

for terminal cost, 175Hilbert’s differentiability condition,

38Hilbert’s invariant integral, see

invariant integral

Index 251

hyperbola

rectangular, 20, 24hyperplane, 168

separating, 168, 172, 174

hypersurface, 167

index set, 135

index theorem, 212

interior minimum, 11, 83

constrained, 122

invariant integralHilbert’s, 93–95, 101, 114

isoperimetrical problem, 119–124,147–148

classical, 119, 123–124

Jacobi’s equation, 59–65

Jacobi’s necessary condition, seenecessary condition

Lagrange multiplier, 123, 125

Lagrangian, 26

least action, principle of, 26, 27

Legendre conditionstrengthened, 58, 110

Legendre’s necessary condition, seenecessary condition

Legendre-Clebsch condition

generalized, 209

limit cycle, 212

little oh, 52, 137

Mathematica�, 3, 63, 187, 189,213, 222, 223

maximization

versus minimization, 13, 22, 54,104, 119, 160, 190, 205

maximum principle

Pontryagin’s, 141–143, 146, 150,159, 160, 162, 167

Michaelis-Menten kinetics, 163

minimization

versus maximization, 13, 22, 54

natural dynamics, 203

navigation

boat, 183–189plane, 189–193

necessary condition

Jacobi’s, 59, 115, 209Legendre’s, 54, 79, 147Weierstrass’s, 78, 110

necessary conditionsfor optimal control, 135–143

negative phase-plane, 144, 151Newton’s method

simple versus double root, 201node, see equilibrium pointnonautonomous control problem,

159, 162

normal case, 177notation, xi, 15, 72, 99numerical integration, 188–189,

201, 214–215

ODE, 29open-loop control, see control

optimal control, see controloptimal control problems, 127optimal harvesting, 129, 203–218optimal trajectory, see trajectory

orthogonally, 89

parabola, 61–62, 96, 98, 105, 144pencil

of extremals, 91–93, 112, 117

point, 91, 102perturbation cone, 172phase-plane analysis, 144–146,

151–157, 203–216piecewise-continuous control, see

control

piecewise-smooth, 8Pontryagin’s principle, see

maximum principlepositive phase-plane, 144, 151

present value, 129, 205Problem B, 164, 166Problem E, 130, 132–134, 136,

159–162, 204Problem P, 128, 131–132, 136,

143–146, 149, 202proof by contradiction, 16, 55, 59pseudo-optimal, 196, 199, 202

regular extremal, see extremalregular problem, 39, 58, 78, 103,

116

252 Index

revenue, 204

river crossing, 188–189, 199–202

saddle point, see equilibrium point

scaling, 127–130, 163–164, 188

second variation, see variation

semi-field, 107, 113

separatrix, 212

simple broken extremal, seeextremal

simply connected, 91

singular control, see control

smooth, 8

state, 128, 135

restricted, 196–197

Stokes’ theorem, 94

strengthened Legendre condition,see Legendre condition

strong minimum, 67, 73, 110, 116

strong variation, see variation, 81

sufficient condition

fundamental, 102, 219

surface of revolution

minimum area of, 6, 14–15, 17,25, 32, 126

switching

curve, 152, 157, 178

function, 150, 160

synthesis of optimal control, seecontrol

target, 137

general, 167–176

Taylor series, 52, 53

Taylor’s theorem, 137, 171

with remainder, 78, 103

terminal cost, see cost

terminality condition, 136, 141

time-optimal control, see control

total variation, see variation

trajectory

optimal, 136–142, 144, 209

nonunique, 202

transversality conditions, 86–87,174–176, 179–180

trial curve, 4, 9, 41, 51, 73

two-parameter, 121

trial function, 4, 46, 74, 81

tumor growth, 163

variationadmissible, 10–12first, 53second, 53strong, 67, 73–74, 81total, 52, 53, 102weak, 9, 67, 73–74, 81

velocity, 2

weak minimum, 67, 73weak variation, see variationWeierstrass’s excess function, see

excess functionWeierstrass’s necessary condition,

see necessary conditionWeierstrass-Erdmann conditions,

see cornerwildlife management, 203, 218

Zermelo’s problem, 183–189,197–202

Published Titles in This Series

50 Mike Mesterton-Gibbons, A primer on the calculus of variations andoptimal control theory, 2009

49 Francis Bonahon, Low-dimensional geometry: From euclidean surfaces

to hyperbolic knots, 2009

48 John Franks, A (terse) introduction to Lebesgue integration, 2009

47 L. D. Faddeev and O. A. Yakubovskiı, Lectures on quantummechanics for mathematics students, 2009

46 Anatole Katok and Vaughn Climenhaga, Lectures on surfaces:(Almost) everything you wanted to know about them, 2008

45 Harold M. Edwards, Higher arithmetic: An algorithmic introduction tonumber theory, 2008

44 Yitzhak Katznelson and Yonatan R. Katznelson, A (terse)introduction to linear algebra, 2008

43 Ilka Agricola and Thomas Friedrich, Elementary geometry, 2008

42 C. E. Silva, Invitation to ergodic theory, 2007

41 Gary L. Mullen and Carl Mummert, Finite fields and applications,2007

40 Deguang Han, Keri Kornelson, David Larson, and Eric Weber,Frames for undergraduates, 2007

39 Alex Iosevich, A view from the top: Analysis, combinatorics and numbertheory, 2007

38 B. Fristedt, N. Jain, and N. Krylov, Filtering and prediction: Aprimer, 2007

37 Svetlana Katok, p-adic analysis compared with real, 2007

36 Mara D. Neusel, Invariant theory, 2007

35 Jorg Bewersdorff, Galois theory for beginners: A historical perspective,2006

34 Bruce C. Berndt, Number theory in the spirit of Ramanujan, 2006

33 Rekha R. Thomas, Lectures in geometric combinatorics, 2006

32 Sheldon Katz, Enumerative geometry and string theory, 2006

31 John McCleary, A first course in topology: Continuity and dimension,2006

30 Serge Tabachnikov, Geometry and billiards, 2005

29 Kristopher Tapp, Matrix groups for undergraduates, 2005

28 Emmanuel Lesigne, Heads or tails: An introduction to limit theorems inprobability, 2005

27 Reinhard Illner, C. Sean Bohun, Samantha McCollum, and Theavan Roode, Mathematical modelling: A case studies approach, 2005

26 Robert Hardt, Editor, Six themes on variation, 2004

25 S. V. Duzhin and B. D. Chebotarevsky, Transformation groups forbeginners, 2004

PUBLISHED TITLES IN THIS SERIES

24 Bruce M. Landman and Aaron Robertson, Ramsey theory on theintegers, 2004

23 S. K. Lando, Lectures on generating functions, 2003

22 Andreas Arvanitoyeorgos, An introduction to Lie groups and thegeometry of homogeneous spaces, 2003

21 W. J. Kaczor and M. T. Nowak, Problems in mathematical analysisIII: Integration, 2003

20 Klaus Hulek, Elementary algebraic geometry, 2003

19 A. Shen and N. K. Vereshchagin, Computable functions, 2003

18 V. V. Yaschenko, Editor, Cryptography: An introduction, 2002

17 A. Shen and N. K. Vereshchagin, Basic set theory, 2002

16 Wolfgang Kuhnel, Differential geometry: curves – surfaces – manifolds,second edition, 2006

15 Gerd Fischer, Plane algebraic curves, 2001

14 V. A. Vassiliev, Introduction to topology, 2001

13 Frederick J. Almgren, Jr., Plateau’s problem: An invitation to varifoldgeometry, 2001

12 W. J. Kaczor and M. T. Nowak, Problems in mathematical analysisII: Continuity and differentiation, 2001

11 Mike Mesterton-Gibbons, An introduction to game-theoreticmodelling, 2000

10 John Oprea, The mathematics of soap films: Explorations with Maple�,

2000

9 David E. Blair, Inversion theory and conformal mapping, 2000

8 Edward B. Burger, Exploring the number jungle: A journey intodiophantine analysis, 2000

7 Judy L. Walker, Codes and curves, 2000

6 Gerald Tenenbaum and Michel Mendes France, The prime numbersand their distribution, 2000

5 Alexander Mehlmann, The game’s afoot! Game theory in myth and

paradox, 2000

4 W. J. Kaczor and M. T. Nowak, Problems in mathematical analysisI: Real numbers, sequences and series, 2000

3 Roger Knobel, An introduction to the mathematical theory of waves,2000

2 Gregory F. Lawler and Lester N. Coyle, Lectures on contemporaryprobability, 1999

1 Charles Radin, Miles of tiles, 1999

ISBN 978-0-8218-4772-5

STML/50AMS on the Webwww.ams.org

For additional informationand updates on this book, visit

www.ams.org/bookpages/stml-50

The calculus of variations is used to fi nd func-tions that optimize quantities expressed in terms of integrals. Optimal control theory seeks to fi nd functions that minimize cost integrals for systems described by differential equations.

This book is an introduction to both the clas-sical theory of the calculus of variations and the more modern developments of optimal control theory from the perspective of an applied math-ematician. It focuses on understanding concepts and how to apply them. The range of potential applications is broad: the calculus of variations and optimal control theory have been widely used in numerous ways in biology, criminology, economics, engineering, fi nance, management science, and physics. Applications described in this book include cancer chemotherapy, navigational control, and renewable resource harvesting.

The prerequisites for the book are modest: the standard calculus sequence, a fi rst course on ordinary differential equations, and some facility with the use of mathematical software. It is suitable for an undergraduate or beginning graduate course, or for self study. It provides excellent preparation for more advanced books and courses on the calculus of variations and optimal control theory.

Phot

o co

urte

sy o

f Sa

rah

Mes

tert

on-G

ibbo

ns