six, lies, and calculators

8/7/2019 Six, Lies, And Calculators

1/8

Six, Lies, and CalculatorsAuthor(s): R. M. CorlessSource: The American Mathematical Monthly, Vol. 100, No. 4 (Apr., 1993), pp. 344-350Published by: Mathematical Association of AmericaStable URL: http://www.jstor.org/stable/2324956 .

Accessed: 09/04/2011 14:50

Your use of the JSTOR archive indicates your acceptance of JSTOR's Terms and Conditions of Use, available at .

http://www.jstor.org/page/info/about/policies/terms.jsp. JSTOR's Terms and Conditions of Use provides, in part, that unlessyou have obtained prior permission, you may not download an entire issue of a journal or multiple copies of articles, and you

may use content in the JSTOR archive only for your personal, non-commercial use.

Please contact the publisher regarding any further use of this work. Publisher contact information may be obtained at .http://www.jstor.org/action/showPublisher?publisherCode=maa. .

Each copy of any part of a JSTOR transmission must contain the same copyright notice that appears on the screen or printed

page of such transmission.

JSTOR is a not-for-profit service that helps scholars, researchers, and students discover, use, and build upon a wide range of

content in a trusted digital archive. We use information technology and tools to increase productivity and facilitate new forms

of scholarship. For more information about JSTOR, please contact [email protected].

Mathematical Association of America is collaborating with JSTOR to digitize, preserve and extend access to

The American Mathematical Monthly.

http://www.jstor.org
http://www.jstor.org/action/showPublisher?publisherCode=maahttp://www.jstor.org/stable/2324956?origin=JSTOR-pdfhttp://www.jstor.org/page/info/about/policies/terms.jsphttp://www.jstor.org/action/showPublisher?publisherCode=maahttp://www.jstor.org/action/showPublisher?publisherCode=maahttp://www.jstor.org/page/info/about/policies/terms.jsphttp://www.jstor.org/stable/2324956?origin=JSTOR-pdfhttp://www.jstor.org/action/showPublisher?publisherCode=maa


2/8

Six,Lies, and CalculatorsR. M. Corless

Thisaitcle attemptso alleviate omedismalmpressionshatmight e inadver-tentlyeft fter eading vesNievergelt'stherwisexcellentaper 1]. n particu-lar, hepresentrticlehows hat n a certainense, heanswers rovidedy hecalculatorr preciselys useful s theunique xact olution,or veryarge lassofproblems, n when hecalculatorolutions renotat all close to theexactsolution. t is furtherhownthateven the exact answerby itself s usuallyinsufficientor completeunderstandingf theproblemunderconsideration,ndthat ome ort fsensitivit nalysissuch s computingheconditionumber fthe problem)s also required. t is also sho thatunexpected ehaviourf acomputedolution an be extremelyseful edagogically.The behaviourfthe alculatorsbynomeans nique.Allotheromputersndsoftwareackages xhibit imilarlyseful uirks. urther,he behaviours notrestricted o the solution of linear equations, holding true also for functionevaluation,rootfinding,he solutionof differentialquations, and manyotherpractical roblems. here is a strong onnection iththe theor of chaoticdynamics ere,whichmayalso be of interest.SUMMARYOF Y. NIEVERGELTS PAPER.Nievergelt'saper 11reports hathappenswhenyou try o solvethe inear ystem x - b, given elow, n theHP28S.

888445 887112 1887112 885781 0Severalmethodsretried,ndonly neproduceshe xact nswer.Professor ievergeltorectly oints ut that hematrix is ey close to asingularmatrix,ndthus s extremelyll-conditioned,aving conditionumbergreater han 1012, A succinct nd accurate lgebraic efinitionf conditionnumbers givenn 1] and hence snotrepeated ere), ndProfessorievergeltalsopoints ut that hedefinition, eaning,nduseoftheconditionumberrenot sufficientlyell known utsideof the numericalnalysis ommunity,ndshould e taughtmorewidely. agreewhole-heartedlyithhis.nfact his houldbe easy,because there s a strong onnection etweenthecondition umber n the

more eneralomputationalituationnd differentialsffirstear alculus,ndsoin some sense the teaching f the "conditioning"f a problems a simpleapplicationfpart fmainstreamalculus.Of coursethere re many extbooks hichdiscuss ondition umbern thecontex of numerical inearalgebraand finding ootsofpolynomialse.g. [2]),andothershatdiscuss heconditionumberffunctionvaluatione.g. 3]), ndstillothers hat do so fordifferentialquationseg. [41). hispaperattemptsnintroductoryverview.344 SIX,IAS, AND CALCULARS [April


3/8

ERRORS IN MODELLING, ERRORS IN DATA, AND ERRORS IN COMPUTA-TION. The key rationale for having to deal with condition numbers s notnumerical nalysis nd the problemof computational rror.A very arge numberof mathematical roblems re derivedfrom real-world" rigins, nd containbothmodelling rror e.g. neglected erms n the equations) and data or measurementerror.These are unavoidable,while computational rror s avoidable, at least inprinciple, fyou wishto pay the price fordoingexact arithmetic,or xampleusinga symbolicmanipulation ackage such as Maple [5].It is a veryuseful featureof numerical nalysisthat the techniquesused formonitoringhe effects f computational rrors an oftenbe used to monitor heeffects fmodelling rror nd measurement rror.The basic principle s Wilkinson's dea of backward error nalysis: a goodnumericalmethodwill giveyou the exact solution f a nearbyproblem.This verypowerful dea reduces the study f computational rrors o the study f modellingor measurementrrors,whichwehave tostudy nyway.This principle was first lucidated in the contextof the solution of linearsystems f equations and in the contextof polynomialrootfinding.nstead ofrepeating etailsofthese, urgethereaderto examine he technical nd historicaldiscussionsn [2]. Examination fFigure1 at thispoint maymake the basic ideamore clear.

"Real World"Problem

Modeling Process

residual .'Exact Solution loe

"Nearby"Problem _\, ~~~~~~Exactolution { odelSpecifiedProblem AFigure1. Modelingbased on a nearby roblem.

The fundamentaloint s thatwe get nsight rom nowingxact olutions-thatis,from nowing oththequestion nd the answer. f what thecomputer roducesis theexactsolution f ust as gooda model ofthephysical ystems was originallywritten own,we can get ust as much nsight rom he computer olution s wecan from he exactsolution f theoriginallypecified roblem.The idea of allowing he calculatoror computer o changetheproblem, lbeitnotbyverymuch, nsomenorm,supsettingomanymathematicians,ecause oneof the mostpowerful deas in mathematicss that the irrelevant etails of thephysical ontext f a problemcan be ignored.However,mostpeople willhave todeal eventuallywiththe factthat mathematical roblems ncounteredn science1993] SIX, LIES, AND CALCULATORS 345


4/8

and engineeringre usuallymerely ne representativeut of an infinitelass ofmathematicalodels or hephenomenonnquestion,ndfurtherhat he nputdata to themodel willusually e of low accuracy ompared o theprecisionavailable n most omputersr calculators.n such ases,fanaticalbsession ithaccurately olving he specifiedmodel problem s neithernecessarynorappropri-ate,while nalysis f theeffect f perturbationsf the nputdata and/orthemodel s essential.I remark hat ometimeshe method r the programr the computer illchange heproblemy large mount,r by small mountn a nonphysicalay.For example, he computermtay ot preserve he physicalnvariantsf theproblem.n suchcases,we saythat he numericalmethods at fault nd mustsearch or bettermethod.NIEVERGELT'SPROBLEMREVISITED. The object f backwardrrornalysisis toput computationalrrorsnto he context fmodellingrrors. owever, o44real-world"ontext orNievergelt'sxamplewasgiven n [1], o what ollowssspeculative.The elements f A are ntegers,nd A is symmetric.erhaps,hen,A arose sthe normal quationsfrom ome least-squaresroblemwherethe entries fthe possiblyectangular)atrix whereA = BTBwere lso ntegers. e foundthatf

[666 6651B 667 666jthenA = BTB.Theremaywellbe other uchB (ofdifferentimensions),s anexhaustiveearchwasnotcarried ut. The occurrence fthedigit 6" in theentriesfB isone reason or he Six" n the itle, y heway.)It is well-knownnnumericalnalysis ircles hatusing he normal quationscan increase he condition umber rastically,akingheproblemmuchmoresensitivehan tneedbe, andit is certainlyhe case herethatA is muchmoreill-conditionedhanB.On theother and, et(B) = det(A)= I andthe ntriesre ntegers,o itmaybe that heproblem as combinatorialnorigin.nthis ase,nothinghort fexactarithmetics sufficientinceno perturbationn thedata s allowed. he fact hatCramer'sulegot herightnsweror his roblemn 1]was accidental:ramer'srule sunstable,venfor he2 by case 6].Wenote hat here remany ifferentways f doing xact omputationsn computersowadays,ncludingheuse ofsymbolic anipulationrogramsuch s Maple,orperhapsntervalrithmetic.Wenow ooka bitmore loselyt the pparentlyandom igitsf the nswersproducedy hevariousmethods riedn theNievergeltaper.Note hat fterheresults fthedifferentethods eregiven,hequestion asasked:"Whatmightnstructorsnd textbooksell tudents,hocarefullyopydowntheir upercalculator'sesults ithout ver uspectinghat hey recopyingan-domdigits?"If wecomputeheratiox/yfor achof the olutionsroduced,ncludingheexact solution,we get x/y = - 0.9984996258,forall solutions xcept hesingularvaluedecompositionolution. he solutions re not randomdigits!What,then, rethey? learlyhevectorx y] produceds tohigh ccuracymultiplef omeunitvector.Whatunit ector? he answeries na detailedook at the ingularaluedecomposition.hismight e beyondwhat s desired o teach, t least n anintroductoryourse see [7]for omedetails ftheuseof theHP28SatWestern).346 SIX, LIES, AND CALCULATORS [April


5/8

However, t is instructiveo pursue this here. We firstnote that if the singularvalue decompositionsee [8]) of B is B = UYVT whereU and V are orthogonaland I = diag(o1, o2) where a, > a2> 0, then A = Vi2VT and the singularvaluesof A are thesquaresof the singular aluesof B. We can write he solutionto Ax = b in terms f these singular alues asX = U)1 2 l[Vi] + 02 2V[ V121

and we note thato- 1332.00075075033while r2= 0.00075075033.The singularvalues were computedbyMATLAB [9], and separately heckedbyhand and byMaple [5].For small u2, thesolution s clearlydominatedbythe second term, nd hencethe near-constantatioof thecomputed olutions bserved arlier.However, fo2is smallcompared o o1, thenthe matrixs ill-conditioned,nd perturbationsnthedata will change the value of o-2 drastically. he singular-value ecompositionsolutionobtained n [1] is just the first ermabove, on the otherhand,and it isrobustunder perturbations.It is remarkable hatthe solutionproduced bythe HP keysdirectlyorrespondto the above withOr2 1/1601,whileusingGaussian eliminations programmedgives o-2 1/1413, and the eigenvalueapproachgivesu2 1/1332.13,whereastheexactsolutionhas 0U2 1/1332.00075075.MORAL 1: Each method gives the exact solutionfor a slightly erturbedmatrix, llwith mallU2.Each solution s justas good as theexactsolution,n viewof the possibilityf data error.Note particularlyhatthisproblemdoes not goaway fyouuse another oftware ackage,even f tgivestheexact solution.MORAL 2: Explainingthe unexpectedbehaviourof the calculator nvolvesmore seriousmathematicshan perhaps)was anticipated. his seems to be one ofthe betterarguments or using such equipment n a classroomsetting,n thatusingthecalculator an enrich hemathematical ontent f the course.CONDITIONING OF COMPUTATIONAL PROBLEMS. It turnsout that thisbehaviourof computational quipment s quite general.The idea of conditionnumber an be used in any computational roblem, o assess (to first rder)theeffects fperturbationsn themodel or thedata. This is a good idea, even ifyouhave the exact solution o yourproblem, nd provides n excellentmotivation ostudy erturbationheory. givesomeexamplesbelow.Function valuation.n evaluating he function = f(x), we consider he relativeeffect fa smallchangeAx on y.Of coursethis eads to thefirst rderexpression

Ay (xf'(x) Axy f(x) jx

and the expression n brackets s called the conditionnumberof f, and oftendenotedby C (see e.g. [3]). Note that this s just an applicationof the theory fdifferentials,standard opic nmostcalculuscourses.ThisnumberC is an appropriate umber o look at ifx is not zero and f(x) isnot zero, in which case absolute errors, s opposed to relativeerrors, re thequantities f nterest. ere,we lookat only woexamples, hetangent unction,orwhichC = x/(sin(x)cos(x)), and theexponential unction,orwhich C = x. Con-sidertheexponential unction irst.f x = 100,then C = 100. Ifwe know x only1993] SIX, LIES, AND CALCULATORS 347


6/8

to two figures, ow accurately an we know y = exp(x)? The relative rror n y ison the orderof 1. That is, we are notsure of any figures f our answer.Note thatthe precisionused to calculate exp(x) is almost rrelevant,nd in particular oingto "double precision"doesn't help. The difficultys thatdata error s amplified,not that computational rrors re amplified.Now considerr/2 which s 1.57079632680 o the precisionof the calculator.The calculated value of tan(7/2), using the calculator in radians, is- 195948537906.This result shows that someone took fanatical care withtheprogramming f the calculator, because it is correctto all figures giventheassumption hat we wanted the tangentof 1.570796326800000000... and nottan(&/2). In degree mode, taking an(90) gives TAN Error: nfiniteResult,withtheinfinite esultflag et. What s the conditionnumber f tan(x) near x = 7/2?-T 1C ( - /2) - 1 - - x - w/2) + O((x - 7/2)

giving learevidenceof the difficulty.n the ight fthis xtreme ensitivityo dataerror, what purpose is served by getting 12 figures correct fortan(1.570796326800000000.. )? The assumption hatall the unknown igures rezero is simply otalways enable.In some sense,the paper [1] contendedthattheHP28S is not precise enough.The above example howsthat n fact, t is too precise, t leastfor omeproblems.Rootfinding.Wilkinson10]has given superbly lear account of thesensitivityfsomepolynomial ootsto very mallchanges n thepolynomial oefficients.willnot repeat the analysishere of the now-famousWilkinsonpolynomialp(x) =(x - 1)(x - 2) ... (x - 20), except o note thatmanypeople takethesensitivityfan input polynomial oot as a motivation o use higherprecision n rootfinding,perhaps even using arbitrary recision.The sad truth s that if the roots areill-conditioned,hen small changes n the inputdata can drastically hange theroots, rrespectivef the olutionmethod sed.Solution f Differentialquations. As a final xample,we look at somedifferentialequations.These, too, have conditionnumbers, nd not surprisinglye findthatsome ill-conditioned .e.'s are of great interest: ll chaotic problemsare (bydefinition) xponentiallyll-conditioneds initial-value roblems. t turns ut thata system s chaotic if the trajectories re bounded but C exp(At) for someA > 0. In fact,A is precisely helargestLyapunov xponent. hismeans thatforchaoticproblem,data errors i.e. in the initialcondition),modelling rrors, ndcomputationalrrors re amplifiedt an exponential ate.This implies hat goodapproximate olution,good in the sense ofbeingclose to the "exact" solutionofthespecified roblem,s impracticalo compute, incegood accuracy t the end ofthe range of integration equires exponentiallyhigh precisionfor the initialconditionsregardless f the solution echnique-thiswouldhold even fyouknewthe exact solution). However, n the context f modelling rrors, nd giventhepointof view that a good numericalmethodwillgive you the exact solutionof anearbydifferentialquation i.e. just as good a model of theunderlying hysicalproblemas the specified model), much insight an still be gained fromsuchnumericalcomputations, ven thoughthe computed solutions are practicallyguaranteed o be nothingiketheexactsolutions. ee [11]fordetails.

348 SIX, LIES, AND CALCULATORS [April


7/8

So whatdoes this ll have to do with hecalculator? his materials almostcertainlyoo advanced or firstourse n calculus, fter ll. However,here s anexample f a differentialquation hat oes comeup in any irst ear ourse, ndthe HP28S even has a keyfornumericallyolvingt. Of course, his s the d.e.y'(t) = f(t)-that is,compute hedefinitentegral,'Iffr)r.The performancefthiskey s quiteremarkable,nd f heHP28S is used n class, he tudentsike tverymuch ndeed.The followingxampleperhaps venmorehorrifyinghat helinear lgebra xample f [1] at first lance)provides n excellent edagogicalopportunity.Problem: ompute,n thecalculator,J(dr/r).The calculator ill akemore han n hour, lmost egardlessfthe nput rrortolerance, o returnts answer 2.82... -which, of course, s nothingike thecorrectnswerf nfinity.This xamples accessible o studentsery arly n in the urriculum.any reextremelytartled y the peculiarbehavior f the calculator. f course, hecalculatoreturnsn errormessagealbeitnthevery rypticndeasily verlookedform fa negative eportedstimatedrror),o it s notquite s badas itseems.However, hisexampleprovokes verydesirable egreeof skepticismn thestudent, nd gives trongmotivationorthe study f improperntegralsandanalyticntegration).he observationhat he alculators infact ivinghem heexact nswerof1(dr/r)+ fJO(r) or = 10-10 ndKr) = 10-12 provideshemwithwelcome elief nd understandingfwhat hecalculator as done.After herelief omesthe realization hatmost f the area of the figureies next o thesingularity,hich s a valuable edagogical oint.Conclusions. he pointof view of backward rror nalysis,.e. that a goodnumerical ethod ives heexact olution oa nearby roblem,s very elpfulnexplainingheunexpected ehaviourfcalculators nd computersn sensitiveproblems. hispoint fview s not panacea, s someproblem ontextsrecludethenecessaryhangesntheproblem.nparticular,rofessor .Kahancautionsthatbackwardrror nalysiss intendednly s explanationndnot as justifica-tion, ndwarns hat here reproblemsorwhich ackwardrror nalysisails.Asimple xamples compositionffunctions-if e have way fcomputing(x)with oodbackwardrror,nd a way fcomputing(u)with oodbackwardrror,then t snotnecessarilyrue hatwecancomputeg of)(x) with oodbackwarderror. ut, heres noquestionhat ackwardrrornalysisoeshelpwith largeclassofpractical roblems.Thispaper choes he all of 1]for he eachingfthe heoryfconditioningfproblems.ince his smerelynapplicationfdifferentials,r thefirstermn aTaylor eries, r the first erm na perturbationxpansion,his ask s actuallydesirable or everal easons.In essence, hispaperhas shown hat hedifficultiesxhibitedn[1]werenotthe fault f thecalculator,ut rather he fault f theproblem,n some sense.Further,hesedifficultiesctually rovidemotivationorthe student o learnsensitivitynalysisnd theuse ofdifferentials,orms, erturbationeries, ndothermoresophisticated athematicalopicsthan those ust to "solve" theproblem.n view f dataerror,hese opicshould e learnednyway.t hasalsobeen made clear thatthe theory f condition umbers s not restrictedonumericalinear lgebra, ut s in fact fwidepractical tility.hisusefulnesssonly nhanced y he xistencefsupercalculators,ywhichmanyfour tudents

1993] SIX, LIES, AND CALCULATORS 349


8/8

are exposed o very owerfulnd sophisticatednvironmentsor cientificompu-tation.ACKNOWLEDGMENTS. would iketo thankMs. AmandaConnell orprogrammingssistancenfinding . Figure was prepared y Professor eorgeCorliss. oug Moseley nd ProfessorhrisEssex ogetheruggestedhe itle.REFERENCES1. Y. Nievergelt,umericalinearAlgebra n theHP-28 r How to Lie with upercalcualtors,hisMONTHLY,VOl98,no.6, pp. 539-543, une-July991.2. D. Kahaner, . Moler, nd S. Nash,Numerical ethodsndSoftware,rentice-Hall,989.3. G. Dahlquist nd Ake Bj6rk,Numericalethods,rentice-Hall,974.4. U. Ascher, . M. M.Mattheij,ndR. D. Russell, umericalolutionfBoundaryalue roblemsforOrdinaryifferentialquations, rentice-Hall988.5. B. Char,K. 0. Geddes,G. H. Gonnet,M. B. Monagan,ndStephenWatt, heMapleReferenceManual, th d.,WATCOM1988.6. G. W.Stewart,Cramer's ule,"USENET postingosci.math.num-analysis,o. 249,Message-ID:(42829?mimsy.umd.edu),2 Nov91.7. R. M. Corless, . Essex, . J.Sullivan,nd P. A. Rosati,Use oftheHP28S SupercalculatornFirstYear Engineering athematicsourses, roc.7thCanadianConferencen EngineeringEducation, oronto,une 990.8. G. Golub nd C. VanLoan,Matrix omputations,ohns opkins,983.9. The MATLAB Reference uide, he MathWorks,989.10. J.H. Wilkinson,oundingrrorsnAlgebraicrocesses,rentice-Hall,963.11. R. M. Corless,Defect-Controlledumerical ethodsndShadowingorChaotic ifferentialEquations",hysica , vol.60,1992pp.323-334.

DepartmentfAppliedMathematicsUniversityf Western ntarioLondon,Canada N6A 5B9

According o cable reportsfromLondon,the Councilof Trinity ol-lege,Cambridge,as removed rofes-sor BERTRANDRUSSELLfromhis lec-tureship n logic and principles fmathematicsn account fhis havingbeen convicted nderthedefense fthe realmact forpublishing leafletdefendinghe "Conscientious bjec-tor" to service n the British rmy.Professorussell s wellknownn thiscountryhroughismathematicalrit-ings.-AmericanMathematicalonthly23, 1916)p. 317

350 SIX, LIES, AND CALCULATORS [April

six, lies, and calculators

Documents