Chapter7ApproximationTheoryThe primaryaimof a general approximationis to represent non-arithmeticquantitiesbyarithmeticquantitiessothattheaccuracycanbeascertainedtoadesireddegree. Secondly, wearealsoconcernedwiththeamountofcompu-tationrequiredtoachievethisaccuracy. Thesegeneral notionsareapplicabletofunctions f(x) as well as tofunctionals F(f) (Afunctional is amappingfromthesetof functionstothesetof real orcomplexnumbers). Typical ex-amples of quantitiesto beapproximated are transcendental functions, integralsand derivatives of functions, and solutions of dierential or algebraic equations.Dependinguponthenaturetobeapproximated,dierenttechniquesareusedfordierentproblems.Acomplicatedfunctionf(x)usuallyisapproximatedbyaneasierfunctionoftheform(x; a0, . . . , an)wherea0, . . . , anareparameterstobedeterminedsoastocharacterizethebestapproximationoff. Dependingonthesenseinwhichtheapproximation isrealized,therearethreetypesofapproaches:1. Interpolatoryapproximation: Theparametersaiarechosensothatonaxedprescribedsetofpointsxi, i = 0, 1, . . . , n,wehave(xi; a0, . . . , an) = f(xi) := fi. (7.1)Sometimes, we even further require that, for each i, the rst riderivativesofagree withthoseoffatxi.12 CHAPTER7. APPROXIMATION THEORY2. Least-square approximation: TheparametersaiarechosensoastoMinimize|f(x) (x; a0, . . . , an)|2. (7.2)3. Min-Maxapproximation: theparametersaiarechosen soastominimize|f(x) (x; a0, . . . , an)|. (7.3)Denition7.0.1We say is a linear approximationof fif dependslinearlyontheparametersai,thatis,if(xi; a0, . . . , an) = a00(x) + . . . + an(xn) (7.4)wherei(x)aregivenandxedfunctions.Choosingi(x)=xi,theapproximatingfunctionbecomesapolynomial.Inthiscase, thetheoryforall theabovethreetypesof approximationiswellestablished. Thesolutionfor the min-maxapproximationproblemis the socalledChebyshevpolynomial. Westatewithoutprooftwofundamentalresultsconcerningthersttwotypesofapproximation:Theorem7.0.1Letf(x)beapiecewisecontinuousfunctionovertheinterval[a, b]. Thenforany > 0,thereexistanintegernandnumbersa0, . . . , ansuchthat_baf(x) n

i=0aixi2dx < .Theorem7.0.2(WeierstrassApproximationTheorem)Letf(x)beacontinu-ousfunctionon[a, b]. Forany > 0,thereexistanintegernandapolynomialpn(x)of degreensuchthat maxx[a,b][f(x) pn(x)[ j(xi xj). Sinceall xisaredistinct, wecanuniquelysolve(7.10)fortheunknowns4 CHAPTER7. APPROXIMATION THEORYa0, . . . , an1. Notethat each i(x)isapolynomial of degree n 1and i(xj) =ij, theKroneckerdeltanotation. Therefore, byuniqueness, (7.7)is proved.Letx0 [a, b]andx0 ,= xiforanyi = 1, . . . , n. ConstructtheCn-functionF(x) = f(x) p(x) (f(x0) p(x0))n

i=1(x xi)n

i=1(x0 xi).ItiseasytoseethatF(xi) = 0fori = 0, . . . , n. BytheRollestheorem,thereexistsbetweenx0, . . . , xnsuchthatF(n)() = 0. Itfollowsthatf(n)() (f(x0) p(x0))n!n

i=1(x0xi)= 0.ThusE(x0)=f(x0) p(x0)=n

i=1(x0xi)n!f(n)(). Sincex0isarbitrary, thetheoremisproved. Denition7.1.1Thepolynomial p(x)denedby(7.7)iscalledtheLagrangeinterpolationpolynomial.Remark. Theevaluationof apolynomial p(x)=a0+ a1x + . . . + anxnforx = maybedonebythesocalledHornerscheme:p() = (. . . ((an + an1) + an2) . . . + a1) + a0(7.11)whichonlytakesnmultiplicationsandnadditions.Remark. While theoretically important, Lagranges formula is, in general, notecient for applications. The eciency is especially bad when new interpolatingpoints are added, since then the entire formula is changed. In contrast, Newtonsinterpolationformula, beingequivalenttotheLagrangesformulamathemati-cally,ismuchmoreecient.Remark. Supposepolynomialsare usedtointerpolatethefunctionf(x) =11 + 25x2(7.12)7.2. NEWTONSINTERPOLATIONFORMULA 5in the interval [1, 1] at equally spaced points. Runge (1901) discovered that asthedegreenoftheinterpolatingpolynomialpn(x)tendstoward innity,pn(x)divergesintheintervals.726 . . . [x[ < 1whilepn(x)worksprettywellinthecentral portionoftheinterval.7.2 NewtonsInterpolationFormulaInterpolatingafunctionbyaveryhighdegreepolynomial isnotadvisableinpractice. One reason isbecausewehave seenthedangerofevaluatinghighde-greepolynomials(e.g. theWilkinsonspolynomialandtheRungesfunction).Anotherreasonisbecauselocal interpolation(asopposedtoglobal interpola-tion)usuallyissucientforapproximation.Oneusuallystartstointerpolateafunctionoverasmallersetsof supportpoints. If thisapproximationisnotenough, onethenupdatesthecurrentin-terpolatingpolynomialbyaddinginmoresupportpoints. Unfortunately,eachtimethedatasetischangedLagrangesformulamustbeentirelyrecomputed.Forthisreason, NewtonsinterpolatingformulaispreferredtoLagrangesin-terpolation formula.LetPi0i1...ik(x)represent thek-thdegree polynomialforwhichPi0i1...ik(xij) = f(xij) (7.13)forj= 0, . . . , k.Theorem7.2.1Therecursionformulapi0i1...ik(x) =(x xi0)Pi1...ik(x) (x xik)Pi0...ik1(x)xik xi0(7.14)holds.(pf): Denotetheright-handsideof(7.14)byR(x). ObservethatR(x)isapolynomialofdegree k. Bydenition,itiseasytoseethatR(xij)=f(xij)forallj= 0, . . . , k. Thatis,R(x)interpolatesthesamesetofdataasdoesthepolynomialPi0i1...ik(x). ByTheorem7.1.1theassertion isproved. 6 CHAPTER7. APPROXIMATION THEORYThe dierence Pi0i1...ik(x) Pi0i1...ik1(x) is a k-th degree polynomial whichvanishesatxijforj= 0, . . . , k 1. ThuswemaywritePi0i1...ik(x) = Pi0i1...ik1(x) + fi0...ik(x xi0)(x xi1) . . . (x xik1). (7.15)Theleadingcoecientsfi0...ikcanbedeterminedrecursivelyfromtheformula(7.14),i.e.,fi0...ik=fi1...ik fi0...ik1xik xi0(7.16)where fi1...ikand fi0...ik1are the leading coecients of the polynomials Pi1...ik(x)andPi0...ik1(x),respectively.Remark. Notethattheformula(7.16) startsfromfi0= f(xi0).Remark. ThepolynomialPi0...ik(x)isuniquelydeterminedbythesetofsup-portdata (xij, fij). Thepolynomial isinvarianttoanypermutationof theindicesi0, . . . , ik. Therefore, thedivideddierences(7.16)are invariant toper-mutationoftheindices.Denition7.2.1Let x0, . . . , xkbesupport arguments(but not necessarilyinanyorder)overtheinterval [a, b]. WedenetheNewtonsdivideddierenceasfollows:f[x0] : = f(x0) (7.17)f[x0, x1] : =f[x1] f[x0]x1x0(7.18)f[x0, . . . , xk] : =f[x1, . . . , xk] f[x0, . . . , xk1]xk x0(7.19)It follows that the k-th degree polynomial that interpolates the set of supportdata (xi, fi)[i = 0, . . . , kisgivenbyPx0...xk(x) = f[x0] + f[x0, x1](x x0) (7.20)+ . . . + f[x0, . . . , xk](x x0)(x x1) . . . (x xk1).7.3. OSCULATORYINTERPOLATION 77.3 OsculatoryInterpolationGiven xi, i = 1, . . . kandvaluesa(0)i, . . . , a(ri)iwhereriarenonnegativeinte-gers. WewanttoconstructapolynomialP(x)suchthatP(j)(xi) = a(j)i(7.21)for i = 1, . . . , kand j= 0, . . . , ri. Such apolynomial issaid tobe an osculatoryinterpolatingpolynomialofafunctionfifa(j)i= f(j)(xi) . . .Remark. Thedegree ofP(x)isatmostk

i=1(ri + 1) 1.Theorem7.3.1Given the nodes xi, i = 1, . . . , k and values a(j)i, j= 0, . . . , ri,thereexistsauniquepolynomial satisfying(7.21).(pf): Fori = 1, . . . , k,denoteqi(x) = c(0)i+ c(1)i(x xi) + . . . + c(ri)i(x xi)ri(7.22)P(x) = q1(x) + (x x1)r1+1q2(x) + . . . (7.23)+ (x x1)r1+1(x x2)r2+1. . . (x xk1)rk1+1qk(x).ThenP(x)isofdegree k

i=1(ri +1) 1. Now P(j)(x1) = a(j)1forj= 0, . . . , r1impliesa(0)1=c(0)1, . . . , a(j)1=c(j)1j!. Soq1(x)isdeterminedwithc(j)1=a(j)1j!.Nowwerewrite(7.23)asR(x) :=P(x) q1(x)(x x1)r1+1= q2(x) + (x x2)r2+1q3(x) + . . .NotethatR(j)(x2)are knownforj= 0, . . . , r2sinceP(j)(x2)areknown. Thusall c(j)2, henceq2(x), maybedetermined. Thisprocedurecanbecontinuedtodetermineallqi(x). SupposeQ(x) = P1(x) P2(x)whereP1(x)andP2(x)aretwopolynomials of thetheorem. ThenQ(x) is of degree k

i=1(ri+ 1) 1,andhaszerosatxiwithmultiplicityri + 1. Countingmultiplicities, Q(x)hask

i=1(ri + 1)zeros. ThisispossibleonlyifQ(x) 0.8 CHAPTER7. APPROXIMATION THEORYExamples. (1)Supposek= 1, x1= a, r1= n 1,thenthepolynomial(7.23)becomesP(x)=n1

j=0f(j)(a)(x a)jj!whichistheTaylorspolynomial of f atx = x1.(2)Supposeri= 1foralli = 1, . . . , k. Thatis,supposevaluesoff(xi)andf

(xi)aretobeinterpolated. Thentheresultant(2k 1)-degreepolynomialiscalledtheHermiteinterpolatingpolynomial. Recall thatthe(k 1)-degreepolynomial

i(x) =k

j=1j=ix xjxi xj(7.24)hastheproperty

i(xj) = ij. (7.25)Denehi(x) = [1 2(x xi)

i(xi)]2i(x) (7.26)gi(x) = (x xi)2i(x). (7.27)Notethatbothhi(x)andgi(x)areofdegree2k 1. Furthermore,hi(xj) = ij;gi(xj) = 0; (7.28)h

i(xj) = [1 2(x xi)

i(xi)]2i(x)

i(x) 2

i(xi)2i(x)[x=xj= 0;g

i(xj) = (x xi)2i(x)

i(x) + 2i(x)[x=xj= ij.SotheHermiteinterpolatingpolynomialcanbewrittendownasP(x) =k

i=1f(xi)hi(x) + f

(xi)gi(x)). (7.29)(3)Supposeri= 0foralli. ThenthepolynomialbecomesP(x) = c1 + c2(x x1) + . . . + ck(x x1) . . . (x xk1)whichisexactlytheNewtonsformula.7.4 SplineInterpolationThusfarforagivenfunctionfofaninterval[a, b], theinterpolationhasbeentoconstructapolynomialovertheentireinterval[a, b]. Thereareatleasttwodisadvantagesfortheglobal approximation:7.4. SPLINEINTERPOLATION 91. For better accuracy, we needtosupplymoresupport data. But thenthedegreetheresultantpolynomialgetshigherandsuchapolynomialisdiculttoworkwith.2. Supposef is not smoothenough. Thenthe error estimateof anhighdegreepolynomialisdiculttoestablish. Infact,itisnotclearwhetherornotthattheaccuracywillincreasewithincreasingnumberofsupportdata.Asanalternativewayof approximation, thesplineinterpolationisalocalapproximation ofafunctionf,which,nonetheless,yieldsglobalsmoothcurvesandis less likelytoexhibit thelargeoscillationcharacteristicof high-degreepolynomials. (Ref: APracticalGuidetoSplines,Springer-Verlga,1978,byC.deBoor).Wedemonstratetheideaofcubicsplineasfollows.Denition7.4.1Lettheinterval[a, b]bepartitionedintoa = x1< x2< . . . 0forallx M. By(7.88),itfollowsthatthepolynomialpmustchangesignsatleastntimesin[a, b]. Thatis,pmusthave atleastnzeros. Thiscontradictwiththeassumptionthatpisnotidenticallyzero.Remark. The above theorem asserts only that g is a best approximation when-ever there are atleast n+1 points satisfying (7.87) and (7.88). In general, therecanbemorepointswherethemaximaldeviationisachieved.Example. Supposewewanttoapproximatef(x) =sin3xovertheinterval[0, 2]. It follows from the theorem that if n1 4, then the polynomial g= 0is a best approximation of f. Indeed, in this case the dierence f galternatesbetweenits maximal absolutevalueat sixpoints, whereas thetheoremonlyrequires n +1points. Ontheotherhand,forn 1 = 5we have n +1 = 7, andg= 0nolonger satisesconditions (7.87) and (7.88). In fact,in thiscase g= 0isnotabestapproximation from T5.Remark. Theonlypropertyof Tn1wehaveusedtoestablishTheorem 7.7.1isaweakerformoftheFundamentalTheoremofAlgebra,i.e.,anypolynomialof degree n1 has at most n1 distinct zeros in [a, b]. This property is in factshared byalargerclassoffunctions.Denition7.7.1Supposethat g1, . . . , gn C[a, b] arenlinearlyindependentfunctionssuchthateverynon-trivial elementg U:= spang1, . . . , gnhasatmost n 1distinct zerosin[a,b]. Thenwesaythat UisaHaarspace. Thebasis g1, . . . , gnofaHaarspaceiscalledaChebyshevsystem.7.7. UNIFORMAPPROXIMATION 23Remark. Wehavealreadyseenthat 1, x, x2, . . . , xn1formsaChebyshevsystem. Twootherinterestingexamplesare1. 1, ex, e2x, . . . , e(n1)xoverR.2. 1, sinx, . . . , sinmx, cos x, . . . , cos mxover[0, 2].Wenowstatewithout proof thefamous resultthatTheorem7.7.1is notonly sucient but is also necessary for a polynomial gto a best approximation.ThefollowingtheoremisalsoknownastheAlternationTheorem:Theorem7.7.2Thepolynomial g Tn1isabestapproximationofthefunc-tionf [a, b] if andonlyif thereexist pointsa x1