stepanov rose from.mathematics.to.generic.programming

AboutThiseBook

ePUBisanopen,industry-standardformatforeBooks.However,supportofePUBanditsmanyfeaturesvariesacrossreadingdevicesandapplications.Useyourdeviceorappsettingstocustomizethepresentationtoyourliking.Settingsthatyoucancustomizeoftenincludefont,fontsize,singleordoublecolumn,landscapeorportraitmode,andfiguresthatyoucanclickortaptoenlarge.Foradditionalinformationaboutthesettingsandfeaturesonyourreadingdeviceorapp,visitthedevicemanufacturer’sWebsite.Manytitlesincludeprogrammingcodeorconfigurationexamples.Tooptimizethepresentationofthese

elements,viewtheeBookinsingle-column,landscapemodeandadjustthefontsizetothesmallestsetting.Inadditiontopresentingcodeandconfigurationsinthereflowabletextformat,wehaveincludedimagesofthecodethatmimicthepresentationfoundintheprintbook;therefore,wherethereflowableformatmaycompromisethepresentationofthecodelisting,youwillseea“Clickheretoviewcodeimage”link.Clickthelinktoviewtheprint-fidelitycodeimage.Toreturntothepreviouspageviewed,clicktheBackbuttononyourdeviceorapp.

FromMathematicstoGenericProgramming

AlexanderA.StepanovDanielE.Rose

UpperSaddleRiver,NJ•Boston•Indianapolis•SanFranciscoNewYork•Toronto•Montreal•London•Munich•Paris•Madrid

Capetown•Sydney•Tokyo•Singapore•MexicoCity

Manyofthedesignationsusedbymanufacturersandsellerstodistinguishtheirproductsareclaimedastrademarks.Wherethosedesignationsappearinthisbook,andthepublisherwasawareofatrademarkclaim,thedesignationshavebeenprintedwithinitialcapitallettersorinallcapitals.

Theauthorsandpublisherhavetakencareinthepreparationofthisbook,butmakenoexpressedorimpliedwarrantyofanykindandassumenoresponsibilityforerrorsoromissions.Noliabilityisassumedforincidentalorconsequentialdamagesinconnectionwithorarisingoutoftheuseoftheinformationorprogramscontainedherein.

Forinformationaboutbuyingthistitleinbulkquantities,orforspecialsalesopportunities(whichmayincludeelectronicversions;customcoverdesigns;andcontentparticulartoyourbusiness,traininggoals,marketingfocus,orbrandinginterests),pleasecontactourcorporatesalesdepartmentatcorpsales@pearsoned.comor(800)382-3419.

Forgovernmentsalesinquiries,[email protected].

ForquestionsaboutsalesoutsidetheUnitedStates,[email protected].

VisitusontheWeb:informit.com/aw

LibraryofCongressCataloging-in-PublicationData

Stepanov,AlexanderA.Frommathematicstogenericprogramming/AlexanderA.Stepanov,DanielE.Rose.pagescmIncludesbibliographicalreferencesandindex.ISBN978-0-321-94204-3(pbk.:alk.paper)1.Genericprogramming(Computerscience)—Mathematics.2.Computeralgorithms.I.Rose,DanielE.II.Title.QA76.6245.S742015005.1'1—dc232014034539

Copyright©2015PearsonEducation,Inc.

Allrightsreserved.PrintedintheUnitedStatesofAmerica.Thispublicationisprotectedbycopyright,andpermissionmustbeobtainedfromthepublisherpriortoanyprohibitedreproduction,storageinaretrievalsystem,ortransmissioninanyformorbyanymeans,electronic,mechanical,photocopying,recording,orlikewise.Toobtainpermissiontousematerialfromthiswork,pleasesubmitawrittenrequesttoPearsonEducation,Inc.,PermissionsDepartment,OneLakeStreet,UpperSaddleRiver,NewJersey07458,oryoumayfaxyourrequestto(201)236-3290.

Photocreditsarelistedonpage293.

ISBN-13:978-0-321-94204-3ISBN-10:0-321-94204-3TextprintedintheUnitedStatesonrecycledpaperatRRDonnelleyinCrawfordsville,Indiana.Firstprinting,November2014

mailto:[email protected]



http://informit.com/aw

Contents

AcknowledgmentsAbouttheAuthorsAuthors’Note

1WhatThisBookIsAbout1.1ProgrammingandMathematics1.2AHistoricalPerspective1.3Prerequisites1.4Roadmap

2TheFirstAlgorithm2.1EgyptianMultiplication2.2ImprovingtheAlgorithm2.3ThoughtsontheChapter

3AncientGreekNumberTheory3.1GeometricPropertiesofIntegers3.2SiftingPrimes3.3ImplementingandOptimizingtheCode3.4PerfectNumbers3.5ThePythagoreanProgram3.6AFatalFlawintheProgram3.7ThoughtsontheChapter

4Euclid’sAlgorithm4.1AthensandAlexandria4.2Euclid’sGreatestCommonMeasureAlgorithm4.3AMillenniumwithoutMathematics4.4TheStrangeHistoryofZero4.5RemainderandQuotientAlgorithms4.6SharingtheCode4.7ValidatingtheAlgorithm4.8ThoughtsontheChapter

5TheEmergenceofModernNumberTheory5.1MersennePrimesandFermatPrimes5.2Fermat’sLittleTheorem

5.3Cancellation5.4ProvingFermat’sLittleTheorem5.5Euler’sTheorem5.6ApplyingModularArithmetic5.7ThoughtsontheChapter

6AbstractioninMathematics6.1Groups6.2MonoidsandSemigroups6.3SomeTheoremsaboutGroups6.4SubgroupsandCyclicGroups6.5Lagrange’sTheorem6.6TheoriesandModels6.7ExamplesofCategoricalandNon-categoricalTheories6.8ThoughtsontheChapter

7DerivingaGenericAlgorithm7.1UntanglingAlgorithmRequirements7.2RequirementsonA7.3RequirementsonN7.4NewRequirements7.5TurningMultiplyintoPower7.6GeneralizingtheOperation7.7ComputingFibonacciNumbers7.8ThoughtsontheChapter

8MoreAlgebraicStructures8.1Stevin,Polynomials,andGCD8.2GöttingenandGermanMathematics8.3NoetherandtheBirthofAbstractAlgebra8.4Rings8.5MatrixMultiplicationandSemirings8.6Application:SocialNetworksandShortestPaths8.7EuclideanDomains8.8FieldsandOtherAlgebraicStructures8.9ThoughtsontheChapter

9OrganizingMathematicalKnowledge9.1Proofs

9.2TheFirstTheorem9.3EuclidandtheAxiomaticMethod9.4AlternativestoEuclideanGeometry9.5Hilbert’sFormalistApproach9.6PeanoandHisAxioms9.7BuildingArithmetic9.8ThoughtsontheChapter

10FundamentalProgrammingConcepts10.1AristotleandAbstraction10.2ValuesandTypes10.3Concepts10.4Iterators10.5IteratorCategories,Operations,andTraits10.6Ranges10.7LinearSearch10.8BinarySearch10.9ThoughtsontheChapter

11PermutationAlgorithms11.1PermutationsandTranspositions11.2SwappingRanges11.3Rotation11.4UsingCycles11.5Reverse11.6SpaceComplexity11.7Memory-AdaptiveAlgorithms11.8ThoughtsontheChapter

12ExtensionsofGCD12.1HardwareConstraintsandaMoreEfficientAlgorithm12.2GeneralizingStein’sAlgorithm12.3Bézout’sIdentity12.4ExtendedGCD12.5ApplicationsofGCD12.6ThoughtsontheChapter

13AReal-WorldApplication13.1Cryptology

13.2PrimalityTesting13.3TheMiller-RabinTest13.4TheRSAAlgorithm:HowandWhyItWorks13.5ThoughtsontheChapter

14ConclusionsFurtherReading

ANotation

BCommonProofTechniquesB.1ProofbyContradictionB.2ProofbyInductionB.3ThePigeonholePrinciple

CC++forNon-C++ProgrammersC.1TemplateFunctionsC.2ConceptsC.3DeclarationSyntaxandTypedConstantsC.4FunctionObjectsC.5Preconditions,Postconditions,andAssertionsC.6STLAlgorithmsandDataStructuresC.7IteratorsandRangesC.8TypeAliasesandTypeFunctionswithusinginC++11C.9InitializerListsinC++11C.10LambdaFunctionsinC++11C.11ANoteaboutinline

BibliographyIndex

Acknowledgments

Wewouldliketothankallthepeoplewhocontributedtomakingthisbookareality.OurmanagementatA9.comactivelysupportedthisprojectfromthebeginning.BillStasiorinitiatedthecreationofthecoursethisbookisbasedon,andselectedthetopicfromamongseveraloptionsweoffered.BrianPinkertonnotonlyattendedthecourse,butalsostronglyencouragedourideaofturningthematerialintoabook.WealsowouldliketothankMatMarcus,whocollaboratedwithAlexonasimilarcourseatAdobein2004–2005.TheothermembersoftheFundamentalDataStructuresandAlgorithmsforSearchteamplayed

importantrolesthroughouttheprocess.AnilGangollihelpedshapethecontentofthecourse,RyanErnstprovidedmuchoftheprogramminginfrastructure,andParamjitOberoigaveinvaluablefeedbackduringthewritingstage.Wehaveenjoyedworkingwithallofthemandaregratefulfortheirinput.Wearegratefultooureditors,PeterGordonandGregDoench,andtotheteamofexpertsassembledby

Addison-Wesley,includingmanagingeditorJohnFuller,productioneditorMaryKeselWilson,copyeditorJillHobbs,andcompositor/LaTeXexpertLoriHughesforalltheirworkinturningourroughmanuscriptintoapolishedbook.Finally,we’dliketothankthemanyfriends,familymembers,andcolleagueswhoreadearlierdraftsof

thebookand/orgaveuscomments,corrections,suggestions,advice,orotherhelp:GašperAžman,JohnBanning,CynthiaDwork,HernanEpelman,RyanErnst,AnilGangolli,SusanGruber,JonKalb,RobertLehr,DmitryLeshchiner,TomLondon,MarkManasse,PaulMcJones,NicolasNicolov,GorNishanov,ParamjitOberoi,SeanParent,FernandoPelliccioni,JohnReiser,RobertRose,StefanVargyas,andAdamYoung.Thebookismuchbetterasaresultoftheircontributions.

http://A9.com

AbouttheAuthors

AlexanderA.StepanovstudiedmathematicsatMoscowStateUniversityfrom1967to1972.Hehasbeenprogrammingsince1972:firstintheSovietUnionand,afteremigratingin1977,intheUnitedStates.Hehasprogrammedoperatingsystems,programmingtools,compilers,andlibraries.HisworkonfoundationsofprogramminghasbeensupportedbyGE,PolytechnicUniversity,BellLabs,HP,SGI,Adobe,and,since2009,A9.com,Amazon’ssearchtechnologysubsidiary.In1995hereceivedtheDr.Dobb’sJournalExcellenceinProgrammingAwardforthedesignoftheC++StandardTemplateLibrary.DanielE.RoseisaresearchscientistwhohasheldmanagementpositionsatApple,AltaVista,Xigo,Yahoo,andA9.com.Hisresearchfocusesonallaspectsofsearchtechnology,rangingfromlow-levelalgorithmsforindexcompressiontohuman-computerinteractionissuesinwebsearch.RoseledtheteamatApplethatcreateddesktopsearchfortheMacintosh.HeholdsaPh.D.incognitivescienceandcomputersciencefromUniversityofCalifornia,SanDiego,andaB.A.inphilosophyfromHarvardUniversity.

http://A9.com

http://A9.com

Authors’Note

Theseparationofcomputersciencefrommathematicsgreatlyimpoverishesboth.Thelecturesthatthisbookisbasedonweremyattempttoshowhowthesetwoactivities—anancientonegoingbacktotheverybeginningsofourcivilizationandthemostmodernone—canbebroughttogether.IwasveryfortunatethatmyfriendDanRose,underwhosemanagementourteamwasapplying

principlesofgenericprogrammingtosearchenginedesign,agreedtoconvertmyrathermeanderinglecturesintoacoherentbook.Bothofushopethatourreaderswillenjoytheresultofourcollaboration.

—A.A.S.

Thebookyouareabouttoreadisbasedonnotesfroman“AlgorithmicJourneys”coursetaughtbyAlexStepanovatA9.comduring2012.ButasAlexandIworkedtogethertotransformthematerialintobookform,werealizedthattherewasastrongerstorywecouldtell,onethatcenteredongenericprogramminganditsmathematicalfoundations.Thisledtoamajorreorganizationofthetopics,andremovaloftheentiresectiononsettheoryandlogic,whichdidnotseemtobepartofthesamestory.Atthesametime,weaddedandremoveddetailstocreateamorecoherentreadingexperienceandtomakethematerialmoreaccessibletolessmathematicallyadvancedreaders.WhileAlexcomesfromamathematicalbackground,Idonot.I’vetriedtolearnfrommyownstruggles

tounderstandsomeofthematerialandtousethisexperiencetoidentifyideasthatrequireadditionalexplanation.Ifinsomecaseswedescribesomethinginaslightlydifferentwaythanamathematicianwould,orusingslightlydifferentterminology,orusingmoresimplesteps,thefaultismine.

—D.E.R.

http://A9.com

1.WhatThisBookIsAbout

Itisimpossibletoknowthingsofthisworldunlessyouknowmathematics.

RogerBacon,OpusMajus

Thisbookisaboutprogramming,butitisdifferentfrommostprogrammingbooks.Alongwithalgorithmsandcode,you’llfindmathematicalproofsandhistoricalnotesaboutmathematicaldiscoveriesfromancienttimestothe20thcentury.Morespecifically,thebookisaboutgenericprogramming,anapproachtoprogrammingthatwas

introducedinthe1980sandstartedtobecomepopularfollowingthecreationoftheC++StandardTemplateLibrary(STL)inthe1990s.Wemightdefineitlikethis:

Definition1.1.Genericprogrammingisanapproachtoprogrammingthatfocusesondesigningalgorithmsanddatastructuressothattheyworkinthemostgeneralsettingwithoutlossofefficiency.

Ifyou’veusedSTL,atthispointyoumaybethinking,“Waitaminute,that’sallthereistogenericprogramming?Whataboutallthatstuffabouttemplatesanditeratortraits?”Thosearetoolsthatenablethelanguagetosupportgenericprogramming,andit’simportanttoknowhowtousethemeffectively.Butgenericprogrammingitselfismoreofanattitudetowardprogrammingthanaparticularsetoftools.Webelievethatthisattitude—tryingtowritecodeinthisgeneralway—isonethatallprogrammers

shouldembrace.Thecomponentsofawell-writtengenericprogramareeasiertouseandmodifythanthoseofaprogramwhosedatastructures,algorithms,andinterfaceshardcodeunnecessaryassumptionsaboutaspecificapplication.Makingaprogrammoregenericrendersitsimultaneouslybothsimplerandmorepowerful.

1.1ProgrammingandMathematicsSowheredoesthisgenericprogrammingattitudecomefrom,andhowdoyoulearnit?Itcomesfrommathematics,andespeciallyfromabranchofmathematicscalledabstractalgebra.Tohelpyouunderstandtheapproach,thisbookwillintroduceyoutoalittlebitofabstractalgebra,whichfocusesonhowtoreasonaboutobjectsintermsofabstractpropertiesofoperationsonthem.It’satopicnormallystudiedonlybyuniversitystudentsmajoringinmath,butwebelieveit’scriticalinunderstandinggenericprogramming.Infact,itturnsoutthatmanyofthefundamentalideasinprogrammingcamefrommathematics.Learning

howtheseideascameintobeingandevolvedovertimecanhelpyouthinkaboutsoftwaredesign.Forexample,Euclid’sElements,abookwrittenmorethan2000yearsago,isstilloneofthebestexamplesofhowtobuildupacomplexsystemfromsmall,easilyunderstoodpieces.Althoughtheessenceofgenericprogrammingisabstraction,abstractionsdonotspringintoexistence

fullyformed.Toseehowtomakesomethingmoregeneral,youneedtostartwithsomethingconcrete.Inparticular,youneedtounderstandthespecificsofaparticulardomaintodiscovertherightabstractions.Theabstractionsthatappearinabstractalgebralargelycomefromconcreteresultsinoneoftheoldest

branchesofmathematics,callednumbertheory.Forthisreason,wewillalsointroducesomekeyideasfromnumbertheory,whichdealswithpropertiesofintegers,especiallydivisibility.Thethoughtprocessyou’llgothroughinlearningthismathcanimproveyourprogrammingskills.But

we’llalsoshowhowsomeofthemathematicalresultsthemselvesturnouttobecrucialtosomemodernsoftwareapplications.Inparticular,bytheendofthebookwe’llshowhowsomeoftheseresultsareusedincryptographicprotocolsunderlyingonlineprivacyandonlinecommerce.Thebookwillmovebackandforthbetweentalkingaboutmathandtalkingaboutprogramming.In

particular,we’llinterweaveimportantideasinmathematicswithadiscussionofbothspecificalgorithmsandgeneralprogrammingtechniques.We’llmentionsomealgorithmsonlybriefly,whileotherswillberefinedandgeneralizedthroughoutthebook.Acoupleofchapterswillcontainonlymathematicalmaterial,andacouplewillcontainonlyprogrammingmaterial,butmosthaveamixtureofboth.

1.2AHistoricalPerspectiveWe’vealwaysfoundthatit’seasierandmoreinterestingtolearnsomethingifit’spartofastory.Whatwasgoingonatthetime?Whowerethepeopleinvolved,andhowdidtheycometohavetheseideas?Wasoneperson’sworkanattempttobuildonanother’s—oranattempttorejectwhatcamebefore?Soasweintroducethemathematicalideasinthisbook,we’lltrytotellyouthestoryofthoseideasandofthepeoplewhocameupwiththem.Inmanycases,we’veprovidedshortbiographicalsketchesofthemathematicianswhoarethemaincharactersinourstory.Thesearen’tcomprehensiveencyclopediaentries,butratheranattempttogiveyousomecontextforwhothesepeoplewere.Althoughwetakeahistoricalperspective,thatdoesn’tmeanthatthebookisintendedasahistoryof

mathematicsoreventhatalltheideasarepresentedintheorderinwhichtheywerediscovered.We’lljumparoundinspaceandtimewhennecessary,butwe’lltrytogiveahistoricalcontextforeachoftheideas.

1.3PrerequisitesSincealotofthebookisaboutmathematics,youmaybeconcernedthatyouneedtohavetakenalotofmathclassestounderstandit.Whileyou’llneedtobeabletothinklogically(somethingyoushouldalreadybegoodatasaprogrammer),wedon’tassumeanyspecificmathematicalknowledgebeyondhighschoolalgebraandgeometry.Inacoupleofsections,weshowsomeapplicationsthatusealittlelinearalgebra(vectorsandmatrices),butyoucansafelyskiptheseifyouhaven’tbeenexposedtothebackgroundmaterialbefore.Ifyou’reunfamiliarwithanyofthenotationweuse,it’sexplainedinAppendixA.Animportantpartofmathematicsisbeingabletoprovesomethingformally.Thisbookcontainsquitea

fewproofs.You’llfindthebookeasiertounderstandifyou’vedonesomeproofsbefore,whetherinhighschoolgeometry,inacomputerscienceclassonautomatatheory,orinlogic.We’vedescribedsomeofthecommonprooftechniquesweuse,alongwithexamples,inAppendixB.Weassumethatifyou’rereadingthisbook,you’realreadyaprogrammer.Inparticular,youshouldbe

reasonablyproficientinatypicalimperativeprogramminglanguagelikeC,C++,orJava.OurexampleswilluseC++,butweexpectyou’llbeabletounderstandthemevenifyou’veneverprogrammedinthatlanguagebefore.WhenwemakeuseofaconstructuniquetoC++,weexplainitinAppendixC.IrrespectiveofouruseofC++,webelievethattheprinciplesdiscussedinthisbookapplytoprogrammingingeneral.Manyoftheprogrammingtopicsinthisbookarealsocoveredfromadifferentperspective,andmore

formally,inElementsofProgrammingbyStepanovandMcJones.Readersinterestedinadditionaldepthmayfindthatbooktobeausefulcompaniontothisone.Throughoutthisbook,weoccasionallyreferinterestedreaderstoarelevantsectionofElementsofProgramming.

1.4RoadmapBeforedivingintothedetails,it’susefultoseeabriefoverviewofwherewe’reheaded:

•Chapter2tellsthestoryofanancientalgorithmformultiplication,andhowtoimproveit.•Chapter3looksatsomeearlyobservationsaboutpropertiesofnumbers,andanefficientimplementationofanalgorithmforfindingprimes.•Chapter4introducesanalgorithmforfindingthegreatestcommondivisor(GCD),whichwillbethebasisforsomeofourabstractionsandapplicationslateron.•Chapter5focusesonmathematicalresults,introducingacoupleofimportanttheoremsthatwillplayacriticalrolebytheendofthebook.•Chapter6introducesthemathematicalfieldofabstractalgebra,whichprovidesthecoreideaforgenericprogramming.•Chapter7showshowthesemathematicalideasallowustogeneralizeourmultiplicationalgorithmbeyondsimplearithmetictoavarietyofpracticalprogrammingapplications.•Chapter8introducesnewabstractmathematicalstructures,andshowssomenewapplicationstheyenable.•Chapter9talksaboutaxiomsystems,theories,andmodels,whichareallbuildingblocksofgenericprogramming.•Chapter10introducesconceptsingenericprogramming,andexaminesthesubtletiesofsomeapparentlysimpleprogrammingtasks.•Chapter11continuestheexplorationofsomefundamentalprogrammingtasks,examininghowdifferentpracticalimplementationscanexploittheoreticalknowledgeoftheproblem.•Chapter12looksathowhardwareconstraintscanleadtoanewapproachforanoldalgorithm,andshowsnewapplicationsofGCD.•Chapter13putsthemathematicalandalgorithmicresultstogethertobuildanimportantcryptographyapplication.•Chapter14isasummaryofsomeoftheprincipalideasinthebook.

Thestrandsofprogrammingandmathematicsareinterwoventhroughout,thoughoneortheothermayliehiddenforachapterortwo.Buteverychapterplaysapartintheoverallchainofreasoningthatsummarizestheentirebook:

Tobeagoodprogrammer,youneedtounderstandtheprinciplesofgenericprogramming.Tounderstandtheprinciplesofgenericprogramming,youneedtounderstandabstraction.Tounderstandabstraction,youneedtounderstandthemathematicsonwhichit’sbased.

That’sthestorywe’rehopingtotell.

2.TheFirstAlgorithm

Mosesspeedilylearnedarithmetic,andgeometry....ThisknowledgehederivedfromtheEgyptians,

whostudymathematicsaboveallthings.PhiloofAlexandria,LifeofMoses

Analgorithmisaterminatingsequenceofstepsforaccomplishingacomputationaltask.Algorithmsaresocloselyassociatedwiththenotionofcomputerprogrammingthatmostpeoplewhoknowthetermprobablyassumethattheideaofalgorithmscomesfromcomputerscience.Butalgorithmshavebeenaroundforliterallythousandsofyears.Mathematicsisfullofalgorithms,someofwhichweuseeveryday.Eventhemethodschoolchildrenlearnforlongadditionisanalgorithm.Despiteitslonghistory,thenotionofanalgorithmdidn’talwaysexist;ithadtobeinvented.Whilewe

don’tknowwhenalgorithmswerefirstinvented,wedoknowthatsomealgorithmsexistedinEgyptatleastasfarbackas4000yearsago.

***

AncientEgyptiancivilizationwascenteredontheNileRiver,anditsagriculturedependedontheriver’sfloodstoenrichthesoil.TheproblemwasthateverytimetheNileflooded,allthemarkersshowingtheboundariesofpropertywerewashedaway.TheEgyptiansusedropestomeasuredistances,anddevelopedproceduressotheycouldgobacktotheirwrittenrecordsandreconstructthepropertyboundaries.Aselectgroupofpriestswhohadstudiedthesemathematicaltechniqueswereresponsibleforthistask;theybecameknownas“rope-stretchers.”TheGreekswouldlatercallthemgeometers,meaning“Earth-measurers.”Unfortunately,wehavelittlewrittenrecordoftheEgyptians’mathematicalknowledge.Onlytwo

mathematicaldocumentssurvivedfromthisperiod.TheoneweareconcernedwithiscalledtheRhindMathematicalPapyrus,namedafterthe19th-centuryScottishcollectorwhoboughtitinEgypt.Itisadocumentfromabout1650BCwrittenbyascribenamedAhmes,whichcontainsaseriesofarithmeticandgeometryproblems,togetherwithsometablesforcomputation.Thisscrollcontainsthefirstrecordedalgorithm,atechniqueforfastmultiplication,alongwithasecondoneforfastdivision.Let’sbeginbylookingatthefastmultiplicationalgorithm,which(asweshallseelaterinthebook)isstillanimportantcomputationaltechniquetoday.

2.1EgyptianMultiplicationTheEgyptians’numbersystem,likethatofallancientcivilizations,didnotusepositionalnotationandhadnowaytorepresentzero.Asaresult,multiplicationwasextremelydifficult,andonlyafewtrainedexpertsknewhowtodoit.(ImaginedoingmultiplicationonlargenumbersifyoucouldonlymanipulatesomethinglikeRomannumerals.)Howdowedefinemultiplication?Informally,it’s“addingsomethingtoitselfanumberoftimes.”

Formally,wecandefinemultiplicationbybreakingitintotwocases:multiplyingby1,andmultiplyingbyanumberlargerthan1.Wedefinemultiplicationby1likethis:

Nextwehavethecasewherewewanttocomputeaproductofonemorethingthanwealready

computed.Somereadersmayrecognizethisastheprocessofinduction;we’llusethattechniquemoreformallylateron.

Onewaytomultiplynbyaistoaddinstancesofatogetherntimes.However,thiscouldbeextremelytediousforlargenumbers,sincen–1additionsarerequired.InC++,thealgorithmlookslikethis:Clickheretoviewcodeimage

intmultiply0(intn,inta){

if(n==1)returna;

returnmultiply0(n-1,a)+a;

}

Thetwolinesofcodecorrespondtoequations2.1and2.2.Bothaandnmustbepositive,astheywerefortheancientEgyptians.ThealgorithmdescribedbyAhmes—whichtheancientGreeksknewas“Egyptianmultiplication”and

whichmanymodernauthorsrefertoasthe“RussianPeasantAlgorithm”1—reliesonthefollowinginsight:

1ManycomputerscientistslearnedthisnamefromKnuth’sTheArtofComputerProgramming,whichsaysthattravelersin19th-centuryRussiaobservedpeasantsusingthealgorithm.However,thefirstreferencetothisstorycomesfroma1911bookbySirThomasHeath,whichactuallysays,“Ihavebeentoldthatthereisamethodinusetoday(somesayinRussia,butIhavenotbeenabletoverifythis),....”

Thisoptimizationdependsonthelawofassociativityofaddition:

a+(b+c)=(a+b)+c

Itallowsustocomputea+aonlyonceandreducethenumberofadditions.Theideaistokeephalvingnanddoublinga,constructingasumofpower-of-2multiples.Atthetime,

algorithmswerenotdescribedintermsofvariablessuchasaandn;instead,theauthorwouldgiveanexampleandthensay,“Nowdothesamethingforothernumbers.”Ahmeswasnoexception;hedemonstratedthealgorithmbyshowingthefollowingtableformultiplyingn=41bya=59:

Eachentryontheleftisapowerof2;eachentryontherightistheresultofdoublingthepreviousentry(sinceaddingsomethingtoitselfisrelativelyeasy).Thecheckedvaluescorrespondtothe1-bitsinthebinaryrepresentationof41.Thetablebasicallysaysthat

41×59=(1×59)+(8×59)+(32×59)

whereeachoftheproductsontherightcanbecomputedbydoubling59thecorrectnumberoftimes.Thealgorithmneedstocheckwhethernisevenandodd,sowecaninferthattheEgyptiansknewof

thisdistinction,althoughwedonothavedirectproof.ButancientGreeks,whoclaimedthattheylearned

theirmathematicsfromtheEgyptians,certainlydid.Here’showtheydefined2evenandodd,expressedinmodernnotation:3

2Thedefinitionappearsinthe1st-centuryworkIntroductiontoArithmetic,BookI,ChapterVII,byNicomachusofGerasa.Hewrites,“Theevenisthatwhichcanbedividedintotwoequalpartswithoutaunitinterveninginthemiddle;andtheoddisthatwhichcannotbedividedintotwoequalpartsbecauseoftheaforesaidinterventionofaunit.”3Thearrowsymbol“ ”isread“implies.”SeeAppendixAforasummaryofthemathematicalnotationusedinthisbook.

Wewillalsorelyonthisrequirement:

odd(n) half(n)=half(n–1)

ThisishowweexpresstheEgyptianmultiplicationalgorithminC++:Clickheretoviewcodeimage


if(n==1)returna;

intresult=multiply1(half(n),a+a);

if(odd(n))result=result+a;

returnresult;

}

Wecaneasilyimplementodd(x)bytestingtheleastsignificantbitofx,andhalf(x)byasinglerightshiftofx:Clickheretoviewcodeimage

boolodd(intn){returnn&0x1;}

inthalf(intn){returnn>>1;}

Howmanyadditionsismultiply1goingtodo?Everytimewecallthefunction,we’llneedtodotheadditionindicatedbythe+ina+a.Sincewearehalvingthevalueaswerecurse,we’llinvokethefunctionlogntimes.4Andsomeofthetime,we’llneedtodoanotheradditionindicatedbythe+inresult+a.Sothetotalnumberofadditionswillbe

#+(n)= logn +(ν(n)–1)

4Throughoutthisbook,whenwewrite“log,”wemeanthebase2logarithm,unlessspecifiedotherwise.

wherev(n)isthenumberof1sinthebinaryrepresentationofn(thepopulationcountorpopcount).SowehavereducedanO(n)algorithmtoonethatisO(logn).Isthisalgorithmoptimal?Notalways.Forexample,ifwewanttomultiplyby15,thepreceding

formulawouldgivethisresult:

#+(15)=3+4–1=6

Butwecanactuallymultiplyby15withonly5additions,usingthefollowingprocedure:Clickheretoviewcodeimage

intmultiply_by_15(inta){

intb=(a+a)+a;//b==3*a

intc=b+b;//c==6*a

return(c+c)+b;//12*a+3*a

}

Suchasequenceofadditionsiscalledanadditionchain.Herewehavediscoveredanoptimaladditionchainfor15.Nevertheless,Ahmes’salgorithmisgoodenoughformostpurposes.

Exercise2.1.Findoptimaladditionchainsforn<100.

Atsomepointthereadermayhaveobservedthatsomeofthesecomputationswouldbeevenfasterifwefirstreversedtheorderoftheargumentswhenthefirstisgreaterthanthesecond(forexample,wecouldcompute3×15moreeasilythan15×3).That’strue,andtheEgyptiansknewthis.Butwe’renotgoingtoaddthatoptimizationhere,becauseaswe’llseeinChapter7,we’reeventuallygoingtowanttogeneralizeouralgorithmtocaseswheretheargumentshavedifferenttypesandtheorderoftheargumentsmatters.

2.2ImprovingtheAlgorithmOurmultiply1functionworkswellasfarasthenumberofadditionsisconcerned,butitalsodoeslogn recursivecalls.Sincefunctioncallsareexpensive,wewanttotransformtheprogramtoavoidthisexpense.Oneprinciplewe’regoingtotakeadvantageofisthis:Itisofteneasiertodomoreworkratherthan

less.Specifically,we’regoingtocompute

r+na

whererisarunningresultthataccumulatesthepartialproductsna.Inotherwords,we’regoingtoperformmultiply-accumulateratherthanjustmultiply.Thisprincipleturnsouttobetruenotonlyinprogrammingbutalsoinhardwaredesignandinmathematics,whereit’softeneasiertoproveageneralresultthanaspecificone.Here’sourmultiply-accumulatefunction:

Clickheretoviewcodeimage

intmult_acc0(intr,intn,inta){

if(n==1)returnr+a;

if(odd(n)){

returnmult_acc0(r+a,half(n),a+a);

}else{

returnmult_acc0(r,half(n),a+a);

}

}

Itobeystheinvariant:r+na=r0+n0a0,wherer0,n0anda0aretheinitialvaluesofthosevariables.

Wecanimprovethisfurtherbysimplifyingtherecursion.Noticethatthetworecursivecallsdifferonlyintheirfirstargument.Insteadofhavingtworecursivecallsfortheoddandevencases,we’lljustmodifythevalueofrbeforewerecurse,likethis:Clickheretoviewcodeimage


if(n==1)returnr+a;

if(odd(n))r=r+a;


}

Nowourfunctionistail-recursive—thatis,therecursionoccursonlyinthereturnvalue.We’lltakeadvantageofthisfactshortly.Wemaketwoobservations:•nisrarely1.•Ifniseven,there’snopointcheckingtoseeifit’s1.

Sowecanreducethenumberoftimeswehavetocomparewith1byafactorof2,simplybycheckingforoddnessfirst:Clickheretoviewcodeimage


if(odd(n)){

r=r+a;

if(n==1)returnr;

}


}

Someprogrammersthinkthatcompileroptimizationswilldothesekindsoftransformationsforus,butthat’srarelytrue;theydonottransformonealgorithmintoanother.Whatwehavesofarisprettygood,butwe’reeventuallygoingtowanttoeliminatetherecursionto

avoidthefunctioncalloverhead.Thisiseasierifthefunctionisstrictlytail-recursive.

Definition2.1.Astrictlytail-recursiveprocedureisoneinwhichallthetail-recursivecallsaredonewiththeformalparametersoftheprocedurebeingthecorrespondingarguments.

Again,wecanachievethissimplybyassigningthedesiredvaluestothevariableswe’llbepassingbeforewedotherecursion:Clickheretoviewcodeimage


if(odd(n)){

r=r+a;

if(n==1)returnr;

}

n=half(n);

a=a+a;

returnmult_acc3(r,n,a);

}

Nowitiseasytoconvertthistoaniterativeprogrambyreplacingthetailrecursionwithawhile(true)construct:Clickheretoviewcodeimage


while(true){

if(odd(n)){

r=r+a;

if(n==1)returnr;

}

n=half(n);

a=a+a;

}

}

Withournewlyoptimizedmultiply-accumulatefunction,wecanwriteanewversionofmultiply.Ournewversionwillinvokeourmultiply-accumulatehelperfunction:Clickheretoviewcodeimage


if(n==1)returna;

returnmult_acc4(a,n-1,a);

}

Noticethatweskiponeiterationofmult_acc4bycallingitwithresultalreadysettoa.Thisisprettygood,exceptwhennisapowerof2.Thefirstthingwedoissubtract1,whichmeansthat

mult_acc4willbecalledwithanumberwhosebinaryrepresentationisall1s,theworstcaseforouralgorithm.Sowe’llavoidthisbydoingsomeoftheworkinadvancewhenniseven,halvingit(anddoublinga)untilnbecomesodd:Clickheretoviewcodeimage


while(!odd(n)){

a=a+a;

n=half(n);

}

if(n==1)returna;

returnmult_acc4(a,n-1,a);

}

Butnowwenoticethatwe’remakingmult_acc4dooneunnecessarytestforn=1,becausewe’recallingitwithanevennumber.Sowe’lldoonehalvinganddoublingontheargumentsbeforewecallit,givingusourfinalversion:Clickheretoviewcodeimage


while(!odd(n)){

a=a+a;

n=half(n);

}

if(n==1)returna;

//even(n–1) n–1≠1

returnmult_acc4(a,half(n-1),a+a);

}

RewritingCodeAswehaveseenwithourtransformationsofthemultiplyalgorithm,rewritingcodeisimportant.Noonewritesgoodcodethefirsttime;ittakesmanyiterationstofindthemostefficientorgeneralwaytodosomething.Noprogrammershouldhaveasingle-passmindset.Atsomepointduringtheprocessyoumayhavebeenthinking,“Onemoreoperationisn’tgoing

tomakeabigdifference.”Butitmayturnoutthatyourcodewillbereusedmanytimesformanyyears.(Infact,atemporaryhackoftenbecomesthecodethatlivesthelongest.)Furthermore,thatinexpensiveoperationyou’resavingnowmaybereplacedbyaverycostlyoneinsomefutureversionofthecode.Anotherbenefitofstrivingforefficiencyisthattheprocessforcesyoutounderstandtheproblem

inmoredepth.Atthesametime,thisincreaseddepthofunderstandingleadstomoreefficientimplementations;it’savirtuouscircle.

2.3ThoughtsontheChapterStudentsofelementaryalgebralearnhowtokeeptransformingexpressionsuntiltheycanbesimplified.InoursuccessiveimplementationsoftheEgyptianmultiplicationalgorithm,we’vegonethroughananalogousprocess,rearrangingthecodetomakeitclearerandmoreefficient.Everyprogrammerneedstogetinthehabitoftryingcodetransformationsuntilthefinalformisobtained.We’veseenhowmathematicsemergedinancientEgypt,andhowitgaveusthefirstknownalgorithm.

We’regoingtoreturntothatalgorithmandexpandonitquiteabitlaterinthebook.Butfornowwe’regoingtomoveaheadmorethanathousandyearsandtakealookatsomemathematicaldiscoveriesfromancientGreece.

3.AncientGreekNumberTheory

Pythagoreansappliedthemselvestothestudyofmathematics....Theythoughtthatitsprinciplesmustbetheprinciplesofallexistingthings.

Aristotle,Metaphysics

Inthischapter,we’regoingtolookatsomeoftheproblemsstudiedbyancientGreekmathematicians.Theirworkonpatternsand“shapes”ofnumbersledtothediscoveryofprimenumbersandthebeginningsofafieldofmathematicscallednumbertheory.Theyalsodiscoveredparadoxesthatultimatelyproducedsomemathematicalbreakthroughs.Alongtheway,we’llexamineanancientalgorithmforfindingprimes,andseehowtooptimizeit.

3.1GeometricPropertiesofIntegersPythagoras,theGreekmathematicianandphilosopherwhomostofusknowonlyforhistheorem,wasactuallythepersonwhocameupwiththeideathatunderstandingmathematicsisnecessarytounderstandtheworld.Healsodiscoveredmanyinterestingpropertiesofnumbers;heconsideredthisunderstandingtobeofgreatvalueinitsownright,independentofanypracticalapplication.AccordingtoAristotle’spupilAristoxenus,“Heattachedsupremeimportancetothestudyofarithmetic,whichheadvancedandtookoutoftheregionofcommercialutility.”

Pythagoras(ca.570BC–ca.490BC)

PythagoraswasbornontheGreekislandofSamos,whichwasamajornavalpoweratthetime.Hecamefromaprominentfamily,butchosetopursuewisdomratherthanwealth.AtsomepointinhisyouthhetraveledtoMiletustostudywithThales,thefounderofphilosophy(seeSection9.2),whoadvisedhimtogotoEgyptandlearntheEgyptians’mathematicalsecrets.DuringthetimePythagoraswasstudyingabroad,thePersianempireconqueredEgypt.

PythagorasfollowedthePersianarmyeastwardtoBabylon(inwhatisnowIraq),wherehelearnedBabylonianmathematicsandastronomy.Whilethere,hemayhavemettravelersfromIndia;whatweknowisthathewasexposedtoandbeganespousingideaswetypicallyassociatewithIndianreligions,includingthetransmigrationofsouls,vegetarianism,andasceticism.PriortoPythagoras,theseideaswerecompletelyunknowntotheGreeks.AfterreturningtoGreece,PythagorasstartedasettlementinCroton,aGreekcolonyinsouthern

Italy,wherehegatheredfollowers—bothmenandwomen—whosharedhisideasandfollowedhisasceticlifestyle.Theirliveswerecenteredonthestudyoffourthings:astronomy,geometry,numbertheory,andmusic.Thesefoursubjects,laterknownasthequadrivium,remainedafocusofEuropeaneducationfor2000years.Eachofthesedisciplineswasrelated:themotionofthestarscouldbemappedgeometrically,geometrycouldbegroundedinnumbers,andnumbersgeneratedmusic.Infact,Pythagoraswasthefirsttodiscoverthenumericalstructureoffrequenciesinmusicaloctaves.Hisfollowerssaidthathecould“hearthemusicofthecelestialspheres.”AfterthedeathofPythagoras,thePythagoreansspreadtoseveralotherGreekcoloniesinthe

areaanddevelopedalargebodyofmathematics.However,theykepttheirteachingssecret,somanyoftheirresultsmayhavebeenlost.TheyalsoeliminatedcompetitionwithintheirranksbycreditingalldiscoveriestoPythagorashimself,sowedon’tactuallyknowwhichindividualsdidwhat.AlthoughthePythagoreancommunitiesweregoneafteracoupleofhundredyears,theirwork

remainsinfluential.Aslateasthe17thcentury,Leibniz(oneoftheinventorsofcalculus)describedhimselfasaPythagorean.

Unfortunately,Pythagorasandhisfollowerskepttheirworksecret,sononeoftheirwritingssurvive.However,weknowfromcontemporarieswhatsomeofhisdiscoverieswere.Someofthesecomefromafirst-centurybookcalledIntroductiontoArithmeticbyNicomachusofGerasa.Theseincludedobservationsaboutgeometricpropertiesofnumbers;theyassociatednumberswithparticularshapes.

Triangularnumbers,forexample,whichareformedbystackingrowsrepresentingthefirstnintegers,arethosethatformedthefollowinggeometricpattern:

Oblongnumbersarethosethatlooklikethis:

Itiseasytoseethatthenthoblongnumberisrepresentedbyann×(n+1)rectangle:

n=n(n+1)

It’salsocleargeometricallythateachoblongnumberistwiceitscorrespondingtriangularnumber.Sincewealreadyknowthattriangularnumbersarethesumofthefirstnintegers,wehave

Sothegeometricrepresentationgivesustheformulaforthesumofthefirstnintegers:

AnothergeometricobservationisthatthesequenceofoddnumbersformstheshapeofwhattheGreekscalledgnomons(theGreekwordforacarpenter’ssquare;agnomonisalsothepartofasundialthatcaststheshadow):

Combiningthefirstngnomonscreatesafamiliarshape—asquare:

Thispicturealsogivesusaformulaforthesumofthefirstnoddnumbers:

Exercise3.1.Findageometricproofforthefollowing:takeanytriangularnumber,multiplyitby8,andadd1.Theresultisasquarenumber.(ThisproblemcomesfromPlutarch’sPlatonicQuestions.)

3.2SiftingPrimesPythagoreansalsoobservedthatsomenumberscouldnotbemadeintoanynontrivialrectangularshape(ashapewherebothsidesoftherectanglearegreaterthan1).Thesearewhatwenowcallprimenumbers—numbersthatarenotproductsofsmallernumbers:

2,3,5,7,11,13,...

(“Numbers”fortheGreekswerealwayswholenumbers.)SomeoftheearliestobservationsaboutprimescomefromEuclid.Whileheisusuallyassociatedwithgeometry,severalbooksofEuclid’sElementsactuallydiscusswhatwenowcallnumbertheory.Oneofhisresultsisthistheorem:

Theorem3.1(EuclidVII,32):Anynumberiseitherprimeordivisiblebysomeprime.

Theproof,whichusesatechniquecalled“impossibilityofinfinitedescent,”goeslikethis:11Euclid’sproofofVII,32actuallyreliesonhispropositionVII,31(anycompositenumberisdivisiblebysomeprime),whichcontainsthe

reasoningshownhere.

Proof.ConsideranumberA.Ifitisprime,thenwearedone.Ifitiscomposite(i.e.,nonprime),thenitmustbedivisiblebysomesmallernumberB.IfBisprime,wearedone(becauseifAisdivisiblebyBandBisprime,thenAisdivisiblebyaprime).IfBiscomposite,thenitmustbedivisiblebysomesmallernumberC,andsoon.Eventually,wewillfindaprimeor,asEuclidremarksinhisproofofthepreviousproposition,“aninfinitesequenceofnumberswilldividethenumber,eachofwhichislessthantheother;andthisisimpossible.”

ThisEuclideanprinciplethatanydescendingsequenceofnaturalnumbersterminatesisequivalenttotheinductionaxiomofnaturalnumbers,whichwewillencounterinChapter9.

***

Anotherresult,whichsomeconsiderthemostbeautifultheoreminmathematics,isthefactthatthereareinfinitelymanyprimes:

Theorem3.2(EuclidIX,20):Foranysequenceofprimes{p1,...,pn},thereisaprimepnotinthesequence.

Proof.Considerthenumber

wherepiistheithprimeinthesequence.Becauseofthewayweconstructedq,weknowitisnotdivisiblebyanypi.Theneitherqisprime,inwhichcaseitisitselfaprimenotinthesequence,orqisdivisiblebysomenewprime,whichbydefinitionisnotinthesequence.Therefore,thereareinfinitelymanyprimes.

Oneofthebest-knowntechniquesforfindingprimesistheSieveofEratosthenes.Eratostheneswasa3rd-centuryGreekmathematicianwhoisrememberedinpartforhisamazinglyaccuratemeasurementofthecircumferenceoftheEarth.Metaphorically,theideaofEratosthenes’sieveisto“sift”allthenumberssothatthenonprimes“fallthrough”thesieveandtheprimesremainattheend.Theactualprocedureistostartwithalistofallthecandidatenumbersandthencrossouttheonesknownnottobeprimes(sincetheyaremultiplesofprimesfoundsofar);whateverisleftaretheprimes.TodaytheSieveofEratosthenesisoftenshownstartingwithallpositiveintegersuptoagivennumber,butEratosthenesalreadyknewthatevennumberswerenotprime,sohedidn’tbothertoincludethem.FollowingEratosthenes’convention,we’llalsoincludeonlyoddnumbers,sooursievewillfind

primesgreaterthan2.Eachvalueinthesieveisacandidateprimeuptowhatevervaluewecareabout.Ifwewanttofindprimesuptoamaximumofm=53,oursieveinitiallylookslikethis:

Ineachiteration,wetakethefirstnumber(whichmustbeaprime)andcrossoutallthemultiplesexceptitselfthathavenotpreviouslybeencrossedout.We’llhighlightthenumbersbeingcrossedoutinthecurrentiterationbyboxingthem.Here’swhatthesievelookslikeafterwecrossoutthemultiplesof3:

Nextwecrossoutthemultiplesof5thathavenotyetbeencrossedout:

Andthentheremainingmultiplesof7:

Weneedtorepeatthisprocessuntilwe’vecrossedoutallthemultiplesoffactorslessthanorequalto,wheremisthehighestcandidatewe’reconsidering.

Inourexample,m=53,sowearedone.Allthenumbersthathavenotbeencrossedoutareprimes:

Beforewewriteourimplementationofthealgorithm,we’llmakeafewobservations.Let’sgobackto

whatthesievelookedlikeinthemiddleoftheprocess(say,whenwewerecrossingoutmultiplesof5)andaddsomeinformation—namely,theindex,orpositioninthelist,ofeachcandidatebeingconsidered:

Noticethatwhenwe’reconsideringmultiplesoffactor5,thestepsize—thenumberofentriesbetweentwonumbersbeingcrossedout,suchas25and35—is5,thesameasthefactor.Anotherwaytosaythisisthatthedifferencebetweentheindexesofanytwocandidatesbeingcrossedoutinagiveniterationisthesameasthefactorbeingused.Also,sincethelistofcandidatescontainsonlyoddnumbers,thedifferencebetweentwovaluesistwiceasmuchasthedifferencebetweentwoindexes.Sothedifferencebetweentwonumbersbeingcrossedoutinagiveniteration(e.g.,between25and35)istwicethestepsizeor,equivalently,twicethefactorbeingused.You’llseethatthispatternholdsforallthefactorsweconsideredinourexampleaswell.Finally,weobservethatthefirstnumbercrossedoutineachiterationisthesquareoftheprimefactor

beingused.Thatis,whenwe’recrossingoutmultiplesof5,thefirstonethatwasn’tpreviouslycrossedoutis25.Thisisbecausealltheothermultipleswerealreadyaccountedforbypreviousprimes.

3.3ImplementingandOptimizingtheCodeAtfirstglanceitseemslikeouralgorithmwillneedtomaintaintwoarrays:onecontainingthecandidatenumberswe’resifting—the“values”—andanothercontainingBooleanflagsindicatingwhetherthecorrespondingnumberisstillthereorhasbeencrossedout.However,afterabitofthoughtitbecomesclearthatwedon’tactuallyneedtostorethevaluesatall.Mostofthevalues(namely,allthenonprimes)areneverused.Whenwedoneedavalue,wecancomputeitfromitsposition;weknowthatthefirstvalueis3andthateachsuccessivevalueis2morethanthepreviousone,sotheithvalueis2i+3.SoourimplementationwillstorejusttheBooleanflagsinthesieve,usingtrueforprimeandfalse

forcomposite.Wecalltheprocessof“crossingout”nonprimesmarkingthesieve.Here’safunctionwe’llusetomarkallthenonprimesforagivenfactor:Clickheretoviewcodeimage

template<RandomAccessIteratorI,IntegerN>

voidmark_sieve(Ifirst,Ilast,Nfactor){

//assert(first!=last)

*first=false;

while(last-first>factor){

first=first+factor;

*first=false;

}

}

Weareusingtheconventionof“declaring”ourtemplateargumentswithadescriptionoftheirrequirements.Wewilldiscusstheserequirements,knownasconcepts,indetaillateroninChapter10;fornow,readerscanconsultAppendixCasareference.(IfyouarenotfamiliarwithC++templates,thesearealsoexplainedinthisappendix.)Aswe’llseeshortly,we’llcallthisfunctionwithfirstpointingtotheBooleanvaluecorresponding

tothefirst“uncrossed-out”multipleoffactor,whichaswesawisalwaysfactor’ssquare.Forlast,we’llfollowtheSTLconventionofpassinganiteratorthatpointsjustpastthelastelementinourtable,sothatlast-firstisthenumberofelements.

***

Beforeweseehowtosift,weobservethefollowingsiftinglemmas:•Thesquareofthesmallestprimefactorofacompositenumbercislessthanorequaltoc.•Anycompositenumberlessthanp2issiftedby(i.e.,crossedoutasamultipleof)aprimelessthanp.•Whensiftingbyp,startmarkingatp2.•Ifwewanttosiftnumbersuptom,stopsiftingwhenp2>m.

Wewillusethefollowingformulasinourcomputation:

stepbetweenmultiplekandmultiplek+1ofvalueati:

indexofsquareofvalueati:

Wecannowmakeourfirstattemptatimplementingthesieve:Clickheretoviewcodeimage


voidsift0(Ifirst,Nn){

std::fill(first,first+n,true);

Ni(0);

Nindex_square(3);

while(index_square<n){

//invariant:index_square=2i^2+6i+3

if(first[i]){//ifcandidateisprime

mark_sieve(first+index_square,

first+n,//last

i+i+3);//factor

}

++i;

index_square=2*i*(i+3)+3;

}

}

ItmightseemthatweshouldpassinareferencetoadatastructurecontainingtheBooleansequence,sincethesieveworksonlyifwesiftthewholething.Butbyinsteadpassinganiteratortothebeginningoftherange,togetherwithitslength,wedon’tconstrainwhichkindofdatastructuretouse.ThedatacouldbeinanSTLcontainerorinablockofmemory;wedon’tneedtoknow.Notethatweusethesizeofthetablenratherthanthemaximumvaluetosiftm.

Thevariableindex_squareistheindexofthefirstvaluewewanttomark—thatis,thesquareofthecurrentfactor.Onethingwenoticeisthatwe’recomputingthefactorweusetomarkthesieve(i+i+3)andotherquantities(showninslantedtext)everytimethroughtheloop.Wecanhoistcommonsubexpressionsoutoftheloop;thechangesareshowninbold:Clickheretoviewcodeimage


voidsift1(Ifirst,Nn){

Ilast=first+n;

std::fill(first,last,true);

Ni(0);

Nindex_square(3);

Nfactor(3);


//invariant:index_square=2i^2+6i+3,

//factor=2i+3

if(first[i]){

mark_sieve(first+index_square,last,factor);

}

++i;

factor=i+i+3;

index_square=2*i*(i+3)+3;

}

}

Theastutereaderwillnoticethatthefactorcomputationisactuallyslightlyworsethanbefore,sinceithappenseverytimethroughtheloop,notjustoniterationswhentheiftestistrue.However,weshallseelaterwhymakingfactoraseparatevariablemakessense.Abiggerissueisthatwestillhavearelativelyexpensiveoperation—thecomputationofindex_square,whichinvolvestwomultiplications.Sowewilltakeacuefromcompileroptimizationanduseatechniqueknownasstrengthreduction,whichwasdesignedtoreplacemoreexpensiveoperationslikemultiplicationwithequivalentcodethatuseslessexpensiveoperationslikeaddition.2Ifacompilercandothisautomatically,wecancertainlydoitmanually.

2Whilemultiplicationisnotnecessarilyslowerthanadditiononmodernprocessors,thegeneraltechniquecanstillleadtousingfeweroperations.

Let’slookatthesecomputationsinmoredetail.SupposewereplacedClickheretoviewcodeimage

factor=i+i+3;

index_square=3+2*i*(i+3);

withfactor+=δfactor;index_square+=δindex_square;

whereδfactorandδindex_squarearethedifferencesbetweensuccessive(ithandi+1st)valuesoffactorandindex_square,respectively:

δfactoriseasy;thevariablescancelandwegettheconstant2.Buthowdidwesimplifytheexpressionforδindex_square?Weobservethatbyrearrangingtheterms,wecanexpressitusingsomethingwealreadyhave,factor(i),andsomethingweneedtocomputeanyway,factor(i+1).(Whenyouknowyouneedtocomputemultiplequantities,it’susefultoseeifonecanbecomputedintermsofanother.Thismightallowyoutodolesswork.)Withthesesubstitutions,wegetourfinalversionofsift;again,ourimprovementsareshowninbold:



voidsift(Ifirst,Nn){

Ilast=first+n;

std::fill(first,last,true);

Ni(0);

Nindex_square(3);

Nfactor(3);


//invariant:index_square=2i^2+6i+3,

//factor=2i+3

if(first[i]){

mark_sieve(first+index_square,last,factor);

}

++i;

index_square+=factor;

factor+=N(2);

index_square+=factor;

}

}

Exercise3.2.Timethesieveusingdifferentdatasizes:bit(usingstd::vector<bool>),uint8_t,uint16_t,uint32_t,uint64_t.

Exercise3.3.Usingthesieve,graphthefunction

π(n)=numberofprimes<n

fornupto107andfinditsanalyticapproximation.Wecallprimesthatreadthesamebackwardandforwardpalindromicprimes.Herewe’vehighlighted

theonesupto1000:

Interestingly,therearenopalindromicprimesbetween1000and2000:

Exercise3.4.Aretherepalindromicprimes>1000?Whatisthereasonforthelackofthemintheinterval[1000,2000]?Whathappensifwechangeourbaseto16?Toanarbitraryn?

3.4PerfectNumbersAswesawinSection3.1,theancientGreekswereinterestedinallsortsofpropertiesofnumbers.Oneideatheycameupwithwasthatofaperfectnumber—anumberthatisthesumofitsproperdivisors.3Theyknewoffourperfectnumbers:

3Aproperdivisorofanumbernisadivisorofnotherthannitself.

Perfectnumberswerebelievedtoberelatedtonatureandthestructureoftheuniverse.Forexample,thenumber28wasthenumberofdaysinthelunarcycle.WhattheGreeksreallywantedtoknowwaswhethertherewasawaytopredictotherperfectnumbers.

Theylookedattheprimefactorizationsoftheperfectnumberstheyknew:

andnoticedthefollowingpattern:

Theresultofthisexpressionisperfectwhenthethesecondtermisprime.ItwasEuclidwhopresentedtheproofofthisfactaround300BC.

Theorem3.3(EuclidIX,36):

UsefulFormulasBeforewelookattheproof,itisusefultorememberacoupleofalgebraicformulas.Thefirstisthedifferenceofpowers:

Thisresultcaneasilybederivedusingthesetwoequations:

Theleftandrightsidesof3.2and3.3areequalbythedistributivelaw.Ifwethensubtract3.3from3.2,weget3.1.Thesecondusefulformulaisforthesumofoddpowers:

whichwecanderivebyconvertingthesumtoadifferenceandrelyingonourpreviousresult:

Wecangetawaywiththisbecause–1toanoddpowerisstill–1.Wewillrelyheavilyonbothoftheseformulasintheproofsahead.

Nowweknowthatforn>0

bythedifferenceofpowersformula:

2n–1=(2–1)(2n–1+2n–2+...+2+1)

(orjustthinkofthebinarynumberyougetwhenyouaddpowersof2).

Exercise3.5.UsingEquation3.1,provethatif2n–1isprime,thennisprime.

WearegoingtoproveEuclid’stheoremthewaythegreatGermanmathematicianCarlGaussdid.(We’lllearnmoreaboutGaussinChapter8.)First,wewilluseEquation3.5,substituting2n–1forbothoccurrencesof inEuclid’stheorem,torestatethetheoremlikethis:

If2n–1isprime,then2n–1(2n–1)isperfect.

Next,wedefineσ(n)tobethesumofthedivisorsofn.Iftheprimefactorizationofnis

thenthesetofalldivisorsconsistsofeverypossiblecombinationoftheprimedivisorsraisedtoeverypossiblepoweruptoai.Forexample,24=23·31,sothedivisorsare{20·30,21·30,22·30,20·31,21

·31,22·31,23·31}.Theirsumis20·30+21·30+22·30+20·31+21·31+22·31+23·31=(20+21+22+23)(30+31)Thatis,wecanwritethesumofthedivisorsforanynumbernasaproductofsums:

wherethelastlinereliesonusingthedifferenceofpowersformulatosimplifythenumerator.(Inthisexample,andfortherestofthebook,whenweusepasanintegervariableinourproofs,weassumeit’saprime,unlesswesayotherwise.)

Exercise3.6.Provethatifnandmarecoprime(havenocommonprimefactors),then

σ(nm)=σ(n)σ(m)

(Anotherwaytosaythisisthatσisamultiplicativefunction.)Wenowdefineα(n),thealiquotsum,asfollows:

α(n)=σ(n)–n

Inotherwords,thealiquotsumisthesumofallproperdivisorsofn—allthedivisorsexceptnitself.Nowwe’rereadyfortheproofofTheorem3.3,alsoknownasEuclidIX,36:

If2n–1isprime,then2n–1(2n–1)isperfect.

Proof.Letq=2n–1(2n–1).Weknow2isprime,andthetheorem’sconditionisthat2n–1isprime,so2n–1(2n–1)isalreadyaprimefactorizationoftheform ,wherem=2,p1=2,a1=n–1,p2=2n–1,anda2=1.Usingthesumofdivisorsformula(Equation3.6):

Then

α(q)=σ(q)–q=2q–q=q

Thatis,qisperfect.

WecanthinkofEuclid’stheoremassayingthatifanumberhasacertainform,thenitisperfect.Aninterestingquestioniswhethertheconverseistrue:ifanumberisperfect,doesithavetheform2n–1(2n–1)?Inthe18thcentury,Eulerprovedthatifaperfectnumberiseven,thenithasthisform.Hewasnotabletoprovethemoregeneralresultthateveryperfectnumberisofthatform.Eventoday,thisisanunsolvedproblem;wedon’tknowifanyoddperfectnumbersexist.

Exercise3.7.Provethateveryevenperfectnumberisatriangularnumber.

Exercise3.8.Provethatthesumofthereciprocalsofthedivisorsofaperfectnumberisalways2.Example:

3.5ThePythagoreanProgramForPythagoreans,mathematicswasnotaboutabstractsymbolmanipulation,asitisoftenviewedtoday.Instead,itwasthescienceofnumbersandspace—thetwofundamentalperceptibleaspectsofourreality.Inadditiontotheirfocusonunderstandingfiguratenumbers(suchassquare,oblong,andtriangularnumbers),theybelievedthattherewasdiscretestructuretospace.Theirchallenge,then,wastoprovideawaytogroundgeometryinnumbers—essentially,tohaveaunifiedtheoryofmathematicsbasedonpositiveintegers.Todothis,theycameupwiththeideathatonelinesegmentcouldbe“measured”byanother:

Definition3.1.AsegmentVisameasureofasegmentAifandonlyifAcanberepresentedasafiniteconcatenationofcopiesofV.

Ameasuremustbesmallenoughthatanexactintegralnumberofcopiesproducesthedesiredsegment;thereareno“fractional”measures.Ofcourse,differentmeasuresmightbeusedfordifferentsegments.Ifonewantedtousethesamemeasurefortwosegments,ithadtobeacommonmeasure:

Definition3.2.AsegmentVisacommonmeasureofsegmentsAandBifandonlyifitisameasureof

both.

Foranygivensituation,thePythagoreansbelievedthereisacommonmeasureforalltheobjectsofinterest.Therefore,spacecouldberepresenteddiscretely.

***

Sincetherecouldbemanycommonmeasures,theyalsocameupwiththeideaofthegreatestcommonmeasure:

Definition3.3.AsegmentVisthegreatestcommonmeasureofAandBifitisgreaterthananyothercommonmeasureofAandB.

ThePythagoreansalsorecognizedseveralpropertiesofgreatestcommonmeasure(GCM),whichwerepresentinmodernnotationasfollows:

Usingtheseproperties,theycameupwiththemostimportantprocedureinGreekmathematics—perhapsinallmathematics:awaytocomputethegreatestcommonmeasureoftwosegments.ThecomputationalmachineryoftheGreeksconsistedofrulerandcompassoperationsonlinesegments.UsingC++notation,wemightwritetheprocedurelikethis,usingline_segmentasatype:Clickheretoviewcodeimage

line_segmentgcm(line_segmenta,line_segmentb){

if(a==b)returna;

if(b<a)returngcm(a-b,b);

/*if(a<b)*/returngcm(a,b-a);

}

Thiscodemakesuseofthetrichotomylaw:thefactthatifyouhavetwovaluesaandbofthesametotallyorderedtype,theneithera=b,a<b,ora>b.Let’slookatanexample.What’sgcm(196,42)?

Sowe’redone:gcm(196,42)=14.Ofcourse,whenwesaygcm(196,42),wereallymeanGCMofsegmentswithlength196and42,but

fortheexamplesinthischapter,we’lljustusetheintegersasshorthand.We’regoingtouseversionsofthisalgorithmforthenextfewchapters,soit’simportanttounderstandit

andhaveagoodfeelforhowitworks.Youmaywanttotrycomputingafewmoreexamplesbyhandto

convinceyourself.

3.6AFatalFlawintheProgramGreekmathematiciansfoundthatthewell-orderingprinciple—thefactthatanysetofnaturalnumbershasasmallestelement—providedapowerfulprooftechnique.Toprovethatsomethingdoesnotexist,provethatifitdidexist,asmalleronewouldalsoexist.Usingthislogic,thePythagoreansdiscoveredaproofthatunderminedtheirentireprogram.4We’re

goingtousea19th-centuryreconstructionofthisproofbyGeorgeChrystal.4Wedon’tknowifPythagorashimselfmadethisdiscovery,oroneofhisearlyfollowers.

Theorem3.4:Thereisnosegmentthatcanmeasureboththesideandthediagonalofasquare.

Proof.Assumethecontrary,thattherewereasegmentthatcouldmeasureboththesideandthediagonalofsomesquare.5Letustakethesmallestsuchsquareforthissegment:

5Thisisanexampleofproofbycontradiction.Formoreaboutthisprooftechnique,seeAppendixB.1.

Usingarulerandcompass,6wecanconstructasegment withthesamelengthas ,andthencreateasegmentstartingatFandperpendicularto .

6Althoughmodernreadersmaythinkofarulerasbeingusedtomeasuredistances,forEucliditwasonlyawaytodrawstraightlines.Forthisreason,somepeoplepreferthetermstraightedgetodescribeEuclid’sinstrument.Similarly,althoughamoderncompasscanbefixedtomeasureequaldistances,Euclid’scompasswasusedonlytodrawcircleswithagivenradius;itwascollapsible,soitdidnotpreservedistancesoncelifted.

Nowweconstructtwomoreperpendicularsegments, and :

Weknowthat∠CFE=90°(byconstruction)andthat∠ECF=45°(sinceit’sthesameas∠BCA,whichistheangleformedbythediagonalofasquare,andthereforeishalfof90°).Wealsoknowthatthethreeanglesofatrianglesumto180°.Therefore

∠CEF=180°–∠CFE–∠ECF=180°–90°–45°=45°

So∠CEF=∠ECF,whichmeansCEFisanisoscelestriangle,sothesidesoppositeequalanglesareequal—thatis, .Finally,weaddonemoresegment :

TriangleABFisalsoisosceles,with∠ABF=∠AFB,sinceweconstructed .And∠ABC=∠AFE,sincebothwereconstructedwithperpendiculars.So

Now,weknow ismeasurablesincethat’spartofourpremise,andweknow ismeasurable,sinceit’sthesameas ,whichisalsomeasurablebyourpremise.Sotheirdifference isalsomeasurable.SincewejustshowedthatΔCEFandΔBEFarebothisosceles,

weknow ismeasurable,againbyourpremise,andwe’vejustshownthat ,andtherefore ,ismeasurable.So ismeasurable.Wenowhaveasmallersquarewhoseside anddiagonal arebothmeasurablebyourcommon

unit.Butouroriginalsquarewaschosentobethesmallestforwhichtherelationshipheld—acontradiction.Soouroriginalassumptionwaswrong,andthereisnosegmentthatcanmeasureboththesideandthediagonalofasquare.Ifyoutrytofindone,you’llbeatitforever—ourline_segment_gcm(a,b)procedurewillnotterminate.

Toputitanotherway,theratioofthediagonalandthesideofasquarecannotbeexpressedasarationalnumber(theratiooftwointegers).Todaywewouldsaythatwiththisproof,thePythagoreanshaddiscoveredirrationalnumbers,andspecificallythat isirrational.Thediscoveryofirrationalnumberswasunbelievablyshocking.ItunderminedthePythagoreans’s

entireprogram;itmeantthatgeometrycouldnotbegroundedinnumbers.Sotheydidwhatmanyorganizationsdowhenfacedwithbadnews:theysworeeveryonetosecrecy.Whenoneoftheorderleakedthestory,legendhasitthatthegodspunishedhimbysinkingtheshipcarryinghim,drowningallonboard.

***

Eventually,Pythagoras’followerscameupwithanewstrategy.Iftheycouldn’tunifymathematicsonafoundationofnumbers,theywouldunifyitonafoundationofgeometry.Thiswastheoriginoftheruler-and-compassconstructionsstillusedtodaytoteachgeometry;nonumbersareusedorneeded.Latermathematicianscameupwithanalternate,number-theoreticproofoftheirrationalityof .One

versionwasincludedasproposition117insomeeditionsofBookXofEuclid’sElements.WhiletheproofpredatesEuclid,itwasaddedtoElementssometimeafterthebook’soriginalpublication.Inanycase,itisanimportantproof:

Theorem3.5: isirrational.

Proof.Assume isrational.Thenitcanbeexpressedastheratiooftwointegersmandn,wherem/nisirreducible:

m2iseven,whichmeansthatmisalsoeven,7sowecanwriteitas2timessomenumberu,substitutetheresultintotheprecedingequation,anddoabitmorealgebraicmanipulation:

7Thisiseasilyshown:Theproductoftwooddnumbersisanoddnumber,soifmwerenoteven,m2couldnotbeeven.EuclidprovedthisandmanyotherresultsaboutoddandevennumbersearlierinElements.

n2iseven,whichmeansthatnisalsoeven.Butifmandnarebotheven,thenm/nisnotirreducible—acontradiction.Soourassumptionisfalse;thereisnowaytorepresent astheratiooftwointegers.

3.7ThoughtsontheChapterTheancientGreeks’fascinationwith“shapes”ofnumbersandotherpropertiessuchasprimeandperfectwerethebasisofthemathematicalfieldofnumbertheory.Someofthealgorithmstheyused,suchastheSieveofEratosthenes,arestillveryelegant,thoughwesawhowtoimprovetheirefficiencyfurtherbyusingsomemodernoptimizationtechniques.

***

Towardtheendofthechapter,wesawtwodifferentproofsthat isirrational,onegeometricandonealgebraic.Thefactthatwehavetwocompletelydifferentproofsofthesameresultisgood.Itisactuallyessentialformathematicianstolookformultipleproofsofthesamemathematicalfact,sinceitincreasestheirconfidenceintheresult.Forexample,Gaussspentmuchofhiscareercomingupwithmultipleproofsforoneimportanttheorem,thequadraticreciprocitylaw.ThediscoveryofirrationalnumbersemergedfromthePythagoreans’attemptstorepresentcontinuous

realitywithdiscretenumbers.Whileatfirstglancewemightthinktheywerenaivetobelievethattheycouldaccomplishthis,computerscientistsdothesamethingtoday—weapproximatetherealworldwithbinarynumbers.Infact,thetensionbetweencontinuousanddiscretehasremainedacentralthemeinmathematicsthroughthepresentday,andwillprobablybewithusforever.Butratherthanbeingaproblem,thistensionhasactuallybeenthesourceofgreatprogressandrevolutionaryinsights.

4.Euclid’sAlgorithm

Thewholestructureofnumbertheoryrestsonasinglefoundation,namelythealgorithmforfindingthegreatestcommondivisor.

Dirichlet,LecturesonNumberTheoryInthepreviouschapter,wemetPythagorasandthesecretiveorderhefoundedtostudyastronomy,geometry,numbertheory,andmusic.WhilethePythagoreans’failuretofindacommonmeasureofthesideandthediagonalofasquareendedthedreamofreducingtheworldtonumbers,theideaofagreatestcommonmeasure(GCM)turnedouttobeanimportantoneformathematics—andeventuallyforprogramming.Inthischapter,we’llintroduceanancientalgorithmforGCMthatwe’llbeexploringthroughouttherestofthebook.

4.1AthensandAlexandriaTosetthestageforthediscoveryofthisalgorithm,wenowturntooneofthemostamazingtimesandplacesinhistory:Athensinthe5thcenturyBC.For150yearsfollowingthemiraculousdefeatoftheinvadingPersiansinthebattlesofMarathon,Salamis,andPlatea,Athensbecamethecenterofculture,learning,andscience,layingthefoundationsformuchofWesterncivilization.ItwasinthemiddleofthisperiodofAthenianculturaldominancethatPlatofoundedhisfamous

Academy.AlthoughwethinkofPlatotodayasaphilosopher,thecenteroftheAcademy’sprogramwasthestudyofmathematics.AmongPlato’sdiscoverieswerewhatwenowcallthefivePlatonicsolids—theonlythree-dimensionalshapesinwhicheveryfaceisanidenticalregularpolygon.

Plato(429BC–347BC)

PlatowasbornintooneoftheancientnoblefamiliesofAthens.Asayoungman,hebecameafollowerofSocrates,oneofthefoundersofphilosophy,whotaughtandlearnedbyquestioning,especiallyexaminingone’sownlifeandassumptions.Socrateswasugly,bug-eyed,shabbilydressed,old,andonlyalowlystonemasonbytrade,but

hisideaswererevolutionary.Atthetime,self-proclaimedwisemen(“Sophists”)promisedtoteachtheirstudentstotakeanysideofanargumentandmanipulatethevoters.SocrateschallengedtheSophists,questioningtheirsupposedwisdomandmakingthemlookfoolish.WhiletheSophistschargedsubstantialfeestosharetheirknowledge,Socrates’followersreceivedhistrainingforfree.Tothisday,thetechniqueSocratesintroducedofaskingquestionstogetatthetruthisknown

astheSocraticmethod.AlthoughSocrateswasadmiredbysome,andsomeofhisfollowerswentontobeprominentleaders,hewasgenerallyconsideredtobeanotorioustroublemaker,andwaspubliclyridiculedinAristophanes’famousplayClouds.Eventually,in399BC,Socrateswasputontrialforcorruptingthecity’syouth,andwascondemnedtodeathbypoisoning.PlatowasprofoundlyinfluencedbySocrates,andmostofhisownwritingstaketheformof

dialoguesbetweenSocratesandvariousopponents.PlatowasdevastatedbySocrates’execution,andbythefactthatasocietywoulddestroyitswisestandmostjustmember.HeleftAthensindespair,studyingforawhilewiththepriestsinEgypt,andlaterlearningmathematicsfromthePythagoreansinsouthernItaly.Adecadeorsolater,hereturnedtoAthensandfoundedwhatisessentiallytheworld’sfirstuniversityataplacecalledtheAcademy,namedafteranancientheroAcademus.UnlikethesecretteachingsofthePythagoreans,theAcademy’sprogramofstudywaspublicandavailabletoeveryone:menandwomen,Greeksandbarbarians,freeandslave.ManyofPlato’sdialogues,suchasApology,Phaedo,andSymposium,areasbeautifullywritten

asanypoetry.AlthoughPlato’sbest-knownworkstodayareconcernedwithavarietyofethicalandmetaphysicalissues,mathematicsplayedacentralroleinthecurriculumoftheAcademy.Infact,Platohadtheinscription“Letnooneignorantofgeometryenter”writtenovertheentrance.HegatheredmanytopmathematiciansofthetimetoteachattheAcademyanddevelopauniformcourseofstudy.WhilePlatodidnotleaveusanymathematicalworks,manymathematicalideasarespreadthroughouthisdialogues,andoneofthem,Meno,isdesignedtodemonstratethatmathematicalreasoningisinnate.Onseveraloccasions,PlatotraveledtoSyracusetoinfluencethelocalrulertointroduceajust

society.Hewasunsuccessful;infact,oneofthetripsannoyedtherulersomuchthathearrangedforPlatotobesoldintoslavery.Fortunately,thephilosopherwasquicklyransomedbyanadmirer.ItishardtoexaggeratetheinfluenceofPlatoonEuropeanthought.AstheprominentBritish

philosopherWhiteheadsaid,“ThesafestgeneralcharacterizationoftheEuropeanphilosophicaltraditionisthatitconsistsofaseriesoffootnotestoPlato.”

AthenianculturespreadthroughouttheMediterranean,especiallyduringthereignofAlexandertheGreat.Amonghisachievements,AlexanderfoundedtheEgyptiancityofAlexandria(namedforhimself),whichbecamethenewcenterofresearchandlearning.Morethanathousandscholarsworkedinwhatwewouldnowthinkofasaresearchinstitute,theMouseion—the“InstitutionoftheMuses”—fromwhichwegetourword“museum.”Thescholars’patronsweretheGreekkingsofEgypt,thePtolemys,whopaidtheirsalariesandprovidedfreeroomandboard.PartoftheMouseionwastheLibraryofAlexandria,whichwasgiventhetaskofcollectingalltheworld’sknowledge.Supposedlycontaining500,000scrolls,thelibrarykeptalargestaffofscribesbusycopying,translating,andeditingscrolls.

***

ItwasduringthisperiodthatEuclid,oneofthescholarsattheMouseion,wrotehisElements,oneofthemostimportantbooksinthehistoryofmathematics.Elementsincludesderivationsoffundamentalresultsingeometryandnumbertheory,aswellastheruler-and-compassconstructionsthatstudentsstilllearntoday.

Euclid(flourishedca.300BC)

WeknowverylittleaboutEuclid—notevenexactlywhenhelived.Whatwedoknowisthathetookgeometryveryseriously.AccordingtoastorytoldbythephilosopherProclusDiadochus,oneofPlato’slatersuccessorsasheadoftheAcademy:“Ptolemy[thekingofEgypt]onceaskedEuclidwhethertherewasanyshorterwaytoaknowledgeofgeometrythanbystudyoftheElements,whereuponEuclidansweredthattherewasnoroyalroadtogeometry.”ItisprobablethatEuclidstudiedattheAcademysometimeafterPlato’sdeathandbroughtthemathematicshelearnedtoAlexandria.AlthoughweknowalmostnothingelseaboutEuclid’slife,wedoknowabouthiswork.His

Elementsincorporatedmathematicalresultsandproofsfromseveralexistingtexts.Acarefulreadingrevealssomeoftheselayers;forexample,sinceancienttimestheworkonthetheoryofproportionsinBookVhasgenerallybeenbelievedtobebasedontheworkofPlato’sstudent,Eudoxus.ButitwasEuclidwhowovetheseideastogethertoformacarefullycraftedcoherentstory.InBookI,hestartswiththefundamentaltoolsforgeometricconstructionwithrulerandcompassandendswithwhatwenowcallthePythagoreanTheorem(PropositionI,47).Inthe13thandfinalbook,heshowshowtoconstructthefivePlatonicsolids,andprovesthattheyaretheonlyregularpolyhedra(bodieswhosefacesarecongruent,regularpolygons)thatexist.Euclid’sElementsshowsasenseofpurposeuniqueinthehistoryofmathematics.Each

propositionandproofisthereforareason;nounnecessaryresultsarepresented.Howeverbeautiful,notheoremispresentedunlessitisneededforthelargerstory.Euclidalsoprefersproofsthatconstructasmanyusefulresultsaspossiblewiththefewestruler-and-compassoperations.Hisapproachisreminiscentofamodernprogrammerstrivingforminimalelegantalgorithms.Fromitspublicationaround300BCuntilthebeginningofthe20thcentury,Euclid’sElements

wasusedasthebasisofmathematicaleducation.ItwasnotonlyscientistsandmathematicianswhostudiedEuclid;greatpoliticalleaderssuchasThomasJeffersonandAbrahamLincolnalsoadmiredandstudiedElementsthroughouttheirlives.Evennow,manypeoplebelievethatstudentswouldstillbenefitfromthisapproach.

4.2Euclid’sGreatestCommonMeasureAlgorithmBookXofEuclid’sElementscontainedaconcisetreatmentofincommensurablequantities:

Proposition2.If,whenthelessoftwounequalmagnitudesiscontinuallysubtractedinturnfromthegreater,thatwhichisleftnevermeasurestheonebeforeit,thenthetwomagnitudesareincommensurable.

Essentially,Euclidissayingwhatweobservedearlierinthechapter:ifourprocedureforcomputinggreatestcommonmeasureneverterminates,thenthereisnocommonmeasure.EuclidthengoesontoexplicitlydescribethealgorithmandprovethatitcomputestheGCM.This

diagrammaybeusefulinfollowingtheproof:

Sincethisisthefirstalgorithmterminationproofinhistory,we’reincludingtheentiretext,usingSirThomasHeath’stranslation:

Proposition3.Giventwocommensurablemagnitudes,tofindtheirgreatestcommonmeasure.

Proof.

LetthetwogivencommensurablemagnitudesbeAB,CDofwhichABistheless;thusitisrequiredtofindthegreatestcommonmeasureofAB,CD.NowthemagnitudeABeithermeasuresCDoritdoesnot.Ifthenitmeasuresit—anditmeasuresitselfalso—ABisacommonmeasureofAB,CD.Anditismanifestthatitisalsothegreatest;foragreatermagnitudethanthemagnitudeABwillnot

measureAB.Next,letABnotmeasureCD.Then,ifthelessbecontinuallysubtractedinturnfromthegreater,thatwhichisleftoverwill

sometimemeasuretheonebeforeit,becauseAB,CDarenotincommensurable;[cf.X.2]letAB,measuringED,leaveEClessthanitself,letEC,measuringFB,leaveAFlessthanitself,andletAFmeasureCE.Since,then,AFmeasuresCE,whileCEmeasuresFB,thereforeAFwillalsomeasureFB.Butitmeasuresitselfalso;thereforeAFwillalsomeasurethewholeAB.ButABmeasuresDE;thereforeAFwillalsomeasureED.ButitmeasuresCEalso;thereforeitalsomeasuresthewholeCD.ThereforeAFisacommonmeasureofAB,CD.Isaynextthatitisalsothegreatest.For,ifnot,therewillbesomemagnitudegreaterthanAFwhichwillmeasureAB,CD.LetitbeG.SincethenGmeasuresAB,whileABmeasuresED,thereforeGwillalsomeasureED.ButitmeasuresthewholeCDalso;thereforeGwillalsomeasuretheremainderCE.ButCEmeasuresFB;thereforeGwillalsomeasureFB.ButitmeasuresthewholeABalso,anditwillthereforemeasuretheremainderAF,thegreater

[measuring]theless:whichisimpossible.ThereforenomagnitudegreaterthanAFwillmeasureAB,CD;thereforeAFisthegreatestcommon

measureofAB,CD.ThereforethegreatestcommonmeasureofthetwogivencommensurablemagnitudesAB,CDhas

beenfound.

This“continualsubtraction”approachtoGCMisknownasEuclid’salgorithm(orsometimestheEuclideanalgorithm).It’saniterativeversionofthegcmfunctionwesawinChapter3.Aswedidbefore,wewilluseC++-likenotationtoshowitsimplementation:Clickheretoviewcodeimage

line_segmentgcm0(line_segmenta,line_segmentb){

while(a!=b){

if(b<a)a=a-b;

elseb=b-a;

}

returna;

}

InEuclid’sworld,segmentscannotbezero,sowedonotneedthisasaprecondition.

Exercise4.1.gcm0isinefficientwhenonesegmentismuchlongerthantheother.Comeupwithamoreefficientimplementation.Rememberyoucan’tintroduceoperationsthatcouldn’tbedonebyruler-and-compassconstruction.

Exercise4.2.Provethatifasegmentmeasurestwoothersegments,thenitmeasurestheirgreatestcommonmeasure.

Toworktowardamoreefficientversionofline_segment_gcm,we’llstartbyrearranging,checkingforb<aaslongaswecan:Clickheretoviewcodeimage

line_segmentgcm1(line_segmenta,line_segmentb){

while(a!=b){

while(b<a)a=a-b;

std::swap(a,b);

}

returna;

}

Wecouldavoidaswapinthecasewherea=b,butthatwouldrequireanextratest,andwe’renotquitereadytooptimizethecodeanyway.Instead,weobservethattheinnerwhileloopiscomputingtheremainderofaandb.Let’sfactoroutthatpieceoffunctionality:Clickheretoviewcodeimage

line_segmentsegment_remainder(line_segmenta,line_segmentb){

while(b<a)a=a-b;

returna;

}

Howdoweknowtheloopwillterminate?It’snotasobviousasitmightappear.Forexample,ifourdefinitionofline_segmentincludedthehalflinestartingatapointandcontinuinginfinitelyinonedirection,thecodewouldnotterminate.Therequiredassumptionsareencapsulatedinthefollowing

axiom:AxiomofArchimedes:Foranyquantitiesaandb,thereisanaturalnumbernsuchthata≤nb.Essentially,whatthissaysisthattherearenoinfinitequantities.

***

NowwecanrewriteourGCMfunctionwithacalltosegment_remainder:Clickheretoviewcodeimage

line_segmentgcm(line_segmenta,line_segmentb){

while(a!=b){

a=segment_remainder(a,b);

std::swap(a,b);

}

returna;

}

Sofarwehaverefactoredourcodebutnotimproveditsperformance.Mostoftheworkisdoneinsegment_remainder.Tospeedupthatfunction,wewillusethesameideaasinEgyptianmultiplication—doublingandhalvingourquantities.Thisrequiresknowingsomethingabouttherelationshipofdoubledsegmentstoremainder:

Lemma4.1(RecursiveRemainderLemma):Ifr=segment_remainder(a,2b),then

Suppose,forexample,thatwewantedtofindtheremainderofsomenumberndividedby10.We’lltrytotaketheremainderofndividedby20.Iftheresultislessthan10,we’redone.Iftheresultisbetween11and20,we’lltakeaway10fromtheresultandgettheremainderthatway.Usingthisstrategy,wecanwriteourfasterfunction:


line_segmentfast_segment_remainder(line_segmenta,

line_segmentb){

if(a<=b)returna;

if(a-b<=b)returna-b;

a=fast_segment_remainder(a,b+b);

if(a<=b)returna;

returna-b;

}

It’srecursive,butit’salessintuitiveformofupwardrecursion.Inmostrecursiveprograms,wegodownfromnton–1whenwerecurse;here,we’remakingourargumentbiggereverytime,goingfromnto2n.It’snotobviouswheretheworkisdone,butitworks.Let’slookatanexample.Supposewehaveasegmentaoflength45andasegmentboflength6,and

wewanttofindtheremainderofadividedbyb:

RememberthatsincetheGreekshadnonotionofazero-lengthsegment,theirremainderswereintherange[1,n].Westillhavetheoverheadofrecursion,sowe’lleventuallywanttocomeupwithaniterativesolution,

butwe’llputthatasidefornow.Finally,wecanplugthiscodeintoourGCMfunction,providingasolutiontoExercise4.1:


line_segmentfast_segment_gcm(line_segmenta,line_segmentb){

while(a!=b){

a=fast_segment_remainder(a,b);

std::swap(a,b);

}

returna;

}

Ofcourse,nomatterhowfastitis,thiscodewillstillneverterminateifaandbdonothaveacommonmeasure.

4.3AMillenniumwithoutMathematicsAswehaveseen,ancientGreecewasasourceofseveralcenturiesofastonishingmathematicaldevelopments.Bythe3rdcenturyBC,mathematicswasaflourishingfieldofstudy,withArchimedes(bestknowntodayforastoryaboutdiscoveringtheprincipleofbuoyancyinhisbathtub)itsmostdominantfigure.Unfortunately,theriseofRomanpowerledtoastagnationinWesternmathematicsthatwouldlastforalmost1500years.WhiletheRomansbuiltgreatworksofengineering,theyweregenerallyuninterestedinadvancingthemathematicsthatmadethesestructurespossible.AsthegreatRomanstatesmanCicerosaidinhisTusculanDisputations:

AmongtheGreeksgeometrywasheldinhighesthonor;nothingcouldoutshinemathematics.Butwehavelimitedtheusefulnessofthisarttomeasuringandcalculating.

WhiletherewereGreekmathematiciansworkinginRomantimes,itisaremarkablefactthatthereisnorecordofanyoriginalmathematicaltextwritteninLatin(thelanguageofancientRome)atthattime.TheperiodofhistorythatfollowedwasnotkindtotheformerlygreatsocietiesofEurope.In

Byzantium,theGreek-speakingEasternremnantoftheformerRomanEmpire,mathematicswasstillstudied,butinnovationdeclined.Bythe6thto7thcenturies,scholarsstillreadEuclid,butusuallyjustthefirstbookofElements;Latintranslationsdidn’tevenbothertoincludetheproofs.Bytheendofthefirstmillennium,ifyouwereaEuropeanwhowantedtostudymathematics,youhadtogotocitieslikeCairo,Baghdad,orCordobaintherealmoftheArabs.

OtherMathematicalTraditionsThroughoutancienttimes,mathematicsdevelopedinmanypartsoftheworld.Civilizationdependsonmathematics.Allmajorcivilizationsdevelopednumbersystems,whichwereafundamentalrequirementfortwocriticalcivicactivities:collectingtaxesandcomputingcalendarstodeterminecultivationdates.Furthermore,allmajorcivilizationsdevelopedcommonmathematicalconcepts,suchas

Pythagoreantriples(setsofthreeintegersa,b,cwherea2+b2=c2).WhilesomehavearguedthatthisimpliesacommonNeolithicsourceofmathematicalknowledgethatspreadthroughouttheworld,thereisnoevidenceforthisclaim.Todayitseemsmorelikelythatthisissimplythemathematicalequivalentofconvergentevolutioninbiology,wherethesamecharacteristicsevolveindependentlyinunrelatedspecies.Thefactthatthesesamemathematicalideaswererediscoveredindependentlysuggeststheirfundamentalnature.Manycivilizationsdevelopedimportantmathematicaltraditionsatsomepointintheirhistory.

Forexample,inChina,3rd-centurymathematicianandpoetLiuHuiwroteimportantcommentariesonanearlierbook,NineChaptersontheMathematicalArt,andextendedthework.Amongotherdiscoveries,hedemonstratedthatthevalueofπmustbegreaterthan3,andprovidedseveralgeometrictechniquesforsurveying.InIndia,5th-centurymathematicianandastronomerAryabhatawroteafoundationaltextcalledtheAryabhatiya,whichincludedalgorithmsforcomputingsquareandcuberoots,aswellasgeometrictechniques.IdeasofIndianmathematicswerefurtherdevelopedbyArab,Persian,andJewishscholars,allwritinginArabic,whointurnheavilyinfluencedtherebirthofEuropeanmathematicsintheearly13thcentury.ComputerscienceemergedfromthisreinvigoratedEuropeanmathematics,sothisiswhatwe

arefocusingon.Asprogrammers,weareallheirsofthistradition.

4.4TheStrangeHistoryofZeroThenextdevelopmentinthehistoryofEuclid’salgorithmrequiredsomethingtheGreeksdidn’thave:zero.Youmayhaveheardthatancientsocietieshadnonotionofzero,andthatitwasinventedbyIndiansorArabs,butthisisonlypartiallycorrect.Infact,Babylonianastronomerswereusingzeroasearlyas1500BC,togetherwithapositionalnumbersystem.However,theirnumbersystemusedbase60.Therestoftheirsocietyusedbase10—forexample,incommerce—withouteitherzeroorpositionalnotation.Amazingly,thisstateofaffairspersistedforcenturies.GreekastronomerseventuallylearnedtheBabyloniansystemandusedit(stillinbase60)fortheirtrigonometriccomputations,butagain,thisapproachwasusedonlyforthisoneapplicationandwasunknowntotherestofsociety.(ItwasalsotheseGreekastronomerswhostartedusingtheGreekletteromicron,whichlooksjustlikeourletter“O,”torepresentzero.)Whatisparticularlysurprisingaboutthelackofzerooutsideofastronomyisthatitpersisteddespite

thefactthattheabacuswaswellknownandcommonlyusedforcommerceinnearlyeveryancientcivilization.Abaciconsistofstonesorbeadsarrangedincolumns;thecolumnscorrespondto1s,10s,100s,andsoon,andeachbeadrepresentsoneunitofagivenpowerof10.Inotherwords,ancientsocietiesusedadevicethatrepresentednumbersinbase10positionalnotation,yettherewasnocommonlyusedwrittenrepresentationofzerountil1000yearslater.TheunificationofawrittenformofzerowithadecimalpositionalnotationisduetoearlyIndian

mathematicianssometimearoundthe6thcenturyAD.ItthenspreadtoPersiabetweenthe6thand9thcenturiesAD.Arabscholarslearnedthetechniqueandspreaditacrosstheirempire,fromBaghdadintheeasttoCordobainthewest.ThereisnoevidencethatzerowasknownanywhereinEuropeoutsidethisempire(evenintherestofSpain);300yearswouldpassbeforethisinnovationcrossedfromoneculturetotheother.Thebreakthroughcamein1203whenLeonardoPisano,alsoknownasFibonacci,publishedLiber

Abaci(“TheBookofCalculation”).Inadditiontointroducingzeroandpositionaldecimalnotation,thisastonishingbookdescribedtoEuropeans,forthefirsttime,thestandardalgorithmsfordoingarithmeticthatwearenowtaughtinelementaryschool:longaddition,longsubtraction,longmultiplication,andlongdivision.Withonestroke,LeonardobroughtmathematicsbacktoEurope.

LeonardoPisano(1170–ca.1240)

TheItaliancityofPisa,whichtodayislandlocked,wasamajorportandnavalpowerinthe12thand13thcentury.ItcompetedwithVeniceasthedominanttradingcenterintheMediterranean.ThousandsofPisantraderscrisscrossedthesearoutestotheMiddleEast,Byzantium,NorthAfrica,andSpain,andthePisangovernmentsenttraderepresentativestomajorcitiestoensuretheirsuccess.Oneoftheserepresentatives,GuglielmoBonacci,waspostedtoAlgeria.HedecidedtobringhissonLeonardoalong,adecisionthatchangedthecourseofmathematicalhistory.Leonardolearned“Hindudigits”fromtheArabs,andcontinuedhisstudiesduringbusinesstrips

toEgypt,Syria,Sicily,Greece,andProvence.InhisbookLiberAbaci,hewouldgoontointroducetheirinnovations(includingzero)toEurope.ButLiberAbaciwasnotjustatranslationofotherpeople’swork:itwasafirst-classmathematicaltreatisewithmanyfundamentalnewcontributions.Leonardowouldgoontowriteseveralmorebooksonvariousbranchesofmathematics,includingsomeofthemostimportantmathematicaldevelopmentsincenturies.HecalledhimselfLeonardoPisano(“LeonardothePisan”),althoughsincethe19thcenturyhe

hasusuallybeenknownasFibonacci,anabbreviationoffiliusBonacci(“sonofBonacci”).Leonardo’sfamereachedtheHolyRomanEmperor,FrederickII,agreatintellectualconversant

inmanylanguagesandapatronofscienceandmathematics,whosecourtwasinPalermo,Sicily.FrederickcametoPisaandorganizedachallengetoLeonardobyhiscourtmathematicians.Leonardoperformedwellandimpressedthevisitingdignitaries.Towardtheendofhislife,thecityofPisagavehimasalaryasarewardforhisgreatcontributions.

LeonardoPisano’slaterworkLiberQuadratorum(“TheBookofSquares”),publishedin1225,isprobablythegreatestworkonnumbertheoryinthetimespanbetweenDiophantus1000yearsearlierandthegreatFrenchmathematicianPierredeFermat400yearslater.Hereisoneoftheproblemsfromthebook:

Exercise4.3(easy).Provethat .

WhywasaproblemlikethisdifficultfortheGreeks?Theyhadnoterminatingprocedureforcomputingcuberoots(infact,itwaslaterproventhatnosuchprocessexists).Sofromtheirperspective,the

problemstartsout:“First,executeanonterminatingprocedure....”Leonardo’sinsightwillbefamiliartoanymiddle-schoolalgebrastudent,butitwasrevolutionaryin

the13thcentury.Basically,whathesaidwas,“EventhoughIdon’tknowhowtocompute ,I’lljustpretendIdoandassignitanarbitrarysymbol.”Here’sanotherexampleofthekindofproblemLeonardosolved:

Exercise4.4.ProvethefollowingpropositionfromLiberQuadratorum:Foranyoddsquarenumberx,thereisanevensquarenumbery,suchthatx+yisasquarenumber.

Exercise4.5(hard).ProvethefollowingpropositionfromLiberQuadratorum:Ifxandyarebothsumsoftwosquares,thensoistheirproductxy.(ThisisanimportantresultthatFermatbuildson.)

4.5RemainderandQuotientAlgorithmsOncezerowaswidelyusedinmathematics,itactuallytookcenturieslongerbeforeitoccurredtoanyonethatasegmentcouldhavezerolength—specifically,thesegment .Zero-lengthsegmentsforceustorethinkourGCMandremainderprocedures,becauseArchimedes’

axiomnolongerholds—wecanaddazero-lengthsegmentforever,andwe’llneverexceedanonzerosegment.Sowe’llallowthefirstargumentatobezero,butweneedapreconditiontoensurethatthesecondargumentbisnotzero.Havingzeroalsoletsusshiftourremainderstotherange[0,n–1],whichwillbecrucialformodulararithmeticandotherdevelopments.Here’sthecode:Clickheretoviewcodeimage

line_segmentfast_segment_remainder1(line_segmenta,

line_segmentb){

//precondition:b!=0

if(a<b)returna;

if(a-b<b)returna-b;

a=fast_segment_remainder1(a,b+b);

if(a<b)returna;

returna-b;

}

Theonlythingwe’vechangedaretheconditions;everywhereweusedtosaya<=b,wenowchecka<b.Let’sseeifwecangetridoftherecursion.Everytimewerecursedown,wedoubleb,sointhe

iterativeversion,we’dliketoprecomputethemaximumamountofdoublingwe’llneed.Wecandefineafunctionthatfindsthefirstrepeateddoublingofbthatexceedsthedifferencea–b:Clickheretoviewcodeimage

line_segmentlargest_doubling(line_segmenta,line_segmentb){

//precondition:b!=0

while(a-b>=b)b=b+b;

returnb;

}

Nowweneedouriterativefunctiontodothesamecomputationthathappensonthewayoutoftherecursion.Eachtimeitreturns,thevalueofbhasthevalueithadbeforethemostrecentrecursivecall(i.e.,themostrecentdoubling).Sotosimulatethis,theiterativeversionneedstorepeatedly“undouble”thevalue,whichitwilldobycallingafunctionhalf.Remember,we’restill“computing”withrulerandcompass.Fortunately,thereisaEuclideanprocedurefor“halving”asegment,1sowecanuseahalffunction.Nowwecanwriteaniterativeversionofremainder:

line_segmentremainder(line_segmenta,line_segmentb){

//precondition:b!=0

if(a<b)returna;

line_segmentc=largest_doubling(a,b);

a=a-c;

while(c!=b){

c=half(c);

if(c<=a)a=a-c;

}

returna;

}

1Drawacirclewiththecenteratoneendofthesegmentandradiusequaltothesegment;repeatfortheotherend.Userulertoconnectthetwopointswherethecirclesintersect.Theresultinglinewillbisecttheoriginalsegment.

Thefirstpartofthefunction,whichfindsthelargestdoublingvalue,doeswhatthe“downward”recursiondoes,whilethelastpartdoeswhathappensonthewaybackupoutoftherecursivecalls.Let’slookagainatourexampleoffindingtheremainderof45dividedby6,thistimewiththenewremainderfunction:

Noticethatthesuccessivevaluesofcintheiterativeimplementationarethesameasthevaluesofbfollowingeachrecursivecallintherecursiveimplementation.Also,comparethistothetraceofourearlierversionofthealgorithmattheendofSection4.2.Observehowtheresultsofthefirstpart(c=24anda=21)herearethesameastheinnermostrecursionintheoldexample.Thisisanextremelyefficientalgorithm,nearlyasfastasthehardwareimplementedremainder

operationinmodernprocessors.

***

Whatifwewantedtocomputequotientinsteadofremainder?Itturnsoutthatthecodeisalmostthesame.Allweneedareacoupleofminormodifications,showninbold:Clickheretoviewcodeimage

integerquotient(line_segmenta,line_segmentb){

//Precondition:b>0

if(a<b)returninteger(0);

integern(1);

a=a-c;

while(c!=b){

c=half(c);n=n+n;

if(c<=a){a=a-c;n=n+1;}

}

returnn;

}

Quotientisthenumberoftimesonelinesegmentfitsintoanother,soweusethetypeintegertorepresentthiscount.Basically,wearegoingtocountmultiplesofb.Ifa<b,thenwedon’thaveanymultiplesofbandwereturn0.Butifa≥b,weinitializethecounterto1,thendoubleiteachtimewehalvec,addingonemoremultipleforeachiterationwhenitfits.Again,let’sworkthroughanexample.Thistime,insteadoffindingtheremainderof45dividedby6,we’llfindthequotientof45dividedby6.

Essentially,thisistheEgyptianmultiplicationalgorithminreverse.AndAhmesknewit:aprimitivevariantofthisalgorithm,knowntotheGreeksasEgyptiandivision,appearsintheRhindpapyrus.

4.6SharingtheCodeSincethemajorityofthecodeissharedbetweenquotientandremainder,itwouldmakemuchmoresensetocombinethemintoasinglefunctionthatreturnsbothvalues;thecomplexityofthecombinedfunctionisthesameaseitherindividualfunction.NotethatC++11allowsustouseinitializerlistsyntax{x,y}toconstructthepairthatthefunctionreturns:Clickheretoviewcodeimage

std::pair<integer,line_segment>

quotient_remainder(line_segmenta,line_segmentb){

//Precondition:b>0

if(a<b)return{integer(0),a};

integern(1);

a=a-c;

while(c!=b){

c=half(c);n=n+n;

if(c<=a){a=a-c;n=n+1;}

}

return{n,a};

}

Infact,anyquotientorremainderfunctiondoesnearlyalltheworkoftheother.

ProgrammingPrinciple:TheLawofUsefulReturnOurquotient_remainderfunctionillustratesanimportantprogrammingprinciple,whichwecallthelawofusefulreturn:

Ifyou’vealreadydonetheworktogetsomeusefulresult,don’tthrowitaway.Returnittothecaller.

Thismayallowthecallertogetsomeextraworkdone“forfree”(asinthequotient_remaindercase)ortoreturndatathatcanbeusedinfutureinvocationsofthefunction.Unfortunately,thisprincipleisnotalwaysfollowed.Forexample,theCandC++programming

languageshaveseparatequotientandremainderoperators;thereisnowayforaprogrammertogetbothresultswithonecall—despitethefactthatmanyprocessorshaveaninstructionthatreturnsboth.

Mostcomputingarchitectures,whetherruler-and-compassormodernCPUs,provideaneasywaytocomputehalf;forus,it’sjusta1-bitrightshift.However,ifyoushouldhappentobeworkingwithanarchitecturethatdoesn’tsupportthisfunctionality,thereisaremarkableversionoftheremainderfunctiondevelopedbyRobertFloydandDonaldKnuththatdoesnotrequirehalving.It’sbasedontheideaoftheFibonaccisequence—anotherofLeonardoPisano’sinventions,whichwewilldiscussmoreinChapter7.Insteadofthenextnumberbeingdoublethepreviousone,we’llmakethenextnumberbethesumofthetwopreviousones:2


line_segmentremainder_fibonacci(line_segmenta,line_segmentb){

//Precondition:b>0

if(a<b)returna;

line_segmentc=b;

do{

line_segmenttmp=c;c=b+c;b=tmp;

}while(a>=c);

do{

if(a>=b)a=a-b;

line_segmenttmp=c-b;c=b;b=tmp;

}while(b<c);

returna;

}

2Notethatthissequencestartsatb,sothevalueswillnotbethesameasthetraditionalFibonaccisequence.

Thefirstloopisequivalenttocomputinglargest_doublinginourpreviousalgorithm.Thesecondloopcorrespondstothe“halving”partofthecode.Butinsteadofhalving,weusesubtractiontogetbacktheearliernumberintheFibonaccisequence.Thisworksbecausewealwayskeeponepreviousvalue

aroundinatemporaryvariable.

Exercise4.6.Tracetheremainder_fibonaccialgorithmasitcomputestheremainderof45and6,inthewaywetracedtheremainderalgorithminSection4.5.

Exercise4.7.Designquotient_fibonacciandquotient_remainder_fibonacci.

Nowthatwehaveanefficientimplementationoftheremainderfunction,wecanreturntoouroriginalproblem,thegreatestcommonmeasure.Usingournewremainderfunctionfromp.54,wecanrewriteEuclid’salgorithmlikethis:Clickheretoviewcodeimage

line_segmentgcm_remainder(line_segmenta,line_segmentb){

while(b!=line_segment(0)){

a=remainder(a,b);

std::swap(a,b);

}

returna;

}

Sincewenowallowremaindertoreturnzero,theterminationconditionforthemainloopiswhenb(thepreviousiteration’sremainder)iszero,insteadofcomparingaandbaswedidoriginally.Wewillusethisalgorithmforthenextfewchapters,leavingitsstructureintactbutexploringhowit

appliestodifferenttypes.Wewillleaveourruler-and-compassconstructionsbehindandimplementthealgorithmwithadigitalcomputerinmind.Forexample,forintegers,theequivalentfunctionisthegreatestcommondivisor(GCD):Clickheretoviewcodeimage

integergcd(integera,integerb){

while(b!=integer(0)){

a=a%b;

std::swap(a,b);

}

returna;

}

Thecodeisidentical,exceptthatwehavereplacedline_segmentwithintegerandusedthemodulusoperator%tocomputetheremainder.Sincecomputershaveinstructionstocomputetheintegerremainder(asinvokedbytheC++modulusoperator),it’sbettertousethemthantorelyondoublingandhalving.

4.7ValidatingtheAlgorithmHowdoweknowthattheintegerGCDalgorithmworks?Weneedtoshowtwothings:first,thatthealgorithmterminates,andsecond,thatitcomputestheGCD.Toprovethatthealgorithmterminates,werelyonthefactthat

0≤(amodb)<b

Therefore,ineachiteration,theremaindergetssmaller.Sinceanydecreasingsequenceofpositiveintegersisfinite,thealgorithmmustterminate.ToprovethatthealgorithmcomputestheGCD,westartbyobservingthatineachiteration,the

algorithmcomputesaremainderofaandb,whichbydefinitionis

r=a–bq

whereqistheintegerquotientofadividedbyb.Sincegcd(a,b)bydefinitiondividesaandalsodividesb(andthereforebq),itmustalsodivider.Wecanrewritetheremainderequationasfollows:

a=bq+r

Bythesamereasoning,sincegcd(b,r)bydefinitiondividesb(andthereforebq),andalsodividesr,itmustalsodividea.Sincepairs(a,b)and(b,r)havethesamecommondivisors,theyhavethesamegreatestcommondivisor.Thereforewehaveshownthat

Ateachiteration,thealgorithmreplacesgcd(a,b)withgcd(b,r)bytakingtheremainderandswappingthearguments.Hereisthelistofremainders,startingwitha0andb0,theinitialargumentstothefunction:

Usingthedefinitionofremainder,werewritethesequencecomputedbythealgorithmlikethis:

WhatEquation4.1guaranteesisthattheGCDstaysthesameeachtime.Inotherwords:

gcd(a0,b0)=gcd(b0,r1)=gcd(r1,r2)=···=gcd(rn–1,rn)

Butweknowthattheremainderofrn-1andrnis0,becausethat’swhattriggerstheterminationofthealgorithm.Andgcd(x,0)=x.So

gcd(a0,b0)=···=gcd(rn–1,rn)=gcd(rn,0)=rn

whichisthevaluereturnedbythealgorithm.Therefore,thealgorithmcomputestheGCDofitsoriginalarguments.

4.8ThoughtsontheChapterWe’veseenhowanancientalgorithmforcomputingthegreatestcommonmeasureoftwolinesegmentscouldbeturnedintoamodernfunctionforcomputingtheGCDofintegers.We’velookedatvariantsofthealgorithmandseenitsrelationshiptofunctionsforfindingthequotientandtheremainder.DoestheGCDalgorithmworkforotherthingsbesideslinesegmentsandintegers?Inotherwords,isthereawaytomakethealgorithmmoregeneral?We’llcomebacktothatquestionlaterinthebook.

5.TheEmergenceofModernNumberTheory

Mathematicianshavetriedinvaintothisdaytodiscoversomeorderinthesequenceofprimenumbers,

andwehavereasontobelievethatitisamysteryintowhichthehumanmindwillneverpenetrate.

LeonhardEulerInthepreviouschapter,wesawhowthefledgingfieldofnumbertheory,whichhadfascinatedtheancientGreeks,wasrevivedinmedievalEuropeafteralongperiodofdormancy.Butnumbertheoryinitsmodernsensereallyemergedafewhundredyearslater,in17th-centuryFrance.Forthischapter,wearegoingtoputprogrammingasideforabitandlearnsomeoftheresultsdiscoveredby17th-centuryFrenchmathematicians,whichwe’lluseforsomeimportantcomputerapplicationslateron.

5.1MersennePrimesandFermatPrimesMathematiciansoftheRenaissancerekindledtheancientGreeks’fascinationwithprimenumbers.Theywonderedwhethertherewerecertainpredictablepatternsofprimes.Theywereparticularlyinterestedinprimesoftheform2n-1,since(aswesawinSection3.4)thiswasthesourceofperfectnumbers.Mathematiciansfromthe15thtothe18thcenturies,liketheGreeksbeforethem,feltthatthesenumbershadspecialimportance.Lettersofthe17th-centurymathematiciansFermat,Mersenne,andDescartescontainmanyreferencestoperfectnumbers,aswellasacloselyrelatedconcept,amicablenumbers.Inthe18thcentury,thegreatmathematicianLeonhardEulerstillfoundthesubjecttobeofprimaryimportance.AswesawinChapter3,theGreeksknewthattheycouldgenerateperfectnumbersfromprimesofthe

form2n-1.Theyknewthatnumbersofthatformareprimeforn=2,3,5,and7,andpossibly13.In1536,HudalricusRegiusshowedthattheexpressionwasnonprimeforn=11,byfinding

211-1=2047=23x89

PietroCataldiaddedseveralmorevaluesofntothelistin1603—17,19,(23),(29),31,and(37)—buthalfofthese(showninparentheses)wereincorrect.PierredeFermatdiscoveredthat

Inhis1644bookCogitataPhysicoMathematica,theFrenchmathematicianMersennestatesthatforn≤257,2n-1isprimeifandonlyif

n=2,3,5,7,13,17,19,31,(67),117,(257)

Twoofthesewerewrong(showninparentheses),andhemissed89and107.BecauseofMersenne’sconjecture,primesofthisformbecameknownasMersenneprimes.Westilldonotknowwhetherthereisaninfinitenumberofthem,buteventodayMersennenumbersarestillusedtosearchforlargeprimes.

MarinMersenne(1588–1648)

Startingaround1624,whenCardinalRichelieubecamechiefminister,Francebegantoriseasamilitary,political,cultural,andscientificpower.WhilescholarsattraditionaluniversitiesstillinterpretedAristotle’sancientworks,philosopherslikeDescartesinFrance,workingoutsideoftheuniversitysystem,wererevolutionizingthewaypeoplethoughtabouttheworld.Richelieucreatedthefirstmodernstate,withacarefullyorganizedcentralbureaucracyandmilitary,andevenofficialcontroloftheFrenchlanguage.By1660,FrancehadbecometheundisputedleaderofEurope,andFrenchbecamethedominantlanguagefordiplomatsandaristocratsinmostWesterncountriesforthenext250years.ItwasduringthisperiodthatMarinMersenne,aFrenchpolymathandamemberofthestrictreligiousorderofMinims,hadanenormousimpactonscience.AlthougheducatedbytheintellectualJesuitsandanaccomplishedclassicalscholarandmathematician,MersennechosetheextremeasceticismoftheMinims,whoheldnoproperty(evenincommon),ateastrictvegandiet,anddranknoalcohol.Mersenne’shumilityextendedtohisprofessionallife;whileotherscientistsproclaimedtheirownimportance,hiscausewastohelpothersdisseminatetheirworkandlearnabouteachother’sresults.Mersennedidsomeimportantworkonthetheoryofsoundandotherareas,buthisgreatestcontributionwasthecreationofasharedscientificcommunity.Scientificjournalsdidnotyetexist,butMersenneservedasa“virtualscientificjournal,”byexchangingletterswithfriendsandinformingthemofeachother’sresults.Mersenne’sfriendsincludedpeoplelikeGalileo,Huygens,Torricelli,Descartes,Fermat,andPascal.Infact,MersennearrangedforpublicationofGalileo’sworkinProtestantHolland,despiteGalileo’scondemnationbytheCatholicChurch.LaterinMersenne’slife,scholarswouldmeettogetherinhiscellinakindofinformalweeklyconference.Whenhisletterswerepublishedafterhisdeath,theywereinessencetheworld’sfirstscientificproceedings.

InalettertoMersenneinJune1640,Fermatwrotethathisfactorizationof237-1dependsonthefollowingthreediscoveries:

1.Ifnisnotaprime,2n-1isnotaprime.2.Ifnisaprime,2n-2isamultipleof2n.

3.Ifnisaprime,andpisaprimedivisorof2n-1,thenp-1isamultipleofn.We’lllookattheproofofdiscovery1inabit,butfornowlet’sassumethatallthreestatementsaretrue.Fermatreasonedthatif237-1isnotprime,itmusthaveaprimefactorp,whichmustbeodd.By

observation3,p-1isamultipleof37,whichisequivalenttosayingthat

p=37u+1

Also,sincepisodd,p-1=37umustbeeven,soumustbeeven.Thatmeanswecanexpressuas2v,whichgivesus:

p=74v+1

Fermatthereforenarrowedthefactoringtaskfromtryingallpossiblenumberstojustthoseprimesproducedbythisformula.Testingtheseinsequence:Whataboutv=1?No,75isnotaprime.Whataboutv=2?No,149isprime,butisnotadivisorof237-1.Whataboutv=3?Yes!223isprime,andisadivisorof237-1.So237-1isnotprime.

***

Nowlet’slookatFermat’sproofofdiscovery1,whichwestateinitscontrapositive1form.1Anyimplicationp qislogicallyequivalenttoitscontrapositive,whichistheexpression¬q ¬p.See“Implicationandthe

Contrapositive”inAppendixAformoredetails.

Theorem5.1:If2n-1isprime,thennisprime.

Proof.Supposenisnotprime.Thentheremustbefactorsuandvsuchthat

n=uv,u>1,v>1

Then

wherethelaststepusesEquation3.1,thedifferenceofpowersformula.Sinceu>1,weknowthatbothofthefollowingaretrue:

So5.1showsthatwehavefactored2n-1intotwonumberseachgreaterthan1.Butthiscontradictstheconditionofthetheoremisthat2n-1isprime.Sotheinitialassumptioninourproofmustbefalse,andnmustbeprime.

Asfordiscoveries2and3,Fermatneversharedtheproofs.InalettertohisfriendFrenicle,Fermat

wrotethat“hewouldsend[theproof]ifhedidnotfearbeingtoolong.”Weshallreturntothemsoon.

PierredeFermat(1601–1665)

PierredeFermatwasalawyerandprovincialmagistratefromToulouseinthesouthofFrance.HewasaRenaissancemaninthetraditionofMontaigne,interestedinavarietyofsubjectsincludingclassicalliterature,andfluentinLatinandGreek.Thelastofthegreatamateurmathematicians,FermatbecameinterestedinnumbertheoryafterreadingBachet’stranslationofDiophantus’ancientGreektextArithmetic.Althoughhemadeenormouscontributionstomathematics,heneverpersonallyinteractedwithothermathematicians.Infact,MersennerepeatedlyinvitedhimtovisitParis,butasfarasweknow,Fermatneverwent.Fermatoftenboastedofhisresultswhilekeepinghismethodssecret.Hewouldoftensaythathehadaproofofsomething,yetcomeupwithanexcusenottoprovideit.Whenhedidpublicizearesult,hewouldtrytodivulgeaslittleaspossibleabouthowhedidit.Duringhislife,Fermatneverpublishedhiswork,althoughhecorrespondedwithMersenneandothersthroughletters.AfterFermat’sdeath,hissonpublishedtheeditionofDiophantuswithFermat’smarginalnotes.Thesenotescontainedmanytheorems,whichweregraduallyconfirmedbyothermathematiciansinlateryears.Thelasttobesolved—whichbecameknownasFermat’sLastTheorem—wasthestatementthatan+bn=cnhasnosolutionsinpositiveintegersforn>2.Itwasfinallyprovedin1994byAndrewWiles.Fermatnotoriouslywrote“theproofistoolargetofitinthemargin”nexttohisstatementoftheLastTheorem.Asmentionedearlier,thiswashiscommonpattern;heoftengavesimilarexcusestoavoidsharinghisproofs.Althoughallbutoneofhisconjectureshavebeenconfirmed,someoftheproofsaresocomplexandlengthythatlatermathematicianssuchasGausshavebeenskepticalthatFermatactuallydiscoveredthem.Inadditiontohisworkonnumbertheory,Fermatmademajorcontributionstootherareasofmathematics.Heinventedanalyticgeometry—thestudyofequationsofcurves—beforeDescartes,butdescribedtheworkinanunpublishedmanuscript.Healsoco-inventedprobabilitytheoryinthecourseofalengthycorrespondencewithBlaisePascal.

Fermatmadealotofconjecturesforwhichheleftnoproofs,buteveryonehassincebeenproventrue

exceptone:

2n+1isprime n=2i

(Thedouble-arrowsymbolisread“ifandonlyif”;seeAppendixAfordetails.)Sincethen,numbersofthisform(22

i+1)havebeenknownasFermatprimes.It’seasytoproveapartofhisconjecture:

Theorem5.2:2n+1isprime n=2i.

Proof.Supposen≠2i.Thenoneofn’sfactorsmustbeodd,sowecanexpressthatfactoras2q+1.Thisis>1,sowecanexpressnas

n=m(2q+1)

Substitutingm(2q+1)fornandthenusingtheformulaforsumofoddpowers(Equation3.4),wefactor2n+1:

Butfactoring2n+1contradictsthepremiseoftheconjecture;itcan’thavenontrivialfactorsifit’sprime.Soourinitialassumptionintheproofisfalse,andn=2i.

Whataboutotherprimesoftheform22i+1?Fermatstatesthat3,5,17,257,65537,4294967297,and

18446744073709551617areprime,andsoarealltherestofthisform.Unfortunately,hewaswrongabouttwoofhisexamples—onlythefirstfiveareprime—andabouthisconjecture.In1732,Eulershowedthat

232+1=4294967297=641×6700417

Infact,weknowthatfor5≤i≤32,thenumbersarecomposite.ArethereanymoreFermatprimesbesidesthesefive?Asofthiswriting,nooneknows.

5.2Fermat’sLittleTheoremWenowcometooneofthemostimportantresultsinnumbertheory.

Theorem5.3(Fermat’sLittleTheorem):

Ifpisprime,ap–1–1isdivisiblebypforany0<a<p.

Fermatclaimedtohaveaproofofthetheoremin1640,butdidnotpublishit.Leibnizdiscoveredaproofsometimebetween1676and1680,butdidnotpublishiteither.Finally,Eulerpublishedtwodifferentproofsin1742and1750.Wewillprovethetheoremhere,butfirstweneedtoderiveseveralotherresults.Whilethesemayatfirstseemtobeunrelated,wewillseeshortlyhowtheycometogether.

LeonhardEuler(1707–1783)

LeonhardEuler(pronounced“OILer”)wasborninawell-educatedmiddle-classfamilyinSwitzerland.Atalentedandwell-roundedstudentwithanamazingmemory,hestudiedwithJohannBernoulli,thegreatestmathematicianofthetimeandafriendofEuler’sfather.(BernoullihimselfwasastudentofLeibniz,theco-inventorofcalculus.)Formostofthe18thcentury,CzarPetertheGreatandhissuccessorsconductedaperiodofdramaticreformthat“Europeanized”Russiansocietyandculture.OneoftheresultsofthesereformswasthecreationoftheImperialAcademyofSciencesinSt.Petersburg,whichrecruitedEuropeanscholars.Itwastherein1727thatEuler,atage20,gotajobdoingmathematicalresearch.Within10years,hisresultsinmathematics,mechanics,andevenshipbuildingestablishedhisreputationasoneofthetopscientificmindsinEurope.BythetimeFredericktheGreatrecruitedhimtocometoBerlinin1741,Eulerwasaninternationalsuperstar.Atthetime,kingsandqueensconsideredassociatingwithscientistsandotherintellectualstobeanimportantwaytoincreasetheirownstatus.ThroughoutEuler’syearsinBerlin,theFrenchandRussianroyalcourtscompetedtowoohimaway.Eventually,in1766,hereturnedtoSt.Petersburg,wherehespenttherestofhiscareer.Euler’scontributionstomathematics(andphysics)wereenormous.Heworkedinmanyareas;hefoundedmoderngraphtheoryandmadefundamentaldiscoveriesinnumbertheory.However,hisgreatestachievementwasthedevelopmentofmodernanalysis—calculusanddifferentialequations—fromtheindividualtechniquesinventedbyNewtonandLeibniztoasystematicdiscipline.Histhreebooksoncalculus(IntroductiontoAnalysisoftheInfinite,DifferentialCalculus,andIntegralCalculus)werethedefinitivetextsfornearlyacenturyandstilldeservecarefulstudy.Eulerwrotethefirstbookonpopularscience,LetterstoaGermanPrincess,inwhichheexplainstheNewtonianviewoftheworldtoalayperson.Healsowroteanelementaryalgebratextbookintendedfornon-mathematicians,whichisstillinprint.Eulerwassoprolificthatafterhisdeath,theRussianAcademyofSciencestook60yearstopublishalltheadditionalworkhehadsubmitted.Hewasgenerallyregardedasthegreatestmathematicianintheworldinhistime,andafter200years,westillshareLaplace’sviewthat“heisthemasterofusall.”

OurfirststepisanotherpropositionfromEuclid:

Theorem5.4(EuclidVII,30):Theproductoftwointegerssmallerthanaprimepisnotdivisiblebyp.

(Anotherwaytosaythisisthatifpisprimeandaandbaresmallerthanp,thenabisnotdivisiblebyp.)Ifsomenumberxisdivisiblebysomeothernumbery,thenxisamultipleofy:x=my.Ifxisnotdivisiblebyy,thendividingxbyyleavesaremainderr:x=my+r.Sowecanrestatethepropositionlikethis:

pisprime∧0<a,b<p ab=mp+r∧0<r<p

Proof.Assumethecontrary,thatabisamultipleofp.Thenforagivena,letbbethesmallestintegersuchthatab=mp.Thensincepisprime,weknowdividingpbybleavesaremainderv<b:

p=bu+v∧0<v<b

Multiplyingbothsidesoftheequationbyaandthensubstitutingwithab=mpgives

Butthismeansthatvisanintegersmallerthanbsuchthatavisamultipleofp.That’sacontradiction,sincewechosebtobethesmallestsuchnumber.Soourassumptionisfalse,andabisnotdivisiblebyp.

ThisapproachwasactuallyacommonpatternforproofsinancientGreekmathematics:choosethesmallestofsomething,andthenshowthatcertainassumptionswouldleadtoasmallerone.

***

Next,weprovearesultaboutremainders:

Lemma5.1(PermutationofRemaindersLemma):Ifpisprime,thenforany0<a<p,

where

0<ri<p∧i≠j ri≠rj

Inotherwords,ifwetakeallthemultiplesofafrom1ato(p-1)a,andexpresseachmultipleintheformqp+r,everyremainderrwillbeuniqueandthesetofremainderswillbeapermutationof{1,...,p-1}.(Weknoweachremainderislessthanp,sowehavep-1uniquenumbersintherange[1,p-1].)Example:Ifp=7anda=4,thenthelemmasaysthat

{4,8,12,16,20,24}={0·7+4,1·7+1,1·7+5,2·7+2,2·7+6,3·7+3}sotheremaindersare

{4,1,5,2,6,3}

whichisapermutationof{1,...,7-1}.

Proof.Supposeri=rjandi<j;thatis,twooftheremaindersareequal.Thenwecouldtakethedifferenceofthetwocorrespondingelementsintheset,andtheremaindersriandrjwouldcancel:

Sincetheithandjthelementsofthesetaretheproductsaiandaj,wecouldequivalentlywritethedifferenceofthesetwoelementsasaj-ai.Thatis:

Butthisisoftheformab=mp,whichimpliesthattheproductoftwointegerssmallerthanpisdivisiblebyp.SincethiscontradictsEuclidVII,30,whichwejustproved,ourassumptionmustbefalse,andtheremaindersmustbeunique.

5.3CancellationNowwewilllookatsomeresultsthatdealwiththenotionofcancellation.Ifwearemultiplyingtwonumbersxandy,theycancel(i.e.,theirproductis1)whenoneisthemultiplicativeinverseoftheother.

CancellationandModularArithmeticOnewaytoviewcancellationisinthecontextofmodulararithmetic,whichwasintroducedbyCarlGauss,whomweshallmeetinChapter8.AlthoughEulerdidnotusethistechniqueinhisproofofFermat’sLittleTheorem,youmayfinditusefultounderstandthelogic.Agoodanalogyformodulararithmeticisastandard12-hourclock.Ifit’s10o’clock,andyouhavetodosomethingthat’sgoingtotake5hours,you’llbedoneat3o’clock.Inasense,you’resayingthat10+5=3.Moreprecisely,you’resayingthat(10+5)mod12=3.(Mathematicianswouldcallnoon“0,”though.)Ofcourse,wecandomodulararithmeticinanybase.Hereareacoupleofexamplesusing7:

(6+4)mod7=3

(3x3)mod7=(3+3+3)mod7=2

Noticeinthelattercasethatwecouldalsocalculateourproductinthetraditionalway,thenexpressitintermsofmultiplesofthemodularbaseandaremainder:

(3×3)=9=(1×7)+2

Inotherwords,avaluemodulonisequivalenttotheremainderafterusingnasadivisor.Inelementaryarithmetic(forexample,arithmeticofrationalnumbers),iftheproductoftwotermsxandyis1,thentheyaresaidtocancel,andxandyarecalledinversesofeachother.Thesameistrueinmodulararithmetic,onlytheinverseswillbothbeintegers.Forexample,

(2×4)mod7=1

so2and4cancel,andareeachother’sinverse.Anegativenumberxmodnisequaltothepositivenumbern-x;it’sthepositionyou’dgettoifyou“turnedtheclockback”byxhours.Inparticular,-1modn=n–1.Justaswithelementaryarithmetic,wecanwritemultiplicationtablesformodulararithmetic.Here’soneforintegersmodulo7:

Firstweexpresstheproductasamultipleof7andaremainder;themodularproductisthenjusttheremainder.Forexample,5×4=20=(2×7)+6=6mod7,sothereisa6inthetableattheintersectionofrow5andcolumn4.Observethateveryrowisapermutationofeveryotherrow,andthateveryrowcontainsa1.Recallthatiftheproductis1,thetwofactorsareinverses.Inthetableabove,wesee,forexample,that2and4areinversessince2×4=1mod7.Here’saversionofthetablewiththeinverseofeachfactorontheleftshowninthecolumnontheright:

Formally,forintegern>1andintegeru>0,wecallvamultiplicativeinversemodulonifthereisanintegerqsuchthatuv=1+qn.Inotherwords,uandvareinversesiftheirproductdividedbynyieldsaremainderof1.Wewillrelyheavilyonthisinthefollowingproofs.

***

Thenextresultreliesonthisgeneralizednotionofcancellation:

Lemma5.2(CancellationLaw):Ifpisprime,thenforany0<a<pthereisanumber0<b<psuchthatab=mp+1.

Inotherwords,aandbcancelmodulop.Example:Supposeagainthatp=7anda=4.Isthereavalueofbthatsatisfiestheequationab=mp

+1?Let’stryallthevaluesofbuntilwefindonethatworks:

Proof.BythePermutationofRemaindersLemma,weknowthatoneofthepossibleproductsintheset

a·{1,...,p–1}

willhavearemainderof1.Inthiscasewehavep-1uniqueremaindersgreaterthan0andlessthanp,sooneofthemmustbe1.Therefore,theremustbeanotherelementbthatcancelsa.

Notethat1andp-1areself-cancelingelements—thatis,ifyoumultiplyeachbyitself,theresultis1modp,or(equivalently)theresultcanbeexpressedintheformmp+1.It’sobviousthat1·1canbeexpressedinthisform,sinceit’s0p+1.Whataboutp-1?

(p-1)2=p2-2p+1=(p-2)p+1=mp+1

Infact,1andp-1aretheonlyself-cancelingelements,whichwe’llnowdemonstrate.

Lemma5.3(Self-CancelingLaw):

Forany0<a<p,a2=mp+1 a=1∨a=p–1

Proof.Assumethereisaself-cancelingathat’sneither1norp-1:

a≠1∧a≠p–1 1<a<p–1

Rearrangingtheconditionoftheproof,wehave

a2-1=mp

Factoringtheexpressionontheleft,wehave

(a-1)(a+1)=mp

Butsincebyourassumption0<a-1,a+1<p,whichmeanswehaveaproductoftwointegerssmallerthanpthatisdivisiblebyp,acontradictionwithEuclidVII,30(seep.70).Soourassumptionisfalse,andtheonlyself-cancelingelementsare1andp-1.

WearealmostreadytoproveFermat’sLittleTheorem,butwestillneedonemoreresult:Wilson’stheorem,announcedbyEdwardWaringin1770,whoattributedittohisstudent,JohnWilson.Atthetime,Waringstatedthathewasunabletoprovethetheoremsincehedidnothavetherightnotation—inresponsetowhichGausslaterremarked,“Oneneedsnotion,notnotation.”

Theorem5.5(Wilson’sTheorem):Ifpisprime,thereexistsanintegermsuchthat

(p-1)!=mp+(p-1)

orinotherwords

(p-1)!=(p-1)modp

Proof.Bydefinition,

(p-1)!=1·2·3...(p-1)

BytheCancellationLaw,everynumberabetween1andp-1hasanumberbinthatrangethat’sitsinverse;bytheSelf-CancelingLaw,only1andp-1aretheirowninverses.Soeveryothernumberintheproductexcept1andp-1iscancelledbyitsinverse;thatis,theirproductdividedbyphasremainder1.Inotherwords,wecouldexpressallthecancelledtermstogether—allthetermsbetween1andp-1—asnp+1forsomen.Westillhaveouruncanceledterms1andp-1,soourproductnowbecomes

Thenm=np-nsatisfiesthetheorem.

Exercise5.1.Provethatifn>4iscomposite,then(n–1)!isamultipleofn.

5.4ProvingFermat’sLittleTheoremFinally,usingtheresultswe’vejustderived,wecanproveFermat’sLittleTheorem:

Ifpisprime,ap–1–1isdivisiblebypforany0<a<p.

Proof.Considertheexpression .Wecanmovetheatermsoutsidetheproduct,sowehave

Wilson’sTheoremcanbewrittenas

ThereforewecanmaketheabovesubstitutionintoEquation5.2,giving

Nowlet’sreturntotheexpression .Itsexpansioncontainsalltheterms{a,2a,3a,...,(p-1)a},whichbythePermutationofRemaindersLemma(p.71)isthesameas{q1p+r1,...,qp–1p+rp–1}.Sowecanwrite

Whenweexpandtheproductontheright,wegetasumcontainingmanytermswithp,andonethatistheproductofallri.Wegroupalltheptermstogether;theygiveussomemultipleup.Whatremainsistheproductofallri:

NowwecanapplyWilson’sTheoremagaintotheproductontheright,thenagaingroupmultiplesofp:

wherew=u+v+1.Weknowexpressions5.3and5.4areequal,andneedonlysomesimplerearrangement:

Again,wecancombinemultiplesofpontheright,givingourdesiredresult:

ap-1-1=np

Soap-1-1isdivisiblebyp.

Wealsoobservethatap-2isaninverseofa,sinceap-2·a=ap-1,whichFermat’sLittleTheoremtellsusismp+1.(Rememberthatbeinganinversewithrespecttopmeanshavingaremainderof1afterdividingbyp.)

***

WhatabouttheconverseofFermat’sLittleTheorem?Toprovethat,weneedonemoreintermediateresult:

Lemma5.4(Non-invertibilityLemma):Ifn=uv∧u,v>1,thenuisnotinvertiblemodulon.

Proof.Letn=uvandwbeaninverseofu(i.e.,wu=mn+1).Then

Soifwedefinez=(w-mv),then

(w-mv)n=zn=v

Sincen>v,thenzn>v,whichisacontradictionwithzn=v.Soucannothaveaninverse.

Definition5.1.Twonumbersmandnarecoprimeifgcd(m,n)=1.Equivalently,mandnarecoprimeiftheyhavenocommonfactorsgreaterthan1.

TheNon-invertibilityLemmatellsusthatwhenwearedealingwithnumbersmodulon,wherenisnotprime,thereareinvertibleelementsandnon-invertibleelements;elementsthatarenotcoprimetonarenotinvertible.

Theorem5.6(ConverseofFermat’sLittleTheorem):Ifforalla,0<a<n,

an-1=1+qan

thennisprime.

Proof.Supposenisnotprime;thatis,n=uv.ThenbytheNon-invertibilityLemma,uisnotinvertible.Butbytheconditionofthetheorem,un-1=un-2u=1+qun.Inotherwords,uhasaninverseun-2,whichisacontradiction.Sonmustbeprime.

5.5Euler’sTheoremLikeanygreatmathematician,EulerwasnotsatisfiedwithjustprovingFermat’sLittleTheorem;hewantedtoseeifitcouldbegeneralized.SinceFermat’sLittleTheoremwasonlyforprimes,Eulerwonderedwhethertherewasasimilarresultthatwouldincludecompositenumbers.Butcompositenumbersdostrangethingsinmodulararithmetic.Toillustratethis,let’stakealookatthemultiplicationtablemodulo10,whichwe’veannotatedbyshowinginversesoftheleft-handfactorontheright-handsideofthetable:

Thetableshouldlookabitfamiliar,becauseit’sjustlikethetraditional10×10multiplicationtable,ifyoukeeponlythelastdigitofeachproduct.Forexample,7×9=63,whichis3mod10.Immediatelywecanseedifferencesfromthetablewedidfor7,whichwasprime(seep.74).Foronething,therowsarenolongerpermutationsofeachother.Moreimportantly,somerowsnowcontain0.That’saproblemformultiplication—howcantheproductoftwothingsbe0?Thatwouldmeanthatwegetintoasituationwherewecanneverescapezero—anyproductoftheresultwillbezero.Theotherpropertywenotedearlieraboutprimes—thatonly1and-1areself-canceling—happensto

betruefor10aswell,butisnotalwaystrueforcompositenumbers.(Theinteger8,forexample,hasfourself-cancelingelements:1,3,5,and7.)Let’slookatthemultiplicationtablefor10again,focusingoncertainentries:

Therowsthatcontainonly“good”products(i.e.,nozeros)aretheoneswhosefirstfactorisshownina

rectangularboxontheleft—whichalsohappentobetherowswherethatfactorhasaninverse,shownontherightsideofthetable.Whichrowshavethisproperty?Thosethatrepresentnumbersthatarecoprimewith10.(Remember,beingcoprimemeanshavingnocommonfactorsgreaterthan1.)Socouldwejustusethegoodrowsandleaveouttherest?Notquite,becausesomeoftheresultsin

goodrowswouldthemselvesleadtobadrowsifusedinasuccessiveproduct.(Forexample,3isagoodrow,but(3×5)×2=0.)Euler’sideawastouseonlytheentriesingoodcolumnsaswellasgoodrows—thenumbersinshadedcells.Noticethatthosenumbershaveallthenicepropertieswesawforprimes:theshadednumbersineachrowarepermutationsofeachother,eachsetofshadednumberscontainsa1,andsoon.

***

ToextendFermat’sLittleTheoremforcompositenumbers,Eulerusesonlytheseboldvalues.Hestartsbydefiningthesizeofthesetofcoprimes:

Definition5.2.Thetotientofapositiveintegernisthenumberofpositiveintegerslessthannthatarecoprimewithn.Itisgivenbytheformula:

φ(n)=|{0<i<n∧coprime(i,n)}|

ThisisknownastheEulertotientfunctionorEulerφfunction.φ(n)givesusthenumberofrowscontainingshadedentriesinthemultiplicationtablemodulon.For

example,φ(10)=4,andφ(7)=6,aswecanseefromthemultiplicationtablesgivenearlier.Sinceprimesbydefinitiondon’tshareanyprimefactorswithsmallernumbers,thetotientofaprime

numberis

φ(p)=p–1

Inotherwords,allnumberslessthanagivenprimearecoprimewithit.WhatEulerrealizedwasthatthep-1inFermat’stheoremisjustaspecialcase;it’swhatφhappensto

beforprimes.NowwecanstateEuler’sgeneralizationofFermat’sLittleTheorem.

Theorem5.7(Euler’sTheorem):coprime(a,n) aφ(n)-1isdivisiblebyn.

Exercise5.2.ProveEuler’sTheorembymodifyingtheproofofFermat’sLittleTheorem.Steps:

•ReplacePermutationofRemaindersLemmawithPermutationofCoprimeRemaindersLemma.(Essentially,usethesameproofbutlookonlyat“good”elements.)•Provethateverycoprimeremainderhasamultiplicativeinverse.(Wejustshowedthattheremaindersformapermutation,so1hastobesomewhereinthepermutation.)•UsetheproductofallcoprimeremainderswheretheproofofLittleFermathastheproductofallnonzeroremainders.

***

Wewouldliketobeabletocomputetheφfunctionforanyinteger.Sincewecanexpressanyintegerastheproductofpowersofprimes,we’llstartbyseeinghowtocomputethetotientofapowerofaprimep.

Wewanttoknowthenumberofcoprimesofpm.Weknowthereareatmostpm-1ofthem,becausethat’sallthepossiblenumberslessthanpm.Butwealsoknowthatthosedivisiblebyp(i.e.,multiplesofp)arenotcoprime,soweneedtosubtracthowevermanyofthesetherearefromourtotal:

Whathappensifwehaveφ(puqv),wherepandqarebothprimes?Again,westartwiththemaximumpossibleandthensubtractoffallthemultiples.Sowe’llsubtractthenumberofmultiplesofpandalsothenumberofmultiplesofq,butthenwehavetoaddbackmultiplesofbothpandq,becauseotherwisethey’dbesubtractedtwice.(Thisgeneraltechnique,knownastheinclusion-exclusionprinciple,isoftenusedincombinatorics.)Letusassumen=puqv:

Asaspecialcasewhenwehaveasimpleproductoftwoprimes,p1andp2,wenowknowthat

Forexample,since10=5×2,

φ(10)=φ(5)φ(2)=4

Althoughthecasewecaremostaboutistheonegivenhere,wecangeneralizetheformulatohandleaproductofanynumberofprimesraisedtopowers,notjusttwo.Forexample,ifwehadthreefactorsp,q,andr,we’dsubtractallthemultiplesofeach,thenaddbackthedouble-countedmultiplesofpq,pr,andqr,andthencompensateforourovercompensationbyagainsubtractingmultiplesofpqr.Extendingthistomprimesgivesthisformula,where :

Euler’sinterestinprovinghistheoremledtohisneedtocountcoprimes.Hisderivationoftheφfunctiongavehimatoolthatallowedhimtoefficientlycomputethiscountinthecaseswheretheprimedecompositionisknown.

5.6ApplyingModularArithmeticInSection5.3,wesawhowmodularmultiplicationwasrelatedtoremainders.Let’stakealookatacoupleofourimportantresultsfromearlierinthechapterandseewhatsomeexampleslooklikeifwedothemmodulo7.Wilson’sTheoremstatesthatforaprimep,thereexistssomemsuchthat

(p-1)!=(p-1)+mp

Anotherwaytosaythisis

(p-1)!=(p-1)modp

Let’sseeifwecanconfirmthatresultifpis7.p-1is6,sowestartbyexpanding6!intoitsfactors,rearrangingthem,andusingourmodularmultiplicationtabletocancelinverses:

whichiswhatWilson’sTheorempredicts.Similarly,let’susemodularmultiplicationtoseewhatFermat’sLittleTheoremsays.Theoriginalform

is

Ifpisprime,ap-1–1isdivisiblebypforany0<a<p.

Butwithmodulararithmetic,wecouldrestateitas

Ifpisprime,ap–1–1=0modpforany0<a<p.

or

Ifpisprime,ap–1=1modpforany0<a<p.

Again,let’susep=7,andtrya=2.Thistimewe’llexpandourexpression,multiplybothsidesby6!,andthenusemodularmultiplicationtocancelterms:

whichiswhatFermat’sLittleTheoremtellsus.

5.7ThoughtsontheChapterEarlier,wesawhowtheancientGreekswereinterestedinperfectnumbers.Therewasn’tanypracticalvaluetothiswork;theyweresimplyinterestedinexploringpropertiesofcertainkindsofnumbersfortheirownsake.Yetaswehaveseeninthischapter,overtimethesearchforthese“useless”perfectnumbersledtothediscoveryofFermat’sLittleTheorem,oneofthemostpracticallyusefultheoremsinallofmathematics.We’llseewhyit’ssousefulinChapter13.Thischapteralsogaveusafirstlookattheprocessofabstractioninmathematics.Eulerlookedat

Fermat’sLittleTheoremandrealizedthathecouldextenditfromonespecificsituation(primes)toamoregeneralone(integers).HesawthattheexponentinFermat’stheoremwasaspecialcaseofamoregeneralconcept,thenumberofcoprimes.Thatsameprocessofabstractionliesattheheartofgenericprogramming.Generalizingcodeislikegeneralizingtheoremsandtheirproofs.JustasEulersawhowtoextendFermat’sresultfromonetypeofmathematicalobjecttoanother,soprogrammerscantakeafunctionthatwasdesignedforonetypeofcomputationalobject(say,vectors)andextendittoworkequallywellonanother(perhapslinkedlists).

6.AbstractioninMathematics

Mathematiciansdonotstudyobjects,buttherelationsbetweenobjects;tothemitisamatterofindifferenceiftheseobjectsarereplacedbyothers,

providedthattherelationsdonotchange.Matterdoesnotengagetheirattention,theyareinterestedinformalone.

Poincaré,ScienceandHypothesisThehistoryofmathematicsisfilledwithdiscoveriesofnewabstractions:findingwaystosolveamoregeneralproblem.Forexample,wesawinChapter5howEulergeneralizedFermat’sLittleTheoremsoitwouldworkwithcompositenumbersaswellasprimes.Eventually,mathematiciansrealizedthattheycouldgeneralizebeyondnumbers,andderiveresultsaboutabstractentitiescalledalgebraicstructures—collectionsofobjectsthatfollowcertainrules.Thisledtothedevelopmentofanentirelynewbranchofmathematics,abstractalgebra.Inthischapter,we’llintroducethefirstexamplesoftheseabstractentities,andprovesomeoftheirproperties.Aswedidinthepreviouschapter,we’regoingtoputprogrammingasidewhilewebuildthefoundationsweneedtoderiveagenericalgorithminChapter7.

6.1GroupsThefirstandmostimportantofthesealgebraicstructures,discoveredbyFrenchmathematicianÉvaristeGaloisin1832,iscalledagroup.

Definition6.1.Agroupisasetonwhichthefollowingaredefined:

operations:x y,x–1

constant:eandonwhichthefollowingaxiomshold:

Theconstanteistheidentityelement(alsosometimeswrittenid),whichisoftenwrittenas1inmultiplicativecontexts.Theoperationx–1istheinverseoperation;applyingtheoperationtoanitemanditsinverseresultsintheidentityelement,asthelastaxiomshows.Thegroupoperationisbinary,whichsimplymeansthatittakestwoarguments(ithasnothingtodowiththebinaryrepresentationofnumbersusedincomputers).Thesymbol (sometimeswritten*)canrepresentanybinaryoperation,aslongasitfollowstheaxioms.Weoftentreatthegroupoperationasmultiplication,andevenreferto“multiplying”twoelementsofa

group,althoughwhatwereallymeanisapplyingthegroupoperation,whateveritmightbe.Justaswithmultiplication,thesymbolfortheoperationisoftendropped;thatis,x ymaybewrittenxy,andx x=xx=x2.Thegroupoperationisnotnecessarilycommutative(commutativitymeansthat∀x,y:x y=y x).

Whenwewanttorequirecommutativity,weneedtospecifyaparticularkindofgroup:

Definition6.2.Anabeliangroupisagroupwhoseoperationiscommutative.

Onekindofabeliangroupistheadditivegroup:

Definition6.3.Anadditivegroupisanabeliangroupwherethegroupoperationisaddition.

Additivegroupsaretheexceptiontothenamingconventionsgivenearlier.Foranadditivegroup,thesymbol+isusedtorepresentitsoperationand0itsidentityelement.Eventhoughthename“additivegroup”saysnothingaboutcommutativity,itisbyconventionassumedtobecommutative.Groupsareclosedundertheiroperation.Thismeansthatifyoutakeanytwoelementsofthegroupand

applythegroupoperation,theresultwillitselfbeamemberofthegroup.Similarly,theyareclosedundertheinversefunction:ifyoutaketheinverseofanyelementofthegroup,theresultisstillanelementofthegroup.Someexamplesofgroupsfollow:•Additivegroupofintegers:theelementsareintegersandtheoperationisaddition.•Multiplicativegroupofnonzeroremaindersmodulo7:theelementsarethenumbers1through6andtheoperationismultiplicationmodulo7.•Groupofrearrangementsofadeckofcards:theelementsarepermutationsofthedeckandtheoperationiscompositionofthesepermutations.•Multiplicativegroupofinvertiblematrices(thosewithnonzerodeterminants)withrealcoefficients:theelementsarematricesandtheoperationismatrixmultiplication.•Groupofrotationsoftheplane:theelementsaredifferentrotationsaroundtheoriginandtheoperationiscompositionoftheserotations.

Notethatintegersdonotformamultiplicativegroup,becausethemultiplicativeinverseofmostintegersisnotaninteger.Inotherwords,integersarenotclosedundermultiplicativeinverse.

Exercise6.1.Howmanyintegershavemultiplicativeinversesthatareintegers?Whatarethey?

Let’slookatoneoftheexamplesinabitmoredetail.InChapter5,welookedatthemultiplicationtableforintegersmodulo7:

Theuniquevaluesinthetable—theset{1,2,3,4,5,6}—arealsocalled“nonzeroremaindersmodulo7,”andaswenotedearlier,theseformamultiplicativegroup.Whatdoesthatmean?Sinceit’samultiplicativegroup,thegroupoperationismultiplication,anditsidentityelementis1.Wecanseefromthefirstrowandcolumnofthetablethat1istheidentity,becausetheproductofanyelementxand1isx.Sincegroupsareclosedundertheiroperation,ifwemultiplyanytwomembersofthegroup,weget

anothermemberofthegroup.Forexample:

Associativityandcommutativityofmodularmultiplicationfollowsfromassociativityandcommutativityofintegermultiplication.Thecommutativityorabeliannessofthegroupisevidentbyobservingthatthemultiplicationtableissymmetricwithrespecttothemaindiagonal.Sincegroupsareclosedunderinverse,ifwetaketheinverseofanymemberofthegroup,weget

anothermemberofthegroup.(Recallthattheinverseofanelementxistheelementthatproduces1whenmultipliedbyx.Fromthemultiplicationtable,youcanseethepairsofinversesbylookingattherowsandcolumnswhosecellscontain1s.)Forexample:

2–1=4mod7

4–1=2mod7

5–1=3mod7

ÉvaristeGalois(1811–1832)

TheconceptofgroupsstartedwiththeworkofÉvaristeGalois,ayoungFrenchcollegedropoutinvolvedinarevolutionarymovement,andthemostromanticfigureinthehistoryofmathematics.Intheearly19thcentury,aromanticspiritspreadthroughEurope;youngpeopleidolizedtheEnglishpoetByron,whodiedfightingforGreekindependence,andotherswhowerewillingtogivetheirlivesforacause.TheyrememberedNapoleonnotasatyrant,butasayoungherowhoabolishedfeudalismthroughoutEurope.Parisintheearly1830swasaflamewithrevolutionaryactivity.Galois,whowasabohemianhothead,joinedtherevolutionarymovement.Asaromanticrebel,Galoisdidnotfollowtheconventionalpaththroughauniversityeducation.Afterfailingtobeadmittedtooneschoolandbeingexpelledfromanother,hestudiedmathematicsonhisown,becominganexpertonLagrange’stheoryofpolynomials.Heservedbriefprisonsentencesforvariousprotestactivities,suchasmarchingthroughthestreetsintheuniformofabannednationalguardunitwhilecarryingseveralloadedweapons—butkeptdoingmathematicswhileinprison.Atage20,Galois,defendingthehonorofawomanwhomheapparentlybarelyknew,issuedachallenge(orwaschallenged)toaduel.Thenightbeforetheduel,certainofhisimpendingdeath,hewrotealonglettertoafrienddescribinghismathematicalideas.Thismanuscriptcontainedtheseedsofthetheoryofgroups,fields,andtheirautomorphisms(mappingsontothemselves).Theseideaslaidthefoundationsforamajornewfieldofmathematics,abstractalgebra.AccordingtomathematicianHermannWeyl,“Thisletter,ifjudgedbythenoveltyandprofundityofideasitcontains,isperhapsthemostsubstantialpieceofwritinginthewholeliteratureofmankind.”Thenextday,Galoisfoughttheduelanddiedasaresultofhiswounds.Itisironicthatwhileheonlyplayedatbeingarevolutionaryinpolitics,hewasatruerevolutionaryinmathematics.

6.2MonoidsandSemigroupsInsomesituations,weareinterestedinalgebraicstructuresthathavefewerrequirementsthangroups.(We’llseesomeoftheseapplicationsinthenextchapter.)Forexample,therearetimeswhenwedon’tneedtorequireaninverseoperation,butwewanttomaintaintheotherpropertiesofagroup.Thisiscalledamonoid.Moreformally:

Definition6.4.Amonoidisasetonwhichthefollowingaredefined:

operation:x y

constant:e

andonwhichthefollowingaxiomshold:

Thisdefinitionisliterallythesameastheoneforgroups,exceptwe’veleftouttheinverseoperationandtheaxiomofcancellationthatusesit.Aswithgroups,wecandefineparticularkindsofmonoidsbyspecifyingtheoperation,suchasanadditivemonoid(wheretheoperationisaddition)andamultiplicativemonoid(wheretheoperationismultiplication).Someexamplesofmonoidsfollow:•Monoidoffinitestrings(freemonoid):theelementsarestrings,theoperationisconcatenation,andtheidentityelementistheemptystring.•Multiplicativemonoidofintegers:theelementsareintegers,theoperationismultiplication,andtheidentityelementis1.

Wecanrelaxtherequirementsevenfurtherbydroppingtheidentityelement.Thisiscalledasemigroup:

Definition6.5.Asemigroupisasetonwhichthefollowingisdefined:

operation:x y

andonwhichthefollowingaxiomholds:

Again,allwe’vedoneistakenthepreviousdefinitionandremovedsomething—inthiscase,therequirementthattherebeanidentityelement,andtheaxiomthatusesit.Andaswedidwithgroupsandmonoids,wecandefineadditivesemigroupsandmultiplicativesemigroupsassemigroupsthatusethoseoperations.Someexamplesofsemigroupsfollow:•Additivesemigroupofpositiveintegers:theelementsarepositiveintegersandtheoperationisaddition.•Multiplicativesemigroupofevenintegers:theelementsareevenintegersandtheoperationismultiplication.

Aswementionedearlier,mathematicianswriterepeatedapplicationsofthesemigroup,monoid,orgroupoperationwiththesameconventionsasordinarymultiplication.Forexample:

x x x=xxx=x3

Moreformally,wedefineraisingasemigrouptoapowerasfollows:

Exercise6.2.Whycan’twedefinepowerforsemigroupsstartingwithn=0?

Equation6.1showsthesemigroupoperationhappeningontheleftsideofthepower(i.e.,itsaysxxn–1,notxn–1x).Whatifwewantedtowritetheexpansiontheotherway?Wecandothat,too,aswewillnowprove:

Lemma6.1:Forn≥2,xxn–1=xn–1x.

Proof.Weprovethisbyinduction.1

Basis:n=2.It’sobviouslytrue,because

xx1=xx=x1x

1Forarefresheronthisprooftechnique,seeAppendixB.2.

Astheinductionhypothesis,weassumethestatementistrueforn=k–1:

xx(k –1)–1=xxk–2=xk–2x=x(k–1)–1x

andthenderivetheresultforn=k:

Eventhoughasemigroupguaranteesonlyassociativityandnotcommutativityofitsoperation,itturnsoutthatpowersofagivensemigroupelementalwayscommute,whichwecanalsoprovebygeneralizingthisresult.Thisisperhapsthemostimportanttheoremonsemigroups.

Theorem6.1(CommutativityofPowers):xnxm=xmxn=xn+m

Proof.Proofbyinductiononm.

Basis:m=1:

Inductivestep:Assumetrueform=k.Showform=k+1:

Wehaveshownthatxnxm=xn+m.Therefore,itisalsotruethatxmxn=xm+n.Sinceintegeradditioniscommutative,xn+m=xm+n,soxnxm=xmxn.

Asemigroupistheweakestinterestingalgebraicstructure.Theonlyrequirementlefttorelaxistheassociativityaxiom.Analgebraicstructurecalledmagmadropseventhataxiom,butitturnsoutnottobeveryuseful.Sincetherearenoaxiomsleft,notheoremscanbeproved.

6.3SomeTheoremsaboutGroupsNowlet’sreturntogroupsandlookatsomeoftheirproperties.Animportantobservationisthatallgroupsaretransformationgroups.Inotherwords,everyelementa

ofthegroupGdefinesatransformationofGontoitself:

x→ax

Forexample,withtheadditivegroupofintegers,ifwechoosea=5,thenthisactsasa“+5”operation,transformingthesetofelementsxtothesetx+5.Thesetransformationsareone-to-onebecauseofourinvertibilityaxiom:

a–1(ax)→x

Inourexample,wecanundoour+5transformationbyapplyingtheinverse,–5.

Theorem6.2:Agrouptransformationisaone-to-onecorrespondence.2

2Aone-to-onecorrespondencebetweentwosetsisamappingbetweenthemthatisbothone-to-oneandonto.

ThisisequivalenttosayingthatforanyfinitesetSofelementsofgroupGandanyelementaofG,asetofelementsaShasthesamenumberofelementsasS.

Proof.IfS={s1,...,sn},thenaS={as1,...,asn}.WeknowthatthesetaScan’tcontainmoreuniqueelementsthanS,butcoulditcontainfewer?(ThatwouldbethecaseiftwooftheelementsinSweremappedtothesameelementinaS.)SupposetwoelementsofaSwerethesame:asi=asj.Then

Soiftworesultsofthetransformationaskareequal,thentheirinputsskmustbeequal.Equivalently(bythecontrapositiveofthepreviousstatement),iftheinputsarenotequal,thentheresultsofthetransformationmustnotbeequal.Sincewestartedwithndistinctarguments,wewillhavendistinctresults.Inotherwords,thesetaShasthesamenumberofelementsasS.

Hereareafewmoresimpleresultsaboutgroups:

Theorem6.3:Thereisauniqueinverseforeveryelement.

ab=e b=a–1

Proof.Supposeab=e.Thenwecanmultiplybothsidesbya–1ontheleft,likethis:

Theorem6.4:Theinverseofaproductisthereversedproductoftheinverses.

(ab)–1=b–1a–1

Proof.Thetwoexpressionsareequalifandonlyifmultiplyingonebytheinverseoftheotheryieldstheidentityelement.We’llusetheinverseof(ab)–1,whichbydefinitionis(ab),andmultiplyitbyb–1a–1:

Theorem6.5:Thepowerofaninverseistheinverseofthepower.

(x–1)n=(xn)–1

Proofbyinduction.Basis:n=1

(x−1)1=x−1=(x1)−1

Inductivestep:Assumetrueforn=k−1;thatis,(x−1)k−1=(xk−1)−1.Thenproveforn=k.Wewanttoshowthat(x–1)k=(xk)–1;thatis,theinverseofxkis(x–1)k.Ifthat’strue,thenwhenwe

multiplythemtogether,weshouldgettheidentityelement.Usingthedefinitionofpowerandthecommutativityofpowerstheorem,we’llrewritexkasxk –1xand(x–1)kasx–1(x–1)k–1,regroupthetermstogetsometocancel,andthensubstituteusingourinductiveassumption:

Therefore(xn)–1=(x–1)n.

Exercise6.3(veryeasy).Provethatanygrouphasatleastoneelement.

Definition6.6.Ifagrouphasn>0elements,niscalledthegroup’sorder.Ifagrouphasinfinitelymanyelements,itsorderisinfinite.

Thereisalsothenotionoftheorderofanelementinagroup:

Definition6.7.Anelementahasanordern>0ifan=eandforany0<k<n,ak≠e.(Inotherwords,theorderofaisthesmallestpowerofathatproducese.)Ifsuchndoesnotexist,ahasaninfiniteorder.

Exercise6.4(veryeasy).Whatistheorderofe?Provethateistheonlyelementofsuchorder.

***

Wenowcometoanimportanttheoremaboutgroups:

Theorem6.6:Everyelementofafinitegrouphasfiniteorder.

Proof.Ifnisanorderofthegroup,thenforanyelementa,{a,a2,a3,...,an+1}hasatleastonerepetitionaiandaj.Letusassumethat1≤i<j≤n+1,aiisthefirstrepeatedelementandajisitsfirstrepetition.Then

andj–i>0istheorderofa.Thisproofusesaversionofthepigeonholeprinciple.(Tolearnmoreaboutthepigeonholeprinciple

andhowtouseit,seeAppendixB.3.)Theresultguaranteesthatthissimplealgorithmforcomputingtheorderofanelementwillterminate:justkeepmultiplyingbyitselfuntilyougete.

Exercise6.5.Provethatifaisanelementofordern,thena–1=an–1.

6.4SubgroupsandCyclicGroupsDefinition6.8.AsubsetHofagroupGiscalledasubgroupofGif

Inotherwords,tobeasubgroup,Hmustbeasubsetandagroup.AssociativityoftheoperationonGimpliesitsassociativityonH,sowedonotneedtoexplicitlystatetheassociativityrequirementinthedefinition;bythesamereasoning,wedonotneedtoexplicitlyrestatetheidentityandcancellationaxioms.

Forexample,theadditivegroupofevennumbersisasubgroupoftheadditivegroupofintegers;soistheadditivegroupofnumbersdivisibleby5.Somegroupshavemanysubgroups,butalmostallgroupshaveatleasttwo:thegroupitselfandthe

groupconsistingofjusttheelemente.Thesetwosubgroupsarecalledtrivialsubgroups.(Theonlygroupthatdoesn’thaveatleasttwosubgroupsisthegroupthatconsistsofjusttheidentityelement.)Let’sreturnagaintothemultiplicativegroup{1,2,3,4,5,6}ofnonzeroremaindersmodulo7andits

multiplicationtable:

Ourgrouphasfourmultiplicativesubgroups:

{1},{1,6},{1,2,4},{1,2,3,4,5,6}

Howcanwetell?Tobeasubgroup,eachonefirstneedstobeasubsetoftheoriginalgroup.Eachoftheseisobviouslyasubsetof{1,2,3,4,5,6},sincealloftheirelementsarecontainedinthelargerset.Next,eachsubsetalsocontains1(theidentityelement)andisclosedunderitsoperation(multiplicationmodulo7)anditsinverseoperation.Forexample,considertheset{1,2,4}:ifwemultiplyanyelementofthesetbyitselforanyother

memberofthesetanynumberoftimes(mod7),wewillstillgetaresultintheset.

Exercise6.6.Findordersofeveryelementof:

•Themultiplicativegroupofremaindersmod7•Themultiplicativegroupofremaindersmod11

***

Thesimplestkindofgroupsarecyclicgroups.

Definition6.9.Afinitegroupiscalledcyclicifithasanelementasuchthatforanyelementb,thereisanintegernwhere

b=an

Inotherwords,everyelementinthegroupcanbegeneratedbyraisingoneparticularelementtodifferentpowers.Suchanelementiscalledageneratorofthegroup;agroupmayhavemultiplegenerators.Theadditivegroupofremaindersmodulonisanexampleofacyclicgroup.Inourpreviousexampleofremaindersmodulo7,wecantellthatthegeneratorsare3and5,because

theyarenotinanynontrivialsubgroupoftheoriginalgroup.

Exercise6.7.Provethatanysubgroupofacyclicgroupiscyclic.

Exercise6.8.Provethatacyclicgroupisabelian.

Lemma6.2:Powersofagivenelementinafinitegroupformasubgroup.

Inotherwords,everyelementofafinitegroupiscontainedinacyclicsubgroupgeneratedbythiselement.

Proof.Forasettobeasubgroup,itmustbeanonemptysubsetanditmustbeagroup.Tobeasubset,itmustbeclosedunderthegroupoperation.Weknowit’sclosedundertheoperationbecausetheproductoftwopowersisapower.Tobeagroup,itsoperationneedstobeassociative,itmustcontaintheidentityelement,anditmusthaveaninverseoperation.Weknowtheoperationisassociative,becauseit’sthesameoperationfromtheoriginalgroup.Weknowthesetofpowersofagivenelementhasanidentityelement,sinceweshowedearlier(Theorem6.6)thateveryelementofafinitegrouphasfiniteorder.3Andweknowthattheyhaveaninverse,becauseforeverypowerak,an–kisitsinverse,wherenistheorderofa.

3Recallthatpartofthedefinitionofanelementahavingafiniteorderisthatan=e.

6.5Lagrange’sTheoremOneoftheremarkablethingsaboutabstractalgebraisthatwecanproveresultsforstructuressuchasgroupswithoutknowinganythingabouteitherthespecificitemsinthegrouportheoperation.Toseeanimportantexampleofthis,wewillstartbyprovingafewsimpleresultsaboutcosets.

Definition6.10.IfGisagroupandH⊂GisasubgroupofG,thenforanya Gthe(left)cosetofabyHisaset

aH={g G|∃h H:g=ah}

Inotherwords,acosetaHisasetofallelementsinGobtainablebymultiplyingelementsofHbya.Asanexample,considertheadditivegroup4ofintegers anditssubgroup,integersdivisibleby4,4 .

(Theuseof asasymbolforintegerscomesfromtheGermanwordZahlen,whichmeans“numbers.”)Ithasfourdistinctcosets:4n,4n+1,4n+2,and4n+3.Addingotherintegerswillresultonlyinvaluesthatarealreadyinoneofthesecosets;forexample,thecoset4n+5willcontainthesameelementsasthecoset4n+1.(Theleftandrightcosetsarethesame,sinceintegeradditioniscommutative.)

4Remember,inanadditivegroup,theroleofgroup“multiplication”isplayedbyaddition.SothecosetaHconsistsofelementsofGobtainablebyaddingatoelementsofH.

Lemma6.3(SizeofCosets):InafinitegroupG,foranyofitssubgroupsH,thenumberofelementsinacosetaHisthesameasthenumberofelementsinthesubgroupH.

Proof.Wealreadyprovedtheone-to-onecorrespondenceoftransformationsaSwhenSisasubsetofG.Sinceasubgroupisbydefinitionalsoasubset,thenweknowthemappingfromHtoaHisalsoaone-to-onecorrespondence.Ifthereisaone-to-onecorrespondencebetweentwofinitesets,theyarethesamesize.

Lemma6.4(CompleteCoveragebyCosets):EveryelementaofagroupGbelongstosomecosetofsubgroupH.

Proof.a aH.Thatis,everyelementabelongstothecosetaHgeneratedbyitself,becauseH,beingasubgroup,containstheidentityelement.

Lemma6.5(Cosetsareeitherdisjointoridentical):IftwocosetsaHandbHinagroupGhaveacommonelementc,thenaH=bH.

Proof.Supposethecommonelementcisahainonecosetandbhbintheothercoset.

aha=bhb

Multiplyingbothsidesontherightby ,weget

ThetermontherightisbtimessomethingfromH(weknowit’sfromHbecausehbisfromHand isfromH,andsinceHisasubgroup,itisclosedundermultiplication).Nowlet’smultiplybothsidesontherightbyx,anarbitraryelementfromH:

WeknowbydefinitionthataxisinthecosetaH.WealsoknowthatthetermontherightisbtimessomethingfromH,soit’salsointhecosetbH.SincewecandothisforanyxinH,aH⊆bH.Wecanthenrepeattheprocessfromthebeginning,thistimeusing insteadof ,toshowthatbH⊆aH.SobH=aH.

Withtheseresults,wecanstateanimportanttheoremingrouptheory,whichillustratesthepowerofabstractreasoning.Ifyouwanttolearnonetheoremingrouptheory,thisistheone.Whileit’sverysimpletostate,itprovidesthefoundationforthetheoryoffinitegroups.

Theorem6.7(Lagrange’sTheorem):TheorderofasubgroupHofafinitegroupGdividestheorderofthegroup.

Proof.

1.ThegroupGiscoveredbycosetsofH.(Provedearlier,Lemma6.4)2.Differentcosetsaredisjoint.(Provedearlier,Lemma6.5)3.Theyareofthesamesizen,wherenistheorderofH.(Provedearlier,Lemma6.3.Specifically,theresultsaysthat|aH|=|H|foranya,soeverycosethassize|H|andthereforethesamesizeaseveryothercoset.)

ThereforetheorderofGisnm,wheremisthenumberofdistinctcosets,whichmeansthattheorderofGisamultipleoftheorderofH.Inotherwords,theorderofHdividestheorderofG.

Asanexample,supposeagroupGhastwodistinctcosetsofitssubgroupH.TheneveryelementofGmustbecoveredbyoneortheother(butnotboth)ofthosecosets,sotheorderofHmustbe|G|/2.Interestingly,theconverseofLagrange’sTheoremisnottrue:inagroupofordern,noteverydivisorof

nwillhaveasubgroupofcorrespondingorder.

Joseph-LouisLagrange(1736–1813)

TheleadingmathematicianinEuropeattheendofthe18thcenturywasJoseph-LouisLagrange,whowasasuccessorofLeonhardEulerbothintermsofintellectualleadershipandasholderofapositionasdirectorofmathematicsatthePrussianAcademyofSciencesinBerlin.LagrangewasbornGiuseppeLuigiLagranciainTurin,inthenorthernItalianregionofPiedmont.Ashisoriginalnamesuggests,hegrewupspeakingItalian,althoughhisfamilyclaimedFrenchancestry.LagrangediscoveredmathematicslargelyonhisownwhileastudentinTurin,andwithinafewyearsbecameaninstructorandbeganpublishinghiswork.Sometimearoundage20,hebegancorrespondingwithEuler,whowasinBerlinatthetime.ImpressedbyLagrange’swork,Euleractedasamentortotheyoungermathematician,encouraginghimandpromotinghisdiscoveries.EulerbeganlobbyingtobringLagrangetoBerlin,butbythetimethisplancametofruitionin1766,EulerhadmovedbacktoRussia.WithEulergone,Lagrangewasappointedtohismentor’sformerposition,andsoonestablishedhimselfasthesecondmostprominentmathematicianinEurope.Lagrangespentthenext20yearsinBerlin,anddidhismostimportantworkduringthisperiod,contributingtoseveralareasofmathematicsandphysics.HisbookAnalyticalMechanics,perhapsoneofthe10mostimportantbooksinthehistoryofmathematics,describedatop-downapproachtosolvingmechanicalproblemsthatwasmoregeneralthanusingNewtonianmechanics.MuchofmodernphysicsreliesonLagrange’swork.Healsodidextensiveworkonpolynomialequations,realizingthatcoefficientscouldbeexpressedasfunctionsofapolynomial’sroots,andlaidthegroundworkforalaterbreakthroughbyGalois.Andinnumbertheory,Lagrangefiguredoutwhencontinuedfractionswillbeperiodic,complementingEuler’searlierworkonthesubject.Afterthedeathofhispatronandfriend,KingFriedrichIIofPrussia,LagrangewasapproachedbytheFrenchambassador(actingonordersfromKingLouisXVI)toconvincethegreat

mathematiciantomovetoFrance.Lagrangeagreed,andwouldliveinParisfrom1786untilhisdeath.Despitehisprominence,Lagrangewasashy,unpretentiousmanwithfewfriendsandlittlesociallife.Hewaspronetoboutsofdepression,andmadelittleprogressonhisworkforyearsatatime,particularlyduringhisfirstyearsinFrance.However,theFrenchRevolutionrevivedhisinterest—althoughheoftenfearedhewouldneedtofleethecountryduetotherevolutionaries’expulsionofmostforeigners.Hebecameinvolvedineffortstocomeupwithanewsystemofweightsandmeasures,andwasoneofacommitteeoffiveprominentscientistswhovotedtoapprovewhatwenowcallthemetricsystem.Lagrangealsobeganteachingagain,althoughbymanyaccountshewasnotpopularwithstudents,whohadtroubleunderstandingbothhisideasandhisItalianaccent.Inhislateryears,LagrangewasfavoredbythenewemperorNapoleonBonaparte—himselfquiteanaccomplishedmathematician.NapoleonrecognizedLagrange’sgenius,andmadehimaCountoftheEmpire.

***

Nowwe’llproveacoupleofcorollariestoLagrange’sTheorem:Corollary6.7.1:Theorderofanyelementinafinitegroupdividestheorderofthegroup.

Proof.ThepowersofanelementofGformasubgroupofG.Sincetheorderofanelementistheorderofthesubgroup,andsincetheorderofthesubgroupmustdividetheorderofthegroup,thentheorderoftheelementmustdividetheorderofthegroup.

(Reminder:Theorderofanelementisequaltotheorderofthecyclicgroupofitspowers.)Corollary6.7.2:GivenagroupGofordern,ifaisanelementofG,thenan=e.

Proof.Ifahasanorderm,thenmdividesn(bythepreviouscorollary),son=qm.am=e(bydefinitionoforderofanelement).Therefore(am)q=eandan=e.

Notethatthisdoesn’tsaythattheorderofaisn;itcouldbesmaller.Lagrange’sTheoremletsuseasilyprovesomeimportantresultsfromChapter5inmuchsimpler

fashionthanwedidthefirsttime:Fermat’sLittleTheorem:Ifpisprime,ap–1–1isdivisiblebypforany0<a<p.

Proof.Letustakethemultiplicativegroupofremaindersmodulop.Itcontainsp–1nonzeroremainders.Sincep–1istheorderofthegroup,itfollowsimmediatelyfromCorollary6.7.2that

ap–1=e

Sincetheidentityelementforamultiplicativegroupis1(specifically,1modpinourgroupofremainders),wehave

ap–1=1modp

ap–1–1=0modp

whichiswhatitmeanstobedivisiblebyp.

Euler’sTheorem:For0<a<n,whereaandnarecoprime,aφ(n)–1isdivisiblebyn.

Proof.Letustakethemultiplicativegroupofinvertibleremaindersmodulon.Sinceφ(n)isbydefinitionthenumberofcoprimes,andeverycoprimeisinvertible,φ(n)givesustheorderofthegroup.ItfollowsimmediatelyfromCorollary6.7.2that

aφ(n)=e

aφ(n)=1modn

orequivalently

aφ(n)–1=0modn

Thelogicisexactlythesameasthepreviousproof.

Exercise6.9(veryeasy).Whataresubgroupsofagroupoforder101?

Exercise6.10.Provethateverygroupofprimeorderiscyclic.

6.6TheoriesandModelsGroups,monoids,andsemigroupsareexamplesofwhatmathematicianscalltheories.Peopleusetheword“theory”inmanyways,oftenmeaningsomethinglike“conjecture”—thatis,anunprovenexplanation.Buttomathematicians,“theory”hasaveryspecificmeaning,whichdoesnotincludethissenseofbeingunproven.

Definition6.11.Atheoryisasetoftruepropositions.

Fromnowon,whenweusetheword“theory,”we’llmeanthisspecificmathematicalsense.Herearesomeimportantfactsabouttheories:

•Atheorycanbegeneratedbyasetofaxiomsplusasetofinferencerules.•Atheoryisfinitelyaxiomatizableifitcanbegeneratedfromafinitesetofaxioms.•Asetofaxiomsisindependentifremovingonewilldecreasethesetoftruepropositions.•Atheoryiscompleteifforanyproposition,eitherthatpropositionoritsnegationisinthetheory.•Atheoryisconsistentiffornopropositionitcontainsboththatpropositionanditsnegation.

Let’slookataspecificexample:thenotionofgroupsthatwe’vebeendiscussingthroughoutthechapter.Agroupisatheoryinthesensewe’vejustdefined—specifically,atheorythat,givenoperationsx yandx–1andtheidentityelemente,hastheseaxioms:

x (y z)=(x y) z

x e=e x=x

x x–1=x–1 x=e

Startingwiththese,wecanderiveanynumberoftruepropositions(theorems),suchas

x y=x y=e

(x y)–1=y–1 x–1

Wearen’tenumeratingallthetruepropositionsthatconstitutethetheoryofgroups.Rather,we’regeneratingthepropositionsbyderivingthemfromtheaxiomsandfrompreviouslyprovenpropositions.Forexample,wecanprovethefirsttheorembymultiplyingbothsidesoftheequationbyx–1.Likebasisvectorsinlinearalgebra,axiomsformabasisforthetheory.Alsolikethelinearalgebracase,wecanhavemorethanonebasisforthesametheory.

***

Closelyassociatedwiththenotionofatheoryisthatofamodel.Again,themathematicalmeaningisquitedifferentfromtheeverydaymeaning:

Definition6.12.Asetofelementswherealltheoperationsinthetheoryaredefinedandallthepropositionsinthetheoryaretrueiscalleditsmodel.

Inasense,amodelisaparticularimplementationofatheory.Amodelgivesyouthespecificsetofelements;atheorydoesnot.Justastherecanbemultipleimplementationsof,say,analgorithm,sotherecanbemultiplemodelsofatheory.Forexample,theadditivegroupofintegersandthemultiplicativegroupofnonzeroremaindersmodulo7arebothmodelsofthetheoryofabeliangroups.Themore5propositionsthereareinatheory,thefewerdifferentmodelsthereare.Ifwegenerate

propositionsfromaxiomsandinferencerules,thenfeweraxiomsmeansfewerpropositions,andthereforemoremodels.Thismakessenseintuitively:axiomsandpropositionsareconstraintsonatheory;themoreofthemyouhave,theharderitistosatisfyallofthem,sothefewermodelstherewillbethatdoso.

5Thenotionof“more”thatwe’reinterestedinhereisthatofadditional.IfthesetofpropositionsfortheoryAcontainsallthepropositionsoftheoryB,plussomeadditionalones,thenwesayAhasmore,evenifbothhaveacountablyinfinitenumberofpropositions.

Conversely,themoremodelsthereareforatheory,thefewerpropositionsthereare.Iftherearemorewaystodosomething,theremustbefewerconstraintsonhowyoucandoit.

Definition6.13.Twomodelsareisomorphicifthereisaone-to-onecorrespondencebetweenthemthatpreservestheiroperations.Thismeanswecanapplythemapping(oritsinverse)beforeoraftertheoperationandwe’llgetthesameresult.

Forexample,wecanmapnaturalnumberstoevennaturalnumberswiththemapping“multiplyby2,”anduseadditionasouroperation.Ifweaddtwonaturalnumbersandthenapplythemapping(i.e.,multiplyby2),wegetthesameresultasifwefirstmultiplyby2andthenaddthenumbers:

Anisomorphismofamodelwithitselfiscalledanautomorphism.

Definition6.14.A(consistent)theoryiscalledcategoricalorunivalentifallofitsmodelsareisomorphic.6

6ThisistheoriginaldefinitionbyOswaldVeblen.Modernlogicianstalkaboutκ-categoricaltheories:allmodelsofthecardinalityκareisomorphic.Fulltreatmentofmodernmodeltheoryiswelloutsidethescopeofthisbook.

Aninconsistenttheoryisonethathasnomodels—there’snowaytosatisfyallthepropositionswithoutacontradiction.

CategoricalTheoriesversusSTLForalongtime,peoplebelievedthatonlycategoricaltheoriesweregoodforprogramming.WhentheC++StandardTemplateLibrary(STL)wasfirstproposed,somecomputerscientistsopposeditonthegroundsthatmanyofitsfundamentalconcepts,suchasIterator,wereunderspecified.Infact,itisthisunderspecificationthatgivesthelibraryitsgenerality.Whilelinkedlistsandarraysarenotcomputationallyisomorphic,manySTLalgorithmsaredefinedoninputiteratorsandworkonbothdatastructures.Ifyoucangetbywithfeweraxioms,youallowforawiderrangeofimplementations.

6.7ExamplesofCategoricalandNon-categoricalTheoriesLet’slookatanexampleofacategoricaltheorywithtwoisomorphicmodels:cyclicgroupsoforder4.Thefirstmodelis ,theadditivegroupofremaindersmodulo4(consistingof{0,1,2,3});thesecondmodelis( ),themultiplicativegroupofnonzeroremaindersmodulo5(consistingof{1,2,3,4}).Thesegroupshavethefollowing“multiplication”tables:

Eventhoughthenumbersaredifferent,thesetwomodelsareisomorphic;wecouldmapelementsofonetoelementsoftheother.Intheory,thereare4!=12possiblemappings—0inthefirstmodelcouldmapto1,2,3,or4inthesecond,then1couldmaptothethreeremainingchoices,andsoon.Butinthiscase,the

actualnumberofpossibilitiesismuchsmaller.Weobservethatinthefirstmodel,thevalues1and3aregeneratorsofthegroup—wecouldstartwith

eitherofthem,raiseittoapowerusingthegroupoperation,andgetalltheotherelements.Inthesecondmodel,2and3aregenerators.Thishelpsusnarrowthechoices:ageneratorfromonegrouphastomaptoageneratorfromtheothergroup,whichinthiscasegivesustwodifferentmappings.Forexample,wecansay,“theroleof1inthefirstmodelisplayedby2inthesecondmodel,and3inthefirstmodelcorrespondsto3inthesecond.”Whatabouttheothertwovalues?Weknowfromthemultiplicationtablesthat0inthefirstmodelisthe

identityelement,aroleplayedby1inthesecondmodel.Finally,weknowthat2inthefirstmodelmapsto4inthesecondmodel,becauseinbothcasesitistheonlynon-identityelementthatgivesidentitywhenappliedtoitself.Sowehavetwopossiblemappings:

Howdoweknowthesemappingsproduceoursecondmodel?Onewayistoseeifwecanusethemtotransformthe multiplicationtableintothe( )multiplicationtable.Let’stryitusingthesecondmapping.First,wereplacethevaluesfromthe tablewiththemappedvalues.Thatgivesus:

Thenwepermutetherowsandcolumnssotheheadersareintherightorder.We’llstartbyswappingthelasttwocolumnsandthelasttworows(shadedintheprecedingtable):

Finally,we’llswapthemiddletworowsandcolumns:

Thisisexactlythemultiplicationtablefor( )weshowedonp.105,whichiswhatwewanted.

***

Nowlet’slookatanexampleofanon-categoricaltheory:allgroupsoforder4.Whilethereisonlyonenon-isomorphicgroupoforder1,2,or3,therearetwonon-isomorphicgroupsoforder4:thecyclicgroup ,whichwejustdescribed,andagroupcalledtheKleingroup.TherearetwoimportantmodelsoftheKleingroup:themultiplicativegroupofunitsmodulo8(consistingof{1,3,5,7})andthegroupofisometriestransformingarectangleintoitself(theidentitytransform,verticalsymmetry,horizontalsymmetry,and180°rotation).Thesetwokindsofgroupshavethefollowingmultiplicationtables.Sincewedon’tknowthe

individualelementsofatheory,thistimewe’llwritethemusingetomeantheidentityelementanda,b,andctomeantheirotherelements:

Inthetableontheleft,additionisouroperationandwecanthinkof“e”asbeing0(theadditiveidentity)anda,b,andcasrepresentingtheintegers1,2,and3.So,forexample,a b=1+2=3=c;thereforethevalueatrowaandcolumnbisc.Arethesetwogroupsreallydifferent(i.e.,notisomorphic),oristhereawaytotransformoneintothe

other,aswedidbefore?Inotherwords,isthereadistinguishingpropositionforgroupsoforder4?Yes,theproposition

∀x G:x2=e

istruefortheKleingroupbutfalsefor .Wecanseethisbylookingatthediagonalofthemultiplicationtable.Anotherwaytoseethattheyaredifferentistonoticethatthecyclicgroupcontainstwogenerators,whiletheKleingroupcontainsnone.

6.8ThoughtsontheChapterInthischapterweintroducedtheideaofalgebraicstructures,abstractsetsofelementsthatobeycertainproperties.Welookedatgroups,themostimportantofthesestructures,andtheirweakercousinsmonoidsandsemigroups.Thesearesummarizedinthefollowingtable,whichwe’llbeaddingtolater:

Eachrowofthetableincludesallthepropertiesofthepreviousrow,withoneormoreadditions.Wecanviewtherelationshipsbetweenthestructureslikethis:

Forexample,thediagramshowsthatamonoidisasemigroupthatalsohasanidentityelement(andtheidentityaxiom).Afewadditionalstructureswetalkedaboutaremosteasilydefinedintermsofothers:

Wealsosawhowwecouldderivepropertiesofgroups(liketheonestatedbyLagrange’sTheorem)withoutknowinganythingabouttheparticularelementsbeingmanipulated.Inotherwords,wesawhowtoderiveresultsabouttheorieswithoutspecifyingaparticularmodel.Nowwe’rereadytoputthesealgebraicstructurestopracticaluse.

7.DerivingaGenericAlgorithm

Togeneralizesomethingmeanstothinkit.Hegel,PhilosophyofRight

Inthischapterwe’lltaketheEgyptianmultiplicationalgorithmfromChapter2and,byusingthemathematicalabstractionsintroducedinthepreviouschapter,generalizeittoapplytoawidevarietyofproblemsbeyondsimplearithmetic.

7.1UntanglingAlgorithmRequirementsTwostepsarerequiredtowriteagoodpieceofcode.Thefirststepistogetthealgorithmright.Thesecondstepistofigureoutwhichsortsofthings(types)itworksfor.Now,youmightbethinkingthatyoualreadyknowthetype—it’sintorfloatorwhateveryoustartedwith.Butthatmaynotalwaysbethecase;thingschange.Nextyearsomeonemaywantthecodetoworkwithunsignedintsordoublesorsomethingelseentirely.Wewanttodesignourcodesowecanreuseitinthesedifferentsituations.Let’stakeanotherlookatthemultiply-accumulateversionoftheEgyptianmultiplicationalgorithmwe

developedinChapter2.Recallthatwearemultiplyingnbyaandaccumulatingtheresultinr;alsorecallthatwehaveapreconditionthatneithernnoracanbezero:Clickheretoviewcodeimage


while(true){

if(odd(n)){

r=r+a;

if(n==1)returnr;

}

n=half(n);

a=a+a;

}

}

Thistimewe’vewrittensomeofthecodeinaslantedtypefaceandsomeofitinaboldtypeface.Noticethattheslantedandboldpartsaredisjointed;therearenoplaceswherea“slantedvariable”anda“boldvariable”arecombinedorinteractwitheachotherinanyway.Thismeansthattherequirementsforbeingaslantedvariabledon’thavetobethesameastherequirementsforbeingaboldvariable—ortoputitinprogramminglanguageterms,theycanbedifferenttypes.Sowhataretherequirementsforeachkindofvariable?Untilnow,we’vedeclaredthevariablesas

ints,butitseemsasifthealgorithmwouldworkformanyothersimilartypesaswell.Slantedvariablesrandamustbesometypethatsupportsadding—wemightsaythattheymustbea“plusable”type.Theboldvariablenmustbeabletobecheckedforoddness,comparedwith1,andmustsupportdivisionby2(itmustbe“halvable”).Notethatdivisionby2isamuchmorerestrictedoperationthandivisioningeneral.Forexample,anglescanbedividedby2withruler-and-compassconstruction,whiledividingthemby3isimpossibleinthatframework.We’veestablishedthatrandaarethesametype,whichwe’llwriteusingthetemplatetypenameA.

Similarly,wesaidthatnisadifferenttype,whichwe’llcallN.Soinsteadofinsistingthatalltheargumentsbeoftypeint,wecannowwritethefollowingmoregenericformoftheprogram:Clickheretoviewcodeimage

template<typenameA,typenameN>

Amultiply_accumulate(Ar,Nn,Aa){

while(true){

if(odd(n)){

r=r+a;

if(n==1)returnr;

}

n=half(n);

a=a+a;

}

}

Thismakestheproblemeasier—wecanfigureouttherequirementsforAseparatelyfromtherequirementsonN.Let’sdigabitdeeperintothesetypes,startingwiththesimplerone.

7.2RequirementsonAWhatarethesyntacticrequirementsonA?Inotherwords,whichoperationscanwedoonthingsbelongingtoA?Justbylookingathowvariablesofthistypeareusedinthecode,wecanseethattherearethreeoperations:

•Theycanbeadded(inC++,theymusthaveoperator+).•Theycanbepassedbyvalue(inC++,theymusthaveacopyconstructor).•Theycanbeassigned(inC++,theymusthaveoperator=).

Wealsoneedtospecifythesemanticrequirements.Thatis,weneedtosaywhattheseoperationsmean.Ourmainrequirementisthat+mustbeassociative,whichweexpressasfollows:

A(T) ∀a,b,c T:a+(b+c)=(a+b)+c

(InEnglish,wemightreadthepartbeforethecolonlikethis:“IftypeTisanA,thenforanyvaluesa,b,andcinT,thefollowingholds:....”)Eventhough+isassociativeintheory(andinmathgenerally),thingsarenotsosimpleoncomputers.

Therearereal-worldcaseswhereassociativityofadditiondoesnothold.Forexample,considerthesetwolinesofcode:

w=(x+y)+z;

w=x+(y+z)

Supposex,y,andzareoftypeint,andzisnegative.Thenitispossiblethatforsomeverylargevalues,x+ywilloverflow,whilethiswouldnothavehappenedifweaddedy+zfirst.Theproblemarisesbecauseadditionisnotwelldefinedforallpossiblevaluesofthetypeint;wesaythat+isapartialfunction.Toaddressthisproblem,weclarifyourrequirements.Werequirethattheaxiomsholdonlyinsidethe

domainofdefinition—thatis,thesetofvaluesforwhichthefunctionisdefined.11Foramorerigoroustreatmentofthistopic,seeSection2.1ofElementsofProgrammingbyStepanovandMcJones.

***

Infact,thereareacouplemoresyntacticrequirementsthatwemissed.Theyareimpliedbycopyconstructionandassignment.Forexample,copyconstructionmeanstomakeacopythatisequaltotheoriginal.Todothis,weneedtheabilitytotestthingsbelongingtoAforequality:

•Theycanbecomparedforequality(inC++,theymusthaveoperator==).

•Theycanbecomparedforinequality(inC++,theymusthaveoperator!=).Accompanyingthesesyntacticrequirementsaresemanticrequirementsforwhatwecallequational

reasoning;equalityonatypeTshouldsatisfyourexpectations:•Inequalityisthenegationofequality.

(a≠b) ¬(a=b)

•Equalityisreflexive,symmetric,andtransitive.

a=a

a=b b=a

a=b∧b=c a=c

•Equalityimpliessubstitutability.

foranyfunctionfonT,a=b f(a)=f(b)

Thethreeaxiomsinthemiddle(reflexivity,symmetry,andtransitivity)providewhatwecallequivalence,butequationalreasoningrequiressomethingmuchstronger,soweaddthesubstitutabilityrequirement.Wehaveaspecialnamefortypesthatbehaveinthe“usualway”—regulartypes:

Definition7.1.AregulartypeTisatypewheretherelationshipsbetweenconstruction,assignment,andequalityarethesameasforbuilt-intypessuchasint.

Forexample:•Ta(b);assert(a==b);unchanged(b);•a=b;assert(a==b);unchanged(b);•Ta(b);isequivalenttoTa;a=b;

Foranextensivetreatmentofregulartypes,seeChapter1ofElementsofProgramming.Alltypesthatweuseinthisbookareregular.

***

NowwecanformalizetherequirementsonA:•Regulartype•Providesassociative+

AswesawinChapter6,algebraicstructuresthathaveabinaryassociativeoperationarecalledsemigroups(seeDefinition6.5).Also,aregulartypeguaranteestheabilitytocomparetwovaluesforequality,whichweneedforourassociativityaxiom.ThereforewecansaythatAisasemigroup.Itsoperationisaddition,sowemightbetemptedtocallitanadditivesemigroup.Butrecallthatbyconvention,additivesemigroupsareassumedtobecommutative.Sincewedon’tneedcommutativityforouralgorithm,we’llsaythatAisanoncommutativeadditivesemigroup.Thismeansthatcommutativityisnotrequired;itdoesnotmeanthatcommutativityisnotallowed.Inotherwords,every(commutative)additivesemigroupisalsoanoncommutativeadditivesemigroup.

Definition7.2.Anoncommutativeadditivesemigroupisasemigroupwheretheassociativebinaryoperationis+.

Someexamplesofnoncommutativeadditivesemigroupsarepositiveevenintegers,negativeintegers,realnumbers,polynomials,planarvectors,Booleanfunctions,andlinesegments.Theseexampleshappentoalsobeadditivesemigroups,butthatisnotalwaysthecase.Asweshallsee,+mayhavedifferentinterpretationsforthesedifferenttypes,butitwillalwaysbeassociative.Formanycenturies,thesymbol“+”hasbeenused,byconvention,tomeanacommutativeoperationas

wellasanassociativeone.Manyprogramminglanguages(e.g.,C++,Java,Python)use+forstringconcatenation,anoncommutativeoperation.Thisviolatesstandardmathematicalpractice,whichisabadidea.Themathematicalconventionisasfollows:

•Ifasethasonebinaryoperationanditisbothassociativeandcommutative,callit+.•Ifasethasonebinaryoperationanditisassociativeandnotcommutative,callit*.

20th-centurylogicianStephenKleeneintroducedthenotationabtodenotestringconcatenation(sinceinmathematics*isusuallyelided).

TheNamingPrincipleIfwearecomingupwithanameforsomething,oroverloadinganexistingname,weshouldfollowthesethreeguidelines:1.Ifthereisanestablishedterm,useit.2.Donotuseanestablishedterminconsistentlywithitsacceptedmeaning.Inparticular,overloadanoperatororfunctionnameonlywhenyouwillbepreservingitsexistingsemantics.

3.Ifthereareconflictingusages,themuchmoreestablishedonewins.ThenamevectorinSTLwastakenfromtheearlierprogramminglanguagesSchemeand

CommonLisp.Unfortunately,thiswasinconsistentwiththemucholdermeaningoftheterminmathematicsandviolatesRule3;thisdatastructureshouldhavebeencalledarray.Sadly,ifyoumakeamistakeandviolatetheseprinciples,theresultmightstayaroundforalongtime.

7.3RequirementsonNNowthatweknowthatAmustbeanoncommutativeadditivesemigroup,wecanspecifythatinourtemplateinsteadofjustsayingtypename:Clickheretoviewcodeimage

template<NoncommutativeAdditiveSemigroupA,typenameN>

Amultiply_accumulate(Ar,Nn,Aa){

while(true){

if(odd(n)){

r=r+a;

if(n==1)returnr;

}

n=half(n);

a=a+a;

}

}

Herewe’reusingNonCommutativeAdditiveSemigroupasaC++concept,asetofrequirementsontypesthatwe’lldiscussinChapter10.Insteadofsayingtypename,wenametheconceptwewishto

use.Sinceconceptsarenotyetsupportedinthelanguageasofthiswriting,we’redoingabitofpreprocessorslight-of-hand:Clickheretoviewcodeimage

#defineNonCommutativeAdditiveSemigrouptypename

Asfarasthecompilerisconcerned,Aisjustatypename,butforus,it’saNonCommutativeAdditiveSemigroup.We’llusethistrickfromnowonwhenwewanttospecifythetyperequirementsintemplates.Althoughthisbehaviorisnotneededforabstractmathematics,inprogrammingweneedourvariables

tobeconstructibleandassignable,whicharealsoguaranteedbybeingregulartypes.Fromnowon,whenwespecifyanalgebraicstructureasaconcept,wewillassumethatweareinheritingalloftheregulartyperequirements.Whatabouttherequirementsonourotherargument’stype,N?Let’sstartwiththesyntactic

requirements.Nmustbearegulartypeimplementing•half•odd•==0•==1

HerearethesemanticrequirementsonN:•even(n) half(n)+half(n)=n•odd(n) even(n–1)•odd(n) half(n–1)=half(n)•Axiom:n≤1∨half(n)=1∨half(half(n))=1∨...

WhichC++typessatisfytheserequirements?Thereareseveral:uint8_t,int8_t,uint64_t,andsoon.TheconceptthattheysatisfyiscalledNInteger.

***

Nowwecanfinallywriteafullygenericversionofourmultiply-accumulatefunctionbyspecifiyingthecorrectrequirementsforbothtypes:Clickheretoviewcodeimage

template<NoncommutativeAdditiveSemigroupA,IntegerN>

Amultiply_accumulate_semigroup(Ar,Nn,Aa){

//precondition(n>=0);

if(n==0)returnr;

while(true){

if(odd(n)){

r=r+a;

if(n==1)returnr;

}

n=half(n);

a=a+a;

}

}

We’veaddedonemorelineofcode,whichreturnsrwhenniszero.Wedothisbecausewhenniszero,wedon’tneedtodoanything.However,thesameisnottrueformultiply,aswe’llseeinamoment.

Here’sthemultiplyfunctionthatcallstheprecedingcode;ithasthesamerequirements:Clickheretoviewcodeimage

template<NoncommutativeAdditiveSemigroupA,IntegerN>

Amultiply_semigroup(Nn,Aa){

//precondition(n>0);

while(!odd(n)){

a=a+a;

n=half(n);

}

if(n==1)returna;

returnmultiply_accumulate_semigroup(a,half(n-1),a+a);

}

WecanalsoupdateourhelperfunctionsoddandhalftoworkforanyInteger:Clickheretoviewcodeimage

template<IntegerN>

boolodd(Nn){returnbool(n&0x1);}

template<IntegerN>

Nhalf(Nn){returnn>>1;}

7.4NewRequirementsOurpreconditionformultiplysaysthatnmustbestrictlygreaterthanzero.(Wemadethisassumptionearlier,sincetheGreekshadonlypositiveintegers,butnowweneedtomakeitexplicit.)Whatshouldanadditivesemigroupmultiplicationfunctionreturnwhenniszero?Itshouldbethevaluethatdoesn’tchangetheresultwhenthesemigroupoperator—addition—isapplied.Inotherwords,itshouldbetheadditiveidentity.Butanadditivesemigroupisnotrequiredtohaveanidentityelement,sowecan’tdependonthisproperty.Inotherwords,wecan’trelyontherebeinganequivalentofzero.(Remember,anolongerhastobeaninteger;itcanbeanyNoncommutativeAdditiveSemigroup,suchaspositiveintegersornonemptystrings.)That’swhyncan’tbezero.Butthereisanalternative:insteadofhavingarestrictiononthedatarequiringn>0,wecanrequire

thatanytypeweuseknowhowtodealwith0.Wedothisbychangingtheconceptrequirementonnfromadditivesemigrouptomonoid.RecallfromChapter6thatinadditiontoanassociativebinaryoperation,amonoidcontainsanidentityelementeandanidentityaxiomthatsays

xοe=eοx=x

Inparticular,we’regoingtouseanoncommutativeadditivemonoid,wheretheidentityelementiscalled“0”:

x+0=0+x=x

Thisisthemultiplyfunctionformonoids:Clickheretoviewcodeimage

template<NoncommutativeAdditiveMonoidA,IntegerN>

Amultiply_monoid(Nn,Aa){


if(n==0)returnA(0);

returnmultiply_semigroup(n,a);

}

Whatifwewanttoallownegativenumberswhenwemultiply?Weneedtoensurethat“multiplyingbyanegative”makessenseforanytypewemighthave.Thisturnsouttobeequivalenttosayingthatthetypemustsupportaninverseoperation.Again,wefindthatourcurrentrequirement—noncommutativeadditivemonoid—isnotguaranteedtohavethisproperty.Forthis,weneedagroup.Agroup,asyoumayrecallfromChapter6,includesallofthemonoidoperationsandaxioms,plusaninverseoperationx–1thatobeysthecancellationaxiom

xοx–1=x–1οx=e

Inourcase,wewantanoncommutativeadditivegroup,onewheretheinverseoperationisunaryminusandthecancellationaxiomis:

x+–x=–x+x=0

Havingstrengthenedourtyperequirements,wecanremoveourpreconditionsonntoallownegativevalues.Again,we’llwrapthelastversionofourfunctionwithournewone:Clickheretoviewcodeimage

template<NoncommutativeAdditiveGroupA,IntegerN>

Amultiply_group(Nn,Aa){

if(n<0){

n=-n;

a=-a;

}

returnmultiply_monoid(n,a);

}

7.5TurningMultiplyintoPowerNowthatourcodehasbeengeneralizedtoworkforanyadditivesemigroup(ormonoidorgroup),wecanmakearemarkableobservation:

Ifwereplace+with*(therebyreplacingdoublingwithsquaring),wecanuseourexistingalgorithmtocomputeaninsteadofn·a.

Here’stheC++functionwegetwhenweapplythistransformationtoourcodeformultiply_accumulate_semigroup:Clickheretoviewcodeimage

template<MultiplicativeSemigroupA,IntegerN>

Apower_accumulate_semigroup(Ar,Aa,Nn){


if(n==0)returnr;

while(true){

if(odd(n)){

r=r*a;

if(n==1)returnr;

}

n=half(n);

a=a*a;

}

}

Thenewfunctioncomputesran.Theonlythingsthathavechangedarehighlightedinbold.Notethatwe’vechangedtheorderofargumentsaandn,soastomatchtheorderofargumentsinstandard

mathematicalnotation(i.e.,wesayna,butan).Here’sthefunctionthatcomputespower:


template<MultiplicativeSemigroupA,IntegerN>

Apower_semigroup(Aa,Nn){


while(!odd(n)){

a=a*a;

n=half(n);

}

if(n==1)returna;

returnpower_accumulate_semigroup(a,a*a,half(n-1));

}

Herearethewrappedversionsformultiplicativemonoidsandgroups:Clickheretoviewcodeimage

template<MultiplicativeMonoidA,IntegerN>

Apower_monoid(Aa,Nn){


if(n==0)returnA(1);

returnpower_semigroup(a,n);

}

template<MultiplicativeGroupA,IntegerN>

Apower_group(Aa,Nn){

if(n<0){

n=-n;

a=multiplicative_inverse(a);

}

returnpower_monoid(a,n);

}

Justasweneededanadditiveidentity(0)forourmonoidmultiply,soweneedamultiplicativeidentity(1)inourmonoidpowerfunction.Also,justasweneededanadditiveinverse(unaryminus)forourgroupmultiply,soweneedamultiplicativeinverseforourgrouppowerfunction.There’snobuilt-inmultiplicativeinverse(reciprocal)operationinC++,butit’seasytowriteone:Clickheretoviewcodeimage

template<MultiplicativeGroupA>

Amultiplicative_inverse(Aa){

returnA(1)/a;

}

7.6GeneralizingtheOperationWe’veseenexamplesoftwosemigroups—additiveandmultiplicative—eachwithitsassociatedoperation(+and*,respectively).Thefactthatwecouldusethesamealgorithmforbothiswonderful,butitwasannoyingtohavetowritedifferentversionsofthesamecodeforeachcase.Inreality,therecouldbemanysuchsemigroups,eachwithitsassociativeoperations(forexample,multiplicationmod7)thatworkonthesametypeT.Ratherthanhavinganotherversionforeveryoperationwewanttouse,wecangeneralizetheoperationitself,justaswegeneralizedthetypesoftheargumentsbefore.Infact,therearemanysituationswhereweneedtopassanoperationtoanalgorithm;youmayhaveseenexamplesofthisinSTL.Here’stheaccumulateversionofourpowerfunctionforanarbitrarysemigroup.Westillrefertothe

functionitcomputesas“power,”eventhoughtheoperationit’srepeatedlyapplyingmaynotnecessarilybemultiplication.Clickheretoviewcodeimage

template<RegularA,IntegerN,SemigroupOperationOp>

//requires(Domain<Op,A>)

Apower_accumulate_semigroup(Ar,Aa,Nn,Opop){


if(n==0)returnr;

while(true){

if(odd(n)){

r=op(r,a);

if(n==1)returnr;

}

n=half(n);

a=op(a,a);

}

}

Noticethatwe’veaddeda“requires”commenttoourtemplatethatsaysthedomainofoperationOpmustbeA.IffutureversionsofC++supportconcepts,thiscommentcouldbeturnedintoastatement(similartoanassertion)thatthecompilercouldusetoensurethecorrectrelationshipholdsbetweenthegiventypes.Asitis,we’llhavetomakesureasprogrammersthatwecallthisfunctiononlywithtemplateargumentsthatsatisfytherequirement.Also,sincewenolongerknowwhichkindofsemigrouptomakeA—itcouldbeadditive,

multiplicative,orsomethingelsealtogether,dependingonOp—wecanrequireonlythatAbearegulartype.The“semigroupness”willcomefromtherequirementthatOpbeaSemigroupOperation.Wecanusethisfunctiontowriteaversionofpowerforanarbitrarysemigroup:


template<RegularA,IntegerN,SemigroupOperationOp>


Apower_semigroup(Aa,Nn,Opop){


while(!odd(n)){

a=op(a,a);

n=half(n);

}

if(n==1)returna;

returnpower_accumulate_semigroup(a,op(a,a),

half(n-1),op);

}

Aswedidbefore,wecanextendthefunctiontomonoidsbyaddinganidentityelement.Butsincewedon’tknowinadvancewhichoperationwillbepassed,wehavetoobtaintheidentityfromtheoperation:Clickheretoviewcodeimage

template<RegularA,IntegerN,MonoidOperationOp>


Apower_monoid(Aa,Nn,Opop){


if(n==0)returnidentity_element(op);

returnpower_semigroup(a,n,op);

}

Hereareexamplesofidentity_elementfunctionsfor+and*:Clickheretoviewcodeimage

template<NoncommutativeAdditiveMonoidT>

Tidentity_element(std::plus<T>){returnT(0);}

template<MultiplicativeMonoidT>

Tidentity_element(std::multiplies<T>){returnT(1);}

Eachofthesefunctionsspecifiesthetypeoftheobjectitexpectstobecalledwith,butdoesn’tnameit,sincetheobjectisneverused.Thefirstonesays,“Theadditiveidentityis0.”Ofcourse,therewillbedifferentidentityelementsfordifferentmonoids—forexample,themaximumvalueofthetypeTformin.

***

Toextendpowertogroups,weneedaninverseoperation,whichisitselfafunctionofthespecifiedGroupOperation:Clickheretoviewcodeimage

template<RegularA,IntegerN,GroupOperationOp>


Apower_group(Aa,Nn,Opop){

if(n<0){

n=-n;

a=inverse_operation(op)(a);

}

returnpower_monoid(a,n,op);

}

Examplesofinverse_operationlooklikethis:Clickheretoviewcodeimage

template<AdditiveGroupT>

std::negate<T>inverse_operation(std::plus<T>){

returnstd::negate<T>();

}

template<MultiplicativeGroupT>

reciprocal<T>inverse_operation(std::multiplies<T>){

returnreciprocal<T>();

}

STLalreadyhasanegatefunction,but(duetoanoversight)hasnoreciprocal.Sowe’llwriteourown.We’lluseafunctionobject—aC++objectthatprovidesafunctiondeclaredbyoperator()andisinvokedlikeafunctioncall,usingthenameoftheobjectasthefunctionname.Tolearnmoreaboutfunctionobjects,seeAppendixC.Clickheretoviewcodeimage

template<MultiplicativeGroupT>

structreciprocal{

Toperator()(constT&x)const{

returnT(1)/x;

}

};

Thisisjustageneralizationofthemultiplicative_inversefunctionwewroteintheprevioussection.2

2Thistimewedonotneedapreconditionpreventingxfrombeingzero,becauseMultiplicativeGroupdoesnotcontainanon-invertiblezeroelement.IfinpracticewehaveatypesuchasdoublethatotherwisewouldsatisfytherequirementsofMultiplicativeGroupifitdidnotcontainazero,wecanaddapreconditiontoeliminatethatcase.

ReductionThepoweralgorithmisnottheonlyimportantalgorithmdefinedonsemigroups.Anotherkeyalgorithmisreduction,inwhichabinaryoperationisappliedsuccessivelytoeachelementofasequenceanditspreviousresult.Twocommonlyseenexamplesofthisinmathematicsarethesummation(∑)functionfor

additivesemigroupsandtheproduct(∏)functionformultiplicativesemigroups.Wecangeneralizethistoanarbitrarysemigroup.Thisgeneralizedversionofreductionwasinventedin1962bycomputerscientistKenIversonin

hislanguageAPL.InAPLterminology,the/representedthereductionoperator.Forexample,summationofasequenceisexpressedas+/123

Theideaofreductionhasappearedinmanycontextssincethen.JohnBackus,inventorofthefirsthigh-levelprogramminglanguage,includedasimilaroperatorcalledinsertinhislanguageFPin1977.(Hecalledoperators“functionalforms.”)Anearlypaperongenericprogramming,“OperatorsandAlgebraicStructures,”byKapur,Musser,andStepanov,extendedtheideatoparallelreductionin1981andclarifiedtherelationshiptoassociativeoperations.ThelanguageCommonLisp,popularinthe1980sforartificialintelligenceapplications,includedareducefunction.Google’sMapReducesystem,anditsopen-sourcevariantHadoop,isacurrentpracticalapplicationoftheseideas.

7.7ComputingFibonacciNumbersNote:Thissectionassumessomebasicknowledgeoflinearalgebra.Therestofthebookdoesnotdependonthematerialcoveredhere,andthissectionmaybeskippedwithoutaffectingthereader’sunderstanding.InChapter4wemettheearly13th-centurymathematicianLeonardoPisano,oftenknowntodayasFibonacci.Oneofthethingshe’sbestknownforisafamousproblemheposed:ifwestartwithonepairofrabbits,howmanypairswillwehaveafteracertainnumberofmonths?Tosimplifytheproblem,Leonardomadesomeassumptions:theoriginalpairofrabbitsandeverylitterafterwardconsistsofamaleandafemale;femalerabbitstakeonemonthtoreachsexualmaturityandhaveonelitterpermonthafterthat;rabbitsliveforever.Initiallywehave1pairofrabbits.Atthestartofmonth2,therabbitsmate,butwestillhaveonly1

pair.Atthestartofmonth3,thefemalegivesbirth,sowehave2pairs.Atthestartofmonth4,theinitialfemalegivesbirthanothertime,sowehave3pairs.Atthestartofmonth5,theinitialfemalegivesbirthagain,butsodoesthefemaleborninmonth3,sonowwehave5pairs,andsoon.Ifwesaythatwehad0rabbitsinmonth0(beforetheexperimentbegan),thenthenumberofpairsineachmonthlookslikethis:

0,1,1,2,3,5,8,13,21,34...

Eachmonth’spopulationcanbeobtainedsimplybyaddingthepopulationsofeachoftheprevioustwomonths.Today,elementsofthissequencearecalledFibonaccinumbers,andsuchasequenceisdefinedformallylikethis:

F0=0

F1=1

Fi=Fi–1+Fi–2

HowlongdoesittaketocomputethenthFibonaccinumber?The“obvious”answerisn–2,buttheobviousansweriswrong.ThenaivewaytoimplementthisinC++issomethinglikethis:


intfib0(intn){

if(n==0)return0;

if(n==1)return1;

returnfib0(n-1)+fib0(n-2);

}

However,thiscodedoesanawfullotofrepeatedwork.Considerthecalculationstocomputefib0(5):

Eveninthissmallexample,thecomputationdoes17additions,andjustthequantityF1+F0isrecomputed5times.

Exercise7.1.Howmanyadditionsareneededtocomputefib0(n)?

Recomputingthesamethingoverandoverisunacceptable,andthere’snoexcuseforcodelikethat.Wecaneasilyfixthecodebykeepingarunningstateoftheprevioustworesults:Clickheretoviewcodeimage

intfibonacci_iterative(intn){

if(n==0)return0;

std::pair<int,int>v={0,1};

for(inti=1;i<n;++i){

v={v.second,v.first+v.second};

}

returnv.second;

}

Thisisanacceptablesolution,whichtakesO(n)operations.Infact,giventhatwewanttofindthenthelementofasequence,itmightappeartobeoptimal.ButtheamazingthingisthatwecanactuallycomputethenthFibonaccinumberinO(logn)operations,whichformostpracticalpurposesislessthan64.SupposewerepresentthecomputationofthenextFibonaccinumberfromtheprevioustwousingthe

followingmatrixequation:3

3AbriefrefresheronmatrixmultiplicationmaybefoundatthebeginningofSection8.5.

ThenthenthFibonaccinumbermaybeobtainedby

Inotherwords,wecancomputethenthFibonaccinumberbyraisingacertainmatrixtoapower.Aswewillsee,matrixmultiplicationcanbeusedtosolvemanyproblems.Matricesareamultiplicativemonoid,sowealreadyhaveanO(logn)algorithm—ourpoweralgorithmfromSection7.6.

Exercise7.2.ImplementcomputingFibonaccinumbersusingpower.

Thisisaniceapplicationofourpoweralgorithm,butcomputingFibonaccinumbersisn’ttheonlythingwecando.Ifwereplacethe+withanarbitrarylinearrecurrencefunction,wecanusethesametechniquetocomputeanylinearrecurrence.

Definition7.3.Alinearrecurrencefunctionoforderkisafunctionfsuchthat

Definition7.4.Alinearrecurrencesequenceisasequencegeneratedbyalinearrecurrencefrominitialkvalues.

TheFibonaccisequenceisalinearrecurrencesequenceoforder2.Foranylinearrecurrencesequence,wecancomputethenthstepbydoingmatrixmultiplicationusing

ourpoweralgorithm:

Thelineof1sjustbelowthediagonalprovidesthe“shifting”behavior,sothateachvalueinthesequencedependsonthepreviousk.

7.8ThoughtsontheChapterWebeganthischapterbyanalyzingtherequirementsonourcodefromChapter2,abstractingthealgorithmstouseanassociativeoperationonarbitrarytypes.Wewereabletorewritethecodesothatitisdefinedoncertainalgebraicstructures:semigroups,monoids,andgroups.Next,wedemonstratedthatthealgorithmcouldbegeneralized,firstfrommultiplicationtopower,then

toarbitraryoperationsonouralgebraicstructures.We’llusethisgeneralizedpoweralgorithmagainlateroninthebook.Theprocesswewentthrough—takinganefficientalgorithm,generalizingit(withoutlosingefficiency)

sothatitworksonabstractmathematicalconcepts,andthenapplyingittoavarietyofsituations—istheessenceofgenericprogramming.

8.MoreAlgebraicStructures

ForEmmyNoether,relationshipsamongnumbers,functions,andoperationsbecametransparent,amenabletogeneralization,andproductiveonlyaftertheyhavebeen

dissociatedfromanyparticularobjectsandhavebeenreducedtogeneralconceptualrelationships.

B.L.vanderWaerdenWhenwefirstintroducedEuclid’salgorithminChapter4,itwasforcomputingthegreatestcommonmeasureoflinesegments.Thenweshowedhowtoextendittoworkforintegers.Doesitworkforotherkindsofmathematicalentities?Thisisthequestionwe’llbeinvestigatinginthischapter.Aswe’llsee,attemptstoansweritledtoimportantdevelopmentsinabstractalgebra.We’llalsoshowhowsomeofthesenewalgebraicstructuresenablenewprogrammingapplications.

8.1Stevin,Polynomials,andGCDSomeofthemostimportantcontributionstomathematicswereduetooneofitsleast-knownfigures,the16th-centuryFlemishmathematicianSimonStevin.Inadditiontohiscontributionstoengineering,physics,andmusic,Stevinrevolutionizedthewaywethinkaboutandoperateonnumbers.AsBartelvanderWaerdenwroteinhisHistoryofAlgebra:

[With]onestroke,theclassicalrestrictionsof“numbers”tointegersortorationalfractionswaseliminated.[Stevin’s]generalnotionofarealnumberwasacceptedbyalllaterscientists.

Inhis1585pamphletDeThiende(“TheTenth”),publishedinEnglishasDisme:TheArtofTenths,or,DecimallArithmetike,Stevinintroducesandexplainstheuseofdecimalfractions.ThiswasthefirsttimeanyoneinEuropeproposedusingpositionalnotationintheotherdirection—fortenths,hundredths,andsoon.Disme(pronounced“dime”)wasoneofthemostwidelyreadbooksinthehistoryofmathematics.ItwasoneofThomasJefferson’sfavorites,andisthereasonwhyU.S.currencyhasacoincalleda“dime”andusesdecimalcoinageratherthantheBritishpounds,shillings,andpenceinuseatthetime.

SimonStevin(1548–1620)

SimonStevinwasborninBruges,Flanders(nowpartofBelgium),butlatermovedtoLeideninHolland.Priortothisperiod,theNetherlandsprovinces(whichincludedbothFlandersandHolland)werepartoftheSpanishempire,ledbyitsHapsburgkingsandheldtogetherbyitsinvincibleprofessionalarmy.In1568,theDutchbeganawarofindependence,unitedbyacommoncultureandlanguage,ultimatelycreatingarepublicandthenanempireoftheirown.Stevin,aDutchpatriotandmilitaryengineer,joinedintherebellionandbecamefriendswithPrinceMauriceofOrange,itsleader.InparthelpedbyStevin’sdesignsforfortificationsandhiscleveruseofasystemofsluicestofloodinvadingSpanishtroops,therebellionsucceeded,creatinganindependentDutchnationcalledtheUnitedProvinces.ThisbegantheDutch“GoldenAge,”whenthecountrybecameacultural,scientific,andcommercialpower,rememberedtodayinthegreatworksofartistslikeRembrandtandVermeer.StevinwasatrueRenaissanceman,withmanyfar-reachinginterestsbeyondmilitaryengineering.WhileStevin’sofficialjobformostofhiscareerwasquartermaster-generalofthearmy,inpracticehealsobecameakindofscienceadvisortoPrinceMaurice.Inadditiontohisinventionofdecimalfractions,polynomials,andothermathematicalwork,Stevinmademanycontributionstophysics.Hestudiedstatics,realizingthatforcescouldbeaddedusingwhatwenowcallthe“parallelogramofforces,”andpavedthewayfortheworkofNewtonandothers.Hediscoveredtherelationshipoffrequenciesinadjacentnotesofa12-tonemusicalscale.Stevinevendemonstratedconstantaccelerationoffallingobjects,afewyearsbeforeGalileo.StevinwasalsoanardentproponentoftheDutchlanguage,whichuntilthenhadbeenconsideredasortofsecond-ratedialectofGerman.HehelpedPrinceMauricecreateanengineeringschoolwhereteachingwasinDutch,andwrotetextbooksinthelanguage.Heusedwordfrequencyanalysisandwordlengthto“prove”thatitwasthebest(mostefficient)languageandthebesttodoscience.StevininsistedonpublishinghisownresultsinDutchratherthanLatin,whichmayexplainwhyhedidnotbecomebetterknownoutsidehiscountry.

InDisme,Stevinexpandsthenotionofnumbersfromintegersandfractionsto“thatwhichexpresseththequantitieofeachthing”(asanEnglishtranslationatthetimeputit).Essentially,Stevininventedtheentireconceptofrealnumbersandthenumberline.Anyquantitycouldgoonthenumberline,including

negativenumbers,irrationalnumbers,andwhathecalled“inexplicable”numbers(bywhichhemayhavemeanttranscendentalnumbers).Ofcourse,Stevin’sdecimalrepresentationshadtheirowndrawbacks,particularlytheneedtowriteaninfinitenumberofdigitstoexpressasimplevalue,suchas

Stevin’srepresentationenabledthesolutionofpreviouslyunsolvableproblems.Forexample,heshowedhowtocomputecuberoots,whichhadgiventheGreekssomuchtrouble.HisreasoningwassimilartowhateventuallybecameknownastheIntermediateValueTheorem(seethe“OriginsofBinarySearch”sidebarinSection10.8),whichsaysthatifacontinuousfunctionisnegativeatonepointandpositiveatanother,thentheremustbeanintermediatepointwhereitsvalueiszero.Stevin’sideawastofindthenumber(initially,aninteger)wherethefunctiongoesfromnegativetopositive,thendividetheintervalbetweenthatnumberandthenextintotenths,andrepeattheprocesswiththetenths,hundredths,andsoon.Herealizedthatby“zoomingin,”anysuchproblemcouldbesolvedtowhateverdegreeofaccuracywasneeded,orasheputit,“onemayobtainasmanydecimalsof[thetruevalue]asonemaywishandcomeindefinitelyneartoit.”AlthoughStevinsawhowtorepresentanynumberasapointalongaline,hedidnotmaketheleapto

showingpairsofnumbersaspointsonaplane.Thatinvention—whatwenowcallCartesiancoordinates—camefromthegreatFrenchphilosopherandmathematicianRenéDescartes(RenatusCartesiusinLatin).

***

Stevin’snextgreatachievementwastheinventionof(univariate1)polynomials,alsointroducedin1585,inabookcalledArithmétique.Considerthisexpression:

4x4+7x3–x2+27x–3

1Univariatepolynomialsarepolynomialswithasinglevariable.Fortherestofthischapter,wewillassume“polynomial”meansunivariatepolynomial.

PriortoStevin’swork,theonlywaytoconstructsuchanumberwasbyperforminganalgorithm:Takeanumber,raiseittothe4thpower,multiplyitby4,andsoon.In,fact,onewouldneedadifferentalgorithmforeverypolynomial.Stevinrealizedthatapolynomialissimplyafinitesequenceofnumbers:{4,7,–1,27,–3}fortheprecedingexample.Inmoderncomputerscienceterms,wemightsaythatStevinwasthefirsttorealizethatcodecouldbetreatedasdata.WithStevin’sinsight,wecanpasspolynomialsasdatatoagenericevaluationfunction.We’llwrite

onethattakesadvantageofHorner’srule,whichusesassociativitytoensurethatweneverhavetomultiplypowersofxhigherthan1:

4x4+7x3–x2+27x–3=(((4x+7)x–1)x+27)x–3

Forapolynomialofdegreen,weneednmultiplicationsandn–madditions,wheremisthenumberofcoefficientsequaltozero.Usuallywewillsettlefordoingnadditions,sincecheckingwhethereachadditionisneededismoreexpensivethanjustdoingtheaddition.Usingthisrule,wecanimplementapolynomialevaluationfunctionlikethis,wheretheargumentsfirstandlastspecifytheboundsofasequenceofcoefficientsofthepolynomial:Clickheretoviewcodeimage

template<InputIteratorI,SemiringR>

Rpolynomial_value(Ifirst,Ilast,Rx){

if(first==last)returnR(0);

Rsum(*first);

while(++first!=last){

sum*=x;

sum+=*first;

}

returnsum;

}

Let’sthinkabouttherequirementsonthetypessatisfyingIandR.Iisaniterator,becausewewanttoiterateoverthesequenceofcoefficients.2Butthevaluetypeoftheiterator(thetypeofthecoefficientsofthepolynomial)doesnothavetobeequaltothesemiring3R(thetypeofthevariablexinthepolynomial).Forexample,ifwehaveapolynomiallikeax2+bwherethecoefficientsarerealnumbers,thatdoesn’tmeanxhastobearealnumber;infact,itcouldbesomethingcompletelydifferent,likeamatrix.

2We’llexplainiteratorsmoreformallyinChapter10,butfornowwecanthinkofthemasgeneralizedpointers.3Asemiringisanalgebraicstructurewhoseelementscanbeaddedandmultipliedandhasdistributivity.Wewillgiveitsformaldefinition

inSection8.5.

Exercise8.1.WhataretherequirementsonRandthevaluetypeoftheiterator?Inotherwords,whataretherequirementsoncoefficientsofpolynomialsandontheirvalues?

Stevin’sbreakthroughallowedpolynomialstobetreatedasnumbersandtoparticipateinnormalarithmeticoperations.Toaddorsubtractpolynomials,wesimplyaddorsubtracttheircorrespondingcoefficients.Tomultiply,wemultiplyeverycombinationofelements.Thatis,ifaiandbiaretheithcoefficientsofthepolynomialsbeingmultiplied(startingfromthelowest-orderterm)andciistheithcoefficientoftheresult,then

Todividepolynomials,weneedthenotionofdegree.

Definition8.1.Thedegreeofapolynomialdeg(p)istheindexofthehighestnonzerocoefficient(orequivalently,thehighestpowerofthevariable).

Forexample:deg(5)=0deg(x+3)=1deg(x3+x–7)=3

Nowwecandefinedivisionwithremainder:

Definition8.2.Polynomialaisdivisiblebypolynomialbwithremainderriftherearepolynomialsqandrsuchthat

a=bq+r∧deg(r)<deg(b)

(Inthisequation,qrepresentsthequotientofa÷b.)Doingpolynomialdivisionwithremainderisjustlikedoinglongdivisionofnumbers:

Exercise8.2.Provethatforanytwopolynomialsp(x)andq(x):

1.p(x)=q(x)·(x–x0)+r p(x0)=r

2.p(x0)=0 p(x)=q(x)·(x–x0)

***

StevinrealizedthathecouldusethesameEuclideanalgorithm(theonewelookedatintheendofSection4.6)tocomputetheGCDoftwopolynomials;allwereallyneedtodoischangethetypes:Clickheretoviewcodeimage

polynomial<real>gcd(polynomial<real>a,polynomial<real>b){

while(b!=polynomial<real>(0)){

a=remainder(a,b);

std::swap(a,b);

}

returna;

}

Theremainderfunctionthatweuseimplementsthealgorithmforpolynomialdivision,althoughwedonotcareaboutthequotient.ThepolynomialGCDisusedextensivelyincomputeralgebrafortaskssuchassymbolicintegration.Stevin’srealizationistheessenceofgenericprogramming:analgorithminonedomaincanbe

appliedinanothersimilardomain.JustasinSection4.7,weneedtoshowthatthealgorithmworks—specifically,thatitterminatesand

computestheGCD.Toshowthatthealgorithmterminates,weneedtoshowthatitcomputestheGCDinafinitenumberof

steps.Sinceitrepeatedlyperformspolynomialremainder,weknowbyDefinition8.2that

deg(r)<deg(b)

Soateverystep,thedegreeofrisreduced.Sincedegreeisanon-negativeinteger,thedecreasingsequencemustbefinite.ToshowthealgorithmcomputestheGCD,wecanusethesameargumentfromSection4.7;itappliesto

polynomialsaswellasintegers.

Exercise8.3(fromChrystal,Algebra).FindtheGCDofthefollowingpolynomials:

8.2GöttingenandGermanMathematicsInthe18thand19thcenturies,startinglongbeforeGermanyexistedasaunifiedcountry,Germancultureflourished.ComposerslikeBach,Mozart,andBeethoven,poetslikeGoetheandSchiller,andphilosopherslikeKant,Hegel,andMarxwerecreatingtimelessworksofdepthandbeauty.GermanuniversitiescreatedauniqueroleforGermanprofessorsascivilservantsboundbytheircommitmenttothetruth.Eventuallythissystemwouldproducethegreatestmathematiciansandphysicistsoftheirage,manyofthemteachingorstudyingattheUniversityofGötttingen.

TheUniversityofGöttingenThecenterofGermanmathematicswasaseeminglyunlikelyplace:theUniversityofGöttingen.UnlikemanygreatEuropeanuniversitiesthathadstartedhundredsofyearsearlierinmedievaltimes,Göttingenwasrelativelyyoung,foundedin1734.AndthecityofGöttingenwasnotamajorpopulationcenter.Despitethis,theUniversityofGöttingenwashometoanastonishingseriesoftopmathematicians,includingGauss,Riemann,Dirichlet,Dedekind,Klein,Minkowski,andHilbert,someofwhomwewilldiscusslaterinthebook.Bytheearly20thcenturyitscommunityofphysicistswasequallyimpressive,includingquantumtheoristsMaxBornandWernerHeisenberg.Göttingen’sgreatnesswasdestroyedin1933bytheNazis,whoexpelledallJewsfromthefacultyandstudentbody—includingmanyofthetopphysicistsandmathematicians.Someyearslater,theNaziMinisterofEducationaskedthegreatGermanmathematicianDavidHilbert,“HowismathematicsinGöttingennowthatithasbeenfreedofJewishinfluence?”Hilbertreplied,“MathematicsinGöttingen?Thereisnoneanymore.”

PerhapsthemostimportantmathematiciantocomeoutofGöttingenwasCarlFriedrichGauss,whowasthefounderofGermanmathematicsinthemodernsense.Amonghismanyaccomplishmentswashisseminalworkonnumbertheory,describedinhis1801bookDisquisitionesArithmeticae(“InvestigationsofArithmetic”).Gauss’sbookistonumbertheorywhatEuclid’sElementsistogeometry—thefoundation

onwhichalllaterworkinthefieldisbased.Amongotherresults,itincludestheFundamentalTheoremofArithmetic,whichstatesthateveryintegerhasauniqueprimefactorization.

CarlFriedrichGauss(1777–1855)

CarlFriedrichGaussgrewupinBrunswick,Germany,andwasrecognizedasachildprodigyearlyinhislife.Accordingtoafamousstory,hiselementaryschoolteachertriedtokeeptheclassoccupiedbyaskingthemtoaddalltheintegersfrom1to100.Nine-year-oldGausscameupwiththeanswerinseconds;hehadobservedthatthefirstandlastnumbersaddedto101,asdidthesecondandsecondtolast,andsoon,sohesimplymultiplied101by50.Gauss’stalentscametotheattentionoftheDukeofBrunswick,whopaidfortheyoungstudent’seducationstartingatage14,firstataschoolinhishometownandlaterattheUniversityofGöttingen.Initially,Gaussconsideredacareerinclassics,which,unlikemathematics,wasoneofthestrengthsoftheuniversityatthetime.However,hecontinuedtodomathematicsonhisown,andin1796hemadeadiscoverythathadeludedmathematicianssinceEuclid:howtoconstructa17-sidedregularpolygonusingarulerandacompass.Infact,Gausswentfurther,provingthatconstructionofaregularp-gonwaspossibleforprimeponlyifpisaFermatprime—thatis,anumberoftheform22k+1.Thisbreakthroughconvincedhimtopursueacareerinmathematics.Infact,hewassoproudofthediscoverythatheplannedtohavethe17-gonengravedonhistomb.InhisPh.D.dissertation,GaussprovedwhatbecameknownastheFundamentalTheoremofAlgebra,whichstatesthateverynonconstantpolynomialwithcomplexcoefficientshasacomplexroot.Gausswrotehisgreatnumbertheorytreatise,DisquisitionesArithmeticae,whilehewasstillastudent,andhaditpublishedin1801whenhewasjust24.Whilegreatmathematiciansthroughouthistory,suchasEuclid,Fermat,andEuler,hadworkedonnumbertheory,Gausswasthefirsttocodifythefieldandplaceitonaformalfoundationbyintroducingmodulararithmetic.Disquisitionesisstillstudiedtoday,and,infact,someimportantdevelopmentsin20th-centurymathematicsweretheresultofcarefulstudyofGauss’swork.Gaussbecameworldfamousin1801whenhepredictedthelocationoftheasteroidCeresusinghismethodofleastsquares.Becauseofthisresult,hewaslaterappointeddirectoroftheastronomicalobservatoryatGöttingen.Thispatternofpracticalproblemsinspiringhismathematicalresults

continuedthroughouthiscareer.Hisworkongeodesy(thescienceofmeasuringtheEarth)ledtohisinventionofthenewfieldofdifferentialgeometry.HisobservationsoferrorsindataledtotheideaoftheGaussiandistributioninstatistics.ThroughoutGauss’scareer,hechosehisworkverycarefully,andpublishedonlywhatheconsideredtobehisbestresults—asmallfractionofhispotentialoutput.Oftenhewoulddelaypublication,sometimesbyyears,untilhefoundtheperfectwaytoproveaparticularresult.Hismottowas“Few,butripe.”Becauseofthebreadthanddepthofhiscontributions,GausswasknownasPrincepsMathematicorum,thePrinceofMathematicians.

AnotherofGauss’sinnovationswasthenotionofcomplexnumbers.Mathematicianshadusedimaginarynumbers(xiwherei2=–1)forover200years,butthesenumberswerenotwellunderstoodandwereusuallyavoided.Thesamewastrueforthefirst30yearsofGauss’scareer;wehaveevidencefromhisnotebooksthatheusedimaginarynumberstoderivesomeofhisresults,butthenhereconstructedtheproofssothepublishedversionswouldnotmentioni.(“Themetaphysicsofiisverycomplicated,”hewroteinaletter.)Butin1831,Gausshadaprofoundinsight:herealizedthatnumbersoftheformz=x+yicouldbe

viewedaspoints(x,y)onaCartesianplane.Thesecomplexnumbers,hesaw,werejustaslegitimateandself-consistentasanyothernumbers.Hereareafewdefinitionsandpropertieswe’lluseforcomplexnumbers:

Theabsolutevalueofacomplexnumberzisthelengthofthevectorzonthecomplexplane,whiletheargumentistheanglebetweentherealaxisandthevectorz.Forexample,|i|=1andarg(i)=90°.JustasStevindidforpolynomials,Gaussdemonstratedthatcomplexnumberswereinfactfull-fledged

numberscapableofsupportingordinaryarithmeticoperations:

Multiplyingtwocomplexnumberscanalsobedonebyaddingtheargumentsandmultiplyingtheabsolutevalues.Forexample,ifwewanttofind ,weknowitwillalsohaveanabsolutevalueof1andanargumentof45°(since1·1=1and45+45=90).

***

GaussalsodiscoveredwhatarenowcalledGaussianintegers,whicharecomplexnumberswithintegercoefficients.Gaussianintegershavesomeinterestingproperties.Forexample,theGaussianinteger2is

notprime,sinceitcanbeexpressedastheproductoftwootherGaussianintegers,1+iand1–i.Wecan’tdofulldivisionwithGaussianintegers,butwecandodivisionwithremainder.Tocompute

theremainderofz1andz2,Gaussproposedthefollowingprocedure:

1.Constructagridonthecomplexplanegeneratedbyz2,iz2,−iz2,and−z2.2.Findasquareinthegridcontainingz1.3.Findavertexwofthegridsquareclosesttoz1.

4.z1–wistheremainder.

Gaussrealizedthatwiththisremainderfunction,hecouldapplyEuclid’sGCDalgorithmtocomplexintegers,aswe’vedonehere:Clickheretoviewcodeimage

complex<integer>gcd(complex<integer>a,complex<integer>b){

while(b!=complex<integer>(0)){

a=remainder(a,b);

std::swap(a,b);

}

returna;

}

Theonlythingwe’vechangedarethetypes.

***

Gauss’sworkwasextendedbyanotherGöttingenprofessor,PeterGustavLejeune-Dirichlet.WhileGauss’scomplexnumberswereoftheform(inDirichlet’sterminology) ,Dirichletrealizedthatthiswasaspecialcaseof whereadidnothavetobe1,andthatdifferentpropertiesfollowedfromtheuseofdifferentvalues.Forexample,thestandardGCDalgorithmworksonnumbersofthisformwhena=1,butitfailswhena=5sincethereendupbeingnumbersthatdon’thaveauniquefactorization.Forexample:

ItturnsoutthatifEuclid’salgorithmworks,thenthereisauniquefactorization.Sincewehavenouniquefactorizationhere,thenEuclid’salgorithmdoesn’tworkinthiscase.Dirichlet’sgreatestresultwashisproofthatifaandbarecoprime(thatis,ifgcd(a,b)=1),thenthere

areinfinitelymanyprimesoftheformak+b.MostofDirichlet’sresultsweredescribedinthesecondgreatbookonnumbertheory,appropriately

calledVorlesungenüberZahlentheorie(“LecturesonNumberTheory”).Thebookcontainsthefollowingimportantinsight,whichweusedinourepigraphforChapter4:

[T]hewholestructureofnumbertheoryrestsonasinglefoundation,namelythealgorithmforfindingthegreatestcommondivisoroftwonumbers.Allthesubsequenttheorems...arestillonlysimpleconsequencesoftheresultofthisinitialinvestigation....

ThebookwasactuallywrittenandpublishedafterDirichlet’sdeathbyhisyoungerGöttingencolleague,RichardDedekind,basedonDedekind’snotesfromDirichlet’slectures.DedekindwassomodestthathepublishedthebookunderDirichlet’sname,evenafteraddingmanyadditionalresultsofhisowninlatereditions.Unfortunately,Dedekind’smodestyhurthiscareer;hefailedtogettenureatGöttingenandendeduponthefacultyofaminortechnicaluniversity.DedekindobservedthatGaussianintegersandDirichlet’sextensionsofthemwerespecialcasesofa

moregeneralconceptofalgebraicintegers,whicharelinearintegralcombinationsofrootsofmonicpolynomials(polynomialswherethecoefficientofthehighest-ordertermis1)withintegercoefficients.Wesaythatthesepolynomialsgeneratethesesetsofalgebraicintegers.Forexample:

Dedekind’sworkonalgebraicintegerscontainedalmostallthefundamentalbuildingblocksofmodernabstractalgebra.ButitwouldtakeanothergreatGöttingenmathematician,EmmyNoether,tomakethebreakthroughtofullabstraction.

8.3NoetherandtheBirthofAbstractAlgebraEmmyNoether’srevolutionaryinsightwasthatitispossibletoderiveresultsaboutcertainkindsofmathematicalentitieswithoutknowinganythingabouttheentitiesthemselves.Inprogrammingterms,wewouldsaythatNoetherrealizedthatwecoulduseconceptsinouralgorithmsanddatastructures,withoutknowinganythingaboutwhichspecifictypeswouldbeused.Inaveryrealsense,Noetherprovidedthetheoryforwhatwenowcallgenericprogramming.Noethertaughtmathematicianstoalwayslookforthemostgeneralsettingofanytheorem.Inasimilarway,genericprogrammingdefinesalgorithmsintermsofthemostgeneralconcepts.

EmmyNoether(1882–1935)

EmmyNoether(pronouncedalmostlike“Nerter,”butwithoutfinishingthefirst“r”sound)wasbornintoanacademicGerman-Jewishfamily.HerfatherwasadistinguishedprofessorofmathematicsattheUniversityofErlangen.Althoughitwasveryunusualforwomenatthetime,Noetherwasabletostudyattheuniversityandgotadoctorateinmathematicsin1907.ShethenstayedonforseveralyearsatErlangen,assistingherfatherandteachingwithoutapositionorsalary.Womenhadbeenexcludedfromacademiccareersforcenturies.WiththesingleexceptionofSofiaKovalevskaya,aRussianmathematicianwhobecameaprofessorinStockholmin1884,therewerenowomeninfacultypositionsinmathematicsatuniversitiesatthetime.Twoofthegreatestmathematiciansoftheday,FelixKleinandDavidHilbert,recognizedNoether’stalent,andfeltthatshedeservedanacademicposition.Theyalsobelievedasamatterofprinciplethatwomenshouldnotbeexcludedfrommathematics.TheyarrangedforNoethertocometoGöttingenin1915.Unfortunately,shestillwasnotofficiallyallowedtoteach;thefacultyresistedherappointment.Forthenextfouryears,allofNoether’scourseswerelistedunderHilbert’sname;shewastreatedasakindofunofficialsubstituteteacher.Evenin1919,whenshewasfinallygiventherighttoteachunderherownname,itwasanunpaidpositionasaPrivatdozent,akindofadjunctprofessor.DuringhertimeatGöttingen,Noethermadeenormouscontributionsintwofields,physicsandmathematics.Inphysics,sheisresponsibleforNoether’stheorem,whichfundamentallyconnectedcertainsymmetriesandphysicalconservationlaws(e.g.,conservationofangularmomentum).AlbertEinsteinwasimpressedbyNoether’stheorem,whichisoneofthemostprofoundresultsintheoreticalphysics.Herresultunderliesmuchofmodernphysics,fromquantummechanicstothetheoryofblackholes.Inmathematics,Noethercreatedthefieldofabstractalgebra.AlthoughearliermathematicianssuchasCauchyandGaloishadworkedwithgroups,rings,andotheralgebraicobjects,theyalwaysusedspecificinstances.Noether’sbreakthroughwastorealizethatthesealgebraicstucturescouldbestudiedabstractly,withoutlookingatparticularimplementations.Noetherwasknownasanoutstandingteacher,andattractedstudentsfromallovertheworld.Underherleadership,theseyoungresearchers(oftencalled“Noether’sBoys”)werecreatinga

newkindofmathematics.In1933,whentheNazisexpelledJewsfromuniversities,NoetherfledtotheUnitedStates.Despitebeingoneofthegreatestmathematiciansintheworld,nomajorresearchuniversitywouldhireher,primarilybecauseshewasawoman.SheendedupwithavisitingappointmentatBrynMawr,asmallundergraduatewomen’scollege.Tragically,EmmyNoetherdiedin1935atage53,afewdaysaftersurgerytoremoveanovariancyst.Sincethen,hercontributionstomathematicshaveincreasinglybeenrecognizedasfundamentalandrevolutionary.

Noetherwaswellknownforherwillingnesstohelpstudentsandgivethemherideastopublish,butshepublishedrelativelylittleherself.Fortunately,ayoungDutchmathematician,BartelvanderWaerden,auditedhercourseandwroteabookbasedonherlectures(whichhecreditsonthetitlepage).CalledModernAlgebra,itwasthefirstbooktodescribetheabstractapproachshehaddeveloped.Thisbook,ModernAlgebra,ledtoafundamentalrethinkingofthewaymodernmathematicsis

presented.Itsrevolutionaryapproach—theideathatyouexpressyourtheoremsinthemostabstractterms—isNoether’screation.Mostofmathematics—notjustalgebra—changedasaresultofherwork;shetaughtpeopletothinkdifferently.

8.4RingsOneofNoether’smostimportantcontributionswasthedevelopmentofthetheoryofanalgebraicstructurecalledaring.4

4Theterm“ring,”coinedbyHilbert,wasintendedtousethemetaphorofabunchofpeopleinvolvedinacommonenterprise,likeacriminalring.Ithasnothingtodowithjewelryrings.

Definition8.3.Aringisasetonwhichthefollowingaredefined:

operations:x+y,–x,xy

constants:0R,1R


Rings5havethepropertiesweassociatewithintegerarithmetic—operatorsthatactlikeadditionandmultiplication,whereadditioniscommutativeandmultiplicationdistributesoveraddition.Indeed,rings

maybethoughtofasanabstractionofintegers,andthecanonicalexampleofaringisthesetofintegers,.Alsoobservethateveryringisanadditivegroupandthereforeanabeliangroup.The“addition”

operatorisrequiredtohaveaninverse,butthe“multiplication”operatorisnot.5Somemathematiciansdefineringswithoutthemultiplicativeidentity1anditsaxioms,andcallringsthatincludethemunitaryrings;we

donotmakethatdistinctionhere.

Inpractice,mathematicianswritethezeroeswithouttheirsubscripts,justaswe’vedoneintheaxioms.Forexample,indiscussingaringofmatrices,“0”refersnottothesingleintegerzerobuttotheadditiveidentitymatrix.Besidesintegers,otherexamplesofringsincludethefollowingsets:•n×nmatriceswithrealcoefficients•Gaussianintegers•Polynomialswithintegercoefficients

Wesaythataringiscommutativeifxy=yx.Noncommutativeringsusuallycomefromtherealmoflinearalgebrawherematrixmultiplicationdoesnotcommute.Incontrast,polynomialringsandringsofalgebraicintegersdocommute.Thesetwotypesofringsleadtotwobranchesofabstractalgebra,knownascommutativealgebraandnoncommutativealgebra.Ringsareoftennotexplicitlylabeledas“commutative”or“noncommutative”;instead,onetypeofringortheotherisassumedfromthebranchofalgebra.WiththeexceptionofSections8.5and8.6,therestofthisbookwilldealwithcommutativealgebra—thekindthatDedekind,Hilbert,andNoetherworkedon—sofromthenonwewillassumeourringsarecommutative.

Definition8.4.Anelementxofaringiscalledinvertibleifthereisanelementx–1suchthat

xx–1=x–1x=1

Everyringcontainsatleastoneinvertibleelement:1.Theremaybemorethanone;forexample,intheringofintegers ,both1and–1areinvertible.

Definition8.5.Aninvertibleelementofaringiscalledaunitofthatring.

Exercise8.4(veryeasy).Whichringcontainsexactlyoneinvertibleelement?WhatareunitsintheringofGaussianintegers?

Theorem8.1:Unitsareclosedundermultiplication(i.e.,aproductofunitsisaunit).

Proof.Supposeaisaunitandbisaunit.Then(bydefinitionofunits)aa–1=1andbb–1=1.So

1=aa–1=a·1·a–1=a(bb–1)a–1=(ab)(b–1a–1)

Similarly,a–1a=1andb–1b=1,so

1=b–1b=b–1·1·b=b–1(a–1a)b=(b–1a–1)(ab)

Wenowhaveatermthat,whenmultipliedbyabfromeitherside,gives1;thattermistheinverseofab:

(ab)–1=b–1a–1

Soabisaunit.

Exercise8.5.Provethat:

•1isaunit.•Theinverseofaunitisaunit.

Definition8.6.Anelementxofaringiscalledazerodivisorif:

1.x≠02.Thereexistsay≠0,xy=0

Forexample,inthering ofremaindersmodulo6,2and3arezerodivisors.

Definition8.7.Acommutativeringiscalledanintegraldomainifithasnozerodivisors.

It’scalled“integral”becauseitselementsactlikeintegers—youdon’tgetzerowhenyoumultiplytwononzerothings.Herearesomeexamplesofintegraldomains:

•Integers•Gaussianintegers•Polynomialsoverintegers•Rationalfunctionsoverintegers,suchas (Arationalfunctionistheratiooftwopolynomials.)

Theringofremaindersmodulo6isnotanintegraldomain.(Whetheraringofremaindersisintegraldependsonwhetherthemodulusisprime.)

Exercise8.6(veryeasy).Provethatazerodivisorisnotaunit.

8.5MatrixMultiplicationandSemiringsNote:Thissectionandthenextassumesomebasicknowledgeoflinearalgebra.Therestofthebookdoesnotdependonthematerialcoveredhere,andthesesectionsmaybeskippedwithoutimpactingthereader’sunderstanding.Inthepreviouschapter,wecombinedpowerwithmatrixmultiplicationtocomputelinearrecurrences.Itturnsoutthatwecanusethistechniqueformanyotheralgorithmsifweuseamoregeneralnotionofmatrixmultiplication.

LinearAlgebraReviewLet’squicklyreviewhowsomebasicvectorandmatrixoperationsaredefined.Innerproductoftwovectors:

Inotherwords,theinnerproductisthesumoftheproductsofallthecorrespondingelements.Theresultofinnerproductisalwaysascalar(asinglenumber).Matrix-vectorproduct:

Multiplyingann×mmatrixwithanm-lengthvectorresultsinann-lengthvector.Onewaytothinkoftheprocessisthattheithelementoftheresultistheinnerproductoftheithrowofthematrixwiththeoriginalvector.Matrix-matrixproduct:

InthematrixproductAB=C,ifAisak×mmatrixandBisanm×nmatrix,thenCwillbeak×nmatrix.TheelementinrowiandcolumnjofCistheinnerproductoftheithrowofAandthejthcolumnofB.Notethatmatrixmultiplicationisnotcommutative:thereisnoguaranteethatAB=BA.Indeed,it’softenthecasethatonlyoneofABandBAwillbewelldefined,sincethenumberofcolumnsofthefirsttermhastomatchthenumberofrowsofthesecond.Evenwhenbothproductsaredefined,theyarealmostalwaysdifferent.

Justaswegeneralizedourpowerfunctiontoworkwithanyoperation,wecannowgeneralizethenotionofmatrixmultiplication.Normallywethinkofmatrixmultiplicationasconsistingofaseriesofsumsofproducts,asshownintheearlierformula.Butwhat’smathematicallyessentialisactuallythattherebetwooperations,a“plus-like”onethatisassociativeandcommutative(denotedby⊕)anda“times-like”onethatisassociative(denotedby⊗),wherethelatteroperationdistributesoverthefirst:

a⊗(b⊕c)=a⊗b⊕a⊗c

(b⊕c)⊗a=b⊗a⊕c⊗a

We’vejustseenanalgebraicstructurethathasoperationslikethis,aring.However,ringshaveafewrequirementswedon’tneed,specificallythoseinvolvingtheadditiveinverseoperation.Instead,whatwewantisasemiring,aringwithoutminus(–).

Definition8.8.Asemiringisasetonwhichthefollowingaredefined:

operations:x+y,xy

constants:0R,1R


Ourdefinitionfollowsthemathematicalconventionofreferringtotheoperationsas+and×ratherthan⊕and⊗.Butaswithallthealgebraicstructureswe’vebeendiscussing,thesymbolsrefertoanytwooperationsthatbehaveinthemannerspecifiedbytheaxioms.Thecanonicalexampleofasemiringisthesetofnaturalnumbers .Whilenaturalnumbersdonot

haveadditiveinverses,youcaneasilyperformmatrixmultiplicationonmatriceswithnon-negativeintegercoefficients.(Infact,wecouldrelaxtherequirementsfurtherbyremovingtheadditiveidentity0andthemultiplicativeidentity1,aswellastheircorrespondingaxioms;matrixmultiplication6wouldstillwork.Wemightrefertoasemiringwithout0and1asaweaksemiring.)

6Hereweareassumingthestraightforwardalgorithmformatrixmultiplication;fasteralgorithmsrequirestrongertheories.

8.6Application:SocialNetworksandShortestPathsWecanusesemiringstosolveavarietyofproblems.Forexample,supposewehaveagraphoffriendships,asinasocialnetwork,andwewanttofindallthepeopleyouareconnectedtothroughanypath.Inotherwords,wewanttoknowwhoyourfriendsare,thefriendsofyourfriends,thefriendsofthefriendsofyourfriends,andsoon.Findingallsuchpathsisknownasfindingthetransitiveclosureofthegraph.Tocomputethetransitive

closure,wetakeann×nBooleanmatrixwhereentryxijis1iftherelationholdsbetweeniandj(inthiscase,ifpersoniisfriendswithpersonj),and0otherwise;we’llalsoassumepeoplearefriendswiththemselves.Here’sasmallexample:

Thematrixtellsuswhoeachperson’sfriendsare.Wecanapplygeneralizedmatrixmultiplicationwherewereplace⊕byBooleanOR(∨)and⊗byBooleanAND(∧).WesaythisisthematrixmultiplicationgeneratedbyaBooleanor{∨,∧}-semiring.Multiplyingthematrixbyitselfusingtheseoperationstellsuswhothefriendsofourfriendsare.Doingthismultiplicationn–1timeswilleventuallyfindallthepeopleineachnetworkoffriends.Sincemultiplyingthematrixbyitselfseveraltimesisjustraisingittoapower,wecanuseourexistingpoweralgorithmtodothecomputationefficiently.Ofcourse,wecanusethisideatocomputethetransitiveclosureofanyrelation.

Exercise8.7.UsingthepoweralgorithmfromChapter7withmatrixmultiplicationonBooleansemirings,writeaprogramforfindingtransitiveclosureofagraph.Applythisfunctiontofindthesocialnetworksofeachpersonintheprecedingtable.

Anotherexampleofaclassicproblemwecansolvethiswayisfindingtheshortestpathbetweenanytwonodesinadirectedgraphlikethisone:

Asbefore,wecanrepresentthegraphasann×nmatrix—thistimeonewhosevaluesaijrepresentthedistancefromnodeitonodej.Ifthereisnoedgefromonenodetoanother,we’llinitiallylistthedistanceasinfinity.

Thistime,weusematrixmultiplicationgeneratedbyatropicalor{min,+}-semiring:

Thatis,the⊕operationismin,andthe⊗operationis+.Again,weraisetheresultingmatrixtothen–1power.Theresulttellsustheshortestpathofanylengthupton–1steps.

Exercise8.8.UsingthepoweralgorithmfromChapter7withmatrixmultiplicationontropicalsemirings,writeaprogramforfindingthelengthoftheshortestpathinagraph.

Exercise8.9.ModifytheprogramfromExercise8.8toreturnnotjusttheshortestdistancebuttheshortestpath(asequenceofedges).

8.7EuclideanDomainsWebeganthischapterbyseeinghowEuclid’sGCDalgorithmcouldbegeneralizedbeyondintegers,firsttopolynomials,thentocomplexnumbers,andsoon.Howfarcouldthisgeneralizationgo?Inotherwords,whatarethemostgeneralmathematicalentitiesthattheGCDalgorithmworkson(thedomainorsettingforthealgorithm)?WiththeabstractionsNoetherhaddeveloped,shewasfinallyableanswerthisquestion:thedomainoftheGCDalgorithmiswhatNoethercalledtheEuclideandomain;itisalsosometimesknownasaEuclideanring.

Definition8.9.EisaEuclideandomainif:

•Eisanintegraldomain•Ehasoperationsquotientandremaindersuchthat

b≠0 a=quotient(a,b)·b+remainder(a,b)

•Ehasanon-negativenorm||x||:E→ satisfying

Theterm“norm”hereisameasureofmagnitude,butitshouldnotbeconfusedwiththeEuclideannormyoumaybefamiliarwithfromlinearalgebra.Forintegers,thenormistheirabsolutevalue;for

polynomials,itisthedegreeofthepolynomial;forGaussianintegers,itisthecomplexnorm.Theimportantideaisthatwhenyoucomputetheremainder,thenormdecreasesandeventuallygoestozero,sinceitmapsintonaturalnumbers.WeneedthispropertytoguaranteethatEuclid’salgorithmterminates.

***

NowwecanwritethefullygenericversionoftheGCDalgorithm:Clickheretoviewcodeimage

template<EuclideanDomainE>

Egcd(Ea,Eb){

while(b!=E(0)){

a=remainder(a,b);

std::swap(a,b);

}

returna;

}

Theprocesswe’vegonethroughintransformingtheGCDalgorithmfromsomethingthatworksonlyonlinesegmentstosomethingthatworksonverydifferenttypesillustratesthefollowingimportantprinciple:

Tomakesomethinggeneric,youdon’taddextramechanisms.Rather,youremoveconstraintsandstripdownthealgorithmtoitsessentials.

8.8FieldsandOtherAlgebraicStructuresAnotherimportantabstractionisthefield.7

7Theterm“field”reliesonthemetaphorofafieldofstudy,notafieldofwheat.

Definition8.10.Anintegraldomainwhereeverynonzeroelementisinvertibleiscalledafield.

Justasintegersarethecanonicalexampleofrings,sorationalnumbers( )arethecanonicalexampleoffields.Otherimportantexamplesoffieldsareasfollows:

•Realnumbers•Primeremainderfields•Complexnumbers

Aprimefieldisafieldthatdoesnothaveapropersubfield(asubfielddifferentfromitself).Itturnsoutthateveryfieldhasoneoftwokindsofprimesubfields: or .Thecharacteristicofafieldispifitsprimesubfieldis (thefieldofintegerremaindersmodulop),and0ifitsprimesubfieldis .

***

Allfieldscanbeobtainedbystartingwithaprimefieldandaddingelementsthatstillsatisfythefieldproperties.Thisiscalledextendingthefield.Inparticular,wecanextendafieldalgebraicallybyaddinganextraelementthatisarootofa

polynomial.Forexample,wecanextend with ,whichisnotarationalnumber,sinceitistherootofthepolynomialx2–2.Wecanalsoextendafieldtopologicallyby“fillingintheholes.”Rationalnumbersleavegapsinthe

numberline,butrealnumbershavenogaps,sothefieldofrealnumbersisatopologicalextensionofthefieldofrationalnumbers.Wecanalsoextendthefieldtotwodimensionswithcomplexnumbers.

Surprisingly,therearenootherfinitedimensionalfieldscontainingreals.88Therearefour-andeight-dimensionalfield-likestructurescalledquaternionsandoctonions.Thesearenotquitefields,becausethey

aremissingcertainaxioms;bothquaternionsandoctonionslackcommutativityofmultiplication,andoctonionsalsolackassociativityofmultiplication.Therearenootherfinite-dimensionalextensionsofrealnumbers.

Uptonow,everyalgebraicstructurewe’veintroducedinthisbookhasoperatedonasinglesetofvalues.Buttherearealsostructuresthataredefinedintermsofmorethanoneset.Forexample,animportantstructurecalledamodulecontainsaprimaryset(anadditivegroupG)andasecondaryset(aringofcoefficientsR),withanadditionalmultiplicationoperationR×G→Gthatobeysthefollowingaxioms:

IfringRisalsoafield,thenthestructureiscalledavectorspace.Agoodexampleofavectorspaceistwo-dimensionalEuclideanspace,wherethevectorsarethe

additivegroupandtherealcoefficientsarethefield.

8.9ThoughtsontheChapterInthischapter,wefollowedthehistoricaldevelopmentofgeneralizingtheideaof“numbers”andthecorrespondinggeneralizationoftheGCDalgorithm.Thisledtothedevelopmentofseveralnewalgebraicstructures,someofwhichweusedtogeneralizematrixmultiplicationandapplyittosomeimportantgraphproblemsincomputerscience.Let’sextendourtablefromSection6.8toincludethenewstructuresweintroducedinthischapter.

Notethateveryrowofthetableincludesalltheaxiomsfromearlierrows.(Inthecaseofsemiringsandrings,the“times”operationinheritsalltheaxiomsfrommonoids,whilethe“plus”operationinheritstheaxiomsfromabeliangroups.)Toillustratethis,we’vegrayedoutoperations,elements,andaxiomsthatappearedpreviouslyinthetable.

Aswedidbefore,wecanalsodefinesomeotherstructuresmoreconciselyintermsofothers:

Thisdiagramshowstherelationshipsbetweensomeofthemostimportantstructuresdiscussedinthischapter:

Thefirsttimeyouencounteralgebraicstructures,itmightseemasiftherearesomanyvarietiesthatit’shardtokeeptrackoftheirproperties.However,theyfitintoamanageabletaxonomythatmakestheirrelationshipsclear—ataxonomythathasenabledgreatprogressinmathematicsoverthelasthundredyears.

9.OrganizingMathematicalKnowledge

Allthetruthsofmathematicsarelinkedtoeachother,andallmeansofdiscoveringthemareequallyadmissible.

Legendre

Nowwe’regoingtolookatsomeofthebuildingblocksfororganizingknowledge,particularlymathematicalknowledge.We’llstartbyexploringthenotionofproofsandtheintroductionoftheideaoftheorems.Thenwe’llexaminesomeimportantexamplesofattemptstobuildupbodiesofknowledgefromaxioms.Mathematicianshavebeenthinkingabouthowtoorganizeknowledgeforthousandsofyears.As

programmers,wewillusetheirorganizationalprinciplesinourdomainofalgorithmsanddatastructures.

9.1ProofsPeoplehadbeendiscoveringandusingmathematicalresultslongbeforetheystartedprovingthem.Yetmathematicalproofsarealsoasurprisinglyoldinvention.Forcenturiesmathematiciansreliedonvisualproofs.TheancientGreeksrealizedthattheycoulduseourinnatespatialreasoningtoprovealgebraicfacts.Herearesomeexamplesofvisualproofs.Commutativityofaddition:a+b=b+a

Ifwehavetwostripsofpaperandtapethemtogethertomakeonestrip,wegetthesamelengthregardlessofwhichoneisontheleftandwhichoneisontheright.Wecanseethisbecausethefigureontherightisamirrorimageofthefigureontheleft.Associativityofaddition:(a+b)+c=a+(b+c)

Ifwehavethreestripsofpaperandwetapethepiecestogethertomakeonelongstrip,itdoesn’tmatterifwetapethefirsttwopiecestogetherandthentapethethirdonetotheresult,orifinsteadwetapethelasttwoandthenthefirst.Eitherwaywe’llendupwithastripofthesamelengthintheend.Commutativityofmultiplication:ab=ba

Arectanglehasacertainlengthandacertainwidth.Ifyouturnitsideways,you’vereversedlengthandwidth,butyouobviouslystillhavethesamerectangle.Infact,thisessentialargumentappearsina19th-centurybookbyDirichlet,whosaysthatwhetheryouarrangesoldiersinrowsorcolumns,youstillhavethesamenumber.Associativityofmultiplication:(ab)c=a(bc)

Whetheryouslicethisrectangularprismalongoneaxisoralonganother,whenyouputtheslicesbacktogether,youstillhavethesamevolume.(a+b)2=a2+2ab+b2:

It’sclearjustbylookingthattherectangleonthelowerleftisthesameareaastherectangleontheupperright:notonlydobothhaveareaab,butyoucouldliterallycutoneout,turnitsideways,andlayitontheother.π>3:

Herewe’veinscribedaregularunithexagon(onewhosesidesarealloflength1)inthecircle.It’sevidentthattheperimeterofthehexagonisshorterthanthecircumferenceofthecircle,becausewheneverwehavetwointersectionpointsbetweenthetwofigures,theshortestpathfromonepointtothenextisalongthehexagon,notthecircle.Sincethetrianglesthatmakeupthehexagonareequilateral,alltheirsidesarelength1,sothediameterofthecircleislength2.Sotheratioofthecircle’scircumferencetoitsdiameter(i.e.,π)mustbegreaterthantheratioofthehexagon’sperimeter(6)toitsdiameter(2).

Exercise9.1.Designvisualproofsforthefollowing:

Exercise9.2.Usingavisualproof,findanupperboundforπ.

***

Asusefulasvisualproofsare,thistechniqueisn’tsufficienttoproveeverytypeofpropositioninmathematics,andsomeoftheproofsarenolongerconsideredrigorousenough.Modernmathematicianshaveavarietyofprooftechniquesavailabletothem,someofwhichwe’veusedthroughoutthisbook,andwhicharesummarizedinAppendixB.Proofsshowconnectionsbetweendifferenttruths.Butwhatexactlyconstitutesaproof?Today,weusethefollowingdefinition:

Definition9.1.Aproofofapropositionis

•Anargument•Acceptedbythemathematicalcommunity•Toestablishthepropositionasvalid

Thesecondpointisoftenoverlooked:proofisfundamentallyasocialprocess,andonethatchangesovertime.Ourconfidenceinaproofincreasesasmorepeopleunderstandandagreewithit.Atthesametime,whatisconsideredavalidprooftodaymightnotbeconsideredavalidproof300yearsfromnow,justassomeproofsthatwereviewedasvalidbyEuler—thegreatest18th-centurymathematician—arefrownedupontoday.Nowwe’llturntoanotherbuildingblockofmathematicalknowledge,theorems.

9.2TheFirstTheoremAswediscussedinChapter2,ancientMediterraneancivilizationsbelievedthattheEgyptianswerethesourceofmathematicalknowledge.WhenGreekcivilizationwasjuststarting,Egyptiancivilizationhadalreadyexistedforthousandsofyears,soitisnotsurprisingthattheleadingthinkersofancientGreecewouldtraveltoEgypttostudywiththeirpriestsandlearntheirwisdom.ThefirstsuchpersonknowntousisThalesofMiletus.ThaleslearnedgeometryfromtheEgyptians,buthewentbeyondtheirwork.WhiletheEgyptianshadalgorithms,Thaleshadatheorem—infact,heinventedtheverynotionofatheorem,whichisapropositionderivablefromotherpropositions.TodayThalesisregardedasthefounderofWesternphilosophy,andmightalsobeconsideredthefirstmathematician.

ThalesofMiletus(flourishedearly6thcenturyBC)

Sometimearoundtheyear750BCanewsocietystartedtoappearindifferentcoastalregionsoftheMediterraneanandevenasfarnorthastheBlackSea.TheycalledthemselvesHellenes;wecallthemGreeks.Theycamefromasmallmountainouscountrywherethegeographypreventedtheemergenceofalargeunifiedkingdomaswascommonelsewhere.Greekslivedinsmall,independentcity-statesunifiednotbyacentralgovernment,butbyacommonlanguageandculture.Wheneveracity’spopulationexceededitsresources,itwouldsendsomeofitscitizenstosettleacolony,anewpracticallyindependentcityonsomeconvenientlylocatedbaywithariver.Within200yearsGreekssettledaroundtheMediterranean,asPlatoputit,“likefrogsaroundapond.”Bysomewherearound600BC,theGreekcoloniesinAsiaMinor(whatisnowTurkey)were

gettingwealthy.Insteadofspendingalltheextramoneyonluxuries,someofthemstartedsupportingintellectualpursuits.Forthefirsttimeinhistorytheylookedbeyondmythologyfortheanswerstoeternalquestionssuchaswhatthingsweremadeof.ThefirstpersontodothisinafundamentalwaywasThalesofMiletus.ThaleswastheoriginatorofwhatancientGreekswouldeventuallycallphilosophyandwhatwenowcallscience.Hewantedtofindthenatural,non-mythologicalexplanationofreality.Heproposedthatallvisiblerealityismadeoutofonesinglesubstance:water.Therefore,visiblerealityexistsinoneofthreestates—gas,liquid,orsolid—andtherearetransitionsbetweenthestates.

WhileinEgypt,Thalescollectedmanygeometricalgorithmsand,probably,someBabylonianastronomicalknowledge.HerodotusreportsthatThaleswasabletopredictatotalsolareclipseayearinadvance.Aristotle—usuallyareliablesource—tellsusthatThaleswasabletopredictanexceptionallylargeharvestofolivesbystudyingweatherpatternsand,bybuyingoptionsontheuseofalltheolivepressesintheregion,madeafortune.Therearemanyotherstoriesabouthisaccomplishments,suchashisdiscoveryofstaticelectricity.Whilewedonotknowexactlywhichstoriesaretrue,Thalesclearlyamassedalargebodyofscientificknowledgeandwasabletoapplyittopracticalproblems.Hisknowledgedidnotperishwithhim;hisstudentscarriedtheprogramforward.Butmoreimportantthananyofhisspecificdiscoverieswashisapproachtounderstandingtheworld,whichisstillthebasisofallscience.

Theorem9.1(Thales’Theorem):ForanytriangleABCformedbyconnectingthetwoendsofacircle’sdiameter(AC)withanyotherpointBonthecircle,∠ABC=90°.

Proof.ConsiderthetrianglesformedbyjoiningpointBwiththecenterofthecircle,D:

SinceDAandDBarebothradiiofthecircle,theyareequalandtriangleADBisisosceles.ThesameistrueforDB,DC,andtriangleBDC.Therefore

wherewegetthethirdequationbyaddingtheprevioustwo.Itwasalsoknownthattheanglesofatriangleaddupto180°,andwecanseethat∠CBAisthesumof∠DBAand∠DBC,so

∠DAB+∠DCB+∠DBA+∠DBC=180°

Bysubstitutingusingtheequalityweestablished,wecanwritethisasfollows:

WhywasThales’discoverysoimportant?Whatherealizedisthattruthsareconnected.Hesawthatifyouhaveonepieceofknowledge,youcanuseittofindanother.Furthermore,theoremsareessentialtotheideaofabstraction,forthevalueofatheoremisthatitappliestoallentitiesthathavecertainproperties.

9.3EuclidandtheAxiomaticMethodIfwewanttobuildupasystemofknowledge,proofsandtheoremsareessentialtools.Butwealsoneedtohaveasetofstartingassumptions,oraxioms,asafoundationforoursystem.Thefirstappearanceoftheaxiomaticmethod,inwhichanentiremathematicalsystemwasbuiltonthe

basisofafewformalprinciples,isinEuclid’sElements.Infact,forcenturiesEuclid’sweretheonlyknownexamplesofaxioms,andtheyappliedonlytogeometry.Eucliddividedhisprinciplesintothreegroups:definitions,postulates,andcommonnotions.Hestarts

withhis23definitions,whichrelatetogeometricfigures.Hereareafewofthem:11Asbefore,weuseSirThomasHeath’stranslationofEuclid’sElements.

1.Apointisthatwhichhasnoparts.2.Alineisabreadthlesslength.

23.Parallelstraightlinesarestraightlineswhich,beinginthesameplaneandbeingproducedindefinitelyinbothdirections,donotmeetoneanotherineitherdirection.

Next,hegavethefollowingfive“commonnotions”:1.Thingswhichareequaltothesamethingarealsoequaltooneanother.2.Ifequalsbeaddedtoequals,thewholeareequal.3.Ifequalsbesubtractedfromequals,theremaindersareequal.4.Thingswhichcoincidewithoneanotherareequaltooneanother.5.Thewholeisgreaterthanthepart.

Todaywewouldexpressthesenotionsasfollows:1.a=c∧b=c a=b2.a=b∧c=d a+c=b+d3.a=b∧c=d a−c=b−d4.a b a=b5.a<a+b

What’sinterestingaboutthesecommonnotionsisthat,unlikethe23definitions,thenotionsarenotlimitedtogeometry;theyalsoapplytopositiveintegers.Infact,thesecommonnotions,suchastransitivityofequality,areessentialtoprogramming.2

2ThedefinitionofregulartypesinChapter7isderivedfromtheseEuclideannotions.

Finally,Euclidintroducedhisfamousfivepostulates.Thesearestatedintermsofallowableoperationsinthe“computationalmachinery”ofhisgeometricsystem.Youcanreadthefirstthreeasbeingprefixedwithastatementlike“Thereisaprocedure...”:

1.Todrawastraightlinefromanypointtoanypoint.2.Toproduceafinitestraightlinecontinuouslyinastraightline.3.Todescribeacirclewithanycenteranddistance.4.Thatallrightanglesareequaltooneanother.5.That,ifastraightlinefallingontwostraightlinesmakestheinterioranglesonthesamesidelessthantworightangles,thetwostraightlines,ifproducedindefinitely,meetonthatsideonwhicharetheangleslessthanthetworightangles.

IfwewerewritingEuclid’ssystemtoday,wewouldconsiderboth“commonnotions”and“postulates”tobeaxioms—unprovableassumptionsonwhichtherestofthesystemisbuilt.Euclid’sfifthpostulate,whichprovidesthebasisforreasoningaboutparallellines,isthemost

importantaxiominthehistoryofmathematics.Alsoknownastheparallelpostulate,itexpressestherelationshowninthefollowingdiagram:

However,therearemanyequivalentwaystoexpressthesamenotion:•Givenalineandapointnotonit,atmostoneparalleltothegivenlinecanbedrawnthroughthepoint.3

3Thisformulation,whichisoftentaughtas“theparallelpostulate”insecondaryschoolgeometry,wasactuallypublishedbyScottishmathematicianJohnPlayfairin1795,andisproperlyknownasPlayfair’sAxiom.

•Thereexistsatrianglewhoseanglesaddupto180°.•Thereexisttwosimilartrianglesthatarenotcongruent.

9.4AlternativestoEuclideanGeometryAlmostfromthetimeEuclidstatedhisfivepostulates,mathematiciansfeltthattherewassomethingdifferentaboutthefifthone.Intuitively,theyfeltthatthefirstfourpostulatesweresomehowmorefundamental;perhapsthefifthpostulatecouldbederivedfromtheothers,andthereforewasnotatrueaxiom.Thusbegana2000-yearsearchforaproofofthefifthpostulate,onepursuedbysuchluminariesastheastronomer(andmathematician)Ptolemy(90–168),thepoet(andmathematician)OmarKhayyam(1050–1153),andtheItalianpriest(andmathematician)GiovanniGirolamoSaccheri,S.J.(1667–1733).SaccheriwroteabookcalledEuclidusVindicatus(“EuclidVindicated”)inwhichheconstructedawholegeometricalsystembasedonthetheassumptionthatthefifthpostulateisfalse,thenclaimedthattheconsequenceswouldbesobizarrethatthepostulatemustbetrue.

Whilemost18th-centurymathematiciansdidn’tcareaboutaxioms,themoodshiftedinthe19thcentury.Mathematiciansstartedtofocusonthefoundationsoftheirwork.Theyrevisitedgeometry,nolongertakingEuclidforgranted,butexamininghisassumptions.Around1824,RussianmathematicianNikolaiLobachevskywasworkingontheproblem.Atsome

point,herealizedthattheparallelpostulatewasjustonepossibleassumption,andthatthecontraryassumptionisequallyvalid.Insteadofsaying“thereisatmostonelinethroughapointparalleltoagivenline,”Lobachevskyessentiallyexploredtheideathat“therearemanylines....”UnlikeSaccheri,Lobachevskyrealizedthattheresultingsystemofgeometrywasentirelyconsistent.Inotherwords,heinventedanentirelynewnon-Euclideangeometry,sometimescalledhyperbolicgeometry.InLobachevsky’sgeometry,therearenosimilartrianglesexceptforcongruentones.Bywayof

analogy,thinkoftrianglesonthesurfaceofasphere.Forsmalltriangles,thesurfaceisalmostplanar,sothesumoftheanglesiscloseto180°.Butasthetrianglesgetbigger,theanglesneedtogetbiggerbecauseofthecurvatureofthesurface.Lobachevsky’smodelwassimilar,butwithspacecurvedintheoppositeway,sothatbiggertrianglescorrespondedtosmallerangles.Lobachevsky’sresults,firstpublishedin1826,weremetwithdismissalandscornfromtheRussian

mathematicalcommunity,andLobachevskyhimselfwasmarginalized.OnepersonwhodidrecognizethevalidityofLobachevsky’sworkwasGauss,wholearnedRussiantoreadLobachevsky’sbook.Butingeneral,itwouldtakemanyyearsbeforehisworkbecameanacceptedpartofmathematics.Today,Lobachevsky’sdiscoveryisconsideredtobeamonumentalturningpointinthehistoryofmathematics.

NikolaiIvanovichLobachevsky(1792–1856)

Intheearly19thcentury,Russiawasnotamajorcenterformathematics(despiteEulerspendingmuchofhiscareerinSt.Petersburg).TherewerenogreatRussianmathematicians.Yetbythemiddleofthe20thcentury,Russiawasamathematicalsuperpower.ThistransformationbeganwiththefirstgreatRussianmathematician,NikolaiIvanovichLobachevsky.Lobachevskydidnotcomefromamajorcity,nordidheattendoneofthetwogreatuniversities

(MoscowandSt.Petersburg);hewasnotsentabroadtolearnfromtheleadingthinkersofEurope.Hedidnotcomefromthearistocracyoreventheuppermiddleclass;heandhisbrotherwere

charitystudentsattheirlocalschool.HegrewupinKazan,aprovincialcityontheVolgariverthatdidnotevenhaveauniversityuntil1805.Lobachevskyenteredtherecentlyfoundeduniversityin1807.(Interestingly,TolstoyandLeninattendedthesameschooldecadeslater.)WhenLobachevskystartedattheUniversityofKazan,therewasnoonetoteachmathematics—

studentsstudiedontheirown.Fortunately,MartinBartels,oneofGauss’sformerprofessors,soonjoinedthefaculty.Afterreceivingamaster’sdegreeandcontinuingtostudyprivatelywithBartels,Lobachevskywasappointedasanadjunctprofessorin1814.Hewouldgoontospendmostofhiscareerattheuniversity,eventuallybeingelecteditsrector(similartopresident)in1827.Despitehishumbleorigins,Lobachevskywasneverafraidtochallengeconventionalopinions.

Hisgroundbreakingworkonnon-Euclideangeometrywassubmittedin1826,butwasnotwidelyknownuntilitwaspublishedasabookin1832.ThebookwaspubliclyridiculedinareviewbyOstrogradsky,animportantRussianmathematicianwhostudiedwithCauchy.Lobachevskycontinuedhisworkonnon-Euclideangeometryfortherestofhislife,refiningit

andpublishingbooksaboutitinvariouslanguages.Bythe1840s,Gaussrecognizedtheimportanceofthework,evenreadingsomeofLobachevsky’sbooksintheoriginalRussian.GaussnominatedhimformembershipintheGöttingenAcademyofSciences,agreathonoratthetime.YetLobachevskywasstillostracizedbytheRussianmathematicalestablishmentuntiltheendofhiscareer.Inhislateryears,Lobachevsky’slifetookatragicturn.Helosthisjobattheuniversity,his

house,andmostofhisproperty,sufferedthedeathsoftwoofhischildren,andthenbecameblind.Evenunderthesecircumstances,hepersistedinhiswork,dictatingamajornewbook,Pangeometry,justbeforehisdeathin1856.

Oftenwhenanewideaemergesinmathorscience,itisdiscoveredindependentlybymultiplepeopleatroughlythesametime.Thiswasthecasewithnon-Euclideangeometry.AtaboutthesametimeLobachevskywasworkinginKazan,ayoungHungarianmathematiciannamedJánosBolyaimadeasimilardiscovery.Afewyearslater,Bolyai’sfatherFarkasBolyai,awell-knownmathprofessorandfriendofGauss,includedtheson’sresultsasanappendixtooneofhisownbooks.FarkassentGaussthebook.AlthoughGaussprivatelyremarkedthatyoungBolyaiwasagenius,theletterhesentFarkashadadiscouragingmessage:

IfIcommencedbysayingthatIamunabletopraisethiswork,youwouldcertainlybesurprisedforamoment.ButIcannotsayotherwise.Topraiseitwouldbetopraisemyself.Indeedthewholecontentsofthework,thepathtakenbyyourson,theresultstowhichheisled,coincidealmostentirelywithmymeditations,whichhaveoccupiedmymindpartlyforthelastthirtyorthirty-fiveyears.

TheletteristypicalofGauss,bothinhisrefusaltogivecredittoothersandinhisinsistencethathisownunpublishedthoughtsgavehimpriority.(WenowknowthatGausshadindeeddiscoveredmanyofthesameideas,buthaddecidednottopublishthembecausehewasafraidofthereaction.)WhyheacknowledgedLobachevsky’sworkbutdismissedBolyai’swewillneverknow.Butwhateverthereason,theresultsweretragic.BolyaiwasdevastatedbyGauss’sresponseandneverattemptedtopublishinmathematicsagain.Evensadder,hebecamementallyunstable.WhenhecameacrossLobachevsky’sbooksometimelater,hewasconvincedthat“Lobachevsky”wasactuallyapseudonymforGauss,whomhebelievedhadstolenhisideas.

***

Oncenon-Euclideangeometrywasdiscovered,manymathematicianswrestledwithwhattheyconsideredtobeanimportantquestion:whichgeometryisactuallycorrect,Euclid’sorLobachevsky’s?Gausstookthequestionquiteseriously,andproposedaningeniousexperimenttotestthetheory.First,findthreemountainsformingatrianglethataresomedistanceapart,butcloseenoughsothata

personstandingontopofeachwithatelescopecanseetheothers.Thensetupsurveyingequipmentoneachpeaktoaccuratelymeasuretheanglesofthetriangle.Iftheanglesaddupto180°,thenEuclidisright;iftheirsumislessthan180°,thenLobachevskyis.Theactualexperimentwasneverconducted.Butovertime,thequestionbecamemoot.Other

mathematicianswouldultimatelyprovetheindependenceofthefifthpostulate,showingthatifEuclideangeometryisconsistent,thensoisLobachevskiangeometry.Meanwhile,mathematiciansbegantotreatquestionsofrealityasirrelevant.Whilemathwasoriginallyinventedtounderstandaspectsoftheworldwelivein,bytheendofthe19thcentury,itbegantobeseenasapurelyformalexercise.

9.5Hilbert’sFormalistApproachOnemustbeabletosay“tables,chairs,beer-mugs”eachtimeinplaceof“points,lines,planes.”

—DavidHilbert

DavidHilbert,perhapsthegreatestmathematicianoftheearly20thcentury,wastheleaderofthisformalistapproach.Inaviewthateventuallybecamestandardthroughoutmathematics,hesaidthatifatheorywasconsistent,itwastrue.WhileallofEuclid’stheoremsandproofsarecorrect,bymodernstandardstheaxiomsaresomewhat

shaky.Ittook2400yearsbeforeanyonetriedtocomeupwithabetterfoundationforgeometry.Hilbertspent10yearsrethinkingEuclidandconstructinghisownaxiomaticsystemforgeometry.Asthequotationsuggests,Hilbertbelievedthatthevalidityofhisaxiomaticsystemshouldnotrelyonanyintuitionsaboutgeometry.Hilbert’ssystemcontainedmanymoreaxiomsthanEuclid’s,makingexplicitmanythingsthatEuclidtookforgranted.Hilberthad:

•7axiomsofconnection(e.g.,iftwopointslieonaplane,thenallpointsonthelinegoingthroughthesepointsareonthisplane)•4axiomsoforder(e.g.,thereisapointbetweenanytwopointsonaline)•1axiomofparallels•6axiomsofcongruence(e.g.,twotrianglesarecongruentifside-angle-side...)•1Archimedes’axiom•1completenessaxiom

Hilbert’sgeometricsystemisquitecomplex,andwasthesubjectofseveralofhiscourses.Unfortunately,bythetimehewasdoneconstructingtheaxioms,hehadnoenergylefttoprovemanygeometrictheorems.Hilbert’sworkontheaxiomsofEuclideangeometrywasthelastmajorworkdoneonthatsubject.

DavidHilbert1862–1943

DavidHilbertwasbornintheGermancityofKönigsberg(nowKaliningrad,Russia).HestudiedmathematicsatthetheUniversityofKönigsberg,continuedforaPh.D.,andeventuallyjoinedthefaculty.Atage33,heacceptedanoffertobecomeaprofessorattheUniversityofGöttingen.Hewould

staytherefortherestofhiscareer.Aswesawearlier(Section8.2),Göttingenwasthecenterofthemathematicaluniverse,andHilberteventuallybecametheleaderofthemathematicalcommunitythereduringthepinnacleofthedepartment’sfame.ItisdifficulttoconveytheastoundingvarietyoffundamentalworkdonebyHilbertandthe

profoundeffecthehadonallofmathematics.Inhisinitialworkoninvarianttheory,Hilbertchampionedtheuseofnonconstructiveproofs,

whichwasaradicalideaatthetime.Infact,heinitiallybecamefamousasmuchfortheapproachasfortheactualresult.Today,nonconstructiveproofsarecommon.Whenaskedtosummarizeworkonalgebraicnumbertheory(theareaDedekindhadbeen

workingon),Hilbertwrotea600-pagevolumecalledZahlbericht(“ReportonNumbers”).Thisbookcapturedandexplainedallthemajordevelopmentsinthefield.Whilehemostlysummarized(andcredited)theworkofothers,Hilbert’sunificationdrovethefieldforward,eventuallyleadingtoNoether’sworkonabstractalgebra.Forthenext10years,whileworkingongeometry,HilbertwentbeyondLobachevskyand

examinedthevalidityofallofEuclid’saxioms.HisbookFoundationsofGeometrynotonlyintroducedhisnewaxioms,butalsotaughtpeopleforthefirsttimehowtothinkaboutandrigorouslyanalyzeanyaxiomaticsystem.Hilbertalsoworkedonphysics,co-inventinggeneralrelativitytheory,largelyindependentlyand

atroughlythesametimeasEinstein.HisinventionofHilbertspaces—anextensionofvectorspacestoinfinitedimensions—becameanimportantbuildingblockinthemathematicalfoundationofquantummechanics.In1900,HilbertgavealectureattheSorbonneinPariswherehelisted10importantunsolved

problemsinmathematicsandchallengedthecommunitytoworkonthem.Thelistwaslater

expandedto23problemsinapublishedpaper.Workontheseproblems,whichbecameknownasHilbert’sproblems,definedmuchofmathematicsinthe20thcentury.Healsospentmuchofthelast25yearsofhiscareerworkingonmechanizingthefoundationsofmathematics,aneffortknownas“Hilbert’sprogram.”AlthoughHilbert’sprogramwasshowntobeflawedbytheworkofKurtGödelandAlanTuring,thatsameworkledtothedevelopmentofthemoderntheoryofcomputation.Hilbertwasnotonlyagreatmathematician,butalsoagreatmentorandsupporterofyounger

colleagues.WhenhisbestfriendHermannMinkowskidied,HilbertspentseveralyearseditingandpublishingMinkowski’swork.HechampionedthecareerofEmmyNoether(Section8.3).Healsocollaboratedwithmanyresearchersinseveralfields;hislecturesonphysicsbecamethebasisofaclassictextco-authoredbyRichardCourant.Hilbert’soneblindspotwashisprideinGermanculture.Withgoodreason,hesawmathematics

inGermanyastheculminationof200yearsofadvances.HewelcomedpeoplefromallovertheworldtojointheGermanmathematicalcommunity.ButhealsobelievedthatonlytheresearchinGermanywasworthyofattention.PerhapsthemostegregiousexamplewashisunwillingnesstocitetheworkofGiuseppePeanoinItalyorrecognizeitsseminalimportancetothefoundationsofmathematics.Atthesametime,HilbertwascompletelyopposedtotheviewsoftheNazis(whocametopowerafewyearsafterhisretirementin1930),havingspentmuchofhiscareerpromotingtheworkofmanycolleagueswhohappenedtobeJewish,includinghisbestfriendMinkowskiandhisprotegéNoether.Sadly,Hilbertlivedtoseeeverythinghecaredaboutdestroyed.Hisfriendsweredriveninto

exile,hisonce-greatdepartmentwasreducedtomediocrity,andhisbelovedcountryembracedbeliefshedespised.Buthismathematicallegacywascarriedaroundtheworldbyhismanystudentsandcollaborators,andlivesontoday.

9.6PeanoandHisAxiomsCertainlyitispermittedtoanyonetoputforwardwhateverhypotheseshewishes,andtodevelopthelogicalconsequencescontainedinthosehypotheses.ButinorderthatthisworkmeritthenameofGeometry,itisnecessarythatthesehypothesesorpostulatesexpresstheresultofthemoresimpleandelementaryobservationsofphysicalfigures.

—GiuseppePeano

EvenbeforeHilbertannouncedhisprogramonformalizingmathematics,othershadbeenworkingonsimilarideasaboutformalizingmathematicalsystems.OneofthesewasItalianmathematicianGiuseppePeano.Asthequotationshows,Peanowasstillinterestedintheconnectionsbetweenmathematicsandreality.In1891,hebeganwritingFormularioMathematico(“MathematicalFormulas”),whichwouldbecomeacomprehensiveworkcontainingallessentialtheoremsinmathematicsexpressedinasymbolicnotationPeanoinvented.Muchofhisnotation,suchasthesymbolsforquantifiersandsetoperations,isstillusedtoday.In1889,Peanopublishedasetofaxiomsthatprovidedaformalbasisforarithmetic.Therewerefive,

justlikeEuclid’s:Thereisaset calledthenaturalnumbers:1.∃0 2.∀n :∃n′ -calleditssuccessor

3.∀ ⊂ :(0 ∧∀n:n n′ ) =4.∀n,m :n′=m′ n=m5.∀n :n′≠0InEnglish,wemightwritethemlikethis:1.Zeroisanaturalnumber.2.Everynaturalnumberhasasuccessor.3.Ifasubsetofnaturalnumberscontainszero,andeveryelementinthesubsethasasuccessorinthesubset,thenthesubsetcontainsallnaturalnumbers.

4.Iftwonaturalnumbershavethesamesuccessor,thentheyareequal.5.Zeroisnotthesuccessorofanynaturalnumber.

Thethirdaxiom,knownastheaxiomofinduction,isthemostimportant.ItsaysthatifwetakeanysubsetSof thatcontainszeroandobeystherulethatthesuccessorofeveryelementisalsoinS,thenSisthesameas .Anotherwaytoputthisis“therearenounreachablenaturalnumbers”;ifyoustartwithzeroandkeeptakingthesuccessor,you’lleventuallygettoeverynaturalnumber.Manymoderntextsputthisaxiomlast,butweusePeano’sorder.4

4Moderntextsoftenalsostartnaturalnumberswith1ratherthan0.

Peano’saxiomstransformedarithmetic.Infact,hewasbuildingonearlierworkbyRichardDedekindandHermannGrassman,bothofwhomshowedhowtoderivesomebasicprinciplesofarithmetic.ButPeanowentfurther,andhiscontributionsweresoimportantthatmathematicianssincethentalkaboutPeanoarithmetic,notjustarithmetic.

GiuseppePeano(1858–1932)

GiuseppePeanowasbornintoapeasantfamilynearTurininthenorthofItaly,rightaroundthetimeItalybecameaunifiedcountry.HeattendedtheUniversityofTurinandeventuallyjoinedthefacultythere.Later,healsobeganteachingattheRoyalMilitaryAcademy.Amonghisbest-knownachievementswasthediscoveryofthespace-fillingcurve,knownasaPeanocurve,which

providedacontinuousmappingfromaone-dimensionalsegmenttoeverypointonatwo-dimensionalsquare.Formostofthe1890s,Peanoworkedonthefoundationsofmathematicsandhisgreatbook

FormularioMathematico.Formulariowasmeanttobeacompendiumofallmathematicalresults,writtenformally.Itwasamasterpiece,notonlyprovidingafoundationformathematicsbutalsocoveringavarietyoftopics,togetherwithreferencestothesourcesintheiroriginallanguages.PeanogaveacopyofthebooktotheBritishphilosopherBertrandRussell,anditstronglyinfluencedRussell’sworkwithWhitehead,PrincipiaMathematica,whichwouldcometoplayanimportantroleinearlytheoriesofcomputation.Initially,PeanopublishedFormularioMathematicoinFrench,buthewasfrustratedbythe

ambiguityinherentinanynaturallanguage.Eventually,around1900,hedecidedthattheonlysolutionwastoinventanunambiguousuniversallanguageforscienceandmathematics,andtothenusethisforhiswriting.ThelanguagePeanodesignedwascalledLatinesineFlexione(“LatinwithoutInflection”),later

renamed“Interlingua.”HisideawastostartwithLatin,buttoreplaceallitsconfusingdeclensions,conjugations,andirregularwordswithasimple,logicalsetofrules.PeanorewroteFormularioinhisnewlanguage,andthiseditionwaspublishedin1908.Here’s

whathisfamousaxiomslookedlikeinInterlingua:0.N0esclasse,vel“numero”esnomencommune.

1.Zeroesnumero.2.Siaesnumero,tuncsuosuccessivoesnumero.3.N0esclasseminimo,quesatisfacadconditione0,1,2;[...]

4.Duonumero,quehabesuccessivoaequale,esaequaleinterse.5.0nonsequeullonumero.Peanoalsostartedusinghisformalnotationinhisteaching,whichprobablydidnotendearhim

tohisstudents.Inaddition,heturnedeverycoursehewassupposedtobeteachingintoadiscussionofthefoundationsofmathematics,whicheventuallycausedhimtolosehispositionatthemilitaryacademy.Peano’sdreamwasthatotherscientistswouldstartpublishingtheirworkinInterlingua,butthis

didnothappen.Infact,fewpeopleevenattemptedtoreadPeano’sbook,andhisworkwaslargelyignored.Towardtheendofhislife,PeanospentmuchofhistimetryingtopromoteInterlingua,andhewasmostlyforgottenbythemathematicalcommunity;theyweremoreinterestedintheworkofHilbertandothersatGöttingen.Eventoday,despiteembracingPeano’sfoundationalaxiomsofarithmetic,mostmathematicians

haveneverreadmorethanthefirstpageofhismonumentalwork.Ithasbeenoutofprintforyears,andhasneverbeentranslatedintoEnglish.

Toprovethateveryaxiomisneeded,weneedtoremoveeachonefromthesetanddemonstratethattheremainingsethasconsequencesthatdonotmeetourintent—inthiscase,thattheydonotcorrespondtowhatwemeanbynaturalnumbers.Removingexistenceof0axiom.Ifweremovethisaxiom,weareforcedtodropallaxiomsthatreferto

zero.Sincewehavenoelementstostartwith,theotheraxiomsneverapplyandcanbesatisfiedbytheemptyset,whichisclearlynotamodelofnaturalnumbers.

Removingtotalityofsuccessoraxiom.Ifweremovetherequirementthateveryvaluehaveasuccessor,thenweendupallowingfinitesetslike{0}or{0,1,2}.Clearly,nofinitesetsatisfiesournotionofnaturalnumbers.(However,oncomputers,wegiveupthisaxiom,sinceallofourdatatypesarefinite;forexample,auint64canexpressonlythefirst264integers.)Removinginductionaxiom.Ifweremovetheinductionaxiom,thenweendupwiththesituationwhere

wehavemoreinteger-likethingsthanthereareintegers.These“unreachable”numbersarecalledtransfiniteordinalsandaredesignatedbyω.Sowecouldendupwithsetslike{0,1,2,3,...,ω,ω+1,ω+2,...},{0,1,2,3,...,ω1,ω1+1,ω1+2,...,ω2,ω2+1,ω2+2,...},andsoon.

Removinginvertibilityofsuccessoraxiom.Ifweremovetherequirementthatequalsuccessorshaveequalpredecessors,thenwe’reallowing“ρ-shaped”structureswhereanitemcanhavemultiplepredecessors,someearlierinthesequenceandsomelater,suchas{0,1,1,1,...},{0,1,2,1,2,...},or{0,1,2,3,4,5,3,4,5,...}.Sinceallofthesestructuresarefinite,theyclearlydonotincludeallnaturalnumbers.Removing“nothinghas0asitssuccessor”axiom.Ifweremovethisaxiom,thenwe’dallow

structuresthatloopbacktozero,like{0,0,...}and{0,1,0,1,...}.Again,thesestructuresarefinite,sotheydonotcaptureournotionofnaturalnumbers.

9.7BuildingArithmeticNowthatwehaveestablishedthatallofPeano’saxiomsareindependent,andthereforenecessaryforournotionofnaturalnumbers,wecanbuilduparithmeticfromfirstprinciples.We’lldothisnowbydefiningexactlywhatitmeanstoaddandmultiplytwonaturalnumbers.DefinitionofAddition:

Wearenotprovingthesestatements;wearedefiningadditiontobethesestatements.Allpropertiesofaddingnaturalnumbersfollowfromthisdefinition.Forexample,here’showweprovethat0istheleftadditiveidentity:

Inthebasisstep,weassertthatit’struewhenaiszero.WeknowthisbecauseofEquation9.1inthedefinitionofaddition.Intheinductivestep,weassumeit’strueforanya.ByEquation9.2,weknowthat0+a′=(0+a)′.Butbytheassumptionoftheinductivestep,wecansubstituteafor0+a,soourresultisa′,andtherefore0+a′=a′.DefinitionofMultiplication:

Wecannowprovethat0·a=0,muchaswedidforaddition:

Definitionof1.Wealsodefine1asthesuccessorof0:

Nowweknowhowtoadd1:

Wealsoknowhowtomultiplyby1:

a·1=a·0′=a·0+a=0+a=a

Wecanderivefundamentalpropertiesofadditionaswell;theyfollowfromtheaxioms.AssociativityofAddition:(a+b)+c=a+(b+c)

inductivestep:

Togetcommutativity,we’llstartbyprovingitforthespecialcase:

inductivestep:

CommutativityofAddition:a+b=b+a

inductivestep:

Exercise9.3.Usinginduction,prove:

•Associativityandcommutativityofmultiplication•Distributivityofmultiplicationoveraddition

Exercise9.4.Usinginduction,definetotalorderingbetweennaturalnumbers.

Exercise9.5.Usinginduction,definethepartialfunctionpredecessoronnaturalnumbers.

***

DoPeanoaxiomsdefinenaturalnumbers?No;asPeanoputit,“number(positiveinteger)cannotbedefined(seeingthattheideasoforder,succession,aggregate,etc.,areascomplexasthatofnumber).”Inotherwords,ifyoudon’talreadyknowwhattheyare,Peano’sdefinitionswon’ttellyou.Instead,theydescribeourexistingideaofnumbers,formalizingournotionsofarithmetic,whichhelpsprovideawaytostructureproofs.Ingeneral,wecansaythataxiomsexplain,notdefine.Theexplanationmaynotbeconstructive;thatis,

itmightnotsayhowtheresultisachieved.Evenifitdoessuggestanalgorithm,thealgorithmcouldbecomputationallyveryinefficient.Nosanepersonwoulddoadditionbyrepeatedapplicationofthesuccessorfunction.Buttheseaxiomsstillserveausefulpurpose;theygetustothinkaboutwhichpropertiesofnaturalnumbersareessentialandwhicharenot.Thisapproachisagoodattitudetotakewhenstudyingthedocumentationforaprogramminginterface.

Whyisthatrequirementimposed?Whatwouldtheconsequencesbeifitwerenotthere?

9.8ThoughtsontheChapterWebeganthechapterbylookingatthenotionofproof,aformal—yetsocial—processfordemonstratingthetruthofaproposition.Wesawhowproofsshowconnectionsbetweentruths;proofsystemsareawaytoorganizeknowledge.Wealsolookedatthediscoveryoftheorems,andtheimportantabstractiontheyprovide.Nextwelookedataricherformalismfororganizingknowledge,theaxiomaticsystem,andsawhow

geometryandarithmeticcouldbebuiltupfromfirstprinciples.Thecriticalroleofaxiomaticsystemsistheirabilitytoreducethecomplexityofknowledge.Youdon’tneedtomemorizeallthetruepropositions,becauseyoucanderivethemfromafewaxiomsandinferencerules.However,it’simportanttorememberthathistorically,mathematiciansdidnotreallystartwithaxioms

andderivetheoremsfromthem.Theaxiomswereproposedonlyaftertheinterrelationshipsbetweenthe

theoremswerewellunderstoodandtheassumptionsunderlyingthemidentified.Thesameprocessholdsforprogramming:designinggoodabstractionsrequiresexaminingalargenumberofrealalgorithmsandunderstandingtheirinterrelationships.Whileaxiomaticsystemsallowustoorganizeknowledge,theypresupposethatwealreadyhavesome

knowledgetoorganize.Discoveryofatheoremisamoreimportantthingthanprovingit—youcannotattempttoprovesomethingunlessyouhavereasontobelieveitisatruth.Sometimesmodernmathematiciansforgettheempiricaloriginsofknowledge.InhisbookTheMethod,

thegreatGreekmathematicianArchimedesdiscussedhowanymeansforacquiringmathematicalknowledgewasvalid,includingmeasurementandexperimentation.Onlyafterdiscoveringmathematicaltruthsshouldoneattempttoderivearigorousproof.Thesameprincipleappliestoprogramming:beforetryingtoproveaprogramcorrect,weshouldtrytowritecorrectprograms—evenifourattemptsinvolvetrialanderror.

10.FundamentalProgrammingConcepts

Allhumansnaturallydesiretoknow.Aristotle,MetaphysicsI,1

Inthischapterwewillintroducesomeoftheimportantideasassociatedwithgenericprogramming,includingconceptsanditerators.We’llalsoconsidersomecommonprogrammingtasksthatrelyonthem.Butwe’llstartbylookingattheoriginsofthenotionofabstraction.

10.1AristotleandAbstractionTheSchoolofAthens,afamouspaintingbyItalianRenaissancepainterRaphael,depictsmanyancientGreekphilosophers(seedetailofpaintingonthenextpage).AtthecenterarePlatoandAristotle,thetwomostimportantphilosophersoftheancientworld.Platoispointingupward,whilehisstudentAristotleholdshishandoutovertheground.Accordingtopopularinterpretation,Platoispointingtotheheavens,indicatingthatweshouldcontemplatetheeternal,whileAristotleisindicatingthatweneedtostudytheworld.Infact,itcouldbesaidthatPlatoinventedmathematicsandAristotleinventedthestudyofeverythingelse,especiallyscience.Aristotle’sworkscovereverythingfromaestheticstozoology.

Aristotle(384BC–322BC)

AristotlecamefromStageira,acityinthefarnorthofGreece.Weknowverylittleabouthisearlylife,butweknowthatatsomepointhedecidedtomovetoAthensinsearchofwisdom.Hestudied(andatsomepoint,probablytaught)atPlato’sAcademyforabout20years.AroundthetimeofPlato’sdeath—perhapsdisappointedbecausehewasnotappointedPlato’ssuccessor—heleftAthens.In343BC,KingPhilipofMacedonappointedAristotletobeatutorforhissonAlexanderand

Alexander’scompanions.Wedon’tknowmuchaboutAristotle’srelationshipwiththeprince,butinlateryearswhenAlexanderbecamekingandbegantheAsianconqueststhatearnedhimthenickname“theGreat,”hesentAristotleexoticplantandanimalspecimensforthephilosopher’scollection.Around335BC,AristotlereturnedtoAthensandcreatedhisowngreatschool,theLyceum.Duringthe

next12years,heproducedthemostastonishingcollectionofknowledgeeverassembled.WhilehisteacherPlatohadfocusedonstudyingeternaltruths,Aristotlewantedtounderstandtheworldasitreallywas.Forexample,whenPlatowantedtowriteaboutpolitics,hedescribedwhattheidealsocietywouldbe.WhenAristotlewroteaboutpolitics,heaskedhisstudentstovisiteachoftheimportantGreekcity-statesandthenreportontheirconstitutions.Aristotle’sapproachwastoobserveeverything,describeit,andcomeupwithexplanationsforwhathesaw.Aristotletaught,andwroteabout,nearlyeverysubjectimaginable.Accordingtoseveralimportant

writersoftheperiod,Aristotlewrotebeautifully;unfortunatelythebooksheintendedforpublication,suchashisdialogues,haveallbeenlost.Whatwehaveinsteadaretersetreatisesthatwereprobablylecturenotes.Nevertheless,manyofhisworks,suchasNicomacheanEthics,Politics,andMetaphysicsarestillessentialreadingtoday.Andregardlessoftheirstyle,hiscollectedworksconstitutethefirstencyclopedictreatmentofknowledge.AlthoughAristotle’sscientificdescriptionsincludefactualerrors,hewasthefirstpersonto

systematicallydescribethescientificworld,withobservationsontopicsasdetailedashowtheoctopusreproduces.AristotleleftAthensaround322BCanddiedsoonafterward.Whileotherphilosophicaltraditions

(e.g.,Stoicism,Platonism)persistedinancienttimeslongaftertheirfounders’deaths,Aristotle’sdidnot.GreekphilosophybecameincreasinglyintrospectiveandlostinterestinAristotle’sideaofstudyingobservablereality.TheLyceumrapidlydeclinedinimportance,andfewscholarsinlaterGreekand

RomantimesconsideredthemselvesAristotelians.EvenwhenhisworkwasrediscoveredintheMiddleAges,medievalscholarsslavishlystudiedhiswritingsratherthanfollowinghismethodologyofgoingoutandobservingtheworld.Aristotle’sgreatlegacyishisempiricalapproach,whichisthefoundationalprincipleofallmodernscience.Furthermore,theorganizationofknowledgereflectedinthedepartmentsofmodernuniversitiesisadirectdescendentofthetaxonomythatAristotleproposed.

Aristotle’swritingswerepreservedtowardtheendofthefirstmillenniumADbyArabphilosophers.Inthe12thcentury,whenChristiankingdomsrecapturedmuchofSpainfromtheIslamicstateknownasAl-Andalus,theyfoundalibraryintheSpanishcityofToledocontainingagreatnumberofbooksinArabic,includingtranslationsofAristotle’sworks.EventuallythebooksweretranslatedfromtheoriginalGreekintoLatin,andanAristotelianrenaissancespreadthroughEurope.Aristotlebecameknownassimply“thePhilosopher,”andhisworksbecameapartofgenerallyacceptedknowledge.ThegreatAndalusianphilosopherIbnRushd(knownasAverroestoEuropeans)wrotewidelyreadcommentariesonAristotle,reconcilinghisphilosophywiththeteachingsofIslam;hebecameknownsimplyas“theCommentator.”Inthe13thcentury,ChristianscholarssuchasThomasAquinasandDunsScotussimilarlyshowedthatAristotelianismwascompatiblewithChristianity;thiswasoftendescribedas“baptizing”Aristotle.Asaresult,Aristotle’sworkswerepartoftherequiredstudyatEuropeanuniversitiesforliterallyhundredsofyears.AmongAristotle’smostimportantworksistheOrganon,acollectionofsixtreatisesonvarious

aspectsoflogicthatwoulddefinethefieldforthenext2600years.1InCategories,thefirstpartoftheOrganon,Aristotleintroducedthenotionofabstraction.Hewroteaboutthedistinctionbetweenanindividual,aspecies,andagenus.Whiletodaymostpeoplethinkoftheseasbiologicaldistinctions,forAristotletheyappliedtoeverything.Aspeciesincorporatesallthe“essential”propertiesofatypeofthing.Agenusmaycontainmanyspecies,eachidentifiedbyitsdifferentia—thethingsthatdifferentiateitfromotherspeciesinthegenus.

1UnlikeAristotle’sotherworks,aLatinversionoftheOrganonwasavailabletoEuropeansmuchearlier;Boethiusprovidedthetranslationintheearly6thcentury.

ItisAristotle’sideaofgenusthatinspiredthetermgenericprogramming—awaytothinkaboutprogrammingthatfocusesonthelevelofgenera(thepluralofgenus)ratherthanspecies.

10.2ValuesandTypesNowwewillseehowsomeoftheideaswe’vebeendiscussingfitincomputerprogramming.First,weneedafewdefinitions:

Definition10.1.Adatumisasequenceofbits.

01000001isanexampleofadatum.

Definition10.2.Avalueisadatumtogetherwithitsinterpretation.

Adatumwithoutaninterpretationhasnomeaning.Thedatum01000001mighthavetheinterpretationoftheinteger65,orthecharacter“A,”orsomethingelsealtogether.Everyvaluemustbeassociatedwithadatuminmemory;thereisnowaytorefertodisembodiedvaluesinalanguagelikeC++orJava.

Definition10.3.Avaluetypeisasetofvaluessharingacommoninterpretation.

Definition10.4.Anobjectisacollectionofbitsinmemorythatcontainavalueofagivenvaluetype.

Thereisnothinginthedefinitionthatsaysthatallthebitsofanobjectmustbecontiguous.Infact,it’squitecommonforpartsofanobjecttobelocatedatdifferentplacesinmemory;thesearecalledremoteparts.Anobjectisimmutableifthevalueneverchanges,andmutableotherwise.Anobjectisunrestrictedif

itcancontainanyvalueofitsvaluetype.

Definition10.5.Anobjecttypeisauniformmethodofstoringandretrievingvaluesofagivenvaluetypefromaparticularobjectwhengivenitsaddress.

Whatwecall“types”inprogramminglanguagesareobjecttypes.C++,Java,andotherprogramminglanguagesdonotprovidemechanismsfordefiningvaluetypes.2Everytyperesidesinmemoryandisanobjecttype.

2Theiteratortraitvalue_typeinC++,despiteitsname,doesnotactuallyreturnavaluetypeinthesensedescribedhere.Rather,itreturnstheobjecttypeofthevaluepointedtobytheiterator.

10.3ConceptsTheessenceofgenericprogrammingliesintheideaofconcepts.Aconceptisawayofdescribingafamilyofrelatedobjecttypes.Therelationshipbetweenconceptandtypeisexactlytherelationshipbetweentheoryandmodelinmathematics,andbetweengenusandspeciesinthescientifictaxonomyintroducedbyAristotle.

HerearesomeexamplesofconceptsandsomeoftheirtypesinC++:•Integral:3int8_t,uint8_t,int16_t,...

3Thisrefersspecificallytothelistofbuilt-inC++integraltypes.ThereisabroaderconceptIntegerthatincludesalloftheseplusotherrepresentationsofintegerssuchasinfinite-precisionintegers.

•UnsignedIntegral:uint8_t,uint16_t,...•SignedIntegral:int8_t,int16_t,...

Whileconceptsexistinmanylanguagesimplicitly,veryfewlanguagesprovideanexplicitwaytotalkaboutthem.4

4TherehavebeenproposalstoincludeconceptsinC++;theworkisstillinprogress.Therearealsoconcept-likefeaturesinsomefunctionalprogramminglanguagessuchasHaskell.

Manyprogramminglanguagesprovideamechanismtospecifytheinterfaceofatypetobeimplementedlater:abstractclassesinC++,interfacesinJava,andsoon.However,thesemechanismscompletelyspecifytheinterface,includingstrictrequirementsonthetypesofargumentsandreturnvalues.Incontrast,conceptsallowinterfacestobespecifiedintermsoffamiliesofrelatedtypes.Forexample,inbothJavaandC++,youcanspecifyaninterfacecontainingafunctionsize()returningavalueoftypeint32.Intheworldofconcepts,youcanhaveaninterfacewithafunctionsize()thatreturnsavalueofanyintegraltype—uint8,int16,int64,etc.

Aconceptcanbeviewedasasetofrequirementsontypes,orasapredicate5thattestswhethertypesmeetthoserequirements.Therequirementsconcern

5Apredicateisafunctionthatreturnstrueorfalse.

•Theoperationsthetypesmustprovide•Theirsemantics•Theirtime/spacecomplexity

Atypeissaidtosatisfyaconceptifitmeetstheserequirements.Whenfirstencounteringconcepts,programmersoftenwonderwhythethirdrequirement,forspace/time

complexity,isincluded.Isn’tcomplexityjustanimplementationdetail?Toanswerthisquestion,considerareal-worldexample.Supposeyoudefinedtheabstractdatatypestack,butyouimplementeditasanarray,insuchawaythateverytimeyoupushedsomethingontothearray,youhadtomoveeveryexistingelementtothenextpositiontomakeroom.Insteadofpushingonthestackbeingafastoperation(constanttime),it’snowaslowoperation(lineartime).Thisviolatesaprogrammer’sassumptionofhowstacksshouldbehave.Inasense,astackthatdoesn’thavefastpushandpopisnotreallyastack.Thereforetheseverybasiccomplexityconstraintsarepartofwhatitmeanstosatisfyaconcept.

***

Twousefulideasrelatedtoconceptsaretypefunctionsandtypeattributes.Atypefunctionisafunctionthat,givenatype,returnsanaffiliatedtype.Forexample,itwouldbenicetohavetypefunctionslikethis:

•value_type(Sequence)•coefficient_type(Polynomial)•ith_element_type(Tuple,size_t)

Unfortunately,mainstreamprogramminglanguagesdonotcontaintypefunctions,eventhoughtheywouldbeeasytoimplement.(Afterall,thecompileralreadyknowsthingslikethetypeofelementsinasequence.)Atypeattributeisafunctionthat,givenatype,returnsavaluerepresentingoneofitsattributes.For

example:•sizeof•alignment_of•Numberofmembersinastruct•Nameofthetype

Somelanguagesprovidesometypeattributes,likesizeof()inCandC++.

***

Let’slookatsomeverygeneralconcepts.ThefirstoneisRegular,6whichweintroducedbackinChapter7.Roughlyspeaking,atypeisregularifitsupportstheseoperations:

6Byconvention,wewriteconceptnameswithinitialcapitals,anddisplaythemwithaSansSeriftypeface.

•Copyconstruction•Assignment•Equality

•DestructionHavingacopyconstructorimplieshavingadefaultconstructor,sinceTa(b)shouldbeequivalenttoTa;a=b;.TodescribethesemanticsofRegular,we’llexpresstherequirementsasaxioms:

Thefirstaxiomsaysthatifyoucopyconstructafromb,thenanythingthatwasequaltobwillnowalsobeequaltoa.Thesecondaxiomsaysthatifyouassignbtoa,thenanythingthatwasequaltobwillnowalsobeequaltoa.Thethirdaxiomusesthenotionofaregularfunction(nottobeconfusedwitharegulartype),whichisonethatproducesequalresultsgivenequalinputs.It’stheresponsibilityoftheprogrammertospecifywhichfunctionsaresupposedtoberegular;onlythencanotherprogrammers(or,inthefuture,thecompiler)relyonthefactthatthesefunctionswillpreserveequality.ThecomplexityrequirementsonRegulararethateachoperationisnoworsethanlinearintheareaof

theobject,whereareaisdefinedasallspaceoccupiedbytheobject,bothitsheaderanditsremoteparts,bothitsdataanditsconnectors.7

7ForamoreformaltreatmentoftheconceptRegular,seeElementsofProgrammingSection1.5.

TheconceptRegularisuniversal—it’snotspecifictoanyprogramminglanguage.Atypeinanylanguageisregularifitsatisfiestherequirements.ArelatedconceptisSemiregular,whichisjustlikeRegularexceptthatequalityisnotexplicitly

defined.Thisisneededinafewsituationswhereitisverydifficulttoimplementanequalitypredicate.Eveninthesesituations,equalityisassumedtobeimplicitlydefined,sothataxiomsthatcontrolcopyingandassignmentarestillvalid.Afterall,aswesawearlier,themeaningofassigningatobisthatb’svaluewillbeequaltoa’svalueafterward.

10.4IteratorsAniteratorisaconceptusedtoexpresswhereweareinasequence.Infact,iteratorswereoriginallygoingtobecalled“coordinates”or“positions”;theymaybeviewedasageneralizationofpointers.Insomeprogramminglanguages,whattheycalliteratorsareheavyweightbundlesoffunctionality,buttheconceptofaniteratorjustexpressesthisverysimplenotionofposition.Tobeaniterator,atypemustsupportthreeoperations:•Regulartypeoperations•Successor•Dereference

Onewaytothinkofaniteratoris“somethingthatletsyoudolinearsearchinlineartime.”Theessenceofaniteratoristhenotionofsuccessor.Indeed,iteratorscometousdirectlyfromPeano’saxioms;essentially,theconceptIteratoris“atheorywithsuccessor.”However,ouriteratorconceptswillbelessstrict,becausewedon’trequireallofPeano’saxioms.Forexample,inPeanoarithmetic,everynumberhasasuccessor,whilewithiterators,thisisnotalwaysthecase—sometimeswegettotheendofourdata.Peanoalsotellsusthatifsuccessorsareequal,thepredecessorsmustbeequal,andthatwecan’thaveloops.Theserequirementsarealsonotthecaseforprogrammers;we’reallowedtohavedatastructuresthatlinkbacktoearlierelementsandformloops.Infact,thisissometimesexactlywhatweneedtodoacomputationaltaskefficiently.

Theseconditeratoroperation,dereferencing,isawaytogetfromaniteratortoitsvalue.Dereferencinghasatimecomplexityrequirement;it’sassumedtobe“fast,”whichmeansthatthereisnotafasterwayofgettingtodatathanthroughtheiterator.Iteratorsaresometimesbiggerthantraditionalpointers,insituationswheretheyneedtostoresomeadditionalstatetoenablefastnavigation.Iteratorsmayalsosupportspecialvaluesindicatingthatwearepasttheendoftheobject,aswellassingularvalueslikethenullpointerthatcannotbedereferenced.It’sokaythatdereferencingisapartialfunction(i.e.,thatitisn’tdefinedforallvalues);afterall,mathematicianshavenotroublesayingwhatdivisionis,eventhoughdivisionbyzeroisnotdefined.Dereferencingandsuccessorarecloselyconnected,8andthisrelationshipimposesthefollowing

restrictions:8SeeChapter6ofElementsofProgrammingformoreabouttherelationshipbetweendereferencingandsuccessor.

•Dereferencingisdefinedonaniteratorifandonlyifsuccessorisdefined.•Ifyouarenotattheendoftherange,youcandereference.

Whydoweneedequalityasarequirementforiterators?Inotherwords,whydoweneediteratorstoberegularratherthansemiregular?Becauseweneedtobeabletoseewhenoneiteratorreachesanother.

10.5IteratorCategories,Operations,andTraitsThereareseveralkindsofiterators,whichwecalliteratorcategories.Herearethemostimportant:

•Inputiteratorssupportone-directionaltraversal,butonlyonce,asisfoundinsingle-passalgorithms.Thecanonicalmodelofaninputiteratoristhepositioninaninputstream.Bytesarecomingoverthewireandwecanprocessthemoneatatime,butoncetheyareprocessed,theyaregone.Inparticular,withinputiteratorsi==jdoesnotimply++i==++j;forexample,ifyou’vealreadyconsumedacharacterfromaninputstream,youcan’tconsumethesamecharacteragainwithadifferentiterator.Keepinmindthatjustbecauseanalgorithmonlyrequiresinputiteratorsdoesnotmeanitislimitedtooperatingoninputstreams.•Forwarditeratorsalsosupportonlyone-directionaltraversal,butthistraversalcanberepeatedasneeded,asinmulti-passalgorithms.Thecanonicalmodelofaforwarditeratoristhepositioninasinglylinkedlist.9

9Weassumethatlinkstructureofthelistisnotmodifiedasitistraversed.

•Bidirectionaliteratorssupportbidirectionaltraversal,repeatedasneeded(i.e.,theyalsocanbeusedinmulti-passalgorithms).Thecanonicalmodelofabidirectionaliteratoristhepositioninadoublylinkedlist.Bidirectionaliteratorshaveaninvertiblesuccessorfunction:ifanelementxhasasuccessory,thenyhasapredecessor.•Random-accessiteratorssupportrandom-accessalgorithms;thatis,theyallowaccesstoanyelementinconstanttime(bothfarandfast).Thecanonicalmodelisthepositioninanarray.

Inaddition,thereisanothercommoniteratorcategorythatbehavesdifferentlyfromtheothers:•Outputiteratorssupportalternatingsuccessor(++)anddereference(*)operations,buttheresultsofdereferencinganoutputiteratorcanappearonlyontheleft-handsideofanassignmentoperator,andtheyprovidenoequalityfunction.Thecanonicalmodelofanoutputiteratoristhepositioninanoutputstream.Wecan’tdefineequalitybecausewecan’tevengettotheelementsoncethey’vebeenoutput.

WhiletheiteratorsdescribedsofararetheonlyonesincludedinC++,otherusefuliteratorconceptsalsoexist:

•Linkediteratorsworkinsituationswherethesuccessorfunctionismutable(forexample,alinkedlistwherethelinkstructureismodified).•Segmentediteratorsareforcaseswherethedataisstoredinnoncontiguoussegments,eachcontainingcontiguoussequences.std::deque,adatastructurethatisimplementedasasegmentedarray,wouldimmediatelybenefit;insteadofneedingeachsuccessoroperationtocheckwhethertheendofthesegmenthasbeenreached,a“toplevel”iteratorcouldfindthenextsegmentandknowitsbounds,whilethe“bottomlevel”iteratorcoulditeratethroughthatsegment.

Iteratorslikethesecaneasilybeimplemented.Justbecauseaconceptisnotbuiltintothelanguagedoesnotmeanit’snotuseful.Ingeneral,STLshouldbeviewedasasetofwell-chosenexamples,notanexhaustivecollectionofallusefulconcepts,datastructuresandalgorithms.

***

Asimplebutimportantthingwemaywanttodoisfindthedistancebetweentwoiterators.Foraninputiterator,wemightwriteourdistance()functionlikethis:Clickheretoviewcodeimage

template<InputIteratorI>

DifferenceTypedistance(If,Il,std::input_iterator_tag){

//precondition:valid_range(f,l)

DifferenceTypen(0);

while(f!=l){

++f;

++n;

}

returnn;

}

Therearethreenotablethingsaboutthiscode:theuseofthetypefunctionDifferenceType,theuseoftheiteratortagargument,andtheprecondition.We’lldiscussallofthesesoon,butbeforewedo,let’scomparethistoadifferentimplementation—onethat’soptimizedforrandomaccessiterators:Clickheretoviewcodeimage

template<RandomAccessIteratorI>

DifferenceTypedistance(If,Il,

std::random_access_iterator_tag){

//precondition:valid_range(f,l)

returnl-f;

}

Sincewehaverandomaccess,wedon’thavetorepeatedlyincrement(andcount)fromoneiteratortotheother;wecanjustuseaconstanttimeoperation—subtraction—tofindthedistance.Thedifferencetypeofaniteratorisanintegraltypethatislargeenoughtoencodethelargestpossible

range.Forexample,ifouriteratorswerepointers,thedifferencetypeinC++couldbeptrdiff_t.Butingeneralwedon’tknowinadvancewhichtypetheiteratorwillbe,soweneedatypefunctiontogetthedifferencetype.AlthoughC++doesnothaveageneralmechanismfortypefunctions,STLiteratorshaveaspecialsetofattributesknownasiteratortraits,oneofwhichgivesusthedifferencetype.Thecompletesetofiteratortraitsis

•value_type•reference•pointer

•difference_type•iterator_category

We’vementionedvalue_typebefore;itreturnsthetypeofthevaluespointedtobytheiterator.Thereferenceandpointertraitsarerarelyusedincurrentarchitectures,10buttheothersareveryimportant.

10EarlierversionsoftheIntelprocessorarchitectureincludeddifferenttypesforshorterandlongerpointers,soitwasimportanttoknowwhichtouseforagiveniterator.Today,ifthevaluetypeofaniteratorisT,thepointeriteratortraitwouldnormallybeT*.

Sincethesyntaxforaccessingiteratortraitsisratherverbose,we’llimplementourowntypefunctionforaccessingdifference_type,withtheusingconstructofC++11.(SeeAppendixCformoreinformationaboutusing.)Clickheretoviewcodeimage


usingDifferenceType=

typenamestd::iterator_traits::difference_type;

ThisgivesustheDifferenceTypetypefunctionusedintheearliercode.Theiteratortraititerator_categoryreturnsatagtyperepresentingthekindofiteratorwe’re

dealingwith.Objectsofthesetagtypescontainnodata.AswedidforDifferenceType,wedefinethefollowingtypefunction:Clickheretoviewcodeimage


usingIteratorCategory=

typenamestd::iterator_traits::iterator_category;

Nowwecanreturntotheuseoftheiteratortagargumentinthedistancefunctions.Theiteratortagsshownintheexamples(input_iterator_tagandrandom_access_iterator_tag)arepossiblevaluesoftheiteratorcategorytrait,sobyincludingthemasarguments,wearedistinguishingthetypesignatureofthetwofunctionimplementations.(WewillseemoreexamplesofthisinChapter11.)Thisallowsustoperformcategorydispatchonthedistancefunction;thatis,wecanwriteageneralformofthefunctionforanyiteratorcategory,andthefastestonewillbeinvoked:Clickheretoviewcodeimage


DifferenceTypedistance(If,Il){

returndistance(f,l,IteratorCategory());

}

Notethatthethirdargumentisactuallyaconstructorcallcreatinganinstanceoftheappropriatetype,becausewecannotpasstypestofunctions.Whentheclientcallsdistance(),itusesthetwo-argumentversionshownhere.Thatfunctiontheninvokestheimplementationthatmatchestheiteratorcategory.Thisdispatchhappensatcompiletimeandthegeneralfunctionisinline,sothereisliterallynoperformancepenaltyforchoosingtherightversionofthefunction.Theuseoftagtypesasargumentstodistinguishversionsofthefunctionmayseemredundant,sincewe

alreadyspecifieddifferentconceptsinthetemplates.However,recallthatouruseofconceptsservesonlyasdocumentationfortheprogrammer;currentC++compilersdon’tknowanythingaboutconcepts.Onceconceptsareaddedtothelanguage,thearcaneiteratorcategorytagmechanismwillnolongerbeneeded.

10.6RangesArangeisawayofspecifyingacontiguoussequenceofelements.Rangescanbeeithersemi-openorclosed;11aclosedrange[i,j]includesitemsiandj,whileasemi-openrange[i,j)includesibutendsjustbeforej.Itturnsoutthatsemi-openrangesarethemostconvenientfordefininginterfaces.Thisisbecausealgorithmsthatoperateonsequencesofnelementsneedtobeabletoreferton+1positions.Forexample,therearen+1placestoinsertanewitem:beforethefirstelement,betweenanytwoelements,orafterthelastelement.Also,semi-openranges,unlikeclosedranges,candescribeanemptyrange.Furthermore,asemi-openemptyrangecanbespecifiedatanyposition;itprovidesmoreinformationthanasimple“nil”oremptylist.

11Inmathematics,therearealsoopenranges,buttheyarelessusefulinprogramming,sowedonotincludethemhere.

Arangecanbespecifiedinoneoftwoways:aboundedrangehastwoiterators(onepointingtothebeginningandonepointingjustpasttheend),whileacountedrangehasaniteratorpointingtothebeginningandanintegernindicatinghowmanyitemsareincluded.Thisgivesusfourkindsofrangesaltogether:

(Aclosedcountedrangemusthaven>0.)Asweshallsee,therearedifferentsituationswhereboundedorcountedrangesarepreferable.Whilemathematicaltextsindexsequencesfrom1,computerscientistsstartfrom0,andwewillusethe

latterconventionforourranges.Interestingly,although0-basedindexingincomputersciencewasinitiallyusedasawaytoindicatetheoffsetinmemory,thisconventionturnsouttobemorenaturalregardlessofimplementation,sinceitmeansthatforasequencewithnelements,theindicesareintherange[0,n)andanyiterationisboundedbythelength.

***

Nowwecanreturntothethirdnotablefeatureofourdistancefunctions:thevalid_rangeprecondition.Itwouldbeniceifwecouldhaveavalid_rangefunctionthatreturnedtrueiftherangespecifiedbythetwoiteratorswasvalidandfalseotherwise,butunfortunately,it’snotpossibletoimplementsuchafunction.Forexample,iftwoiteratorseachrepresentcellsinalinkedlist,wehavenowayofknowingifthere’sapathfromonetotheother.Butevenifwe’redealingwithsimplepointers,westillcannotcomputevalid_range:thereisnowayinCorC++todetermineiftwopointerspointtoasinglecontiguousblockofmemory;theremightbegapsinthemiddle.Sowecan’twriteavalid_rangefunction,butwecanstilluseitasaprecondition.Insteadof

guaranteeingthecorrectbehaviorincode,we’lluseaxiomsthat,ifsatisfied,ensurethatourdistancefunctionwillbehaveasintended.Specifically,wepostulatethefollowingtwoaxioms:

Thefirstaxiomsaysthatifit’sacontainer,therangefrombegin()toend()isvalid.Thesecondaxiomsaysthatif[x,y)isanonemptyvalidrange,thentherange[successor(x),y)isalsovalid.AllSTL-stylecontainers,aswellasC++arrays,mustobeytheseaxioms.Thisallowsustoprovethealgorithmscorrect.Forexample,ifyougobacktoouroriginaldistancefunctionforinputiteratorsinSection10.5,you’llseethatthesecondaxiomensuresthatifwestartwithavalidrange,we’llstillhaveoneeachtime

throughtheloop.

***

Inadditiontothesuccessor(++)anddistanceoperations,it’susefultohaveawaytomoveaniteratorbyseveralpositionsatonce.Wecallthisfunctionadvance.Asbefore,we’llimplementtwoversions,oneforinputiterators:Clickheretoviewcodeimage


voidadvance(I&x,DifferenceTypen,std::input_iterator_tag){

while(n){

--n;

++x;

}

}

andanotherforrandomaccessiterators:Clickheretoviewcodeimage


voidadvance(I&x,DifferenceTypen,

std::random_access_iterator_tag){

x+=n;

}

We’llalsoprovidewithatop-levelfunctionfordoingthedispatch:Clickheretoviewcodeimage


voidadvance(I&x,DifferenceTypen){

advance(x,n,IteratorCategory());

}

10.7LinearSearchLinearsearchisafundamentalprogrammingtaskthatallprogrammersshouldunderstand.Thesimplestideaoflinearsearchistoscanalinearlistuntilwefindaspecificelement.Butwewillgeneralizethattoafunctionthatscansthelistuntilitfindsanelementthatsatisfiesagivenpredicate.Soinadditiontobeingabletosearchforaspecificvalue,wecouldfind,forexample,thefirstoddelement,orthefirstelementthathasnovowels,orwhateverelsewelike.Ofcourse,theremightnotbesuchanelement,soweneedsomewaytoindicatethatnoitemisfound.We’llcallourfunctionfind_if—“finditifit’sthere”:12

12ThisnameoriginatedintheprogramminglanguageCommonLisp.


template<InputIteratorI,PredicateP>

Ifind_if(If,Il,Pp){

while(f!=l&&!p(*f))++f;

returnf;

}

Thisfunctionreliesonequality,dereference,andsuccessor.Ifnoitemthatsatisfiesthepredicateexists,thereturnedvalueoffwillbethesameasl,theiteratorthatpointspasttheendoftherange.Thecallingfunctionusesthiscomparisontodeterminewhetheramatchingitemhasbeenfound.CandC++guarantee

thatapointerisvalidonepositionpasttheendofanarray.However,suchapointershouldneverbedereferenced.AllSTLcontainersprovidesimilarguaranteesfortheiriterators.Thisfunctionhasanimplicitsemanticprecondition:thevaluetypeoftheiteratorandtheargumenttype

ofthepredicatemustbethesame;otherwise,thereisnowaytoapplythepredicatetotheitemsintherange.Here’savariationofourlinearsearchfunctionfortheinputiteratorcase.Althoughwecouldhave

overloadedthename,we’vedeliberatelyadded_ntoemphasizethatthisversionusesacountedrange:Clickheretoviewcodeimage

template<InputIteratorI,PredicateP>

std::pair<I,DifferenceType>

find_if_n(If,DifferenceTypen,Pp){

while(n&&!p(*f)){++f;--n;}

return{f,n};

}

Whydowereturnapair?Wouldn’titbesufficienttoreturntheiteratorthatpointstothefoundelement,aswedidinthepreviousversion?No.Inthepreviousversion,thecallerhadthe“last”iteratortocompareto;here,itdoesnot.Sothesecondreturnedvaluetellsthecallerwhetherwe’reattheend,inwhichcasetheitemwasnotfoundandthereturnediteratorcannotbedereferenced.Butjustasimportantly,ifwedofindamatchingitem,thisallowsustorestartthesearchwhereweleftoff.Withoutthis,therewouldbenowaytosearcharangeforanythingbutthefirstoccurrenceofadesireditem.Thisillustratesanimportantpoint:justastherecanbebugsincode,sotherecanalsobebugsinthe

interface.We’llseeanexampleofoneinthenextsection.

10.8BinarySearchIfwehaveasortedsequence,wecansearchitmuchmoreefficientlybyusinganotheressentialalgorithm,binarysearch.Thisfunctioniseasytodescribe,buthardtoimplementcorrectlyandevenhardertospecify(designaninterfacefor)correctly.

OriginsofBinarySearchTheideaofbinarysearchoriginatedintheIntermediateValueTheorem(IVT),alsoknownastheBolzano-CauchyTheorem:Iffisacontinuousfunctioninaninterval[a,b]suchthatf(a)<f(b),then∀u [f(a),f(b)]thereisc [a,b]suchthatu=f(c).

Theproofofthetheoremconsistsofdoingbinarysearch.Suppose,asanexample,wehaveacontinuousfunctionsuchthatf(a)is–3andf(b)is5.TheIVTtellsusthatforaparticularpointintheimageofthefunction—let’ssay,0—thereisapointcinthedomainsuchthatf(c)=0.Howcanwefindthatpoint?Wecanstartbyfindingthepointx1thatishalfthedistancebetweenaandb,andcomputingf(x1).Ifitequals0,we’redone;we’vefoundc.Ifit’sgreaterthan0,werepeatwithapointx2that’shalfthedistancefromatox1.Ifit’sless,wetakehalfthedistancefromx1tob.Aswekeeprepeatingtheprocess,weasymptoticallyconvergetoc.SimonStevinhadasimilarideain1594,whenhedevisedaversionoftheIVTforpolynomials.

ButsinceStevinwasinterestedindoingeverythingwithdecimals,heactuallydividedtheintervalintotenthsratherthanhalves,andexaminedallofthemuntilhefoundthetenththatcontainedthedesiredvalue.ItwasLagrangewhofirstdescribedthebinaryapproachforpolynomialsin1795.BolzanoandCauchygeneralizedtheIVTintheearly19thcentury,anditistheirversionusedbymathematicianstoday.

Binarysearchwasfirstdiscussedasaprogrammingtechniquein1946byphysicistandcomputingpioneerJohnMauchly,co-creatoroftheENIAC(thefirstelectronicgeneral-purposecomputer).However,manydetailswereleftunspecified.Thefirst“correct”algorithmforperformingbinarysearchwaspublishedin1960byD.H.Lehmer,amathematicianwhohadworkedontheENIACyearsearlier.However,Lehmer’sversiondidnotusethecorrectinterface,andthiserrorwasrepeatedforseveraldecadesafterward.AnexampleoftheerroneousinterfaceremainstodayintheUNIXbsearch()function.Accordingto

thePOSIX13standard:13POSIXisthesetofstandardsforUNIX-likeoperatingsystems.Linux,forexample,isaPOSIX-compliantOS.

Thebsearch()functionshallreturnapointertoamatchingmemberofthearray,oranullpointerifnomatchisfound.Iftwoormorememberscompareequal,whichmemberisreturnedisunspecified.14

14http://www.unix.com/man-page/POSIX/3posix/bsearch/

Therearetwofundamentalflawswiththisinterface.Thefirstconcernsthereturnofanullpointertoindicatethattheitemisnotfound.Oftenyouaredoingthesearchbecauseyouwanttoinsertanitemifitisn’tthere,attheplaceitwouldhavebeenifitwerethere.Withthisinterface,youhavetostartfromscratchtofindyourinsertposition,thistimeusinglinearsearch!Furthermore,therearemanyapplicationswhereyouactuallywanttofindtheclosestornextvaluetowherethemissingitemwouldbe;theitemyou’researchingformaysimplybeaprefixofthekindsofitemsyouhopetofind.Thesecondflawconcernsthesituationwheretherearemultiplematches.Thematchesmaybekeysof

itemsthatyouneedtoretrieve.Sohowdoyouobtaintheentirerangeofequalitems,ifyoudon’tknowwhichoneyou’reon?Youhavetodolinearsearchbothbackwardandforwardtofindtheendsofthematchingsequence.

http://www.unix.com/man-page/POSIX/3posix/bsearch/

***

Therightwaytoimplementbinarysearchbeginswiththeideaofapartitionpoint.Assumethatwehaveasequenceofitems[f,l)thatisarrangedsuchthatacertainpredicateistrueoftheitemsintherange[f,m)andfalsefor[m,l).15Thenthepartitionpointisthepositionm.Formally,ifwehaveafunctionthatcomputesthepartitionpoint,thenitspreconditionis

15Inretrospect,itwouldhavebeenbettertohavethefalseitemsfirst,sincetheBooleanvaluefalsesortsbeforetrue;unfortunately,the“wrong”orderisnowpartoftheC++languagestandard.

(i.e.,theelementsarealreadypartitionedasdescribedearlier).Itspostconditionisthatitreturnsthevaluemfromtheprecondition.Notethatfinthisexpressionreferstothefirstelementintherange,nottoafunction.Foracountedrange,thepartitionpointalgorithmlookslikethis:


template<ForwardIteratorI,PredicateP>

Ipartition_point_n(If,DifferenceTypen,Pp){

while(n){

Imiddle(f);

DifferenceTypehalf(n>>1);

advance(middle,half);

if(!p(*middle)){

n=half;

}else{

f=++middle;

n=n-(half+1);

}

}

returnf;

}

Thisisanextremelyimportantalgorithm,andit’sworthspendingsometimetomakesureyouunderstandit.Itusesabinary-search-stylestrategyjustliketheIntermediateValueTheorem.Recallthatthegoalistoreturnthefirst“bad”element—thatis,thefirstelementforwhichthepredicateisfalse.Theouterloopcontinuesuntilniszero.Insidetheloop,wepositiontheiteratormiddletoapointhalfwaybetweenfandf+n.Ifourpredicateisfalsefortheelementatthatposition,wesetntohalfofitspreviousvalueandrepeat.Otherwise,weknowthepredicateistrue,sowesetourstartingpointftothenextvalueafterthemiddle,adjustntoreflectthenumberofitemsleft,andrepeat.Whenwearedone,thereturnvalueisthepartitionpoint:thepointafterthelasttruevalue.Noticethatourfunctionusesadvancetomovetheiterator.Sincewedon’tknowtheiteratortype,we

can’tassumethatadditionisallowed.However,ifwehavearandomaccessiterator,theadvancefunctionwewroteearlierwillcallthefastimplementation.(Ifwedon’thavearandomaccessiterator,wemighthavetomakeasmanyasnmoves,butineithercasewe’llneverhavetodomorethanlogncomparisons.)Ifwearegivenaboundedrangeinstead,wesimplycomputethedistanceandinvokethecountedrange

version:Clickheretoviewcodeimage

template<ForwardIteratorI,PredicateP>

Ipartition_point(If,Il,Pp){

returnpartition_point_n(f,distance(f,l),p);

}

Nowlet’sreturntothegeneralbinarysearchproblem.Tosolveit,we’regoingtomakeuseofthefollowinglemma:

Lemma10.1(BinarySearchLemma):Foranysortedrange[i,j)andavaluea(theitemyou’researchingfor),therearetwoiterators,lowerboundblandupperboundbusuchthat

wherevkisthevalueatpositionk.Theseboundsalwaysexist,thoughinthespecialcasewherethereisnomatchingitem,bl=bu.Itmayhelptopicturethedataintherange:

Exercise10.1.ProvetheBinarySearchLemma.

Nowwecanuseourpartitionpointfunctiontoperformbinarysearch.Ifwehavetotalordering,anysortedrangeispartitionedforanyvalueaaccordingtothepredicatex<a.STLactuallyprovidesafewfunctionsthatperformbinarysearch,dependingonourtask.Ifwewanttofindthefirstpositionwheretheitemisfound,weusethelower_boundfunction.UsingthefeaturesinC++11,wecanwritelower_boundlikethis:Clickheretoviewcodeimage

template<ForwardIteratorI>

Ilower_bound(If,Il,ValueTypea){

returnpartition_point(f,l,

[=](ValueTypex){returnx<a;});

}

Thelastlinedefinesanewanonymousfunction(alsoknownasalambdaexpression16)thatreturnstrueifitsargumentislessthana,thenpassesthatfunctionasthepredicateusedbypartition_point.Thelower_boundfunctionreturnsthepositionoftheitema,orthepositionitwouldbeinifitwerethere.ValueTypeisjustashorthandtypefunctionforaccessingtheappropriateiteratortrait,liketheonewewroteearlierforDifferenceType:Clickheretoviewcodeimage


usingValueType=typenamestd::iterator_traits::value_type;

16SeeAppendixCfordetailsonhowtouselambdaexpressions.

Inthecasewherethereturnedpositionisnotl,westillneedtoknowwhetherwefoundtheitem.Todothis,thecallerneedstocheckwhetherthedereferencedreturnvalueisequaltoa.Ifinsteadwewanttofindthelastpositionwheretheitemisfound,weuseupper_bound.Thecodeis

almostthesame,exceptthatwedefinethepredicatetocheckifthevalueislessthanorequaltoa,rather

thanstrictlylessthana:Clickheretoviewcodeimage


Iupper_bound(If,Il,ValueTypea){

returnpartition_point(f,l,

[=](ValueTypex){returnx<=a;});

}

Somereadersmaybewonderingwhichfunctionisthe“real”binarysearch.Theansweristhatitdependsonthetask.Ifyouwanttofindthepositionofthefirstmatchingelement,thenlower_boundisthe“real”binarysearch.Ifyouwanttofindthepositionofthelastmatchingelement,thenit’supper_bound.Ifyouwanttoknowtheentirerangeofmatchingelements,STLprovidesathirdfunction,equal_range.Andifallyoucareaboutiswhethertherewasamatch,thereisafunctionbinary_search—butkeepinmindthatallit’sdoingiscallinglower_boundandtestingwhetherthedereferencedreturnvalueisequaltotheitem.Theadditionalfunctionalityofequal_rangeclearlybenefitsfromtheSTLconventionofusing

semi-openranges.Evenwhentheelementisnotpresent,thefunctionreturnsanemptyrangeatthepositionwhereitcouldbeinserted.

10.9ThoughtsontheChapterWebeganthischapterbyseeinghowAristotle’slevelsofabstraction(individual,species,genus)correspondtotheprogrammingnotionsofinstance,type,andconcept.Itisthenotionofconceptthatallowsagenericprogramtoworkinavarietyofsettings.Oneofyourcentralgoalsasaprogrammershouldbetoidentifyexistingconceptsinyourapplication.

Youwilloftendevelopnewalgorithms,occasionallydevelopanewdatastructure,andonlyrarelydefineanewconcept.Inthatraresituation,alotofworkisneededtoensurethatitisatrueconceptandnotjustacollectionofunrelatedrequirements.TorestateOccam’sRazor,oneshouldnotintroducenewconceptswithoutnecessity.Wethenintroducedtheconceptofiterators,andsawtheroletheyplayinsomefundamentalalgorithms.

Byusingcompile-timetypedispatchondifferentkindsofiterators,wecanensurethatthemostefficientimplementationgetsexecutedinagivensituation.Finally,wesawtheimportanceofwritingnotonlycorrectcode,butalsocorrectinterfaces.An

incorrectinterfacecanseverelylimittheutilityofafunction;acorrectinterfaceallowsittobeusedinavarietyofsituationswithoutlossofefficiency.

11.PermutationAlgorithms

Analgorithmmustbeseentobebelieved.DonaldKnuth

Complexcomputerprogramsarebuiltupfromsmallerpiecesthatperformcommonlyusedfundamentaltasks.Inthepreviouschapter,welookedatsometasksinvolvingsearchingfordata.Inthischapter,we’lllookattasksthatinvolveshiftingdataintonewpositions,andshowhowtoimplementtheminagenericway.We’llseehowthesetasksalsoendupusingtwoideasdiscussedearlierinthebook:groupsfromabstractalgebraandthegreatestcommondivisor(GCD)fromnumbertheory.Thetaskswewillfocuson—rotateandreverse—allowustointroducealgorithmsthatdothesametask

differentlydependingontheconceptoftheiteratortowhichtheyapply.Inadditiontoillustratingsomegenericprogrammingtechniques,thesealgorithmsareofgreatpracticalimportance.TherotatealgorithminparticularisprobablythemostusedalgorithminsidetheimplementationofSTLcomponentsfromvectortostable_sort.

11.1PermutationsandTranspositionsOurexplorationoftheGCDalgorithmledustolearnaboutgroupsandotheralgebraicstructures.Usingthisknowledge,we’regoingtostartinvestigatingthemathematicaloperationspermutationandtransposition,whichplayanimportantroleinsomefundamentalalgorithms.

Definition11.1.Apermutationisafunctionfromasequenceofnobjectsontoitself.

Theformalnotationforpermutations1lookslikethis:1Thisisthesamenotationmathematiciansuseformatrices;hopefullythecontextwillmakeitclearwhichinterpretationisintended.

Thefirstrowrepresentstheindexes(positions)ofasequenceofobjects;mathematiciansstartnumberingfrom1.Thesecondrowrepresentswheretheitemsinthosepositionsendupafterapplyingthepermutation.Inthisexample,theitemthatusedtobeatposition1willendupinposition2,theitemthatwasinposition2willendupinposition4,andsoon.Inpractice,permutationsareusuallywrittenusingashorthandformatthatomitsthefirstrow:

(2413)

Inotherwords,atpositioni,youwritewheretheithoriginalelementwillendup.Anexampleofapplyingapermutationis

(2413):{a,b,c,d}={c,a,d,b}

Wecanusethenotionofpermutationtodefineasymmetricgroup:

Definition11.2.ThesetofallpermutationsonnelementsconstitutesagroupcalledthesymmetricgroupSn.

Asymmetricgrouphasthefollowinggroupproperties:

binaryoperation:composition(associative)inverseoperation:inversepermutationidentityelement:identitypermutation

Thisisthefirstexamplewe’veseenwheretheelementsofagrouparethemselvesfunctions,andthegroupoperationisanoperationonfunctions.Ifwehaveapermutationxthatshiftsitemstwopositionstotheright,andanotherpermutationythatshiftsitemsthreepositionstotheright,thenthecompositionx yshiftsitemsfivepositionstotheright.Thisisperhapsthemostimportantgrouptoknowabout,sinceeveryfinitegroupisasubgroupofa

symmetricgroup.ItisknownasCayley’sTheorem.

Exercise11.1.ProveCayley’sTheorem.

Exercise11.2.WhatistheorderofSn?

Nowlet’slookataspecialcaseofpermutation,transposition.

Definition11.3.Atransposition(i,j)isapermutationthatexchangestheithandjthelements(i≠j),leavingtherestinplace.

Thenotationfortranspositionsindicateswhichtwopositionsshouldgetexchanged:

(23):{a,b,c,d}={a,c,b,d}

Inprogramming,wehaveasimplernamefortransposition:wecallitswap.InC++,wemightimplementitlikethis:

template<SemiregularT>

voidswap(T&x,T&y){

Ttmp(x);

x=y;

y=tmp;

}

TheswapoperationrequiresonlythatthetypesofitsargumentssatisfytheconceptSemiregular.2Wecanseethatswaprequirestheabilitytocopy-construct,assign,anddestructitsdata,sincethoseoperationsareusedinthecode.Itdoesnotneedtoexplicitlytestforequality,sowedonotneedthetypestobeRegular.Whenwedesignanalgorithm,we’llwanttoknowwhichconceptsthetypesneedtosatisfy,butwe’llalsowanttomakesurenottoimposeextrarequirementswedon’tneed.

2AdiscussionofC++movesemanticsisbeyondthescopeofthisbook.

***

Thetranspositionlemmademonstrateshowfundamentaltheswapoperationis:

Lemma11.1(TranspositionLemma):Anypermutationisaproductoftranspositions.

Proof.Onetranspositioncanputatleastoneelementintoitsfinaldestination.Therefore,atmostn–1

transpositionswillputallnelementsintotheirfinaldestinations.

Whydoweneedonlyn–1transpositions?Becauseoncen–1itemsareintherightplace,thenthitemmustalsobeintherightplace;there’snoplaceelseforittogo.

Exercise11.3.Provethatifn>2,Snisnotabelian.

Everypermutationdefinesadirectedgraphofnelements.Afterapplyingthepermutationenoughtimes,agivenelementwilleventuallybeputbackinitsoriginalposition,representingacycleinthegraph.Everypermutationcanbedecomposedintocycles.Forexample,considerthepermutation(235614).Theelementinposition4movestoposition6,andtheelementinposition6movestoposition4,soafterapplyingthepermutationtwice,bothofthoseelementswillendupwheretheystarted.Weseeasimilarpatternfortheelementsatpositions1,2,3,and5,althoughthistimeittakesfouroperationsbeforetheygetbacktothebeginning.Wesaythatthepermutation(235614)canbedecomposedintotwocycles,andweshowthemgraphicallylikethis:

Wewritethisdecompositionas(235614)=(1235)(46).Thecyclenotation,usedontheright,maybethoughtofasanextensionofthetranspositionnotation.

Althoughthenotationisambiguous(isthisacycleorapermutation?),itisusuallyclearfromthecontext.Also,permutationsalwayscontainalltheintegersfrom1ton,whilecyclesmightnot.Cyclesaredisjoint.Ifyouareatapositioninacycle,youcangettoallotherpositionsinthatcycle.

Therefore,iftwocyclesshareoneposition,theyshareallpositions;thatis,theyarethesamecycle.Sotheonlywaytohaveseparatecyclesisiftheydon’tshareanypositions.

Definition11.4.Acyclecontainingoneelementiscalledatrivialcycle.

Exercise11.4.Howmanynontrivialcyclescouldapermutationofnelementscontain?

Theorem11.1(NumberofAssignments):Thenumberofassignmentsneededtoperformanarbitrarypermutationinplaceisn–u+v,wherenisthenumberofelements,uisthenumberoftrivialcycles,andvisthenumberofnontrivialcycles.

Proof.Everynontrivialcycleoflengthkrequiresk+1assignments,sinceeachelementneedstobemoved,plusweneedtosavethefirstvaluebeingoverwritten;sinceeverynontrivialcyclerequiresoneextramove,movingallvcyclesrequiresvextramoves.Elementsintrivialcyclesdon’tneedtobemovedatall,andthereareuofthose.Soweneedtomoven–uelements,plusvadditionalmovesforthecycles.

Acommonpermutationthathasexactlyn/2cyclesisreverse.Insomesensethisisthe“hardest”permutationbecauseitrequiresthelargestnumberofassignments.We’lllookatreverseingreaterdetailinthenextchapter.

Exercise11.5.Designanin-place3reversealgorithmforforwarditerators;thatis,thealgorithmshouldworkforsinglylinkedlistswithoutmodifyingthelinks.

3We’lldiscussthenotionofin-placealgorithmsmoreinSection11.6;seeDefinition11.6.

11.2SwappingRangesSometimeswewanttoswapmorethanoneitematatime.Infact,acommonoperationinprogrammingistoswapallthevaluesinonerangeofdatawiththecorrespondingvaluesinanother(possiblyoverlapping)range.Wedoitinaloop,oneswapatatime:Clickheretoviewcodeimage

while(condition)std::swap(*iter0++,*iter1++);

whereiter0anditer1areiteratorspointingtotherespectivevaluesineachrange.RecallfromChapter10thatweprefertousesemi-openranges—thosewheretherangeincludesallelementsfromthefirstbounduptobutnotincludingthesecondbound.Whenweswaptworanges,oftenonlyoneofthemneedstobespecifiedexplicitly.Here,first0and

last0specifytheboundsofthefirstrange,whilefirst1referstothestartofthesecondrange:Clickheretoviewcodeimage

template<ForwardIteratorI0,ForwardIteratorI1>

//ValueType<I0>==ValueType<I1>

I1swap_ranges(I0first0,I0last0,I1first1){

while(first0!=last0)swap(*first0++,*first1++);

returnfirst1;

}

There’snopointinspecifyingtheendofthesecondrange,sincefortheswaptowork,itmustcontain(atleast)thesamenumberofelementsasthefirstrange.Whydowereturnfirst1?Becauseitmightbeusefultothecaller.Forexample,ifthesecondrangeislonger,wemightwanttoknowwheretheunmodifiedpartofthesecondrangebegins.It’sinformationthatthecallerdoesn’thave,anditcostsusalmostnothingtoreturn.Thisisagoodtimetoreviewthelawofusefulreturn,whichweintroducedinSection4.6.

TheLawofUsefulReturn,RevisitedWhenwritingcode,it’softenthecasethatyouendupcomputingavaluethatthecallingfunctiondoesn’tcurrentlyneed.Later,however,thisvaluemaybeimportantwhenthecodeiscalledinadifferentsituation.Inthissituation,youshouldobeythelawofusefulreturn:Aprocedureshouldreturnallthepotentiallyusefulinformationitcomputed.

Thequotient-remainderfunctionthatwesawinChapter4isagoodexample:whenwefirstwroteit,weneededonlytheremainder,butwehadalreadydonealltheworktofindthequotient.Later,wesawotherapplicationsthatusebothvalues.Thelawdoesnotimplydoingunneededextracomputations,nordoesitimplythatuselessinformationshouldbereturned.Forexample,inthecodegivenearlier,it’snotusefultoreturnfirst0,becausethealgorithmguaranteesthatattheendit’sequaltolast0,whichthecalleralreadyhas.It’suselesstogivemeinformationIalreadyhave.

TheLawofSeparatingTypesTheswap_rangescodeillustratesanotherimportantprogrammingprinciple,thelawofseparatingtypes:Donotassumethattwotypesarethesamewhentheymaybedifferent.

Ourfunctionwasdeclaredwithtwoiteratortypes,likethis:Clickheretoviewcodeimage


//ValueType<I0>==ValueType<I1>

I1swap_ranges(I0first0,I0last0,I1first1);

ratherthanassumingthatthey’rethesametype,likethis:Clickheretoviewcodeimage


Iswap_ranges(Ifirst0,Ilast0,Ifirst1);

Thefirstwaygivesusmoregeneralityandallowsustousethefunctioninsituationswewouldn’totherwisebeableto,withoutincurringanyadditionalcomputationcost.Forexample,wecouldswaparangeofelementsinalinkedlistwitharangeofelementsinanarrayanditwouldjustwork.However,justbecausetwotypesaredistinctdoesnotmeanthereisnorelationshipbetween

them.Inthecaseofswap_ranges,fortheimplementationtobeabletocallswaponthedata,weneedtoensurethatthevaluetypeofI0isthesameasthevaluetypeofI1.Whilethecompilercannotcheckthisconditiontoday,wecanindicateitwithacommentinthecode.

Insituationswherewe’renotsureifthesecondrangeislongenoughtoperformalltheneededswaps,wecanmakebothrangesexplicitandhaveourwhiletestchecktomakesureneitheriteratorhasrunofftheendoftherange:Clickheretoviewcodeimage


std::pair<I0,I1>swap_ranges(I0first0,I0last0,

I1first1,I1last1){

while(first0!=last0&&first1!=last1){

swap(*first0++,*first1++);

}

return{first0,first1};

}

Thistimewedoreturnbothfirst0andfirst1,becauseonerangemaybeexhaustedbeforetheother,sowewon’tnecessarilyhavereachedlast0andlast1.Swappingcountedrangesisalmostthesame,exceptthatinsteadofcheckingtoseeifwereachedthe

endoftherange,wecountdownfromnto0:Clickheretoviewcodeimage

template<ForwardIteratorI0,ForwardIteratorI1,IntegerN>

std::pair<I0,I1>swap_ranges_n(I0first0,I1first1,Nn){

while(n!=N(0)){

swap(*first0++,*first1++);

--n;

}

return{first0,first1};

}

TheLawofCompletenessObservethatwecreatedbothswap_rangesandswap_ranges_n.Eventhoughinaparticularsituationtheprogrammermightneedonlyoneoftheseversions,lateronaclientmightneedtheotherversion.Thisfollowsthelawofcompleteness:Whendesigninganinterface,considerprovidingalltherelatedprocedures.

Iftherearedifferentwaysofinvokinganalgorithm,provideinterfacesforthoserelatedfunctions.Inourswapexample,we’vealreadyprovidedtworelatedinterfacesforboundedranges,andwe’llalsoprovideinterfacesforcountedranges.Thisruleisnotsayingthatyouneedtohaveasingleinterfacetohandlemultiplecases.It’s

perfectlyfinetohaveonefunctionforcountedrangesandanotherforboundedranges.Youespeciallyshouldnothaveasingleinterfacefordisparateoperations.Forexample,justbecausecontainersneedtoprovideboth“insertelement”and“eraseelement”functionalitydoesn’tmeanyourinterfaceshouldhaveasingle“insert_or_erase”function.

Countedrangesareeasierforthecompiler,becauseitknowsthenumberofiterations,calledthetripcount,inadvance.Thisallowsthecompilertomakecertainoptimizations,suchasloopunrollingorsoftwarepipelining.

Exercise11.6.Whydon’tweprovidethefollowinginterface?


pair<I0,I1>swap_ranges_n(I0first0,I1first1,N0n0,N1n1)

11.3RotationOneofthemostimportantalgorithmsyou’veprobablyneverheardofisrotate.It’safundamentaltoolusedbehindthescenesinmanycommoncomputingapplications,suchasbuffermanipulationintexteditors.Itimplementsthemathematicaloperationrotation.

Definition11.5.Apermutationofnelementsbykwherek≥0:

(kmodn,k+1modn,...,k+n–2modn,k+n–1modn)

iscalledannbykrotation.Ifyouimagineallnelementslaidoutinacircle,we’reshiftingeachone“clockwise”byk.Atfirstglance,itmightseemthatrotationcouldbeimplementedwithamodularshift,takingthe

beginningandendoftherange,togetherwiththeamounttoshift,asarguments.However,doingmodulararithmeticoneveryoperationisquiteexpensive.Also,itturnsoutthatrotationisequivalenttoexchangingdifferentlengthblocks,ataskthatisextremelyusefulformanyapplications.Viewedthisway,itisconvenienttopresentrotationwiththreeiterators:f,m,andl,where[f,m)and[m,l)arevalidranges.4Rotationtheninterchangesranges[f,m)and[m,l).Iftheclientwantstorotatekpositionsinarange[f,l),thenitshouldpassavalueofmequaltol–k.Asanexample,ifwewanttodoarotationwithk=5onasequencespecifiedbytherange[0,7),wechoosem=l–k=7–5=2:

4Thenamesf,m,andlaremeanttobemnemonicsforfirst,middle,andlast.

0123456

fml

whichproducesthisresult:2345601

Essentially,we’removingeachitem5spacestotheright(andwrappingaroundwhenwerunofftheend).

Exercise11.7.Provethatifwedorotate(f,m,l),itperformsdistance(f,l)bydistance(m,l)rotation.

AnimportantrotatealgorithmwasdevelopedbyDavidGries,aprofessoratCornellUniversity,togetherwithIBMscientistHarlanMills:Clickheretoviewcodeimage


voidgries_mills_rotate(If,Im,Il){

//u=distance(f,m)&&v=distance(m,l)

if(f==m||m==l)return;//u==0||v==0

pair<I,I>p=swap_ranges(f,m,m,l);

while(p.first!=m||p.second!=l){

if(p.first==m){//u<v

f=m;m=p.second;//v=v-u

}else{//v<u

f=p.first;//u=u-v

}

p=swap_ranges(f,m,m,l);

}

return;//u==v

}

Thealgorithmfirstusestheregularswap_rangesfunctiontoswapasmanyelementsaswecan—asmanyelementsasthereareintheshorterrange.Theiftestcheckswhetherwe’veexhaustedthefirstrangeorthesecondrange.Dependingontheresult,weresetthestartpositionsoffandm.Thenwedoanotherswap,andrepeattheprocessuntilneitherrangehasremainingelements.Thisiseasiertofollowwithanexample.Let’slookathowarangeistransformed,andhowthe

iteratorsmove,asthealgorithmruns:Start:

0123456

fml

Swap[0,1]and[2,3].Haveweexhaustedbothranges?No,onlythefirstone,sowesetf=mandm=p.second,whichpointstothefirstelementinthesequencethathasn’tyetbeenmoved:

2301456

fml

Swap[0,1]and[4,5].Haveweexhaustedbothranges?No,onlythefirstone,soagainwesetf=mandm=p.second:

2345016

fml

Swap[0]with[6].Haveweexhaustedbothranges?No,onlythesecondonethistime,sowesetf=p.first:

2345610

fml

Swap[1]with[0].Haveweexhaustedbothranges?Yes,sowe’redone.2345601

fm

l

Nowlookatthecommentsinthegries_mills_rotatecode(showninboldface).Wecalluthelengthofthefirstrange[f,m),andvthelengthofthesecondrange[m,l).Wecanobservesomethingremarkable:theannotationsareperformingourfamiliarsubtractiveGCD!Attheendofthealgorithm,u=v=GCDofthelengthsoftheinitialtworanges.

Exercise11.8.Ifyouexamineswap_ranges,youwillseethatthealgorithmdoesunnecessaryiteratorcomparisons.Rewritethealgorithmsothatnounnecessaryiteratorcomparisonsaredone.

Itturnsoutthatmanyapplicationsbenefitiftherotatealgorithmreturnsanewmiddle:apositionwherethefirstelementmoved.Ifrotatereturnsthisnewmiddle,thenrotate(f,rotate(f,m,l),l)isanidentitypermutation.First,weneedthefollowing“auxiliaryrotate”algorithm:Clickheretoviewcodeimage


voidrotate_unguarded(If,Im,Il){

//assert(f!=m&&m!=l)



f=p.first;

if(m==f)m=p.second;


}

}

ThecentralloopisthesameasintheGries-Millsalgorithm,justwrittendifferently.(Wecouldhavewrittenitthiswaybeforebutwantedtomaketheuandvcomputationsclearer.)Weneedtofindm′—theelementwhosedistancetothelastisthesameasm’sdistancefromthefirst.

It’sthevaluereturnedbythefirstcalltoswap_ranges.Togetitback,wecanembedacalltorotate_unguardedinourfinalversionofrotate,whichworksaslongaswehaveforwarditerators.Aswe’llexplainshortly,theforward_iterator_tagtypeintheargumentlistwillhelpusinvokethisfunctiononlyinthiscase:Clickheretoviewcodeimage


Irotate(If,Im,Il,std::forward_iterator_tag){

if(f==m)returnl;

if(m==l)returnf;



if(p.second==l){

rotate_unguarded(p.first,m,l);

returnp.first;

}

f=m;

m=p.second;


}

returnm;

}

Howmuchworkdoesthisalgorithmdo?Untilthelastiterationofthemainloop,everyswapputsoneelementintherightplaceandmovesanotherelementoutoftheway.Butinthefinalcalltoswap_ranges,thetworangesarethesamelength,soeveryswapputsbothelementsitisswappingintotheirfinalpositions.Inessence,wegetanextramoveforfreeoneveryswap.Thetotalnumberofswapsisthetotalnumberofelementsn,minusthenumberoffreeswapswesavedinthelaststep.Howmanyswapsdidwesaveinthelaststep?Thelengthoftherangesattheend,aswesawearlier,isgcd(n–k,k)=gcd(n,k),wheren=distance(f,l)andk=distance(m,l).Sothetotalnumberofswapsisn–gcd(n,k).Also,sinceeachswaptakesthreeassignments(tmp=a;a=b;b=tmp),thetotalnumberofassignmentsis3(n–gcd(n,k)).

11.4UsingCyclesCanwefindafasterrotatealgorithm?Wecanifweexploitthefactthatrotations,likeanypermutations,havecycles.Considerarotationofk=2forn=6elements:

Theiteminposition0endsupinposition2,2endsupinposition4,and4endsupbackinposition0.Thesethreeelementsformacycle.Similarly,item1endsupin3,3in5,and5backin1,forminganothercycle.Sothisrotationhastwocycles.RecallfromSection11.1thatwecanperformanypermutationinn–u+vassignments,wherenisthenumberofelements,uisthenumberoftrivialcycles,andvisthenumberofnontrivialcycles.Sincewenormallydon’thaveanytrivialcycles,weneedn+vassignments.

Exercise11.9.Provethatifarotationofnelementshasatrivialcycle,thenithasntrivialcycles.(Inotherwords,arotationeithermovesallelementsornoelements.)

Itturnsoutthatthenumberofcyclesisgcd(k,n),soweshouldbeabletodotherotationinn+gcd(k,n)assignments,5insteadofthe3(n–gcd(n,k))weneededfortheGries-Millsalgorithm.Furthermore,inpracticeGCDisverysmall;infact,itis1(thatis,thereisonlyonecycle)about60%ofthetime.Soarotatealgorithmthatexploitscyclesalwaysdoesfewerassignments.

5SeeElementsofProgramming,Section10.4,fortheproof.

Thereisonecatch:Gries-Millsonlyrequiredmovingonestepforward;itworksevenforsinglylinkedlists.Butifwewanttotakeadvantageofcycles,weneedtobeabletodolongjumps.Suchanalgorithmrequiresstrongerrequirementsontheiterators—namely,theabilitytodorandomaccess.Tocreateournewrotatefunction,we’llfirstwriteahelperfunctionthatmoveseveryelementina

cycletoitsnextposition.Butinsteadofsaying“whichpositiondoestheiteminpositionxmoveto,”we’llsay“inwhichpositiondowefindtheitemthat’sgoingtomovetopositionx.”Eventhoughthesetwooperationsaresymmetricmathematically,itturnsoutthatthelatterismoreefficient,sinceitneedsonlyonesavedtemporaryvariablepercycle,insteadofoneforeveryitemthatneedstobemoved(exceptthelast).Here’sourhelperfunction:


template<ForwardIteratorI,TransformationF>

voidrotate_cycle_from(Ii,Ffrom){

ValueTypetmp=*i;

Istart=i;

for(Ij=from(i);j!=start;j=from(j)){

*i=*j;

i=j;

}

*i=tmp;

}

Notethatwe’reusingtheValueTypetypefunctionwedefinedneartheendofSection10.8.Howdoesrotate_cycle_fromknowwhichpositionanitemcomesfrom?Thatinformationwill

beencapsulatedinafunctionobjectfromthatwepassinasanargument.Youcanthinkoffrom(i)as“computewheretheelementmovingintopositionicomesfrom.”Thefunctionobjectwe’regoingtopasstorotate_cycle_fromwillbeaninstanceof

rotate_transform:Clickheretoviewcodeimage


structrotate_transform{

DifferenceTypeplus;

DifferenceTypeminus;

Im1;

rotate_transform(If,Im,Il):

plus(m-f),minus(m-l),m1(f+(l-m)){}

//m1separatesitemsmovingforwardandbackward

Ioperator()(Ii)const{

returni+((i<m1)?plus:minus);

}

};

Theideaisthateventhoughweareconceptually“rotating”elements,inpracticesomeitemsmoveforwardandsomemovebackward(becausetherotationcausedthemtowraparoundtheendofourrange).Whenrotate_transformisinstantiatedforagivensetofranges,itprecomputes(1)howmuchtomoveforwardforitemsthatshouldmoveforward,(2)howmuchtomovebackwardforthingsthatmovebackward,and(3)whatthecrossoverpointisfordecidingwhentomoveforwardandwhentomovebackward.Nowwecanwritethecycle-exploitingversionofalgorithmforrotation,whichisavariationofthe

algorithmdiscoveredbyFletcherandSilverin1965:Clickheretoviewcodeimage


Irotate(If,Im,Il,std::random_access_iterator_tag){

if(f==m)returnl;

if(m==l)returnf;

DifferenceTypecycles=gcd(m-f,l-m);

rotate_transformrotator(f,m,l);

while(cycles-->0)rotate_cycle_from(f+cycles,rotator);

returnrotator.m1;

}

Afterhandlingsometrivialboundarycases,thealgorithmfirstcomputesthenumberofcycles(theGCD)andconstructsarotate_transformobject.Thenitcallsrotate_cycle_fromtoshiftalltheelementsalongeachcycle,andrepeatsthisforeverycycle.Let’slookatanexample.Considertherotationk=2forn=6elementsthatweusedatthebeginningof

thissection.Forsimplicity,we’llassumethatourvaluesareintegersstoredinanarray:012345

Wealsoassumeouriteratorsareintegeroffsetsinanarray,startingat0.(Becarefultodistinguishbetweenthevaluesatapositionandthepositionitself.)Toperformak=2rotation,we’llneedtopassthethreeiteratorsf=0,m=4,andl=6:

012345

fml

Theboundarycasesofournewrotatealgorithmdon’tapply,sothefirstthingitdoesiscomputethenumberofcycles,whichisequaltogcd(m–f,l–m)=gcd(4,2)=2.Thenitconstructstherotatorobject,initializingitsstatevariablesasfollows:

plus←m–f=4–0=4minus←m–l=4–6=–2m1←f+(l–m)=0+(6–4)=2

Themainloopofthefunctionrotatesallelementsofacycle,thenmovesontothenextcycle.Letusseewhathappenswhenrotate_cycle_fromiscalled.Initially,wepassf+d=0+2=2asthefirstargument.Soinsidethefunction,i=2.Wesavethevalue

atposition2,whichisalso2,toourtmpvariableandsetstarttoourstartingpositionof2.Nowwegothroughaloopaslongasanewvariable,j,isnotequaltostart.Eachtimethroughthe

loop,wearegoingtosetjbyusingtherotatorfunctionobjectthatwepassedinthroughthevariablefrom.Basically,allthatobjectdoesisaddthestoredvaluesplusorminustoitsargument,dependingonwhethertheargumentislessthanthestoredvaluem1.Forexample,ifwecallfrom(0),itwillreturn0+4,or4,since0islessthan2.Ifwecallfrom(4),itwillreturn4+(–2),or2,since4isnotlessthan2.

Here’showthevaluesinourarraychangeaswegothroughtheloopinrotate_cycle_from:

i←2,j←from(2)=0

012345

ji

*i←*j

010345

ji

i←j=0,j←from(0)=4

010345

ij

*i←*j

410345

ij

i←j=4,j←from(4)=2whichisstart,soloopends

410345

ji

*i←tmp

410325

ji

Thiscompletesthefirstcalltorotate_cycle_frominthewhileloopofourrotatefunction.

Exercise11.10.Continuetotracetheprecedingexampleuntiltherotatefunctionfinishes.

Noticethatthesignaturesofthisrotatefunctionandthepreviousonedifferbythetypeofthelastargument.Inthenextsection,we’llwritethewrapperthatletsthefastestimplementationforagivensituationbeautomaticallyinvoked.

WhenIsanAlgorithmFasterinPractice?Wehaveseenanexamplewhereonealgorithmdoesfewerassignmentsthananotheralgorithm.Doesthatmeanitwillrunfaster?Notnecessarily.Inpractice,theabilitytofitrelevantdataincachecanmakeadramaticdifferenceinthisspeed.Analgorithmthatinvolveslargejumpsinmemory—thatis,onethathaspoorlocalityofreference—mayendupbeingslowerthanonethatrequiresmoreassignmentsbuthasbetterlocalityofreference.

11.5ReverseAnotherfundamentalalgorithmisreverse,which(obviously)reversestheorderoftheelementsofasequence.Moreformally,reversepermutesak-elementlistsuchthatitem0anditemk−1areswapped,item1anditemk−2areswapped,andsoon.

Ifwehavereverse,wecanimplementrotateinjustthreelinesofcode:Clickheretoviewcodeimage

template<BidirectionalIteratorI>

voidthree_reverse_rotate(If,Im,Il){

reverse(f,m);

reverse(m,l);

reverse(f,l);

}

Forexample,supposewewanttoperformourk=2rotationonthesequence012345.Thealgorithmwouldperformthefollowingoperations:Clickheretoviewcodeimage

fml

start012345

reverse(f,m)321045

reverse(m,l)321054

reverse(f,l)450123

Exercise11.11.Howmanyassignmentsdoes3-reverserotateperform?

Thiselegantalgorithm,whoseinventorisunknown,worksforbidirectionaliterators.However,ithasoneproblem:itdoesn’treturnthenewmiddleposition.Tosolvethis,we’regoingtobreakthefinalreversecallintotwoparts.We’llneedanewfunctionthatreverseselementsuntiloneofthetwoiteratorsreachestheend:Clickheretoviewcodeimage


pair<I,I>reverse_until(If,Im,Il){

while(f!=m&&m!=l)swap(*f++,*--l);

return{f,l};

}

Attheendofthisfunction,theiteratorthatdidn’thittheendwillbepointingtothenewmiddle.Nowwecanwriteageneralrotatefunctionforbidirectionaliterators.Whenitgetstowhatwouldhave

beenthethirdreversecall,itdoesreverse_untilinstead,savesthenewmiddleposition,andthenfinishesreversingtherestoftherange:Clickheretoviewcodeimage


Irotate(If,Im,Il,bidirectional_iterator_tag){

reverse(f,m);

reverse(m,l);

pair<I,I>p=reverse_until(f,m,l);

reverse(p.first,p.second);

if(m==p.first)returnp.second;

returnp.first;

}

Wehaveseenthreedifferentimplementationsofrotate,eachoptimizedfordifferenttypesofiterators.However,we’dliketohidethiscomplexityfromtheprogrammerwho’sgoingtobeusingthesefunctions.So,justaswedidwiththedistancefunctionsinSection10.5,we’regoingtowriteasimplerversionthatworksforanytypeofiterator,andusecategorydispatchtoletthecompilerdecidewhichimplementationwillgetexecuted:Clickheretoviewcodeimage

Irotate(If,Im,Il){

returnrotate(f,m,l,IteratorCategory());

}

Theprogrammerjustneedstocallasinglerotatefunction;thecompilerwillextractthetypeoftheiteratorbeingusedandinvoketheappropriateimplementation.

***

We’vebeenusingareversefunction,buthowmightweimplementit?Forbidirectionaliterators,thecodeisfairlystraightforward;wehaveapointertothebeginningthatmovesforward,andapointertotheendthatmovesbackward,andwekeepswappingtheelementstheypointtountiltheyrunintoeachother:Clickheretoviewcodeimage


voidreverse(If,Il,std::bidirectional_iterator_tag){

while(f!=l&&f!=--l)std::swap(*f++,*l);

}

Exercise11.12.Explainwhyweneedtwotestsperiterationintheprecedingwhileloop.

Itmightappearthataccordingtothelawofusefulreturnweshouldreturnpair<I,I>(f,l).However,thereisnoevidencethatthisinformationisactuallyuseful;thereforethelawdoesnotapply.Ofcourse,ifwealreadyknewinadvancehowmanytimeswehadtoexecutetheloop(thetripcount),

wewouldn’tneedtwocomparisons.Ifwepassthetripcountntoourfunction,wecanimplementitwithonlyn/2tests:Clickheretoviewcodeimage

template<BidirectionalIteratorI,IntegerN>

voidreverse_n(If,Il,Nn){

n>>=1;

while(n-->N(0)){

swap(*f++,*--l);

}

}

Inparticular,ifwehavearandomaccessiterator,wecancomputethetripcountinconstanttime,andimplementreverseusingreverse_n,likethis:Clickheretoviewcodeimage


voidreverse(If,Il,std::random_access_iterator_tag){

reverse_n(f,l,l-f);

}

Whatifwehaveonlyforwarditeratorsandwestillwanttoreverse?We’llusearecursiveauxiliaryfunctionthatkeepspartitioningtherangeinhalf(thehinthecode).Theargumentnkeepstrackofthelengthofthesequencebeingreversed:Clickheretoviewcodeimage

template<ForwardIteratorI,IntegerN>

Ireverse_recursive(If,Nn){

if(n==0)returnf;

if(n==1)return++f;

Nh=n>>1;

Im=reverse_recursive(f,h);

if(odd(n))++m;

Il=reverse_recursive(m,h);

swap_ranges_n(f,m,h);

returnl;

}

Exercise11.13.Usingthesequence{0,1,2,3,4,5,6,7,8}asanexample,tracetheoperationofthereverse_recursivealgorithm.

Thefunctionreturnstheendoftherange,sothefirstrecursivecallreturnsthemidpoint.Thenweadvancethemidpointby1or0dependingonwhetherthelengthisevenorodd.Nowwecanwriteourreversefunctionforforwarditerators:



voidreverse(If,Il,std::forward_iterator_tag){

reverse_recursive(f,distance(f,l));

}

Finally,wecanwritethegenericversionofreversethatworksforanyiteratortype,justaswedidforrotateearlier:Clickheretoviewcodeimage


voidreverse(If,Il){

reverse(f,l,IteratorCategory());

}

TheLawofInterfaceRefinementWhatisthecorrectinterfaceforrotate?Originally,std::rotatereturnedvoid.Afterseveralyearsofusage,itbecameclearthatreturningthenewmiddle(thepositionwherethefirstelementmoved)madeimplementationofseveralotherSTLalgorithms,suchasin_place_mergeandstable_partition,simpler.Unfortunately,itwasnotimmediatelyobvioushowtoreturnthisvaluewithoutdoinganyextra

work.Onlyafterthisimplementationproblemwassolvedwasitpossibletoredesigntheinterfacetoreturntherequiredvalue.Itthentookmorethan10yearstochangetheC++languagestandard.Thisisagoodillustrationofthelawofinterfacerefinement:

Designinginterfaces,likedesigningprograms,isamulti-passactivity.

Wecan’treallydesignanidealinterfaceuntilwehaveseenhowthealgorithmwillbeused,andnotalltheusesareimmediatelyapparent.Furthermore,wecan’tdesignanidealinterfaceuntilweknowwhichimplementationsarefeasible.

11.6SpaceComplexityWhentalkingaboutconcretealgorithms,programmersneedtothinkaboutwheretheyfallintermsoftimeandspacecomplexity.Therearemanylevelsoftimecomplexity(e.g.,constant,logarithmic,quadratic).However,thetraditionalviewofspacecomplexityputalgorithmsintojusttwocategories:thosethatperformtheircomputationsinplaceandthosethatdonot.

Definition11.6.Analgorithmisin-place(alsocalledpolylogspace)ifforaninputoflengthnitusesO((logn)k)additionalspace,wherekisaconstant.

Initially,in-placealgorithmswereoftendefinedasusingconstantspace,butthiswastoonarrowarestriction.Theideaofbeing“inplace”wassupposedtocapturealgorithmsthatdidn’tneedtomakeacopyoftheirdata.Butmanyofthesenon-data-copyingalgorithms,likequicksort,useadivide-and-conquertechniquethatrequireslogarithmicextraspace.Sothedefinitionwasformalizedinawaythatincludedthesealgorithms.Algorithmsthatarenotin-placeusemorespace—usually,enoughtocreateacopyoftheirdata.

***

Let’suseourreverseproblemtoseehowanon-in-placealgorithmcanbefasterthananin-placealgorithm.First,weneedthishelperfunction,whichcopieselementsinreverseorder,startingattheendofarange:Clickheretoviewcodeimage

template<BidirectionalIteratorI,OutputIteratorO>

Oreverse_copy(If,Il,Oresult){

while(f!=l)*result++=*--l;

returnresult;

}

Nowwecanwriteanon-in-placereversealgorithm.Itcopiesallthedatatoabuffer,thencopiesitbackinreverseorder:Clickheretoviewcodeimage

template<ForwardIteratorI,IntegerN,BidirectionalIteratorB>

Ireverse_n_with_buffer(If,Nn,Bbuffer){

Bbuffer_end=copy_n(f,n,buffer);

returnreverse_copy(buffer,buffer_end,f);

}

Thisfunctiontakesonly2nassignments,insteadofthe3nweneededfortheswap-basedimplementations.

11.7Memory-AdaptiveAlgorithmsInpractice,thedichotomyofin-placeandnon-in-placealgorithmsisnotveryuseful.Whiletheassumptionofunlimitedmemoryisnotrealistic,neitheristheassumptionofonlypolylogextramemory.Usually25%,10%,5%,oratleast1%ofextramemoryisavailableandcanbeexploitedtogetsignificantperformancegains.Algorithmsneedtoadapttohowevermuchmemoryisavailable;theyneedtobememoryadaptive.Let’screateamemory-adaptivealgorithmforreverse.Ittakesabufferthatwecanuseas

temporaryspace,andanargumentbufsizeindicatinghowbigthebufferis.Thealgorithmisrecursive—infact,it’salmostidenticaltothereverse_recursivefunctionintheprevioussection.Buttherecursionhappensonlyonlargechunks,sotheoverheadisacceptable.Theideaisthatforagiveninvocationofthefunction,ifthelengthofthesequencebeingreversedfitsinthebuffer,wedothefastreversewithbuffer.Ifnot,werecurse,splittingthesequenceinhalf:Clickheretoviewcodeimage

template<ForwardIteratorI,IntegerN,BidirectionalIteratorB>

Ireverse_n_adaptive(If,Nn,Bbuffer,Nbufsize){

if(n==N(0))returnf;

if(n==N(1))return++f;

if(n<=bufsize)returnreverse_n_with_buffer(f,n,buffer);

Nh=n>>1;

Im=reverse_n_adaptive(f,h,buffer,bufsize);

advance(m,n&1);

Il=reverse_n_adaptive(m,h,buffer,bufsize);

swap_ranges_n(f,m,h);

returnl;

}

Thecallerofthisfunctionshouldaskthesystemhowmuchmemoryisavailable,andpassthatvalueasbufsize.Unfortunately,suchacallisnotprovidedinmostoperatingsystems.

ASadStoryaboutget_temporary_bufferWhentheC++STLlibrarywasdesignedbythefirstauthorofthisbook,herealizedthatitwouldbehelpfultohaveafunctionget_temporary_bufferthattakesasizenandreturnsthelargestavailabletemporarybufferuptosizenthatfitsintophysicalmemory.Asaplaceholder(sincethecorrectversionneedstohaveknowledgethatonlytheoperatingsystemhas),hewroteasimplisticandimpracticalimplementation,whichrepeatedlycallsmallocaskingforinitiallyhugeandgraduallysmallerchunksofmemoryuntilitreturnsavalidpointer.Heputinaprominentcommentinthecodesayingsomethinglike,“Thisisabogusimplementation,replaceit!”Tohissurprise,hediscoveredyearslaterthatallthemajorvendorsthatprovideSTLimplementationsarestillusingthisterribleimplementation—buttheyremovedhiscomment.

11.8ThoughtsontheChapterOneofthethingswe’veseeninboththischapterandthepreviousoneishowsimplecomputationaltasksofferrichopportunitiestoexploredifferentalgorithmsandlearnfromthem.Theprogrammingprinciplesthatarisefromtheseexamples—thelawsofusefulreturn,separatingtypes,completeness,andinterfacerefinement—carryoverintonearlyeveryprogrammingsituation.Thischapterhasalsopresentedsomegoodexamplesofhowtheoryandpracticecometogetherin

programming.Ourknowledgeofthetheoryofpermutations—itselfbasedongrouptheory—allowedustocomeupwithamoreefficientalgorithmforrotation,onethatexploitedthepropertiesthatourtheoryguaranteed.Atthesametime,theexampleofmemory-adaptivealgorithmsdemonstratedhowpracticalconsiderationssuchastheamountofavailablememorycanhaveaprofoundimpactonthechoiceofalgorithmanditsperformance.Theoryandpracticearetwosidesofthesamecoin;goodprogrammersrelyonknowledgeofboth.

12.ExtensionsofGCD

Iswearbytheevenandtheodd.Quran,SurahAl-Fajr

Programmersoftenassumethatsinceadatastructureoralgorithmisinatextbookorhasbeenusedforyears,itrepresentsthebestsolutiontoaproblem.Surprisingly,thisisoftennotthecase—evenifthealgorithmhasbeenusedforthousandsofyearsandhasbeenworkedonbyeveryonefromEuclidtoGauss.Inthischapterwe’lllookatanexampleofanovelsolutiontoanoldproblem—computingtheGCD.Thenwe’llseehowprovingatheoremfromnumbertheoryresultedinanimportantvariationofthealgorithmstillusedtoday.

12.1HardwareConstraintsandaMoreEfficientAlgorithmIn1961,anIsraeliPh.D.student,Josef“Yossi”Stein,wasworkingonsomethingcalledRacahAlgebraforhisdissertation.Heneededtodorationalarithmetic,whichrequiredreducingfractions,whichusestheGCD.Butbecausehehadonlylimitedtimeonaslowcomputer,hewasmotivatedtofindabetterway.Asheexplains:

Using“RacahAlgebra”meantdoingcalculationswithnumbersoftheforma/b. ,where,a,b,cwereintegers.IwroteaprogramfortheonlyavailablecomputerinIsraelatthattime—theWEIZACattheWeizmanninstitute.Additiontimewas57microseconds,divisiontookabout900microseconds.Shifttooklessthanaddition....Wehadneithercompilernorassembler,andnofloating-pointnumbers,butusedhexadecimalcodefortheprogramming,andhadonly2hoursofcomputer-timeperweekforRacahandhisstudents,andyouseethatIhadtherightconditionsforfindingthatalgorithm.FastGCDmeantsurvival.1

1J.Stein,personalcommunication,2003.

WhatSteinobservedwasthattherewerecertainsituationswheretheGCDcouldbeeasilycomputed,oreasilyexpressedintermsofanotherGCDexpression.HelookedatspecialcasesliketakingtheGCDofanevennumberandanoddnumber,oranumberanditself.Eventually,hecameupwiththefollowingexhaustivelistofcases:

Usingtheseobservations,Steinwrotethefollowingalgorithm:Clickheretoviewcodeimage

template<BinaryIntegerN>

Nstein_gcd(Nm,Nn){

if(m<N(0))m=-m;

if(n<N(0))n=-n;

if(m==N(0))returnn;

if(n==N(0))returnm;

//m>0&&n>0

intd_m=0;

while(even(m)){m>>=1;++d_m;}

intd_n=0;

while(even(n)){n>>=1;++d_n;}

//odd(m)&&odd(n)

while(m!=n){

if(n>m)swap(n,m);

m-=n;

dom>>=1;while(even(m));

}

//m==n

returnm<<min(d_m,d_n);

}

Let’slookatwhatthecodeisdoing.ThefunctiontakestwoBinaryIntegers—thatis,anintegerrepresentationthatsupportsfastshiftandeven/oddtesting,liketypicalcomputerintegers.First,iteliminatestheeasycaseswhereoneoftheargumentsiszero,andinvertsthesignifanargumentisnegative,sothatwearedealingwithtwopositiveintegers.Next,ittakesadvantageoftheidentitieswithevenarguments,removingfactorsof2(byshifting)while

keepingtrackofhowmanytherewere.Wecanuseasimpleintforthecounts,sincewhatwe’recountingisatmostthetotalnumberofbitsintheoriginalarguments.Afterthispart,weareoperatingontwooddnumbers.Nowcomesthemainloop.Werepeatedlysubtractthesmallerfromthelargereachtime(sincewe

knowthedifferenceoftwooddnumbersiseven),andagainuseshiftstoremoveadditionalpowersof2fromtheresult.2Whenwe’redone,ourtwonumberswillbeequal.Sincewe’rehalvingatleastonceeachtimethroughtheloop,weknowwe’lliteratenomorethanlogntimes;thealgorithmisboundedbythenumberof1-bitsweencounter.

2Weusedo-whileratherthanwhilebecausewedon’tneedtorunthethetestthefirsttime;weknowwe’restartingwithanevennumbersoweknowwehavetodoatleastoneshift.

Finally,wereturnourresult,usingashifttomultiplyournumberby2foreachoftheminimumnumberof2swefactoredoutatthebeginning.Wedon’tneedtoworryabout2sinthemainloop,becausebythatpointwe’vereducedtheproblemtotheGCDoftwooddnumbers;thisGCDdoesnothave2asafactor.Here’sanexampleofthealgorithminoperation.SupposewewanttocomputeGCD(196,42).The

computationlookslikethis:

Aswesaw,Steintooksomeobservationsaboutspecialcasesandturnedthemintoafasteralgorithm.Thespecialcaseshadtodowithevenandoddnumbers,andplaceswherewecouldfactorout2,whichiseasyonacomputer;that’swhyStein’salgorithmisfasterinpractice.(Eventoday,whentheremainderfunctioncanbecomputedinhardware,itisstillmuchslowerthansimpleshifts.)Butisthisjustacleverhack,oristheremoreherethanmeetstheeye?Doesitmakesenseonlybecausecomputersusebinaryarithmetic?DoesStein’salgorithmworkjustforintegers,orcanwegeneralizeitjustaswedidwithEuclid’salgorithm?

12.2GeneralizingStein’sAlgorithmToanswerthesequestions,let’sreviewsomeofthehistoricalmilestonesforEuclid’sGCD:

•Positiveintegers:Greeks(5thcenturyBC)•Polynomials:Stevin(ca.1600)•Gaussianintegers:Gauss(ca.1830)•Algebraicintegers:Dirichlet,Dedekind(ca.1860)•Genericversion:Noether,vanderWaerden(ca.1930)

Ittookmorethan2000yearstoextendEuclid’salgorithmfromintegerstopolynomials.Fortunately,ittookmuchlesstimeforStein’salgorithm.Infact,just2yearsafteritspublication,Knuthalreadyknewofaversionforsingle-variablepolynomialsoverafield .Thesurprisinginsightwasthatwecanhavexplaytheroleforpolynomialsthat2playsforintegers.

Thatis,wecanfactoroutpowersofx,andsoon.Carryingtheanalogyfurther,weseethatx2+x(oranythingelsedivisiblebyx)is“even,”x2+x+1(oranythingelsewithazero-ordercoefficient)is“odd,”andx2+x“shifts”tox+1.Justasdivisionby2iseasierthangeneraldivisionforbinaryintegers,sodivisionbyxiseasierthangeneraldivisionforpolynomials—inbothcases,allweneedisashift.(Rememberthatapolynomialisreallyasequenceofcoefficients,sodivisionbyxisliterallyashiftof

thesequence.)Stein’s“specialcases”forpolynomialslooklikethis:

Noticehoweachofthelasttworulescancelsoneofthezero-ordercoefficients,soweconvertthe“odd,odd”casetoan“even,odd”case.TogettheequivalenceexpressedbyEquation12.6,werelyontwofacts.First,ifyouhavetwo

polynomialsuandv,thengcd(u,v)=gcd(u,av),whereaisanonzerocoefficient.Sowecanmultiplythesecondargumentbythecoefficient ,andwe’llhavethesameGCD:

Second,gcd(u,v)=gcd(u,v-u),whichwenotedwhenweintroducedGCDearlyinthebook(Equation3.9).Sowecansubtractournewsecondargumentfromthefirst,andwe’llstillhavethesameGCD:

Finally,wecanusethefactthatifoneoftheGCDargumentsisdivisiblebyxandtheotherisnot,wecandropxbecausetheGCDwillnotcontainitasafactor.Sowe“shift”outthex,whichgives

whichiswhatwewanted.Wealsoseethatineachtransformation,thenorm—inthiscase,thedegreeofthepolynomial—gets

reduced.Here’showthealgorithmwouldcomputegcd(x3–3x–2,x2–4):

Firstweseethattheratiooftheirfreecoefficients(thecanddinEquations12.6and12.7)is1/2,sowewillmultiplynby1/2andsubtractitfromm(showninthefirstlineoftheprecedingtable),resultinginthenewmshownonthesecondline.Thenwe“shift”mbyfactoringoutx,resultinginthethirdline,and

soon.In2000,AndreWeilertfurthergeneralizedStein’salgorithmtoGaussianintegers.Thistime,1+iplays

theroleof2;the“shift”operationisdivisionby1+i.In2003,DamgårdandFrandsenextendedthealgorithmtoEisensteinintegers.In2004,AgarwalandFrandsendemonstratedthatthereisaringthatisnotaEuclideandomain,but

wheretheSteinalgorithmstillworks.Inotherwords,therearecaseswhereStein’salgorithmworksbutEuclid’sdoesnot.IfthedomainoftheSteinalgorithmisnottheEuclideandomain,thenwhatisit?Asofthiswriting,thisisanunsolvedproblem.WhatwedoknowisthatStein’salgorithmdependsonthenotionofevenandodd;wegeneralizeeven

tobedivisiblebyasmallestprime,wherepisasmallestprimeifanyremainderwhendividingbyitiseitherzerooraninvertibleelement.(Wesay“asmallestprime”ratherthan“thesmallestprime”becausetherecouldbemultiplesmallestprimesinaring.Forexample,forGaussianintegers,1+i,1–i,–1+i,and–1–iareallsmallestprimes.)Whydowefactorout2whenwe’recomputingtheGCDofintegers?Becausewhenwerepeatedly

divideby2,weeventuallyget1asaremainder;thatis,wehaveanoddnumber.Oncewehavetwooddnumbers(twonumberswhoseremaindersmodulo2arebothunits),wecanusesubtractiontokeepourGCDalgorithmgoing.Thisabilitytocancelremaindersworksbecause2isthesmallestintegerprime.Similarly,xisthesmallestprimeforpolynomials,andi+1forGaussianintegers.3Divisionbythesmallestprimealwaysgivesaremainderofzerooraunit,becauseaunitisthenumberwiththesmallestnonzeronorm.So2worksforintegersbecauseit’sthesmallestprime,notbecausecomputersusebinaryarithmetic.Thealgorithmispossiblebecauseoffundamentalpropertiesofintegers,notbecauseofthehardwareimplementation,althoughthealgorithmisefficientbecausecomputersusebinaryarithmetic,makingshiftsfast.

3Notethat2isnotprimeintheringofGaussianintegers,sinceitcanbefactoredinto(1+i)(1-i).

Exercise12.1.ComparetheperformanceoftheSteinandEuclidalgorithmsonrandomintegersfromtheranges[0,216),[0,232),and[0,264).

12.3Bézout’sIdentityTounderstandtherelationshipofGCDandringstructures,weneedtointroduceBézout’sidentity,whichalsoleadstoanimportantpracticalalgorithmforcomputingthemultiplicativeinverse.TheidentitysaysthatforanytwovaluesaandbintheEuclideandomain,thereexistcoefficientssuchthatthelinearcombinationgivestheGCDoftheoriginalvalues.

Theorem12.1(Bézout’sIdentity):

Forexample,ifa=196andb=42,thenthissaystherearevaluesxandysuchthat196x+42y=gcd(196,42).Sincegcd(196,42)=14,inthiscasex=–1andy=5.We’llseelaterinthechapterhowtocomputexandyingeneral.Likemanyresultsinmathematics,thisoneisnamedaftersomeoneotherthanitsdiscoverer.Although

18th-centuryFrenchmathematicianÉtienneBézoutdidprovetheresultforpolynomials,itwasactuallyshownfirstforintegersahundredyearsearlier,byClaudeBachet.

ClaudeGasparBachetdeMéziriac(1581–1638)

ClaudeGasparBachetdeMéziriac,generallyknownasBachet,wasaFrenchmathematicianduringtheRenaissance.Althoughhewasascholarinmanyfields,heisbestknownfortwothings.First,hetranslatedDiophantus’ArithmeticfromGreektoLatin,thecommonlanguageofscienceandphilosophyinEuropeatthetime.Itishis1621translationthatmostmathematiciansreliedon,anditwasinacopyofhistranslationthatFermatfamouslywrotethemarginalnotedescribinghislasttheorem.Second,Bachetwrotethefirstbookonrecreationalmathematics,ProblèmesPlaisants,originallypublishedin1612.Throughthisbook,mathematicsbecameapopulartopicamongeducatedpeopleinFrance,somethingtheywoulddiscussandspendtimeonasahobby.ProblèmesPlaisantsintroducedmagicsquares,aswellasprovingwhatisnowknownasBézout’sidentity.BachetwaschosenasoneoftheoriginalmembersofAcadémieFrançaise(FrenchAcademy),

theorganizationcreatedbyCardinalRichelieutobetheultimateauthorityontheFrenchlanguage,andtaskedwiththecreationandmaintenanceoftheofficialFrenchdictionary.

Recallthataringisanalgebraicstructurethatbehavessimilartointegers;ithasbothplus-likeandtimes-likeoperations,butonlyanadditiveinverse.(YoucanreviewthedefinitionofaringinDefinition8.3inSection8.4.)ToproveBézout’sidentity,weneedtoshowthatthecoefficientsxandyalwaysexist.Todothis,we

needtointroduceanewalgebraicstructure,theideal.

Definition12.1.AnidealIisanonemptysubsetofaringRsuchthat

1.∀x,y I:x+y I2.∀x I,∀a R:ax I

Thefirstpropertysaysthattheidealisclosedunderaddition;inotherwords,ifyouaddanytwoelementsoftheideal,theresultisintheideal.Thesecondpropertyisabitmoresubtle;itsaysthattheidealisclosedundermultiplicationwithanyelementofthering,notnecessarilyanelementoftheideal.Anexampleofanidealisthesetofevennumbers,whichareanonemptysubsetoftheringofintegers.

Ifyouaddtwoevennumbers,yougetanevennumber.Ifyoumultiplyanevennumberbyanynumber(not

necessarilyeven),youstillgetanevennumber.Otherexamplesofidealsareunivariatepolynomialswithroot5,andpolynomialswithxandyandfreecoefficient0(e.g.,x2+3y2+xy+x);we’llseeshortlywhythislastcaseisimportant.Notethatjustbecausesomethingisasubringdoesn’tmeanit’sanideal.IntegersareasubringofGaussianintegers,buttheyaren’tanidealofGaussianintegers,becausemultiplyinganintegerbytheimaginarynumberidoesnotproduceaninteger.

Exercise12.2.

1.ProvethatanidealIisclosedundersubtraction.2.ProvethatIcontains0.

Lemma12.1(LinearCombinationIdeal):Inaring,foranytwoelementsaandb,thesetofallelements{xa+yb}formsanideal.

Proof.First,thissetisclosedunderaddition:

(x1a+y1b)+(x2a+y2b)=(x1+x2)a+(y1+y2)b

Next,itisclosedundermultiplicationbyanarbitraryelement:

z(xa+yb)=(zx)a+(zy)b

Therefore,itisanideal.

Exercise12.3.Provethatalltheelementsofalinearcombinationidealaredivisiblebyanyofthecommondivisorsofaandb.

Lemma12.2(IdealsinEuclideanDomains):AnyidealinaEuclideandomainisclosedundertheremainderoperationandunderEuclideanGCD.

Proof.

1.Closedunderremainder:Bydefinition,

remainder(a,b)=a–quotient(a,b)·b

Ifbisintheideal,thenbythesecondaxiomofideals,anythingmultipliedbybisintheideal,soquotient(a,b)·bisintheideal.ByExercise12.2,thedifferenceoftwoelementsofanidealisintheideal.

2.ClosedunderGCD:SincetheGCDalgorithmconsistsofrepeatedlyapplyingremainder,thisimmediatelyfollowsfrom1.

Definition12.2.AnidealIoftheringRiscalledaprincipalidealifthereisanelementa RcalledtheprincipalelementofIsuchthat

x I ∃y R:x=ay

Inotherwords,aprincipalidealisanidealthatcanbegeneratedfromoneelement.Anexampleofaprincipalidealisthesetofevennumbers(2istheprincipalelement).Polynomialswithroot5areanotherprincipalideal.Incontrast,polynomialswithxandyandfreecoefficient0areideals,butnotprincipalideals.Rememberthepolynomialx2+3y2+xy+x,whichwegaveasanexampleofanideal?There’snowaytogenerateitstartingwithjustx(itwouldnevercontainy),andviceversa.

Exercise12.4.Provethatanyelementinaprincipalidealisdivisiblebytheprincipalelement.

Recallthatanintegraldomainisaringwithnozerodivisors(Definition8.7).

Definition12.3.Anintegraldomainiscalledaprincipalidealdomain(PID)ifeveryidealinitisaprincipalideal.

Forexample,theringofintegersisaPID,whiletheringofmultivariatepolynomialsoverintegersisnot.

Theorem12.2:ED PID.EveryEuclideandomainisaprincipalidealdomain.

Proof.AnyidealIinaEuclideandomaincontainsanelementmwithaminimalpositivenorm(a“smallestnonzeroelement”).Consideranarbitraryelementa I;eitheritisamultipleofmorithasaremainderr:

Butwechosemasthesmallestelement,sowecannothaveasmallerremainder—thatwouldbeacontraction.Thereforeourelementacan’thavearemainder;a=qm.Sowecanobtaineveryelementfromoneelement,whichisthedefinitionofaPID.

NowwecanproveBézout’sidentity.Sinceitsaysthatthereisatleastonevalueofxandonevalueofythatsatisfytheequationxa+yb=gcd(a,b),wecanrestateitassayingthatthesetofallpossiblelinearcombinationsxa+ybcontainsthedesiredvalue.

Bézout’sIdentity,Restated:AlinearcombinationidealI={xa+yb}ofaEuclideandomaincontainsgcd(a,b).

Proof.ConsiderthelinearcombinationidealI={xa+yb}.aisinIbecausea=1a+0b.Similarly,bisinIbecauseb=0a+1b.ByLemma12.2,anyidealinaEuclideandomainisclosedundertheGCD;thusgcd(a,b)isinI.

WecanalsouseBézout’sidentitytoprovetheInvertibilityLemma,whichweencounteredinChapter5:

Lemma5.4(InvertibilityLemma):

Proof.ByBézout’sidentity,

∃x,y :xa+yn=gcd(a,n)

Soifgcd(a,n)=1,thenxa+yn=1.Therefore,xa=–yn+1,and

xa=1modn

Exercise12.5.UsingBézout’sidentity,provethatifpisprime,thenany0<a<phasamultiplicativeinversemodulop.

12.4ExtendedGCDTheproofofBézout’sidentitythatwesawinthelastsectionisinteresting.Itshowswhytheresultmustbetrue,butitdoesn’tactuallytellushowtofindthecoefficients.Thisisanexampleofanonconstructiveproof.Foralongtime,therewasadebateinmathematicsaboutwhethernonconstructiveproofswereasvalidasconstructiveproofs.Thosewhoopposedtheuseofnonconstructiveproofswereknownasconstructivistsorintuitionists.Attheturnofthe20thcentury,thetideturnedagainsttheconstructivists,withDavidHilbertandtheGöttingenschoolleadingthecharge.Thelonemajordefenderofconstructivism,HenriPoincaré,lostthebattle,andtodaynonconstructiveproofsareroutinelyused.

HenriPoincaré(1854–1912)

JulesHenriPoincaréwasaFrenchmathematicianandphysicist.Hecamefromadeeplypatrioticfamily,andhiscousinRaymondwasaprimeministerofFrance.Hepublishedmorethan500papersonavarietyofsubjects,includingseveralonspecialrelativitydevelopedindependentlyof,andinsomecasespriorto,Einstein’sworkonthesubject.Althoughheworkedonmanypracticalproblems,suchasestablishingtimezones,hemostlyvaluedscienceasatoolforunderstandingtheuniverse.Ashewrote:Thescientistmustnotdallyinrealizingpracticalaims.Henodoubtwillobtainthem,butmustobtaintheminaddition.Henevermustforgetthatthespecialobjectheisstudyingisonlyapartofthisbigwhole,whichmustbethesolemotiveofhisactivity.Sciencehashadmarvelousapplications,butasciencethatwouldonlyhaveapplicationsinmindwouldnotbescienceanymore,itwouldbeonlycookery.

Poincaréalsowroteseveralimportantbooksaboutthephilosophyofscience,andwaselectedamemberoftheAcadémieFrançaise.Poincarécontributedtoalmosteverybranchofmathematics,originatingseveralsubfields,such

asalgebraictopology.PeopleatthetimedebatedaboutwhetherPoincaréorHilbertwasthegreatestmathematicianintheworld.However,Poincaré’scriticismofsettheoryandtheformalistagendaofHilbertputhimonthewrongsideoftherivalrybetweenFranceandarecentlyunifiedGermany.TherejectionofPoincaré’sintuitionistapproachbytheformalistswasagreatlossfor20th-centurymathematics.Bothpointsofviewcomplementeachother.

Whatevertheviewofformalistmathematicians,fromaprogrammingperspective,itisclearlymoresatisfactorytoactuallyhaveanalgorithmthansimplytoknowthatoneexists.Sointhissection,wewillderiveaconstructiveproofofBézout’sidentity—inotherwords,analgorithmforfindingxandysuchthatxa+yb=gcd(a,b).Tounderstandtheprocess,it’shelpfultoreviewwhathappensinseveraliterationsofEuclid’s

algorithmtocomputetheGCDofaandb.Clickheretoviewcodeimage


Egcd(Ea,Eb){

while(b!=E(0)){

a=remainder(a,b);

std::swap(a,b);

}

returna;

}

Eachtimethroughthemainloop,wereplaceabytheremainderofaandb,thenswapaandb;ourfinalremainderwillbetheGCD.Sowearecomputingthissequenceofremainders:

Notehowthesecondargumenttotheremainderfunctiononiterationkshiftsovertobecomethefirstargumentoniterationk+1.Sincetheremainderofaandbiswhat’sleftoverfromaafterdividingabyb,wecanwritethe

sequencelikethis:

wheretheqtermsarethecorrespondingquotients.Wecansolveeachequationforthefirsttermontheright—thefirstargumentoftheremainderfunction:

InSection4.7,weshowedhowthelastnonzeroremainderrninthesequenceisequaltotheGCDoftheoriginalarguments.ForBézout’sidentity,we’dliketoshowthatrn=gcd(a,b)=xa+yb.Ifeachoftheequationsintheprecedingseriescouldbewrittenaslinearcombinationsofaandb,thenextonecouldaswell.Thefirstthreeareeasy:

Thelastequationcomesfromouroriginaldefinitionofr1inthefirstremaindersequence(Equation12.8).Thenextonerequiressubstitutingtheexpansionofr1andrearrangingterms:

Next,wehaveaniterativerecurrence.Assumethatwealreadyfiguredouthowtorepresenttwosuccessiveremaindersaslinearcombinations:

ri=xia+yib

ri+1=xi+1a+yi+1b

Thenwecanuseourpreviousobservationabouthowtodefineanarbitraryremainderinthesequenceusingthetwopreviousones(Equation12.9),substitutingthosepreviousvaluesandagaindistributingandrearrangingtogroupallthecoefficientswithaandallthecoefficientswithb:

Wehaveseenhowwecanexpresseveryentryinthesequenceasalinearcombinationofaandb,andourproceduregivesusthecoefficientsxandyateverystep.Furthermore,weobservethatthecoefficientsonaaredefinedintermsofthepreviouscoefficientsona(i.e.,theonlyvariablesarexs),andthecoefficientsonbaredefinedintermsofthepreviouscoefficientsonb(i.e.,theonlyvariablesareys).Inparticular,thecoefficientsoniterationi+2are

Whenwereachtheend,wehavecoefficientsxandysuchthatxa+yb=gcd(a,b),whichwaswhatwewantedallalong.We’veseenthatthattheysdonotdependonthexsandthexsdonotdependontheys.Sinceweknow

xa+yb=gcd(a,b),thenaslongasb≠0,wecanrearrangethesetermstodefineyas

Thismeanswedon’tneedtogothroughthetroubleofcomputingalltheintermediatevaluesofy.

Exercise12.6.Whatarexandyifb=0?

***

NowthatweknowhowtocomputethecoefficientxfromBézout’sidentity,wecanenhanceourGCDalgorithmtoreturnthisvalueaswell.ThenewalgorithmiscalledextendedGCD(alsoknownastheextendedEuclidalgorithm).ItwillstillcomputetheusualseriesofremaindersforGCD,butitwillalsocomputetheseriesofxcoefficientsfromourearlierequations.Aswehaveseen,wedon’tneedtokeepeveryvalueintherecurrence;wejustneedthetwoprevious

ones,whichwe’llcallx0andx1.Ofcourse,weneedtoknowthefirsttwovaluestogettheseriesstarted,butfortunatelywehavethem—theyarethecoefficientsofainthefirsttwolinearcombinations(Equations12.10and12.11),namely1and0.Thenwecanusethexi+2formulawehavejustderived(Equation12.12)tocomputethenewvalueeachtime.Eachnewxcomputationrequiresaquotient,sowe’llneedafunctionforthat.Meanwhilewe’restillcomputingtheGCD,sowestillneedtheremainder.Sinceweneedbothquotientandremainder,we’llusethegenericversionofthequotient_remainderfunctionweintroducedinSection4.6,whichreturnsapairwhosefirstelementisthequotientandwhosesecondelementistheremainder.HereisourextendedGCDcode:Clickheretoviewcodeimage


std::pair<E,E>extended_gcd(Ea,Eb){

Ex0(1);

Ex1(0);

while(b!=E(0)){

//computenewrandx

std::pair<E,E>qr=quotient_remainder(a,b);

Ex2=x0-qr.first*x1;

//shiftrandx

x0=x1;

x1=x2;

a=b;

b=qr.second;

}

return{x0,a};

}

Attheend,whenbiszero,thefunctionreturnsapairconsistingofthevalueofxthatwewantedforBézout’sidentity,andtheGCDofaandb.

Exercise12.7.DevelopaversionoftheextendedGCDalgorithmbasedonStein’salgorithm.

12.5ApplicationsofGCDTowrapupourdiscussionofGCD,weconsidersomeimportantusesofthealgorithm.Cryptography.Asweshallseeinthenextchapter,moderncryptographicalgorithmsrelyonbeingable

tofindthemultiplicativeinversemodulonforlargenumbers,andtheextendedGCDalgorithmallowsustodothis.WeknowfromBézout’sidentitythat

xa+yb=gcd(a,b)

so

xa=gcd(a,b)–yb

Ifgcd(a,b)=1,then

xa=1–yb

Thusmultiplyingxandagives1plussomemultipleofb,ortoputitanotherway,

xa=1modb

AswelearnedinChapter5,twonumberswhoseproductis1aremultiplicativeinverses.Sincetheextended_gcdalgorithmreturnsxandgcd(a,b),iftheGCDis1,thenxisthemultiplicativeinverseofamodb;wedon’tevenneedy.RationalArithmetic.Rationalarithmeticisveryusefulinmanyareas,anditcan’tbedonewithout

reducingfractionstotheircanonicalform,whichrequirestheGCDalgorithm.SymbolicIntegration.Oneoftheprimarycomponentsofsymbolicintegrationisdecomposinga

rationalfractionintoprimitivefractions,whichusestheGCDofpolynomialsoverrealnumbers.RotationAlgorithms.WesawinChapter11howtheGCDplaysaroleinrotationalgorithm.Infact,the

std::rotatefunctioninC++reliesonthisrelationship.

12.6ThoughtsontheChapterInthischapter,wesawtwoexamplesofhowcontinuedexplorationofanoldalgorithmcanleadtonewinsights.Stein’sobservationsaboutpatternsofoddandevennumberswhencomputingtheGCDallowedhimtocomeupwithamoreefficientalgorithm,onethatexposedsomeimportantmathematicalrelationships.Bachet’sproofofatheoremabouttheGCDgaveustheextendedGCDalgorithm,whichhasmanypracticaluses.Inparticular,thediscoveryoftheSteinalgorithmillustratesafewimportantprogrammingprinciples:1.Everyusefulalgorithmisbasedonsomefundamentalmathematicaltruth.WhenSteinnoticedsomeusefulpatternsincomputingtheGCDofoddandevennumbers,hewasn’tthinkingaboutsmallestprimes.Indeed,it’sverycommonthatthediscovererofanalgorithmmightnotseeitsmostgeneralmathematicalbasis.Thereisoftenalongtimebetweenthefirstdiscoveryofthealgorithmanditsfullunderstanding.Nevertheless,itsunderlyingmathematicaltruthisthere.Forthisreason,everyusefulprogramisaworthyobjectofstudy,andbehindeveryoptimizationthereissolidmathematics.

2.Evenaclassicalproblemstudiedbygreatmathematiciansmayhaveanewsolution.Whensomeonetellsyou,forexample,thatsortingcan’tbedonefasterthannlogn,don’tbelievethem.Thatstatementmightnotbetrueforyourparticularproblem.

3.Performanceconstraintsaregoodforcreativity.ThelimitationsthatSteinfacedusingtheWEIZACcomputerin1961arewhatdrovehimtolookforalternativestothetraditionalapproach.Thesameistrueinmanysituations;necessityreallyisthemotherofinvention.

13.AReal-WorldApplication

Iamfairlyfamiliarwithallformsofsecretwritings,andammyselftheauthorofatriflingmonographuponthesubject,inwhichIanalyzeonehundredandsixtyseparateciphers,but

Iconfessthatthisisentirelynewtome.SherlockHolmes

Throughoutthisbook,we’veseenexamplesofimportantalgorithmsthatcameoutofworkonnumbertheory.We’vealsoseenhowattemptstogeneralizethosemathematicalresultsbroughtaboutthedevelopmentofabstractalgebra,andhowitsideasaboutabstractionleddirectlytotheprinciplesofgenericprogramming.Nowwe’regoingputitalltogether.We’llshowhowourmathematicalresultsandourgeneralizedalgorithmscanbeusedtoimplementareal-worldapplication:aparticularkindofsystemforsecurecommunication,knownasapublic-keycryptosystem.

13.1CryptologyCryptologyisthescienceofsecretcommunication.Cryptographyisconcernedwithdevelopingcodesandciphers;1cryptanalysiswithbreakingthem.

1Technically,acodeisasystemforsecretcommunicationwhereameaningfulconceptsuchasthenameofaperson,place,oreventisreplacedwithsomeothertext,whileacipherisasystemformodifyingtextatthelevelofitsrepresentation(lettersorbits).Butwe’llusethetermsinterchangeably;inparticular,we’lluseencodeanddecodeinformallytomeanencipheranddecipher.

Theideaofsendingsecretmessagesdatesbacktoancienttimes,withexamplesofcryptographyinmanysocietiesincludingSpartaandPersia.JuliusCaesarusedthetechniqueofreplacingletterswiththoseina“rotated”alphabet(nowknownastheCaesarcipher)tosendmilitarymessages.Inthe19thcentury,cryptographyand“cryptograms,”puzzlesthatuseasimplesubstitutioncipher,caughtthepublic’simagination.Inan1839magazinearticle,EdgarAllenPoeclaimedhecoulddecipheranysuchmessageshisreaderssubmitted—andapparentlysucceeded.Afewyearslaterhepublishedashortstorycalled“TheGoldBug,”whichincludesanaccountofhowtobreaksuchacode.Asubstitutioncipheralsofeaturedprominently,severaldecadeslater,inSirArthurConanDoyle’sSherlockHolmesmystery,“TheAdventureoftheDancingMen.”Buttheimportanceofcryptographywentwellbeyondcasualentertainment.Codesandciphersplayed

animportantroleindiplomacy,espionage,andwarfare.Bytheearly20thcentury,creatingbettercryptographicschemeswasatoppriorityforthemilitaryoftheworld’sleadingpowers.Theabilitytobreakthesecodescouldmakethedifferencebetweensuccessandfailureonthebattlefield.

BletchleyParkandtheDevelopmentofComputersInWorldWarII,themainBritishcryptanalysisgroupwasbasedatanestateinthecountrysidecalledBletchleyPark.Atthetime,theGermannavywasusinganenhancedversionofacommercialencryptionmechanismcalledtheEnigmamachine.AnearlierversionoftheEngimahadbeencrackedbyPolishcryptographerMarianRejewski,usinganelectromechanicaldevicethattestedmanypossibleEnigmasettingsinparallel.AtBletchleyPark,abrilliantyoungmathematiciannamedAlanTuring,whosepreviouswork

providedmuchofthefoundationsofwhatbecamecomputerscience,designedamuchmoresophisticatedversionofRejewski’sdevicecalleda“bombe.”ThroughtheworkofTuringandmanyothers,theAllieswereabletodeciphertheEnigmamessages,agreathelpinwinningthewar.AnotherencryptionmechanismusedbytheNaziswascalledtheLorenzmachine.Tobreakthe

Lorenzcipher,theBritishcryptographersrealizedthattheelectromechanicalbombeswerenotfastenough.SoanengineernamedTommyFlowersdesignedamuchmorepowerfulelectronicdeviceusingvacuumtubes,called“Colossus.”Althoughitwasnotageneral-purposemachineandwasonlypartiallyprogrammable,Colossusmayhavebeentheworld’sfirstprogrammableelectronicdigitalcomputer.

Acryptosystemconsistsofalgorithmsforencryptinganddecryptingdata.Theoriginaldataiscalledtheplaintext,andtheencrypteddataiscalledtheciphertext.Asetofkeysdeterminethebehavioroftheencryptionanddecryptionalgorithms:

ciphertext=encryption(key0,plaintext)plaintext=decryption(key1,ciphertext)

Thesystemissymmetricifkey0=key1;otherwise,itisasymmetric.Manyearlycryptosystemsweresymmetricandusedsecretkeys.Theproblemwiththisisthatthe

senderandthereceiverofthemessagebothmusthavethekeysinadvance.Ifthekeyiscompromisedandthesenderwantstoswitchtoanewone,hehastheadditionalproblemoffiguringouthowtosecretlyconveythenewkeytothereceiver.

***

Apublic-keycryptosystemisanencryptionschemethatmakesuseofapairofkeys:apublickeypubforencrypting,andaprivatekeyprvfordecrypting.IfAlicewantstosendamessagetoBob,sheencryptsthemessagewithBob’spublickey.TheciphertextisthenunreadabletoanyonebutBob,whouseshisprivatekeytodecryptthemessage.Tohaveapublic-keycryptosystem,thefollowingrequirementsmustbesatisfied:1.Theencryptionfunctionneedstobeaone-wayfunction:easytocompute,withaninversethatishardtocompute.Here,“hard”hasitstechnicalcomputersciencemeaningoftakingexponentialtime—inthiscase,exponentialinthesizeofthekey.

2.Theinversefunctionhastobeeasytocomputewhenyouhaveaccesstoacertainadditionalpieceofinformation,knownasthetrapdoor.

3.Bothencryptionanddecryptionalgorithmsarepubliclyknown.Thisensurestheconfidenceofall

partiesinthetechniquebeingused.Afunctionmeetingthefirsttworequirementsisknownasatrapdoorone-wayfunction.Perhapsthebest-knownandmostwidelyusedpublic-keycryptosystemistheRSAalgorithm,named

afteritscreators(Rivest,Shamir,andAdleman).Aswe’llseeshortly,RSAdependsonsomemathematicalresultsaboutprimes.

WhoInventedPublic-KeyCryptography?Foryears,itwasbelievedthatStanfordprofessorMartinHellman,togetherwithtwograduatestudents,WhitfieldDiffieandRalphMerkle,inventedpublic-keycryptographyin1976.Theyproposedhowsuchasystemwouldwork,andrealizeditwouldrequireatrapdoorone-wayfunction.Unfortunately,theydidn’tgiveanexampleofsuchafunction—itwasjustahypotheticalconstruct.In1977,MITresearchersRonRivest,AdiShamir,andLenAdlemancameupwithaprocedureforcreatingatrapdoorone-wayfunction,whichbecameknownastheRSAalgorithmaftertheinventors’initials.In1997,theBritishgovernmentdisclosedthatoneoftheirintelligenceresearchers,Clifford

Cocks,hadactuallyinventedaspecialcaseofRSAin1973—butittook20yearsafterthepublicationoftheRSAalgorithmbeforetheydeclassifiedCocks’memo.Afterthat,AdmiralBobbyRayInman,theformerheadoftheU.S.NationalSecurityAgency,claimedthathisagencyhadinventedsomesortofpublic-keycryptographictechniqueevenearlier,inthe1960s,althoughnoevidencewasgiven.Whoknowswhichcountry’sintelligenceagencywillcomeforwardnextwithanearlierclaim?

13.2PrimalityTestingTheproblemofdistinguishingprimenumbersfromcomposite...isknowntobeoneofthemostimportantandusefulinarithmetic.

C.F.Gauss,DisquisitionesArithmeticae

Animportantprobleminmoderncryptographyisdeterminingwhetheranintegerisprime.Gaussbelievedthat(1)decidingwhetheranumberisprimeorcompositeisaveryhardproblem,andsois(2)factoringanumber.Hewaswrongabout#1,asweshallsee.Sofar,heseemstoberightabout#2,whichisagoodthingforus,sincemoderncryptosystemsarebasedonthisassumption.Tofindoutifanumbernisprime,ithelpstohaveapredicatethattellsuswhetherit’sdivisiblebya

givennumberi:Clickheretoviewcodeimage

template<IntegerI>

booldivides(constI&i,constI&n){

returnn%i==I(0);

}

Wecancallthisrepeatedlytofindthesmallestdivisorofagivennumbern.JustaswedidwiththeSieveofEratosthenesinChapter3,ourloopfortestingdivisorswillstartat3,advanceby2,andstopwhenthesquareofthecurrentcandidatereachesn:Clickheretoviewcodeimage

template<IntegerI>

Ismallest_divisor(In){

//precondition:n>0

if(even(n))returnI(2);

for(Ii(3);i*i<=n;i+=I(2)){

if(divides(i,n))returni;

}

returnn;

}

Nowwecancreateasimplefunctiontodeterminewhethernisprime:Clickheretoviewcodeimage

template<IntegerI>

Iis_prime(constI&n){

returnn>I(1)&&smallest_divisor(n)==n;

}

Thisismathematicallycorrect,butit’snotgoingtobefastenough.Itscomplexityis.Thatis,it’sexponentialinthenumberofdigits.Ifwewanttotesta200-digit

number,wemaybewaitingformoretimethanthelifeoftheuniverse.Toovercomethisproblem,we’regoingtoneedadifferentapproach,whichwillrelyontheabilityto

domodularmultiplication.We’lluseafunctionobjectthatprovideswhatweneed:Clickheretoviewcodeimage

template<IntegerI>

structmodulo_multiply{

Imodulus;

modulo_multiply(constI&i):modulus(i){}

Ioperator()(constI&n,constI&m)const{

return(n*m)%modulus;

}

};

We’llalsoneedanidentityelement:Clickheretoviewcodeimage

template<IntegerI>

Iidentity_element(constmodulo_multiply&){

returnI(1);

}

Nowwecancomputeamultiplicativeinversemoduloprimep.ItusestheresultweshowedinChapter5that,asaconsequenceofFermat’sLittleTheorem,theinverseofanintegera,where0<a<p,isap−2(seeSection5.4,rightaftertheproofofFermat’sLittleTheorem).ItalsousesthepowerfunctionwecreatedinChapter7:Clickheretoviewcodeimage

template<IntegerI>

Imultiplicative_inverse_fermat(Ia,Ip){

//precondition:pisprime&a>0

returnpower_monoid(a,p-2,modulo_multiply(p));

}

Withthesepieces,wecannowuseFermat’sLittleTheoremtotestifanumbernisprime.RecallthatFermat’sLittleTheoremsays:

Ifpisprime,thenap−1−1isdivisiblebypforany0<a<p.

Equivalently:

Ifpisprime,thenap–1=1modpforany0<a<p.

Wewanttoknowifnisprime.Sowetakeanarbitrarynumberasmallerthann,raiseittothen−1powerusingmodularmultiplication(modn),andcheckiftheresultis1.(Wecallthenumberawe’reusingawitness.)Iftheresultisnotequalto1,weknowdefinitelybythecontrapositiveofthetheoremthatnisnotprime.Iftheresultisequalto1,weknowthatthere’sagoodchancethatnisprime,andifwedothisforlotsofrandomwitnesses,there’saverygoodchancethatitisprime:Clickheretoviewcodeimage

template<IntegerI>

boolfermat_test(In,Iwitness){

//precondition:0<witness<n

Iremainder(power_semigroup(witness,

n-I(1),

modulo_multiply(n)));

returnremainder==I(1);

}

Thistimeweusepower_semigroupinsteadofpower_monoid,becauseweknowwe’renotgoingtoberaisinganythingtothepower0.TheFermattestisveryfast,becausewehaveafastwaytoraiseanumbertoapower—ourO(logn)generalizedEgyptianmultiplicationalgorithmfromChapter7.

***

WhiletheFermattestworksthevastmajorityofthetime,itturnsoutthattherearesomepathologicalcasesofnumbersthatwillfoolitforallwitnessescoprimeton;theyproduceremaindersof1eventhoughthey’recomposite.ThesearecalledCarmichaelnumbers.

Definition13.1.Acompositenumbern>1isaCarmichaelnumberifandonlyif

∀b>1,coprime(b,n) bn−1=1modn

172081isanexampleofaCarmichaelnumber.Itsprimefactorizationis7·13·31·61.

Exercise13.1.Implementthefunction:

boolis_carmichael(n)

Exercise13.2.FindthefirstsevenCarmichaelnumbersusingyourfunctionfromExercise13.1.

13.3TheMiller-RabinTestToavoidworryingaboutCarmichaelnumbers,we’regoingtouseanimprovedversionofourprimalitytester,calledtheMiller-Rabintest;itwillagainrelyonthespeedofourpoweralgorithm.Weknowthatn−1iseven(itwouldbeawfullysillytorunaprimalitytestonanevenn),sowecan

representn−1astheproduct2k·q.TheMiller-Rabintestusesasequenceofsquaresw20q,w2

1q,...,w2

kq,wherewisarandomnumberlessthantheonewearetesting.Thelastexponentinthissequenceisn−1,thesamevaluetheFermattestuses;we’llseewhythisisimportantshortly.We’realsogoingtorelyontheself-cancelinglaw(Lemma5.3),exceptwe’llwriteitwithnew

variablenamesandassumingmodularmultiplication:

Forany0<x<n∧prime(n),x2=1modn x=1∨x=−1

Rememberthatinmodulararithmetic,−1modnisthesameas(n−1)modn,afactthatwerelyoninthefollowingcode.Ifwefindsomex2=1modnwherexisneither1nor−1,thennisnotprime.Nowwecanmaketwoobservations:(1)Ifx2=1modn,thenthere’snopointinsquaringxagain,

becausetheresultwon’tchange;ifwereach1,we’redone.(2)Ifx2=1modnandxisnot−1,thenweknownisnotprime(sincewealreadyruledoutx=1earlierinthecode).Here’sthecode,whichreturnstrueifnisprobablyprime,andfalseifitdefinitelyisnot:


template<IntegerI>

boolmiller_rabin_test(In,Iq,Ik,Iw){

//preconditionn>1∧n–1=2kq∧qisodd

modulo_multiplymmult(n);

Ix=power_semigroup(w,q,mmult);

if(x==I(1)||x==n-I(1))returntrue;

for(Ii(1);i<k;++i){

//invariantx=w2i–1q

x=mmult(x,x);

if(x==n-I(1))returntrue;

if(x==I(1))returnfalse;

}

returnfalse;

}

Notethatwepassinqandkasarguments.Sincewe’regoingtocallthefunctionmanytimeswithdifferentwitnesses,wedon’twanttorefactorn–1everytime.Whycanwereturntrueatthebeginningifthepower_semigroupcallreturns1or–1?Becausewe

knowthatsquaringtheresultwillgive1,andsquaringisequivalenttomultiplyingtheexponentinthepowercalculationbyafactorof2,anddoingthisktimeswillmaketheexponentn–1,thevalueweneedforFermat’sLittleTheoremtohold.Inotherwords,ifwqmodn=1or–1,thenw2kqmodn=wn–1modn=1.Let’slookatanexample.Supposewewanttoknowifn=2793isprime.Wechoosearandomwitness

w=150.Wefactorn–1=2792into22·349,soq=349andk=2.Wecompute

x=wqmodn=150349mod2793=2019

Sincetheresultisneither1nor–1,westartsquaringx:

i=1;x2=15021·349mod2793=1374

i=2;x2=15022·349mod2793=2601

Sincewehaven’treached1or–1yet,andi=k,wecanstopandreturnfalse;2793isnotprime.

LiketheFermattest,theMiller-Rabintestisrightmostofthetime.UnliketheFermattest,theMiller-Rabintesthasaprovableguarantee:itisrightatleast75%ofthetime2forarandomwitnessw.(Inpractice,it’sevenmoreoften.)Randomlychoosing,say,100witnessesmakestheprobabilityoferrorlessthan1in2200.AsKnuthremarked,“Itismuchmorelikelythatourcomputerhasdroppedabit,due...tocosmicradiations.”

2Infact,therequirementthatqmustbeoddisneededforthisprobabilityguarantee.

AKS:ANewTestforPrimalityIn2002,NeerajKayalandNitinSaxena,twoundergraduatestudentsattheIndianInstituteofTechnologyatKanpur,togetherwiththeiradvisor,ManindraAgrawal,cameupwithadeterministicpolynomial-timealgorithmforprimalitytesting,andpublishedtheirresult.Thisisaproblemthatpeopleinnumbertheoryhadbeenworkingonforcenturies.ThereisaveryclearpaperbyAndrewGranvilledescribingthetechnique.Althoughitisa

densemathematicalpaper,itisunderstandabletoasurprisinglywideaudience.Thisisunusual;mostimportantmathematicalresultsbeingpublishedinrecentdecadesrequireyearsofpriormathematicalstudytobeunderstood.Determinedreaderswhoarewillingtoputinseriouseffortareencouragedtoreadit.DespitethefactthattheAKSalgorithmisagreataccomplishment,we’renotgoingtouseithere,

becausetheprobabilisticMiller-Rabinalgorithmisstillconsiderablyfaster.

13.4TheRSAAlgorithm:HowandWhyItWorksTheRSAalgorithmisoneofthemostimportantandwidelyusedcryptosystemsinusetoday.Itisoftenusedforauthentication—toprovethatusers,companies,websites,andotherentitieswithanonlinepresencearewhotheysaytheyare.Itisalsooftenusedtoexchangeprivatekeysthatareusedinaseparate,fastersymmetriccryptosystemusedtoencodedatabeingcommunicated.SomeoftheimportantcommunicationprotocolsthatuseRSAare:

Weusemanyoftheseprotocolsdaily.Forexample,anytimeyouvisita“secure”website(onewhoseURLhasanhttpsprefix),you’rerelyingonSSL/TLS,whichinturnusesRSAor(dependingontheimplementation)asimilarpublic-keycryptosystem.RSAreliesonthemathematicalresultswe’vejustshownforprimalitytesting.RSArequirestwosteps:

keygeneration,whichneedstobedoneonlyrarely,andencoding/decoding,whichisdoneeverytimeamessageissentorreceived.Keygenerationworksasfollows.First,thefollowingvaluesarecomputed:•Tworandomlargeprimes,p1andp2(theMiller-Rabintestmakesthisfeasible)•Theirproduct,n=p1p2•TheEulerfunctionoftheirproduct,whichwecancomputeusingEquation5.5fromChapter5:φ(p1p2)=(p1–1)(p2–1)

•Arandompublickeypub,coprimewithφ(p1p2)•Aprivatekeyprv,themultiplicativeinverseofpubmoduloφ(p1p2)(Tocomputethis,we’llusetheextendedGCDfunctionwederivedinChapter12.)

Whenthesecomputationsarecomplete,p1andp2aredestroyed;pubandnarepublished,andprviskeptsecret.Atthispoint,thereisnofeasiblewaytofactorn,sinceitissuchalargenumberwithsuchlargefactors.Theencodinganddecodingprocessissimpler.Thetextisdividedintoequalsizeblocks,say256bytes

long,whichareinterpretedaslargeintegers.Themessageblocksizesmustbechosensothatn>2s.Toencodeaplaintextblock,weuseourfamiliarpoweralgorithm:Clickheretoviewcodeimage

power_semigroup(plaintext_block,pub,modulo_multiply(n));

Decodinglookslikethis:Clickheretoviewcodeimage

power_semigroup(ciphertext_block,prv,modulo_multiply(n));

Observethatwedoexactlythesameoperationtoencodeanddecode.Theonlydifferenceiswhichtextandwhichkeywepassin.

***

HowdoesRSAwork?Encryptionconsistsofraisingamessagemtoapowerpub;decryptionconsistsofraisingtheresulttothepowerprv.Weneedtoshowthattheresultofapplyingthesetwooperationsistheoriginalmessagem(modulon):

(mpub)prv=mmodn

Proof.Recallthatwespecificallycreatedprvtobethemultiplicativeinverseofpubmoduloφ(p1p2),sobydefinition,theproductofpubandprvissomemultipleqofφ(p1p2)witharemainderof1.Wecanmakethatsubstitutionintheexponentontheright.

NowwecanapplyEuler’stheoremfromChapter5,whichsaysthataφ(n)–1isdivisiblebyn;thatis,aφ(n)=1+vn.Makingthatsubstitution,wehave

=m(1+vn)q

Whenweexpand(1+vn)q,everytermwillbeamultipleofnexcept1,sowecancollapsealloftheseandjustsaywehave1plussomeothermultipleofn:

=m+wn

=mmodn

TheEulertheoremstepdependsonmbeingcoprimewithn=p1p2.Sincethemessagemcouldbeanything,howdoweknowthatitwillbecoprimetop1p2?Sincep1andp2areenormousprimes,thatprobabilityispracticallyindistinguishablefrom1,andpeoplenormallydonotworryaboutit.However,ifyouwanttoaddressthis,youcanaddoneextrabytetotheendofm.Theextrabyteisnotactuallypartofthemessage,butisthereonlytoensurethatwehaveacoprime.Whenwecreatem,wecheckwhetheritiscoprime;ifitisn’t,wesimplyadd1tothisextrabyte.

***

WhydoesRSAwork?Inotherwords,whydowebelieveit’ssecure?Thereasonisthatfactoringishard,andthereforecomputingφisnotfeasible.Perhapsifquantumcomputersprovetoberealizableinthefuture,itwillbepossibletorunanexponentialnumberofdivisortestsinparallel,makingfactoringatractableproblem.Butfornow,wecanrelyonRSAformanysecurecommunicationsapplications.

ProjectExercise13.3.ImplementanRSAkeygenerationlibrary.

Exercise13.4.ImplementanRSAmessageencoder/decoderthattakesastringandthekeyasitsarguments.

Hints:•Ifyourlanguagedoesnotsupportarbitrary-precisionintegers,you’llneedtoinstallapackageforhandlingthem.•RememberthattwonumbersarecoprimeiftheirGCDis1.Thiswillcomeinhandyforoneofthekeygenerationsteps.•You’llneedtheresultsthatwederivedinChapter12.Therearetworelevantfunctions,extended_gcdandmultiplicative_inverse.

Recallthattheextended_gcdfunctionfromChapter12returnsapair(x,y)suchthatax+ny=gcd(a,n).Youcanusethisfunctiontocheckforcoprimes.It’salsopartoftheimplementationofmultiplicative_inverse,afunctionthatreturnsthemultiplicativeinverseofamodulonifitexists,or0ifitdoesnot.Unlikethefunctionmultiplicative_inverse_fermatthatweintroducedinSection13.2,thisoneworksforanyn,notjustprimes:Clickheretoviewcodeimage

template<IntegerI>

Imultiplicative_inverse(Ia,In){

std::pair<I,I>p=extended_gcd(a,n);

if(p.second!=I(1))returnI(0);

if(p.first<I(0))returnp.first+n;

returnp.first;

}

You’llneedthistogettheprivatekeyfromthepublickey.

13.5ThoughtsontheChapterIssuesofidentity,privacy,andsecurityarebecomingincreasinglyimportantasmoreofourpersonaldatalivesonlineandmoreofourpersonalcommunicationtravelsovertheInternet.Aswehaveseen,manyimportantprotocolsforkeepingdataprivateandsecurefromtamperingrelyonRSAorsimilarpublickeycryptosystemsforauthentication,exchangeofkeysusedforencryption,orothersecurityfeatures.Theseimportantpracticalcapabilitiesowetheirexistencetoresultsfromoneofthemosttheoretical

branchesofmathematics,numbertheory.Thereisaperceptionamongprogrammersthatmathematiciansarepeoplewhodon’tknoworcareaboutpracticalconcernsandthatmathematics,particularlyinitsmoreabstractareas,haslittlepracticalvalue.Lookingatthehistory,wecanseethatbothoftheseperceptionsarefalse.Thegreatestmathematiciansenthusiasticallyworkedonextremelypracticalproblems—forexample,Gaussworkedononeofthefirstelectromechanicaltelegraphs,andPoincaréspentyearsdevelopingtimezones.Perhapsmoreimportantly,itisimpossibletoknowwhichtheoreticalideasaregoingtohavepracticalapplications.

14.Conclusions

Thestrongestargumentsprovenothingsolongastheconclusionsarenotverifiedbyexperience.

RogerBacon,OpusTertiumWestartedthisbookbycharacterizinggenericprogrammingasanattitudetowardprogrammingthatfocusesonabstractingalgorithmstotheirmostgeneralsettingwithoutlosingefficiency.Throughoutthebook,we’veseenexamplesofthisabstractionprocessinmathematicsandinprogramming.Wesawhowmathematicians’attemptstofindthemostgeneralsettingforEuclid’sGCDalgorithmledtothedevelopmentofabstractalgebra,anentireareaofmathematicsdevotedtoabstractstructures,whichitselfprovidedthebasisforgenericprogramming.Wealsosawhowtousethosesameprinciplesofabstractiontogeneralizeanancientalgorithmformultiplyingpositiveintegerstoafastpowerfunctiononsemigroups,enablingarangeofapplicationsrangingfromcomputingFibonaccinumberstofindingtheshortestpathinagraphtoencryptingdatainInternetcommunicationprotocols.Thisprocess—startingwithaspecificefficientsolutionand,wheneverpossible,relaxingtherequirements—isattheheartofgenericprogramming.Whiletheideaofabstractioningenericprogrammingcomestousdirectlyfromabstractalgebra,as

programmerswealsocareaboutefficiency.Agenericalgorithmthatrunsmoreslowlythanitstype-specificcounterpartwillnotgetused.That’swhyefficiencyisalsopartofthedefinitionofgenericprogramming.We’veshownexamplesthroughoutthebookofspecifictechniquesforimprovingefficiency,fromrewritingcodetousestrengthreduction,tousingmemory-adaptivealgorithms,toexploitingcompile-timetypedispatchsothecomputercaninvokethefastestavailableimplementationforagivensituation.Moregenerally,wehavefoundthatattemptstofindgenericversionsofalgorithmsoftenleadtosimplerandmoreefficientsolutions.We’vealsoseenhowthecorrectnessofaprogramminginterfacecanbejustasimportantasthe

correctnessoftheprogramitself.Acorrectinterfacecanenableawiderrangeofapplications.Itcanalsobringefficiencybenefits—forexample,byreturningalltherelevantcomputations(thelawofusefulreturn)toavoidaduplicationofeffort.Incontrast,anincorrectinterfacecripplestheapplicationbylimitingwhatitcando.Forexample,wesawhowafindfunctionthatreturnsonlyaBooleanvalueinsteadofthepositionofafounditemmakesitimpossibletoseeifasecondmatchingitemexists.Andjustasyouneedtorewriteaprogramafewtimestogetitright,soyouneedtoredesigntheinterface;thecorrectinterfaceusuallywon’tbeclearuntilyou’vealreadyimplementedanalgorithmandexploreditsusecases.Anotherideathat’sessentialtounderstandinggenericprogrammingisthedistinctionbetweentypeand

concept.Inmuchthesamewaythataxiomsinamathematicaltheoryarerequirementsthattelluswhatitmeanstobeacertainkindofabstractmathematicalentity(suchasagroup),conceptsinprogramingarerequirementsontypes;theytelluswhatitmeanstobeacertainkindofcomputationalentity.Choosingtherightconceptsforanalgorithmordatastructureisessentialtogoodprogramming.Choosingaconceptwithtoomanyrequirementsplacesunnecessarylimitationsontherangeofsituationsinwhichanalgorithmcanbeused.Choosingaconceptwithtoofewrequirementsmakesitimpossibletodefinealgorithmsthatdoanythinguseful.Nexttimeyousetouttowriteaprogram,trytoadoptthegenericprogrammingattitude.Startwith

specificimplementationsofyourfunctions,thenreviseandrefinethemtobemoreefficientandmoregeneral.Asyourefineyourcode,thinkcarefullyabouthowthepiecesfittogetherandhowtoprovidean

interfacethatwillstillbeusefulinthefuture.Chooseconceptsthatprovidejusttherightrequirementsforyourdata,withoutimposingunnecessaryassumptions.Andrememberthatyouaretheinheritorofalongmathematicaltraditionofalgorithmicthought.Infollowingtheprinciplesofgenericprogramming,youarealreadybenefitingfromtheworkofthosewhocamebefore,fromEuclidtoStevintoNoether.Bydesigningbeautiful,generalalgorithms,youareaddingyourownsmallcontributiontotheirwork.

FurtherReading

Readerswhoareinterestedinlearningmoreaboutthetopicsdiscussedinthisbookmaywishtolookatsomeofthereferencesmentionedhere.CompletecitationsareincludedintheBibliography.

Chapter1GenericProgramming.ThelanguageTecton,whichfirstusedgenericprogrammingconcepts,isdescribedin“Tecton:ALanguageforManipulatingGenericObjects”byKapur,Musser,andStepanov(1981).TheAdalibraryisdescribedinthepaper“GenericProgramming”byMusserandStepanov(1988),andC++STLin“TheStandardTemplateLibrary”byStepanovandLee(1994).Allofthesematerialsareavailableonwww.stepanovpapers.com.

Chapter2HistoryofMathematics.Agoodcomprehensivereference,notonlyforthischapterbutalsoformuchofthemathematicalhistoryinthebook,isKatz’sAHistoryofMathematics:AnIntroduction(2009).Thisgeneraltextbookcombinesthoroughnessandmathematicalrigorwithaccessibilitytothelayperson.AnincisivebookthatexplainsthehistoricaldevelopmentofsomemajormathematicalideasisMathematicsandItsHistorybyJohnStillwell(2010).RhindPapyrus.ToseeareproductionoftheRhindPapyrus,togetherwithitstranslation,seeThe

RhindMathematicalPapyrus:AnAncientEgyptianTextbyRobinsandShute(1987).VanderWaerdenincludesadiscussionoftheRhindPapyrusinGeometryandAlgebrainAncientCivilizations(1983).

Chapter3EgyptianandGreekMathematics.InadditiontoKatz,twoexcellentresourcesareVanderWaerden’sScienceAwakening(1963)andthetwo-volumeHistoryofGreekMathematicsbySirThomasHeath(originallypublishedin1921butavailableina1981reprint).Bothareveryaccessibletothegeneralreader.FigurateNumbers.ThebestintroductiontoPythagoreanarithmeticisthebookbyNicomachusof

Gerasa,whichcaneasilybefoundinthe10thvolumeoftheBritannica’sGreatBooksoftheWesternWorld,editedbyMortimerAdler.ThisvolumealsocontainsthecompleteworksofEuclidandArchimedes.BasicNumberTheory.AgoodintroductiontobasicnumbertheoryisinChapterIIIofGeorge

Chrystal’sAlgebra:AnElementaryText-Book.

Chapter4GreatestCommonMeasure.ForgeneralhistoryofGreekmathematics,includingthetopicscoveredinthischapter,thebestreferenceisstillHeath’sAHistoryofGreekMathematics,mentionedinthetopicsforChapter3.ForafascinatingandmathematicallysophisticatedaccountofthemathematicalstudiesinPlato’sAcademy,includingthealgorithmforthegreatestcommonmeasure,seeDavidFowler’sTheMathematicsofPlato’sAcademy,aNewReconstruction.Forthoseinterestedinreadingthesource,Plato’scompleteworksareavailableinaone-volumeedition,withaclearmoderntranslation,editedbyJohnM.Cooper.AgoodexplanationoftheGCDmaybefoundinChapterIIIofGeorgeChrystal’sAlgebra:AnElementaryText-Book.

http://www.stepanovpapers.com

DeclineofGreekScience.AnimportantaccountoftheriseanddeclineofGreekmathematicsispresentedinthebookbyLucioRusso,TheForgottenRevolution:HowScienceWasBornin300BCandWhyItHadtoBeReborn.HistoryofZero.OuraccountofthehistoryofzeroislargelytakenfromvanderWaarden’sScience

Awakening,pp.56–57.LeonardoPisano(Fibonacci).AshortautobiographyofLeonardoPisanohasbeentranslatedby

RichardGrimm.HisgreatworkLiberAbaciisavailableinanEnglishtranslationbyLaurenceSigler.AbriefbutthoroughdescriptionofLeonardoPisano’snumbertheoretictreatiseisgiveninMcClenon’s1919article“LeonardoofPisaandHisLiberQuadratorum.”ReadersinterestedinhistoryofarithmeticalgorithmsmaywanttoreadLeonardoPisano’soriginalLiberQuadratorum,availableinamodernEnglishtranslationbyL.E.Sigler(seeFibonacciintheBibliography).RemainderandQuotient.ThefulltreatmentoftheextensionofGCDtoremainderandquotientisin

Chapter5ofElementsofProgrammingbyStepanovandMcJones(2009).FloydandKnuth’salgorithmforremainderappearsintheir1990article“AdditionMachines.”

Chapter5Fermat’sandEuler’sNumberTheoryWork.AsourceformuchofthematerialinthischapterisNumberTheory:AnApproachthroughHistoryfromHammurapitoLegendrebyAndréWeil.Whilethisbookdoesnotassumeanyadvancedmath,itisprobablytoodetailedformostcasualreaders.TheclassicnumbertheorytextsbyGauss(DisquisitionesArithmeticae)andDirichlet(LecturesonNumberTheory)arestillofgreatvalue,butwouldbeofinteresttoseriousscholarsonly.Euler’sBooks.OurbiographyofEuleralsomentionshisfoundationalworksoncalculus.Althoughthey

arenotdirectlyrelatedtothetopicsofthisbook,theyarestillworthcarefulreading.Euler’sfirstbookonthesubject,IntroductiontoAnalysisoftheInfiniteisavailableinEnglish.Sadly,onlythefirsthalfofhissecondbook,FoundationsofDifferentialCalculus,hasanEnglishtranslation,andallofhisIntegralCalculusstillawaitsanEnglishtranslation.ThebookLetterstoaGermanPrincessisavailableontheInternet.

Chapter6GroupTheory.AclassicbookongrouptheoryisBurnside’sTheoryofGroupsofFiniteOrder.Thoughfirstpublishedin1897,itstillgivesanunparalleledintroductiontowhatgrouptheoryisaboutandincludesmanymoreexamplesthanmostmodernbooks.ItwasreprintedbyDoverPressin2004.ModelTheory.Sadly,wearenotawareofanintroductiontomodeltheoryaccessibletoalayperson.

Foramoreadvancedreader,agoodintroductionisH.JeromeKeisler’s“FundamentalsofModelTheory,”achapterintheHandbookofMathematicalLogic.

Chapter7RequirementsonTypes.ManytopicsinthischapterarediscussedmoreformallyinElementsofProgrammingbyStepanovandMcJones.Reduction.Iversondiscussesreductioninhispaper“NotationasaToolofThought”(1980).Theidea

isalsodiscussedinBackus’“CanProgrammingBeLiberatedfromtheVonNeumannStyle?”(1978).Usingreductionforparallelcomputationwasdiscussedin“OperatorsandAlgebraicStructures”byKapur,Musser,andStepanov(1981).Dean’suseofreductionisdescribedin“Map-Reduce:SimplifiedDataProcessingonLargeClusters”(2004).

Chapter8SimonStevin.DespiteStevin’sgreatcontributionstoscienceandmathematics,verylittlehasbeenpublishedabouthiswork.AgoodoverviewisSarton’s“SimonStevinofBruges.”PolynomialDivisionandGCD.ForarefresheronpolynomialdivisionandpolynomialGCD,see

Chapters5and6ofChrystal’sAlgebra.OriginsofAbstractAlgebra.AgoodintroductiontoGaussianintegersisChapter6ofStillwell’s

ElementsofNumberTheory.TheclassictextthatintroducedthegeneralnotionofalgebraicintegersisRichardDedekind’sTheoryofAlgebraicIntegers.Stillwell’stranslationincludesanexcellentintroductionthatexplainsmanyoftheideas.LeoCorry’sModernAlgebraandtheRiseofMathematicalStructuresisanexhaustivescholarlytreatmentoftheemergenceofabstractalgebrafromDedekindtoNoetherandlaterdevelopments.AbstractAlgebra.Forthereaderwhowantstotakethenextstepinunderstandingabstractalgebra,a

seriousbutaccessible(andhistoricallyinformed)textisStillwell’sElementsofAlgebra.Rings.Stillwell’sElementsofNumberTheorycoversthesetopicsthoroughlyandisquiteaccessible.

(WhileitstitlemightsuggestthatthistopiciscoveredinStillwell’sElementsofAlgebra,thatbookfocusesonGalois’sworkandthereforedoesnotcoverrings.)

Chapter9SocialNatureofProof.Theideathatproofisasocialprocessisdiscussedin“SocialProcessesandProofsofTheoremsandPrograms,”byDeMillo,Lipton,andPerlis(1979).Euclid.SirThomasHeath’stranslationofEuclid,TheThirteenBooksoftheElements,iswidely

available.Thiseditionincludesveryextensivecommentarybythetranslator.Inaddition,thereisnewreproductionofOliverByrne’s1847uniqueedition,whichdemonstratesalltheproofsthroughvisualillustrations.RobinHartshorne’sGeometry:EuclidandBeyondisatextbookaimedatuniversitymathmajors,butthefirstchapter(describingEuclid’sgeometry)isquiteaccessible.AxiomsofGeometry.Chapters1and2ofHartshorne’sGeometry:EuclidandBeyondprovideagood

introductiontoEuclid’saxiomaticmethodandHilbert’smodernversionofEuclideanaxioms.Hilbert’sFoundationsofGeometryisstillthedefinitivetreatmentofhisaxiomsforGeometry.Non-EuclideanGeometry.AclassictreatmentofthistopicisRobertoBonola’sNon-Euclidean

Geometry:ACriticalandHistoricalStudyofItsDevelopment.Amodern(butstillaccessible)mathematicaltreatmentmaybefoundinChapter7ofHartshorne’sGeometry.PeanoArithmetic.Forreadersinterestedinhowarithmeticcanbebuiltrigorouslyfromthegroundup,

EdmundLandau’sFoundationsofAnalysisdescribeshowtoconstructintegers,rationals,reals,andcomplexnumbersstartingwithPeano-likeaxioms.Peano’smagnumopusFormularioMathematicoactuallycoveredmanyareasofpracticalmathematics,notjusttheaxiomsthatbecamefamous.Unfortunately,thisgreatbookhasneverbeentranslatedfromitsoriginalinventedlanguage.FormoreaboutPeano’swork,seeTwelveArticlesonGiuseppePeanobyHubertKennedy.

Chapter10Aristotle’sOrganizationofKnowledge.AgoodintroductiontoAristotle’slifeandphilosophyisSirDavidRoss’sAristotle.ThereareseveralgoodeditionsofAristotle’scompleteworks,includingthetwo-volumeBollingenFoundationeditionandthemultivolumeLoebClassicalLibraryedition.ReadersinterestedonlyinAristotle’sCategories,theworkdiscussedinthischapter,canchoosetheappropriatevolume.Concepts.Chapter1ofElementsofProgrammingbyStepanovandMcJonescoversthismaterialmore

formallyandinmoredetail.IteratorsandSearch.Chapter6ofElementsofProgrammingcoversthismaterialmoreformallyand

inmoredetail.

Chapter11PermutationsandTranspositions.AgoodintroductiontopermutationsisinChapterXXIIIofChrystal’sAlgebra.Chapter1ofBurnside’sTheoryofGroupsofFiniteOrder,mentionedunderChapter6,givesmoredetailsaboutthesetopics.RotateandReverse.ThealgorithmsinthischapteraredescribedinmoredetailinChapter10of

ElementsofProgramming.

Chapter12Stein’sAlgorithm.Stein’soriginalpaperonthefasterGCDalgorithmis“ComputationalProblemsAssociatedwithRacahAlgebra.”KnuthdescribesthealgorithminSection4.5.2ofTheArtofComputerProgramming,Vol.2.RecreationalMathematics.Manydevelopmentsinmathematicscamefromstudyingseemingly

frivolousproblems,andmanydistinguishedmathematiciansbecameinterestedinmathematicsthroughexposuretomathematicalgames.AclassicbookonthesubjectisMathematicalRecreationsandEssaysbyW.W.RouseBall.

Chapter13Cryptography.AnentertaininghistoryofcryptographyisDavidKahn’sTheCode-breakers:TheComprehensiveHistoryofSecretCommunicationfromAncientTimestotheInternet.Tolearnmoreaboutmethodsusedinmoderncryptography,astandardtextisIntroductiontoModernCryptography:PrinciplesandProtocolsbyKatzandLindell.NumberTheory.Agoodmodernintroductiontonumbertheory,whichincludesadiscussionoftheRSA

algorithm,isJohnStillwell’sElementsofNumberTheory(2003).ItalsoincludessomematerialthatwecoverinChapters5and8.AKSPrimalityTesting.Thedeterministicpolynomial-timealgorithmforprimalitytestingisdescribed

inGranville’spaper“ItIsEasytoDetermineWhetheraGivenIntegerIsPrime”(2005).

AppendixA.Notation

Thefollowingaresymbolsusedinthebookthatmaynotbefamiliartoanonmathematicalreaderandthatarenotexplainedinthemaintext.(Othernewsymbolsareexplainedwhentheyareintroduced.)We’lllistthesymbolsfirst,thengiveafewexamplesoftheiruse.¬pLogicalnegation.Read“notp.”Ifpistrue,then¬pisfalse,andviceversa.

p∨qLogicaldisjunction.Read“porq.”Thestatementp∨qistrueifeitherpistrue,orqistrue,ortheyarebothtrue.

p∧qLogicalconjunction.Read“pandq.”Thestatementp∧qistrueonlywhenbothpandqaretrue.

p qLogicalimplication.Read“pimpliesq”or“ifp,thenq.”Notethatthestatementp qisfalseonlywhenpistrueandqisfalse.Itmaynotbeintuitivethattheexpressionistruewhenpisfalse,butonewaytothinkofitis“youcanmakeanargumentforanythingifyougettostartbyassumingsomethingthatisn’ttrue.”Formoreaboutlogicalimplication,see“ImplicationandtheContrapositive”attheendofthisappendix.

p qLogicalequivalence.Read“pifandonlyifq”andsometimeswritten“piffq.”Thestatementp q

istruewhenbothpandqaretrue,orwhenbothpandqarefalse.Thisisexactlythesameassaying(pq)∧(q p).

x SSetmembership.Read“xisanelementofS”or“xisinS.”

x∉SNegationofsetmembership.Read“xisnotanelementofS”or“xisnotinS.”

∀x SUniversalquantifier.Read“forallxinS”or“foranyxinS.”SometimesmembershipinthesetSisassumedfromthecontext,sowejustwrite∀x.

∃x SExistentialquantifier.Read“thereexistsanxinS”or“thereisanxinS.”SometimesmembershipinthesetSisassumedfromthecontext,sowejustwrite∃x.

S∪TSetunion.Read“theunionofSandT.”AnelementxisintheunionofSandTifitisineitherSorTorboth.

S∩T

Setintersection.Read“theintersectionofSandT.”AnelementxisintheintersectionofSandTifitisinbothSandT.

S={x|...}Setdefinition.Read“Sisthesetofallxsuchthat...”(wherethe“...”couldbeanylistofconditionsaboutx).

Thesetofnaturalnumbers0,1,2,3,...—numbersusedforcounting.(Someauthorsdonotinclude0inthesetofnaturalnumbers.)

Thesetofintegers,whichincludesallthenaturalnumbersand(forallexceptzero)theirnegations.

Theset{0,1,2,...,n–1}ofremaindersmodulon.

Theset ofrationalnumbers(theratiooftwointegers).

Thesetofrealnumbers.

Thesetofcomplexnumbersa+bi,whereaandbarerealnumbersandi2=–1.

Exampleseven(x) ¬odd(x)“xisevenifandonlyifxisnotodd.”

S={x|x ∧even(x)}“Sisthesetofallx’ssuchthatxisinthesetofintegersandxiseven”or,moreconcisely,“Sisthesetofallevenintegers.”

∀x∃y:y=x+1“Foranyx,thereisaysuchthatyequalsxplus1.”

x {S∩T} x {S∪T}“IfxisintheintersectionofSandT,thenxisintheunionofSandT.”

ImplicationandtheContrapositiveTheimplicationp q(alsoknownasaconditional)islogicallyequivalenttoavariantcalledthecontrapositive,whichhasthefollowingform:

¬q ¬p

Considerthisexample:

Ifn=2,thenniseven.

Hereourpropositionpis“n=2”andourpropositionqis“niseven”;thisconditionalhappenstobetrue.Toformthecontrapositive,welogicallynegatebothsidesandreversethedirectionoftheimplication.Sothecontrapositiveoftheprecedingstatementis

Ifnisnoteven,thenn≠2

whichisalsotrue.Sinceaconditionalstatementanditscontrapositivearelogicallyequivalent,weoccasionallyreplace

onewiththeother,insituationswherethelatterformismoreconvenient.Don’tconfusethecontrapositiveofp qwithitsconverse,whichisq p.Justbecausean

implicationistruedoesnotmeanitsconverseistrue;thetwoareindependent.Continuingtheearlierexample,eventhoughouroriginalstatementwastrue,itsconverse

Ifniseven,thenn=2

isclearlyfalse.

AppendixB.CommonProofTechniques

Thereareafewstandardprooftechniquesthatoccurfrequentlyinmathematicsandcomputerscience,andwhichweuseinthisbook.Ifyouarehavingtroubleunderstandingtheproofsinthemaintext,youmaywanttoreviewthissection.

B.1ProofbyContradictionManythingswewanttoprovehavetheform“ifp,thenq”(alsosometimeswritten“p q”),wherepandqaretwopropositions.Wealwaysstartwiththepremisethatpistrue;otherwise,wewouldbesolvingadifferentproblem.Theideaofproofbycontradictionistoassumetheoppositeofwhattheoriginalconjectureconcludes(i.e.,assumethatqisnottrue),andthenshowthatthisassumptionwouldleadtoalogicalcontradiction—inparticular,thatpropositionpwouldbefalse,whichweknowisnotthecase.Thisforcesustoconcludethatpropositionqmustbetrueafterall,whichiswhatwereallywantedallalong.Let’slookatanexample.Supposewewanttoprovethatforallintegersn:

Ifn2isodd,thennisodd.

Here“n2isodd”isourpand“nisodd”isourq.Solet’sassumetheoppositeconclusionistrue,thatnisnotodd—thatniseven.Whatdoesitmeanforanintegerntobeeven?Itmeanswecanwriteitastwicesomeotherintegerm:

n=2m

Whathappensifwesquaren?

n2=2·2·m2

Let’sintroduceanewvariablex,andsetx=2m2.Thenwecansubstitute:

n2=2·2m2=2x

Nowweseethatn2canbeexpressedastwicesomeotherintegerx.Butthat’sthedefinitionofeven,andourpremisewasthatn2isodd.n2can’tbebothevenandodd—that’salogicalcontradiction.Sotheassumptionwemadeatthebeginningthatnisevenmustbefalse;nmustthereforebeodd,andwe’veprovedtheoriginalresult.

B.2ProofbyInductionSomeresultswe’dliketoproveinvolveinfinitesetsofthings.Obviouslyinthesesituationswecan’tenumerateallthecases,butwecanoftenusemathematicalinductiontoobtainourresult.Toprovesomethingbyinduction,youneedtodotwothings:

•Provethatit’strueforthefirstelementintheset.Thisiscalledthebasis.•Provethatifit’strueforanarbitraryelementintheset(theinductionhypothesis),thenit’salsotrueforthesuccessorofthatelement.Thisiscalledtheinductivestep.

Forexample,supposewewanttoprovethatforanypositiveintegern:

Basis:Doestheequationholdifn=1?Inotherwords,isittruethat

Wecanjustdothearithmetic,andseethattheansweris“yes.”Inductivestep:

Assumethattheequationistrueforn=k.Ifthatweretrue,woulditalsobetruefork+1?Thisiswhatitmeanstobetruefork(i.e.,thisisourinductionhypothesis):

Let’saddk+1tobothsides:

Thisisjust ,wheren=k+1.Sowehaveprovedthatiftheequationistruefork,it’struefork+1.

Sincewehaveshownboththebasisandtheinductivestep,wehaveprovedouroriginalstatement.

B.3ThePigeonholePrincipleThepigeonholeprinciple(sometimesknownastheDirichletprinciple)isverysimple:ifyouhavenpigeonholesandmorethannpigeons,thenatleastonepigeonholemustcontainmorethanonepigeon.Therearelotsofexamplesofthisinreallife.Forexample,ifyouhave367people,atleasttwoofthemmusthavethesamebirthday.Buttheideaalsoturnsouttobeusefulinsomemathematicalproofs.Oftenwhenyouseeatheoremthat’stryingtoprovethattwothingswillbethesame,thepigeonholeprincipleisagoodapproach.Here’sanexample:

Provethatanysetof10positiveintegerssmallerthan100willalwayscontaintwosubsetswiththesamesum.

First,let’sconsiderhowmanypossiblesumswecanget.Thesumofany10positiveintegerslessthan100can’tbesmallerthan10orlargerthan990,soitmustbesomewhereintherange[10,990].Thatrangecontains990−10=980values,sothat’sthemaximumnumberofpossiblesums.1Next,let’sseehowmanypossiblesubsetsofthose10integersthereare.Wecanrepresenteachsubsetasabinarynumberwheretheithbitis1iftheithintegerinthesetisinthatsubset,and0otherwise.Thereare10

elementsintheset,andweuseonebitforeachelement,sothereare210=1024possiblesubsets.Sincethereareonly980possiblesums,andthereare1024possiblesubsets,bythepigeonholeprinciple,atleasttwoofthesubsetsmusthavethesamesum.

1Actually,theproblemsaysthatwehaveasetof10integers,andasetmaynotcontainrepeatedelements.Sotheactualnumberofpossiblesumsissmallerthan980,buttheresultoftheproofisthesame.

AppendixC.C++forNon-C++Programmers

ThisbookgenerallyusesasubsetofC++thatshouldbeeasilyunderstandabletomostprogrammerswhohaveusedalanguagelikeCorJava.However,thereareafewimportantfeaturesandidiomsspecifictoC++thatwerelyon.ThesearedescribedinthisAppendix.Exceptwherenoted,weuseonlyfeaturesofC++thathavebeenavailableinthe1998standardversionofthelanguage.ForagoodbriefintroductiontoC++11,seeATourofC++byBjarneStroustrup.Foracompletereference,seeStroustrup’sTheC++ProgrammingLanguage.

C.1TemplateFunctionsOnewayC++supportsthegenericprogrammingparadigmisthroughtheuseoftemplates.Supposeyouhaveafunctionlikethis:

intmy_function(intx){

inty;

...dosomethingcomplicated...

returny;

}

Nowyouwanttodothesamesetofcomputations,butthistimeyouwantthefunctiontotakeandreturnadouble-precisionfloating-pointnumber.C++allowsyoutooverloadthefunctionname,soyoucanwriteawholenewfunctionwiththesamenamethatworkswithdifferenttypes:

doublemy_function(doublex){

doubley;


returny;

}

Butifeverythingisthesameexceptforthetype,writingawholeseparatefunctioniswasteful.Templatesavoidthisproblem.Withtemplates,youcanwriteasinglefunctiontoworkonanytypethat

satisfiesboththesyntacticandsemanticrequirementsofthecode,likethis:template<typenameT>

Tmy_function(Tx){

Ty;


returny;

}

NowwehaveafunctionthattakestypeTandreturnstypeT,whereTdependsonhowthefunctioniscalled.Ontheonehand,ifyousay

intx(1);

inty=my_function(x);

thenmy_function()willbecalledwithTsettoint.Ontheotherhand,ifyousaydoublex(1.0);

doubley=my_function(x);

thenmy_function()willbecalledwithTsettodouble.Thisisdoneatcompiletime,sothereisnoperformancepenaltyforusingtemplates.

C.2ConceptsConceptsaretheessentialpartofgenericprogramming,andwediscusstheminsomedetailinSection10.3.Thefollowingdiscussionisintendedtobeaquickreference.Ideally,wewouldliketohaveawaytotelltheprogrammerwhattherequirementsareonagiven

templateargument.Forinstance,we’dliketosaytemplate<NumberN>

andhavethatmeanthatwhatevertypethisfunctiongetscalledwithmustbeanumber.Thismeansthecodeisintendedtoworkforthingsintsanddoublesanduint64_ts,butnot,say,forstrings.Arestrictionlike“Number”isanexampleofaconcept.Unfortunately,asofthiswriting,C++doesnotsupportconceptsasabuilt-inpartofthelanguage—thatis,C++doesnothaveanywaytoenforcerequirementsontemplatetypes.Despitethislimitation,wewillwriteourcodeexamplesasifconceptswerepresentinthelanguage.

Wecanimplementthisbyjustdefiningourfavoriteconceptsasaliasesfortypename:#defineNumbertypename

Sowhenwewritetemplate<NumberN>

asfarasthecompilerisconcerned,it’sasifwewrotetemplate<typenameN>

butthehumanprogrammerwillunderstandtheintendedrestriction.

C.3DeclarationSyntaxandTypedConstantsC++providesmultiplewaystodeclareandinitializeavariable.WhilethetraditionalCsyntax

intx=y;

islegal,thecommonwaytowritethisinC++isintx(y);

ThisismoreconsistentwiththesyntaxusedtoconstructarbitraryC++objects.(Note:ThecurrentversionofC++supportsanadditionalwaytodoinitialization:

intx{y};

However,thisusageisstilllesswidespread,sowedonotuseitinourexamples.)Whenusingnumericconstants,we’llbeverycarefulabouttypes.Forexample,atraditionalCprogram

mightcontainalinelikethis:if(something)return0;

Thisisabitsloppy.Whichtypeisthe0returnedbythefunction?Bydefault,it’sanint,butsupposeourprogramwassupposedtoreturnaspecifickindofinteger,likea16-bitunsignedinteger,oronespecifiedbyatemplateargument.Ratherthanrelyonimplicittypeconversion,we’lltrytobeexplicitaboutwhatwe’rereturningbywritingsomethinglikethis:

if(something)returnuint16_t(0);

or,inthecaseofatemplateargument:if(something)returnT(0);

whereTisthetypespecifiedinthetemplate.

C.4FunctionObjectsOftenwe’dliketohaveafunctionthatrequiressomeinitializationandmaintainssomestate.AcommonwaytoimplementthisinC++isbytheuseofafunctionobject,alsocalledafunctor.Afunctionobjectisanobjectthatencapsulatesasingle(unnamed)function.Let’slookatasimpleexample—afunctionobjectfordoingcurrencyconversion:Clickheretoviewcodeimage

structconverter{

doubleexchange_rate;

converter(doubleex):exchange_rate(ex){}

doubleoperator()(doubleamt){

returnamt*exchange_rate;

}

};

Notetheuseofthesyntaxoperator()todeclaretheunnamedfunctionthatbelongstotheobject.Tousethisfunction,wefirstconstructaninstanceofourconverterobject(whichinitializesthe

exchangerate).InthisexamplewewanttoconverteurostoU.S.dollars,sowe’llnameourinstanceeur_to_usd.Thenwecaninvokethefunctionbyusingthatinstance:Clickheretoviewcodeimage

intmain(){

convertereur_to_usd(1.3043);

doubleeuros;

do{

std::cout<<"EnteramountinEuros:";

std::cin>>euros;

std::cout<<euros<<"eurosis"

<<eur_to_usd(euros)<<"dollars"

<<std::endl;

}while(euros>0.0);

}

Functionobjectshavethebenefitthatwecanpassthemasargumentstofunctions.(C++doesnotallowpassingfunctionsdirectly,onlyfunctionpointers,whichrequirestheaddedcostofanindirectfunctioncall.)Inaddition,functionobjectscancontainstateinformation.

C.5Preconditions,Postconditions,andAssertionsGivenvalidarguments,afunctionperformsacertaincomputation.Anotherwaytoputthisisthatifitspreconditionsaresatisfied,certainpostconditionswillbetrue.Sometimeswewritethesepreconditionsandpostconditionsascommentsinourcode,likethis:Clickheretoviewcodeimage

//precondition:y!=0.0

doublemy_ratio(doublex,doubley){

returnx/y;

}

//postcondition:returnedvalueisx/y

However,thelibraryalsoprovidesamechanismcalledassertforcheckingsomeconditions.Sowecouldwrite:Clickheretoviewcodeimage

doublemy_ratio(doublex,doubley){

assert(y!=0.0);

returnx/y;

}

Iftheassertexpressionevaluatestotrue,nothinghappens.Butifitevaluatestofalse,executionoftheprogramhaltsandanerrormessageisprinted.Inproductioncode,assertionsaretypicallydisabledtoavoidaperformancepenalty.

C.6STLAlgorithmsandDataStructuresTheC++languagecontainsalibraryofstandardsoftwarecomponents,knownastheStandardTemplateLibrary(STL).Thislibraryincludesdatastructures,algorithms,andotherutilitiescommonlyusedbyC++programmers.AllSTLcomponentsbelongtothenamespacestd;wewillexplicitlyusetheprefixstd::whenwerefertotheminourcodeexamples.STLisagenericlibrary,meaningthateachcomponentcanbeusedwithanyappropriatetype.Inthe

caseofdatastructures,thetypesarespecifiedastemplateargumentswhentheobjectisdeclared.Forexample,

std::vector<int>v;

declaresavectorofintegers,whilestd::vector<bool>v;

declaresavectorofBooleans.ThefollowingSTLcomponentsareusedinthisbook:

Functionobjectsforarithmeticoperationsandcomparisons(seeSectionC.4forexplanationoffunctionobjects):

•std::plus—Computesthesumofitsoperator’stwoarguments.•std::multiplies—Computestheproductofitsoperator’stwoarguments.•std::negate—Computesthenegationofitsoperator’sargument.•std::less—Returnstruewhenitsoperator’sfirstargumentislessthanitssecond,falseotherwise.•std:less_equal—Returnstruewhenitsoperator’sfirstargumentislessthanorequaltoitssecond,falseotherwise.

Datastructures:•std::pair—Astructthatstorestwoarbitraryobjects;typicallyusedtoreturntwothingsfromafunction.•std::vector—Acontainerforasequenceofelementsofasingletypethatsupportsconstant-timerandomaccess.

Algorithms:•std::fill—Fillstherangespecifiedbyitsfirsttwoargumentswiththevaluespecifiedbythethirdargument.•std::swap—Exchangesthecontentsofitsarguments.•std::partition_point—Returnsaniteratortothefirstelementinanalreadypartitionedrange(specifiedbythefirsttwoarguments)forwhichthegivenpredicate(thirdargument)isnottrue.SeediscussioninSection10.8.

Otherutilities:•std::advance—Incrementsthepositionofaniterator(itsfirstargument)byadistance(secondargument).

ForamoredetaileddescriptionoftheseandotherSTLcomponents,seePartIVofStroustrup’sTheC++ProgrammingLanguage.

C.7IteratorsandRangesIteratorsareanimportantpartofgenericprogramming,andwediscussthemingreaterdetailinSection10.4.Thefollowingdiscussionisintendedtobeaquickreference.Iteratorsareanabstractionofpointers;aniteratorindicatesapositioninasequence.Theexamplesin

thisbookusethesefourtypesofiterators,eachwithitsowniteratortag.•Inputiteratorssupportone-directionaltraversal,butonlyonce,asisfoundinsingle-passalgorithms.Thecanonicalmodelofaninputiteratoristhepositioninaninputstream.Tag:std::input_iterator_tag•Forwarditeratorsalsosupportonlyone-directionaltraversal,butthistraversalcanberepeatedasneeded,asinmulti-passalgorithms.Thecanonicalmodelofaforwarditeratoristhepositioninasinglylinkedlist.Tag:std::forward_iterator_tag•Bidirectionaliteratorssupportbidirectionaltraversal,repeatedasneeded(i.e.,theyalsocanbeusedinmulti-passalgorithms).Thecanonicalmodelofabidirectionaliteratoristhepositioninadoublylinkedlist.Tag:std::bidirectional_iterator_tag•Random-accessiteratorssupportrandom-accessalgorithms;thatis,theyallowaccesstoanyelementinconstanttime.Thecanonicalmodelisthepositioninanarray.Tag:std::random_access_iterator_tag

Theiteratortagsarespecialtypesthatmaybeusedinfunctionsignaturestoensurethatthecorrectversionofanoverloadedfunctionwillbeinvokedwhenagiveniteratorisused;seeChapter11foranexample.

***

STLfunctionsoftentaketwoiteratorsrepresentingthebeginningandendofarangeofdata.Byconvention,theiteratorfortheendofthedata(oftencalledlast)pointstothepositiondirectlyafterthelastelement.Iteratorsalsohavespecialattributescallediteratortraits.Theonesweuseare:•value_type:thetypeoftheobjectspointedtobytheiterator.

•difference_type:anintegraltypelargeenoughtoexpressthenumberofincrementoperationsneededtogetfromoneiteratortoanother.•iterator_category:theiteratortag,describedearlier.

Thesyntaxtoaccessaniteratortraitforaparticulariteratortypexis(forexample)std::iterator_traits<x>::value_type

Formoreinformationoniterators,seeChapter10.

C.8TypeAliasesandTypeFunctionswithusinginC++11C++11,thecurrentstandardversionofC++,hasafeaturecalledusingthatallowsprogrammerstoprovidealiasesfortypesandotherconstructs.Thisistypicallyusedtoprovideashortwaytowritealongandcomplicatedtype.Forexample:Clickheretoviewcodeimage

usingmyptr=long_complicated_class_name*;

Afterwritingthisstatement,programmerscouldrefertomyptrinthecodewherevertheywouldhavepreviouslywrittenlong_complicated_class_name*.UsersofCandearlierversionsofC++maybefamiliarwithanolderaliasingmechanism,typedef,

butusingismoreflexible.Forexample,theusingfeatureallowsustowritetemplatizedtypefunctionsforiteratortraits.IfwewriteClickheretoviewcodeimage


usingIteratorCategory=

typenamestd::iterator_traits::iterator_category;

theneverytimewewanttoknowthecategoryofaniterator,wecansayIteratorCategory

ratherthanClickheretoviewcodeimage

std::iterator_traits::iterator_category

C.9InitializerListsinC++11InCandC++,youcanconvenientlyinitializeanarraybyenclosingthelistofvaluesincurlybraces,likethis:Clickheretoviewcodeimage

charmy_array[5]={'a','e','i','o','u'};

C++11extendsthissyntaxbeyondarrays,sonowyoucanalsowritethingslikethis:Clickheretoviewcodeimage

std::vector<char>={'a','e','i','o','u'};

std::pair<int,double>={42,3.1415};

C.10LambdaFunctionsinC++11C++11includessupportforlambdafunctions.Theseareanonymousfunctionsthatareneededonlyonce,oftenasargumentstoanotherfunction.Supposeforsomeapplication,wewantedtotakeafunctionthatcomputedthecubeofitsargumentand

passittoanotherfunction.Traditionally,we’dhavetoimplementthisasafunctionobject,declareitseparately,instantiateit,andpasstheinstance,likethis:Clickheretoviewcodeimage

structcuber{

cuber(){};//constructor

intoperator()(intx){

returnx*x*x;

}

};

intmain(){

...

cubercube;

inta=some_other_function(cube);

...

}

Butifthisistheonlytimeweeverneedthecubefunction,that’salotofwork.Whybothercreatingthefunctionobject,orevengivingthefunctionaname,whenwe’renevergoingtouseitagain?Instead,wecanwritealambdafunctioninlineandpassthewholeexpressionasanargument:Clickheretoviewcodeimage

intmain(){

...

inta=some_other_function([=](intx){returnx*x*x;});

...

}

Thesyntaxforlambdafunctionsisjustlikethesyntaxforimplementinganyotherfunction,exceptthatthenameofthefunctionisreplacedbytheexpression[=],andthereturntypeusuallydoesnotneedtobespecified;thecompilerfiguresitout.

C.11ANoteaboutinlineTheC++directiveinlinebeforeafunctionisahintthattellsthecompilerthattheprogrammerwouldlikethebodyofthefunctiontobeincludedaspartofthecodeofthecaller,avoidingtheusualfunctioncalloverhead.Inpractice,manyfunctionsinthisbookwouldbenefittodayfrombeingdeclaredinline.Inliningmakessenseonlyforrelativelysmallpiecesofcode.Largerinlinedfunctionscouldendup

increasingthesizeofthecallingcodeenoughtodisruptcodecachingorcauseotherperformanceproblems.Compilerstakethisintoaccountandignoretheinlinerequestinthosecases.Atthesametime,compilersaregettingsmartenoughtoinlinecodeautomaticallywhenitmakessense.Forthesereasons,theinlinedirectivewillsoonbeobsolete,sowedidnotuseitinourexamples.

Bibliography

Adler,MortimerJ.(Ed.).(1991).GreatBooksoftheWesternWorld,Vol.10:Euclid,Archimedes,Nicomachus.Chicago:EncyclopaediaBrittanica.

Aristotle.(1938).Aristotle:Categories,OnInterpretation,PriorAnalytics,Vol.325.TranslatedbyH.P.CookeandHughTredennick.Cambridge,MA:LoebClassicalLibrary.

Aristotle.(1984).TheCompleteWorksofAristotle:TheRevisedOxfordTranslation.EditedbyJonathanBarnes.Princeton,NJ:PrincetonUniversityPress.

Backus,John.(1978).“CanProgrammingBeLiberatedfromtheVonNeumannStyle?:AFunctionalStyleandItsAlgebraofPrograms.”CommunicationsoftheACM21(8),613-641.

Ball,W.W.Rouse,andH.S.M.Coxeter.([1922]2010).MathematicalRecreationsandEssays(10thed.).Reprint,NewYork:DoverPublications.Originaleditionpublished1892.

Bonola,Roberto.([1955]2010).Non-EuclideanGeometry:ACriticalandHistoricalStudyofItsDevelopment.TranslatedbyH.S.Carslaw.Reprint,NewYork:DoverPublications.OriginallypublishedasLaGeometrianon-Euclidea,1912.

Burnside,William.([1911]2004).TheoryofGroupsofFiniteOrder(2nded.).Reprint,Mineola,NY:DoverPublications.

Byrne,Oliver.(2010).TheFirstSixBooksoftheElementsofEuclid.Taschen.Facsimileof1847edition.

Chrystal,George.([1964]1999).Algebra:AnElementaryText-Book(7thed.).Reprint,Providence,RI:AmericanMathematicalSociety.Originaleditionpublished1886.

Cohen,MorrisR.,andI.E.Drabkin.(1948).ASourceBookinGreekScience.Cambridge,MA:HarvardUniversityPress.

Corry,Leo.(2004).ModernAlgebraandtheRiseofMathematicalStructures(2ndreviseded.).Basel,Switzerland:Birkhäuser.

Dean,Jeffrey,andSanjayGhemawat.(2008).“MapReduce:SimplifiedDataProcessingonLargeClusters.”CommunicationsoftheACM51(1),107-113.

Dedekind,Richard.(1996).TheoryofAlgebraicIntegers.TranslatedbyJohnStillwell.Cambridge,UK:CambridgeUniversityPress.OriginallypublishedasÜberdieTheoriederganzenalgebraicschenZahlen,1877.

DeMillo,RichardA.,RichardJ.Lipton,andAlanJ.Perlis.(1979).“SocialProcessesandProofsofTheoremsandPrograms.”CommunicationsoftheACM22(5),271-280.

Dirichlet,P.G.L.(1999).LecturesonNumberTheory.SupplementsbyR.Dedekind.TranslatedbyJohnStillwell.Providence,RI:AmericanMathematicalSociety.OriginallypublishedasVorlesungenüberZahlentheorie,1863.

Euclid.(1956).Euclid:TheThirteenBooksoftheElements.TranslatedbyThomasL.Heath.(2nded.).NewYork:DoverPublications.

Euler,Leonhard.(1988).IntroductiontoAnalysisoftheInfinite,Vol.1and2.TranslatedbyJohnD.Blanton.NewYork:Springer.OriginallypublishedasIntroductioinanalysininfinitorum,1748.

Euler,Leonhard.(2000).FoundationsofDifferentialCalculus.TranslatedbyJohnD.Blanton.New

York:Springer.OriginallypublishedasInstitutionesCalculiDifferentialis,1755.Fibonacci,LeonardoPisano,andL.E.Sigler(Trans.).(1987).TheBookofSquares:AnAnnotatedTranslationintoModernEnglish.Boston:AcademicPress.OriginallypublishedinLatinasLiberQuadratorum,1225.

Floyd,RobertW.,andDonaldE.Knuth.(1990).“AdditionMachines.”SIAMJournalonComputing19(2),329-340.

Fowler,DavidH.(1987).TheMathematicsofPlato’sAcademy:ANewReconstruction.Oxford,UK:ClarendonPress.

Gauss,CarlFriedrich.(1965).DisquisitionesArithmeticae.TranslatedbyArthurA.Clarke,S.J.NewHaven,CT:YaleUniversityPress.OriginalLatinedition,1801.

Granville,Andrew.(2005).“ItIsEasytoDeterminewhetheraGivenIntegerIsPrime.”BulletinoftheAmericanMathematicalSociety42(1),3-38.

Gries,David,andGaryLevin.(1980).“ComputingFibonacciNumbers(andSimilarlyDefinedFunctions)inLogTime.”InformationProcessingLetters11(2),68-69.

Grimm,RichardE.(1973).“TheAutobiographyofLeonardoPisano.”FibonacciQuarterly11(1),99-104.

Hartshorne,Robin.(2000).Geometry:EuclidandBeyond.NewYork:Springer.Heath,Thomas.([1921]1981).AHistoryofGreekMathematics.Reprint,NewYork:DoverPublications.

Hilbert,David.([1971]1999).FoundationsofGeometry(10thed.).TranslatedbyLeoUngerandrevisedbyPaulBernays.Chicago:OpenCourt.OriginallypublishedasGrundlagenderGeometrie,1899.

Iverson,KennethE.(1962).“AProgrammingLanguage.”InProceedingsoftheMay1-3,1962,SpringJointComputerConference,AIEE-IRE’62,pp.345–351.ACM.

Iverson,KennethE.(1980).“NotationAsaToolofThought.”CommunicationsoftheACM35(1-2),2-31.

Kahn,David.(1996).TheCodebreakers:TheComprehensiveHistoryofSecretCommunicationfromAncientTimestotheInternet(Reviseded.).NewYork:Scribner.

Kapur,D.,D.R.Musser,andA.A.Stepanov.(1981a).“OperatorsandAlgebraicStructures.”InProceedingsofthe1981ConferenceonFunctionalProgrammingLanguagesandComputerArchitecture,FPCA’81,NewYork,NY,pp.59-64.ACM.

Kapur,D.,D.R.Musser,andA.A.Stepanov.(1981b).“Tecton:ALanguageforManipulatingGenericObjects.”InProgramSpecification,ProceedingsofaWorkshop,pp.402-414.Springer-Verlag.

Katz,Jonathan,andYehudaLindell.(2008).IntroductiontoModernCryptography.BocaRaton,FL:CRCPress.

Katz,VictorJ.(2009).AHistoryofMathematics:AnIntroduction(3rded.).Boston:Addison-Wesley.Keisler,H.Jerome.(1989).“FundamentalsofModelTheory.”InJ.Barwise(Ed.),HandbookofMathematicalLogic.NorthHolland.

Kennedy,Hubert.(2002).TwelveArticlesonGiuseppePeano.SanFrancisco:PeremptoryPublications.Knuth,DonaldE.(2007).TheArtofComputerProgramming,Vol.2:SeminumericalAlgorithms.

Boston:Addison-Wesley.Landau,Edmund.([1966]2001).FoundationsofAnalysis(3rded.).TranslatedbyF.Steinhardt.Reprint,Providence,RI:Chelsea.

McClenon,R.B.(1919).“LeonardoofPisaandHisLiberQuadratorum.”TheAmericanMathematicalMonthly26(1),1-8.

Musser,DavidR.,andAlexanderA.Stepanov.(1988).“GenericProgramming.”InProceedingsoftheInternationalSymposiumISSAC’88onSymbolicandAlgebraicComputation,pp.13-25.Springer-Verlag.

Peano,Giuseppe.(1960).FormularioMathematico.EdizioniCremonense.Originaleditionpublished1908.

Plato.(1997).Plato:CompleteWorks.EditedbyJ.M.CooperandD.S.Hutchinson.Indianapolis,IN:HackettPublishing.

Robins,Gay,andCharlesShute.(1987).TheRhindMathematicalPapyrus:AnAncientEgyptianText.London:BritishMuseumPublications.

Ross,David.(2004).Aristotle(6thed.).London:Routledge.Russo,Lucio.(2004).TheForgottenRevolution:HowScienceWasBornin300BCandWhyItHadtoBeReborn.TranslatedbySilvioLevy.NewYork:Springer.OriginallypublishedasLarivoluzionedimenticata,1996.

Sarton,George.(1934).“SimonStevinofBruges.”Isis21(2),241-303.Sigler,Laurence.(1987).Fibonacci’sLiberAbaci:ATranslationintoModernEnglishofLeonardoPisano’sBookofCalculation.NewYork:Springer.OriginalLatinedition,1202.

Stein,Josef.(1967).“ComputationalProblemsAssociatedwithRacahAlgebra.”JournalofComputationalPhysics1(3),397-405.

Stepanov,Alexander,andMengLee.(1995).TheStandardTemplateLibrary.Hewlett-PackardLaboratories,TechnicalPublicationsDepartment.

Stepanov,Alexander,andPaulMcJones.(2009).ElementsofProgramming.Boston:Addison-WesleyProfessional.

Stillwell,John.(1994).ElementsofAlgebra.NewYork:Springer.Stillwell,John.(2002).ElementsofNumberTheory.NewYork:Springer.Stillwell,John.(2010).MathematicsandItsHistory.NewYork:Springer.Stroustrup,Bjarne.(2013a).TheC++ProgrammingLanguage(4thed.).Boston:Addison-WesleyProfessional.

Stroustrup,Bjarne.(2013b).ATourofC++.Boston:Addison-WesleyProfessional.vanderWaerden,B.L.(1983).GeometryandAlgebrainAncientCivilizations.Berlin:Springer-Verlag.

vanderWaerden,B.L.(1988).ScienceAwakening:Egyptian,Babylonian,andGreekMathematics.TranslatedbyArnoldDresden.Dordrecht,Netherlands:KluwerAcademicPublishers.

Weil,André.(2007).NumberTheory:AnApproachthroughHistoryfromHammurapitoLegendre.Cambridge,MA:BirkhäuserBoston.

Index

*(operator),mathematicalconventionfor,115+(plussign),mathematicalconventionfor,115α.SeeAliquotsumϕ.SeeEulertotientfunctionσ(sumofdivisors)formula,31

AAbeliangroup,86,108,153Abstractalgebra

birthof,85,140–145Euclideandomains,150–151,153fields,151–153groups,85–88,92–95,108,152ideals,226–228modules,151,154monoids,89,108–109,152,154andprogramming,2,141,249principalideals,227–228rings,142–145,153semigroups,90–91,108–109,152semirings,145–149,153vectorspaces,152,154

AbstractionAristotle,177,180,196inmathematics,84,85–109andprogramming,2,5,249

Academy(Plato’s),41–44,178Addition

associativityof,9,156,174commutativityof,155–156,174–175definition,173

Additionchains,11Additivegroups,86Additivemonoids,89,109Additivesemigroups,90,109Address,181Adleman,Len,239advance,190Agrawal,Manindra,244

Ahmes,8–9,57Ahmesalgorithm.SeeEgyptianmultiplication,EgyptiandivisionAKSprimalitytest,244–245AlexandertheGreat,43,178–179Alexandria,43–44Algebraicintegers,140Algebraicstructures,85.SeealsoAbstractalgebraAlgorithms

inancientEgypt,7–11definition,7domainorsetting,150firstrecorded,8generalizing,111,119–123,126–127,151historyof,7–11in-place,215–216memoryadaptive,216–217polylogspace,215–216spacecomplexity,215–216performanceinpractice,211

Aliases,272Aliquotsum,31Amicablenumbers,63–64AnalyticalMechanics,99APL,124Apology,43Archimedes

onacquiringmathematicalknowledge,176axiomof,47placeinhistory,50

Aristophanes,42Aristotle,17,177–180Aristoxenus,17Arithmétique,132TheArtofComputerProgramming,9Aryabhata,51Aryabhatiya,51Assertions,269Associativebinaryoperation( ),108

ingroups,85–86inmonoids,89insemigroups,90

Associativityaxiom,semigroups,91

Associativityofaddition,9,113definition,174visualproof,156

Associativityofmultiplication,visualproof,157Asymmetrickeys,238Athens,41–43Automorphism,104Averroes.SeeIbnRushdAxiomofArchimedes,47Axiomaticmethod,161–162Axioms

definition,163Euclid’s,162–163Hilbert’s,167Peano’s,170–171

BBachetdeMéziriac,ClaudeGaspar,67,235

profile,225–226Backus,John,124Bacon,Roger,1,249Bartels,Martin,165Bernoulli,Johann,69Bézout’sidentity,225–229Bidirectionaliterators,185Binarysearch,191–196.SeealsoPartitionpointsBinarysearchlemma,194–195BletchleyPark,238Bolyai,Farkas,166Bolyai,János,166Bolzano-CauchyTheorem.SeeIVT(IntermediateValueTheorem)Booleansemirings,148Boundedranges,189,203–204bsearch,192

CC++,3,265–273C++11,57,187,195,265,272–273TheC++ProgrammingLanguage,265,270C++StandardTemplateLibrary.SeeSTLCaesarcipher,237Cancellation

CancellationLaw,74–75definition,72–73inversenumbers,73andmodulararithmetic,72–76Self-CancelingLaw,75–76

CancellationLaw,74–75Carmichaelnumbers,242Cartesiancoordinates,131,138Cataldi,Peter,64Categoricaltheories

vs.STL,104definition,104examplesof,104–106

Categorydispatch,188,190,196,213Cayley’sTheorem,198Chinesemathematics,51Chrystal,George,34Cicero,50Ciphertext,238Closedranges,188Clouds,42Cocks,Clifford,240Codes,definition,237CogitataPhysicoMathematica,64Colossusmachine,238CommonLisp,116,124,190Commonmeasureofsegments,33Commonnotions,Euclid’saxiomaticmethod,162Commutativealgebra,rings,143–144Commutativityofaddition,155–156,174–175Commutativityofmultiplication,visualproof,156Commutativityofpowers,semigroups,91Compile-timedispatch.SeeCategorydispatchCompleteness,lawof,203–204Completeness,theories,102Complexnumbers,137–138Compositenumbers.SeealsoPrimenumbers

definition,21distinguishingfromprime,240–245

Conceptsandabstractalgebra,141definition,181

choosing,250examples,116–117,181namingconventions,183overview,181–184,266–267Regular,183–184requirementsontypes,24,182Semiregular,184typeattributes,182–183typefunctions,182–183

Consistency,theories,102,104Constructivists,229Contradiction,proofby,35,261–262Contrapositive,259Coprime,31,78,80–81,246–247Cosets,97.SeealsoLagrange’sTheoremCountedranges,189,203–204Cryptanalysis,237–238Cryptography,233–234,237Cryptology

asymmetrickeys,238BletchleyPark,238Caesarcipher,237ciphertext,238codes,definition,237Colossusmachine,238cryptanalysis,237–238cryptography,237cryptosystems,238Enigmamachine,238keys,238Lorenzmachine,238plaintext,238public-keycryptosystems,239–240RSAalgorithm,239–240,245–247symmetrickeys,238trapdoorone-wayfunctions,239

Cryptosystems,238Cycles,ofpermutations,200,207–211Cyclicgroups,96,109

generator,96Cyclicsubgroups,96

DDatum,180Decimalfractions,129–131Declarationsyntax,267Dedekind,Richard,140,171Degreeofpolynomials,133Dereferencing,iterators,184–185Descartes,René,64,131Differenceofpowersformula,30Differencetype,iterators,187difference_typeiteratortrait,187DifferentialCalculus,70Diffie,Whitfield,239Diophantus,67,225Dirichlet,PeterGustavLejeune,41,139–140,156Dirichletprinciple.SeePigeonholeprincipleDisme:TheArtofTenths,129–131DisquisitionesArithmeticae(“InvestigationsofArithmetic”),136–137distance,186-188divides,240Dividingpolynomials,133Domainofalgorithm,150Domainofdefinition,113Doublylinkedlists,185

EEgyptiandivision,57Egyptianmultiplication,8–11

requirements,111–118generalizingtopower,120

Elements(ofEuclid),2,21,43–45,161–163Proposition[VII,30],70Proposition[VII,32],21Proposition[IX,36],29,31–32Proposition[X,2],45Proposition[X,3],45–46Proposition[X,117],37

ElementsofProgramming,3,113–114,183,185,208Enigmamachine,238Equationalreasoning,114Equivalence,114

Eratosthenes,22Euclid.SeealsoElements

theaxiomaticmethod,161–163GCM(greatestcommonmeasure)algorithm,45–49incommensurablequantities,45–49onnumbertheory,21profile,44–45

Euclideandomains(ED),150–151,153Euclideangeometry

alternativesto,164–167fifthpostulate,163–164vs.hyperbolicgeometry,164–167vs.non-Euclidean,166–167

Euclideanrings.SeeEuclideandomains(ED)Euclid’salgorithm,45–47Euler,Leonhard,84,85

Euler’stheorem,79–83andLagrange,99perfectnumbers,32,63–64primenumbers,63,68profile,69–70

Eulertotientfunction,80,245Euler’sTheorem,79–83,246

proofusingLagrange’sTheorem,101Evenandoddnumbers,9–10,117

inGCD,219–220,224,234Existenceofzeroaxiom,172ExtendedGCDalgorithm,229–235,245,247extended_gcd,233

FFast-multiplicationalgorithm.SeeEgyptianmultiplicationFermat,Pierrede,63,65–69

profileof,67–68proofs,65–66,68

Fermatprimes,63–68,137Fermat’sLastTheorem,67Fermat’sLittleTheorem

converseof,77–79description,69non-invertibilitylemma,79proofbyLagrange’sTheorem,101

proof,77testingforprimenumbers,241–242restatementusingmodulararithmetic,84

Fermattest,241–242fermat_test,242

Fibonacci.SeeLeonardoPisanoFibonaccinumbers,computing,124–127

Fibonaccisequence,58–59Fields

characteristicof,151definition,151,153extensions,151prime,151,154

FifthpostulateofEuclideangeometry,163–164Figuratenumbers

gnomons,20oblongnumbers,19overview,17,19–20triangularnumbers,19squarenumbers,20

find_if,190–191find_if_n,191Finiteaxiomatizabilityoftheories,102Flowers,Tommy,238Floyd,Robert,58Formalistphilosophyofmathematics,167–169FormularioMathematico,170–172Forwarditerators,185FP,124Functionobjects,123–124,268,270Functors.SeeFunctionobjects

GGalois,Évariste

discoveryofgroups,85–88profileof,88–89

Gauss,CarlFriedrich,31,72,136–140,166,240profileof,136–137

Gaussianintegers,138–139,224GCD(greatestcommondivisor)

applicationsof,234ofpolynomials,134

description,59computing,59Euclid’salgorithm,45–46extendedGCD,229–235,245,247historicalmilestones,222andrationalarithmetic,234andringstructures,225–229rotationalgorithms,234Stein’salgorithm,219–225symbolicintegration,234validating,59–60

gcd,150,230GCM(greatestcommonmeasure)33,41

Euclid’salgorithm,45–49properties,33

Generatorelementsinsubgroups,96Genericprogramming

inC++,265–266,270concepts,181essence,127,249–250history,124,134,141,180andmathematics,84overview,1–2,5

get_temporary_buffer,217Gnomons,20Gödel,Kurt,169Göttingen,Universityof

CarlGauss,136–140DavidHilbert,168–169EmmyNoether,140–145profile,135–136

Granville,Andrew,244Grassman,Hermann,171Greatestcommondivisor(GCD).SeeGCD(greatestcommondivisor)Greatestcommonmeasure(GCM).SeeGCM(greatestcommonmeasure)Gries,David,205Gries-Millsalgorithm,204–208Groups

abelian,86,108,153additive,86binaryoperations,86cyclic,96,109

definition,85discoveryof,85examplesof,86–88identityelements,86inverseoperations,86Kleingroup,106orderofelements,94summarydescription,108,152symmetric,198theoremsabout,92–95

Hhalf,118Heath,Thomas,9,45Hegel,G.W.F.,111Hellman,Martin,239Hilbert,David,141,167–169,229

profile,168–169Hilbertspaces,168–169Hilbert’sproblems,169Hilbert’sprogram,169HistoryofAlgebra,129Horner’srule,132Hyperbolicgeometry,164–167

IIbnRushd,180Ideals.SeealsoRings

definition,226idealsinEuclideandomainslemma,227linearcombinationideallemma,227PID(principalidealdomains),228principalideals,227–228principalelements,227

IdealsinEuclideandomainslemma,227Identityelement,108–109,121

ingroups,86inmonoids,89inrings,143

identity_element,123,241Immutableobjects,181Impossibilityofinfinitedescent,21

Inclusion-exclusionprinciple,82–83Incommensurablequantities,45–49Independence,theories,102Indianmathematics,51Induction,proofby,262–263Inductionaxiom,21,170,172–173Inman,BobbyRay,240Innerproductoftwovectors,145–146In-placealgorithms,215–216Inputiterators,185IntegralCalculus,70Integraldomains,145,153Interfacerefinement,lawof,215Interfaces,designing,215Interlingua,171IntermediateValueTheorem(IVT),131,192IntroductiontoAnalysisoftheInfinite,70IntroductiontoArithmetic,10,19Intuitionistphilosophyofmathematics,229Inversenumbers,73inverse_operation,123Inverseoperation,86,119,121

ingroups,86Invertibilitylemma,229Invertibilityofsuccessoraxiom,173Invertibleelements.SeeUnitsIrrationalnumbers,38–39is_prime,241Isomorphism,models,103–104Iteratorcategories

bidirectional,185forward,185input,185output,186random-access,185

Iteratortraits,187iterator_categoryiteratortrait,187Iterators

inarrays,185bidirectional,185definition,184dereferencing,184–185

differencetype,187findingthedistancebetween,186–187forward,185input,185innoncontiguousdatasegments,186linked,186output,186overview,184–185randomaccess,185segmented,186successors,184

Iverson,Kenneth,124IVT(IntermediateValueTheorem),131,192

JJefferson,Thomas,44,130

KKapur,Deepak,124Kayal,Neeraj,244Keys,cryptography,238Khayyam,Omar,164Kleene,Stephen,115–116Klein,Felix,106–107,141Kleingroup,106–107Knuth,DonaldE.,9,58,197Kovalevskaya,Sofia,141

LLagrange,Joseph-Louis,99–100,192Lagrange’sTheorem,97–99,100–101Lambdaexpressions,195,272–273Laplace,Pierre-Simon,70largest_doubling,54LatinesineFlexione,171Lawofcompleteness,203–204Lawofinterfacerefinement,215Lawofseparatingtypes,202–203Lawofusefulreturn,57–58,201–202,213LecturesonNumberTheory(VorlesungenüberZahlentheorie),140Legendre,Adrien-Marie,155Lehmer,D.H.,192

LeonardoPisanoFibonaccisequence,58–59introductionofzero,52profile,52–53

LetterstoaGermanPrincess,70LiberAbaci,52LiberQuadratorum,52LibraryofAlexandra,43Lincoln,Abraham,44Linearalgebra

innerproduct,145–146matrix-matrixproduct,146matrix-vectorproduct,146review,145–147

Linearcombinationideallemma,227Linearrecurrencefunctions,127Linearrecurrencesequences,127Linearsearch,190–191Linkediterators,186Liu,Hui,51Lobachevsky,Nikolai,164–166Lorenzmachine,238lower_bound,195–196Lyceum,179

MMagmas,91,108mark_sieve,24Mathnotationinthisbook,257–259Matrixmultiplication,145–147Matrix-matrixproduct,146Matrix-vectorproduct,146Mauchly,John,192McJones,Paul,3Measureofasegment,33Memory-adaptivealgorithms,216–217Meno,43Mersenne,Marin,64–65Mersenneprimes,63–68Metaphysics,179Miller-Rabintest,243–245miller_rabin_test,243

Mills,Harlan,205Models.SeealsoTheories

definition,103isomorphism,103–104

ModernAlgebra,142Modulararithmetic,72–74,83–84

Fermat’sLittleTheorem,83–84Wilson’sTheorem,83

Modules,definition,151,154modulo_multiply,241Monoids.SeealsoGroups

additive,89,109,154definition,89examplesof,89multiplicative,89,154summarydescription,108,152

Mouseion,43Multiplication

definition,8,173–174Egyptian,8–11RussianPeasantAlgorithm.9

Multiplicativefunctions,31multiplicative_inverse,121,247multiplicative_inverse_fermat,241Multiplicativemonoids,89Multiplicativesemigroups,90Multiply-accumulatefunction,11–14Musser,DavidR.,124Mutableobjects,181

NNamingconventions,concepts,183Namingprinciple,115–116Naturalnumbers,147,170,172,175,258NicomacheanEthics,179NicomachusofGerasa,10,19NineChaptersontheMathematicalArt,51Noether,Emmy,129,140–145

profile,141–142Non-categoricaltheories,106–107Noncommutativeadditivemonoids,119Noncommutativeadditivesemigroups,115

Noncommutativealgebra,rings,143–144Nonconstructiveproofs,229Noncontiguousdatasegments,iterators,186Non-Euclideangeometry,164–167Non-invertibilitylemma,79Notationinthisbook,257–259Numberline,131Numberofassignmentstheorem,200–201Numbersystems,ancientEgypt,8Numbertheory2,41,43

inancientGreece,17–39Bezout’sidentity,225–229Euler’sTheorem,79–83,101Fermat’sLittleTheorem69–78,101figuratenumbers,17–20,33Gauss,136–137andGCD,140LiberQuadratorum,53modulararithmetic,72–74perfectnumbers,28–32primalitytesting240–245primenumbers,21–2817thand18thcentury,63–72,74–84sieveofEratosthenes,22–23Wilson’sTheorem,76,83

OObjecttypes,definition,181Objects

definition,180immutable,181mutable,181remoteparts,181unrestricted,181

Oblongnumbers,19Octonions,151odd,118Oddnumbers.SeeEvenandoddnumbersOne-to-onecorrespondence,92Openranges,188“OperatorsandAlgebraicStructures,”124Orderofgroupelements,94

Organon,180Outputiterators,186

PPalindromicprimes,28Parallelpostulate.SeeFifthpostulateofEuclideangeometryPartitionpoints,193partition_point,194partition_point_n,193Peano,Giuseppe,169–175

profile,171–172Peanoarithmetic,170–171,173–175Peanoaxioms,170–173Peanocurve,171Perfectnumbers

inancientGreece,28–32,38definition,28–29mathematicians’interestin,63

Permutationofremainderslemma,71–72Permutations,197–201Phaedo,43PhiloofAlexandria,7PID(principalidealdomains),228Pigeonholeprinciple,95,263Pisano,LeonardoSeeLeonardoPisanoPlaintext,238Plato,41–43,177–179

profile,42–43PlatonicQuestions,20Platonicsolids,41,44Playfair’saxiom,163Plussign(+),mathematicalconventionfor,115Plutarch,20Poincaré,JulesHenri,85,229–230,248

profile,229–230pointeriteratortrait,187Politics,179Polylogspace,215–216Polynomials

computingGCDfor,134degreeof,definition,133divisionwithremainder,133

historyof,132–135Horner’srule,132treatingasnumbers,133–135

polynomial_value,132Populationcount,10Postconditions,269Postulates,Euclid’saxiomaticmethod,162,163Poweralgorithm,119–123,249

computingFibonaccinumbers,126computinglinearrecurrence,127useincryptology,241–243,246useingraphapplications,148–149

power_accumulate_semigroup,121power_group,123power_monoid,122power_semigroup,122Primalitytesting,240–245Primefactorization,29,31–32,65,136,139–140Primefields,151,154Primenumbers

inancientGreece,21–28definition,21distinguishingfromcomposite,240–245Fermatprimes,63–68finding.SeesieveofEratosthenesinfinitenumberof,21Mersenneprimes,63–68primalitytesting,240–245

Principalelement,227Principalidealdomains(PID),228Principalideals,227–228ProblèmesPlaisants,225–226Proof

bycontradiction,35,261–262definition,158–159byinduction,262–263nonconstructive,229pigeonholeprinciple,95,263visual,155–159

Properdivisor,32Ptolemy,164Public-keycryptosystems,239–240

Pythagoras,17profile,18–19

Pythagoreanprogram,33–38PythagoreanTheorem,44Pythagoreantriples,50–51

QQuadrivium,18Quaternions,151Quotient,55–57,150,153,202

forpolynomials,133quotient_remainder,57

RRandom-accessiterators,185Ranges

bounded,189,203–204closed,188counted,189,203–204definition,188open,188overview,188–189partitionpoints,193semi-open,188swapping,201–204

Rationalarithmetic,GCDapplications,234Rationalnumbers,151,258Realnumbers,131,258reciprocal,124Recreationalmathematics,225–226Recursiveremainderlemma,48–49Reductionalgorithm,124referenceiteratortrait,187Regius,Hudalricus,64Regularconcepts,183–184Regularfunctions,183Regulartypes,114Rejewski,Marian,238Remainder,47–49,53–55,57–59,150,153,222

Floyd-Knuthalgorithm,58permutationofremainders,71–72inmodulararithmetic,73–75

ofGaussianintegers,138–139ofpolynomials,133–134

remainder,54–55remainder_fibonacci,58Remotepartsofobjects,181Requirementsonalgorithm,111–119reverse,212–215reverse_copy,216reverse_n,214reverse_n_adaptive,217reverse_n_with_buffer,216reverse_recursive,214Reversepermutation,201,212–215Rewritingcode,14–15RhindMathematicalPapyrus,8,57Rings.SeealsoIdeals;Semirings

definition,142–143andtheGCD,225–229integraldomains,145summarydescription,153unitary,143units,144zerodivisors,145

Rivest,Ron,239Rotatealgorithms,204–213rotate,207,210,213rotate_cycle_from,208rotate_transform,210rotate_unguarded,206Rotation,204–207Rotationalgorithms,GCDapplications,234RSAalgorithm,239–240,245–247Russell,Bertrand,171RussianPeasantAlgorithm,9.SeealsoEgyptianmultiplication

SSaccheri,GiovanniGirolamo,164Saxena,Nitin,244Scheme,116TheSchoolofAthens,177–178Searches

binary,191–196linear,190–191

Segmentediterators,186Self-CancelingLaw,75–76Semanticrequirementsforgenericalgorithms,113Semigroups.SeealsoGroups

additive,90,109associativityaxiom,91commutativityofpowers,91definition,90,109examples,90multiplicationalgorithm,115multiplicative,90summarydescription,108,152

Semi-openranges,188Semiregularconcepts,184Semirings.SeealsoRings

Boolean,148description,145–147matrixmultiplication,146shortestpath,148–149summarydescription,153tracingsocialnetworks,147–148transitiveclosures,147–148tropical,149weak,147

Separatingtypes,lawof,202–203Settingofalgorithm,150Shamir,Adi,239Shortestpath,finding,148–149SieveofEratosthenes,22–23

implementation23–28sift,27smallest_divisor,240Socialnetworkconnections,tracing,147–148Socrates,42Socraticmethod,42Sophists,42Spacecomplexity,215–216Squarerootof2,anirrationalnumber,37–38StandardTemplateLibrary.SeeSTLStein,Josef,219–222

Stein’salgorithm,219–225stein_gcd,220Stepanov,AlexanderA.,3,124Stevin,Simon,129–135,192

profile,130–131STL(StandardTemplateLibrary)

algorithms,195–196,215,217applicationofgenericprogramming,1,186containers,190–191conventions,24non-categorical,104

Strengthreduction,26Stroustrup,Bjarne,265Subgroups.SeealsoGroups

cyclic,96definition,95,109generatorelements,96trivial,95

Successors,170,184Sumofoddpowersformula,30swap,199swap_ranges,201–203Symbolicintegration,GCDapplications,234Symmetricgroups,198Symmetrickeys,238Symposium,43Syntacticrequirementsforgenericalgorithms,113

TTail-recursivefunctions,12–14Templatefunctions,265–266ThalesofMiletus,18,159–161Thales’Theorem,160–161Theories.SeealsoModels

categorical,104–106characteristicsof,102completeness,102consistency,102,104definition,102determiningtruthof,167finiteaxiomatizability,102independence,102

non-categorical,106–107univalent,104

Totalityofsuccessoraxiom,172Totientofaninteger,80ATourofC++,265Transfiniteordinals,172–173Transformationgroup,92Transitiveclosures,finding,147–148Transpositionlemma,199Transpositions,197,199–201Trapdoorone-wayfunctions,239Triangularnumbers,19TrichotomyLaw,34Tripcount,204,213Trivialcycles,200,208Trivialsubgroups,95Tropicalsemirings,149Turing,Alan,169,238TusculanDisputations,50Typeattributes,182–183Typedispatch.SeeCategorydispatchTypefunctions,182–183

UUnitaryrings,143Units,rings,144Univalenttheories,104Univariatepolynomials.SeePolynomialsUniversityofGöttingen.SeeGöttingen,UniversityofUnreachablenumbers,172–173Unrestrictedobjects,181upper_bound,195Usefulreturn,lawof,57–58,201–202,213

VValuetypes,definition,180Values,definition,180value_typeiteratortrait,187vanderWaerden,Bartel,129,142Veblen,Oswald,104vectorcontainer,116,Vectorspace,152,154

Visualproofs,155–159VorlesungenüberZahlentheorie(LecturesonNumberTheory),140

WWaring,Edward,76Weaksemirings,147Weilert,Andre,224Well-orderingprinciple,34Whitehead,AlfredNorth,43Wiles,Andrew,67Wilson,John,76Wilson’sTheorem

description,76usingmodulararithmetic,83

Witnesses,primalitytesting,242

ZZero

inEgyptiannumbersystem,8introductionof,52originsof,51–53

Zerodivisors,rings,145,154

PhotoCredits

Page18:Vkara/Fotolia;42,44:AlexeyPavluts/Fotolia;52:Bettman/Corbis;64:Lisztcollection/Alamy;67:Marzolino/Shutterstock;69:GeorgiosKollidas/Fotolia;88:Bettman/Corbis;99:GeorgiosKollidas/Fotolia;130:BilwisseditionLtd.&Co.KG/Alamy;136:GeorgiosKollidas/Fotolia;141:Interfoto/Alamy;159:DesignPics/Newscom;165:RIANovosti/Alamy;168:AKGImages/Newscom;171:RobertGruberRose;178:WorldHistoryArchive/Alamy;178:Midosemsem/Fotolia;225:RMN-GrandPalais/ArtResource,NY;229:AKGImages/Newscom.

CodeSnippets

stepanov rose from.mathematics.to.generic.programming

Documents