6ThingsAboutProgrammingThatEveryComputerProgrammerShouldKnow

byVincentP.North

2015

Introduction:JustCallItACareersExperience,CondensedIntoSixKeyTopics

Bymannerofintroduction,Ihavebeenaprofessionalcomputerprogrammerand,softwareprojectmanagerforaboutthirty-fiveyears.Iwasfortunatetohavebegunthatcareerwhenthepersonalcomputerhadnotquiteyetarrivedonthescene.(Theywerestill,forthemostpart,atantalizingfuturecuriosity,thenfoundonlyinthepagesofRadio-ElectronicsandPopularElectronicsmagazines.)Computerseitherfilledair-conditionedroomsor,morerecently,wereaboutthesizeofabreadbox.Althoughintegratedcircuits,includingmicroprocessors,wereastandardpartofelectronicdesign,computersatthattimewereneithersmallnorfast.Computers,also,weredevicesthatotherpeopleowned:companieswhocouldaffordtobuytheboxesandtheair-conditionedroomstoputthemin.ItwasinthiscontextthatIfirstgotmystart.

Asyouwellknow,thesubsequentyearshavewitnesseddramaticchanges.MooresLaw1continuestoholdtrue,assemiconductormanufacturerscontinuetoastonishusallwithhowmuchmorecomputingpowertheycansomehowcramintoasingletinypieceofsand.Buildersofflat-screenandtouch-sensitivedisplaysalsocontinuetoastounduswithwhattheycando,ashavethebuildersofsound-chipsandnearly-microscopicdigitalcameras.Weliveininterestingtimesthatcertainlyshownosignofslackingoff.

Intothiswhirlingdervishofatechnologicalworld,then,comesyou,GentleReader.2Taskednotonlywithkeepingupwiththehardwaretechnologythatwillneverallowitselftofullybekept-upwith,butalsowiththetaskofcomputerprogrammingitself.

Ithasbeensaid,bysome,thatcomputerprogrammingisaninnateability.Thatyoueitherhaveit(whateveritis),oryoudont.But,Idontagree.Ineverthoughtofmyselfasbeinganaturalinwhatbecamemycareer.Iwasmerelynaturallyinterestedinit,asIstillam.Iwasinterestedenough,andfoundittobeengagingenough,thatIperseveredtolearnhowtodoit.ButImadealotofdumbmistakesalongthewaywhichIwouldliketotrytohelpyouavoid.

Mostofall,myexperiencehasshownmealistoffundamentaltechnicalskillsofaprofessionalprogrammer,whichIhavenowsoughttoreducetoasuccinctlistofsixitems.Theactuallistislongermaybe,muchlongerbutaftersomereflection,herearethesixthatinmyhumbleopinionarekey.Thesearethethingsallofthemtechnicalabilities,proceduresandperspectiveswhichyouwilluseonthejobmosteveryday.Thesearesixthingsforwhich,inmyopinionatleast,yourcommandoftheseskillswillmakethegreatestdifferenceinyoursuccessandlongevityinthecraft.

Computerprogramming,bytheway,stillisacraft.3Atthiswriting,itstillrequiresskilledworkandtheproductofexperience.Butweshouldrecognizethatthisaspectalsoischanging.Thankstotheinfluenceofopen-sourceprogramsandcooperativedevelopment,computerprogrammingistiltingin-parttowardtheassemblyofnewsolutionsbasedsubstantiallyonpre-existingcomponentsthattheprogrammer(s)in

questiondidnotthemselvesdevelop.This,itself,constitutesanewfundamentaltechnicalskillthatisoneofthesixthatyouwillnowfindinthisbook.

AndSo,AndWithoutFurtherAdo:HereAreTheSixAsIsaidintheIntroduction,thefollowingisalistofwhatIconsidertobesix

fundamentaltechnicalskillsthateverycomputerprogrammerneedstoknow.Technicalmeansthatthesearethingswhichyouneedtoknowandwhichyouwillapplywhencrafting(andtroubleshooting)thecomputersoftwarethatyouwrite,and/orthatyou(andyourteam)maintain,foryourclientoremployer.Theseskillsarenotparticulartoanysinglesize,type,orbrandofcomputerhardware,andforthemostpartalsoarenotlimitedtoanycomputerprogramminglanguageortool.Thesealsoarenotsocialnororganizationalskills.(Thatwouldbeaseparatelist,entirely.)

TheList1. Internalmemorymanagementanddatastructures.

2. Objects.

3. SQLDatabaseQueriesandConcepts.

4. Precisespecification,strategy,andimplementation.

5. Front-End,Back-End.User-interfacesandframeworks.

6. PragmaticDebuggingskills.

Thislistisnotinanyparticularorder,althoughIwillchoosetoaddressitinthesequencegiven.Mytreatmentofeachtopicwillalsonotbeextremelydetailed.PleaseunderstandthatIamseekingtoprovideyouwitha30,000-footview,andtopointyouinspecificdirectionsfromwhichyoucanpursueadditionalresearchonyourown.

Thislistisalsonotaprimer,andIemphaticallydonotbyusingthisphraseintendanynegativeslighttoyou.ThetopicsthatIwillpresentheremightwell,section-by-section,requirere-reading.(And,theymightrequireclarification.Sincethisisane-book,wecandothat.)

DavidIntersimone,theoriginaldirectorofdevelopmenta(now,long-defunct)BorlandInternational,referredtothisexperienceasasipfromthefire-hose.Ifindmyselfunablenowtoacknowledgethatyoursuperficialexperiencewithregardstotheforthcomingmaterialmightwellbethesame.However,asyourGentleAuthor,Ihopethatyouwillnotinfactexpectanythinglessfromthetextthatyouareabouttoconsume.

Andso,withallthatnowsaid:Letusbegin.

One:InternalMemoryManagementandDataStructures:Everydigitalcomputerroom-sizedorpocket-sizedconsistsofthesamethree

functionalparts:

1. CPU=CentralProcessingUnit(themicroprocessor,GPU,etc.).

2. I/O=Input/Output.

3. MainMemory.

Memory,ofcourse,consistsof(today)billionsofindividualstorage-compartments,eachonecharacter(byte)wide,eachwithanindividualaddress.TheCPUretrievesbothinstructionsanddatafrommemory,whichistheonlypartofthecomputersystemwhichis(mustbe)[nearly]asfastastheCPUitself.

Allmodernoperatingsystemsthroughinterestingdevisingsthatneednotconcernusherearemagicallyabletoprovideeachexecutingprogramwiththefunctionalillusionthattheyhavesomecertainamountofmemoryalltothemselves.Theyneverhavetoworryaboutstumblingintoanyoneelsesmemory,becausenooneelsesmemoryisevervisibleoraccessibletothem,unlessbothprogramsmakespecialarrangementstoshareacertainpartofitwhichvery-interestingtopicIherebysimplydeclaretobeoutofscopeforthepurposesofourpresentconversation.4

Thus,wehave,foreachprocess5,aplay-penalltheirown,whichtheydonothavetosharewithanyoneunlesstheywantto.However,asitturnsout,thisallocationofmemoryisnotpristineandundisturbed.Everyprocessrunsundertheauspicesofanoperatingsystem6whichcompletelydefineshoweachprocessactuallyperceives[itsprivateviewof]MainMemory.Thisview,asitturnsout,consistsofexactlytwothings:

1. TheStack:Everyprocess,fromtheoperatingsystemspointofview,consistsofonesubroutine.Theoperatingsystemlaunchestheprocessbycallingthatonesubroutine,and,whenthatonesubroutinefinallyreturnstoitscaller,theentireprocessends.Thissubroutine,directlyorindirectly,launchesmanyothersubroutines,eachoneofwhichiscalledandthen,finally,returnstoitscaller.Eachsubroutine,duringitsfinitelifetime,possessessomecertainsetoflocalvariableswhicharepeculiartoitself,suchthat,ifthesubroutine(bywhatevermeans)happenstocallitself,thelocalvariablesownedbyeachinstancewillbedistinct.Theentireportionofmemorywhichisusedtoaccomplishthisfeatiscalled,theStack,becauseithastheeffectivefunctionalorganizationofasetofdishesstoredatthestartofanycafeterialine.Thecall-and-returnflowofcontrol,andthestorageofalllocalvariables,ismanagedusingthissingleareaofstorage.

2. TheHeap:Thisis,quitesimply,everythingelse.Storageinthisareaisnotallocatedautomatically:itisobtainedonlyonrequest,anditislikewisemadeavailableforre-allocationonlyonrequest.(Itisrather-rudelycalledtheheapbecausethisareaofstoragehasnoinherentstructure,unlikethe

(clearly,push-down)stack.)

Almostallofthestoragethatyourprocesswillactuallyuseistakenfromtheheap,notfromthestack.Furthermore,allreferencestothisstorageareactuallyindirect,accomplishedthroughtheuseofpointers.

Apointerissimplyavariablewhosevalueisunderstoodtobeamemoryaddress.Thestack,yousee,ismerelyanamorphouspoolofavailablestorage.Therefore,inordertoobtaintheuseofachunkofhowever-manybytes,forwhateverpurposesyoumaydevise,itwillbenecessaryforyourprocesstorequestit.Theoperatingsystemwillprovideyouwiththememoryaddressofasuitablearea.Inordertomakeuseofthearea,youmustrefertoitindirectly,usingtheaddressthatyouweregiven.Pointersarethebasemechanismbywhichthistrickisdone.

Mostprogramminglanguages,however,clearlyrecognizethatthisridingtheponybarebackstrategyisfraughtwithquite-unnecessarydanger.(Ifprogramsfailtocomplywiththeoperatingsystemsexpectations,atanypointorforanyreason,theentireprogramwillcrash.)Hence,mostlanguagesimplementamuchmoresophisticatedmemorymanagementstrategy,andhereafterIwillelecttopresumethatyoursituationismore-or-lessalsolikethisone.

Typically,modernprogramminglanguagesblurthedistinctionbetweenapointerandavalue.Whenyourefertoavariableinyourprogram,thelanguagewillautomatically(and,transparently)deducewhetherthisvariablesvalueistheactualvalue,orapointertoit,andinanycasewillrespondaccordingly(and,transparently).

Thelanguagesystemwillalsotransparentlykeeptrackofhowmanyvariablescontaintheaddressesof(referencesto)anyparticularpieceofstorage.Theywill,bysomeveryclevermechanism,usethistodiscoverwhenagivenpieceofstorageisnolongerbeingreferencedbyanyone.Inthisway,programsnolongerhavetobeexplicitlyconcernedwithcleaninguptheirownmess.Blocksofstoragewillbeallocatedautomaticallyandtransparently,and,whentheyarenolongerbeingused,theywillbeharvestedandre-used.

These,then,becomethetwobasicgroundrulesuponwhichallprocesses(or,threads7)may,inmostprogramminglanguages,depend:

Allprogramsdirectlyorindirectlyconsistofsubroutines,eachsubroutine[instance]ofwhichhavetheirownprivatecopiesoflocalvariables,whichcomeintoexistencewhentheydo,anddisappearwhentheydo.

Forallotherpurposesofnecessarymemoryallocation,programsmaysimplyaskfornewstoragetobeallocatedandmayblissfullyignorewhere,exactly,anyparticularpieceofstorageactuallyis.Theymayusethestorageuntiltheynolongerhaveuseofit,andthensimplyabandonit.Wheneveryonehasabandonedit,thestoragewillbesilentlyandreliablyre-used.

Uponthefoundationsofthisentiretobeassumedmemoryinfrastructure,providedforthemgratisbytheprogramming-languagesystemwhichtheyuse,allcomputer

programsmust,sufficientlyfortheirownpurposes(whatevertheyare)constructafoundationwhichissufficientforwhatever-it-isthattheyaresupposedtodo.

Thismeans,essentially,twothings:

1. Anyincoming(orcomputed)datausedbytheprogrammustbestoredinsuchawaythattheprogram,duringitsexecution,can(ofcourse)obtainitagain.

2. Theprogrammustbecapableofhandlingavariableandunpredictablequantityofdata.Theremustbenopre-conceivedlimitsastojusthowmanycopiesofdatamightbestored(or,storable).

Ordinarily,thesechoresaredelegatedtopre-existingstoragestrategieswhichareanintrinsicpartofwhateverlanguagesystemisbeingused.Thereareusuallytwotypes:thosewhichstoreavalueunderaparticularkey(requiringthatexactkeytobeprovidedinordertoretrieveitagain),andthosewhichstoreanarbitrarily-sizedlistof(zeroormore)values.Thesetwoarefrequentlyusedincombination,toallowzeroormorevaluestobereferencedbyanyuniquekey:eachelementofthekeyeddata-store(suchasahashortree),referstoaseparatelist.

Letusnowconsiderwhatsortofthingscangowrongwiththesearrangements.Whatsortofthingscancauseaprogramtomisbehave,orfail?Herearethemost-commonculprits:

1. StackOverrun,causedbyendlessrecursion:AsIsaidearlier,thestackistheportionofmemorythatsusedtomanagesubroutinecalls.Whenasubroutineiscalled,informationisstoredinthestacktofacilitatereturningfromthesubroutine,andthesubroutineinstanceslocalvariablesarealsostoredthere.Thestackisofalimitedsize.Therefore,itispossibletooverruntheboundariesofthestackiftoomanynestedsubroutinecallsaremade.Pragmatically,thismeansthatsubroutinesarecallingthemselves,so-calledrecursively,withouteverreturningfromanyofthosecalls.Theeffectofthissortofbugonaprogramisinstantlyfatal,butrelativelyeasytodebug.

2. HeapCorruption(theinfamousDoubleFree):Thisalways-fatalproblemiscausedbycorruptionoftheinternaldatastructureswhichmanagetheallocationandreturnofmemoryintheHeap.Therearetworoutines:amalloc()routine,whichrequestsablockofmemoryofaspecifiedsize(returninganaddress),andafree()routine,whichreleasesablockofmemoryataspecifiedaddress(whichmusthavepreviouslybeenobtainedfrommalloc()).Programsarerequiredtofree()onlyaddressesthattheyobtainedfrommalloc(),andtofree()anyparticularaddressonlyonce.Theyarealsorequiredtoconstraintheirmemory-modificationsonlytotherangeofaddressesgiven,nevermodifyinganyadjacentbytes.Mostmodernprogramminglanguagesprotectfromthesetypesofproblemsbymanagingthelow-levelmalloc()andfree()callsthemselves.

3. HeapExhaustion(MemoryLeaks):Programsarerequiredtotimely

releaseanystoragethattheyarenolongerusing.Mostmodernprogramminglanguagestakecareofthischorethroughsomemechanismwhichdetectsautomaticallywhenaparticularstorageblockisnolongerbeingreferenced,butso-calledleakscanoccurwhen,forinstance,aseriesofstorage-blocksallcontainreferencestooneanother,butthereremainnootherreferenceselsewheretoanyofthoseblocks.(Sincealloftheblocksarestillreferenced,theynevergetreleased.)Heapexhaustioncanalsobecausedbyinefficientprogramdesign.

4. Failuretodetectwhenastorage-allocationrequestcouldnotbesatisfied:Whenamalloc()requestcannot,forwhateverreason,obtaintheamountofstoragerequested,itwilltypicallyreturnzeroaspecialvaluealsoknownasNULL.Programsshouldbedetectifthisoccurs,andrespondaccordingly,buttheyrarelydo.

5. Exhaustionoffixed-sizestoragearrays:Someearlyprogramminglanguagesdonotallowstoragetobedynamicallyallocatedfromtheheap.Instead,theprogrammermustspecifyafixedsizeforthestructure.Programsaresupposedtodetermineifthespacewithinthesefixedstructureshasbeenexhausted,buttheyrarelydo.Programminglanguagesalsousuallydonotdetectthatareferencehasbeenattemptedwhichliesoutsideoftheproscribedboundariesofthestructure.Theusualconsequenceisstackorheapcorruption.

Memoryissuesareacommonsourceofproblemsinsoftwarethatisintheprocessofbeingdeveloped,buttheyaremuchlesscommoninprogramsthatareinproduction.

Two:ObjectsManyoftheinfluentialbooksincomputerscienceliteratureshareacommon

characteristic:theyaresmall.CertainlyoneofthemostimportantofthesewasentitledAlgorithms+DataStructures=Programs,byDr.NiklausWirth.8

Truly,thetitleofthisbooksaysitall.Anycomputerprogramconsistsofalgorithms(thestep-by-stepexecutionofinstructionsthatiscalled-for),appliedtodatastructuressuchastheonesalludedtointheprecedingsectionofthisbook.

Inearlyprogramminglanguages,thesetwoconcerns(algorithmanddata)wereaddressedseparately,indifferentpartsoftheprogram.(TheCOBOLlanguagewasaparticularexample,definingall(fixed)datastructuresintheso-calledDATADIVISION,andallalgorithmsinthePROCEDUREDIVISION.)Thisisnotsomuchofaproblemwithregardtothelocalvariablesthatmightbeassociatedwithaparticularprocedureorfunction,butitisaveryvexingconcernwithregardtotheglobalstoragethatisusedbymultipleproceduresandfunctionsthroughoutaprogram.

Inaword,theproblemisthatthetwothingsareseparated:thedatastructureswhicharemanipulatedbyalgorithms,areseparatefromthealgorithmswhichmanipulatethedatastructures.Ifdecisionsneedtobemadeastowhichalgorithmshouldbeappliedtowhichdata,thesedecisionswindupbeingredundantlyscatteredthroughouttheentireprogram.

Toaddressthisconcern,thenotionofobjectswasinvented.

Anobject,forourpurposes,isaself-describingpieceofstorage,allocatedfromtheheap.Itcontains,notonlyspacefortheindividualvalues(properties)whichmightneedtobestoredthere,butalsoadditionaldescriptivedata(metadata)whichservestodirectlyassociatetheobjectwiththeproceduralcode(methods)thataredesignedtooperateinconjunctionwithit.

Significantly,givenaparticularobjectandarequesttoapplyaparticularfunctionagainstit(aso-calledmethodcall),thecomputerisabletodeterminewhichfunctionisthecorrectonetocallbasedonlyonthemetadatacontainedwithintheobjectitself.Theexactmechanismsbywhichthisdeterminationismadeareconcealedfromtheprogrammer,buttheyareveryefficient.

Theparadigmthatisusuallyquotedis:Hey,you!Dothis!Whereas,inaconventionalprogramminglanguage,aspecifiedsubroutinewouldbecalledandareferencetothedatawouldbesuppliedtoitasaparameter,inanobject-orientedprogramminglanguagetheprimaryreferenceistotheobject(Hey,you!)whichistheninstructedtocalloneofitsmethods(Dothis!).Theactualsequenceofeventsthatsubsequentlytakesplacemayvaryfromobjecttoobject,andfromonemethod-calltothenext,becausethedecisionismadeliterallyon-the-fly.9

Sincelate-bindingisthefundamentalcharacteristicofanyobject-orientedprogrammingsystem,therearemanyapproachesthatexistinglanguagesusetoobtainit.Somelanguagesaredesignedstrictlyfromthegrounduptouseanobject-oriented

approach,whereasotherlanguagespermitobject-orientedandconventional(procedural)techniquestobeusedinthesameprogramsatthesametime.Languagesalsodiffersometimes,quitemarkedlyinexactlywhatfeaturestheydoanddonotoffer.

Akeynotionofanyobject-orientedsystemisconcealment.Objectsaresaidtoexposecertainmethods(functionalitythatcanbecalled)andproperties(valuesassociatedwiththeobjectwhichcanbeexaminedorset).(Theactofexaminingorsettingthevalueofapropertymightcauseproceduralcodetobeexecutedso-calledsideeffects.)Theobjects,intheactofexposingwhattheydoandwhatcanbedonetothem,alsoconcealthedetailsofhowtheydoit.Thisisdonesothatdifferentandderivativedefinitionsofanobjectcanbedevised,allofwhichappear(andappeartoact)thesametotheirclientstotheotherpartsoftheprogramwhichmayuseandreferencethemwhileperhapshavingmanyentirely-differentimplementations.Thiswhat,nothowapproachavoidsunnecessarydependenciescouplingbetweendifferentflavorsofobjects,andalsobetweentheobjectsandtheirclients.

Object-orientedlanguagesaredesignedtoallowyoutodefineataxonomyofrelatedobjectdefinitions.Forexample,aso-calledbaseclassofvehiclecouldbedefined,fromwhicharederivedsubclassessuchasautomobile,motorcycle,andtruck.Eachsubclassinheritsthepropertiesandmethodsofitsbaseclass(e.g.themethoddrive()andpropertiessuchascolorandnumber_of_wheels),butimplementsthemindifferentways.Subclassescancompletelyoverride(replace)thepropertiesandmethodsoftheirancestors,orcanaugmentthem.

Somelanguagessupportmultipleinheritance,whereasubclasscaninheritcharacteristicsfrommorethanonebaseclass.(Andsometimes,whenthelanguageallowsthesemultiplebaseclassestodefinethesamemethodsandproperties,theresultscanbequiteinteresting.Thelanguagewillalwaysdefinesomekindofbright-lineruletodetermineexactlywhatwillactuallyhappen.)

Everyobject-orientedlanguagewilldefineoneortwospecialmethodsthatcanbeappliedtoanyobject,callingtheseconstructorsanddestructors.Aconstructorisaninitializationsubroutine,calledbythelanguageimmediatelyaftermemoryhasbeencarved-outforanewobjectinstance.Adestructorisacleanupsubroutine,andacompaniontotheconstructor,beingcalledbythelanguageimmediatelybeforetheobjectmemoryisfreed.Constructorsanddestructorsofsubclassesusuallyareobligedtocalltheconstructorsanddestructorsoftheirancestors,andtodosoatparticularpoints.

Object-orientedlanguagesopenlyencouragetheuseofthephrase,isa.Acarobjectisacar,andadune_buggyobjectisadunebuggyaswellasacar.Thesameistrueofasedan.Iftheclientofaparticularobjectonlywishestodosomethingthatbothasedanandadune-buggycandoequallywell(howboring),thenthatclientdoesntneedtoknoworcareexactlywhatvarietyofcarheisusing.(Thethirty-ninecentwordforthisideais,polymorphism.)Iftheclientwantstoasktheobjecttodosomethingthatasedanandadune-buggywouldcarryoutindifferentways,thentheclientalsodoesntneedtobeconcernedwiththesedifferencesofimplementation.

Therearemanyfundamentaladvantagestoobject-orientedlanguages,andverylittle

runtimecost.Butprogramsaresubjecttocertaindesigndifficultiesoncetheyhavebeeninserviceforanumberofyears,mostlyduetotheinheritanceschemesaforementioned.Aslongasthebusinessrequirementsdonotchangeinanywaythatisnotperfectlyreflectedintheobjectinheritancestratagemoriginallydevisedfortheprogram,suchprogramscanhaveaverylongservice-life,indeed.However,ifrequirementsdochangefundamentally,inheritancecanbecomeanintractableformofcouplingbetweenthevarioussubclasseswhicharederivedfromacommonancestor.

Three:SQLQueriesandConceptsSQL(StructuredQueryLanguage)isthenow-ubiquitouslinguafrancaofthe

databaseworld.Withfew(legacy)exceptions,alldatabasesystemsinusetodayareengineeredtouseit.Althoughtheimplementationsarenotthesame,theyareatthispointsufficientlysimilarthatIcannowmakeseveralimportantandgeneralstatementsaboutthem.

TheSQLlanguagewasinventedaspartoftheSEQUELprojectatIBM,basedinpartonapioneeringpaperbyDr.E.F.Codd,whichwaspublishedintheJune,1970issueoftheCommunicationsoftheACM.Akeyfeatureofthislanguagewasthatitwasessentiallydeclarative.SQLallowsyoutospecifywhatdatayouwishtoobtain.Itisuptothedatabaseengineto,onthefly,deviseaplanforobtainingtheseanswers,andthentodoso.Thedatabaseenginemayexploitcharacteristicsofthedatabasesuchasindexesthathavebeencreatedoncertainfieldsinordertoproduceitsanswersmoreefficiently,butanSQLquerydoesnotspecifyhowtheworkistobecarriedout.

InformationinanSQLdatabaseisorganizedintotables,whichcontainanunlimitednumberofrows.Eachrowconsistsofanidenticalsetofcolumns,eachofwhich(usually)containseitherasinglevalue(ofasinglespecifieddatatype),ornovalueatall.(Whenthisisthecase,wesaythatthecolumnISNULL.)Therowsinthetableareinanunspecifiedorder,althoughyoucanrequestthatquery-resultsshouldbereturnedtoyousortedinanyorderyouwish.

Conceptually,everySQLquery(theSELECTstatement)consistsofaspecificationofthefollowing:

1. Alistofthecolumnswhosevaluesyouwanttosee.(Ifthesamecolumnnameappearsinmorethanonetable,youmustbespecific.)

2. Thetablesfromwhencethedataistocome,andhowthesetablesarerelatedtooneanotherforthepurposesofthisquery.(Seebelow)

3. Theselectioncriteria(WHEREclause)thatistobeused.

4. Ifyoudliketoreceivesummarystatistics,specifywhatthedatashouldbeGROUPedBY.

5. Ifyoudliketoreceiveonlycertainsummaryrows,specifywhatcharacteristicsthegroups-of-interestshouldbeHAVING.

6. Ifyoudliketoreceivetheresultssortedinaparticularway,specifywhattherowsshouldbeORDERedBY.

SQLqueriescanobtainresultsfromanynumberoftablesatatime.Thequeryspecifieshowtherowsinthevarioustablesshouldbeconsideredtoberelatedtooneanother.Thisrelationshipisexpressedbythepresenceofidenticalvaluesinone-or-morespecifiedcolumnsinanypairoftables.Forexample,acustomer_idfieldinanOrderstableissufficienttoenableanysortofinformationaboutthatcustomertoberetrievedfromaCustomerstableinthesamequery.Thisisreferredtoasajoin.

Therearethreetypesofjoinsthatcanbeused:innerjoins,whichreturnonlyrowswhichhaveidenticalvaluesinbothtables,andleftorrightouterjoins,whichalwaysreturnalloftherowsfromoneofthetablesortheother.(AninnerjoinagainstOrdersandCustomerswouldreturnOrdersthatareassociatedwithCustomers,whileanouterjoinmightreturn[leftouter-join]allOrders,withorwithoutCustomers,or[right]allCustomers,withorwithoutOrders.)

SinceSQLqueriesdonotspecifyhowthedatabaseengineistoobtainthespecifiedresults,itisveryimportanttounderstandhowyourquerieswillbeinterpreted.Itiscertainlypossibletowritetwodifferentqueriesthatwillproducethesameresults,butthatwilldosoindramaticallymore-orless-efficientways.MostdatabasesystemsprovideanEXPLAINcommandwhichwilltellyou(inratherarcane,system-specificterms)exactlyhowthedatabaseenginewouldgoaboutcarryingoutaparticularquery.

AverysignificantproblemwithSQLqueries,intoo-typicaldeployedapplications,isthatthewebserver(or,whoeverisissuingaparticularquery)candoanythingandeverything.EverySQLserverhassomekindofpermissions-systemwhichspecifiesexactlywhatanyuserisandisnotpermittedtodo.Ifsomeweb-sitehackeris,bywhatevermeans,abletopersuadeyourweb-servertoissuetheDROPTABLE(oreventheDROPDATABASE(!!))command,andyourweb-serverisauthorizedtoissuesuchacommand,then(atleastavery-significantpartof)yourdatabasejustdisappeared.Nuffsaid.

WhenyoudealwithSQLdatabases,youmustalsodealwiththeissueofconcurrency.Onatypicaldatabaseserver,hundredsofqueriesmightbeexecutingatthesametime,andthesequeriesmayormaynotbespecificallyconcernedwithwhattheotherqueriesaredoing.(Forinstance,ifauserismerelybrowsingaproductcatalog,itsessentiallyacertaintythatthecatalogscontentswontbechangingatthetime.Accountingdata,however,isadifferentmatter.)SQLdatabasesystemshaveaspecificstrategyfordealingwiththisissue:transactions.10

Atransactionisdefinedasasingleunitofworkitcouldbeasetofmodifications,deletions,and/orupdates,oritcouldsimplybeasetofquerieswhichisconsideredtobeatomic.Thatistosay,asingle,indivisiblegroup.Inthecaseofmodifications,eithertheentiresetofmodificationshappens,or,noneofthemdo.Inanycase,atransactionhassomespecifieddegreeofisolationfromeveryothertransactionthatisoccurringatthesametime.Forexample,anaccountingreportmightneedtosecureasnapshotviewofavery-busydatabaseasitexistedataparticularinstantintime.

Four:PreciseSpecification,Strategy,andImplementationAsaprofessionalcomputerprogrammer,youwillalwaysbeconfrontedwiththetask

ofturninguser-requirementsinto1sand0s.

Butfirst,youwillbeconfrontedwiththerealizationthattheuser-requirementswhichyouhavebeengiven,arenotyetsuitabletobetreatedinthatway!

Peopledescribetheirrequirementsinhumanandbusinessterms.Tothem,itisanInvoice,andeverythingintheirexistingexperienceconcerninginvoicesis:assumed,implied,well,ofcourse,andperhapsworstofall,Iforgottomention.

AsIdiscussinmye-book,ManagingtheMechanism,11whenyoubuildcomputersoftware,youarebuildingaself-directingmachine.Itisamachinecomposedofif/then/elsedecisions,variables,internalandexternalstate.Amachinecomposedofliterallybillionsofmovingparts,allpotentiallyinterconnectedtooneanotherandsoinfluencingoneanother.Intruebinaryfashion,suchamechanismiseither( )correct,or( )not.Thereisnothingin-between.

Ordinarily,specificationswillcometoyourteamfrombusinessanalysts,whosejobitistoflesh-outthedescriptionoftheprojecttofullyencompassalloftherelevantbusinessimplicationsincludingtheimplicationsthatthestakeholdersthemselvesprobablydidntthinkabout.Buteventhesewillbebusinessrequirements,notyetexpressedintermsofnewsource-codethatistobewritten,norchangesthataretobemadetoexistingcode.

Theworstthingthataprogrammercandoatthispoint,istodowhatisordinarilydoneatthispoint:tojuststartwritingcode.Effectively,makingitupastheygoalong.Addressingeachproblemorissueasitisencountered.Thekeyproblemwiththislack-ofstrategystrategyissimplythatthepiecesofsource-codethatyouarewritingnowmustinteractwithotherpiecesthathavenotyetbeendevelopedorevendesigned!Likewise,ifathoroughanalysisoftheexistingsource-codebasehasnotfirstbeendone,thereisanoverwhelmingprobabilitythatthenewcodewontmeshproperlywithit.Theincompatibilitywillbediscoveredattheworstpossibletime:whenthepreviously-writtenmaterialisbeingputintoservice.

Software-writingisnotmustnotbeavoyageofdiscovery.Nooneintheirrightmindsetssailfromaharborortakesofffromanairportwithoutaplan;aplanthatspecificallyincludescontingencies.Sure,theoverallplanusuallyconsistsofanumberofshorterhops,butcarefulattentionisplaceduponanticipatingthecourseoftheprojectandlookingforanythingthatmightinterferewithitssuccess.

Thisformofproject-planning,inaveryrealsense,iscode-writing.Figuringoutwhatcodeneedstobewritten,whatitneedstodo,whatinputsandpre-conditionsitwillencounter,andexactlywhatoutputsandresponsesitmustproduceinallcases,trulyisthehardpartofdevelopingcomputersoftware.Bycomparison,writingsource-codeandgettingittocompileisentirelysecondary.

Five:Front-End/Back-End.UserInterfacesandFrameworks.

Allreal-worldproductionapplicationswillbefoundtohaveamulti-tierarchitecture.Theywillinvolvetheinteractionofthemachineinthecustomershands(or,onherdesk),connectingtosomeserver(s)whichareresponsibleforperformingallorpartofthework.Eachoftheseservers,inturn,maycommunicatewithotherservers.

AJAXisthetermthatismost-commonlyusedtodescribetheinteractionbetweentheJavaScriptonatypicalweb-page(whichisexecutingontheusersowncomputer),andremoteHTTPservers.ThisisaspecificcaseofatechniquecalledIPC=Inter-ProgramCommunication,RPC=RemoteProcedure-Calls,and/orClient/Server.

InthecaseofAJAX,theJavaScriptfront-endcodeismakingspecificrequeststotheback-endwebserver.AlthoughtheserequestsarebeingmadeusingtheHTTP(orHTTPS)protocol,theyarenotrequestsforHTMLcontent.Instead,thefront-endcreatesapacketofinformationsimilartofilling-outastandardformandsendsitviaHTTPtotheserver,whichcarriesouttherequestandreturnsanotherpacketofinformationdescribingtheoutcome.Hencetheterm,remoteprocedure-call.Thefront-endclienteffectivelyaskstheservertocallasubroutine(aprocedure),andtoreturntotheclienttheresultsthereof.

SincetheprotocolthatisbeingusedisHTTP,boththerequestandtheresponsemustbeencodedinsuchawayastobecompatiblewiththerequirementsofHTTP.ThisistypicallydoneusingtechniquessuchasJSONandYAML.OthercharacteristicsofHTTP,suchascookies,areusedtomaintainstateandtoidentifythesourceoftherequests.

Inlargersystems,aformatcalledXMLisused,andtheprocessisaltogethermuchmoreformalized.Inthisarrangement,calledSOAP,thecommunicatingcomputersmightnotknoweachotherbutmightdiscoveroneanother.Formalmethodshavebeendevised(WSDL,etc.)bywhichthecomputerscanbroadcasttheircapabilitiestooneanotheraspartofthisprocess.

Inallofthesecases,theworkthatisbeingdoneisbothvery-complexandvery-familiar.Itsbeendoneathousandtimesbefore.Therefore,developershaveconcoctedandperfectedmanyframeworkswhichcanbeusedasfoundationsforbuildingsubstantialpartsoftheapplication.Everytimeyouworkonanapplicationsuchasthese,youwillencounteraframeworkmostlikely,adifferentonethanyouhaveeverusedbefore.Soitgoes.

Frameworksarealsousedtoconstructfront-enduserinterfaces.Sometoolkitsareusedtogloss-overthedifferencesbetweenwebbrowsers.Othersgloss-overthedifferencesbetweendifferenttypes(andbrands)ofmobiledevices.

Eachframeworkmakesitveryeasytodocertainthingstocreatecertainvisualeffectsonadevice,forexample.Bewarned,however,thattheresultsareverybeguiling.Itseasytoconfuseapparentprogress(whichappearstohappenveryfastindeed)with

thenot-sovisiblesupportingsoftwareinfrastructurewhichmustbebuiltunderthesurface.Itisalsoveryeasytofallinlovewithwhatyouvedone,onlytodiscoverthatadifferentapproachorpresentationmightworkbetter.Onceagain,thesediscoveriesareoftenmadeatinopportunemoments,andrequiresometimesdeep-seatedandfar-reachingchangestothesystem,whichquicklyde-stabilizeit.

Six:PragmaticDebuggingSkillsDebuggingseemslikeanarcaneart.(Donewell,itseemslikevoodoo.)However,in

itsentirety,debuggingisonlyonefacetofamuchlargerconcern:keepingdefectsfromgettingintothecodeinthefirstplace,andmakingthemOIL=obvious,identifiable,andlocatablewhentheydo.

Thisisaconcernthatshouldpermeateyoursoftware-writingatmanylevels.Itwillcauseyoutoaddthingstoyourcodethatyourcolleaguesmightcallunnecessaryorinefficient.(Butlettherecordshowthatyouspendfarlesstimedebuggingthantheydo!)Doneproperly,thiswillresultinrobustsoftwarethatkeepsyourpocket-pagersilentallnightlong.

ThefirstprinciplethatIwillnowofferisthat:thecomputersoftware,itself,isactuallytheonlypartythatistrulyinthepositiontodetectadefectwithinitself.Thefirstandhardeststepintroubleshootinganypieceofsoftwareisliterallytobeawareofthedefectsexistence.Therefore,codesuspiciously.Subroutinesshouldcheckalloftheirinputsandtheirassumptions.Inanychainofif/then/elselogicwhereoneofthesecasesmustmatch,alwaysremembertoaddonemorecase:thecasethatcantpossiblyhappen,butjustdid.Ifyoudontdothis,itsprobablyimpossibleforanyonetoknow.

So,whenyoursuspiciouscodenowfindsthatsomethingwhichcantpossiblyhappen,justdid,whatsitsupposedtodoaboutit?Thebeststrategyistothrowanexception.Thisissimilartodividing-by-zero,oryellingFire!Executionstopsinitstracks,andtheprogramminglanguagelooksforanexceptionhandler(whichyouvesetupinadvance).Whenitfindsone,controlisimmediatelytransferredtothathandler(nevertoreturn),andanexceptionobjectdescribingtheincidentishandedtoit.Amongthisinformationistheexactlocationfromwhencetheexceptionwasthrown.Yoursuspiciousprogramming,havingdiscoveredafire,hasjustnotifiedthefire-department.

Anotherexcellentdebuggingtechniqueistologinformativeprogress-messagestosomekindoffile,ortoanevent-logmechanismprovidedbytheoperatingsystem.Generationofdebuggingmessagesisoftenanoptionthatcanbeturnedonoroff.

Manyprogramminglanguagesprovideafeaturecalledassertions.Thesetaketheformofasubroutine,usuallynamed ,whichisgivenaparameterthatmustbetrue.Iftheparameterisnottrue,anexceptionisautomaticallythrown.Furthermore,thisfeatureisimplementedinsuchawaythatitcanbeturned-onorturned-off;includedintheprogramoromittedfromit.Sprinkleyoursource-codeliberallywiththeseassertions.

Afinalusefuldebugging-techniqueisatracetable.Thisissimplyanarrayorbuffer,ofsomecertainsize,intowhichin-progressmessagesarerecorded.Thetrace-tableiscircular:newmessagesreplacetheoldestones.(Eachentryistypicallytimestamped.)Theadvantageofthistechniqueisthatitisveryfast,becauseitdoesntinvolveinput/output.Facilitiesmustbeprovidedtoextractthepresentcontentofthetracetable.

Debugging,unabashedly,isdetectivework.ButthesetwotechniquesthatIhavenowdiscussedwillgreatlyimprovetheeffectivenessofthisprocess.Bymakingtheprogramsuspiciousofitsownbehavior,youimprovetheoddsthatdefectswillbediscoveredandcorrectedearly.Bybuildingachronologyofwhathappenedrecently,youmakeiteasiertodiscovertheinternal-stateofthesystemwhichenabledthedefectivebehaviortooccur.

Allproductionsystemsshouldalsobeaccompaniedbyacomprehensivetestsuite,thepurposeofwhichistoexerciseandre-exercisecomponentsofthesystematvariouslevels.Thetest-suiteisrunandre-runconstantly,andbothsuccessfulandunsuccessfuloutcomesarelogged.Ifasource-codechangeisintroduced(orisabouttobeintroduced)whichcausesatest-casetofailwell,forewarnedisforearmed.

InClosing:Wellthen,thereyouhaveit.Mysix.And,alongtheramblingway,mypragmatic

recommendations.MostifnotallofthetopicsthatIhavequicklydescribedinthislittlebookwillcallforfurtherexplorationonyourpart,andIhopethatIhavesucceededinsettingthestageofunderstandingforyoutodoso.

Computerprogramminghaschangedenormouslyoverthepastsixtyyearsandcounting,butinmanywaysithaschangednot-at-all.Werestillwritinginstructionsforelectronicmachinesto(unthinkingly)carryout,andtheprocessrequiresalotofhumanthought.Capturingthebigpicture,inspiteofthemyriaddetails,issomethingthatcaneasilybecomelostintheshuffle.Ihopethatthesewordshavehelpedyouinsomesmallway,andwelcomeyourcomments,reviewsandfeedback.

VinceNorth

1Theobservation,attributedtoDr.GordonMoore,thatsemiconductorswoulddoubleinspeedanddensityeverytwoyear.

2Atermmost-likelyoriginallycoinedbyCharlotteBrontinherbook,JaneEyre:GentleReader,mayyouneverfeelwhatIthenfelt!Mayyoureyesnevershedsuchstormy,scalding,heart-wrungtearsaspouredfrommineOhyes,Ms.Bronthadquitethewaywithwords.

3Wikipediadefinescraftas:apastimeoraprofessionthatrequiresparticularskillsandknowledgeofskilledwork.

4Hey,ImtheAuthorhere.Icandothat.Please,readon

5Allright,allright.Ihavenopragmaticchoice,atthispoint,buttoimposeanimportanttechnicalterm:process,wherepreviouslyIsaid,executingprogram.Onalmostanycomputertoday,youcanrunmore-than-onecopyofthesameprogramatthesametime,justaseasilyasyoucanrundifferentprograms.Operatingsystemsroutinelycalleachdistinctinstanceof,runninghereaprocess.

6Theoperatingsystemisthefoundationallayerofsoftwarewhichgovernstheoperationoftheentirecomputersystem.Unix,Linux,OS/X,Z-OS,etc.areallexamplesofthis.Thesecreateandmanagetheoperatingenvironmentunderwhichallprocessesultimatelyoperate,anddefineandimplementtheentireworldthatisavailabletothem.

7Athreadisanindependentthreadofexecutionrunningwithintheauspicesofasingleprocess.Forourpurposesnow,simplythinkofitashavingitsownstack.

8ISBN:978-0-13-022418-7.

9Thischaracteristicissometimescalled,latebinding.

10NoteverySQLdatabasesystemsupportstransactions.Mostdo,butsomehavestringsattached.Forinstance,theever-popularMySQLdatabaseonlysupportstransactionsiftheInnoDBphysicalfile-formatisused.

11North,VincentP.,ManagingtheMechanism:WhySoftwareProjectsArentLikeAnyOtherProjectYouveEverTriedtoManage(AndHowToDoItSuccessfully).Published2012.ISBN:978-0-715-74383-7.

Introduction: Just Call It A Careers Experience, Condensed Into Six Key TopicsAnd So, And Without Further Ado: Here Are The SixThe List

One: Internal Memory Management and Data Structures:Two: ObjectsThree: SQL Queries and ConceptsFour: Precise Specification, Strategy, and ImplementationFive: Front-End / Back-End. User Interfaces and Frameworks.Six: Pragmatic Debugging SkillsIn Closing:Footnotes