TreadingonPythonSeriesIllustratedGuidetoPython3

ACompleteWalkthroughofBeginningPythonwithUniqueIllustrationsShowinghowPythonReallyWorks

MattHarrison

TechnicalEditors:RogerA.Davidson,AndrewMcLaughlinVersion:Oct2,2017Copyright©2017While every precaution has been taken in the preparation of this book, thepublisher and author assumes no responsibility for errors or omissions, or fordamagesresultingfromtheuseoftheinformationcontainedherein.

TableofContents

IntroductionWhyPython?WhichVersionofPython?TheInterpreterRunningProgramsWritingandReadingDataVariablesMoreaboutObjectsNumbersStringsdir,help,andpdbStringsandMethodsComments,Booleans,andNoneConditionalsandWhitespaceContainers:Lists,Tuples,andSetsIterationDictionariesFunctionsIndexingandSlicingFileInputandOutputUnicodeClassesSubclassingaClassExceptionsImportingLibrariesLibraries:PackagesandModulesACompleteExampleOntoBiggerandBetterFileNavigationUsefulLinksAbouttheAuthor

AlsoAvailableOnemorething

Introduction

AREYOUREADYTOJUMPSTARTYOURPYTHONPROGRAMMINGCAREER?THISBOOKWILL

armyouwithyearsofknowledgeandexperiencethatarecondensedintoaneasyto follow format. Rather than taking months reading blogs, websites, andsearchingmailinglistsandgroups,thisbookwillallowaprogrammertoquicklybecomeknowledgeableandcomfortablewithPython.ProgrammingisfunandPythonmakesitdelightful.BasicPythonisnotonly

easy,butapproachableforallages.Ihavetaughtelementarystudents,teenagers,“industry” professionals and “golden years” folks the Python programminglanguage.Ifyouarewillingtoreadandtype,youareabouttobeginanexcitingpath.Where it ultimately takes you depends on how hard you are willing towork.TherearedifferentlevelsofPython.BasicPythonsyntaxissmallandeasyto

learn.OnceyoumasterbasicPython,doorswillopentoyou.Youshouldbeableto read a lot of Python and understand it. From there, you can learn moreadvanced topics, specific toolkits, start contributing to open source projectswritteninPython,orusethatknowledgetolearnotherprogramminglanguages.Arecommendedapproachfor learning the language is to readachapterand

then sit down at a computer and type out some of the examples found in thechapter. Python makes it easy to get started, eliminating much of the hasslefoundinrunningprogramsinotherlanguages.Thetemptationwill likelybetomerelyreadthebook.Byjumpinginandactuallytypingtheexamplesyouwilllearnalotmorethanbyjustreading.

WhyPython?

PYTHONISBOOMING!ITISTHETOPLANGUAGEBEINGTAUGHTINUNIVERSITIES.PYTHONdevelopersareamongthehighestpaid.Withtheboomindatascience,Pythonisquickly becoming one of themost desired skills for analytics. Operations arealso adopting Python tomanage their backend systems. They are discoveringwhatweb developers using Python have known for a long time; namely, thatPythonwillmakeyouproductive.Pythonhascrossedthetippingpoint.Nolongerareonlysmall,agilestartups

relyingonit.Lookingtotakeadvantageofitspowerandefficiency,enterpriseshave also converged on Python. Over the past year, I've taught Python tohundreds of seasoneddeveloperswith years of experience at large companies,whoaremovingtoPython.Python enables increased productivity. I came to Python as a Perl

programmer.Atwork,Iwasassignedtoateamwithaco-workerwhowaswellversed in Tcl. Neither of us wanted to jump ship though both of us wereinterested in learning Python. In 3 days our prototype was completed, muchfaster than we expected, and we both promptly forgot our previous “goto”language.Whatappealed tomeaboutPythonwas that it fitmybrain. I firmlybelieveifyouhavesomeexperienceinprogramming,youcanlearnthebasicsofPythoninafewdays.Python is easy to learn. For beginning programmers, Python is a great

steppingstone.Learningtowritesimpleprogramsisprettyeasy,yetPythonalsoscalesuptocomplex“enterprise”systems.Pythonalsoscaleswithage—Ihavepersonallyseenpeoplefrom7-80+learnbasicprogrammingskillsusingPython.

WhichVersionofPython?

THIS BOOK WILL FOCUS ON PYTHON 3. PYTHON 2 HAS SERVED US WELL OVER THEyears. The Python Software Foundation, which manages releases of thelanguage,hasstatedthatPython2iscomingtoanend.Assuch,after2020theywillnolongersupportthelanguage.Python3,hasbeenoutforabitnowandissomewhatbackwardincompatible

with the2series.Forgreen fielddevelopment,youshouldmove forwardwithPython3.IfyouhavelegacysystemsonPython2,donotfret.Infact,mostofthecontentinthisbookisperfectlyapplicabletoPython2.IfyouwanttofocusonPython2,checkoutthepriorversionofthisbook.

Pythoninstallation

Python3isnotinstalledonmostplatforms.SomeLinuxdistributionsshipwithit,butWindowsandMacuserswillneedtoinstallit.ForWindowsfolks,gotothedownloadareaofthePythonwebsite1andfinda

linkthatsays“Python3.6WindowsInstaller”.Thiswill linktoa .msifile thatwill install Python on yourWindowsmachine. Download the file, open it bydouble-clickingit,andfollowtheinstructionstofinishtheinstallation.

NOTE

On the Windows installer there is an option labeled "Add Python toPATH".Pleasemake sure it is checked.Thatway,whenyou runpythonfrom the command prompt, it will know where to find the Pythonexecutable.Otherwise,youcangotoSystemproperties(clickWIN+Pause,orrunenvironfromthestartmenu),Advancedsystemsettings,andclick

on the Environment Variables button. There you can update the PATHvariablebyaddingthefollowing:

C:\ProgramFiles\Python3.6;C:\ProgramFiles\Python3.6\Scripts

If you have UAC (User Account Control) enabled onWindows, thenpathis:

C:\Users\<username>\AppData\Local\Programs\Python\Python36

Likewise, Mac users should download the Mac installer from the Pythonwebsite.

NOTE

Anotheroptionfor installingPython is touse theAnaconda 2distribution.ThisrunsonWindows,Mac,andLinux,andalsoprovidesmanypre-builtbinariesfordoingscientificcalculations.Traditionally,theselibrarieshavebeenannoyingtoinstallastheywraplibrarieswritteninCandFortran,thatrequiresomesetupforcompiling.Macusersmightalsowant to lookinto theHomebrewversion 3. Ifyou

arealreadyfamiliarwithHomebrew,itisasimplebrewinstallpython3away.

Whicheditor?

Inadditionto installingPython,youwillneeda texteditor.Aneditor isa toolforwritingcode.Askilledcraftsmanwillinvestthetimetolearntousetheirtoolappropriatelyanditwillpaydividends.Learningtousethefeaturesofaneditorcan make churning out code easier. Many modern editors today have somesemblanceofsupportforPython.

IfyouarejustbeginningwithPythonandhavenothadmuchexperiencewithreal text editors, most Python installations include IDLE, which has decentPython editing features. The IDLE development environment also runs onWindows,MacandLinux.Afeature to lookfor ineditors is integrationwith thePythonREPL 4.Later

youwillseeanexamplewithIDLE.Hopefullyyoureditorofchoicewillhavesimilarfeatures.Popular editors with decent Python support include Emacs, Vim, Atom,

Visual Studio Code, and Sublime Text. If you are interested in more fancyeditors that have support for refactoring tools and intelligent completion,PyCharmandWingIDEarealsopopular.

Summary

Python 3 is the current version of Python.Unless you areworking on legacycode,youshouldfavorusingthisversion.YoucanfindthelatestversiononthePythonwebsite.Most modern editors contain some support for Python. There are various

levels of features that editors and IDEs provide. If you are getting startedprogramming,givetheIDLEeditoratry.Itisagreatplacetostartout.

Exercises

1. InstallPython3onyourcomputer.MakesureyoucanstartPython.2. Ifyouareusedtoaparticulareditor,dosomeinvestigationintoitssupport

forPython.Forexample,doesit:

DosyntaxhighlightingofPythoncode?RunPythoncodeinaREPLforyou?ProvideadebuggerforsteppingthroughyourPythoncode?

1-https://www.python.org/download

2-https://www.anaconda.com/download/3-https://brew.sh/4 -REPLstands forRead,Evaluate,Print, andLoop.Youwill soon seean

exampleusingit.

TheInterpreter

PYTHONISCOMMONLYCLASSIFIEDASANINTERPRETEDLANGUAGE.ANOTHERTERMUSEDto describe an interpreted language is scripting language. To run a computerprogramontheCPU,theprogrammustbeinaformatthattheCPUunderstands,namelymachinecode.Interpretedlanguagesdonotcompiledirectlytomachinecode,instead,thereisalayerabove,aninterpreterthatperformsthisfunction.

Differencebetweenacompiledlanguageandaninterpretedlanguage.Acompilerrunstocreateanexecutable.Aninterpreterisanexecutablethatloadscodeandrunsitontopofitself.

There are pros and cons to this approach. As you can imagine, on the flytranslatingcanbe timeconsuming.InterpretedcodelikePythonprogramstend

to runon theorderof10–100 timesslower thanCprograms.On the flipside,writingcodeinPythonoptimizesfordeveloper time.It isnotuncommonforaPython program to be 2–10 times shorter than its C equivalent. Also, acompilation step can be time consuming and actually a distraction duringdevelopmentanddebugging.Manydevelopersandcompaniesarewillingtoaccept this trade-off.Smaller

programs (read fewer lines of code) take less time to write and are easier todebug.Programmerscanbeexpensive—ifyoucanthrowhardwareataproblem,itcanbecheaperthanhiringmoreprogrammers.Debugging10linesofcodeiseasierthandebugging100linesofcode.Studieshaveshownthatthenumberofbugsfoundincodeisproportionaltothenumbersoflinesofcode.Hence,ifalanguagepermitsyoutowritefewer linesofcodetoachieveagiventask,youwill likely have fewer bugs. Sometimes program execution speed is not thatimportantandPythonissufficientlyfastformanyapplications.Inaddition,thereareeffortstoimprovethespeedofPythoninterpreterssuchasPyPy5.

REPL

PythonhasaninteractiveinterpreterorREPL(ReadEvaluatePrintLoop).Thisisaloopthatwaitsuntilthereisinputtoreadin,thenevaluatesit(interpretsit),andprints out the result.Whenyou run thepython3 executableby itself, youlaunchtheinteractiveinterpreterinPython.Otherenvironments,suchasIDLE,alsoembedaninteractiveinterpreter.

TolaunchtheREPL,typepython3inthepromptanditwillstartaPythonsession

NOTE

This book generally starts Python 3 with the python3 executable. OnWindows,theexecutableisnamedpython.IfyouareonWindows,replacepython3 with python. On Unix systems you shouldn't have to changeanything.

Whenyou start the interpreter, itwill printout theversionofPython, someinformationaboutthebuild,andsomehintstotype.Finally,theinterpreterwillgiveyouaprompt,>>>.Another option is to start IDLE, the editor includedwithPython, by typing

python3-midlelib.idle.

TolaunchtheREPLinIDLE,clickontheIDLEiconortypepython3-midlelib.idle

NOTE

Some Linux distributions do not ship with all of the libraries from thePython standard library.This is annoying, but justified by the idea that aserver doesn't need libraries to create client side applications. As such,UbuntuandArch (amongothers)don'tprovide thegui librariesnecessaryforIDLEonthedefaultinstallation.Ifyouseeanerrorlike:

$python3-midlelib.idle**IDLEcan'timportTkinter.YourPythonmaynotbeconfiguredforTk.**

itisanindicationyouaremissingthetkinterlibrary.OnUbuntu,youneedtorun:

$sudoapt-getinstalltk-dev

OnArch,youneedtorun:

$sudopacman-Stk

AREPLexample

BelowisanexampletoshowwhytheRead,Evaluate,PrintLoopwasgiventhisname. Ifyou typedpython3 from thecommand line 6,or launched IDLE,youwillsee>>>.Type2+2likeshownbelowandhitenter:

$python3>>>2+24>>>

Intheaboveexample,python3wastyped,whichopenedtheinterpreter.Thefirst>>>couldbethoughtofasthereadportion.Pythoniswaitingforinput.2+

2istypedin,read,andevaluated.Theresultofthatexpression—4—isprinted.Thesecond>>> illustrates the loop,becausetheinterpreter iswaitingformoreinput.The REPL, by default, prints the result of an expression to standard out

(unless the result is None, which will be explained in a later chapter). ThisbehaviorisinconsistentwithnormalPythonprograms,wheretheprintfunctionmustbeexplicitlyinvoked.ButitsavesafewkeystrokeswhenintheREPL.

NOTE

The>>>promptisonlyusedonthefirstlineofeachinput.IfthestatementtypedintotheREPLtakesmorethanoneline,the...promptfollows:

>>>sum([1,2,3,4,5,...6,7])

Thesepromptsaredefinedinthesysmodule:

>>>importsys>>>sys.ps1'>>>'>>>sys.ps2'...'

Alaterchapterwillexplainwhatmodulesare.Fornow,knowthattherearevariablesthatdefinewhatthepromptslooklike.

TheREPLendsupbeingquitehandy.Youcanusetheinteractiveinterpreterto write small functions, to test out code samples, or even to function as acalculator. Perhapsmore interesting is to go the other way. Run your PythoncodeintheREPL.Yourcodewillrun,butyouwillhaveaccesstotheREPLtoinspectthestateofyourcode.(YouwillseehowtodothisinIDLEsoon).The >>> is a prompt. That is where you type your program. Type

print("helloworld")afterthe>>>andhittheenterkey.Makesuretherearenotanyspacesortabsbeforethewordprint.Youshouldseethis:

>>>print("helloworld")helloworld

If you see this, congratulations, you are writing Python. Consider yourselfinductedinto theworldofprogramming.Youhavejustrunaprogram—“helloworld”. Hello world is the canonical program that most people write whenencounteringanewlanguage.ToexittheREPLfromtheterminal,typequit().UnixusersmayalsotypeCtl-D.

NOTE

Programmingrequiresprecision. Ifyouwerenotcareful in typingexactlyprint("helloworld")youmighthaveseensomethinglikethis:

>>>print("helloworldFile"<stdin>",line1print("helloworld^SyntaxError:EOLwhilescanningstringliteral

Computers are logical, and if your logic does not make sense, thecomputercanwarnyou,performirrationally(oratleastwhatappearssotoyou), or stop working. Do not take it personally, but remember thatlanguageshaverules,andanycodeyouwritehastofollowthoserules.Inthepreviousexample, therules thatstates ifyouwant toprint texton thescreen you need to start and end the text with quotes was violated. AmissingquoteontheendofthelineconsequentlyconfusedPython.

IDLEwilltrytohighlightwheretheerroroccurred.Thehighlightfollowingworldissupposedtoindicatewherethesyntaxerroroccurred.Itisnormallyasalmoncolor.

Summary

AsPython is an interpreted language, it provides aREPL.This allows you toexplorefeaturesofPythoninteractively.Youdon'tneedtowritecode,compileit,andrunit.YoucanlaunchaREPLandstarttryingoutcode.For users who have used compiled languages, this might seem somewhat

novel.Giveitatrythough,itcanmakedevelopmenteasyandquick.Also,don'tbeafraidtotryoutcodefromtheREPL.IfindthatnewPythonuserstendtobetimidtousetheREPL.Don'tfeartheREPL!ThereareotherREPLsforPython.OnepopularoneisJupyter,whichpresents

aweb-basedREPL 7.StartusingtheREPL,thenyoucanjumpintoothermoreadvancedREPLs.

Exercises

1. OpenthePython3REPLandrun"helloworld".Reviewthechaptertoseewhatthisonelineprogramlookslike.

2. OpentheREPLIDLEandrun"helloworld".

5-https://www.pypy.org6 -From theWindows runmenu, typecmd (orWin-R) togetaprompt.On

Macs,youcanquicklylaunchaterminalbytypingcommand-space,thetypingTerminal and return. If you have installed Python 3, you should be able tolaunchitnowoneitherplatformbytypingpython3.7-https://jupyter.org/

RunningPrograms

WHILEUSINGTHEINTERACTIVEINTERPRETERCANBEUSEFULDURINGDEVELOPMENT,YOU(andothers)willwanttodeployyourprogramandrunitoutsideoftheREPL.InPython,thisiseasyaswell.TorunaPythonprogramnamedhello.py,openaterminal,gotothedirectorycontainingthatprogramandtype:

$python3hello.py

NOTE

Whenrunningacommandfromthecommandline,thisbookwillprecedethe command with a $. This will distinguish it from interpreter contents(>>>or...)andfilecontents(nothingprecedingthecontent).

NOTE

Thepreviouscommand,python3hello.py,willprobablyfailunlessyouhaveafilenamedhello.py.

In the previous chapter you used theREPL to run “helloworld”, howdoesonerunthesameprogramstandalone?Createafilenamedhello.pyusingyourfavoritetexteditor.Inyourhello.pyfiletype:

print("helloworld")

Save the file, go to its directory, andexecute the file (hereexecute and runhave the same meaning, i.e. type python3 before the file name, and let the

Pythoninterpreterevaluatethecodeforyou.)

NOTE

Typing python3 standalone launches the interpreter. Typing python3some_file.pyexecutesthatfile.

Ifyouweresuccessfulatrunninghello.py,itwouldprintouthelloworld.

RunningfromIDLE

YoucanalsoeditandrunprogramsfromIDLE.StartbylaunchingIDLE,eitherbyclickingontheapplicationicononyourcomputerorbytyping:

$python3-midlelib.idle

WhenIDLElaunches,youseetheShellwindow.ThisisaPythonREPL.YoucantypecodeandPythonwillimmediatelyevaluateit.TocreateaprogramyouneedtocreateafilewithPythoncodeinit.Todothat,clickonthe“File”menu,andselect“NewFile”.Youshouldseeanewwindowpopup.ThiswindowisnotaShellwindow,itisaneditorwindow.YouwillnoticethatitisemptyandthereisnoPythonpromptinit.Inthiswindow,youwilltypePythoncode.

IDLEhasbeenlaunchedandaneweditorwindow(Untitledbecauseithasnotbeensaved)hasbeenopened

Type your code into the window. Since you are doing the hello worldexample,type:

print("helloworld")

You'llnoticethatIDLEusesadifferentcolorforprintand"helloworld".Thisiscalledsyntaxhighlightingandismeanttoaidinunderstandingyourcode.Now,youneedtorunthecode.TheeasiestwaytodothisistohittheF5keyonyourkeyboard.Alternatively,youcanclickon“Run”,andselect“RunModule”.IDLEwillaskyoutosaveyourfile.Saveitashello.py. IDLEwillbringtheshellwindowtofocus,printalinestatingthatithasrestartedtheshell,andprintouthelloworld.

Codetypedintotheeditorwindow.

Thismightseemtrivial,buttheshellwindownowhasthestateofyourcode.Inthiscase,youonlyprintedtothescreen,sothereisnotmuchstate.Infutureexamples,youwillseehowyoucanusetheintegrationbetweentheeditorandshell to provide an environment where you can quickly try out code, see theresults,andinspecttheoutputofaprogram.

ResultofrunningHelloWorldfromIDLE

If you need help navigating from the command prompt check out theappendix.

Unixyembellishments

OnUnixplatforms(LinuxandOSXamongothers),filessuchashello.pyareoftenreferredtoasscripts.Ascript isaprogram,but thetermisoftenusedtodistinguishnativecodefrominterpretedcode.Inthiscase,scriptsareinterpretedcode,whereastheoutputfromthecompilationstepofalanguagethatcompilestomachinecode(suchasC)isnativecode.

NOTE

Itisnotuncommontohearaboutshellscripts,Perlscripts,Pythonscripts,etc.WhatisthedifferencebetweenaPythonscriptandaPythonprogram?Nothing, really it is only semantics. A Python script usually refers to aPythonprogramrunfromthecommandline,whereasaPythonprogramisany programwritten inPython (which run the gamut of small 1-liners tofancyGUIapplications,to“enterprise”classservices).

Unixenvironmentsprovideahandywaytomakeyourscriptexecutableonitisown.Byputtingahashbang(#!)onthefirstlineofthefile,followedbythepath to the interpreter, andbychanging theexecutablebiton the file,youcancreateafilethatcanrunitself.To have the script execute with the Python interpreter found in the

environment,updateyourhello.pyfileto:

#!/usr/bin/envpython3print("helloworld")

NOTE

Thisnewfirstlinetellstheshellthatexecutesthefiletoruntherestofthefilewith the#!/usr/bin/envpython3 executable. (Shell scripts usuallystartwith#!/bin/bashor#!/bin/sh.)Savehello.pywiththenewinitialline.

TIP

#!/usr/bin/env is a handy way to indicate that the first python3executable found on your PATH environment variable should be used.Because thepython3executable is located indifferentplacesondifferentplatforms, thissolutionturnsout tobecross-platform.NotethatWindowsignores this line.Unlessyouareabsolutelycertain thatyouwant to runaspecificPythonversion,youshouldprobablyuse#!/usr/bin/env.Usinghardcodedhashbangssuchas:

#!/bin/python3

#!/usr/bin/python3.3

mightworkfineonyourmachine,butcouldleadtoproblemswhenyoutrytoshareyourcodeandothersdonothavepython3whereyouspecified

it.IfyourequireaspecificversionofPython,itiscommontospecifythatintheREADMEfile.

Now you need to make the file executable. Open a terminal, cd to thedirectorycontaininghello.pyandmakethefileexecutablebytyping:

$chmod+xhello.py

This sets theexecutablebit on the file.TheUnix environment hasdifferentpermissions (set by flipping a corresponding bit) for reading, writing, andexecutingafile.Iftheexecutablebitisset,theUnixenvironmentwilllookatthefirstlineandexecuteitaccordingly,whenthefileisrun.

TIP

If you are interested in knowingwhat thechmod command does, use theman(manual)commandtofindoutbytyping:

$manchmod

Nowyoucanexecute the fileby typing itsname in the terminalandhittingenter.Type:

$./hello.py

And your program (or script) should run. Note the ./ included before thenameoftheprogram.Normallywhenyoutypeacommandintotheterminal,theenvironment looksforanexecutable in thePATH (anenvironmentvariable thatdefinesdirectorieswhereexecutablesreside).Unless.(ortheparentdirectoryofhello.py)isinyourPATHvariableyouneedtoinclude./beforethename(orthefullpathtotheexecutable).Otherwise,youwillgetamessagelikethis:

$hello.pybash:hello.pycommandnotfound

Yes,all thatwork just toavoid typingpython3hello.py.Why?Themainreason is thatyouwantyourprogram tobenamedhello (without the trailing.py).AndperhapsyouwanttheprogramonyourPATHsoyoucanrunitatanytime.Bymakingafileexecutable,andaddingahashbang,youcancreateafilethatlookslikeanordinaryexecutable.Thefilewillnotrequirea.pyextension,norwillitneedtobeexplicitlyexecutedwiththepython3command.

Summary

Running Python programs is easy. There is no lengthy compilation step.Youneed topointPython to theprogramyouwould like to run.ManyeditorsalsohavetheabilitytorunPythoncode.Itisworthwhiletoinvestigatehowtodoitwithyoureditor.WithIDLE,itissimple:youhitF5.

Exercises

1. Createafilehello.pywiththecodefromthischapterinit.2. Runhello.pyfromaterminal.3. Runhello.pyfromwithinIDLE.4. Ifyouhaveanothereditorthatyouprefer,runhello.pyfromit.5. If you are on a Unix platform, create a file called hello. Add the hello

worldcode to it, andmake theappropriateadjustments such thatyoucanrunthecodebytyping:

./hello

WritingandReadingData

PROGRAMSWILLTYPICALLYHAVE INPUTANDOUTPUT.THISCHAPTERWILLSHOWHOWtoprintvaluestothescreenandallowtheendusertotypeinavalue.InPython,bothofthesearereallystraightforward.

Simpleoutput

The easiestway to provide the userwith output is to use theprint function,which writes to standard out. Standard out is where the computer writes itsoutput.Whenyouareinaterminal,standardoutisprintedontheterminal

>>>print('Hellothere')Hellothere

If youwant to print outmultiple items, you can provide them separated bycommas. Python will insert a space between them. You can put strings andnumbersinaprintfunction:

>>>print('Iam',10,'yearsold')Iam10yearsold

Alaterchapterwilllookatstringsindetail.Itwilldiscusshowtoformatthemtogetoutputtolookacertainway.

Gettinguserinput

Thebuilt-ininputfunctionwillreadtextfromaterminal.Thisfunctionacceptstextwhichitprintsoutasaprompttothescreenandwaitsuntil theusertypessomethingonstandardinandhitsenter.Standardiniswherethecomputerreadsitsinput.Inaterminal,standardinputcanbereadfromwhatyoutypein:

>>>name=input('Enteryourname:')

If you typed the above into the interpreter (the spaces around the= are notrequired but convention suggests that you type them tomake your codemorereadable), it might look like your computer is frozen. In reality, Python iswaitingforyoutotypeinsomeinputandhitenter.Afteryoutypesomethinginandpressenter,thevariablenamewillholdthevalueyoutyped.TypethenameMatt andpress theenterkey. Ifyouprintname itwillprint thevalueyou justtyped:

>>>print(name)Matt

NOTE

Thevalueenteredintotheterminalwheninputiscalledisalwaysastring.If you tried to performmath operations on it, it might not give you theansweryouwant:

>>>value=input('Enteranumber:')3>>>other=input('Enteranother:')4

Ifyoutrytoaddvalueandotherrightnow,youconcatenatethem(orjointhemtogether)becausetheyarestrings:

>>>type(value)<class'str'>>>>value+other'34'

If youwant to add these strings as if theywere numbers you need tochange them from a string into a number type. To convert a string toanothertypelikeaninteger(awholenumber)orfloat(adecimalnumber),youwillneedtousetheintandfloatfunctionsrespectively.If youwant to numerically addvalue andother, youhave to convert

themtonumbersusingint:

>>>int(value)+int(other)7

7

Afuturechapterwilltalkmoreaboutstringsandnumbertypes.

Summary

Pythongivesyoutwofunctionsthatmakeitreallyeasytoprintdataouttothescreen and read input from the user. These functions are print and input.Rememberthatwhenyoucall theinput function,youwillalwaysgetastringback.

Exercises

1. CreatesomePythoncodethatwillpromptyoutoenteryourname.PrintoutHelloandthenthename.

2. Createaprogramthatwillaskauserhowoldtheyare.Printoutsometexttellingthemhowoldtheywillbenextyear.

Variables

NOW THAT YOU KNOW ABOUT RUNNING PROGRAMS VIA THE INTERPRETER (OR THEREPL) and the command line, it is time to start learning about programming.Variablesarethebasicbuildingblocksofcomputerprograms.VariablesareimportantinPython,becauseinthePythonworld,everythingis

anobject.(Thisisnotquitetrue,keywordsarenotobjects).Variablesallowyoutoattachnamestotheseobjectssoyoucanrefertotheminfuturecode.

Mutationandstate

Twoimportantprogrammingconceptsarestateandmutation.Statedealswithadigitalrepresentationofamodel.Forexample,ifyouwanttomodelalightbulb,you may want to store its current status—is it on or off? Other possiblyinterestingstatesyoucouldstoreincludethetypeofbulb(CFLorincandescent),wattage,size,dimmable,etc.Mutationdealswithchangingthestatetoanewordifferentstate.Forthelight

bulbexample,itcouldbeusefultohaveapowerswitchthattogglesthestateandchangesitfromofftoon.How is this related to variables?Remember that inPython everything is an

object.Objectshavestate,andmightbemutated.Tokeeptrackoftheseobjects,youusevariables.Onceyouhaveobjectsthatholdstateandaremutable,thenyouhaveopened

aworld of possibilities.You canmodel almost anything youwant if you candeterminewhatstateitneeds,andwhatactionsormutationsneedtoapplytoit.

Pythonvariablesareliketags

Variablesarethebuildingblocksofkeepingtrackofstate.Youmightthinkofavariableasalabelortag.Importantinformationistaggedwithavariablename.Tocontinuethelightbulbexample,supposethatyouwanttorememberthestateofyourlightbulb.Havingthatdataisonlyusefulifyouhaveaccesstoit.Ifyouwant toaccess itandkeep trackof itsstateyouneed tohaveavariable to tagthatdata.Herethestateofthebulbisstoredinavariablenamedstatus:

>>>status="off"

Thisrequiresalittlemoreexaminationbecausethereisabitgoingonthere.Startingfromtheright,thereistheword"off"surroundedbyquotes.Thisisastringliteral,orabuiltindatatypethatPythonhasspecialsyntaxfor.ThequotestellPython that thisobject isastring.SoPythonwillcreatea stringobject.Astringstorestextualdata—inthiscase,thelettersoff.Thisobjecthasafewpropertiesofinterest.First,ithasanid.Youcanthinkof

the idaswherePythonstores thisobject inmemory. Italsohasa type, in thiscase, a string. Finally, it has a value, here the value is'off', because it is astring.The= sign is theassignmentoperator inmanyprogramming languages.Do

not be afraid of these technical terms, they aremore benign than they appear.The assignment operator connects or binds together a variable name and itsobject.Itindicatesthatthenameontheleftofitisavariablethatwillholdtheobjectontheright.Inthiscase,thevariablenameisstatus.When Python creates a variable, it tells the object to increase its reference

count.Whenobjectshavevariablesorotherobjectspointingtothem,theyhavea positive reference count.When variables go away (an example iswhen youexitafunction,variablesinthatfunctionwillgoaway),thereferencecountgoesdown.When this countgoesdown to zero, thePython interpreterwill assumethatnoonecaresabouttheobjectanymoreandgarbagecollectsit.Thismeansitremovesitfromitsmemory,soyourprogramdoesn'tgetoutofcontrolanduseallthememoryofyourcomputer.

NOTE

If you want to inspect the reference count of an object, you can callsys.getrefcountonit:

>>>importsys>>>names=[]>>>sys.getrefcount(names)2

Do note that as this count may seem high, the documentation for thisfunctionstates:

Returnthereferencecountofobject.Thecountreturnedisgenerallyonehigherthanyoumightexpect,becauseitincludesthe(temporary)referenceasanargumenttogetrefcount().

Even though Python gives you this ability, typically you don't worryaboutthereferencecountandletPythonhandlecleaningupobjectsforus.

This is a feature of Python, and typically Python does this for youautomatically,withoutanypromptingfromauser.Inotherlanguages,youneedtomanuallytelltheprogramtoallocateanddeallocatememory.To drive the point home, reading the code again from the left this time.

statusisavariablethatisassignedtotheobjectthatPythoncreatedforus.Thisobjecthasatype,string,andholdsthevalueof"off".

Twostepsofanassignmenttoaliteral.First,Pythoncreatesanobject.Theobjecthasavalue,"off",atype,string,andanid(thelocationoftheobjectinmemory).Aftertheobjectiscreated,Pythonlooksforanyvariablenamedstatus.Ifitexists,Pythonupdateswhatobjectthevariableis

pointingto,otherwise,Pythoncreatesthevariableandpointsittotheobject.

Cattletags

Mygrandfatherhadacattleranch,soIwillpulloutanon-nerdyanalogy.Ifyouhaveacattleranchofanysize,itmakessensetohaveagoodforemanwhowillkeeptrackofyourcattle(yourinvestment).Onewaytokeeptrackofcattleistousecattletags.Thesesmalltagsattached

totheearofacowcanbeusedtoidentifyandtrackindividualcows.Topullthisbacktoprogramming,ratherthanrunningacattleranch,youare

managingaspectsofyourprogram.Aprogramcanholdmanydistinctpiecesofinformationthatyouwanttorememberorkeeptrackof.Thisinformationisthe

state.Forexample, ifyouneeded to trackdatapertinent tohumansyoumightwanttotrackage,address,andname.Just like how ranchers tag their cattle to keep track of them, programmers

createvariablestokeeptrackofdata.Lookattheexampleagain:

>>>status="off"

ThistellsPythontocreateastringwiththecontentsofoff.Createavariablenamedstatus, and attach it to that string. Later on,when you need to knowwhatthestatusis,youcanaskyourprogramtoprintitoutlikeso:

>>>print(status)off

It isentirelypossible tocreatestateandloseit to theether ifyouneglect toputit inavariable.It issomewhatuselesstocreateobjectsthatyouwouldnotuse, but again it is possible. Suppose you want to keep track of the bulb’swattage.Ifyouwrite:

>>>"120watt"

IttellsPythontocreateastringobjectwiththecontentof120watt.Thisisproblematicbecauseyouforgottoassignittoavariable.Now,youhavenowayof using this object. Python will only let you access data that is stored invariables,soitisimpossibleforyoutousethisitemnow.Objectsareaccessedbyusing theirvariablenames. If thiswas information thatyouneeded inyourprogram,abettersolutionwouldbethefollowing:

>>>wattage="120watt"

Lateron,inyourprogramyoucanaccesswattage,youcanprintitout,andyou can even assign another variable to it, or assignwattage to another newvalue (say if your incandescent bulb broke and you replaced it with an LEDbulb):

>>>incandescent=wattage>>>wattage="25watt">>>print(incandescent,wattage)

>>>print(incandescent,wattage)120watt25watt

Managing state is a core aspect of programming. Variables are onemechanismtomanageit.

Rebindingvariables

Muchlikecowtags,variables tend tostaywithanobject forawhile,but theyaretransferable.Pythonletsyoueasilychangethevariable:

>>>num=400>>>num='400'#nownumisastring

Intheaboveexamplenumwasoriginallypointingtoanintegerbutthenwastoldtopointtoastring.

Thisillustratesrebindingvariables.Variablescanbereboundtoanytype.Pythonmakesnoefforttopreventthisorcomplain.Whenanobjectnolongerhasanyvariablepointingtoit,itiscleanedup

byPython,orgarbagecollected.

NOTE

Thevariabledoesnotcareaboutthetype.InPython,thetypeisattachedtotheobject.

Thereisnolimittohowoftenyoucanchangeavariable.Butyoushouldbecarefulnottochangeavariableifyoustillneedaccesstotheolddata.Onceyou

removeallvariablesfromanobject,youareessentiallytellingPythontodestroy(garbagecollectisthepropergeekyterm)theobjectwhenithasthechance,tofreeupanyinternalmemoryitoccupies.

TIP

ThisisacasewherePythonallowsyoutodosomething,butyouprobablydon'twanttodoit inreallife.Justbecauseyoucanrebindavariabletoadifferenttype,doesn'tmeanyoushould.Changingthetypeofavariableisconfusing to you when you read your code later. It is also confusing toothers who are using your code. Don't use the same variable to point todifferenttypes.This is a point of confusion for those who are new to Python. They

sometimes reuse the same variable throughout their code because theymistakenlybelieve that itwill savememory.Asyouhaveseen this isnotthe case. The variable itself is very lightweight. The object is what usesmemory. Reusing a variable is not going to change howmemory on theobject is handled, but itwill be confusing to thosewho have to read thecodelater.

Namingvariables

Python is somewhat particular about namingvariables. It has conventions thatmost Python programmers follow. Some of these are enforced, some are not.Onethatisenforcedbytheinterpreterisavariableshouldnothavethenameofa keyword. The word break is a keyword and hence cannot be used as avariable.Youwill get aSyntaxError if you try to use it as a variable. Eventhoughthiscodelooksperfectlylegal,Pythonwillcomplain:

>>>break='foo'File"<stdin>",line1break='foo'^SyntaxError:invalidsyntax

SyntaxError:invalidsyntax

IfyoufindyourselfwithaSyntaxErrorthatlookslikenormalPythoncode,checkthatthevariablenameisnotakeyword.

NOTE

KeywordsarereservedforuseinPythonlanguageconstructs,soitconfusesPythonifyoutrytomakethemvariables.Themodulekeywordhasakwlistattribute,whichisalistcontainingall

thecurrentkeywordsforPython:

>>>importkeyword>>>print(keyword.kwlist)['False','None','True','and','as','assert','break','class','continue','def','del','elif','else','except','finally','for','from','global','if','import','in','is','lambda','nonlocal','not','or','pass','raise','return','try','while','with','yield']

AnothermethodforexaminingkeywordsintheREPListorunhelp().This puts you in a help utility in the REPL, from which you can typecommands(thataren'tPython).Thentypekeywordsandhitenter.Youcantype any of the keywords and it will give you some documentation andrelatedhelptopics.Toexitthehelputilityhitenterbyitself.

Additionalnamingconsiderations

In addition to the aforementioned rule about not naming variables afterkeywords,thereareafewbestpracticesencouragedbythePythoncommunity.Therulesaresimple—variablesshould:

belowercaseuseanunderscoretoseparatewordsnotstartwithnumbersnotoverrideabuiltinfunction

Hereareexamplesofvariablenames,bothgoodandbad:

>>>good=4>>>bAd=5#bad-capitalletters>>>a_longer_variable=6#thisstyleisfrownedupon>>>badLongerVariable=7#bad-startswithanumber>>>3rd_bad_variable=8File"<stdin>",line13rd_bad_variable=8^SyntaxError:invalidsyntax#bad-keyword>>>for=4File"<stdin>",line1for=4^SyntaxError:invalidsyntax#bad-builtinfunction>>>compile=5

TIP

RulesandconventionsfornaminginPythoncomefromadocumentnamed“PEP 8 – Style Guide for Python Code” 8. PEP stands for PythonEnhancementProposal,which is a communityprocess for documenting afeature, enhancement, or best practice for Python. PEP documents arefoundonthePythonwebsite.

NOTE

AlthoughPythonwillnotallowkeywordsasvariablenames, itwillallowyou touseabuiltin nameasavariable.Builtinsare functions, classes,orvariablesthatPythonautomaticallypreloadsforyou,soyougeteasyaccessto them.Unlikekeywords,Pythonwill let youuse a builtin as a variable

namewithoutsomuchasapeep.However,youshouldrefrainfromdoingthis,itisabadpractice.Using a builtin nameas a variablename shadows thebuiltin.Thenew

variable name prevents you from getting access to the original builtin.Doingsoessentially takes thebuiltinvariableandco-opts it foryouruse.Asaresult,accesstotheoriginalbuiltinmayonlybeobtainedthroughthe__builtins__module.But it ismuchbetter not to shadow it in the firstplace.Here is a list of Python’s builtins that you should avoid using as

variables:

>>>dir(__builtins__)

['ArithmeticError','AssertionError','AttributeError','BaseException','BlockingIOError','BrokenPipeError','BufferError','BytesWarning','ChildProcessError','ConnectionAbortedError','ConnectionError','ConnectionRefusedError','ConnectionResetError','DeprecationWarning','EOFError','Ellipsis','EnvironmentError','Exception','False','FileExistsError','FileNotFoundError','FloatingPointError','FutureWarning','GeneratorExit','IOError','ImportError','ImportWarning','IndentationError','IndexError','InterruptedError','IsADirectoryError','KeyError','KeyboardInterrupt','LookupError','MemoryError','NameError','None','NotADirectoryError','NotImplemented','NotImplementedError','OSError','OverflowError','PendingDeprecationWarning','PermissionError','ProcessLookupError','RecursionError','ReferenceError','ResourceWarning','RuntimeError','RuntimeWarning','StopAsyncIteration','StopIteration','SyntaxError','SyntaxWarning','SystemError','SystemExit','TabError','TimeoutError','True','TypeError','UnboundLocalError','UnicodeDecodeError','UnicodeEncodeError','UnicodeError','UnicodeTranslateError','UnicodeWarning','UserWarning','ValueError','Warning','ZeroDivisionError','','_build_class__','__debug__','__doc_',

'_import__','__loader__','__name_','_package__','__spec__','abs','all','any','ascii','bin','bool','bytearray','bytes','callable','chr','classmethod','compile','complex','copyright','credits','delattr','dict','dir','divmod','enumerate','eval','exec','exit','filter','float','format','frozenset','getattr','globals','hasattr','hash','help','hex','id','input','int','isinstance','issubclass','iter','len','license','list','locals','map','max','memoryview','min','next','object','oct','open','ord','pow','print','property','quit','range','repr','reversed','round','set','setattr','slice','sorted','staticmethod','str','sum','super','tuple','type','vars','zip']

TIP

Herearea fewbuiltins thatwouldbe temptingvariablenamesotherwise:dict,id,list,min,max,open,range,str,sum,andtype.

Summary

InPython, everything is anobject.Objectshold state,which is alsocalled thevalue.Tokeeptrackofobjectsyouusevariables.Pythonvariablesarelikecattletags, they are attached to the object and have a name. But the object has theimportantdata,thevalueandtypeofdata.This chapter alsodiscussed rebindingof variables.Python allowsyou todo

this,butyoushouldbecarefulnottochangethetypeofthevariable,asthatcanbeconfusingtoreadersof thesaidcode.Finally, thechapterdiscussednamingconventionsforPythonvariables.

Exercises

1. Create a variable,pi, that points to an approximation for the value of π.Create a variable, r, for the radius of a circle that has a value of 10.

Calculatetheareaofthecircle(πtimestheradiussquared).Youcandomultiplicationwith*andyoucansquarenumbersusing**.Forexample,3**2is9.

2. Createavariable,b,thatpointstothebaseofarectanglewithavalueof10.Createavariable,h,thatpointstotheheightofarectanglewithavalueof2.Calculatetheperimeter.Changethebaseto6andcalculatetheperimeteragain.

8-https://www.python.org/dev/peps/pep-0008/

MoreaboutObjects

THIS CHAPTERWILLDIVE INTOOBJECTSA LITTLE BITMORE. YOUWILL COVER THREEimportantpropertiesofobjects:

identitytypevalue

Identity

Identity at its lowest level refers to an object’s location in the computer’smemory.Pythonhasabuiltinfunctioncalledidthattellsyoutheidentityofanobject:

>>>name="Matt">>>id(name)140310794682416

When you type this, the identity of the string "Matt" will appear as140310794682416 (which refers to a location in theRAMof your computer).Thiswillgenerallyvaryfromcomputertocomputerandforeachtimeyoustarttheshell,buttheidofanobjectisconsistentacrossthelifetimeofaprogram.It is possible for a single cow to have two tags on its ears and it is also

possible for two variables to refer to the same object. If you want anothervariable—first—toalsorefertothesameobjectreferredtobyname,youcoulddothefollowing:

>>>first=name

Thisillustrateswhathappenswhenyoubindavariabletoanexistingvariable.Theybothpointtothesameobject.Notethatthisdoesnotcopythevariable!Alsonotethattheobjecthasavalue,

"Matt",atype,andanid.

ThistellsPythontogivethefirstvariablethesameidasname.Runningidoneitherofthetwovariableswillreturnthesameid:

>>>id(first)140310794682416>>>id(name)140310794682416

Whatisidentityusedfor?Actuallynotmuch.WhenyouprograminPython,youtypicallyarenotconcernedwithlow-leveldetailssuchaswheretheobjectislivingintheRAMofthecomputer.Butidentityisusedtoillustratewhenobjectsarecreatedand if theyaremutable. It isalsoused indirectly fordoing identitycheckswithis.Theisoperatorchecksforidentityequalityandvalidateswhetherornottwo

variablespointtothesameobject:

>>>firstisnameTrue

If you print eitherfirst orname at theREPL, itwill print the same valuebecausetheyarebothpointingtotheexactsameobject:

>>>print(first)Matt>>>print(name)

>>>print(name)Matt

IfIhadacattletag,Icouldtakeitoffofonecowandattachittoanother.Justlikeacattletag,youcantakeavariableandpointittoanewobject.Icanmakenamepointtoanewobject.Youwillseethattheidentityofnamehaschanged.Butfirstisstillthesame:

>>>name='Fred'>>>id(name)140310794682434>>>id(first)140310794682416

Type

Another property of an object is its type.Common types are strings, integers,floats, andbooleans.There aremanyotherkindsof types, andyoucan createyourownaswell.Thetypeofanobjectreferstotheclassofanobject.Aclassdefinesthestateofdataanobjectholds,andthemethodsoractionsthat itcanperform.Pythonallowsyoutoeasilyviewthetypeofanobjectwiththebuiltinfunction,type:

>>>type(name)<class'str'>

Thetypefunctiontellsyouthatthevariablenamepointstoastring(str).ThetablebelowshowsthetypesforvariousobjectsinPython.

OBJECT TYPE

String str

Integer int

Floatingpoint float

List list

Dictionary dict

Tuple tuple

function function

User-definedclass(subclassobject) type

Instanceofclass(subclassofclass) class

Builtinfunction builtin_functionormethod

type type

Duetoduck-typing,thetypefunctionisnotusedtoofrequently.Ratherthancheckifanobject isofacertaintypethatprovidesanoperation,normallyyoutryanddothatoperation.Sometimesthoughyouhavedataandneedtoconvertittoanothertype.This

iscommonwhenreadingdatafromstandardin.Typicallyitwouldcomeinasastring,andyoumightwanttochangeit intoanumber.Pythonprovidesbuiltinclasses,str,int,float,list,dict,andtuplethatconvert(orcoerce)totheappropriatetypeifneeded:

>>>str(0)'0'>>>tuple([1,2])(1,2)>>>list('abc')['a','b','c']

NOTE

Duck typing comes from a saying used in the 18th century to refer to amechanicalduck.MuchliketheTuringTest,thesayingwent:

Ifitlookslikeaduck,walkslikeaduckandquackslikeaduck,thenit’saduck

ButIpreferthesceneinMontyPythonandtheHolyGrail,wherealadyisdeterminedtobeawitchbecausesheweighsasmuchasaduck.(Ifthisis confusing go watch the movie, I find it enjoyable). The idea is thatbecauseshehadcharacteristics(herweight)thatwerethesameasaduck’s,shecouldbeconsideredawitch.

Pythontakesasimilarapproach.Ifyouwanttoloopoveranobject,youputitinaforloop.Youdon'tcheckfirsttoseeifitisalistorasubclassofa list,youjust loopover it. Ifyouwant touseaplusoperation,youdon'tcheck to see if the object is a number, or string (or another type thatsupports addition). If theoperation fails, that’s ok, it is an indication thatyouarenotprovidingthecorrecttype.Ifyouarefamiliarwithobject-orientedprogramming,ducktypingeases

the requirement for subclassing.Rather than inheritingmultipleclasses totake advantage of behaviors they provide, you need to implement theprotocols (usuallybydefiningamethodor two).Forexample, tocreateaclassthatadds,youneedtoimplementa.__add__method.Anyclasscandefinethatmethodandrespondtotheplusoperation.

Mutability

Another interesting property of an object is itsmutability. Many objects aremutablewhileothersare immutable.Mutableobjectscanchangetheirvalueinplace,inotherwords,youcanaltertheirstate,buttheiridentitystaysthesame.Objectsthatareimmutabledonotallowyoutochangetheirvalue.Instead,youcan change their variable reference to a new object, but this will change theidentityofthevariableaswell.In Python, dictionaries and lists aremutable types. Strings, tuples, integers,

and floats are immutable types. Here is an example demonstrating that theidentity of a variable holding an integerwill change if you change the value.First,youwillassignanintegertothevariableageandinspecttheid:

>>>age=1000>>>id(age)140310794682416

Noticethatifyouchangethevalueoftheinteger,itwillhaveadifferentid:

>>>age=age+1>>>id(age)140310793921824

140310793921824

Thisillustratesthatwhenyoutrytochangeaninteger,youwillnecessarilycreateanewinteger.Integersareimmutable,andyoucan'tchangetheirvalue.

Hereisanexampleofchangingalist.Youwillstartwithanemptylist,andexaminetheid.Notethatevenafteryouaddanitemtothelist,theidentityof

thelistisunchanged,henceitismutable.First,youwillcreatealistandlookattheid:

>>>names=[]>>>id(name)140310794682432

Now,addastringintothelist.Thereareafewthingstonote.Thereturnvalueofthe.appendmethoddidn'tshowanything(ie, it isnotreturninganewlist).Butifyouinspectthenamesvariable,youwillseethatthenewnameisinthere.Also,theidofthelistisstillthesame.Youhavemutatedthelist:

>>>names.append("Fred")>>>names['Fred']>>>id(name)140310794682432

Thisillustratesthatwhenyouappendanobjectintoalist,youchangethevalueofthelist.Thelistismutated.Youcanaddandremoveitemsfromit,buttheidoftheliststaysthesame.

Mutableobjects shouldnot beused forkeys indictionaries and canpresentproblemswhenusedasdefaultparametersforfunctions.

UsingIDLE

At thispoint, itwouldbegood to try thisout in IDLE(oryour favoriteeditorthathasREPLintegration).BecauseIDLEcomeswithaREPL,youcouldtypeinthepreviouscodeandinspectitfromthere.Butyoucanalsowritecode,runit, and then inspect it from theREPL.To try it, openanew file, and type thefollowingcodeintoit:

name="Matt"first=nameage=1000print(id(age))age=age+1print(id(age))names=[]print(id(names))names.append("Fred")print(id(names))

Savethisasafile,callitiden.py.Thenrunthefile.InIDLE,youneedtohitF5todothis. In theREPL,youshouldseefournumbersprintedout.Thefirsttwo shouldbe different, illustrating that an integer is immutable.The last twonumbersare thesame.Theyare thesamebecauseeven thoughthe list,names,wasmutated,theidisstillthesame.Thisbyitselfisnothingparticularlynovel.Theinterestingpartnowisthat ifyoutypedir() intheREPL,itwillshow

youthevariables.Youwillseethattheglobalvariablesfromiden.pyarenowavailable.

ThisillustratesrunningcodeandtheninspectingitfromtheREPLinIDLE.IfyoudonotuseIDLE,figureouthowtodothisfromyoureditor.

FromtheREPLinIDLEyouhaveaccesstoalltheglobalvariables.Youcaninspect name or names. You can even call functions or methods likenames.append("George").Theabilitytoinspectwhatjustrangivesyouthechancetoquicklyinspectthe

code,andtrythingsout.ItisnotuncommonforexperiencedPythondeveloperstowritecodeintheREPL,pasteitintoafile,re-runthefile,writemorecodeintheREPL,andcontinuewritingcodeinthismanner.

Summary

InPython,everythingisanobject.Objectshavethreeproperties:

AType-Indicateswhattheclassisfortheobject.AValue-Thedatathattheobjectholds.Whenyoutestifanobjectisequaltoanotherobject(with==),youarecheckingagainstthevalue.An Id - A unique id for the object. In the Python version found atwww.python.org, this is essentially the location inmemory of the object,

whichwillbeauniquevalue.Whenyoucheckwhether twoobjectshavethesameidentity(withis),youarecheckingwhethertheidisthesame.

Youalsoexaminedhowmutableobjects,suchaslists,canchangetheirvalue,whileimmutableobjects,likenumbersorstrings,cannotbechanged.

Exercises

1. Create a variable that points to a floating point number. Examine the id,type,andvalueofthatnumber.Updatethevariablebyadding20toit.Re-examinetheid,type,andvalue.Didtheidchange?Didthevaluechange?

2. Createavariablepointingtoanemptylist.Examinetheid,type,andvalueofthelist.Appendthenumber300tothelist.Re-examinetheid,type,andvalue.Didtheidchange?Didthevaluechange?

Numbers

THISCHAPTERWILLDISCUSSMANIPULATINGNUMBERSWITHPYTHON.INTEGERS(WHOLE

numberslike-1,5,or2000)andFloatingPoints(thecomputer’sapproximationofrealnumberslike.333,0.5or-1000.234)areavailableinPythonandprovideeasy numerical manipulation. Out of the box, Python provides support foraddition,subtraction,multiplication,division,power,modulo,andmore!Unlikeotherlanguages,inPython,everythingisanobject,includingnumbers.

Integersareofclassint:

>>>type(1)<class'int'>

Floatingpointnumbersareofclassfloat:

>>>type(2.0)<class'float'>

Everything has a level of precision. It’s up to the user to determine if it issufficientfortheircalculations.Python’sfloatsarerepresentedinternallyusingabinaryrepresentation(aspertheIEEE754standardforfloatingpointnumbers).Floats have a certain amount of precision and rounding errors are possible. Infact, one should expect rounding errors. (If you need more precision, thedecimalmoduleprovidesamoreprecisealbeitslowerimplementation).As aquick exampleof precision, examinewhat happenswhenyouperform

thisapparentlysimplesubtractionoperation:

>>>print(1.01-.99)0.020000000000000018

TIP

Ifyouareinterestedinunderstandingmoreaboutfloatsandhowcomputersrepresentthem,Wikipedia9hasmoreinformationonthesubject.

Addition

ThePythonREPLcanbeusedas a simple calculator. If youwant to add twointegers,typeintheexpression:

>>>2+68

NOTE

Themathexampleabovedidnotbindtheresulttoavariable.Forasimplecalculationprintingouttheresult totheterminalmaybesufficient.Ifyouneedtheresultafterthefact,thePythoninterpreterstoresthelastresultinavariablenamed_:

>>>2+68>>>result=_>>>result8

Notethataddingtwointegerstogetherresultsinaninteger.Likewise,youcanalsoaddtwofloatstogether:

>>>.4+.010.41000000000000003

This example illustrates once again that care is neededwhen using floatingpointnumbers,asyoucanloseprecision(therealresultwouldbe0.41).Whathappenswhenyouaddanintegerandafloat?

>>>6+.26.2

Pythondecidedthatbecauseyouareaddinganintegerandafloat,youneedfloatingpoint arithmetic. In this case,Python converts orcoerces,6 to a floatbehindthescenes,beforeaddingitto.2.Pythonhasgivenyoutheanswerbackasafloat.

NOTE

Ifyouhaveanoperation involving twonumerics,coerciongenerallydoestherightthing.Foroperationsinvolvinganintegerandafloat,theintegeriscoercedtoafloat.

NOTE

Coercion between strings and numerics does not occur with mostmathematicaloperations.Twoexceptionsarethestringformattingoperator,andthemultiplicationoperator.Whenyouuse%withastringontheleftside(theleftoperand)andany

object (including numbers) on the right side (the right operand), Pythonperformstheformattingoperator:

>>>print('num:%s'%2)num:2

Iftheleftoperandisastringandyouusethemultiplicationoperator,*,Pythonperformsrepetition:

>>>'Python!'*2'Python!Python!'>>>'4'*2'44'

NOTE

Explicit conversion can be donewith theint andfloat built-in classes.(Althoughtheselooklikefunctionstheyarereallyclasses):

>>>int(2.3)2>>>float(3)3.0

Subtraction

Subtraction is similar to addition. Subtraction of two integers or two floatsreturnsanintegerorafloatrespectively.Formixednumerictypes,theoperandsarecoercedbeforeperformingsubtraction:

>>>2-6-4>>>.25-0.20.04999999999999999>>>6-.25.8

Multiplication

Inmanyprogramminglanguages,the*(asterisk)isusedformultiplication.Youcanprobablyguesswhatisgoingtohappenwhenyoumultiplytwointegers:

>>>6*212

Ifyouhavebeenfollowingcarefully,youwillalsoknowwhathappenswhenyoumultiplytwofloats:

>>>.25*12.03.0

Andifyoumixthetypesoftheproductyouendupwithafloatasaresult:

>>>4*.31.2

Notethatthefloatresultintheseexamplesappearscorrect,thoughyoushouldbecareful,duetofloatingpointprecisionissues,nottoassumethatyouwouldalwaysbesolucky.

Division

InPython(likemanylanguages),the/(slash)symbolisusedfordivision:

>>>12/43.0

Python3alsoaddressedwhatmanyconsideredtobeawartinpriorversionsofPython.Theresultofdividingtwointegersisafloat:

>>>3/40.75

Previously,Pythonperformedintegerdivision.Ifyouwantthatbehavior,youcan use the// operator.What integer does Python use? The floor of the realresult—taketheanswerandrounddown:

>>>3//40

Modulo

The modulo operator (%) calculates the modulus. This is the same as theremainderofadivisionoperationwhentheoperatorsarepositive.Thisisusefulfordeterminingwhetheranumberisoddoreven(orwhetheryouhaveiteratedover1000items):

#remainderof4dividedby3>>>4%31>>>3%2#oddif1isresult1>>>4%2#evenif0isresult0

TIP

TIP

Be careful with themodulo operator and negative numbers.Modulo canbehavedifferently,dependingonwhichoperandisnegative.Itmakessensethatifyouarecountingdown,themoduloshouldcycleatsomeinterval:

>>>3%30>>>2%32>>>1%31>>>0%30

What should-1%3 be?Sinceyouare countingdown it should cycleoverto2again:

>>>-1%32

Butwhenyouswitchthesignofthedenominator,thebehaviorbecomesweird:

>>>-1%-3-1

Pythonguarantees that the signof themodulo result is the sameas thedenominator(orzero).Tofurtherconfuseyou:

>>>1%-3-2

Thetakeawayhereis thatyouprobablydonotwanttodomodulowithnegativenumbersonthedenominatorunlessyouaresurethatiswhatyouneed.

Power

Pythonalsogivesyouthepoweroperatorbyusing**(doubleasterisks).Ifyouwantedtosquare4(4isthebase,2istheexponent),thefollowingcodewilldo

it:

>>>4**216

Exponential growth tends to let numbers get large pretty quickly. Considerraising10tothe100thpower:

>>>10**10010000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000

Programsneedtouseacertainamountofmemorytostoreintegers.Becauseintegers are usually smaller numbers,Pythonoptimizes for them, to notwastememory.Underthecovers,itcancoercesystemintegerstolongintegerstostorelargernumbers.Python3doesthisautomaticallyforyou.Ananalogymightbeascale.Ifyouarealwaysweighingsmallamounts,you

mightwantasmallscale.Ifyoudealinsand,youwillprobablywanttoputthesandinabagtomakeiteasiertohandle.Youwillhaveabunchofsmallbagsthatyouwilluse.Butifyouoccasionallyneedtoweighlargeritemsthatdonotfiton thesmall scale,youneed topulloutabiggerscale,andabiggerbag. Itwouldbeawastetousethebiggerbagandscaleformanyofthesmalleritems.Similarly,Pythontriestooptimizestoragespaceforintegerstowardssmaller

sizes.When Python does not have enoughmemory (a small bag) to fit largerintegers in, itcoerces the integer intoa long integer.This isactuallydesirablebecause,insomeenvironments,yourunintoanoverflowerrorhere,wheretheprogramdies(orPac-Manrefusestogooverlevel255—sinceitstoredthelevelcounterinan8-bitnumber).

NOTE

Python includes the operator module which provides functions forcommonmathematicaloperations.Whenusingmoreadvancedfeaturesof

Python such as lambda functions or list comprehensions, this modulecomesinhandy:

>>>importoperator>>>operator.add(2,4)#sameas2+46

Orderofoperations

Whenyouareperformingmath,youdonotapplyalltheoperationsfromlefttoright.Youdothemultiplicationanddivisionbeforetheadditionandsubtraction.Computersworkthesameway.Ifyouwanttoperformaddition(orsubtraction)first,useparenthesestoindicatetheorderofoperations:

>>>4+2310>>>(4+2)318

Asillustratedintheexample,anythinginparenthesesisevaluatedfirst.

Otheroperations

The help section from the REPL is pretty useful. There is a topic calledNUMBERMETHODSthatexplainshowallofthenumberoperationswork.

Summary

Python has built-in support for the basic mathematical operations. Addition,subtraction,multiplication,anddivisionareallincluded.Inaddition,thepowerandmodulus operations are available. If you need to control which order theoperationsoccur,wrapparenthesesaroundtheoperationthatyouwanttohappenfirst.Ifyouneedasimplecalculator, rather thanopeningacalculatorapplication,

givePythonatry.Itshouldbemorethancapableformosttasks.

Exercises

1. Youslept for6.2,7,8,5,6.5,7.1,and8.5hours thisweek.Calculate theaveragenumberofhoursslept.

2. Is297divisibleby3?3. Whatis2raisedtothetenthpower?4. Wikipediadefinesleapyearsas:

Everyyear that is exactlydivisibleby four isa leapyear, except foryears that are exactly divisibleby100, but these centurial yearsareleapyearsiftheyareexactlydivisibleby400.Forexample,theyears1700,1800,and1900arenotleapyears,buttheyears1600and2000are.—https://en.wikipedia.org/wiki/Leap_year

WritePythoncode todetermine if 1800,1900, 1903,2000, and2002areleapyears.

9-https://en.wikipedia.org/wiki/Floating_point

Strings

STRINGSAREIMMUTABLEOBJECTSTHATHOLDCHARACTERDATA.ASTRINGCOULDHOLDasinglecharacter,aword,alineofwords,aparagraph,multipleparagraphs,orevenzerocharacters.Python denotes strings bywrapping themwith ' (single quotes), " (double

quotes),"""(tripledoubles)or'''(triplesingles).Herearesomeexamples:

>>>character='a'>>>name='Matt'>>>with_quote="Iain'tgonna">>>longer="""Thisstringhas...multiplelines...init""">>>latin='''Lorumipsum...dolor'''>>>escaped='Iain\'tgonna'>>>zero_chars=''>>>unicode_snake="Ilove\N{SNAKE}"

Noticethatthestringsalwaysstartandendwiththesamestyleofquote.Asillustrated in the with_quote example you can put single quotes inside of adouble-quoted string—andvice versa. Furthermore, if you need to include thesametypeofquotewithinyourstring,youcanescapethequotebyprecedingitwitha\ (backslash).Whenyouprintoutanescapedcharacterthebackslashisignored.

NOTE

Attentive readersmaywonder how to include a backslash in a string.Toincludeabackslashinanormalstring,youmustescapethebackslashwith...youguessedit,anotherbackslash:

>>>backslash='\\'

>>>backslash='\\'>>>print(backslash)\

NOTE

HerearethecommonwaystoescapecharactersinPython:

ESCAPESEQUENCE OUTPUT

\\ Backslash

\' Singlequote

\" Doublequote

\b ASCIIBackspace

\n Newline

\t Tab

\u12af Unicode16bit

\U12af89bc Unicode32bit

\N{SNAKE} Unicodecharacter

\o84 Octalcharacter

\xFF Hexcharacter

TIP

Ifyoudonotwanttouseanescapesequence,youcanusearawstringbypreceding thestringwithanr.Rawstringsarenormallyused twoplaces.Theyareusedinregularexpressions,wherethebackslashisalsousedasanescapecharacter.Youcanuseregularexpressionstomatchcharacters(suchas phone numbers, names, etc) from text. The re module in the Pythonstandard libraryprovides support for regular expressions.Raw strings arealsousedinWindowspathswherethebackslashisadelimiter.

Raw strings interpret the character content literally (ie. there is noescaping).Thefollowingillustratesthedifferencebetweenrawandnormalstrings:

>>>slash_t=r'\tText\\'>>>print(slash_t)\tText\\>>>normal='\tText\\'>>>print(normal)Text\

Pythonalsohasatriplequotingmechanismfordefiningstrings.Triplequotesare useful for creating strings containing paragraphs ormultiple lines. Triple-quoted strings are also commonly used in docstrings. Docstrings will bediscussedinthechapteronfunctions.Belowisanexampleofamulti-linetriple-quotedstring:

>>>paragraph="""Loremipsumdolor...sitamet,consecteturadipisicing...elit,seddoeiusmodtemporincididunt...utlaboreetdoloremagnaaliqua.Ut...enimadminimveniam,quisnostrud...exercitationullamcolaborisnisiut...aliquipexeacommodoconsequat.Duis...auteiruredolorinreprehenderitin...voluptatevelitessecillumdoloreeu...fugiatnullapariatur.Excepteursint...occaecatcupidatatnonproident,sunt...inculpaquiofficiadeseruntmollit...animidestlaborum."""

Anicebenefitofusingtriple-quotedstringsisthatyoucanembedsingleanddoublequotesinsideitwithoutescapingthem:

>>>"""Thisstringhasdouble"andsingle...quotes'insideofit"""'Thisstringhasdouble"andsingle\nquotes\'insideofit'

Unlesstheybuttuptotheendofthestring,thenyouwillneedtoescapethefinalquote:

>>>"""Hesaid,"Hello""""File"<stdin>",line1"""Hesaid,"Hello""""^SyntaxError:EOLwhilescanningstringliteral>>>"""Hesaid,"Hello\""""'Hesaid,"Hello"'

FormattingStrings

Storing strings in variables is nice, but being able to compose stringsof otherstringsandmanipulatethemisalsonecessary.Onewaytoachievethisistousestringformatting.InPython3,thepreferredwaytoformatstringsistousethe.formatmethod

ofstrings.Below,youtellPythontoreplace{}(aplaceholder)withthecontentsofnameorthestringMatt:

>>>name='Matt'>>>print('Hello{}'.format(name))HelloMatt

Anotherusefulpropertyof formatting is thatyoucanalso formatnon-stringobjects,suchasnumbers:

>>>print('I:{}R:{}S:{}'.format(1,2.5,'foo'))I:1R:2.5S:foo

Formatstringsyntax

Formatstringshaveaspecialsyntaxforreplacementfields.Ifanobjectispassedinto the format string, attributes can be looked up using .attribute_namesyntax.Thereisalsosupportforpullingindex-ableitemsoutbyusing[index]as well. The Python documentation refers to these as field names. The fieldnamescanbeempty,anameofakeywordargument,anumberofapositionalargument,orindexofalistordictionary(insquarebrackets):

>>>'Name:{}'.format('Paul')

>>>'Name:{}'.format('Paul')'Name:Paul'>>>'Name:{name}'.format(name='John')'Name:John'>>>'Name:{[name]}'.format({'name':'George'})'Name:George'

The curly braces ({ and}) can also contain an integer inside of them. Theinteger refers to the zerobasedpositionof the argumentpassed into.format.Below is an example of using the numbers of positional arguments in theplaceholders. The first argument to .format, 'Paul', is at position 0, thesecond,'George',isposition1,and'John'isat2:

>>>'Last:{2}First:{0}'.format('Paul','George',...'John')'Last:JohnFirst:Paul'

There is a whole language for formatting strings. If you insert a colonfollowing the fieldname, you canprovide further formatting information.Theformatisbelow.Anythinginsquarebracketsisoptional:

:[[fill]align][sign][#][0][width][grouping_option][.precision][type]

Thefollowingtablesliststhefieldsandtheirmeaning.

FIELD MEANING

fill Characterusedtofillinalign(defaultisspace)

align Alightoutput<(leftalign),>(rightalign),^(centeralign),or=(putpaddingaftersign)

sign Fornumbers+(showsignonbothpositiveandnegativenumbers,-(default,onlyonnegative),orspace(leadingspaceforpositive,signonnegative)

# Prefixintegers.0b(binary),0o(octal),or0x(hex)

0 Enablezeropadding

width Minimumfieldwidth

grouping_option Numberseparator,(usecommaforthousandsseparator),_(Useunderscoreforthousandsseparator)

.precision Forfloats(digitsafterperiod(floats),fornon-numerics(maxlength)

typeNumbertypeors(stringformatdefault)seeIntegerandFloatcharts

type

Thetablesbelowliststhevariousoptionsyouhaveforformattingintegerandfloatingpointnumbers.

INTEGERTYPES MEANING

b binary

c character-converttoUnicodecharacter

d decimal(default)

n decimalwithlocale-specificseparators

o octal

x hex(lower-case)

X hex(upper-case)

FLOATTYPES MEANING

e/E Exponent.Lower/upper-casee

f Fixedpoint

g/G General.Fixedwithexponentforlarge,andsmallnumbers(gdefault)

n gwithlocale-specificseparators

% Percentage(multipliesby100)

Someformatexamples

Hereareafewexamplesofusing.format.Toformatastringinthecenterof12characterssurroundedby*,use thecodebelow.* is the fillcharacter,^ is thealignfield,and12isthewidthfield:

>>>"Name:{:*^12}".format("Ringo")'Name:***Ringo****'

Next,youformatapercentageusingawidthof10,onedecimalplace,andthesignbeforethewidthpadding.=isthealignfield,+ensuresthatthereisalwaysasign(negativeorpositive),10.1arethewidthandprecisionfields,and%isthefloattype,whichconvertsthenumbertoapercentage:

>>>"Percent:{:=+10.1%}".format(-44/100)'Percent:-44.0%'

'Percent:-44.0%'

Belowareabinaryandahexconversion.Theintegertypefieldissettobandxrespectively:

>>>"Binary:{:b}".format(12)'Binary:1100'>>>"Hex:{:x}".format(12)'Hex:c'

NOTE

The.formatmethodonastringprovidesanalternativeforthe%operatorwhich issimilar toC’sprintf.The%operator isstillavailableandsomeuserspreferitasitrequireslesstypingforsimplestatementsandbecauseitissimilartoC.%s,%d,and%xarereplacedbytheirstring,integer,andhexvaluesrespectively.Herearesomeexamples:

>>>"Num:%dHex:%x"%(12,13)'Num:12Hex:d'>>>"%s%s"%('hello','world')'helloworld'

TIP

A great resource for formatting is in the built-in help documentation,availableintheREPL.Type:

>>>help()

Whichputsyouinthehelpmode,andgivesyouahelp>prompt.Thentype:

help>FORMATTING

Youcanscrollthroughhereandfindmanyexamples.Abarereturnfromthehelp>promptwillreturnyoutothenormalprompt.

Anotherresourceisfoundathttps://pyformat.info/.Thiswebsitecontainsmanyformattingexampleswithboth.formatandtheolder%operator.Thereisalsoanentryinhelpforstrings,locatedunderSTRINGS.

F-Strings

Python 3.6 introduced a new type of string, called f-string. If you precede astringwith anf, it will allow you to include code inside of the placeholders.Hereisabasicexample:

>>>name='matt'>>>f'Mynameis{name}''Mynameismatt'

Pythonwilllookintheplaceholderandevaluatethecodethere.Notethattheplaceholdercancontainfunctioncalls,methodcalls,oranyotherarbitrarycode:

>>>f'Mynameis{name.capitalize()}''MynameisMatt'

Youcanalsoprovideformatstringsfollowingacolon:

>>>f'Squarerootoftwo:{2**.5:5.3f}''Squarerootoftwo:1.414'

Summary

In this chapter, strings were introduced. Strings can be defined with variousdelimiters. Unlike other languages, which may distinguish between a stringdefined with " and one defined with ', Python makes no distinction. Triple-quotedstrings,however,mayspanmultiplelines.We also looked at the .format method and gave examples of formatting

strings.Finally,thechapterintroducedanewfeatureinPython3.6,f-strings.

Exercises

1. Create a variable, name, pointing to your name. Create another variable,age,holdinganintegervalueforyourage.Printoutastringformattedwith

bothvalues.IfyournamewasFredandagewas23itwouldprint:

Fredis23

2. Createavariable,paragraph,thathasthefollowingcontent:

"Python is a great language!", said Fred. "I don't ever rememberhavingthismuchfunbefore."

3. Gotohttps://unicode.organdfindthesymbolomegaintheGreekcharactercodechart.Create a string thatholds theomegacharacter, usingboth theUnicodecodepoint(\uform)andUnicodename(\Nform).Thecodepointis the hex number in the chart, the name is the bolded capital namefollowing the code point. For example, the theta character has the codepointof03f4andanameofGREEKCAPITALTHETASYMBOL.

4. Make a variable, item, that points to a string, "car". Make a variable,cost, that points to 13499.99. Print out a line that has item in a left-justified area of 10 characters, and cost in a right-justified area of 10characters with 2 decimal places and commas in the thousands place. Itshouldlooklikethis(withoutthequotes):

'car13,499.99'

dir,help,andpdb

YOUHAVEONLYTOUCHEDTHESURFACEOFSTRINGS,BUTYOUNEEDTOTAKEABREAKtodiscuss twoimportantfunctionsandonelibrarythatcomewithPython.Thefirstfunctionisdir,whichillustrateshowpowerfulandusefultheREPLis.Thedir functionreturnstheattributesofanobject. IfyouhadaPythoninterpreteropen and wanted to knowwhat the attributes of a string are, you can do thefollowing:

>>>dir('Matt')['__add__','__class__','__contains__','__delattr__','__dir__','__doc__','__eq__','__format__','__ge__','__getattribute__','__getitem__','__getnewargs__','__gt__','__hash__','__init__','__iter__','__le__','__len__','__lt__','__mod__','__mul__','__ne__','__new__','__reduce__','__reduce_ex__','__repr__','__rmod__','__rmul__','__setattr__','__sizeof__','__str__','__subclasshook__','capitalize','casefold','center','count','encode','endswith','expandtabs','find','format','format_map','index','isalnum','isalpha','isdecimal','isdigit','isidentifier','islower','isnumeric','isprintable','isspace','istitle','isupper','join','ljust','lower','lstrip','maketrans','partition','replace','rfind','rindex','rjust','rpartition','rsplit','rstrip','split','splitlines','startswith','strip','swapcase','title','translate','upper','zfill']

dirlistsalltheattributesoftheobjectpassedintoit.Sinceyoupassedinthestring'Matt'todir,thefunctiondisplaystheattributesofastring.Thishandyfeature of Python illustrates its “batteries included” philosophy. Python givesyouaneasymechanismtodiscovertheattributesofanyobject.Otherlanguages

might require special websites, documentation or IDEs to access similarfunctionality.But in Python, because you have theREPL, you can get at thisinformationquicklyandeasily.Theattributelistisinalphabeticalorder,andyoucannormallyignorethefirst

couple of attributes startingwith__. Later on, youwill see attributes such ascapitalize(whichisamethodthatcapitalizesastring),format(whichaswasillustrated previously, allows for formatting of strings), or lower (which is amethod used to ensure the string is lowercase). These attributes happen to bemethods,which are functions that are attached to objects.To call, or invoke afunction, you place a period after the object, then the method name, thenparentheses.Thethreemethodsareinvokedbelow:

>>>print('matt'.capitalize())Matt>>>print('Hi{}'.format('there'))Hithere>>>print('YIKES'.lower())yikes

Dundermethods

Youmightbewonderingwhatalltheattributesstartingwith__are.Peoplecallthemspecialmethods,magicmethods,ordundermethodssincetheystart(andend)withdoubleunderscores(DoubleUNDERscores).“Dunderadd”isonewaytosay__add__,“theaddmagicmethod”isanother.Specialmethodsdeterminewhathappensunderthecoverswhenoperationsareperformedonanobject.Forexample, when you use the + or % operator on a string, the .__add__ or.__mod__methodisinvokedrespectively.Beginner Pythonistas can usually ignore dunder methods. When you start

implementingyourownclassesandwantthemtoreacttooperationssuchas+or%,youcandefinethem.

TIP

In thehelp()documentation isanentry,SPECIALMETHODS, thatdescribesthesemethods.AnotherplacewherethesearedescribedisonthePythonwebsite.Goto

Documentation, Language Reference, Data Model. It is tucked away inthere.

help

helpisanotherbuiltinfunctionthatisusefulincombinationwiththeREPL.Thebook previouslymentioned invokinghelp(),without any arguments, to bringuphelpdocumentation.Thehelpfunctionalsoprovidesdocumentationforamethod,module,class,

orfunctionifyoupasstheminasanargument.Forexample,ifyouarecuriouswhat the attribute upper on a string does, the following gives you thedocumentation:

>>>help('somestring'.upper)Helponbuiltinfunctionupper:upper(...)methodofbuiltins.strinstanceS.upper()->strReturnacopyofSconvertedtouppercase.

The help function, combined with the REPL, allows you to read up ondocumentationwithouthavingtogotoabrowser,orevenhaveinternetaccess.If you were stranded on a desert island, you should be able to learn Python,providedyouhadacomputerwithPythoninstalledandapowersource.

pdb

Pythonalso includesadebugger to step throughcode. It is found inamodulenamedpdb.ThislibraryismodeledafterthegdblibraryforC.TodropintothedebuggeratanypointinaPythonprogram,insertthecode:

importpdb;pdb.set_trace()

importpdb;pdb.set_trace()

These are two statements here, but I typically type them in a single lineseparated by a semicolon—that way I can easily remove them with a singlekeystrokefrommyeditorwhenIamdonedebugging.ThisisalsoabouttheonlyplaceIuseasemicoloninPythoncode(twostatementsinasingleline).Whenthislineisexecuted,itwillpresenta(pdb)prompt,whichissimilarto

theREPL.Codecanbeevaluatedatthispromptandyoucaninspectobjectsandvariablesaswell.Also,breakpointscanbesetforfurtherinspection.Belowisatablelistingusefulpdbcommands:

COMMAND PURPOSEh,help Listthecommandsavailable

n,next Executethenextline

c,cont,continue Continueexecutionuntilabreakpointishit

w,where,bt Printastacktraceshowingwhereexecutionis

u,up Popupalevelinthestack

d,down Pushdownalevelinthestack

l,list Listsourcecodearoundcurrentline

NOTE

InProgrammingPearls,JonBentleystates:

WhenIhavetodebugalittlealgorithmdeepinsideabigprogram,Isometimesusedebuggingtools... though,printstatementsareusuallyfastertoimplementandmoreeffectivethansophisticateddebuggers.

I've heard Guido van Rossum, the creator of Python, voice the sameopinion:heprefersprintdebugging.Printdebuggingiseasy,simplyinsertprint functions to provide clarity as to what is going on. This is oftensufficient to figure out a problem. Make sure to remove these debug

statementsorchangethemtologgingstatementsbeforereleasingthecode.Ifmoreexplorationisrequired,youcanalwaysusethepdbmodule.

Summary

Python provides many tools to make your life easier. If you learn to use theREPL,youcantakeadvantageofthem.Thedirfunctionwillhelpyouseewhatthe attributesof anobject are.Then,youcanuse thehelp function to inspectthoseattributesfordocumentation.Thischapteralsointroducedthepdbmodule.Thismoduleallowsyoutostep

throughcode,whichcanbeusefulfordebugging.

Exercises

1. OpenaREPL,andcreateavariable,name,withyourname in it.List theattributesofthestring.Printoutthehelpdocumentationforthe.findand.titlemethods.

2. Open a REPL, and create a variable, age, with your age in it. List theattributes of the integer. Print out the help documentation for the.numeratormethod.

StringsandMethods

INTHEPREVIOUSCHAPTERYOULEARNEDABOUTTHEBUILT-IN DIRFUNCTIONANDSAWsomemethods you can call on string objects. Because strings are immutable,thesemethodsdonotmutatethestring,butratherreturnanewstringoranewresult. Strings allow you create a new version that is capitalized, return aformattedstring,orcreatealowercasestring,aswellasmanyotheractions.Youdothisbycallingmethods.Methodsarefunctionsthatarecalledonaninstanceofatype.Whatdoesthis

mean? The string type allows you to call a method (another term for call isinvoke)byplacinga.(period)andthemethodnamedirectlyafterthevariablename holding the data (or the data itself), followed by parentheses withargumentsinsideofthem.

NOTE

Inthisbook,Iplaceaperiodinfrontofmethods.Thisismeanttoremindyouthatyouneedtohaveanobjectbeforethemethod.Iwillmentionthe.capitalize method, rather than saying capitalize. The invocationlookslikethisonthetextobject:

text.capitalize()

This is in contrast to a function, likehelp,whichyou invokeby itself(thereisnoobjectorperiodbeforeit):

help()

Illustrationofcallingamethodonastring.Themethoddoesnotchangethestringbecauseitisimmutable.Rather,themethodreturnsanewstring.

Hereisanexampleofcallingthe.capitalizemethodonavariablepointingtoastringandastringliteral.Notethatthisdoesnotchangetheobjectonwhichit is called. Because a string is immutable, the result of themethod is a newobjectwiththecapitalizedvalue:

>>>name='matt'#invokedonvariable>>>correct=name.capitalize()>>>print(correct)Matt

Notethatnamedoesnotchange:

>>>print(name)matt

Note,the.capitalizemethoddoesnothavetobecalledonavariable.Youcaninvokethemethoddirectlyonastringliteral:

>>>print('fred'.capitalize())Fred

In Python,methods and functions are first-class objects.Aswas previouslymentioned,everythingisanobject.Iftheparenthesesareleftoff,Pythonwillnotthrowanerror,itwillonlyshowareferencetoamethod,whichisanobject:

>>>print('fred'.capitalize)<built-inmethodcapitalizeofstrobjectat0x7ff648617508>

Having first-class objects enablesmore advanced features like closures anddecorators(thesearediscussedinmyintermediatePythonbook).

NOTE

Do integersand floatshavemethods?Yes,again,everything inPython isandobjectandobjectshavemethods.Thisiseasytoverifybyinvokingdironaninteger(oravariableholdinganinteger):

>>>dir(4)['__abs__','__add__','__and__','__class__',...'__subclasshook__','__truediv__','__trunc__','__xor__','conjugate','denominator','imag','numerator','real']

Invokingamethodonanumberissomewhattrickyduetotheuseofthe. to denote calling a method. Because . is common in floats, it wouldconfusePythonif.werealsousedtocallmethodsonnumbers.Forexample, the.conjugatemethodreturns thecomplexconjugateof

aninteger.Butifyoutrytoinvokeitonaninteger,youwillgetanerror:

>>>5.conjugate()Traceback(mostrecentcalllast):...5.conjugate()^SyntaxError:invalidsyntax

Onesolutiontothisistowrapthenumberwithparentheses:

>>>(5).conjugate()5

Anotheroptionwouldbetoassignavariableto5andinvokethemethodonthevariable:

>>>five=5>>>five.conjugate()5

However,inpractice,itisfairlyraretocallmethodsonnumbers.

Commonstringmethods

Hereareafewstringmethodsthatarecommonlyusedorfoundinthewild.Feelfreetoexploreothersusingdirandhelp(ortheonlinedocumentation).

endswith

If you have a variable holding a filename, you might want to check theextension.Thisiseasywith.endswith:

>>>xl='Oct2000.xls'>>>xl.endswith('.xls')True>>>xl.endswith('.xlsx')False

NOTE

Noticethatyouhadtopassinaparameter (orargument),'xls', intothemethod.Methods have a signature, which is a funkyway of saying thattheyneedtobecalledwiththecorrectnumber(andtype)ofparameters.Forthe.endswithmethod,itmakessensethatifyouwanttoknowifastringendswithanotherstringyouhavetotellPythonwhichendingyouwanttocheckfor.Thisisdonebypassingtheendstringtothemethod.

TIP

Again, it isusuallyeasytofindout thissortof informationviahelp.Thedocumentationshouldtellyouwhatparametersarerequiredaswellasanyoptionalparameters.Hereisthehelpforendswith:

>>>help(xl.endswith)Helponbuilt-infunctionendswith:endswith(...)S.endswith(suffix[,start[,end]])->boolReturnTrueifSendswiththespecifiedsuffix,Falseotherwise.Withoptionalstart,testSbeginningatthatposition.Withoptionalend,stopcomparingSatthatposition.suffixcanalsobeatupleofstringstotry.

Noticetheline:

S.endswith(suffix[,start[,end]])->bool

TheSrepresentsthestring(orinstance)youareinvokingthemethodon,in thiscase, thexlvariable..endswith is themethodname.Betweentheparentheses,( and), are theparameters.suffix is arequired parameter,the.endswithmethodwillcomplainifyoudonotprovideit:

>>>xl.endswith()Traceback(mostrecentcalllast):File"<stdin>",line1,in<module>

File"<stdin>",line1,in<module>TypeError:endswith()takesatleast1argument(0given)

The parameters between the square brackets [ and ] are optionalparameters.Inthiscase,startandendallowyoutoonlycheckaportionof the string. If you wanted to check if the characters starting at 0 andendingat3endswithOctyoucoulddothefollowing:

>>>xl.endswith('Oct',0,3)True

Stringsalsohavea.startswithmethod,sothecorrectwaytocheckifastringstartswith'Oct'is:

>>>xl.startswith('Oct')True

find

The.findmethodallowsyou tofindsubstrings insideotherstrings. It returnstheindex(offsetstartingat0)ofthematchedsubstring.Ifnosubstringisfounditreturns-1:

>>>word='grateful'#0isg,1isr,2isa>>>word.find('ate')2>>>word.find('great')-1

format

formatallowsforeasycreationofnewstringsbycombiningexistingvariables.Thechapteronstringsdiscussedthismethod:

>>>print('name:{},age:{}'.\...format('Matt',10))name:Matt,age:10

NOTE

NOTE

In the above example, the print function is spread across two lines. Byplacinga\followinga.youindicatetoPythonthatyouwanttocontinueonthenextline.Ifyouhaveopenedaleftparenthesis,(,youcanalsoplacetheargumentsonmultiplelineswithouta\:

>>>print("word"....find('ord'))1>>>print("word".find(...'ord'))1

Tohelpmake thecodemore readable, indent thecontinued lines.Notethat indenting with four spaces serves to indicate to anyone reading thecodethatthesecondlineisacontinuationofapreviousstatement:

>>>print("word".\...find('ord'))1>>>print("word".find(...'ord'))1

Whyspreadcodethatcouldresideinasinglelineacrossmultiplelines?Wherethiscomesintoplayinmostcodeisdealingwithcodestandardsthatexpect lines to be less than 80 characters in length. If a method takesmultiplearguments, itmaybehardtofollowthe80character limit.(NotethatPythonitselfdoesnotcareaboutlinelength,butreadersofyourcodemight). It is not uncommon to see a separate line for each argument to amethod:

>>>print('{}{}{}{}{}'.format(...'hello',...'to',...'you',...'and',...'you'...))hellotoyouandyou

join

Oftentimes you have a list (lists are discussed later in the book) of items andneedtoinsertsomethingbetweenthem.The.joinmethodcreatesanewstringfromasequencebyinsertingastringbetweeneverymemberofthelist:

>>>','.join(['1','2','3'])'1,2,3'

TIP

For most Python interpreters, using .join is faster than repeatedconcatenationusingthe+operator.Theaboveidiomiscommon.

lower

The.lowermethodreturnsacopyofthestringconvertedtolowercase.Theisoften useful for validating if the input matches a string. For example, someprogramscapitalizefileextensions,whileothersdonot.IfyouwantedtoknowifafilenamehadTXTortxtasanextension,youcoulddothefollowing:

>>>fname='readme.txt'>>>fname.endswith('txt')orfname.endswith('TXT')True

AmorePythonicversionwouldread:

>>>fname.lower().endswith('txt')True

startswith

The.startswithmethodisanalogousto.endswithexceptthatitchecksthatastringstartswithanotherstring:

>>>'Book'.startswith('B')True>>>'Book'.startswith('b')False

strip

The .strip method returns a new string that removes preceding and trailingwhitespace (spaces, tabs, newlines). This may come in handy if you have tonormalizedataorparseinputfromauser(ortheweb):

>>>'hellothere'.strip()'hellothere'

Notethatthreespacesatthefrontofthestringwereremovedaswerethetwoat the end. But the two spaces between thewordswere left intact. If you areinterestedinremovingonlytheleadingwhitespaceorrightmostwhitespace,themethodslstripandrstriprespectivelywillperformthoseduties.

upper

The.uppermethodisanalogousto.lower.Itreturnsacopyofthestringwithalloftheletterscapitalized:

>>>'yell'.upper()'YELL'

Othermethods

Thereareotherstringmethods,buttheyareusedlessoften.Feelfreetoexplorethembyreadingthedocumentationandtryingthemout.Theappendixprovidesalistofthem.

NOTE

The STRINGMETHODS entry in the help section from the REPL containsdocumentationforallofthestringmethodsaswellassomeexamples.

Summary

This chapter talked about methods. Methods are always called by putting anobject and a period before themethod name.You also looked at some of themore common methods of strings. One thing to remember is that a string is

immutable. If youwant to change a string’s value, you need to create a newstring.

Exercises

1. Createastring,school,withthenameofyourelementaryschool.Examinethemethodsthatareavailableonthatstring.Usethehelpfunctiontoviewtheirdocumentation.

2. Create a string, country, with the value 'usa'. Create a new string,correct_country, that has the value in uppercase, by using a stringmethod.

3. Createastring,filename,thathasthevalue'hello.py'.Checkandseeifthefilenameendswith'.java'.Findtheindexlocationof'py'.Seeifitstartswith'world'.

4. Open a REPL. Enter the help documentation and scan through theSTRINGMETHODSentry.

Comments,Booleans,andNone

THISCHAPTERWILLINTRODUCECOMMENTS,BOOLEANS,ANDNONE.COMMENTSHAVETHEpotentialtomakeyourcodemorereadable.ThebooleanandNonetypesareverycommonthroughoutPythoncode.

Comments

CommentsarenotatypepersebecausetheyareignoredbyPython.Commentsserve as reminders to the programmer. There are various takes on comments,theirpurpose,andtheirutility.Thereisacontinuumfromthosewhoareagainstanyandallcomments,tothosewhocommentalmosteverylineofcode,aswellas those who are in between. If you are contributing to a project, try to beconsistent with their commenting scheme. A basic rule of thumb is that acomment should explain thewhy rather than the how (code alone should besufficientforthehow).TocreateacommentinPython,startalinewitha#.Anythingthatfollowsthe

hashisignored:

>>>#ThislineisignoredbyPython

Youcanalsocommentattheendofaline:

>>>num=3.14#PI

TIP

Arogueuseofcomments is to temporarilydisablecodeduringediting.Ifyoureditorsupportsthis,itissometimeseasiertocommentoutcoderatherthanremoveitcompletely.Butthebestpracticeistoremovecommented-outcodebeforesharingthecodewithothers.

Otherlanguagessupportmulti-linecomments,butPythondoesnot.Theonlywaytocommentmultiplelinesistostarteverylinewith#.

TIP

You may be tempted to comment out multiple lines of code by makingthoselinesatriple-quotedstring.Thisisuglyandconfusing.Trynottodothis.

Booleans

Booleansrepresentthevaluesfortrueandfalse.Youhavealreadyseentheminpreviouscodeexamples,suchastheresultof.startswith:

>>>'bar'.startswith('b')True

Youcanalsoassignthosevaluestovariables:

>>>a=True>>>b=False

NOTE

TheactualnameofthebooleanclassinPythonisbool,notboolean:

>>>type(True)<class'bool'>

Itcanbeusefultoconvertothertypestobooleans.InPython,theboolclasscandothat.Convertingfromonetypetoanotheriscalledcasting.However,thisis usually unnecessary due to the implicit casting Python performs whenconditionals are evaluated. The conditional statement will do this casting foryou.

InPythonparlance, it is common tohearofobjectsbehavingas“truthy”or“falsey”—that means that non-boolean types can implicitly behave as thoughtheywerebooleans. Ifyouareunsurewhat thebehaviormightbe,pass in thetypetotheboolclassforanexplicitconversion(orcast).For strings, an empty string is “falsey”, while non-empty values coerce to

True:

>>>bool('')False>>>bool('0')#Thestringcontaining0True

Since a non-empty string behaves as truthy, you can testwhether the stringhascontent.Inthecodebelownamehasbeenset,butimaginethatitcamefromuserinput:

>>>name='Paul'>>>ifname:...print("Thenameis{}".format(name))...else:...print("Nameismissing")ThenameisPaul

Youdon'thavetotestifthenamehasalength.Sodon'tdothis:

>>>iflen(name)>0:...print("Thenameis{}".format(name))

Also,youdon'tneedtodothis:

>>>ifbool(name):...print("Thenameis{}".format(name))

becausePythonwillevaluatethecontentsoftheifstatement,andcoercetoabooleanforyou.BecauseastringisTruewhenithascontentyouonlyneed:

>>>ifname:...print("Thenameis{}".format(name))

NOTE

Thebuiltintypes,int,float,str,andbool,areclasses.Eventhoughtheircapitalization(lowercase)makesitlookasiftheywerefunctions,theyareclasses.Invokinghelp(str)willconfirmthis:

>>>help(str)Helponclassstrinmodulebuiltins:classstr(object)|str(object='')->str|str(bytes_or_buffer[,encoding[,errors]])->str|

This is one of those slight inconsistencies with Python. User-definedclasses typically follow PEP8, which suggests camel cased naming ofclasses.

For numbers, zero coerces to False while other numbers have “truthy”behavior:

>>>bool(0)False>>>bool(4)True

Whileexplicitcastingviatheboolfunctionisavailable,itisusuallyoverkill,because variables are implicitly coerced to booleanswhen used in conditionalstatements. For example, container types, such as lists and dictionaries, whenempty,behaveas“falsey”.Ontheflipside,whentheyarepopulatedtheyactas“truthy”.

TIP

Becarefulwhenparsingcontentthatyouwanttoturnintobooleans.Stringsthat are non-empty evaluate toTrue.One example of a string thatmightbiteyouisthestring'False'whichevaluatestoTrue:

>>>bool('False')True

Hereisatableoftruthyandfalseyvalues:

TRUTHY FALSEYTrue False

Mostobjects None

1 0

3.2 0.0

[1,2] [](emptylist)

{'a':1,'b':2} {}(emptydict)

'string' ""(emptystring)

'False'

'0'

TIP

Do not test boolean values to check if they are equal to True. Do notexplicitlycastexpressionstobooleanresults.Ifyouhaveavariable,done,containingaboolean,thisissufficient:

ifdone:#dosomething

Whilethisisoverkill:

ifdone==True:#dosomething

Asisthis:

ifbool(done):#dosomething

Similarly, if you have a list and need to distinguish between an emptyandnon-emptylist,thisissufficient:

members=[]ifmembers:

#dosomethingifmembers#havevalueselse:#memberisempty

Likewise, this test is superfluous. It is not necessary to determine thetruthinessofalistbyitslength:

iflen(members)>0:#dosomethingifmembers#havevalueselse:#memberisempty

NOTE

If youwish to define the implicit truthiness for user-defined objects, the.__bool__methodspecifiesthisbehavior.ItcanreturnTrue,orFalse.Ifthismagicmethod is not defined, the.__len__method is checked for anon-zerovalue.Ifneithermethodisdefined,anobjectdefaultstoTrue:

>>>classNope:...def__bool__(self):...returnFalse>>>n=Nope()>>>bool(n)False

Ifthisisconfusing,feelfreetocomebacktothisexampleafterreadingaboutclasses.

None

None isaninstanceofNoneType.Otherlanguageshavesimilarconstructssuchasnil,NULL orundefined.Variables can be assigned toNone to indicate thattheyarewaitingtoholdarealvalue.NonecoercestoFalseinabooleancontext:

>>>bool(None)False

NOTE

A Python function defaults to returning None if no return statement isspecified:

>>>defhello():...print("hi")>>>result=hello()hi>>>print(result)None

NOTE

None isasingleton (PythononlyhasonecopyofNone in the interpreter).Theidforthisvaluewillalwaysbethesame:

>>>a=None>>>id(a)140575303591440>>>b=None>>>id(b)140575303591440

AsanyvariablecontainingNoneisthesameobjectasanyothervariablecontaining None. You typically use is to check for identity with thesevariablesratherthanusing==tocheckforequality:

>>>aisbTrue>>>aisnotbFalse

isisfasterthan==andconnotestotheprogrammerthatidentityisbeingcomparedratherthanthevalue.Youcanputtheisexpressioninanifstatement:

>>>ifaisNone:...print("Aisnotset!")Aisnotset!

SinceNoneevaluatestoFalseinabooleancontextyoucouldalsodothefollowing:

>>>ifnota:...print("Aisnotset!")Aisnotset!

But,youshouldbecarefulasothervaluesalsoevaluatetoFalse,suchas0,[],or''(emptystring).CheckingagainstNoneisexplicit.

Summary

In this chapter, you learned about comments inPython.Comments are startedwith a hash, and any content following the hash until the end of the line isignored.Therearenomulti-linecomments.Thechapter alsodiscussedTrue,False, andbooleancoercion.Mostvalues

areTrueinabooleancontext(whenusedinanifstatement).TheFalsevaluesarezero,None,andemptysequences.Finally, the None object was mentioned. It is a singleton that is used to

indicatethatyouhaveavariablethatmaybeassignedavalueinthefuture.Itisalsotheresultofafunctionthatdoesnotexplicitlyreturnavalue.

Exercises

1. Create a variable,age, set to your age.Create another variable,old, thatuses a condition to testwhether you are older than 18.The value ofoldshouldbeTrueorFalse.

2. Create a variable, name, set to your name. Create another variable,second_half,thattestswhetherthenamewouldbeclassifiedinthesecondhalfofthealphabet?Whatdoyouneedtocompareitto?

3. Createalist,names,withthenamesofpeopleinaclass.Writecodetoprint'The class is empty!' or 'Class has enrollments.', based onwhethertherearevaluesinnames.(Seethetipinthischapterfordetails).

4. Createavariable,car,settoNone.Writecodetoprint'Taxiforyou!',or'Youhaveacar!',basedonwhetherornotcarisset(Noneisnotthenameofacar).

ConditionalsandWhitespace

IN THIS CHAPTER, YOUWILL LEARNMORE ABOUTMAKING COMPARISONS IN PYTHON.Most code needs tomake decisions aboutwhich path to execute, so youwilllookathowthisisdone.Inadditiontothebooleanvalues,TrueandFalse,inPython,youcanalsouse

expressionstogetbooleanvalues.Ifyouhavetwonumbers,youmightwanttocompare them to check if they are greater than or less than each other. Theoperators,>and<,dothisrespectively:

>>>5>9False

Hereisatableofcomparisonoperationstocreatebooleanvalues:

OPERATOR MEANING

> Greaterthan

< Lessthan

>= Greaterthanorequalto

<= Lessthanorequalto

== Equalto

!= Notequalto

is Identicalobject

isnot Notidenticalobject

Theseoperationsworkonmosttypes.Ifyoucreateacustomclassthatdefinestheappropriatemagicmethods,yourclasscanusethemaswell:

>>>name='Matt'>>>name=='Matt'True

>>>name!='Fred'True>>>1>3False

NOTE

The“richcomparison”magicmethods,__gt__,__lt__,__ge__,__le__,__eq__, and __ne__ correspond to >, <, >=, <=, ==, and != respectively.Defining all of these canbe somewhat tedious and repetitive. For classeswhere these comparisons are commonly used, thefunctools.total_ordering class decorator gives you all of thecomparison functionality as long as you define __eq__ and __le__. Thedecorator will automatically derive the remainder of the comparisonmethods.Otherwise,allsixmethodsshouldbeimplemented:

>>>importfunctools>>>@functools.total_ordering...classAbs(object):...def__init__(self,num):...self.num=abs(num)...def__eq__(self,other):...returnself.num==abs(other)...def__lt__(self,other):...returnself.num<abs(other)>>>five=Abs(-5)>>>four=Abs(-4)>>>five>four#notusinglessthan!True

Decoratorsareconsideredanintermediatesubjectandarenotcoveredinthisbeginningbook.

TIP

Theisandisnotstatementsareforcomparingidentity.Whentestingforidentity—if twoobjects are the same actual objectwith the sameid (not

justthesamevalue)—useisorisnot.SinceNoneisasingletonandonlyhasoneidentity,isandisnotareusedwithNone:

>>>ifnameisNone:...#initializename

Combiningconditionals

Conditionalexpressionsarecombinedusingbooleanlogic.Thislogicconsistsoftheand,or,andnotoperators.

BOOLEANOPERATOR MEANING

xandy BothxandymustevaluatetoTruefortrueresult

xory IfxoryisTrue,resultistrue

notx Negatethevalueofx(TruebecomesFalseandviceversa)

Belowisasimpleexampleforsettingagradebasedonascoreusingandtotestwhetherthescoreisbetweentwonumbers:

>>>score=91>>>ifscore>90andscore<=100:...grade='A'

NOTE

Pythonallowsyoutodotheaboveexampleusingarangecomparisonlikethis:

>>>if90<score<=100:...grade='A'

Either style works, but range comparisons are not common in otherlanguages.

Hereisanexampleforcheckingifagivennameisamemberofaband:

>>>name='Paul'>>>beatle=False

>>>beatle=False>>>ifname=='George'or\...name=='Ringo'or\...name=='John'or\...name=='Paul':...beatle=True...else:...beatle=False

NOTE

In theaboveexample the\at theendof"'George'or\" indicates thatthestatementwillbecontinuedonthenextline.Like most programming languages, Python allows you to wrap

conditional statements in parentheses. Because they are not required inPython,mostdevelopersleavethemoutunlesstheyareneededforoperatorprecedence.Butanothersubtletyofusingparenthesesisthattheyserveasahinttotheinterpreterwhenastatementisstillopenandwillbecontinuedonthenextline,hencethe\isnotneededinthatcase:

>>>name='Paul'>>>beatle=False>>>if(name=='George'or...name=='Ringo'or...name=='John'or...name=='Paul'):...beatle=True...else:...beatle=False

TIP

An idiomaticPythonversionof checkingmembership is listedbelow.Tocheck if a value is found across a variety of values, you can throw thevaluesinasetandusetheinoperator:

>>>beatles={'George','Ringo','John','Paul'}>>>beatle=nameinbeatles

Alaterchapterwilldiscusssetsfurther.

Hereisanexampleofusingthenotkeywordinaconditionalstatement:

>>>last_name='unknown'>>>ifname=='Paul'andnotbeatle:...last_name='Revere'

ifstatements

Booleans(TrueandFalse)areoftenusedinconditionalstatements.Conditionalstatementsareinstructionsthatsay“ifthisstatementistrue,performablockofcode, otherwise execute some other code.” Branching statements are usedfrequently in Python. Sometimes, the “if statement” will check values thatcontainbooleans,othertimesitwillcheckexpressionsthatevaluatetobooleans.Anothercommoncheckisforimplicitcoercionto“truthy”or“falsey”values:

>>>debug=True>>>ifdebug:#checkingaboolean...print("Debugging")Debugging

elsestatements

Anelsestatementcanbeusedincombinationwithanifstatement.ThebodyoftheelsestatementwillexecuteonlyiftheifstatementevaluatestoFalse.Hereisanexampleofcombininganelsestatementwithanifstatement.Theschoolbelowappearstohavegradeinflation:

>>>score=87>>>ifscore>=90:...grade='A'...else:...grade='B'

Notethattheexpression,score>=90,isevaluatedbyPythonandturnsintoaFalse. Because the “if statement”was false, the statements under the elseblockareexecuted,andthegradevariableissetto'B'.

Morechoices

The previous example does not work for most schools. You can add moreintermediatestepsifneededusingtheelifkeyword.elifisanabbreviationfor“elseif”.Belowisacompletegradingscheme:

>>>score=87>>>ifscore>=90:...grade='A'...elifscore>=80:...grade='B'...elifscore>=70:...grade='C'...elifscore>=60:...grade='D'...else:...grade='F'

Theif,elif,andelsestatementsaboveeachhavetheirownblock.PythonwillstartfromthetoptryingtofindastatementthatevaluatestoTrue,whenitdoes,itrunstheblockandthencontinuesexecutingatthecodefollowingalloftheelifandelseblocks.IfnoneoftheiforelifstatementsareTrue,itrunstheblockfortheelsestatement.

NOTE

The if statement can have zero or more elif statements. The elsestatementisoptional.Ifitexists,therecanonlybeone.

Whitespace

A peculiarity you may have noticed is the colon (:) following the booleanexpressionintheifstatement.Thelinesimmediatelyaftertheifstatementwereindentedbyfourspaces.TheindentedlinesaretheblockofcodethatisexecutedwhentheifexpressionevaluatestoTrue.Inotherlanguages,anifstatementmightlooklikethis:

if(score>=90){grade='A';}

Inmanyoftheselanguages,thecurlybraces,{and},denotetheboundariesof theifblock.Anycodebetween thesebraces isexecutedwhen the score isgreaterthanorequalto90.Python,unlikethoseotherlanguages,usestwothingstodenoteblocks:

acolon(:)indentation

Ifyouhaveprogrammedinotherlanguages,itiseasytolearnthewhitespacerules inPython.Allyouhave todo is replace the left curlybracket ({)withacolon(:)andindentconsistentlyuntiltheendoftheblock.

TIP

Whatisconsistent indentation?Normallyeither tabsorspacesareusedtoindentcode.ThePythoninterpreteronlycaresaboutconsistencyonaper-filebasis. It ispossible tohaveaprojectwithmultiple files thateachusedifferentindentationschemes,butthiswouldbesilly.InPython,usingfourspacesisthepreferredwaytoindentcode.Thisis

described inPEP8. Ifyoumix tabsandspaces in thesamefile,youwilleventuallyrunintoproblems.Although spaces are the preferred mechanism, if you are working on

code that already uses tabs, it is better to be consistent. In that case,continueusingtabswiththecode.Thepython3executablewillcomplainwithaTabErrorwhenyoumix

tabsandspaces.

Summary

This chapter discussed theif statement. This statement can be used to createarbitrarily complex conditionswhen combining expressionswithand,or, andnot.

Blocks, indentation, and whitespace were also discussed. Sometimes whenpeople encounter Python, the required whitespace rules may seem like anuisance. I've runacross suchpeople inmy trainings.Whenasked if andwhytheyindentcodeinotherlanguages,theyreply“Ofcourse,itmakescodemorereadable”.InPython,youemphasizereadability,andenforcingwhitespacetendstoaidinthat.

Exercises

1. Write an if statement to determinewhether a variable holding ayear is aleapyear.(SeetheNumberschapterexercisesfortherulesforleapyears).

2. Writeanifstatementtodeterminewhetheravariableholdinganintegerisodd.

3. Writeanifstatement.Lookattheindentedblockandcheckifyoureditorindented with tabs or spaces. If it is indented with tabs, configure youreditor to indentwith spaces. Some editors show tabs differently, if yoursdoesnotdistinguishbetweentabsandspaces,aneasywaytocheckif thespacing is a tab is to cursor over it. If the cursor jumps four or eightcharacters,thenitinsertedatabcharacter.

Containers:Lists,Tuples,andSets

MANY OF THE TYPES DISCUSSED SO FAR HAVE BEEN SCALARS,WHICH HOLD A SINGLEvalue.Integers,floats,andbooleansareallscalarvalues.Containersholdmultipleobjects(scalartypesorevenothercontainers).This

chapterwilldiscusssomeofthesetypes—lists,tuples,andsets.

Lists

Lists,asthenameimplies,areusedtoholdalistofobjects.InPython,alistmayholdanytypeofitemandmaymixitemtype.Thoughinpractice,youonlystorea single item type in a list.Anotherway to thinkof a list is that theygive anorder to a sequence of items.They are amutable type,meaning you can add,remove,andalterthecontentsofthem.Therearetwowaystocreateemptylists,oneistoinvokethelistclass,andtheotheristousethesquarebracketliteralsyntax—[and]:

>>>names=list()>>>other_names=[]

Ifyouwanttohaveprepopulatedlists,youcanprovidethevaluesinbetweenthesquarebrackets,usingtheliteralsyntax:

>>>other_names=['Fred','Charles']

NOTE

The list class can also create prepopulated lists, but it is somewhatredundantbecauseyouhavetopassalistintoit:

>>>other_names=list(['Fred','Charles'])

Typically,thisclassisusedtocoerceothersequencetypesintoalist.Forexample,astringisasequenceofcharacters.Ifyoupassastringintolist,yougetbackalistoftheindividualcharacters:

>>>list('Matt')['M','a','t','t']

Lists,likeothertypes,havemethodsthatyoucancallonthem(usedir([])toseeacompletelistofthem).Forexample,toadditemstotheendofalist,usethe.appendmethod:

>>>names.append('Matt')>>>names.append('Fred')>>>print(names)['Matt','Fred']

Rememberthatlistsaremutable.Pythondoesnotreturnanewlistwhenyouappendtoalist.Noticethatthecallto.appenddidnotreturnalist(theREPLdidn'tprintanythingout).RatheritreturnsNoneandupdatesthelistinplace.(InPython,thedefaultreturnvalueisNoneforafunctionoramethod.Thereisnowaytohaveamethodthatdoesn'treturnanything).

Sequenceindices

AlistisoneofthesequencetypesinPython.Sequencesholdorderedcollectionsof objects. To work effectively with sequences, it is important to understandwhatanindexis.Everyiteminalisthasanassociatedindex,whichdescribesitslocationinthelist.Forexample,theingredientsinpotatochipsarepotatoes,oil,andsalt,andtheyarenormallylistedinthatorder.Potatoesarefirstinthelist,oilissecond,andsaltisthird.Inmanyprogramminglanguages,thefirstiteminasequenceisatindex0,the

second item is at index1, the thirdat index2, and soon.Countingbeginningwithzeroiscalledzero-basedindexing.

Youcanaccessan item ina listusing thebracketnotationand the indexofsaiditem:

>>>names[0]'Matt'>>>names[1]'Fred'

Listinsertion

Toinsertanitematacertain index,usethe.insertmethod.Calling.insertwillshiftanyitemsfollowingthatindextotheright:

>>>names.insert(0,'George')>>>print(names)['George','Matt','Fred']

Thesyntaxforreplacementatanindexisthebracketnotation:

>>>names[1]='Henry'>>>print(names)['George','Henry','Fred']

Toappenditemstotheendofalistusethe.appendmethod:

>>>names.append('Paul')>>>print(names)['George','Henry','Fred','Paul']

NOTE

CPython’s underlying implementation of a list is actually an array ofpointers. This provides quick random access to indices. Also, appendingand removing at the end of a list is quick (O(1)), while inserting andremoving from themiddle of a list is slower (O(n)). If you find yourselfinserting and popping from the front of a list, collections.deque is abetterdatastructuretouse.

Listdeletion

Toremoveanitem,usethe.removemethod:

>>>names.remove('Paul')>>>print(names)['George','Henry','Fred']

Youcanalsodeletebyindexusingthebracketnotation:

>>>delnames[1]>>>print(names)['George','Fred']

Sortinglists

Acommonoperationonlistsissorting.The.sortmethodordersthevaluesinthelist.Thismethodsortsthelistinplace.Itdoesnotreturnanew,sortedcopyofthelist,ratheritupdatesthelistwiththeitemsreordered:

>>>names.sort()>>>print(names)['Fred','George']

Thisillustratessortingalistwiththe.sortmethod.Notethatthelistissortedin-place.Theresultofcallingthe.sortmethodisthelistismutated,andthemethodreturnsNone.

If the previous order of the list was important, you canmake a copy of itbefore sorting. A more general option for sorting sequences is the sortedfunction. Thesorted functionworkswith any sequence. It returns a new listthatisordered:

>>>old=[5,3,-2,1]>>>nums_sorted=sorted(old)>>>print(nums_sorted)[-2,1,3,5]>>>print(old)[5,3,-2,1]

Thisillustratessortingalistbyusingthesortedfunction.Notethatthelistisnotmodified,ratheranewlistiscreated.Also,notethatPythonreusedtheitemsofthelist.Itdidnotcreatenewitems.

Finalstepsshowingthatthevariableassignmentisusedtocreateavariablepointingtothenewlist.Notethatthesortedfunctionworkswithanysequence,notonlylists.

Becarefulaboutwhatyousort.Pythonwantsyoutobeexplicit.InPython3,whenyoutry tosorta list thatcontainsheterogeneous types,youmightgetanerror:

>>>things=[2,'abc','Zebra','1']>>>things.sort()Traceback(mostrecentcalllast):...TypeError:unorderabletypes:str()<int()

Boththe.sortmethodandsortedfunctionallowarbitrarycontrolofsortingbypassing ina function for thekeyparameter.Thekeyparametercanbeanycallable(function,class,method)thattakesasingleitemandreturnssomethingthatcanbecompared.

Inthisexample,bypassinginstrasthekeyparameter,everyiteminthelistissortedasifitwereastring:

>>>things.sort(key=str)>>>print(things)['1',2,'Zebra','abc']

Usefullisthints

As usual, there are othermethods found on lists. Do not hesitate to open thePythoninterpreterandtypeinafewexamples.Rememberthatdirandhelpareyourfriends.

TIP

rangeisabuilt-infunctionthatconstructsintegersequences.Thefollowingwillcreatethenumberszerothroughfour:

>>>nums=range(5)>>>numsrange(5)

Python3tendstobelazy.Notethatrangedoesnotmaterializethelist,butrathergivesyouaniterablethatwillreturnthosenumberswheniteratedover. By passing the result into list you can see the numbers it wouldgenerate:

>>>list(nums)[0,1,2,3,4]

Notice that range does not include 5 in its sequence. Many Pythonfunctionsdealingwithfinalindicesmean“uptobutnotincluding”.(Slicesareanotherexampleofthisyouwillseelater).If you need to start at a non-zero number, range will accept two

parameters.Whentherearetwoparameters,thefirstisthestartingnumber(includingitself),andthesecondisthe“uptobutnotincluding”number:

#numbersfrom2to5>>>nums2=range(2,6)>>>nums2

>>>nums2range(2,6)

range also has an optional third parameter—stride. A stride of one(which is the default)means the next number in the sequence thatrangereturns shouldbeonemore than theprevious.A strideof2would returneveryothernumber.Below isanexample that returnsonlyevennumbersbeloweleven:

>>>even=range(0,11,2)>>>evenrange(0,11,2)>>>list(even)[0,2,4,6,8,10]

NOTE

The“uptobutnotincluding”constructismoreformallyknownasthehalf-openintervalconvention.Itiscommonlyusedwhendefiningsequencesofnaturalnumbers.Thisconstructhasafewniceproperties:

ThedifferencebetweentheendandstartisthelengthwhendealingwithasequenceofconsecutivenumbersTwosubsequencescanbesplicedtogethercleanlywithoutoverlap

Pythonadoptsthishalf-openintervalidiomwidely.Hereisanexample:

>>>a=range(0,5)>>>b=range(5,10)>>>both=list(a)+list(b)>>>len(both)#10-010>>>both[0,1,2,3,4,5,6,7,8,9]

Ifyouaredealingwithnumericsequencesyoumightwanttofollowsuit,especiallywhendefiningAPIs.

Famous computer scientist, EdsgerDijkstra, posited on the reasons forusingzero-based indexing,andwhy it is thecorrectchoice. 10Heendsbysaying:

Many programming languages have been designed without dueattentiontothisdetail.

Luckily,Pythonisnotoneofthoselanguages.

Tuples

Tuples(commonlypronouncedaseither“two”-plesor“tuh”-ples)areimmutablesequences.Youshouldthinkofthemasorderedrecords.Onceyoucreatethem,you cannot change them. To create a tuple using the literal syntax, useparenthesesaroundthemembersandcommasinbetween.Thereisalsoatupleclassthatyoucanusetoconstructanewtuplefromanexistingsequence:

>>>row=('George','Guitar')>>>row('George','Guitar')>>>row2=('Paul','Bass')>>>row2('Paul','Bass')

You can create tuples with zero or one items in them. Though in practice,because tuples hold record type data, this isn't super common. There are twoways to create an empty tuple, using either the tuple function or the literalsyntax:

>>>empty=tuple()>>>empty()>>>empty=()>>>empty()

Herearethreewaystocreateatuplewithoneiteminit:

>>>one=tuple([1])>>>one(1,)>>>one=(1,)>>>one(1,)>>>one=1,>>>one(1,)

NOTE

Parentheses are used for denoting the calling of functions or methods inPython.Theyarealsousedtospecifyoperatorprecedence.Inaddition,theymaybeusedintuplecreation.Thisoverloadingofparenthesescanleadtoconfusion.Here is thesimple rule, if there isone item in theparentheses,then Python treats the parentheses as normal parentheses (for operatorprecedence),suchasthosethatyoumightusewhenwriting(2+3)*8.Iftherearemultipleitemsseparatedbycommas,thenPythontreatsthemasatuple:

>>>d=(3)>>>type(d)<class'int'>

In the above example, d might look like a tuple with parentheses butPythonclaims it isan integer.For tupleswithonlyone item,youneed toputacomma(,)followingtheitem—orusethetupleclasswithasingleitemlist:

>>>e=(3,)>>>type(e)<class'tuple'>

Herearethreewaystocreateatuplewithmorethanoneitem.Typically,youwouldusethelastonetobePythonic.Becauseithasparenthesesitiseasiertoseethatitisatuple:

>>>p=tuple(['Steph','Curry','Guard'])>>>p('Steph','Curry','Guard')>>>p='Steph','Curry','Guard'>>>p('Steph','Curry','Guard')>>>p=('Steph','Curry','Guard')>>>p('Steph','Curry','Guard')

Becausetuplesareimmutableyoucannotappendtothem:

>>>p.append('GoldenState')Traceback(mostrecentcalllast):File"<stdin>",line1,in<module>AttributeError:'tuple'objecthasnoattribute'append'

NOTE

Whythedistinctionbetweentuplesandlists?Whynotuselistssincetheyappeartobeasuper-setoftuples?Themaindifferencebetweentheobjectsismutability.Becausetuplesare

immutable, theyareable toserveaskeys indictionaries.Tuplesareoftenused to represent a record of data such as the row of a database query,whichmaycontainheterogeneous typesofobjects.Perhapsa tuplewouldcontainaname,address,andage:

>>>person=('Matt','123North456East',24)

Tuplesareusedforreturningmultipleitemsfromafunction.Tuplesalsoserveasahinttothedeveloperthatthistypeisnotmeanttobemodified.Tuplesalsouse lessmemory than lists. Ifyouhavesequences thatyou

arenotmutating,considerusingtuplestoconservememory.

Sets

AnothercontainertypefoundinPythonisaset.Asetisanunorderedcollectionthatcannotcontainduplicates.Likea tuple, itcanbeinstantiatedwitha listoranythingyoucaniterateover.Unlikealistoratuple,asetdoesnotcareaboutorder. Sets are particularly useful for two things, removing duplicates andcheckingmembership.Becausethelookupmechanismisbasedontheoptimizedhash function found in dictionaries, a lookup operation takes very little time,evenonlargesets.

NOTE

Becausesetsmustbeabletocomputeahashvalueforeachitemintheset,sets can only contain items that are hashable. A hash is a semi-uniquenumberforagivenobject.Ifanobjectishashable,itwillalwaysgeneratethesamehashnumber.InPython,mutable itemsarenothashable.Thismeans thatyoucannot

hasha list ordictionary.Tohashyourownuser-createdclasses,youwillneedtoimplement__hash__and__eq__.

Sets can be specified by passing in a sequence into the set class (anothercoercionclassthatappearsasafunction):

>>>digits=[0,1,1,2,3,4,5,6,...7,8,9]>>>digit_set=set(digits)#removeextra1

Theycanalsobecreatedwithaliteralsyntaxusing{and}:

>>>digit_set={0,1,1,2,3,4,5,6,...7,8,9}

As was mentioned, a set is great for removing duplicates. When a set iscreatedfromasequence,anyduplicatesareremoved.Theextra1wasremoved

fromdigit_set:

>>>digit_set{0,1,2,3,4,5,6,7,8,9}

Tocheckformembership,youusetheinoperation:

>>>9indigit_setTrue>>>42indigit_setFalse

NOTE

ThereisacontainsprotocolinPython.Ifaclass(setorlist,oruser-definedclass) implements the __contains__ method (or the iteration protocol),you can use in with to check for membership. Due to how sets areimplemented, membership tests against them can be much quicker thantestsagainstalist.

Below,asetcalledoddiscreated.Thissetwillaidinthefollowingexamples:

>>>odd={1,3,5,7,9}

Setsprovidesetoperations,suchasunion(|),intersection(&),difference(-),andxor(^).Thedifference(-)operatorremovesitemsinonesetfromanother:

>>>odd={1,3,5,7,9}#difference>>>even=digit_set-odd>>>even{0,8,2,4,6}

Noticethattheorderoftheresultissomewhatarbitraryatacasualglance.Iftheorderisimportant,asetisnotthedatatypeyoushoulduse.

Theintersection(&)operation(youcanthinkofitastheareawheretworoadsintersect)returnstheitemsfoundinbothsets:

>>>prime=set([2,3,5,7])#thoseinboth>>>prime_even=prime&even>>>prime_even{2}

Theunion(|)operationreturnsasetcomposedofalltheitemsfrombothsets,withduplicatesremoved:

>>>numbers=odd|even>>>print(numbers){0,1,2,3,4,5,6,7,8,9}

Xor(^)isanoperationthatreturnsasetofitemsthatonlyarefoundinonesetortheother,butnotboth:

>>>first_five=set([0,1,2,3,4])>>>two_to_six=set([2,3,4,5,6])>>>in_one=first_five^two_to_six>>>print(in_one){0,1,5,6}

TIP

Why use a set instead of a list? Remember that sets are optimized formembership and removing duplicates. If you find yourself performingunionsordifferencesamonglists,lookintousingasetinstead.Setsarealsomuchquickerfortestingmembership.Theinoperatorruns

faster forsets than lists.However, thisspeedcomesatacost.Setsdonotkeeptheelementsinanyparticularorder,whereaslistsandtuplesdo.Iftheorderisimportantforyou,useadatastructurethatrememberstheorder.

Summary

Thischapterdiscussedafewofthebuilt-incontainertypes.Yousawlists,whichareorderedsequences thataremutable.Remember thata listmethoddoesnotreturn a new list, but typicallywill change the list in place.Listswill supportinsertinganyitem,butyoutypicallyputitemsofthesametypeinalist.Youalsosawtuples.Tuplesarealsoorderedlikelists,unlikelists,however,

tuplesdonotsupportmutation.Inpractice,youusethemtorepresentarecordofdata,likearowretrievedfromadatabase.Whentheyarerepresentingrecords,theymightholddifferenttypesofobjects.Finally,setswereintroduced.Setsaremutable,buttheyareunordered.They

are used to removeduplicates and checkmembership.Because of the hashingmechanism theyuse, these setoperations are fast andefficient.But it requiresthattheitemsinthesetarehashable.

Exercises

1. Createalist.Appendthenamesofyourcolleaguesandfriendstoit.Hastheidofthelistchanged?Sortthelist.Whatisthefirstiteminthelist?Whatistheseconditeminthelist?

2. Create a tuple with your first name, last name, and age. Create a list,people, and append your tuple to it. Make more tuples with thecorresponding information fromyour friends and append them to the list.Sort the list. When you learn about functions, you can use the keyparametertosortbyanyfieldinthetuple,firstname,lastname,orage.

3. Createalistofthenamesofthefirstnamesyourfriends.Createalistwiththetoptencommonnames.Usesetoperationstoseeifanyofyourfriendshavecommonnames.

4. GotoProjectGutenberg 11andfindapageoftextfromShakespeare.Pasteitintoatriple-quotedstring.CreateanotherstringwithaparagraphoftextfromRalphWaldoEmerson.Usethestring's.splitmethodtogetalistofwordsfromeach.Usingsetoperationsfindthecommonwordsandwordsuniquetobothauthors.

5. Tuplesandlistsaresimilarbuthavedifferentbehavior.Usesetoperationstofindtheattributesofalistobjectthatarenotinatupleobject.

10-https://www.cs.utexas.edu/users/EWD/transcriptions/EWD08xx/EWD831.html11-https://www.gutenberg.org/

Iteration

ACOMMON IDIOMWHENDEALINGWITHSEQUENCES ISTOLOOPOVERTHECONTENTSOFthesequence.Youmightwanttofilteroutoneoftheitems,applyafunctiontoit, or print it out. Thefor loop is oneway to do this.Here is an example ofprintingoutthestringsinalist:

>>>forletterin['c','a','t']:...print(letter)cat>>>print(letter)t

NOTE

Notice that a for loop construct contains a colon (:) followed by theindentedcode.(Theindentedcodeistheblockoftheforloop).

Thisillustrateshowaforloopcreatesavariable.Anewvariable,letter,iscreated.Atfirst,itpointsto'c',whichisprinted.Thentheloopcontinuesandthevariablepointsto'a'.

Theforloopcontinueswiththefinalloop.After't'isprinted,theloopexits.Atthispoint,thelistliteralisgarbagecollected.Becausetheonlythingpointingto'c'and'a'wasthelist,thosearealsogarbagecollected.However,thelettervariableisstillpointingto't'.Pythondoesn'tcleanupthis

variable,ithangsaroundaftertheforloopisdone.

Duringaforloop,Pythonmakesanewvariable,letter,thatholdstheitemof iteration.Note that thevalueofletter isnot the indexposition,but ratherthestring.ThisvariableisnotcleanedupbyPythonaftertheforloopisdone.

Loopingwithanindex

InlanguageslikeC,whenyouloopoverasequence,youdonot loopover theitemsinthesequence,ratheryouloopovertheindices.Usingthoseindicesyou

canpullouttheitemsatthoseindexvalues.HereisonewaytodothatinPythonusingthebuilt-infunctionsrangeandlen:

>>>animals=["cat","dog","bird"]>>>forindexinrange(len(animals)):...print(index,animals[index])0cat1dog2bird

Theabovecodeisacodesmell.ItindicatesthatyouarenotusingPythonasyoushould.Usually,thereasonforiteratingoverasequenceistogetaccesstothe items in thesequence,not the indices.Butoccasionallyyouwillalsoneedtheindexpositionoftheitem.Pythonprovidesthebuilt-inenumeratefunctionthat makes the combination of range and len unnecessary. The enumeratefunctionreturnsatupleof(index,item)foreveryiteminthesequence:

>>>animals=["cat","dog","bird"]>>>forindex,valueinenumerate(animals):...print(index,value)0cat1dog2bird

Because the tuple returns a pair of index and value, you can use tupleunpackingtocreatetwovariables,indexandvaluedirectlyintheforloop.Youneed to put a commabetween the variable names.As long as the tuple is thesamelengthasthenumberofvariablesyouincludeintheforloop,Pythonwillhappilycreatethemforus.

Breakingoutofaloop

Youmayneedtostopprocessingaloopearly,withoutgoingovereveryitemintheloop.Thebreakkeywordwilljumpoutofthenearestloopyouarein.Belowisaprogramthataddsnumbersuntilitcomestothefirstnegativeone.Itusesthebreakstatementtostopprocessingwhenitcomesacrossanegativenumber:

>>>numbers=[3,5,9,-1,3,1]>>>result=0>>>foriteminnumbers:...ifitem<0:

...ifitem<0:

...break

...result+=item>>>print(result)17

NOTE

Theline:

result+=item

useswhatiscalledaugmentedassignment.Itissimilartowriting:

result=result+item

Augmentedassignment isslightlyquickeras the lookupfor theresultvariableonlyoccursonce.Anotheraddedbenefitisthatitiseasiertotype.

NOTE

Theifblockinsidetheforblockisindentedeightspaces.Blockscanbenested,andeachlevelneedstobeindentedconsistently.

Skippingoveritemsinaloop

Anothercommonloopingidiomistoskipoveritems.Ifthebodyoftheforlooptakesawhiletoexecute,butyouonlyneedtoexecuteitforcertainitemsinthesequence,thecontinuekeywordcomesinhandy.ThecontinuestatementtellsPythontodisregardprocessingofthecurrentitemintheforloopand“continue”fromthetopoftheforblockwiththenextvalueintheloop.Hereisanexampleofsummingallpositivenumbers:

>>>numbers=[3,5,9,-1,3,1]>>>result=0>>>foriteminnumbers:...ifitem<0:...continue...result=result+item

...result=result+item>>>print(result)21

Theinstatementcanbeusedformembership

Youhaveusedtheinstatementinaforloop.InPython,theinstatementcanalsobeusedtocheckformembership.Ifyouwanttoknowifalistcontainsanitem,youcanusetheinstatementtocheckthat:

>>>animals=["cat","dog","bird"]>>>'bird'inanimalsTrue

Ifyouwanttheindexlocation,usethe.indexmethod:

>>>animals.index('bird')2

Removingitemsfromlistsduringiteration

Itwasmentionedpreviouslythatlistsaremutable.Becausetheyaremutableyoucanaddorremoveitemsfromthem.Also,listsaresequences,soyoucanloopoverthem.Donotmutatethelistatthesametimethatyouareloopingoverit.For example, if you wanted to filter a list of names so it only contained

'John'or'Paul',thiswouldbethewrongwaytodoit:

>>>names=['John','Paul','George',...'Ringo']>>>fornameinnames:...ifnamenotin['John','Paul']:...names.remove(name)>>>print(names)['John','Paul','Ringo']

Whathappened?Pythonassumesthatlistswillnotbemodifiedwhiletheyarebeingiteratedover.Whentheloopgotto'George',itremovedthenamefromthelist.InternallyPythontrackstheindexlocationoftheforloop.Atthatpointthereareonlythreeitemsinthelist,'John','Paul',and'Ringo'.Buttheforloopinternallythinksitisonindexlocation3(thefourthitem),andthereisnofourthitemsotheloopstops,leaving'Ringo'in.

Therearetwoalternativestotheabovecontructofremovingitemsfromalistduring iteration. The first is to collect the items to be removed during a passthroughthelist.In a subsequent loop, iterate over only the items that need to be deleted

(names_to_remove),andremovethemfromtheoriginallist(names):

>>>names=['John','Paul','George',...'Ringo']>>>names_to_remove=[]>>>fornameinnames:...ifnamenotin['John','Paul']:...names_to_remove.append(name)>>>fornameinnames_to_remove:...names.remove(name)>>>print(names)['John','Paul']

Anotheroptionistoiterateoveracopyofthelist.Thiscanbedonerelativelypainlessly using the[:] slice copy construct that is covered in the chapter onslicing:

>>>names=['John','Paul','George',...'Ringo']>>>fornameinnames[:]:#copyofnames...ifnamenotin['John','Paul']:...names.remove(name)>>>print(names)['John','Paul']

elseclauses

A for loop can also have an else clause. Any code in an else block willexecuteiftheforloopdidnothitabreakstatement.Belowissamplecodethatchecksifnumbersinalooparepositive:

>>>positive=False>>>fornuminitems:...ifnum<0:...break...else:...positive=True

...positive=True

Note thatcontinue statements do not have any effect onwhether an elseblockisexecuted.Theelse statement is somewhat oddly named. For afor loop, it indicates

thatthewholesequencewasprocessed.Atypicaluseofanelsestatementinaforloopistoindicatethatanitemwasnotfound.

whileloops

Pythonwill letyou loopoverablockofcodewhileaconditionholds.This iscalledawhileloopandyouusethewhilestatementtocreateait.Awhileloopis followed by an expression that evaluates to True or False. Then, a colonfollows it. Remember what follows a colon (:) in Python? Yes, an indentedblock of code. This block of code will continue to repeat as long as theexpressionevaluatestoTrue.Thisallowsyoutoeasilycreateaninfiniteloop.You usually try to avoid infinite loops because they cause your program to

"hang", forever caught in the processing of a loop with no way out. Oneexception to this isaserver, that loopsforever,continuing toprocess requests.Another exception seen in more advanced Python is an infinite generator. Ageneratorbehaveslikealazylistandonlycreatesvalueswhenyouloopoverit.Ifyouarefamiliarwithstreamprocessing,youcouldthinkofitasastream.(Idon'tcovergeneratorsinthisbook,butdoinmymoreadvancedbook.)Typically,ifyouhaveanobjectthatsupportsiteration,youuseaforloopto

iterateovertheitems.Youusewhileloopswhenyoudon'thaveeasyaccesstoaniterableobject.Acommonexampleiscountingdown:

>>>n=3>>>whilen>0:...print(n)...n=n-1321

Youcanalsousethebreakstatementtoexitawhileloop:

>>>n=3>>>whileTrue:...print(n)...n=n-1...ifn==0:...break

Summary

Thischapterdiscussedusing thefor loop to iterateoverasequence.Yousawthatyoucanloopoverlists.Youcanalsoloopoverstrings,tuples,dictionaries,andotherdatastructures.Infact,youcandefineyourownclassesthatrespondtotheforstatement,byimplementingthe.__iter__method.Afor loop creates a variable during iteration. This variable is not garbage

collected after the for loop, it sticks around. If your for loop is inside of afunction,thevariablewillbegarbagecollectedwhenthefunctionexits.Theenumeratefunctionwasintroduced.Thisfunctionreturnsasequenceof

tuplesofindex,itempairsforthesequencepassedintoit.Ifyouneedtogettheindexlocationaswellastheitemofiteration,useenumerate.Finally, you saw how to break out of loops, continue to the next item of

iteration, and use else statements. These constructs enable you to adeptlyconfigureyourloopinglogic.

Exercises

1. Createalistwiththenamesoffriendsandcolleagues.Calculatetheaveragelengthofthenames.

2. Createalistwiththenamesoffriendsandcolleagues.SearchforthenameJohn using a for loop. Print not found if you didn't find it. (Hint: useelse).

3. Createalistoftuplesoffirstname,lastname,andageforyourfriendsandcolleagues. If youdon't know the age,put inNone.Calculate the average

age,skippingoveranyNonevalues.Printouteachname,followedbyoldoryoungiftheyareaboveorbelowtheaverageage.

Dictionaries

DICTIONARIESAREAHIGHLYOPTIMIZEDBUILT-INTYPEINPYTHON.You can compare a Python dictionary to anEnglish dictionary.AnEnglish

dictionaryhaswordsanddefinitions.Thepurposeofadictionaryistoallowfastlookupofthewordinordertofindthedefinition.Youcanquicklylookupanyword by doing a binary search (open up the dictionary to the midpoint, anddeterminewhichhalfthewordisin,andrepeat).APython dictionary also haswords and definitions, but you call them keys

andvaluesrespectively.Thepurposeofadictionaryistoprovidefastlookupofthekeys.Youcanquicklylookforakeyandpulloutthevalueassociatedwithit.LikeanEnglishdictionary,whereitwouldtakealongtimetodeterminethewordfromadefinition(ifyoudidn'tknowthewordbeforehand),lookingupthevalueisslow.InPython3.6,thereisafeaturethatisnewfordictionaries.Thekeysarenow

sortedbyinsertionorder. IfyouarewritingPythoncodethatneeds toworkinpriorversions,youwillneedtorememberthatpriorto3.6,keyshadanarbitraryorder(thatallowedPythontodoquicklookupsbutwasn'tparticularlyusefultoendusers).

Dictionaryassignment

Dictionaries provide a link from a key to a value. (Other languages call themhashes,hashmaps,maps,orassociativearrays).Supposeyouwanted to storeinformationaboutanindividual.Yousawhowyoucoulduseatupletorepresentarecord.Adictionaryisanothermechanism.Becausedictionariesarebuiltinto

Python, you can use a literal syntax to create one. This one has first and lastnames:

>>>info={'first':'Pete','last':'Best'}

NOTE

Analternatewaytocreateadictionaryiswiththebuilt-indictclass.Ifyoupassalistoftuplepairsintotheclass,itwillreturnadictionary:

>>>info=dict([('first','Pete'),...('last','Best')])

Youcanalsousenamedparameterswhenyoucalldict:

>>>info=dict(first='Pete',last='Best')

Ifyouusenamedparameters,theymustbevalidPythonvariablenames,andtheywillbeconvertedtostrings.

Youcanuseindexoperationstoinsertvaluesintothedictionary:

>>>info['age']=20>>>info['occupation']='Drummer'

Thisillustratesthecreationofadictionary.Inthiscase,youuseanexistingvariableforthevalue.Notethatthedictionarydoesnotcopythevariable,butitpointstoit(increasingthereferencecount).

In the above example, the keys are 'first', 'last', 'age', and'occupation'.Forexample,'age' is thekey thatmaps to the integer20, thevalue.Youcanquicklylookupthevaluefor'age'byperformingalookupwithanindexoperation:

>>>info['age']20

On the flip side, finding what the key is that has the value 20 is a slowoperation. It would be like giving someone a definition from an EnglishDictionaryandaskingwhatwordhasthatdefinition.The above example illustrates the literal syntax for creating an initially

populateddictionary. It also showshow the squarebrackets (indexoperations)areusedtoinsertitemsintoadictionaryandpullthemout.Anindexoperation

associates a key with a value when used in combination with the assignmentoperator(=).Whentheindexoperationhasnoassignment,itlooksupthevalueforagivenkey.

Retrievingvaluesfromadictionary

As you have seen, the square bracket literal syntax can pull a value out of adictionarywhenyouusethebracketswithoutassignment:

>>>info['age']20

Be careful though, if you try to access a key that does not exist in thedictionary,Pythonwillthrowanexception:

>>>info['Band']Traceback(mostrecentcalllast):File"<stdin>",line1,in<module>KeyError:'Band'

BecausePython likes tobe explicit and fail fast, raising an exception is thedesired behavior. You know that the 'Band' key is not in the dictionary. Itwouldbebad ifPythondecided toreturnanarbitraryvalueforakey thatwasnot in thedictionary.Doingsocouldallowerrors topropagate further inyourprogramandmaskalogicerror.

Theinoperator

Pythonprovidesanoperator,in,thatallowsyoutoquicklycheckifakeyisinadictionary:

>>>'Band'ininfoFalse>>>'first'ininfoTrue

As you have seen, in also works with sequences. You can use the instatementwithalist,set,orstringtocheckformembership:

>>>5in[1,3,6]False

>>>'t'in{'a','e','i'}False>>>'P'in'Paul'True

NOTE

Python 3 removed the .has_key method, which provided the samefunctionalityastheinstatement,butisspecifictoadictionary.Hoorayforstepsforwardtowardsconsistency!

Theinstatementwillworkwithmostcontainers.Inaddition,youcandefineyourownclassesthatrespondtothisstatement.Yourobjectneedstodefinethe.__contains__orbeiterable.

Dictionaryshortcuts

The .get method of a dictionary will retrieve a value for a key. .get alsoacceptsanoptionalparametertoprovideadefaultvalueifthekeyisnotfound.Ifyouwantedthegenretodefaultto'Rock',youcoulddothefollowing:

>>>genre=info.get('Genre','Rock')>>>genre'Rock'

TIP

The.getmethod of dictionaries is oneway to get around theKeyErrorthrownwhentryingtousethebracketnotationtopulloutakeynotfoundinthedictionary.It is fine touse thismethodbecauseyou arebeing explicit aboutwhat

willhappenwhen thekey ismissing.Youshouldprefer to fail fastwhenyouaren'tbeingspecificaboutthefailingcase.

setdefault

A useful, but somewhat confusingly named, method of dictionaries is the.setdefaultmethod.Themethodhasthesamesignatureas.getandinitiallybehaveslikeit,returningadefaultvalueifthekeydoesnotexist.Inadditiontothat,italsosetsthevalueofthekeytothedefaultvalueifthekeyisnotfound.Because.setdefaultreturnsavalue,ifyouinitializeittoamutabletype,suchasadictorlist,youcanmutatetheresultinplace.

.setdefaultcanbeusedtoprovideanaccumulatororcounterforakey.Forexample, if youwanted to count the number of people with same name, youcoulddothefollowing:

>>>names=['Ringo','Paul','John',...'Ringo']>>>count={}>>>fornameinnames:...count.setdefault(name,0)...count[name]+=1

Withoutthe.setdefaultmethod,abitmorecodeisrequired:

>>>names=['Ringo','Paul','John',...'Ringo']>>>count={}>>>fornameinnames:...ifnamenotincount:...count[name]=1...else:...count[name]+=1>>>count['Ringo']2

TIP

Performingthesecountingtypesofoperationswassocommonthatlaterthecollections.Counter class was added to the Python standard library.Thisclasscanperformtheaboveoperationsmuchmoresuccinctly:

>>>importcollections>>>count=collections.Counter(['Ringo','Paul',...'John','Ringo'])>>>countCounter({'Ringo':2,'Paul':1,'John':1})

Counter({'Ringo':2,'Paul':1,'John':1})>>>count['Ringo']2>>>count['Fred']0

Here is a somewhat contrived example illustrating mutating the result of.setdefault.Assumeyouwanttohaveadictionarymappinganametobandsthat theyplayed in. If apersonnamedPaul is in twobands, the result shouldmapPaultoalistcontainingbothofthosebands:

>>>band1_names=['John','George',...'Paul','Ringo']>>>band2_names=['Paul']>>>names_to_bands={}>>>fornameinband1_names:...names_to_bands.setdefault(name,...[]).append('Beatles')>>>fornameinband2_names:...names_to_bands.setdefault(name,...[]).append('Wings')>>>print(names_to_bands['Paul'])['Beatles','Wings']

Tobelaborthepoint,withoutsetdefaultthiscodewouldbeabitlonger:

>>>band1_names=['John','George',...'Paul','Ringo']>>>band2_names=['Paul']>>>names_to_bands={}>>>fornameinband1_names:...ifnamenotinnames_to_bands:...names_to_bands[name]=[]...names_to_bands[name].\...append('Beatles')>>>fornameinband2_names:...ifnamenotinnames_to_bands:...names_to_bands[name]=[]...names_to_bands[name].\...append('Wings')>>>print(names_to_bands['Paul'])['Beatles','Wings']

TIP

The collections module from the Python standard library includes ahandyclass—defaultdict.Thisclassbehaveslikeadictionarybutitalsoallows for setting thedefaultvalueofakey toanarbitrary factory. If thedefaultfactoryisnotNone,itisinitializedandinsertedasavalueanytimeakeyismissing.Thepreviousexamplere-writtenwithdefaultdictisthefollowing:

>>>fromcollectionsimportdefaultdict>>>names_to_bands=defaultdict(list)>>>fornameinband1_names:...names_to_bands[name].\...append('Beatles')>>>fornameinband2_names:...names_to_bands[name].\...append('Wings')>>>print(names_to_bands['Paul'])['Beatles','Wings']

Usingdefaultdictisslightlymorereadablethanusingsetdefault.

Deletingkeys

Another common operation on dictionaries is the removal of keys and theircorresponding values. To remove an item from a dictionary, use the delstatement:

#remove'Ringo'fromdictionary>>>delnames_to_bands['Ringo']

TIP

Python will prevent you from adding to or removing from a dictionarywhileyouareloopingoverit.PythonwillthrowaRuntimeError:

>>>data={'name':'Matt'}>>>forkeyindata:...deldata[key]Traceback(mostrecentcalllast):...RuntimeError:dictionarychangedsizeduringiteration

Dictionaryiteration

Dictionariesalsosupportiterationusingtheforstatement.Bydefault,whenyouiterateoveradictionary,yougetbackthekeys:

>>>data={'Adam':2,'Zeek':5,'Fred':3}>>>fornameindata:...print(name)AdamZeekFred

NOTE

Thedictionaryhasamethod—.keys—thatwillalso listout thekeysofadictionary. In Python 3, the.keysmethod returns a view. The view is awindowintothecurrentkeysfoundinthedictionary.Youcaniterateoverit, like a list. But, unlike a list, it is not a copy of the keys. If you laterremoveakeyfromthedictionary,theviewwillreflectthatchange.Alistwouldnot.

Toiterateoverthevaluesofadictionary,iterateoverthe.valuesmethod:

>>>forvalueindata.values():...print(value)253

Theresultof.valuesisalsoaview.Itreflectsthecurrentstateofthevaluesfoundinadictionary.Toretrievebothkeyandvalueduringiteration,usethe.itemsmethod,which

returnsaview:

>>>forkey,valueindata.items():...print(key,value)Adam2Zeek5

Zeek5Fred3

Ifyoumaterializetheviewintoalist,youwillseethatthelistisasequenceof(key,value)tuples—thesamethingthatdictacceptstocreateadictionary:

>>>list(data.items())[('Adam',2),('Zeek',5),('Fred',3)]

TIP

Remember that a dictionary is ordered based on key insertion order. If adifferentorderisdesired,youwillneedtosortthesequenceofiteration.The built-in function sorted will return a new sorted list, given a

sequence:

>>>fornameinsorted(data.keys()):...print(name)AdamFredZeek

Thesortedfunctionhasanoptionalargument,reverse,tofliptheorderoftheoutput:

>>>fornameinsorted(data.keys(),...reverse=True):...print(name)ZeekFredAdam

NOTE

Itispossibletohavekeysofdifferenttypes.Theonlyrequirementforakeyis that it behashable. For example, a list isn't hashable because you canmutate it, andPython can't generate a consistent hashvalue for it. If youusedalistasakeyandthenmutatedthatkey,shouldthedictionaryreturn

thevaluebasedontheoldlist,orthenewone,orboth?Pythonrefusestoguesshereandmakesyouusekeysthatdon'tchange.Itwouldbepossibletoinsertitemsintoadictionaryusingbothintegers

andstringsaskeys:

>>>weird={1:'first',...'1':'anotherfirst'}

Typically,youdon'tmixkeytypes,becauseitisconfusingtoreadersofthe code and alsomakes sorting keys harder. Python 3won't sortmixedtypelistswithoutakey functiontellingPythonexplicitlyhowtocomparedifferent types. This is one of those areas where Python gives you theabilitytodosomething,butthatdoesn'tmeanyoushould.InthewordsofPythoncoredeveloperRaymondHettinger:

Many"CanIdoxinPython"questionsequateto"CanIstopacaron train tracks when no trains are coming?" Yes you can, No youshouldn't—@raymondh

Summary

Thischapterdiscussedthedictionary.ThisdatastructureisimportantbecauseitisabuildingblockinPython.Classes,namespaces,andmodulesinPythonareallimplementedusingadictionaryunderthecovers.Dictionariesprovidequick lookupor insertion for akey.Youcanalsodoa

lookupbyvalue,butitisslow.Ifyoufindyourselfdoingthisoperationoften,itisacodesmellthatyoushoulduseadifferentdatastructure.Yousawhowtomutateadictionary.Youcaninsertandremovekeysfroma

dictionary. You can also check a dictionary for membership using the instatement.You looked at some fancier constructs, using .setdefault to insert and

return values in one operation. You saw that Python includes specializeddictionaryclasses,Counteranddefaultdictinthecollectionsmodule.Because dictionaries aremutable, you can delete keys from them using the

del statement.You can also iterate over the keys of a dictionary using aforloop.RememberthatPython3.6introducedorderingtothedictionary.Thekeysare

orderedbyinsertionorder,notalphabeticornumericorder.

Exercises

1. Createadictionary,info,thatholdsyourfirstname,lastname,andage.2. Createanemptydictionary,phone,thatwillholddetailsaboutyourphone.

Addthescreensize,memory,OS,brand,etc.tothedictionary.3. Write a paragraph in a triple-quoted string. Use the .split method to

createalistofwords.Createadictionarytoholdthecountforeverywordintheparagraph.

4. CounthowmanytimeseachwordisusedinaparagraphoftextfromRalphWaldoEmerson.

5. Writecodethatwillprintouttheanagrams(wordsthatusethesameletters)fromaparagraphoftext.

6. ThePageRankalgorithmwasusedtocreatetheGooglesearchengine.Thealgorithmgivesascoretoeachpagebasedonincominglinks.Ittakesoneinput:alistofpagesthatlinktootherpages.Eachpageinitiallygetsascoresetto1.Thenmultipleiterationsofthealgorithmarerun,typicallyten.Foreachiteration:

Apagetransfersitsscoredividedbythenumberofoutgoinglinkstoeachpagethatitlinksto.Thescoretransferredismultipliedbyadampingfactor,typicallysetto.85.

Writesomecodetorun10iterationsofthisalgorithmonalistoftuplesofsourceanddestinationlinks,ie:

links=[('a','b'),('a','c'),('b','c')]

Functions

WE HAVE COME A LONGWAYWITHOUT DISCUSSING FUNCTIONS,WHICH ARE A BASICbuildingblockofPythonprograms.Functionsarediscreteunitsofcode,isolatedinto theirownblock.Youhaveactuallybeenusingbuiltin functionsalong theway such as dir, help, (and the classes that behave like coercion functions—float,int,dict,list,andbool).

Afunctionislikeabox.Itcantakeinputandreturnoutput.Itcanbepassedaroundandreused.

Oneway to thinkofa function isasablackbox thatyoucansend input to(though input is not required). The black box then performs a series ofoperations and returns output (it implicitly returns None if the function endswithoutreturnbeingcalled).Anadvantageofafunctionisthatitenablescodereuse.Onceafunctionisdefined,youcancallitmultipletimes.Ifyouhavecodethatyourepeatedlyruninyourprogram,ratherthancopyingandpastingit,youcanput it ina functiononce, thencall that functionmultiple times.Thisgivesyoulesscodetoreasonabout,makingyourprogramseasiertounderstand.Itisalsoeasier to changecode (or fixbugs) later asyouonlyhave todo it inoneplace.

Hereisasimpleexampleofafunction.Thisfunction,namedadd_2,takesanumberasinput,adds2tothatvalue,andreturnstheresult:

>>>defadd_2(num):...'''...return2morethannum...'''...result=num+2...returnresult

Whatarethedifferentcomponentsofafunction?Thiswholegroupofcodeisknownasa functiondefinition.First,youseethedefstatement,whichisshortfordefine(i.e.defineafunction).Followingdefisarequiredspace(onespaceisfine)andthenthefunctionname—add_2.Thisisthenamethatisusedtoinvokethe function. Synonyms for invoke include call, execute, or run the function.WhenPythoncreatesafunction,itwillcreateanewvariable,usingthenameofthefunction.

Thisillustratesthecreationofafunction.NotethatPythoncreatesanewfunctionobject,thenpointsavariabletoitusingthenameofthefunction.Usethedirfunctiononthefunctionnametosee

theattributesofthenewlycreatedfunction.

Followingthefunctionnameisaleftparenthesisfollowedbynumandarightparenthesis. The names between the parentheses (there can be an arbitrary

number of names, in this case, there is only one), are the input parameters.Thesearetheobjectsthatyoupassintoafunction.Aftertheparenthesescomesacolon(:).WheneveryouseeacoloninPython,

you should immediately think the colon is followed by an indented block—similar to the body of the for loops shown previously. Everything that isindentedisthebodyofthefunction.Itisalsocalledablockofcode.Thebodyofafunctioniswherethelogiclives.Youmightrecognizethefirst

3linesofindentedcodeasatriple-quotedstring.Thisisnotacomment,thoughitappearstohavecomment-liketext.Itisastring.Pythonallowsyoutoplaceastringimmediatelyafterthe:, thisstringiscalledadocstring.Adocstringisastring used solely for documentation. It should describe the block of codefollowingit.Thedocstringdoesnotaffectthelogicinthefunction.

TIP

Thehelp functionhasbeenemphasizedthroughthisbook.It is importantto note that the help function gets its content from the docstring of theobject passed into it. If you call help on add_2, you should see thefollowing(providedyouactuallytypedouttheadd_2codeabove):

>>>help(add_2)Helponfunctionadd_2inmodule__main__:add_2()return2morethannum(END)

Providing docstrings can be useful to you as a reminder of what yourcode does. If they are accurate, the docstrings are invaluable to anyonetryingtouseyourcode.

Followingthedocstring(notethatadocstringisoptional),comesthelogicofthe function. The result is calculated. Finally, the function uses a return

statementtoindicatethatithasoutput.Areturnstatementisnotnecessary,andifitismissingafunctionwillreturnNonebydefault.Furthermore,youcanusemorethanonereturnstatement,andtheydonotevenhavetobeattheendofthefunction.Forexample,aconditionalblockmaycontainareturnstatementinsideofanifblockandanotherinanelseblock.Torecap,themainpartsofafunctionare:

thedefkeywordafunctionnamefunctionparametersbetweenparenthesesacolon(:)indentation

docstringlogicreturnstatement

Creatingfunctionsiseasy.Theyallowyoutoreusecode,whichmakesyourcodeshorterandeasiertounderstand.Theyareusefultoremoveglobalstatebykeeping short-lived variables inside of the function body. Use functions toimproveyourcode.

Invokingfunctions

Whenyoucallorexecuteafunction,youareinvokingafunction.InPython,youinvokefunctionsbyaddingparenthesesfollowingthefunctionname.Thiscodeinvokesthenewlydefinedfunctionadd_2:

>>>add_2(3)5

To invoke a function, list its name followed by a left parenthesis, the inputparameters,andarightparenthesis.Thenumberofparametersshouldmatchthenumber of parameters in the function declaration. Notice that the REPL

implicitly prints the result—the integer 5. The result is what the returnstatementpassesback.Youcanpassinwhateverobjectyouwanttotheadd_2function.However,if

that object doesn't support additionof a number, youwill get an exception. Ifyoupassastringin,youwillseeaTypeError:

>>>add_2('hello')Traceback(mostrecentcalllast):...TypeError:mustbestr,notint

Scope

Python looks for variables in various places.You call these places scopes (ornamespaces). When looking for a variable (remember that functions are alsovariablesinPython,asareclasses,modules,andmore),Pythonwilllookinthefollowinglocations,inthisorder:

Localscope-Variablesthataredefinedinsideoffunctions.Globalscope-Variablesthataredefinedatthegloballevel.Builtinscope-VariablesthatarepredefinedinPython.

Inthefollowingcodeisafunctionthatwill lookupvariables ineachof thethreescopes:

>>>x=2#Global>>>defscope_demo():...y=4#Localtoscope_demo...print("Local:{}".format(y))...print("Global:{}".format(x))...print("Builtin:{}".format(dir))>>>scope_demo()Local:4Global:2Builtin:<builtinfunctiondir>

Note that after scope_demo is invoked, the local variable y is garbagecollectedandnolongeravailableintheglobalscope:

>>>y

>>>yTraceback(mostrecentcalllast):...NameError:name'y'isnotdefined

Variablesdefinedinsideofafunctionormethodwillbelocal.Ingeneral,youshouldtrytoavoidglobalvariables.Theymakeithardertoreasonaboutcode.Globalvariablesarecommoninbooks,blogs,anddocumentationbecauseusingthem requireswriting less code, and allows you to focus on conceptswithoutthembeingwrappedinafunction.Usingfunctionsandclassesaresomeofthetoolsthathelpyouremoveglobalvariables,makeyourcodemoremodular,andeasiertounderstand.

NOTE

Pythonwillallowyoutooverridevariablesintheglobalandbuiltinscope.Atthegloballevel,youcoulddefineyourownvariablecalleddir.Atthatpoint, the builtin function, dir, is shadowed by the global variable. Youcouldalsodothiswithinafunction,andcreatealocalvariablethatshadowsaglobalorbuiltin:

>>>defdir(x):...print("Dircalled")>>>dir('')Dircalled

Youcanuse thedelstatement todeletevariables in the localorglobalscope. In practice though, you don't see this, as it is better to avoidshadowingbuiltinsinthefirstplace:

>>>deldir>>>dir('')['__add__','__class__','__contains__',...]

HINT

Youcanusethelocalsandglobalsfunctionstolistthesescopes.Thesereturndictionarieswiththeircurrentscopeinthem:

>>>deffoo():...x=1...print(locals())>>>foo(){'x':1}

The __builtins__ variable lists the builtin names. If you access its__dict__ attribute, it will give you a dictionary similar to globals andlocals.

Multipleparameters

Functions can have many parameters. Below is a function that takes twoparametersandreturnstheirsum:

>>>defadd_two_nums(a,b):...returna+b

NotethatPythonisadynamiclanguageandyoudon'tspecifythetypesoftheparameters.Thisfunctioncanaddtwointegers:

>>>add_two_nums(4,6)10

Itcanalsoaddfloats:

>>>add_two_nums(4.0,6.0)10.0

Stringstoo:

>>>add_two_nums('4','6')'46'

Notethatthestringsuse+toperformstringconcatenation(joiningtwostringstogether).However,ifyoutrytoaddastringandanumber,Pythonwillcomplain:

>>>add_two_nums('4',6)Traceback(mostrecentcalllast):...TypeError:Can'tconvert'int'objecttostrimplicitly

This is an instancewherePythonwantsyou tobemore explicit aboutwhatoperationisdesired,andit isn'tgoingtoguessforyou.Ifyouwantedtoaddastringtypetoanumber,youmightwanttoconvertthemtonumbersfirst(usingfloatorint).Likewise,itmightbeusefultoconvertfromnumberstostringsifconcatenationisthedesiredoperation.Pythondoesnotimplicitlychoosewhichoperation to perform. Rather, it throws an error which should force theprogrammertoresolvetheambiguity.

Defaultparameters

One cool feature of Python functions is default parameters. Like the nameimplies,defaultparametersallowyoutospecifythedefaultvaluesforfunctionparameters.Thedefault parameters are thenoptional, thoughyoucanoverridethemifyouneedto.Thefollowingfunctionissimilartoadd_two_nums,butwilldefaulttoadding

3ifthesecondnumberisnotprovided:

>>>defadd_n(num,n=3):..."""defaultto...adding3"""...returnnum+n>>>add_n(2)5>>>add_n(15,-5)10

Tocreate adefault value for aparameter, follow theparameternameby anequalsign(=)andthechosenvalue.

NOTE

Default parameters must be declared after non-default parameters.Otherwise,PythonwillgiveyouaSyntaxError:

>>>defadd_n(num=3,n):...returnnum+nTraceback(mostrecentcalllast):...SyntaxError:non-defaultargumentfollowsdefaultargument

Because default parameters are optional, Python forces you to declarerequired parameters before optional ones. The above codewouldn'tworkwith an invocation like add_n(4) because the required parameter ismissing.

TIP

Donotusemutabletypes(lists,dictionaries)fordefaultparametersunlessyou know what you are doing. Because of the way Python works, thedefaultparametersarecreatedonlyonce—atfunctiondefinitiontime,notatfunctionexecutiontime.Ifyouuseamutabledefaultvalue,youwillendupreusing the same instance of the default parameter during each functioninvocation:

>>>defto_list(value,default=[]):...default.append(value)...returndefault>>>to_list(4)[4]>>>to_list('hello')[4,'hello']

Thefactthatdefaultparametersareinstantiatedatthetimethefunctioniscreatedisconsideredawartbymany.Thisisbecausethebehaviorcanbesurprising. The workaround is to push the creation of the default valuesfromfunctiondefinitiontime(whichonlyoccursonce)tofunctionruntime(whichwillcreateanewvalueeverytimethefunctionisexecuted).Change the mutable default parameters so that they default to None.

Then,createaninstanceofthedesiredmutabletypewithinthebodyofthe

functionifthedefaultisNone:

>>>defto_list2(value,default=None):...ifdefaultisNone:...default=[]...default.append(value)...returndefault>>>to_list2(4)[4]>>>to_list2('hello')['hello']

Thefollowingcode:

...ifdefaultisNone:

...default=[]

Canalsobewrittenasasinglelineusingaconditionalexpression:

...default=defaultifdefaultisnotNoneelse[]

Namingconventionsforfunctions

Function naming conventions are similar to variable naming conventions (andare also found in the PEP 8 document). This is called snake case, which isclaimedtobeeasiertoread.Functionnamesshould:

belowercasehave_an_underscore_between_wordsnotstartwithnumbersnotoverridebuiltinsnotbeakeyword

LanguagessuchasJavahaveadopted thecamelcasenamingconvention. Inthat style youmight have a variable namedsectionList or amethod namedhasTimeOverlap. In Python, you would typically name these section_listandhas_time_overlaprespectively.WhilePythoncodeshouldfollowthePEP

8conventions, there isalsoanallowanceinPEP8forconsistency.If thecodeyouareworkingonhasadifferentstyleofnamingconvention,followsuitandbe consistent with the existing code. Indeed, the unittest module in thestandardlibrarystillhasJavastyleconventions(becauseitwasoriginallyaportfromtheJavalibraryjunit).

Summary

Functionsallowyoutoencapsulatechangeandsideeffectswithintheirbody.Inthischapter,youlearnedthatfunctionscantake inputandreturnoutput.Therecan bemultiple input parameters, and you can also provide default values forthem.Remember thatPython isbasedonobjects,andwhenyoucreatea function,

youalsocreateavariablewiththefunctionnamethatpointstoit.Functionscanalsohaveadocstringwhichisastringwrittenimmediatelyafter

the declaration. These strings are used to present documentation when thefunctionispassedintothehelpfunction.

Exercises

1. Write a function, is_odd, that takes an integer and returns True if thenumberisoddandFalseifthenumberisnotodd.

2. Write a function,is_prime, that takes an integer and returnsTrue if thenumberisprimeandFalseifthenumberisnotprime.

3. Write a binary search function. It should take a sorted sequence and theitem it is looking for. It should return the index of the item if found. Itshouldreturn-1iftheitemisnotfound.

4. Write a function that takes camel cased strings (i.e. ThisIsCamelCased),and converts them to snake case (i.e. this_is_camel_cased). Modify thefunction by adding an argument, separator, so it will also convert tokebabcase(i.e.this-is-camel-case)aswell.

IndexingandSlicing

PYTHONPROVIDESTWOCONSTRUCTSTOPULLDATAOUTOFSEQUENCE-LIKETYPES(LISTS,tuples,andevenstrings).Thesearetheindexingandslicingconstructs.Indexingallowsyoutoaccesssingleitemsoutofasequence,whileslicingallowsyoutopulloutasubsequencefromasequence.

Indexing

You have seen indexingwith lists. For example, if you have a list containingpets,youcanpulloutanimalsbyindex:

>>>my_pets=["dog","cat","bird"]>>>my_pets[0]'dog'

TIP

RememberthatinPythonindicesstartat0.Ifyouwanttopulloutthefirstitemyoureferenceitby0,not1.Thisiscalledzero-basedindexing.

Thisillustratespositiveandnegativeindexpositions.

Python has a cool feature where you can reference items using negativeindices. -1 refers to the last item, -2 the second to last item, etc. This iscommonlyusedtopulloffthelastiteminalist:

>>>my_pets[-1]'bird'

Guido van Rossum, the creator of Python, tweeted to explain how tounderstandnegativeindexvalues:

[The]properwaytothinkof[negativeindexing]istoreinterpreta[-X]asa[len(a)-X]

—@gvanrossum

Youcanalsoperformanindexoperationonatupleorastring:

>>>('Fred',23,'Senior')[1]23>>>'Fred'[0]'F'

Sometypes,suchassets,don'tsupportindexoperations.Ifyouwanttodefineyour own class that supports index operations, you should implement the.__getitem__method.

Slicingsublists

Inadditiontoacceptinganintegertopulloutasingleitem,aslicemaybeusedtopulloutasubsequence.Aslicemaycontain thestart index,anoptionalendindex,andanoptionalstride,allseparatedbyacolon.Hereisaslicethatpullsoutthefirsttwoitemsofalist:

>>>my_pets=["dog","cat","bird"]#alist>>>print(my_pets[0:2])['dog','cat']

Remember thatPythonuses thehalf-open interval convention.The listgoesup tobutdoesnot include theend index.Asmentionedpreviously, therangefunctionalsobehavessimilarlywithitssecondparameter.

Thisillustratesslicingoffthefirstfourcharactersofastring.Threeoptionsareshown.Thelastoptionisthepreferredway.Youdon'tneedthezeroindexsinceitisthedefault.Usingnegative

indicesisjustsilly.Slicingoffthefinalthreecharactersisshownaswell.Again,thefinalexampleistheidiomaticwaytodoit.Thefirsttwoassumethestringhasalengthofeight,whilethefinalcode

willworkwithanystringthathasatleastthreecharacters.

Whenyouslicewithacolon(:),thefirstindexisoptional.Ifthefirstindexismissing, the slice defaults to starting from the first itemof the list (the zerothitem):

>>>print(my_pets[:2])

>>>print(my_pets[:2])['dog','cat']

Youcanalsousenegativeindiceswhenslicing.Anegativeindexcanbeusedinthestartlocationorendinglocation.Theindex-1representsthelastitem.Ifyousliceuptothelastitem,youwillgeteverythingbutthatitem:

>>>my_pets[0:-1]['dog','cat']>>>my_pets[:-1]#defaultsto0['dog','cat']>>>my_pets[0:-2]['dog']

Thefinalindexisalsooptional.Ifthefinalindexismissing,theslicedefaultstotheendofthelist:

>>>my_pets[1:]['cat','bird']>>>my_pets[-2:]['cat','bird']

Finally, you can default both the start and end indices. If both indices aremissing,theslicereturnedwillrunfromthestarttotheend(whichwillcontainacopyofthelist).ThisisaconstructyoucanusetoquicklycopylistsinPython:

>>>print(my_pets[:])['dog','cat','bird']

Stridingslices

Slices also take a stride following the starting andending indices.Thedefaultvalueforanunspecifiedstrideis1.Astrideof1meanstakeeveryitemfromasequence between the indices.A stride of2would take every second item.Astrideof3wouldtakeeverythirditem:

>>>my_pets=["dog","cat","bird"]>>>dog_and_bird=my_pets[0:3:2]>>>print(dog_and_bird)['dog','bird']>>>zero_three_six=[0,1,2,3,4,5,6][::3]>>>print(zero_three_six)[0,3,6]

[0,3,6]

NOTE

Again,therangefunctionhasasimilarthirdparameterthatspecifiesstride:

>>>list(range(0,7,3))[0,3,6]

Stridescanbenegative.Astrideof-1meansthatyoumovebackwardrighttoleft.Touseanegativestride,youshouldmakethestartslicegreaterthantheendslice.Theexceptionis ifyouleaveout thestartandendindices,astrideof-1willreversethesequence:

>>>my_pets[0:2:-1][]>>>my_pets[2:0:-1]['bird','cat']>>>print([1,2,3,4][::-1])[4,3,2,1]

Thenexttimeyouareinajobinterviewandtheyaskyoutoreverseastring,youcandoitinasingleline:

>>>'emerih'[::-1]'hireme'

Ofcourse, theywillprobablywantyoutodoit inC.Just tell themthatyouwanttoprograminPython!

Summary

Youuseindexingoperationstopullsinglevaluesoutofsequences.Thismakesiteasytogetacharacterfromastring,oranitemfromalistortuple.Ifyouwantsubsequences,youcanuse theslicingconstruct.Slicingfollows

thehalf-openintervalandgivesyouthesequenceuptobutnotincludingthelastindex.Ifyouprovideanoptionalstride,youcanskipelementsinaslice.Pythonhasanice feature thatallowsyou tousenegativevalues to indexor

slicerelativetotheendofthesequence.Thisletsyoudoyouroperationsrelativetothelengthofthesequence,andyoudon'thavetoworryaboutcalculatingthelengthofsequenceandsubtractingfromthat.

Exercises

1. Create a variable with your name stored as a string. Use indexingoperations to pull out the first character. Pull out the last character.Willyourcodetopulloutthelastcharacterworkonanameofarbitrarylength?

2. Createavariable,filename.Assumingthat ithasathreeletterextension,and using slice operations, find the extension. For README.txt, theextension should betxt.Write code using slice operations thatwill givethe name without the extension. Does your code work on filenames ofarbitrarylength?

3. Createa function,is_palindrome, todetermine if a suppliedword is thesameifthelettersarereversed.

FileInputandOutput

READING AND WRITING FILES IS COMMON IN PROGRAMMING. PYTHON MAKES THESEoperationseasy.Youcanreadbothtextandbinaryfiles.Youalsohaveachoiceof readinga fileby thenumberofbytes, lines,oreven thewhole fileatonce.Therearesimilaroptionsavailableforwritingfiles.

Openingfiles

The Python function, open, returns a file object. This function has multipleoptionalparameters:

open(filename,mode='r',buffering=-1,encoding=None,errors=None,newline=None,closefd=True,opener=None)

HINT

Windows paths use \ as a separator, which can be problematic. Pythonstringsalsouse\asanescapecharacter.Ifyouhadadirectorynamed"test"andwrote"C:\test",Pythonwouldtreat\tasatabcharacter.TheremedyforthisistouserawstringstorepresentWindowspaths.Put

anrinfrontofthestring:

r"C:\test"

Youtypicallyareonlyconcernedwiththefirst twoorthreeparameters.Thefirst parameter, the filename, is required. The second parameter is the mode,whichdeterminesifyouarereadingorwritingafileandifthefileisatextfileorbinaryfile.Therearevariousmodes.

Filemodes

FilemodesMODE MEANING

'r' Readtextfile(default)

'w' Writetextfile(overwritesifexists)

'x' Writetextfile,throwFileExistsErrorifexists.

'a' Appendtotextfile(writetoend)

'rb' Readbinaryfile

'wb' Writebinary(overwrite)

'w+b' Openbinaryfileforreadingandwriting

'xb' Writebinaryfile,throwFileExistsErrorifexists.

'ab' Appendtobinaryfile(writetoend)

Fordetailson therestof theoptions to theopen function,pass it intohelp.Thisfunctionhasverydetaileddocumentation.Ifyouaredealingwith text files, typicallyyouwilluse'r' as themode to

readafile,and'w'towriteafile.Hereyouopenthefiletmpa.txtforwriting.Pythonwillcreatethatfile,oroverwriteitifitalreadyexists:

>>>a_file=open('tmpa.txt','w')

Thefileobject,returnedfromtheopencall,hasvariousmethodstoallowforreadingandwriting.Thischapterdiscusses themostcommonlyusedmethods.To read the fulldocumentationona fileobjectpass it to thehelp function. Itwilllistallofthemethodsanddescribewhattheydo.

NOTE

/tmpiswhereUnixsystemsplacetemporaryfiles.IfyouareonWindows,youcaninspecttheTempvariablebytyping:

c:\>ECHO%Temp%

Thevalueisusually:

C:\Users\<username>\AppData\Local\Temp

Readingtextfiles

Pythonprovidesmultiplewaystoreaddatafromfiles.Ifyouopenafileintextmode(thedefault),youreadstringsfromthefileoryouwritestringstoit.Ifyouopenafile inbinarymode(usinga'rb' forreadingor'wb' forwriting),youwillreadandwritebytestrings.Ifyouwanttoreadasinglelinefromanexistingtextfile,usethe.readline

method. Ifyoudon't specifyamode,Pythonwilldefault to readinga text file(modeofr):

>>>passwd_file=open('etcpasswd')>>>passwd_file.readline()'root:x:0:0:root:/root:/bin/bash'

Becareful,ifyouopenafileforreadingthatdoesnotexist,Pythonwillthrowanerror:

>>>fin=open('bad_file')Traceback(mostrecentcalllast):...IOError:[Errno2]Nosuchfileordirectory:'bad_file'

TIP

Theopenfunctionreturnsafileobjectinstance.Thisobjecthasmethodstoreadandwritedata.Commonvariablenames for fileobjects arefin (file input),fout (file

output),fp (filepointer,usedforeither inputoroutput)ornamessuchaspasswd_file.Names likefin andfout are useful because they indicatewhetherthefileisusedforreadingorwritingrespectively.

Asillustratedabove,the.readlinemethodwillreturnonelineofafile.Youcan call it repeatedly to retrieve every line or use the .readlinesmethod toreturnalistcontainingallthelines.Toreadallthecontentsofafileintoonestringusethe.readmethod:

>>>passwd_file=open('etcpasswd')>>>print(passwd_file.read())root:x:0:0:root:/root:/bin/bashbin:x:1:1:bin:/bin:/bin/falsedaemon:x:2:2:daemon:/sbin:/bin/falseadm:x:3:4:adm:/var/adm:/bin/false

Youshouldalwayscloseyourfileswhenyouaredonewiththembycalling.close:

>>>passwd_file.close()

Closingyourfilesiseasy.Abitlateryouwilldigintowhyyoushouldmakesureyouclosethem.

Readingbinaryfiles

Toreadabinaryfile,passin'rb'(readbinary)asthemode.Whenyoureadabinaryfileyouwillnotgetstrings,butbytestrings.Don'tworry,astheinterfaceforbytestringsisverysimilartostrings.HerearethefirsteightbytesofaPNGfile.Toreadeightbytes,youwillpass8tothe.readmethod:

>>>bfin=open('img/dict.png','rb')>>>bfin.read(8)b'\x89PNG\r\n\x1a\n'

Noticethebinfrontofthestring,specifyingthatthisisabytestring.Youcanalso use .readline on binary files and it will read until it gets to a b'\n'.Binarystringsversusnormalstringswillbediscussedinalaterchapter.

Iterationwithfiles

Iterationoversequenceswasdiscussedpreviously.InPython,itiseasytoiterateover the lines ina file.Whendealingwitha text file,youcan iterateover the.readlinesmethodtogetalineatatime:

>>>fin=open('etcpasswd')>>>forlineinfin.readlines():...print(line)

However, because .readlines returns a list, Python will need to read thewhole file tocreate that list, and thatcouldbeproblematic.Say the filewasalarge logfile, itcouldpossiblyconsumeallofyourmemorytoread it.Pythonhasatrickupitssleevethough.Pythonallowsyoutoloopoverthefileinstanceitselftoiterateoverthelinesofthefile.Whenyouiteratedirectlyoverthefile,Pythonislazy,andonlyreadsthelinesoftextasneeded:

>>>fin=open('etcpasswd')>>>forlineinfin:...print(line)

How can you iterate directly over the file instance? Python has a dundermethod, .__iter__, that defines what the behavior is for looping over aninstance.Itjustsohappensthatforthefileclass,the.__iter__methoditeratesoverthelinesinthefile.The.readlinesmethodshouldbe reservedforwhenyouaresure thatyou

can hold the file inmemory and you need to access the linesmultiple times.Otherwise,directlyiteratingoverthefileispreferred.

Writingfiles

Towrite toa file,youmust firstopen the file inwritemode. Ifmode is set to'w',thefileisopenedforwritingtextdata:

>>>fout=open('tmpnames.txt','w')>>>fout.write('George')

The above method will try to overwrite the file tmpnames.txt if it exists,otherwise,thefilewillbecreated.Ifyoudon'thavepermissiontoaccessthefile,a Permission Error will be thrown. If the path is bad, you will see aFileNotFoundError.Twomethodsusedtoplacedatainafileare.writeand.writelines.The

.write method takes a string as a parameter and writes it to the file. The

.writelinesmethodtakesasequencecontainingstringdataandwritesittothefile.

NOTE

Toincludenewlinesinyourfile,youneedtoexplicitlypassthemtothefilemethods.OnUnix platforms, strings passed into.write should endwith\n. Likewise, each of the strings in the sequence that is passed into to.writelinesshouldendin\n.OnWindows,thenewlinestringis\r\n.Toprograminacrossplatformmanner,thelinesepstringfoundinthe

osmoduledefinesthecorrectnewlinestringfortheplatform:

>>>importos>>>os.linesep#Unixplatform'\n'

TIP

Ifyouaretryingthisoutontheinterpreterrightnow,youmaynoticethatthe tmpnames.txt file is empty even though you told Python to writeGeorgeinit.Whatisgoingon?File output is buffered by the operating system. In order to optimize

writestothestoragemedia,theoperatingsystemwillonlywritedataafteracertain thresholdhasbeenpassed.OnLinuxsystems, this isnormally4Kbytes.Toforcewritingthedata,youcancallthe.flushmethod,whichflushes

thependingdatatothestoragemedia.Amoreheavy-handedmechanism,toensurethatdataiswritten,istocall

the.closemethod.This informsPython thatyouaredonewriting to thefile:

>>>fout2=open('tmpnames2.txt',...'w')>>>fout2.write('John\n')>>>fout2.close()

Closingfiles

Closingfiles

Asmentioned previously, calling.closewillwrite the file buffers out to thestoragemedia.ThebestpracticeinPythonistoalwaysclosefilesafteryouaredonewiththem(whetherforwritingorreading).Hereareafewreasonswhyyoushouldexplicitlycloseyourfiles:

If youhave a file in a global variable, itwill never be closedwhile yourprogramisexecuting.cPython will automatically close your files for you if they are garbagecollected.OtherPythonimplementationsmaynot.Unlessyoucall.flushyouwon'tknowwhenyourdataiswritten.Youroperatingsystemprobablyhasalimitofopenfilesperprocess.Someoperatingsystemswon'tletyoudeleteanopenfile.

Pythonusuallytakescareofgarbagecollectionforyouandyoudon'thavetoworry about cleaning up objects. Opening and closing files are exceptions.Pythonwillautomaticallyclosethefileforyouwhenthefileobjectgoesoutofscope. But this is a case where you shouldn't rely on garbage collection. Beexplicitandcleanupafteryourself.Makesureyoucloseyourfiles!Python 2.5 introduced thewith statement.Thewith statement is usedwith

context managers to enforce conditions that occur before and after a block isexecuted.Theopen functionalso servesasacontextmanager toensure that afileisopenedbeforetheblockisenteredandthatitisclosedwhentheblockisexited.Belowisanexample:

>>>withopen('tmpnames3.txt','w')asfout3:...fout3.write('Ringo\n')

Thisistheequivalentof:

>>>fout3=open('tmpnames3.txt','w')>>>fout3.write('Ringo\n')>>>fout3.close()

Noticethatthewithlineendswithacolon.WhenalineendswithacoloninPython,youalwaysindentthecodethatfollows.Theindentedcontentfollowinga colon is called a block or the body of the context manager. In the aboveexample, theblockconsistsofwritingRingo toafile.Thentheblockfinishes.Youcan't reallysee it in theaboveexample,butyouknowawithblockendswhen the code is dedented. At this point, the context manager kicks in andexecutestheexitlogic.TheexitlogicforthefilecontextmanagertellsPythontoautomaticallyclosethefileforyouwhentheblockisfinished.

TIP

Usethewithconstructforreadingandwritingfiles.Itisagoodpracticetoclosefiles,andifyouusethewithstatementwhenusingfiles,youdonothavetoworryaboutclosingthem.Thewithstatementautomaticallyclosesthefileforyou.

Designingaroundfiles

You have seen how to use functions to organize and compartmentalizecomplicated programs. One benefit to using functions is that you can re-usethosefunctionsthroughoutyourcode.Hereisatipfororganizingfunctionsthatdealwithfiles.Assume that you want to write code that takes a filename and creates a

sequenceof the lines from the filewith the line number inserted before everyline. At first thought it might seem that you want the API (applicationprogramminginterface)foryourfunctionstoacceptthefilenameofthefilethatyouwanttomodify,likethis:

>>>defadd_numbers(filename):...results=[]...withopen(filename)asfin:...fornum,lineinenumerate(fin):...results.append(...'{0}-{1}'.format(num,line))...returnresults

...returnresults

This codewill work okay. Butwhatwill happenwhen you are required toinsertlinenumbersinfrontoflinesfromasourceotherthanafile?Ifyouwantto test the code, now you have access to the file system. One solution is torefactortheadd_numbersfunctionsothatitwillonlyopenthefileinacontextmanager,andthencallanotherfunction,add_nums_to_seq.Thisnewfunctioncontainslogicthatoperatesonasequenceratherthandependingonafilename.Since a file behaves as a sequence of strings, you preserve the originalfunctionality:

>>>defadd_numbers(filename):...withopen(filename)asfin:...returnadd_nums_to_seq(fin)>>>defadd_nums_to_seq(seq):...results=[]...fornum,lineinenumerate(seq):...results.append(...'{0}-{1}'.format(num,line))...returnresults

Nowyouhave amore general function,add_nums_to_seq, that is easier totest and reuse, because instead of depending on a filename, it depends on asequence.Youcanpassinalistofstrings,orcreateafakefiletopassintoit.

HINT

Thereareother typesthatalsoimplement thefile-likeinterface(readandwrite).Anytimeyoufindyourselfcodingwithafilename,askyourselfifyoumaywanttoapplythelogictoothersequence-likethings.Ifso,usethepreviousexampleof refactoring the functions toobtaincode that ismucheasiertoreuseandtest.

Summary

Python provides a single function to interact with both text and binary files,open.Byspecifyingthemode,youcantellPythonwhetheryouwanttoreadorwritetothefile.Whenyouaredealingwithtextfiles,youreadandwritestrings.Whenyouaredealingwithbinaryfiles,youreadandwritebytestrings.Makesureyoucloseyourfiles.Usingthewithstatementistheidiomaticway

todothis.Finally,makesureyourfunctionsdealwithsequencesofdatainsteadoffilenamesasthismakesyourcodemoregenerallyuseful.

Exercises

1. Write a function towrite a comma separated value (CSV) file. It shouldacceptafilenameandalistoftuplesasparameters.Thetuplesshouldhaveaname,address,andage.Thefileshouldcreateaheaderrowfollowedbyarowforeachtuple.Ifthefollowinglistoftupleswaspassedin:

[('George','4312AbbeyRoad',22),('John','54LoveAve',21)]

itshouldwritethefollowinginthefile:

name,address,ageGeorge,4312AbbeyRoad,22John,54LoveAve,21

2. WriteafunctionthatreadsaCSVfile.Itshouldreturnalistofdictionaries,using the first row as key names, and each subsequent row as values forthosekeys.Forthedatainthepreviousexampleitwouldreturn:

[{'name':'George','address':'4312AbbeyRoad','age':22},{'name':'John','address':'54LoveAve','age':21}]

Unicode

WE'VESEENSTRINGSALLOVERTHEPLACE,BUTWEHAVEN'TREALLYTALKEDABOUTONEof the biggest changes that came in Python 3,Unicode strings! Python 2 hadsupportforUnicodestrings,butyouneededtoexplicitlycreatethem.Thisisnolongerthecase,everythingisUnicode.

Background

What is Unicode? It is a standard for representing glyphs (the characters thatcreatemostwritten language,aswellassymbolsandemoji).Thestandardcanbe found on the Unicode website 12, and is frequently updated. The standardconsists of various documents or charts that map code points (hexadecimalnumberssuchas0048or1F600)toglyphs(suchasHorᛡ),andnames(LATINCAPITALH andGRINNING FACE). The code points and names are unique,thoughmanyglyphsmaylookverysimilar.

Thisillustrateshowtoreadcodechartsfoundatunicode.org.Thechartshavetableslistingglyphsbytheirhexcodepoints.Followingthattableisanothertablewiththecodepoint,glyph,andname.

Youcanuseaglyph,thename,orthecodepoint.Ifthecodepointhasmorethan4digits,youneedtousecapitalUandleftpadwith0untilyouhave8digits.Ifithas4orlessdigits,youneedtousea

lowercaseuandleftpadwith0untilyouhave4digits.AlsoshownisanexampledecodingtheUTF-8bytestringtothecorrespondingglyph.

Here is a condensed history. As computers became prevalent, differentprovidershaddifferentschemestomapbinarydatatostringdata.Oneencoding,ASCII,woulduse7bitsofdatatomapto128symbolsandcontrolcodes.ThatworksfineinaLatincharacter-centricenvironmentsuchasEnglish.Having128different glyphs would provide enough space for lowercase characters,uppercasecharacters,digits,andcommonpunctuation.

Assupportfornon-Englishlanguagesbecamemorecommon,ASCIIwasnotsufficient.WindowssystemsthroughWindows98supportedanencodingcalledWindows-1252, that had support for various accented characters, and symbolssuchastheEurosign.Allof theseencodingschemesprovidedaone-to-onemappingofbytes toa

character.InordertosupportChinese,Korean,andJapanesescripts,manymorethan 128 symbolswere needed.Using four bytes provided support for over 4billioncharacters.Butthisencodingcameatacost.ForthemajorityofpeopleusingonlyASCII-centriccharacters,requiringthosecharacterstobeencodedinfourtimesasmuchdataseemedlikeacolossalwasteofmemory.Acompromisethatprovidedboththeabilitytosupportallcharactersbutnot

wastememory,wastostopencodingcharacterstoasequenceofbits.Rather,thecharacters were abstracted. Each character instead mapped to a unique codepoint(thathasahexvalueandauniquename).Variousencodingsthenmappedthesecodepointstobitencodings.Unicodemapsfromcharactertoacodepoint—it is not the encoding. For different contexts, an alternate encoding mightprovidebettercharacteristics.One encoding, UTF-32, uses four bytes to store the character information.

This representation is easy for low-level programmers to use as indexingoperationsaretrivial.ButitusesfourtimestheamountofmemoryasASCIItoencodeaLatinsequenceofletters.Thenotionofvariablewidthencodingsalsohelpedalleviatememorywaste.

UTF-8isonesuchencoding.Itusesbetweenoneandfourbytestorepresentacharacter.Inaddition,UTF-8isbackwardcompatiblewithASCII.UTF-8isthemost common encoding on theweb, and is a great choice for encoding sincemanyapplicationsandoperatingsystemsknowhowtodealwithit.

TIP

You can find out what the preferred encoding is on your machine byrunningthefollowingcode.Hereismineona2015Mac:

>>>importlocale>>>locale.getpreferredencoding(False)'UTF-8'

Let’sbeclear.UTF-8isanencodingofbytesforUnicodecodepoints.TosaythatUTF-8isUnicodeissloppyatbestandconnotesalackofunderstandingofcharacter encodings at worst. In fact, the name is derived from UnicodeTransformationFormat-8-bit,ie.itisaformatforUnicode.Let’s look at an example. The character named, SUPERSCRIPT TWO, is

definedintheUnicodestandardascodepointU+00b2.Ithasaglyph(awrittenrepresentation) of ². ASCII has no way to represent this character.Windows-1252does,itisencodedasthehexbyteb2(whichhappenstobethesameasthecodepoint, note that this is not guaranteed). InUTF-8 it is encoded asc2b2.Likewise,UTF-16encodesitasfffeb200.TherearemultipleencodingsforthisUnicodecodepoint.

BasicstepsinPython

You can createUnicode strings in Python. In fact, you have been doing it allalong. Remember in Python 3, strings are Unicode strings. But you probablywanttocreateastringthathasanon-ASCIIcharacter.Belowiscodetomakeastringthathasxsquared(x²)init.Ifyoucanfindtheglyphyouwanttouse,youcancopythatcharacterinintoyourcode:

>>>result='x²'

Thisworksingeneral.Thoughyoumightrunintoproblemsifyourfontdoesnotsupportsaidglyph.Thenyouwillgetwhatissometimescalledtofubecauseit lookseither likeanemptyboxoradiamondwithaquestionmark.Another

way to include non-ASCII characters is to use the hex Unicode code pointfollowing\u:

>>>result2='x\u00b2'

Notethatthisstringisthesameasthepreviousstring:

>>>result==result2True

Finally,youcanusethenameofthecodepointinsideofthecurlybracesin\N{}:

>>>result3='x\N{SUPERSCRIPTTWO}'

Allofthesework.TheyallreturnthesameUnicodestring:

>>>print(result,result2,result3)x²x²x²

Thethirdoptionisalittleverboseandrequiresalittlemoretyping.Butifyoudon'tspeaknativeUnicodeorhavefontsupport,itisperhapsthemostreadable.

TIP

The Python help documentation has an entry forUnicode. Type help(),then UNICODE. It actually doesn't discuss Unicode that much, rather thedifferentwaystocreatePythonstrings.

Encoding

PerhapsoneofthekeystogrokkingUnicodeinPythonistounderstandthatyouencodeaUnicodestringtoabytestring.Youneverencodeabytestring.YoudecodeabytestringtoaUnicodestring.Likewise,youneverdecodeaUnicodestring. Another way of looking at encoding and decoding is that encodingtransforms a human-readable or meaningful representation into an abstractrepresentation meant for storage (Unicode to bytes or letter to bytes), and

decoding transforms that abstract representation back to a human readable ormeaningfulrepresentation.

Imageillustratestheencoding(inthiscaseusingUTF-8)ofaUnicodestringintoitsbyterepresentation,andthenthedecodingofthesamebytestringbackintoUnicode(alsousingUTF-8).Notethatyoushouldbeexplicitwhendecodingasthereareotherencodingsthatifused,might

produceerroneousdataormojibake(charactertransformation).

Given a Unicode string, you can call the .encode method to see therepresentationinvariousencodings.Let’sstartoffwithUTF-8:

>>>x_sq='x\u00b2'>>>x_sq.encode('utf-8')b'x\xc2\xb2'

IfPythondoesnotsupportanencoding,itwillthrowaUnicodeEncodeError.ThismeansthattheUnicodecodepointsarenotsupportedinthatencoding.Forexample,ASCII,doesnothavesupportforthesquaredcharacter:

>>>x_sq.encode('ascii')Traceback(mostrecentcalllast):...UnicodeEncodeError:'ascii'codeccan'tencodecharacter'\xb2'inposition1:ordinalnotinrange(128)

If you use Python long enough, you will probably encounter this error. Itmeansthatthespecifiedencodingdoesnothavetheabilitytorepresentallofthe

characters. If you are certain that youwant to force Python to this encoding,therearea fewdifferentoptionsyoucanprovide to theerrorsparameter.ToignorecharactersPythoncan'tencode,useerrors='ignore':

>>>x_sq.encode('ascii',errors='ignore')b'x'

Ifyouspecifyerrors='replace',Pythonwillinsertaquestionmarkfortheunsupportedbytes:

>>>x_sq.encode('ascii',errors='replace')b'x?'

NOTE

TheencodingsmodulehasamappingofaliasesofencodingsthatPythonsupports.Thereare99encodingsinPython3.6.MostmodernapplicationstrytostayinUTF-8,butifyouneedsupportforotherencodings,thereisagoodchancethatPythonhassupportforit.Thismapping is found inencodings.aliases.aliases.Alternatively,

thedocumentationonthemoduleatthePythonwebsite13hasatableoftheencodings.

Herearesomeotherpossibleencodingsforthisstring:

>>>x_sq.encode('cp1026')#Turkishb'\xa7\xea'>>>x_sq.encode('cp949')#Koreanb'x\xa9\xf7'>>>x_sq.encode('shift_jis_2004')#Japaneseb'x\x85K'

ThoughPythonsupportsmanyencodings,theirusageisbecomingmorerare.Typically they are used for legacy applications. Today, most applications useUTF-8.

Decoding

DecodinghasaspecificmeaninginPython.ItmeanstotakeasequenceofbytesandcreateaUnicodestringfromthem.InPython,youneverencodebytes,youdecodethem(infactthereisno.encodemethodonbytes).Ifyoucanrememberthatrule,itcansaveabitoffrustrationwhendealingwithnon-ASCIIcharacters.Let’sassumethatyouhavetheUTF-8bytesequenceforx²:

>>>utf8_bytes=b'x\xc2\xb2'

Whenyouaredealingwithcharacterdata representedasbytes,youwant toget it into aUnicode string as soon as possible. Typically, youwill only dealwithstringsinyourapplicationandusebytesasaserializationmechanism(i.e.whenpersistingtofilesorsendingoverthenetwork).Ifyoureceivedthesebytessomehowand the frameworkor library that youwere usingdid not convert itintoastringforyou,youcoulddothefollowing:

>>>text=utf8_bytes.decode('utf-8')>>>text'x²'

Anotherthingtonoteisthatingeneral,youcannotdivinewhatasequenceofbyteswasencodedas.YouknowthatthesequenceinthecodelistedisUTF-8because you just created it, but if unidentified bytes were sent to you fromanothersourcethenitcouldcertainlybeanotherencoding.

NOTE

Youneedtobetoldtheencoding.Otherwise,youmighttrytodecodewiththewrongencodingandhavebaddata.When thishappens, themiscodedstring is calledmojibake. This is a Japanese word that means charactermutation,butmojibakesoundscooler.Herearesomeexamplesoferroneousdecoding:

>>>b'x\xc2\xb2'.decode('cp1026')#Turkish'ÌB¥'

>>>b'x\xc2\xb2'.decode('cp949')#Korean'x'

>>>b'x\xc2\xb2'.decode('shift_jis_2004')#Japanese'xﾂｲ'

Butsomeencodingsdon'tsupportallbytesequences.Ifyoudecodewiththewrong encoding, sometimes insteadof gettingbaddata, youget an exception.Forexample,ASCII,doesnotsupportthisbytesequence:

>>>b'x\xc2\xb2'.decode('ascii')Traceback(mostrecentcalllast):...UnicodeDecodeError:'ascii'codeccan'tdecodebyte0xc2inposition1:ordinalnotinrange(128)

AUnicodeDecodeErrormeansthatyouweretryingtoconvertabytestringto a Unicode string and the encoding did not have support to create Unicodestrings forallof thebytes.Typically, thismeans thatyouareusing thewrongencoding.Forbestresults,trytodetermineandusethecorrectencoding.If you really don't know the encoding, you can pass the errors='ignore'

parameter,butthenyouarelosingdata.Thisshouldonlybeusedasalastresort:

>>>b'x\xc2\xb2'.decode('ascii',errors='ignore')'x'

Unicodeandfiles

When you read a text file, Pythonwill give youUnicode strings. By default,Python will use the default encoding(locale.getpreferredencoding(False)).Ifyouwanttoencodeatextfileinanotherencoding,youcanpassthatinformationwiththeencodingparametertotheopenfunction.Likewise, you can specify the encoding when you open a file for writing.

Remember,youshouldtreatencodingsasaserializationformat(usedtotransferinformationovertheinternetorstorethedatainafile).Herearetwoexamplesofwriting to a file.With the first file no encoding is defined, and it therefore

defaults toUTF-8 (per the system's default encoding). In the second example,CP949(Korean)issetastheencodingparameter:

>>>withopen('tmpsq.utf8','w')asfout:...fout.write('x²')

>>>withopen('tmpsq.cp949','w',encoding='cp949')asfout:...fout.write('x²')

From theUNIX terminal, lookat the files.You see that theyhavedifferentcontents,becausetheyareusingdifferentencodings:

$hexdumptmpsq.utf8000000078c2b20000003

$hexdumptmpsq.cp949000000078a9f70000003

Youcanreadthedataback:

>>>data=open('tmpsq.utf8').read()>>>data'x²'

Remember,thatinPython,explicitisbetterthanimplicit.Whendealingwithencodings, you need to be specific. Python on a UTF-8 system will try anddecodefromUTF-8whenyoureadafile.ThisisfineifthefileisencodedwithUTF-8,butifitisnot,youwilleithergetmojibake,oranerror:

>>>data=open('tmpsq.cp949').read()Traceback(mostrecentcalllast):...UnicodeDecodeError:'utf-8'codeccan'tdecodebyte0xa9inposition1:invalidstartbyte

IfyouareexplicitandtellPythonthatthisfilewasencodedwiththeCP949encoding,youwillgetthecorrectdatafromthisfile:

>>>data=open('tmpsq.cp949',encoding='cp949').read()>>>data'x²'

Ifyouaredealingwithtextfilesthatcontainnon-ASCIIcharacters,makesureyouspecifytheirencoding.

Summary

Unicode is a mapping of code points to glyphs. In Python, strings can holdUnicode glyphs. You can encode the Unicode string to a byte string usingvariousencodings.YouneverdecodePythonstrings.Whenyoureadatextfile,youcanspecifytheencodingtomakesurethatyou

getaUnicodestringcontainingthecorrectcharacters.Whenyouwritetextfiles,youcanusetheencodingparametertodeclaretheencodingtouse.UTF-8isthemostpopularencodingthesedays.Unlessyouhaveareasonto

useanotherencoding,youshoulddefaulttousingUTF-8.

Exercises

1. Go to http://unicode.org and download a chart that has code points on it.Chooseanon-ASCIIcharacterandwritePythoncodetoprintthecharacterbyboththecodepointandname.

2. There are various Unicode characters that appear to be upside downversions of ASCII characters. Find a mapping of these characters (theyshouldbeasearchaway).Writeafunction that takesastringwithASCIIcharacters,andreturnstheupsidedownversionofthatstring.

3. Writetheupsidedownversionofyournametoafile.Whataresomeofthepossibleencodingsthatsupportyourname?Whataresomeencodingsthatdon't?

4. Smart quotes (or curly quotes) are not supported with ASCII. Write afunctionthattakesanASCIIstringasinputandreturnsastringwherethedouble quotes are replaced with smart quotes. For example, the stringPythoncomeswith"batteriesincluded",shouldbecomePythoncomeswith“batteries included” (if you look closely, you will see that with smartquotes,theleftquotescurvedifferentlythantherightquotes).

5. Writea function that takes textwitholdschoolemojis in it (:),:P,etc.).Using the emoji chart 14, add code to your function that replaces the textemojis,withUnicodeversionsfoundinthechart.

12-https://unicode.org13-https://docs.python.org/3/library/codecs.html14-http://unicode.org/emoji/charts/full-emoji-list.html

Classes

STRINGS,DICTIONARIES, FILES,AND INTEGERSAREALLOBJECTS.EVEN FUNCTIONSAREobjects. InPython, almost everything is anobject.There are someexceptions,keywords(suchasin)arenotobjects.Also,variablenamesarenotobjects,buttheydopointtothem.Thischapterwilldelvedeeperintowhatanobjectreallyis.Object is a somewhat ambiguous term. One definition of “Object-Oriented

Programming” is using structures to group together data (state) and methods(functionstoalterstate).Manyobject-orientedlanguagessuchasC++,Java,andPythonuseclasses todefinewhat stateanobject canholdand themethods toalter that state. Whereas classes are the definition of the state and methods,instancesareoccurrencesofsaidclasses.Generallywhenpeoplesayobjecttheymeananinstanceofaclass.InPython,str is thenameof the classused to store strings.Thestr class

definesthemethodsofstrings.Youcancreateaninstanceofthestrclass,b,byusingPython’sliteralstring

syntax:

>>>b="I'mastring">>>b"I'mastring"

Therearemanytermsthatyouwillseethrownabouttotalkaboutb.Youmayhear,“bisastring”,“bisanobject”,“bisaninstanceofastring”.Thelatterisperhapsthemostspecific.But,bisnotastringclass.

Thisillustratesastringobject.Allobjectshavetypes,whicharereallyjusttheclassoftheobject.Inthiscasetheclassisstr.

Thisillustratesanupdatedversionofastringobject.Typehasbeenchangedto__class__,becausewhenyouinspecttheobject,the__class__attributepointstotheclassoftheobject.Notethatthisclasshasvariousmethods,thisimageonlyshowscapitalize,buttherearemanymore.Methodsare

alsoobjects,asshownintheimage.

NOTE

Thestr classcanalsobeused tocreate strings,but isnormallyused forcasting. Strings are built into the language, so it is overkill to pass in astringliteralintothestrclass.Youwouldn'tsay:

>>>c=str("I'mastring")

Because Python automatically creates a string when you put quotesaroundcharacters.ThetermliteralmeansthatthisisaspecialsyntaxbuiltintoPythontocreatestrings.Ontheflipside,ifyouhaveanumberthatyouwanttoconverttoastring,

youcouldcallstr:

>>>num=42>>>answer=str(num)>>>answer'42'

It is said that Python comes with “batteries included”—it has libraries andclassespredefinedforyouruse.Theseclassestendtobegeneric.Youcandefineyour own classes that deal specifically with your problem domain. You cancreatecustomizedobjectsthatcontainstate,andaswellaslogictochangethatstate.

Planningforaclass

First of all, classes are not always needed in Python. You should give somethoughttowhetheryouneedtodefineaclassorwhetherafunction(orgroupoffunctions) would be sufficient. Classes might be useful to provide aprogrammatic representation of something that is a physical object, or aconceptual object. Something that is a description such as speed, temperature,average,orcolorarenotgoodcandidatesforclasses.Onceyouhavedecided thatyouwant tomodel somethingwitha class, ask

yourselfthefollowingquestions:

Doesithaveaname?Whatarethepropertiesthatishas?Arethesepropertiesconstantbetweeninstancesoftheclass?ie:

Whichofthesepropertiesarecommontotheclass?

Whichofthesepropertiesareuniquetoeachmember?

Whatactionsdoesitdo?

Here’saconcreteexample.Assumethatyouworkforaskiresortandwanttomodelhowpeopleusethechairlifts.Onewaytodosowouldbetocreateaclassthatdefinesachaironachairlift.Achairlift,ifyouarenotfamiliarwithit,isacontraption that has many chairs. These chairs are like a bench that multipleskiersmaysiton.Skiersqueueupatthebottomofahilltoboardthechairs,thenunboardoncethechairhasliftedthemtothetopofthehill.Obviously,youwillneedtoabstractyourmodeltosomedegree.Youarenot

goingtomodeleverylow-levelpropertyofachair.Forexample,youdon'tcarewhether a chair ismade of steel or aluminum or wood for calculating usage.Thatmightmatterforotherpeople.Doesthisthingyouwanttomodelhaveaname?Yes,chair.Afewproperties

ofthechairincludechairnumber,capacity,whetherithasasafetybar,andifithaspaddedseats.Todive ina littledeeper, capacitycanbebrokendown intomaximum capacity and current occupants. Maximum capacity should stayconstant,whereasoccupantscanchangeatapointintime.Achairisinvolvedwithafewactions.Someactionsincludeaddingpeopleat

thebottomofthehillandremovingpeoplewhentheyreachthetopofthehill.Another actionmight be the position of the safety bar. You are going handleloadingandunloadingbutignorebarpositionforyourmodel.

Definingaclass

HereisaPythonclasstorepresentachaironachairlift.Belowisasimpleclassdefinition.Thecommentsarenumberedfordiscussionfollowingthecode:

>>>classChair:#1...'''AChaironachairlift'''#2...max_occupants=4#3......def__init__(self,id):#4...self.id=id#5...self.count=0

...self.count=0

...

...defload(self,number):#6

...self.count+=number

...

...defunload(self,number):#7

...self.count-=number

TheclassstatementinPythondefinesaclass.Youneedtogiveitaname(1),followed by a colon.Remember that in Python, you indent following a colon(unless it is a slice). Note, that you indented consistently below the classdefinition.Alsonotethatthenameoftheclasswascapitalized.

NOTE

Classnamesareactuallycamelcased.BecauseChairisasingleword,youmightnothavenoticed.Unlikefunctionswherewordsare joined togetherwith underscores, in camel casing, you capitalize the first letter of eachword and then shove them together.Normally class names are nouns. InPython, they cannot start with numbers. The following are examples ofclassnames,bothgoodandbad:

Kitten#good

jaguar#bad-startswithlowercase

SnowLeopard#good-camelcase

White_Tiger#bad-hasunderscores

9Lives#bad-startswithanumber

SeePEP8forfurtherinsightintoclassnaming.Note that many of the builtin types do not follow this rule: str, int,

float,etc.

Immediatelyfollowingthedeclarationoftheclass,youcaninsertadocstring(2). This is just a string. Note that if it is a triple-quoted string, it may span

multiple lines. Docstrings are optional, but they are useful to readers of yourcode,andalsoappearwhenyouusethehelpfunctiontoinspectyourcodeintheREPL.Usethemjudiciouslyanditwillpaygreatdividends.Inside the indentedbodyof the class, you can createclass attributes (3).A

classattributeisusedtoholdstatethatissharedamongallinstancesofaclass.In this example, any chair that you create will have a maximum of fouroccupants.(Skierscall thesetypesofchairliftsquads).Thereareadvantagestocreatingclassattributes.Becausetheclassissettingthisnumber,youdon'thavetorepeatyourselfandsetthevalueeachtimeyoucreateachair.Ontheflipside,youhavehardcodedyourchair toonlysupportfourseatsonit.Later,youwillseehowtooverridetheclassattribute.Next you see adef statement (4). It looks like you are defining a function

insideoftheclassbody.Andyouare,exceptwhenyoudefineafunctiondirectlyinsideofaclassbody,youcall itamethod.Because thismethodhasaspecialname,__init__,youcallitaconstructor.Ithastwoparameters,selfandid.Mostmethodshaveselfasthefirstparameter.Youcaninterpretselfasbeingtheinstanceoftheclass.

Thisillustrateswhathappenswhenyoudefineaclass.Pythonwillcreateanewtypeforyou.Anyclassattributesormethodswillbestoredasattributesofthenewclass.Instanceattributes(idand

count)arenotfoundintheclassastheywillbedefinedontheinstances.

Aconstructoriscalledwhenyoucreateaninstanceofaclass.Ifyouconsideraclasstobeafactorythatprovidesatemplateorblueprintforinstances,thentheconstructoriswhatinitializesthestatefortheinstances.Theconstructortakesaninstanceasinput(theselfparameter),andupdatesitinsidethemethod.Pythontakescareofpassingaroundtheinstanceforus.Thiscanbeconfusingandwillbediscussedmorelater.

Inside the body of the constructor (5) (it is indented because it follows acolon), you attach two attributes that will be unique to the instance, id andcount. On most chairlifts, each chair on the chairlift has a unique numberpaintedon the chair.Theid represents this number.Also, a chairmayhold anumberofskiers—youstorethisinthecountattributeandinitializeit tozero.Note, that theconstructordoesnotreturnanything,butrather itupdatesvaluesthatareuniquetotheinstance.

NOTE

RememberthatidisabuiltinfunctioninPython,butyoucanalsousethatasanattributenameinaclass.Youwillstillhaveaccesstotheidfunction.Everytimeyouwanttoaccesstheidattributeyouwilldoalookupontheinstance.Iftheinstancewasnamedchair,youwouldgettheidbycallingchair.id.So,thisisnotshadowingthebuiltinfunctionandisok.

You can tell that the constructor logic is finished when you see that theindentation level has stepped back out. You see another method defined (6),load.Thismethodrepresentsanactionthataninstanceoftheclasscanperform.In this case, a chair may load passengers onto it, and this method tells theinstancewhat todowhen thathappens.Again,self (the instance), is the firstparameter to the method. The second parameter, number, is the number ofpeople that get on the chair.Remember that a chair on a chairlift can usuallyholdmorethanoneperson,andinthiscase,yousaidthatyourchairmayholdupto fourpeople.Whenskiersboard thechairyouwouldwant tocall theload`methodon the chair, and insideof thebodyof thatmethod,update thecountattributeontheinstance.Likewise,thereisacorrespondingmethod,unload(7),thatshouldbecalled

whenskiersdismountatthetopofthehill.

NOTE

NOTE

Don't be intimidated bymethods.You have already seenmanymethods,such as the .capitalize method defined on a string. Methods arefunctionsthatareattachedtoaclass.Insteadofcallingthemethodbyitself,youneedtocallitonaninstanceoftheclass:

>>>'matt'.capitalize()'Matt'

Insummary,creatingaclassiseasy.Youdefinetheattributesyouwant.Theattributesthatareconstantareputinsidetheclassdefinition.Theattributesthatareuniquetoaninstanceareputintheconstructor.Youcanalsodefinemethodsthat contain actions that modify the instance of the class. After the class isdefined,Pythonwillcreateavariablewith thenameof theclass thatpoints totheclass:

>>>Chair<class'__main__.Chair'>

Youcaninspecttheclasswiththedirfunction.Notethattheclassattributesaredefinedontheclass,andyoucanaccessthemdirectlyontheclass:

>>>dir(Chair)['__class__','__delattr__','__dict__','__dir__','__doc__','__eq__','__format__','__ge__','__getattribute__','__gt__','__hash__','__init__','__le__','__lt__','__module__','__ne__','__new__','__reduce__','__reduce_ex__','__repr__','__setattr__','__sizeof__','__str__','__subclasshook__','__weakref__','load','max_occupants','unload']>>>Chair.max_occupants4

Thefigureinthissectionshowedhowtheattributesandmethodsoftheclassarestored.WecanalsoinspectthemfromtheREPL.BecauseeverythingisanobjectinPython,theywillallhavea__class__attribute:

>>>Chair.__class__<class'type'>>>>Chair.max_occupants.__class__

>>>Chair.max_occupants.__class__<class'int'>>>>Chair.__init__.__class__<class'function'>>>>Chair.load.__class__<class'function'>>>>Chair.unload.__class__<class'function'>

Themethodsarealsodefinedontheclass,buttheinstanceattributesarenot.Becauseinstanceattributesareuniquetotheinstance,theywillbestoredontheinstance.Ifyouhavedocstringsdefinedonyourclassor itsmethods,youcaninspect

themwithhelp:

>>>help(Chair)HelponclassChairinmodule__main__:classChair(builtins.object)|AChaironachairlift||Methodsdefinedhere:||__init__(self,id)||load(self,number)||unload(self,number)||------------------------------------------------------|Datadescriptorsdefinedhere:||__dict__|dictionaryforinstancevariables(ifdefined)||__weakref__|listofweakreferencestotheobject(ifdefined)||------------------------------------------------------|Dataandotherattributesdefinedhere:||max_occupants=4

Creatinganinstanceofaclass

Nowthatyouhavedefinedaclasstorepresentachair,youcancreateinstancesofchairs.OnewaytothinkoftheChairclassistocompareittoafactory.Thefactorytakesinbareobjectsandchurnsoutchairobjects.To be more specific, this is pretty much what the constructor method,

__init__ does. You'll notice that self is the first parameter, this is the bareobject.Pythonassignsthe__class__attribute(pointingtheChairclass)totheobjectbeforepassingitintotheconstructor.Another analogy that might help understand classes is to consider how

cartoons depict the delivery of babies. The unborn babies live in the cloudssomewhere.Atsomepoint in time,astorkcomesalong,picksupababyfromthe cloud and delivers it to a crib in the parent's house. When you call theconstructor,Pythongoesoutandpicksababyupfromthecloud(getsanobjectforyou).Itdeliversittothehouse,makingitamemberofthefamily(itsetsthe__class__ to Chair or whatever your class is). Once the baby is inside thehouse,youcanmoldher,cutherhair,etc.Rememberthatobjectsstorestateandmutatethestate.Below is code to create chair number 21 in Python.When you invoke the

class(i.e.putparenthesesaftertheclassname),youtellPythonthatyouwanttocalltheconstructor.Unlikesomelanguages,youdon'tneedtosayneworaddthevariable type, you just add parentheses with the constructor parametersfollowingthenameoftheclass:

>>>chair=Chair(21)

Again,tobespecificwithterminology,thechairvariablepointstoanobjector instance. It does not point to a class. The object has a class, Chair. Theinstancehasafewattributes,includingcountandid.

Thisillustratesconstructinganobject.Whentheconstructoriscalled,theproverbialPython"stork"dropsababyobjectasselfintotheconstructor.Thebabyobjecthasthe__class__

attributeset,buttheconstructorisfreetoaltertheinstance,byaddingattributes.Thisobjectwillturnintochair.

Youcanaccessaninstanceattributefromtheinstance,chair:

>>>chair.count0>>>chair.id21

Pythonhasahierarchyforlookingupattributes.First,Pythonwilllookfortheattributeontheinstance.Ifthatfails,Pythonwilltrytofindtheattributeontheclass.Becauseinstancesknowabouttheirclass,Pythonwill looktherenext.If

that fails Python will raise an AttributeError, an apt error for a missingattribute.Themax_occupants attribute is actually storedon the class, but youcanaccessitfromtheinstance:

>>>chair.max_occupants4

UnderthecoversPythonisdoingthisforyou:

>>>chair.__class__.max_occupants4

Attributelookupisdifferentthatvariablelookup.RecallthatPythonlooksforvariables first in the localscope, thenglobalscope, then thebuiltinscope,andfinally raisesaNameError if itwasunsuccessful.Attribute lookupoccurs firston the instance, then the class, then the parent classes, and will raise aAttributeErroriftheattributewasnotfound.

Callingamethodonaclass

Whenyouhaveaninstanceofaclass,youcancallmethodsonit.Methods—likefunctions—are invoked by adding parentheses with any arguments inside ofthem.Hereyouwillcallthe.loadmethodtoadd3skierstothechair:

>>>chair.load(3)

Here’s a review of the syntax of method invocation. First, you list theinstance,chair,which is followedby aperiod.Aperiodmeans search for anattribute in Python (unless it is following a number literal).When you see aperiodfollowinganinstance,rememberthatPythonisgoingtosearchforwhatfollowstheperiod.First,Pythonwilllookontheinstanceforload.Thisattributewasnotfound

ontheinstance,recallthatonlycountandidweresetontheinstanceinsideoftheconstructor.Buttheinstancehasalinktoitsclass.Soifthesearchfailsonthe instance, Python will look for those attributes on the class. The .loadmethod is definedon theChair class, soPython returns that.Theparentheses

mean invoke or call the method. You are passing 3 in as a parameter to themethod.Recallthatthedeclarationofloadlookedlikethis:

...defload(self,number):#6

...self.count+=number

In the declaration, therewere twoparameters,self andnumber.But in theinvocation,youonlypassedasingleparameter,3.Whythemismatch?Theselfparameterrepresentstheinstance,whichischairinyourcase.Pythonwillcallthe.loadmethodbyinsertingchairastheselfparameterand3asthenumberparameter.Ineffect,Pythonhandlespassingaroundtheselfparameterforyouautomatically.

NOTE

Whenyoucall:

chair.load(3)

Whathappensunderthecoversissimilartothis:

Chair.load(chair,3)

Youcantrythisouttovalidatethatitworks,butyouwouldn'tdothisinreallifebecauseitishardertoread,andalsorequiresmoretyping.

Examininganinstance

Ifyouhaveaninstanceandwanttoknowwhatitsattributesare,youhaveafewoptions.Youcanlookupthedocumentation(ifitexists).Youcanreadthecodewheretheclasswasdefined.Oryoucanusedirtoexamineitforyou:

>>>dir(chair)['__class__','__delattr__','__dict__','__dir__','__doc__','__eq__','__format__','__ge__','__getattribute__','__gt__','__hash__','__init__','__le__','__lt__','__module__','__ne__','__new__','__reduce__','__reduce_ex__','__repr__','__setattr__','__sizeof__','__str__',

'__subclasshook__','__weakref__','count','id','load','max_occupants','unload']

Recallthatthedirfunctionliststheattributesofanobject.Ifyoulookatthedocumentationfordir,youseethatthedefinitionpreviouslyprovidedfordirisnotquitecorrect.Thedocumentationreads:

returnanalphabetizedlistofnamescomprising(someof)theattributesofthegivenobject,andofattributesreachablefromit.—help(dir)(emphasisadded)

This function shows the attributes that are reachable from an object. Theactual state of the instance is stored in the __dict__ attribute, a dictionarymappingattributenamestovalues:

>>>chair.__dict__{'count':3,'id':21}

So, the instance really only stores count and id, the other attributes areavailable through the class. Where is the class stored? In the __class__attribute:

>>>chair.__class__<class'__main__.Chair'>

Itisimportantthataninstanceknowwhatitsclassisbecausetheclassstoresthemethodsandclassattributes.

Privateandprotected

Some languages have the notion of private attributes or methods. These aremethods that are meant to be implementation details and end users can't callthem.Infact,thelanguagemaypreventaccesstothem.Pythondoesnotmakeanefforttopreventusersfromdoingmuchofanything.

Rather, it takes the attitude that you are an adult and you should takeresponsibilityforyouractions.Ifyouwant toaccesssomething,youcandoit.Butyoushouldbewillingtoaccepttheconsequences.

Python programmers realize that it can be convenient to store state andmethodsthatareimplementationdetails.Tosignifytoendusersthattheyshouldnotaccessthesemembers,youprefixtheirnamewithanunderscore.Hereisaclassthathasahelpermethod,._check,thatisnotmeanttobecalledpublicly:

>>>classCorrectChair:...'''AChaironachairlift'''...max_occupants=4......def__init__(self,id):...self.id=id...self.count=0......defload(self,number):...new_val=self._check(self.count+number)...self.count=new_val......defunload(self,number):...new_val=self._check(self.count-number)...self.count=new_val......defcheck(self,number):...ifnumber<0ornumber>self.maxoccupants:...raiseValueError('Invalidcount:{}'.format(...number))...returnnumber

The._checkmethodisconsideredprivate,onlytheinstanceshouldaccessitinside the class. In the class, the.load and.unloadmethods call the privatemethod.Ifwanted,youcouldcallitfromoutsidetheclass.Butyoushouldn't,asanythingwithanunderscoreshouldbeconsideredanimplementationdetailthatmightnotexistinfutureversionsoftheclass.

Asimpleprogrammodelingflow

Let’susetheclassabovetomodelflowofskiersupahill.Youwillmakesomebasic assumptions, such as every chair has an equal probability of 0 tomax_occupantsridingonit.Itwillturnonthechairlift,loadit,andrunforever.Fourtimesaseconditprintsoutthecurrentstatistics:

importrandomimporttime

classCorrectChair:'''AChaironachairlift'''max_occupants=4

def__init__(self,id):self.id=idself.count=0

defload(self,number):new_val=self._check(self.count+number)self.count=new_val

defunload(self,number):new_val=self._check(self.count-number)self.count=new_val

defcheck(self,number):ifnumber<0ornumber>self.maxoccupants:raiseValueError('Invalidcount:{}'.format(number))returnnumber

NUM_CHAIRS=100

chairs=[]fornuminrange(1,NUM_CHAIRS+1):chairs.append(CorrectChair(num))

defavg(chairs):total=0forcinchairs:total+=c.countreturntotal/len(chairs)

in_use=[]transported=0whileTrue:#loadingloading=chairs.pop(0)in_use.append(loading)in_use[-1].load(random.randint(0,CorrectChair.max_occupants))

#unloadingiflen(in_use)>NUM_CHAIRS/2:unloading=in_use.pop(0)transported+=unloading.countunloading.unload(unloading.count)chairs.append(unloading)print('LoadingChair{}Count:{}Avg:{:.2}Total:{}'.format

(loading.id,loading.count,avg(in_use),transported))time.sleep(.25)

Thisprogramwillrunforeverprintingouthowmanypeopleareridingonthechairlift.Itoutputstotheterminal,buttheprintfunctioncouldbereplacedwithcodethatwritesaCSVfileaswell.By changing two numbers, the global NUM_CHAIRS and the class attribute

CorrectChair.max_occupants, you could change how themodel behaves torepresent larger or smaller chairlifts. The call to random.randint could bereplacedwithafunctionthatmorepreciselyrepresentstheusagedistribution.

Summary

Wedoveintoclassesabit inthischapter.Theterminologyaroundclasseswasdiscussed.Youcansayobjectorinstance;theyareinterchangeable.Eachobjecthasaclass.Theclass is likea factory thatdetermineshowits instanceobjectsbehave.An object is created by the constructor method. This method is named

__init__.Youmaydefineothermethodsforclassesaswell.Givesomethoughttocreatingaclass.Whataretheattributes?Iftheattributes

remainconsistentacrosstheobjects,definethemontheclass.Iftheyareuniquetotheobject,setthemintheconstructor.

Exercises

1. Imagineyouaredesigningabankingapplication.Whatwouldacustomerlooklike?Whatattributeswouldshehave?Whatmethodswouldshehave?

2. ImagineyouarecreatingaSuperMariogame.YouneedtodefineaclasstorepresentMario.Whatwoulditlooklike?Ifyouaren'tfamiliarwithSuperMario,useyourownfavoritevideoorboardgametomodelaplayer.

3. Create a class that could represent a tweet. If you aren't familiar withTwitter,Wikipediadescribesitas:

[...] an online news and social networking service where users postandinteractwithmessages,"tweets",restrictedto140characters.—https://en.wikipedia.org/wiki/Twitter

4. Createaclass thatcouldrepresentahouseholdappliance(toaster,washer,refrigerator,etc.).

SubclassingaClass

BESIDESGROUPINGSTATEANDACTIONINACOHERENTPLACE,CLASSESALSOENABLERE-use. Ifyoualreadyhaveaclassbutwantone thatbehaves slightlydifferently,onewaytore-usetheexistingclassistosubclassit.Theclassthatyousubclassfrom is called the superclass. (Another common term for superclass isparentclass).Supposethatyouwantedtocreateachairthatcanholdsixpeople.Tocreatea

class representing a six-person chair, Chair6, which is a more specializedversionof aChair, youcancreatea subclass.Subclassesallowyou to inheritmethodsofparentclasses,andoverridemethodsyouwanttochange.HereistheclassChair6,whichisasubclassofCorrectChair:

>>>classChair6(CorrectChair):...max_occupants=6

Illustrationofthe__bases__attributeinasubclass.Thisconnectionbetweenasubclassanditsparentclassesallowsyoutolookupattributesinawell-definedmanner.Iftheinstanceofasubclass

hasanattributedefined,itusesthatattribute.Ifnot,aftersearchingtheinstance,theclass(__class__)oftheinstanceissearched.Failingthat,parentclasses(__bases__)oftheclassare

searched.

Note that you put the parent class,CorrectChair, in parentheses followingthe class name. Notice that Chair6 doesn't define a constructor inside of the

body,yetyoucanstillcreateinstancesoftheclass:

>>>sixer=Chair6(76)

Illustrationdepictingtheinstantiationofasubclass.Notethattheinstancepointstoitsclassandthattheclasspointstoanyparentclassesusingthe__bases__attribute.

How does Python create an object, when the class doesn't define theconstructor? Here is what happens: when Python looks for the .__init__method,itwillsearchfor itonChair6 first.SincetheChair6classonlyhasamax_occupantsattribute,Pythonwillnotfindthe.__init__methodthere.But,becauseChair6isasubclassofCorrectChair,ithasa__bases__attributethatliststhebaseclassesinatuple:

>>>Chair6.__bases__(__main__.CorrectChair,)

Pythonwill then search the base classes for the constructor. Itwill find theconstructorinCorrectChairanduseittocreatethenewclass.Thesamelookuphappenswhenyoucall.loadonaninstance.Theinstance

doesn'thaveanattributematchingthemethodname,soPythonlooksattheclassoftheinstance.Chair6doesnothavethe.loadmethodeither,soPythonlooksfor the attribute on the base class, CorrectChair. Here the .load method iscalledwithacountthatistoolarge,leadingtoaValueError:

>>>sixer.load(7)Traceback(mostrecentcalllast):File"/tmp/chair.py",line30,in<module>sixer.load(7)File"/tmp/chair.py",line13,inloadnew_val=self._check(self.count+number)File"/tmp/chair.py",line23,in_checknumber))ValueError:Invalidcount:7

Pythonfindsthemethodonthebaseclass,butthecalltothe._checkmethodraisesaValueError.

Countingstalls

Asemi-commonoccurrenceisthataskierfailstomountthechaircorrectly.Inthat case, a chairlift operatorwill slowdownor stop the chairlift to assist theskier.YoucanusePythontocreateanewclassthatwillbeabletoaccountforthenumberoftimesastallhappens.

Assumethatyouwanttocallafunctioneverytime.load iscalled,andthisfunctionwillreturnabooleanindicatingwhetherastalloccurred.Thefunctiontakesasparametersthenumberofskiersandthechairobject.Below is a class that accepts anis_stalled function in the constructor. It

willcall this functionevery time.load is invoked todetermine if thechairlifthasstalled:

>>>classStallChair(CorrectChair):...def__init__(self,id,is_stalled):...super().__init__(id)...self.is_stalled=is_stalled...self.stalls=0......defload(self,number):...ifself.is_stalled(number,self):...self.stalls+=1...super().load(number)

Tocreateaninstanceofthisclass,youmustprovideanis_stalledfunction.Hereisasimplefunctionthatstallstenpercentofthetime:

>>>importrandom>>>deften_percent(number,chair):..."""ReturnTrue10%oftime"""...returnrandom.random()<.1

Now you can create an instance using the ten_percent function as youris_stalledparameter:

>>>stall42=StallChair(42,ten_percent)

Afigureshowingthecodeforcreatingasubclasswithcustommethods.Notethatyouusesuper()tocallthemethodontheparentclass.Thisillustrateswhatobjectsarecreatedwhenyoucreatea

classthatisasubclass.

super

RememberthatStallChairdefinesitsown.__init__method,whichiscalledwhentheinstanceiscreated.Notethatthefirstlineintheconstructoris:

super().__init__(id)

Whensuper is called insideof amethod, it givesyouaccess to the correctparentclass.ThislineintheconstructorallowsyoutoinvoketheCorrectChairconstructor.Rather than repeat the logicof setting theid andcount attribute,you can reuse the logic from the parent class. Because StallChair hasadditionalattributesitwantstosetontheinstance,youcandothatfollowingthecalltotheparentconstructor.Notethatthe.loadmethodalsohasacalltosuper:

defload(self,number):ifself.is_stalled(number,self):self.stalls+=1super().load(number)

In the .load method, you call your is_stalled function to determinewhether the chair was stalled. Then you dispatch back to the original .loadfunctionalityfoundinCorrectChairbyusingsuper.Havinggeneralcodeappearinoneplace(thebaseclass)eliminatesbugsand

repeatingcode.

NOTE

There are two cases where super really comes in handy. One is forresolving method resolution order (MRO) in classes that have multipleparents.superwillguaranteethatthisorderisconsistent.Theotheriswhenyouchangethebaseclass,super is intelligentaboutdeterminingwhothenewbaseis.Thisaidsincodemaintainability.

Summary

Thischapterdiscussedsubclasses,whicharenewspecializedclassesthatreusecodefromtheirbaseclass(alsoreferredtoassuperclassorparentclass).Foranymethod that you don't implement in a subclass, Python will reuse the parent

class' functionality. You can choose to implement a method to overridecompletely, or you can call out tosuper.When you callsuper, it gives youaccesstotheparentclasssoyoucanreuseanyfunctionalityfoundthere.

Exercises

1. Createaclasstorepresentacat.Whatcanacatdo?Whatarethepropertiesof a cat? Create a subclass of a cat for a tiger. How does it behavedifferently?

2. In the previous chapter, you created a class to representMario from theSuperMarioBrothersvideogame.Inlatereditionsofthegame,thereweredifferent characters that you could play. They all had the same basicfunctionality 15 but differed in skill. Create a base class for the character,thenimplementfoursubclasses.OneforMario,Luigi,Toad,andPrincessToadstool.

NAME MARIO LUIGI TOAD PRINCESSTOADSTOOL

Speed 4 3 5 2

Jump 4 5 2 3

Power 4 3 5 2

SpecialSkill Floatjump

15-https://www.mariowiki.com/Super_Mario_Bros._2#Playable_characters

Exceptions

ACOMPUTERMAYBETOLDTOPERFORMANACTIONTHATITCANNOTDO.READINGFILESthatdonotexistordividingbyzeroaretwoexamples.Pythonallowsyoutodealwith such exceptions when they occur. In these cases, Python throws anexceptionorraisesanexception.Normallywhen an exception occurs, Pythonwill halt and print out a stack

traceexplainingwheretheproblemoccurred.Thisstacktraceisusefulinthatittellsyouthelineandfileoftheerror:

>>>3/0Traceback(mostrecentcalllast):File"<stdin>",line1,in<module>ZeroDivisionError:divisionbyzero

The above states that in line 1 of the file<stdin> (this is the name of theinterpreter“file”)therewasadividebyzeroerror.Whenyouexecuteaprogramwithanexception,thestacktracewillindicatethefilenameandlinenumberofwheretheproblemoccurred.Thisexamplefromtheinterpreterisn'tparticularlyhelpfulasthereisonlyasinglelineofcode.Butinbiggerprogramsyoucouldhave deeply nested stack traces due to functions calling other functions andmethods:If you had the following in a file and a tried to run it, you would see a

traceback:

deferr():1/0defstart():returnmiddle()defmiddle():returnmore()

returnmore()defmore():err()start()

Hereisthestacktrace:

Traceback(mostrecentcalllast):File"/tmp/err.py",line13,in<module>start()File"/tmp/err.py",line5,instartreturnmiddle()File"/tmp/err.py",line8,inmiddlereturnmore()File"/tmp/err.py",line11,inmoreerr()File"/tmp/err.py",line2,inerr1/0ZeroDivisionError:divisionbyzero

Tracebacksareeasiesttoreadwhenyoustartfromthebottom,findtheerror,and see where it occurred. As you move up the traceback, you are in effectlookingupthecallchain.Thiscanhelpyoupinpointwhat isgoingoninyourprogram.

Lookbeforeyouleap

Suppose you have a program that performs division. Depending on how it iscoded,itmaybepossiblethatittriestodividebyzeroatsomepoint.Therearetwo styles for dealing with exceptions that programmers commonly use. Thefirstislookbeforeyouleap(LBYL).Theideaistocheckforexceptionalcasesbefore performing an action. In this case, the program would examine thedenominatorvalueanddetermineifitiszeroornot.Ifitisnotzero,theprogramcouldperformthedivision,otherwise,itcouldskipit.InPython,lookbeforeyouleapcanbeimplementedwithifstatements:

>>>numerator=10>>>divisor=0>>>ifdivisor!=0:...result=numerator/divisor...else:...result=None

...result=None

NOTE

Lookbeforeyouleapisnotalwaysaguaranteeofsuccess.Ifyoucheckthatafileexistsbeforeyouopenit(lookingbeforeleaping),thatdoesnotmeanthatthefilewillstillbearoundlater.Inmulti-threadedenvironments,thisisknownasaracecondition.

NOTE

None is used to represent the undefined state. This is a common idiomthroughoutPythondom.Becarefulthough,trynottoinvokemethodsonavariablethatisassignedtoNone,asthatwillraiseanexception.

Easiertoaskforforgiveness

Another option for dealing with exceptions is known as easier to ask forforgiveness than permission (EAFP). The idea here is to always perform theoperationinsideofatryblock.Iftheoperationfails,theexceptionwillbecaughtbytheexceptionblock.The try...except construct provides a mechanism for Python to catch

exceptionalcases:

>>>numerator=10>>>divisor=0>>>try:...result=numerator/divisor...exceptZeroDivisionErrorase:...result=None

Notice that the try construct creates a block following the try statement(because there is a colon and indentation). Inside of the try block are thestatements that might throw exceptions. If the statements actually throw an

exception, Python looks for an except block that catches that exception (or aparentclassofit).Inthecodeabove,theexceptblockstatesthatitwillcatchanexceptionthatis

an instance (or subclass) of the ZeroDivisionError class. When the statedexceptionisthrowninthetryblock,theexceptblockisexecutedandresultissettoNone.Notethatline:

exceptZeroDivisionErrorase:

has as e: on the end. This part is optional. If it is included then e (orwhatever you choose as a valid variable name)will point to an instance of aZeroDivisionError exception. You can inspect the exception as they oftenhavemoredetails.Theevariablepointstotheactiveexception.Ifyouleave"ase"off theendoftheexceptstatement,youwillstillhaveanactiveexception,butyouwon'tbeabletoaccesstheinstanceofit.

TIP

Trytolimitthescopeofthetryblock.Insteadofincludingallthecodeinafunction insideatryblock, includeonly the line thatwillpossibly throwtheerror.

Because the look before you leap style of handling is not guaranteed toprevent errors, in general, most Python developers favor the easier to ask forforgiveness style of exception handling. Here are some rules of thumb forexceptionhandling:

Gracefully handle errors you know how to handle and can reasonablyexpect.Do not silence exceptions that you cannot handle or do not reasonablyexpect.

Useaglobalexceptionhandlertogracefullyhandleunexpectederrors.

TIP

If you were making a server type application that needs to run withoutstopping, here is one way to do it. The functions process_input andlog_errordon'texistbutserveasplaceholders:

while1:try:result=process_input()exceptExceptionase:log_error(e)

Multipleexceptioncases

If there aremultiple exceptions that your code needs to be aware of, you canchainalistofexceptstatementstogether:

try:some_function()exceptZeroDivisionErrorase:#handlespecificexceptExceptionase:#handleothers

Inthiscase,whensome_functionthrowsanexception,theinterpreterchecksfirstifitisaZeroDivisionError(orasubclassofit).Ifthatisnotthecase,itchecks if the exception is a subclass ofException.Once anexcept block isentered,Pythonnolongerchecksthesubsequentblocks.Ifanexceptionisnothandledbythechain,codesomewhereupthestackmust

dealwiththeexception.Iftheexceptionisunhandled,Pythonwillstoprunningandwillprintthestacktrace.An example of dealing with multiple exceptions is found in the standard

library.Theargparsemodulefromthestandardlibraryprovidesaneasywaytoparsecommandlineoptions.Itallowsyoutospecifyatypeforcertainoptions,

suchas integersor files (theoptionsall come inas strings). In the.getvaluemethodthereareexamplesofmultipleexceptclauses.Basedonthetypeoftheexceptionthatoccursadifferenterrormessageisprovided:

defgetvalue(self,action,arg_string):

type_func=self._registry_get('type',action.type,action.type)ifnotcallable(type_func):msg=('%risnotcallable')raiseArgumentError(action,msg%typefunc)

#convertthevaluetotheappropriatetypetry:result=type_func(arg_string)

#ArgumentTypeErrorsindicateerrorsexceptArgumentTypeError:name=getattr(action.type,'__name__',repr(action.type))msg=str(_sys.exc_info()[1])raiseArgumentError(action,msg)

#TypeErrorsorValueErrorsalsoindicateerrorsexcept(TypeError,ValueError):name=getattr(action.type,'__name__',repr(action.type))args={'type':name,'value':arg_string}msg=_('invalid%(type)svalue:%(value)r')raiseArgumentError(action,msg%args)

#returntheconvertedvaluereturnresult

NOTE

This code example also shows that a single except statement can catchmorethanoneexceptiontypeifyouprovideatupleofexceptionclasses:

except(TypeError,ValueError):

NOTE

This example also shows an older style of string formatting using the %operator.Thelines:

msg=_('invalid%(type)svalue:%(value)r')raiseArgumentError(action,msg%args)

wouldbewritteninamoremodernstyleas:

msg=_('invalid{type!s}value:{value!r}')raiseArgumentError(action,msg.format(**args))

finallyclause

Anotherclauseforerrorhandlingisthefinallyclause.Thisstatementisusedto place code that will always execute, regardless of whether an exceptionhappens or not. If the try block succeeds, then the finally block will beexecuted.Thefinallyalwaysexecutes.Iftheexceptionishandled,thefinallyblock

will execute after the handling. If the exception is not handled, the finallyblockwillexecuteandthentheexceptionwillbere-raised:

try:some_function()exceptExceptionase:#handleerrorsfinally:#cleanup

Usually, thepurposeof thefinally clause is tocleanupexternal resources,suchasfiles,networkconnections,ordatabases.Theseresourcesshouldbefreedregardlessofwhetheranoperationwassuccessfulornot.Anexamplefromthetimeitmodulefoundinthestandardlibrarymightaid

in seeing the utility of the finally statement. The timeit module allowsdeveloperstorunabenchmarkontheircode.Oneofthethingsthatitdoeswhilerunning the benchmark is tell the Python garbage collector to disable itselfduringtherun.But,youwanttoensurethatgarbagecollectionisworkingafterthebenchmarkisdoneregardlessofwhethertherunworkedorerroredout.Hereisthemethod,timeitthatdispatchestorunthebenchmark.Itchecksif

thegarbagecollectorwasenabled,thenturnsoffthegarbagecollector,runsthe

timing code, and finally, re-enables garbage collection if it was previouslyenabled:

deftimeit(self,number=default_number):"""Time'number'executionsofthemainstatement.Tobeprecise,thisexecutesthesetupstatementonce,andthenreturnsthetimeittakestoexecutethemainstatementanumberoftimes,asafloatmeasuredinseconds.Theargumentisthenumberoftimesthroughtheloop,defaultingtoonemillion.Themainstatement,thesetupstatementandthetimerfunctiontobeusedarepassedtotheconstructor."""it=itertools.repeat(None,number)gcold=gc.isenabled()gc.disable()try:timing=self.inner(it,self.timer)finally:ifgcold:gc.enable()returntiming

Itispossiblethatthecalltoself.innerwillthrowanexception,butbecausethestandardlibraryusesfinally,garbagecollectionwillalwaysbeturnedbackonregardless(ifthegcoldbooleanistrue).

NOTE

This book doesn't address contextmanagers, but to prepare you for yourfutureasaPythonexpert,hereisahint.Thetry/finallycombinationisacode smell in Python. Seasoned Python programmers will use a contextmanager in these cases. Put this on your list of things to study after youhavemasteredbasicPython.

elseclause

Theoptionalelse clause inatry statement isexecutedwhennoexception israised. Itmust followanyexcept statementsandexecutesbefore thefinallyblock.Hereisasimpleexample:

>>>try:...print('hi')...exceptExceptionase:...print('Error')...else:...print('Success')...finally:...print('atlast')hiSuccessatlast

Here is an example from the heapq module found in the in the standardlibrary.According to thecomments, there isashortcut ifyouwant theget thesmallestvaluesandyouaskformorevaluesthanthesizeoftheheap.However,ifyoutrytogetthesizeoftheheapandgetanerror,thecodecallspass.Thisignores the error and continues on with the slower way of getting the smallitems.Iftherewasnoerror,youcanfollowtheelsestatementandtakethefastpathifnisgreaterthanthesizeoftheheap:

defnsmallest(n,iterable,key=None):#Coderemovedhere....#Whenn>=size,it'sfastertousesorted()try:size=len(iterable)except(TypeError,AttributeError)ase:passelse:ifn>=size:returnsorted(iterable,key=key)[:n]#Coderemovedhere....Tryslowerway

Raisingexceptions

In addition to catching exceptions,Python also allowsyou to raise exceptions(orthrowthem).RememberthattheZenofPythonwantsyoutobeexplicitandrefusethetemptationtoguess.Ifinvalidinputispassedintoyourfunctionandyou know that youwill not be able to handle it, youmay raise an exception.

ExceptionsaresubclassesoftheBaseExceptionclass,andareraisedusingtheraisestatement:

raiseBaseException('Programfailed')

Normally youwill not raise the genericBaseException class butwill raisesubclassesthatarepredefined,ordefinedbyyou.Anothercommonwayofusing theraise statement is touse itallby itself.

Recallthatwhenyouareinsideofanexceptstatement,youhavewhatiscalledanactiveexception. If that is thecase,youcanuseabareraise statement.Abare raise statement allows you to deal with the exception, but re-raise theoriginalexception.Ifyoutrytouse:

except(TypeError,AttributeError)ase:log('Hitanexception')raisee

You will succeed in raising the original exception, but the stack trace willstate that the original exceptionnowoccurred in the linewithraisee ratherthanwheretheexceptionfirstoccurred.Youhavetwooptionstodealwiththis.Thefirst isabare ``raise`` statement.Thesecond isusingexceptionchaining,describedlater.Here is an example from theconfigparsermodule in the standard library.

ThismodulehandlesreadingandcreatingINIfiles.AnINIfileisusuallyusedfor configurationandwaspopularbefore JSONandYAMLwerearound.The.read_dictmethodwill try toread theconfigurationfromadictionary. If theinstanceisinstrictmode,itwillraiseanerrorifyoutrytoaddthesamesectionmorethanonce.Ifyouarenotinstrictmode,themethodallowsduplicatekeys;the last one wins. Here is a portion of the method showing the bare raisemethod:

defread_dict(self,dictionary,source='<dict>'):elements_added=set()forsection,keysindictionary.items():section=str(section)try:self.add_section(section)

self.add_section(section)except(DuplicateSectionError,ValueError):ifself._strictandsectioninelements_added:raiseelements_added.add(section)#coderemovedfromhere....

Whenaduplicateisaddedinstrictmode,thestacktracewillshowtheerrorinthe.add_sectionmethod,becausethatiswhereitoccurred.

Wrappingexceptions

Python3hasintroducedanotherfeaturesimilartoabareraisein“PEP3134–ExceptionChaining andEmbeddedTracebacks”When handling an exception,another exceptionmay occur in the handling code. In this case, it is useful toknowaboutbothexceptions.Here is somebasic code.The function,divide_work,might haveproblems

dividingbyzero.Youcancatchthaterrorandlogthatitoccurred.Supposethatyour log function is calling out to a cloud-based service that is down (you'llsimulatethatbyhavinglograiseanexception):

>>>deflog(msg):...raiseSystemError("Loggingnotup")>>>defdivide_work(x,y):...try:...returnx/y...exceptZeroDivisionErrorasex:...log("Systemisdown")

When you call divide_work with 5 and 0 as input, Pythonwill show twoerrors, the ZeroDivisionError, and the SystemError. It will showSystemErrorlastbecauseithappenedlast:

>>>divide_work(5,0)Traceback(mostrecentcalllast):File"begpy.py",line3,individe_workreturnx/yZeroDivisionError:divisionbyzeroDuringhandlingoftheaboveexception,anotherexceptionoccurred:Traceback(mostrecentcalllast):

Traceback(mostrecentcalllast):File"begpy.py",line1,in<module>divide_work(5,0)File"begpy.py",line5,individe_worklog("Systemisdown")File"begpy.py",line2,inlograiseSystemError("Loggingnotup")SystemError:Loggingnotup

Suppose your cloud logging service is now working (the log function nolonger throws an error). If you want to change the type of theZeroDivisionError in divide_work to ArithmeticError, you can use asyntaxdescribedinPEP3134.Youcanusetheraise...fromsyntax:

>>>deflog(msg):...print(msg)>>>defdivide_work(x,y):...try:...returnx/y...exceptZeroDivisionErrorasex:...log("Systemisdown")...raiseArithmeticError()fromex

Youwill see twoexceptionsnow: theoriginalZeroDivisionError and theArithmeticErrorwhichisnolongershadowedbyZeroDivisionError:

>>>divide_work(3,0)Traceback(mostrecentcalllast):File"begpy.py",line3,individe_workreturnx/yZeroDivisionError:divisionbyzeroTheaboveexceptionwasthedirectcauseofthefollowingexception:Traceback(mostrecentcalllast):File"begpy.py",line1,in<module>divide_work(3,0)File"begpy.py",line6,individe_workraiseArithmeticError()fromexArithmeticError

Ifyouwanttosuppresstheoriginalexception,theZeroDivisionError,youcan use the following code. This is described in “PEP 0409 – Suppressingexceptioncontext”:

>>>defdivide_work(x,y):...try:...returnx/y...exceptZeroDivisionErrorasex:...log("Systemisdown")...raiseArithmeticError()fromNone

Nowyouonlyseetheoutermosterror,ArithmeticError:

>>>divide_work(3,0)Traceback(mostrecentcalllast):File"begpy.py",line1,in<module>divide_work(3,0)File"begpy.py",line6,individe_workraiseArithmeticError()fromNoneArithmeticError

Definingyourownexceptions

Pythonhasmanybuilt-inexceptionsdefinedintheexceptionsmodule.Ifyourerror correspondswellwith the existing exceptions, you can re-use them.Thefollowingliststheclasshierarchyforthebuilt-inexceptions:

BaseExceptionSystemExitKeyboardInterruptGeneratorExitExceptionStopIterationArithmeticErrorFloatingPointErrorOverflowErrorZeroDivisionErrorAssertionErrorAttributeErrorBufferErrorEnvironmentErrorIOErrorOSErrorEOFErrorImportErrorLookupErrorIndexErrorKeyErrorMemoryErrorNameErrorUnboundLocalErrorReferenceErrorRuntimeError

RuntimeErrorNotImplementedErrorSyntaxErrorIndentationErrorTabErrorSystemErrorTypeErrorValueErrorUnicodeErrorUnicodeDecodeErrorUnicodeEncodeErrorUnicodeTranslateErrorWarningDeprecationWarningPendingDeprecationWarningRuntimeWarningSyntaxWarningUserWarningFutureWarningImportWarningUnicodeWarningBytesWarning

Whendefiningyourownexception,youshouldsubclassfromExceptionorbelow. The reason for this is that other subclasses of BaseException are notnecessarily“exceptions”.Forexample,ifyoucaughtKeyboardInterrupt,youwouldn’t be able to stop the process with control-C. If you caughtGeneratorExit,generatorswouldstopworking.Hereisanexceptionfordefiningthataprogramismissinginformation:

>>>classDataError(Exception):...def__init__(self,missing):...self.missing=missing

Usingyourcustomexceptioniseasy:

>>>if'important_data'notinconfig:...raiseDataError('important_datamissing')

Summary

This chapter introduced strategies fordealingwithexceptions. InLookBeforeYouLeap,youmakesuretheenvironmentwillnotthrowanerrorbeforetryingsomething.InEasiertoAskforForgivenessthanPermission,youwrapanycode

thatyouknowmightthrowanerrorwithatry/catchblock.YoushouldfavorthelatterstyleofprogramminginPython.Thevariousmechanismsforcatchingerrors,raisingthem,andre-raisingthem

werediscussed.Finally,thechaptershowedhowyoucouldsubclassanexistingexceptiontocreateyourown.

Exercises

1. Writeaprogramthatservesasabasiccalculator.Itasksfortwonumbers,thenitasksforanoperator.Gracefullydealwithinputthatdoesn'tcleanlyconverttonumbers.Dealwithdivisionbyzeroerrors.

2. Writeaprogramthatinsertslinenumbersinfrontthelinesofafile.Accepta filenamebeing passed in on the command line. Import thesysmoduleandreadthefilenamefromthesys.argvlist.Gracefullydealwithabogusfilebeingpassedin.

ImportingLibraries

THEPREVIOUSCHAPTERSHAVECOVEREDTHEBASICCONSTRUCTSFORPYTHON. INTHISchapter,you'lllearnaboutimportingcode.Manylanguageshavetheconceptoflibraries or reusable chunks of code. Python comes with a whole swath oflibraries,commonlyreferredtoas“batteriesincluded”.Youneedtoknowhowtousethebatteriesthatarefoundintheselibraries.To use a library, you have to load the code from that library into your

namespace.Thenamespaceholdsthefunctions,classes,andvariablesyouhaveaccessto.Ifyouwanttocalculatethesineofanangleyouwillneedtodefineafunction thatdoes thator loadapreexistingfunction.Thebuilt-inmath libraryhas asin function that calculates the sineof an angle expressed in radians. Italsohasavariablethatdefinesavalueforpi:

>>>frommathimportsin,pi>>>sin(pi/2)1.0

The above code loads the math module. But it doesn't put math in yournamespace.Ratheritcreatesavariablethatpointstothesinfunctionfromthemathmodule.Italsocreatesavariablethatpointstothepivariablefoundinthemathmodule.Ifyouinspectyourcurrentnamespaceusingthedirfunctionyoucanconfirmthis:

>>>'sin'indir()True

Multiplewaystoimport

Inthepreviousexample,youimportedasinglefunctionfromalibrary.Itisalsopossibletoloadthelibraryintoyournamespaceandreferenceallofitsclasses,

functions,andvariables.Toimportthemathmoduleintoyournamespacetype:

>>>importmath

Thisillustratesimportingamodule.Notethatthiscodecreatesanewvariable,math,thatpointstoamodule.Themodulehasvariousattributesthatyoucanaccessusingaperiod.

In the above, you imported the math library. This created a new variable,math,thatpointstothemodule.Themodulehasvariousattributes.Youcanusethedirfunctiontolisttheattributes:

>>>dir(math)['__doc__','__file__','__loader__','__name__','__package__','__spec__','acos','acosh','asin','asinh','atan','atan2','atanh','ceil','copysign','cos','cosh','degrees','e','erf','erfc','exp','expm1','fabs','factorial','floor','fmod','frexp','fsum','gamma','gcd','hypot','inf','isclose','isfinite','isinf','isnan','ldexp','lgamma','log','log10','log1p','log2','modf','nan','pi','pow','radians','sin','sinh','sqrt','tan','tanh','trunc']

Mostoftheseattributesarefunctions.Ifyouwanttocallthetanfunction,youcan'tinvokeit,becausetanisn'tinyournamespace,onlymathis.But,youcando a lookup on the math variable using the period (.) operator. The periodoperatorwilllookupattributesonanobject.BecauseeverythinginPythonisanobject,youcanusetheperiodtolookupthetanattributeonthemathobject:

>>>math.tan(0)0.0

Ifyouwanttoreadtheassociateddocumentationforthetanfunction,youcanenlistthehelpfunctionforaid:

>>>help(math.tan)Helponbuilt-infunctiontaninmodulemath:tan(...)tan(x)Returnthetangentofx(measuredinradians).

TIP

When would you import a function using from versus the importstatement? If you are only using a couple attributes from a library, youmightwant touse thefrom style import. It ispossible tospecifymultiplecomma-delimitedattributesinthefromconstruct:

>>>frommathimportsin,cos,tan>>>cos(0)1.0

Howeverifyouneedtoaccessmostofthelibrary,itrequireslesstypingtoimportthelibraryusingtheimportstatement.Itisalsoahinttoanyonereading your code (including you), as to where the function (or class orvariable)camefrom.

NOTE

Ifyouneedtoimportmultipleattributesfromalibrary,youmightneedtospanmultiple lines. If you continue importing functions on the next line,youwillrunintoanerror:

>>>frommathimportsin,...cosTraceback(mostrecentcalllast):File"<stdin>",line1,in<module>SyntaxError:trailingcommanotallowedwithoutsurroundingparentheses

Youwillneed touseabackslash, to indicate that the linecontinuesonthenextline:

>>>frommathimportsin,\...cos

Otherwise, you can specify multiple imports with parentheses. Openparentheses(orbraces,orbrackets)indicatethatthestatementcontinuesonthenextline:

>>>frommathimport(sin,...cos)

ThislatterformismoreidiomaticPython.

Conflictingimportnames

Ifyouwereworkingonaprogramthatperformstrigonometricoperations,youmightalreadyhaveafunctionnamedsin.Whatifyoualsowanttousethesinfunctionfromthemath library?Oneoptionistoimportmath, thenmath.sinwouldreferencethelibraryandsinwouldreferenceyourfunction.Pythonhasanotheroptiontoo.Youcanredefinethenameofwhatyouwant

toimportusingtheaskeyword:

>>>frommathimportsinasother_sin>>>other_sin(0)0.0

Now, other_sin is a reference to the sin found in math and you maycontinueusingyoursinwithouthavingtorefactoryourcode.The as keyword construct also works on import statements. If you had a

variable(orafunction) thatconflictedwith themathnameinyournamespace,thefollowingwouldbeonewaytogetaroundit:

>>>importmathasother_math>>>other_math.sin(0)0.0

TIP

The as keyword can also be used to eliminate typing. If your favoritelibraryhasoverly longandverbosenamesyoucaneasilyshortentheminyour code. Users of the Numpy 16 library have adopted the standard ofreducingkeystrokesbyusingatwo-letteracronym:

>>>importnumpyasnp

ThePandas17libraryhasadoptedasimilarstandard:

>>>importpandasaspd

Starimports

Pythonalsoallowsyoutoclobberyournamespacewithwhatareknownasstarimports:

>>>frommathimport*>>>asin(0)0.0

Noticethattheabovecodecallsthearcsinefunction,whichwasnotdefined.Thelinewhereasinisinvokedisthefirstreferencetoasininthecode.Whathappened?Whenyousayfrommathimport*,thatisastarimport,andittellsPython to throweverything from themath library (classdefinitions, functions,

andvariables) into thelocalnamespace.While thismightappearhandyatfirstglance,itisquitedangerous.Star imports make debugging harder because it is not explicit where code

comes from. Even worse are star imports frommultiple libraries. Subsequentlibraryimportsmightoverridesomethingdefinedinanearlierlibrary.Assuch,starimportsarediscouragedandfrownedupon.

TIP

Donotusestarimports!Thepossibleexceptionstothisrulearewhenyouarewritingyourown

testingcode,ormessingaroundin theREPL.Libraryauthorsdothisasashortcuttoimportingeverythingfromthelibrarythattheywanttotest.Andthisoftenendsupinthedocumentation.Donotbetemptedbecauseyouseeitinother’scode,tousestarimportsinyourcode.RemembertheZenofPython:

Explicitisbetterthanimplicit

Nestedlibraries

SomePythonpackageshaveanestednamespace.Forexample,theXMLlibrarythatcomeswithPythonhassupportforminidomandetree.Bothlibrariesliveunderthexmlparentpackage:

>>>fromxml.dom.minidomimport\...parseString>>>dom=parseString(...'<xml><foo/></xml>')>>>fromxml.etree.ElementTreeimport\...XML>>>elem=XML('<xml><foo/></xml>')

Noticethatthefromconstructallowsimportingonlythefunctionsandclassesneeded.Using theimportconstruct (withoutfrom)wouldrequiremore typing(butalsoallowaccesstoeverythingfromthepackage):

>>>importxml.dom.minidom>>>dom=xml.dom.minidom.parseString(...'<xml><foo/></xml>')>>>importxml.etree.ElementTree>>>elem=xml.etree.ElementTree.XML(...'<xml><foo/></xml>')

Importorganization

According toPEP8, import statements shouldbe locatedat the topof the filefollowingthemoduledocstring.Thereshouldbeoneimportperlineandimportsshouldbegroupedby:

Standardlibraryimports3rdpartyimportsLocalpackageimports

Anexamplemodulemighthavethefollowingatthestart:

#!/usr/bin/envpython3"""ThismoduleconvertsrecordsintoJSONandshovesthemintoadatabase"""importjson#standardlibsimportsysimportpsycopg2#3rdpartylibimportrecordconverter#locallibrary...

TIP

Itisusefultoorganizethegroupedimportsalphabetically.

TIP

Itcanbeusefultopostponesomeimportsto:

Avoid circular imports. A circular import is where modulesmutuallyimportoneanother.Ifyouarenotable(orwilling)torefactorto remove the circular import, it is possible to place the importstatement within the function or method containing the code thatinvokesit.Avoidimportingmodulesthatarenotavailableonsomesystems.Avoidimportinglargemodulesthatyoumaynotuse.

Summary

ThischapterdiscussedimportinglibrariesinPython.Pythonhasalargestandardlibrary 18, andyouwill need to import those libraries to use them.Throughoutthisbook, ithasemphasized that everything is anobject, andyouoftencreatevariables thatpoint to thoseobjects.Whenyou import amodule,youcreate avariablethatpointstoamoduleobject.Anythingwithinthemodule’snamespacecanbeaccessedusingthelookupoperation(.).Youcanalso selectively importpartsoutof themodule’snamespace,using

thefromstatement.Ifyouwanttorenamewhatyouareimporting,youcanuseanasstatementtochangewhatthevariablenamewillbe.

Exercises

1. Find a package in the Python standard library for dealing with JSON.Importthelibrarymoduleandinspecttheattributesofthemodule.Usethehelp function to learn more about how to use the module. Serialize adictionarymapping'name'toyournameand'age'toyourage,toaJSONstring.DeserializetheJSONbackintoPython.

2. FindapackageinthePythonstandardlibraryforlistingdirectorycontents.Using that package, write a function that accepts a directory name. Thefunctionshouldgetallthefilesinthatdirectoryandprintoutareportofthecountoffilesbyextensiontype.

16-numpy.scipy.org17-pandas.pydata.org18-https://docs.python.org/3/library/index.html

Libraries:PackagesandModules

THEPREVIOUSCHAPTERDISCUSSEDHOWTOIMPORTLIBRARIES.THISCHAPTERWILLDIVEa little deeper into what constitutes a library. There are two requirements forimportingalibrary:

1. Thelibrarymustbeamoduleorapackage2. The library must exist in the PYTHONPATH environment variable or

sys.pathPythonvariable.

Modules

Modules arePython files that end in.py, andhaveaname that is importable.PEP 8 states that module filenames should be short and in lowercase.Underscoresmaybeusedforreadability.

Packages

ApackageinPythonisadirectorythatcontainsafilenamed__init__.py.Thefile named __init__.py can have any implementation it pleases or it can beempty. In addition, the directorymay contain an arbitrary number ofmodulesandsubpackages.Whenwritingcode,shouldyoupreferamoduleorapackage?Iusuallystart

simpleanduseamodule.WhenIneedtobreakcoherentpartsoutintotheirownmodules,thenIrefactorintomodulesinapackage.Here isanexample fromthedirectory layoutof thepopularSQLAlchemy 19

project(anObjectRelationalMapperfordatabases).

sqlalchemy/__init__.pyengine/__init__.py

__init__.pybase.pyschema.py

PEP8statesthatdirectorynamesforpackagesshouldbeshortandlowercase.Underscoresshouldnotbeused.

Importingpackages

To import a package, use the import statement with the package name (thedirectoryname):

>>>importsqlalchemy

This will import the sqlalchemy/__init__.py file into the currentnamespaceifthepackageisfoundinPYTHONPATHorsys.path.If you wanted to use the Column and ForeignKey classes found in the

schema.pymodule,eitherofthecodesnippetsbelowwouldwork.Thefirstputssqlalchemy.schema in your namespace, while the latter only puts schema inyournamespace:

>>>importsqlalchemy.schema>>>col=sqlalchemy.schema.Column()>>>fk=sqlalchemy.schema.ForeignKey()

or:

>>>fromsqlalchemyimportschema>>>col=schema.Column()>>>fk=schema.ForeignKey()

Alternatively,toaccessonlytheColumnclass,importthatclassinoneofthefollowingtwoways:

>>>importsqlalchemy.schema.Column>>>col=sqlalchemy.schema.Column()

or:

>>>fromsqlalchemy.schemaimportColumn>>>col=Column()

PYTHONPATH

PYTHONPATH is an environment variable listing non-standard directories thatPythonlooksformodulesorpackagesin.Thisvariableisusuallyempty.Itisnotnecessary to changePYTHONPATH unless you are developing code andwant touselibrariesthathavenotbeeninstalled.

TIP

LeavePYTHONPATHemptyunlessyouhaveagoodreasontochangeit.Thissectionillustrateswhatcanhappenifyouchangeit.Thiscanbeconfusingtoothers tryingtodebugyourcodewhoforget thatPYTHONPATHhasbeenchanged.

If you had some code in hometest/a/plot.py, but were working out ofhometest/b/, using PYTHONPATH allows access to that code. Otherwise, ifplot.py was not installed using system or Python tools, trying to import itwouldraiseanImportError:

>>>importplotTraceback(mostrecentcalllast):File"<stdin>",line1,in<module>ImportError:Nomodulenamedplot

IfyoustartPythonbysettingthePYTHONPATH,itindicatestoPythonwheretolookforlibraries:

$PYTHONPATH=hometest/apython3Python3.6.0(default,Dec242016,08:01:42)>>>importplot>>>plot.histogram()...

TIP

Python packages can be installed via package managers, WindowsexecutablesorPythonspecifictoolssuchaspip20.

sys.path

The sys module has an attribute, path, that lists the directories that Pythonsearchesforlibraries.Ifyouinspectsys.path,youwillseeallthelocationsthatarescanned:

>>>importsys>>>sys.path

['','usrlib/python35.zip','usrlib/python3.6','usrlib/python3.6/plat-darwin','usrlib/python3.6/lib-dynload','usrlocal/lib/python3.6/site-packages']

TIP

Ifyouseeerrorslike:

ImportError:Nomodulenamedplot

Look at the sys.path variable to see if it has the directory holdingfoo.py (if it isamodule). Ifplot isapackage, then theplot/directoryshouldbelocatedinoneofthepathsinsys.path:

>>>importplotTraceback(mostrecentcalllast):File"<stdin>",line1,in<module>ImportError:Nomodulenamedplot>>>sys.path.append('hometest/a')>>>importplot>>>plot.histogram()

Alternatively,youcansetPYTHONPATHtopointtothatdirectoryfromthecommandusedtoinvokePython.Again, typically you don't manually set sys.path or PYTHONPATH,

normally you install libraries, and the installer puts them in the correctlocation.

TIP

TIP

Ifyouwant toknowthelocationof thelibraryonthefilesystem,youcaninspectthe__file__attribute:

>>>importjson>>>json.__file__'usrlib/python3.6/json/__init__.py'

ThisonlyworkswithlibrariesimplementedinPython.ThesysmoduleisnotimplementedinPython,sothisfails:

>>>importsys>>>sys.__file__Traceback(mostrecentcalllast):...AttributeError:module'sys'hasnoattribute'__file__'

Summary

This chapter discussed modules and packages. A module is a Python file. Apackageisadirectorythathasafilenamed__init__.pyinit.Apackagemayalsocontainothermodulesandpackages.ThereisalistofpathsthatdetermineswherePythonlookstoimportlibraries.

This list is stored insys.path.Youcan inspect it to seewherePython looks.You can also update its value via the PYTHONPATH environment variable, ormutate the list directly. But typically, you don't set this variable to installpackages,youusesomethinglikepiptoinstallpackages.

Exercises

1. Create a module, begin.py, that has a function named prime in it. Theprime function should take a number and return a boolean indicatingwhetherthenumberisaprimenumber(divisibleonlyby1anditself).Gotoanotherdirectory,launchPython,andrun:

frombeginimportprime

It should fail. Update the sys.path variable, so that you can import thefunctionfromthemodule.Then,setPYTHONPATHtogetittoload.

2. Create a package,utils. In the__init__.py file, place theprime codefrom the previous exercise. Go to a different directory in the terminal,launchPython,andrun:

fromutilsimportprime

It should fail. Update the sys.path variable, so that you can import thefunctionfromthepackage.Then,setPYTHONPATHtogetittoload.

19-https://www.sqlalchemy.org/20-https://pip.pypa.io/

ACompleteExample

THISCHAPTERCOVERSHOWTOLAYOUTCODEWITHINASCRIPT.ITINCLUDESTHESOURCEforasimplified implementationof theUnixcommandcat.Thecatcommandreadsfilenamesfromthecommand lineandprints theircontents to thescreen.Therearevariousoptionstodothingslikeaddlinenumbering.ThisscriptwillillustratehowthecodeislaidoutinatypicalPythonfile.

cat.py

BelowarethecontentsofthePythonimplementationoftheUnixcommandcat.Itonlyincludesanoptionforaddinglinenumbers(--number),butnoneoftheothercatoptions.Youcanputthisinafilenamedcat.py:

#!usrbin/envpython3

r"""Asimpleimplementationoftheunix``cat``command.Itonlyimplementsthe``--number``option.ItisusefulforillustratingfilelayoutandbestpracticesinPython.

Thisisatriplequoteddocstringforthewholemodule(thisfile).Ifyouimportthismodulesomewhereelseandrun``help(cat)``,youwillseethis.

Thisdocstringalsocontainsa``doctest``whichservesasanexampleofprogrammaticallyusingthecode.Italsofunctionsasadoctest.The``doctest``modulecanexecutethisdocstringandvalidateitbycheckinganyoutput.

>>>importio>>>fin=io.StringIO(\...'hello\nworld\n')>>>fout=io.StringIO()>>>cat=Catter([fin],...show_numbers=True)

>>>cat.run(fout)>>>print(fout.getvalue())1hello2world

"""

importargparseimportloggingimportsys

__version__='0.0.1'

logging.basicConfig(level=logging.DEBUG)

classCatter(object):"""Aclasstoconcatenatefilestostandardout

Thisisaclassdocstring,``help(cat.Catter)``wouldshowthis."""

def__init__(self,files,show_numbers=False):self.files=filesself.show_numbers=show_numbers

defrun(self,fout):#use6spacesfornumbersandrightalignfmt='{0:>6}{1}'forfininself.files:logging.debug('catting{0}'.format(fin))forcount,lineinenumerate(fin,1):ifself.show_numbers:fout.write(fmt.format(count,line))else:fout.write(line)

defmain(args):"""Logictorunacatwitharguments

"""parser=argparse.ArgumentParser(description='ConcatenateFILE(s),or''standardinput,tostandardoutput')parser.add_argument('--version',action='version',version=__version__)parser.add_argument('-n','--number',action='store_true',help='numberalloutputlines')parser.add_argument('files',nargs='*',type=argparse.FileType('r'),default=[sys.stdin],metavar='FILE')parser.add_argument('--run-tests',action='store_true',help='runmoduletests')args=parser.parse_args(args)

ifargs.run_tests:importdoctestdoctest.testmod()else:cat=Catter(args.files,args.number)cat.run(sys.stdout)logging.debug('donecatting')

if__name__=='__main__':main(sys.argv[1:])

Ifyoudon'tfeelliketypinginthiscode,youcangrabacopyhere21.

Whatdoesthiscodedo?

This codewill echo the contents of a file (optionallywith line numbers) to aterminalonWindowsandUnixsystems:

$python3cat.py-nREADME.md1#IllustratedPy323Ifyouhavequestionsorconcerns,clickonIssuesabove.

Ifyou run thiscodewith-h itwillprintout thehelpdocumentation for thecommandlinearguments:

$python3cat.py-husage:cat.py[-h][--version][-n][--run-tests][FILE[FILE...]]ConcatenateFILE(s),orstandardinput,tostandardoutput

positionalarguments:FILEoptionalarguments:-h,--helpshowthishelpmessageandexit--versionshowprogram'sversionnumberandexit-n,--numbernumberalloutputlines--run-testsrunmoduletests

Thecommandlineparsingfunctionalityisimplementedinthemainfunctionand provided by the argparse module found in the standard library. Theargparsemodulehandlescommandlineargumentparsing.If youwant to knowwhat theargparsemodule does, you can lookup the

documentationontheweboryoucanusethehelpfunction.ThebasicideaofthemoduleisthatyoucreateaninstanceoftheArgumentParserclassandcall.add_argumentforeachcommandlineoption.Youprovidethecommandlineswitches,tellitwhatkindofactiontodo(thedefaultstoresthevaluefollowingtheswitch),andprovidehelpdocumentation.Afteryouhaveaddedarguments,youcallthe.parse_argsmethodonthecommandlinearguments(wegetthatfrom sys.argv). The result of .parse_args is an object that has attributesattached to it based on the option names. In this case, therewill be a.filesattributeanda.numberattribute.

NOTE

YoucanalsousetheREPLtoinspectthismoduleandanydocumentationthatitshipswith.Rememberthehelpfunction?Youcanpassamoduletoitanditwillprintoutthemoduleleveldocstring.Ifyouwantedtolookatthesourcecodeforthismodule,youcandothat

too. Remember the dir function? It lists attributes of an object. If youinspecttheargparsemodule,youseethatithasa__file__attribute.Thispointstothelocationonyourcomputerwherethefileis:

>>>importargparse>>>argparse.__file__'usrlocal/Cellar/python3/3.6.0/Frameworks/

Python.framework/Versions/3.6/lib/python3.6/argparse.py'

BecauseitiswritteninPython(somemodulesarewritteninC),youcaninspectthesource.Atthispoint,youshouldbeabletoreadthemoduleandunderstandwhatitistryingtodo.

Afterparsingthearguments,youcreateaninstanceoftheCatterclassthatisdefinedinthecode,andcallthe.runmethodonit.Thisexamplemightseemalittleoverwhelming,butyouhavecoveredallof

thesyntaxthatitillustratesoverthecourseofthebook.Therestofthischapterwilldiscussthelayoutandotheraspectsofthiscode.

Commonlayout

Here are the commoncomponents found in aPythonmodule and theorder inwhichtheyarefound:

#!usrbin/envpython3(shebang)(usedifmodulealsoservesasascript.)moduledocstringimportsmetadata/globalsloggingimplementationif__name__=='__main__':(usedifmodulealsoservesasascript.)argparse

NOTE

The above list is a recommendation. Most of those items can be in anarbitraryorder.Andnoteveryfilewillhavealltheseitems.Forinstancenoteveryfileneedstobeexecutableasashellscript.Youarefreetoorganizefileshowyouplease,butyoudosoatyourown

peril. Users of your code will likely complain (or submit patches). As a

readerofcode,youwillappreciatecodethatfollowstherecommendation,sinceitwillbequicklydiscoverable.

Shebang

The first line in a file (that is also used as a script) is the shebang line(#!usrbin/env python3). On Unix operating systems, this line is parsed todeterminehowtoexecutethescript.Thus,thislineisonlyincludedinfilesthataremeanttobeexecutableasscripts.Itshouldsaypython3,aspythonreferstoPythonversion2onmostsystems.

NOTE

TheWindowsplatformignorestheshebangline.Sothisissafetoinclude.IndeedyouwillfinditinlibrariesthatarealsopopularonWindows.

NOTE

RatherthanhardcodingaspecificpathtoaPythonexecutable,usrbin/envselectsthefirstpython3executablefoundontheuser’sPATH.Toolssuchasvenv22willmodifyyourPATHtouseacustompython3executableandwillworkwiththisconvention.

TIP

OnUnixsystems,ifthedirectorycontainingthefileispresentintheuser’sPATH environment variable, and the file is executable, then the file namealoneissufficientforexecutionfromashell.Type:

$chmod+x<path/to/file.py>

tomakeitexecutable.

Docstring

Amodulemay have amodule-level docstring at the top of the file. It shouldfollowtheshebanglinebutprecedeanyotherPythoncode.Adocstringservesasan overview of themodule and should contain a basic summary of the code.Also,itmaycontainexamplesofusingthemodule.

TIP

Python includes a library, doctest that can verify examples from aninteractive interpreter. Using docstrings that contain REPL code snippetscanservebothasdocumentationandsimplesanitytestsforyourlibrary.cat.py includesdoctestcodeat theendof itsdocstring.Whencat.py

runswith--run-tests,thedoctestlibrarywillcheckanydocstringsandvalidatethecodefoundinthem.Thiswasincludedforillustrationpurposesonly.Normallyanon-developerenduserwouldnotseeoptionsforrunningtestsinascript,thoughyoucouldincludedoctestsinthedocstrings.Inthiscase, the --run-tests option is included as an example of using thedoctestmodule.

Imports

ImportsareusuallyincludedatthetopofPythonmodules.Theimportlinesarenormallygroupedbylocationofthelibrary.First,listanylibrariesfoundinthePythonstandard library.Next list third-party libraries.Finally, list the librariesthat are local to the current code. Such organization allows end users of yourcodetoquicklyseeimports,requirements,andwherethecodeiscomingfrom.

Metadataandglobals

If you have legitimate module-level global variables, define them after theimports.Thismakes it easy to scan amodule and quickly determinewhat the

globalsare.Globalvariablesaredefinedatthemodule-levelandareaccessiblethroughout

that module. Because Python allows any variable to be modified, globalvariables are potential sources of bugs. In addition, it is easier to understandcodewhenvariables are defined andmodified onlywithin the function scope.Thenyoucanbesureofwhatdatayouhaveandwhoischangingit.Ifyouhavemultipleplaceswhereaglobalvariableisbeingmodified(especiallyifitisinadifferentmodule)youaresettingyourselfupforalongdebuggingsession.One legitimate use for globals is to emulate constants found in other

programminglanguages.Aconstantvariableisavariablewhosevaluedoesnotchange. Python doesn't support variables that don't change, but you can use aconventiontoindicatetotheuserthattheyshouldtreatavariableasread-only.PEP 8 states that global constants should have names that are the same asvariablesexcept theyshouldbecapitalized.Forexample, ifyouwanted tousethegoldenratioyoucoulddefineitlikethis:

>>>GOLDEN_RATIO=1.618

If thiscodewasdefined inamodule, thecapitalizationservesasahint thatyoushouldnotrebindthisvariable.

NOTE

By defining constants as globals, and using well thought-out variablenames,youcanavoidaproblemfoundinprogramming—magicnumbers.Amagicnumberisanumbersittingincodeoraformulathatisnotstoredin a variable. That in itself is bad enough, especiallywhen someone elsestartsreadingyourcode.Another problem with magic numbers is that the same value tends to

propagatethroughthecodeovertime.Thisisn’taproblemuntilyouwanttochange thatvalue.Doyoudoa searchand replace?What if themagicnumberactuallyrepresentstwodifferentvalues,i.e.thenumberofsidesof

a triangle and number of dimensions? In that case, a global search andreplacewillintroducebugs.Thesolutiontoboththeseproblems(contextandrepetition)istoputthe

value in a named variable. Having them in a variable gives context andnamingaroundthenumber.Italsoallowsyoutoeasilychangethevalueinoneplace.

Inadditiontoglobalvariables,therearealsometadatavariablesfoundatthatlevel.Metadatavariablesholdinformationaboutthemodule,suchasauthorandversion. Normally metadata variables are specified using “dunder” variablessuchas__author__.For example, PEP 396 recommends that the module version should be

specifiedinastring,__version__,attheglobalmodulelevel.

NOTE

It is a good idea to define a version for your library if you intend onreleasingittothewild.PEP396suggestsbestpracticesforhowtodeclareversionstrings.

Othercommonmetadatavariablesincludeauthor,license,date,andcontact.Ifthesewerespecifiedincodetheymightlooklikethis:

__author__='MattHarrison'__date__='Jan1,2017'__contact__='matt_harrison<at>someplace.com'__version__='0.1.1'

Logging

Onemorevariable that isoftendeclaredat theglobal level is the logger for amodule. The Python standard library includes thelogging library that allowsyoutoreportdifferentlevelsofinformationinwell-definedformats.

Multiple classes or functions in the same module will likely need to loginformation. It is common to perform logger initialization once at the globallevelandthenreusetheloggerhandlethatyougetbackthroughoutthemodule.

Otherglobals

You should not use a global variable when local a variable will suffice. ThecommonglobalsfoundinPythoncodearemetadata,constants,andlogging.Itisnotuncommontoseeglobalsscatteredaboutinsamplecode.Donotfall

tothetemptationtocopythiscode.Placeitintoafunctionoraclass.Thiswillpaydividendsinthefuturewhenyouarerefactoringordebuggingyourcode.

Implementation

Followinganyglobalandloggingsetupcomestheactualmeatofthecode—theimplementation.Functionsandclasseswillbeasubstantialportionofthiscode.TheCatterclasswouldbeconsideredthecorelogicofthemodule.

Testing

Normally,bonafidetestcodeisseparatedfromtheimplementationcode.Pythonallows a small exception to this. Python docstrings can be defined atmodule,function, class, and method levels. Within docstrings, you can place PythonREPL snippets illustrating how to use the function, class, or module. Thesesnippets, if well crafted and thought-out, can be effective in documentingcommonusageofthemodule.Another nice feature of doctest is validation of documentation. If your

snippetsonceworked,butnowtheyfail,eitheryourcodehaschangedoryoursnippets are wrong. You can easily find this out before end users startcomplainingtoyou.

TIP

doctest code can be in a stand-alone text file.To execute arbitrary filesusingdoctest,usethetestfilefunction:

importdoctest

importdoctestdoctest.testfile('module_docs.txt')

NOTE

Inadditiontodoctest,thePythonstandardlibraryincludestheunittestmodule that implements the common xUnit style methodology—setup,assert and teardown. There are pro’s and con’s to both doctest andunittest styles of testing. doctest tends to bemore difficult to debug,while unittest contains boilerplate code that is regarded as too Java-esque.Itispossibletocombinebothtoachievewelldocumentedandwelltestedcode.

if__name__=='__main__':

Ifyourfileismeanttoberunasascript,youwillfindthissnippetatthebottomofyourscript:

if__name__=='__main__':sys.exit(main(sys.argv[1:])or0)

Tounderstandthestatement,youshouldunderstandthe__name__variable.

__name__

Pythondefinesthemodulelevelvariable__name__foranymoduleyouimport,oranyfileyouexecute.Normally__name__’svalueisthenameofthemodule:

>>>importsys>>>sys.__name__'sys'>>>importxml.sax>>>xml.sax.__name__'xml.sax'

There isanexception to this rule.Whenamodule isexecuted (i.e.python3some_module.py),thenthevalueof__name__isthestring"__main__".Ineffect,thevalueof__name__indicateswhetherafileisbeingloadedasa

library,orrunasascript.

NOTE

It is easy to illustrate__name__.Create a file,some_module.py,with thefollowingcontents:

print("The__name__is:{0}".format(__name__))

NowrunaREPLandimportthismodule:

>>>importsome_moduleThe__name__is:some_module

Nowexecutethemodule:

$python3some_module.pyThe__name__is:__main__

It isacommonidiomthroughoutPythondomtoplaceachecksimilar to thefollowingatthebottomofamodulethatcouldalsoserveasascript.Thischeckwilldeterminewhetherthefileisbeingexecutedorimported:

if__name__=='__main__':#executesys.exit(main(sys.argv[1:])or0)

This simple statementwill run themain functionwhen the file is executed.Conversely,ifthefileisusedasamodule,mainwillnotberunautomatically.Itcallssys.exitwiththereturnvalueofmain(or0ifmaindoesnotreturnanexitcode)tobehaveasagoodcitizenintheUnixworld.Themain function takes thecommand lineoptionsasparameters.sys.argv

holds the command line options. It also containspython3 at the front, so youneedtoslicesys.argvtoignorethat,beforeyoupasstheoptionsintomain.

TIP

Somepeopleplacetheexecutionlogic(thecodeinsidethemainfunction)directlyundertheif__name__=='__main__':test.Reasonstokeepthelogicinsideafunctioninclude:

ThemainfunctioncanbecalledbyothersThemainfunctioncanbetestedeasilywithdifferentargumentsReducetheamountofcodeexecutingatthegloballevel

Summary

Inthischapter,youdissectedPythoncodeinascript.Thechapterdiscussedbestpracticesandcommoncodingconventions.By layingout your code as described in this chapter, youwill be following

bestpracticesforPythoncode.This layoutwillalsoaidothersneedingtoreadyourcode.

Exercises

1. Copy the cat.py code. Get it working on your computer. This isn't justbusy work. Much of the time when you are programming, you aren'tcreating something from scratch, but rather are reusing code that othershavewritten.

2. Write a script,convert.py, thatwill convert a file fromoneencoding toanother.Acceptthefollowingcommandlineoptions:

AninputfilenameAninputencoding(defaulttoutf-8)AnoutputencodingAnoptionforhandlingerrors(ignore/raise)

21-https://github.com/mattharrison/IllustratedPy3/

22-https://docs.python.org/3/library/venv.html

OntoBiggerandBetter

AT THIS POINTYOU SHOULDHAVEA PRETTYGOODUNDERSTANDINGOFHOW PYTHONprograms work. You should feel comfortable using the Python REPL andexploringclassesusingdirandhelp.Whatcomesnext?Thatisuptoyou.Youshouldnowhavetheprerequisites

tousePythontocreatewebsites,GUIprograms,ornumericalapplications.One huge benefit of Python is the various communities associatedwith the

different areas of programming. There are local user groups, newsgroups,mailing lists and social networks for many aspects of Python. Most of thesegroups are welcoming to new programmers and willing to share theirknowledge.Donot be afraid to try something new,Pythonmakes it easy andmostlikelythereareotherswhohavesimilarinterests.

FileNavigation

IFYOUARENOTFAMILIARWITHFILENAVIGATIONFROMATERMINAL,HERE ISABASICintroduction. First, you need to open a terminal. A terminal is one of thosewindowsyouseeon themovieswherehackers type ina lotof text.Youdon'thavetouseonetoprogram,butbeingabletonavigateandruncommandsfromtheterminalisausefulskilltohave.

MacandUnix

On Mac, type command-space to bring up Spotlight, then type terminal tolaunchtheterminalapplicationincludedonMacs.On Linux systems, launching a terminal depends on your desktop

environment.Forexample,onUbuntusystemsyoucanhitthekeyboardshortcutctr-alt-T.Aminimalterminalfoundinmostsystemsiscalledxterm.Thereareafewcommandstoknow:

cd-Changedirectorychangestoadifferentdirectory.Typing:

$cd~/Documents

willchange to theDocumentsdirectoryfound inyourhomefolder (~ isashortcut for home, which is Users<username> on Mac andhome<username>onLinux.pwd-Printworkingdirectorywilllistthecurrentdirectorythatyouarein.ls-Listdirectorywilllistthecontentsofthecurrentdirectory.

If you had a Python script located at~/work/intro-to-py/hello.py, youcouldtypethefollowingtorunit:

$cd~/work/intro-to-py

$cd~/work/intro-to-py$python3hello.py

Windows

OnWindows, typeWin-R to bring up the "RunWindow" interface, then typecmdtolaunchthecommandprompt.Thereareafewcommandstoknow:

cd-Changedirectorychangestoadifferentdirectory.Typing:

c:>cdC:\Users

willchangetotheC:\Usersdirectory.echo%CD%-Willlistthecurrentdirectorythatyouarein.dir-Listdirectorywilllistthecontentsofthecurrentdirectory.

If you had a Python script located at C:\Users\matt\intro-to-

py\hello.py,youcouldtypethefollowingtorunit:

C:>cdC:\Users\matt\intro-to-pyC:\Users\matt\intro-to-py>pythonhello.py

UsefulLinks

HEREARESOMEUSEFULPYTHONLINKS:

https://python.org/-Pythonhomepagehttps://github.com/mattharrison/Tiny-Python-3.6-Notebook - Python 3.6referencehttp://docutils.sourceforge.net/ - reStructuredText - lightweight markuplanguageforPythondocumentationhttps://pyformat.info-Usefulstringformattingreferencehttps://pypi.python.org/pypi-PythonPackageIndex-3rdpartypackageshttps://www.python.org/dev/peps/pep-0008/-PEP8-Codingconventionshttps://www.anaconda.com/download/ - Anaconda - Alternate Pythoninstallerwithmany3rdpartypackagesincludedhttps://www.djangoproject.com/-Django-Popularwebframeworkhttp://scikit-learn.org/-MachinelearningwithPythonhttps://www.tensorflow.org/-DeeplearningwithPythonhttps://www.reddit.com/r/Python/-NewsforPython

AbouttheAuthor

MATT HARRISON HAS BEEN USING PYTHON SINCE 2000. HE RUNS METASNAKE, APythonandDataScienceconsultancyandcorporatetrainingshop.Inthepast,hehas worked across the domains of search, build management and testing,businessintelligence,andstorage.He has presented and taught tutorials at conferences such as Strata, SciPy,

SCALE,PyCON,andOSCONaswellas localuserconferences.Thestructureand content of this book is based on first-hand experience teaching Python tomanyindividuals.He blogs at hairysun.com and occasionally tweets useful Python related

informationat@__mharrison__.

AbouttheTechnicalEditors

Roger A. Davidson is currently the Dean of Mathematics at American RiverCollege in Sacramento,CA.His doctorate is in aerospace engineering, but he

also holds degrees in computer science, electrical engineering and systemsengineeringinadditiontoarecentgraduatecertificateindatascience(wherehiscurrentloveofPythonbegan).DuringhiscareerthusfarRogerhasworkedforNASA, Fortune 50 companies, startups, and community colleges. He ispassionate about education, science (not just the data kind), wild blackberrycobblerandguidingdiverseteamstosolvebigproblems.AndrewMcLaughlin isaprogrammeranddesigner,asysadminbyday,and

familymanbynight.Withalovefordetail,he'sbeenbuildingthingsforthewebsince1998.AgraduatewithhonorsfromGeorgeFoxUniversity,Andrewhasadegree in Management and Information Systems. In his free time he enjoyshikingwithhiswifeand twokids,andoccasionallyworking inhiswoodshop.Hehasalltenfingers.Followhimontwitter:@amclaughlin

AlsoAvailable

BeginningPythonProgramming

TreadingonPython:BeginningPythonProgramming 23byMattHarrisonisthecompletebook to teachyouPythonfast.Designed toupyourPythongamebycoveringthebasics:

InterpreterUsageTypesSequencesDictionaries

FunctionsIndexingandSlicingFileInputandOutputClassesExceptionsImportingLibrariesTestingAndmore…

Reviews

Matt Harrison gets it, admits there are undeniable flaws and schisms inPython, and guides you through it in short and to the point examples. IboughtbothKindleandpaperbackeditionstoalwayshaveatthereadyforcontinuingtolearntocodeinPython.—S.Oakland

ThisbookwasagreatintrotoPythonfundamentals,andwasveryeasytoread.IespeciallylikedallthetipsandsuggestionsscatteredthroughouttohelpthereaderprogramPythonically:)—W.Dennis

Youdon'tneed1600pagestolearnPythonLast timeIwasusingPythonwhenLutz'sbookLearningPythonhadonly300pages.Forwhateverreasons, Ihave toreturn toPythonrightnow. Ihavediscoveredthatthesamebooktodayhas1600pages.Fortunately, I discoveredHarrison's books. I purchasedall of them,bothKindleandpaper.JustfewdayslaterIwasontrack.Harrison'spresentationisjustright.Shortandclear.Thereisno50pagesabouttheZenandphilosophyofusingif-then-elseconstruct.Justfacts.

—A.Customer

TreadingonPython:Vol2:IntermediatePython

Treading on Python: Vol 2: Intermediate Python 24 by Matt Harrison is thecomplete book on intermediate Python.Designed to up your Python game bycovering:

FunctionalProgrammingLambdaExpressionsListComprehensionsGeneratorComprehensionsIteratorsGeneratorsClosures

DecoratorsAndmore…

Reviews

Complete!All youmust knowaboutPythonDecorators: theory, practice,standarddecorators.Allwritteninaclearanddirectwayandveryaffordableprice.NicetoreadinKindle.—F.DeArruda(Brazil)

Thisisaverywellwrittenpiecethatdelivers.Nofluffandrighttothepoint,Mattdescribeshowfunctionsandmethodsareconstructed,thendescribesthevaluethatdecoratorsoffer.…Highlyrecommended,evenifyoualreadyknowdecorators,asthisisaverygoodexampleofhowtoexplainthissyntaxillusiontoothersinawaytheycangrasp.—JBabbington

DecoratorsexplainedthewaytheySHOULDbeexplained…Thereisanoldsayingtotheeffectthat“Everystickhastwoends,onebywhich itmay be picked up, and one bywhich itmay not.” I believe thatmostexplanationsofdecorators failbecause theypickup thestickby thewrongend.What I like about Matt Harrison’s e-book “Guide to: Learning PythonDecorators”isthatitisstructuredinthewaythatIthinkanintroductiontodecoratorsshouldbestructured.Itpicksupthestickbytheproperend…Whichisjustasitshouldbe.—S.Ferg

This bookwill clear up your confusions about functions even before youstart toreadaboutdecorationatall. Inaddition togettingstraightaboutscope,you'llfinallygetclarityaboutthedifferencebetweenargumentsandparameters, positional parameters, named parameters, etc. The authorconcedesthatthisintroductorymaterialissomethingthatsomereaderswillfind“pedantic,”butreportsthatmanypeoplefindithelpful.He’sbeingtoomodest. The distinctions he draws are essential to moving yourprogrammingskillsbeyonddoingaprettygoodimitationtorealfluency.—R.Careago

EbookFormatting:KF8,Mobi&EPUB

EbookFormatting:KF8,Mobi&EPUBbyMattHarrisonisthecompletebookon formatting for all Kindle and EPUB devices. A good deal of the eBooksfound in online stores today have problematic formatting.Ebook Formatting:KF8,Mobi&EPUBismeanttobeanaidinresolvingthoseissues.Customershate poorly formattedbooks.Read through the reviewsonAmazon andyou'llfindmany examples. This doesn't just happen to self-published books. BookscomingoutofbigPublishingHousesoftensuffersimilar issues. In therush toput out a book, one of the most important aspects affecting readability isignored.Thisbookscoversallaspectsofebookformattingonthemostpopulardevices

(Kindle,iPad,AndroidTablets,NookandKobo):

Covers

TitlePagesImagesCenteringstuffBoxmodelsupportTextStylesHeadingsPagebreaksTablesBlockquotesFirstsentencestylingNon-ASCIIcharactersFootnotesSidebarsMediaQueriesEmbeddedFontsJavascriptandmore…

23-http://hairysun.com/books/tread/24-http://hairysun.com/books/treadvol2/

Onemorething

THANKYOUFORBUYINGANDREADINGTHISBOOK.If you have found this book helpful, I have a big favor to ask. As a self-

published author, I don't have a big Publishing House with lots of marketingpowerpushingmybook.Ialsotrytopricemybookssothattheyaremuchmoreaffordable.Ifyouenjoyed thisbook, Ihope thatyouwould takeamoment to leavean

honest reviewonAmazon.Ashortcommentonhowthebookhelpedyouandwhatyour learnedmakesahugedifference.Aquickreviewisuseful tootherswhomightbeinterestedinthebook.Thanksagain!Writeareviewhere.