TableofContentsTheNode.jsHandbook

Introduction

IntroductiontoNode

AbriefhistoryofNode

HowtoinstallNode

HowmuchJavaScriptdoyouneedtoknowtouseNode?

DifferencesbetweenNodeandtheBrowser

v8

Basics

RunNode.jsscriptsfromthecommandline

HowtoexitfromaNode.jsprogram

Howtoreadenvironmentvariables

Nodehostingoptions

CommandLine

UsetheNodeREPL

Passargumentsfromthecommandline

Outputtothecommandline

Acceptinputfromthecommandline

Nodemodulesandnpm

ExposefunctionalityfromaNodefileusingexports

npm

Wheredoesnpminstallthepackages

Howtouseorexecuteapackageinstalledusingnpm

Thepackage.jsonfile

Thepackage-lock.jsonfile

Findtheinstalledversionofannpmpackage

Howtoinstallanolderversionofannpmpackage

HowtoupdatealltheNodedependenciestotheirlatestversion

Semanticversioningrules

Uninstallingnpmpackages

2

Globalorlocalpackages

npmdependenciesanddevDependencies

npx

Workingwiththeeventloop

Theeventloop

nextTick

setImmediate

Timers

Asynchronousprogramming

Callbacks

Promises

async/await

TheNodeEventEmitter

Networking

HTTP

HowHTTPRequestswork

BuildanHTTPserver

MakingHTTPrequests

Axios

Websockets

HTTPS,secureconnections

FileSystem

Filedescriptors

Filestats

Filepaths

Readingfiles

Writingfiles

Workingwithfolders

Someessentialcoremodules

Thefsmodule

Thepathmodule

Theosmodule

Theeventsmodule

3

Thehttpmodule

Miscellaneous

Streams

WorkingwithMySQL

Differencebetweendevelopmentandproduction

4

TheNode.jsHandbook

TheNodeHandbookfollowsthe80/20rule:learnin20%ofthetimethe80%ofatopic.

Ifindthisapproachgivesawell-roundedoverview.ThisbookdoesnottrytocovereverythingunderthesunrelatedtoNode.Ifyouthinksomespecifictopicshouldbeincluded,tellme.

YoucanreachmeonTwitter@flaviocopes.

Ihopethecontentsofthisbookwillhelpyouachievewhatyouwant:learnthebasicsNode.js.

ThisbookiswrittenbyFlavio.Ipublishwebdevelopmenttutorialseverydayonmywebsiteflaviocopes.com.

Enjoy!

TheNode.jsHandbook

5

IntroductiontoNodeThispostisagettingstartedguidetoNode.js,theserver-sideJavaScriptruntimeenvironment.Node.jsisbuiltontopoftheGoogleChromeV8JavaScriptengine,andit'smainlyusedtocreatewebservers-butit'snotlimitedtothat

OverviewThebestfeaturesofNode.js

FastSimpleJavaScriptV8AsynchronousplatformAhugenumberoflibraries

AnexampleNode.jsapplicationNode.jsframeworksandtools

OverviewNode.jsisaruntimeenvironmentforJavaScriptthatrunsontheserver.

Node.jsisopensource,cross-platform,andsinceitsintroductionin2009,itgothugelypopularandnowplaysasignificantroleinthewebdevelopmentscene.IfGitHubstarsareonepopularityindicationfactor,having46000+starsmeansbeingverypopular.

IntroductiontoNode

6

Node.jsisbuiltontopoftheGoogleChromeV8JavaScriptengine,andit'smainlyusedtocreatewebservers-butit'snotlimitedtothat.

ThebestfeaturesofNode.js

Fast

OneofthemainsellingpointsofNode.jsisspeed.JavaScriptcoderunningonNode.js(dependingonthebenchmark)canbetwiceasfastthancompiledlanguageslikeCorJava,andordersofmagnitudefasterthaninterpretedlanguageslikePythonorRuby,becauseofitsnon-blockingparadigm.

Simple

Node.jsissimple.Extremelysimple,actually.

JavaScript

Node.jsrunsJavaScriptcode.ThismeansthatmillionsoffrontenddevelopersthatalreadyuseJavaScriptinthebrowserareabletoruntheserver-sidecodeandfrontend-sidecodeusingthesameprogramminglanguagewithouttheneedtolearnacompletelydifferenttool.

Theparadigmsareallthesame,andinNode.jsthenewECMAScriptstandardscanbeusedfirst,asyoudon'thavetowaitforallyouruserstoupdatetheirbrowsers-youdecidewhichECMAScriptversiontousebychangingtheNode.jsversion.

V8

IntroductiontoNode

7

RunningontheGoogleV8JavaScriptengine,whichisOpenSource,Node.jsisabletoleveragetheworkofthousandsofengineersthatmade(andwillcontinuetomake)theChromeJavaScriptruntimeblazingfast.

Asynchronousplatform

Intraditionalprogramminglanguages(C,Java,Python,PHP)allinstructionsareblockingbydefaultunlessyouexplicitly"optin"toperformasynchronousoperations.IfyouperformanetworkrequesttoreadsomeJSON,theexecutionofthatparticularthreadisblockeduntiltheresponseisready.

JavaScriptallowstocreateasynchronousandnon-blockingcodeinaverysimpleway,byusingasinglethread,callbackfunctionsandevent-drivenprogramming.Everytimeanexpensiveoperationoccurs,wepassacallbackfunctionthatwillbecalledoncewecancontinuewiththeprocessing.We'renotwaitingforthattofinishbeforegoingonwiththerestoftheprogram.

Suchmechanismderivesfromthebrowser.Wecan'twaituntilsomethingloadsfromanAJAXrequestbeforebeingabletointerceptclickeventsonthepage.Itallmusthappeninrealtimetoprovideagoodexperiencetotheuser.

Ifyou'vecreatedanonclickhandlerforawebpageyou'vealreadyusedasynchronousprogrammingtechniqueswitheventlisteners.

ThisallowsNode.jstohandlethousandsofconcurrentconnectionswithasingleserverwithoutintroducingtheburdenofmanagingthreadsconcurrency,whichwouldbeamajorsourceofbugs.

Nodeprovidesnon-blockingI/Oprimitives,andgenerally,librariesinNode.jsarewrittenusingnon-blockingparadigms,makingablockingbehavioranexceptionratherthanthenormal.

IntroductiontoNode

8

WhenNode.jsneedstoperformanI/Ooperation,likereadingfromthenetwork,accessadatabaseorthefilesystem,insteadofblockingthethreadNode.jswillsimplyresumetheoperationswhentheresponsecomesback,insteadofwastingCPUcycleswaiting.

Ahugenumberoflibraries

npmwithitssimplestructurehelpedtheecosystemofnode.jsproliferateandnowthenpmregistryhostsalmost500.000opensourcepackagesyoucanfreelyuse.

AnexampleNode.jsapplicationThemostcommonexampleHelloWorldofNode.jsisawebserver:

consthttp=require('http')

consthostname='127.0.0.1'

constport=3000

constserver=http.createServer((req,res)=>{

res.statusCode=200

res.setHeader('Content-Type','text/plain')

res.end('HelloWorld\n')

})

server.listen(port,hostname,()=>{

console.log(`Serverrunningathttp://${hostname}:${port}/`)

})

Torunthissnippet,saveitasa server.jsfileandrun nodeserver.jsinyourterminal.

ThiscodefirstincludestheNode.js httpmodule.

Node.jshasanamazingstandardlibrary,includingafirst-classsupportfornetworking.

The createServer()methodof httpcreatesanewHTTPserverandreturnsit.

Theserverissettolistenonthespecifiedportandhostname.Whentheserverisready,thecallbackfunctioniscalled,inthiscaseinformingusthattheserverisrunning.

Wheneveranewrequestisreceived,the requesteventiscalled,providingtwoobjects:arequest(an http.IncomingMessageobject)andaresponse(an http.ServerResponseobject).

Those2objectsareessentialtohandletheHTTPcall.

Thefirstprovidestherequestdetails.Inthissimpleexample,thisisnotused,butyoucouldaccesstherequestheadersandrequestdata.

IntroductiontoNode

9

Thesecondisusedtoreturndatatothecaller.

Inthiscasewith

res.statusCode=200

wesetthestatusCodepropertyto200,toindicateasuccessfulresponse.

WesettheContent-Typeheader:

res.setHeader('Content-Type','text/plain')

andweendclosetheresponse,addingthecontentasanargumentto end():

res.end('HelloWorld\n')

Node.jsframeworksandtoolsNode.jsisalow-levelplatform,andtomakethingseasierandmoreinterestingfordevelopersthousandsoflibrarieswerebuiltuponNode.js.

Manyofthoseestablishedovertimeaspopularoptions.Hereisanon-comprehensivelisttotheonesIconsiderveryrelevantandworthlearning:

Express,oneofthemostsimpleyetpowerfulwaystocreateawebserver.Itsminimalistapproach,unopinionated,focusedonthecorefeaturesofaserver,iskeytoitssuccess.Meteor,anincrediblypowerfulfull-stackframework,poweringyouwithanisomorphicapproachtobuildingappswithJavaScript,sharingcodeontheclientandtheserver.Onceanoff-the-shelftoolthatprovidedeverything,nowintegrateswithfrontendlibsReact,VueandAngular.Canbeusedtocreatemobileappsaswell.koa,builtbythesameteambehindExpress,aimstobeevensimplerandsmaller,buildingontopofyearsofknowledge.Thenewprojectbornoutoftheneedtocreateincompatiblechangeswithoutdisruptingtheexistingcommunity.Next.js,aframeworktorenderserver-siderenderedReactapplications.Micro,averylightweightservertocreateasynchronousHTTPmicroservices.Socket.io,areal-timecommunicationenginetobuildnetworkapplications.

IntroductiontoNode

10

AbriefhistoryofNodeAlookbackonthehistoryofNode.jsfrom2009totoday

Believeitornot,Node.jsisjust9yearsold.

Incomparison,JavaScriptis23yearsoldandthewebasweknowit(aftertheintroductionofMosaic)is25yearsold.

9yearsissuchalittleamountoftimeforatechnology,butNode.jsseemstohavebeenaroundforever.

I'vehadthepleasuretoworkwithNodesincetheearlydayswhenitwasjust2yearsold,anddespitethelittleinformationavailable,youcouldalreadyfeelitwasahugething.

Inthispost,IwanttodrawthebigpictureofNodeinitshistory,toputthingsinperspective.

Alittlebitofhistory2009201020112012201320142015201620172018

AlittlebitofhistoryJavaScriptisaprogramminglanguagethatwascreatedatNetscapeasascriptingtooltomanipulatewebpagesinsidetheirbrowser,NetscapeNavigator.

PartofthebusinessmodelofNetscapewastosellWebServers,whichincludedanenvironmentcalledNetscapeLiveWire,whichcouldcreatedynamicpagesusingserver-sideJavaScript.Sotheideaofserver-sideJavaScriptwasnotintroducedbyNode.js,butit'soldjustlikeJavaScript-butatthetimeitwasnotsuccessful.

OnekeyfactorthatledtotheriseofNode.jswastiming.JavaScriptsinceafewyearswasstartingbeingconsideredaseriouslanguage,thanksforthe"Web2.0"applicationsthatshowedtheworldwhatamodernexperienceonthewebcouldbelike(thinkGoogleMapsor

AbriefhistoryofNode

11

GMail).

TheJavaScriptenginesperformancebarraisedconsiderablythankstothebrowsercompetitionbattle,whichisstillgoingstrong.Developmentteamsbehindeachmajorbrowserworkhardeverydaytogiveusbetterperformance,whichisahugewinforJavaScriptasaplatform.V8,theenginethatNode.jsusesunderthehood,isoneofthoseandinparticularit'stheChromeJSengine.

Butofcourse,Node.jsisnotpopularjustbecauseofpureluckortiming.ItintroducedmuchinnovativethinkingonhowtoprograminJavaScriptontheserver.

2009Node.jsisbornThefirstformofnpmiscreated

2010ExpressisbornSocket.ioisborn

2011npmhits1.0BigcompaniesstartadoptingNode:LinkedIn,UberHapiisborn

2012Adoptioncontinuesveryrapidly

2013FirstbigbloggingplatformusingNode:GhostKoaisborn

2014Bigdrama:IO.jsisamajorforkofNode.js,withthegoalofintroducingES6supportandmovefaster

2015

AbriefhistoryofNode

12

TheNode.jsFoundationisbornIO.jsismergedbackintoNode.jsnpmintroducesprivatemodulesNode4(no1,2,3versionswerepreviouslyreleased)

2016TheleftpadincidentYarnisbornNode6

2017npmfocusesmoreonsecurityNode8HTTP/2V8introducesNodeinitstestingsuite,officiallymakingNodeatargetfortheJSengine,inadditiontoChrome3billionnpmdownloadseveryweek

2018Node10ESmodules.mjsexperimentalsupport

AbriefhistoryofNode

13

HowtoinstallNodeHowyoucaninstallNode.jsonyoursystem:apackagemanager,theofficialwebsiteinstallerornvm

Node.jscanbeinstalledindifferentways.Thisposthighlightsthemostcommonandconvenientones.

Officialpackagesforallthemajorplatformsareavailableathttps://nodejs.org/en/download/.

OneveryconvenientwaytoinstallNode.jsisthroughapackagemanager.Inthiscase,everyoperatingsystemhasitsown.

OnmacOS,Homebrewisthede-factostandard,and-onceinstalled-allowstoinstallNode.jsveryeasily,byrunningthiscommandintheCLI:

brewinstallnode

OtherpackagemanagersforLinuxandWindowsarelistedinhttps://nodejs.org/en/download/package-manager/

nvmisapopularwaytorunNode.ItallowsyoutoeasilyswitchtheNodeversion,andinstallnewversionstotryandeasilyrollbackifsomethingbreaks,forexample.

ItisalsoveryusefultotestyourcodewitholdNodeversions.

Seehttps://github.com/creationix/nvmformoreinformationaboutthisoption.

Mysuggestionistousetheofficialinstallerifyouarejuststartingoutandyoudon'tuseHomebrewalready,otherwise,Homebrewismyfavoritesolution.

Inanycase,whenNodeisinstalledyou'llhaveaccesstothe nodeexecutableprograminthecommandline.

HowtoinstallNode

14

HowmuchJavaScriptdoyouneedtoknowtouseNode?IfyouarejuststartingoutwithJavaScript,howmuchdeeplydoyouneedtoknowthelanguage?

Asabeginner,it'shardtogettoapointwhereyouareconfidentenoughinyourprogrammingabilities.

Whilelearningtocode,youmightalsobeconfusedatwheredoesJavaScriptend,andwhereNode.jsbegins,andviceversa.

IwouldrecommendyoutohaveagoodgraspofthemainJavaScriptconceptsbeforedivingintoNode.js:

LexicalStructureExpressionsTypesVariablesFunctionsthisArrowFunctionsLoopsLoopsandScopeArraysTemplateLiteralsSemicolonsStrictModeECMAScript6,2016,2017

Withthoseconceptsinmind,youarewellonyourroadtobecomeaproficientJavaScriptdeveloper,inboththebrowserandinNode.js.

Thefollowingconceptsarealsokeytounderstandasynchronousprogramming,whichisonefundamentalpartofNode.js:

AsynchronousprogrammingandcallbacksTimersPromisesAsyncandAwaitClosuresTheEventLoop

HowmuchJavaScriptdoyouneedtoknowtouseNode?

15

LuckilyIwroteafreeebookthatexplainsallthosetopics,andit'scalledJavaScriptFundamentals.It'sthemostcompactresourceyou'llfindtolearnallofthis.

Youcanfindtheebookatthebottomofthispage:https://flaviocopes.com/javascript/.

HowmuchJavaScriptdoyouneedtoknowtouseNode?

16

DifferencesbetweenNodeandtheBrowserHowwritingJavaScriptapplicationinNode.jsdiffersfromprogrammingfortheWebinsidethebrowser

BoththebrowserandNodeuseJavaScriptastheirprogramminglanguage.

BuildingappsthatruninthebrowserisacompletelydifferentthingthanbuildingaNode.jsapplication.

Despitethefactthatit'salwaysJavaScript,therearesomekeydifferencesthatmaketheexperienceradicallydifferent.

AsafrontenddeveloperthatwritesNodeappshaveahugeadvantage-thelanguageisstillthesame.

Youhaveahugeopportunitybecauseweknowhowharditistofully,deeplylearnaprogramminglanguage,andbyusingthesamelanguagetoperformallyourworkontheweb-bothontheclientandontheserver,you'reinauniquepositionofadvantage.

Whatchangesistheecosystem.

Inthebrowser,mostofthetimewhatyouaredoingisinteractingwiththeDOM,orotherWebPlatformAPIslikeCookies.ThosedonotexistinNode,ofcourse.Youdon'thavethedocument, windowandalltheotherobjectsthatareprovidedbythebrowser.

Andinthebrowser,wedon'thavealltheniceAPIsthatNode.jsprovidesthroughitsmodules,likethefilesystemaccessfunctionality.

AnotherbigdifferenceisthatinNode.jsyoucontroltheenvironment.Unlessyouarebuildinganopensourceapplicationthatanyonecandeployanywhere,youknowwhichversionofNodeyouwillruntheapplicationon.Comparedtothebrowserenvironment,whereyoudon'tgettheluxurytochoosewhatbrowseryourvisitorswilluse,thisisveryconvenient.

ThismeansthatyoucanwriteallthemodernES6-7-8-9JavaScriptthatyourNodeversionsupports.

SinceJavaScriptmovessofast,butbrowserscanbeabitslowandusersabitslowtoupgrade,sometimesontheweb,youarestucktouseolderJavaScript/ECMAScriptreleases.

YYoucanuseBabeltotransformyourcodetobeES5-compatiblebeforeshippingittothebrowser,butinNode,youwon'tneedthat.

AnotherdifferenceisthatNodeusestheCommonJSmodulesystem,whileinthebrowserwearestartingtoseetheESModulesstandardbeingimplemented.

DifferencesbetweenNodeandtheBrowser

17

Inpractice,thismeansthatforthetimebeingyouuse require()inNodeand importinthebrowser.

DifferencesbetweenNodeandtheBrowser

18

v8V8isthenameoftheJavaScriptenginethatpowersGoogleChrome.It'sthethingthattakesourJavaScriptandexecutesitwhilebrowsingwithChrome.V8providestheruntimeenvironmentinwhichJavaScriptexecutes.TheDOMandtheotherWebPlatformAPIsareprovidedbythebrowser.

V8isthenameoftheJavaScriptenginethatpowersGoogleChrome.It'sthethingthattakesourJavaScriptandexecutesitwhilebrowsingwithChrome.

V8providestheruntimeenvironmentinwhichJavaScriptexecutes.TheDOM,andtheotherWebPlatformAPIsareprovidedbythebrowser.

ThecoolthingisthattheJavaScriptengineisindependentbythebrowserinwhichit'shosted.ThiskeyfeatureenabledtheriseofNode.js.V8waschosenforbeingtheenginechosenbyNode.jsbackin2009,andasthepopularityofNode.jsexploded,V8becametheenginethatnowpowersanincredibleamountofserver-sidecodewritteninJavaScript.

TheNode.jsecosystemishugeandthankstoitV8alsopowersdesktopapps,withprojectslikeElectron.

v8

19

OtherJSenginesOtherbrowsershavetheirownJavaScriptengine:

FirefoxhasSpidermonkeySafarihasJavaScriptCore(alsocalledNitro)EdgehasChakra

andmanyothersexistaswell.

AllthoseenginesimplementtheECMAES-262standard,alsocalledECMAScript,thestandardusedbyJavaScript.

ThequestforperformanceV8iswritteninC++,andit'scontinuouslyimproved.ItisportableandrunsonMac,Windows,Linuxandseveralothersystems.

InthisV8introduction,IwillignoretheimplementationdetailsofV8:theycanbefoundonmoreauthoritativesites(e.g.theV8officialsite),andtheychangeovertime,oftenradically.

V8isalwaysevolving,justliketheotherJavaScriptenginesaround,tospeeduptheWebandtheNode.jsecosystem.

Ontheweb,thereisaraceforperformancethat'sbeengoingonforyears,andwe(asusersanddevelopers)benefitalotfromthiscompetitionbecausewegetfasterandmoreoptimizedmachinesyearafteryear.

CompilationJavaScriptisgenerallyconsideredaninterpretedlanguage,butmodernJavaScriptenginesnolongerjustinterpretJavaScript,theycompileit.

Thishappenssince2009whentheSpiderMonkeyJavaScriptcompilerwasaddedtoFirefox3.5,andeveryonefollowedthisidea.

JavScriptisinternallycompiledbyV8withjust-in-time(JIT)compilationtospeeduptheexecution.

Thismightseemcounter-intuitive,butsincetheintroductionofGoogleMapsin2004,JavaScripthasevolvedfromalanguagethatwasgenerallyexecutingafewdozensoflinesofcodetocompleteapplicationswiththousandstohundredsofthousandsoflinesrunninginthebrowser.

v8

20

Ourapplicationsnowcanrunforhoursinsideabrowser,ratherthanbeingjustafewformvalidationrulesorsimplescripts.

Inthisnewworld,compilingJavaScriptmakesperfectsensebecausewhileitmighttakealittlebitmoretohavetheJavaScriptready,oncedoneit'sgoingtobemuchmoreperformantthatpurelyinterpretedcode.

v8

21

RunNode.jsscriptsfromthecommandlineHowtorunanyNode.jsscriptfromtheCLI

TheusualwaytorunaNodeprogramistocallthe nodegloballyavailablecommand(onceyouinstallNode)andpassthenameofthefileyouwanttoexecute.

IfyourmainNodeapplicationfileisin app.js,youcancallitbytyping

nodeapp.js

RunNode.jsscriptsfromthecommandline

22

HowtoexitfromaNode.jsprogramLearnhowtoterminateaNode.jsappinthebestpossibleway

TherearevariouswaystoterminateaNode.jsapplication.

Whenrunningaprogramintheconsoleyoucancloseitwith ctrl-C,butwhatIwanttodiscusshereisprogrammaticallyexiting.

Let'sstartwiththemostdrasticone,andseewhyyou'rebetteroffnotusingit.

The processcoremoduleisprovidesahandymethodthatallowsyoutoprogrammaticallyexitfromaNode.jsprogram: process.exit().

WhenNode.jsrunsthisline,theprocessisimmediatelyforcedtoterminate.

Thismeansthatanycallbackthat'spending,anynetworkrequeststillbeingsent,anyfilesystemaccess,orprocesseswritingto stdoutor stderr-allisgoingtobeungracefullyterminatedrightaway.

Ifthisisfineforyou,youcanpassanintegerthatsignalstheoperatingsystemtheexitcode:

process.exit(1)

Bydefaulttheexitcodeis 0,whichmeanssuccess.Differentexitcodeshavedifferentmeaning,whichyoumightwanttouseinyourownsystemtohavetheprogramcommunicatetootherprograms.

Youcanreadmoreonexitcodesathttps://nodejs.org/api/process.html#process_exit_codes

Youcanalsosetthe process.exitCodeproperty:

process.exitCode=1

andwhentheprogramwilllaterend,Nodewillreturnthatexitcode.

Aprogramwillgracefullyexitwhenalltheprocessingisdone.

ManytimeswithNodewestartservers,likethisHTTPserver:

constexpress=require('express')

constapp=express()

app.get('/',(req,res)=>{

res.send('Hi!')

})

HowtoexitfromaNode.jsprogram

23

app.listen(3000,()=>console.log('Serverready'))

Thisprogramisnevergoingtoend.Ifyoucall process.exit(),anycurrentlypendingorrunningrequestisgoingtobeaborted.Thisisnotnice.

InthiscaseyouneedtosendthecommandaSIGTERMsignal,andhandlethatwiththeprocesssignalhandler:

Note: processdoesnotrequirea"require",it'sautomaticallyavailable.

constexpress=require('express')

constapp=express()

app.get('/',(req,res)=>{

res.send('Hi!')

})

constserver=app.listen(3000,()=>console.log('Serverready'))

process.on('SIGTERM',()=>{

server.close(()=>{

console.log('Processterminated')

})

})

Whataresignals?SignalsareaPOSIXintercommunicationsystem:anotificationsenttoaprocessinordertonotifyitofaneventthatoccurred.

SIGKILListhesignalsthattellsaprocesstoimmediatelyterminate,andwouldideallyactlikeprocess.exit().

SIGTERMisthesignalsthattellsaprocesstogracefullyterminate.Itisthesignalthat'ssentfromprocessmanagerslike upstartor supervisordandmanyothers.

Youcansendthissignalfrominsidetheprogram,inanotherfunction:

process.kill(process.pid,'SIGTERM')

OrfromanotherNode.jsrunningprogram,oranyotherapprunninginyoursystemthatknowsthePIDoftheprocessyouwanttoterminate.

HowtoexitfromaNode.jsprogram

24

HowtoreadenvironmentvariablesLearnhowtoreadandmakeuseofenvironmentvariablesinaNode.jsprogram

The processcoremoduleofNodeprovidesthe envpropertywhichhostsalltheenvironmentvariablesthatweresetatthemomenttheprocesswasstarted.

HereisanexamplethataccessestheNODE_ENVenvironmentvariable,whichissettodevelopmentbydefault.

Note: processdoesnotrequirea"require",it'sautomaticallyavailable.

process.env.NODE_ENV//"development"

Settingitto"production"beforethescriptrunswilltellNodethatthisisaproductionenvironment.

Inthesamewayyoucanaccessanycustomenvironmentvariableyouset.

Howtoreadenvironmentvariables

25

NodehostingoptionsANode.jsapplicationcanbehostedinalotofplaces,dependingonyourneeds.Thisisalistofallthevariousoptionsyouhaveatyourdisposal

Hereisanon-exhaustivelistoftheoptionsyoucanexplorewhenyouwanttodeployyourappandmakeitpubliclyaccessible.

Iwilllisttheoptionsfromsimplestandconstrainedtomorecomplexandpowerful.

Simplestoptionever:localtunnelZeroconfigurationdeployments

GlitchCodepen

ServerlessPAAS

ZeitNowNanoboxHerokuMicrosoftAzureGoogleCloudPlatform

VirtualPrivateServerBaremetal

Simplestoptionever:localtunnelEvenifyouhaveadynamicIP,oryou'reunderaNAT,youcandeployyourappandservetherequestsrightfromyourcomputerusingalocaltunnel.

Thisoptionissuitedforsomequicktesting,demoaproductorsharingofanappwithaverysmallgroupofpeople.

Averynicetoolforthis,availableonallplatforms,isngrok.

Usingit,youcanjusttype ngrokPORTandthePORTyouwantisexposedtotheinternet.Youwillgetangrok.iodomain,butwithapaidsubscriptionyoucangetacustomURLaswellasmoresecurityoptions(rememberthatyouareopeningyourmachinetothepublicInternet).

Anotherserviceyoucanuseishttps://github.com/localtunnel/localtunnel

Zeroconfigurationdeployments

Nodehostingoptions

26

Glitch

Glitchisaplaygroundandawaytobuildyourappsfasterthanever,andseethemliveontheirownglitch.comsubdomain.Youcannotcurrentlyhaveaacustomdomain,andthereareafewrestrictionsinplace,butit'sreallygreattoprototype.Itlooksfun(andthisisaplus),andit'snotadumbeddownenvironment-yougetallthepowerofNode.js,aCDN,securestorageforcredentials,GitHubimport/exportandmuchmore.

ProvidedbythecompanybehindFogBugzandTrello(andco-creatorsofStackOverflow).

Iuseitalotfordemopurposes.

Codepen

Codepenisanamazingplatformandcommunity.Youcancreateaprojectwithmultiplefiles,anddeployitwithacustomdomain.

ServerlessAwaytopublishyourapps,andhavenoserveratalltomanage,isServerless.Serverlessisaparadigmwhereyoupublishyourappsasfunctions,andtheyrespondonanetworkendpoint(alsocalledFAAS-FunctionsAsAService).

Toverypopularsolutionsare

ServerlessFrameworkStandardLibrary

TheybothprovideanabstractionlayertopublishingonAWSLambdaandotherFAASsolutionsbasedonAzureortheGoogleCloudoffering.

PAASPAASstandsforPlatformAsAService.Theseplatformstakeawayalotofthingsyoushouldotherwiseworryaboutwhendeployingyourapplication.

ZeitNow

Zeitisaninterestingoption.Youjusttype nowinyourterminal,andittakescareofdeployingyourapplication.Thereisafreeversionwithlimitations,andthepaidversionismorepowerful.Yousimplyforgetthatthere'saserver,youjustdeploytheapp.

Nodehostingoptions

27

Nanobox

Nanobox

Heroku

Herokuisanamazingplatform.

ThisisagreatarticleongettingstartedwithNode.jsonHeroku.

MicrosoftAzure

AzureistheMicrosoftCloudoffering.

CheckouthowtocreateaNode.jswebappinAzure.

GoogleCloudPlatform

GoogleCloudisanamazingstructureforyourapps.

TheyhaveagoodNode.jsDocumentationSection

VirtualPrivateServerInthissectionyoufindtheusualsuspects,orderedfrommoreuserfriendlytolessuserfriendly:

DigitalOceanLinodeAmazonWebServices,inparticularImentionAmazonElasticBeanstalkasitabstractsawayalittlebitthecomplexityofAWS.

SincetheyprovideanemptyLinuxmachineonwhichyoucanwork,thereisnospecifictutorialforthese.

TherearelotsmoreoptionsintheVPScategory,thosearejusttheonesIusedandIwouldrecommend.

BaremetalAnothersolutionistogetabaremetalserver,installaLinuxdistribution,connectittotheinternet(orrentonemonthly,likeyoucandousingtheVultrBareMetalservice)

Nodehostingoptions

28

Nodehostingoptions

29

UsetheNodeREPLREPLstandsforRead-Evaluate-Print-Loop,andit'sagreatwaytoexploretheNodefeaturesinaquickway

The nodecommandistheoneweusetorunourNode.jsscripts:

nodescript.js

Ifweomitthefilename,weuseitinREPLmode:

node

Ifyoutryitnowinyourterminal,thisiswhathappens:

❯node>

thecommandstaysinidlemodeandwaitsforustoentersomething.

Tip:ifyouareunsurehowtoopenyourterminal,google"Howtoopenterminalon".

TheREPLiswaitingforustoentersomeJavaScriptcode,tobemoreprecise.

Startsimpleandenter

>console.log('test')

test

undefined

>

Thefirstvalue, test,istheoutputwetoldtheconsoletoprint,thenwegetundefinedwhichisthereturnvalueofrunning console.log().

WecannowenteranewlineofJavaScript.

UsethetabtoautocompleteThecoolthingabouttheREPListhatit'sinteractive.

Asyouwriteyourcode,ifyoupressthe tabkeytheREPLwilltrytoautocompletewhatyouwrotetomatchavariableyoualreadydefinedorapredefinedone.

UsetheNodeREPL

30

ExploringJavaScriptobjectsTryenteringthenameofaJavaScriptclass,like Number,addadotandpress tab.

TheREPLwillprintallthepropertiesandmethodsyoucanaccessonthatclass:

ExploreglobalobjectsYoucaninspecttheglobalsyouhaveaccesstobytyping global.andpressing tab:

UsetheNodeREPL

31

The_specialvariableIfaftersomecodeyoutype _,thatisgoingtoprinttheresultofthelastoperation.

DotcommandsTheREPLhassomespecialcommands,allstartingwithadot ..Theyare

.help:showsthedotcommandshelp.editor:enableseditormore,towritemultilineJavaScriptcodewithease.Onceyouareinthismode,enterctrl-Dtorunthecodeyouwrote..break:wheninputtingamulti-lineexpression,enteringthe.breakcommandwillabortfurtherinput.Sameaspressingctrl-C..clear:resetstheREPLcontexttoanemptyobjectandclearsanymulti-lineexpression

UsetheNodeREPL

32

currentlybeinginput..load:loadsaJavaScriptfile,relativetothecurrentworkingdirectory.save:savesallyouenteredintheREPLsessiontoafile(specifythefilename).exit:existstherepl(sameaspressingctrl-Ctwotimes)

TheREPLknowswhenyouaretypingamulti-linestatementwithouttheneedtoinvoke.editor.

Forexampleifyoustarttypinganiterationlikethis:

[1,2,3].forEach(num=>{

andyoupress enter,theREPLwillgotoanewlinethatstartswith3dots,indicatingyoucannowcontinuetoworkonthatblock.

...console.log(num)

...})

Ifyoutype .breakattheendofaline,themultilinemodewillstopandthestatementwillnotbeexecuted.

UsetheNodeREPL

33

PassargumentsfromthecommandlineHowtoacceptargumentsinaNode.jsprogrampassedfromthecommandline

YoucanpassanynumberofargumentswheninvokingaNode.jsapplicationusing

nodeapp.js

Argumentscanbestandaloneorhaveakeyandavalue.

Forexample:

nodeapp.jsflavio

or

nodeapp.jsname=flavio

ThischangeshowyouwillretrievethisvalueintheNodecode.

Thewayyouretrieveitisusingthe processobjectbuiltintoNode.

Itexposesan argvproperty,whichisanarraythatcontainsallthecommandlineinvocationarguments.

Thefirstargumentisthefullpathofthe nodecommand.

Thesecondelementisthefullpathofthefilebeingexecuted.

Alltheadditionalargumentsarepresentfromthethirdpositiongoingforward.

Youcaniterateoverallthearguments(includingthenodepathandthefilepath)usingaloop:

process.argv.forEach((val,index)=>{

console.log(`${index}:${val}`)

})

Youcangetonlytheadditionalargumentsbycreatinganewarraythatexcludesthefirst2params:

constargs=process.argv.slice(2)

Passargumentsfromthecommandline

34

Ifyouhaveoneargumentwithoutanindexname,likethis:

nodeapp.jsflavio

youcanaccessitusing

constargs=process.argv.slice(2)

args[0]

Inthiscase:

nodeapp.jsname=flavio

args[0]is name=flavio,andyouneedtoparseit.Thebestwaytodosoisbyusingtheminimistlibrary,whichhelpsdealingwitharguments:

constargs=require('minimist')(process.argv.slice(2))

args['name']//flavio

Passargumentsfromthecommandline

35

OutputtothecommandlineHowtoprinttothecommandlineconsoleusingNode,fromthebasicconsole.logtomorecomplexscenarios

BasicoutputusingtheconsolemoduleCleartheconsoleCountingelementsPrintthestacktraceCalculatethetimespentstdoutandstderrColortheoutputCreateaprogressbar

BasicoutputusingtheconsolemoduleNodeprovidesa consolemodulewhichprovidestonsofveryusefulwaystointeractwiththecommandline.

Itisbasicallythesameasthe consoleobjectyoufindinthebrowser.

Themostbasicandmostusedmethodis console.log(),whichprintsthestringyoupasstoittotheconsole.

Ifyoupassanobject,itwillrenderitasastring.

Youcanpassmultiplevariablesto console.log,forexample:

constx='x'

consty='y'

console.log(x,y)

andNodewillprintboth.

Wecanalsoformatprettyphrasesbypassingvariablesandaformatspecifier.

Forexample:

console.log('My%shas%dyears','cat',2)

%sformatavariableasastring%dor %iformatavariableasaninteger

Outputtothecommandline

36

%fformatavariableasafloatingpointnumber%Ousedtoprintanobjectrepresentation

Example:

console.log('%O',Number)

Cleartheconsoleconsole.clear()clearstheconsole(thebehaviormightdependontheconsoleused)

Countingelementsconsole.count()isahandymethod.

Takethiscode:

constx=1

consty=2

constz=3

console.count(

'Thevalueofxis'+x+'andhasbeenchecked..howmanytimes?'

)

console.count(

'Thevalueofxis'+x+'andhasbeenchecked..howmanytimes?'

)

console.count(

'Thevalueofyis'+y+'andhasbeenchecked..howmanytimes?'

)

Whathappensisthatcountwillcountthenumberoftimesastringisprinted,andprintthecountnexttoit:

Youcanjustcountapplesandoranges:

constoranges=['orange','orange']

constapples=['justoneapple']

oranges.forEach(fruit=>{

console.count(fruit)

})

apples.forEach(fruit=>{

console.count(fruit)

})

Outputtothecommandline

37

PrintthestacktraceTheremightbecaseswhereit'susefultoprintthecallstacktraceofafunction,maybetoanswerthequestionhowdidyoureachthatpartofthecode?

Youcandosousing console.trace():

constfunction2=()=>console.trace()

constfunction1=()=>function2()

function1()

Thiswillprintthestacktrace.Thisiswhat'sprintedifItrythisintheNodeREPL:

Trace

atfunction2(repl:1:33)

atfunction1(repl:1:25)

atrepl:1:1

atContextifyScript.Script.runInThisContext(vm.js:44:33)

atREPLServer.defaultEval(repl.js:239:29)

atbound(domain.js:301:14)

atREPLServer.runBound[aseval](domain.js:314:12)

atREPLServer.onLine(repl.js:440:10)

atemitOne(events.js:120:20)

atREPLServer.emit(events.js:210:7)

CalculatethetimespentYoucaneasilycalculatehowmuchtimeafunctiontakestorun,using time()and timeEnd()

constdoSomething=()=>console.log('test')

constmeasureDoingSomething=()=>{

console.time('doSomething()')

//dosomething,andmeasurethetimeittakes

doSomething()

console.timeEnd('doSomething()')

}

measureDoingSomething()

stdoutandstderrAswesawconsole.logisgreatforprintingmessagesintheConsole.Thisiswhat'scalledthestandardoutput,or stdout.

console.errorprintstothe stderrstream.

Outputtothecommandline

38

Itwillnotappearintheconsole,butitwillappearintheerrorlog.

ColortheoutputYoucancolortheoutputofyourtextintheconsolebyusingescapesequences.Anescapesequenceisasetofcharactersthatidentifiesacolor.

Example:

console.log('\x1b[33m%s\x1b[0m','hi!')

YoucantrythatintheNodeREPL,anditwillprint hi!inyellow.

However,thisisthelow-levelwaytodothis.Thesimplestwaytogoaboutcoloringtheconsoleoutputisbyusingalibrary.Chalkissuchalibrary,andinadditiontocoloringitalsohelpswithotherstylingfacilities,likemakingtextbold,italicorunderlined.

Youinstallitwith npminstallchalk,thenyoucanuseit:

constchalk=require('chalk')

console.log(chalk.yellow('hi!'))

Using chalk.yellowismuchmoreconvenientthantryingtoremembertheescapecodes,andthecodeismuchmorereadable.

ChecktheprojectlinkIpostedaboveformoreusageexamples.

CreateaprogressbarProgressisanawesomepackagetocreateaprogressbarintheconsole.Installitusing npminstallprogress

Thissnippetcreatesa10-stepprogressbar,andevery100msonestepiscompleted.Whenthebarcompleteswecleartheinterval:

constProgressBar=require('progress')

constbar=newProgressBar(':bar',{total:10})

consttimer=setInterval(()=>{

bar.tick()

if(bar.complete){

clearInterval(timer)

}

},100)

Outputtothecommandline

39

Outputtothecommandline

40

AcceptinputfromthecommandlineHowtomakeaNode.jsCLIprograminteractiveusingthebuilt-inreadlineNodemodule

HowtomakeaNode.jsCLIprograminteractive?

Nodesinceversion7providesthe readlinemoduletoperformexactlythis:getinputfromareadablestreamsuchasthe process.stdinstream,whichduringtheexecutionofaNodeprogramistheterminalinput,onelineatatime.

constreadline=require('readline').createInterface({

input:process.stdin,

output:process.stdout

})

readline.question(`What'syourname?`,(name)=>{

console.log(`Hi${name}!`)

readline.close()

})

Thispieceofcodeaskstheusername,andoncethetextisenteredandtheuserpressesenter,wesendagreeting.

The question()methodshowsthefirstparameter(aquestion)andwaitsfortheuserinput.Itcallsthecallbackfunctiononceenterispressed.

Inthiscallbackfunction,weclosethereadlineinterface.

readlineoffersseveralothermethods,andI'llletyoucheckthemoutonthepackagedocumentationIlinkedabove.

Ifyouneedtorequireapassword,it'sbesttonowechoitback,butinsteadshowinga *symbol.

Thesimplestwayistousethe readline-syncpackagewhichisverysimilarintermsoftheAPIandhandlesthisoutofthebox.

AmorecompleteandabstractsolutionisprovidedbytheInquirer.jspackage.

Youcaninstallitusing npminstallinquirer,andthenyoucanreplicatetheabovecodelikethis:

constinquirer=require('inquirer')

varquestions=[{

type:'input',

Acceptinputfromthecommandline

41

name:'name',

message:"What'syourname?",

}]

inquirer.prompt(questions).then(answers=>{

console.log(`Hi${answers['name']}!`)

})

Inquirer.jsletsyoudomanythingslikeaskingmultiplechoices,havingradiobuttons,confirmations,andmore.

It'sworthknowingallthealternatives,especiallythebuilt-inonesprovidedbyNode,butifyouplantotakeCLIinputtothenextlevel,Inquirer.jsisanoptimalchoice.

Acceptinputfromthecommandline

42

ExposefunctionalityfromaNodefileusingexportsHowtousethemodule.exportsAPItoexposedatatootherfilesinyourapplication,ortootherapplicationsaswell

Nodehasabuilt-inmodulesystem.

ANode.jsfilecanimportfunctionalityexposedbyotherNode.jsfiles.

Whenyouwanttoimportsomethingyouuse

constlibrary=require('./library')

toimportthefunctionalityexposedinthe library.jsfilethatresidesinthecurrentfilefolder.

Inthisfile,functionalitymustbeexposedbeforeitcanbeimportedbyotherfiles.

Anyotherobjectorvariabledefinedinthefilebydefaultisprivateandnotexposedtotheouterworld.

Thisiswhatthe module.exportsAPIofferedbythe modulesystemallowsustodo.

Whenyouassignanobjectorafunctionasanew exportsproperty,thatisthethingthat'sbeingexposed,andassuch,itcanbeimportedinotherpartsofyourapp,orinotherappsaswell.

Youcandosoin2ways.

Thefirstistoassignanobjectto module.exports,whichisanobjectprovidedoutoftheboxbythemodulesystem,andthiswillmakeyourfileexportjustthatobject:

constcar={

brand:'Ford',

model:'Fiesta'

}

module.exports=car

//..intheotherfile

constcar=require('./car')

Thesecondwayistoaddtheexportedobjectasapropertyof exports.Thiswayallowsyoutoexportmultipleobjects,functionsordata:

ExposefunctionalityfromaNodefileusingexports

43

constcar={

brand:'Ford',

model:'Fiesta'

}

exports.car=car

ordirectly

exports.car={

brand:'Ford',

model:'Fiesta'

}

Andintheotherfile,you'lluseitbyreferencingapropertyofyourimport:

constitems=require('./items')

items.car

or

constcar=require('./items').car

What'sthedifferencebetween module.exportsand exports?

Thefirstexposestheobjectitpointsto.Thelatterexposesthepropertiesoftheobjectitpointsto.

ExposefunctionalityfromaNodefileusingexports

44

npmAquickguidetonpm,thepowerfulpackagemanagerkeytothesuccessofNode.js.InJanuary2017over350000packageswerereportedbeinglistedinthenpmregistry,makingitthebiggestsinglelanguagecoderepositoryonEarth,andyoucanbesurethereisapackagefor(almost!)everything.

IntroductiontonpmDownloads

InstallingalldependenciesInstallingasinglepackageUpdatingpackages

VersioningRunningTasks

IntroductiontonpmnpmisthestandardpackagemanagerforNode.js.

npm

45

InJanuary2017over350000packageswerereportedbeinglistedinthenpmregistry,makingitthebiggestsinglelanguagecoderepositoryonEarth,andyoucanbesurethereisapackagefor(almost!)everything.

ItstartedasawaytodownloadandmanagedependenciesofNode.jspackages,butithassincebecomeatoolusedalsoinfrontendJavaScript.

Therearemanythingsthat npmdoes.

Yarnisanalternativetonpm.Makesureyoucheckitoutaswell.

Downloadsnpmmanagesdownloadsofdependenciesofyourproject.

Installingalldependencies

Ifaprojecthasa packages.jsonfile,byrunning

npminstall

itwillinstalleverythingtheprojectneeds,inthe node_modulesfolder,creatingitifit'snotexistingalready.

Installingasinglepackage

Youcanalsoinstallaspecificpackagebyrunning

npminstall<package-name>

Oftenyou'llseemoreflagsaddedtothiscommand:

--saveinstallsandaddstheentrytothe package.jsonfiledependencies--save-devinstallsandaddstheentrytothe package.jsonfiledevDependencies

ThedifferenceismainlythatdevDependenciesareusuallydevelopmenttools,likeatestinglibrary,while dependenciesarebundledwiththeappinproduction.

Updatingpackages

Updatingisalsomadeeasy,byrunning

npmupdate

npm

46

npmwillcheckallpackagesforanewerversionthatsatisfiesyourversioningconstraints.

Youcanspecifyasinglepackagetoupdateaswell:

npmupdate<package-name>

VersioningInadditiontoplaindownloads, npmalsomanagesversioning,soyoucanspecifyanyspecificversionofapackage,orrequireaversionhigherorlowerthanwhatyouneed.

Manytimesyou'llfindthatalibraryisonlycompatiblewithamajorreleaseofanotherlibrary.

Orabuginthelatestreleaseofalib,stillunfixed,iscausinganissue.

Specifyinganexplicitversionofalibraryalsohelpstokeepeveryoneonthesameexactversionofapackage,sothatthewholeteamrunsthesameversionuntilthe package.jsonfileisupdated.

Inallthosecases,versioninghelpsalot,and npmfollowsthesemanticversioning(semver)standard.

RunningTasksThepackage.jsonfilesupportsaformatforspecifyingcommandlinetasksthatcanberunbyusing

npmrun<task-name>

Forexample:

{

"scripts":{

"start-dev":"nodelib/server-development",

"start":"nodelib/server-production"

},

}

It'sverycommontousethisfeaturetorunWebpack:

{

"scripts":{

npm

47

"watch":"webpack--watch--progress--colors--configwebpack.conf.js",

"dev":"webpack--progress--colors--configwebpack.conf.js",

"prod":"NODE_ENV=productionwebpack-p--configwebpack.conf.js",

},

}

Soinsteadoftypingthoselongcommands,whichareeasytoforgetormistype,youcanrun

$npmrunwatch

$npmrundev

$npmrunprod

npm

48

WheredoesnpminstallthepackagesHowtofindoutwherenpminstallsthepackages

Readthenpmguideifyouarestartingoutwithnpm,it'sgoingtogoinalotofthebasicdetailsofit.

Whenyouinstallapackageusing npm(oryarn),youcanperform2typesofinstallation:

alocalinstallaglobalinstall

Bydefault,whenyoutypean npminstallcommand,like:

npminstalllodash

thepackageisinstalledinthecurrentfiletree,underthe node_modulessubfolder.

Asthishappens, npmalsoaddsthe lodashentryinthe dependenciespropertyofthepackage.jsonfilepresentinthecurrentfolder.

Aglobalinstallationisperformedusingthe -gflag:

npminstall-glodash

Whenthishappens,npmwon'tinstallthepackageunderthelocalfolder,butinstead,itwilluseagloballocation.

Where,exactly?

The npmroot-gcommandwilltellyouwherethatexactlocationisonyourmachine.

OnmacOSorLinuxthislocationcouldbe /usr/local/lib/node_modules.OnWindowsitcouldbe C:\Users\YOU\AppData\Roaming\npm\node_modules

Ifyouuse nvmtomanageNode.jsversions,however,thatlocationwoulddiffer.

Iforexampleuse nvmandmypackageslocationwasshownas/Users/flavio/.nvm/versions/node/v8.9.0/lib/node_modules.

Wheredoesnpminstallthepackages

49

HowtouseorexecuteapackageinstalledusingnpmHowtoincludeanduseinyourcodeapackageinstalledinyournode_modulesfolder

Whenyouinstallusing npmapackageintoyour node_modulesfolder,oralsoglobally,howdoyouuseitinyourNodecode?

Sayyouinstall lodash,thepopularJavaScriptutilitylibrary,using

npminstalllodash

Thisisgoingtoinstallthepackageinthelocal node_modulesfolder.

Touseitinyourcode,youjustneedtoimportitintoyourprogramusing require:

const_=require('lodash)

Whatifyourpackageisanexecutable?

Inthiscase,itwillputtheexecutablefileunderthe node_modules/.bin/folder.

Oneeasywaytodemonstratethisiscowsay.

Thecowsaypackageprovidesacommandlineprogramthatcanbeexecutedtomakeacowsaysomething(andotheranimalsaswellࢤ ).

Whenyouinstallthepackageusing npminstallcowsay,itwillinstallitselfandafewdependenciesinthenode_modulesfolder:

Howtouseorexecuteapackageinstalledusingnpm

50

Thereisahidden.binfolder,whichcontainssymboliclinkstothecowsaybinaries:

Howdoyouexecutethose?

Youcanofcoursetype ./node_modules/.bin/cowsaytorunit,anditworks,butnpx,includedintherecentversionsofnpm(since5.2),isamuchbetteroption.Youjustrun:

npxcowsay

andnpxwillfindthepackagelocation.

Howtouseorexecuteapackageinstalledusingnpm

51

Howtouseorexecuteapackageinstalledusingnpm

52

Thepackage.jsonfileThepackage.jsonfileisakeyelementinlotsofappcodebasesbasedontheNode.jsecosystem.

IfyouworkwithJavaScript,oryou'veeverinteractedwithaJavaScriptproject,Node.jsorafrontendproject,yousurelymetthe package.jsonfile.

What'sthatfor?Whatshouldyouknowaboutit,andwhataresomeofthecoolthingsyoucandowithit?

The package.jsonfileiskindofamanifestforyourproject.Itcandoalotofthings,completelyunrelated.It'sacentralrepositoryofconfigurationfortools,forexample.It'salsowhere npmand yarnstorethenamesandversionsofthepackageitinstalled.

ThefilestructurePropertiesbreakdown

name

author

contributors

bugs

homepage

version

license

keywords

description

repository

main

private

scripts

dependencies

devDependencies

engines

browserslist

Command-specificpropertiesPackageversions

ThefilestructureHere'sanexamplepackage.jsonfile:

Thepackage.jsonfile

53

{

}

It'sempty!Therearenofixedrequirementsofwhatshouldbeina package.jsonfile,foranapplication.TheonlyrequirementisthatitrespectstheJSONformat,otherwiseitcannotbereadbyprogramsthattrytoaccessitspropertiesprogrammatically.

Ifyou'rebuildingaNode.jspackagethatyouwanttodistributeover npmthingschangeradically,andyoumusthaveasetofpropertiesthatwillhelpotherpeopleuseit.We'llseemoreaboutthislateron.

Thisisanotherpackage.json:

{

"name":"test-project"

}

Itdefinesa nameproperty,whichtellsthenameoftheapp,orpackage,that'scontainedinthesamefolderwherethisfilelives.

Here'samuchmorecomplexexample,whichIextractedthisfromasampleVue.jsapplication:

{

"name":"test-project",

"version":"1.0.0",

"description":"AVue.jsproject",

"main":"src/main.js",

"private":true,

"scripts":{

"dev":"webpack-dev-server--inline--progress--configbuild/webpack.dev.conf.js",

"start":"npmrundev",

"unit":"jest--configtest/unit/jest.conf.js--coverage",

"test":"npmrununit",

"lint":"eslint--ext.js,.vuesrctest/unit",

"build":"nodebuild/build.js"

},

"dependencies":{

"vue":"^2.5.2"

},

"devDependencies":{

"autoprefixer":"^7.1.2",

"babel-core":"^6.22.1",

"babel-eslint":"^8.2.1",

"babel-helper-vue-jsx-merge-props":"^2.0.3",

"babel-jest":"^21.0.2",

"babel-loader":"^7.1.1",

"babel-plugin-dynamic-import-node":"^1.2.0",

"babel-plugin-syntax-jsx":"^6.18.0",

Thepackage.jsonfile

54

"babel-plugin-transform-es2015-modules-commonjs":"^6.26.0",

"babel-plugin-transform-runtime":"^6.22.0",

"babel-plugin-transform-vue-jsx":"^3.5.0",

"babel-preset-env":"^1.3.2",

"babel-preset-stage-2":"^6.22.0",

"chalk":"^2.0.1",

"copy-webpack-plugin":"^4.0.1",

"css-loader":"^0.28.0",

"eslint":"^4.15.0",

"eslint-config-airbnb-base":"^11.3.0",

"eslint-friendly-formatter":"^3.0.0",

"eslint-import-resolver-webpack":"^0.8.3",

"eslint-loader":"^1.7.1",

"eslint-plugin-import":"^2.7.0",

"eslint-plugin-vue":"^4.0.0",

"extract-text-webpack-plugin":"^3.0.0",

"file-loader":"^1.1.4",

"friendly-errors-webpack-plugin":"^1.6.1",

"html-webpack-plugin":"^2.30.1",

"jest":"^22.0.4",

"jest-serializer-vue":"^0.3.0",

"node-notifier":"^5.1.2",

"optimize-css-assets-webpack-plugin":"^3.2.0",

"ora":"^1.2.0",

"portfinder":"^1.0.13",

"postcss-import":"^11.0.0",

"postcss-loader":"^2.0.8",

"postcss-url":"^7.2.1",

"rimraf":"^2.6.0",

"semver":"^5.3.0",

"shelljs":"^0.7.6",

"uglifyjs-webpack-plugin":"^1.1.1",

"url-loader":"^0.5.8",

"vue-jest":"^1.0.2",

"vue-loader":"^13.3.0",

"vue-style-loader":"^3.0.1",

"vue-template-compiler":"^2.5.2",

"webpack":"^3.6.0",

"webpack-bundle-analyzer":"^2.9.0",

"webpack-dev-server":"^2.9.1",

"webpack-merge":"^4.1.0"

},

"engines":{

"node":">=6.0.0",

"npm":">=3.0.0"

},

"browserslist":[

">1%",

"last2versions",

"notie<=8"

]

}

therearelotsofthingsgoingonhere:

Thepackage.jsonfile

55

namesetstheapplication/packagenameversionindicatesthecurrentversiondescriptionisabriefdescriptionoftheapp/packagemainsettheentrypointfortheapplicationprivateifsetto truepreventstheapp/packagetobeaccidentallypublishedon npmscriptsdefinesasetofnodescriptsyoucanrundependenciessetsalistof npmpackagesinstalledasdependenciesdevDependenciessetsalistof npmpackagesinstalledasdevelopmentdependenciesenginessetswhichversionsofNodethispackage/appworksonbrowserslistisusedtotellwhichbrowsers(andtheirversions)youwanttosupport

Allthosepropertiesareusedbyeither npmorothertoolsthatwecanuse.

PropertiesbreakdownThissectiondescribesthepropertiesyoucanuseindetail.Ireferto"package"butthesamethingappliestolocalapplicationswhichyoudonotuseaspackages.

Mostofthosepropertiesareonlyusedonthehttps://www.npmjs.com/,otherbyscriptsthatinteractwithyourcode,like npmorothers.

name

Setsthepackagename.

Example:

"name":"test-project"

Thenamemustbelessthan214characters,mustnothavespaces,itcanonlycontainlowercaseletters,hyphens( -)orunderscores( _).

Thisisbecausewhenapackageispublishedon npm,itgetsitsownURLbasedonthisproperty.

IfyoupublishedthispackagepubliclyonGitHub,agoodvalueforthispropertyistheGitHubrepositoryname.

author

Liststhepackageauthorname

Example:

Thepackage.jsonfile

56

{

"author":"FlavioCopes<flavio@flaviocopes.com>(https://flaviocopes.com)"

}

Canalsobeusedwiththisformat:

{

"author":{

"name":"FlavioCopes",

"email":"flavio@flaviocopes.com",

"url":"https://flaviocopes.com"

}

}

contributors

Aswellastheauthor,theprojectcanhaveoneormorecontributors.Thispropertyisanarraythatliststhem.

Example:

{

"contributors":[

"FlavioCopes<flavio@flaviocopes.com>(https://flaviocopes.com)"

]

}

Canalsobeusedwiththisformat:

{

"contributors":[

{

"name":"FlavioCopes",

"email":"flavio@flaviocopes.com",

"url":"https://flaviocopes.com"

}

]

}

bugs

Linkstothepackageissuetracker,mostlikelyaGitHubissuespage

Example:

{

"bugs":"https://github.com/flaviocopes/package/issues"

Thepackage.jsonfile

57

}

homepage

Setsthepackagehomepage

Example:

{

"homepage":"https://flaviocopes.com/package"

}

version

Indicatesthecurrentversionofthepackage.

Example:

"version":"1.0.0"

Thispropertyfollowsthesemanticversioning(semver)notationforversions,whichmeanstheversionisalwaysexpressedwith3numbers: x.x.x.

Thefirstnumberisthemajorversion,thesecondtheminorversionandthethirdisthepatchversion.

Thereisameaninginthesenumbers:areleasethatonlyfixesbugsisapatchrelease,areleasethatintroducesbackward-compatiblechangesisaminorrelease,amajorreleasecanhavebreakingchanges.

license

Indicatesthelicenseofthepackage.

Example:

"license":"MIT"

keywords

Thispropertycontainsanarrayofkeywordsthatassociatewithwhatyourpackagedoes.

Example:

Thepackage.jsonfile

58

"keywords":[

"email",

"machinelearning",

"ai"

]

Thishelpspeoplefindyourpackagewhennavigatingsimilarpackages,orwhenbrowsingthehttps://www.npmjs.com/website.

description

Thispropertycontainsabriefdescriptionofthepackage

Example:

"description":"Apackagetoworkwithstrings"

Thisisespeciallyusefulifyoudecidetopublishyourpackageto npmsothatpeoplecanfindoutwhatthepackageisabout.

repository

Thispropertyspecifieswherethispackagerepositoryislocated.

Example:

"repository":"github:flaviocopes/testing",

Noticethe githubprefix.Thereareotherpopularservicesbakedin:

"repository":"gitlab:flaviocopes/testing",

"repository":"bitbucket:flaviocopes/testing",

Youcanexplicitlysettheversioncontrolsystem:

"repository":{

"type":"git",

"url":"https://github.com/flaviocopes/testing.git"

}

Youcanusedifferentversioncontrolsystems:

Thepackage.jsonfile

59

"repository":{

"type":"svn",

"url":"..."

}

main

Setstheentrypointforthepackage.

Whenyouimportthispackageinanapplication,that'swheretheapplicationwillsearchforthemoduleexports.

Example:

"main":"src/main.js"

private

ifsetto truepreventstheapp/packagetobeaccidentallypublishedon npm

Example:

"private":true

scripts

Definesasetofnodescriptsyoucanrun

Example:

"scripts":{

"dev":"webpack-dev-server--inline--progress--configbuild/webpack.dev.conf.js",

"start":"npmrundev",

"unit":"jest--configtest/unit/jest.conf.js--coverage",

"test":"npmrununit",

"lint":"eslint--ext.js,.vuesrctest/unit",

"build":"nodebuild/build.js"

}

Thesescriptsarecommandlineapplications.Youcanrunthembycalling npmrunXXXXoryarnXXXX,where XXXXisthecommandname.Example: npmrundev.

Youcanuseanynameyouwantforacommand,andscriptscandoliterallyanythingyouwant.

Thepackage.jsonfile

60

dependencies

Setsalistof npmpackagesinstalledasdependencies.

Whenyouinstallapackageusingnpmoryarn:

npminstall<PACKAGENAME>

yarnadd<PACKAGENAME>

thatpackageisautomaticallyinsertedinthislist.

Example:

"dependencies":{

"vue":"^2.5.2"

}

devDependencies

Setsalistof npmpackagesinstalledasdevelopmentdependencies.

Theydifferfrom dependenciesbecausetheyaremeanttobeinstalledonlyonadevelopmentmachine,notneededtorunthecodeinproduction.

Whenyouinstallapackageusingnpmoryarn:

npminstall--dev<PACKAGENAME>

yarnadd--dev<PACKAGENAME>

thatpackageisautomaticallyinsertedinthislist.

Example:

"devDependencies":{

"autoprefixer":"^7.1.2",

"babel-core":"^6.22.1"

}

engines

SetswhichversionsofNode.jsandothercommandsthispackage/appworkon

Example:

"engines":{

Thepackage.jsonfile

61

"node":">=6.0.0",

"npm":">=3.0.0",

"yarn":"^0.13.0"

}

browserslist

Isusedtotellwhichbrowsers(andtheirversions)youwanttosupport.It'sreferencedbyBabel,Autoprefixer,andothertools,toonlyaddthepolyfillsandfallbacksneededtothebrowsersyoutarget.

Example:

"browserslist":[

">1%",

"last2versions",

"notie<=8"

]

Thisconfigurationmeansyouwanttosupportthelast2majorversionsofallbrowserswithatleast1%ofusage(fromtheCanIUse.comstats),exceptIE8andlower.

(seemore)

Command-specificproperties

The package.jsonfilecanalsohostcommand-specificconfiguration,forexampleforBabel,ESLint,andmore.

Eachhasaspecificproperty,like eslintConfig, babelandothers.Thosearecommand-specific,andyoucanfindhowtousethoseintherespectivecommand/projectdocumentation.

PackageversionsYouhaveseeninthedescriptionaboveversionnumberslikethese: ~3.0.0or 0.13.0.Whatdotheymean,andwhichotherversionspecifierscanyouuse?

Thatsymbolspecifieswhichupdatesyoupackageaccepts,fromthatdependency.

Giventhatusingsemver(semanticversioning)allversionshave3digits,thefirstbeingthemajorrelease,thesecondtheminorreleaseandthethirdisthepatchrelease,youhavetheserules:

~:ifyouwrite ~0.13.0,youwanttoonlyupdatepatchreleases: 0.13.1isok,but

Thepackage.jsonfile

62

0.14.0isnot. :ifyouwrite 0.13.0,youwanttoupdatepatchandminorreleases: 0.13.1, 0.14.0andsoon.*:ifyouwrite *,thatmeansyouacceptallupdates,includingmajorversionupgrades.>:youacceptanyversionhigherthantheoneyouspecify>=:youacceptanyversionequaltoorhigherthantheoneyouspecify<=:youacceptanyversionequalorlowertotheoneyouspecify<:youacceptanyversionlowertotheoneyouspecify

Thereareotherrules,too:

nosymbol:youacceptonlythatspecificversionyouspecifylatest:youwanttousethelatestversionavailable

andyoucancombinemostoftheaboveinranges,likethis: 1.0.0||>=1.1.0<1.2.0,toeitheruse1.0.0oronereleasefrom1.1.0up,butlowerthan1.2.0.

Thepackage.jsonfile

63

Thepackage-lock.jsonfileThepackage-lock.jsonfileisautomaticallygeneratedwheninstallingnodepackages.Learnwhatit'sabout

Inversion5,npmintroducedthe package-lock.jsonfile.

What'sthat?Youprobablyknowaboutthe package.jsonfile,whichismuchmorecommonandhasbeenaroundformuchlonger.

Thegoalofthefileistokeeptrackoftheexactversionofeverypackagethatisinstalledsothataproductis100%reproducibleinthesamewayevenifpackagesareupdatedbytheirmaintainers.

Thissolvesaveryspecificproblemthat package.jsonleftunsolved.Inpackage.jsonyoucansetwhichversionsyouwanttoupgradeto(patchorminor),usingthesemvernotation,forexample:

ifyouwrite ~0.13.0,youwanttoonlyupdatepatchreleases: 0.13.1isok,but 0.14.0isnot.ifyouwrite 0.13.0,youwanttoupdatepatchandminorreleases: 0.13.1, 0.14.0andsoon.ifyouwrite 0.13.0,thatistheexactversionthatwillbeused,always

Youdon'tcommittoGityournode_modulesfolder,whichisgenerallyhuge,andwhenyoutrytoreplicatetheprojectonanothermachinebyusingthe npminstallcommand,ifyouspecifiedthe ~syntaxandapatchreleaseofapackagehasbeenreleased,thatoneisgoingtobeinstalled.Samefor andminorreleases.

Ifyouspecifyexactversions,like 0.13.0intheexample,youarenotaffectedbythisproblem.

Itcouldbeyou,oranotherpersontryingtoinitializetheprojectontheothersideoftheworldbyrunning npminstall.

Soyouroriginalprojectandthenewlyinitializedprojectareactuallydifferent.Evenifapatchorminorreleaseshouldnotintroducebreakingchanges,weallknowbugscan(andso,theywill)slidein.

The package-lock.jsonsetsyourcurrentlyinstalledversionofeachpackageinstone,andnpmwillusethoseexactversionswhenrunning npminstall.

Thisconceptisnotnew,andotherprogramminglanguagespackagemanagers(likeComposerinPHP)useasimilarsystemforyears.

Thepackage-lock.jsonfile

64

The package-lock.jsonfileneedstobecommittedtoyourGitrepository,soitcanbefetchedbyotherpeople,iftheprojectispublicoryouhavecollaborators,orifyouuseGitasasourcefordeployments.

Thedependenciesversionswillbeupdatedinthe package-lock.jsonfilewhenyourun npmupdate.

AnexampleThisisanexamplestructureofa package-lock.jsonfilewegetwhenwerun npminstallcowsayinanemptyfolder:

{

"requires":true,

"lockfileVersion":1,

"dependencies":{

"ansi-regex":{

"version":"3.0.0",

"resolved":"https://registry.npmjs.org/ansi-regex/-/ansi-regex-3.

0.0.tgz",

"integrity":"sha1-7QMXwyIGT3lGbAKWa922Bas32Zg="

},

"cowsay":{

"version":"1.3.1",

"resolved":"https://registry.npmjs.org/cowsay/-/cowsay-1.3.1.tgz"

,

"integrity":"sha512-3PVFe6FePVtPj1HTeLin9v8WyLl+VmM1l1H/5P+BTTDkM

Ajufp+0F9eLjzRnOHzVAYeIYFF5po5NjRrgefnRMQ==",

"requires":{

"get-stdin":"^5.0.1",

"optimist":"~0.6.1",

"string-width":"~2.1.1",

"strip-eof":"^1.0.0"

}

},

"get-stdin":{

"version":"5.0.1",

"resolved":"https://registry.npmjs.org/get-stdin/-/get-stdin-5.0.

1.tgz",

"integrity":"sha1-Ei4WFZHiH/TFJTAwVpPyDmOTo5g="

},

"is-fullwidth-code-point":{

"version":"2.0.0",

"resolved":"https://registry.npmjs.org/is-fullwidth-code-point/-/

is-fullwidth-code-point-2.0.0.tgz",

"integrity":"sha1-o7MKXE8ZkYMWeqq5O+764937ZU8="

},

"minimist":{

"version":"0.0.10",

"resolved":"https://registry.npmjs.org/minimist/-/minimist-0.0.10

.tgz",

Thepackage-lock.jsonfile

65

"integrity":"sha1-3j+YVD2/lggr5IrRoMfNqDYwHc8="

},

"optimist":{

"version":"0.6.1",

"resolved":"https://registry.npmjs.org/optimist/-/optimist-0.6.1.tgz",

"integrity":"sha1-2j6nRob6IaGaERwybpDrFaAZZoY=",

"requires":{

"minimist":"~0.0.1",

"wordwrap":"~0.0.2"

}

},

"string-width":{

"version":"2.1.1",

"resolved":"https://registry.npmjs.org/string-width/-/string-width-2.1.1.tgz",

"integrity":"sha512-nOqH59deCq9SRHlxq1Aw85Jnt4w6KvLKqWVik6oA9ZklXLNIOlqg4F2yrT1MVaT

jAqvVwdfeZ7w7aCvJD7ugkw==",

"requires":{

"is-fullwidth-code-point":"^2.0.0",

"strip-ansi":"^4.0.0"

}

},

"strip-ansi":{

"version":"4.0.0",

"resolved":"https://registry.npmjs.org/strip-ansi/-/strip-ansi-4.0.0.tgz",

"integrity":"sha1-qEeQIusaw2iocTibY1JixQXuNo8=",

"requires":{

"ansi-regex":"^3.0.0"

}

},

"strip-eof":{

"version":"1.0.0",

"resolved":"https://registry.npmjs.org/strip-eof/-/strip-eof-1.0.0.tgz",

"integrity":"sha1-u0P/VZim6wXYm1n80SnJgzE2Br8="

},

"wordwrap":{

"version":"0.0.3",

"resolved":"https://registry.npmjs.org/wordwrap/-/wordwrap-0.0.3.tgz",

"integrity":"sha1-o9XabNXAvAAI03I0u68b7WMFkQc="

}

}

}

Weinstalled cowsay,whichdependson

get-stdin

optimist

string-width

strip-eof

Inturn,thosepackagesrequireotherpackages,aswecanseefromthe requirespropertythatsomehave:

Thepackage-lock.jsonfile

66

ansi-regex

is-fullwidth-code-point

minimist

wordwrap

strip-eof

Theyareaddedinalphabeticalorderintothefile,andeachonehasa versionfield,aresolvedfieldthatpointstothepackagelocation,andan integritystringthatwecanusetoverifythepackage.

Thepackage-lock.jsonfile

67

FindtheinstalledversionofannpmpackageHowtofindoutwhichversionofaparticularpackageyouhaveinstalledinyourapp

Toseethelatestversionofallthenpmpackageinstalled,includingtheirdependencies:

npmlist

Example:

❯npmlist/Users/flavio/dev/node/cowsay

└─┬cowsay@1.3.1

├──get-stdin@5.0.1

├─┬optimist@0.6.1

│├──minimist@0.0.10

│└──wordwrap@0.0.3

├─┬string-width@2.1.1

│├──is-fullwidth-code-point@2.0.0

│└─┬strip-ansi@4.0.0

│└──ansi-regex@3.0.0

└──strip-eof@1.0.0

Youcanalsojustopenthe package-lock.jsonfile,butthisinvolvessomevisualscanning.

npmlist-gisthesame,butforgloballyinstalledpackages.

Togetonlyyourtop-levelpackages(basically,theonesyoutoldnpmtoinstallandyoulistedinthe package.json),run npmlist--depth=0:

❯npmlist--depth=0/Users/flavio/dev/node/cowsay

└──cowsay@1.3.1

Youcangettheversionofaspecificpackagebyspecifyingthename:

❯npmlistcowsay/Users/flavio/dev/node/cowsay

└──cowsay@1.3.1

Thisalsoworksfordependenciesofpackagesyouinstalled:

Findtheinstalledversionofannpmpackage

68

❯npmlistminimist/Users/flavio/dev/node/cowsay

└─┬cowsay@1.3.1

└─┬optimist@0.6.1

└──minimist@0.0.10

Ifyouwanttoseewhat'sthelatestavailableversionofthepackageonthenpmrepository,runnpmview[package_name]version:

❯npmviewcowsayversion

1.3.1

Findtheinstalledversionofannpmpackage

69

HowtoinstallanolderversionofannpmpackageLearnhowtoinstallanolderversionofannpmpackage,somethingthatmightbeusefultosolveacompatibilityproblem

Youcaninstallanoldversionofannpmpackageusingthe @syntax:

npminstall<package>@<version>

Example:

npminstallcowsay

installsversion1.3.1(atthetimeofwriting).

Installversion1.2.0with:

npminstallcowsay@1.2.0

Thesamecanbedonewithglobalpackages:

npminstall-gwebpack@4.16.4

Youmightalsobeinterestedinlistingallthepreviousversionofapackage.Youcandoitwithnpmview<package>versions:

❯npmviewcowsayversions

['1.0.0',

'1.0.1',

'1.0.2',

'1.0.3',

'1.1.0',

'1.1.1',

'1.1.2',

'1.1.3',

'1.1.4',

'1.1.5',

'1.1.6',

'1.1.7',

'1.1.8',

'1.1.9',

'1.2.0',

'1.2.1',

Howtoinstallanolderversionofannpmpackage

70

'1.3.0',

'1.3.1']

Howtoinstallanolderversionofannpmpackage

71

HowtoupdatealltheNodedependenciestotheirlatestversionHowdoyouupdateallthenpmdependenciesstoreinthepackage.jsonfile,totheirlatestversionavailable?

Whenyouinstallapackageusing npminstall<packagename>,thelatestavailableversionofthepackageisdownloadedandputinthe node_modulesfolder,andacorrespondingentryisaddedtothe package.jsonand package-lock.jsonfilesthatarepresentinyourcurrentfolder.

npmcalculatesthedependenciesandinstallsthelatestavailableversionofthoseaswell.

Let'ssayyouinstall cowsay,acoolcommandlinetoolthatletsyoumakeacowsaythings.

Whenyou npminstallcowsay,thisentryisaddedtothe package.jsonfile:

{

"dependencies":{

"cowsay":"^1.3.1"

}

}

andthisisanextractof package-lock.json,whereIremovedthenesteddependenciesforclarity:

{

"requires":true,

"lockfileVersion":1,

"dependencies":{

"cowsay":{

"version":"1.3.1",

"resolved":"https://registry.npmjs.org/cowsay/-/cowsay-1.3.1.tgz",

"integrity":"sha512-3PVFe6FePVtPj1HTeLin9v8WyLl+VmM1l1H/5P+BTTDkMAjufp+0F9eLjzRnOHz

VAYeIYFF5po5NjRrgefnRMQ==",

"requires":{

"get-stdin":"^5.0.1",

"optimist":"~0.6.1",

"string-width":"~2.1.1",

"strip-eof":"^1.0.0"

}

}

}

}

HowtoupdatealltheNodedependenciestotheirlatestversion

72

Nowthose2filestellusthatweinstalledversion 1.3.1ofcowsay,andourruleforupdatesis 1.3.1,whichforthenpmversioningrulesmeansthatnpmcanupdatetopatchandminorreleases: 0.13.1, 0.14.0andsoon.

Ifthereisanewminororpatchreleaseandwetype npmupdate,theinstalledversionisupdated,andthe package-lock.jsonfilediligentlyfilledwiththenewversion.

package.jsonremainsunchanged.

Todiscovernewreleasesofthepackages,yourun npmoutdated.

Here'sthelistofafewoutdatedpackagesinonerepositoryIdidn'tupdateforquiteawhile:

Someofthoseupdatesaremajorreleases.Running npmupdatewon'tupdatetheversionofthose.Majorreleasesareneverupdatedinthiswaybecausethey(bydefinition)introducebreakingchanges,and npmwanttosaveyoutrouble.

Toupdatetoanewmajorversionallthepackages,installthe npm-check-updatespackageglobally:

npminstall-gnpm-check-updates

thenrunit:

ncu-u

HowtoupdatealltheNodedependenciestotheirlatestversion

73

thiswillupgradealltheversionhintsinthe package.jsonfile,to dependenciesanddevDependencies,sonpmcaninstallthenewmajorversion.

Youarenowreadytoruntheupdate:

npmupdate

Ifyoujustdownloadedtheprojectwithoutthe node_modulesdependenciesandyouwanttoinstalltheshinynewversionsfirst,justrun

npminstall

HowtoupdatealltheNodedependenciestotheirlatestversion

74

SemanticversioningrulesSemanticVersioningisaconventionusedtoprovideameaningtoversions

Ifthere'sonegreatthinginNode.jspackages,isthatallagreedonusingSemanticVersioningfortheirversionnumbering.

TheSemanticVersioningconceptissimple:allversionshave3digits: x.y.z.

thefirstdigitisthemajorversiontheseconddigitistheminorversionthethirddigitisthepatchversion

Whenyoumakeanewrelease,youdon'tjustupanumberasyouplease,butyouhaverules:

youupthemajorversionwhenyoumakeincompatibleAPIchangesyouuptheminorversionwhenyouaddfunctionalityinabackward-compatiblemanneryouupthepatchversionwhenyoumakebackward-compatiblebugfixes

Theconventionisadoptedallacrossprogramminglanguages,anditisveryimportantthatevery npmpackageadherestoit,becausethewholesystemdependsonthat.

Whyisthatsoimportant?

Because npmsetsomeruleswecanuseinthe package.jsonfiletochoosewhichversionsitcanupdateourpackagesto,whenwerun npmupdate.

Therulesusethosesymbols:

~

>

>=

<

<=

=

-

||

Let'sseethoserulesindetail:

:ifyouwrite 0.13.0whenrunning npmupdateitcanupdatetopatchandminorreleases: 0.13.1, 0.14.0andsoon.~:ifyouwrite ~0.13.0,whenrunning npmupdateitcanupdatetopatchreleases:0.13.1isok,but 0.14.0isnot.

Semanticversioningrules

75

>:youacceptanyversionhigherthantheoneyouspecify>=:youacceptanyversionequaltoorhigherthantheoneyouspecify<=:youacceptanyversionequalorlowertotheoneyouspecify<:youacceptanyversionlowertotheoneyouspecify=:youacceptthatexactversion-:youacceptarangeofversions.Example: 2.1.0-2.6.2||:youcombinesets.Example: <2.1||>2.6

Youcancombinesomeofthosenotations,forexampleuse 1.0.0||>=1.1.0<1.2.0toeitheruse1.0.0oronereleasefrom1.1.0up,butlowerthan1.2.0.

Thereareotherrules,too:

nosymbol:youacceptonlythatspecificversionyouspecify( 1.2.1)latest:youwanttousethelatestversionavailable

Semanticversioningrules

76

UninstallingnpmpackagesHowtouninstallannpmNodepackage,locallyorglobally

Touninstallapackageyouhavepreviouslyinstalledlocally(using npminstall<package-name>inthe node_modulesfolder,run

npmuninstall<package-name>

fromtheprojectrootfolder(thefolderthatcontainsthenode_modulesfolder).

Usingthe -Sflag,or --save,thisoperationwillalsoremovethereferenceinthepackage.jsonfile.

Ifthepackagewasadevelopmentdependency,listedinthedevDependenciesofthepackage.jsonfile,youmustusethe -D/ --save-devflagtoremoveitfromthefile:

npmuninstall-S<package-name>

npmuninstall-D<package-name>

Ifthepackageisinstalledglobally,youneedtoaddthe -g/ --globalflag:

npmuninstall-g<package-name>

forexample:

npmuninstall-gwebpack

andyoucanrunthiscommandfromanywhereyouwantonyoursystembecausethefolderwhereyoucurrentlyaredoesnotmatter.

Uninstallingnpmpackages

77

GlobalorlocalpackagesWhenisapackagebestinstalledglobally?Why?

Themaindifferencebetweenlocalandglobalpackagesisthis:

localpackagesareinstalledinthedirectorywhereyourun npminstall<package-name>,andtheyareputinthe node_modulesfolderunderthisdirectoryglobalpackagesareallputinasingleplaceinyoursystem(exactlywheredependsonyoursetup),regardlessofwhereyourun npminstall-g<package-name>

Inyourcode,theyarebothrequiredinthesameway:

require('package-name')

sowhenshouldyouinstallinonewayoranother?

Ingeneral,allpackagesshouldbeinstalledlocally.

Thismakessureyoucanhavedozensofapplicationsinyourcomputer,allrunningadifferentversionofeachpackageifneeded.

Updatingaglobalpackagewouldmakeallyourprojectsusethenewrelease,andasyoucanimaginethismightcausenightmaresintermsofmaintenance,assomepackagesmightbreakcompatibilitywithfurtherdependencies,andsoon.

Allprojectshavetheirownlocalversionofapackage,evenifthismightappearlikeawasteofresources,it'sminimalcomparedtothepossiblenegativeconsequences.

Apackageshouldbeinstalledgloballywhenitprovidesanexecutablecommandthatyourunfromtheshell(CLI),andit'sreusedacrossprojects.

Youcanalsoinstallexecutablecommandslocallyandrunthemusingnpx,butsomepackagesarejustbetterinstalledglobally.

Greatexamplesofpopularglobalpackageswhichyoumightknoware

npm

create-react-app

vue-cli

grunt-cli

mocha

react-native-cli

gatsby-cli

Globalorlocalpackages

78

forever

nodemon

Youprobablyhavesomepackagesinstalledgloballyalreadyonyoursystem.Youcanseethembyrunning

npmlist-g--depth0

onyourcommandline.

Globalorlocalpackages

79

npmdependenciesanddevDependenciesWhenisapackageadependency,andwhenisitadevdependency?

Whenyouinstallannpmpackageusing npminstall<package-name>,youareinstallingitasadependency.

Thepackageisautomaticallylistedinthepackage.jsonfile,underthe dependencieslist(asofnpm5:beforeyouhadtomanuallyspecify --save).

Whenyouaddthe -Dflag,or --save-dev,youareinstallingitasadevelopmentdependency,whichaddsittothe devDependencieslist.

Developmentdependenciesareintendedasdevelopment-onlypackages,thatareunneededinproduction.Forexampletestingpackages,webpackorBabel.

Whenyougoinproduction,ifyoutype npminstallandthefoldercontainsa package.jsonfile,theyareinstalled,asnpmassumesthisisadevelopmentdeploy.

Youneedtosetthe --productionflag( npminstall--production)toavoidinstallingthosedevelopmentdependencies.

npmdependenciesanddevDependencies

80

npxnpxisaverycoolwaytorunNodecode,andprovidesmanyusefulfeatures

Inthispost,Iwanttointroduceaverypowerfulcommandthat'sbeenavailableinnpmstartingversion5.2,releasedinJuly2017:npx.

Ifyoudon'twanttoinstallnpm,youcaninstallnpxasastandalonepackage

npxletsyouruncodebuiltwithNodeandpublishedthroughthenpmregistry.

EasilyrunlocalcommandsNodedevelopersusedtopublishmostoftheexecutablecommandsasglobalpackages,inorderforthemtobeinthepathandexecutableimmediately.

Thiswasapainbecauseyoucouldnotreallyinstalldifferentversionsofthesamecommand.

Running npxcommandnameautomaticallyfindsthecorrectreferenceofthecommandinsidethenode_modulesfolderofaproject,withoutneedingtoknowtheexactpath,andwithoutrequiringthepackagetobeinstalledgloballyandintheuser'spath.

Installation-lesscommandexecutionThereisanothergreatfeatureof npm,whichisallowingtoruncommandswithoutfirstinstallingthem.

Thisisprettyuseful,mostlybecause:

1)youdon'tneedtoinstallanything2)youcanrundifferentversionsofthesamecommand,usingthesyntax@version

Atypicaldemonstrationofusing npxisthroughthe cowsaycommand. cowsaywillprintacowsayingwhatyouwroteinthecommand.Forexample:

cowsay"Hello"willprint

_______

<Hello>

-------

\^__^

\(oo)\_______

(__)\)\/\

||----w|

npx

81

||||

Now,thisifyouhavethe cowsaycommandgloballyinstalledfromnpmpreviously,otherwiseyou'llgetanerrorwhenyoutrytorunthecommand.

npxallowsyoutorunthatnpmcommandwithouthavingitinstalledlocally:

npxcowsay"Hello"

willdothejob.

Now,thisisafunnyuselesscommand.Otherscenariosinclude:

runningthe vueCLItooltocreatenewapplicationsandrunthem: npxvuecreatemy-vue-app

creatinganewReactappusing create-react-app: npxcreate-react-appmy-react-app

andmanymore.

Oncedownloaded,thedownloadedcodewillbewiped.

RunsomecodeusingadifferentNodeversionUsethe @tospecifytheversion,andcombinethatwiththe nodenpmpackage:

npxnode@6-v#v6.14.3

npxnode@8-v#v8.11.3

Thishelpstoavoidtoolslike nvmortheotherNodeversionmanagementtools.

RunarbitrarycodesnippetsdirectlyfromaURLnpxdoesnotlimityoutothepackagespublishedonthenpmregistry.

YoucanruncodethatsitsinaGitHubgist,forexample:

npxhttps://gist.github.com/zkat/4bc19503fe9e9309e2bfaa2c58074d32

Ofcourse,youneedtobecarefulwhenrunningcodethatyoudonotcontrol,aswithgreatpowercomesgreatresponsibility.

npx

82

npx

83

TheeventloopTheEventLoopisoneofthemostimportantaspectstounderstandaboutJavaScript.Thispostexplainsitinsimpleterms

IntroductionBlockingtheeventloopThecallstackAsimpleeventloopexplanationQueuingfunctionexecutionTheMessageQueueES6JobQueue

IntroductionTheEventLoopisoneofthemostimportantaspectstounderstandaboutJavaScript.

I'veprogrammedforyearswithJavaScript,yetI'veneverfullyunderstoodhowthingsworkunderthehoods.It'scompletelyfinetonotknowthisconceptindetail,butasusual,it'shelpfultoknowhowitworks,andalsoyoumightjustbealittlecuriousatthispoint.

ThispostaimstoexplaintheinnerdetailsofhowJavaScriptworkswithasinglethread,andhowithandlesasynchronousfunctions.

YourJavaScriptcoderunssinglethreaded.Thereisjustonethinghappeningatatime.

Thisisalimitationthat'sactuallyveryhelpful,asitsimplifiesalothowyouprogramwithoutworryingaboutconcurrencyissues.

Youjustneedtopayattentiontohowyouwriteyourcodeandavoidanythingthatcouldblockthethread,likesynchronousnetworkcallsorinfiniteloops.

Ingeneral,inmostbrowsersthereisaneventloopforeverybrowsertab,tomakeeveryprocessisolatedandavoidawebpagewithinfiniteloopsorheavyprocessingtoblockyourentirebrowser.

Theenvironmentmanagesmultipleconcurrenteventloops,tohandleAPIcallsforexample.WebWorkersrunintheirowneventloopaswell.

Youmainlyneedtobeconcernedthatyourcodewillrunonasingleeventloop,andwritecodewiththisthinginmindtoavoidblockingit.

Theeventloop

84

BlockingtheeventloopAnyJavaScriptcodethattakestoolongtoreturnbackcontroltotheeventloopwillblocktheexecutionofanyJavaScriptcodeinthepage,evenblocktheUIthread,andtheusercannotclickaround,scrollthepage,andsoon.

AlmostalltheI/OprimitivesinJavaScriptarenon-blocking.Networkrequests,Node.jsfilesystemoperations,andsoon.Beingblockingistheexception,andthisiswhyJavaScriptisbasedsomuchoncallbacks,andmorerecentlyonpromisesandasync/await.

ThecallstackThecallstackisaLIFOqueue(LastIn,FirstOut).

Theeventloopcontinuouslychecksthecallstacktoseeifthere'sanyfunctionthatneedstorun.

Whiledoingso,itaddsanyfunctioncallitfindstothecallstackandexecuteseachoneinorder.

Youknowtheerrorstacktraceyoumightbefamiliarwith,inthedebuggerorinthebrowserconsole?Thebrowserlooksupthefunctionnamesinthecallstacktoinformyouwhichfunctionoriginatesthecurrentcall:

Theeventloop

85

AsimpleeventloopexplanationLet'spickanexample:

constbar=()=>console.log('bar')

constbaz=()=>console.log('baz')

constfoo=()=>{

console.log('foo')

bar()

baz()

}

foo()

Theeventloop

86

Thiscodeprints

foo

bar

baz

asexpected.

Whenthiscoderuns,first foo()iscalled.Inside foo()wefirstcall bar(),thenwecallbaz().

Atthispointthecallstacklookslikethis:

Theeventlooponeveryiterationlooksifthere'ssomethinginthecallstack,andexecutesit:

Theeventloop

87

untilthecallstackisempty.

QueuingfunctionexecutionTheaboveexamplelooksnormal,there'snothingspecialaboutit:JavaScriptfindsthingstoexecute,runstheminorder.

Theeventloop

88

Let'sseehowtodeferafunctionuntilthestackisclear.

Theusecaseof setTimeout(()=>{}),0)istocallafunction,butexecuteitonceeveryotherfunctioninthecodehasexecuted.

Takethisexample:

constbar=()=>console.log('bar')

constbaz=()=>console.log('baz')

constfoo=()=>{

console.log('foo')

setTimeout(bar,0)

baz()

}

foo()

Thiscodeprints,maybesurprisingly:

foo

baz

bar

Whenthiscoderuns,firstfoo()iscalled.Insidefoo()wefirstcallsetTimeout,passing barasanargument,andweinstructittorunimmediatelyasfastasitcan,passing0asthetimer.Thenwecallbaz().

Atthispointthecallstacklookslikethis:

Theeventloop

89

Hereistheexecutionorderforallthefunctionsinourprogram:

Theeventloop

90

Whyisthishappening?

TheMessageQueueWhensetTimeout()iscalled,theBrowserorNode.jsstartthetimer.Oncethetimerexpires,inthiscaseimmediatelyasweput0asthetimeout,thecallbackfunctionisputintheMessageQueue.

Theeventloop

91

TheMessageQueueisalsowhereuser-initiatedeventslikeclickorkeyboardevents,orfetchresponsesarequeuedbeforeyourcodehastheopportunitytoreacttothem.OralsoDOMeventslike onLoad.

Theloopgivesprioritytothecallstack,anditfirstprocesseseverythingitfindsinthecallstack,andoncethere'snothinginthere,itgoestopickupthingsintheeventqueue.

Wedon'thavetowaitforfunctionslike setTimeout,fetchorotherthingstodotheirownwork,becausetheyareprovidedbythebrowser,andtheyliveontheirownthreads.Forexample,ifyousetthe setTimeouttimeoutto2seconds,youdon'thavetowait2seconds-thewaithappenselsewhere.

ES6JobQueueECMAScript2015introducedtheconceptoftheJobQueue,whichisusedbyPromises(alsointroducedinES6/ES2015).It'sawaytoexecutetheresultofanasyncfunctionassoonaspossible,ratherthanbeingputattheendofthecallstack.

Promisesthatresolvebeforethecurrentfunctionendswillbeexecutedrightafterthecurrentfunction.

Ifindnicetheanalogyofarollercoasterrideatanamusementpark:themessagequeueputsyoubackinqueuewithafteralltheotherpeopleinthequeue,whilethejobqueueisthefastpassticketthatletsyoutakeanotherriderightafteryoufinishedthepreviousone.

Example:

constbar=()=>console.log('bar')

constbaz=()=>console.log('baz')

constfoo=()=>{

console.log('foo')

setTimeout(bar,0)

newPromise((resolve,reject)=>

resolve('shouldberightafterbaz,beforebar')

).then(resolve=>console.log(resolve))

baz()

}

foo()

Thisprints

foo

Theeventloop

92

baz

shouldberightafterbaz,beforebar

bar

That'sabigdifferencebetweenPromises(andAsync/await,whichisbuiltonpromises)andplainoldasynchronousfunctionsthrough setTimeout()orotherplatformAPIs.

Theeventloop

93

nextTickTheNode.jsprocess.nextTickfunctioninteractswiththeeventloopinaspecialway

AsyoutrytounderstandtheNode.jseventloop,oneimportantpartofitisprocess.nextTick().

Everytimetheeventlooptakesafulltrip,wecallitatick.

Whenwepassafunctionto process.nextTick(),weinstructtheenginetoinvokethisfunctionattheendofthecurrentoperation,beforethenexteventlooptickstarts:

process.nextTick(()=>{

//dosomething

})

Theeventloopisbusyprocessingthecurrentfunctioncode.

Whenthisoperationends,theJSenginerunsallthefunctionspassedto nextTickcallsduringthatoperation.

It'sthewaywecantelltheJSenginetoprocessafunctionasynchronously(afterthecurrentfunction),butassoonaspossible,notqueueit.

Calling setTimeout(()=>{},0)willexecutethefunctioninthenexttick,muchlaterthanwhenusing nextTick().

Use nextTick()whenyouwanttomakesurethatinthenexteventloopiterationthatcodeisalreadyexecuted.

nextTick

94

setImmediateTheNode.jssetImmediatefunctioninteractswiththeeventloopinaspecialway

Whenyouwanttoexecutesomepieceofcodeasynchronously,butassoonaspossible,oneoptionistousethe setImmediate()functionprovidedbyNode.js:

setImmediate(()=>{

//runsomething

})

AnyfunctionpassedasthesetImmediate()argumentisacallbackthat'sexecutedinthenextiterationoftheeventloop.

Howis setImmediate()differentfrom setTimeout(()=>{},0)(passinga0mstimeout),andfrom process.nextTick()?

Afunctionpassedto process.nextTick()isgoingtobeexecutedonthecurrentiterationoftheeventloop,afterthecurrentoperationends.ThismeansitwillalwaysexecutebeforesetTimeoutand setImmediate.

A setTimeout()callbackwitha0msdelayisverysimilarto setImmediate().Theexecutionorderwilldependonvariousfactors,buttheywillbebothruninthenextiterationoftheeventloop.

setImmediate

95

TimersWhenwritingJavaScriptcode,youmightwanttodelaytheexecutionofafunction.LearnhowtousesetTimeoutandsetIntervaltoschedulefunctionsinthefuture

setTimeout()

ZerodelaysetInterval()

RecursivesetTimeout

setTimeout()

WhenwritingJavaScriptcode,youmightwanttodelaytheexecutionofafunction.

Thisisthejobof setTimeout.Youspecifyacallbackfunctiontoexecutelater,andavalueexpressinghowlateryouwantittorun,inmilliseconds:

setTimeout(()=>{

//runsafter2seconds

},2000)

setTimeout(()=>{

//runsafter50milliseconds

},50)

Thissyntaxdefinesanewfunction.Youcancallwhateverotherfunctionyouwantinthere,oryoucanpassanexistingfunctionname,andasetofparameters:

constmyFunction=(firstParam,secondParam)=>{

//dosomething

}

Timers

96

//runsafter2seconds

setTimeout(myFunction,2000,firstParam,secondParam)

setTimeoutreturnsthetimerid.Thisisgenerallynotused,butyoucanstorethisid,andclearitifyouwanttodeletethisscheduledfunctionexecution:

constid=setTimeout(()=>{

//shouldrunafter2seconds

},2000)

//Ichangedmymind

clearTimeout(id)

Zerodelay

Ifyouspecifythetimeoutdelayto 0,thecallbackfunctionwillbeexecutedassoonaspossible,butafterthecurrentfunctionexecution:

setTimeout(()=>{

console.log('after')

},0)

console.log('before')

willprint beforeafter.

ThisisespeciallyusefultoavoidblockingtheCPUonintensivetasksandletotherfunctionsbeexecutedwhileperformingaheavycalculation,byqueuingfunctionsinthescheduler.

Somebrowsers(IEandEdge)implementa setImmediate()methodthatdoesthissameexactfunctionality,butit'snotstandardandunavailableonotherbrowsers.Butit'sastandardfunctioninNode.js.

setInterval()

setIntervalisafunctionsimilarto setTimeout,withadifference:insteadofrunningthecallbackfunctiononce,itwillrunitforever,atthespecifictimeintervalyouspecify(inmilliseconds):

setInterval(()=>{

//runsevery2seconds

},2000)

Timers

97

Thefunctionaboverunsevery2secondsunlessyoutellittostop,using clearInterval,passingittheintervalidthat setIntervalreturned:

constid=setInterval(()=>{

//runsevery2seconds

},2000)

clearInterval(id)

It'scommontocall clearIntervalinsidethesetIntervalcallbackfunction,toletitauto-determineifitshouldrunagainorstop.ForexamplethiscoderunssomethingunlessApp.somethingIWaithasthevalue arrived:

constinterval=setInterval(()=>{

if(App.somethingIWait==='arrived'){

clearInterval(interval)

return

}

//otherwisedothings

},100)

RecursivesetTimeoutsetIntervalstartsafunctioneverynmilliseconds,withoutanyconsiderationaboutwhenafunctionfinisheditsexecution.

Ifafunctiontakesalwaysthesameamountoftime,it'sallfine:

Maybethefunctiontakesdifferentexecutiontimes,dependingonnetworkconditionsforexample:

Andmaybeonelongexecutionoverlapsthenextone:

Timers

98

Toavoidthis,youcanschedulearecursivesetTimeouttobecalledwhenthecallbackfunctionfinishes:

constmyFunction=()=>{

//dosomething

setTimeout(myFunction,1000)

}

setTimeout(

myFunction()

},1000)

toachievethisscenario:

setTimeoutand setIntervalareavailableinNode.js,throughtheTimersmodule.

Node.jsalsoprovides setImmediate(),whichisequivalenttousing setTimeout(()=>{},0),mostlyusedtoworkwiththeNode.jsEventLoop.

Timers

99

CallbacksJavaScriptissynchronousbydefault,andissinglethreaded.Thismeansthatcodecannotcreatenewthreadsandruninparallel.Findoutwhatasynchronouscodemeansandhowitlookslike

AsynchronicityinProgrammingLanguagesJavaScriptCallbacksHandlingerrorsincallbacksTheproblemwithcallbacksAlternativestocallbacks

AsynchronicityinProgrammingLanguagesComputersareasynchronousbydesign.

Asynchronousmeansthatthingscanhappenindependentlyofthemainprogramflow.

Callbacks

100

Inthecurrentconsumercomputers,everyprogramrunsforaspecifictimeslot,andthenitstopsitsexecutiontoletanotherprogramcontinueitsexecution.Thisthingrunsinacyclesofastthat'simpossibletonotice,andwethinkourcomputersrunmanyprogramssimultaneously,butthisisanillusion(exceptonmultiprocessormachines).

Programsinternallyuseinterrupts,asignalthat'semittedtotheprocessortogaintheattentionofthesystem.

Iwon'tgointotheinternalsofthis,butjustkeepinmindthatit'snormalforprogramstobeasynchronous,andhalttheirexecutionuntiltheyneedattention,andthecomputercanexecuteotherthingsinthemeantime.Whenaprogramiswaitingforaresponsefromthenetwork,itcannothalttheprocessoruntiltherequestfinishes.

Normally,programminglanguagesaresynchronous,andsomeprovideawaytomanageasynchronicity,inthelanguageorthroughlibraries.C,Java,C#,PHP,Go,Ruby,Swift,Python,theyareallsynchronousbydefault.Someofthemhandleasyncbyusingthreads,spawninganewprocess.

JavaScriptJavaScriptissynchronousbydefaultandissinglethreaded.Thismeansthatcodecannotcreatenewthreadsandruninparallel.

Linesofcodeareexecutedinseries,oneafteranother,forexample:

consta=1

constb=2

constc=a*b

console.log(c)

doSomething()

ButJavaScriptwasborninsidethebrowser,itsmainjob,inthebeginning,wastorespondtouseractions,like onClick, onMouseOver, onChange, onSubmitandsoon.Howcoulditdothiswithasynchronousprogrammingmodel?

Theanswerwasinitsenvironment.ThebrowserprovidesawaytodoitbyprovidingasetofAPIsthatcanhandlethiskindoffunctionality.

Morerecently,Node.jsintroducedanon-blockingI/Oenvironmenttoextendthisconcepttofileaccess,networkcallsandsoon.

Callbacks

Callbacks

101

Youcan'tknowwhenauserisgoingtoclickabutton,sowhatyoudois,youdefineaneventhandlerfortheclickevent.Thiseventhandleracceptsafunction,whichwillbecalledwhentheeventistriggered:

document.getElementById('button').addEventListener('click',()=>{

//itemclicked

})

Thisistheso-calledcallback.

Acallbackisasimplefunctionthat'spassedasavaluetoanotherfunction,andwillonlybeexecutedwhentheeventhappens.WecandothisbecauseJavaScripthasfirst-classfunctions,whichcanbeassignedtovariablesandpassedaroundtootherfunctions(calledhigher-orderfunctions)

It'scommontowrapallyourclientcodeina loadeventlisteneronthe windowobject,whichrunsthecallbackfunctiononlywhenthepageisready:

window.addEventListener('load',()=>{

//windowloaded

//dowhatyouwant

})

Callbacksareusedeverywhere,notjustinDOMevents.

Onecommonexampleisbyusingtimers:

setTimeout(()=>{

//runsafter2seconds

},2000)

XHRrequestsalsoacceptacallback,inthisexamplebyassigningafunctiontoapropertythatwillbecalledwhenaparticulareventoccurs(inthiscase,thestateoftherequestchanges):

constxhr=newXMLHttpRequest()

xhr.onreadystatechange=()=>{

if(xhr.readyState===4){

xhr.status===200?console.log(xhr.responseText):console.error('error')

}

}

xhr.open('GET','https://yoursite.com')

xhr.send()

Handlingerrorsincallbacks

Callbacks

102

Howdoyouhandleerrorswithcallbacks?OneverycommonstrategyistousewhatNode.jsadopted:thefirstparameterinanycallbackfunctionistheerrorobject:error-firstcallbacks

Ifthereisnoerror,theobjectis null.Ifthereisanerror,itcontainssomedescriptionoftheerrorandotherinformation.

fs.readFile('/file.json',(err,data)=>{

if(err!==null){

//handleerror

console.log(err)

return

}

//noerrors,processdata

console.log(data)

})

TheproblemwithcallbacksCallbacksaregreatforsimplecases!

Howevereverycallbackaddsalevelofnesting,andwhenyouhavelotsofcallbacks,thecodestartstobecomplicatedveryquickly:

window.addEventListener('load',()=>{

document.getElementById('button').addEventListener('click',()=>{

setTimeout(()=>{

items.forEach(item=>{

//yourcodehere

})

},2000)

})

})

Thisisjustasimple4-levelscode,butI'veseenmuchmorelevelsofnestingandit'snotfun.

Howdowesolvethis?

AlternativestocallbacksStartingwithES6,JavaScriptintroducedseveralfeaturesthathelpuswithasynchronouscodethatdonotinvolveusingcallbacks:

Promises(ES6)Async/Await(ES8)

Callbacks

103

Callbacks

104

PromisesPromisesareonewaytodealwithasynchronouscodeinJavaScript,withoutwritingtoomanycallbacksinyourcode.

IntroductiontopromisesHowpromiseswork,inbriefWhichJSAPIusepromises?

CreatingapromiseConsumingapromiseChainingpromises

ExampleofchainingpromisesHandlingerrors

CascadingerrorsOrchestratingpromises

Promise.all()

Promise.race()

CommonerrorsUncaughtTypeError:undefinedisnotapromise

IntroductiontopromisesApromiseiscommonlydefinedasaproxyforavaluethatwilleventuallybecomeavailable.

Promisesareonewaytodealwithasynchronouscode,withoutwritingtoomanycallbacksinyourcode.

Althoughbeingaroundsinceyears,theyhavebeenstandardizedandintroducedinES2015,andnowtheyhavebeensupersededinES2017byasyncfunctions.

AsyncfunctionsusethepromisesAPIastheirbuildingblock,sounderstandingthemisfundamentalevenifinnewercodeyou'lllikelyuseasyncfunctionsinsteadofpromises.

Howpromiseswork,inbrief

Onceapromisehasbeencalled,itwillstartinpendingstate.Thismeansthatthecallerfunctioncontinuestheexecution,whileitwaitsforthepromisetodoitsownprocessing,andgivethecallerfunctionsomefeedback.

Promises

105

Atthispoint,thecallerfunctionwaitsforittoeitherreturnthepromiseinaresolvedstate,orinarejectedstate,butasyouknowJavaScriptisasynchronous,sothefunctioncontinuesitsexecutionwhilethepromisedoesitwork.

WhichJSAPIusepromises?

Inadditiontoyourowncodeandlibrariescode,promisesareusedbystandardmodernWebAPIssuchas:

theBatteryAPItheFetchAPIServiceWorkers

It'sunlikelythatinmodernJavaScriptyou'llfindyourselfnotusingpromises,solet'sstartdivingrightintothem.

CreatingapromiseThePromiseAPIexposesaPromiseconstructor,whichyouinitializeusing newPromise():

letdone=true

constisItDoneYet=newPromise(

(resolve,reject)=>{

if(done){

constworkDone='HereisthethingIbuilt'

resolve(workDone)

}else{

constwhy='Stillworkingonsomethingelse'

reject(why)

}

}

)

Asyoucanseethepromisechecksthe doneglobalconstant,andifthat'strue,wereturnaresolvedpromise,otherwisearejectedpromise.

Using resolveand rejectwecancommunicatebackavalue,intheabovecasewejustreturnastring,butitcouldbeanobjectaswell.

Consumingapromise

Promises

106

Inthelastsection,weintroducedhowapromiseiscreated.

Nowlet'sseehowthepromisecanbeconsumedorused.

constisItDoneYet=newPromise(

//...

)

constcheckIfItsDone=()=>{

isItDoneYet

.then((ok)=>{

console.log(ok)

})

.catch((err)=>{

console.error(err)

})

}

Running checkIfItsDone()willexecutethe isItDoneYet()promiseandwillwaitforittoresolve,usingthe thencallback,andifthereisanerror,itwillhandleitinthe catchcallback.

ChainingpromisesApromisecanbereturnedtoanotherpromise,creatingachainofpromises.

AgreatexampleofchainingpromisesisgivenbytheFetchAPI,alayerontopoftheXMLHttpRequestAPI,whichwecanusetogetaresourceandqueueachainofpromisestoexecutewhentheresourceisfetched.

TheFetchAPIisapromise-basedmechanism,andcalling fetch()isequivalenttodefiningourownpromiseusing newPromise().

Exampleofchainingpromises

conststatus=(response)=>{

if(response.status>=200&&response.status<300){

returnPromise.resolve(response)

}

returnPromise.reject(newError(response.statusText))

}

constjson=(response)=>response.json()

fetch('/todos.json')

.then(status)

Promises

107

.then(json)

.then((data)=>{console.log('RequestsucceededwithJSONresponse',data)})

.catch((error)=>{console.log('Requestfailed',error)})

Inthisexample,wecall fetch()togetalistofTODOitemsfromthe todos.jsonfilefoundinthedomainroot,andwecreateachainofpromises.

Running fetch()returnsaresponse,whichhasmanyproperties,andwithinthosewereference:

status,anumericvaluerepresentingtheHTTPstatuscodestatusText,astatusmessage,whichis OKiftherequestsucceeded

responsealsohasa json()method,whichreturnsapromisethatwillresolvewiththecontentofthebodyprocessedandtransformedintoJSON.

Sogiventhosepremises,thisiswhathappens:thefirstpromiseinthechainisafunctionthatwedefined,called status(),thatcheckstheresponsestatusandifit'snotasuccessresponse(between200and299),itrejectsthepromise.

Thisoperationwillcausethepromisechaintoskipallthechainedpromiseslistedandwillskipdirectlytothe catch()statementatthebottom,loggingthe Requestfailedtextalongwiththeerrormessage.

Ifthatsucceedsinstead,itcallsthejson()functionwedefined.Sincethepreviouspromise,whensuccessful,returnedthe responseobject,wegetitasaninputtothesecondpromise.

Inthiscase,wereturnthedataJSONprocessed,sothethirdpromisereceivestheJSONdirectly:

.then((data)=>{

console.log('RequestsucceededwithJSONresponse',data)

})

andwesimplylogittotheconsole.

HandlingerrorsIntheexample,intheprevioussection,wehada catchthatwasappendedtothechainofpromises.

Whenanythinginthechainofpromisesfailsandraisesanerrororrejectsthepromise,thecontrolgoestothenearest catch()statementdownthechain.

Promises

108

newPromise((resolve,reject)=>{

thrownewError('Error')

})

.catch((err)=>{console.error(err)})

//or

newPromise((resolve,reject)=>{

reject('Error')

})

.catch((err)=>{console.error(err)})

Cascadingerrors

Ifinsidethe catch()youraiseanerror,youcanappendasecond catch()tohandleit,andsoon.

newPromise((resolve,reject)=>{

thrownewError('Error')

})

.catch((err)=>{thrownewError('Error')})

.catch((err)=>{console.error(err)})

Orchestratingpromises

Promise.all()

Ifyouneedtosynchronizedifferentpromises, Promise.all()helpsyoudefinealistofpromises,andexecutesomethingwhentheyareallresolved.

Example:

constf1=fetch('/something.json')

constf2=fetch('/something2.json')

Promise.all([f1,f2]).then((res)=>{

console.log('Arrayofresults',res)

})

.catch((err)=>{

console.error(err)

})

TheES2015destructuringassignmentsyntaxallowsyoutoalsodo

Promise.all([f1,f2]).then(([res1,res2])=>{

Promises

109

console.log('Results',res1,res2)

})

Youarenotlimitedtousing fetchofcourse,anypromiseisgoodtogo.

Promise.race()

Promise.race()runswhenthefirstofthepromisesyoupasstoitresolves,anditrunstheattachedcallbackjustonce,withtheresultofthefirstpromiseresolved.

Example:

constfirst=newPromise((resolve,reject)=>{

setTimeout(resolve,500,'first')

})

constsecond=newPromise((resolve,reject)=>{

setTimeout(resolve,100,'second')

})

Promise.race([first,second]).then((result)=>{

console.log(result)//second

})

Commonerrors

UncaughtTypeError:undefinedisnotapromise

Ifyougetthe UncaughtTypeError:undefinedisnotapromiseerrorintheconsole,makesureyouuse newPromise()insteadofjust Promise()

Promises

110

async/awaitDiscoverthemodernapproachtoasynchronousfunctionsinJavaScript.JavaScriptevolvedinaveryshorttimefromcallbackstoPromises,andsinceES2017asynchronousJavaScriptisevensimplerwiththeasync/awaitsyntax

IntroductionWhywereasync/awaitintroduced?HowitworksAquickexamplePromiseallthethingsThecodeismuchsimplertoreadMultipleasyncfunctionsinseriesEasierdebugging

IntroductionJavaScriptevolvedinaveryshorttimefromcallbackstopromises(ES2015),andsinceES2017asynchronousJavaScriptisevensimplerwiththeasync/awaitsyntax.

Asyncfunctionsareacombinationofpromisesandgenerators,andbasically,theyareahigherlevelabstractionoverpromises.Letmerepeat:async/awaitisbuiltonpromises.

Whywereasync/awaitintroduced?Theyreducetheboilerplatearoundpromises,andthe"don'tbreakthechain"limitationofchainingpromises.

WhenPromiseswereintroducedinES2015,theyweremeanttosolveaproblemwithasynchronouscode,andtheydid,butoverthe2yearsthatseparatedES2015andES2017,itwasclearthatpromisescouldnotbethefinalsolution.

Promiseswereintroducedtosolvethefamouscallbackhellproblem,buttheyintroducedcomplexityontheirown,andsyntaxcomplexity.

Theyweregoodprimitivesaroundwhichabettersyntaxcouldbeexposedtothedevelopers,sowhenthetimewasrightwegotasyncfunctions.

Theymakethecodelooklikeit'ssynchronous,butit'sasynchronousandnon-blockingbehindthescenes.

async/await

111

HowitworksAnasyncfunctionreturnsapromise,likeinthisexample:

constdoSomethingAsync=()=>{

returnnewPromise((resolve)=>{

setTimeout(()=>resolve('Ididsomething'),3000)

})

}

Whenyouwanttocallthisfunctionyouprepend await,andthecallingcodewillstopuntilthepromiseisresolvedorrejected.Onecaveat:theclientfunctionmustbedefinedasasync.Here'sanexample:

constdoSomething=async()=>{

console.log(awaitdoSomethingAsync())

}

AquickexampleThisisasimpleexampleofasync/awaitusedtorunafunctionasynchronously:

constdoSomethingAsync=()=>{

returnnewPromise((resolve)=>{

setTimeout(()=>resolve('Ididsomething'),3000)

})

}

constdoSomething=async()=>{

console.log(awaitdoSomethingAsync())

}

console.log('Before')

doSomething()

console.log('After')

Theabovecodewillprintthefollowingtothebrowserconsole:

Before

After

Ididsomething//after3s

Promiseallthethings

async/await

112

Prependingthe asynckeywordtoanyfunctionmeansthatthefunctionwillreturnapromise.

Evenifit'snotdoingsoexplicitly,itwillinternallymakeitreturnapromise.

Thisiswhythiscodeisvalid:

constaFunction=async()=>{

return'test'

}

aFunction().then(alert)//Thiswillalert'test'

andit'sthesameas:

constaFunction=async()=>{

returnPromise.resolve('test')

}

aFunction().then(alert)//Thiswillalert'test'

ThecodeismuchsimplertoreadAsyoucanseeintheexampleabove,ourcodelooksverysimple.Compareittocodeusingplainpromises,withchainingandcallbackfunctions.

Andthisisaverysimpleexample,themajorbenefitswillarisewhenthecodeismuchmorecomplex.

Forexamplehere'showyouwouldgetaJSONresource,andparseit,usingpromises:

constgetFirstUserData=()=>{

returnfetch('/users.json')//getuserslist

.then(response=>response.json())//parseJSON

.then(users=>users[0])//pickfirstuser

.then(user=>fetch(`/users/${user.name}`))//getuserdata

.then(userResponse=>response.json())//parseJSON

}

getFirstUserData()

Andhereisthesamefunctionalityprovidedusingawait/async:

constgetFirstUserData=async()=>{

constresponse=awaitfetch('/users.json')//getuserslist

constusers=awaitresponse.json()//parseJSON

constuser=users[0]//pickfirstuser

constuserResponse=awaitfetch(`/users/${user.name}`)//getuserdata

async/await

113

constuserData=awaituser.json()//parseJSON

returnuserData

}

getFirstUserData()

MultipleasyncfunctionsinseriesAsyncfunctionscanbechainedveryeasily,andthesyntaxismuchmorereadablethanwithplainpromises:

constpromiseToDoSomething=()=>{

returnnewPromise(resolve=>{

setTimeout(()=>resolve('Ididsomething'),10000)

})

}

constwatchOverSomeoneDoingSomething=async()=>{

constsomething=awaitpromiseToDoSomething()

returnsomething+'andIwatched'

}

constwatchOverSomeoneWatchingSomeoneDoingSomething=async()=>{

constsomething=awaitwatchOverSomeoneDoingSomething()

returnsomething+'andIwatchedaswell'

}

watchOverSomeoneWatchingSomeoneDoingSomething().then((res)=>{

console.log(res)

})

Willprint:

IdidsomethingandIwatchedandIwatchedaswell

EasierdebuggingDebuggingpromisesishardbecausethedebuggerwillnotstepoverasynchronouscode.

Async/awaitmakesthisveryeasybecausetothecompilerit'sjustlikesynchronouscode.

async/await

114

TheNodeEventEmitterHowtoworkwithcustomeventsinNode

IfyouworkedwithJavaScriptinthebrowser,youknowhowmuchoftheinteractionoftheuserishandledthroughevents:mouseclicks,keyboardbuttonpresses,reactingtomousemovements,andsoon.

Onthebackendside,Nodeoffersustheoptiontobuildasimilarsystemusingthe eventsmodule.

Thismodule,inparticular,offersthe EventEmitterclass,whichwe'llusetohandleourevents.

Youinitializethatusing

consteventEmitter=require('events').EventEmitter()

Thisobjectexposes,amongmanyothers,the onand emitmethods.

emitisusedtotriggeraneventonisusedtoaddacallbackfunctionthat'sgoingtobeexecutedwhentheeventistriggered

Forexample,let'screatea startevent,andasamatterofprovidingasample,wereacttothatbyjustloggingtotheconsole:

eventEmitter.on('start',()=>{

console.log('started')

})

Whenwerun

eventEmitter.emit('start')

theeventhandlerfunctionistriggered,andwegettheconsolelog.

Youcanpassargumentstotheeventhandlerbypassingthemasadditionalargumentstoemit():

eventEmitter.on('start',(number)=>{

console.log(`started${number}`)

})

eventEmitter.emit('start',23)

TheNodeEventEmitter

115

Multiplearguments:

eventEmitter.on('start',(start,end)=>{

console.log(`startedfrom${start}to${end}`)

})

eventEmitter.emit('start',1,100)

TheEventEmitterobjectalsoexposesseveralothermethodstointeractwithevents,like

once():addaone-timelistenerremoveListener()/ off():removeaneventlistenerfromaneventremoveAllListeners():removealllistenersforanevent

Youcanreadalltheirdetailsontheeventsmodulepageathttps://nodejs.org/api/events.html

TheNodeEventEmitter

116

HTTPAdetaileddescriptionofhowtheHTTPprotocol,andtheWeb,work

HTTP(HyperTextTransferProtocol)isoneoftheapplicationprotocolsofTCP/IP,thesuiteofprotocolsthatpowerstheInternet.

Letmefixthat:it'snotoneoftheprotocols,it'sthemostsuccessfulandpopularone,byallmeans.

HTTPiswhatmakestheWorldWideWebwork,givingbrowsersalanguagetocommunicatetoremoteserversthathostwebpages.

HTTPwasfirststandardizedin1991,asaresultoftheworkthatTimBerners-LeedidatCERN,theEuropeanCenterofNuclearResearch,since1989.

Thegoalwastoallowresearcherstoeasilyexchangeandinterlinktheirpapers.Itwasmeantasawayforthescientificcommunitytoworkbetter.

BackthentheinternetmainapplicationsbasicallyconsistedinFTP(theFileTransferProtocol),EmailandUsenet(newsgroups,todayalmostabandoned).

In1993Mosaic,thefirstgraphicalwebbrowser,wasreleased,andthingsskyrocketedfromthere.

TheWebbecamethekillerappoftheInternet.

OvertimetheWebandtheecosystemaroundithavedramaticallyevolved,butthebasicsstillremain.Oneexampleofevolution:HTTPnowpowers,inadditiontowebpages,RESTAPIs,onecommonwaytoprogrammaticallyaccessaserviceovertheInternet.

HTTPgotaminorrevisionin1997withHTTP/1.1,andin2015itssuccessor,HTTP/2,wasstandardizedandit'snowbeingimplementedbythemajorWebServersusedacrosstheglobe.

TheHTTPprotocolisconsideredinsecure,justlikeanyotherprotocol(SMTP,FTP..)notservedoveranencryptedconnection.ThisiswhythereisabigpushnowadaystowardsusingHTTPS,whichisHTTPservedoverTLS.

Thatsaid,thebuildingblocksofHTTP/2andHTTPShavetheirrootsinHTTP,andinthisarticleI'llintroducehowHTTPworks.

HTMLdocuments

HTTP

117

HTTPisthewaywebbrowserslikeChrome,Firefox,Edgeandmanyothers(alsocalledclientsfromhereon)communicatewithwebservers.

ThenameHyperTextTransferProtocolderivesfromtheneedoftransferringnotjustfiles,likeinFTP-the"FileTransferProtocol",buthypertexts,whichwouldbewrittenusingHTML,andthenrepresentedgraphicallybythebrowserwithanicepresentationandinteractivelinks.

Linkswerethedrivingforcethatdroveadoption,alongwiththeeaseofcreationofnewwebpages.

HTTPiswhattransferthosehypertextfiles(andaswe'llseealsoimagesandotherfiletypes)overthenetwork.

HyperlinksInsideawebbrowser,adocumentcanpointtoanotherdocumentusinglinks.

Alinkiscomposedbyafirstpartthatdeterminestheprotocolandtheserveraddress,eitherthroughadomainnameoranIP.

ThispartisnotuniquetoHTTP,ofcourse.

Thenthere'sthedocumentpart.Anythingappendedtotheaddresspartrepresentsthedocumentpath.

Forexample,thisdocumentaddressis https://flaviocopes.com/http/:

httpsistheprotocol.flaviocopes.comisthedomainnamethatpointstomyserver/http/isthedocumentURLrelativetotheserverrootpath.

Thepathcanbenested: https://flaviocopes.com/page/privacy/andinthiscasethedocumentURLis /page/privacy.

Thewebserverisresponsibleforinterpretingtherequestand,onceanalyzed,servingthecorrectresponse.

ArequestWhat'sinarequest?

ThefirstthingistheURL,whichwe'vealreadyseenbefore.

Whenweenteranaddressandpressenterinourbrowser,underthehoodstheserversendstothecorrectIPaddressarequestlikethis:

HTTP

118

GET/a-page

where/a-pageistheURLyourequested.

ThesecondthingistheHTTPmethod(alsocalledverb).

HTTPintheearlydaysdefined3ofthem:

GET

POST

HEAD

andHTTP/1.1introduced

PUT

DELETE

OPTIONS

TRACE

We'llseethemindetailinaminute.

ThethirdthingthatcomposesarequestisasetofHTTPheaders.

Headersareasetof key:valuepairsthatareusedtocommunicatetotheserver-specificinformationthatispredefined,sotheservercanknowwhatwemean.

IdescribedthemindetailintheHTTPrequestheaderslist.

Givethatlistaquicklook.Allofthoseheadersareoptional,except Host.

HTTPmethods

GET

GETisthemostusedmethodhere.It'stheonethat'susedwhenyoutypeanURLinthebrowseraddressbar,orwhenyouclickalink.

Itaskstheservertosendtherequestedresourceasaresponse.

HEAD

HEADisjustlikeGET,buttellstheservertonotsendtheresponsebodyback.Justtheheaders.

HTTP

119

POST

TheclientusesthePOSTmethodtosenddatatotheserver.It'stypicallyusedinforms,forexample,butalsowheninteractingwithaRESTAPI.

PUT

ThePUTmethodisintendedtocreatearesourceatthatspecificURL,withtheparameterspassedintherequestbody.MainlyusedinRESTAPIs

DELETE

TheDELETEmethodiscalledagainstanURLtorequestdeletionofthatresource.MainlyusedinRESTAPIs

OPTIONS

WhenaserverreceivesanOPTIONSrequestitshouldsendbackthelistofHTTPmethodsallowedforthatspecificURL.

TRACE

Returnsbacktotheclienttherequestthathasbeenreceived.Usedfordebuggingordiagnosticpurposes.

HTTPClient/ServercommunicationHTTP,asmostoftheprotocolsthatbelongtotheTCP/IPsuite,isastatelessprotocol.

Servershavenoideawhat'sthecurrentstateoftheclient.Alltheycareaboutisthattheygetrequestandtheyneedtofulfillthem.

Anypriorrequestismeaninglessinthiscontext,andthismakesitpossibleforawebservertobeveryfast,asthere'slesstoprocess,andalsoitgivesitbandwidthtohandlealotofconcurrentrequests.

HTTPisalsoverylean,andcommunicationisveryfastintermsofoverhead.ThiscontrastswiththeprotocolsthatwerethemostusedatthetimeHTTPwasintroduced:TCPandPOP/SMTP,themailprotocols,whichinvolvelotsofhandshakingandconfirmationsonthereceivingends.

HTTP

120

Graphicalbrowsersabstractallthiscommunication,butwe'llillustrateithereforlearningpurposes.

Amessageiscomposedbyafirstline,whichstartswiththeHTTPmethod,thencontainstheresourcerelativepath,andtheprotocolversion:

GET/a-pageHTTP/1.1

Afterthat,weneedtoaddtheHTTPrequestheaders.Asmentionedabove,therearemanyheaders,buttheonlymandatoryoneis Host:

GET/a-pageHTTP/1.1

Host:flaviocopes.com

Howcanyoutestthis?Usingtelnet.Thisisacommand-linetoolthatletsusconnecttoanyserverandsenditcommands.

Openyourterminal,andtype telnetflaviocopes.com80

Thiswillopenaterminal,thattellsyou

Trying178.128.202.129...

Connectedtoflaviocopes.com.

Escapecharacteris'^]'.

YouareconnectedtotheNetlifywebserverthatpowersmyblog.Youcannowtype:

GET/axios/HTTP/1.1

Host:flaviocopes.com

andpressenteronanemptylinetofiretherequest.

Theresponsewillbe:

HTTP/1.1301MovedPermanently

Cache-Control:public,max-age=0,must-revalidate

Content-Length:46

Content-Type:text/plain

Date:Sun,29Jul201814:07:07GMT

Location:https://flaviocopes.com/axios/

Age:0

Connection:keep-alive

Server:Netlify

Redirectingtohttps://flaviocopes.com/axios/

HTTP

121

See,thisisanHTTPresponsewegotbackfromtheserver.It'sa301MovedPermanentlyrequest.SeetheHTTPstatuscodeslisttoknowmoreaboutthestatuscodes.

Itbasicallytellsustheresourcehaspermanentlymovedtoanotherlocation.

Why?Becauseweconnectedtoport80,whichisthedefaultforHTTP,butonmyserverIsetupanautomaticredirectiontoHTTPS.

Thenewlocationisspecifiedinthe LocationHTTPresponseheader.

Thereareotherheaders,alldescribedintheHTTPresponseheaderslist.

Inboththerequestandtheresponse,anemptylineseparatestherequestheaderfromtherequestbody.Therequestbodyinthiscasecontainsthestring

Redirectingtohttps://flaviocopes.com/axios/

whichis46byteslong,asspecifiedinthe Content-Lengthheader.Itisshowninthebrowserwhenyouopenthepage,whileitautomaticallyredirectsyoutothecorrectlocation.

Inthiscasewe'reusingtelnet,thelow-leveltoolthatwecanusetoconnecttoanyserver,sowecan'thaveanykindofautomaticredirect.

Let'sdothisprocessagain,nowconnectingtoport443,whichisthedefaultportoftheHTTPSprotocol.Wecan'tusetelnetbecauseoftheSSLhandshakethatmusthappen.

Let'skeepthingssimpleanduse curl,anothercommand-linetool.WecannotdirectlytypetheHTTPrequest,butwe'llseetheresponse:

curl-ihttps://flaviocopes.com/axios/

thisiswhatwe'llgetinreturn:

HTTP/1.1200OK

Cache-Control:public,max-age=0,must-revalidate

Content-Type:text/html;charset=UTF-8

Date:Sun,29Jul201814:20:45GMT

Etag:"de3153d6eacef2299964de09db154b32-ssl"

Strict-Transport-Security:max-age=31536000

Age:152

Content-Length:9797

Connection:keep-alive

Server:Netlify

<!DOCTYPEhtml>

<htmlprefix="og:http://ogp.me/ns#"lang="en">

<head>

<metacharset="utf-8">

HTTP

122

<metahttp-equiv="X-UA-Compatible"content="IE=edge">

<title>HTTPrequestsusingAxios</title>

....

Icuttheresponse,butyoucanseethattheHTMLofthepageisbeingreturnednow.

OtherresourcesAnHTTPserverwillnotjusttransferHTMLfiles,buttypicallyitwillalsoserveotherfiles:CSS,JS,SVG,PNG,JPG,lotsofdifferentfiletypes.

Thisdependsontheconfiguration.

HTTPisperfectlycapableoftransferringthosefilesaswell,andtheclientwillknowaboutthefiletype,thusinterpretthemintherightway.

Thisishowthewebworks:whenanHTMLpageisretrievedbythebrowser,it'sinterpretedandanyotherresourceitneedstodisplayproperty(CSS,JavaScript,images..)isretrievedthroughadditionalHTTPrequeststothesameserver.

HTTP

123

HowHTTPRequestsworkWhathappenswhenyoutypeanURLinthebrowser,fromstarttofinish

TheHTTPprotocolIanalyzeURLrequestsonlyThingsrelatetomacOS/LinuxDNSLookupphase

gethostbynameTCPrequesthandshakingSendingtherequest

TherequestlineTherequestheaderTherequestbody

TheresponseParsetheHTML

ThisarticledescribeshowbrowsersperformpagerequestsusingtheHTTP/1.1protocol

Ifyoueverdidaninterview,youmighthavebeenasked:"whathappenswhenyoutypesomethingintotheGooglesearchboxandpressenter".

It'soneofthemostpopularquestionsyougetasked.Peoplejustwanttoseeifyoucanexplainsomeratherbasicconceptsandifyouhaveanycluehowtheinternetactuallyworks.

Inthispost,I'llanalyzewhathappenswhenyoutypeanURLintheaddressbarofyourbrowserandpressenter.

It'saveryinterestingtopictodissectinablogpost,asittouchesmanytechnologiesIcandiveintoinseparateposts.

Thisistechthatisveryrarelychanged,andpowersonethemostcomplexandwideecosystemseverbuiltbyhumans.

TheHTTPprotocolFirst,ImentionHTTPSinparticularbecausethingsaredifferentfromanHTTPSconnection.

IanalyzeURLrequestsonly

HowHTTPRequestswork

124

ModernbrowsershavethecapabilityofknowingifthethingyouwroteintheaddressbarisanactualURLorasearchterm,andtheywillusethedefaultsearchengineifit'snotavalidURL.

IassumeyoutypeanactualURL.

WhenyouentertheURLandpressenter,thebrowserfirstbuildsthefullURL.

Ifyoujustenteredadomain,like flaviocopes.com,thebrowserbydefaultwillprependHTTP://toit,defaultingtotheHTTPprotocol.

ThingsrelatetomacOS/LinuxJustFYI.Windowsmightdosomethingsslightlydifferently.

DNSLookupphaseThebrowserstartstheDNSlookuptogettheserverIPaddress.

Thedomainnameisahandyshortcutforushumans,buttheinternetisorganizedinsuchawaythatcomputerscanlookuptheexactlocationofaserverthroughitsIPaddress,whichisasetofnumberslike 222.324.3.1(IPv4).

First,itcheckstheDNSlocalcache,toseeifthedomainhasalreadybeenresolvedrecently.

ChromehasahandyDNScachevisualizeryoucanseeatchrome://net-internals/#dns

Ifnothingisfoundthere,thebrowserusestheDNSresolver,usingthe gethostbynamePOSIXsystemcalltoretrievethehostinformation.

gethostbyname

gethostbynamefirstlooksinthelocalhostsfile,whichonmacOSorLinuxislocatedin/etc/hosts,toseeifthesystemprovidestheinformationlocally.

Ifthisdoesnotgiveanyinformationaboutthedomain,thesystemmakesarequesttotheDNSserver.

TheaddressoftheDNSserverisstoredinthesystempreferences.

Thoseare2popularDNSservers:

8.8.8.8:theGooglepublicDNSserver1.1.1.1:theCloudFlareDNSserver

MostpeopleusetheDNSserverprovidedbytheirinternetprovider.

HowHTTPRequestswork

125

ThebrowserperformstheDNSrequestusingtheUDPprotocol.

TCPandUDParetwoofthefoundationalprotocolsofcomputernetworking.Theysitatthesameconceptuallevel,butTCPisconnection-oriented,whileUDPisaconnectionlessprotocol,morelightweight,usedtosendmessageswithlittleoverhead.

HowtheUDPrequestisperformedisnotinthescopeofthistutorial

TheDNSservermighthavethedomainIPinthecache.Itnot,itwillasktherootDNSserver.That'sasystem(composedof13actualservers,distributedacrosstheplanet)thatdrivestheentireinternet.

TheDNSserverdoesnotknowtheaddressofeachandeverydomainnameontheplanet.

Whatitknowsiswherethetop-levelDNSresolversare.

Atop-leveldomainisthedomainextension: .com, .it, .pizzaandsoon.

OncetherootDNSserverreceivestherequest,itforwardstherequesttothattop-leveldomain(TLD)DNSserver.

Sayyouarelookingfor flaviocopes.com.TherootdomainDNSserverreturnstheIPofthe.comTLDserver.

NowourDNSresolverwillcachetheIPofthatTLDserver,soitdoesnothavetoasktherootDNSserveragainforit.

TheTLDDNSserverwillhavetheIPaddressesoftheauthoritativeNameServersforthedomainwearelookingfor.

How?Whenyoubuyadomain,thedomainregistrarsendstheappropriateTDLthenameservers.Whenyouupdatethenameservers(forexample,whenyouchangethehostingprovider),thisinformationwillbeautomaticallyupdatedbyyourdomainregistrar.

ThosearetheDNSserversofthehostingprovider.Theyareusuallymorethan1,toserveasbackup.

Forexample:

ns1.dreamhost.com

ns2.dreamhost.com

ns3.dreamhost.com

TheDNSresolverstartswiththefirst,andtriestoasktheIPofthedomain(withthesubdomain,too)youarelookingfor.

ThatistheultimatesourceoftruthfortheIPaddress.

HowHTTPRequestswork

126

NowthatwehavetheIPaddress,wecangooninourjourney.

TCPrequesthandshakingWiththeserverIPaddressavailable,nowthebrowsercaninitiateaTCPconnectiontothat.

ATCPconnectionrequiresabitofhandshakingbeforeitcanbefullyinitializedandyoucanstartsendingdata.

Oncetheconnectionisestablished,wecansendtherequest

SendingtherequestTherequestisaplaintextdocumentstructuredinaprecisewaydeterminedbythecommunicationprotocol.

It'scomposedof3parts:

therequestlinetherequestheadertherequestbody

Therequestline

Therequestlinesets,onasingleline:

theHTTPmethodtheresourcelocationtheprotocolversion

Example:

GET/HTTP/1.1

Therequestheader

Therequestheaderisasetof field:valuepairsthatsetcertainvalues.

Thereare2mandatoryfields,oneofwhichis Host,andtheotheris Connection,whilealltheotherfieldsareoptional:

Host:flaviocopes.com

Connection:close

HowHTTPRequestswork

127

Hostindicatesthedomainnamewhichwewanttotarget,while Connectionisalwayssettocloseunlesstheconnectionmustbekeptopen.

Someofthemostusedheaderfieldsare:

Origin

Accept

Accept-Encoding

Cookie

Cache-Control

Dnt

butmanymoreexist.

Theheaderpartisterminatedbyablankline.

Therequestbody

Therequestbodyisoptional,notusedinGETrequestsbutverymuchusedinPOSTrequestsandsometimesinotherverbstoo,anditcancontaindatainJSONformat.

Sincewe'renowanalyzingaGETrequest,thebodyisblankandwe'llnotlookmoreintoit.

TheresponseOncetherequestissent,theserverprocessesitandsendsbackaresponse.

Theresponsestartswiththestatuscodeandthestatusmessage.Iftherequestissuccessfulandreturnsa200,itwillstartwith:

200OK

Therequestmightreturnadifferentstatuscodeandmessage,likeoneofthese:

404NotFound

403Forbidden

301MovedPermanently

500InternalServerError

304NotModified

401Unauthorized

TheresponsethencontainsalistofHTTPheadersandtheresponsebody(which,sincewe'remakingtherequestinthebrowser,isgoingtobeHTML)

HowHTTPRequestswork

128

ParsetheHTMLThebrowsernowhasreceivedtheHTMLandstartstoparseit,andwillrepeattheexactsameprocesswedidnotforalltheresourcesrequiredbythepage:

CSSfilesimagesthefaviconJavaScriptfiles...

Howbrowsersrenderthepagethenisoutofthescope,butit'simportanttounderstandthattheprocessIdescribedisnotjustfortheHTMLpages,butforanyitemthat'sservedoverHTTP.

HowHTTPRequestswork

129

BuildanHTTPserverHowtobuildanHTTPserverwithNode.js

HereistheHTTPwebserverweusedastheNodeHelloWorldapplicationintheNode.jsintroduction

consthttp=require('http')

constport=3000

constserver=http.createServer((req,res)=>{

res.statusCode=200

res.setHeader('Content-Type','text/plain')

res.end('HelloWorld\n')

})

server.listen(port,()=>{

console.log(`Serverrunningathttp://${hostname}:${port}/`)

})

Let'sanalyzeitbriefly.Weincludethe httpmodule.

WeusethemoduletocreateanHTTPserver.

Theserverissettolistenonthespecifiedport, 3000.Whentheserverisready,the listencallbackfunctioniscalled.

Thecallbackfunctionwepassistheonethat'sgoingtobeexecuteduponeveryrequestthatcomesin.Wheneveranewrequestisreceived,the requesteventiscalled,providingtwoobjects:arequest(an http.IncomingMessageobject)andaresponse(an http.ServerResponseobject).

requestprovidestherequestdetails.Throughit,weaccesstherequestheadersandrequestdata.

responseisusedtopopulatethedatawe'regoingtoreturntotheclient.

Inthiscasewith

res.statusCode=200

wesetthestatusCodepropertyto200,toindicateasuccessfulresponse.

WealsosettheContent-Typeheader:

BuildanHTTPserver

130

res.setHeader('Content-Type','text/plain')

andweendclosetheresponse,addingthecontentasanargumentto end():

res.end('HelloWorld\n')

BuildanHTTPserver

131

MakingHTTPrequestsHowtoperformHTTPrequestswithNode.jsusingGET,POST,PUTandDELETE

IusethetermHTTP,butHTTPSiswhatshouldbeusedeverywhere,thereforetheseexamplesuseHTTPSinsteadofHTTP.

PerformaGETRequest

consthttps=require('https')

constoptions={

hostname:'flaviocopes.com',

port:443,

path:'/todos',

method:'GET'

}

constreq=https.request(options,(res)=>{

console.log(`statusCode:${res.statusCode}`)

res.on('data',(d)=>{

process.stdout.write(d)

})

})

req.on('error',(error)=>{

console.error(error)

})

req.end()

PerformaPOSTRequest

consthttps=require('https')

constdata=JSON.stringify({

todo:'Buythemilk'

})

constoptions={

hostname:'flaviocopes.com',

port:443,

path:'/todos',

method:'POST',

headers:{

MakingHTTPrequests

132

'Content-Type':'application/json',

'Content-Length':data.length

}

}

constreq=https.request(options,(res)=>{

console.log(`statusCode:${res.statusCode}`)

res.on('data',(d)=>{

process.stdout.write(d)

})

})

req.on('error',(error)=>{

console.error(error)

})

req.write(data)

req.end()

PUTandDELETEPUTandDELETErequestsusethesamePOSTrequestformat,andjustchangetheoptions.methodvalue.

MakingHTTPrequests

133

AxiosAxiosisaveryconvenientJavaScriptlibrarytoperformHTTPrequestsinNode.js

IntroductionInstallationTheAxiosAPIGETrequestsAddparameterstoGETrequestsPOSTRequests

IntroductionAxiosisaverypopularJavaScriptlibraryyoucanusetoperformHTTPrequests,thatworksinbothBrowserandNode.jsplatforms.

Itsupportsallmodernbrowsers,includingsupportforIE8andhigher.

Itispromise-based,andthisletsuswriteasync/awaitcodetoperformXHRrequestsveryeasily.

UsingAxioshasquiteafewadvantagesoverthenativeFetchAPI:

Axios

134

supportsolderbrowsers(Fetchneedsapolyfill)hasawaytoabortarequesthasawaytosetaresponsetimeouthasbuilt-inCSRFprotectionsupportsuploadprogressperformsautomaticJSONdatatransformationworksinNode.js

InstallationAxioscanbeinstalledusingnpm:

npminstallaxios

oryarn:

yarnaddaxios

orsimplyincludeitinyourpageusingunpkg.com:

<scriptsrc="https://unpkg.com/axios/dist/axios.min.js"></script>

TheAxiosAPIYoucanstartanHTTPrequestfromthe axiosobject:

axios({

url:'https://dog.ceo/api/breeds/list/all',

method:'get',

data:{

foo:'bar'

}

})

butforconvenience,youwillgenerallyuse

axios.get()

axios.post()

(likeinjQueryyouwoulduse $.get()and $.post()insteadof $.ajax())

AxiosoffersmethodsforalltheHTTPverbs,whicharelesspopularbutstillused:

Axios

135

axios.delete()

axios.put()

axios.patch()

axios.options()

andamethodtogettheHTTPheadersofarequest,discardingthebody:

axios.head()

GETrequestsOneconvenientwaytouseAxiosistousethemodern(ES2017)async/awaitsyntax.

ThisNode.jsexamplequeriestheDogAPItoretrievealistofallthedogsbreeds,usingaxios.get(),anditcountsthem:

constaxios=require('axios')

constgetBreeds=async()=>{

try{

returnawaitaxios.get('https://dog.ceo/api/breeds/list/all')

}catch(error){

console.error(error)

}

}

constcountBreeds=async()=>{

constbreeds=awaitgetBreeds()

if(breeds.data.message){

console.log(`Got${Object.entries(breeds.data.message).length}breeds`)

}

}

countBreeds()

Ifyoudon'twanttouseasync/awaityoucanusethePromisessyntax:

constaxios=require('axios')

constgetBreeds=()=>{

try{

returnaxios.get('https://dog.ceo/api/breeds/list/all')

}catch(error){

console.error(error)

}

}

constcountBreeds=async()=>{

constbreeds=getBreeds()

Axios

136

.then(response=>{

if(response.data.message){

console.log(

`Got${Object.entries(response.data.message).length}breeds`

)

}

})

.catch(error=>{

console.log(error)

})

}

countBreeds()

AddparameterstoGETrequestsAGETresponsecancontainparametersintheURL,likethis: https://site.com/?foo=bar.

WithAxiosyoucanperformthisbysimplyusingthatURL:

axios.get('https://site.com/?foo=bar')

oryoucanusea paramspropertyintheoptions:

axios.get('https://site.com/',{

params:{

foo:'bar'

}

})

POSTRequestsPerformingaPOSTrequestisjustlikedoingaGETrequest,butinsteadof axios.get,youuse axios.post:

axios.post('https://site.com/')

AnobjectcontainingthePOSTparametersisthesecondargument:

axios.post('https://site.com/',{

foo:'bar'

})

Axios

137

Axios

138

WebsocketsWebSocketsareanalternativetoHTTPcommunicationinWebApplications.Theyofferalonglived,bidirectionalcommunicationchannelbetweenclientandserver.

WebSocketsareanalternativetoHTTPcommunicationinWebApplications.

Theyofferalonglived,bidirectionalcommunicationchannelbetweenclientandserver.

Onceestablished,thechanneliskeptopen,offeringaveryfastconnectionwithlowlatencyandoverhead.

BrowsersupportforWebSocketsWebSocketsaresupportedbyallmodernbrowsers.

Websockets

139

HowWebSocketsdifferfromHTTPHTTPisaverydifferentprotocol,andalsoadifferentwayofcommunicate.

HTTPisarequest/responseprotocol:theserverreturnssomedatawhentheclientrequestsit.

WithWebSockets:

theservercansendamessagetotheclientwithouttheclientexplicitlyrequestingsomethingtheclientandtheservercantalktoeachothersimultaneouslyverylittledataoverheadneedstobeexchangedtosendmessages.Thismeansalowlatencycommunication.

WebSocketsaregreatforreal-timeandlong-livedcommunications.

HTTPisgreatforoccasionaldataexchangeandinteractionsinitiatedbytheclient.

HTTPismuchsimplertoimplement,whileWebSocketsrequireabitmoreoverhead.

SecuredWebSocketsAlwaysusethesecure,encryptedprotocolforWebSockets, wss://.

ws://referstotheunsafeWebSocketsversion(the http://ofWebSockets),andshouldbeavoidedforobviousreasons.

CreateanewWebSocketsconnection

consturl='wss://myserver.com/something'

constconnection=newWebSocket(url)

connectionisaWebSocketobject.

Whentheconnectionissuccessfullyestablished,the openeventisfired.

Listenforitbyassigningacallbackfunctiontothe onopenpropertyofthe connectionobject:

connection.onopen=()=>{

//...

}

Ifthere'sanyerror,the onerrorfunctioncallbackisfired:

Websockets

140

connection.onerror=error=>{

console.log(`WebSocketerror:${error}`)

}

SendingdatatotheserverusingWebSocketsOncetheconnectionisopen,youcansenddatatotheserver.

Youcandosoconvenientlyinsidethe onopencallbackfunction:

connection.onopen=()=>{

connection.send('hey')

}

ReceivingdatafromtheserverusingWebSocketsListenwithacallbackfunctionon onmessage,whichiscalledwhenthe messageeventisreceived:

connection.onmessage=e=>{

console.log(e.data)

}

ImplementaWebSocketsserverinNode.jswsisapopularWebSocketslibraryforNode.js.

We'lluseittobuildaWebSocketsserver.Itcanalsobeusedtoimplementaclient,anduseWebSocketstocommunicatebetweentwobackendservices.

Easilyinstallitusing

yarninit

yarnaddws

Thecodeyouneedtowriteisverylittle:

constWebSocket=require('ws')

constwss=newWebSocket.Server({port:8080})

Websockets

141

wss.on('connection',ws=>{

ws.on('message',message=>{

console.log(`Receivedmessage=>${message}`)

})

ws.send('ho!')

})

Thiscodecreatesanewserveronport8080(thedefaultportforWebSockets),andaddsacallbackfunctionwhenaconnectionisestablished,sending ho!totheclient,andloggingthemessagesitreceives.

SeealiveexampleonGlitchHereisaliveexampleofaWebSocketsserver:https://glitch.com/edit/#!/flavio-websockets-server-example

HereisaWebSocketsclientthatinteractswiththeserver:https://glitch.com/edit/#!/flavio-websockets-client-example

Websockets

142

HTTPS,secureconnectionsTheHTTPSprotocolisanextensionofHTTP,theHyperTextTransferProtocol,thatprovidesecurecommunication

HTTPininsecurebydesign.

Whenyouopenyourbrowserandaskawebservertosendyouawebpage,yourdataperforms2trips:1fromthebrowsertothewebserver,and1fromthewebservertothebrowser.

Then,dependingonthecontentofthewebpage,youmighthavemoreconnectionsrequiredtogettheCSSfiles,theJavaScriptfiles,images,andsoon.

Duringanyofthoseconnections,anynetworkyourdataisgoingtocrosscanbeinspectedandmanipulated.

Theconsequencescanbeserious:youmighthaveallyournetworkactivitymonitoredandlogged,bya3rdpartyouarenotevenawareitexist,somenetworksmightinjectads,andyoumightbesubjecttoaman-in-the-middleattack,asecuritythreatwheretheattackercanmanipulateyourdataandevenimpersonateyourcomputeroverthenetwork.It'sveryeasyforsomeonetojustlistentoHTTPpacketsbeingtransmittedoverapublicandunencryptedWi-Finetwork.

HTTPSaimstosolvetheproblemattheroot:theentirecommunicationbetweenyourbrowserandthewebserverisencrypted.

Privacyandsecurityareamajorconcernintoday'sinternet.Afewyearsago,youcouldgetawaywithjustusinganencryptedconnectioninlogin-protectedpages,orduringane-commercecheckout.AlsobecauseofSSLcertificatespricingandcomplications,mostwebsitesjustusedHTTP.

TodayHTTPSisarequirementonanysite.Morethan50%ofthewholeWebusesitnow.GoogleChromerecentlystartedmarkingHTTPsitesasinsecure,justtogiveyouavalidreasontohaveHTTPSmandatory(andforced)onallyourwebsites.

WhenusingHTTPthedefaultserverportis80,andonHTTPSit's443.Itdoesnotneedtobeexplicitlyaddediftheserverusesthedefaultport,ofcourse.

HTTPSisalsosometimescalledHTTPoverSSL,orHTTPoverTLS.

Thedifferencebetweenthetwoissimple:TLSisthesuccessorofSSL.

WhenusingHTTPS,theonlythingthatisnotencryptedisthewebserverdomain,andtheserverport.

HTTPS,secureconnections

143

Everyotherinformation,includingtheresourcepath,headers,cookiesandqueryparametersareallencrypted.

Iwon'tgointhedetailsofanalyzinghowtheTLSprotocolworksunderthehoods,butyoumightthinkit'saddingagoodamountofoverhead,andyouwouldberight.

Anycomputationthat'saddedtotheprocessingofnetworkresourcescausesoverheadbothontheclient,theserver,andtothetransmittedpacketssize.

HoweverHTTPSenablestheuseofthenewestprotocolHTTP/2,whichhasahugeadvantageoverHTTP/1.1:itwayfaster.

Why?Therearemanyreasons,oneisheadercompression,oneisresourcemultiplexing.Oneisserverpush:theservercanpushmoreresourceswhenoneresourceisrequested.So,ifthebrowserrequestsapage,itwillalsoreceivealltheresourcesneeded(images,CSS,JS).

Detailsaside,HTTP/2isahugeimprovementoverHTTP/1.1anditrequiresHTTPS.ThismeansthatHTTPS,despitehavingtheencryptionoverhead,happenstobewayfasterthanHTTP,ifthingsareproperlyconfiguredwithamodernsetup.

HTTPS,secureconnections

144

FiledescriptorsHowtointeractwithfiledescriptorsusingNode

Beforeyou'reabletointeractwithafilethatsitsinyourfilesystem,youmustgetafiledescriptor.

Afiledescriptoriswhat'sreturnedbyopeningthefileusingthe open()methodofferedbythefsmodule:

constfs=require('fs')

fs.open('/Users/flavio/test.txt','r',(err,fd)=>{

//fdisourfiledescriptor

})

Noticethe rweusedasthesecondparametertothe fs.open()call.

Thatflagmeansweopenthefileforreading.

Otherflagsyou'llcommonlyuseare

r+openthefileforreadingandwritingw+openthefileforreadingandwriting,positioningthestreamatthebeginningofthefile.Thefileiscreatedifnotexistingaopenthefileforwriting,positioningthestreamattheendofthefile.Thefileiscreatedifnotexistinga+openthefileforreadingandwriting,positioningthestreamattheendofthefile.Thefileiscreatedifnotexisting

Youcanalsoopenthefilebyusingthe fs.openSyncmethod,whichinsteadofprovidingthefiledescriptorobjectinacallback,itreturnsit:

constfs=require('fs')

try{

constfd=fs.openSync('/Users/flavio/test.txt','r')

}catch(err){

console.error(err)

}

Onceyougetthefiledescriptor,inwhateverwayyouchoose,youcanperformalltheoperationsthatrequireit,likecalling fs.open()andmanyotheroperationsthatinteractwiththefilesystem.

Filedescriptors

145

Filedescriptors

146

FilestatsHowtogetthedetailsofafileusingNode

EveryfilecomeswithasetofdetailsthatwecaninspectusingNode.

Inparticular,usingthe stat()methodprovidedbythe fsmodule.

Youcallitpassingafilepath,andonceNodegetsthefiledetailsitwillcallthecallbackfunctionyoupass,with2parameters:anerrormessage,andthefilestats:

constfs=require('fs')

fs.stat('/Users/flavio/test.txt',(err,stats)=>{

if(err){

console.error(err)

return

}

//wehaveaccesstothefilestatsin`stats`

})

Nodeprovidesalsoasyncmethod,whichblocksthethreaduntilthefilestatsareready:

constfs=require('fs')

try{

conststats=fs.stat('/Users/flavio/test.txt')

}catch(err){

console.error(err)

}

Thefileinformationisincludedinthestatsvariable.Whatkindofinformationcanweextractusingthestats?

Alot,including:

ifthefileisadirectoryorafile,using stats.isFile()and stats.isDirectory()ifthefileisasymboliclinkusing stats.isSymbolicLink()thefilesizeinbytesusing stats.size.

Thereareotheradvancedmethods,butthebulkofwhatyou'lluseinyourday-to-dayprogrammingisthis.

constfs=require('fs')

fs.stat('/Users/flavio/test.txt',(err,stats)=>{

if(err){

console.error(err)

return

}

Filestats

147

stats.isFile()//true

stats.isDirectory()//false

stats.isSymbolicLink()//false

stats.size//1024000//=1MB

})

Filestats

148

FilepathsHowtointeractwithfilepathsandmanipulatetheminNode

GettinginformationoutofapathWorkingwithpaths

Everyfileinthesystemhasapath.

OnLinuxandmacOS,apathmightlooklike:

/users/flavio/file.txt

whileWindowscomputersaredifferent,andhaveastructuresuchas:

C:\users\flavio\file.txt

Youneedtopayattentionwhenusingpathsinyourapplications,asthisdifferencemustbetakenintoaccount.

Youincludethismoduleinyourfilesusing

constpath=require('path')

andyoucanstartusingitsmethods.

GettinginformationoutofapathGivenapath,youcanextractinformationoutofitusingthosemethods:

dirname:gettheparentfolderofafilebasename:getthefilenamepartextname:getthefileextension

Example:

constnotes='/users/flavio/notes.txt'

path.dirname(notes)///users/flavio

path.basename(notes)//notes.txt

path.extname(notes)//.txt

Youcangetthefilenamewithouttheextensionbyspecifyingasecondargumenttobasename:

Filepaths

149

path.basename(notes,path.extname(notes))//notes

WorkingwithpathsYoucanjointwoormorepartsofapathbyusing path.join():

constname='flavio'

path.join('/','users',name,'notes.txt')//'/users/flavio/notes.txt'

Youcangettheabsolutepathcalculationofarelativepathusing path.resolve():

path.resolve('flavio.txt')//'/Users/flavio/flavio.txt'ifrunfrommyhomefolder

InthiscaseNodewillsimplyappend /flavio.txttothecurrentworkingdirectory.Ifyouspecifyasecondparameterfolder, resolvewillusethefirstasabaseforthesecond:

path.resolve('tmp','flavio.txt')//'/Users/flavio/tmp/flavio.txt'ifrunfrommyhomefold

er

Ifthefirstparameterstartswithaslash,thatmeansit'sanabsolutepath:

path.resolve('/etc','flavio.txt')//'/etc/flavio.txt'

path.normalize()isanotherusefulfunction,thatwilltryandcalculatetheactualpath,whenitcontainsrelativespecifierslike .or ..,ordoubleslashes:

path.normalize('/users/flavio/..//test.txt')///users/test.txt

Bothresolveandnormalizewillnotcheckifthepathexists.Theyjustcalculateapathbasedontheinformationtheygot.

Filepaths

150

ReadingfilesHowtoreadfilesusingNode

ThesimplestwaytoreadafileinNodeistousethe fs.readFile()method,passingitthefilepathandacallbackfunctionthatwillbecalledwiththefiledata(andtheerror):

constfs=require('fs')

fs.readFile('/Users/flavio/test.txt',(err,data)=>{

if(err){

console.error(err)

return

}

console.log(data)

})

Alternatively,youcanusethesynchronousversion fs.readFileSync():

constfs=require('fs')

try{

constdata=fs.readFileSync('/Users/flavio/test.txt','utf8')

console.log(data)

}catch(err){

console.error(err)

}

Thedefaultencodingisutf8,butyoucanspecifyacustomencodingusingaasecondparameter.

Both fs.readFile()and fs.readFileSync()readthefullcontentofthefileinmemorybeforereturningthedata.

Thismeansthatbigfilesaregoingtohaveamajorimpactonyourmemoryconsumptionandspeedofexecutionoftheprogram.

Inthiscase,abetteroptionistoreadthefilecontentusingstreams.

Readingfiles

151

WritingfilesHowtowritefilesusingNode

TheeasiestwaytowritetofilesinNode.jsistousethe fs.writeFile()API.

Example:

constfs=require('fs')

constcontent='Somecontent!'

fs.writeFile('/Users/flavio/test.txt',content,(err)=>{

if(err){

console.error(err)

return

}

//filewrittensuccessfully

})

Alternatively,youcanusethesynchronousversion fs.writeFileSync():

constfs=require('fs')

constcontent='Somecontent!'

try{

constdata=fs.writeFileSync('/Users/flavio/test.txt',content)

//filewrittensuccessfully

}catch(err){

console.error(err)

}

Bydefault,thisAPIwillreplacethecontentsofthefileifitdoesalreadyexist.

Youcanmodifythedefaultbyspecifyingaflag:

fs.writeFile('/Users/flavio/test.txt',content,{flag:'a+'},(err)=>{})

Theflagsyou'lllikelyuseare

r+openthefileforreadingandwritingw+openthefileforreadingandwriting,positioningthestreamatthebeginningofthefile.Thefileiscreatedifnotexistingaopenthefileforwriting,positioningthestreamattheendofthefile.Thefileiscreatedifnotexisting

Writingfiles

152

a+openthefileforreadingandwriting,positioningthestreamattheendofthefile.Thefileiscreatedifnotexisting

(youcanfindmoreflagsathttps://nodejs.org/api/fs.html#fs_file_system_flags)

AppendtoafileAhandymethodtoappendcontenttotheendofafileis fs.appendFile()(anditsfs.appendFileSync()counterpart):

constcontent='Somecontent!'

fs.appendFile('file.log',content,(err)=>{

if(err){

console.error(err)

return

}

//done!

})

UsingstreamsAllthosemethodswritethefullcontenttothefilebeforereturningthecontrolbacktoyourprogram(intheasyncversion,thismeansexecutingthecallback)

Inthiscase,abetteroptionistowritethefilecontentusingstreams.

Writingfiles

153

WorkingwithfoldersHowtointeractwithfoldersusingNode

TheNode.js fscoremoduleprovidesmanyhandymethodsyoucanusetoworkwithfolders.

CheckifafolderexistsUse fs.access()tocheckifthefolderexistsandNodecanaccessitwithitspermissions.

CreateanewfolderUse fs.mkdir()or fs.mkdirSync()tocreateanewfolder.

constfs=require('fs')

constfolderName='/Users/flavio/test'

try{

if(!fs.existsSync(dir)){

fs.mkdirSync(dir)

}

}catch(err){

console.error(err)

}

ReadthecontentofadirectoryUse fs.readdir()or fs.readdirSynctoreadthecontentsofadirectory.

Thispieceofcodereadsthecontentofafolder,bothfilesandsubfolders,andreturnstheirrelativepath:

constfs=require('fs')

constpath=require('path')

constfolderPath='/Users/flavio'

fs.readdirSync(folderPath)

Youcangetthefullpath:

Workingwithfolders

154

fs.readdirSync(folderPath).map(fileName=>{

returnpath.join(folderPath,fileName)

}

Youcanalsofiltertheresultstoonlyreturnthefiles,andexcludethefolders:

constisFile=fileName=>{

returnfs.lstatSync(fileName).isFile()

}

fs.readdirSync(folderPath).map(fileName=>{

returnpath.join(folderPath,fileName)).filter(isFile)

}

RenameafolderUse fs.rename()or fs.renameSync()torenamefolder.Thefirstparameteristhecurrentpath,thesecondthenewpath:

constfs=require('fs')

fs.rename('/Users/flavio','/Users/roger',(err)=>{

if(err){

console.error(err)

return

}

//done

})

fs.renameSync()isthesynchronousversion:

constfs=require('fs')

try{

fs.renameSync('/Users/flavio','/Users/roger')

}catch(err){

console.error(err)

}

RemoveafolderUse fs.rmdir()or fs.rmdirSync()toremoveafolder.

Removingafolderthathascontentcanbemorecomplicatedthanyouneed.

Workingwithfolders

155

InthiscaseIrecommendinstallingthe fs-extramodule,whichisverypopularandwellmaintained,andit'sadrop-inreplacementofthe fsmodule,providingmorefeaturesontopofit.

Inthiscasethe remove()methodiswhatyouwant.

Installitusing

npminstallfs-extra

anduseitlikethis:

constfs=require('fs-extra')

constfolder='/Users/flavio'

fs.remove(folder,err=>{

console.error(err)

})

Itcanalsobeusedwithpromises:

fs.remove(folder).then(()=>{

//done

}).catch(err=>{

console.error(err)

})

orwithasync/await:

asyncfunctionremoveFolder(folder){

try{

awaitfs.remove(folder)

//done

}catch(err){

console.error(err)

}

}

constfolder='/Users/flavio'

removeFolder(folder)

Workingwithfolders

156

ThefsmoduleThefsmoduleofNode.jsprovidesusefulfunctionstointeractwiththefilesystem

The fsmoduleprovidesalotofveryusefulfunctionalitytoaccessandinteractwiththefilesystem.

Thereisnoneedtoinstallit.BeingpartoftheNodecore,itcanbeusedbysimplyrequiringit:

constfs=require('fs')

Onceyoudoso,youhaveaccesstoallitsmethods,whichinclude:

fs.access():checkifthefileexistsandNodecanaccessitwithitspermissionsfs.appendFile():appenddatatoafile.Ifthefiledoesnotexist,it'screatedfs.chmod():changethepermissionsofafilespecifiedbythefilenamepassed.Related:fs.lchmod(), fs.fchmod()fs.chown():changetheownerandgroupofafilespecifiedbythefilenamepassed.Related: fs.fchown(), fs.lchown()fs.close():closeafiledescriptorfs.copyFile():copiesafilefs.createReadStream():createareadablefilestreamfs.createWriteStream():createawritablefilestreamfs.link():createanewhardlinktoafilefs.mkdir():createanewfolderfs.mkdtemp():createatemporarydirectoryfs.open():setthefilemodefs.readdir():readthecontentsofadirectoryfs.readFile():readthecontentofafile.Related: fs.read()fs.readlink():readthevalueofasymboliclinkfs.realpath():resolverelativefilepathpointers( ., ..)tothefullpathfs.rename():renameafileorfolderfs.rmdir():removeafolderfs.stat():returnsthestatusofthefileidentifiedbythefilenamepassed.Related:fs.fstat(), fs.lstat()fs.symlink():createanewsymboliclinktoafilefs.truncate():truncatetothespecifiedlengththefileidentifiedbythefilenamepassed.Related: fs.ftruncate()fs.unlink():removeafileorasymboliclink

Thefsmodule

157

fs.unwatchFile():stopwatchingforchangesonafilefs.utimes():changethetimestampofthefileidentifiedbythefilenamepassed.Related:fs.futimes()

fs.watchFile():startwatchingforchangesonafile.Related: fs.watch()fs.writeFile():writedatatoafile.Related: fs.write()

Onepeculiarthingaboutthe fsmoduleisthatallthemethodsareasynchronousbydefault,buttheycanalsoworksynchronouslybyappending Sync.

Forexample:

fs.rename()

fs.renameSync()

fs.write()

fs.writeSync()

Thismakesahugedifferenceinyourapplicationflow.

Node10includesexperimentalsupportforapromisebasedAPI

Forexamplelet'sexaminethe fs.rename()method.TheasynchronousAPIisusedwithacallback:

constfs=require('fs')

fs.rename('before.json','after.json',(err)=>{

if(err){

returnconsole.error(err)

}

//done

})

AsynchronousAPIcanbeusedlikethis,withatry/catchblocktohandleerrors:

constfs=require('fs')

try{

fs.renameSync('before.json','after.json')

//done

}catch(err){

console.error(err)

}

Thekeydifferencehereisthattheexecutionofyourscriptwillblockinthesecondexample,untilthefileoperationsucceeded.

Thefsmodule

158

Thefsmodule

159

ThepathmoduleThepathmoduleofNode.jsprovidesusefulfunctionstointeractwithfilepaths

The pathmoduleprovidesalotofveryusefulfunctionalitytoaccessandinteractwiththefilesystem.

Thereisnoneedtoinstallit.BeingpartoftheNodecore,itcanbeusedbysimplyrequiringit:

constpath=require('path')

Thismoduleprovides path.sepwhichprovidesthepathsegmentseparator( \onWindows,and /onLinux/macOS),and path.delimiterwhichprovidesthepathdelimiter( ;onWindows,and :onLinux/macOS).

Thesearethe pathmethods:

path.basename()

path.dirname()

path.extname()

path.isAbsolute()

path.join()

path.normalize()

path.parse()

path.relative()

path.resolve()

path.basename()

Returnthelastportionofapath.Asecondparametercanfilteroutthefileextension:

require('path').basename('/test/something')//something

require('path').basename('/test/something.txt')//something.txt

require('path').basename('/test/something.txt','.txt')//something

path.dirname()

Returnthedirectorypartofapath:

require('path').dirname('/test/something')///test

require('path').dirname('/test/something/file.txt')///test/something

Thepathmodule

160

path.extname()

Returntheextensionpartofapath

require('path').dirname('/test/something')//''

require('path').dirname('/test/something/file.txt')//'.txt'

path.isAbsolute()

Returnstrueifit'sanabsolutepath

require('path').isAbsolute('/test/something')//true

require('path').isAbsolute('./test/something')//false

path.join()

Joinstwoormorepartsofapath:

constname='flavio'

require('path').join('/','users',name,'notes.txt')//'/users/flavio/notes.txt'

path.normalize()

Triestocalculatetheactualpathwhenitcontainsrelativespecifierslike .or ..,ordoubleslashes:

require('path').normalize('/users/flavio/..//test.txt')///users/test.txt

path.parse()

Parsesapathtoanobjectwiththesegmentsthatcomposeit:

root:therootdir:thefolderpathstartingfromtherootbase:thefilename+extensionname:thefilenameext:thefileextension

Example:

require('path').parse('/users/test.txt')

Thepathmodule

161

resultsin

{

root:'/',

dir:'/users',

base:'test.txt',

ext:'.txt',

name:'test'

}

path.relative()

Accepts2pathsasarguments.Returnsthetherelativepathfromthefirstpathtothesecond,basedonthecurrentworkingdirectory.

Example:

require('path').relative('/Users/flavio','/Users/flavio/test.txt')//'test.txt'

require('path').relative('/Users/flavio','/Users/flavio/something/test.txt')//'something

/test.txt'

path.resolve()

Youcangettheabsolutepathcalculationofarelativepathusing path.resolve():

path.resolve('flavio.txt')//'/Users/flavio/flavio.txt'ifrunfrommyhomefolder

Byspecifyingasecondparameter, resolvewillusethefirstasabaseforthesecond:

path.resolve('tmp','flavio.txt')//'/Users/flavio/tmp/flavio.txt'ifrunfrommyhomefold

er

Ifthefirstparameterstartswithaslash,thatmeansit'sanabsolutepath:

path.resolve('/etc','flavio.txt')//'/etc/flavio.txt'

Thepathmodule

162

TheosmoduleTheosmoduleofNode.jsprovidesusefulfunctionstointeractwithunderlyingsystem

Thismoduleprovidesmanyfunctionsthatyoucanusetoretrieveinformationfromtheunderlyingoperatingsystemandthecomputertheprogramrunson,andinteractwithit.

constos=require('os')

Thereareafewusefulpropertiesthattellussomekeythingsrelatedtohandlingfiles:

os.EOLgivesthelinedelimitersequence.It's \nonLinuxandmacOS,and \r\nonWindows.

WhenIsayLinuxandmacOSImeanPOSIXplatforms.ForsimplicityIexcludeotherlesspopularoperatingsystemsNodecanrunon.

os.constants.signalstellsusalltheconstantsrelatedtohandlingprocesssignals,likeSIGHUP,SIGKILLandsoon.

os.constants.errnosetstheconstantsforerrorreporting,likeEADDRINUSE,EOVERFLOWandmore.

Youcanreadthemallonhttps://nodejs.org/api/os.html#os_signal_constants.

Let'snowseethemainmethodsthat osprovides:

os.arch()

os.cpus()

os.endianness()

os.freemem()

os.homedir()

os.hostname()

os.loadavg()

os.networkInterfaces()

os.platform()

os.release()

os.tmpdir()

os.totalmem()

os.type()

os.uptime()

os.userInfo()

Theosmodule

163

os.arch()

Returnthestringthatidentifiestheunderlyingarchitecture,like arm, x64, arm64.

os.cpus()

ReturninformationontheCPUsavailableonyoursystem.

Example:

[{model:'Intel(R)Core(TM)2DuoCPUP8600@2.40GHz',

speed:2400,

times:

{user:281685380,

nice:0,

sys:187986530,

idle:685833750,

irq:0}},

{model:'Intel(R)Core(TM)2DuoCPUP8600@2.40GHz',

speed:2400,

times:

{user:282348700,

nice:0,

sys:161800480,

idle:703509470,

irq:0}}]

os.endianness()

Return BEor LEdependingifNodewascompiledwithBigEndianorLittleEndian.

os.freemem()

Returnthenumberofbytesthatrepresentthefreememoryinthesystem.

os.homedir()

Returnthepathtothehomedirectoryofthecurrentuser.

Example:

'/Users/flavio'

Theosmodule

164

os.hostname()

Returnthehostname.

os.loadavg()

Returnthecalculationmadebytheoperatingsystemontheloadaverage.

ItonlyreturnsameaningfulvalueonLinuxandmacOS.

Example:

[3.68798828125,4.00244140625,11.1181640625]

os.networkInterfaces()

Returnsthedetailsofthenetworkinterfacesavailableonyoursystem.

Example:

{lo0:

[{address:'127.0.0.1',

netmask:'255.0.0.0',

family:'IPv4',

mac:'fe:82:00:00:00:00',

internal:true},

{address:'::1',

netmask:'ffff:ffff:ffff:ffff:ffff:ffff:ffff:ffff',

family:'IPv6',

mac:'fe:82:00:00:00:00',

scopeid:0,

internal:true},

{address:'fe80::1',

netmask:'ffff:ffff:ffff:ffff::',

family:'IPv6',

mac:'fe:82:00:00:00:00',

scopeid:1,

internal:true}],

en1:

[{address:'fe82::9b:8282:d7e6:496e',

netmask:'ffff:ffff:ffff:ffff::',

family:'IPv6',

mac:'06:00:00:02:0e:00',

scopeid:5,

internal:false},

{address:'192.168.1.38',

netmask:'255.255.255.0',

family:'IPv4',

mac:'06:00:00:02:0e:00',

Theosmodule

165

internal:false}],

utun0:

[{address:'fe80::2513:72bc:f405:61d0',

netmask:'ffff:ffff:ffff:ffff::',

family:'IPv6',

mac:'fe:80:00:20:00:00',

scopeid:8,

internal:false}]}

os.platform()

ReturntheplatformthatNodewascompiledfor:

darwin

freebsd

linux

openbsd

win32

...more

os.release()

Returnsastringthatidentifiestheoperatingsystemreleasenumber

os.tmpdir()

Returnsthepathtotheassignedtempfolder.

os.totalmem()

Returnsthenumberofbytesthatrepresentthetotalmemoryavailableinthesystem.

os.type()

Identifiestheoperatingsystem:

Linux

DarwinonmacOSWindows_NTonWindows

os.uptime()

Theosmodule

166

Returnsthenumberofsecondsthecomputerhasbeenrunningsinceitwaslastrebooted.

os.userInfo()

Theosmodule

167

TheeventsmoduleTheeventsmoduleofNode.jsprovidestheEventEmitterclass

The eventsmoduleprovidesustheEventEmitterclass,whichiskeytoworkingwitheventsinNode.

Ipublishedafullarticleonthat,sohereIwilljustdescribetheAPIwithoutfurtherexamplesonhowtouseit.

constEventEmitter=require('events')

constdoor=newEventEmitter()

Theeventlistenereatsitsowndogfoodandusestheseevents:

newListenerwhenalistenerisaddedremoveListenerwhenalistenerisremoved

Here'sadetaileddescriptionofthemostusefulmethods:

emitter.addListener()

emitter.emit()

emitter.eventNames()

emitter.getMaxListeners()

emitter.listenerCount()

emitter.listeners()

emitter.off()

emitter.on()

emitter.once()

emitter.prependListener()

emitter.prependOnceListener()

emitter.removeAllListeners()

emitter.removeListener()

emitter.setMaxListeners()

emitter.addListener()

Aliasfor emitter.on().

emitter.emit()

Theeventsmodule

168

Emitsanevent.Itsynchronouslycallseveryeventlistenerintheordertheywereregistered.

emitter.eventNames()

ReturnanarrayofstringsthatrepresenttheeventsregisteredonthecurrentEventListener:

door.eventNames()

emitter.getMaxListeners()

GetthemaximumamountoflistenersonecanaddtoanEventListenerobject,whichdefaultsto10butcanbeincreasedorloweredbyusing setMaxListeners()

door.getMaxListeners()

emitter.listenerCount()

Getthecountoflistenersoftheeventpassedasparameter:

door.listenerCount('open')

emitter.listeners()

Getsanarrayoflistenersoftheeventpassedasparameter:

door.listeners('open')

emitter.off()

Aliasfor emitter.removeListener()addedinNode10

emitter.on()

Addsacallbackfunctionthat'scalledwhenaneventisemitted.

Usage:

door.on('open',()=>{

Theeventsmodule

169

console.log('Doorwasopened')

})

emitter.once()

Addsacallbackfunctionthat'scalledwhenaneventisemittedforthefirsttimeafterregisteringthis.Thiscallbackisonlygoingtobecalledonce,neveragain.

constEventEmitter=require('events')

constee=newEventEmitter()

ee.once('my-event',()=>{

//callcallbackfunctiononce

})

emitter.prependListener()

Whenyouaddalistenerusing onor addListener,it'saddedlastinthequeueoflisteners,andcalledlast.Using prependListenerit'sadded,andcalled,beforeotherlisteners.

emitter.prependOnceListener()

Whenyouaddalistenerusing once,it'saddedlastinthequeueoflisteners,andcalledlast.Using prependOnceListenerit'sadded,andcalled,beforeotherlisteners.

emitter.removeAllListeners()

Removesalllistenersofaneventemitterobjectlisteningtoaspecificevent:

door.removeAllListeners('open')

emitter.removeListener()

Removeaspecificlistener.Youcandothisbysavingthecallbackfunctiontoavariable,whenadded,soyoucanreferenceitlater:

constdoSomething=()=>{}

door.on('open',doSomething)

door.removeListener('open',doSomething)

Theeventsmodule

170

emitter.setMaxListeners()

SetsthemaximumamountoflistenersonecanaddtoanEventListenerobject,whichdefaultsto10butcanbeincreasedorlowered.

door.setMaxListeners(50)

Theeventsmodule

171

ThehttpmoduleThehttpmoduleofNode.jsprovidesusefulfunctionsandclassestobuildanHTTPserver

TheHTTPcoremoduleisakeymoduletoNodenetworking.

Propertieshttp.METHODS

http.STATUS_CODES

http.globalAgent

Methodshttp.createServer()

http.request()

http.get()

Classeshttp.Agent

http.ClientRequest

http.Server

http.ServerResponse

http.IncomingMessage

Itcanbeincludedusing

consthttp=require('http')

Themoduleprovidessomepropertiesandmethods,andsomeclasses.

Properties

http.METHODS

ThispropertylistsalltheHTTPmethodssupported:

>require('http').METHODS

['ACL',

'BIND',

'CHECKOUT',

'CONNECT',

'COPY',

'DELETE',

'GET',

Thehttpmodule

172

'HEAD',

'LINK',

'LOCK',

'M-SEARCH',

'MERGE',

'MKACTIVITY',

'MKCALENDAR',

'MKCOL',

'MOVE',

'NOTIFY',

'OPTIONS',

'PATCH',

'POST',

'PROPFIND',

'PROPPATCH',

'PURGE',

'PUT',

'REBIND',

'REPORT',

'SEARCH',

'SUBSCRIBE',

'TRACE',

'UNBIND',

'UNLINK',

'UNLOCK',

'UNSUBSCRIBE']

http.STATUS_CODES

ThispropertylistsalltheHTTPstatuscodesandtheirdescription:

>require('http').STATUS_CODES

{'100':'Continue',

'101':'SwitchingProtocols',

'102':'Processing',

'200':'OK',

'201':'Created',

'202':'Accepted',

'203':'Non-AuthoritativeInformation',

'204':'NoContent',

'205':'ResetContent',

'206':'PartialContent',

'207':'Multi-Status',

'208':'AlreadyReported',

'226':'IMUsed',

'300':'MultipleChoices',

'301':'MovedPermanently',

'302':'Found',

'303':'SeeOther',

'304':'NotModified',

'305':'UseProxy',

'307':'TemporaryRedirect',

'308':'PermanentRedirect',

'400':'BadRequest',

Thehttpmodule

173

'401':'Unauthorized',

'402':'PaymentRequired',

'403':'Forbidden',

'404':'NotFound',

'405':'MethodNotAllowed',

'406':'NotAcceptable',

'407':'ProxyAuthenticationRequired',

'408':'RequestTimeout',

'409':'Conflict',

'410':'Gone',

'411':'LengthRequired',

'412':'PreconditionFailed',

'413':'PayloadTooLarge',

'414':'URITooLong',

'415':'UnsupportedMediaType',

'416':'RangeNotSatisfiable',

'417':'ExpectationFailed',

'418':'I\'mateapot',

'421':'MisdirectedRequest',

'422':'UnprocessableEntity',

'423':'Locked',

'424':'FailedDependency',

'425':'UnorderedCollection',

'426':'UpgradeRequired',

'428':'PreconditionRequired',

'429':'TooManyRequests',

'431':'RequestHeaderFieldsTooLarge',

'451':'UnavailableForLegalReasons',

'500':'InternalServerError',

'501':'NotImplemented',

'502':'BadGateway',

'503':'ServiceUnavailable',

'504':'GatewayTimeout',

'505':'HTTPVersionNotSupported',

'506':'VariantAlsoNegotiates',

'507':'InsufficientStorage',

'508':'LoopDetected',

'509':'BandwidthLimitExceeded',

'510':'NotExtended',

'511':'NetworkAuthenticationRequired'}

http.globalAgent

PointstotheglobalinstanceoftheAgentobject,whichisaninstanceofthe http.Agentclass.

It'susedtomanageconnectionspersistanceandreuseforHTTPclients,andit'sakeycomponentofNodeHTTPnetworking.

Moreinthe http.Agentclassdescriptionlateron.

Methods

Thehttpmodule

174

http.createServer()

Returnanewinstanceofthe http.Serverclass.

Usage:

constserver=http.createServer((req,res)=>{

//handleeverysinglerequestwiththiscallback

})

http.request()

MakesanHTTPrequesttoaserver,creatinganinstanceofthe http.ClientRequestclass.

http.get()

Similarto http.request(),butautomaticallysetstheHTTPmethodtoGET,andcallsreq.end()automatically.

ClassesTheHTTPmoduleprovides5classes:

http.Agent

http.ClientRequest

http.Server

http.ServerResponse

http.IncomingMessage

http.Agent

Nodecreatesaglobalinstanceofthe http.AgentclasstomanageconnectionspersistanceandreuseforHTTPclients,akeycomponentofNodeHTTPnetworking.

Thisobjectmakessurethateveryrequestmadetoaserverisqueuedandasinglesocketisreused.

Italsomaintainsapoolofsockets.Thisiskeyforperformancereasons.

http.ClientRequest

An http.ClientRequestobjectiscreatedwhen http.request()or http.get()iscalled.

Thehttpmodule

175

Whenaresponseisreceived,the responseeventiscalledwiththeresponse,withanhttp.IncomingMessageinstanceasargument.

Thereturneddataofaresponsecanbereadin2ways:

youcancallthe response.read()methodinthe responseeventhandleryoucansetupaneventlistenerforthe dataevent,soyoucanlistenforthedatastreamedinto.

http.Server

Thisclassiscommonlyinstantiatedandreturnedwhencreatinganewserverusinghttp.createServer().

Onceyouhaveaserverobject,youhaveaccesstoitsmethods:

close()stopstheserverfromacceptingnewconnectionslisten()startstheHTTPserverandlistensforconnections

http.ServerResponse

Createdbyan http.Serverandpassedasthesecondparametertothe requesteventitfires.

Commonlyknownandusedincodeas res:

constserver=http.createServer((req,res)=>{

//resisanhttp.ServerResponseobject

})

Themethodyou'llalwayscallinthehandleris end(),whichclosestheresponse,themessageiscompleteandtheservercansendittotheclient.Itmustbecalledoneachresponse.

ThesemethodsareusedtointeractwithHTTPheaders:

getHeaderNames()getthelistofthenamesoftheHTTPheadersalreadysetgetHeaders()getacopyoftheHTTPheadersalreadysetsetHeader('headername',value)setsanHTTPheadervaluegetHeader('headername')getsanHTTPheaderalreadysetremoveHeader('headername')removesanHTTPheaderalreadysethasHeader('headername')returntrueiftheresponsehasthatheadersetheadersSent()returntrueiftheheadershavealreadybeensenttotheclient

Thehttpmodule

176

Afterprocessingtheheadersyoucansendthemtotheclientbycalling response.writeHead(),whichacceptsthestatusCodeasthefirstparameter,theoptionalstatusmessage,andtheheadersobject.

Tosenddatatotheclientintheresponsebody,youuse write().ItwillsendbuffereddatatotheHTTPresponsestream.

Iftheheaderswerenotsentyetusing response.writeHead(),itwillsendtheheadersfirst,withthestatuscodeandmessagethat'ssetintherequest,whichyoucaneditbysettingthestatusCodeand statusMessagepropertiesvalues:

response.statusCode=500

response.statusMessage='InternalServerError'

http.IncomingMessage

An http.IncomingMessageobjectiscreatedby:

http.Serverwhenlisteningtothe requesteventhttp.ClientRequestwhenlisteningtothe responseevent

Itcanbeusedtoaccesstheresponse:

statususingits statusCodeand statusMessagemethodsheadersusingits headersmethodor rawHeadersHTTPmethodusingits methodmethodHTTPversionusingthe httpVersionmethodURLusingthe urlmethodunderlyingsocketusingthe socketmethod

Thedataisaccessedusingstreams,since http.IncomingMessageimplementstheReadableStreaminterface.

Thehttpmodule

177

StreamsLearnwhatstreamsarefor,whyaretheysoimportant,andhowtousethem.

WhatarestreamsWhystreamsAnexampleofastreampipe()Streams-poweredNodeAPIsDifferenttypesofstreamsHowtocreateareadablestreamHowtocreateawritablestreamHowtogetdatafromareadablestreamHowtosenddatatoawritablestreamSignalingawritablestreamthatyouendedwritingConclusion

WhatarestreamsStreamsareoneofthefundamentalconceptsthatpowerNode.jsapplications.

Theyareawaytohandlereading/writingfiles,networkcommunications,oranykindofend-to-endinformationexchangeinanefficientway.

StreamsarenotaconceptuniquetoNode.js.TheywereintroducedintheUnixoperatingsystemdecadesago,andprogramscaninteractwitheachotherpassingstreamsthroughthepipeoperator( |).

Forexample,inthetraditionalway,whenyoutelltheprogramtoreadafile,thefileisreadintomemory,fromstarttofinish,andthenyouprocessit.

Usingstreamsyoureaditpiecebypiece,processingitscontentwithoutkeepingitallinmemory.

TheNode.js streammoduleprovidesthefoundationuponwhichallstreamingAPIsarebuild.

WhystreamsStreamsbasicallyprovidetwomajoradvantagesusingotherdatahandlingmethods:

Memoryefficiency:youdon'tneedtoloadlargeamountsofdatainmemorybeforeyou

Streams

178

areabletoprocessitTimeefficiency:ittakeswaylesstimetostartprocessingdataassoonasyouhaveit,ratherthanwaitingtillthewholedatapayloadisavailabletostart

AnexampleofastreamAtypicalexampleistheoneofreadingfilesfromadisk.

UsingtheNode fsmoduleyoucanreadafile,andserveitoverHTTPwhenanewconnectionisestablishedtoyourhttpserver:

consthttp=require('http')

constfs=require('fs')

constserver=http.createServer(function(req,res){

fs.readFile(__dirname+'/data.txt',(err,data)=>{

res.end(data)

})

})

server.listen(3000)

readFile()readsthefullcontentsofthefile,andinvokesthecallbackfunctionwhenit'sdone.

res.end(data)inthecallbackwillreturnthefilecontentstotheHTTPclient.

Ifthefileisbig,theoperationwilltakequiteabitoftime.Hereisthesamethingwrittenusingstreams:

consthttp=require('http')

constfs=require('fs')

constserver=http.createServer((req,res)=>{

conststream=fs.createReadStream(__dirname+'/data.txt')

stream.pipe(res)

})

server.listen(3000)

Insteadofwaitinguntilthefileisfullyread,westartstreamingittotheHTTPclientassoonaswehaveachunkofdatareadytobesent.

pipe()Theaboveexampleusestheline stream.pipe(res):the pipe()methodiscalledonthefilestream.

Streams

179

Whatdoesthiscodedo?Ittakesthesource,andpipesitintoadestination.

Youcallitonthesourcestream,sointhiscase,thefilestreamispipedtotheHTTPresponse.

Thereturnvalueofthe pipe()methodisthedestinationstream,whichisaveryconvenientthingthatletsuschainmultiple pipe()calls,likethis:

src.pipe(dest1).pipe(dest2)

Thisconstructisthesameasdoing

src.pipe(dest1)

dest1.pipe(dest2)

Streams-poweredNodeAPIsDuetotheiradvantages,manyNode.jscoremodulesprovidenativestreamhandlingcapabilities,mostnotably:

process.stdinreturnsastreamconnectedtostdinprocess.stdoutreturnsastreamconnectedtostdoutprocess.stderrreturnsastreamconnectedtostderrfs.createReadStream()createsareadablestreamtoafilefs.createWriteStream()createsawritablestreamtoafilenet.connect()initiatesastream-basedconnectionhttp.request()returnsaninstanceofthehttp.ClientRequestclass,whichisawritablestreamzlib.createGzip()compressdatausinggzip(acompressionalgorithm)intoastreamzlib.createGunzip()decompressagzipstream.zlib.createDeflate()compressdatausingdeflate(acompressionalgorithm)intoastreamzlib.createInflate()decompressadeflatestream

DifferenttypesofstreamsTherearefourclassesofstreams:

Readable:astreamyoucanpipefrom,butnotpipeinto(youcanreceivedata,butnotsenddatatoit).Whenyoupushdataintoareadablestream,itisbuffered,untilaconsumerstartstoreadthedata.Writable:astreamyoucanpipeinto,butnotpipefrom(youcansenddata,butnot

Streams

180

receivefromit)Duplex:astreamyoucanbothpipeintoandpipefrom,basicallyacombinationofaReadableandWritablestreamTransform:aTransformstreamissimilartoaDuplex,buttheoutputisatransformofitsinput

HowtocreateareadablestreamWegettheReadablestreamfromthe streammodule,andweinitializeit

constStream=require('stream')

constreadableStream=newStream.Readable()

Nowthatthestreamisinitialized,wecansenddatatoit:

readableStream.push('hi!')

readableStream.push('ho!')

HowtocreateawritablestreamTocreateawritablestreamweextendthebase Writableobject,andweimplementits_write()method.

Firstcreateastreamobject:

constStream=require('stream')

constwritableStream=newStream.Writable()

thenimplement _write:

writableStream._write=(chunk,encoding,next)=>{

console.log(chunk.toString())

next()

}

Youcannowpipeareadablestreamin:

process.stdin.pipe(writableStream)

Howtogetdatafromareadablestream

Streams

181

Howdowereaddatafromareadablestream?Usingawritablestream:

constStream=require('stream')

constreadableStream=newStream.Readable()

constwritableStream=newStream.Writable()

writableStream._write=(chunk,encoding,next)=>{

console.log(chunk.toString())

next()

}

readableStream.pipe(writableStream)

readableStream.push('hi!')

readableStream.push('ho!')

Youcanalsoconsumeareadablestreamdirectly,usingthe readableevent:

readableStream.on('readable',()=>{

console.log(readableStream.read())

})

HowtosenddatatoawritablestreamUsingthestream write()method:

writableStream.write('hey!\n')

SignalingawritablestreamthatyouendedwritingUsethe end()method:

constStream=require('stream')

constreadableStream=newStream.Readable()

constwritableStream=newStream.Writable()

writableStream._write=(chunk,encoding,next)=>{

console.log(chunk.toString())

next()

}

readableStream.pipe(writableStream)

Streams

182

readableStream.push('hi!')

readableStream.push('ho!')

writableStream.end()

ConclusionThisisanintroductiontostreams.Therearemuchmorecomplicatedaspectstoanalyze,andIhopetocoverthemsoon.

Streams

183

WorkingwithMySQLMySQLisoneofthemostpopularrelationaldatabasesintheworld.FindouthowtomakeitworkwithNode.js

MySQLisoneofthemostpopularrelationaldatabasesintheworld.

TheNodeecosystemofcoursehasseveraldifferentpackagesthatallowyoutointerfacewithMySQL,storedata,retrievedata,andsoon.

We'lluse mysqljs/mysql,apackagethathasover12.000GitHubstarsandhasbeenaroundforyears.

InstallingtheNodemysqlpackageYouinstallitusing

npminstallmysql

InitializingtheconnectiontothedatabaseYoufirstincludethepackage:

constmysql=require('mysql')

andyoucreateaconnection:

constoptions={

user:'the_mysql_user_name',

password:'the_mysql_user_password',

database:'the_mysql_database_name'

}

constconnection=mysql.createConnection(options)

Youinitiateanewconnectionbycalling:

connection.connect(err=>{

if(err){

console.error('AnerroroccurredwhileconnectingtotheDB')

throwerr

}

})

WorkingwithMySQL

184

TheconnectionoptionsIntheaboveexample,the optionsobjectcontained3options:

constoptions={

user:'the_mysql_user_name',

password:'the_mysql_user_password',

database:'the_mysql_database_name'

}

Therearemanymoreyoucanuse,including:

host,thedatabasehostname,defaultsto localhostport,theMySQLserverportnumber,defaultsto3306socketPath,usedtospecifyaunixsocketinsteadofhostandportdebug,bydefaultdisabled,canbeusedfordebuggingtrace,bydefaultenabled,printsstacktraceswhenerrorsoccurssl,usedtosetupanSSLconnectiontotheserver(outofthescopeofthistutorial)

PerformaSELECTqueryNowyouarereadytoperformanSQLqueryonthedatabase.Thequeryonceexecutedwillinvokeacallbackfunctionwhichcontainsaneventualerror,theresultsandthefields.

connection.query('SELECT*FROMtodos',(error,todos,fields)=>{

if(error){

console.error('Anerroroccurredwhileexecutingthequery')

throwerror

}

console.log(todos)

})

Youcanpassinvalueswhichwillbeautomaticallyescaped:

constid=223

connection.query('SELECT*FROMtodosWHEREid=?',[id],(error,todos,fields)=>{

if(error){

console.error('Anerroroccurredwhileexecutingthequery')

throwerror

}

console.log(todos)

})

WorkingwithMySQL

185

Topassmultiplevalues,justputmoreelementsinthearrayyoupassasthesecondparameter:

constid=223

constauthor='Flavio'

connection.query('SELECT*FROMtodosWHEREid=?ANDauthor=?',[id,author],(error,

todos,fields)=>{

if(error){

console.error('Anerroroccurredwhileexecutingthequery')

throwerror

}

console.log(todos)

})

PerformanINSERTqueryYoucanpassanobject

consttodo={

thing:'Buythemilk'

author:'Flavio'

}

connection.query('INSERTINTOtodosSET?',todo,(error,results,fields)=>{

if(error){

console.error('Anerroroccurredwhileexecutingthequery')

throwerror

}

})

Ifthetablehasaprimarykeywith auto_increment,thevalueofthatwillbereturnedintheresults.insertIdvalue:

consttodo={

thing:'Buythemilk'

author:'Flavio'

}

connection.query('INSERTINTOtodosSET?',todo,(error,results,fields)=>{

if(error){

console.error('Anerroroccurredwhileexecutingthequery')

throwerror

}}

constid=results.resultId

console.log(id)

)

Closetheconnection

WorkingwithMySQL

186

Whenyouneedtoterminatetheconnectiontothedatabaseyoucancallthe end()method:

connection.end()

Thismakessureanypendingquerygetssent,andtheconnectionisgracefullyterminated.

WorkingwithMySQL

187

DifferencebetweendevelopmentandproductionLearnhowtosetupdifferentconfigurationsforproductionanddevelopmentenvironments

Youcanhavedifferentconfigurationsforproductionanddevelopmentenvironments.

Nodeassumesit'salwaysrunninginadevelopmentenvironment.YoucansignalNode.jsthatyouarerunninginproductionbysettingthe NODE_ENV=productionenvironmentvariable.

Thisisusuallydonebyexecutingthecommand

exportNODE_ENV=production

intheshell,butit'sbettertoputitinyourshellconfigurationfile(e.g. .bash_profilewiththeBashshell)becauseotherwisethesettingdoesnotpersistincaseofasystemrestart.

Youcanalsoapplytheenvironmentvariablebyprependingittoyourapplicationinitializationcommand:

NODE_ENV=productionnodeapp.js

Thisenvironmentvariableisaconventionthatiswidelyusedinexternallibrariesaswell.

Settingtheenvironmentto productiongenerallyensuresthat

loggingiskepttoaminimum,essentiallevelmorecachinglevelstakeplacetooptimizeperformance

ForexamplePug,thetemplatinglibraryusedbyExpress,compilesindebugmodeifNODE_ENVisnotsetto production.Expressviewsarecompiledineveryrequestindevelopmentmode,whileinproductiontheyarecached.Therearemanymoreexamples.

Expressprovidesconfigurationhooksspecifictotheenvironment,whichareautomaticallycalledbasedontheNODE_ENVvariablevalue:

app.configure('development',()=>{

//...

})

app.configure('production',()=>{

//...

})

app.configure('production','staging',()=>{

//...

Differencebetweendevelopmentandproduction

188

})

Forexampleyoucanusethistosetdifferenterrorhandlersfordifferentmode:

app.configure('development',()=>{

app.use(express.errorHandler({dumpExceptions:true,showStack:true}));

})

app.configure('production',()=>{

app.use(express.errorHandler())

})

Differencebetweendevelopmentandproduction

189