ESPM5295

16-Oct-17

1

IntroductiontoLiDAROurgoalsherearetointroduceyoutoLiDARdata,toshowyouhowtodownloaditforanareaofinterest,andtobetterunderstandthedataandusesthroughsomesimplemanipulations.Inourprojectwe’llbecalculatingbuildingandtreeheightsforyourprojectarea.ThestateonMinnesotahasflownLiDARfortheentirestate.DatamaybeinteractivelydownloadedviatheMNDNRMNTopoviewer(http://dnr.state.mn.us/maps/mntopo/index.html),orforlargerareasviatheMNGEOftpsite(ftp://ftp.lmic.state.mn.us/pub/data/elevation/lidar/).RasterDEMs,hillshades,contours,andthe“raw”dataor“pointcloud”datamaybedownloaded,inESRIgeodatabasesorincommonlysupportedformats. WewilldownloadLiDARdataandderivedlayersandpreparethemforinclusioninourCampusGIS.Wewillfirstworkwiththeprocessedlayers,andlatertherawdata,primarilyforbuildingheights.WeneedtomergetogetherseveraltilesforthecompleteSt.PaulCampus. ThisiscommonwhenworkingwithLiDAR. Ourindividualstudyareasaresmall,sowe’veonlyafewtiles.OpentheMnTOPOwebsite,andfamiliarizeyourselfwiththetoolsneartheupperright.Usethedefaultdisplayandthezoombuttonintheupperleft,andclick-hold-dragtopan,togettheapproximateCampusareadisplayed.First,tousethedownloaddatatool(diskdrivewitharrowicon,inthetoolbarrow,upper/left),specifyasmallpolygonthatiscenteredonyourindividualprojectarea.Downloadonlyonetileforeachofthefollowing:

• 1mdem,• 1mhillshade• therawlidarpointclouddata(thesewillbe.lasor.laz(acompressedform)

You’llseetheDEMandhillshadearerasters,andtherawlidarpointdataareinfileswitha.lazextension.Thecodesaretilenumber,tiledandcompressedtoa.lazformat.

Weonlyhaveyoudownloadasmallbitofdatasothatyougetabitofexperiencewiththetool.Thismighthelpyourememberitsexistencelater,ifyouneeditonanotherproject.However,youmaybeputinaratherlengthyqueue,sotoavoidwaiting,we’vedownloadedalltherelevantdatafortheclassproject.ItisontheLdrive,inthe5295directory(notthe5295share),underLiDAR.YoushouldcopytheDEMandhillshade,andtherelevantlaz/lasfilesfromtheLdrivedirectorytoanewdirectory,inthisexamplewe’llcallit\LASData.Notethatyoudon’tneedallthe.laz/.laspointfilesthatMnTOPOdistributes,butonlyasubsetoftiles,dependingonyourstudyarea.Youshouldrefertotheimagebelow,andonlycopythefilesyouneedforyourprojectarea.Theboundariescorrespondtothetileborders,andthelabelsthe.laz/.lasfilenames.Alsonote,yourworkingdirectoryshouldNOTbeonanetworkdrive,e.g.,theclassLdrive.LASfilesareoftenquitelarge,andtheoverheadofcopyingandthenwritingbacktothedriveacrossanetworkisoftentooslow.Besttoworkonalocaldrive,thatis,thedriveonyourcomputer,ora

ESPM5295

16-Oct-17

2

USBdrivepluggedintooneoftheports.

DisplaytheDEMandhillshadesurfaces.Thesearelargelyfromanautomatedprocessthatfirsttriestoidentifythe“bareearth,”orgroundonlyreturns,andthenbuildaDEMandahillshade.Noticetheoddshapes,sometimesliketriangles,onthehillshadesurfacewithinthebuildingfootprints.Theseareartifactsoflidardataprocessing.Thealgorithmshaveahardtimeidentifyingthegroundnearbuildingsperfectly,sothereareoftentriangularshapesonornearbuildings.Youalsoseearoughnessandoccasionalsmallbumpsovergrassorforestareas,thesearealsoartifactsoftheprocessing.

AlsonoticetheDEMrastersareinUTM15NAD83(1986).YouwillneedtotransformthemtoCORS96ifyouwishtoavoidthewarningsaboutdatumconflictswhendisplayedwithyourotherprojectdata.

YoucandoallthefollowingstepsinArcGIS,wewillbeusingLASTools,awidely-used,generalpurposelidarprocessingtoolboxthatismuchfasterthanprocessinginArc,andquiteflexible.TheUMNhasasitelicenceforLASTools,sowe’llbeusingitforthissmallexercise.Youshouldmaketwosub-directorieswithinyour\LasDatadirectory,onenamedbuildings,andonenamedtallveg,intowhichwe’llbesubsettingthelasdata.BecauseLASToolsismodular,andyouwanttobeabletoworkathomeoronothercomputers,wecancopythespecifictoolswe’llbeusingtothedirectorywithourdata.FindtheLASToolsdirectoryontheL:\5295directory,thenthe\binsubdirectory,cancopythelas2las.exeandlas2shp.exefilestoyourlocalworkingdirectory.Alsocopythecorresponding.txtfilesthatcontainthedocumentationforeachofthesetoolsWhenyou’redone,yourdirectoryshouldlooksomethinglikethiswindowbelow.

ESPM5295

16-Oct-17

3

YounoticeI’veconvertedcopiesofthe.lazto.lasfiles.Youdon’tneedto,butnoticethe.lazfilesaremuchsmallerthanthe.las.Becauselidardataareoftensolarge,thereisusuallyatradeoffbetweensavingspaceandprocessingspeed.We’reworkingwithasmallarea,soitdoesn’tmuchmatter,butinmanyrealprojectsitdoes.AlsonotethatIonlyshowasubsetofthe.lasfiles,yoursetwilldependonyourstudyarea.YoushouldknowafewmorethingsaboutLiDARdata.Remember,theseLiDARarefrompulsesoflightsentdownfromaplane,thenreflectedbackfromatarget.Precisegeometryallowsoperatorstodeterminethex,y,andzcoordinatestowithinafewinchesofeachobjectthatreflectsbackapulse.Therecanbemanyreturnsmeasuredfromeachpulse,althoughwe’remostofteninterestedinthefirstreturns,tomeasureobjectheights,andthelastreturns,tomeasuregroundelevation.Thedataareprocessedbeforethey’redelivered,andthecollectingorganizationusuallyappliesalgorithmstotrytodetectgroundhits(notalllastreturnsarefromtheground),objectheights(notallfirstreturnsarefromthetopsofobjects),andtoprovideotherinformationabouteachdatapoint.Mosttimeseachreturnpointcontainsinformationonthereturnnumber(first,last,orinbetween),thefeatureclassification(2=ground,4=mid-heightvegetation,5=highvegetation,6=building,9=water,0=unclassified),aswellasthescanangle,thestrengthofthereturnsignal,andotherinformationaboutthepoint.LASTools,andmostotherlidarprocessingtools,allowustoaccessthesedata.We’llusethecommandprompttorunLAStools.ThecommandpromptisfoundviatheWindowsStartbutton(pinkarrow,right),theninthe

ESPM5295

16-Oct-17

4

Accessoriesfolder–>CommandPrompt(greenarrowandcircled,atright)InWindows10,asinthelab,youclickonthesomewhatsimilariconinthelowerrightcorner,andtypeCMDinthesearchwindow,thenclickontheCMDcommandthatpopsup.Thiswillopenasmallwindowthatonlyallowsonlytextentry(seebelow).

Thepromptshowsthecurrentpathstring.Wanttosetthepathtoyourdatalocation.Firsttypeinthedriveletter,inmycasetheYdrive,soItypeinthedriveletterandthenhitreturntochangetothatdrive:Y:

Thepromptshouldnowshowthedrive,inmycase:Y:\IthenusetheCD(ChangeDirectory)commandtochangetheactivedirectory,typingin:CDdownloads\lasdataSothepromptnowbecomes:Y:\downloads\lasdataIcantypeDIR(fordirectory)andthenhitreturn,itshouldlistthedirectorycontents.IcancheckitagainsttheWindowsExplorerviewofthedirectory,showatthebottomofpage2,toverifyI’matthecorrectsubdirectory.Now,ontoprocessing.LASToolsisabitoldschool,youdon’thaveaccesstoaGUI,youspecifyoptionsthrough“flags”onacommandline.EachflaginLASToolsstartswithadash,e.g.,“-i”meansthefollowingtextspecifiestheinputfile(s).Thisisabitcumbersomeforsinglefileprocessing,butitisgreatfor

ESPM5295

16-Oct-17

5

writingsmallprogramstodorepetitivetasks.Anotherkeybitofinformationisthattheasterisk*standsfor“matcheverything,”andhelpswhenspecifyingmultiplefiles.Ifyou’vecopiedtheLASToolsprogramsfromtheCLDdrive,andareworkingathomeoranothernon-Skok35computer,you’llhavetotypesomethinglikethefollowingline,andenter:setPATH=%PATH%;c:\yourdirectory\lastools\binwherec:\yourdirectoryisthepathtothedirectoryonyourcomputerthatcontainsthelastoolsdirectory.Ifyou’reworkingin35Skok,ignorethis.Thefirstcommandwe’lltypeis:Las2las–i*.laz–odirbuildings–keep_class6Thisusesthelas2lastoolto:

• Readallthe.lazfilesasinput(-imeansinput,the*.lazusesthewildcardsymbol*tomatchanythingthatcomesbeforea.lazextension).NOTEHERE:IFI’vealreadyuncompressedmy.lazfilesto.lasfiles,IHAVETOSPECIFY-i*.lasastheinput

• Outputthemtothedirectorynamedbuildings(-odirbuildings),and• Onlykeeptheclasslabeled6(-keep_class6),inthiscasethepointsclassifiedasbuilding

pointsYoumaygetacommentsayingtheuseofLasToolsisunauthorized,ignoreit.Wedohavealicense,butsettingituprequiresafewmoresteps,andthechangestheymightwarnaboutareminor,anddon’tchangeouroutcome.WecanuseCDchangetothatbuildingsdirectoryafterthecommandruns,andverifythecommandworked:

Noticethefileshavethesamenameastheoriginals(that’swhyweputtheminaclearlylabeled

ESPM5295

16-Oct-17

6

sub-directory),butthatthey’remuchsmaller,thesubsetbuildingfileis16,337,316bytes(16.3Mb),whilethefulldatasetwasalmost166Mb;we’vesampledoutonlythebuildingreturns,andthefileislessthan10%oftheoriginalsize.Wecanapplyasimilarcommandtospecifykeepingonlythevegetationclasses(3,lowveg,4,mediumtrees,and5,hightrees):Las2las–i*.laz–odirtallveg–keep_class345Notetheoutputdirectoryissettotallveghere.Youshouldcheckyourselftomakesurethecommandworked,andnotethedecreaseinsize.ConverttheselaspointstoESRIshapefileswiththelas2shpcommand.Toconverttypein:las2shp–ibuildings\*.las–single_pointsandhitreturn.Again,youmaygetawarningthatyou’reunauthorized,andthey’readdingsmallerrorstothedata,youcanignorethemhere.Thecommandaccessesthelas2shpcommand(las2shp),tellsittouseall.lasfilesinthebuildingsdirectoryasinput(–ibuildings\*.las),andtooutputsingle-pointshapefiles(-single_points),notmultipointfiles,whichclusterpointsinblocks,reducingfilesize,butencumberingaccess.Verifytheabovecommandworked,firstbylookinginthebuildingsdirectoryandcheckingthatshapefile(s)appeared,andthenbydisplayingthedatainanArcMapview:Youcanseethepoints,hereinyellow,moreorlessmatchthebuildinglocations.Theslightmismatchisbecauseofbuildingleanintheaerialphotographs.Youmightrememberthatorthophotoscorrectalllocationsasiftheywereontheground,andthetopsofbuildingsaren’t.Thus,thetopsgetdisplaced,usuallyoutwardfromtheimagingcenterrelativetothecameraposition.

ESPM5295

16-Oct-17

7

That’swhyskyscrapersseemtoleaninmost“vertical”orthophotos.Nowweshouldlookatheights.First,weneedtocreateavariabletoholdoureventualheight,measuredfromthegroundtothetopoftheobject.Openthetableforeachofyourpointshapefiles,andaddanewfieldnamedzHeight,doubleorfloat,precision12,scale1.Thismaytakesometimeforeachtile,youcouldtrytousetheAddFieldcommandasanArcMapbatchjob.Youarestronglyencouragedtodothisandthefollowingoperationsasbatchjobs(seecourseresources,oraskAndyorPaulforhelp).Thiswillgreatlyeaseyourwork.Next,weneedtoextractheightsfromthepoints.ItisusuallymoreefficienttodoitasabatchjobwiththeAddGeometryAttributestool,specifymetersandsquaremetersforunits,andourprojectcoordinatesystem.Notethatthistoolwillalsoaddthexandycoordinatesforeachpoint.Whentheprocessisfinished,usetheidentifybuttontoqueryafewbuildingheights(zvalues).Remember,thevaluesareinmeters.Dothesevaluesmakesense?Arethebuildingsover200meters(650ft)tall?Whydoyouthinkyou’regettingthesevalues?TheZvaluerepresentstheorthometricheightofthetopofthebuilding.Thisistheheightrelativetoourstandardsurface,nearsealevel,andnotrelativetothelocalgroundsurface.Weneedtosubtractthelocalgroundsurfacefromthebuildingzvaluetogetbuildingheight.OnewayistoquerytheDEMrasterbeloweachpoint,andaddittothetablerecordforeachpoint.WecanthensubtracttheDEMheightfromthezvalueswecalculatedfromtheAddGeometryAttributes,andgetabuildingheight.TheExtractValuestoPointstool,foundatArcToolbox->SpatialAnalystTools->Extract->Extractvaluestopointsdoesjustthis.Specifyingtheinputandoutputswillqueryeachcellbeloweachpoint,andaddacolumntothenewdatalayerspecified.Thismaytakequitealongtime,sobepatient,andagain,thisprocesscanberunasabatchjobformultiplefiles.

ESPM5295

16-Oct-17

8

ThisgeneratesanewcolumnnamedRASTEVALthatcontainsthegroundelevationsnearateachpoint:(ignorethezheightvalueinthetableatleft,thisisfromatestrun,inyourfilesitwillbezeroornull/notassigned)Nowyoucancalculatethebuildingheightsforeachpoint.Youcandothismanually,butitisbettertodoitasabatchjobwiththeCalculateFieldcommand.Createanewfieldcalledsomethinglikebuild_hghtfortheheightabove-groundsampledateachpoint,andthenbatchjobthefieldcalculation.Whenyouaredone,youshouldhaveatablethatlookssomethingliketheonetotheright:Now,calculatetheaverageheightfrompointsamplesforoneofyourbuildings.Thereareseveralwaystodothis,anditwouldeasiestiswiththeSpatialJointool.However,itdoesn’tworkmuchofthetime,freezingup,orworse,returningerroneousanswers,withnowarningthatthevaluesarewrong.Thereisanalternativewhichappearstoworkinmostcases,thatinvolvesmultiplesteps.FirstapplytheIntersecttool,withtheinputbuildingpolygons,andtheprocessedLiDARpointfile.First,weneedpolygonsforabuildingslayer,withIDsyou’veassigned(typicallyshortinteger),andforclarity,abuildingname.Youshouldalreadyhavecreatedthisbuildinglayeraspartofyourdatabase.YoumayhavetodisplayboththeLiDARpointsandanimageofthestudyareawithyourbuilding,andadjustthebuildingfootprintpolygonabittoaccountforbuildinglean.

ESPM5295

16-Oct-17

9

Afterensuringyouhaveaproperbuildingsfile,readtheIntersecttool’sdocumentation.StarttheIntersecttool,andspecifyyourbuildingandprocessed(heightscalculated)LiDARpoints.IfyouspecifyAllforthejoinattributes,itwilljointheattributesbyspatiallocation,intoapointfile,withthebuildingassignedtoeachpoint.

Wecanseetheattributetableforthepointfeaturesbelow.Thelastthreecolumnsarefromthebuildingslayer,withthebuildingFID,ID,andname,andallthecolumnsbefore,fromFID*throughRASTERVAL,arefromthepointfile,includingtheheightforeachLiDARpointthatlandedonthebuilding.

ESPM5295

16-Oct-17

10

Nowweneedtoaggregatethepointsforeachbuilding.TheeasiestwayisthroughtheSummaryStatisticstool.Thissummarizedfeatures,andIcanassign“case”attributes.Thiswillcalculatethesummarystatisticsforeverydifferentvalueofthecaseattribute.IfImakethebuildingIDmycaseattribute,itwillcreateatablewithsummarystatisticsforeachuniquebuilding.Thetoolisshownbelow:HereI’vespecifiedmyintersectedbuilding/LiDARpointsetastheinput,anoutputtable,andthefield,statistics,andcaseattribute.Iamcalculatingthemeanheightforthebuilding.YoumightargueIwantthetallestpointforabuilding,butmostroofsdon’tvarymuchinheight,andIwasconcernedthatabuildingmighthaveatowerorscaffoldontop,andhencegetthealargeerror.Fortreecanopies,amaximumstatisticwouldprobablymakemoresense.Runningthiscreatesanoutputtable:NotethereisameanheightforeachcaseofthebuildingID.Icanthensimplyjointhistomybuildinglayerby

ESPM5295

16-Oct-17

11

theIDcolumn,andcopy/savethejoinedfiletogetmyheightforeachbuilding.Therearesomenuancesinapplying.Multi-leveledbuildingsmayhavetobesplitintothevariouspieces,andtheheightscalculatedseparatelytomakemuchsense.Ifthebuildingfootprintsaretoolarge,ortherearemanyerrorsinclassification,thentheremaybeabitofbiasdownward.Generally,theseerrorsappeartobesmallinourapplication,andthismeasurementisgoodenoughforouruse.Performtheseprocessingstepsforboththebuildingsandthecanopydatalayer,sothateachfeaturehasaLiDARheight.Asnotedearlier,youshouldprobablycalculatethemaximumstatisticinforthecanopydatalayer,andthemeanstatisticsforthebuildingslayer,butevaluateanddecideforyourself.TurninamapofshowingyoursubsetLiDARdataforyourprojectarea,yourbuildingpolygon,andlabeleachbuildingwithyourcalculatedaveragebuildingheights.Useanimageforthebackground,eitheroneofthehighresolutionimagesontheclassdrive,oroneoftheWMSlayers.Includetheusualtitle,legend,scale,andotherstandardmapelements.Dothissameexerciseforthetreesinyourprojectarea.Again,createandturninasecondmapofyourresults,withanimagebackground,thetreepolygons,labeledbyheight.