omnidirectional human intrusion detection … human intrusion detection system 415 busses, etc.) and...
TRANSCRIPT
411
Chapter 16
Omnidirectional Human Intrusion Detection System Using Computer Vision Techniques
Wai Kit Wong, Chu Kiong Loo, and Way Soong Lim
Contents16.1 Introduction....................................................................................................................41216.2 HumanIntruderSurveillanceSystem.............................................................................414
16.2.1BurglarAlarmSystem..........................................................................................41416.2.1.1PassiveInfraredMotionDetectorSystem..............................................41516.2.1.2UltrasonicMotionDetectorSystem......................................................41516.2.1.3Glass-BreakDetectorSystem.................................................................41616.2.1.4PhotoelectricBeamSystems...................................................................41716.2.1.5VibrationSensorSystem........................................................................41716.2.1.6PassiveMagneticFieldDetectionSystem...............................................41716.2.1.7MicrophonicDetectionSystem..............................................................41816.2.1.8TautWirePerimeterSecuritySystem.....................................................418
16.2.2Radar-BasedHumanIntruderDetectionSystem.................................................41916.2.3 ImageProcessing-BasedHumanIntruderDetectionSystem.............................. 420
16.2.3.1VisionSpectrumImageProcessing-BasedHumanIntruderDetectionSystem.......................................................421
16.2.3.2NightVision/InfraredSpectrumImageProcessing-BasedTrespasserDetectionSystem.................................................................................. 423
K13920_C016.indd 411 1/4/2013 7:24:10 PM
412 ◾ Effective Surveillance for Homeland Security
16.1 IntroductionHomelandsecurityisaneffortbygovernment(normallyparkedundernationaldefensedepart-ment)topreventterroristattack inacountryandreduceacountry’svulnerabilitytoterrorism[1].Thescopesofhomelandsecurityonhumantrespasserdoincludetheprotectionofacriticalinfrastructure’sperimeterandthebordersecurity(countryborderofterritorialland,water,andairspace).Homelandthreatsrefertothecrimesthathaveanimmediateandvisibleimpactonthelocalcommunityandaffectcitizenqualityoflife.Inthefaceofunknownfutureterroristthreats,illegal immigrants that will flush out a peaceful and economically stable country as refugees,whichmightbringinthefts,smugglers,etc.,issues,however,nationalsecuritydepartmentandconvergentsecurityengineerswillhavetodevelopstrategiesandsecurity/surveillancesystemtofulfilltherequirementofhomelandsecurity,ontrespassers’threats.Thischapterproposessomehomelandsecuritysystemsonhumanintruderdetection.
Intrusiondetectionistheactofdetectingatrespasserinaguardzone.Humanintrusiondetec-tionsystemisasystemusedtodetecthumantrespasserenteringaprohibitedarea.Conventionalhumanintrusiondetectionsystemusesburglaralarmsystem(activeorpassivesensors),wherebymodernhumanintrusiondetectionsystemappliescomputervisiontechniques,bothtotraceoutwhetherthereisanexistenceofatrespasser/humanbeingornotinaprohibitedarea.Themainmeritofmodernhumanintrusiondetectionsystemascomparedtoconventionalhumanintru-siondetectionsystemisthattheimageprocessing-basedhumanintrusiondetectionsystemcanhelpcapturepictures.Sincepicturesarecaptured,therearemorechancesoftheintrudersbeingrecognizedandcaught.Theauthoritycanwiselyplacesecuritycamerasineveryvulnerableplaceoftheguardedarea,indoorandoutdoor,thatwouldbeaccessibletoahumanintruder.Thisallowstheauthoritytostaysafeinsidethepremisewhilestillbeingabletoseewhatishappeningintheoutdoorareaofthepremise.Italsogivesthemmoretimetocallforhelporbackupiftheynoticeanysecuritythreat.
AQ1
AQ2
16.2.4DirectionalversusOmnidirectionalViewing...................................................... 42316.2.4.1OpticalApproachversusMechanicalApproach................................... 42416.2.4.2VisionSpectrum-BasedOmnidirectionalSurveillanceSystem............. 42716.2.4.3Thermal/InfraredSpectrum-BasedOmnidirectional
SurveillanceSystem.............................................................................. 42816.3 UnwarpingMethods.......................................................................................................431
16.3.1DiscreteGeometryTechniqueMethod............................................................... 43216.3.2Pano-MappingTableMethod............................................................................. 43316.3.3Log-PolarMappingMethod................................................................................43516.3.4PerformanceEvaluation...................................................................................... 436
16.4 AutomaticHumanIntruderDetectionAlgorithm......................................................... 43916.4.1PartitionedRegionofInterestAlgorithm............................................................ 43916.4.2HumanHeadCurveTestAlgorithm.................................................................. 44116.4.3ExperimentalResults.......................................................................................... 447
16.4.3.1ExperimentalResultsforPartitionedROI-BasedHumanIntruderDetectionAlgorithm................................................. 448
16.4.3.2ExperimentalResultsforHumanHeadCurveTestAlgorithm.............45016.4.4ComparisonbetweenTwoProposedHumanIntruderDetectionAlgorithms.....452
16.5 ConclusionandFutureResearchDirections...................................................................453References................................................................................................................................454
K13920_C016.indd 412 1/4/2013 7:24:10 PM
Omnidirectional Human Intrusion Detection System ◾ 413
Ingeneral,themoderncomputervisiontechnique-basedhumanintrusiondetectionsystemcanbedividedintotwomaincategories:oneisvisionspectrum-basedhumanintrusiondetectionsystemandanotheroneisnightvision/infrared(IR)spectrum-basedhumanintrusiondetectionsystem.Visionspectrum-basedhumanintrusiondetectionsystemappliesvisionspectrumrangeimagingtoolstocaptureimages.Oneproblemencounteredinthistypeofsurveillancesystemisthechangeinambientlight,especiallyinanoutdoorenvironmentwherethelightingconditionvariesnaturally.Thismakestheconventionaldigitalcolorimagesanalysistaskinsmartsurveil-lancesystembecomeverydifficult.Onecommonapproachtoeliminatethisproblemistotrainthesystemtocompensateforanychangeintheillumination[2].However,thisisgenerallynotenoughforhumanintrusioninthedark.Itisbettertoapplysomesortofnightvisionimagingtoolthathelpsinimagingobjectsinthedark.ThencomestheapplicationofIRspectrum-basedhumanintrusiondetectionsystem.Thermalcameraisanexcellentnightvisionsecuritycamera.ItperceivesIRradiationanddoesnotneedasourceofillumination.Thermalcameraisidealforanylow-lightareas,notjustforthenighttime.Itproducesanimageinthedarkestofnightsandcanviewthroughlightfog,rain,andsmoke.Thermalimagingcamerasmakesmalltemperaturedifferencesvisible.Thermalimagingcamerasarecurrentlyappliedwidelyinmanyneworexistingsecuritynetworks.
Ifa single imagingtool is tomonitora singleangleofa location, thenformore locationsindifferentanglesofview,more imagingtoolsarerequired.Hence, itwillcostmore,besidescomplicatingthesurveillancenetwork.Therefore,anomnidirectionalhumanintrusiondetec-tionsystemusingminimumhardwareisdevelopedtoovercomethecostandnetworkcomplica-tionproblems.Themethodappliedtoobtainomnidirectionalimagescanbeclassifiedintotwoapproaches[3]:(1)mechanicalapproachand(2)opticalapproach.Sincemechanicalapproachleads to many problems on discontinuity and inconsistency, therefore, optical approach wasfavoredbypractitioners.
Thecapturedomnidirectionalimagesnormallyhavesomedifferentpropertiescomparingtoperspectiveimagesintermsofimagingdeformation.Suchdistortionleadstotheimagesbeingdif-ficulttobedirectlyimplemented.Thus,itisnecessarytoworkoutanefficientmethodtounwarptheomni-image.Unwarping,generally,isamethodusedindigitalimageprocessingin“opening”up an omnidirectional image into a panoramic image, making the information on the imagetobeablefordirectimplementation.Unwarpingmethodisactivelyadoptedintheapplicationofvisual surveillance systems.Thereare currently threeunwarpingmethods activelypracticedaroundtheworld,whicharethepano-mappingtablemethod[4],discretegeometrytechniques(DGT)method[5],andlog-polarmappingmethod[6].Thischapterstudiestheadvantagesanddisadvantagesofeachmethod,andtheirperformanceiscomparedandevaluated.
Conventionalsurveillancesystemnormallyemployshumanobserverstoanalyzethesurveil-lancevideo.Sometimesthisismorepronetoerrorduetolapsesinattentionofthehumanobserver[7].Itisafactthatahuman’svisualattentiondropsbelowacceptablelevelswhenassignedtovisualmonitoring,andthisfactholdstrueevenforatrainedpersonnel[8,9].Theweaknessinconven-tionalsurveillancesystemhasraisedtheneedforasmartsurveillancesystemwhereitemployscomputerandpatternrecognitiontechniquestoanalyzeinformationfromsituatedimagingtoolsandautomaticallydetectatrespasser[10].Twoautomatichumanintrusiondetectionalgorithmsarediscussed in this chapter; this includes partitioned regionof interest (ROI) algorithm [11]andhumanheadcurvetestalgorithm[12,13].Withthealgorithmsproposedinthischapter,itissimpletodetectthehumanintrusionofmorethanonelocationinasingleviewcapturedbytheimagingtool.Thesemonitoringandsubsequentanalysesoftheimagesfromtheinspectioncanalertsecuritypersonneltotakefurtheractiontoeithercatchorhustlethetrespassereffectively.
AQ4
AQ3
K13920_C016.indd 413 1/4/2013 7:24:10 PM
414 ◾ Effective Surveillance for Homeland Security
Inthischapter,thefundamentalsofhumanintrusiondetectionsystem,classicalburglaralarmsystem(activeorpassivesensorsystem)andradarsystemversuscomputervisiontechniquesystem,visionspectrumimagingversusIRimagingsystem,anddirectionalversusomnidirectionalview-ing,arefirstdiscussed.Thealgorithmandimplementationofsomeuniversalunwarpingmethodswill be discussed too, such as discrete geometric transforms (DGTs) [4], pano-mapping tablemethod[5],andlog-polarmappingmethod[6]proposedintransformingthecapturedomnidi-rectionalimagesintopanoramicform,providingobserverorimageprocessingtoolsawideangleof view.Besides that, automatic human intrusiondetection is implemented in the omnidirec-tionalimagingsystems(bothinvisionspectrumandinIRspectrum,respectively).ThedevelopedhumanintrusionalgorithmsarepartitionedROIalgorithm[11]andhumanheadcurvetestalgo-rithm[12,13],andtheirdesignprocedureswillbeincludedhere.Later,someexperimentalresultstoprovethealgorithmsproposedforthehumanintrusiondetectionsystemareshown.Inthelastsectionofthischapter,wesummarizetheworkandenvysomefutureenhancement.
16.2 Human Intruder Surveillance SystemAccording to tort law,property law,andcriminal law[14], ahuman intruder is apersonwhocommitstheactoftrespassing/intrudingonaprohibitedarea,thatis,withoutthepermissionoftheauthority.Ahumanintrudertrespassestoacriticalinfrastructure’sperimeterandthebordersecurity isdefinedas “an intentional interferencewith the infringeontonational security thatproximatelywillcauseinjury,vandalism,terrorism,theft,etc.”InUnitedKingdomjurisdictions,trespassing has been codified to clearly define the scope of the remedy, and in most jurisdic-tions,trespassingremainsapurelycommonlawremedy,thescopeofwhichvariesbyjurisdiction.Surveillanceisthemonitoringoftheactivities,behavior,orotherchanginginformation,normallywithpeopleinasurreptitiousmannerandattheentranceofprohibitedarea.Surveillanceisveryusefultosecurityauthoritytorecognizeandmonitorthreatsandpreventcriminalactivity.
Humanintrudersurveillancesystemcanbeusedtohelpsecurityauthorityguardacriticalinfrastructure’sperimeterandthebordersecurity.Itisdesignedtodetectanintrusion,activateawarningdeviceupondetectionofanintrusion,determinecrime,protectlifeandproperty,bringanappropriate response toanemergency,andenhance theapprehensionofcriminals.Humanintruder surveillance systemcanbedivided into threemaincategories,whichare theconven-tionalburglar alarmsystem, the radar-basedhuman intruderdetection system, and the imageprocessing-basedhumanintruderdetectionsystem.
16.2.1 Burglar Alarm SystemBurglar(orintrusion)alarmsystemsareelectronicalarmsdesignedtoalerttheusertoaspecificintruder.DetectionsensorsareconnectedtoacontrolunitviaanarrowbandRFsignalorlow-voltagewiringthatisusedtocommunicatewitharesponsedevice.Newconstructionsystemsare predominately hardwired for efficient, more economical hardware installation. Refurbishconstructionoftenapplieswirelesssystemsforafaster,moreeconomicalchannelinstallation,duetoneednotdiggingwall,ceiling,andfloorforrewiring.Somesystemsserveasinglepur-poseofeitherburglarorfireprotectionandsomecombinationsystemsprovidebothfireandintrusion protections. Systems range from small, self-contained noisemakers to complicated,multi-zonedsystemswithcolor-codedcomputermonitoroutputs.Manyoftheseburglaralarmsystemconceptsalsoapplytoportablealarmsystemsforprotectingmotorvehicles(cars,trucks,
K13920_C016.indd 414 1/4/2013 7:24:10 PM
Omnidirectional Human Intrusion Detection System ◾ 415
busses,etc.)andtheircontents.Burglaralarmsystems(orintrusiondetectionsystems,perimeterdetection systems, perimeter security systems, perimeter protection systems, andmanymoretermsfortheidenticalitem)aredividedintomanytypes,suchaspassiveIR(PIR)motiondetec-torsystem,ultrasonicmotiondetectorsystem,glass-breakdetectorsystem,photoelectricbeamsystem,vibrationsensorsystem,passivemagneticfielddetectionsystem,microphonicsystem,and taut wire perimeter security system. Each of these burglar alarm systems will be brieflyillustratedinthefollowing.
16.2.1.1 Passive Infrared Motion Detector System
APIRsensor isanelectronicsensorthatmeasuresIRlightradiatingfromobjectswithinitsfieldofview.PIRsensorsareoftenusedintheconstructionofPIR-basedmotiondetectors.ThePIR-basedmotiondetectorisoneofthemostcommondetectorsfoundinhouseholdandsmallbusinessenvironmentsbecauseitoffersaffordableandreliablefunctionality.Theterm“passive”meansthedetectorisabletofunctionwithouttheneedtogenerateandradiateitsownenergy.Thisisdifferentfromultrasonicandmicrowavevolumetricintrusiondetectorsinwhichtheyare“active” inoperation.IfanIR-emittingobjectexists inthecoveragearea, thePIR-basedmotiondetectorisabletoidentifybyfirstlearningtheambienttemperatureofthemonitoredspaceandthendetectingachangeinthetemperaturecausedbythepresenceofthatobject.Applyingthedifferentiationprinciple(creatingindividualzonesofdetectionwhereeachzonecomprisesoneormorelayers)canachievedifferentiation.Betweenthezones,thereareareasofnosensitivity(deadzones)thatareusedbythesensorforcomparison,thatis,acheckofpres-enceornon-presence;PIR-basedmotiondetectorcanverifywhetheranintruderorobjectisactuallyinplace.
InaPIR-basedmotiondetector,thePIRsensoristypicallymountedonaprintedcircuitboardcomprisingtherequiredelectronicsusedtointerpretthesignalsfromthepyroelectricsensorchip[15].Thecompleteassemblageisconfinedwithinahousingattachedinasitewherethesensorcanviewtheareatobemonitored.IRenergyisabletoreachthepyroelectricsensorthroughthewin-dowbecausetheplasticusedistransparenttoIRradiationbutsomehowtranslucenttovisiblelightspectrum.Thisplasticsheetalsoinhibitstheintrusionofdustand/orinsectsfromobscuringthesensor’sfieldofviewand,inthecaseofinsects,fromgeneratingfalsealarms.SomemechanismshavebeenusedtoconcentratethedistantIRenergyontothesensorsurface.
ThePIR-basedmotiondetectorworkingasahumanintruderdetectionsystemhasthemeritsofsimpleandlowerinstallationcostandlesssensitivetoilluminationchanges.However,PIR-basedmotiondetectorhasthesedemeritswhenworkingasahumanintruderdetectionsystem:(1)itcanbeeasilytriggeredbymovinganimals,blowingshrubs,etc.;(2)itcannotdetectpeoplewhoarestationary,thusmayleadtoalargenumberoffalsealarms;(3)itsoutputishighlybursty(somecommercialoff-the-shelfsensorsuseaheuristicsolutiontomakeupforthis,byignoringdetectionsthatfallwithinarefractoryperiodofanearlierevent.Theseissuesarelargelyignoredby thevastmajorityofPIR-basedresearchby limiting their systemtosingle-personscenariosand/orassumingpeoplearealwaysmoving);and(4)itdoesnottoleratelargeareasorlargetem-peraturechanges.
16.2.1.2 Ultrasonic Motion Detector System
Thetransmitterof theultrasonicdetector is radiating anultrasonic signal into the areaundersurveillance.Theultrasonicsoundwavesarereflectedbysolidobjects(suchasthesurrounding
AQ5
AQ6
K13920_C016.indd 415 1/4/2013 7:24:11 PM
416 ◾ Effective Surveillance for Homeland Security
walls,floor,andceiling)andthendetectedbythereceiver.Sinceultrasonicwavesaretransmit-ted through air, the hard-surfaced objects tend to reflect most of the ultrasonic energy, whilesoftsurfacestendtoabsorbmostenergy.Thereceivedfrequencywillbeequaltothetransmittedfrequencywhenthesurfacesarestationary.However,achangeinfrequencywilloccurasaresultoftheDopplerprinciple,duetoapersonorobjectmovingtowardorawayfromthedetector.Thiseventwillinitiateanalarmsignal.
Ingeneral,ultrasonicmotiondetectorcanbecategorizedintotwotypes:activeandpassive.Activeultrasonicmotiondetectoremitsultrasonicsoundsthatareinaudibletohumanear(fre-quenciesbetween15and75kHz).Passiveultrasonicmotiondetectorconsistsofonlyreceiversthatsimplyreceivetheemittedsounds.Asthesedevicesareoneofthemostsensitiveamongthehumanintruderdetectionsystems,theyarealsoexpensiveincost.
Ultrasonicmotiondetector systemsutilizeadvanced technology,but somehowunder someconditions,theyarepronetofalsealarmsbystuffslikepassingbirdsorinsects,gustsofwind,orvibrationscausedbyairplanespassingoverhead.Passiveultrasonicmotiondetector systemsdonotprovidecompletedetectioninareaswithlargeobjects,thuscreatinga“dead”zone.Hence,inthesecases,anothertypeofdetectionsystemmayberequiredtoworktogethertheultrasoundsystemasaseconddetectorformoreaccuratealarm.Duetoitspooreffectiveness,thistechnologyisconsideredobsoletebymanyalarmprofessionalsandisnotactivelyinstalled.
16.2.1.3 Glass-Break Detector System
Aglass-breakdetectorisasensorusedinelectronicburglaralarmsystemsfordetectingifthereisapaneofglassshatteredorbroken.Thesedetectorsarecommonlyplacednearglassdoorsorglassstorefrontwindowstodetectwhetherthereisanintrudertryingtobreaktheglassandenterthepremises.Glass-breakdetectorsnormallyapplyamicrophone,whichmonitorsanynoiseorvibra-tionscomingfromtheglass.Ifthevibrationsexceedacertainthreshold(userselectable/preset),theyareanalyzedbydetectorcircuitry.Simplerdetectorsjustsimplyapplynarrowbandmicrophonesthattunedtofrequenciestypicalofglassshatteringandreacttosoundabovecertainthreshold,whereasmorecomplexdesignswillcomparethesoundanalysistooneormoreglass-breakprofilesusingsignaltransformssuchasdiscretecosinetransformorfastFouriertransformandreactifboththeamplitudethresholdandstatisticallyexpressedsimilaritythresholdarebreached.
Theglass-breakdetectorcanbeappliedforinternalperimeterbuildingprotection.Whenglassbreaks, itactuallycreates sound inawidebandof frequencies ranging frominfrasonic (below20Hz, this frequency range is inaudible tohumanear) to theaudioband (20Hz to20kHzthatisaudibletohumanear)rightuptoultrasonic(whichisabove20kHzandagainitfallsinrangeinaudibletohumanear).Therearetwotypesofglass-breakdetectorsingeneral:glass-breakacousticdetectorandseismicglass-breakdetector.Glass-breakacousticdetectorsaremountedincloseproximitytotheglasspanesandlistenforsoundfrequenciesassociatedwithglassbreaking.Seismicglass-breakdetectorsaredifferentinthattheyareinstalledontheglasspane.Whenglassbreaks,itproducesspecificshockfrequenciesthattravelthroughtheglassandoftenthroughthewindow frame and the surrounding walls and ceiling. Typically, the most intense frequenciesgeneratedarebetween3and5kHz,dependingonthetypeofglassandthepresenceofaplasticinterlayer.Seismicglass-breakdetectorssensetheseshockfrequenciesandgenerateanalarmcon-ditionaccordingly.
However,glass-breakdetectorscanonlybeappliedatlimitedareas,forexample,building/premiseswithwindows.Glass-breakdetectorsarealsosensitivetoenvironmentaleffect,suchas
AQ7
AQ8
K13920_C016.indd 416 1/4/2013 7:24:11 PM
Omnidirectional Human Intrusion Detection System ◾ 417
waterspreading/rain,insects,birds,orobjecthittingwindows(butwindownotclashing)andcansometimesgeneratefalsealarm.
16.2.1.4 Photoelectric Beam Systems
PhotoelectricbeamsystemsdetectthepresenceofanintruderbytransmittingvisibleorIRlightbeamsacrossanarea,wherethesebeamsmaybeobstructed.PhotoelectricbeamsensorstransmitabeamofIRlighttoaremotereceivercreatingan“electronicfence.”Thesesensorsareoftenusedto“cover”openingssuchasdoorwaysorhallways,actingessentiallyasatripwire.Oncethebeamisbroken/interrupted,analarmsignalisgenerated.Photoelectricbeamsystemconsistsoftwomaincomponents:atransmitterandareceiver.Thetransmitterusesalight-emittingdiode(LED)asalightsourceandtransmitsaconsistentIRbeamoflighttoareceiver.Thereceiverconsistsofaphotoelectriccellthatdetectswhenthebeamispresent.Ifthephotoelectriccellfailstoreceiveatleast90%ofthetransmittedsignalforasbriefas75ms(timeofanintrudercrossingthebeam),analarmsignalisgenerated.
Thephotoelectricbeamsystemworkingasatrespasserdetectionsystemhasthemeritsofeasyinstallationandhighimmunitytoambientlight,anditsfunctionalityisnotaffectedbyelectricalandmagneticfields.However,photoelectricbeamsystemhasthesedemeritswhenconsideredtobeworkedasatrespasserdetectionsystem:(1)highinstallationcostwithtwodeviceshavingtobemounted,wired,andadjustedand(2)detectionofverysmallobjects;thismaysomehowleadtoalargenumberoffalsealarmsduetoanimals,passingobjectsblownbywinds,orevenpassinginsectstriggeringanalarm.
16.2.1.5 Vibration Sensor System
Vibrationsensorsrelyonanunstablemechanicalconfigurationthatformspartoftheelectricalcircuit.Theworkingoperationforvibrationsensoristhatwhenmovementorvibrationoccurs,theunstableportionofthecircuitmovesandbreaksthecurrentflow;thisleadstoanalarm.Thetech-nologyofthedevicesisvaryingandcanbesensitivetodifferentlevelsofvibration.Themediumtransmittingthevibrationmustbecorrectlyselectedforthespecificsensorsastheyarebestsuitedtodifferenttypesofstructuresandconfigurations.
Vibrationsensorsareveryreliablesensors.Itgenerateslowfalsealarmrateintrespasserdetec-tionsystemandmoderateinpricerange.However,thistypeofdetectionsystemmustalwaysbefencemounted.Also,vibrationsensorsareanewtechnologywithanunprovenrecordasopposedtothemechanicalsensor,whichinsomecaseshasafieldrecordinexcessof20years.That’swhyitisnotwidelyseeninthemarketyet.
16.2.1.6 Passive Magnetic Field Detection System
Thistypeofburiedsecuritysystemisbasedonthemagneticanomalydetection(MAD)principleofoperation.TheprincipleoftheMADisbasedontheabilitytosensetheanomalyintheEarthmagneticfieldproducedbythetarget[16].Thesystemappliesanelectromagneticfieldgeneratorpoweredbytwowiresrunninginparallel.Bothwiresrunalongtheperimeterandareusuallyinstalledabout5in.apartontopofawallorabout12in./30cmbelowground.Thewiresareconnectedtoasignalprocessorthatanalyzesanychangeinthemagneticfield.Passivemagnetic
AQ9
K13920_C016.indd 417 1/4/2013 7:24:11 PM
418 ◾ Effective Surveillance for Homeland Security
fielddetectionsystemhasverylowfalsealarmrate.Itcanbeputontopofanywallandhasveryhighchanceofdetectingrealburglars.However,itishavinghighinterferenceifitisinstallednearhigh-voltagelinesorradars.
16.2.1.7 Microphonic Detection System
Microphonic-baseddetectionsystemshaveavarietyofdesign,butallaregenerallybasedonthedetectionof a trespasser attempting tocutor climbover a chainwire fence.Themicrophonicdetectionsystemsareusuallyinstalledassensorcablesattachedtorigidchainwirefences.Oneexampleisthemicrophonicfencedisturbancesensorsystem.Microphonicfencedisturbancesen-sorsapplythesignalsgeneratedbytheminuteflexingoftriboelectriccoaxialsensorcable,whichareanalyzedbypowerfulsignalprocessorstodetectthesoundassociatedwithcutting,climbing,orliftingthefencestructure.Thesystemscanalsobeembeddedwithaspecialaudiochannelthatenablessecuritiesto“listen”toactivityalongeachzoneofthefencefortheprotectionofexistingfencesandstructuresagainstcutting,climbing,orlifting.Theycanalsobefittedtocoiledrazorwirefences.
Inoperation,microphonicfencedisturbancesensorsystemsaredesignedtodetectandanalyzeincomingelectronicsignalsreceivedfromthesensorcableandthentogeneratealarmsfromsig-nalsthatexceedsomepresetconditions.Thesystemsofferadjustableelectronicstoallowinstallerstochangethesensitivityofthedetectors’alarmtosuitaspecificenvironmentalcondition.Thetuningofthesystemisnormallyaccomplishedbeforethedetectiondevicesareputincommis-sioning.Microphonicfencedisturbancesensorsystemsareverycheapincost,easytoinstall,andsimpleinconfiguration.However,thesesystemshaveahighrateoffalsealarmsbecausesomeofthesesensorsmightbetoosensitivetoextremeweather,contactby largeanimals,badlymain-tainedfences,andovergrownvegetation.
16.2.1.8 Taut Wire Perimeter Security System
Atautwireperimetersecuritysystemisnormallyastreamoftensionedtripwiresusuallymountedonawallorfence.Itisparticularlyusefulfordetectingclimbing(ontopofawall)orwhereitisnecessarytobuildupaphysicalbarrier(fence).Thissystemisdesignedtodetectanyphysicalattempttopenetratethebarrier.
Tautwireperimetersecuritysystemcanoperatewithavarietyofdetectorsorswitchesthatdetectmovementateachendofthetensionedwires.Thesedetectorsorswitchescanbeanelec-tronicstraingauge,astaticforcetransducer,orasimplemechanicalcontact.Falsealarmscausedbybirds and animals canbe avoidedby tuning thedetectors to omit objects that exert smallamountsofpressureonthewires.However,thistypeofsystemisvulnerabletotrespassersdig-gingunderthefence.Hence,aconcretefootingisinstalleddirectlybelowthefencetopreventsuchtrespassing.Tautwireperimetersecuritysystemsarehavingveryreliablesensors,lowrateoffalsealarms,andhighrateofdetection.However,thistypeoftrespasserdetectionsystemisveryexpensive,itiscomplicatedtoinstall,andthetechnologyisquiteancient.
Ingeneral,conventionalburglaralarmsystemsaresimpleandlowerininstallationandmain-tenancecost.However,theyhavealowerprobabilityofdetectinghumanintrudersandhighfalsealarmrate.Thisisduelargelytomanyuncontrollablefactors,suchasenvironmentalissues(rain,ice, wind, standing water), random animals, and human activities, as well as other electronicinterferencesources.
K13920_C016.indd 418 1/4/2013 7:24:11 PM
Omnidirectional Human Intrusion Detection System ◾ 419
16.2.2 Radar-Based Human Intruder Detection SystemThesecondcategoryofhumanintrudersurveillancesystemisradar-basedhumanintruderdetec-tionsystem.Radarisradiodetectionandranging,whichisanobjectdetectionsystemthatappliesradiowavestodeterminetherange,attitude,direction,andspeedofanobject.Inhumanintru-siondetectionsystem,radarcanbeusedtodetecthumanintruder.Theradardishorantennatransmitspulsesof radiowavesormicrowave thatwill bounceoffanyobject in theirpath.Aradar’scomponentconsistsof(1)atransmitterthatgeneratestheradiofrequencysignalwithanoscillatorandcontrolsitsdurationbyamodulator,(2)awaveguidethatbondsthetransmitterandantenna,(3)aduplexerthatactsastheswitchamongtheantennaandtransmitterorthereceiverforthesignalwhentheantennaisusedinbothsituations,(4)areceiverthatknowstheshapeofthedesiredreceivedsignal(so-calledapulse),and(5)anelectronicsectionthatcontrolsallthosedevicesandtheantennatoperformtheradarscanorderedbysoftware.
For theworkingprinciplesof radar, the transmitterwill emit radiowaves (radar signal) inpredetermineddirections.Whenthesesignalscomeincontactwithanobject,theyusuallyreflect/scatterinmorethanonedirection.Theradarsignalsarereflectedbacktowardthetransmitter.Inhumandetection(movingobjectdetection),iftheobjectismovingeithercloserorfartheraway,thereisaslightchangeinthefrequencyoftheradiowaves.Dopplerradarisoneofsuchcommonperimetermonitoringsystems.However,thiskindofradarsystemrequiredthecoverageareatobeclearoffoliageandobstaclesthatmightcreatecoverageshadowsandfalsealarm.Thisrequirementmightnot suitmanyoutdoorenvironments,andeven though in indoorusage, itmightcreateundesirableinstallationandmaintenanceexpenses.Also,slow-movingtargetssometimesmightnotbedetectedonthisradarsystemduetolow-resolutiondetectableDopplershift[17].InRef.[17],engineersareprovingthattheultra-wideband(UWB)RFcanovercomethedeficienciesonconventionalDopplerradar.Someotherrecentadvancedradarsystemsalsodevelopedforhome-landsecurityincludedtheReutechRadarSystem[18]andtheHARRIERGroundSurveillanceRadar(GSR)[19].
ReutechRadarSystemdevelopedandlaunchedtheSpiderRSR940inJuly2009.ThefigureofSpiderRSR940 is shown inFigure16.1. It is ahighlymobile land-based,360°continuous
Figure 16.1 Spider RSR 940. (From http://www.rrs.co.za/products/homeland-security.html) AQ10
K13920_C016.indd 419 1/4/2013 7:24:11 PM
420 ◾ Effective Surveillance for Homeland Security
scanningsurveillance radar that iscapableofdetectingsurfaceandair targets. Italsoprovidessector scanningsurveillanceused fordetectionand identificationof slow-movingsurface-basedtargetssuchashumanbeings,smallboats,andevenhelicopters.Inadditiontothelocalcontrolstation,aremotecontrolandmonitoringcapabilityisalsoprovidedfortypicalapplicationinclud-ingthebordercontroloperations,monitoringofcoastaltraffic,coastlinecontrol,andmonitoringofprivateandunattendedairfield.Theradarsystemhasaninstrumentedrangeupto40km.Ontheotherhand,DeTect’sradarprocessingtechnologydevelopedtheHARRIERGSRinyear2011,asshowninFigure16.2.HARRIERGSRusesstate-of-the-artsolid-stateDopplerradartechnol-ogyavailable inS-,X-,andcombinedS/X-bandradarfrequencies.Solid-stateradartechnologyappliedinHARRIERGSRofferssignificantincreasedperformance,longerusefullife,andlowermaintenancecostoverconventionalmagnetron-basedsystems.HARRIERGSRuseshigh-speedscanningforenhancedsmalltargetdetectioninhigh-clutterenvironmentssuchasdevelopedareas,terrain,andhighseastates.Italsohasautomaticdetectionandtrackingcapabilitiesandincludesuser-definedmonitoringandalarmzones.HARRIERGSRisofferedinfixedandmobileconfigu-rationsandcanbelinearlynetworkedtocoverlargeareassuchasbordercrossingsandcoastlines.
Ingeneral,radar-basedhumanintruderdetectionsystemisexpensiveinhardware,sinceradarantennas(transmitterandreceiver),duplexer,waveguides,andelectronictoolsarerequiredtobesetup,anditalsorequiresspecificoperatingsoftware.Besides,thetargetmightbedetectedbytheradar,buttheirtypesorclassisnotknown.Itcanbesometimesvehicles,animals,ormovingobjects,whichgivefalsealarm.Also,somenarrowbandradarsmightsometimesfinddifficultiesinhavinginsufficientrangeresolutiontodiscriminatebetweenasmallernearbytargetandalargerlonger-rangetarget.
16.2.3 Image Processing-Based Human Intruder Detection SystemThe third category of human intruder surveillance system is image processing-based humanintruderdetectionsystem.Itisbymeansofusingimagetotraceoutwhetherthereisanexistenceof trespasser/humanintruderornot.Imageprocessing-basedhumanintruderdetectionsystem
Figure 16.2 HARRIER GSR. (From http://www.detect-inc.com/security.html)
K13920_C016.indd 420 1/4/2013 7:24:12 PM
Omnidirectional Human Intrusion Detection System ◾ 421
iswidelyfavoredbymanypractitionersascomparedtoburglaralarmsystemsandradar-basedhumanintruderdetectionsystem,mainlyduetothesefourreasons:
1.Ithelpscapturepictures.Theappliedsecuritycameraisagreattooltocaptureapictureoftheburglar/terroristwhentheyaretryingtotrespass/breakintoaprohibitedterritory.Thisisveryimportantbecausethesecuritycameragivestheauthoritiessomethingthattheycanusetohelpthemidentifytheburglar/terrorist.Havingasecuritysystemthattriggersanalarmisessential,butwithoutsecuritycameras,theauthoritieswillneverknowwhoorhowmanyburglars/terroriststriedtogetintotheirterritory.
2.Morechanceoftheburglars/terroristsbeingcaught.Whentheauthoritiesareabletoviewthepicturesfromthecamerastoidentifytheburglars/terrorists,itprovideshigherpossibleratefortheburglars/terroriststogetcaught.Inmanyoccasionswhenauthoritiesarriveattheirincidentsite,theburglars/terroristswillbelonggone.Withoutthecapturedpictures,thereisnotmuchchancefortheburglars/terroristsbeingcaught.
3.Securitycamerasaregreatpreventiontools.Theyaresomethingthatallburglars/terroristswilllookthroughbeforetheydecidetobreakinto/trespassaterritory.Mostoftheburglars/terroristswillnotevenattemptaterritoryiftheydetecttheexistenceofsecuritycamerasbecausetheyknowthatthisisgoingtoworkagainstthemandcausethemtogetcaught.Burglars/terrorists areknown for avoiding territory thathasgood security, especially theones that aremonitoredwith security cameras.Thecameraspose toobigof a threat forthem,sotheywillmoveontoatargetthatdoesn’thavegoodsecurity.
4.Securitycamerascansecurevulnerableareas.Whenauthorityisinsidetheinfrastructure’sperimeterandneedstoseewhatishappeningoutsideofthenearbybuildingforsecurity,securitycamerasarethebestwaytoachievethatgoalsafely.Theauthoritycanwiselyplacesecuritycamerasineveryvulnerableareaofhisorherbuilding,indoorsandoutdoors,thatwouldbeaccessiblebyaburglar/terrorist.Thisallowstheauthoritytostaysafeinsidehisorherbuildingwithhisorherprotectedpersonwhilestillbeingabletoseewhatishappen-ingintheoutdoorareaofthebuilding.Italsogivesthemmoretimetoseekforhelpiftheynoticeanysecuritythreat.
Ingeneral,theimageprocessing-basedtrespasserdetectionsystemcanbedividedintotwomaincategories:oneisvisionspectrumimageprocessing-basedhumanintruderdetectionsystemandanotheroneisnightvision/IRspectrumimageprocessing-basedhumanintruderdetectionsystem.
16.2.3.1 Vision Spectrum Image Processing-Based Human Intruder Detection System
Visionspectrumimageprocessing-basedhumanintruderdetectionsystemcanbedividedintotwocategories:oneisanalogvideosurveillancesystemandtheotheristhedigitalvideosurveil-lancesystem.
16.2.3.1.1 Analog Video Surveillance System
Datingbacktoasearlyas1965,analogvideosurveillancewasfirstbegunwithsimpleclosedcircuittelevision(CCTV)monitoring.TheU.S.pressreportssuggestingpolicealreadystartusingsurveil-lance cameras inmonitoringpublicplaces’ security. In1969,police installed sets of surveillancecamerasinNewYorkCityatthemunicipalbuildingnearthecityhall.Thispracticelaterspreadsto
AQ11
K13920_C016.indd 421 1/4/2013 7:24:12 PM
422 ◾ Effective Surveillance for Homeland Security
othercitiesintheUnitedStateswithCCTVsystemsandkeepsaneyebypoliceofficersatalltimes.CCTVistheapplicationofvideocamerastosendimagesignaltoaspecificlocation,onalimitedsetofmonitors.ThefirstCCTVappliedinpublicplaceswascrude,conspicuous,lowdefinition,andinblackandwhitesystemsthatareunabletopanorzoomintoparticularview.Inmoderndays,CCTVsystemsapplysmaller-sizeandhigh-definitioncolorvideocamerasthatcanfocustoresolveminutedetailandalsocanlinkthecontrolofthevideocamerastoacomputer.Thismakestheobjectstobetrackedsemiautomatically.Thistechnology,so-calledvideocontentanalysis(VCA),iscurrentlyusedbyalargenumberoftechnologicalcompaniesaroundtheworldtoenablethesystemstorecognizewhetheramovingobjectisawalkingperson,acrawlingperson,ananimal,avehicle,etc.
However, in themid-1990s, the emergingof digital technologyhas superseded the analogtechnologyinvideosurveillancesystem.Digitalmakesvideosurveillanceclearer,faster,andmoreefficient. Digital video surveillance has made complete sense as the price of digital recordingdroppedwiththecomputerrevolution.Insteadofchanginganalogvideotapesdaily,thedigitaluserscouldnowreliablyrecordamonth’sworthofsurveillancecontentsonharddrivebecauseof itshighcompressioncapabilityand lower storagecost.Thedigitally recorded imagesare somuchclearerthantheanalog-recordedimages.Thisleadstotherecognitionprocessimmediatelyimprovingforpolice,privateinvestigators,andotherusersthatusevideosurveillanceforidenti-ficationpurposes.Byusingdigitaltechnology,theimagescouldalsobemanipulatedtofurtherimproveclaritybyaddinglight,enhancingtheimage,zoominginonframes,etc.
16.2.3.1.2 Digital Video Surveillance System
Digitaltechnologyisadatatechnologythatusesdiscretevalues[20].Digitalvideoisatypeofvideorecordingsystemapplyingdigitaltechnology.Thereisabroadrangeofdigitalvideosurveil-lancecamerasavailableinthemarket:
◾ Fake security cameras:thesecameraslooksimilartothosesurveillancecamerasavailableinthemarket,buttheyarenotactualcameras.Theyhavenorecordingcapability.Thesecam-erascanactasdeterrentcamerastoscareburglars/theft.Ifsomethinghappens,theywillnothavearecordsincetheyhavenorecodingcapability.
◾ Covert surveillance cameras: these cameras look like regular items, to hide its identityasasurveillancecamera, forexample,awallclockinashop,afacingfrontdoorteddybear,andapottedplantattheshop’scorner.Eachoneofthemcouldveryeasilyembedasurveillancecamera.Thesurveillancecamerascanrecordthescenesanytimewithoutanybodyknowingitsexistence.
◾ Wireless security digital cameras:thesesurveillancecamerasareeasytoinstallandremoved,areoftensmallinsize,havenowiringconnectionseen,andoffermoreflexibilityinsetup.Thesecamerastransmitimagesignalswirelesslytoacenterhubthatareshownonamonitorscreeninamonitoringroom.
◾ Wired surveillance digital cameras: these surveillance cameras arewired and lackflexibilityinsetup.Theyareappropriateforpermanentsetup.Thesecamerastransmit imagesignalsthroughawiretoacenterhubthatareshownonamonitorscreeninamonitoringroom.
◾ Home surveillance cameras:thesecamerascomeinapackagethatoftenincludessomeextrafeaturessuchastimersforlamps,motionsensors,andautomaticgatedoorlock.
Oneproblemencounteredinmostvisionspectrumimageprocessing-basedsurveillancesystemsisthechangeinambientlight,especiallyinanoutdoorenvironmentwherethelightingcondition
AQ12
AQ13
AQ14
AQ15
K13920_C016.indd 422 1/4/2013 7:24:12 PM
Omnidirectional Human Intrusion Detection System ◾ 423
variesnaturally.Thismakestheconventionaldigitalcolorimageanalysistaskinsmartsurveil-lanceverydifficult.One commonapproach to alleviate thisproblem is to train the system tocompensateforanychangeintheillumination[2].However,thisisgenerallynotenoughforatrespasserinthedark.Itisbettertoapplysomesortofnightvisionimagingtoolsthatcanhelpimagingobjectsinthedark.
16.2.3.2 Night Vision/Infrared Spectrum Image Processing-Based Trespasser Detection System
Nightvisionistheabilitytoseeinadarkenvironment.Nightvisionismadepossiblebyacombi-nationoftwoapproaches:(1)sufficientspectralrangeand(2)sufficientintensityrange.Humanbeingshavepoornight vision ability compared tomany animalsbecausehumaneyes lack anelement,so-calledthetapetumlucidum.Thetapetumlucidum[2]isalayeroftissueintheeyeofmanyvertebrateanimals,whichliesimmediatelybehindorsometimeswithintheretina.Itreflectsvisible lightback through the retina, increasing the light available to thephotoreceptors.Thisimprovesvisioninlow-lightconditionsbutcancausetheperceivedimagetobeblurryfromtheinterferenceofthereflectedlight.Thetapetumlucidumcontributestothesuperiornightvisionofsomeanimals.Manyoftheseanimalsarenocturnal,especiallycarnivoresthathunttheorganismatnight,whileothersaredeepseaanimals.
Thermalcameraisanexcellentnightvisionsecuritycamera.ItperceivesIRradiationanddoesnotneedasourceofillumination.Thermalcameraisidealforanylow-lightareas,notjustforthenighttime.Itproducesanimageinthedarkestofnightsandcanviewthroughlightfog,rain,andsmoke.Thermalimagingcamerasmakesmalltemperaturedifferencesvisible.Thermalimagingcamerasarecurrentlyappliedwidelyinmanyneworexistingsecuritynetworks.
16.2.4 Directional versus Omnidirectional ViewingIn spite of the availability of many modern sophisticated surveillance monitoring products inthe market, majority of the systems have the limitation in the viewing angle of the camera.Omnidirectionalpromptstotheconceptoftheexistenceinalldirection,with360°areacoverageonasingleplane/axis.Inimagingpointofview,anomnidirectionalvisualizationhasvisualizationcapabilityofa360°fieldofviewaroundthehorizontalplaneorwithvisualfieldthatcoverstheentiresphere.Omnidirectionalvisualizationsystemisimportantinareasthatneedlargevisualfieldcoverage,suchasinpanoramicimagingandinrobotics.Aconventionalimagingtoolnor-mallyhasafieldofviewwiththerangeofafewdegreestomaximumof180°.Itcancaptureonlyasemisphereimagewithlightfallingontotheimagingtool’sfocalpoint.However,ontheotherhand,anomnidirectional imagingtoolcancapture light fromalldirections (surrounded360°fieldofview)fallingontoitsfocalpoint,coveringafullsphere.
Convergentsecuritysystemsaresecuritysystemsthatintegrateintrusion,holdup,fire,videosurveillance,accesscontrol,andmonitoringapplicationsinphysicalsecuritysystemsandITinfra-structures. However, the current convergent security systems apply digital CCTV monitoringsystems, inwhichthecoverageareaisdirectional.Eventheycandoit inomnidirectional,butitrequiresmorehardware.Conventionalapproachestoobtainpanoramic(wideview)imageforanomnidirectionalviewmainlyconsistofcombiningsnapshotscapturedseparatelyintoasingleandcontinuousimage.Thiscombinationofimagesiscomputationallyintensivesometimes.AnexampleisbyusingaRANSACiterativealgorithm[21]tocombinethesnapshots.RANSACisanabbreviationfor“RANdomSampleConsensus.”ThisalgorithmwasfirstpublishedbyFischler
AQ16
AQ17
K13920_C016.indd 423 1/4/2013 7:24:12 PM
424 ◾ Effective Surveillance for Homeland Security
andBollerin1981andfoundtobeusedinsolvingthecorrespondingproblem(partsofanimagecorrespondtopartsofanotherimage;aftertheimagingtoolhasmoved,timehaselapsedorthefocusingobjectsmovedaround)andcalculatesthefundamentalmatrixcorrespondingtoapairofstereoimagingtools.RANSACcanestimatetheparameterswithahighdegreeofaccuracybutwithlimitationthatthereisnoupperboundonthetimeittakestocomputetheseparameters.Toprocessanimage,itissometimestimeconsumingorendless.Evenifanuppertimeboundisused(setwithamaximumnumberofiterations),theresultsobtainedmaynotbetheoptimalone.Itmaynotbeonethatgeneratestheimageinagoodway.
Besidescomputational intensive, thecombiningof images to formapanoramic imagealsodependsonthequalityandconsistencyofthesnapshotsused.Thesnapshotimagesmighthaveanumberofdeficienciesthatwillfurtherimpairthequalityoftheoutputpanoramicimage.Incomparison,anomnidirectionalimagingtoolcanbeusedtocreatereal-timepanoramicart,with-outpost-processingrequirement,andsomehowwillprovidemuchbetteroutputqualityimage.
In robotics and computer vision, omnidirectional imaging tools are widely used in visualodometry [22] and also help solve the simultaneous localization and mapping (SLAM) [23]problemsvisually.Visualodometryistheprocessofdefiningthepositionandorientationofarobotbyanalyzingthecapturingimagesfromtheattachedimagingtools,whereasSLAMisatechniqueappliedbyautonomousvehiclesandmobilerobotstoformamapwithinanunknownenvironmentor toupdate amapwithin a known environment and in themeantimekeepon tracking theircurrentlocation.Duetotheomnidirectionalvisualization’sabilitytoobtaina360°view,roboticandcomputervisiontaskscanhavebetterresultsforopticalflowandinfeatureselectionandmatching.
Besidespanoramicart,robotics,andcomputervision,applicationofomnidirectionalvisual-izationalsoincludessurveillance,inwhichitisimportanttocoveralargevisualfield,andtele-conferencing,inwhichitisofgreatinteresttocoverasmanyparticipantsaspossibleinthesameimage,andyetthereareunlimitedapplicationsthatwillbediscoveredinthefuturesoon.Nextsectionwillfurtherdiscussthemethodsappliedtoacquireomnidirectionalviews.
16.2.4.1 Optical Approach versus Mechanical Approach
Omnidirectionalvisualizationpossessessignificantapplicationpotentialsintheareassuchaspan-oramicart,mobilerobotnavigationandcomputervision,qualitycontrol,andsurveillance.Themethodsappliedtoobtainomnidirectionalimagescanbeclassifiedintotwoapproaches[3]:
1.Mechanical approach:themethodofgatheringimagestogenerateanomnidirectionalimage 2.Optical approach:themethodofcapturinganomnidirectionalimageatonce
Inaddition,theyareclassifiedintotwocategoriesbytheviewpointoftheimage[3]:singleview-pointandmultipleviewpoints.
For the mechanical approach, the images captured on a single viewpoint are continuous.Oneexampleistherotatingcamerasystem[24–27].Insuchasystem,thecamerarotatesaroundthecenteroftheprojection.Itgeneratesanomnidirectionalimagefromasingleviewpoint.Theproperorderofimagesobtainedbyrotationis joinedtogethertoacquireapanoramicviewforthescene.AnexampleofrotatingcameraisshowninFigure16.3.Arotatingmotorisrequiredtorotatethevideocamerainordertoscantheomnidirectionalview.However,sinceitisnecessarytorotateavideocamerainafullcircleinordertoacquireasingleomnidirectionalimage,itisimpossibletogeneratereal-timeomnidirectionalimage.Otherdisadvantagesofrotatingcamerasystemarethatitrequirestheuseofmovingpartsandprecisepositioning.Theimagecaptured
AQ18
K13920_C016.indd 424 1/4/2013 7:24:12 PM
Omnidirectional Human Intrusion Detection System ◾ 425
atmultipleviewpointsisrelativelyeasytoconstruct,asshowninFigure16.4.Asinglecameraormorecamerasareappliedtogathermultipleimagesatmultipleviewpointsandcombinethemintoanomnidirectionalimage.QuickTimeVRsystem[28]adoptedsuchtechnologiesandhasmanymarketapplications.However,theimagesgeneratedbythesystemarenotalwayscontinuousandconsistent,anditalsocannotcapturethedynamicsceneatvideorate.
Sincemechanicalapproachleadstomanyproblemsondiscontinuityandinconsistency,opti-calapproachwasinuse.Thisapproachismostappropriateforreal-timeapplicationsanditisofsingleviewpoint.Thistypeofapproachneednotusemotorsandinthemeantimeitcancaptureomnidirectionalimageatonce,noextracombinationworkrequired,anditisfast.Twoalterna-tiveshavebeenproposed,namely,theuseofspecial-purposelens(suchasthefish-eyelens[29])andtheuseofhyperbolicopticalmirrors[30].Fish-eyelens,asshowninFigure16.5,areusedtoreplaceaconventionalcameralensthathaveveryshortfocallengththatallowsthecameratoviewobjectsasmuchasinahemispherescene.Fish-eyelenshavebeenwidelyusedforwide-angleimagingareasasnotedinRefs.[31,32].However,Nalwa’sworksinRef.[33]foundoutthatitisdifficulttodesignfish-eyelensthatensurethatall incomingprincipalraysintersectatasinglepointtoyieldafixedviewpoint.Theacquiredimageusingfish-eyelensnormallydoesnotpermittheconstructionofdistortion-freeperspectiveimagesoftheviewedscene.Hence,tocaptureanomnidirectionalview,thedesignofoptimalfish-eyelensmustbequitecomplexandlarge,andtherefore,theyareexpensiveincost.Besides,accordingtoRef.[34],therelativeilluminationforafish-eyelensdesigniswidelyvarying.Inaddition,theexistencesofdistortionacrossthehemi-sphericalfieldofviewneedtobeinconsiderationwhendesigningagoodqualityfish-eyelens.Sincefish-eye lensareexpensiveandcomplexindesignandalmostprovidethesamereflectivequalityashyperbolicopticalmirror,hyperbolicopticalapproachisplannedtoadapt.
AQ19
Figure 16.3 Rotating camera. (From Stun-ningsales.com, Weatherproof CCD color rotating video security camera, redirecting from http://www.stun-ningsales.com/homethings/outdoor_securitycameras.htm)
K13920_C016.indd 425 1/4/2013 7:24:13 PM
426 ◾ Effective Surveillance for Homeland Security
Figure 16.5 Fish-eye lens. (From The-Digital Image.com, Fisheye lens, redirecting from http://www.the-digital-picture.com/Reviews/)
Figure 16.4 Multiple cameras system. (From Chen, S.E., Quick time VR: An image-based approach to virtual environment navigation, in Proceedings of the 22nd Annual ACM Conference on Computer Graphics, Los Angeles, CA, pp. 29–38, 1995.)
K13920_C016.indd 426 1/4/2013 7:24:14 PM
Omnidirectional Human Intrusion Detection System ◾ 427
16.2.4.2 Vision Spectrum-Based Omnidirectional Surveillance System
The proposed vision spectrum-based omnidirectional surveillance system model is shown inFigure16.6.
Inthismodel,omnidirectional imagesofanobservedscenearecapturedusing thecombi-nation of a web camera (webcam) and a specific design hyperbolic optical mirror. MATLAB®inthe laptopcomputerwillperformunwarpingonthe imagescaptured intopanoramic form.Thehuman intruderdetection algorithm that is programmed inMATLABwill thenbeusedtoprocess thepanoramic images todetect thepresenceofhumanintruder. Ifhumanintruderisdetected,alarmwillbesignaledandportionsofsuspectedimagewithhumanintruderwillbestoredinadatabaseforfurtheridentificationpurposes.
ThesurveillancecamerasetusedinthissurveillancesystemconsistsofawebcameraandanattachedspecificdesignhyperbolicopticalmirrorasshowninFigure16.7.
Laptopcomputer
Alarm
Webcamspecificdesign
hyperbolicmirror
+
Figure 16.6 Omnidirectional surveillance system model.
Figure 16.7 Surveillance camera set (web camera and specific design hyperbolic optical mirror).
K13920_C016.indd 427 1/4/2013 7:24:15 PM
428 ◾ Effective Surveillance for Homeland Security
ThewebcamerausedinthesurveillancesystemisE3500PlusQuickCambyLogitech.Itisadomesticwebcamerathatcaptureshigh-qualityVGA(640×480)videosand1.3megapixel(soft-ware-enhanced)images.ThewebcameraissmallinsizeandcheapcomparedtodigitalandCCTVcamerawithfine-resolutionoutput.ThedigitalcontrolisalsoaccomplishedthroughtheUSBportconnectedtoalaptoporpersonalcomputer(PC)viaplugandplayonWindowsXPorVista.ItcanbeinterfacedwithMATLABtoo.Therefore,itisbestoutfittedinomnidirectionalsurveillancesystem.
Thespecificdesignhyperbolicopticalmirrorusedintheomnidirectionalsurveillancesystemisasmall-sizewide-viewtype,withouterdiameterof40mmandangleofview30°abovehorizon-talplanemanufacturedbyACCOWLEVISION.Themirrorcanreflecta360°viewsurroundedbyitself,andasthewebcameraplugsonit,omnidirectionalimageswithinaguardedperimetercanbecapturedandsenttoalaptopcomputerinamonitoringroomtobeprocessedforsurveil-lancepurpose.A custom-madebracket that is shown inFigure16.8 is designed to attach thehyperbolicmirrortothewebcameraviaasocket.
Alaptopcomputercanbeusedforimageprocessingofanobservedroom.Acore2duolaptopcomputerwithspecs1.83GHzprocessorand2GBDDR2RAMwithMATLABver.7.0ischo-sentobeusedhere.MATLABhasadataacquisitiontoolboxinterfacethatenablesattainmentofvideosandimagesthroughtheE3500PlusQuickCam.Asetofconnectingspeakerstothelaptopcomputersoundsthealarmifhumanintruderisdetected.
16.2.4.3 Thermal/Infrared Spectrum-Based Omnidirectional Surveillance System
Oneproblemencounteredinmostsurveillancesystemsisthechangeinambientlight,especiallyinoutdoorenvironmentwherethelightingconditionisnaturallyvarying.Thismakesthevideoanalysistaskinsmartsurveillanceverydifficult.Onecommonapproachtoalleviatethisproblemistotrainthesystemtocompensateforanychangeintheillumination.However,thisisgenerallynot enough forobject trackingandmonitoring in thedark. In recent times, severalmanufac-turershavecomeupwithhighlysophisticatedthermalcameraforimagingobjectsinthedark.ThecamerausesIRsensorsthatcaptureIRcomingfromdifferentobjectsinthesurroundingand
Figure 16.8 Front view of the custom-made bracket.
K13920_C016.indd 428 1/4/2013 7:24:15 PM
Omnidirectional Human Intrusion Detection System ◾ 429
formanIRimage.SinceIRradiationfromanobjectisduetothethermalradiation,theimageformationwilldependontheobjecttemperatureandnotonthelightreflectedfromtheobject.Hence,suchcameracanbeconvenientlyusedfornightvision.Thepresentstateoftheartevenallowsthermalcameratocaptureobjectevenfromaverylongdistance.
Ifasinglethermalcameraistomonitorthesecurityofasinglelocation,thenformoreloca-tions indifferentanglesofview,more thermalcamerasarerequired.Hence, itwillcostmore,besidescomplicatingthesurveillancenetwork.Theproposedthermal/IRspectrum-basedomnidi-rectionalsurveillancesystemmodelisshowninFigure16.9.Thissystemrequiresacustom-madeIR-reflectedhyperbolicmirror,acameramirrorholder,afine-resolutionthermalcameraandalaptoporPC installedwithMATLABprogramming (versionR2007bor later), andanalarmsignalingsystem.Thealarmsignalingsystemcanbeassimpleasacomputer’sspeaker.
Thebestshapeofpracticaluseomnidirectionalmirrorishyperbolic.AsderivedbyChahlandSrinivasaninRef.[37],allthepolynomialmirrorshapes(conical,spherical,parabolic,etc.)donotprovideacentralperspectiveprojection,exceptforthehyperbolicone.Theyalsoshowthatthehyperbolicmirrorguaranteesalinearmappingbetweentheangleofelevationθandtheradialdistancefromthecenteroftheimageplaneρ.Anotheradvantageofhyperbolicmirroriswhenusingitwithacamera/imagerofhomogenouspixeldensity,theresolutionintheomnidirectionalimagecapturedisalsoincreasingwithgrowingeccentricity,andhence,itwillguaranteeauniformresolutionforthepanoramicimageafterunwarping.
TheresearchgroupofOMNIVIEWSproject fromCzechTechnicalUniversityfurtherdevel-opedMATLABsoftwarefordesigningomnidirectionalmirror[38].FromtheMATLABsoftware,omnidirectionalhyperbolicmirrorcanbedesignedbyinputtingsomeparametersthatspecifythemirrordimension.Thefirstparameteristhefocallengthofthecameraf,inwhichforthethermalcamerainuseis12.5mmandthedistanced(ρzplane)fromtheoriginissetto2m.Theimageplaneheighthissetto20cm.Theradiusofthemirrorrimischosent1=3.6cmasmodifiedfromSvoboda’sworkinRef.[39],withradiusforfovearegion0.6cmandretinaregion3.0cm.Foveaangleissetbetween0°and45°,whereasretinaangleisfrom45°to135°.ThecoordinatesaswellastheplotofthemirrorshapearegeneratedusingMATLABandshowninFigure16.10.ThecoordinatesaswellasmechanicaldrawingusingAutoCADareprovidedtoprecisionengineeringcompanytofabricate/custommadethehyperbolicmirror.Thehyperbolicmirrorismilledfromaluminumbarandthenchromed.Chromiumisofgreatinterestbecauseofitslustrous(goodinIRreflection)property,highcorrosionresistance,highmeltingpoint,andhardness.ThefabricatedmirrorisshowninFigure16.11.
Thecameramirrorholderisself-designedandcustommadewithaluminummaterialasshowninFigure16.12.Thethermalcamerausedisanaffordableandaccuratetemperaturemeasurementmode:ThermoVisionA-20MismanufacturedbyFLIRSystems.Thethermalcamerahasatemper-aturesensitivityof0.10rangingfrom−20°Cto350°C.However,forhumandetection,thetemper-aturerangeissettorangefrom30°Cto40°C.Thethermalcameracancapturethermalimageswithfineresolutionupto320×240pixelsofferingmorethan76,000individualmeasurementpoints
Custom made IRreflected hyperbolicmirror + cameramirror holder set
Thermalcamera
Capturethermalimage
Laptop/PC
Processimage and
signal alarm
Figure 16.9 Omnidirectional thermal imaging surveillance system model.
K13920_C016.indd 429 1/4/2013 7:24:16 PM
430 ◾ Effective Surveillance for Homeland Security
perimageatarefreshrateof50/60Hz.TheA-20Mfeaturesachoiceofconnectivityoptions.Forfastimageanddatatransferofreal-timefullyradiometric16bitimages,anIEEE-1394FireWiredigitaloutputcanbechosen.Fornetworkand/ormultiplecamerainstallations,Ethernetconnec-tivityisalsoavailable.EachA-20McanbeequippedwithitsownuniqueURLallowingittobeaddressedindependentlyviaitsEthernetconnection,anditcanbelinkedtogetherwitharoutertoformanetwork.Therefore,itisbestoutfittedforhumanintruderdetection.
Figure 16.11 Fabricated mirror.
3
2.5
2
1.5
1
0.5
0 –3 –2 –1 0 1 2 3
cm
cm
Figure 16.10 Mirror coordinates plot in MATLAB.
K13920_C016.indd 430 1/4/2013 7:24:16 PM
Omnidirectional Human Intrusion Detection System ◾ 431
Aproblemencountered in thermalcamera selection is theexistenceof thehaloingeffect inuncalibratedferroelectricbariumstrontiumtitanate(BST)sensors.Haloingeffectisthepresenceofhalosaroundobjectshavingahighthermalcontrastwiththebackground[40].A-20Mischo-senbecauseitusestheuncooledmicrobolometerFPAdetectortechnologythatdoesnotproducethehaloingeffect.A laptoporPCcanbeusedas imageprocessor,placedeitheronsiteor inamonitoringroom.MATLABversionR2007bprogramming ischosentobeusedbecause ithasuser-friendlysoftwareforperforminglog-polarmappingtechniquetounwraptheomnidirectionalthermal image into panoramic form and it can partition the panoramic thermal images easilyaccordingtoeachsinglelocationtobemonitoredandprocessthemsmoothlywiththetrespasserorfaintdetectionalgorithmuserprogrammedin.Thealarmwillbetriggeredonceahumanbeingisdetectedinatestedimageforhumanintruderdetectionmode.TheoverallfabricatedsystemmodelisshowninFigure16.12.
16.3 Unwarping MethodsThecapturedomnidirectionalimageshavedifferentpropertiescomparedtoperspectiveimagesintermsofimagingdeformation.Suchdistortionleadstotheimagesbeingdifficulttobedirectlyimple-mented.Thus,itisnecessarytoworkoutanefficientmethodtounwarptheomni-image.Unwarping,generally,isamethodusedindigitalimageprocessingin“opening”upanomnidirectionalimageintoapanoramicimage,makingtheinformationontheimagebeabletobedirectlyimplementedand
AQ20
AQ21
Figure 16.12 Overall fabricated omnidirectional thermal imaging surveillance system model.
K13920_C016.indd 431 1/4/2013 7:24:17 PM
432 ◾ Effective Surveillance for Homeland Security
understood.Thissubsectionstudiesthreeuniversalunwarpingmethodsthatarecurrentlyappliedactivelyaroundtheworldintransformingomnidirectionalimagetopanoramicimage,namely,theDGTmethod[4],thepano-mappingtablemethod[5],andthelog-polarmappingmethod[6].
16.3.1 Discrete Geometry Technique MethodDGTmethod,bythenameitself,meansthatthistechniqueisusedbyapplyingonebyonethegeometryoftheimage,discretely,inordertosuccessfullyunwarptheomnidirectionalimageintoapanoramicimage.ThismethodispracticallyusedintransformingtheomnidirectionalimagesintopanoramicimagesonacylindricalsurfaceusingPDE-basedresamplingmodels[4].
InDGTmethod,itisrequiredtoperformthecalculationofeachandeverypixelintheomni-imagefirstandthencheckforitscorrespondingradiusfromthecenteroftheomnidirectionalimageandlaterdeterminewhetheritshouldbeconsideredornot.Thecalculationsstartfromafixedposi-tionanddirection,suchasfromtherightgoingcounterclockwisefor360°.Foraradiusof1,acircleofradius1willbevisualizedinthecenterofomnidirectionalimage,whichinotherwordsmeansthatthecirclewillbeinsizeof3×3pixels.Allthepixelsinthisboundaryof3×3pixelswillbeconsidered,andtheircorrespondingradiuswillbecalculated.Allpixelsthatfallwithintheradiusof1,whichistheradiusofconcern,willbeconsideredintheconversion.Duetothepixelsthataregenerallyanareaofdatainformation,itispossiblethatthecirclewilllieinbetweenthepixels.Therefore,atoleranceof±½radiusissettocounterthisproblem.Inotherwords,acircleofradius1willconsiderthepixelslyingbetweenradiusof0.5and1.4,andacircleofradius2willconsiderthepixelslyingbetweenradiusof1.5and2.4,andsoon.AnexampleisshowninFigure16.13.
Assoonasapixel intheboundaryisdeemedtobeconsideredorinrangeoftheradius, itwillbemappedintoanewmatrixofpanoramicimage.However,sincethepixelsmappedintothepanoramicimagemustbeinordersothattheimagewillnotbedistorted,theimagewillbesplitintofoursectionsof90°each,asshowninFigure16.14,whereeachsectionwillperformthecalculationbasedonthemovingdirectionofthecircle.Forexample,foracircledrawn,startingfromtherightinacounterclockwisedirection,thepixelsinthesectionattheupperrightpartwillbetakenandcalculatedonebyone,fromthebottompartofthesectionandfromrighttoleft,whichwillthenbeincreasedonebyone,tilltheupperpartofthesection,fromrighttoleftaswell.Ontheotherhand,forthelowerleftpartofthesection,thecalculationwillgofromthetopofthesection,goingfromlefttoright.
However,duetothepixelsbeingconsideredfordifferentcirclesofdifferentradiithatwillbenonuniform,asshowninFigure16.15,aresamplingprocessisneededtostandardizethepixelsin
Pixels lyingin between
Figure 16.13 Circle lying in between pixels.
K13920_C016.indd 432 1/4/2013 7:24:18 PM
Omnidirectional Human Intrusion Detection System ◾ 433
everyrowofthepanoramicimage.Therefore,aftereverypixelinthewholeomni-imageismappedontothepanoramicimageplane,spacingwillbeinsertedinbetweenpixelsineveryrow(asshowninFigure16.16)inordertostandardizetheresolutionofthepanoramicimageforeachrow.Thiswillgenerateastandardresolutionofpanoramicimage.However,duetospacingthatisgenerallyemptypixelswithnodatainformation,arowwithverylittlepixelswillbehardlyunderstandable.Therefore,thepixelswillbeduplicatedoverthespacing,insteadofinsertingemptypixelsintoit,andanunderstandableuniformresolutionpanoramicimagecanbegenerated.
16.3.2 Pano-Mapping Table MethodThis method uses a table, which is so-called the pano-mapping table, to process the imageconversion.Pano-mappingtablewillbecreated“onceandforall,”consistingofmanycoordi-natescorresponding to thecoordinates taken fromtheomnidirectional image thatwill thenbemappedintoanewpanoramicimage,respectively.Itispracticallyusedinomnidirectionalvisual tracking[41]andtheunwarpingprocessofomni-images takenbyalmostanykindofomni-camerasprior to requiringanyknowledgeabout thecameraparameters inadvance, asproposedbyJengetal.[5,8].
Figure 16.14 Circle being split into four sections.
Figure 16.15 Nonuniform resolution of panoramic image.
Figure 16.16 Spacing is inserted in between pixels, denoted by black dots.
K13920_C016.indd 433 1/4/2013 7:24:18 PM
434 ◾ Effective Surveillance for Homeland Security
Inpano-mappingtablemethod,itisrequiredtoselectfivelandmarkpointsfromtheomni-directionalimagefirst.Thesepointswillbetakenfromthesameline,drawingfromthecenteroftheomni-imagetothecircumferenceoftheimage,whichinotherwordsiscalledtheradiusoftheimage.Fivepointsinbetweentheendofthislinewillbepicked,andthevaluecorrespondingtotheirradiusfromthecenterisobtained.Itisthenusedinordertoobtainthefivecoefficientsofa0througha4inthe“radialstretchingfunction,”fr(ρ),describedbythefollowing4th-degreepolynomialfunctionof
r f a a a a ar 1 1 2 2 3 3 4 4= ( ) = + + + +ρ ρ ρ ρ ρ0 (16.1)
wherercorrespondstotheradiusρistheparticularradiusfortheeachofthefivepointstakena0–a4arefivecoefficientstobeestimatedusingthevaluesobtainedfromthelandmarkpoints
Oncethefivecoefficientsareobtained,thepano-mappingtable,TMN,canthenbegenerated.Thesizeofthetablewillfirstbedeterminedmanually,bysettingittoatableofsizeM×N.Hence,inordertofillupatablewithM×Nentries,thelandmarkpointρ,whichcorrespondstotheradiusof theomnidirectional image,willbedivided intoMseparatedparts, and theangleθwillbedividedintoNpartsasfollows:
ρij
i radiusM
= × (16.2)
θij
j 36
N= × 0°
(16.3)
andthecalculationwillbeprocessed,bytakingthefirstpointwherei=1andj=1,whichgivesρ11=radius/Mandθ11=360°/N.Thevalueofρijwillthenbesubstitutedintothe“radialstretchingfunction”inordertoobtaintheparticularradiusatthatparticularlandmarkpoint.Thisradiusobtainedwillthenbesubstitutedintotheequationasfollowstoberoundedup,inordertogetthecorrespondingcoordinatesintheomnidirectionalimage:
v r cos= θ (16.4)
u r sin= θ (16.5)
wherevanducorrespondtothex-andy-coordinatesoftheomnidirectionalimage.Thiscoor-dinate(u,v)obtainedisinsertedintothepano-mappingtableTMN=Tij.TheuandvwillthenbeprocessedforNtimesbyincreasingjforNtimestoobtaindifferentangles,θ,tolaterdetermineall the coordinates corresponding to the value of landmarkpoint.These coordinates obtainedareinsertedintothetableofi=1withtheircorrespondingj=1toj=N,andtheiwillthenbeincreasedby1,andtheprocessisrepeatedforj=1toj=Ntodetermineallcoordinatesrelatedtoi=2.ThisiwillberepeatedforMtimes,andatableofM×Nentrieswithallthecoordinatescanbegenerated.Thecoordinatesineachoftheentriesaretakenonebyone,inordertomapeachandeverypixelintheomnidirectionalimagewiththecoordinateinthecurrententry,intoanewpanoramicimage.Theconversioniscompletedupontheendofmappingofthetable.
K13920_C016.indd 434 1/4/2013 7:24:19 PM
Omnidirectional Human Intrusion Detection System ◾ 435
16.3.3 Log-Polar Mapping MethodLog-polarmappingisatypeofspatiallyvariantimagerepresentationwherebypixelseparationsincreaselinearlywithdistance.ItenablestheconcentrationofcomputationalresourceonanROIaswellasmaintainingtheinformationfromawiderview.Thismethodisimplementedbyapply-inglog-polargeometryrepresentations.ThecapturedomnidirectionalimagewillfirstbesampledbyspatiallyvariantgridfromaCartesianformintoalog-polarform.Thespatiallyvariantgridrepresenting log-polarmappingwill thenbe formedby inumberof concentric circleswithNnumberofsamples,andtheomnidirectionalimagewillthenbeunwarpedintoapanoramicimageinanotherCartesianform.
Thismethodispracticallyusedinrobustimageregistration[43],orinroboticvision,particu-larlyinvisualattention,targettracking,egomotionestimation,and3Dperception[44],aswellasinvision-basednavigation,environmentalrepresentations,andimaginggeometries[45],byJoséSantos-VictorandAlexandreBernardino.Inlog-polarmappingmethod,thecenterpixelforlog-polarsamplingiscalculatedby
ρ x y x x y yi i i c i c,( ) = −( ) + −( )2 2
(16.6)
θ
πx y
N y yx x
i ii c
i c
, tan( ) =
−−
−
21 (16.7)
andthecenterpixelforlog-polarmappingiscalculatedby
x xo cρ θ ρ θ, cos( ) = + (16.8)
y yo cρ θ ρ θ, sin( ) = + (16.9)
wherexc,ycarethecenterpointsofouroriginalCartesianformcoordinateNisthenumberofsamplesineachandeveryconcentriccircletaken
Theoriginal(xi,yi)inCartesianformissampledintolog-polarcoordinateof(ρ,θ),asshowninFigure16.17.Thecenterpointiscalculatedbyusing(16.6)and(16.7)togettherespectiveρandθ,whichcoveraregionoftheoriginalCartesianpixelsofradius:
r brn n= −1 (16.10)
and
b
NN
= +−
ππ
(16.11)
whereristhesamplingcircleradiusbistheratiobetweentwoapparentsamplingcircles
Figure16.18showsthecircularsamplingstructureandtheunwarpingprocessdonebyusingthelog-polarmappingmethod[43].Themeanvalueofpixelswithineachandeverycircularsampling
K13920_C016.indd 435 1/4/2013 7:24:20 PM
436 ◾ Effective Surveillance for Homeland Security
iscalculatedandwillbeassignedtothecenterpointofthecircularsampling.Theprocesswillthencontinuebymappingthemeanvalueatlog-polarpixel(ρ,θ)intoanotherCartesianformusingEquations16.8and16.9,andtheunwarpingisdoneattheendofmapping.
16.3.4 Performance EvaluationThis subsection reports the performance evaluation for different unwarping methods. Fewimportantfactorsareselectedfortheperformanceevaluationoftheunwarpingmethods.Thesefactorsincluderesolutionoftheimagegenerated,qualityofimage,algorithmusedinperform-
A
Center ofimage
AB
BB΄ B΄
A΄A΄
y2
x2
θ
ρ
Figure 16.18 Circular sampling structure and the unwarping process.
Capturedimage
y1
x1Sampling Mapping
x2
y2
N
ρ
ρθ (0,0)
Figure 16.17 Process of log-polar mapping.
K13920_C016.indd 436 1/4/2013 7:24:21 PM
Omnidirectional Human Intrusion Detection System ◾ 437
ingtheunwarpingprocess,complexity,processingtime,anddatacompression.SomecapturedomnidirectionalimagesasshowninFigure16.19willbeusedtotesttheunwarpingmethods.
1.Resolution of the image generated:Theresolutionofeachgeneratedpanoramicimageusinglog-polarmappingmethod,DGTs, andpano-mapping tablemethod isdiscussed in thissubsection.Thelog-polarmappingmethodprovidessmallerresolutionofdimensionthatequalsto1/4-foldoftheomnidirectionalimage,whereasfortheDGTmethodandpano-mapping tablemethod, the resolutionof thepanoramic imageproducedcanbe as largeasthelengthoftheperimeteroftheomnidirectionalimage,withthewidthequalstotheradiusoftheomnidirectionalimage.However,duetotheimagesbeingrescaledforviewingpurposes,thedifferenceisnotobviousinthischapter.
2.Quality of image: Since the images are rescaled, thedifference inquality isnot apparentaswell.However,pano-mappingtablemethodisfoundtoproducethehighestqualityofimage,followedbythelog-polarmappingmethod,andtheDGTmethodcorrespondinglyindescendentqualityorder.
(a-1) (a-2)
(b-2)
(c-2)
(d-2)
(b-1)
(c-1)
(d-1)
Figure 16.19 Performance evaluation (a-1, a-2). Samples of omnidirectional images (b-1, b-2). Panoramic images generated using DGT method (c-1, c-2). Panoramic images generated using pano-mapping table method (d-1, d-2). Panoramic images generated using log-polar method.
K13920_C016.indd 437 1/4/2013 7:24:22 PM
438 ◾ Effective Surveillance for Homeland Security
3.Algorithm used in performing the unwarping process: In log-polar mapping algorithm, theomnidirectionalimageisconsideredintheformofanumberofsectorsinwhicheachsectorconsistsofagroupofpixelsthatwillbeextractedlaterinsectorbysectortobearrangedintoarectangularformofimage,whereasfortheDGTmethod,pixelbypixelistobeextractedandarrangedintoarectangularformimage.Thesepixelswillthenbereproduced,ordupli-cated,inordertostandardizethenumberofpixelsavailableineachrowofthepanoramicimage.Forthepano-mappingtablemethod,analgorithmisusedwherebyatableiscreatedatinitialization,toindicatethecoordinatesofthepixelstobeextractedfromtheomnidirec-tionalimage.Oncethetableiscreated,itwillthenbeusedoverandoveragaintomapeachofthepixelatthatparticularcoordinate,onebyone,fromtheomnidirectionalimageintoapanoramicimage,hencethename“onceandforall.”
4.Complexity:Table16.1showsthebig-Ocomplexityof log-polarmappingmethod,DGTmethod,andpano-mappingtablemethod.
5.Processing time:Theprocessingtimeforallthethreeunwarpingmethodstotransformanomnidirectional image into a panoramic image is calculated using MATLAB function“cputime.”Theprogram isprocessedfive timesonfivedifferent images, and theaverageprocessingtimeiscomputed.Itisfoundthatpano-mappingtablemethodhasthefastestcomputationtime,whichis1.220s,followedbylog-polarmappingmethodbeing2.003sand3.426sfortheDGTmethod.
6.Data compression:Thegeneratedpanoramicimageproducedbylog-polarmappingmethodhastheresolutionof473×114;DGTmethodhasaresolutionof1472×235and1146×243forpano-mappingtablemethod,inwhichtheoriginalomnidirectionalimageisofresolu-tion473×473.Fromtheoutputresolution,itisclearthatlog-polarmappinghasthehighestcompression,whichcompressestheimageuptofourfold,comparedtoDGTmethod(0.65-foldimageexpansion)andpano-mappingtablemethod(0.80-foldimageexpansion).
Intermsofresolution of the image generated,althoughtheimagegeneratedbyDGTmethodandpano-mappingtablemethodsislargerascomparedtotheimagegeneratedbylog-polarmappingmethod, these twomethods seemtoelongate theactual sizeof the image. Inotherwords, this
Table 16.1 Big-O Complexity
DGT Log-Polar Mapping
Pano-Mapping Table
AdditionO(XY2) O(X2Y2) O(Y2)
Subtraction
MultiplicationO(Y) O(X2) O(Y2)
Division
Logarithmic —O
log XY
( )( )
log
—
X, length of the panoramic image = perimeter of the omni-directional image taken into consideration’ Y = height of the panoramic image = radius of the omnidirectional image taken into consideration.
K13920_C016.indd 438 1/4/2013 7:24:22 PM
Omnidirectional Human Intrusion Detection System ◾ 439
methodtendstomaketheobjectsintheimageextendedand“broader”thantheoriginalimage.Duetothiselongation,itwillbehardertoexaminethepictureandtheobjects,asthesenseofthesizehadbeeneliminated.Forlog-polarmappingmethod,theextensionisnotmuch,anditisnotasobviousasDGTmethodandpano-mappingtablemethods.Intermsofquality of image,pano-mappingtablemethodproducesthehighestqualityamongthethreemethods,followedbylog-polarmappingmethodwithaslightlylowerimagequalitybutstillwithinanacceptablerange,andlastlytheblurredDGTmethod.Intermsofalgorithm used in performing the unwarping process,pano-mappingtablemethodusesthesimplestandeasiestalgorithm,followedbyaslightlycomplexalgorithmthatisthelog-polarmethod,andlastly,acomplicatedandcomplexalgorithmfromtheDGTmethod.Intermsofcomplexity,itisfoundthatpano-mappingtablemethodhastheleastcomplexity,followedbyDGTmethod,andlastlylog-polarmappingmethodinbig-Onotation.Intermsofprocessing time,onaverage,pano-mappingtablemethodhasthefastestprocessingtimetotransformanomnidirectionalimageintoapanoramicimage,followedbylog-polarmappingmethodandDGTmethod.Intermsofdata compression,log-polarmappingmethodhasthebestdatacompressionratecomparedtopano-mappingtablemethodandDGTmethod.ThisisverygoodinpreservingCPU’smemory,asthememoryavailableisusuallyverylimited.
16.4 Automatic Human Intruder Detection AlgorithmAutomatichumanintruderdetectionis implementedintheproposedomnidirectional imagingsystemtoanalyzeinformationfromthepositionoftheimagingtoolsandautomaticallydetectatrespasser.Twoautomatichumanintruderdetectionalgorithmsarediscussedinthissubsection;thisincludespartitionedROIalgorithm[11]andhumanheadcurvetestalgorithm[12,13].
16.4.1 Partitioned Region of Interest AlgorithmThepartitionedROI-basedhumanintruderdetectionalgorithmissummarizedasfollows:
Step1:Adjustthethermalcameradetectionrangingfrom30°Cto40°Csothatobjectwithhumanbodytemperaturerangecanbedetected.
Step2:Unwarptheomnidirectionalthermalimageintopanoramicthermalimageusinglog-polarmappingtechnique.
Step3:CaptureimagescontinuouslyfromthermalcameraintolaptopandnameitasPxwherex=1,2,3,…isthediscrete-timeinstant.
Step4:Divideeach imagecaptured fromthermalcamera into (m×n) regions.Each regionconsistsofequalnumberofpixels.
Step5:DefineamatrixMwithsizeof(m×n)torepresentthecharacteristicofeachcorrespond-ingregion.
Step6:DefineathresholdvalueQ.QisthethresholdvalueofthedifferencebetweensumsofRGBvalueforaparticularcurrentimagepixelandpreviousimagepixel.
Step7:DefineavariablehforcountingthenumberofpixelsexceedingQ.Initially,h issetto0.
Step8:DefineHasaminimumnumberofpixelswithdifferenceexceedingQ.Step9:ComparecurrenttakenimagePxwithprevioustakenimagePx−1.Foreachcorrespond-
ingregion,findoutthedifferencebetweenaparticularcurrentimageandpreviousimagepixels’sumofRGBvalues.IfthedifferencebetweensumsofRGBvaluesforaparticular
K13920_C016.indd 439 1/4/2013 7:24:22 PM
440 ◾ Effective Surveillance for Homeland Security
currentimagepixelandpreviousimagepixel≥Q,thenh=h+1.Ifh≥H,marka“1”intothecorrespondingelementofM;elseifh<H,marka“0”intothecorrespondingelementofM.AnexampleisshowninFigure16.20.
Step10:LetFbethenumberofdifferentelementsthatalignverticallyandcontinuously.SomeexamplesofcalculationofFareshownasfollows:
Examples
Example 1
Compare
0 0 0 0 00 0 0 0 00 0 0 0 00 0 0 0 00 0 0 0 0
0 0 0 0 00
with11 0 0 0
0 1 0 0 00 1 0 0 00 1 0 0 0
F=4inthisexample.Example 2
Compare with
0 0 0 0 00 0 0 0 00 0 0 0 00 0 0 0 00 0 0 0 0
0 0 0 0 00
11 0 0 00 0 0 1 00 0 0 1 00 0 0 1 0
F=3inthisexamplebecauseonlythreedifferentelementsarealignedverticallyandcontinuously.Example3
Compare with
0 0 0 0 00 0 0 0 00 0 0 0 00 0 0 0 00 0 0 0 0
0 0 0 0 00
11 0 0 00 1 0 1 00 0 0 1 00 0 0 1 0
Iftherearemorethantwogroupsofverticallyandcontinuouslydifferentelements,thenwewilltakethelargestnumber.Inthiscase,F=3.
Step11:DefineGasminimumregionsthatahumanbeingwillappearon-screen.
IfF≥G,thenalarmunknowntrespasserdetected.
Divide ROI into (m × n)regions
00000
00000
00000
00000
00000
00000
00000
00000
00000
00000
Figure 16.20 Example for partitioning of ROI for trespasser detection surveillance system.
K13920_C016.indd 440 1/4/2013 7:24:23 PM
Omnidirectional Human Intrusion Detection System ◾ 441
16.4.2 Human Head Curve Test AlgorithmThehumanheadcurvetestalgorithmissummarizedasfollows:
1.Algorithm for Trespasser DetectionStep1:Acquirethermalimagethroughhyperbolicreflector.RefertoFigure16.21forthe
exampleofimagecaptured.Step2:Imageunwarping:Unwarptheacquiredthermalimageintopanoramicimage.Refer
toFigure16.22fortheexampleonresultingpanoramicimage.Step3:Imagecropping:Croptheimagetoobtainthethermalimageoftheinterestedarea
only.PleaserefertoFigure16.23fortheexampleontheresultingcroppedimage.Step4:Binaryimageconversion:Convertthermalimageintopurelyblackandwhiteimage
(BW)using
BW i, j
1, T1 Temp i, j Th0, otherwise
( ) ( )
=
≤ ≤ (16.12)
wheretemperatureatpoint(i,j)ofthermalimageTlandTharetheminimumandmaximumpossiblehumanbodytemperaturesi,jarethepixel’srowandcolumncoordinates,respectively
Step5:Objectidentification:IdentifyobjectsinsidetheBWwhereagroupofdiscontinuouswhitepixelsisconsideredasasingleobject.
Step6:Noisefiltering:a. RemovefromBWallconnectedcomponents(objects)thathavefewerthanSpixels.b. Createaflat,disk-shapedstructuringelement(SE)withradius,R.AnexampleofSE
isshowninFigure16.24.PerformmorphologicalclosingontheBW.Morphologicaloperationdilatesan
imageandthenerodesthedilatedimageusingthesameSEforbothoperations.
AQ22
AQ23
Figure 16.21 Thermal image captured through hyperbolic reflector.
K13920_C016.indd 441 1/4/2013 7:24:23 PM
442 ◾ Effective Surveillance for Homeland Security
Step7:Boundaryextraction:FindboundarylineofeachidentifiedobjectinsideBWandrecorditintoanarrayofcoordinates.
Step8:Headdetection:Foreachobject’sboundary,performtheheaddetectionalgorithmasexplainedin“AlgorithmforHeadDetection.”
Step9:Ifahumanbeingisdetected,triggerthealarm.
SE =
R = 3
0 0 0 11111
1
1
0 0 000
00
0
0
111110
111111
111110
111110
001000
Origin
Figure 16.24 Disk-shaped SEs (e.g., R = 3).
Figure 16.23 Thermal image after cropping process.
Figure 16.22 Panoramic view of the inspected scene after the thermal image is unwarped.
K13920_C016.indd 442 1/4/2013 7:24:24 PM
Omnidirectional Human Intrusion Detection System ◾ 443
2.Algorithm for Head DetectionStep1:Startingpointidentification:Calculatethestartingpointforheadtopdetection.The
startingpointshallbethehighestpoint(smallesty-coordinatevalue)amongtheintersectionpointsoftheboundarywiththehorizontalmiddleline,wherehorizontalmiddlelineisgivenbyx=xmandxmisthemeanofallx-coordinatesintheboundaryoftheobject.PleaserefertoFigure16.25aforbetterunderstanding.
Step2:Peakpointdetection:Fromthestartingpoint,followtheboundaryinclockwisedirec-tionandsearchforthefirstpeakpointencountered(Ppeak1).Again,fromthestartingpoint,followtheboundaryinanticlockwisedirectionandsearchforthefirstpeakpointencoun-tered(Ppeak1).Peakpointisdefinedasthepointinwhichithasthesmallesty-coordinatevaluecomparedtoalltheDproceedingpoints.Disthenumberofnextofpointtobetested.RefertoFigure16.25bforbetterunderstanding.
Step3:Headtoppointdetection:ComparePpeak1andPpeak2obtainedinstep2.Recordthehighestpoint(withsmallery-coordinate)astheheadtoppoint,Ppeak1.Forexample,inFigure16.25b,Ppeak2ishigherthanPpeak1.Thus,Ppeak1=PHT=Ppeak2.
Step4:Boundarylinesplitting:Splittheboundaryintoleftboundary(Bl)andrightboundary(Br)fromtheheadtoppointtowardthebottom.Takeonlyonepointforeachy-coor-dinatetofilteroutunwantedinformationsuchasraisedhands(refertoFigure16.26forbetterunderstanding):
Bl xl yl Br xr yri i i i= [ ] = [ ], , , (16.13)
wherexli,yli,xri,yriarethepixels’y-coordinatesandx-coordinatesforBlandBr,respectivelyi=1,…,NistheindexnumberNisthesizeoftheboundarymatrix(NisthenumberofpixelsforBlandBr)
Starting point
Horizontalmiddle line
Startingpoint
Ppeak1
Ppeak2 (PHT)
X
Y
(a) (b)
Figure 16.25 (a) Horizontal middle line and the starting point as in step 1. (b) Detection of (c/w from starting point) and (counter c/w from starting point).
K13920_C016.indd 443 1/4/2013 7:24:26 PM
444 ◾ Effective Surveillance for Homeland Security
Step 5: Left significant point detection: Search downward along Bl from PHT for thefirstleftmostpointencountered(Plp).Next,searchfortherightmostpointrightafterPlpwhichisPld(refertoFigure16.27forbetterunderstanding):
Pl xl yl Pl xl ylp lp lp d ld ld= =( , ), ( , ) (16.14)
wheresubscriptlpisanindexnumberofthefirstleftmostpointsubscriptldisanindexnumberoftherightmostpointrightafterPlp
PHT
Prd
Prp
Pld
Plp
B(left)B(right)
Figure 16.27 Example of points found in steps 3, 5, and 6.
Left boundaryRight boundary
(a) (b)
Figure 16.26 (a) Original object boundary. (b) Left and right object boundary after the splitting process (step 4).
K13920_C016.indd 444 1/4/2013 7:24:26 PM
Omnidirectional Human Intrusion Detection System ◾ 445
Step6:Rightsignificantpointdetection:SearchdownwardalongBrfromPHTforthefirstrightmostpointencountered(Prp).Next,detecttheleftmostpointrightafterPrp,whichisPrd(refertoFigure16.27forbetterunderstanding):
Pr xr yr Pr xr yrp rp rp d rd rd= =( , ), ( , ) (16.15)
wheresubscriptrpisanindexnumberofthefirstrightmostpointsubscriptrdisanindexnumberofthefirstleftmostpointrightafterPrp
Step7:Headsymmetrictest:Definehl=verticaldistancebetweenPHTandPldhr=verticaldistancebetweenPHTandPrd.
Testtheratiobetweenhlandhr.Ifhl/hrorhr/hl>2,thentheobjectisnotconsideredasahumanbeingandthenextsubsequentstepsinthisalgorithmcanbeskippedandproceedwiththenextobject.Else,iftheobjectispossiblyahumanbeing,continuestep8forfurtherdetection.
Step8:Neck–bodyposition test:Calculate∆x,which is thedistancebetweenxcandxmwherexc=horizontalcenterbetweenPldandPrdandxmisobtainedinstep1.Definewn=horizontaldistancebetweenPldandPrd.
If∆x≥2wn,thenthisobjectisnotclassifiedasahumanbeingandthenextsubsequentstepsinthisalgorithmcanbeskippedandproceedwiththenextobject.Elseif∆x<2wn,thentheobjectispossiblyahumanbeing.Continuestep9forfurtherdetection.
Step9:Curvetests:a. Topcurvetest
Definest=floor(min(lp,rp)/(F/2))asthestepsizefortopcurvetest.
Calculate
C
1, yl yl
0, otherwisetl
1 s k 1 S (k 1)t t=≤+ ∗ + ∗ +
C
1, yr yr
0, otherwisetr
1 s k 1 S (k 1)t t=≤+ ∗ + ∗ +
(16.16)
C C Ct tl tr= ∑ + ∑
wherek=0,…,F/2–1Fisthestep-sizepartitionvariable.FisevenintegerandF≥2.For example, if F=6, st=8, then the y-coordinates tested are shown in Figure
16.28.Thesameconceptgoesforleftcurvetestandrightcurvetest.
Note:Thesymbol“*”meansmultiply.b. Leftcurvetest
Definesl=floor(min(lp,ld–lp)/(F/2))asthestepsizeforleftcurvetest.
K13920_C016.indd 445 1/4/2013 7:24:26 PM
446 ◾ Effective Surveillance for Homeland Security
Calculate
C =
1, xl xl
0, otherwisell
lp s k lp S (k 1)t l+ ∗ + ∗ +≤
C =
1, xl xl
0, otherwisel2
lp s k lp S (k 1)t l+ ∗ + ∗ +≤
(16.17)
C C Cl l1 l2= ∑ + ∑
wherek=0,…,F/2−1c. Rightcurvetest
Definesr=floor(min(rp,rd–rp)/(F/2))asthestepsizeforrightcurvetest.
Calculate
C
1, xr xr
0, otherwiserl
rp + s k rp S (k 1)r r=≤∗ + ∗ +
C
, xr xr
0, otherwiser2
p s k rpr=≤
+ +1 r + ∗ ∗S (k 1)r (16.18)
C C Cr r1 r2= ∑ + ∑
wherek=0,…,F/2–1Step10:Humanidentification:
Definecurvetestcondition:
C F 1t ≥ − (16.19)
C F 1l ≥ − (16.20)
C F 1r ≥ − (16.21)
Checkconditions(16.19)through(16.21).Ifanytwoormoreconditionsaretrue,theobjectisverifiedasahumanbeing.Else,theobjectisnotconsideredasahumanbeing.
yl25
yr25yl17
yr17yl9 yl1
yr1 yr9
Figure 16.28 Example on top curve test.
K13920_C016.indd 446 1/4/2013 7:24:28 PM
Omnidirectional Human Intrusion Detection System ◾ 447
16.4.3 Experimental ResultsIn this section, the application of the proposed omnidirectional human intruder detectionsystem is briefly illustrated. An omnidirectional image captured using digital camera onthe site is shown in Figure 16.29. An omnidirectional thermal image also captured usingthermal camera on the site is shown in Figure 16.30. The unwarped form of Figure 16.29
Figure 16.29 Case studies of trespasser detection (digital color form).
Figure 16.30 Case studies of trespasser detection (thermal image).
K13920_C016.indd 447 1/4/2013 7:24:29 PM
448 ◾ Effective Surveillance for Homeland Security
(digitalcolorpanoramicform)isshowninFigure16.31,whereastheunwarpedformofFigure16.30(thermalimagepanoramicform)isshowninFigure16.32,respectively.InFigure16.32,thelog-polarmappingprocessisby4:1reductionmappingscale,whichmeansthat320×240omnidirectional thermal image’s Cartesian pixels are mapped to one-fourth of the thermalimage Cartesian pixels (320×60) in panoramic view, with fourfold data compression com-pared to original omnidirectional thermal image as inFigure 16.30.The captured thermalimagesaretestedfortwotrespasserfaintdetectionalgorithmsasproposedinSections4.1and4.2intheprecedingtext.
16.4.3.1 Experimental Results for Partitioned ROI-Based Human Intruder Detection Algorithm
InpartitionedROIalgorithmfortrespasserdetection,therearethreeparametersthatneedtobeoptimized,whichareQ,H,andG,whereQisthethresholdvalueofthedifferencebetweensumofRGBvalueforaparticularcurrentimagepixeltopreviousimagepixel,HisminimumnumberofpixelswithdifferenceexceedingQandGisminimumregionsthatahumanbeingwillappearon-screen.
SincetheimagecapturedisinRGBform,thedifferenceofthesumofRGBvaluesbetweenaparticularcurrentimagepixelandpreviousimagepixelisbetween0and765.ForQparameter,1000sampleimages(withorwithouthumanbeing)areusedtotesteverydifferentpointwithstepsizeof15.TheaccuracyversusdifferenceofsumofRGBvaluesisplottedinFigure16.33.Fromtheplot,theoptimumQvalueis345withhighestaccuracyof95.30%.
AQ24
Figure 16.31 Unwarp form of Figure 16.29 (digital color panoramic form).
Figure 16.32 Unwarp form of Figure 16.30 (thermal image panoramic form).
K13920_C016.indd 448 1/4/2013 7:24:30 PM
Omnidirectional Human Intrusion Detection System ◾ 449
Theunwarpedpanoramicimageispartitionedinto50regions(m=10,n=5)witheachregionconsistingofequalnumberofpixels(384).AsforHvalue,thealgorithmwithpets(hamster,cat,anddog)andhuman,movingtowardandawayfromthecapturedregion.Onethousandsampleimagesarecaptured.Byusingthesampleimages,werepeatedthesimulationwithH=10%,20%,30%,…,100%ofnumberofpixeldifferencetototalpixelsinoneregionratio.ThegraphaccuracyversusnumberofpixeldifferencetototalpixelsinoneregionratioisplottedinFigure16.34.Fromtheplot,theoptimumHvalueis50%oftotalpixelsinaregion,withthehighestaccuracyof97%.
AsforGvalue,thealgorithmistestedwithhumanmovingtowardandawayfromthecap-tured region with minimum regions that a human being will appear on-screen, G=1–5. Thegraphofaccuracyversusminimumregionsthatahumanbeingappearson-screenGisshowninFigure16.35.Fromthegraph,theoptimumGvalueis3withhighestaccuracyof93.5%.
For testing the trespasserdetectionperformanceofpartitionedROI-basedtrespasserdetec-tion algorithm, a total of10,000 imageswith test subjects (humanbeingor animal) roamingrandomlyinthetestsite(asshowninFigure16.37)visibletotheproposedsystemaretakenassamples.Thisincludesthermalimageswithasingletrespasser,morethanonetrespasser,withoutatrespasser,andanimals(cats,birds,etc.,whicharenotcountedastrespassers).The“operatorperceivedactivity”(OPA)[46]isusedandtheoperatorwillcommentontheimagescaptured,
AQ25
100.00%90.00%80.00%70.00%60.00%
40.00%50.00%
30.00%20.00%10.00%
Accuracy vs. difference of sum of RGB values
0.00%0 60 120 180 240 300 360 420 480 540 600 660 720
Figure 16.33 Accuracy versus difference of sum of RGB values.
Accuracy vs. number of pixel differenceto total pixels in one region
1
0
0.5
20% 30%10% 40% 50% 60% 70% 80% 90% 100%
Figure 16.34 Accuracy versus number of pixel difference to total pixels in one region.
K13920_C016.indd 449 1/4/2013 7:24:31 PM
450 ◾ Effective Surveillance for Homeland Security
whetherthereisanytrespasserornot,andcomparewiththedetectedresultofthesurveillancesystem.Fromthe totalof10,000samples images forevaluation,7,080weredetectedperfectly(trespasser-or-not condition agreedbybothobserver and surveillance system), that is,with anaccuracyof70.8%.
16.4.3.2 Experimental Results for Human Head Curve Test Algorithm
Thesame10,000testedsamplesasmentionedinSection16.4.3.1areappliedhere.TodetermineoptimumvalueforparameterTl,arandomsampleimageischosenandconvertedintoBWimageusingstep2ofthehumanheadcurvetestalgorithmwithvalueofTlrangingfrom0to510(sumofRandGcomponentsinRGBimage).Performthebinaryimageconversionrepeatedlywithincreasingstepsizeof10forTlandsearchfortheoptimumTlwherethenoisecanbeminimizedandthehumanshapeisnotdistortedintheresultingimage.Basedon1000observableimages,Tlisbestsuitablesetat150inthisexperiment.Foranexample,ifTl=130isused,excessivenoisewillbeintroduced.IfTl=170isused,therewillbetoomuchdistortiontohumanbeingintheresultingimage.RefertoFigure16.36forbetterunderstanding.
Todetermine theoptimumvalue forparameterTh,performthebinary imageconversionrepeatedlywithdecreasingstepsizeof10forThandsearchfortheminimumvalueofThthatdoesnotinfluencetheappearanceofthehumanobject.Basedon1000observableimages,Thisbestsetat430inthisexperiment.Forexample,ifTh=400isused,theimageofhumanbeingisdistorted. IfTh=460 isused, therewillbeno improvement for the image.LowerThvalueis preferredbecause itwill filter outmorenoise component.Refer toFigure16.37 forbetterunderstanding.
Humanshape’sparametersSandRareapproximatedfrom1000testingimagesatadistanceof5mfromtheimagingsystem.Fromthoseimages,thehuman’spixelsareapproximately30.Hence,Sissetto30.LargerSEspreservelargerfeatures,whilesmallerelementspreservethefinerdetailsofimagefeatures.Figure16.38showsexampleswithdifferentRvaluesforSEselection.ItisobservedthatwhenR≥3,theneckpartisunidentifiedfromtheimages.Sincefinerhumanheadshapeisofconcern,RforSEisbestsuittosetattheminimum,whichis2,aswellaswithlowercomputationalcomplexity.
Theaccuracyoftheproposedalgorithmisthenevaluatedusing“OPA”inwhichthepro-posedalgorithmisevaluatedwithrespecttotheresultsinterpretedbyahumanobserver[46].Firstly, the panoramic images are tested using the proposed algorithm. Then, the result is
60.00%
40.00%
20.00%
0.00%1
Accuracy vs. minimum regions that a humanbeing will appear on screen
2 3 4 5
80.00%
100.00%
Figure 16.35 Accuracy versus minimum regions that a human being will appear on-screen.
K13920_C016.indd 450 1/4/2013 7:24:31 PM
Omnidirectional Human Intrusion Detection System ◾ 451
(a) (b) (c)
Figure 16.36 (a) T1 = 130, (b) T1 = 150, and (c) T1 = 170.
(a) (b) (c)
Figure 16.37 (a) Th = 400, (b) Th = 430, and (c) Th = 460.
K13920_C016.indd 451 1/4/2013 7:24:32 PM
452 ◾ Effective Surveillance for Homeland Security
comparedwith the resultof thehumanobserver.Theaccuracyof theproposed algorithm isthepercentageofinterpretation(trespasserornot)agreedbyboththehumanobserverandtheproposedalgorithm.
TodetermineparameterDandF,allofthe10,000sampleimagesaretestedwithdifferentcombinationsofDandF.AsshowninthegraphinFigure16.39,theoptimumvaluesforparam-etersDandFare7and2,whichcontributetoaccuracyof81.38%.
16.4.4 Comparison between Two Proposed Human Intruder Detection Algorithms
ThefirstproposedalgorithmthatisthepartitionedROI-basedhumanintruderdetectionalgorithmisregionalbasedwherebyanobjectthatoccupiesmorethanacertainnumberofpartitionsinapanoramicimageisconsideredasahumanbeingandviceversa.However,thealgorithmhastwo
79.00%
75.00%73.00%71.00%69.00%67.00%65.00%
3 4 5 6
Accuracy vs. D and F parameters
7
Acc
urac
y
D parameter8 9 10 11
F = 2
F = 4
F = 6
F = 8
F = 10
F = 12
77.00%
81.00%
Figure 16.39 Accuracy of proposed algorithm for different combinations of D and F.
(a) (b) (c) (d) (e)
Figure 16.38 Examples with different R values for SE selection: (a) R = 1, no changes; (b) R = 2; (c) R = 3; (d) R = 4; and (e) R = 5.
K13920_C016.indd 452 1/4/2013 7:24:34 PM
Omnidirectional Human Intrusion Detection System ◾ 453
majorconcerns.Firstconcernisthedistance,inwhichahumanbeingthatisfarawayfromtheimagingsystemwillnotbeidentifiedasahuman.Thesecondconcernisifananimal(suchascat,dog)ismovingtooclosetothesystem(whichoccupiesmorethanthethresholdpartitioned),itwillbemissconsideredasahumanbeingtoo.
Hence,asecondeffectivehumanintruderdetectionalgorithm,whichisthehumanheadcurvetestalgorithmwithhumanheaddetectioncapability,isproposed.Bycomparingthetwohumanintruderdetectionalgorithms,humanheadcurvetestalgorithmrequiredcomplicatedheadsym-metrictestandcurvetest.PartitionedROI-basedhumanintruderdetectionalgorithmissuper-seding human head curve test algorithm in terms of simplicity and lower computational timeconsumption(averageroutinetimeforprocessingonesampleis1.3sforpartitionedROI-basedhumanintruderdetectionalgorithmand2.27sforhumanheadcurvetestalgorithm).However,intermsofefficiency,humanheadcurvetestalgorithmwithanaccuracyof81.38%ishigherthanpartitionedROI-basedhumanintruderdetectionalgorithmwithanaccuracyof70.8%inthesamesetof10,000testedimages.
16.5 Conclusion and Future Research DirectionsThis chapter presented omnidirectional human intrusion detection system using computervision techniques. Two imaging methods, namely, vision spectrum imaging and IR imag-ing,areappliedincomputervision-basedomnidirectionalhumanintrusiondetectionsystem.Simulationresultsshowthatlog-polarmappingproposedintransformingthecapturedomnidi-rectionalimagesintopanoramicformhasgoodqualityinoutputimagewithhighdatacompres-sionrateandfastprocessingspeedinprovidingobserverorimageprocessingtoolsawideangleof view. Automatic human intrusion detection algorithms are implemented in the proposedomnidirectionalimagingsystem,bothinvisionspectrumimagingandinIRspectrumimaging,respectively.TheproposedhumanintrusionalgorithmincludespartitionedROIalgorithmandhumanheadcurvetestalgorithm.ExperimentalresultsalsoshowthatpartitionedROI-basedhumanintruderdetectionalgorithmissupersedinghumanheadcurvetestalgorithmintermsof simplicity and lower computational time consumption. However, human head curve testalgorithmcantraceouthumanintruderfromthepanoramicimagesmoreaccuratelycomparedtoROIalgorithm.
Currently the omnidirectional human intrusion detection systems are applied in indoorbuildingsecurityfori-habitat(smarthome),fossilpowerplant,etc.,andprototypingforborderintrusiondetection,onhumantargets(includingsmugglers,illegalimmigrants,orterrorists).Inthefuture,itwillbeembeddedwithfacialrecognitioncapabilitiestorecordandidentifycrimi-nals’andsuspects’identity.Also,amobilerobotcanbebuiltformovingaroundthesurveillancesitecarryingsuchomnidirectionalsurveillancesystem.Theimagingtoolpowerisdesignedtobesuppliedbyabatteryinsteadofapowerplug.Itallowstherobottocarrythesurveillanceimagingtoolsetwithoutlimitationofthepowercables’length.Byusingamobilerobot,severalsitescanbemonitoredbyusingonlyoneomnidirectionalsurveillancesystem.Itisalsoaplantoemploymicroprocessor modules such as field programmable gate array (FPGA) and Advanced RISCMachine(ARM)forimageprocessingandanalyzingtasksinsteadofacomputertoeffectivelyreducethecostsandpowerconsumptionoftheproposedsystem.Thesetopicswillbeaddressedinfutureworks.
AQ26
K13920_C016.indd 453 1/4/2013 7:24:34 PM
454 ◾ Effective Surveillance for Homeland Security
References 1. Moseley,T.M.2006.Homelandoperations.AirForceDoctrineDocument2-10. 2. Lu,C.andDrew,M.S.2007.Automaticcompensationforcamerasettingsforimagestakenunderdif-
ferentilluminants.Technicalpaper,SchoolofComputerScience,SimonFraserUniversity,Vancouver,BritishColumbia,Canada,pp.1–5.
3. Kawanishi,T.,Yamazawa,K.,Iwasa,H.,Takemura,H.,andYokoya,N.1998.Generationofhigh-resolution stereo panoramic images by omnidirectional imaging sensor using hexagonal pyramidalmirrors.InProceedings of the 14th International Conference in Pattern Recognition,Brisbane,Australia,Vol.1,pp.485–489.
4. Akihiko,T. andAtsushi, I.2004.Panoramic image transformofomnidirectional imagesusingdis-cretegeometrytechniques.InProceedings of the 2nd International Symposium on 3D Data Processing, Visualization, and Transmission (3DPVT’04),Washington,DC.
5. Jeng,S.W.andTsai,W.H.2007.Usingpano-mappingtablesforunwarpingofomni-imagesintopan-oramicandperspective-viewimages,IET Image Processing,1(2),149–155.
6. Jurie,F.1999.Anewlog-polarmappingforspacevariantimaging:Applicationtofacedetectionandtracking,Pattern Recognition,32(55),865–875.
7. Hampapur, A., Brown, L., Connell, J., Pankanti, S., Senior A. et al. 2003. Smart surveillance:Applications, technologies and implications, Information, Communications and Signal Processing, 2,1133–1138.
8. Hampapur,A.,Brown,L.,Connell,J.,Ekin,A.,Haas,N.etal.2005.Smartvideosurveillance,IEEE Signal Processing Magazine,pp.39–51.
9. Green,M.W.1999.TheappropriateandeffectiveuseofsecuritytechnologiesinU.S.schools.Aguideforschoolsandlawenforcementagencies,ResearchReport,SandiaNationalLaboratories,Albuquerque,NM,NCJ178265.
10. Shu,C.,Hampapur,A.,Lu,M.,Brown,L.,Connell,J.etal.2005.IBMsmartsurveillancesystem(S3):Aopenandextensibleframeworkforeventbasedsurveillance,Advanced Video and Signal Based Surveillance (AVSS 2005),Como,Italy,pp.318–323.
11. Wong,W.K.,Tan,P.N.,Loo,C.K.,andLim,W.S.2010.Omnidirectionalsurveillancesystemusingthermalcamera,Journal of Computer Science and Engineering,3(2),42–51(ISSN2043-9091).
12. Lee,L.H.2008.Smartsurveillanceusingimageprocessingandcomputervisiontechniques,BachelorDegreethesis,MultimediaUniversity,Melaka,Malaysia.
13. Wong,W.K.,Chew,Z.Y.,Lim,H.L.,Loo,C.K.,andLim,W.S.2011.Omnidirectionalthermalimag-ingsurveillancesystemfeaturingtrespasserandfaintdetection,International Journal of Image Processing, CSC Journals, IJIP-279,4(6),518–538.
14. Holmes,O.W.1881.The Common Law.Boston,MA:LittleBrownandCo. 15. Tsai, C.F. and Young, M.S. 2003. Pyroelectric infrared sensor-based thermometer for monitoring
indoorobjects,Review of Scientific Instruments,74(12),5267–5273. 16. Hirota, M., Furuse,T., Ebana, K., Kubo, H.,Tsushima, K., Inaba,T., Shima, A., Fujinuma, M.,
andTojyo,N.2001.MagneticdetectionofasurfaceshipbyanairborneLTSSQUIDMAD,IEEE Transactions on Applied Superconductivity,11,884–887.
17. Paul,W.,Herbert,F.,andSoumya,N.2003.EnhancinghomelandsecuritywithadvancedUWBsen-sors,IEEE Microwave Magazine,pp.51–58.
18. http://www.rrs.co.za/products/homeland-security.html 19. http://www.detect-inc.com/security.html 20. Tocci,R.2006.Digital Systems: Principals and Applications,10thedn.UpperSaddleRiver,NJ:Prentice
Hall.ISBN013172. 21. Fischler,M.A.andBolles,R.C.1981.Randomsampleconsensus:Aparadigmformodelfittingwith
applicationstoimageanalysisandautomatedcartography,Communications of the ACM,24,381–395. 22. Corke,P.,Strelow,D.,andSingh,S.2004.Omnidirectionalvisualodometryforaplanetaryrover.In
International Conference on Intelligent Robots and Systems (IROS 2004),Sendai,Japan,4,4007–4012. 23. Durrant,W.H.andBailey,T.2006.Simultaneouslocalizationandmapping(SLAM):PartI.Theessen-
tialalgorithms,Robotics and Automation Magazine,13,99–110.
AQ27
AQ28
AQ29
K13920_C016.indd 454 1/4/2013 7:24:34 PM
Omnidirectional Human Intrusion Detection System ◾ 455
24. Ishiguro,H.,Yamamoto,M.,andTsuji,S.1992.Omni-directionalstereo,IEEE Transactions on Pattern Analysis and Machine Intelligence,14(2),257–262.
25. Huang,H-C.andHung,Y.P.1998.Panoramicstereoimagingsystemwithautomaticdisparitywarpingandseaming,Graphical Models and Image Processing,60(3),196–208.
26. Peleg,S.andBen-Ezra,M.1999.Stereopanoramawithasinglecamera.InProceedings of the IEEE Conference on Computer Vision and Pattern Recognition,FortCollins,CO,395–401.
27. Shum,H.andSzeliski,R.1999.Stereoreconstructionfrommulti-perspectivepanoramas.InProceedings of the Seventh International Conference on Computer Vision,Kerkyra,Greece,pp.14–21.
28. Chen,S.E.1995.Quick timeVR:An image-basedapproach tovirtual environmentnavigation. InProceedings of the 22nd Annual ACM Conference on Computer Graphics,LosAngeles,CA,pp.29–38.
29. Kumar,J.andBauer,M.2000.Fisheyelensdesignandtheirrelativeperformance,Proceedings of SPIE,4093,360–369.
30. Padjla,T.andRoth,H.2000.Panoramic imagingwithSVAVISCAcamera-simulationsandreality,ResearchReportsofCMP,CzechTechnicalUniversity,Prague,No.16.
31. Oh,S.J. andHall,E.L.1987.Guidanceofamobile robotusinganomnidirectionalvisionnaviga-tion system. InProceedings of the Society of Photo-Optical Instrumentation Engineers,SPIE,Vol.852,pp.288–300.
32. Kuban,D.P.,Martin,H.L.,Zimmermann,S.D.,andBusico,N.1994.Omniviewmotionlesscamerasurveillancesystem,UnitedStatesPatentNo.5,359,363.
33. Nalwa,V.1996.Atrueomnidirectionalviewer,TechnicalReport,BellLaboratories,Holmdel,NJ. 34. James,J.K.andMartin,B.2000.Fisheyelensdesignsandtheirrelativeperformance.InProceedings of
the Current Developments in Lens Design and Optical Systems Engineering,SPIE,Vol.4093,360–369. 35. Stun-ningsales.com, Weatherproof CCD color rotating video security camera, Redirecting from
http://www.stun-ningsales.com/homethings/outdoor_securitycameras.htm 36. The-DigitalImage.com,Fisheyelens,redirectingfromhttp://www.the-digital-picture.com/Reviews/ 37. Chahl,J.andSrinivasan,M.1997.Reflectivesurfacesforpanoramicimaging,Applied Optics,36(31),
8275–8285. 38. Gachter,S.2001.Mirrordesignforanomnidirectionalcamerawithauniformcylindricalprojection
whenusingtheSVAVISCAsensor,ResearchReportsofCMP,OMNIVIEWSProject,CzechTechnicalUniversity,Prague,No.3,2001.Redirectedfrom:http://cmp.felk.cvut.cz/projects/omniviews/
39. Svoboda,T.1999.Centralpanoramiccamerasdesign,geometry,egomotion.PhDtheses,CenterofMachinePerception,CzechTechnicalUniversity,Prague,1999.
40. Davis,J.W.andSharma,V.2007.Background-subtractioninthermalimageryusingcontoursaliency,International Journal of Computer Vision71(2),161–181.
41. Huang,D.S.,Wunsch,D.C.,Levine,D.S.,andJo,K-H.2008.Advancedintelligentcomputingtheo-riesandapplications:withaspectsoftheoreticalandmethodological issues.InProceedings of the 4th International Conference on Intelligent Computing (ICIC 2008),Shanghai,China.
42. Wu,C.J.andTsai,W.H.2009.Unwarpingofimagestakenbymisalignedomnicameraswithoutcam-eracalibrationbycurvedquadrilateralmorphingusingquadraticpatternclassifiers,Optical Engineering,48,087003(1)–087003(11).
43. George,W.andSiavash,Z.2000.Robustimageregistrationusinglog-polartransform,InProceedings of the IEEE International Conference on Image Processing,Vancouver,BritishColumbia,Canada.
44. Traver,V.J.andAlexandre,B.2010.Areviewoflog-polarimagingforvisualperceptioninrobotics,Robotics and Autonomous Systems,58,378–398.
45. José,S.V.andAlexandre,B.2003.Vision-basednavigation,environmentalrepresentationsandimag-inggeometries,InVisLab-TR 01/2003, 10th International Symposium on Robotics Research,R.JarvisandA.Zelinsky(Eds.),Springer,NewYork.
46. Owens, J.,Hunter,A., andFletcher,E. 2002.A fastmodel-freemorphology-basedobject trackingalgorithm,InBritish Machine Vision Conference,Cardiff,U.K.,Vol.2,pp.767–776.
AQ30
AQ31
AQ32
K13920_C016.indd 455 1/4/2013 7:24:34 PM
AUTHORQUERIES[AQ1] Pleasechecksentencestarting“Inthefaceof…”forclarity.[AQ2] Pleasecheckifedittosentencestarting“Theauthoritycan…”isokay.[AQ3] Pleasecheckifthefixedrunningheadisok.[AQ4] Pleasecheckentrystarting“makingtheinformation…”forclarity.[AQ5] Pleasecheckifinsertedclosingparenthesisisokay.[AQ6] Pleasecheckifedittosentencestarting“However,PIR-basedmotion…”isokay.[AQ7] Pleasecheckifedittosentencestarting“Hence,inthesecases…”isokay.[AQ8] Pleasecheckifedittosentencestarting“Glass-breakdetectors…”isokay.[AQ9] Pleasecheckifedittosentencestarting“thismaysomehow…”isokay.[AQ10] Figures16.1through16.4areofpoorquality.Pleaseprovidebetterqualityfigures.[AQ11] Pleasechecktheorderofsectionheadings.[AQ12] Pleasecheckentrystarting“keepsaneye…”forclarity.[AQ13] Pleasecheckifedittosentencestarting“Thismakesthe…”isokay.[AQ14] Pleasecheckifedittosentencestarting“Thisleadsto…”isokay.[AQ15] Pleasecheckifedittosentencestarting“thesecameraslook…”isokay.[AQ16] Pleasecheckifedittosentencestarting“However,thecurrent…”isokay.[AQ17] Pleasecheckifedittosentencestarting“Thisalgorithmwas…”isokay.[AQ18] Pleasecheckif“capturingimages”shouldbechangedto“capturedimages”.[AQ19] Pleasecheckifedittosentencestarting“Sincefish-eyelens…”isokay.[AQ20] Pleasecheckif“unwrap”shouldbechangedto“unwarp”.[AQ21] Pleasecheckentrystarting“processthemsmoothly…”forclarity.[AQ22] Pleasenote thatEquations16.20 to16.29have changed toEquations16.12 to16.21
forsequentialorder.Pleasecheck.[AQ23] PleasecheckthewherestatementprovidedforEquation16.12forcorrectness.[AQ24] Pleasecheckifthephrase“whereQisthethresholdvalueofthedifferencebetweensum
ofRGBvalue for aparticular current imagepixel toprevious imagepixel” shouldbemodifiedas“whereQisthethresholdvalueofthedifferenceofthesumofRGBvaluesbetweenaparticularcurrentimagepixelandpreviousimagepixel”.
[AQ25] Pleasechecksentencestarting“AsforHvalue…”forcompleteness.[AQ26] Pleasecheckifedittosentencestarting“Inthefuture…”isokay.[AQ27] Originallyreferences2and21and3and25wereoneandthesame;hence,therepeated
references have been deleted and renumbered both in the text and list accordingly.Pleasecheck.
[AQ28] PleaseprovidepagerangeforRefs.[4,41,33].[AQ29] Pleaseprovidetheauthor/owner,articletitle,andtheaccesseddatesforRefs.[18,19].[AQ30] PleaseprovidethelocationoftheconferenceforRefs.[31,34].[AQ31] PleaseprovidetheaccesseddatesforRefs.[35,36].[AQ32] Pleaseprovidein-textcitationsforRefs.[42].
K13920_C016.indd 456 1/4/2013 7:24:34 PM