eyedetect - hofstra university€¦ · analytics, the process gained from visualization. people...

RESEARCH POSTER PRESENTATION DESIGN © 2012

www.PosterPresentations.com

Detecting human behaviors in video surveillance is a daunting task for realworld applications. For example, if a security guard is on the job whilemonitoring multiple live video streams, and they’re supposed to make surethat nothing odd is going on, let alone if there are any human activity, thenthere’s certainly room for error for that security guard to make. They canmistakenly spot a human for something else, or even look away frommonitoring the screen. Another example is if a room can only fit a certainamount of people and anything more than that threshold is a fire hazard,one person would certainly try to do a head count to see if they arebreaking the law or not. For these reasons, an interactive tool that isdesigned to support this task to abstract and automate the complexity andprocess of manual surveillance is an important step towards the capabilitiesof surveillance. This poster is about the construction of EyeDetect and itsimplementation. We introduce the concept of integrating human-detectingalgorithms to a Raspberry Pi with an attached camera and the benefits itproduces towards Data Analytics which presents the potential it hastowards automated Visual Analytics.

ABSTRACT

INTRODUCTION

• Setup EyeDetect system in the Raspberry Pi where human shape wouldbe detected in a video stream

• Transition the process of manual surveillance to automated videosurveillance

• i.e. if EyeDetect detects X amount of people in a room, thensend an automated email notification to the user

• Be able to be notified even if there’s no one at the premise. An examplewould be if a person is away from his/her house but has EyeDetectinstalled. They then would know if the mailman dropped off their mail,or even if someone was trying to get in their car!

GOALS

There was heavy use of the OpenCV library in our EyeDetect code.Implementation flow of EyeDetect is:• Detecting motion in the video stream• We detect motion by computing the absolute difference between the

current frame and the last frame• Min_area -> variable or threshold value to determine if EyeDetect

should ignore this type of movement or not• Draw bounding box of that motion• After detecting motion, now run human-detecting algorithm

The reason we detect motion first is because the Raspberry Pi is verylimited in terms of resources, so running the human detection algorithm onevery frame would hurt the quality of performance of thesystem. Therefore we pre-process the frame by using a motion detectionalgorithm and if there is motion detected then run the human detectionalgorithm.

Integrating Dropbox upload was implemented by using the DropboxAPI. One thing to mention here is we used OAuth 2.0 authenticationthrough Dropbox. This is a way for users to share account informationwith third party applications, in this case EyeDetect.

Some basic configuration settings can also be tweaked such as the fps thatthe Pi should run, the resolution of the frame, the time difference betweeneach dropbox upload, the minimum area for the system to consider motion,and more.

METHODS

Conclusions

Although results of the EyeDetect code shows good results of humandetection, they aren’t perfect. We get multiple bound bounding boxes eventhough there is only one person moving around the room – this is far fromideal. We do have a fix for this using the non-maxima suppressiontechnique, but if we’re able to detect one one human person per onebounding object detected by the algorithm, then that would be perfect.In the motion detection technique, we also see some false-positives whenthere’s shadows and reflections on a wall. In order to relieve theseproblems, there’s more pre-processing that needs to happen using theOpenCV library, but that takes away a lot of resources that the Pi needs.By keeping our motion detection and human detection simple and fast, thesystem is a compact and light weight framework for automatedsurveillance. Admittingly, it is not a perfect detection system, but it stilldoes a fairly good job considering the resources provided by the RaspberryPi.

REFERENCES1. Itseez, OpenCV Reference Manual, 2.4.9.0, http://opencv.org

2. Dropbox, Python Dropbox Core API https://www.dropbox.com/developers-v1/core/start/python

3.Raspberry Pi, Raspberry Pi Documentation, https://github.com/raspberrypi/documentation

4. Adrian Rosebrock, OpenCV3 Tutorials, www.pyimagesearch.com

AcknowledgementsMarvin Dizon – Owner of EyeDetectAmeer Humza Malik – Co-Owner of EyeDetectDr. Simona Doboli – Senior Seminar AdvisorDr. Krishnan Pillaipakkamnatt – Marvin’s AdvisorDr. Ed Currie – Ameer’s Advisor

Over the years, surveillance systems have allowed humans to track andmonitor a certain area for the purpose of managing, protecting, orinfluencing that certain area. Currently, surveillance systems have to bemanually overlooked and monitored by people such as security guards inorder to decipher data from the video stream itself. This is key to VisualAnalytics, the process gained from visualization. People have usedtechnology to amplify the ability of surveillance through methods such asautomated analysis. When automated analysis is used, then data gatheringmethods are used which is then applied to generate predictive models ofthe original data. EyeDetect brings everything in full circle as it focuses onvisual analytics that produces results that ultimately increases a user’scapability of surveillance. The core of EyeDetect is it’s ability to detecthuman shape in live video streams. Data can then be gathered from thissuch as how many people in a room, if someone shouldn’t be in a room, orif the mailman is delivering the mail, and the user can act upon thisinformation.

Advisors:Dr.KrishnanPillaipakkamnat,Dr.EdwardH.Currie Spring2017MarvinDizonandAmeerHumzaMalik

EyeDetect

SYSTEMDESIGN

When running human detection algorithm our CPU was using around65%-80% of it’s resources. With the algorithm of detecting motion first,then human detection, our CPU was using around 40%-60%. Definitely animprovement.Another optimization technique we used was the preprocessing of framesbefore motion was detected. There were three transformations done beforetrying to detect motion:1. Rescale the frame to the lowest possible resolution (set the width =

500)2. Convert the image to greyscale (for less color)3. Apply the Gaussian Filter on the image. The Gaussian Filter just blurs

the image. Not every frame will be 100% the same, therefore we needto blur the image in order to lessen the possibility of false-positives sothat the system doesn’t detect any non-important small movements,high noise frequency, etc

The results definitely showed improvement as there was previousdetections where there was clearly no movement.

Results showing positive detection of human shape

RESULTS

IMPLEMENTATION

Algorithms that was used in our methods included functions from theOpenCV library, Dropbox API integration, the picamera package foreasy access to the modular camera, imutils package for simple imageprocessing operations (like rescaling, color translation, and more) andmany built in functions in python such as: smtplib, email, json, time,and more.

Histogram of Oriented Gradients and Linear Support VectorMachinesWe used the technique Histogram of Oriented Gradients and LinearSupport Vector Machines from OpenCV to detect a certain object wewant to detect. In this case, EyeDetect wants to detect humans.OpenCV is very simple, yet powerful that, to initialize a function thatdetects an object, the simple call issetSVMDetector(cv2.HOGDescriptor_getDefaultPeopleDetecter())What this means is that we are passing, as arguments, the data set forwhat a shape for a person is based on the very popular INRIA Persondataset.

Non-Maxima SuppressionAnother method used from OpenCV is the non-maxima suppressiontechnique to threshold bounding boxes. Although OpenCV provides thefunctions to detect human shape, it most certainly will have many false-positives, therefore there would be many bounding boxes that wouldconsider one person, to be many persons. Applying non-maximasuppression takes care of this and maintains any overlapping boxes thatis the same person.

A Raspberry Pi is a small, lightweight but compact computer. Itwill use a camera module. The EyeDetect code will then be ran onthe Raspberry Pi and will be processing images frame by framefrom the camera. EyeDetect uses many functionalities from theOpenCV library along with infusing Dropbox integration using it’sCore API technology.

FUTURE

• MakeasimpleGUItobeabletoaccessEyeDetectandconfiguresettingsremotely

• MakeRaspberryPivideostreamaccessibleviasmartphone

• UpgradePiandimplementserver-sidelivedataprocessinginordertoeliminatetheproblemofCPUbeingalimitedresource