facial recognition access control - edgeedge.rit.edu/edge/c12407/public/documents/proposal.pdf ·...
TRANSCRIPT
Facial Recognition Access Control 1
Facial Recognition Access ControlA project to develop and implement a hands-free and secure method of gaining
entry to a home.
Michael CresswellJosh KramerChris Ziemba
04/27/2013
Facial Recognition Access Control 2
Table of ContentsI. Overview ................................................................................................................. 3
Needs Statement ..........................................................................................…... 3Objective Statement ............................................................................................ 3
Figure 1.1 System Overview....................................................... 3II. Requirements Specification .................................................................................. 4
Customer Needs or Marketing Requirements ..................................................... 4Engineering Specifications .................................................................................. 4
Table 2.1 Engineering Specifications ......................................... 4Objective Tree...................................................................................................... 5
Figure 2.1 Objective Tree with appropriate weights ................... 5III. Concept Selection ................................................................................................ 6
Table 3.1 Existing Systems ........................................................ 6Table 3.2 Comparison of Biometric Identification Methods......... 6Table 3.3 Comparison of Interface Choices................................. 7
IV. Design .................................................................................................................. 8Table 4.1 Comparison of Computing Platform Options ............. 8Figure 4.1 System Level Diagram ............................................. 8Figure 4.2 State Diagram of the Application............................... 9Figure 4.3 Functional diagram of Kinect..................................... 9Figure 4.4 Schematic for circuit to control 12V Door
Power Line with BeagleBoard GPIO ........................ 10Figure 4.5 OpenNI flowchart for retrieving Kinect data............... 10
V. Constraints and Considerations ........................................................................... 13VI. Cost Estimates .................................................................................................... 15VII. Testing Strategy .................................................................................................. 16
Unit Tests …......................................................................................................... 17Partial Integration ................................................................................................ 20Full Integration ..................................................................................................... 24
VIII. Risks .................................................................................................................. 27Risk Mitigation ..................................................................................................... 27
IX. Milestone Chart .................................................................................................... 29
Facial Recognition Access Control 3
I. Overview
1. Needs StatementMechanisms for door locking have not come very far in the last several hundred years.
Jagged pieces of metal are still being used to verify the identity of someone trying to gainaccess. Even given the increasing complexity of locks and the relatively recent improvements ofenterprise solutions, there is a need for a more advanced solution for the home and smallbusiness that allows convenient access while reducing chances of loss or theft and maintainingease and speed of use.
2. Objective StatementThe objective of this project is to develop and implement a concise and secure method of
gaining entry to a home without the use of a key. The system will use facial recognition to restrictaccess through a door. Access will be granted without any specific intervention of the user. Thissystem will achieve the goal of modernizing the basic homeowner’s door without compromisingon security.
Figure 1.1 System Overview
The Kinect will reside mounted to the door and will be directed where any potential usercan be detected, similar to the figure above. The system will start by scanning for a face throughfacial detection and then will proceed to recognize the face while also checking to make sure it isnot a flat image being held to the camera.
Facial Recognition Access Control 4
II. Requirements Specification
1. Customer Needs or Marketing Requirements1. Authorized users must be able to enter their home at all times of the day.2. The owners must be able to allow other persons to enter remotely.3. The system must be able to be configured by the customer.4. The system must recognize a user relatively quickly, without the need to wait for a
long time.5. The system must be reasonable for a consumer to install and use in their home.6. The system must only allow authorized users to enter.
2. Engineering Specifications
RelatedCustomerNeeds
Engineering Requirement Justification
1 1, 4, 5 The system must perform facialrecognition and take appropriate actionwithin 5 seconds of detecting a face in thefield of vision.
Any more than 5 seconds ofwaiting and a user will getimpatient. Any less may notbe possible for the hardware.
2 1, 6 The system must correctly authorize atleast 80% of authorized access attempts,and the system may only allow at most10% of unauthorized access attempts.
The system must have a levelof accuracy that makes itreliable and consistent.
3 1, 5, 6 The system must have an overridemechanism to allow access even when anauthorized user is mistakenly deniedaccess.
The user should be able toopen the door even when thefacial recognition algorithmcontinues to not correctlyidentify him/her.
4 1, 5, 6 The system must have an overridemechanism to allow access even whenpower to the device is cut.
The user should be able toopen the door even during apower outage.
5 2, 3, 6 The system must provide an interface toadd and remove up to 10 authorizedusers.
The addition of familymembers and friends to thesystem is as essential as theability to duplicate a key orlend one to a friend.
Table 2.1 Engineering Specifications
Facial Recognition Access Control 5
3. Objective Tree
Figure 2.1 Objective Tree with appropriate weights
Facial Recognition Access Control 6
III. Concept Selection
There are many current systems that allow for restricted access through a door. Thegoal of restricted access is to make it easy for an authorized user to get through the door whilemaking it difficult for a nonauthorized user to get through it. Table 3.1 shows several differentways of authenticating a user before allowing them through a door.
Pros Cons
Manual Lock(metal key)
• Reliable• Trusted by users
• ‘Pickable’• Keys can be lost or stolen• No record of who has entered when• Not much improved security in the last several decades
Magnetic swipe • Reliable• Data can’t be skimmed like RFID
• Card data can be spoofed• Cards can be lost or stolen
RFID (badge,chip, card)
• More obscure• Reliable• Convenient chip can remain in pocket while authenticating
• Chips can be lost/stolen.• Chip data can be skimmed
Biometric(fingerprint,retinal scan, etc.)
• Secure• Can’t be stolen
• Expensive• Can be inconvenient
Table 3.1 Existing Systems
In general, we want to combine the secure elements of a biometric solution with the lowuser interaction of an RFID solution to get the perfect lowcost solution for home security. To dothis, several conceptual components need to be weighed against one another such that the bestsolutions will stand out as the optimal choice. Easeofuse and convenience are the priorities,followed by the security and price.
Weight Fingerprint Facial Recognition Retinal Scan
.4 Easeofuse .25 .6 .15
.25 Secure .4 .25 .35
.35 Cheap .5 .4 .1
1 Total .375 .4425 .1825Table 3.2 Comparison of Biometric Identification Methods
Facial Recognition Access Control 7
To physically unlock the door there are 2 main options. The first is to physically attach aservo to a structure that will turn the knob. However, this system is prone to failure, isn’tcompatible with all doors, and could stop working if the door was too tight. The other option is touse a door latch commonly used with magnetic swipe access. Instead of the door knob turning,the door frame gives way to the door, without actually turning the knob. This second option ismuch more reliable and unobtrusive. Additionally, the hardware is relatively cheap and it alreadycan be controlled electrically.
It is necessary for a facial recognition product have an administration interface toconfigure access settings such as access times and authorized users. Several options areconsidered in the following table. The most practical method for providing this interface isthrough a web interface. A selfhosted website is a very standard way of configuringnetworkconnected devices in the home, such as a wireless router.
Dedicated TouchScreen
Web Interface Dedicated ASCIIScreen
Cheap .05 .65 .30
Easytouse .4 .4 .2
Featurerich .3 .6 .1
Total .75 1.65 .6
There are several options to train the system to specific users. The basic idea behindtraining the system is giving it example images of the user. When it tries to recognize a person itcompares the reference images to the new image. The source of these reference images will bestored images from the past.
Facial Recognition Access Control 8
IV. Design
The first and most critical component to the overall design is the platform on which thesystem will be running. Below is Table 4.1 which shows the various options as far as computinggoes and weighs them based on their speed and cost. In weighing these key aspects itbecomes clear that the BeagleBoard is the best choice. The BeagleBoard will offer us a goodamount of CPU power while remaining low cost which is a chosen goal for this project.
Weight Raspberry Pi BeagleBoard PandaBoard
Speed/Power 0.25 0.2 0.35 0.45
Cost/Availablity 0.2 0.4 0.4 0.2
Compatibility 0.55 0 0.5 0.5
Total 1 0.13 0.4425 0.4275
Table 4.1 Comparison of Computing Platform Options
As seen in the figure below BeagleBoard is the central controller in the project. All otherdevices interact with it in some way. It will run Ubuntu, a Debianbased linux distribution. The SDcard is inserted into the BeagleBoard and is used to store Ubuntu, the facial recognitionsoftware, and all associated training images and profiles. The BeagleBoard is powered through a5V power adapter.
Figure 4.1 System Level Diagram
Facial Recognition Access Control 9
The processing hardware will be running the control software which will determine theoperations being performed such as door unlocking and detection. The state diagram belowdisplays the generalized flow of the system and the reason for moving between states. It issimplified in a way that error cases and UI interactions are not detailed while still providing asense of flow and logic.
Such error cases which are ones where the manual lock is triggered and the door isdetected as open, which will stop the control flow and reset the system to standby.
If any computation or processing step is met with errors, the system will go into a softreset where it will reenter the standby mode. This soft reset will consist of disposing of all localvariables (frames, crops, etc.) and returning to the lowframe rate mode.
Figure 4.2 State Diagram of the Application (C.I. = Confidence Interval)
The Kinect is a camera device used to gather images of the surrounding environment forprocessing. It can gather RGB and IR images which can be used to determine depth informationin addition to seeing the surroundings. This information is sent to the BeagleBoard over a USB2.0 port. The Kinect also requires a standard wall adapter since it draws more power than canbe delivered over USB.
Figure 4.3 Functional diagram of Kinect.
Facial Recognition Access Control 10
An electronic door strike will be used to allow access through the door. A GPIO on theBeagleBoard is used to signal the door strike to unlock. The circuit in Figure 4.3 shows how aBJT is used to enable and disable the power to the electronic door strike. This is requiredbecause the BeagleBoard cannot supply neither the required voltage nor the required current forthe strike.
Figure 4.4 Schematic for circuit to control 12V Door Power Line with BeagleBoard GPIO
OpenNI is used to interface directly with the Kinect via USB. It works through the use ofwhat it calls production nodes. Each production node is assigned to what the Kinect can outputin a way that they are configurable and separate. For example, the RGB camera and depthcamera will be configured as two separate production nodes that output 640x480 (VGA)resolution. These nodes are captured each time a call to the OpenNI function which refreshesthe production nodes for new information (i.e. images). They can then be copied into arrayobjects and operated upon by other processing functions such as those from the OpenCVlibraries.
Figure 4.5 OpenNI flowchart for retrieving Kinect data
OpenCV will provide us with the bulk of the processing required to be done on eachimage in order to properly recognize authorized users. The facial recognition process will require
Facial Recognition Access Control 11
the modeling function to be trained which requires equal sized crops of the person and anassociated label. The crops will be obtained through facial detection which processes a wholeframe (RGB) for any suitable faces and then returns the cropped section of the frame. Thesecan then be used to train the model or run it through the prediction model with a returnedconfidence interval.
To connect the device to the network for configuration and training an Ethernetconnection will be used. The BeagleBoard will be assigned a dynamic IP address from theDHCP server. If this IP address is visited through a browser using port 80 the configurationwebsite will be shown.
This website will be served from the BeagleBoard using a lightweight web server. Thefollowing three are the most popular and most uptodate lightweight web servers: Hiawatha,lighttpd, and nginx. Hiawatha was built for security in mind, and nginx was built for hightrafficweb sites and flexibility with different languages and server configurations. For a simple,straightforward, lightweight install, lighttpd seems to be the choice. It is the most commonly usedserver on embedded devices, and it seems to be the best choice for this project. Due to priorteam member knowledge, SQLite will be used as a database and PHP will be used as aprogramming language.
The face training will happen with two methods. The first will be with the new userstanding at the device and clicking a “capture” or “recognize” button in the web interface. Thesecond will involve adding previous images to a user profile using integration with Facebook. Thesite will contact Facebook and request access to a user’s photos and the photos of their friends.These images will be verified by the user and then used to construct a profile of the new user.
The image below shows the unstyled structure of the configuration site once anadministrator logs in. It lists which users currently have profiles in the system, and which cancurrently unlock the door names highlighted in green can currently unlock the door, whilenames highlighted in red can not. The administrator can delete these users. Additionally, itindicates which user’s profiles were generated using Facebook images. There is a button to adda profile to the system, either by using the Kinect’s camera or a Facebook profile. Clicking on auser’s name will bring the administrator to the settings page for that user.
Facial Recognition Access Control 12
A mockup of the settings page for specific users is shown below. The times that theuser can use the system can be viewed and changed. Multiple date and time ranges can bespecified. The administrator can also recapture the person’s profile with the Kinect’s camera, orrescan the person’s Facebook profile .
Facial Recognition Access Control 13
V. Constraints and Considerations
ExtensibilityThis project could serve to prove the practicality of having “smart” appliances. If
implemented correctly and with enough accuracy, families could learn to trust the technologyand help create a push for more technologically advanced homes. The facial recognitionspecifically could lend itself to other inhome applications once it is realized how lowcost it canbe.
ManufacturabilityAssembly of this system would include affixing a mounting bracket to the door while
allowing it to remain protected from external conditions (rain, snow, etc.). The processing wouldbe internal to the door, thus requiring a few alterations to the door itself. The power and control(USB) lines will be run in some weathersafe manner and the system will need to be plugged intoa wall socket at the endpoint. Given all of these requirements of the system it is quite feasible tomanufacture on a large scale simply by making the alterations to the door standard andassembling the system at a later stage.
ReliabilityThe performance of the system will rely heavily on the ability for faces to be detected in a
multitude of environments. Theoretically, if the recognition model is trained and updated in amanner that it remains accurate, there should not be any issue with the subject changing orgrowing so the performance remains tied to the versatility of the system. With enoughpreprocessing and adequate lighting, the system should be able to recognize any subjectregardless of the weather or time of day.
Intellectual PropertySince the bulk of our software will be written with open source libraries and interfaces,
there is no risk of paying licensing fees for the release of this system. However, were thissystem be released as a full featured product, the Kinect component would need to be revisitedfor an alternative that would not require the support and consent of Microsoft.
The use of the Facebook logo throughout the interface is permitted as long as it is usedto communicate the project’s relationship with Facebook. As long as it being used in the contextof integrating the recognition with Facebook there should be no issue.
Ethical IssuesMounting a user controlled camera at the face of a moderately trafficked area poses an
ethical issue of what the system, when controlled by the user, is capable of. The camera itself iscapable of taking high resolution captures which in the end are going to be accessible by theowner. This issue will need to be addressed through the means of notifying any user that thedoor is featured with a camera. This can be done through a sticker or warning placed on the
Facial Recognition Access Control 14
door with the message explaining the features of the door.
Environmental ContextThis system will have a small impact on the environment. The BeagleBoard and the
Kinect together use a small amount of electricity. The system will not generate any wasteproducts or other substances that may harm the environment.
Health and Safety IssuesInaccurate facial recognition poses a large safety risk. While an override mechanism will
allow owners back into the house, authorized users denied access or unauthorized usersgranted access both pose risks. An authorized user refused access may not be able to care forpets, plants, etc., like they may need to while an owner is away. An unauthorized person wouldhave the ability to burgle or perform other harmful actions within the house.
Facial Recognition Access Control 15
VI. Cost Estimates
Item Quantity Cost Availability Our Cost
Kinect 1 $110 Department $0
Electronic Door Strike 1 $29 Department $0
BeagleBoard 1 $150 Department $0
4GB SD Card 1 $7 Team member $0
12v Power Supply 1 $3 Department $0
Total $299 $0
Facial Recognition Access Control 16
VII. Testing Strategy
Our testing strategy will be such that each individual aspect of the project is focused onso as to fully test the whole system. The tests will divided into unit tests, partial integration tests,and full integration tests. The unit tests will be focused on a component and the result willdetermine whether or not that component functions on its own. The partial integration tests willcombine two or more components in a test to make sure the system will be cohesive at anacceptable level. And finally, the full integration tests will test the entire system in its workingconfiguration for any final bugs that arise in combining each component. It is important that thelower level tests be white box so that we can deduce whether or not the inner working areperforming as desired.
Facial Detection and RecognitionThe facial recognition system will be tested in a variety of lighting environments and with
a multitude of people. Since the goal is to be able to authorize multiple persons for one doorway,there will need to be tests involving a range of faces. The recognition itself will also be testedseparately from facial detection since detection is the first step in recognition. Once the detectionand recognition is confirmed to be working in an acceptable variety of conditions, the system canbe integrated further for testing purposes.
Door Unlocking MechanismThe unlocking mechanism consists of the door strike wired up to the BeagleBoard. This
component will need to be tested in a way where we can unlock the door for a period of timebefore reengaging the lock. Since the door strike operates with a 12V line, the BeagleBoard willcontrol the electricity via a transistor. After verifying the operation of the door strike using theBeagleBoard, it will then be integrated with the Kinect and tested when coupled with therecognition.
User InterfaceThe user interface will need to be tested in a way that will verify the owner can easily train
and alter the database for access control. It will also possess the ability to control the lockdirectly from the interface so it is important that this feature does not interfere with the algorithmalready running on the BeagleBoard. A large portion of this testing can be done without the restof the components integrated as the interface is fairly independent and as long as it sends theappropriate requests to the system it can be deemed functional.
Facial Recognition Access Control 17
Unit Tests
Facial Recognition Access Control 18
Facial Recognition Access Control 19
Facial Recognition Access Control 20
Partial Integration Tests
Facial Recognition Access Control 21
Facial Recognition Access Control 22
Facial Recognition Access Control 23
Facial Recognition Access Control 24
Full Integration Tests
Facial Recognition Access Control 25
Facial Recognition Access Control 26
VIII. Risks
Risk Potential Sources of Failure
1 Use of BeagleBoard and Kinect Image processing workload Storage and RAM
2 Communication with Kinect Getting camera data correctly Configuration possibilities Reliability
3 Web Interface for Facial Recognition Making interface secure Seamless Facebook integration Web server reliability
4 Facial Recognition Algorithm Lighting conditions and its effect onrecognition
Misrecognition Process time
Table 8.1 Summary of Potential Risks
Risk Mitigation
1. Use of BeagleBoard and KinectThe BeagleBoard provides the means to process images and retrieve data from
the Kinect. The BeagleBoard is powered by a 720MHz ARM CortexA8 CPU. This is agood processor considering the price of the unit, but it may not be the best available forimage processing. The BeagleBoard also features a midlevel GPU that can be used byOpenCV library functions capable of using OpenCL. This will provide an additional corefor offloading some processing to.
Memory on the BeagleBoard is shared between main memory and the GPU. TheGPU and operating system will take a significant portion of the 256MB of main memoryand not leave a lot for image processing. An SD card can be attached to theBeagleBoard for storing images and the recognition logic.
The risk of limited processing power and memory will be minimized through thefacial detection and recognition algorithms. Hardware requirements will be a major factorwhen selecting algorithms.
2. Communication with KinectIn order to interface with the Kinect, a thirdparty open source driver will be used
for retrieval of the camera’s information. The driver being used is developed by OpenNIand allows for easy retrieval of both the depth and RGB frames and as a bonus works
Facial Recognition Access Control 27
well with our image processing library, OpenCV. The use of this preprogrammed librarywill minimize the risk of interfacing with the Kinect through a library or feature we wouldhave to develop.
3. Web Interface for Facial RecognitionAn issue with providing a web interface is its susceptibility to hacking. The web
interface is the easiest attack vector besides physically trying to break down the door. Forinstance, a hacker could add themselves to the facial database and then the systemwould let them in. To solve this problem the web interface will, by default, only beaccessible from the local network. Therefore, assuming it is secured, access will beprotected with the router.
Facebook integration will be possible only for friends that have allowed Apps offriends to see their pictures. Many Facebook users have turned this option off, which maylimit the usefulness of this feature. However, this will not make the system break; rather,it will simply make this feature slightly less useful.
Another risk of Facebook integration is the fact that many users tag themselves inimages they are not actually in. If these images are used to construct a profile naïvely itcould ruin the profile. Therefore, the images retrieved will need to be reviewed by theadministrator before they are used in the profile.
If the web server aboard the BeagleBoard stops working this interface will nolonger be accessible. This is an issue that would hopefully be solved with a reboot of themicrocontroller. If the web server goes down the facial recognition should continue tofunction, so this risk hopefully does not have any immediate consequences should itoccur.
4. Facial Recognition AlgorithmThe facial recognition process requires that first, a face be detected, and if there
is a face it needs to be cropped from the frame and sent to the recognition functionand/or training function of the algorithm. There will be additional preprocessing done aswell to attempt to minimize the differences in the environmental effects on the algorithms.
For facial detection, we will use haarcascades for the frontal face only. Thereason for this is since the subject will only ever be looking at the door straight on, itwould be beneficial for the algorithm to not pick up side profiles and the like.
The recognition portion will be handled by a fisherface algorithm. This algorithmshows a higher tolerance to changes in the lighting conditions which is crucial for thepurpose of this system. It is required that the cropped faces be of the same dimensionsfor this algorithm to work. This means there will be a standard sizing for each of thecrops.
IX. Milestone Chart
Facial Recognition Access Control 28
Week Milestone Task Expected CompletionDate
ResponsibleParty
1 20123W4 Submit Preliminary Project Proposal March 25 2013 Kramer
2 20123W4 Obtain Microcontroller March 29 2013 Cresswell
3 20123W8 Facial Detection SubsystemDemonstration
April 22 2013 Ziemba
4 20123W8 Establish point of contact with Dr.Andreas Savakis for consultation onfacial recognition algorithms
April 25 2013 Cresswell
5 20123W8 Facial Recognition SubsystemDemonstration
April 26 2013 Ziemba
6 20123W8 Facebook Image Fetching SubsystemDemonstration
April 27 2013 Kramer
7 20123W10 Finalized Project Proposal May 8 2013 Cresswell,Kramer
8 20124W2 Web interface settings, writing toSQLite database.
June 6, 2013 Kramer
9 20124W3 Facebook integration working, creatingprofiles
June 13 2013 Kramer
10 20124W3 Facial Recognition Identifies FacesAccurately 2D
June 13 2013 Ziemba
11 20124W4 Facial Detection Running onBeagleBoard
June 20 2013 Cresswell
12 20124W5 Facial Recognition Identifies FacesAccurately distinct from image 3D
June 27 2013 Ziemba
13 20124W5 Door Constructed, BeagleBoard GPIOtalking to Door Strike
June 27 2013 Cresswell,Kramer
14 20124W7 Full Integration Test Success July 11 2013 Cresswell,Kramer,Ziemba