fpga accelerated abandoned object detection · rajesh rohilla, aman raj, saransh kejriwal, dr....
TRANSCRIPT
![Page 1: FPGA Accelerated Abandoned Object Detection · Rajesh Rohilla, Aman Raj, Saransh Kejriwal, Dr. Rajiv Kapoor DELHI TECHNOLOGICAL UNIVERSITY. Problem Statement •Abandoned objects](https://reader034.vdocuments.us/reader034/viewer/2022051807/600381b23e66da0e46324fa0/html5/thumbnails/1.jpg)
FPGA Accelerated Abandoned Object Detection
Rajesh Rohilla, Aman Raj, Saransh Kejriwal, Dr. Rajiv Kapoor
DELHI TECHNOLOGICAL UNIVERSITY
![Page 2: FPGA Accelerated Abandoned Object Detection · Rajesh Rohilla, Aman Raj, Saransh Kejriwal, Dr. Rajiv Kapoor DELHI TECHNOLOGICAL UNIVERSITY. Problem Statement •Abandoned objects](https://reader034.vdocuments.us/reader034/viewer/2022051807/600381b23e66da0e46324fa0/html5/thumbnails/2.jpg)
Problem Statement
• Abandoned objects - a common sight at public places like railwaystation, public transport, marketplace etc.
• Can be dangerous for people if they contain explosive materialplanted by terrorists.
• CCTV monitoring for such objects needs manpower which can bedifficult if area to be monitored is large.
• An automatic system is needed in place to detect such abandonedobjects.
![Page 3: FPGA Accelerated Abandoned Object Detection · Rajesh Rohilla, Aman Raj, Saransh Kejriwal, Dr. Rajiv Kapoor DELHI TECHNOLOGICAL UNIVERSITY. Problem Statement •Abandoned objects](https://reader034.vdocuments.us/reader034/viewer/2022051807/600381b23e66da0e46324fa0/html5/thumbnails/3.jpg)
Snapshot of a typical overhead surveillance footage
What if the camera itself could locate unattended objects, and display their highlighted images to the security personnel ?
![Page 4: FPGA Accelerated Abandoned Object Detection · Rajesh Rohilla, Aman Raj, Saransh Kejriwal, Dr. Rajiv Kapoor DELHI TECHNOLOGICAL UNIVERSITY. Problem Statement •Abandoned objects](https://reader034.vdocuments.us/reader034/viewer/2022051807/600381b23e66da0e46324fa0/html5/thumbnails/4.jpg)
How do we do it?
• We utilized the following concept:
A video frame- I without object and II with objectVariation of pixel intensity at encircled position inFig.(I)-(II) with incoming frames. Averaging is performedover all the coming inputs video frames, highlighting effectof introduction of an object that is black in this case.
![Page 5: FPGA Accelerated Abandoned Object Detection · Rajesh Rohilla, Aman Raj, Saransh Kejriwal, Dr. Rajiv Kapoor DELHI TECHNOLOGICAL UNIVERSITY. Problem Statement •Abandoned objects](https://reader034.vdocuments.us/reader034/viewer/2022051807/600381b23e66da0e46324fa0/html5/thumbnails/5.jpg)
Algorithm
Obtaining Reference Static Frame
For each such pixel value a queue Q(i,j) of
size N, a sum of pixel values S(i,j) and
average of pixel values A(i,j) is maintained
over the incoming frames.
![Page 6: FPGA Accelerated Abandoned Object Detection · Rajesh Rohilla, Aman Raj, Saransh Kejriwal, Dr. Rajiv Kapoor DELHI TECHNOLOGICAL UNIVERSITY. Problem Statement •Abandoned objects](https://reader034.vdocuments.us/reader034/viewer/2022051807/600381b23e66da0e46324fa0/html5/thumbnails/6.jpg)
Algorithm
• When n = N, we get the sum S(i,j) and the average A(i,j) for eachcorresponding pixel in the frame using it’s queue Q(i,j)
• Computes a reference static background by forming an image using averages of each pixel in first N frames. Saved on disk , can be updated after every X minutes.
• N = 100, is used.
![Page 7: FPGA Accelerated Abandoned Object Detection · Rajesh Rohilla, Aman Raj, Saransh Kejriwal, Dr. Rajiv Kapoor DELHI TECHNOLOGICAL UNIVERSITY. Problem Statement •Abandoned objects](https://reader034.vdocuments.us/reader034/viewer/2022051807/600381b23e66da0e46324fa0/html5/thumbnails/7.jpg)
Algorithm
Updating Current Static Frame
• We keep updating our current static
background using same computation
• For n > N, queue becomes
![Page 8: FPGA Accelerated Abandoned Object Detection · Rajesh Rohilla, Aman Raj, Saransh Kejriwal, Dr. Rajiv Kapoor DELHI TECHNOLOGICAL UNIVERSITY. Problem Statement •Abandoned objects](https://reader034.vdocuments.us/reader034/viewer/2022051807/600381b23e66da0e46324fa0/html5/thumbnails/8.jpg)
Algorithm
![Page 9: FPGA Accelerated Abandoned Object Detection · Rajesh Rohilla, Aman Raj, Saransh Kejriwal, Dr. Rajiv Kapoor DELHI TECHNOLOGICAL UNIVERSITY. Problem Statement •Abandoned objects](https://reader034.vdocuments.us/reader034/viewer/2022051807/600381b23e66da0e46324fa0/html5/thumbnails/9.jpg)
Algorithm•We model a background using:
Current Frame Comparison
•Updated current static frame and reference static frame compared bycalculating difference to detect abandoned objects.
Blob Detection & Decision Making
•Attempts to remove the effect of small blobs caused due tointermittent movement in video feed
![Page 10: FPGA Accelerated Abandoned Object Detection · Rajesh Rohilla, Aman Raj, Saransh Kejriwal, Dr. Rajiv Kapoor DELHI TECHNOLOGICAL UNIVERSITY. Problem Statement •Abandoned objects](https://reader034.vdocuments.us/reader034/viewer/2022051807/600381b23e66da0e46324fa0/html5/thumbnails/10.jpg)
Flowchart
![Page 11: FPGA Accelerated Abandoned Object Detection · Rajesh Rohilla, Aman Raj, Saransh Kejriwal, Dr. Rajiv Kapoor DELHI TECHNOLOGICAL UNIVERSITY. Problem Statement •Abandoned objects](https://reader034.vdocuments.us/reader034/viewer/2022051807/600381b23e66da0e46324fa0/html5/thumbnails/11.jpg)
FPGA Implementation
• Serial processing of such pixelqueues on a conventional computingplatform is a relatively slow process,so algorithm synthesized on FPGA.
•Hardware implementation speeds upalgorithm execution by exploiting itparallel nature.
• Xilinx Zynq-7020 all programmablesystem on chip (SoC) FPGA board used.
Xilinx FPGA Board
![Page 12: FPGA Accelerated Abandoned Object Detection · Rajesh Rohilla, Aman Raj, Saransh Kejriwal, Dr. Rajiv Kapoor DELHI TECHNOLOGICAL UNIVERSITY. Problem Statement •Abandoned objects](https://reader034.vdocuments.us/reader034/viewer/2022051807/600381b23e66da0e46324fa0/html5/thumbnails/12.jpg)
FPGA Implementation
• Processing System (PS) that contains DualARM Cortex-A9 Processor
• Programmable Logic (PL) that contains Artix-7FPGA
• Our logic is programmed on PL part usingVivado High-Level Synthesis (HLS) libraryprovided by Xilinx.
• Data transfer using the AXI-Stream bus whichis highly efficient and fast for real-time high-bandwidth data transfer
Block diagram of our FPGA system
![Page 13: FPGA Accelerated Abandoned Object Detection · Rajesh Rohilla, Aman Raj, Saransh Kejriwal, Dr. Rajiv Kapoor DELHI TECHNOLOGICAL UNIVERSITY. Problem Statement •Abandoned objects](https://reader034.vdocuments.us/reader034/viewer/2022051807/600381b23e66da0e46324fa0/html5/thumbnails/13.jpg)
Results (AVSS2007)
• Algorithm tested on AVSS2007 video dataset, thatcontains abandoned objects in public places
Detection results of the sequence AB-Easy of AVSS2007
![Page 14: FPGA Accelerated Abandoned Object Detection · Rajesh Rohilla, Aman Raj, Saransh Kejriwal, Dr. Rajiv Kapoor DELHI TECHNOLOGICAL UNIVERSITY. Problem Statement •Abandoned objects](https://reader034.vdocuments.us/reader034/viewer/2022051807/600381b23e66da0e46324fa0/html5/thumbnails/14.jpg)
Results (our Dataset)
• Horizontally placed camera on table top in minimallycrowded place, our lab:
![Page 15: FPGA Accelerated Abandoned Object Detection · Rajesh Rohilla, Aman Raj, Saransh Kejriwal, Dr. Rajiv Kapoor DELHI TECHNOLOGICAL UNIVERSITY. Problem Statement •Abandoned objects](https://reader034.vdocuments.us/reader034/viewer/2022051807/600381b23e66da0e46324fa0/html5/thumbnails/15.jpg)
Results (our Dataset)
• An overhead Surveillance camera in very crowded place.
![Page 16: FPGA Accelerated Abandoned Object Detection · Rajesh Rohilla, Aman Raj, Saransh Kejriwal, Dr. Rajiv Kapoor DELHI TECHNOLOGICAL UNIVERSITY. Problem Statement •Abandoned objects](https://reader034.vdocuments.us/reader034/viewer/2022051807/600381b23e66da0e46324fa0/html5/thumbnails/16.jpg)
Thank You