harris corner detector on fpga rohit banerjee jared choi 15-418: parallel computer architecture and...
TRANSCRIPT
![Page 1: Harris Corner Detector on FPGA Rohit Banerjee Jared Choi 15-418: Parallel Computer Architecture and Programming](https://reader037.vdocuments.us/reader037/viewer/2022110100/56649ddf5503460f94ad802a/html5/thumbnails/1.jpg)
Harris Corner Detector on FPGA Rohit BanerjeeJared Choi
15-418: Parallel Computer Architecture and Programming
![Page 2: Harris Corner Detector on FPGA Rohit Banerjee Jared Choi 15-418: Parallel Computer Architecture and Programming](https://reader037.vdocuments.us/reader037/viewer/2022110100/56649ddf5503460f94ad802a/html5/thumbnails/2.jpg)
Introduction to the problem
![Page 3: Harris Corner Detector on FPGA Rohit Banerjee Jared Choi 15-418: Parallel Computer Architecture and Programming](https://reader037.vdocuments.us/reader037/viewer/2022110100/56649ddf5503460f94ad802a/html5/thumbnails/3.jpg)
Corner detection
Corner Detection
• Corners provide a lot of information
• Spending time to detect corners can significant reduce computation time* Image matching* Motion tracking* Robot Navigation
![Page 4: Harris Corner Detector on FPGA Rohit Banerjee Jared Choi 15-418: Parallel Computer Architecture and Programming](https://reader037.vdocuments.us/reader037/viewer/2022110100/56649ddf5503460f94ad802a/html5/thumbnails/4.jpg)
Application Specific Need
Pixel Stream
OutputCORNERDETECT
OR
• Want to be able to process images in real time.
• Corner detection is very memory intensive.
• CPU cannot process data fast enough because it had to store data to DRAM first.
![Page 5: Harris Corner Detector on FPGA Rohit Banerjee Jared Choi 15-418: Parallel Computer Architecture and Programming](https://reader037.vdocuments.us/reader037/viewer/2022110100/56649ddf5503460f94ad802a/html5/thumbnails/5.jpg)
Exploit arbitrary hardware parallelism to process more within the given time window
![Page 6: Harris Corner Detector on FPGA Rohit Banerjee Jared Choi 15-418: Parallel Computer Architecture and Programming](https://reader037.vdocuments.us/reader037/viewer/2022110100/56649ddf5503460f94ad802a/html5/thumbnails/6.jpg)
Harris Detector Basic Idea
![Page 7: Harris Corner Detector on FPGA Rohit Banerjee Jared Choi 15-418: Parallel Computer Architecture and Programming](https://reader037.vdocuments.us/reader037/viewer/2022110100/56649ddf5503460f94ad802a/html5/thumbnails/7.jpg)
Hardware Implementation
![Page 8: Harris Corner Detector on FPGA Rohit Banerjee Jared Choi 15-418: Parallel Computer Architecture and Programming](https://reader037.vdocuments.us/reader037/viewer/2022110100/56649ddf5503460f94ad802a/html5/thumbnails/8.jpg)
Processing Pipeline
![Page 9: Harris Corner Detector on FPGA Rohit Banerjee Jared Choi 15-418: Parallel Computer Architecture and Programming](https://reader037.vdocuments.us/reader037/viewer/2022110100/56649ddf5503460f94ad802a/html5/thumbnails/9.jpg)
Processing Pipeline
![Page 10: Harris Corner Detector on FPGA Rohit Banerjee Jared Choi 15-418: Parallel Computer Architecture and Programming](https://reader037.vdocuments.us/reader037/viewer/2022110100/56649ddf5503460f94ad802a/html5/thumbnails/10.jpg)
Grayscale
![Page 11: Harris Corner Detector on FPGA Rohit Banerjee Jared Choi 15-418: Parallel Computer Architecture and Programming](https://reader037.vdocuments.us/reader037/viewer/2022110100/56649ddf5503460f94ad802a/html5/thumbnails/11.jpg)
Processing Pipeline
![Page 12: Harris Corner Detector on FPGA Rohit Banerjee Jared Choi 15-418: Parallel Computer Architecture and Programming](https://reader037.vdocuments.us/reader037/viewer/2022110100/56649ddf5503460f94ad802a/html5/thumbnails/12.jpg)
Processing Pipeline
![Page 13: Harris Corner Detector on FPGA Rohit Banerjee Jared Choi 15-418: Parallel Computer Architecture and Programming](https://reader037.vdocuments.us/reader037/viewer/2022110100/56649ddf5503460f94ad802a/html5/thumbnails/13.jpg)
Sobel Filter
![Page 14: Harris Corner Detector on FPGA Rohit Banerjee Jared Choi 15-418: Parallel Computer Architecture and Programming](https://reader037.vdocuments.us/reader037/viewer/2022110100/56649ddf5503460f94ad802a/html5/thumbnails/14.jpg)
2D convolution
![Page 15: Harris Corner Detector on FPGA Rohit Banerjee Jared Choi 15-418: Parallel Computer Architecture and Programming](https://reader037.vdocuments.us/reader037/viewer/2022110100/56649ddf5503460f94ad802a/html5/thumbnails/15.jpg)
Convolution Buffer
![Page 16: Harris Corner Detector on FPGA Rohit Banerjee Jared Choi 15-418: Parallel Computer Architecture and Programming](https://reader037.vdocuments.us/reader037/viewer/2022110100/56649ddf5503460f94ad802a/html5/thumbnails/16.jpg)
Convolution Buffer
![Page 17: Harris Corner Detector on FPGA Rohit Banerjee Jared Choi 15-418: Parallel Computer Architecture and Programming](https://reader037.vdocuments.us/reader037/viewer/2022110100/56649ddf5503460f94ad802a/html5/thumbnails/17.jpg)
Convolution Buffer
![Page 18: Harris Corner Detector on FPGA Rohit Banerjee Jared Choi 15-418: Parallel Computer Architecture and Programming](https://reader037.vdocuments.us/reader037/viewer/2022110100/56649ddf5503460f94ad802a/html5/thumbnails/18.jpg)
Convolution Buffer
![Page 19: Harris Corner Detector on FPGA Rohit Banerjee Jared Choi 15-418: Parallel Computer Architecture and Programming](https://reader037.vdocuments.us/reader037/viewer/2022110100/56649ddf5503460f94ad802a/html5/thumbnails/19.jpg)
Convolution Buffer
![Page 20: Harris Corner Detector on FPGA Rohit Banerjee Jared Choi 15-418: Parallel Computer Architecture and Programming](https://reader037.vdocuments.us/reader037/viewer/2022110100/56649ddf5503460f94ad802a/html5/thumbnails/20.jpg)
Convolution Buffer
![Page 21: Harris Corner Detector on FPGA Rohit Banerjee Jared Choi 15-418: Parallel Computer Architecture and Programming](https://reader037.vdocuments.us/reader037/viewer/2022110100/56649ddf5503460f94ad802a/html5/thumbnails/21.jpg)
Convolution Buffer
![Page 22: Harris Corner Detector on FPGA Rohit Banerjee Jared Choi 15-418: Parallel Computer Architecture and Programming](https://reader037.vdocuments.us/reader037/viewer/2022110100/56649ddf5503460f94ad802a/html5/thumbnails/22.jpg)
Convolution Buffer
![Page 23: Harris Corner Detector on FPGA Rohit Banerjee Jared Choi 15-418: Parallel Computer Architecture and Programming](https://reader037.vdocuments.us/reader037/viewer/2022110100/56649ddf5503460f94ad802a/html5/thumbnails/23.jpg)
Convolution Buffer
![Page 24: Harris Corner Detector on FPGA Rohit Banerjee Jared Choi 15-418: Parallel Computer Architecture and Programming](https://reader037.vdocuments.us/reader037/viewer/2022110100/56649ddf5503460f94ad802a/html5/thumbnails/24.jpg)
Convolution Buffer
After a while……
![Page 25: Harris Corner Detector on FPGA Rohit Banerjee Jared Choi 15-418: Parallel Computer Architecture and Programming](https://reader037.vdocuments.us/reader037/viewer/2022110100/56649ddf5503460f94ad802a/html5/thumbnails/25.jpg)
Convolution Buffer
This column is ready to be processed
0
1
N-2
N-1
![Page 26: Harris Corner Detector on FPGA Rohit Banerjee Jared Choi 15-418: Parallel Computer Architecture and Programming](https://reader037.vdocuments.us/reader037/viewer/2022110100/56649ddf5503460f94ad802a/html5/thumbnails/26.jpg)
Convolution Buffer
0
1
N-2
N-1
![Page 27: Harris Corner Detector on FPGA Rohit Banerjee Jared Choi 15-418: Parallel Computer Architecture and Programming](https://reader037.vdocuments.us/reader037/viewer/2022110100/56649ddf5503460f94ad802a/html5/thumbnails/27.jpg)
Convolution Buffer
0
1
N-2
N-1
![Page 28: Harris Corner Detector on FPGA Rohit Banerjee Jared Choi 15-418: Parallel Computer Architecture and Programming](https://reader037.vdocuments.us/reader037/viewer/2022110100/56649ddf5503460f94ad802a/html5/thumbnails/28.jpg)
Processing Pipeline
![Page 29: Harris Corner Detector on FPGA Rohit Banerjee Jared Choi 15-418: Parallel Computer Architecture and Programming](https://reader037.vdocuments.us/reader037/viewer/2022110100/56649ddf5503460f94ad802a/html5/thumbnails/29.jpg)
Processing Pipeline
![Page 30: Harris Corner Detector on FPGA Rohit Banerjee Jared Choi 15-418: Parallel Computer Architecture and Programming](https://reader037.vdocuments.us/reader037/viewer/2022110100/56649ddf5503460f94ad802a/html5/thumbnails/30.jpg)
Gaussian Filter
![Page 31: Harris Corner Detector on FPGA Rohit Banerjee Jared Choi 15-418: Parallel Computer Architecture and Programming](https://reader037.vdocuments.us/reader037/viewer/2022110100/56649ddf5503460f94ad802a/html5/thumbnails/31.jpg)
Processing Pipeline
![Page 32: Harris Corner Detector on FPGA Rohit Banerjee Jared Choi 15-418: Parallel Computer Architecture and Programming](https://reader037.vdocuments.us/reader037/viewer/2022110100/56649ddf5503460f94ad802a/html5/thumbnails/32.jpg)
Processing Pipeline
![Page 33: Harris Corner Detector on FPGA Rohit Banerjee Jared Choi 15-418: Parallel Computer Architecture and Programming](https://reader037.vdocuments.us/reader037/viewer/2022110100/56649ddf5503460f94ad802a/html5/thumbnails/33.jpg)
Harris Response
Harris Input
∑Gadxdx
∑Gadydy∑Gadxdy
∑Gadxdy
=Sxx
Sxy
Sxy
Syy
![Page 34: Harris Corner Detector on FPGA Rohit Banerjee Jared Choi 15-418: Parallel Computer Architecture and Programming](https://reader037.vdocuments.us/reader037/viewer/2022110100/56649ddf5503460f94ad802a/html5/thumbnails/34.jpg)
Harris Response
![Page 35: Harris Corner Detector on FPGA Rohit Banerjee Jared Choi 15-418: Parallel Computer Architecture and Programming](https://reader037.vdocuments.us/reader037/viewer/2022110100/56649ddf5503460f94ad802a/html5/thumbnails/35.jpg)
Results
OpenCV parallel software implementation SystemVerilog(50MHz)0
2
4
6
8
10
12
14
16
18
20
Execution Time for Different platforms
Platform
Executi
on t
ime/m
s
Platform Execution Time/ms
Naïve Serial Implementation 351.20
OpenCV 18.12
SystemVerilog(50MHz) 1.31
![Page 36: Harris Corner Detector on FPGA Rohit Banerjee Jared Choi 15-418: Parallel Computer Architecture and Programming](https://reader037.vdocuments.us/reader037/viewer/2022110100/56649ddf5503460f94ad802a/html5/thumbnails/36.jpg)
Results
Speedup over serial implementation Speedup over OpenCV implementaion0.00
50.00
100.00
150.00
200.00
250.00
300.00
Speedup
Speedup
PlatformSpeedup of
FPGA Implementation
Naïve Serial Implementation 268.09
OpenCV 13.83
![Page 37: Harris Corner Detector on FPGA Rohit Banerjee Jared Choi 15-418: Parallel Computer Architecture and Programming](https://reader037.vdocuments.us/reader037/viewer/2022110100/56649ddf5503460f94ad802a/html5/thumbnails/37.jpg)
Results
Intel i3-4330 Cyclone IVE FPGA0.00
5.00
10.00
15.00
20.00
25.00
30.00
35.00
40.00
45.00
Energy Efficiency
Platform
Appro
x.
Pow
er
Consum
pti
on/W
Hardware Approx. Power Consumption/W
Intel i3-4330 42.00
Cyclone IVE FPGA 6.30
![Page 38: Harris Corner Detector on FPGA Rohit Banerjee Jared Choi 15-418: Parallel Computer Architecture and Programming](https://reader037.vdocuments.us/reader037/viewer/2022110100/56649ddf5503460f94ad802a/html5/thumbnails/38.jpg)
Thank you for your attention