hardware architecture for calculating lbp-based image region descriptors

IntroductionThe implemented algorithm

The implemented systemResults

Conclusions

Hardware Architecture for Calculating LBP-BasedImage Region Descriptors

Michał Fularz1 Marek Kraft1

1Poznań University of TechnologyInstitute of Control and Information Engineering

May 27, 2015

M. Fularz, M. Kraft A Hardware Architecture for Calculating LBP-Based...



Conclusions

Table of contentsGoal and motivation

1 IntroductionTable of contentsGoal and motivation

2 The implemented algorithmNon-redundant uniform local binary patterns (NRULBP)Region descriptor formation

3 The implemented systemSystem architectureNRULBP computationLocal histogram computation

4 ResultsSummary of used resourcesProcessing speed

5 Conclusions




Conclusions


GoalTo implement an efficient hardware architecture, enabling thecomputation of LBP-based image region descriptors withcorresponding occurrence histograms

MotivationLBP-based descriptors can be used in a wide range ofcomputer vision applicationsComputation of features as well as their distributions in theform of the local occurrence histograms for the whole image isa compute-intensive task




Conclusions


Sample applications of LBP-based descriptors




Conclusions

Non-redundant uniform local binary patterns (NRULBP)Region descriptor formation

Descriptor formationRegular LBP is formed based on gray level value comparisonoperations in 8-neighborhood, giving rise to 256 different LBPvariants

Over 90% of LBPs are ones with less than 3 transitions in thebinary vector - so called uniform LBPs (ULBP) with 59different variantsFurther reduction with non-redundant ULBPs - ULBPs andtheir binary negation (e.g. 0b00000110 - 0b11111001)considered the same, reducing the number of variants to 30




Conclusions

Non-redundant uniform local binary patterns (NRULBP)Region descriptor formation

Region descriptor formationLBPs or their variants can be used to construct region descriptors:

histograms of descriptors are computed within smaller cellsthe cells form a grid whose dimensions correspond to theexpected dimensions of described objectindividual histograms are concatenated to form the fulldescriptor




Conclusions

System architectureNRULBP computationLocal histogram computation

Block diagram of the coprocessorThe input data are the image pixels in progressive scan mode




Conclusions


Block diagram of the coprocessorThe input pixels are arranged to form a 3× 3 window




Conclusions


Block diagram of the coprocessor - NRULBP computationthe raw LBP value is computed based on the results from a setof comparatorsthe NRULBP is computed from the raw value using a lookuptable




Conclusions


Block diagram of the coprocessor - local histogram computationThe resulting NRULBP stream is arranged as a 50× 1 window,enabling parallel computation of local histograms




Conclusions


Block diagram of the coprocessor - local histogram computationlocal histogram modified based on the entry and exit histogramfully pipelined and systolic approach




Conclusions


Block diagram of the coprocessor - data arrangementFinal spatial arrangement ensured by the cell delay blocks




Conclusions


Block diagram of the coprocessorDescriptor type, cell size and arrangement can be easily changed




Conclusions

Summary of used resourcesProcessing speed

Summary of programmable logic resources used for implementationThe designations are: FF – flipflops, LUT – lookup tables, BRAM –BlockRAM memory blocks. The total amount of resources availablein example target devices is given for reference in the bottom rowof the table.

FF LUT BRAMResource utilization used % of avail. used % of avail. used % of avail.10 × 5 13816 12,98 23223 46,63 26 18,575 × 5 6966 6,55 12789 24,04 13 9,296 × 3 6536 6,14 12163 22,86 16 11,07XC7Z020 106400 53200 140XC7Z045 437200 218600 545




Conclusions

Summary of used resourcesProcessing speed

Processing speedTests were performed using a 10× 5 grid of 5× 5 cells on all imagepixels. The power consumption is well below 3W. Clock frequencyis 75MHz.Number of frames per second that the proposed hardwareaccelerator can process for different image sizes:

resolution processing time [ms] frames per second640 x 480 3,84 260,41280 x 720 11,52 86,81920 x 1080 25,92 38,6

PC implementation achieves 1,5 FPS on VGA resolution images.




Conclusions

ConclusionsThe system offers high performance at low power in a smallfootprintThe architecture can be easily adapted to a range of tasksrequiring various cell size or cell grid size. Change of descriptortype (e.g. to HoG) requires more work, but is also possibleFuture work will be focused on integrating the implementedarchitecture with a coprocessor performing a classification task




Conclusions

Thank you for your attention

The project was financed by the National Science Center under the contract decision numberDEC-2011/03/N/ST6/03022,

New concept of the network of smart cameras with enhanced autonomy for automatic surveillancesystems


hardware architecture for calculating lbp-based image region descriptors

Technology

computation of lbp

ecient hardware architecture

dierent lbp variants

motivation goal

histograms of descriptors

local occurrence histograms

nonredundant ulbps ulbps

binary negation