Discrete Point Based Signatures and Applications to Document MatchingNemanja Spasojevic, Guillaume Poncin, and Dan Bloomberg
September 15th 2011, Ravenna, Italy
Overview
● Background● Algorithm● Applications
○ duplicate page detection○ text image lockup
● Conclusion
Background (Duplicate Page Detection)
● Find duplicate pages for a given set of scans of physically same book. Assumption:
○ has to handle at time corpus of ~10k text pages○ pages rich in text ○ < 4o of rotation from image to image○ some translation○ needs to be quick ○ simple to use (discrete signatures, for easy indexing /
lookup)
Background (example)
Background (Image Lookup)
● See how well we perform in image lookup mode. Test how robust algorithm is for something it was not designed for:
○ index clean, images○ lookup by image take with cell phone camera○ skew○ rotation ○ blur○ part of original page
Other Aproaches
● Image matching well studied problem ○ SURF, SIFT, FIT work well at point matching across
images and image lookup ○ do not work as well for repetetive patterns such as text
documents ● Document page matching
○ Locally Likely Arrangement Hashing (LLAH), Nakai, et. al.■ affine invariant ■ produces thousands of signatures per page■ precise ■ handles 10k image corpus
Algorithm
Possible inputs:● raw image (operate on word centroids)● OCR-ed text with word bounding boxes (operate on word
bounding box center)● PDF with word bounding box info (operate on word
bounding box center)
Algorithm (Image Processing)
Signature Generation Algorithm
Signature Instability
Signature is composed of N sub-signatures: S = [s(0)][s(1)]...[s(N-1)]
Instability of signatures comes from: ● Small shifts may lead to changes in discretized angle value
(e.g. s(0) flipping from 13 to 14 due to small word position shifts)
● order of sub-signatures may change (s(0) and s(1) swap as they had almost same radial distance)
Signature Filtering Based on Estimated Risk
Superposition of AmbiguousSignature
[s1][{s2,s'2}][s3][s4] => { [s1][s2][s3][s4], [s1][s'2][s3][s4] }
Duplicate Page Detection (metrics)
● The similarity based on signature sets is calculated as:
● The similarity based on matched (aligned) word bounding boxes:
Duplicate Page Detection (example)
Js = 19% Jb = 93%
Duplicate Page Detection (example)
Js = 5% Jb = 37%
Image Lookup
Image Lookup Examples
Image Lookup Examples
Image Lookup Result
1M pages index (32bit signatures) stats:● 386M signatures total● stored as sorted array (<signature, book_id, page_pid, x, y>) fits
in ~4GB of memory● 0.8% of all signatures filtered (those repeated on 1k or more
pages)● each query on average returns 2000 canidates
Index Size [pages] Accuracy Signature Size [bits]
4.1k 0.966 164.1k 0.949 321M 0.871 32
Conclusion
● Simple schema for point cloud based discrete signature generation
● Filtering based on signature stability estimate● Superposing signatures ● Duplicate page detection ● Image lookup by cell phone camera image query (87.1%
accuracy for 1M pages indexed)
Q & A
Thank You!
Synthetic Data Evaluation
Synthetic Data Evaluation
