Signboard Optical Character RecognitionIsaac Wu, Hsiao-Chen Chang

Department of Electrical Engineering, Stanford University

Motivation System Pipeline

Results

Having the ability to recognize any store just by taking

a picture of its signboard is a powerful asset for

business reviews and ratings companies such as Yelp

to incorporate into their mobile app.

Phase 1: Training A BC

E

…D

Phase 2:

Segmentation

MSER GrayscaleDetect MSER

RegionsRemove Non-Text Regions

Create Bounding Boxes of Each Region

Merge Boxes and Keep the

Longest

Morphology GrayscaleIncrease Contrast

Adaptive Thresholding

Morphological Opening

Small Region Removal

Region Labeling

Remove Non-Text Regions

Create Bounding Box

Phase 3: RecognitionA B

CE

…D

MCDONALDS

OCRRestrict OCR Matching to

English LettersPerform OCR

Remove short length words

Remove Spaces

Success Rate: 86%

# Testing Images: 113

# Correctly Determined: 97

AlgorithmtrainDatabase()

if (SIFT with MSER has many matches) return result

elseif (SIFT with Morphology has many matches) return result

elseif (OCR with MSER seems valid) return result

elseif (OCR with Morphology seems valid) return result

else return null

* Multi-scale approach

SIFTExtract SIFT Descriptors

Match Descriptors

to Codes

Top 5 Database Matches

Perform SIFT Match with

RANSAC

Return the Most

Matches

Manually Segmented Images

Extract SIFT Descriptors

Construct K-Means Codebook

Match Descriptors to Codes

89.7%

7.2%3.1%

Techniques Used

MSER x SIFT

MORPH x SIFT

MSER x OCR

MORPH xOCR