•  Bad alpha-num recognition •  excessive false positives

Augmented Reality Equation Plotter"Salman Naqvi, Uzair Sikora"

Department of Electrical Engineering, Stanford University

Motivation Method

Experimental Results Future Work

•  Graphical Visualization of an equation can often help better understand underlying trends of real variables.

•  Mobile Application with the use of Android platform, OpenCV, Tesseract OCR, and Template Matching with a server to overlay a plot on top of user’s axes.

•  Typing out a complex equation made easier •  Decently Complex equations can be solved including

explicit equations composed of exponentials, polynomials, trigonometric functions, simple derivatives and integrals

Client Side Server Side

Acknowledgements

Equation Image Rank Filtered Background Recursive Segmentation

& Exponentiation

Canny Edge Detection

Harris Keypoint Detection

Create Plot

•  Perspective Insensitive plotting •  Robust keypoint detection for faster

and efficient axis detection Skew and Perspective Insensitive OCR using RANSAC and VLFeat

•  Handwritten equation detection capability by training an SVM.

•  3D plotting capability by adding plot transformation and multiple variable equation detection

•  Interactive plot reading features by allowing the user to place dots where the plot readings are needed

•  Detailed Confusion Matrix for addressing all edge cases in character recognition

Equa%on  Image  Capture  

Binarize  Image  

Recursively  Segment  Image  

Template  Match  

Form  Equa%on  String  

Create  Plot  Points  

Query  user  for  equa%on  approval  

Canny  Edge  Detector  

Harris  Keypoint  Detector  

Create  Plot  

•  Extremely grateful to Sam Tsai, for crafting pathway for character recognition

•  And the TA, David Chen, continuous help along the way

Coordinate Axis Preview Frame

Close Operation

Binarized Tesseract Math Chars 72(x6(2)+2x+26(x))/8(x6(3)+5x)  

Template Matching Database

y=(x^(2)+2x+2^(x))xe^(x^(s)+5x))  

Template Matching Result

•  Astounding! but still false pos. •  Need character properties

•  5 fonts, 35 characters each •  Database quickly updated •  calculus can thus also be used

y=(x^(2)+2x+2^(x))/e^((x^(5)+5x))  

Smart Character Fixing

•  Utilize region properties •  Aspect Ratio, Solidity, # Holes,

y-Centroid, Hole Centroid •  Fixes problems with 6,(,e,0;

(,l,|,t,i; s,5; 3,a; /,x; & other pairs

y=((x.^(2)+2.*x+2.^(x)))./(exp(x.^(5)+5.*x));  Equation Parsing

Manual  Correc,on  

Char. Confusion Matrix

Eqn.  Solved:  Data  Points  Sent  

/ * + - = ^ √ d π ∫ y x ( ) | 0 1 2 3 4 5 6 7 8 9 e l s i n t a c o .

/ * +

- =

^ √

d π

∫ y

x ( )

| 0

1 2

3 4

5 6

7 8

9 e

l s i

n t a

c o

.

Rec

ogni

zed

Cha

ract

ers

Database Characters

EE368 Spring 2013