playing card recognizer ece 4025 february 28, 2002 group 5 robert barrett jason hodkin chung tse mar...
DESCRIPTION
Project Scope Image Restrictions: –One card per image –No occlusion of card –High contrast solid background –Constant light source Image Allowances: –Scaling –Rotation –Font styleTRANSCRIPT
![Page 1: Playing Card Recognizer ECE 4025 February 28, 2002 Group 5 Robert Barrett Jason Hodkin Chung Tse Mar Jay Silver David Winkler Yu Ming Wu](https://reader036.vdocuments.us/reader036/viewer/2022062413/5a4d1b617f8b9ab0599ad4b1/html5/thumbnails/1.jpg)
Playing Card RecognizerPlaying Card RecognizerECE 4025ECE 4025
February 28, 2002February 28, 2002
Group 5Group 5Robert BarrettRobert BarrettJason HodkinJason Hodkin
Chung Tse MarChung Tse MarJay SilverJay Silver
David WinklerDavid WinklerYu Ming WuYu Ming Wu
![Page 2: Playing Card Recognizer ECE 4025 February 28, 2002 Group 5 Robert Barrett Jason Hodkin Chung Tse Mar Jay Silver David Winkler Yu Ming Wu](https://reader036.vdocuments.us/reader036/viewer/2022062413/5a4d1b617f8b9ab0599ad4b1/html5/thumbnails/2.jpg)
MotivationMotivation
Short-term:Short-term:– Demonstrate effectiveness and usefulness of Demonstrate effectiveness and usefulness of
character recognitioncharacter recognition
Long-term:Long-term:Design autonomous on-line card playing Design autonomous on-line card playing systemsystem
![Page 3: Playing Card Recognizer ECE 4025 February 28, 2002 Group 5 Robert Barrett Jason Hodkin Chung Tse Mar Jay Silver David Winkler Yu Ming Wu](https://reader036.vdocuments.us/reader036/viewer/2022062413/5a4d1b617f8b9ab0599ad4b1/html5/thumbnails/3.jpg)
Project ScopeProject ScopeImage Restrictions:Image Restrictions:– One card per imageOne card per image– No occlusion of cardNo occlusion of card– High contrast solid backgroundHigh contrast solid background– Constant light sourceConstant light source
Image Allowances:Image Allowances:– ScalingScaling– RotationRotation– Font styleFont style
![Page 4: Playing Card Recognizer ECE 4025 February 28, 2002 Group 5 Robert Barrett Jason Hodkin Chung Tse Mar Jay Silver David Winkler Yu Ming Wu](https://reader036.vdocuments.us/reader036/viewer/2022062413/5a4d1b617f8b9ab0599ad4b1/html5/thumbnails/4.jpg)
How does it work?How does it work?System diagramSystem diagram
Down Sample & Convert to
Binary
High Resolution
Color Image
Binary Image of Card Corner
Label Connected Objects & Flood Fill Card Area
Cross-correlate
with Templates
Compute Scores
and Compare
Card Value and Suit
Use Major Axis to Find Corners for Scaling and
Rotation
Extract Corner for
Comparison
![Page 5: Playing Card Recognizer ECE 4025 February 28, 2002 Group 5 Robert Barrett Jason Hodkin Chung Tse Mar Jay Silver David Winkler Yu Ming Wu](https://reader036.vdocuments.us/reader036/viewer/2022062413/5a4d1b617f8b9ab0599ad4b1/html5/thumbnails/5.jpg)
Color to Gray Scale ConversionColor to Gray Scale Conversion[R G B] YR UR VR
YG UG VG
YB UB VB
= Y U V
We are only interested in intensity (Y)
[R G B] YR
YG
YB
= intensity
YR = 0.299 YG =0.587 YB =0.114
![Page 6: Playing Card Recognizer ECE 4025 February 28, 2002 Group 5 Robert Barrett Jason Hodkin Chung Tse Mar Jay Silver David Winkler Yu Ming Wu](https://reader036.vdocuments.us/reader036/viewer/2022062413/5a4d1b617f8b9ab0599ad4b1/html5/thumbnails/6.jpg)
Gray Scale to Binary ConversionGray Scale to Binary ConversionIf grayScaleValue > mean + 2*Pixel = 1; Else, Pixel = 0
25 13 1 50 28 40 247 5 225 13 18 254 190 12 5 7 210 18 255 32 11 5 36 20 13
0 0 0 0 0 0 1 0 1 0 0 1 1 0 0 0 1 0 1 0 0 0 0 0 0
![Page 7: Playing Card Recognizer ECE 4025 February 28, 2002 Group 5 Robert Barrett Jason Hodkin Chung Tse Mar Jay Silver David Winkler Yu Ming Wu](https://reader036.vdocuments.us/reader036/viewer/2022062413/5a4d1b617f8b9ab0599ad4b1/html5/thumbnails/7.jpg)
LabelingLabeling
1 1 1 0 0 0 0 0 1 1 1 0 1 1 0 0 1 1 1 0 1 1 0 0 1 1 1 0 0 0 1 0 1 1 1 0 0 0 1 0 1 1 1 0 0 0 1 0 1 1 1 0 0 1 1 0 1 1 1 0 0 0 0 0
1 1 1 0 0 0 0 0 1 1 1 0 2 2 0 0 1 1 1 0 2 2 0 0 1 1 1 0 0 0 3 0 1 1 1 0 0 0 3 0 1 1 1 0 0 0 3 0 1 1 1 0 0 3 3 0 1 1 1 0 0 0 0 0
Identify blobs
Determine characteristics of blobs
Area = 24
Area = 4
Area = 5
![Page 8: Playing Card Recognizer ECE 4025 February 28, 2002 Group 5 Robert Barrett Jason Hodkin Chung Tse Mar Jay Silver David Winkler Yu Ming Wu](https://reader036.vdocuments.us/reader036/viewer/2022062413/5a4d1b617f8b9ab0599ad4b1/html5/thumbnails/8.jpg)
Flood FillFlood FillFloodFill ( pixelToEliminate ) FloodFill ( allNeighboringPixels) Erase all neigboring pixels end
![Page 9: Playing Card Recognizer ECE 4025 February 28, 2002 Group 5 Robert Barrett Jason Hodkin Chung Tse Mar Jay Silver David Winkler Yu Ming Wu](https://reader036.vdocuments.us/reader036/viewer/2022062413/5a4d1b617f8b9ab0599ad4b1/html5/thumbnails/9.jpg)
Calculate Angle of Major Axis of Calculate Angle of Major Axis of Best Fit Ellipse (orientation)Best Fit Ellipse (orientation)
![Page 10: Playing Card Recognizer ECE 4025 February 28, 2002 Group 5 Robert Barrett Jason Hodkin Chung Tse Mar Jay Silver David Winkler Yu Ming Wu](https://reader036.vdocuments.us/reader036/viewer/2022062413/5a4d1b617f8b9ab0599ad4b1/html5/thumbnails/10.jpg)
RotateRotate
25 13 1 50 28 40 247 5 225 13 18 254 190 12 5 7 210 18 255 32 11 5 36 20 13
25 13 1 50 28
40 247 5 225
13 18 254 190 12
5 7 210 18
255 32 11 5 36
20 13
![Page 11: Playing Card Recognizer ECE 4025 February 28, 2002 Group 5 Robert Barrett Jason Hodkin Chung Tse Mar Jay Silver David Winkler Yu Ming Wu](https://reader036.vdocuments.us/reader036/viewer/2022062413/5a4d1b617f8b9ab0599ad4b1/html5/thumbnails/11.jpg)
Isolate the Important InformationIsolate the Important Information
Right Side of Card
Bottom of Card
Top of Card
Left Side of Card
Scan each row and column for a white pixel
![Page 12: Playing Card Recognizer ECE 4025 February 28, 2002 Group 5 Robert Barrett Jason Hodkin Chung Tse Mar Jay Silver David Winkler Yu Ming Wu](https://reader036.vdocuments.us/reader036/viewer/2022062413/5a4d1b617f8b9ab0599ad4b1/html5/thumbnails/12.jpg)
Locate Identifiable Symbol and isolate itLocate Identifiable Symbol and isolate it
top
left
bottom
right
![Page 13: Playing Card Recognizer ECE 4025 February 28, 2002 Group 5 Robert Barrett Jason Hodkin Chung Tse Mar Jay Silver David Winkler Yu Ming Wu](https://reader036.vdocuments.us/reader036/viewer/2022062413/5a4d1b617f8b9ab0599ad4b1/html5/thumbnails/13.jpg)
Choose an Appropriate Template and Choose an Appropriate Template and Scale to Match the TemplateScale to Match the Template
Choose a font that should be consistent with most card fonts, but that is different from the data set.
Template
Image
Freecell (A card game on most windows OS’s)
![Page 14: Playing Card Recognizer ECE 4025 February 28, 2002 Group 5 Robert Barrett Jason Hodkin Chung Tse Mar Jay Silver David Winkler Yu Ming Wu](https://reader036.vdocuments.us/reader036/viewer/2022062413/5a4d1b617f8b9ab0599ad4b1/html5/thumbnails/14.jpg)
Take a Normalized Cross CorrelationTake a Normalized Cross Correlation[X(r+m,c+n)*H(r,c)] = Correlation(m,n)n m
![Page 15: Playing Card Recognizer ECE 4025 February 28, 2002 Group 5 Robert Barrett Jason Hodkin Chung Tse Mar Jay Silver David Winkler Yu Ming Wu](https://reader036.vdocuments.us/reader036/viewer/2022062413/5a4d1b617f8b9ab0599ad4b1/html5/thumbnails/15.jpg)
Take a Normalized Cross CorrelationTake a Normalized Cross Correlation
![Page 16: Playing Card Recognizer ECE 4025 February 28, 2002 Group 5 Robert Barrett Jason Hodkin Chung Tse Mar Jay Silver David Winkler Yu Ming Wu](https://reader036.vdocuments.us/reader036/viewer/2022062413/5a4d1b617f8b9ab0599ad4b1/html5/thumbnails/16.jpg)
Highest “Score” is a MatchHighest “Score” is a MatchMax Corr is 0.5581
Max Corr is 0.3016
![Page 17: Playing Card Recognizer ECE 4025 February 28, 2002 Group 5 Robert Barrett Jason Hodkin Chung Tse Mar Jay Silver David Winkler Yu Ming Wu](https://reader036.vdocuments.us/reader036/viewer/2022062413/5a4d1b617f8b9ab0599ad4b1/html5/thumbnails/17.jpg)
Implementation OverviewImplementation Overview
EVM
Correlation Calculation
Templates
Final Result
Color to Binary
HOST
Corner Extraction
ScalingLabel
Flood Fill Rotate
Decision Model
RTDX
Camera
![Page 18: Playing Card Recognizer ECE 4025 February 28, 2002 Group 5 Robert Barrett Jason Hodkin Chung Tse Mar Jay Silver David Winkler Yu Ming Wu](https://reader036.vdocuments.us/reader036/viewer/2022062413/5a4d1b617f8b9ab0599ad4b1/html5/thumbnails/18.jpg)
Implementation OverviewImplementation Overview
Image File
Decision Model
HOST
Corner Extraction
ScalingRTDX
EVM
Correlation Calculation
Templates
Final Result
![Page 19: Playing Card Recognizer ECE 4025 February 28, 2002 Group 5 Robert Barrett Jason Hodkin Chung Tse Mar Jay Silver David Winkler Yu Ming Wu](https://reader036.vdocuments.us/reader036/viewer/2022062413/5a4d1b617f8b9ab0599ad4b1/html5/thumbnails/19.jpg)
Realtime ConstraintRealtime Constraint
Realtime constraint depends on the Realtime constraint depends on the applicationapplication– How fast the cards changeHow fast the cards change– Bounded by camera frame rateBounded by camera frame rate
Set goal of 1 frame/secondSet goal of 1 frame/second
![Page 20: Playing Card Recognizer ECE 4025 February 28, 2002 Group 5 Robert Barrett Jason Hodkin Chung Tse Mar Jay Silver David Winkler Yu Ming Wu](https://reader036.vdocuments.us/reader036/viewer/2022062413/5a4d1b617f8b9ab0599ad4b1/html5/thumbnails/20.jpg)
Computational ComplexityComputational Complexity
1 frame/second1 frame/second64x64 = 4096 pixels/template64x64 = 4096 pixels/template100 cross-correlations/template100 cross-correlations/template13 templates for number, 4 templates for 13 templates for number, 4 templates for suitesuite
100 * 4096 * 17 * 1 = 7 Million MACs/second
![Page 21: Playing Card Recognizer ECE 4025 February 28, 2002 Group 5 Robert Barrett Jason Hodkin Chung Tse Mar Jay Silver David Winkler Yu Ming Wu](https://reader036.vdocuments.us/reader036/viewer/2022062413/5a4d1b617f8b9ab0599ad4b1/html5/thumbnails/21.jpg)
Next StepsNext Steps
Implementing processing on EVMImplementing processing on EVM– Cross-correlationCross-correlation
Implementing processing on hostImplementing processing on host– Corner extractionCorner extraction– ScalingScaling– Decision model and displayDecision model and display
Synchronizing host with EVM using RTDX Synchronizing host with EVM using RTDX APIAPI