features for handwriting recognition
DESCRIPTION
Features for handwriting recognition. The challenge. “Rappt JD 10 Feb no 175, om machtiging om af”. Short processing pipeline. Learning. “machtiging”. Feature extraction. Classification. 82,34,66,…. “machtiging”. 0.12. Processing pipeline. Preprocessing. Feature extraction. - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Features for handwriting recognition](https://reader036.vdocuments.us/reader036/viewer/2022062322/5681505a550346895dbe5c8b/html5/thumbnails/1.jpg)
Features for handwriting recognition
![Page 2: Features for handwriting recognition](https://reader036.vdocuments.us/reader036/viewer/2022062322/5681505a550346895dbe5c8b/html5/thumbnails/2.jpg)
| 2
The challenge
“Rappt JD 10 Feb no 175, om machtiging om af”
![Page 3: Features for handwriting recognition](https://reader036.vdocuments.us/reader036/viewer/2022062322/5681505a550346895dbe5c8b/html5/thumbnails/3.jpg)
| 3
Short processing pipeline
“machtiging”
Feature extraction
Classification
82,34,66,…0.12
“machtiging”
Learning
![Page 4: Features for handwriting recognition](https://reader036.vdocuments.us/reader036/viewer/2022062322/5681505a550346895dbe5c8b/html5/thumbnails/4.jpg)
| 4
Processing pipeline
Feature extraction
Classification
Preprocessing
![Page 5: Features for handwriting recognition](https://reader036.vdocuments.us/reader036/viewer/2022062322/5681505a550346895dbe5c8b/html5/thumbnails/5.jpg)
| 5
Input image types
› Color:
› Grayscale:
› Binary:
![Page 6: Features for handwriting recognition](https://reader036.vdocuments.us/reader036/viewer/2022062322/5681505a550346895dbe5c8b/html5/thumbnails/6.jpg)
Preprocessing
› Goal: enhance the foreground while reducing other visual symptoms (stains, noise, pictures, ...)
› Methods:• Contrast stretching• Highpass filtering• Despeckling• Change color representation (RGB, HSV, grayscale, black/white, …)
• Remove selected connected components ()• …
| 6
![Page 7: Features for handwriting recognition](https://reader036.vdocuments.us/reader036/viewer/2022062322/5681505a550346895dbe5c8b/html5/thumbnails/7.jpg)
| 7
Connected components
![Page 8: Features for handwriting recognition](https://reader036.vdocuments.us/reader036/viewer/2022062322/5681505a550346895dbe5c8b/html5/thumbnails/8.jpg)
| 8
Processing pipeline
Segmentation
Feature extraction
Classification
Preprocessing
![Page 9: Features for handwriting recognition](https://reader036.vdocuments.us/reader036/viewer/2022062322/5681505a550346895dbe5c8b/html5/thumbnails/9.jpg)
| 9
Object of classification› Sentences› Words› Characters
(use grammar)(use dictionary)(use alphabet)
![Page 10: Features for handwriting recognition](https://reader036.vdocuments.us/reader036/viewer/2022062322/5681505a550346895dbe5c8b/html5/thumbnails/10.jpg)
| 10
Object representations
› Image› Unordered vectors (in a coco)› Contour vectors› On-line vectors› Skeleton image› Skeleton vectors
(x, y)i
(x, y)k
(x, y)k
(x, y)k
I(x, y)
I(x, y)
![Page 11: Features for handwriting recognition](https://reader036.vdocuments.us/reader036/viewer/2022062322/5681505a550346895dbe5c8b/html5/thumbnails/11.jpg)
| 11
A full processing pipeline
Segmentation
Normalization
Feature extraction
Classification
Preprocessing
![Page 12: Features for handwriting recognition](https://reader036.vdocuments.us/reader036/viewer/2022062322/5681505a550346895dbe5c8b/html5/thumbnails/12.jpg)
| 12
Invariance
› Luminance / contrast› Position› Size› Rotation› Shear› Writer style› Ink thickness› …
![Page 13: Features for handwriting recognition](https://reader036.vdocuments.us/reader036/viewer/2022062322/5681505a550346895dbe5c8b/html5/thumbnails/13.jpg)
| 13
Invariance by normalization
› Luminance / contrast› Position› Size› Rotation› Shear› Writer style› Ink thickness› …
Center on center of gravity
Contrast stretchi
ngScale to standard size
![Page 14: Features for handwriting recognition](https://reader036.vdocuments.us/reader036/viewer/2022062322/5681505a550346895dbe5c8b/html5/thumbnails/14.jpg)
| 14
Invariance by trying many deformations› Luminance / contrast› Position› Size› Rotation› Shear› Writer style› Ink thickness› …
Try different scale factors
Try different rotations
… and use the best recognition result
Try different deformation
s
![Page 15: Features for handwriting recognition](https://reader036.vdocuments.us/reader036/viewer/2022062322/5681505a550346895dbe5c8b/html5/thumbnails/15.jpg)
| 15
Invariance by using invariant features› Luminance / contrast› Position› Size› Rotation› Shear› Writer style› Ink thickness› …
Zernike invariant moments
![Page 16: Features for handwriting recognition](https://reader036.vdocuments.us/reader036/viewer/2022062322/5681505a550346895dbe5c8b/html5/thumbnails/16.jpg)
| 16
A full processing pipeline
Segmentation
Normalization
Feature extraction
Classification
Preprocessing
82,34,66,…
![Page 17: Features for handwriting recognition](https://reader036.vdocuments.us/reader036/viewer/2022062322/5681505a550346895dbe5c8b/html5/thumbnails/17.jpg)
| 17
Feature ROI types
› Whole object› Zones› Windowing
![Page 18: Features for handwriting recognition](https://reader036.vdocuments.us/reader036/viewer/2022062322/5681505a550346895dbe5c8b/html5/thumbnails/18.jpg)
| 18
Whole object (“holistic”)
![Page 19: Features for handwriting recognition](https://reader036.vdocuments.us/reader036/viewer/2022062322/5681505a550346895dbe5c8b/html5/thumbnails/19.jpg)
| 19
Zones
![Page 20: Features for handwriting recognition](https://reader036.vdocuments.us/reader036/viewer/2022062322/5681505a550346895dbe5c8b/html5/thumbnails/20.jpg)
| 20
Windowing
![Page 21: Features for handwriting recognition](https://reader036.vdocuments.us/reader036/viewer/2022062322/5681505a550346895dbe5c8b/html5/thumbnails/21.jpg)
| 21
Feature types
› Image itself› Statistical› Structural› Abstract
› Image (off-line) features (1—20)› Contour / on-line features (21 – 28)
![Page 22: Features for handwriting recognition](https://reader036.vdocuments.us/reader036/viewer/2022062322/5681505a550346895dbe5c8b/html5/thumbnails/22.jpg)
| 22
Feature 1 – 3
› Connected component images
› Scaled image
› Distance transform
![Page 23: Features for handwriting recognition](https://reader036.vdocuments.us/reader036/viewer/2022062322/5681505a550346895dbe5c8b/html5/thumbnails/23.jpg)
| 23
Feature 4: density histogram
![Page 24: Features for handwriting recognition](https://reader036.vdocuments.us/reader036/viewer/2022062322/5681505a550346895dbe5c8b/html5/thumbnails/24.jpg)
| 24
Feature 5: radon transform
![Page 25: Features for handwriting recognition](https://reader036.vdocuments.us/reader036/viewer/2022062322/5681505a550346895dbe5c8b/html5/thumbnails/25.jpg)
| 25
Feature 6: run count pattern
3
6
2 3
![Page 26: Features for handwriting recognition](https://reader036.vdocuments.us/reader036/viewer/2022062322/5681505a550346895dbe5c8b/html5/thumbnails/26.jpg)
| 26
Feature 7: run length pattern
avg
stdev
avg
stdev
![Page 27: Features for handwriting recognition](https://reader036.vdocuments.us/reader036/viewer/2022062322/5681505a550346895dbe5c8b/html5/thumbnails/27.jpg)
| 27
Feature 8: Autocorrelation
![Page 28: Features for handwriting recognition](https://reader036.vdocuments.us/reader036/viewer/2022062322/5681505a550346895dbe5c8b/html5/thumbnails/28.jpg)
| 28
Feature 9: Polar zones
![Page 29: Features for handwriting recognition](https://reader036.vdocuments.us/reader036/viewer/2022062322/5681505a550346895dbe5c8b/html5/thumbnails/29.jpg)
| 29
Feature 10: radial zones (tip!)
![Page 30: Features for handwriting recognition](https://reader036.vdocuments.us/reader036/viewer/2022062322/5681505a550346895dbe5c8b/html5/thumbnails/30.jpg)
| 30
Feature 11: zone histograms
![Page 31: Features for handwriting recognition](https://reader036.vdocuments.us/reader036/viewer/2022062322/5681505a550346895dbe5c8b/html5/thumbnails/31.jpg)
Feature 12: Hinge
| 31
(By Marius Bulacu)
![Page 32: Features for handwriting recognition](https://reader036.vdocuments.us/reader036/viewer/2022062322/5681505a550346895dbe5c8b/html5/thumbnails/32.jpg)
Feature 13: Fraglets
| 32
![Page 33: Features for handwriting recognition](https://reader036.vdocuments.us/reader036/viewer/2022062322/5681505a550346895dbe5c8b/html5/thumbnails/33.jpg)
| 33
Regelmatigheden
Singulariteiten
Feature 14: J.C. Simon (1/2)
![Page 34: Features for handwriting recognition](https://reader036.vdocuments.us/reader036/viewer/2022062322/5681505a550346895dbe5c8b/html5/thumbnails/34.jpg)
| 34
"million" ==> convex:concave:3(north:concave) :(north:LOOP):concave:(north:LOOP) :concave:north :concave:HOLE :2(convex:concave)
(J.-C. Simon, 1989)
Feature 14: J.C. Simon (2/2)
![Page 35: Features for handwriting recognition](https://reader036.vdocuments.us/reader036/viewer/2022062322/5681505a550346895dbe5c8b/html5/thumbnails/35.jpg)
| 35
Feature 15: Structure of background (1/3)
![Page 36: Features for handwriting recognition](https://reader036.vdocuments.us/reader036/viewer/2022062322/5681505a550346895dbe5c8b/html5/thumbnails/36.jpg)
| 36
Feature 15: Structure of background (2/3)
![Page 37: Features for handwriting recognition](https://reader036.vdocuments.us/reader036/viewer/2022062322/5681505a550346895dbe5c8b/html5/thumbnails/37.jpg)
| 37
Feature 15: Structure of background (3/3)
![Page 38: Features for handwriting recognition](https://reader036.vdocuments.us/reader036/viewer/2022062322/5681505a550346895dbe5c8b/html5/thumbnails/38.jpg)
| 38
Feature 16: Structure of foreground + background
![Page 39: Features for handwriting recognition](https://reader036.vdocuments.us/reader036/viewer/2022062322/5681505a550346895dbe5c8b/html5/thumbnails/39.jpg)
| 39
Feature 17: Fourier transform (1/2)
From: http://ccp.uchicago.edu/~dcbradle/pages/5.23.06.html
![Page 40: Features for handwriting recognition](https://reader036.vdocuments.us/reader036/viewer/2022062322/5681505a550346895dbe5c8b/html5/thumbnails/40.jpg)
| 40
Feature 17: Fourier transform (2/2)
Fig. 1 and 3 from: http://www.csse.uwa.edu.au/~wongt/matlab.html
Fig. 2 from: http://www.chemicool.com/definition/fourier_transform.html
![Page 41: Features for handwriting recognition](https://reader036.vdocuments.us/reader036/viewer/2022062322/5681505a550346895dbe5c8b/html5/thumbnails/41.jpg)
| 41
Feature 18: Wavelet transform
From: http://www.regonaudio.com/Audio%20Measurement%20via%20Wavelets.html
![Page 42: Features for handwriting recognition](https://reader036.vdocuments.us/reader036/viewer/2022062322/5681505a550346895dbe5c8b/html5/thumbnails/42.jpg)
| 42
Feature 19: Hu invariant moments
€
M p,q = i(x,y)x p
D
∫∫ y qdxdy
0,0M area of the object
€
M1,0
M0,0
,M0,1
M0,0
⎛
⎝ ⎜
⎞
⎠ ⎟= center of mass
Slide adapted from: http://www.cedar.buffalo.edu/~govind/CSE717/lectures/CSE717_3.ppt
› Invariant for scale, position and rotation
› Derived from moments› Moments describe the image distribution with respect to its axes
› Works on (x, y) vectors
![Page 43: Features for handwriting recognition](https://reader036.vdocuments.us/reader036/viewer/2022062322/5681505a550346895dbe5c8b/html5/thumbnails/43.jpg)
| 43
Feature 20: Zernike moments
From: Trier, O. D., Jain, A. K., and Taxt, T. (1996). Feature extraction methods for character recognition - a survey. Pattern Recognition,29:641–662.
› Invariant for scale, position and rotation
› Reconstructing original
![Page 44: Features for handwriting recognition](https://reader036.vdocuments.us/reader036/viewer/2022062322/5681505a550346895dbe5c8b/html5/thumbnails/44.jpg)
| 44
Feature 21 – 28: Contour features
› (cos, sin) of running angle› (cos, sin) of running angular difference› Angular difference› Fourier transform› Ink density (horizontal or vertical)› Radon transform: (ink density, computed
radially from the c.o.g.)› Angular histogram› Curvature scale space ()
![Page 45: Features for handwriting recognition](https://reader036.vdocuments.us/reader036/viewer/2022062322/5681505a550346895dbe5c8b/html5/thumbnails/45.jpg)
| 45
Feature 28: Curvature scale space
From: http://www.christine.oppe.info/blog/category/formen-und-farben/formenvergleich/
pos
itera
tion
![Page 46: Features for handwriting recognition](https://reader036.vdocuments.us/reader036/viewer/2022062322/5681505a550346895dbe5c8b/html5/thumbnails/46.jpg)
End