urdu ocr using feedforward neural networks thesis presentation 5-2-09

8/7/2019 Urdu OCR using Feedforward Neural Networks Thesis Presentation 5-2-09

1/71

Urdu Optical Character

Recognition Using FeedforwardNeural Network

By

Zaheer AhmadMS-IT

Institute of Management Science, Peshawar, Pakistan

6th

February,09 1


2/71

Optical Character Recognition

Optical Character Recognition (OCR) is themechanical or electronic translation / readingof images of handwritten, typewritten or

printed text (usually captured by a scanner)into machine- editable text

OCR is Branch of Pattern Recognition and Machine Vision

2


3/71

Optical Character Recognition

Main Three Steps

Scanner--------with new tech not difficult Analyzing Image ---script/style dependent

Classification-----------script dependent

3


4/71


5/71

To reduce errors: Standardization of print

fonts, paper, and ink qualities

In the 1970s New fonts such as OCRA and

OCRB were designed

These efforts revolutionalized data entry

process ...loss of jobs

5


6/71

Urdu and Arabic Script OCR History

Little work in the field, mostly on standalonealphabetsIndia.and Pak

BUT

Some work on Arabic and Farsian but still.. Different style used for Arabic, Farsian and

Urdu.i.e nastaleeq .naskh.

Some work on Pashto

6


7/71

Some Applications Of Urdu OCR

It will expand and multiply already available

knowledge in hard copies i.e.

Centuries old rare script in Arabic, Urdu and

Persian will become available to common man

improve the interaction between man and

machine in many applications, including

office automation, check verification of banking,

business and data entry applications,

7


8/71

Some Applications Of OCR

library archives,

documents identifications,

e-books producing,

invoice and shipping receipt processing,

subscription collections,

questionnaires processing,

exam papers processing and

online address and signboard reading

8


9/71

Pattern Recognition

The act of taking in raw data and taking an actionbased on the category of the data (also known asclassification or pattern classification)

It uses methods from statistics, machine learningand other areas.

Some popular techniques for pattern recognitioninclude:

Neural Networks

Hidden Markov Models ----Probability

Bayesian networks .

9


10/71

Urdu Script( ) National Language of Pakistan and

One of the popular script in the Indiansubcontinent evolved in the subcontinent fromthe mixture of Arabic, Turkish, Farsi and HindiLanguages

spoken by more than 60 million speakers in over20 countries

58 Character Set by NLA Pakistan

40 Basic plus onedo-chashmi-hey ( ) is used toform all composite alphabets; therefore theworking set is consists of 41 alphabets.

10


11/71

Character Set (58 alphabets) of Urdu Script

11


12/71

Urdu is a modification of the Persian alphabet,

which is itself a derivative of the Arabic

alphabet

and adopted some characters like Rhe( )fromHindi script.

Urdu is a right to left Script written in the

calligraphic Nasta'liq script where as Naskh

style is used for Arabic.

12


13/71

Most combined Characters form a degree of

about 45 to the horizontal line.

because of which Urdu script reading is faster

than roman script but

It makes it harder the machines to recognize

the word or segment one character from the

rest.

for the novice readers as well

13


14/71

No Capital or Small Character but the Last character isconsidered to be capital as it is in its full form.

Stand alone and joining forms ----changes shape but alsoits size .

It increase the number of classes to be recognized from. Inour experiments we have used 54 different classes for 41different Urdu characters

s

jm

The word Urdu ( ) or of the similar categoryare not joinable or cannot be connected

14


15/71

Problems Of Urdu Script

A large Character set but most of which are

similar:

15


16/71

Other forms ( initial, middle) of these characterreveals that ein() is similar to hamza(),

Waw , () ) might be confusing with)

Ze ) resembles) noon () Zanl ,() dhal ) is close match to initial form of)

tay ( )

Mem() at middle of a word can be confusedwith middle form of ein () and with standalone goal-he .()

16


17/71

Some characters contain closed loop theCharacter contains two loops.

The open portion of characters Jim , Hey and Khe forms a triangle.

The loop of character Mem ,Waw and Ein sometimes becomes too small that the internalopening part is disappeared

Hamza ) zigzag shape, is not really a letter but it)can cause difficulty in segmentation process as itresembles with the segmented middle form ofein ( ).

17


18/71

Dots may appear as two separated dots,

touched dots, hat or as a stroke.

Another style of Urdu handwriting is the

artistic or decorative calligraphy.

Which is usually full ofoverlapping making

the recognition process even more difficult by

human being rather than by computers

18


19/71

What Have Been Done

--------------------NOTHING------------

No text databases or dictionary available, exceptthe one under preparation by the Urdu LanguageAuthority but their Web shows a slow progress sofar.

Even no standard keyboard exits, NationalLanguage Authority of Pakistan has devised akeyboard in which the most used characters are

set under the main fingers but it is very differentfrom the one already in use ( phonetic keyboardof Inpage).

19


20/71

Moreover still to be adopted by software vendors

as even Windows Vista is using its own version of

Urdu keyboard.

The research carried out on Urdu language ismostly scattered and outside from the Urdu

world.

There are no specialized conferences orsymposium conducted so far.

There is no financial support from government.

20


21/71

Neural Networks

21


22/71

Inputs Outputs

Connection between cells

NN A Brain-Inspired Model

in

out

22


23/71

NN A Brain-Inspired Model

A neural network acquires knowledge through

learning.

A neural network's knowledge is stored within

inter-neuron connection strengths known as

synaptic weights.

The largest modern neural networksachieve the complexity comparable to anervous system of a fly.

23


24/71

Historical Background

1943 McCulloch and Pitts proposed the firstcomputational models of neuron.

1949 Hebb proposed the first learning rule.

1958 Rosenblatts work in perceptrons.

1969 Minsky and Paperts exposed limitation of thetheory.

1970s Decade of dormancy for neural networks.

1980-90s Neural network return (self-organization,back-propagation algorithms, etc)

24


25/71

NN Applications

Process Modeling and Control- Creating a neural network model for a physicalplant then using that model to determine the best control settings for the plant.

Machine Diagnosis- Detect when a machine has failed so that the system canautomatically shut down the machine when this occurs.

Target Recognition- Military application which uses video and/or infrared image data todetermine if an enemy target is present.

Medical Diagnosis- Assisting doctors with their diagnosis by analyzing the reportedsymptoms and/or image data such as MRIs or X-rays.

Target Marketing- Finding the set of demographics which have the highest responserate for a particular marketing campaign.

Voice Recogntion- Transcribing spoken words into ASCII text.

Financial Forecasting( Stock predication) - Using the historical data of a security topredict the future movement of that security.

Quality Control - Attaching a camera or sensor to the end of a production process toautomatically inspect for defects.

Intelligent Search - An internet search engine that provides the most relevant contentand banner ads based on the users' past behavior.

Fraud Detection - Detect fraudulent credit card transactions and automatically declinethe charge. 25
http://www.nd.com/apps/science.html


26/71

How NN Work ( Mathematically)

Linear and Non Linear Pattern / Classification Regression / Function Estimation

Curve Fitting

Why to USE NN

Parallel Processing

Fault tolerance Self-organization

Generalization ability

Continuous adaptivity26


27/71

27

ArtificialNeurons

Neural networks are made up of nodes which have

Input edges, each with some weight

Output edges (with weights)

An activation level (a function of the inputs)

Weights of edges can be positive or negative and may change

over time (learning)

The output function is the weighted sum of the activation levels

of inputs

The activation level is a linear or non-linear transfer function a

of the input :

Some nodes are inputs, some are outputs.


28/71

A Model of Artificial Neuron

bias

x1

x2

xm= 1

wi1

wi2

wim =i

.

.

.

yi

f(.) a (.)

28


29/71

bias

x1

x2

xm= 1

wi1

wi2

wim =i

.

.

.

A Model of Artificial Neuron

yi

f(.) a (.)

1

( )m

i ij j

j

f w x

)()1( fatyi

otherwise

ffa

0

01)(

29


30/71

Structural Types of NN

Un-weighted -- McCullochPitts ( 1943 ) Weighted---- Introduced by Hebb

Supervised

Perceptron -- by Frank Rosenblattfoundation ADALIN and MADALIN

FFNN

Unsupervised ART1 and ART2

Kohenons Self Organizing Maps(SOM)..etc

30


31/71

The Perceptron

31

x1

x2

xn

.

.

.

w1

w2

wn

wn+1

Biasxn+1=-1

a= bias+wi xi

y

1 ifa0y= 0 ifa


32/71

Linear Separability Problem

If two classes of patterns can be separated by a decision boundary,

represented by the linear equation

then they are said to be linearly separable. The simple network can

correctly classify any patterns.

Decision boundary of linearly separable classes can be determined

either by some learning procedures or by solving linear equation

systems based on representative patterns of each classes

If such a decision boundary does not exist, then the two classes are

said to be linearly inseparable. Linearly inseparable problems cannot be solved by the simple

network , more sophisticated architecture is needed.

01

n

i iiwxb

32


33/71

Examples of linearly separable classes

- Logical AND function

patterns (bipolar) decision boundary

x1 x2 y w1 = 1-1 -1 -1 w2 = 1-1 1 -1 b = -11 -1 -1 = 01 1 1 -1 + x1 + x2 = 0

- Logical OR function


x1 x2 y w1 = 1-1 -1 -1 w2 = 1-1 1 1 b = 11 -1 1 = 01 1 1 1 + x1 + x2 = 0

x

oo

o

x: class I (y = 1)o: class II (y = -1)

x

xo

x


33

Equation of Line ( Decision Boundary )


34/71

Examples of linearly inseparable classes

- Logical XOR (exclusive OR)function


x1 x2 y-1 -1 -1-1 1 11 -1 1

1 1 -1

o

xo

x


34


35/71

Multilayer NN

Neural Net for Nonlinear Classification

Combination of Perceptron

Back propagation learning

35


36/71

What do each of the layers do?

1st layer draws

linear boundaries

2nd layer combines

the boundaries

3rd layer can generate

arbitrarily complex boundaries

Multilayer FFNN

A NN with one or more than one hidden layers

36


37/71

Back propagation Algorithm Multiple outputs.

Forward pass:

Error calculation:

Backward propagation:

No guarantee to in getting best possibleweights after correcting.

Classifies inputs into multiple classes.

Can be modified to represent any function.


38/71

MATLAB and NN Toolbox

The name MATLAB stands for matrix

laboratory.MATLAB is a high-performance language fortechnical computing. It integrates computation,visualization, and programming in an easy-to-useenvironment where problems and solutions areexpressed in familiar mathematical notation. Typicaluses include:

Math and computation

Algorithm development

Modeling, simulation, and prototyping

Data analysis, exploration, and visualization

Scientific and engineering graphics

Application development, including Graphical UserInterface building

38


39/71

Urdu Optical Character

Recognition Using Feedforward

Neural Networks

The Proposed System

39


40/71

Introduction

UOCR developed forUrdu Script

Ariel 36 Font

Single line of Urdutext image.

Segmentation Part

Neural Network /Classification Part

Input Urdu Text Image

Preprocessing

Segmentation

Segmented Character

Binary Character ( Resized )

Character Code (Results) 40


41/71

Feature Extraction and Segmentation

Algorithm developed

Pixels strength is measure where the pixelsstrength or energy is the number of black pixelsin a specific direction. Down-up or right-left orany degree

A search for finding a path, bottom to top, rightto left or any degree is made during which blackpixels are counted and selected , the path on

which minimum number of black pixels areencountered (minimum number of black pixelsare found)

41


42/71

i ii iii iv v vi vii viii

i 0 0 0 0 1 0 1 0

ii 0 0 1 0 0 0 1 0

iii 0 1 1 1 1 0 1 0

iv 0 1 0 1 0 0 1 0v 0 1 0 1 0 1 1 0

vi 0 0 0 1 0 1 1 0

vii 0 0 1 1 0 1 1 0

viii 0 0 1 1 0 1 1 0

Ix 0 0 1 1 0 1 1 0 42


43/71

Energy Level

For Words segmentation zero level energy is

selected

For character segmentation energy of the seam ofis calculated and compared with the average

energy of all the seam(of the image )

Character Size and threshold values Large Segments further segmented

Small Segments are merged

43


44/71

Garbage Charactersthe main problem in the algorithm

44


45/71

Training Patterns

Ariel font of size 36 was

resized

Enlarged in some cases

reduced in some case

imresize function with

nearest parameter

54 different classes

100 samples

Ms Paint, photoshop andMs Word

45


46/71

Neural Network Training and Sim

A Multilayer Feedforward Neural Network(FFNN)

Input layer21x15 (315)----size of character

Hidden layer with 2000------trial and error

Output layer 6 nodes ----to cover 56 Alphabets

Activation Functions Tansig and logsig

epochs = 2000 to get trained/meet the goal of

0.0005----goal selected from results

Time 5-7 hours on 2 GHZ, Dual Core with 2 Gb of

RAM46


47/71

Segmented character again resized beforefeeding to NN

sim function returns a 6 digit binary number The number is matched with the 54 character

set (used as target during the training).

The No Character Found message

47


48/71

Simulation Results

Recognition of character family of ( ), pee (),tee ( ) tay () ,cee ( ) ) and fee ) is around80 %

same is the case of character family of kaf ( (and gaf ( ) as these are the most simplecharacters and despite their similarity with eachother they are totally different from the othercharacters.

The character lam ( ) when used in middle of aword behave like and alif ( ) which decrease itsrecognition percentage but alif is notmisunderstood as lam ( ) in most of the cases.

48


49/71

Simulation Results

The character waw() and choty yee ( = ) aredifficult to be differentiated by the NN as thesegment of choty yee () after it produce thegarbage is very similar to waw () .

Characters fee () , mem ( = ) and ein ( = )when used in the middle form of a character candeceive neural network for each other during therecognition process which leads to a low

percentage for their recognition. Character noon ( = ) when used in the

beginning it looks like ze ( ) ) and zal ) and thusproduces low results.

49


50/71

Simulation Results

The combination of lam () and alif ( ) whenused in ( ) like words make some what anew character, in the segmentation phase as

shown in figure below.

50

C l i


51/71

Conclusions The results show 70% success at the neural network output

but the algorithm developed shows about 85% results whenseen through human eye.

Most of the error (garbage characters) are produced at the

end character of a word when the word is ending on noon or

a character having similar shape like noon.

It is hard to find which character is the end character

therefore the problem cannot be overcome easily.

A large percentage of error is produced by the character

seen() ,sheen() ,swad(),dwad(), noon(), noonghuna() which in most of the cases get passes the charactertest during segmentation,

where as bee ( ), pee (), tee ( ) , tay () , cee ( ) andfee ( .) also produces garbage characters in some cases

51


52/71

Thanks

52


53/71

53

10

x


54/71

54

The Role of a Bias Weight

with a fixed input -1

1a

2a

02211 aWaW

The decision boundary :

Without the bias weight!

1a

2a

002211 WaWaW

The decision boundary :

With the bias weight!

1x

2x

nx

0W

1W

2W

nW

g


55/71

55


56/71

Binary

AND OR NOT

56


57/71

AND

input output

00011011

0001

y

x1 x2

w1 w2

f(x1w1 + x2w2) = y

f(0w1 + 0w2) = 0f(0w1 + 1w2) = 0f(1w1 + 0w2) = 0

f(1w1 + 1w2) = 1

= 0.5

f(a) =1, for a >

0, for a

some possible values for w1 and w2

w1 w2

0.20

0.20

0.25

0.40

0.35

0.40

0.30

0.20


58/71

XOR

input output

00011011

0110

y

x1 x2

w1 w2

f(x1w1 + x2w2) = y

f(0w1 + 0w2) = 0f(0w1 + 1w2) = 1f(1w1 + 0w2) = 1

f(1w1 + 1w2) = 0

= 0.5

f(a) =1, for a >

0, for a

some possible values for w1 and w2

w1 w2


59/71

XOR

input output00011011

0110

y

x1 x2

= 0.5

f(a) =1, for a >

0, for a

z = 0.5

w 3 w4

f(w1, w2, w3, w4, w5)

w5

a possible set of values for ws

(w1, w2, w3, w4, w5)

(0.3,0.3,1,1,-2)

w1 w2


60/71

XOR

input output00011011

0110

f(a) =1, for a >

0, for a

f(w1, w2, w3, w4, w5 , w6)

a possible set of values for ws

(w1, w2, w3, w4, w5 , w6)

(0.6,-0.6,-0.7,0.8,1,1)

w1 w4w3w2

w5 w6= 0.5 for all units


61/71

XOR

can be solved by a more complex network with hidden units

Y

z2

z1x1

x2

2

2

2

2

-2

-2

1

0

(-1, -1) (-1, -1) -1(-1, 1) (-1, 1) 1(1, -1) (1, -1) 1

(1, 1) (1, 1) -1

61


62/71

Linear Separation


63/71

Y = a(X) + b Y =1 + e-a(X) + b

1

Linear Logistic

RegressionDiscriminant


64/71

AND, OR, NOT

w1

w2

wn

x1

x2

xn

ThresholdIntegrate

1.5

1.0

1.0


65/71

AND, OR, NOT

w1

w2

wn

x1

x2

xn

ThresholdIntegrate

.9

1.0

1.0


66/71

AND, OR, NOT

w1

w2

wn

x1

x2

xn

ThresholdIntegrate

.5

-1.0


67/71

67


68/71

68

Perceptron Learning Algorithm:


69/71

69

p g g

Initialise weights and threshold.

Set wi(t), (0


70/71

70

Neural Networks

Training

Backpropagation training cycle


71/71

urdu ocr using feedforward neural networks thesis presentation 5-2-09

Documents