deep convolutional neural network for recognition of...

XXX-X-XXXX-XXXX-X/XX/$XX.00 ©20XX IEEE

Deep Convolutional Neural Network for Recognition of Unified Multi-Language Handwritten Numerals

Ghazanfar Latif Computer Science Department,

Prince Mohammad Bin Fahd University, Al Khobar, Saudi Arabia.

[email protected]

M. Muzzamal NaseerCollege of Engineering,

Australian National University, Canberra, Australia.

[email protected]

Jaafar Alghazo Computer Engineering Department,

Prince Mohammad Bin Fahd University, Al Khobar, Saudi Arabia.

[email protected]

Loay Alzubaidi Computer Science Department,

Prince Mohammad Bin Fahd University, Al Khobar, Saudi Arabia. [email protected]

Yazan Alghazo Humanities Department, Prince

Mohammad Bin Fahd University, Al Khobar, Saudi Arabia. [email protected]

Abstract— Deep learning systems have recently gained importance as the architecture of choice in artificial intelligence (AI). Handwritten numeral recognition is essential for the development of systems that can accurately recognize digits in different languages which is a challenging task due to variant writing styles. This is still an open area of research for developing an optimized Multilanguage writer independent technique for numerals. In this paper, we propose a deep learning architecture for the recognition of handwritten Multilanguage (mixed numerals belongs to multiple languages) numerals (Eastern Arabic, Persian, Devanagari, Urdu, Western Arabic). The overall accuracy of the combined Multilanguage database was 99.26% with a precision of 99.29% on average. The average accuracy of each individual language was found to be 99.322%. Results indicate that the proposed deep learning architecture produces better results compared to methods suggested in the previous literature.

Keywords—Arabic Numberals; Mul-Language Numerals Recognition; Hand Written Numerals; Deep Convolutional Neural Networks

I. INTRODUCTION

Due to the infinite number of variations of handwritten numerals that are subject to many variable, the recognition of handwritten numeral is a complex problem that has been subject of research. A large number of algorithms have been developed to tackle this problem for both offline and online recognition. Offline recognition refers to recognizing numerals written on paper using a pen and online recognition refers to numerals written on handheld devices e.g. stylus. Handwritten numerals can differ even for the same writer when writing on paper or a handheld device and can vary even in the same medium. The algorithm consisting of pre-processing, feature extraction and the final phase of classification is used for both cases. In the preprocessing phase, the image is extracted whether in the spatio-temporal or spatio-luminance representation, noise reduction, centering the image, etc. In the feature extraction phase, researchers proposed different methods for feature extraction that can be used for in final step of classification of

handwritten. The classification phase involves using well-developed classification techniques such as Multilayer Perceptron (MLP) and Support Vector Machines (SVM) among many others. This phase involves training the Artificial Intelligence (AI) system and then recognizing the remaining portion of the dataset or new data. These algorithms can also utilize either supervised or unsupervised learning while training the AI system on the dataset. It is observed that mostly research papers have only tackled numerals written in one language

The importance of recognizing handwritten numerals is apparent due to the increase usage of handheld devices in both personal and professional settings. Banking industry relies heavily on handwritten numeral recognition especially for numerals written on checks. Other industries also would benefit from handwritten numeral recognition algorithms in addition to individuals with different handheld device applications developed that allow users to write numbers rather than input them using a keyboard.

In this research, a deep Convolutional Neural Network (CNN) architecture is developed for handwritten Eastern Arabic and Persian numerals while experiments are also performed on some other languages including Urdu, Devanagari, Western Arabic which in total, are spoken by approximately 1.86 billion people. The rest of this paper is structured as follows: section 2 confers the literature review, section 3 presents the proposed system, section 4 details the experimental datasets, section 5 discusses the results of the proposed recognition system and section 6 concludes the paper.

II. LITERATURE REVIEW

Though the concept of Deep Learning has been around for quite some time, yet the utilization of Deep learning to its full potential was not achieved until recently with advances in technology, computing power and computing resources. Deep learning algorithms have achieved new records in different fields such as image recognition and speech recognition among others [1-2]. The concept of Deep Learning relies on what is known as Convolutional Neural Networks (ConvNets or CNN).

2018 IEEE 2nd International Workshop on Arabic and Derived Script Analysis and Recognition (ASAR)

90978-1-5386-1459-4/18/$31.00 ©2018 IEEE

Authorized licensed use limited to: Prince Mohammad Bin Fahd University. Downloaded on August 13,2020 at 11:49:58 UTC from IEEE Xplore. Restrictions apply.

pc

Tampon

LeNet 5 developed by LeCun, Y. et al. was one of the first convolutional networks used in the 1990s [3]. In 2012, Krizhevsky, A. et al. [4] introduced a newer CNN that was wider and deeper than LeNet and subsequently won the ImageNet-2012 Large Scale Visual Recognition Challenge (ILSVRC). This is considered by many as the initial point of the subsequent interest in Deep Learning. The accuracy rate achieved was 84.7%. The Architecture of [4] was modified into a new architecture proposed by M. D. Zeiler and R. Fergus in [5]. This new architecture ended up winning the ILSVRC 2013. It achieved an accuracy rate of 88.3%. An Inception module was developed that shrinks the number of parameters in the network and was the contribution of the CNN developed by Szegedy, C. et al. and ended up winning the ILSVRC 2014 [6]. The accuracy rate achieved was 93.33%. In the same year and in the same competition Simonyan, K., & Zisserman, A. introduced a CNN proving that the number of layers in a deep network affects the performance [7]. They showed that results are improved by increasing the depth to 16-19 weighted layers. Their network ended up winning second place in the 2014 ILSVRC. He, K. et al. in [8] introduced residual network CNN and won the ILSVRC 2015. The error rate achieved was 3.6%. Huang et al. [9] introduced the densely connected CNN which exhibitedimprovement over some previous architectures. Alghazo et al.also proposed an efficient geometric feature based numeralrecognition system for Arabic and Persian numerals [10]. Yetthe CNN introduced in [8] is still the most popular and preferredCNN architecture for practical applications.

A number of papers have already been completed on digit or character recognition using some types of Deep Learning Architectures. For example, Singh et al. [11] compared the use of the deep Convolutional Neural Network and the fully connected Feed-forward Neural Network for the recognition of the Devangari handwritten characters. They reported a 98.11% recognition rate. In [12], Deep-Learning Feed forward Backpropagation Neural Network (DFBNN) along with Extreme Learning Machine (ELM) were used in handwritten Numeral Recognition. The methods were applied to datasets in Thai, Bangla and Devangari. DRBNN outperformed ELM slightly. In DFBNN, the accuracy rates were 98.4% for Thai, 95.84% for Bangla, and 78.4% for Devangari. Deep Belief Networks (DBN), A Deep learning Structure, was used in [13] for Bangla Handwritten Numeral Recognition. The recognition accuracy was 90.27%. DBN was also used in [14] for high-resolution Synthetic Aperture Radar (SAR) image classification. Alghazo proposed an online numerals recognition system using structural features [15]. In [16], Deep Learning algorithm consisting of Back-propagation with many hidden layers and deformed images for training was applied on the MNIST Handwritten dataset and achieved a low 0.35% error rate. In [17], deep convolutional Neural Networks (CNN) is used for numeral and character recognition respectively. The error rate was as low as 0.19% and 0.27% respectively.

Two hidden layers were used in [18] for the recognition of postal code in Urdu, Telugu, English, Tamil and Devanagari and achieved a 96.53%. Deep Autoencoder (DA) based on MLP was used in [19] for Bangali handwritten numeral recognition with 97.74% recognition rate for deep learning DA algorithm. Neural Network Language Models and Convolutional neural networks

are used in [20] for improving handwritten Chinese text recognition. Deep Neural Network is used as part of an algorithm implemented in [21] for SAR target configuration recognition. Fisher Vector (FV) and CNN are used in [22] for visual recognition of the ancient inscriptions.

Since Deep learning algorithm handwritten numeral recognition rate will be compared to widely studied Artificial Intelligence Algorithms, a few references will be mentioned here in the literature review. Different novel feature extraction methods were developed and used in [23-25] for numeral and character recognition. Different classifier have also been used to test the proposed algorithms.

III. PROPOSED SYSTEM

A. Preprocessing

Numerals in Arabic, Urdu and Persian are written from leftto right. Their digits are mostly similar except for the numbers (4) four, (5) five and (6) six. In this paper, we target therecognition of handwritten digits of these languages in additionto Western Arabic (English) and Devanagari which are allshown in table 1.

TABLE I. REPRESENTATION NUMERALS IN EASTERN ARABIC, WESTERN ARABIC, PERSIAN, URDU AND DEVANAGARI

Western Arabic

Eastern Arabic

Persian Urdu Devanagari

0 ٠ ٠ ०

1 ١ ١ १

2 ٢ ٢ २

3 ٣ ٣ ३

4 ٤ ۴ ४

5 ٥ ۵ ५

6 ٦ ۶ ६

7 ٧ ٧ ७

8 ٨ ٨ ८

9 ٩ ٩ ९

Handwriting styles are different from one person to another and even the handwriting of the same person can be different according to several variables. For example, more than 52 writing classes exist for Arabic and Persian digits only [26]. The similarity between the digits in different languages shown in table 1 makes the recognition of handwritten numerals a difficult task. Fig. 1 shows a sample of handwritten numbers for the different languages targeted in this work.

The preprocessing phase is important in automatic handwritten numeral recognition due to the variations of size, location, shape, noise and angle of handwritten numerals. The following steps are performed in the preprocessing phase of the proposed system.

• Binarization of handwritten numerals from grayscale isdone through Otsu Thresholding [27].


91


• Removing noise from the binarized image which is done by a 3 3 window of disk shaped structure throughmorphological operation.

• Separating each digit referred to as Segmentation.

• Each digit is then centered within the window.

Each digit of all the languages is normalized and convertedto a size of28 28 to make identical input for the Deep CNN.

Fig. 1. Samples of Handwritten numerals for most popular five languages. Columns from left to right representing Eastern Arabic, Persian, Urdu, Western Arabic (English) and Devanagari digits.

B. Deep CNN Architecture

The proposed algorithm in this article is based on deeplearning. A convolution neural network CNN, a three dimensional (3D) volume of neurons, to classify numerals. The deep learning architecture proposed in this study is designed as follows: the input layer is equivalent to image size which is 28x28 units then two hidden convolution layers with 5X5 window size are used also called local receptive field and both convolution layers are then followed by pooling layers with max-pooling of 2×2. The convolution layers in the proposed network extract and learn image features that are distributed across the entire image. Every convolution layer is composed of multiple feature map with different weights to extract multiple features from each location of the input image. All units in a single feature map share the same set of weights (5×5) and biases to detect the same feature at all possible locations on the input. In this way, every feature map is trying to detect different local feature. The purpose of the pooling layers is to condense and simplify the information at the output of these convolution layers. There are several reasons to choose convolution layers over fully connected neural nets, some of these includes.

• From experience convolution networks tends to perform better than the fully connected neural networks.

• Convolutional networks are robust to shifts anddistortions in the image as output of the feature map willbe shifted by the same amount as the input image [28].

• Using individual image pixel as input in fully connectedneural nets would not take advantage of the fact thatimages are highly spatially coordinated.

• Since hidden convolution layers use shared weights andbiases, so it greatly reduces the number of parameters

which in turn helps to train deep networks due to computational advantages.

• Another disadvantage of fully connected neural network is that they are very prone to overfitting when comparedto convolution neural networks. The problem ofoverfitting occurs when a neural network cannotgeneralize well on the unseen data.

The proposed network architecture contains input layer of 28 28 which is equal to image size, next is the hidden convolution layer with 20 24 24 units followed by a pooling layer with 20 12 12 units and next is another convolution layer with 100 8 8 units again followed by pooling layer with 100 4 4 units and finally, the fully connected output layer generating actual results of classification. Fig. 2 shows the architecture of our convolution neural network.

In the proposed convolution neural network architecture, a regularization technique called "dropout" [29] to deal with the problem of overfitting is used. Dropout technique ignores randomly selected neurons during training. These ignored neurons cannot contribute to activations during forwardpass and their corresponding weights are not updated during backward pass. Dropout technique causes a neural network to learn multiple internal representations which in turn makes the neural network to better generalize and less likely to overfit the training data. In addition to using dropout as regularization, a small weight penalty is used while training our network to keep weights under control and avoid overfitting.

We used stochastic gradient descent equation 1 with a batch size of one as an optimization technique to update the weights during training of our architecture.

Where is the cost function, is the step size and is the sample size and equal to one.

IV. EXPERIMENTAL DATA

As previously shown, the complexity of recognition of handwritten numerals depends on many variables and is increased due to variables such as the variation of handwritings. The proposed method is validated with 5 well known large databases of 5 different languages. Modified Arabic Handwritten Digits Databases (MADBase) is the first database used. MADBase was compiled from 700 writers [30], and comprises of 70,000 handwritten Arabic digits with 300 dpi resolution at 28 ×28 pixels. The Modified National Institute of Standards and Technology (MNIST) is the second database used is Eastern Arabic. MNIST was developed using 250 writers and consists of 60,000 training and 10,000 testing numerals [31]. The HODA database is used for Persian numerals [32]. HODA contains 80,000 Persian numerals compiled from 12,000 registration forms from university entrance exams. In this paper, a database was developed for the Urdu numerals database consisting 8,500 samples from which 6,500 numerals are used for training and 2,000 are used for testing. Experiments are also performed on Devanagari numerals datasets named DHCD respectively [33]. A summary of the datasets used for the testing of the proposed system are shown in Table 2.


92


Fig. 2. Proposed Deep CNN Architecture for the Numeral Recognition

V. RESULTS AND DISCUSSIONS

The proposed deep learning architecture was tested on the above mentioned datasets. After preprocessing the digits in the databases, the numerals are input to the deep learning architecture developed in this study. As shown in table 3, the database were input to the proposed CNN with 1 hidden layer and 2 hidden layers respectively. As seen, the average recognition rate using 1 hidden layer is 99.10% and the average recognition rate using 2 hidden layers is 99.32%. It has been proven in previous literature that more hidden layers would produce better recognition rate. The results indicate that with two hidden layers better recognition rates were achieved. However, with more hidden layers, the more complex the network and more computational time is needed to produce results. For the purpose of this study, the recognition rate of 99.32% is adequate and adding more layers increasing the time to produce results will not be justified. The rest of the results in this section are tested on a 2 hidden layer architecture.

TABLE II. DETAILS OF THE DATASETS USED FOR TESTING THE PROPOSED SYSTEM

Numeral Language

Database Data Source Training Dataset

Testing Dataset

Eastern Arabic MADBase 700 volunteers 60,000 10,000 Western Arabic

(English) MNIST 250 participants 60,000 10,000

Persian HODA 12000 Forms 60,000 20,000Urdu PMUdb 170 volunteers 6,500 2,000

Devanagari DHCD NA 17,000 3,000

Fig. 3 shows the error rate for each numeral in each language and indicates the confusion between certain digits in each language. For example, in Arabic the error rate for the digit zero is about 0.996%, however, what Fig. 2 shows is that the digit 0 in Arabic is confused as the digits 5 for about 0.896% of the error rate. This is a logical results since the numeral 5 has similar geometrical features with the numeral 0. In Persian, the digit 0 is mostly confused with the digit 1 and digit 5. To explain the definition of target and predicted, an example will be used. For example if we input an image for the digit 0 (target image) and after all three phases are complete, the system recognizes the image wrongly as 5 (predicted).

TABLE III. COMPARISON OF RECOGNITION RATES FOR DIFFERENT LANGUAGES NUMERALS WITH 1 CNN LAYER AND 2 CNN LAYERS

CNN 1 Layer CNN 2 Layers

Eastern Arabic 99.21% 99.30%

Persian 98.57% 98.82%

Devanagari 99.47% 99.73%

Urdu 99.14% 99.33%

Western Arabic (English) 99.13% 99.43%

The combined Multilanguage unified system can recognize numerals of any language without predefining the specific language which make the recognition task more challenging. Table 4 show the accuracy rates for all 5 languages separately and for each digit within each language. For example, the accuracy rate for digit 0 has an average of 98.96% in all languages while digit 0 in Western Arabic (English) has 99.29% accuracy rate. In general, the average accuracy for all digits in all 5 languages is greater than 99.33%. The comparison of the proposed method shows better recognition rates as compared to those methods discussed in the previous studies in the literature review section along with their experimental results.

TABLE IV. COMPARISON OF ACCURACY RATES FOR EACH INDIVIDUAL NUMERAL OF ALL LANGUAGES

Digit Class

Eastern Arabic

Persian Western Arabic

Devanagari Urdu Average

0 98.01% 98.73% 99.29% 100.00% 98.76% 98.96%

1 99.10% 97.87% 99.91% 99.67% 98.74% 99.06%

2 98.52% 98.22% 99.32% 99.34% 98.57% 98.79%

3 99.90% 98.74% 99.51% 100.00% 99.73% 99.58%

4 99.80% 98.71% 99.19% 99.67% 99.20% 99.31%

5 98.80% 98.81% 99.22% 99.67% 99.13% 99.12%

6 99.70% 99.00% 99.79% 99.67% 99.60% 99.55%

7 99.90% 99.55% 99.22% 100.00% 99.90% 99.71%

8 99.90% 99.65% 99.59% 99.34% 99.90% 99.68%

9 99.40% 98.95% 99.60% 100.00% 99.77% 99.54%


93


Fig. 3. Comparison of Targets vs. Predicted digits error rates

Finally and as an ultimate test for the proposed CNN, the database for all 5 languages is combined into one gigantic database and table 6 shows the results. The average accuracy rate achieved was 99.26% and average FP rate 0.02% and average precision of 99.29%. These results show the superiority of the proposed architecture in the recognition of the multi-language numerals.

VI. CONCLUSION

In this study, a novel deep learning CNN is proposed for the recognition of multi-language numerals. It was shown that the more the hidden layers the better the recognition rate, however, the more the hidden layers the more time and complexity in producing results. It was determined that 2 hidden layers is a compromise that produces adequate recognition rates. The experimental results indicate that the proposed architecture produces better accuracies on both the individual languages or combined languages database that have similar geometrical features or have combined different geometrical features respectively. The overall average accuracy for the combined Multilanguage database was 99.26% with an average precision of 99.29. The average accuracy for each individual language was 99.322%. The results achieved in this study far exceeds those mentioned in previous literature for systems proposed to recognize these languages. For future work, the authors will explore a deep learning architecture for universal numeral recognition. This will be also expanded for character recognition as well.

TABLE V. RECOGNITION RATES FOR COMBINED ALL 5 LANGUAGES NUMERALS

Language Digit Class

TP Rate FP

Rate Precision

Eastern Arabic, Persian, Urdu

0 98.10% 0.10% 99.10%

2 99.20% 0.10% 98.20%

3 98.40% 0.00% 99.40%

5 98.80% 0.10% 98.70%

8 99.90% 0.00% 99.90%

9 99.10% 0.10% 99.30%

Eastern Arabic, Persian 7 99.60% 0.00% 99.50%

Eastern Arabic 4 99.20% 0.00% 99.90%

Eastern Arabic, Urdu 6 99.70% 0.00% 99.60%

Persian 6 99.20% 0.00% 99.30%

4 99.20% 0.00% 99.30%

Urdu 4 99.20% 0.10% 99.40%

7 99.90% 0.00% 99.70%

Western Arabic

2 99.50% 0.00% 99.40%

4 99.40% 0.00% 99.20%

5 99.20% 0.00% 99.00%

6 98.90% 0.00% 99.50%

7 99.40% 0.00% 98.80%

8 99.30% 0.00% 99.40%

Western Arabic, Devanagari 0 98.80% 0.10% 98.40%

3 99.20% 0.00% 99.40%

Devanagari

1 99.70% 0.00% 99.30%

2 99.00% 0.00% 98.00%

4 99.70% 0.00% 99.70%

5 99.00% 0.00% 99.70%

6 98.70% 0.00% 99.70%

7 99.70% 0.00% 100.00%

8 100.00% 0.00% 99.70%

Eastern Arabic, Persian, Urdu. Western Arabic

1 99.60% 0.10% 98.80%

Average 99.26% 0.02% 99.29%

0.00%

0.30%

0.60%

0.90%

1.20%

1.50%

1.80%

2.10%

2.40%

2.70%

3.00%

0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9

Arabic Persian English Dawanageri Urdu

Err

or R

ate

Numerals of different Languages

0 1 2 3 4 5 6 7 8 9


94


REFERENCES

[1] LeCun, Y., Bengio, Y., & Hinton, G. (2015). Deeplearning. Nature, 521(7553), 436-444.

[2] Schmidhuber, J. (2015). Deep learning in neural networks: Anoverview. Neural networks, 61, 85-117.

[3] Y. LeCun, L. Bottou, Y. Bengio and P. Haffner: Gradient-Based Learning Applied to Document Recognition, Proceedings of the IEEE,86(11):2278-2324, November 1998

[4] Krizhevsky, A., Sutskever, I. and Hinton, G. E. ImageNet Classificationwith Deep Convolutional Neural Networks NIPS 2012: NeuralInformation Processing Systems, Lake Tahoe, Nevada

[5] M. D. Zeiler and R. Fergus. Stochastic pooling for regularization of deepconvolutional neural networks. CoRR, abs/1301.3557, 2013.

[6] Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V. & Rabinovich, A. (2015). Going deeper withconvolutions. In Proceedings of the IEEE Conference on ComputerVision and Pattern Recognition (pp. 1-9).

[7] Simonyan, K., & Zisserman, A. (2014). Very deep convolutionalnetworks for large-scale image recognition. arXiv preprintarXiv:1409.1556

[8] He, K., Zhang, X., Ren, S., & Sun, J. (2016). Deep residual learning forimage recognition. In Proceedings of the IEEE Conference on ComputerVision and Pattern Recognition (pp. 770-778).

[9] Huang, G., Liu, Z., Weinberger, K. Q., & van der Maaten, L. (2016).Densely connected convolutional networks. arXiv preprintarXiv:1608.06993.

[10] Alghazo, J. M., Latif, G., Elhassan, A., Alzubaidi, L., Al-Hmouz, A., &Al-Hmouz, R. (2017). An Online Numeral Recognition System UsingImproved Structural Features–A Unified Method for Handwritten Arabic and Persian Numerals. Journal of Telecommunication, Electronic andComputer Engineering (JTEC), 9(2-10), 33-40.

[11] Singh P., Verma A., Chaudhari N.S. (2016) Deep Convolutional NeuralNetwork Classifier for Handwritten Devanagari Character Recognition.In: Satapathy S., Mandal J., Udgata S., Bhateja V. (eds) InformationSystems Design and Intelligent Applications. Advances in IntelligentSystems and Computing, vol 434. Springer, New Delhi.

[12] Iamsa-at, S., & Horata, P. (2013, December). Handwritten characterrecognition using histograms of oriented gradient features in deeplearning of artificial neural network. In IT Convergence and Security(ICITCS), 2013 International Conference on (pp. 1-5). IEEE.

[13] Sazal, M. M. R., Biswas, S. K., Amin, M. F., & Murase, K. (2014,February). Bangla handwritten character recognition using deep beliefnetwork. In Electrical Information and Communication Technology(EICT), 2013 International Conference on (pp. 1-5). IEEE.

[14] Zhao, Z., Jiao, L., Zhao, J., Gu, J., & Zhao, J. (2017). Discriminant deepbelief network for high-resolution sar image classification. Pattern Recognition, 61, 686-701.

[15] Alghazo, J. M., Latif, G., Elhassan, A., Alzubaidi, L., Al-Hmouz, A., &Al-Hmouz, R. (2017). An Online Numeral Recognition System UsingImproved Structural Features–A Unified Method for Handwritten Arabic and Persian Numerals. Journal of Telecommunication, Electronic andComputer Engineering (JTEC), 9(2-10), 33-40.

[16] Ciresan, D. C., Meier, U., Gambardella, L. M., & Schmidhuber, J., 2010, Deep Big Simple Neural Nets Excel on Handwritten Digit Recognition, Neural Computation, 22(12) 2010, pp. 1-14.

[17] Ciresan, D. C., Meier, U., Gambardella, L. M., & Schmidhuber, J. (2011, September). Convolutional neural network committees for handwrittencharacter classification. In Document Analysis and Recognition (ICDAR), 2011 International Conference on (pp. 1135-1139). IEEE.

[18] Asthana, S., Haneef, F., & Bhujade, R. K. (2011). Handwritten multiscript numeral recognition using artificial neural networks. International Journal of Soft Computing and Engineering, 1(1), 1-5.

[19] Pal, A., Khonglah, B. K., Mandal, S., Choudhury, H., Prasanna, S. R. M., Rufiner, H. L., & Balasubramanian, V. N. (2016, March). Online Bengali handwritten numerals recognition using Deep Autoencoders.In Communication (NCC), 2016 Twenty Second National Conferenceon (pp. 1-6). IEEE.

[20] Wu, Y. C., Yin, F., & Liu, C. L. (2017). Improving handwritten Chinesetext recognition using neural network language models and convolutional neural network shape models. Pattern Recognition, 65, 251-264.

[21] Huang, X., Nie, X., Wu, W., Qiao, H., & Zhang, B. (2017). SAR targetconfiguration recognition based on the biologically inspiredmodel. Neurocomputing, 234, 185-191.

[22] Amato, G., Falchi, F., & Vadicamo, L. (2016). Visual recognition ofancient inscriptions using convolutional neural network and fishervector. Journal on Computing and Cultural Heritage (JOCCH), 9(4), 21.

[23] Boukharouba, A., & Bennia, A. (2015). Novel feature extractiontechnique for the recognition of handwritten digits. Applied Computingand Informatics.

[24] Karimi, H., Esfahanimehr, A., Mosleh, M., Salehpour, S., & Medhati, O.(2015). Persian Handwritten Digit Recognition Using EnsembleClassifiers. Procedia Computer Science, 73, 416-425.

[25] Jie, M., Aziz, I. A., Hasbullah, H., & Azizan, S. A. B. (2016, August).Handwritten digits recognition based on improved label propagationalgorithm. In Computer and Information Sciences (ICCOINS), 2016 3rdInternational Conference on (pp. 345-350). IEEE.

[26] Al_barraq, M.O. and Mehrotra, S.C., 2015. Recognition of ArabicHandwritten Amount in Cheque through Windowing Approach.International Journal of Computer Applications, 115(10), pp. 33-38.

[27] Fan, J. L., & Zhao, F. (2007). Two-dimensional Otsu's curve thresholding segmentation method for gray-Level images. Dianzi Xuebao(ActaElectronica Sinica), 35(4), 751-755.

[28] LeCun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998). Gradient-basedlearning applied to document recognition. Proceedings of theIEEE, 86(11), 2278-2324.

[29] Srivastava, N., Hinton, G. E., Krizhevsky, A., Sutskever, I., &Salakhutdinov, R. (2014). Dropout: a simple way to prevent neuralnetworks from overfitting. Journal of Machine Learning Research, 15(1), 1929-1958.

[30] Abdelazeem, S., 2009. Comparing arabic and latin handwritten digitsrecognition problems. World Academy of Science, Engineering andTechnology, 54.

[31] Deng, L., 2012. The MNIST database of handwritten digit images formachine learning research [best of the web]. IEEE Signal ProcessingMagazine, 29(6), pp.141-142.

[32] Khosravi, H. and Kabir, E., 2007. Introducing a very large dataset ofhandwritten Farsi digits and a study on their varieties. Pattern recognition letters, 28(10), pp.1133-1141.

[33] Acharya, Shailesh, Ashok Kumar Pant, and Prashnna Kumar Gyawali."Deep learning based large scale handwritten Devanagari characterrecognition." In Software, Knowledge, Information Management andApplications (SKIMA), 2015 9th International Conference on, pp. 1-6.IEEE, 2015.


95


deep convolutional neural network for recognition of...

Documents