1

“A Novel Text detection System Based on Character and Link Energies”

Presented by: Arun PatelRoll No.: 15EC65R18

M.Tech 1st year VIPES, IIT Kharagpur

2

Algorithm• This algorithm can detect most text object in various condition including different lightening,

different colors, complex background and low contrast text.• This method is robust to the font, size, color and orientation of text and discriminate text object

from others effectively.

Fig(1) Algorithm

3

Initialization of Candidate Text Objects

Localize the candidate Part Euler number Let and be two candidate parts with widths W and Wvj , heights Hvi and Hvj , and centroids Cvi

and Cvj ;

dist.(Cvi ,Cvj)wd .min(max(Wvi ,Hvi), max(Wvj ,Hvj)) Finally, the candidate character parts that are reachable by one another via one or more links are

grouped to form a candidate text.

4Fig.2 Initialization of candidate text objects

5

Character Features

• One important characteristic that discriminate text object from other object is that character are made up of strokes that typically have approximately uniform thickness resulting in two near parallel edges sets in their boundaries.

• Two edges sets have high similarities in length, orientation and curvature.• Similarities of two stroke edges is captured by gradient vector of each point on the boundary.

Fig.3 (a) edge pairs of strokes (b)Gradient vectors of ‘R’

6

• For a character, it has two near parallel edges sets and the gradients of an edge point and its corresponding point should have approximately opposite direction.

• Distance between the points and their corresponding are similar because the change of stroke width is usually small.

Fig.4 Corresponding pairs and links

7

Average Angle Difference of Corres. Pairs(Dangle)

• Let N denote the number of edge points of a candidate part. P(i)(1 ≤ i ≤ N) is the ith edge point with the corresponding point P(i) corr .The difference of the gradient directions of the corresponding pair (P(i) , P(i)

corr.) is defined as:

=abs(-)

• Dangle measures the average gradient direction difference of all corresponding pairs of a candidate part.

• Dangle =

• For an ideal character Dangle reaches the maximum value 1.

8

Fraction of non-noise pair (Fnon-

noise) • In some cases, however, a character may have a smaller due to noise or deformations. We

compute Fnon−noise to measure the noise and deformation levels of a part based on d (i)

angle .

• Fnon-noise =

• h(,)=1 if d (i)

angle >β

=0 else

• Fnon-noise is the fraction of all pairs for which the angle difference d (i)

angle is greater than β.

Fig5.Noise connections and non-noise connections Ref.(1)

9

R A C E

Dangle 0.889 0.865 0.925 0.897Fnon-noise 0.754 0.684 0.897 0.806

Fig.6 Dangle and Fnon-noise Ref. (1)

10

• we divide the non-noise connections into two types: stroke-length connection and stroke-width connection.

• By doing so, we can separate circle like objects and compute the feature vector of stroke width.

• Let k(i)(1 ≤ i ≤ N) be one of N non-noise connections of a part and have Ik(i) intersections with other

non-noise connections. We define stroke-length connection and stroke-width as follows:• K(i)

• For circle, every connection intersects with all other connections at its center. Hence, all non-noise connection of a circle are stroke length connection.

• Character have much more stroke-width connection than the non-characters.

11Fig 7. Percentages of stroke-width links of two example images. Ref.(1)

12

Vector of Stroke Width ()• The vector of stroke width Vwidth is defined as: =, .• Characters typically have one or two dominating stroke widths depending on their fonts.

• Then, we estimate dominating stroke-width w(i)d through a weighted average computation using w(i)

p and its two immediately adjacent neighbors:

=

Fig.8 Histogram of the lengths of stroke width connections Ref. (1)

13

Character Energy• For a part vi , we consider that its and are equally important for text detection and define the

character energy of vi as follows: = ,0 1.

• It can be treated as a measure of the probability that vi is a character.

• Character have larger Echar can discriminate text objects from other objects and it is robust to the font,size,color and orientation of characters.

• and are correlated.

14

(a) (b)

Fig.9 two character with different noise/deformation levels Ref. (1)

(a) 0.8846 0.5950 0.5950

(b) 0.8847 0.5261 0.7054

15Fig.10 character energy Ref. (1)

16

Link energy• Link energy is computed for every candidate link to measure the probability that two parts

connected by the link are both characters.• Link energy is computed by measuring two values: 1. Similarity in the properties of neighboring parts, such as the color, stroke width, and size. 2.Spatial consistency in the direction and distance between neighboring parts in a string of parts.

• For two connected parts vi and vj ,we use color, stroke width(Vwidth),character width, and character height to capture similarities between them.

=.)

• Higher the higher the similarities between two parts.

17

Similarity Computation Of two Character

Fig.11 Link energy Ref.(1)

colour =

Vwidth ==

Character width

, =

Character Height

=Simi(), =

18

Text Unit Energy • For the text unit containing two parts vi and v j , the text unit energ is computed using character

energies and link energy :• =[()+ ]• To refine the detected text objects, text units whose text unit energies are smaller than a pre-

defined threshold Ttext are removed from the text objects.• choice of this threshold depends upon the characteristic of the datasets, a threshold of of 0.7

worked well for several datasets used for testing this algorithm.

19Fig.12 Text energy Ref(1)

20Fig.13 Threshold Etext and txt detection outputs Ref(1)

21

Result on ICDAR 2003/2005 Dataset Objects

Fig.14 experimental outputs Ref.(2)

22

Evalution ResultsAlgorithm Precision

Ashida 0.55

Hinneck Becker 0.62

SWT 0.73

Novel Text detection 0.74

Recall

0.46

0.67

0.60

0.69

23

References:• [1] Jing Zhang and Rangachar Kasturi ,“A novel text detection system based on characters

and link energies”, image processing, IEEE trans., vol.23, No.9, pp.4187-4198, September 2014.• [2] S.M.Lucas, A. Panaretos, L.Sosa,A. Tang, S. Wong, and R. Young, “ICDAR 2003 robust reading

competitions”, in Proc. 7th Int. Conf. Document And Recognit.,vol.2,pp. 682,2003.• [3]D,.Marr and Hildreth, “Theory of edge detection,” Proc.Roy.Soc. London B,vol.

207,No.1167,pp. 187-217,1980.

24

Thank You