building synthetic graphical documents for performance evaluation of symbol recognition m....
Post on 19-Dec-2015
215 views
TRANSCRIPT
Building Synthetic Graphical Documents for Performance Evaluation of Symbol Recognition
M. Delalandre1, E. Valveny1 and T. Pridmore2 1 CVC, Barcelona, Spain2 SCSIT, Nottingham, UK
SymbolRec MeetingCVC, Barcelona, Spain
Thursday 13th December 2007
Plan
• Introduction• Building Engine• Constraint Editor• Use Cases• Conclusions
Introduction (1/4)Graphics Analysis Systems and Performance Evaluation
AnalysisDocument
Document
Document
Characterisation
Groundtruth
Groundtruth
Groundtruth
Groundtruthing
Results
Results
Results
Performance evaluation
Evaluation
Performance evaluation
1. Analysis and Evaluation2. Groundtruthing3. Problematic & Approach4. Our System
Symbol recognition systems
results label + localization
SymbolRecognition
spotting, retrieval, understanding, …
Introduction (2/4)Groundtruthing
Document
Document
Document
Groundtruth
Groundtruth
Groundtruth
Groundtruthing
Based on real-life documents
Advantages : based on real life documentsDrawbacks : copyright, cost, length, error, content control
Akoy’00
binary noise
vectorial noise
Valveny’03
Based on synthetic documents
Document
Document
Document
Groundtruth
Groundtruth
GroundtruthGroundtruthingSetting
Advantages : easy to produce, few constraintDrawbacks : not real
Past works
1. Analysis and Evaluation2. Groundtruthing3. Problematic & Approach4. Our System
Introduction (3/4)Problematic & Approach
1. Analysis and Evaluation2. Groundtruthing3. Problematic & Approach4. Our System
symbol
background
Graphical documents are composed of two layers
To use a same background layer with different symbol layers
To associate symbols to positioning constraints and to select them at random
Our approach
c2
c1
ProblematicHow to build whole documents ?
How to deal with the real in the synthetic ?
How to make an usable approach ?
-
++
-
2-connected
1-connected
1-connected
1-connected
2-connected
Production rules
0-n
0-n
1
1
O-n
‘Creasy’ but well formed drawing
Introduction (4/4)Our System
GTGT
GTGT
SymbolModels
Building Manager
SymbolFactory
SymbolPositioningPositioning
Constraints
Four components• The Engine
• Symbol Model & Factory• Symbol Positioning• Building Manager
• Constraint Editor
Plan
• Introduction• Building Engine• Constraint Editor• Use Cases• Conclusions
Building EngineSymbol Models & Factory
SymbolModels
Building Manager
SymbolFactory
SymbolPositioningPositioning
Constraints
(x,y)begin
end
width
begin
end
center
width width
center
radius
Symbol models
One file, one model
Vector graphics format(point, line, arc, circle)
Symbol factory
x,y
x’,y’
dx
dy
l lxx cos'
lyy sin'
arc is directed to because is lower than γ
02Π
γ
trigonometric
d1
d2
d2
021 dd
2,0,
,2,0
2π -
d1 and d2 are the two lines’ directions
Building EngineConstraints & Symbol Positioning
SymbolModels
Building Manager
SymbolFactory
SymbolPositioningPositioning
Constraints
cos2xdl
cos2 xdl
2cos2
ydl
2cos2
ydl
2
x
y
d
da tan
x
y
d
da tan
2
dx
dy
(2)(3)
(4)l
(1)
(1)
(2)
(3)
(4)
How to select the control point ?
Define for each constraint by polar unit coordinate (ρ,) computed from bounding box
0
10,5
r
How to select the anchor point ?
(a) (b)
0
1
0,5l = r L
Lb
e
rpl
point
0
01
The polygon includes the point if | 01 + 12 + …… 60 | = 2π
3 4
5
62
112
+
-x0
y0
y0+dy
x0+dxx0+rxdx
y0+rydy
0 rx,y 1
Just a fixed value (x,y)
Taken at random belong
a line
Taken at random inside a zone
control point anchor point selection
(4) positioning on background
Constraint, main principle
(2) computation of control point
(1) rotation (3) sloping
Building EngineBuilding Manager (1/2)
SymbolModels
Building Manager
SymbolFactory
SymbolPositioningPositioning
Constraints
L overlap p if dx1 dx2 < 0 or dy1 dy2 < 0
dx1 dx2
dy1 dy2p
L
L overlaps L if e or b are overlapped
b e
Boxes are overlapped if L1 overlaps L2 or L2 overlaps L1
and L3 overlaps L4 or L4 overlaps L3
L1
L2L3
L4
Symbol overlapping
L covers L if e and b are overlapped
b e
L
left
bottom
right
up
symbol overflows L if right or up or left
or bottom is not covered by L
Line overflow
p1 p2
p3p4
symbol overflows P if P doesn’t include p1 or p2 or p3 or p4
P
P
Zone overflow
sc nw
1
acn
icis ww
1
c
ss n
wp
wc1 = 0.33
wc2 = 0.33
wc3 = 1.00
ws1 = 0.33
ws2 = 0.33
ws3 = 0.66
ws4 = 0.33
ws5 = 0.33
ws6 = 1.00
ps1 = 0.11
ps2 = 0.11
ps3 = 0.22
ps4 = 0.11
ps5 = 0.11
ps6 = 0.33
s1
s2
s3
s4
s5
s6
c1
c2
c3
(c)onstraint and (s)ymbol
Symbol selection Stopping criterionWhen building failures (overlapping, overflow, capacity) become upper to number of wished symbolwe stop the process
Plan
• Introduction• Building Engine• Constraint Editor• Use Cases• Conclusions
Constraint Editor Introduction
SymbolModels
Building Manager
SymbolFactory
SymbolPositioningPositioning
Constraints
Constraint Setting
Model Selector
Constraint Selector
Model Viewer
Building Bar
Constraint Locator
Building Viewer
SymbolModels
BuildingEngine
Run
View
Edit
Constraint Editor Background and Models
SymbolModels
Building Manager
SymbolFactory
SymbolPositioningPositioning
Constraints
Model loading, selection and viewing
loading
select‘scroll & click’
automatic zooming
attach
focus‘click and well’
shift‘drag’
Background attachment and browsing
Constraint Editor Locate and Link the Constraints
‘click’
‘click, move, click’or’
‘pressed, drag, release’
‘click(s), double click’
Locate the constraints
select‘click’
link model
add‘right double click’
delete‘left double click’
Link the constraints
SymbolModels
Building Manager
SymbolFactory
SymbolPositioningPositioning
Constraints
Constraint Editor Building View
SymbolModels
Building Manager
SymbolFactory
SymbolPositioningPositioning
Constraints
view‘CRTL+V’
view
focus‘click and well’
shift‘drag’
keeping view
Viewing and browsing Some views
Constraint Editor Wrap, Scale and Rotate
SymbolModels
Building Manager
SymbolFactory
SymbolPositioningPositioning
Constraints
Wrap
1.0
1.2
1.5
Scale Rotate
Constraint Editor Control and Slope
SymbolModels
Building Manager
SymbolFactory
SymbolPositioningPositioning
ConstraintsControl
Slope
Plan
• Introduction• Building Engine• Constraint Editor• Use Cases• Conclusions
Use CasesWork Group
Marçal Rusinol (CVC), Ernest Valveny (CVC), Mathieu Delalandre (CVC), Jean-Yves Ramel (LI), Rashid Qureshi (LI), Philippe Dosch (LORIA), Tony Pridmore (SCSIT) and Hervé Locteau (LITS).
http://docs.google.com/Doc?id=dgjczxf6_4ccc8kh LITIS
Rouen
CVC
Barcelona
SCSIT
Nottingham
LI
Tours
LORIA
Nancy
Use CasesArchitectural drawings
Dataset: 1000 images, 18 000 symbols, 16 models Constraints: connectivity and heterogeneous backgrounds
•10 backgrounds•filled, empty, textured
Use CasesBags of symbol
Dataset: 1600 images, 15 000 symbols, 150 modelsConstraints: scalability, rotation and scaling
• rotation & scaling• 25,50, 100 and 150 models
16 bases 100 images (1024)2 / base 10 symbols / image
Conclusions and Perspectives
• ConclusionsFirst valid system to build whole graphic synthetic documents
• Built documents kept meaning and seem realistic• No constraint on database sizes (Thousands …..)• Contents of documents can be controlled • Can be use on several backgrounds with “reasonable” work costs using the GUI (1/4 hour per background)
• Perspectives• Databases under packaging for GREC contest 2007• To build different document types (diagram, maps)• To propose a characterization algorithm working with the produced
groundtruth