inria, nice, december 7 th -8 th 2006 data set and ground truth

INRIA, NICE, December 7th-8th 2006

Data Set and Ground Truth


Data set and Ground truth

• Data set contents

• Data set resources

• Ground truth contents

• Ground truth resources


Data set contents

Goal:

Capture realistic video sequences with graduate difficulties:

• lighting variations,

• occlusions …

covering predefined scenarios.


Data set contents

Video providers, main characteristics

• Inrets: Building Entrance, Indoor/Outdoor, Multi-view

• CEA: Corridor + Road, Indoor/Outdoor, Visible/IR

• RATP: Metro, Indoor, Crowded environment

• iLids: Road, Outdoor, Real street context

• Silogic: Aircraft parking zone, Multi-view


Data set contents

5 Topics

• Apron

• Building Corridor

• Building Entrance

• Metro

• Road


Data set contents

APRON2 points of 2 points of

viewview

Different Different weather weather

conditionsconditions


Data set contents

BUILDING CORRIDOR

Visible and IR clipVisible and IR clip


Data set contents

BUILDING ENTRANCE

Entrance andEntrance andCar ParkCar Park 4 points of 4 points of

viewview


Data set contents

METRO

Real subway sceneReal subway scene


Data set contents

ROAD

Visible and IR clipVisible and IR clip

Real sceneReal scene


Data set resources

Data set 1 2 Total

Nb sequences

5 AP

3 BC

19 = 3 BE

4 MO

4 RD

5 AP

3 BC

21 = 3 BE

5 MO

5 RD

10 AP

6 BC

40 = 6 BE

9 MO

9 RD

Nb clips

(x Nb camera)

14 AP

6 BC

41 = 9 BE

4 MO

8 RD

10 AP

6 BC

37 = 9 BE

5 MO

7 RD

24 AP

12 BC

78 = 18 BE

9 MO

15 RD


Data set resources

Data set 1 2 Total

Total sequences Duration

38 min 03 s12 min 04s AP

06 min 40s BC

02 min 24s BE

11 min 52s MO

05 min 03s RD


07 min 24s BC

02 min 46s BE

10 min 16s MO

09 min 59s RD

1 h 15 min 01 s18 min 37s AP

14 min 04s BC

05 min 10s BE

22 min 08s MO

15 min 02s RD

Total Duration 1 h 23 min 19 s 1 h 00 min 02 s 2 h 23 min 21 s

Mean Duration / sequence


02 min 13s BC

00 min 48s BE

02 min 58s MO

01 min 16s RD


02 min 28s BC

00 min 55s BE

02 min 03s MO

02 min 00s RD

1 min 50 s


Data set resources

Data set 1Participation

(13 participants)2

Participation

(16 participants)

Priority sequences

1 AP

0 BC

1 BE

1 MO

1 RD

92 %

---

69 %

69 %

100 %

1 AP

0 BC

1 BE

1 MO

2 RD

94 %

---

94 %

75 %

94 %

Wished sequences

1 AP

1 BC

1 BE

2 MO

0 RD

46 %

15 %

8 %

0 %

---

1 AP

1 BC

1 BE

2 MO

1 RD

25 %

25 %

19 %

22 %

19 %

Others

AP

BC

BE

MO

RD

25 %

0 %

7 %

0 %

5 %

AP

BC

BE

MO

RD

23 %

15 %

25 %

22 %

19 %


Data set resources

• 2 video data set (test + evaluation)

• 2*2 DVD provided

• Documentation (metrics, annotation rules, structures and formats)

• Context and calibration information

• Data base (xls file)


Data set resources

Topics Event of interest

APVehicle movements:

empty_area, inside_zone, enters_zone, stopped

BCPerson and objects interactions:

opens/closes (door), picks_up/puts_down, abandoned_baggage, changes_zone, running

BEPerson, vehicle and objects interactions (complex scene):

opens/closes (door), gets_in/gets_out, go_down/up_stairs, door_control, inside_zone, enters/exist_zone, changes_zone

MOAbandoned baggage in crowded environment:

Waiting, picks_up/puts_down, abandoned_baggage, people counting, exchange_objects, (ticket machine)

RDPerson, vehicle and objects interactions (complex scene):

opens/closes (door), gets_(in/on)/gets_(out/of), biking, crossing, walking_with, overtaking


Ground truth contents

• Context information

• Physical objects of interest and events annotation

Annotation rules provided to participant in a document and updated during the project

Examples…


Ground truth resources

Data set 1 2 Total

Nb ground truth files

14 AP

3 BC

34 = 9 BE

4 MO

4 RD

10 AP

3 BC

32 = 9 BE

5 MO

5 RD

24 AP

6 BC

66 = 18 BE

9 MO

9 RD

Sequence time1 h 11 min 36 s

(61 470 frames)

48 min 31 s

(56 422 frames)

2 h 00 min 07s

(117 892 frames)


Conclusion & Feedback

Difficulties encountered:

• Definition of event

• Annotation subjectivity

Proposition for other evaluation project:

• Ground truth done before data set distribution: Easier to identify difficulties that may encountered participants and add explanations and specific annotation rules

• Ground truth given between the deadline submission and the results processing to be able to correct ground truth before evaluation

-> Time consuming

inria, nice, december 7 th -8 th 2006 data set and ground truth

Documents