Image Features and Neural Network Classifiers for Animal

Behaviour Recognition

Carlos Fernando Crispim Junior, BCSDoctorate student

Advisor: José Marino Neto, Dr.Sc.Co-Advisor: Fernando Mendes de Azevedo, Dr.Sc.

2007

2/18

What is behaviour?

3/18

Sistematic Behaviour Score

Ethograph

PharmacologyNeuroscience

Video-trackingLesion

LEHNER (1996)

4/18

Automatic Behaviour Score

Locomotion Imobility Grooming

Behavioural stream

Locomotion

LABORAS™

Vibration

seconds

A

B

5/18

Classification of rat behaviour with animage-processing method and a neural network

• RNA Multilayer Perceptron (MLP)

• Morphologic descriptors[animal postures]

27 descriptors 3 frames per sample

ROUSSEAU et al. (2000)

6/18

Summary

• Few image processing solution using image descriptors (for behavioural phenomena);

• Almost no use of kinematic descriptors;

• Lack of evaluation of behavioural descriptors for relevance;

• Low focus on the Temporal relationship among a set of frames

• How to evaluate ANN classifiers without golden pattern;

7/18

General Objective

Study the use of image features (behavioural descriptors) and descriptive statistics attributes to describe behavioural events of lab animals, and automatically identify them using artificial neural networks (ANN), evaluated by different performance indexes.

8/18

EthoWatcher®: behaviour score

LEHNER (1996)

Time (seconds) Behaviour

Behaviour Frequency Duration Latency

Locomotion 2 90 s 10 s

Risk assesment 1 60 s 60 s

Immobillity 1 95 s 95 s

Rearing 1 XXX 120s

[010 s] Locomotion[060 s] Risk assesment[080 s] Locomotion[095 s] Immobillity[120 s] Rearing

9/18

Rat naive to treatment

Rat treated with caffeine

EthoWatcher®: Activity Analysis

10/18

Frame 329

Animal length Distance travelled[kinematic descriptor]

Animal area[morphological descriptor]

Number of modified pixels[kinematic descriptor]

Frame 328

Animal orientation angle[morphological descriptor]

Animal estimated position

Behavioural descriptors

11/18

Frame 329

Frame 330

Behavioural time-relationship profile

Frame 331

Fonte: Benjamini et al., 2010

Descriptive Statistics Attributes Mean; Mode; Variance; Skewness; Kurtosis; e etc.

12/18

Statistical AnalysisDescritor=NpxPosSubtracao

Boxplot by Group

Variable: Std.Dev.

Mean Mean±SE Mean±SD

LocomocaoExp.Vertical

ImobilidadeExp.Horizontal

Auto-limpeza

Comportamento

0

50

100

150

200

250

300

350

Std

.Dev.

Descritor=VariacaoAngularBoxplot by Group

Variable: Mean

Mean Mean±SE Mean±SD

LocomocaoExp.Vertical

ImobilidadeExp.Horizontal

Auto-limpeza

Comportamento

-4

-2

0

2

4

6

8

10

12

14

16

Mean

• One-way analysis of variance - ANOVA (parametric)• One-way Kruskal-Wallis (non-

parametric)

Which behavioural descriptors are relevant?

13/18

Descriptive Statistics DistributionBehaviour=Grooming, Descriptor=Animal Area

Histogram: Mean

Expected Normal76

5,99

5777

8,35

0579

0,70

5380

3,06

0181

5,41

4882

7,76

9684

0,12

4485

2,47

9186

4,83

3987

7,18

8788

9,54

3490

1,89

8291

4,25

3092

6,60

7893

8,96

2595

1,31

7396

3,67

2197

6,02

6898

8,38

1610

00,7

364

1013

,091

110

25,4

459

1037

,800

710

50,1

555

1062

,510

210

74,8

650

1087

,219

810

99,5

745

1111

,929

311

24,2

841

1136

,638

811

48,9

936

1161

,348

411

73,7

032

1186

,057

911

98,4

127

1210

,767

512

23,1

222

1235

,477

012

47,8

318

1260

,186

5

X <= Category Boundary

0

1

2

3

No.

of o

bs.

14/18

Descriptive Statistics DistributionBehaviour=Rearing, Descriptor=Number of Modified Pixels

Histogram: Variance

Expected Normal15

26,0

8430

52,1

6845

78,2

5261

04,3

3676

30,4

2091

56,5

0410

682,

588

1220

8,67

213

734,

756

1526

0,84

016

786,

924

1831

3,00

819

839,

092

2136

5,17

622

891,

260

2441

7,34

425

943,

428

2746

9,51

228

995,

596

3052

1,68

032

047,

764

3357

3,84

835

099,

932

3662

6,01

638

152,

100

3967

8,18

441

204,

268

4273

0,35

244

256,

436

4578

2,52

047

308,

604

4883

4,68

850

360,

772

5188

6,85

653

412,

940

5493

9,02

456

465,

108

5799

1,19

259

517,

276

6104

3,36

062

569,

444

X <= Category Boundary

0

1

2

3

4

5

6

7

No.

of o

bs.

15/18

ANN input

Attribute of Descriptor 01

Ex.: ANN Locomotion

Attribute of Descriptor 02

Attribute of Descriptor 03

Attribute of Descriptor n

RELEVANT descriptors for Locomotion identification

Original behaviour sample length

Example: ½, 1, 2 seconds.

• Locomotion

• Imobility

• Grooming

• Rearing

Multi-layer perceptron

16/18

– Kappa coefficient, – ROC Curves and Area under ROC Curve (AUC).– Accuracy. 1

10

1 - Especificity

Sens

ibili

tyOperation dotAUC

Classifiers evaluation

17/18

Ongoing activities• Determination of Statistical procedures power for descriptors relevance identification;

(in progress).

•Analysis ANN classifiers performance and best training epochs;

(in progress);

• Evaluation of Descriptive Statistics attributes length influences on events

identification;

(in progress);

Image Features and Neural Network Classifiers for Animal

Behaviour Recognition

Carlos Fernando Crispim Junior, BCScrispim@ieb.ufsc.br