human action recognition optimization based on evolutionary feature
DESCRIPTION
Human action recognition constitutes a core component of advanced human behavior analysis. The detection and recognition of basic human motion enables to analyze and understand human activities, and to react proactively providing different kinds of services from human-computer interaction to health care assistance. In this paper, a feature-level optimization for human action recognition is proposed. The resulting recognition rate and computational cost are significantly improved by means of a low-dimensional radial summary feature and evolutionary feature subset selection. The introduced feature is computed using only the contour points of human silhouettes. These are spatially aligned based on a radial scheme. This definition shows to be proficient for feature subset selection, since different parts of the human body can be selected by choosing the appropriate feature elements. The best selection is sought using a genetic algorithm. Experimentation has been performed on the publicly available MuHAVi dataset. Promising results are shown, since state-of-the-art recognition rates are considerably outperformed with a highly reduced computational cost.TRANSCRIPT
Amsterdam, The Netherlands July 06-10, 2013
Real World Applications: RWA4.
Room: 02A00 10:40 – 12:20
Session Chair: Alexandros Andre Chaaraoui (University of Alicante, Spain)
ALEXANDROS ANDRE CHAARAOUI AND FRANCISCO FLÓREZ-REVUELTA
HUMAN ACTION RECOGNITION
OPTIMIZATION BASED ON EVOLUTIONARY FEATURE
SUBSET SELECTION
… …
Amsterdam, July 6-10, 2013
Gen
etic
an
d
Evolu
tion
ary
C
om
pu
tatio
n
Con
fere
nce 2
01
3
3
© A
lexandro
s A
ndre
Ch
aara
oui and F
ranci
sco F
lóre
z-R
evu
elt
a
(GEC
CO
’13
)
Contents1. Introduction2. Radial Summary Feature3. Evolutionary Feature Subset
Selection4. Human Action Recognition
Method5. Experimentation & Results6. Conclusions7. ReferencesQ & A and Discussion
4
© A
lexandro
s A
ndre
Ch
aara
oui and F
ranci
sco F
lóre
z-R
evu
elt
a
(GEC
CO
’13
)
1. Introduction
Motivation and starting point Recognition of actions such as walking,
jumping or falling. Requirements:
High and stable recognition ratesReal-time suitability
Proposal of a visual feature with reduced extraction cost and low dimensionality
Feature subset selection
5
© A
lexandro
s A
ndre
Ch
aara
oui and F
ranci
sco F
lóre
z-R
evu
elt
a
(GEC
CO
’13
)
2. Radial Summary Feature
Human Silhouettes Relatively simple extraction
process Rich shape information Contour points
Radial Summary feature proposal Spatial alignment Feature
selection Low dimensionality, reduced
extraction cost, … Fig 1: Sample silhouette of the MuHAVi dataset [1].
6
© A
lexandro
s A
ndre
Ch
aara
oui and F
ranci
sco F
lóre
z-R
evu
elt
a
(GEC
CO
’13
)
2. Radial Summary Feature
Fig 2: Overview of the proposed Radial Summary feature.
7
© A
lexandro
s A
ndre
Ch
aara
oui and F
ranci
sco F
lóre
z-R
evu
elt
a
(GEC
CO
’13
)
3. Evolutionary Feature Subset Selection
Binary selection using a genetic algorithm Binary individual representation:
Active radial bin: uj = 1
Disabled radial bin: uj = 0
Random initial population (but one with all selected)
Fitness based on the evaluation of the feature Individuals with less active bins are favoured One-point crossover combination operator with
ranking selection Flip bit mutation operator Convergence termination criteria
8
© A
lexandro
s A
ndre
Ch
aara
oui and F
ranci
sco F
lóre
z-R
evu
elt
a
(GEC
CO
’13
)
4. Human Action Recognition Method
Pose Representations
Bag-of-Key-Poses Model
Sequences of Key Poses
9
© A
lexandro
s A
ndre
Ch
aara
oui and F
ranci
sco F
lóre
z-R
evu
elt
a
(GEC
CO
’13
)
4. Human Action Recognition Method
Learning based on Bag-of-Key-Poses Model The available pose representations
are reduced to a representative subset of key poses
We use the K-means clustering algorithm
10
© A
lexandro
s A
ndre
Ch
aara
oui and F
ranci
sco F
lóre
z-R
evu
elt
a
(GEC
CO
’13
)
4. Human Action Recognition Method
Sequence recognition Sequences of key poses Nearest-neighbour key poses Sequence matching (dynamic time
warping)
Fig 3: Sequences of key poses.
11
© A
lexandro
s A
ndre
Ch
aara
oui and F
ranci
sco F
lóre
z-R
evu
elt
a
(GEC
CO
’13
)
5. Experimentation & Results
Tested on the MuHAVi-MAS Dataset [1]
Two versions with 14 and 8 actions Manually Annotated Silhouettes Leave-one-actor-out (LOAO) and leave-one-
sequence out (LOSO) cross validations
Dataset Test Chaaraoui et al.
[2]
Radial Summar
y
Feature Selectio
n
State of the Art Rate [3]
MuHAVi-14
LOSO 94.1% 95.6% 98.5% 91.9%
MuHAVi-14
LOAO 86.8% 91.2% 94.1% 77.9%
MuHAVi-8 LOSO 98.5% 100% 100% 98.5%
MuHAVi-8 LOAO 95.6% 97.1% 100% 85.3%
12
© A
lexandro
s A
ndre
Ch
aara
oui and F
ranci
sco F
lóre
z-R
evu
elt
a
(GEC
CO
’13
)
5. Experimentation & Results Result of the feature
selection ~47% feature size
reduction
~14% temporal reduction
96 FPS overall recognition rate Fig 4: Resulting feature subset
selection of the MuHAVi-14 LOSO cross validation test (dismissed radial bins are shaded in gray).
13
© A
lexandro
s A
ndre
Ch
aara
oui and F
ranci
sco F
lóre
z-R
evu
elt
a
(GEC
CO
’13
)
6. Conclusions
Conclusions An evolutionary algorithm has been applied to
optimize action recognition. An appropriate feature for feature subset
selection has been proposed. We demonstrated that a guided selection of
feature elements can improve the recognition rate and reduce the computational cost.
Future work Real-valued weights instead of binary selection Action-class specific feature selection
14
© A
lexandro
s A
ndre
Ch
aara
oui and F
ranci
sco F
lóre
z-R
evu
elt
a
(GEC
CO
’13
)
7. References
[1] Singh, S., Velastin, S.A., Ragheb, H.: Muhavi: A multicamera human action video dataset for the evaluation of action recognition methods. In: 2010 Seventh IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS), pp. 48–55 (2010)
[2] Chaaraoui, A.A., Climent-Perez, P., Florez-Revuelta, F.: An Efficient Approach for Multi-view Human Action Recognition based on Bag-of-Key-Poses. In Salah, A., ed.: Human Behavior Understanding. Lecture Notes in Computer Science. Springer Berlin / Heidelberg (2012)
[3] A. Eweiwi, S. Cheema, C. Thurau, and C. Bauckhage. Temporal key poses for human action recognition. In Computer Vision Workshops (ICCV Workshops), IEEE International Conference on, pp. 1310-1317 (2011)
15 Q & A and Discussion
ALEXANDROS ANDRE CHAARAOUI AND FRANCISCO FLÓREZ-REVUELTA
HUMAN ACTION RECOGNITION
OPTIMIZATION BASED ON EVOLUTIONARY FEATURE
SUBSET SELECTION
… …
Amsterdam, July 6-10, 2013
Gen
etic
an
d
Evolu
tion
ary
C
om
pu
tatio
n
Con
fere
nce 2
01
3