SwATrack: A Swarm Intelligence-based Abrupt Motion Tracker

Mei Kuan Lim, Chee Seng ChanUniversity of Malaya, Center of Image and Signal Processing

50603 Kuala Lumpur, Malaysiaimeikuan@siswa.um.edu.my, cs.chan@um.edu.my

Dorothy MonekossoUniversity of West England, Eng. & Maths.

Bristol BS16 1QY, United Kingdomdorothy.monekosso@uwe.ac.uk

Paolo RemagninoKingston University, Comp. & Info. Sys.

Surrey KT1 2EE, United Kingdomp.remagnino@kingston.ac.uk

Abstract

Conventional tracking solutions are not feasi-ble in handling abrupt motion as they are basedon smooth motion assumption or constrained motionmodel; where the motion is often governed by a fixedGaussian distribution. Abrupt motion however, isnot subjected to motion continuity and smoothness.To assuage this, we propose a novel abrupt motiontracker that is based on swarm intelligence - the SwA-Track. Unlike existing swarm-based filtering methods,we firstly introduce an optimised swarm-based samplingstrategy to enrich the trade-off between the explorationand exploitation of the search space in search for theoptimal proposal distribution. Secondly, we proposeadaptive acceleration parameters to allow on the flytuning of the best mean and variance of the distributionfor sampling. The adaptive strategy requires no train-ing stage thus allowing flexibility in the motion model,while relaxing the number of particles deployed. Exper-imental results in both the quantitative and qualitativemeasures demonstrate the effectiveness of the proposedmethod in tracking abrupt motions.

1 Introduction

Visual tracking is pertinent in the tasks of motion-based recognition, automated surveillance and human-computer interaction [1]. In general, tracking is oftensimplified by assuming that the prior knowledge aboutthe motion is governed by Gaussian distribution basedon the Brownian or constant-velocity motion models[1, 2]. These assumptions however, do not hold true formany real world scenarios that exhibit abrupt motionssuch as in low frame rate videos, camera switching andfast motion. In this study, the abrupt motion refers tomotion (position) that is random and changes at irreg-ular intervals with unknown pattern; and cannot bemodelled simply as the Brownian or constant-velocitymotion model.While considerable research efforts exist in relation

to visual tracking, only a handful correspond to abruptmotion [3, 4, 5, 6]. Amongst them, most have been fo-cused on online learning techniques to handle abruptchanges in the appearance instead of motion. In whatconstitutes the closer work to ours, Wong and Dooley[3] proposed a template-matching algorithm to tracktable tennis ball. Their method applied an automatedtwo-pass segmentation method to detect the ball, fol-lowed by a Block Matching Detection (BMD) to specifythe position of the ball. This detection-based strategy

however, is impractical as it is highly dependent on thesearch window size and is computationally expensive.

Kwon and Lee [4] recently proposed the integrationof Wang Landau sampling strategy into the MarkovChain Monte Carlo framework (A-WLMC) to dealwith abrupt motion. The current frame is dividedinto N equal size sub-regions, and then the densityof each sub-region is learned by the proposed algo-rithm to guide the state transition. In another vari-ation, a feature-driven motion model from acceleratedsegment test feature matching is integrated into theparticle-filtering (PF) framework by Liu et al. [5]. In[6], another adaptive sampling strategy coupled witha density-grid-based predictive model (IA-MCMC) isintroduced to cope with abrupt motion. While theseworks have shown good results, they are still based onMonte Carlo sampling and require training stages thatconsequently increase the computational requirement.In short, an accurate tracking algorithm with reducedcomputational requirement remains a key challenge.

In summary, the contributions of this paper includes:1) The proposal of a swarm intelligence-based abruptmotion tracker. To the best of our knowledge, thereis yet to be any swarm intelligence-based method thathandles abrupt motion. Most of the abrupt motiontrackers are based on Monte Carlo sampling or its vari-ations that combines Monte Carlo sampling + swarmintelligence; which are computationally expensive. 2)No learning stage is required from specific trainingdata, thus providing flexibility while reducing the com-putational cost.

The rest of this paper is organised as follows. InSection 2, the motivation of the proposed work is pre-sented, followed by a detailed explanation of the pro-posed algorithm in Section 3. Experimental resultsand discussion are given in Section 4, while Section 5concludes this study.

2 Motivation

Visual tracking using stochastic solution is popularand involves a searching process for inferring the mo-tion of a target known as the state, xt from uncer-tain and ambiguous observations, zt at a given time, t.Given observations, z1:t−1 = {z1,... zt − 1} from timet=0 to t=1, the prediction stage applies the probabilis-tic transition model p(xt |xt−1) to predict the posterior,p(xt| z1:t−1) as Eq. 1.

To facilitate efficient tracking, in general, it is verycommon to simply use Brownian or constant-velocitymotion models governed by Gaussian distribution with

MVA2013 IAPR International Conference on Machine Vision Applications, May 20-23, 2013, Kyoto, JAPAN4-1

37

fixed mean and variance. However, these assumptionsfail when the motion of the target is abrupt. In suchcase, tracking tend to drift from the actual positioneven though the appearance model is flawless. This isdue to the inefficient samples which are drawn fromthe incorrect state space. Conventional particle-basedtracking are known to suffer from degeneracy and sam-ple impoverishment phenomenon. The former impliesthat a large computational effort is devoted to updat-ing particles whose contribution to the approximationto p(xt|z1:t) is almost zero, while the latter leads to aloss of diversity among the particles.Tracking with particle swarm optimisation (PSO)

however, assuage the degeneracy and sample impov-erishment phenomenon by optimising the search forthe optimal distribution without any assumptions orprior knowledge on the target’s motion. In general,PSO tracking involves propagating a swarm of parti-cles over the image at random with the aim of searchingfor the best-fit search window or proposal distributionof a target. The state and velocity of the ith particleare updated by

p(xt | z1:t−1) =

∫p(xt | x1:t−1)p(xt−1 | z1:t−1)dxt−1

(1)

xi,jt+1 = xi,j

t + vi,jt+1 (2)

vi,jt+1 = (w ∗ vi,jt ) + (c1 ∗ r1 ∗ (pBesti,jt − xi,jt ))

+(c2 ∗ r2 ∗ (gBestt − xi,jt ))

(3)

where v is the velocity of the particle, ω, r1, r2, c1 andc2 are the acceleration constants, pBest is the best po-sition from the particle’s individual search history andgBest is the swarm’s best solution which is also theoptimal position. The cooperative interaction and in-formation exchange between particles in PSO offers anorganised way to escape the local maxima and achievethe global maximum. A known drawback of the con-ventional PSO is its sensitivity to the parameters set-tings and the lack of a reasonable mechanism to controlthe acceleration parameters. This, when is applied intotracking abrupt motion will constrain the explorationand exploitation of the search space. Thus, the fullpotential of the PSO algorithm has not been utilised.

3 Proposed Tracking - SwATrack

The proposed SwATrack is a variant of the conven-tional PSO to track target with abrupt motion. SwA-Track is composed of two main components, i) appear-ance model that is usually the visual appearance cueand ii) velocity, v - we denote this as the motion modelthat describe the evolution of the state with time. Inthis paper, the appearance model is fixed - the HSVcolour histogram and we focus on the study of a novelrepresentation of the motion model that copes withabrupt motion efficiently.

3.1 Dynamic Model

The conventional PSO velocity estimation, v wherewe denote as the motion model in Eq. 3 is still sub-jected to a constraint search space and may fail to

cope with some degree of abrupt motion. This is dueto the fixed constant acceleration variables that re-quire prior fine-tuning. In addition, we discovered thatthere is statistical relationship between these parame-ters. Thus, to ease self-tuned of these parameters byobserving the quality of estimation, we introduced anovel motion model, v′:

v′ i,jt+1 = Ei,j

t (w ∗ v′ i,jt ) + (c1 ∗ r1 ∗ (pBesti,jt − xi,j

t ))

+(c2 ∗ r2 ∗ (gBestt − xi,jt ))(4)

where E is the exploration rate and c, r, ω are adaptiveparameters with condition p(w ∩ c1 ∩ c2) = 1.

3.1.1 Exploration rate, E

We define the exploration rate, E as the parametersthat adaptively i) increase the exploration with highvariance and ii) increase the exploitation with low vari-ance.

At every iteration, the quality of estimation for eachparticle is evaluated by its corresponding fitness valuef(xi

t). f(xit) → 1 indicates high likelihood whereas

f(xit) → 0 indicates low likelihood or no similarity

between an estimation and target. Thus, if f(xit) ≤

TMinF , E is increased along with the maximum num-ber of iterations, T by empirically determined stepsizesm and n respectively. This drives the swarm of par-ticles to explore the region beyond the current localmaxima, when the target cannot be tracked within asmaller region specified earlier (increase exploration).In contrary, when the particle search quality improves,(f(xi

t) ≤ TMinF ), E is decreased alongside T, con-straining the search around the current local maximum(exploitation). By utilising these exploitation and ex-ploration abilities [7], our method is capable to trackabrupt and smooth motions accurately and robustly;since there is no fixed assumption on the region forparticles sampling.

3.1.2 Adaptive Acceleration Parameters

A drawback of the conventional PSO is the lack of areasonable mechanism to control the acceleration pa-rameters (ω, c and r); which are constant. This lim-its the search space and therefore could not cope withabrupt motion. To overcome this, we propose a mech-anism to self-tune the acceleration parameters by util-ising the velocity information of the particles. Firstly,we normalised the acceleration parameters so that theycan be compared fairly with respect to the estimatedvelocity: p(w ∩ c1 ∩ c2) = 1.

The motion of the target throughout K number offrames is analysed to self-tune the settings of the in-ertia, w, cognitive, c1, and social weight, c2. Whenan object moves consistently in a particular direction,the inertia, w and cognitive weight, c1 values are in-creased to allow resistance of any changes in its stateof motion in the later frames. Otherwise, the socialweight c2 is increased by a stepsize to reduce its re-sistance to motion changes. The increase of the socialweight allows global influence and exploration of thesearch space, which is relevant when the motion of a

38

target is dynamic. Finally, the positions of the par-ticles, Xi,j

t+1(xi,jt+1,y

i,jt+1,w

i,jt+1,h

i,jt+1), are updated accord-

ing to Eq. 2 until the swarm reaches convergence.

4 Experimental Results

This section experimentally validates our approach,showing that SwATrack can consistently track a targetthat moves with abrupt motion. Here, the tracking im-plementation and evaluation criteria are discussed. Weevaluate the performance of the proposed method us-ing three public table tennis dataset. Two separatecomparisons are performed; i) comparison with thetwo state-of-the-art methods, a variation of template-matching method, BMD [3] and PF as proposed in [8];and (ii) comparison with conventional PSO tracking(where the acceleration parameters are of fixed con-stants). Note that the parameters in BDM and PFsare optimised to obtain the best results for compari-son. In addition, we validate our tracking results quali-tatively using three videos obtained from Youtube andthe benchmarked Tennis sequence [4, 6].We initialise manually the 2D position of the target

to be tracked in the first frame, and define a 15x15pixels patch around the 2D position of the target asthe reference frame. We employ colour histogram asour appearance model. The quality of the estimationis measured by its fitness value, which is representedby Bhattacharyya coefficient. Here, the initial valuesfor SwATrack are E=25, TMinF=0.6, w=0.4, c1=0.3,c2=0.3, T=10, m=0.1 and n=0.1, respectively. Notethat the choices of the initial values are not as crucial,given the adaptive tuning of these parameters in theproposed method.

4.1 Performance Evaluation

We quantify the accuracy using detection rate. Adetection is deemed correct if both of the criteria aremet: Firstly, the same identity should be given to thecorresponding object and secondly, the F −measure,Fj of j th object is > 0.5. Otherwise, tracking is classi-fied as incorrect. The detection rate indicates the ratiobetween the number of correctly detected frames andthe total number of frames in which the object appearsin the scene.

4.2 Quantitative Analysis

A set of 3 test sequences that are obtained from tabletennis game 1,2,3 were used for evaluation. Besidesthe abrupt motion of the ball, these sequences alsoexhibit (i) highly textured background; (ii) occlusions;(iii) small target size - about 8x8 to 15x15 pixels for animage resolution of 352x240 and (iv) low frame rate.As shown in Table 1, the proposed algorithm out-

performs the conventional PSO, PF and BDM eventhough the scale of ball becomes very small with minorocclusion. This is because existing trackers often fail totrack the table tennis ball accurately when it exhibits

1SIF data from http://www.sfu.ca/ ibajic/datasets.html2ITTF training dataset from http://www.ittf.com/

committees/umpires_referees/video/training/index.html3Xgmt open dataset from http://xgmt.open.ac.uk/table_

tennis

(a) Frame 11 of Dataset 1

(b) Frame 85 of Dataset 2

(c) Frame 32 and 35 of Dataset 3Figure 1. A comparison between our proposedmethod - SwATrack, PF, PSO and BDM on threepublicly available datasets.

SwATrack PSO PF [8] BDM [3]

Dataset 11

Accuracy(%) 92.1 86.4 82.0 81.3

Time (ms) 20.9 42.2 92.6 100.0

Dataset 22

Accuracy(%) 98.8 95.0 86.3 62.5

Time (ms) 22.8 30.6 90.8 105.0

Dataset 33

# particles 15 15 1000 N\AAccuracy(%) 98.4 95.5 80.3 98.4

Time (ms) 19.5 35.8 62.8 100.0

Table 1. Comparison of the tracking results be-tween SwATrack, PSO, PF and BDM

sudden change in motion as shown in Fig. 1. Althoughan increase in the number of particles in PF and an in-crease in the search space size in BDM would oftenincrease their tracking accuracy, this is however, infea-sible in practice. This is due to the large search space ofthe object state, which will lead to extremely expensivecomputational cost. In another alternative, an accu-rate motion model can be used to estimate the searchspace. However, an accurate motion model needed tobe learned from a set of training data and thus does notcope well with unknown motion. In comparison, ourproposed SwATrack does not make assumption on themotion model. Instead, the motion model is optimisedat each frame by utilising the interactions between par-ticles in a swarm for a more accurate tracking of abruptmotion. In addition, the proposed SwATrack requiresminimal number of particles; about 10-20% of the sam-ples used in PF. When the ball exhibits sudden changein its motion, the proposal distribution changes drasti-cally and thus leads to inaccurate tracking when sam-ples are drawn using the presumed Gaussian model.In PF, resampling from the incorrect distribution overa number of frames propagates the error and thus the

39

Figure 2. Sample shots of SwATrack results.

(a) PF

(b) A-WLMC

(c) IA-MCMC

(d) SwATrackFigure 3. A comparison between SwATrack, PF,A-WLCM[4] and IA-MCMC[6].

estimation is trapped in local optima. Meanwhile, theconventional PSO performs better tracking than thePF and BDM methods. However, the acceleration pa-rameters are not fully utilised in PSO, making it moretime-consuming and inconsistent as compared to theproposed method.

4.3 Qualitative Analysis

Further evaluation on SwATrack on videos obtainedfrom Youtube is as depicted in Fig. 2. (i) Abruptmotion: The first and second sequences aim to tracka synthetic ball which moves randomly, and a soc-cer ball which is being juggled in a free-style man-ner with highly textured background. It is observedthat SwATrack is able to track the targets accurately.(ii) Multiple targets: The third sequence demon-strates the capability of the proposed system to trackmultiple targets; two simulated balls moving at ran-dom. Most of the existing solutions are focused onsingle target. Finally, (iii) Low-frame-rate video:

The fourth sequence aims to track a tennis player ina low-frame rate video, which is down-sampled froma 700 frames sequence by keeping one frame in every20 frames. Here, the target (player) exhibits frequentabrupt changes which violate the smooth motion andconstant velocity assumptions. Thus, motion that isgoverned by Gaussian distribution based on the Brow-nian or constant-velocity motion models will not workin this case. Fig. 3 shows sample shots to compare theperformance between conventional PF tracking (500samples), A-WLMC (600 samples)[4], IA-MCMC (300samples)[6] and SwATrack (50 samples). It is observedthat the tracking accuracy of SwATrack is better thanPF and A-WLMC even by using fewer samples. Whilethe performance of SwATrack is comparable to IA-MCMC, SwATrack requires fewer samples and thusrequires less processing requirement. These results fur-ther verify that the proposed method is able to trackthe moving targets accurately and effectively, regard-less of the variety of change in the target’s motion.

5 Conclusion

We presented a novel swarm intelligence-basedtracker for visual tracking that copes with abrupt mo-tion efficiently. The proposed SwATrack optimised thesearch for the optimal distribution without making as-sumptions or need to learn the motion model before-hand. In addition, we introduced an adaptive mecha-nism that detects and responds to changes in the searchenvironment to allow on the fly tuning of the param-eters for a more accurate and effective tracking. Ex-perimental results show that the proposed algorithmimproves the accuracy of tracking while significantlyreduces the computational overheads, since it requiresless than 20% of the samples used by PF. In future, wewould like to further investigate the robustness of theproposed method as well as its behaviour change withthe different parameter settings and sampling strategy.

References

[1] H. Yang, L. Shao, F. Zheng, L. Wang, and Z. Song,“Recent advances and trends in visual tracking: A re-view,” Neurocomp., vol. 74, pp. 3823–3831, 2011.

[2] C. Garcıa Cifuentes, M. Sturzel, F. Jurie, and G. J.Brostow, “Motion models that only work sometimes,”in BMVC, 2012.

[3] K. C. P. Wong and L. S. Dooley, “Tracking table tennisballs in real match scenes for umpiring applications,”BJMCS, vol. 1(4), pp. 228–241, 2011.

[4] J. Kwon and K. Lee, “Wang-landau monte carlo-basedtracking methods for abrupt motions.” PAMI, 2012.

[5] Y. Liu, S. Lai, B. Wang, M. Zhang, and W. Wang,“Feature-driven motion model-based particle-filter track-ing method with abrupt motion handling,” Opt. Eng.,vol. 51(4), 2012.

[6] X. Zhou, Y. Lu, J. Lu, and J. Zhou, “Abrupt motiontracking via intensively adaptive markov-chain montecarlo sampling,” TIP, vol. 21, pp. 789 –801, 2012.

[7] L. P. Kaelbling, M. L. Littman, and A. W. Moore, “Re-inforcement learning: a survey,” JAIR, vol. 4, pp. 237–285, 1996.

[8] F. Yan, W. Christmas, and J. Kittler, “A tennis balltracking algorithm for automatic annotation of tennismatch,” Sig. Proc., pp. 619–628, 2005.

40