i

Tracking the Orientation and Axes Lengths of anElliptical Extended Object

Shishan Yang and Marcus Baum

Abstract—Extended object tracking considers the simultaneousestimation of the kinematic state and the shape parameters of amoving object based on a varying number of noisy detections.A main challenge in extended object tracking is the nonlinearityand high-dimensionality of the estimation problem. This workpresents compact closed-form expressions for a recursive Kalmanfilter that explicitly estimates the orientation and axes lengths ofan extended object based on detections that are scattered overthe object surface (according to a Gaussian distribution). Existingapproaches are either based on Monte Carlo approximations ordo not allow for explicitly maintaining all ellipse parameters. Theperformance of the novel approach is demonstrated with respectto the state-of-the-art by means of simulations.

Index Terms—Target tracking, extended object tracking, mul-tiplicative error, Kalman filter

I. INTRODUCTION

The objective of extended object tracking is to simultane-ously determine both the kinematic state and the shape param-eters of a moving object. With the development of novel near-field and high-resolution sensors, extended object tracking isbecoming increasingly important in many applications suchas autonomous driving [1], [2] and maritime surveillance [3].Recent overviews of extended object tracking methods andapplications are given in [4] and [5].

Most sensors for extended object tracking, e.g., LiDAR orradar devices, provide a varying number of spatially distributeddetections (measurements) per scan from the object. Depend-ing on the specific sensor and target, different scatteringpatterns can be distinguished. For example, in two-dimensionalspace, measurements can be scattered on the surface of theobject, or on the boundary of the object.

In case of spatially dense measurements, it might be pos-sible to extract detailed shape information from the object.For example, star-convex shape approximations as in [6], [7],[8], [9], [10] are widely-used for this purpose. In scenarioswith high measurement noise and a relatively low number ofmeasurements from the object, it is common to approximatethe object shape with an ellipse (see Fig. 1). The randommatrix approaches [11], [12], [13], [14] pioneered by Koch[11] can be seen as the state-of-the-art for estimating ellipticshape approximations in case of surface scattering. By meansof representing the shape estimate and its uncertainty withan Inverse-Wishart density, it is possible to derive compactclosed-form expression for a Bayesian measurement update.

S. Yang and M. Baum are with the Institute of Computer Science, Universityof Goettingen, Goettingen 37077, Germany (e-mail: shishan.yang@cs.uni-goettingen.de; marcus.baum@cs.uni-goettingen.de).

Fig. 1: Illustration of the extended object tracking problem.Multiple spatially distributed measurements are received fromthe target object and the objective is to determine an ellipticshape approximation in addition to the kinematic target state.

The Inverse Wishart density is defined on symmetric positive-definite (SPD) matrices, where it is specified by

• a d× d SPD scale matrix V and the• scalar degree of freedom v ∈ R.

For d = 2, the SPD scale matrix V can be interpreted as a two-dimensional elliptic shape estimate with an uncertainty that isspecified by the scalar v. An advantage of this representationis that the ellipse shape (including orientation, and semi-axeslengths) is uniquely defined by a single scale matrix. However,the uncertainty of the complete ellipse shape is encoded ina single one-dimensional value v. For this reason, it is notpossible to distinguish explicitly between the uncertainty ofthe semi-axes and the orientation, which is often necessary inpractical applications.

A. Contribution

The main contribution of this work is a novel elliptic shapetracking method that explicitly maintains an

• estimate for the orientation and semi-axes lengths, i.e., athree-dimensional vector, and

• the 3× 3 joint covariance of the shape estimate.By this means, it becomes possible to explicitly model thetemporal evolution of individual shape parameters and theirinterdependencies, which is highly relevant for numerouspractical applications. For example, it can be directly modeledthat the semi-axes are fixed (and unknown) but the orientationvaries.

We derive compact closed-form expressions for a recursiveupdate of the kinematic state and the aforementioned shapeparameters plus the respective covariance matrices. Due to thehigh degree of nonlinearity of the problem, a naıve applica-tion of standard estimation techniques such as the ExtendedKalman Filter (EKF), Unscented Kalman Filter (UKF) orSecond-Order Extended Kalman Filter (SOEKF) [15], [16]

arX

iv:1

805.

0327

6v2

[cs

.SY

] 3

Sep

201

9

ii

result in unsatisfactory estimation results. In fact, we have evenshown in our previous work [17] that the Linear MinimumMean Squared Estimator (LMMSE) is already for the specialcase of axis-aligned ellipses inconsistent.

The key components that lead to the compact closed-formexpressions are the followings:C1 An explicit measurement equation (corrupted by multi-

plicative noise) is formed that relates a measurement tothe kinematic state and shape parameters

C2 The kinematic state and the shape parameters are decou-pled, i.e., treated independently (as in the random matrixapproach)

C3 The kinematic state estimate is updated using the actualmeasurement. However, the shape parameters are updatedwith a pseudo-measurement constructed from the actualmeasurement

C4 As the measurement equation for the kinematic parame-ters involves multiplicative noise, a linearization is per-formed for the kinematic parameters, but the multiplica-tive noise is kept as a random variable for the momentcalculation.

C5 A standard linearization of the measurement equation forthe shape parameters does not yield a feasible estimatordue to the high nonlinearities. For this reason, we derivea problem-tailored second-order approximation. In orderto avoid the complex calculation of Hessian matrices,we exploit that the first two moments of the pseudo-measurement can be directly derived from the covariancematrix of the actual measurement.

This article is based on the two conference papers [18],[19]. Early ideas about the use of a multiplicative noise termto model a spatial distribution were discussed in [17]. In[18], we introduce a variant of the Second Order ExtendedKalman filter (SOEKF) for estimating the orientation andsemi-axes lengths of an ellipse. Unfortunately, it involvescomplex calculations of several Hessian matrices. In [19],we develop a method that works completely without Hessianmatrices. To track an unknown number of extended objects,an implementation that combines [19] and Probability DensityHypotheses (PHD) filter is presented in [20], [21]. The methodintroduced in this work improves over [19] by a more preciseapproximation of the covariance of the predicted measurement.Furthermore, a much more detailed evaluation and comparisonis provided.

The shape modeling with multiplicative noise in (C1) iscalled Multiplicative Error Model (MEM). The approximations(C1)-(C5) are the key to a Kalman filter-based update. For thisreason, the new method is called MEM-EKF*. EKF stands forExtended Kalman Filter (EKF) and the “*” emphasizes that(C1)-(C5) are problem tailored linearization techniques andmoment approximations.

B. Related WorkRelated work exists in the context of random matrix ap-

proaches, random hypersurface models, and particle filtering,and optimization-based approaches.

In [22], an alternative prediction for the random matrixapproach is derived that allows for kinematic state dependent

predictions. The model from Lan et al. [23] can captureorientation changes using a rotation matrix and and isotropicscaling. However, both methods [22], [23] still work withInverse Wishart densities, i.e., do not allow for explicitlymaintaining the uncertainty of individual shape parameters.The work [24] assumes the principal components of the mea-surements to be Gaussian and [25] independently estimates theorientation and semi-axes lengths based on fitting ideas, i.e.,both methods [24], [25] are not based on the common spatialdistribution model.

The random hypersurface approach [8], [26] also allowsto estimate elliptical shapes. However, the method discussedin [8], [26] uses the Cholesky decomposition of the shapematrix as a state vector, which has no intuitive meaning.Furthermore, the standard update in the random hypersurfaceapproach requires a point estimate for the angle from the centerto the measurement source, which is a poor approximation incase of high measurement noise. The roots of the proposedmethod here lie in the random hyperface approach, however,while the original random hypersurface approach uses a one-dimensional scaling factor, we here use a two-dimensionalscaling.

A recent overview of particle filter methods for extendedobject and group tracking is given in [5]. Based on Rao-Blackwellization, the random matrix approach has been com-bined with particle filtering techniques [27]. In [28], a con-volution particle filter is developed for tracking ellipticalshaped extended objects. Based on a hierarchical point processand particle-based approximations, multiple elliptic targets aretracked in [29] within the PHD filter framework. Particle-basedmethods are also widely-used for group object tracking, e.g.,[30] considers a virtual leader model for groups and a MarkovChain Monte Carlo (MCMC) method for approximating theposterior density.

A further class of methods optimizes over an entire batchof measurements in order to determine an estimate for thetarget object and its shape. In [31], the Probabilistic Multi-Hypothesis Tracking (PMHT) in combination with the randommatrix model is used to track multiple extended targets.In [7], a PMHT approach for estimating star-convex shapeapproximations is developed. In [32], a problem-tailored prob-ability density function is introduced in order to determine themaximum likelihood estimate of elliptical target objects.

C. Structure

This article is structured as follows: The next sectionintroduces the basic models that are used for tracking anelliptical shape approximation of a single extended object. Thefollowing Section III derives the compact closed-form expres-sions for a recursive measurement and time update. A detailedevaluation of the proposed method is provided in Section IV.Subsequently, this article is concluded in Section V.

II. MODELING AN ELLIPTICAL EXTENDED OBJECT

This section introduces the shape parameterization, mea-surement model, and process model for a single extendedobject whose shape is approximated as an ellipse.

iii

m

h1l1

α

yy

x

z h2l2

v

xRyR

Fig. 2: An illustration of our parameterization and measure-ment model. We omit the time index k and measurement indexi in this figure. The location of the object is m =

[m1,m2

]T.

The object shape is denoted as p =[α, l1, l2

]T. The measure-

ment y is measurement source z corrupted with measurementnoise v. The measurement source z is related to p usingmultiplicative noise h =

[h1, h2

]T. By anticlockwise rotating

coordinates system x-y through an angle of α, we have thedepicted ellipse axes-aligned in reference frame xR-yR.

A. Parameterization

The kinematic state of the object at time step k

rk =[mTk , m

Tk , . . .

]T(1)

consists of the center mk ∈ R2, velocity mk, and possiblefurther quantities such as acceleration. As motivated in theintroduction, the elliptical shape parameters at time k

pk =[αk, lk,1, lk,2

]T ∈ R3, (2)

contains the• angle αk, which indicates the counterclockwise angle of

rotation from the x-axis, and the• semi-axes lengths lk,1 and lk,2.

Although this ellipse parameterization is not unique, it isquite common in tracking applications (for other measurementmodels), see for example [33]. The parameterization of theobject extent as an SPD matrix V in the random matrixapproaches is unique. However, the scalar parameter v forthe uncertainty does not allow to distinguish between theuncertainties of the orientation and the semi-axes lengths. Aconversion from a covariance matrix of pk to scalar v inducesa loss of information.

B. Measurement Model

We adopt the widely-used spatial distribution model [34],[35] for modeling the object extent. The extended object givesrise to a varying number of independent two-dimensionalCartesian detections

Yk = y(i)k nk

i=1

in each time step k. Each individual measurement (detection)y

(i)k originates from a measurement source z(i)

k , which iscorrupted by an additive Gaussian measurement noise v(i)

k

with covariance of Cv .Each measurement source z(i)

k lies on the object extent andfollows a (uniform) spatial distribution. Similar to the random

matrix approach (e.g., [12]), we approximate the uniformspatial distribution with a Gaussian spatial distribution.

A key step to the proposed method – see (C1) in theintroduction – is the formulation of an explicit measurementequation, which relates a measurement source and the objectstate with the help of a multiplicative error term h. Consideran axis-aligned ellipse that lies in the origin and its semi-axeslengths are l1 and l2. Any point z(i) that lies on the ellipsecan be written as

z(i) =

[l1 00 l2

][h

(i)1

h(i)2

]︸ ︷︷ ︸:=h(i)

. (3)

To describe elliptical distributed measurement sources, weassume that the random variable h(i) is zero mean anddistributed on a unit circle. Note that if h(i)

1 and h(i)2 would be

independent and uniformly distributed on the interval [−1, 1],we would model a rectangular spatial distribution with length2 · l1 and width 2 · l2.

For an ellipse with orientation α and center m (see Fig. 2),a rotation and translation transformation of (3) gives us

z(i) = m+

[cosα − sinαsinα cosα

] [l1 00 l2

]︸ ︷︷ ︸

:=S

[h

(i)1

h(i)2

], (4)

where S specifies the orientation and size of the extendedobject. Incorporating the time index and sensor noise in (4)results in the measurement equation

y(i)k = Hrk + Skh

(i)k + v

(i)k , (5)

where H =[I2 0

]picks the object location out of the

kinematic state.In the same way as [12], we assume h(i)

k ∼ N (0,Ch) with

Ch =1

4I2 (6)

in order to match the covariance of an elliptical uniformdistribution.

Remark 1. By assuming additive Gaussian measurementnoise, the measurement likelihood becomes

p(y(i)k |rk,pk) = N (y

(i)k ; Hrk,SkC

hSTk + Cv) . (7)

We would like to note that the measurement likelihood (7)is equivalent to the likelihood used in the random matrixapproach [12], which is

p(y(i)k |rk,Xk) = N (y

(i)k ; Hrk, zXk + Cv) . (8)

As SkSTk is extension matrix Xk, (7) and (8) are equivalent

with Ch = 1z I2.

C. Dynamic Model

In general, there are no restrictions on the dynamic modelsfor the temporal evolution of the state and shape parameters.

iv

For the sake of simplicity, we here assume linear equationsaccording to

rk+1 = Arkrk +wr

k , (9)pk+1 = Ap

kpk +wpk , (10)

where• Ar

k and Apk are process matrices;

• wrk and wp

k are zero-mean Gaussian process noises withcovariance matrices Cw

r and Cwp .

III. ESTIMATION

This section presents closed-form expressions for the mea-surement and time update step based on the Kalman filter.For this purpose, we factorize the joint density for the objectkinematics and extension similar to [12], [23], see also (C2)in the introduction. By this means, it is not necessary tomaintain the cross-correlation between the kinematic state andthe shape parameters. However, it is important to note thatinterdependencies between the object kinematics and extensionare incorporated in the update and prediction formulas.

The derivation of a (nonlinear) Kalman filter is particu-larly difficult due to the high nonlinearities and zero-meanmultiplicative noise in the measurement equation. To solvethese issues, we derive problem-tailored approximations of therequired moments by means of combining linearization andanalytic moment calculation techniques.

A. Measurement Update

The measurements y(j)k nk

j=1 from time step k are incorpo-rated sequentially in the measurement update. For this purpose,let

r(i−1)k , p

(i−1)k and C

r(i−1)k , C

p(i−1)k .

denote the estimates for the kinematic state r(i−1)k and shape

parameters p(i−1)k plus the corresponding covariance matrices,

having incorporated all measurements up to time k − 1 plusthe measurements y(j)

k i−1j=1 from time k.

In the measurement update, the next measurement y(i)k is

incorporated in order to obtain the updated estimates

r(i)k , p

(i)k and C

r(i)k , C

p(i)k .

Note that – according to this notation – the predictedestimates for time k are denoted as (•)(0)

k , correspondingly.

Remark 2. It is important to note that the measurementsfrom a single time scan are incorporated sequentially (inan arbitrary order). Due to approximations, slightly differentresults might be obtained for different orderings.

As shown in [17], the object extent cannot be estimated witha linear estimator that works with the actual measurement,i.e., the shape parameters do not change when updated witha single measurement y(i)

k in the Kalman filter framework.For this reason, a pseudo-measurement is constructed basedon y(i)

k in order to update the shape parameters. This can beseen as an uncorrelated transformation as discussed in [36].

TABLE I: Measurement update of the MEM-EKF* algorithm.Source code: https://github.com/Fusion-Goettingen/

Input: Measurements y(i)k

nki=1, predicted estimates r

(0)k , p

(0)k , C

r(0)k ,

Cp(0)k , measurement noise covariance Cv , H as defined in (5), multiplicative

noise covariance Ch, F and F as defined in (23)Output: updated estimates r

(nk)k , p

(nk)k and C

r(nk)k , C

p(nk)k

For i = 1, · · · , nk[α l1 l2

]T= p

(i−1)k

S =

[S1

S2

]=

[cosα − sinαsinα cosα

] [l1 00 l2

]J1 =

[−l1 sinα cosα 0−l2 cosα 0 − sinα

]J2 =

[l1 cosα sinα 0−l2 sinα 0 cosα

]CI = SChST

CII = [εmn] = trCp(i−1)

k JTnC

hJm

for m,n = 1, 2

M =

2S1ChJ1

2S2ChJ2

S1ChJ2 + S2ChJ1

y(i)k = Hri−1

k

Cry(i)

k = Cr(i−1)k HT

Cy(i)

k = HCr(i−1)k HT + CI + CII + Cv

Y(i)k = F

((y

(i)k − y

(i)k )⊗ (y

(i)k − y

(i)k ))

Y(i)k = Fvect

Cy(i)

k

C

pY (i)k = C

p(i−1)k MT

CY (i)k = F(C

y(i)k ⊗C

y(i)k )(F + F)T

r(i)k = r

(i−1)k + C

ry(i)k

(C

y(i)k

)−1 (y(i)k − y

(i)k

)C

r(i)k = C

r(i−1)k −C

ry(i)k

(C

y(i)k

)−1 (C

ry(i)k

)Tp(i)k = p

(i−1)k + C

pY (i)k

(C

Y (i)k

)−1 (Y

(i)k − Y

(i)k

)C

p(i)k = C

p(i−1)k −C

pY (i)k

(C

Y (i)k

)−1 (C

pY (i)k

)TEnd

1) Kinematic State Update: The kinematic state estimate isupdated according to the Kalman filter update equations usingthe actual measurement y(i)

k , see (C3),

y(i)k = Hr

(i−1)k , (11)

r(i)k = r

(i−1)k + C

ry(i)k

(C

y(i)k

)−1 (y

(i)k − y

(i)k

),(12)

Cr(i)k = C

r(i−1)k −C

ry(i)k

(C

y(i)k

)−1 (C

ry(i)k

)T

.(13)

The challenge is to find compact closed-form approximationsto the required moments, i.e., the covariance of the mea-surement C

y(i)k and the cross-correlation C

ry(i)k between the

measurement and kinematic state.The measurement equation (5) is linear in the kinematic

state but nonlinear in the shape parameters due to the shapematrix Sk. Linearizing Skh

(i)k with respect to pk at p(i−1)

k

and keeping h(i)k as a random variable (C4) gives us

Skh(i)k ≈ S

(i−1)k h

(i)k︸ ︷︷ ︸

I

+

(h

(i)k

)T

J1

(i−1)

k(h

(i)k

)T

J2

(i−1)

k

(pk − p(i−1)k

)︸ ︷︷ ︸

II(14)

v

where •(i−1)k denotes matrix • evaluated at the (i−1)-th shape

estimate p(i−1)k , J1 and J2 are the Jacobian matrices of the

first row and second row of S, i.e.,

J1 =∂S1

∂p=

[−l1 sinα cosα 0−l2 cosα 0 − sinα

], (15)

J2 =∂S2

∂p=

[l1 cosα sinα 0−l2 sinα 0 cosα

], (16)

with

S1 =[l1 cosα −l2 sinα

]and S2 =

[l1 sinα l2 cosα

]. (17)

Note that the terms I and II in (14) are uncorrelated. Thecovariance of Skh

(i)k is approximated as the sum of CI and

CII, where

CI = S(i−1)k Ch(S

(i−1)k )T , (18)

[εmn]︸ ︷︷ ︸CII

= tr

Cp(i−1)

k

(Jn

(i−1)

k

)T

ChJm(i−1)

k

,(19)

for m,n ∈ 1, 2.The derivation of (19) is shown in Appendix A-A. The

cross-covariance and covariance are

Cry(i)

k = Cr(i−1)k HT (20)

Cy(i)

k = HCr(i−1)HT + CI + CII + Cv . (21)

2) Shape Update: A pseudo-measurement is constructedusing the 2-fold Kronecker product (C3). For a two-dimensional vector y =

[y1 y2

]T, its 2-fold Kronecker

product ⊗ is defined as

y ⊗ y =[y2

1 y1y2 y2y1 y22

]T. (22)

Furthermore, each measurement is shifted by the expectedmeasurement, and multiplied by a matrix

F =

1 0 0 00 0 0 10 1 0 0

or F =

1 0 0 00 0 0 10 0 1 0

(23)

to remove the duplicate element resulting from the 2-foldKronecker product. All told, the pseudo-measurement is

Y(i)k = F

((y

(i)k − y

(i)k )⊗ (y

(i)k − y

(i)k )). (24)

Note that (24) is an uncorrelated conversion (c.f., Theorem 3in [36]), which means the pseudo-measurement is uncorrelatedwith the actual measurement.

The shape parameters are updated with the pseudo-measurement Y (i)

k using the Kalman filter update formulas

p(i)k = p

(i−1)k + C

pY (i)k

(C

Y (i)k

)−1 (Y

(i)k − Y

(i)k

)(25)

Cp(i)k = C

p(i−1)k −C

pY (i)k

(C

Y (i)k

)−1 (C

pY (i)k

)T

(26)

where Y (i) denotes the predicted pseudo-measurement, CY (i)k

is the covariance of the pseudo-measurement, and CpY (i)k is

the cross-covariance between the pseudo-measurement and theshape parameters.

By constructing the pseudo-measurement in this way, theexpected pseudo-measurement happens to consist of all cen-tralized second moments of the actual measurements, whichcan be extracted directly from (21), see (C5). To show this,we introduce the vect-operator, which constructs a columnvector from a matrix by stacking its column vectors. Given

the covariance matrix of measurement Cy(i)k =

[c11 c12

c12 c22

],

a vect-operator gives us

vect

Cy(i)k

=[c11 c12 c12 c22

]T, (27)

which equals

E

(y(i)k − y

(i)k )⊗ (y

(i)k − y

(i)k )

. (28)

The expected i-th pseudo-measurement is

Y(i)k = Fvect

C

y(i)k

. (29)

The predicted pseudo-measurement covariance is

CY (i)k =

2c211 2c212 2c11c12

2c212 2c222 2c22c12

2c11c12 2c22c12 c11c22 + c212

, (30)

= F(Cy(i)k ⊗C

y(i)k )(F + F)T . (31)

Equation (30) is obtained using Isserlis’s theorem [37] (seeAppendix A-B). Equation (31) is a compact formulation of(30).

The cross-covariance between the pseudo-measurement andthe shape parameters is approximated by linearization of (24)according to

CpY (i)k = C

p(i−1)k

(M

(i−1)k

)T

, (32)

with

M =

2S1ChJ1

2S2ChJ2

S1ChJ2 + S2ChJ1

. (33)

The derivation of (33) is shown in the Appendix A-B1. Thepseudo code of measurement update is given in Table I.

B. Time Update

As the temporal evolution of both the kinematic state andthe shape parameters follow a linear model, the time updatecan be performed with the standard Kalman filter time updateformulas, i.e.,

r(0)k+1 = Ar

kr(nk)k , (34)

Cr(0)k+1 = Ar

kCr(nk)k (Ar

k)T + Cwr . (35)

and

p(0)k+1 = Ap

k p(nk)k , (36)

Cp(0)k+1 = Ap

kCp(nk)k (Ap

k)T + Cwp . (37)

vi

−10 0 10

−10

0

10

x→

Exa

mpl

arE

stim

ates

No Measurement Noise, Cv = 0

Ground Truth

Measurement

MEM-MC

MEM-SOEKF

MEM-EKF*

−10 0 10

−10

0

10

x→

y→

High Measurement Noise, Cv = diag[l1, l2]

0 20 40 60 80 1000

2

4

6

8

i→

Est

imat

ion

Err

or[m

]

0 20 40 60 80 1000

2

4

6

8

i→

MEM-MCMEM-SOEKFMEM-EKF*

Fig. 3: Simulation with a stationary ellipse. The first row shows the ground truth, exemplar measurements, and the estimatesafter 100 measurement updates. The bottom row plots the root mean squared Gaussian Wasserstein distance averaged over 100runs.

0 1 2 3 4 5 6 7 8 9 10

−4

−3

−2

−1

0

1

x [km]

y[km]

Measurement

Ground Truth

Feldmann et al.

Lan et al.

MEM-EKF*

0 1 2 3 4 5 6 7 8 9 10

−4

−3

−2

−1

0

1

x [km]

y[km]

Measurement

Ground Truth

Feldmann et al.

Lan et al.

MEM-EKF*

0 1 2 3 4 5 6 7 8 9 10

−4

−3

−2

−1

0

1

x [km]

y[km]

Measurement

Ground Truth

Feldmann et al.

Lan et al.

MEM-EKF*

0 1 2 3 4 5 6 7 8 9 10

−4

−3

−2

−1

0

1

x [km]

y[km]

Measurement

Ground Truth

Feldmann et al.

Lan et al.

MEM-EKF*

0 1 2 3 4 5 6 7 8 9 10

−4

−3

−2

−1

0

1

x [km]

y[km]

Measurement

Ground Truth

Feldmann et al.

Lan et al.

MEM-EKF*Fig. 4: The measurements, trajectory, and estimation results of a single example run.

IV. EVALUATION

In this section, we first evaluate the accuracy of the de-veloped moment approximations. Then, the benefits of thedeveloped shape tracker are demonstrated with respect to therandom matrix approach in a simple extension dynamics. Inthe end, we integrate turn rate estimation and tested in ascenario in which object extent is coupled with its kinematics.

We assess location and extent errors simultaneously with asingle score by means of the Gaussian Wasserstein distance[38] as proposed [39]. It is very important to note thatorientation and axes-lengths errors are combined in a singlescalar value.

The Gaussian Wasserstein distance compares two ellipsesaccording to

d(µ1,Σ1,µ2,Σ2)2 =‖ µ1 − µ2 ‖2

+ tr

Σ1 + Σ2 − 2

√√Σ1Σ2

√Σ1

, (38)

where the ellipses are specified by their locations µ1 ∈ R2

and µ2 ∈ R2 and SPD shape matrices Σ1 ∈ R2×2 and Σ2 ∈R2×2. In this case, the first ellipse is the ground truth and thesecond one is the extended object tracking method estimate.Note that an SPD shape matrix is computed using SkS

Tk .

vii

0 50 100 1500

50

100

150

k →

[m]

Feldmann et al. Lan et al. MEM-EKF*

Fig. 5: Extent error based on the mean squared GaussianWasserstein distance.

A. Evaluation of Moment Approximations

First, we evaluate the quality of the proposed momentapproximations for the kinematic state (12) and (13), andthe shape parameters (25) and (26) compared to the MonteCarlo moment approximation and our Second-Order EKF[18], which requires the calculation of Hessian matrices. Bothmethods are computationally much more complex than theproposed tracker. As we focus on the moment approximationsof the measurement update, we restrict ourselves to a scenariowith a non-moving object. The considered object is locatedin the origin with semi-axes lengths 2 and 9 meters andit is counter-clockwise rotated π

3 . The prior for the shapeparameters is

r(0)1 =

[1 1

], C

r(0)1 = diag

[1 1

],

p(0)1 =

[0 2 12

], C

p(0)1 = diag

[1 4 9

]for all three methods. Two different measurement noise co-variance matrices are evaluated and the simulation results areshown in Fig. 3.

As expected, the Monte Carlo moments approximationoutperforms the analytic approaches in both low and highmeasurement noises scenarios. In case of high measurementnoise, our moments approximation is almost as good as theMonte Carlo approximation unexpected. Fig. 3 shows thatthere are no significant visual differences between the meth-ods. We can conclude that the derived moment approximationsnearly matches the true exact moments in low and high noisescenarios.

B. Comparison with Random Matrix Approaches using Con-stant Velocity Motion Models

In the second simulation, we compare our algorithm withthe two random matrix approaches by Feldmann et al. [12]and Lan et al. [23].

The considered scenario involves a target object with anunknown extent. However, only its orientation is changing overtime, its semi-axes are fixed. With the MEM-EKF* we canassume a low system noise on the semi-axes lengths, and highsystem noise on the orientation. However, the random matrixapproaches [12], [23] cannot model this scenario precisely (asthere is only a single parameter for the extent uncertainty). Forthis reason, the MEM-EKF* is able to outperform the random

0 1 2 3 4 5

0

1

2

3

x [km]

y[k

m]

Fig. 6: True trajectory of variable turn simulation.

matrix approaches in this scenario. The true track is similar asin [12] and [23]. The extended object has diameters of 340mand 80m. It starts at the coordinate origin and it moves witha constant speed of 50km/h. At each time step, measurementsources are generated from a uniform distribution on theelliptical extent. The number of measurements per time scanis drawn from a Poisson distribution with mean 20 as in [23].The variances of the measurement noise are 10000m2 and400m2 for each dimension.

We set τ to 50 in the approach of Feldmann et al. ;δ to 40 in the approach of Lan et al. ; v to 56 in bothrandom matrix approaches. The extension transition matrixin Lan et al. ’s approach is 1

δkI2. For our method, the prior

of the shape variables is specified by the covariance matrixCp

0 = diag [1, 702, 702]. The process noise covariance isset to Cw

p = diag [0.1, 1, 1], and the transition matrix isApk = I3. The process noise covariance for the kinematic state

is diag [100, 100, 1, 1] for all three estimators.Measurements, trajectory, and estimation results of an ex-

ample run are depicted in Fig. 4. Both random matrix trackershave worse results during turns. The extent error accordingto the Gaussian Wasserstein distance is depicted in Fig. 5.From Fig. 5 we can conclude that Feldmann et al. and Lan etal. perform similarly overall. This is expected as both randommatrix methods are the same if no special dynamic model forthe extend is used. For all three turns the proposed tracker hasa lower error compared to both random matrix approaches dueto the aforementioned assumption of (nearly) static semi-axis.

C. Comparison with Random Matrix Approaches using Con-stant Turn Motion Models

For a manoeuvring target, the object orientation is typicallycoupled with the turn rate [22]. Assuming that the size of theextended object is constant (plus noise), we can integrate thecorrelation between orientation and turn rate in the predictionof the MEM-EKF* according to

p(0)k+1 = Br

(nk)k + p

(nk)k , (39)

Cp(0)k+1 = BC

r(nk)k BT + C

p(nk)k + Cw

p , (40)

where B =

[01×4 T02×4 02×1

]picks out turn rate Ω

(nk)k .

In this simulation, a target object that follows a variableturn-rate model is simulated, similar to [22]. The diameters of

viii

0 20 40 60 80

50

100

150

Feldmann et al. Granstrom et al. M4MEM-EKF* (1) MEM-EKF* (2)

Fig. 7: Mean squared Gaussian Wasserstein distance for 100runs.

the simulated object are 170m and 40m and the true track isshown in Fig. 6. In the first 25 time steps, the object moveswith a constant velocity of 150m/s. Afterwards, its turn rateincreases from 0 to 20 degree per second in 20 time steps, anddecreases to 0 in 20 time steps. The object evolves additional5 time steps according to a constant velocity model in the endof the trajectory. The number of measurements for each timestep is Poisson distributed with mean 20 and the measurementnoise covariance is Cv = diag[10000 400]. We compare theMEM-EKF* with the random matrix approach M4 proposed[22]. M4 approximates the density of the object extent withan inverse Wishart density by minimizing Kullback-Leiblerdivergence in the prediction step. As a baseline, we alsoinclude the random matrix approach from [12], which doesnot incorporate the turn-rate.

For all estimators, the object kinematics is modeled as aconstant turn model and the prior is

r(0)1 =

[100 100 100 20 0.001

]T(41)

Cr(0)1 = diag[1600I2 16I2 0.001]. (42)

The parameters for the random matrix approaches are v = 56,τ = 5, and T = 1s. The prior for our shape variables is

p(0)1 =

[π3 200 90

]T, C

p(0)1 = diag[0.2 360I2]. (43)

The process noise covariace matrices for the location, velocity,and turn rate are

Cwr = diag[1000I2 100I2], Cw

Ω = 0.001.

The MEM-EKF* constrains the temporal evolution of theextent via process a suitable noise covariance for the shapeparameters. In the same way as the simulation in Section IV-B,the process noise covariance matrix is tuned such that the semi-axes are only allowed to change slightly over time.

To demonstrate this effect, we choose two different processnoise covariance matrices for our tracker and refer them as

MEM-EKF* (1) with Cwp = diag[.01 I2] (44)

MEM-EKF* (2) with Cwp = diag[.1 40I2] (45)

MEM-EKF* (1) has less process noise on the shape variables,i.e., only slight changes in the lengths of the semi-axes are

possible. MEM-EKF* (2) has rather high process noise on theshape variables such that larger changes are possible.

The root mean squared Gaussian Wasserstein distance isgiven in Fig. 7. As expected, the tracking performance im-proves significantly with turn rate estimation and a constantturn motion model. MEM-EKF* (1) has the least estimationerror, as size changes are constrained using small process noisecorrespondingly. However, note that, compared to randommatrix approaches, MEM-EKF* involves more parameters.This is an advantage as well as a disadvantage. Its performanceis sensitive to the choice of the parameters, e.g., the results ofMEM-EKF* (2) are worse as the used parameters do not fit.

V. CONCLUSION

Extended object tracking is challenging – especially for thecase of measurements that are scattered on the surface ofthe object. As the underlying estimation problem is highlynonlinear, closed-form solutions are rarely available.

In this work, we introduced a new closed-form trackercalled MEM-EKF* for the orientation and axes-lengths ofan elliptical extended object. For this purpose, an explicitmeasurement equation is formulated via a multiplicative error.A problem-tailored combination of analytic moment calcu-lation and linearization techniques then allows to derive aKalman filter-based measurement update. A major benefit ofour method is that it provides an intuitive parameterizationof an ellipse, which allows for directly modeling relevantmotion models. Furthermore, the full joint covariance of theellipse parameters is available. The closed-form formulas arecompact, i.e., they are not significantly more complex than thestandard Kalman filter formulas.

In the future, we will investigate extensions of the approach,e.g., for non-elliptical shapes or known extent [40] and it willbe embedded into multi-object trackers such as the extendedtarget PHD filter [21], [41].

ACKNOWLEDGMENT

This work was supported by the German Research Founda-tion (DFG) under grant BA 5160/1-1.

APPENDIX ADERIVATIONS FOR THE JOINT MOMENTS

A. Equation (19)

As h is zero-mean,1 for m,n = 1, 2, we have

cov

[hTJ1

hTJ2

]p,

[hTJ1

hTJ2

]p

= [εmn] , (46)

1For the sake of compactness, we omit the measurement index (i) and timeindex k

ix

with

εmn = EhTJmpp

TJn

Th, (47)

= E

trhTJmpp

TJn

Th

, (48)

= E

trppTJn

ThhTJm

, (49)

= tr

EppTJn

ThhTJm

, (50)

= tr

CpJn

TChJm

. (51)

Equation (48) follows from the fact that hTJmppTJn

Th is

1 × 1. As trace is invariant under cyclical permutations, wehave (49). Equation (50) follows from the property that trace isa linear operator and can commute with expectation. Equation(51) follows form the independence between h and p.

B. Pseudo-measurement Covariance

To calculate the covariance of the, pseudo-measurement weneed the fourth centralized moments of original measurement,which can be calculated using Isserlis’s theorem [37] or Wick’stheorem [42]. Given a measurement y =

[y1 y2

]T, the

corresponding pseudo-measurement isY 1

Y 2

Y 3

=

(y1 − y1)2

(y2 − y2)2

(y1 − y1)(y2 − y2)

. (52)

From Isserlis’ theorem, we get

E

(Y 1)2

= 3c211 , (53)

E

(Y 2)2

= 3c222 , (54)E Y 3Y 1 = 3c11c12 , (55)E Y 3Y 2 = 3c22c12 , (56)E

(Y 3)2

= E Y 1Y 2 = c11c22 + 2c212 , (57)

where cmn denotes E (ym − ym)(yn − yn) for m,n ∈1, 2. Based on the results above, the calculation of mn-th entry of pseudo-measurement covariance matrix simplyfollows

cov Y m,Y n = E Y mY n − E Y mE Y n , (58)

for m,n ∈ 1, 2, 3. After a few further calculations, we getcovariance of pseudo-measurement as in (30).

1) Linearization of the Pseudo-measurement Equation:Let S1 and S2 denote the first and second row of matrixS. Similarly, H1 and H2 refer to the first and second rowof H. Accordingly, the pseudo-measurement equation (24) isrewritten as (time and measurement indices are omitted)

g(r,p) =

(H1r + S1h+ v1 − y1)2

(H2r + S2h+ v2 − y2)2

(H1r + S1h+ v1 − y1) (H2r + S2h+ v2 − y2)

(59)

The cross-covariance of the pseudo-measurement and shapeparameters are approximated using

CpY = cov

∂g

∂p

∣∣∣∣p=p

(p− p), p

(60)

= Cp

E

∂g

∂p

∣∣∣∣p=p

︸ ︷︷ ︸

M

T

(61)

After applying the chain rule, ∂g∂p equals 2 (H1r + S1h + v1 − y1)hTJ1

2 (H2r + S2h + v2 − y2)hTJ2

(H1r + S1h + v1 − y1)hTJ2 + (H2r + S2h + v2 − y2)hTJ1

(62)

with J1 and J2 are given in (15) and (16). Evaluating thefirst row of (62) at p, we have

2(H1r − y1)hTJ1 + 2S1hhTJ1 + 2v1h

TJ1 (63)

Taking the expectation of (63) gives us

2S1ChJ1 (64)

After a similar derivation for the second and third row of (62),we get

M =

2S1ChJ1

2S2ChJ2

S1ChJ2 + S2ChJ1

. (65)

REFERENCES

[1] F. Kunz, D. Nuss, J. Wiest, H. Deusch, S. Reuter, F. Gritschneder,A. Scheel, M. Stubler, M. Bach, P. Hatzelmann, C. Wild, and K. Di-etmayer, “Autonomous Driving at Ulm University: A Modular, Robust,and Sensor-Independent Fusion Approach,” in IEEE Intelligent VehiclesSymposium (IV), Jun. 2015, pp. 666–673.

[2] T. Hirscher, A. Scheel, S. Reuter, and K. Dietmayer, “Multiple ExtendedObject Tracking using Gaussian Processes,” in Proceedings of the 19thInternational Conference on Information Fusion (Fusion 2016), Jul.2016, pp. 868–875.

[3] G. Vivone and P. Braca, “Joint Probabilistic Data Association Trackerfor Extended Target Tracking Applied to X-Band Marine Radar Data,”IEEE Journal of Oceanic Engineering, vol. 41, no. 4, pp. 1007–1019,Oct. 2016.

[4] K. Granstrom, M. Baum, and S. Reuter, “Extended Object Tracking:Introduction, Overview and Applications,” ISIF Journal of Advances inInformation Fusion, vol. 12, no. 2, Dec. 2017.

[5] L. Mihaylova, A. Carmi, F. Septier, A. Gning, S. Pang, and S. Godsill,“Overview of Bayesian Sequential Monte Carlo Methods for Group andExtended Object Tracking,” Digital Signal Processing, vol. 25, pp. 1–16,Feb. 2014.

[6] M. Baum and U. D. Hanebeck, “Shape Tracking of Extended Objectsand Group Targets with Star-Convex RHMs,” in Proceedings of the14th International Conference on Information Fusion (Fusion 2011),Chicago, Illinois, USA, Jul. 2011.

[7] H. Kaulbersch, M. Baum, and P. Willett, “EM Approach for TrackingStar-Convex Extended Objects,” in Proceedings of the 20th InternationalConference on Information Fusion (Fusion 2017), Xi’an, P.R. China, Jul.2017.

[8] M. Baum and U. D. Hanebeck, “Extended Object Tracking with RandomHypersurface Models,” IEEE Transactions on Aerospace and ElectronicSystems, vol. 50, pp. 149–159, Jan. 2014.

[9] N. Wahlstrom and E. Ozkan, “Extended Target Tracking Using GaussianProcesses,” IEEE Transactions on Signal Processing, vol. 63, no. 16, pp.4165–4178, Aug. 2015.

x

[10] E. Ozkan, N. Wahlstrom, and S. J. Godsill, “Rao-Blackwellised ParticleFilter for Star-Convex Extended Target Tracking Models,” in Proceed-ings of the 19th International Conference on Information Fusion (Fusion2016), July 2016, pp. 1193–1199.

[11] W. Koch, “Bayesian Approach to Extended Object and Cluster Trackingusing Random Matrices,” IEEE Transactions on Aerospace and Elec-tronic Systems, vol. 44, no. 3, pp. 1042–1059, Jul. 2008.

[12] M. Feldmann, D. Franken, and W. Koch, “Tracking of Extended Objectsand Group Targets using Random Matrices,” IEEE Transactions onSignal Processing, vol. 59, no. 4, pp. 1409–1420, 2011.

[13] U. Orguner, K. Granstrom, and C. Lundquist, “Extended Target Trackingwith a Cardinalized Probability Hypothesis Density Filter,” in Proceed-ings of the 14th International Conference on Information Fusion (Fusion2011), Chicago, Illinois, USA, Jul. 2011.

[14] J. Lan and X. Rong Li, “Tracking of Extended Object or TargetGroup Using Random Matrix - Part I: New Model and Approach,” inProceedings of the 15th International Conference on Information Fusion(Fusion 2012), Jul. 2012, pp. 2177–2184.

[15] R. Henriksen, “The Truncated Second-Order Nonlinear Filter Revisited,”IEEE Transactions on Automatic Control, vol. 27, no. 1, pp. 247–251,1982.

[16] M. Roth and F. Gustafsson, “An Efficient Implementation of the SecondOrder Extended Kalman Filter,” in Proceedings of the 14th InternationalConference on Information Fusion (Fusion 2011), Chicago, Illinois,USA, July 2011.

[17] M. Baum, F. Faion, and U. D. Hanebeck, “Modeling the Target Extentwith Multiplicative Noise,” in Proceedings of the 15th InternationalConference on Information Fusion (Fusion 2012), Singapore, Jul. 2012.

[18] S. Yang and M. Baum, “Second-Order Extended Kalman Filter forExtended Object and Group Tracking,” in Proceedings of the 19th Inter-national Conference on Information Fusion (Fusion 2016), Heidelberg,Germany, Jul. 2016.

[19] ——, “Extended Kalman Filter for Extended Object Tracking,” inProceedings of the 42nd IEEE International Conference on Acoustics,Speech, and Signal Processing (ICASSP 2017), New Orleans, USA, Mar.2017.

[20] F. Teich, S. Yang, and M. Baum, “GM-PHD filter for Multiple ExtendedObject Tracking based on the Multiplicative Error Shape Model andNetwork Flow Labeling,” in Proceedings of the IEEE Intelligent VehiclesSymposium (IV 2017), Redondo Beach, CA, USA, Jun. 2017.

[21] S. Yang, F. Teich, and M. Baum, “Network Flow Labeling for ExtendedTarget Tracking PHD Filters,” IEEE Transactions on Industrial Infor-matics, pp. 1–1, 2019.

[22] K. Granstrom and U. Orguner, “A New Prediction for Extended Targetswith Random Matrices,” IEEE Transactions on Aerospace and ElectricalSystems, 2013.

[23] J. Lan and X. R. Li, “Tracking of Extended Object or Target GroupUsing Random Matrix: New Model and Approach,” IEEE Transactionson Aerospace and Electrical Systems, vol. 52, no. 6, pp. 2973–2988,Dec. 2016.

[24] J. Degerman, J. Wintenby, and D. Svensson, “Extended Target Trackingusing Principal Components,” in Proceedings of the 14th InternationalConference on Information Fusion (Fusion 2011), Chicago, Illinois,USA, Jul. 2011.

[25] B. Li, C. Mu, Y. Bai, J. Bi, and L. Wang, “Ellipse Fitting based Approachfor Extended Object Tracking,” Mathematical Problems in Engineering,vol. 2014, 2014.

[26] M. Baum, M. Feldmann, D. Franken, U. D. Hanebeck, and W. Koch,“Extended Object and Group Tracking: A Comparison of Random Ma-trices and Random Hypersurface Models,” in Proceedings of the IEEEISIF Workshop on Sensor Data Fusion: Trends, Solutions, Applications(SDF 2010), Leipzig, Germany, Oct. 2010.

[27] G. Vivone, K. Granstrom, P. Braca, and P. Willett, “Multiple SensorMeasurement Updates for the Extended Target Tracking Random MatrixModel,” IEEE Transactions on Aerospace and Electronic Systems,vol. 53, no. 5, pp. 2544–2558, 2017.

[28] D. Angelova, L. Mihaylova, N. Petrov, and A. Gning, “A ConvolutionParticle Filtering Approach for Tracking Elliptical Extended Objects,” inProceedings of the 16th International Conference on Information Fusion(Fusion 2013), Istanbul, Turkey, Jul. 2013, pp. 1542–1549.

[29] A. Swain and D. Clark, “The PHD Filter for Extended Target Trackingwith Estimable Extent Shape Parameters of Varying Size,” in 15th Inter-national Conference on Information Fusion (FUSION 2012), Singapore,Jul. 2012.

[30] F. Septier, S. K. Pang, S. Godsill, and A. Carmi, “Tracking of Coordi-nated Groups using Marginalised MCMC-based Particle Algorithm,” in2009 IEEE Aerospace conference, Big Sky, MT, USA, Mar. 2009.

[31] M. Wieneke and W. Koch, “A PMHT Approach for Extended Objectsand Object Groups,” IEEE Transactions on Aerospace and ElectronicSystems, vol. 48, no. 3, pp. 2349–2370, Jul. 2012.

[32] P. Broßeit, B. Duraisamy, and J. Dickmann, “The Volcanormal Densityfor Radar-based Extended Target Tracking,” in 2017 IEEE 20th Inter-national Conference on Intelligent Transportation Systems (ITSC), Oct2017, pp. 1–6.

[33] K. Granstrom, C. Lundquist, and U. Orguner, “Tracking Rectangular andElliptical Extended Targets Using Laser Measurements,” in Proceedingsof the 14th International Conference on Information Fusion (Fusion2011), Chicago, Illinois, USA, Jul. 2011.

[34] K. Gilholm, S. Godsill, S. Maskell, and D. Salmond, “Poisson Modelsfor Extended Target and Group Tracking,” in SPIE: Signal and DataProcessing of Small Targets, 2005.

[35] K. Gilholm and D. Salmond, “Spatial Distribution Model for TrackingExtended Objects,” IEE Proceedings on Radar, Sonar and Navigation,vol. 152, no. 5, pp. 364–371, Oct. 2005.

[36] J. Lan and X. R. Li, “Nonlinear Estimation by LMMSE-Based Estima-tion With Optimized Uncorrelated Augmentation,” IEEE Transactionson Signal Processing, vol. 63, no. 16, pp. 4270–4283, Aug 2015.

[37] L. Isserlis, “On A Formula For The Product-Moment Coefficient of AnyOrder of a Normal Frequency Distribution in Any Number of Variables,”Biometrika, vol. 12, no. 1-2, pp. 134–139, 1918.

[38] C. R. Givens and R. M. Shortt, “A Class of Wasserstein Metrics forProbability Distributions,” The Michigan Mathematical Journal, vol. 31,no. 2, pp. 231–240, 1984.

[39] S. Yang, M. Baum, and K. Granstrom, “Metrics for Performance Eval-uation of Elliptic Extended Object Tracking Methods,” in Proceedingsof the 2016 IEEE International Conference on Multisensor Fusion andIntegration for Intelligent Systems (MFI 2016), Baden-Baden, Germany,Sep. 2016.

[40] J. S. Fowdur, M. Baum, and F. Heymann, “Tracking Targets withKnown Spatial Extent Using Experimental Marine Radar Data,” in22nd International Conference on Information Fusion (FUSION 2019),Ottawa, Canada, Jul. 2019.

[41] K. Granstrom and U. Orguner, “A PHD Filter for Tracking MultipleExtended Targets Using Random Matrices,” IEEE Transactions onSignal Processing, vol. 60, no. 11, pp. 5657–5671, Nov. 2012.

[42] G. Wick, “The Evaluation of the Collision Matrix,” Physical Review,vol. 80, no. 2, pp. 268–272, 1950.