a probabilistic method for event stream registration · a probabilistic method for event stream...

A probabilistic method forevent stream registration

BACHELOR THESIS

submitted byFlorian Kalber

born on: 23.06.1990

Brachvogelweg 1482140 Olching

Tel.: 0178 5236902

NEUROSCIENTIFIC SYSTEM THEORYTechnische Universitat Munchen

Prof. Dr Jorg Conradt

Supervisor: M.Sc. Lukas EverdingStart: 02.05.2016Intermediate Report: 22.06.2016Final Submission: 15.09.2016

In your final hardback copy, replace this page with the signed exercise sheet.

Abstract

The biologically inspired dynamic vision sensors are a recently developed cameratype which has a fundamentally different working principal compared to conven-tional cameras. They are recording a stream of pixel events, generated by the pixelsindependently when they sense a change in brightness above a certain threshold.Their characteristics allow for a whole array of new applications, but also requirethe development of new processing algorithms. This thesis deals with the possibil-ity of adapting the point set registration algorithm Coherent Point Drift for eventstream data of dynamic vision sensors. The attention of this probabilistic approachshould lie on finding the event stream stereo correspondence and the associated dis-parity values, with the purpose of the extraction of depth and other 3D information.The algorithms performance with event stream data is investigated in several ex-periments. Limitations and prerequisites for the application are also examined andpresented.

CONTENTS 3

Contents

1 Introduction 51.1 Dynamic vision sensors . . . . . . . . . . . . . . . . . . . . . . . . . . 61.2 Related work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6

2 Mathematical background 92.1 Probability density estimation . . . . . . . . . . . . . . . . . . . . . . 92.2 Mixture models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92.3 Maximum likelihood . . . . . . . . . . . . . . . . . . . . . . . . . . . 102.4 Expectation-maximization algorithm . . . . . . . . . . . . . . . . . . 11

3 Coherent Point Drift Algorithm 133.1 Implementation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 163.2 Adapting CPD for event stream data . . . . . . . . . . . . . . . . . . 16

3.2.1 From DVS-stream to point sets . . . . . . . . . . . . . . . . . 173.2.2 Event slices . . . . . . . . . . . . . . . . . . . . . . . . . . . . 173.2.3 Dimensionality . . . . . . . . . . . . . . . . . . . . . . . . . . 183.2.4 Field of view . . . . . . . . . . . . . . . . . . . . . . . . . . . 183.2.5 Horizontal disparity . . . . . . . . . . . . . . . . . . . . . . . . 20

4 Experimental results 234.1 Synthetic data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23

4.1.1 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 244.1.2 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30

4.2 Disparity measurement . . . . . . . . . . . . . . . . . . . . . . . . . . 394.2.1 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 404.2.2 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43

5 Conclusion 47

List of Figures 49

List of Tables 51

Acronyms 53

4 CONTENTS

Bibliography 55

5

Chapter 1

Introduction

The ability of orientation in the surrounding environment is a crucial skill for hu-mans, animals and robots alike. In this context, the perception of depth plays animportant role. Slightly different lateral positions of the eyes of humans and manyanimals lead to slightly different images projected to the eyes retinas. The resultingdisparity of the images can be processed by the visual cortex of the brain in orderto obtain depth information. While this biological process is called stereopsis, incomputer vision this task is referred to as stereo vision. Rigs of two cameras withoverlapping fields of view take images of the same scene from a slightly differentposition. Then, through a process called image registration, the images are trans-formed into a single coordinate system and a correspondence between the imagescan be found. This in turn, enables the computation of disparity and therefore thecomputation of the distance of objects. A wide field of application is then opened.For example, this could enhance the ability of robots for autonomous navigationand orientation by detection of obstacles or the mapping of their surroundings.While this principle is successfully implemented to the images of conventional, frame-based cameras, the perception of depth and stereo vision for the relatively new andbiological inspired Dynamic vision sensor (DVS) is still an active field of research.In a similar way as the photoreceptor cells of a retina, the pixel of a DVS is gen-erating vision information independently, whenever sensing a change in brightness.Changes in the recorded scene are described by a continuous stream of single pixelevents which are the vision information. The working principal and character ofthe data recorded by DVS is completely different when compared to conventionalcameras. This enforces different processing methods than in conventional computervision.The contribution of this thesis is to explore the suitability of a probabilistic regis-tration technique in the DVS domain. Therefore, the Coherent Point Drift (CPD)algorithm [AM09], which is designed for mapping and registration of point clouds,will be adapted for event stream data. Some initial experiments with synthetic dataare followed by experiments of disparity measurements, in order to evaluate the ef-ficacy of the proposed method.

6 CHAPTER 1. INTRODUCTION

The rest of this thesis is structured as follows: After a brief introduction of DVSand a section with related work, chapter 2 gives a description of the mathematicalbasics of the CPD. Chapter 3 explains the coherent point drift algorithm, as wellas prerequisites when adapting CPD for event stream data, followed by an experi-mental part with results and a dedicated discussion in chapter 4. The last chapterends the thesis with some concluding remarks and a brief outlook on possible futurework.

1.1 Dynamic vision sensors

The working principal of a typical DVS, as described by Lichtensteiner et al. [LPD08],is fundamentally different from conventional cameras which record images at a fixedfrequency. All pixels of the DVS operate independently and asynchronously. Whenthe pixel sense a relative change in light intensity above a certain threshold, a pixelevent is generated. These events are in the form of a so-called address-event repre-sentation (AER). They contain the location of the pixel on the sensor, a time stampindicating the time of occurrence, and a parity bit indicating whether the event gen-eration was caused by an increase or decrease in brightness. In this thesis, the DVSresolution is 128 x 128 pixel. Additionally, when using a stereo rig of two DVS, aindication bit for indicating the particular sensor (left or right view) which recordedthe event can be added. All the events form an event stream with a temporal resolu-tion in the range of microseconds. This makes DVS highly applicable for recordingsof high-speed motion, in comparison to frame based cameras which often then sufferfrom motion blur. With image recording frequencies of e.g. 30 or 60Hz, they onlyhave millisecond accuracy. Because of the pixels only generating events with changesin luminosity, a motionless sensor and a static scene leads to a rather empty eventstream with no data transmission (except for noise events). A further advantage,when compared to common cameras where new images are recorded continuously,regardless whether the scene has changed or not. The consequence is redundant andtherefore unneeded data. Usually events, which are generated through motion inthe scene or movement of the sensor, are located at the edges of salient features ofobjects. The DVS pixels do not have the fixed exposure time of frame base cameras,hence dynamic range (e.g. 120 dB) increases and thus the ability to handle unsteadylight conditions. Typically, lower power consumption makes for another advantageof the DVS.

1.2 Related work

Image registration algorithms have the goal to match and to find the correspondenceof images taken from the same scene, but from different point of views . They can beclassified into feature-base and intensity-based algorithms. Intensity-base algorithmsare searching for intensity of gradient patterns, while feature-based algorithms are

1.2. RELATED WORK 7

comparing image features such as lines, edges and points [Gos05]. Many point setregistration algorithms, like CPD, are also suitable for the task of image registration.

Point set registration

By far the most common algorithm for registration of point sets is the iterativeclosest point algorithm (ICP) introduced by Besl and McKay [BM92] and Zhang[Zha94]. One point set is fixed, the other point set is transformed to match the fixedset. ICP assumes that every point in the first point set corresponds with the closespoint in the second point set. The algorithm proceeds by finding the least-squaresrigid transformation. S. Rusinkiewicz et al. [RL01] presented an example for themany variants of ICP which are affecting every aspect of the original algorithm likethe minimization strategy. Other variants deal with the limitations (e.g. initialposition of point sets need to be as close as possible) with probabilistic approachessuch as Luo and Hancock [LH01, LH03] which grasp the matching problem as amaximum likelihood estimation problem which can be solved via the EM-algorithmand the use of mixture models. The rotation matrix is found through singular valuedecomposition. Gold et al. [GLR+98] introduced the robust point matching (RPM),which uses techniques like deterministic annealing and soft assignments of the cor-respondences between points. RPM is the basis for Chuia and Rangarajanb [CR03]which can perform non-rigid registration by a parametrization of the transformationas a thin plate spline. Myronenkos and Songs [AM09] coherent point drift algorithmis similar to [LH01, LH03], but provides a closed form solution to the maximizationstep (M-step) of the EM algorithm for a general multidimensional case for rigid andaffine point set registration. A solution for smooth non-rigid registration is alsopresented.

Stereo Vision for DVS

Despite DVS being a relatively new technology, there are a few approaches existentfor event stream registration and solving the stereo vision correspondences problem.Schraml et al. [SSM07] proposed a synchronous approach to stereo event matchingwhere the events of a certain time slot are collected for a frame-like image repre-sentation. Inside a window of fixed size around the pixel of interest, the algorithmsearches for the best matching candidate in the image recorded by the other sen-sor. The choice for the corresponding pixel is made by finding the minimum ofthe Normalized Sum of Absolute Differences. This algorithm is already used forDVS applications like a real-time tracking algorithm presented by Schraml et al.[SBMS10].Kogler at al. [KHS11] introduced and compared area-based matching, event-image-based matching, and time-based matching for stereo vision with DVS. The first andsecond approach are again collecting events for an image representation. The first

8 CHAPTER 1. INTRODUCTION

area based matching technique uses a form of conventional stereo matching. In thesecond approach the event stream is transformed into an image with a three statelogic (on, off or no event). Vectors, with the neighborhood of a pixel in the tristate logic representation, are created. After that a function is used to compare thevectors between left and right image and to calculate a matching cost to find thecorresponding event pairs. The last approach uses the time between events for thematching cost calculation and therefore renounces the frame representation. Againa neighborhood (with the dimensions space and time) of a pixel is taken into con-sideration in order to find the match. Polarity correlations of the events are alsotaken into account.The work from Rogister et al. [RBI+12] is based on a similar idea. Correlatedevents are likely to have the shortest distance in space and time to the epipolar line.Small variations in timing are tolerated. Additionally, they showed that furtherconstraints, making use of the polarity and ordering of the events, can enhance thematching quality.Asynchronous approaches for stereo vison matching are mainly based on the cooper-ative network of Marr. In order to separate true machtes from all possible matches,Marr and Poggio [MP76] defined two constraints. Uniqueness which says there isonly one disparity value for a certain feature and smoothness which suggests thatdue to the coherence of matter, object surfaces are usually smooth and should resultin smooth varying disparities.Firouzi and Conrad [FC16] extended Marrs static cooperative network by taking thetemporal aspect of the stereo events into account. The inputs of the network arethe location of an event and the detection time. According to the network’s internalstate, which is arranged by previous events, the disparity is extracted through acooperative mechanism. One further approach, also based on the cooperative net-work, is presented by Piatkowska et al. [PBG13]. A completely different approachwas chosen by Weikersdorfer et al. [WACC14] where the required depth informa-tion is obtained by adding a dedicated depth sensor to a DVS. The depth sensoris frame-based, therefore the approach is a compromise between frame-based andevent-based.

9

Chapter 2

Mathematical background

This thesis will introduce the Coherent Point Drift algorithm for DVS data. CPDis considered as a probability density estimation problem. It is based on the EM-Algorithm and Gaussian mixture model (GMM). Therefore, the next sections willgive a brief overview over these basics. This chapter is based on Bishop [Bis95].

2.1 Probability density estimation

The concept of probability density estimation is to find the unknown probabilitydensity function p(x) of a given, observed set of data. The data is containing afinite number of data points xn, n = 1, ..., N and it is assumed that each data pointis drawn as a random sample from this unknown density function. There are threecommonly known approaches for density estimation, each of it with its own benefitsand drawbacks. The parametric approach estimates the density function out ofa known parametric family of basic distributions. The parameters of the chosenmodel are then optimized in such way that the data fits to the model. The secondtechnique is non-parametric estimation where the data determines the form of thedensity function entirely on its own. In many cases parametric estimation mightbe a very imprecise representation of the true density. In comparison to that, non-parametric estimation can deliver a much preciser model but the number of modelparameters increases with growing data complexity. The third approach, sometimescalled semiparametric, tries to combine the advantages of the other two approachesby utilizing the concept of mixture models. These are the model flexibility of non-parametric estimation and the finite amount of model parameters of the parametricapproach.

2.2 Mixture models

For a more flexible representation of the data, a density function can be consideredas a linear combination of several basic functions. In other words, the model is

10 CHAPTER 2. MATHEMATICAL BACKGROUND

a weighted sum of several of these component functions, each with its own densityfunction p(x|m) where the membership of the datapoints is hidden. Mixture Modelscome in handy especially when the statistical model of the observed data includes la-tent variables. For example, this could be an observable structure of different classeswhere the assignment of the data points to corresponding classes is hidden. Theseclasses could be representing some underlying, hidden random variables. Therefore,from a statistics point of view, a mixture model is the representation of a mixturedistribution in the form:

p(x) =M∑m=1

p(x|m)P (m) (2.1)

P (m) is representing the weight of component m of the mixture model. It is alsothe prior probability of a data point having been generated from component m.In this thesis only mixture models based on Gaussian distributions are used. Ina one-dimensional case, the variance σ2 describes how far data points spread outfrom the mean µ of the distribution. For a multi-dimensional case, this character-istic is described by the covariance matrix Σ. Assuming that the gaussians have acovariance matrix which is a scalar σ2 multiplied with the identity matrix I as in

Σm = σ2mI (2.2)

the multi-dimensional component density functions of the mixture model are definedas follows:

p(x|m) =1

(2πσ2m)

d2

exp

{− ‖x− µm‖

2

2σ2m

}(2.3)

2.3 Maximum likelihood

One possible way to find values for the parameters of the density function is theMaximum Likelihood method. It seeks to find the optimum values for the parametersby maximizing a likelihood function derived from the training data. Intuitively italigns the resulting distribution to the observed data by optimizing its parameters insuch a way that it maximizes the likelihood of the data set. Assuming that the dataset consists of N vectors X ≡ {x1, ..., xN} and a density function p(x) dependingon a set of parameters θ = (θ1, ..., θJ)T . Furthermore assume that the data vectorsare generated independently from this distribution p(x), then the joint probabilitydensity estimation of the whole data set is given by

p(X|θ) =N∏n=1

p(xn|θ) ≡ L(θ) (2.4)

where L(θ) is the likelihood function of θ for the data set X. In the next step, L(θ) ismaximized to determine the values of θ. In many cases it is more convenient to findthe minimum of the negative logarithm of the likelihood rather than to maximize

2.4. EXPECTATION-MAXIMIZATION ALGORITHM 11

the likelihood function itself. For example, bringing some numerical advantages.Multiplications transform into sums and small magnitudes of the likelihood becomelarger negative values which can be handled much better by machines with finiteprecision.

E = − lnL(θ) = −N∑n=1

ln p(xn|θ) (2.5)

The negative logarithm is a monotonically decreasing function, hence an equivalentto maximizing the Likelihood function. The negative logarithm of a function reachesits minimum at the same points as the function reaches its maximum. The resultingfunction E of the negative log-likelihood can be seen as a error function. Thismethod can easily be adapted for mixture models so equation 2.5 takes the form(with the assumption of independence of observations)

E = − lnL(θ) = −N∑n=1

ln

{M∑m=1

p(xn|j)P (m)

}(2.6)

However, the minimization of this function leads to a non-linear optimization prob-lem without an analytic solution. An iterative scheme like the Expectation Maxi-mization algorithm can solve this problem.

2.4 Expectation-maximization algorithm

The Expectation Maximization algorithm (EM), developed by Dempster et al. [APD77],can evaluate maximum likelihood estimates from incomplete data. There are sev-eral application possibilities, for example finding missing values in a data set. Thischapter is only dealing with the case of finding parameters for maximum likelihoodestimates of mixture models. This is also a case of incomplete data because ofthe missing class or component labels, addressed in the section ”Mixture models”.Since these class labels of the data are missing, it is not known which componentof the mixture model generated which data point. If one had that information, themaximum likelihood problem could be solved analytically for each single gaussian.When using a mixture model representation for unlabeled data, EM can handle thiscase of incomplete data. The algorithm starts with guessing some initial parametervalues, then it iterates between the expectation (E) step and the maximization (M)step. The E step uses the ”old” parameters of the mixture model to compute theexpectation of the complete data log-likelihood. Since the component which gener-ated which data point is stil unknown, the posterior probabilities P (m|xn) of thegaussians with given data point can be utilized. The posterior probability can becomputed, making use of Bayes’ theorem in the way

P (m|x) =p(x|m)P (m)

p(x)(2.7)

12 CHAPTER 2. MATHEMATICAL BACKGROUND

where p(x|m) is given by 2.3 and p(x) given by 2.1. This probability can be seen ashow much credit a gaussian takes for generating the data point or as the correspon-dence probability between data point and gaussian component. The M step thenfinds the new parameter values by maximizing the likelihood computed in the E step,in this case minimizing the the expectation of the complete negative log-likelihood:

E − step : Q(θ|θi) = E[− logL(θ|θi)]M − step : θi+1 = arg max

θQ(θ|θi) (2.8)

The variable i is the current iteration. Bishop [Bis95] shows that the function of theexpectation of the complete negative log-likelihood Q is defined as follows:

Q = −N∑n=1

M+1∑m=1

P i(m|xn) log(P i+1(m)pi+1(xn|m)) (2.9)

This function is also an upper bound of the error function 2.6. EM convergesby alternating between the E- and the M-step. EM guarantees convergence, butconverging into local minima is possible.

13

Chapter 3

Coherent Point Drift Algorithm

Coherent Point Drift, published by Andri Mironenko and Xubo Song in 2007 [AM09],is a point set registration algorithm. The two main goals are to discover correspon-dences between the two point sets and to recover the transformation that maps onepoint set to the other one. As shown by the authors, CPD can fulfill the desirableproperties of practical point set registration. These goals are to recover the transfor-mation for point set alignment accurately and with sufficient computational costs,to handle point sets with high dimensionality, and finally robustness to any form ofnoise.The authors introduce algorithms for rigid, affine and non-rigid mapping transfor-mations. In this thesis, CPD is used to map the recordings of two synchronizedDVS which observe the same scene at the same time. Hence deformations betweenthe particular recordings are impossible and non-rigid transformation would be un-physical. In fact, camera recordings are perspective projections of a 3-dimensionalscene to a 2-dimensional image representation. This would suggest the use of pro-jective transformation which does not preserve length, parallelism, and angle butcollinearity. However, CPD in its native version is not capable of dealing withhomogeneous coordinates which are necessary for projective transformations. Forthat reason, the algorithm for affine point set registration is used and covered inthis section. Affine transformation is a special case of projective transformation.The properties are identical with the exception that parallelism is preserved. Affinetransformations extend the elementary transformations of the rigid case by skewsand non-uniform(anisotropic) scalings. The elementary transformations of the rigidcase are translation, rotation, and uniform scaling.The alignment of two point sets is considered a probability density estimation prob-lem. Therefore, one point set represents the data points and the other one thecentroids (means) of a GMM. The algorithm moves the GMM centroids coher-ently as a group until it fits to the data in the best possible way and the likelihoodis maximized. To fulfill the coherence constraint and to preserve the topologicalstructure of the point set, the GMM centroids are moved by applying the mappingtransformation to all of them.

14 CHAPTER 3. COHERENT POINT DRIFT ALGORITHM

Mathematical methodology: The first point set XN×D = (x1, ..., xN)T is rep-resenting the data points. The second point set YM×D = (y1, ..., yM)T are the GMMcentroids. N,M are the number of points in the point sets and D the dimension ofthem. N and M do not have to be equal. The transformation T (Y, θ) is appliedto Y, with θ being a set of transformation parameters, to move the GMM centroidscoherently. The basis of the GMM densitiy function are equations 2.1 and 2.3. Thefunction is extended by one additional uniform distribution p(x|M + 1) = 1

N. The

reason for this is to deal with noiseand outliers and to smaller its impact in thedataset. All of the variances σ2 are equal and satisfy equation 2.2. The varianceis reparametrized during each iteration but stays the same for all GMM compo-nents. The prior probabilities are also equal for all GMM components and fixedas P (m) = 1

M. Except for the uniform distribution, which has its own weight

w, 0 ≤ w ≤ 1. The mixture model of CPD is defined as

p(x) = w1

N+ (1− w)

M∑m=1

1

Mp(x|m) (3.1)

Instead of maximizing the likelihood, again the negativ log-likelihood gets minimizedby reparametrizion of the parameter set θ. So that the error function changes fromequation 2.6 to

E(θ, σ2) = −N∑n=1

logM+1∑m=1

P (m)p(x|m) (3.2)

Then EM ([APD77, Bis95]) is used to find the parameters θ for relocalization ofthe centroids and σ2. During the E-step, the posterior probability distributionsP i(m|xn) of all of the mixture components are computed, making use of the ”old”parameter values. Bayes’ theorem(2.7) and 3.1 result in the following equation:

P i(m|xn) =exp− 1

2

∥∥xn−T (ym,θ)

σi

∥∥2

∑Mk=1 exp− 1

2

∥∥xn−T (ym,θ)

σi

∥∥2

+(2πσ2)D/2 w1−w

MN

(3.3)

In the M-step, the expectation of the complete negative log-likelihood, which isshown in 2.9, gets minimized with respect to the ”new” parameter values. Thisfunction is called the objective function and can be rewritten to

Q(θ, σ2) =1

2σ2

M,N∑m,n=1

P i(m|xn)‖xn − T (ym, θ)‖2 +NPD

2log σ2 (3.4)

by ignoring all the constants independent of θ and σ2. NP is defined as NP =∑Nn=1

∑Mm=1 P

old(m|xn) ≤ N (with N = NP only if w = 0). Since the Q function isan upper bound of the error function E(3.2), its minimization leads necessarily toa decrease of E. The convergence behavior is identical to EM.

15

Affine point set registration: Affine transformation is defined through an affinetransformation matrix BD×D and a translation vector tD×1 which leads to the trans-formation T (ym, θ) = Bym + t. The closed form solution is obtained by taking thepartial derivatives of Q and equate them to zero. The solving of the resulting linearsystem of equations leads to the affine point set registration algorithm, shown inFig.3.1. The mapping of the aligned point set is defined as

T (Y) = YBT + 1tT (3.5)

where 1 is a column vector of all ones.

Figure 3.1: Affine point set registration algorithm taken from [AM09]. d(a) is adiagonal matrix formed from the vector a.

Computational complexity: The default algorithm takes O(MN) operations.However, the authors show that the computational complexitiy can be reduced toO(M + N) operations and to linear up to a multiplicative constant by using fastGauss transform (FGT), introduced by Greengard and Strain [GS91]. FGT is usedfor fast computation of the sum of exponentials but shall not be described in anyfurther detail at this point. A runtime test, with the native approach and also FGT,can be seen in section ”Synthetic data”.


3.1 Implementation

For the experimental part of this thesis, the algorithm was implemented in a Matlabtoolbox, provided by A. Myronenko [Myr]. The toolbox contains the Matlab FGTimplementation by Sebastien Paris [Par]. Note that the matrix-vector products wereimplemented in the programming language C as Matlab mex functions to avoid thestorage of P directly in Matlab. Unless otherwise stated, the CPD algorithm runswith the native implementation and not FGT. The hardware, used for all of theexperiments and tests , is an Intel R© Core

TMi5-5200U Processor, 2x2.20GHz and

with 8GB RAM. Before the point set registration, both point sets are always pre-normalized to zero mean and unit variance. CPD hast two possible exit conditions.The algorithm terminates either when reaching an adjustable number of maximumiterations or when satisfying the adjustable tolerance criterion ∆L which is definedas

∆L =

∣∣∣∣L− LoldL

∣∣∣∣ (3.6)

with L being the score of the error function 3.2. This means when the decreaseof the negative log-likelihood function in the current iteration falls below ∆L, thealgorithm terminates. In the experimental part, the algorithm always exits becauseof the tolerance criterion, which is denoted for every experiment separately. Theauthors of CPD are recommending values in the magnitude of w = 0.5 for theweight of the uniform distribution. Initial experiments have shown that rather highvalues for w are bringing the best results, at least in the presence of noise andoutliers.

3.2 Adapting CPD for event stream data

When adapting CPD for DVS data, the first step is to assign recorded events ofthe particular cameras to the two point sets which get registered by the algorithm.Image registration of frame-based cameras has one main objective to assign point-to-point correspondences between the images. However, the nature of DVS data makesit difficult to recover direct point-to-point correspondences between the recordingsof the particular cameras. In fact, a group of events, recorded by one camera,can be seen as a distorted (in space and time) and noisy version of a group ofevents, recorded by the other camera in the same range of time. This is caused byjitter. Moreover, there is a possibility that the sensors capture different numbers ofevents. Therefore the abilities of recovering the transformation of alignment and therobustness to noise is of special importance when using the algorithm with DVS data.The point-to-point correspondences are ignored in this report because the focus lieson the single correspondence of the whole point set. This correspondence can beobtained by recovering the mapping transformation. In the following subsections,some prerequisites are stated out in order to run CPD algorithm with event streamdata.

3.2. ADAPTING CPD FOR EVENT STREAM DATA 17

3.2.1 From DVS-stream to point sets

A event, recorded by a DVS, contains the following information:

• The time when the event was recorded. The unit is µs.

• The x-coordinate of the event in pixel-coordinates

• The y-coordinate of the event in pixel-coordinates

• The polarity of the event

• The camera-ID of the event

A small excerpt of a DVS recording can be seen in table 3.1. The DVS recordingswere imported to Matlab as .tsv files. Then the events with camera-ID ”0” canbe used as the points of point set X (data points) and the events with camera-ID”1” can be used as point set Y (the GMM centroids). After this allocation, the lasttwo columns of the event data is deleted. The polarity of events is ignored for thisthesis, it means in effect that the point sets contain events with both polarities.

Table 3.1: Excerpt from a DVS recording

time/µs x-value/pixel y-value/pixel polarity camera-ID

48653949 106 75 1 0

48654150 27 73 0 1

48654563 106 76 1 0...

......

......

3.2.2 Event slices

Changes in the recorded scene are causing the DVS to generate events continuousover time. To run the point set registration, all the events from a certain time rangeshould be used as input for the algorithm. In other words, the eventstream has to beportioned in slices of time. Ideally, the time range of the slices should be infinitelysmall or as small as possible. A larger time range results in a worse time resolution.Changes and movements in the recorded scene during a single time-slice can resultin a continuous changing disparity value, but the algorithm can only provide onedisparity value for the Slice. However, the limiting parameter of this concept is thenumber of Events inside these time-slices. If the the number of events is getting tosmall, the algorithm probably computes inaccurate or even wrong results. In thiscase, the time slice has to be extended until it contains the minimum number ofevents. When the number is getting larger, a longer computation time is expected.Computation time is addressed in chapter ”Synthethic data”. Therefore, in the


following experiments often slices with a certain amount of events are used. Thismeans, the number of events is always the same whereas the time range can vary.In contrast to the time-slices where the time range is constant but the amount ofevents can vary. Most of the time the point sets have different sizes due to jitterof the DVS output data. Therefore, a continuous sequence of event slices can bedefined by the number of events from both point sets combined, instead of usingthe event numbers of the particular event streams. Slices containing equal amountof events in both point sets would result into more or less different time ranges.

3.2.3 Dimensionality

Intuitively, the 3-dimensional DVS data suggests to register the point sets also in3-D. Tests have shown that DVS event-data is afflicted with jitter naturally. Thisleads to transformations into the time-dimension when aligning both point sets.However, these transformations do not make any sense from a physically point ofview and are therefore unwanted. They falsify the results and can not be extractedeasily.In Fig. 3.2 an example of 3-dimensional point set registration with CPD algorithmand DVS event stream data as input is shown. The images in the top row show thebefore-point-set-registration-situation and the after-point-set-registration-situationwith view of the x-y-plane. The images in the bottom row show the view of thetime-y-plane. The number of events in both point sets combined is 500, the weightof the uniform distribution is 0.7 and the tolerance criterion is set to ∆L = 1e− 6.The time-value was normalized to values between 0 and 127. The view of the x-y-plane shows that the point sets are getting aligned quite well, but the t-y-planeshows some distortions into the time dimension after the alignment (e.g. blue circleson the left of the 0-value of the time axis). The resulting transformation, (cf. 3.7)can confirm this, especially the significant high value tt of the translation vectort = (tt, tx, ty)

T .

T (Y) = Y

1.0140 0.0667 0.05770.0077 0.8824 −0.0830−0.0012 −0.0086 0.8935

T

+ 1

−6.404821.30798.7078

T

(3.7)

To solve this problem, the event data is mapped from 3-D space to 2-D space beforerunning the point set registration which is then executed in 2D. The mapping isperformed by simply ignoring the time-values.

3.2.4 Field of view

The distance between both DVS implies that both cameras do not have completelyoverlapping field of views and therefore do not necessarily register the same move-ments and respectively events. For example, objects which enter the left edge of thefield of view of the left camera may not be registered at all by the right camera.


20 40 60 80 100 120

x/pixel

20

40

60

80

100

120

y/pi

xel

Before (x-y-plane)

Data pointsGMM Centroids

20 40 60 80 100 120

x-value

20

40

60

80

100

120

y/pi

xel

After (x-y-plane)

0 20 40 60 80 100 120

time

20

40

60

80

100

120

y/pi

xel

Before (t-y-plane)

0 20 40 60 80 100 120

time

20

40

60

80

100

120

y/pi

xel

After (t-y-plane)

Figure 3.2: 3D point set registration of recorded event stream of a hand. Y(bluecircles) is aligned onto X(red stars). Before and after registration situations areshown in the x-y and in the t-y plane. Distortions into the time dimension arevisible in the bottom right plot.

In terms of point set registration, these events which are only registered by one ofthe two cameras can be regarded as noise and worsen the results. A preprocessingstep of deleting all events which are not inside the overlapping field of view couldsolve this problem. Fig. 3.3 gives an example of the difference between runningthe algorithm with preprocessed data and without preprocessed data. The numberof events in both point sets is 1000, the weight of the uniform distribution is 0.7and the tolerance stopping criterion is set to ∆L = 1e− 6. The top row shows thebefore and after situation without preprocessing. The algorithm can not find analignment between the point sets because of the noise points on the left as part ofthe data points. The bottom row shows the same, but with preprocessed data. Thealgorithm finds the alignment.


-20 0 20 40 60

x/pixel

40

60

80

100

120

140

y/pi

xel

After (without preprocessing)

0 10 20 30 40 50 60 70

x/pixel

40

60

80

100

y/pi

xel

Before (without preprocessing)


0 10 20 30 40 50 60 70

x/pixel

40

60

80

100

y/pi

xel

Before (with preprocessing)

0 10 20 30 40 50 60 70

x/pixel

40

60

80

100

y-va

lue

After (with preprocessing)

Figure 3.3: 2D point set registration of recorded event stream of a hand. Y(bluecircles) is aligned onto X(red stars). Before and after registration situations areshown for differing field of views and identical field of views respectively. Registrationsucceeds for preprocessed data with identical field of vies, but fails with native data.

3.2.5 Horizontal disparity

The lateral difference of the two stereo sensors results in an horizontal differenceof the same objects in the recordings of these cameras. Based on this horizontaldisparity, further 3D-information e.g. depth information can be obtained.

Image rectification: The following paragraph is adapted from A. Fusiello et al.[FTV00]. In the left sketch of Fig.3.4, the optical centers C1 and C2 and their be-longing image planes R1 and R2 are shown. The projection of a point W onto theimage planes results into the points M1 and M2. All of the possible 3D-points onthe optical ray of M1 lead to the projection of M1 on R1, but when projected ontoR2 through C2, they lie on a so-called epipolar line between M2 and E2. All ofthese epipolar lines run through the points E2 and E1 which are so-called epipoles.These epipoles are defined as the projection of the optical center of the other camerarespectively. When moving the image planes until they are co-planar, the epipoles


are at infinity. That means that all of the epipolar lines are parallel to each otherand also horizontal. This process is called image rectification. On the right side ofFig.3.4, two rectified cameras with co-planar image planes can be seen. All of theepipolar lines and the base line are parallel to the horizontal axis and therefore thepoints M1 and M1 have the same vertical coordinate.

Figure 3.4: On the left: The epipole(E1, E2) of one camera is the projection of theoptical center(C1, C2) of the other camera. On the right: rectified cameras havecoplanar image planes which are parallel to the baseline.

Assuming that the DVS stereo cameras are calibrated and rectified and the eventsof both cameras share the same coordinate system, the horizontal disparity is thedistortion between both point sets in direction of the x-axis. CPD aligns the pointsets and recovers a mapping transformation in the form of equation 3.5. The pointsets are in camera coordinates and transformations are always with respect to theorigin. Therefore, multiplying a point set with the transformation matrix results inan additional translation. Provided that the point set is not centered in the origin.This is illustrated in the sketch in Fig. 3.5 where two identical objects are multipliedwith the same shear (shear factor 1) in y-direction. The blue square has its mean inthe origin, before and after the transformation respectively. The red square is shiftedaway from its original position by the transformation. This shift can be computedas the difference between transformation matrix, multiplied with the mean vector,and the mean vector. Finally, this translation and the translation vector t of theresulting mapping transformation can be added up to

d = t + BµY − µY (3.8)


-1 1 2 3 4 5

x

-2

-1

1

2

3

4

5

6y

Transformation example

Bµ

µ

Bµ - µ

Figure 3.5: Transformations are always with respect to the origin. Transformationsof points not located in the origin are leading to additional translations. As aexample, two squares are transformed by multiplication with the matrix B.

where dD × 1 is the disparity vector and µY the mean vector of Y. The x-elementis the horizontal disparity of the camera recordings and the y-element the verticaldisparity which should be zero, given perfect calibrated cameras and ideal point setregistration.

23

Chapter 4

Experimental results

In this section, the CPD point set registration with DVS data is tested and eval-uated. In order to have the ability to monitor and evaluate the results of pointset registration with CPD qualitatively, the first series of experiments is executedwith synthetic data. In this case synthetic data means that both point sets, thedata and the GMM centroids, are identical until one point set gets distorted by anknown distortion transformation. Then it can be tested whether CPD can registerthe point sets and recover the known distortion transformation. Experiments whichare investigating the runtime of the algorithm, as well as the robustness to outliersand noise, are also performed with synthetic data.In the second experimental part, which is all about the measurement of the dispar-ity, the whole DVS recording with events from both cameras are used.The DVS recordings which are used for the experiments in the following sectionsare noise filtered through pre-processing of the data.

4.1 Synthetic data

The transformation for distortion is T (X) = XBXT + 1tX

T . The advantage for ap-plying the distortions to point set X is, that is allows direct comparison of the resultsafter point set registration. X gets distorted away from Y and the algorithm tries tomove Y behind until re-alignment. In practice, one or more elementary transforma-tions were multiplied into the distortion matrix Bd. According to equation 3.5, pointset Y can be matched with the distorted point set X after the registration process.In the synthetic data experiments, the point sets are additional pre-normalized tozero mean before applying the distortion. This makes sure that the point gets trans-formed with respect to the centroid of the point set. The transformations chosenfor the distortions are mainly based on shear transformations. The expected changewhen recording an object from two positions, differing by a slight spatial shift, canbe replicated best by shear transformations.

24 CHAPTER 4. EXPERIMENTAL RESULTS

4.1.1 Results

Two examples of event slices are tested. The Events of ”Camera 1” are ignored,both point sets contain the events of ”Camera 0”. Point set X, the data points, arethen distorted by an know transformation of coordinates. It is tested whether CPDcan register the point set and recover the distortion transformation. Each exampleis tested with 7 different distortions which are identical for both examples. Theweight of the uniform distribution is set w = 0.5, the tolerance stopping criterion isset to ∆L = 1e− 6 for both examples.

Example 1 ”hand”

The event-slice used for the first example is extracted out of the DVS recordinghandWithWatch denoised.tsv. It contains the first 250 events of Camera ”0”,counted from overall event number 2000 of the recording. Both point sets are con-taining these 250 events, that is why the overall amount is 500. In test (a), the pointset of data points X is distorted by a translation of tX = (tx, ty)

T = (−25,−5)T .The result for the first example can be seen in table 4.1 in addition with the numberof iterations and time the algorithm needed for the registration. CPD recovers tYsuccessfully and it is identical to tX . For all of the following tests in this section,this translation is applied.

Table 4.1: Synthetic data example ”hand”

Distortion Result

AX tX AY tY Iter. time in s

(a) -

(−25

−5

) (1 0.000

0 1

) (−25

−5

)24 0.37s

(b)

(0.965 0.258

−0.258 0.965

) (−25

−5

) (0.965 0.258

−0.258 0.965

) (−25

−5

)24 0.35s

(c)

(1 1

0 1

) (−25

−5

) (1 1

0.000 1

) (−25

−5

)27 0.86s

(d)

(1 2

0 1

) (−25

−5

) (1 2

0.000 1

) (−25

−5

)288 6.39s

(e)

(1 3

0 1

) (−25

−5

) (1.668 2.079

0.514 0.842

) (−33.120

−6.574

)162 3.48s

(f)

(0.7 2

−0.3 1

) (−25

−5

) (0.564 0.504

0.444 0.997

) (−28.757

−5.257

)84 4.55s

(g)

(1.4 0.8

−0.6 0.8

) (−25

−5

) (1.4 0.8

−0.6 0.8

) (−25

−5

)29 1.01s

4.1. SYNTHETIC DATA 25

In test (b), the data points are additionally distorted by an rotation of -15 degreesaround the mean of the point set before the translation is applied. In Fig. 4.8, avisualization of this test and the following two tests is shown. CPD aligns both pointsets and the affine transformation matrix Bd, as well as the translation vector, arerecovered completely. In the tests (c), (d), and (e) the distortion transformationsare shears parallel to the x axis. In (c), the shear factor is 1 and the alignment isfound. In (d), the alignment is also found for shear factor 2. However, the numberof iterations is more then 10 times higher then in (c). A visualization of the nextthree tests can be seen in Fig. 4.9. With the shear factor being 3, and thereforeeven higher, CPD fails to find the alignment as can be seen in test (e). In test (f),in addition to a shear in x-direction with shear factor 1, a shear in y-direction withshear factor −0.3 is applied. CPD again fails to find the alignment. In test (g), theregistration is successful. In this test, the distortion of (f) was multiplied with annon-uniform scaling of 2 in x-direction and 0.8 in y-direction.

Example 2 ”pen”

The event-slice used for the second example is extracted out of the DVS recordingJitteringPen New denoised.tsv. It contains the first 250 Events of ”Camera 0”,counted from overall event number 1500 of the recording.

Table 4.2: Synthetic data example ”pen”

Distortion Result

AX tX AY tY Iter. time in s

(h) -

(−25

−5

) (1 0.000

0.000 1

) (−25

−5

)29 0.97s

(i)

(0.965 0.258

−0.258 0.965

) (−25

−5

) (0.965 0.258

−0.258 0.965

) (−25

−5

)31 1.00s

(k)

(1 1

0 1

) (−25

−5

) (1 1

0.000 1

) (−25

−5

)23 0.84s

(l)

(1 2

0 1

) (−25

−5

) (1.136 1.429

0.621 0.706

) (−32.768

−8.357

)53 1.42s

(m)

(1 3

0 1

) (−25

−5

) (1.802 2.118

0.632 0.701

) (−38.484

−9.158

)102 2.31s

(n)

(0.7 2

−0.3 1

) (−25

−5

) (0.7 1

−0.3 1

) (−28, 757

−5, 257

)123 2.99s

(o)

(1.4 0.8

−0.6 0.8

) (−25

−5

) (1.4 0.8

−0.6 0.8

) (−25

−5

)39 1.14s


The results can be seen in table 4.2. The distortions used for this example are thesame as the ones in the first example. Again, the first test (h), where the distortionscontains only the translation, is successful. The rotation test (i) is visualized in Fig.4.10, together with the tests (k) and (l). In (i) and (k), point set registration issuccessful, in (l) is the shear factor of 2 enough to lead to failure of the algorithm.Same goes for shear factor 3 in test (m) which can be seen in Fig. 4.11. In comparisonto the first example, the distortion of a combined x- and y-shear can be recoveredin this example, demonstrated in (n). The addition of the non-uniform scaling alsoleads to success in (o).

Outliers

This experiment shows the performance of the algorithm with respect to outliersin both point sets. Outliers are the noise events which are not referring to anyfeatures of the recorded scene. The eventslice was extracted out of the DVS recordinghandWithWatch denoised.tsv.

-100 -50 0 50

x/pixel

-60

-40

-20

0

20

40

60

y/pi

xel

50 outliers, Initialization

-80 -60 -40 -20 0 20 40

x/pixel

-100

-50

0

50

y/pi

xel

50 outliers, Result( iteration 80)

GMM centroidsData points

-50 0 50

x/pixel

-50

0

50

y/pi

xel

500 outliers, Initialization

-80 -60 -40 -20 0 20 40

x/pixel

-50

0

50

y/pi

xel

500 outliers, Result( iteration 85)

Figure 4.1: Synthetic event stream registration of recording of a hand. Y(bluecircles) is aligned onto X(red stars). Before and after registration situations areshown for 50 added outliers and 500 added outliers. For the latter example, theregistration fails.


It contains the first 500 events of Camera ”0”, counted from the overall event num-ber 2000 of the recording. The point set, containing the data points, is distortedwith a transformation based on shears in both directions and a non-uniform scal-

ing. The transformation matrix is BX =

(0.96 −0.20.72 0.6

), the translation vector is

tX =

(−25−5

)The weight of the uniform distribution is set to w = 0.7, the tolerance

stopping criterion is set to ∆L = 1e − 6. Different numbers of outliers are added.Starting at 50, the number increases up to 500 Events in addition to the 500 Eventsof the denoised DVS recording. The outliers were generated from an uniform distri-bution and added to both point sets independently. Furthermore they were addedin the range 0 to 127 for both axes and before the distortion was applied, meansthat the outliers were also transformed by the distortions.

0 50 100 150 200 250 300 350 400 450 500

Number of outliers

0

0.01

0.02

0.03

0.04

0.05

0.06

0.07

0.08

0.09

0.1

Err

or o

f B

Influence of outliers

Error of B as a function of the amount of outliers

Difference between distorted B and the identity matrix

Figure 4.2: Influence of outliers on the affine transformation matrix B. The error isthe difference of the frobenius norm of the distortion matrix and the frobenius normof the resulting transformation matrix. respectively.

In Fig. 4.1, the first column shows two examples of initial point sets with outliersadded. The second column shows these examples after the point set registrationwith CPD. With increasing number of outliers, the results for independent runs butwith the same initial paramters become different. Therefore, for each level of theoutliers, 10 independent runs were executed. Fig. 4.2 shows the error values of theresulting transformation matrices BY as a function of the amount of outliers added,


according to a similiar experiment of Myronenko et. al [AM09]. The error is theabsolute value of the difference between the Frobenius norms of distortion matrixand the resulting, estimated transformation matrix. Mean and standard deviationare added. Until up to 200 Outliers, that is ' 29% of the overall event number,the estimated transformation matrix is identical to the initial matrix in 10 out of 10runs. Between 250 and 300 Outliers (' 33−38 of overall events), CPD starts to failin finding the alignment of the point sets. From 350 up to 500 Outliers (' 41%−50%of overall events), CPD is not able to register the point sets successful. Noticeableis that after 400 outliers, the standard deviation is still increasing, but not the meanof the error.

Noise

Because of jitter, the two stereo DVS are not recording the exact same events. Thisbehavior was simulated by applying noise to one of the point sets. The event slice forthis experiment was extracted out of the DVS recording handWithWatch denoised.tsv.It contains the first 250 events of Camera ”0”. The distortion matrix, the translationvector, the weight of the uniform distribution and the tolerance stopping criterionwere the same as in the outliers experiment.

-60 -40 -20 0 20 40 60

x/pixel

-30

-20

-10

0

10

20

30

y/pi

xel

Added noise with std 1, Inizialization

-60 -40 -20 0 20 40 60

x/pixel

-30

-20

-10

0

10

20

30

y/pi

xel

Added noise with std 1, Result (iteration 55)

GMM centroidsData points

Figure 4.3: Synthetic event stream registration with recording of a hand. Y(blue cir-cles) is aligned onto X(red stars). X was additionally distorted with noise (standartdeviation(std) of 1). Before and after registration situations are shown.

The noise was applied to the data points X (before the distortion through BX andtX) by randomly relocating them with a normal distribution where the mean wasthe initial location of the data point. The standard deviation was increased from 0up to 2 pixel. 20 independent runs were performed for each level of the standard


deviation. In Fig. 4.3, an example of the registration with added noise is shown.Fig. 4.4 shows the error values of the resulting transformation matrices BY as afunction of the amount of noise added. The error was computed equally as in theoutliers experiment. Mean and standard deviation of the error values are added.With increasing amount of noise, the error increases also.

0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 2

standart deviation of noise distortion (in pixel)

0

0.01

0.02

0.03

0.04

0.05

0.06

0.07

0.08

0.09

0.1

Err

or o

f B

Influence of noise

Error of B as a function of the intensitiy of noise

Figure 4.4: Influence of noise on the affine transformation matrix B. The error isthe difference of the frobenius norm of the distortion matrix and the frobenius normof the resulting transformation matrix. respectively.

Runtime

In this experiment, the registration time of CPD with synthetic DVS data is exam-ined. They contain events of Camera ”0”, counted from the overall event number2000 of the recording handWithWatch denoised.tsv. The point set containing the

data points is distorted with a transformation of B =

(1 −0.752 1.5

). The weight of

the uniform distribution is set to w = 0.3, the tolerance stopping criterion is set to∆L = 1e−6. The runtime is measured for different number of events, starting at 200events in each point set and increasing up to 5000. In addition to the default algo-rithm, the time was also measured for the execution with FGT. The measurementscan be seen in table 4.3 and also in Fig.4.5 which shows the runtime as a function ofthe number of events in one point set. Until 300 events, the computational time for


the native approach is smaller then with FGT. With 1000 Events, FGT is alreadymore then 3-times faster and at 2000 it is 5 times faster then the native algorithm.

Table 4.3: Runtime per number of events

Events default fast Gauss transform

200 0.1574s 0.22157s

300 0.2632s 0.27709s

400 0.3976s 0.31813s

500 0.5956s 0.39993s

600 0.8669s 0.50426s

700 1.1910s 0.56713s

800 1.5780s 0.65836s

900 1.9472s 0.73551s

1000 2.4206s 0.81368s

1500 6.1608s 1.4218s

2000 11.7997s 2.2754s

2500 19.9213s 3.1737s

3000 29.6356s 4.2931s

3500 44.8217s 5.7065s

4000 59.8872s 7.4489s

4500 78.4930s 9.8418s

5000 149.3324s 16.0860s

4.1.2 Discussion

Example 1 ”hand” and Example 1 ”pen”: The normalization of the pointsets to zero mean, means that the point sets are already pre-aligned before the ac-tual algorithm proceeds. As a result, large translations between the point sets areovercome easily by the registration algorithm. With the distortions getting moresignificant, the number of iterations seem to increase, independently of the successof the registration. This can be seen in the tests (d) and (n) in the case of successfulregistration and in tests (e) and (m) in the case of failure.It should be added that all of the experiments where CPD failed to find the align-ment between the point set, were repeated with a much smaller weight w for theuniform distribution. Then the registration was successful (except for example (m))which makes sense since the synthetic data examples contain zero noise, simply be-cause the point sets are identical. Also note that the used shear factors are relativelylarge and may not occur in realistic scenarios. It can be assumed that the weight


200 500 1000 2000 3000 4000 5000

number of events per point set

10-1

100

101

102

103

runt

ime/

sRuntime per events

nativewith fast gauss transform

Figure 4.5: Runtime as a function of number of events in one point set. After 400events, FGT outperforms the native algorithm.

w plays quite a significant role whether the alignment is successful or not. Theintuitive approach would be to choose a bigger w’s for data, containing more noiseand vice versa. However, choosing a larger w could lead to a situation where thealgorithm could treat outlying parts of the objects of heavy distorted point sets asnoise and the registration fails . This may have happened in the failed tests of thesynthetic data experiments (tests (e) and (f)). A more detailed look on the behaviorof the algorithm with respect to the weight w is not a part of this report.

Variance initialization Eye-catching is that in the early iterations of all the tests,the point set with the GMM centroids contracts. During higher iterations, the pointset expands again. In Fig.4.6, the initialization (after the normalization) and somesmall iterations of test (c) were plotted. The plot of the point sets was supplementedby a plot of the probability density function (pdf) of the GMM as contour lines. Theinitial variance sigma2 is large enough that the pdf’s of the gaussian componentsmelt into a pdf of the mixture model, which has approximately the same contours asthe pdf of a single gaussian. Since the algorithm tries to maximize the likelihood ofthe complete model, all of the data points are ”pulling” at all of the GMM centroids


-2 -1 0 1 2

x/variance

-2

-1

0

1

2

y/va

rianc

e

Initialization

-2 -1 0 1 2

x/variance

-2

-1

0

1

2

y/va

rianc

e

Iteration 1

-2 -1 0 1 2

x/variance

-2

-1

0

1

2

y/va

rianc

e

Iteration 5

-2 -1 0 1 2

x/variance

-2

-1

0

1

2

y/va

rianc

e

Iteration 10GMM pdfGMM centroidsData points

Figure 4.6: Synthetic event stream registration of recording of a hand and nativevariance initialization. Y(blue circles) is aligned onto X(red stars). A contour plotof the probability density (pdf) function of the GMM is shown during initializationand some early iterations. Point set Y is contracting during early iterations.

with the effect that the point set contracts to the center of the pdf. I takes someiterations until σ2 gets small enough so that the contours of the pdf of the GMMdefine them self’s more around the means of the single components. This leads fi-nally to the effect that, in order to maximize the models likelihood, particular datapoints only ”pull” at single GMM centroids or at least smaller groups of them. Test(c) was repeated with variance 10 times smaller then initial default, computed asshown in Fig. 3.1. The results are shown in Fig.4.7. The contours probability den-sity function is already much more defined around the outlines of the hand, due tothe smaller variance. The plot of iteration 10 shows that the point sets are closer tothe final alignment, in comparison to the plot of iteration 10 in Fig. 4.6. Eventually,a smart initialization of the variance could be a optimization of the runtime, sincethe number of iterations can be reduced. In the example of test (c), the 10 timessmaller initial σ2 has led to a decrease of iterations from 24 iterations to 16 iterations.


-2 -1 0 1 2

x-value/variance

-2

-1

0

1

2

y/va

rianc

eInitialization

-2 -1 0 1 2

x/variance

-2

-1

0

1

2

y/va

rianc

e

Iteration 1

-2 -1 0 1 2

x/variance

-2

-1

0

1

2

y/va

rianc

e

Iteration 5

-2 -1 0 1 2

x/variance

-2

-1

0

1

2

y/va

rianc

e

Iteration 10GMM pdfGMM centroidsData points

Figure 4.7: Synthetic event stream registration with recording of a hand and varianceinitialization 10 times smaller then default. Y(blue circles) is aligned onto X(reddots). A contour plot of the GMM pdf is shown during initialization situation andsome early iterations. Point set Y is not contracting during early iterations.

Outliers & noise: In the outliers experiment, CPD shows robustness of outliersup to an amount of ' 29% of outliers in the overall point set. The steady increase ofoutliers, did not led to the expected, accordingly increase of the mean of the trans-formation matrix error at some point. A possible explanation could be the errorbetween initial distortion matrix and identity matrix, which can also be seen in Fig.4.2. If the event-slice would contain 100% outliers, the expected random results ofthe transformation matrix would distribute around the identity matrix randomly.The expected error would be the error between initial distortion matrix and identitymatrix. Therefore, an convergence up to the error of the identity matrix , produceyby an increasing amount of outliers, seems plausible.When adding noise to one point set, a resulting matching error is inevitable. How-ever, the error is rather small and increases slightly. The alignment of the point setswas successful up to the maximal value (2) of the standard deviation, except for thesmall errors. Robustness to noise is apparently provided up to a certain degree.


Runtime: For small event numbers, the native implementation seems to be thefastes runtime-wise. For medium event numbers, the FGT implementation should bepreferred. However, the promised linear rise of computational time by Myroneko etal. [AM09] could not be accomplished in this runtime experiment since the measuredcurve shows a bending in Fig. 4.5. The runtime as a function of number of eventsincreases steeper then linear.


-50

050

-20020

y-value

Init

ializ

atio

nIt

erat

ion

5It

erat

ion

10

Iter

atio

n 1

5

-60

-40

-20

020

-20

020

y-value

Res

ult

( it

erat

ion

24)

-50

050

-20020

y-value

Init

ializ

atio

n

-20020

Iter

atio

n 5

Iter

atio

n 1

0It

erat

ion

15

-40

-30

020

40-4

0

-20

020

y-value

Res

ult

( it

erat

ion

27)

-50

050

x-va

lue

-20020

y-value

Init

ializ

atio

n

x-va

lue

Iter

atio

n 5

x-va

lue

Iter

atio

n 5

0

x-va

lue

Iter

atio

n 1

50

-50

050

x-va

lue

-20

020

y-value

Res

ult

( it

erat

ion

288

)

(b)

(c)

(d)

Fig

ure

4.8:

Sev

eral

2Dp

oint

set

regi

stra

tion

exam

ple

sw

ith

synth

etic

even

tst

ream

dat

a”h

and”.

Y(b

lue

circ

les)

isal

igned

onto

X(r

edst

ars)

.T

he

colu

mns

show

the

alig

nm

ent

pro

gres

s.


-100-50

050

-20

-10 0 10y/pixel

Initializatio

nIteratio

n 5

Iteration

40Iteratio

n 100

-100-50

050

-20

-10

0 10

Resu

lt( iteration

162)

-500

50

-20

-10 0 10

y/pixel

Initializatio

nIteratio

n 5

Iteration

30Iteratio

n 60

-500

50

-20

-10

0 10

Resu

lt( iteration

84)

-500

50

x/pixel

-40

-20 0 20

y/pixel

Initializatio

n

x/pixel

Iteration

5

x/pixel

Iteration

10Iteratio

n 15

x/pixel-50

050

x/pixel

-40

-20

0

Resu

lt( iteration

29)

(e)

(f)

(g)

Figu

re4.9:

2Dp

oint

setregistration

exam

ples

with

synth

eticeven

tstream

data

”han

d”.

Y(b

lue

circles)is

aligned

onto

X(red

stars).T

he

colum

ns

show

the

alignm

ent

progress.


-60

-40

-20

020

40

-2002040

y/pixelIn

itia

lizat

ion

Iter

atio

n 5

Iter

atio

n 1

0It

erat

ion

20

-60

-40

-20

020

40

-20

02040

Res

ult

( it

erat

ion

31)

-50

050

-2002040

y/pixel

Init

ializ

atio

nIt

erat

ion

5It

erat

ion

10

Iter

atio

n 1

5

-50

050

-20

02040R

esu

lt(

iter

atio

n 2

3)

-50

050

x/pi

xel

-20020

y/pixel

Init

ializ

atio

n

x/pi

xel

Iter

atio

n 5

x/pi

xel

Iter

atio

n 1

5

x/pi

xel

Iter

atio

n 3

0

-50

050

x/pi

xel

-20

020

Res

ult

( it

erat

ion

53)

(i) (k)

(l) Fig

ure

4.10

:2D

poi

nt

set

regi

stra

tion

exam

ple

sw

ith

synth

etic

even

tst

ream

dat

a”p

en”.

Y(b

lue

circ

les)

isal

igned

onto

X(r

edst

ars)

.T

he

colu

mns

show

the

alig

nm

ent

pro

gres

s.


-500

50100

-20 0 20

y-value

Initializatio

nIteratio

n 5

Iteration

15Iteratio

n 30

-500

50100

-20

0 20 40R

esult( iteratio

n 102)

-40-20

020

40

-20 0 20 40

y-value

Initializatio

nIteratio

n 5

Iteration

15Iteratio

n 100

-40-20

020

40

-20

0 20

Resu

lt( iteration

123)

-100-50

050

x-value

-20 0 20

y-value

Initializatio

n

x-value

Iteration

5

x-value

Iteration

10

x-value

Iteration

20

-500

50100

x-value

-20

0 20

Resu

lt( iteration

39)

(m)

(n)

(o)

Figu

re4.11:

Several

2Dp

oint

setregistration

exam

ples

with

synth

eticeven

tstream

data

”pen

”.Y

(blu

ecircles)

isalign

edon

toX

(redstars).

The

colum

ns

show

the

alignm

ent

progress.

4.2. DISPARITY MEASUREMENT 39

4.2 Disparity measurement

This section deals with the computation of the horizontal disparity with CPD. Forthis experiment, the recording of a dropped pen is used. The cameras were facingnearly in vertical direction towards the floor. The respective event stream can beseen in Fig. 4.12 as a three dimensional figure with the time value as the z-axisand also in Fig. 4.13 as a projection on the x/y-plane. Since the pen falls towardsthe floor, the distance between pen and cameras is steadily increasing. As shownby L. Shapiro et al. [SS01], the coherence between depth (distance between objectand cameras) and disparity is z = fb

d. Where z is the depth, b is the baseline

(distance between the centers of projection of both cameras respectively) and f isthe focal length (distance between center of projection and image plane). Hence,the disparity is decreasing with increasing depth of the object (d ∼ 1

z). Fig. 4.13

shows this relation which can be observed when looking at the edges of the pointsets. The free fall of the pen, a case of a uniformly accelerated motion, is definedas h = gt2

2(h ∼ t2) with g being the gravity, h the height of the fall and t the time.

This relation can be observed in Fig. 4.12.

20

0

40

20

60

40

x/pixel

80

y/pi

xel

60

100

80

200

t/ms

Dropped pen from above

150

120

100 10050

0


Figure 4.12: 3D plot of the recording ”Droppen pen from above”


The baseline and focal length of the cameras, as well as the angle between falldirection of the pen and camera orientation are unknown. Therefore the qualityof the disparity computation is evaluated by fitting the measured values to theexpected family of curves. Assuming z ∼ h and the relations h ∼ t2 for the free falland d ∼ 1

zfor the disparity, the relation between disparity and time is d ∼ 1

t2. The

event stream is divided into a continuous sequence of event slices with an constantnumber of events in both event streams combined. CPD is executed with everyevent slice as input. From the results, one disparity value and one time value iscomputed.

0 10 20 30 40 50 60 70 80 90 100

x/pixel

30

40

50

60

70

80

90

100

110

120

y/pi

xel

Dropped pen from above (x-y-plane)

Data PointsGMM Centroids

Figure 4.13: 2D plot (x-y-plane) of the recording ”Dropped pen from above”

4.2.1 Results

The event-slices used for this experiment are extracted out of the DVS recordingDroppedPenFromAbove denoised.tsv. They contain the events of both cameras,counted from overall event number 1000 of the recording up to 18000. The testedsizes of the event slices were 200, 500 and 1000 events. The number of events in theparticular point sets were generally differed from each other, due to the nature ofDVS data. The weight of the uniform distribution is set to w = 0.7, the tolerancestopping criterion is set to ∆L = 1e−6 for all examples. The variance was initialized


according to Fig. 3.1 and additionally divided by 100. The disparity value wascomputed according to equation 3.8. The time value was computed as the mean ofthe time values of all the events included in the slice. In the first test, the pointsets of 17 slices, each containing 1000 Events, were registered with CPD. Duringthe registrations, it was noticeable that the point set of some slices were not alignedproperly. In all of these cases, the GMM centroids were lining up along the axis ofsymmetry of the pen. A example of this miss alignment can be seen in Fig. 4.14(example 2), alongside with an example of correct alignment (example 1).

0 20 40 60 80

x/pixel

40

60

80

100

120

140

y/pi

xel

Example 1 initialization

40 60 80 100

x/pixel

60

80

100

120

140

y/pi

xel

Example 1 result

0 20 40 60 80 100

x/pixel

70

80

90

100

110

120

130

y/pi

xel

Example 2 initialization

GMM centroids

Data points

40 60 80 100

x/pixel

60

80

100

120

Example 2 result

Figure 4.14: Examples of event stream registration of a falling pen. Y(blue circles)is aligned onto X(red dots). Before and after registration situations are shown. Inexample 1, the alignment is found. In example 2, the GMM centroids are lining upalong the axis of symmetry of the pen.

However, since these miss alignments are located in the center along the symmetryaxis, it does not necessarily mean that these miss alignments falsify the resultingdisparity values in a significant way. In the test with 1000 Events, 3 out of 17 sliceswere not aligned properly. The average time range per slice was 10.86ms. From theresults, the disparity values were computed and are shown in Fig. 4.15 as a functionof the time.


40 60 80 100 120 140 160 180 200

t/ms

30

35

40

45

50

55

60

65

disp

arity

in p

ixel

Disparity "dropped pen from above" (1000 events)

disparityFitted curve

Figure 4.15: Disparity measurement of a ”Dropped pen from above”. The eventstream was partitioned into slices with 1000 events (both point sets combined).For every slice, the point set registration was performed and the disparity valuecomputed. A curve was fitted to the resulting values.

The curve fitting application of matlab was able to find a fitting curve. It is alsoshown in Fig. 4.15. The model used for the curve fitting was

f(t) =1

(at+ b)2(4.1)

as this was the only model of the 1t2

-family matlab accepted for fitting. The fittingparameters a and b were calculated by matlab and can be seen in table 4.4, togetherwith the sum of square error (SSE) and the root mean square error(RMSE) of thecurve fitting. The average computational time and average number of iterationscan be found in this table too. The same approach was used when repeating theexperiment with 500 and 200 events per slice respectively. In the case of 500 events,the point sets of 34 slices were registered, from which 10 were miss aligned accordingto Fig. 4.14. The average time range per slice was 5.43ms. In the case of 200 events,the point sets of 85 slices were registered, from which 30 were miss aligned. Theaverage time range per slice was 2.17ms. The resulting time/disparity points andassociated curve fittings can be seen in Fig. 4.16 and in Fig. 4.17.Table 4.4 shows that the average number of iterations decreases significantly with


Table 4.4: Disparity results & curve fitting parameters

1000 Events 500 Events 200 Events

� runtime 4.1611s 1.1432s 0.2319s

� iterations 246 209 134

a -0.0003797 -0.0003846 -0.0003899

b -0.1074 -0.1069 -0.1068

SSE 6.094 10.91 52.31

RMSE 0.6374 0.5839 0.7939

event slices containing a smaller number of events. The resulting parameter a isidentical for the three slice sizes up to ±1.33%. Parameter b is identical up to±0.56%. The best fitting curve has the 500 events experiment with the smallestvalue of the root mean square error(RMSE).

4.2.2 Discussion

The resulting curve fitting parameters a and b of the three tests are relatively close toeach other. This fact suggests that CPD is capable of stable disparity measurementswith event stream data. However, this statement cannot be made with reasonablecertainty, since the actual disparity values are unknown due to the missing sensorparameters and falling pen parameters. The 500 events test shows the smoothestfitting with the lowest RMSE, even better the 1000 events test. So it can not beassumed that larger number of events are automatically improving the quality ofthe point set registration. When reducing the number of events down to 200, theRMSE is significantly higher which means in effect higher deviations between thedisparity values of the event slices. The results in table 3.1 show that with increasingnumber of events, the average number of iterations is also increasing. The runtimeof the test with 1000 events was already approximately 4-times higher then the 500events test. Taking the results of the disparity experiment and runtime experimentinto consideration, an operational range in the area of 500 Events (both point setscombined) is suggested. The time resolution for the 500 events test is an average of5, 43ms per disparity value or ' 184Hz.The failed point set alignments, as shown in Fig. 4.14, are suggesting that CPDhas difficulties with objects which are symmetric or have very simple shapes andoutlines. One the one hand, the disparity values maybe unaffected by these kind oferrors, but on the other hand, the transformation matrix can not be utilized for anyfurther applications. In addition to the disparity values, the horizontal difference ofthe point set means can be seen in Fig. 4.16. This disparity of the means showsa quite similar trend when compared with the actual disparity values. While thecomputation of these values is trivial and computationally cheap, they are definitelyfalsified by outliers.


40 60 80 100 120 140 160 180 200

t/ms

30

35

40

45

50

55

60

65

70

disp

arity

in p

ixel

Disparity "Dropped pen from above" (500 events)

disparityfitted curvemean disparity

Figure 4.16: Disparity measurement of a ”Dropped pen from above”. The eventstream was partitioned into slices with 500 events (both point sets combined). Forevery slice, the point set registration was performed and the disparity value com-puted. A curve was fitted to the resulting values. The red dots are disparity values,computed from the difference of the means of the point sets respectively.


40 60 80 100 120 140 160 180 200 220

t/ms

25

30

35

40

45

50

55

60

65

70

disp

aritt

y in

pix

elDisparity "Dropped pen from above" (200 events)

DisparityFitted curve

Figure 4.17: Disparity measurement of a ”Dropped pen from above”. The eventstream was partitioned into slices with 200 events (both point sets combined). Forevery slice, the point set registration was performed and the disparity value com-puted. A curve was fitted to the resulting values.

47

Chapter 5

Conclusion

This thesis proposed the idea of adapting the coherent point drift algorithm forevent stream registration and mapping of dynamic vision sensor data. The resultsobtained from the implementation of CPD with event stream data suggest that theevent stream registration could perform well but it is also quite sensitive towardsthe input parameters. The alignment of point sets tend to either be successful or tofail completely. Further investigations in how to choose the weight of the uniformdistribution, the number of events and the initial variance, would be a requirementfor any practical application. It was shown that pre-processing steps are either nec-essary or at least would improve the result, since CPD is robust against outliers andnoise, but not unlimitedly. To increase the accuracy, it was proposed to executethe registration in two dimensions. Future work could be done about the analysisand decomposition of the affine transformation matrix. This, for example couldbe, assuming reliable and precise point set alignment, the computation of severaldisparity values for certain features of an object instead of one average disparityvalue as it was the case in this thesis. The fragmentation of the event stream intotwo dimensional event slices, as proposed in this thesis, is quite similar to the frameconcept of conventional cameras. However, applications would still benefit fromthe much smaller data rate of DVS. The disparity measurements showed promis-ing results but the computation time for the point set registration could inhibitsome applications. An implementation in C++ could provide runtime optimization.However, this could prepare the ground for the possibility of a continuous real-timeevent stream registration and disparity measurement. The smoothness constraintof Marr [MP76] suggests that disparity changes smoothly, due to the coherence ofmatter. Therefore, only a few iterations would be necessary in order to respond tochanges of the disparity and to update the event set alignment. Always providedthat computational time could keep up with the changes.

48 CHAPTER 5. CONCLUSION

LIST OF FIGURES 49

List of Figures

3.1 Affine point set registration algorithm . . . . . . . . . . . . . . . . . . 153.2 3D event stream registration . . . . . . . . . . . . . . . . . . . . . . . 193.3 Event stream registration with differing or identical field of views . . 203.4 Image rectification . . . . . . . . . . . . . . . . . . . . . . . . . . . . 213.5 Transformations with respect to the origin . . . . . . . . . . . . . . . 22

4.1 Synthetic event stream registration with outliers . . . . . . . . . . . . 264.2 Influence of oultiers . . . . . . . . . . . . . . . . . . . . . . . . . . . . 274.3 Synthetic event stream registration with noise . . . . . . . . . . . . . 284.4 Influence of noise . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 294.5 Runtime per amount of events . . . . . . . . . . . . . . . . . . . . . . 314.6 Contour plot of GMM pdf with native variance initialization . . . . . 324.7 Contour plot of GMM pdf with smaller variance initialization . . . . 334.8 synthetic event stream registration ”hand” 1 . . . . . . . . . . . . . . 354.9 synthetic event stream registration ”hand” 2 . . . . . . . . . . . . . . 364.10 synthetic event stream registration ”pen” 1 . . . . . . . . . . . . . . . 374.11 synthetic event stream registration ”pen” 2 . . . . . . . . . . . . . . . 384.12 ”Dropped pen from above” 3D . . . . . . . . . . . . . . . . . . . . . . 394.13 ”Dropped pen from above” 2D . . . . . . . . . . . . . . . . . . . . . . 404.14 ”Dropped pen from above” registration examples . . . . . . . . . . . 414.15 Disparity measurement ”dropped pen” with 1000 events . . . . . . . 424.16 [Disparity measurement ”dropped pen” with 500 events . . . . . . . . 444.17 Disparity measurement ”dropped pen” with 200 events . . . . . . . . 45

50 LIST OF FIGURES

LIST OF TABLES 51

List of Tables

3.1 Excerpt from a DVS recording . . . . . . . . . . . . . . . . . . . . . . 17

4.1 Synthetic data example ”hand” . . . . . . . . . . . . . . . . . . . . . 244.2 Synthetic data example ”pen” . . . . . . . . . . . . . . . . . . . . . . 254.3 Runtime per number of events . . . . . . . . . . . . . . . . . . . . . . 304.4 Disparity results & curve fitting parameters . . . . . . . . . . . . . . 43

52 LIST OF TABLES

LIST OF TABLES 53

Acronyms

CPD Coherent Point Drift

GMM Gaussian mixture model

EM Expectation Maximization algorithm

FGT fast Gauss transform

DVS Dynamic vision sensor

pdf probability density function

54 LIST OF TABLES

BIBLIOGRAPHY 55

Bibliography

[AM09] Xubo Song Andriy Myronenko. Point-Set Registration: Coherent PointDrift. IEEE Trans. on Pattern Analysis and Machine Intelligence,32:2262–2275, 2009.

[APD77] D. B. Rubin A. P. Dempster, N. M. Laird. Maximum likelihood fromincomplete data via the EM algorithm. Journal of Royal StatisticalSociety, Series B, 39(1):1–38, 1977.

[Bis95] Christopher M. Bishop. Neural Networks for Pattern Recognition, chap-ter 2. Clarendon Press, 1995.

[BM92] P. J. Besl and N. D. McKay. A method for registration of 3-Dshapes. IEEE Transactions on Pattern Analysis and Machine Intelli-gence, 14(2):239–256, 1992.

[CR03] Haili Chuia and Anand Rangarajanb. A new point matching algorithmfor non-rigid registration. Computer Vision and Image Understanding,89:114–141, 2003.

[FC16] Mohsen Firouzi and Jorg Conradt. Asynchronous Event-based Coop-erative Stereo Matching Using Neuromorphic Silicon Retinas. NeuralProcessing Letters, 43(2):311–326, 2016.

[FTV00] Andrea Fusiello, Emanuele Trucco, and Alessandro Verri. A compactalgorithm for rectification of stereo pairs. Machine Vision and Applica-tions, 12:16–22, 2000.

[GLR+98] S. Gold, C. P. Lu, A. Rangarajan, S. Pappu, and E. Mjolsness. NewAlgorithms for 2D and 3D Point Matching: Pose Estimation and Cor-respondence. Pattern Recognition, 31(8), 1998.

[Gos05] A. Ardeshir Goshtasby. 2-D and 3-D Image Registration, chapter 10.4.2,page 210. John Wiley and Sons, 2005.

[GS91] L. Greengard and J. Strain. The fast gauss transform. SIAM JSSC,12(1):79–94, 1991.

56 BIBLIOGRAPHY

[KHS11] J. Kogler, M. Humenberger, and C. Sulzbachner. Event- Based StereoMatching Approaches for Frameless Address Event Stereo Data. InISVC’11 Proceedings of the 7th international conference on Advancesin visual computing - Volume Part I, pages 674–685, 2011.

[LH01] B. Luo and E. R. Hancock. Structural graph matching using the em algo-rithm and singular value decomposition. IEEE Transactions on PatternAnalysis and Machine Intelligence, 23(10):1120–1136, 2001.

[LH03] B. Luo and E. R. Hancock. A unified framework for alignment andcorrespondence. Computer Vision and Image Understanding, 92(1):26–55, 2003.

[LPD08] Patrick Lichtsteiner, Christoph Posch, and Tobi Delbruck. A 128128×120dB15µs Latency Asynchronous Temporal Contrast Vision Sensor.IEEE Journal of Solid-State Circuits, 43(2):566–576, 2008.

[MP76] D. Marr and T. Poggio. Cooperative Computation of Stereo Disparity.Science, 1976.

[Myr] Andriy Myronenko. Matlab toolbox for coherent point drift.https://sites.google.com/site/myronenko/research/cpd. Accessed:2016-09-03.

[Par] Sebastien Paris. Fast gaussian transform mex implementa-tion. https://de.mathworks.com/matlabcentral/fileexchange/17438-fast-gaussian-transform-mex-implementation.

[PBG13] Ewa Piatkowska, Ahmed Nabil Belbachir, and Margrit Gelautz. Asyn-chronous Stereo Vision for Event-Driven Dynamic Stereo Sensor Usingan Adaptive Cooperative Approach. In Computer Vision Workshops(ICCVW), 2013.

[RBI+12] P. Rogister, R. Benosman, S.H. Ieng, P. Lichtsteiner, and T. Delbruck.Asynchronous event-based binocular stereo matching. IEEE Trans. Neu-ral Netw. Learn. Syst., NNLS-23:347–353, 2012.

[RL01] S. Rusinkiewicz and M. Levoy. Eficient variants of the ICP algorithm.International Conference on 3D Digital Imaging and Modeling, pages145–152, 2001.

[SBMS10] S. Schraml, A.N. Belbachir, N. Milosevic, and P. Schoen. DynamicStereo Vision for Real-time Tracking. In Proceedings of 2010 IEEE In-ternational Symposium on Circuits and Systems, 2010.

[SS01] Linda Shapiro and George Stockman. Computer Vision, chapter 12.6,pages 431–438. Pearson, 2001.

BIBLIOGRAPHY 57

[SSM07] S. Schraml, P. Schon, and N. Milosevic. Smartcam for real-time stereovision - address-eventbased Stereo Vision. In Proceedings of the SecondInternational Conference on Computer Vision Theory and Applications,volume 2, 2007.

[WACC14] David Weikersdorfer, David B. Adrian, Daniel Cremers, and Jorg Con-radt. Event-based 3D SLAM with a depth-augmented dynamic visionsensor. IEEE International Conference on Robotics & Automation, 2014.

[Zha94] Z. Zhang. Iterative point matching for registration of free-form curvesand surfaces. International Journal of Computer Vision, 13(2):119–152,1994.

58 BIBLIOGRAPHY

LICENSE 59

License

This work is licensed under the Creative Commons Attribution 3.0 Germany License.To view a copy of this license, visit http://creativecommons.org or send a letter toCreative Commons, 171 Second Street, Suite 300, San Francisco, California 94105,USA.

http://creativecommons.org/licenses/by/3.0/de/

a probabilistic method for event stream registration · a probabilistic method for event stream...

Documents