Whom to talk to? Estimating user interest from movement trajectories

Steffen Muller, Sven Hellbach, Erik Schaffernicht, Antje Ober, Andrea Scheidig, and Horst-Michael Gross

Abstract— Correctly identifying people who are interested inan interaction with a mobile robot is an essential task for asmart Human-Robot Interaction.

In this paper an approach is presented for selecting suitabletrajectory features in a task specific manner from a hugeamount of different forms of possible representations. Differentsub-sampling techniques are proposed to generate trajectorysequences from which features are extracted. The trajectorydata was generated in real world experiments that includeextensive user interviews to acquire information about userbehaviors and intentions. Using those feature vectors in aclassification method enables the robot to estimate the user’sinteraction interest.

For generating low-dimensional feature vectors, a commonmethod, the Principle Component Analysis, is applied. Theselection and combination of useful features out of a set ofpossible features is carried out by an information theoreticapproach based on the Mutual Information and Joint MutualInformation with respect to the user’s interaction interest.The introduced procedure is evaluated with neural classifiers,which are trained with the extracted features of the trajectoriesand the user behavior gained by observation as well as userinterviewing. The results achieved indicate that an estimationof the user’s interaction interest using trajectory informationis feasible.

I. INTRODUCTION

A future application of interactive service robots, likethe robot platforms HOROS [1] or SCITOS [2], will be toprovide information and other services to people in publicenvironments and office buildings. A main question is howthe robot should attract the attention of its interaction part-ners. It would be inconvenient to let the robot address everydetectable person in the surrounding area, or to let him waitmotionless for people to approach the robot themselves.

Intelligent robots should be able to show a natural human-like behavior. For instance, they should meet halfway withsomeone who is on the verge of starting an interaction andattract people who are not sure about the robot’s abilities.In addition, they should avoid disturbing people who areunwilling to interact with them.

In order to do so, a robot must have the ability to estimateon its own, who is willing to interact and who is not. Thisestimation has to take place early enough before the person isreaching the robot, to enable the system to accomplish a user-adaptive welcoming. Human beings infer the intention ofother people from their mimics, gestures, and body language.As a result of limited camera resolution, a robot is not able

This work has partially received funding from the European Com-munity’s Seventh Framework Programme (FP7/2007-2013) under grantagreement n◦ 216487. All authors are with Neuroinformatics and CognitiveRobotics Lab, Ilmenau University of Technology, 98684 Ilmenau, Germanysteffen.mueller@tu-ilmenau.de

to specify those at far distances. A promising idea, followedin this paper, is to extract the required information from aperson’s movement trajectory which is available as soon asthe person is detected by the robot.

Section II gives a short overview on recent research in thisparticular field of Human-Robot Interaction. This is followedby a description of the scenario in section III. Afterwards asystematization on different methods for trajectory represen-tation is presented in section IV. How to select the rightrepresentation method is discussed in section V. After theresults are presented in section VI, the paper concludes withsection VII.

II. RELATED WORK

The estimation of a user’s interest to interact with a robotwith the common goal of adapting the robot’s behavior instarting an interaction became focus of attention in differentworks. Most approaches consider only the distance andcertain zones to infer a user’s intention since one-to-oneinteractions are typically carried out in small distances. Finkeet al. [3] trained HMMs with data from sonar sensors torecognize people that were closer than 1 m to the robot.It was assumed that people which entered that zone areinterested in an interaction. Nabe et al. [4] discovered ina field trial with a ROBOVIE-M robot that three zones existin which a person shows either a watching, a talking or aphysical interacting behavior depending on their distances tothe robot. They announced to regulate the robot’s behaviorjust by these distances. Michalowski et al. [5] subdivided thesurrounding area of the roboceptionist VALERIE into certainzones, also depending on the distance to the reception desk.These zones were used together with head pose informationfrom a camera to determine the degree of the person’sengagement in a Human-Robot Interaction in order to controlthe rule-based behavior of Valerie in contacting people. Anapproach with the main focus set on the selection of therobot’s actions can be found in Schulte et al. [6]. The tour-guide robot MINERVA measured the probability distibution ofpeople in the surrounding area as well as their distance to therobot via laser. This information is used in a memory-basedreinforcement learning approach to learn the best action ofthe robot for attracting people and involving them in aninteraction.

A known problem in the use of distance and zones ascriteria is that these measures might be applicable in somescenarios but it seems insufficient when people are movingquickly in the robot’s surroundings. Preferably, the classifica-tion takes place as early as possible to let the robot react fastenough. The incorporation of movement trajectories enables

this course of action. A first step into this direction was madeby Holzapfel et al. [7]. They trained a neural network withdata from a stereo camera tracker. At each classification step,some angle and distance features were used as input. Peopleheading towards the robot were classified as being interestedin an interaction, and people passing the robot as being notinterested.

Within the discussed approaches the classification problemis always seen from the robot’s view by using a robotcentered coordinate system. In comparison, our approachinterpretes the scene from the person’s point of view whichbrings some advantages. Moreover, often only raw trajectorydata is considered as classification input so far. Applyingspecific preprocessing and automatic evaluation steps mightlead to more suitable classification results which is proposedin this work.

III. SCENARIO AND DATA ACQUISITION

To realize the estimation of the users’ intention to interactwith a robot, it is necessary to capture movement trajectorydata. A spatio-temporal classifier, in particular a standardfeedforward Neural Network, is trained with this data. Thissection discusses the data aquisition scenario.

We used our experimental robot plattform HOROS for thispurpose. The robot is shown in Fig. 2, while the details ofthe applied multimodal people tracking system have beenintroduced in [8].

This approach is using dif-

���

���

���

���

Fig. 2. The robot plattformHOROS. The marked devices are(a) microphones, (b) touch screen,(c) laser range finder, and (d)sonar sensors.

ferent sensors of the robot toget a set of hypotheses of peo-ple’s positions in the robot’ssurroundings. A local occu-pancy gridmap which is builtup from the measurements ofthe robot’s sonar sensors iscompared to a static map ofthe environment, yielding hy-potheses based on unexpecteddifferences. Besides the hy-potheses from this cue, a SICKlaser range scanner is usedto detect legs in greater dis-tances. Using a difference ap-proach, individual rays of therange scan are compared toa dynamic background model,which adapts slowly to varia-tions e.g. due to slight move-ments of the robot. The dis-

advantage of the laser model, that people become part ofthe background if they don’t move for a certain time, iscompensated by the combination of the different sensors bymeans of probabilistic sensor fusion techniques. The systemgenerates a list of trajectory points for people moving in a6 m circle around the robot with an update frequency of 10Hz.

For the purpose of recording meaningful trajectory data anappropriate location is needed. This comes with some stringsattached. Besides having enough space to detect and track aperson soon enough for accomplishing a robot action beforethe person leaves the robot’s interactive operation area, theplace should have a high frequency of single individualspassing by. One disadvantage of the used tracking systemis that people who walk very close to each other are hard toseparate.

However, two entrance halls at our university were chosen,which seemed appropriate for this task. The robot was placedin three different positions on 15 non-consecutive days forabout 6 hours each in order to gain trajectories from differentrobot views. This allowed a position invariant estimation ofthe user’s intention to interact.

Besides the choice of the place, different design decisionshad to be made before the experiments could be carriedout. First of all, a promising scenario for the target group,consisting of students, employees and visitors, had to befound. As the robot was supposed to be deployed for a longerperiod of time, we wanted to offer an application which leadto both, interactions with new people as well as repeatedinteractions with people passing the robot regularly in orderto integrate the robot into their everyday life.

The resulting information terminal application providedthe daily menus of different cafeterias, the ongoing cinemaprogram, bus timetables, jokes, events and as a specialfeature, an audio-based gender estimation accomplished bythe robot. An eye-catching GUI was created especially for theapplication which was presented on the robots touch display.It had a simple navigation structure to ensure that even peoplewho are not familiar with using technical devices are ableto interact with the robot intuitively. To let the robot differsufficiently from a static information terminal, we focusedon the integration of a broad spectrum of robot activitiesby varying the robots voice responses, changing its facialexpressions and trying to produce a continuous dialogueprocess.

The robot behaved as follows during the experiments:Whenever a person was detected by HOROS, a randomlychosen voice output started which was intended to attract theuser’s attention and influence the user’s decision to interact.

Each possible voice output had a different effect on theuser. These actions had specifiable moods, they were meantto be funny, happy, or provocative. Several actions weretested to find the most suited to the people’s expectations andwishes. After an action was initiated, the robot waited for theperson to approach or to pass by. Via its sensors the robottook note of hesitating people, too. In that case, the robotstarted an additional animating voice output to convince theperson to interact. During each trial hidden experimentatorswatched the scene and made notes about what happened. Theexperimentators waited until the current person had left therobot’s surroundings. Afterwards they tracked the person andstarted an interview to gain new knowledge about Human-Robot Interactions besides just recording trajectory data.

Fig. 1. Overview of different kinds of representation for trajectories. Choosing the right reference system (a), coordinate system (b) and samplingtechnique (c) is highly task dependent. Optionally, it is possible to apply dimension reduction (d). In (d) the influence for two of the Eigentrajectories onthe recomposed trajectory are shown. Choosing informative representations via feature selection (e) and the neural network as final classifier (f) round outthe system.

Especially the process of establishing dialogs with humanswas our field of interest, concerning getting the attentionof people, signalizing the robot’s interest to interact andfinally establishing the interaction. A bunch of questionsarised in this context that we tried to answer by designinga questionnaire for user interviews. Details regarding thequestions, the collected answers and conclusion are providedin section VI.

IV. TRAJECTORY REPRESENTATION

One important step after data acquisition using a trackeris to transform the trajectories into an appropriate rep-resentation. This helps to provide necessary informationabout the trajectory more explicitely, which will support theclassifier. Similar as performing image pre-processing stepsbefore classifying the image content, trajectory data needs toundergo an adequate transformation.

Being confronted with the fact that collecting real worlddata for training is time consuming, one has to deal withfew training examples. Having only a small amount of dataimplicitely means that the number of input dimensions of theclassifier is not supposed to exceed a certain amount.

The final classification algorithm, as well as all necessarypreprocessing steps, have to work online on the robot. Thismeans the method has to deal with the fact, that during theclassification process only a part of the trajectory is known.Hence, the classifier is trained with trajectory fragments.Those fragments are extracted by sliding a window witha fixed size over the trajectory (highlighted end of thetrajectory in Fig. 3(a)). For obtaining the right window size,methods from time series analysis could be used [9]. Sincethe system aims at online classification, a compromise has tobe found. On one hand having an adequate length to provideenough information is crucial and on the other hand, beingable to estimate the user’s interest in a reasonable short timebefore passing by is important, too.

(a)

(b) (c)

Fig. 3. (a) Reference systems: Contextual Information is provided byspecifying the polar coordinates of the trajectory’s origin (φO, dO) in astationary coordinate system originating at the robot’s position (which isin this case facing to the right). The positions of the robot (φR, dR) orother target objects (like a door) (φT , dT ) are provided in a trajectorycentered coordinate system. (b) Trajectory centered coordinate system: Theupper plot shows a point (xi, yi) on the trajectory represented in cartesiancoordinates, while the lower one shows its intrinsic representation (αi, li).(c) Sampling techniques: In the upper graph the trajectory is sampled withfour points (×) being equally spaced with distance b. The lower one showsthe cartesian x-axis ploted against time. Equidistant sampling concerningthe t-axis leads to different points on the trajectory (+).

Unfortunately, choosing the right representation is highlytask dependent. Most recent publications either spend mucheffort in designing a very task specific solution using expertknowledge or skip it at all. This paper’s intention is topresent an evaluation procedure to support the designer inthe decision-making process. For this purpose, Fig. 1 givesa short systematization of necessary steps to be considered.So far, this systematization is not complete, but it helps toget an idea of the necessary steps to follow.

The user’s situation is described within the tracking data.To allow varying views on this situation, the data is splitup into different forms of representations. These forms areachieved by using diverse reference systems (Fig. 1(a)),which may contain a different amount of information. Inthis paper three versions are discussed.

Choosing the right reference system has a high influenceon the classification task. For example, if the robot wouldpresent general information, e.g. weather report, the directionfrom which a person approaches would play no decisive role.So a target referring coordinate system would be sufficient.But if the robot presents information about the place it islocated in, e.g. in a museum, arriving people would probablybe more interested than leaving ones. In this case a contextualview is essential.

First among the considered representation is a contextualreference system (Fig. 3(a)), modeling information about thesurroundings. This paper suggests to provide informationabout the origin of the trajectory in a robot-centered way.This is done by specifying the first detected point of thetrajectory (φO, dO) in relation to the robot’s fixed position.Most approaches are limited to such global types of represen-tation. For the classification task presented in this paper usingthis technique is expected to lead to unsatisfying results.

The intention of the user to interact with the robot can beobserved in a better way by centering the coordinate systemto the user and specifying the relative position to objects ofinterest (Fig. 3(a)). Literarily spoken this helps the robot toput itself into the person’s position. Such a target referringrepresentation is done for the last point in time of eachtrajectory. Hence, the relative position of the nearest target(φT , dT ) with the smallest angle and the robot (φR, dR) isstored. This is done by specifying the distance d and theangle φ relative to the person’s motion direction.

Furthermore, to rid the representation of the surroundingsa purely trajectory centered reference system is used. So,the characteristics of the trajectory is put into focus, whichcontains implicitly the velocity, acceleration and straightness.

Two different ways are suggested for describing the tra-jectory in a trajectory centered reference system (Fig. 1(b)).The first one (lower graph in Fig. 3(b)) can be comparedwith a route description. The trajectory is represented in anintrinsic way, which leads from one trajectory point to itssubsequent one. This is done by specifying the distance liand the angle αi with respect to the motion direction to thenext point.

For the second one (upper graph in Fig. 3(b)) the originof the cartesian (x, y)-coordinate system is set to the lasttrajectory point in time. The coordinate system’s orientationis determined by the corresponding first trajectory point inthe window. In our case the orientation is chosen in the waythat this point lies on the x-axis.

Data originating from the tracker is sampled in a specificway. But what kind of sampling technique (Fig. 1(c) andFig. 3(c)) helps to get most of the information? Samplingin a spatial manner helps to keep the number of data pointsconstant for a certain trajectory length. For example, if a

person is standing, only one data point is sampled insteadof a point cloud. So, the shape of the trajectory is in thefocus of this type of representation. This is similar to theway a HMM would handle the observations on the trajectory,compensating variations in speed and time.

On the other hand, sampling can be done in a temporalequidistant way. Hence, the speed of a person is codedimplicitly in the trajectory.

Some of the information implied in each of the trajec-tory representations might be correlated. Finding out thecorrelated data helps to reduce the number of informationthat needs to be provided to the classifier. For this taskPrincipal Component Analysis (PCA) is commonly used,which provides a set of Eigenvectors and Eigenvalues. TheEigenvectors, which are called Eigentrajectories (Fig. 1(d)) inthis paper, span a new space, while the Eigenvalues indicatethe variance of each dimension of the new space.

The systematization presented in this section is to beunderstood as a first proposal. Further efforts have to bemade to achieve an exhaustive taxonomy. Still, it becomesclear, that a large number of representations can be generated.But which one is the correct choice for the given task ofestimating the user’s interest in an interaction with the robot?To make this determination, information theoretic measureslike Mutual Information and Joint Mutual Information (Fig.1(e)) are applied together with the class labels gained fromthe user observations and interviews. Both are discussed inthe next section.

V. FEATURE SELECTION AND CLASSIFICATION

The decision whether an observed person is interested inan interaction with the robot is estimated by a neural classi-fier, more specifically a Multi Layer Perceptron (MLP). Thisclassifier is trained with a part of the recorded data referredto as training set, and the evalution is done subsequentlywith the rest of the available data termed test set. Eachvariable or dimension of a specific trajectory representationis considered to be a feature.

Due to the abundance of possible input feature represen-tations, the resulting neural classifier using all features is notonly prone to overfitting and related problems like increasedtime demands for training, but we are interested in finding thebest way to preprocess our data, too. Hence, we incorporateda feature selection step. These feature selection methods areapplied to identify irrelevant and redundant features in theinput space.

Feature selection methods are commonly distinguishedinto “Filter”, “Wrapper” and “Embedded” approaches (see[10]). Both wrapper and embedded methods utilize a learningmachine to evaluate the features, while the filter algorithmsare independent of the used classifier. For this work a filterapproach is chosen, because of this independence property.The relevance of each feature is assessed with a statisticalmeasure, e.g. simple Pearson correlation coefficient, Fisherdiscriminant analysis or information theoretic measures.

In this case, Mutual Information (MI) is applied (see [10],chapter 6). MI expresses the correlation between a feature

Fig. 4. Interpretation of the information-theoretic measure Mutual Informa-tion for our scenario. The approaching person sends signals of interactioninterest via gestures, facial expression and movement trajectories. Up tonow, the presented system only captures the movement trajectories includingsome measurement noise, while e.g. gestures are lost. Additionally, maybesome parts of the movement trajectory are not correlated to the interactioninterest at all. The relevant signals observable by the robot are measuredwith MI.

xi, e.g. the x-coordinate of the position at the first trajectorypoint, and the target Y , the class information representingthe intention to interact or not to interact. High MI valuesindicate a strong correlation, while small values are hintingat irrelevant features. It can be determinded by the followingequation, which is based on the well-known Kullback-Leiblerdivergence:

MI(xi, Y ) =∫

xi

∫

Y

P (xi, Y ) logP (xi, Y )

P (xi)P (Y )dxidY (1)

A graphical interpretation of the meaning of Mutual In-formation in the context of the scenario is given in Fig. 4.

The computation of the probability densities is quitedemanding, due to the necessary integration. In the practicalimplementation a histogram based approach was used to es-timate the Probability Density Functions (PDF), simplifyingthe above equation by replacing the integrals with sums overthe bins of the histogram. For more details on the histogramtechnique and other approaches for PDF estimation theinterested reader is refered to [11].

The resulting MI value indicates the relevance of a featurewith respect to the target, but it does not include informationabout possible redundancies between features. This drawbackcan be overcome by computing the pairwise Mutual Infor-mation MI(xi, xj) between the features. In this case a highMI value indicates redundancy, low values independence.

A step further, the use of Joint Mutual Information (JMI)eliminates another possible problem, that occurs if twofeatures are unimportant themselves, but provide informationif they are used in combination (think of the classical XORproblem). JMI computes the relevance of a set of featuresX = {x1, .., xn} with respect to the target in the followingway:

JMI(X,Y ) =∫

X,Y

P (x1, .., xn, Y ) logP (x1, .., xn, Y )

P (x1, .., xn)P (Y )(2)

Selecting feature sets from different trajectory representa-tions with high relevance for the user’s interest is the finalstep before the training of the classifier. The next sectiondiscusses which representations proved to be relevant.

VI. EXPERIMENTS & RESULTS

This section discusses the conducted experiments, theobtained results and what conclusions can be drawn.

After the trajectory recording sessions, we were left witha very unbalanced data set. The number of 43 interactionswith the robot is quite low, compared to the number of 450non-interaction cases. However, an interaction took place ifand only if the user provided input to the system, e.g. pusheda button. Additionally, 34 individuals stated that they wereundecided but finally did not interact with HOROS.

The analysis of the questionnaires revealed some ofthe reasons for this unexpected behavior. The main issuescovered with our questions were:

• How should a robot behave in public environments,especially in office environments?As expected, the results showed that friendly and funnyactions were preferred from the users and lead to mostinteractions. Sad or provocative robot voice outputs arenot recommendable since only in a few cases success-ful interactions could be established by this behavior.Moreover those actions had a slightly negative influenceon the users. Interestingly the results indicate that aninformal way to talk to people is more suitable thanbeing too formal.

• Which robot actions are appropriate for robots in publicenvironments, especially in office buildings, e.g. whichvoice output, robot movements and facial expressionsshould be used?The choice of voice outputs was already discussed in theprevious question. Additionally, people wanted to get asmuch information as possible about the robot’s abilitiesduring dialog initialization but with as little amount ofwords as possible. We found out that next to a voiceoutput a movement of the robot is very important. Manypeople noticed that HOROS didn’t move during theexperiments at all and suggested to include movementsin order to recognize the robot earlier while passingby the robot. Regarding the robot’s facial expression, afriendly and smart smiling was the best choice.

• Is a user-adaptive robot behavior necessary for suchenvironments during dialog initiation and beneficial forfuture work?In our long-term experiment we figured out that auser-adaptive robot behavior is necessary during dialoginitiation. People who had to pass by the robot regularlyand didn’t have interest in an interaction tried to avoidthe robot and partially changed their way through thebuilding in order to be not recognized and contactedby the robot. Also we couldn’t convince all undecidedpeople just by using voice output. For those people, amore active robot behavior would be necessary. For peo-ple who had interest in an interaction a time-consuminggreeting was needless. A short greeting and meetinga person half way is completely appropriate for thesecases.

• How good is HOROS perceived in an environment bypeople who don’t know about its presence and howmuch is the robot able to arouse interest in an inter-action?Since HOROS was standing static during the experi-ments, it partially was perceived too late. The eye-catching appearance of HOROS did not compensate thelack of movements.

• Which robot dependent and independent variables influ-ence the user’s descisions to interact with the robot, e.g.the person’s mood, haste and attitude towards robots?In our questionnaires we tried to collect informationabout all possible facts that could have an effect ona user’s intention to interact with the robot. Othervariables like a persons mood, gender or a generalinterest in robots were apparently no factor for engagingin interactions.

• Why did people dodge an interaction?One motive are time constraints. Because of them, manypeople, students and employees alike, rushed throughthe hall in a hurry. Another reason was, that peopleavoided the robot because they were unsure if theywere allowed to use the system or expressed fear todamage the robot. Those people who interacted withthe robot once, refused to use him again, mostly becausethe application of the information terminal did not yieldenough benefit for them.

The main insight gained during the data selection phasewas, that people in a working environment need a strongreason to use a robot.

In the following, the recorded trajectories are examined indetail. The different forms of trajectory representation, seesection IV, are compared to each other to identify a usefulsubset of trajectory features. At first, the time equidistant re-sampling method is compared to the spatial equidistant one,while intrinsic and cartesian trajectory centered coordinates,are analyzed in more detail. The following evaluations havebeen done once for both resampling methods, all leadingto the conclusion that, contrary to expectations, the disad-vantage of losing velocity information in the case of spatialresampling is not that relevant for the given classificationtask. For all further illustrations, we will concentrate on timeequidistant resampling, knowing that spatial resampling willyield similar results.

The next point of interest is the amount of informationcontained in the raw data points considering the intrinsicand cartesian descriptions (see section IV).

Fig. 5 shows a plot of the Mutual Information betweenthe individual features and the interest of people to inter-act, computed as described in section V. A look at thefigure shows that in comparison cartesian coordinates (xi, yi)seem to contain more information about the target interestthan intrinsic ones (αi, li). Especially the x-coordinate ofcartesian representation is coding the speed and thus seemsto be the most relevant. By plotting the pairwise MutualInformation between the cartesian coordinates (see Fig. 6(a)) it is observable that there is a high redundancy between

Fig. 5. This image shows the Mutual Information for the individual featuresof the datapoints to the interest in an interaction. Columns 1 to 12 show theresampled datapoints in cartesian coordinates (first x then y components),13 to 18 describe the principal components for the cartesian representation,19 to 30 show the data using intrinsic representation (angle α and length lof segments), 31 to 36 principal components of intrinsic representation. Theright diagram shows the information of target refering data (relation to therobot dR, φR and to the closest target point dT , φT ) on a different scale.

Fig. 6. Illustration of the pairwise Mutual Information matrices for theraw data in cartesian coordinates (a) the respective principal components(b), as well as the intrinsic representation (c) and corresponding principalcomponents (d). Yellow/Brighter color represents high values and indicatesthat both variables share a lot of information, which means that they areredundant, at least partially. e.g. the MI between x6 and x5 is very highand thus one of the variables is redundant. Green/Darker ones imply lowervalues and an independence of the considered variables. (a) and (b) havethe same scale which is different to the one of (c) and (d) which is lowerat all.

the features in the trajectory window. For example, lookingat matrix entry (x5, x6), which has a high pairwise MutualInformation leads to the conclusion that only one of thetwo features is necessary for the classification process. Incomparison with entry (y1, y4) the low value between thoseleads to the conjecture that both features are not redundant.The x-coordinates have higher correlations because theyresult from translative motion, whereas the y-coordinates arecoding the shape of the trajectory in the window includingthe curvature and therefore are less correlated among eachother than the x’s. An absolute quantitative comparison isfairly hard but the relation is observable.

In contrast to this, angular descriptors (Fig. 6 (c)) ofthe intrinsic representation are less redundant. But for thisrepresentation the length of the segments is highly correlated,

MLP inputs Network complexity errorraw data (x1, y1, . . . , x6, y6) input 12, hidden 16 0.2252PCA data (e1, . . . , e12) input 12, hidden 16 0.2454raw data (x4, y4, x5, y5) input 4, hidden 10 0.2057PCA data (e1, . . . , e4) input 4, hidden 10 0.193(x4, y4, x5, y5), (φR, dR, φT , dT ) input 8, hidden 16 0.1756

TABLE I

Classification error of MLPs using different subsets of features (first column)extracted from the windows of people’s trajectories (see section IV). Thebinary output is compared to the class information and the errors areaveraged over the test data set to compute the final error rate. The bestresult was achieved using the target refering information additionally.

because people usually do not vary their velocity that fast.

As introduced in section IV, the method used to reducelinear redundancies is PCA. In Fig. 5, the information ofthe most significant principal components ei and fj (theprojection on Eigentrajectories) are given for both kindsof representation. In the cartesian case, the information isconcentrated in the most significant components, which arenearly decorrelated, as a look on the pairwise Mutual In-formation matrix Fig. 6(b) shows clearly. These componentsdescribe the local shape of the trajectory, almost as well asthe complete raw data does.

In contrast the intrinsic representation decorrelates thepoints on themselves. Considering the different scale, theMutual Information matrix shows that the correlations be-tween the principal components does not decrease at all.As a result we conclude, that the shape information of thetrajectory centered data is expressed in the most compactway by projecting the cartesian raw data onto the mostrelevant Eigentrajectories (principal components).

The experiments with the classifier that has been trainedonce with the raw data and once with the data compressedwith PCA, reached similar performances with a quite com-plex network. But as given in Table I, the compressedrepresentations (line 2 and 4) allow to keep the networksimple and, therefore, a better generalization over the testdata set can be reached. During the learning phase, thetarget is set to one for interaction interest and zero for non-interacting trajectories. By means of this, the classificationis done by thresholding the real-valued output in the interval[0, 1] at 0.5. Hence, an expected interaction is coded asone and a non-interaction as zero. The classification errorresults from averaging the binary classification results overall trajectory windows in the test data set. This includes theambiguous parts observed in the beginning of a trajectory.

A last experiment considering the obtainable classificationaccuracy was conducted. The expectation from Fig. 5 isthat the classification rate could be increased once moreby taking into account the target refering information foreach trajectory window. Finally, without optimization of anynetwork structure, a rate of about 82% correct classifications(see line 5 in Table I) promises a great benefit when selectingthe right action to attract interested people or avoid incuriousones.

VII. CONCLUSION

This work concerns the task of classifying people in thesurroundings of the robot as interested in an interaction oras incurious. The available hypotheses about the environmentcome from the people tracking system of the robot, whichis used to create movement trajectories. After discussingdifferent approaches to represent trajectories, a method waspresented to prune these possibilites by means of informationtheoretic measures.

This method conveniently supports the design process andhelps to break dependecies from expert knowledge. Hence,the presented method is applicable for a wider spectrumof applications than the one presented here. Projecting thecartesian raw data into a PCA subspace proved to be the bestsuited approach for the trajectory classification task.

In our future, work this knowledge about the user’s currentinterest in an interaction will be used to automatically adaptthe welcoming strategy of the robot.

Further on, the presented approach will be applied toestimate other user specific properties, like the degree ofurgency. This will allow us to refine and personalize thedialog between human and robot even further.

REFERENCES

[1] H.-M. Gross, J. Richarz, S. Muller, A. Scheidig, and C. Martin,“Probabilistic multi-modal people tracker and monocular pointing poseestimator for visual instruction of mobile robot assistants,” in Proc.2006 IEEE World Congress on Computational Intelligence (WCCI2006), International Joint Conference on Neural Networks (IJCNN-06), 2006, pp. 8325–8333.

[2] H.-J. Bohme, A. Scheidig, T. Wilhelm, C. Schroter, C. Martin,A. Konig, S. Muller, and H.-M. Gross, “Progress in the developmentof an interactive mobile shopping assistant,” in Proc. of the Joint Conf.on Robotics: 37th International Symposium on Robotics (ISR 2006)and 4th German Conference on Robotics (Robotik 2006), 2006.

[3] M. Finke, K. L. Koay, K. Dautenhahn, C. Nehaniv, M. Walters, andJ. Saunders, “Hey, i’m over here - how can a robot attract people’sattention?” in in Proc. 14th IEEE International Workshop on Robotand Human Interactive Communication (RO-MAN 2005), Nashville,USA, 2005, pp. 7–12.

[4] S. Nabe, T. Kanda, K. Hiraki, H. Ishiguro, K. Kogure, and N. Hagita,“Analysis of human behavior to a communication robot in an openfield,” in Proceedings of the Conference on Human-Robot Interaction(HRI). Salt Lake City, USA: ACM, 2006, pp. 234–241.

[5] M. P. Michalowski, S. Sabanovic, and R. Simmons, “A spatialmodel of engagement for a social robot,” in Proceedings of the 9thInternational Workshop on Advanced Motion Control (AMC 2006).Pittsburgh, USA: IEEE Computer Society, 2006, pp. 762–767.

[6] J. Schulte, C. R. Rosenberg, and S. Thrun, “Spontaneous, short-terminteraction with mobile robots,” in Proceedings of the IEEE Inter-national Conference on Robotics and Automation (ICRA). Detroit,USA: IEEE Computer Society, 1999, pp. 658–663.

[7] H. Holzapfel, T. Schaaf, H. K. Ekenel, C. Schaa, and A. Waibel, “Arobot learns to know people - first contacts of a robot,” in KI, vol.4314. Bremen, Germany: Springer, 2006, pp. 302–316.

[8] A. Scheidig, S. Muller, C. Martin, and H.-M. Gross, “Generatingperson’s movement trajectories on a mobile robot,” in Proc. RO-MAN2006 - 15th IEEE Int. Symposium on Robot and Human InteractiveCommunication, 2006, pp. 747–752.

[9] B. Schelter, M. Winterhalder, and J. Timmer, Eds., Handbook of TimeSeries Analysis. Wiley-VCH, 2000.

[10] I. Guyon, S. Gunn, M. Nikravesh, and L. Zadeh, Eds., FeatureExtraction, Foundations and Applications. Springer, 2006.

[11] D. W. Scott, Multivariate density estimation: theory, practice, andvisualization. John Wiley & Sons, 1992.