[ieee fifth annual ieee international conference on pervasive computing and communications workshops...

Ontology-based Pervasive Spatial Knowledge for Car Driver Assistance

Marcus Tonnis, Gudrun KlinkerTechnische Universitat Munchen

Boltzmannstraße 385748 Garching, Germany

[toennis, klinker]@in.tum.de

Jan-Gregor FischerSiemens AG, Corporate Technology

Otto-Hahn-Ring 681739 Munchen, Germany

[email protected]

Abstract

Deducing spatial knowledge for car driver assistanceis of special importance for upcoming Advanced DriverAssistance Systems. Such systems can not only rely oncar mounted sensors, but also require environmental track-ing. We illustrate our approach of a distributed ad-hocinfrastructure that collects and disseminates tracking dataof environmental objects and thus allows for vehicle- andontology-based reasoning. We discuss, where such a sys-tem can facilitate in driver assistance systems, that requirespatial knowledge.

1 Introduction

Advanced Driver Assistance Systems (ADAS) take care

of safety aspects for car drivers [7]. One of these, for in-

stance, is the Adaptive Cruise Control (ACC), that automat-

ically senses a leading car and controls the speed so that a

safe breaking distance is maintained. But just sensing the

next car in front is not quite sufficient because cars from

other lanes might change to the own lane. In-car systems

need to monitor a wider environment if they want to pro-

vide timely and accurate safety-related information or in-

tervene in emergency situations. Enriched sensor systems

must track all cars and obstacles in the near environment of

the car.

Therefore we proposed a hybrid approach at the Ubi-

Comp associated workshop UbiSys last year. On the one

hand, environmentally fixed tracking systems track certain

areas, while car mounted systems track each car’s near en-

vironment. We are exploring such concepts in a system

that supervises location, speed and trajectory of other vehi-

cles and deduces relevant behavioral traffic data to be used

within a vehicle-centric service-oriented architecture.

For the area of pervasive computing, the challenges of a

highly dynamic environment in conjunction with dynamic

short range network connections are addressed. It is the re-

sponsibility of the individual cars to gather and interpret ac-

cumulated traffic data from such stationary providers along

the road – enhanced by data from on-board sensors such as

radar, near and far infrared and range scanning devices [21].

ADAS systems then need to infer critical events from such

traffic data and provide appropriate feedback to the driver.

ADAS systems, among other things, provide intuitive

and minimally distractive information to the driver of a

car. To fokus of the user interface, such systems require

underlying deduction of a car’s spatial context. To enable

ADAS systems to receive such spatially related information

about traffic situations, we designed and implemented a dis-

tributed system to meet the general issues of such systems.

Our context deduction system, SCORE, a Spatial Con-

text Ontology Reasoning Environment, constitutes, what

pervasive distributed systems need: a peer to peer middle-

ware in conjunction with an applied contextual deduction

system. The system consists of separate autonomous com-

ponents which federate and reason about contextual data

that is related to spatial properties [9, 14]. We show that Se-

mantic Web technologies and logic/rule-based systems are

applicable to a spatial context.

As part of an ongoing research project towards creating

and evaluating novel assistance systems for car drivers, the

SCORE system has helped us overcome the limitations of

the sensory equipment in current cars and look a few years

ahead. In this paper, we illustrate the approach of our con-

text deduction system and the underlying middleware. We

extend the system’s illustration with ADAS systems, under

research in our group, for future use in intelligent trans-

portation systems. Especially ADAS systems, that use AR

for egocentric and therefore minimally distractive informa-

tion presentation are presented.

2 Related Work

Research projects as the project INVENT [5] deal,

among other things, with the subject of car to car com-

munication. Stationary sensors as well as mobile sensors

Proceedings of the Fifth Annual IEEE International Conferenceon Pervasive Computing and Communications Workshops(PerComW'07)0-7695-2788-4/07 $20.00 © 2007

mounted to cars acquire data for use in a traffic routing sys-

tem. Diverse and heterogeneous sources of data are com-

bined to obtain a prognosis of traffic state. In this man-

ner, a comprehensive knowledge base is built up to support

optimal individual route guidance. In order to reconstruct

the traffic state based on fused data coming from differ-

ent sources methods such as computer simulation of traf-

fic flows are used. Route planning then is performed us-

ing a roadway network with dynamic attributes. Waypoint-

algorithms are used as well as graph-theory algorithms in

a in-car computer system. Here, each car deduces its own

relevant information for use in 3rd generation navigation

systems.

The goal of the work of Kosch [13] is to provide pre-

cise and up-to-date information to car drivers, according

to the needs of their individual situation. The developed

CARISMA system independently and autonomously self-

organizes direct wireless transfer of sensor data between au-

tomobiles. By adding new protocols below and besides the

TCP/IP stack, the used protocol design takes special net-

work properties and very dynamic topology into account.

The developed methods allow targeted exchange of infor-

mation between vehicles on the road. The system assesses

the situation-dependent benefit of information and decen-

trally controls the communication with its distributed com-

ponents.

3 Tracking and Overall System Architecture

An increasing number of sensors, such as GPS, radar,

infrared, range and ultrasound sensors, are integrated into

next generations’ cars [21]. Such tracking data is still not

sufficient since directly adjacent cars may be occluding fur-

ther cars that contribute significantly to the overall traffic

conditions.

In our approach, we use a hybrid concept towards gath-

ering and maintaining up-to-date traffic models. The ap-

proach involves continuous communication between car-

based (”inside-out”) mobile tracking equipment in cars

and road-based (”outside-in”) stationary tracking equip-

ment along the road. To provide ubiquitous support along

the road, the street network is separated into road sections,

with each of them containing a number of vehicles at a cer-

tain point in time. For each road section, a logically related

tracking system observes the explicit spatial context states

of all vehicles within that area.

Bearing the prerequisites of such ubiquitous information

management [11, 18] and high dynamics of spatial data in

associated scenarios in mind, SCORE was designed to sup-

port applications on two separate layers that set up on the

observed reality. In the federation layer, all data of a cer-

tain area is federated in a spatial context model. In the next

higher layer, ontology-based reasoning components can ac-

Figure 1. Combining Spatial Context Modelsand Contextual Ontologies

cess the model. Figure 1 shows a sketch of this architecture.

This approach is based on work by [3] and supports the

combination of spatial context models for efficiently pro-

cessing low-level spatial data with ontological context rep-

resentations for information deduction as top layer.

The two layers clearly separate highly dynamic as-

sertional knowledge in traditional coordinate-based mod-

els and terminological information of real world concepts

within interpretable ontology spaces [10]. We strictly dis-

tinguish between the continuous acquisition of context and

its usage by aware applications. This enables facilitated ac-

cess to context data and also its sharing between applica-

tions [12]. Figure 2 illustrates the general architecture.

Our approach combines the advantages of coordinate-

based models with those of ontological knowledge repre-

sentation. Therefore SCORE profits on the one hand from

flexibility regarding spatial scope and detail, and on the

other hand from scalability with respect to dynamics of con-

text acquisition [17].

4 Federation

Federation components mediate between various infor-

mation sources and reasoning components and beyond that,

the ADAS systems. Regarding the volume of communica-

tion traffic, several stationary computers – each logically re-

lated to one road section – run federation components that

collect spatial data within the respective areas. Trackers,


Figure 2. Architectural Overview

act as spatial context providers (see figure 2). These inform

the federation layer about spatial data with frequencies high

enough to permit real time use in safety-relevant applica-

tions in ubiquitous computing and in Augmented Reality-

based applications in general. One application that has to

rely on such data is founded by an ADAS system that is in-

tended to guide a car driver’s attention to the direction on an

imminent danger in the car’s near environment [19]. Figure

3 shows a 3D arrow floating in the Head-up Display (HUD)

of a car pointing into the certain direction of that danger.

Since obstacles change their relative position quickly, the

system requires very high update rates to give an immer-

sive and accurate presentation. Our experiments indicate

reduced reaction times with respect to mini-map presenta-

tions.

5 Reasoning

To deduce relevant information from federated data, the

reasoning system uses an ontology rule-based approach. In

each car a reasoning component accesses spatial data nec-

essary for interpreting the vehicle’s situational information

from a nearby federation server.Each query to the reasoning system consists of a

triplet comprising an ontological object property of interest.Hence, the important prerequisite for SCORE’s functional-ity is the automated matching of entity classes to ontologicalclasses, entity class relations to ontological binary relationsbetween classes, and entity class attributes to ontologicalproperties, that assign data types to ontological classes. Thefollowing two examples show queries for contextual infor-mation whether vehicle ”car007” overtakes ”car001”, andwhich lorry represents a spatial obstacle for the motorcycle”bike27”.

Vehicle=car007 overtakes Car=car001Lorry=* isSpatialObstacleFor

Figure 3. An indicator for the direction of animminent danger in the car’s environment,shown in the Head-up Display

Motorcycle=bike27

Such queries, especially those in a forward direction can

be used to determine possible forward collisions. An ADAS

system (see Figure 4), currently under development in our

group uses information of this kind. In general, the system

shows a bar floating in the HUD to extend a driver’s an-

ticipation about the car’s physical behavior [20]. The bar

indicates the position, the car would come to a stop when

fully pushing the breaking pedal and shows, which area the

car will cover on the actual setting of the steering wheel.

First results point to superior capability to stay in the own

lane without increasing overall cognitive workload. Current

extensions change the color of the bar, when approaching a

leading car and therefore give an implicit follow-up distance

warning.

The terminological interpretation process in the reason-

ing system is based on ontologies known from the Semantic

Web [4, 8]. To define static knowledge, a global ontologyknowledge base plays a central role in the architecture. It

comprises extensible ontologies for a homogeneous appli-

cation domain terminology as well as rules that describe

the validity of binary relations between ontology classes.

Ontologies are well-suited for logic-based reasoning, espe-

cially when description logic [1, 6] is applied. This is due

to the similarity between the language constructs of descrip-

tion logics and those of ontologies. Additional information

can be deduced automatically from pre-defined ontologies

with properties such as equality, symmetry, transitivity, in-

version, disjointness and others.

Ontological knowledge bases are not well-suited for


(a) Bar presentation undr normal conditions

(b) Bar presentation as follow-up distance warning

Figure 4. In HUD presentation of the currentdriving state of the own vehicle showing dif-ferent opportunities for open roads or ladingtraffic

maintaining a detailed history of highly dynamic assertional

data due to the continuous growth of information resulting

in inefficient data management. In spite of that and in order

to cope with the increasing amount of data without giving

up the benefits of an ontological representation of concept

knowledge in application domains dynamic spatial context

data is managed on the federation tier and only requested for

interpreting situational information. The reasoning process

is rule-based, since it addresses the objectivity of spatial re-

lations, e.g. traffic rules, that in its own sense disregard a

scope of varying interpretation [10].

6 Ad-hoc Networking

Interconnections between the federation and the rea-

soning components require dynamic bindings, when a car

leaves one federation sector and moves to another. Then

once more the car built-in reasoning system must get con-

nected to the forthcoming sector’s federation component

and disconnect from the previous one.

For this kind of ubiquitous networking we used the

DWARF [2] architecture to describe and interconnect all

components. DWARF specifies a peer to peer framework

that manages services to dynamically interconnect them de-

pending on their contextual attributes [15]. For instance lo-

calization and tracking distance of services are specified by

such attributes. The DWARF middleware supervises these

attributes and interconnects corresponding data connectors,

thus enabling the data flow.

When a car is going to leave the sector of a specific feder-

ation component, the DWARF middleware determines, that

another environmental federation service has contextual at-

tributes that now fulfill the changed need of the reasoning

service. The new connection between the entered sector’s

federation component and the car’s reasoning system is es-

tablished and the present connection between the two ser-

vices is removed. Figure 5 shows in a sketch how intercon-

nection is executed while a car drives along a X-axis. While

this operation is performed, some of the reasoning service’s

attributes are automatically passed to the federation service,

adjusting and configuring it for addressed queries and noti-

fication frequency [16].

To provide additional tracking data, among other various

information sources a car built in GPS system can also act as

a spatial context provider and therefore propagate the own

car’s position to the federation service that is connected at

this moment.

Figure 5. Sketch of dynamic reconnectionof car internal reasoning service betweentwo federation service sectors as car movesalong X-axis

Inconsistencies that may arise during a hand-over from

one federation service to another, which are similar to hand-

overs of WiFi systems, are not dealt with at the moment.

Our work currently focuses on applied federation and rea-

soning, which we will investigate further on in the future.


7 Performance

To test the performance of our system, that was initially

proposed on UbiSys in 2006, we used a driving simulator to

simulate additional cars, whose amount reflected an aver-

agely frequented road. We developed an ADAS application

that observes the time to collision (TTC) with various addi-

tional cars. Two federation services managed two areas (left

and right of a horizontal X axis dividing the world into two

areas), see figure 6.

Figure 6. The road course of the driving sim-ulator. Federation1 manages the positive sideof the X-Axis, Federation2 the negative

The own car’s position in the virtual world was computed

by a single-lane driving dynamics model – we simulated a

GPS tracker. Depending on the own car’s position in the vir-

tual world, the car’s reasoning service (running on a Linux

PC) was connected to one federation service, querying for

spatial data. Depending on its location in the world, the

reasoner updated its predicate for the selection of a suitable

federation.

The upper part of figure 7 illustrates a snapshot of the

SCORE system’s service structure. Within this snapshot,

the car is on the positive side of the simulated world and

therfore is connected to the service named Federation1. The

lower part of figure 7 shows the predicate that is adjusted in

correspondence to the current position of the own car.

The federation service gives a response time below

10 ms for up to five spatial context providers with up to 250

entities per provider. Compared to an average amount of

cars on an road interval, these response times are clearly ac-

ceptable. The reasoning service responds to n-to-n queries

in 37.1ms seconds for 100 entities. This result is a worst

case result, because general reasoning systems are intended

to be integrated in each car and therefore only have to man-

Figure 7. A snapshot of the services’ inter-connections: The own car is in the positiveside of the federation sector ,,border”

age 1-to-n queries. Going down to 50 entities, which still is

a lot on open fast traveled roads like highways, the response

time sinks to values around 10 ms, as shown in Figure 8.

Taking network delays of cabled installations into account,

the cumulated response or notification time of the system is

below 30 ms. These measurements do not match any wire-

less communication and also do not take any computation

time of tracking systems into account.

Figure 8. Overall response times of the feder-ation service’s entity histories access inter-face method (n-to-n query)

8 Conclusion

In this paper, we presented our system that uses

ontology-based reasoning, resting upon distributed feder-


ating components. As such a system requires a suitable

middleware, we used DWARF for this purpose and showed,

that, besides the tracking infrastructure, such a system is ca-

pable of performing spatially related contextual reasoning.

We also showed that an overall highly dynamic traffic en-

vironment can be split into separate independent areas of

stationary data acquisition, while traveling cars access this

data and conclude information that is relevant for them. The

system demonstrates how Semantic Web technologies and

logic/rule-based systems are applicable to spatial context.

On certain examples we illustrated, that the general func-

tionality, does facilitate innovative ADAS systems.

Our further research on ADAS applications intendes to

simulate a perfect driver in order to give advises to the

real car’s driver. For instance, the proposed system recom-

mends, which lane to choose in heavy traffic or, in other

cases, for instance, just to accelerate to comfortably let an-

other car get on the own lane. Identifying such a system’s

ontological structure will have a certain impact for ubiq-

uitous computing technologies in the sector of intelligent

ground transportation systems. Therefore, the concept of

Ubiquitous Augmented Reality [15] can focus new chal-

lenges in HUDs, enabling further distraction reduced driver

assistance systems.

References

[1] F. Baader, D. Calvanese, D. L. McGuiness, D. Nardi, and

P. F. Patel-Schneider, editors. The Description Logic Hand-book: Theory, Implementation, Applications. Cambridge

University Press, Cambridge, UK, 2003.

[2] M. Bauer, B. Brugge, G. Klinker, A. MacWilliams, T. Re-

icher, S. Riss, C. Sandor, and M. Wagner. Design of a

component-based augmented reality framework. In Pro-ceedings ISAR, Oct. 2001.

[3] C. Becker and D. Nicklas. Where do spatial context-models

end and where do ontologies start? a proposal of a com-

bined approach. In First Intl Workshop on Advanced Con-text Modelling, Reasoning and Management, pages 48–53.

University of Stuttgart: SFB 627 (Nexus), 2004.

[4] T. Berners-Lee, J. Hendler, and O. Lassila. The SemanticWeb. Scientific American, 2001.

[5] Bundesministerium fur Bildung und Forschung (BMBF).

INVENT - Driver Assistance Systems - Active Safety, 2005.

http://www.invent-online.de/.

[6] D. Calvanese, M. Lenzerini, and D. Nardi. Description log-

ics for conceptual data modeling. In Logics for Databasesand Information Systems, pages 229–263, 1998.

[7] C. Christen. Intelligent Safety Systems - Sustainability Re-

port in Economics and Mobility, 2002. Adam Opel AG.

[8] J. Davies, D. Fensel, and F. van Harmelen. Towards the

Semantic Web: Ontology-driven Knowledge Management.

Technical report, Wiley, UK, 2002.

[9] A. K. Dey and G. D. Abowd. Towards a better understanding

of context and context-awareness. In 1st International Sym-

posium on Handheld and Ubiquitous Computing (HUC),1999.

[10] Fischer, J.-G. Spatial Context Management for Augmented

Reality Applications, SCORE: Spatial Context Ontology

Reasoning Environment. Master’s thesis, Technische Uni-

versitat Munchen, Chair for Computer Aided Medical Pro-

cedures and Augmented Reality, 2005.[11] A. Galloway. Resonances and everyday life: Ubiquitous

computing and the city, 2003.[12] H.-W. Gellersen. Ubiquitous Computing. In Mobile Net-

works and Applications, 2001.[13] T. Kosch. Situation-Aware Communication in Vehicle Ad-

hoc Networks. PhD thesis, Technische Universitat Munchen,

2005.[14] C. Kray. Spatial reasoning and agents: Exploring possibili-

ties, 2001.[15] A. MacWilliams. A Decentralized Adaptive Architecture for

Ubiquitous Augmented Reality Systems. PhD thesis, Tech-

nische Universitat Munchen, Chair for Applied Software

Engineering, 2005.[16] A. MacWilliams, T. Reicher, and B. Brugge. Decentral-

ized coordination of distributed interdependent services. In

IEEE Distributed Systems Online – Middleware ’03 Work inProgress Papers, Rio de Janeiro, Brazil, June 2003.

[17] K. Rothermel, M. Bauer, and C. Becker. Digitale Weltmod-

elle - Grundlage kontextbezogener Systeme, 2003.[18] P. P. L. Siu, N. Belaramani, C. L. Wang, and F. C. M. Lau.

Context-Aware State Management for Ubiquitous Applica-

tions. In Lecture Notes in CS, 2004.[19] M. Tonnis and G. Klinker. Effective Control of a Car Drivers

Attention for Visual and Acoustic Guidance towards the Di-

rection of Imminent Dangers. In Proceedings of the ISMAR,

Oct. 2006.[20] M. Tonnis, C. Lange, G. Klinker, and H. Bubb. Transfer

of Flight-Tunnel-Presentations into the Head-Up Display of

Cars. In Integrierte Sicherheit und Fahrerassistenzsysteme,Vol. 1960, VDI Wissensforum, Germany, 2006. in german.

[21] L. Walchshaeusl, R. Lindl, K. Vogel, and T. Tatschke. Detec-

tion of road users in fused sensor data streams for collision

mitigation. In Proceedings of the 10th AMAA), Berlin, April

2006. VDI/VDE/IT.


[ieee fifth annual ieee international conference on pervasive computing and communications workshops...

Documents