seminar info (autosaved)

8/7/2019 seminar info (Autosaved)

1/21

The Processing System

Architectural Issues

Two different architectural solutions have been pointed out and

evaluated: special-purpose and standard processing system. In

particular during the project a massively parallel architecture was

developed: PAPRICA, based on a square array of 256 single-bit

Processing Elements was able to deliver a sufficiently highcomputational power to allow real-time image processing when

other general-purpose processors weren't (at that time only 80286

and 80386 were commercially available).

The advantages offered by the first solution, such as an ad-hoc

design of both the processing paradigm and the overall system

architecture, are diminished by the necessity of managing thecomplete project, starting from the hardware level (design of the

ASICs) up to the design of the architecture, of the programming

language along with an optimizing compiler, and finally to the

development of applications using the specific computational

paradigm. Conversely the latter takes advantage of standard

development tools and environments but suffers from a less

specific instruction set and a less oriented system architecture.

In addition also the following technological aspects need to be

considered:


2/21

the fast technological improvements, which tend to reduce

the life time of the system;

the costs of the system design and engineering, which are

justified for productions based on large volumes only.

For these reasons the architectural solution currently under

evaluation on the ARGO vehicle is based on a standard 200 MHz

MMX Pentium processor.

The MMX Technology

MMX technology represents an enhancement of the Intel

processor family, adding instructions, registers, and data types

specifically designed for multimedia data processing. Namely

software performance are boosted exploiting a SIMD technique:

multiple data elements can be processed in parallel using a single

instruction. The new general-purpose instructions supported by

MMX technology perform arithmetic and logical operations on

multiple data elements packed into 64-bit quantities. These

instructions accelerate the performance of applications based on

compute-intensive algorithms that perform localized recurring

operations on small native data. More specifically in the

processing of gray level images, data is represented in 8 bit

quantities, hence an MMX instruction can operate on 8 pixels

simultaneously.

Basically the MMX extensions provide the programmers with the

following new features:


3/21

MMX Registers:

the MMX technology provides eight general-purpose 64-bit new

registers. MMX registers have been overlapped to the floating

point registers to assure the backward compatibility with the

existing software and specifically with the multitasking operating

systems[3].

Unfortunately this solution has two drawbacks:

theprogrammer is expected to not mix MMX instructions and

floating point code in any way, but is forced to use a specific

instruction (EMMS) at the end of every MMX enhanced

routine. The EMMS instruction empties the floating point tag

word, thus allowing the correct execution of floating point

operations;

frequent transitions between the MMX and floating-point

instructions may cause significant performance degradation.

MMX Data Types:

the MMX instructions can handle four different 64-bit data types:

8 bytes packed into one 64-bit quantity,

4 words packed into one 64-bit quantity,

2 double-words packed into one 64-bit quantity, or

1 quadword (64-bit).

This allows to process multiple data using a single instruction or

to directly manage 64-bit data.


4/21

MMX arithmetics:

the main innovation of the MMX technology consists in the two

different methods used to process the data:

saturation arithmetic and

wraparound mode.

Their difference depends on how the overflow or underflow

caused by mathematical operations is managed. In both cases

MMX instructions do not generate exceptions nor set flags, but in

wraparound mode, it results that overflow or underflow are

truncated and only the least significant part of the result isreturned; conversely, the saturation approach consists in setting

the result of an operation that overflows to the maximum value of

the range, as well as the result of an operation that underflows is

set to the minimum value. For example packed unsigned bytes for

results that overflow or underflow are saturated to 0x00 or to 0xFF

respectively.

The latter approach is very useful for grey-level image processing,

in fact, saturation brings grey value to pure black or pure white,

without allowing for an inversion as in the former approach.

MMX instructions:

MMX processors are featured by 57 new instructions, which may

be grouped into the following functional categories: arithmetic

instructions, comparison instructions, conversion instructions,

logical instructions, shift instructions, data transfer instructions,

and the EMMS instruction.


5/21

Automatic Vehicle Driving is a generic term used to address a

technique aimed at automating -enterely or in part- one or more

driving tasks. The automation of these tasks carries a large number

of economical and social benefits, such as:

a higher exploitation of the road network,

lower fuel and energy consumption,

reduction of personnel for people and goods mobility, and -of

course-

improved quality of mobility and

improved safety conditions compared to the current scenario.

The tasks that automatically driven vehicles are able to perform

range from the following:

the possibility to follow the road and keep within the right

lane,

maintaining a safe distance between vehicles,

regulating the vehicle's speed according to traffic conditions

and road characteristics,

moving across lanes in order to overtake vehicles and avoid

obstacles,

helping to find the correct and shortest route to a destination,

and

the movement and parking within urban environments.


6/21

Car Functionalities:

Manual Driving:

the system monitors the driver activity; in case of potentialdangerous situations the system warns the driver with acoustic and

optic signals

Supervised Driving:

the system warns the driver with acoustic and optic signals and, in

case of dangerous situations, takes the control of the vehicle inorder to keep it in a safe condition

Automatic Driving:

the system drives automatically, following the lane and localizing

obstacles on the path; it is able to perform lane changes

The Input System

Only passive sensors (cameras) are used on ARGO to sense the

surrounding environment, since they offer the possibility toacquire data in a non-invasive way, namely without altering the

environment. Because of the large number of vehicles that could

be moving simultaneously, this is a prominent advantage with

respect to invasive ways of perceiving the environment, which

could lead to an unacceptable pollution of the environment.


7/21

Figure 1: The left and right cameras

The vision system

The ARGO vehicle is equipped with a stereoscopic vision system

(figure 1) consisting of two synchronized cameras able to acquire

pairs of grey level images. The installed devices are small (3.2cm

X 3.2cm) low cost cameras featuring a 6.0 mm focal length and a

360 lines resolution which can receive the synchronism from an

external signal.

The cameras lie inside the car at the top corners of the windscreen,

so that the longitudinal distance between the two cameras is

maximum. The optical axes are parallel and, in order to handle

also non flat roads, part of the scene over the horizon is captured,even if the framing of a portion of the sky can be critical for the

image brightness: in case of high contrast the sensor may acquire

oversaturated images.

The acquisition system

Images are acquired by a PCI Matrox board, which is able to grab

three 768 X 288 pixel images simultaneously. The images are

directly stored into the main memory of the host computer thanks

to the use of DMA. The acquisition can be performed in real time,


8/21

at a 25Hz rate in case of full frames or at a 50Hz rate in case of

single field acquisition.

The output

The steering capabilities

The ARGO vehicle has autonomous steering capabilities: the

result of the processing (position of obstacles and geometry of the

road) is used to drive an actuator on the steering wheel. More

precisely, the output provided by the GOLD vision system, suchas the vehicle lateral offset, the vehicle yaw relative to the road

centerline and the upcoming road curvature, are combined to

determine the lane center at a given distance ahead of the vehicle.

The steering wheel is turned to head the vehicle at that lane center

point ahead of the vehicle. In addition, the information coming

from a speedometer will be integrated to handle also changes in

the vehicle speed.

Acoustic warnings

For debug purposes, the result of the processing is also fed to the

driver through a set of output devices installed on-board of the

vehicle. An acoustical device warns the driver in case dangerousconditions are detected, e. g. when the distance to the leading

vehicle is under a safety threshold or when the vehicle position

within the lane is not safe.


9/21

The visual feedback

Moreover, a visual feedback is supplied to the driver by displaying

the results both on an on-board monitor and on a led-based control

panel: the monitor presents the acquired left image with markershighlighting the lane markings as well as the position of the

eventual obstacles, while the leds encode the offset of the vehicle

with respect to the road center line.


10/21

Future Trends Will I still be driving in the near future?

The research carried out worldwide on intelligent

vehicles -after its first explorative stage- has now reached its

second stage: many different approaches have been evaluated

during these years and the most promising ones are being

engineerized, installed on prototype vehicles, and extensively

tested on the road.

However, a long period of exhaustive tests and refinement must

precede the availability of these systems on the general market,

and a fully automated highway system with intelligent vehicles

driving and exchanging information is not expected for a couple

of decades. For the time being, complete automation will be

restricted to special infrastructures such as industrial applications

or public transportation. Then, automatic vehicle technology will

be gradually extended to other key transportation areas such as the

transportation of goods, for example on expensive trucks, where

the cost of an autopilot is negligible with respect to the cost of the

vehicle itself and the service it provides. Finally, once technology

has been stabilised and after freezing the most promising solutions

and the best algorithms, a massive integration and a widespread

use of such systems will also take place with private vehicles, but

this will not happen for another two or more decades.

So, it's a bit too early to be looking for autonomous vehicles in

your automobile dealers' showrooms.


11/21


12/21

Lane Detection by

means of IPM

This section presents a possible solution to the problem oflane detection

in images acquired from a camera installed on a mobile vehicle which is

reduced to the detection of lane markings. In this case the a-priori

knowledge exploited by the IMP transform is the assumption of a flatroad

in front of the vehicle. The advantage offered by the use of the IPM is that

in the remapped image (see figure 2.c) the road markings width is almost

invariant within the whole image. This simplifies the following detection

steps and allows its implementation with a traditional pattern matching

technique on a SIMD system.


13/21

The basic assumption lane detection relies on is that road markings after

the IPM transform are represented by quasi-vertical constant width lines,

brighter than their surrounding region. Hence the first step of road

markings detection is a low-level processing aimed to detect the pixels that

have a higher brightness value than their horizontal neighbors at a given

distance. The following processing is in charge of the reconstruction of the

road geometry.

Figure 2: IPM applied to a road environment: (a) 3D representation of the

environment, (b) the acquired image, (c) the remapped image

Obstacle Detection bymeans of Stereo IPM

As mentioned in paragraph 3.2, when obstacle detection means the mere

localization of objects that can obstruct the vehicle's path without their

complete identification orrecognition, stereo IPM can be used in

conjunction with a geometrical model of the road in front of the vehicle[2].

Assuming the flatroadhypothesis introduced in the previous section, IPM


14/21

is performed using the same relations. This is of basic importance since in

a system aimed to both obstacle and lane detection the IPM transform can

be performed only once and its result can be shared by the two processes.

The flat road model is checked through a pixel-wise difference between

the two remapped images: in correspondence to a genericobstacle in front

of the vehicle, namely anything rising up from the road surface, the

difference image features sufficiently large clusters of non-zero pixels that

have a particular shape.

Due to the different angles of view of the stereo cameras, an ideal

homogeneous square obstacle produces two clusters of pixels with a

triangular shape in the difference image, in correspondence to its vertical

edges.

Unfortunately triangles found in real cases (see figure 4) are not so clearly

defined and often not clearly disjoint because of the texture, irregular

shape, and non-homogeneous color of real obstacles. Nevertheless clusters

of pixels having an almost triangular shape are anyway recognizable in the

difference image (see figure 4.e). The obstacle detection process is thus

based on the localization of these triangles.

Moreover, this process is complicated by the possible presence oftwoor

moreobstacles in front of the vehicle at the same time, thus producing

more than one pair of triangles, orpartially visibleobstacles, thus

producing a single triangle; a further processing step is thus needed in

order to classify triangles that belong to the same obstacle.


15/21

Vehicle Detection


16/21

The platooning functionality is based on the detection of the distance,

speed, and direction of the preceeding vehicle, which is localized and

tracked using a single monocular image sequence.

The vehicle detection algorithm is based on the following considerations: a

vehicle is generally symmetric, characterized by a rectangular bounding

box which satisfies specific aspect ratio constraints, and placed in a

specific region of the image. These features are used to identify vehicles in

the image in the following way: first an area of interest is identified on the

basis of road position and perspective constraints. This area is searched for

possible vertical symmetries; not only gray level symmetries are

considered, but vertical and horizontal edges symmetries as well, in order

to increase the search robustness. Once the symmetry position and width

has been detected, a new search begins, which is aimed at the detection of

the two bottom corners of a rectangular bounding box. Finally, the top

horizontal limit of the vehicle is searched for, and the preceeding vehicle

localized.

The tracking phase is performed through the maximization of the

correlation beetween the portion of the image cintained into the bounding

box of the previous frame (partially stretched and reduced to take into

account small size variations due to the increment and reduction of the

relative distance) and the new frame.

System calibration


17/21

Since the processing is based on stereo vision, camera calibration plays a

basic role for the success of the approach. It is divided in two steps.

Supervised calibration:

the first part of the calibration process is an interactive step: a grid with

known size (figure 1.a) has been painted onto the ground and two stereo

images (figure 1.b and 1.c) are captured and used for the calibration.

Thanks to an X-Window based graphical interface a user selects the

intersections of the grid lines using a mouse; these intersections represent a

small set of homologous points whose world coordinates are known to the

system; this mapping is used to compute the calibration parameters. Theset of homologous points is used to minimize different cost functions, such

as, the distance between each point and its neighbors and line parallelism.

This first step is intended to be performed only once when the orientation

of the cameras or the vehicle trim have changed. Since the set of

homologous points is small and their coordinates may be affected by

human imprecision, this calibration represents only a rough guess of the

parameters, and a further process is required.

Automatic parameters tuning:

after the supervised phase, the computed calibration parameters have to be

refined. Moreover small changes in the vision system setup or in the

vehicle trim require a periodic tuning of the calibration. For this purpose

an automatic tool has been developed [2]. Since this step is only a

refinement, a structured environment, such as the grid, is no more required

and a mere flat road in front of the vision system suffices. The parameters

tuning consists of an iterative procedure based on the application of the

IPM transform to stereo images (see the extension of IPM to stereo vision)

and takes about 20 seconds.


18/21

The Internet Connection

Video broadcast requirements

During the Tour across Italy, ARGO broadcasted a live video

stream to the Internet. To accomplish this target a permanent link

to the Internet was established.

The problem

A permanent link from a moving vehicle implies the use of mobile

telecommunications facilities, such as GSM modems.

Unfortunately the use of GSM modems for live video streaming

compels to face the following constraints:

low bit-rate band for transmission (usually 9600 bps, namely

a little more than 1 kbyte/s);

high variability of the bandwidth during movements;

The solution

In order to increase the throughput of the link the simultaneous

use of two GSM modems has been studied. Two different

techniques were conceived to exploit multiple phone lines:

MLPPP (Multi-Link Point to Point Protocol);


19/21

EQL (EQualizer Load-balancer);

Both share the underlying idea of splitting the network traffic

across multiple serial links. The former relies on the presence of

homogeneous serial links. Conversely the latter can handle also

links that feature different throughput and protocols. But the most

important capability for our application is that EQL dynamically

adapt the network traffic according to the number of working

serial links.

The implementation

Another personal computer was installed on ARGO; it wasequipped with a parallel port QuickcamColor camera and two

GSM modems able to work up to 9600 bps using the MNP10-EC

protocol. The computer runs the Linux operating system and a

specific application that grabs images from the Quickcam, convert

them in the jpeg format, and send them to our web server through

the two GSM modems exploiting EQL.


20/21

Pair of stereo cameras: it is used to detect the obstacles in front of

the car and is an input for the processor.

Stereo loudspeakers: during emergency gives warnings for a

person sitting in the car.

Hall effect speedometer: to detect the speed of the car.

Emergency pedal: incase the system fails and if someone is sittin

in the car the person can use the emergency pedal to stop the car.

Electric engine: is used to power up the car and also acts as a

supply for the processing system.

Control panel: to control the movement of

the car.

Tv monitor: to see the visuals taken from the camera.

Emergency joystick: this is also used during emergency.

Personel computer: it acts as a processing system for the car.

Inverter: the dc supply given from the electric engine is converted

to ac and then sent to the processing system.


21/21

Gsm antennas: are used to connect to the internet and locate cars

position through gps.

Internal cameras: to get the internal view of the car.

seminar info (autosaved)

Documents