a fog computing infrastructure for autonomous driving in

A Fog Computing Infrastructure for

Autonomous Driving in Urban Environments

Modena Automotive Smart Area (MASA)

Marko Bertogna

University of Modena, Italy

marko.bertogna@unimore.it

http://hipert.unimore.it/

Research on High-Performance Real-Time Systems

~30 people

– 4 faculties

– 10 post docs

– 7 PhD students

– 3 tech/admin

– Multiple students…

Ongoing EU projects:

Past EU projects:

Industrial collaborations:

2

HiPeRT Lab

>3MEuro funding

http://hipert.unimore.it/

Just started:

PRYSTINE,

SECREDAS

Copyright of the University of Modena - HiPeRT Lab

The Motor Valley

3Copyright of the University of Modena - HiPeRT Lab

Copyright of the University of Modena - HiPeRT Lab 4

Copyright of the University of Modena - HiPeRT Lab 5

What is MASA?MASA is a living lab created to develop, test and

validate automated and connected vehicles.

MASA MAIN ASSETS:

Simulator Proving ground Public road

Advanced

training

Research & product

development

Support for

experimental test

implementation

Public-private partnership that offers OEMs,

Tier 1 and Tier 2 a complete environment to

develop and validate solutions for Cooperative

Connected Automated Mobility

Modena racetrack

Copyright of the University of Modena - HiPeRT Lab 7

Copyright of the University of Modena - HiPeRT Lab 8

Urban Area

▶ 4sq km urban area with a wide variety of real scenarios

▶ Roundabouts, crossroads, R/L turns, parking areas

▶ Dedicated road side equipment installed (Cameras, fog

nodes, Fiber, parking sensor…)

▶ Test new applications in real-life situations

▶ smart traffic lights, parking and environment sensors,

LTE dedicated antennas, cameras integrated on the

edge computing platform

Which dedicated services are available?MASA Public Road activities

Urban area

EU project CLASS:Edge Cloud Computation: A Highly Distributed Software Architecture for Big Data AnalyticS

Funded by H2020-RIA ICT-16-2017 GA n°780622

Duration: 36 month (2018-2020)

Budget: 3.900.803

European Funding: 100%

Fog computing infrastructure for Urban Autonomous driving: a public-private partnership within Modena Automotive Smart Area (MASA)

• Accurate awareness of road users and obstacles in real-time

• Distributed traffic monitoring and enforcement in metropolitan areas

• Enabling technology for advanced AD applications in urban settings

V2I obstacle detection, Coordinated intersection crossing, Dynamic traffic signalling, Green routes forpublic vehicles, Smart parking: free lot detection and valet parking

Real-time fog computing system

• Hundreds of smart cameras installed

• Cameras are locally connected to a

high-performance embedded board

• Edge nodes elaborate video streams detecting road users in real-time

• Elaborated information is sent in V2I

to vehicles for enhanced perception to L3/L4 autonomous driving

• Edge nodes are connected to local

servers receiving data at block level (a.k.a. Fog nodes)

• Fog nodes are fiber connected to

main control center

Control Room

Copyright of the University of Modena - HiPeRT Lab 11

Car as a sensor

Copyright of the University of Modena - HiPeRT Lab 12

Precise Localization and Mapping

Copyright of the University of Modena - HiPeRT Lab 13

Real-time Urban Awareness

In-vehicle sensors

Infrastructure sensors Real-time detection Low-latency V2X communication

Autonomous vehiclesTraffic enforcementPublic authoritiesData analytics

14Copyright of the University of Modena - HiPeRT Lab

Predictable communication infrastructure- 4G/LTE 5G

Low-latency V2X Infrastructure

Vehicle-to-vehicle

V2V

Vehicle-to-infrastructure

V2I

eNB

S-PGW PCRF

Athonet 4G+ Mobile Core

MME Local HSS

Local apps

eNB

IP

N e t w o r kApplication

layer

Copyright of the University of Modena - HiPeRT Lab 15

Multiple infrastructure cameras detect road users and obstacles

– vehicles, buses, trucks, pedestrians, bicycles, etc.

A consistent representation of RU’s is sent to L3/L4 vehicles (V2I) in real-time

Edge-side: Real-time road user detection

To cloud infrastructure

Camera-to-car < 100ms !

• V2I sensing system for harsh

urban environment• Redundant and robust perception

• Safer obstacle detectionCopyright of the University of Modena - HiPeRT Lab 16

Distributed infrastructure for

real-time urban awareness

Distributed intelligence system for real-time urban awareness

Smart city map updated every second

Tight reaction time to urban hazards

Traffic/parking monitoring & enforcement

Emergency vehicle routing

Cloud side: Advanced control center

Copyright of the University of Modena - HiPeRT Lab 17

Highly valuable dataset for smart urban mobility enforcement and understanding

Highly detailed and accurate mobility data

Traffic/mobility analysis and prediction

Critical scenario identification

Road user behaviour understanding and prediction

– Improved algorithms for autonomous driving

Data anonymization for GDPR compliance

– Implemented at source/edge level

Big data analytics stack developed with IBM, UNIMORE, ATOS, Barcelona

Supercomputing Center

Copyright of the University of Modena - HiPeRT Lab 18

Use cases – Data analytics

MASA Real-Time Smart Cameras

MASA platform

Copyright of the University of Modena - HiPeRT Lab 21

CLASS Software infrastructure

Data Analytics Platform

Programming Models

OpenWhisk

Invoker

COMPSs + DataClay

Connectors

RotterdamCaaS API

Kubernetes

Docker

Linux

Edge

Co

mp

ute

Co

nti

nu

um

Copyright of the University of Modena - HiPeRT Lab 22

Traditional Smart City flow: high latency

IP cameras: >200ms

YOLO v3 with tensor cores on

TitanV: 14ms

Private 4G network:

16ms latency

Metadata

Copyright of the University of Modena - HiPeRT Lab 23

CLASS Smart City flow: under 100 ms

Smart cameras

YOLO v2 with

TensorRT on TX2: 56ms

Private 4G

network:16ms latency

<10ms

Train a

PredictiveModel of the

city using FogNodes and City

Cloud

Metadata

Copyright of the University of Modena - HiPeRT Lab 24

Example from yesterday’s feed

Copyright of the University of Modena - HiPeRT Lab 25

Small datagram using UDP

Sender ID: 1 Byte

Timestamp: 1 Byte (consider 1/200 of a second as an incremental modular

value)

# objects that will be sent: 1 Byte

– Latitude: 4 Bytes floating point precision

– Longitude: 4 Bytes floating point precision, should be enough for our needs

– Speed: 1 Byte (unsigned) 0 to 128 KM/h with 0,5 KM/h increment

– Orientation: 1 Byte (unsigned) 0 to 360 with 1,40625 degrees increment

– Category: half Byte with other half for parity

For 50 objects to send datagram of 553 Bytes = about half a KB

26

Data exchange

Copyright of the University of Modena - HiPeRT Lab

Hybrid edge/fog DNN

Same layers

Edge result

Fog/Cloud result

Copyright of the University of Modena - HiPeRT Lab 27

Open source simulator for transportation

Simulates large scale artificial agents, with daily transportation routines and other behavioural factors.

Easy interfaced to mapping systems (e.g. OpenStreet Maps)

Useful to simulate MASA traffic flow, V2X interactions and emerging behaviour before the actual real world deployment

Copyright of the University of Modena - HiPeRT Lab 28

MATSim: Multi-Agent Transport simulator

Misbehaving drivers modelled using ISTAT (Italian National Institute of Statistics) data– Car Accident frequency

– Average driver behaviours in traffic situations– Traffic flow recovery time after accidents

MASA historical traffic analyses, forecasts and driver interactions by interfacing MATSim to Visum– Extract drivers’ transportation routines as aggregate data provided by the city council

900 Agents, represented as “independent traffic flows”

Each agent has its own routine and behavior

One way streets, traffic lights, yields, parking spaces as in the MASA

Contingencies:

○ Accidents

○ Misbehaving drivers

○ Emergency vehicles

Copyright of the University of Modena - HiPeRT Lab 29

MATSim on MASA

Not only cars..

Copyright of the University of Modena - HiPeRT Lab 30

Confidential, all rights reserved

Confidential, all rights reserved

Control Unit: NVIDIA Jetson Xavier

Chassis: 6 motor wheels (independent)

able to climb over curbs

Speed: up to 6 km/h in pedestrian areas

up to 20 km/h in bike lane

Range: > 20 km

Load: Max Volume 50x50x45 cm

Max load 50 kg

Autonomous driving: localization and mapping, object detection, obstacle avoidance,

human overtaking in case of issues

Connectivity: Smart city data, destination/route request, Smart buildings

systems

LIFETOUCH MOVEO: Characteristics

Thank you!

Marko Bertogna

marko.bertogna@unimore.ithttp://hipert.unimore.it

Research Education

Competition

WP3, MODENA

Chassis Design Sensor Integration

Software System Architecture Cloud-Based Simulation Tool

GPU accelerated

libraries

Vehicle and

environment

models in GazeboROSbag dataScenario Sim

F1/10 racing platform

With simple IKEA style instructions at http://f1tenth.org

Fastware

Approximately $2,700

2 times per year

Co-located with major conferences around the world

Each time more challenging!

PAST: CPSWeek @Porto, POR, ESweek @Turin, IT

NEXT: CPSWeek @Montreal, CAN (April '19)

Get Involved !