towards perception integrity ? evidential road detection ...proposal we propose to rely on an...
TRANSCRIPT
![Page 1: Towards perception integrity ? Evidential road detection ...Proposal We propose to rely on an explicit road detection step, at the point level, to generate road grids from LIDAR scans](https://reader035.vdocuments.us/reader035/viewer/2022062602/5e8ac1a58227f03de443d31d/html5/thumbnails/1.jpg)
Edouard CAPELLIER - Franck DAVOINE – Véronique CHERFAOUI – You LI
November 4th 2019, PPNIV – IROS 2019 Workshop, Macau
Transformation-adversarial network for road detection in LIDAR rings,
and model-free evidential road grid mapping
![Page 2: Towards perception integrity ? Evidential road detection ...Proposal We propose to rely on an explicit road detection step, at the point level, to generate road grids from LIDAR scans](https://reader035.vdocuments.us/reader035/viewer/2022062602/5e8ac1a58227f03de443d31d/html5/thumbnails/2.jpg)
2
Rationale (I)
➢Raw point-clouds need to be processed into significant representations
➢before being used by an autonomous vehicle
➢In mobile robotics, it is common to convert LIDAR scans into occupancy grids
➢Occupancy grids are 2D maps of the
➢environment, splitted into regular cells
➢Each cell is either be occupied
➢(presence of obstacles), or free (no
➢obstacle: the robot can navigate).Example of occupancy grid obtained from a 3D LIDAR
![Page 3: Towards perception integrity ? Evidential road detection ...Proposal We propose to rely on an explicit road detection step, at the point level, to generate road grids from LIDAR scans](https://reader035.vdocuments.us/reader035/viewer/2022062602/5e8ac1a58227f03de443d31d/html5/thumbnails/3.jpg)
3
Rationale (II)
➢Most of the time, ad-hoc parameters or strong geometrical assumptions
➢are used in the ground detection and classification steps (e.g.: thresholding,
➢ray tracing, flat-ground assumption)
-> Lack of flexibility in complex or non-typical areas
➢The ground is a semantically poor concept: it is composed of areas that are
➢drivable (road) and areas that are not drivable (sidewalk, grass,…)
-> Need to rely on an explicit road detection step in the context of AD
![Page 4: Towards perception integrity ? Evidential road detection ...Proposal We propose to rely on an explicit road detection step, at the point level, to generate road grids from LIDAR scans](https://reader035.vdocuments.us/reader035/viewer/2022062602/5e8ac1a58227f03de443d31d/html5/thumbnails/4.jpg)
4
Proposal
➢We propose to rely on an explicit road detection step, at the point level, to
➢generate road grids from LIDAR scans
➢A deep-learning approach was investigated, so as not to rely on strong
➢assumptions nor ad-hoc parameters
➢We rely on the evidential framework, in
➢order to properly represent the fact that a
➢cell either belongs to the road, to an
➢obstacle, or is in an unknown state
Example of road detection result
![Page 5: Towards perception integrity ? Evidential road detection ...Proposal We propose to rely on an explicit road detection step, at the point level, to generate road grids from LIDAR scans](https://reader035.vdocuments.us/reader035/viewer/2022062602/5e8ac1a58227f03de443d31d/html5/thumbnails/5.jpg)
5
What is the evidential framework? Why using it ? (I)
➢Let be the frame of discernment used to model our problem.
➢R corresponds to the fact that a LIDAR point / grid cell belongs to the road, and
➢¬R that it does not
➢The theory of belief functions reasons on and uses
➢the Dempster-Shafer operator to fuse independent information sources
➢ indicates that and a point/cell is in an unknown state
➢Probabilistic grids usually need to explicitly track the transitions from an
➢unobserved to an observed state for advanced functionalities (cf. CMCDOT)
![Page 6: Towards perception integrity ? Evidential road detection ...Proposal We propose to rely on an explicit road detection step, at the point level, to generate road grids from LIDAR scans](https://reader035.vdocuments.us/reader035/viewer/2022062602/5e8ac1a58227f03de443d31d/html5/thumbnails/6.jpg)
6
➢A deep-learning architecture for road detection in LIDAR scans had to be chosen
➢We chose to rely on a network inspired by PointNet, for a first proof of concept
➢PointNet processes raw point-clouds, and relies on a solid mathematical theorem
PointNet: machine learning on raw point-clouds
General PointNet architecture
![Page 7: Towards perception integrity ? Evidential road detection ...Proposal We propose to rely on an explicit road detection step, at the point level, to generate road grids from LIDAR scans](https://reader035.vdocuments.us/reader035/viewer/2022062602/5e8ac1a58227f03de443d31d/html5/thumbnails/7.jpg)
7
➢Previous studies report that PointNet-like networks struggle with large-scale,
➢and sparse point-clouds (typically: LIDAR scans)
What PointNet lacks for our problem
➢Evidential mass values have to be generated from the classification results in a
➢significant manner
➢We propose architectural
➢refinements to address those
➢limitations
A sparse LIDAR scan
![Page 8: Towards perception integrity ? Evidential road detection ...Proposal We propose to rely on an explicit road detection step, at the point level, to generate road grids from LIDAR scans](https://reader035.vdocuments.us/reader035/viewer/2022062602/5e8ac1a58227f03de443d31d/html5/thumbnails/8.jpg)
➢Let a binary GLR classifier predicting the probability p(x) that an input x belongs
➢to the 𝜃 class, and 𝜎 the Sigmoid function.
Evidential theory and generalized logistic regression (GLR) classifiers
![Page 9: Towards perception integrity ? Evidential road detection ...Proposal We propose to rely on an explicit road detection step, at the point level, to generate road grids from LIDAR scans](https://reader035.vdocuments.us/reader035/viewer/2022062602/5e8ac1a58227f03de443d31d/html5/thumbnails/9.jpg)
➢The values still have to be chosen. A cautious choice is to maximize the mass
➢values on the unknown state. This is done by solving the following minimization
➢problem
➢This would require a post-processing step. Doing it on the training data is an
➢arbitrary choice
➢If the final layer of a neural network implements Instance, applying L2
➢regularization gives that lead to cautious evidential mass functions
The Instance Normalization trick
![Page 10: Towards perception integrity ? Evidential road detection ...Proposal We propose to rely on an explicit road detection step, at the point level, to generate road grids from LIDAR scans](https://reader035.vdocuments.us/reader035/viewer/2022062602/5e8ac1a58227f03de443d31d/html5/thumbnails/10.jpg)
10
➢Instead of relying on a PointNet that extract a global feature at the scan level, we propose to perform the road detection at the ring-level
➢Lidar rings are usually dense, which is likely to facilitate the road detection
Ring-level road detection
➢Yet: Lidar rings are acquired at very varying
➢distance.
➢So as to perform road Detection in any LIDAR
➢ring, an homothety rescaling factor can be
➢used to realign the LIDAR rings togetherLIDAR points colored according to their ring ID
![Page 11: Towards perception integrity ? Evidential road detection ...Proposal We propose to rely on an explicit road detection step, at the point level, to generate road grids from LIDAR scans](https://reader035.vdocuments.us/reader035/viewer/2022062602/5e8ac1a58227f03de443d31d/html5/thumbnails/11.jpg)
11
Ring-level pointnet with homothety rescaling for road detection
➢ An additional H-Net predicts an homothety rescaling factor➢ The network predicts the ID of the ring that it is processing. This information is
used in the training, to supervise the predicted rescaling factors➢ Instance-Normalization is added at the end of the network, to facilitate the
generation of evidential mass functions
![Page 12: Towards perception integrity ? Evidential road detection ...Proposal We propose to rely on an explicit road detection step, at the point level, to generate road grids from LIDAR scans](https://reader035.vdocuments.us/reader035/viewer/2022062602/5e8ac1a58227f03de443d31d/html5/thumbnails/12.jpg)
➢The system is trained under the assumption that it is hard to predict the ID➢of rings that are properly realigned together, and share similar dimensions
Transformation-adversarial training
![Page 13: Towards perception integrity ? Evidential road detection ...Proposal We propose to rely on an explicit road detection step, at the point level, to generate road grids from LIDAR scans](https://reader035.vdocuments.us/reader035/viewer/2022062602/5e8ac1a58227f03de443d31d/html5/thumbnails/13.jpg)
13
Training data collection and labelling
Data collection vehicle: front view Data collection vehicle: back view –Velodyne VLP32C and GNSS receiver
![Page 14: Towards perception integrity ? Evidential road detection ...Proposal We propose to rely on an explicit road detection step, at the point level, to generate road grids from LIDAR scans](https://reader035.vdocuments.us/reader035/viewer/2022062602/5e8ac1a58227f03de443d31d/html5/thumbnails/14.jpg)
14
Training data collection and labelling
➢ 2334 LIDAR scans sere recorded in Guyancourt, France, and automatically labelled from a lane-level map
➢A classical Gaussian error model is used to generate soft-labels for each point
Ground detection and map skeleton Automatically labelled LIDAR scan
![Page 15: Towards perception integrity ? Evidential road detection ...Proposal We propose to rely on an explicit road detection step, at the point level, to generate road grids from LIDAR scans](https://reader035.vdocuments.us/reader035/viewer/2022062602/5e8ac1a58227f03de443d31d/html5/thumbnails/15.jpg)
15
Results on the validation set
➢We report the results on a validation set composed of 30% percent of the labelled scans
➢ The validation set is composed of the first and last 15% of the sequence
➢We compare our network with regular PointNets trained on either scans or ring. All the shared hyperparameters have the same values among the threeapproaches
![Page 16: Towards perception integrity ? Evidential road detection ...Proposal We propose to rely on an explicit road detection step, at the point level, to generate road grids from LIDAR scans](https://reader035.vdocuments.us/reader035/viewer/2022062602/5e8ac1a58227f03de443d31d/html5/thumbnails/16.jpg)
16
➢ A grid can be generated by projecting theevidential mass values at the point level into thexy-plane.
➢ The road detection results can be accumulatedover time to densify the grid
➢ An evidential decay is used to handle movingobjects, and outdated observations:
Utilization in an evidential grid mapping framework (I)
Evidential grid mapping algorithm from the proposed neural network
![Page 17: Towards perception integrity ? Evidential road detection ...Proposal We propose to rely on an explicit road detection step, at the point level, to generate road grids from LIDAR scans](https://reader035.vdocuments.us/reader035/viewer/2022062602/5e8ac1a58227f03de443d31d/html5/thumbnails/17.jpg)
17
Utilization in an evidential grid mapping framework (II)
Mass values for LIDAR points and grid cells
![Page 18: Towards perception integrity ? Evidential road detection ...Proposal We propose to rely on an explicit road detection step, at the point level, to generate road grids from LIDAR scans](https://reader035.vdocuments.us/reader035/viewer/2022062602/5e8ac1a58227f03de443d31d/html5/thumbnails/18.jpg)
18
Utilization in an evidential grid mapping framework (III)
![Page 19: Towards perception integrity ? Evidential road detection ...Proposal We propose to rely on an explicit road detection step, at the point level, to generate road grids from LIDAR scans](https://reader035.vdocuments.us/reader035/viewer/2022062602/5e8ac1a58227f03de443d31d/html5/thumbnails/19.jpg)
19
Summary
➢We proposed a first grid mapping framework, that fuses road detection results
➢Our system follows the theory of belief function, which allows it to quantifythe amount of knowledge for each LIDAR point and grid cell
But:
➢ We lack proper evaluation on a manually labelled and representative test set
➢ The grid mapping algorithm is sensitive to moving objects, and does not run in real time, mainly due to the inference time of the network
-> Those points have been addressed in an upcoming paper
![Page 20: Towards perception integrity ? Evidential road detection ...Proposal We propose to rely on an explicit road detection step, at the point level, to generate road grids from LIDAR scans](https://reader035.vdocuments.us/reader035/viewer/2022062602/5e8ac1a58227f03de443d31d/html5/thumbnails/20.jpg)
20
Coeverage of the new training dataset
![Page 21: Towards perception integrity ? Evidential road detection ...Proposal We propose to rely on an explicit road detection step, at the point level, to generate road grids from LIDAR scans](https://reader035.vdocuments.us/reader035/viewer/2022062602/5e8ac1a58227f03de443d31d/html5/thumbnails/21.jpg)
21
Manually labelled test dataset
![Page 22: Towards perception integrity ? Evidential road detection ...Proposal We propose to rely on an explicit road detection step, at the point level, to generate road grids from LIDAR scans](https://reader035.vdocuments.us/reader035/viewer/2022062602/5e8ac1a58227f03de443d31d/html5/thumbnails/22.jpg)
22
Evidential road surface mapping and object detection
![Page 23: Towards perception integrity ? Evidential road detection ...Proposal We propose to rely on an explicit road detection step, at the point level, to generate road grids from LIDAR scans](https://reader035.vdocuments.us/reader035/viewer/2022062602/5e8ac1a58227f03de443d31d/html5/thumbnails/23.jpg)
23
Thank you