1 1 low cost gnss and computer vision based data fusion solution for driverless vehicles marc...
TRANSCRIPT
![Page 1: 1 1 Low Cost GNSS and Computer Vision based data fusion solution for driverless vehicles Marc POLLINA pollina@m3systems.net TAXISAT PROJECT](https://reader033.vdocuments.us/reader033/viewer/2022042821/56649dab5503460f94a99d22/html5/thumbnails/1.jpg)
11
Low Cost GNSS and Computer Vision based data fusion solution
for driverless vehicles
Marc POLLINA
TAXISAT PROJECT
![Page 2: 1 1 Low Cost GNSS and Computer Vision based data fusion solution for driverless vehicles Marc POLLINA pollina@m3systems.net TAXISAT PROJECT](https://reader033.vdocuments.us/reader033/viewer/2022042821/56649dab5503460f94a99d22/html5/thumbnails/2.jpg)
22
Outline
• Importance of ITS• In-vehicle systems: Future Technologies• System Architecture• Results Analysis• Conclusions
![Page 3: 1 1 Low Cost GNSS and Computer Vision based data fusion solution for driverless vehicles Marc POLLINA pollina@m3systems.net TAXISAT PROJECT](https://reader033.vdocuments.us/reader033/viewer/2022042821/56649dab5503460f94a99d22/html5/thumbnails/3.jpg)
33
Importance of ITS
• The Global Market for ITS Technologies is estimated to grow to €50BN by 2020.
• Automotive Industry is one of the most innovative sectors– Active: Continuously monitor an aspect of the user, vehicle, environment or
transport network and alert the user to potential danger, or intervene with the driving task to avoid danger
– Passive: These are crash mitigation or minimisation technologies that act to enhance the safety of the driver or other road users by minimising the severity.
– Combined active and passive systems (CAPS): These systems monitor the environment, vehicle or driver for potential danger and then apply passive safety measures if a crash is deemed unavoidable
![Page 4: 1 1 Low Cost GNSS and Computer Vision based data fusion solution for driverless vehicles Marc POLLINA pollina@m3systems.net TAXISAT PROJECT](https://reader033.vdocuments.us/reader033/viewer/2022042821/56649dab5503460f94a99d22/html5/thumbnails/4.jpg)
44
Example of test case ( GUIDE Laboratory – Toulouse)Blue : GNSS , Green : reference ( PPK + high grade IMU)
GNSS Sensor in Urban Area
![Page 5: 1 1 Low Cost GNSS and Computer Vision based data fusion solution for driverless vehicles Marc POLLINA pollina@m3systems.net TAXISAT PROJECT](https://reader033.vdocuments.us/reader033/viewer/2022042821/56649dab5503460f94a99d22/html5/thumbnails/5.jpg)
55
Example of test case ( GUIDE Laboratory – Toulouse)Blue : GNSS , Green : reference ( PPK + high grade IMU)
GNSS Sensor in Urban Area
![Page 6: 1 1 Low Cost GNSS and Computer Vision based data fusion solution for driverless vehicles Marc POLLINA pollina@m3systems.net TAXISAT PROJECT](https://reader033.vdocuments.us/reader033/viewer/2022042821/56649dab5503460f94a99d22/html5/thumbnails/6.jpg)
66
• Sensor Fusion is essential : no sole positioning sensor covers all requirements and constraints
• Combination of computer vision, 3D Maps and GNSS technologies are fostering new solutions not only for driving assistance but for unmanned vehicles
Future Technologies
![Page 7: 1 1 Low Cost GNSS and Computer Vision based data fusion solution for driverless vehicles Marc POLLINA pollina@m3systems.net TAXISAT PROJECT](https://reader033.vdocuments.us/reader033/viewer/2022042821/56649dab5503460f94a99d22/html5/thumbnails/7.jpg)
77
• GNSS : new constellations & new frequencies
New GNSS satellite constellations, signals, and associated frequency diversity is stimulating innovations in user equipment design leading to improved capabilities of positioning
• 3D Maps : city mapping3D city mapping has the potential to revolutionize positioning in challenging urban areas. Adding height information to street maps can be used to aid GNSS positioning for land vehicle and pedestrian navigation.
• Computer vision: intelligent cameraThe major new navigation sensor of the next decade could well be the camera. Visual odometry, is a form of dead reckoning
Future Technologies
![Page 8: 1 1 Low Cost GNSS and Computer Vision based data fusion solution for driverless vehicles Marc POLLINA pollina@m3systems.net TAXISAT PROJECT](https://reader033.vdocuments.us/reader033/viewer/2022042821/56649dab5503460f94a99d22/html5/thumbnails/8.jpg)
88
Architecture
![Page 9: 1 1 Low Cost GNSS and Computer Vision based data fusion solution for driverless vehicles Marc POLLINA pollina@m3systems.net TAXISAT PROJECT](https://reader033.vdocuments.us/reader033/viewer/2022042821/56649dab5503460f94a99d22/html5/thumbnails/9.jpg)
99
Architecture
Traditional SensorsCost/Accuracy Trade off
Odometers for:-Wheels speed-Front Axle orientation
Gyro:-Optical-MEMS
![Page 10: 1 1 Low Cost GNSS and Computer Vision based data fusion solution for driverless vehicles Marc POLLINA pollina@m3systems.net TAXISAT PROJECT](https://reader033.vdocuments.us/reader033/viewer/2022042821/56649dab5503460f94a99d22/html5/thumbnails/10.jpg)
1010
Architecture
Position SensorsCost/Accuracy Trade off
Trimble bullet III: compact antenna- Low cost and good gain
LEA-6T : GPS/EGNOS receiver- Accurate, reliable
![Page 11: 1 1 Low Cost GNSS and Computer Vision based data fusion solution for driverless vehicles Marc POLLINA pollina@m3systems.net TAXISAT PROJECT](https://reader033.vdocuments.us/reader033/viewer/2022042821/56649dab5503460f94a99d22/html5/thumbnails/11.jpg)
1111
Architecture
Computer VisionCost/Accuracy Trade off
SLAMEnhancing performance level compared to usual INSTransversal displacements and estimations of velocity and orientationMatching between a live map of the scene structure and a new acquired image
FLEA 3, Point grey, stereo pair
FOLLOW THE LANE Improve security, reliability and 24/7 operation possibilityExtra feature derived from ADAS to assist continuously the car’s control loops
![Page 12: 1 1 Low Cost GNSS and Computer Vision based data fusion solution for driverless vehicles Marc POLLINA pollina@m3systems.net TAXISAT PROJECT](https://reader033.vdocuments.us/reader033/viewer/2022042821/56649dab5503460f94a99d22/html5/thumbnails/12.jpg)
1212
Architecture
EDAS Connection Module
Local server - Hosting the EDAS client software (EDAS server connection software)- Filtering routine
3G communication- Communication between the local server and the vehicle
![Page 13: 1 1 Low Cost GNSS and Computer Vision based data fusion solution for driverless vehicles Marc POLLINA pollina@m3systems.net TAXISAT PROJECT](https://reader033.vdocuments.us/reader033/viewer/2022042821/56649dab5503460f94a99d22/html5/thumbnails/13.jpg)
1313
• Tight Hybridization module composed of
• An Inertial Navigation System (INS) which integrates the gyrometer/odometer data (100Hz)
• A Navigation filter which updates and corrects the INS according to the measurements from the Vision or GNSS modules when available and valid
• 3 platforms -> Time synchronisation of measurement required
Architecture
![Page 14: 1 1 Low Cost GNSS and Computer Vision based data fusion solution for driverless vehicles Marc POLLINA pollina@m3systems.net TAXISAT PROJECT](https://reader033.vdocuments.us/reader033/viewer/2022042821/56649dab5503460f94a99d22/html5/thumbnails/14.jpg)
1414
Real Distance / Location
Information in pixels
.
Mapping of real world information
to 2D image
Real World
Captured image
Info
rmat
ion
Rat
io
Camera/Vehicle position and Orientation in Real Time
(lat,lon)
(x0,y0) - (lat0,lon0)
(x1,y1) - (lat1,lon1) (x2,y2) - (lat2,lon2)
(x3,y3) - (lat3,lon3)(x4,y4) - (lat4,lon4)
No Geo-Referenced informationA-priori Unknown scenario
.Captured image
Known relationDepth Information
Measured Information: - GNSS Position Device- Orientation by Sensors
Measured Reference
- Future GIS Hibridization Capabilities- Precise Map Building - Usable information for control loops: predictive
Real Time Scenario GeoPositioning
![Page 15: 1 1 Low Cost GNSS and Computer Vision based data fusion solution for driverless vehicles Marc POLLINA pollina@m3systems.net TAXISAT PROJECT](https://reader033.vdocuments.us/reader033/viewer/2022042821/56649dab5503460f94a99d22/html5/thumbnails/15.jpg)
1515
Follow the Lane•Tx: (lateral) translation in x•Vx: linear velocity in x•Wx: width of the lane•dWx: linear velocity of the change of width•Self Assesment•Active Control of Light Conditions
Vision Sensor: FtL results
![Page 16: 1 1 Low Cost GNSS and Computer Vision based data fusion solution for driverless vehicles Marc POLLINA pollina@m3systems.net TAXISAT PROJECT](https://reader033.vdocuments.us/reader033/viewer/2022042821/56649dab5503460f94a99d22/html5/thumbnails/16.jpg)
1616
SLAM (Simultaneous Location & Mapping ) : Visual odometry + Mapping•Visual odometry: Estimation of the EgoMotion (6D camera/vehicle pose) in real time •Real time 3D scene map generation
Vision Sensor: SLAM
![Page 17: 1 1 Low Cost GNSS and Computer Vision based data fusion solution for driverless vehicles Marc POLLINA pollina@m3systems.net TAXISAT PROJECT](https://reader033.vdocuments.us/reader033/viewer/2022042821/56649dab5503460f94a99d22/html5/thumbnails/17.jpg)
1717
FtL Evaluation•Recorded video sequences: 337 minutes
SLAM Module - 2 step evaluation•Laboratory computer using the KITTI odometry evaluation dataset with ground truth
– 22 sequences of images recorded with a stereo pair of cameras embedded in a car.
•Evaluation in San Sebastian – Running predifined paths
Evaluation
![Page 18: 1 1 Low Cost GNSS and Computer Vision based data fusion solution for driverless vehicles Marc POLLINA pollina@m3systems.net TAXISAT PROJECT](https://reader033.vdocuments.us/reader033/viewer/2022042821/56649dab5503460f94a99d22/html5/thumbnails/18.jpg)
1818
Accuracy and precision of the odometry•Translation error max 0.29% •rotational error 0.0122 deg/m •Runtime 9.0 ms
Evaluation
![Page 19: 1 1 Low Cost GNSS and Computer Vision based data fusion solution for driverless vehicles Marc POLLINA pollina@m3systems.net TAXISAT PROJECT](https://reader033.vdocuments.us/reader033/viewer/2022042821/56649dab5503460f94a99d22/html5/thumbnails/19.jpg)
1919
– Computer Vision as one key sensor for enabling autonomous driving– Enable autonomous or semi-autonomous driving of your vehicle
even in situations when GNSS Signal is unreliable or not available at all (i.e. indoors, in tunnels, under dense vegetation, etc.).
– Know the position of your vehicle even when no GNSS reception is available.
– Improve position precision and reliability considerably when compared to GNSS-only solutions
– Improve availability compared to GNSS solutions. SLAM is possible 24/7 while GNSS reception might be unreliable or not available at all for several minutes
– Create a map in Real Time and Geo-locate all the point of an image in Real Time
Conclusions