covariance estimation of gps-lidar sensor fusion for...
TRANSCRIPT
![Page 1: Covariance Estimation of GPS-LiDAR Sensor Fusion for UAVsweb.stanford.edu/.../S03.Akshay_Shetty.pdf · Akshay Shetty and Grace Xingxin Gao SCPNT, November 2017. University of Illinois](https://reader034.vdocuments.us/reader034/viewer/2022050315/5f7786b29135f93d797600b3/html5/thumbnails/1.jpg)
University of Illinois at Urbana-Champaign
GPS-LiDAR Sensor Fusion
Aided by 3D City Models for UAVs
Akshay Shetty and Grace Xingxin Gao
SCPNT, November 2017
![Page 2: Covariance Estimation of GPS-LiDAR Sensor Fusion for UAVsweb.stanford.edu/.../S03.Akshay_Shetty.pdf · Akshay Shetty and Grace Xingxin Gao SCPNT, November 2017. University of Illinois](https://reader034.vdocuments.us/reader034/viewer/2022050315/5f7786b29135f93d797600b3/html5/thumbnails/2.jpg)
University of Illinois at Urbana-Champaign
1
1
Positioning in Urban Areas
• GPS signals blocked or reflected
• Additional sensors: LiDAR, cameras, etc
![Page 3: Covariance Estimation of GPS-LiDAR Sensor Fusion for UAVsweb.stanford.edu/.../S03.Akshay_Shetty.pdf · Akshay Shetty and Grace Xingxin Gao SCPNT, November 2017. University of Illinois](https://reader034.vdocuments.us/reader034/viewer/2022050315/5f7786b29135f93d797600b3/html5/thumbnails/3.jpg)
University of Illinois at Urbana-Champaign
2
2
LiDAR State Estimation Challenge
• Surrounding features affect accuracy
• Need to characterize covariance accordingly
Start
End
[Google Earth][https://github.com/ethz-asl/ethzasl_icp_mapping]
![Page 4: Covariance Estimation of GPS-LiDAR Sensor Fusion for UAVsweb.stanford.edu/.../S03.Akshay_Shetty.pdf · Akshay Shetty and Grace Xingxin Gao SCPNT, November 2017. University of Illinois](https://reader034.vdocuments.us/reader034/viewer/2022050315/5f7786b29135f93d797600b3/html5/thumbnails/4.jpg)
University of Illinois at Urbana-Champaign
3
3
State Estimation Covariance
Adequate features Poor features Lack of features
[Google Earth] [Google Earth][Google Earth]
![Page 5: Covariance Estimation of GPS-LiDAR Sensor Fusion for UAVsweb.stanford.edu/.../S03.Akshay_Shetty.pdf · Akshay Shetty and Grace Xingxin Gao SCPNT, November 2017. University of Illinois](https://reader034.vdocuments.us/reader034/viewer/2022050315/5f7786b29135f93d797600b3/html5/thumbnails/5.jpg)
University of Illinois at Urbana-Champaign
4
4
Approach
• Deep sensor fusion
− Characterize LiDAR-based position covariance based
on features
• Eliminate NLOS satellites
− Use 3D city model to detect and eliminate NLOS GPS
satellites
![Page 6: Covariance Estimation of GPS-LiDAR Sensor Fusion for UAVsweb.stanford.edu/.../S03.Akshay_Shetty.pdf · Akshay Shetty and Grace Xingxin Gao SCPNT, November 2017. University of Illinois](https://reader034.vdocuments.us/reader034/viewer/2022050315/5f7786b29135f93d797600b3/html5/thumbnails/6.jpg)
University of Illinois at Urbana-Champaign
5
5
Outline
• Approach:
− Overall Architecture
− 3D City Model
− LiDAR-based State Estimation and Covariance
− GPS Measurement Model
• Experimental Setup and Results
• Summary
• Future Work: Deep Learning
![Page 7: Covariance Estimation of GPS-LiDAR Sensor Fusion for UAVsweb.stanford.edu/.../S03.Akshay_Shetty.pdf · Akshay Shetty and Grace Xingxin Gao SCPNT, November 2017. University of Illinois](https://reader034.vdocuments.us/reader034/viewer/2022050315/5f7786b29135f93d797600b3/html5/thumbnails/7.jpg)
University of Illinois at Urbana-Champaign
6
6
Overall Architecture
![Page 8: Covariance Estimation of GPS-LiDAR Sensor Fusion for UAVsweb.stanford.edu/.../S03.Akshay_Shetty.pdf · Akshay Shetty and Grace Xingxin Gao SCPNT, November 2017. University of Illinois](https://reader034.vdocuments.us/reader034/viewer/2022050315/5f7786b29135f93d797600b3/html5/thumbnails/8.jpg)
University of Illinois at Urbana-Champaign
7
7
3D City Model
• Illinois Geospatial Data Clearinghouse provides
top-view point cloud [https://clearinghouse.isgs.illinois.edu]
• OpenStreetMap provides building footprint
information [www.openstreetmap.com]
Building wall
information from
OpenStreetMap
Top-view point
cloud from
geospatial data
![Page 9: Covariance Estimation of GPS-LiDAR Sensor Fusion for UAVsweb.stanford.edu/.../S03.Akshay_Shetty.pdf · Akshay Shetty and Grace Xingxin Gao SCPNT, November 2017. University of Illinois](https://reader034.vdocuments.us/reader034/viewer/2022050315/5f7786b29135f93d797600b3/html5/thumbnails/9.jpg)
University of Illinois at Urbana-Champaign
8
8
LiDAR Odometry
• Use Iterative Closest Point (ICP) algorithm
• Match consecutive point clouds to estimate
incremental motion
Reference Point Cloud
Input Point Cloud
ICP
![Page 10: Covariance Estimation of GPS-LiDAR Sensor Fusion for UAVsweb.stanford.edu/.../S03.Akshay_Shetty.pdf · Akshay Shetty and Grace Xingxin Gao SCPNT, November 2017. University of Illinois](https://reader034.vdocuments.us/reader034/viewer/2022050315/5f7786b29135f93d797600b3/html5/thumbnails/10.jpg)
University of Illinois at Urbana-Champaign
9
9
LiDAR – 3D City Model
• Use ICP algorithm
• Match LiDAR point cloud with 3D city model
Before Matching
After Matching
ICP
![Page 11: Covariance Estimation of GPS-LiDAR Sensor Fusion for UAVsweb.stanford.edu/.../S03.Akshay_Shetty.pdf · Akshay Shetty and Grace Xingxin Gao SCPNT, November 2017. University of Illinois](https://reader034.vdocuments.us/reader034/viewer/2022050315/5f7786b29135f93d797600b3/html5/thumbnails/11.jpg)
University of Illinois at Urbana-Champaign
10
10
LiDAR – 3D City Model
True Position
Initial Positions
Final Positions
Feature
distribution
Position
accuracy
Adequate High
Poor Low
[Google Earth][Google Earth]
[Google Earth] [Google Earth]
![Page 12: Covariance Estimation of GPS-LiDAR Sensor Fusion for UAVsweb.stanford.edu/.../S03.Akshay_Shetty.pdf · Akshay Shetty and Grace Xingxin Gao SCPNT, November 2017. University of Illinois](https://reader034.vdocuments.us/reader034/viewer/2022050315/5f7786b29135f93d797600b3/html5/thumbnails/12.jpg)
University of Illinois at Urbana-Champaign
11
11
LiDAR Point Cloud Features
• LiDAR-based position covariance as function of
features
• Extract feature points based on curvature values [Zhang et al., 2014]
LiDAR Point Cloud
Surface Points
Edge Points
![Page 13: Covariance Estimation of GPS-LiDAR Sensor Fusion for UAVsweb.stanford.edu/.../S03.Akshay_Shetty.pdf · Akshay Shetty and Grace Xingxin Gao SCPNT, November 2017. University of Illinois](https://reader034.vdocuments.us/reader034/viewer/2022050315/5f7786b29135f93d797600b3/html5/thumbnails/13.jpg)
University of Illinois at Urbana-Champaign
12
12
Surface Feature Points
Covariance Ellipsoid
Surface Normal
Orthonormal Basis
![Page 14: Covariance Estimation of GPS-LiDAR Sensor Fusion for UAVsweb.stanford.edu/.../S03.Akshay_Shetty.pdf · Akshay Shetty and Grace Xingxin Gao SCPNT, November 2017. University of Illinois](https://reader034.vdocuments.us/reader034/viewer/2022050315/5f7786b29135f93d797600b3/html5/thumbnails/14.jpg)
University of Illinois at Urbana-Champaign
13
13
Edge Feature Points
Covariance Ellipsoid
Edge Direction
Orthonormal Basis
![Page 15: Covariance Estimation of GPS-LiDAR Sensor Fusion for UAVsweb.stanford.edu/.../S03.Akshay_Shetty.pdf · Akshay Shetty and Grace Xingxin Gao SCPNT, November 2017. University of Illinois](https://reader034.vdocuments.us/reader034/viewer/2022050315/5f7786b29135f93d797600b3/html5/thumbnails/15.jpg)
University of Illinois at Urbana-Champaign
14
14
Combined Position Covariance
LiDAR-based position covariance:
Covariance Ellipsoid
LiDAR Point Cloud
Surface Points
Edge Points
![Page 16: Covariance Estimation of GPS-LiDAR Sensor Fusion for UAVsweb.stanford.edu/.../S03.Akshay_Shetty.pdf · Akshay Shetty and Grace Xingxin Gao SCPNT, November 2017. University of Illinois](https://reader034.vdocuments.us/reader034/viewer/2022050315/5f7786b29135f93d797600b3/html5/thumbnails/16.jpg)
University of Illinois at Urbana-Champaign
15
15
GPS Measurement Model
• Pseudorange measurement:
• Double-difference measurement:
• Measurement covariance:
Clock biasesSpeed of light Atmospheric errors Measurement noise
![Page 17: Covariance Estimation of GPS-LiDAR Sensor Fusion for UAVsweb.stanford.edu/.../S03.Akshay_Shetty.pdf · Akshay Shetty and Grace Xingxin Gao SCPNT, November 2017. University of Illinois](https://reader034.vdocuments.us/reader034/viewer/2022050315/5f7786b29135f93d797600b3/html5/thumbnails/17.jpg)
University of Illinois at Urbana-Champaign
16
Eliminate satellites blocked by 3D city model
16
Non-line-of-sight (NLOS) Satellites
![Page 18: Covariance Estimation of GPS-LiDAR Sensor Fusion for UAVsweb.stanford.edu/.../S03.Akshay_Shetty.pdf · Akshay Shetty and Grace Xingxin Gao SCPNT, November 2017. University of Illinois](https://reader034.vdocuments.us/reader034/viewer/2022050315/5f7786b29135f93d797600b3/html5/thumbnails/18.jpg)
University of Illinois at Urbana-Champaign
17
17
Outline
• Approach:
− Overall Architecture
− 3D City Model
− LiDAR-based State Estimation and Covariance
− GPS Measurement Model
• Experimental Setup and Results
• Summary
• Future Work: Deep Learning
![Page 19: Covariance Estimation of GPS-LiDAR Sensor Fusion for UAVsweb.stanford.edu/.../S03.Akshay_Shetty.pdf · Akshay Shetty and Grace Xingxin Gao SCPNT, November 2017. University of Illinois](https://reader034.vdocuments.us/reader034/viewer/2022050315/5f7786b29135f93d797600b3/html5/thumbnails/19.jpg)
University of Illinois at Urbana-Champaign
18
18
Experimental Setup
Custom-built iBQR UAV
LiDAR
GPS
Antenna
Onboard
Computer
GPS
Receiver
IMU
![Page 20: Covariance Estimation of GPS-LiDAR Sensor Fusion for UAVsweb.stanford.edu/.../S03.Akshay_Shetty.pdf · Akshay Shetty and Grace Xingxin Gao SCPNT, November 2017. University of Illinois](https://reader034.vdocuments.us/reader034/viewer/2022050315/5f7786b29135f93d797600b3/html5/thumbnails/20.jpg)
University of Illinois at Urbana-Champaign
19
19
Results: Individual Measurements
GPS unweighted least
squares estimate
contains large errors
LiDAR odometry drifts
over time, due to poor
distribution of features in
some sections
LiDAR – 3D city model
matching contains errors
where ICP might
converge to local minima
![Page 21: Covariance Estimation of GPS-LiDAR Sensor Fusion for UAVsweb.stanford.edu/.../S03.Akshay_Shetty.pdf · Akshay Shetty and Grace Xingxin Gao SCPNT, November 2017. University of Illinois](https://reader034.vdocuments.us/reader034/viewer/2022050315/5f7786b29135f93d797600b3/html5/thumbnails/21.jpg)
University of Illinois at Urbana-Champaign
20
20
Results: Sensor Fusion
Our covariance model v/s fixed covariance model
Our algorithm matches
true path more accurately
compared to a fixed
covariance model
![Page 22: Covariance Estimation of GPS-LiDAR Sensor Fusion for UAVsweb.stanford.edu/.../S03.Akshay_Shetty.pdf · Akshay Shetty and Grace Xingxin Gao SCPNT, November 2017. University of Illinois](https://reader034.vdocuments.us/reader034/viewer/2022050315/5f7786b29135f93d797600b3/html5/thumbnails/22.jpg)
University of Illinois at Urbana-Champaign
21
21
Outline
• Approach:
− Overall Architecture
− 3D City Model
− LiDAR-based State Estimation and Covariance
− GPS Measurement Model
• Experimental Setup and Results
• Summary
• Future Work: Deep Learning
![Page 23: Covariance Estimation of GPS-LiDAR Sensor Fusion for UAVsweb.stanford.edu/.../S03.Akshay_Shetty.pdf · Akshay Shetty and Grace Xingxin Gao SCPNT, November 2017. University of Illinois](https://reader034.vdocuments.us/reader034/viewer/2022050315/5f7786b29135f93d797600b3/html5/thumbnails/23.jpg)
University of Illinois at Urbana-Champaign
22
22
Summary
• Proposed a deep sensor fusion architecture for
GPS and LiDAR
• Implemented a novel method to characterize
LiDAR-based position covariance
• Applied a 3D city model to eliminate NLOS
satellites
• Validated improvement in positioning accuracy
using proposed technique
![Page 24: Covariance Estimation of GPS-LiDAR Sensor Fusion for UAVsweb.stanford.edu/.../S03.Akshay_Shetty.pdf · Akshay Shetty and Grace Xingxin Gao SCPNT, November 2017. University of Illinois](https://reader034.vdocuments.us/reader034/viewer/2022050315/5f7786b29135f93d797600b3/html5/thumbnails/24.jpg)
University of Illinois at Urbana-Champaign
23
23
Outline
• Approach:
− Overall Architecture
− 3D City Model
− LiDAR-based State Estimation and Covariance
− GPS Measurement Model
• Experimental Setup and Results
• Summary
• Future Work: Deep Learning
![Page 25: Covariance Estimation of GPS-LiDAR Sensor Fusion for UAVsweb.stanford.edu/.../S03.Akshay_Shetty.pdf · Akshay Shetty and Grace Xingxin Gao SCPNT, November 2017. University of Illinois](https://reader034.vdocuments.us/reader034/viewer/2022050315/5f7786b29135f93d797600b3/html5/thumbnails/25.jpg)
University of Illinois at Urbana-Champaign
24
24
Deep Learning for Sensor Fusion
Develop deep learning for different components
![Page 26: Covariance Estimation of GPS-LiDAR Sensor Fusion for UAVsweb.stanford.edu/.../S03.Akshay_Shetty.pdf · Akshay Shetty and Grace Xingxin Gao SCPNT, November 2017. University of Illinois](https://reader034.vdocuments.us/reader034/viewer/2022050315/5f7786b29135f93d797600b3/html5/thumbnails/26.jpg)
University of Illinois at Urbana-Champaign
25
25
Deep Learning Dataset for GPS
• Experimental vehicle with 10 GPS receivers
• Collected data near San Francisco: downtown,
underground, open areas, etc.
• Intermediate measurements such as
pseudoranges, carrier phases, SNR
• High-grade IMU for ground truth
![Page 27: Covariance Estimation of GPS-LiDAR Sensor Fusion for UAVsweb.stanford.edu/.../S03.Akshay_Shetty.pdf · Akshay Shetty and Grace Xingxin Gao SCPNT, November 2017. University of Illinois](https://reader034.vdocuments.us/reader034/viewer/2022050315/5f7786b29135f93d797600b3/html5/thumbnails/27.jpg)
University of Illinois at Urbana-Champaign
26
26
Deep Learning Dataset for LiDAR
Simulations in Unity Game Engine
![Page 28: Covariance Estimation of GPS-LiDAR Sensor Fusion for UAVsweb.stanford.edu/.../S03.Akshay_Shetty.pdf · Akshay Shetty and Grace Xingxin Gao SCPNT, November 2017. University of Illinois](https://reader034.vdocuments.us/reader034/viewer/2022050315/5f7786b29135f93d797600b3/html5/thumbnails/28.jpg)
University of Illinois at Urbana-Champaign
27
Acknowledgement
We would like to thank Kalmanje Krishnakumar and his group at NASA
Ames for supporting this work under the grant NNX17AC13G
27
Thank you!