monitoring, modelling, and predicting with real-time control dr ian oppermann director, csiro ict...
TRANSCRIPT
![Page 1: Monitoring, Modelling, and Predicting with Real-Time Control Dr Ian Oppermann Director, CSIRO ICT Centre](https://reader036.vdocuments.us/reader036/viewer/2022062500/56649e9d5503460f94b9eb0e/html5/thumbnails/1.jpg)
Monitoring, Modelling, and Predicting with Real-Time Control
Dr Ian OppermannDirector, CSIRO ICT Centre
![Page 2: Monitoring, Modelling, and Predicting with Real-Time Control Dr Ian Oppermann Director, CSIRO ICT Centre](https://reader036.vdocuments.us/reader036/viewer/2022062500/56649e9d5503460f94b9eb0e/html5/thumbnails/2.jpg)
Understanding the world …in real time
CSIRO. Monitoring, Modelling, and Predicting with Real-Time Control
It is so natural for us.Nothing is “natural” in Robotics How do we do it in “real time”?We focus on four components.
Sensing Perception, Control Actuation
![Page 3: Monitoring, Modelling, and Predicting with Real-Time Control Dr Ian Oppermann Director, CSIRO ICT Centre](https://reader036.vdocuments.us/reader036/viewer/2022062500/56649e9d5503460f94b9eb0e/html5/thumbnails/3.jpg)
Understanding the world : A Robot’s Viewpoint
CSIRO. Monitoring, Modelling, and Predicting with Real-Time Control
operations performed, from sensing to actuation, within the timeframe required for effective operation in a dynamic environment
![Page 4: Monitoring, Modelling, and Predicting with Real-Time Control Dr Ian Oppermann Director, CSIRO ICT Centre](https://reader036.vdocuments.us/reader036/viewer/2022062500/56649e9d5503460f94b9eb0e/html5/thumbnails/4.jpg)
Globally Consistent Real-time Perception
Sensing(Lidar. Radar, Stereo)
Sensing(Lidar. Radar, Stereo)
Semantics(Path, Obstacles)
Semantics(Path, Obstacles)
SLAM
Long Term Mapping
The difference between MOT and CD is in the time
scale and whether the object is transitory or
permanent
The difference between MOT and CD is in the time
scale and whether the object is transitory or
permanent
The difference between Localization and MOT is the difference between recording the pose of
ourselves versus recording the pose of
somebody else
The difference between Localization and MOT is the difference between recording the pose of
ourselves versus recording the pose of
somebody else
Globally Consistent Model(map, trajectories etc.)
CSIRO. Monitoring, Modelling, and Predicting with Real-Time Control
![Page 5: Monitoring, Modelling, and Predicting with Real-Time Control Dr Ian Oppermann Director, CSIRO ICT Centre](https://reader036.vdocuments.us/reader036/viewer/2022062500/56649e9d5503460f94b9eb0e/html5/thumbnails/5.jpg)
Mobile Mapping: What’s available now?
Big, Bulky and Expensive• Rooftop-mounted sensors
• 2D lidars with high-end GPS/INS
• Cost: $ X00,000
Limited Environments• Reliance on GPS which is
challenged in urban canyons, underground, near large infrastructure, forests, mines
Not Real Time• Processing straightforward if
accurate position is known at all times but NOT REAL TIME
CSIRO. Monitoring, Modelling, and Predicting with Real-Time Control
![Page 6: Monitoring, Modelling, and Predicting with Real-Time Control Dr Ian Oppermann Director, CSIRO ICT Centre](https://reader036.vdocuments.us/reader036/viewer/2022062500/56649e9d5503460f94b9eb0e/html5/thumbnails/6.jpg)
CSIRO. Monitoring, Modelling, and Predicting with Real-Time Control
What problems to address?
Cost
move from expensive hardware to software
Accuracy
improve location estimation
loop closure (knowing where you have been)
A dynamic environment
Long term versus short term changes in environment
Speed
Real time means being able to sense, aggregate, decide or re-plan in the time frames affected by limits of safety, fuel, task completion constraints
![Page 7: Monitoring, Modelling, and Predicting with Real-Time Control Dr Ian Oppermann Director, CSIRO ICT Centre](https://reader036.vdocuments.us/reader036/viewer/2022062500/56649e9d5503460f94b9eb0e/html5/thumbnails/7.jpg)
CSIRO. Monitoring, Modelling, and Predicting with Real-Time Control
Mapping and Location Improving accuracy, reducing cost
3D Map created with spinning 2D LIDAR on Bobcat
With Scan Matching (without any additional sensors)
Data A
ssociation R
obust Optim
izationD
ata Association
Robust O
ptimization
• No GPS• No odometry• No encoders• No IMU (inertial measurement Unit)
![Page 8: Monitoring, Modelling, and Predicting with Real-Time Control Dr Ian Oppermann Director, CSIRO ICT Centre](https://reader036.vdocuments.us/reader036/viewer/2022062500/56649e9d5503460f94b9eb0e/html5/thumbnails/8.jpg)
CSIRO. Monitoring, Modelling, and Predicting with Real-Time Control
Loop Closure – Improving accuracy
With Place Recognition (loop closure)
2D LIDAR on moving 4WD
Data A
ssociation R
obust Optim
izationD
ata Association
Robust O
ptimization
![Page 9: Monitoring, Modelling, and Predicting with Real-Time Control Dr Ian Oppermann Director, CSIRO ICT Centre](https://reader036.vdocuments.us/reader036/viewer/2022062500/56649e9d5503460f94b9eb0e/html5/thumbnails/9.jpg)
CSIRO. Monitoring, Modelling, and Predicting with Real-Time Control
Loop Closure Reliability and Accuracy over long times and long distances
2 Lasers
No GPS
No odometry
No encoders
No IMU
No Calibration
![Page 10: Monitoring, Modelling, and Predicting with Real-Time Control Dr Ian Oppermann Director, CSIRO ICT Centre](https://reader036.vdocuments.us/reader036/viewer/2022062500/56649e9d5503460f94b9eb0e/html5/thumbnails/10.jpg)
Object Tracking
Long term Tracking
Under different conditions Passive actuationCSIRO. Monitoring, Modelling, and Predicting with Real-Time Control
![Page 11: Monitoring, Modelling, and Predicting with Real-Time Control Dr Ian Oppermann Director, CSIRO ICT Centre](https://reader036.vdocuments.us/reader036/viewer/2022062500/56649e9d5503460f94b9eb0e/html5/thumbnails/11.jpg)
Example : Mapping Jenolan Caves
CSIRO. Monitoring, Modelling, and Predicting with Real-Time Control
![Page 12: Monitoring, Modelling, and Predicting with Real-Time Control Dr Ian Oppermann Director, CSIRO ICT Centre](https://reader036.vdocuments.us/reader036/viewer/2022062500/56649e9d5503460f94b9eb0e/html5/thumbnails/12.jpg)
Example : Mapping Jenolan Caves Sample Point Cloud
CSIRO. Monitoring, Modelling, and Predicting with Real-Time Control
![Page 13: Monitoring, Modelling, and Predicting with Real-Time Control Dr Ian Oppermann Director, CSIRO ICT Centre](https://reader036.vdocuments.us/reader036/viewer/2022062500/56649e9d5503460f94b9eb0e/html5/thumbnails/13.jpg)
Example : Mapping Jenolan Caves Watertight Surface Generation
CSIRO. Monitoring, Modelling, and Predicting with Real-Time Control
![Page 14: Monitoring, Modelling, and Predicting with Real-Time Control Dr Ian Oppermann Director, CSIRO ICT Centre](https://reader036.vdocuments.us/reader036/viewer/2022062500/56649e9d5503460f94b9eb0e/html5/thumbnails/14.jpg)
Example : Mapping Jenolan Caves Chifley Cave Surface Model
CSIRO. Monitoring, Modelling, and Predicting with Real-Time Control
![Page 15: Monitoring, Modelling, and Predicting with Real-Time Control Dr Ian Oppermann Director, CSIRO ICT Centre](https://reader036.vdocuments.us/reader036/viewer/2022062500/56649e9d5503460f94b9eb0e/html5/thumbnails/15.jpg)
Example : Mapping Jenolan Caves Registered Point Cloud
CSIRO. Monitoring, Modelling, and Predicting with Real-Time Control
![Page 16: Monitoring, Modelling, and Predicting with Real-Time Control Dr Ian Oppermann Director, CSIRO ICT Centre](https://reader036.vdocuments.us/reader036/viewer/2022062500/56649e9d5503460f94b9eb0e/html5/thumbnails/16.jpg)
Example : Mapping Jenolan Caves Point Cloud Overlay
CSIRO. Monitoring, Modelling, and Predicting with Real-Time Control
![Page 17: Monitoring, Modelling, and Predicting with Real-Time Control Dr Ian Oppermann Director, CSIRO ICT Centre](https://reader036.vdocuments.us/reader036/viewer/2022062500/56649e9d5503460f94b9eb0e/html5/thumbnails/17.jpg)
Example: Bringing it all together
CSIRO. Monitoring, Modelling, and Predicting with Real-Time Control
![Page 18: Monitoring, Modelling, and Predicting with Real-Time Control Dr Ian Oppermann Director, CSIRO ICT Centre](https://reader036.vdocuments.us/reader036/viewer/2022062500/56649e9d5503460f94b9eb0e/html5/thumbnails/18.jpg)
Example : Airborne Terrain Mapping and Static Object Avoidance
CSIRO. Monitoring, Modelling, and Predicting with Real-Time Control
![Page 19: Monitoring, Modelling, and Predicting with Real-Time Control Dr Ian Oppermann Director, CSIRO ICT Centre](https://reader036.vdocuments.us/reader036/viewer/2022062500/56649e9d5503460f94b9eb0e/html5/thumbnails/19.jpg)
The Future?Connecting the Virtual to the Real (Robot)
Mixed-RealityTele-Robotic
Robot
Tele-Operation
Machine Autonomy
SharedAutonomy
Autonomy
Manual
User Interface
Intelligent Behavior
Extent of Knowledge
CommunicationsLatency
Global
LocalReactive
Proactive
AugmentedReality
AugmentedVirtuality
Supervisory
Assistive
Real
Virtual
CSIRO. Monitoring, Modelling, and Predicting with Real-Time Control
![Page 20: Monitoring, Modelling, and Predicting with Real-Time Control Dr Ian Oppermann Director, CSIRO ICT Centre](https://reader036.vdocuments.us/reader036/viewer/2022062500/56649e9d5503460f94b9eb0e/html5/thumbnails/20.jpg)
The Future? Convergence of Technology
Realistic Simulation of Environment.
Realistic Visualization of
Environment
Realistic Interaction with
Environment
CSIRO. Monitoring, Modelling, and Predicting with Real-Time Control
![Page 21: Monitoring, Modelling, and Predicting with Real-Time Control Dr Ian Oppermann Director, CSIRO ICT Centre](https://reader036.vdocuments.us/reader036/viewer/2022062500/56649e9d5503460f94b9eb0e/html5/thumbnails/21.jpg)
Simulation and Computation Spectrum
Off-line
Analysis
Multiple
Scenarios
CSIRO. Monitoring, Modelling, and Predicting with Real-Time Control
![Page 22: Monitoring, Modelling, and Predicting with Real-Time Control Dr Ian Oppermann Director, CSIRO ICT Centre](https://reader036.vdocuments.us/reader036/viewer/2022062500/56649e9d5503460f94b9eb0e/html5/thumbnails/22.jpg)
Data Importance Latency Bandwidth
Safety (automation) Failsafe Low Low
Control & State Critical Med Med
Sensing Desirable Variable High
Peer-to-Peer
MBWA
Peer-to-Peer
WiMAXBackhaul Network
WiFi/MiMo Mesh Access Network
Whole of system view
CSIRO. Monitoring, Modelling, and Predicting with Real-Time Control
![Page 23: Monitoring, Modelling, and Predicting with Real-Time Control Dr Ian Oppermann Director, CSIRO ICT Centre](https://reader036.vdocuments.us/reader036/viewer/2022062500/56649e9d5503460f94b9eb0e/html5/thumbnails/23.jpg)
The Challenge: Unified User Interface
Command & Control
Tele - Robotic SCADA
Whole of Mine Planning
by Paul BourkeCSIRO. Monitoring, Modelling, and Predicting with Real-Time Control
![Page 24: Monitoring, Modelling, and Predicting with Real-Time Control Dr Ian Oppermann Director, CSIRO ICT Centre](https://reader036.vdocuments.us/reader036/viewer/2022062500/56649e9d5503460f94b9eb0e/html5/thumbnails/24.jpg)
Summary
• Automation and Teleoperation in Realtime Environments require:• Knowing state of environment [Mapping]• Knowing where you are [Localization]
• SLAM provides:• Localization in GPS denied areas• Mapping over large scales• Independent real-time sensor• Integration with other sensors (GPS,IMU etc)
• Future Realtime Automation will require:• Pervasive Tracking
• SLAM-MOT (SLAM with Moving Object Tracking)• Collaborative / Distributed real time mapping
• Deal with uncertainty, latency, trust and different sensing modalities• Life long mapping and long term map management
• Scalable (semantic mapping)• Maps get better with age, rather than “blurry
CSIRO. Monitoring, Modelling, and Predicting with Real-Time Control
![Page 25: Monitoring, Modelling, and Predicting with Real-Time Control Dr Ian Oppermann Director, CSIRO ICT Centre](https://reader036.vdocuments.us/reader036/viewer/2022062500/56649e9d5503460f94b9eb0e/html5/thumbnails/25.jpg)
CSIRO – more than 80 years of achievements
CSIRO. Monitoring, Modelling, and Predicting with Real-Time Control
![Page 26: Monitoring, Modelling, and Predicting with Real-Time Control Dr Ian Oppermann Director, CSIRO ICT Centre](https://reader036.vdocuments.us/reader036/viewer/2022062500/56649e9d5503460f94b9eb0e/html5/thumbnails/26.jpg)
Contact UsPhone: 1300 363 400 or +61 3 9545 2176
Email: [email protected] Web: www.csiro.au
Thank you
With Thanks to Dr Rob ZlotDr Mike BosseDr Elliot DuffDr Jonathan Roberts
For further information:Dr Ian OppermannDirector, CSIRO ICT Centre
Email: [email protected]: www.csiro.au/ict