challenges in automated driving - matlab...8 sensor development in matlab and simulink radar vision...
TRANSCRIPT
![Page 1: Challenges in Automated Driving - Matlab...8 Sensor Development in MATLAB and Simulink Radar Vision Wireless Communications • Feature detection, extraction and matching • Image](https://reader033.vdocuments.us/reader033/viewer/2022060806/608b0c6afb14d739d42edde3/html5/thumbnails/1.jpg)
1© 2015 The MathWorks, Inc.
Automated Driving Developmentwith MATLAB® and Simulink®
Marc Willerton
Application Engineering, MathWorks
![Page 2: Challenges in Automated Driving - Matlab...8 Sensor Development in MATLAB and Simulink Radar Vision Wireless Communications • Feature detection, extraction and matching • Image](https://reader033.vdocuments.us/reader033/viewer/2022060806/608b0c6afb14d739d42edde3/html5/thumbnails/2.jpg)
2
Challenges in Automated Driving
Understanding the
area around me
Understanding the
best way to get there
Understanding how
to move
Should I stop?How should I
navigate this
path?
How should I
get there?
Perception Planning Control
![Page 3: Challenges in Automated Driving - Matlab...8 Sensor Development in MATLAB and Simulink Radar Vision Wireless Communications • Feature detection, extraction and matching • Image](https://reader033.vdocuments.us/reader033/viewer/2022060806/608b0c6afb14d739d42edde3/html5/thumbnails/3.jpg)
3
Designing a forward collision warning system
Perception Control
Planning
Deep Learning
V2V
Radar
Computer Vision
Model Predictive Control
ROS
GPU
Tracking
Vehicle Dynamics
CAN
Scenario Testing
SLAM
Sensor Fusion
Mapping
Data logging
Lidar
Detection
IMU
GPS
Labelling
Certification
FPGA
Depth PerceptionWheel Encoder
Surface mapping
Weather effects
Data generation
![Page 4: Challenges in Automated Driving - Matlab...8 Sensor Development in MATLAB and Simulink Radar Vision Wireless Communications • Feature detection, extraction and matching • Image](https://reader033.vdocuments.us/reader033/viewer/2022060806/608b0c6afb14d739d42edde3/html5/thumbnails/4.jpg)
4
Voyage Develops longitudinal controls for self-driving taxis
Designed longitudinal model predictive controller,
generated ROS node, and integrated with their open
source software framework for perception and planning
“We were searching for a prototyping solution
that was fast for development and robust for
production. We decided to go with Simulink for
controller development and code generation,
while using MATLAB to automate development
tasks.”
- Alan Mond, Voyage
Voyage’s self driving car in San Jose, California.
Voyage user story, 2017
![Page 5: Challenges in Automated Driving - Matlab...8 Sensor Development in MATLAB and Simulink Radar Vision Wireless Communications • Feature detection, extraction and matching • Image](https://reader033.vdocuments.us/reader033/viewer/2022060806/608b0c6afb14d739d42edde3/html5/thumbnails/5.jpg)
5
Scania presentation recording from 2015 MathWorks Automotive Conference, Germany
Source: Model-Based Design Approach to Resource Efficient Object Fusion for an Autonomous Braking System, Jonny Andersson, Scania
Scania Re-simulation workflow to develop AEB sensor fusion algorithm
![Page 6: Challenges in Automated Driving - Matlab...8 Sensor Development in MATLAB and Simulink Radar Vision Wireless Communications • Feature detection, extraction and matching • Image](https://reader033.vdocuments.us/reader033/viewer/2022060806/608b0c6afb14d739d42edde3/html5/thumbnails/6.jpg)
6
Perception Control
Planning
How can I make a forward collision warning system?
Control
Planning
Perception
Sensor detection
visualization
Path planning
Control and Vehicle
Dynamics
Sensor fusion and tracking
Sensor development
Classification
(Deep learning)
Scenario testing
ROSSLAM
![Page 7: Challenges in Automated Driving - Matlab...8 Sensor Development in MATLAB and Simulink Radar Vision Wireless Communications • Feature detection, extraction and matching • Image](https://reader033.vdocuments.us/reader033/viewer/2022060806/608b0c6afb14d739d42edde3/html5/thumbnails/7.jpg)
7
Perception Control
Planning
How can I make a forward collision warning system?
Control
Planning
Perception
Sensor detection
visualization
Path planning
Control and Vehicle
Dynamics
Sensor fusion and tracking
Sensor development
Classification
(Deep learning)
Scenario testing
ROSSLAM
![Page 8: Challenges in Automated Driving - Matlab...8 Sensor Development in MATLAB and Simulink Radar Vision Wireless Communications • Feature detection, extraction and matching • Image](https://reader033.vdocuments.us/reader033/viewer/2022060806/608b0c6afb14d739d42edde3/html5/thumbnails/8.jpg)
8
Sensor Development in MATLAB and Simulink
Radar Vision Wireless Communications
• Feature detection, extraction and matching
• Image segmentation, registration, enhancement
• Detect and Track objects
• Image overlay
• Feed to deep learning algorithms
• Deploy to processor / GPU / FPGA
• Antenna Modelling
• Antenna Array Design
• Beamforming and Direction Finding Algorithms
• Detect and Track objects
• Develop RF component architectures and model
imperfections
• Radar control
• Extract DSRC + C-V2X performance metrics
• PHY Communications Modelling
• Pre-built standards based functions
• RF propagation models and visualisations
• Prototype with Software Defined Radio
![Page 9: Challenges in Automated Driving - Matlab...8 Sensor Development in MATLAB and Simulink Radar Vision Wireless Communications • Feature detection, extraction and matching • Image](https://reader033.vdocuments.us/reader033/viewer/2022060806/608b0c6afb14d739d42edde3/html5/thumbnails/9.jpg)
9
Perception Control
Planning
How can I make a forward collision warning system?
Control
Planning
Perception
Sensor detection
visualization
Path planning
Control and Vehicle
Dynamics
Sensor fusion and tracking
Sensor development
Classification
(Deep learning)
Scenario testing
ROSSLAM
![Page 10: Challenges in Automated Driving - Matlab...8 Sensor Development in MATLAB and Simulink Radar Vision Wireless Communications • Feature detection, extraction and matching • Image](https://reader033.vdocuments.us/reader033/viewer/2022060806/608b0c6afb14d739d42edde3/html5/thumbnails/10.jpg)
10
Classification (Deep Learning)
Deep Learning approach
…𝟗𝟓%𝟑%
𝟐%
Truck
Car ✓
Bicycle
Learned featuresEnd-to-end learning
Feature learning + Classification
![Page 11: Challenges in Automated Driving - Matlab...8 Sensor Development in MATLAB and Simulink Radar Vision Wireless Communications • Feature detection, extraction and matching • Image](https://reader033.vdocuments.us/reader033/viewer/2022060806/608b0c6afb14d739d42edde3/html5/thumbnails/11.jpg)
11
Deep Learning Made Easy in MATLAB
Train free space detection network using deep learning
Computer Vision
System ToolboxTM
![Page 12: Challenges in Automated Driving - Matlab...8 Sensor Development in MATLAB and Simulink Radar Vision Wireless Communications • Feature detection, extraction and matching • Image](https://reader033.vdocuments.us/reader033/viewer/2022060806/608b0c6afb14d739d42edde3/html5/thumbnails/12.jpg)
12
Ground Truth Labeling App for labelling Training Data
automatically
![Page 13: Challenges in Automated Driving - Matlab...8 Sensor Development in MATLAB and Simulink Radar Vision Wireless Communications • Feature detection, extraction and matching • Image](https://reader033.vdocuments.us/reader033/viewer/2022060806/608b0c6afb14d739d42edde3/html5/thumbnails/13.jpg)
13
Generate CUDA Code for Embedded Deployment
function out = segnet_predict(in) %#codegen
persistent mynet;
if isempty(mynet)
mynet = coder.loadDeepLearningNetwork('SegNet.mat');
end
out = predict(mynet,in);
% Save network to MAT file
save('SegNet.mat', 'net’)
% Generate CUDA code
cfg = coder.config('lib');
cfg.TargetLang = 'C++';
codegen -config cfg segnet_predict -args
{ones(360,480,3,'uint8')} -report
![Page 14: Challenges in Automated Driving - Matlab...8 Sensor Development in MATLAB and Simulink Radar Vision Wireless Communications • Feature detection, extraction and matching • Image](https://reader033.vdocuments.us/reader033/viewer/2022060806/608b0c6afb14d739d42edde3/html5/thumbnails/14.jpg)
14
Automated Driving System Toolbox introduced examples to:
Accelerate the process of Ground Truth Labeling
▪ Add your own automation
algorithm to Ground Truth
Labeler App
▪ Extend connectivity of
Ground Truth Labeler App
to visualize lidar data
▪ Label detections with
Ground Truth Labeler App
![Page 15: Challenges in Automated Driving - Matlab...8 Sensor Development in MATLAB and Simulink Radar Vision Wireless Communications • Feature detection, extraction and matching • Image](https://reader033.vdocuments.us/reader033/viewer/2022060806/608b0c6afb14d739d42edde3/html5/thumbnails/15.jpg)
15
Learn more about developing deep learning perception algorithms
with these examples
▪ Train free space detection network using deep learning
Computer Vision
System ToolboxTM
▪ Add semantic segmentation
automation algorithm to
Ground Truth Labeler AppAutomated Driving
System ToolboxTM
▪ Generate CUDA® code to
execute directed acyclic graph
network on an NVIDIA GPU
GPU CoderTM
![Page 16: Challenges in Automated Driving - Matlab...8 Sensor Development in MATLAB and Simulink Radar Vision Wireless Communications • Feature detection, extraction and matching • Image](https://reader033.vdocuments.us/reader033/viewer/2022060806/608b0c6afb14d739d42edde3/html5/thumbnails/16.jpg)
16
Perception Control
Planning
How can I make a forward collision warning system?
Control
Planning
Perception
Sensor detection
visualization
Path planning
Control and Vehicle
Dynamics
Sensor fusion and tracking
Sensor development
Classification
(Deep learning)
Scenario testing
ROSSLAM
![Page 17: Challenges in Automated Driving - Matlab...8 Sensor Development in MATLAB and Simulink Radar Vision Wireless Communications • Feature detection, extraction and matching • Image](https://reader033.vdocuments.us/reader033/viewer/2022060806/608b0c6afb14d739d42edde3/html5/thumbnails/17.jpg)
17
Example: Synthesizing scenarios to test sensor fusion
algorithms
Synthesize driving
scenario to test multi-
object tracker
Build scenarios
programmatically or interactively
Load prebuilt scenarios (e.g.
NCAP)
Export to MATLAB / Simulink
![Page 18: Challenges in Automated Driving - Matlab...8 Sensor Development in MATLAB and Simulink Radar Vision Wireless Communications • Feature detection, extraction and matching • Image](https://reader033.vdocuments.us/reader033/viewer/2022060806/608b0c6afb14d739d42edde3/html5/thumbnails/18.jpg)
18
Example: Stelvio Pass GPX Data Import
p = gpxread('StelvioPass.gpx’);
flatearth_pos = lla2flat([ p.Longitude' p.Latitude' p.Elevation'],...
[p.Longitude(1),p.Latitude(1)],0,p.Elevation(1));
s=drivingScenario;
roadWidth = 4;
road(s,flatearth_pos,roadWidth);
v1 = vehicle(s,'Position',flatearth_pos(1,:),'Velocity’,[0 0 0],
'Yaw', 140);
speed = 50;
path(v1,flatearth_pos,speed);
chasePlot(v1,'Centerline','on');
player = geoplayer(p.Longitude(1), p.Latitude(1));
restart(s)
while advance(s)
t = targetPoses(v1);
a = flat2lla(s.Actors.Position,roadCentres(1,1:2),...
0,roadCentres(1,3));
plotPosition(player,a(2),a(1));
end
Complex paths overlaid on maps
can be imported from external
logs and then simulated with
models of cars and sensors
![Page 19: Challenges in Automated Driving - Matlab...8 Sensor Development in MATLAB and Simulink Radar Vision Wireless Communications • Feature detection, extraction and matching • Image](https://reader033.vdocuments.us/reader033/viewer/2022060806/608b0c6afb14d739d42edde3/html5/thumbnails/19.jpg)
19
Example: Stelvio Pass GPX Data Import
![Page 20: Challenges in Automated Driving - Matlab...8 Sensor Development in MATLAB and Simulink Radar Vision Wireless Communications • Feature detection, extraction and matching • Image](https://reader033.vdocuments.us/reader033/viewer/2022060806/608b0c6afb14d739d42edde3/html5/thumbnails/20.jpg)
20
Perception Control
Planning
How can I make a forward collision warning system?
Control
Planning
Perception
Sensor detection
visualization
Path planning
Control and Vehicle
Dynamics
Sensor fusion and tracking
Sensor development
Classification
(Deep learning)
Scenario testing
ROSSLAM
![Page 21: Challenges in Automated Driving - Matlab...8 Sensor Development in MATLAB and Simulink Radar Vision Wireless Communications • Feature detection, extraction and matching • Image](https://reader033.vdocuments.us/reader033/viewer/2022060806/608b0c6afb14d739d42edde3/html5/thumbnails/21.jpg)
21
Multi-object tracker to develop sensor fusion algorithms
Detections
Multi-Object Tracker
TracksTracking
Filter
Track
Manager
• Assigns detections to tracks
• Creates new tracks
• Updates existing tracks
• Removes old tracks
• Predicts and
updates state of
track
• Supports linear,
extended, and
unscented Kalman
filters
![Page 22: Challenges in Automated Driving - Matlab...8 Sensor Development in MATLAB and Simulink Radar Vision Wireless Communications • Feature detection, extraction and matching • Image](https://reader033.vdocuments.us/reader033/viewer/2022060806/608b0c6afb14d739d42edde3/html5/thumbnails/22.jpg)
22
Tracking Filter: Kalman Filter Basics
Initial Estimates of unknowns
and filter internals
Sensor
Readings
External
Force
EquationsPredict
predict(filter,T)
Updatecorrect(filter,pos(k,:))
Next Time
StepCurrent
Time Step
A priori
estimate of
unknowns
A posteriori
estimate of
unknowns
e.g. Position / Velocity
e.g. Breaking
e.g. Position / Velocity
![Page 23: Challenges in Automated Driving - Matlab...8 Sensor Development in MATLAB and Simulink Radar Vision Wireless Communications • Feature detection, extraction and matching • Image](https://reader033.vdocuments.us/reader033/viewer/2022060806/608b0c6afb14d739d42edde3/html5/thumbnails/23.jpg)
23
Constant Velocity Kalman FilterinitialState = [5.3;0;3.6;0];
filter = trackingKF('MotionModel','2D Constant…
Velocity','State',initialState);
vx = 0.2;
vy = 0.1;
T = 0.5;
pos = [linspace(0,4,42); linspace(5,12,42)].’;
for k = 1:size(pos,1)
pstates(k,:) = predict(filter,T);
cstates(k,:) = correct(filter,pos(k,:));
end
plotResults
![Page 24: Challenges in Automated Driving - Matlab...8 Sensor Development in MATLAB and Simulink Radar Vision Wireless Communications • Feature detection, extraction and matching • Image](https://reader033.vdocuments.us/reader033/viewer/2022060806/608b0c6afb14d739d42edde3/html5/thumbnails/24.jpg)
24
Fuse and track multiple detections from different sensors
Radar
Detections
Multi-Object Tracker
TracksKalman
Filter
Track
Manager
Time
Position
Velocity
Vision
Detections
Dete
ctio
n P
acker
Time
Position
VelocityDetections
Time
Measurement
Measurement Noise
Time
State
Covariance
Track ID
Age
Is Confirmed
Is Coasted
Forward Collision
Warning Using Sensor
Fusion
• Packing sensor data
into object detections
• Initializing Kalman
filter
• Configuring multi-
object tracker
![Page 25: Challenges in Automated Driving - Matlab...8 Sensor Development in MATLAB and Simulink Radar Vision Wireless Communications • Feature detection, extraction and matching • Image](https://reader033.vdocuments.us/reader033/viewer/2022060806/608b0c6afb14d739d42edde3/html5/thumbnails/25.jpg)
25
More complex tracking: Extended Object Tracking
Enhanced
Trackers and
sensor models
New in R2018b:
Sensor Fusion
and Tracking
Toolbox!
![Page 26: Challenges in Automated Driving - Matlab...8 Sensor Development in MATLAB and Simulink Radar Vision Wireless Communications • Feature detection, extraction and matching • Image](https://reader033.vdocuments.us/reader033/viewer/2022060806/608b0c6afb14d739d42edde3/html5/thumbnails/26.jpg)
26
Perception Control
Planning
How can I make a forward collision warning system?
Control
Planning
Perception
Sensor detection
visualization
Path planning
Control and Vehicle
Dynamics
Sensor fusion and tracking
Sensor development
Classification
(Deep learning)
Scenario testing
ROSSLAM
![Page 27: Challenges in Automated Driving - Matlab...8 Sensor Development in MATLAB and Simulink Radar Vision Wireless Communications • Feature detection, extraction and matching • Image](https://reader033.vdocuments.us/reader033/viewer/2022060806/608b0c6afb14d739d42edde3/html5/thumbnails/27.jpg)
27
Camera (640 x 480 x 3)
239 239 237 238 241 241 241 242 243 243 243 243 243 243 243 243 243 243 243 243
252 252 251 252 252 253 253 253 255 255 255 255 255 255 255 255 255 255 255 255
254 254 253 253 254 253 255 255 255 255 255 255 255 255 255 255 255 255 255 255
250 251 251 251 251 251 253 251 253 253 255 255 253 255 255 255 255 255 255 255
251 252 251 253 253 253 251 251 253 253 253 253 254 253 253 253 253 253 253 253
252 253 253 254 254 253 253 253 253 253 253 253 253 253 254 253 253 253 253 253
252 253 253 254 254 254 253 253 253 253 253 253 253 253 253 253 251 251 251 251
253 253 253 254 254 254 254 253 253 253 253 253 253 253 253 253 253 253 253 253
253 254 253 254 254 254 254 253 253 253 253 253 253 253 253 253 253 253 253 253
254 254 254 254 254 254 254 253 253 253 253 253 253 253 253 253 253 253 253 253
254 254 254 254 254 254 254 253 253 253 253 253 253 253 253 253 253 253 253 253
254 254 254 254 254 254 254 253 253 253 253 253 253 253 253 253 253 253 253 253
255 255 255 255 255 255 255 255 254 254 254 254 254 254 254 254 254 254 254 254
255 255 255 255 255 255 255 255 254 254 254 254 254 254 254 254 254 254 254 254
253 254 255 255 255 255 254 254 253 253 253 253 254 254 254 254 254 254 254 254
253 255 255 255 255 255 254 254 253 253 253 253 254 254 254 254 254 254 254 254
254 254 254 254 253 253 253 253 253 253 253 253 253 253 253 253 253 253 253 253
254 254 254 254 253 253 253 253 253 253 253 253 253 253 253 253 253 253 253 253
253 253 253 253 253 253 251 251 251 251 251 251 253 253 253 253 253 253 253 253
253 253 253 253 253 253 251 251 251 251 251 251 253 253 253 253 253 253 253 253
253 253 253 253 253 253 251 251 251 251 251 251 253 253 253 253 253 253 253 253
253 253 253 253 253 253 251 251 251 251 251 251 253 253 253 253 253 253 253 253
253 253 253 253 253 253 251 251 251 251 251 251 253 253 253 253 253 253 253 253
253 253 253 253 253 253 251 251 251 251 251 251 253 253 253 253 253 253 253 253
Sensor detection Visualisation
Vision based
lane detector
Vision DetectorSensorID = 1;
Timestamp = 1461634696379742;
NumDetections = 6;
Detections(1)
TrackID: 0
Classification: 5
Position: [22.61 -0.43 2.24]
Velocity: [-9.86 0 0]
Size: [0 1.75 0]
Detections(2)
TrackID: 1
Classification: 5
Position: [22.8 3.12 2.24]
Velocity: [-9.37 0 0]
Size: [0 1.8 0]
Radar DetectorSensorID = 2;
Timestamp = 1461634696407521;
NumDetections = 23;
Detections(1)
TrackID: 0
TrackStatus: 6
Position: [56.07 17.73 0.34]
Velocity: [-8.50 2.86 0]
Amplitude: 3
Detections(2)
TrackID: 1
TrackStatus: 6
Position: [35.35 19.59 0.34]
Velocity: [-8.02 4.92 0]
Amplitude: 3
Detections(3)
TrackID: 12
TrackStatus: 5
Position: [57.69 3.13 0.34]
Lane DetectorLeft
IsValid: 1
Confidence: 3
BoundaryType: 3
Offset: 1.68
HeadingAngle: 0.002
Curvature: 0.0000228
Right
IsValid: 1
Confidence: 3
Lidar (47197 x 3)
-12.2911 1.4790 -0.5900
-14.8852 1.7755 -0.6475
-18.8020 2.2231 -0.7396
-25.7033 3.0119 -0.9246
-0.0632 0.0815 1.2501
-0.0978 0.0855 1.2561
-0.2814 0.1064 1.2575
-0.3375 0.1129 1.2650
-0.4611 0.1270 1.2572
-0.6184 0.1450 1.2475
-0.8369 0.1699 1.2319
-14.8815 1.8245 -0.6478
-18.8008 2.2849 -0.7403
-25.7134 3.0970 -0.9265
Inertial Measurement UnitTimestamp: 1461634696379742
Velocity: 9.2795
YawRate: 0.0040
![Page 28: Challenges in Automated Driving - Matlab...8 Sensor Development in MATLAB and Simulink Radar Vision Wireless Communications • Feature detection, extraction and matching • Image](https://reader033.vdocuments.us/reader033/viewer/2022060806/608b0c6afb14d739d42edde3/html5/thumbnails/28.jpg)
28
Example: Explore Logged Vehicle Data
Bring in sensor detection data and build up a visualisation using this example:
web(fullfile(docroot, 'driving/examples/visualize-sensor-coverage-detections-and-tracks.html'))
![Page 29: Challenges in Automated Driving - Matlab...8 Sensor Development in MATLAB and Simulink Radar Vision Wireless Communications • Feature detection, extraction and matching • Image](https://reader033.vdocuments.us/reader033/viewer/2022060806/608b0c6afb14d739d42edde3/html5/thumbnails/29.jpg)
29
Learn more about visualizing vehicle databy exploring examples in the Automated Driving System Toolbox
▪ Transform between
vehicle and image
coordinates
▪ Plot lidar point cloud▪ Plot object detectors
in vehicle coordinates– Vision & radar detector
– Lane detectors
– Detector coverage areas
![Page 30: Challenges in Automated Driving - Matlab...8 Sensor Development in MATLAB and Simulink Radar Vision Wireless Communications • Feature detection, extraction and matching • Image](https://reader033.vdocuments.us/reader033/viewer/2022060806/608b0c6afb14d739d42edde3/html5/thumbnails/30.jpg)
30
Radar
Camera
IMU
CA
N
CAN Tx CAN FD
EthernetTCP/IP
Testing perception algorithms with live data
Algorithm
Visualization
Read
sensor data stream
and video stream
CAN
Rx
TCP/IP
Vision Object
Radar Object
Lane
Vehicle Speed
Yaw Rate
Video frame
https://uk.mathworks.com/help/vnt/examples/forward-
collision-warning-can-fd.html
![Page 31: Challenges in Automated Driving - Matlab...8 Sensor Development in MATLAB and Simulink Radar Vision Wireless Communications • Feature detection, extraction and matching • Image](https://reader033.vdocuments.us/reader033/viewer/2022060806/608b0c6afb14d739d42edde3/html5/thumbnails/31.jpg)
31
Perception Control
Planning
How can I make a forward collision warning system?
Control
Planning
Perception
Sensor detection
visualization
Path planning
Control and Vehicle
Dynamics
Sensor fusion
Sensor development
Classification
(Deep learning)
Scenario testing
ROSSLAM
![Page 32: Challenges in Automated Driving - Matlab...8 Sensor Development in MATLAB and Simulink Radar Vision Wireless Communications • Feature detection, extraction and matching • Image](https://reader033.vdocuments.us/reader033/viewer/2022060806/608b0c6afb14d739d42edde3/html5/thumbnails/32.jpg)
32
What is Path Planning?Parking Valet Example
▪ Grid-based search
– Split the area in to a grid
– Move between adjacent grid squares by
solving an optimization problem
▪ Motion planning – use Rapidly exploring
Random Tree (RRT)
▪ Path smoothing – so a controller can
execute
▪ Trajectory generation – create speed
profile
▪ Vehicle control – get steering angle and
generate acceleration commands
▪ Goal checking
▪ May also need to derive map
▪ May also be dynamic obstacles
https://uk.mathworks.com/help/releases/R2018a/driving/e
xamples/automated-parking-valet.html
![Page 33: Challenges in Automated Driving - Matlab...8 Sensor Development in MATLAB and Simulink Radar Vision Wireless Communications • Feature detection, extraction and matching • Image](https://reader033.vdocuments.us/reader033/viewer/2022060806/608b0c6afb14d739d42edde3/html5/thumbnails/33.jpg)
33
Perception Control
Planning
How can I make a forward collision warning system?
Control
Planning
Perception
Sensor detection
visualization
Path planning
Control and Vehicle
Dynamics
Sensor fusion
Sensor development
Classification
(Deep learning)
Scenario testing
ROSSLAM
![Page 34: Challenges in Automated Driving - Matlab...8 Sensor Development in MATLAB and Simulink Radar Vision Wireless Communications • Feature detection, extraction and matching • Image](https://reader033.vdocuments.us/reader033/viewer/2022060806/608b0c6afb14d739d42edde3/html5/thumbnails/34.jpg)
34
What about when there is no map?
MATLAB has SLAM algorithms which can be used to derive maps
during the localisation process
![Page 35: Challenges in Automated Driving - Matlab...8 Sensor Development in MATLAB and Simulink Radar Vision Wireless Communications • Feature detection, extraction and matching • Image](https://reader033.vdocuments.us/reader033/viewer/2022060806/608b0c6afb14d739d42edde3/html5/thumbnails/35.jpg)
35
Learn about developing path planning algorithms
with these examples
▪ Plot map tiles using
World Street Map (Esri)
Automated Driving
System ToolboxTM
▪ Plan path for automobile
given pre-defined map
Automated Driving
System ToolboxTM
![Page 36: Challenges in Automated Driving - Matlab...8 Sensor Development in MATLAB and Simulink Radar Vision Wireless Communications • Feature detection, extraction and matching • Image](https://reader033.vdocuments.us/reader033/viewer/2022060806/608b0c6afb14d739d42edde3/html5/thumbnails/36.jpg)
36
Perception Control
Planning
How can I make a forward collision warning system?
Control
Planning
Perception
Sensor detection
visualization
Path planning
Control and Vehicle
Dynamics
Sensor fusion
Sensor development
Classification
(Deep learning)
Scenario testing
ROSSLAM
![Page 37: Challenges in Automated Driving - Matlab...8 Sensor Development in MATLAB and Simulink Radar Vision Wireless Communications • Feature detection, extraction and matching • Image](https://reader033.vdocuments.us/reader033/viewer/2022060806/608b0c6afb14d739d42edde3/html5/thumbnails/37.jpg)
37
ROS is important in Automated Driving
?
![Page 38: Challenges in Automated Driving - Matlab...8 Sensor Development in MATLAB and Simulink Radar Vision Wireless Communications • Feature detection, extraction and matching • Image](https://reader033.vdocuments.us/reader033/viewer/2022060806/608b0c6afb14d739d42edde3/html5/thumbnails/38.jpg)
38
Perception Control
Planning
How can I make a forward collision warning system?
Control
Planning
Perception
Sensor detection
visualization
Path planning
Control and Vehicle
Dynamics
Sensor fusion
Sensor development
Classification
(Deep learning)
Scenario testing
ROSSLAM
![Page 39: Challenges in Automated Driving - Matlab...8 Sensor Development in MATLAB and Simulink Radar Vision Wireless Communications • Feature detection, extraction and matching • Image](https://reader033.vdocuments.us/reader033/viewer/2022060806/608b0c6afb14d739d42edde3/html5/thumbnails/39.jpg)
39
Developing Forward Collision Warning using Model Predictive
Control
Traditional Control Model Predictive Control
MPC is good for:
• Complex Nonlinear Systems (e.g. MIMO) – outputs and non-linearities are all considered
together and there are fewer parameters
• Constraints – changing hard and soft constraints (e.g. speed) can be considered within the
optimisation
• Preview – it is forward looking
![Page 40: Challenges in Automated Driving - Matlab...8 Sensor Development in MATLAB and Simulink Radar Vision Wireless Communications • Feature detection, extraction and matching • Image](https://reader033.vdocuments.us/reader033/viewer/2022060806/608b0c6afb14d739d42edde3/html5/thumbnails/40.jpg)
40
Model Predictive Control – the basics
Parameters:
➢ Period
➢ Prediction horizon
➢ Control horizon
➢ Constraints
Main Goal: Try iterations to get to the final position with
the minimum cost
![Page 41: Challenges in Automated Driving - Matlab...8 Sensor Development in MATLAB and Simulink Radar Vision Wireless Communications • Feature detection, extraction and matching • Image](https://reader033.vdocuments.us/reader033/viewer/2022060806/608b0c6afb14d739d42edde3/html5/thumbnails/41.jpg)
41
Manipulated variableReferences
Measured disturbance
Measured outputs
1. Relative distance (𝐷𝑟𝑒𝑙𝑎𝑡𝑖𝑣𝑒)2. Ego velocity (𝑉𝑒𝑔𝑜)
Control variable
Acceleration (𝑢)
References
1. Safe distance (𝐷𝑠𝑎𝑓𝑒)
2. Ego velocity set point
(𝑉𝑠𝑒𝑡)
Disturbances
Most Important Object velocity
Hidden Connector
Hidden Connector
Example: Developing a Cruise Control Algorithm
minimize: 𝑉𝑒𝑔𝑜 − 𝑉𝑠𝑒𝑡2
subject to: 𝐷𝑟𝑒𝑙𝑎𝑡𝑖𝑣𝑒 ≥ 𝐷𝑠𝑎𝑓𝑒
𝑢𝑚𝑖𝑛 ≤ 𝑢 ≤ 𝑢𝑚𝑎𝑥
Ego
vehicle
MPC controller
Optimizer
Ego vehicle
model
EgoDistance to Most Important Object (𝐷𝑟𝑒𝑙𝑎𝑡𝑖𝑣𝑒)
Safe distance (𝐷𝑠𝑎𝑓𝑒)
Most
Important
Object
![Page 42: Challenges in Automated Driving - Matlab...8 Sensor Development in MATLAB and Simulink Radar Vision Wireless Communications • Feature detection, extraction and matching • Image](https://reader033.vdocuments.us/reader033/viewer/2022060806/608b0c6afb14d739d42edde3/html5/thumbnails/42.jpg)
42
Model Vehicle Dynamics using Vehicle Dynamics Blockset
Library of blocks
Pre-built reference applications
Game engine
![Page 43: Challenges in Automated Driving - Matlab...8 Sensor Development in MATLAB and Simulink Radar Vision Wireless Communications • Feature detection, extraction and matching • Image](https://reader033.vdocuments.us/reader033/viewer/2022060806/608b0c6afb14d739d42edde3/html5/thumbnails/43.jpg)
43
Chassis Controls Study: Braking Test with ABS
![Page 44: Challenges in Automated Driving - Matlab...8 Sensor Development in MATLAB and Simulink Radar Vision Wireless Communications • Feature detection, extraction and matching • Image](https://reader033.vdocuments.us/reader033/viewer/2022060806/608b0c6afb14d739d42edde3/html5/thumbnails/44.jpg)
44
▪ Simulate vehicle
dynamics for closed
loop design
Vehicle Dynamics BlocksetTM
▪ Co-simulate with Unreal
Engine to set actor positions
and get camera image
Vehicle Dynamics BlocksetTM
Model Vehicle Dynamics using Vehicle Dynamics Blockset
![Page 45: Challenges in Automated Driving - Matlab...8 Sensor Development in MATLAB and Simulink Radar Vision Wireless Communications • Feature detection, extraction and matching • Image](https://reader033.vdocuments.us/reader033/viewer/2022060806/608b0c6afb14d739d42edde3/html5/thumbnails/45.jpg)
45
Perception Control
Planning
Control
Planning
Perception
Sensor detection
visualization
Path planning
Control and Vehicle
Dynamics
Sensor fusion
Sensor development
Classification
(Deep learning)
Scenario testing
ROSSLAM