human factors in (tele) operation of autonomous vehicles
TRANSCRIPT
Oren Musicant, [email protected]
Mobile-Lab, Ariel University
Human Factors in (Tele) operation of
Autonomous Vehicles
Teleoperator
Local Operator
1
2
“It would be impossible to have autonomous vehicles driving around without never needing help. Any autonomous system is built by humans for humans, the vehicles will have to interact with human beings”
M. Sierhuis, Director
"Limited FSD beta releasing on Tuesday next week, as promised. This will, at first, be limited to a small number of people who are expert and careful drivers"
Elon Musk, CEO,
Self-driving cars will require driver assistance for many years to come...”- John Krafcik, CEO of Waymo(Bloomberg, 14 November 2018)
“Our videos are showing work in progress and, as such, are not flawless.”
Shai Shalev-Shwartz, CTO of Mobileye and senior fellow at Intel (EE Times 7, October, 2020)
AGENDA
❏ Use cases for (tele) operators in the age of increasingly autonomous vehicles
❏ The challenges that (tele)operators face
❏ Ways to meet these challenges
❏ ANDROMEDA consortium
❏ Mobile-Lab
3
Teleoperator
Local Operator
Use Case 1: Robotaxis
4
Egil Juliussen 10.23.2020. Robotaxis: Where Are We?https://www.eetimes.com/robotaxis-where-are-we/#
Local OperatorPreliminary experiments with limited number of passengers
Special permits for operators
20XX
Local OperatorPaid service
20XX
Tele-OperatorsPaid service
20XX
Tele-OperatorsLimited, free service to
20XX
5
Use Case 2: Taxis (low level autonomy)
Source: Francis Kim (September 22, 2020). The future of rideshare is a remote control rental car
“A large-scale empirical evaluation study (based on data from Porto) ...shows that the implementation of the system can reduce, on average, the number of drivers to between 15% and 27% when considering teleoperation during pickup/dropoff and service, respectively” (d'Orey, et al. 2016, IEEE)
So what about these services?
Use case 3: Teleoperators Advising Autonomous Vehicles
6
1. Lundgard, A., Yang, Y., Foster, M. L., & Lasecki, W. S. (2018, April). Bolt: Instantaneous crowdsourcing via just-in-time training. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems (pp. 1-7).
2. Daw, A., Hampshire, R. C., & Pender, J. (2019). Beyond Safety Drivers: Staffing a Teleoperations System for Autonomous Vehicles. arXiv preprint arXiv:1907.12650.3. Hampshire, R. C., Bao, S., Lasecki, W. S., Daw, A., & Pender, J. (2020). Beyond safety drivers: Applying air traffic control principles to support the deployment of
driverless vehicles. PLoS one, 15(5), e0232837.
Human-in-the-loop AI look ahead assistance (adopted from Daw et al. 2019)
“Assuming a 10% penetration rate of driverless vehicles, we show that one remote driver can replace 14,840 human drivers” (Hampshire et al. 2020)
7
Andromeda: Challenges & Vision Autonomous Vehicles (AV) Among UsAVs of various types will soon move in human space (vehicles, taxis, couriers, trucks, drones, flying taxis)
● Millions of AVs from different manufacturers, operated by different service providers. How can we avoid total chaos?
● How will a remote human operator be able to effectively control a fleet of robots?
● How will the teleoperation and control transitions between the robot and the operator be safe?
Vision: Technological infrastructure for real-time remotely manage fleets of various types of mobile autonomous robotic systems for diverse fields where the ratio of teleoperators to vehicles is 1:1000
Characterization & execution
WG 1
Characterizing of use cases and needs, planning a demonstrator, business
analysisAerospace
Teleoperation
WG 4 ControlWG 3
Architecture
WG 2
,System interfaces, end-to-end information flow Definition and development of tools for dealing
with a limited/variable communicationAerospace
Characterization and development of human/control interfaces at the individual machine/fleet level,interfaces between fleets
Aerospace
HMI tools & AI, Training,Cognitive Load
Aerospace
Deliverables: Demonstrators, Business Analysis, Hackaton
Deliverables: Architectural characterization, application of algorithms for communication treatment
8
Andromeda: Partners
Human Factors Challenges for (Tele) OperatorsWork in progress...
9
Maintaining VigilanceSituation
AwarenessMotion
SicknessExperiencing AV Dynamics (Haptic sensing) Time lags in Data
Transmission
Allocating tasks between the AV and the Operator
Deskilling
AV can make robotic errors
Managing ‘Take Over Requests’
Developing Mental models
Monitoring the operation of the AV & the environment
Monitoring multiple AVs
Switching between AVs
Switching between Operating Interfaces
Spatial Perception
Operator motivation (Social distancing, Awareness to the state of the passenger)
Collaborative Work In Progress with:● Oren Musicant, Ariel University, Israel● Anat Meir, HIT, Israel● Einat Grimberg, The University of Queensland, Brisbane, Australia
ResponseTime
Self driving cars can make robotic errors
10
Missing baby buggy? Red box ghost appears on the visualization software
Where is the motorcycle?
Junko Yoshida
We’re grateful that Mobileye engaged in our public dialog. When more and more devices — including self-driving cars — are supposedly getting smarter than people, it becomes more and more urgent that we ask: “Show us how that works.”
Unedited Ride in Mobileye’s Camera-Driven Autonomous Vehicle
0:20: why the vehicle tried to stop?
0:30: It should move over to the other lane
0:50: Thinks the road is 2-way
1:43: Turn left come on - what are you doing?
2:05: Was going to crash making late left turn - Operator took control
3:30: Would have hit a curb while making right hand turn - Driver took control
3:50: Thinks the road is 2-way
4:53: Drifts too far to the right
6:30: Did not detect crossing car - Operator took control
Example for challenges: AV errors, Monitoring Automation, Mental Model...
Operator think,Tesla (FSD Beta Version) Does
Brandon M (23, Aug, 2020) youtube link
The Mobile-Lab
Mobile- Lab & Andromeda
14
Link
