an ethics code for automated and connected driving€¦ · the introduction of more highly...
TRANSCRIPT
Prof. Dr. Christoph Lütge
Director, TUM Institute for Ethics in Artificial Intelligence
Technical University of Munich
An Ethics Code for Automated and Connected Driving
Peter Löscher Chair of Business Ethics and Global GovernanceTUM School of GovernanceTechnical University Munich
2
Autonomous Driving: Enthusiasm and Mistrust Related to Age
39%36%
33%
24%
14%
54%
63%60%
72%
80%
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
14-29 Jahre 30-39 Jahre 40-49 Jahre 50-59 Jahre 60+ Jahre
Begeisertung Misstrauen
Source: Bertelsmann Stiftung (2017), S. 3.
Peter Löscher Chair of Business Ethics and Global GovernanceTUM School of GovernanceTechnical University Munich
3
Autonomous Driving: Reasons for Skepticism in Germany
55%
56%
62%
74%
83%
84%
Verlust der Privatsphäre
Datendiebstahl
Fehlender Fahrspaß
Fremdkontrolle des Fahrzeugs durch Hackerangriffe
Verlust der eignenen Kontrolle über das Auto
Angst vor Unfällen
Source: Bertelsmann Stiftung (2017), S. 4.
Peter Löscher Chair of Business Ethics and Global GovernanceTUM School of GovernanceTechnical University Munich
Accidents involving autonomous cars:
4
Uber Volvo XC90
was hit.
▪Police: Autonomous
technology was not
responsible
▪No injuries
März 2017:
Fatal accident with
Tesla X
▪Autopilot was
activated
▪Driver ignored
several warnings
▪1 dead
23. März 2018:
Uber AV hitting
woman
▪Car driving
autonomously
▪Sensor technology
was deactivated on
purpose (false
positives)
▪1 dead
20. März 2018:
Tesla S colliding with
police car
▪Autopilot activated
▪Tesla: even with
autopilot activated,
drivers still need to
keep control of the
car
▪1 injured
29. Mai 2018:
Sources: manager magazin (2017); auto motor sport (2018); Welt (2018); tagesschau (2018).
Peter Löscher Chair of Business Ethics and Global GovernanceTUM School of GovernanceTechnical University Munich
Autonomous Driving
• Autonomous cars:
• Major problem of cars: High fatality rates
• Car accidents caused by human error by ca. 90-95%
• Autonomous cars as solution to reduce car accidents
• In Germany, this would mean ultimately saving around 3,000 lives each year – which clearly is
an ethical goal in itself.
But: Rules needed!
Prof. Dr. Christoph Lütge 5
“The introduction of more highly automated driving systems,
especially with the option of automated collision prevention, may be
socially and ethically mandated if it can unlock existing potential for
damage limitation.” (German Ethics Code for Automated and Connected Driving, 2017)
Peter Löscher Chair of Business Ethics and Global GovernanceTUM School of GovernanceTechnical University Munich
• Some problems cannot be addressed by manufacturers alone
• In particular: questions of liability:
• Who is to be held accountable in case of a crash?
• Which cars are to be licensed?
• These questions require a legal foundation in order to make
costs and benefits calculable for manufacturers
• If well-designed, such guidelines will in turn not restrain,
but accelerate the development of new technologies
Prof. Dr. Christoph Lütge | The New German Ethics Code for Autonomous Cars
The Need for a Code (for Manufacturers)
6
Peter Löscher Chair of Business Ethics and Global GovernanceTUM School of GovernanceTechnical University Munich
• More fundamentally, the introduction of autonomous cars
poses larger questions of technology management:
• How much dependence on technologically complex systems
(e.g. AI and machine learning) are we willing to accept in order
to achieve, in return, more safety, mobility and convenience?
• What precautions need to be taken to ensure controllability,
transparency and data autonomy?
• How can technological development be
advanced without blurring the contours of
a human society that places the freedom
and integrity of individuals and social
respect at the heart of its legal regime?
Prof. Dr. Christoph Lütge | The New German Ethics Code for Autonomous Cars
The Need for a Code (for Society at large)
7
Source: bpb.de
Peter Löscher Chair of Business Ethics and Global GovernanceTUM School of GovernanceTechnical University Munich
• Before 2017, a number of local detail regulations on
AVs had already been passed
• However no broader ethics code for AVs
• Against this background, the “Ethics Commission
on Automated and Connected Driving”
was appointed by the German Federal
Minister of Transport and Digital
Infrastructure in September 2016
Prof. Dr. Christoph Lütge | The New German Ethics Code for Autonomous Cars
The German Ethics Code for Automatedand Connected Driving – Procedure
8
Source: mercedes-benz.com
Peter Löscher Chair of Business Ethics and Global GovernanceTUM School of GovernanceTechnical University Munich
• The committee consisted
of 14 experts from a wide
range of disciplines:
• philosophy
• law
• the automotive industry
• consumer organisation
• In addition, external experts from other disciplines were consulted
• Head was Udo di Fabio, former Federal Constitutional Court judge
Prof. Dr. Christoph Lütge | The New German Ethics Code for Autonomous Cars
Procedure
9
Source: bmvi.de
Peter Löscher Chair of Business Ethics and Global GovernanceTUM School of GovernanceTechnical University Munich
• In June 2017, the Commission
presented its final report
• The core of the report are 20
“Ethical rules for automated
and connected vehicular traffic”:
1. The primary purpose of partly and fully automated transport
systems is to improve safety for all road users. Another purpose
is to increase mobility opportunities and to make further benefits
possible. Technological development obeys the principle of
personal autonomy, which means that individuals enjoy
freedom of action for which they themselves are responsible.
Prof. Dr. Christoph Lütge | The New German Ethics Code for Autonomous Cars
Positions: The Code
10
Source: miscw.com
Peter Löscher Chair of Business Ethics and Global GovernanceTUM School of GovernanceTechnical University Munich
2. The protection of individuals takes
precedence over all other utilitarian
considerations.
• The objective is to reduce the level of
harm until it is completely prevented
• The licensing of automated systems is
not justifiable unless it promises to
produce at least a diminution in harm
compared with human driving,
i.e. a positive balance of risks
Prof. Dr. Christoph Lütge | The New German Ethics Code for Autonomous Cars
Positions
11
Source: rci-online.org
Peter Löscher Chair of Business Ethics and Global GovernanceTUM School of GovernanceTechnical University Munich
3. The public sector is responsible for guaranteeing
the safety of the automated and connected systems
introduced and licensed in the public street environment.
• Driving systems need official licensing and monitoring
• The guiding principle is the avoidance of accidents, although
technologically unavoidable residual risks do not militate against
the introduction of
automated driving if
the balance of risks
is very positive
Prof. Dr. Christoph Lütge | The New German Ethics Code for Autonomous Cars
Positions
12
Source: imd.org; creators-rev.com
Peter Löscher Chair of Business Ethics and Global GovernanceTUM School of GovernanceTechnical University Munich
4. The personal responsibility of individuals for taking decisions is
an expression of a society centred on individual human beings.
• The purpose of all governmental and political regulatory
decisions is to promote the free development and
the protection of individuals
• Technology is to be regulated so that a balance is achieved
between maximum personal freedom of choice and the freedom
and safety of others
Prof. Dr. Christoph Lütge | The New German Ethics Code for Autonomous Cars
Positions
13
Peter Löscher Chair of Business Ethics and Global GovernanceTUM School of GovernanceTechnical University Munich
5. Automated and connected technology should prevent accidents
wherever this is practically possible.
• Based on the state of the art, the technology must be designed in
such a way that critical situations do not arise in the first place.
• the entire spectrum of technological options should be used and
constantly evolved:
• limiting the scope of application to controllable traffic environments
• preventing hazards by means of “intelligent” road infrastructure
• vehicle sensors and braking performance
• signals for persons at risk
Prof. Dr. Christoph Lütge | The New German Ethics Code for Autonomous Cars
Positions
14
Peter Löscher Chair of Business Ethics and Global GovernanceTUM School of GovernanceTechnical University Munich
6. The introduction of more highly automated driving systems,
especially with the option of automated collision prevention,
may be socially and ethically mandated if it can unlock new
potentials for damage limitation.
In contrast, mandatory fully automated transport systems
or practical inescapabilty would be ethically questionable
if it entailed submission to technological imperatives.
Prof. Dr. Christoph Lütge | The New German Ethics Code for Autonomous Cars
Positions
15
Source: scienceabc.com
Peter Löscher Chair of Business Ethics and Global GovernanceTUM School of GovernanceTechnical University Munich
7. In hazardous situations that prove to be unavoidable, the
protection of human life enjoys top priority in a balancing of
legally protected interests.
• Thus, within the constraints of what is
technologically feasible, the systems
must be programmed to accept damage
to animals or property in a conflict
if this means that personal injury
can be prevented.
Prof. Dr. Christoph Lütge | The New German Ethics Code for Autonomous Cars
Positions
16
Source: gearheads.org
Peter Löscher Chair of Business Ethics and Global GovernanceTUM School of GovernanceTechnical University Munich
8. Genuine dilemmatic decisions, such as a decision between one
human life and another, depend on the actual specific situation
and the “unpredictable” behaviour by parties affected.
They can thus not be clearly standardized, nor can they be
programmed such that they are ethically unquestionable.
• Technological systems must be
designed to avoid accidents
• they can’t replace or anticipate
the decision of a responsible
driver with the moral capacity
to make correct judgements
Prof. Dr. Christoph Lütge | The New German Ethics Code for Autonomous Cars
Positions
17
Source: autocar.co-uk
Peter Löscher Chair of Business Ethics and Global GovernanceTUM School of GovernanceTechnical University Munich
• A human driver would be acting unlawfully if he killed a person
in an emergency to save the lives of one or more other persons,
but he would not necessarily be acting culpably
• Such retrospective legal judgements cannot be transformed into
general ex ante appraisals or corresponding programming
• Rather, an independent public
sector agency should monitor
and process lessons learned
(e.g. a “Federal Bureau for the
Investigation of Accidents
Involving Automated
Transport Systems”)
Prof. Dr. Christoph Lütge | The New German Ethics Code for Autonomous Cars
Positions
18
Source: gearheads.org
Peter Löscher Chair of Business Ethics and Global GovernanceTUM School of GovernanceTechnical University Munich
9. In the event of unavoidable accident situations, any distinction
based on personal features is strictly prohibited.
• E.g. based on age, gender, physical or mental constitution
• It is also prohibited to offset victims against one another
• General programming to reduce the number
of personal injuries may be justifiable
• Those parties involved in the
generation of mobility risks
must not sacrifice non-
involved parties
Prof. Dr. Christoph Lütge | The New German Ethics Code for Autonomous Cars
Positions
19
Source: wixstatic.com
Peter Löscher Chair of Business Ethics and Global GovernanceTUM School of GovernanceTechnical University Munich
10. For automated and connected driving systems,
the accountability shifts from the motorist to:
• manufacturers and operators of technological systems
• bodies responsible for taking decisions on
infrastructure, policy and legal decisions
11. Liability for damage caused by automated driving systems is
governed by the same principles as in other product liability.
• Manufacturers and operators are thus obliged to continuously
optimize their systems and also to observe systems they have
already delivered and to improve them where this is
technologically possible and reasonable.
Prof. Dr. Christoph Lütge | The New German Ethics Code for Autonomous Cars
Positions
20
Source: eqgroup.com
Peter Löscher Chair of Business Ethics and Global GovernanceTUM School of GovernanceTechnical University Munich
12. The public is entitled to be informed about new technologies
and their deployment in a sufficiently differentiated manner.
Guidance for the deployment and
programming of automated vehicles
should be derived in a form that is
• as transparent as possible
• communicated in public
• reviewed by an independent body
Prof. Dr. Christoph Lütge | The New German Ethics Code for Autonomous Cars
Positions
21
Peter Löscher Chair of Business Ethics and Global GovernanceTUM School of GovernanceTechnical University Munich
13. The complete connectivity and central control of all
motor vehicles is ethically questionable if a total
surveillance of road users and manipulation
of vehicle control cannot safely be ruled out.
14. Automated driving is justifiable only if attacks,
in particular manipulation of the IT system or
innate system weaknesses, do not shatter
people’s confidence in road transport.
15. Owners and users decide if their vehicle data
that are generated are to be forwarded and used.
Prof. Dr. Christoph Lütge | The New German Ethics Code for Autonomous Cars
Positions
22Source: auone.jp; sophos.com; waxdigital.com
Peter Löscher Chair of Business Ethics and Global GovernanceTUM School of GovernanceTechnical University Munich
16. It must be possible to clearly distinguish if a driverless system is
being used or if a driver retains accountability with the option of
overruling the system.
• Human-machine interfaces must be designed so that it is clearly
apparent on which side individual control responsibilities lie
• Human-to-technology handovers (and vice versa) should be
documented and stored
• International standardization
of the handover and logging
procedures is to be sought
Prof. Dr. Christoph Lütge | The New German Ethics Code for Autonomous Cars
Positions
23
Peter Löscher Chair of Business Ethics and Global GovernanceTUM School of GovernanceTechnical University Munich
17. The software and technology in highly automated vehicles must
be designed such that the need for an abrupt handover of
control to the driver (“emergency”) is virtually obviated.
• To enable efficient, reliable and secure human-machine
communication and prevent overload, the systems must adapt
to human communicative behaviour (rather than vice versa)
18. Self-learning systems are only ethically
allowed if they generate safety gains.
• They must not be deployed unless they
meet the safety requirements for functions
relevant to vehicle control
Prof. Dr. Christoph Lütge | The New German Ethics Code for Autonomous Cars
Positions
24Source: roblox.com; ccli.com
Peter Löscher Chair of Business Ethics and Global GovernanceTUM School of GovernanceTechnical University Munich
19. In emergency situations, the vehicle must
autonomously, without human assistance,
enter into a “safe condition”.
• Harmonization, especially of the definition of
safe conditions or of handover routines, is desirable
20. The proper use of automated systems should
form part of people’s general digital education.
• The proper handling of automated driving systems
should be taught in during driving tuition and tested
Prof. Dr. Christoph Lütge | The New German Ethics Code for Autonomous Cars
Positions
25
Source: teknoaware.com; llowvelder.com
Peter Löscher Chair of Business Ethics and Global GovernanceTUM School of GovernanceTechnical University Munich
26
The Moral Machine (Nature 2018)
• Findings/conclusions: • Cross cultural ethical variations: 3 major
clusters of countries (see figures)
• „Differences corelate with modern
instituions and deep cultural traits“
• E.g. „Collectivist cultures, which
emphasize the respect that is due to
older members of the community,
show(ing) a weaker preference for
sparing younger characters.“
• Authors state that their findings can
„contribute to developing global,
socially acceptable principles for
machine ethics.”
Source of images (b) & quotes: Awad et al. (2018)
Peter Löscher Chair of Business Ethics and Global GovernanceTUM School of GovernanceTechnical University Munich
29
The Moral Machine (Nature 2018)
• Findings/conclusions: • Cross cultural ethical variations: 3 major
clusters of countries (see figures)
• „Differences corelate with modern
instituions and deep cultural traits“
• E.g. „Collectivist cultures, which
emphasize the respect that is due to
older members of the community,
show(ing) a weaker preference for
sparing younger characters.“
• Authors state that their findings can
„contribute to developing global,
socially acceptable principles for
machine ethics.”
Source of images (b) & quotes: Awad et al. (2018)
Peter Löscher Chair of Business Ethics and Global GovernanceTUM School of GovernanceTechnical University Munich
27
Findings of Moral Machine vs. Ethics Code
Rather kill one than many
Children should rather be
spared than elderly
How should we deal with illegally
crossing pedestrians? There are
different tendencies in different
cultures
Moral Machine Ethics Code
No recommendation
Any distinction based on
personal features, such as
age, should be prohibited vs
children
Is it possible to detect the
person who is responsible for
the dilemma situation?
Peter Löscher Chair of Business Ethics and Global GovernanceTUM School of GovernanceTechnical University Munich
28
Findings of Moral Machine vs. Ethics Code
Children should rather be
spared than elderly
How should we deal with illegally
crossing pedestrians? There are
different tendencies in different
cultures
Problems
Constitutional law does not
differentiate between young
and old when dealing with e.g.
value of life, murderer etc.
Is it possible to have programs
to act according to the drivers
morality, especially when there
are various differences in
cultures?
Peter Löscher Chair of Business Ethics and Global GovernanceTUM School of GovernanceTechnical University Munich
29
Critique: Moral Machine
• Programming Away Human Rights and
Responsibilities? (Kochupillai & Lütge, 2019)
• Well established categories of Human Rights:
• Absolute non-derogable rights – Right to Life
• Absolute derogable rights
• Qualified Rights – Right to data protection &
privacy
• Recital 4 of GDPR: “The right to the protection of
personal data is not an absolute right; it must be
considered in relation to its function in society and be
balanced against other fundamental rights, in
accordance with the principle of proportionality.”
• Article 20, GDPR: “The data subject shall have the
right to have the personal data transmitted directly
from one controller to another, where technically
feasible.”
(1) Kochupillai, M. & Lütge C. Programming Away Human Rights
& Responsibilities? A Response to Awad et al. (Forthcoming)
Peter Löscher Chair of Business Ethics and Global GovernanceTUM School of GovernanceTechnical University Munich
30
Critique: Moral Machine
• Needed: More experimental &
empirical research for determining
how to organize qualified rights to
balance out competing concerns – e.g.
privacy v. safety. (Kochupillai & Lütge,
2019)
• Focus of AVs research needs to move
away from resolving dilemma
situations to finding means of ensuring
they do not arise at all – e.g. by re-
organizing infrastructure, GDPR
compliant data collection for efficient
operation of V2x systems (Kochupillai &
Lütge, 2019)
(1) Kochupillai, M. & Lütge C. Programming
Away Human Rights & Responsibilities? A
Response to Awad et al. (Forthcoming)
Source: Tabarrock, A. The Google-Trolley
Problem, June 15, 2012
https://marginalrevolution.com/
Peter Löscher Chair of Business Ethics and Global GovernanceTUM School of GovernanceTechnical University Munich
• The German Ethics Code for Automated Driving does not
address / leaves open a large number of questions, because
• Technologies not advanced enough
• Policies are up to the political system - and not to an ethics
committee
• No agreement on ethical questions could be reached
• Examples:
• Overruling?
• Irreversibility?
• Power Centralisation?
• Mandatory Introduction at some point?
Prof. Dr. Christoph Lütge | The New German Ethics Code for Autonomous Cars
Outlook: Open Questions
31
32
Thank you very much for your attention!
Prof. Dr. Christoph Lütge