United Nations Group of Governmental Experts Meeting:Lethal Autonomous Weapons Systems
Military Effects Panel
Dr. Lydia Kostopoulos@LKCYBER
November 14, 2017
College of Information and CyberspaceNational Defense University
U.S.A.
DISCLAIMER:
The views expressed in this presentation are those of the author and do not necessarily reflectthe views of the National Defense University, the College of Information and Cyberspace, or theUnited States Government.
TRUSTCULTURE AVAILABILITY
Autonomous machines perform as expected and within parameters outlined by human.
• Authenticity of the machine learning environment.
• Integrity of the data used.
Source: Geospatial World
HumanOnly
MachineOnlyHuman is
informed by machine.
Human is assisted by machine.
Human supervises work done by machine.
Close visual supervision
Remote supervision
Supervision through
alerts
With ability for human to modify, intervene and change or stop order/process.
Human gives command
without ability to change
course.
Human has confidence in
outcome.
Machine performs a specific, unambiguous task
Human gives a command in the form of objectives with guidance.
The machine has the the authority to execute full command without
reporting back.
Human has ability to
track machine’s activities.
Human has no ability to
track machine’s activities.
Machine performs a specific, unambiguous, or ambiguous task
Human-Machine Teaming Range for Lethal Autonomous Weapons Systems Discussion
Lydia Kostopoulos, PhD2017
Human gives a command in the form
of objectives with guidance.
The machine does not have the authority to execute full command
without reporting back.
Human has ability to track, modify,
intervene and/or stop the order or process.
Human has ability to modify,
intervene and/or stop the order or
process.
MachineOnlyHuman gives a command in the form of objectives with
guidance.
The machine has the the authority to execute full command without
reporting back.
Human has ability to
track machine’s activities.
Human has no ability to
track machine’s activities.
Machine performs a specific, unambiguous, or ambiguous task
Human gives a command in the form
of objectives with guidance.
The machine does not have the authority to execute full command
without reporting back.
Human has ability to track, modify,
intervene and/or stop the order or process.
Human has ability to modify,
intervene and/or stop the order or
process.
I. Reduction of cost (through process optimization, efficient management of limited resources)
II. Reducing burden on the soldier/commander (physically or intellectually)
Confidence Growth
III. Successful synthetic training & physical testing Demonstrated ability to successfully:- Operate on intent with minimal to no call-back for additional information or permissions.- Assimilate knowledge transfer from other autonomous systems’ data, experience, decisions etc (speed advantage).- Operate within “moral” bounds/limits assigned to the algorithm complying with rules of distinction and proportionality; while finding creative solutions to achieve mission objectives.- Communicate (to humans) actionable anticipatory intelligence in meaningful ways that directly fall inline with its mission and mandate; and to autonomously act successfully with the incorporation of anticipatory intelligence (speed advantage).
IV. Explainable AI/AlgorithmsAbility to explain (and confidently “fix”) AI algorithms will lead to greater trust in (expected) outcomes.
Detect, Select, Engage
TRUSTCULTURE
AVAILABILITY
“New” technology is normal
Anecdote
New technology is novel and interesting
New technology goes against
human nature
Digital Culture EvolutionFood for thought
• Acceptability of changing technology in battlefield, seeing technology as one with AI and a justifiable and “normal” and fair use of military tech assets to achieve mission success and political objectives.
• Moving forward, as digital natives become commanders, their perception of what types of weapons are acceptable in the battlefield may be different from ours.
• An AI 9/11 style attack conducted by a terrorist organization could push society to embrace the need to incorporate machine learning in all facets of society with a renewed drive to make LAWS fit within the values of our society.
TRUST CULTUREAVAILABILITY
• Availability refers to the availability of the technology to be used.
• The democratization of technologies also will reach hostile non-state political actors who do not abide by international norms.
• Tech won’t matter as much as creativity in it’s use that will bring decisive advantage.
CULTURE AVAILABILITY TRUST
Asymmetric Force Application: Alternative perspective for non-state hostile actors
[ex: terrorist organizations]