human-centered artificial intelligenceriedl/talks/yconf.pdf · fortune favors the brave. harrison,...
TRANSCRIPT
-
Human-Centered Artificial IntelligenceMark Riedl [email protected] @mark_riedl
-
Alien intelligences
2
-
Alien intelligences• Artificial intelligences are
inscrutable to most humans
2
-
Alien intelligences• Artificial intelligences are
inscrutable to most humans
• Humans are inscrutable to artificial intelligences
2
-
3
Human-centered artificial intelligence
-
3
Understanding humans
Human-centered artificial intelligence
-
3
Understanding humans
Helping humans understand them
Human-centered artificial intelligence
-
3
Understanding humans
Helping humans understand them
Computational creativity
Human-centered artificial intelligence
-
3
Understanding humans
Helping humans understand them
Human-centered artificial intelligence
-
3
Understanding humans
Helping humans understand them
Challenges & opportunities
-
3
Understanding humans
Challenges & opportunities
-
Specifying goals
4
-
Specifying goals
4
-
Commonsense goal failure• Do what I want…
6
-
Commonsense goal failure• Do what I want…
6
… the way I would do it!
-
Commonsense goal failure• Do what I want…
6
… the way I would do it!
• Knowledge bases?
• Lots of sensors?
• Demonstration?
-
Learning from stories
7
• If computers could comprehend stories then humans can transfer commonsense procedural knowledge to computers by telling stories
-
Machine enculturation• Human cultural values are implicitly encoded in stories
told by members of a culture
• Allegorical tales
• Fables
• Contemporary fictional literature, TV, & movies
8
Riedl. CHI Workshop on Human-Centered Machine Learning, 2016.
-
Natural language• Natural language processing is not a solved problem
• Humans are noisy (variable)
• Humans shouldn’t need to know autonomous system capabilities or execution environment
9
-
Quixote• Reinforcement learning: AI devises a “program” for
operating in an environment through trial and error
• Intuition: Reward the agent for performing actions that mimic the stories that it has been told
10
Harrison & Riedl. AIIDE Conference, 2016.
-
Quixote
11
10
1015
Model learning
Trajectory tree creation
Reward assignment
Reinforcement learning
Exemplar stories A model A trajectory tree
A trajectory tree with events assigned reward valuesA policy mapping
states to actions
Environment
-
Quixote
11
10
1015
Model learning
Trajectory tree creation
Reward assignment
Reinforcement learning
Exemplar stories A model A trajectory tree
A trajectory tree with events assigned reward valuesA policy mapping
states to actions
Environment
-
12
choose restaurant
drive to restaurant
walk/go into restaurant
read menu
choose menu item
wait in line
drive to drive-thru
take out wallet place order
pay for food
wait for food
drive to window
get food
find table
sit down
eat food
clear trash
leave restaurant
drive home
Fast food restaurant
-
13
arrive at theatre
wait for ticket
go to ticket booth
buy tickets
choose movie
go to concession stand
order popcorn / soda show tickets
buy popcorn
enter theatre
find seats
turn off cellphone sit down
eat popcorn watch movie
hold handsuse bathroom discard trash
talk about movie
leave movie
drive home
kiss
Going on a date to the
movies
-
Quixote
14
10
1015
Model learning
Trajectory tree creation
Reward assignment
Reinforcement learning
Exemplar stories A model A trajectory tree
A trajectory tree with events assigned reward valuesA policy mapping
states to actions
Environment
-
• Fill gaps between events
Reinforcement learning
15
World state space
Leave House
Go to bank Go to hospital Go to doctor
Don't get prescription hospital Don't get prescription doctor
Get prescription hospital Get prescription doctorWithdraw money
Go to pharmacy
Buy strong drugs Buy weak drugs
Go home
Harrison & Riedl. AIIDE Conference, 2016.
-
• Fill gaps between events
Reinforcement learning
15
World state space
Leave House
Go to bank Go to hospital Go to doctor
Don't get prescription hospital Don't get prescription doctor
Get prescription hospital Get prescription doctorWithdraw money
Go to pharmacy
Buy strong drugs Buy weak drugs
Go home
Harrison & Riedl. AIIDE Conference, 2016.
-
• Fill gaps between events
Reinforcement learning
15
World state space
Leave House
Go to bank Go to hospital Go to doctor
Don't get prescription hospital Don't get prescription doctor
Get prescription hospital Get prescription doctorWithdraw money
Go to pharmacy
Buy strong drugs Buy weak drugs
Go home
leave house
Harrison & Riedl. AIIDE Conference, 2016.
-
• Fill gaps between events
Reinforcement learning
15
World state space
Leave House
Go to bank Go to hospital Go to doctor
Don't get prescription hospital Don't get prescription doctor
Get prescription hospital Get prescription doctorWithdraw money
Go to pharmacy
Buy strong drugs Buy weak drugs
Go home
leave house
go bank
go hospital
go doctor
Harrison & Riedl. AIIDE Conference, 2016.
-
• Fill gaps between events
Reinforcement learning
15
World state space
Leave House
Go to bank Go to hospital Go to doctor
Don't get prescription hospital Don't get prescription doctor
Get prescription hospital Get prescription doctorWithdraw money
Go to pharmacy
Buy strong drugs Buy weak drugs
Go home
leave house
go bank
go hospital
go doctor
Drive Main St.
Stairs
Harrison & Riedl. AIIDE Conference, 2016.
-
Machine enculturation• Social conventions prevent conflict
• Robots that follow the “rules” of society will be safer
17
Riedl. CHI Workshop on Human-Centered Machine Learning, 2016.
-
Challenges & opportunities
18
Understanding humans
Helping humans understand them
-
Challenges & opportunities
18
Helping humans understand them
-
Autonomous system failures
19
-
Possible solution: open the black box
-
AI rationalization
21
Harrison, Ehsan, Riedl. arXiv 1702.07826, 2017.
-
AI rationalization• Creating an explanation comparable to what a human
would say if he or she were controlling the robot in the same situation
21
Harrison, Ehsan, Riedl. arXiv 1702.07826, 2017.
-
AI rationalization• Creating an explanation comparable to what a human
would say if he or she were controlling the robot in the same situation
• Takes inspiration from what humans do
21
Harrison, Ehsan, Riedl. arXiv 1702.07826, 2017.
-
AI rationalization• Creating an explanation comparable to what a human
would say if he or she were controlling the robot in the same situation
• Takes inspiration from what humans do
• Human understandable
21
Harrison, Ehsan, Riedl. arXiv 1702.07826, 2017.
-
AI rationalization• Creating an explanation comparable to what a human
would say if he or she were controlling the robot in the same situation
• Takes inspiration from what humans do
• Human understandable
• Helps build trust; useful in time-critical situations
21
Harrison, Ehsan, Riedl. arXiv 1702.07826, 2017.
-
Neural machine translation
23
Harrison, Ehsan, Riedl. arXiv 1702.07826, 2017.
-
Neural machine translation
23
4 10 3 0 3 0 3 0 3 3 3 2 2 2 3 3 -1 -1 -1 -1 -1 -1 -1 2 3 3 2 3 2 3 2 2 2 3 3 3 3 2 2 3 2 3 2 2 0 0 0 0 0 0 0 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 0 0 0 0 1 1 1 1 0 0 0 1 1 0 0 0 0 1 1 0 0 0 0 0 0 0 0 0 1 3 10 -1 -1 -1 -1 -1 -1 -1 2 3 3 2 2 2 3 3 2 2 3 3 3 2 3 2 3 3 2 3 2 -1 -1 -1 -1 -1 -1 -1 3 2 2 3 2 3 2 0 0 0 0 0 0 0 0 0 1 0 1 0 1 0 0 0 1 1 1 0 0 0 0 0 0 1 1 0 0 0 1 1 0 0 1 0 0 0 1 1 0 -1 -1 -1 -1 -1 -1 -1
Woah! Car beside me and a gap above. Fortune favors the brave.
Harrison, Ehsan, Riedl. arXiv 1702.07826, 2017.
-
Neural machine translation
24
Harrison, Ehsan, Riedl. arXiv 1702.07826, 2017.
-
Neural machine translation
24
4 10 3 0 3 0 3 0 3 3 3 2 2 2 3 3 -1 -1 -1 -1 -1 -1 -1 2 3 3 2 3 2 3 2 2 2 3 3 3 3 2 2 3 2 3 2 2 0 0 0 0 0 0 0 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 0 0 0 0 1 1 1 1 0 0 0 1 1 0 0 0 0 1 1 0 0 0 0 0 0 0 0 0 1 3 10 -1 -1 -1 -1 -1 -1 -1 2 3 3 2 2 2 3 3 2 2 3 3 3 2 3 2 3 3 2 3 2 -1 -1 -1 -1 -1 -1 -1 3 2 2 3 2 3 2 0 0 0 0 0 0 0 0 0 1 0 1 0 1 0 0 0 1 1 1 0 0 0 0 0 0 1 1 0 0 0 1 1 0 0 1 0 0 0 1 1 0 -1 -1 -1 -1 -1 -1 -1
Woah! Car beside me and a gap above. Fortune favors the brave.
Harrison, Ehsan, Riedl. arXiv 1702.07826, 2017.
-
AI Rationalization
25
-
AI Rationalization• Target users are those without technical backgrounds
25
-
AI Rationalization• Target users are those without technical backgrounds
• Meant to convey fast, approximate explanations
25
-
AI Rationalization• Target users are those without technical backgrounds
• Meant to convey fast, approximate explanations
• Meant to foster rapport and trust
25
-
AI Rationalization• Target users are those without technical backgrounds
• Meant to convey fast, approximate explanations
• Meant to foster rapport and trust
• Coupled with more thorough explanations & visualizations
25Work by Alex Endert,
Georgia Tech
-
Challenges & opportunities
26
Understanding humans
Helping humans understand them
-
Challenges & opportunities
26
Understanding humans
Helping humans understand them
-
Understanding helps AI
27
4 10 3 0 3 0 3 0 3 3 3 2 2 2 3 3 -1 -1 -1 -1 -1 -1 -1 2 3 3 2 3 2 3 2 2 2 3 3 3 3 2 2 3 2 3 2 2 0 0 0 0 0 0 0 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 0 0 0 0 1 1 1 1 0 0 0 1 1 0 0 0 0 1 1 0 0 0 0 0 0 0 0 0 1 3 10 -1 -1 -1 -1 -1 -1 -1 2 3 3 2 2 2 3 3 2 2 3 3 3 2 3 2 3 3 2 3 2 -1 -1 -1 -1 -1 -1 -1 3 2 2 3 2 3 2 0 0 0 0 0 0 0 0 0 1 0 1 0 1 0 0 0 1 1 1 0 0 0 0 0 0 1 1 0 0 0 1 1 0 0 1 0 0 0 1 1 0 -1 -1 -1 -1 -1 -1 -1
Woah! Car beside me and a gap above. Fortune favors the brave.
-
Understanding helps AI
27
4 10 3 0 3 0 3 0 3 3 3 2 2 2 3 3 -1 -1 -1 -1 -1 -1 -1 2 3 3 2 3 2 3 2 2 2 3 3 3 3 2 2 3 2 3 2 2 0 0 0 0 0 0 0 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 0 0 0 0 1 1 1 1 0 0 0 1 1 0 0 0 0 1 1 0 0 0 0 0 0 0 0 0 1 3 10 -1 -1 -1 -1 -1 -1 -1 2 3 3 2 2 2 3 3 2 2 3 3 3 2 3 2 3 3 2 3 2 -1 -1 -1 -1 -1 -1 -1 3 2 2 3 2 3 2 0 0 0 0 0 0 0 0 0 1 0 1 0 1 0 0 0 1 1 1 0 0 0 0 0 0 1 1 0 0 0 1 1 0 0 1 0 0 0 1 1 0 -1 -1 -1 -1 -1 -1 -1
Woah! Car beside me and a gap above. Fortune favors the brave.
-
Punchline
28
Training iterations (x100)
Aver
age
rew
ard
-
Punchline
28
Training iterations (x100)
Aver
age
rew
ard
Standard Q-learning
-
Punchline
28
Training iterations (x100)
Aver
age
rew
ard
Standard Q-learning
Learning from demonstration
-
Punchline
28
Training iterations (x100)
Aver
age
rew
ard
Standard Q-learning
Learning from demonstration
Language-based guidance
-
29
Understanding humans
Helping humans understand them
Computational creativity
Human-centered artificial intelligence
-
29
Computational creativity
Human-centered artificial intelligence
-
Computational creativity
30
-
Computational creativity
30
-
Computational creativity
30
-
Computational creativity
30
-
Computational creativity
31
-
Computational creativity• Most computational creativity is learning a pattern from
data and trying to make new inputs fit the pattern
31
-
Computational creativity• Most computational creativity is learning a pattern from
data and trying to make new inputs fit the pattern
• AI can’t reach human-level creativity without making intuitive leaps
31
-
Computational creativity• Most computational creativity is learning a pattern from
data and trying to make new inputs fit the pattern
• AI can’t reach human-level creativity without making intuitive leaps
• AI can’t augment human creativity if AI can’t keep up with human collaborator’s intuitive leaps
31
-
Computational creativity• Most computational creativity is learning a pattern from
data and trying to make new inputs fit the pattern
• AI can’t reach human-level creativity without making intuitive leaps
• AI can’t augment human creativity if AI can’t keep up with human collaborator’s intuitive leaps
• Computational creativity is about making AI gracefully handle novel situations it was never trained for
31
-
32
+ = ?
-
Concluding thoughts• AI appears less “alien”
• Maybe safer?
• Computational creativity to handle contingencies very different from input
• Human-centered AI is an essential mix of capabilities for robots in the human world
33