humanoides- osaka university
TRANSCRIPT
-
8/19/2019 Humanoides- Osaka University
1/46
CNR101x Week2 Handout
© 2016 Osaka University. All Rights Reserved.
1
Android Science
Distinguished Professor of Osaka University
Visiting Director & Fellow of ATR Hiroshi Ishiguro Laboratories
www.is.sys.es.osaka-u.ac.jp, www.geminoid.jp
1
Hiroshi
ISHIGURO
2
Robot Society
-
8/19/2019 Humanoides- Osaka University
2/46
CNR101x Week2 Handout
© 2016 Osaka University. All Rights Reserved.
2
3
4
RobotSociety
Android
Android
Human
-
8/19/2019 Humanoides- Osaka University
3/46
CNR101x Week2 Handout
© 2016 Osaka University. All Rights Reserved.
3
5
Why do we use thehumanoid?
Humans have brains that
recognize humans
The ideal interface for
humans are humans.
Studies on humans and
humanoids are studies on
principle of interface.
6
-
8/19/2019 Humanoides- Osaka University
4/46
CNR101x Week2 Handout
© 2016 Osaka University. All Rights Reserved.
4
8
Near future society
supported by humanoids and android
Elderly care
Public places Learning
Department stores Stations
Humanoids that interact
with humans by using
multiple modalities
-
8/19/2019 Humanoides- Osaka University
5/46
CNR101x Week2 Handout
© 2016 Osaka University. All Rights Reserved.
5
9
Research topics in the past and future
Neuro-
science
Robotics Pattern
Recognition
Information Society
1990 Cerebellum Mobile robot
Kansei
Model-based
Parametric
Web service
Sharing information
2000 Cerebrum Active visionBehaviorism
Learning
Learning XXX Social NetworkSociety on the net
2010 BMI Interaction
Development
Deep Learning Smart Phone
Real time small society2020 Frontal
cortex
Intention
Language
Intention
Consciousness
Society
Language
Non-parametric
Big data
Language
Personal Agent
Language processing
10
Android Science
-
8/19/2019 Humanoides- Osaka University
6/46
CNR101x Week2 Handout
© 2016 Osaka University. All Rights Reserved.
6
Robotics for understanding humans
Appearance
Movement
Perception
Conversation
Learning and Development of the softwareBio-mimetic mechanisms
Human
11
12
Studies on humanlikeness
-
8/19/2019 Humanoides- Osaka University
7/46
CNR101x Week2 Handout
© 2016 Osaka University. All Rights Reserved.
7
13
Development of interactive robots
Studies on Human-Robot Interaction300 action patterns and 700 transition
rules
14
Robots working in a shopping mallCombination of robots and sensor networks
Semi-autonomous robots
-
8/19/2019 Humanoides- Osaka University
8/46
CNR101x Week2 Handout
© 2016 Osaka University. All Rights Reserved.
8
15
Development of child androids
Very humanlike appearanceBut not humanlike movements
内部メカニズム
女性アンドロイド16
Android: very humanlike humanoid robot
-
8/19/2019 Humanoides- Osaka University
9/46
CNR101x Week2 Handout
© 2016 Osaka University. All Rights Reserved.
9
Subconscious and reactive movements
Subconscious movement Reactive movement• The subconscious movement gives human likeness to the android. However,
professionals find the difference.
• Reactive movements of a human is more complicated since it has mental states.
• Studies on cognitive and brain science improve the android.
17
Overcoming the uncanny valley
Neuroscience
study on
uncanny valley
The female android is not uncanny anymore.
Cognitive science and neuroscience with androids
Android Science: Integration of human studies and robotics
18
Similarity
F a m i l i a r i t y
Doll
Toy robot
Uncanny
valley
Human
Moving corpse100%
Child android
Adult
android
-
8/19/2019 Humanoides- Osaka University
10/46
CNR101x Week2 Handout
© 2016 Osaka University. All Rights Reserved.
10
Science
Engineering PatternRecognition
MechanicalEngineering
Understanding onHuman behaviors
Hypothesis
and
verification
Android Science
Robotics and Cognitive Science
Scientific issue
Human likeness (appearance , movement, perception)
Engineering issue
Simple and interactive communication tasks
Systems
Engineering
Brain
science
Psychology
RoboticsDevelopment of mechanical humans
Cognitive
science
20
Practical application ofinteractive robots
-
8/19/2019 Humanoides- Osaka University
11/46
CNR101x Week2 Handout
© 2016 Osaka University. All Rights Reserved.
11
21
Human archiving
Representation of human presence
Eternal life as a national treasure
22
Android vs. the person
Which presence is more strong?
What is the human presence?
-
8/19/2019 Humanoides- Osaka University
12/46
CNR101x Week2 Handout
© 2016 Osaka University. All Rights Reserved.
12
23
Story-telling android
24
The original show window
Emotional states and 65 behaviors
Reactions to visitors
Practical role of the android
-
8/19/2019 Humanoides- Osaka University
13/46
CNR101x Week2 Handout
© 2016 Osaka University. All Rights Reserved.
13
Sensor
25
Kinect
Camera
The control system
Sensor info
processionHow many people
How many peoplelook at F
Position of people
Gesture of People
Basic motionNeutral
(Waiting)
Looking around
Behavior model
Gazing someone
Looking around … Emotion
control O u t p u t
Emotion
-1HappyUnhappy
Arousal
1
1
PeaceDepression
Tension Excitation
Normal Happy Sad
Russel’s Model
-
8/19/2019 Humanoides- Osaka University
14/46
CNR101x Week2 Handout
© 2016 Osaka University. All Rights Reserved.
14
27
Idol singer
Ideally beautiful singer
Many fans
28
Which is more beautiful, humans or androids?
What is the perfect idol singer?
What is beauty?
-
8/19/2019 Humanoides- Osaka University
15/46
CNR101x Week2 Handout
© 2016 Osaka University. All Rights Reserved.
15
29
Sales clerk of the department storeDialogue system using tablet computers (without voice
recognition).
It handles more customers than human clerks.
30
Verbal conversationVoice recognition system and Cleverbot (Winner of
Loebner Prize in 2005 and 2006)
One question and one answer
-
8/19/2019 Humanoides- Osaka University
16/46
CNR101x Week2 Handout
© 2016 Osaka University. All Rights Reserved.
16
31
Conversation among robots
They seem conversational.Feeling of intelligence from the robot conversation.
32
What is intelligence?
Is a human more intelligent than computers?
How can we define to think?
What is intelligence? What is thinking?
-
8/19/2019 Humanoides- Osaka University
17/46
CNR101x Week2 Handout
© 2016 Osaka University. All Rights Reserved.
17
33
Selection of
related topics by
Google search
Understanding
concepts with
Wikipedia
Facial
expression
Gaze
control
Representation
of emotion
Emotion is the fastest way to connect among humansand robots.
Knowledge on the net for more intelligent conversation.
Intelligence in the cloud
Conversational pattern
34
Humanlike conversation
Practical teleoperation
and data collection
-
8/19/2019 Humanoides- Osaka University
18/46
CNR101x Week2 Handout
© 2016 Osaka University. All Rights Reserved.
18
35
GeminoidTeleoperated android of an existing person
I n t e r n
e t
Operator
ジェミノイドを用いた会議36
Meeting with Geminoid
-
8/19/2019 Humanoides- Osaka University
19/46
CNR101x Week2 Handout
© 2016 Osaka University. All Rights Reserved.
19
Comparison with video and speaker
Geminoid
Video
Speaker
The Geminoid has stronger presence than
others.
But it is more uncanny….
What is identity?38
Who has the identity? Can the human have identity?
Is there the peak for the
identity?
The identity is relative and it is
in the society.
-
8/19/2019 Humanoides- Osaka University
20/46
CNR101x Week2 Handout
© 2016 Osaka University. All Rights Reserved.
20
39
Adaptation to
the different body
I n t e r n e t
40
Virtual tactile sensation
Operator
-
8/19/2019 Humanoides- Osaka University
21/46
CNR101x Week2 Handout
© 2016 Osaka University. All Rights Reserved.
21
Body Ownership Model [Tsakiris M. et al 2005]
Visual
afference
Proprioceptiveafference
Motor
command
Motor
plan
Self-bodyRecognition
Arm
displacement
Arm
displacement
Comparator
Efference copy
Forward model
Body Ownership Model
during Geminoid Opereation
Visual
afference
Proprioceptive
afference
Motor
command
Motor
plan Arm
displacement
Arm
displacement
Comparator
Efference copy
Forward model
Geminoid
Body
Ownership
Illusion
-
8/19/2019 Humanoides- Osaka University
22/46
CNR101x Week2 Handout
© 2016 Osaka University. All Rights Reserved.
22
Body Ownership Model
during Geminoid Opereation
Visual
afference
Proprioceptiveafference
Motor
command
Motor
plan Arm
displacement
Arm
displacement
Comparator
Efference copy
Forward model
Geminoid
Body
Ownership
Illusion
Does the body ownership transferoccur in the absence of
proprioceptive feedback?
?
BCI Configuration
The subject imagines a left or right hand movement
Geminoid’s hands move according to the subject’s
brain activity
Amplifier
Signal
processing
Raw
data
gTEC BCI system
Motion command
Visual
Feedback
Geminoid system
classifierGeminoid
server
-
8/19/2019 Humanoides- Osaka University
23/46
CNR101x Week2 Handout
© 2016 Osaka University. All Rights Reserved.
23
Experimental setup
The Procedure
Task Stimulus QuestionnaireTraining
12
Q: When the robot’s hand was injected, did you feel as if
your own hand was injected?
I didn’t feel at all 1 2 3 4 5 6 7 I felt very strongly
-
8/19/2019 Humanoides- Osaka University
24/46
CNR101x Week2 Handout
© 2016 Osaka University. All Rights Reserved.
24
Evaluation method
SCR peak value after stimulus in each condition
were averaged and compared
Participants' score to the question in each condition were averaged and
compared
Questionnaire
Skin Conductance Response (SCR)
0
1
2
3
4
5
6
7
Still Match Raw
S c o r e
Results with 19 subjects
Significant difference in
questionnaire seen as
Match > {Still, Raw}
* ( p < 0.05)
Significant difference for SCR
seen as
Match > Still
* ( p < 0.05)
*
0
0.5
1
1.5
2
2.5
Still Match Raw
P
e
V
a
u
S
)
**
Questionnaire SCR
-
8/19/2019 Humanoides- Osaka University
25/46
CNR101x Week2 Handout
© 2016 Osaka University. All Rights Reserved.
25
Body Ownership Model
during Geminoid Opereation
Visual
afference
Proprioceptiveafference
Motor
command
Motor
plan Arm
displacement
Arm
displacement
Comparator
Efference copy
Forward model
Geminoid
Self-bodyRecognition
Body ownership
transfer
in the absence of
proprioception
50
Do we know ourselves?We do not know ourselves as much as we know others.
-
8/19/2019 Humanoides- Osaka University
26/46
CNR101x Week2 Handout
© 2016 Osaka University. All Rights Reserved.
26
New media
that transfer our presence
to distant places
51
52
Conference with Geminoid
-
8/19/2019 Humanoides- Osaka University
27/46
CNR101x Week2 Handout
© 2016 Osaka University. All Rights Reserved.
27
53
Experience is in our brain.
Experience is in others’ brain.
We, myself and the Geminoid can share the
experience.
What is experience?
54
Humanlike emotionand heart/mind
-
8/19/2019 Humanoides- Osaka University
28/46
CNR101x Week2 Handout
© 2016 Osaka University. All Rights Reserved.
28
55
Emotional facial expression
Is it richer than human facial expression?
56
What is emotion?
Which is more emotional and more
sympathetic?
Can the android have human emotion?
-
8/19/2019 Humanoides- Osaka University
29/46
CNR101x Week2 Handout
© 2016 Osaka University. All Rights Reserved.
29
57
Android theater
Moving and impressive theater withandroids.
It is world famouswith Hirata
with Hirata and Kuroki
58
Robot Theater 働く私Representation of humanlike heart
-
8/19/2019 Humanoides- Osaka University
30/46
CNR101x Week2 Handout
© 2016 Osaka University. All Rights Reserved.
30
with Hirata and Kuroki
59
Robot Theater 森の奥
60
Can the android havehumanlike heart/mind?
Which is more strong?
What is heart/mind?
-
8/19/2019 Humanoides- Osaka University
31/46
CNR101x Week2 Handout
© 2016 Osaka University. All Rights Reserved.
31
Development of
f ully autonomous robot
61
JST ERATO ISHIGURO
Symbiotic Human RobotInteraction
Project
2014-2020
62
I have developed the Geminoid originallyfor gathering data to make it autonomous.
Development of autonomous robots
I n t e r n e t
OperatorInterlocutor
Teleoperated
Android(Geminoid)
The teleoperated androids and robots
AutonomousAndroid
Autonomous androids and robots
-
8/19/2019 Humanoides- Osaka University
32/46
CNR101x Week2 Handout
© 2016 Osaka University. All Rights Reserved.
32
63
Big data approach for developing autonomous robots
I n t e r n e t
OperatorInterlocutorTeleoperated
Android
(Geminoid)
Speech and gesture
recognition
Utterance andbehavior generation
Hierarchical
models ofutterance andbehavior
generation
Behavior
Intention
Desire
Robot
Enhancement of the internal humanlikeness
Role assignment
in particular situations
Development of an acoustic model for
effective speech recognition by
Deep LearningCollect a great deal
of conversational
patternsincluding movements and
emotional expressions
Classification of
the patterns
64
Hierarchical models of
utterance and behavior generation
Behavior
Intention
Desire
Real
worldInternal
representation
Sensing
Action
Sensing(Prediction)
Action
(Revision)
Robot
Hierarchical models studied for decision-making in mobile
robotics.
The models are based on the idea accepted in cognitive science
and artificial intelligence.
The desires generate intentions, and intention generate
behaviors and utterances.
-
8/19/2019 Humanoides- Osaka University
33/46
CNR101x Week2 Handout
© 2016 Osaka University. All Rights Reserved.
33
65
Acquisition of the Hierarchical Models
Behavior
Intention
Desire
Behavior
Intention controls activation or selection of behavior.
Desire controls activation or selection of intention.
Machine learning techniques classify collected data with
intention flags given by the operator and find paths for satisfying
intentions.
BehaviorBehavior
Behavior
BehaviorBehavior
Behavior
Behavior
Behavior
BehaviorBehavior
BehaviorBehavior
Behavior
Behavior
BehaviorBehavior
Intention Intention
Desire Desire
IntentionIntention
Intention
66
Inference of people’s desires and
intentions
based on the hierarchical models Robots need to infer other’s intention and desire in social situations for having
proper conversations.
The robots assume the hierarchical models not only for other robots but alsofor humans, and infer humans’ intentions and desires.
What is mind? What is consciousness?
Geminoid
Geminoid
Human
Speech and gesture
recognition
Utterance and
behavior generation
Hierarchical
models ofutterance and
behaviorgeneration
Behavior
Intention
Desire
Robot
Speech and gesture
recognition
Utterance and
behavior generation
Hierarchical
models ofutterance andbehaviorgeneration
Behavior
Intention
Desire
Human
-
8/19/2019 Humanoides- Osaka University
34/46
CNR101x Week2 Handout
© 2016 Osaka University. All Rights Reserved.
34
67
The minimal human
68
How do we recognize
humans and geminoids?
-
8/19/2019 Humanoides- Osaka University
35/46
CNR101x Week2 Handout
© 2016 Osaka University. All Rights Reserved.
35
69
How do we recognize
the human presence with Telenoid?
70
Human has imagination
Humans interact withothers by using the
imagination.
The minimum design
maximizes the imagination.
-
8/19/2019 Humanoides- Osaka University
36/46
CNR101x Week2 Handout
© 2016 Osaka University. All Rights Reserved.
36
71
Telenoid
I n t e r n e t
The imagined face is mentally projected onto the neutral
appearance.
72
Communication service for elderlies
-
8/19/2019 Humanoides- Osaka University
37/46
CNR101x Week2 Handout
© 2016 Osaka University. All Rights Reserved.
37
73
Field tests with Telenoids
Practical service for elderly care.In Denmark and Japan.
-
8/19/2019 Humanoides- Osaka University
38/46
CNR101x Week2 Handout
© 2016 Osaka University. All Rights Reserved.
38
75
Portable media
for transmitting human presenceNew information media after smartphone
From voice and visual to human presence
76
The 3G prototype
-
8/19/2019 Humanoides- Osaka University
39/46
CNR101x Week2 Handout
© 2016 Osaka University. All Rights Reserved.
39
77
78
The minimum condition
to feel human presenceIt requires two modalities.
Hugvie: the minimum media.
Smartphone holder
Talk by phones
-
8/19/2019 Humanoides- Osaka University
40/46
CNR101x Week2 Handout
© 2016 Osaka University. All Rights Reserved.
40
79
Relax effect of Hugvie
Cortisol decrease by using Hugvie
p
-
8/19/2019 Humanoides- Osaka University
41/46
CNR101x Week2 Handout
© 2016 Osaka University. All Rights Reserved.
41
81
With hugvies
82
More than two modalities
Voice + Tactile sensation
Appearance + Tactile sen.
Smell + Tactile sensation
Voice + Smell
What do we need for feeling human
presence?
0 1 2 3 4 5
Number of modalities
Strength
of the feeling
-
8/19/2019 Humanoides- Osaka University
42/46
CNR101x Week2 Handout
© 2016 Osaka University. All Rights Reserved.
42
83
Humanlike Mechanism
Engineering explanation
of Yuragi formula
activity x f xdt
)()(x f x dt
d
YuragiOptimization measureControl model with attractors
Yuragi
formula
Currentcontrol
Control model
-
8/19/2019 Humanoides- Osaka University
43/46
CNR101x Week2 Handout
© 2016 Osaka University. All Rights Reserved.
43
Engineering explanation
of Yuragi formula activity x f x
dt )()(x f x
dt
d
u( x ) does not have to be precise or
accurate
activity Attractor
State valuable x
activit x u
)(
u( x )・activity is minimized by theyuragi formula
When activity is small, Yuragi η becomes relatively large, and it searches semi-
optimal solution based on Yuragi. Then, activity becomes large and it stays there.
It can be applied for complicated cases where proper models do not exist.
It does not compute all combinations, but it rather searches by Yuragi anddrastically reduce the computational cost/energy.
dx
x du x f
)()(
Potential
u( x )
YuragiOptimization measureControl model with attractors
Yuragi
formula
Current
control
Control model
85
86
Control of complicated robots based
on the biological principle
Bacteria robot with one motor
and one sensorRobotic arm that has humanlike bone structure
and muscle arrangement
-
8/19/2019 Humanoides- Osaka University
44/46
CNR101x Week2 Handout
© 2016 Osaka University. All Rights Reserved.
44
87
Simulation ofhumanlike development
with Asada (Osaka Univ.)
Humanlike developmental mechanism
• Yuragi simulates the infant
developmental process.
with Asada (Osaka Univ.)
88
U-shape development of infant
Variations
of themovements
Age
General
movements
Identification of the
elemental behaviors
Combinations of theelemental behavior
-
8/19/2019 Humanoides- Osaka University
45/46
CNR101x Week2 Handout
© 2016 Osaka University. All Rights Reserved.
45
89
Difference between humans and robot?
Can we distinguish them with
the fresh body?
Why do humans want to be
humans?
Someday, the boundary maydisappear. Is the boundary
necessary?
90
What is the definition of Human?
It is changing.
Are humans always more valuable than robots?
Do humans have the unlimited value to survive?
-
8/19/2019 Humanoides- Osaka University
46/46
CNR101x Week2 Handout
91
Android
Android
Human
Robot society
Ubiquitous
systems
Cognitive science
Standardization
Economy
Marketing
Robotics
Sociology
Field test
Complex systemsChaotic systems
Biology
Material
Engineering Sensor
Actuator
Artificial Life
Pattern
Recognition
Systems
Engineering
Neuroscience
User
Interface
Media & Art
Philosophy
Psychology