11/17/17

Tol

uca,

Mex

ico,

Sep

tem

ber

1-4,

200

2 {p

aren

thoe

n,bu

che,

tisse

au}@

enib

.fr

Marc Parenthoën,Cédric Buche,

Jacques Tisseau

Action Learning for AutonomousVirtual Actors

Laboratory of Software IngeneeringLi2/ENIB, EA 2215/UBO, BREST, FRANCE

22/17/17

Tol

uca,

Mex

ico,

Sep

tem

ber

1-4,

200

2 {p

aren

thoe

n,bu

che,

tisse

au}@

enib

.fr

Plan Autonomous Virtual Actors

Simulation in the Simulation

Action Learning

Principle of autonomy

Behavioral culture, Active perception

Autonomous adaptation of behavior

33/17/17

Tol

uca,

Mex

ico,

Sep

tem

ber

1-4,

200

2 {p

aren

thoe

n,bu

che,

tisse

au}@

enib

.fr

virus

interaction

immersion

3D-movie

flight simulator

presence

video-game

0 1

1

autonomy

1

VR : History

virtual theatre

interactive fiction

in virtuo virtual reality

44/17/17

Tol

uca,

Mex

ico,

Sep

tem

ber

1-4,

200

2 {p

aren

thoe

n,bu

che,

tisse

au}@

enib

.fr

Deficiencies sensorial motor mental

loss of autonomy

Abilities of perception of action of decision

Autonomy of execution

Towards Intrinsic Autonomous Models

55/17/17

Tol

uca,

Mex

ico,

Sep

tem

ber

1-4,

200

2 {p

aren

thoe

n,bu

che,

tisse

au}@

enib

.fr

FCM

++

-

specify behaviorBehavioral Modeling

closeenemy

farenemy

wish toescape

fear

-

+

+

perceptions

actions

deci

sion

sensors

effectors

1 ;,,1for , )1(),()( tNiNitatfGSN

ita tT

t L

dyna

mic

s

fuzzyfication

defuzzyfication

Explicative Emotional Perceptive Predictive Adaptative

FCMs [Kosko86]

control movement

66/17/17

Tol

uca,

Mex

ico,

Sep

tem

ber

1-4,

200

2 {p

aren

thoe

n,bu

che,

tisse

au}@

enib

.fr

autonomous virtual entity

perception

action

Reactive Virtual Actor

reaction

sensors

effectors

FCMs

perception

emotions

character

77/17/17

Tol

uca,

Mex

ico,

Sep

tem

ber

1-4,

200

2 {p

aren

thoe

n,bu

che,

tisse

au}@

enib

.fr

Plan Autonomous Virtual Actors

Simulation in the Simulation

Action Learning

Behavioral culture, Active perception

88/17/17

Tol

uca,

Mex

ico,

Sep

tem

ber

1-4,

200

2 {p

aren

thoe

n,bu

che,

tisse

au}@

enib

.fr

Behavioral Prototypes for Simulation

Virtual World : Real for Agent

-0.5

+0.5 +0.6

+1 -1 +1

-0.5

+0.5 +0.6

+1 -1 +1 -

0.5

+0.5 +0.6

+1 -1 +1

-0.5

+0.5 +0.6

+1 -1 +1

Imaginary World : Simulation

Key for learning

Anticipation Choice of strategy

99/17/17

Tol

uca,

Mex

ico,

Sep

tem

ber

1-4,

200

2 {p

aren

thoe

n,bu

che,

tisse

au}@

enib

.fr

Active Perception [Droulez & Berthoz 88]

sensorimotor inputs

effector command

operators on state

variables

inner maps operations

input switch

output switch

} proj

ectiv

e

}

cons

erva

tive

behaviormodification

comparisonreal/simulated

1010/17/17

Tol

uca,

Mex

ico,

Sep

tem

ber

1-4,

200

2 {p

aren

thoe

n,bu

che,

tisse

au}@

enib

.fr

autonomous virtual entity

perception

action

Predictive Virtual Actor

predictioninhibition

inhibition

proj

ectio

nst

rate

gy

reaction

sensors

effectors

FCMs

FCMs

self-

perc

eptio

n

library of behaviors

perc

eptio

n of

oth

ers Prey

Predator

Friend

behavioral model simulation sensorimotor strategy choice

active perception

1111/17/17

Tol

uca,

Mex

ico,

Sep

tem

ber

1-4,

200

2 {p

aren

thoe

n,bu

che,

tisse

au}@

enib

.fr

Plan Autonomous Virtual Actors

Simulation in the Simulation

Action Learning

Autonomous adaptation of behavior

1212/17/17

Tol

uca,

Mex

ico,

Sep

tem

ber

1-4,

200

2 {p

aren

thoe

n,bu

che,

tisse

au}@

enib

.fr

Virtual Gathering

• another sheepdog

• shepherd driven

• specific sheep

Imitations

sheepdogFCMs={it-self, prey}

paranoiac sheepFCMs={it-self, predator}• specific dog

• normal sheep

remaining paranoiac

guard point

pasture

1313/17/17

Tol

uca,

Mex

ico,

Sep

tem

ber

1-4,

200

2 {p

aren

thoe

n,bu

che,

tisse

au}@

enib

.fr Prey prototype from libraryBehavioral Prototype-Based Imitation

estimated sensors

estimated effectors

++

-

closeenemy

farenemy

wish toescape

fear

-

+

+0.6

predicted effectors

calling into question (crisp choice)

differential hebbian learning to update links

1414/17/17

Tol

uca,

Mex

ico,

Sep

tem

ber

1-4,

200

2 {p

aren

thoe

n,bu

che,

tisse

au}@

enib

.fr

Prey-prototype from library Prey identified by imitation

Sheepdog Imagining a Stressed Sheep

1515/17/17

Tol

uca,

Mex

ico,

Sep

tem

ber

1-4,

200

2 {p

aren

thoe

n,bu

che,

tisse

au}@

enib

.fr

autonomous virtual entity

perception

action

Autonomous Action Learning

learning

com

pari

son

adap

tatio

n

selection by active perception

modification

reaction

sensors

effectors

FCMspr

ojec

tion

stra

tegy

prediction

FCMs

self-

perc

eptio

n

library of behaviors

perc

eptio

n of

oth

ers Prey

Predator

Friend

FCM

learning process

ImitationCharacterisationIdentification

inhibition

inhibition inhibition

inhibition

inhibition

inhibition

1616/17/17

Tol

uca,

Mex

ico,

Sep

tem

ber

1-4,

200

2 {p

aren

thoe

n,bu

che,

tisse

au}@

enib

.fr

Plan Autonomous Virtual Actors

Simulation in the Simulation

Action Learning

1717/17/17

Tol

uca,

Mex

ico,

Sep

tem

ber

1-4,

200

2 {p

aren

thoe

n,bu

che,

tisse

au}@

enib

.fr

European Center of Virtual Reality

CERV

… Brest (France), september 2003

Li2CERV

Questions ?

1818/17/17

Tol

uca,

Mex

ico,

Sep

tem

ber

1-4,

200

2 {p

aren

thoe

n,bu

che,

tisse

au}@

enib

.fr

Animation

what IS the model

Simulation

what doesthe model DO

ModelingWhat BECOMES of the model

Model

sense action

language

real time interactive

on line

Exploitation of Models

1919/17/17

Tol

uca,

Mex

ico,

Sep

tem

ber

1-4,

200

2 {p

aren

thoe

n,bu

che,

tisse

au}@

enib

.fr

Participative SimulationInteractive SimulationNumerical Simulation

model

slave/master

model

master/slave

actorspectator

cre-actorinteractive simulationaction

real time animationsense

on line modellinglanguage

interactivity

0

1

2

User Status

model

avatar

equal/equal

2020/17/17

Tol

uca,

Mex

ico,

Sep

tem

ber

1-4,

200

2 {p

aren

thoe

n,bu

che,

tisse

au}@

enib

.fr

Multi-Agent Systems

perception

action

decision

Numerical model descriptive causal behavioral

2121/17/17

Tol

uca,

Mex

ico,

Sep

tem

ber

1-4,

200

2 {p

aren

thoe

n,bu

che,

tisse

au}@

enib

.fr

VOWELS Agent Environment Interaction Organisation

“extended” VOWELS User

model

model

A E I O

avatar

avatar

U

Participative Multi-Agent Systems

2222/17/17

Tol

uca,

Mex

ico,

Sep

tem

ber

1-4,

200

2 {p

aren

thoe

n,bu

che,

tisse

au}@

enib

.fr

Reactive Virtual Skipper

LandmarkGyroscope Landmark

Trim

ObstacleJibe-Luff

ObstacleLuff

SpecificNo Surf

SpecificSurf

GatewayVMG

GatewayRestart

-1 -1

+1

+1 -1

-1

-1

-1 -1-1

-1

-1-1

-1

-1

-1

compasscrollrrollrtrimtcompasschelmboatv2

Adaptative Virtual Skipper

HF strategy

LF fuzzyfication

and dynamics

2323/17/17

Tol

uca,

Mex

ico,

Sep

tem

ber

1-4,

200

2 {p

aren

thoe

n,bu

che,

tisse

au}@

enib

.fr

Human and Virtual Helmsman

compasscrollrrollrtrimtcompasschelmboatv

2

• associated to each affordance

• part of strategy

• no HF strategy