a computational perspective on mind and brain · on mind and brain 3. abstraction, event...

Martin Butz, [email protected]

A Computational Perspective On Mind and Brain3. Abstraction, event-predictive cognition, language, & cognitive architectures

Institute of Computer Science & Institute of Psychology

2 | Martin Butz | A Computational Perspective on Mind and Brain

Nexta-priori

predictivestate of mind

SensoryInformation

Sensory expectationsfocusing on “relevant”aspects

Sensor fusion of perceptions with

expectations (yielding

information gain)

Adaptation ofinternal estimates by

means of neuralactivity adapta-tions (i.e. state inference) and

(slower) weight adaptations

(i.e. learning)

Activeinference-based futureconsiderationsand consequentdecision making.

The Dynamically Active, Anticipatory Mind

Temporal

prediction

Δt

Maximalconsistenta-posteriori anticipatorystate of mind

Temporal forward and inverse homeostasis-oriented

considerations

Maximalconsistenta-posteriori predictive

state of mind

Con

sist

ency

enfo

rcem

ent

Local posterior predictive

state of mindInformation integration

A-priori predictive

state of mind

Butz, M. V. (2016). Towards a Unified Sub-Symbolic Computational Theory of Cognition. Frontiers in Psychology, 7, 10.3389/fpsyg.2016.00925. Butz, M. V. & Kutter, E. F. (2017). How the Mind Comes Into Being: Introducing Cognitive Science from a Functional and Computational Perspective. Oxford, UK: Oxford University Press.

Control & tem-poral predictionof motor, motion, & other force consequences


Brain as a Generative Model

• The brain develops a generative, predictive model of the encountered environment in order to interact with it in an effective, goal-directed manner.

• The free energy principle allows the formalization of this development –including the involved behavior – but the formalism is overly general.

Structural, inductive biases –that is, tendencies to develop certain encoding structures –must shape cognitive development further!



Structural Systematics

• At least three fundamental types of predictive encodings should be distinguished:

- spatial, top-down, & temporal predictive encodings (PEs)

• Moreover, sudden free energy signal changes within the active PEs lead to the encoding of

- events & event boundary / event transitions.

• As a result, event-schemata are formed, which encode- Preconditions necessary to initiate an event. - Final consequences that are typically reached when the event is over.- The event code itself – a sensory-force code, which predicts the unfolding

dynamics.

• Hierarchically organized event encodings enable conceptual, goal-directed planning and goal-directed behavior – as well as conceptual reasoning.



Nested Event Schemata Yield Hierarchical Structure

• Events- Predictive (probabilistic)

encoding network of how (sensorimotor, -force, & more abstract) dynamics unfold over time.

• Event Boundaries- Conditional predictive

encodings of event transitions.

• Event Hierarchies- Hierarchical / taxonomical

structure of events, event boundaries and their typical interactions.

time

Drinking from a glass in front of me

Moving &drinking

Gettinghold of it

drinkingPuttingdown

Trans-port

gras-ping

Hand to glass



NEURAL ACTIVE-INFERENCE-BASED IMAGINATIONS AND CONTROL

Sensorimotor forward model used for control

Sebastian Otte, Theresa Schmitt, Karl Friston, & Martin V. Butz (2017). Inferring adaptive goal-directed behavior within recurrent neural networks. ICANN 2017.


The RNN Model

• Recurrent neural network architecture (LSTM)learns a sensorimotor forward model.

• Scenario: A “rocket” with simulated gravity and two thrust motors

• Input to the LSTM:- Current location of rocket and current motor activity

• Output of the LSTM:- Resulting velocity and next location of rocket

• During training: - Training samples of somewhat systematic random explorations

• During testing: - Active inference-based motor control derivation

“Rocket” scenario



Controlling a Glider(Otte, Schmitt, Friston, & Butz, ICANN 2017)

• Learning a sensorimotor forward model with an RNN (LSTM)

• Control via active inference:- Attempting to reach goal state 𝑴𝑴𝑮𝑮- Inferring policy 𝜋𝜋 by maximizing

expected 𝑄𝑄𝜏𝜏 over expected future model states 𝑴𝑴𝝉𝝉 and observations 𝒐𝒐𝝉𝝉.

𝑄𝑄𝜏𝜏 𝜋𝜋, 𝑠𝑠𝑡𝑡 =−𝐷𝐷𝐾𝐾𝐾𝐾 𝑃𝑃 𝒐𝒐𝝉𝝉|𝑴𝑴𝝉𝝉 ||𝑃𝑃 𝒐𝒐𝝉𝝉|𝑴𝑴𝑮𝑮−𝐸𝐸𝜋𝜋 𝐻𝐻 𝑃𝑃 𝒐𝒐𝝉𝝉|𝑴𝑴𝝉𝝉



REPRISE: A RETROSPECTIVE & PROSPECTIVE INFERENCE SCHEME

Concurrent Event Inference and Goal-Directed Control

Martin V. Butz, David Bilkey, Alistair Knott, & Sebastian Otte (in press). REPRISE: A Retrospective and Prospective Inference Scheme. CogSci 2018.


The Setup“Rocket” event

• Recurrent neural network architecture (LSTM)learns a sensorimotor forward model. With dedicated, stable hidden state event indicator neurons

• 3 - “Event” Scenario: 1. “Rocket” simulation (2 thrust directions, inertia gravity)2. Stepper simulation (4 combinable directional step

motions, no inertia)3. Glider simulation (4 thrust directions, inertia)

• Input to the LSTM:- Current location of the “vehicle” and current motor activity

• Output of the LSTM:- Resulting motion of rocket

• During training: - Switching between three vehicles. - Switch triggers

1. Retrospective (stable) hidden state inference2. BPTT learning

• During testing: - Retrospection: hidden state inference- Prospection: Active inference-based motor control derivation

“Stepper” event

“Glider” event

Martin V. Butz, David Bilkey, Alistair Knott, & Sebastian Otte (submitted).REPRISE: A Retrospective and Prospective Inference Scheme. CogSci 2018.


REPRISE: Retro- and Prospective Inference Dynamics

Prospection(active motor

inference)

Retrospection(hidden state & event inference)

PredictiveCurrentState

GOAL



REPRISE Performance

Random motor commands



Event Boundary Indicator

Training

Prospection(active motor

inference)



GOAL

Testing



ANTICIPATING UPCOMINGEVENT BOUNDARIES

Goal-Oriented Object Interactions

Belardinelli, A. & Butz, M. V. (2013). Gaze strategies in object identification and manipulation. Annual Conference on Cognitive Science (CogSci 2013), 1875-1880.Belardinelli, A., Herbort, O., & Butz, M. V. (2015). Goal-oriented gaze strategies afforded by object interaction. Vision Research, 106, 47–57.Belardinelli, A., Stepper, M. Y., & Butz, M. V. (2016). Itś in the eyes: Planning precise manual actions before execution. Journal of Vision, 16(1), 18. doi:10.1167/16.1.18Belardinelli, A., Barabas, M., Himmelbach, M., & Butz, M. V. (2016). Anticipatory eye fixations reveal tool knowledge for tool interaction. Experimental Brain Research, 234, 2415–2431.


Videos of Typical Trial Interactions

Task: Grasp & Drink from object

Task: Grasp & Handover object

Belardinelli, A., Stepper, M. Y., & Butz, M. V. (2016). Itś in the eyes: Planning precise manual actions before execution. Journal of Vision, 16(1), 18. doi:10.1167/16.1.18


Objects arefixated in the light of the final event

Task: Grasp & drink from object

Task: Grasp and hand over object

Belardinelli, A., Stepper, M. Y., & Butz, M. V. (2016). Itś in the eyes: Planning precise manual actions before execution. Journal of Vision, 16(1), 18. doi:10.1167/16.1.18


ANTICIPATING BODILY STATES AT FUTURE EVENT BOUNDARY

Manual peripersonal space maps into the future

Belardinelli, A., Lohmann, J., Farnè, A., & Butz, M. V. (2018). Mental space maps into the future. Cognition, 176, 65–73. doi:10.1016/j.cognition.2018.03.007


Event-Predictive Encodings in Action

• Focus:- Manual interactions with objects- Peripersonal space around the handKnown to integrate multiple sensory information sources.Surrounds hand / face and other body parts.Can be modified via tool usage.

• Dual task paradigm with 1. Expected cross-modal congruency

Does a light flash close to FUTURE FINGER POSITIONS influence the detection of vibrotactile stimulations of the respective fingers?

2. Task-oriented object interaction Transport an object to the right and place it upright

Belardinelli, A., Lohmann, J., Farnè, A., & Butz, M. V. (2018). Mental space mapsinto the future. Cognition, 176, 65–73. doi:10.1016/j.cognition.2018.03.007


Event-Predictive Encodings in Action – Trial Schedule

SOAs:1. 200ms after visual

onset2. Upon motion onset3. 200ms after motion

onset



Cross-Modal Congruency Effect

Vibration detection response time is influenced by the visual distractor in anticipation of future hand posture (finger locations).

(data shown is averaged over all three SOAs.)



LEARNING EVENT TAXONOMIES FROM SENSORIMOTOR EXPERIENCES

A computational model of event & event transition detection, abstraction, and planning via free-energy based (active) inference

Gumbsch, C., Kneissler, J., & Butz, M. V. (2016). Learning behavior-grounded event segmentations. Proceedings of the 38th Annual Meeting of the Cognitive Science Society (pp. 1787–1792). Austin, TX: Cognitive Science Society.Gumbsch, C., Otte, S., & Butz, M. V. (2017). A computational model for the dynamical learning of event taxonomies. Proceedings of the 39th Annual Meeting of the Cognitive Science Society (pp. 452–457). London, UK: Cognitive Science Society.


• Predictive System- Continuously predicts next perceptions 𝑠𝑠′𝑡𝑡+1 by means of currently active

predictive event models 𝑀𝑀(𝑡𝑡) – monitoring the currently unfolding event(s).- Improves currently active event models 𝑀𝑀(𝑡𝑡 ) by means of error signal 𝑒𝑒𝑡𝑡+1

• Error Detection system monitors the error dynamics over time.

- Online surprise measure(transient error detection) detects and learns to predictevent transitions: Changes in active

event models 𝑀𝑀𝑛𝑛,𝑗𝑗 → 𝑀𝑀(𝑡𝑡 + 1) Creation of new event model𝑀𝑀𝑛𝑛,𝑗𝑗 (in interaction with predictive system).

• Event Models module maintains all available event models 𝑀𝑀.

• Event Boundary Models modulelearns to predict / expect particularevent transitions

- Specifying relevant event boundary characteristics & expected event transition.

Basic Components ofComputational Model

Computational EST Model Schema

Gumbsch, C., Kneissler, J., & Butz, M. V. (2016). Learning behavior-grounded event segmentations. Proceedings of the 38th Annual Meeting of the Cognitive Science Society (pp. 1787–1792). Austin, TX: Cognitive Science Society.


Learning Temporal Predictive Models (i.e. event models)

Forward model predictions (event monitoring):• Algorithm maintains active predictive, temporal forward models, which predict

current sensory changes 𝑠𝑠′𝑡𝑡+1 given motor command �⃗�𝑥𝑡𝑡. Independent forward models 𝑀𝑀𝑛𝑛,𝑖𝑖 for each sensory dimension 𝑛𝑛. Temporal forward models are learned by recursive least squares.

• Algorithm additionally maintains moving average error �̅�𝑒𝑀𝑀𝑛𝑛,𝑖𝑖 and error d𝐞𝐞𝐞𝐞𝐞𝐞𝐞𝐞𝐞𝐞𝐞𝐞𝐞𝐞𝐞𝐞�𝜎𝜎𝑀𝑀𝑛𝑛,𝑖𝑖 of each active temporal forward model.

Surprise and revision of active models (event boundary detection): • Surprise is defined as an error that is more than two standard deviations away from

the current moving average error of a forward model: 𝑒𝑒𝑛𝑛,𝑡𝑡 > �̅�𝑒𝑀𝑀𝑛𝑛,𝑖𝑖 + 2 �𝜎𝜎𝑀𝑀𝑛𝑛,𝑖𝑖.• Surprise triggers a searching (reorientation) period.

Searching period• Resets all respective temporal forward models M and

monitors their prediction error over the next time steps.• Also a novel forward model is trained and monitored.• The forward model with the lowest error takes over.



Learning Event Boundaries / Event Transitions

• Once a temporal forward model transition occurred, event boundary model learning is triggered.

• Essentially, the density P(𝑠𝑠|𝑀𝑀𝑛𝑛,𝑖𝑖 → 𝑀𝑀𝑛𝑛,𝑗𝑗) is learned. Approximated by a multivariate Gaussian

(via covariance matrix of encountered transition states 𝑠𝑠𝑖𝑖).

• Consequences: - Event boundaries become predictable given the current sensory state. - Covariance matrix identifies relevant sensory information and

correlations within. Note that event boundaries are assumed

to be local and unimodal in sensory space.



Computational Model of Event Processing and Inference

Predictive processing & learning Hierarchical, active-inference-based goal-directed behavior

• Event Models as sensorimotor forward models• Event Boundary Models as multivariate Gaussians • Predictive Perceptual Space yields information fusion and imaginations• Motivations activate EBs to reach anticipated reward• Motor system executes motor commands and provides efference copies• Observations yield relative locations of hand, mouth, & object + object properties.

Gumbsch, C., Otte, S., & Butz, M. V. (2017). A computational model for the dynamical learning of event taxonomies. In Proceedings of the 39th Annual Meeting of the Cognitive Science Society. Cognitive Science Society


Results: Development of a hierarchical Event Taxonomy

Simple 3D Environment

Developing hierarchical event taxonomy dependent on signal noise and surprise threshold 𝜽𝜽.

• 3 types of objects (heavy, light sticky object, marble)

• Training by demonstration.

• Testing shows: - Prediction error reduction (ANN slower than Recursive least squares; both effective)- Successful goal-directed behavior dependent on noise and surprise threshold

(100% success when surprise threshold 𝜃𝜃 = 10 and noise 𝜎𝜎 = 0.01 ).- Event taxonomy development (see schema on the right)

Gumbsch, C., Otte, S., & Butz, M. V. (2017). A computational model for the dynamical learning of event taxonomies. In Proceedings of the 39th Annual Meeting of the Cognitive Science Society. Cognitive Science Society


A COGNITIVE ARCHITECTURE FOR LEARNING ABOUT ACTION GAME ENVIRONMENTS

Action game environments offer a very good test bed tostudy the development of understanding (≈common sense).

Focus: a Super Mario® clone: Actively learning conceptual schematic knowledge, Actively pursuing goals in a self-motivated manner, Linking the developing knowledge to language, Pursuing social cooperation.

Schrodt, F., Kneissler, J., Ehrenfeld, S., & Butz, M. V. (2017). Mario becomes cognitive. Topics in CognitiveScience, 9(2), 343–373. doi:10.1111/tops.12252Schrodt, F., Röhm, Y., & Butz, M. V. (2017). An Event-Schematic, Cooperative, Cognitive Architecture Plays Super Mario. In Proceedings of EUCognition 2016: Cognitive Robot Architectures (pp. 10–15).


Mario‘s Brain – SEMLINCS Architecture

Schrodt, F., Kneissler, J., Ehrenfeld, S., & Butz, M. V. (2017). Mario becomes cognitive. Topics in Cognitive Science, 9(2), 343–373. doi:10.1111/tops.12252

Nexta-

priori

predictivestate of mind

SensoryInforma-tion

Temporal

prediction

Δt

Maximalconsistenta-

posteriori anticipatorystate of mind

Temporal forward and inverse homeostasis-oriented considerations

Maximalconsistenta-

posteriori predictive state of mind

Con

sist

ency

enfo

rcem

ent

Local

posterior predictivestate of mind

Information integration

A-prio

ri predictive state of mind


Main Features of SEMLINCS Architecture

• Motivational system - Based on homeostatic variables;- Allows the autonomous selection of

goals.

• Event schema knowledge in the form of condition-action-effect rules…

- ... is learned from surprising event signals (e.g. disappearing object);

- … allows temporal forward predictions and inverse planning on a conceptual level.

• Planning: - Sensorimotor planning is currently

hard-coded (A*) relying on game engine (simulator)

- Schematic planning relies on event schema rule knowledge (i.e. production rules)

• Speech interface enables• querying and manipulating:

- Event schema knowledge- Goal selection- Motivational system state- Current motor commands.

• autonomously agreeing on (sequential) joint action plans.

Schrodt, F., Kneissler, J., Ehrenfeld, S., & Butz, M. V. (2017). Mario becomes cognitive. Topics in Cognitive Science, 9(2), 343–373. doi:10.1111/tops.12252


Example 1: Learning from Object Interaction Events and Observations

Schrodt, F., Kneissler, J., Ehrenfeld, S., & Butz, M. V. (2017). Mario becomescognitive. Topics in Cognitive Science, 9(2), 343–373. doi:10.1111/tops.12252 “Mario Becomes Social!” video available online on YouTube.


Example 2:Coordinating Plans

Schrodt, F., Röhm, Y., & Butz, M. V. (2017). An Event-Schematic, Cooperative, Cognitive Architecture Plays Super Mario. In Proceedings of EUCognition 2016: Cognitive Robot Architectures (pp. 10–15).


Summary, Limits, & Outlook

• SEMLINCS is a continuously active, hierarchical learning architecture.- It actively learns conceptual schema rules (i.e. a hierarchical, generative

model) of the encountered environment based on event boundary signals (non-linear error transients).

• Learned generative model is useful for• … goal-directed, homeostatic behavior;• … communication; • … collaboration.

• SEMLINCS has focused on a single, complex task (i.e. Mario) and has solved some aspects of human behavior….

- NOT directly modeling player behavior but aspects of cognitive development; - NOT solving the game but learning about the game environment.

• Hard-coded aspects should be learned (e.g. low-level control). • SEMLINCS needs to be applied / generalized to more complex (virtual) worlds.• Similarities to other cognitive architectures need to be spelled-out.

Schrodt, F., Kneissler, J., Ehrenfeld, S., & Butz, M. V. (2017). Mario becomes cognitive. Topics in Cognitive Science, 9(2), 343–373. doi:10.1111/tops.12252Schrodt, F., Röhm, Y., & Butz, M. V. (2017). An Event-Schematic, Cooperative, Cognitive Architecture Plays Super Mario. In Proceedings of EUCognition 2016: Cognitive Robot Architectures (pp. 10–15).


(Many)Open Challenges

- More complex, scalable models

- More real-world environments

- Episodic memory

- Conceptual abstractions beyond sensorimotor behaviors

- Learning the linkage to linguistic structure

Thank you

for the attention!

• John could not see the stage with Billy in front of him because he is so short / tall. Who is so short / tall?

• Joan made sure to thank Susan for all the help she had given / received. Who had given / received the help?

A Computational Perspective �On Mind and BrainThe Dynamically Active, Anticipatory MindBrain as a Generative ModelStructural SystematicsNested Event Schemata Yield Hierarchical StructureNeural Active-Inference-Based Imaginations and ControlThe RNN ModelControlling a Glider�(Otte, Schmitt, Friston, & Butz, ICANN 2017)REPRISE: a Retrospective & Prospective Inference SchemeThe SetupREPRISE: Retro- and Prospective Inference DynamicsREPRISE PerformanceAnticipating Upcoming�Event BoundariesVideos of Typical �Trial InteractionsObjects are�fixated in the light of the final event Anticipating Bodily States AT Future Event BoundaryEvent-Predictive Encodings in ActionEvent-Predictive Encodings in Action – Trial ScheduleCross-Modal Congruency EffectLearning Event Taxonomies From Sensorimotor ExperiencesBasic Components of�Computational ModelLearning Temporal Predictive Models (i.e. event models)Learning Event Boundaries / Event TransitionsComputational Model of Event Processing and InferenceResults: Development of a hierarchical Event TaxonomyA Cognitive Architecture for Learning About Action Game EnvironmentsMario‘s Brain – SEMLINCS ArchitectureMain Features of SEMLINCS ArchitectureExample 1: Learning from Object Interaction Events and ObservationsExample 2:�Coordinating PlansSummary, Limits, & Outlook(Many)�Open Challenges

a computational perspective on mind and brain · on mind and brain 3. abstraction, event...

Documents