Matching brain and body dynamics

• Daniel Wolpert: – "Why don't plants have brains?"– "Plants don't have to move!"

• Early phases of embodied artificial intelligence: – Navigation and orientation – Wheeled robots: Mismatch in complexity levels of sensory,

neural and motor systems (unsuccessful, and inconsistent with the “ecological balance principle”)

Matching brain and body dynamics

• Research inspired by rats– Place cells (O’Keefe and Dostrovsky, 1971)– Rat-mimicking robots

• Problem 1: complex sensory-motor skills not accounted for• Problem 2: rat cognition not limited to navigation

An important goal of developmental robotics is to construct a robot which develops into an intelligent adult human-like robot

•Development of sensory and motor systems– Biological organisms changes shape

• Growth cannot yet be modeled well on robots– Robot experiment by Max Lungarella

• From low-precision to high-precision motor system• From low-resolution to high-resolution sensory system

•Using previously aquired skills to learn new ones•Time– Many time scales involved (millisecs -> years)– Today, robot learn a skill very quickly

Other aspects of development

Other aspects of development

• Motivation– Why would a robot want to develop/improve over time?– Autotelic principle

• Good for more complicated tasks• A single motivating principle• Inspired by the concept of “flow”: Experience/feeling when

performing an expert activity• Maintain constant flow of information through system• Example: learn to reach -> learn to grasp -> learn to drink

Other aspects of development

Learning in embodied systems

• Exploration guided by embodiment facilitates learning

• Example: Baby learning to categorize objects– Hypothetically, random signal are sent to arm muscles– Movements constrained by anatomy and muscle-tendon

properties– Objects grasped due to (hypothetical) grasp reflex– At some point, object is brought to mouth, leading to strong

sensory input (sensory-motor correlation)– Eventually, categorization possible with less interaction

Social interaction

• Important aspect of development• Imitation– Powerful strategy for humans– Requires sophisticated perceptual abilities– Mirror neurons (Gallese et al, 1996)– Shared/joint attention– Scaffolding

• Natural language– Essential for human development– Can emerge from interaction among robot agents

Summary

• Human-like robots– Has potential, but are not very human-like– Haptic system, mouth region, tongue, speech system,

advanced muscle-tendon system

• Design principles for developmental systems– Time-scale integration principle– Development as an incremental process principle– Discovery principle– Social interaction principle– Motivated complexity principle

Place cells, grid cells, head-direction cells, border cells, and more..

A manifold of spatial maps in the brainDerdikman & Moser, TICS, 2010

Place cells

• Fire specifically when animal is at certain position

• Allow read-out of position• Located in the hippocampus• Strongly influenced by distal

sensory cues• Active both in light and dark

– i.e. not dependent on vision only

Place cells• Can also encode – Olfactory information– Time intervals

• Remapping can occur when environment is changed

Grid cells• Multiple firing locations, forming a

hexagonal grid of the entire space• Located in the parahippocampal

cortex (input/output to hippocampus)• Grid space, orientation and phase• Modulated by velocity

KTH research contributing to grid cell research

Head-direction cells

• Cells that only fire when head is pointing in certain direction

• Located in the parahippocampal cortex

Border cells

• Cells that only fire when animal is close to a geometric boundary

• Located in the parahippocampal cortex

• Discovered in 2008

Multi-compartment environment

Rapid transition between spatial maps

Development of spatial mapsCritical spatial representation components are already present when rat explores open environment outside the nest for the first time (Langston et al, & Wills et al, Science, 2010)•Head-direction cells:

– present 2 days after eye-opening•Place cells:

– present 2 days after eye-opening•Grid cells:

– present 4-8 days after eye-opening•In line with Immanuel Kant’s theory that the concept of space does not require experience

Questions remaining

• How does place cells relate to grid cells?• Do humans have neurons with similar functions?– Place cells (Ekstrom et al, 2003)– Grid cells (Doeller et al, 2010)– Path cells (Jacobs et al, 2010)

• Clockwise/counterclockwise paths• Neurosurgical patients in VR-game• Entorhinal cortex

Mouse place cells in virtual reality experimentVirtual reality environment created with Quake2 video game engine

youtube

Mouse place cells in virtual reality experiment