FROM ANIMALS TO ANIMATS 4

Proceedings of the Fourth International Conference on Simulation of Adaptive Behavior

edited by Pattie Maes, Maja }. Mataric, Jean-Arcady Meyer, Jordan Pollack,

and Stewart W. Wilson

A Bradford Book

Hie MIT Press Cambridge, Massachusetts

London, England

CONTENTS

Preface XI

THE ANIMAT APPROACH TO ADAPTIVE BEHAVIOR

Building "Fungus Eaters": Design Principles of Autonomous Agents 3 Rolf Pfeifer

Human Simulation of Adaptive Behavior: Interactive Studies of Pursuit, Evasion, Courtship, Fighting, and Play 13

Philip W. Blythe, Geoffrey F. Miller, and Peter M, Todd

The Engineering ofMind 23 James S. Albus

PERCEPTION AND MOTOR CONTROL

Seli-TaughtVisually-GuidedPointingforaHumanoid Robot 35 Matthew Marjanovic, Brian Scassellati, and Matthew Williamson

A Self-Organizing Model of the Antennal Lobes 45 Rainer Malaka, Stefan Schmitz, and WayneM. Getz

Some Adaptive Movements of Animats with Single Symmetrical Sensors 55 Owen Holland and Chris Melhuish

Categorization in a Real-World Agent Using Haptic Exploration and Active Perception 65 Christian Scheier and Dimitrios Lambrinos

How to Attract Females: Further Robotic Experiments in Cricket Phonotaxis 75 Barbara Webb and John Hallam

Coordination in a Six-Legged Walking System: Simple Solutions to Complex Problems by Exploitation of Physical Properties 84

Molk Cruse, Christian Bartling, Jeffrey Dean, Thomas Kindermann, Josef Schmitz, Michael Schumm, and Hendrik Wagner

Orientation Behavior Using Registered Topographie Maps 94 Cynthia Ferrell

Locating Odor Sources in Turbulence with a Lobster Inspired Robot 104 Frank Grosso, Thomas Consi, David Mountain, and Jelle Atema

Dynamics for Vision-Based Autonomous Mobile Robots U3 Hartmut Neuen, Axel Steinhage, Martin Giese, and Carsten Bruckhoff

Postural Primitives: Interactive Behavior for a Humanoid Robot Arm 124 Matthew M. Williamson

vi Contents

ACTION SELECTION AND BEHAVIORAL SEQUENCES

Action Selection Methods using Reinforcement Learning 135 Mark Humphrys

Towards Adaptive Behavior System Integration Using Connectionist Infinite State Automata X45

Tom Ziemke

Centrally-Generated and Reflexive Control Strategies in the Adaptive Behavior of Real and Simulated Animals 155

JimH. Belanger and Mark A. WiUis

Variable Binding and Predicate Representation in a Behavior-Based Architecture 163 Ian Horswill

The Experimental Study and Computer Simulation of Fish Behavior in the Uniform Environment 173

V. A. Nepomnyashchikh and Vera A. Gremyatchikh

Handling Time-Warped Sequences with Neural Networks 180 Claudia Ulbricht

INTERNAL WORLD MODELS AND NAVIGATION

How Insects Learn about the Sun's Course: Alternative Modeling Approaches 193 Jeffrey Dickinson and Fred Dyer

Spatial Exploration, Map Learning, and Self-Positioning with MonaLysa 204 Jean-Yves Donnart and Jean-Arcady Meyer

Adaptive Animat Navigation Based on a Flexibility Model for the Environment 214 Peter Veelaert and Herbert Peremans

Maze Navigation Using Optical Flow , 224 Andrew P. Duchon

An Autonomous System for Extracting Fuzzy Behavioral Rules in Mobile Robotics.... 233 A. G. Pipe and A. Winfield

MOTIVATION AND EMOTIONS

A New Control Architecture Combining Reactivity, Planning, Deliberation and Motivation for Situated Autonomous Agent 245

Frangois Michaud, Gerard Lachiver, and Chon Tarn Le Dinh

A Finer-Grained Motivational Model of Behaviour Sequencing 255 Emmet Spier and David McFarland

What Are Emotions For? Commitments Management and Regulation Within Animals/Animats Encounters 264

Michel Aube and Alain Senteni

Contents vii

Reinfprcement Leaming and Anirnat Emotions 272 Ian Wright

LEARNING

Skinnerbots . 285 David S. Touretzky and Lisa M. Saksida

No Bad Dogs: Ethological Lessons for Leaming in Hamsterdam 295 Bruce M. Blumberg, Peter M. Todd, and Pattie Maes

Generalization in Instrumental Leaming.... 305 Christian Balkenius

Leaming to Use Selective Attention and Short-Term Memory in Sequential Tasks 315 Andrew Kachites McCallum

Explore/Exploit Strategies in Autonomy 325 Stewart W. Wilson

Leaming Control Composition in a Complex Environment 333 E. G. Araujo and R. A. Grupen

Modular Growing Network Architectures for TD Leaming 343 Pascal Blanchet

Leaming to Detour & Schema-based Leaming 353 Fernando ]. Corbacho and Michael A. Arbib

Emergent Hierarchical Control Structures: Leaming Reactive/Hierarchical Relationships in Reinforcement Environments 363

Bruce L. Digney

A Model of Operant Conditioning for Adaptive Obstacle Avoidance 373 Paolo Gaudiano, Eduardo Zalatna, Carolina Chang, and Juan Lopez Coronado

Leaming Navigational Behaviors Using a Predictive Sparse Distributed Memory 382 Rajesh P. N. Rao and Olac Fuentes

EVOLUTION

A Developmental Model for the Evolution of Complete Autonomous Agents 393 Frank Dellaert and Randall D. Beer

Evolution of Plastic Neurocontrollers for Situated Agents 402 Dario Floreano and Francesco Mondada

Increasing Adaptivity through Evolution Strategies 411 RalfSalomon

Toward the Evolution of Dynamkal Neural Networks for Minimally Cognitive Behavior 421 Randall D. Beer

viii Contents

Emergence of a Multi- Agent Architecture and New Tactics for the Ant Colony Food Foraging Problem Using Genetic Programming 430

Forrest H Bennett III

Cell Interactions as a Control Tool of Developmental Processes for Evolutionary Robotics 440 Peter Eggenberger

Evolution of the Sensorimotor Control in an Autonomous Agent 449 Susanne A. Huber, Hanspeter A. Mallot, and Heinrich H. Bülthoff

The Evolutionary Cost of Learning 458 Giles Mayley

Evolving Obstacle Avoidance Behavior in a Robot Arm 468 David E. Moriarty and Risto Miikkulainen

Automatic Generation of Adaptive Programs 476 Lee Spector and Kilian Stoffel

Sexual Swimmers: Emergent Morphology and Locomotion without a Fitness Function.... 484 Jeffrey Ventrella

COEVOLUTION

Cooperative Versus Competitive System Elements in Coevolutionary Systems 497 Robert E. Smith and H. B. Cribbs III

Co-evolution of Pursuit and Evasion II: Simulation Methods and Results 506 Dave Cliffand Geoffrey F. Miller

Incremental Self-Improvement for Life-Time Multi-Agent Reinforcement Learning 516 Jieyu Zhao and Jürgen Schmidhuber

Dynamics of Co-evolutionary Learning 526 Hugues Juille and Jordan B. Pollack

COIXECTIVE BEHAVIOR

Oscillation-Enhanced Adaptability in the Vicinity of a Bifurcation: The Example of Foraging in Ants

Eric Bonabeau and Frangois Cogne

Dominance Interactions, Spatial Dynamics and Emergent Reciprocity in a Virtual World Charlotte K. Hemelrijk

A Study of Territoriality: The Role of Critical Mass in Adaptive Task Division Miguel Schneider Fontän and MajaJMataric

Emergent Adaptive Lexicons Luc Steels

Synthetic Robotic Language Acquisition by Observation Alexandros Moukas and Gillian Hayes

.537

.545

,553

.562

.568

Contents ix

The Evolution of Communication Scheines over Continuous Channels 580 Gregory M. Saunders and Jordan B. Pollack

Using A-Life to Study Bee Life: The Economics of Central Place Foraging 590 P. de Bourcier

An Evolved Fuzzy Reactive Control System for Co-operating Autonomous Robots 599 Robert Ghanea-Hercock and David P Barnes

On Simulating the Evolution of Communication 608 Jason Noble and Dave Cliff

Collective Behavior by Modular Reinforcement-Learning Animats 618 Norihiko Ono, Kenji Fukumoto, and Osamu Ikeda

Robotic "Food" Chains: Extemalization of State and Program for Minimal-Agent Foiaging 625 Barry Brian Werger and Maja J Mataric

(Not) Evolving Collective Behaviours in Synthetic Fish 635 Nahum Zaera, Dave Cliff, and Janet Brüten

Author Index 645