From motor babbling to hierarchical learning by

imitation: a robot developmental pathway

Yiannis Demiris and Anthony Dearden

Biologically-inspired Autonomous Robots Team (BioART)Intelligent Systems and Networks Group

Department of Electrical and Electronic EngineeringImperial College London, South Kensington Campus, SW7 2BT

{y.demiris, anthony.dearden}@imperial.ac.ukhttp://www.iis.ee.ic.ac.uk/yiannis

Abstract

How does an individual use the knowledgeacquired through self exploration as a manip-ulable model through which to understandothers and benefit from their knowledge?How can developmental and social learning becombined for their mutual benefit? In thispaper we review a hierarchical architecture(HAMMER) which allows a principled wayfor combining knowledge through explorationand knowledge from others, through the cre-ation and use of multiple inverse and forwardmodels. We describe how Bayesian Belief Net-works can be used to learn the associationbetween a robot’s motor commands and sen-sory consequences (forward models), and howthe inverse association can be used for imi-tation. Inverse models created through selfexploration, as well as those from observingothers can coexist and compete in a princi-pled unified framework, that utilises the sim-ulation theory of mind approach to mentallyrehearse and understand the actions of others.

1. Introduction

Equiping a robot with the ability to learn enables itto adapt its behaviour in order to improve its perfor-mance on certain tasks. Similarly, imitation (amongother social learning algorithms) has been shown tobe a powerful way for transferring behaviour pat-terns from one agent to another in order to reducethe time it takes the observer to achieve a task. Al-though powerful frameworks exist for both social anda-social learning, there hasn’t been a lot of work incombining techniques from the two domains in a uni-fied principled representational frameworks. Suchframeworks would allow a robot to combine (anduse) knowledge acquired from self-exploration, withknowledge acquired through observation, and allow

both to be used in a unified way.

In this paper we review our architecture,HAMMER (Hierarchical, Attentive, MultipleModels for Execution and Recognition), whichadopts the simulation theory of mind approach(Carruthers and Smith, 1996) to understanding theactions of others, using a distributed network of cou-pled inverse (Narendra and Balakrishnan, 1997) andforward (Miall and Wolpert, 1996) models. Newmodels can be added either through self-exploration(Dearden and Demiris, 2005), or by observing oth-ers, through imitation (Demiris and Hayes, 2002,Demiris and Johnson, 2003).

First we describe how Bayesian Belief Networkscan be used to learn the mapping between motorcommands and corresponding visual feedback states(the forward models). We subsequently invert thismapping to learn basic inverse models. We then pro-ceed to explain how having the same structure, a cou-pled inverse and forward model allows the robot to ei-ther execute, rehearse or perceive an action. Havinga distributed network of coupled inverse and forwardmodels helps us develop hierarchies of increasinglycomplex inverse models, and combine both learn-ing through self exploration and learning from oth-ers. We conclude by outlining current challengesin this developmental framework, namely object-oriented actions, and development of more complexbody schemas.

2. Background

Current approaches in robotics emphasize the needfor mechanisms to allow a robot to adapt tonew situations and be able to perform new tasks.Environmental conditions, task requirements andcontext cannot be assumed to be known in ad-vance. Robot Learning mechanisms have thus re-ceived extensive attention in the last few years(Conell and Mahadevan, 1993, Franklin et al., 1996,

31

Berthouze, L., Kaplan, F., Kozima, H., Yano, H., Konczak, J., Metta, G., Nadel, J., Sandini, G., Stojanov, G. and Balkenius, C. (Eds.)Proceedings of the Fifth International Workshop on Epigenetic Robotics: Modeling Cognitive Development in Robotic Systems

Lund University Cognitive Studies, 123. ISBN 91-974741-4-2

Demiris and Birk, 2000, Schaal, 2002). These tech-niques can be generally grouped in social and asociallearning. In the latter, the robot attempts to learnto perform a task by interacting with its environ-ment, without considering the solutions that otheragents in its environment possesss with respect tothis task. Techniques can be generally divided intosupervised and unsupervised learning, with one ofthe most widely used being reinforcement learning(Sutton and Barto, 1998).

On the other hand, recent approaches haveemphasized the importance of social learningin robotics, including learning by observation,demonstration and imitation. Equipping robotswith the ability to imitate enables them to learnto perform tasks by observing a human demon-strator (Schaal, 1999). In the center of thisability lies a mechanism that matches demon-strated actions with motor actions available tothe robot (Demiris and Hayes, 2002, Billard, 2000,Schaal et al., 2003). Several architectures havebeen proposed for implementing this mechanism(for reviews see (Breazeal and Scassellati, 2002,Schaal, 1999, Schaal et al., 2003,Dautenhahn and Nehaniv, 2002)), including anumber of proposals utilizing a shared substratebetween execution, planning, and recognition of ac-tions (Demiris and Hayes, 2002, Schaal et al., 2003,Wolpert et al., 2003). This methodological shift,compared to other successful approaches to learningby demonstration (Kuniyoshi et al., 1994) wasinspired by the discovery of the mirror system(di Pellegrino et al., 1992, Decety et al., 1997),which indicated that, at least in primates, thereis indeed a shared neural substrate between themechanisms of action execution and those of actionrecognition. Apart from being compatible with themotor theories of perception, from an engineeringperspective this approach is also attractive since itallows reuse of subsystems in multiple roles.

Despite extensive work on social and asocial learn-ing, there isn’t a lot of work being done in bring-ing these together. In this paper we will reviewour computational framework, HAMMER, that al-lows a principled way for combining knowledge fromthe two sources, and use them to incrementally learnmore complex structures. We will start by explaininghow our system can learn primitive forward and in-verse models by motor babbling, and demonstratingthis on an ActivMedia Peoplebot, learning to predictthe consequences of its motor commands on its grip-pers. We brieflly demonstrate how these primitiveforward models can be inverted and used to imitategestures done by a demonstrator. Having primitiveinverse and forward models, we explain how they canbe used to build more complex hierarchical inversemodels (Demiris and Johnson, 2003).

3. Learning through self-exploration

Drawing inspiration from motor babbling in infants(Meltzoff and Moore, 1997), we aim at building asystem that enables a robot to autonomously learn aforward model with no a priori knowledge of its mo-tor system or the external environment. The robotsends out random motor commands to its motor sys-tem and uses this, together with the informationfrom its vision system to learn the structure and pa-rameters of a Bayesian belief network (BBN), whichrepresents the forward model. This autonomouslylearnt model can then be used to enable the robot topredict the effects of its own actions, or imitate theactions of others.

3.1 Learning forward models

Bayesian Belief Networks (BBN) are well suited forrepresenting forward models which can be learnt.They enable the causal relationship between therobot’s motor commands, its motor system and theobservations from the robot’s sensor system to bemodelled and learnt. Each motor command, stateof the robot or sensor observation is represented as arandom variable in the BBN, and the causal relation-ships between them are represented with arcs. TheBBN therefore represents a learnt probability dis-tribution across the previous N motor commands,M1:N [t − d], the P current states S1:P [t], and the P

observations of the state, O1:P [t]. The variable d

represents the delay between a motor command be-ing issued and robot’s state changing; in real roboticsystems it cannot be assumed that the effect of amotor command will occur after just one time-step,so this is a parameter that must be modelled andlearnt. Forward models predict the consequence of amotor command. Therefore, the BBN can be usedas a forward model by performing inference with thenetwork to calculate the probabilities of robot statesor observations given a particular set of motor com-mands: P(S[t ]|M[t-d]=m) or P(O[t ]|M[t-d]=m). Figure1 shows this prototype network structure.

Several aspects of this BBN structure need to belearnt. The robot first needs to learn the number ofstate variables, P, and what they represent. Here,this is done by using information suppled from a thecomputer vision system, which is able to track ob-jects in a scene using no prior information about thenumber or properties of the objects. Objects arefound by clustering regions in the image with similarposition and movement properties. The state of eachobject found can be represented with a node on theBBN, and the tracking information from the visionsystem can be represented with an associated obser-vation node. The remaining parts of the structurethat need to be learnt are which motor commandscontrol which state variables, and which observations

32

Figure 1: The BBN of a forward model. The underlying

state of the robot is represented with hidden nodes. The

question marks represent the part of the structure which

needs to be learnt: the causal association between the mo-

tor commands and the state of the robot, and the state of

the robot with the observations from the computer vision

system.

from the computer vision system are the correct ob-servation of an object’s state.

To train a BBN, a set of evidence is required:a data set containing actual observed values ofthe observed nodes in the network. This is a setof motor commands executed at each time step,m1:N [t − d], together with the observations of thestate o1:P [t]. Two issues need to be addressed: howto create this set of evidence, and how to use thisevidence to learn the structure and parameters ofthe BBN. The inspiration for solving these prob-lem here is taken from developmental psychology.(Gopnik and Schulz, 2004) compares the mechanisman infant uses to learn to that of a scientist: scientistsform theories, make observations, perform experi-ments, and revise existing theories. Applying thisidea here, forward models can be seen as a theory ofthe robot’s own world and how it can interact withit. The process a robot uses to obtain its data andinfer a forward model from it is its experiment. Toperform an experiment, a robot decides what mo-tor commands to use, gathers a corresponding setof evidence and uses this evidence to learn the BBNstructure. The motor commands are chosen at ran-dom using a Markov model to randomly change thecurrent motor command. To learn the structure andparameter of a BBN, it is necessary to perform asearch through the space of possible structures, withthe goal of finding the one that maximises the log-likelihood of the model given the data. This searchspace would normally be extremely large. However,for a forward model the number of possible struc-tures is made much smaller because of the constraintsplaced on the dependence of particular nodes, e.g.the motor command nodes can only be connected tothe robot’s state nodes.

3.2 Experiments

An experiment was carried out to show how forwardmodels can be autonomously learnt and used. Thetask was to learn the forward model for the move-ment of the robot’s grippers; the robot needed tolearn to predict the effects of the motor commands.No prior information about the appearance of thegrippers or the nature of the motor commands wasspecified. The robot was to perform an experiment:it babbled with its motor commands, gathered evi-dence of the motor commands and corresponding ob-servations, and then learnt the relationship betweenthese using a BBN.

When the gripper motor system was babbled, thevision system correctly calculated and tracked thepositions of the moving grippers in the scene. Thisprovided the random variables for the states S[t ] and

observations O[t ] in the BBN for the forward model:each gripper’s state was represented as a discretenode for the state, and a set of continuous nodes forthe observation, both of which were represented as aGaussian distribution. The observations, O[t ] couldbe either the velocity VEL[t] or the position POS[t]of the tracked grippers. Learning the structure of theBBN of the forward was done by searching throughthe space of structures for different motor delays andobservations to find the structure that maximised thelikelihood of the model given the data. This was per-formed by simutaneously training each possible for-ward model structure, and tracking its log-likelihoodgiven the data. The parameters of each BBN werelearnt online using the recursice expectation maximi-sation (EM) algorithm. Learning multiple forwardmodels simultaneously allows the best one at anyparticular step of the training process to be imme-diately available. As shown in figure 2 the motorcommand was learnt to be M[t -11], correspondingto a delay of 200ms, and the best observation wascorrectly learnt to be VEL[t].

M[t-11]

S1[t] S2[t]

VEL1[t] VEL2[t]

Figure 2: The BBN of the forward model which was learnt

by the robot

Once the BBN was learnt, it could be used as a for-

33

0 .25 .5 .75 1 1.25 1.5 1.75 2 2.5 2.75−15

−10

−5

0

5

10

15

20

Time (s)

Vel

ocity

(pi

xels

/s)

Right handLeft handRight gripperLeft gripper

Figure 3: Imitation using a forward model. The top im-

ages are corresponding frames from the human demon-

stration (left) and the robot’s imitating grippers (right).

The graph shows the trajectory of the demonstrating

hands and the imitating trajectory of the grippers.

ward model: a prediction for a given motor commandis the probability of an observation of a gripper mov-ing at a particular velocity given a particular motorcommand: P(VEL[t ]|M[t-11 ]=m). Alternatively, themost likely gripper command to achieve a particu-lar movement can be inferred: P(M[t-11 ]|VEL[t]=v).

This can now be used a primitive inverse model -it gives a conditional probability distribution of themotors commands given a particular observation, sotaking the most likely motor command, the robotcan try and recreate a particular movement observa-tion. This can be used for imitation:. if the obser-vations given to the inverse model are of a human’shands, the robot is able to reproduce the motor com-mands that would be most likely to recreate that hu-man movement in its own gripper system. Figure 3shows the results of a simple imitation experiment:the robot is able to imitate a simple hand waving mo-tion. The human’s hand movements are tracked andrecognised using the same vision system. The ob-servations are used as evidence for the BBN alreadylearnt by the robot. This network is then used to pre-dict the most likely motor command that would pro-duce this observation. These motor commands aregiven to the robot enabling it to perform the actionthat best replicates the observed movement. Otherforward and inverse models can be created and usedthis way (but see the concluding discussion aboutlimitations of the vision-based comparison process)for the other possible degrees of freedom of the robot.

4. Learning from others

Having established this critical correspondence be-tween learning forward models and their use asinverse models through inverse mapping, we arenow in position to incorporate into the same ar-chitecture the ability to learn from others. We doso by utilising the simulation theory of mind ap-proach (Carruthers and Smith, 1996), which postu-lates that when we observe others, we put our-selves in their position, and internally simulate theiractions to understand what they are doing. In(Demiris and Johnson, 2003) we have done so usingHAMMER, a competitive, multiple inverse and for-ward models architecture.

4.1 Coupling Inverse and forward models

The fundamental structure of HAMMER is an in-verse model paired with a forward model (figure 4).In order to execute an action within this structure,the inverse model module receives information aboutthe current state (and, optionally, about the tar-get goal(s)), and it outputs the motor commandsthat it believes are necessary to achieve or maintainthe implicit or explicit target goal(s). The forwardmodel provides an estimate of the next state. Thisis fed back to the inverse model, allowing it to ad-just any parameters of the behaviour (an example ofthis would be achieving different movement speeds(Demiris and Hayes, 2002)). Additionally, the samestructure can be used in order to match a visuallyperceived demonstrated action with the imitator’sequivalent one. This is done by feeding the demon-strator’s current state as perceived by the imitator tothe inverse model and having it generate the motorcommands that it would output if it was in that stateand wanted to execute this particular action. Themotor commands are inhibited from being sent tothe motor system. The forward model outputs an es-timated next state, which is a prediction of what thedemonstrator’s next state will be. This prediction iscompared with the demonstrator’s actual state at thenext time step. This comparison results in an errorsignal that can be used to increase or decrease thebehaviour’s confidence value, which is an indicatorof how confident the particular imitator’s behaviouris that it can match the demonstrated behaviour.

HAMMER consists of several of thestructures described above, operat-ing in parallel (Demiris and Hayes, 2002,Demiris and Johnson, 2003). Fig. 5 shows thebasic structure. When the demonstrator executesan action the perceived states are fed into the imita-tor’s available inverse model. This generates motorcommands that are sent to the forward models.The forward models generate predictions about thedemonstrator’s next state: these are compared with

34

Figure 4: Coupling an inverse with a forward model;

this structure can be used to execute an action, as well

as rehearse it and match it with an external observation

(Demiris and Hayes, 2002).

� �

� �

� �

� �

� �

� �

� � � � � � � � � � � �

� � � � � �� � � � � � � �

� � � � � � �

� �

� � � � � �� � � � � � � �

� � � � � � �

� �

� �� �

� � � � � �� � � � � � � �

� � � � � � �

� �

� �� �

� � � �

� � � � � �� � � � �

� � � !

Figure 5: The basic architecture, showing multiple in-

verse models (B1 to Bn) receiving the system state, sug-

gesting motor commands (M1 to Mn), with which the

corresponding forward models (F1 to Fn) form predic-

tions regarding the system’s next state (P1 to Pn); these

predictions are verified at the next time state, resulting

in set of error signals (E1 to En)

the actual demonstrator’s state at the next timestep, and the error signal resulting from this com-parison affects the confidence values of the inversemodels. At the end of the demonstration (or earlierif required) the inverse model with the highest con-fidence value, i.e. the one that is the closest matchto the demonstrator is selected. This architecturehas been implemented in real-dynamics robot simu-lations and real robots (Demiris and Johnson, 2003,Johnson and Demiris, 2004) and has offered plau-sible explanations and testable predictions re-garding the behaviour of biological imitationmechanisms in humans and monkeys (review in(Demiris and Johnson, 2005)).

4.2 Learning hierarchical structures

More recently we have designed and implemented ahierarchical extension (Demiris and Johnson, 2003),to this arrangement: primitive inverse models arecombined to form higher more complex sequences,

with the eventual goal of achieving increasingly moreabstract inverse models. Given a set of primitiveinverse models and forward models (for examplein (Demiris and Johnson, 2003) we used raise andlower gripper platform and open and close grip-per) more complex ones can be created throughobservation (details of the learning algorithm in(Demiris and Johnson, 2003)). Incremental learningand scaffolding have already been highlighted as cru-cial factors in development (Lungarella et al., 2003,Weng, 2004).

One of the crucial issues in the learning ofmore complex structures is that of resetting(Demiris, 2002). If the demonstrator is executing aprimitive inverse model, then selection of the appro-priate inverse model for the imitator is performed bya winner-take-all approach: the inverse model withthe higher confidence is selected. However, in caseswhere the demonstrator performs a more complex se-quence, the sychronization and timing of the activa-tion of the inverse models in the imitator’s repertoirebecome critical.

More specifically, the fact that an inverse modelalready present in the imitator’s repertoire might ap-pear at any point during the demonstration as part ofthe demonstrated sequence requires the existence ofa mechanism that can reinitialise (reset) the imita-tor’s inverse model during the demonstration. Thatwill involve re-running the initialisation steps thateach inverse model takes at the start of the experi-ment, i.e. the state of the inverse model is set to thatof the demonstrator, its confidence is set to zero, andthe inverse model starts moving towards its first (oronly) goal, so even if the inverse model has a low con-fidence because it didn’t match well previous partsof the demonstrated sequence, it will be given a newchance for the current part.

The crucial issue here is, during the demonstra-tion, when should an inverse model in the imitator’srepertoire be reset? The available options are: (a)Reset upon completion of this inverse model. (b) Re-set when the confidence level of this inverse modelhas dropped below a certain value (c) Reset when allinverse model in the imitator’s repertoire have beencompleted

We have performed experiments (Demiris, 2002)investigating the behavioural effects of these threeoptions, as well as combinations; the best resultswere obtained through the adoption of the third op-tion, where inverse models are reset when all inversemodels have been completed. This was implementedthrough a mutual-inhibition mechanism: each in-verse model inhibits the resetting of all other inversemodels until it reaches its goal. When all inversemodels have reached their goals, all inhibitions havebeen removed, and all inverse models are reset. Thishas the side effect that we might get better perfor-

35

mance if we keep the different hierarchical inversemodels balanced in terms of length; we are currentlyinvestigating whether this is indeed the case.

5. Discussion

5.1 Advantages and disadvantages from a

distributed approach

HAMMER stores procedural knowledge in the formof multiple inverse models. By doing so, it allowsdifferent solutions to a problem to co-exist withoutconsiderable interference between them. This is asignificant advantage over approches where a globalcentralised controller is learned, and catastrophicinterference (Schaal and Atkeson, 1998) is observed.This is particularly true for situations where thelearning data are obtained incrementally, rather thanbeing available in advance (Schaal, 2002).

However our architecture will over time continueto learn an increasingly high number of inverse mod-els. Although the inverse models run in parallel, theydo compete on the basis of the quality of predic-tions they generate, and as discussed in the previoussection, when placed in sequences and hierarchicalstructures, can mutually inhibit each other, thus thedemand for an efficient organisation. One way ofdealing with the combinatorial explosion is to removeor combine inverse models based for example on us-age, and thus implementing forgetting or memoryconsolidation mechanisms (Robertson et al., 2004).

5.2 Open challenges

There are a number of challeges that remain un-solved:

• In our current experiments, forward models arelearned between the motor commands and theimage plane based visual perception of the effectsof these actions. A generalisation of these to moreabstract representations (for example a 3-D bodyschema) will allow more complex relationships tobe addressed (the visual information containedin a Japanese bow for example, is different whenyou perform it from when you observe it)

• Object oriented actions - although we have per-formed a number of experiments learning hierar-chical structure involving object oriented actions(Johnson and Demiris, 2004), we have used hard-wired object oriented action primitives. We are inthe process of applying the BBN approach aboveto learning the effects of motor commands on ob-jects.

Elsewhere (Demiris and Khadhouri, 2005) we alsoelaborate on the issue of top-down control and jointattention (Kozima et al., 2003, Nagai et al., 2003) in

the selection of features to observe and imitate. De-spite the need for further research on these high-lighted challeges, our approach so far demonstrates adevelopmental pathway that allows a robot to com-bine learning from self-exploration and learning fromimitation in a principled way.

Acknowledgements

This research was supported by the Engineeringand Physical Sciences Research Council (EPSRC -Grant GR/S11305/01 to YD, and a Doctoral Train-ing Award to AD) and the Royal Society. Thanksto all the BioART members, including MatthewJohnson, Bassam Khadhouri, Gavin Simmons andPaschalis Veskos, to Oliver Grau of BBC Researchand Development, and to three anonymous review-ers for their feedback.

References

Billard, A. (2000). Learning motor skills by im-itation: a biologically inspired robotic model.Cybernetics and Systems, 32:155–193.

Breazeal, C. and Scassellati, B. (2002). Robots thatimitate humans. Trends in Cognitive Sciences,6(11):481–487.

Carruthers, P. and Smith, P. K. (1996). Theories ofTheories of Mind. Cambridge University Press.

Conell, J. H. and Mahadevan, S., (Eds.) (1993).Robot Learning. Kluwer Academic Publishers.

Dautenhahn, K. and Nehaniv, C. (2002). Imita-tion in Animals and Artifacts. MIT Press, Cam-bridge, MA, USA.

Dearden, A. and Demiris, Y. (2005). Learning for-ward models for robotics. In Proceedings of IJ-CAI.

Decety, J., Grezes, J., Costes, N., Perani, D., Jean-nerod, M., Procyk, E., Grassi, F., and Fazio, F.(1997). Brain activity during observation of ac-tions: Influence of action content and subject’sstrategy. Brain, 120:1763–1777.

Demiris, Y. (2002). Imitation, mirror neurons, andthe learning of movement sequences. In Proceed-ings of the International Conference on NeuralInformation Processing (ICONIP-2002), pages111–115. IEEE Press.

Demiris, Y. and Birk, A., (Eds.) (2000). Interdis-ciplinary approaches to robot learning. WorldScientific.

Demiris, Y. and Hayes, G. (2002). Imitation asa dual route process featuring predictive and

36

learning components: a biologically-plausiblecomputational model. In Dautenhahn, K. andNehaniv, C., (Eds.), Imitation in Animals andArtifacts. MIT Press.

Demiris, Y. and Johnson, M. (2003). Distributed,prediction perception of actions: a biologicallyinspired architecture for imitation and learning.Connection Science, 15(4):231–243.

Demiris, Y. and Johnson, M. (2005). Simulationtheory of understanding others: A robotics per-spective. In Dautenhahn, K. and Nehaniv, C.,(Eds.), Imitation and Social Learning in Robots,Humans and Animals: Behavioural, Social andCommunicative Dimensions. Cambridge Uni-versity Press.

Demiris, Y. and Khadhouri, B. (2005). Hierarchi-cal, attentive multiple models for execution andrecognition. In Proceedings of the IEEE ICRAWorkshop on Social Mechanisms of Robot Pro-gramming by Demonstration.

di Pellegrino, G., Fadiga, L., Fogassi, L., Gallese,V., and Rizzolatti, G. (1992). Understandingmotor events: a neurophysiological study. Ex-perimental Brain Research, 91:176–180.

Franklin, J., Mitchell, T., and Thrun, S., (Eds.)(1996). Recent advances in Robot Learning.Kluwer Academic Publishers, Boston, MA,USA.

Gopnik, A. and Schulz, L. (2004). Mechanisms oftheory formation in young children. Trends inCognitive Sciences, 8(8):371–377.

Johnson, M. and Demiris, Y. (2004). Abstraction inrecognition to solve the correspondence problemfor robot imitation. In Proceedings of TAROS,pages 63–70, Essex.

Kozima, H., Nakagawa, C., and Yano, H. (2003).Attention coupling as a prerequisite for socialinteraction. In Proceedings of the IEEE Inter-national Workshop on Robot and Human In-teractive Communication, pages 109–114. IEEEPress.

Kuniyoshi, Y., Inaba, M., and Inoue, H. (1994).Learning by watching: Extracting reusable taskknowledge from visual observation of humanperformance. IEEE Transactions on Roboticsand Automation, 10(6):799–822.

Lungarella, M., Metta, G., Pfeifer, R., and Sandini,G. (2003). Developmental robotics: a survey.Connection Science, 15(4):151–190.

Meltzoff, A. N. and Moore, M. K. (1997). Explain-ing facial imitation: a theoretical model. EarlyDevelopment and parenting, 6(2):157.1–14.

Miall, R. C. and Wolpert, D. M. (1996). Forwardmodels for physiological motor control. NeuralNetworks, 9(8):1265–1279.

Nagai, Y., Hosoda, K., Morita, A., and Asada, M.(2003). A constructive model for the develop-ment of joint attention. Connection Science,15(4):211–229.

Narendra, K. S. and Balakrishnan, J. (1997). Adap-tive control using multiple models. IEEE Trans-actions on Automatic Control, 42(2):171–187.

Robertson, E. M., Pascual-Leone, A., and Miall,R. C. (2004). Current concepts in proceduralconsolidation. Nature Reviews Neuroscience, 5.

Schaal, S. (1999). Is imitation learning the way tohumanoid robots? Trends in cognitive sciences,3(6):233–242.

Schaal, S. (2002). Learning robot control. In Arbib,M., (Ed.), The Handbook of brain theory andneural networks, pages 983–987. MIT Press.

Schaal, S. and Atkeson, C. (1998). Constructive in-cremental learning from only local information.Neural Computation, 10:2047–2084.

Schaal, S., Ijspeert, A., and Billard, A. (2003).Computational approaches to motor learningby imitation. Phil. Trans. R. Soc London B,358:537–547.

Sutton, R. S. and Barto, A. G. (1998). Reinforce-ment learning: an introduction. MIT Press.

Weng, J. (2004). Developmental robotics: the-ory and experiments. Journal of HumanoidRobotics, 1(2):199–236.

Wolpert, D. M., Doya, K., and Kawato, M. (2003).A unifying computational framework for motorcontrol and social interaction. Phil. Trans. ofthe Royal Society of London B, 358:593–602.

37