bioinspired character animations: a mechanistic and cognitive view

FTC 2016 - Future Technologies Conference 20166-7 December 2016 | San Francisco, United States

Bio-Inspired Animated Characters: A Mechanistic &Cognitive View

Ben KenwrightSchool of Media Arts and Technology

Southampton Solent UniversityUnited Kingdom

Abstract—Unlike traditional animation techniques, which at-tempt to copy human movement, ‘cognitive’ animation solutionsmimic the brain’s approach to problem solving, i.e., a logical (in-telligent) thinking structure. This procedural animation solutionuses bio-inspired insights (modelling nature and the workings ofthe brain) to unveil a new generation of intelligent agents. As withany promising new approach, it raises hopes and questions; anextremely challenging task that offers a revolutionary solution,not just in animation but to a variety of fields, from intelligentrobotics and physics to nanotechnology and electrical engineering.Questions, such as, how does the brain coordinate muscle signals?How does the brain know which body parts to move? With allthese activities happening in our brain, we examine how ourbrain ‘sees’ our body and how it can affect our movements.Through this understanding of the human brain and the cognitiveprocess, models can be created to mimic our abilities, such as,synthesizing actions that solve and react to unforeseen problemsin a humanistic manner. We present an introduction to theconcept of cognitive skills, as an aid in finding and designing aviable solution. This helps us address principal challenges, suchas: How do characters perceive the outside world (input) andhow does this input influence their motions? What is requiredto emulate adaptive learning skills as seen in higher life-forms(e.g., a child’s cognitive learning process)? How can we controland ‘direct’ these autonomous procedural character motions?Finally, drawing from experimentation and literature, we suggesthypotheses for solving these questions and more. In summary,this article analyses the biological and cognitive workings ofthe human mind, specifically motor skills. Reviewing cognitivepsychology research related to movement in an attempt toproduce more attentive behavioural characteristics. We concludewith a discussion on the significance of cognitive methods forcreating virtual character animations, limitations and futureapplications.

Keywords—animation; life-like; movement; cognitive; bio-mechanics; human; reactive; responsive; instinctual; learning;adapting; biological; optimisation; modular; scalable

I. INTRODUCTION

Movement is Life Animated films and video games arepushing the limits of what is possible.

In today’s virtual environments, animations tends to bedata-driven [1], [2]. It is common to see animated charactersusing pre-recorded motion capture data, but it is rare tosee the animated characters driven using purely proce-dural solutions. With the dawn of Virtual Reality (VR) andAugmented Reality (AR) there is an ever growing need forcontent - to create indistinguishably realistic virtual worldsquickly and cost effectively. While rendered scenes may appearhighly realistic, the ‘movement’ of actively driven systems(e.g., biological creatures) is an open area of research [2].

Specifically, the question of how to ‘automatically’ createrealistic actions that mimic the real-world. This includes, theability to learn and adapt to unforeseen circumstances in alife-like manner. While we are able to ‘record’ and ‘playback’highly realistic animations in virtual environments, they havelimitations. The motions are constrained to specific skeletontopologies, not to mention, time consuming and challengingto create motions for non-humans (creatures and aliens). Whatis more, the recording of animations for dangerous situations isimpossible using motion capture (so must be manually doneusing artistic intervention). Another key thing to remember,in dynamically changing environments (video games), pre-recorded animations are unable to adapt automatically tochanging situations.

This article attempts to solve these problems using biologi-cally inspired concepts. We investigate neurological, cognitiveand behavioural methods. These methods provide inspirationalsolutions for creating adaptable models that synthesize life-like character characteristics. We examine how the humanbrain ‘thinks’ to accomplish tasks; and how the brain solvesunforeseen problems. Exploiting the knowledge of how thebrain functions, we formulate a system of conditions thatattempt to replicate humanistic properties. We discusses novelapproaches around solving these problems, by questioning,analysing and formulating a system based on the humancognitive processes.

Cognitive vs Machine Learning Essentially, cognitivecomputing has the ability to reason creatively about data,patterns, situations, and extended models (dynamically). How-ever, most statistics-based machine learning algorithms cannothandle problems much beyond what they have seen and learned(match). The machine learning algorithm has to be pairedwith cognitive capabilities to deal with truly ‘new situation’.Cognitive science therefore raises challenges for, and drawsinspiration from, machine learning; and insights about thehuman mind to help inspire new directions for animation.Hence, cognitive computing along with many other disciplineswithin the field of artificial intelligence are gaining popularity,especially in character systems, so in the not so distant futurewill have a colossal impact on the animation industry.

Automation The ability to ‘automatically’ generate phys-ically correct humanistic animations is revolutionary. Removeand add behavioural components (happy and sad). Createanimations for different physical skeletons using a single set oftraining data. Perform a diverse range of actions, for instance,getting-up, jumping, dancing, and walking. The ability to reactto external interventions, while completing assigned task (i.e.,combining motions with priorities). These problem-solving

978-1-5090-4171-8/16/$31.00 c©2016 IEEE 1079 | P a g e


skills are highly valued. We want character agents to learnand adapt to the situation. This includes:

• physically based models (e.g., rigid bodies) thatare controlled through internal joint torques (muscleforces)

• controllable adjustable joint signals to accomplishspecific actions (trained)

• learn and retain knowledge from past experiences• embed personal traits (personality)

Problems We want the method to be automatic (i.e., notdepend too heavily on pre-canned libraries). Avoid simplyplaying back captured animations, but instead parama-terizing and re-using animations for different contexts(provide stylistic advice to the training algorithm). Wewant the solution to have the ability to adapt on-the-fly tounforeseen situations in a natural life-like manner. Havingsaid that, we also want to accommodate a diverse range ofcomplex motions, not just balanced walking, but getting-up,climbing, and dancing actions. With a physics-based modelat the heart of the system (i.e., not just a kinematic skeletonbut joint torques/muscles), we are able to ensure a physicallycorrect solution. While a real-world human skeleton has a hugenumber of degrees-of-freedom, we accept that a lower fidelitymodel is able to represent the necessary visual characteristics(enable reasonable computational overheads). Of course, evena simplified model possesses a large amount of ambiguitywith singularities. All things considered, we do not want tofocus on the ‘actions’ - but embrace the autonomous emotion,behaviour and cognitive properties that sit on top of the motion(intelligent learning component).

Fig. 1. Homunculus Body Map - The somato-sensory homunculus is a kindof map of the body [3], [4]. The distorted model/view of a person (see

Figure 2) represents the amount of sensory information a body part sends tothe central nervous system (CNS)

Geometric to Cognitive Synthesizing animated charactersfor virtual environments addresses the challenges of automat-ing a variety of difficult development tasks. Early researchcombined geometric and inverse kinematic models to simplifykey-framing. Physical models for animating particles, rigidbodies, deformable solids, fluids, and gases have offeredthe means to generate copious quantities of realistic motionthrough dynamic simulation. Bio-mechanical models employsimulated physics to automate the lifelike animation of animals

with internal muscle actuators. In recent years, research in be-havioral modeling has made progress towards ‘self-animating’characters that react appropriately to perceived environmentalstimuli [5], [6], [7], [8]. It has remained difficult, however, toinstruct these autonomous characters so that they satisfy theprogrammer’s goals. As pointed out by Funge et al. [9], thecomputer graphics solution has evolved, from geometric solu-tions to more logical mathematical approaches, and ultimatelycognitive models, as shown in Figure 3.

A large amount of work has been done into motion re-targeting (i.e., taking existing pre-recorded animations andmodifying them to different situations) [10], [11], [12]. Tar-geted solutions that generate animations for specific situations,such as, locomotion [13] and climbing [14]. Kinematic modelsdo not take into account the physical properties of the model, inaddition, are only able to solve local problems (e.g., readingand stepping and not complex rhythmic actions) [15], [16],[17]. Procedural models may not converge to natural lookingmotions [18], [19], [20]. Cognitive models go beyond be-havioral models, in that they govern what a character knows,how that knowledge is acquired, and how it can be used toplan actions. Cognitive models are applicable in instructing anew breed of highly autonomous, quasi-intelligent char-acters that are beginning to find use in interactive virtualenvironments. We decompose cognitive modeling into tworelated sub-tasks: (1) domain knowledge specification and(2) character instruction. This is reminiscent of the classicdictum from the field of artificial intelligence (AI) that triesto promote modularity of design by separating out knowledgefrom control.

knowledge + instruction = intelligent behavior (1)

Domain (knowledge) specification involves administeringknowledge to the character about its world and how that worldcan change. Character instructions tell the character to try tobehave in a certain way within its world in order to achievespecific goals. Like other advanced modeling tasks, both ofthese steps can be fraught with difficulty unless developersare given the right tools for the job.

Components We wanted to avoid a ‘single’ amalgamatedalgorithm (e.g., Neural Networks or connectionist models[21]). Instead we investigate modular or dissectable learningmodels for adapting joint signals to accomplish tasks. Forexample, genetic algorithms [18], in combination with Fouriermethods to subdivide complex actions into components (i.e.,extract and identify behavioural characteristics [22]). Coupledwith the fact that, joint motions are essentially signals, whilethe physics-based model ensures the generated motions arephysically correct [23]. To say nothing of the advancements inparallel hardware - we envision the exploitation of massivelyparallel architecture constitutional.

Contribution The novel contribution of this technicalarticle is the amalgamation of numerous methods, for instance,bio-mechanics, psychology, robotics, and computer animation,to address the question of ‘how can we make virtual characterssolve unforeseen problems automatically and in a realisticmanner?’ (i.e., mimic the human cognitive learning process).

978-1-5090-4171-8/16/$31.00 c©2016 IEEE 1080 | P a g e


Fig. 3. Timeline - Computer Graphics Cognitive Development Model (Geometric, Kinematic, Physical, Behavioural, and Cognitive) ([9]. Simplified illustrateof milestones over the years that have contributed novel animation solutions - emphasises the gradual transition from kinematic and physical techniques to

intelligent behavioural models. [A] [24]; [B] [20]; [C] [19]; [D] [25]; [E] [26]; [F] [27]; [G] [28]; [H] [18]; [I] [29]; [J] [30]; [K] [31]; [L] [32]; [M] [33]; [N][34]; [O] [35]; [P] [8]; [Q] [36]; [R] [7]; [S] [5]; [T] [6]; [U] [37]; [V] [38];

Fig. 2. Homunculus Body Map - Reinert et al [4], presented a graphicalpaper on mesh deformation to visualize the somato-sensory information of

the brain-body. The figure conveys the importance of the neuronalhomunculus - i.e., the human body part size relation to neural density and

the brain.

II. BACKGROUND & RELATED WORK

Literature Gap The research in this article brings togethernumerous diverse concepts and while in their individual fieldthey are well studied, in their whole and applied to virtualcharacter animations, there is a serious gap in the referentialliterature. Hence, we begin by exploring branches of researchfrom cognitive psychology and bio-mechanics before takingthem across and combining them with computer animation androbotics concepts.

Autonomous Animation Solutions Formal approaches toanimation, such as, genetic algorithms [18], [19], [20], maynot converge to natural looking motions without additional

work, such as, artist intervention or constrained/complex fit-ness functions. This causes limitations and constrains the ‘au-tomation’ factor. We see autonomy as the emergent of salient,novel, action discovery, through self organisation of high levelgoal directed orders. The behavioural aspect emerges fromthe physical (or virtual) constraints and fundamental low levelmechanisms. We adapt bodily motor controls (joint signals)from randomness to purposeful actions based on cognitivedevelopment (Lee [39] referred to this process as evolvingfrom babbling to play). Interestingly, this intrinsic method ofbehavioural learning has also been demonstrated in biologicalmodels (known as action discovery) [40].

Navigation/Controllers/Mechanical Synthesizing humanmovement that mimics real-world behaviours ‘automatically’is a challenging and important topic. Typically, reactive ap-proaches for navigation and pursuit [24], [41], [42], [27], maynot readily accommodate task objectives, sensing costs, andcognitive principles. A cognitive solution adapts and learns(finds answers to unforeseen problems).

Expression/Emotion Humans exhibit a wide variety ofexpressive actions, which reflect their personalities, emotions,and communicative needs [25], [26], [28]. These variationsoften influence the performance of simpler gestural or facialmovements.

Components Essential Components:

• Fourier - subdivide actions into components, extractand identify behavioural characteristics [22]

• Heuristic Optimisation [18] - adapting non-linearsignals (with purpose)

• Physics-Based [43], [23] - torques and forces tocontrol the model

• Parallel Architecture - exploit massively parallel pro-cessor architecture, such as, the graphical processingunit (GPU)

978-1-5090-4171-8/16/$31.00 c©2016 IEEE 1081 | P a g e


• Randomness - inject awareness and randomness(blood flow, repository signals, background noise)[44], [45]

Brain Body Map As shown in Figure 1, we are ableto map the minds awareness of different body parts. This isknown as the homunculus body map. So why is it importantfor movement? Helps understanding the neural mechanismsof human sensori-motor coordination and cognitive connec-tion. While we are a complex biological organism, we needfeedback and information (input) to be able to move and thuslive (i.e., movement is life). The motor part of the brain relieson information from the sensory systems. The control signalsare dynamically changing depending on our state. Simply put,the better the central representation, the better the motoroutput will be and the more life-like and realistic the finalanimations will be. Our motor systems need to know the stateof our body. If the situation is not known or not very clear, themovements will not be good, because the motor systems willbe ‘afraid’ to go all out. Very similar to driving a car on anunknown road in misty conditions with only an old, worn andworm eaten map. We drive slow and tense, to avoid hittingsomething or getting of road. This is safety behaviour: safe,but taxing on the system.

Cognitive Science The cognitive science of motion is aninterdisciplinary scientific study of the mind and its processes.We examines what cognition motion is, what it does and howit works. This includes research in to intelligence and be-haviour, especially focusing on how information is represented,processed, and transformed (in faculties such as perception,language, memory, attention, reasoning, and emotion) withinnervous systems (humans or other animals) and machines(e.g. computers). Cognitive motion science consists of multipleresearch disciplines, including robotics, psychology, artificialintelligence, philosophy, neuroscience, linguistics, and anthro-pology. The subject spans multiple levels of analysis, fromlow level learning and decision mechanisms to high levellogic and planning; from neural circuitry to modular brainorganization. However, the fundamental concept of cognitivemotion is the understanding of instinctual thinking in terms ofthe structural mind and computational procedures that operateon those structures. Importantly, cognitive solutions are notonly adaptive but also anticipatory and prospective, thatis, they need to have (by virtue of their phylogeny) or develop(by virtue of their ontogeny) some mechanism to rehearsehypothetical scenarios.

Neural Networks and Cognitive Simulators Compu-tational Neuroscience [46], [29], [47] biologically inspiredsolutions for neural models for simulating information process-ing and cognition and behaviour modelling. The majority ofthe research has focused on modelling ‘isolated components’.Cognitive architectures [48] using biologically based modelsfor goal driven learning and behaviours. Publically availableneural network simulators are available [49].

Motor Skills Our brain sees the world in ‘maps’. The mapsare distorted, depending on how we use each sense, but they arestill maps. Almost every sense has a map. Most senses havemultiple maps. We have a ‘tonotopic’ map, which is a mapof sound frequency, from high pitched to low pitched, whichis how our brain processes sound. We have a ‘retinotopic’map, which is a reproduction of what you are seeing, and itis how the brain processes sight. Our brain loves maps. Most

importantly, we have maps of our muscles. The mappingfrom sensory information to motor movement is shown inFigure 1. For muscle movements, the finer, more detailed themovements are, the more brain space those muscles have.Hence, we can address which muscles take priority and underwhat circumstances (i.e., sensory input). This also opens thedoor to lots of interesting and exciting questions, such as, whathappens to the maps if we lose a body part, such as, a finger.

Psychology Aspect A number of interesting facts arehidden in the psychology aspect of movement that are oftentaken for granted or overlooked. Incorporating them in adynamic system allows us to solve a number of problems.For example, when we observe movements which are slightlydifferent from each other but possess similar characteristics.The work by Armstrong [50], showed that when a movementssequence is speeded up as a unit, the overall relative move-ment or ‘phasing’ remains constant. Led to the discovery ofrelative forces or the relationship among forces in the musclesparticipating in the action.

How the Brain Controls Muscles Let us pretend thatwe want to go to the kitchen, because we are hungry. First,an area in our brain called the parietal lobe comes up witha lots of possible plans. We could get to the kitchen byskipping, sprinting, uncoordinated somersaulting, or walking.The parietal lobe sends these plans to another brain area calledthe basal ganglia. The basal ganglia picks ‘walking’ as thebest plan (with uncoordinated somersaulting as close secondoption). It tells the parietal lobe the plan. The parietal lobeconfirms it, and sends the ‘walk to kitchen’ plan down thespinal cord and to the muscles. The muscles move. As theymove, our cerebellum kicks into high gear, making sure weturn right before we crash into the kitchen counter, and thatwe jump over the dog. Part of the cerebellum’s job is tomake quick changes to muscle movements while they arehappening (see Figure 4).

Visualizing the Solution (Offline) We visualize a goal.In our mind, over and over and over again. We picture themovements. We see ourself catching that ball. Dancing thattoe touch. Swimming that breaststroke. We watch it in themovie of our mind whenever we can. Scrutinize it. Is our wristturning properly? Is our kick high enough? If not, we changethe picture. See ourself doing the movement perfectly. As faras our parietal lobe and basal ganglia are concerned, this isexactly the same as doing the movement. When we visualizethe movement, we activate all those planning pathways. Thoseneurons fire, over and over again. Which is what needs tohappen for our synapses to strengthen. In other words, bypicturing the movements, we are actually learning them. Thismakes it easier for the parietal lobe to send the right message tothe muscles. So when we actually try to perform a movement,we will get better, faster. We will need less physical practiceto be good at sports. This does not work for general fitness(i.e., increased strength). We still need to train our muscles,heart, and lungs to become strong. However, its good forskilled movements. Basketball lay ups. Gymnastics routines.For improved technique, visualization works. We train ourbrain, which makes it easier to control our muscles. Whatdoes this have to do with character simulations? We are ableto mimic the ‘visualization’ approach by having our systemconstantly run simulations in the background. Exploit all thatparallel processing power. Run large numbers of simulations

978-1-5090-4171-8/16/$31.00 c©2016 IEEE 1082 | P a g e


Fig. 4. Brain and Actions - The phases (left-to-right) the human brain goes through - from thinking about doing a task to accomplishing it (e.g., walking tothe kitchen to get a drink from the cupboard).

Fig. 5. Overview - High level view of interconnected components and their justifications. (a) We have a current (starting) state and a final state. The unknownmiddle transitioning states is what we are searching for. The transition state is a dynamic problem that is specific to the problem. For instance, the terrain or

the situation may vary (slopes or crawling under obstacles). (b) A heuristic model would be able to train a set of trigonometric functions (e.g., Fourier series),to create rhythmic motions that are able to accomplish the task. The low level task (fitness function), being a simple ‘overall centre of mass trajectory’. (c)

With (b) on its own, the solution is plagued with issues, such as, how to steer or control the type of motion and if the final motion is ‘humanistic’ or‘life-like’. Hence, we have a ‘pre-defined’ library of motions that are chosen based on the type of animation we are leaning towards (standard walk or

hopping). The information from the animation is fed back into the fitness function in (b). Providing a multi-objective problem, centre of mass, end-effectors,and frequency components for ‘style’. (d) The solution from each problem is ‘stored’ in a sub-bank of the animation and used for future problems. This builds

upon using previous knowledge to help solve new problems faster in a coherent manner (e.g., previous experiences will cause different characters to createslightly different solutions over time).

978-1-5090-4171-8/16/$31.00 c©2016 IEEE 1083 | P a g e


one or two seconds in advance and see how the result leads out.If the character’s food it a few centimetres forward, if we usemore torque on the knee muscle, how does this compare withour ideal animation we are aiming for? As we find solutions,we store them and improve upon them each time a similarsituation arises.

Physically Correct Model Our solution controls a physicsbased model using joint torques as in the real world. Thismimics the real world more closely, not only do we requirethe model to move in a realistic manner but it also has tocontrol joint muscles in sufficient ratios to achieve the finalmotion (e.g., balance control). Adjusting the physical model,for instance, muscle strength or leg lengths, allows the modelto retrain to achieve the action.

(Get Up) Rise Animations Animation is diverse andcomplex area, so rather than try and create solutions for everypossible situation, we focus on a particular set of actions,that is, rising movements. Rise animations require a suitablydiverse range of motor skills. We formate a set of tasksto evaluate our algorithm, such as, get up from front, getup from back, get up on uneven ground and so on. Themodel also encapsulates underlying properties, such as, visualattention and expressive qualities (tired, unsure, eager) andhuman expressiveness. We consider a number of factors, suchas, inner and outer information, emotion, personality, primaryand secondary goals.

III.OVERVIEW

High Level Elements The system is driven by three keysources of information:

1) the internal information (e.g., logistics of the brain,experience, mood)

2) the aim or action3) external input (e.g., environmental, contacts, comfort,

lighting)4) memory and information retrieval (e.g., parallel mod-

els and associative memory)

Motion Capture Data (Control) We have a library ofactions as reference material for look-up and comparison.Some form of ‘control’ and ‘input’ to steer the characters toperform actions in a particular way (e.g., instead of the artistcreating a large look-up array of animations for every singlepossible solution), we provide fundamental poses and simplepre-recorded animations to ‘guide’ the learning algorithm. Assearch models are able to explore their diverse search-spaceto reach the goal (e.g., heuristically adjusting joint muscles),however, a reference ‘library’ allows us to steer the solutiontowards what is ‘natural-looking’. As there are a wide numberof ways of accomplishing a task - but what is ‘normal’and what is ‘strange’ and uncomfortable. The key points weconcentrate on are:

1) the animations requires basic empirical information(e.g., reference key-poses) from human movementand cognitive properties;

2) the movement should not simply reply pre-recordedmotions, but adapt and modify them to differentcontexts;

3) the solution must react to disturbances and changesin the world while completing the given task;

4) the senses provide unique pieces of information,which should be combined with internal personalityand emotion mechanisms to create the desired actionsand/or re-actions.

Blending/Adapting Animation Libraries During motorskill acquisition, the brain learns to map between ‘intended’limb motion and requisite muscular forces. We propose thatregions (i.e., particular body segments) in the animation libraryarea are blended together to find a solution that is aestheticallypleasing. (i.e., based upon pre-recorded motions instead ofrandomly searching).

Virtual Infant (or Baby) Imagine a baby with no knowl-edge or understanding. As we explained, a bottom up view,starting with nothing and educating the system to mimichumanistic (organic) qualities. Learning algorithms to tuneskeletal motor signals to accomplish high-level tasks. As with achild - ‘trial-and-error’ approach to learning - exploring whatis possible and impossible - to eventually reach a solution.This requires continuously integrating in corrective guidance(as with a child - without knowing what is right and wrong- the child will never learn). This guidance is through fitnesscriteria and example motion clips (as children do - see andcopy - or try to). Performing multiple training exercises overand over again to learn skills. Having the algorithm activelyimprove (e.g., proprioception - how the brain understands thebody). As we learn to perform motions, there are thousandsof small adjustments that our body as a whole is makingevery millisecond to ensure optimal (quickest, energy efficient,closest idea/style). Constantly monitoring the body by sendingand receiving sensory information (e.g., to and from everyjoint, limb, and contact). Over time, the experience strengthensthe model’s ability to accomplish tasks quicker and moreefficiently.

Stability Autonomous systems have ‘stability’ issues (i.e.,they are far from equilibrium stability) [51]. Due to thedynamic nature of a character’s actions, they are dependentfor their environment (external factors) requiring interaction,which are open processes (exhibit closed self-organization).However, we can measure stability in relation to referenceposes, energy, and balance to draw conclusions of the effec-tiveness of the learned solution.

Memory Learn through explorative searching (i.e, withquantative measures for comfort, security, and satisfaction).While a character may find an ‘optimal’ solution that meets thespecified criteria - it will continue to expand its memory reper-toire of actions. This is a powerful component, increasing theefficiency in achieving a goal (e.g., the development of walkingand retention of balanced motion in different circumstanceswould be more effective). The view that exploration and re-tention (memory) is crucial to ontogenetic development, whichis supported by research findings in developmental psychology[52]. Hofsten [53] explains that it is not necessarily successat achieving task-specific goals that drives development butthe discovery of new way of doing something (through explo-ration). Forms a solution that builds upon ‘prior knowledge’with an increased reliance on machine learning and statisticalevaluation (i.e., for tuning the system parameters). This leadsto an model that constantly acquires new knowledge both forthe current and future task.

978-1-5090-4171-8/16/$31.00 c©2016 IEEE 1084 | P a g e


IV.COMPLEXITY

Experimenting with optimisation algorithms (i.e., differentfitness criteria for specific situations). Highly dynamic ani-mations (jumping or flying through the air). Close proximitysimulations (dancing, wrestling, getting in/out of a vehicle).Exploring ‘beyond’ human but creative creatures (multiplelegs and arms). Instead of aesthetic qualities, investigate ‘in-teresting’ behaviours. As the system and training evolves touse a ‘control language’ to give orders. Not just limited togeneric motions (i.e., walking and jumping), but the ability tolearn and search for solutions (whatever the method). Introducerisk, harm, and comfort to ‘limit’ the solutions to be more‘human’ and organic. Avoid unsupervised learning since itleads to random unnatural and uncontrollable motions. Simpleexamples (i.e., training data) to steer the learning. Gatherknowledge and extend the memory of experiences to help solvefuture problems (learn from past problems). This method isvery promising for building organic real-life systems (handleunpredictable situations in a logical natural manner). Tech-nique is scalable and generalizes across topologies. Learnedsolutions can be shared and transferred between characters(i.e., accelerated learning through sharing).

Fig. 6. Complexity - As animation and behavioural character models becomeincreasing complex, it becomes more challenging and time consuming to

customize and create solutions for specific environments/situations.

An physically correct, self-adapting, learning animationsystem to mimic human cognitive mechanics is a complex taskthat embodies a wide range of biologically based concepts. Abottom up approach (i.e., starting with nothing). This formsa foundation from which greater details can be added. As themodel grows in complexity and details more expressive andautonomous animations appear. Leading on to collaborativeagents, i.e., social learning and interaction (i.e., behaviourin groups). The enormous complexity of the human brainand its ability to problem solve cannot be underestimated- however, through simple approximations we are able todevelop autonomous animation models that embody and pos-sess humanistic qualities, such as, cognitive and behaviourallearning abilities.

Tackle a complex problem - our movement allows us toexpress a vast array of behaviours in addition to solvingphysical problems, such as, balance and locomotion. We haveonly scraped the surface of what is possible - constructing andexplaining a simple solution (for a relatively complex neuro-behavioural model) - to investigate a modular extendible

framework to synthesize human movement (i.e., mappingfunctionality, problem solving, mapping of brain to anatomy,and learning/experience).

Body Language The way we ‘move’ says a lot. How westand and how we walk expels ‘emotional’ details. We humansare very good at spotting these underlying characteristics.These fundamental physiological motions are important inanimation - if we want to synthesize life-like characters. Whilethese subtle underlying motions are aesthetic (i.e., sitting ontop of the physical action or goal), they are non the less equallyimportant. Emotional synthesis is often classified as a low-level biological process [54]. Chemical reactions in the brainfor stress and pain - correlate and modulate various behaviours(including motor control) - vast array of effects - influencingsensitivity, mood, and emotional responses. We have took aview that the motion and learning is driven by a high levelcognitive model (avoid the various underlying physiologicaland chemical parameters).

Input (Sensory Data) The brain has a vast array of sensorydata, such as, the eyes, sound, temperature, smell, and feelings,that feed in to make the final decision. Technically, our simpleassumption is analogous to a blind person taking lots of shortexploratory motions to discover how to accomplish the task.Reduce the skeleton complexity compared to a full humanmodel (numerical complexity). Physical information from theenvironment, like contacts, centre of mass, and end-effectorlocations. The output motor control signals - with behaviouralselection, example learning motion library, emotion, and fitnessevaluation.

V. CONCLUSION

We have specified a set of simple constraints to steer andcontrol the animation (e.g., get-up poses). We developed amodel based on biology, cognitive psychology, and adaptiveheuristics to create animations to control a physics-basedskeleton that adapts and re-trains parameters to meet changingsituations (e.g., different physical and environmental informa-tion). We inject personality and behavioural components tocreate animations that capture life-like qualities (e.g., mood,tired, and scared).

This article addresses several possibilities for future work.It would be valuable to do further tests on specific hypothesesand assumptions by constructing more focused and rigorousexperiments. However, these hypotheses are hard to stateprecisely, and thus have mixed feelings - since we are tryingto model humanistic cognitive abilities. A practical approachmight be to directly compare and contrast real-world andsynthesized situations. For instance, an experiment of an actordealing with difficult situations, such as, stepping over objectsand walking under bridges. Younger children approach theproblem in a different way - similar to our computer agent- learning through trial and error, behaving less mechanicallyand more consciously. Further, communication between direc-tor (e.g., example animations and posses for control) mightlead to more formal languages of commands. This would helpus learn precisely what sorts of commands are needed andwhen there should be issued. Finally, we could go furtherby developing richer cognitive models and control languagesfor describing motion and style to solve questions not evenimagined.

978-1-5090-4171-8/16/$31.00 c©2016 IEEE 1085 | P a g e


We have taken a simplified view of cognitive modelling.We will continue to see cognitive architectures develop overthe coming years that are capable of adapting and self-modifying, both in terms of parameter adjustment phylogeneticskills. This will be through learning and, more importantly,through the modification of the very structure and organizationof the system itself (memory and algorithm) so that it iscapable of altering its system dynamics based on experience,to expand its repertoire of actions, and thereby adapt to newcircumstances [52]. A variety of learning paradigms will needto be developed to accomplish these goals, including, butnot necessarily limited to, unsupervised, reinforcement, andsupervised learning.

Learning through watching Providing the ability totranslate 2D video images to 3D animation sequences wouldallow cognitive learning algorithms the ability to constantly‘watch’ and learn from people. Watching people in the streetwalking and avoiding one another, climbing over obstacles,and interacting to reproduce similar characteristics virtually.

REFERENCES[1] D. Vogt, S. Grehl, E. Berger, H. B. Amor, and B. Jung, “A data-driven

method for real-time character animation in human-agent interaction,”in Intelligent Virtual Agents. Springer, 2014, pp. 463–476. 1

[2] T. Geijtenbeek and N. Pronost, “Interactive character animation usingsimulated physics: A state-of-the-art review,” in Computer GraphicsForum, vol. 31, no. 8. Wiley Online Library, 2012, pp. 2492–2515. 1

[3] E. N. Marieb and K. Hoehn, Human anatomy & physiology. PearsonEducation, 2007. 2

[4] B. Reinert, T. Ritschel, and H.-P. Seidel, “Homunculus warping: Con-veying importance using self-intersection-free non-homogeneous meshdeformation,” Computer Graphics Forum (Proc. Pacific Graphics 2012),vol. 5, no. 31, 2012. 2, 3

[5] T. Conde and D. Thalmann, “Learnable behavioural model for au-tonomous virtual agents: low-level learning,” in Proceedings of the fifthinternational joint conference on Autonomous agents and multiagentsystems. ACM, 2006, pp. 89–96. 2, 3

[6] F. Amadieu, C. Marine, and C. Laimay, “The attention-guiding effectand cognitive load in the comprehension of animations,” Computers inHuman Behavior, vol. 27, no. 1, 2011, pp. 36–40. 2, 3

[7] E. Lach, “fact-animation framework for generation of virtual charactersbehaviours,” in Information Technology, 2008. IT 2008. 1st Interna-tional Conference on. IEEE, 2008, pp. 1–4. 2, 3

[8] J.-S. Monzani, A. Caicedo, and D. Thalmann, “Integrating behaviouralanimation techniques,” in Computer Graphics Forum, vol. 20, no. 3.Wiley Online Library, 2001, pp. 309–318. 2, 3

[9] J. Funge, X. Tu, and D. Terzopoulos, “Cognitive modeling: knowledge,reasoning and planning for intelligent characters,” in Proceedings ofthe 26th annual conference on Computer graphics and interactivetechniques. ACM Press/Addison-Wesley Publishing Co., 1999, pp.29–38. 2, 3

[10] S. Tak and H.-S. Ko, “A physically-based motion retargeting filter,”ACM Transactions on Graphics (TOG), vol. 24, no. 1, 2005, pp. 98–117. 2

[11] S. Baek, S. Lee, and G. J. Kim, “Motion retargeting and evaluation forvr-based training of free motions,” The Visual Computer, vol. 19, no. 4,2003, pp. 222–242. 2

[12] J.-S. Monzani, P. Baerlocher, R. Boulic, and D. Thalmann, “Using anintermediate skeleton and inverse kinematics for motion retargeting,”in Computer Graphics Forum, vol. 19, no. 3. Wiley Online Library,2000, pp. 11–19. 2

[13] B. Kenwright, R. Davison, and G. Morgan, “Dynamic balancing andwalking for real-time 3d characters,” in Motion in Games. Springer,2011, pp. 63–73. 2

[14] C. Balaguer, A. Gimenez, J. M. Pastor, V. Padron, and M. Abderrahim,“A climbing autonomous robot for inspection applications in 3d com-plex environments,” Robotica, vol. 18, no. 03, 2000, pp. 287–297. 2

[15] K. Grochow, S. L. Martin, A. Hertzmann, and Z. Popovic, “Style-basedinverse kinematics,” in ACM Transactions on Graphics (TOG), vol. 23,no. 3. ACM, 2004, pp. 522–531. 2

[16] D. Tolani, A. Goswami, and N. I. Badler, “Real-time inverse kinematicstechniques for anthropomorphic limbs,” Graphical models, vol. 62,no. 5, 2000, pp. 353–388. 2

[17] T. B. Moeslund, A. Hilton, and V. Kruger, “A survey of advances invision-based human motion capture and analysis,” Computer vision andimage understanding, vol. 104, no. 2, 2006, pp. 90–126. 2

[18] B. Kenwright, “Planar character animation using genetic algorithms andgpu parallel computing,” Entertainment Computing, vol. 5, no. 4, 2014,pp. 285–294. 2, 3

[19] K. Sims, “Evolving virtual creatures,” in Proceedings of the 21st annualconference on Computer graphics and interactive techniques. ACM,1994, pp. 15–22. 2, 3

[20] J. T. Ngo and J. Marks, “Spacetime constraints revisited,” in Proceed-ings of the 20th annual conference on Computer graphics and interactivetechniques. ACM, 1993, pp. 343–350. 2, 3

[21] J. A. Feldman and D. H. Ballard, “Connectionist models and theirproperties,” Cognitive science, vol. 6, no. 3, 1982, pp. 205–254. 2

[22] M. Unuma, K. Anjyo, and R. Takeuchi, “Fourier principles for emotion-based human figure animation,” in Proceedings of the 22nd annualconference on Computer graphics and interactive techniques. ACM,1995, pp. 91–96. 2, 3

[23] P. Faloutsos, M. Van de Panne, and D. Terzopoulos, “Composablecontrollers for physics-based character animation,” in Proceedings ofthe 28th annual conference on Computer graphics and interactivetechniques. ACM, 2001, pp. 251–260. 2, 3

[24] H. Noser, O. Renault, D. Thalmann, and N. M. Thalmann, “Navigationfor digital actors based on synthetic vision, memory, and learning,”Computers and graphics, vol. 19, no. 1, 1995, pp. 7–19. 3

[25] H. H. Vilhjalmsson, “Autonomous communicative behaviors in avatars,”Ph.D. dissertation, Massachusetts Institute of Technology, 1997. 3

[26] J. Cassell, H. H. Vilhjalmsson, and T. Bickmore, “Beat: the behaviorexpression animation toolkit,” in Life-Like Characters. Springer, 2004,pp. 163–185. 3

[27] X. Tu and D. Terzopoulos, “Artificial fishes: physics, locomotion,perception, behavior,” in Proceedings of the 21st annual conference oncomputer graphics and interactive techniques. ACM, 1994, pp. 43–50.3

[28] J. Cassell, C. Pelachaud, N. Badler, M. Steedman, B. Achorn, T. Becket,B. Douville, S. Prevost, and M. Stone, “Animated conversation: rule-based generation of facial expression, gesture & spoken intonationfor multiple conversational agents,” in Proceedings of the 21st annualconference on Computer graphics and interactive techniques. ACM,1994, pp. 413–420. 3

[29] X. Yao, “Evolving artificial neural networks,” Proceedings of the IEEE,vol. 87, no. 9, 1999, pp. 1423–1447. 3, 4

[30] H. A. ElMaraghy, “Kinematic and geometric modelling and animationof robots,” in Proc. of Graphics Interface’86 Conference. ACM, 1986,pp. 15–19. 3

[31] C. W. Reynolds, “Computer animation with scripts and actors,” in ACMSIGGRAPH Computer Graphics, vol. 16, no. 3. ACM, 1982, pp. 289–296. 3

[32] N. Burtnyk and M. Wein, “Interactive skeleton techniques for enhancingmotion dynamics in key frame animation,” Communications of theACM, vol. 19, no. 10, 1976, pp. 564–569. 3

[33] C. Csuri, R. Hackathorn, R. Parent, W. Carlson, and M. Howard,“Towards an interactive high visual complexity animation system,” inACM SIGGRAPH Computer Graphics, vol. 13, no. 2. ACM, 1979,pp. 289–299. 3

[34] R. A. Goldstein and R. Nagel, “3-d visual simulation,” Simulation,vol. 16, no. 1, 1971, pp. 25–31. 3

[35] A. Bruderlin and T. W. Calvert, “Goal-directed, dynamic animation ofhuman walking,” ACM SIGGRAPH Computer Graphics, vol. 23, no. 3,1989, pp. 233–242. 3

[36] I. Mlakar and M. Rojc, “Towards ecas animation of expressive complexbehaviour,” in Analysis of Verbal and Nonverbal Communication andEnactment. The Processing Issues. Springer, 2011, pp. 185–198. 3

[37] M. Soliman and C. Guetl, “Implementing intelligent pedagogical agentsin virtual worlds: Tutoring natural science experiments in openwonder-land,” in Global Engineering Education Conference (EDUCON), 2013IEEE. IEEE, 2013, pp. 782–789. 3

[38] J. Song, X.-w. Zheng, and G.-j. Zhang, “Method of generating intel-ligent group animation by fusing motion capture data,” in Ubiquitous

978-1-5090-4171-8/16/$31.00 c©2016 IEEE 1086 | P a g e


Computing Application and Wireless Sensor. Springer, 2015, pp. 553–560. 3

[39] M. H. Lee, “Intrinsic activitity: from motor babbling to play,” in De-velopment and Learning (ICDL), 2011 IEEE International Conferenceon, vol. 2. IEEE, 2011, pp. 1–6. 3

[40] K. Gurney, N. Lepora, A. Shah, A. Koene, and P. Redgrave, “Actiondiscovery and intrinsic motivation: a biologically constrained formal-isation,” in Intrinsically Motivated Learning in Natural and ArtificialSystems. Springer, 2013, pp. 151–181. 3

[41] W.-Y. Lo, C. Knaus, and M. Zwicker, “Learning motion controllerswith adaptive depth perception,” in Proceedings of the ACM SIG-GRAPH/Eurographics Symposium on Computer Animation. Euro-graphics Association, 2012, pp. 145–154. 3

[42] C. W. Reynolds, “Flocks, herds and schools: A distributed behavioralmodel,” in ACM Siggraph Computer Graphics, vol. 21, no. 4. ACM,1987, pp. 25–34. 3

[43] K. Erleben, J. Sporring, K. Henriksen, and H. Dohlmann, Physics-basedanimation. Charles River Media Hingham, 2005. 3

[44] K. Perlin, “Real time responsive animation with personality,” Visual-ization and Computer Graphics, IEEE Transactions on, vol. 1, no. 1,1995, pp. 5–15. 4

[45] B. Kenwright, “Generating responsive life-like biped characters,” in Pro-ceedings of the The third workshop on Procedural Content Generationin Games. ACM, 2012, p. 1. 4

[46] T. Trappenberg, Fundamentals of computational neuroscience. OUPOxford, 2009. 4

[47] P. Dayan and L. Abbott, “Theoretical neuroscience: computationaland mathematical modeling of neural systems,” Journal of CognitiveNeuroscience, vol. 15, no. 1, 2003, pp. 154–155. 4

[48] A. V. Samsonovich, “Toward a unified catalog of implemented cognitivearchitectures.” BICA, vol. 221, 2010, pp. 195–244. 4

[49] R. Brette, M. Rudolph, T. Carnevale, M. Hines, D. Beeman, J. M.Bower, M. Diesmann, A. Morrison, P. H. Goodman, F. C. Harris Jret al., “Simulation of networks of spiking neurons: a review of toolsand strategies,” Journal of computational neuroscience, vol. 23, no. 3,2007, pp. 349–398. 4

[50] T. R. Armstrong, “Training for the production of memorized movementpatterns,” Ph.D. dissertation, The University of Michigan, 1970. 4

[51] M. H. Bickhard, “Autonomy, function, and representation,” Communi-cation and Cognition-Artificial Intelligence, vol. 17, no. 3-4, 2000, pp.111–131. 6

[52] D. Vernon, G. Metta, and G. Sandini, “A survey of artificial cognitivesystems: Implications for the autonomous development of mental ca-pabilities in computational agents,” IEEE Transactions on EvolutionaryComputation, vol. 11, no. 2, 2007, p. 151. 6, 8

[53] C. von Hofsten, On the development of perception and action. London:Sage, 2003. 6

[54] M. Sagar, P. Robertson, D. Bullivant, O. Efimov, K. Jawed, R. Kalarot,and T. Wu, “A visual computing framework for interactive neural systemmodels of embodied cognition and face to face social learning,” inUnconventional Computation and Natural Computation. Springer,2015, pp. 71–88. 7

978-1-5090-4171-8/16/$31.00 c©2016 IEEE 1087 | P a g e