Dance: A Declarative Language for the Control

of Humanoid Robots

Research Report YALEU/DCS/RR-1253

Liwen Huang Paul HudakDepartment of Computer Science, Yale University

New Haven, CT, 06520-8280, U.S.A.{liwen.huang, paul.hudak}@yale.edu

July 25, 2003

1

Abstract

Robots are becoming increasingly common in, and important to, manycommercial, industrial, and military applications. This paper focuses onhow to program humanoid robots, and in particular how to program theirmovements and interactions as easily and as effectively as possible. Thecore of this effort is the design of a domain-specific language called Dancethat is highly abstract, easy to use, yet has enough expressive power todescribe a wide range of useful robot movement.

Dance consists of a non-reactive base inspired by Labanotation, agraphical notation for human motion, and a reactive layer based on Yampa,our latest incarnation of functional reactive programming. The reactivelayer is structured using arrows, a generalization of monads, and gives arobot the ability to react to its environment in critical ways. Dance is alsoindependent of the robot platform being used. Our target is two humanoidrobots being designed and constructed in the Yale Vision and RoboticsLaboratory. We have also written a simulator in which a 3D-rendering ofa humanoid robot is controlled by a Dance program.

1 Introduction

Humanoid robots (often called androids), are becoming increasingly useful asthey advance in sophistication, for two reasons: (a) they can perform in envi-ronments that have been engineered specifically for humans, and (b) they make

1This research was supported in part by NSF Grants CCR-0205542 and CCR-0306046.Any opinions, findings, and conclusions contained in this document are those of the authorsand do not reflect the views of these agenties

1

it easier for humans to interact with computers and other kinds of automationin a natural way.

We are interested in how to program humanoid robots, and in particularhow to program their movements and interactions as easily and as effectively aspossible. The focus is not on developing new algorithms for robot movement orsensing. Rather, once an algorithm is in hand, how does one program a robotto walk, wave its arms, clap, or pick up an object? How does one do so in ahigh-level way that is devoid of unnecessary detail, yet is expressive enough tocapture all desirable movements and interactions?

Current approaches to robot control tend to either focus on very low-levelactuator control, or very high-level control of gross robot motion. The low-levelapproach does not scale well, as it is tedious and error-prone to simultaneouslycontrol large numbers of motors, and it is difficult to abstract over these low-levelcontrollers to encapsulate complex motions involving multiple limbs. The high-level approach also does not scale well, and tends to be very platform-specific:controllers that run on one robot can rarely be reused on others.

Our approach is to capitalize on the common morphology of humanoid robotsand look for “middle-level” abstractions for motion. By expressing the controlof a humanoid robot using these abstractions, we can design a domain specificlanguage (DSL) called Dance that is highly abstract, easy to use, and has enoughexpressive power to describe almost any humanoid robot movement.

The design of Dance is partially inspired by Labanotation [12], a graphicalnotation for expressing human movement developed 75 years ago. In Laban-otation, a dance is captured by a sequence of poses, while how to perform itsmoothly is left to the dancer. This idea is analogous to what we often claimabout functional programming: “say what to do, not how to do it.”

Dance also incorporates ideas from by our previous work on domain-specificlanguages (DSLs) for computer music (Haskore [10, 6, 8]), computer animation(Fran [3, 4]), and software-enabled control (FRP [21], Yampa [9, 17]).

The advantages of using Dance are as follows:

1. Dance is hardware independent and scalable. No two humanoid robotdesigns are going to be alike, so it is important that the mechanism used toprogram them does not depend on a specific design. In Dance, humanoidrobot movement is described at an abstract level that can be interpretedby any particular robot, regardless of whether it only has a bare minimumof degrees of freedom or a highly refined motor system.

2. Dance is expressive. Its core abstractions capture the key aspects of hu-manoid robot movement, and its embedding in Haskell permits the con-struction of higher-level abstractions using Haskell’s higher-order func-tions, lazy evaluation, and so on.

3. Dance is reactive. Borrowing ideas from Yampa [17], we use a declarativeevent-based calculus to give a robot the ability to respond to its environ-ment (through tactile, aural, and visual sensors), to its own body (such as

2

interactions between limbs), and to internal programmatic events (timers,remote messages, user commands, and so on).

4. Dance is a language of behavior, capable of either driving the actions of arobot or representing the actions of other perceived agents. Dance datastructures permit the analysis of robot movement, not just conceptually byhumans, but algorithmically by other programs. Sequences of movementsmay be taken apart, spliced together in new ways, compared for equalityor similarity, and so on.

5. Dance is also amenable to formal reasoning based on a formal algebraicsemantics.

The paper is organized as follows. Section 2 briefly introduces Labanotation.Section 3 presents the basic design of non-reactive Dance. Section 4 discussesthe issue of how to add reactivity with the help of Yampa. In section 5, weshow how to interpret Dance using abstract performance and how to generatea graphics simulator. Sections 6 and 7 discuss related work and conclusions.

2 Introduction to Labanotation

The core ideas in Dance were inspired by Labanotation, and thus we begin witha brief overview of Labanotation’s basic functionality.

Labanotation is a graphical notation for expressing human movement thathas become popular within certain groups of educators, analysts, designers, andperformers of all forms of dance and choreography. Labanotation was inventedin 1928 by Rudolf von Laban (thus the name “Labanotation”), who was animportant figure in European modern dance. Its modern form is due in a largepart to Ann Hutchinson; her book [12] is the most often-referenced Labanotationsource. Labanotation aficionados state that it is possible to express any kind ofhuman motion in the notation, and thus it is natural to look to it for ideas forcontrolling humanoid robots.

A Labanotation score is similar to that used in music, but instead of beingread from left to right, it is read up the page, from bottom to top. As shown inFigure 1(a), the staff consists of three solid vertical lines (analogous to a musicalstaff), with small horizontal marks across the center line to mark the passage oftime (analogous to bar lines). Additional dashed vertical lines represent bodyparts (arm, leg, hand, foot, and so on). Each column contains a sequence ofactions for its corresponding body part. The left side of the body is shown onthe left side of the staff, and the right side of the body on the right side of thestaff. The head is the furthest column to the right.

The innermost column is called the support column, and is assumed to de-scribe the feet unless another body part’s symbol is placed there. This columndescribes which body part holds the weight at any given moment. Jumps aredescribed by an absence of symbols in this column, while walking would be de-scribed by switching forward arrows between the right and left support columns.

3

Figure 1: Labanotation

Moving outward from the center of the staff, we find first the support column,then the leg column, the body column, the arm column, and finally the headcolumn. Other parts (hands, feet, etc.) may be described by adding additionalcolumns.

A system of arrows and other symbols describes the motion for each bodypart, as shown in Figure 1(b). A set of symbols captures three important ele-ments of movement: duration, level, and direction. The duration of a movementis indicated by the length of the symbol. Shading indicates the vertical level ofthe movement: dark for low, a single dot for middle, and slashed for high. Thedirection of the movement is indicated by the way in which the symbol faces,and can indicate either forward, backward, left, right, or middle (neutral). Forexample, a shaded arrow pointing to the right indicates a movement to the rightside of the body in the low position.

Figure 1(c) shows a simple but complete example. The first few bars can beread as follows: Begin with equal weight on both feet, with both arms extendeddownward and outward. Step forward with the left foot, while moving the armsforward and level. Then step to the right with the right foot, while extending

4

the arms outward and level. Then while retaining weight on the right foot, shiftweight centrally to the left, while extending the arms downward and outward.(And so on.) While this example is relatively simple, the full Labanotation con-tains a (somewhat mind-boggling) assortment of annotations for other motionsand motion refinements, such as rotating limbs, refining direction, specifyingdegree of floor contact, turning, jumping, and so on.

Although it is tempting to use Labanotation directly as a means of con-trolling humanoid robots, and indeed graphical software tools exist for editingLabanotation scores, it has several drawbacks:

1. The large assortment of notations, although arguably expressive, is cum-bersome and ad hoc. It takes a great deal of training to learn all of theLabanotation features.

2. There are no abstraction mechanisms, such as the ability to express indi-vidual limb motions and then combine them into a whole, or the abilityto repeat a sequence of moves a certain number of times. In programminglanguage terms, motions and motion sequences are not first-class, and thusthere is no way to abstract over them.

3. Labanotation has no means for expressing reactivity. A human or hu-manoid robot interacts with its environment in many ways, whether throughphysical (i.e. tactile) contact or through its visual or auditory subsystems.It is vital that humanoid robots be able to react to their environments ineffective and programmable ways.

Despite these limitations, the fundamental ideas behind Labanotation areappealing, and we can use them as a basis for the design of a more powerfulrobot programming language.

3 Dance: A DSL for Humanoid Robot Motion

In this section we describe Dance, a domain-specific language for humanoidrobot movement. We first describe a non-reactive version of Dance that isinfluenced largely by Labanotation, discussed in the previous section. We thendescribe a reactive version of Dance, borrowing ideas from Yampa, our owndomain-specific language for reactive hybrid systems [9, 17].

Our methodology for designing domain-specific languages, used successfullymany times in the past, is to first embed the design in the functional languageHaskell [11], from which mundane language features can be inherited, and withinwhich it is easy to prototype new languages that have a look and feel quitedifferent from that of Haskell [7]. Once the design is stable, consideration canbe given to a concrete syntax and semantics.

3.1 Non-reactive Dance

Inspired by Labanotation, we first define some data types that capture Laban-otation’s notions of direction, level, and duration:

5

data Direction = Left | Right | Forward| Backward | Center| LFD | LBD | RFD | RBD

data Level = Low | Mid | Highdata Length = Zero | Quarter | Half

| ThreeFourths | Fulltype Duration = Double

Direction represents the directional Labanotation symbols from figure 1(b),and Level captures its three possible levels. Length represents the relativemovement of a prismatic joint which has only one translational degree of free-dom. The type Duration represents how long a dance lasts.

In addition, we define a data type that captures different kinds of actions,which are somewhat more explicit than that used in Labanotation:

data Action = Move Direction Level| Turn Direction| Transpose Direction| Extend Length| Support

Move d l represents a movement in direction d and level l, just like a movein Labanotation. It is suitable to describe the action of a robot limb when therotating axis is not the axis of the limb. Turn d is used to capture the actionthat a limb turns around its own axis. The above two actions are suitable fordescribing rotational joints. While Transpose d is suitable to describe the rel-ative position of a wheeled robot, Extend l is good for a robot with prismaticjoints. Support is a special movement for support, again drawing from Laban-otation: the idea is that the limb(s) supporting the body may have to move invarious ways to keep the robot from falling over, but we want these moves tobe induced implicitly by the physical constraint of keeping the balance, ratherthan having to describe them explicitly.

There are many kinds of robot body parts, and a particular robot may haveone or more of a particular kind. Thus we do not want to constrain Dance to afixed set of body parts or a fixed number of them. Dance treats a body part asabstract, but captures common operations on them and between them using atype class:

class Parts b wheresuperPart :: b -> [b]childPart :: b -> [b]symmetricPart :: b -> bsize :: b -> XYZjointAt :: b -> b -> Maybe (XYZ, XYZ)

type XYZ = (Double, Double, Double)

6

Given any body part b, superPart b is the body part to which b is attached,if it exists. childPart b does the oposite, it returns the body parts attachedto it. Because most animate creatures and humanoid robots are symmetric, thefunction symmetricPart establishes a mapping between symmetric body parts.size b is the size of body part b. Finally, jointAt b1 b2 returns a pair of thepositions which correspond to the relative position of the joint on body partsb1 and b2, if they are connected. Later we will show how to use these functionsto automatically generate a simulator of any robot.

We now turn to the main data type: Dance. In Labanotation, dances arespecified by several columns (representing different body parts), each of which isa sequence of movements which are performed simultaneously. Dance capturesthese idea in following data type:

data (Parts b) =>Dance b = Prim Action Duration

| Rest Duration| WithPart b (Dance b)| Dance b :+: Dance b| Dance b :=: Dance b

Prim a t constructs a simple dance with only one action a lasting for duration t,just like a single move in Labanotation. We use Rest t to convey no movement,which is expressed through empty space in Labanotation. Constructor WithPartassociates a dance with a particular body part. A dance can also be composedfrom other dances. d1 :+: d2 sequentially composes d1 and d2, like a columnof movement in Labanotation. Any two columns in Labanotation representmovements of different body parts and can be performed simultaneously; thisis captured by constructor :=: which composes two dances concurrently.

There are other implicit rules and constraints in Dance other than the im-plicit rule for Support discussed earlier. These rules affect the interpretation ofa Dance as low-level robot or simulator control. These implicit rules are:

• Although a particular body part b may be “at rest” (via constructor Rest),the movement of other body parts may affect the actual position of b.

• The movement of a super body part of b will affect the pose of b, unlessa different movement (other than rest) is specified explicitly for b.

• When we say body part b is performing a movement Support, we meanthe extremity of b is fixed to the ground, and other body parts will adjusttheir pose to keep the robot from falling over.

Because dances are first class, it is possible to define operations that modifythem in various ways, including time and space transforms, such as speeding up,slowing down, or changing direction or level. Indeed, we have a library of suchcommon operations, which are grouped together based on their functionality:

1. Action Function:

7

changeLevel :: Level -> Action -> ActionflipLR :: Action -> Action

Function changeLevel changes the level of an action. flipLR flips the leftand right directionality

2. Basic Dances: Dance programs with only one action can serve as build-ing blocks for more complex programs:

support, rest, mvFwdl, turnR:: (Parts a) => Duration -> Dance a

support = Prim Supportrest = RestmvFwdl = Prim (Move Forward Low)turnR = Prim (Turn Right)

3. Dance Composition The following functions are used to combine a listof dances into a bigger one:

seqL, parL :: (Parts a) => [Dance a] -> Dance aseqL = foldr (:+:) (Rest 0)parL = foldr (:=:) (Rest 0)

seqL is used to sequentially compose dances, and parL composes dancesin parallel.

4. Dance transformation: These functions provide ways to transformDance programs in useful ways:

delay :: (Parts a) =>Duration -> Dance a -> Dance a

repeatn :: (Parts a) =>Int -> Dance a -> Dance a

repeat :: (Parts a) => Dance a -> Dance atempo :: (Parts a) =>

Double -> Dance a -> Dance atransform :: (Parts a) => (Action -> Action)

-> Dance a -> Dance arewire :: (Parts a) =>

(a -> b) -> Dance a -> Dance bcut :: (Parts a) =>

Duration -> Dance a -> Dance a

Function delay adds a period of time of no action before a dance. repeatnn repeats a Dance fragment n times. tempo changes the speed of a dance.transform updates every action in a dance using the function supplied.rewire allows the reusing of the dance defined for robot type a in robottype b by providing a map function from a to b. cut t d truncates adance at time t.

8

5. Utility Functions:

dur :: (Parts a) => Dance a -> Dur

dur computes the total duration of a dance.

3.2 Examples

With these functions, we can now look at some examples. Even without actualbody parts, we can still define many reusable move sequences, i.e. movementsfor some unspecified body part:

mvl, mvr, wave, pointFwd, noDance:: (Parts a) => Dance a

mvl = seqL [mvFwdl 1.0, support 1.0, mvl]mvr = seqL [support 1.0, mvFwdl 1.0, mvr]wave = seqL [mvFwdh 6.0, mvLfdh 5.0, mvRfdh 5.0,

mvLfdh 5.0, mvRfdh 5.0, mvCntl 5.0]noDance = (rest 1.0) :+: noDance

turn, pointMid :: (Part a) => Direction -> Dance aturn dir = Prim (Turn dir) 2.0pointMid dir = Prim (Move dir Mid) 7.0

Note that mvl, mvr, and noDance are recursively defined; indeed, they are con-ceptually infinite movement sequences. Here the basic rest duration is chosenarbitrary as 1 second. mvl is an infinite sequence of a move forward followed bysupport, whereas mvr is a sequence of support followed a move forward.

In order to give a more concrete example, we suppose we are working on arobot having body parts:

data BpName = Head | Torso | LArm | RArm| LLeg | RLeg

instance Parts BpName where...

We then can proceed to define complete dances by associating the sequencesdefined above with specific body parts and, where appropriate, grouping suchlabeled sequences into dances specifying the simultaneous movement of multiplebody parts:

walk, waveRArm, putDownRArm, pointLArm:: Dance BpName

walk = (WithPart LLeg mvl) :=:(WithPart RLeg mvr)

waveRArm = WithPart RArm wave

turnTorso, pointTo :: Direction -> Dance BpName

9

turnTorso dir = WithPart Torso (turn dir)pointTo dir = WithPart LArm (pointMid dir)

sample = waveRArm :=:((Rest 5) :+: pointTo Forward)

Note how walk makes use of the infinite sequences mvl and mvr. When combinedin parallel, we get a dance in which each leg in turn moves forward while theother is supporting. Thus walk constitutes an infinite dance that specifies thatthe robot walks forward forever. That might not seem very useful by itself, butsuch a sequence may be used in a context that requires only a portion of thatinfinite sequence. In particular, the reactivity that we will introduce shortly willallow the interruption of an infinite sequence upon the occurrence of a specifiedevent, such as approaching a wall, receiving a remote message, or an internaltime-out.

We note that already, by using functional abstraction, we can do quite afew more things than basic Labanotation allows. Body parts, directions, levels,moves, move sequences, and complete dances are all first-class citizens, thus canbe named, passed as arguments to functions, stored in data structures, and soon. Lazy evaluation permits the construction of infinite sequences, and higher-order functions provide even greater levels of abstraction. We believe that thisflexibility will be very useful to a robotics programmer.

We also notice that Dance does not forbid the specification of potentiallynon-sensual actions such as:

WithPart LLeg (mvFwdl 2) :=: WithPart LLeg (mvLfdh 5)

which is a dance requiring the left leg to do two different things simultaneously.This could mean that there are two left legs in a particular robot, or that therobot will choose one of the actions and ignore the other, or that it is simply anillegal dance. The choice depends on the robot details and the programmer’sintentions.

4 Reactive Dance

So far we have presented a language for describing fixed sequences of motion.But in the real world, a humanoid robot must also react to what is happeningin its environment. For this purpose we adopt the ideas of functional reactiveprogramming, as embodied in Yampa [17]. This approach to reactivity is tofirst provide a declarative interface to the real world in which objects and theirattributes are perceived as continuous, time-varying quantities, or what we callsignals. These signals may then be tested for certain properties, thus inducingan event, which in turn triggers the desired reactivity. For example, we maytest the distance from a wall (a signal) to see if it drops below some threshold(thus creating an event), or the temperature of the room (a signal) to see if itexceeds a threshold (an event), and so on. To properly describe reactivity in

10

Dance, we must first explain Yampa’s notions of signals, signal functions, andevents.

Intuitively, a signal function is a pure function mapping signals onto signals,where a signal is a function from time to an instantaneous signal value. Thetype of a signal function mapping a signal of type A onto a signal of type Bis written SFAB. This is captured in the following isomorphism, although theactual implementation is different:

SFAB = (Time → A) → (Time → B)

Another fundamental concept is that of an event. Events occur at discretepoints in time. This idea is captured in Yampa through an option type calledEvent that has one form indicating non-occurrence of an event and one indicat-ing occurrence:

data Event a = NoEvent | Event a

Event isomorphics to Maybe, but it is an abstract data type whose constructorsare not exposed. Yampa provides a rich set of functions for generating eventsources and for operating point-wise on events. The following are ones used inthis paper.

now :: b -> SF a (Event b)after :: Time -> b -> SF a (Event b)attach :: Event a -> b -> Event (a, b)

now x generates an immediate event carrying value x. after t x generates asingle event at time t carrying value x. attach e b extends the content ofevent e with b.

Our approach to introducing reactivity into Dance is to treat a Dance pro-gram as a signal function generating events of type Dance. In other words:

type (Parts b) =>RDance a b = SF a (Event (Dance b))

Here, type a represents the signal type a robot reacts to. It can be anything, forexample it could be readings from the robot’s radio sensors, camera, or touchsensors. Now Yampa’s extensive facilities for reactivity become immediatelyavailable to us. Among those, signal functions and event manipulating functionsare most useful. For example, Yampa provides the following functions, used laterin this paper, to compose and modify signal functions.

identity :: SF a aconstant :: b -> SF a b&&& :: SF a b -> SF a c -> SF a (b, c)>>> :: SF a b -> SF b c -> SF a carr :: (a -> b) -> SF a bswitch :: SF a (b, Event c) -> (c -> SF a b)

-> SF a b

11

identity directly connects the input signal to the output signal. constant bis a signal function generating a signal that yields the value b continuously. s1&&& s2 is a signal function that sends the same input to s1 and s2, and pairstheir outputs. s1 >>> s2 connects the output of s1 and the input of s2. arrf lifts function f to a signal function. Finally, the function switch is the key toreactivity: the expression (sf1 &&& es) ‘switch‘ \e -> sf2 behaves as sf1until the first event in the event stream es occurs, at which point the event’svalue is bound to e and the behavior switches over to sf2.

4.1 An Example of Reactive Dance

In order to give a concrete example, we will assume a simplified high-level formof sensor readings that allow the robot to detect the direction and distance ofhumans and toys from the robot, and that at any point of time there is at mostone event happening:

type SensorReading = Event (Object, Distance, Direction)data Object = Toy | Humantype Distance = Double

Thus, whenever a human or a toy is detected, the sensors will generate an eventindicating the type of object found and where it is.

Consider now the following scenario: A robot starts in a quiescent statedoing nothing. If a human is detected, the robot starts turning in the human’sdirection and waves at the human. If a toy is detected, it starts walking towardthe toy if it is more than 3 meters away, and stops after 30 seconds; otherwise itpoints at the toy. The robot continuously switches between these two behaviorsas the situation evolves. We can achieve this behavior using the following Danceprogram:

interact ::(Parts a) => RDance SensorReading ainteract =

let loop init =switch (init &&& identity)

(\e -> loop (reaction e))in loop (now noDance)

reaction :: (Parts a) =>(Object, Distance, Direction) -> RDance b areaction (Human, dis, dir) =

now (seq_D (turnTorso dir) waveRArm)reaction (Toy, dis, dir) =

if (dis > 3)then (switch

(now (seq_D (turnTorso dir) walk))&&& after 30 ()) \_ -> now noDance

else (now (pointTo dir))

12

The function interact defines two sub-functions, loop and reaction. Thebody of interact is just a call to loop with initial action noDance. Functionnow is applied to a Dance to generate the event representing the start of a Dance.loop is a recursive switch function that starts with a dance of no movement,and watches the input signal for an event. When an event happens, it switchesback to itself again, but this time with a new dance computed from the functionreaction and the event value.

If a human is detected, the function reaction will cause the robot to firstturn toward the human and then wave. If a toy is detected, a choice is madedepending on the distance to the toy. If it is far away, reaction first initiatesa dance of walking toward the toy, but imposes a forced termination after 30seconds. If the toy is close, the robot will simply point in the toy’s generaldirection.

5 The Meaning of Dance

In a real implementation there are many nuances of robot motion relating to theresponse time of motors, the kinematics of motion, physical interactions withobstacles, and so on. Nevertheless, it is useful to specify an abstract notion ofthe meaning of a Dance program that is devoid of these details, and that yieldsan abstract notion of equivalence of robot motion sequences.

5.1 The Semantics of Dance

Our first goal is to interpret the structure of the Dance data type. For thispurpose we define the semantics of a Dance program as an abstract performancewhich is a temporally ordered sequence of action events:

type Time = Doubletype (Parts b) => Performance b = [ActEvent b]type (Parts b) => ActEvent b =

(Time, Maybe b, Action, Duration)

ActEvent t b a d captures the fact that at time t, action a is performed bybody part b for duration d.

We can then define the meaning of a Dance program as follows:

meaning :: (Parts b) => Dance b -> Time -> Performance bmeaning (Rest d) t = []meaning (Prim a d) t = [(t, None, a, d)]meaning (d1 :+: d2) t = let t’ = dur d1

in meaning d1 t ++ meaning d2 (t+t’)meaning (d1 :=: d2) t = merge (meaning d1 t) (meaning d2 t)meaning (WithPart p d) t = mapBp p (meaning d t)

merge [] b = b

13

merge b [] = bmerge (a:as) (b:bs)=

if a < b then a : merge as (b:bs)else b : merge (a:as) bs

mapBp p l = [(t, Just p, a, d)|(t,_,a,d)<-l]

Given the start time t, the semantic function meaning d t generates thecomplete performance for dance d. merge is simply a time-ordered merge func-tion. The definition of < is omitted here, as it compares entire events to ensurethat merge is commutative. Function dur computes the duration of a Danceprogram.

We can now define a notion of equivalence:

Definition 1 Two dance programs d1 and d2 are equivalent, written d1 ≡ d2,if and only if:

(∀t) meaning d1 t = meaning d2 t

where “=” is equality on values.

One of the most useful things we can do with this notion of equivalence isestablish the validity of certain axioms about dance programs; for example:

Axiom 1 For any dance d1 and d2

d1 :=: d2 ≡ d2 :=: d1

Proof: For any t:

meaning (d1 :=: d2) t= merge (meaning d1 t) (meaning d2 t) -- unfold= merge (meaning d2 t) (meaning d1 t) -- commutativity= meaning (d2 :=: d1) t

Many other useful axioms can be stated and proved correct in this way. We listthe most basic ones in Figure 2.

5.2 From Performance to Simulation

Performance is good enough to do formal reasoning about Dance at an abstractlevel. However, if we want to simulate dance, we need to know the exact pose ofthe robot at any given point of time. We begin by defining an abstract notionof robot pose:

type (Parts b) => RobotPose b = [(Maybe b, Pose)]data Pose = M (Angle, Angle) | R Angle

| E Pos1D | T (Pos1D, Pos1D)type Angle = Floattype Pos1D = Float

14

d ≡ dd1 :=: d2 ≡ d2 :=: d1

d1 : + : (d2 : + : d3) ≡ (d1 : + : d2) : + : d3d1 :=: (d2 :=: d3) ≡ (d1 :=: d2) :=: d3

WithPart p d1 :=: WithPart p d2 ≡ WithPart p (d1 :=: d2)d ≡ d : + : (Rest 0)

(Rest 0) : + : d ≡ dRest (t1 + t2) ≡ (Rest t1) : + : (Rest t2)

d :=: Rest t ≡ d, (dur d ≥ t)(d1 : + : d2) :=: (d3 : + : d4) ≡ (d1 :=: d3) : + : (d2 :=: d4), dur d1 = dur d3

Figure 2: Equality Axioms of Dance

Each constructor in Pose corresponds to one action type. RobotPose is a list oftuples binding Pose with a body part and is a suitable abstraction of a robot’spose.

Then we must map each robot action into a pose, as follows:

mapAct :: Action -> PosemapAct (Move Forward Mid) = M (270, 0)mapAct (Move Left Mid) = M (0, 0)

...

The pose we get from mapAct is just the target pose. To get the pose of a robotat some given point of time in a performance, we must know how does the robotget there, i.e. the trajectory of that action. We assume our ideal trajectory is astraight line that goes from previous position to the final position of an actionand the angle speed is a constant. We then define:

pose :: (Parts b) => RobotPose b -> Performance b-> Time -> RobotPose b

pose rpi [] t = rpipose rpi ((t’,b,a,d):plist) t =

if (t’+d) < tthen pose (updateRP rpi (b, a)) telse let pb = getPose rpi b

pn = mapAct ain (b, poseAct pn pb (t-t’) d):

pose rpi plist t

Function pose rpi plist t does two things. One is keeping track of the latestcompleted actions of the robot which provides the initial positions of activeactions. The other is generating the pose at time t. poseAct is the actualfunction which computes the pose of an action at a given point of time based onthe trajectory assumption we made. getPose rpi b returns the pose of part b,and updateRP rpi rp updates part of a robot pose with rp.

15

As a language for reactive hybrid systems, Yampa is well suited to expressthe mapping from the abstract motions of Dance onto motions in a real robot.In fact, we can think of Dance as a simplified signal function. For normal signalfunctions, the exact value of the output signal is given at any point of time.While in Dance, it gives a sequence of pose information at a set of key points.Now with the help of functions meaning and pose, we can fill the gap betweenthose key points and give a smooth simulation of a dance using signal functions,as follows.

simulate :: (Parts b) => Dance b -> Time ->RobotPose b -> SF a (RobotPose b)

simulate d t rp = let plist = meaning d tin time >>> arr (pose rp plist)

Finally, with the help of simulate, the interpreter of reactive Dance can beconstructed quite easily.

simulateR :: (Parts b) => RobotPose b-> RDance a b -> SF a (RobotPose b)

simulateR rp sf =let loop init =

switch ((init &&& sf)>>>(arr (\(d, e) -> (d, attach e d)))

(\(d, rp) -> loop (simulate d 0 rp))in loop (constant rp)

simulateR takes arguments of a initial pose rp and a reactive Dance sf. Itstarts at the initial pose. Whenever sf generates a Dance event of Dance d, itpasses d along with the pose to simulate to start the new dance and waits forthe next event at the same time.

5.3 A Graphics Simulator

The functions in the previous section simulate a dance using signal functions.A 3D graphics simulator, however, is more direct and is helpful in visualizinga dance. We assume an idealized robot that has no restriction on acceleration,speed, or range of motion, and we choose HOpenGL as our 3D graphics library.

Generally speaking, writing a graphics simulator requires knowledge of therobot and experience with a graphics library. It can be tedious and error prone,especially when the robot’s design changes a lot. Our approach is to specify therobot design abstractly, relying on type classes in particular, in such a way thatthe robot details necessary for proper rendering can be extracted automaticallyfrom the robot description.

In fact, type class Parts described in Section 3.1 captures a sufficient specifi-cation of robots needed for 3D animation. To show how the Parts methods areused, we outline several functions for generating code fragments of the simulatorbelow:

16

Figure 3: Graphics Simulator

• An image of some body part can be obtained via the following function:

drawPart :: (Parts b) => Handle -> b -> IO()drawPart h b =

let (x, y, z) = size b indo hPutStr h "pushMatrix\n"

hPutStr h ("scale ("++(show x) ++" :: GLfloat) " ++ (show y) ++" " ++ (show z) ++ "\n")

hPutStr h "solidSphere 0.5 10 8\n"hPutStr h "popMatrix\n"

Function drawPart generates code pieces. when inserted into the properposition in a complete HOpenGL program, it will draw a 3D image of thecorresponding body part.

• Using the joint information, we can translate the center of the image ofthe body part into the proper point using function genTrans:

genTrans h Nothing p1 = return ()genTrans h (Just p) p1 =

let (Just (a, b)) = jointAt p p1

17

indo hPutStr h ("translate (" ++ toVector a’

++ " :: Vector3 GLfloat)\n")genRotate h p1 "v1"hPutStr h ("translate (" ++ toVector b’

++ " :: Vector3 GLfloat)\n")

toVector generates a string corresponding to the vector value of the input.

• Function genRotate generates the proper code to reproduce the pose ofa body part according to the simulation. The output of the simulationfunction is passed to animation through IO references.

genRotate ::(Parts b) => Handle -> b -> IO()genRotate h b =

do hPutStr h ("(a, b) <- readIORef (getPart "++ (show b) ++ ")\n")

hPutStr h "rotate (a) (Vector3 0 1 0)\n"hPutStr h "rotate (b) (Vector3 1 0 0)\n"

• Using the above functions, drawBody generates HOpenGL code which willreproduce the 3D image of a given robot:

drawBody h p [] = return ()drawBody h p (p1:ps) =

let cs = childPart p1 indo hPutStr h "pushMatrix\n"

genTrans h p p1drawPart h p1drawBody h (Just p1) cshPutStr h "popMatrix\n\n"drawBody h p ps

With these and several other functions, a 3D animation program is auto-matically generated, thus allowing us to adjust to new robot designs quickly,or to simulate many different robots simultaneously. At the same time, theanimation is controlled using the output of the simulation function defined inprevious section, which improves the modularity of our program. Figure 3 isthe snapshots of the simulation of dance sample.

6 Related Work

Considerable effort has been made in developing robot programming languages.Some are focused primarily on providing high-level control. For example, Mis-sionLab is a visual programming tool set that implements the “Societal Agent”

18

theory [16, 15]. It makes the robot programming considerably easier for noviceprogrammers but relies on a predefined set of very high level primitive actionsthat cannot be easily extended. GOLOG [14] is a high-level robot programminglanguage based on the situation calculus. Programs may be written at a muchhigher level of abstraction than is usually possible. It requires a goal along witha set of constraints, then a theorem prover is used to find a solution in thisrestricted search space. Unfortunately the theorem prover is costly during run-time. Schrott [18] introduces a task level programming environment, in whichhe uses a knowledge base to analyze and transform a task specification into anaction sequence. These languages provide a high-level goal-based way of pro-gramming. However, it is also goal-bound, programs written for some specifictasks are often difficult to extend in different tasks. A Dance program canbe seen as a detailed specification of some behavior. By dividing a behaviorinto multiple steps, we simplify the task of motion planning and increase theextensibility of the language.

Some researchers have focused on low level controls. ARCL [2] is a portableC library of robot control routines that supports sensor-based motion control.Colbert is a C-like language for low-level robot control tied to the Sapphiraarchitecture and uses a semantics based on finite state automata[13]. CES de-veloped by Thrun [20, 19] is a special-purpose language extension of C++ forrobots, integrating probabilistic computation and learning into the language.Horswill’s GRL [5] is an architecturally neutral robot language embedded inScheme. It implements behavior-based robot architectures as reusable softwareabstractions. While similar to our language in the use of functional abstraction,it does not embody a “language of motion,” but rather focuses on interconnect-ing controllers. ROBCL [1] is a distributed language for programming severalactors and sensors in a heterogeneous system architecture. These languages pro-vide flexible and efficient robot controllers, but do not feature the abstractioncapabilities in Dance.

Dance is somewhere in between these two classes of language. It allows thedefinition of new behaviors and separates the motor control and other low-levelaspects of the system from its high-level specification. It is also amenable toformal reasoning based on a formal algebraic semantics.

7 Conclusions

Robot programming is a field that provides an interesting and challenging basisfor programming language research. With the development of humanoid robots,the high degree of freedom makes it even more of a challenge. The applicationarea of humanoid robots is different from traditional robots at which it focusesmore on acting like a human in environments that are fit for humans. Mostrobot programming languages are not tailored for humanoid robots. Inspiredby Labanotation, we design a language called Dance for humanoid robots. Itprovides a high-level abstract motion description capability. It is hardwareindependent, expressive, and reactive. We also define a semantics for Dance,

19

which make it amenable to formal reasoning.There are other issues not addressed here but that are worth further re-

search. First, Dance is working with a simulator now, but our final target is tocontrol the real robots designed and constructed in the Yale Vision and RoboticsLaboratory. Secondly, at the present stage, Yampa is used to provide reactivity.However Yampa is for a much more general framework, so we would like to in-vestigate the possibility of a more specific syntax for introducing reactivity intoDance. Lastly, we plan to develop techniques to map normal human movement– acquired through a motion capture system – into Dance. In other words,instead of generating robot movements from Dance, we also wish to use Danceto capture existing movements.

References

[1] B. Baginski, A. Baumann, and S. Riesner. Robcl - an distributed object-oriented robot programming language. In Proceedings of the 8th Inter-national Workshop on Robotics in Alpe-Adria-Danube Region (RAAD-99),pages 229–234, 1999.

[2] P. Corke and R. Kirkham. The arcl robot programming system. In Robotsfor Competitive Industries, pages 484–493, Brisbane, July 1993. MechanicalEngineering Publications.

[3] C. Elliott. Modeling interactive 3D and multimedia animation with anembedded language. In Proceedings of the first conference on Domain-Specific Languages, pages 285–296. USENIX, Oct. 1997.

[4] C. Elliott and P. Hudak. Functional reactive animation. In InternationalConference on Functional Programming, pages 263–273, June 1997.

[5] I. Horswill. Functional programming of behavior-based systems. Au-tonomous Robots, 9(1):83–93, 2000.

[6] P. Hudak. Haskore music tutorial. In Second International School on Ad-vanced Functional Programming, pages 38–68. Springer Verlag, LNCS 1129,Aug. 1996.

[7] P. Hudak. Modular domain specific languages and tools. In Proceedingsof Fifth International Conference on Software Reuse, pages 134–142. IEEEComputer Society, June 1998.

[8] P. Hudak. The Haskell School of Expression – Learning Functional Pro-gramming through Multimedia. Cambridge University Press, New York,2000.

[9] P. Hudak, A. Courtney, H. Nilsson, and J. Peterson. Arrows, robots, andfunctional reactive programming. In Summer School on Advanced Func-tional Programming, Oxford University. Springer Verlag, to appear, 2003.

20

[10] P. Hudak, T. Makucevich, S. Gadde, and B. Whong. Haskore music nota-tion – an algebra of music. Journal of Functional Programming, 6(3):465–483, May 1996.

[11] P. Hudak, S. Peyton Jones, and P. Wadler (editors). Report on the Pro-gramming Language Haskell, A Non-strict Purely Functional Language(Version 1.2). ACM SIGPLAN Notices, 27(5), May 1992.

[12] A. Hutchinson. Labanotation. Routledge Theatre Arts Books, New York,1991.

[13] K. Konolige. COLBERT: A language for reactive control in sapphira. InKI - Kunstliche Intelligenz, pages 31–52, 1997.

[14] H. J. Levesque, R. Reiter, Y. Lesperance, F. Lin, and R. B. Scherl. GOLOG:A logic programming language for dynamic domains. Journal of LogicProgramming, 31(1-3):59–83, 1997.

[15] D. MacKenzie and R. Arkin. Evaluating the usability of robot programmingtoolsets. The International Journal of Robotics Research, 17(4):381–401,April 1998.

[16] D. MacKenzie, R. Arkin, and J. Cameron. Multiagent mission specificationand execution. Autonomous Robots, 4(1):29–52, 1997.

[17] H. Nilsson, A. Courtney, and J. Peterson. Functional reactive programming,continued. In Proceedings of the 2002 ACM SIGPLAN Haskell Workshop(Haskell’02), pages 51–64, Pittsburgh, Pennsylvania, USA, Oct. 2002. ACMPress.

[18] G. Schrott. An experimental environment for tasklevel programming. InProceedings of Second Int. Symposium on Experimental Robotics (ISER),pages 196–206, Toulouse, France, June 1991. Springer.

[19] S. Thrun. A framework for programming embedded systems: Initial designand results. Technical Report CMU-CS-98-142, Carnegie Mellon University,Pittsburgh, PA, 1998.

[20] S. Thrun. A programming language extension for probabilistic robot pro-gramming. In Workshop notes of the IJCAI Workshop on Uncertainty inRobotics (RUR), Seattle, WA, 2000. IJCAI, Inc.

[21] Z. Wan and P. Hudak. Functional reactive programming from firstprinciples. In Proceedings of PLDI’01: Symposium on Program-ming Language Design and Implementation, pages 242–252, June 2000.http://haskell.org/frp/publication.html#frp-1st

21