The Automated Marionette Project

Elliot JohnsonTodd Murphey

University of Colorado at Boulder

Puppet choreography is a highly-developed language for controlling mechanically complex mar-ionettes. It has evolved over centuries into a largely standardized form that allows puppeteers toaddress issues that arise as a result of the complex systems with which they are working. As such,the standardization of the choreography can be thought of as the puppeteers’ response to com-plexity. The research, in collaboration with Magnus Egerstedt and Patrick Martin at the GeorgiaInstitute of Technology, focuses on understanding how puppeteers address complex tasks in theirchoreographic descriptions of plays and using that understanding to solve questions of importanceto computer science and engineering. These goals will be achieved by creating an automated pup-pet play, which will use insights about puppet choreography to implement embedded control ofmechanically complex marionettes engaged in complex coordination tasks.

(a) (b)

Figure 1: (a) Simulation of a string marionette and (b) a marionette mechanical systemand its associated graph representation. For a specific example of a string marionette, seehttp://puppeteer.colorado.edu.

In order to automate a puppet play, there are three key technical hurdles that must be ad-dressed: real-time embedded motion control; strategically handling the complexity associated withcoordination of high degree-of-freedom systems; and computer animation and simulation. Thesethree areas are of critical interest to the computer science community, entirely aside from theirapplication to puppetry. In most current engineering applications real-time embedded control ofphysical systems only addresses comparatively simple systems, such as single degree-of-freedommechanisms. Puppets often have more than forty degrees-of-freedom (as seen in Fig. 1, where thealgebraic graph on the right is used to represent the puppet on the left), indicating that the asso-ciated embedded control must take their complexity into account. To facilitate understanding howpuppet choreography addresses this problem, the puppetry artists will teach the researchers how tomanipulate puppets and use choreography. Other additional complexity issues must be resolved inorder to successfully put on a play with multiple puppets. These include communication betweenthe embedded controllers, dynamic resource allocation, and synchronizing motion. The proposed

1

(a) (b) (c)

Figure 2: (a) A Mathematica graphical representation of the mechanical system we plan to even-tually build; (b) a first-generation 3D experimental puppet platform at the Georgia Institute ofTechnology that cannot spatially translate in the environment; (c) a first-generation 2D experimen-tal puppet platform at the University of Colorado that can spatially translate in its environment.

automated puppets will use a programming language that directly mimics the constructions usedin puppet choreography. The puppeteers will additionally provide feedback on the implementation,indicating whether the implementation is not simple enough, is too simple, is abstracted improperly,or needs to be corrected in other ways. The goal will be to formalize how puppeteers strategicallydeal with complexity, task description, deadlock, problem specification, etcetera, in the contextof puppetry. Lastly, the ability to simulate and visualize complex mechanical puppets is essentialfor testing and validating a potential choreographic specification of a play. However, current fastanimation techniques use heuristics to resolve motions involving mechanical contact and are notamenable to control design and synthesis, so new animation techniques will be developed that areamenable to these tools, and will lead to tools that the puppeteers themselves will be able to use inthe design of plays. Hence, the project’s goal is to use puppetry and puppet choreography as a sourceof insight into pressing computer science and engineering problems. Experimental implementationssuch as those seen in Fig. 2 will be used to validate the techniques as well as put on a play at theAtlanta Center for Puppetry Arts.

Acknowledgments

We gratefully acknowledge the National Science Foundation and the Creative IT program for theirsupport of this project under award IIS-0757378.

2