Do Digital Agents Do Dada?Paper type: CC Bridges

Gunter LoeselZurich University of the Arts

Zurich, Switzerlandgunter.loesel@zhdk.ch

Piotr Mirowski and Kory W. MathewsonImprobotics

London, UK and Montreal, QC, Canadacontact@improbotics.org

Abstract

Do digital agents do Dadaism? To answer this question,we review a series of theatrical experiments involvinghuman improvisers and AI-controlled Cyborgs in frontof audiences. We describe these experiments and dis-cuss the use of conversational digital agents (DA) onthe stage. We identify two basic strategies of stagingmachines: the “immersive approach”, and the “Dadais-tic approach”. We draw on Dadaistic conceptions ofembracing modernity specifically in the Dadaists’ ob-session with androids and cyborgs. Through analysisof several stage experiments we contend that digitalagents, while attempting to build believable characters,do Dada, only if we do Dada too.

Motivation: Improvisation and Digital AgentsImprovised theatre, impro or improv, is a diverse art formmodelled on natural human interaction and relies on spon-taneity. Some improv connects scenes into a consistent nar-rative and aiming to reproduce the experience of a scriptedplay. Doing so demands from performers constant adap-tation to an evolving context. Researchers in computa-tional performance have established parallels between im-prov theatre and jazz music and formulated both as “real-time dynamical problem solving” (Bruce et al. 2000;Johnson-Laird 2002; Magerko et al. 2009).

The first problem improvisers try to solve is collectivelycreating stories by impersonating believable characters andincorporating narrative elements suggested by the audiencearising over the course of the performance. The secondproblem is how to be “truthful” and “in the moment” of thescene, while accepting the offers made by the other impro-visers, the audience, or their own cultural background (John-stone 1979; Merritt and Hines 2019). A third problem isfinding the limits of an audience’s expectations.

Logic in Improv and the Circle of ExpectationWhen improvising scenes, improvisers follow the rules oflogic while establishing a specific “universe” in which theimprovised scene takes place. In this way, the performersand the audience can follow the story, and they can predicthow the scene continues (Merritt and Hines 2019). Practi-tioners have introduced the notion of the Circle of Expec-tation as a concept to qualify the difference between adding

Figure 1: TL: Improv with a robotic digital agent controlled byan AI chatbot. TR: Improv where a human actor reads lines fromthe AI chatbot displayed on a screen. BL: Psychotherapist chatbotELIZA appearing as a projection. BR: Improv (downstage) withone actor receiving lines from an AI chatbot controlled by a com-puter operator (upstage). Credits at improbotics.org

obvious narrative elements that make the story more specificversus those that violate the expectations of the audienceand each other (Johnstone 2014). The circle contains theassumptions and associations that define the dramatic world(Mathewson et al. 2020). Improvisors make offers by takinginto account the “obvious” next-steps from the mind of theaudience. In practice, improvisers follow the rules of logicfirst by establishing the platform (e.g. who, what, where,when) and the relationship between the characters, and thenby “doing the most obvious thing” to move the story for-ward (Johnstone 1979). This is similar to the circumstancesformalized by Konstantin Stanislavski (Benedetti 1999).

Digital Agents in Improvisational Theatre

Adhering to logic in improv while following the Circle ofExpectation with digital agents based on randomness seemsan impossible task. Nevertheless, the field has adoptedtechnological trends and offerings. Recent work buildsupon computational improvisation in music and dance per-formance where humans interact and co-create with artifi-cial systems (Fiebrink 2011; Hoffman and Weinberg 2011;

Long et al. 2020), and collaborative storytelling (Per-lin and Goldberg 1996; Hayes-Roth and Van Gent 1996;Riedl and Stern 2006; Magerko and others 2011).

There have been attempts at generating high-level nar-rative structures for improvised theatre. Digital agents(DA) have acted as directing narrators grounding theperformances of human improvisers with generated plotpoints (Eger and Mathewson 2018). Other DAs have quan-tified and shaped the narrative arc of generated text in in-teractive human-machine dialogue, in order to reveal orconceal information according to the Circle of Expecta-tion (Mathewson et al. 2020). DAs have been used else-where in improv theatre (O’Neill and others 2011; Baumerand Magerko 2010; Jacob 2019) and placed directly on thestage, such as in (Bruce et al. 2000), Heather Knight’s stand-up robot (Knight and others 2011), and Arthur Simone’s BotParty: Improv Comedy with Robots.

Progress in machine learning for natural language pro-cessing, specifically neural networks for text genera-tion (Vinyals and Le 2015; Radford et al. 2019), encour-aged practitioners to build DAs for improv by focusing onthe conversational and storytelling aspects. Kory Mathew-son and Piotr Mirowski innovated in this space with theircollaborative human-robot improv group HumanMachine.1They developed A.L.Ex., an advanced conversational chatbottrained on film dialogue from OpenSubtitles2 (Tiedemann2009) which interfaced with speech recognition, synthetictext-to-speech, and controlled a humanoid robot stage part-ner (Mathewson and Mirowski 2017a; 2017b).

Advancing Digital Agents for the StageThe current study focuses on extensions of AI improv prac-tice, where the robot is replaced by a human, who performslines provided by an AI chatbot. Examples of such showsinclude Improbotics3 by Mirowski and Mathewson, Yes, An-droid4 by Etan Muskat and Almost Human5 by Gunter Loe-sel. In Improbotics, digital agent A.L.Ex. is adapted tosend lines to control a human performer Cyborg who wearsheadphones and receives lines of dialogue generated by theAI. While the Cyborgs are free to move and to expressnon-verbal acting and emotional subtext, they can only saycomputer-generated lines. Those lines of dialogue are gener-ated in response to context typed by a human Operator whocan also serve as curator in the case when multiple choicesare offered. Others are free to perform as they wish.

By replacing the robot avatar by a human improviser get-ting lines from the chatbot and reintroducing the humaninto the AI improv loop, opportunities for theatrical exper-imentations arise. The AI plays the role of an interactiveplaywright, giving lines to a specific performer, while chal-lenging the other improvisers to justify what the AI pro-duced. Live experiments investigate what kinds of the-atrical frames emerge through the interaction of humans

1https://humanmachine.live

2https://www.opensubtitles.org/

3https://improbotics.org/

4https://baddogtheatre.com/yes-android

5https://www.stupidlovers.de/de/show-termin/almost-human

and machines on the stage, how one can describe and ex-plain these theatrical phenomena, and what value thesepartly machine-generated dialogues can have in an artis-tic and aesthetic sense (Martin, Harrison, and Riedl 2016;Mathewson and Mirowski 2017b).

Similar to previous work using artistic lenses to contextu-alize novel AI technology (Horswill 2012; 2016), we ana-lyze digital agent improv through the lens of Dadaism. Dadais a a suitable artistic frame to understand human-machineco-creativity improvisation.

Avant-garde Movements and DadaismThe avant-garde movements at the beginning of the 20th cen-tury were the first artistic response to modernity and theindustrial age. As the art historian Matthew Biro pointsout, the Dada movement was quite obsessed with the figureof the cyborg (Biro 2009)—though the word “cyborg” didnot exist at that time. Dada artists, especially in the BerlinDada group, explored and involved machines and mechani-cal products in various ways to express their ambiguous re-lationship towards modernity. They were furious activistsagainst the political and military “machinery” that led to thefirst world war. Hausmann’s drawing “Der eiserne Hinden-burg” (1920) displays the German general Hindenburg ashalf-human, half-machine indicating the de-humanizing im-pact of technological war and chauvinistic patriotism.

Not only did Dada artists point towards the politicalaspects of automatization, they also reflected on the hu-man self turning more machine-like. In two famous self-portraits, George Grosz’s “Daum marries her pedantic au-tomaton George” (1920) and Hausmann’s “Selfportrait ofthe Dadasoph” (1920), the Dadaists displayed themselves ashalf-machine and half-human. In opposition to the famousnarratives of encounters between humans and robots, theseworks brought a new quality. The humans had no outsideperspective to look at the machine as a human. They arealready changed and turned into cyborgs and are unable todistance themselves from the machine.

This notion reflection is explored in (Hausmann 1921):“Why can’t we paint pictures today like those of Botticelli,Micheangelo, Leonardo, or Titian? Because human beingshave completely changed in terms of their consciousness.This is the case not simply because we have the telephone,the airplane, the electric piano, and the escalator, but ratherbecause these experiences have transformed our entire psy-chophysical condition.”

The naıve perspective of the human as a counterpart oftechnology is being questioned even at this early stage ofreflecting on modernity. Consequently, the Dadaists pro-posed an art not made by humans. Grosz stated this clearlyat the First International Dada Fair in 1920, when he de-clared: “Art is dead. Long live the new machine art ofTatlin” (Broeckmann 2016).

Dadaism has a couple of identifiable traits. One of par-ticular interest is that the artists deconstructed meaning, ar-riving at the smallest units of expression like letters andphonemes, resulting in poems that consist of single lettersonly (Hausmann, “fmsbw”, 1918) or in optophonetic poetry(Hausmann, 1918). The ideal unit for the Dadaist does not

bear any meaning, it is under the surface of meaning. Thisresults in the Dadaistic practice of reducing the world to itsbasic units. Then putting those pieces together again in ameaningless way, thereby uncovering the meaninglessnessunder the smooth surface of sense-making. Optophoneticpoetry is an early example of the ambiguity of “cyborg art”.While subverting the meaning of language, it was performedin a expressive way, as can be experienced through record-ings with Hausmann reading his own poetry.6

Dadaists did not just embrace modernity and the machineage. They squeezed it so hard, and twisted it so maliciously,that modernity stopped being something you could look atfrom the outside. It became a feature internalized by modernhuman beings.

Artistic Strategies for Cyborg ImprovIn appearance, trying to build a digital agent that tries toact exactly like a human seems somehow superfluous in anartistic sense. Even if it were possible, which at the momentit is not, such an endeavour would degenerate to mimickinghuman-to-human interaction. Quoting (Turing 1951): “But,I certainly hope and believe that no great efforts will be putinto making machines with the most distinctively human,but non-intellectual characteristics such as the shape of thehuman body; it appears to me to be quite futile to make suchattempts and their results would have something like the un-pleasant quality of artificial flowers.”

Digital agents might however be employed to create newtypes of stage interaction. The benefits owing more to theDA’s deficits in social and psychological abilities than totheir high-fidelity imitation. We now analyze digital agentsin the context of such interactions.

Subverting the Turing Test for DeceptionOne component of AI improv shows consists of performinga few scenes while hiding the identity of the actor(s) who gettheir lines from an AI chatbot. All improvisers wear head-phones, whether they hear the AI or not. The performers areasked to improvise a poem based on a word suggested bythe audience, one line at a time, and after a few verses, theaudience is asked to guess the identity of the Cyborg. Thisgame is based on the Turing Test or Imitation Game (Turing1950), where the machine needs to convince an audience ofhuman judges that it is human. This dynamic is explored inother games (Treanor et al. 2015) including Spy Party.7

In order to understand and to quantify the performanceof humans and AI models alike, (Mathewson and Mirowski2018) collected feedback of a large number of human per-formers who interacted with AI co-stars. The human per-formers highlighted the limitations of the digital agents;agents generated sentences inconsistent with respect tologic, social conventions and emotions. Interestingly, theselimitations were welcomed by the performers as they felt theAI acted like an “X factor” and forced humans to becomebetter improvisers.

6https://www.youtube.com/watch?v=2lVqiCURmFQ

7http://spyparty.com

IMMERSIVE DADAISTICHiding technology Displaying technology

Celebrating humanity Celebrating the CyborgMimic human behaviour Expose machine behaviour

Within Circle of Expectation Break Circle of ExpectationTry to pass Turing Test Inspire the Cyborg in us

Table 1: Approaches for staging Digital Agents in improv.

Anecdotal evidence8 suggests that, while performing theTuring Test for the audience in the context of AI improv, theactors naturally introduced a slight modification to the prin-ciples of the game. Not only was the robot “trying” to passas human, the humans tried to pass as robots, meaning thatthe non-Cyborg performers tried to behave like the Cyborgimprovisers, aping the sometimes nonsensical nature of AI-generated text. This artistic choice was made by the humanperformers to deceive the audience into thinking that theyactually were the Cyborg. The audience was fooled by thebehaviour of the human actors, to great comedic success.

Based on both the feedback from the actors and on the ob-servation of how they acted, the human actors gave the im-pression of not being “in their heads” while coming up withAI-sounding nonsense, but rather that this nonsense was pro-duced naturally. By trying to be robotic, the actors behavedhumanly upholding the audience connection.

Cyborgian DadaismContrasting the immersive setup inspired by the TuringTest described above, Almost Human (Zurich and Germany,2018) used an alternative approach. In Almost Human, themachine does not pretend to be a human. The technology isprovocatively not hidden. The show focuses on questioningthe “machineliness” of the human.

This is quite a change of perspective. It is opposed to themore mainstream developments of “immersive” theatre. Inimmersive theatre, creators aim to immerse the performersand spectator in the fiction, making them forget the technol-ogy being used. Thus, there are two antagonistic strategiesof dealing with modernity which we called the “immersiveapproach” and the “Dadaistic approach”.

Almost Human staged two chatbots in leading roles in atheatre piece—to our knowledge this is the first time this wasdone in a feature-length theatre performance. One chatbotwas embodied by a human actor who got their lines froman in-ear-monitoring system (similar to Improbotics), andthe lines came from JANN, the Just Approximate NearestNeighbor chatbot.9

For the show, JANN was trained on a large corpus of pop-song lines in order to make it respond directly and emotion-ally. The second chatbot was a modern version of ELIZA,the chatbot therapist (Weizenbaum 1966). ELIZA was “em-bodied” through a real-time avatar, projected on the screen(see Fig. 1). The audience was informed which lines camefrom a machine, so there was no confusion about who washuman and who was machine. The show’s creative team

8Recording of an AI improv show: https://youtu.be/bMSigawTuJs

9https://github.com/korymath/jann

made it explicit that the chatbots were seen as co-creators ofthe theatre performance.

Often times the digital agents added to the story, othertimes they acted unpredictably and added absurd and illog-ical material to the scene. By letting the audience know theresponses were created by an algorithm, they were “invitedin” on the conceit of the show. They knew that the hu-mans did not have complete control; the scene could movein unforeseen directions due to the contributions of JANNor ELIZA. At the same time the actors reacted emotional toall lines from the machine, providing a sense of urgency anddrama. The audience could watch a play that was generatedpartly by a machine and partly by humans, making it “cy-borg art”. The intention was to leave the audience with someirritation about who was the author of this performance, if itwas a machine or a human—or if there was an author at all.

Discussion and ConclusionA transfer of Dadaistic principles to today’s improvised per-formances is possible and consistent with the avant-gardes’strategies for dealing with modernity. The use of AI in arthas been heavily inspired by the Turing Test—training ma-chines in their ability to imitate humans. Drawing on theavant-garde gives us a second frame for appreciating artistichuman-machine collaboration. These theatrical experimentsuse two strategies. First, an immersive strategy that hidestechnology from the spectators and creates insecurity aboutthe humanness of the actors. Second, a Dadaistic strategythat displays technology and invites the audience to reflectabout their “machineliness” and “humanness”.

The Dadaist deconstructed meaning, they concernedthemselves with nonsense and irrationality. They focusedon breaking expectation because anything was possible. Theanti-logic and anti-reason mentality of Dada broke the Circleof Expectation wide open. Dadaism defined itself by break-ing those expectations and by shocking audience with seem-ingly disconnected associations. Audiences grew to expectthe shock and eventually the subversion fell out of favour.

In summary, we provide several conclusions supportingthe continued creation of AI improv using the frame ofDadaism. The Dadaistic strategy has a logic that integratesformulaic aspects (e.g. the use of probability). The im-mersive approach is over-represented, as large parts of thefilm and gaming industries continue to pursue this strategy.Much of the computational text generation underlying AIimprov involuntarily frees the language from semantics andsometimes even syntax. This quality aligns well with thefundamental underpinnings of the Dadaistic perspective.

We suggest to use the Dadaistic strategy as a measure ofthe “realism-absurdity degree” of an improvised scene. Theoptimal balance of absurdism and realism for artistic pur-poses remains an open question. Most previous work on AIimprov focused on the justification of what the AI wouldsay, viewing the machine-generated nonsense as a challengeto good improv. We encourage investigation of how the ab-surdity of AI-generated dialogue uplifts improv. We suspectthe “Circle of Expectation” will be crucial for further ex-plorations in digital agent co-creation. Finally, we concludethat DA do Dada, only if we do Dada too.

ReferencesBaumer, A., and Magerko, B. 2010. An analysis of narrative moves in improvisational theatre. InJoint International Conference on Interactive Digital Storytelling, 165–175. Springer.

Benedetti, J. 1999. Stanislavski: his life and art. Methuen Drama.

Biro, M. 2009. The Dada cyborg: visions of the new human in Weimar Berlin. U of MinnesotaPress.

Broeckmann, A. 2016. Machine art in the twentieth century. MIT Press.

Bruce, A.; Knight, J.; Listopad, S.; Magerko, B.; and Nourbakhsh, I. R. 2000. Robot improv: Usingdrama to create believable agents. In IEEE International Conference on Robotics and Automation.

Eger, M., and Mathewson, K. W. 2018. dairector: Automatic story beat generation through knowl-edge synthesis. arXiv preprint arXiv:1811.03423.

Fiebrink, R. A. 2011. Real-time human interaction with supervised learning algorithms for musiccomposition and performance. Princeton University.

Hausmann, R. 1921. Presentismus. gegen den puffkeismus der teutschen seele. In Hausmann TexteBis 1933, II, edited by Michael Erlhoff, Edition , 1982. Munchen.

Hayes-Roth, B., and Van Gent, R. 1996. Improvisational puppets, actors, and avatars. In Proc CompGame Dev Conf.

Hoffman, G., and Weinberg, G. 2011. Interactive improvisation with a robotic marimba player. InMusical robots and interactive multimodal systems. Springer. 233–251.

Horswill, I. D. 2012. Punch and judy ai playset: A generative farce manifesto, or, the tragicalcomedy or comical tragedy of predicate calculus. In Eighth Artificial Intelligence and InteractiveDigital Entertainment Conference.

Horswill, I. D. 2016. Dear leader’s happy story time: A party game based on automated storygeneration. In Twelfth Artificial Intelligence and Interactive Digital Entertainment Conference.

Jacob, M. 2019. Improvisational Artificial Intelligence for Embodied Co-creativity. Ph.D. Disserta-tion, Georgia Institute of Technology.

Johnson-Laird, P. N. 2002. How jazz musicians improvise. Music Perception 19(3).

Johnstone, K. 1979. Impro: Improvisation and the Theatre. Faber and Faber Ltd.

Johnstone, K. 2014. Impro for storytellers. Routledge.

Knight, H., et al. 2011. A savvy robot standup comic: Online learning through audience tracking.In Conf. Tangible & Emb. Int.

Long, D.; Liu, L.; Gujrania, S.; Naomi, C.; and Magerko, B. 2020. Visualizing improvisation inluminai, an ai partner for co-creative dance. In Proceedings of the 7th International Conference onMovement and Computing, 1–2.

Magerko, B., et al. 2011. Employing fuzzy concept for digital improvisational theatre. In AAAIAIIDE, 53–60.

Magerko, B.; Manzoul, W.; Riedl, M.; Baumer, A.; Fuller, D.; Luther, K.; and Pearce, C. 2009. Anempirical study of cognition and theatrical improvisation. In ACM Conf. on Creat. & Cog.

Martin, L. J.; Harrison, B.; and Riedl, M. O. 2016. Improvisational computational storytelling inopen worlds. In International Conference on Interactive Digital Storytelling. Springer.

Mathewson, K. W., and Mirowski, P. 2017a. Improvised theatre alongside artificial intelligences. InAAAI AIIDE.

Mathewson, K. W., and Mirowski, P. 2017b. Improvised comedy as a turing test. arXiv:1711.08819.

Mathewson, K. W., and Mirowski, P. 2018. Improbotics: Exploring the imitation game usingmachine intelligence in improvised theatre. In AAAI AIIDE.

Mathewson, K. W.; Castro, P. S.; Cherry, C.; Foster, G.; and Bellemare, M. G. 2020. Shapingthe narrative arc: An information-theoretic approach to collaborative dialogue. In InternationalConference on Computational Creativity.

Merritt, B., and Hines, W. 2019. Pirate Robot Ninja: An Improv Fable. Pretty Good Publishing, Inc.

O’Neill, B., et al. 2011. A knowledge-based framework for the collaborative improvisation of sceneintroductions. In ICIDS, 85–96. Springer.

Perlin, K., and Goldberg, A. 1996. Improv: A system for scripting interactive actors in virtualworlds. In Conf on Comp. G. & Int. Tech. ACM.

Radford, A.; Wu, J.; Child, R.; Luan, D.; Amodei, D.; and Sutskever, I. 2019. Language models areunsupervised multitask learners. OpenAI Blog 1(8).

Riedl, M. O., and Stern, A. 2006. Believable agents and intelligent story adaptation for interactivestorytelling. In Intl Conf on Tech for Int Dig St and Ent, 1–12. Springer.

Tiedemann, J. 2009. News from opus-a collection of multilingual parallel corpora with tools andinterfaces. In Recent Advances in Natural Language Processing, volume 5.

Treanor, M.; Zook, A.; Eladhari, M. P.; Togelius, J.; Smith, G.; Cook, M.; Thompson, T.; Magerko,B.; Levine, J.; and Smith, A. 2015. Ai-based game design patterns. In Proceedings of the 10thInternational Conference on the Foundations of Digital Games 2015. Society for the Advancementof Digital Games.

Turing, A. M. 1950. Computing machinery and intelligence. Mind 59(236).

Turing, A. M. 1951. Can digital computers think? The Turing Test: Verbal Behavior as the Hallmarkof Intelligence.

Vinyals, O., and Le, Q. 2015. A neural conversational model. arXiv preprint arXiv:1506.05869.

Weizenbaum, J. 1966. Eliza—a computer program for the study of natural language communicationbetween man and machine. Communications of the ACM 9(1):36–45.