kuileixi: a chinese open-ended text adventure game
TRANSCRIPT
Proceedings of the Joint Conference of the 59th Annual Meeting of the Association for Computational Linguistics and the 11thInternational Joint Conference on Natural Language Processing: System Demonstrations, pages 175–184, August 1st - August 6th, 2021.
©2021 Association for Computational Linguistics
175
KuiLeiXi: a Chinese Open-Ended Text Adventure GameYadong Xi1∗, Xiaoxi Mao1†∗, Le Li1, Lei Lin1, Yanjiang Chen1,
Shuhan Yang1, Xuhan Chen1, Kailun Tao1, Zhi Li1, Gongzheng Li1Lin Jiang1, Siyan Liu1, Zeng Zhao1,Minlie Huang2, Changjie Fan1, Zhipeng Hu1
1 Fuxi AI Lab, NetEase Inc., Hangzhou, China2 Department of Computer Science and Technology, Institute for Artifical Intelligence, State Key
Lab of Intelligent Technology and Systems, Beijing National Research Center forInformation Science and Technology, Tsinghua University, Beijing, China.{xiyadong, maoxiaoxi, lile, lin lei}@corp.netease.com,
Abstract
There is a long history of research related toautomated story generation, dating back as faras the 1970s. Recently, the rapid developmentof pre-trained language models has spurredgreat progresses in this field. Equipped withGPT-2 and the latest GPT-3, AI Dungeon hasbeen seen as a famous example of the power-ful text generation capabilities of large-scalepre-trained language models, and a possibil-ity for future games. However, as a game,AI Dungeon lacks incentives to players andrelies entirely on players to explore on theirown. This makes players’ enthusiasm declinerapidly. In this paper, we present an open-ended text adventure game in Chinese, namedas KuiLeiXi1. In KuiLeiXi, players need tointeract with the AI until the pre-determinedplot goals are reached. By introducing theplot goals, players have a stronger incentiveto explore ways to reach plot goals, whilethe AI’s abilities are not abused to generateharmful contents. This limited freedom allowsthis game to be integrated as a part of a ro-mance simulation mobile game, Yu Jian Love2.Since KuiLeiXi was launched, it has receiveda lot of positive feedbacks from more than100,000 players. A demo video is available athttps://youtu.be/DyYZhxMRrkk.
1 Introduction
The past few years have seen a significant improve-ment in the capabilities of neural networks for textgeneration(Radford et al., 2019; Brown et al., 2020;Zhang et al., 2020b). Large-scale pre-trained lan-guage models with tens of billions of parametersare capable of producing human-like text. Thiscapability has spawned a range of revolutionary ap-plications(Roller et al., 2020; Zhang et al., 2020a;
∗ Equal contribution† Corresponding Author
1http://yujian.163.com/klx-silk/redirect.html2https://yujian.163.com
Guan et al., 2020). AI Dungeon is a typical ex-ample of them. It is an open-ended text adventuregame, where players are allowed to create theirown adventures in any way they like. The originalAI Dungeon is based on GPT-2 large, fintuned on adataset of text adventures collected online3. Sincethe launch of its Colab version, AI Dungeon hasgained a lot of attention on social networks.
However, from the point of view of game devel-opers, AI Dungeon suffers from several problems,hindering it from becoming mainstream gaming.The first problem is that it relies entirely on playersto explore on their own. The lack of incentivesmay lead to a rapid decline of players’ enthusiasm.The second problem is the boundaryless nature ofgenerated contents. Every game is associated witha certain series of world settings where the storiestake place. To integrate AI Dungeon-like technol-ogy in a game, considerable adaptation works arenecessary. On the other hand, in the absence ofnecessary guidance and restraints, players tend toabuse AI Dungeon to create malicious or offensivecontents4. In areas with more conservative values,it is of high risk to launch an AI Dungeon-likefeature in a commercial product.
Considering the problems described above, weextended the original AI Dungeon so that it couldbe accommodated in a commercial game. Whenplaying AI Dungeon, depending on the player’schoice of different topics, the AI will generate astory beginning, and then the player is free to ex-plore the development of the story. Unlike AI Dun-geon, in our game, players need to play a fixedcharacter of their choice and interact with the AIto develop the story according to the pre-definedstory background until they reach the specified plotgoal to obtain the mission rewards. Multiple plot
3https://chooseyourstory.com/4https://www.wired.com/story/ai-fueled-dungeon-game-
got-much-darker/
176
goals are contained in a story script. By elaboratelydesign the plot goals, the difficulty of game and thefreedom of players to create could be manipulated.The game supports multiplayer, scripts are createdboth by the game developers and by the playersthemselves. Because in the gaming process, theplayer seems to be maninulating the puppet of thecharacter, we figuratively call this game KuiLeiXi,which refers to the puppetry in the Song Dynasty.
Deploying a neural text generation model formany players is quite expensive. So we adopteda range of methods to reduce the cost, includinglayer drop and knowledge distillation. In addition,we implemented a highly optimized transformer inCUDA for inference. After applying these methods,the inference speed of the model is increased by10 times, the throughput is increased by 20 times,greatly reducing the deployment cost.
KuiLeiXi has been launched as a part of Yu JianLove. Yu Jian Love is a mobile romance simulationgame where players can role play as a girl that livesin the era of Northern Song Dynasty and develop ro-mantic relationship with different handsome malecharacters. Since launched, it received a lot of posi-tive feedbacks from players and industry. We hopeKuiLeiXi could inspire fellow game developers andNLP researchers to bring more NLP capabilitiesinto games and make game content more dynamicand personalized.
2 Architecture
In this section, we will describe the implementa-tion and optimization of KuiLeiXi in detail. Asseen in Figure 1, the system consists of three com-ponents: Input Processor, Story Generator andCandidates Ranker. As both the Story Genera-tor and Candidates Ranker are based on our in-house pre-trained language model, we will firstlydescribe the pre-training details. Then we willpresent the implementation details of the three com-ponents in order. Finally, we will introduce theoptimization details for deployment.
2.1 Pre-training
Our in-house pre-trained language model for storygeneration is based on GPT-2 large. It has 36 layers,1280 hidden size, 20 self-attention heads, and 725million parameters. It is pre-trained on a datasetconsisted of around 30 gigabytes of Chinese web-novels collected online. The vocabulary size is13762 and the context length is 1024. In addition,
we pre-trained a Roberta-large(Liu et al., 2019)based bidirectional transformer model(Vaswaniet al., 2017) on the same dataset. It has 24 layers,1024 hidden size, 16 self-attention heads and 317million parameters. We used fairseq5 for trainingof the models.
2.2 Input ProcessorThe input text of a player will firstly be checkedby a toxicity detection service6 to avoid potentialrisks. It is then processed by a semantic similar-ity detection model to determine if it is too se-mantically close to the plot goal. This is to avoidmaking it too easy for players to reach the plotgoal. The semantic similarity detection model isbased on Sentence-Bert(Reimers and Gurevych,2019), trained on the combination of several Chi-nese NLI datasets(Bowman et al., 2015; Williamset al., 2018; Hu et al., 2020). The virtual adversar-ial training(Zhu et al., 2020) is also adopted. Thisapproach improves the generalization of the modelby adding small perturbations to the input embed-dings. For every plot goal, at least three textualdescriptions of that goal should be prepared. Theinput text will be compared with all the textualdescriptions of current plot goal. If any of the simi-larity scores is above a certain threshold, the playerwill receive a message telling the player to inputagain. After the input text has passed the toxicitydetection and semantic similarity detection, it willbe concatenated to the context to form the input forstory generation.
2.3 Story GeneratorThe story generator is in charge of generating con-sistent and fluent story contents based on the con-text and player input. In below we will describe indetail how the story generator is implemented.
2.3.1 FinetuningBecause KuiLeiXi is supposed to be launched asa part of Yujian Love, the generated text needs tobe consistent with the original stories of the gamein terms of language style and backdrop. There-fore, the game’s existing story scripts are criticalfor finetuning. However, these scripts only con-tain approximately 2 million tokens, barely enoughfor effective finetuning. So we carefully selected10 online novels with similar language styles andbackdrops to form an augmented dataset along with
5https://github.com/pytorch/fairseq6https://dun.163.com/locale/en
177
Figure 1: Architecture of KuiLeiXi. The user input is first passed through the input processor module, whichdetects whether it contains toxic content and whether it is too semantically similar to the current plot goal; after theprocessing, the user input is concatenated to the existing context and truncated to ensure that the length is withinthe context length of the story generation model; the story generation model generates a series of candidate storiesthat are then sent to the candidate ranker for ranking; the ranker contains a filter that removes inappropriate storiesbased on multiple rules, and the remaining candidate stories are ranked based on their overlapping with the contextand how smoothly they connect to the plot goal, with the highest ranked being output to the player as the finalresult.
in-game story scripts. For scripts from the game,we assign every line a label indicating if it is dia-logue or narrative content, as seen in Figure 2. Itis easy because the dialogues and narratives arenaturally separated in different lines in the scriptsand the dialogues are with double quotation marks.This allows the finetuned model to control the gen-erate subsequent story contents to be dialogue ornarrative content. In addition, the label can guidethe model to generate more consistent content withthe story’s background similar to (Keskar et al.,2019).
2.3.2 InferenceInput Truncation: At inference, the generationmodel receives a concatenation of the player inputand the previous context as the input. As gamecontinues, the input length will easily exceed thecontext length 1024. So we need to design a trun-cation strategy. Naively keeping the latest storycontext is not feasible in this application, as thepre-written story beginning corresponding to thecurrent plot goal is neccessary to keep the storyunfold without straying too far from the currentplot goal. Therefore, we keep the pre-written storybeginning corresponding to the current plot goalalong with the latest story context as the input.Decoding Strategy: We use the top-k sam-pling(Fan et al., 2018) for decoding. Samplingtemperature and k are set to 0.8 and 8 respectively.We observed that the model tend to copy from the
input. To alleviate this issue, we adopt the penal-ized sampling technique(Keskar et al., 2019; Seeet al., 2019). In general, penalty sampling penal-izes words that occur throughout the context bydefault, reducing their sampling probability. How-ever, we argue that this is inappropriate, especiallyfor penalizing words that are far from the decod-ing position. The reasons are twofold. Firstly weobserved that, the model tends to copy from wordscloser to the decoding position, rather than a verydistant context, like contents with more than 200words away from the decoding position. Secondly,we conducted statistics in the webnovel corpus, andthe probability of the next word appearing in theprevious 800 words reached 75%, indicating thatcopying from context is also common in real worldtexts. In summary, if the probability of words oc-curring in very distant contexts is also penalizedat inference, the distribution of the generated textwill be significantly different from the real worldtext distribution, which may reduce the generationperformance. Therefore, we only penalize the prob-ability of words that have appeared in previous 200words prior to the decoding position.
Given the input tokens G[1, 2, .., t] and the con-text window size c, the probability distribution pifor the next token is defined as:
pi =exp(xi/(I(i ∈ g[(t− c) : t]))∑j(exp(xj/(I(j ∈ g[(t− c) : t])))
(1)
I(e) = θ if e is True else 1 (2)
178
Figure 2: A story fragment from the preprocessed story dataset.
We set θ to 1.2, which makes a balance betweengenerated text quality and elimination of duplica-tion.
2.4 Candidates Ranker
For each player input, we generate 5 candidatestories for re-ranking. The candidate stories arethen sent to the ranker to select the best one toreturn to the user.
To ensure the quality, we developed a series offiltering rules to remove inappropriate candidatesin the stories. Firstly, if a candidate story containsa character name that does not appear in YujianLove, the story will be moved out of the candidates.Secondly, candidate stories that contain a lot of con-tent copied from context will be removed. Thirdly,stories with inappropriate content detected by tox-icity detection service will be removed. Fourthly,if a character described in a story behaves incon-sistently with his or her gender, that story will alsobe removed. We trained a discriminator model todetect whether a character in the story behaves in-consistently with his/her gender. The training datais generated automatically. We use the original textas a positive sample and the text after the charac-ter name replacement as a negative sample. Whenreplacing character names, a character is replacedwith the name of a character having the other gen-der.
For the remaining stories, we rank them basedon the weighted sum of two metrics. The first is theoverlapping score, which is calculated based on theoverlapping of tokens in the generated story andcontext. Generally, when the overlapping score ishigher, repetition is heavier and will hurt the textquality. The second is the goal matching score,which measures how likely a story entails the cur-
rent plot goal. Given the list of context tokens C,the list of generated story tokens G and the lengthl of G, the overlapping score is defined below:
Scoreoverlap =
∑i I(i ∈ G)
l(3)
I(i) = 1 if i in G else 0 (4)
Determining whether a story contains a speci-fied plot is a typical textual entailment problem.However. because players can create story scriptsand submit them to the game community, it is in-tractable to create a dataset dealing with numerouspossible plot goals. So we had to approach theproblem from a different angle. We argue that itis easier to solve this problem by transforming itinto a problem similar to Next Sentence Prediction(NSP), i.e., determining whether a plot goal canbe coherently connected to a generated story. It iswell known that the original NSP task proposed inBERT(Devlin et al., 2019) is too easy, many lat-est pre-trained language models have abandonedit(Liu et al., 2019; Lan et al., 2019). We arguethat discriminating the randomly sampled negativeexamples is relatively easy so we adopt a novelstrategy to enhance the difficulty of NSP. Whengenerating the training dataset, in addition to therandomly sampled sentences, we also take the nextsentence of next sentence as a negative sample witha certain probability. We finetuned the pre-trainedRoberta-large based model as described in Section2.1 on this generated dataset. The finetuned modelis then used as a discriminator to detect whetherthe plot goal can be smoothly connected to thegenerated story.
179
(a) Script selection interface (b) Character selection interface
Figure 3: The script and character selection interfaces
(a) (b)
Figure 4: The screenshot at the beginning of the game. The right figure is the English translation of the left figure.Text in the orange box shows the story background. Text in red box shows the current plot goal. Text in the whiteboxes are the players’ inputs. Text in the purple boxes are the generated stories.
180
(a) (b)
Figure 5: The screenshot when a plot goal is reached. The right figure is the English translation of the left figure.
2.5 Optimization
Our original story generation model is of 36 layersand 725 millions of parameters. It takes around10 seconds to generate a piece of story with oneRTX 2080ti, which is totally unacceptable. To im-prove the inference speed, we need to compress theoriginal model. We firstly adopted the layerdroptechnique(Fan et al., 2020), reducing the numberof layers to 20. We then used the knowledge dis-tillation technique(Hinton et al., 2015) to distillthis 20-layer model. Finally, we finetuned the dis-tilled model over the story dataset.Our experimentsshowed that combining layerdrop and knowledgedistillation performs better than directly perform-ing knowledge distillation.
In addition, we optimized the incremental de-coding implementation in fairseq to reduce com-putation overhead. We developed custom CUDAkernels for better support of long sequence andlarge hidden size. We also developed an inferenceserver supporting dynamic batching and variableinput length. After applying these methods, the in-ference speed is increased by 10 times, the through-put is increased by 20 times. We have integratedthese optimization techniques into a python librarynamed as Easy and Efficient Transformer(?). It hasbeen opensourced at https://github.com/NetEase-FuXi/EET.
3 Demonstration
In this section, we demonstrate how to playKuiLeiXi.
First, we demonstrate how to start a game. Afterentering the game, if there is no ready game forjoining, you can click the create stage button tostart a new game. You then need to pick a storyscript from the candidates, as demonstrated in Fig-ure 3a. The scripts are both created by game de-velopers and players. Scripts submitted by playerswill be voted by all players and the winners be-come playable. After picking the script, you needto choose the character you want to play, as demon-strated in Figure 3b. The playable characters ineach script are different. Wait for other playersjoining your game until the number of players ex-ceeds the minimum player limit. Then you couldeither start the game or wait for other players join-ing as additional characters or audience.
After the game starts, all players can see the storybackground as well as the first plot goal. Playerswill play in order. The order is randomly decidedat the start of the game and does not change duringthe game. When it is your turn to play, you canchoose to write a dialogue with your character ordescribe a narration. Figure 4a shows the situationat the beginning of the game. Overall, you need toconsider the development of the current story, thepersona of the character you play and the plot goal.After completing the input, the AI will generate thecorresponding story to unfold based on the input,
181
so on and so forth until the current plot goal isreached. Once the current plot goal is reached, theAI will show a pre-written storyline, as well as anew plot goal, as seen in the Figure 5a. Usually ascript has multiple plot goals. When the final plotgoal is reached, players win and are rewarded withgame props. The whole process of playing will besaved in the database, and players can share it withtheir friends or make it public on social networks.
4 Conclusion
In this paper, we demonstrate KuiLeiXi, an open-ended text adventure game in Chinese. In orderfor it to be released as part of a commercial game,we have made many innovations based on AI Dun-geon. We believe that the current advances in NLPtechnology can not only reduce the cost of gamecontent development to a certain extent, but alsomake the game world more dynamic and person-alized. We hope our work will be of interest tofellow game developers and NLP researchers. Infuture work, we will further explore the genera-tion of game quests and ambient dialogues withup-to-date NLP techniques.
ReferencesSamuel R. Bowman, Gabor Angeli, Christopher Potts,
and Christopher D. Manning. 2015. A large anno-tated corpus for learning natural language inference.In Proceedings of the 2015 Conference on Empiri-cal Methods in Natural Language Processing, pages632–642, Lisbon, Portugal. Association for Compu-tational Linguistics.
Tom B. Brown, Benjamin Mann, Nick Ryder, MelanieSubbiah, Jared Kaplan, Prafulla Dhariwal, ArvindNeelakantan, Pranav Shyam, Girish Sastry, AmandaAskell, Sandhini Agarwal, Ariel Herbert-Voss,Gretchen Krueger, Tom Henighan, Rewon Child,Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu,Clemens Winter, Christopher Hesse, Mark Chen,Eric Sigler, Mateusz Litwin, Scott Gray, BenjaminChess, Jack Clark, Christopher Berner, Sam Mc-Candlish, Alec Radford, Ilya Sutskever, and DarioAmodei. 2020. Language models are few-shot learn-ers. In Advances in Neural Information ProcessingSystems 33: Annual Conference on Neural Informa-tion Processing Systems 2020, NeurIPS 2020, De-cember 6-12, 2020, virtual.
Jacob Devlin, Ming-Wei Chang, Kenton Lee, andKristina Toutanova. 2019. BERT: Pre-training ofdeep bidirectional transformers for language under-standing. In Proceedings of the 2019 Conferenceof the North American Chapter of the Associationfor Computational Linguistics: Human Language
Technologies, Volume 1 (Long and Short Papers),pages 4171–4186, Minneapolis, Minnesota. Associ-ation for Computational Linguistics.
Angela Fan, Edouard Grave, and Armand Joulin. 2020.Reducing transformer depth on demand with struc-tured dropout. In 8th International Conference onLearning Representations, ICLR 2020, Addis Ababa,Ethiopia, April 26-30, 2020. OpenReview.net.
Angela Fan, Mike Lewis, and Yann Dauphin. 2018. Hi-erarchical neural story generation. In Proceedingsof the 56th Annual Meeting of the Association forComputational Linguistics (Volume 1: Long Papers),pages 889–898, Melbourne, Australia. Associationfor Computational Linguistics.
Jian Guan, Fei Huang, Zhihao Zhao, Xiaoyan Zhu, andMinlie Huang. 2020. A Knowledge-Enhanced Pre-training Model for Commonsense Story Generation.Transactions of the Association for ComputationalLinguistics, 8:93–108.
Geoffrey Hinton, Oriol Vinyals, and Jeff Dean. 2015.Distilling the knowledge in a neural network. arXivpreprint arXiv:1503.02531.
Hai Hu, Kyle Richardson, Liang Xu, Lu Li, SandraKubler, and Lawrence Moss. 2020. OCNLI: Orig-inal Chinese Natural Language Inference. In Find-ings of the Association for Computational Linguis-tics: EMNLP 2020, pages 3512–3526, Online. As-sociation for Computational Linguistics.
Nitish Shirish Keskar, Bryan McCann, Lav R Varshney,Caiming Xiong, and Richard Socher. 2019. Ctrl: Aconditional transformer language model for control-lable generation. arXiv preprint arXiv:1909.05858.
Zhenzhong Lan, Mingda Chen, Sebastian Goodman,Kevin Gimpel, Piyush Sharma, and Radu Sori-cut. 2019. ALBERT: A lite BERT for self-supervised learning of language representations.CoRR, abs/1909.11942.
Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Man-dar Joshi, Danqi Chen, Omer Levy, Mike Lewis,Luke Zettlemoyer, and Veselin Stoyanov. 2019.Roberta: A robustly optimized bert pretraining ap-proach. arXiv preprint arXiv:1907.11692.
Alec Radford, Jeffrey Wu, Rewon Child, David Luan,Dario Amodei, and Ilya Sutskever. 2019. Languagemodels are unsupervised multitask learners. OpenAIblog, 1(8):9.
Nils Reimers and Iryna Gurevych. 2019. Sentence-BERT: Sentence embeddings using Siamese BERT-networks. In Proceedings of the 2019 Conference onEmpirical Methods in Natural Language Processingand the 9th International Joint Conference on Natu-ral Language Processing (EMNLP-IJCNLP), pages3982–3992, Hong Kong, China. Association forComputational Linguistics.
182
Stephen Roller, Emily Dinan, Naman Goyal, Da Ju,Mary Williamson, Yinhan Liu, Jing Xu, Myle Ott,Kurt Shuster, Eric M Smith, et al. 2020. Recipesfor building an open-domain chatbot. arXiv preprintarXiv:2004.13637.
Abigail See, Aneesh Pappu, Rohun Saxena, AkhilaYerukola, and Christopher D. Manning. 2019. Domassively pretrained language models make betterstorytellers? In Proceedings of the 23rd Confer-ence on Computational Natural Language Learning(CoNLL), pages 843–861, Hong Kong, China. Asso-ciation for Computational Linguistics.
Ashish Vaswani, Noam Shazeer, Niki Parmar, JakobUszkoreit, Llion Jones, Aidan N. Gomez, LukaszKaiser, and Illia Polosukhin. 2017. Attention is allyou need. In Advances in Neural Information Pro-cessing Systems 30: Annual Conference on NeuralInformation Processing Systems 2017, December 4-9, 2017, Long Beach, CA, USA, pages 5998–6008.
Adina Williams, Nikita Nangia, and Samuel Bowman.2018. A broad-coverage challenge corpus for sen-tence understanding through inference. In Proceed-ings of the 2018 Conference of the North AmericanChapter of the Association for Computational Lin-guistics: Human Language Technologies, Volume1 (Long Papers), pages 1112–1122, New Orleans,Louisiana. Association for Computational Linguis-tics.
Rongsheng Zhang, Xiaoxi Mao, Le Li, Lin Jiang, LinChen, Zhiwei Hu, Yadong Xi, Changjie Fan, andMinlie Huang. 2020a. Youling: an AI-assisted lyricscreation system. In Proceedings of the 2020 Con-ference on Empirical Methods in Natural LanguageProcessing: System Demonstrations, pages 85–91,Online. Association for Computational Linguistics.
Zhengyan Zhang, Xu Han, Hao Zhou, Pei Ke, YuxianGu, Deming Ye, Yujia Qin, Yusheng Su, HaozheJi, Jian Guan, Fanchao Qi, Xiaozhi Wang, YananZheng, Guoyang Zeng, Huanqi Cao, Shengqi Chen,Daixuan Li, Zhenbo Sun, Zhiyuan Liu, MinlieHuang, Wentao Han, Jie Tang, Juanzi Li, XiaoyanZhu, and Maosong Sun. 2020b. Cpm: A large-scalegenerative chinese pre-trained language model.
Chen Zhu, Yu Cheng, Zhe Gan, Siqi Sun, Tom Gold-stein, and Jingjing Liu. 2020. Freelb: Enhanced ad-versarial training for natural language understanding.In 8th International Conference on Learning Repre-sentations, ICLR 2020, Addis Ababa, Ethiopia, April26-30, 2020. OpenReview.net.
5 Appendices
In the figures below, we present a complete game-play record of KuiLeiXi.
Figure 6: The first part of the game play.
183
Figure 7: The second part of the game play. Figure 8: The third part of the game play.
184
Figure 9: The fourth part of the game play.Figure 10: The fifth part of the game play.