Proceedings of the NAACL HLT 2010 Workshop on Computational Approaches to Analysis and Generation of Emotion in Text, pages 125–130,Los Angeles, California, June 2010. c©2010 Association for Computational Linguistics

NewsViz:Emotional Visualization of News Stories

Eva Hanser, Paul Mc Kevitt, Tom Lunney and Joan CondellSchool of Computing & Intelligent Systems

Faculty of Computing & EngineeringUniversity of Ulster, Magee

Derry/Londonderry, BT48 7JLNorthern Ireland

hanser-e@email.ulster.ac.uk,{p.mckevitt, tf.lunney, j.condell}@ulster.ac.uk

Abstract

The NewsViz system aims to enhance newsreading experiences by integrating 30 secondslong Flash-animations into news article webpages depicting their content and emotionalaspects. NewsViz interprets football matchnews texts automatically and creates abstract2D visualizations. The user interface en-ables animators to further refine the anima-tions. Here, we focus on the emotion extrac-tion component of NewsViz which facilitatessubtle background visualization. NewsViz de-tects moods from news reports. The origi-nal text is part-of-speech tagged and adjec-tives and/or nouns, the word types convey-ing most emotional meaning, are filtered outand labeled with an emotion and intensityvalue. Subsequently reoccurring emotions arejoined into longer lasting moods and matchedwith appropriate animation presets. Differ-ent linguistic analysis methods were tested onNewsViz: word-by-word, sentence-based andminimum threshold summarization, to find aminimum number of occurrences of an emo-tion in forming a valid mood. NewsViz provedto be viable for the fixed domain of footballnews, grasping the overall moods and somemore detailed emotions precisely. NewsVizoffers an efficient technique to cater for theproduction of a large number of daily updatednews stories. NewsViz bypasses the lack ofinformation for background or environmentdepiction encountered in similar applications.Further development may refine the detectionof emotion shifts through summarization withthe full implementation of football and com-mon linguistic knowledge.

1 Introduction

News reports are regarded as objective facts, com-monly delivered in an objective, unbiased mannerand represented in a neutral and formal format: typ-ically a static headline, a summarizing paragraphwith one image and eventually the body text withone to three more images. Even though reportersfind the content of news stories worth mentioning foremotional reasons and the content often affects read-ers emotionally, story brevity, scarce background in-formation and poor combination of visual and verbalinformation hinders learning and feeling by view-ers. In order to reach the audience emotionally, toeducate and to entertain, emphasis on visual ele-ments is important as they tend to be more memo-rable than verbal ones. The emphasis of NewsVizlies on expression, impacting on the reader’s under-standing of the article and making it more memo-rable. The software prototype, NewsViz, automat-ically creates animations from news articles. Ab-stract design elements show emotions conveyed inthe stories. The main objective of NewViz remainsinformation provision and thus our focus is emotionextraction which is universally applicable and with-out opinion bias. NewsViz is an efficient softwaretool for designers to be able to build daily updatedanimations. Input for NewsViz is natural languagetext. Multimodal systems automatically mappingtext to visuals face challenges in interpreting humanlanguage which is variable, ambiguous, impreciseand relies on the communicative partners possess-ing common knowledge. Enabling a machine to un-derstand a natural language text involves feeding the

125

machine with grammatical structures, e.g. part-of-speech, semantic relations, e.g. emotion value andintensity, and visual descriptions, e.g. colors andmotion direction, to match suitable graphics.

2 Background and Related Research

Text-to-visual mapping relates to the areas of natu-ral language processing (NLP) and multimodal sto-rytelling which attempt to enable computers to in-terpret and generate natural human language andmental images. Text-to-visual mapping starts withlinguistic analysis of the text. Despite variability,ambiguity and imprecision, syntactic analysis toolsachieve mostly reliable results, such as trainablepart-of-speech tagger software tools which identifyparts of speech with 97% accuracy. For example,Qtag (Mason, 2003) attaches a tag to each word la-beling it as noun, verb, adjective or other.

Semantic interpretation and actual understandingof the meaning of a text is more difficult, because itdepends largely on commonsense knowledge. Com-monsense knowledge and mental images need to bestructured, related through logical rules and enteredinto databases before computational text interpreta-tion is possible. WordNet (Miller, 1995) determinessemantic relations between words and is an extendeddictionary specifying word relations such as simi-larity, part-of relations, hierarchy or manner. Storysegmentation is performed by e.g. SeLeCT (Stokes,2003), an example application based on semanticanalysis to find story or subtopic changes within atext. Groups of semantically related words calledcohesive ‘lexical chains’ are extracted from a text.They are determined through WordNet’s seman-tic relations and additionally through statisticallyacquired co-occurrences (e.g. Diego Maradonna,Hand of God). Their starting and end points indi-cate topical unit boundaries.

Sensing emotions from multimodal input hasmainly been investigated with the objective of de-veloping human-like agents. The football commen-tary system, Byrne (Binsted and Luke, 1999), in-cludes a commentator with emotions influenced byhis personality and intentions. SOCCER (Retz-Schmidt, 1988) analyses football scenes visually inorder to simultaneously add linguistic descriptionsof the events. SOBA (Buitelaar et al., 2006) ex-

tracts information from soccer match reports, an-notates relevant expressions (e.g. players, teams,goals.) and generates knowledge base entities. Thecollected football knowledge can set preconditionsand context to consequently evaluate current eventsand assign appropriate emotions. The MoodNewswebsite (Mitchell, 2005) demonstrates a very sim-ple linguistic method to distinguish positive, neg-ative and neutral content in BBC news headlines.It effectively ranks them on a color scale betweengood to bad. The three kinds of emotions are ap-pointed through keyword scoring based on a smallvocabulary of 160 words and phrases. The Emo-tion Sensitive News Agent (ESNA) (Shaikh et al.,2007) chategorizes news stories from different RSSsources into eight emotion categories according totheir emotional content, determined through a cog-nitive evaluation and user preferences.

Automated story visualization systems deliverinitial results for object and action depiction, as inWordsEye (Coyne and Sproat, 2001), creating static3D images from written descriptions. Additionally,automated camera and character animation, inter-action and speech synthesis is realized in CONFU-CIUS (Ma, 2006). ScriptViz (Liu and Leung, 2006)renders 3D scenes from NL screenplays immedi-ately during the writing process, extracting verbsand adverbs to interpret events and states in sen-tences. The Unseen Video (Scheibel and Wein-rother, 2005), is a good example of abstract moodvisualization. Local weather data is automaticallyretrieved from news websites and influences the lookand feel of the Flash animation through shapes, col-ors and images. The Story Picturing Engine (Joshiet al., 2004) visualizes texts selecting and matchingpictures and their annotations from image databases.

The work discussed here demonstrates that suffi-cient subsets of the English language can be mappedto computer understandable language for the visual-ization of stories.

3 The NewsViz System

NewsViz takes online news articles as input and out-puts animations reflecting the content of these newsstories. NewViz consists of three main components:the linguistic analysis, the animation composer andan interface for editing text and animations (Figure

126

Figure 1: NewsViz System Architecture.

1). The linguistic component constructs three ele-ments of the animation in different processes. Theemotion extraction tool creates atmospheric back-ground visuals, the action visualizer depicts people,objects and their actions and the audio creator se-lects music and sound effects. The composer syn-chronizes the different outputs. Here, we focus onthe emotion extraction component (Figure 2) devel-oped in Flash MX and Photoshop. Emotional as-pects within the news story are identified and linkedto appropriate presets of background animations.

3.1 Emotion Extraction

The first step in processing the text is to tag partsof speech for all words. The part-of-speech tagger,Qtag (Mason, 2003), attaches tags to nouns, verbs,adjectives and other parts of speech. The tagged textis sent on to the adjective and noun detector. These

Figure 2: Emotion Extraction Component.

two types of words are selected for further process-ing because they are most central to conveying emo-tional meaning and sufficient for the visualisation ofthe emotional content. Nouns and adjectives are theparts of speech which represent the highest num-ber of affective words as found in WordNet-Affect(Strapparava and Valitutti, 2004). Verbs and adverbswill be addressed in future work to increase sensi-tivity and precision, but their impact on the resultinganimations may not be as significant. Next, the emo-tion word selector checks the adjectives and nounsin the emotion dictionary and attaches emotion tagsindicating their kind of emotion and intensity. Thedictionary holds manually created emotion-indicesand default intensity values of all affective words.

Figure 3: Animations for Sadness (blue), Boredom (green), Tension (red) and Happiness (yellow).

127

Four emotions have been found relevant in relationto football matches - happiness, sadness, tension andboredom. Words with a neutral emotion index donot describe football relevant emotions. To achievea coherent course of emotion and animation, neutralphrases are replaced by the previous mood with de-creasing intensity. The list of emotion tagged wordsis handed to the emotion summarizer. During thesummarization process subsequent emotions of thesame type are combined to form one longer-lastingmood. Each mood is labeled with its type, averageintensity and display duration. With the ‘word-by-word’ summarization method mood boundaries ap-pear as soon as the emotion type of the next worddiffers. In order to reduce error and excessive moodswings, the minimum threshold method sets a mini-mum number of words required to represent a mood.Alternatively, the sentence-based method assumesthat one sentence conveys one idea and consequentlyone emotion. Hence, it calculates an average emo-tion for each sentence, before combining identicalemotions. A chronological list of mood chunks iscreated.

3.2 Animation Construction

The animation selection component loads the in-dividual animation elements from the graphicsdatabase and combines them in a 30 seconds longanimation. The graphics database contains prefab-ricated graphics sorted by an emotion index whichare combined and adjusted according to mood in-tensities. Based on the weighted mood list, the emo-tion sequence order, the type of graphic element, itsdisplay duration, and the background color are de-termined. The intensity value specifies the elementsize and the number of objects loaded. An emo-tion change causes the current animation elementsto fade out and to load different elements. Anima-tion examples are shown in Figure 3.

3.3 User Interface

NewsViz provides users with options to load or typenews stories into the text editor. The options menuoffers different emotion extraction and mood sum-marization methods. By pressing the ‘run’ buttonthe visualization can be watched in the preview win-dow. The text processing runs ‘on the fly’ in thebackground. If the user is satisfied they can save

the animation. If the user prefers to alter the anima-tion manually, they have the option to edit the orig-inal text or the animation elements frame by frame.Figure 4 shows the user interface with animationplayer. The final animations are integrated at the topof the news article’s internet page (Figure 5).

Figure 4: NewsViz User Interface.

Figure 5: Animation Integrated into Website.

4 Evaluation and Testing

NewsViz was tested on a set of four news articlesrelated to the same news domain - football matchreports. The articles were taken from BBC andFIFA online describing the same two World Cup2006 matches. The three different emotion extrac-tion methods, word-by-word, sentence-based and

128

Figure 6: Results Analysis of all Test Texts.

threshold were run on these news stories with vary-ing word types or word type combinations. The out-put of NewsViz is evaluated against two forms ofhuman interpretation of the articles. A short man-ual description outlines the general course of emo-tion of a match as reported in each article namingthree to five emotions. A second more fine grainedinterpretation assigns one (or two) emotions to eachsentence. In correspondence to Beeferman’s proba-bilistic error metric (Beeferman et al., 1999) threetypes of emotion extraction error are distinguished.Falsely detected emotions are rated with zero points.Missing emotions were assessed depending on theirsignificance in the text. If the overall feeling of thematch was represented, two to three points wouldbe given, but if the main emotions were missing, nopoints were assigned. Very close, but not exact emo-tions, got a value of four. A correct representationof the course of emotion received five points. Thegrain counts the number of the extracted emotionsper text. The results for correctness of emotionalfindings and amount of emotions detected (grain)of each method run on each part-of-speech or wordtype combination are presented in Figure 6.

The results analysis shows that the effectivenessof adjectives or nouns varies from text to text, butgenerally the best results are achieved with the ex-traction of both kinds of words. On average theword-by-word method produces emotion sequenceswith the closest correctness, but unfortunately itsoutput is too fine grained for visualization. Thirtysecond long animations are best visualized with twoto ten mood swings. This means that some formof summarization is needed. Combining emotionsof logically structured chunks of text, namely sen-

tences, in the sentence-based summarization methodachieved better results than the minimum subse-quent occurrence of two or three emotions withthe threshold method. The sentence-based sum-marizaion as well as the threshold method with aminimum value of 3 produce the most appropriategrain/number of emotions. Some misinterpretationis due to false part-of-speech tagging by Qtag whichhas particular trouble with proper nouns. More accu-racy can be achieved through training Qtag on foot-ball reports. Overall the results for NewsViz are sat-isfactory and it demonstrates that it is possible to ex-tract emotions from news texts. The generally differ-ent sensations of the two described football matchesare distinguishable. Three of the four test texts showgood results, but for one article the extracted emo-tions do not seem to match the human sensation.

5 Relation to Other Work

NewsViz uses natural human language as input tocreate animated output. NewsViz aims to solely re-flect emotions as they are mentioned in the news ar-ticle to keep the objective and formal character ofnews reporting. Therefore, NewsViz applies a re-duced, universal and ’personality-free’ version ofexisting concepts for emotion and mood construc-tion. Instead of facial expressions and gesturesNewsViz combines and illustrates emotions with de-sign principles. NewsViz offers manual reediting ofthe automatically created animations.

6 Conclusion and Future Work

NewsViz extracts emotion-bearing words from on-line football news reports based on an extended dic-tionary with emotion-indices assigned to each en-

129

try. The extracted emotions are processed and illus-trated in abstract background animations. Resultsfrom initial testing demonstrate that this automatedprocess has satisfactory performance. Technolog-ically, NewsViz is viable for the fixed domain offootball reports and offers a sound basis for moreaffective text-to-visual mapping. Future work willaim to improve the linguistic and semantic process-ing of emotions. This involves the extension of theparts of speech selection to include verbs and ad-verbs, assuming that more input data will lead to bet-ter results. Rules for common and linguistic knowl-edge will be integrated. Linguistic knowledge iden-tifies emotions in context applying language rulesto emotion interpretation, i.e. it solves negation byinverting emotions. With the integration of a de-pendency parser, which relates words according totheir sentence structure, emotions of related wordscan be found and their average emotion determined.Domain-specific knowledge (e.g. football) providesbackground information including match statistics,players’ and teams’ names, team colors and leaguetables. It also accommodates game rules or matchsituations with their emotional consequences. Themood list is refined through moods discovered withcommonsense knowledge and football facts whichset pre-conditions and context representing long-term moods influencing current event-based emo-tions. Future work will reveal whether NewsViz isfeasible when extended to different domains. Theemotion database could be extended through theWordNet-Affect dictionary (Strapparava and Vali-tutti, 2004). NewsViz enriches standard news web-sites with attractive and informative animations andcan track emotional aspects of people’s views onworld events. NewsViz brings news reported onthe internet closer to readers, making it more eas-ily understood and memorized which is much appre-ciated by online users overloaded with information.NewsViz assists animation designers in the produc-tion of daily updated visualizations creating initialscenes.

References

D. Beeferman, A. Berger and J. Lafferty. 1999. Statisti-cal models for text segmentation. Machine Learning,34:177–210. Springer Netherlands.

K. Binsted and S. Luke. 1999. Character Design for Soc-cer Commentary. Lecture Notes in Computer Science.RoboCup-98: Robot Soccer World Cup II, 1604:22–33. Springer-Verlag, London, UK.

P. Buitelaar, T. Eigner, G. Gulrajani, A. Schutz, M.Siegel, N. Weber, P. Cimiano, G. Ladwig, M. Mantel,H. Zhu. 2006. Generating and Visualizing a SoccerKnowledge Base. Proceedings of the EACL06 DemoSession, 4/2006:123–126.

B. Coyne and R. Sproat. 2001. WordsEye: an auto-matic text-to-scene conversion system. Proceedingsof the 28th Annual Conference on Computer Graph-ics and Interactive Techniques, 487–496. ACM Press,Los Angeles, USA.

D. Joshi, J. Z. Wang and J. Li. 2004. The Story Pictur-ing Engine: Finding Elite Images to Illustrate a StoryUsing Mutual Reinforcement. Proceedings of the 6thACM SIGMM International Workshop on MultimediaInformation Retrieval, 119–126. ACM Press, NewYork, USA.

Z. Liu and K. Leung. 2006. Script visualiza-tion (ScriptViz): a smart system that makes writ-ing fun. Soft Computing, 10(1), 34–40. SpringerBerlin/Heidelberg, Germany.

Minhua Ma. 2006. Automatic Conversion of NaturalLanguage to 3D Animation. Ph.D. Thesis. Schoolof Computing and Intelligent Systems, University ofUlster, UK.

O. Mason. 2003. Qtag. http://phrasys.net/uob/om/software.

G. A. Miller. 1995. WordNet: a lexical database forEnglish. Communications of the ACM, 38(11):39–41.

Davy Mitchell. 2005. MoodNews. http://www.latedecember.com/sites/moodnews.

G. Retz-Schmidt. 1988. A REPLAI of SOCCER Recog-nizing intensions in the domain of soccer games. Proc.European Conf. AI (ECAI-88), 8:455-457.

Daniel Scheibel and Ferdinand Weinrother.2005. The Unseen Video. http://www.theunseenvideo.com.

Mostafa Al Masum Shaikh, Helmut Prendinger and Mit-suru Ishizuka. 2007. Emotion Sensitive News Agent:An Approarch Towards User Centric Emotion Sensingfrom the News. Proceedings 2007 IEEE/WIC/ACMInternational Conference on Web Intelligence (WI07),614–620. Silicon Valley, USA.

N. Stokes. 2003. Spoken and Written News Story Seg-mentation Using Lexical Chains. Proceedings of HTL-NAACL 2003, 49–54. Edmonton, Canada.

C. Strapparava and A. Valitutti. 2004. WordNet-Affect:an Affective Extension of WordNet. Proceedings ofthe 4th International Conference on Language Re-sources and Evaluation (LREC 2004), 1083–1086.

130