storyflow - visually tracking evolution of stories
DESCRIPTION
Storyline visualizations, which are useful in many applications, aim to illustrate the dynamic relationships between entities in a story. However, the growing complexity and scalability of stories pose great challenges for existing approaches. In this paper, we propose an efficient optimization approach to generating an aesthetically appealing storyline visualization, which effectively handles the hierarchical relationships between entities over time. The approach formulates the storyline layout as a novel hybrid optimization approach that combines discrete and continuous optimization. The discrete method generates an initial layout through the ordering and alignment of entities, and the continuous method optimizes the initial layout to produce the optimal one. The efficient approach makes real-time interactions (e.g., bundling and straightening) possible, thus enabling users to better understand and track how the story evolves. This work was presented in IEEE InfoVis 2013. Project page: http://research.microsoft.com/en-us/um/people/ycwu/projects/infovis13.htmlTRANSCRIPT
![Page 1: StoryFlow - Visually Tracking Evolution of Stories](https://reader036.vdocuments.us/reader036/viewer/2022062418/556251ddd8b42a1b4b8b4e04/html5/thumbnails/1.jpg)
1
StoryFlow: Tracking the Evolution of Stories
Shixia Liu, Yingcai Wu, Enxun Wei, Mengchen Liu, Yang LiuMicrosoft Research Asia
![Page 2: StoryFlow - Visually Tracking Evolution of Stories](https://reader036.vdocuments.us/reader036/viewer/2022062418/556251ddd8b42a1b4b8b4e04/html5/thumbnails/2.jpg)
Outline Introduction
Optimization Framework
StoryFlow Layout
Interactive Exploration
Experiments
Conclusion
![Page 3: StoryFlow - Visually Tracking Evolution of Stories](https://reader036.vdocuments.us/reader036/viewer/2022062418/556251ddd8b42a1b4b8b4e04/html5/thumbnails/3.jpg)
Outline Introduction
Optimization Framework
StoryFlow Layout
Interactive Exploration
Experiments
Conclusion
![Page 4: StoryFlow - Visually Tracking Evolution of Stories](https://reader036.vdocuments.us/reader036/viewer/2022062418/556251ddd8b42a1b4b8b4e04/html5/thumbnails/4.jpg)
Storytelling
![Page 5: StoryFlow - Visually Tracking Evolution of Stories](https://reader036.vdocuments.us/reader036/viewer/2022062418/556251ddd8b42a1b4b8b4e04/html5/thumbnails/5.jpg)
Who, When, and Where
![Page 6: StoryFlow - Visually Tracking Evolution of Stories](https://reader036.vdocuments.us/reader036/viewer/2022062418/556251ddd8b42a1b4b8b4e04/html5/thumbnails/6.jpg)
Stories Are Complicated The dynamic relationships of characters
![Page 7: StoryFlow - Visually Tracking Evolution of Stories](https://reader036.vdocuments.us/reader036/viewer/2022062418/556251ddd8b42a1b4b8b4e04/html5/thumbnails/7.jpg)
Randall Munroe’s Storyline Visualization
![Page 8: StoryFlow - Visually Tracking Evolution of Stories](https://reader036.vdocuments.us/reader036/viewer/2022062418/556251ddd8b42a1b4b8b4e04/html5/thumbnails/8.jpg)
Storyline Visualization
time
![Page 9: StoryFlow - Visually Tracking Evolution of Stories](https://reader036.vdocuments.us/reader036/viewer/2022062418/556251ddd8b42a1b4b8b4e04/html5/thumbnails/9.jpg)
Storyline Visualization
T-Rex
One character
time
DinosaursHuman
![Page 10: StoryFlow - Visually Tracking Evolution of Stories](https://reader036.vdocuments.us/reader036/viewer/2022062418/556251ddd8b42a1b4b8b4e04/html5/thumbnails/10.jpg)
Storyline Visualization
Five characters in the same scene
time
DinosaursHuman
![Page 11: StoryFlow - Visually Tracking Evolution of Stories](https://reader036.vdocuments.us/reader036/viewer/2022062418/556251ddd8b42a1b4b8b4e04/html5/thumbnails/11.jpg)
Storyline Visualization
DinosaursHuman
time
![Page 12: StoryFlow - Visually Tracking Evolution of Stories](https://reader036.vdocuments.us/reader036/viewer/2022062418/556251ddd8b42a1b4b8b4e04/html5/thumbnails/12.jpg)
Storyline Visualization
time
![Page 13: StoryFlow - Visually Tracking Evolution of Stories](https://reader036.vdocuments.us/reader036/viewer/2022062418/556251ddd8b42a1b4b8b4e04/html5/thumbnails/13.jpg)
Storyline Visualization Applications
Tracing genealogical data Tracking community evolution
Kim et al. 2010 Reda et al. 2011
![Page 14: StoryFlow - Visually Tracking Evolution of Stories](https://reader036.vdocuments.us/reader036/viewer/2022062418/556251ddd8b42a1b4b8b4e04/html5/thumbnails/14.jpg)
General Storyline Layout Yuzuru Tanahashi and Prof. Kwan-Liu Ma’s work
Dreams inside dreams
![Page 15: StoryFlow - Visually Tracking Evolution of Stories](https://reader036.vdocuments.us/reader036/viewer/2022062418/556251ddd8b42a1b4b8b4e04/html5/thumbnails/15.jpg)
Hierarchical Locations
![Page 16: StoryFlow - Visually Tracking Evolution of Stories](https://reader036.vdocuments.us/reader036/viewer/2022062418/556251ddd8b42a1b4b8b4e04/html5/thumbnails/16.jpg)
StoryFlow Real-time interactions
Level-of-detail rendering
Location hierarchy
First debate VP debate Second debate Third debate
![Page 17: StoryFlow - Visually Tracking Evolution of Stories](https://reader036.vdocuments.us/reader036/viewer/2022062418/556251ddd8b42a1b4b8b4e04/html5/thumbnails/17.jpg)
Outline Introduction
Optimization Framework
StoryFlow Layout
Interactive Exploration
Experiments
Conclusion
![Page 18: StoryFlow - Visually Tracking Evolution of Stories](https://reader036.vdocuments.us/reader036/viewer/2022062418/556251ddd8b42a1b4b8b4e04/html5/thumbnails/18.jpg)
System
![Page 19: StoryFlow - Visually Tracking Evolution of Stories](https://reader036.vdocuments.us/reader036/viewer/2022062418/556251ddd8b42a1b4b8b4e04/html5/thumbnails/19.jpg)
Input Data
Location hierarchy
Session list
![Page 20: StoryFlow - Visually Tracking Evolution of Stories](https://reader036.vdocuments.us/reader036/viewer/2022062418/556251ddd8b42a1b4b8b4e04/html5/thumbnails/20.jpg)
Objectives
Crossings
Wiggles
White Space
![Page 21: StoryFlow - Visually Tracking Evolution of Stories](https://reader036.vdocuments.us/reader036/viewer/2022062418/556251ddd8b42a1b4b8b4e04/html5/thumbnails/21.jpg)
Optimization Strategy
Crossings
Number of wiggles
White space
Importance decrease
Wiggle distance
Wiggles
Discrete
Continuous
Wiggle distance
![Page 22: StoryFlow - Visually Tracking Evolution of Stories](https://reader036.vdocuments.us/reader036/viewer/2022062418/556251ddd8b42a1b4b8b4e04/html5/thumbnails/22.jpg)
Outline Introduction
Optimization Framework
StoryFlow Layout
Interactive Exploration
Experiments
Conclusion
![Page 23: StoryFlow - Visually Tracking Evolution of Stories](https://reader036.vdocuments.us/reader036/viewer/2022062418/556251ddd8b42a1b4b8b4e04/html5/thumbnails/23.jpg)
Discrete and Continuous optimization
Discrete optimization– Edge crossings
– Number of wiggles
Continuous optimization– Wiggle distance
– White space
![Page 24: StoryFlow - Visually Tracking Evolution of Stories](https://reader036.vdocuments.us/reader036/viewer/2022062418/556251ddd8b42a1b4b8b4e04/html5/thumbnails/24.jpg)
Hierarchy Generation
Location tree Session list
Relationship trees
![Page 25: StoryFlow - Visually Tracking Evolution of Stories](https://reader036.vdocuments.us/reader036/viewer/2022062418/556251ddd8b42a1b4b8b4e04/html5/thumbnails/25.jpg)
Ordering 1. Sorting location nodes using a
greedy algorithm from bottom to top 2. Ordering sessions based on a DAG
barycenter sweeping algorithm
![Page 26: StoryFlow - Visually Tracking Evolution of Stories](https://reader036.vdocuments.us/reader036/viewer/2022062418/556251ddd8b42a1b4b8b4e04/html5/thumbnails/26.jpg)
Alignment Longest common subsequence ABCDEFG
BCDGK BCDG
![Page 27: StoryFlow - Visually Tracking Evolution of Stories](https://reader036.vdocuments.us/reader036/viewer/2022062418/556251ddd8b42a1b4b8b4e04/html5/thumbnails/27.jpg)
Compaction Quadratic programming
12 2
, , 1 ,1 1 1
( )e t e tn n n n
i j i j i ji j i j
Minimize y y y
1 2 1 2, , , ,
, , 1 , , 1
, 1, , 1,
, 1, , 1,
, if ;
, if ;
, if ( ) ( );
, if (
) ( ).
i j i j i j i j
i j i j i j i j
i j i j in i j i j
i j i j out i j i j
y y S S
y y S S
y y d SID S SID S
y y d SID S SID S
Subject to
Line order
Line alignment
Line adjacency
Line separate
![Page 28: StoryFlow - Visually Tracking Evolution of Stories](https://reader036.vdocuments.us/reader036/viewer/2022062418/556251ddd8b42a1b4b8b4e04/html5/thumbnails/28.jpg)
Outline Introduction
Optimization Framework
StoryFlow Layout
Interactive Exploration
Experiments
Conclusion
![Page 29: StoryFlow - Visually Tracking Evolution of Stories](https://reader036.vdocuments.us/reader036/viewer/2022062418/556251ddd8b42a1b4b8b4e04/html5/thumbnails/29.jpg)
System
![Page 30: StoryFlow - Visually Tracking Evolution of Stories](https://reader036.vdocuments.us/reader036/viewer/2022062418/556251ddd8b42a1b4b8b4e04/html5/thumbnails/30.jpg)
User Interactions
![Page 31: StoryFlow - Visually Tracking Evolution of Stories](https://reader036.vdocuments.us/reader036/viewer/2022062418/556251ddd8b42a1b4b8b4e04/html5/thumbnails/31.jpg)
User Interactions
![Page 32: StoryFlow - Visually Tracking Evolution of Stories](https://reader036.vdocuments.us/reader036/viewer/2022062418/556251ddd8b42a1b4b8b4e04/html5/thumbnails/32.jpg)
User Interactions
![Page 33: StoryFlow - Visually Tracking Evolution of Stories](https://reader036.vdocuments.us/reader036/viewer/2022062418/556251ddd8b42a1b4b8b4e04/html5/thumbnails/33.jpg)
User Interactions
![Page 34: StoryFlow - Visually Tracking Evolution of Stories](https://reader036.vdocuments.us/reader036/viewer/2022062418/556251ddd8b42a1b4b8b4e04/html5/thumbnails/34.jpg)
Outline Introduction
Optimization Framework
StoryFlow Layout
Interactive Exploration
Experiments
Conclusion
![Page 35: StoryFlow - Visually Tracking Evolution of Stories](https://reader036.vdocuments.us/reader036/viewer/2022062418/556251ddd8b42a1b4b8b4e04/html5/thumbnails/35.jpg)
Quantitative Analysis1
Movie Examples2
Case Study3
Evaluation
![Page 36: StoryFlow - Visually Tracking Evolution of Stories](https://reader036.vdocuments.us/reader036/viewer/2022062418/556251ddd8b42a1b4b8b4e04/html5/thumbnails/36.jpg)
Quantitative Analysis Intel i7-2600 CPU (3.4GHz)
8GB memory
Data Time(s) Crossings Wiggles
#Entity #Frame Ours GA Ours GA Ours GA
Star Wars 14 50 0.16 129.79 48 93 82 133
Inception 8 71 0.16 149.67 23 99 88 162
Matrix 14 42 0.16 172.47 14 43 54 94
MID 79 523 0.60 >10^5 1267 1871 831 874
GA refers to Tanahashi and Ma’s method based on Genetic Algorithm (GA)
![Page 37: StoryFlow - Visually Tracking Evolution of Stories](https://reader036.vdocuments.us/reader036/viewer/2022062418/556251ddd8b42a1b4b8b4e04/html5/thumbnails/37.jpg)
Jurassic ParkOur method
GA method
Randall’s work
(a)
![Page 38: StoryFlow - Visually Tracking Evolution of Stories](https://reader036.vdocuments.us/reader036/viewer/2022062418/556251ddd8b42a1b4b8b4e04/html5/thumbnails/38.jpg)
Inception
Our method
GA method
![Page 39: StoryFlow - Visually Tracking Evolution of Stories](https://reader036.vdocuments.us/reader036/viewer/2022062418/556251ddd8b42a1b4b8b4e04/html5/thumbnails/39.jpg)
King LearOur method
GA method
![Page 40: StoryFlow - Visually Tracking Evolution of Stories](https://reader036.vdocuments.us/reader036/viewer/2022062418/556251ddd8b42a1b4b8b4e04/html5/thumbnails/40.jpg)
The Lord of the Rings Trilogy
![Page 41: StoryFlow - Visually Tracking Evolution of Stories](https://reader036.vdocuments.us/reader036/viewer/2022062418/556251ddd8b42a1b4b8b4e04/html5/thumbnails/41.jpg)
US 2012 Presidential Election– 2012 US presidential election Twitter Data
• 89,174,308 tweets from May 01, 2012 to November 20, 2012• 900 users: politicians (334), media (288), and grassroots (276 )
• Two-level location hierarchy– Five hot topics: Welfare, Defense, Economy, Election, and Horse race– 2,344 hot hashtags
• Session List
ID Hashtag Start End Members
0 Hashtag1 140 167 Opinion leader A, Opinion leader B
1 Hashtag2 145 180 Opinion leader C, Opinion leader D
![Page 42: StoryFlow - Visually Tracking Evolution of Stories](https://reader036.vdocuments.us/reader036/viewer/2022062418/556251ddd8b42a1b4b8b4e04/html5/thumbnails/42.jpg)
Overall Patterns (1/2) Five significant events on Election
– First debate, VP debate, second debate, and third debate
Defense
Election
Economy
Welfare
Horse Race
Media
Political Figures
Grassroots
First debate VP debate Second debate Third debate Voting
Timeline
![Page 43: StoryFlow - Visually Tracking Evolution of Stories](https://reader036.vdocuments.us/reader036/viewer/2022062418/556251ddd8b42a1b4b8b4e04/html5/thumbnails/43.jpg)
Overall Patterns (2/2) Three user groups focused mainly on Election
– Grassroots also focused on Economy and switched frequently
– Political figures were more focused
– Media occasionally switched
Defense
Election
Economy
Welfare
Horse Race
Media
Political Figures
Grassroots
Timeline
![Page 44: StoryFlow - Visually Tracking Evolution of Stories](https://reader036.vdocuments.us/reader036/viewer/2022062418/556251ddd8b42a1b4b8b4e04/html5/thumbnails/44.jpg)
Defense
Election
Economy
Welfare
Horse Race
Media
Political Figures
Grassroots
First debate VP debate Second debate Third debate Voting
Timeline
Significant Transition Transition from Election to Economy
Sensata
tlotteaparty
gop
think Romney is tough on china? ask the workers of #sensata about that as they train their Chinese replacements
![Page 45: StoryFlow - Visually Tracking Evolution of Stories](https://reader036.vdocuments.us/reader036/viewer/2022062418/556251ddd8b42a1b4b8b4e04/html5/thumbnails/45.jpg)
Defense
Election
Economy
Welfare
Horse Race
Media
Political Figures
Grassroots
First debate VP debate Second debate Third debate Voting
Timeline
Significant Transition Transition from Election to Economy
sandy
fema
Issue-attention cycle
![Page 46: StoryFlow - Visually Tracking Evolution of Stories](https://reader036.vdocuments.us/reader036/viewer/2022062418/556251ddd8b42a1b4b8b4e04/html5/thumbnails/46.jpg)
Outline Introduction
Optimization Framework
StoryFlow Layout
Interactive Exploration
Experiments
Conclusion
![Page 47: StoryFlow - Visually Tracking Evolution of Stories](https://reader036.vdocuments.us/reader036/viewer/2022062418/556251ddd8b42a1b4b8b4e04/html5/thumbnails/47.jpg)
Conclusion A Storyline visualization system
– An efficient hybrid optimization approach
– A hierarchy-aware storyline layout
– A method for interactively and progressively rendering
Future improvements– Flashback narrative
![Page 48: StoryFlow - Visually Tracking Evolution of Stories](https://reader036.vdocuments.us/reader036/viewer/2022062418/556251ddd8b42a1b4b8b4e04/html5/thumbnails/48.jpg)
Acknowledgements Prof. Jonathan J.H. Zhu @ CityU, Hong Kong
Prof. Tai-Quan Peng @ NTU, Singapore
Prof. Kwan-Liu Ma and Yuzuru Tanahashi @ UC Davis