chen, c. (2008) an information-theoretic view of visual analytics. ieee computer graphics &...

Chen, C. (2008) An information-theoretic view of visual analytics. IEEE Computer Graphics & Applications, Jan/Feb 2008.

Visual Analysis of Macroscopic Patterns Chaomei ChenCollege of Information Science and Technology

Visual Analysis of Macroscopic Patterns Chaomei ChenCollege of Information Science and Technology

Drexel Computer Science Colloquium. November 12, 2007


Questions

Question 1:

How do we recognize that something is interesting, or suspicious, or worth pursuing?


Questions

Question 2:

What does it take for us to decide whether it will be worthwhile going through a collection of information or a complex network of data?


Questions

Question 3:

How can we strategically ‘fast forward’ through a complex web of information at a higher-level of aggregation?


Outline

• Puzzles and mysteries• Information foraging and scent following• Bayesian reasoning• Detect surprises and semantic outliers• The role of structural holes in information

networks• Understanding high-profile and low-profile

information patterns


The Connecting-the-Dots Problem

"I don't think anybody could have predicted that they would try to use an airplane as a missile, a hijacked airplane as a missile," said national security adviser Condoleeza Rice on May 16, 2002.

"How is it possible we have a national security advisor coming out and saying we had no idea they could use planes as weapons when we had FBI records from 1991 stating that this is a possibility," said Kristen Breitweiser, one of four New Jersey widows who lobbied Congress and the president to appoint the commission.

The widows want to know why various government agencies didn't connect the dots before Sept. 11, such as warnings from FBI offices in Minnesota and Arizona about suspicious student pilots.


First Monday - Uncloaking Terrorist Networks by Valdis Krebs


Connectable Dots?

• Prior to the 9/11 terrorist attacks, – several foreign nationals enrolled in different

civilian flying schools to learn how to fly large commercial aircraft.

– They were interested in learning how to navigate civilian airlines, but not in landings or takeoffs.

– And they all paid cash for their lessons. – ……


Puzzles .vs. Mysteries

Puzzles• Where is bin Laden?

Mysteries• Why did Enron collapse?


Solving Mysteries

• We may have all the necessary information in front of us and yet fail to see the connection or recognize an emergent pattern.

• To solve a mystery, one needs to ask the right question.


Solving Mysteries

decomposition

aggregation


Macroscopic and Microscopic Levels

words

phrases

sentences

documents

digital libraries

concepts

associations

clusters

specialties

domains

disciplines


Information Foraging and Sense Making


What is my profitability here?

Gain=?Cost=?


Information Foraging Theory

• People adapt their search strategies to maximize their profitability, or the profit-investment ratio. – Profit: finding relevant information– Cost: time spent

• People may adapt their search by reconfiguring the information environment.


Information Scent

• Information scent is the perception of the value, cost, or accessible path of information sources.


Information Foraging at Macroscopic Levels through Information Networks


Information Entropy and Uncertainty


Uncertainty

• A good example– Voting in political elections

• deal with overwhelmingly diverse information• differentiate political positions• accommodate conflicting views• update beliefs in light of new evidence• make macroscopic, categorical decisions


Evidence and Beliefs

• The USS Scorpion was lost from the sea in May 1968.

• The search for the USS Scorpion nuclear submarine is a frequently told story of a successful application of Bayesian reasoning.


NSF Awards

PreprintsPublications

Citations Grants Long-Term PlansPatentsTextbook

Fo

resi

gh

tnes

s

arXiv, ADS

Science, Nature

USPTO

Web of Science

NSF (SGER)

NSF Annual

Budget Requests


NSF Small Grants for Exploratory Research (SGER) (2000-2007)


NSF Budget Requests FY2004-FY2008

CISEp=0.5


Saliency and Novelty


Structural Holes and Brokerage

• The lack of comprehensive connectivity among components in a social network.

• Information flows are restricted to the privileged few who are strategically positioned over structural holes.

• The presence of a structural hole has a potential for gaining distinct advantages.


Previous hot Previous hot topic?topic?

Turning Turning point?point?

Transition Transition path?path?

Current hot Current hot topic?topic?


Macroscopic Views of Information Contents Information Entropy (Vocabulary)


relative entropy


Information Indices


Interestingness, Unexpectedness and Actionability

interestingness

objective subjective

unexpectedness actionability

Δbeliefs

interested in learning how to

navigate civilian airlines, but not

in landings or takeoffs.


natural languagenatural languageprocessingprocessing

entity-relationentity-relationextractionextraction

statistical statistical modelingmodeling

featurefeatureselectionselection

association rulesassociation rulesclassificationclassification

summarizationsummarization

InformationInformationtheorytheory

informationinformationindicesindices

graphical graphical modelsmodels

belief belief networksnetworks

ontology constructionontology constructionpredictive modelspredictive models

decision treesdecision trees

information information scentscent

interestingnessinterestingnessnoveltynovelty

uncertaintyuncertainty predictabilitypredictability

search search strategiesstrategies

emergent propertiesemergent propertiesnovelty detectionnovelty detection

topic trackingtopic tracking

sense makingsense making

formulate formulate hypotheseshypotheses

evaluateevaluateevidenceevidence

decision makingdecision making

macroscopic viewsmacroscopic viewsnew theoriesnew theories

solved mysteriessolved mysteries

Analytical Analytical ReasoningReasoning

Information Information ForagingForaging

Aggregation & Aggregation & TransformationTransformation

Microscopic Microscopic StructuresStructures


Technical Contentshttp://www.pages.drexel.edu/~cc345/papers/papers.html

• CiteSpace– http://cluster.cis.drexel.edu/~cchen/citespace – Chen, C. (2006) CiteSpace II: Detecting and visualizing emerging trends and

transient patterns in scientific literature. Journal of the American Society for Information Science and Technology, 57(3), 359-377. http://cluster.cis.drexel.edu/%7Ecchen/citespace/doc/jasist2006.pdf

• Differentiating Conflicting Opinions– Chen, C., SanJuan, F. I., SanJuan, E., & Weaver, C. (2006) Visual analysis of

conflicting opinions. IEEE Symposium on Visual Analytics Science and Technology (VAST 2006), Baltimore, MA. Oct 31-Nov 2, 2006. pp. 59-66. http://cluster.cis.drexel.edu/%7Ecchen/papers/confs/vast2006-chen.pdf

• Scientific Discoveries– Chen, C., Zhang, J., Zhu, W., Vogeley, M. (2007) Delineating the citation impact of

scientific discoveries. IEEE/ACM Joint Conference on Digital Libraries (JCDL 2007). June 17-22, 2007. Vancouver, British Columbia, Canada. http://cluster.cis.drexel.edu/%7Ecchen/papers/confs/jcdl2007.pdf

• Knowledge Diffusion– Chen, C., Zhu, W., Tomaszewski, B., MacEachren, A. (2007) Tracing conceptual

and geospatial diffusion of knowledge. HCI International 2007. Beijing, China. July 22-27, 2007. http://cluster.cis.drexel.edu/%7Ecchen/papers/confs/hcii2007.pdf

http://cluster.cis.drexel.edu/~cchen/citespace


Acknowledgements

National Visualization and Analytics Center (NVAC)

Northeast Visualization and Analytics Center (NEVAC)

NSF IIS Award #0612129SEI: Coordinated Visualization and Analysis of Sky Survey Data and Astronomical Literature

http://www.nsf.gov/

http://nvac.pnl.gov/

http://www.dhs.gov/dhspublic/


Credits• http://www.care2.com/c2c/groups/disc.html?gpp=12960&pst=600297&archival=&posts=7

• http://www.princeton.edu/~rvdb/JAVA/election2004/

•


Before …

After!


**#1729:#1729: Catholic Bashing Lite (2003-12-25)

I have just finished "The Da Vinci Code". What an utter waste of time! Dan Brown adds nothing to the murder mystery genre with this book. Furthermore, the entire premise is implausible as the key element of the mystery, the "Priory of Sion" and its guardianship of the Holy Grail has been proven to be a complete hoax. Mr. Brown's biblical scholarship is shoddy, his analyses of the artworks of Leonardo are facile and, of course, he provides no motive for the secret, which has been kept for so long, to be kept. I am truly amazed that this book has received as much attention as it has. I am sorry to say that I wasted my money on this book. A much better read would be Umberto Eco's "Foucault's Pendulum" or "Badolino".

Holy Grail

Umberto Eco

chen, c. (2008) an information-theoretic view of visual analytics. ieee computer graphics &...

Documents

information networks

collection of information

mysteries information

complex web of information

valdis krebs slide

mysteries puzzles

questions question

dots problem