kmi hyper 2009

1

The Hypermedia Discourse Project Tools for Annotating, Visualizing & Navigating Literature as Discourse Networks

Simon Buckingham Shum

Knowledge Media Institute The Open University Milton Keynes, UK

http://people.kmi.open.ac.uk/sbs http://projects.kmi.open.ac.uk/hyperdiscourse

HypER Workshop: Hypotheses, Evidence and Relationships11-12 May 2009, Elsevier, Amsterdam

2

Compendium Institute

http://projects.kmi.open.ac.uk/hyperdiscourse

3

Compendium Institute

http://projects.kmi.open.ac.uk/hyperdiscourse

questions

4

5

1665 throws a long shadow

Philosophical Transactions of the Royal Society of London March 1665

Le Journal des Sçavans January 1665

From: To…?

Chaomei Chen, 2006: Citation analysis

Buckingham Shum et al, 2003: lineage analysis

Buckingham Shum, S. (2007). Digital Research Discourse? Computational Thinking Seminar Series, School of Informatics, University of Edinburgh, 25 Apr. 2007. http://kmi.open.ac.uk/projects/hyperdiscourse/docs/Simon-Edin-CompThink.pdf

The question we used to ask in 2001 at the start of the ScholOnto project

  In 2010, will we still be publishing scientific results

primarily as prose papers, or will a complementary

infrastructure emerge that exploits the power of

the social, semantic web to model the literature as

a network of claims and arguments?

6

The question we used to ask in 2001 at the start of the ScholOnto project

  In 2010, will we still be publishing scientific results

primarily as prose papers, or will a complementary

infrastructure emerge that exploits the power of

the social, semantic web to model the literature as

a network of claims and arguments?

7

20xx?

8

Questions the next generation scientific infrastructure should help answer

•  “What is the evidence for this claim?”

•  “Was this prediction accurate?”

•  “What are the conceptual foundations for this idea?”

•  “Who’s built on this idea? How?”

•  “Who’s challenged this idea? Why? How?”

•  “Are there distinctive perspectives on this problem?”

•  “Are there inconsistencies within this school of thought?”

assumptions

9

  Researchers read meanings into texts that are not there, and with which the author might disagree   so we will always require manual annotation tools   we need ways to make connections to connections   extremely complex connections may remain the province of human sensemaking

(e.g. is analogous to)

  Good user interfaces will be needed   to view, edit and navigate HypERnets, whether manually or automatically constructed

  Scientific discourse is a social process   we take huge care in our writing about how we position ourselves in relation to our

peers — will we trust unsupervised machines to extract and position our more complex claims?

10

modelling schemes: IBIS

11

12

Rittel’s IBIS: Issue-Based Information System

13

Compendium: customisable, collaborative, hypermedia IBIS mapping

Buckingham Shum, S., Selvin, A., Sierhuis, M., Conklin, J., Haley, C. and Nuseibeh, B. (2006). Hypermedia Support for Argumentation-Based Rationale: 15 Years on from gIBIS and QOC. In: Rationale Management in Software Engineering (Eds.) A.H. Dutoit, R. McCall, I. Mistrik, and B. Paech. Springer-Verlag: Berlin

14

IBIS mapping of Iraq debate

Buckingham Shum, S., and A. Okada. 2008. Knowledge cartography for controversies: The Iraq debate. In Knowledge cartography: Software tools and mapping techniques, ed. A. Okada, S. Buckingham Shum, and T. Sherborne, 249–66. London: Springer.

Mapping a nuclear power debate on a blog

Mapping a nuclear power debate on a blog

Mapping a nuclear power debate on a blog

modelling schemes: ScholOnto

18

19

ScholOnto schema Connecting freeform tags with naturalistic connections (“dialects”) grounded in a formal set of relations (from semiotics and coherence relations)

Mancini, C. and Buckingham Shum, S.J. (2006). Modelling Discourse in Contested Domains: A Semiotic and Cognitive Framework. International Journal of Human Computer Studies, 64, (11), pp.1154-1171. [PrePrint: http://oro.open.ac.uk/6441]

20

Paper: “The Scent of a Site: A System for Analyzing and Predicting Information Scent,

Usage, and Usability of a Web Site”

“Web User Flow by Information Scent (WUFIS)”

Paper: “Information foraging”

“Information foraging theory”

“Information scent models”

“People try to maximise their rate of gaining information”

?

applies

Scholarly discourse as CKS… Beyond document citations…

These annotations are freeform summaries of an

idea, as one would also find in researchers’ journals,

fieldnotes, lit. review notes or blog entries

Making formal connections between ideas creates a

semantic citation network —> novel literature navigation, querying and visualization

Method

Theory

Claim

topic maps and subject centric federation

21

Schematic: Documents, Subjects,and Relations

Document

Subjects in documents

Relations between subjects

Occurrence links

Topic Map of documents and their subjects

Federated Subjects

interaction design

24

25

Interaction design for literature visualization: pilot study: paper-based literature modelling

S. Buckingham Shum, V. Uren, G. Li, B. Sereno, and C. Mancini. Computational Modelling of Naturalistic Argumentation in Research Literatures: Representation and Interaction Design Issues. International Journal of Intelligent Systems, 22(1):17–47, 2006

26

Interaction design for lit. visualization From paper prototype to semiformal mapping tool   The ClaiMapper tool

Evaluated in: V. Uren, S. Buckingham Shum, G. Li, and M. Bachler. Sensemaking Tools for Understanding Research Literatures: Design, Implementation and User Evaluation. International Journal of Human Computer Studies, 64(5):420–445, 2006

…to formal argument maps

Starting from paper-based modelling, move from literature sketches…

27

Interaction design for doc. annotation Pilot study: paper-based annotation

Pilot study reported in: B. Sereno, S. Buckingham Shum, and E. Motta. (2005). ClaimSpotter: an Environment to Support Sensemaking with Knowledge Triples. Proc. Int. Conf. Intelligent User Interfaces, pages 199–206, ACM

28

The ClaimSpotter annotation tool   Web 2.0-style tagging with optional community/system tag

recommendations

Sereno, B., Buckingham Shum, S. and Motta, E. (2007). Formalization, User Strategy and Interaction Design: Users’ Behaviour with Discourse Tagging Semantics. Workshop on Social and Collaborative Construction of Structured Knowledge, 16th Int. World Wide Web Conference, Banff, Canada; 8-12 May 2007.

29

Lessons Learnt & Design Principles   Untrained users can do it: in their first hour they created

coherent claims. UI design validated to this degree. —future work: longitudinal evaluation at scale

  New users attend to what is highlighted for them (matching tags; primary doct.), and generally don’t click down a level —next version combines visualizations and document-centric features

  Support incremental formalization —cf. use of is-about as a placeholder link; provide an Other… category and try to map automatically to the ontology

  Users’ strategies vary — don’t assume a strong workflow a paper-based pilot study can provide insights into this

  Web 2.0 UI simplicity: good design needed to provide high functionality, walk-up-and-use tools —we overwhelmed some users with overlaid suggestions for tags

Sereno, B., Buckingham Shum, S. and Motta, E. (2007). Formalization, User Strategy and Interaction Design: Users’ Behaviour with Discourse Tagging Semantics. Workshop on Social and Collaborative Construction of Structured Knowledge, 16th Int. World Wide Web Conference, Banff, Canada; 8-12 May 2007.

30

Cohere: from tag clouds to idea webs

Buckingham Shum, Simon (2008). Cohere: Towards Web 2.0 Argumentation. In: Proc. COMMA'08: 2nd International Conference on Computational Models of Argument, 28-30 May 2008, Toulouse, France. IOS Press [PrePrint: http://oro.open.ac.uk/10421]

31

Cohere: embedding an Idea or Map in another website (a blog post)

Buckingham Shum, Simon (2008). Cohere: Towards Web 2.0 Argumentation. In: Proc. COMMA'08: 2nd International Conference on Computational Models of Argument, 28-30 May 2008, Toulouse, France. IOS Press [PrePrint: http://oro.open.ac.uk/10421]

32

Cohere: a mashup visualization merging different connections around a common Idea

Buckingham Shum, Simon (2008). Cohere: Towards Web 2.0 Argumentation. In: Proc. COMMA'08: 2nd International Conference on Computational Models of Argument, 28-30 May 2008, Toulouse, France. IOS Press [PrePrint: http://oro.open.ac.uk/10421]

33

Cohere: semantically filtering a focal Idea by “contrasting” connections

Buckingham Shum, Simon (2008). Cohere: Towards Web 2.0 Argumentation. In: Proc. COMMA'08: 2nd International Conference on Computational Models of Argument, 28-30 May 2008, Toulouse, France. IOS Press [PrePrint: http://oro.open.ac.uk/10421]

34

Cohere: semantically filtering a focal Idea by “contrasting” connections

Buckingham Shum, Simon (2008). Cohere: Towards Web 2.0 Argumentation. In: Proc. COMMA'08: 2nd International Conference on Computational Models of Argument, 28-30 May 2008, Toulouse, France. IOS Press [PrePrint: http://oro.open.ac.uk/10421]

35

“What papers contrast with this paper?”

1.   Extract concepts for this document 2.   Trace concepts on which they build 3.   Trace concepts challenging this set 4.   Show root documents

Evaluated in: V. Uren, S. Buckingham Shum, G. Li, and M. Bachler. Sensemaking Tools for Understanding Research Literatures: Design, Implementation and User Evaluation. International Journal of Human Computer Studies, 64(5):420–445, 2006

“What is the lineage of this idea?”

36

Buckingham Shum, S.J., Uren, V., Li, G., Sereno, B. and Mancini, C. (2007).Modelling Naturalistic Argumentation in Research Literatures: Representation and Interaction Design Issues. International Journal of Intelligent Systems, (Special Issue on Computational Models of Natural Argument, Eds: C. Reed and F. Grasso, 22, (1), pp.17-47. [PrePrint: http://oro.open.ac.uk/6463]

Current projects: scientific collective intelligence through discourse

  OLnet: Open Learning Network to connect the open educational resource movement’s discourse/evidence base: http://olnet.org

  ESSENCE: e-Science/Sensemaking/Climate Change testing and integrating Web argumentation tools: http://events.kmi.open.ac.uk/essence

  SocialLearn: Web 3.0 social learning/sensemaking platform with semantic discourse connections (launches end of year)

37

Marriage made in heaven?

Human annotation

38

Machine annotation

Workshop Qs:

  Corpus you are working on; community, type of content (abstracts, full-text, book..)full text:   scholarly/scientific, blogs, newspapers, real time

discussions (and video of it), mission doctrine/policy

  Granularity of knowledge element you are identifying   arbitrary: statements, single words, paragraphs

  Relationships between knowledge elements you have identified   IBIS: relational types + node types   ScholOnto: relations + roles...   Cohere 39

Workshop Qs:

  Type of annotation: automatic, manual, combination   manual annotation   partial automatic highlighting of text based on

Simone Teufel's work on Argumentative Zoning   Size of corpus you have annotated so far

  40 pages of blog debate   12 hours of video   distill 2 cm of policy docts into IBIS maps   several books in a literature   10-30 papers in a sample literature   30 articles on Iraq   5 days workshop discussions 40

Workshop Qs:

  Data standards, outline of architecture of system built (if relevant)   Compendium: XML DTD; SQL   Cohere API: RDF; XML; JSON   TopicSpaces: XML Topic Map; RDF; OWL

  Visualisations   Compendium/ClaiMapper manual maps   ClaiMaker/Cohere/TopicSpaces generated maps

41

Workshop Qs:

  User studies: yes, focusing on interaction design and usage patterns in both field trials and lab studies   IUI 2005: evaluation of ClaimSpotter   IJHCS 2006: evaluation of ClaiMaker   WWW'07 CKC: evaluation of ClaimSpotter   IJRME 2008: evaluation of Compendium for mapping

climate change arguments   Space Ex. Conf 2005: NASA Ames field trials   DIAC 2008: evaluation of Compendium for mapping

planning discourse   HCI (under review): evaluation of Compendium

mapping for hostage recovery 42