Beyond Pencil and Paper

• Topics for today– Interactive texts– Integrating paper and digital

content

• What, where, and how do you read?

• Interviews with people indicated they tend to discount reading on the computer.– When asked if they read online they said no but ignored

email, web news, etc.– Do you think this has this changed? Why?

Interactive Texts

• Many forms of interactive text– Hypertext– Chatterbots– Computer-

generated texts– Social texts (email, etc.) are for next class so let’s ignore

them for now

• How do these forms of text change the connection between the author and reader?

• What are they good for?

Fluid Texts

• Stretchtext• Fine grained hypertext

structure– Differs from the web

• Novel animated text to maintain reader context– Differs from traditional hypertext where the new

lexia replaces (or opens in a separate window) from the prior lexia

• Authoring is still a challenge

Fluid Reader

Fluid Reader (2)

Fluid Reader (3)

Fluid Structure

Fluid Writer

Fluid Writer (2)

Character Development in Fluid Texts

Beyond Strings

Paper-Based Spatial Expression

Understanding Spatial Expression

Integrating Paper and Digital

• We see it all the time– Barcodes– QR codes– TAG codes

• Characteristics– Visible– Takes space from other content– No features of the link are human interpretable– May not fit into the aesthetic of linked content

Code-base Invisible Links

• We don’t see it all the time– RFID– Data glyphs– Anoto

• Characteristics– Rely on special hardware– Not human interpretable– Does not take space– Does not interfere with aesthetic

Paper++

Authoring in Paper++

Recap of Last Lecture

• Stretchtext – extensible/collapsible strings• Virtual Notebook System – integrated

reading/writing interface• Hyper-Object Substrate – object-oriented

authoring with proactive support for incremental formalization

• VIKI/VKB – expression via spatial/visual structure with system recognizing structures

• Visible and invisible links from paper to digital content

Links from Content

• Can we create links from standard paper content without augmented technology or visible marks?

• 1st possibility: Use the text (OCR)– What are the limitations?

• 2nd possibility: Treat as a photo– How to index?

The Big Issue

• Relying on a camera– Scale, skew, parallax

• Scale Invariant Feature Transform– Features that are resilient to different angles– Used for lots of

image recognition tasks

Links Using SIFT (or Similar) Features

• Links can be from text and image content but– Printed text is not good for distinctive SIFT

features– Could combine with lossy OCR

• Readers– Do not know where there is a link and where

there is not a link– Do not know the boundary of links

Embedded Media Markers (EMMs)

EMM Characteristics

• Printed as an layer on top of existing content– Does not take space but can impact aesthetics– Readers know where links are– Readers have some information about results of

taking a link• Challenges– Recognition still has problems of SIFT features– Need authoring support for unambiguous EMMs

EMM Authoring Tool

Merging Paper and Digital

• Older and Newer technologies– E Ink– Flexible OLED