semantic scrolling how to navigate structured documents
TRANSCRIPT
In the World of Data• Data exists in multiple forms
INT
RO
DU
CT
ION
• Many ways of navigating data
• What is Semantic Scrolling?
A hypothetical example.• Let’s look at Twitter
Scrolling by pixels
Why can’t we scroll by tweets?
Or by time?
INT
RO
DU
CT
ION
Google Maps Directions.• Side Panel: a list of directions from A to B
Each item maps from a heading in the side panel to a geographic location
scroll semantically
INT
RO
DU
CT
ION
Microsoft Word Search.• Little known function of the navigation panel
Move through document by Graphics, Tables, Equations, etc.
scroll semantically
INT
RO
DU
CT
ION
How to categorize?• By the document structure?
INT
RO
DU
CT
ION
• By the mechanics?
rendering, storage etc.
Input, Interactive elements, etc.
1. The Data Structure
• Linear – one level of detail
DO
CU
ME
NT
ST
RU
CT
UR
E
• Hierarchical – multiple levels of detail
• Data Structure = how the data is stored
2. Rendering• Rendering = how the data is represented to the user for consumption
DO
CU
ME
NT
ST
RU
CT
UR
E
• Linear & Hierarchical applies here as well
ex. Linear data rendered linearly
ex. Hierarchical data rendered linearly
ex. Hierarchical data rendered hierarchically
ex. Linear data rendered hierarchically ex. Linear data rendered hierarchically
3. Exposed• Exposed = how the data is represented to the user for navigation
DO
CU
ME
NT
ST
RU
CT
UR
E
• Linear & Hierarchical applies here as well
• Note: How a document is Exposed can be independent from how it is Rendered
Putting it Together
DO
CU
ME
NT
ST
RU
CT
UR
E
Data Rendered Exposed Example
Data Rendered Exposed Example
Linear Linear Linear - Simple lists
Hierarchy - Lists w/ categorization
Data Rendered Exposed Example
Linear Linear Linear - Simple lists
Hierarchy - Lists w/ categorization
Hierarchy Linear Linear - List of nodes in tree (ordered?)
Hierarchy - eBooks table of contents
Data Rendered Exposed Example
Linear Linear Linear - Simple lists
Hierarchy - Lists w/ categorization
Hierarchy Linear Linear - List of nodes in tree (ordered?)
Hierarchy - eBooks table of contents
Hierarchy Linear - List of nodes in tree (ordered?)
Hierarchy - Semantic zooming, i.e. expand / collapse tree at a depth
1. Inputs• Touch:
ME
CH
AN
ICS
• Mouse:
tap, swipe, drag, pinch, etc.
click, drag, mouse wheel, secondary buttons
• Keyboard: (not really interesting)
arrow keys, hot keys
2. Targeting
up-down semantic movements in data (up/down level of detail)
ME
CH
AN
ICS
prev-next semantic movements in data (across same level of detail)
• Absolute movements in data (random access)
• Relative movements in data
3. Interaction Manifestation• Invisible – interaction is done to the content direction, no other feedback
ME
CH
AN
ICS
• Invoke – an element is invoked for the semantic scrolling to take place
• Persistent – a persistent element exists for semantic scrolling
Doc Structure & Mechanics
FR
AM
EW
OR
K
Data Rendered Exposed
Linear Linear Linear
Hierarchy
Hierarchy Linear Linear
Hierarchy
Hierarchy Linear
Hierarchy
• InputTouch, Mouse,
Keyboard
• TargetingRelative, Absolute
• Interaction Manifestation Invisible, Invoke,
Persistent
Size of the design space
FR
AM
EW
OR
K• Total Doc Structure categories = 6
• Total Mechanics categories = 18 Input = 3, Targeting = 2, IM = 3
• So, 108 categories in total?
at least 108 prototypes?
NOT EXACTLY
Focus• Certain categories are dull
FR
AM
EW
OR
K
• Certain combinations are dull
Keyboard inputs, Data storage structure, etc.
Ex. Keyboard & persistent/invoked
Ex. Touch & absolute movement
• Focus: Mechanics that compliment certain document structures
Demo #1 overview• Document structure
DE
MO
Data: Hierarchical
• Mechanics
Render: LinearExposed: Hierarchical
Input: Keyboard (hotkeys)Targeting: Relative Interaction Manifestation: Invisible
Demo #1
DE
MO
• Purpose:Show the backend scrolling algorithmDocument has 4 levels of detail (LOD)
Parts, Chapters, Sections, SubsectionsScrolling at a certain LOD will scroll through
the parts of the document at that LOD and higher, but not lower. Like a depth limited tree traversal Prevents jumps, gives context
Demo #2 overview• Document structure (same document)
DE
MO
Data: Hierarchical
• Mechanics
Render: LinearExposed: Hierarchical
Input: TouchTargeting: Relative Interaction Manifestation: Invisible
Demo #2a
DE
MO
• Purpose:Touch gesture for relative movement with 2
taps scrolling2 taps to right = more fine (higher LOD)2 taps to left = more coarse (lower LOD)
Demo #2b
DE
MO
• Purpose:Touch gesture for relative movement with 2
finger scrollingWidth of space between 2 fingers determines
LOD