ooms - cognitive user evaluation of digital maps: findings and challenges
DESCRIPTION
Presentation from Third InDOG Doctoral Conference in Olomouc, Czech Republic. 13. - 16. October 2014TRANSCRIPT
![Page 1: Ooms - Cognitive user evaluation of digital maps: findings and challenges](https://reader034.vdocuments.us/reader034/viewer/2022051514/548b491bb47959e3338b46ac/html5/thumbnails/1.jpg)
Cognitive user evaluation of digital maps: Findings and challenges
Kristien OomsCartoGIS Research Unit
Department of Geography Ghent University
![Page 2: Ooms - Cognitive user evaluation of digital maps: findings and challenges](https://reader034.vdocuments.us/reader034/viewer/2022051514/548b491bb47959e3338b46ac/html5/thumbnails/2.jpg)
Research Objectives
Contribute to the understanding of how map users read, interpret, store, and use the presented visual information on screen
maps.
AttentionRead
Store
Use
Understand
InfluenceImprovedesign
Retrieve
Interpret
![Page 3: Ooms - Cognitive user evaluation of digital maps: findings and challenges](https://reader034.vdocuments.us/reader034/viewer/2022051514/548b491bb47959e3338b46ac/html5/thumbnails/3.jpg)
Research Objectives
Contribute to the understanding of how map users read, interpret, store, and use the presented visual information on screen
maps.
After van Elzakker and Wealands (2007)
![Page 4: Ooms - Cognitive user evaluation of digital maps: findings and challenges](https://reader034.vdocuments.us/reader034/viewer/2022051514/548b491bb47959e3338b46ac/html5/thumbnails/4.jpg)
Research Objectives
• PhD: • Static maps or simulations
– controlled experiments:» simple map design (Part I)» complex realistic maps design (Part II)
• Current research:• Dynamic & interactive maps
– methodology? user logging– User tests
After van Elzakker and Wealands (2007)
![Page 5: Ooms - Cognitive user evaluation of digital maps: findings and challenges](https://reader034.vdocuments.us/reader034/viewer/2022051514/548b491bb47959e3338b46ac/html5/thumbnails/5.jpg)
Maps are …visual
Eye Tracking– Evaluate maps: UCD
• Log users’ Point of Regard– Location– Duration– …in screen-coordinates (px)
• Combination with other methods– Reaction time measurements– Thinking Aloud– Questionnaires– Sketch maps– ….
![Page 6: Ooms - Cognitive user evaluation of digital maps: findings and challenges](https://reader034.vdocuments.us/reader034/viewer/2022051514/548b491bb47959e3338b46ac/html5/thumbnails/6.jpg)
Part I - Basic Map Design
• Task:– Visual search
• Techniques:– Eye tracking– Reaction times– Questionnaire
• Analyses:– Statistical– Visual
![Page 7: Ooms - Cognitive user evaluation of digital maps: findings and challenges](https://reader034.vdocuments.us/reader034/viewer/2022051514/548b491bb47959e3338b46ac/html5/thumbnails/7.jpg)
Part I – Experts vs. novices
• Aims:– Study cognitive processes– Difference experts vs. novices?– Explain by Cognitive Load Theory
• Structure WM: Cognitive load• Influence of map design
– Content– Symbolisation
• Room for learning
![Page 8: Ooms - Cognitive user evaluation of digital maps: findings and challenges](https://reader034.vdocuments.us/reader034/viewer/2022051514/548b491bb47959e3338b46ac/html5/thumbnails/8.jpg)
Part I – Efficient and effective labels?
• Aims:– Evaluate different map designs
• Label placement algorithm– Improved efficiency– Lower map quality
– Influence on (novice) users?• Effectiveness of the map?
bord
er-d
esig
nto
tal-d
esig
nor
igin
al v
iew
![Page 9: Ooms - Cognitive user evaluation of digital maps: findings and challenges](https://reader034.vdocuments.us/reader034/viewer/2022051514/548b491bb47959e3338b46ac/html5/thumbnails/9.jpg)
Chapter 2
Part I – Quantitative Results
• Results:– Reaction time measurements– Fixation duration– Fixation count– Fixation distribution
![Page 10: Ooms - Cognitive user evaluation of digital maps: findings and challenges](https://reader034.vdocuments.us/reader034/viewer/2022051514/548b491bb47959e3338b46ac/html5/thumbnails/10.jpg)
Part I – Visual Analytics
• Time series• Aggregation• Simplification• Selection
![Page 11: Ooms - Cognitive user evaluation of digital maps: findings and challenges](https://reader034.vdocuments.us/reader034/viewer/2022051514/548b491bb47959e3338b46ac/html5/thumbnails/11.jpg)
Part I – Conclusion
• Conclusion– Experts significantly faster at locating the names
– No influence of label placement algorithm
Shorter fixations
More fixation per second
Locates the names faster
Can interpret a larger part of the map in the same amount of time
Interprets map more efficiently
Can interpret the map’s content more efficiently
![Page 12: Ooms - Cognitive user evaluation of digital maps: findings and challenges](https://reader034.vdocuments.us/reader034/viewer/2022051514/548b491bb47959e3338b46ac/html5/thumbnails/12.jpg)
Part II – Complex Map Design
• Task:– Study & draw
• Techniques:– Eye tracking– Thinking aloud– Sketch maps– Questionnaire
• Analyses:– Statistical– Visual
![Page 13: Ooms - Cognitive user evaluation of digital maps: findings and challenges](https://reader034.vdocuments.us/reader034/viewer/2022051514/548b491bb47959e3338b46ac/html5/thumbnails/13.jpg)
Part II – Research Objectives
• Communication process:– Cognitive structure
– Expertise?– Influence of deviations
SensoryInput
WorkingMemory
TransferredLong Term
Memory
Transferred
Retrieved
Virtually unlimited
Limited in -size -time (debate)
Using links, pointers with previous knowledge
![Page 14: Ooms - Cognitive user evaluation of digital maps: findings and challenges](https://reader034.vdocuments.us/reader034/viewer/2022051514/548b491bb47959e3338b46ac/html5/thumbnails/14.jpg)
Part II – Reading and Interpretation
• Eye movement:– Metrics
• Average fixation duration:Experts significantly shorter
• Number of fixations per secondExperts significantly more
Same findings as in previous studies
![Page 15: Ooms - Cognitive user evaluation of digital maps: findings and challenges](https://reader034.vdocuments.us/reader034/viewer/2022051514/548b491bb47959e3338b46ac/html5/thumbnails/15.jpg)
Part II – Reading and Interpretation
• Eye movements– Gridded visualisations
• Fixation count• Total dwell time• Average fixation duration
• Average per user group• Maximum per user group
Variable Classification and colour schemes
FixCount [0-1[ [1-2[ [2-4[ [4-6[ [6-8[ [8-10[ [10-20[ [20-...]
FixDur [.000-.325[
[.325-.650[
[.650-1.300[
[1.300-1.950[
[1.950-2.600[
[2.600-3.250[
[3.250-6.500[
[6.500-…......]
Colour (RGB) 255 247 217 189 150 99 37 0
![Page 16: Ooms - Cognitive user evaluation of digital maps: findings and challenges](https://reader034.vdocuments.us/reader034/viewer/2022051514/548b491bb47959e3338b46ac/html5/thumbnails/16.jpg)
Part II – Reading and Interpretation
• Eye movements– 2D gridded visualisations
Average fixation count
Maximum fixation count
![Page 17: Ooms - Cognitive user evaluation of digital maps: findings and challenges](https://reader034.vdocuments.us/reader034/viewer/2022051514/548b491bb47959e3338b46ac/html5/thumbnails/17.jpg)
Part II – Reading and Interpretation
• Eye movements– 3D gridded visualisation
Average total fixation duration
Average fixation duration per fixation
![Page 18: Ooms - Cognitive user evaluation of digital maps: findings and challenges](https://reader034.vdocuments.us/reader034/viewer/2022051514/548b491bb47959e3338b46ac/html5/thumbnails/18.jpg)
Part II – Reading and Interpretation
• Eye movements– Gridded visualisation: statistical
comparisonStatistical comparison (ANOVA)
![Page 19: Ooms - Cognitive user evaluation of digital maps: findings and challenges](https://reader034.vdocuments.us/reader034/viewer/2022051514/548b491bb47959e3338b46ac/html5/thumbnails/19.jpg)
Part II – Reading and Interpretation
• Eye movements– Scanpaths
![Page 20: Ooms - Cognitive user evaluation of digital maps: findings and challenges](https://reader034.vdocuments.us/reader034/viewer/2022051514/548b491bb47959e3338b46ac/html5/thumbnails/20.jpg)
Part II – Reading and Interpretation
• Eye movements– Conclusion
• Focus on general structuring elements– Experts: more pronounced– Experts fixate more on the left side
• Influence of deviations– No influence for less important elements– Confusion for structuring elements
» Colour water bodies» Mirrored map elements
– Novices: more pronounced
![Page 21: Ooms - Cognitive user evaluation of digital maps: findings and challenges](https://reader034.vdocuments.us/reader034/viewer/2022051514/548b491bb47959e3338b46ac/html5/thumbnails/21.jpg)
Part II – Cognition and Memory
• Thinking aloud– Word segmentation (count in ‰)
Based on theme Based on frequency
![Page 22: Ooms - Cognitive user evaluation of digital maps: findings and challenges](https://reader034.vdocuments.us/reader034/viewer/2022051514/548b491bb47959e3338b46ac/html5/thumbnails/22.jpg)
Part II – Cognition and Memory
• Thinking aloud– ‘Full thought’
• 4 Levels of codes:Level 1: Map Level
Orientate – Execute - EvaluateLevel 2: Item Level
Gather Thougts – Draw – Correct - EvaluateLevel 3: Confidence
Confident – Neutral – Not ConfidentLevel 4: Actions
Check – Correct – Draw – Erase – Fill Colour – Talk – Take Pencil
• Time ratio for each code: [0-1]
PsychologicalTheories
Task Analysis
Psychological Model
Proposed Codes
CodingScheme
Segmented Protocols
Transcriptions(Raw Protocols)
Coded Protocols
T H E O R Y
USERDATA
![Page 23: Ooms - Cognitive user evaluation of digital maps: findings and challenges](https://reader034.vdocuments.us/reader034/viewer/2022051514/548b491bb47959e3338b46ac/html5/thumbnails/23.jpg)
Part II – Cognition and Memory
• Sketch maps– Order of drawing
– Scores on maps
• Questionnaire– Stated confidence
![Page 24: Ooms - Cognitive user evaluation of digital maps: findings and challenges](https://reader034.vdocuments.us/reader034/viewer/2022051514/548b491bb47959e3338b46ac/html5/thumbnails/24.jpg)
Part II – Cognition and Memory
• Conclusion– General structures: similar
• Novices: store more information– Descriptions, locations, etc.– No extra knowledge– Not derive extra information
• Experts: can retrieve more information– Know objects’ names – Background information
» Derive information– Larger chunks in WM
![Page 25: Ooms - Cognitive user evaluation of digital maps: findings and challenges](https://reader034.vdocuments.us/reader034/viewer/2022051514/548b491bb47959e3338b46ac/html5/thumbnails/25.jpg)
More [email protected]
1. Ooms, K., De Maeyer, P., Fack, V., Van Assche, E., & Witlox, F. (2012). Interpreting maps through the eyes of expert and novice users. International Journal of Geographical Information Science, 26 (10), 1773-1788.
2. Ooms, K., Andrienko, G., Andrienko, N., De Maeyer, P., & Fack, V. (2012). Analysing the spatial dimension of eye movement data using a visual analytic approach. Expert Systems with Applications, 39(1), 1324-1332.
3. Ooms, K., De Maeyer, P., Fack, V., Van Assche, E., & Witlox, F. (2012). Investigation the effectiveness of an efficient label placement method using eye movement data. The Cartographic Journal, 49(3), 234-246.
4. Ooms, K., De Maeyer, P., & Fack, V. (2014). Understanding expert and novice map users: Reading and interpretation. Cartography and Geographic Information Science, 41(1), 37-54.
5. Ooms, K., De Maeyer, P., & Fack, V. (in press). Listen to the map user: Cognition, memory, and expertise. The Cartographic Journal.
![Page 26: Ooms - Cognitive user evaluation of digital maps: findings and challenges](https://reader034.vdocuments.us/reader034/viewer/2022051514/548b491bb47959e3338b46ac/html5/thumbnails/26.jpg)
Maps are …interactive
– ‘Maps on the Internet/Web’– Typical user interactions
• Panning changing extent
• Zooming changing scale & extent
– Influence on users’ cognitive processes?
Read
Interpet
Store
Retrieve
Benifical for user?e.g. memory, change blindness, …
![Page 27: Ooms - Cognitive user evaluation of digital maps: findings and challenges](https://reader034.vdocuments.us/reader034/viewer/2022051514/548b491bb47959e3338b46ac/html5/thumbnails/27.jpg)
Eye Tracking & Interactivity?
• Level of experimental control…– High: simulations of interactions
same stimuli high comparability easy to analyse
– Low: free interactions different stimuli low comparability difficult to analyse
At certain timestamp: - different scale - different extent …for each participant
Less intrusion on cognitive processes Higher realism
… vs. ecological validity
![Page 28: Ooms - Cognitive user evaluation of digital maps: findings and challenges](https://reader034.vdocuments.us/reader034/viewer/2022051514/548b491bb47959e3338b46ac/html5/thumbnails/28.jpg)
Logging Interactivity?
• Mouse actions– Mouse down – Mouse up – Scroll wheel – Time stamp & location (x and y in px)
– Eye Tracking software– Existing tools– APIs web mapping software– Javascript, AJAX, PHP, SQL, DB (with proxy server)– Desktop tool
based on JAVA: JNativeHook based on Python: PyHook
Panning Zooming
![Page 29: Ooms - Cognitive user evaluation of digital maps: findings and challenges](https://reader034.vdocuments.us/reader034/viewer/2022051514/548b491bb47959e3338b46ac/html5/thumbnails/29.jpg)
Eye Tracking & Interactivity?
• Free interactions
Changing point of origin
Applying map projection formula
Spherical Mercator(inverse)
![Page 30: Ooms - Cognitive user evaluation of digital maps: findings and challenges](https://reader034.vdocuments.us/reader034/viewer/2022051514/548b491bb47959e3338b46ac/html5/thumbnails/30.jpg)
Evaluation of panning in Google Maps
• Panning along a route– Count intersections– Zoom level 13– Alteration map - satellite view
![Page 31: Ooms - Cognitive user evaluation of digital maps: findings and challenges](https://reader034.vdocuments.us/reader034/viewer/2022051514/548b491bb47959e3338b46ac/html5/thumbnails/31.jpg)
Evaluation of panning in Google Maps
• Find Belgium– Zoom level 7– Alteration map - satellite view
Start: Fiji Start: Quttinirpaaq NP Canada
![Page 32: Ooms - Cognitive user evaluation of digital maps: findings and challenges](https://reader034.vdocuments.us/reader034/viewer/2022051514/548b491bb47959e3338b46ac/html5/thumbnails/32.jpg)
Preliminary Results
![Page 33: Ooms - Cognitive user evaluation of digital maps: findings and challenges](https://reader034.vdocuments.us/reader034/viewer/2022051514/548b491bb47959e3338b46ac/html5/thumbnails/33.jpg)
Future Work
• Zooming?– In theory: same concept, only change in R value– Logging change in zoom levels
• Scroll wheel…
• Other map projections?– In theory: same concept, only change in map projection
formula– Example: Google Earth
• Spherical General Perspective Azimuthal projection
![Page 34: Ooms - Cognitive user evaluation of digital maps: findings and challenges](https://reader034.vdocuments.us/reader034/viewer/2022051514/548b491bb47959e3338b46ac/html5/thumbnails/34.jpg)
Thank you for your attention!
Questions?Kristien.Ooms@UGe
nt.be