decision-point panorama-based indoor navigation
DESCRIPTION
We present a novel user interface concept for indoor navigation which uses directional arrows and panorama images at decision points. The interface supports the mental model of landmark-based navigation, can be used on- and offline and is highly tolerant to localization inaccuracy.TRANSCRIPT
Decision-Point Panorama-Based Indoor Navigation
Andreas Möller, Matthias Kranz, Luis Roalter, Stefan Diewald, Kåre Synnes
MCPT Workshop on Eurocast 2013, February 15, 2013
Technische Universität München Institute for Media Technology Distributed Multimodal Information Processing Group
Overview
15.2.2013 A. Möller, M. Kranz, L. Roalter, S. Diewald, K. Synnes
Motivation: Vision-Based Localization
UI Concept and First Evaluation
Decision-Point Based Navigation
User Study
Discussion and Future Work
Technische Universität München Institute for Media Technology Distributed Multimodal Information Processing Group
Background and Motivation § Location information still the
most important contextual information § Indoor localization is a hot topic
and useful for a lot of locations: □ Airports □ Hospitals □ Conference venues □ Large environments
§ Various indoor localization methods possible: □ WLAN/cell-based localization □ Sensor-based localization □ Beacon-based localization □ Vision-based localization
15.2.2013 A. Möller, M. Kranz, L. Roalter, S. Diewald, K. Synnes
Technische Universität München Institute for Media Technology Distributed Multimodal Information Processing Group
Vision-Based Localization
§ Principle □ Comparing of query images, taken with
camera, to reference images based on visual features
§ Advantages □ No infrastructure and augmentation of the environment needed □ Immune to disturbances (reflection/refraction…) □ Works with existing
(modern) hardware
Live features
15.2.2013 A. Möller, M. Kranz, L. Roalter, S. Diewald, K. Synnes
Technische Universität München Institute for Media Technology Distributed Multimodal Information Processing Group
Overview
15.2.2013 A. Möller, M. Kranz, L. Roalter, S. Diewald, K. Synnes
Motivation: Vision-Based Localization
UI Concept and First Evaluation
Decision-Point Based Navigation
User Study
Discussion and Future Work
Technische Universität München Institute for Media Technology Distributed Multimodal Information Processing Group
HCI Perspective: Interaction Concept
§ Augmented Reality View for intuitiveness □ But: Consequence when location estimate
is inaccurate: wrong overlays!
§ Permanent re-localization during path required □ High computational effort □ Uncomfortable camera pose □ Not all scenes adequate for visual localization
(sometimes insufficient features for algorithm)
?
15.2.2013 A. Möller, M. Kranz, L. Roalter, S. Diewald, K. Synnes
Technische Universität München Institute for Media Technology Distributed Multimodal Information Processing Group
Interaction Concept (cont.)
Instead: § Re-localization from time to time
(when possible) □ Estimation of intermediate position
with odometry (device sensors) □ More comfortable usage
§ “Virtual Reality” View □ Regularly show 360° panorama images
of the environment with embedded navigation instructions (those panoramas are already available from reference images)
□ More robust: user matches virtual and real world
15.2.2013 A. Möller, M. Kranz, L. Roalter, S. Diewald, K. Synnes
Technische Universität München Institute for Media Technology Distributed Multimodal Information Processing Group
Initial User Study
§ Comparison of Augmented Reality and Virtual Reality
§ Simulation of different levels of accuracy
§ Survey of user preferences
15.2.2013 A. Möller, M. Kranz, L. Roalter, S. Diewald, K. Synnes
See Full Paper: A. Möller, M. Kranz, R. Huitl, S. Diewald, L. Roalter A Mobile Indoor Navigation System Interface Adapted to Vision-Based Localization In: 11th Intl. Conf. on Mobile and Ubiquitous Multimedia (MUM 2012)
Technische Universität München Institute for Media Technology Distributed Multimodal Information Processing Group
Initial User Study: Summary
§ Panoramas provide better guidance when location estimate is inaccurate than Augmented Reality
§ But: Subjects experienced frequent changes (~2m) as irritating, as □ Shown locations are not always correct □ Locations sometimes do not look like in reality □ Frequent changes in the UI are visually straining
§ Subjects manage finding their goal, also when not looking at the screen all the time
§ How to improve? □ Reduce frequent visual changes □ Simplify match between panoramas and real world (problem of
misplaced panoramas?) 15.2.2013 A. Möller, M. Kranz, L. Roalter, S. Diewald, K. Synnes
Technische Universität München Institute for Media Technology Distributed Multimodal Information Processing Group
Overview
15.2.2013 A. Möller, M. Kranz, L. Roalter, S. Diewald, K. Synnes
Motivation: Vision-Based Localization
UI Concept and First Evaluation
Decision-Point Based Navigation
User Study
Discussion and Future Work
Technische Universität München Institute for Media Technology Distributed Multimodal Information Processing Group
Decision-Point Based Navigation (DPBN)
§ Show only panoramas of decision points § Compliance with familiar mental model of self-orientation § Users can always re-localize and retrieve the panorama
of their current location § High error tolerance § Offline usage possible (by swiping through list of route instructions)
15.2.2013 A. Möller, M. Kranz, L. Roalter, S. Diewald, K. Synnes
Technische Universität München Institute for Media Technology Distributed Multimodal Information Processing Group
Decision-Point Based Navigation (DPBN)
§ Show only panoramas of decision points § Compliance with familiar mental model of self-orientation § Users can always re-localize and retrieve the panorama
of their current location § High error tolerance § Offline usage possible
15.2.2013 A. Möller, M. Kranz, L. Roalter, S. Diewald, K. Synnes
Technische Universität München Institute for Media Technology Distributed Multimodal Information Processing Group
Decision-Point Based Navigation (DPBN)
§ Show only panoramas of decision points § Compliance with familiar mental model of self-orientation § Users can always re-localize and retrieve the panorama
of their current location § High error tolerance § Offline usage possible
15.2.2013 A. Möller, M. Kranz, L. Roalter, S. Diewald, K. Synnes
Technische Universität München Institute for Media Technology Distributed Multimodal Information Processing Group
Decision-Point Based Navigation (DPBN)
§ Show only panoramas of decision points § Compliance with familiar mental model of self-orientation § Users can always re-localize and retrieve the panorama
of their current location § High error tolerance § Offline usage possible
15.2.2013 A. Möller, M. Kranz, L. Roalter, S. Diewald, K. Synnes
Technische Universität München Institute for Media Technology Distributed Multimodal Information Processing Group
Decision-Point Based Navigation (DPBN)
§ Show only panoramas of decision points § Compliance with familiar mental model of self-orientation § Users can always re-localize and retrieve the panorama
of their current location § High error tolerance § Offline usage possible
15.2.2013 A. Möller, M. Kranz, L. Roalter, S. Diewald, K. Synnes
Technische Universität München Institute for Media Technology Distributed Multimodal Information Processing Group
Decision-Point Based Navigation (DPBN)
§ Show only panoramas of decision points § Compliance with familiar mental model of self-orientation § Users can always re-localize and retrieve the panorama
of their current location § High error tolerance § Offline usage possible
15.2.2013 A. Möller, M. Kranz, L. Roalter, S. Diewald, K. Synnes
Technische Universität München Institute for Media Technology Distributed Multimodal Information Processing Group
Decision-Point Based Navigation (DPBN)
§ Show only panoramas of decision points § Compliance with familiar mental model of self-orientation § Users can always re-localize and retrieve the panorama
of their current location § High error tolerance § Offline usage possible
15.2.2013 A. Möller, M. Kranz, L. Roalter, S. Diewald, K. Synnes
Technische Universität München Institute for Media Technology Distributed Multimodal Information Processing Group
Overview
15.2.2013 A. Möller, M. Kranz, L. Roalter, S. Diewald, K. Synnes
Motivation: Vision-Based Localization
UI Concept and First Evaluation
Decision-Point Based Navigation
User Study
Discussion and Future Work
Technische Universität München Institute for Media Technology Distributed Multimodal Information Processing Group
User Study: Research Questions
§ RQ1: Does DPBN have an effect on efficiency? Are users as fast as with continuous panoramas?
§ RQ2: Is DPBN more convenient than continuous panoramas? What mode do users prefer? How well do they feel guided?
§ RQ3: What usage patterns can be identified? What do we learn for designing an “ideal” route description?
15.2.2013 A. Möller, M. Kranz, L. Roalter, S. Diewald, K. Synnes
Technische Universität München Institute for Media Technology Distributed Multimodal Information Processing Group
User Study – Quick Facts
3 conditions - Continuous panoramas (automatic *) - Decision points (automatic *) - Decision points (manual navigation)
3 routes (avoidance of learning effects) Measured data: time, user behavior, questionnaires 12 Participants, 75% male, 25% female, average 25 years Within-subjects design 75% own a smartphone, only one has experience with indoor navigation
15.2.2013 A. Möller, M. Kranz, L. Roalter, S. Diewald, K. Synnes
Source: iconsdb.com
* Wizard of Oz
Technische Universität München Institute for Media Technology Distributed Multimodal Information Processing Group
Results: RQ1
196 208 263
0
50
100
150
200
250
300
350
Continuous (automatic)
Decision-Point (automatic)
Decision-Point (manual)
Average Time per Mode
15.2.2013 A. Möller, M. Kranz, L. Roalter, S. Diewald, K. Synnes
Tim
e in
sec
onds
Difference not
significant
Technische Universität München Institute for Media Technology Distributed Multimodal Information Processing Group
Results: RQ2
15.2.2013 A. Möller, M. Kranz, L. Roalter, S. Diewald, K. Synnes
I found the method pleasing to use.
Continuous (automatic)
Decision-Point (automatic)
Decision-Point (manual)
5: fully agree 1: fully disagree
- Median + Mean
Technische Universität München Institute for Media Technology Distributed Multimodal Information Processing Group
I felt guided well to the goal.
Continuous (automatic)
Decision-Point (automatic)
Decision-Point (manual)
Results: RQ2
15.2.2013 A. Möller, M. Kranz, L. Roalter, S. Diewald, K. Synnes
5: fully agree 1: fully disagree
- Median + Mean
Technische Universität München Institute for Media Technology Distributed Multimodal Information Processing Group
Which system would you choose?
Results: RQ2
15.2.2013 A. Möller, M. Kranz, L. Roalter, S. Diewald, K. Synnes
75%
17%
8% Continuous (automatic) Decision-Point (automatic) Decision-Point (manual)
Panoramas of decision points are sufficient for
orientating
5: fully agree 1: fully disagree
Technische Universität München Institute for Media Technology Distributed Multimodal Information Processing Group
0
10
20
30
40
20 40 60 80 100 120 140 160 180 200 220 240 260 280 300 320 340
Dis
tan
ce (
m)
Time (s)
Participant 12 - Path 1
swipe relocalize Δd position
Results: RQ3
15.2.2013 A. Möller, M. Kranz, L. Roalter, S. Diewald, K. Synnes
Technische Universität München Institute for Media Technology Distributed Multimodal Information Processing Group
-60
-50
-40
-30
-20
-10
0
10
20
30
40
50
60
20 40 60 80 100 120 140 160 180 200 220 240 260 280 300 320 340
Dis
tance
(m
)
Time (s)
Participant 2 - Path 2
swipe relocalize Δd position
Results: RQ3
15.2.2013 A. Möller, M. Kranz, L. Roalter, S. Diewald, K. Synnes
Technische Universität München Institute for Media Technology Distributed Multimodal Information Processing Group
-20
-10
0
10
20
30
20 40 60 80 100 120 140 160 180 200 220
Dis
tan
ce (
m)
Time (s)
Participant 5 - Path 3
swipe relocalize Δd position
Results: RQ3
15.2.2013 A. Möller, M. Kranz, L. Roalter, S. Diewald, K. Synnes
Technische Universität München Institute for Media Technology Distributed Multimodal Information Processing Group
Results: RQ3 – Manual Mode
§ Average numbers □ 52.5 continuous locations (automatic, continuous mode) □ 10.3 decision points (automatic, decision-point mode) □ 15.3 manual swipes □ 9.3 times relocalized
§ Different usage patterns/strategies □ Showing “ahead” panoramas, no use of automatic re-localization □ Frequent use of re-localization, (sometimes only in parts of the
route) – equivalent to continuous panorama mode □ “Memorizing” of the entire route not observed, subjects only
looked at the “next” panorama location
15.2.2013 A. Möller, M. Kranz, L. Roalter, S. Diewald, K. Synnes
Technische Universität München Institute for Media Technology Distributed Multimodal Information Processing Group
Overview
15.2.2013 A. Möller, M. Kranz, L. Roalter, S. Diewald, K. Synnes
Motivation: Vision-Based Localization
UI Concept and First Evaluation
Decision-Point Based Navigation
User Study
Discussion and Future Work
Technische Universität München Institute for Media Technology Distributed Multimodal Information Processing Group
Discussion
§ Subjects were equally fast with DPBN as with continuous panoramas (RQ1)
§ Subjects felt guided better with continuous panoramas (RQ2) □ Due to more certainty whether they are on the right way □ Confirmed by frequent swipes and re-localizations in manual mode
§ DPBN is desirable from a technical point of view (reliability…), but acceptance must be increased
§ Requires stronger confirmation whether subjects are (still) on the right way
15.2.2013 A. Möller, M. Kranz, L. Roalter, S. Diewald, K. Synnes
Technische Universität München Institute for Media Technology Distributed Multimodal Information Processing Group
How could this be achieved? (Lessons learned) § Compromise between continuous mode and DPBN as it is now
§ Distance information § Verification when decision point was passed § Add intermediate landmarks in decision point mode □ Especially if decision point lies too far away (distance threshold)
§ Simplify the mapping of picture content to real environment □ Highlight significant objects in panorama
15.2.2013 A. Möller, M. Kranz, L. Roalter, S. Diewald, K. Synnes
Technische Universität München Institute for Media Technology Distributed Multimodal Information Processing Group
Summary and Outlook
§ With Decision-Point Based Navigation, we have presented a novel interface for visual indoor navigation □ Very robust in case of localization inaccuracy □ Works even if localization fails (manual mode) □ Supports users’ mental model of navigation (landmarks)
§ Real-world study results □ DPBN as efficient as full panorama navigation □ DPBN needs more information to reduce users’ uncertainty
(landmarks, confirmations) to increase satisfaction § Future work □ See previous slide ;-) □ Identification of good candidates for (additional) key points
15.2.2013 A. Möller, M. Kranz, L. Roalter, S. Diewald, K. Synnes
Technische Universität München Institute for Media Technology Distributed Multimodal Information Processing Group
Thank you for your attention! Questions?
www.vmi.ei.tum.de/team/andreas-moeller.html
This research project has been supported by the space agency of the German Aerospace Center with funds from the Federal Ministry of Economics and Technology on the basis of a resolution of the German Bundestag under the reference 50NA1107.
15.2.2013 A. Möller, M. Kranz, L. Roalter, S. Diewald, K. Synnes
Technische Universität München Institute for Media Technology Distributed Multimodal Information Processing Group
Paper Reference
§ Please find the associated paper at: https://vmi.lmt.ei.tum.de/publications/2013/MCPT2013-IndoorNav_preprint.pdf
§ Please cite this work as follows: § A. Möller, M. Kranz, L. Roalter, S. Diewald, Kåre Synnes
Decision-Point Panorama-Based Indoor Navigation In: 14th International Conference on Computer Aided Systems Theory (EUROCAST 2013), pp. 325-326, Las Palmas de Gran Canaria, Spain, February 2013
15.2.2013 A. Möller, M. Kranz, L. Roalter, S. Diewald, K. Synnes
Technische Universität München Institute for Media Technology Distributed Multimodal Information Processing Group
If you use BibTex, please use the following entry to cite this work:
15.2.2013 A. Möller, M. Kranz, L. Roalter, S. Diewald, K. Synnes
@INPROCEEDINGS{MCPT13IndoorNav, author = {Andreas M{\"o}ller and Matthias Kranz and Luis Roalter and Stefan Diewald}, title = {{Decision-Point Panorama-Based Indoor Navigation}}, booktitle = {14th International Conference on Computer Aided Systems Theory (EUROCAST 2013)}, editor = {Alexis Quesada-Arencibia and Jos\'{e} Carlos Rodriguez and Roberto Moreno-Diaz jr. and Roberto Moreno-Diaz}, year = {2013}, month = feb, pages = {325--326}, ISBN = {978-84-695-6971-9}, location = {Las Palmas de Gran Canaria, Spain}, }