a mobile indoor navigation system interface adapted to vision-based localization

A Mobile Indoor Navigation System Interface Adapted to Vision-Based Localization

Andreas Möller1, Matthias Kranz2, Robert Huitl1, Stefan Diewald1, Luis Roalter1

1Technische Universität München, Germany2Luleå University of Technology, Department of Computer Science,

Electrical and Space Engineering, Luleå, Sweden

MUM 2012, Ulm, Germany

Technische Universität MünchenInstitute for Media TechnologyDistributed Multimodal Information Processing Group

04.12.2012 A. Möller, M. Kranz, R. Huitl, S. Diewald, L. Roalter

Outline

§ Motivation for Vision-Based Navigation§ Challenges§ Proposed User Interface Concepts§ Evaluation§ Discussion and Conclusion

2



Background and Motivation§ Location information still the

most important contextual information§ Indoor localization is a hot topic

and useful for a lot of scenarios:□ Airports□ Hospitals□ Conference venues□ Large environments

§ Various indoor localization methods possible:□ WLAN/cell-based localization□ Sensor-based localization□ Beacon-based localization□ Vision-based localization

3



Why vision-based localization?

§ Compare query image to reference data set

§ Use existing features in the environment

§ No additional augmentation needed

§ Modern devices are equipped with a camera

and powerful processor

4

Live features



Challenges of Vision-Based Localization

§ Query images need to be distinctive□ Repeating structures throughout the environment (e.g. corridors)□ Little visual features (e.g. uniform walls)

§ Orientation of the user’s smartphone□ Good candidates for query images (windows, adverts, posters) are

typically found in eye height □ In a typical pose, user holds the device in a way that it points at

the floor

§ User interface needs to address those challenges

5



User Interface Concept

§ No 1 Challenge: Localization inaccuracy□ When insufficient number of features

in the query image

§ Solution A: Adaption of the user interface□ Augmented Reality (AR) with a live view of the environment□ Virtual Reality (VR) with panorama images

(compare to Google Street View)taken in advance

§ Solution B: Corrective actions□ The user needs to point at interesting regions with the camera

6

Ourcontribution: Address problem from a UI perspective



Augmented Reality (AR)

• Display the live camera image

• Impose navigation instructions over the image

• Smartphone held ineye height

• Direct match of information and the real environment

7


04.12.2012 A. Möller, M. Kranz, R. Huitl, S. Diewald, L. Roalter 8

Virtual Reality (VR)

• Uses preloaded panorama images

• Navigation instructions are drawn in panorama

• Smartphone can be directed to the ground

• User matches environment and virtual reality


04.12.2012 A. Möller, M. Kranz, R. Huitl, S. Diewald, L. Roalter 9

Pointing Towards Discriminative Areas



Highlighting Objects of Interest

10



Research Questions

§ RQ1: Which concept (AR or VR) is preferable in terms of perceived accuracy?

§ RQ2: Which concept (AR or VR) is preferred by users?

§ RQ3: Which visualizations are appropriate to acquire sufficient visual features?

§ RQ4: Can object highlighting be improved with a soft border visualization?

11



Methodology

§ Online Survey§ Videos and images of mockup system§ 81 participants (18-59 years) recruited via Mobileworks

12



RQ1: Perceived accuracy

§ Simulated error conditions□ No Error□ Location Error

□ Orientation Error□ Both Errors

13

No Error Orientation Error Location Error Loc.+Ori. ErrorResponse Std.Dev. Response Std.Dev. Response Std.Dev. Response Std.Dev.

2.5 0.9 0.8 2.0 1.7 1.5 0.6 2.01.7 1.6 1.4 1.8 1.4 1.8 1.0 1.72.4 1.0 0.2 2.1 1.2 1.8 0.4 2.11.7 1.5 1.1 1.8 1.3 1.7 0.9 1.82.3 1.0 -0.2 2.0 0.4 1.9 -0.5 1.91.8 1.4 1.4 1.6 1.4 1.7 0.9 1.8

Augmented RealityVirtual Reality


2.5 0.9 0.8 2.0 1.7 1.5 0.6 2.01.7 1.6 1.4 1.8 1.4 1.8 1.0 1.72.4 1.0 0.2 2.1 1.2 1.8 0.4 2.11.7 1.5 1.1 1.8 1.3 1.7 0.9 1.82.3 1.0 -0.2 2.0 0.4 1.9 -0.5 1.91.8 1.4 1.4 1.6 1.4 1.7 0.9 1.8

Augmented Reality 1: The system seemed to know well my location.2: The system seemed to know well my orientation.

Virtual Reality 3: I perceived the navigation instructions as correct.


2.3 1.0 -0.2 2.0 0.4 1.9 -0.5 1.91.8 1.4 1.4 1.6 1.4 1.7 0.9 1.8

Augmented Reality

Virtual Reality

Perceived Accuracy of Virtual Reality and Augmented Reality Views

my orientation.I perceived the navigation instructions as correct.


The system seemed to know well my location.The system seemed to know well

3:


1:

2:

-3: strongly disagree+3: strongly agree



RQ2: User Preferences

§ In total, AR more popular with users than VR

14




RQ3: Feature Indicator

§ Clarity of the indicator‘s meaning

a)b)c)d)

15

a) b)

c) d)




RQ4: Highlightingmethods

§ Objects of interest highlighting□ Potentially feature-rich□ Interaction points for

location-based services§ Two types□ Frame□ Soft border

§ Both equally raise attention, but Frame distracts more

16




Discussion

§ Accuracy perception through the interface□ VR beneficial for lower localization accuracy□ AR preferred for reliable estimate

§ Feature indicators and object highlights potentially contribute to good reference images

§ Situational use□ VR for 45° angle□ Less features visible□ Less accuracy required

□ AR when holding the phone up□ Target to visual features□ Highlight objects of interaction

17



Future andOngoing Work§ Evaluating interfaces in the

real world§ Modeling the system’s

and user’s state□ Localization accuracy□ Location/environment□ User context□ User mental model□ …

18



Thank you for your attention!Questions?

??

19

[email protected]/team/andreas-moeller.html

This research project has been supported by the space agency of the German Aerospace Center with funds from the Federal Ministry of Economics and Technology on the basis of a resolution of the German Bundestag under the reference 50NA1107.



Paper References

§ Please find the full paper at:http://dx.doi.org/10.1145/2406367.2406372

§ Please cite this work as follows:§ Andreas Möller, Matthias Kranz, Robert Huitl, Stefan Diewald, and Luis

Roalter. 2012. A mobile indoor navigation system interface adapted to vision-based localization. In Proceedings of the 11th International Conference on Mobile and Ubiquitous Multimedia (MUM '12). ACM, New York, NY, USA, , Article 4 , 10 pages. DOI=10.1145/2406367.2406372 http://doi.acm.org/10.1145/2406367.2406372

§

20

http://dx.doi.org/10.1145/2406367.2406372

http://dx.doi.org/10.1145/2406367.2406372

http://doi.acm.org/10.1145/2406367.2406372

http://doi.acm.org/10.1145/2406367.2406372



Please use the following BibTex file:§ @inproceedings{Moller:2012:MIN:2406367.2406372, author = {M\"{o}ller, Andreas and Kranz, Matthias and Huitl, Robert and Diewald, Stefan and Roalter, Luis}, title = {A mobile indoor navigation system interface adapted to vision-based localization}, booktitle = {Proceedings of the 11th International Conference on Mobile and Ubiquitous Multimedia}, series = {MUM '12}, year = {2012}, isbn = {978-1-4503-1815-0}, location = {Ulm, Germany}, pages = {4:1--4:10}, articleno = {4}, numpages = {10}, url = {http://doi.acm.org/10.1145/2406367.2406372}, doi = {10.1145/2406367.2406372}, acmid = {2406372}, publisher = {ACM}, address = {New York, NY, USA}, keywords = {augmented reality, indoor navigation, user interface, virtual reality, vision-based localization},}

[21

http://doi.acm.org/10.1145/2406367.2406372%7D

http://doi.acm.org/10.1145/2406367.2406372%7D

a mobile indoor navigation system interface adapted to vision-based localization

Technology

localization inaccuracyution

floor user interface

live features

query images windows

live camera image

image smartphone

live view

panorama images