evaluating indoor positioning errors - the errors of a positioning system are often presented in...

Copyright © 2013 Ekahau, Inc. All rights reserved.

Teemu Pulkkinen, Johannes Verwijnen, Ekahau OyICTC 2015, Oct 28, Jeju Island

Evaluating Indoor Positioning Errors

Copyright © 2013 Ekahau, Inc. All rights reserved. 2© Copyright 2014 Ekahau, Inc. 2

• Ekahau & WiFi Positioning

• Positioning errors

• Evaluation suggestions

• Q&A

Outline


• Spin-off from University of Helsinki Complex System Computation Group (2000)

• HQ in Reston, Virginia, R&D in Helsinki, Finland• 10 patent families for radio-based positioning• 1,400+ RTLS systems worldwide

– 20000+ positioning tags tracked– US healthcare as main market

• Two basic use cases– Asset tracking– Staff safety

About Ekahau


• The received signal strengths (RSS) of WiFi Access Points (APs) vary over locations

• The combination of RSSs measured is somewhat unique to a location

• Off-line phase:– Collect information on the RSSs observed at discrete locations

• On-line phase:– Scan for RSSs and compare to information collected during off-line

phase(Bahl & Padmanabhan, 2000)

• We use a probabilistic model for positioning and a particle filter to help tracking

WiFi Positioning in one minute


• [One-off] positioning– No prior information of positions

• Tracking– A continuous stream of positions

Positioning vs tracking


• The errors of a positioning system are often presented in measures of accuracy and precision as defined by Hightower & Borriello (2001)

• Accuracy: 3-4.3m, precision: 50% (MSR RADAR)• “MSR RADAR can locate positions to within 3-4.3m for

approximately 50 percent of measurements”

• Thus; accuracy is the target “granularity” and precision is the measurement of hitting this target

Accuracy and precision


• RADAR 3-4.3m 50%

• “Histogram method” 1.45m 50%, 1.81m 67%, 2.76m 90%(Roos et al, 2002)

• “Semi-supervised method” 1.5m 50%(Pulkkinen et al, 2011)

• “Dynamic radio method” 2m 75%(Wang et al, 2011)

• How does this compare to my 138-acre campus with over 15 multi-storey buildings? (Napa State Hospital)

Comparison of systems


• Positioning or tracking?– Often the end use case of the positioning system involves tracking,

however accuracy/precision only address one-off positioning

• Environment size– How does this error relate to the size of the environment?

• Sample bias– Positioning errors are often sampled uniformly over the environment,

whereas in the real use case positions are clustered

Relevance of errors


• Environment-normalization– Measuring the error as a proportion of the largest error possible in the

environment

• Shortest path– Using the shortest traversable path instead of the Euclidean distance.

• Zone/cell accuracy– End users are often more interested in

whether a location is inside a roomthan the precise co-ordinates

Normalization


• Nearest AP error– The position error compared to the error when positioning to the location

of the AP with the highest RSS– Measures the density of the WiFi network, which is relevant for WiFi

positioning

• Random/center error– The position error compared to either a random location, or the center

point of the environment.

Baselines


• For sequential estimates we’re more interested in the overall “shape” of the route than point errors

• Continuous environment-normalized error– Creating a polygon measuring the distance of estimated positions from

ground truth positions. The normalized size of this polygon is a score of accuracy.

• Movement vector error– Compute error using differences in estimated position movement

vectors compared to ground truth movement vectors to penalize routes that change direction frequently.

Tracking


• Dynamic Time Warping (DTW)– The distance of points along two routes– Time is “warped”, allowing for temporary

differences in speeds

• Line Segment Distance– As presented by Lee et al (2007)– Inspired by Hausdorff distance– Weighed sum of perpendicular, parallel and angle distances

More tracking

Picturecredit:Pulkkinen (2014)


Example


• ISO is working on a standard, ISO/IEC DIS 18305 “Information technology — Real time locating systems — Test and evaluation of localization and tracking systems”

• Unfortunately at its current state the standard does not address the problems identified earlier

Related


Questions?

Copyright © 2013 Ekahau, Inc. All rights reserved.© Copyright 2014 Ekahau, Inc.© Copyright 2015 Ekahau, Inc.

Ekahau East Coast(Headquarters)1851 Alexander Bell DriveSuite 105Reston, VA 20191Tel: 1-866-4EKAHAUFax: [email protected]

Ekahau EuropeHiilikatu 300180 Helsinki, FinlandTel: +358-20-743 5910Fax: +358-20-743 [email protected]


• This work was supported by the Finnish Funding Agency for Technology and Innovation (project D2I).

• Bahl, Paramvir, and Venkata N. Padmanabhan. "RADAR: An in-building RF-based user location and tracking system." INFOCOM 2000. Nineteenth Annual Joint Conference of the IEEE Computer and Communications Societies. Proceedings. IEEE. Vol. 2. Ieee, 2000.

• Hightower, Jeffrey, and Gaetano Borriello. "Location systems for ubiquitous computing." Computer 8 (2001): 57-66.

• Lee, Jae-Gil, Jiawei Han, and Kyu-Young Whang. "Trajectory clustering: a partition-and-group framework." Proceedings of the 2007 ACM SIGMOD international conference on Management of data. ACM, 2007.

• Pulkkinen, Teemu, Teemu Roos, and Petri Myllymäki. "Semi-supervised learning for wlanpositioning." Artificial Neural Networks and Machine Learning–ICANN 2011. Springer Berlin Heidelberg, 2011. 355-362.

• Roos, Teemu, et al. "A probabilistic approach to WLAN user location estimation." International Journal of Wireless Information Networks 9.3 (2002): 155-164.

• Wang, Huimin, et al. "Dynamic radio map construction for WLAN indoor location." Intelligent Human-Machine Systems and Cybernetics (IHMSC), 2011 International Conference on. Vol. 2. IEEE, 2011.

Acknowledgments & References

evaluating indoor positioning errors - the errors of a positioning system are often presented in...

Documents