user!and!design!perspectives!of!mobile!augmented!reality …
TRANSCRIPT
USER AND DESIGN PERSPECTIVES OF MOBILE AUGMENTED REALITY
A THESIS
SUBMITTED TO THE GRADUATE SCHOOL
IN PARTIAL FULFILLMENT OF THE REQUIREMENTS FOR THE DEGREE
MASTER OF ARTS BY
DANIEL COOPER
BALL STATE UNIVERSITY
ADVISOR: JENNIFER GEORGE-‐PALILONIS
MAY 2011
ii
ABSTRACT
Augmented Reality is one of the hottest trends in mobile media, yet there is a
substantial lack of user studies within this field of research. The purpose of this
study is to evaluate the value of mobile augmented reality as a vehicle for
information delivery on the basis of a usability and design analysis. Using a multi-‐
dimensional qualitative method, this study examines the impressions of users and
designers regarding mobile augmented reality, along with a heuristic evaluation of
select mobile applications. This analysis finds that (1) users believe mobile
augmented reality offers great promise as a medium for visual communication, and
(2) it’s current execution is limited by technical restraints, design flaws and a lack of
compelling content. It is concluded that mobile augmented reality will one day
revolutionize how consumers engage geographical and time-‐based information. Yet,
like all new technologies, mobile augmented reality needs time to mature in order to
address these limitations. Finally, based on the findings of this study, this thesis
offers suggested guidelines for future developments in this medium.
iii
ACKNOWLEDGEMENTS
A great deal of thanks is due to my thesis committee. My advisor, Jennifer
George-‐Palilonis provided invaluable guidance and support toward the completion
of this study. Her passion for excellence and unwavering support she shows toward
her students has profoundly influenced my professional career. I would also like to
thank James Chesebro and Michael Holmes who routinely challenged me to examine
my work with acute criticism and curiosity. Their constructive feedback was both
invaluable and enlightening.
Special thanks is also due to my colleagues at the Center for Media Design,
particularly Michelle Prieb and Jennifer Milks who graciously allowed me the
opportunity to bounce methodological ideas off them when this study first took
shape. Thanks are also due to Angie Faller and Shawna Pierson for assisting with the
videography and documentation of the focus groups and field research.
Finally, I thank my participants for taking the time to share their experiences.
iv
TABLE OF CONTENTS
Abstract ......................................................................................................................................................... ii Acknowledgements ................................................................................................................................. iii Chapter 1: Introduction .......................................................................................................................... 1
Defining Augmented Reality .................................................................................................. 5 Research Questions ................................................................................................................ 11 Significance to the Field ........................................................................................................ 11
Chapter 2: Review of the Literature ............................................................................................... 12
History .......................................................................................................................................... 13 Consumer Mobile Augmented Reality Applications .................................................. 15 Development Concerns and Limitations ........................................................................ 17 The current state of user research and evaluations ................................................. 20 A potential solution ................................................................................................................. 21
Chapter 3: Research Methods ........................................................................................................... 26
Data Collection and Procedures ......................................................................................... 31 Ethical Considerations .......................................................................................................... 34
Chapter 4: Results .................................................................................................................................. 35
First Impressions ...................................................................................................................... 36 User Experiences and Feedback ........................................................................................ 41 Heuristic Evaluations .............................................................................................................. 47
Chapter 5: Discussion ........................................................................................................................... 58
Guidelines for Future Mobile Augmented Reality Applications .......................... 63 Limitations ................................................................................................................................. 65 Recommendations for Future Research ........................................................................ 67
References ................................................................................................................................................. 71
Appendix A: Focus Group Discussion Guide Protocol ............................................................ 75 Appendix B: Designer Perceptions Questionnaire .................................................................. 77 Appendix C: Heuristic Evaluation Protocol ............................................................................... 78
CHAPTER 1: INTRODUCTION
Mobile augmented reality is a disruptive technology that is redefining how
people perceive data by transforming the visual environment into an immersive,
information-‐based ecosystem. By blending our world with virtuality, augmented
reality can enhance the perception of one’s personal space by overlaying digital
information in the form of text, video, audio and other forms of imagery.
Geographical and time-‐based information has been made relevant to the unique
perspective of individual users, simply through the push of a button on one’s cell
phone.
However, in order to realize the full potential of mobile augmented reality,
there is fundamental need for understanding how users engage and perceive
augmented reality content. Most research within the field is currently focused on
the technological state of the art, in contrast to substantial user studies.
Furthermore, a valuable means for evaluating the effectiveness of mobile
augmented reality applications ability to deliver information in a clear and concise
way is also required.
To that end, a concise understanding of how information should be designed
for an augmented reality system is paramount for creating a desirable user
experience. If augmented reality is to become a commercial success, designers and
developers alike need to understand the needs of their user bases and establish
2
fundamental design principles in order to prove value for displaying information
over more traditional means.
This thesis posits that a usability and design analysis of augmented reality is
both feasible and useful, and perhaps provides a conception of value for mobile
augmented reality as a vehicle for the visual communication of personally relevant
information.
Statement of the Problem
Mobile Augmented Reality is struggling to find its niche. Marketers are
banking on its hype by fusing this new technology with trading cards, cereal boxes
and movie posters while dozens of mobile developers are investing on their hopes
that augmented reality app will become the “next big thing” (Hurley, 2009; Husson,
2010). Yet, mobile augmented reality has many problems to overcome if it is to
become a commercial success. Technical issues such as slow network speeds and
information display errors continue to plague users (Haller, Billinghurst & Thomas,
p. 26). It also remains to be seen how augmented reality can be used to solve real-‐
world problems rather than serve simply as a marketing gimmick. If solutions to
technical constraints and practical benefits for consumers are not realized,
augmented reality could meet the same fate as other overhyped technologies, such
as virtual reality worlds like Second Life (Wen, 2010). While augmented reality may
be “next big thing” just as virtual worlds once were, tomorrow it could fade into
obscurity due to a lack of consumer interest.
3
Nevertheless, the commercial sector sees potential business in augmented
reality. According to market projections by Juniper Research, revenues from mobile
augmented reality applications will reach $732 million by 2014. However, the
current user base consists primarily of “early adopters,” a demographic consisting of
“trendsetters” who are eager to try out new technologies before their peers, despite
existing limitations (Rogers, 1962). Additionally, this user pool is rather small
considering the number of existing smartphone users. According to Mashable.com,
of the 83% of Americans who own a cell phone, only 25% are smartphone users
(Schroder, 2010). Furthermore, according to research by Nielson, a third of all smart
phone users do not subscribe to a data plan, which is a necessary requirement for
operating augmented reality based apps (Nielson, 2010).
Furthermore, the most popular augmented reality apps have only realized
limited is distribution. As of 2010, the most downloaded augmented reality app,
Layar has a user base of one million active users. Meaio’s Junaio is a distant second,
having been downloaded only half a million times (Bryne, 2010).
Not only are there a limited number of users; we also know little about them.
Most research within the industry is focused on the advancement of augmented
reality technologies and software applications, rather than on the user experience.
In academia, understanding the problem of how users perceive augmented reality
content has been virtually ignored. As of 2010, only a handful of usability tests have
been conducted, most of which have been informal. (Schmalstieg et al.) Nor have the
4
opinions of users on the desirability and viability of existing mobile augmented
reality applications been addressed.
This lack of user research, combined with technological limitations of current
augmented reality systems, is providing unique challenges for content designers and
developers alike. Peter Meier, CTO of Metaio explained in a 2010 interview with
Veture Beat, that current user experience needs improvement and a greater
emphasis must be placed on design. Claire Boonstra, co-‐founder of Layar, also tells
Venture Beat that mobile augmented reality is moving from “functional AR to
experience AR.” If the mobile augmented reality industry is to move forward,
developers must convert the existing hype into engaging user experiences for larger
audiences (Bryne, 2010).
Moving forward, mobile augmented reality’s greatest strength over prior
digital technologies is its ability to localize geographic and time-‐based information
within the perspective of the user’s personal environment (Rutledge, 2010). Yet,
most companies are struggling to make this work (Husson, 2010). In addition to a
lack of user research, there are also few options for evaluating the ability of mobile
augmented reality applications to display information effectively (Schmalstieg, et
al.). Effective information design is increasingly important when users will navigate
a complex information ecosystem (Baer, 2008).
5
Defining Augmented Reality
Unlike virtual reality, augmented reality attempts to enhance one’s personal
environment, rather than replace it. By employing a technological lens that allows a
user to see and interact with the information that surrounds us, mobile augmented
reality allows users to interact with data on a more personal level. New technologies
and applications are rapidly emerging in the areas of information search,
entertainment, gaming and location-‐based services such as tourism and directional
mapping.
Defining augmented reality: Although many researchers have broadened the
definition and scope of augmented reality, the most commonly accepted defining
criteria were conceived by Ronald Azuma in 1997 (Zhou, 2008). Azuma states that
augmented reality systems share the following three characteristics:
1. Combines real and virtual objects in the real world.
2. Possesses interactivity and is presented in real-‐time.
3. Registers and aligns virtual and physical objects with each other in 3-‐D.
The rationale for these criteria is to avoid limiting augmented reality to specific
technologies. Prior to Azuma, researchers had primarily defined augmented reality
through the use of head-‐mounted displays, an approach unable to distinguish some
AR and VR applications. These new criteria expand the applications of augmented
reality to handheld mobile devices such as smartphones and monitor-‐based systems
that read specialized coded tags that can present augmented reality information on
a stationary computer screen (Azuma, 1997).
6
Augmented reality vs. virtual reality: Augmented reality is often looked upon
as a “middle ground” between virtual reality and the physical environment
(Milgram, 1994). In contrast to augmented reality and the physical world, the realm
of virtual reality lies entirely within the synthetic. Virtual reality can be defined as,
“a computer generated, interactive, three-‐dimensional environment in which a
person is immersed” (Aukstakalnis & Blatner, 1992). Virtual realities, such as those
found in computer games like World of Warcraft and Second Life, are experienced by
the player through control of an avatar in a computer-‐generated environment
presented on a computer screen. Users may also experience virtual environments by
wearing head-‐mounted displays such as those are commonly used by the military
for training purposes.
The fundamental difference between virtual and augmented realities lies in
the user perspective and immersion methods employed by each system. Virtual
reality systems strive to be a completely immersive experience, similar to the level
of immersion users experience with everyday living. In contrast, augmented reality
distinguishes itself as a “mixed reality” between two worlds by blending virtual
elements within a real-‐world environment (Milgram, 1994).
Paul Milgram expanded upon this concept with his “Reality-‐Virtuality
continuum” (Figure 1). Milgram argues that augmented reality is a “mixed reality”,
sharing characteristics of both real and virtual environments. Milgram’s “virtuality
continuum” visualizes the degree of user immersion and classes of objects displayed
within a specific reality. For example, a user in a virtual world is completely
7
immersed within an artificial environment consisting entirely of synthetic artifacts,
while an augmented reality environment mixes the real world with virtual elements.
In contrast, augmented virtuality is the merging of real world objects into virtual
environments (Milgram, 1994).
Figure 1. Milgram’s reality-‐virtuality continuum model.
Mobile augmented reality: A typical mobile augmented reality system
presents virtual elements to the user by overlaying data and visuals over a real
environment by employing either a handheld or wearable display device equipped
with a camera and position/orientation tracking system. The two most common
augmented reality display technologies are head-‐worn devices and smart phones.
A HWD provides the most immersive augmented reality experience, since the
visual information is directly tied to the view of the user. The device can be as small
as a pair of sunglasses, which are available from established electronic companies
such as Minolta and Sony. HWD devices can display augmented reality visuals in two
ways: through a mounted video display, or through an opaque AR overlay while the
user can still see his surroundings through a transparent display device (Azuma
2001).
Smart phones work along the same principles as HWDs but they are held
instead of worn. Typically, the user points the device in the direction of an item of
Usability Testing of Augmented / Mixed Reality Systems
Mark Billinghurst [email protected]
1 Introduction For many decades researchers have been trying to blend Reality and Virtual Reality in interesting ways to create intuitive computer interfaces. For example, in the area of Tangible Interfaces real objects are used as interface widgets [Ishii 97], researchers in Augmented Reality (AR) overlay three-dimensional virtual imagery onto the real world [Feiner 93], while in Virtual Reality (VR) interfaces the real world is replaced entirely with a computer-generated environment.
As Milgram points out [Milgram 94], these types of computer interfaces can be placed along a continuum according to how much of the users environment is computer generated (figure 1). On this Reality-Virtuality line Tangible Interfaces lie far to the left, immersive Virtual Environments are placed at the rightmost extreme, while Augmented Reality and Augmented Virtuality interfaces occupy the middle ground. To encompass this broad range of interfaces Milgram coined the term “Mixed Reality”.
Figure 1: Milgram’s Reality-Virtuality Continuum In this section of the tutorial we describe factors that should be taken into account when developing Mixed Reality usability studies and promising areas of research where usability studies could be conducted. As the previous presenters have shown there has been a wide range of user studies conducted in immersive Virtual Reality. However, there have been fewer experiments presented in the wider context of Mixed or Augmented Reality. It is in the areas outside of immersive VR on the Reality-Virtuality continuum that we are interested in, and particularly in Augmented Reality interfaces. In many ways AR interfaces may have more near term application, so it is important that rigorous users studies are conducted in this area. 2 Types of User Studies In general user studies in Augmented and Mixed Reality can fall into one or more of the following categories:
Perception: How do users perceive virtual information overlaid on the real world? What perceptual cues can be used to distinguish between real and virtual content?
Interaction: How do users interact with virtual information overlaid on the real world? How can real world objects be used to interact with augmented content?
Collaboration: How can MR / AR interfaces be used to enhance face-to-face and remote collaboration?
Real Environment
Virtual Environment
Augmented Reality (AR)
Augmented Virtuality (AV)
8
interest, and the camera output will augment the display with additional
information about the environment (Gotow, 2010). Due to the portable nature of
smart phones, mobile augmented reality devices have become the most widely
deployed consumer augmented reality display device and show promise for
becoming the first commercial success for augmented reality technologies.
Additionally, augmented reality graphical information can be presented
through the use of wearable spatial-‐display devices that project visuals on top of
physical objects. At a TED conference in February 2009, Pattie Maes and Pranav
Mistry presented SixthSense, a gestural AR interface that is worn around the neck.
The device itself consists of a camera, projector, smart phone and mirror, which is
able to project an image on any nearby surface such as a hand or a wall. The user
also wears four colored caps on the fingers, which are used by the AR unit to detect
hand gestures made by the wearer to interact with the projected images (TED,
2009).
An augmented reality system must be able to recognize what users are
looking at, acquire the appropriate virtual graphics, and register them in the correct
position on the display device:
1. Recognition: Mobile AR applications are able to recognize what a user is
looking at by triangulating its location by means of information provided by
GPS, cell-‐phone towers, or Wi-‐Fi hotspots. Another approach used by
augmented reality developers is to use physical markers, or ARTags, which
can appear alone or appear on objects such as an advertisement in a
9
magazine. A mobile phone equipped with augmented reality software and a
camera will recognize the marker and accurately overlay a 3D graphic image
in the real world on screen.
2. Registration: In addition to recognizing what a user is looking at, an
augmented reality device must be able to accurately align and register virtual
objects over the existing ones in physical surroundings. Augmented reality
display technologies are able to accomplish this through landmark
recognition by utilizing images of the surrounding area stored on websites
such as Google, Bing, and Flickr (Vaughan-‐Nichols, 2009). The system is able
to accurately pinpoint the exact position of the artifact through the camera’s
level of focus and image-‐recognition software (Saite, 1996).
3. Image acquisition: Once the augmented reality system accurately
recognizes and registers a fixed location in the real world, it will pull in the
correct data and images to overlay on screen from the Internet (Vaughan-‐
Nichols, 2009).
Uses: Mobile augmented Reality is used in a variety of disciplines ranging
from military training programs and architectural surveys, to aiding surgeons
during medical procedures or replacing engineering technical manuals in the
manufacturing and repair sectors. Consumer-‐based applications have recently
emerged in the form of games, information browsers and immersive storytelling
experiences.
10
Purpose of Study
As augmented reality continues to mature as a communication medium for
information design, there is a need to explore the advantages, potential uses and
limitations of this immersive technology for smart phone users and content
developers. This research will investigate (1) the perceptions held by first-‐time
mobile augmented reality users, (2) the opinions of digital content developers about
potential applications of augmented reality in information design and visual
storytelling. Another aim of this research is to establish a paradigm for effective
information design when applied to mobile augmented reality.
In this study, the researcher will explore users' impressions of this
technology and how they interact with smart-‐phone based augmented reality
content. Additionally, this study will explore designers' and content developers’
ideas about how augmented reality can be used as a medium for information design
and interactive storytelling experiences. The combination of user and designer
perspectives, along with an evaluation of existing mobile applications will provide a
comprehensive perspective on augmented reality design. This research will help us
understand:
• Users' first impressions of mobile augmented reality applications.
• The desirability of augmented reality as a delivery model for interactive
content.
• Perceived limitations of augmented reality design as seen by users and
content developers.
11
• Potential applications of augmented reality when applied to digital
storytelling and information design.
Research Questions
RQ1: What are mobile augmented reality users' impressions of the technology
before and after using it for the first time?
RQ2: What do users and content developers see as current advantages and
limitations of mobile augmented reality?
RQ3: Do current information design principles need to be revised in order to apply
them to augmented reality experiences?
Significance to the Field
The power of augmented reality lies in its ability to place the world in context to the
needs of the consumer. Combined with mobile devices, this technology can filter
massive amounts of data in relation to the time and space on an individual user. The
market has recognized mobile augmented reality’s potential for redefining
information distribution and is banking heavily on its success. To that end, it is
paramount that developers understand how users perceive augmented reality
content and define the best practices for information design for this medium. Doing
so will be the first-‐step in ensuring mobile augmented reality’s commercial success
CHAPTER 2: REVIEW OF THE LITERATURE
Consumer applications of mobile augmented reality are still in their infancy,
yet the technology itself is decades old. Only recently has augmented reality gained
popularity due to the convergence of smartphones, faster networks and cloud
computing (Hurley, 2009). Yet many problems continue to plague mobile
augmented reality as the field struggles to find its niche. Most research within the
field is focused on developing better tools, while few researchers have looked at
how users perceive this technology. This gap in empirical user research represents a
need for understanding how to create a better user experience.
Such knowledge would benefit developers as their struggle to move past mobile
augmented reality’s initial “wow factor,” and avoid the stigma of being a victim of
marketing hype (Bryne, 2010).
This literature review will provide a brief overview of augmented reality
advancements, including a survey of existing applications and development
limitations. Additionally, the current state of user research and application
evaluation methods will be addressed. Finally, the last section will discuss using
prior research in the field of information design as a basis for evaluating augmented
reality applications.
13
A brief history
Although augmented reality is currently one of the hottest trends in mobile
media, the technology itself is not new. The earliest example of rudimentary
augmented reality can be traced back to the first heads-‐up display employed by the
British Royal Air Force during World War II. A radar screen superimposed on a
pilot’s windshield would display information about nearby aircraft, including the
identities of enemy targets (Vaughn-‐Nichols, 2009).
In 1968, Ivan Sutherland created the first modern example of an augmented
reality system that fulfills the modern definition by Azuma and Milgram (see
Chapter 1). Sutherland invented a head-‐mounted display device capable of
displaying wireframe 3D graphics, which moved as the wearer turned his head. By
1992, Tom Caudwell was the first to coin the term “augmented reality” when he
designed a head-‐mounted device for Boeing to assist engineers wire an airplane
during the manufacturing process. His system displayed the aircraft’s schematics on
the factory floor (Zhou, 2008).
Hirokazu Kato and Mark Billinghurst introduced the first computer vision-‐
tracking library, called ARToolKit (Kato & Billinghurst, 1999). Virtual imagery could
now be overlaid in the real world through digital markers called ARTags that could
be read by desktop computers equipped with cameras. When held in front of a
camera, printed ARTags are recognized by the system and a 3D image is
superimposed in its place on a computer screen. Today, advertisers and magazines
commonly use ARTags as a way to bring interactivity to a printed product.
14
The December 2009 issue of Esquire magazine demonstrated how print and
digital content could be merged through the employment of ARTags. After
downloading augmented reality software from the Esquire website, readers could
use their computer’s webcam to view the magazine’s digital features. This issue
includes the cover featuring Robert Downey, Jr. sitting on an ARTag that displays a
video promoting the film Sherlock Homes. More than half a dozen pages showcased
augmented reality content, including two advertisements from Lexus (Esquire,
2009). Other marketers and publications have followed suit including Popular
Mechanics and MINI.
Recently, state of the art research in augmented reality has shifted toward
mobile, hand-‐held devices. Researchers are studying how portable devices can be
used to enhance vision through AR tracking (Wagner, 2008), while industry
developers are employing directional-‐based augmented reality experiences through
mobile phone applications. Early mobile AR during the 1990s relied on Head-‐Worn
Devices controlled by wearable “backpack systems” and later expanded to PDAs and
cell phones (Azuma 2001, Schmalstieg et al., 2011). With the advent of powerful
smartphones equipped with GPS systems, faster processing speeds and data
transfer, and larger displays, today’s mobile augmented reality devices are capable
of overlaying virtual information on a smartphone's camera output in real-‐time.
Combined with the popular iPhone and Android mobile platforms, mobile
augmented reality has moved beyond its traditional military and industry
applications and has begun to enter the hands of consumers. A variety of
15
applications have been introduced, such as: information browsers, educational
tools, and games.
Consumer mobile augmented reality applications
Consumer-‐based mobile augmented reality application development has
seen tremendous growth over the past few years. According to Augmented Planet,
as of June 2010 there were 396 applications listed in Apple’s iTunes store with
descriptions listing augmented reality as a feature. 1 As of March 2011, there were
630 iPhone applications matching the same description3. The applications range
from information services and navigational tools to games and entertainment.
Table 2.1 Mobile Augmented Reality Categories
Information Browsers
Informational point of interest overlaid within a smartphone camera’s field of vision. Users can find local points of interest, restaurants, photos, and social media content.
Social Networking Shares and displays geo-‐tagged social networking content
Tourism City guides and sightseeing applications Entertainment and Games 360-‐degree shooters, virtual scavenger hunts,
virtual pets and puzzle cheat overlays. Education Learning based applications such as satellite finders
or constellation identifiers. Shopping Virtually try out products and or shop for
merchandise by using image or bar code scanners Navigation Assist users with navigating to a fixed location such
as a local restaurant or bus stop. Utilities Compasses, head up displays and overlays such as
color identifiers, rulers or assembly instructions. Translation Dynamically translates text in images from a
camera’s perspective into one’s native language. 1 Figure determined by an information search using the term “augmented reality” in iTunes App Store. 3 Updated figures based on search based on same terminology by the researcher on March 1, 2011
16
The most popular and largest application category to date is information
browsers. Mobile augmented reality browsers display data points of interest within
a mobile phone camera’s field of vision. They are used to find local points of interest
such as landmarks, public transportation, restaurants or retail outlets.
In 2009, SPRX mobile created Layar, the first mobile augmented reality
browser. The application offers several options for overlaying digital information
relating to brands, services and social networking. When a user searches for local
restaurants or businesses, Layar will overlay navigational information directing the
user toward the nearest location. There are over 1,500 “layers” to choose from
including: Wikipedia entries, restaurant guides, games and social networking
integration. Additionally, Layar is an open platform, which encourages other
developers and content providers to create additional layers showcasing their
information or services.
Other competitors have surfaced in the browser category. Junaio, a general-‐
purpose browser similar to Layar, is the first augmented reality application to
employ LLA (latitude, longitude, altitude) markers. LAA markers are considered to
be a more accurate location system when employed indoors, which tends to be
problematic under GPS-‐based systems (Junaio, 2011) due to the difficulty of indoor
reception of GPS satellite signals.
Other information browsers employ more targeted functions. These can
include city guides, subway locators, or wikis. For example, Zagat To Go, based on
17
the popular Zagat Dining guide, directs diners to the nearest fine dining locations.
Wikitude World Browser is a smart phone travel guide application that
superimposes information about historical landmarks or other points of interest
from Wikipedia.
In addition to information-‐based services, mobile augmented reality has
become integrated into new forms of social networking, shopping and games.
Integrating social networking has been a popular trend. It can appear either
integrated into a larger application or as a separate mobile application. Social
networking applications such as TwittAround allow augmented reality users to
virtually geo-‐tag their tweets, which can be visualized in real time on a mobile
display for others to see. Shopping apps such as Red Laser by Occiptal can read
labels and bar codes in order to compare prices and discover discounts and sale
items. Mobile augmented reality is also delivering new gaming experiences.
Smartphone gamers can participate in the virtual scavenger hunt game, Crimsonfox
(Toto, 2010) or play a 360 degree shooter like Sky Siege, in which the user shoots
down enemy aircraft flying around their living room or office.
Development Concerns and Limitations
Augmented Reality is far from the point at which consumers and developers
universally accept it as a viable technology. While most people have seen examples
of how augmented reality works in the news or other popular culture sources, few
people have experienced AR first hand due to access barriers and limited adoption
of AR-‐ready smart phones. Significant advances have been made over the last 15
18
years in AR display devices, especially in mobile media. However, mobile augmented
reality still faces significant hardware and software limitations. Additionally,
developers must deal with a lack of open standards and a growing concern over
user privacy and security.
Tobias Höllerer, an associate professor of computer science at UC Santa
Barbara, has said in a Wired.com interview:
Augmented reality is stifled by limitations in software and hardware, he
explained. Cell phones require superb battery life, computational power,
cameras and tracking sensors. For software, augmented reality requires
a much more sophisticated artificial intelligence and 3D modeling
applications. And above all, this technology must become affordable to
consumers. The best possible technology that is available today would
nearly cost $100,000 for a solid augmented-‐reality device. (Chen, 2009)
Smart phones also provide limited screen space for displaying augmented
information. Registration problems continue to plague augmented reality systems,
since GPS is only accurate to within 30 feet and does not work well indoors (Metz,
2009). Registration errors are considered the most significant limitation of mobile
augmented reality systems. They are often the result of tracking and alignment
issues caused by sensor and/or camera calibration errors and network latency
(Haller, Billinghurst & Thomas, p. 26). Registration errors are of particular concern
in the scope of information design as misaligned visual representation of data leads
19
to inaccurate reporting of information. Furthermore, it hinders the viability of the
application by diminishing user satisfaction.
Interoperability and open standards will also be necessary to encourage
content developers to adopt augmented reality. Currently, users are required to
access content data through a specific application. For example, the only way to
access Wikitude data is to use the Wikitude mobile application. Lack of cross-‐
platform interoperability could result in “AR wars” similar to the web browser wars
of the 1990s (Kirkpatrick, 2009). If augmented reality development is to become
mainstream, standardized data formats, are necessary. Concurrently, application
developers also lack easy to use authoring tools for creating mobile augmented
reality content (Schmalstieg, et al., 2011).
In addition to these technological limitations, augmented reality has several
image and security concerns. Science-‐fiction author and renowned augmented
reality critic Bruce Sterling discussed the issues the augmented reality industry is
facing in his keynote address, “At the Dawn of the Augmented Reality Industry.”
According to Sterling, augmented reality is an exciting technology with tremendous
promise, yet it still has a sleazy, gimmicky feel to it (2009). Sterling also warned that
security and spam could be potential problems in the future as porn and
pharmaceutical companies and hackers attempt to crowd the augmented view with
unwanted spam. In addition, augmented reality adopters can potentially face
information overload or dependence, similar to Internet addiction.
20
Moving forward, the mobile augmented reality industry must address the
monetary and technical limitations of existing display systems if this technology is
to achieve any significant market penetration. Furthermore, open development
standards must be realized if the industry wishes to attract developers for creating
new mobile augmented reality content.
The current state of user research and evaluations
The majority of published research in the field of augmented reality has
focused on the development of new technologies and application prototypes. Few
studies have addressed user experience and mobile application evaluation. Studies
that have focused on users have been informal and under structured testing setups
(Schmalstieg, et al.). According to a 2005 literature survey by Swann et al. (2005)
reviewing all augmented reality literature published in leading journals and
conferences, less than 8% had any reference to formal user evaluations. The HIT Lab
NZ conducted a similar study in 2008 and found that only 29% of published
augmented reality articles in major computer science journals included references
to user research. As of 2011 according to scholarly reference searches conducted by
this researcher, no published research studies address perceptions and experiences
of consumers using mobile augmented reality applications.
If mobile augmented reality is to become a viable medium for delivering
visual information, empirical research must be conducted that seeks to discover
how commercially available applications are perceived and employed. According to
Schmalstieg, et al., if mobile augmented reality applications are to become
21
commonplace, attention must be devoted to user studies. There is a need for novel
approaches to evaluating application interfaces and the consumers that use them.
Research gaps in this area can be found in the evaluation of mobile applications that
employ social networking and location-‐based user collaborations (p. 33); These
features are commonly employed by information-‐ bowser application such as Layar.
A potential solution
Mobile augmented reality’s potential lies in its ability to make location-‐based
information contextually relevant to the perception of the individual user. The
amount of available data is limitless. The Internet has brought worldwide access to
data sources including news, information services, shopping and social networking.
However, the problem lies in how the end-‐user wades through the exorbitant
amount of data. A way to make sense of this data from one’s personal point of view
is needed. Paul Saffo, a technology forecaster and analyst states:
“’Point of view’ is that quintessentially human solution to information
overload, and intuitive process of reducing things to an essential
relevant and manageable minimum… In a world of hyperabundant
content, point of view will become the scarcest of resources.” (Baer,
18)
The principles of information design, combined with the technological
advantages offered by mobile augmented reality displays, can be a filtering lens that
brings order to this chaos. Information design is a framework for clear
communication of data. It is employed as an organization and presentation method
22
designed to bring meaning to unstructured data. Data alone is considered worthless
without user context (Shedroff, 270). Through effective information design, one can
create value by defining filtering constraints that communicate data in such a way
that carries both meaning and purpose in context to the user's point of view (Dervin,
43; Shedroff, 270).
The presentation of a tailored information scheme is important for mobile
augmented reality since it relies on the placement of visual artifacts within the
user's environment. Additionally, the aesthetics of information design are closely
related to graphic design, where principles of visual communication are applied to
the delivery of information. Foremost among information visualizers is Edward
Tufte, the leading authority on the visual presentation of data (AIGA, 2004). Tufte is
a pioneer in the field who established the general principles of information design,
using visual language as a means to enhance the delivery and clarity of information
(Horn, 20).
In his three books, The Visual Display of Quantitative Information (1983),
Envisioning Information (1990), and Visual Explanations (1997), Tufte describes his
general principles of information design. These principles are a means for enabling
users to understand and “envision” information. The foundational principle is
avoidance of visual overload, a condition Tufte refers to as “chartjunk.” Tufte defines
chartjunk as any frivolous visual elements not required for the comprehension of
information, or distracting the viewer from said information (1983). This is
significant for the design of mobile augmented reality applications. Designers must
23
take into account competing environmental elements within the users field of
vision, which the designer has no control over. To counter this, designers and
mobile application developers can look to Tufte’s information design principles:
micro/macro design, layering and separation, small multiples, color and
information, and the integration of words and images.
Micro/Macro Design: Tufte describes Micro/Macro compositions as
visualizations that contain large amounts of detail, yet within which patterns
emerge. "Micro" refers to the smaller details that make up the greater whole
(macro), thus creating structure (Tufte, 1990, p. 33). Tufte argues that macro/micro
design is both a critical and valuable method for displaying information in ways that
readers can understand the complexities of the “big picture,” yet also place the
smaller details in context (1990, p. 50).
Layering and Separation: By layering information, one can establish a visual
hierarchy that emphasizes content important to the reader and deemphasizes
what’s not. By creating salience through design, Tufte argues that the reader can
easily navigate through a structured, organized construct for presenting
information.
The separation of data categories is equally as important as hierarchal
presentation structures. According to Tufte, Designers should separate layers of
information by means of weight, shape, size, value or color. Failure to do so results
in “jumbled up, blurry, incoherent, [and] chaotic” designs (1990, p. 58).
24
Small Multiples: A small multiple is a single design element, repeated multiple
times in order to show variations of a theme. It is a reliable method for allowing
users to see changes and differences among similar objects (1990, p. 67). Through
the principle of small multiples, Tufte argues one can create contrast in detail, yet
maintain context without the need for changing the design (1983, p. 170). Designers
often employ this method to create a sense of unity by using similar colors and
typefaces, yet may use larger type for headlines or categories to visualize
significance.
Color and Information: The principle of applying color to information is a
means to label categories, quantify intensity, imitate reality, and to enliven or
beautify data (1990, p. 81). When using color, Tufte suggests that strong, darker
tones should be used sparingly since they are both “loud” and “disruptive” and can
distort one’s visual field, thus creating unpleasant effects. Secondly, using light and
brighter colors next to each other can also cause visual distortions (1990, p. 83).
Instead, Tufte suggests that designers use earth tones, and lighter shades of blue,
yellow and grey (1990, p. 90).
Integrating words and images: Words and images are the fundamental
building blocks of information design. Designers employ this principle in order to
present information in story form. Tufte argues that words and images should never
be separated, as it would burden the reader with the task of having to cognitively
associate the separate elements. Designers who effectively integrate text and images
25
can control the navigation and flow of the reader’s experience to facilitate
understanding (1990, p. 116).
Although these principles were originally conceptualized with print in mind,
Tufte and information designers consider these principles to be “timeless” and
applicable to any medium. Tufte argues his principles of information design “are
universal-‐like mathematics and are not tied to unique features of a particular
language or culture” (1990, p. 10) Tufte’s influence has expanded into web design
and other forms of information interfaces, including the iPhone (Tufte, 2008).
This point has also been proven through academic research. Beverly
Zimmerman of Brigham Young University conducted a study in 1997 to test if
Tufte’s principles of information design could be applied to effective web design.
The study used Tufte’s principles to evaluate the design effectiveness of two
websites. Her results indicated Tufte’s principles are applicable to interactive media
like the web, and can also serve as a measurement tool for evaluating the
effectiveness of a web page in conveying information (Zimmerman, 1997). This
research is relevant to mobile augmented reality since there is a need for effective
measurement of the effectiveness of mobile augmented reality application designs.
Edward Tufte’s principles could potentially fill this need.
CHAPTER 3: RESEARCH METHODS
Evaluating a new communication medium such as mobile augmented reality
requires a multi-‐dimensional qualitative method that takes into account both user
and design perspectives, along with an evaluation of existing applications. This
study implemented a qualitative thematic analysis and heuristic evaluations in
order to answer the following:
RQ1: What are mobile augmented reality users' impressions of the
technology before and after using it for the first time?
RQ2: What do users and content developers see as the advantages and
limitations of mobile augmented reality?
RQ3: Do information design principles need to be revised in order to apply
them to augmented reality experiences?
This study describes the initial perceptions of college students before and
after they engaged with a mobile augmented reality application for the first time.
Additionally, a small number of questionnaires were conducted to explore the
problem from the perspective of augmented reality designers and content
developers. Focus groups and questionnaires were used to collect data in the areas
of usability and desirability as well as the advantages and disadvantages of the
medium. The primary goal of this research is to evaluate the viability of mobile
27
augmented reality applications as a vehicle for information design. The narrative
data was transcribed, coded and categorized into main themes related to RQ 1-‐2.
To answer RQ 3, the researcher conducted a heuristic evaluation of Junaio
and Layar, the two most popular information-‐based mobile augmented reality
applications available at the time of this study. A heuristic evaluation is an effective
method for identifying the strengths and limitations of an interactive system, based
on accepted standards of practice. For this study, evaluative criteria were derived
from established principles of information design.
Methods overview
Thematic analysis: This study relied on thematic analysis to identify user
perceptions of mobile augmented reality application and the designers who create
them. Braun and Clark (2006) define thematic analysis as “method for identifying,
analyzing and reporting patterns (themes) within data” (p. 79). A themes, as defined
by Boyatiz (1998), is:
“...a pattern found in the information that at the minimum describes and
organizes possible observations or at the maximum interprets aspects of the
phenomenon. Themes can be generated inductively from raw data sets, or
deducted through the use of theory and prior research.” (p. 161)
A theme may be identified at the manifest level (directly observable in the
information) or at the latent level (categorizing issues underlying the phenomenon)
(p. 9).
28
The advantage to using thematic analysis is that it is a flexible methodology
used to quickly highlight similarities and differences within a data set (Braun and
Clark, p. 97). Thematic analysis is sound way to compare and contrast the strengths
and weaknesses of mobile augmented reality when applied to information design
from the perspective of users and designers. Furthermore, this method can be used
to categorize problems found in usability tests, such as heuristic evaluations.
Researchers often employ a thematic analysis when data focuses on experiences,
practices or compliance-‐oriented evaluations.
Heuristic Evaluation: Heuristic Evaluation is a common tool for interaction
design analysis . It is an easy and inexpensive way to evaluate the viability of a
design, such as a mobile phone application (Unger & Chandler, 2009, p. 71). Rogers,
et al. (2007) define heuristic evaluation as, “an approach to evaluation in which
knowledge of typical users is applied, often guided by heuristics, to identify usability
problems“ (p. 590). Heuristics are principles based on common-‐sense knowledge, or
rules of thumb and design usability guidelines. A key feature of this method is that
does not require the presence of users and can be conducted by an expert reviewer
(Nielsen & Tahir, 2002).
For the purpose of this study, the heuristics used in the evaluation of smart
phone applications are based on the principles of information design derived from
the works of Edward Tufte. Described by The New York Times as “The Leonardo Da
Vinci of data,” Tufte is considered the leading authority on the visual presentation of
data. (AIGA, 2004) His principles are regarded as “universal” across all forms of
29
media. For this reason, Tufte’s guidelines are the ideal criterion for evaluating
information design practices on mobile augmented reality platforms.
Setting
This study was conducted at Ball State University, a mid-‐sized university
located in Muncie, Indiana. Participants were selected from a population of 22,000
students. Additional field research was conducted in Indianapolis, for the purpose
of heuristically evaluating mobile augmented reality applications usage within a
metropolitan environment.
Participants
First-‐time users: The researcher employed convenience sampling to collect
subjects for Test Group 1. Selection of participants was restricted to undergraduate
and graduate students currently enrolled at Ball State University. Participants were
also selected because they currently owned an iPhone or android-‐based smartphone
and had no prior experience using mobile augmented reality applications. Six non-‐
design students were chosen at random to participate.
The participants in the study represented a variety of majors and
departments within the University. Four students were male; two were female. Two
of the four male students were graduate students while the rest of the participants
were undergraduates. All student participants were in their early to mid-‐twenties.
Designers and content developers: Two participants were identified and
selected to complete questionnaires by the researcher through Internet searches
30
and recommendations. Participants were selected based on their skills and
experience in mobile augmented reality development, along with a visual
communication background in one or more of the following areas: information
design, visual storytelling, multimedia graphics, mobile platforms, and or art
direction. Both participants have been recognized as leaders in their field, based on
their publication and presentation records.
Research Instruments
Focus group: A focus group explored initial user perceptions of augmented
reality, before and after using a mobile augmented reality application for the first
time. The focus group met for two sessions, both lasting one and a half hours in
length. Sessions were separated by a seven-‐day period during which participants
were asked to complete an assigned task with an AR application. The total time
commitment for participants was five hours (see Appendix A).
I. Session One: Participants were presented two short videos introducing the
concept of augmented reality and the Layar mobile application. Both of these
videos were produced by Layar and demonstrate the application’s purpose,
along with a preview of its information, social media and gaming features.
Following the videos, subjects were asked to discuss their first impressions,
how they might use the application, whether they found it desirable, and/or
potential concerns. At the conclusion of the session, the participants were
instructed to download the Layar application onto their personal smart
31
phone and to use it to complete an assigned task. The task was to try out and
evaluate the Layar application for at least two hours over a seven-‐day period.
II. Session Two: Students reconvened to discuss their experiences using the
Layar application. Discussion probes covered usability, viability as a medium
for information delivery, features, suggestions for future applications,
problems and concerns, desirability, and willingness to use the application in
the future.
Questionnaires: Perceptions about mobile augmented design and its
implications in communicating information were measured by questionnaires
completed by two design professionals. Questions explored respondents opinions
about how augmented reality can be used to advance information design and
interactive storytelling, including its advantages and disadvantages relative to other
communication mediums (see Appendix B).
Heuristic Evaluations: An evaluation checklist was devised using judging
protocols derived from the information design principles written by Edward Tufte
in his books: The Visual Display of Quantitative Information (1983), Envisioning
Information (1990) and Visual Explanations (1997). Problems and instances are
noted for each applied principle on the basis of structure, presentation and
dynamics in the application design (see Appendix C).
Data Collection and Procedures
Data was collected through moderated focus groups, surveys and heuristic
evaluations:
32
I. Focus groups: Data was collected through interactive group discussion,
moderated by the researcher. The focus groups were conducted under
natural settings using a discussion guide protocol (see Appendix A).
Discussions took place on campus at Ball State University, where all
participants were currently registered students. The researcher guided the
group’s discussion through predefined probes (see Appendix A) and asked
follow-‐up questions based on participant responses. The data collection
process took place during two focus group sessions conducted over a two-‐
week period, totaling three hours of discussion time. Each session was video-‐
recorded for accuracy and lasted one-‐and-‐a-‐half hours.
II. Surveys: An open-‐ended questionnaire (see Appendix B) that consisted of
unstructured and sentence completion type questions were sent to
participating design experts, along with instructions for how to complete and
submit the survey by e-‐mail. Participants completed the questionnaires at
home or work on their own time.
III. Heuristic Evaluation: The researcher evaluated two mobile augmented
reality applications, Layar and Junaio by using an evaluation checklist based
on heuristics derived from the following information design principles by
Edward Tufte: micro/macro design, layering and separation, small multiples,
effective use of color and integration of words and images. The evaluation
was conducted by testing the mobile applications at Ball State University’s
main campus and its surrounding neighborhoods in Muncie, Indiana. Every
33
function was tested twice for reliability, and all problems were documented
for further analysis. Additional field research was also conducted in
Indianapolis to provide contrast regarding how the applications preformed
in a metropolitan environment versus rural area. Observations were
recorded on the heuristics scorecard during a one-‐week period, totaling six
hours of evaluation time.
Data Analysis
The collected narrative data from the user focus groups and designer surveys
were transcribed and categorized by thematic analysis. Subject responses were
categorized into themes applicable to answering the research questions. The
researcher used a coding method to organize collect data into themes that
illuminated the key concepts of this study. Additionally, interview responses from
designers were used to provide supplemental evidence in support of user responses
The applications Layar and Junaio were analyzed in accordance with the
information design principles defined on the heuristic evaluation scorecard (see
Appendix C). A thematic analysis was applied to categorize the strengths and
weaknesses of the applications in accordance with the design principles to establish
a paradigm for information-‐based augmented reality design. Emergent themes from
the heuristic evaluations were cross-‐referenced and validated by the narrative data
sets.
34
Ethical Considerations
As this study required the participation of human subjects, ethical issues
were addressed to protect the privacy and safety of all participants. The most
significant issues considered during this study were informed consent and
confidentiality. Prior to participation, each subject was fully informed of all-‐
important aspects of the study, including its aim and purpose. Furthermore,
participants were advised of their right to withdraw from the study at any time.
Subject confidentiality was ensured by not disclosing their names or personal
information in the research. Only relevant data pertinent to answering the research
questions were included.
Conclusion
This study employed a qualitative method to answer the three stated
research questions. Data collected from focus groups, questionnaires and
evaluations of existing augmented reality applications were thematically analyzed
to gauge the viability of mobile augmented reality as a medium for visual
information communication.
CHAPTER 4: RESULTS
This study employed a qualitative methodology to (1) discover and identify
the initial perceptions of first time augmented reality users and (2) identify the
advantages and limitations of mobile augmented reality systems from a design
perspective. Focus groups were used to collect data in the areas of usability and
desirability, along with the perceived advantages and disadvantages of using mobile
augmented reality as a medium for information delivery. Additionally,
questionnaires submitted by two professional designers with experience in the field
provided supplemental insights into mobile augmented reality from a designer’s
perspective. Finally, a heuristic evaluation was conducted to evaluate how existing
mobile augmented reality applications conform to established information design
principles.
A thematic analysis revealed that mobile augmented reality is struggling to
find its niche among consumers. Themes derived from participant responses were
problematic technical limitations and a lack of compelling content and user
experiences. Additionally, a heuristic evaluation of two augmented reality
applications discovered limitations in how augmented reality content is displayed
on mobile devices.
36
First Impressions
Focus group discussions provided insight into how participants perceived
the technology before using it for the first time. Prior to discussion, participants
watched two short videos (totaling 10 minutes in length) that introduced the
concepts of mobile augmented reality and the Layar application. Both videos
demonstrated Layar and the social media, gaming and information-‐delivery
capabilities of mobile augmented reality. Following this brief overview, the
researcher facilitated discussion by asking participants to share their initial
thoughts and opinions based on the introductory videos. Additional questions
explored how they might use the application, its desirability and/or potential
concerns. Although the discussions were structured, participants were encouraged
to speak freely about their thoughts and opinions. In addition to the group’s first
impressions, two themes emerged from the patterns of discussion: perceived uses
with an emphasis on information-‐based services and a potential invasion of user
privacy.
First impressions: Initial judgments about mobile augmented reality were
generally positive, with most participants viewing mobile augmented reality as
potentially useful. The entire group was in agreement that this was an application
they would be willing to try out for themselves. Participants described the
application as a potential tool they could use for socializing with friends or as an
immersive information guide they could employ when exploring a new city. Some
37
participants viewed mobile augmented reality as the next evolution of virtual
spaces:
“(Mobile Augmented Reality) Almost seems like a cross between
second life and reality. You are meeting in a virtual space, but you are
actually meeting in the same space. It's kind of like Second Life 2.0, the
evolution of second life. Instead of trying and take everything that is
reality and make it virtual, it creates the best of both worlds.”
Perceived information-‐based uses: When participants were asked to describe
how they might use an augmented reality application such as Layar, each person
described a function or use relating to information based services. The most
common responses could be subdivided into information-‐based searches,
navigation, tourism and shopping. Social networking and gaming were also
mentioned as potentially valuable reasons for using mobile augmented reality.
However, all participants considered these to be secondary.
Being able to point your mobile phone at an artifact within one’s own
surroundings was of particular interest to these potential mobile augmented reality
users. They were intrigued by the possibility of being able to use their mobile
devices to discover additional information, such as a restaurant’s hours of
operations or the historical significance of a local landmark. A common idea among
all participants was to use this technology within a museum as a personal tour guide
for learning more about artifacts on display:
38
“I would use AR for information purposes. Like if I was going to a
museum, finding out about the artifacts that I'm looking at. If you put
the phone over the art, it gives you information about the piece and
it's history. I recently went over to the Air Force museum in Dayton
over spring break. They have some of the stations have podcasts, but I
think an augmented reality would be a better suited since it could feed
more content.”
Participants recognized tourism-‐based information as one of the most
potentially useful applications of mobile augmented reality. Everyone in the focus
group saw this as a tool they would want to take with them on their next vacation or
use to explore an unknown destination or city. Paired with the built in navigational
features of a smart phone, several participants described mobile augmented reality
as a “personal tour guide:”
“I like the idea of having a device like this when I was traveling last
year. I had never been to the place, and it would have been great to
have a guided tour that continually updates live information like
traffic flow, highlights in the city, that sort of thing.”
This concept of using mobile augmented reality as a personal tour guide was
also used as a metaphor for describing how this technology could be employed as a
personal shopping guide:
“If you tell the app what you want, and the app points out a route for
you... so as you are walking through the store, telling you where the
39
items are located. A guided experience, like walking through IKEA... A
virtual tour guide as you are looking for your products.”
Being able to use a mobile phone as a shopping guide was conceived as a
valuable tool for both consumers and marketers. Participants envisioned using
mobile augmented reality for comparative shopping, virtually trying out
merchandise and researching additional information about a product by simply
pointing the cell phone’s camera at the item of interest. Many also recognized the
value to marketers and companies who could use the application to attract
customers by offering special deals to mobile augmented reality users.
Other uses: In addition to information-‐based services, several focus group
members were also attracted to the notion of using mobile augmented reality for
social networking and gaming. Students expressed how essential social networking
tools such as Twitter and Facebook are to how they communicate with their peers.
Participants conceptualized using mobile augmented reality as a way to virtually
communicate and/or meet new people in their immediate surroundings.
Participants suggested they would want to use this technology as a means for
communicating anonymously with others on their daily commute to class or work.
They saw this as an innovative way for connecting to new people in their
communities.
Gaming was also mentioned both as an extension of social networking and as
a potentially attractive feature of mobile augmented reality. Younger students, who
did not own a car and/or had limited spending money, saw mobile augmented
40
reality games as a cheap and entertaining way to socialize with friends. Other
participants described mobile augmented reality gaming as a “virtual theme park” in
which children could play within the confinements of their own backyards.
Privacy Risks: The perceived risk of privacy when operating a mobile
augmented reality application was of significant concern to many of the subjects.
Participants feared that their actions and locations could potentially be tracked
through location-‐based technologies:
“I have issues with privacy; I don't use any applications that are
location-‐based because I don't want people following me. Not that I'm
afraid of a ‘Big Brother’ idea. But at the same time, I like having some
anonymity of where I’m at. I don't think it’s really relevant that people
know my whereabouts. With augmented reality, it's obvious that
people would know where I or other people are located.”
The majority of participants in this study made clear that they want direct
control over what information may potentially be shared:
“I do want control. All of the stories that have come out in the news
about these privacy issues are causing me to trust these entities less
and less. Privacy controls absolutely need to be involved. You need to
be able to have the option to turn off your location, because if you
don't you are going to get people who don't understand the risks
involved.”
41
Although the majority of students had concerns about potential privacy risks,
a few members of this study considered it a non-‐issue. Those who were concerned
however, also recognized that a minimum amount of personal information sharing
would be necessary. Otherwise, the user experience could be significantly crippled.
User Experiences and Feedback
Following the group’s sharing of their first impressions about mobile
augmented reality, the research subjects were instructed to try out the Layar
application on their own and report back in one week to discuss their experience.
Participants were asked to discuss what they liked and disliked about the
application, any problems they incurred, suggestions for improvement, and their
desire and willingness to use the application in the future.
After having a chance to use the application for the first time, a majority of
the participants reported that they had “mixed feelings” about their experiences.
Although they liked the concept of mobile augmented reality, in practice the
application left much to be desired. Participants pointed out design, contextual and
technical limitations which hindered their experience. Additionally, several of the
features they reported trying did not work as expected (or at all), while those that
did were not considered to be intuitive:
“I feel it’s not incredibly intuitive. It took a lot of poking around to
figure out exactly what I was actually doing. I considered at many
42
points to go online to look up what I was doing and try to find a ‘How
To’ guide.”
Other students shared this same sentiment and suggested that an in-‐
application help guide would be both helpful and necessary, since the types of
interfaces used for mobile augmented reality applications are likely unfamiliar to
most new users.
“A tutorial would be good, just like a lot of mobile phone games have
a tutorial option. The same thing would work here. This is what you
can do.... even if your first screen options are asking what do you want
to do, and list specific options ... you tap that.... then it guides you
through that process.”
Understanding how to use the application was not the only concern. By far,
user interface and information design problems were the most significant
complaints among all participating users. Analysis of the responses revealed that
information overload and a lack of hierarchy between displayed elements caused
the greatest hindrance to usability as a result of the application’s design. Other
notable factors included technological constraints, a lack of content in rural areas,
and a general lack of compelling content.
Design concerns: Participants were dismayed by how the application
displayed augmented reality content. Although they found the aesthetics of the
interface to be visually pleasing, they also found it difficult to differentiate between
the multiple data points displayed on screen. The larger the pool of content
43
available within the user’s vicinity, the harder it was to navigate through the
information displayed on screen. Content was continually overlapping, making it
nearly impossible to see or access the information that was covered up. Most of the
subjects described the situation as “overwhelming,” especially within larger
metropolitan areas. As one student describes:
“I went downtown to Indianapolis circle and the application was
throwing hits from every store in the mall. The screen on my phone
was overloaded and I wasn’t even in the mall!”
Even limiting the distance parameters within the application did not improve
the experience. Another subject explained:
“I don't know what's the best way to go about solving that issue, I did
mess with the parameters a bit, it helped some, but at the same time
on the lowest area in a heavy populated area, you are going to get
overlap.”
Participants also suggested they would like to see options for hiding content,
such as a particular user’s tweets or a retail location tag they are not interested in.
They said this might serve as a means for combating the application’s display issues.
Additionally, many participants considered that it may be valuable to add “like” and
“dislike” buttons as a way for the application to learn what types of content a user
might be interested in and filter out any irrelevant data.
Technological constraints: Most subjects reported that the quality of the
application’s performance on their smartphones was less than adequate. Several
44
issues were raised, including slower processing speeds when running the
application and detrimental effects on battery life. Several subjects described their
experiences as “glitchy” and often reported the incorrect location of data. A subject
describes his experience:
“My phone wouldn't calibrate proper north and south. I tried it out a
couple different times and it would tell me that Thai Smile was ‘that
way’, while I'm standing right there staring at it. I don't know, I really
hate things that are glitchy, so I'm really hesitant to use it again.”
Subjects suggested that a “lite” version of the Layar application should be
released. They said they are willing to accept fewer features in return for better
hardware performance and battery life. The participants were also in agreement
that the screen size of a typical smartphone is inadequate for an application of this
nature. One of the subjects commented, “I think the screen size is too small to cram
so much information into it.” Others suggested that the larger screens employed by
tablet devices such as the iPad would be a more suitable medium for displaying
augmented reality content.
The design respondents shared this same sentiment. Smaller screens,
in combination with limited tracking systems and a lack of computing power,
diminish a designer’s ability to create immersive, three-‐dimensional
experiences. Instead, designers are limited to developing two-‐dimensional
overlays in a three-‐dimensional environment, which designers and end-‐users
find to be limiting. Display technologies must evolve to the point that a
45
seamless presentation of augmented reality content becomes possible within
a physical space. As one designer stated, “3D content is really what works
best for an augmented reality application.”
Furthermore, both designers argued that today’s display technologies offer
more of a “forced” experience, rather than an immersive one. The user is limited by
what they see and is forced to experience augmented reality through the
perspective of a hand-‐held device, rather than through the user’s own eyes. As one
designer explained:
“When you are looking around through your device, you are always
being mediated through something. You are always looking through
the screen. The point of focus is always the device. You're not really
experience augmented reality, you are experiencing a trick on your
mobile phone.”
In one designer’s opinion, head-‐worn display devices will eventually
mature to the point that they will usurp smartphones and the leading display
device for mobile augmented reality content. A head-‐worn display device
would be able to provide a seamless, immersive viewing experience and
open the door to new forms of interactivity based on physical movement,
rather than a touchscreen interface.
Lack of content in rural areas: In contrast to the “information overload”
concerns many of the participants had, several subjects felt that the amount of
content available within smaller communities was significantly lacking. Although
46
most of the subjects had the opportunity to try the application in larger
metropolitan cities, nearly all of them reside in a smaller mid-‐western city with
approximately 70,000 citizens.4 Subjects argued they would be less likely to
regularly use the application in their own neighborhoods if local content was
limited.
Lack of compelling content: At the conclusion of the study, participants were
asked if they intended to use Layar in the future. Each stated they would either
seldom use the application, or not at all. Despite its glitches, the students argued
that the application itself offers little compelling content to warrant repeated use.
Rather than focusing on static content such as location-‐based information, the
subjects said they would be more interested in seeing non-‐static content such as
local news, events such as concerts and sports, or current specials run by local
business.
Likewise, the design respondents agreed that existing applications
have little to offer. They blame this lack of quality content on an existing top-‐
down application development cycle where corporate objectives overshadow
the need for a compelling user experience. In their opinion, most mobile
applications are being employed as marketing tools, which can be seen as
overhyped and gimmicky. A culture driven by content developers is
necessary to support the maturation of this new form of interaction and
4 Population count as reported by 2010 census.
47
presentation. In the meantime, as one designer laments: “Augmented reality
is going to be a really gimmicky field for a very long time.”
Heuristic Evaluations
In order to validate the design limitations raised by the focus group, this
study employed a heuristic evaluation process to appraise mobile augmented reality
applications on their ability to effectively present information. The evaluation was
conducted by the researcher using information design standards as evaluative
criteria. A set of heuristics was devised from Edward Tufte’s principles of
information design to appraise the mobile applications Layar and Junaio on the
basis of structure, presentation and dynamics in the applications’ information
design (Appendix C).
For this study, five heuristics were used to examine how Layar and Junaio
present augmented reality content within a smartphone camera’s view. The
research compared how each application displayed Twitter and Wikipedia content.
The purpose of this evaluation was to identify major information design flaws
within Layar and Juniao through the application of five heuristics:
1. Presents individual data points within a framework relating to a larger
context;
2. Effectively layers and separates information to emphasize hierarchy and
structure;
3. Employs small multiples as a means of comparing differences between
related data;
48
4. Effectively uses color for information display purposes;
5. Integrates words and images effectively.
Although usability is an important component of interactive information
design, a usability evaluation is beyond the scope of this study. Usability studies are
standard practice for any interaction design project prior to deployment. Instead,
this study examines the unique problems mobile augmented reality faces when
visually presenting information.
After the evaluation was complete, 10 problem areas were identified as
violations of established information design principles and rated according to
severity:
1 — Design problem that occurs infrequently and/or with minor user hindrance.
2 — Moderate design problem that occurs occasionally and may result in
readability issues.
3 — Severe design flaws that occurs frequently and damages the readability of
data.
Issue 1: Oversized icons cover up the background (camera view) and other data points.
Application: Layar
# Problem Severity Rating
Heuristic Number Broad Heuristic
1
Oversized icons cover up the background (camera view) and other data points.
3 1, 2 Micro design elements should not overshadow the main view.
49
Problem: Although this problem does not occur within the Twitter and Wikipedia
components in the application, it is worth mentioning since it severely affects other
layers within the application such as Livespot (See image 4.1). By covering up the
entire screen, the application violates the micro/macro design principle. The minor
detail (restaurant icon) overshadows the macro view by failing to display its
location in context to the user’s visual perception of the environment as seen
through the smartphone’s camera. It is also problematic when content completely
covers and eliminates access to other points of interest; this is a violation of Tufte’s
layering and separation principle.
Image 4.1: Livespot on Layar Image 4.2: Layar’s “Bird’s Eye” view
Image 4.3: Reality view on Layar Image 4.4: Twitter on Layar
50
Issue 2: Oversized icons cover up the background (camera view) and other data points.
Application: Layar Problem: Layar offers two options for viewing augmented reality data in camera
mode: “Reality” view and “Bird’s Eye” view. The reality view (Image 4.3) employs
the layering and separation principle by presenting data points in a visual hierarchy
according to its geographical proximity to the user. Closer points of interest appear
closer in the foreground, while distant location points are placed further back.
Additionally, the subtle, yet noticeable changes in dot sizes are an effective
execution of the small multiples rule. The size differences between the dots
represent their relative distance to the user, thus maintaining an accurate visual
perspective. In contrast, the “bird’s eye” view (Image 4.2) fails to layer information
points appropriately in addition to stripping much of its location-‐based context in
relation to the user’s position. Furthermore, large data sets, such as the one depicted
in image 4.1, can result in unnecessary data point overlapping. This can lead to
usability issues since information can be missed or difficult to access.
# Problem Severity Rating
Heuristic Number Broad Heuristic
2
“Bird’s eye” view feature results in data point overlap and diminishes location-‐based context.
2 2 Data should be appropriately layered in proximity to the user’s location
51
Issue 3: Difficult to recognize which tweet is currently selected.
Application: Layar Problem: As seen in image 4.4, Layar uses Twitter avatars to represent
geographically tagged tweets within the camera view. While this method offers the
benefit of placing tweets in context with its creator, it does create confusion for the
user. The only way to determine which tweet is currently being viewed is the small
arrow connected to the corresponding dialog box and the identical avatar image
located within. This is the result of a lack of separation between other displayed
avatar images. A better approach would be to model it after the Wikipedia layer
(Image 4.3), which represents Wikipedia entries as consistent visual icons, while
separating selected data points by a change in color.
Issue 4: A visual disconnect exists between data points and their related text and/or images.
# Problem Severity Rating
Heuristic Number Broad Heuristic
3 Difficult to recognize which tweet is currently selected
2 2, 3 Selected objects should be highlighted. Icons should be simple, and consistent.
# Problem Severity Rating
Heuristic Number Broad Heuristic
4
A visual disconnect exists between data points and their related text and/or images
2 5 Data points should have strong visual associations with related images and text.
52
Application: Layar
Problem: Layar loosely presents text and images relating to a point of interest by
means of a static dialog box located always at the bottom of the screen. Content
appearing inside the box is dependent on which data point is currently being
viewed. The application employs a small arrow that points at the highlighted dot to
create a visual association. (images 4.3, 4.4) Although this presentation form loosely
adheres to the integrating words and images principle, it does create an
unnecessary burden for the user who is tasked to associate related content by
means of a subtle visual clue. This problem is particularly troublesome when it
occurs in conjunction with issue 3.
Image 4.5: Layar (4.0) dialog box Image 4.6: Layar (3.0) dialog box Issue 5: Large text boxes take up a considerable amount of screen real estate.
# Problem Severity Rating
Heuristic Number Broad Heuristic
5 Large text boxes take up a considerable amount of screen real estate.
2 1, 2
The display of secondary content should not overpower the framework of the bigger scene.
53
Application: Layar
Problem: Dialog boxes in Layar (version 4.0) take up one-‐third of the available
screen real estate, thus diminishing the environmental view. Tufte’s layering and
separation principle suggests this content requires additional visual emphasis since
it takes priority over other content on screen. However, this execution unnecessarily
obscures the user’s view by covering up other data points and their surroundings.
Furthermore, the information presented with the dialog box is poorly laid out,
resulting in a waste of precious space on the small screen. Earlier versions of Layar
were more conservative in the dialog box design. Version 3.0 dialog box designs
(Image 4.6) presented only basic identification information in one-‐sixth of the
available screen space. If a user wanted more information, she could open an
expanded dialog box that overlays on top of the camera view. This display option
appropriates layered content to establish visual dominance by dimming the
background.
Issue 6: White text on bright colored backgrounds difficult to read.
Application: Layar
# Problem Severity Rating
Heuristic Number Broad Heuristic
6 White text on bright colored backgrounds difficult to read.
1 4 Poor color selection and lack of contrast.
54
Problem: Brightly colored backgrounds can make white type illegible. This is not a
consistent problem within the Layar application, since each layer employs different
visual styles. However, when it does occur, the small type size, combined with a lack
of contrast with the background, makes the text especially hard to read. Tufte
suggests that designers avoid brightly colored backgrounds for this very reason,
since it causes too much visual distortion. Instead, designers should use earth tones
such as lighter colors of blue, yellow and gray with black type.
Issue 7: User-‐interface controls do not orient correctly when viewing in landscape mode.
Application: Junaio
Problem: When switching from portrait to landscape mode, the navigational
interface elements do not reorient to conform to the new view (image 4.7). This is
more of a usability issue, but it also presents a flaw in the information design as it
hinders the readability of presented information. The small multiples principle
suggests that repeated navigational devices such as links, arrows, or icons should
maintain a consistent design. Failing to orient navigational elements correctly is
both visually unappealing and unnecessarily burdens the user.
# Problem Severity Rating
Heuristic Number Broad Heuristic
7
User-‐interface controls do not orient correctly when viewing in landscape mode.
3 3 User interface should be consistent throughout the application.
55
Image 4.7: Junaio
Issue 8: User-‐interface controls do not orient correctly when viewing in landscape mode.
Application: Junaio
Problem: When operating the application in landscape mode, expanded dialog
boxes display text sideways. This problem is the result of the same design flaw as
issue 7. In this instance, the poor alignment of text creates significant readability
# Problem Severity Rating
Heuristic Number Broad Heuristic
8
Expanded dialog boxes are presented sideways when using application in landscape mode.
3 5 Text presented sideways is difficult to read.
56
issues, thus breaking the principle of integrating words effectively. Large amounts of
copy should never be oriented on its side, as it makes the text difficult to read.
Issue 9: Radar size is inadequate for presenting surrounding information in 360 degrees.
Application: Junaio
Problem: The radar graphic (see image 4.7) used to display nearby points of
interest within a 360-‐degree radius of the user is too small. There is not enough
separation between elements to adequately gauge the distance between them.
Layar’s execution of the radar navigational element would be more appropriately
size (image 4.4).
Issue 10: Text displayed within individual data points is difficult to read.
Application: Junaio
# Problem Severity Rating
Heuristic Number Broad Heuristic
9
Radar size is inadequate for presenting surrounding information in 360 degrees.
2 3 Difficult to separate content effectively on a smaller scale.
# Problem Severity Rating
Heuristic Number Broad Heuristic
10 Text displayed within individual data points is difficult to read.
1 4
Reversing type out of dark backgrounds requires strong color contrast and larger type sizes.
57
Problem: In contrast to Layar, the Junaio application employs a tighter integration
of words and images using the dialog boxes themselves as geographically pinned
data. This execution offers many advantages over Layar since it does not
unnecessarily burden the reader with having to visually associate separate
components. However, the execution could use refinement of the color of the text.
White text, instead of gray over the dark translucent background would provide
greater contrast, thereby making the text easier to read.
Conclusion
According to the respondents, mobile augmented reality is a compelling
medium for displaying visual information. Participants in this study see mobile
augmented reality as an immersive tool for engaging everyday information in a
more personal context. However in its present form, mobile augmented reality is
lacking in functionality and desirability. User studies and design evaluations suggest
that mobile augmented reality is limited by poor information display factors and
technological limitations of mobile devices. Furthermore, there is a lack of
compelling content and an overhyped push by corporations and marketers.
Despite these limitations, the participants in this study agreed that mobile
augmented reality has potential. However, it needs time to mature. As one student
concluded, “I see potential in this, but it's just like anything else, it's just going to
take time before it finds its niche.”
CHAPTER 5: DISCUSSION
This thesis postulated that a usability and design analysis is a feasible and
useful method for comprehending the value of mobile augmented reality as a vehicle
for the visual communication of personally relevant information. To ascertain this
theory, the following questions were posed and answered by means of user studies,
questionnaires and application evaluations.
RQ1: What are mobile augmented reality users' impressions of the technology
before and after using it for the first time?
Users welcome the idea of mobile augmented reality with open arms. They
recognize its potential for revolutionizing social media and information services by
connecting with content on a more personal level. In their opinions, mobile
augmented reality applications could be employed as immersive tour guides for
exploring one’s community or to interact with neighbors whom they are unlikely to
meet by conventional means in a face-‐to-‐face reality.
Additionally, these same users perceive mobile augmented as a new tool for
marketers and companies to employ as immersive advertising and shopping
experiences. Commercial applications of mobile augmented reality could offer
59
guided tours through shops, comparative pricing tools or bargains tailored
specifically for mobile users.
These same users were less optimistic after using a mobile augmented reality
application for the first time. Participants reported that their experience was
hindered by design, contextual and technical limitations. Oftentimes the test
application, Layar, would perform erratically and failed to operate as expected.
Subjects frequently complained about information overload due to the small size of
mobile phones. Additionally, users were dismayed by a general lack of compelling
content. Participants were more interested in seeing non-‐static content such as local
news and social media, rather than commercially based information.
Although mobile augmented reality users are open to using such
applications, they are weary of the privacy risks that come with sharing their
personal information, particularly their current locations. Prior to using any mobile
augmented reality service, users want direct control over what information they are
sharing and or receiving from content providers.
RQ2: What do users and content developers see as current advantages and
limitations of mobile augmented reality?
Mobile augmented reality currently offers little value for smartphone users
and content designers. Technical limitations and a lack of compelling features are
the leading factors for a poor user experience. Consumers frequently complained
that the test application Layar was plagued by slow operating speeds, data
60
inaccuracies and detrimental effects on battery life. Additionally, smaller screens
were found to be inadequate for displaying augmented reality content. Smaller
screens forcibly burden the user by requiring the use of a hand held device as a
point of visual perception when displaying augmented reality content. Furthermore,
using the application within heavily populated areas may result in information
overload for the user since there are limits to how much information can be
efficiently displayed on a smaller screen at one time.
These same technical limitations have constrained the potential for
developing content by designers as well. Mobile phone technologies have not yet
advanced to the point in which they can accurately and efficiently display three-‐
dimensional visualizations. Instead, designers are limited to creating mobile
applications that provide geographically tagged, augmented reality content within a
two-‐dimensional overlay on a camera screen.
Designers consider these limitations to be frustrating and as a result, are
finding that mobile augmented reality has yet to live up to its expectations. Current
applications are being viewed as limited and boring, while consumers are left
wanting. Instead of geographically tagging information one could easily find in the
yellow pages, consumers are more interested in seeing dynamic content such as
local news and events or daily specials currently offered by local business.
Mobile augmented reality does have a potential advantage over other
communication technologies in that it offers an immersive experience by merging
the physical and virtual worlds. However, this advantage has yet to be realized. Both
61
designers and consumers foresee mobile augmented reality having a major
influence on how we play games or access information in the future. But due its
limitations, it remains an underdeveloped technology.
RQ3: Do current information design principles need to be revised in order to apply
them to augmented reality experiences?
A heuristic evaluation of two mobile augmented reality applications revealed
the traditional principles of information design remain applicable for augmented
reality content displayed on hand held devices. Furthermore, these principles can
also serve as an effective benchmark for measuring the viability of mobile
augmented reality as a medium for presenting visual information from a user’s
geographical context.
Using Edward Tufte’s principles of information design for print and
multimedia as judging criteria, several presentation issues were noted ranging from
poor color usage to a failure to properly separate and layer visual information.
Furthermore, the identified limitations were validated by prior focus group
responses pertaining to the same issues. Many of these problems are a result of
limited screen space offered by the mobile device. To rectify this, a conservative
approach is required where only the minimum amount of detail is presented
without loosing context. Tufte refers to this concept as the “Data-‐Ink ratio,” where
the largest use of ink (or pixels) should present data-‐information. Everything else is
expendable (Tufte, 1983). Oversized icons and wasted space in dialog boxes take up
62
valuable screen real estate, which often conceals other sets of information. This is
especially problematic in larger populated areas where the amount of augmented
reality content available outpaces the visual limitations of the small screen. The
resulting effect is information overload for the user.
It is evident applications designers have attempted to make the best
presentation decisions possible within the technical limitations. By following the
suggestions mentioned above, designers can improve the presentation of
information and create a better experience for the user. However, this will not solve
the underlying problem presented by all mobile augmented reality applications: The
ineffective use of a two-‐dimensional content overlay within a three-‐dimensional
space.
Augmented reality content is often displayed within a location-‐based context,
and current mobile applications are failing in this regard. Simply overlaying a dot on
screen to signify a location provides no context to distance or relative proximity to
any nearby points of interest. It merely provides a compass direction. This may
work if there are only one or two information points displayed, but when there are
several present, the metaphor is less effective. Tufte’s micro/macro design principle
states that the smaller details must be visualized in context to the greater whole.
When applied to augmented reality, it’s not enough to place individual location
markers in context with other data points, since it shares a direct relationship with
its environment as well.
63
The user’s environment displayed on screen is visual information that also
needs to be taken into account when it’s augmented with supplemental information.
The principle of layering and separation states that a visual hierarchy should be
established to emphasize which content is most important to the reader, while
deemphasizing what is not. For the user, this means distinguishing which content is
in close proximity in relation to physical objects in their surroundings. The optimum
way to accomplish this is to spatially display data points within three dimensions.
However, this option is not yet feasible due to the technical limitations of modern
mobile technology.
Guidelines for Future Mobile Augmented Reality Applications
While this study has highlighted the shortcomings of mobile augmented
reality, it also recognized potential areas for improvement. Based on user feedback
and heuristic design evaluations of current applications, the following guidelines
would serve to improve the design and user experience of future applications:
Applications should implement navigational controls that are both intuitive
and easy to use. User controls should be easy to navigate with minimum instruction
or prompting. Users find multiple viewing options confusing and would prefer a
single modular interface. Additionally, users would also benefit from a simple visual
cue that highlights the central viewpoint on their mobile devices. By employing a
small box or cross hairs in the middle of the screen, users would better gauge the
geographical location of mobile augmented reality content. This implementation
64
would effectively work in concert with the existing practice of highlighting selected
data points for separation purposes.
Augmented content should be spatially oriented in 3D. Mobile display
technologies will eventually mature to the point that they can accurately display
augmented reality content in 3D. When this occurs, developers should move beyond
the magic mirror approach and seek to seamlessly place points of interest within the
users environment. Such implementations would offer significant advantages over
traditional 2D overlays since it presents an accurate representation of geophysical
locations.
Augmented reality content should be filtered on the basis of user interests and
proximity. Information overload is a significant usability hindrance of existing
applications due to the limited screen space afforded by mobile phones. Developers
must understand these limitations and design around them. Existing applications
are currently addressing this issue by filtering content based on user proximity.
Filtering content based on user interests could strengthen this system, thus
reducing the risk of information overload. Participants in this study expressed their
desire for a filtering features that hiding uninteresting or non-‐applicable
information. A progressive approach would be to devise an educated filter that
learns by means of users marking content as “liked” or “disliked.”
Applications should emphasize social bonding and sharing of information to
create compelling user experiences. Social media has taken a central role in how
people communicate today. Applications like Facebook and Twitter have
65
strengthened social bonds while offering new ways of connecting to new people.
The participants of this study have noted that social media is an important part of
their lives and expect mobile augmented reality applications to take advantage of
this. Furthermore, these same users are also interested in seeing local content that
is both dynamic and useful such as restaurant specials or community events. If this
content is both useful and engaging, users are more likely to share it with friends,
thus expanding an application’s active user base.
Information should be interoperable between all applications. If augmented
reality is to become a successful medium for information delivery, there needs to be
open standards for sharing content among all applications. This current lack of
interoperability burdens content developers, as they must reformat information for
multiple augmented reality applications. It’s especially problematic when
formatting dynamic content that must be updated on a regular basis. Consumers
would also benefit from standardization. In the future, users will be the driving force
in generating new augmented reality content by means of geographically tagging
social media and personally relevant content. These users will be able to reach a
wider audience if tagged information can be shared between multiple applications.
Limitations
Although this research helps to explain how users and designers perceive
mobile augmented reality, there were several limitations. The first was related to
the participant sample for the user focus groups. Participants in this study were
pooled from a smaller audience (college students) rather than addressing all
66
demographics representing smart phone users. A second limitation was that while
student participants tested the Layar application in both rural and urban settings,
the primary testing location was in a smaller community. Since the analysis of this
study suggests that population density has a direct effect on the user experience, a
second sample derived from residents in a metropolitan area would have been
beneficial.
Additional limitations were related to the implementation of the heuristic
evaluation of the Layar and Junaio applications. Ideally, multiple expert reviewers
are employed to access and document the limitations of an application when
applying the heuristic evaluation method. Due to time and financial constraints, the
evaluation was limited to the findings and opinions of a single researcher. Although
the final analysis has merit, multiple evaluators could have further validated the
results.
Furthermore, Edward Tufte’s information design principles required
modification in order to apply them to mobile augmented reality. While Tufte’s
principles are applicable to this medium, they were originally intended for print or
static mediums. In his original works, Tufte never addressed interactivity or spatial
design within a 3D environment. While Tufte’s guideline for presenting information
within a 3D projection presented in 2D proved applicable to augmented reality, it’s
meaning had to be inferred when applied to spatial environments.
67
Recommendations for Future Research
As mobile augmented reality is relatively young, there are many avenues for
future research in this area. New applications are introduced on a weekly basis and
mobile technologies continue to improve every year. Future studies would benefit
by examining how users perceive and interact with other forms of mobile
augmented reality applications besides information browsers. Applications in social
media, gaming and navigation are worthy of a closer look. Furthermore, as
augmented reality applications are refitted for tablet computers, it would be
valuable to reassess the limitations found in this study’s analysis in context to
mobile phones to see if they still apply.
Finally, it would be advantageous to expand the designer’s perspective
component into a full research study. The initial research in this area suggests that
designers have a central role in defining the direction mobile augmented reality
takes. Their thoughts and opinions would be valuable to future studies exploring
mobile augmented reality content.
Conclusion
Mobile augmented reality is on the cusp of revolutionizing how people
interact with digital information in their daily lives. Content will no longer be tied to
a specific medium such as print or the web; rather it will seamlessly blend with
one’s surroundings. This disruptive technology will soon redefine how consumers
engage with news, entertainment, and other forms of data on a daily basis. In all
68
respects, mobile augmented reality may one day become the ultimate immersive
user experience.
Nevertheless, mobile augmented reality requires time to mature. Just as
every disruptive technology that has come before, mobile augmented reality must
address its initial flaws and limitations if it’s to become a commercial success. While
augmented reality has demonstrated that it can be an innovative medium for
displaying information, it’s mobile counterpart has yet to create a sense of “need”
for consumers. Further advancements in content development are necessary in
order to generate universal acceptance and demand and to shed the “gimmicky”
perceptions that users and content designers have about this technology.
Additionally, mobile technologies and applications have not advanced to the
point in which they can deliver on augmented reality’s promise of immersive
information delivery. Despite the hype generated by marketers and corporations,
current hand-‐held devices are incapable of displaying pure augmented reality
experiences. By definition, augmented reality systems must be able to register and
align virtual and physical objects in relation to each other within a 3D space. Current
mobile offerings such as Layar or Junaio fail to meet this criterion, as they instead
rely on visual trickery by employing a two-‐dimensional overlay on top of a
smartphone camera-‐viewfinder. To that end, consumers are left with a “magic
mirror,” which plasters virtual content over their small screens rather than fully
integrating within the user’s surroundings.
69
Although mobile augmented reality has been limited by technical constraints,
the recent arrival of faster processers and larger displays will have a profound
impact on the depth and viability of future applications. Tablet computers such as
the iPad are beginning to redefine how developers approach augmented reality
content. These devices are equipped with powerful 3D graphics hardware and
larger screens, which move beyond the limitations of mobile phone displays.
However, tablet and mobile devices are only the next evolution, not the final step in
mobile augmented reality’s development. Most designers and users find using a
hand-‐held device to be cumbersome since it restrains their visual perception to the
device’s limited perspective. Eventually head-‐worn displays such as augmented
reality glasses will become the standard. Users will gain unparalleled access to
content from a first-‐person viewpoint, thus eliminating the need for employing
hand-‐held displays as windows for viewing augmented reality content.
Furthermore, augmented reality systems will expand to allow users to interact with
virtual content in the same way they would use physical objects by means of
gestural interfaces.
Once mobile augmented reality moves beyond its limitations, the potential
benefits for information delivery and entertainment are enormous. Mobile
augmented reality could be used to hyper-‐localize content such as dining reviews or
present statistical overlays during live sporting events. Additionally, the immersive
nature of augmented reality could also be used to pull gamers and readers into
stories by interacting with virtual characters in their own space in real time. The
70
opportunities for content designers and users alike are potentially limitless. Unlike
overhyped technologies of the past, mobile augmented reality has the greatest
potential for redefining how consumers view the world by merging the physical and
virtual into a single reality.
71
REFERENCES
AIGA (2004). 2004 AIGA Medaliists -‐ Edward Tufte. Retrieved January 11, 2011,
from AIGA web site: http://www.aiga.org/content.cfm/medalist-‐edwardtufte
Aukstakalnis, S. & Blatner, D. (1992). Silicon Mirage -‐ The Art and Science of
Virtual Reality. Berkeley, CA: Peachpit Press. Azuma, R. (1997, August). A Survey of Augmented Reality. Presence: Teleoperators
and Virtual Environments, 355-‐385. Azuma, R., Baillot, Y., Behringer, R., Feiner, S., Julier, S., MacIntyre, B. (2001).
Recent Advances in Augmented Reality. Computers & Graphics, November 2001.
Baer, K. (2009). Information Design Workbook: Graphic Approaches, Solutions, and
Inspirations + 30 Case Studies. Beverly, MA: Rockport Publishers. Boyatzis, R. (1998). Transforming Qualitative Information. Thousand Oaks, CA: SAGE
Publications. Bryne, C. (2010, December 22). Most augmented reality companies not doing
augmented reality? Retrieved January 13, 2011, from VentureBeat: http://venturebeat.com/2010/12/22/forrester-‐most-‐augmented-‐reality-‐companies-‐not-‐doing-‐augmented-‐reality/
Braun, V., & Clarke, V. (2006). Using thematic analysis in psychology. Qualitative
Research in Psychology, 77-‐101. Chen, B. (2009, August 25). If You’re Not Seeing Data, You’re Not Seeing. Retrieved
April 17, 2010, from Wired: http://www.wired.com/gadgetlab/2009/08/augmented-‐reality/
Dervin, B. (1999). Chaos, Order and Sense-‐Making: A Proposed Theory
for Information Design. In R. Jacobson, Information Design (pp. 35-‐57). Cambridge: MIT Press.
Dünser, A., Grasset, R. & Billinghurst M. (2008). A Survey of Evaluation Techniques
Used in Augmented Reality Studies. HIT Lab NZ Technical Report.
72
Entner, R. (2010, June 30). Quantifying the Mobile Data Tsunami and its
Implications. Retrieved July 19, 2010, from Nielsen: http://blog.nielsen.com/nielsenwire/online_mobile/quantifying-‐the-‐mobile-‐data-‐tsunami-‐and-‐its-‐implications/
(2009, December). Esquire Magazine . Gotow, J., Zienkiewicz, K., White, J., Schmidt, D. (2010). Addressing Challenges in
Delivering Augmented Reality Applications to Smartphones. Third International ICST Conference on MOBILe Wireless MiddleWARE, Operating Systems, and Applications. Chicago: Mobileware.
Haller, M., Billinghurst, M., & Thomas, B. (2006). Emerging Technologies of
Augmented Reality: Interfaces and Design. Hershey, PA: Idea Group Publishing.
Horn, R. (1999). Information Design: Emergence of a New Profession. In R. Jacobson,
Information Design (pp. 15-‐33). Cambridge: MIT Press. Hurley, W. (2009, November 3). Augmented Reality: Getting Beyond the Hype.
(Bloomberg) Retrieved January 30, 2011, from Businessweek: http://www.businessweek.com/technology/content/nov2009/tc2009112_353477.htm
Husson, T. (2010, December 20). Mobile Augmented Reality: Beyond The Hype, A
Glimpse Into The Mobile Future. Retrieved January 9, 2011, from Forrester Blogs: http://blogs.forrester.com/thomas_husson/10-‐12-‐20-‐mobile_augmented_reality_beyond_the_hype_a_glimpse_into_the_mobile_future
Junaio. (2011). (Metaio, Producer) Retrieved from http://www.junaio.com/ Kato, H. & Billinghurst, M. (1999). Marker tracking and hmd calibration for a video
based augmented reality conferencing system. 2nd IEEE and ACM International Workshop on Augmented Reality, (pp. 85-‐94).
Kirkpatrick, M. (2009, August 24). Augmented Reality: 5 Barriers to a Web That's
Everywhere. Retrieved April 2, 2010, from Readwriteweb: http://www.readwriteweb.com/archives/augmented_reality_five_barriers_to_a_web_thats_eve.php
73
Maes, P. & Mistry, P. (2009, November). Pranav Mistry: The thrilling potential of SixthSense technology. Retrieved March 9, 2010, from TED: http://www.ted.com/talks/pranav_mistry_the_thrilling_potential_of_sixthsense_technology.html
Metz, R. (2009, October 4). Another use for your phone: augmented reality.
Retrieved from Associated Press. Milgram, P. and Kishino, F. (1994, September). A Taxonomy of Mixed Reality Virtual Displays. IEICE Transactions on Information and Systems. Nielson, J. & Molich, R. (1990). Heuristic evaluation of user interfaces. CHI '90
Proceedings of the SIGCHI conference on Human factors in computing systems: Empowering people, (pp. 249-‐256).
Rogers, E. (2003). Diffusion of Innovations (5th ed.) New York: Free Press. Rutledge, P. (2010, August 26). Augmented Reality: Who wants more reality?
Retrieved January 22, 2011, from Media Psychology Research Center: http://mprcenter.org/blog/2010/08/26/who-‐wants-‐more-‐reality/
Sharp, H., Rogers, Y. & Preece, J. (2007). Interaction Design: Beyond Human-‐Computer
Interaction. West Sussex, England: John Wiley & Sons. Schmalstieg, D., Langlotz, T., and Billinghurst, B. (2011). Augmented Reality 2.0.
In Virtual Realities. New York: Springer Vienna. Schroeder, S. (2010, September 29). How Popular Is the iPhone, Really?
[INFOGRAPHIC]. Retrieved February 4, 2011, from Mashable: http://mashable.com/2010/09/29/popular-‐iphone-‐infographic/
Shedroff, N. (1999). Information Interaction Design: A Unified Field of Theory of
Design. In R. Jacobson, Information Design (pp. 267-‐292). Cambridge: MIT Press.
Sterling, B. (2009, August 20). At the Dawn of the Augmented Reality Industry.
Retrieved March 29, 2010, from Layar conference keynote (video): http://site.layar.com/company/blog/video-‐bruce-‐sterlings-‐keynote-‐at-‐the-‐dawn-‐of-‐the-‐augmented-‐reality-‐industry/
Swan J., & Gabbard J, (2005). Survey of User-Based Experimentation in Augmented
Reality, 1st International Conference on Virtual Reality, Las Vegas.
74
Toto, S. (2010, March 15). Crimsonfox: Augmented reality-‐powered scavenger hunt n Tokyo. Retrieved January 29, 2011, from CrunchGear: http://www.crunchgear.com/2010/03/15/crimsonfox-‐augmented-‐reality-‐powered-‐scavenger-‐hunt-‐in-‐tokyo-‐video/
Tufte, E. (1997). Visual Explanations: Images and Quantities, Evidence and Narrative.
Cheshire: Graphics Press. Tufte, E. (1990). Envisioning Information. Cheshire: Graphics Press. Tufte, E. (2008). iPhone video. Retrieved March 1, 2011, from The Work of Edward
Tufte and Graphics Press: http://www.edwardtufte.com/bboard/iphone-‐video.adp
Tufte, E. (1983). The Visual Display of Quantitative Information.
Cheshire: Graphics Press. Uger, R. & Chandler, C. (2009). A Project Guide to UX Design: For User Experience
Designers in the Field or in the Making. Berkeley, CA: New Riders. Vaughan-‐Nichols, S. (2009). Augmented Reality: No Longer a Novelty?.
IEEE Computer, 4. Wagner D., L. T. (2008). Robust und Unobtrusive Marker Tracking on Mobile Phones.
ISMAR’08, (pp. 121–124) Wen, H. (2010, January 26). Augmented reality: Pure hype or Next Big Thing in
mobile? Retrieved December 19, 2010, from Computerworld: http://www.computerworld.com/s/article/9145418/Augmented_reality_Pure_hype_or_Next_Big_Thing_in_mobile_
Zhou, F., Duh, H. & Billinghurst, M. (2008). Trends in Augmented Reality Tracking,
Interaction and Display: A Review of Ten Years of ISMAR. ISMAR 2008. Cambridge.
Zimmermann, B. (1997). Applying Tufte's principles of information design to
creating effective Web sites. 15th annual international conference on Computer documentation (pp. 309-‐317). New York: ACM.
75
APPENDIX A: FOCUS GROUP DISCUSSION GUIDE PROTOCOL
Session 1 Introduction (10 minutes)
Explanation of Focus Group: Today I am going to be introducing to you a mobile augmented reality application called Layar. Over the next 1 ½ hours we are going to be discussing, sometimes individually, and other times as a whole group your thoughts about augmented reality and the Layar application.
To begin, we’re going to introduce ourselves. If you would, please state your name, major, year in school and a little bit about the types of mobile apps you typically use.
Activity: Presentation of Augmented Reality and Layar concept videos (15 minutes) Participants will be presented with two videos about augmented reality and the Layar mobile application, produced by Layar. Video 1: http://www.youtube.com/watch?v=b64_16K2e08 Video 2: http://www.youtube.com/watch?v=EtpNx7Y14d0&feature=related
After watching the video clips, the moderator will open the floor to discussion asking participants what do they think of augmented reality and the Layar app based on what they just watched.
Discussion: First impressions of mobile augmented reality (40 minutes)
The moderator will facilitate discussion about the participants first impressions about mobile augmented reality and the application, Layar by utilizing the following probing questions:
• Does Layar seem like an application you would be interested in trying out for
yourself? Why or Why not?
• How do you think you might use this application?
• Are you more likely to use Layar for tourism, social networking, shopping, or games? Why did you choose that category?
• Can you think of any other potential augmented reality applications you would like to see? How would they work, and why would you want to use it?
76
• Can you think of reasons why you would not use an application like Layar?
Assignment: Testing out the Layar application (10 minutes)
Participants will be provided directions on how to download the Layar application to their smart phone, and asked to try out the application for a period of at least two hours over the next week. The focus group will reconvene a week from today.
Session 2 Introduction (5 minutes)
Explanation of Focus Group: Today we are going to share and discuss your experiences using the Layar mobile reality application. What I’m interested in hearing from you are you’re your likes and dislikes about the app, how you used the app, and whether or not you intend to continue using this app in the future.
Discussion: First impressions of mobile augmented reality (70 minutes)
The moderator will facilitate discussion about the participants’ first experiences using a mobile augmented reality application, by utilizing the following probing questions: • So, what did you think of the Layar application now that you have had a chance
to try it out?
• Did the application work as you expected it to? If not, what surprised you?
• Did you have any problems using the application? If so, what were they?
• If you could change one aspect of the application or add one feature, what would it be?
• Please fill in the blank. It would be really cool if an augmented reality application could do __________.
• If you were to recommend Layar to a friend? Who would it be, and why do you think they would like using it?
• Do you plan to continue using this application in the future? If not, why?
77
APPENDIX B: DESIGNER PERCEPTIONS QUESTIONARE
Name: Occupation: 1. What types of content do you think works best in an augmented reality application? 2. How do you utilize mobile augmented reality as a medium for content creation? What are your current plans on using augmented reality in the future?
3. Please fill in the blank. It would be really cool if an augmented reality application could do __________.
4. How is the design process for creating augmented reality applications different from other forms of interactive media?
5. What do you see as the advantages and limitations to developing augmented reality information and storytelling based applications?
6. What are the must have elements to every augmented reality design?
7. What do you think are mobile augmented reality’s greatest design challenges?
8. In what direction do you see augmented reality taking information design and digital storytelling in the future?
78
APPENDIX C: HEURISTIC EVALUATION PROTOCOL
This document provides a detailed analysis of the heuristic evaluation of two mobile augmented reality applications: Layar and Junaio. Evaluation criteria were derived from established principles of information design written by Edward Tufte in his books: The Visual Display of Quantitative Information (1983), Envisioning Information (1990) and Visual Explanations (1997). Problems and instances are noted for each applied principle on the basis of structure, presentation and dynamics in the application design. Evaluative heuristics
1. Individual elements are presented within a framework relating to a larger context
2. Effectively layers and separates information to emphasize hierarchy and structure
3. Employs small multiples as a means of comparing differences between related data
4. Effectively uses color for information display purposes 5. Integrates words and images effectively
Severity rankings 1 — Minimum design problem that occurs infrequently and/or with minor user hindrance. 2 — Moderate design problem that occurs occasionally and may result in readability issues. 3 — Severe design flaws that occurs frequently and damages the readability of data.
79
Evaluation 1: Layar
# Problem Severity Rating
Heuristic Number Broad Heuristic
1 Oversized icons cover up the background (camera view) and other data points.
3 1, 2 Micro design elements should not overshadow the main view.
2
“Bird’s eye” view feature results in data point overlap and diminishes location-‐based context.
2 2
Data should be appropriately layered in proximity to the user’s location
3 Difficult to recognize which tweet is currently selected 2 2, 3
Selected objects should be highlighted. Icons should be simple and consistent.
4
A visual disconnect exists between data points and their related text and/or images
2 5
Data points should have strong visual associations with related images and text.
5 Large text boxes take up a considerable amount of screen real estate.
2 1
The display of secondary content should not overpower the framework of the bigger scene.
6 White text on bright colored backgrounds difficult to read.
1 4 Poor color selection and lack of contrast.
80
Evaluation 2: Junaio # Problem Severity
Rating Heuristic Number Broad Heuristic
7
User interface controls do not orient correctly when viewing in landscape mode
3 3 User interface should be consistent throughout the application.
8
Expanded dialog boxes are presented sideways when using application in landscape mode
3 5 Text presented sideways is difficult to read.
9
Radar size is inadequate for presenting surrounding information in 360 degrees.
2 3 Difficult to separate content effectively on a smaller scale.
10 Text displayed within individual data points is difficult to read
1 5
Reversing type out of dark backgrounds requires strong color contrast and larger type sizes.