-
8/8/2019 3D Interfaces for Mobile Phones_public_20080529
1/25
-
8/8/2019 3D Interfaces for Mobile Phones_public_20080529
2/25
3D User Interfaces for Mobile
Phones
1. Introduction
This document explores the use of 3D graphics in user interfaces (UIs) on mobiledevices. We present a framework for analyzing and working with 3D in UIs, point toinspirational examples, and give suggestions and benefits of various 3D enabledinterfaces.
Working with this report we've looked at and benchmarked existing examples of 3D usagein user interfaces among competitors and in TATs own products. From testing andanalyzing these examples we derived a practical framework that encompasses principal
features and benefits of 3D interfaces.
Typically the palette of "3D techniques" has been comprised visualizations of lightning,shadows, focus, depth, camera angles and similar. Based on our exploration, we conceivethat 3D is more than 3D graphics. We understand "3D" in user interfaces as a paradigmwhere the appearance of the interface displays three-dimensional or real world qualities,while the technology behind the visualization is not necessarily built up as a real 3dimensional simulation. To take benefit of the full potential of 3D in mobile user interfaceswe also need to consider user input, perception and understanding as important parts thatadd to the total user experience.
2
-
8/8/2019 3D Interfaces for Mobile Phones_public_20080529
3/25
2. 3D put in the bigger picture of mobile UIs
It is important to recognize that the time for 3D in the mobile market may not have arrived
yet. As mobile technology drives forward UI design must consider what functionality and
visualizations can be displayed appropriately.
We have recently moved from phones like the Sony Ericsson T610 with indirect input(keys) and a color graphical output to the current time of the iPhone. The major changes inthe UI are partially enabled by the change from indirect input to direct input (touch).
As the mobile industry moves towards a ubiquitous future, where the UI is adaptive andcontextually sensitive and reactive, we can project that a well thought out 3D interface,represented with physical properties such as gravity or shadow, has a chance to blur theborders between the real world and the UI space. Though its not suggested that 3D is
the only future, as handsets and user expectations of mobile experiences evolve, 3D canplay a large part in expanding our existing toolkit in addressing the needs of the user.
3
-
8/8/2019 3D Interfaces for Mobile Phones_public_20080529
4/25
3. Framework for analyzing and using 3D inmobile user interfaces
The framework is a tool for orienting a project in the conceptualization and design phases.
It offers a means of recognizing the needs of a current project and relating them to generalaspects and benefits of 3D in user interface solutions. By concrete examples and benefitsit also serves as device for enhancing communication when working on any projectinvolving 3D.
We have found that there are three major aspects of 3D that can be utilized in the designof applications and UIs:
Flexible Information Visualization Naturalized Interaction Visual Style & Feedback
Flexible Information Visualization (FIV)Provides the possibility of scalable and pliable UIs to organize data-sets ranging fromsmall to endless amounts. In contrast to 2D UIs which thrive by displaying small amountsof data or utilizing search functions, 3D UIs can give scalable overviews and exploitvarious navigational metaphors.
Naturalized Interaction (NI)Using 3D for representations that builds upon innate human understanding of space andphysical materiality of objects to offer a set of meaningful and easily recognized interaction
possibilities. By using 3D to support these tacit understandings, the cognitive load of theuser is reduced, and the use and learning of an application or interface is more intuitive.
Visual Style & Feedback (VSF)As a stylistic element, 3D offers the possibility of product differentiation and an enrichedvisual language that offers both functional and aesthetic appeal.
While the three major aspects gives a more general understanding of the range of qualitiesthat 3D enables, each aspects encompasses a set of more concrete potential benefits. Inthe following section, we highlight these benefits together with a set of examples that wellillustrates the benefit. Please note that the examples used often show benefits in more
than one aspect or category, although not always explicitly stated in this section. For amore comprehensive description the examples, please turn to the appendix.
4
-
8/8/2019 3D Interfaces for Mobile Phones_public_20080529
5/25
3.1 Flexible Information Visualization - Benefits and Examples
3.1.1 De-clutter screen spaceThe use of depth, multi-axis rotation, point of view, and assumed persistent spatialmemory can allow for information to be visualized, so as to reduce the need of overloadingthe screen with all information at one time.
Tagging an image in Perceptive Pixel.
3.1.2 Optimize screen space
A successful use of object rotation or movement and lens choice is one way to allow for
the most pertinent information to be given proper focus, while still allowing the user to havemany choices available in the margins.
Navigating the music library in iTunes by Cover Flow or alist.
5
-
8/8/2019 3D Interfaces for Mobile Phones_public_20080529
6/25
3.1.3 Add focus through depth of field
While a 2D UI allows for layering or stacking, adding depth of field can allow for focus tosingle points or planes of information.
Navigating a collection of RSS feeds by time in RSSVoyage. Entries from selected timeframe appear in focuswhile older entries are blurred in the background.
3.1.4 Provide means for zoom and overview
When displaying large amounts of data there are times when total overview of theavailable options is desired. Zooming outwards can provide overview, while zooming incan show details or options which are more specific to a certain object or application.
Switching between applications with Apples active
corners
6
-
8/8/2019 3D Interfaces for Mobile Phones_public_20080529
7/25
3.1.5 Create alternative relationships and context between objects.
The relation between information changes the way in which it is navigated and perceived.3D allows for many more relations to be established.
Google search results presented by Tianamo, mapped aspeaks with diverse spatial relationships between differentresults.
7
-
8/8/2019 3D Interfaces for Mobile Phones_public_20080529
8/25
3.2 Naturalized Interaction - Benefits and Examples
3.2.1 Show affordances in 3D objects to suggest possibilities for actions and interactions
We can go beyond even the affordances established in industrial design with objects tolook at how the elements of space, itself, guides actions and behavior.
In Call of Duty fog and other 3D elements guide useractions.
3.2.2 Show properties on applications and information through physical properties (utilizingphysics engines, textures)
We can create ambient and intuitive metaphors for certain kinds of information, such asstatus of a file transfer or relevance in an inbox, without explicitly labeling them by usingmaterial qualities or physical forces like gravity which were not available or as appropriatein 2D.
Bumptop desktop showing importance ofobjects through size.
Bumptop desktop showing character of objects byreshaping.
8
-
8/8/2019 3D Interfaces for Mobile Phones_public_20080529
9/25
3.2.3 Reducing cognitive load by providing familiar and comfortable spatial metaphors.
A 3D spatial metaphor can range from literal representations to a more abstract use ofspace. Apples widgets backside flip to show more information finds a nice balance toallow the organization of information to make sense to the user.
Bumtop desktop. When moving objects, only one object need to be selected to move severalobjects. When the selected object is thrown towards other objects, they behave according tophysical laws and move according to the force of the thrown object.
3.2.4 Flattening learning curve of commands
When using indirect input, such as a keypad, we utilize a tool. There is often a learningperiod before the user can remember commands without having to be prompted. Duringthat time the user is always thinking about their next step.
3D makes sense with more natural input methods and commands as the visualized resultsof our actions is more in parallel with what we would expect in the natural world, or can beassociated quickly to a physical memory. A successful intuitive tool, such as PerceptivePixel, allows for the sense of input as a tool to disappear and be more of an extension ofthe users body.
Active corners on MacOSX. By dragging themouse towards corners of the screenseveral views can accessed, in a way that iseasy to recall (and discover).
Multi-touch in Perceptive Pixel. Zooming actionsperformed directly on the surface by simplegestures.
9
-
8/8/2019 3D Interfaces for Mobile Phones_public_20080529
10/25
3.3 Visual Style and Feedback - Benefits and Examples
3.3.1 Allows for greater differentiation of products
Adding a set of simple 3D properties, such as skewing or tilting, zooming, and physicalproperties such as momentum, can greatly differentiate a product from another. Even if
content is the same. This can be seen in the Firefox plugin PicLens.
This particular benefit might be considered temporary until more UI work is done in 3D, butthe options for representation are exponentially expanded by the added dimension.
Image search with PicLens plugin for Firefox, presentedon a 3D wall. The content is taken from a standardimage search engine, such as Google Image Search.
3.3.2 Adds a wow-effect
Often the wow-effect is achieved by utilizing physical properties of the real world, andexceeding what is really possible in the representation. Use 3D to enable the unexpected.
iTunes Cover Flow. TATs Photo River 3D.
10
-
8/8/2019 3D Interfaces for Mobile Phones_public_20080529
11/25
3.3.3 Create visual immersion
The use of depth and well thought out 3D graphics can create a more immersivevisualization than a 2D UI. 3D allows for more focus to be shifted into experiencing thespace between information or objects.
Actfaces Rhythm concept. Upon navigation and selections visual effects create a pleasant and eyecatching experience that adds to immersion in the use.
3.3.4 Possibility of more visually holistic UIs with seamless edges
When trying to achieve a seamless experience, pop-ups, new frames, or shifting betweenfunctions can be hard to negotiate visually.
3D Desktop with touch screen on Linux. Flipping between different stages with seamlessboundaries.
11
-
8/8/2019 3D Interfaces for Mobile Phones_public_20080529
12/25
123.3.5 Spatial movements gives process feedback
A value of the user waiting for an application to load, or a connection to be made, can beestablished by providing more immersive graphics which in some way represent thenormally invisible process which is occurring.
Activating different content and applications on NVIDIA Shiny. When selecting a new window,transition effects give feedback of the process (compare to situations where all computation takesplace in the background an application is ready to launch).
12
-
8/8/2019 3D Interfaces for Mobile Phones_public_20080529
13/25
3.4 Mapping of existing products and concepts in the framework
We can use the three major aspects of 3D, Flexible Information Visualization (FIV),Naturalized Interaction(NI), and Visual Style & Feedback(VSF), as vertexes in a space tomap 3D UIs. By doing this we can see where the strengths are in a particular design, and
which areas may be lacking.
The Perceptive Pixel concept isvery balanced between allelements in NI, FIV, and VSF. Itis a complex system and hasneeded to consider all elements
in its design.
Coverflows primary strengths lie in itsvisual appeal and the naturalizedinteraction.
TATs 3D demosreflect thecompanysstrength indeveloping
visual style.
(See appendix for links to all examples)
13
-
8/8/2019 3D Interfaces for Mobile Phones_public_20080529
14/25
4. Communicating With the 3D Layman
The 3D UI Framework is not only a tool for designers, but can be used to bettercommunicate with all people, some of whom know what they want in a UI, but find it
difficult to articulate their desires.
Flexible Information Visualization (FIV) describes the elements of 3D which allow a UI tobetter display large amounts of data, like playlists, in interesting ways relevant to the userand to the application being used.
Naturalized Interaction (NI) describes the elements of a 3D UI which minimize how much auser needs to think about how theyre using an application, and allows them to simply useit.
Visual Style and Feedback (VSF) describes how 3D can add a wow effect to a productand better differentiate itself from competitors.
4.1 Understanding needs by comparisonBy understanding where other 3D concepts and products are mapped, the conversationwith the layman does not have to rely on this terminology, as they can speak freely incomparisons.
I really want something like PhotoSynth.or I like the way PicLens shows pictures.
Now both of you can refer specifically to which 3D elements should be concentrated on.
However, if youre working together on defining a project, it might be a good idea to plotthe established needs and values to show the specific benefits of FIV, NI, and VSF and to
justify certain design choices or suggestions.
4.2 Offering benefits and solutionsThe framework can also be used to check if what they want to achieve is in linewith their visual expectations, or assumptions about what it should look and behave like. Ifsomeone is talking about a highly immersive environment with avatars (NI) because theywant to better visualize massive amounts of information (FIV), the reasoning may not have
been thoroughly considered yet, and it presents an opportunity to show a morecomprehensive solution which incorporates more appropriate qualities of 3D.Communicating around specific benefits and results of using 3D also allows theopportunity to ask pointed questions, and boil the design problem down to its core values.
14
-
8/8/2019 3D Interfaces for Mobile Phones_public_20080529
15/25
5. Summary
3D is not a solution or a goal in itself. What it offers us is an opportunity to expand ourtoolkit and our offerings. The tools it offers us can be complex and nearly endless, as they
get closer to representing the real world.
In the mobile world, 3D is still new ground, and most fall back on old tricks. What isneeded is to understand what the different tools of 3D will specifically mean to the userexperience.
The framework of FIV, NI, and VSF are a beginning to gaining this understanding, andrepresent a broader understanding of the benefits of 3D in the UI.
Remember that 3D is more than 3D graphics.
15
-
8/8/2019 3D Interfaces for Mobile Phones_public_20080529
16/25
6. Appendix
This appendix presents a list of examples which demonstrate the benefits of NaturalizedInteraction (NI), Flexible Information Visualization (FIV), and Visual Style & Feedback(FSV). The QR-Code allows you to link to examples with your mobile as you read the
document.
Apple Cover Flow
Cover Flow is integrated with iTunes and allowsthe user to visually rummage through files anddigital media libraries.
Benefits: Strong in VSF & FIV
http://www.youtube.com/watch?v=1nCeJWxp4yk&feature=related
Apple OS X Expose
Expose gives the user a simple way to show alltheir overlapping layers of content.
Benefits: Strong in FIV and NI
http://www.youtube.com/watch?v=XwX6BbeDgRk
16
http://www.youtube.com/watch?v=XwX6BbeDgRkhttp://www.youtube.com/watch?v=XwX6BbeDgRkhttp://www.youtube.com/watch?v=XwX6BbeDgRkhttp://www.youtube.com/watch?v=XwX6BbeDgRkhttp://www.youtube.com/watch?v=1nCeJWxp4yk&feature=relatedhttp://www.youtube.com/watch?v=1nCeJWxp4yk&feature=relatedhttp://www.youtube.com/watch?v=1nCeJWxp4yk&feature=relatedhttp://www.youtube.com/watch?v=1nCeJWxp4yk&feature=related -
8/8/2019 3D Interfaces for Mobile Phones_public_20080529
17/25
BumpTop
BumpTop is meant to enhance the desktopmetaphor by supporting normal behaviour of a realworld desk.
Benefits: Strong in NI
http://www.youtube.com/watch?v=M0ODskdEPnQ
Call of Duty / 3D Gaming
3D games give a strong sense of immersion dueto their consideration of creating environments
representing objects with realistic physicalproperties.
Benefits: Strong in NI, VSF
http://www.youtube.com/watch?v=LO4caTLqqoM
17
http://www.youtube.com/watch?v=LO4caTLqqoMhttp://www.youtube.com/watch?v=LO4caTLqqoMhttp://www.youtube.com/watch?v=M0ODskdEPnQhttp://www.youtube.com/watch?v=M0ODskdEPnQ -
8/8/2019 3D Interfaces for Mobile Phones_public_20080529
18/25
HTC Diamond Touch
A 3d environment displaying and organising video.
Benefits: Good elements of VSF
http://www.youtube.com/watch?v=vvpYJsV0fk0
Microsoft Photosynth
Photosynth takes a large collection of photos of a
place or an object, analyzes them for similarities,and then displays the photos in a reconstructedthree-dimensional space, showing how each onerelates to the next.
Benefits: Strong in VSF & NI
http://www.ted.com/index.php/talks/view/id/129
18
http://www.ted.com/index.php/talks/view/id/129http://www.ted.com/index.php/talks/view/id/129http://www.youtube.com/watch?v=vvpYJsV0fk0http://www.youtube.com/watch?v=vvpYJsV0fk0 -
8/8/2019 3D Interfaces for Mobile Phones_public_20080529
19/25
NVIDIA Shiny Demo
The Shiny demo demonstrates how processfeedback can be represented in a 3D space,moving closer towards a holistic user experience.
Benefits: Strong in VSF
http://www.youtube.com/watch?v=8kLFPfaxQ6U
Perceptive Pixel
Multi-touch in Perceptive Pixel. Zooming actionsperformed directly on the surface by simplegestures.
Benefits: Strong in FIV, NI, & VSF
http://www.youtube.com/watch?v=tr_RgOTum3M
19
http://www.youtube.com/watch?v=tr_RgOTum3Mhttp://www.youtube.com/watch?v=tr_RgOTum3Mhttp://www.youtube.com/watch?v=8kLFPfaxQ6Uhttp://www.youtube.com/watch?v=8kLFPfaxQ6U -
8/8/2019 3D Interfaces for Mobile Phones_public_20080529
20/25
PicLens
PicLens provides an alternative view of image andvideo searches on a 3D wall and allows for amore immersive browsing experience.
Benefits: Strong in FIV, NI, & VSF
http://www.youtube.com/watch?v=FjetHe-DTVs
Photoboard
Photoboard allows iPhone users to navigate,scale, and organize images in layers.
Benefits: Good use of VSF, NI, and FIV
http://www.youtube.com/watch?v=GJ7BdniMeck
20
http://www.youtube.com/watch?v=FjetHe-DTVshttp://www.youtube.com/watch?v=GJ7BdniMeckhttp://www.youtube.com/watch?v=GJ7BdniMeckhttp://www.youtube.com/watch?v=FjetHe-DTVshttp://www.youtube.com/watch?v=FjetHe-DTVs -
8/8/2019 3D Interfaces for Mobile Phones_public_20080529
21/25
PureDepth
PureDepth Multi-Layer Display brings actual depthto display devices of any size, by layering multipleLCD screens on top of each other.
Benefits: Excellent VSF
http://www.youtube.com/watch?v=8UOY4kG_CGA
RSS Voyage
The user can navigate RSS feeds chronologically
by bringing headers in and out of focus in a 3Dspace.
Benefits: Strong in FIV
http://www.rssvoyage.com
21
http://www.youtube.com/watch?v=JD6BiKnLzck&NRhttp://www.youtube.com/watch?v=JD6BiKnLzck&NRhttp://www.youtube.com/watch?v=8UOY4kG_CGAhttp://www.youtube.com/watch?v=8UOY4kG_CGA -
8/8/2019 3D Interfaces for Mobile Phones_public_20080529
22/25
Screenvader
Screenvader creates a curved navigation UI in a
3D space, and focuses information with depth offield.
Benefits: Excellent use of VSF
http://www.screenvader.com/root.html
Searchme
3D web navigation takes advantage of 3D spaceto organize content.
Benefits: Good elements FIV
http://www.youtube.com/watch?v=TsGR5HP2ffw
22
http://www.youtube.com/watch?v=TsGR5HP2ffwhttp://www.youtube.com/watch?v=TsGR5HP2ffwhttp://www.screenvader.com/root.htmlhttp://www.screenvader.com/root.html -
8/8/2019 3D Interfaces for Mobile Phones_public_20080529
23/25
TATs 3D Concepts
TATs 3D concepts demonstrate the companysfocus in presenting visually stunning graphics andanimation in mobile UIs.
Benefits: Very strong in VSF
http://www.tat.se/conceptlab/
Tianamo
Tianamo is a browser-based, visual interface forsearching, exploring and interacting generally withstructured and unstructured information.
Benefits: Strong in FIV
http://www.youtube.com/watch?v=2i4Njh1w8DU
23
http://www.tat.se/conceptlab/http://www.youtube.com/watch?v=2i4Njh1w8DUhttp://www.youtube.com/watch?v=2i4Njh1w8DUhttp://www.tat.se/conceptlab/http://www.tat.se/conceptlab/ -
8/8/2019 3D Interfaces for Mobile Phones_public_20080529
24/25
Ubuntu 3D Desktop Effects
The Ubuntu 3D desktop effects allow for seamlessand natural transitions between windows and
spaces.
Benefits: Strong in FIV & NI
http://www.youtube.com/watch?v=JD6BiKnLzck&NR
Whitevoid
3D web navigation takes advantage of 3D spaceto organize content.
Benefits: Good elements of VSF and FIV
http://www.whitevoid.com/application
24
http://www.whitevoid.com/applicationhttp://www.whitevoid.com/applicationhttp://www.youtube.com/watch?v=JD6BiKnLzck&NRhttp://www.youtube.com/watch?v=JD6BiKnLzck&NRhttp://www.youtube.com/watch?v=JD6BiKnLzck&NRhttp://www.youtube.com/watch?v=JD6BiKnLzck&NR -
8/8/2019 3D Interfaces for Mobile Phones_public_20080529
25/25
Wideo
A 3d environment displaying and organising video.
Benefits: Good elements of VSF
http://www.youtube.com/watch?v=iHyqwi-gFso
WiiRemote Hacks
Using the infrared camera in the Wii remote and ahead mounted sensor bar you can accurately trackthe location of your head and render viewdependent images on the screen.
Benefits: Strong in VSF
http://www.youtube.com/watch?v=8kLFPfaxQ6U
http://www.youtube.com/watch?v=8kLFPfaxQ6Uhttp://www.youtube.com/watch?v=8kLFPfaxQ6Uhttp://www.youtube.com/watch?v=iHyqwi-gFsohttp://www.youtube.com/watch?v=iHyqwi-gFso