3.9 Appearance design

Although the section on appearance design is placed here, towards the end of the book, the appearance of something is of major importance, both for the user experience and for marketing reasons. In a highly competitive world impressions can be directly translated into money.

Definition: Appearance: the state, condition, manner, or style in which a person or

object appears; outward look or aspect or appearances, outward impressions, indications, or circumstances. (Dictionary.com)

There are appearances at many levels of design, in particular appearance of services, information, and of interactions. The levels are interdependent and for a service each device in the orchestrated ensemble contributes to the appearance. The whole of an appearance emerges from the interaction of the parts. Typically the designer tries to find a way that enforces a consistent appearance over the service. The devices of a service are instances of the interactors (H, I and T), but a service is not necessary for an appearance of interaction. All H-H interactions can for instance be assigned a social status in a particular context, which is a kind of appearance. For H we have physical disposition, sun glasses, mobile phone, clothes and a person’s car that are often related to a sought impression by appearance. We are in other words well versed in creating and evaluating appearances. We also know that appearance is not everything. Wine taste better from a crystal stemware than from a paper cup, but thirst is still the best motivator for drinking. A service is often an example of H-I/T interaction and its appearance emanates from experiencing the dynamic behaviour of interaction objects/structures and the information elements. We will exemplify appearance in services below by web design, graphic design, and 2D screen appearance, but also give some comments on interaction and physical product appearance. Visual appearance will be in focus but much of what is said can be applied to other media.

The interface is the message.

Appearance is a complex experience with both emotional and cognitive components. As with all human perception it is based on pattern matching past experiences with the current situation, a process impossible to fully control by someone else. The resulting experience is a type of “problem solving” working bottom up from details at the same time as it approximates a holistic guess top down. To support this process convention is important. It cues the perceiver for instance on convenience, simplicity, ease of use, priming and affordance. At the same time the designer needs to purposely break conventions to direct and attract attention and for emphasis (Ware, 2008). The designer needs to know about the rules in order to break them creatively and constructively.

Product appearance can play one or more of the following roles (Creusen, 2004):

� Attention drawing – invokes surprise or other emotional and intellectual experiences. Salient features can be added to the expressions to catch attention, e.g. differences in siXe. Matching clothes worn and the mobile device could attract some looks, integrating the device into the clothes is another idea (Juhlin, 2013).

� Categorisation – classifying, differentiation and sorting based on the patterns found. Simple patterns, as in an easily perceived arrayed structure, or more subtle such as in something repeating itself over time in long cycles.

� Ergonomics – consequences of use by showing overall appearance aspects such as size, roundness, indication of weight. A large man in a small car or a small man in a large car can make a difference on how we judge a situation.

� Symbolic – symbolic product associations, e.g. branding.

� Functional – displays functions or possible uses (affordances), functions, features and technical qualities.

� Aesthetic – provisioning of beauty, taste.

Our goal is not only experiences but also engagement. The world is full of choices and we want our offerings to be selected.

The subsections that follow will first add some details on interface design. This is followed by a short section on affordance. The next three sections will look at three frameworks for appearance, aesthetics, story and semiotics. After this fair amount of theory we will delve into branding, web page design and appearance in pervasive design will bring us home, almost, we end by discussing evaluation of appearance. While we focus here on our main sense of vision the other senses should never be ignored. It is for instance quite difficult to start your car if you are deaf.

3.9.1 Interface design

At some stage, we as interaction designers have to decide on how to present the interaction objects and information elements. It is a mistake to allocate this work by default to the end of the design process. For some products, e.g. the iPhone, it is of utmost importance and appearance serves as a guiding star throughout the project. As another example a beautiful, well designed, physical object, received as a part of a service moment transaction, can give an experience stretching far into the post-service

period. Customer loyalty is important and customer verdict on today’s service indicates the status of a company tomorrow. In this section we will not take on the full challenge of exterior design but mostly limit the discussion to graphic design.

Virtual objects are easier than physical to modify, but this still demands a fair amount of time and competence. For a pop-up form we need to choose fonts, sizes, and background colour, decide on its position, how it should appear and disappear, select accompanying sounds, and make many other decisions before we implement it. All relevant data should be available on screen and organised so that searching and scanning are simplified. If the pop-up form displays speed then 144 km/h is probably a better formulation then 144.353784 km/h. Or maybe a green circle is enough to show if the speed is within speed limits and a red one otherwise?

The graphics design knowhow and theoretical knowledge-base is huge and goes back for centuries. It includes facts such as that symmetry is an important clue to beauty and that symmetric objects attract attention. The basic visual elements are also well known: dot, line, space, size, luminance, colour, text and motion, see also 2.6xxxHG.

GOLD

Humans are not as sensitive to colours as they are to different shades of grey, but nonetheless colour is important and gives clues to behaviour and functionality. When colours are used, the same rules apply as for most other interface items; consistency promotes learning, reuse of familiar colouring simplifies interpretation, and colouring should enhance understanding and use in general.

The graphical elements can be modified in different respects. Techniques at hand are related to shadow, distance, separation, size, rotation, value change, font, blur, path of movement, direction and grouping. When the elements are arranged into a composition, the possibilities multiply. Groups of elements acquire properties of their own and when the different groups are arranged new, higher-level patterns, can be created. Foreground and background (figure, ground) can be used for effects along with different applications of contrast, balance, symmetry, harmony, repetition and rhythm. At this level of design text can be seen as graphical elements and different fonts can be combined to achieve interesting effects. Joining or separating groups creates new forms that can be extended into linear or radial clusters or grid based organization. Depth, perspective and occlusion add even more possibilities. Each arrangement will be evaluated in context and there is no guarantee that two viewers will perceive the same patterns or see the same changes.

Typically in graphic design we can assume a rectangular canvas and from this we have a set of dichotomies to play with: center/ margin, above/ below and left/right. Other rules of composition are the golden ratio and rule of third. To this we can add diagonal tension, strength and power of the vertical, stable horizontal elements, triangular relationships connoting spirituality and the stability from a rectangular shape.

Motion is another important attribute. Motion attracts attention across the entire visual field and the pattern of moving objects is easily identified. If the dots in the illustration to the left move they are easily identified as a person walking, given that users’ attention is directed to the figure. Attention should be directed to important areas without annoying the user and to grab attention a pattern needs to be broken, or a pattern found where there was none before. There are numerous ways to do this and moving something around against a

fixed background is one obvious solution.

Other ways to do it is blinking, colour, intensity, or bold text, interesting faces, eyes, or a sound. A logical balanced focus simplifies use, but too much attentatives distracts. In some applications our goal is not a smooth efficient user experience. We might crave for cognitive dissonance to change attitudes or to persuade. Advertisements use surprises to be

noticed, or use even more dubious tactics such as suggesting that diamonds are necessary to prove love. In general, something standing out from its surroundings is remembered. That a noticeably different thing is more likely to be recalled is named the von Restorff effect, a name that stands out.

“Difficult to write a PhD thesis on audio or tactile interfaces.” / Bill Buxton

We live in a culture that obviously favours visual literacy, and this section has provided quite a list related to visual appearance. However, we must not forget that the other senses can add to the image. Audio is important even though it is not as well researched for use in user interfaces. The importance of music indicates the potential for audio in appearance design for pervasive services.

As noted the aspects mentioned above are possible to vary in an almost infinite number of ways, but the lazy, or smart, designer enjoys the work of others by reusing platform styles, keeping guidelines in mind, explore already existing (successful) interfaces. There are pattern galleries that should be considered, consistency is perhaps the most important design guideline and consistency implies patterns (Neil, 2012). Style guides are available for WIMP from many system developers, e.g. Microsoft and Apple. Guidelines have been proposed also for some limited pervasive computing application areas, e.g. for informative art (Ferscha, 2007). Reuse simplifies building the system and many times helps the user to better understand the interface. You remember a tasty cake.

3.9.2 Affordance

Design is a multi-valued word when used in the process of product development. For some, design is construction, e.g. how to build a computer. For others, design is arranging exterior features for artistic or usability reasons. In this section design of a thing means the latter, i.e. how the exterior is arranged to present internal states, or other properties. For a relatively complex product it is not possible, and maybe not even desirable, to show every aspects to the user, and hence there will be misinterpretations and surprises when the external behaviour is studied.

Highlighting

When the design and the user establish a relation, the former gives away (affords) information about itself and its uses, its affordance. Consider the phones to the left where the leftmost one is connected to the telephone system, using a physical wire. The wire gives you a clue about how to hold the phone, and where to talk and listen. The second phone is connected using a wireless system and you have no idea whatsoever what part that holds the microphone. As another example, imagine a thermostat where you cannot tell if you have set it to the desired temperature. Affordance can be perceived also by other senses. Think about a train whistle or a hissing snake. A shining surface feels cold even before we touch it.

There are affordances for actions of different kinds. If we consider planning for an action, then we first have precondition affordances that must be fulfilled, and secondly there are properties or mechanisms that reduce the cost for the interactor to reach its goals, and these are called effect affordances (st Amant, 1999). For the example of a chair, a precondition can be that the chair is located out in the sunshine where the user wants to sit. A nice property of the chair is that it is at the right height for sitting, and comfortable enough to sit in for a couple of hours. The action oriented affordances described can be complemented by two affordances for evaluation. A precondition evaluation affordance helps the interactor to understand that preconditions are met. Maybe the chair has a sign with “CHAIR” written on it. An effect evaluation affordance instead helps the user to understand that the user’s goal has been, or will be reached, i.e. it provides feedback. Testing different positions for sitting in the chair will provide such feedback.

A product’s affordance should be designed with care, but the constraints involved must also be considered and could in fact be helpful in the design process. Constraints complement affordances by describing the limitations of an object. What it cannot do. We are often constrained by social, psychological and physical limitations and cannot easily adapt, so the things around us have to!

Sensory recognition is function based. In other words you start with an intention and try to map this to the

affordance of a thing. If your intention is to sit down, a chair is the perfect choice. With no chair nearby, but only a rusty bucket, you turn it upside down to make it sittable. If a chair is blocking your way, you are no longer interested in if it is sittable, only if it is moveable. It is not often that you look at an object and wonder what it is for, and maybe even less often that you imagine uses that were explicitly not intended for an object.

3.9.3 Visual and other types of appearance

Attractive things work better Don Norman

We have already touched on the subject of visual appearance in Section 2.6xxxHG and aesthetics in section 2.8xxxHG where we introduced the concept, its history and some basic elements, such as form and expression, mostly referring to Pragmatism philosophy and a visual point of view.

Button Button

Volume

Designing for appearance is possibly the most challenging task of the design process, and especially so for pervasive computing. Aesthetics, in all its defined flavours, is clearly involved and a lot can be learned from architecture, movies, and interactive art in general, from computer graphics, generalizing what has been learnt from the web, and from research in psychology and physiology (Locher, 2010).

In the following section we will briefly add some detail more related to design. The value of a service is affected by its perceived functionality and a beautiful one will be perceived as more useful.

In this book we have followed a structured methodology and at this design stage information architecture, interaction objects and functions are already decided on. We “only” have to design form and expression for the dynamic information elements involved. Our choices will eventually emerge as an interaction design form and result in a user experience. The crux of the matter is to do informed guesses on what expressions of the design that gives the sought experiences. A model to ponder on is shown in the following figure.

Figure 3.9.1 Dimensions of aesthetics (1) form and expression, (2) experience, (3) interaction aesthetics evaluation based on the H-H interactions presented or produced.

The figure adds a social dimension that we have not discussed here. Since service design often involves many stakeholders, the social experiences are obviously important and this is true also of temporal aspects of interaction. There can be no interaction without change of time and no service design (Landin, 2009, p. 6). Furthermore, the aesthetics of the figure will be experienced in use and in a particular situation that the designer cannot fully control.

Without a theory for the design of the particular application on the one hand and too many theories for the general case on the other, a sensible approach is to use a guess and evaluate loop. Users and others are important in the evaluation phase in order to quickly improve on ideas. Another approach is to aim for realism in graphics, which has its own set of problems, e.g. that humans are sensitive to how reality is presented and realism is computer intensive

One way to approach the problem is to start from the philosophy of Pragmatism and its conception of aesthetics. Ross et al. (2010) do this and suggest four principles where the first is that a product or service should be useful, at least provide a value. The other three principles stress the importance of

� Social and ethical dimension of resulting behaviour.

� Satisfying dynamic form, timing, rhythm, and tempo of the interactions designed are important, e.g. for creating tension and fulfilment.

HT/I 1 2

H

3

� Involving the whole human being, i.e. cognitive, perceptual-motor, emotional and social skills.

3.9.3.1 Dynamic form

The form is the message.

Dynamic form is a key issue in understanding aesthetics of interactive systems. Ross et al. (2010) describe a strategy for designing dynamic form, starting from the Laban motion analysis (LMA) that dancers use to describe motion. According to them traditional sketching is not optimal for illustrating dynamic motion. The authors also want to qualitatively describe the social implications of interaction dynamics. The subset of LMA they use is:

� Effort – the amount of energy, e.g. fighting or urgent indulging

� Body – what parts of the body moves and how physical contact is made

� Kinespheric reach – size of the movement, inscribed in a sphere (kinesphere)

� Shape qualities – shape of the movement, described for instance as if the body opens up (growing larger) or closes itself off

� Initiative – who acts and who reacts

� Interaction dynamic development – energy fluctuation, e.g. rhythmic interaction.

What the authors describe is not a finished design process, but the point here is that it is possible to describe quality of motion, including gestures, systematically irrespective of if we want to describe user(s) behaviour, the system we design, or the composition of user(s) and our design. What is not clear how to describe is the character of the moving body. It makes a difference pushing a baby stroller or a Volvo and this situatedness is difficult to describe and to automatically characterize (Mentis, 2013)

Interaction design form can be related to beauty for all the different experiences we have discussed, e.g. to the behaviour of emotional and social experiences. In particular we hypothesize that aesthetics can be applied to all meaningful experiences, but now we are getting into pretty deep water. Questions to further ponder on are; “Is beauty a relevant semantic concept for describing all meaningful experiences?”, “Can freedom be characterized as beautiful?”.

This triggers new questions such as; How do meaning and meaningful experiences relate to values for dynamic systems? How are values prioritized in such systems? Should we for instance aim for being strong and free or for safety in the tight social group? Can style overcome substance in a dynamic system? How does expression depend on form, and form on material? Does the material affect form and when does a change of form improve expression? These are difficult questions especially since it seems that the relations between the layers are non-linear in nature, that is, a small change on a lower level can give new life to the higher levels, and establish or strengthen the wholeness of the design.

3.9.3.2 Visual aesthetics for design

Aesthetics is used as a framework to describe beauty in a painting or a statue, i.e. for the static visual appearance of artful objects. An extensive review of research on visual aesthetics by Moshagen et al. (2010) listed some facets of perceived visual aesthetics in web sites, see also 2.8xxxHG. They suggested the following:

� Simplicity

� Diversity – visual richness, dynamics, novelty and creativity

� Colourfulness – selection, placement and combination of colours

� Craftsmanship – skilful and coherent integration.

Note that these facets are incompatible to some extent. It is for instance impossible to both aim for simplicity and open up for full diversity.

On their way to the facets they found much evidence from research on the benefits of aesthetics for increase of ease and effectiveness of learning, increased motivation, positive affects, urge to buy, customer loyalty, intention to revisit, appreciation of content and positive judgement of usability. They also listed a number of terms related to quality that are useful when describing visual appearance:

Balance, Equilibrium, Symmetry, Sequence, Cohesion, Unity, Proportion, Simplicity, Density, Regularity, Economy, Homogeneity, Rhythm, Order and Complexity, Clean, Clear, Pleasant, Symmetrical, Aesthetic, Original, Sophisticated, Fascinating, Creative, Uses Special Effects, Admirable, Dull, Noisy, Unique character, Complex, Intriguing, Pleasing, Colourful, Vulgar, Exciting, Old-fashioned, Fun, Lacks imagination, Standard, Enjoyable, Realistic appearance, Harmonic, Modern, Beautiful, Monotonous, Artistic, Skilfully designed, Symmetrical Applies good taste, Energetic, Challenging, Convenient Wretched, Simple, Overloaded, Professional. (Mbipom, 2009)

3.9.3.3 Sound aesthetics for design

We can also use sound.

Often peak enjoyment correlates with peak emotional arousal, so how does sound affect emotion? Sonnenschein (2001) referred to a list of acoustic expression corresponding to emotional states, originally researched by Friedrich Marpurg (1718-1795):

� Compassion – soft, smooth, lamenting melody; slow; repeating bass figure.

� Contentment – a more steady and tranquil melody than with happiness.

� Fear – tumbling downward progressions, mainly in the low register.

� Happiness – fast movement; animated and triumphant melody; more constant harmony.

� Hate – rough harmony and melody.

� Hopefulness – a proud and exultant melody.

� Laughter – drawn out, languid tones.

� Love – consonant harmony; soft flattering melody in broad movements.

� Repentance – like sorrow, but with a turbulent, lamenting melody.

� Sorrow – slow, languid melody; prevailing dissonant harmony.

Anticipation is important for dramatic storytelling, we strain ourselves to try to figure out what will happen next. This state can be supported by sound. Another factor supporting storytelling, is setting the mood. For music there is a language for describing tempo and mood of the music. Some examples describing musical tempo in beats per minute (BPM) from Wikipedia are:

� Dagio – slow and stately, literally, "at ease" (55–65 BPM)

� Andante – at a walking pace (73–77 BPM)

� Moderato – moderately (86–97 BPM)

� Allegro – fast, quickly and bright (109–132 BPM).

And some terms for musical moods:

� Con brio – with vigour and spirit

� Con fuoco – with fire

� Deciso – decidedly, decisively

� Misterioso – mysterious

� Poco a poco – little by little

� Sostenuto – sustained, prolonged

� Subito – suddenly.

Sound tells us a lot of our environment and this makes it important also in movies. It shapes what we see in a scene. Imagine a large train coming out, in smoke, from the tunnel, sounding like miau, miau. Music in film adds to the experience, but is only supportive, thus should not be so good that it is noticed (and not that bad either). According to Sonnenschein (2001, p. 155) sound can be used in the following ways:

� Emotional signifier

� Continuity in sound

� Narrative queuing

� Narrative unity – by repetition, variation, and counterpoint

� Programmatic – represents action or event in the world, e.g. music is played and the pace of a runner perfectly follows the tempo of the music

� Anempathetic – the music does not follow the emotional content.

Furthermore the properties of sound loudness, pitch, timbre are useful to define the “sonic texture” of a film (Bordwell, 1985). Basic dimensions of film sound are:

� Rhythm– from rhythmic to irregular, can emphasize the rhythm of the movie, multiple rhythms overlays, synchronized or purposely not.

� Fidelity – support immersion, the feeling of reality in the movie is supported, it feels real.

� Space – space can be suggested by sound, e.g. by an echo or a characteristic reverberation.

� Time – sound mapped to moving lips or sound made to illustrate story time. One example of this is that a shot heard could be repeated in another scene and we will adjust time frames so that the shots match. Sounds could also provide flashbacks or flash forward. Story time can also be accentuated by slowing down the music and perhaps lowering a high intensity or pitch.

A sound is diegetic if the source comes from a character in the film, an object such as an instrument or a door. In physical reality sound occurs when materials interact and it informs us about the materials (given that someone does not trick us). A solid metal door will definitely sound different when slammed shut than a mosquito door does. Non-diegetic sound is not heard by the characters of a movie and has no direct relation an event in the story. It could be used as a voice over to introduce a mood or tell a parallel story that is not relevant to the main story and its space. How then can diegetic sounds be used? One possibility is to present events that happen out of sight, suspension can be created with footsteps coming, thoughts can be made audible, heartbeats can be amplified. A sound that cannot be clearly identified can be most frightening. The camera mostly is used as an eye and only reaches the surface of things.

In pervasive systems we need to consider the devices used. A short flashback illustrated development and some possibilities. In the beginning Radio programs were listened to using headphones out as headphones, advanced to an external point source and from that to stereo. When TV entered the living room the radio was simply turned off. The service assumed that there was one source only and could be designed from that assumption. Design of pervasive computing will face the design of 3D soundscapes and on top of that has the problem of how to handle multiple sound sources, without annoying the users too much. Think about when you listen to the stereo and there is a sound clip suddenly invoked on a web page. What combination of sources, information, and intensity will give the best emergent result for the user(s) over time?

3.9.3.4 New materiality

WYSIWYG

Phenomena exist in the material world Material makes thoughts tangible Materials manifest the world /Viray

Mobile technology, smart materials, new and accessible sensor technology Internet of things and other developments makes it possible to visualize what has previously been invisible and only assumed. Computing, interaction and productivity and enery could be seen as new types of “materials”. At the same time IT gets embedded in previously non-computational materials, for example cars, glasses and doors. Acknowledging the trend where materials are given new important interactor roles has been dubbed the material turn in HCI (Wiberg, 2013). Tangible interaction is one example of the trend, where physical material is merged with other computational representations.

In essence the virtual and the physical are increasingly entangled and interdependent. As people adapt to this they become parts of the system.

A new way of studying an exemplar of such a composite is to focus os its materiality rather than its purpose. This allows us to ask new questions such as what its character is, rather than the traditional HCI question about how we can use it (Wiberg, 2013b).

Plenty of open questions here, for instance, what is the terminology to desctibe this materiality and what are the proper methods for exploring the material? What new materials are there? What are the properties of combinations of interactive materials? How do we find them out in a structured way? If we do not then we might end up fighting to adapt the application to the material. Some new terminology is suppleness, collaborative rythms, material interaction, interaction gestalt, digital form, inspirational bits, hybrid crafting, design by trace and worn materials, material probes, texture, media as material, transmaterials, computational composites, composite materials, interactive play objects, intangible, immaterial, becoming and computational materals, just a truncated list of all approaches currently attempted.

What a mess �

Still, there is yet at least another leap ahead of us as we materialize emotions, behavioural, social and cultural aspects of our life world through virtual reality. Interesting times ahead as the Chinese say.

3.9.4 Story and visual appearance

How does it all connect?

Stories about a product at this design stage can be considered from two points of views. First we have the appearance of the story itself, i.e. is it an engaging story told in a good way? Secondly, we can think about what the story says about of the behaviour and appearance of the design. The story about Apple and the iPhone says a lot about the design and use of the iPhone. This section will briefly discuss both of these aspects starting from literature research and aesthetics.

Why are certain stories more interesting than others? What and when in a story makes it interesting, fascinating, compulsory reading and engaging in a way that could even change your life?

A major feature of most such stories is that they connect in parallel to your own life as reader or author. You experience difficulties and opposition also in your own life, i.e. the equivalence to a plot. You need change to feel alive. You have needs. You have dreams. A story is a promise to experience the above as a relief/escape/hope from the past, as a substitution of the current, and as a challenge/possibility/hope for the future (Johnson, 2000).

Researchers in literature have tried in different ways to figure out the mystery of a good story. Some of the more prominent explorers and an indication of their approaches are:

� Barthes – The plot is used to create suspense through unanswered questions and anticipation of the result of an action (Barthes, 1977).

� Iser – The reader fills in the gaps with his or her personal reflections, especially about what is left out of the story creates the basis for a good experience (Iser,

1972). The challenge to find a unifying gestalt for the text makes it interesting, perhaps forcing the reader to accept something new, explore the unknown.

� Greimas – Story is built from oppositions that create tension (Chandler, 2002 on Greimas).

� Brooks – Applies terminology and knowledge from Freud and psychoanalysis. One example is a discussion on how the death drive pushes the story toward the end, while the story develops by exploiting eros (sexual drive, and other creative, life-producing drives built by their energy libido), stimulation into tension and the desire of narrative. The plot in the middle is shaped as a detour, “a struggle toward the end” (Brooks, 1992, p. 107).

A plot must be represented to be perceived and each medium has its pros and cons for portraying meaning, unfolding actions and events, and processes of change. There are also numerous techniques for story telling according to each type of medium and we will certainly not have time to cover them all. Just to take video as an example we would have to discuss (to begin with) image composition, perspective, framing, camera angles, camera movements, lighting, scenography, costumes, sound effects, music, and cutting.

Media using the visual sense such as photographs, paintings, and video are efficient information bearers. Oddly, how and why an image affects us seems to be less known to the typical reader/listener/audience compared to for instance speech and written text. One reason for this might be that written text has a well-established and well-known framework for describing its syntax and semantics.

What we will do now is to describe some aspects and terms of such a narrative representation scheme at a general level and introduce some terms that can be used to analyse a story. This we will do by following and extending the references (Kress, 1996) and Arnheim (2004). We will mostly use images and visual aesthetics for our examples and discussion, not literature.

To start with, being and acting are fundamental concepts of reality, roughly equivalent to noun and verb in written text. Kress (1996) instead of nouns and verbs uses other concepts such as participants and processes, which can be used to analyse narrative structures. Participants represents objects and elements, i.e. visual representations of characters such as actors, but also of goals, receivers, senders, attributes and props could be referred to as volumes or masses. Participants are selected from what is perceptually salient in the image, e.g. a ’heavy’ volume might dominate the image space and be seen as a participant. What is salient in a story is determined on a higher level of cognition. A tree could for instance play an important role in a story without saying anything or moving around.

A process is what is ‘going on’ in an image or a language construct (doing, sensing, being, happening, …). Processes are ‘vectors’, tensions, or dynamic forces found in the image corresponding to ‘action verbs’ in written text, e.g. transport. Once again a story has many additional ways when it comes to representation as compared to an image. If we for instance are told that “a door is slammed shut” we immediately construct a story for what is happening, given any contextual evidence we have from the rest of the story. It is possible to represent a slamming door also in an image, and to add circumstances for the slamming such as a person running down the steps. The text based story (or film) however has many more degrees of freedom of designing this context.

A narrative process involves actions and events. A vector represents an action and can be exemplified by the diagonal line formed by an outstretched arm, or the pointing of a gun. Other examples of vectors are a glance and an eyeline where visible eyes, distinct pupils and facial expressions are prerequisites. The target of a glance or what the eyeline is directed at is called the phenomenon.

What is done by an actor towards a goal is called a transaction, i.e. two participants (actor, goal) are needed. One example is connecting a person in the image to a thought balloon. This transactional structure is one type of structure. Another type is the classification structure where the actor-goal transaction is exchanged by a carrier-attribute relation where the carrier (participant) is built or designed by attributes (participants). If we instead only have an actor and no goal the structure is called non-transactional. If a visual structure only has a goal, but no actor, it is called an event. One example of an event is an image with an arrow in the bull’s eye of a target but no one in sight responsible for throwing the arrow. Something happens to a participant and who or what that makes it happen is not known. A similar visual structure is formed if we see only the face of a person, but not what this person is intensely staring at. Illustrating action is important in scenarios and storyboards, both as a means to illustrate action and to persuade the reader some way or other.

Think about the recruiting ad where Uncle Sam is pointing his finger at you. This kind of image act can be compared to the speech acts advanced by the philosopher Searle (1970). He suggested that normal utterances in speech, i.e. saying something, forms a hierarchy of speech acts on different levels where the message affects the addressee, see also Section 2.6xxxHG. The speaker wants the listener to recognise the meaning of the utterance, i.e. to do or think something. One example is the statement “You’ll do that”. What Searle calls an illocutionary act can be expressed as a performative, i.e. a verb, operating on some content, i.e. “do” and “that” in the previous example. Searle proposes the following taxonomy for illocutionary acts (Searle, 1970):

� Assertive acts, used to commit the speaker to the truth of the expression “There’s too little salt in the soup”.

� Directive acts, trying to persuade the listener to perform something, “Pass the salt, please”.

� Promissive acts, are attempts to commit the speaker to do something, “I will pass the salt, some day”.

� Expressive acts, expresses the speaker’s feelings about a state of affairs, “I am sorry, the salt has been stolen”.

� Declarative acts, perform an act by the utterance, “I curse you” (the effect is uncertain, but maybe some salt might help?).

One interesting feature of all speech acts is that they are independent of the cultural setting and the linguistic form chosen for it. Utterances such as “Pass the salt, please” and “Would you mind passing the salt” are equivalent from the speech act point of view. The theory of speech acts is quite general and currently difficult to use directly in applications, but it is useful in the analysis of conversations, and can be used to analyse other interactions, for instance a graphical user interface or the interpretation of an image.

Image acts can achieve the corresponding results if a contact or relation is established between the represented participants and the viewer. A way to do this is to have the person in

an image look straight at the viewer, e.g. as in Mona Lisa by Da Vinci. You as a viewer will look for a message in her eyes, face expression and posture. Another way to involve the viewer is as an invisible onlooker. This could for instance be done by showing the content of the image framed by a key hole, indicating that the user looks through the key hole. Television news reading attempts to accomplish a relation with the viewer, while on the other hand a viewer is not supposed to feel part of a TV drama. A reporter gazing at the viewer demands something from the viewer, while looking away gives the message away for free. If the viewer felt present in person, the meaning of a scene can change. Stories can be seen as generalised speech acts. Since their expressivity is much higher than that of an utterance, there are more degrees of freedom for achieving an intended effect. The utterance has another advantage since much more of the context is shared, at least in face-to-face conversation. This is usually not the case for an image or a written story.

At this stage one might ask what it is that shapes an utterance. One answer to this, which is suggested by the Russian author Bakhtin is that both the speaker and the audience are involved with their respective histories. This is perhaps obvious, but Bakhtin adds another parameter, the speech genre, which to start with includes the style of speaking. Speech genres are “modes of speaking or writing that people learn to mimic, weave together, and manipulate (such as "formal letter" and "grocery list", or "university lecture" and "personal anecdote") (Wikipedia, Genre). We could for instance compare a command from a military officer to a whispering conversation between lovers. If we mix genres we can create new effects.

Settings in the model by Kress are called circumstances and are secondary participants not related to the main participants with means of vectors. One example is the locative circumstance, which forms the background of the image, for example grass and a tree. Tools used in the transactions of the image or a hand forming a gesture are other circumstances, circumstances of means. A third kind is the circumstance of accompaniment, which can be exemplified by a teddy bear carried by a child. This would be an image of a child with a teddy bear. In general the circumstance is the answer to questions such as when, where, what and who. There are many tips and tricks for managing the circumstances. One is that readers/viewers assume that the setting, e.g. the location is the same if nothing else is said. This also means that whenever a scene or a viewing angle is changed it is important to think about continuity, not confusing the viewers/readers without a reason. For films we need to consider that humans usually do not pan or zoom, but follow objects. Extensive camera movements not tied to objects or reason will hence be quite disturbing.

Symbolic processes are about what a participant means or is (Kress, 1996, p. 105). This is illustrated by an attribute with a symbolic value. The symbolic meaning of such attributes can be well known, indicated by having a salient representation or pointed at by objects in the image. The symbolism can also be suggested by that the attribute might not seem to belong in the image or that the image somehow points it out (Kress, 1996). One example of this could be an apple placed where it is seldom found. Symbolism can also be suggested by extreme lighting, soft focus or an unmotivated golden glow.

The viewing distance chosen when portraying a human object suggests a level of intimacy to the viewer. Face only on a close shot implies intimate face-to-face interaction, while a medium half-body shot suggests socialisation. Taking in the whole figure is more impersonal and suggests a public meeting. Camera angles can also be used to indicate interpersonal relationships. One example is that a view from a high angle empowers the viewer and signal dominance, while a low angle indicates the opposite.

We rank credibility higher for visual information than for what we hear. What we choose to believe as credible is something that is established socially. Naturalistic images are considered as details from reality and are given high credibility. If we reduce the level of detail we lose credibility. Another parameter that affects the truth value of an image is colour saturation where reduced colour will give less credibility. Decontextualisation where the background is faded out, replaced or with no depth, absence of light variations and shading, and reduced variations in the levels of brightness also reduce credibility. Also other effects in an image can increase its apparent reality and credibility, even to the extent of hyperreality. An image could for instance make heavy use of blue colours visually making the scene appear colder, or use bright red areas to energise an image. An educated audience will appreciate added symbolism and abstractions that in turn might give a deeper sense of credibility to the image. A variation of this is to use established company brands, such as Bank of England, to increase the credibility of an image.

There are meanings conventionally assigned to visual shapes (Kress, 1996, p. 53). The square implies honesty, straightness, and workmanlike results. The triangle associated with action, conflict, and tension. Circles instead are interpreted as endlessness, warmth, protection, and eternity. The square and angular are associated with technology, mechanical, and the inorganic as opposed to curved shapes. Visual shapes also can be amplified in different ways, for instance by changing line width or changing the size of the arrowhead. A curved arrow could give a more organic impact.

Not just the elements but also the whole composition of an image or a story highly affects what it tells an audience. How elements relate to each other and are integrated in a meaningful whole can form a plausible reading path, which in turn can be used to strengthen the narrative effect of the image or the story. In the following we will explore some of the effects that can be used, beginning with concepts from art theory, as indicated in the following table.

Story/Theme

Setting Character Plot Props

Tension/contrast/opposition

Balance/harmony/symmetry

Space/ground Spatialextension Spatialrelationships/proportion

Repetition Dynamics Movement Rhythm/tempo Pointofview/mood

Motive/Intention

Table 3.9.1 Matrix correlating aesthetic values with story components.

Some of the aesthetic values in the first column of the table will be discussed below and applied to story design. Only brief, sketchy illustrations and examples will be given. Filling in the cells of the array is an exercise for the reader.

A major component in a good story is repetition. Repetition of the elements in your day-to-day life can be contrasted to the transformations throwing you off course, beyond your life. Repetition can bind energy through increased tension, but also allows for variations of rhythm, and symmetry; “rhyme, alliteration, assonance, meter, refrain, … allow the ear, the eye, the mind to make connections conscious or unconscious between different textual moments, to see part and present as related and as establishing a future that will be noticeable as some variation of the pattern” (Brooks, 1992, p. 99). Choice is another important component; “choice stands in the place of necessity, of destiny. In this way man overcomes death, which he has recognized intellectually.” (Brooks, 1992, p. 98). Repetition together with choice means a second chance, the possibility of revenge or striving towards perfection. Choices also imply oppositions, challenges to the main characters, once again something for the reader to relate to. The reader is challenged both to fill in the blanks, by presenting problems to solve, and by the question of resolution (how will this end?).

Contrast in an image is the use of opposing elements, such as colours, forms, or lines, placed in proximity to produce an intensified effect. It is the degree of difference in tone, brightness, or color from point to point or from highlight to shadow. In a story contrast implies opposition and is important to create tension, i.e., a balanced relation between strongly opposing elements. In a story the "pressure for response" can take the form of a challenge, a surprise, a time restraint or the suspense of not knowing. If there are two characters in a scene they should be different in some way, to make the scene interesting. One could be aggressive while the other is passive, or they could differ in size. By varying the emotional expression of a scene over time and also between scenes a dynamic tension can be created. Each scene should ultimately be a story in miniature, with its own goal (theme) and development, e.g. it could start with a pleasant conversation and end in an argument.

As a special case of contrast we have balance weighting two or more things against each other over a given space. For cultures where reading left to right dominates, the left part of an image can be given more weight, still maintaining the balance of the whole image. Consequently a focus upon the object to the right will be more interesting, and if an object is placed to the left of an image it can be placed further away while still preserving balance. Balance in stories is even more subtle to manage. It for instance depends on how the author places the peaks of action of the story, how the ending is planned, but also on how the beginning involves the reader from the start. Proportion also relates things over a given space, but here the relationships are more complex, not just equalizing the sums of a number of weights. Beautiful proportions mean that relationships follow conventions not always easy to specify, even though there are exceptions such as the golden ratio. Proportions in stories also relate to conventions, in analogy with the image. Here, however, the space to divide is not a simple two-dimensional image space, but the dimensions are given by the relations between lengths of scenes, from character descriptions, and from many other dimensions of the story.

Images are inherently spatial and composing space is an important aspect of their implementation. Correspondingly stories can be seen as occupying room in a number of spaces such as time, plot, character space etc. The spatial arrangement can affect meaning of the elements in the space and we can agree on meanings for combinations of participants and their behaviours in a story. One example is identifying that someone is ill from how they walk and talk. The centre of an image is an important part of the image; often this is the focus where information density by convention is supposed to be the highest. A person in the middle of an image will be seen as more important than on in the periphery. To avoid overloading the viewer, focus should be limited both in the number of interesting areas in an image, and by concentrating the size of the message to a smaller area of the image. In a story intense action can be focused to create a similar contrast.

A vertical line can be used to divide an image up into two areas, for instance separating ‘before’ and ‘after’, ‘from’ and ‘to’, or ‘given’ to ‘new’. For a story the effect of breaking up story space, e.g. the plot can be achieved in many ways. One is to have a secondary plot that interleaves with the first. Another is to advance story time by a sequence of scenes. On the other hand, the horizontal line in an image serves as horizon, or as an attractive promise of the future that the product that could realise this future. Depending on their positions participants could be seen as grouped together or placed separately. A disconnected object can be visually separated using a line or empty spaces or a discontinuity of colour. This framing of elements in the image affects its information content. Strong framing for instance supports individuality, differentiation and opposition.

Framing disconnects or connects participants in different places of the image, and could have dramaturgic effects. Spatial composition affects the rhythm that can be found in media where cyclic change over time matters, for instance in a video or a linear story. Rhythm has a tempo, which is true also for repetition, an attribute that lends it character. Rhythm and tempo imply movement, which is the most important appeal for visual attention. With the ability to describe motion comes another possibility to illustrate dynamic tension, i.e. by exploiting tendencies of motion. Rudolf Arnheim (2004) describes our relation to dynamics this way: “Brain and mind envisage change and crave it; they strive for growth, invite challenge and adventure. Man prefers life to death, activity to inactivity”.

One important aspect of a representation of a scene is the mood it creates, framing the dramatic events. This mood is built by more or less subtle cues, and we have all learnt a veritable library of them. Typical cues are colour or other visual tweaking (black and white versus colour, dark clouds versus sunshine), sound (footsteps in the hallway, screams), voice of the narrator, choice of words (dark and gloomy, harsh, radiating), background music (spooky). Another way in which the author can change the mode is by manipulating length of scenes, i.e. the tempo. Such manipulations can be compared to how rhythm is used in music. Short scenes create a high tempo, but in the long-run consistent short scenes bore the reader. A long scene could build tension, which the author could create release by a series of short scenes. Scene length of course also depends on other aspects of the story, e.g. the amount of information to convey. The last row in Table 3.9.1, “intention/motive” brings us right back to the reasons for acting, the needs, and the quest for finding meaning. Any rules stated above could, and even should, be broken at times to surprise the reader. Any detour from convention will shake the user.

3.9.4.1 Sound and narrative

Sound is also important for communication and information presentation. Sound and music can support the narrative and even provide a narrative by themselves. The author is renamed composer and there are also other music-story relationships, for example (Sonnenschein, 2001, p. 175):

� Music – story

� Melody – character

� Dissonance – conflict

� Rhythm – pace

The correspondence between music and story can be further refined by the use of particular musical genres and their impacts (Sonnenschein, 2001):

� Sacred, hymns, gospel, shamanic drumming – grounding, deep peace, spiritual awareness, transcend and release pain

� Gregorian chant – regular breathing, openness, lowered stress, contemplation

� Big band, pop, country/western – centred feeling of goodness, contained movement

� Rock – aggressive movement, building or releasing tension

� Heavy metal, punk, rap, hip – animating the nervous system, rebellious behaviour

3.9.5 Using semiotics for analysis

What possible use could a designer have of semiotics, the theory of signs, icons, indices and symbols and its semantics? See 2.6.4xxxHG for a short background to semiotics.

One idea is that we can relate the three basic signs into modes of sign functions, i.e. modes of the functionality provided by products working as mediators of messages. The following is a slightly edited list from (Routio,

2003), which is not exhaustive.

Functionality provided by indices (real and dynamic connections to object): Light, sound, smell or haptic signal from a product, e.g. lifting a container to find out if it is empty. Smells and sounds could be disagreeable or pleasant. Noise from a product is a special case.

� Material – touch might indicate quality, e.g. kick the ball to determine if it is properly inflated.

� Graphic figures – if they are an integral part of the product, such as numbers on a yardstick.

� Pointing form – the archetype is the pointing hand

� Traces of tools – which are used in design or manufacturing, e.g. a typical Windows ® pop-up menu.

� Marks of use – exemplified by flaws or dirt.

Functionality provided by icons (resembles object):

� Form, shape, sound, smell, haptic – conformity, tradition and divergence are used.

� Colour – red can refer to blood,

� Material – gilding indicates wealth and concrete emotional coldness

� Metaphor – resemblance of form to a not designed object, e.g. the Volkswagen is compared to a “Beetle”

� Style – follow a style such as impressionism or rococo.

� Context/Environment – tools designed for a kitchen share well known characteristics. We think of a kitchen and this in turn gives meaning to the sign. Try to think of an example of other contexts that impose meaning to a sign.

Functionality provided by symbols (sign by convention):

� Graphic, sound, smell or haptic – e.g. logotype, on-off-buttons, washing instruction, … --- …

� Colour – red carpet, white could refer to the quality of cleanliness.

� Material – dress codes

”Chair”

� Form – military uniform, swastika, �.

� Position and posture – closeness, above or below, flag on the American flag mailbox.

From an evaluation point of view semiotics gives us a perspective to study our designs, a structure that helps us leave our frame of mind as creators and become evaluators of a depersonalized space of signs.

3.9.6 Branding

At the appearance design level branding establishes an identity and we use it here as an example of an important visual appearance. A company has to have a logotype, physical products a packaging identifying the company, and service design can be described as “total brand experience” (Saffer, 2007). This is however the company’s point of view and in this book we are more interested in the user’s perception of the company. From this perspective branding, or styling as it is sometimes referred to, starts by associating the company and its symbols with values such as reliability, luxury and caring. From the company profile branding makes a difference all the way to product interaction where for instance smartphones of a certain brand behave in the same way. This specific look and feel affects customer expectations and trust in the product. It could trigger a good mood and a positive user experience. From a service point of view, most services include many touchpoints and for each of them there are numerous possibilities to enhance the user experience while at the same time promoting the brand.

Branding is a lot more than a graphical style and a logotype. A story told of the product and the company is one example. Think about the “It’s the real thing” slogan by Coca cola or Nokia’s “Connecting people”. Some factors affecting the styling are product predecessors, competing products and brand identity. When we choose between two products the symbols associated with the brand and what they suggest are important. How do they relate to how we perceive ourselves? Do symbols associated with environmental sustainability attract us more than those related to decadent and exotic mystery? New symbols need to be communicated and imprinted, which takes time and costs money. Instead of creating a new symbol we can adjust product appearance such that is adheres with values that are already established. The problem is to present the values while at the same time signalling that this particular product or service is unique and superior to its competitors.

It is easy to make mistakes and establish the wrong mental image. Authenticity for a web shop is for instance important and falters already for such a simple reason as a long response time. Adding a “work in progress” icon efficiently and immediately alienates customers. Valuable customisable experiences should be signalled, which help users develop, get a glimpse of a higher goal, or give a foretaste of a better way, see also Section 3.1.

Above we have briefly discussed branding of companies and products, but also the designer can benefit from branding. A personal way of expressing a design and a nice looking web page could open new doors.

3.9.7 Webpage design and guidelines

We assume that a structured design process has been followed and that we know what information we want to display. We also know what is most important to communicate. From the perspective of this book webpage design is used here as the prototype design for any

display in any pervasive design ensemble. Sound will not be discussed, but should not be forgotten.

Right from the start of a webpage appearance design project we can use a mood board to learn more about the emotional landscape at hand. The illustration to the left might be a good start for the ultimate mood board for environmental action. A mood board is a collage of images, texts, and anything else that can convey the right feeling for the product. You can generate it yourself, or you can get users/clients to help you.

Next thing to do is to establish a visual hierarchy, i.e. the order in which visual objects will be visited by a viewer. The most important information elements should be the most apparent and then the design should lead users through the rest of the page taking reading norms into account. A visual flow is established where the eye tends to enter the page at the highest point of contrast, which gives us a chance to define the page entry point. From the starting point the eye follows lines, gesture, movement, (actual or implied), and aligned objects. Temporary stops are made on masses, blobs, significant shapes. The visual hierarchy could be structured according to; kind, from general to detail, according to complexity, chronology, or relevance.

Different approaches can be explored and evaluated using simple sketches. There are many visual elements to play with, e.g. consistency versus dynamics, while making sure that the graphics serves communication efficiency. Brainstorming and the power of a group working together could support the process. One example is group sketching, e.g. experience sketching where a group tries to catch user experience by sketching together (Moritz, 2005).

Less is more, Negative space is powerful, Symmetry is the ultimate evil. (Samara, 2010)

Appearance is increasingly dynamic, both dynamic as in interaction, and in content as in videos. Moodboards with audio and video, i.e. “Moodfilms” can be useful to quickly find the right expressions (Moritz, 2005). Animation is an important area with a terminology of its own and well worth a study. Some important key principles are: timing, slow-in and slow-out, squash and stretch, line of action, anticipation, exaggeration, follow through and overlapping action. These are terms that also signal the human and physical qualities of animation.

The following two subsections will list guidelines for webpage design. The information is terse and the sections are included more as a reminder of some important principles.

3.9.7.1 Webpage design guidelines

Rules can be broken but never ignored.

This section complements Section 3.7xxxHG on information design and presents a compact list of guidelines for detailed website and webpage design (Samara, 2007). To begin with a webpage should support the user objectives. Other things to consider are:

� A graphic composition with a strong visual hierarchy that organizes and presents the former aspects by forming a visual flow. Start from the informations architecture and interaction structure identified in previous design steps.

� Group navigational cues (buttons, text and icon links, navigation bars …).

� Logical positioning (most used cue on top), consistency and repetition.

� Include whitespace!

� The colors selected should add to the concept.

� Easy to understand labels and cues, visible, descriptive rather than artistic.

� The form of graphic elements should communicate with images.

� Images should be consistent in their message.

� Typography detail should relate to images and graphics, and convey appropriate messages.

� Legibility is important.

� Keep all writing brief, half of what you would normally write on paper.

� When arranging typographic elements, consider balance, emphasis, rhythm, unity, as well as positive and negative space.

� Follow conventions.

� Break them knowing why.

The more info you try to pack in, the more crucial your composition becomes. Evaluate every graphical element with respect to the whole. Is this graphic serving a purpose or is it there just because I can do it? Can this graphic be removed, made smaller/simpler and still do the same job? Can I get this graphic to do more than one thing? Reuse it, make it carry more information? Work in both the hierarchy and as a compositional element?

3.9.7.2 Webpage design process

Here we will hint at a design process for a web page, i.e. the design process of a snapshot of the screen of an individual device in the ensemble, presented in a compact list form. We once again assume that the required elements to include in the page are known:

1. Decide the priorities for the elements in the visual hierarchy.

2. Build a hierarchy tree of the elements assigning them descending values of contrast (change in texture, colour, size).

3. Design the visual flow from big masses to small, high level of contrast to low level.

4. What groupings can you create? Build hierarchy within groups and ensure strong grouping using color, size, shape, font, texture. Use no more than 5 ungrouped elements at the same intensity in order to avoid visual clutter.

5. Combine, add, adjust and remove elements. Simplify!

6. Analyze the design. Where’s the design working, not working? Where is your eye going? Why?

7. Iterate, ask others, often.

The last item of the list is the most important. As a rule the first run through the list is just a warming up exercise.

3.9.8 Mobile devices

The context of a mobile device changes, which makes it difficult to design an appearance to match its environment. The one aspect that can be trusted to be there is the user who faithfully follows the device around. If the device is used outside it must be rugged enough to stand the environment, a fact that might affect the materials used and the appearance of the information displayed. To be of any use a device must be small enough to carry, on wheels, or be

motorised. If the device is carried there will be a natural trend to make it as small as possible, as long as usability is not threatened or you lose your device too often. Another constraint whenever a human user carries a device is that it should be smooth rather than edgy, especially if it is to be carried in a pocket. If it is intended for handheld use then there are also ergonomic aspects, such as fitting the device comfortably in the palm of your hand.

The design should focus on two things. The first is to fulfil the fundamental needs that the application promises, and the second is to establish an emotional connection between user and product. Only after that should we start looking for the latest trends in graphical design, for instance by evaluating the most popular apps in the app stores from different device types.

Branding, also discussed in a previous section, refers to helping a product establish a visual identity and familiarity and reinforces any message that you deliver. For mobile devices the opportunities are limited since the device is so small. However, competition is hard and a device only has a split second to get the first impression through. Search with a sparse, minimalistic design has been “branded” by Google also on the mobile phone. What would a mobile screen dump from Disney look like? What message is sent by it? Check it out for yourself. Other sites to ponder are Wikipedia, Amazon, Facebook, NY times, and Stardoll. The style guides that can be found for all mobile operating systems also say a lot about the particular brand.

Too much fancy graphics on a small screen will not be appreciated, even if it is an ever-so-genial artwork. The user will not see the details and if the bandwidth is low the delay of loading a large image can be annoying. Another problem is that the mobile web browser could rearrange items on a webpage hiding well thought out effects. Input adds to the problems. A keyboard consumes a lot of surface space as the touch sensitive display doubles as a keyboard on demand. The limitations of input also have another effect. Focusing on them means less time can be devoted to appearance and other aesthetic aspects.

One way to increase emotional bonding between device and user is customization. Let the user change its appearance, e.g. to put an orange sticker on the iPhone. If functionality is changed, rather than appearance only, then we talk about personalization. Adding ten apps to an Android phone is one way to personalize it. If the device changes its behavior without the user’s active intervention we have adaptation. This can be done in many small steps over time, e.g. by automatic updates of the software on the phone.

3.9.9 Pervasive systems

If we take a step back from the individual web page or the single device we will see the need to coordinate appearance over all screens visible at the moment, and for all touchpoints throughout the whole service. In this section we will add some short observations on the appearance of pervasive services. Most of the following discussion will be on visuals but most of the discussion can easily be extended to other media. The area of pervasive system design is new and for now the basic principle is remediation, i.e. to reuse conventions from the

individual media used in the pervasive service. It might seem that appearance is not as important in pervasive systems, after all, computers have disappeared. However, in pervasive systems we will often have multiple active displays to manage and it will often be necessary also to add visual features to the context to indicate functionality (or lack of it) and feedback from actions.

Three typical examples are driving a car, watching TV and travelling by air. All of them illustrate how the context, e.g. the car interior and the use of its digital equipment interact with performing the activity. Designing the context shapes the experience, such that for instance driving comes as a consequence within the service; rather than first designing the driving of the car and getting a context for driving, as well as for other services as a result of this. Each information element and every interaction object involved in the context and the activity is a target for design.

When functionality is computerised and hidden it is not always easy to figure out how to use that functionality or even what it is good for. We have to learn conventions on how to interact, and what the limitations of an application are. Imagine a remote control for a TV with a “record” button. What does pressing this button mean if you cannot see a VCR in the room? Is it some functionality added just in case, or is the remote control connected to the internet and if you press “record” the program will be stored somewhere (and you will be billed)?

One important design principle is the ambient signifier, small subtle changes of context that inform users. One example is short melodies played at railways stations, both identifying the station and welcoming passengers. Another idea that has been explored in research is a visual artwork in the living room changing slightly to signal the arrival of an email.

A Service safari (Mood service safari) can be used to get started. The designer goes out and explores services to identify trends in appearance and to be inspired. The idea is to step into the shoes of the customer/user and find out what works and what does not (Stickdorn, 2011).

3.9.9.1 Design guidelines

This section will list some guidelines for appearance design of pervasive services. Some of them have been presented earlier in this book but are important enough to repeat here. A typical scenario is a family watching a TV show in the living room while chatting away on the mobiles and looking for more information on the iPads. A large ambient display is hanging on the wall and there is an interesting radio show on philosophy coming up in just a couple of minutes that the father in the family wants to follow. A timer is set for a cake in the oven and a friend is coming over later to show a newly released computer game.

The guidelines are:

� Visual consistency – always, whenever possible

� A graphic composition with a strong visual hierarchy that organizes and presents the former aspects by forming a visual flow. It starts from the information architecture and interaction structure identified in previous design steps. Logical positioning of information and group related information, e.g. navigational cues are important.

� Provide for invitations, even the most revolutionary functionality will go unnoticed if it is hidden.

� Design context to optimize attention on the display.

� Position displays close to eye height to encourage glances.

� Transition between activities and displays should be smooth, keeping focus.

� Support the process of switching concentration between the service and surrounding factors of importance, and different everyday contexts. Immersion is maintained when the user to shift focus between the virtual and the physical parts of the service environment.

� System should react in a consistent, challenging, and exciting way to the users’ actions (e.g., appropriate audio feedback with the action). Calm aesthetcs for displays not in forcus and for notifications.

� Consistent cross-display object movement. This for instance includes mouse control if one or more mices are available.

� Content should be formatted to be adapted to different types and combinations of screens.Adapt content to screen and when this is not possible minimize content on screen and set appropriate legibility level. Some touch-screen devices allow for simple zooming, but a TV set does not.

� Bare necessities, only. Start with the content that is most valuable to the user. Easy to understand labels and cues, visible, descriptive rather than artistic.

� Reduce the amount of text, generous use of headings, split large texts into pages and use short paragraphs, and legible fonts on ever display. When arranging typographic elements, consider balance, emphasis, rhythm, unity, as well as positive and negative space.

� Effects of any artificial intelligence are clearly visible and consistent with the player’s reasonable expectations of the AI actor.

� Follow conventions (for the type of service and system).

� Break them only knowing why.

If there are several modalities involved in the interaction then these guidelines might apply, see also Section 2.8:

� Only choose multimodal options if there is a compelling, real and beneficial need.

� Match modality to task, need and context of use.

� Be consistent over modalities.

� Make sharing and updating of user data over modalities transparent to users.

3.9.10 Evaluation of appearance

Product and service experiences are the results of interactions, which can be seen as a conversations on many experiential levels, and from many viewpoints, among them the aesthetic. In a conversation, the user and the product get to know each other and the product tells a story about the user and the relationship between them. This relationship deepens from the moment that they meet. The trend towards digitally enabled service avatars, i.e. Internet of Things, makes user adaptation of services possible. This is a major distinguishing feature of

any service, and important in a harsh competition environment. All of the above need to be evaluated.

One problem is that appearance is dynamic and distributed in 3D for all but the most trivial service touch points. Physical products force us to use 3D animations to visualize them before the actual system is built. Given more resources in time and equipment we can build physical prototypes, e.g. moulding plastics. To equip these prototypes with interaction capability, technical expertise and creative engineering are needed, which are expensive resources indeed. For virtual subsystems we already have methodologies that we can borrow from web design, cinematography or image-based art.

The first impression is important and can be tested by prototyping appearance. This is a rapid, involuntary and aggregate assessment (Ulrich, 2007). Users prefer a beautiful artefact to an ugly one, and the aesthetic response is the primary response. Humans have many hardwired behaviours and the fastest responses are those that were useful in our evolutionary past. We for instance prefer landscapes that provide protection and vantage points. A VW beetle is cute because its form is associated with the face of a baby. It signals an honest aesthetics, i.e. “unlikely to be faked by the sender and therefore can be relied on” (Ulrich, 2007). This impression can however be tangled with since a product is always judged in context, a context that could be influenced by stories planted by the designer. Context is very important when we judge appearance. Consider the artwork “Fountain” by Duchamp for instance. It consisted of a urinal and replicas have been shown in many of the great museums of the world. Compare finding this piece of art in a toilet, in a museum or as a surprise gift in your kitchen at home.

A slightly different view from the evolutionary above is that contributions to the first impression can be characterized as hedonic and pragmatic (Hassenzahl, 2007). Pragmatic attributes are those related to the user’s goals and usability, such as whether the product is perceived as useful, controllable, easy to understand, clear, or supportable. Hedonic attributes are the rest. Pragmatic attributes are a kind of generalized affordances, not only considering identifying functionality, while hedonic attributes on the other hand “emphasize individuals’ psychological well-being” (ibid, p5). This could mean products that stimulate curiosity or perfection of knowledge and skills. Hassenzahl further refines hedonic attributes into those that stimulate personal development, help us express ourselves, and products that can help us remember. The close-to-immediate first impression means that we judge it an emotional, behavioural and other unconscious grounding, which in-turn will affect how well we think it works, and how much time we want to spend with it. As/If the system is used this impression can change and we might need to re-evaluate the appearance as well as the interaction after some time of use.

The simplest test of the appeal of an object is to try to buy it second hand. What other ways are there? When evaluating a cell phone we could ask: “Imagine that the cell phone is a person. How would you describe him/her: Playful, charismatic, dull, unlikeable, clean, or otherwise?” We in other words tell the story of the phone.

Using a similar analysis based on characteristic descriptive words pragmatics can also be measured. Why this works is because we have a good grip on most of the adjectives and adverbs we use, i.e. they are meaningful to us. It is not too difficult to build a story around any of them within a couple of seconds. An example set of words is listed in Table 3.9.2.

Comprehensible Incomprehensible

Supporting Obstructing

Simple Complex

Clear Confusing

Trustworthy Untrustworthy

Controllable Uncontrollable

Familiar Strange

Table 3.9.2 Dichotomies spanning pragmatics (Hassenzahl, 2007).

Hassenzahl identifies two characteristic types of user. The first one holds a low profile and avoids arousal, stress, and attracting attention. This user is sober, friendly and wants something that is easy to use. A user that is serious, planning, and goal oriented (Hassenzahl). Quite a different kind of user is the one that seeks arousal and wants to impress others, to be free. This user is cheerful and wants a playful distinctive product, signalling progress, and achievement (Hassenzahl, 2007). When the user types are identified we can list dichotomies for the categories, and try to position products or prototypes within the ranges of these dichotomies, e.g. ask on a scale from 1 to 7 how interesting–boring our product is. Some suggestions for values and dichotomies to use for the proposed characteristics are:

Hedonic dichotomies: (Interesting-Boring),(Extravagant-Cheap),(Exciting-Dull),(Exclusive-Standard), (Impressive-Nondescript), (Original-Ordinary), (Innovative-Conservative) User characterisations (Isolating-Integrating), (Amateurish-Professional), (Gaudy-Classy), (Cheap-Valuable), (Non-inclusive-Inclusive), (Takes me distant from people-Brings me closer to people), (Unpresentable-Presentable), (Typical-Original), (Standard- Creative), (Cautious-Courageous), (Conservative-Innovative), (Lame-Exciting), (Easy-Challenging), (Commonplace-New), (Technical-Human), (Complicated-Simple), (Impractical -Practical), (Cumbersome-Direct), (Unpredictable-Predictable), (Confusing-Clear), (Unruly-Manageable).

Donald Norman presents a three level model of user experience suggesting the visceral, behavioural and reflective experience levels. See also Section 3.1 (Norman, 2005). At the visceral level appearance and first impression matter and there are plenty of rules of thumb that should be kept in mind. Red text on a green background is difficult to read, and a soft, musical voice is preferred to a harsh, gruff, metallic one. A (partial) list of characteristics that can be used or avoided in a design are shown in the table below (Norman, 2005).

Preferred Avoided Warm, comfortably lit Cold or too hot, bright light Sweet smell and taste Rotting smells, bitter taste Rhythmic beats, soft, harmonious sounds

Loud, abrupt, discordant sounds

Rounded, smooth Sharp, edgy Smiling face Empty flat, or to crowded terrains Attractive people Even worse things, bläh

Table 3.9.3 Variables on the visceral level that can be used in design (Norman, 2005).

Desmet and Hekkert propose a similar model to the one by Norman (Desmet, 2003). Their three levels are

� Aesthetic experience – how a product is experienced by our senses.

� Experience of meaning, cognitive processing of stimuli – luxury and attachment are typical examples at this level.

� Emotional experience – emotions as interpretations of stimuli.

Also aesthetic experiences can be evaluated against a number of basic principles (Heckert, 2008). One idea behind the principles is that they are applicable to all of our senses. Aesthetic experiences are preferred when:

� They give maximum effect for minimum means; e.g. an interface that achieves its objective with as little as possible is preferred. The designer Dieter Rams formulated a similar principle; “good design is less design”.

� Unity in variety, one way is by grouping according to Gestalt laws and another is using contrast while still preserving a unified design, e.g. a dinner with complementary contrasting courses.

� Most advanced, yet acceptable, here we can compare with the principle of Occams razor, i.e. accept the simplest law that explains the data

� Congruency/appropriateness, we prefer “products that convey a similar message to all our senses” and where “form follows function”.

Another idea suggested by Desmet et al. is to measure emotions by selecting cartoon figures, or 3D figures with peculiar forms. When we do this we need to “calibrate“ participants and the emoticons to compare evaluations. The idea is that the direct association of an emotion to a figure will bring forward unconscious, affective information showing different levels of arousal and valence (Desmet, 2003). If the participants are trained to correlate figures and emotions, i.e. making the behaviour tacit the results might improve. Also, since context is important it needs to be neutralized, or if the context of use is determined, the context should be matched when evaluating alternatives.

Frameworks such as the ones suggested above can never give a final value when evaluating something. The complementary effects of the whole range of affects, preferences, moods and emotions in a particular situation will always be personal to a user. The techniques discussed in section 3.4 are valuable and applicable here. A persona can for instance be used to evaluate the decisions made on appearance.

Subconscious affect works at the visceral level, and a bad mood for instance indicates that something is wrong. Emotions are transient unconscious affective states that guide reflective thought and speed up decisions, which means that they are crucial factors when someone decides to spend money. This is one reason why there is music playing in the grocery store. How we design a product will manipulate the affective state of the user. Furthermore, reflective behaviour and affect interact, we think better when we are in a good mood, for instance when we are well fed, and reach for the coffee cup. Elation makes a person impulsive, speeds up decisions, and helps us to better solve problems, for instance to manage

a complicated user interface. We have all of us at some time even used a bad mood as an excuse for failure.

Short and by themselves insignificant experiences of a product are often forgotten, but are merged into diffuse opinions. The final verdict of a product will emerge over time and situations (Battarbee, 2007). The first experience will heavily influence the following experiences, but the most recent experience will be more important. One single failure could override an impeccable record of a long time of unremarkable use. A story could frame experience snippets and might mitigate otherwise negative responses, if the failures are natural consequence of the story presented. The summed response will be more stable, but can still be disrupted by a large, perhaps frustrating event.