physical computing theme on designing interactive...

Physical Computing theme on Designing Interactive Artifacts F-2013

Proximity-based interaction

Thomas Pederson <[email protected]>Associate Professor

Pervasive Interaction Technology Lab – http://itu.dk/pit/ Interaction Design research group – http://www.itu.dk/research/ixd/

1

Thomas Pederson DDDA-‐F2013 Proximity-‐based interaction

AgendaReality-Based Interaction (Jacob et al., 2008)


• Proxemics for Ubicomp (Greenberg et al., 2011)• Situative Space Model (Pederson et al., 2012)

exercise

• proximity-based interaction using ultrasonic sensor

virtual world

physical world

2


AgendaReality-Based Interaction (Jacob et al., 2008)


• Proxemics for Ubicomp (Greenberg et al., 2011)• Situative Space Model (Pederson et al., 2012)

exercise

• proximity-based interaction using ultrasonic sensor

virtual world

physical world

3


last week: mixed-reality perspective on physical computing

virtual world

physical world

physical computing

Reality-Based Interaction

4


last week: mixed-reality perspective on physical computingAugmented/Mixed Reality

Ubiquitous/Pervasive Computing

Graspable/Tangible User Interfaces

Wearable Computing

Context(ual) Aware Systems

virtual world

physical world


5


New interfaces are breaking the boxAugmented/Mixed Reality

Ubiquitous/Pervasive Computing

Graspable/Tangible User Interfaces

Wearable Computing

Context(ual) Aware Systems

virtual world

physical world


WIMPcommandline

6


Reality-Based Interaction (Jacob et al., 2008)

a concept that ties together many new post-WIMP interaction styles

post-WIMP

• interfaces "containing at least one interaction technique not dependent on classical 2D widgets such as menus and icons" (van Dam)

the RBI framework helps us analyse these new interactive systems


7




8



common sense knowledge about the physical world

• gravity, friction, velocity, the persistence of objects, relative scaleRBI examples

• TUI constraints enforced by the size and shape of physical tokens• illusion of mass in iPhone menus ("rubber-band effect")


9



the understanding of our own body and it's state

• relative position of the limbs (proprioception), motion range, senses• muscle coordination skills acquired early in life (limbs, head, eyes)

RBI examples

• two-handed interaction• full-body interaction


10



the understanding of spatial relationships among objects, near and far

• facilitate our sense of orientation and spatial understanding• example clues: horizon, atmospheric color, fog, lighting, and shadow

RBI examples

• Augmented/Mixed Reality systems make use of these visual clues• Context-Aware systems base their behaviour on the physical context


11



People possess skills for social interaction and adapt to presence of others

• e.g. verbal and non-verbal communicationRBI examples

• TUIs support co-located collaboration• VR massively multiplayer online role-playing games (MMORPG) exploit

social awareness and skills


12


implications for designbasing a user interface on pre-existing real-world knowledge can...

• speed up learning• decrease mental effort• improve performance

... but will not necessarily lead to a useful system:

• computer systems are powerful tools because they can help us do things beyond what can be done in the real world


13


RBI design tradeoffssuggested approach

• give up reality only explicitly and only in return for other desired qualities


14


Proxemics for Ubicomp (Greenberg et al., 2011)

"When you walk up to your computer, does the screen saver stop and the working windows reveal themselves? Does it even know if you are there? How hard would it be to change this? Is it not ironic that, in this regard, a motion-sensing light switch is “smarter” than any of the switches in the computer...?"

(Buxton, 1997)

15


devices are too blindDevices are blind to the presence of

• other computational devices• non-computational aspects of the environment- people

- non-digital objects

- the room's fixed and semi-fixed features

Proxemics for Ubicomp

16


devices are too blinddesign proposal

• "Just as people expect increasing engagement and intimacy as they approach others, so should they naturally expect increasing connectivity and interaction possibilities as they bring their devices in close proximity to one another and to other things in the ecology."


17


five dimensions of proxemics


18


five dimensionsof proxemicsdistance is continuous, but can be interpreted in discrete steps


19


five dimensionsof proxemicsdistance is continuous, but can be interpreted in discrete steps, e.g.

• "personal reaction bubbles" (Hall, 1966)


20


five dimensionsof proxemicsdistance is continuous, but can be interpreted in discrete steps, e.g.

• "personal reaction bubbles" (Hall, 1966)

• "interaction zones" (Vogel & Balakrishnan, 2004)


21


five dimensionsof proxemicscontinuous

• pitch, yaw, roll angles• used in- gesture-based interfaces

- tangible user interfaces

discrete

• used in for instance attentive UIs- looking at the screen

- looking nearby the screen

- not looking at the screen


22


five dimensionsof proxemicscaptures the distance and orientation of an entity over time

• speed of motion

• moving and turning toward versus away from another entity

used in many kinds of post-WIMP UIs

• full-body UIs (Nintendo Wii, Microsoft Kinect)• gesture-based UIs• tangible UIs


23


five dimensionsof proxemicsuniquely describes the entity

different levels of detail

• exact identity and attributes (e.g. Thomas Pederson, the prof at ITU)

• only object type (a prof at ITU)• a person


24


five dimensionsof proxemicsthe physical context in which the entities reside

• e.g. a specific room and its characteristics


25


example system:the ViconFace

The Proximity Toolkit and ViconFace: The Video

• http://www.youtube.com/watch?v=BhT0QgTSddM


26


Proxemic Interaction: Designing for a Proximity and Orientation-Aware Environment

• http://www.youtube.com/watch?v=OHm9teVoNE8


example system:the proxemic media player

27


exercise: apply RBI to the proximity-based media player

how much of the interaction is something we know from the real world, how much is new?


what RBI design tradeoffs have the designers made?

28


Situative Space Model (Pederson et al., 2012)

29


Situative Space Modelused for modeling interaction situations in the emergingegocentric interaction paradigm:

• ‘users’ are transformed into agents that are active in a mixed-reality environment

• also the classical concepts of (system) ‘input’ and ‘output’ become transformed into (human agent) action and perception, respectively- object manipulation and perception are processes that can involve any

modality: tactile, visual, aural, etc.

- virtual objects are manipulated andobserved through "mediators"

30


Situative Space Modelintended to capture what an agent can perceive and not perceive, reach and not reach, at any moment in time

• uses a number of spaces and sets for this purpose

31


Situative Space Model

SpacesWorld Space

• encompasses all physical and virtual objects that are part of a specific model

32



SpacesWorld Space


Perception Space

• the part of the space around the agent that can be perceived by the agent- varies continuously with agent movements

- different senses have differently shaped Perception Spaces

33



SpacesWorld Space


Perception Space

• the part of the space around the agent that can be perceived by the agent- varies continuously with agent movements

- different senses have differently shaped Perception Spaces

Action Space

• the part of the space around the agent that is accessible for physical actions by the agent

34



SetsRecognizable Set

• set of objects in Perception Space currently within their recognition distances

35





Examinable Set

• set of objects in Perception Space currently within their examination distances

36





Examinable Set


Selected Set

• set of objects in Action Space currently being physically or virtually handled

37





Examinable Set


Selected Set

• set of objects in Action Space currently being physically or virtually handled

Manipulated Set

• set of objects in Action Space whose states are currently being changed by the agent

38



An illustration: having breakfast

39



An illustration: having breakfast

40


The Egocentric Interaction Paradigm

An application

The easyADL home

Prototyping wearable activity support for persons suffering from mild dementia

41



UsesActivity recognition

Providing data for a multimodalinteraction manager

42


The challenge of determining the spaces and sets in realtime


43


time to play with arduino and proximity!activities

• lecture 1: why and how computing is getting physical – a historical perspective and some example areas of research- exercise: arduino basics

• lecture: 2 proximity-based interaction- exercise: proximity-based interaction

• lecture 3: materiality- exercise: physical displays

• mandatory hand-in: reflective report on physical computing

virtual world

physical world

44

physical computing theme on designing interactive...

Documents