touchless interaction from an embodied perspective
TRANSCRIPT
Touchless
from
an
Embodied
Interaction
Perspective
Touchless
from
an
Embodied
Interaction
Perspective
Exploring Perceptual and Motor
Gestalt in Touchless Interactions with
Distant Displays
Debaleena ChattopadhyayAdvisor: Davide Bolchni
January 16, 2015
In our everyday life..
we are constantly using our
hands to interact with objects..
Opening a door Closing a door
Holding an objectReaching for
an objectPushing a button
For example..
Wiping
These “natural” gestural
interactions have two typical
properties
Haptic Feedback
A 3D Worldview
But touchless interactions
with distant displays lack
both
Haptic Feedback
A 3D Worldview
Everyday
gestural
interactions
Touchless
interactions
with distant
displays
Public
Spaces
Sterile
Operating
Rooms
Interactive
TVs
Distant
Displays
Touchless
Interaction
Usage contexts
Large-scale
InfoVis
Walter, et al., 2013
O’Hara, et al., 2014
Vatavu, et al., 2014
Dostal, et al., 2014
Haptic FeedbackA 3D Worldview
Touchless
Explore touchless from an
embodied interaction
perspective
Heidegger’s phenomenological view
Gibson’s ecological psychology
Gestalt Psychology
Heidegger’s
phenomenological view
Device – less
interaction
Device – based
interaction
there is no embodied conception of a tool—no
transition of an input device from present-at-
hand (an object of activity) to ready-to-hand
(absorbed in the fabric of the activity)
But touchless interactions
with distant displays lack
both
Haptic Feedback
A 3D Worldview
Gibson’s ecological
psychology
cognition as a construct involving the ability of
an entity, action, and the environment
3D Worldview 2D Display
Violation of how seeing and acting is related in
our familiar 3D environment
Gestalt Psychology
the psychological principles
affecting visual perception, such
as Gestalt principles, will have a
significant effect on touchless
motor action
Preliminary Findings
Perceptual grouping by Similarity of
Orientation principle affected the efficiency of
touchless interaction: Expert users were faster
when crossing-to-select a rectangular menu
option than a circular menu option with a circular
cursor. 1
Perceptual
grouping by
Continuity affected
the effectiveness of
touchless
interactions: Users
were less precise
when aiming an
isolated rectangular
target (A) compared
with a rectangle as
part of a semi-
circular
arrangement (B).2
When intending to make mid-air movements
perpendicular to a vertical display, such as a
pull gesture, users repeatedly made oblique
motions toward the center of their torso—to
optimally reach static equilibrium, thus
minimizing their body’s energy expenditure.
Law of Prägnanz
affected gesture intuitiveness.
3
Our findings are posteriori
arguments …Chattopadhyay, D., & Bolchini, D. (2014). Touchless circular menus: toward an
intuitive UI for touchless interactions with large displays. Proc. AVI, 33 – 40.
Chattopadhyay, D., & Bolchini, D. (2014). Motor-Intuitive Interactions Based on
Image Schemas: Aligning Touchless Interaction Primitives with Human
Sensorimotor Abilities, [Special Issue on Intuitive Interaction] Interacting with
Computers (in press).
Chattopadhyay, D., & Bolchini, D. (2014). Understanding Visual Feedback in
Large-Display Touchless Interactions: An Exploratory Study. [Research Report]
IUPUI Scholar Works, Indiana University.
Proposed Research
Aim 1: Identify how Gestalt principles of visual
perception and motor action affect touchless
user experience (UX).
Aim 2: Design and evaluate Gestalt-informed
touchless interaction techniques for large-
display information visualizations (InfoVis).
controlled, quantitative studies
iteration, formative evaluation
At a Glance
Thank you.We thank the study participants and UITS AVL. This research is partially supported by an IUPUI Research Support Funds Grant.
1
2
3
4
Touchless interaction with distant, 2D displays lack haptic
feedback and a 3D world view.
I explore the mismatch between gestural interaction in
everyday life and touchless interaction with distant
displays from an embodied interaction perspective.
I argue that a lack of haptic feedback and a 3D worldview
violates how seeing and acting is related in a familiar
environment.
My preliminary findings suggest that Gestalt principles of
proximity, similarity and Prägnanz may affect touchless.