laterotactile displays &...

Laterotactile Displays & Rendering : from Illusion to Application

Jerome Pasquero and Vincent Levesque

IEEE VR 2008 Tutorial on:“Integration of Haptics in Virtual Environments:

from Perception to Rendering”March 9th, 2008

From the tutorial abstract…

“The design of virtual environments using haptic interfaces remains often driven more by the availability of technologythan by the necessity to solve real users' issues. There is a need today for a clear change of perspective ...”

Two Complementary PerspectivesAn Example

Engineering DesignPerspective

Gap Interaction DesignPerspective

THMB device

The Key Main Players

Interaction DesignPerspective

• Prof. Vincent Hayward• Jerome Pasquero• Vincent Levesque• Qi Wang

• Prof. Karon MacLean• Joseph Luk• Shannon Little

Initial Stageinspiration

Observe tactile phenomenon & reduce it to its essence.

Identify context for which hapticfeedback offers potential benefit.

A Simple Illusionlateral skin deformation (laterotactile)

normal indentation

lateral deformation

instead of …

Hayward, V., Cruz-Hernandez, M. 2000. Tactile Display Device Using Distributed Lateral Skin Stretch. Proc. of the Haptic Interfaces for Virtual Environments and Teleoperator Systems Symposium, ASME Vol. DSC-69-2, pp. 1309-1314.

Mobile Interfacestypical issues & challenges

• High cognitive workload & competition for attention

• Limited screen real estate & awkward keypad space

• Unavailability of auditory and/or visual channel

Actuator Technologypicking the right one

Develop a suitable actuation technology.

Survey different technologies available & identify the most promising.

Tactile Feedback Technologystate-of-the-art

Electrostatic

Normal Indentation

Friction

Vibration

Lateral Skin Stretch

Electrocutaneous

Suction Pressure

Acoustic Radiation Pressure

Piezoelectric

MotorsShape Memory Alloys

Electromagnetic Micro-Coils

Peltier Elements

Tang & Beebe, 1998

Jungman, Schlaak, 2002Van Doren et al., 1987

Pneumatic Valve and Dimple

Taylor et al., 1997

Shinohara et al., 1998

Ino et al., 1993

Moy et al.,

Caldwell et al., 1999

ElectrostimulationKajimoto et al., 2003

Pressure Valve

Surface Acoustic Wave Nara et al., 2001

Makino et al., 2003

Ionic Conducting Polymer gel Film (ICPF)

et al.

Konyo et al., 2000

Technologies Interaction Modes

Air Jet

Asamura et al.,

1998Khoudja & Hafez, 2004

Summers &

Chanter, 2002

Pasquero, J. 2006. Survey on communication through touch. Technical Report TR-CIM-06.04. Center for Intelligent Machines, McGill University.

(laterotactile)

Tactile Feedback Technologylaterotactile transducer with piezo-actuators

laterotactiletransducer

piezoelectric bender

Prototypesincremental improvements

Virtual Braille Display(early iteration)

THMB Transducer(new design)

Tactile Iconsexploring the device’s expressive space

example of 6 different tactile icons(traveling waves)

tactile icon editor

transducer

Design Stagean iterative process

Develop specific usage scenarios that take into account the known strengths & limitations of the technology.

Design & build iterative prototypes with apps in mind.

The Resulting Device: THMBTactile Handheld Miniature Bimodal device

Pasquero, J., Luk, J., Levesque, V., Wang, Q., Hayward, V., MacLean K. E., Haptically Enabled Handheld Information Display with Distributed Tactile Transducer, IEEE transactions on Multimedia, 9(4), p. 746-753, June 2007..

Mobile Interaction Design3 different interactions

Spatial Navigationsection links

header

Browsing of large web pages

Luk, J., Pasquero, J., Little, S., MacLean, K. E., Levesque, V. and Hayward, V. 2006. A Role for Haptics in Mobile Interaction: Initial Design Using a Handheld Tactile Display Prototype. Proc. of the ACM 2006 Conference on Human Factors in Computing Systems, CHI 2006. pp. 171-180

target

Perceptual Characterizationwhere the two perspectives merge

Characterize the expressive perceptual capabilities of the prototype.

Perceptual Characterizationevaluating the device’s expressive capability

5 waveforms

0.35 m/s0.24 m/s0.18 m/s

100%50%

3 speeds 2 amplitudesX X

Perceptual characterization using 30 different tactile icons

Pasquero, J., Luk, J., Little, S. MacLean, K. E. 2006. Perceptual Analysis of Haptic Icons: an Investigation into the Validity of Cluster Sorted MDS. Proc. 14th Symposium on Haptic Interfaces For Virtual Environment And Teleoperator Systems IEEE VR 2006. pp. 437-444

Perceptual Characterizationconclusions

Most Distinguishable Stimulus Characteristics:1. Direction2. Waveform (e.g., texture vs. no texture, repeating vs. non-repeating)3. Amplitude4. Duration

Therefore,• Use direction and waveform as primary parameters for designing

haptic icons.• Use amplitude and duration as secondary.

Luk, J., Pasquero, J., Little, S., MacLean, K. E., Levesque, V. and Hayward, V. 2006. A Role for Haptics in Mobile Interaction: Initial Design Using a Handheld Tactile Display Prototype. Proc. of the ACM 2006 Conference on Human Factors in Computing Systems, CHI 2006. pp. 171-180

Back to the applications

Reconsider device design and applications in light of results from perceptual characterization.

Mobile Interaction DesignHow it all fits together

Spatial Navigationsection links

linksheader

Browsing of Large Web Pages

Use traveling waves icons to communicate speed and direction of movement.

Use different waveformsto give a sense of information flow when navigating

Experimental Validation

Validate the application.

Showing Added-Valuea challenge

section linkslinks

Header

Browsing of Large Web Pages

Could not find

improvement in

performancesl

Summary

Validate the application.

Back to the drawing boardlooking deeper for added–value

• Better awareness of the environment

• Less conspicuous gestures

• More appropriate in social contexts

Can laterotactile feedback reduce the number of glances at the screen ? Why would it be useful?

Overviewpart 2

2D Tactile Display Applications forvisually impaired persons

1. Braille

2. Tactile graphics

2D Tactile DisplaySTReSS2

Q. Wang, V. Hayward (2006) Compact, Portable, Modular, High-performance, Distributed Tactile Transducer Device Based on Lateral Skin Deformation. Proc. HAPTICS’06, pp. 67-72.

Brailleintroduction

Commercial braille displays

single line

40 or 80 braille cells

one actuator per pin

Laterotactile braille display

finger-sized display

virtual page of arbitrary size

cheaper and more portable

http://www.flickr.com/photos/44124267260@N01/2154495003

Brailletactile rendering

1. Implement rendering algorithms

based on

intuition

experience

experimentation

adjust with graphical interface

2. Select promising algorithms

session with reference users

3. Evaluate experimentally

with visually impaired subjects

V. Levesque, J. Pasquero and V. Hayward (2007) Braille Display by Lateral Skin Deformation with the STReSS2 Tactile Transducer. Proc. World Haptics Conference 2007, Tsukuba, Japan, pp. 115-120.

Brailledot rendering

Texture=0% Texture=25% Texture=100%

No texture

smooth bump

similar to braille dot

low contrast

Low texture

textured bump

increased constrast

No texture

smooth bump

low contrast

Texture alone

textured patch

different from braille

very high constrast

No texture

smooth bump

low contrast

Low texture

textured bump

increased constrast

Braillecell rendering

method 1 method 2 method 4method 3

Brailleletter identification experiment

S1 S2 S3 S4 S5 S6 S7 mean

T=100%

percentage of letters correctly read

Conclusions

display could be optimized for braille

rendering could be optimized for contrast

Task: read meaningless strings

Tactile Graphicsintroduction

Current technology

raised lines on paper or plastic

talking tactile tablets

experimental devices

2) dots1) grating

3) vibration 4) combined

Laterotactile display

3 rendering methods (so far)

programmable

interactive

http://www.rnib.org.uk

Tactile Graphics1) grating

Tactile Graphics2) dots

Tactile Graphics3) vibration

Tactile Graphics4) combined renderings

Tactile Graphicsexperimental results

• V. Levesque and V. Hayward (2008) Tactile Graphics Rendering Using Three Laterotactile Drawing Primitives. Proc. Haptic Symposium 2008, Reno, Nevada.

Shape identification

Icon identification

Orientation identification

Spatial frequency scaling

Tactile Graphicsapplications

tactile illustrations for schoolbooks

mathematical concepts, diagrams, bar charts, maps

collaboration with Université de Montréal

Demonstrations

tactile graphics

Haptic Symposium, March 13-14

mobile haptics

laterotactile displays &...

Documents

simulating haptic feedback using vision: a survey of...

six degrees-of-freedom haptic interaction with...

haptic ppt

haptic-emoticon: haptic content creation and sharing ... ·...

haptic plug in -...

haptic tech

haptic interface

haptic playback

pasquero et al. 1 haptically enabled handheld information...

english11 haptic

the rice haptic rocker: skin stretch haptic feedback with...

haptic technology

haptic technologies

contents haptic interaction -...

haptic interface using sensory illusion -...

haptic-emoticon: haptic content creation and sharing system...

observations on haptic experience and haptic perception...

haptic material: a holistic approach for haptic texture...

haptic line-drawings for blind persons - lth · haptic...

tactile graphics rendering using three laterotactile drawing