communication via the skin: the challenge of tactile displays
Post on 23-Feb-2016
32 Views
Preview:
DESCRIPTION
TRANSCRIPT
Communication via the Skin: The Challenge of
Tactile Displays
Lynette Jones Department of Mechanical Engineering,
Massachusetts Institute of TechnologyCambridge, MA
Spectrum of Tactile DisplaysSensory substitution Human-computer interactions Navigation/orientation
Visual impairments
Hearing impairments
Vestibular (balance) impairments
TSAS (Rupert et al.)
MIT tactile display
CyberTouch
Data Glove EST
Tactile mouse
Tactile belt
DataGlove
Tactile Displays - CTA ADA FocusUtilize a relatively underused sensory channel to convey
information that is private and discreet• Assist in navigation or threat
location in the battlefield• Increase SA in virtual
environments used for training• Enhance the representation of
information in displays
Torso-based Tactile Displays
0.1 1 10 100 1000-20
-10
0
10
20
30
40
Frequency (Hz)
Disp
lace
men
t (dB
re 1
um
pea
k)
Abdomen
Finger
Forearm
•Function as an alert•Orientation and direction information •Sequential activation of array – vector conveys “movement” in environment•Effective in environments with reduced visibility – enhances situation awareness
Vibrotactile Sensitivity
Development of Tactile Display• Actuator (tactor) selection and
characterization• Development of body-based system
(configuration of display, power, wireless communication)
• Perceptual studies – optimize design of the display in terms of human perceptual performance
• Develop a framework for creating a tactile vocabulary – tactons
• Field studies – measure the efficacy of display for navigation, identifying location of environmental events, and examine robustness of system (e.g. impact of body armor)
Characteristics of the Actuators EvaluatedCylindrical
MotorPancakeMotor
R1Rototactor
LengthDiameter (mm)
12.84.0
14 25.46.4
Mass (g) 0.87 1.6 3
Peak frequency (Hz)
110(at 4 V)
103(at 8.8 V)
200(at 10 V)
Peak accel.(ms-2) 492 254
Voltage (V) Rated: 3Range: 2-4
82.5-8.8 10
Rated maximum current (mA) 130 65 170
Current (mA)at 3.3 V 91 39 56
0.6 1.0 1.4 1.8 2.2 2.6 3.0 3.40
25
50
75
100
Freq
uenc
y (H
z)
Voltage (V)
Pancake Motor
0.5 0.7 0.9 1.1 1.3 1.5 1.7 1.9 2.10
25
50
75
100
125
150
175
200
Freq
uenc
y (H
z)
Voltage (V)
Cylindrical motor
Rototactor
Pancakemotor
Cylindricalmotor
(Jones, Lockyer & Piateski, 2006)
C2 tactor
Tactaid
Prototypes 2003-2007
Tactile Display - Final ElementsCore components - Pancake motors, Wireless Tactile Control UnitContact area - ~ 300 mm2 (encased in plastic)Input signal – 130 Hz at 3.3 V, sinusoidal waveformPower – 9 V battery or 7.2 V Li-ion rechargeable, 2200 mAhDisplay – vest, waist band, sleeve
Visual Basic GUI
Actuator Evaluation – Frequencies and Forces
0 1 2 3 4 5 6 7 8 9 10 11 12 13 1480
100
120
140
160
180
Motor number
Freq
uenc
y (H
z)
0 1 2 3 4 5 6 7 8 9 10 11 12 13 140
0.5
1
1.5
2
2.5
Motor number
Forc
e (N
)
Mechanical properties not affected by encasing motors (Jones & Held, 2008)
Actuator Evaluation – Tactor spacing and Intensity
y = 0.3765x + 11.46R2 = 0.979
0
10
20
30
40
50
60
70
80
90
0 50 100 150 200 250
Peak Frequency in Impedance Head (Hz)
Peak
Fre
quen
cy o
n Sk
insi
m (H
z)
Mechanical testing of skin
Skinsim with accelerometers(Jones & Held, 2008)
Transitions – MIT Tactile DisplayARL
Investigated the efficacy of tactile and multimodal alerts on decision making by Army Platoon Leaders (Krausman et al., 2005, 2007) Analyzed the effectiveness of tactile cues in target search and localization tasks and when controlling robotic swarms (Hass, 2009)Evaluated Soldiers’ abilities to interpret and respond to tactile cues while they navigated an Individual Movement Techniques (IMT) course (Redden et al., 2006) Measured the effects of tactile cues on target acquisition and workload of Commanders and Gunners and determined the detectability of vibrotactile cues while combat assault maneuvers were being performed (Krausman & White, 2006; White et al., in press).
The MIT tactile displays have also been incorporated into multi-modal platforms developed by the University of Michigan, ArtisTech in the CTA test bed, and Alion MA&D for a robotics control environment.
Questions addressed – MIT Research• Can tactile signals be used to provide spatial cues about
the environment that are accurately localized? • How does the location and configuration of the tactile display
influence the ability of the user to identify tactile patterns?• What is the maximum size of a tactile vocabulary that could
be used for communication? • Which characteristics of vibrotactile signals are optimal for
generating a tactile vocabulary? • Can a set of Army Hand and Arm Signals be translated into
tactile signals that are accurately identified when the user is involved in concurrent tasks?
Localization of Tactile Cues for Navigation and Orientation
Navigation– Way-finding– Location of events – real and
simulated environments– Control of robots
Waist BackExperiments10 subjects in each
experimentEach tactor activated 5
times (randomly)Subject indicate location
of tactor vibrated
Navigation – Tactile Belt – One-dimensional Display
1
2
3
4
8
6
5
7
Navel
Spine
Left Right
Identification of tactor locationEight locations – 98% correct
(Inter-tactor spacing: 80-100 mm)Twelve locations – 74% correct
(Spacing: 55-66 mm)(Jones & Ray, 2008)
Inter-tactor distance 80-100 mm
Localization – Two-dimensional Display
Identification of tactor location16 locations – 59% correct (40-82%)
Within 1 tactor location: 95%Inter-tactor spacing: 40 mm vertical
60 mm horizontal Darker the shading, the more accurate the localization
(Jones & Ray, 2008)
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 160
102030405060708090
100
Tactor number
Perc
ent c
orre
ct
Results• Spatial localization becomes more difficult as the number of
tactors increases and the inter-tactor distance decreases• Two-dimensional 16-tactor array on the back is unable to
support precise spatial mapping, for example between tactile location and visual target –driving or to highlight on-screen information
• One-dimensional array is very effective for conveying directions
Questions addressed• Can tactile signals be used to provide spatial cues about the
environment that are accurately localized? • How does the location and configuration of the tactile
display influence the ability of the user to identify tactile patterns?
• What is the maximum size of a tactile vocabulary that could be used for communication?
• Which characteristics of vibrotactile signals are optimal for generating a tactile vocabulary?
• Can a set of Army Hand and Arm Signals be translated into tactile signals that are accurately identified when the user is involved in concurrent tasks?
Location and Configuration of Tactile Display• Tested vibrotactile pattern recognition on forearm and back• Fabricated 3x3 (arm) and 4x4 (torso) arrays both controlled
by Wireless Tactile Control Unit (WTCU)• Tactile patterns varied with respect to spatial cues (location),
amplitude (number of tactors simultaneously active) and spatio-temporal sequence.
Group mean percentage of correct responses – averaged across tactors – 89%
Actual Pattern A B C D E F G H
A 80% 0% 1% 8% 2% 1% 8% 0%B 1% 86% 6% 3% 0% 1% 3% 0%C 2% 0% 95% 0% 2% 1% 0% 0%D 0% 8% 0% 90% 2% 0% 0% 0%E 0% 0% 1% 1% 96% 2% 0% 0%F 1% 4% 0% 1% 5% 86% 2% 1%G 0% 5% 3% 3% 0% 3% 84% 2%H 0% 1% 0% 1% 0% 0% 1% 97%
Subject Response
A
C D
E F
G H
B
Up Down
Right Left
Left, right, left
Blink X-shape 3 times
1
2
3
1 1
2
2
3
2
1
1
2
2
2
1
1
2
3
1,3
3 2
3
3
1,3
2
3
1,3
2
2
3
3
1
2 3
1
1
2
32
3
1
1
1, 2, 3
22
1,3
2
1,3
1,3
1, 2, 3
Top, bottom, top
Blink center 3 times
Results
(Piateski & Jones, 2005)
Tactile Patterns
2
3
4
2 2
4
3 3
4
2
3
4
1 1 1 1
3
2
1
3 3
1
2 2
1
3
2
1
4 4 4 4
1
1
2
2 3
1
2 3
3
4
4
4
1 2 3 4
4
4
3
3 2
4
3 2
2
1
1
1
4 3 2 1
A
2, 4
2, 4
2, 4
2, 4
1, 3
1, 3
1, 3
1, 3
2, 4 2, 4 2, 42, 4
1, 3 1, 3 1, 31, 31,2, 3,4
1,2, 3,4
B C D
EF G H
Up Down Right Left
Left, right, left, right
Top, bottom, top, bottom
Blink corners 4 times
Blink single motor 4 times100%100%100%
100%100%99%
99%
97%
Back -Tactile Pattern Recognition
(Piateski & Jones, 2005)
1 2 3 43,4
NM2,4
1,3
4,8 3,7 2,6 1,51,3 2,41,3
2,4Top, bottom, top, bottom
Bottom, top, bottom,
top
Right, left, right, left
Each corner vibrates twice
Up
H
Down
ED1,2
5,6
7,8
1
RightLeft
Left, right, left, right
Single tactor vibrates four
times
1234
4321
A B FC G
I
I
1,3
2,4
4 3 2 1
K
Corners vibrate together four times
Middle two rows
Outer corners then inner twice
L
Diagonal vibrates four
times
O
Two corners vibrate in turn twice
Tactile Vocabulary – 15-20 Tactons?
A B C D E F G H I J K L M N O70
75
80
85
90
95
100
Tactile pattern
Perc
ent c
orre
ct
Mean: 96%
J
(Jones, Kunkel, & Torres, 2007)
Experiment 1A
3
3 3
A C D
E F G H
B
Up Down Right Left
Up and right Blink X-shape 3 times
2
3 3
1,3
1,3
1,3
1
1, 2, 3
1 1
1
Up and left Left, right, left
3
3
3
3
3
3
3 3 3 3 2
2
2
2
2
2
2 2 2
2
2
2
2 2
1 1 1
1 1 1
1
1
1 1
1
2
2
2
3 3
3
3 3
3
Experiment 1B
A B C D E F G H0
20
40
60
80
100
Tactile pattern
Cor
rect
res
pons
es (%
)
Mean correct response rate:62% in Expt 1A 85% in Expt 1B IT: 1.48 bits IT: 2.15 bitsConfusion matrix (Expt 1A): A misidentified as F, whereas F mis-identified as D – errors not symmetrical.Tactile patterns that “moved” across the arm more accurately perceived than those that “moved” along the arm
Tactile Pattern Recognition – Effect of Stimulus Set
(Jones, Kunkel, & Piateski, 2009)
A C D
E F G H
B
Up Down Right Left
Blink center 3 times Blink X-shape 3 times
2
3 3
1
1
1, 2, 3 1,3 1,3 1,3
2
Top, bottom, top Left, right, left
3
3
3
3
3
3
3 3 3 3
2
2
2
2
2
2
2
2 2 2
2 2 2
2 2
1 1 1 1
1 1 1
1
1 1
2
2
1,3
1,3
1,3
Summary of FindingsArm vs back – both provide effective substrates for
communicationArray dimensions – marked effect on spatial localizationAsymmetries in spatial processing on the skinNeed to evaluate patterns in the context of the “vocabulary” usedTactile vocabulary size – absolute identification vs communicationInterceptor Body Armor - no effect on performance
Direction and orientation
Saltation
Tap on shoulder
Navigation path
Field ExperimentsFive subjects participatedEight patterns with five repetitions
Familiarization with visual analog initially
Brief training period outdoors
Navigation using only tactile cues, without feedback100% accuracy for 7/8 patterns presentedSingle error on 8th patternDemonstrated that navigation is accurate using only tactile cues as directions
(Jones, Lockyer & Piateski, 2006)
Questions addressed• Can tactile signals be used to provide spatial cues about the
environment that are accurately localized? • How does the location and configuration of the tactile display
influence the ability of the user to identify tactile patterns?• What is the maximum size of a tactile vocabulary that could
be used for communication? • Which characteristics of vibrotactile signals are optimal
for generating a tactile vocabulary? • Can a set of Army Hand and Arm Signals be translated into
tactile signals that are accurately identified when the user is involved in concurrent tasks?
Tactons (tactile icons)Structured tactile messages that can be used to communicate
information.These tactons must be intuitive and salient.
1
1
2
2 3
1
2 3
3
4
4
4
1 2 3 4
4
4
3
3 2
4
3 2
2
1
1
1
4 3 2 1
2, 4
2, 4
2, 4
2, 4
1, 3
1, 3
1, 3
1, 3
2, 4 2, 4 2, 4 2, 4
1, 3 1, 3 1, 3 1, 3
1,2, 3,4
C D
E F
G
Turn right Turn left
Arm horizontal Arm vertical
Stop at next cone
1,2, 3,4
H
Hop
Move forward Turn around
2
3
4
2 2
4
3 3
4
2
3
4
1 1 1 1
3
2
1
3 3
1
2 2
1
3
2
1
4 4 4 4
A B
Navigation tactons
Communication tactons
Assemble/rally
1,23,4
5,6 7,8
Tactons for hand-based
communicationFrequency
Duration, repetition rate
Waveform complexity
(Jones & Sarter, 2008)
Frequency Intensity Waveform Duration Location
Range:0.4-1000 Hz.
Optimal sensitivity:
150-300 Hz1
Absolute thresholds
across body sites: 0.07-14 m at 200 Hz3
Relatively insensitive to waveforms: sinusoidal, triangular,
square wave5
Burst duration: 80-500 ms (typical)
Differential thresholds: 7-
50%5
Localization accuracy varies with body site7
Body site influences perceived frequency
Changes with increased
voltage to a single tactor
and with number of
tactors activated
Amplitude modulation of
sinusoids effective for
varying roughness of
signals6
Pulse repetition rate (create temporal patterns - rhythms)
Inter-tactor spacing and array
configuration important
Differential thresholds:
18-50%2
Differential thresholds: 5-
30%4
Number of pulses:
1-5 (typical)
Localization superior near
anatomical points of reference (elbow,
spine)7
Tacton building blocks: Relevant properties of each variable
(Jones, Kunkel, & Piateski, 2009)
Arm and Hand Signals for Ground ForcesIdentify a set of structured
tactile messages (tactons) that can be used to communicate information.
1,7
4 4 4 4 1 1 1 1
2 2 2 2
3 3 3 3
4 4 4 41,7
1,7
1,7
2,6
2,6
2,6
2,6
3,5
3,5
3,5
3,5 4
41
1
2
2 3
3
Take coverIncrease speed Danger area
4
4
4
4
1
1
1
1
Advance or Move Out
2
2
2
2
3
3
3
3
Halt
4
3
2
1
4
2
3
1 11
2 2
3 3
4 4 2, 4
2, 4
2, 4
2, 4
1, 3
1, 3
1, 3
1, 3
Attention
Each tactile hand signal was designed to keep some of the iconic information of the matching visual hand signal
A B C D E F G0
20
40
60
80
100
Tactile pattern
Cor
rect
res
pons
es (%
)
Mean (N=10) percentage of correct responses (35 trials per subject) when identifying the hand signal with both the illustration and schematic available
(black - 98% correct) and with only the illustration available (red – 75% correct).
Hand and Arm Signals – Tactile-visual mapping
(Jones, Kunkel, & Piateski, 2009)
Questions addressed• Can tactile signals be used to provide spatial cues about the
environment that are accurately localized? • How does the location and configuration of the tactile display
influence the ability of the user to identify tactile patterns?• What is the maximum size of a tactile vocabulary that could
be used for communication? • Which characteristics of vibrotactile signals are optimal for
generating a tactile vocabulary? • Can a set of Army Hand and Arm Signals be translated
into tactile signals that are accurately identified when the user is involved in concurrent tasks?
Field Experiments – Concurrent activities
1 2 3 4 5 6 7 80
20
40
60
80
100
Hand Signal Identification
Walking
Jogging
Cognitive task
Hand signal
Perc
ent c
orre
ct
(Jones, Kunkel, & Piateski, 2009)
Nuclear, biological and chemical attack
AssembleAttentionIncrease speed
Take cover
Advance to left
Danger area
Halt
91%91%93%
Conclusions Vibrotactile patterns easily perceived on torso with little
training and single stimulus exposure Demonstrated feasibility of using sites that are non-
intrusive and body movements are not impeded Shown that the ability to perceive tactile patterns is not
affected by concurrent physical and cognitive activities Directional patterns are intuitive and can readily be used
as navigational and instructional cues Two-dimensional arrays provide greater capabilities for
communication, but one-dimensional arrays are effective for simple commands
AcknowledgementsEdgar TorresAmy LamDavid HeldChrista MargossianKatherine Ray
Brett LockyerMealani NakamuraErin PiateskiJacquelyn Kunkel
Research was supported through the Advanced Decision Architectures Collaborative Technology Alliance sponsored by the U.S. Army Research Laboratory under Cooperative Agreement DAAD19-01-2-0009.
ReferencesJones, L.A., Kunkel, J. & Piateski, E. (2009). Vibrotactile pattern recognition on the
arm and back. Perception, 38, 52-68.Jones, L.A. & Held, D.A. (2008). Characterization of tactors used in vibrotactile
displays. Journal of Computing and Information Sciences in Engineering, 044501-1-044501-5.
Jones, L.A. & Ray, K. (2008). Localization and pattern recognition with tactile displays. Proceedings of the Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems, 33-39.
Jones, L.A. & Sarter, N. (2008). Tactile displays: Guidance for their design and application. Human Factors, 50, 90-111.
Jones, L.A., Kunkel, J., & Torres, E. (2007). Tactile vocabulary for tactile displays. Proceedings of the Second Joint Eurohaptics Conference and Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems, 574-575.
Jones, L.A., Lockyer, B., & Piateski, E. (2006). Tactile display and vibrotactile pattern recognition on the torso. Advanced Robotics, 20, 1359-1374.
Piateski, E. & Jones, L.A. (2005). Vibrotactile pattern recognition on the arm and torso. Proceedings of the First Joint Eurohaptics Conference and Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems, 90-95.
top related