VibPress

Estimating Pressure Input

Using Vibration Absorption on Mobile Devices

Presenter: S.J. Hwang

Ph.D. Candidate | 6th Semester

BACKGROUND VibPress

BACKGROUND

Human interacts with environmental objects by grasping,

holding, twisting, pressing with pressure.

BACKGROUND

However, current touch screen

does not support pressure-sensitive input.

BACKGROUND

1. It can free the user from spatial restrictions and repetitive movement (Miyaki, & Rekimoto, 2009).

2. It is suitable for one-handed interaction, adds a degree of

freedom to touch locations (Boring et al., 2012).

3. Pressure input allows relatively stable and accurate interactions

when user is in mobile context (Wilson et al., 2011).

4. It can be used to alleviate occlusion problems and provide

ways for rich contextual selections (Ramos et al., 2004).

Advantages of pressure input for mobile device

PREVIOUS WORK VibPress

PREVIOUS WORK

Hardware augmentation approach Pressure estimation in software

Pressure input techniques for mobile devices

PREVIOUS WORK

GraspZoom (Miyaki et al., 2009)

Pressure-based Text Entry (Brewster et al., 2009)

Squeezing the Sandwich (Essl, 2009)

Pressure-based Menu Selection (Wilson et al., 2010)

ForceGestures (Heo et al., 2011)

Indirect Shear Force (Heo et al, 2013)

Multi-digit Pressure Input (Wilson et al., 2012)

ForceDrag (Heo et al., 2012)

Hardware Augmentation Approach

PREVIOUS WORK

Hardware Augmentation Approach to Enable Pressure Interaction

PREVIOUS WORK

Hardware Augmentation Approach

Specialize Hardware offers reliable and fast input pressure

readings.

However,

- It is still not widely supported in current mobile devices.

- It represents additional costs (production cost for vendors and

maintenance costs for users).

c

t

a

PREVIOUS WORK

Software Approach to Enable Pressure Interaction

A tangible controller (Kato et al., 2009)

Vision-based Force Sensor (Sato et al., 2012)

The Fat Thumb (Boring et al., 2012)

Muscle Tremors (Strachan et al., 2004)

Use the Force or something (Essl et al., 2010)

ForceTap (Heo et al., 2011)

Expressive typing (Iwasaki et al., 2009)

camera

touchscreen

accelerometer

c t

a

Technique (Author) Sensor

Repurposed P levels Modality emulated (type) Property Used A/P

A tangible controller

(Kato et al., 2009) Camera (cont) Pressure (S) Marker position and size P

Vision-based Force Sensor

(Sato et al., 2012) Camera (cont) Pressure (S) Marker position and size P

Muscle Tremors

(Strachan et al., 2004) accelerometer 2 Pressure (S) Vibration P

Expressive typing

(Iwasaki et al., 2009)

Accelerometer

Keyboard 2 Pressure (E) physical movement P

ForceTap

(Heo et al., 2011)

Accelerometer

Touchscreen 2 Pressure (E) physical movement P

Use the Force or something

(Essl et al., 2010) Accelerometer N/A Pressure (E) physical movement P

The Fat Thumb

(Boring et al., 2012) Touchscreen 2 Pressure (E) Contact size P

A, Active; P, Passive; P/A, Passive/Active; S, Streaming based; E, Event-based; R, Recognition-based;

PREVIOUS WORK

Software Approach to Enable Pressure Interaction

PREVIOUS WORK

Software Approach to Enable Pressure Interaction

- They do not continuously measure pressure. (e.g., ForceTap, MicPen, and Expressive typing)

- The interaction area is limited by hardware settings.

(e.g., PseudoButton)

- An additional calibration process is needed.

(e.g, The Fat Thumb, and Muscle tremor)

These approaches successfully estimated the input pressure using only inertial sensors on mobile devices.

However,

OUR METHOD VibPress

OUR METHOD

Vibration Absorption As Proxy For Pressure Input

Amount of pressure on a mobile device can be approximated by using an

accelerometer to measure the spatial displacement generated when the internal

vibration motor vibrations.

OUR METHOD

Implementation

- Implemented on the Samsung Galaxy S3, running Android OS 4.0.4.

- Activated internal linear vibration motor with a continuous pulse at

maximum amplitude (setting on OS).

- Sampled the data from the internal three-axis accelerometer at 100Hz.

- Passed through a low-pass filter to reduce sampling noise and

Euclidean distance between last reading and its predecessor is

computed.

- Finally, we used the sum of these values to reach a good estimation of

the amount of pressure exerted on the device.

OUR METHOD

VibPress

10-2012-0110731, Method and apparatus for sending touch strength on user terminal and method, Korea Patent, Pended (2012.10.05) 10-2012-0066225, Apparatus and method for touch Sensing, Korea Patent, Pended (2012.06.19)

Finally, we built software that reliably and quickly estimate the input pressure by

measuring the spatial displacement absorbed. And we filed some patents based

on the technique.

PREVIOUS WORK

GripSense (Goel et al., Oct, 2012)

Muscle tremor (Gyroscope) + + Finger size (Touchscreen)

Machine learning algorithm

Damping effect (Gyroscope)

PREVIOUS WORK

Main differences from the previous work

GripSense (Goel et al., 2012)

Technique P levels Sensor Used Estimation

Method A/P

Postures

Explored

GripSense

(Goel et al., 2012) <= 3

Gyroscope

Touchscreen Machine-learning Active Pressure

VibPress

(Hwang et al., 2013) <= 6 Accelerometer Real-time readings Active

Pressure

Squeeze

Our approach is fast and lightweight, supporting two types of gestures.

EVALUATION VibPress

EVALUATION

Pilot study

The goal of the pilot study was to establish the average minimum

and maximum pressure that user could exert on the mobile device in

order to derive an appropriate estimation of the maximum number

of distinguishable pressure levels.

Usability study

The goal of the usability study was to test the input performance

(time, errors, and cognitive load) for different pressure levels and

hand gripping gestures derived from the pilot.

Pilot Test

- 4 participants (2 women) aged between 26 and 33 (average

29.5 , SD 3.1)

- Asked to sequentially apply maximum and minimum

pressure with their dominant hands using two postures

(touch and grip).

- For touch gesture, participants asked to press the center of

the screen with their thumbs.

- For the grip gesture, participants asked to hold the device

with the dominant hand and activated the vibration motor by

clicking a button with the other hand.

- Every press event took 1.5 seconds and repeated ten times.

- We discarded the first 500ms to allow for pressure

stabilization.

Touch

Grip

Pilot Test : Result

Ranges of average minimum and maximum pressure according to hand-gripping type.

Usability Test

The graphical interface used for testing pressure input with 2

levels (L2), 4 levels (L4) and 6 levels (L6).

Continuous visual feedback + Dwell

Usability Test

- 12 volunteers (6 women) with ages between 24 and 33 (average 27.5, SD 2.5).

- Participants were asked to familiarize themselves with the device for a maximum of five

minutes.

- We asked users to perform 36 successful targeting trials (the first 12 discarded as

practice) for each pressure level (L2, L4, L6) with two hand gestures (press and

squeeze).

- NATA TLX questionnaire for each pressure level.

- Every target level was evenly represented and balanced, collecting data for a total of

288 trials per hand gesture type.

- We recorded measures of performance (e.g., the selection time of successful trials,

number of wrong selections, and the variations of the pressure within the targets).

- The experiment took approximately one hour.

Usability Test : Result

Metrics for selection times and error rate (low is better).

(F(2,11)=126.6, p<0.01) *

* (F(1,11)=6.3, p<0.05)

Overall error rates for levels L2, L4, and L6 are, respectively, 0.3%, 7.6%

and 12.5% for the press gesture, and 0.7%, 11.5%, and 33% for the

squeeze posture.

(F(2,11)=25.1, p<0.01) *

* (F(1,11)=5.1, p<0.05)

Usability Test : Result

Users’ cognitive workload with a TLX.

The error rate is also reflected in the results from the TLX (cognitive

load is directly proportional to the number of pressure levels).

(F(2,11)=84.5, p<0.01) *

Usability Test : Result

% of error distribution among the selection levels for L6 (low is better).

The distribution of errors was uneven and most errors

happened in the central targets (especially in the box5)

rather than at the extremes.

Box 6

Box 1 268 (MAX PRESSURE)

0 (MIN PRESSURE)

coarse

smooth

Usability Test : Result

- Most participants expressed that VibPress is enjoyment and quick adoption.

- However, some reported that “continuous vibrations are somehow disturbing.”

- Overall, participants agreed that for L2 and L4, the performance and the usability level were satisfactory

- Despite the accuracy level, users preferred the squeeze gesture to the press one, as they felt it took less physical effort.

Interview

Usability Test : Result

VibPress shows …

- Selection times similar to those obtained with specialized

hardware (Cechanowicz et al., 2007; Ramos et al., 2004).

- But faster (Goel et al., 2012).

- With fewer errors than software pressure techniques (Goel et al.,

2012; Heo et al., 2011; Hwang et al., 2012).

When comparing these results with those from previous

work for analogous dwell-based targeting tasks,

VIDEO VibPress

CONCLUSION VibPress

LIMITATION

Compatibility across devices

(different motors or device cases propagate vibrations

differently).

Battery consumption due to the continuous activation

of the vibration motor. (However, these issues could be

alleviated by using frequent but short vibration patterns

at lower intensities).

CONCLUSION

- We showed evidence that, with continuous visual feedback,

users can reliably and quickly input with accuracy ranging from

99.7% to 67% and from ~1 to ~4 seconds interaction time,

depending on the different pressure levels and hand gripping

used.

- We also described how different users favorably judged the

usability and potential applications of the system.

- Future work will aim to identify hand postures and specific

pressure for selective portions of the screen. Finally, we also

plan to measure pressure inputs using tangible objects (e.g.,

styli, tokens) rather than fingers.

END

Contact : www.UXInventor.com

Demo Here !

Thank you so much !