new interaction techniques department of computer sciences university of tampere, finland september...

New Interaction Techniques

Department of Computer SciencesUniversity of Tampere, Finland

Department of Computer SciencesUniversity of Tampere, Finland

September – December, 2003

Grigori Evreinov

www.cs.uta.fi/~grse/

Dwell Time Selection & more…

TAUCHI MMIG G. Evreinov 01_26 10.10.2003

Cursor Object Inter-actions

Dwell Time

Time

X

Y

Time

demo1

[1]


MagicCursor 2000: Dwell Selection Clicking Solution, http://www.gstsdesigns.com/madentec/MagicCursor.htm

Gus! Dwell Cursor, http://www.gusinc.com/dwell.html

The Smart-Nav AT package (Dwell clicking 0.2-2.0 s), http://www.naturalpoint.com/prod/d_dwell.htm

Dwell Time

http://www.gusinc.com/dwell.html

http://www.naturalpoint.com/prod/d_dwell.htm


Gaze Control

[1] http://www.metrovision.fr/mv-vi-notice-us.html[2] http://www.delta7.asso.fr/Deltavision%202001/ecrire3.html

Dwell Time

http://www.metrovision.fr/mv-vi-notice-us.html







http://www.delta7.asso.fr/Deltavision%202001/ecrire3.html








[31] http://www.metrovision.fr/mv-vi-notice-us.html

LC Technologies, Inc.

The Eye-Gaze System Makers

Dwell Time



Dwell Time

[9] Myers, B.A., et al. Interacting at a Distance: Measuring the Performance of Laser Pointers and Other Devices. CHI 2002.

[12] Olsen Jr., D.R. and Nielsen, T. Laser Pointer Interaction. CHI 2001.

Laser Control


Dwell Time

Natural Point trackIRTM

http://www.naturalpoint.com ,

http://www.headtracker.com/

Marsden, R. 100% Hands-free Computer Access- Madentec’s 2000 Serieshttp://www.madentec.com/products/comaccess/2000/whitepaper.html

http://www.csun.edu/cod/conf/2000/proceedings/0185Marsden.htm

demo2

demo3

Head Control


Usability-testing software for… txtEntry Eye-

Gaze

Dwell Time

Clear Data


TWords.txt /phrases

GridData1:test words/chars

entered text /charstime per char, ms

GridData2:

char per word /phrasenum. of entered words

num. of strokes /clicks per word /phrase

time per word/phrase, s

lblTestSymbol

txtText1

On-screen Keyboard

Break test

Test initialization

Ctrl+K=>move keys

SetSigns

SetCharacters

BackSp

Statistics()

Rtime, mss (st.dev), ms

Errorswpm

TestTime, s

lblSave_Click()

GridData2_Click()to save column

GridData1_Click() to save column

txtPersonData comments…

fraDataWordsLoading

SetTestlblOpen_Click()

SetData

Trial start

Timer1

Timer2

Lesson6 txtEntry_EyeGaze3

SpotActivate

lblDwDelay

lblDwell

Ctrl+L=>EditCapts.

Up/Down

Timer3 Timer4

Ctrl+W=>Dwell

MouseInput SymbDec

HideBorders

Dwell Time

Clear Data


lblTestSymbol

txtText1

On-screen Keyboard

Break test

Test initialization

Ctrl+K=>move keys

SetSigns

SetCharacters

fraData

SetData

Trial start

Lesson6 txtEntry_EyeGaze3

SpotActivate

lblDwDelay

lblDwell

Ctrl+L=>EditCapts.

Up/Down

tmrDwDelay tmrDwell

Ctrl+W=>Dwell

MouseInput SymbDec

HideBorders

Dwell Time


Dwell TimeAdaptive

Dwellingaccording to Windows Interface Design Guide*, the action associated with

the control is initiated when the mouse button is released

if the pointer is dragged to another location, the control will return to its

original state and undesirable click will be stopped

for instance, the user can put cursor over icon or command button and

choose it by clicking the mouse

sometimes, clicks are undesirable or impossible (laser pointer, gaze / head

/ ‘brain’ control…)

to provide productivity and easy access during communication with

computer it is necessary a balance between flexibility of the dialogue

structure, adaptability on each level of the interface, cognitive abilities of

the person and limitations used interaction techniques

semantic, syntactic, lexical* The Windows Interface. An Application Design Guide. Microsoft Press, Redmond, Washington, 1992.


Dwell Timethe time is one of the most important critical parameters of the system

feedback loop

a normal time of simple visual-motor control task in accomplishing with the

help of intact muscles of the finger is about 150-250 ms

the time is progressively increased up to 1000 ms and more, if an

additional semantic analysis or physical limitations take place

to prevent wrong selection external (dwell) timer, mental and motor

activities of the user should strictly be coordinated

the feedback cues (highlighted areas) could play a role of strobe-signals to

stimulate user behavior and to support, to stabilize temporal framework for

rhythmically-alternating cognitive and motor activities

if the system could individually and dynamically turn, dwell mode could be

more flexible


Dwell Timehow we could know what temporal window satisfies of user requirements

and when it should be changed?

menu pointing could be considered as temporal process or stimulus-

dependent goal-directed behavior of the user; herewith, a behavior model

includes a sequence of actions both on the user side – cognitive

processing and motor actions, and procedures are within interface –

highlighting or other verification about the state of a particular alternative

the physical parameters of feedback signals may either facilitate

synchronization of the interactive process or hinder performance

in both cases, physical stimulus starts the motor reaction that could be

measured

based on real-time analysis the visual-motor reaction time, we could

predict or optimize dwell interval*.

* Bourhis, G., Pino, P. Temporal analysis of the use of an augmentative communication device. AAATE’03. IOS Press, Netherlands (2003), 294-298.

Tthreshold

selectionhighlighting

the menu item


Dwell Time

the temporal diagram of the algorithm for measuring the user performance

through visual-motor reaction time and correction of the dwell interval

T0 – the first variable interval, T1 – the second variable interval and T2 = T1

Tthresh. - dynamical threshold Tthresh. = T0 + T1

Time

Tdwell

RTi

T0 T2T1

selectionselection


Dwell Time

after each highlighting the menu item, we can record a time of user reply in

a kind of selection time

besides that, we can count an average magnitude for AveRT on each 5

realizations

now, if (AveRT < Tthresh. – dT) then we may decrease T0 on dT too

that is, a new scan interval will equal

Tdwell = (T0 + 2 T1) – dT or Tdwell = (T0 – dT) + 2 T1

demo4


Dwell Timechanging scan interval can occurred with equal probability in plus and

minus, if user reactions are symmetrical regarding dynamical threshold

Tthresh.

if user changed typing speed, each user replies

AveRT < Tthresh. and Tdwell

are permanently decreased due to decreasing T0, therefore the time to

change Tthresh.

in a kind of criterion could be index of some number of dT

for instance, if T0 have changed on ((-3) dT), the magnitude of Tthresh.

may also be decreased, at least, on one half of this magnitude

then, a new scan interval will equal

Tdwell = (T0 – 3 dT) + 2 (T1 – 3 dT/2)

after changing dynamical threshold, probability of user replies in a field T2

will be higher and AveRT > Tthresh.

this situation will lead, or not, to increasing T0


Dwell Time

Usability-testing software for… Adaptive

dwelling


Dwell Time

Shape1 Shape2

Shape9 Shape10

MenuSave

Open

ShapePos

Data

Timer1

lblGrid1

fraData

Timer2

lblDwell

lblThresh

lblDelta

lblWait

txtTest comments…

chkColor

chkBorder

chkSound

chkTactile

lblNAve

chkAve

chkSAve

lblIndexC

chkCThresh

T0 changing T1 changing

lblGrid2, 3 Clear

lblDrawGraphlblError chkCursor chkSpots

picGraph1

picGraph2


Dwell TimebPos = False

kThresh = CInt(t2 - t1)

t1 = GetTickCountTimer1.Enabled = True

Timer1.Interval = tDwell

Yes

No

Yes

Exit

Private Sub Form_MouseMove(…)

Xtmp1 = CInt(X) Ytmp1 = CInt(Y)

cursor is inside Shape1

bShape1 = True

NoYes

bThresh = True

Yest2 = GetTickCount

t1 > 0

Yes

reply's time or (T0 + T1)

Shape1.BorderColor = &H80&

Change Border Color =1

Yes

bShape1 = True

bShape2 = True

bShape2 = False reset BorderColor

reset FillColor

Yes

Timer1.Enabled = 0, Int. = 0Timer2.Enabled = 0, Int. = 0ImmWebControl1.StopEffect

bShape1 = Falsereset BorderColor, reset FillColor

If bThresh = True ThenChangeDwell

If bThresh = True ThenChangeDwell

chkSAve.Value = 0 And chkAve.Value = 0


Dwell Time

Private Sub ChangeDwell()

kThresh > 100 or tDwell/2

Yes

No

ChangeDwell

ArrThresh(nClick) = tThresh ArrCThresh(nClick) = tThresh ArrScan(nClick) = tDwell t1 = 0: t2 = 0

ArrThresh(nClick) = kThresh

nClick < (nAve + 1)

Yes

No

ArrDTmp(nClick) = kThresh

aveThresh = tThresh

Shift of array and count of the sliding aveThresh

aveThresh < min lower limit = tDelta + 20

Yes

NoaveThresh = min

aveThresh < tThresh - tDelta

aveThresh > tThresh + tDelta

chkSAve.Value = 1

chkAve.Value = 1

aveThresh = tThresh

Line1.BorderColor = &HFF00FF

Line1.Y2 = picGraph1.ScaleHeight - tThresh Line1.Y1 = Line1.Y2


Dwell Time

aveThresh < tThresh - tDelta aveThresh > tThresh + tDelta

tDwell = tDwell - tDelta

sDelta = sDelta - tDelta

ArrScan(nClick) = tScan

tDwell = tDwell + tDelta

sDelta = sDelta + tDelta

sDelta > tIndex tDelta sDelta < (-1) tIndex tDelta

new current value

tThresh = tThresh + CInt(sDelta / 2)

sDelta = 0


sDelta = 0

tThresh < 50 tThresh = 50

lower limit

ArrCThresh(nClick) = tThresh new current value

Private Sub ChangeDwell()

tDwell < 50

tDwell = 50lower limit


Dwell Time

after some kind of calibration the proposed algorithm will automatically

keep dwell interval near convenient magnitude with given speed or

increment dT

adaptive temporal interval could be useful for many applications, which

require of the periodic time correction in dependence on user performance,

for instance, in the systems (head / eye / finger tracking) using dwell time

to simulate mouse actions etc.


Dwell Time

T-Adaptive Unit

form transfers buttons' events to the TextBox therefore you can use KeyPreview or

directly txtControl TextBox


Dwell Time

Private Sub ChangeScan()

rTime > 100 or tScan/2

Yes

No

ChangeScan

ArrThresh(nClick) = tThresh ArrCThresh(nClick) = tThresh ArrScan(nClick) = tScan t1 = 0: t2 = 0

ArrThresh(nClick) = rTime - tScan

nClick < (nAve + 1)

Yes

No

ArrDTmp(nClick) = rTime - tScan

aveThresh = tThresh

Shift of array and count of the sliding aveThresh

aveThresh < min lower limit

Yes

NoaveThresh = min

aveThresh < tThresh - tDelta

aveThresh > tThresh + tDelta

T0

TAUCHI MMIG G. Evreinov p 24_26 10.10.2003

Dwell Time

aveThresh < tThresh - tDelta aveThresh > tThresh + tDelta

tScan = tScan - tDelta

sDelta = sDelta - tDelta

ArrScan(nClick) = tScan

tScan = tScan + tDelta

sDelta = sDelta + tDelta

sDelta > tIndex tDelta sDelta < (-1) tIndex tDelta

new current value


sDelta = 0


sDelta = 0

tThresh < 50 tThresh = 50

lower limit

ArrCThresh(nClick) = tThresh new current value

Private Sub ChangeScan()

tScan < 100

tScan = 100lower limit


References[1] Accot, J., Zhai, Sh. More than dotting the i’s — Foundations for crossing-based interfaces, CHI 2002,

April 20-25, 2002, Minneapolis, Minnesota, USA. / AccotZhai2002.pdf

[2] Zhai, Sh., Morimoto, C., Ihde, S. Manual And Gaze Input Cascaded (MAGIC) Pointing. In Proc. CHI’99:

ACM Conference on Human Factors in Computing Systems. 246-253, Pittsburgh, 15-20 May1999. /

magic.pdf

[3] Jacob, R.J.K. Eye Movement-Based Interaction Techniques and the Elements of Next-Generation,

Non-WIMP User Interfaces, http://www.roetting.de/eyes-tea/history/020919/jacob.html

[4] Jacob, R.J.K. What You Look At Is What You Get: Eye Movement-based Interaction Techniques.

CHI’90. / EyeMovBasedInteraction.pdf

[5] Jacob, R.J.K. Eye Movement-based Human-Computer Interaction Techniques: Toward Non-Command

Interfaces. / EyeMovementBased.pdf

[6] Shell, J.S., Vertegaal, R., Skaburskis, A.W. EyePliances: Attention-Seeking Devices that Respond to

Visual Attention. CHI2003 / EyePliances.pdf

[7] Hyrskykari, A. Gaze Control as an Input Device. / gazecontrol.pdf

[8] Aoki, H., Itoh, K., Sumitomo, N. and Hansen, J.P. Usability of Gaze Interaction Compared to Mouse

and Head-Tracking in Typing Japanese Texts on a Restricted On-Screen Keyboard for Disabled

People. / GazeInteraction_iea2003-aoki.pdf

[9] Myers, B.A., Bhatnagar, R., Nichols, J., Choon Hong Peck, Kong, D., Miller, R. and Long, A.Chr.

Interacting at a Distance: Measuring the Performance of Laser Pointers and Other Devices. CHI 2002,

April 20-25, 2002, Minneapolis, Minnesota, USA. / InteractingAtDistance.pdf

[10] Cheng, K., Pulo, K. Direct Interaction with Large-Scale Display Systems using Infrared Laser Tracking

Devices. Australasian Symposium on Information Visualisation, Adelaide, 2003. /

DirectInteractionLargeScaleDisplays.pdf

Dwell Time

http://www.roetting.de/eyes-tea/history/020919/jacob.html


[11] Hwang, F. Partitioning Cursor Movements in “Point and Click” Tasks. ACM 1-58113-630-7/03/0004.

CHI 2003, April 5-10, 2003, Ft. Lauderdale, Florida, USA. / PartitioningCursor Movements.pdf

[12] Olsen Jr., D.R. and Nielsen, T. Laser Pointer Interaction. CHI2001, SIGCHI’01, March 31-April 4,

2001, Seattle, WA, USA. / LaserPointerInteraction.pdf

[13] Steriadis, C.E. and Constantinou, Ph. Designing Human-Computer Interfaces for Quadriplegic People.

ACM Transactions on Computer-Human Interaction, Vol. 10, No. 2, June 2003. /

Interafce_QuadriplegicPeople.pdf

[14] Lankford, C. Effective Eye-gaze Input Into Windows tm. Eye Tracking Research & Applications

Symposium 2000 Palm Beach Gardens, FL, USA. / EffectiveEyeGazeInput.pdf

[15] Bates, R. Have Patience with Your Eye Mouse! Eye-Gaze Interaction with Computers Can Work. pp.

33-37. / EyeMouseGazeInteraction.pdf

[16] Ware, C., Mikaelian, H.H. An Evaluation of an Eye Tracker as a Device for Computer Input. CHI + GI

1987 / EvaluationEyeTracker_Input.pdf

[17] Hinckley, K., Pausch, R., Goblel, J.C. and Kassell, N.F. A Survey of Design Issues in Spatial Input. /

Spatial_InputSurvey.pdf

[18] Edwards, G. A Tool for Creating Eye-aware Applications that Adapt to Changes in User Behavior. /

EyeAwareApplications_Adapt.txt

[19] Youngblut, C., Johnson, R.E. et al. Review of Virtual Environment Interface Technology, Institute for

Defence Analyses, available at http://www.hitl.washington.edu/scivw/scivw-ftp/publications/IDA-pdf/

Dwell Time

new interaction techniques department of computer sciences university of tampere, finland september...

Documents

dwell time selection

dwell selection

dwell cursor

package dwell

dwell time adaptive

series http

time time x y demo1

word phrase time