training wheels in an user interface

8/7/2019 Training wheels in an user interface.

http://slidepdf.com/reader/full/training-wheels-in-an-user-interface 1/7

RESEARCH CONTRIBUTIONS

H u m a n A s p e c t s o f

C o m p u t i n g

He nry F . Ledgard

E d i tor

T r a in i n g W h e e l s in a

U s e r I n t e r f a c e

JOHN M . CARRO LL and CARO LINE CARRITHERS

A B S T R A C T : N e w u se rs o f h i g h - fu n c t io n a p p l i ca t io n

sys tems can become f rus tra ted and con fused by the e rrors

they m ake in the ear ly s tages o f learn ing . A t ra in ing

in ter face fo r a commerc ia l word processor was des igned to

make typ ica l and t roub lesome error s ta te s "unreachab le , "

t h u s e l i m i n a t i n g t h e s o ur c es o f so m e n e w - u s e r l e a r n in g

prob lems . Crea t ing a t ra in ing env ironmen t f rom the bas ic

func t io n o f the sys tem i t se l f a f fo rded subs ta n t ia l ly fa s te r

l e a r n in g c o u p l e d w i t h b e t t e r l ea r n i n g a c h i e v e m e n t a n d

be t te r per formance on a comp rehens ion pos t - te s t . A con tro l

group spen t a lmos t a quar ter o f the ir t ime recover ing f rom

the error s tates th at the train ing interface blocked off . We

specu la te on how th is t ra in ing s t ra tegy mig h t be re fined, 1 .

and more genera l ly , on how func t ion shou ld be organ ized in

a user interface.

E m p i r i c a l c h a r a c t e r i z a t i o n s o f c o m p u t e r n o v i c e s l ea r n -

i n g t o u s e a p p l i c a ti o n s y s t e m s p a i n t a d r e a r y p i c t u r e o f

s i d e t r a c k s a n d e r r o r t a n g l es f r o m w h i c h r e c o v e r y , o r

even diagnos i s , i s d i f f i cul t ( e .g . , [ 6] ) . In th i s pape r , we

s h o w t h a t a " t ra i n i n g w h e e l s " i n t e r f a c e - - d e s i g n e d t o

b l o c k t y p i c a l s i d e t r a c k s a n d e r r o r s t a t e s - - c a n f a c i l i t a t e

t h e l e a r n i n g p r o c e s s f o r n e w u s e r s .

T H E T R A I N I N G W H E E L S I N T E R F A C E

W e s t u d i e d a s t an d - a l o n e, c o m m e r c i a l w o r d p r o c e s s i n g

s y s t e m a n d o b s e r v e d n e w u s e r s , p e o p l e w h o h a d n e v e r

u s e d a c o m p u t e r b e f o r e , t r y i n g t o l e a r n t h e s y s t e m ' s 2 .

T h i s p a p e r w a s p r e s e n t e d a t t h e 2 4 t h A n n u a l M e e t i n g o f t h e P s y c h o n o m i cSoc ie ty . on N ove m be r 1 7 , 1 9 8 3 . in Sa n D ie go . C a l i fo rn ia .

© 1 9 8 4 A C M O O 0 1 - 0 7 8 2 / 8 4 / 0 8 0 0 - 0 8 0 0 7 5¢

b a s ic f u n c ti o n . F r o m t hi s, w e d e v e l o p e d a n i n v e n t o r y o f

t y p i c a l n e w - u s e r e r r o r s . T h e s y s t e m ' s i n t e r f a c e w a s

t he n m o d i f i e d t o b l oc k t he s e e r r o r s t a te s , t ha t i s , t o

m a k e t h e m u n r e a c h a b l e . T h i s t ra i n i n g w h e e l s i n t e r fa c e

a f f o r d e d t h e b a s i c f u n c t i o n f o r d o c u m e n t c r e a t i o n , r e v i -

s i o n, a n d p r in t i n g , b u t n o a d v a n c e d w o r d p r o c e s s i n g

f un c t i on ( e. g. , t a b l e f o r m a t t i ng ) . A va r i e t y o f s pe c i f i c

e r r o rs w e r e i d e n t i fi e d a n d b l o c k e d . T h e m o s t i m p o r t a n t

( in t e rm s o f t h e s e v e r i t y o f th e i r c o n s e q u e n c e s ) w e r e

t h e s e s e v e n ( t h e e r r o r s a r e n u m b e r e d a n d l a b e l e d f o r

l a t e r r e f e r e nc e ) :

T h e E x o t i c M e n u C h o i ce E r r or . N e w u s e r s o f te n

r e c k l e s s ly t r i e d o u t m e n u c h o i c e s i n t h e i r e a r l y

e n c o u n t e r s w i t h t h e s y s t e m . T h e s e u s e r s t y p i c a l l y

b e c a m e t a n g le d i n a d v a n c e d m e n u s ( fo r e x a m p l e ,

t h o s e p e r t a i n i n g t o s y s t e m a n d d i s k e t t e m a i n t e -

n a n c e ) a n d w e r e u n a b l e t o g et b a c k t o t h e s i m p l e

f u n c t i o n t h e y h a d o r i g i n a l l y s e t o u t t o l e a r n . T h i s

e r r o r w a s e l im i n a t e d b y m a k i n g o n l y b a s ic m e n u

c h o i c e s a c c e s s i b l e i n t h e t r a i n i n g w h e e l s i n t e r f a c e .

W h e n i n a p p r o p r i a t e , e x o t i c c h o i c e s w e r e s e l e c t e d ,

t he m e ss ag e " x x x n o t a v a i l a b l e o n t h eT r a i n i n g S y s t e m " w a s d is p la y ed ( w h er e " x x x "

s t a n d s ' f o r t h e n a m e o f t h e s e l e c t e d f u n c t i o n ) , a n d

t h e a d v a n c e d f u n c t i o n w a s n o t e n g a g e d .

T h e P r i n t F i r s t E r r or . N e w u s e r s w a n t t o u n d e r -

t a k e r e al w o r k i m m e d i a t e l y . W e o b s e r v e d n o v i c e s

w h o r e q u e s t e d a p r i n t j o b b e f o r e t h e y h a d e v e n

c r e a te d a n d s t o re d a d o c u m e n t . T h i s e r r o r w a s

800 Communica t ionso f the A CM A ugus t 1984 V o lume 27 Number 8



Research Contr ibu t ions

b l o c k e d b y m a k i n g p r i n t re q u e s ts d e p e n d e n t o n

i m m e d i a t e l y p r e c e d i n g c r e a t e o r r e v i s e r e q u e s t s .

W h e n a p r i n t w a s s e l e c te d i n a p p r o p r i a t e l y , t h e

m es s ag e " P r i n t i s a v a i l a b l e o n l y a f t e r a

C r e a t e o r a R e v i s e o n t h e T r a i n i n g S y s -

t e m " w a s d i sp l ay e d.

3 . T h e P a r a m e t e r L o o p E r ro r . S e v e r a l o f t h e w o r d

p r o c e s s o r ' s m e n u s i n v o l v e a s er i e s o f p a r a m e t e r s .

U s e r s h a v e t h e o p t i o n o f a c c e p t i n g d e f a u l t v a l u e s

b y m e r e l y p r e s s i n g E N T E R o r o f i n t e r a c t i n g w i t h

t h e m e n u t o s p e c i f y o r r e s p e c if y p a r a m e t e r v a l u e s

( fo r e x a m p l e , t h e d e f a u l t o f s i n g le s p a c i n g c a n b e

r e s p e c if i e d t o d o u b l e s p a ci n g ). W e o b s e r v e d t h a t

n e w u s e r s f a il e d t o a v a i l t h e m s e l v e s o f t h e d e f a u l t

o p t i o n . T h e y q u i t e t y p i c a l l y g o t i n t o a l o o p o f s p e c -

i f ic a t io n a n d r e s p e c i fi c a t io n o f p a r a m e t e r s ( p e r-

h a p s b e c a u s e t h e s c r e e n p r o m p t i n v i t i n g t h i s a c -

t i o n w a s m o r e v i s u a l l y s a l ie n t t h a n w a s t h e

p r o m p t i n v i t i n g t h e d e f a u l t s ) . T h e s e p a r a m e t e r

l o o p s c a n b e l e n g t h y a n d f r u s tr a t in g , a n d t h e y a r e

t o t a ll y u n n e c e s s a r y . F o r t h e n o v i c e , t h e d e f a u l t s

a r e f in e . T h i s e r r o r s t a te w a s b l o c k e d b y d i s a b l in g

t h e m e n u p a r a m e t er s ; w h e n a n o v i c e s e l ec t e d a

p a r a m e t e r, t h e d i sa b l e m e n t m e s s a g e " x x x n o t

a v a i l a b l e o n t h e T r a i n i n g S y s t e m " w a s

displayed (wh ere "x xx " stands for the n a m e of the

s e l e c te d p a r a m e t e r ) .

4 . T h e A l t e r n a t e S h i f t E r r o r . W e o b s e r v e d a n e r r o r

i n v o l v i n g th e a l te r n a t io n o f t w o k e y s t r o k e c o m -

m a n d s . B o t h a r e l o c a t e d o n t h e s a m e p h y s i c a l k e y .

O n e c o m m a n d , w h i c h c a n c e l s m e n u s a n d s e l e c -

t io n s , r e q u i r e s a n a l t e r n a t e s h if t. T h e o t h e r , w h i c h

d i s p l ay s a s p e c ia l s y s t e m u t i l it y m e n u , d o e s n o t .

U n d e r s t a n d a b l y , i t i s t h e a l t e r n a t e k e y t h a t i s

m o r e i m p o r t a n t t o n o v i c e u s e r s ( w h o m u s t f r e -

q u e n t l y c a n c e l a n i n c o r r e c t m e n u o r s el e c ti o n ) .

H o w e v e r , t h e s e u s e r s o f t e n f a i l t o h o l d d o w n t h e

a l t e r n a t e s h i f t a n d e n d u p w i t h t h e u t i l i t y m e n u

o f f e r in g s e e m i n g l y i n s c r u t a b l e c h o i c e s . T h i s e r r o r

s t a t e w a s b l o c k e d b y d i s a b l i n g t h e n o n a l t e r n a t e d

c o m m a n d . P r e ss i ng t h e k e y w i t h o u t t h e a l te r n a t e

s h i f t e l i c i t e d t he m e s s a ge " T he U t i l i t y key i s

n o t a v a i l a b l e o n t h e T r a i n i n g S y s t e m . "

5 . T h e P r i n t Q u e u e E r r o r . T h e w o r d p r o c e s s o r w e

s t u d i e d a l l o w s m u l t i p l e p r i n t j o b s to b e q u e u e d .

N o v i c e s l a c k t h e c o n c e p t o f a " q u e u e " a n d o f t e n

i s s ue m u l t i p l e p r i n t r e que s t s . T h i s le a ds t o p r ob -

l e m s w h e n t h e u s e r l a t e r r e q u e s t s a n o t h e r p r i n t o f

a n o t h e r d o c u m e n t . T h e o u t p u t w i ll b e w h a t w a s

q u e u e d b e f o r e a n d n o t w h a t w a s j u s t r e q u e s t e d .T h e u s e r i s a l s o p r e v e n t e d f r o m r e v i s i n g d o c u -

m e n t s t h a t a r e q u e u e d t o p ri n t , h e n c e m u l t i p l e

p r i n t r e q u e s t s e f f e c t iv e l y m a k e a d o c u m e n t u n r e -

v i sa b l e. T h e e r r o r s t a t e w a s b l o c k e d b y l i m i t i n g

t h e s i ze o f t h e p r i n t q u e u e t o o n e d o c u m e n t .

T r y i n g to q u e u e a s e c o n d d o c u m e n t m e r e l y e li c-

i te d t h e m e s sa g e " O n l y o n e d o c u m e n t a t a

t i m e c a n b e p r i n t e d o n t h e T r a i n i n g S y s -

tern. "

6 . T h e D i s k e t t e N a m e E r r o r. T h e w o r d p r o c e s s o r

s t o r es d a t a o n d i s k e tt e s . I n t h e c o u r s e o f cr e a t in g ,

r e v i s i n g , a n d p r i n t i n g a d o c u m e n t , t h e u s e r i s

p r o m p t e d t o s p e c i f y a d i s k e t t e n a m e , a n d t h e n t o

i n s e r t t ha t d i s ke t t e . I f t he u s e r m i s s pe c i f i e s t he

d i s k e t te n a m e , t h e s y s t e m p r o m p t s f o r t h e ( p e r -

h a p s n o n e x i s t e n t ) d i s k e t t e. T h e r e i s a c o m m a n d t o

c a n c e l t h i s r e q u e s t , b u t ! t i s n o t a n e l e m e n t a r y

c o m m a n d . F o r n o v i c e s , t h i s e r r o r i s a d e a d e n d .T h i s e r r o r s t at e w a s e l i m i n a t e d b y a n t i c i p a t i n g t h e

n a m e o f t h e d i s k e t t e t h a t n o v i c e s u s e a n d b y p r o -

g r a m m i n g t h e s y s t e m t o a c c ep t o n l y t h a t n a m e .

H e n c e , m i s s p e c i f y i n g t h e d i s k e t t e n a m e e l i c i t e d

t h e m e s s a g e " D i s k e t t e X X X i s n o t a v a i l -

a b l e o n t h e T r a i n i n g S y s t e m . "

7 . T h e U n p r i n t a b l e C h a r a c t e r E r r o r . D i f f e r e n t

p r i n t e r c o n f i g u r a t i o n s f o r t h e w o r d p r o c e s s o r a r e

c o m p a t i b l e w i t h d i f f e r en t s u b s e t s o f it s p o s s ib l e

k e y b o a r d c h a r a c t e r s. I n p a r ti c u l a r , f o r th e p r i n t e r

w e u s e d , i t w a s p o s s i b l e t o e n t e r k e y b o a r d c h a r a c -

t e rs i n t o a t e x t fi le w h i c h c o u l d n o t b e p r i n t e d . I n

s u c h a c as e , t h e w o r d p r o c e s s o r p r o d u c e d a n e r r o r

m e s s ag e w h i c h p r o m p t e d t h e u s e r t o o v e rr i d e t h e

p r i n t e r, a n d h a v e a n u n d e r s c o r e p r i n t i n p l a c e o f

t h e u n p r i n t a b l e c h a r a c t e r , o r t o c a n c e l t h e p r i n t

j o b a lt o g e t h e r ( a n d t h e n , b y i m p l i c a t i o n , t a k e f u r -

t h e r a c t i o n , s u c h a s r e - e d i t in g t h e d o c u m e n t o r

c h a n g i n g t h e p r i n t e r) . T h i s e r r o r s t a t e w a s e l i m i -

n a t e d b y f o r c in g a c h o i c e o n t h e u s e r : U n p r i n t a b l e

c h a r a c te r s w e r e a l w a y s r e p l a ce d w i t h u n d e r -

s c o r e s .

T h e r e w e r e o f c o u r s e a v a r i e t y o f o t h e r p o s s i b le e r -

r o r s u s e r s c o u l d m a k e . T o c o n t r a s t t h e m t e r m i n o l o g i -

c a l l y w i t h t h e " b l o c k e d " e r r o r s a b o v e , w e w i l l r e fe r t o

t h e s e a s " n o n b l o c k e d " e r r o r s . W e w i l l c o m m e n t o n

s o m e o f th e s e l a te r .O u r h y p o t h e s i s w a s th a t b l o c k i n g th e c o n s e q u e n c e s

o f t h e s e s e v e n e r r o r s w o u l d e l i m i n a t e s o m e o f t h e f r u s -

t r a t i o n a n d c o n f u s i o n c a u s e d i n t h e e a r l y s t a g e s o f

l e a r n in g , a n d t h e r e b y f a c il i ta t e l e a r n i n g o f t h e s y s t e m

b y n o v i ce s . W e p e r f o r m e d a n e x p l o r a t o r y e x p e r i m e n t a l

i nve s t i g a t i on t o t e s t t h i s hypo t h e s i s .

M E T H O D A N D R E S U L T S

O u r e x p e r i m e n t a l a p p r o a c h w a s e s s e n t i a l l y o b s e r v a -

t i o n a l , r e f l e c t i n g o u r c o n c e r n w i t h t h e q u a l i t a t i v e d i f -

f e r e n c e s b e t w e e n t w o t r a i n i n g e x p e r i e n c e s . H o w e v e r ,

t h e o b s e r v e r i n o u r s t u d y d i d n o t i n t e r f e r e w i t h b e h a v -

i o r ( e. g., b y q u e s t i o n i n g o r p r o m p t i n g ) , a n d h e n c e w e

w i l l r e p o r t s h e e r p e r f o r m a n c e d i f f e r e n c e s a s w e l l .I n t h e e x p e r i m e n t , 1 2 o f f ic e t e m p o r a r i e s l e a r n e d t o

u s e a c o m m e r c i a l w o r d p r o c e s si n g s y s te m . T h e y w e r e

o b t a i n e d f r o m a n a g e n c y a n d s e le c t ed t o h a v e n o p r i o r

w o r d - p r o c e s s i n g e x p e r i e n c e . A t t h e s t a r t , t h e y w e r e

e a c h g i v e n a b r i e f s e l f - s tu d y m a n u a l a n d a h a n d w r i t t e n

l e tt e r. T h e y w e r e t o l d to i m a g i n e t h a t t h e y h a d b e e n

h i r e d t o r e p l a c e a ty p i s t f o r o n e w e e k , w e r e t o l e a r n t h e

e s s e n ti a ls o f t h e w o r d p r o c e s s i n g s y s t e m , a n d t h e n t y p e

u p t h e l e t t e r - - a l l t h i s as q u i c k l y a s p o ss i b le . T h e y w e r e

A u g u s t 1 9 8 4 V o l u m e 2 7 N u m b e r 8 C o m m u n i c a ti o n s o f t h e A C M 8 01



R e s e a r c h C o n t r ib u t i o n s

a s k e d t o f u r t h e r i m a g i n e t h a t t h e o n l y p e r s o n w h o

k n e w a b o u t t h e w o r d p r o c e s s o r w a s t h e o n e t h e y w e r e

r e p l a c i n g - - h e n c e , t h e y w o u l d b e t o t al ly o n t h e ir o w n .

T h e o b s e r v e r o f f e r e d e n c o u r a g e m e n t w h e n e v e r i t

s e e m e d a p p r o p r i a t e, b u t t h e o n l y " h e lp " p r o v i d e d w a s

t h e s u g g e s t io n t o r e r e a d t h e m a n u a l .

S i x o f t h e t e m p o r a r i e s w e r e a r b i t r a r i l y a s s i g n e d t o

t h e t r a i n i n g w h e e l s i n t e r f a ce a n d s i x t o t h e c o m p l e t e

s y s t e m i n t e rf a c e . T h e m a n u a l t h e y u s e d w a s s p e c ia l l yde s i gn e d t o be b r i e f. ( I n 1 2 pa ge s , it i n t r od uc e d w o r d

p r o c e s s i n g a n d t a u g h t s y s t e m i n i t ia l i z at i o n , d o c u m e n t

c r e a t i o n , a n d p r i n t i n g . ) T h e t r a i n i n g w h e e l s v e r s i o n o f

t h e m a n u a l d i f f e r e d f r o m t h e c o m p l e t e s y s t e m v e r s i o n

o n l y w h e r e i t h a d t o b e f a c t u a l l y a c c u r a t e i n o r d e r t o

i n f o r m t h e u s e r o f t h e d i s a b l e m e n t m e s s a g e s.

T h e o b s e r v e r s a t w i t h t h e p a r t i c i p a n t d u r i n g t h e e n -

t ir e s e ss i o n , t a k i n g d e t a i l ed n o t e s , i n c l u d i n g t i m i n g s o n

a c t i o n s a n d o u t c o m e s . P a r t i c u l a r a t t e n t i o n w a s g i v e n t o

e r r o r s (l ik e 1 - 7 a b o v e ) - - b o t h t o t h e i r f r e q u e n c y a n d t o

t h e a m o u n t o f t im e r e q u i r e d t o r e c o v e r f r o m t h e m .

( " E r ro r " w a s d e f i n e d a s a d e p a r t u r e f r o m t h e c r e a t e a n d

p r i n t a c t i o n p a t h ; " r e c o v e r y " w a s d e f i n e d a s a s u b s e -

q u e n t r e t u r n t o t h a t p a th ) .

Performance Dif ferences

O v e r a l l t i m e a n d s u c c e s s d a t a i n d i c a t e t h a t t h e t r a i n i n g

w h e e l s i n t e r f a c e p r o v i d e d a b e t t e r l e a r n i n g e n v i r o n -

m e n t f o r o u r p a r t i c i p a n ts . T r a i n i n g w h e e l s u s e r s t o o k

a n a v e r a g e o f 9 2 m i n u t e s t o c o m p l e t e t h e l e t t e r - ty p i n g

t a sk ; c o m p l e t e s y s t e m u s e r s t o o k 1 1 6 m i n u t e s . T h u s ,

t h e t r a i n i n g w h e e l s p a r t i c i p a n t s w e r e 2 1 p e r c e n t f a s t e r

ov e ra l l , t ( 10) = 2 .11 , p < 0 .05 . (See [5] for de sc r ip t ion of

t s t at is ti c s. ) P a r t i c ip a n t s w h o f a i l e d t o ty p e a n d / o r p r i n t

o u t w e r e a r b i t r a r il y a s s i g n e d a ti m e o f 1 2 0 m i n u t e s . A s

w e w i ll s e e, m o s t o f t h e s e f a i l u r e s w e r e i n t h e c o m p l e t e

s y s t e m c o n d i t i o n , h e n c e t h e t i m e a n a l y s is m a y b e

s o m e w h a t c o n s e r v a ti v e . T h e p e r f o r m a n c e , e r r o r a n dp o s t - te s t m e a s u r e s w e t o o k a r e s u m m a r i z e d i n T a b l e I .

O v e r a l l s u c c e s s f o l l o w e d t h e s a m e p a t t e r n . F o u r o f

t h e s i x p a r t i c i p a n t s u s i n g t h e t r a i n i n g w h e e l s i n t e r f a c e

s u c c e e d e d i n p r i n t i n g o u t t h e l e t t e r w e g a v e t h e m

W i t h i n t h e a l lo t t e d t i m e . ( M e a n t i m e t o c o m p l e t e t h e

t a s k f o r t h e s e f o u r p e o p l e w a s 7 8 m i n u t e s . ) T h e o t h e r

t w o t y p e d t h e l e t t e r i n t o t h e s y s t e m b y t h e e n d o f t h e

t w o h o u r s b u t d i d n o t p r i n t . I n c o n t r a s t , o n l y t w o o f t h e

s i x p a r t i c i p a n t s u s i n g t h e c o m p l e t e s y s t e m s u c c e e d e d i n

p r i n t i n g t h e i r l e t t er s . ( M e a n t i m e f o r th e s e t w o w a s 1 0 7

m i n u t e s , 3 7 p e r c e n t l o n g e r t h a n t h e f o u r tr a i n i n g

w h e e l s i n t e rf a c e p a r t ic i p a n t s w h o p r i n t e d .) T w o o t h e r

p a r t i c i p a n t s h a d o n l y t y p e d t h e l e t t e r a f t e r t w o h o u r s :

t w o h a d n e i t h e r t y p e d n o r p r i n t e d t h e i r l e tt e rs .W e p e r f o r m e d W i l c o x o n ' s r a n k e d s u m t es t [ 5] o n t h e

o v e r a l l s u c c es s d a t a , g r o u p i n g t h e p a r t i c ip a n t s i n t o

t h r e e c l a s s e s - - t h o s e t h a t p r i n t e d , t h o s e t h a t o n l y ty p e d ,

a n d t h o s e t h a t d i d n e i t h e r . T h e s i g n i f i c a n c e l e v e l f o r

t h i s c on t r a s t w a s on l y m a r g i na l , p < 0 . 1 . I f w e a l s o

i n c l u d e p e r f o r m a n c e t i m e i n f o r m a t i o n i n t h i s r a n k i n g ,

t h a t i s, if w e r a n k b y o v e r a l l t i m e w i t h i n t h e t h r e e

s u c c e s s c l as s es , t h e W i l c o x o n r a n k e d s u m b e c o m e s s ta -

t i s t i c a l ly s igni f i cant , p < 0 .05 .

TABLE I. Performance, Error and Post - test Summ ary

Training Complete

Wheels System

O v e r a l l M e a s u r e s

T o t a l t i m e o n t a s k 9 2 m i n . 1 1 6 m i n .

N u m b e r o f s u b j e c ts w h o 4 2

p r i n t e d

N u m b e r o f s u b j e c t s w h o o n l y 2 2

typed

N u m b e r o f s u b j e c t s w h o failed 0 2

I n it ia l t i m e t o g e t t o t h e t y p i n g 2 8 m i n . 5 3 . 6 m i n . , n = 5

area

T o t a l t i m e s p e n t i n t h e t y p i n g 1 4 . 7 r a i n . 1 0 . 2 m i n .

area

T o t a l t i m e s p e n t o n e r r o r r e - 4 5 . 5 m i n . 6 5 min .

c o v e r y

T o t a l r e c o v e r y t i m e f o r 5 . 7 m i n . 2 5 . 2 m i n .

b l o c k e d e r r o rs

L e a r n in g v e r s u s L e t te r - T y p i n g P h a s e :

T o t a l t i m e s p e n t i n l e a r n i n g 7 5 m i n . 8 8 . 5 r a in .

p h a s e

T o t a l t i m e s p e n t i n l e t t e r - t y p - 1 6 . 8 r a i n . 4 0 . 5 r a i n ., n = 4

i n g p h a s e

I n i t ia l t im e t o t y p i n g a r e a i n l e t - 3 . 5 r a i n . 10 . 0 r a i n . , n = 3

te r ph ase

E r r o r r e c o v e r y t i m e i n l e a r n i n g 3 9 r a i n . 4 7 . 8 r a in .

ph ase

E r r o r r e c o v e r y t i m e i n l e t t e r - 6 . 5 r a i n . 2 5 . 8 r a i n ., n = 4

t y p i n g p h a s e

P o s t - t e s t s

C o m p r e h e n s i o n s c o r e s 2 . 7 / 6 1 . 6 / 6

W o r k - A t t i t u d e s c o r e s 1 2 . 5 / 1 6 9 . 8 / 1 6

I n b r ie f , th e s e o v e r a l l q u a n t i t a t i v e m e a s u r e s i n d i c a t e

t h a t t h e t r a i n i n g w h e e l s i n t e r f a c e a f f o r d e d m o r e e f f i -

c i e n t l e a r n i n g p r o g r e s s. I n d e e d , a t h i r d o f t h e c o m p l e t e

s y s t e m p a r ti c ip a n t s w e r e u n a b l e t o p r o d u c e a n y d e m -o n s t r a t i o n o f p r a c t i c a l c o m p e t e n c e a f t e r t w o h o u r s o f

t r a i n i n g t i m e . T h e p e o p l e u s i n g t h e t r a i n i n g w h e e l s i n -

t e r fa c e w e r e a l l a b l e t o d o s o m e t h i n g , a n d t h e m a j o r i t y

o .f t he m w e r e a b l e t o c om pl e t e t he e n t i r e t a s k . I n pa s s -

i n g , i t i s a l s o w o r t h n o t i n g t h a t - - l o o k i n g a c r o s s b o t h

c o n d i t i o n s - - h a l f o f o u r p a r t i c ip a n t s d i d n o t s u c c e e d i n

p r i n t i n g a o n e - p a g e l e t t e r w i t h i n t w o h o u r s . T h i s r e -

m a i n s a m a j o r d e s i g n c h a l l e n g e i n w o r d p r o c e s s i n g .

B e y o n d t h e o v e r a l l d i ff e r e n c es , w e w a n t e d t o u n d e r -

s t a n d w h a t t h e t r a i n i n g w h e e l s l e a r n e r s w e r e d o i n g d if -

f e r e n t l y t h a n t h e c o m p l e t e s y s t e m l e a r n e r s . M o s t o f t h e

m o d i f i c a t i o n s i n t h e t r a i n i n g i n t e r f a c e a r e i n t e n d e d t o

h e l p t h e n e w u s e r g e t t h r o u g h t h e c o n t r o l s t r u c t u r e o f

t h e w o r d p r o c e s s o r a n d o n t o t h e m o r e c o n c r e t e t a s k s o fc r e a t i n g a n d p r i n ti n g . A c c o r d i n g l y , o n e f u r t h e r p e r -

f o r m a n c e m e a s u r e w e f o c u s e d o n w a s t h e t i m e e l a p s ed

w h e n t h e u s e r f i r s t e m e r g e d f r o m t h i s c o n t r o l s t r u c t u r e

a n d a c c e s s e d t h e t y p i n g d is p l a y . T h e m e a n t i m e t o

r e a c h t h e t y p i n g a r e a f o r t h e u s e r s i n t h e t r a i n i n g

w h e e l s c o n d i t i o n w a s 2 8 m i n u t e s ; f o r t h o s e u s i n g t h e

c o m p l e t e s y s t e m , t h e m e a n t i m e w a s 5 3 . 6 m i n u t e s . T h e

t r a i n i ng w he e l s l e a r n e r s w e r e 4 8 pe r c e n t f a s te r , t( 1 0 ) =

2.47, p < 0.05.

802 C o m m u n i c a ti o n s o f t h e A C M A u g u s t 1 9 84 V o l u m e 2 7 N u m b e r 8




Traversing the system's control structure to get to thetyping display was indeed an important aspect to focus

attention on. Across all 12 participants, only 12 percent

of the 2-hour experimental session was spent working

on the typing display. The fact that the training wheels

participants got through this control structure more

quickly may have left them relatively more time to

work on the typing display. Training wheels partici-

pants spent an average of 14.7 minutes, or 16.5 percentof their time, in the typing display; the complete system

participants spent 10.2 minutes, or 9 percent of their

time, t(10) = 2.97, p < 0.01.

How were the training wheels learners able to trav-

erse the system's control structure so much more rap-

idly? At least a part of the performance difference be-

tween the two groups can be directly ascribed to the

amount of time the complete system learners wasted

recovering from the particular errors (1-7 above) that

the training wheels learners wer e protected against. In-

deed, the complete systems learners spent almost 22

percent of their time on task (25.2 minutes) trying to

recover from these seven particular errors. The training

wheels learners were blocked from suffering the conse-quences of these errors and spent an average of less

than 6 minutes (7 percent of their overall time on task)

recovering from the blocked errors. (Generally, this was

time spent reading and interpreting the training inter-

face's disablement messages.) This difference in error

recovery times between the two groups was significant,

t(10) = 3.20, p < 0.005.

Again, we must keep this result in perspective. Both

groups spent a huge amount of time recovering from

errors. Learners using the training interface spent 50

percent of their time in error recovery as compared to

those using the complete system who spent 56 percent

of their time. (The training wheels learners were of

course recovering from errors other tha n 1-7 above, thenonblocked errors).

The summary figures for time spent on error-

recovery suggest a simple additive model differentiating

the two learning conditions. It turned out that learners

in both groups spent an average of about 40 minutes

recovering from nonblocked errors, but that the com-

plete system learners spent almost 20 minutes more, on

average, recovering from the seven errors blocked for

the training wheels learners. Recall now that the train-

ing wheels group spent about 24 minutes less on the

overall letter-typing task; it seems that one could sum-

marize the effect of the training interface as saving the

learner 20 minutes by blocking 20 minutes worth of

error-recovery time.We do not believe, however, that the training inter-

face merely saved people from making errors. There

are several sources of evidence for this. First, there is

evidence in the overall times that participants using the

training interface enjoyed most of their advantage

when they were actually performing with the system,

as opposed to learning it. We were able to divide the

participants' overall time into two phaseswtime spent

initially learning, and time spent typing and printing

the letter. We did not ask participants to organize their

time into two sequenced phases, but they all did so.

Training-wheels learners spent an average of 75 min-

utes in the learning phase and nearly 17 minutes in the

letter phase. Only four of the complete system learners

entered both phases; for these four an average of nearly

75 minutes was spent in the learning phase but over 40

minutes in the letter phase. Clearly, the time advantage

of the training wheels group was in the letter phaseof the experiment. This difference was significant,

t(8) = 2.90, p ~ 0.01.

In order to separate the fact that training wheels

learners spent less time on the experimental task over-

all from the contrast of time spent on the final letter

phase, we re computed the contrast using the proportion

of each participant's time spent on the final letter

phase. The difference is still reliable by Wilcoxon's

ranked sums test, p < 0.05. Hence, the training wheels

learners not only spent significantly less time on the

final letter phase of the task, they spent a smaller pro-

portion of their time on this phase of the task. These

differences were obtained despite the fact that both

groups spent comparable amounts of time in the initiallearning phase. This indicates that the training wheels

people were able to make better use of their learning

time.

But what were they learning? Other performance ev-

idence indicates that the training interface participants

learned to avoid errors. Considering both types of errors

(blacked and nonblocked), the training wheels learners

spent 52 percent of their time, or an average of 39

minutes, recovering from errors during the learning

phase of the experiment. The complete system people

spent 54 percent of their time, an average of 48 min-

utes, in error recovery during the learning phase. In the

letter phase, the training wheels group spent only 39

percent of their time, an average of less than 7 minuteson error recovery, while the complete system group (of

only four participants) spent 64 percent of their time, 26

minutes on average. Thus, the training interface signifi-

cantly reduced the proportion of time dedicated to er-

ror recovery between the learning phase and the letter

phase relative to the complete syst em group, t(8) = 2.1,

p < 0.05.

When we further divided these error recovery times

into blocked versus nonblocked errors, we found that

most (over 90 percent) of the recovery time decrease for

the training wheels group came from the nonblocked

errors. This is not entirely surprising; the training

wheels people spend so little time recovering from the

blocked errors that there was little room left for im-provement. However, their improvement in the non-

blocked errors suggested that blocking of even s o m e

learner errors can have a generalized benefit for learners.

Training wheels learners also seemed to become effi-

cient users more rapidly. In the letter-typing phase, the

training wheels people were able to traverse the system

control structure and get to the typing area in an aver-

age of 3.5 minutes. Complete system learners took 10.0

minutes or more than three times as long, t(8} = 3.69,

A u g u s t 1 9 84 V o l u m e 2 7 N u m b e r 8 C o m m u n i c a ti o n s o f t h e A C M 80 3




p < 0.005. Between the learning phase and the letter

phase, the training wheels learners r educed the amount

of time it took them to get to the typing area by a factor

of 8 (3.5 versus 28 minutes); the complete system

learners reduced their times by a factor of 5 (10.0 ver-

sus 53.6 minutes}. While both groups were improving,

the training wheels group improved more dramatically.

Finally, two post-tests we administered also indicated

that the training wheels manipulation had done morethan merely limit error recovery time. We administered

a brief comprehension test which asked participants in

both groups about diskettes and documents, and about

several of the function keys and display symbols. The

results are an indication that the group on the training

wheels did learn more than the group on the complete

system. Mean n umber correct for the group using the

training wheels was 2.7 out of 6; for the group using the

complete system the mean number correct was 1.6. The

training wheels learners did 41 percent better. This re-

sult is significant by Wilcoxon's ranked sum test, p <

0.05.

After the experiment proper, we also gave all partici-

pants a questionnaire designed to reveal their generalattitude toward work. The group using the training

wheels scored significantly higher than the group using

the complete system, t(10) = 2.32, p < 0.05. The partici-

pants using the training interface, because they were

more successful, may have felt better about themselves

and about work in general.

Ind iv id u a l D i f f erences

We observed vastly differing individual learning styles

among the participants in our study. At one extreme is

the reckless explorer who immediately begins to play

with the system, frequently with only a superficial

reading of the manual. This type of learner commitsmany errors and spends much of the time in error re-

covery, but will sometimes stumble on the correct solu-

tion by mistake. One of our partic ipants made 106 er-

rors during the two-hour test period.

At the other end of the contin uum is the plodder

who will not try anything until assured of the results.

This type of learner will read and reread the ma nual

and frequently sit and stare at the screen for periods of

time. These participants make fewer errors but may

have more trouble recovering from the effects of errors.

One of our participants made only 15 errors total in two

hours, but the fifteenth error was fatal and this person

was unable to accomplish anything more in the final 16

minute s of the expe rimen tal session. Neither of theselearning styles seems to be better-suited tha n the other

to mastering the system. The two learners in these ex-

amples were equally successful: both were able to type

in the letter, but neither was able to print.

One particular learner in the training wheels group

caught our attention. All by herself, she accounted for

nearly half of the errors committed by the learners in

this group. Indeed, looking at the nonblocked errors

made during the letter phase, this participant made 36

of the group's 43 errors. Not surprisingly, removing her

from the experiment strengthened most of the statisti-

cal results we have cited. Curiously, she is the reckless

explorer described above. (The plodder was one of our

complete system participants.} Unfortunately, at pres-

ent we have no independent basis for understanding

these individual differences, and their causal relations

to training methods an.d performance success.

Error Speci f icsExamining the particular errors and error recovery out-

comes can further illuminate how the training interface

worked. First though, we digress to comment on a few

of the principal "nonblocked" errors. Mechanical errors

involving the loading of program or data diskettes con-

sumed the most time among the nonblocked errors. A

typical error of this sort was attempting to mount a

diskette in an improper orientation. A second type of

nonblocked error-- with an assortment of variants- -

was naming errors, producing names with too many

characters, naming a document with the same name as

a menu choice, an ID letter, or a data diskette or using

a double name (ignoring a system-prompted na me a nd

suffixing a duplication of the name). Another non-blocked error was making an over ly literal response to

a system prompt, for example, typing the literal string

"ID let te r" when prompted by the system to "t yp e

ID le t t e r " (the system accepts only ID letters like a,

b, c, etc.).

Of the seven errors inventor ied in previous studies of

the word processing system, and blocked in the training

interface, four did not affect the performance of partici-

pants in either experimental condition. Only three par-

ticipants added more than one document to the print

queue and since they were all from the training wheels

group they received the message that this was not al-

lowed (Print Queue Error, #5). And only two partici-

pants typed characters that could not be printed (Un-printable Character Error, #7); both were in the train-

ing wheels condition and neither even noticed the un-

derscores that the system substituted. The fact that all

of these errors occurred in the training wheels group

seemed likely to be due to the fact that only two of the

complete system learners ever printed anything. Over-

all, the training wheels group printed nine documents

to three for the complete system group.

Two other blocked errors also were of little impor-

tance in participants' performance. No participant at-

tempted to print before creating or revising (Print First

Error, #2), and only three participants made an exotic

menu choice unrelated to their task, but were able to

recover quickly (Exotic Menu Choice Error, #1). Twoaccoun ts of this are obvious. First, we prepared our

own manual for this experiment--chiefly to control

"manual effects" between the two system conditions.

We designed the manual to be usable, based on what

we had seen in the course of our earlier work and

perhaps we simply succeeded with respect to Errors 1

and 2.

A second account appealed to our learner s' task ori-

entation. Both groups were explicitly instructed to type

out a particular letter. Thus for them, learning the sys-

804 C o m m u n i c a ti o n s o f t h e A C M A u g u s t 1 9 84 V o l u m e 2 7 N u m b e r 8



Research Con tr ibut ions

t e rn w a s a fu n c t i o n a l m e a n s o f a c c o m p l i s h i n g a s

q u i c k l y a s p o s s i b l e a r e a l t a s k t h e y c o u l d a l r e a d y a p p r e -

c i at e . L e a r n e r s i n o u r p r e v i o u s s t u d i e s , a n d i n d e e d t y p i -

c a l l y i n su c h s t u d i es , w e r e i n s t r u c t e d t o " l e a r n h o w t o

u s e t h e s y s t e m . " T h i s r e l a t i v e l y a b s t r a c t g o a l m a y m a k e

l e a r ne r s m or e s u s c e p t i b l e t o c e r t a i n e r r o r s . T he s e pos s i -

b i li ti e s a r e b e i n g p u r s u e d i n f o l l o w - u p r e s e a r c h .

T h e o t h e r t h r e e b l o c k e d e r r o r s in o u r i n v e n t o r y a c -

c o u n t f o r t h e m a j o r i t y o f t h e p e r f o r m a n c e d i f f e re n c e sb e t w e e n o u r t w o g r o u p s . F i v e o f t h e 1 2 p a r t ic i p a n t s

s p e c i f i e d u n n e c e s s a r y p a r a m e t e r s i n m e n u s ( P a r a m e t e r

L o o p E r ro r , # 3 ) . O f th e s e f iv e , f o u r w e r e i n t h e c o m -

p l e t e s y s t e m g r o u p a n d o n e i n t h e t r a i n i n g w h e e l s

g r o u p . . O n e o f t h e s e c o m p l e t e s y s t e m p a r t i c i p a n t s

s e e m e d d i f f e r e n t f r o m t h e o t h e r s w h o m a d e t h i s e r r o r :

S h e m a d e o n l y E r ro r 3 o n o n e m e n u , m a d e t h e e r r o r

o n l y t w i c e i n a l l , a n d w a s t h e o n l y p e r s o n t o m a k e t h e

e r r o r a f t e r h a v i n g s u c c e s s f u l l y a v o i d e d i t e a r li e r i n t h e

e x p e r i m e n t . T h e o t h e r f o u r l e a n e r s w h o m a d e t h i s er -

r o r m a d e i t r e p e a t e d l y a n d i n d if f e r e n t m e n u e n v i r o n -

m e n t s .

W e c a n n o t s a y t h a t t h e t r a i n i n g w h e e l s i n t e r f a c e

h e l p e d o u r n e w u s e r s l e a r n n o t t o m a k e E r r o r 3 b e -c a us e t h i s e r r o r s e e m s t o be s e l f - s u s t a i n i ng onc e i t i s

c o m m i t t e d . H o w e v e r , t h e r a w f i g u r es d o su g g e s t t h a t

pe op l e u s i ng t he t r a i n i ng i n t e r f a c e a r e l e s s l i ke l y t o

m a k e t h e e r r o r i n t h e f i r st p l ac e . P e r h a p s , h a v i n g a l -

r e a d y se e n t h a t m a n y m e n u c h o i c es a n d k e y p r e s s c o m -

m a n d s a r e d i s a b l e d i n t h e t r a i n i n g i n t e r f a c e , t h e s e n e w

u s e r s a re d i s i n c l in e d t o g o b e y o n d t h e t r a i n i n g m a n u a l

i n s tr u c t io n s b y t r y in g a n o n r e q u i r e d m e n u o p t io n .

A l t h o u g h o u r l e a r n e r s a m p l e i s q u i t e s m a l l , s o m e

p r e l i m i n a r y o b s e r v a t i o n s s e e m s t r ik i n g . N o p a r t i c ip a n t

i n e i t h e r g r o u p c l e a r l y l e a r n e d n o t t o m a k e E r r o r 3 . In

f a c t, in t he on l y une qu i vo c a l c a s e o f l e a r n i ng , a pa r t i c i -

p a n t l e a rn e d h o w t o r e c o v e r q u i c k l y ; t h i s d i d n o t p r e -

v e n t h e r f r o m m a k i n g t h e e r r o r . T h e p a r t i c i p a n t s ' f i r s te n c o u n t e r w i t h a p o t e n t i a l e r r o r s i t u a t i o n s e e m e d t o

d e t e r m i n e p e r f o r m a n c e i n f u t u r e s i tu a t i o n s : T h o s e p a r -

t i c i p a n t s w h o c o m m i t t e d t h e e r r o r a t t h e i r f i r s t o p p o r -

t u n i t y a l w a y s s e e m e d t o re p e a t i t ; t h o s e p a r t i c i p a n t s

w h o d i d n o t c o m m i t i t a t t h e f i r s t o p p o r t u n i t y , t e n d e d

no t t o c om m i t i t a t a l l .

S e v e n o f t h e 1 2 p a r t i c i p a n t s m a d e t h e A l t e r n a t e S h i f t

E r r o r ( #4 ) , i m p r o p e r l y u s i n g t h e a l t e r n a t e s h i ft k e y

w h e n a t t e m p t i n g t o c a n c e l m e n u s o r se l e c ti o n s . T h r e e

o f t h e s e s e v e n u s e d t h e t r a i n i n g w h e e l s i n t e r f a c e; f o u r

u s e d t h e c o m p l e t e s y s t e m . I n c o n t r a s t t o E r r o r 3 , f i v e o f

t h e s e v e n p a r ti c i p a n t s w h o m a d e t h is e r r o r d id d e m o n -

s t ra t e t h a t t h e y h a d l e a r n e d t o c o r r e c t t h i s m i s t ak e .

T h i s i n c l u d e d a ll t h r e e o f t h e t r a i n i n g w h e e l s l e a r n e r sa n d t w o o f t h e c o m p l e t e s y s t e m l e a rn e r s . O n e o f t h e

r e m a i n i n g p a r t i c i p a n t s u s e d t h e c a n c e l f u n c t i o n c o r -

r e c t ly f o r a ti m e , t h e n b e g a n t o m a k e t h e e r ro r . T h e

r e m a i n i n g p a r t i c i p a n t n e v e r c o r r e c t l y u s e d i t , a l t h o u g h

s he ke p t t r y i ng .

E r r o r 4 w a s f a i r l y pe r va s i ve ; f ou r o f t he f i ve pa r t i c i -

p a n t s w h o a v o i d e d t h e e r r o r s i m p l y h a d n o o c c a s i o n to

u s e t h e c a n c e l f u n c t i o n a n d t h e r e f o r e n e v e r r i s k e d

m a k i n g t h e e r r o r . I n d e e d , o n l y o n e o f t h e p a r t i c i p a n t s

w h o t r i e d t h e o p e r a t i o n a t a l l u s e d i t s u c c e s s f u l ly

t h r o u g h o u t . P e r h a p s w e do h a v e h e r e a n i n d i c a ti o n t h a t

t h e t r a i n i n g i n t e r f a c e i s s u p p o r t i n g s u c c e s s f u l l e a r n i n g .

L e a r n e r s u s i n g b o t h s y s t e m s m a d e E r r o r 4 , b u t a l l o f t h e

t r a i n i n g w h e e l s l e a r n e r s c o r r e c t e d t h e e r r o r w h e r e o n l y

h a l f o f t h e c o m p l e t e s y s t e m l e a r n e r s w e r e a b l e t o d o

this .

B y f ar t h e m o s t c o s t l y e r r o r f o r p a r ti c i p a n t s w a s m i s -

na m i ng a da t a d i s ke t t e ( D i s ke t t e N a m e E r r o r , #6 ) . I t

c o n s u m e d l a r g e s p a n s o f l e a r n i n g t i m e a n d i t w a s f a i r l yc o m m o n ; s e v e n o f t h e 1 2 l e a r n e r s m a d e t h e e r ro r .

T h r e e o f t h e s e w e r e u s i n g t h e t r a i n i n g w h e e l s i n t e rf a c e ,

f o u r w e r e u s i n g t h e c o m p l e t e s y s t e m . O f th e s e s e v e n ,

o n l y t w o s e e m e d t o h a v e l e a r n e d t h e c o r r e c t co n c e p t

b y t h e e n d o f t h e e x p e r i m e n t . B o t h w e r e i n t h e t ra i n i n g

w h e e l s g r o u p . T h r e e o f t h e r e m a i n i n g p a r t i c i p a n t s , o n e

i n t h e t r a i n i n g w h e e l s g r o u p , t w o o n t h e c o m p l e t e s y s -

t e m , m a n a g e d t o m a k e p r o g r e s s i n s p it e o f t h e f a c t t h a t

t h e y s e e m e d t o h a v e n o c l e a r c o n c e p t f o r a d a t a d i sk -

e t t e n a m e . T h e l a s t t w o p a r t i c i p a n t s , b o t h o n t h e c o m -

p l e t e s y s t e m , w e r e n e v e r a b l e t o u s e t h e d i s k e t t e n a m e

c o r r e c tl y a n d w e r e u n a b l e t o c o n t i n u e m a k i n g p r o g r e s s

o n c e t h e y h a d c o m m i t t e d t h e e r r o r. O n e l e a r n e r s p e n t

9 0 m i n u t e s a t t e m p t i n g t o c o r r e c t l y n a m e a d a t a d i s k -e t te . A ga i n , i n s p i t e o f t he s m a l l nu m b e r s , i t i s w o r t h

no t i ng t ha t t he t r a i n i ng i n t e r f a c e f o s t e r e d t he l e a r n i r i g

t h a t d i d o c c u r ; p a r t i c i p a n t s w h o l e a r n e d t o c o r r e c t l y

n a m e d i s k e t t e s w e r e u s i n g t h e t r a i n i n g w h e e l s i n t e r f a c e

w h i l e t h e p a r t i c ip a n t s w h o w e r e u n a b l e t o r e c o v e r i n

t h e s p e c i f i e d t i m e w e r e o n t h e c o m p l e t e s y s t e m .

D I S C U S S I O N

T h e t r a i n i n g s y s t e m t e s te d i n t h i s s t u d y p r e s e n t s a fi r st

a p p r o x i m a t i o n o f a s o l u t i o n f o r i n t r o d u c i n g n e w u s e r s

t o t h e m e c h a n i c s o f c o m p l e x s y s te m s . O n e o b v i o u s n e x t

s t ep i n t h i s r e s e a r c h w o u l d b e t o re i t e r at e o u r s e l e c t i o n

o f b l oc ke d e r r o r s . E r r o r s w e d i d no t b l oc k ( e .g ., na m i ng

a d o c u m e n t w i t h t h e s a m e n a m e a s th a t o f t h e d a t ad i s k e tt e , a n d t h e n c o n f u s i n g t h e t w o ) t u r n e d o u t t o b e

t r o u b l e s o m e f o r o u r p a r t i c ip a n t s . C o n v e r s e l y , f o u r o f

t h e s e v e n e r r o r s w h o s e c o n s e q u e n c e s w e d i d b l o c k , a c -

t u a l l y h a d l it tl e e f f e c t o n t h e p e r f o r m a n c e o f p a r t ic i -

p a n t s . A n o t h e r d i r e c t i o n f o r r e s e a r c h w o u l d b e a c lo s e r

e x a m i n a t i o n o f t h e d i s a b l e m e n t m e s s a g e s, w h i c h i n -

f o r m e d u s e r s t h a t a r e q u e s t e d s y s t e m f u n c t i o n w a s u n a -

v a i la b l e . In t h e t r a i n i n g w h e e l s i n t e r f a c e , E r r o r 4 - - n e -

g l e c t in g t o h o l d d a w n t h e a l t e r n a t e s h i f t w h i l e c a n c e l -

l i n g a m e n u o r s e l e c t i o n - - e l i c i t e d a m e s s a g e t h a t t h e

u t i l i t y k e y w a s n o t a v a i l a b l e . T h i s m i g h t b e u n d e r -

s t a n d a b l y c o n f u s in g i n th a t t h e p e r s o n m a y n e v e r h a v e

i n t e n d e d t o r e q u e s t t h e s y s t e m u t i l it y m e n u i n th e f i rs t

p l a c e ! A b e t t e r m e s s a g e m i g h t i n f o r m t h e p e r s o n t h a t i ti s n e c e s s a r y t o h o l d d o w n t h e a l t e r n a t e s h i ft k e y w h i l e

c a nc e l l i ng .

B e y o n d t h e s e t e c h n i c a l i s su e s , t h o u g h , t h e v a r i e t y o f

r e s u l t s w e h a v e p r e s e n t e d , b o t h q u a l i t a t i v e a n d q u a n t i -

t a t i v e , d e m o n s t r a t e t h a t t h e t r a i n i n g w h e e l s i n t e r f a c e

f a c i l i t a t e d l e a r n i ng . M or e ove r , t h i s f a c i l i t a t i on i s m or e

t h a n a m e r e r e l e a s e o f e r r o r r e c o v e r y t im e t h r o u g h

e r r o r b lo c k i n g . N e v e r t h e l e s s , t h i s d e m o n s t r a t i o n i t se l f

f u r t h e r r a i s e d a se r i o u s q u e s t i o n o f i n t e r p r e t a ti o n . O n

t h e o n e h a n d , t h e t r a i n i n g w h e e l s i n t e r f a c e c a n b e

A u g u s t 1 9 8 4 V o l um e 2 7 N u m b e r 8 C o m m u n i ca t io n s o f t he A C M 805



Research Contr ibut ions

t h o u g h t o f a s a t a c t i c a l t e c h n i q u e f o r s a l v a g i n g t h e

l e a r n a b i l i t y o f d i f f i c u l t s y s t e m i n t e r f a c e s . B u t o n t h e

o t h e r h a n d , i t c a n b e t h o u g h t o f a s a s tr a t e g i c t e c h n i q u e

f o r a r c h i t e c t i n g t h e f u n c t i o n o f s y s t e m s a n d i n c o r p o r a t -

i n g o n l i n e t r a i n i n g i n t o s y s t e m d e s i gn s . T h e s e t w o

v i e w s a r e n o t c o n t r a d i c t o r y , b u t t h e y a r e d i f f e re n t .

O n t h e f i rs t v i ew , t h e t r a i n i n g w h e e l s i n t e r f a c e i s a n

i l l u s t r a t i o n o f a s i m p l e t e c h n i q u e f o r r e t ro f i t t i n g a u s e r

i n t e r f a c e to a c h i e v e e n h a n c e d e a s e - o f - le a r n i n g . In o u rc a s e, t h e t r a i n i n g w h e e l s a l t e r a t i o n w a s m a d e t o a n

e x i s t e n t c o m m e r c i a l s y s t e m , a t t h e a s s e m b l y c o d e l e v e l ,

i n l e s s t h a n o n e m a n - m o n t h . G i v e n t h e e m p i r i c a l re -

s u l t s o f o u r e x p e r i m e n t , o n e c o u l d i n d e e d h a z a r d t h a t

w e h a v e s u c c e e d e d i n g i v in g t h e l e a r n a b i l i t y of a s ta t e -

o f - t h e - a r t s y s t e m a l i t t le p u s h i n t h e r i g h t d i r e c t io n , a n d

a t a m o d e s t m a n - m o n t h c o st . B u t i s t h e r e a n y l o n g e r

t e r m i m p l i c a t i o n i n t h e t r a i n i n g w h e e l s i n t e r f a c e ?

C l e a r l y , a m o r e s i g n i f i c a n t a n d l o n g e r t e r m g o a l i n

s y s t e m d e s i g n i s t h a t o f d e f i n i n g a u s e r i n t e r f a c e a r c h i -

t e c t u r e w i t h w h i c h t o c o n f r o n t l e a r n a b i l i t y a n d t r a i n i n g

i s s ue s , a n d w i t h w h i c h t o e l i m i n a t e t h e n e e d t o r e t ro f i t

f o r l e a r n a b i l i t y i n t h e f i r s t p l a c e . T h i s i s n o t a s i m p l e

i s su e ; e v e n t h e f i r s t - o r d e r d i s t i n c t i o n s i n u s e r i n t e r f a c ea r c h i t e c t u r e a r e o n l y n o w b e i n g t h r a s h e d o u t ( e .g ., [ 2] ).

N e v e r t h e l e s s , i t i s w o r t h r e f l e c t i n g o n t h e p r o p o s i t i o n

t h a t t h e t r a i n i n g w h e e l s a p p r o a c h , n a m e l y " s t a g i n g " t h e

p r e s e n t a t i o n o f f u n c t i o n t o u s e r s, i s a n a r c h i t e c t u r a l

p r i n c i p l e .

C o n s i d e r t h e c u r r e n t c o n f l i c t b e t w e e n t h e r a p i d d i-

v e r s i f i c a t i o n o f u s e r i n t e r f a c e s t y l e a n d s y s t e m f u n c -

t i o n , o n t h e o n e h a n d , a n d t h e g o a l o f p r o v i d i n g e f fe c -

t i v e t r a i n i n g fo r n e w u s e r s , o n t h e o t h e r. S t a n d a r d a p -

p r o a c h e s t o tr a i n i n g , n a m e l y , o n l i n e t u t o r i a l s a n d s e l f -

i n s t r u c t i o n m a n u a l s , h a v e f a i l e d u p to n o w t o r e s o l v e

t h i s c o n f l i c t - - a n d i t is a r g u a b l e t h a t t h e y a r e i n d e e d

l o s in g g r o u n d a s s y s t e m s b e c o m e m o r e d i v e r s e a n d

c o m p l e x . A t r a i n i n g w h e e l s i n t e r f a c e p r o v i d e s a n a l t e r -n a t i v e a p p r o a c h f o r t h e d e s i g n o f t r a in i n g : a n i n i t i a l l y

s i m p l e a n d e r r o r - c u s h i o n e d s y s t e m i n t e r f a c e t o h e l p d e -

v e l o p a f o u n d a t i o n o f c o n c e p t s a n d c o n f i d e n c e , a n d a

b r i d g e f ro m m a s t e r y o f th e b a s i c f u n c t i o n t o a c q u a i n t -

a n c e w i t h t h e c o m p l e t e s y s t e m ( b y k e e p i n g m e n u s ,

m e s s ag e s , p r o m p t s a n d h a r d w a r e i d e n t ic a l b e t w e e n t h e

t r a i n i n g w h e e l s i n t e r f a c e a n d t h e c o m p l e t e s y s t e m i n -

t e r f a c e ) .

T h e r a t i o n a l e f o r s u c h a n a p p r o a c h i s th a t a t r a i n i n g

w h e e l s i n t e r f a ce p r o v i d e s a n e x p l o r a t o r y e n v i r o n m e n t

[ 1] t o t h e n e w u s e r , a n e n v i r o n m e n t t h a t a f f o rd s a c t iv e

i n v o l v e m e n t i n t h e l e a r n i n g p r o c e s s ( l e a rn i n g b y do i n g )

w i t h r e a s o n a b l e p r o t e c t i o n f r o m t h e c o n s e q u e n c e s o f

e r r o r s t h a t a c t i v e l e a r n i n g i n e v i t a b l y e n t r a i n s [ 3, 4] .T h i s i s i n c o n t r a s t t o s t a n d a r d a p p r o a c h e s t o t h e d e s i g n

o f t r a i n i n g ( i n c l u d i n g o n l i n e c o m p u t e r t u t o r i a ls ) w h i c h

p l a c e t h e l e a r n e r i n a r e l a t i v e l y p a s s i v e r o le , a n d w h i c h

e i t h e r o b s t r u c t l e a r n e r i n i t i a t e d a c t i v i t y ( m o s t o n l i n e

t u t o r i a l s a l l o w o n l y o n e " c o r r e c t " a n s w e r i n a n y i n p u t

f i el d ) o r d i s c o u r a g e i t a n d p r o v i d e n o p r o t e c t i o n f r o m

t h e c o n s e q u e n c e s o f e r r o r s ( m o s t s e lf - i n s t r u c t i o n m a n -

u a l s a r e w r i t t e n u n d e r t h e a s s u m p t i o n t h a t l e a r n e r s

w i l l fo l l o w t h e m t o th e l e t t e r, a n d n e v e r m a k e a m i s -

t a k e ) . O f c o u r s e a l l b e t s a r e o f f i f s y s t e m i n t e r f a c e s

s h o u l d e v e r b e c o m e s o s im p l e t h a t l e a r n i n g i s v i r t u a l l y

i n s t a n t a n e o u s a n d t r a i n i n g u n n e c e s s a r y . T h e r e i s, u n -

f o r t u n a t e l y , n o t h i n g t o i n d i c a t e t h a t a n y s u c h c i r c u m -

s t a n c e s a r e i n t h e o f f i n g f o r t h e f o r e s e e a b l e f u t u r e .

A n e w u s e r i s v e r y m u c h l i k e a n o u t - o f - t o w n e r s e t

d o w n i n t h e m i d d l e o f a s t r a n g e c i t y . A g oa l , a m a p , a n d

s o m e d i r e c t i o n s a l l m a y h e l p , b u t t h e p o s s i b i l i t i e s f o r

g e t t in g l o s t a re o v e r w h e l m i n g . M o s t i n s i d i o u s o f al l i st h e f a ct t h a t i f t h e o u t - o f - t o w n e r m a k e s a w r o n g t u r n

h e o r s h e m a y n o t e v e n b e c o m e a w a r e o f it , b e c a u s e

e v e r y t h i n g l o o k s s t r a n g e . T h e p r e s e n t r e s e a r c h s u g g e s t s

t h a t a p r a c t i c a l a l t e r n a t i v e t o t h e c u r r e n t m e t h o d s o f

i n d u c i n g n e w u s e r s i n t o t h e m y s t e r i e s o f s m a l l s y s t e m s

i s t o c r e a t e a n e n v i r o n m e n t i n w h i c h t h e l e a r n e r i s

e i t h e r c o r r e c t o r e ls e is c o r r e c t e d w i t h o u t p e n a l t y . T h e

c o n s e q u e n c e s o f e r r o r s a r e b l u n t e d , a n d a p p r o p r i a t e

f e e d b a c k i s i m m e d i a t e . I n s u c h a n a p p r o a c h , t h e n e w

u s e r i s a b le t o t a k e c o n t r o l o f t h e l e a r n i n g s i t u a t i o n a n d

o f t h e s y s t e m , t o d o s o m e t h i n g r e a l a n d r e c o g n i z a b l e

i m m e d i a t e l y , a n d t o a v o i d t h e s i d e t r a c k s a n d e r r o r

t a n g l e s t h a t l o o m s o l a r g e t o d a y .

A c k n o w l e d g m e n t s , W e a r e g r a t e f u l t o D a v i d B o o r , w h o

d e s i g ne d t h e i m p l e m e n t a t i o n f o r t h e T r a i n i n g W h e e l s

I n t e rf a c e . T h i s c o l l a b o r a t i o n m a d e i t p o s si b l e to i t e r a -

t i v e l y d e s i g n a t r a i n i n g e n v i r o n m e n t f r o m a n e m p i r i -

c a l l y d e r i v e d i n v e n t o r y o f u s e r e r r o rs . W e a l s o t h a n k

C l a y t o n L e w i s , A n n G r u h n , M a r y B e t h R o s so n , a n d

M a u r e e n R u s k i e w h o c o m m e n t e d o n s e v e r a l d r a ft s of

t h i s p a p e r . C l a y t o n L e w i s s u g g e s t e d t h e t e r m " t r a i n i n g

w h e e l s . "

REFERENCES

1. Carroll. J.M. The ad venture of getting to know a computer. Computer15, 11 (No v. 1982). 49-58 .

2. Carroll. ].M. Presentation and form in user-interface architecture.BYTE 8, 12 (D ec. 1983). 113-122.

3. Carro ll. J.M.. and M ack. R.L. Activ ely learning to use a w ord proces-sor. In Cognitive Aspects of S killed Typew riting, W.E. Coope r, ed.Springer-Verlag,New York, 1983.

4. Carro ll, J.M.. and M ack, R.L. Learn ing to use a word proc essor:Bydoing, by thinking, and by knowing. In Hu ma n Factors in Com putingSystems, J.C. Tho mas and M . Schne ider, eds. ABLEX,Norwo od, N.J.,1984.

5 . Fe rgu son . G.A. S ta t i s t i c a l A na ly s i s in P s y c ho logy and E duc a t ion . M c-

Gra w-Hi l l , Ne w York . 1971 .6 . M a c k , R . L. , Le wi s , C .H. , a nd C a r ro l l , LM. Le a rn i ng t o use wo rd

p r o c e s s o rs : P r o b l e m s a n d p r o s p e c ts . AC M Tra ns . Off . lnfo. Sys t . 1, 3

(July 1983), 254-271,

CR Categories and Subject Descriptors: H.1.2 [Models a nd Princi-ples]: U s e r / M a c h i n e S y s t e m s - - h u m a n f a c to r s , h u m a n i n f o r m a t i on p r o ce s s -

ing; H.4.1 [ In format ion System s Ap pl icat ions] : O f f ic e A u t o m a t i o n - -w or d pr oc e s s ing

General Terms: H u m a n F a c to r sAddit ional K ey Words and Phrases: u s e r i n t e r f a c e a r c h i t e c t u r e , u s a -

b i l i t y , e a s e o f l e a r n i n g , h u m a n - c o m p u t e r i n t e r a c t i o n , h u m a n l e a r n i n g ,

t r a i n i n g , e d u c a t i o n

R e c e i v e d 7 / 8 3 : r e v i s e d 2 / 8 4 : a c c e p t e d 3 / 8 4

A u t h o r s ' P r e s e n t A d d r e s s e s : J o hn M . C a r r o l l , C o m p u t e r S c i e n c e D e p t ..

I B M T h o m a s W a t s o n R e s e a r c h C e n t e r , Y o r k t o w n H e i g h t s , N Y 1 0 5 9 8 :C a r o l i n e C a r r i t h e r s , P s y c h o l o g y D e p t. , C o l u m b i a U n i v e r s i t y , N e w Y o r k ,NY 1O027.

P e r m i s s i o n t o c o p y w i t h o u t f e e a l l o r p a r t o f th i s m a t e r i a l i s g r a n t e d

p r o v i d e d t h a t t h e c o p i e s a r e n o t m a d e o r d i s t r i b u t e d f o r d i r e c t c o m m e r -c i a l a d v a n t a g e , t h e A C M c o p y r i g h t n o t i c e a n d t h e t i t l e o f t h e p u b l i c a t i o na n d i t s d a t e a p p e a r , a n d n o t i c e i s g i v e n t h a t c o p y i n g i s b y p e r m i s s i o n o f

t h e A s s o c i a t i o n f or C o m p u t i n g M a c h i n e r y . T o c o p y o t h e r w i s e , o r t or e p u b l i s h , r e q u i r e s a f e e a n d / o r s p e c i f i c p e r m i s s i o n .

806 C o m m u n i ca t io n s o f t h e A C M A u g u s t 1 9 8 4 V o l u m e 2 7 N u m b e r 8

training wheels in an user interface

Documents