Int. J. Man-Machine Studies (1991) 35, 235-250

Informing HCI design through conversation analysts

M. A. NORMAN AND P. J. TIIOMAS

Department of Computer Science, The University of Hull, thdl, UK

(Received 27January 1989 and in revised form 5January 1990)

HCI encompasses a variety of disciplines to provide knowledge of the user and seeks to make this knowledge available to designers in a variety of practical ways. In seeking to enrich the multidisciplinary base of ltCI, to provide designers with enhanced knowledge of the user and to facilitate interface design, the authors are currently exploring conversation analysis, a social-scientific approach to the investi- gation of interaction. Conversation analysis provides a methodology, a set of attalytic constructs and a collection of established findings about interaction which will prove important in the investigation and design of human-computer interaction in a way that transcends technological issues. The paper discusses issues concerned with the applicability of conversation analysis to HCI, and provides examples of studies carried out on a corpus of video-rccorded human-computer interaction. These examples demonstrate that the findings of conversation analysis can inform the design of interactive systcms, and that its methods may be uscd productively in the investigation of human-computer interaction.

1. Introduction

The role of HCI design as an activity within the design process has been problematic. In part this is attributable to the nature and comparatively limited scope of our available knowledge--which in any activity which seeks to include the human component as part of its area of study is inevitably problemat ic- -and in part to the way in which our available knowledge is made available and applied.

It is possible to trace the development of HCI as it has moved from a discipline which contributes to design after the artefact is constructed, through the develop- ment of various sets of guidelines to aid the designer (Smith & Mosier, 1984; Shackel, Eason, Gardener & Mckenzie, 1988), to the provision of software tools, techniques and methods (Farooq & Dominick, 1988). In these last cases, the focus has been on informing the designer about aspects of HCI through various characterization of computer-based systems and the features of the interface. The tendency has been toward finding the pragmatic solution. However , as a discipline, HCI seeks to understand and thereby advance our knowledge of users in the context of carrying out some task through the use of computing systems. The breadth of this enterprise is such that it involves the conjunction of work from apparently disparate disciplines (with the attendant overheads and difficulties of all multidisciplinary activities), the identification of applicable areas of study, and the development of principled approaches.

This paper reports on one such discipline where a considerable body of prior work exists which may be used to inform HCI design, although it represents an investigative paradigm quite different to those presently involved in HCI research. The studies reported here suggest that conversation analysis, a sociological approach

235 0020-7373/91/020235 + 16503.00/0 (~) 199 i Academic Press Limited

236 M. A. NORMAN AND P. J. TIIOMAS

to the analysis of human interaction, offers a fruitful approach to the characteriza- tion and investigation of human-computer interaction.

We view conversation analysis as relevant to HCI in several sp~ecific ways:

�9 the findings of conversation analysis present a source of design knowledge which has so far been unavailable to designers;

�9 these findings relate to fundamental features of interaction and are as such widely applicable;

�9 the possibility exists that the findings may be stated in a simple form and made accessible to designers;

�9 the methods of conversation analysis provide a productive approach to the analysis of human-computer interaction;

�9 the design of systems around the existing features of interaction, and the pervasive abilities of users for interaction, will result in more effective human-computer interaction.

The following section outlines the rationale for the studies, discussing previous accounts of the relationship between conversation and human-computer interac- tion, and providing a brief overview of conversation analysis, and its status vis-h-vis other approaches to interaction analysis. The way in which conversation analysis is employed in the studies, with particular reference to methodological issues, is discussed in section 3, and the corpus of data on which the studies are based is described. The remainder of the paper describes and investigates representative sequences of human-computer interaction.

2. Conversation analysis and human-computer interaction

2.1. C O M P U T E R S A N D C O N V E R S A T I O N

Prominent among the concerns of HCI are those of "meaning" and "inter- pretation". It would be appropriate therefore to look to disciplines which deal with such issues for applicable findings. In this context there is a fundamental aspect of interaction which has not been systematically examined in relation to human- computer interaction--the interaction between conversational partners. Of course the "conversational metaphor" has been common currency in studies of human- computer interaction, from, for example, Orr's Conuersational Compttters (1968), through Gaines and Shaw's The Art of Computer Conuersation (1974), to studies by Foley and Wallace (1974), and Nickerson (1976).

However, the juxtaposition of conversation and human-computer interaction in this paper is not to be taken as a proposal that human-computer interaction and conversational interaction are isomorphic. Rather it is concerned with the proposal that a discipline which investigates human interaction is applicable to investigation and design of human-computer interaction. Any similarities which may exist between conversation and human-computer interaction--and one finding of this work is that there are such similarities--are located at the fundamental bases of interaction, rather than at the level of simple phatic interactional routines. Adopting this perspective will lead to a principled approach to the design of user-system interaction.

CONVERSATION ANALYSIS AND |ICI 237

2.2. C O N V E R S A T I O N A N A L Y S I S

As one of a number of disciplines which focus on the nature of human interaction, conversation analysis is concerned with the description of interactions coffducted though talk. Even though conversation analysis has been hitherto primarily concerned with human spoken interaction, it has much to offer to HCI research. Specifically the methods of conversation analysis offer the HCI researcher access to the interpretations generated by users in the course of interaction; the analytic constructs of conversation analysis provide a technical vocabulary for describing features of interaction; and the findings of conversation analysis provide a resource for the design of human-computer interaction.

2.Z 1. The perspective and findings of conversation analysis The following sections provide only a brief account of conversation analysis. Readers who wish to pursue a more detailed inquiry into conversation analysis are directed to introductory chapters in collections of conversation analytic studies by Atkinson and Drew (1979), Atkinson and Heritage (1984), and Button and Lee (1987).

Essentially, conversation analysis builds on the premise that conversational interaction is an accomplishment. Conversation analysis suggests that two important characteristics of interaction are that it is orderly, and that it allows mutual h~telligibility between speakers. Interaction is orderly in the sense that not just anything may be said to be engaged in "sensible" interaction; mutual intelligibility is possible since conversation is specifically designed to provide for display, and thus understanding of, the intentions, actions and motives of speakers.

Order and mutual intelligibility are achieved without any noticeable effort on the part of speakers, and so are based upon tacit abilities. In this sense, conversation analysis builds on the ethnomethodological premise tfiat speakers engage in "practical reasoning" or "sense-making" about the social world (Garfinkel, 1967). Conversation analysis examines one particular aspect of the process of sense-making in social interaction--the production and recognition of systematic interactional structures. It is the organization of interaction around familiar and expected structures which makes conversation orderly and sensible, rather than random and meaningless, and makes mutual intelligibility possible.

One particular finding may be taken as an example of conversation analytic interests. The adjacency pair (Schegloff & Sacks, 1973) consists of a balanced pair of utterances from successive speakers, and is a format around which activities as greetings, requests and accusations are organized. In the examples below, A and B indicate speakers' identities:

Greeting Request A ccusation A: Hello A: What time is it? A: It's your fault! B: Hi B: Five-forty B: No wily.

From the perspective of conversation analysis, it is possible to make several observations about the adjacency pair. Firstly, it is not the case that the two parts of the pair are merely adjacent, but that when the first part of a pair is produced a second part is then required--"second speakers" operate under the constraint that they should produce a response. This constraint has been termed the conditional

238 M. A. N O R M A N A N D P. J. T I IOMAS

relevance which holds between the two parts of the pair. The way in which adjacency is not to be equated with proximity, but rather with an obligation is shown in such conversations as

A: Can I have a box of matches? B: Large or small? A: Small B: Sure

where the second part of the pair, (B's "Sure"), does not occur immediately following the first (A's "Can I have a box of matches?"), but after intervening talk. Nevertheless, although "second speakers" operate under a constraint to produce a second part, this does not mean that they are compelled to do so. This is partly to observe that conversation is an improvised and creative, rather than determined and pre-scripted, activity. However, if a second speaker does not produce a response, the first may infer that the second is, for example, sulking or angry. This means that what might be considered as merely conversational silences or lacunae may be, effectively, owned by particular speakers. Thus "no reply" to a greeting from speaker A, may be "speaker-B's-no-reply-to-a-greeting", rather than a silence belonging to neither A nor B.

A second observation may be made regarding the cohesion between the two parts of the pair. This is that not any kind of second part is adequate. This is so since the first part of the pair projects the kind of activity which should follow, and requires that an appropriate second part must be produced. Thus the adjacency pair

A: Hi B: Drop dead

would not be seen as accomplishing a greeting (at least, presumably not by A or any of his friends). Second speakers are thus under an additional constraint to produce appropriate second parts of adjacency pairs, and if they do not, again the first speaker may make inferences and draw conclusions about the second (here, A might infer that B is thoroughly unpleasant and conclude that he is best avoided).

Thirdly, the notion that speakers are constrained, yet do not operate under any absolute prohibition on their actions is illustrated by the observation that there may be a number of second parts which are possible after any particular first. So, for example, accusations may be followed by denials or admissions, and invitations by acceptances or rejections.

Although the adjacency pair as briefly described here is only one of a large number of findings, it provides a view of the major concerns of conversation analysis. These may be summarized as:

�9 the importance roles of speaker and hearer and the ways that those roles are established and interchanged;

�9 the importance of structure and format in interaction; �9 the importance of constraints and expectations; �9 the importance of the sequential context of interaction (utterances must occur in, influence,

and be influenced by, an immediate context); �9 the way in which conversation is organized from withbl by the participants, rather than

being the product of external "rules for speaking"; �9 the aim to present an insider's view of conversation.

CONVERSATION ANALYSIS AND llCI 239

This last concern encapsulates the notion that the kinds of interpretations, meanings and processes which are of interest to conversation analysis are those important to speakers themselves rather than those which are merely "an~alysts" readings".

This final concern also defines the difference between conversation analysis and other approaches to interaction analysis such as Speech Act Theory (Searle, 1970), or Discourse Analysis (Sinclair & Coulthard, 1975). Speech Act Theory attempts analysis of the function of utterances on the basis of syntactic and semantic properties independent of discursive contexts, and is perhaps most familiar for analyses of "indirect" speech acts based on the relationship between utterance and intention. While it treats language as social action, as does Conversation analysis, Speech Act Theory clearly stands at odds with the conversation analytic perspective which asserts that the meaning of a conversational action cannot be established without reference to context. Discourse analysis, in contrast, is an attempt to apply the notion of hierarchical linguistic sentence structure at the level of interaction by decomposing discourse into grammars of functional categories. Although discourse analysis, unlike Speech Act Theory, bases its analyses on "naturally-occurring" data, these are not scrutinized for the ways in which they are understood by participants. It is the way in which conversation analysis attempts to provide this "insider's view" of interaction which makes it attractive as a framework within which to investigate user-centred HCI design, and it is this which such approaches as Speech Act Theory and discourse analysis fail to provide.

Conversation analytic studies have examined a wide range of phenomena. Studies have investigated the ways in which speaking turns are organized, describing sets of schematic "rules" for turn-taking (Sacks, Schegloff & Jefferson, 1974), the systema- tic organization of corrections (Schegloff, Sacks & Jefferson, 1977; Jefferson, 1987), and the opening and closing of conversations as "technical problems" resolved through the use of adjacency pairs (Schegloff, 1968; Schegloff & Sacks, 1973). A definitive bibliography of conversation analytic studies appears in Heritage (1988).

22.2. The methods of conversation attalysis Conversation analytic methods seek to preserve the "integrity of the phenomena". Anxious to avoid the use of idealized or "clean" data, conversation analysis analyses "real-life" tape-recorded interaction. This is then meticulously transcribed using a modified orthography which seeks to record detailed features of conversation. This attention to detail is motivated by the assertion that it is impossible to decide aprioristically what important phenomena might come to light. In this way, features uncovered by detailed transcription are made available for investigation in their own right. So, for example, phenomena such as the exact points where speakers start talking (Jefferson, 1973), the duration and placement of pauses (Jefferson, 1988), gaze (Goodwin, 1981; Beattie, 1983) and posture (Heath, 1982) which might have gone unnoticed, have been investigated.

A second aspect of the attempt to preserve the integrity of the phenomena appears in the approach to analysis taken by conversation analysis. Analysis commences with a inductive search through the minutiae of the transcripts for systematic patternings in interaction. However, in an attempt to provide an "insider's view", analysis also involves the demonstration that analysts' inter-

240 r,l. A. N O R M A N A N D P. J. T i l O M A S

pretations of the interaction are supported by those of speakers. For example, if an analysis proposes an utterance as ambiguous, an attempt will be made to show that it is an ambiguity for those in the conversation, rather than a l~ossible ambiguity conjured up by the analyst (Schegloff, 1984). One way in which these warrants for the analysis are provided is through the examination of the relationship between consecutive utterances by successive speakers. Simply, the analysis is buttressed if it can be shown that a previous speaker's utterance is treated as ambiguous by the current speaker.

It should be clear that this approach to data collection and analysis makes conversation analysis less than attractive to those whose work lies squarely with the experimental paradigm. Additionally, the somewhat bellicose attitude of conversa- tion analysts may be at least partly attributable to the puzzlement, if not outright hostility that conversation analysis has sometimes provoked. Conversation analysts vociferously eschew experimental methods, preferring to collect data of "mundane" activities in "everyday" situations. This means that in practice--if not principle--the methods of conversation analysis do not lend themselves to theories framed in quantitative and predictive terms. Conversation analytic studies are not "replicable" in any strict sense since they are based on particular sets of data, which comprises conversation produced by particular speakers, about a particular topic, in a particular (physical or social) setting. However, underlying the findings of these studies is the common emphasis on the skills and expectations involved in interaction, and it is this emphasis which makes conversation analysis directly applicable to HCI.

3. Conversation analysis and HCI design

3.1. CONVERSATIONAL INTERACTION AND IIUMAN-COMPUTER INTERACTION: TIIE ROLE OF CONVERSATION ANALYTIC FINDINGS

From the perspective of conversation analysis, interaction is an activity organized from within--managed by participants rather than the product of explicitly agreed rules. In contrast, human-computer interaction is shaped by two determinants. The first is the way in which technology available to the designer has been employed. The second is the collaboration between the designed system and the user. The proposal that the findings of conversation analysis are relevant to human-computer interaction addresses both determinants. This amounts to the assertion that we may inform the way in which system features are deployed by the designer, to provide for a change in the ways in which that system integrates with the abilities of users. Put simply, if systems are designed in accordance with the interactional predisposi- tions, expectancies, skills and abilities of users, as revealed by conversation analysis, they will be superior hlteractive artefacts. There are three powerful reasons to support this view.

Firstly, the formats in which certain actions in interaction appear, the expec- tancies they generate, and the abilities on which they are based, are natural to us. Conversation analysis asserts that such knowledge resides at a level below our conscious awareness: this must be the case, since if we were to reflect deeply upon the ways in which we had understood some utterance or arrived at some meaning,

CONVERSATION ANALYSIS AND ttCl 241

social encounters would become impossibly disconcerted. Similarly, if interactive systems were designed so that they operated in markedly different ways to pervasive expectations, users might be forced into such conscious speculation, since they would be engaged in interactions which seemed "odd" or "unnatural". Ultimately, users would need to learn new and unfamiliar interactional techniques, and crucially, users would revert to familiar hlteractional patterns which would conflict with the current design of systems.

Secondly, and as a consequence, systems designed in this way would be easier to learn, since there would be little in the way of cognitive overheads in finding conscious routes to achieve specific interactional goals. This means that they would be simpler to operate since interactions would bear some resemblance to conversa- tional interactions and, ultimately, would lead to more effective work practices.

Finally, guidance derived from conversation analysis would provide a more secure and consistent basis for hzteraction than the differing conventions that are presently devised and adopted for each new generation of technology. As a principled approach to design it offers the prospect of portability, in a way which subsumes technology to the user rather than vice versa.

3.2. CONVERSATION ANALYTIC METHODS IN THE INVESTIGATION OF H U M A N - COMPUTER INTERACTION

3.Z1. The corpus of data In seeking to follow some of the general precepts of conversation analysis, the basis for the studies were naturally-occurring rather than experimental user-system interactions. The data in this preliminary study takes the form of interaction between six users and several pieces of software recorded on videotape. This permanent record is important, in that it allows repeated investigation, discussion and re-investigation of the user's interaction with the system, something not possible with casual observational or anecdotal data.

Each recording session lasted from 1 to 2 hours. The users were not selected by any particular criterion, nor was there any attempt to arrive at a representative sample of users. All were members of staff in the Department of Computer Science, some with extensive and varied experience of computing, others with comparatively brief and restricted experience. In the majority of cases users were recorded in their usual environment, encouraged to go about whatever tasks they had planned for the day, and were not instructed as to how they should behave (in terms of leaving the system, pausing and resuming work, or requesting help).

The material was recorded in ambient light using a single VHS Panasonic NewViCon A1 Camera and Panasonic VHS NV-180 portable recorder and the recordings were time-stamped to allow the relocation of particular sequences of interaction. It should be stressed that a "sequence" is a strip of interaction defined by analysis, rather then by the user: they are represented by start/end tape reference numbers and may be edited-out for comparison and scrutiny. The corpus shows the collaborative use of SmallTalk80 by two experienced users, the use of software such as AppleScan and Microsoft Word by two expert users, and two

242 M. A. NORMAN AND P. J. TtlOMAS

novice users of HyperCardt engaged in authoring interactive educational materials. It is from this last section of the corpus that the examples in this paper are drawn.

3.2.2. Specific methodological considerations There are several points which may be raised in connection with the methods employed in this investigation and their relationship to the methods of conversation analysis.

Firstly, since in these interactions one partner is a human and one a computer system, this means that putative analysts of human-computer interactions do not have the same degree of access to interpretations and meanings that are available to analys.ts of conversation. This is the case since one premise on which conversation analysis rests is that each successive utterance in conversation displays an under- standing, or "performs an analysis of", previous utterances; by virtue of their understanding of language, investigators of conversation h~ve immediate access to this "analysis". Nevertheless, in the case of human-computer interaction it is possible to scrutinize the responses of the user, and examine the internal construction of the system, to arrive at an understanding of the interpretations of the user, and the meaning of the user's actions for the system.

Secondly, the primary aim of the analysis is not only to provide merely a descriptive account of the detailed structure of interactional sequences, but to provide a thorough enough characterization to indicate points of contact between conversation, human-computer interaction, and the way in which conversation analytic findings can be of practical value to design.

Thirdly, the user of the HyperCard system, from which the examples here are drawn, is a novice user (although similar interactions were observed with ex- perienced users in the corpus). An initial study involving novices may be seen as fruitful in two particular ways. Firstly, users are all novices at some stage, and experts may be inexpert users of particular systems. Secondly, expert users have developed operational shortcuts to deal with certain aspects of systems which they find problematic. In this sense, the study of novices allows us to see to what extent features of the interface appear as problematic without the masking effects of experience.

Finally, in attempting to adhere as much as possible to conversation analytic methods, the analysis takes the steps characteristic of a conversation analytic investigation: this is to examine inductively records of naturally-occurring interac- tion, and to explain the structure of recurrent patterns of activity and the reasoning and interpretations involved in particular sequences of interaction. The additional step necessary here is to undertake an assessment of the role of the particular" features of the system which occasion, or are implicated in, the interaction.

4. Investigating human-computer interaction through conversation analysis: some examples

The examples which follow have been selected as representative instances. Although we assert that the application of conversation analysis in this fashion will lead to the

t Microsoft @ is a registered trademark of Microsoft Corporation. Apple @ and Macintosh @ a r e T M "I b . / registered trademarks, and HyperCard and HyperTalk , trademarks of Apple Computer Inc.

1 M Smalltalk80 " is a trademark of the Xerox Corporation.

CONVERSATION ANALYSIS AND llCl 243

production of design guidelines, we do not attempt any definition of guidelines in this paper. A preliminary attempt at such formalization appears in Thomas (1990) and is the subject of our continuing research (Thomas & Norman, 1990).

4.1. PROVIDING A RESPONSE

The instances are all selected from a section of the corpus showing a novice user's interaction with HyperCard. They are all instances of problematic situations. "Problematic" here means problematic for the user, in that they are encountered as interruptions to normal events or the current task. The recordings are a rich source of these sequences. Problematic episodes merit attention not least because they are severely damaging to the progress of the interaction: there are several very lengt~.y sequences where the user is visibly confused, repeats the same series of actions agam and again, and finally requests help in desperation. They also merit specific attention since they are, paradoxically, concerned with what the user and designer see as purely "administrative" or "housekeeping" tasks such as the opening and closing of flies.

Specifically, these examples turn on the notion of adequate response, and the failure of the system to provide a response which may be seen as appropriate by the user. The notion of response is a central feature of human interaction, and should be a central concern in HCI design. The discussion centres on the ways in which the user finds the system's responses problematic, the expectations which the user may hold about the system in the light of conversational interaction and the resources employed by the user to obtain an adequate response. The notion of adequate response is linked to the inherent problems of the designer's view of interaction, the features of the system which support or occasion interaction, and the general resources available for designers to provide responses for users. In particular, a prominent feature of many interactive systems--the separation of selection and action--is identified as a feature of such sequences.

4. I.I. The locked card The first sequence involves a user of HyperCard constructing a hypertext manual for the use of the Apple Macintosh. HyperCard is a hypertext authoring system in which authors create stacks of cards, using the various tools provided to implement multidirectional links between cards, and to create textual and pictorial layouts on each card. The author defines the behaviour of these links, and the specific behaviour of objects such as active screen regions (or buttons), by attaching instructions written in the language HyperTalk to scripts associated with particular objects. Objects' scripts define their behaviour on receipt of messages which are, typically, generated by mouse or keyboard events. The end-user is then free to browse the information contained on each card, navigating through the stack of cards constrained by tile links provided by the author. (A more detailed view of HyperCard may be found in Goodman, 1987). Figure 1 shows the HyperCard display, with drop-down menu, interpreter window, and in the background, buttons.

One concern of hypertext authors is to control the sequential presentation of material. This is done by the prohibition of certain navigational routes. To this end the user in this example has placed a HyperTalk lockmessages instruction in the

244 M.A. NORMAN AND P. J. "DtOMAS

File 1 ~ Go Tools Paint Options Patterns Undo ~Z

. . . . . . . . . . . . . . . . . . . . . . ' O I N T I N G Cut ~B Copy ~C Paste ~V 1 Clear

. . . . . . . . . . . . . . . . . . . . . . . .

New Card ~N Delete Card ~ ~ - - ~ Cut Card 3 Copy Card

Text Style... ~T Background ~B :3

FIGURE 1. A hypercard showing pull-down menu and active screen regions (buttons).

script of the first card of the stack. This inhibits the receipt of any messages by the card, such as those generated by mouse of keyboard events. The intention of the user is to create a frontispiece, which presents an opening message, and is not intended to elicit further action by the reader. Unfortunately, having done this, the user (as author) finds that it is impossible to manipulate the card or its script at all, since the lockmessages instruction is followed by a wait command which further inhibits any action until several seconds have elapsed and the next card is automatically displayed.

The user is here faced with the practical problem of accessing the script to change in the instructions which it contains. The nature of the problem is to prevent the execution of the instructions, and to access the script of the card where the instruction resides, to remove the errant instruction. It must be emphasized that the designer of the system is not at fault here, since it is a problem of the user's own making: the designer is responsible only in so far as he or she has created a language command, along with a set of possible circumstances, which is, at least without substantial knowledge of the system, irreversible. The interest that this sequence of interaction holds is therefore not strictly concerned with the efficacy or otherwise of the design of this system. It is rather that this interaction provides a illustration (and possibly a caricature) of interaction where users attempt to comprehend and contend with the unresponsiveness of a system.

The recording shows that in a first at tempt to prevent the execution of the instruction, the user attempts a mouse-click on the card. This has no effect on the system's actions, since receipt of all mouse events is inhibited. The user then returns to the card, an action which causes the re-execution of the script (since doing so

CONVERSATION ANALYSIS AND I[CI 245

re-executes its script). The interaction is then repeated, but with several pronounced mouse-clicks at different locations on the card. These again have no effect on the system's actions. Subsequently the user tries to "unlock" the card using menu commands available within the system to examine the script of the card, and then issues key-equivalents of the menu-selectable commands, all of which are unsuccess- ful. Finally, the user attempts to issue various textual HyperTalk commands through an interpreter window.

One description of this sequence of events might be in terms of the user attempting to solve a simple puzzle. However from a perspective which emphasized the blteraction between user and system, this episode may be seen somewhat differently. This involves seeing the user and system as contributing mutually to the interaction. The user's contributions are actions involving features of the interface; the system's contribution is the repetition of a sequence of actions. Seen in these terms, the user's repeated attempts to "unlock" the card may be seen as attempts to elicit a response from a non-responsive co-interactant.

There are a set of equivalent circumstances in human conversation where responses are not forthcoming and must be elicited. One inference that is drawn when a response is not forthcoming, in the absence of any information to the contrary, is that there is some problem of hearing. In this case the most appropriate way of eliciting a response is merely to repeat the original utterance. The user's repeated, and then more pronounced, mouse-clicks would seem to be contributions of this order. This in itself is a relatively simple observation. However , the ways that the user goes on to reformulate contributions would seem to parallel the way that conversationalists, in the terms of Pomerantz (1984), "pursue responses" in conversation. One of Pomerantz 's findings was that in order to gain a response, speakers reformulate utterances by locating problematic parts of those utterances and substituting more comprehensible ones. In this way speakers use the control they have over the construction of utterances. In relation to the episode here, one explanation for the user's shift from using the mouse, to using key-shortcuts, to using the message box, is that each potentially allow more control over the content of contributions.

This sequence is also interesting for other reasons. The description provided of this interaction left the sequence at the point where the user's contributions were in the form of textual entries in the interpreter window. Another set of observations may be made about subsequent events. At one point the user enters an instruction into the interpreter window (visible continuously in the lower section of the display) to return to the first card, but does not terminate it with a carriage return. The user then selects a menu item which sends the system message openstack to this, the current stack. The result of this is to execute the script of the first card of the stack. The user then employs the text in the interpreter window to interrupt the execution of the script by pressing the return key. The user is attempting, it would seem, to circumvent the system's contribution by "interject ing", "gett ing in before" , "anti- cipating", or "butting in". The user would thus seem to assume that the machine's contribution should have the property of interruptability, in the same way that conversational turns are interruptable, potentially anywhere, but more usually at specific "transition-relevance places" (Sacks et al., 1974). The user expects tlmt, at certain points in the machine's turn, there are expectable points at which it is

246 r~t. A. NORMAN AND P. J. TIIOMAS

appropriate for it to be interrupted. In contrast, the design of system contributions are non-compositional in nature.

4.1.2. The "open fi/e" dialogue box The second episode concerned with the problem of providing appropriate responses relates to use of a standard file selection box. In Apple Macintosh applications, users have access to the file system through standard dialogue boxes which are invoked by menu selection. The dialogue box provides a combination of buttons which-allow the user to search external disk drives, open a file, and cancel tt}e dialogue,box, and pull-down menus to browse the file structure.

The user invoked this file selection dialogue box from within HyperCard. However, the user was searching for an object which was not a HyperCard stack, and as such it did not appear to exist in the directory in which the user knew it to be located. As a consequence, the display area in which selectable files are shown remained empty. The entire episode amounted to the repeated opening and closing of the file selection box representing unsuccessful attempts to locate the desired file. The user's problem arises from the fact that there is effectively no response from the system in response to repeated, unsuccessful actions.

Viewing this episode again as a sequence of interaction, shows that the problem is not a matter of some signal being misinterpreted by the user, or one concerning a lack of knowledge about how to use this particular dialogue box: the consistency of the design of file selection mechanisms across Apple applications means that the user would have come across this generic mechanism before. In fact, in the normal course of events this dialogue box would not be problematic, and it is in general terms an efficient and pragmatic mechanism for the specific transaction of selecting a file. Rather the problem is far more obvious: the general purpose nature of the dialogue box leads one to expect that any file is selectable through it. This is not the case: the user has fallen foul of that old b~te noir--hidden modes. The user is in the wrong context to select the file, and although it is not indicated anywhere, the file search space is constrained by the current context. The operation of the dialogue box is therefore context sensitive.

One obvious solution would be for the file selection box to either make explicit the "current context", or to provide a prompt with appropriate sub-dialogue to accomplish the required context change. Alternatively one could consider removing the mode constraints from the file selection (allowing applications to handle the problem individually). These however are simply pragmatic responses to the "hidden modes" problem to which there are any number of solutions of varying merit. The important feature of this interaction is that a repeated sequence of actions receives no response. The repetitive nature of the user's actions indicates that a response was required. The problematic character of interactions such as these seems to have come about since in the development of uncomplicated interactional methods we have separated the selection and action components o f a task as discrete elements and thereby removed the requirement that designers need see them as related actions. As a result, users may interact with systems by simply undertaking a never-ending sequence of selections, without any related action being performed. The assumption in the design of the dialogue box in the example is that there are only two possible outcomes---go ahead with the identified selection, or

CONVERSATION ANALYSIS AND llCl 247

cancel. The possibility that there might be something problematic--in this case tile constraints of "moded" operation--which requires some response to the user over and above the two simple options offered is not catered for. The problem arises because in the separation of selection and action, the designer is not obliged to take account of the prior actions of the user as a determinant of the provision of adequate responses.

5. Discussion

The particular focus of the examples in this paper has been on the problem of providing an appropriate response to the user's actions. In the first example, the lack of an appropriate response gave rise to observations regarding the ways in which expectations and processes characteristic of conversation enter into human- computer interaction. In the second example, the lack of an appropriate response was shown to be a result of the separation of selection and action, characteristic of direct manipulation interfaces.

Probably the most substantive and fundamental point that has come from these examples is that the possibility of multiple user selections is simply not catered for by the designer. It is evidently a situation where the user wishes to proceed with some interaction but where the system is unresponsive. In conversation there are mechanisms whereby conversationlists attempt to rectify such situations by, for example, reformulating utterances, and it is possible to envisage ways in which a system might pursue clarification through similar mechanisms. Equally it would be possible for the system to provide features which actively directed the user to employ such mechanisms, particularly in the reformulation of requests. Advice to the designer would thus relate the findings of conversation analysis to spec!fic features of human-computer interaction, and indicate appropriate interactional structures to be incorporated within the design.

These comparatively simple examples provide clear evidence of the need for interface designers to take account of considerations based on the user's view of interaction. Conversation analysis provides a perspective which emphasizes the interactive nature of user's behaviour, and allows investigators access to the interpretations and "interactional logics" of the user which are shaped partly by expectations from conversation.t Moreover, it is clear that any combination of the ethnomethodological "family of practices" used to make sense of interaction: for example the interpretive procedures described by Cicourel (1973), and the abilities for practical reasoning discussed by Garfinkel (1967), may be important factors in user-system interaction. One direction for our future work is to investigate the extent to which the nature of human-computer interaction is shaped by these practices, and the ways in which this may be articulated in order to influence design.

Prospectively, it is possible to see how conversation analysis can inform HCI research and influence HCI design through the development of guidelines. One specific situation identified in this paper is the case where repeated multiple selections are possible without some action be_;ng performed. Whilst it is clear in this situation that the user is in some kind of difficulty, there may be a variety of

"i" A number of theoretical issues regarding the status of conversation analytic findings are addressed in Norman and Thomas (1990).

248 M. A. N O R M A N A N D P. J. T i l O M A S

proposed solutions. One view that the designer may take is to see the problem as trivial, and suggest that what is needed is not more support built into the system, but that users will merely have to learn how to "use the system properly". It is possible to concede that the difficulties that have been identified here are those of a user who has little experience or understanding of the system. Yet it is clear from our studies that the problematic character of interaction is not restricted to novice users. Whilst the "catastrophic" type of problem is largely absent from the expert user-system interaction, there remain the sort of problematic episodes discussed hero. An alternative solution might be for the designer to have provided suitable context information, in the case of the dialogue box, so that a more appropriate course of action was evident to the user. This is of course no more than an ad hoc solution to a problem which could not have been envisaged by the designer prior to the construction of the system. Another solution might be to suggest that an "intelligent system" could infer that the user was in difficulty and provide a suitable form of help: an approach not without substantial problems, particularly those concerned with generating accurate inferences about the meaning of the user's actions. Design guidance provided through conversation analysis stands in contrast to these solutions. In this case the observation that the separation of selection and action components of user contributions to an interaction gives rise to a prediction that there will be problems both where the user receives no adequate response to some contribution, and where the designer permits repeated selections without accompanying actions.

The use of both the methods and findings of conversation ' analysis in the examination of human-computer interaction provides a route into providing designers with ways to understand problems evident in human-computer interac- tion, and to engineer better interaction. The issues that are raised relate to clearly identifiable features of interaction that the designer might draw upon in developing a design for an application. Information about the user's abilities and expectations obtained from conversation analysis can be formulated into guidelines which relate to the structure of interactional sequences. It is not a requirement that each particular situation is identifiable and that guidelines or internal representation be provided for it, but that interaction should conform to expected and familiar formats. The proposed design solution is not to identify this particular difficulty but to rely on the general mechanisms from conversation to guide procedures which will extract the user from the impasse. In this way we can see that it is possible to provide an interpretation of the "rules" of conversation in terms of the features of interactive systems.

The formulation of guidance from conversation analysis may lead to the resolution of the inherent problems of previous design guidelines: their generality means that many are only usable with unwelcome interpretive effort by designers; their specificity results in a lack of applicability (Maguire, 1982; Gould & Lewis, 1985). Guidance derived from conversation analysis may address such difficulties, since it both addresses the structure of interaction in a principled fashion, yet does not prescribe the substantive content of the interaction. The additional claim for such guidelines, as has been noted, would be for their universality, since they are based on users' held-in-common interactional abilities and expectations. Although our work in this respect is still in its early stages, it would seem that it holds considerable promise.

CONVERSATION ANALYSIS AND HCI 249

This paper has sought to establish that an approach employing the methods and established findings of conversation analysis is appropriate for the examination of human-compute r interaction, and can provide routes to inform design practice. Of course, the interactions examined here are merely illustrative, but it has become clear that many aspects of human-compute r interaction can be productively analysed even using relatively small amounts of data. Potentially there are a considerable number of issues concerned with the structure of interaction which will prove amenable to similar investigation. The possibility of employing conversation analytic techniques to identify interactional problems in the interface, and to offer design solutions, would seem a fruitful and methodologically sound course to pursue.

References ATKINSON, J. M. & DREW, P. (1979). Order in Court: The Organisation of Verbal Interaction

in Judicial Settings. London: Macmillan. A'rKINSON, J. M. & HEm-rAGE, J. C. (Eds.) (1984). Structures of Social Interaction: Studies h~

Conversation Analysis. Cambridge, UK: Cambridge University Press. BEA'r'rm, G. (1983). Talk: An Analysis of Speech and Non-Verbal Behaviour. Milton

Keynes: Open University Press. BtrrroN, G. & LEE, R. E. (Eds.) (1987). Talk and Social Organisation. Clevedon, Phil.:

Multilinguai Matters. CICOUREL, A. V. (1973). Cognitive Sociology--Language and Meaning in Social Interaction.

Harmondsworth, UK: Penguin. FAROOO, M. U. & DOr, UNlCK, W. D. (1988). A survey of formal tools and models for

developing user interfaces. International Journal of Man-Machine Studies, 29, 479-496. FOELY, J. D. & WALLACE, V. L. (1974). The art of natural graphic man-machine

conversation. IEEE Proceedhzgs, Special Issue on Computer Graphics, 62, 462-471. GAINES, B. R. & SHAW, M. L. G. (1984). The Art of Computer Conversation: a New

Medhtm for Comnmnication. Englewood Cliffs, N J: Prentice-Hall. GARFINKEL, H. (1967). Studies in Ethnomethodology. Englewood Cliffs, N J: Prentice-Hall. GOODMAN, D. (1987). The Complete HyperCard T M Handbook. New York: Bantam Books. GOODWIN, C. (1981). Conversational Organisation: Interaction Between Speakers and

Hearers. New York: Academic Press. GOULD & Lnwm (1985). Designing for usability: key principles and what designers thinly.

Communications of the ACM, 28, 300-311. HEATH, C. (1982). The display of recipiency: an instance of a sequential relationship in

speech and body movement. Semiotica, 42, 147-167. HERrrAGE, J. C. (1988). Current developments in conversation analysis. In D. ROGER & P.

BULL, Eds. Conversation: An Interdisciplinary Perspective, pp. 9-21. Clevedon, Phil: Multilingual Matters.

JEFFERSON, G. (1973). A case of precision timing in ordinary conversation: overlapped tag-positioned address terms in closing sequences. Semiotica, 9, 47-97.

JE~-nRSON, G. (1987). On exposed and embedded correction in conversation, In G. Bu'rroN & J. R. E. LEE, Eds. Talk attd Social Organization, pp. 86-99. Clevedon, Phil: Multilingual Matters.

JEFFERSON, G. (1988). Preliminary notes on a possible metric which provides for a "standard maximum" silence of approximately one second in conversation. In D. ROGER & P. BULL, Eds. Talk and Social Organization, pp. 166-197. Clevedon, Phil: Multilingual Matters.

MAGUmE, M. (1982). An evaluation of published recommendations on the design of man-computer dialogues. International Journal of Man-Machine Studies, 16~ 237-261.

NXCKERSGN, R. S. (1976). On conversational interaction with computers. In Proceedings of the ACM/SIGGRAPH Workshop, Pittsburg, pp. 101-113.

250 ~t. A. NORMAN AND P. I. TtlOMAS

NORMAN, M. A. & Tllo.,.t,,,s, P. J. (1990). The very idea: informing HCI design from conversation. In P. LuI'F, D. FaOHLtO, & G. N. GtLnnRT, Eds. Computers and Conversation Analysis. London: Academic Press.

OJ~l~, W. D. (Ed.) (1968). Conversational Computers. New York: Wiley. PO.',IERAN'rz, A. M. (1984). Pursuing a response. In J. M. ATKtNSON & J. C. HERtTAGn,

Eds. Structures of Social Interaction: Studies in Conoersation Analysis, pp. 152-163. Cambridge, UK: Cambridge University Press.

SACKS, H., SOIEGLorT, E. & JEFFERSON, G. (1974). A simplest systematics for the organization of turn-taking in conversation. Language, 50, 696-735.

SEAI~LE, J. R. (1970). Speech Acts: An Essay in the Philosophy of Language. Cambridge: Cambridge University Press.

SCHEOLOZ-3;, E. (1968). Sequencing in conversational openings. American Anthropologist, 70, 1075-1095.

SCHEGLOFF, E. (1979). Identification and recognition in telephone conversation openings. In G. PSAt.THAS, Ed. Everyday Language: Studies in Ethnomethodology, pp. 23-78. New York: lrvington.

SCHUGLO~, E. (1984). On Some Questions and Ambiguities in Conversation. In J. M. ATKINSON ~r J. C. HERITAGE, Eds. Structures of Social Interaction: Studies in Conversational Analysis, pp. 28-52. Cambridge, UK: Cambridge University Press.

SCHEGLOFF, E. & SACKS ii. (1973). Opening up closings. Semiotica, 7, 289-327. SCHEOt.O~V, E., SACKS, I I. & JEV~-ERSON, G. (1977). The preference for self-correction in the

organisation of repair in conversation. Language, 53, 361-382. SIIACKEL, B., EASON, K., GARDENER, A. & MCKEIqZIE, J. (1988). Human Factors

Guidelines for the design of Computer-Based Systems. London: I IMSO. SINCLAm, J. MCII. & COtJL-rHARt~, R. M. (1975). Towards an Analysis of Discourse.

London: Oxford University Press. Sr, wrH, S. L. & MosIER, J. N. (1984). Design guidelines for user-system interface software,

Mitre Corporation Research Report MTR-9420. TllOMAs, P. J. (1990). Conversation analysis in bzteractive computer system design, Ph.D.

thesis, University of I lull, UK. THOXIAS, P. J. & NORMAN, M. A. (1990). Interacting with hypertext: functional simplicity

without conversational competence. In C. GIw-Erq, & R. McALt~Esn, Eds. llyperText: Theory into Practice IL Oxford, UK: Intellect.