evaluating three museum installations of a hypertext systemben/papers/brethauer-3... · bischof,...

Evaluating Three Museum Installations of a Hypertext System

Ben Shneiderman,* Dorothy Brethauer, Catherine Plaisant, and Richard Potter Department of Computer Science and Human-Computer Interaction Laboratory, University of Maryland, College Park, MD 20742

This study explores the use of a hypertext system by patrons of three museums. Data on the articles accessed, the time spent in each, the number of times the index was accessed, etc., were collected from a total of more than 5000 sessions. Results show that visitors at all three museums used the embedded menus of the Hyperties hypertext system in moving from one article to another far more than the traditional index. Article selection appears to reflect anticipated interests of patrons at each museum suggesting success in travers- ing the database. At the third museum, the data collection was complemented by direct observation and interviews of the museum patrons. This last approach appears to be more appropriate to analyze the usage data as well as to improve the user interface and database structure.

Introduction

Interactive databases have been widely used in settings such as business offices, airline reservation counters, li- braries, and supermarkets for the purpose of storing, retrieving, and manipulating data. The use of databases for browsing and educational purposes is less common, and research on the use of interactive databases in museums is nonexistent. This article describes a study of Hyperties, an interactive encyclopedia system, in three museum settings.

Hyperties* was developed at the University of Mary- land for use by novices in a museum and other environ- ments. It is a software tool for constructing and reading a database composed of articles containing text and pictures on a given topic (Shneiderman, 1987). Such hypertext systems (Conklin, 1987) have become popular, especially since the recent development of Apple’s HyperCard.

*Hyperties is available from Cognetics Corp., 55 Princeton-Hightstown

Rd., Princeton Junction, NJ 08550, Phone (609) 799-5005.

*Author to whom all correspondence should be addressed.

01989 by John Wiley & Sons, Inc

Within each Hyperties article certain words and phrases, determined by the author of the database, appear in color or boldface. When selected (by using arrow keys, a mouse, or a touchscreen), a brief definition of the word or phrase appears near the bottom of the screen. The reader can then select a FULL ARTICLE on the topic. The goal of the database author is to create logical links between articles so that readers can progress from one to another in a natural manner and according to personal interests. At all times the user has the option of selecting NEXT PAGE, BACK PAGE, RETURN TO [previous article], or INDEX (see Figure 1).

One aim of designers of systems such as Hyperties is to facilitate browsing and information retrieval by novices. In over 20 experiments at the University of Maryland, stu- dent volunteers have generally needed little or no instruction. In a museum setting, however, patrons may have a broader range of backgrounds, abilities, and expectations (Marchionini and Shneiderman, 1988). This study was done to determine how readily museum visitors would use Hyperties and to see whether they would exploit the hypertext feature, as opposed to using the index, in moving from one article to the next. In related studies (Wang et al., 1988), library science students used the index more than 80% of the time and only exploited the embedded menus after gaining familiarity with the system.

At least one study (Young, 1987) has explored how users of an online help facility locate information when given the choice of several methods, such as reading a more general article versus doing a string search for a particular word or phrase or reading a specific article about the topic. It appeared that new users tended to read the more general articles and performed string searches while expert users immediately read specific articles. It is diffi- cult to generalize such results to our study, however, as Hyperties had no “string search” feature at the time of these studies.

This evaluation of Hyperties utilization in three museum settings is divided in two parts. In part 1, a common database was used in two different museums and only quantita- tive data was logged by the computer. In part 2 of the

JOURNAL OF THE AMERICAN SOCIETY FOR INFORMATION SCIENCE. 40(3):172-182, 1989 CCC 0002-8231/89/030172-11$04.00

Chim--an overview PAGE 1 OF 2

David Seymour (1911-1956), known to his friends as "Chim", applied his

skilled hand, warm heart, and perceptive eye to photographing poignant

and dramatic events of the 20th century. Chim's photographs have

appeared in many magazine articles, &c&a, and exhibits, and a number

of memorials have been dedicated to his memory. His photographs of

famous personalities of the 20th century are widely know..

This exhibit "David Seymour 'Cbim' The Early Years 1933-1939

will remain on view at the International Center of Photography

through January 1987. ----------------________________________-------------------------------

BOOKS WITH DAVID SEYMOUR'S PHOTOS - a description of books contaunng photographs taken by David Seymcur

FULL ARTICLE ON BOOKS WITH DAVID SEYMOUR'S PHOTOS NEXT PAGE RETURN TO CAPA, ROBERT INDEX

FIG. 1. This screen shows the first page of the Introduction article used

at the International Center of Photography in New York. The bold items

were selectable by pressing the left and right arrow keys to move a selec-

tor box. The selector box is now resting on the word books and the brief

definition for the referenced article appears on the lower part of the

screen. The user can now select the full article, turn to the next page, re-

turn to the previously read article, or choose the index. The back page

command appears on the screen only when the reader has gone on to the

second page of an article.

evaluation at a third museum, (Smithsonian Institution’s Natural History (SINH)) we used a different database and were able to couple direct observations of patrons with more detailed log data.

1. Log Data Only

The Museums

B’nai B’rith Klutznick Museum (BBK). The B’nai B’rith Klutznick Museum is a well-established but modest- sized museum in downtown Washington, DC. Typical attendance is 150 people per day. The exhibit “Chim: Photographs by David Seymour” occupied one gallery of the museum from May to November 1986. A specially built painted wood cabinet housing an IBM PC-XT computer was situated across from the entrance to the gallery, behind a freestanding partition that held exhibit materials. The monochrome monitor was at eye level for a seated person, and three chairs were provided. The keyboard was covered by a clear plexiglass plate with holes punched in it to allow users to press the left and right arrow keys and the RETURN key. A silk-screened plexiglass label was at- tached with brief instructions.

International Center of Photography (ICP). The In- ternational Center of Photography is a 14-year-old museum located on upper Fifth Avenue in New York City. Typical attendance is 400 people per day. The exhibit “David Sey- mour ‘Chim’ the Early Years 1933-1939” was on display at the ICP from November 1986 to January 1987. An elegantly fashioned freestanding podium that housed an IBM

PC-XT computer was installed near the gallery’s center. The monitor was at about waist level at a 45 degree angle for comfortable viewing while standing up. The sturdy podium enabled viewers to rest their arms or elbows while viewing the CGA monitor. A wooden plate completely covered the keyboard. The arrow and RETURN keys were manipulated by pressing wooden pegs that protruded from holes in the plate and rested on the keys below.

The Databases

The databases used at both museums were entitled “David Seymour: Chim” by Jeremy Symington, and consisted of 26 and 30 articles at BBK and ICP, respectively. The screen in Figure 2 was displayed on the monitor when the database was not in use. The index (here an alphabetical list of the article titles) is shown in Figure 3. Articles were from 1 to 8 screens long, some single and others double spaced, according to author preferences.

The INTRODUCTION article gave an overview to the topics and included highlighted references to key articles. The contents included information about David Seymour’s life including a time-line of his accomplishments, the exhibit, the host museum, related photographers, and various testimonials.

Data Collection

The computer automatically recorded the time at which each session began, the name of each article accessed, and

JOURNAL OF THE AMERICAN SOCIETY FOR INFORMATION SCIENCE-May 1989 173

DAVID SEYMOUR: CHIM edited by Jeremy Symington

You can learn more about David Seymour by reading article on this computer. Move the blue highlight from yellow word to vellow word by pressing the left or right arrow keys. Then, to select the word in blue highlights, press the ENTER key.

-------------------_____________________-------------------------------

FIG. 2. The title page seen by visitors at the ICP museum. A silk-

screened plexiglass note with instructions for use was pasted to the cabi-

net at the BBK. The instructions were slightly different because the monitor was monochrome at the BBK.

INDEX OF ARTICLES Startins index entry 1 out of 26

1. 2. 3. 4. 5. 6. 7. 8. 9.

10. 11. 12. 13. 14. 15.

INTRODUCTION: CHIN--AN OVERVIEW

ARTICLES ABOUT DAVID SEYMOUR B'NAI B'RITH KLUTZNICK MUSEUM HISTORY B'NAI B'RITH KLUTZNICK MUSEUM OVERVIEW BISCHOF, WERNER BOOKS WITH DAVID SEYMOUR'S PHOTOS CAPA, CORNELL CAPA, ROBERT CARTIER-BRESSON, HENRI CHIM-- AN OVERVIEW CHIM: PHOTOGRAPHS BY DAVID SEYMOUR ~1911-19561 CHRONOLOGY OF DAVID SEYMOUR'S LIFE EXHIBITS WITH DAVID SEYMOUR'S WORK HAAS, ERNST INTERNATIONAL CENTER OF PHOTOGRAPHY ISRAEL

32 54 40 9

28 14 18 33

490 0

30 20 10 31

117

(Second screen of mdex)

16 17 18 19 20 21 22 23 24 25 26

MAGNUM PHOTOS MEMORIALS IN DAVID SEYMOUR'S HONOR NEWSWEEK'S TRIBUTE TO CHIM PERMANENT COLLECTIONS OF DAVID SEYMOUR'S WORK PERSONALITIES PHOTOGRAPHED BY DAVID SEYMOUR QUOTATIONS ABOUT DAVID SEYMOUR RODGER, GEORGE SEYMOUR, DAVID SHNEIDEFtMAN, EILEEN SPANISH CIVIL WAR VANDIVERT, WILLIAM

FIG. 3a. Index for BBK database with total number of accesses for

32 10 13 I

50 38 2

55 26 33 12

each article.

the number of seconds spent in each article. Index accesses were also recorded, but the time spent in the index was not. If the user did not execute a QUIT command at the end of a session, the screen reverted to the title page after five minutes of inactivity, and a “timed out” message was stored in the data for the last article.

Results

Data from 373 sessions at BBK and 361 sessions at ICP were recorded and analyzed. The data included:

Nonarticle-Specific Data:

l Articles per session --the number of articles accessed during one session (index accesses were not counted).

l Zndex accesses per session -the number of times the index was accessed during one session, excluding any access just prior to the end of the session, since such an access was necessary to execute the QUIT command.

l Articles read per index access -Average number of articles read per session divided by average number of index accesses per session.

174 JOURNAL OF THE AMERICAN SOCIETY FOR INFORMATION SCIENCE-May 1989

INDEX OF ARTICLES Starting index entry 1 cut of 30 INTRODUCTION: CHIM--AN OVERVIEW

1. 2. 3. 4. 5. 6. 7. 8. 9.

10. 11. 12. 13. 14. 15.

ARTICLES ABOUT DAVID SEYMOUR BISCHOF, WERNER BOOKS WITH DAVID SEYMOUR'S PHOTOS CAPA, CORNELL CAPA, ROBERT CARTIER-BRESSON, HENRI CHIM, A MAN OF PEACE CHIM, A MAN OF PEACE: PART 2 CHIM-- AN OVERVIEW CHRONOLOGY OF DAVID SEYMOUR'S LIFE DAVID SEYMOUR 'CHIM' THE EARLY YEARS 1933-1939 EXHIBITS WITH DAVID SEYMOUR'S W0P.K ICP'S BROAD RANGE OF PROGRAMS ICP/MIDTOWN INTERNATIONAL CENTER OF PHOTOGRAPHY

6econd screen of indeXJ

49 21 26 52

112 93 74 22

551 71 45 12 48 74

157

16. 17. 18. 19. 20. 21. 22. 23. 24. 25. 26. 27. 28. 29. 30.

ISRAEL 47 MAGNUM PHOTOS 72 MEMORIALS IN DAVID SEYMOUR'S HONOR 13 NEWSWEEK'S TRIBUTE TO CHIM 14 PERMANENT COLLECTIONS OF DAVID SEYMOUR'S WORK 11 PERSONALITIES PHOTOGRAPHED BY DAVID SEYMOUR 75 PHILLIPS, CHRISTOPHER 12 QUOTATIONS ABOUT DAVID SEYMOUR 57 REGARDS MAGAZINE 18 RODGER, GEORGE 15 SEYMOUR, DAVID 168 SHNEIDERMAN, EILEEN 49 SPANISH CIVIL WAR 44 TARO, GERDA 39 VANDIVERT, WILLIAM 26

FIG. 3b. Index for ICP database with total number of accesses for each article.

Duration of session -the sum of the time spent in each article for which a valid time value had been recorded. In sessions that were “abandoned” the time for the last article was not included. Percentage of sessions abandoned -the proportion of sessions not terminated by a QUIT command. Average time per article -the sum of all time values recorded divided by the numbers of articles from which the time values were obtained (last articles of aban-

doned sessions ignored).

l Corrected duration -the duration of a session plus the average time per article for each article whose time value was not included in the duration.

Figures 4-7 show the frequency distribution for key data items at each museum. The average length of a session was slightly more than two minutes at BBK and approximately six minutes at ICP. The average number of articles accessed per session was 3.2 at BBK and 5.7 at

Percent of sessions with given number 01 articles per session

n BE3K q ICP q SINH

40

35

30

25

20

15

10

5

0 ‘1 2-3 4-6 7 or more

FIG. 4. A histogram showing the percent of sessions that accessed the

given number of articles in the session at all three museums. Visitors at

ICP viewed more articles on the average than visitors at BBK. Visitors at

SINH viewed even more articles on the average than the visitors to ICP.


Percent of sessions with given number of index accesses per session

3 or more

FIG. 5. A histogram showing the percent of sessions with the given

number of index accesses per session. ICP visitors used the index more

per session than BBK visitors, possibly reflecting deeper interest in the

photographers or more sophistication with computer usage. SINH visitors

used the index less per session, possibly reflecting the difficulty in mak-

ing good use of the much larger index.

Percent of sessions with

average time per article (seconds)

IfEK ICP SINH

100

80

60

40

0 set 1-29 30-59 60-119 120 01 more

FIG. 6. A histogram of the percent of sessions with the average time

per article in seconds. Visitors at ICP spent an average of 64 seconds per

article versus 41 seconds at BBK. Visitors at SINH spent an average of

16 seconds per article, possibly reflecting the ease of traversal with the

touchscreen, the greater connectivity, and the brevity of the articles.

Percent of sessions with

duration of session (seconds)

30

25 20 15

10 5 0

0 set 1-59 60-119 120-239 240-479 480 or more

FIG. 7. A histogram showing the percent of sessions with the given du-

ration of the session in seconds. Visitors at ICP spent an average of approximately six minutes per session versus just over two minutes at BBK. Visitors at SINH spent an average of 2.6 minutes per session. We conjec-

ture that this reflects the ease of exploration with the touchscreen that en-

abled users to quickly explore a few articles (notice the brief times spent

per article in Figure 6) and satisfy their information needs or their sense

of the system. Also the SINH crowds may have encouraged people to

move on quickly.

ICP. This difference may reflect the differing interest of the museum patrons or their skill with computers. The average time spent in each article was approximately 41 seconds at BBK and 64 seconds at ICP. We assume that the greater persistence of ICP visitors reflects a different profile of usage. Forty-seven percent of the sessions at BBK were abandoned, compared to 39% of the sessions at ICP. Articles read per index access was similar for the two museums. Note: Eight sessions were not included in the above analysis (seven at BBK and one at ICP) because they consisted of one INDEX access (no articles were read).

Article-Specific Data:

First access -The proportion of sessions beginning with the index and the proportion beginning with the introduction.

Second article in session -The proportion of sessions (of those containing two or more items) that had each item as their second member. Second article in sessions beginning with the introduction -The proportion of sessions, of those containing two or more items and beginning with the introduction, that had each item as their second member. Second article in sessions beginning with the index - The proportion of sessions, of those containing two or more items and beginning with the index, that had each item as their second member. Third article in all sessions -The proportion of sessions, of those containing three or more items, that had each item as their third member. All articles in all sessions -The proportion of total accesses accounted for by each item.

The first item in 80% of the sessions at ICP and in 84% of the sessions at BBK was the introduction (lead article) as opposed to the index. This contrasts with the results in Wang et al. (1988) who found that graduate library students answering fact questions used the index more than 80% of the time. At both museums, approximately 50% of sessions beginning with the introduction had the index as their second item. The second and third most popular items consisted of the same two articles at both museums.

In those sessions in which the index was selected first, the most popular article at the BBK was “Israel,” while the most popular article at ICP was “Chim- an overview” (the introduction). The total accesses per article in all sessions also reflects heavy use of the index and introduction (Figures 3 and 8).

Discussion

Based on the comments of curators at both museums, Hyperties was well-received by visitors. Of course, these comments are biased and interviews with visitors should be used in future studies. As designers of the software we were interested in suggestions from visitors, curators, and colleagues to improve the system, but the data logging turned out to be insufficient for this purpose. Nevertheless


Percent of access with

Ten popular amcles

30

25

20

15

1c

5

0

Index Lead A23 Al4 A7 A8 A20 Al6 All A21

arllcle

FIG. 8. A histogram showing the percent of accesses for the index and

the most popular articles that were included in the databases at both BBK

and ICP. The numbers refer to the article numbers as shown in the BBK

index in Figure 3a. The BBK visitors viewed the lead article often and

did not explore as widely as the ICP visitors.

we identified some ideas that might produce greater in- volvement and satisfaction. The lack of instructions on the title page of the BBK database may have contributed to the short duration of many sessions there. In addition, both databases lacked instructions on how to quit. Users had to discover the QUIT commands on the index screen to ter- minate the session. The authors conjecture that the longer sessions at the ICP were due to a more knowledgeable and interested user population, both with respect to the subject matter of the database and to computers in general. The ICP users also spent more time reading each article and explored a wider range of topics.

Both user groups utilized the embedded menu feature of Hyperties more frequently than the index, as evidenced by the number of articles read per index access. In addition, the distinctive distribution of articles accessed at the two museums seems to indicate that users selected articles based on their perceived content, rather than at random. Visitors at BBK, for example, were more likely to read the articles on Israel and on the museum itself, while visitors at ICP were more likely to read articles about the various photographers profiled in the database.

The problem of “abandoned” sessions could have been reduced by adding QUIT to the commands that appear at the bottom of each screen when an article is displayed, and by mentioning the quit procedure in the instructions on the title page.

There are a number of pitfalls that must be avoided if one wishes to use the data collected in this study to evalu- ate a particular database. The average duration of a session, for instance, is not reliable since there is no way of knowing whether one session was the work of one individ- ual or of a series of people. The heavier volume of visitors at ICP may have accounted for part of the longer duration of the sessions.

The proportion of abandoned sessions may only be an indication of the clarity of the instructions. Average time per article is a more important measure of the utility or ef- ficacy of the system, and the proportion of total accesses

accounted for by each article should give a good indication of which articles were most attractive to users.

2. Direct Observation and Log Data

The Museum

The third museum is the well known Smithsonian Insti- tution’s National Museum of Natural History. Two IBM Personal Computers are installed in the final room of the exhibit on “King Herod’s Dream”-the ancient Roman port city of Caesarea, located in Israel. The exhibit focuses on the rise of urbanism in ancient times and the archaeological methods used during the past 20 years of excava- tions (Holum, 1988). This last station of the exhibit invites visitors to learn more about the sites that welcome volunteers and how to join a dig. This exhibit organized by the Smithsonian Institution Traveling Exhibition Service was in Washington DC from March 22 to June 19, 1988. It is now traveling to five other museums for the next two years (Los Angeles, Denver, St. Paul, Boston, and Ottawa). Two freestanding podiums house the IBM PC-AT computers. The EGA monitors, each equipped with a touchscreen, are at about waist level at a 45degree angle. There is no keyboard. The only permanent instruction is “Touch the screen,” written above the monitor.

The Database

The GOVA (Guide to Opportunities in Volunteer Ar- chaeology) database was developed under the direction of Ken Holum (Professor of History at the University of Mary- land). It consists of about 200 articles. The contents in- clude information about archaeological digs taking place in Israel, the United States, and around the world, and practical questions about how to join a dig. A special effort was made to cover the local sites near the current exhibiting museum, and the database will be updated during its trav- els. The archaeological sites are organized both geographi-


tally and by historical periods. The information can also be accessed by direct selection on 11 maps.

The Touch-Screen Interface

In this exhibit a touch-screen interface is used. The museum patrons need only touch highlighted words on the screen to see the related article. A standard strategy indi- cates the touchable items: in the text as well as the maps only the blue words are selectable. This third study offered an opportunity to test the “touch-mouse” strategy developed at the Human-Computer Interaction Laboratory (Pot- ter et al., 1988). In this strategy users receive feedback about the exact touch position by a cursor just above their fingers. When the cursor is on a selectable area, the area is highlighted. The selection is activated when users lift their fingers from the screen. This method produces low error rates and high user satisfaction.

Observations in the Museum-The Effect on the User

Inte$ace Design.

In this study the authors observed museum patrons using our hypertext system and the touchscreen interface in a natural environment. The main goal of this pilot study was to verify the accessibility of the information. Were patrons really able to explore the database with ease and satisfaction? The Hyperties concepts with the touchscreen interface had been thoroughly tested in the laboratory, and it was clear that the system was easy to use with about 15 seconds of demonstration or explanation, but we had less experience with an unattended public access situation.

Three observers spent four sessions of about two hours each at the exhibit, observing and discussing three poten- tial problems: the touchscreen interface, the Hyperties mechanism, and the database structure. Each session allowed the authors to pick out the weaknesses of the system and put in place a productive mechanism of criticism and modification back and forth between laboratory and museum.

The first observation was that the suppression of our 15- second demonstration was transforming an easy to use system into a frustrating mystery game for some of these totally untutored users. The time spent at the computer shouldn’t have been spent guessing and missing. Three main problems were observed: the misselection rate was still very high (the screen is touched but nothing is selected), the few lines of instructions were not clear or accessible enough to get started, and many patrons didn’t figure out the two steps required to get a new article (first select a word and get a definition, then select its full article).

During the first sessions no usage data were logged by the computer. The data would have been able to show the average very small number of articles accessed and the high number of misselections, but the simple observation of the patrons and some informal interviews were suffi- cient to give us the hints necessary to understand why the

system wasn’t working as hoped and how to improve it. In fact, direct observation and interviews worked better to improve the system than the data collection in part 1 of this study.

After implementing the changes described below, an obvious and satisfactory improvement was seen. Most of the patrons were able to traverse smoothly from article to article and focus on their reading and not on the mechanism. The authors were then able to concentrate on improving the structure of the database (contents and links) to make the information more accessible.

About the touchscreen. A surprisingly wide variety of ways to touch a screen were observed! Some users pressed hard or rubbed the screen, some touched with three fingers, some swished elegantly with a fingernail, etc. In spite of our efforts to design the screens clearly, there was confusion about where to touch. In this Hyperties applica- tion all the touchable areas were words written in blue, an easy concept only if you have read the instructions. To re- duce the user’s confusion on both of these issues, the authors provided better instructions and made the touch selection mechanism more forgiving. The initial selection mechanism was very accurate and easy to use with a few seconds of explanation. For the museum purpose, the touchable zones as well as the cursor, were enlarged and we installed a help line at the bottom of the screen, continuously updated according to the current step of the selection.

About the instructions. Written paper instructions at- tached on the side of the screens are not esthetically attractive. To give directions in the introduction article of the database seemed more appropriate. Therefore it has to be accessible to the patron coming to the screen. First, the RESTART button was made available at anytime, well separated from the others to avoid accidental hits. The time-out delay was also shorter than during the first installations. Modifying the introduction article and immediately observing the reaction of the patrons allowed us to find a more successful writing style. Patrons were more successful when they were asked to try specific examples than to read about the general hypertext concept. It is better to write “For example, if you touch the blue word “volunteer” on the third line of this page, you will get an article about volunteers. . . ” , than to explain that the blue words will give you more information by accessing an article about the related subject. Our observations showed the im- portance of the introduction article. It should give clear instructions, a brief overview of the main database topics, and also easy and clear links to the most attractive parts of the database (maps for example) (Fig. 9).

About the Hyperties mechanism. In the initial system the user had to first select a word, then select the command FULL ARTICLE to jump to a new article. This mechanism had been designed for a keyboard interface with which users simply pressed the Return-Key twice to get an article-thanks to the judicious setting of the de- fault cursor position. In the touchscreen version that mechanism doubles the selection process time and difficulty.


GOVA - Introduction PAGE 1 OF 2

Welcome to GOVA, the Guide to Opportunities in Volunteer Archaeology.

GOVA will tell you about archaeological projects all over the world

(see Hap 9) that welcomes volunteers. You can use GOVA to answer

practical questions like how to join a dig, or how much it coete.

TO Use GOVA, simply touch a blue word with your fingertip-- not

your fingernail-- and release. For example, if you touch the blue word

'volunteers" at the third line of this page, you will get an article abou:

volunteers. TO turn to the next page touch the blue ward NEXT PAGE. _________-----____--------------------------------------------------------

INDEX NEXT PAGE

RE!lWP.NTOMAP 9, THEWORLD RESTART

Touch a blue word with your fingertIp

FIG. 9. This screen shows the first page of the Introduction article used

at the National Museum of Natural History. The bold items appeared blue

on the screen and were the touchable areas. Here, the brief definition

space at the bottom of the screen has been removed. The RESTART is

available at anytime. The style of writing for the introduction article is

also different. It gives instructions and examples, and provides the links

to the major database topics. In a later version we added a TABLE OF

TOPICS command at the bottom of the screen.

The FULL ARTICLE command was also hard to reach in a corner of the screen and our interviews revealed that many patrons were annoyed by it. The solution was to sup- press the definition and give direct access to the full article. Each selection had now a very visible and dramatic effect, the trade-off being a weaker continuity from screen to screen.

About the contents of the database. After making the software changes, it was possible to check the database structure and accessibility. To find information the user can follow links from article to article, or select an article from an index, which consisted here of an alphabetical list of the 200 articles. The authors observed that users seemed to be disoriented by the very large index and often left the index without selecting anything. In the previous experi- ment the index was used often. The hypothesis is that in the first database the index was small enough (two pages) to be browsed rapidly, and give the user the possibility of forming a quick idea of the topics covered by the database. With a large number of articles the index was mainly dis- appointing, except for some people searching for a specific city. It is believed that a table of contents will be more useful. The authors have made the table of contents accessible as an option from any page, but we won’t get any feedback from this change until the exhibit is installed in the next museum-Los Angeles. The data analyzed in the next chapter was also recorded before this change.

Different Types of Use

During our stay at the museum we observed quite diverse behaviors. By making the browsing extremely simple

and rapid, it was also observed that it can become an attractive game for young children who enjoyed their power over the rapid flashing of the screen. Some other patrons come to the computer driven by their interest in new tech- nology (the touchscreen interface or the software). The authors saw them study the software, “play” with the selection mechanism, but not always take interest in the information available. A lot of visitors just stop in front of the screen, touch it once or twice and leave. They don’t seem to be interested or are exhausted- the computer is the last station of a long exhibit. On the other hand, some patrons were observed spending a long time at the computer- or even coming for a second time! -to explore several sites, take notes of names and addresses.

The Data Collection and Analysis

After improving the usability of the system, usage data was collected during four weeks. Unfortunately one of the two monitors was out of order at that time and only one of the two computers was used. All selections were recorded, allowing a more precise analysis. The RESTART command being available at all times and the time-out delay being reduced, most of the patrons started their exploration from the title page. Unlike for the Part 1 data, the authors know that each session has most likely been conducted by the same person or group of persons.

In order to collect the most complete information possible for the GOVA exhibit, every selection and the selection’s time were logged by the computer. Since all of the user input to the computer was captured, any information about what the museum patron saw on the display could


be recreated. The authors chose to concentrate on the time spent within articles. To get this information the log files were “played through” the museum software, recreat- ing the visitor’s experience. At the same time a new con- densed log file was generated that contained the pertinent information.

There were 27 days of data collected which contained 4461 sessions. A session in the GOVA exhibit starts when patrons touch the title page. The session ends either after 60 seconds of inactivity (time-out) or when patrons select RESTART. In GOVA, RESTART appears on every screen except the title page. Some sessions could not be included in the analysis. There were 143 sessions that for some rea- son would not replay through the museum software. The last session for each day was excluded because of the effect of the museum closing and power being cut off from the machine. Because of an oversight, if the user started GOVA by touching the screen in a certain spot, the RE- START on the next page would be selected causing a very short unrepresentative session. This oversight eliminated 730 sessions from analysis. The following results are from the remaining 3561 sessions.

In the average session, patrons looked at 8.5 articles and referenced the index 1.3 times. Thus 6.6 articles were looked at per index reference. The average time per article was 16.4 seconds. The average session lasted 162.2 seconds (2.6 minutes). However 16% of the sessions restarted automatically after timing out (after no user activity for

60 seconds), so the time spent looking at the computer during the time-out period is uncertain. Assuming they spent the 16.4 seconds from the result above, we can in- clude this value into those timed-out sessions and conclude that the average session lasted 165.9 seconds (See Fig- ures 4-7 for histograms).

Of particular interest is the relatively infrequent use of the index. This can be attributed to the effective hypertext writing that encouraged using embedded references. GOVA was written from scratch for the hypertext format. Also, the index was long and arranged strictly alphabeti- cally, not by topic. Patrons would have to have a very specific interest to use the index effectively.

Data on the number of accesses for each article were also compiled. Accesses were determined by two criteria: session accesses were the number of sessions that accessed the article. If an article was accessed several times by one session, it counted as only one session access. Total access counted every time the article was visited. In addition, these two criteria were further subdivided according to where the patron was when accessing the article. Three places were identified: any place, an article, and the index.

There were eight different measures computed for each article. Sorting on each of these measures showed interest- ing results. The articles embedded in the Introduction page topped the list that was sorted by session accesses from any place. One would expect this result since these are pragmatically the most accessible articles. There are two

TABLE 1. This table shows the top 20 articles sorted by the number of sessions that

looked at each article. Underlined articles appear on the introductory page and could be

accessed with one selection. A strong interest in the sites local to the Washington, DC

are as shown in this list. All these numbers consider more than one visit during one session as one access.

Number

of

sessions Accesses Average

that from Accesses time per

accessed another from visit

article article index (seconds)

Introduction 3561 483 259 31

Index of Articles 1802 1802 0 45

Archaeological Projects 1379 1213 310 21

Volunteers 1022 1004 31 24

costs 773 737 45 28

Age and Health Requirements 658 390 303 26

How to Join a Dig 603 549 68 25

Accommodations 487 388 115 25

MAP 9: The World 447 431 30 27

Periods of the past 369 368 6 15

Local sites 334 327 11 14

Academic credit 329 215 132 13

Excavation season 321 306 18 13

Baltimore 304 148 216 23

Annapolis 293 135 189 24

Mediterranean sites 292 288 6 15

Sites in the U.S. and Canada 290 278 16 15

New World Sites 270 265 11 16

Biblical archaeology 262 220 52 19

Special skills 262 258 5 20


peculiar notes though. The article PRACTICAL QUES- TIONS was by far the least popular article of this group, not even appearing with the others in the top 20. The MAP OF THE WORLD, ninth in overall popularity, was the second least popular of this group, which is surprising since patrons can get an overall view of archaeological digs around the world from this eye-pleasing graphic. It appears that it takes a reference with specific information to get the patron’s attention. A user would rather read about ARCHAEOLOGICAL PROJECTS than read about PRACTICAL QUESTIONS (Table 1).

Sorting by session access from the index revealed that the index was too long. All the top 20 articles selected by this method started with ‘a’, ‘b’, or ‘c’ except HOW TO JOIN A DIG and a map showing LOCAL SITES. The strong interest in digs in the local area was shown by most of these sorted lists. Counting by total accesses did not generate results much different from session accesses.

Sorting by the total time spent in the article revealed that while the maps were not accessed relatively often, time was spent with them once they were found. This may be a distorted result since the time for the map to display itself on the screen was longer than for an article.

The total time spent in each article was determined. To better judge which articles caught the patron’s attention, the total time for each article was divided by the number of sessions in which the article was actually inspected. Sort- ing by this result gave an entirely different view on the ar-

ticles. Certain articles obviously required indepth reading. While five maps made this top 20 list, the map of THE WORLD did not. Except for two articles that started with ‘a’ and ‘c’, embedded references were by far the preferred way to find these articles. For example, the article on the HELLENISTIC PERIOD was accessed 61 times from another article and only five times from the index (Table 2).

Conclusions

For Museum Exhibit Designers

This study demonstrates the feasibility of using interactive databases in conjunction with museum exhibits. Hy- perties is not perfect and there is a great deal more to be done to understand how to make successful applications, but the central message is that computers can contribute to an exhibit by offering an engaging, interactive, and personal experience. While only a fraction of the visitors tried the computer, many of those that did succeeded in travers- ing the articles and index. This study also illustrates the wide differences in database use by different visitor popu- lations in terms of topics selected and willingness to invest time. Exhibit designers can add a computer database to provide detailed information, vary the level of presenta- tion, offer multiple languages, or permit patrons to explore vast databases.

TABLE 2. This table shows the top 20 articles sorted by the average number of sec-

onds that each session that visited the article spent there. The authors conjecture that

these are the articles that the patrons stopped to read. Except for articles that were lo-

cated at the start of the index, these articles were mostly accessed from other articles.

All these numbers consider more than one visit during one session as one access.

Nabataean fine ware

Index of articles

Map 1: Northern Israel

Tel Dor

Caesarea’s harbor

Map 2: Southern Israel

Petra

Map 11: Local sites

Hellenistic period

Introduction

Map 5: Britain and Ireland

Map 6: Europe

Ashkelon

Life on a dig

Mount Ebal North Straiton

Published listings

Map 3: Middle East

costs

Kalllak

Number

of

sessions

that

accessed

article

1

1802

180

18

32

136

49

162

64

3561

157

102

68

145

31 42

200

152

773

86

Accesses

from

another

article

1

1802

171

15

16

120

40

130

61

483

145

90

24

127

29 40

194

137

737

79

Accesses

from

index

0

0

21 3

18

19

10

46 5

259

24

22

58

21

2 2

10

28

45 8

Average

time per

visit

(seconds)

62

45

43

41

37

34

33

32

32

31

31

30

30

30

29 29

28

28

28

28


For Researchers Interested in Methodological Issues

Although log data was useful in creating some portrait of usage patterns, the direct observation and rapid iterative refinement led to the most dramatic increases in usability of our system in the museum environment. While the authors were tempted to proceed more slowly and make in- cremental changes while collecting data, this process would have taken many months. Dozens of changes were made in a few weeks and although no statistical data is available to confirm the conjectures, the authors feel quite confident that the system was dramatically improved by their efforts at the Smithsonian.

In future studies, the authors encourage more data collection about who the users were. Age, gender, interests, goals in using the computer, previous interests with the database topic, and previous experience with computers would be worthwhile information to correlate with usage patterns. Converse’s recent study of a shopping mall directory (Converse et al., 1987), found that certain user groups (the elderly) avoided the system, and 20% of users failed to get the information they desired. Such statistics are important in evaluating the effectiveness of a computer system. A second important source of information is post- usage interviews to ascertain whether the users succeeded in retrieving desired information and to determine their subjective satisfaction with the search process. Observa- tions in the third museum had a great effectiveness on improving the user interface and the structure of the database but were still insufficient to analyze search strategies. The authors cannot judge whether two or six minutes with the system is a successful use; only subjective comments can indicate attitudes. However, there now exists a baseline of usage data against which to judge future installations.

For User Inter$ace Designers

The museum environment is very tough to design for since the users are so diverse and even 15 seconds of train- ing may be unrealistic. The initial steps to take should be obvious to the patrons so that they become engaged and experience success. Then the user should be presented with concise information about the database contents, the metaphors for usage, and the syntax for action. The touch-

screen appears to be a convenient and reliable device and the take-off strategy is effective if care is taken to provide a comfortable physical installation, visible feedback about the current touch position, and reasonably sized targets. Instructions should be specific and results should be achieved with a small number of steps. The database structure does make a dramatic difference, and overviews in the form of a table of contents are necessary to complement a lengthy index. In spite of the best efforts in design, testing with real users is still necessary. However, testing need not be lengthy or burdensome. Four to six hours should be enough to give an indication of the problems and successes that patrons have as well as provide ample suggestions for improvement.

Acknowledgments

The authors greatly appreciate the cooperation of the museum staffs at each location, especially Frederica Adel- man of the BBK, Miles Barth of the ICP, and Myriam Springuel of the Smithsonian Institution.

References

1.

2.

3.

4.

5.

6.

7.

8.

Conklin, J. (1987). Hypertext: An introduction and survey, IEEE Computer 20, 17-41. Converse, Sharolyn, et al. (1987). Where can I find. ? An evalu-

ation of a computerized directory information system. Unpublished

manuscript. Holum, K. (1988). Reliving King’s Herod’s Dream. Archaeology, 88,

44-47. Marchionini, G., & Shneiderman, B. (1988). Finding facts vs. brows-

ing knowledge in hypertext systems. IEEE Computer 21, 70-79.

Potter, R., Shneiderman, B., & Weldon, L. (1988). Improving the accuracy of touch-screens: an experimental evaluation of three strate-

gies. Proceedings of the ACM SIGCHI Conference on Human Factors

in Computing Systems, 27-30.

Shneidennan, B. (1987). User interface design for the Hyperties elec-

tronic encyclopedia. Proceedings, Hypertext ‘87 Workshop, (pp. 199-

204), University of North Carolina, Department of Computer Sci-

ence, Chapel Hill, NC, November 13-15.

Wang, X., Liehscher, P., & Marchionini, G. (1988). Improving infor-

mation seeking performance in Hypertext: roles of display format and

search strategy. (Report No. CAR-TR-353 CS-TR-2006.). Human-

Computer Interaction Laboratory: University of Maryland, College

Park. Young, E., & Walker, J. H. (1987). A case study of using a manual

online, unpublished manuscript.


evaluating three museum installations of a hypertext systemben/papers/brethauer-3... · bischof,...

Documents