an evaluation of a “time tunnel” display format for the...

INTRODUCTION

Practitioners in virtually all domains of appli-cation need to be concerned with changes inresources, properties, and variables over time.The term temporal information will be used to re-fer to the historical trace of these changes. Theprimary benefit of temporal information is that“trends” are revealed (i.e., the fact that the valueof variables have changed in particular ways, asopposed to others). It is difficult to imagine adomain in which temporal information is not atleast potentially useful. For example, it is poten-tially useful when the behavior of the system isdriven primarily by the goals and intentions ofits users (e.g., to illustrate trends in the stockmarket). However, temporal information shouldbe especially useful for domains in which thebehavior of the system is driven by the laws ofnature (e.g., process control). In these domainspast system states will determine current and

future system states, at least under normal cir-cumstances. Therefore temporal informationshould be very useful in understanding currentsystem states, predicting future system states,choosing appropriate control inputs, and de-tecting the presence of system faults.

Temporal information has traditionally beenpresented in trend or strip chart displays, whichplot changes in the value of a variable or a re-source as a function of time. These formats havebeen demonstrated to facilitate the detection oftrends with the static presentation of data (e.g.,Schutz, 1961). Despite their intuitive appealand widespread use, there has been surprising-ly little research conducted on trend displays indynamic settings. At least one study has pro-duced negative findings (Spenkelink, 1990).From a theoretical perspective, one potentialdrawback to trend displays is that they are essen-tially “separable” in nature: Each variable hasits own unique representation. As a result the

An Evaluation of a “Time Tunnel” Display Format for thePresentation of Temporal Information

Kevin B. Bennett, Michael Payne, and Brett Walters, Wright State University, Dayton,Ohio

The time tunnel display design technique combines the benefits of configural dis-plays (salient visual properties corresponding to critical domain semantics) withthe benefits of temporal information (i.e., the value of variables and propertiesover time). In Experiment 1 a baseline configural display and a time tunnel displaywere evaluated using real-time measures of system control, fault detection, and stateestimation in a simulated process control task. The results provided little evidencein support of the time tunnel format. In Experiment 2 access to the temporal con-text was limited: Participants performed the detection and estimation tasks withstatic “snapshots” of system states that had been generated in Experiment 1. Theoverall pattern of results indicates that the time tunnel display was more effectivefor state estimation tasks than was the baseline configural display and or a trend dis-play. Issues in the design of temporal displays are discussed, including representa-tional formats and the choice of temporal time frames. Issues in the evaluation oftemporal displays are also discussed, including the role of temporal information andthe critical nature of participants’ access to this information. Actual or potentialapplications of this research include design techniques for improving graphicaldisplays and methodological insights to guide future evaluations.

Address correspondence to Kevin B. Bennett, Department of Psychology, 335 Fawcett Hall, Wright State University,Dayton, OH 45435; [email protected]. HUMAN FACTORS, Vol. 47, No. 2, Summer 2005, pp. 342–359. Copy-right © 2005, Human Factors and Ergonomics Society. All rights reserved.

Reprinted with permission from Human Factors, Vol. 47, No.2, 2005. Copyright 2005by the Human Factors and Ergonomics Society. All rights reserved.

TIME TUNNELS 343

relationships between variables are not empha-sized or, at least, are not emphasized to the ex-tent that is possible with alternative displayformats. Because these relationships are criticalin complex, dynamic domains this limitation isa potentially serious one.

Configural displays explicitly emphasize therelationships between variables. Individual var-iables are arranged in spatial patterns (oftenconnected with contour lines) to produce geo-metrical forms that change shape as a functionof changes in the value of these variables. Thesalient, high-level visual properties that are pro-duced (e.g., symmetry) are usually referred to as“emergent features” (Pomerantz, 1986). A sub-stantial body of laboratory research indicatesthat configural displays can be effective whenthe consideration of relationships between var-iables is essential to the completion of domaintasks (Bennett & Flach, 1992). The degree ofsuccess is determined by the quality of the map-ping between the visual properties of the displayand the physical, functional, and goal-relatedproperties of the domain (Bennett, Nagy, &Flach, 1997).

An example of a configural display is present-ed in Figure 1a (based on the “funnel” metaphorof Vicente, 1991). This display presents threevariables that are critical to a process controltask, the manual control of feedwater. The levelof coolant in a steam generator (indicated level)is plotted as a vertical bar graph on the leftside of the display. The flow rate of mass enter-ing the steam generator (feed flow) is plottedas a horizontal bar graph at the top of the dis-play. The flow rate of mass leaving the steamgenerator (steam flow) is plotted as a horizon-tal bar graph at the bottom of the display. Thedomain property of mass balance (the relation-ship between mass in and mass out) is rep-resented directly by the line connecting thesteam flow and the feed flow bar graphs (themass balance indicator). The orientation of thisline is an emergent feature that specifies massbalance.

From a theoretical perspective configural dis-plays have a potential drawback, even when theyare designed properly and are otherwise effec-tive: There is no explicit representation of tem-poral information. As our previous analysissuggests, this could be a serious drawback, par-

ticularly for systems driven by the laws of na-ture. Pawlak and Vicente (1996) obtained someempirical evidence that supports these concernsin a process control setting. They evaluated aninterface that contained configural displays (theP + F interface). Although this interface wasgenerally effective, Pawlak and Vicente (1996)noted, “The masking problem experienced byP + F subjects during the second fault trial wasprobably due, in part, to the fact that there wasno history [temporal] information available forreservoir volume” (p. 683).

In summary, configural displays and trenddisplays possess complementary strengths andweaknesses for data presentation requirementsin complex sociotechnical systems. The nextlogical step is to consider if, and how, these twogeneral formats might be combined. As Pawlakand Vicente (1996, p. 683) have observed, “thechallenge lies in integrating historical [tempo-ral] information with the existing emergentfeatures of the display” (p. 683). Hansen (1995)has devised a technique that has the potentialto meet this design challenge. The time tunnelsdisplay design technique allows variations inconfigural displays (i.e., geometric forms) to beseen over time. It accomplishes this goal byscaling multiple versions of a geometric formaccording to the laws of perspective geometryand presenting them in the depth plane of thedisplay. This representational format combinesthe positive aspects of configural displays (directrepresentation of high-level domain properties)with the positive aspects of temporal displays(a trace of these properties over time).

Figure 1b illustrates a variation of Hansen’s(1995) time tunnel technique applied to thefunnel display of Figure 1a. A static framework,or perspective grid, is plotted in the depth plane.The outermost rectangle represents display axesthat correspond to the current time frame. Eachsuccessive rectangle is scaled and plotted deep-er in the depth plane to represent the displayaxes at a point more distant in time. Temporalinformation (individual variables, relationships,and goals over time) is presented within this per-spective grid. Perspective trends are formed byconnecting the values of individual variables incontiguous time frames. Similarly, mass balancerelationships over time are represented by a se-ries of mass balance indicator lines that are

344 Summer 2005 – Human Factors

Figure 1. The displays investigated in Experiment 1: (a) the baseline configural display and (b) the time tunneldisplay. FF = feed flow, SF = steam flow, ISGL = indicated (steam generator) level.

TIME TUNNELS 345

formed by connecting the values of steam andfeed flow within a time frame.

The present study continues a line of researchinvestigating time tunnel displays. Bennett andZimmerman (2001) evaluated a previous imple-mentation of the time tunnel display using themanual control of feedwater simulation. Theyfound very little evidence to suggest that thetime tunnel display improved performance atsystem control, fault estimation, or state estima-tion tasks. One potential explanation is relatedto issues in representation. The specific repre-sentational conventions used in their version ofa time tunnel display (i.e., geometrical planes,gray scale shading, occlusion) produced a dis-play that was highly complex. One goal of thepresent experiment was to determine if theserepresentational issues were responsible for thelack of performance benefits. To examine thispossibility, we developed an alternative versionof the time tunnel display that drastically re-duced the visual complexity (Figure 1b).

A second potential explanation involves thenature of the information that was present inthe time tunnel display. Previous analyses of themanual control of feedwater task have indicat-ed that the counterintuitive energy effects andtime lags that are associated with indicated lev-el (the primary variable to be controlled) arefactors that contribute to its difficulty (Roth &Woods, 1988). These insights led to the develop-ment of a form of decision support referred toas compensated level, a calculated variable thatprovides an estimate of the value that indicatedlevel will assume after these transitory effectshave dissipated. Bennett and Zimmerman (2001)evaluated both compensated level and the timetunnel technique. In contrast to the results ob-tained for a time tunnel display, compensatedlevel was extremely effective in improving per-formance at a variety of tasks. It is possible thatthe presence of this powerful, predictor-likevariable may have deterred participants frombecoming attuned to the temporal informationthat was present in the display. The present studyexplores this possibility by removing compen-sated level from the displays.

The baseline display (Figure 1a) and the re-designed time tunnel display (Figure 1b) wereevaluated using several dependent measures.Performance at the real-time system control and

fault detection tasks was measured using thesame methodological procedures of the previousevaluation. Participants also performed twotypes of information estimation tasks. One taskrequired participants to provide an estimate ofcompensated level (to allow us to assess theirunderstanding of current system state). A secondtask required participants to estimate indicatedlevel (the primary variable to be controlled) atthree points in time (current, –20 s, or –40 s).

EXPERIMENT 1

Method

Participants. Five men and 3 women (20–32 years of age) were paid $5.00/hr for theirparticipation. All participants had normal ornormal-corrected vision with no color blindnessdeficiencies. All participants had completed atleast three similar experiments and had a mini-mum of 30 hr of experience.

Apparatus. All experimental events were con-trolled by a general purpose laboratory comput-er (Sun Microsystem Workstation, Model 4-110)located in an enclosed experimental room. Acolor video monitor (Sony Trinitron, modelGDM1604-15, 40.64 cm, 1152 × 900 resolu-tion) and a standard keyboard were used.

Simulation model. For a more detailed de-scription of the simulation model, see Bennett,Toms, and Woods (1993).

Stimuli. The primary axes of the baseline con-figural display (Figure 1a) formed a 12.70-cmsquare. These axes contained scale markers andlabels (0%–100%, black). The bar graphs forthree variables (steam flow, feed flow, indicatedlevel) were 1.18 cm wide; had a maximum ex-tension of 12.70 cm; were green, blue, or mus-tard in color; and were “stenciled” (every otherpixel assuming the background color, mediumgray). A fourth variable (“adjusted” indicatedlevel = indicated level + [feed flow – steamflow]) appeared on the same axis as indicatedlevel. The current mass balance indicator con-sisted of three lines (a black line bracketed bytwo lines the color of the associated bar graphs).Two trip set points (20% and 80%, red) and atarget indicated level (50%, white) were present.

The time tunnel display (Figure 1b) retainedthese physical characteristics and presentedtemporal information over the last 40 s at 2-s


intervals. It was constructed in the followingmanner. A monocular observer views a pictureplane (100 cm square) that is located 500 cmaway (centered in the observer’s field of view).This picture plane corresponds to the currenttime slice of data (i.e., the outermost rectanglecorresponding to the primary axes of the dis-play). Nineteen additional picture planes werecreated to form the tunnel. The size of each suc-cessive picture plane was calculated by assum-ing a displacement of 100 cm away from theobserver. These picture planes were centered inthe display, and the area between them wasfilled with gray scale shading ranging from lightgray (most recent) to dark gray (most distant).Redundant scales appeared on the inside of thetunnel (20%–80%, black). Perspective trends(inside the tunnel) traced the value of these var-iables over a 40-s time span. Twenty mass bal-ance indicators highlighted steam and feed flowrelationships (one for each time frame); histor-ical mass balance indicators were representedby single lines ranging from dark gray (mostrecent) to light gray (most distant).

Procedure. Each participant completed onepractice session and four experimental sessionslasting approximately 1 hr each. In the first prac-tice session the participants were given descrip-tions of the displays, the tasks, and experimentalinstructions. No discussion of specific controlstrategies was provided. The participants weretested individually in an enclosed room. Therewere eight nonfault trials (two displays with fourrepetitions) and two fault trials (one leak, onestuck valve) per experimental session. The pre-sentation order was random. Fault trials occur-red within the first eight trials in a session;additional, nonfault trials for the correspondingdisplays were administered at the end of thesession. Each trial lasted for 5 min. Steam flowand feed flow were 0% initially; indicated leveland compensated level were 35% initially.Every 2 s a 1% increase could occur in steamflow (25% probability) as long as steam flowwas less than 80%. The displays were alsoupdated every 2 s. Participants changed feedflow by pointing and clicking on one of fourboxes (increasing or decreasing feed flow by1% or 4%; see Figure 1a). They were instructedto provide control inputs that moved indicatedlevel to a target level (50%) quickly, to maintain

indicated level close to this target level, and toavoid crossing set point boundaries. Auditoryfeedback (four tones) occurred when a boundarywas crossed. Continuously updated root meansquare error scores were provided.

Two faults could occur. One fault simulateda steam generator leak by decreasing the valueof indicated level by 0.25% at 2-s intervals.The second fault simulated a stuck valve: Con-trol input to feed flow caused the displayed val-ue to change (commanded value) but did notchange the simulation (actual value). When afault was present, it began at a random startingpoint ranging from 30 to 90 s into a trial. Parti-cipants provided a confidence rating for thepresence or absence of a fault at seven points inan experimental session (40, 80, 120, 160, 200,240, and 280 s). Participants pointed and clickedat a 7-point scale (see Figure 1a). Feedback onthe presence or absence of a fault was providedat the end of each trial. Each participant com-pleted two fault trials (one leak, one stuck valve)in each of four experimental sessions for a totalof eight fault trials. Each combination of faulttype (2) and display type (2) occurred twice foreach participant. The experiment-wide presenta-tion order was counterbalanced across partici-pants and days so that each combination of faulttype and display type occurred exactly four timeson each day.

Participants completed six information probesduring each experimental trial. During a probethe simulation was paused, an auditory tonesounded, a textual description of the probe waspresented, and participants entered a numericvalue via the keyboard. The display remainedvisible at all times. The participants were in-structed to respond to probes as accurately andquickly as possible. Feedback on both accuracyand latency was provided. A probe was readmin-istered in the final 25 s of a trial if the partici-pant entered an unacceptable value or changedhis or her estimate.

Two categories of information probes werecompleted. The compensated level estimationtask required participants to estimate the valueof compensated level (produced by the simula-tion model but not presented on the screen).An indicated level estimation task required par-ticipants to estimate (a) the current value ofindicated level, (b) its value 20 s in the past, or

TIME TUNNELS 347

(c) its value 40 s in the past. Six probes (threecompensated level, each of the three indicatedlevel) were completed in each trial. Theseprobes occurred during six time windows(45–75, 85–115, 125–155, 165–195, 205–235,and 245–275 s) and were administered whenthe next screen update was scheduled to occur.The presentation order was determined by (a)randomizing the three indicated level probes,(b) pairing each of these probes with a com-pensated level probe, and (c) randomizing theorder of the two probes within a pair. The colorcoding for the four variables was counterbal-anced across participants.

Results

A similar procedure was followed for themajority of analyses. Outliers were identifiedusing the test described in Lovie (1986, pp.55–56), T1 = (x(n) x–)/s, in which x(n) is a partic-ular observation (one of n observations), x– isthe mean of those observations, and s is the stan-dard deviation of those observations. The indi-vidual observations identified as outliers werenot considered in subsequent analyses; all otherobservations from individual participants wereretained. Nonparametric tests (Friedman analy-sis of variance [ANOVA]) were conducted to de-termine if the outlier distribution was random(none was significant). Only significant effectsinvolving the display manipulation are reported.

Compensated level estimates. Accuracy (errormagnitude) was measured by computing the ab-solute value of the difference between the par-ticipant’s estimate of a variable and the actualvalue. Latency was measured from the appear-ance of the prompt until the first digit of theparticipant’s response (1/100-s accuracy). Datawere averaged across the 12 repetitions occur-ring in an experimental session (three probes infour trials) and a 2 (display) × 4 (session) repeat-ed measures ANOVA was conducted for bothaccuracy and latency. No effects were significant.

Indicated level estimates. Data were averagedacross the four repetitions occurring in an exper-imental session (four trials), and a 2 (display) ×4 (session) × 3 (time: current, –20 s, –40 s) re-peated measures ANOVA was conducted forboth accuracy and latency. No effects involvingthe display manipulation were significant foraccuracy. For latency the main effect of display,

F(1, 7) = 16.10, p < .006, and the Time × Dis-play interaction, F(2, 14) = 7.69, p < .006, weresignificant. Fisher’s least significant differencerevealed that there were no significant differ-ences in performance between displays for cur-rent estimates of indicated level but that the timetunnel display took significantly longer than thebaseline display for estimates of indicated levelin the past (both –20 and –40 s).

Control performance. Six measures of controlperformance were considered: acquisition time,settling time, root mean square error, constantposition error, modulus mean error, and standarddeviation of the error (see Bennett & Zimmer-man, 2001, for formulas). Analyses of controlperformance for nonfault and reservoir leakfault trials were conducted (the stuck valve faulttrials were not analyzed because control inputhad no effect). For nonfault trials, data were av-eraged across the four repetitions occurring inan experimental session, and a 2 (display) × 4(session) repeated measures ANOVA was con-ducted for each of the six control measures. Nosignificant effects involving the display manip-ulation were obtained. For the reservoir leakfault trials, data were averaged across the repe-tition factor and a one-way (display) repeatedmeasures ANOVA was conducted for each con-trol measure. The standard deviation of theerror analysis revealed that the time tunnel dis-play produced significantly less variable controlperformance than did the baseline display, F(1,7) = 5.94, p < .05.

Fault estimates. The estimates of fault certain-ty and the corresponding latencies were aver-aged across repetition and session. Preplannedcontrasts for both linear and quadratic trendswere conducted to compare performance be-tween displays. No significant effects involvingthe display manipulation were obtained.

Discussion

Experiment 1 reinvestigated the time tunneldisplay technique with several modifications(i.e., alternative display design, additional perfor-mance measures, removal of compensated level).The results of this follow-on study did not differsubstantially from the initial evaluation (Bennett& Zimmerman, 2001): There was very little evi-dence that the time tunnel display was moreeffective than the baseline display. The standard


deviation of error during reservoir leak trials didindicate that control was significantly less vari-able with the time tunnel display than with thebaseline display. However, the response laten-cies obtained with the baseline display were sig-nificantly lower for estimates of indicated levelin the past (–20 and –40 s). There were no sig-nificant differences between displays for faultdetection confidence ratings or for estimates ofcompensated level.

These results suggest that the two potentialexplanations outlined in the Introduction (i.e.,issues in representation, the presence of com-pensated level) were probably not responsiblefor the lack of performance benefits for thetime tunnel display. After considerable deliber-ation, we considered an alternative hypothesis.In simple terms, it is possible that the time tun-nel displays were not successful simply becausethey provided no information of which the par-ticipants were not already aware. Several fac-tors may have contributed to this outcome. Theparticipants were very familiar with both gener-al system dynamics and specific behaviors asso-ciated with the start-up sequence as a result oftheir fairly extensive experience. In addition,no divided attention or time-sharing demandswere imposed, given that only the one primarytask was performed. Under these circumstancesit is possible that the participants themselves hadinternally integrated the temporal informationthat was required for successful performance.Stated alternatively, the time tunnel displays maynot have had an impact on performance becausethey provided an external representation of tem-poral information that participants had alreadyinternalized.

Experiment 2 investigated this hypothesisthrough substantial changes in methodology.These methodological changes were designedto eliminate (or at least drastically reduce) theopportunity for participants to internally inte-grate temporal information by cutting off theiraccess to the unfolding temporal context. Theparticipants did not control and monitor thepart-task simulation in real time. Instead, theywere presented with “snapshots” of actual sys-tem states that had been generated in Experi-ment 1; they were required to perform the samefault detection and state estimation tasks onthese snapshots that they had completed in

Experiment 1. The two displays evaluated in Ex-periment 1 (time tunnel and baseline) were alsoevaluated in Experiment 2 with no changes. Athird display represented temporal informationin a more traditional format: a strip chart con-taining the value of individual variables overtime (Figure 2). This display will be referred toas the trend display.

EXPERIMENT 2

Method

Participants. Of the 8 participants in Exper-iment 1, 4 (all men) also participated in Ex-periment 2. Three of the remaining participantshad moved from the area (this study was con-ducted 1 year and 9 months after the first), andthe fourth was unable to participate because ofmedical problems. Participants were paid $5/hr.

Apparatus. This was the same as in Experi-ment 1.

Simulation model. The same simulation mod-el was used in the training sessions.

Stimuli. The implementation of the baselineand time tunnel displays remained the same asin Experiment 1. The trend display (Figure 2)consisted of the baseline display plus a stripchart display, which measured 6.60 × 12.70cm. The values of variables were plotted as afunction of time (current time was located onthe right; 40 s in the past was located on theleft). Trends for individual variables were formedby connecting values occurring at contiguous2-s intervals.

Procedure. Participants were provided withwritten and oral instructions and then complet-ed eight sessions (approximately 1 hr). In thefirst two sessions (training/refresher sessions)participants controlled the simulation. Duringeach training session participants completedfour trials (two nonfault, one reservoir leak, andone stuck valve) with each of the three displays(12 total trials). Feedback regarding the pres-ence and nature of faults were provided.

During the six experimental sessions partici-pants initiated a trial by pointing and clicking ona button. A display appeared depicting a systemstate, and participants completed three experi-mental tasks in the following order. Participantsindicated their certainty regarding the presenceof a fault by pointing and clicking on a 6-point

TIME TUNNELS 349

Likert scale (endpoints labeled definitely no faultand definitely a fault). Participants then com-pleted a two-alternative, forced choice proce-dure that required them to indicate which ofthe two faults (reservoir leak or stuck valve) wasmore likely to have generated the configurationof system variables that was depicted. Finally,participants estimated the current value of com-pensated level. The participants were instructedto respond as accurately and as quickly as pos-sible for all three tasks. Feedback for latency(fault certainty, compensated level) and accura-cy (fault presence, fault type, and compensatedlevel) was provided.

System states were chosen from simulationdata generated during the last three sessions ofExperiment 1. A total of 48 trials were used: 12trials for each of the two fault types (randomlyselected) and the 24 corresponding nonfault tri-als. Data were taken from 5 “onset” times (40,80, 120, 160, and 200 s) measured relative to thebeginning of a fault (for nonfault trials the onsettime corresponded to the beginning of a fault inthe matched fault trial). The resulting 240 sys-tem states (48 trials × 5 onsets) were randomlydivided into six data subsets with the constraintthat the 40 trials in a data subset contained a

factorial combination of the four trial types (twofaults, two matched nonfaults), the five onsettimes, and two repetitions. Participants complet-ed three blocks (one for each display, randomorder) of 40 trials (random order) in each ex-perimental session. Each of these blocks corre-sponded to one of the six data subsets. Eachcombination of data subset and display occur-red exactly once for each participant during thecourse of the experiment.

Results

A similar procedure was followed for themajority of analyses. Outliers were identifiedusing the same procedures outlined in Experi-ment 1. The compensated level estimate (abso-lute error, signed error), the compensated levellatency, and the fault detection latency mea-sures produced 64 (2.22%), 62 (2.15%), 58(2.01%), and 60 (2.08%) outliers, respectively.Nonparametric tests were conducted to deter-mine if the outlier distributions were random(none was significant). Scores were averagedacross repetitions, sessions, and repetitions ofnonfault trials (nonfault trials were not averagedfor the signal detection and the fault identifi-cation analyses). Analytical comparisons were

Figure 2. The trend display investigated in Experiment 2. FF = feed flow, SF = steam flow, HLTT = high levelturbine trip, ISGL = indicated (steam generator) level, LLRT = low level reactor trip.


conducted: Four preplanned orthogonal con-trasts were used to test display by onset timeinteractions. These contrasts were aimed at dis-covering whether or not temporal information(i.e., baseline vs. trend and tunnel displays) andthe representation of temporal information (i.e.,trend vs. tunnel display) had an impact on per-formance. Trend analysis (linear and quadraticcomponents) was used to determine if the pat-terns of performance associated with the vari-ous displays were different across onset times.The four orthogonal contrasts resulted from thecrossing of display (i.e., presence of temporalinformation, representation of temporal in-formation) with onset time (linear, quadratic)components. These orthogonal contrasts wereapplied separately to the three trial types (nor-mal, reservoir leak, stuck valve) for a total of12 contrasts for each dependent variable. Ad-ditional analyses were performed only when apreplanned contrast was significant.

Table 1 lists significant preplanned orthogo-nal contrasts testing performance between thetrend and tunnel displays. The supplementalanalyses for significant contrasts are straight-forward: A comparison to test the simple maineffect of display was conducted for each of thefive onset times. Table 2 lists the significantpreplanned contrasts comparing performancebetween the baseline display and the two dis-plays with temporal information (Contrasts7–12). If a contrast was significant, then a sep-arate interaction comparison was conductedfor each temporal display relative to the base-line. These supplemental interaction compar-isons are labeled in Table 2 with a number anda letter (e.g., 7a). Only when a supplementalinteraction comparison was significant weretests for the simple main effects of display atonset levels conducted. A graphical summaryof the significant simple main effects of displayat onset time is presented in Figure 3. The dis-play by onset means (along with linear bestfits) for the signed error of compensated levelestimates are illustrated in Figure 4.

Discussion

In contrast to the results of Experiment 1,those obtained in Experiment 2 revealed a num-ber of performance differences that were related

to both the presence of temporal informationand the representational format that was used topresent it (Tables 1 and 2; Figure 3). The major-ity of these effects involve the estimation ofcompensated level. The two displays containingtemporal information (i.e., the tunnel and trenddisplays) produced significantly better estimatesof compensated level than did the display with-out temporal information (i.e., the baseline dis-play) during later onset times. As illustrated inFigure 3b, a total of 6 comparisons indicatedthat the trend display produced significantly bet-ter estimates than did the baseline display dur-ing the final two onset times (those contrastsoutlined with thin borders). A total of 11 com-parisons indicated that the time tunnel displayproduced significantly better estimates of com-pensated level than did the baseline display dur-ing the final three onset times (Figure 3c, thosecontrasts filled with gray).

In contrast, the pattern of performance wasreversed during the initial onset times: The base-line display produced better estimates than didthe two displays that contained temporal in-formation. The significant performance de-crements for the trend display (14 significantcomparisons, Figure 3b, thick borders) werefar more numerous than those for the tunneldisplay (6 significant comparisons, Figure 3c,thick borders). Finally, the direct comparisonsbetween the two displays with temporal infor-mation revealed that the tunnel display producedsubstantially better estimates: 14 significantcomparisons favored the tunnel display (Figure3a, gray fill), whereas only 1 comparison favoredthe trend display (Figure 3a, thin outline).

Temporal information and onset time. Theresults indicate very clearly that the presenceof temporal information (in the tunnel displayand, to a lesser extent, in the trend display) im-proved the ability to estimate compensated levelduring later onset times but not during initialonset times. The interpretation of these findingsrequires an explicit consideration of the natureand the utility of the temporal information thatwas present during early and late onset times.Therefore, Figure 5 provides the average valuesof the system variables for normal and fault tri-als in Experiment 1. The average onset times arelabeled on the horizontal axis; the correspondingtemporal information for a particular onset time

TAB

LE 1

:Pre

pla

nned

Ort

hog

ona

l Co

ntra

sts

Co

mp

arin

g P

erfo

rman

ce f

or

Dis

pla

ys W

ith A

ltern

ativ

e R

epre

sent

atio

ns o

f Te

mp

ora

l Inf

orm

atio

n in

Exp

erim

ent

2

Faul

t D

etec

tio

n: E

stim

ates

Faul

tE

stim

ates

of

Co

mp

ensa

ted

Lev

el

Co

ntra

st #

,Fa

ult

Cer

tain

tyLa

tenc

ySD

T d

′A

bso

lute

Err

or

Sig

ned

Err

or

Late

ncy

Verb

al D

escr

ipti

on

F(1,

3) =

p <

F(1,

3) =

p <

F(1,

3) =

p <

F(1,

3) =

p <

F(1

,3) =

p <

F(1,

3) =

p<

Tren

d v

s. t

unne

l:no

nfau

lt t

rial

s1.

Line

ar

93.6

5 .0

0310

8.14

.0

022.

Qua

dra

tic

87.5

8 .0

03

Res

ervo

ir le

ak t

rial

s3.

Line

ar

11.6

9 .0

5 38

.72

.009

69

.44

.004

4.Q

uad

rati

c 64

.34

.005

75

.97

.004

Stuc

k va

lve

tria

ls5.

Line

ar

10.3

4.0

5 18

6.31

.000

9 70

.12

.004

6.Q

uad

rati

c 23

6.98

.000

6

SDT

= s

igna

l det

ecti

on

theo

ry

Det

ecti

on:

351


are those data bracketed by the pair of verticaldotted lines above and to the left of an onsetlabel. Please note that the values of compensat-ed level are provided for illustrative purposesonly and were never actually presented to par-ticipants.

The temporal information during the initialtwo onset times (40 and 80 s) did not vary sub-stantially between normal and fault trials (seeFigure 5). The similarities occurred becausethe effects of the two faults were being maskedby the normal, but substantial, thermodynamiceffects that resulted from the control inputs thatwere executed to move indicated level to thegoal value. As a result, all three types of trialswere characterized by large imbalances betweenmass input (feed flow) and output (steam flow)and/or substantial rates of changes for variables.

In contrast to the initial onset times, in thefinal two onset times (160 and 200 s) the tempo-ral information present provided clear evidence

that differentiated between normal and faulttrials. The pattern of variables that appears dur-ing nonfault trials (Figure 5a) at later onset timesspecifies normal system dynamics and the ab-sence of any transitory thermodynamic effects:mass input/output was balanced and the valueof indicated level was constant over time. In con-trast, the temporal information that is presentduring the reservoir leak trials (Figure 5b) at lat-er onset times clearly specifies a break in systemconstraints. The value of indicated level wasreasonably stable over time. However, this wasattributable only to the fact that participantswere successfully compensating for the reservoirleak by maintaining mass input (feed flow) athigher levels than mass output (steam flow).

The temporal information during stuck valvetrials did indicate the presence of a fault, but theevidence was less clear than that for the reser-voir leak. Participants were attempting to com-pensate for the decreases in indicated level by

TABLE 2: Preplanned Orthogonal Contrasts and Supplemental Interaction Contrasts for Experiment 2

Estimation of Compensated Level

Absolute Error Signed Error Latency

Contrast #, Verbal Description F(1, 3) = p < F(1, 3) = p < F(1, 3) = p <

Baseline vs. Trend and Tunnel: Nonfault trials

7. Linear 192.08 .0009 88.95 .003 11.82 .057a. Interaction comparison: baseline vs. trend 213.64 .0007 18.26 .037b. Interaction comparison: baseline vs. tunnel 154.56 .002 205.34 .0008

8. Quadratic 1170.79 .000068a. Interaction comparison: baseline vs. trend 251.99 .00068b. Interaction comparison: baseline vs. tunnel

Reservoir Leak Trials

9. Linear 1234.59 .000069a. Interaction comparison: baseline vs. trend 112.30 .0029b. Interaction comparison: baseline vs. tunnel 155.82 .002

10. Quadratic 356.23 .0004 27.21 .0210a. Interaction comparison: baseline vs. trend 561.39 .0002 445.79 .000310b. Interaction comparison: baseline vs. tunnel 13.24 .04

Stuck Valve Trials

11. Linear 226.39 .0007 295.22 .0005 23.32 .0211a. Interaction comparison: baseline vs. trend 210.63 .0008 67.38 .00411b. Interaction comparison: baseline vs. tunnel 235.28 .0007 459.57 .0003 12.80 .04

12. Quadratic 14.44 .0412a. Interaction comparison: baseline vs. trend 71.48 .00412b. Interaction comparison: baseline vs. tunnel

TIME TUNNELS 353

increasing feed flow relative to steam flow (Fi-gure 5c). The intended effect of these controlinputs was accurately portrayed as the “com-manded” value of feed flow in the displays.However, this commanded value is actually quitemisleading: The “actual” value of feed flow didnot change after the onset of the stuck valvefault (it remained at approximately 15% afterfault onset).

The problem that this poses for participantsis that they were viewing a configuration of vari-ables that could very easily be confused withnormal shrink effects: Increases in feed flow re-lative to steam flow normally produce thermo-dynamic “shrink” effects, which cause the valueof indicated level to decrease initially, as illus-trated in the beginning of all trials in Figure 5.The evidence that specifies abnormal system

Figure 3. Simple main effects of display at various onset times for preplanned contrasts or significant interactioncomparisons that were significant in Experiment 2: (a) trend versus tunnel, (b) baseline versus trend, (c), base-line versus tunnel.


dynamics is the rapid and sustained decreasesin indicated level that occur over the course ofthe last 150 s: This behavior is clearly not con-sistent with normal system dynamics. However,this evidence accumulates over a time span thatis substantially longer than the 40 s of temporalinformation that was portrayed in the displays.Thus the temporal information for the stuckvalve trials also specifies a break in system con-straints, although the evidence was less obviousthan that for the reservoir leak because of the40-s time frame that was chosen for presenta-tion. These analyses form the basis for the ensu-ing interpretations.

Effects of temporal information on perfor-mance. The two displays with temporal infor-mation produced substantial performanceadvantages for the estimation of compensated

level during the final two onset times. A total of16 comparisons for the accuracy of compensat-ed level estimates (absolute error, signed error)indicated significantly better performance forboth the trend display (Figure 3b, thin outlines)and the tunnel display (Figure 3c, gray fill) rel-ative to the baseline display; there were no sig-nificant comparisons favoring the baselinedisplay.

These results are consistent with the concep-tual analysis of temporal information that wasoutlined in the Introduction. The value of com-pensated level during the final two onset timeswas extremely context sensitive: It could beequal to (Figure 5a), greater than (Figure 5b),or less than (Figure 5c) the value of indicatedlevel. The significant performance advantagesfor the time tunnel and trend displays suggest

Figure 4. Average signed error of compensated level estimates for display and onset times during (a) nonfault,(b) reservoir leak, and (c) stuck valve trials.

TIME TUNNELS 355

that participants used temporal information tofine-tune their estimates of compensated levelduring the final two onset times (Figures 3 and4). The temporal information provided a contextthat was very useful in understanding currentand future system states. Participants used thiscontext, in combination with knowledge aboutsystem dynamics, to form more accurate esti-mates of compensated level.

Unlike the time tunnel and trend displays,the baseline display provided only momentaryand punctate views of system states. This caused

difficulties for estimating compensated level be-cause the momentary configurations of the threevariables could be quite ambiguous and unin-formative (as outlined previously). The partici-pants appear to have adopted a fairly simple andcontext-free strategy to deal with this uncertain-ty: “Take the value of indicated level and addsome number of units to form an estimate ofcompensated level.” The evidence supportingthis interpretation will be discussed in the fol-lowing section.

In contrast to the final two onset times, during

Figure 5. Average value of system variables for display and onset times during (a) nonfault, (b) reservoir leak,and (c) stuck valve trials in Experiment 1. The values of compensated level are for illustrative purposes only.


the initial onset times participants producedmore accurate estimates of compensated levelwith the baseline display than with the tempo-ral displays (Figures 3a and 3b). The positiveresults for the baseline display are somewhatmisleading. During the initial two onset timesthe participants’ control inputs (increases infeed flow) produced large positive net inflowsof mass and therefore larger values of compen-sated level relative to indicated level (see Fi-gure 5). Therefore the context-free strategy ofsimply adding some number of units to the valueof indicated level would produce reasonableperformance during the initial two onset times.This strategy would also produce poor perfor-mance during later onset times, as outlined pre-viously. More specifically, participants shouldoverestimate compensated level during nonfaulttrials (Figure 5a) and stuck valve trials (Figure5c) when the value of indicated level was equalto or less than the value of compensated level.The results for signed error support this inter-pretation (Figure 4). In summary, observers hadno contextual evidence for fine-tuning estimatesof compensated level with the baseline display.They appear to have adopted a simple, context-free response strategy that was effective for themajority of experimental trials (including theinitial onset times) but that also produced glar-ing errors for other conditions. For example, theoverestimation of compensated level (Figure 4a)during the stable and balanced conditions thatcharacterized nonfault trials at the final two on-set times was completely at odds with systemdynamics.

The poor performance in estimating com-pensated level for the temporal displays duringthe initial two onset times requires interpreta-tion. The temporal information presented inthese displays revealed the substantial mass im-balances and rates of changes that characterizedthe system during these onset times. Althoughinaccurate, the estimates of compensated levelproduced with the time tunnel display were con-sistent with system dynamics. The results forsigned error (Figure 4) indicate that participantscorrectly predicted the direction of change infuture system states but overestimated the mag-nitude. This suggests that the errors may havebeen the result of miscalibration, attributableperhaps to the unusual system dynamics present

during the initial onset times. The pattern of re-sults for the trend display is somewhat morepuzzling. With the exception of the final onsettime, there was a consistent tendency to under-estimate the value of compensated level.

GENERAL DISCUSSION

Extensive analyses of the manual control offeedwater task have revealed that understand-ing and anticipating the impact of mass andenergy effects on the value of indicated level isa key component of successful performance(Roth & Woods, 1988). The compensated levelvariable is a form of decision support that wasexplicitly designed to incorporate this knowl-edge (U.S. Patent No. 4,770,841, 1988 [Haley& Woods, 1988]). Empirical evaluations of com-pensated level have indicated that its presencesignificantly improved performance at a varietyof control and fault detection tasks (e.g., Bennett& Zimmerman, 2001). These conceptual analy-ses and empirical results strongly suggest thatthe capability to accurately estimate values ofcompensated level constitutes evidence for animproved understanding of system dynamics.Similarly, the capability to estimate this variablemore accurately with one display, relative toanother, constitutes evidence for more effectivedesign.

The time tunnel display produced the bestoverall performance for the estimation of com-pensated level in Experiment 2. The performanceadvantages for this display relative to the trenddisplay were universal and unequivocal: Thetime tunnel display consistently produced signif-icant performance advantages that were distrib-uted across all onset times (Figure 3a). The timetunnel display also produced better overall lev-els of performance than did the baseline display(Figure 3c). The results obtained during thelater onset times, when the value of compensat-ed level was highly context dependent, revealedthat the time tunnel display produced signifi-cantly better estimates on a consistent basis.The most likely interpretation of these results isthat the presence of temporal information, incombination with the representational formatemployed in the time tunnel display, produceda better understanding of current system state(i.e., the contributions of mass and energy effects

TIME TUNNELS 357

to current value of indicated level) and a corre-sponding increase in the capability to estimatevalues of compensated level (i.e., the value thatindicated level that will assume once the tran-sient thermodynamic effects have dissipated).

The presence of temporal information in thetrend display also significantly improved esti-mation performance relative to the baseline dis-play at late onset times during Experiment 2.However, it is clear that the same temporalinformation had a less positive impact on per-formance when it was presented in the trenddisplay format than when it was presented inthe time tunnel format. There are several poten-tial explanations. One is fairly simple: Partici-pants had less experience interacting with thisdisplay than with the time tunnel display. Asecond potential explanation is that this displayviolated a population stereotype by presentingcurrent values on the right and historical val-ues to the left. The alternative arrangement(i.e., data flowing from left to right over time) ismore consistent with common information dis-plays (e.g., written text) and may have provenmore effective. A third possibility is that trenddisplays are simply not very effective represen-tational formats. Spenkelink (1990) investigatedthe utility of trend displays for the detection offaults in a chemical plant simulation. He con-cluded that “the presence of historical [temporal]information, in the form of trends, seriouslyhampers the early detection of oncoming off-normals” (p. 199). Given that these displays arewidely accepted and widely used, further inves-tigations would appear to be in order.

The estimation of compensated level was leasteffective for the baseline display in Experiment2. The lack of temporal information resulted in agreat deal of uncertainty and ambiguity regard-ing future system states. To deal with this un-certainty, participants appear to have adopted asimple, context-free strategy (i.e., add a numberof units to indicated level). This strategy alsoproduced errors that were glaringly inconsistentwith system dynamics at later onset times, whenthe potential utility of temporal information wasat its highest: The faults had propagated throughthe system, and the corresponding evidence wasnot being masked by normal thermodynamiceffects. It is precisely under these types of condi-tions that the interface and the associated dis-

plays should be providing the most effectivedecision support.

Thus the overall results for compensated levelestimation in Experiment 2 suggest that tempo-ral information improved participants’ under-standing of current and future system states.This understanding should have translated intoan improved capability to detect and identifyfaults, especially with the time tunnel display.However, the results obtained for fault detec-tion and identification in Experiment 2 wereinconclusive. The time tunnel display did pro-duce better average levels of performance forfault detection, particularly under circumstancesin which the temporal information had the high-est level of diagnostic value (i.e., reservoir leaktrials at later onset times). However, only oneof the statistical comparisons between displaysreached significance (Figure 3a).

Establishing a clear link between temporalinformation and improved performance at thesetypes of dynamic tasks, which are representativeof the tasks that occur in the actual domain, isessential. Any number of factors may have con-tributed to the lack of performance benefits inthe present experiment. However, one factorthat is very likely to have had a negative impacton the detection and identification of faults isthe amount of temporal information that waspresented in the time tunnel and trend displays.In retrospect, the 40-s time frame used in thepresent evaluation is much smaller than the timeframe that characterizes the system dynamicsof the manual control of feedwater task. Consi-der the temporal information contained in eachof the isolated individual 40-s time frames, rela-tive to the entire 300-s time frame (see Figure 5).This short time frame is likely to have produceda “keyhole” effect that limited the utility of tem-poral information; it seems highly likely that alonger time frame would have improved faultdetection and identification performance.

These observations underscore a central issuein the design of temporal displays: What is themost effective time frame to be displayed? Onedesign strategy is to analyze the inherent timeconstants of a system and then pick a single,most appropriate time frame for the temporalinformation. An alternative strategy is to designthe display so that it is capable of presenting arange of time frames and to allow the observer


to choose the time frame that is most appropri-ate. For example, a user might manipulate agraphical slider that dynamically changes thetime frame in real time. This would allow flexi-bility in the selection of a time span that matchescurrent goals, needs, and the system context.

Methodological Insights for theEvaluation of Temporal Displays

The patterns of results obtained in the twoevaluations of temporal displays were strikinglydifferent, despite the fact that there was consid-erable overlap in the experimental manipula-tions. The results of Experiment 1 provided verylittle evidence that the time tunnel display im-proved performance at process control, fault de-tection, or state estimation tasks. Experiment 2was conducted with the vast majority of experi-mental factors held constant, including the tasks(fault detection and estimation of compensatedlevel), the displays (time tunnel and baseline),the participants (a subset of those participating inExperiment 1), and the system states (a subset ofthose generated in Experiment 1). In sharp con-trast to Experiment 1, in Experiment 2 numer-ous and substantial differences in performancewere obtained for the estimation of compen-sated level.

The most likely interpretation of these diver-gent empirical results is related to differences inthe availability of temporal information in thetwo settings. In Experiment 1 the fault detectionand the state estimate tasks were interleavedwith real-time process control. Because no othersubstantial demands were being imposed onthe participants, they were free to concentrateon the process control tasks and to internallyintegrate (or “absorb”) the unfolding temporalcontext. As a result, the presence (time tunneldisplay) or absence (baseline display) of an ex-ternal representation of temporal informationdid not have an impact on performance. In Ex-periment 2 the participants were isolated fromthe unfolding temporal context: Static snapshotsof system states were presented to participantsprior to the completion of identical detection andstate estimation tasks. This experimental manip-ulation forced participants to obtain temporalinformation from the external representationsprovided by displays. Under these conditions

the presence or absence of temporal informa-tion did have an impact on performance.

As noted in the Introduction, there are sur-prisingly few empirical evaluations of temporaldisplays in dynamic settings, despite their wide-spread use and acceptance. The divergent resultsobtained in Experiments 1 and 2 provide someindication of why that might be. The capabilityto absorb or integrate temporal information willtend to negate performance benefits for exter-nal representations of the temporal informationwhen participants can focus exclusively on thesedisplays (as in Experiment 1). The utility of ex-ternal representations of temporal informationwill be more apparent when observers do nothave access to the unfolding temporal context(as in Experiment 2). This may explain the lackof empirical evidence in dynamic settings: Pre-vious evaluations of temporal displays may havefailed to restrict access to the temporal contextand therefore failed to obtain supportive evi-dence.

These results indicate that developing dynam-ic evaluation settings that restrict access to theunfolding temporal context in a representativemanner represents a formidable challenge. It isimportant to note that not having access to thetemporal context may actually be more com-monplace than having access to the temporalcontext in applied domains. For example, powerplant operators are required to perform a vari-ety of tasks that are interleaved across time; toperform these tasks they are required to consulta number of controls and displays. In addition,the primary displays and the associated displaysof temporal information (i.e., strip charts) arepresently located in different parts of the con-trol room. Thus the operators cannot afford tofocus exclusively on one task or one set of dis-plays. Similar considerations related to temporalinformation are likely to be found in other do-mains of application.

Summary

Temporal information is critical in many com-plex, dynamic domains. However, there does notappear to be a solid empirical basis to assist de-signers in determining exactly how this infor-mation should be presented. The time tunneldisplay design technique is appealing on several

TIME TUNNELS 359

dimensions. It combines the benefits of configur-al displays (i.e., the emphasis of critical domainproperties through emergent features) with tem-poral information (an explicit representationhow these properties have changed over time).In addition, the time tunnel design techniquemakes effective use of limited display “realestate” by incorporating current and temporalvalues in a single unified display. The resultsobtained with the time tunnel technique in thepresent evaluation are encouraging and com-plement those of Hansen (1995) with a substanti-ally different implementation and a substantiallydifferent set of experimental tasks. Further in-vestigation of the time tunnel display format, aswell as alternative representations of temporalinformation, appears to be a reasonable courseof action to pursue.

ACKNOWLEDGMENTS

The authors would like to thank Raja Para-suraman for his assistance in completing thesignal detection analyses. The part-task simula-tion was originally developed at WestinghouseCorporation.

REFERENCES

Bennett, K. B., & Flach, J. M. (1992). Graphical displays: Implica-tions for divided attention, focused attention, and problemsolving. Human Factors, 34, 513–533.

Bennett, K. B., Nagy, A. L., & Flach, J. M. (1997). Visual displays. InG. Salvendy (Ed.), Handbook of human factors and ergonomics(2nd ed., pp. 659–696). New York: Wiley.

Bennett, K. B., Toms, M. L., & Woods, D. D. (1993). Emergentfeatures and configural elements: Designing more effectiveconfigural displays. Human Factors, 35, 71–97.

Bennett, K. B., & Zimmerman, J. (2001). A preliminary investigationof the time tunnels display design technique. Displays, 22,183–199.

Haley, P. H., & Woods, D. D. (1988). U.S. Patent Number 4,770,841.Hansen, J. P. (1995). An experimental investigation of configural,

digital, and temporal information on process displays. HumanFactors, 37, 539–552.

Lovie, P. (1986). Identifying outliers. In A. D. Lovie (Ed.), New de-velopments in statistics for psychology and the social sciences(pp. 44–69). London: British Psychological Society/Methuen.

Pawlak, W. S., & Vicente, K. J. (1996). Inducing effective operatorcontrol through ecological interface design. InternationalJournal of Human-Computer Studies, 44, 653–688.

Pomerantz, J. R. (1986). Visual form perception: An overview. InH. C. Nusbaum & E. C. Schwab (Eds.), Pattern recognition byhumans and machines: Volume 2. Visual perception (pp. 1–30).Orlando, FL: Academic Press.

Roth, E. M., & Woods, D. D. (1988). Aiding human performance:I. Cognitive analysis. Le Travail Humain, 51, 39–64.

Schutz, H. G. (1961). An evaluation of formats for graphic trenddisplays: Experiment II. Human Factors, 3, 99–107.

Spenkelink, G. P. J. (1990). Aiding the operator’s anticipatory be-haviour: The design of process state information. AppliedErgonomics, 21, 199–206.

Vicente, K. J. (1991). Supporting knowledge-based behaviorthrough ecological interface design (EPRL-91-02). Urbana-Champaign: University of Illinois, Department of MechanicalEngineering, Engineering Psychology Research Laboratory.

Kevin B. Bennett is a professor in the Departmentof Psychology, Wright State University. He receivedhis Ph.D. in applied experimental psychology fromthe Catholic University of America in 1984.

Michael Payne manages a research and design orga-nization at Intel Corporation in Hillsboro, Oregon.He received an M.S. in human factors psychologyfrom Wright State University in 1999.

Brett Walters is a human factors engineer at MicroAnalysis and Design in Boulder, Colorado. He re-ceived a M.S. in human factors psychology fromWright State University in 1997.

Date received: August 5, 2002Date accepted: November 2, 2004

an evaluation of a “time tunnel” display format for the...

Documents