n90-25528 - nasa the human factors of ... on use of a social cybernetic model of workstation ......
TRANSCRIPT
N90-25528
THE HUMAN FACTORS OF WORKSTATION TELEPRESENCE
Thomas J. Smith"
Human Factors GroupBureau of Mines
5629 Minnehaha Ave. S.
Minneapolis, MN 55417
Karl U. SmithEmeritus Professor
Behavioral Cybernetics LaboratoryUniv. of Wisconsin - Madison
I001 Tower Blvd., Lake Wales, FL
ABSTRACTHuman-operated, remotely controlled robots
(telerobots) are projected to play a pivotal role inthe performance of assembly, maintenance, andservicing manipulation tasks, during constructionand operation of the U.S. space station in the nextdecade. To reap the anticipated benefits oftelerobotic systems---increased safety, efficiency,
and productivity of task performance in space,accompanied by reduced costs---it is essential that
the control requirements for telerobot operation arecompliant with control capabilities and limitationsof the human operator. The term workstationtelepresence has been introduced to describe suchhuman-telerobot compliance, which enables the human
operator to effectively project his/her body imageand behavioral skills to control of the telerobot
itself. This report addresses major human-factorsconsiderations for establishing high fidelityworkstation telepresence during human-telerobot
operation. Telerobot workstation telepresence isdefined by the proficiency and skill with which the
operator is able to control sensory feedback fromdirect interaction with the workstation itself, andfrom workstation-mediated interaction with the
telerobot. Numerous conditions influencing suchcontrol have been identified. This raises the
question as to what specific factors most criticallyinfluence the realization of high fidelity
workstation telepresence? The thesis advanced in
this report is that perturbations in sensoryfeedback represent a major source of variability inhuman performance during interactive telerobot
operation. Perturbed sensory Feedback research overthe past three decades has established that spatialtransformations or temporal delays in sensory
feedback engender substantial decrements ininteractive task performance, which training does
not completely overcome. Similar, more recentlaboratory studies with remote telerobotmanipulators have confirmed in part the earlier
findings. The goal of effective and safeinteractive telerobot operation therefore maybenefit from development of techniques which enablethe interactive computer to detect, and compensate
for, perturbations in sensory feedback beforepresentation of such feedback to the operator. Arecently developed social cybernetic model of human-
computer interaction can be used to guide thisapproach, based on computer-mediated tracking andcontrol of sensory feedback. The report will
conclude by indicating how the social cyberneticmodel can be employed for evaluating the variousmodes, patterns, and integrations of interpersonal,
team, and human-computer interactions which play acentral role in workstation telepresence............................................... .--_-
"This is not an official Bureau of Mines report.
The views expressed are not necessarily those of theU.S. Bureau of Mines.
INTRODUCTION
Automation and robotics (A&R) will play a
pivotal role in the development and operation ofpermanently-manned U.S. extraterrestrial outposts.
One such outpost currently under development by NASAis Space Station Freedom, an orbiting spacelaboratory targeted for assembly in the mid-1990s.Projected A&R applications For the U.S. spacestation encompass over twenty major projects,
including use of interactive teleoperated robots(telerobots) for assembly, servicing, andmaintenance functions []-3]. Telerobots are deviceswith mechanical arms for grasping and manipulationwhich are envisioned to play an integral role in
space station operation. The devices are projectedto have some of the same operational capabilities asan astronaut in a space suit. NASA and Bureau ofMines research also indicates that interactive A&R
systems will be employed in mining lunar resources,in support of a lunar base in the 21st century [4-
8]. Anticipated benefits of interactive A&Rapplications in space include a reduced crew size,increased crew productivity, consequent costsavings, and increased safety of the crew,concomitant with use of telerobots instead of crewmembers to conduct risky extravehicular (EVA)
maneuvers [1,2]. For these benefits to be realized,it is essential that the human-factors design and
operational characteristics of the human-computer/human-telerobot interface be compatible with the
behavioral-physiological performance capabilitiesand limitations of the human operator. In the case
of human operation of telerobots at remote sites
(teleoperation), such compliance across theinterface is described by the term telepresence,
defined by NASA as "a teleoperation situation inwhich the operator has sufficient cues to simulate
sensations that would be experienced in directmanual performance of the operations" [9].
This report provides a conceptual andtechnical analysis of the major human-factors issueswhich must be addressed in creating effective
workstation telepresence during telerobot operation.A major focus of the analysis concerns the
phenomenon of perturbed sensory feedback as adecisive influence on workstation telepresence. Theterm refers to the introduction of some sort of
perturbation, or transformation, in the spatial,temporal, or physical properties of one or moremodalities of sensory input to the human operatoracross the interface. Such perturbations alter the
normal closed-loop control mechanisms of cognitiveand motor perceptual behavior, and can thereby evokesubstantial decrements in human performance during
teleoperation.The principle hypothesis advanced in this
report is that perturbations in sensory feedbackconstitute a major obstacle to the creation of high
fidelity workstation telepresence. As a result of
235
https://ntrs.nasa.gov/search.jsp?R=19900016212 2018-05-13T10:18:14+00:00Z
human-factors limitations of interface design,perturbations in sensory feedback arise routinelyduring human interaction with telerobotic systems.
These limitations may be exacerbated by conditionsprevailing in space environments. To address this
problem, the following sections: (I) provide ahuman-factors analysis of telerobotic workstationtelepresence; (2) outline the adverse effects ofperturbed sensory feedback on interactive
performance for behavior generally, and fortelerobotic operation in particular; and (3) discusshow such effects can be managed operationally, basedon use of a social cybernetic model of workstation
telepresence to target research needs.
HUMAN-FACTORS ANALYSIS OF WORKSTATION TELEPRESENCE
During human performance with a telerobot viaa teleoperation workstation, sensory feedback
(visual, tactile, kinesthetic, and/or auditory) tothe operator is generated by two sources: directinteraction with the workstation, and workstation-
computer mediated interaction with the telerobot.To optimize workstation telepresence, the spatial,temporal, and physical properties of workstation and
telerobot sensory feedback, as determined by thehuman-factors design of the teleoperation system,must be compliant with the sensory feedback controlcapabilities of the human operator.
This concept is illustrated in Figure I, usingthe example of a movement-actuated (master-slave)telerobot manipulator. Effective control of themanipulator depends upon space, time, and force
(kinesthetic) compliance between operationalfeedback from the telerobot slave sector, and motorbehavioral control of this sensory feedback by theoperator. If human-factors design problems causespatiotemporal perturbations in sensory feedback,
then compliance between sensory feedback and its_ control is compromised and performance suffers.
Evidence from various sources suggests that becauseof human-factors design problems, both spatial and
temporal perturbations in sensory feedback are apervasive feature of interactive human-computer andtelerobotic systems [10-14].
Figure 2 expands upon Figure I by
schematically illustrating the major human-factorsdesign issues that must be addressed in creatingworkstation telepresence [11]. Figure 2A shows atelerobotic manipulator under direct control, ratherthan computer-mediated remote control, by the
operator. Integrated postural, transport,manipulative, and tremor movements are employed tocontrol the spatiotemporal properties of manipulatoroperation, in relation to its master-control,
actuator, and slave components (Figure 1). For thecontrol task, the operator relies upon three majorsources of sensory feedback: (1) reactive feedback
From movements of body segment and limb effectors;(2) instrumental feedback from effector interactionwith and movement of the manipulator master-controlcomponent; and (3) operational Feedback from action
of the manipulator slave component and its effect onthe environment.
Figure 2B illustrates how these relationshipsare altered in the case of teleoperation of a remotemanipulator. In this case, control of the
manipulator is mediated through a master-controlcomputer workstation, whose actual design for space
station telerobotic applications is still under
development [15,16]. Direct reactive, instrumental,and operational sources of sensory feedback aregenerated by operator interaction with the
workstation, rather than directly with thetelerobot. Through display of video images fromexternal cameras, plus tactile, force, and possiblyauditory sensors mounted on or near the robot, the
workstation provides indirect, computer-mediatedreactive, instrumental, and operational sensoryFeedback information from activity of the
telerobotic manipulator. Thus, during remotetelerobot manipulation, the operator mustsimultaneously control sensory feedback from twomajor modes of interaction, namely direct
interaction with the workstation plus workstation-mediated interaction with the telerobot manipulator.Collectively, this combination of direct plusindirect, computer-mediated feedback information maybe referred to as teleoresence sensory feedback.
Conclusions From Research on CyberneticAnthropomorphic Machines
One of the first comprehensive conceptual and
technical investigations of human-factorsengineering requirements for effective workstationtelepresence, as defined generally in Figures ] and2, was carried out during the fifties and sixties inthe context of an extensive General Electric
research and development program on cyberneticanthropomorphic (manlike) machines (CAMs) []I],
Figure I. Human factors in control-displayrelationships in a manual manipulator. Normalspatial and temporal visual feedback compliance isessential for optimal performance.
236
....tt
Operaeor
Masfer contro_
Master conh'ol ._ ..r [|
I I[ I• _ _ Operational FB N aclualor
t -............ ......-z___ ........... ...;.-" I I
Figure 2. Sources of visual feedback in direct versus remote operation of a manual manipulator.Reactive feedback is from movement of limb or segment effector. Instrumental feedback is from
effector-mediated movement of master-control component. Operational feedback is from action ofslave component. A. Direct operator interaction with telerobot; B. Computer-mediated operatorinteraction with telerobot.
under the direction of Mosher [17,18]. The broad
objective of the program was to develop mechanicaldevices which would serve as operator-controlled
extensions of the body to expand the strength andendurance capabilities of human performance. The
program dealt with four general types of CAMs: arm-claw manipulators, bipedal walking machines,
quadripedal walking machines, and ambulatoryexoskeletons which enclosed the operator within
body-like frames. These were true teleroboticdevices, in that they each comprised master control,actuator, and slave subsystems which provided
computer-mediated, servo-controlled spatial,temporal, and force feedback compliant with allmodes and patterns of motorsensory feedback from thehuman operator's articulated arm, hand-wrist, leg,
foot, and torso movements.To measure system relationships between
operator performance and machine feedback parameters(such as device position, servo gain, and force-feedback ratio), human-factors studies of the four
types of CAMs were carried out. These studies were
guided by the general block diagram of an operator-computer-telerobot system illustrated in Figure 3.Figure 3 is based on the original design for a CAMsystem [18]. However, the figure has been adapted[11] to show a more detailed breakdown of the
operator-workstation-telerobot interactionsschematically illustrated in Figure 2B, includingspecifically the sources of reactive, instrumental,and operational sensory feedback provided to the
operator from the master-control, actuator, andslave subsystems. The figure also has been modifiedto conform to current specifications [31,32,64] for
the Space Station Flight Telerobotic Servicer(SSFTS), to be discussed further in a subsequentsection.
Figure 3 emphasizes the complexity of sensoryfeedback relationships which must be built intotelerobotic systems to provide fully compliant force
and position feedback for effective telepresence.Force feedback to the operator is mediated by
kinesthetic (also termed proprioceptive)mechanoreceptors in the muscles, tendons, and
237
joints, plus tactile receptors in the skin.
Collectively, kinesthetic and tactile receptorsprovide muscle-tendon-joint sensibility, which
encompasses pressure, movement position and dynamics(occurrence, locus, direction, velocity, andacceleration of movement), temperature, and pain[]g, Chap. 14].
Mosher [17,18] established that four differentdimensions of force feedback are needed to control
body movement accurately, and should therefore be
incorporated into a telerobot control system: (])force of movement at different levels of exertion;(2) upper limit of force required; (3) prediction of
force needed to execute movements of a givenvelocity and power; and (4) detection of relativedisplacement between movement and its sensory
feedback. A series of system design and operationalfactors which influence the muscle-tendon-jointsensibility of the operator were identified,
including Force feedback ratios between operator androbot, drift on bias forces, friction thresholds,nonlinearity of force interactions, saturationfactors, and force-signal integrity as influenced bysystem kinematics. As part of this human-factors
system analysis, Mosher also demonstrated that forcefeedback related to the limits of exertion of
powered grips, lifts, and pulls must be built intotelerobots as a safety precaution to prevent their
destructive action, a Finding with implications forpresent-day robotic systems [20].
Although theoretical and experimentaldemonstration of the fact that force feedback is
needed to guide and control telerobotic machines wasfully developed and confirmed by the work of Mosher
(17,18), research conducted by the senior author ofthe present report established that effectiveworkstation telepresence could not be achievedwithout also introducing spatial compliance insensory feedback control of telerobotic action
[11,2],22]. Such compliance occurs only when theposition feedback system designated in Figure 3embodies several spatiotemporal parameters of
sensory feedback, namely position, direction,relative extent, range, velocity, acceleration, andform or pattern of telerobotic movement.
As described in the next section, the roles ofspatiotemporal sensory feedback factors in humanmovement control were established in the course of
an extensive program of experimental research on theeffects of spatially and temporally perturbed
sensory feedback on human behavior and performance[14, 23-29]. Findings from this research
...... ..............................................
VISUALSENSORY
RECEPTORS KINESTHETIC 1
TACTILE
REAC]IVEFCEDBACK
: !
: i. i
:. =, i
MUSCLE ACTIVITY
MOTOR CONTROL
{BODY POSTURE;TRANSPORT ANDMANIPULATIVE
MOVEMENTS OF
ARM AND HAND)
CENTRALNERVOUS
SYSTEM
COGNITION ,'PERCEPTION
WORKSTATION
COMPUTER
WORKSTATION
CONTROL PANEL
POSITIONING DEVICE
DYNAMICS
WORKSTATION
DISPLAY
=
FC#_CEFEEDBACK SERVOS _:
TELEROBOTIC
CAMERAS
EXTERNAL
CAMERAS
I
I
1i
...... .....TASK BOARD
HARDWARE/SENSOR
ONBOARD TILLUMINATION
POSFFIONFEEDBACK SERVOS
MANIPULATORPOSITION H MANIPULATORACTUATORS DYNAMICS
TASKEXECUTK)N
i
MANIPULATORSERVOSFORCE _ MANIPULATOROUTPUTFORCE I
Figure 3. Block diagram of telerobot control system, indicating interactive roles of the
operator, workstation, telerobot, and video cameras in system performance. Visual, position,and Force feedback to the operator combines reactive, instrumental, and operational feedback
generated by the master-control, actuator, and slave components of the system (Figure 2). TheFigure has been modified (from [I],]8]) to include vision system components For a space telerobot[32].
238
demonstrate that all body movements are guided andcontrolled primarily by spatial feedback factors,with different movement systems mediated bydifferent properties and dimensions of spatialfeedback control. In general, tactile-kinestheticmechanisms (for velocity, acceleration, and positioncontrol) are integrated with the visual andvestibular mechanisms (for control of position,direction, range, extent, and pattern of movement inspace) in mediating the overall organization,predictive guidance, and integration of everymovement pattern of the body.
Several characteristics of Mosher's originalCAMs point up the importance of full spatialcompliance for effective telepresence. For example,one problem that emerged in the design of thebipedal walking machines was inability of theoperator to determine whether the machine was goingdown an incline versus falling backwards (or goingup an incline versus falling forwards). To resolvethis problem, Smith [21] demonstrated, first, thatthe operator's normal posture had to be maintainedwith respect to leg motion, and second, that theoperator had to be able to sense any displacement ofeither leg of the machine with respect to theposition of the machine cab. It was concluded thatthese cab-leg displacements had to be sensed by theoperator via active force and position feedback.
After appropriate torso-leg/cab-leg servo feedbackmechanisms were incorporated into the design of thewalking machine, it was found that even a noviceoperator could effectively control the dynamicbalance of the machine.
A second example of the importance of spatialfactors in telepresence emerged with research onmanipulator CAMs. Tests showed that the operator ofa manipulator experienced extension of his/herdynamic body image as long as just the manipulatorwas moved, but that this sensation was lost when themanipulator claw was moved. This loss oftelepresencewas related to the fact that the graspactions of the claw were not spatially compliantwith the operator's hand movements, although theforce compliances were very adequate.
The two examples cited above demonstrate thatbody image is defined in large part by the spatialorganization of body movement, and that the skill,precision, and safety of telerobotic operation areenhanced when the human operator perceives robotmovements as an extension of his/her own movements.The general conclusion is that telerobotic designmust incorporate compliances of both spatial andforce factors betweenoperator movements and machinemovements before effective workstation telepresencecan be established.
Although this conclusion is based on CAMresearch dating back two to three decades[17,18,21,22], we are not aware of more recentevidence which contradicts the insights it providesinto the human factors of workstation telepresence,or that superior approaches to telerobotic designand control have been devised. Indeed,a recognized
expert in the field recently referred to Mosher'smachines as still among the most sophisticatedtelerobots yet developed [30].
EFFECTS OF PERTURBED SENSORY FEEDBACK ON INTERACTIVEPERFORMANCE
The remainder of this report summarizes
scientific evidence from perturbed sensory feedbackresearch which indicates why spatially compliantfeedback is critical to workstation telepresence,and explores the implications of this evidence foroptimizing the human-factors design of interactive,telerobotic systems. Scientific interest in thephenomenon of perturbed sensory feedback dates backover a century to the early work of Helmholtz [33],who studied the effects of and adjustment to spatialdisplacements in visual feedback. Since then, theeffects on human performance of both spatial
perturbations and temporal delays in visual andauditory feedback have been the subject of numerousinvestigations, whose findings are summarized in aseries of reviews published over the past threedecades []4, 23-29,34].
In this report, the conceptual approachadopted to interpreting the performance effects ofperturbations in sensory feedback is grounded inbehavioral cybernetic theory. The basic postulateof this theory is that behavior is a closed-loop,self-regulated process mediated by motor control ofsensory feedback (12,14,23-29]. From a behavioralcybernetic perspective, sensation and perception arefeedback integrated with the motor system throughmotor coordinate feedback control of receptorprocesses and environmental stimuli. During taskperformance, the individual uses movement-basedmechanisms to control spatial, temporal, or othertypes of perturbations in sensory feedback which mayarise as a result of the human-factorscharacteristics of task design.
Since Helmholtz first posed the question, animportant issue in psychology has been whether thespatial organization of behavior is innate orlearned through experience. Although experientiallearning appears to definitely contribute to thespatial guidance of behavior, experimental findingsindicate that complete adaptation to perturbationsin sensory feedback rarely occurs (]4,23-29,34].Instead, evidence indicates that performance andlearning remain suboptimal in the face of persistentexposure to a perturbation condition. This suggeststhat the nature and degree of learning andperformance proficiency achieved for a particulartask depend upon the nature and degree ofspatiotemporal compliance between sensory input andthe motor feedback mechanisms controlling that inputduring task execution.
Figure 4 provides a diagrammatic illustrationof the concept that workstation telepresence iscritically dependent upon spatial compliance betweensensory feedback and its motor control mechanisms,using the same master control-slave telerobot modelas in Figure I. In Figure 4, however, visualfeedback has been spatially perturbed by reversingthe lateral motion relationships between arm-controlled movement of the control component andderivative movement of the slave component.Consequently, the operator experiences aperturbation in the normal manner in which visualfeedback from the te!erobot manipulator iscontrolled by arm movement, telepresence iscompromised, and performance suffers.
In general, four major classes ofperturbations in sensory feedback may be identified,namely: (]) spatial transformations and/ordisplacements; (2) temporal delays; (3) changes inenergy properties; and (4) modality conflicts. Forexample, control of visual feedback from a video-
239
/
/ // I!i
r/.,">,/ li:
Co!lll_l I
Coi11pOil ez_t ._iO =_r_ _
Figure 4. Spatial perturbation (reversed visualfeedback) in visual-manual operation of a telerobot
manipulator. Spatial compliance between movementsof telerobot slave components, and control movements
of the operator, represents one of the key humanFactors of workstation telepresence.
Figure S also assumes that visual-manualperformance is based upon use of the motor system
(19 principle motor control factors specified) tofeedback control the visual system (11 principlevisual perception factors specified), involving 5distinct modes of sensory feedback control [14,23-
2g]. This control process potentially may beinfluenced by a broad range of space telerobotic
system design factors (18 principle designconsiderations specified), some of which may giverise to the perturbation conditions indicated in the
figure. The system design, visual perception, andmovement control factors specified in Figure 5 are
compiled from basic human-factors considerationsregarding telerobotic operation [11], plus recent
analyses by various scientific specialists in spacetelerobotlc research [32,38-4I].
Some appreciation of the scientific challenge
confronting systematic investigation of the humanfactors of workstation telepresence may be gained by
considering that there are over 2.3 million possiblecombinations of performance, design, and feedbackfactors (14x3x3xlgxI1xSx18) which potentially caninfluence operational variability in vlsual-manual
control of a space telerobot, based onspecifications in Figure 5 and summarized above.That it probably is not feasible to completelyevaluate even a reasonable subset of such possiblecombinations raises the question of research
priorities and emphasis in human-factors analysis ofperformance variability in telerobotic operation.The hypothesis adopted in this report is that
spatiotemporal perturbations in sensory feedbackconstitute a major source of variability in humanperformance with automated and telerobotic systems,and that this issue therefore merits attention byhuman-factors research aimed at optimizing
workstation telepresence. Evidence in support ofthis hypothesis is summarized in the next section.
displayed image during remote telerobotic operationmay be perturbed in each of these ways by: (1)
inversion, reversal, angular displacement,magnification, or miniaturization of the displayedimage; (2) feedback delay of the real-time display;(3) modulation of display quality due to glare,brightness, reflectance, contrast, and/or frequency
shift effects; or (4) auditory and/or kinestheticinterference with visual feedback control. In
extraterrestrial environments, microgravityrepresents a fifth class of sensory feedback
perturbation which may exacerbate the effects ofother types of perturbations [2,35-37].
Figure 5 expands upon Figure 4 with aschematic summary of the major sets of factors which
potentially influence visual-manual task performanceand workstation telepresence during operatorinteraction with a space telerobot such as theSSFTS. The diagram indicates that interaction of
the operator (to the right) with the teleroboticsystem (to the left) across the operator-systeminterface is influenced by one or more of 14
possible perturbed sensory feedback factorsspecified in the figure and cited above. Theoperator-system interface specified in Figure 5assumes the feedback relationships depicted in
Figures 2 and 3, namely 3 sources of feedback(reactive, instrumental, operational) from the 3telerobotic subsystems (master-control, actuator,
slave).
Experimental Observations On Perturbed SensoryFeedback Effects
Over the past three decades, the most
extensive program of experimental research onperturbed sensory feedback phenomena has beenconducted by K.U. Smith and colleagues [14,23-
29,34]. This work has focused primarily upondelineating the effects of spatial and temporalperturbations in sensory feedback on learning and
performance. For this purpose, computer- and video-based techniques are used to introduce controlledspatial transformations or displacements, and/ortemporal delays, in sensory feedback (usually visualor auditory) provided to subjects of their own task
performance. A wide range of different cognitiveand motor behavioral skills have been examined in
this manner, including machine performance, tool
using, musical performance, reading, writing and
drawing, tracking and steering, speech, memory,eyemovement control, visual perception, motorcoordination, postural control, behavioral-
physiological integration, and social interaction[28,2g,42,43].
With its use of computer and video-display
techniques, this research essentially represents the
application of interactive, human-computerlaboratory methodology to the study of howperturbations and distortions in sensory feedback
240
influence learning, performance, and workstationtelepresence. However, rather than arising as anincidental and unwelcome concomitant of human-
factors defects in interface design, suchperturbations are introduced deliberately by thecomputer under controlled laboratory conditions, sothat effects can be assessed in an objective andquantitative fashion. The research approachtherefore provides a methodological paradigm forsystematic study of the contribution of sensoryfeedback perturbations to variability in humaninteraction with automated and telerobotic systems,in relation to human-factors design features of theinterface [II,14].
The general conclusion from the extensive bodyof research conducted using this approach is thatthe performance and learning of every behavioraltask so far examined is degraded by perturbations insensory feedback [14,28,29]. Tables I and 2 providerepresentative data on the nature and extent ofthese effects, for spatial transformations andtemporal delays in sensory feedback respectively.All data are derived from controlled laboratorystudy of subject interactionwith computer-or video-based systems. In both tables, performance changesfor a series of tasks under specified perturbationconditions are summarized, along with informationregarding experimental conditions and referencecitations (first column).
Under spatially perturbed visual feedbackconditions, results for various measures of sevendistinct types of tasks are given in Table I (thirdcolumn), namely writing, drawing, reading, assembly,panel control, tool using, and tracking tasks. Thespatial perturbation conditions examined (secondcolumn) encompass inversion, reversal, combinedinversion and reversal, angular displacement, and
size reduction (miniaturization)of visual feedback.The measures used to assess performance (fourthcolumn) vary for different tasks, but generally fallinto the categories of movement characteristics(contact, travel, grasp, or assembly times), orperformance accuracy. Results (last column) areexpressed as percent change in performance, from thecontrol to the spatial perturbation condition, atthe start and at the end of a training period (fifthcolumn) which typically involved one or more trialsessions per day over a series of days. An asterlckis used in the last column to denote the statistical
significance of the performance change observed.Results for the feedback delay studies in
Table 2 are summarized in much the same manner.These studies examined writing, drawing, tracking
(manual, eye, and head movements), speech, memory,and interactive social tracking tasks (secondcolumn), with performance again assessed usingeither movement characteristics or accuracy measures(third column). Results (last three columns) arepresented as the change in performance, for delay ineither visual or auditory feedback, at delayintervals of 0.2 seconds, 0.4 seconds, and the
RCEPTIO_
Figure 5. Human factors which may influence workstation telepresence in space teleroboticoperation. Operator performance involves motor control (movement and sensory feedback controlfactors) of visual feedback (visual perception factors), which is influenced by both systemdesign and perturbed visual feedback factors.
241
maximum delay interval examined in the study,relative to performance under the no delaycondition. Performance change values for the delayconditions are expressed as multiples of the zero-delay levels. In all of the studies cited in Table2, analysis of variance was used to assess theoverall contribution of the delay condition to theperformance variability observed. In every case,the overall contribution was found to bestatistically significant, but tests of significanceat individual delay levels were not performed.
Results summarized in Tables ] and 2, plusfindings from an extensive body of related research,support the following conclusions regarding theeffects of spatial and temporal perturbations insensory feedback on interactive performance [i4,23-29].
1. Performance and learning of every behavioraltask so far examined ate degraded by suchperturbations. Consistent patterns and features ofbehavioral disturbance appear from study to study.Oscillatory instability in movement control becomesmore pronounced, accompanied by increasedvariability and extremes in movement velocities andaccelerations. Accuracy of movement guidance andtracking declines. Perception is disturbed and, insome cases, may disappear entirely; there is aconcomitant deterioration in learning. Subjectsreport feeling confused, uncertain, and/oruncomfortable about their behavior. Skilled
performers are particularly sensitive to theseeffects.
Table I. Changes Inlnteractive Performance and Learning Under Spatially PerturbedVisual Feedback
Reference-Sub.iects
[44!-12n
M
i!
||
[4s!- 4N
||
[241 - lo(p. 1631
I!
w!
||
PerturbationCondition z Task Measure
I .Hand- .Contact
" writing Time" .Travel" " Time
" .Drawin9 .Contact
" Triangles Time" .Travel" " Time" .Reading .Amount" Words Read
I .Reading .Time x• " ErrorR " Score
I-R .Assembly .Grasp" " DurationN I!
11 I!
" " .Assembly" " Duration" " .Nonloaded" " Travel
Duration
24] - 24 I
p..1711 RI-R" I
" R
" I-R" I
" R" I-R
" R
" I-R
[24] - 24 _ I(p. 172) R
W I-R" I" R" I-R
Days ofTraining Performance(Trials)3
0 (I) + 676 % *20 (20) + 153 *0 (I) + 410 *
20 (20) + 80 (1) + 493 *
20 (20) + 71 *0 (I) + 583 *
20 (20) + 330 (1) 81 *5 (5) 12 *
0 (I) + 253 *
5 i 5) + 210 1) + 210 *5 (5) + 6
0 (I) + 5 *4 (4) + 4 *
+ 24 *+ 6
0 (I) + 92 *4 (4) + 60 *0 (I) + 27 *4 (4) + 8
.Loaded Tra- 0 (I)vel Duration 4 (4)
.Hand- .Contact 2 ! 4) + 286 *writing Time + 154 *,' • " + 250 *" .Travel " + 240 *" Time " + 160 *" " " + 200 *
.Drawin9 .Contact " + 171 *Trlangles Time " + 80 *
" " " + ]05 *" .Travel " + 139 *" Time " + 72 *" " " + 94 *
.Haze .Accuracy 0 ! 1)Tracing . ,,
" " 3(91ml I!
72 *51 *3g *37 *15 *25 *
242
Table 1. (continued)
Days of
Reference- Perturbation Training Performance
Subjects i Condition 2 Task Measure T_Trials)3 _e(%__
[24] - 18 .Size .Panel .Contact 0 (I) + ]08 *(p. 218) Reduction Control Time 5 (40) + 104 *
" (to I/3 size) " .Travel 0 (I) + 232 *...... Time 5 (40) + 267 *
[24] - 24 I Tool Using:(p. 239) " .Dial .Manipulation 0 (I) + 273 *
.... Setting Time 7 (21) + I]7 *
.... .Turning " 0 (1) + 191 *
....... 7 (21) + 78 *.... .Pushing " 0 (1) + 200 *....... 7 (21) + 100 *
.... .Pressing " 0 (1) + 250 *
........ 7 (21) + 117 *
.... .Dial .Travel 0 (1) + 257 *
.... Setting Time 7 (21) + 29 *
.... .Turning " 0 (I) + 209 *
........ 7 (21) + 40 *
.... .Pushing " 0 (I) + 218 *
....... 7 (21) + 36 *
.... .Pressing " 0 (I) + 230 *
........ 7 (21) + 40 *
[46] - 24 AD:
" 20 degrees .Maze .Errors 0 ! 1) + 6
" 40 degrees T_acing " + 5" 60 degrees .... + 23 *
" 10 degrees " " 10 !10) - 2" 20 degrees .... 9" 30 degrees ', " " + 2" 40 degrees ..... + 7
" 50 degrees ...... + 63 *" 60 degrees ..... + I48 *" 70 degrees ...... + 287 *
................................................ .................... - ...... _ .................
*Change from nonperturbed visual feedback performance level is statistically significant
(p<.05).IReference in brackets, followed by number of subjects.
21 = Inversion; R = Reversal; I-R = Inversion-Reversal; AD = Angular Displacement (in
degrees).3Number of trials in parentheses.4Performance change, in percent, calculated as:[(perturbed viewing level - nonperturbed viewing level) x 100] / (nonperturbed viewing level)
2. For untrained subjects, spatial perturbations in of 43 cases in which training was provided for 2visual feedback have profound effects on task days or longer. A similar pattern has been observed
performance. In Table I for example, there are in experiments where subjects were exposedstatistically significant performance decrements for continuously to spatially perturbed visual feedback24 of 27 no training cases (fifth column, trial 1, for periods of time lasting days or weeks [24,34].
0 days training), and in 20 of these cases theperformance decrement exceeds 50 percent. 4. The training findings summarized in Point 3
suggest that human learning is refractory to
3. After training, subjects typically exhibit complete adaptation to spatial perturbations inimproved performance under spatially perturbed visual feedback. Neurobiological corroboration of
visual feedback conditions. In Table I for example, this conclusion is provided by recent researchthere are 27 instances in which pro- and post- [52,53] on motor learning in the primate vestibulo-
training performance levels can be compared, and ocular reflex (VOR). Under continuous exposure toperformance improved in 25 of these cases, size distortion (magnifying or miniaturizing
Nevertheless, with rare exceptions, under spatially spectacles), both the reflex gain and activity ofperturbed visual feedback subjects do not perform at brain stem neurons mediating the reflex adjustthe same level as under nonperturbed conditions, accordingly. However, the gain error increases fromeven after a training regimen lasting as long as 20 about zero, with no distortion, to 10-20 percentdays. In Table I for example, statistically under the distortion condition. This lack of
significant performance decrements persisted for 33 complete adaptation to size perturbation may explain
243
Table 2. Changes in Interactive Performance Under Feedback Delay I
Reference-
Subject@ z
[231- 2
II
It
[47]- 8
[48]- 8
[49]- 12
[50] - 8
[S]] - 2
Task Measure
.Writing .Contact TimeLetters
.Drawing "
.Star Tracing "
.Maze Tracing "
.Head Movement .TrackingTracking of ErrorVisual Target
Performance Change (xl)at Specified Delay 3
0.2 sec 0,4 @eC Max (sec)4
+ 2.4x (.52)
+ I.Ix (.52)
+ 4.6x (.52)+ 5.0x (.52)
+ ].3x + 1.6x + 2.3x (0.8)
[auditory delay interval s : O.I secl 0.2 secl]
.Speech .Speech Errors + 4.]x I +]1.]x I +14 x (0.3)
Tracking of Visual Target:
.Eye Movement .Tracking. + ].2x + ].2x + 2.0x (1.6)
.Head Movement Error + 1.2x + 1.2x + 5.0x (1.6)
.Head-Eye " 0 x + 1.2x + 1.9x (1.6)Movement
.Memory of .Recall + ].Sx + 1.9x + 3.3x (0.8)Visual Image Error
.Visual-Manual .TrackingTracking Error
-Individual: " + 1.]x + 1.5x + 2.6x (].5)-Social : " + ].5x + 1.8x + 3.0x (].5)
.............................................................................................
IAll studies examined visual feedback delays, except for Abbs and Smith [48], who examinedauditory feedback delay.
?Reference in brackets, followed by number of subjects.
3performance change, in multiples of one (indicated by small x), observed at delay intervalsof 0.2 sec, 0.4 sec, and maximum delay interval examined _. Performance change is calculatedas:
(Delayed Performance LeveIJ / (No Delay Performance Level)4Performance change, in multiples of ones, at maximum feedback delay interval examined in
study. The maximum delay interval examined, in seconds, is indicated in parentheses.SDelay intervals of 0.I sec, 0.2 sec, and 0.3 sec examined in this study.
the finding cited in Table 1 that performance byhuman subjects in an interactive panel control taskremained significantly impaired after 5 days of
training with use of miniaturizing spectacles thatprovided a two-thirds size reduction [24, p. 218].
5. Three experiments described in Table I [24, pp.171-172; 45] directly compared the effects ofinversion, reversal, and combined inversion-reversalof visual feedback. The results indicate that
inversion has the most adverse impact onperformance, followed in order by combinedinversion-reversal, and reversal alone. Other
research also supports this conclusion [25].
6. Under conditions of angular displacement of
visual feedback, as indicated in the last studycited in Table I [46], performance remainsrelatively unaffected until a displacement angle of
50 degrees is imposed. This finding has beeninterpreted as providing evidence for a breakdown
anqle of spatial displacement, beyond whichperformance becomes progressively more difficult.Typically, the breakdown angle falls in the range of40 to 50 degrees displacement. Results in Table I
indicate that even after IO days of training,performance under displacement conditions exceeding
the breakdown angle remains significantly impaired.
I. Relative to direct viewing conditions,performance on an assembly task is impaired undertelevision viewing of the assembly operation, evenwhen no deliberate spatial distortions are
introduced [24, pp. 161-163]. This may beattributed to inherent spatial distortions in visualfeedback introduced by the video camera and displaysystem.
8. Performance under spatially perturbed visualfeedback varies as a function of both age and gender[45,54].
9. Temporal perturbations (feedback delays) insensory feedback also result in consistent
decrements in performance, as illustrated by thefindings cited in Table 2. Performance decrements
at a delay interval of 0.2 sec occur for eight ofnine tasks listed in the table. At delay intervalsof 0.3 sec and greater, all 13 tasks listed in the
table manifest performance decrements. For speech,a fourfold increase in errors is observed at an
auditory feedback delay of O.I sec [48].
244
10. Trainingeffect dataarenot includedin Table2 becausethe studiesthat havebeendone(onbotheyemovementtrackingandmemory)indicatelittle orno performanceimprovementwith learning underfeedbackdelayconditions[47,55].11.Whenvisual feedbackis bothtemporallydelayedandspatially reversedin either aneye-or a head-movementtracking task, the delayexacerbatesthedecrementin trackingperformanceproducedby thereversalcondition[47,55].12.Socialtrackinginvolvesthemutualexchangeandcontrol of sensoryfeedbackamongtwo or moreindividuals during group activity [13,28,56].Salientfindingsregardingsocial trackingarethat:(a) relative to visual-manualsocial tracking,accuracyis higherfor tactile or auditorysocialtracking {57]; (b) social tracking accuracyiscomparableto individual tracking accuracyexceptunderfeedbackdelayconditions,whenthe latter issuperiorto theformerat all delaylevels [5]]; (c)learningof socialtrackingtasksis limited, highlyvariableandinconsistent,andunstable[58,59];and(d) thepreponderanceof performancevariability insocial trackingis attributable, not to a learningeffect, but to human-factorscharacteristicsof taskdesign{60].
SpaceTeleroboticTelepresence OperationalImplicationsof Perturbationsin SensoryFeedback
A majorassumptionof this report is that thefindingsandconclusionssummarizedin theprecedingsection have direct relevanceto understandingpossibleeffects of perturbedsensoryfeedbackoninteractive performance in extraterrestrialenvironments.Forexample,manyof thetaskslistedin Tables] and2 (reading, tracking, assembly,panelcontrol, tool using, speech,memory,socialtracking) are employedduring interactive humanperformancewith spacetelerobotic systems.Smithandcolleagues{61] haveaddressedthis question,andconcludethat perturbedsensoryfeedbackeffectscouldhavea potentially decisiveinfluenceon theperformanceof useful workwith man-in-the-loopspacetelerobotics. Someexperimentalevidencefromearth-based,human-factorsresearchon teleroboticsystemslends support to this conclusion,assummarizedbelow.
Establishmentof high fidelity workstationtelepresencefor spaceteleroboticoperatorswillrequire that inappropriateor hazardoussensoryfeedbackfromthe spaceenvironmentbe eliminated(i.e., direct sunlight, glare, cold, lackof air),whereasvisual, tactile, kinesthetic,andauditoryfeedbackessential to manipulativeperformanceisprovidedwhichconformsto thecontrol capabilitiesof the operator. Duringperformanceof a remotemanipulationtaskwith a telerobotin space,it maybe possible to view the remotescenedirectlythroughawindow.Formosttasks,however,a directview either will not be available, or will notprovidethe appropriatevisual cuesnecessaryforteleoperation(dueto distanceand/orinterpositionof other objects). For this reason, externalcameraswill providethe primarysourceof visualfeedbackaboutmanipulatorposition, orientation,andrate of movement[62].
Variousstudiesof viewingsystemrequirementsfor space telerobotic systemsindicate thatperturbationsin Visual feedbackmayrepresentapervasivefeatureof teleoperation. A numberofauthorsagree[38-41]that visualguidanceof spacestation telerobotic tasks will be influencedbyextreme illumination, contrast, glare, andreflectanceconditionsknownto exist in space. Inaddition,recentresearchbyR.Smithandcolleagues[32,39,62,63]hasaddressedthe questionof howspatial perturbationsin visual feedback,producedbycamera-inducedtransformationof themanipulatorreferenceplanerelative to that of the operator,could degradetask performanceof the telerobotoperatorandtherebyrisk damageor loss of themanipulatoror payload. In particular, spatialperturbationsof visual feedbackto the operatorfroma spacetelerobotcanbepredictedasa resultof: (]) variable orientations of surveillancecamerasonthe stationgridworkandmodules,andonthetelerobot; (2) lackof a fixedgroundreference;(3) abnormalor missingdepthcues;and(4) extremeviewingconditions,asnotedabove.
Decrementaleffectsof spatial transformationsin visual feedbackon remoteoperation of atelerobothavebeenconfirmedin arecentlaboratorystudy by Stuart and Smith[63], whoshowthattelerobotmanipulationtimesare: (1) significantlyslowerfor video-displayviewingof the worksite,relative to direct viewing;and(2) slowedmostbyinversion of visual feedback,with reversalandinversion-reversalconditionshavingless of aneffect. In particular, theseresearchersfoundthatperformancecompletiontimes in a manipulatorpositioning task wereincreasedby the followingamounts,relative to direct viewingof the task:(I) display viewing,2.0-fold increase(no otherspatial perturbation); (2) inverted-reversedviewing,8.5-fold increase;(3) reversedviewing,lO.2-foldincrease;and(4) invertedviewing,16.1-fold increase. As indicatedin TableI, thesefindings recapitulatethe earlier observationsofK.U.Smithandcolleagues[24], in whichthe sametechniquesof videocameradisplacementandrotationwereusedto spatially perturbvisual feedbackofthe taskprovidedto sub.iects.
Alsorecapitulatingearlier observations[23]are results fromrecentresearchonoperationof alaboratoryversionof a telerobot servicer []5],whichdemonstratethat delaysin visual feedbacksignificantly increasemanipulationperformancetimes. The earlier researchdocumentedthedetrimentaleffectsontrackingperformanceof delayintervalsrangingfromoneto threeseconds(in thesamerangeasthe round-triptransmissiondelayfora signalfromanearth-boundoperatorto anorbitingor moon-basedtelerobot), andwasamongthefirst topoint out that effective guidanceof a remotetelerobotwouldbeseriouslycompromisedby delaysin visual feedback[23, pp. 94-96].
A further important consideration inestablishingtelerobotworkstationtelepresenceisthat of sensoryfeedbackmodalityandits control.Forexample,in anexperimentalstudyof visual andtactile social tracking[25], it wasshownthat anobservercantrack anirregularlymovingobjectmoreaccurately by touching it (tactile-kinestheticfeedback)than by watchingit (visual feedback).The difference is attributable to the slowerfeedbackrelationships of the visual systemas
245
compared with those of the tactile-kinesthetic
system. There also is the question of modalitycompatibility between sensory feedback and sensoryfeedback control. For example, feedback of forceinformation from a telerobot force transducer to a
hand controller, rather than as visual or auditoryfeedback, is compatible with human kinesthetic
senses that are directly interpretable.An appreciation of the potential for
spatiotemporal perturbations in sensory feedback
during SSFTS operation may be gained from aconsideration of current system requirementspecifications [31], as embodied in the general
system diagram in Figure 3. Vision systemspecifications call for at least four telerobot-mounted video cameras, including one on themanipulator wrist, with separate positioning,orientation, zoom, focus, and aperture control
capabilities indicated for each camera. Variablecontrol of attitude and imaging parameters for boththe telerobot-mounted and external cameras, coupled
with possibilities for variability in task board andtelerobot manipulator geometry, introduce avirtually unlimited potential for reversal,inversion, angular displacement, magnification,
and/or miniaturization (Figure 5) of the displayimage presented as visual feedback to the humanoperator.
SSFTS manipulator specifications call For
kinesthetic feedback of force, position, and rateinformation from sensors at each joint [3],64]. Forpurposes of task execution, the specifications alsocall for an orientation accuracy of less than ].0inch in manipulator tool plate center position,
repeatable to an accuracy of less than 0.005 inch.Howe_2r, findings summarized in Tables I and 2, plusobservations from more recent perturbed sensoryfeedback research on telerobots [15,63], suggestthat such accuracy may be compromised by theoccurrence of any type of spatiotemporal
perturbation in visual feedback. Under earthlaboratory conditions, such perturbations causeerrors in movement trajectory guidance plusincreased oscillatory instability in staticpositioning control [23-2g,64]. In addition,experience from earlier work on Mosher's CAM systemsJill, summarized previously, suggests that it may be
necessary to supplement kinesthetic and visualfeedback with tactile feedback From manipulator endeffectors in order to achieve a high degree ofreproducible accuracy in control of manipulator
positioning.
OPTIMIZING WORKSTATION TELEPRESENCECYBERNETIC PERSPECTIVE
A SOCIAL
The analysis in the preceding sectionssuggests that effective workstation telepresencedepends on provision of compliant sensory Feedbackacross different modal]ties which can be readily
controlled by the operator. Because they cancompromise such compliance, perturbations in sensoryfeedback may significantly impair the operationaleffectiveness of space telerobotic systems, inrelation to visual-manual control of manipulator
functions. This raises the question as to whatstrategies might prove useful in reducing possibleadverse effects of sensory feedback perturbations on
human performance in space, in order to achieve high
fidelity workstation telepresence. Training is oneobvious choice. However, evidence cited in Tables
] and 2 suggests that even extended training willnot completely overcome these effects. It maytherefore prove desirable, or even necessary, to
develop techniques which enable the interactivecomputer to detect, and then correct, perturbations
in sensory feedback before presentation of suchfeedback to the human operator.
The idea of enlisting the workstation computeras an intelligent co-participant in optimizingworkstation telepresence is in line with proposalsby various authors that the design of spacetelerobotic systems should evolve from primaryreliance on operator control to reliance on both
operator and autonomous, computer-mediated (robotic)control of task execution [1-3,65-67]. To providea conceptual foundation for dealing with the humanfactors of workstation telepresence at various
levels of interactive complexity, we have developeda social cybernetic model of human-computerinteraction []2-14]. The model, shown in Figure 6A,assumes that behavioral cybernetic principles ofhuman social interaction [28,56,58-60] also can beapplied to an understanding of human-computerinteraction.
The term social tracking used in Figure 6Arefers to the feedback-controlled process by whichan individual engaged in social behavior follows or
tracks one or more social targets. During socialtracking, the activities of one person in a socialgroup effect behavioral-physiological changes in theother social partners, whose own activities in turnhave behavioral-physiological feedback effects onthe first. These effects arise as a consequence of
control by each participant of sensory feedbackgenerated by the other social tracking partners.During interpersonal social tracking for example,the movements of one individual generate sensoryFeedback that is controlled by tracking movements of
the social partner, whose actions generate compliantsensory feedback for the first individual, who inturn tracks this social feedback with further
movement, and so forth. The two social partnersthus become dynamically yoked or interlocked
behaviorally and physiologically, through mutualtracking and control of sensory feedback generatedby each other's social behavior. Through such
social tracking interlocks, participants in a socialgroup begin to operate as an integrated system, withdefinite systems feedback parameters and controlcharacteristics.
The premise of the model in Figure 6A is thatthe interactive computer can be considered as a
machine analog of a human social partner, and can beimbued through adaptive programming with
capabilities For tracking and controlling, acrossdifferent modal]ties, sensory feedback generatedboth by its own activities as well as those of itshuman partner. Unfortunately, because oftechnological and design limitations in such
capapabilities, today's interactive computer systemis a relatively limited and impoverished socialtracking target for the human operator. _roaalyspeaking, the creation of effective workstationtelepresence requires that such limitations beovercome.
A number of authors have suggested that thequality and nature of crew social interaction are animportant determinant of operational effectiveness
246
COMPUTER INTERACTION PER$ NIcOyOTE.I ,NTERACT,ONI O 1L. I _ S'e_,oryI
Ic uu1_l Oulpul Feedback Sensoryeedbad_ Feedback Con o Feeo'oad( oo,,o,J• T U
I °°'°°'°'"°' 1
SOCIAL CYBERNETIC
TRACKING MODEL
F'S "YUoTUADL
Figure 6. The social cybernetics of workstation telepresence. A. The social cybernetic modelof human-computer interaction, which assumes mutual social tracking by operator and machine ofsensory feedback generated by the other partner during interactive performance. B. Possiblesocialcybernetic interactionsduring SSFTSoperation. The diagram assumes that the SSFTS (whichmay be controlled by more than one operator) is conveyed to the task site by an OrbitalManeuvering Vehicle (OMV), controlled by a separate operator. The arrows symbolize possiblesensory feedback exchange and control relationships between operator and machine participantsin the control system. Also indicated are the most likely modes of social tracking for eachrelationship [13].
and safety in space [68]. Factoring possible human-computer interactions into the soclal trackingmatrix further complicates the social cybernetics offlight crew performance. Figure 6B illustratesthispoint with a diagram of some of the interpersonaland human-computer social interactionswhich may berequired for SSFTS operation. Current plans [1-3]call for the SSFTS to be conveyed to its tasklocation by a transport device called the OrbitalManeuvering Vehicle (OMV). It is envisioned thatthe OMV and the SSFTS each will be controlled by aseparate operator. In addition, currentspecifications [3]] call for monitoring oftelerobotlc activities by multiple workstations(such as on the space station and shuttle), with thecapability of handing control from one workstationto another. The various mutual social trackinginteractionssuggested by these plans are indicatedin Figure 6B. The diagram also indicates thespecific type of social tracking []3,56] which ismost likely to be employed for the relationshipspecified.
The general implication of the diagrams inFigure 6 Is that an analysis of the human factors ofworkstationtelepresencerequires consideration, notonly of the social tracking fidelity for aparticular operator-workstation-telerobot system,but also of the social tracking relationshipsbetween all of the players, human and computer,involved in system operation. That high fidelitysocial tracking requires effective control ofsensory feedback from two sources---generated byself-activity and by activity of the socialpartner(s)---suggests two broad themes for humanfactors and engineering research aimed at improvingworkstation telepresence [12,13].
First, computer capabilities for detecting andadaptively controlling sensory feedback from its ownsensors need to be enhanced. One useful outcome of
such research could be automated, computer-medlatedtransformation of spatially perturbed images fromcameras linked to the computer (see Figure 3).Secondly, computer capabilities for detecting andcontrolling sensory feedback from the human operatorneed to be enhanced. One useful outcome of suchresearch could be improved techniques for automateddetection of behavioral and physiologicalmanifestations of operator decremental performanceunder spatiotemporally perturbed sensory feedbackconditions, signalling need for computer-mediatedcorrection of the perturbation. The human factorsof workstation telepresence will benefit from thedevelopment of more robust strategies for automatedcontrol of sensory feedback, growing out of researchon the social cybernetics of human-computer systems,which enable improved system management ofspatiotemporal perturbations in sensory feedback.
SUMMARY
This report has presented a human-factorsanalysis of workstation telepresence which supportsthe following major conclusions:
I. Pioneering human-factors research on operationalrequirements for cybernetic anthropomorphicmachineshas established that spatial compliance betweenpostural, transport, and manipulative movements ofthe operator and those of the CAM is essential tothe creation of high fidelity workstationtelepresence.
247
2. An extensive body of behavioral cyberneticresearch evidence indicates that spatial andtemporal perturbations in visual and auditoryfeedback degrade learning and performance of everybehavioral task so far examined. The findings havedirect relevance to the human factors of human-computer interaction, in that the research itselfmade use of interactive computer- and video-basedmethods to deliberately introduce and evaluate theeffects of such perturbations.
3. Although training improvements occur, thisresearch has found no evidence to indicate thathumans can adapt perfectly to spatial and/ortemporal perturbations in sensory feedback which arenot compliant with established motor feedbackcontrol mechanisms.
4. Because of noncompliant human-factors designfeatures, sensory feedback perturbations ariseroutinely in the course of operational humaninteraction with telerobotic systems. Performancedecrements even occur when video display viewing ofa task is substituted for direct viewing.
5. A number of different modes and sources ofsensory feedback perturbations can be identifiedwhich may potentially influence interactive humanperformance during space telerobot operation.Results from recent laboratory research using remotetelerobot servicers show that manipulation tasks areadversely affected by both spatial perturbations andtemporal delays, confirming in part the findingsfrom earlier behavioral cybernetic research.
6. The human factors of workstation telepresenceencompass the distinct social dynamic attributes oftelerobotic systems, involving mutual sensoryfeedback exchange and control relationships amongmultiple human operators, computers, and telerobots.A social cybernetic model of human-computerinteraction is described which uses social trackingconcepts to characterize the sensory feedbackcontrol relationships among human and machineparticipants in complex, interactive systems.Current plans suggest that such social relationshipswill be an integral part of space stationteleoperations, where station-, shuttle-, andground-based systems may all be involved inmediating telerobotic tasks. One projection Fromthe social cybernetic model is that a researchemphasis on improving the capabilities of theinteractive computer for controlling sensoryFeedback, from its own performance as well as Fromthat of the human partner, will benefit thedevelopment of workstation telepresence.
REFERENCES
].
.
Purves, R.B., Lin, P.S., and Fisher, E.M.,Jr., "A Plan for Time-Phased Incorporation ofAutomation and Robotics on the U.S. SpaceStation," SECOND CONFSRENCE ON ARTIFICIALINTELLIGENCE FOR SPACE APPLICATIONS, NASAScientific and Technical InformationDivision,Washington, D.C., ]988, pp. 237-245.Varsi, G., "Telerobotics for the EfficientUtilization of Space," PROCEEDINGSOF THE 3gTHCONGRESS OF THE INTERNATIONAL ASTRONAUTICALFEDERATION, International AstronauticalFederation, Paris, 1988, IAF-88-023.
3. Varsi, G., and Herman, D.H., "Space Station asa Vital Focus for Advancing the Technologiesof Automation and Robotics," SECOND CONFERENCEON ARTIFICIAL INTELLIGENCE FOR SPACEAPPLICATIONS, NASA Scientific and TechnicalInformationDivision, Washington, D.C., 1988,pp. I-6.
4. Burke, J.D., "Lunar Resources: Evaluation andDevelopment for Use in Lunar Base Program,"PROCEEDINGS OF THE 3gTH CONGRESS OF THEINTERNATIONAL ASTRONAUTICAL FEDERATION,InternationalAstronautical Federation,Paris,1988, iAF-88-586.
5. Burke, J.D., "Precursors and Adjuncts of aLunar Base," PROCEEDINGS OF THE 3gTH CONGRESSOF THE INTERNATIONALASTRONAUTICAL FEDERATION,International Astronautical Federation,Paris,1988, IAF-88-616.
6. Podnieks, E.R., "Environmental Considerationsfor Lunar Base Engineering," ENGINEERING ANDCONSTRUCTION AND OPERATIONS IN SPACE,Aerospace Division/ASCE, Albuquerque, NM,]988, pp. 55-66.
7. Podnieks, E.R., and Roepke, W.W., "Mining forLunar Base Support," LUNAR BASES AND SPACEACTIVITIES OF THE 21ST CENTURY, W.W. Mendell(Ed.), Lunar and Planetary Institute, Houston,I985, pp, 445-452.
8. Podnieks, E.R., and Roepke, W.W., "MiningTechnology Requirements for Lunar ResourceDevelopment," PROCEEDINGS OF SYMPOSIUM '86 -THE FIRST LUNAR DEVELOPMENT SYMPOSIUM, LunarDevelopment Council, Pitman, NJ, ]987, pp.$86-38A to $86-38H.
9. NASA Advanced Technology Advisory Committee,"Advancing Automation and Robotics Technologyfor the U.S. Economy, Volumes I and II," NASATM-87566, NASA, Washington, D.C., 1985.
10. Shaw, J.A., "Reducing Operator Error inDistributed Control Systems", INTECH, March,]988, pp. 45-50.
11. Smith, K.U., and Smith, T.J., "Analysis of theHuman Factors in Automation," SUCCESS FACTORSFOR IMPLEMENTING CHANGE A MANUFACTURING
VIEWPOINT, K.M. Blache (Ed.), Society ofManufacturing Engineers, Dearborn, MI, 1988,pp. 259-338.
12. Smith, T.J., and Smith, K.U., "Motor FeedbackControl of Human Cognition- Implicationsforthe Cognitive Interface," SOCIAL, ERGONOMICAND STRESS ASPECTS OF WORK WITH COMPUTERS, G.Salvendy, S.L. Sauter, and J.J. Hurrell(Eds.), Elsevier, Amsterdam, 1987, pp. 239-254.
13. Smith, T.J., and Smith, K.U., "The SocialCybernetics of Human Interaction withAutomated Systems," ERGONOMICS OF HYBRIDAUTOMATED SYSTEMS I, W. Karwowski, H.R.Parsaei, and M.R. Wilhelm (Eds.), Elsevier,Amsterdam, 1988, pp. 691-711.
14. Smith, T.J., and Smith, K.U., "BehavioralCybernetic Basis of Cognitive Performance.Experimental and Theoretical Analysis,"ERGONOMICS (Special issue on 'MethodologicalIssues in Cognitive Ergonomics'), 1989, InPress.
15. Collins, S.L., and Purves, R.B., "RemoteServicing of Space Systems," PROCEEDINGS OFTHE SECOND CONFERENCE ON ARTIFICIALINTELLIGENCE FOR SPACE APPLICATIONS, NASA,
248
Washington, D.C., ]988, pp. 523-535.16. Keslowitz, S., "Open Control/Display System
for a Telerobotics Work Station," PROCEEDINGSOF THE GODDARD CONFERENCE ON SPACE
APPLICATIONS OF ARTIFICIAL INTELLIGENCE,NASA,Washington, D.C., 1987.
17. Mosher, R.S., "Industrial Manipulators,"SCIENTIFIC AMERICAN, Vol. 2II, No. 4, October,]964, pp. 88-98.
I8. Mosher, R.S., and Murphy, W., "Human ControlFactors in Walking Machines," PROCEEDINGS OFAMERICAN SOCIETY OF MECHANICAl EV_GINEERSHUMAN
FACTORS CONFERENCE, Chicago, 1965.19. Ruch, T.C., Patton, H.D., Woodbury, J.W., and
Towe, A.L., NEUROPHYSIOLOGY, Saunders,Philadelphia, ]965.
20. Rahimi, M., "Human Factors Engineering andSafety in Robotics and Automation," HUMANFACTORS SOCIETY BULLETIN, Vol. 30, No. 7,1987, pp. 3-5.
21. Smith, K.U., HUMAN FACTORS ANALYSIS OF THEMECHANISM OF WALKING WITH SPECIAL REFERENCE TO
DESIGN OF A PEDIPULATORVEHICLE, University ofWisconsin Behavioral Cybernetics Laboratory,Madison, WI, 1965.
22. Smith, K.U., SENSORY-FEEDBACK ANALYSIS OFMANLIKE MACHINE SYSTEMS: A NEW BEHAVIORALTHEORY FOR HUMAN ENGINEERING, University ofWisconsin Behavioral Cybernetics Laboratory,Madison, WI, 1965.
23. Smith, K.U., DELAYED SENSORY FEEDBACK ANDBEHAVIOR, Saunders, Philadelphia, 1962.
24. Smith, K.U., and Smith, W.M., PERCEPTION ANDMOTION: AN ANALYSIS OF SPACE-STRUCTUREDBEHAVIOR, Saunders, Philadelphia, 1962.
25. Smith, K.U., and Smith, M.F., CYBERNETICPRINCIPLES OF LEARNING AND EDUCATIONALDESIGN,Holt, Rinehart and Winston, New York, 1966.
26. Smith, K.U., "Cybernetic Psychology," THEPSYCHOMOTOR DOMAIN, R.N. Singer, Ed., Lea andFebiger, New York, 1972, pp. 285-348.
27. Smith, T.Jo, and Smith, K.U., "CyberneticFactors in Motor Performance and Development,"DIFFERING PERSPECTIVES IN MOTOR LEARNING,MEMORY, AND CONTROL, D. Goodman, R.B. Wilberg,and I.M. Franks, Eds., Elsevier, Amsterdam,1985, pp. 239-283.
28. Smith, T.J., and Smith, K.U., "Feedback-Control Mechanisms of Human Behavior,"HANDBOOK OF HUMAN FACTORS, G. Salvendy, Ed.,Wiley, New York, 1987, pp. 251-293.
29. Smith, T.J., and Smith, K.U., "The CyberneticBasis of Human Behavior and Performance,"CONTINUING THE CONVERSATION. A NEWSLETTER OFIDEAS IN CYBERNETICS,Winter, 1988, Number 15,pp. 1-28.
30. Raibert, M.H., "Machines That Run," ROBOTICSAND MOTOR CONTROL: COMMONALITIES ANDDIFFERENCES, Conference Paper No. 5, SimonFraser University, Burnaby, B.C., Canada, June3, 1987.
31. NASA Goddard Space Flight Center, "SpaceStation Flight Telerobotic ServicerRequirements," FTS REQUIREMENTS DOCUMENT FORPHASE C/D, NASA RFP SS-GSFC-O043, Nov. I,1988, Section 3.0.
32. Smith, R.L., "Human Visual Requirements forControl and Monitoring of a Space Telerobot,"ERGONOMICS OF HYBRID AUTOMATED SYSTEMS I,W. Karwowski, H.R. Parsaei, and M.R. Wilhelm
(Eds.), Elsevier, Amsterdam, 1988, pp. 233-240.
33. Helmholtz, H.L.F. von, HANDBUCH DERPHYSIOLOGISCHEN OPTIK, 3 Volumes, Voss,Hamburg and Leipzig, 1856-1866.
34. Welch, R.B., PERCEPTUAL MODIFICATION.ADAPTING TO ALTERED SENSORY ENVIRONMENTS,Academic Press, New York, 1978.
35. Howard, I.P., and Templeton, W.B., HUMANSPATIAL ORIENTATION, Wiley, New York, 1966.
36. Young, L.R., Oman, C.M., Watt, D.G.D., MoneyK.E., and Lichtenberg, B.K., "SpatialOrientation in Weightlessness andReadaptation to Earth's Gravity," LIFESCIENCES, Vol. 225, 1984, pp. 205-208.
37. Vasyutin, V.V., and Tishchenko, A.A., "SpaceColoristics," SCIENTIFIC AMERICAN, Vol. 26l,No. I, July 1989, pp. 84-90.
38. Huggins, C.T., Malone, T.B., and Shields,N.L., "Evaluation of Human Operator VisualPerformance Capability for TeleoperatorMissions," REMOTELY MANNED SYSTEMS, E. Heer(Ed.), California Institute of Technology,Pasadena, CA, 1973, pp. 337-350.
39. Chandlee, G.O., Smith, R.L., and Wheelwright,C.D., "Illumination Requirements for Operatinga Space Remote Manipulator," ERGONOMICS OFHYBRID AUTOMATED SYSTEMS I, W. Karwowski, H.R.Parsaei, and M.R. Wilhelm (Eds.), Elsevier,Amsterdam, ]988, pp. 241-248.
40. Krishen, K., "Robotic Vision Technology andAlgorithms for Space Applications,"PROCEEDINGS OF THE 39TH CONGRESS OF THEINTERNATIONAL ASTRONAUTICAL FEDERATION, PaperNo. IAF-88-028, International AstronauticalFederation, Paris, 1988.
41. Krishen, K., "Space Robotic Vision SystemTechnology," PROCEEDINGS OF ISA-88INTERNATIONAL CONFERENCE, Paper No. 88-1615,ISA, Research Triangle Park, NC, 1988, pp.1527-1537.
42. Smith, K.U., and Arndt, R., "ExperimentalHybrid-ComputerAutomation in Perceptual-Motorand Social Behavioral Research," JOURNAL OFMOTOR BEHAVIOR, Vol. I, No. I, 1969, pp. 11-27.
43. Smith, K.U., and Smith, T.J., "Systems Theoryof Therapeutic and Rehabilitative Learningwith Television," JOURNA_ERVCIIFS ANDMENTAL DISEASE, Vol. 148, 1969, pp. 386-429.
44. Smith, K.U., and Murphy, T.J., "SensoryFeedback Mechanisms of HandwritingMotions andTheir Neurogeometric Bases," NEW HORIZONS FORRESEARCH IN HANDWRITING, M. Herrick (Ed.),University of Wisconsin Press, Madison, WI,Chap. V, 1963, pp. III- 157.
45. Smith, K.U., Cambria, R., and Steffan, J.,"Sensory-Feedback Analysis of Reading,"JOURNAL OF APPLIED PSYCHOLOGY,Vol. 48, No. 5,1964, pp. 275-286.
46. Gould, J.D., and S_ith, K.U., "AngularDisplacement of Visual Feedback in Motion andLearning," PERCEPTUAL AND MOTOR SKILLS, Vol.17, 1963, pp. 699-710.
47. Smith, K.U., and Ramana, D.S.V.,"Characteristicsof Head Motion with VariableVisual Feedback Delay, Velocity andDisplacement," AMERICAN JOURNAL OF PHYSICALMEDICINE, Vol. 48, No. 6, 1969, pp. 289-300.
249
48. Abbs, J.H., and Smith, K.U., LateralityDifferences in the Auditory Feedback Controlof Speech," JOURNAL OF SPEECH AND HEARINGRESEARCH, Vol. 13, I970, pp. 298-303.
49. Smith, K.U., and Putz, V., "Dynamic MotorFactors in Determining Effects of RetlnalImage Feedback Reversal and Delay," AMERICANJOURNAL OF OPTOMETRY AND ARCHIVES OF AMERICAN
ACADEMY OF OPTOMETRY, Vol. 47, No. S, 1970,pp. 372-383.
50. Sussman, H.M., and Smith, K.U., "SensoryFeedback Persistence in Determining Memory,"JOURNAL OF APPLIED PSYCHOLOGY, Vol. 54, 1970,pp. 503-508.
S]. Rothe, M., SOCIAL TRACKING IN CHILDREN AS AFUNCTION OF AGE, Masters Dissertation,Department of Psychology, University ofWisconsin - Madison, Madison, WI, 1973.
52. Lisberger, S.G., "The Neural Basis forLearning of Simple Motor Skills," SCIENCE,Vol. 242, i988, pp. 728-73S.
53. Lisberger, S.G., and Pavelko, T.A., "BrainStem Neurons in Modified Pathways for MotorLearning in the Primate Vestibulo-OcularReFlex," SCIENCE, Vol. 242, 1988, pp. 771-773.
54. Smith, K.U., and Greene, P., "A CriticalPeriod in Maturation of Performance withSpace-Displaced Vision," PERCEPTUAL AND MOTORSKILLS, Vol. 17, 1963, pp. 627-639.
55. Smith, K.U., and Molitor, K., "Adaptation toCombined Reversal and Delay of Eyemovement-Retinal Feedback," JOURNAL OF MOTOR BEHAVIOR,Vol. ], No. 4, ]969, pp. 296-308.Smith, K.U., INDUSTRIAL SOCIAL CYBERNETICS,University of Wisconsin BehavioralCyberneticsLaboratory, Madison, WI, 1974.Cherry, C., and Sayers, B. McA., "Experimentson Total Inhibition of Stammering by ExternalControl and Some Clinical Results," JOURNAL OFPSYCHOSOMATIC RESEARCH, Vol. 1, 1956, pp. 233-246.
58. Sauter, S., and Smith, K.U., "Social Feedback:Quantitative Division of Labor in SocialInteractions,"JOURNAL OF CYBERNETICS,Vol. I,No. 2, 1971, pp. 80-93.
sg. Smith, K.U., and Kao, H., "Social Feedback:Determination of Social Learning," JOURNAL OFNERVOUS AND MENTAL DISEASE, Vol. 152, 1971,pp. 289-297.
60. Ting, T., Smith, M., and Smith, K.U., "SocialFeedback Factors in Rehabilitative Processesand Learning," AMERICAN JOURNAL OF PHYSICALMEDICINE, Vol. 51, 1972, pp. 86-101.
61. Smith, T.J., Smith, R.L., Stuart, M.A., Smith,S.T., and Smith, K.U. "InteractivePerformance in Space - The Role of PerturbedSensory Feedback," PROCEEDINGS OF THE 3RDINTERNATIONAL CONFERENCE ON HUMAN-COMPUTERINTERACTION, M. Smith (Ed.), Elsevier,Amsterdam, 1989, To be published.
62. Smith, R.L., THE EFFECTS OF TELEVISED ANGULARDISPLACEMENT OF VISUAL FEEDBACK ON A REMOTEMANIPULATION TASK, Unpubl|ShedMasters Thesis,Texas A&M University, College Station, TX,1986.
63. Stuart, M.A., and Smith, R.L., SPATIALLYDISPLACED VISUAL FEEDBACK AND PERFORMANCE OF
FTS-LIKE TASKS, NASA Technical Report JSC-23547, NASA, Houston, TX, 1989.
64. Lawrence, D.A., Chapel, J.D., and Depkovich,T.M., "Issues, Concerns, and InitialImplementation Results for Space BasedTelerobotic Control," PROCEEDINGS, 1987GODDARD CONFERENCE ON SPACE APPLICATIONS OFARTIFICIAL INTELLIGENCE (AI) AND ROBOTICS,NASA Goddard Space Flight Center, Greenbelt,MD, 1987, pp. 1-18.
65. Albus, J.S., Lumia, R., and McCain, H.,Hierarchical Control of Intelligent MachinesApplies to Space Station Telerobots," IEEETRANSACTIONS ON AEROSPACE AND ELECTRONSYSTEMS, Vol. 24, No. 5, ]988, pp. 535-541.
66. McCain, H.G., and Andary, J.F., "The FlightTelerobotic Servicer Project and SystemsOverview," PROCEEDINGS, 2]ST ANNUALELECTRONICS AND AEROSPACE CONFERENCE, IEEE,
Arlington, VA, 1988, pp. 97-102.67. Smith, J.H., Gyamfi, M., Volkmer, K., and
Zimmerman, W., "The Space Station AssemblyPhase: System Design Trade-Offs for theFlight Telerobotic Servicer," PROCEEDINGS,1988 GODDARD CONFERENCE ON SPACE APPLICATIONSOF ARTIFICIAL INTELLIGENCE (AI) AND ROBOTICS,NASA Goddard Space Flight Center, Greenbelt,MD, 1988.
68. Nicholas, J.M., "Interpersonal and group-behavior skills traintng for crews on spacestation," AVIATION, SPACE, AND ENVIRONMENTALMEDICINE, Vol. 60, No. 6, 1989, pp. 603-608.
250