1 90- 22313

SYSTEM CONTROL OF AN AUTONOMOUS PLANETARY MOBILE SPACECRAFT

William C. DiasBarbara A. Zimmerman

Sequence Automation Research GroupMission Profile and Sequencing Section

Jet Propulsion LaboratoryCalifornia Institute of Technology

PHONE: Dias (818) 354-0153 Zimmerman (818) 354-6700MAIL: Jet Propulsion Laboratory, MS 301-250D

4800 Oak Grove Drive, Pasadena, CA 91109

ABSTRACT INTRODUCTION

Our goal is to suggest the scheduling andcontrol functions necessary foraccomplishing mission objectives of a fairlyautonomous interplanetary mobilespacecraft, while maximizing reliability.Goals are (a) to provide an extensible,reliable system conservative in its use ofon-board resources, while (b) getting fullvalue from subsystem autonomy, and (c)avoiding the lure of groundmicromanagement. We propose a functionallayout consisting of four basic elements:GROUND and SYSTEM EXECUTIVE systemfunctions and RESOURCE CONTROL andACTIVITY MANAGER subsystem functions. Thesystem executive includes six subfunctions:SYSTEM MANAGER, SYSTEM FAULTPROTECTION, PLANNER, SCHEDULEADAPTER, EVENT MONITOR and RESOURCEMONITOR. The full configuration is neededfor autonomous operation on Moon or Mars,whereas a reduced version without theplanning, schedule adaption and eventmonitoring functions could be appropriatefor lower-autonomy use on the Moon. Animplementation concept is suggested which isconservative in use of system resources andconsists of modules combined with a networkcommunications fabric. The paper introducesa language concept we have termed a"scheduling calculus" for rapidlyperforming essential on-board scheduleadaption functions.

Interplanetary mobile spacecraft (rovers)require more autonomy than spacecraft inplanetary flybys or orbiters, if they are tobe acceptably productive. This is essentiallybecause knowledge of the environmentchanges over a much shorter time scale thanthe speed at which data could be received andanalyzed and commands generated and sentfrom Earth, given the light-time delays(Wilcox, et al, 1987; Dias, et al, 1987).Even a low autonomy rover on the relativelynearby moon needs more autonomy in thecontrol area than other spacecraft if it isrequired to move continuously (Pivirotto, etal, 1989).

This paper proposes a FUNCTIONAL SYSTEMCONTROL ARCHITECTURE in which a designor requirements for a design could bephrased. We address fairly autonomousrovers first of all. Second, adaption of thecontrol architecture to a low-autonomyLunar rover is discussed. Next, the paperhas a section on the practicalities ofimplementing the control architecture. Last,we discuss ongoing research in the JPLSequence Automation Research Group on thedevelopment of a language in which the rulebase of vital parts of the control systemcould be phrased.

The design process, as well as the designitself, should be responsive to the needs ofoperations managers to ascertain reliabilityand functionality. This is because the controlsystem partly substitutes functionally forground operations. Fairly autonomous roverswould need to be able to reliably perform

223

many of the spacecraft command and controlfunctions now performed only on Earth(Linick, 1985). It will be seen thefunctional layout preserves some of thecurrent division of responsibilities amongtraditional ground system and subsystemelements. Various parts of the COMMANDGENERATION process, including REQUESTGENERATION, REQUEST INTEGRATION,SCHEDULE GENERATION and COMMANDTRANSMISSION, are proposed for on-boardimplementation.

Our proposed architecture assumes aspacecraft with a complex and varied set ofgoals and activities only partly predictableduring design. Activity schedules willrequire some parallelism and optimization,as now provided for on Voyager and Galileo.There will inevitably be a desire for a greatdegree of ground control, to maximizemission return. In fact the command andcontrol system design must walk a tightropeamong the three paradoxically competingconcepts of system autonomy, subsystemautonomy, and maximal ground control, eachproffered in the name of maximizing return.

The control system needs to be as VERSATILEas possible, because the exact desiredoperational modes and combinations ofactivities for a spacecraft are not alwaysfully knowable in advance, and this will beespecially so for a planetary rover. The lessknown in advance about the particulars ofthe environment, the more varied thatenvironment, the more varied and generalthe set of tasks, and the larger the suite ofapproved instruments, the less predictablethe final operational range will be.

To promote rover functional EXTENSIBILITY,the control system design should incorporatefeatures able to enhance the software

development environment. Quick changesmay be needed in the software implementingtraverse and sample acquisition functionsafter landing, whether due to unforseeablehardware failures or unexpected conditions.Depending of course on the mission design,sample return mission surface stay timescould be as short as a few months, addinggreatly to pressures for operationalresponsiveness (Bourke, et al, 1989).

Rapid software turnaround presents a dangerof its own in an operational environment. Bybeing versatile and robust enough tocomprehensively trap and correct faultconditions, a good control system designshould make fast software developmentturnaround possible in the operationalphase. We can define the architecture tomaximize probability of success. We havedone this by incorporating softwareverification in the fault protection scheme.

Our architecture differs considerably fromother proposals, though it takes a layeredapproach often favored by other designers(IKI, 1988; Simmons, et al, 1989).Resulting as it does from the considerationsin the above paragraphs, our proposal isoriented towards providing "generalpurpose" spacecraft functionality byrepresenting what currently exists asground operations functions on board. Thisapproach differs from SubsumptionArchitecture (Brooks, et al, 1989) andfrom the Task Control Architecture(Simmons, et al, 1989). These appear to beoriented towards a predefined (butpresumably robust) set of "behaviors", andtowards missions which could beaccomplished with rovers operating in amore narrowly defined functional envelopethan the kind of mission we foresee. We feelearly interplanetary rover missions willneed to use mobile spacecraft which are asgeneral in capability as is reasonable, for allthe reasons in the above paragraphs. I!seems likely there would be a place for boththe "behavioral" and "general purpose"architectural philosophies in a well funded,longer term solar system planetaryexploration program, however.

The functions required for the rover as awhole are taken generally from Smith andMatijevic (Smith, et al, 1989). We haveadded Global Navigation, Data Handling, andPointing and Articulation to their SystemExecutive, Telecommunications, Power,Thermal, Science, Mobility and Sampling.Thus, our idea on how lhese subsystemfunctions should be defined is slightlydifferent, but exact subsystem delineationsare not our purpose. Instead, our hope is toclarify the system / subsystem interface in

224

general. We find no essential conflictbetween the Smith and Matijevic formulationand ours. The emphasis is different. Theirmethod appears to provide a convenientmeans for designating the control, command,and data paths to be included in a rovingspacecraft from high to low levels, beforedesign begins. Where they provide a generalpurpose tool and framework, we try toprovide and justify the functionalrelationships which need to be used to fill inthe details in the Smith and Matijevicarchitecture matrix, with emphasis on theSystem Executive over the subsystems. Wewanted to show the most meaningfulfunctional interfaces and formulate them so

that people can begin to think of allocation offunctions to modules.

The control system needs to incorporateconcepts from spacecraft FAULT PROTECTION(Riethle, 1983) in order to improveRELIABILITY over research and ground-based robots and robotic vehicles. In a"classic" form of fault protection, signalsfrom one or a few subsystems are used todetermine a fault and then pre-canned,simple and highly structured routines takecontrol of the subsystem or spacecraft andthrow it into a predetermined, safe, butusually non-productive state. This "classic"form of fault protection needs to continue toexist on planetary rovers, but its scopeneeds to be limited in such a way that moreintelligent autonomy is not subverted by itssheer simple-mindedness. More refinedforms of fault protection which take intoaccount the higher level of intelligence onthe spacecraft must be included, or theadvantages of intelligent autonomy would belost.

Finally, it may seem that this architectureis too complex, that it could never execute ina timely fashion. We do not take thispotentially serious problem lightly. Thefunctional description appears complexpartly because we did not want it to appearincomplete through overgeneralization. Wewanted to try to bring out a description ofthe functions and interrelationships thatmight be required, perhaps more completethan available in the past. The design andimplementation could turn out considerably

simpler than the functional descriptionmight cause one to expect, but all functionsshould be addressed in the design process. Wealso feel we have allayed some of thesecomplexity concerns in the implementationsection.

OVERVIEW

This section contains an abbreviateddescription of the overall control flow of thearchitecture. In the subsequent detailedsection on selected elements, we give a fullerdescription of the behavior of the majorelements.

GROUND I

_1 SYSTEM'1 EXECUTIVE

I,p

I . ,o0.cECONTROLLER

tnRESOURCEi

SYSTEM

FAULT

PROTECTION

_r

ACTIVITY

MANAGER

i

I

Figure 1. Basic Functional Control Architecture Layout.

The functional control architecture has fourbasic elements: GROUND, SYSTEM EXECUTIVE(SE), ACTIVITY MANAGERS (AMs), andRESOURCE CONTROLLERS (RCs). These sharecontrol as shown in Figure 1. Ground and SEin combination provide the system-levelcommand and control functions, while thesubsystem functions are performed by theAMs and RCs in various combinations for

different operational modes. The SE and RCseach have separate fault protectionfunctions, but the AMs are oriented morestrictly towards command generation,command, and control, and have no fault

protection role.

225

In the operational scenario for autonomousmodes, which are the ones addressed in thispaper, GROUND provides goals and schedulecontraints -- general and specific -- to therover SE. GROUND uses some sort ofsimulation based on whatever information ithas from orbit, previous traverses,statistical likelihoods, and the rover'ssensors to try to foresee likelihood ofsuccess. There is no guarantee that goals areachievable within time constraints, onlysome probability. Goals should try toencompass at least a few hours of activity,preferably a day or so, with fallbackcontingencies in case operations take longerthan expected, and fill-in items in caseactivities take less time than foreseen.

ACTIVITY

MANAGER

PLAN-

NER

::::::::::::::::::::::::::::::::::::

iiiiii i iiiiiiii::::::::::::::::::::::::::::::::,:.:.:.:.:.:,:-:-:-:-:-:-:-:.:-:.:.:

:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::.:::

Figure 2. System Executive command/control

architecture. All functions required on more

autonomous rovers. Only shaded areas needed

for command/control of low-autonomy rovers.

Figure 2 depicts the internal functionalinterfaces of the System Executive in moredetail. The on-board SE co-ordinates roveroperations through the SYSTEM MANAGER.First the SYSTEM PLANNER and the requisitesubsystem AM PLANNERS agree on a plan to

accomplish the goals. They do this by firsthaving the SYSTEM PLANNER arrive at sub-goal and activity sequences and resource andtime envelopes within which the AMs mustplan, then having the AM PLANNERs arriveat sequences implementing the goals to thedegree possible. A series of "events" wouldbe part of any schedule agreed upon. Theseevents would be more or less at the samelevel as those used by a Voyager or Galileoground sequence team in its higher levelscheduling activities today. The timerelationships among these would be partlyrelative, that is, the time at which an eventwould occur would be phrased as relative toother events, not as an absolute time (thoughthe system would always have a nominalabsolute time for future events.) Sequencesare phrased relative to events, with theexact time of commanding to be determinedlater, in execution.

In the execution phase of the plan, the AMEXECUTION CONTROLLER actually sendscommands to (and reads data from) theRESOURCE CONTROLLERS, which are theonly entities able to address the physicalresources directly. The RESOURCE MONITOR(RM) keeps track of resource status(including progress on events) at the systemlevel, by reports and / or polling techniques.The EVENT MONITOR keeps tabs on theprogress of the schedule with respect to theevents. It does this by receiving both timedand event-tagged progress reports from theAMs, and (redundantly) by RC reportsrelayed from the RM. Discrepancies in thesereports result in fault protection,replanning, or other exception processing.AMs may optionally read event reports fromthe EVENT MONITOR. The SCHEDULEADAPTER constantly modifies the timing ofthe system-level sequence, within agreedparameters, based on event-progressreports from the EVENT MONITOR. The AMsmodify their schedules for data and forcommanding the RCs based on near-realtimescheduling refinements from the SCHEDULEADAPTER. The latter is also able todynamically reconfigure the RCs based onschedule changes, and the AMs are thenlimited to commanding within thoseparameters.

226

Replanning can occur during execution,either because specific events in a scheduleare designated as points for plan refinement,when the next increment of information isavailable for planning, or becauseunforeseen conditions caused an existingschedule to be invalidated. Sometimes,subsystem replanning will be necessarywithout system replanning being needed.

RESOURCE FAULT PROTECTION responses,including reflexes, may be invoked to safe aresource based on resource-internalinformation alone. Signals are then sentsystem-wide. SYSTEM FAULT PROTECTIONmay be invoked in response to conditionssignaled by combinations of subsystems, theschedule becoming dangerously unworkable,disagreeing progress reports from the AMsand RCs, or signaling of computed statusderivatives and trends indicative of faultsfrom the RM. One would try to autonomouslyreplan out of at least some fault protectionresponse modes.

DETAILED FUNCTIONS OF SELECTEDSYSTEM ELEMENTS

This section discusses selected elements ofthe architecture in more detail than in theoverview.

SYSTEM PLANNER

First, the System Planner processesground-supplied GOALS into a serial andparallel collection of MAJOR ACTIVITIESwhich will bring about the desired goals.Time factors are only applied in a gross way,which is enough to eliminate many schedulepossibilities.

Second, the System Planner derives scheduleconstraints and resource utilizationenvelopes within which each major activitymust be planned.

Next, the System Planner waits whilesubsystem AM Planner functions derive aSCHEDULE from the required activities (SeeActivity Managers, below). The SystemPlanner then integrates the resultantsubsystem-derived schedules, which mayinclude changing the previously imposed

resource and schedule constraints and askingfor additional subsystem planning.

The SCHEDULE derived by the planningfunctions must have a form consonant withthe need for on-board schedule maintenancein realtime. This is more than is required ofa contemporary spacecraft schedule. Ingeneral a schedule might be defined as thetimed series of events and states of allresources and subsystems which isdetermined to bring the desired activities tocompletion. In order to allow for laterrealtime adaption, the new type of schedulemust include temporal relations amongevents, of a sufficiently economical nature toallow operations in realtime. In other words,an EVENT-DRIVEN SCHEDULE is needed. Eachevent or state change needs to have its timerequirements described relative to whenother events or state changes take place.

Stated more abstractly, the schedule consistsof a set of functional relationships among thethree elements of states, events and times,such that, where t is an instant in time andF1 and F2 are functions:

1. System Statet = Fl(Subsystem Statest)2. Acceptable System Statet = F2(System

Statet-l)

and, where F3 is a function, Eventa is nexton the schedule and Events b l through bxhave already occurred:

3. Acceptable Time Interval(Eventa)=F3(Times (Eventbl ..... Eventbx))

Applied recursively, what this expresses isthat acceptable activities or states of boththe system and of each subsystem, for anyinstant in time, depend on the states of allthose entities dynamically as mattersprogress. This is similar to a system whichruns according to "control laws". In thisway, schedules are derived without assigningall final times, but with needed time linkageamong events and states. In the last section inthe paper, we discuss a language and some ofthe rules of a "scheduling calculus" whichcould be used to express and enforce thefunctions in a real system.

227

The following are possible examples of"events" at the level of interest of theSystem Planner. These correspond roughlyto a fairly high level of planning,specifically the initial sequence integrationwhich occurs right after request generationfor a spacecraft. They stop short of devicemanagement which in this conception is atthe level of the subsystem AM Planner andRC.

start

start

start

start

/ finish a camera platform slewingoperation/ finish a single maneuver in thecourse of a longer traverse/ finish a sample arm movement (or,maybe a joint movement)/ finish warmup of a sampleprocessing oven

SCHEDULE ADAPTER

The Schedule Adapter converts the schedule,phrased in expressions to be operated on bythe scheduling calculus, into a high-levelSEQUENCE with final times, exact states,etc., assigned. Whereas "schedules" includefunctional relationships among times, eventsand states, "sequences" include only thetimes, events and states which result fromapplying the functions based on knowledge ata given instant. Thus what is known about thespacecraft high level sequence might includeonly a few seconds or minutes of activity.

So the Schedule Adapter routinely uses therules of the scheduling calculus to deriveacceptable ranges for the next set of allsubsystem states from the current state. In aclosely related function, it continuallychanges the system's idea of when events willoccur. These adaptions thus do notnecessarily constitute a need for schedulechanges (i.e., replanning) in ourterminology.

The Schedule Adapter is very dependent oninput from the Event Monitor to keep itinformed of status of all relevant events, oruncertainty in that status. The ScheduleAdapter must have a way to respond tounclear state or event status knowledge,informing System Fault Protection whichmay be the one to decide what to do.

The Schedule Adapter sends sequencerevisions to the relevant AM executioncontrol functions, which adapt in turn.

The Schedule Adapter provides configurationcommands to the RCs. The AMs, whichactually run the rover through an activitysuch as a rolling maneuver, are limited tocommanding the RCs within the envelopesconfigured by the Schedule Adapter for eachmoment in time. This mechanism providessystem level resource control while allowingthe responsible AMs reasonable commandauthority over those resources needed.

There may have been specific points in theschedule designated for system or subsystemreplanning or plan refinement. If so, theseare honored in an event-driven fashion thesame as other dependent events.

The Schedule Adapter may be called upon bySystem Fault Protection to invoke a specialschedule implementing a fault protectionschedule, and to abort a current schedule inan organized way.

Another part of the Schedule Adapter'sfunction is to realize when incompatiblecombinations of conditions occur or areabout to occur. For instance, if the rover isrunning behind schedule in getting to a siteof scientific interest, low priority activitiesmay need to be either rushed through, withcontrolled loss of data quality, or abandonedentirely. It is up to the Schedule Adapter todo this and signal the AMs and RCsaccordingly. If conditions deterioratefurther, the System Planner and AMPlanners must be reinvoked to completelyrework the schedule and salvage what ispossible of the original goals. As a lastresort, the rover should inform Earth of theproblem at the next opportunity. Ground canthen respond by recommanding with adjustedgoals. This can be an expensive solution interms of wasted rover time, a fact that needsto be worked into the original decision onwhat to do if a schedule fails under variousconditions.

228

SYSTEM EVENT MONITOR

This function holds the current systemknowledge of all events previouslycompleted/aborted in the schedule or inprogress. It accomplishes this by seperatelymonitoring both resource and activity status.This redundant approach provides cross-checking considered highly desirable inspacecraft fault monitoring (Riethle, 1983;Reiners, 1985).

First, AM Execution Controllers provideactivity status to the Event Monitor, both atpredetermined intervals and at status changepoints agreed to in the schedule. This is ahigh level check that activities are onschedule and status is acceptable. Activitystatus report frequency will undoubtedlyvary by operational mode. Events to bereported can be unplanned. For instancewhen the AM Execution Controller becomesaware independently that its plan is nolonger workable, that needs to be reported.

Second, event-related resource statusreports are provided from the SystemResource Monitor, which has collected thesefrom the RCs. Unplanned events, such as theinvocation of a reflex action by the RFPCs,also need to be reported.

The Event Monitor integrates these varioussources of event status and reports to theSchedule Adapter to help it make decisions.System Fault Protection is informed in caseevent patterns reported can be used to inferfault conditions. The software validationfunction is served because some (perhapsmost) software problems will show up aserror reports or as inconsistencies in eventreports among the various sources.

SYSTEM FAULT PROTECTION (SFP)

This function includes the separate areas offault detection, fault analysis, and faultreponse. It detects system-level faultconditions from combined messages from theSystem Event Monitor, System ResourceMonitor, Resource Fault ProtectionController, and Resource Controllers.Messages can include both status and datadetermined in the design process. It may

execute hard-coded (or at least high-speed),high-reliability responses to faults detected,forcing the spacecraft into very well definedstates from which recovery will be as easyas possible. It may conceivably also initiateslower fault responses requiring normalschedule planning channels. It seems likelySystem Fault Protection would includereliable, predesigned, canned schedules in aform able to be adapted to specific currentconditions by the on-board Planners and/orSchedule Adapter.

The System Fault Protection functions needto respond directly from primary inputs --otherwise they would be dependent on otherfunctions and therefore less foolproof. Atleast some responses must be designed tooperate without permission from the SystemManager. The System Manager must in turnbe informed as soon as fault protection isinvoked. This requirement poses a problemfaced by all systems with distributedauthority and / or redundant data -- thepossibiiity that different parts of the systemwill be working to cross purposes for someinterval of time, with resultant system stateambiguities. The problem cannot be fullyaddressed until the design phase. Hopefully,the pre-canned fault protection schedulescan be designed so as to be adaptable by theSchedule Adapter to the particulars of thecurrent state.

SYSTEM RESOURCE MONITOR

The System Resource Monitor has threeseparate ways of monitoring resource status.

First, the SE receives a "heartbeat" -- orelementary status message -- from each RCon a regular, timed basis. These messagesare independent of any specific task theresource has been commanded to perform.The total absence of a message, a messageconveying an error status, or a messagecontaining data from which an errorcondition is deduced, can be grounds toinvoke system-level fault response.Heartbeat occurence and frequency may varyfrom subsystem to subsystem, but as a pointof reference, Galileo heartbeats are atapproximately 0.7-second intervals.

229

Second, the RCs report to the ResourceMonitor in connection with the specific tasksthey have been commanded to support. Thesemessages are tagged to the portion of thecommand sequence that brought them about.A "resource event" in this sense includes anyRC status change, or any change in the tasksbeing supported even though there may be noother resource status change. Reports arealso required at regular intervals (muchlike the heartbeats) as a cross-check thatstatus is as expected and things are onschedule.

Third, signals from the RFPC are received inthe event resource-level fault protection isinvoked.

Both planned and unplanned resource statuschanges are events and are duly reported tothe System Event Monitor.

ACTIVITY MANAGERS (AMs)

Identified subsystem functions requiringAMs are: Science Payload, SampleAcquisition, Traverse, Global Navigation,Telecom, Data Handling, and Imaging. Wediscuss the AMs only to the degree necessaryto put them in context with the rest of therover control system. Their design isspecialized, different for each subsystemfunction, and not the subject of this paper.

AMs provide a level of intelligence higherthan the RCs for important spacecraftsubfunctions (e.g., autonomous powersubsystem, Fesq, et al, 1989; autonomousnavigation subsystem, Gat, et al, 1989).They trade or share control depending onoperational mode. On less autonomousspacecraft, subsystem planning andmonitoring would be performed by groundsubsystem engineers or scientists incoordination with system level engineers. Ona more autonomous spacecraft such as afairly autonomous rover, the AMs to someextent represent on-board the functions thesubsystem engineers serve on the ground.AMs have NO FAULT PROTECTIONRESPONSIBILITIES, because it would beredundant, and because they represent areassuch as navigation where fast softwaredevelopment turnaround is desirable. AMs

requiring data from other AMs for planningor execution functions must obtain andupdate information in common data baseareas to maintain controls.

The fully implemented AM is presumed tohave a PLANNER and an EXECUTIONCONTROLLER.

The AM PLANNER utilizes specializedsubsystem knowledge unavailable to the SEplanner to derive (1) a sequence of time-driven and event-driven commands to be

given to the RCs, and (2) a corresponding setof expectations (Gat, et al, 1989). The AMPlanner co-ordinates provisional plans withthe SE Planner. It should be noted that, as inthe case of the SE planner, planning may beincremental. That is, a high level activitysuch as the acquisition of a sample willprobably be worked out as a series of stepswith estimated times, with final planningapplied only as preceding steps complete.

The AM EXECUTION CONTROLLER co-ordinates execution of plans agreed tobetween the AM Planner and the SE Planner.It implements control by commanding theRCs and reading data and status from them. Itmay read event status, if desired, from theSE Event Monitor. It responds to scheduleenvelope changes provided by the ScheduleAdapter in near realtime. It reports itsversion of events and status back to the EventMonitor on both a time- and event-drivenbasis. It may be interrupted by the SE if thelatter decides things are not on track andinvokes fault protection or replanning.

RESOURCE CONTROl I I=RS (RCs)

Every resource is governed by a ResourceController (RC) which has about the samelevel of intelligence as a typicalcontemporary disk controller. All contactbetween a resource and other elements

(except resource-level fault protection) isthrough its RC. RCs for different purposescould have different amounts of memory andCPU power, but they would all have the samequalitative functions: receiving commands,sending status, sending and receiving data,and keeping track of a time-linked stack ofcommands for one resource. The RC accepts

230

reconfiguration commands from the ScheduleAdapter and commands from the AMExecution Controllers, provided these arewithin the configuration envelope providedby the Schedule Adapter. It provides bothtime- and event-tagged status to the SystemResource Monitor and System FaultProtection functions. Status messagesreturned by the RC include identifiers sothat other functions may know whichevent(s) from the sequence the resource iscurrently working on, and the RC's progresson the sequence.

RESOURCE FAULT PROTECTIONCONTROLLERS (RFPCs)

Any precanned, fixed routines which aredesigned to automatically, unconditionallyand unilaterally change individual resourcestates based on sensor fault readings arehandled by the RFPCs. This is a basic,conventional spacecraft fault protectionstrategy which will continue to be needed forsome faults. Examples include fuseprotection, automatic shutdown of electricheaters exceeding temperature specs, ortrend analysis for individual resources.Other system elements such as the RCs, AMsand SE Event Monitor are informed of thefault status through normal channels afterthe fact.

In some cases, a system-wide fault responseis needed, which must be processed bySystem Fault Protection (SFP). In thosepredefined cases, the duty of the RFPC is tosimply send the status to the RC and theimportant fault data to the SFP for action.

In our view, RFPCs would also implementany required resource-level reflexes. Theseinclude any action or behavior, at the levelof the individual resource, required on anunexpected basis and on too short notice fororganized involvement by the planningfunctions. We believe reflexes should beconsidered fault reponses for spacecraftdesign purposes. Of course, not all reflexesresult from true emergencies and willtherefore sometimes result in situationseasily handled by on-board software.

The invocation of any reflexes or resource-level fault protection presumablynecessitates replanning after any furtherimmediate spacecraft sating is complete.

RESOURCES

Resources are commandable elements

providing services, conditions, orcommodities to the requesting elements,through their RCs. All communication to aresource is through the RC in normal modes.The following commandable resources havebeen identified for a planetary rover: Power,Thermal State, Data Handling, SciencePayload, Science Imaging, SamplingMechanisms, Pointing and Articulation,Mobility / Vehicle State, Mass Storage, andTelecom Data.

At this stage of the design, there is alwaysthe possibility that some high-speed controlrequirement will later be found requiringdirect communication with the resource.That discovery will have to await the testbeddevelopment stage.

LOW-AUTONOMY INTERPLANETARYROVERS

True teleoperation, in which both commandand low-level control is in the hands of anoperator, is thought to be an unreasonablemeans of controlling rovers even on theMoon. The light time delay of around threeseconds is too great (Pivirotto, et al, 1989).300 milliseconds may be the maximum forteleoperation.

Our view is that all the functional elementsdiscussed in this paper would be needed insome degree for fairly autonomousinterplanetary rovers, regardless of thedistance from Earth. However, considerableeconomy is possible for a Lunar rover withlower autonomy and interactive commandingfrom Earth, because the round-tripcommunication delay (light-time pluselectronic delays) is likely to be only a fewseconds. All planning (system andsubsystem), schedule maintenance and eventmonitoring functions can be done on Earth.These would be tightly integrated in operatorcommand terminals. Activity execution

231

monitoring, mode switching, resourcehandling and fault protection would be on thespacecraft. See shaded areas on Figure 1.This is more autonomy and on-board controlthan would be provided by teleoperation.

It is likely that Lunar rover programs willnaturally precede Mars rovers (Report ofthe 90-Day Study, 1989). This provides anopportunity to perfect the designs andtechniques for autonomous schedulemaintenance on the ground in an earlierprogram. Those functions would later bemoved on-board to achieve the greaterautonomy necessary for productivity at theup to 40-minute round trip light timedelays presented by Mars, or by moreautonomous Lunar rovers.

IMPLEMENTATION OF THE SYSTEMEXECUTIVE ARCHITECTURE

To the SE the planetary-vehicle domainappears to be made up of a two-levelhierarchy of elements. The top level of thehierarchy represents the vehicle functionsor activities. The lower represents lhehardware that participates in carrying outthe activities. This view of the rover domain

suggests an architecture that supportsmultiple interacting tasks which can accessa pool of resources. A suitable architecturecan be modeled, superficially, by a moderncomputer operating system (Rashid,1986). However, the computer operatingsystem model is not complete or sufficientbecause the control procedures commonlyinvoked fall short of those required for acapable rover's operation. To control afairly autonomous planetary vehicle thesystem design must specify a comprehensiveself-analysis tool with which to track thestate of the vehicle. In this section wedescribe an architecture for the SE's controlfunctions, and a procedure for planning anddiagnosis of the state of the rover as itcarries out a task. The procedure, which wecall a scheduling calculus, combinesqualitative relationships with arithmeticexpressions to render judgements about therover's state, and the validity of a scheduleor sequence given that state.

OVERVIEW

Preliminary system designs (Pivirotto, etal, 1989; Lambert, 1989) have typicalplanetary vehicle functions such as sampleacquisition, navigation, data handling,science, imaging, and telecommunicationscontrolled by several independentprocessing elements (See Figure 3).

RESOURCE

F ."onRESO_OURCE

ScI_ ;ET_O_ _,ng

RESOURCE m _ _ RESOURCE

T_ Handling

RESOURCE

Figure 3. A Distributed Computing Network.

Processing Elements (ovals) and Resources

controlled by independent activity modules

through e common system-level network.

In addition, the processing elements sharesome form of mass storage and systemresources. The processors and the resourcesare joined by a communication fabric whichconnects each of the processing elements toone another and to the resources. We willrefer to this communication fabric as anetwork. We envision a layered architecturewhereby the activities direct their requestsfor resource usage through the SE (Figure4). Access to the SE, the activities, and theresources is by message passing which issupported by SE service routines.

An important concept in the proposed designis that the SE's subfunction modules aremodelled as a collection of one or moreindependent processes that communicatethrough message passing. The modularnature permits the SE to be dynamicallyconfigurable to accomodate the requirementsof a mission. Further, we envision a systemin which one or more of the SE's subfunctionprocesses reside in each processing element.The distributed capability, which isindependent of the number of processors,

232

can be used to partition the demand on therover's computer resources.

Figure 4. Rover Command / Control SystemLayered Architectural Layout.

SYSTEM EXECUTIVE MODULES

The following sections describe the fiveclasses of modules that make up the SE.These are the SYSTEM MANAGER, SYSTEMFAULT PROTECTION, SYSTEM PLANNER,SYSTEM MONITOR, and SYSTEM NETWORKMANAGER. It is assumed that the systemexecutive processes reside on top of someform of computer operating systemenvironment which provides the low-levelfunctionality with which the hardware isaddressed and the data managed. In adistributed system there has to be somemeans by which data of a global nature areprovided to the processes. While the SE, theRCs and the AMs depend on these services,they are not part of the SE functions and arenot discussed in what follows.

System Manager

This element of the SE provides theinterface between the ground and vehicle,and among some of the major on-boardcomponents. The system manager configuresthe control system to support the mission

requirements. For a less autonomous roverthat is to be commanded from the ground,the system manager directs the commandsequence to the relevant AM ExecutionControllers and Resource Controllers. For amore autonomous rover the System Manageractivates the planner processes, and theevent monitor.

System Fault Protection

Both lower- and higher-autonomy roversneed on-board fault protection. It is likelythe great body of fault detection, analysisand response code could be thought of asstored in this separate class of modules.However, it is necessary for efficiency todistribute some of the fault detectionresponsibilities to the System Monitor andNetwork Manager modules, outlined below.In addition some elementary faultresponses, somewhat redundant, willprobably be required as part of the coderunning in each node in a multiprocessornetwork. As stated elsewhere, other faultprotection is implemented at the subsystemlevel, entirely outside the SE. However, it isthe responsibility of the system faultprotection class of modules, in the event of asystem fault, to activate fault protection,retrieve the necessary data to configure theresources and return the rover to a knownstate, and send the necessary commands toorchestrate controlled aborts of on-goingprocesses. In the higher-autonomy rover,stored, configurable schedules can beprovided on-board to the System Plannerand Schedule Adapter for these purposes.

System Planner

The two planner processes, a Planner and aSchedule Adapter, provide a layeredimplementation of the scheduling processes.The system manager directs relatively highlevel goal and constraint data to the systemplanner. The goals are expanded by theplanner into a time-ordered set of tasks forthe Activity Managers to process. TheActivity Managers produce the detailedsequence of events that accomplish the tasks.Before execution, this sequence is returnedto the planner for verification and EventToken extraction (see below). The schedule

233

adapter monitors the plan's execution,adjusting the sequence as needed.

There are several features that reflect thedegree of sophistication represented by theSE planner and schedule adapter processes.The planner must automatically generateand maintain a schedule. This is a significanttask which ground-based plannerscurrently do not fully support. We areinvestigating the Remote Mission Specialist(RMS) system developed by the SequenceAutomation Research Group at JPL (Rokey,et al, 1990), and the research on automaticplanning by the same group (Eggemeyer, etal, 1990) for solutions to the plannerrequirements. In addition, our own work onthe scheduling calculus can be used for theschedule maintenance task.

System Monitor

The system monitor provides for therequired event and resource monitoringfunctions. This includes maintainingcurrent resource status for systempurposes, and serving as a centralrepository of event status around whichother modules can synchronize. It isexpected that some of the fault detection --or preprocessing for that purpose, such astrend analysis -- would be offloaded fromthe system fault protection modules to themonitor modules, for efficiency.

Network Manager

All of the rover elements, including the SE,communicate via message passing. Thenetwork manager provides the systemservices to support this activity. Thesystem services provide a message formatthat characterizes the nature of the

communication. The message formatincludes fields for the message type, thesender, the event tag, and the command ordata, and possibly the destination. Basedupon message type, the messages areexpeditiously forwarded to relevantprocesses. The sender of a message need notknow how a request for data will be filled,or how a message will be routed. Forexample if the message type is that of a"heartbeat" message from a Resource

Controller it is directed by the networkmanager to the Resource Monitor process ofthe SE.

The network manager provides for bothadded security and ease of applicationprogramming. The subsystem implementersaccess all of the rover functions through auniform interface -- a message protocol. Inaddition, the network manager implicitlyprovides the capability to monitor messagetraffic over the network. Statistics gatheredby this process can be used to measure thehealth of the subsystems on the net.However, care must be taken in the design ofthis type of support to insure that theservice does not overwhelm the computerresources.

SCHEDULING CALCULUS

The overall function of the schedule adapterand monitor processes is to integrate theresponses from multiple task activities todetermine whether the activities are leadingto the given objectives or to an undesirablestate or possible fault condition. We arerequired to assign a qualitative value for thestate of the vehicle, based upon quantitativeinformation supplied by the resources. Inaddition, we are required to judge whetherthe events reported during the execution of asequence match the goal set for the task, orwhether conditions of the vehicle or in the

environment require the sequence to bechanged. We chose to investigate the methodsof qualitative processing (Bobrow, 1985)as an approach to solving the above problem.In particular, we are in the process ofdeveloping a language that implements anarithmetic reasoning tool. The work followsclosely that described in the paperCommonsense Arithmetic Reasoning(Simmons, 1986). We are using the UNIX ].yacc procedure (Johnson, 1983) to buildthe scheduling calculus language. Thelanguage combines methods of graph search,interval arithmetic, relational arithmetic,constraint propagation, and functionevaluation to arrive at decisions about the

validity of a schedule step, and conclusionsabout the rover state. A capability of the

1UNIX is a trademark of AT&T.

234

language can be illustrated by one ofSimmons' examples. Given the ranges ofthree quantities A, B and C we want todetermine the acceptable range for a fourthquantity, D:

A is constrained to the interval [3,4]B is constrained to the interval [1,4]C always has the value of 2D is related to A,B,C by (B*C)/(A+B)We assert the relation B >= CB is now constrained to interval [2,4]Using interval arithmetic we determine D

is constrained to [.5,1.6]

REQUIRED KNOWLEDGE BASE

The tool will require a knowledge base thatprovides functions, parameters, values, andconstraints that describe an acceptableoperating environment. We call this a modeldatabase. The high-level goals provided bythe uplink process will provide furtherfunctions and constraints with which toreason about the system. The event tokensderived by the planning process andresource sensor data also contribute to theknowledge base. We call this the derivedknowledge base. Using the model and derivedbases, directed graphs can be built by thelanguage as reasoning tools. Values for thefunctions are the nodes of the graphs andrelations among the functions are the arcs.

REASONING ABOUT RESOURCE STATES

The procedure to reason about theresources' states is simple in principle.Resource Controller output data and statemessages are examined to see if they fallwithin acceptable time and value intervals.The change or lack of change with respect totime is noted. The changes contribute to atable of derivatives that can be used toidentify trends in the behavior of a resourceor a set of related resources. This trendanalysis serves in part as a fault detectiondevice, allowing prediction of futureconditions and intervention before somefaults occur. Data for the trend analysis iscollected and preprocessed locally by themonitor processes, to reduce networkloading. Further fault analysis and response

is performed by the System Fault Protectionmodule.

EVENT TOKENS

The interactions of the system planner andActivity Managers produce a time orderedset of event tokens which represent steps inthe schedule that indicate levels of progress.The event token format is a tag or label, atime interval and expectation values forrover state parameters. The event tokensare assembled into tree structures which

are used by the schedule adapter to measurethe progress of the sequence.

REASONING ABOUT THE SCHEDULE

Conditions derived from information

provided by elements of the event tokentrees are used by the schedule adapter totrigger the scheduling calculus processes.Below are two simple examples. The firstillustrates what may be a common conditionin rover operation -- things taking longerthan predicted.

With the current node of the event tokentree, we have reached the end of one of aseries of traverse segments. The durationof a traverse segment was predicted to be1 hour. The actual time was 1.5 hours,exceeding the specified interval. Thesequence calls for an obligatory downlinkin an hour. The scheduling calculusprocedures are invoked to determine howthe sequence and constraint envelopes forthe planning of the next traverse segmentare to be changed for acceptableproductivity while ensuring the vehiclestops for the downlink telemetry at theproper time.

The second example is one that has thevehicle supporting a science experiment.Again, this example is an illustration of thenon-deterministic nature of planetaryvehicle operations. This example illustratesthe requirement for the scheduling calculusto reason about the rover environment.

Objective: Take a multispectral imageof a designated feature alonga traverse.

235

Parameters: Exposure time vs. amount ofambient light, locationcoordinates, frequencies,objective, and sun angle.

When the rover arrives at the feature,based upon the sun angle and the positionof the scan platform, the schedulingcalculus procedure will reason whetherthere will be sufficient light during thetime interval for the observation. If thereis not, the method can calculate a newexposure time. However the method mustalso reason whether this exposure timecan be used and still meet the overallobjectives of the schedule.

STATUS

The scheduling calculus language currentlysupports the relational, interval, andfunctional arithmetic, as well as built-intyped functions and the ability to createtyped symbols. It can perform operationsillustrated by the Simmons example andwill soon operate on preconstructed graphs.We are in the process of designing thedatabase specification and interface, and theprocedures for automatically constructingthe graphs.

CONCLUSIONS

This rover system control architectureproposal is complex because we have triedto offer something which could eventuallygrow into a comprehensive solution to theproblem of maximizing mission return of arover very remote from Earth, by movingdifficult, complex, time-consuming groundprocesses on-board. Whatever the ultimatesolution to the apparent paradox amongsystem, subsystem, and ground control,rover mission complexity will be reflectedin the command and control system. It stillremains to be proved (1) whether thesefunctions can be implemented, and (2)whether the resultant implementation canbe tested well enough so mission managerswill allow the system to be used. We areoptimistic on both counts.

ACKNOWLEDGEMENTS

The work described in this paper wascarried out at the Jet Propulsion Laboratory/ California Institute of Technology under acontract with the National Aeronautics andSpace Administration.

While the authors have sole responsibilityfor the contents of this paper, we wish toextend special thanks to the following JPLpersonnel for ideas and review of thereport: Jim Burke, Carol Collins, SvenGrenander, Ken Lambert, Jacob Matijevic,William McLaughlin, Donna Pivirotto, DaveSmith, and P. Richard Turner.

A complete list of credits would beimpossible, however the authors wish tothank the following people for ideas:Charles Budney, Curt Eggemeyer, MattGolombek, Mike Hollander, and RobManning.

REFERENCES

Bobrow, D., Editor, (1985). QualitativeReasoning About Physical Systems.

Bourke, R., Kwok, J., and Friedlander, A.(Nov 1989). Design of a Mars Rover andSample Return Mission, Proc. of CNESInternational Symposium on SpaceDynamics, Toulouse, France.

Brooks, R., and Flynn, A., (Oct 1989).Rover on a Chip, Aerospace America.

Dias, W., and Gershman, R., (Oct 1987). ADay in the Life of a Mars Rover. JetPropulsion Laboratory Internal Report,D-5075.

Eggemeyer, W., and Cruz, J., (to appearMay 1990). PLAN-IT-2: The NextGeneration Planning and SchedulingTool, Proc. of Goddard Conference onSpace Applications of AI, 1990.

Fesq, L., and Stephan, A., (1989). On-Board Fault Management Using ModelingTechniques, Proc. Inter-Society EnergyConversion Engineering Conference(IECEC).

236

Gat, E., Firby, R., and Miller, D., (July1989). Planning for ExecutionMonitoring on a Planetary Rover, Proc.of NASA Conference on SpacecraftOperations, Autonomy, and Robotics.Houston, TX.

IKI - Space Research Institute (USSR)(1988). Martian Rover Motion ControlPrinciples Preliminary Concepts, Pr-1422.

Johnson, S., (1983). YACC: Yet AnotherCompiler Compiler. UNIX Programmer'sManual, Vol. 2. Bell TelephoneLaboratories, Murray Hill, N.J.

Lambert, K., (Oct 3, 1989). AComputational System for a Mars Rover,AIAA 89-3026. AIAA Computers inAerospace VII.

Linick, T., (1985). Spacecraft Commandingfor Unmanned Planetary Missions: TheUplink Process, Journal of the BritishInterplanetary Society, Vol. 28 No. 10.

Pivirotto, D. and Dias, W., (1989). UnitedStates Planetary Rover Status-1989,Jet Propulsion Laboratory InternalReport D-6693.

Rashid, R., (1986). Threads of a newSystem. UNIX Review. Vol 4, No. 8.

Reiners, T., (Aug 1985). AutonomousRedundancy and MaintenanceManagement Subsystem (ARMMS)Executive Summary. JPL internalreport D-2414.

Report of the 90-Day Study on HumanExploration of the Moon and Mars, (Nov1989). National Aeronautics and SpaceAdministration.

Riethle, G., (Aug 1983). A SummaryOverview of Technology Appfied to theARMMS Demonstration Project. Fault-Tolerance Techniques. JPL internalreport D-948.

Riethle, G., (Oct 1983). A SummaryOverview of Technology Applied to the

ARMMS Demonstration Project. ARMMSExecutive Software. JPL internal

report D-1122.

Rokey, M., and Grenander, S., (to appearJun 1990). Planning for SpaceTelerobotics: The Remote Mission

Specialist, IEEE Expert.

Simmons, R., (Aug 1986). 'Commonsense'Arithmetic Reasoning, Proceedings ofAAAI-86, Philadelphia, PA.

Simmons, R., and Mitchell, T., (Jul 25,1989). A Task Control Architecture forAutonomous Robots. Proceedings of SOAR'89 Conference.

Smith, D., and Matijevic, J., (Oct 1989). ASystem Architecture for a PlanetaryRover. Proc. of the NASA Conference on

Space Telerobotics, January 1989. JPLPublication 89-7.

Wilcox, B., and Gennery, D., (1987). AMars Rover for the 1990's, Journal ofthe British Interplanetary Society(JBIS), Vol 40, No. 10.

237