A Case Study of Productivity Challenges in Mars Science Laboratory OperationsDaniel Gaines, Gary Doran, Heather Justice, Gregg Rabideau,

Steve Schaffer, Vandana Verma, Kiri Wagstaff, Ashwin Vasavada,William Huffman, Robert Anderson, Ryan Mackey, Tara Estlin

Jet Propulsion LaboratoryCalifornia Institute of Technology

4800 Oak Grove DrivePasadena, California 91109

{firstname.lastname}@jpl.nasa.gov

Abstract

Achieving consistently high levels of productivity has been achallenge for Mars surface missions. While the rovers havemade major discoveries and dramatically increased our un-derstanding of Mars, they often require a great deal of effortfrom the operations teams and achieving mission objectivescan take longer than anticipated. We conducted an in-depthcase study of Mars Science Laboratory operations in order toidentify the productivity challenges facing surface missions.In this paper, we describe how we performed the case studyand analyzed the data. We present and discuss the significantproductivity challenges we identified during the study. In ad-dition to informing future surface exploration missions, thestudy is relevant for a wide range of applications in whichoperators must interact with a robotic system with limitedcommunication opportunities.

IntroductionThe Curiosity rover has been exploring Gale Crater andMount Sharp since its landing in August 2012. During thistime, the Mars Science Laboratory (MSL) mission has ac-complished many significant objectives. It has achieved thesuccess criteria for the prime mission, collected evidencethat indicates Mars was once habitable, collected over 19samples and driven more than 15 kilometers (Grotzinger andothers 2015; Vasavada et al. 2014). Curiosity is currently inits second extended mission and continues to make new dis-coveries as it explores Mount Sharp.

Maintaining high levels of productivity for the Curiosityrover is challenging. While the operations team has madesignificant accomplishments with the rover, doing so oftenrequires a large amount of human effort in planning, coordi-nating, sequencing and validating the development of com-mand products for the rover. Further, limitations in com-munication opportunities and anomalies on the vehicle cansometimes cause delays in accomplishing the team’s ob-jectives. These productivity challenges can result in theunder-utilization of the vehicle’s resources. These produc-tivity challenges are anticipated to increase as our agingfleet of sun-synchronous orbiters are replaced by non-sun-synchronous orbiters, which do not provide a consistent pat-tern of “end-of-day” downlink relays.

Copyright c© 2017, California Institute of Technology. U.S. Gov-ernment sponsorship acknowledged.

To better understand the productivity challenges faced bysurface missions, we conducted a case study of MSL oper-ations. The case study included structured interviews withscience and engineering operations personnel and a detailedanalysis of daily activities during three different sciencecampaigns. For each campaign in the study, we examinedthe activity performed by the operations team, how the teamdecided what to accomplish each day, how well these ob-jectives were achieved and how the team allocated vehicleresources during the campaigns. The interviews with opera-tions personnel were designed to complement the campaignanalysis by providing a broader perspective on surface mis-sion productivity factors.

In the following sections we describe the method we usedto collect and analyze data from MSL science campaignsand provide a discussion of the significant productivity chal-lenges that we identified during the study. These results arebeing used to guide the design and development of flightsystems and ground practices for future surface explorationmissions. In addition, these results are relevant to other ap-plications in which operators interact with robotic systemswith limited communication opportunities.

Background on MSL Mission OperationsWe begin with a brief overview of some important facets ofMSL operations to provide context for the case study. Oneof the challenges of surface missions is the degree to whichoperations are impacted by a priori unknown and changingenvironmental conditions. While orbital imagery providesvaluable information to guide activity, it does not capture allthe conditions that affect the rover. For example, while or-bital imagery may indicate that exploring a particular regionis promising to achieve a science objective, the specific sci-ence targets are not known until additional data is collectedfrom the rover itself.

As such, surface operations must be reactive and respondto the results of activity carried out during the previous sol(Martian day). This daily planning activity is referred to as“tactical” operations and is patterned after the tactical oper-ations developed for the Mars Exploration Rovers (Mishkinet al. 2006).

MSL operations augments this tactical process with“strategic” and “supratactical” phases (Chattopadhyay et al.2014). Strategic planning focuses on developing long-term

10 11 12 13 14 15 16 17 18 19 20 21

22 23 00 01 02 03 04 05 06 07 08 09 10

09

Uplink MRORelay

EngMaint

Pre-DriveScience SleepPost-Drive

ImagingPost-Decisional

Science

DAN Passive

EveImg

ODYRelay Sleep

MRORelay

AMScience

ODYRelaySleep UplinkSleep

RSM Heating

Cool

RSM HeatingMastcam Heating

DAN Passive

AwakeSleep

Sol780Master

Sol780Plan Sol 781 Master

CoolREMS Upload

REMS Get Data

DAN Actv

DAN Actv

EligibleforDecisionalData

RAD Background

RAD Background

REMS Background

REMS Background

Drive withMid-Drive Imaging

11

Turn OnChemCam

Turn OnChemCam

LIBSRaster

Turn OffChemCam

MastcamImaging Margin Cleanup

Pre-Drive Science

Mars Time

Mars Time

Sol 779 Master

Figure 1: Example sol in the life of the rover.

plans, typically spanning weeks or months, to achieve high-level objectives. Examples of strategic planning include thedevelopment of strategies for exploring a large geographicalarea or a high-level traverse route for reaching a distant ob-jective. The supratactical stage provides a bridge betweenthe long-term strategic plan and the day-to-day, highly reac-tive tactical process. The process is designed to coordinatethe complex science instruments and manage the constraintsand resources required to conduct campaigns.

An Example Sol in the Life of the RoverTo provide an idea of how the team operates the rover, Fig-ure 1 illustrates a typical drive sol, derived from an actual Sol780 command products. Following are some key aspects ofthe sol.

The plan for each sol begins with an “Uplink” window inwhich new commands products may be sent to the vehiclefrom Earth. There are various downlink windows through-out the sol in which the rover uses orbiter relays to sendcollected data back to Earth. While there are multiple down-link windows, certain downlinks have increased importancebased on the time that data in the relay will reach operators.If data from a relay will reach operators by the start of thenext tactical planning shift, then they relay is termed “de-cisional” because data from the relay can be used to makedecisions in for the rover’s next plan. In Figure 1 the starred“MRO Relay” represents the decisional relay for this sol. Itis important to realize that for this plan, only the data col-lected prior to this pass could be used to inform the nextplan.

Another important aspect of Figure 1 is how the teamstructures the sol into “blocks” of activity. For example,the main portion of the rover’s day consists of a Pre-DriveScience block, a Drive with Mid-Drive Imaging bock and a

Post-Drive Imaging block. The block structure organizes ac-tivity into related groups and allows a “Master” sequence toenforce timing between these major types of activity. Thelatter has to do with uncertainties in predicting the dura-tion of activity in the plan. Due to environmental conditionssuch as lighting, scene content and terrain, the time to per-form imaging and drive activities varies. The team uses theblock structure to ensure that if activity in one block runslonger than expected, it can be cut off to avoid interferingwith subsequent activity. To protect against loss of data,the team builds “Margin” into each block, to allow activi-ties to run longer than predicted. To deal with cases wheredurations exceed allocated margin, the team also sequences“Cleanups” after each block, to ensure that any activity isfinished before the start of the next block.

Restricted SolsThe vast majority of the surface mission is conducted withthe team restricting operations to daytime hours on Earth.The consequence is that the operations team is often out ofsync with the activity of the rover on Mars. Figure 2 illus-trates the impact this can have on the data available to theteam during planning. In the diagram, the end-of-day relayfrom the rover arrives on the ground late in the Earth day.The team waits until the next Earth day to begin planning.Meanwhile the rover is waking up for its next Mars day with-out a new set of command products from Earth. By the timethe team has completed the tactical process, they must waitfor the subsequent Mars morning to uplink the products tothe vehicle.

This often limits what the team can command the vehicleto do during the middle sol of Figure 2. If the vehicle wereallowed to make significant changes to its state, in particu-lar driving to a new location, this would significantly limit

FullActivity

RestrictedActivity

FullActivity

Planning

Figure 2: Mars activity vs. Earth planning.

the types of activities the team could command on the sub-sequent sol. These limited activity sols are referred to as“restricted sols” because the latency of data often restrictsthe type of activity the team can perform. Overall, 41% ofsols on the MSL mission are restricted sols.

Case Study DesignThe objective of the case study is to understand the factorsthat contribute to and detract from surface mission produc-tivity. We designed an exploratory case study to investigatethis topic (Yin 2014).

MSL is a complex and ever-changing mission. At the timeof our study the surface mission had been operating for al-most 4 Earth years, spanning over 1,300 days on Mars. Thenature of operations changes as the team pursues differenttypes of science campaigns and as different challenges arisedue to the condition of the vehicle and the environment inwhich it is operating.

The dynamic nature of the mission posed a challenge forthe design of our case study. We wanted to obtain an in-depth understanding of the productivity factors that affectsurface operations over a broad range of mission and envi-ronmental conditions. In order to manage the scope of thestudy, we chose to use select science campaigns and performa detailed study of the team’s day-to-day activities duringthis time. We conducted interviews with operational person-nel to contribute to the broad perspective of productivity. Inthis section we describe the selection of the case study cam-paigns and discuss the methodology used to collect data.

Selection of CasesThe primary purpose for conducting our case study wasto inform our research project in which we will developtechnologies and operational practices for enhancing surfacemission productivity. To manage the scope of our researchproject, we decided to primarily focus on remote sensing andnavigation operations. As such, our selection of case studycampaigns was biased toward those that emphasized thesetypes of activities.

There are two major types of campaigns that MSL con-ducts that emphasize remote sensing and navigation: char-acterizing a geographical area and driving to a distant, strate-gically identified location. Because different conditions canimpact operations, we wanted to be able to compare two dif-ferent campaigns of the same type, but performed in dif-ferent contexts. We selected three campaigns, two of whichhad the objective of exploring a geographical area and a third

with the objective of performing a strategic drive. Figure 3shows the campaigns that we have selected for the study.

The objective of the Pahrump Hills campaign was to studythe basal layer of Mount Sharp (Figure 3 (a)) (Stack et al.2015). Pahrump Hills is an interesting case study for multi-ple reasons. The light-toned outcrop of Pahrump Hills wasthe first exposure of bedrock making up the base of MountSharp that was encountered during the mission. The cam-paign was also significant in the way in which the explo-ration of this formation was conducted. The science teamdecided to conduct a “walkabout”, a practice used by fieldgeologists on Earth. In a walkabout, the geologist makes aninitial pass over the area performing a coarse survey whichis used to identify areas for more detailed study on a subse-quent pass.

After completing investigations at Pahrump Hills, theteam drove the rover toward higher levels of Mount Sharpfollowing a route referred to as Artist’s Drive (Figure 3 (b)).Along the way, the science team conducted a science cam-paign to capture images of the surrounding topography inorder to build a record of the stratigraphy (i.e., layering andstructure) of the sedimentary rock layers exposed in the val-ley walls.

The objective of the Marias Pass campaign (Figure 3 (c))was to explore a contact where the Murray formation (thetype of rock from Pahrump Hills) came into contact withan overlying geological formation called the Stimson for-mation (Milliken et al. 2016). This campaign has interest-ing similarities and contrasts with the earlier Pahrump Hillscampaign. Both campaigns sought to explore and character-ize a geological area. However, the more challenging terrainin the area and discoveries made during exploration resultedin a more dynamic campaign than the Pahrump Hills walka-bout campaign.

Conceptual Model of Achieving ObjectivesAlthough we were conducting an exploratory case study, andwanted to minimize unnecessary bias in our study, it is im-portant to have a theory of the topic under investigation tohelp guide the research (Yin 2014). The theory helps iden-tify data to be collected and provides a context for interpret-ing the results of the study.

For the purpose of our case study, we began with the gen-eral notion that mission productivity related to the team’sability to accomplish mission objectives. This can includehow much effort is required by the team to accomplish agiven objective as well as how long it takes, e.g., numberof sols, to achieve objectives. We expanded on this premiseby developing a conceptual model of how objectives are ac-complished in a surface mission. This model is shown inFigure 4. Several of the authors have worked surface opera-tions on the MER and MSL missions and this model is basedon the authors’ experience. We included the evaluation ofthis model in the case study. We presented it to the opera-tions personnel we interviewed to get their feedback and welooked for instances in our data collection where data did notfit well with this model. As will be discussed, we identifiedways in which this model should be expanded to encompass

End$of$Drive$Stops$Mid1Drive$Stops$

(a) Pahrump HillsSeason: SpringSols: 780 - 798 (19 sols)Drive dist: 152m

(b) Artist’s DriveSeason: Late SummerSols: 949 - 972 (24 sols)Drive dist: 567m

(b) Marias PassSeason: Late Summer, Early FallSols: 991 - 997; 1027 - 1043 (24 sols)Drive dist: 130m

Figure 3: Selected campaigns for case study (NASA/JPL-Caltech/MSSS/Univ. of Arizona).

additional productivity factors, but we did not identify errorsin the productivity factors originally described in the model.

The general flow of the diagram begins with the teamidentifying candidate activities that can be used to accom-plish their intent. These activities are developed and refinedduring operations planning until a set of command productsis ready to be uplinked to the vehicle. The vehicle exe-cutes these activities and produces results which are con-veyed back to Earth through telemetry and data products.This information, in turn, is used to support the developmentof subsequent activities and, potentially, new intent. Thecrossed out activities illustrate typical stages in the concep-tual model in which activity is limited in some way. Duringoperations planning, this can include restricting the scope ofan activity, deferring an activity to a later planning day oreven descoping an activity entirely. During execution, it caninclude partial or complete failure of an activity. The fol-lowing subsections describe each stage and the factors thatcan limit productivity in more detail.

Data Collection MethodologyUsing the conceptual model in Figure 4 as a guide, we de-veloped the data collection schema shown in Figure 5 thatincludes intent, activities, constraints and data along withrelations among these entities. We worked through each solof the campaigns, sifting through the plans, acquired dataand telemetry, and written reports from operations person-nel to collect and organize data with respect to this schema.The data gathering process was a combination of manuallyreading through activity plans and operations reports alongwith scripts we developed to assist in the collection process.The scripts we developed included utilities to identify linksbetween data products and the activities that used that dataand utilities to collect data on predicted and actual vehicleresource allocations. The objective in gathering this data isto identify cases of low and high productivity during eachcampaign and to help identify the factors that contributed toeach.

CaveatsIt is important to note some limitations in the case study.First, MSL is a vast and varied mission. While we be-

lieve we have selected campaigns that reflect issues com-mon throughout the mission and are relevant to future sur-face missions, the selected campaigns do not capture all as-pects of the mission. In particular, we did not include contactscience and sampling campaigns in our study. This would bea valuable area to consider in future extensions of this work.

One of the biggest challenges we faced in the data col-lection process was determining the intent for planned ac-tivity. Operations personnel were generally very good aboutdocumenting in reports the specific reason an activity wasincluded in the plan. However, it was not always clear howthis rationale related to specific campaign objectives. Weconcluded that for our objective of understanding productiv-ity factors it was sufficient to perform a “high level” cate-gorization of activity intent. As such, we defined three highlevel intent categories for activities:

Campaign Science: Activities that directly contributed to-ward the campaign’s objectives

Other Science: Activities that contributed to science objec-tives unrelated to the campaign under study

Engineering: Activities carried out primarily for monitor-ing and maintaining the health of the vehicle

Another important limitation of the study is that our infor-mation about what activities were considered during a givenplanning session is limited to what was documented in MSLReports. While the reports provide information about ac-tivities that were part of an earlier plan but removed beforeuplink, and they often document activities that were consid-ered but not included in a plan, they do not document allactivities that the team considered.

Case Study ResultsFollowing is an overview of results from the case study. Acomprehensive report of the case study is available in a tech-nical report (Gaines et al. 2016).

Analysis of Campaign Activity by SolWe begin our comparison of the three campaigns by sum-marizing the sol-by-sol breakdown of campaign activity inTable 1. Sols labeled “Campaign” were those that directly

Figure 4: Factors impacting surface mission productivity.

Complexity Analysis &Activity Refinement

Uplink

VehicleExecution

Downlink

LimitedActivity

DescopedActivity

AlternativeActivity

PreservedActivity

Activity

Data

Intent

Data

ActivityLostActivity

FailedActivity

PartialActivity

SuccessfulActivity

Data

Data

AddedActivity

Constraint

Constraint

DeferredActivity

ActivityDevelopment

Figure 5: Data collection schema.

contributed to the campaign objectives with remote sensingand/or drives. “Campaign Multi-Sol” sols are those in whichsignificant activity was performed toward the campaign ob-jectives as part of a multi-sol plan, either due to a weekendor restricted planning. The reason for calling these sols outseparately is that the presence of the multi-sol plan limitedthe team’s options for these sols. For example, had therenot been a multi-sol plan, the team may have opted for tomove up activity that was performed in a subsequent plan(e.g. a drive activity) which would have reduced the overallnumber of sols required to achieve the campaign objectives.The “Extra Drives” label denotes sols in which unexpectedadditional drives were required to reach objectives. This in-cludes cases where a planned drive faulted out early and hadto be re-planned on a subsequent sol. “Post-Drive Multi-Sol” sols were those sols in which the team was not able toachieve substantial campaign objectives due to lack of datafollowing a drive during a multi-sol plan. The sols labeled“Deferred” were sols in which campaign objectives were un-expectedly deferred due to the need to respond to an issueidentified during tactical plan development or in response toan event from received downlink data. Finally, “Runout” aresols of very low activity that used in cases the team had to

create multi-sol plans but the tactical timeline capacity didnot allow for sufficient time to develop activities for all solsof the plan.

The first takeaway from Table 1 is that there are a largepercentage of sols in each campaign that are not making sig-nificant progress toward campaign objectives. This indicatesthere is significant potential for increasing mission produc-tivity.

Our second observation is that although Pahrump Hillsand Marias Pass shared a common objective, characteriza-tion of a geological region, it is Artist’s Drive and MariasPass campaigns that have a nearly identical campaign activ-ity profile. This indicates, that for the selected campaigns,the overall campaign objectives was not a significant factorin determining productivity. Instead, Table 1 indicates re-stricted sols and terrain conditions appear to be the dominantfactors in the productivity for these campaigns. AlthoughArtist’s Drive and Marias Pass had different campaign ob-jectives, they shared a common pattern of restricted sols andthey took place in similar, challenging, terrain conditions.Pahrump Hills had a total of 9 tactical shifts (days the tac-tical team worked) of which 7 were during restricted peri-ods of the mission. In contrast, Artist’s Drive and MariasPass had more total shifts and fewer restricted shifts thanPahrump Hills. This difference in number of restricted solsbetween Pahrump Hills and the other campaigns is largelyluck of campaign timing. The Pahrump Hills campaign be-gan just as the mission was transitioning into restricted sols,while both Artist’s Drive and Marias Pass occurred during aperiod in which most of the sols were nominal.

Table 1 also highlights the significance of terrain for thesetypes of campaigns. Artist’s Drive and Marias Pass both oc-curred within more complex terrain than Pahrump Hills. Theterrain made drive planning more challenging and resultedin more sols in which a drive faulted out and needed to bere-planned the next day.

Analysis of Resource UsageAnother way to evaluate campaign productivity is to lookat how vehicle resources were used during the campaigns.We performed a detailed series of analyses on how the teamallocated vehicle resources each sol of the campaigns. Thisincluded tracking of predicted and actual allocations of flightcomputer duration, energy and data volume. Figure 6 shows

Sol Type Pahrump Hills Artist’s Drive Marias Pass

Campaign 6 9 10Campaign Multi-Sol 2 4 4Extra Drives 2 6 5Post-Drive Multi-Sol 5 4 3Deferred 3 1 0Runout 1 0 2

Total Sols 19 24 24

Campaign(32%(

Campaign(Mul/0Sol(10%(Extra(

Drives(11%(

Post0Drive(Mul/0Sol(26%(

Runout(5%(

Deferred(16%(

Campaign(37%(

Campaign(Mul/0Sol(17%(

Extra(Drives(25%(

Post0Drive(Mul/0Sol(17%(

Deferred(4%(

Campaign(42%(

Campaign(Mul/0Sol(17%(

Extra(Drives(21%(

Post0Drive(Mul/0Sol(12%(

Runout(8%(

Pahrump Hills Artist’s Drive Marias Pass

Table 1: Breakdown of sols for all campaigns.

780 781 782 783 784 785 786 787 788 789 790 791 792 793 794 795 796 797 798

Sol

0

200

400

600

800

1000

Dur

atio

n (M

inut

es)

Pre

dict

ed

Act

ual (

Est

imat

e)

EngineeringCommHeating

Campaign ScienceOther ScienceDAN Passive Only

MarginCleanupIdle

Wakeup/Shutdown

Figure 6: Allocation of flight computer duration forPahrump Hills Walkabout campaign.

an example using the flight computer duration allocationduring the Pahrump Hills campaign. Multi-sol plans are in-dicated with vertical black lines.

The analyses of resource allocations followed a similarpattern for each of the campaigns. The impact of multi-solplanning due to weekends and restricted sols had the largestimpact in how effectively the team was able to allocate re-sources toward campaign activity. The analysis showed ageneral decrease in overall activity across multi-sol plans.This is likely due to limitations in how much activity canbe developed during the tactical timeline. In addition, theteam is limited in the types of activity that can be performedafter a drive without ground-in-the-loop. The analysis alsoshowed a significant decrease in the allocation of resourcesto campaign objectives following drives during multi-solplans. This is because ground-in-the-loop is required to per-form the majority of the activities needed to accomplish thecampaign objectives.

To quantify this reduction in resource allocation, we plot-ted the per-sol allocation of flight computer duration and en-

Predicted 1 Actual Predicted 2 Actual Predicted 3 ActualSols in Plan

0

200

400

600

800

1000

Dur

atio

n (M

inut

es)

635

min

611

min

557

min

537

min

541

min

519

min

-12% -12% -15% -15%

EngineeringCommHeating

Campaign ScienceOther ScienceDAN Passive Only

MarginCleanupIdle

Wakeup/Shutdown

(a) Per-sol allocation of flight computer duration

Predicted 1 Actual Predicted 2 Actual Predicted 3 ActualSols in Plan

0

500

1000

1500

2000

2500

3000

3500

Ene

rgy

(Wh)

2438

Wh

2216

Wh

2267

Wh

2094

Wh

2164

Wh

1988

Wh

-7% -5% -11% -10%

SurvivalCPU

EngineeringComm

Heating Campaign ScienceOther Science

(a) Per-sol allocation of energy

Figure 7: Per-sol allocation of resource allocations for 1-sol,2-sol and 3-sol plans.

ergy for 1-sol, 2-sol and 3-sol plans. Figure 7 shows theresults. The results show that, on average, there is a reduc-tion of 12% flight computer duration use and 7% reductionof energy use per sol for 2-sol plans. Usage is further re-duced for 3-sol plans, with 15% and 11% reductions forflight computer duration and energy when compared to sin-gle sol plans. This is a reflection of the limited capacity ofthe tactical timeline to develop and validate command prod-ucts to make use of vehicle resources.

The resource analysis indicated that the team was not con-strained by energy during this campaign. There was suf-ficient unused energy and sufficient non-productive vehicle

780 781 782 783 784 785 786 787 788 789 790 791 792 793 794 795 796 797 798

Sol

0

1

2

3

4

5

6

7

8

920

0 W

Act

ivity

+ R

CE

Dur

atio

n (h

)

2.21.7

4.4

5.2

6.5

1.3

2.32.7

4.94.5

5.0

5.7

2.4

4.3

1.1

4.6

3.83.5

5.7

Shunt Energy Available RCE Duration with 100 W Activity

2.2 3.9 8.3 14 20 21 24 26 31 36 41 46 49 53 54 59 63 66 72Cumulative Available Duration (h)

0

200

400

600

800

1000

1200

1400

1600

1800

Equ

ival

ent E

nerg

y (W

h)

Figure 8: Estimate of extra duration availability for PahrumpHills Walkabout campaign.

awake time to support an estimated additional 72 hours ofcampaign-related activity over the span of the 19 sols forthe Pahrump Hills campaign, as shown in Figure 8. Simi-lar analysis estimated an additional 62 hours and 69 hoursof campaign activity could have been performed during theArtist’s Drive and Marias Pass campaigns, respectively.

DiscussionThe selection of campaigns for the case study provides aninteresting basis for comparisons. All three were selectedas examples of campaigns that emphasized driving and re-mote sensing. Two of the campaigns, Pahrump Hills Walk-about and Marias pass had a similar objective of charac-terizing a geological area, while Artist’s Drive had the ob-jective of performing a strategic drive with science imageryalong the way. Though the Pahrump Hills and Marias Passcampaigns had similar high-level objectives, they were con-ducted within very different contexts. The terrain at MariasPass was generally more challenging than at Pahrump Hills.In addition, the team chose to employ very different explo-ration strategies for these two campaigns, using a walkaboutapproach for Pahrump Hills and a linear approach, which ismore conventional for rovers, for Marias Pass.

On a more subjective note, discussions with operationspersonnel, including scientists and engineers, indicate thatthe team generally found the Pahrump Hills campaign to bemore satisfying than the others. Pahrump Hills is often re-ferred to as the way in which the team would like to conductexploration campaigns. This is an especially interesting ob-servation given that Table 1 shows that the number of pro-ductive campaign sols for Pahrump Hills was less than theother two campaigns in this study.

We believe there are two major factors that lead to thesense of satisfaction with the Pahrump Hills campaign. First,the team put together an excellent strategic plan to guide theexploration of Pahrump Hills. This helped the team to main-tain focus during the campaign and make effective use of therelatively limited number of non-restricted sols during thiscampaign. It also gave the team a clear picture of the con-sistent progress being made toward completing objectivesduring the coarse of the campaign.

A second factor is the difference in exploration strategiesemployed at Pahrump Hills and Marias Pass. The walkaboutapproach at Pahrump Hills allows the team to build in the ap-

plication of more cost-intensive resources with lower risk,because they eliminate the “discovery” and “uncertainty”in the first round by identifying the subset of best targets.A walkabout also provides a better match with the scienceteam’s internal processing of the data (both machine pro-cessing and mental). It gives the team a few weeks or monthsto digest what they are discovering before leaving.

A full assessment of the benefits of these two explorationstrategies is beyond the scope of a single case study. The in-terested reader is referred to Yingst et al. for additional dis-cussion on this topic (Yingst et al. 2015). Their conclusion isthat a walkabout approach can take more time to execute, buthas the potential for achieving higher quality results. One ofthe objectives we have with this case study is to leveragewhat we have learned from these productivity challenges toidentify flight and ground approaches that can reduce theoverhead of employing a walkabout approach.

Drawing from our analysis of campaign activity and in-terviews with operations personnel, we developed a list ofthe significant productivity factors we observed in the study.Some of the more significant factors were:

Ground-in-the-loop for target selection and drive planning:This results in a significant drop in productivity on solsthat follow drives during restricted periods of the mission.Even during non-restricted sols, it constrains the timingof activity that can change the state of the vehicle andactivity that acquires decisional data to occur prior to thedecisional pass.

Capacity of tactical timeline to fill multi-sol plans: Dueto the time required to develop and validate commandproducts, the amount of overall activity across a multi-solplan is generally lower than the amount of activity acrossa similar number of single sol plans.

Ground-in-the-loop to respond to outcome of activity:We observed several instances where the team decidedto re-do an activity, or return to a previous location,after observing the data received from the vehicle. Thisincluded the need to re-plan drives that faulted out orrequire observations that did not have intended resultsdue to lighting conditions or targeting problems.

Predicting Available Vehicle Resources: Inaccuracies inresource modeling, including activity power and dura-tion requirements, can result in unnecessarily restrictingplanned activity. Because activity durations tend to beconservative, activities did not usually require time allo-cated for margin and cleanup, resulting in significant idletime during plan execution.

Interpersonal Communication: Effective communicationwas identified as a significant productivity factor by manyof the participants in each of the operations roles we in-terviewed. This includes communication between the sci-ence and engineering teams and among roles within sci-ence and engineering teams.

Science Team Engagement: It is important for scienceteam members to be aware of the current and past con-text of the mission to make informed science decisions.

A significant challenge to engagement is that many teammembers work on the mission part time.

ConclusionsA successfully deployed Martian rover represents an im-mensely valuable asset. The mission has a strong interestin getting the most out of the vehicle to increase the returnon investment. This is further motivated by the fact that therover’s capabilities will inevitably degrade over time. Assuch there is a strong interest in enhancing the productiv-ity of future surface rover missions. We conducted the casestudy of MSL campaigns with the objective to better under-stand the productivity challenges facing surface missions.The study included interviews with mission scientists andengineers along with detailed study of three science cam-paigns.

The responses from the interview participants and ouranalysis of the campaigns showed that there are opportuni-ties for increasing surface mission productivity. In particu-lar, we observed that it is often the case that the vehicle hasmore available resources than the operations team is able touse. The case study identified a variety of issues that arelimiting the productive use of these resources.

Perhaps the largest factor observed in our study of MSLcampaigns was due to restricted sols and, more generally,the reliance on ground-in-the-loop to inform a large por-tion of the rover’s activities. This reliance on ground-in-the-loop places constraints on when certain types of activitiescan be performed. Activities that generate data needed tomake decisions for the next shift must be performed priorto the decisional communication pass. Similarly, activitiesthat change the state of the rover in such a way as to in-validate that decisional data (e.g. driving the rover to a newlocation) cannot be performed after the decisional pass. Theground-in-the-loop reliance is expected to become an evenmore significant liability to surface missions as the fleet ofaging sun-synchronous orbiters are replaced with non-sun-synchronous orbiters.

The next biggest productivity factor was the ground-in-the-loop requirements for responding to problems in previ-ous plans. This resulted in extra sols needed to drive to loca-tions and the need to repeat or give up on observations thatdid not meet expectations.

Our next objective is to identify changes to flight systemsand ground operations practices to overcome these chal-lenges and enable high levels of productivity for future sur-face missions. The findings from this study will guide thedesign and development process by helping to define the ca-pabilities required to meet these productivity challenges. Forexample, we are focusing on how we can enable the engi-neering team to effectively interact with the vehicle withoutknowing the exact state in which the vehicle will be whencommand products are received. We are also investigatingmethods to enable the vehicle to detect when activities arenot going as planned and robustly respond without the needto wait for ground interaction. We are also leveraging ex-amples from the campaigns studied to define scenarios thatwill be used to focus the development and to evaluate theperformance of our work.

AcknowledgmentsThis research was conducted at the Jet Propulsion Labo-ratory, California Institute of Technology, under a contractwith the National Aeronautics and Space Administration.This work was funded by the Jet Propulsion Laboratory Re-search and Technology Development program.

We are extremely grateful to the MSL team for their sup-port. The generous assistance they provided in enabling ac-cess to MSL data, answering our many questions and sup-porting our interviews was invaluable to this study.

References[Chattopadhyay et al. 2014] Chattopadhyay, D.; Mishkin,A.; Allbaugh, A.; Cox, Z. N.; Lee, S. W.; Tan-Wang, G.; andPyrzak, G. 2014. The Mars Science Laboratory supratacticalprocess. In Proceedings of the SpaceOps 2014 Conference.

[Gaines et al. 2016] Gaines, D.; Doran, G.; Justice, H.; Ra-bideau, G.; Schaffer, S.; Verma, V.; Wagstaff, K.; Vasavada,A.; Huffman, W.; Anderson, R.; Mackey, R.; and Estlin, T.2016. Productivity challenges for Mars rover operations: Acase study of Mars Science Laboratory operations. Techni-cal Report D-97908, Jet Propulsion Laboratory.

[Grotzinger and others 2015] Grotzinger, J. P., et al. 2015.Deposition, exhumation, and paleoclimate of an ancient lakedeposit, Gale crater, Mars. Science 350(6257).

[Milliken et al. 2016] Milliken, R.; Hurowitz, J.; Bish, D.;Grotzinger, J.; and Wiens, R. 2016. The chemical and min-eralogical stratigraphy of lower Mt. Sharp: Relating roverobservations to orbital predictions. In Proceedings of the47th Lunar and Planetary Science Conference.

[Mishkin et al. 2006] Mishkin, A. H.; Limonadi, D.;Laubach, S. L.; and Bass, D. S. 2006. Working the Martiannight shift: The MER surface operations process. IEEERobotics and Automation Magazine 13(2):46–53.

[Stack et al. 2015] Stack, K. M.; Grotzinger, J. P.; Gupta, S.;Kah, L. C.; Lewis, K. W.; McBride, M. J.; Minitti, M. E.;Rubin, D. M.; Schieber, J.; Sumner, D. Y.; Beek, L. M. T.J. V.; Vasavada, A. R.; and Yingst, R. A. 2015. Sedimen-tology and stratigraphy of the Pahrump Hills outcrop, lowerMount Sharp, Gale Crater, Mars. In Proceedings of the 46thLunar and Planetary Science Conference.

[Vasavada et al. 2014] Vasavada, A. R.; Grotzinger, J. P.;Arvidson, R. E.; Calef, F. J.; Crisp, J. A.; Gupta, S.;Hurowitz, J.; Mangold, N.; Maurice, S.; Schmidt, M. E.;Wiens, R. C.; Williams, R. M. E.; and Yingst, R. A. 2014.Overview of the mars science laboratory mission: Bradburylanding to yellowknife bay and beyond. Journal of Geophys-ical Research 119(6):1134–1161.

[Yin 2014] Yin, R. K. 2014. Case Study Research: Designand Methods. SAGE Publications.

[Yingst et al. 2015] Yingst, R.; Berger, J.; Cohen, B.; Hynek,B.; and Schmidt, M. 2015. A test of two field methods: De-termining best practices in reconnoitering sites for habitabil-ity potential using a semi-autonomous rover. In Proceedingsof the 46th Lunar and Planetary Science Conference.