operator sanity via automation - fsw...
TRANSCRIPT
![Page 1: Operator Sanity via Automation - FSW Workshopflightsoftware.jhuapl.edu/files/2017/Day-1/11-Parsons... · 2017-12-05 · Dongtaijinai’er Salt Lake, China – May 3, 2016 Operator](https://reader034.vdocuments.us/reader034/viewer/2022042120/5e99c7d7b901035b424899fe/html5/thumbnails/1.jpg)
Dongtaijinai’er Salt Lake, China – May 3, 2016
Operator Sanity via AutomationThe Systems Behind Flock 3p/Flock 2k Commissioning
Nate Parsons, Alex Paul, Lawrence Leung ({nate,apaul,lawrence}@planet.com)
![Page 2: Operator Sanity via Automation - FSW Workshopflightsoftware.jhuapl.edu/files/2017/Day-1/11-Parsons... · 2017-12-05 · Dongtaijinai’er Salt Lake, China – May 3, 2016 Operator](https://reader034.vdocuments.us/reader034/viewer/2022042120/5e99c7d7b901035b424899fe/html5/thumbnails/2.jpg)
Jeddah, Saudi Arabia
Introduction
• Planet Mission 1: image the entire Earth’s landmass every day• Dove Launches, 2017
○ Flock 3p - 88○ Flock 2k - 48○ Flock 3m - 4
• Operations Philosophy: automate as much as possible, see“Commissioning the World’s Largest Satellite Constellation”†
• Until 2017, ADCS commissioning was manual
† https://digitalcommons.usu.edu/smallsat/2017/all2017/138/
![Page 3: Operator Sanity via Automation - FSW Workshopflightsoftware.jhuapl.edu/files/2017/Day-1/11-Parsons... · 2017-12-05 · Dongtaijinai’er Salt Lake, China – May 3, 2016 Operator](https://reader034.vdocuments.us/reader034/viewer/2022042120/5e99c7d7b901035b424899fe/html5/thumbnails/3.jpg)
Shallow Seas, Bahamas – February 4, 2016
CommissioningProblem: ADCS Commissioning
![Page 4: Operator Sanity via Automation - FSW Workshopflightsoftware.jhuapl.edu/files/2017/Day-1/11-Parsons... · 2017-12-05 · Dongtaijinai’er Salt Lake, China – May 3, 2016 Operator](https://reader034.vdocuments.us/reader034/viewer/2022042120/5e99c7d7b901035b424899fe/html5/thumbnails/4.jpg)
CommissioningTimeline of events
Launch
Initial Checkout
Detumble, Panel
Deploy
Calibration Maneuver 1
Calibration Maneuver 2
... Nominal Operations
Calibration Maneuver N
![Page 5: Operator Sanity via Automation - FSW Workshopflightsoftware.jhuapl.edu/files/2017/Day-1/11-Parsons... · 2017-12-05 · Dongtaijinai’er Salt Lake, China – May 3, 2016 Operator](https://reader034.vdocuments.us/reader034/viewer/2022042120/5e99c7d7b901035b424899fe/html5/thumbnails/5.jpg)
Tylihul, Estuary, Ukraine
• Scheduling maneuvers○ After software updates○ Within orbital constraints○ Around contact plan
• Waiting for data• Handling bad data• Tracking progress• Dozens of heterogeneous satellites
CommissioningChallenges
![Page 6: Operator Sanity via Automation - FSW Workshopflightsoftware.jhuapl.edu/files/2017/Day-1/11-Parsons... · 2017-12-05 · Dongtaijinai’er Salt Lake, China – May 3, 2016 Operator](https://reader034.vdocuments.us/reader034/viewer/2022042120/5e99c7d7b901035b424899fe/html5/thumbnails/6.jpg)
London Array Wind Farm, United Kingdom – April 17, 2016
Solution: The Sequencer
![Page 7: Operator Sanity via Automation - FSW Workshopflightsoftware.jhuapl.edu/files/2017/Day-1/11-Parsons... · 2017-12-05 · Dongtaijinai’er Salt Lake, China – May 3, 2016 Operator](https://reader034.vdocuments.us/reader034/viewer/2022042120/5e99c7d7b901035b424899fe/html5/thumbnails/7.jpg)
Sequencer: Goals
• Handle each maneuver• Execute deterministically• System state: easy to determine, understand, communicate• Allow manual intervention• Configurable• Utilize existing infrastructure
○ Automated contact scheduling○ Scheduling API○ Asset config service○ Batch processing system○ Alerts (Slack)
![Page 8: Operator Sanity via Automation - FSW Workshopflightsoftware.jhuapl.edu/files/2017/Day-1/11-Parsons... · 2017-12-05 · Dongtaijinai’er Salt Lake, China – May 3, 2016 Operator](https://reader034.vdocuments.us/reader034/viewer/2022042120/5e99c7d7b901035b424899fe/html5/thumbnails/8.jpg)
Sequencer: DesignCommissioning Events Sequencer State Machine
Schedule Maneuver
Uplink Schedule
Execute Maneuver
Downlink Telemetry
Process Telemetry
Uplink Calibration Parameters
Ground Space Init
Take
Check
Apply
Success
Cancel
Failure
![Page 9: Operator Sanity via Automation - FSW Workshopflightsoftware.jhuapl.edu/files/2017/Day-1/11-Parsons... · 2017-12-05 · Dongtaijinai’er Salt Lake, China – May 3, 2016 Operator](https://reader034.vdocuments.us/reader034/viewer/2022042120/5e99c7d7b901035b424899fe/html5/thumbnails/9.jpg)
scheduleitems.py
Sequencer: Architecture
• Sequencer - command line interface• State Machine
○ Determined via asset config service○ State objects: when invoked, return action items
• Action Items○ Set config value (to set state)○ Skip schedule item○ Take schedule item○ Start processing job○ Apply configuration params
• Schedule Items○ Objects: know how to take and skip themselves
• Maneuver Modules○ Scheduling constraint logic○ Telemetry Validation/Processing○ Determine when to retry maneuver vs fail and alert
statemachine.py
actionitems.py
sequencer.py
maneuver interfaces
maneuver definitions
![Page 10: Operator Sanity via Automation - FSW Workshopflightsoftware.jhuapl.edu/files/2017/Day-1/11-Parsons... · 2017-12-05 · Dongtaijinai’er Salt Lake, China – May 3, 2016 Operator](https://reader034.vdocuments.us/reader034/viewer/2022042120/5e99c7d7b901035b424899fe/html5/thumbnails/10.jpg)
Central Pivot Irrigation, Saudi Arabia
● Manager○ For manual intervention○ Request/cancel/retry/apply/reprocess any sequence
● Kickoff Processes (runs every 2 hours)○ For each sequence,○ if it can advance,○ kicks off state processing
● Process State○ Invoke a single state per run, ○ execute the resulting action items
● Commissioning Report○ Gathers state of all sequences,○ stores in metrics system
Sequencer: UsageServices Deployed
![Page 11: Operator Sanity via Automation - FSW Workshopflightsoftware.jhuapl.edu/files/2017/Day-1/11-Parsons... · 2017-12-05 · Dongtaijinai’er Salt Lake, China – May 3, 2016 Operator](https://reader034.vdocuments.us/reader034/viewer/2022042120/5e99c7d7b901035b424899fe/html5/thumbnails/11.jpg)
Kashaga, Oil Field, Kazakhstan
Sequencer: Testing
1. Unit tests on state machine logic2. Manual tests of action items3. Regression tests on processing of past data
○ Pro: Exercised Sequencer/ADCS code interfaces○ Con: Require satellites with similar hardware
4. Piecewise with virtual satellites○ Pro: real mission control○ Con: no physics model
5. End-to-end retired satellites in space○ Pro: High fidelity○ Con: Hardware mismatch
6. End-to-end with production satellites in space○ Pro: Highest fidelity○ Con: Acceptable loss in production imagery
“All the testing”
![Page 12: Operator Sanity via Automation - FSW Workshopflightsoftware.jhuapl.edu/files/2017/Day-1/11-Parsons... · 2017-12-05 · Dongtaijinai’er Salt Lake, China – May 3, 2016 Operator](https://reader034.vdocuments.us/reader034/viewer/2022042120/5e99c7d7b901035b424899fe/html5/thumbnails/12.jpg)
Muir Woods & Mt. Tamalpais, California, USA – December 23, 2015
Results
![Page 13: Operator Sanity via Automation - FSW Workshopflightsoftware.jhuapl.edu/files/2017/Day-1/11-Parsons... · 2017-12-05 · Dongtaijinai’er Salt Lake, China – May 3, 2016 Operator](https://reader034.vdocuments.us/reader034/viewer/2022042120/5e99c7d7b901035b424899fe/html5/thumbnails/13.jpg)
Results
● All goals met● Satisfied operators● Stable architecture● Reduced ADCS commissioning time● Things to improve
○ Adapt for nominal operations○ Alert on “stalled” sequences○ Reduce scheduling latency
† https://digitalcommons.usu.edu/smallsat/2017/all2017/138/
Success!
![Page 14: Operator Sanity via Automation - FSW Workshopflightsoftware.jhuapl.edu/files/2017/Day-1/11-Parsons... · 2017-12-05 · Dongtaijinai’er Salt Lake, China – May 3, 2016 Operator](https://reader034.vdocuments.us/reader034/viewer/2022042120/5e99c7d7b901035b424899fe/html5/thumbnails/14.jpg)
Mercator Projection
Thank You Flight Software Workshop 2017Visit us at www.planet.com
Nate Parsons | Missions Software Team | [email protected]