the@int software systems
TRANSCRIPT
Introduction Software architecture Data flow
Outline
Introduction
Software architecture
Data flow
2/21
Introduction Software architecture Data flow
Overview
Introduction
Software architecture
Data flow
3/21
Introduction Software architecture Data flow
Overall status of Software Systems
I Preliminary Design Review in May 2017I Updated documents needed
I Prototyping partially doneI Final Design Review planned for December 2019
I Telescope refurb tender on critical path
4/21
Introduction Software architecture Data flow
Mission
I Design and implement a fully robotic control system forthe INT and HARPS3I To carry out the Terra Hunting Experiment and open time
observationsI Must accommodate a possible companion instrumentI Must accommodate a possible solar telescope
I Science requirements – see HARPS3-EXE-0004I Operations model and use cases – see HARPS3-CAM-0007
5/21
Introduction Software architecture Data flow
Overview
Introduction
Software architecture
Data flow
6/21
Introduction Software architecture Data flow
Instrument control software
I Telescope Control System (TCS)I Integrates mount, dome and environment sensing
I Cass Instrument Control Software (CICS)I Cass Unit
I ADCI Cassegrain Fibre AdapterI Polarimeter
I Acquisition and guiding (2 fields)I Calibration Unit
I Spectrograph Control Software (SCS)I Detector Control Software (DCS)
8/21
Introduction Software architecture Data flow
Supervisory software
I Automated schedulerI Schedules daytime calibrations, nighttime observations,
focus, pointing testI Observing Control Software (OCS)
I Observation Sequencer, Fault Manager, FITS BuilderI Data handling
I Engineering data collector and databaseI Local database and automatic transfer to Exeter mirror
I Science data archiverI Local storage, automatic transfer to Exeter, load into database
I Data Reduction SoftwareI Based on ESPRESSO DRS
9/21
Introduction Software architecture Data flow
Overview
Introduction
Software architecture
Data flow
10/21
Introduction Software architecture Data flow
Actors
Open time observer A scientist who wishes to carry outobservations with the INT, in service mode, andmake use of the resulting data
Consortium observer A scientist from the HARPS3consortium working on the Terra-HuntingExperiment (THE)
Consortium engineer An engineer from the HARPS3consortium, responsible for ensuring that bothopen time and THE observations are carried outsuccessfully
WHT operator Responsible for ensuring safe startup andshutdown of the INT. MoU states that the ING willprovide safety override for robotic operation
11/21
Introduction Software architecture Data flow
Automated Scheduler
Phase 2 - Add observation group
THE Scheduler
Observation Pool
Sequencer
Scheduler
Scheduler gets observation block
for 'best' target from pool
Request for target sent across network Observation block
returned back across network to sequencer
Observation blocks added to pool
Phase 1 - New observation group requested
Yes
WEB APPLICATION MESSAGING PROTOCOL (WAMP)
Accepted?
A
A
A A
B
B
BB
B
C
C
C
DD
D
D
D
D
D
16/21
Introduction Software architecture Data flow
Engineering data collection
I INT/HARPS3 sub-systems will publish engineering datacontinuously
I Published data will automatically be collected and storedin the on-site engineering databaseI Data collection must be robust to sub-systems going
offline/onlineI Query facility used by OCS FITS Builder
I Database mirrored in Exeter, updated nightly (TBD)I High-bandwidth data only kept for N days, available for
manual transferI Guide camera videoI Very low-level debug messages
18/21
Introduction Software architecture Data flow
Science/calibration data collectionI FITS files will be assembled from a raw detector frame and
ancillary data/metadata provided by the robotic controlsystemI Science metadata: required by DRS (timestamp, exposure,
target and telescope coordinates, . . . )I Engineering metadata: summarise instrument status
during exposureI Exposure meter time seriesI Acquisition and integrated guide images
I Temporary storage on siteI Transfer to main Archive in Exeter
I Automatic pipeline reduction on importI Retain multiple, traceable reductions of each observation
I Edit metadata (e.g. spectral type) and re-reduceI Upgrade DRS (occasionally) and re-reduce
19/21
Data Reduction Software
I ESPRESSO DRS statusI Validated and operational on ParanalI Able to read HARPS dataI New web-based "trigger" front-end for CPL recipes
I New work for HARPS3I Adapt ESPRESSO DRS for HARPS-X data and test
(Geneva)I FITS Builder (Cambridge)I Integrate/adapt Trigger (Cambridge)
I Details to be discussed with GenevaI Interfaces with HARPS3 Archive for pipeline input/output
files (Exeter)I Science QC tools (includes alerts) for consortium observer
(Exeter)
20/21
Alerts
I Alerts can be generated fromI Engineering dataI DRS-processed data
I FlexibilityI Use any monitored variablesI Complex alert criteria?
I Combinations of variablesI Linked to events
21/21
User interfaces (1)
I Operations UIs (consortium use)I Observing dashboardI Scheduler “queue” UII Engineering archive access
User interfaces (2)
I Engineering GUIs (view only during robotic operation)I Telescope systemI Cass instrumentI SpectrographI Detector
I Observer tools (community/consortium use)I Exposure Time CalculatorI Scheduler simulation tool
I For repeated observation or surveyI Science archive access
Communications framework
I Robotic control of telescope + instrument requires adistributed event-driven software system
I Components communicate over network using WAMPprotocolI Open standard WebSocket subprotocolI Data serialization using JSON or MsgPack
I Use open-source Crossbar.io router and Autobahn clientlibs (many languages)
WAMPWAMP implements two communications patterns:1. Remote Procedure Call (command/response)2. Publish Subscribe
Realistic WAMP example
Python asyncio event loop
WAMP
translate
pexpect
Receive remoteprocedure call
Convert into messagefor target instrument
Dispatch message to instrument and
yield
WAMP
translate
pexpect
Return from remote procedure
call
Convert into return value
Receive message from instrument
Periodic timer
translate
pexpect
Initiate poll of instrument data
Convert into messagefor target instrument
Dispatch message to instrument and
yield
WAMP
translate
pexpect
Publish value
Convert into return value
Receive message from instrument