jon thaler fermilab director’s review 7/26/20061 sispi s urvey i mage s ystem p rocess i...

Jon ThalerJon Thaler Fermilab Director’s Review 7/26/2006 1

SISPI

Survey Image System Process Integration- Realtime Software on the mountain

SISPI’s job:• Coordinate camera operation (filters, shutter, readout, etc.)

• Ensure smooth data flow and logging

• Perform quality analysis

• Provide a user interface (for monitoring and control)


Performance Metrics

Collect the data and deliver it, without loss, to Data Management° Focal plane (CCD) data° Instrument metadata° Status and history logging to database

Perform Quality Analysis (QA), to monitor data integrity° Useful presentation of information via GUI° Software alarms (hardware alarms belong to subsystems)

Perform manual or automated sequences of operations° GUI interface° Instrument calibration and configuration

Maintainable by CTIO° DES only uses 30% of the telescope time.

Image Builder (IB)


GUI

ImageStabilization

Focus

User commands

Sky Cameras

TCS

DM

Alarms

StatusInput Log

DatabasesObservationControl

Guide

ImageAcquisitionand Focus

ImageStabilizationInstrumentControl

ImageStabilization

ImageBuilding &Analysis

Here behumans

SISPI definesthe

interfaces

Logical Connections

Legend:Control paths(response required)

Data paths

Processes

Data

Subsystems


All subsystems need an ICD. OCS draft exists.

Management: JJT supervises. FRD/ICD & architecture definition (milestone) 12/06. Estimate resource requirements 12/06. Assign responsibilities & hire personnel 2/07. Test complete system at FNAL (milestone) 3/09. Install at CTIO 9/09

Infrastructure: OSU - Honscheid + postdoc. Choose message & data protocols. 12/06. Complete focal plane emulation 6/07. Integrate w/o CCDs (using emulators) 12/07. Select database technology 12/06 Fermilab - Buckley-Geer . Demonstrate database functionality 1/08Alarms (needs thought)

Observation Control:OCS (Observation control system) UIUC - Qian & gs. Enumerate command sequences (use cases) 12/06. Demonstrate OCS communications (with FCS) 7/07. Prototype complete OCS 4/08. OCS ready for test (milestone) 10/08

ObsTac (Observation tactics) FNAL - Annis. First version 7/07. Ready for test 9/08GUI UIUC - Qian & gs. List displays and controls 4/07. Prototype display/control (of FCS) 10/07. First complete version 5/08. Ready for test 2/09

Tasks, Milestones, & People


Image Acquisition & FocusFCS (aka Monsoon Supervisor Layer – MSL) UIUC - 2nd postdoc. First prototype 12/06. Ready for test 6/07Focus Chicago - Gladders. Algorithm 9/07. Feedback loop (milestone) 1/08

Image StabilizationGCS (aka Monsoon Supervisor Layer – MSL) UIUC - 2nd postdoc. First prototype (adapt from FCS) 3/07. Ready for test 8/07Guide Barcelona - Castander. Algorithm 6/07. Demonstrate guiding with emulators 10/07

Data collection and analysisIB (Image builder) UIUC - 2nd postdoc. Specify content (metadata) 4/07. Ready for data flow test 10/07. Complete 12/08QA (Quality analysis) UIUC - with DM people. Specify tests 7/07. Software complete 7/08

ICS (Instrumentation control system) FNAL - DiehlFilter control OSU - Honscheid + gs. 1st prototype SW 6/07. HW & SW complete (ready for test) 3/08Focus, cryogenics, & filter control

Cloud Camera FNAL - Tucker. Commission HW & SW 11/09

Tasks, Milestones, & People


End


HardwarePrototyping:Computers $10kEmulator 7k

Installation:Network 25kComputing 36kControl room (displays, etc.) 25k

SoftwareLicenses (IDL, LabVIEW, etc.) 20k

$123kLabor3 faculty (part time) MG, KH, JJT7 staff scientists (part time) JA, LB-G, FC, TD, IK,

WM, DT3 postdocs for 3 years Qian + two more3 graduate students for 3 years tbd

Cost


Proposed SISPI and Front End NetworkShowing Bulk Data Rates (MBps)

Control signalsare not shown

DHE

DHE

DHE

DHE

DHE

DHE

Sci.

Sci.

Sci.

Sci/Focus

Sci/Focus

Guide

Instrumentation

Database

PAN

PAN

PAN

PAN

PAN

PAN

34 MBps

34

34

34

34

8

24-

po

rt s

witc

h

0.01

0.01

5.8

5.8

1.9

1.9

5.8

0.8

(the

sam

e) 2

4-p

ort

sw

itch

Focus

Guide

5.3

5.3

5.3

5.3

1.3

0.8

5.3 + < 5.3

5.3 + < 5.3

5.3 + < 5.3

5.3 + < 5.3

small

small

DM (not SISPI): Data staging on mountain

GUI

21

< 21 ?

IB/QA/disk

IB/QA/disk

IB/QA/disk

IB/QA/disk

PC farm

Fiber3

3

3

2

2

2

4

DAQelectronics PCs Ethernet

FCS & GCS


Other Design Choices

We do not want to reinvent the wheel.

• Control protocols: Adapt the SOAR messaging system?

Supported by CTIO. TCS & ICS will use LabVIEW.

• Data flow: TCP/IP sockets. Client/server or publish/subscribe.

• QA and GUI: Being studied. What diagnostics? What data to humans?

JJT has visited CTIO & Keck control rooms.

• Development: We will be able to operate a partial system.

Before CCDs exist, we wll use emulator FPGAs.

• Mountain storage: 3 days of data:~ 2 TB HD local to IB processors.

Database (calibration, log, SkyCam, etc.)

CTIO uses MySQL.


Other Design Choices (2)

• Subsystems: Interface hardware is the responsibility of the subsystems (FCS, GCS, IC, & TCS).

• Alarms: Safety is a hardware responsibility. SISPI handles other fault conditions.


People

UIUC Physics:Senior Physicists: Inga Karliner, Mats Selen, JJT

Postdocs: Tao Qian, (+ another later in 2006)

Students: Youcai Wang, Curtis Leung (summer)

Engineers: Mike Haney, Todd Moore

UIUC works on DAQ electronics and SISPI. I will only discuss SISPI (the people in green).

Also:

Ohio State University Klaus HonscheidFermilab Liz Buckley-Geer and Wyatt MerrittCTIO Tim Abbott

UIUC and OSU will be the primary sites in the near term.Effort will ramp up over the next six months.Timely assembly of the team is our biggest risk.


Glossary

DC Disk Cache Local image storageDM Data management Offline analysisFCS Focal plane control system Science CCD controllerFD Focal plane data Science CCD dataCGS Guider control system Guide CCD controllerGD Guider data Guide CCD dataGUI Graphical user interface Humans sit here.IB Image builder Assembles an imageICS Instrument control system Moves filters, etc.ID Instrument data TelemetryObsTac Observation tactics Exposure sequencerOCS Observation control system Coordinates everythingQA Quality analysis Real-time feedbackSkyCam Sky cameras Atmospheric monitoringTCS Telescope control system Tracking on the sky


Some Pieces

Some pieces of this are described in other talks:

Image Acquisition T. Shaw & V. SimaitisFocus M. Gladders & S. Kent (optics session)Guide F. CastanderTCS T. Abbott


More Detailed Logical Diagram

IB GUI

FD GD ID

DC

QA

GCSMonsoon B ICS Filter Focus CryoShutter

FCSMonsoon A

Focus

Guide

User commands

Legend:

Control paths

Data paths

Processes

Data

Subsystems Sky CameraTCS

DM

AlarmsEvery process sends/receives data to/from databases.To avoid clutter, arrows are not drawn.

StatusInput Log

DatabasesOCS ObsSeq

Sequencecommands


Observation Control

ObsTac

Control from GUI

Control from Focus

Control from Guide

OCS

Control to:Image acquisitionImage stabilizationInstrument controlTelescope control(and Image builder?)

Sequencedata


Image Acquisition & Focus; Stabilization

Control

from OCS

GD

Monsoon B

Guide

GCD

Guide control

to TCS

via OCS

Data to

IB and GUI

Control

from OCS

FD

Monsoon A

Focus

FCD

Focus control

to ICS

via OCS

Data to

IB and GUI

The same interfaces?


Instrument Control; Image Building

Control

from OCS

ID

Data to

IB and GUI

Filter Focus CryoShutter DC

QA

Data to

DM and GUI

IB

QAD

Data from

Focal plane &

Instrument Control


Electronics: Each ACQ card has 12 input channels. Produces 2 or 4 bytes/pixel.Each DHE crate has 2 ACQ & 1 CBB card or 3 ACQ & 2 CBB cards. Each CBB can control 9 CCDs.

Science CCDs: 62 2k4k CCDs, read out @ 250 kHz, 2 video outputs. 32 MB/CCDDigitization time = 4 Mpixel * 4 µs = 17 sec

Focus/WF CCDs: 8 2k2k CCDs, read out @ 250 kHz, 2 video output. 16 MB/CCDDigitization time = 2 Mpixel * 4 µs = 8.4 sec

Guide CCDs: 4 2k2k CCDs, using 100 x 100 pixel ROI readout @ 1 MHz, 2 video outputs. 20 kB/CCDDigitization time = 104 pixel * 1 µs = 10 msec (+ ROI overhead)

Science/Focus data rate to PAN: (on dedicated fibers)From 3-ACQ DHE: 34.3 MBps (18 CCDs) 3 PANs control 54 science CCDs.From 2-ACQ DHE: 34.3 MBps (8 CCDs) 2 PANs control 8 science and 8 focus CCDs.

Science/Focus data rate out of PAN (to meet 100 sec cadence):3-ACQ PAN: 5.76 MBps2-ACQ PAN: 1.92 MBps

Science/Focus data to IB: 3*5.76 + 2*1.92 MBps = 21.1 MBps. Perhaps /4, depending on SW protocol (push vs pull).Focus data to Focus process: 1.28 MBps

Guide data rate to PAN: 8 MBps (on dedicated fibers)Guide data to Guide process: 0.8 MBps (to send in 0.1 sec)Guide data to IB: 0.08 MBps (if we keep all 100 guide images per 100 sec science image)

Other data sources/sinks:Instrumentation: 100 kB per image to IB

1 MB to GUI (on observer demand)

Database: 100 kB per image to IB1 MB to GUI (on observer demand)

GUI: Receives complete images (2 GB, on observer demand. Instantaneous data rate depends on the desired response time.Other information requires a small data rate. The main GUI issue is the complexity of the human/computer interface.

NOTE: I assume that QA is done by the same computer as IB, and that 3-day data storage is local to each computer. This minimizes network traffic.

Rate calculationNo data compression


Commands and messagesreceived by OCS

Commands andmessagessent by OCS

Take ascienceexposure

StatusData

Buildimage

time

Imageready

Send status

Slew telescope,Change filter,Adjust focusPrepare FCS,Prepare GCS,Prepare Guide

Telescope tracking,Filter ready,Focus adjusted,FCS ready,GCS ready,Guide ready

Prepare for exposure

Shutteris closed

Shutteris open

Openshutter

GuidingNot

guiding

Startguiding

Exposure

OCS Command Sequence

We must specify similar sequences for every situation.

Exposure durationis controlled locally

at the shutter.

ScienceExposure


Division of Labor

This assignment of responsibilities is not yet final.Names are of some who are helping with the design.

Alarms Fermilab, OSU Merritt, HonscheidDB Fermilab, OSU Merritt, Honscheid, Buckley-Geer FCS UIUC Karliner, ThalerFocus Chicago Gladders, AbbottGCS UIUC Karliner, ThalerGUI Fermilab, OSU Merritt, Honscheid, Thaler Guide Chicago, Barcelona CastanderIB Fermilab, UIUC Mohr, StoughtonICS Fermilab, UIUC Alspach, HaneyObsTAC Fermilab AnnisOCS NOAO AbbotQA Fermilab, NOAO, UIUCLin, Rest, MohrSkyCam Fermilab Tucker


• Simulate the response of the front end electronics.. Exercise the HW & SW before CCD’s are available.. Evaluate proposed upgrade or other focal plane modifications.

• Implement with field programmable gate arrays (FPGA’s).. Each FPS element will behave like a CCD.. Accept clock signals.. Generate “fake analog” Vout with programmable patterns.

• Cost to simulate the whole focal plane ~ $7k. . A prototype will exist this summer.

It is important to be able to test the partial system.

Focal Plane Simulator (FPS)

jon thaler fermilab director’s review 7/26/20061 sispi s urvey i mage s ystem p rocess i...

Documents

control slide

ocs ready

complete ocs

software complete

smooth data flow

data flow test1007

image builder ib slide

thought observation