3-feb-2011cs-workshop 2011, dietrich beck overview cs-framework introduction (over)view lessons...

3-Feb-2011 CS-Workshop 2011, Dietrich Beck

Overview Overview CSCS-Framework-Framework

• Introduction

• (Over)view

• Lessons Learned


Starting Point (Starting Point (2001)2001)

• Control System– Control devices (power supplies, function generators, timing, complex

experimental procedure…)– No process control– DAQ and data processing is only a minor issue– Integrate things that were not developed to work together as a system

• Ever changing set-up– The same experiment is never done twice– Reconfiguration of complete hardware (seconds!)– Integration of new hardware (next beam-time)

• Small and medium size experiments– No dedicated staff for control systems– Different experiments but similar problems to solve (configuration,

complex timing schemes, similar hardware,…)• … What about a “Framework”?


"Framework""Framework"

Wikipedia: “…a software framework is an abstraction in which common code providing generic functionality can be selectively overridden or specialized by user code, thus providing specific functionality…”

• Common Code – Standardization – re-usability of software components– maintainability of software components– same look and feel at different experiments– ease transfer of know-how

• User Code - Specialization– concentrates on experiment specific requirements– extension of framework, specializing code


Different Points of View – A Framework is...Different Points of View – A Framework is...

Developer: User:

picture pocket knife

... a tool box ... something ready to use

(sth. to solve a problem) without additional work

(= the solution!)


Definition of the Definition of the CSCS-Framework-Framework

control system = framework + add-ons

EE/KS

and contributors

experiment

bug reports, new features requested

add-ons may become part of framework

bug fixes, new features, maintenance

CycleControl

AFGTiming+DAQ HV

GUI


PHELIX PHELIX (Petawatt High-Energy Laser for Heavy Ion Experiments)(Petawatt High-Energy Laser for Heavy Ion Experiments)

UNILAC

Z6

SIS

PHELIX

HHT

Injection box

Mai

n A

mp

lifi

erS

enso

rFaraday

Isolator

Switch Yard

Compressor

Target Chamber

fs Front End

Fiber ns Front End

Pre-amplifier2 x 19 mm heads1 x 45 mm head

X-ray Lab(low energy)

TW compressor

Target chamber

Z6 experimental

area

Heavy ions

Laser Bay

Double-pass 31.5 cm amplifier

70 m

(slide by S. Götte, GSI)

Up to 3 MJ electrical energy converted to 1kJ light energy or 0.5 Petewatt laser power. 2300 documented “shots” (Dec. 2010). About 10k process variables distributed on 15+ nodes in 2 buildings


Quad-Triple Trap Mass Spectrometer ISOLTRAPQuad-Triple Trap Mass Spectrometer ISOLTRAPHigh-precision mass determination of unstable nuclei with a Penning trap mass spectrometer at ISOLDE/CERN High-precision mass determination of unstable nuclei with a Penning trap mass spectrometer at ISOLDE/CERN

www.cern.ch/isoltrap

– Nuclear-, Astro-, Fundamental PhysicsNuclear-, Astro-, Fundamental Physics

m/m m/m 10 10-8 -8 (typically!)(typically!)

– via cyclotron frequency of stored ionsvia cyclotron frequency of stored ions

– ““sub-µs ion-juggling” every secondsub-µs ion-juggling” every second


Cooking Recipe for the Cooking Recipe for the CSCS Framework Framework

• One development tool LabVIEW

• Standardization object oriented approach, base classes

• Distribution to many nodes DIM (www.cern.ch/dim) – Event driven communication for everything

– Scaling to large systems by distribution

– Remote access

– …

• SCADA functionality (alarming, trending, …) LabVIEW DSC module

http://www.cern.ch/dim


Event Driven CommunicationEvent Driven Communication

• DIM (www.cern.ch/dim) – Peer-to-peer connection, no intrinsic bottle neck– Named service as concept (a name server aids in setting up connections)– “service”: publisher-subscriber-pattern, one-to-many– “command service”: command-pattern, many-to-one

• CS objects communicate via– DIM services: publishing/subscribing to state information– DIM commands:

• triggering actions• synchronous commands (timeout, “expire date”,…)

• everybody may talk to everybody

• Events provide the interface between– device object and it’s GUI object– device object and application layer– hierarchical state machines– …

• Design Approach - First: communication layer, Second: framework• Connectivity to other programming languages and OSs

Caller Callee

Back-end (SCADA, GUI, ...)

Middleware (communication,.)

Front-end (devices, drivers, ...)

3-layers

http://www.cern.ch/dim


Object Orientation (OO) with Object Orientation (OO) with CSCS

• "BaseClass" provides basic functionality (communication layer, active threads, ...)

• "DeviceClass" adds functionality according to specs of device type "DS345".

• Instantiation: one object per device

"BaseClass"

"Device Class"

inheritance

AFG1AFG2

AFG3

• of course: classes for GUIs, Sequencer, State machines, ...

• OO implemented by CS using pure LabVIEW (no LVOOP)


Standardization ofStandardization of Services and Cmds Services and Cmds_birthday, time/date of creation

_classID, class name

_condition, object created successfully…

_systemID, name of (sub)system

_accessID, reservation mechanism...

_evtCounter, number of cmds received

status of threads… Ping, thread alive?

GetDescriptors (Introspection!!!)

Reset (override by child classes)

…

_deviceGUI, name of GUI class Initialize, init interface

_deviceID, ID string of device Close, close interface

_deviceState, OK, ERROR, … IDQuery, query device ID

… …

_nomFrequency, frequency set-value SetFrequency, sets nominal value

_actFrequency, frequency get-value GetFrequency, gets actual value

… …

CSObjTHE base class

CAEObjsend/rec. events

BaseProcessthreads for eventsand periodic action

DeviceBasebasic deviceproperties

AFGBasearbitrary function

generator…


(Over)view (Over)view CSCS-Framework-Framework

• Introduction

• (Over)view

• Lessons Learned


SourceForge: downloads, tracker (bugs, features,…)SourceForge: downloads, tracker (bugs, features,…)


SCC via SubVersion @ GSISCC via SubVersion @ GSI


Domain Management System: Domain Management System: Process Management in a Distributed EnvironmentProcess Management in a Distributed Environment


Process MonitoringProcess Monitoring


PackagingPackaging


Documentation via WikiDocumentation via Wiki


Class Documentation via UML ToolsClass Documentation via UML Tools


Status Status CSCS-Framework-Framework

• Introduction

• Over(view)

• Lessons Learned


Lessons Learned – What Users say about Lessons Learned – What Users say about CSCS(from a talk by Stefan Götte, at the NI BIG PHYSICS Round Table, Paris, 2009)

Stress Field of the Responsible Person:• The OS (XP) and/or the intranet are always unreliable,• the programming language (LabVIEW) never really fulfills the need,• the framework (CS) is only close to the requirement,• the classes of other CS collaborators are typically not usable,• there is no way to test things since there is no test system available

while the real system is always in use, • the users never define what the program has to do, but• are not pleased with what the programmer delivers, and• they misuse the system additionally.• Anyhow: The goal is an easy system where the happy user does not

realize what happens behind the scene, which works reliable and for ever (better: till the next LabVIEW version is installed).


““The OS (XP) and/or the intranet are always The OS (XP) and/or the intranet are always unreliable” (assumption for design phase)unreliable” (assumption for design phase)

Design principles of major importance• A control system must be designed to crash and• (auto-)recover from failures of all kinds

– Fileserver down– Computer down/reboot

• Installation of updates• Switched-Off/On by users• …

– Network down– Power cuts– …

• The system is stressed most, when– everything goes wrong (failures, alarms, sub-system crash…) and– all kinds of asynchronous signals fire at the same time (self-induced

DoS-Attack) and– availability is most important.


Lessons Learned – What Users say about Lessons Learned – What Users say about CSCS(from a talk by Stefan Götte, at the NI BIG PHYSICS Round Table, Paris, 2009)

Stress Field of the Responsible Person:• The OS (XP) and/or the intranet are always unreliable,• the programming language (LabVIEW) never really fulfills the need,• the framework (CS) is only close to the requirement,• the classes of other CS collaborators are typically not usable,• there is no way to test things since there is no test system available

while the real system is always in use, • the users never define what the program has to do, but• are not pleased with what the programmer delivers, and• they misuse the system additionally.• Anyhow: The goal is an easy system where the happy user does not

realize what happens behind the scene, which works reliable and for ever (better: till the next LabVIEW version is installed).


Lessons Learned - DocumentationLessons Learned - Documentation

The fundamental problem about documentation• Two lines of text are not enough, but• Two pages of docs – (almost) nobody will read that*

• Docs out of date …

Solutions• HOW-TOs, FAQs• Auto-generated documentation

– Class documentation, CS2JavaUML– Requires well documented code (again: coding conventions!)– Release notes

• Demo-system (delayed…)

* sometimes not even the first two lines…


Lessons Learned – Generic/Common ToolsLessons Learned – Generic/Common Tools

Generic tools (generic control system GUI, generic device GUIs)

• Sometimes difficult to understand ( documentation problem)

• Lot’s of work to maintain

• Don’t match the users’ needs exactly

• Either– least common denominator (missing features), or– include all requested features (too complex)

“Nobody likes them, but everybody uses them.”


Lessons Learned – Missing FeaturesLessons Learned – Missing Features

Identifying missing features sometimes

• requires telepathic skills

• results in cool solutions that are finally not used (nice to have versus really useful)– CS Access System– Sequencer– ObjectNets, PetriNets– some base classes

• …


Lessons Learned – CompatibilityLessons Learned – Compatibility

Backward Compatibility (“does my old code still work?”)

• Is maybe THE most important feature of a framework

• Often results in a impressive mix of different versions (core system v1, base classes v2, GUI v4, device classes v3,…)

• Makes it hard to correct wrong design decisions

• Requires clear public library routines or class methods, but…

• using conventions on what is public/private does not work


Lessons Learned – The “Saving-Time-Myth”Lessons Learned – The “Saving-Time-Myth”

Wikipedia: “…software frameworks … reducing overall development time” (?)

Amount of time for solving a problem• decreases dramatically, if a problem may be solved with

(generic) existing software. “configuration instead of coding”• does not change – but the solution is much better• may even increase (short-term), compared to a dedicated

solution not (!) using the framework (required: training, courses, understanding and application of conventions)

• decreases (long-term): framework maintained by others, re-usability of code, replacing hardware, coding conventions enforced, maintainability, common language, know-how transfer, …


Experiments using the Experiments using the CSCS framework... framework...

• require high flexibility

• have a large variety of hardware types

• have up to 10,000 (1M possible) process variables

• require fast ( 100 ns) timing control using dedicated hardware

PHELIXPHELIX

Motion CaveAMotion CaveASHIPTRAPSHIPTRAP

ISOLTRAPISOLTRAP

REXTRAPREXTRAP

LEBITLEBIT

GSI, GermanyGSI, Germany

Mainz, GermanyMainz, Germany

Greifswald, GermanyGreifswald, Germany

CERN, SwitzerlandCERN, Switzerland

MSU, USAMSU, USA

Lanzhou, ChinaLanzhou, China

data takingdata taking

developmentdevelopment

commissioningcommissioningFOPIFOPI

RISINGRISING

others ...others ...HITRAPHITRAP LPTLPT

TrigaTRAPTrigaTRAP

ClusterTRAPClusterTRAP

POLARIS, HIJPOLARIS, HIJ


Conclusion and OutlookConclusion and Outlook

• CS 3.21 released for LV2009.

• about 15 active applications

• 1,000,000 PVs demonstrated, an even larger number should be feasible

• 5,000 objects ( hardware devices) demonstrated, an even larger number should be feasible

• stability of a distributed CS system is better than a few hundred hours of continuous operation

• about 15 (60) hardware devices supported on SourceForge (Subversion)

• FAIR: MATS and parts of HITRAP have selected CS as control system framework.


Acknowledgements...Acknowledgements...

Holger Brand, Dietrich Beck, Mathias Richter, Alexander Schwinn, Falk Ziegler, Romain Savreux, Chabouh Yazidjian, Klaus Blaum, Dennis Neidherr, Melanie Wolf, Stefan Schwarz, Josh Savory, Stefan Götte, Maximilian Kugler, Tobias Habermann, Michael Block, Frank Herfurth, Christian Rauth, Stephen Koszudowski, Manas Mukherjee, Thomas Rechel, Martin Feldmann, IT@GSI, ……………...

3-feb-2011cs-workshop 2011, dietrich beck overview cs-framework introduction (over)view lessons...

Documents