Jeffrey O. Hill

LANSCE / LANL

Requirements, a ReviewDesign, a ReviewDesign ChangesStatus, this project and othersLessons LearnedBenefits, a Review

LANSCE timing and flavoring of dataFlavoring

Selection based on - logical combinatorial of beam gatesTiming

Selection based on - time sampling within beam pulseMany permutations

Too many to, a-priori, install records for all of themClient side tools specify flavoring when subscribingCapabilities to forward timing and beam gate

companion data in IOC coreFiltering capabilities in the CA server

LANSCE timed data requires Array Index MetadataMagnitude of zero-eth element index

Floating pointMagnitude of one index increment

Floating pointUnits of these magnitudes

String

CA ServerCA ServerRecordRecord

Event Queue

Legacy Fixed Event

Parameter Set

Scalar ValueTime StampAlarm Status

Legacy Fixed Event

Parameter Set

Scalar ValueTime StampAlarm Status

Upgraded Record Specific Event Parameter Set

Scalar or Vector Value

Time StampAlarm Status

…

Upgraded Record Specific Event Parameter Set

Scalar or Vector Value

Time StampAlarm Status

…

Upgraded Device Specific Event Parameter Set

LANSCE Beam Gate Specification

Upgraded Device Specific Event Parameter Set

LANSCE Beam Gate Specification

RecordRecord

Event Queue

Event Filter

Discard

Event Filter CALC ExpressionEX1beamGateSpec & ( beamGateA | beamGateB ) && ! (beamGateSpec & beamGateC )

EX2Value > 100

Event Filter CALC ExpressionEX1beamGateSpec & ( beamGateA | beamGateB ) && ! (beamGateSpec & beamGateC )

EX2Value > 100

TCP Circuit

Before – Guard based

Ptr < Service > pService = createService ();Ptr < Channel > pChannel = pService->createChannel (…);

After – smart pointer based

Mutex mutex;Guard guard (mutex );Service & service = createService (guard);Channel & channel = Service.createChannel (guard,…);

For event queue implementation and public CA interfaces we have some special requirements – for flexibility and isolationData Access Catalog, CA Channel, and others are

used by different threads than their creatorsRuntime polymorphism – requires virtual interface

User of Ptr must not know at runtime or compile time which of these are chosenReference Counted memory management, or notMutual Exclusion protection for target, or notIntrusive (i.e. within target object) implementation, or not

After some deliberation a custom implementation appeared to be the best choiceWe need mutual exclusion which boost library’s shared_ptr does not provide

The loki library’s smart pointer extensive use of templates is probably not portable enough for EPICS. Legacy vxWorks systems use an old gnu compiler.

A custom implementation currently requires 547 lines of code – not a significant maintenance burden

Internally a smart pointer has a single data member Pointer to a handle

Smart pointer same size as ordinary pointerA handle is runtime polymorphicThe operator-> is overloaded to return

locked smart pointer object

class HandleIntf { virtual TargetMutexPair targetMutexPair () = 0; virtual HandleIntf & clone () = 0; virtual void release () throw () = 0;};

Efficient atomic increment / decrement needed for reference counting RISC architecture requires mutual exclusion

Increment requires two (interruptible) instructions SMP architecture requires memory barriers

Cache sync / flush operations SMP, the future CPU architecture

Atomic increment performance - versus a mutex On typical computer (Intel core2duo 3GHz) 70 μS epicsMutex lock/unlock 23 μS osiAtomicIncr

Increment / Decrement Implementations (so far) GCC 4.1 intrinsic primitives on OS/GNU OS primitives on Solaris, Windows, vxWorks OS, predating SMP, lock interrupts very briefly

Legacy vxWorks, RTEMS, …

MagViz distinguishes potential-threat liquids from harmless shampoos and sodas screened at airport baggage checkpoints

Many quality improvement related changes in the portable server libraryFixed lockup when flow control started and channel

connect requests were pending on the event queueThis may be the cause of DB CA links not reconnecting

through the gateway at Diamond and SLSIncoming (put) data corruption under heavy CAS load.

2nd occurrence foundFaster string conversions - delays reduced, large

arraysScheduling improvements, large arrays, many others

…The portable server library is used by the CA

gateway, and application level services

CA Gateway was modified to behave differently for ca put versus ca put callbackCA put

Caching behavior when client put production rate exceeds put consumption rate of IOC

Classic MEDM slider over-pumps the slow motor record situation – is handled well now

CA put callbackThe original behavior is preserved

Each and every put request is executed

I am working again on this project since the start of the calendar yearSome interruptions this summer due to EPICS

R3.14.11, CA gateway changesGuard to smart pointer locking model

transition required extensive reorganization of the data structures, but is now close to completion, and worth the delay IMHO

The code builds, but there are currently about 40 undefined symbols

Multithreaded and Object OrientedDesigning for both together takes time to masterChanging the primary interface’s locking model

during the middle of the project can be like pulling on a sweater thread.

When designing any modern code SMP must be consideredSome additional locking overhead is justified when

we consider that N processors can work concurrently

Nevertheless, I do worry about the cost of blocking processors while they reload their pipeline due to memory barriers (lesson is pending)

Working on multiple project simultaneously Positive impacts

Returning to a project leads to a fresh object and a desire to review core design decisions

Negative impactsToo much context switchingLost momentum

Breaking large projects up into smaller upgrades is a good ideaIncremental upgrades occur more quicklyWith more frequent releases means better

feedbackMore authors can share the development load

LANSCE style dynamic on-the-fly ad-hoc beam flavoring and beam timing experiments But, in homogenous EPICS system

Tool based approach to LANSCE applicationsApplications have abstract model of hardwareIncremental upgrades hopefully easierHomogeneous communication model for high

level apps might make on-call duties easierMulti-element “Timed” data

COTS digitizerWindow in time selected

Flexible event snapshotsParameters other than alarm status, time

stamp, scalar value correlated on event queueArray update bursts buffered on event queue

Subscription update filteringExpression (string) based means

Project and site independent – tool based approachMinimally invasive for existing client side tools

Array index meta dataExpanding intersection of EPICS with data

acquisition systems