pvmanager gabriele carcassi feb 18 2010. pvmanager goals simplify data collection and aggregation ...

PVManagerGabriele Carcassi

Feb 18 2010

PVManager goals

Simplify data collection and aggregation Simplify (UI) development Re-use code as much as possible

So you want to write a UI…

channel.addListener(new Listener() { public void doSomething(Event evt) { myWidget.setFoo(evt.getWhatever()); }}

TOOLKIT IS SINGLE THREADED!

So you want to write a UI…

channel.addListener(new Listener() { public void doSomething(Event evt) { Toolkit.execute(new Runnable() { myWidget.setFoo(evt.getWhatever()); } }}

LACK OF SYNCHRONIZATION!

So you want to write a UI…

channel.addListener(new Listener() { public void doSomething(Event evt) { Toolkit.execute(new Runnable() { synchronized(evt) { myWidget.setFoo(evt.getWhatever()); } } }}

NO CHANGE BY ANOTHER THREADDURING/BETWEEN NOTIFICATIONS!

So you want to write a UI…

channel.addListener(new Listener() { public void doSomething(Event evt) { Toolkit.execute(new Runnable() { synchronized(evt) { myCopy = copy(evt.getWhatever()); myWidget.setFoo(myCopy); } } }}

SHOULD NOT UPDATE UI AT THE SAME RATEAS DATA ON THE NETWORK!

LACKING DISCONNECTION ANDRECONNECTION NOTIFICATIONS!

NEED TO QUERY METADATAON CONNECTION!

AGGREAGATE EVENTS!

CACHE!

Wish one could write

PV<List<VStatistics>> pv = PVManager.read( listOf(statisticsOf(vDouble(pvNames)))) .atHz(10);

pv.addPVValueChangeListener(new PVValueChangeListener() { public void pvValueChanged() { myWidget.setFoo( pv.getValue().getMoo()); } });

UI subsystem requirements

Single threaded Notification must be done on event thread Data objects must be changed on the event

thread only (avoid inconsistencies and simplify responding to events)

Work must be offloaded to other threads as much as possible (to prevent unresponsiveness)

Refresh rate up to 50Hz (faster is useless and counter productive)

Channel Access

Multi threaded Notification done on connection

threads Data objects changed on connection

threads Rate is not limited: aggregated can

be KHz

Collect, aggregate and notify

Need to collect the data at the source rate Collector could be a queue, a cache, a timed cache Aggregate the data at the UI rate, and notify on UI thread

Notifier Collector Monitor

Transform and calculate

While we are on other thread, we compose and calculate what we need (single array for a table, FFT of a waveform, statistics, prepare synch’ed arrays, …)

Notifier Collector Monitor

CachePV FunctionPV Function

UI or not UI

Everything you can do in the UI you should be able to do at the command line

PVManager can work without a UI (notification is simply done on the timer thread)• Command line clients• Logging/archiving• Republish the data through PVData/PVAccess

PVManager is a standalone library• pvmanager.sourceforge.net

3 APIs for the price of one

Data interfaces: where to put the data• Decoupled from the other two. Needed so that the client

can focus more on what he needs and can work with both EPICS V3 and EPICS V.

• Other talk

Building blocks API: perform computation• API based on interfaces, you can add your pieces though

the goal is to have most of them already there

Expression API: define what you want to read• Internal DSL (Doman Specific Language), type safe

BUILDING BLOCKS API

User objects

Notifier Collector Monitor

CachePV FunctionPV Function

PV<T>

Defines:• void add/removePVValueChangeListener

(PVValueChangeListener listener);• String getName() ;• T getValue();• void close();• boolean isClosed();

PVValueChangeListener defines:• void pvValueChanged();

Under the hood

Notifier Collector Monitor

CachePV FunctionPV Function

Function<T>

Defines:• T getValue();• Class<T> getType()

A function object returns a value of a particular type.• Argument number and types are determined by the

constructor of an actual function.

Under the hood

Notifier Collector Monitor

CachePV FunctionPV Function

Collector<T> extends Function<List<T>>

Defines:• void collect();• List<T> getValue();

Collects a bunch of values of the same type Typically takes a Function<T> as argument

• On a collect call, will take a new value• On getValue returns a number of values

Could be a queue (values returned only once) or a cache (returns the last n values, last 30 sec of values)

Under the hood

Notifier Collector Monitor

CachePV FunctionPV Function

ValueCache<T> extends Function<T>

Defines:• void setValue(T);

You can put a value in (and read a value out)

Under the hood

Notifier Collector Monitor

CachePV FunctionPV Function

DataSource

Defines:• void connect(DataRecipe recipe);• void disconnect(DataRecipe recipe);

DataRecipe defines:• Map<Collector, Map<String, ValueCache>>

getChannelsPerCollectors()

For each collector, list of channels with their caches To update a channel:

• Lock Collector, update cache, Collector.collect()

Under the hood

Notifier Collector Monitor

CachePV FunctionPV Function

Notification

ThreadSwitch defines:• void post(Runnable run);

It allows to execute a piece of code on a target thread

Notifier defines:• Notifier(PV<T> pv, Function<T> function, ThreadSwitch

onThread);• void notifyPv();

On notifyPV() calculates the new value, switches thread and sets the pv’s value.

EXPRESSION API

PVManager

Example:• PV<List<VStatistics>> pv = PVManager.read( listOf(statisticsOf(vDouble(pvNames)))) .atHz(10);

Defines:• static <T> PVManagerExpression<T>

read(SourceRateExpression<T> pvExpression);• static <T> PVManagerExpression<T>

read(DesiredRateExpression<T> pvExpression);

PVManagerExpression<T>

Defines:• PVManagerExpression<T> from(DataSource

connectionManager) • PVManagerExpression<T> andNotify(ThreadSwitch

onThread)• PV<T> atHz(double rate)

Default DataSource and ThreadSwitch can be specified

Source/DesiredRateExpression<T>

Two different types for the two different data rates The collectors change Source to Desired:

• static <T> DesiredRateExpression<List<T>>

queueOf(SourceRateExpression<T> expression)

External libraries for either types or operations defines static constructors for expressions:• static SourceRateExpression<VDouble> vDouble(String

name)• static <T> DesiredRateExpression<List<T>>

listOf(List<DesiredRateExpression<T>> expressions)

Example (again)

PV<List<VStatistics>> pv = PVManager.read( listOf(statisticsOf(vDouble(pvNames)))) .atHz(10);

pv.addPVValueChangeListener(new PVValueChangeListener() { public void pvValueChanged() { myWidget.setFoo( pv.getValue().getMoo()); } });

List<String>

Example (again)

PV<List<VStatistics>> pv = PVManager.read( listOf(statisticsOf(vDouble(pvNames)))) .atHz(10);

pv.addPVValueChangeListener(new PVValueChangeListener() { public void pvValueChanged() { myWidget.setFoo( pv.getValue().getMoo()); } });

List<SourceRateExpression<VDouble>>

Which implies creating the caches for all channels

Example (again)

PV<List<VStatistics>> pv = PVManager.read( listOf(statisticsOf(vDouble(pvNames)))) .atHz(10);

pv.addPVValueChangeListener(new PVValueChangeListener() { public void pvValueChanged() { myWidget.setFoo( pv.getValue().getMoo()); } });

List<DesiredRateExpression<VStatistics>>

Creates all the collectors, and the functions that calculate the statistics

Example (again)

PV<List<VStatistics>> pv = PVManager.read( listOf(statisticsOf(vDouble(pvNames)))) .atHz(10);

pv.addPVValueChangeListener(new PVValueChangeListener() { public void pvValueChanged() { myWidget.setFoo( pv.getValue().getMoo()); } });

DesiredRateExpression<List<VStatistics>>

Creates the function that assembles the list

Example (again)

PV<List<VStatistics>> pv = PVManager.read( listOf(statisticsOf(vDouble(pvNames)))) .atHz(10);

pv.addPVValueChangeListener(new PVValueChangeListener() { public void pvValueChanged() { myWidget.setFoo(pv.getValue().getMoo()); } });

PV<List<VStatistics>>

Creates the Notifier, the DataRecipe, passes it to the DataSource, …

Low complexity point

The API is very expressive• You can combine operators, the syntax is very compact,

you specify things once

But it does not hide choices• You need to specify all transformations, including how to go

from source rate to desired rate

It hides the complexity of the implementation of your choices• The API actually does something!

Eclipse/CSS integration

The library is packaged in org.csstudio.utility.pvmanager• Extension point based on DataSource• The activator configures the default ThreadSwitch to use

SWT and default DataSource to use a CompositeDataSource made up of all extensions point

All other plugins can simply use PVManager.read() Other plugins for extension points

• org.csstudio.utility.pvmanager.sim for simulated data• org.csstudio.utility.pvmanager.epics for epics v3

What’s done

SimulationDataSource• Same syntax for channels as utility.pv.som

JCADataSource• Supports VDouble, VInt, VString and VEnum

Probe was ported to use PVManager (in BNL branch)

OrbitViewer prototype built on PVManager

What’s need to be done

JCADataSource• Mass connection can be optimized to scale better

Operators• Will need to understand which operators are useful, how to

define them for “maximum combination power”• Allow to easily define “custom functions”• Add other common operators

- A CALC operator?- A histogram operator?- An FFT operator?- …