channel archiver overview jan. 2003. channel archiver channel access client stores samples in disk...
DESCRIPTION
Operation Config. file lists channels to archive # Example: channel_X 1 channel_Y 10 # Monitor: might change every 0.2 second channel_Z 0.2 Monitor Scanned Periodically store most recent value Monitor Store all incoming values – up to buffer limit Original time stamps are stored!TRANSCRIPT
Channel ArchiverOverview
Jan. 2003
Channel Archiver Channel Access client Stores samples in disk files Design target: handle 10000 values/sec
Documentation, Snapshots:LANL EPICS Page.From http://www.aps.anl.gov/epics,see Other Sites, LANL, Tools, Archiver.
Operation Config. file lists channels to archive
# Example: <channel> <period [s]>channel_X 1channel_Y 10# Monitor: might change every 0.2 secondchannel_Z 0.2 Monitor
Scanned Periodically store most recent value
Monitor Store all incoming values – up to buffer limit
Original time stamps are stored!
1s Period Data @ 9.96, 10.98, 11.76 s ?
Components
CA
DataDirectory
Storage Retrieval
LibIO (C++)
ArchiveEngine
LibIO (C++)
WinBrowser(Win32)
LibIO (C++)
xarr(Unix)
LibIO (C++)
StripTool(Unix)
LibIO (C++)CGIExport
WebServer
Native:
LibIO (C++)SWIG
TCL tk,BLT
LibIO (C++)SWIG
Pythontk,BLT,PMW
LibIO (C++)SWIG
PerlCGI,DBI,DBDScripting:
Online: (also possible via scripting)
LibIO (C++)
Archive-Export
LibIO (C++)
Archive-Manager
ArchiveEngine Syntax:
ArchiveEngine [options] config [dir]
Options:-d <desciption>-p <port> : change HTTPD from 4812-l <log> : write log file
Engine’s HTTPD URL of engine’s HTTPD: http://<machine>:<port> Status & Config., not data! “Client Pull”: Updates on reload Changes (added groups/channels) written to cfg
subdirectory, original config. files unchanged
“Directory File” File name, not a directory name Specified when running archive engine
Default: “freq_directory”. Better: “dir”. Used by retrieval tools Details of “directory” and “data” files
are given in documentation Note: Keep directory & data files
together, don’t modify them.
ArchiveExport Syntax:
ArchiveExport [options] [dir] {channel names}
Options:-start “mm/dd/yyyy hh:mm:ss”-end “mm/dd/yyyy hh:mm:ss”-fill-interpolate <seconds>-gnuplot-Matlab…
Export Options Original Time Stamps:
ill-suited for Spreadsheets “Fill” missing values
by repetition Linear Interpolation
for given period
(initial #N/A until all channels have valid value)
Time A B3/22/00 17:02:28.700 0.071824 #N/A3/22/00 17:02:28.701 #N/A -0.0860063/22/00 17:02:37.401 0.054358 #N/A3/22/00 17:02:37.511 #N/A -0.111776
...
Time A B3/22/00 17:02:28.700 0.071824 #N/A3/22/00 17:02:28.701 0.071824 -0.086013/22/00 17:02:37.401 0.054358 -0.086013/22/00 17:02:37.511 0.054358 -0.111783/22/00 17:02:39.411 0.139948 -0.11178...
Time A B3/22/00 17:02:28.700 0.071824 #N/A3/22/00 17:02:28.701 #N/A -0.086013/22/00 17:02:30.000 0.069216 -0.089813/22/00 17:02:32.000 0.065201 -0.095663/22/00 17:02:34.000 0.061186 -0.101513/22/00 17:02:36.000 0.057171 -0.10736…
“Fill”, “Interpol.”, ...
6.5
7
7.5
8
8.5
9
0 0.0002 0.0004 0.0006 0.0008 0.001 0.0012 0.0014
Time [s]
Val
ue [a
.u.]
Original Samples"filled"Linear Interpol.
MATLAB Export
Toolsgenerate MATLAB command file ASCII, portable No MATLAB
binaries required Full value info,
time & status Big & slow
WinBrowser Win. Only
CGI Export
Data Access from any Web browser
Download formats: Spreadsheet, Matlab
Data
Web ServerCGIExport
WWW
CGI Export Online Plots
Scripting Allows writing
perl/tcl/python programs for Automated generation
of daily beam statistics for Web page
Specialized scripts to answer questions like:“How often was XX below 10.0 and for how long?”
…
ArchiveEngine Details More options, see manual:
!write_period <seconds> “Groups” for
getting organized conditional archiving!group <other file><channel> <period> Disable
Retrieval f. Multiple Archives Run several ArchiveEngines, creating
several “small enough” data sets Look at them all at once: master_version=1
# List sub-archives, most recent first/archives/2001/july/dir/archives/2001/june/dir/archives/2001/may/dir# …# Then check Fred's "xyz" archive /home/fred/xyzarchive/dir
Sub-archives should not “overlap”
Suggested Setup Long Term Archive
Sample ~10min, kept forever Middle Term
Sample every 60s, restarted after 15 days, switching between two directories
Short TermUp to ‘Monitor’, toggling two directories every shift
High-grained data for last shift, good data for last 2 weeks, some log of everything kept forever
CAManager (Thomas Birke, BESSY) ArchiveEngine control GUI
Start/Stop
Status of (multiple) ArchiveEngines
Periodic re-start in different directory,
updating a MultiArchive File
Data Management Do not separate Directory & Data
files! Periodically restart Engine in new
directory to keep data sets manageable (CAManager helps)
ArchiveManager program can copy selected channels & time range, performing basic repairs and compression while doing this
Summary “ChannelArchiver” is a Toolset for archiving
any ChannelAccess data Some generic retrieval options, scripting
and Matlab allow further analysis Future Plans:
Port to R3.14 and RH8 CORBA interface for data retrieval
(common w/ JLab archiver, replace tcl/perl/python binding, allow Java access to data)