diamon/laser design proposal marek misiowiec be/co/ap may 2010
TRANSCRIPT
Diamon/Laser Design proposal
Marek Misiowiec BE/CO/AP
May 2010
Core OverviewMulti-tier, distributed, fail-safe system processing diagnostic data
collected repeatedly from a variety of sources. Raw data of
different shapes is transformed into unified formats and further
analyzed on a middle layer according to well-defined rules.
Actions may be taken, in form of alarms or monitoring information
delivered to clients. Processed data is made available as JAPC
parameters or for dedicated GUIs. It is stored in long term archive
too. Downward communication with the source devices is possible.2
devs
apps DAQ
raw data
unifieddata
SRV
clients
client view
Core Overview Diamon/Laser core responsibilities:
• data acquisition for various sources – push & pull modes
• communication through layers, publishing
• data transformation, processing, analysis, conditioning
• failover, database separation, scaling
complementary functionalities:• rules engine
• logging, archiving, configuration updates
non-core:• definitions & rules, data provider’s workflow
• GUIs, dedicated client applications
3
Importance of a change
Nowadays: upgrades mess
• stale sources, legacy constraints, dusty components
meeting only part of the requirements• partial data, simplistic thresholds, no conditioning
monitoring based on alarms - Diamon on Laser• misuse of the system, overhead
• mediocre scaling
• badly-behaving sources/clients cannot be detached/isolated
legacy solutions hiding in Laser world• SonicMQ, OC4J, EJBs, Hibernate, Sleepycat
...and no standard CO components in place
4
Importance of a change
We would like to: widely use CO solutions and help to establish others worth it better drive interactions with the bottom and upper tiers consolidate logics in system core be able to draw more conclusions on collected data sustain upgrades, updates, dependency changes
5
Transition
6
JMSNotifier
oc4j-diamon
AlarmArchive
Agent Messages
Checks surveillance
Alarms, filters non registered
Alarms
Result Messages
Sends update/Requests config
CLIC Agents
Central Agents
CMWJMXWIE TWIDO SNMP
PING
jdaemon
Configuration
Agentsto be
configured
configures
commands
commands
Router
CCC Consoles
LASER
ALARMS
YAMI protocol
JMS
Transition
7
JMSNotifier
oc4j-diamon
AlarmArchive
Agent Messages
Checks surveillance
Alarms, filters non registered
Alarms
Result Messages
Sends update/Requests config
CLIC Agents
Central Agents
CMWJMXWIE TWIDO SNMP
PING
jdaemon
Configuration
Agentsto be
configured
configures
commands
commands
Router
CCC Consoles
LASER
ALARMS
YAMI protocol
JMS
OC4J, EJBs, Hibernate,
BerkeleyDB
OC4J, EJBs, Hibernate,
BerkeleyDB
secondSonicMQ
firstSonicMQ
In-house DB
In-house DB
SNMP, CMW, variety of data
formats
directly to devices
not enough data
YAMI specific protocol
Transition
8
JMS
spring AlarmArchive
Agent Messages
Result Messages
Sends update/Requests config
CLIC Agents
Central Agents
CMWJMXWIE TWIDO SNMP
PING
jdaemon
Configuration
Agentsto be
configured
configures
commands
commands
CCC Consoles
YAMI protocol
JMS
LASERPROXY
LASERCMW
Transition
9
spring
AlarmArchive
Result MessagesSends update/
Requests config
CLIC Agents
Central Agents
CMWJMXWIE TWIDO SNMP
PING
jdaemon
Configuration
configures
commands
CCC Consoles
YAMI protocol
JMS
LASERPROXY
LASERCMW
Transition
10
spring
AlarmArchive
Result Messages
Data Acquisition Layer
CLIC AGENTS
CMWJMXWIE TWIDO SNMP
PINGConfiguration
commands
CCC Consoles
JMS
LASERPROXY
LASERCMW
Rules, thresholds, distribution, analysis, publishing...
JAPC
other applications
Technical outcomeThen: unified view of complete data sound construction on CO materials
• OC4J -> Spring, Hibernate -> JDBC Templates, EJBs -> POJOsSonicMQ -> ActiveMQ, multitude of data formats -> JAPCin-house database -> CCDB + Logging
• wider choice of middleware: CMW, YAMI, JMS
focusing logics in one place• rules, thresholds, alarm generation
• conditioning: machine mode, working hours, state of subsystems
driving the whole process • detachable sources, clients
• less restart/reboot downtime
• convergence with similar efforts
11
information exchange
12
japc-ext-yami japc-ext-cmw japc-ext-snmp jms-to-japc proxy japc-ext-?
Core Services
rules thresholds
alarm generator
externalsconditions
publisher w-flow
consoles
client middleware: JMSclientinter
actions
other apps
JAPCprovider
ConfigDB
archiverArchive
DB
Core
DAQ [1] DAQ [2] DAQ [n]
tackleproblematic
sources
Clicagents
CMWsrcs
LaserWie, Twi
?other
sources
source middleware protocols
Available technologies
Building blocks: CO standard components eg. JAPC CO less standard, but demanded eg. japc-ext-yami Laser dev code base, ready to reuse eg. caching, distribution TIM being refurbished and struggling with similar future other CO projects: INCA
13
Lessons learned from Laser eradication of legacy concepts & technologies pays off to scale well, we need to distribute processing power
• a supervisor over multiple workers
• dividing is difficult
separation of concerns single processing unit as a set of loosely coupled actions
• patterns: chain of responsibilities, observable
• upward flow: source input -> processing -> publishing
cater for database outages through local caching JMS+RMI as a middleware for all parties involved
• more advanced features of JMS channel help
...but no JAPC concepts were introduced at all
14
Device/property model to clarify the current incoherent view strong incentives:
• speak one language
• let everyone fit (Logging)
• common tools, well-established in CO world
migration of legacy sources• expose properties
• eradicate laser-source as it is
interaction with devices through JAPC calls domain entities:
• Alarm, Alarms, Diagnostic,...
15
JAPC extensions to unify view on a variety of devices/applications
available now:• CMW (japc-ext-cmwrda) and JMS (remote) as most popular
we need some more:• YAMI through japc-ext-yami is already available,
used in new Clic agent
• SNMP through japc-ext-snmp under development
16
Bottom tier
Experience gained in: current Diamon and Laser
• eg. CMW Alarm Monitor as a huge data acquisition process
LHC Concentrators , similar problems INCA data acqusition layer
Progress:
17
source type current planned/done
Clic Agents JMS, YAMI japc-ext-yami
CMW Alarm Monitor Laser gateway JAPC device
Laser sources C/Java laser-source Laser-to-JAPC proxy
Wiener, Twido SNMP central agents japc-ext-snmp
Ping central agent JAPC device
Already developed JAPC extensions current version of YAMI meets our requirements Diamon Clic agent
• fully YAMI-based, JAPC values
CMW Alarm Monitor refurbished• 6000 subscriptions to GM and FESA devices
• will be incorporated into new core
proxy between Laser production alarms and JAPC values• for the legacy sources
18
Open questions
Still to think over: best components and collaborations to choose level of distribution and scaling shared memory, distributed cache storage: short-term, long-term, Logging DB rules handling: homemade, proprietary ...and many others
19