Download - B est E ver A larm S ystem T oolkit
![Page 1: B est E ver A larm S ystem T oolkit](https://reader036.vdocuments.us/reader036/viewer/2022062323/56815f03550346895dcdc38a/html5/thumbnails/1.jpg)
Best Ever Alarm System Toolkit
Kay Kasemir,
Xihui Chen,
Katia Danilova
SNS/ORNL
April, 2013
![Page 2: B est E ver A larm S ystem T oolkit](https://reader036.vdocuments.us/reader036/viewer/2022062323/56815f03550346895dcdc38a/html5/thumbnails/2.jpg)
2 Managed by UT-Battellefor the U.S. Department of Energy
What is BEAST?
Control SystemAlarm Server
Cool UI Configuration
• An alarm system that monitors Process Variables (PVs) in a Control System
• Effectively help operators take the correct action at the correct time.
![Page 3: B est E ver A larm S ystem T oolkit](https://reader036.vdocuments.us/reader036/viewer/2022062323/56815f03550346895dcdc38a/html5/thumbnails/3.jpg)
3 Managed by UT-Battellefor the U.S. Department of Energy
IOCs
Architecture
Alarm Cfg & StateRDB
FECs/IOCs
Alarm ServerCurrent Alarms: Latched? Annunciated? Acknowledged?
LOG
MessageRDB
JMSto
Speech
JMSto
RDB
Tomcat- Reports
Alarm Client GUI
JMS
Alarm Updates Ack’; Config UpdatesAnnunciationsLog Messages
TALK ALARM_CLIENTALARM_SERVER
PV Updates (Channel Access, …)
![Page 4: B est E ver A larm S ystem T oolkit](https://reader036.vdocuments.us/reader036/viewer/2022062323/56815f03550346895dcdc38a/html5/thumbnails/4.jpg)
4 Managed by UT-Battellefor the U.S. Department of Energy
Levels Of Complexity• Use the Alarm System
– Control Room operator
• Configure the Alarm System– Certain operators, IOC engineers
• Alarm System Setup– CSS maintainer for site
• Coming up with a good configuration– Everybody
Easy
Hard
![Page 5: B est E ver A larm S ystem T oolkit](https://reader036.vdocuments.us/reader036/viewer/2022062323/56815f03550346895dcdc38a/html5/thumbnails/5.jpg)
5 Managed by UT-Battellefor the U.S. Department of Energy
Client GUI: Alarm Area Panel• An overview of all alarm areas
Acknowledge the whole area
![Page 6: B est E ver A larm S ystem T oolkit](https://reader036.vdocuments.us/reader036/viewer/2022062323/56815f03550346895dcdc38a/html5/thumbnails/6.jpg)
6 Managed by UT-Battellefor the U.S. Department of Energy
Client GUI: Alarm Table
All currentalarms– active– ack’ed
Optional:Voice Annunciation
Sort by column
Acknowledge one or multiple alarms– Select by PV or description– BNL/RHIC type un-ack’
![Page 7: B est E ver A larm S ystem T oolkit](https://reader036.vdocuments.us/reader036/viewer/2022062323/56815f03550346895dcdc38a/html5/thumbnails/7.jpg)
7 Managed by UT-Battellefor the U.S. Department of Energy
Client GUI: Alarm Tree
See complete configuration– Active, ack’ed, inactive, disabled
Hierarchical– Optionally only show
active alarms– Ack’/Un-ack’ PVs or sub-tree
![Page 8: B est E ver A larm S ystem T oolkit](https://reader036.vdocuments.us/reader036/viewer/2022062323/56815f03550346895dcdc38a/html5/thumbnails/8.jpg)
8 Managed by UT-Battellefor the U.S. Department of Energy
Guidance, Related Displays, Commands
View Guidance Texts Start related displays (EDM) Open Web pages Run ext. commands
Hierarchical:Including info of parent entriesMerges Guidance etc. from all selected alarms
![Page 9: B est E ver A larm S ystem T oolkit](https://reader036.vdocuments.us/reader036/viewer/2022062323/56815f03550346895dcdc38a/html5/thumbnails/9.jpg)
9 Managed by UT-Battellefor the U.S. Department of Energy
CSS Integration: Alarm Data Browser1. Context menu: Alarm Duration, Guidance, Displays, …
2. Select Data Browser for PV in alarm
3. View history, annotate
3
2
1
![Page 10: B est E ver A larm S ystem T oolkit](https://reader036.vdocuments.us/reader036/viewer/2022062323/56815f03550346895dcdc38a/html5/thumbnails/10.jpg)
10 Managed by UT-Battellefor the U.S. Department of Energy
Data Browser Electronic LogbookAfter inspecting alarm PV’s history, post commented plot to E-Log
![Page 11: B est E ver A larm S ystem T oolkit](https://reader036.vdocuments.us/reader036/viewer/2022062323/56815f03550346895dcdc38a/html5/thumbnails/11.jpg)
11 Managed by UT-Battellefor the U.S. Department of Energy
Directly from Alarm to E-Log
• “Logbook”from context menucreates text w/basic info aboutselected alarms.Edit, submit.
![Page 12: B est E ver A larm S ystem T oolkit](https://reader036.vdocuments.us/reader036/viewer/2022062323/56815f03550346895dcdc38a/html5/thumbnails/12.jpg)
12 Managed by UT-Battellefor the U.S. Department of Energy
.. may require Authentication/Authorization (LDAP)Log in/out
Online Configuration Changes
![Page 13: B est E ver A larm S ystem T oolkit](https://reader036.vdocuments.us/reader036/viewer/2022062323/56815f03550346895dcdc38a/html5/thumbnails/13.jpg)
13 Managed by UT-Battellefor the U.S. Department of Energy
Configure PV
formula-based alarm enablement
Latch highest severity, require acknowledgement
Chatter filter
![Page 14: B est E ver A larm S ystem T oolkit](https://reader036.vdocuments.us/reader036/viewer/2022062323/56815f03550346895dcdc38a/html5/thumbnails/14.jpg)
14 Managed by UT-Battellefor the U.S. Department of Energy
Logging
• ..into generic CSS log also used for error/warn/info/debug messages
• Alarm Server: State transitions, Annunciations• Alarm GUI: Ack/Un-Ack requests, Config changes• Generic Message History Viewer
– Example w/ Filter on TEXT=CONFIG
![Page 15: B est E ver A larm S ystem T oolkit](https://reader036.vdocuments.us/reader036/viewer/2022062323/56815f03550346895dcdc38a/html5/thumbnails/15.jpg)
15 Managed by UT-Battellefor the U.S. Department of Energy
Logging: Get timelineFilter on TYPE, PV
1. PV triggers,clears, triggers again
2. Alarm Server latches alarm
4. Problem fixed
3. Alarm Server annunciates
5. Ack’ed by operator
6. All OK
![Page 16: B est E ver A larm S ystem T oolkit](https://reader036.vdocuments.us/reader036/viewer/2022062323/56815f03550346895dcdc38a/html5/thumbnails/16.jpg)
16 Managed by UT-Battellefor the U.S. Department of Energy
Web Report Examples
• Examples from SNS– Code would need some rework to port to
other sites
![Page 17: B est E ver A larm S ystem T oolkit](https://reader036.vdocuments.us/reader036/viewer/2022062323/56815f03550346895dcdc38a/html5/thumbnails/17.jpg)
17 Managed by UT-Battellefor the U.S. Department of Energy
Creating a good Alarm Configuration
B. Hollifield, E. Habibi,"Alarm Management: Seven Effective Methods for Optimum Performance", ISA, 2007
Hard
![Page 18: B est E ver A larm S ystem T oolkit](https://reader036.vdocuments.us/reader036/viewer/2022062323/56815f03550346895dcdc38a/html5/thumbnails/18.jpg)
18 Managed by UT-Battellefor the U.S. Department of Energy
Alarm Philosophy
Goal:
Help operators take correct actions
– Alarms with guidance, related displays– Manageable alarm rate (<150/day)– Operators will respond to every alarm
(corollary to manageable rate)
![Page 19: B est E ver A larm S ystem T oolkit](https://reader036.vdocuments.us/reader036/viewer/2022062323/56815f03550346895dcdc38a/html5/thumbnails/19.jpg)
19 Managed by UT-Battellefor the U.S. Department of Energy
• DOES IT REQUIRE IMMEDIATE OPERATOR ACTION?
– What action? Alarm guidance!• Not “make elog entry”, “tell next shift”, …
– Consequence of not reacting?
– How much time to react?
What’s a valid alarm?
![Page 20: B est E ver A larm S ystem T oolkit](https://reader036.vdocuments.us/reader036/viewer/2022062323/56815f03550346895dcdc38a/html5/thumbnails/20.jpg)
20 Managed by UT-Battellefor the U.S. Department of Energy
How are alarms added?
• Alarm triggers: PVs on IOCs– But more than just setting HIGH, HIHI, HSV, HHSV– HYST (alarm deadband) is good idea– Dynamic limits, enable based on machine state,...
Requires thought, communication, documentation
• Added to alarm server with– Guidance: How to respond– Related screen: Reason for alarm (limits, …), link to screens
mentioned in guidance– Link to rationalization info (wiki)
![Page 21: B est E ver A larm S ystem T oolkit](https://reader036.vdocuments.us/reader036/viewer/2022062323/56815f03550346895dcdc38a/html5/thumbnails/21.jpg)
21 Managed by UT-Battellefor the U.S. Department of Energy
Example: Elevated Temp/Press/Res.Err./…• Immediate action required?
– Do something to prevent interlock trip
• Impact, Consequence?– Beam off if interlock tripped
• Time to respond?– 10 minutes to prevent interlock
• MINOR? MAJOR?• Guidance: “Open Valve 47 a bit, …”• Related Displays: Screen that shows Temp, Valve, …
![Page 22: B est E ver A larm S ystem T oolkit](https://reader036.vdocuments.us/reader036/viewer/2022062323/56815f03550346895dcdc38a/html5/thumbnails/22.jpg)
22 Managed by UT-Battellefor the U.S. Department of Energy
Avoid Multiple Alarm Levels· Analog PVs for Temp/Press/Res.Err./…:
– Easy to set LOLO, LOW, HIGH, HIHI· Consider:
– Do they require significantly different operator actions?– Will there be a lot of time after the HIGH to react before a
follow-up HIHI alarm?· In most cases, HIGH & HIHI only double the alarm traffic
– Set only HSV to generate single, early alarm– Adding HHSV alarm assuming that the first one is ignored
only worsens the problem
![Page 23: B est E ver A larm S ystem T oolkit](https://reader036.vdocuments.us/reader036/viewer/2022062323/56815f03550346895dcdc38a/html5/thumbnails/23.jpg)
23 Managed by UT-Battellefor the U.S. Department of Energy
Alarms for Redundant Pumps
![Page 24: B est E ver A larm S ystem T oolkit](https://reader036.vdocuments.us/reader036/viewer/2022062323/56815f03550346895dcdc38a/html5/thumbnails/24.jpg)
24 Managed by UT-Battellefor the U.S. Department of Energy
Alarm Generation: Redundant Pumps the wrong way
• Control System– Pump1 on/off status– Pump2 on/off status
• Simple Config setting: Pump Off => Alarm:– It’s normal for the ‘backup’ to be off– Both running is usually bad as well
• Except during tests or switchover– During maintenance, both can be off
![Page 25: B est E ver A larm S ystem T oolkit](https://reader036.vdocuments.us/reader036/viewer/2022062323/56815f03550346895dcdc38a/html5/thumbnails/25.jpg)
25 Managed by UT-Battellefor the U.S. Department of Energy
Redundant Pumps
• Control System– Pump1 on/off status– Pump2 on/off status– Number of running pumps– Configurable number of desired pumps
• Alarm System: Running == Desired?– … with delay to handle tests, switchover
• Same applies to devices that are only needed on-demand
1Required Pumps:
![Page 26: B est E ver A larm S ystem T oolkit](https://reader036.vdocuments.us/reader036/viewer/2022062323/56815f03550346895dcdc38a/html5/thumbnails/26.jpg)
26 Managed by UT-Battellefor the U.S. Department of Energy
Weekly Review: How Many? Top 10?
![Page 27: B est E ver A larm S ystem T oolkit](https://reader036.vdocuments.us/reader036/viewer/2022062323/56815f03550346895dcdc38a/html5/thumbnails/27.jpg)
27 Managed by UT-Battellefor the U.S. Department of Energy
Summary• Easy to use
– Check alarms in Table, Tree, Panel– Fix it: Read Guidance, use Display Links– ✔Acknowledge
• Configuration– Can be changed on the fly– Operators can update guidance or add better
display links
• Alarm System Setup– Somewhat Involved, but only once
• Coming up with a good configuration– Hard
Easy
Hard
![Page 28: B est E ver A larm S ystem T oolkit](https://reader036.vdocuments.us/reader036/viewer/2022062323/56815f03550346895dcdc38a/html5/thumbnails/28.jpg)
28 Managed by UT-Battellefor the U.S. Department of Energy
Thank You!• BEAST Home Page:
– http://sourceforge.net/apps/trac/cs-studio/wiki/BEAST
See also:B. Hollifield, E. Habibi, "Alarm Management: Seven Effective Methods
for Optimum Performance", ISA, 2007
Alarm Rationalization: Practical Experience Rationalizing Alarm Configuration for an Accelerator SubSystem, Xiaosong Geng, etc,. http://info.ornl.gov/sites/publications/files/Pub22522.PDF