monitoring ems infrastructure ann moore san diego gas & electric september 13, 2004 ems users...
TRANSCRIPT
![Page 1: Monitoring EMS Infrastructure Ann Moore San Diego Gas & Electric September 13, 2004 EMS Users Group Meeting-St. Louis](https://reader035.vdocuments.us/reader035/viewer/2022062422/56649eb75503460f94bc0c3e/html5/thumbnails/1.jpg)
Monitoring EMS InfrastructureAnn Moore
San Diego Gas & Electric
September 13, 2004
EMS Users Group Meeting-St. Louis
![Page 2: Monitoring EMS Infrastructure Ann Moore San Diego Gas & Electric September 13, 2004 EMS Users Group Meeting-St. Louis](https://reader035.vdocuments.us/reader035/viewer/2022062422/56649eb75503460f94bc0c3e/html5/thumbnails/2.jpg)
2
Agenda
• SDGE – Who we are and What we do• NERC Near-Term Actions• NERC Urgent Action Standard 1200• EMS Infrastructure• Critical Cyber Assets• IT Monitor• Sample Displays• What’s Next
![Page 3: Monitoring EMS Infrastructure Ann Moore San Diego Gas & Electric September 13, 2004 EMS Users Group Meeting-St. Louis](https://reader035.vdocuments.us/reader035/viewer/2022062422/56649eb75503460f94bc0c3e/html5/thumbnails/3.jpg)
3
Sempra Energy• Sempra Energy is a Fortune 500 energy services holding
company with over 12,000 employees Sempra Energy Utilities
San Diego Gas & Electric (SDG&E) Southern California Gas Company (SoCalGas)
Sempra Energy Global Enterprises Sempra Energy International Sempra Energy LNG Corp. Sempra Energy Solutions Sempra Energy Resources Sempra Energy Trading Sempra Fiber Links
![Page 4: Monitoring EMS Infrastructure Ann Moore San Diego Gas & Electric September 13, 2004 EMS Users Group Meeting-St. Louis](https://reader035.vdocuments.us/reader035/viewer/2022062422/56649eb75503460f94bc0c3e/html5/thumbnails/4.jpg)
4
SDGE & Electric T&D
• 1.3 million customers• 3 million population• Service territory includes San Diego County and
Southern Orange County • 4,150 MW area peak load (9/10/04)• 130 Transmission RTUs (69kV, 138kV, 230kV,
and 500kV) – GE XA21 EMS • 900 Distribution RTUs (12kV)
- ACS Prism DMS
![Page 5: Monitoring EMS Infrastructure Ann Moore San Diego Gas & Electric September 13, 2004 EMS Users Group Meeting-St. Louis](https://reader035.vdocuments.us/reader035/viewer/2022062422/56649eb75503460f94bc0c3e/html5/thumbnails/5.jpg)
5
NERC Near-Term ActionsTo Assure Reliable Operations
Failures of System Monitoring and Control Functions: Review and as necessary, establish a formal means to immediately notify control room personnel when SCADA or EMS functions, that are critical to reliability, have failed and when they are restored.
Establish an automated method to alert power system operators and technical support personnel when power system status indications are not current, or that alarms are not being received or annunciated.
![Page 6: Monitoring EMS Infrastructure Ann Moore San Diego Gas & Electric September 13, 2004 EMS Users Group Meeting-St. Louis](https://reader035.vdocuments.us/reader035/viewer/2022062422/56649eb75503460f94bc0c3e/html5/thumbnails/6.jpg)
6
In-House Implementation• Generating a pseudo Alarm/Event every 5
minutes08/16/04 13:00:01 PDT DCTSta A/E CHECK SUCCESS ... AE 5 Min. Success08/16/04 13:05:01 PDT DCTSta A/E CHECK SUCCESS ... AE 5 Min. Success08/16/04 13:10:01 PDT DCTSta A/E CHECK SUCCESS ... AE 5 Min. Success08/16/04 13:15:01 PDT DCTSta A/E CHECK SUCCESS ... AE 5 Min. Success
• “Check” process to check A/E logs• Sending automatic notifications• How about… Other critical processes? Other system characteristics?
![Page 7: Monitoring EMS Infrastructure Ann Moore San Diego Gas & Electric September 13, 2004 EMS Users Group Meeting-St. Louis](https://reader035.vdocuments.us/reader035/viewer/2022062422/56649eb75503460f94bc0c3e/html5/thumbnails/7.jpg)
7
NERC Cyber Security Standard
• NERC Urgent Action Standard 1200 presents standards to “monitor” and protect critical cyber assets
• At Sempra, we take compliance seriously– SDGE Self-Certification – 3/1/04– “Substantial Compliance” – 3/1/04– “Full Compliance” – 3/1/05
![Page 8: Monitoring EMS Infrastructure Ann Moore San Diego Gas & Electric September 13, 2004 EMS Users Group Meeting-St. Louis](https://reader035.vdocuments.us/reader035/viewer/2022062422/56649eb75503460f94bc0c3e/html5/thumbnails/8.jpg)
8
EMS InfrastructureEMS DMZ
EMS DMZ
Office Network
Primary Control Center
Backup Control Center
WANWAN
C
O
R
P
CAISO DMZ
CAISO DMZ
EMS SYSTEM
EMS SYSTEM
WAN
FIREWALLFIREWALL
FIREWALLFIREWALL
FIREWALLFIREWALL
EMS SYSTEM
EMS SYSTEM
CAISO DMZ
CAISO DMZ
FIREWALLFIREWALL
EMS DMZ
EMS DMZ
Office Network
C
A
I
S
O
![Page 9: Monitoring EMS Infrastructure Ann Moore San Diego Gas & Electric September 13, 2004 EMS Users Group Meeting-St. Louis](https://reader035.vdocuments.us/reader035/viewer/2022062422/56649eb75503460f94bc0c3e/html5/thumbnails/9.jpg)
9
Critical Cyber Assets• EMS nodes: 40+ UNIX boxes
–Application Hosts: IBM AIX–Oracles: IBM AIX–Front End Processors: IBM AIX–Dispatcher Workstations: SUN Solaris
• Windows Servers: 10+ servers
–PI Servers–PI OPC/Interfaces–SQL Servers–Web Servers
• Network Devices: switches, routers, and firewalls
![Page 10: Monitoring EMS Infrastructure Ann Moore San Diego Gas & Electric September 13, 2004 EMS Users Group Meeting-St. Louis](https://reader035.vdocuments.us/reader035/viewer/2022062422/56649eb75503460f94bc0c3e/html5/thumbnails/10.jpg)
10
Monitoring All• A proactive and preventive way to monitor EMS
infrastructure health to ensure the system performance and reliability
• Monitoring all EMS infrastructure for disk, file, paging, cpu, swap and memory, …etc.
• Monitoring EMS processes and applications• Establishing performance baseline standards• Avoid finger pointing• Root cause analysis and problem solving• Automatic notification via email and cell phone
![Page 11: Monitoring EMS Infrastructure Ann Moore San Diego Gas & Electric September 13, 2004 EMS Users Group Meeting-St. Louis](https://reader035.vdocuments.us/reader035/viewer/2022062422/56649eb75503460f94bc0c3e/html5/thumbnails/11.jpg)
11
IT Monitor
OSIsoftPI
OSIsoftPI
EMSEMS
DMSDMS
Non-Scada Non-Scada
DNPDNP
MeteringMetering
CyberAssets
Ping
NetFlow
TCP Response
PerfMon
SNMP
![Page 12: Monitoring EMS Infrastructure Ann Moore San Diego Gas & Electric September 13, 2004 EMS Users Group Meeting-St. Louis](https://reader035.vdocuments.us/reader035/viewer/2022062422/56649eb75503460f94bc0c3e/html5/thumbnails/12.jpg)
12
Main Display Index
![Page 13: Monitoring EMS Infrastructure Ann Moore San Diego Gas & Electric September 13, 2004 EMS Users Group Meeting-St. Louis](https://reader035.vdocuments.us/reader035/viewer/2022062422/56649eb75503460f94bc0c3e/html5/thumbnails/13.jpg)
13
EMS Node Status Overview
![Page 14: Monitoring EMS Infrastructure Ann Moore San Diego Gas & Electric September 13, 2004 EMS Users Group Meeting-St. Louis](https://reader035.vdocuments.us/reader035/viewer/2022062422/56649eb75503460f94bc0c3e/html5/thumbnails/14.jpg)
14
Dispatcher Workstation CPU
![Page 15: Monitoring EMS Infrastructure Ann Moore San Diego Gas & Electric September 13, 2004 EMS Users Group Meeting-St. Louis](https://reader035.vdocuments.us/reader035/viewer/2022062422/56649eb75503460f94bc0c3e/html5/thumbnails/15.jpg)
15
Application Host Index
![Page 16: Monitoring EMS Infrastructure Ann Moore San Diego Gas & Electric September 13, 2004 EMS Users Group Meeting-St. Louis](https://reader035.vdocuments.us/reader035/viewer/2022062422/56649eb75503460f94bc0c3e/html5/thumbnails/16.jpg)
16
Application Host 1
![Page 17: Monitoring EMS Infrastructure Ann Moore San Diego Gas & Electric September 13, 2004 EMS Users Group Meeting-St. Louis](https://reader035.vdocuments.us/reader035/viewer/2022062422/56649eb75503460f94bc0c3e/html5/thumbnails/17.jpg)
17
• Monitoring Oracle-Oracle Database MIB’s • Monitoring ipcs• Implementing SNMP Traps• SNMP agent in facility equipments• Secured SNMP traffic (encrypting SNMP?)• Utilizing Syslog interface• Integrating with Cisco Works, IDS and HP
Openview, …etc.
THANKS!!
What’s next…