presentation outline · 2018-04-04 · 8 presentation outline •resentation goals p • master...

1

Illinois Section AWWA Presents:Planning For Critical Data Systems –SCADA and Technology Master Planning April 26, 2012

Presented by:

William (Bill) Biehl

Cell: 913.601.0104

[email protected]

Presentation Outline

• Presentation Goals

• Master Plan Goals and Objectives

• SCADA Master Planning Drivers

• Master Plan Key Issues

• SCADA Master Planning Methodology

• Summary

2

2

Presentation Goals

�Review the purpose of Master Planning

�Review the Drivers of Technology Master Planning

�Discuss the major components of a SCADA Master

Planning

�Discuss An Approach for SCADA Master Planning

�Understand the benefits of preparing a SCADA Master

Plan

3







• Summary

4

3

Utility Master Planning

• Most Utilities develop Master Plans of one sort or

another

• Water Resources Plan

• Treatment Facilities Master Plan

• Collection Systems Master Plan

• BioSolids Master Plan

• Recycle Water Master Plan

• Master Planning looks at the “big picture”

• Developing long-term objectives and solutions

• Planning time frame can be from 5 years to 20 years.

5

Technology Master Planning

Master Planning for

Technology Systems

• Information Technology

Master Plan

• Information Technology

Strategic Plan

• SCADA Master Plan

• Integrated Technology

Plan

6

4

Why Conduct a Technology Master Plan?

Technology Master Planning aligns Technology and

Governance with the business requirements of the Utility.

Business

Strategy

Technology

Strategy

Business

Performance

Technology

Performance

7

Technology Master Planning

8

5

Utility Technology Systems

9

Simplified Overall System

10

6

SCADA vs. PCS

• SCADA System - monitors many remote sites, utilizes

radio or telephone communications systems; typically

uses Industrial Processors: PLC & RTUs

• PCS - monitors in-plant processes utilizing local area

network technology; typically uses “intelligent” PLCs,

RTUs, or DCUs

• Distinctions have blurred as technology evolves. The

term SCADA is now widely used for both in-plant

control systems and remote site monitoring

applications.

11

Components of SCADA / PCS Systems

• Instrumentation / Final Control Elements

• PLC / DCU / RTU

• Communications Systems

• Networks

• Host / Servers

• Client / Workstations

• Historical Data Collection

• Reporting

12

7

Simplified Overall System

13

• Proactive plan to:

• Define the current state

• Establish strategic vision for

automation maturity

• Prioritize requirements and

determine project

interdependencies

• Plan for the required scope,

budget, time, and resources

What is a Technology Master Plan?

14

8







• Summary

15

SCADA Master Planning Drivers

Most water agencies would be hard-pressed to maintain service levels if their SCADA system suffered a catastrophic (or even partial) failure.

• Operating Knowledge Retained in Control Strategies

• Improved Operating Efficiency

• Regulatory Compliance

• Operating Data for Planning

• Protection of People, Environment and Equipment

SCADA is Mission Critical!

16

9

Role of SCADA in Utility Operations

Critical Need Role of SCADA/Control

Meet Regulatory

Compliance

• Monitor and report regulatory compliance

• Control process within regulatory constraints

Protect Public Health

and Safety

• Early detection and warning of process upsets

• Prevent processes and equipment from

reaching dangerous levels of operation

Maintain Service

Levels

• Operational flexibility and tight control

• Staffing levels and experience

Operate Efficiently • Unattended operations

• Process optimization

Capture and Retain

Knowledge

• Historian maintains operational history

• Operational experience captured in control

strategies and narratives

Plan Capital

Improvements

• Equipment condition and performance data

• Data to calibrate the hydraulic model

17

Planning Approach Should Ensure a Total Solution

18

10

Process

• Difficulty in defining, prioritizing and justifying SCADA projects

• Lack of change management, revision control and back-up procedures

• SCADA Standards do not exist or not enforced

• New regulatory regulations or business requirements

• High costs for maintenance and support

• Inaccurate or missing documentation

• Significant expansion

• Difficulty in sharing SCADA data with other business applications

Common Drivers for SCADA Master Planning

19

People

• Pending retirement of experienced operating or maintenance staff

• Attracting new staff and assimilating cultural changes

• Need for additional training

• Staffing reductions

Technology

• System reliability issues

• Missing or inaccurate historical data

• Obsolete equipment; spare parts hard to get

• Software vendor no longer supports product

• System doesn’t support regulatory and operational reporting

• Accessing meaningful historical data requires a SCADA specialist

• Frequent nuisance alarms

• Minor system/component upgrades cause failures in other components

• Inadequate bandwidth/throughput

• Need for expanded remote access

• Concerns about cyber security

Common Drivers for SCADA Master Planning

20

What are your

challenges?

11

Aligns Technology with Business Need

A proactive plan aligns SCADA technology and

governance with the business requirements of the Utility.

Business

Strategy

Technology

Strategy

Business

Performance

Technology

Performance

21

Bringing Order to Chaos

A logically sequenced, SCADA program meets business

needs within the resource constraints of the Utility.

22

12

Build Stakeholder Consensus

23







• Summary

24

13

What is the Utility’s Vision for SCADA?

25

Automation Maturity Model ™

Technology Areas to Support the Vision

• Overall System Design

• Supervisory and Field Control

• Instrumentation

• Local Area Networks

• Remote Site Communications

• Alarm Management

• Data Management Reporting

• SCADA Security

• SCADA Governance

26

14







• Summary

27

Methodology

28

15

SCADA Planning Methodology

Project Initiation1

Current State2

Requirements3

System Assessment4

Alternative Analysis5 SCADA Master Plan6

29

Key Activities Deliverables

• Kickoff Meeting • Project Charter

• Establish Core Planning Team • Project Plan

• Establish Executive Steering Committee • Project Website

• Review Project Plan & Schedule • Document Templates

• Review Project Deliverables


Project Initiation1

Current State2

Requirements3

System Assessment4


30

16

Project Initiation – Collaborative Approach

31

Project Initiation - Project Core Team

� Core Team:

� Small Group (5 to 6)

� Represents interest of Utility staff

� Integral to Utility/Consultant project team

� Attends all Meetings

� Internal Advocates for Utility

� Executive Steering Committee:

� Utility Management (Vice Presidents / Directors / Assistant Directors)

� Provide Oversight and Direction to Master Planning Process

� Ensure Overall Goals of Plan are aligned with Strategic Business Plan

of Organization

� Committee Briefed at Key milestones

� Actively involved in Development of Strategic Vision

32

17

Project Initiation – Develop Project Charter

33

�Purpose: An overarching document guiding the overall

project.

�Focus on both the goals for the Interceptor SCADA

system and the Preliminary Design Project.

• GROUP ACTIVITY – (10-15 Minutes)

• Review and Discuss A Sample Project Charter


• Review Existing Documentation • Current State Tech Memo

• Field Survey of Representative Sites • “As-Is” SCADA Block Diagram

• Staff Interviews • Facility/Process Summary Table

• Current State TM Review Workshop • Prevalent Standards and Exceptions

• Technology and Governance


Project Initiation1

Current State2

Requirements3

System Assessment4


34

18

Current State – Staff Interviews

35

� Important for Operator and Maintenance Buy-In to the

program

� It is valuable to get the input of these key individuals in

a more personal environment outside influence from

Manager and Peers.

• GROUP ACTIVITY – (10-15 Minutes)

• Review A Sample Staff Interview / Group Workshop Questionnaire


Project Initiation1

Current State2

Requirements3

System Assessment4



• Stakeholder Workshops • SCADA Requirements Tech Memo

• Visioning Workshop •Executive Summary

• Requirements Validation Workshop • Utility Vision for SCADA

• Business Requirements

• Technical Requirements

• Governance Requirements

36

19

Requirements:Steering Committee - Strategic SCADA Vision

37

Any Town Steering Committee SCADA Strategic Vision:

The SCADA systems will provide a reliable means of monitoring and control

for the safe production of quality water and gas services for our customers.

SCADA systems will also utilize technology for the enhancement of job

satisfaction, improvement of utility operational efficiency and optimization

initiatives, reduction in system operating and maintenance costs, and the

reduction or mitigation of potential risks to the utility and public.

The ongoing SCADA enhancement process will include responsible

investment in planned capital SCADA projects that account and utilize existing

SCADA system capital and operational practices where appropriate.

System Control SCADA SystemWhere Do We Want to Be?

38

Any TownAny Town

20


• Assessment Review Workshop • System Assessment Tech Memo

• Presentation to Steering Committee • Executive Summary

• Existing System Overview

• Best Practices Benchmark

• Initial Recommendations


Project Initiation1

Current State2

Requirements3

System Assessment4


39




• Instrumentation





• SCADA Security


40

21




• Instrumentation





• SCADA Security


41

Best Practice: ODMS

• Independent Data Warehouse for Plant and SCADA

Information

• Independent of Plant / SCADA Operation Historian

• SQL compliant Database

• Data Rollup capabilities

• Data Validation

• Ad-hoc reporting / analysis capability

• Reports and KPIs available on web-based system

Examples of Best Practices For Assessment

42

Best Practice: Historical Systems

• Dedicated Operational Historian for each Plant

• Server class hardware with RAID 5 or better

• Independent data collection (buffer data on failure of

Historical server)

• SQL compliant and/or OPC-DA Database

• Variable data collection rates

• Data Rollup capabilities

• Data Validation

• Ad-hoc reporting / analysis capability

Best Practice:

• Standardized, Fully Supported HW and SW Platforms

• Integrated SCADA System to monitor entire system

• Redundant, Meshed Communication Paths between

Control Facilities and to Critical Remote Facilities

• Redundancy at SCADA Server Level, with distributed

Front End Processing

• SCADA Network Separation from Business Network

• Thin-client HMIs available for wide use of SCADA

Information

• RTU Process Control with remote setpoints

Best Practice:

• Up to Date Record Drawings

• Accurate SCADA and Network Block Diagrams

• Up to Date SCADA System O&M Manuals

• Documentation Standards enforced

• SCADA O&M Manuals, including Documented Control

Strategies are complete

• Electronic storage, centralized server or web-based

repository

• Change Management Procedures in place

22

• System architecture that achieves

availability and performance

objectives

• Industry leading products improve

support, product availability,

ease of integration

• Mainstream technologies reduce

cost and avoid obsolescence

• Modular, open architecture

enables the replacement of

components with minimal impact

• Standardization enhances

maintainability & cost of delivery

Overall System Design

Solidifies Automatic

Control

43

Overall System Design

44

23

SCADA System BenchmarksOverall SCADA System Architecture

Least Desirable Most Desirable

AutoDialer

Alarm Only

HMI with

“Dumb”

RTUs, no

control

capability

HMI with

Proprietary

RTUs and

Protocols.

Local-

Automatic

Controls

Multiple

Monitoring

Systems

with

overlapping

capabilities.

Local

Automatic

Control Only

Integrated HMI

(Single System)

supporting

multiple RTUS

and Protocols.

Remote

Supervisory

Control

Proprietary

HMI with

Proprietary

RTUs and

Protocols

Limited

Local

Control

Any TownAny Town

45

• Data Capture• Quality Data – timely and

accurate

• Buffering at RTU/PLC

• Data Accessibility• User-friendly Tools

• Scheduled and Ad-hoc Reporting

• Dashboards/KPIs

• Data Management• Secure, Back-up, and

Recovery

• Enterprise Integration• CMMS, CIS, GIS, LIMS

Data Management & Reporting

46

Any Town, USA – Dewatering Statistics

24


47


48

Any Town, USA – Dewatering Statistics

25

49

Least Desirable Most Desirable

No Historical

Databases

Multiple Plant

Historian Databases

(Separate databases)

Limited Data

Exchange

between

Applications

Dedicated

ODMS, based on

Open Database

(Oracle, etc.).

Web tools for

data analysis

and reporting.

Any TownAny Town

SCADA System BenchmarksOperational Data Management System

Data Mining, Visualization and Analysis

Enterprise Integration

CMMS GIS LIMS FIS WQSHydraulic

Model

Unified Operations Data Model

Consequence

Value

Criticality

Operations Data Management & Reporting

PLCs:

Raw Water, Filtration, Disinfection

Reads and

Rounds – manual data entry

Instrumentation Power

Monitoring

$

ODMS Governance and Management

50

26

SCADA Governance

� SCADA Support

� Staffing

� Training

� External Resources

� Software and Hardware Design and

Development Methodology

� SCADA Standards

� Change Management

� Preventative Maintenance

� Planned Growth versus Organic

Migration

� Security

� Disaster Recovery / Business

Continuity

51

Requirement – Assessed against SCADA System Industry Best Practices

• Overall System Architecture

• Communications Architecture (In-Plant)

• Wide Area Networking

• Communications Architecture (Remotes)

• HMI Hardware Configuration

• In-Plant Control Philosophy

• In-Plant Control Architecture

• Re-Pump Station Control Philosophy

• Re-Pump Station Control Architecture

• RTU Architecture – Other Remote Sites

• RTU and PLC Programming

• Field Instrumentation

• Physical Security (SCADA Components)

• Alarm Subsystem

• SCADA Historical System

• Operational Data Management

• Data Integration

• System Optimization

• System Documentation

• SCADA System Standards

• SCADA Cyber Security

• Disaster Recovery / Business

Continuity

• Service and System Maintenance

52

27


Project Initiation1

Current State2

Requirements3

System Assessment4



• Prioritize Requirements • Alternatives Tech Memo

• Alternatives Review Workshops • Executive Summary

• Vendor Presentations • High Priority Requirements

•Alternatives Validation Workshop • Alternatives Analysis

• Functional Requirements

• Evaluation Process

• Recommended Alternatives

53

Alternative Analysis: Alignment of Projects to Project Charter

54

�Review List of Projects Resulting from Analysis

�Align the projects with the Project Charter & Vision

�Rank the Project from Most Important to Least

Important

�Determine Which Projects Can Executed in Multiple

Manners and Make Quantitative Decision on Direction

of the Projects

28

Any Town, USA

SCADA Master Plan Development Project

Project Charter

� Standardization

� End-Of-Life Equipment Upgrade

� Security Management

� Technology Improvements

� Integration of SCADA Assets

� Continuity of Operations

� Enhancing Reliability

� Operations:

� Coordination

Develop a Master Plan which will start with an audit of existing SCADA

system and operations, and work through the development of the Master

Plan to address the following key items:

55

Any Town, USA


Project Charter

• Build consensus among stakeholders for system requirements

• Be completed in a timely manner to support Utility budgetary

planning efforts.

• Align SCADA technology and governance with the business

requirements of the Utility.

• Outline Short-term and Long-Term Projects with scope

statements, budget estimates, and CIP scheduling

The SCADA Master Plan will:

56

29

Final plan is a fully defined SCADA program

57

Outcome:

Prepare a 5 year program to move

the Utility from current state to the

desired state following best practices

and supporting the Utility’s Business

Requirements

Any Town, USA


Project Charter – Core Team Project Priorities Rating

� Standardization

� (2) End-Of-Life Equipment Upgrade

� (3) Security Management

� Technology Improvements

� Integration of SCADA Assets

� Continuity of Operations

� (4) Enhancing Reliability

� (1) Operations

� Coordination

Develop a Master Plan which will start with an audit of existing SCADA

system and operations, and work through the development of the Master

Plan to address the following key items:

58

30

Core Team Rating of Project Priorities

Ranked Priority – (1) Operations (Section 5.2)

Project Charter Key Item: Operations

Operations: Assessment of staffing, operation, support, and third parties contracts related to meeting the District’s business requirements.

System Assessment Initiatives - Project Title

Priority Tasks:

� Find and Hire a SCADA Engineer to drive the project

� Assess the need for a new position “SCADA Technician”

59


Ranked Priority – (2) End-of-Life Equipment Upgrade (Section 5.3)

Project Charter Key Item: End-Of-Life Equipment Upgrade

End-Of-Life Equipment Upgrade - review which elements of the current SCADA systems have reached their expected end-of-life and replacements are no longer available.


Priority Tasks:

� Christmas WTP SCADA system upgrade required to ensure continuous operations and availability of

spare parts that are not readily available for the existing system.

� Christmas WTP well field communications need to be enhanced to insure remote operation of

equipment is available and reliable.

� Easter Plant communication cables to equipment s monitoring panels should be upgraded from Coax

cable to fiber optic cable to support reliability and enhanced capability.

� Easter Plant upgrade the SCADA system to take advantage of optimization of CIP improvements

project, and insure the system is reliable and available. Some system components are at the end-of-

useful life and require upgrade and replacement.

60

31


Ranked Priority – (3) Security Management (Section 5.4)

Project Charter Key Item: Security Management

Security Management – improve existing SCADA systems’ security and comply with regulatory requirements. This will address aspects of SCADA security such as: system physical/logical isolation, firewalls, anti-virus software, passwords, patch updates, management processes, etc.


Priority Tasks:

� Implementation SCADA System computer virus protection, operating system patching, and general IT

management assistance that will require improvements to the SCADA Network Segmentation on a

Facility by Facility Basis. Each facility will require implementation of:

� Coordinated IP address scheme

� A test and programming SCADA server at each facility

� Implementation of security zones for safe data transfer into and out of the SCADA networks

61


Ranked Priority – (4) Enhancing Reliability (Section 5.5)

Project Charter Key Item: Enhancing Reliability

Enhancing Reliability – review system architecture for redundancy and reliability. Items under this would include communications, primary/standby I/O servers, development nodes, etc.


Priority Tasks:

� Allow backup control center for Remote Monitoring and Control during Emergency

� Improve Historical Data Collection equipment, policies and methods to support operational

requirements.

� Improve System Redundancy at SCADA systems (servers & historians)

� Provide tools to improve start-up of Gas Production Plants

� Easter Plant Network Monitoring Software with Failure Notification & Network IP Addressing

improvements

� Improvements for Pumping Stations to enhance operations through automation and

environment monitoring (building temperature and fire detection monitoring)

62

32


Short Term Projects (Section 5.1)

Short Term Projects

Short Term Projects – The Short-term initiatives consist of those projects which should be undertaken in the next six months to one year. These issues are typically relatively straightforward to correct and normally require a limited budget


Priority Tasks:

� Initial SCADA System Standards required for Christmas WTP Upgrade to avoid additional organic

growth of SCADA system without definition by MUD

� Definition of how Utility will implement Historical Operational Data for report and alignment with

business systems. This project defines what type of database and DB tools are required.

63


Project Initiation1

Current State2

Requirements3

System Assessment4



• Review of Recommended Projects • SCADA Master Plan

• Master Plan Review Workshops • Executive Summary

• Presentation to Steering Committee • Current State Summary

• Requirements Summary

• Recommended Alternatives

• Program Plan

• Scope Definition

• Cost Estimate

• Schedule & Dependencies

64

33

Examples – Any Town, USA

65


66

34


67







• Summary

68

35

Summary – SCADA Master Planning

�Due to the criticality of SCADA to Utility operations, every

Utility should have a formal SCADA Master Plan

�Master Plans should be reviewed and updated on an

annual basis.

�Major updates or revisions should be performed

approximately every five years.

69

Summary – SCADA Master Planning

70

� Stakeholder Input – Plant staff provide input

� Core Team to facilitate and Provide ongoing input to planning

� Executive Steering Committee provided vision and commitment to planning

� Results Should Align with Project Requirements & Project Charter:

� Program addressing highest priority projects to mitigate Utility risks� Each planned project includes a statement on Project Goal, Project Need,

and Project Tasks and Deliverables.

� Projects scheduled to balance annual capital improvement costs over

the plan duration.

� Create Standards / Governance to facilitate efficient maintenance and

operation across systems

36

Lifecycle Management of SCADA

People and

Practices required

to manage and

maintain the

SCADA system in

an “evergreen” state

over its entire

lifecycle

71

Illinois Section AWWA Presents:Planning For Critical Data Systems –SCADA and Technology Master Planning April 26, 2012

Presented by:

William (Bill) Biehl

Cell: 913.601.0104

[email protected]

presentation outline · 2018-04-04 · 8 presentation outline •resentation goals p • master...

Documents