welcome to the spring workshop! - southwest power pool spp re spring workshop... · welcome to the...

Welcome to the Spring Workshop!

SPP.org ->Regional Entity ->2016 Spring Workshop to:

• Download materials • Submit anonymous question/comment

Wireless Select “SPP GUEST” network. A login page will open. Enter your email address.

You may also email comments or questions to [email protected].

March 15 8:00-8:20 Welcome and Introductory Remarks Dave Christiano, SPP RE Trustee

8:20-9:20 1 - NERC Relay Performance Update Sam Chanoski, NERC

9:20-9:35 Break

9:35-10:35 2 - Upcoming Standards: COM-002-4, MOD-025-2, Jim Williams, Jeff Rooker, & MOD-033-1, PRC-002-2 Greg Sorenson, SPP RE

10:35-10:45 Break

10:45-11:45 3 - Compliance with New Versions of PRC-005: Bob Kenyon, NERC Issues to Consider, Tips, and Best Practices 11:45-1:00 Lunch

1:00-2:00 4 - Simplified Process Development Methodology Jack Kutzer, Encari How to Melt the ICE!

2:00-2:10 Break

2:10-3:10 5 - Stakeholder Panel on Internal Controls Donna Maskil-Thompson, BPU Mike Ayotte, ITC3:10-3:30 Coffee and Snack Break

3:30-4:15 6 - Mitigation Update Jenny Anderson, SPP RE 4:15-4:45 7 - New Misoperations Process and Registration Changes Greg Sorenson, SPP RE Thomas Teafatiller, SPP RE

March 168:00-8:05 Opening Remarks

8:05-8:55 8 - CIP Update Steven Keller, SPP RE

8:55-9:05 Break

9:05-10:00 9 - Sampling Handbook/Audit Processes Mike Hughes, SPP RE Shon Austin, SPP RE 10:00-10:10 Break

10:10-11:00 10 - Most Violated Standards in SPP RE - What Went Wrong? Greg Sorenson, SPP RE Jeremy Withers, SPP RE11:00-11:10 Break

11:10-11:50 11 - General Manager Update Ron Ciesiel, SPP RE 11:50-12:00 Evaluation and Closing John Meyer, SPP RE Trustee

2016 SPP RE Spring Workshop SPP Corporate Center, Little Rock

March 15 8:00-8:20 Welcome and Introductory Remarks Dave Christiano, SPP RE Trustee

8:20-9:20 1 - NERC Relay Performance Update Sam Chanoski, NERC

9:20-9:35 Break

9:35-10:35 2 - Upcoming Standards: COM-002-4, MOD-025-2, Jim Williams, Jeff Rooker, & MOD-033-1, PRC-002-2 Greg Sorenson, SPP RE

10:35-10:45 Break

10:45-11:45 3 - Compliance with New Versions of PRC-005: Bob Kenyon, NERC Issues to Consider, Tips, and Best Practices 11:45-1:00 Lunch

1:00-2:00 4 - Simplified Process Development Methodology Jack Kutzer, Encari How to Melt the ICE!

2:00-2:10 Break

2:10-3:10 5 - Stakeholder Panel on Internal Controls Donna Maskil-Thompson, BPU Mike Ayotte, ITC3:10-3:30 Coffee and Snack Break

3:30-4:15 6 - Mitigation Update Jenny Anderson, SPP RE 4:15-4:45 7 - New Misoperations Process and Registration Changes Greg Sorenson, SPP RE Thomas Teafatiller, SPP RE

March 168:00-8:05 Opening Remarks

8:05-8:55 8 - CIP Update Steven Keller, SPP RE

8:55-9:05 Break

9:05-10:00 9 - Sampling Handbook/Audit Processes Mike Hughes, SPP RE Shon Austin, SPP RE 10:00-10:10 Break

10:10-11:00 10 - Most Violated Standards in SPP RE - What Went Wrong? Greg Sorenson, SPP RE Jeremy Withers, SPP RE11:00-11:10 Break

11:10-11:50 11 - General Manager Update Ron Ciesiel, SPP RE 11:50-12:00 Evaluation and Closing John Meyer, SPP RE Trustee

124C

CP

U R

OO

M

106

STO

RA

GE

FIRST FLO

OR

- PUB

LIC SPA

CES

LEGEN

D

AU

DITO

RIU

M

RE

STR

OO

MS

BR

EA

K R

OO

MN

OR

TH

PAR

KIN

G D

ECK

YOU

AR

E HER

E

BU

SIN

ES

S C

EN

TER

AB

C

DE

OU

TSID

E S

MO

KIN

G A

RE

A

CO

NFE

RE

NC

E R

OO

MS

MA

IN EN

TRA

NC

E

Auditorium

Break R

oom/

Lunch

Smoking

Vending Machines

Restroom

s

Restroom

s

ERO Enterprise Relay

Performance Update

Sam Chanoski, Director, Situation Awareness and Event Analysis

Spring SPP RE Workshop

March 15, 2016

RELIABILITY | ACCOUNTABILITY2

• Reliability – addressing real problems to improve the reliability of the grid.

• Assurance – being accountable to customers, the industry and government for the performance of the grid.

• Learning – enabling the industry to learn from experience to improve future reliability performance.

• Risk-based model – focusing actions and programs on issues most important to grid reliability.

NERC Pillars


Performance, Regulation, and Excellence

Normal Performance

Excellence

Practical Minimum

Acceptable

EA, Info Sharing

CMEP

Regula

tory

Cra

ft

Forums, Trades

Regulatory Minimum

Acceptable


Why is this important?S

ever

ity

Inverse

Cost-Benefit

Significance Threshold

Learn and Reduce

Avoid

Frequency

Harms

“Pick important problems and fix them”Dr. Malcolm Sparrow

John F. Kennedy School of Government


All Causes (199 Cat 1a events, eliminating where RC = AZ)

What causes events with misoperations?


Cat 1a event causes

Things break


Cat 1a event causes

…In ways unanticipated by design…


…In The Context Of An Organization


Sometimes it is a human


Cat 1a event causes

…But not as often as you think


The PII Performance Pyramid™


• Latent errors: mutual coupling and ground instantaneous overcurrent

• Focused improvements: relay communication misoperations

• The importance of good barriers: commissioning

Three stories about relays


Latent errors: ground IOC

Line OOS for

maintenance

2100MW

• ~2100 MW generating plant, all units online

• Connected to system by three 345kV lines

• One line OOS for scheduled maintenance



Line OOS for

maintenance

SLG Fault

2100MW

• Single phase to ground fault at remote end of one line



• The effects of a line-end fault on the adjacent line, with end open, were not considered when developing the ground IOC element settings of the mutually coupled line

Line OOS for

maintenance

SLG Fault

Mutual coupling

2100MW



• Ground instantaneous overcurrent protection on the coupled line trips at one end only

• ~2100 MW of generation connected to the system through a single 345kV line

Line OOS for

maintenance

SLG Fault

Mutual coupling

Trips on ground

instantaneous

overcurrent

2100MW



• Generators become unstable due to system configuration

• Remaining 345kV line trips on an unstable power swing

• Isolated generation trips offline

Line OOS for

maintenance

SLG Fault

Mutual coupling

Trips on ground

instantaneous

overcurrent

2100MWGeneration goes

unstable

Line trips on

unstable power

swing


•ERO Event Analysis Process, Category 1a Events

•Most misoperations are overtrips

•Most overtrips are due to incorrect relay settings

•Most incorrect settings are ground instantaneous overcurrent elements Too sensitive for current conditions

Short circuit capacity of the system increased over time

Often, zone 1 ground distance elements already available



• SPP white paper

• Focused on contributing subsystems (disaggregating the harm!)

• Root causes and lessons learned

Focused improvements: communications


• 35% of misops from Station Signal Path

• 29% of misops from Communication Interface

• Consistent with NERC analyses



Good barriers: commissioning

Protective relaying and RAS limit harms

Initiating Events

Major System

Disturbance

No one barrier is perfect, so we use defense in depth

Adapted from James Reason’s Theory of Accident Causation, 1990


• Last step in construction, or standalone event?

• Independent testing, contractors

• Last chance to (easily) expose and fix latent errors Wiring/polarity errors

AC quantities

As-built prints

Good barriers: commissioning


• 2015 Analysis of System Protection Misoperations (http://www.nerc.com/pa/RAPA/PA/Pages/default.aspx)

• ERO Event Analysis Program (http://www.nerc.com/pa/rrm/ea/Pages/EA-Program.aspx)

• NERC Lessons Learned (http://www.nerc.com/pa/rrm/ea/Pages/Lessons-Learned.aspx)

Additional Information

http://www.nerc.com/pa/RAPA/PA/Pages/default.aspx

http://www.nerc.com/pa/rrm/ea/Pages/EA-Program.aspx

http://www.nerc.com/pa/rrm/ea/Pages/Lessons-Learned.aspx


Sam Chanoski

Director, Situation Awareness & Events Analysis

Office (404) [email protected]

mailto:[email protected]

Upcoming/New Standards

Jim Williams, Jeff Rooker, and Greg SorensonMarch 15, 2016 SPP RE workshop

1

Overview

• COM-002-4 (effective 7/1/16)

• MOD-025-2 (effective 7/1/16)

• MOD-033-1 (effective 7/1/17)

• PRC-002-2 (effective 7/1/16)

• Addendum- MOD-032-1 Effective 7/1/15 for R1 and 7/1/16 for R2-R4

2

Today’s Presentation

• Some language has been shortened, paraphrased, or omitted

• We are not covering every footnote or special case listed in the standards

• It is important for you to read each standard in its entirety and consider how your company meets the requirements

3

COM-002-4

OPERATING PERSONNEL COMMUNICATIONS PROTOCOLS

EFFECTIVE 7/1/16

4

COM-002-4

• Enforcement date July 1, 2016

• Applicability: Functional Entities– Balancing Authority (BA)

– Distribution Provider (DP) NEW

– Reliability Coordinator (RC)

– Transmission Operator (TOP)

– Generator Operator (GOP)

5

Terms• Operating Instructions

– A command by operating personnel responsible for the Real-time operation of the interconnected Bulk Electric System to change or preserve the state, status, output, or input of an Element of the Bulk Electric System or Facility of the Bulk Electric System.

– (A discussion of general information and of potential options or alternatives to resolve Bulk Electric System operating concerns is not a command and is not considered an Operating Instruction.)

6

Terms

• Real-Time – Present time as opposed to future time.

• Emergency – Any abnormal system condition that requires automatic or immediate manual action to prevent or limit the failure of transmission facilities or generation supply that could adversely affect the reliability of the Bulk Electric System

• Written Instruction – TLRs (Transmission Loading Relief), protocols, operating guides, email, messaging system, fax

• Oral single-party to multiple-party burst – Satellite phone

7

COM-002-4• R1. Each BA, RC, and TOP shall develop documented

communications protocols for its operating personnel that issue and receive Operating Instructions.

• The protocols shall, at a minimum require its operating personnel:– R1.1. To use English language, unless agreed to

otherwise.

8

COM-002-4– R1.2. That issues an oral two-party, person-to-person

Operating Instruction to take one of the following actions: Confirm the receiver’s response if the repeated information is

correct

Reissue the Operating Instruction if the repeated information is incorrect or if requested by receiver

Take an alternative action if a response is not received or if the Operating Instruction was not understood by the receiver

9

COM-002-4– R1.3. That receives an oral two-party, person-to-person

Operating Instruction to take one of the following actions: Repeat, not necessarily verbatim, the Operating Instruction

and receive confirmation from the issuer that the response was correct

Request that the issuer reissue the Operating Instruction

– R1.4. Require its operating personnel that issue a written or oral single-party to multiple-party burst Operating Instruction to confirm or verify that the Operating Instruction was received by at least one receiver of the Operating Instruction

10

COM-002-4

– R1.5. Specify the instances that require time identification when issuing an oral or written Operating Instruction and the format for that time identification

– R1.6. Specify the nomenclature for Transmission interface Elements and Transmission interface Facilities when issuing an oral or written Operating Instruction

11

COM-002-4

• R2. Each BA, RC, and TOP shall conduct initial training for each of its operating personnel responsible for the Real-time operation of the interconnected BES on the documented communications protocols developed in Requirement R1 prior to that individual operator issuing an Operating Instruction.

12

COM-002-4

• R3. Each DP and GOP shall conduct initial training for each of its operating personnel who can receive an oral two-party, person-to-person Operating Instruction prior to that individual operator receiving an oral two-party, person-to-person Operating Instruction to either:– Repeat, not necessarily verbatim, the Operating

Instruction and receive confirmation from the issuer that the response was correct, or

– Request that the issuer reissue the Operating Instruction.

13

COM-002-4• R4. Each BA, RC, and TOP shall at least once every

twelve (12) calendar months:– R4.1. Assess Operating Personnel’s adherence to the

protocols

– Provide feedback to operating personnel

– Take corrective action to address deviations from the protocols

– R4.2. Assess the protocol’s effectiveness and modify as necessary

14

COM-002-4

• R5. - Each BA, RC, and TOP that issues an oral two-party, person-to-person Operating Instruction during an Emergency, excluding written or oral single-party to multiple-party burst Operating Instructions, shall either:

– Confirm the receiver’s response if the repeated information is correct (in accordance with Requirement R6).

– Reissue the Operating Instruction if the repeated information is incorrect or if requested by the receiver, or

– Take an alternative action if a response is not received or if the Operating Instruction was not understood by the receiver

15

COM-002-4

• R6. - Each BA, DP, GOP, and TOP that receives an oral two-party, person-to-person Operating Instruction during an Emergency, excluding written or oral single-party to multiple party burst Operating Instructions, shall either:– Repeat, not necessarily verbatim, the Operating

Instruction and receive confirmation from the issuer that the response was correct, or

– Request that the issuer reissue the Operating Instruction.

16

COM-002-4

• R7. - Each BA, RC, and TOP that issues a written or oral single-party to multiple-party burst Operating Instruction during an Emergency shall confirm or verify that the Operating Instruction was received by at least one receiver of the Operating Instruction.

17

COM-002-4 Evidence • Documented communication protocols

• Training records– Attendance list, agendas and learning objectives

• Assessments of the protocols– Feedback of the assessments, findings of effectiveness

and changes made to the documented protocols

• For written or oral operating Instructions - Voice recording, transcripts of voice recordings or operator logs

• Data Retention – May be asked to show compliance since the last audit

18

MOD-025-2

REAL AND REACTIVE CAPABILITY VERIFICATION AND REPORTING

EFFECTIVE 7/1/16

19

MOD-B Initiative Status

• MOD-025-2 Real and Reactive Capability Verification for Planning Models

– Effective 7/1/16

• MOD-032-1 Data requirements for Power System Modeling

– Effective 7/1/15 for R1 and 7/1/16 for R2-R4

• MOD-033-1 Steady State and Dynamic Model Validation

– Effective 7/1/17

20

MOD-025-2

• Combined MOD-024-01 & MOD-025-01 into one concise standard to cover Generator Real and Reactive Capability Testing

• Purpose: To ensure that accurate information on generator gross and net Real and Reactive Power capability and synchronous condenser Reactive Power capability is available for planning models used to assess Bulk Electric System (BES) reliability

21

MOD-025-2

• Applicable Entities– Generator Owner (GO)– Transmission Owner (TO) owning synchronous

condensers • Impacted facilities connected to BES (Bulk Electric

System)– Individual units greater than 20 MVA (gross nameplate

rating)– Synchronous condensers greater than 20 MVA (gross

nameplate rating)– Generating plant/facilities greater than 75 MVA (gross

aggregate nameplate rating)22

MOD-025-2

• Phase in testing over time– 40% of applicable units Gross MVA by 7/1/2016

– 60% by 7/1/2017

– 80% by 7/1/2018

– 100% by 7/1/2019

• Wind Farm Verification - If Registered Entity has two wind sites, and verification of one site is complete, the entity is 50% complete regardless of the number of turbines at each site

23

MOD-025-2

• R1-R2 address GO Capability Testing Requirements for Real and Reactive Power, respectively– R1.1 and R2.1 “Attachment 1” outlines the periodicity

for conducting a new verification and specifications– Periodicity for verification:

• Staged test – at least every five years if capability does not change by 10% (otherwise within 12 months of change). First verification must be staged.

• Or verify using historical operational data

• New units or long term shut down units within 12 months of Commercial Operation Date

24

MOD-025-2

• R1.1 and R2.1 “Attachment 1” specifications examples:⁻ Auxiliary equipment and Automatic Voltage Regulator in

service

⁻ Calculate Generator Step Up (GSU) transformer losses if measurements on high side of GSU

⁻ One-line showing sources of auxiliary real and reactive power sources

⁻ Record ambient conditions, dates, voltage, etc.

⁻ Details page 13-16 of standard

25

MOD-025-2

• R1.2 and R2.2 “Attachment 2” outlines what testing has been performed along with the data to be reported to Transmission Planner using the form (pages 17-20 of Standard)

• Submit within 90 days of either:– Date of staged test – Date data selected for historical operational data

• R3. TO provide verification of reactive power capability of synchronous condenser units in accordance with Attachment 1 and 2

26

MOD-025-2 Evidence

• Example Evidence⁻ List of applicable facilities with date of verification

⁻ Performed verification, completed Attachment 2, dated evidence submitted to TP within 90 days

• Evidence Retention⁻ Since last audit or since commercial operation date

27

MOD-033-1

STEADY STATE AND DYNAMIC MODEL VERIFICATION

EFFECTIVE 7/1/17

28

MOD-033-1

• Purpose: To establish consistent validation requirements to facilitate collection of accurate data and building of planning models to analyze reliability interconnected transmission system reliability

• New standard

• Requires each Planning Coordinator (PC) to implement a documented process to perform model validation within its planning area

• Two requirements; both effective 7/1/17

29

MOD-033-1

• Steady-State and Dynamic System Model Validation– R1. Requires each PC implement a documented process

to validate the data from MOD-032-01 from actual system response R1.1 & R1.2. Establishes timeframe for validation Steady

State and Dynamic- within 24 months of effective date R1.3 & R1.4. Guidelines to assess and resolve

unacceptable differences in performance

30

MOD-033-1

• R2. Data Reporting Requirements Establishes obligation of RC and TOP to provide

actual real time system behavior data to the PC – Within 30 days of a request

• Evidence Dated email or notice of receipt of request and

issuance of data

Statement of no notice received

• Data Retention Since last audit or date registered

31

PRC-002-2

DISTURBANCE MONITORING AND REPORTING REQUIREMENTS

EFFECTIVE 7/1/16

32

PRC-002-2

Purpose: To have adequate data available to facilitate analysis of Bulk Electric System Disturbances

Builds on PRC-002-1 but more specific

• Sequence of Events Recording (R1 to R2)

• Fault Recording (R1, R3, R4)

• Dynamic Data Recording (R5 to R9)

• Synchronizing all recording (R10)

• Events (R11)

• Broken recording devices (R12)

33

PRC-002-2 Implementation

• R1, R5 – July 1, 2016

• R12 – October 1, 2016

• All others – 50% compliant (i.e. installed) by July 1, 2020

• 100% compliant by July 1, 2022 – Allows for scheduling outages and upgrades

– Only one BES bus have until July 1, 2022

• After a reassessment by the TO in R1 or PC in R5, all notified parties have 3 years to be 100% compliant with the new list

34

PRC-002-2

• R1. Each TO shall:– R1.1. Identify buses for which sequence of events

recording (SER) and fault recording (FR) data is required per Attachment 1

– R1.2. Notify adjacent owners within 90 days

– R1.3. Re-evaluate all BES buses at least once every 5 calendar years…

35

PRC-002-2 Attachment 1

• Highlights process for determining where to place equipment

• Step 1: Complete list of BES buses– Share a common ground grid

– Ring bus and breaker and a half generally one bus for purposes of this standard

• Step 2: Reduce list to those with 3Ø short circuit of 1500 MVA or greater– If no buses >1500 MVA, no FR or SER required

36

PRC-002-2 Attachment 1

• Step 3: Determine 11 with highest maximum available current– If less than 11 just install at bus with highest fault

current

• Step 4: Determine median MVA level (not mean)

• Step 5 & 6: Reduce the list to those buses with the greater of:– 3Ø > 1500 MVA or

– 3Ø > 20% of median MVA

37

PRC-002-2 Attachment 1• Step 7: If more than 11 buses:

– The 10% with the highest available maximum 3Ø short circuit need SER and FR, AND

• Step 8: – An additional 10% are chosen by the TO to maximize

wide area coverage.

– Considerations: Electrically distant buses Voltage sensitive areas Cohesive load and generation zones BES buses with relatively high number of circuits BES buses with reactive devices Major Facilities connecting outside the TO’s area 38

PRC-002-2

• R2.– TO and GO

– SER data covers circuit breaker position for buses in R1

• R3.– TO and GO

– FR data should include (for each R1 bus): Phase to neutral voltage (all phases)

Phase current and neutral current for:

– Transformers with low side >100 kV– Transmission lines

39

PRC-002-2

• R4.– TO and GO.

– FR must data from R3 must: 2 cycles pre-trigger, 30 cycles post-trigger OR

2 cycles pre-trigger, 3 cycles post-trigger through final cycle of the fault

– Minimum of 16 samples per cycle

– Triggered on (minimum): Neutral overcurrent

Phase undervoltage or overcurrent

40

PRC-002-2• R5. PCs identify Dynamic Disturbance Recording (DDR)

data required for at least:– Single generator units larger than 500 MVA

– Units larger than 300 MVA at a facility larger than 1,000 MVA

– BES elements that are part of an IROL or a stability (angular or voltage) SOL

– HVDC elements larger than 300 MVA

– At least one BES element within a UVLS area

• Minimum of one DDR for every 3,000 MW of Demand

• 90-day notice, 5 year restudy41

PRC-002-2

• R6. (Notified) TOs must have DDR data to determine:– One phase-to-neutral or positive sequence voltage

– Matching phase current or positive sequence current

– Real Power and Reactive Power on a 3Ø basis corresponding to all circuits

– Frequency of any of the voltages

42

PRC-002-2

• R7. (Notified) GO shall have DDR data to determine:– One phase-to-neutral, phase-to-phase, or positive

sequence at either the high or low side of the GSU

– Matching phase current, phase-to-phase current or positive sequence current

– Real Power and Reactive Power on a 3Ø basis

– Frequency of at least one voltage

43

PRC-002-2

• R8. (Notified) GOs and TOs with DDR data must have continuous data recording and storage.

• Legacy equipment must:– Have a triggered record length of at least 3 minutes

– At least one trigger (Eastern Interconnection): Frequency <59.75 Hertz, >61.0 Hertz

Rate of frequency change <-0.03125 Hertz/second, >0.125Hertz/second

Undervoltage at 0.85 per unit for 5 seconds

44

PRC-002-2

• R9. Each (notified) GO and TO responsible for DDR data should meet:– Input data 960 samples per second (!!)

– Output recording rate of electrical quantities of at least 30 times per second

• R10. Each TO and GO shall synchronize all SER and FR data for BES buses … and DDR for BES elements … to meet the following:– Synchronization to UTC (local offset OK)

– Synchronized device clock accuracy +/- 2 milliseconds

45

PRC-002-2

• R11. Each TO and GO shall provide, upon request, all SER and FR data …. and DDR data… to the PC, RE, or NERC upon request.– Data retrievable for 10 calendar days

– Provided within 30 calendar days of a request

– SER use ASCII CSV format as in Attachment 2

– FR and DDR formatted per IEEE C37.111-1999 (or later)

– Data files named per IEEE C37.232-2011 (or later)

46

PRC-002-2

• R12. Each (notified) GO and TO shall, within 90 calendar days of the discovery of a failure of SER, FR, or DDR data shall:– Restore the recording capability

– Submit a Corrective Action Plan to the RE and implement it

• Lots of additional (non-auditable) background material included in the standard

47

PRC-002-2 Implementation

• R1, R5 – July 1, 2016

• R12 – October 1, 2016

• All others – 50% compliant (i.e. installed) by July 1, 2020

• 100% compliant by July 1, 2022 – Allows for scheduling outages and upgrades

– Only one BES bus have until July 1, 2022

• After a reassessment by the TO in R1 or PC in R5, all notified parties have 3 years to be 100% compliant with the new list

48

PRC-002-2 – Key Points

• TO does analysis for SER and FR (Attachment 1)

• PC does analysis for DDR (following a prescriptive list)

• Notified parties (TOs and GOs) install equipment that meets the various specifications

• Legacy equipment can be used, but check against minimum requirements (R1, R8)

• Data helpful during events

49

Questions ?

Jim WilliamsLead Compliance [email protected]

Jeff RookerLead Compliance [email protected]

Greg SorensonSenior Compliance [email protected]

50




Additional Reference Material

MOD-032-1

Data requirements for Power System Modeling

Effective 7/1/15 for R1; 7/1/16 for R2-R4

51

Addendum MOD-032-1

• Provided as reference material

• Addressed at Fall 2014 workshop

• Effective 7/1/15 for R1 and 7/1/16 for R2-R4

52

MOD-032-1 Overview• Purpose:

– To establish consistent modeling data requirements and reporting procedures for development of planning horizon cases necessary to support analysis of the reliability of the interconnected transmission system.

• Applicable to Balancing Authority (BA), Generator Owner (GO), Load Serving Entity (LSE) , Resource Planner (RP), Transmission Owner (TO), Transmission Planner (TP), Transmission Service Provider (TSP) and Planning Coordinator (PC) functions

• Effective Dates

– R1: 7/1/15

– R2-R4: 7/1/16 53

MOD-032-1 Overview

• MOD-032-1 consolidates and replaces MOD-010, 011, 012, 013, 014 and 015

• Requires applicable data owners to submit data to their respective Transmission Planners and Planning Coordinators to support the Interconnection-wide case building (power flow, short circuit, and dynamic cases) process in their Interconnection

54

MOD-032-1 Specific Data Requirements• Steady-State Power Flow

– Bus, Load, Generation, Aux Load, AC and DC Transmission, Transformers, Reactive Compensation, and Static VAR Systems

• Dynamics– Synchronous Machines, Other Technologies, Excitation, Governor, Power

System Stabilizer, Composite Load Models, Wind Turbines Data, Photovoltaic, Static VAR Compensators/FACTS Devices, and DC System Models

• Short Circuit– Positive, Negative, and Zero Sequence Data, and Mutual Line Impedance

Data

• Planning Coordinator established as Entity for data collection and assembling the power flow, dynamic, and short circuit cases

• Listed in a detailed table in “Attachment 1” 55

MOD-032-1 Overview

56

Excerpt from Attachment 1

MOD-032-1 Overview• R1-Data Reporting Requirements

– Consistency of Data being Reported

– Additional Reporting Requirements

Station Service Auxiliary Load

Wind and Solar data

Dynamic load data requirements (TPL-001-4 R2.4)

• R2-Establishes obligation of BAs, GOs, LSEs, RPs, TOs, TPs, TSPs, and PCs to provide the data

Confirmation of No-Change to data

• R3-Establishes methodology to vet questionable data

• R4-Sets expectation of how the PC will report this data 57

MOD-032-1 R1 Overview

R1 Each PC and each TP shall jointly develop steady-state, dynamic, and short circuit modeling data requirements and reporting procedures including:

1.1. Data listed in Attachment 1 for near term and long term planning horizon

1.2 Shared on interconnect wide basis consistent with the following specifications (not limited to this list)

Data format and level of detail on equipment modeled

Case scenarios modeled

Schedule for data submission at least every 13 months

1.3 Specifications on distribution/posting of data requirements and reporting procedures

58


• Who: – PC and each TP

• What:– Jointly develop steady –state, dynamics and short

circuit model data requirements and reporting procedures

– At a minimum, the data listed in Attachment 1 for each reporting entity by function is required (GO, RP, TO, LSE, BA,TSP)

– Specifications for building model cases and schedule

– Specifications for distribution/posting59


• How:– Data provided by data owners

– See Application Guidelines of Standard

TP may collect and aggregate and submit to PC

But no requirement for TP to provide data to the PC

The Submitting entities are responsible

• Why:– To effectively model interconnected transmission

system of Eastern Interconnect

60

MOD-032-1 R1

• Evidence Requested– PC and TP jointly developed model data requirements

and reporting procedures

– Schedule for data submission at least every 13 calendar months

– Distribution of Posting and reporting procedures

61


• R2: Each BA, GO, LSE, RP, TO and TSP shall each provide steady-state, dynamic, and short circuit modeling data to TPs and PCs according to R1:– Data requirements

– Reporting Procedures

• Confirm if no changes since last submission

62

MOD-032-1 R2

• Evidence of:– Entity data submission meets PCs and TPs data

requirements developed in R1

– Entity data submission meets PCs and TPs reporting procedures developed in R1

– Entity provided written confirmation if no changes occurred since last submission

63


• R3: Each BA, GO, LSE, RP, TO and TSP shall respond to PC or TP notification of technical concerns on modeling data submitted as follows:– Provide either updated data or,

– Explanation with technical basis for maintaining current data and,

– Provide response within 90 days of receipt of request unless longer time period agreed upon by PC or TP

64

MOD-032-1 R3

• Question: Has written notification of technical concerns been received from PC or TP in compliance monitoring period?

– If Yes, provide evidence of: Dated Notification from PC or TP

Updated data submission or explanation with technical basis why data not changed

Dated submission to PC or TP within 90 days or agreement if beyond 90 days

65

MOD-032-1 R4

• R4: Each PC shall provide to the ERO (or its designee) the model data provided under R2 for its planning area to support creation of Interconnection-wide cases

• Proposed RSAW evidence requested:– Request from ERO or its designee to provide models

– Dated model data submission

66

Modeling Internal Control Examples

67

Model Data Reporting

Procedures

Joint Development Completeness

DataManagement

Accuracy

PC and TP jointly develop data requirements and reporting procedures (MDWG)

Data owners verify changes to systems are reviewed, approved, and double checked after implementation to ensure accuracy of all study models

Procedure/Process Control Control Activity Control Type

Detective Control

Preventative ControlModel

development

Maintain notices and response

Validity

Records are validated and documented periodically regarding response in specified timeframe.

Preventative Control

Notice of technical

concern with model data

MOD-032-1 Summary

• R1: PC and TP shall jointly develop model data requirements and reporting procedures

• R2: Entities shall provide model data per R1 requirements and time frame

• R3: If PC has a technical concern, entities shall update and submit model data within required time frame

• R4: PC shall provide models to ERO for Interconnection -wide cases

68

Complying With The New Versions Of PRC-005Robert W. Kenyon, P.E. Senior Engineer, Reliability Assurance

SPP Workshop

March 15, 2016


• Review PRC-005-1• Evolution of new versions of PRC-005• Review PRC-005-6• Clarifications of PRC-005-6 • Implementation Plan• Tips• Violations Noted/Best Practices• Useful References• Q & A

Presentation Outline


• When referring to PRC-005-1.1b or the legacy standards, it is generally referring to: PRC-005-1.1b — Transmission and Generation Protection System

Maintenance and Testing PRC-008-0 — Underfrequency Load Shedding Equipment Maintenance

Programs PRC-011-0 — Under Voltage Load Shedding (UVLS) System Maintenance

and Testing PRC-017-0 — Special Protection System Maintenance and Testing

• All of the above prescribe essentially the same requirements for different systems

• All of the above to be retired and consolidated into the new PRC-005 series – but through a lengthy transition

PRC-005-1


In other words, PRC-005-1 requires that:• Entities must have their own justified PS Maintenance program• Entities must document it and execute it

Keep in mind the five elements of the Protection System in the NERC Glossary.

Quick Summary of PRC-005-1


PRC-005-6 includes:• Protection System Maintenance Program (PSMP) - different

from PRC-005-1• List of mandatory minimum maintenance tasks to be performed• Mandatory maximum maintenance intervals• Mandatory tracking of unresolved maintenance issues• Specific performance-based maintenance option• Other items included: Reclosing (R/C) and R/C auxiliaries,

Sudden Pressure Relays, Dispersed Generation, etc.

PRC-005-6 Requirements –New Areas


• R1 – Address the PSMP• R2 – Starting and maintaining a performance-based

maintenance program (optional - only in play if you use performance-based maintenance)

• R3 – Maintenance using fixed, NERC-mandated intervals• R4 – Maintenance using performance-based maintenance• R5 – Unresolved Maintenance Issues

Requirements of PRC-005-6


• Protective relays which respond to electrical quantities • Communications systems necessary for correct operation of

protective functions • Voltage and current sensing devices providing inputs to

protective relays• Station DC supply associated with protective functions

(including station batteries, battery chargers, and non-battery-based DC supply)

• Control circuitry associated with protective functions through the trip coil(s) of the circuit breakers or other interrupting devices

Component Types – Protection System


• Reclosing relay • Supervisory relay(s) or function(s) – relay(s) or function(s) that

perform voltage and/or sync check functions that enable or disable operation of the reclosing relay

• Voltage sensing devices associated with the supervisory relay(s) or function(s)

• Control circuitry associated with the reclosing relay or supervisory relay(s) or function(s)

Component Types – Reclosing


• Fault Pressure Relay – a mechanical relay or device that detects rapid changes in gas pressure, oil pressure, or oil flow that are indicative of faults within liquid-filled, wire-wound equipment

• Control circuitry associated with a fault pressure relay

Component Types – Sudden Pressure Relaying


Note that the R/C requirements in PRC-005-6 are restricted to:• Reclosing on elements connected to the Bulk Electric System

(BES) bus located at generating plant substations where generating plant capacity > capacity of the largest BES unit within the Balancing Authority (BA) Area or, the largest generating unit within the reserve sharing group

• Reclosing applied on the terminals of all BES elements at subs one bus away from above generation plants when the subs < 10 circuit-miles from the generating plant substation

• Reclosing applied as an integral part of an Remedial Action Schemes (RAS)

PRC-005-6 Reclosing Systems


There are two categories of generation covered:• Protection Systems and Sudden Pressure Relaying for generator

facilities that are part of the BES, except for generators identified through Inclusion I4 of the BES definition (4.2.5)

• Protection Systems and Sudden Pressure Relaying for the following BES generator facilities for dispersed power producing resources identified through Inclusion I4 of the BES definition (4.2.6)

• Different for traditional generating facilities vs. aggregated• See BES Definition Reference Document Version 2 April 2014

for clarification

PRC-005-6 Generation Requirements


Typical Dispersed Generation Facility


Typical PRC-005-6 Table Identifying Mandatory Tasks and Intervals


Standard New Topic Area Effective• PRC-005-1.1b BES Protection System 6/18/2007• PRC-005-2 Complete Revision 4/1/2015• PRC-005-2(i) Dispersed Generation 5/29/2015• PRC-005-2(ii) Added RAS/SPS NEVER• PRC-005-3 Added Reclosing NEVER• PRC-005-3(i) Dispersed Generation NEVER• PRC-005-3(ii) RAS/SPS NEVER• PRC-005-4 Sudden Pressure Relays NEVER• PRC-005-5 Dispersed Generation NEVER• PRC-005-6 R/C Supervisory Devices &

Sudden Pressure Relays 1/1/2016

Evolution Of PRC-005 (Does Not Reflect Phase In By Requirement)


Standard New Topic Area Status• PRC-005-1.1b BES Protection System In Force till 3/31/27• PRC-005-2 Complete Revision Retired 5/29/15• PRC-005-2(i) Dispersed Generation In Force till Jan 17• PRC-005-2(ii) Added RAS/SPS Never Enforceable • PRC-005-3 Added Reclosing Never Enforceable • PRC-005-3(i) Dispersed Generation Never Enforceable• PRC-005-3(ii) RAS/SPS Never Enforceable• PRC-005-4 Sudden Pressure Relays Never Enforceable• PRC-005-5 Dispersed Generation Never Enforceable• PRC-005-6 R/C Supervisory Devices In Force

& Sudden Pressure Relays

Present Status of PRC-005 Versions


• PRC-005-6, R1, the PSMP• Transitioning from PRC-005-1 to PRC-005-6• Battery Table 1-4f• PRC-005-6 R5 applicability

Clarifications


• The specifics of R1, PSMP, involves the term Component Type• Component Type Any one of the five specific elements of a protection system Any one of the four specific elements of automatic reclosing Any one of the two specific elements of sudden pressure relaying

• Component Any individual discrete piece of equipment included in a protection

system, automatic reclosing, or sudden pressure relaying

Clarifications PRC-005-6 R1 – the PSMP


• R1 - Establish a PSMP for its Protection Systems, Automatic Reclosing, and Sudden Pressure Relaying identified in Section 4.2, Facilities. The PSMP shall: 1.1. Identify which maintenance method (time-based, and/or

performance-based) is used to address each Protection System, Reclosing, and Sudden Pressure Relaying Component Type

1.2. Include the applicable monitored Component attributes applied to each Component Type where monitoring is used to extend the maintenance intervals beyond those specified for unmonitored Protection System, Automatic Reclosing, etc.

Simplified PRC-005-6 R1 Verbiage


Typical PRC-005-6 Table Identifying Mandatory Tasks and Intervals


Monitoring Attributes

Monitored microprocessor protective relay with the following: • Internal self-diagnosis and alarming (see Table 2) • Voltage and/or current waveform sampling three or more times

per power cycle, and conversion of samples to numeric values for measurement calculations by microprocessor electronics

• Alarming for power supply failure (see Table 2)


• Write a PSMP document• Include information required by parts 1.1 and 1.2• Include information analogous to that in the PRC-005-1 PS

maintenance and testing program• Include Performance-Based Maintenance Information (PBM), if

PBM is used• Recognize that to be compliant with time-based maintenance

requirements, you must meet PRC-005-6 table requirements, executing your program is insufficient for compliance

• If you use PBM, you have to execute your program, a la PRC-005-1 (see Requirement 4, PRC-005-6)

PRC-005-6, R1 Compliance Suggestions – Be on the Safe Side


• Became effective under PRC-005-2 effective 4/1/2015• These continue thru PRC-005-2(i), and PRC-005-6• R1 addresses the PSMP• R2 addresses Performance Based Procedures• R5 address Unresolved Maintenance Issues• R3 and R4 address actual maintenance performance. They are

governed by two complex implementation plans

Implementation, Requirements –One, Two, and Five


• There are now two implementation plans in force• The first covers tables one through three, PRC-005-2(i), (all sub-

tables)• The first table addresses PS, under-frequency, under-voltage,

and SPS.• The second covers tables four through five and addresses

reclosing and sudden pressure relaysMake sure you check out the precise language of all elements covered. As an example, the requirements for reclosing covers a very small number of schemes on most systems

Implementation Plans – Phase In Dates


1 Or, for generating plants with scheduled outage intervals exceeding two years, at the conclusion of the first succeeding maintenance outage.2 Or, for generating plants with scheduled outage intervals exceeding three years, at the conclusion of the first succeeding maintenance outage.

PRC-005-2/-2(i) R3 and R4 Implementation Timelines

Max. Maintenance Interval

% Compliant By

Less than 1 year(T 1-1 thru 1-5)

100% Oct. 1, 2015 (1D/1Q 18 mo. following regulatory approval of PRC-005-2)

1-2 calendar years(T 1-1 thru 1-5)

100% Apr. 1, 2017 (1D/1Q 36 mo. following regulatory approval of PRC-005-2)

3 calendar years(T 1-1 thru 1-5)

30% Apr. 1, 2016 (1D/1Q 24 mo. following regulatory approval of PRC-005-2)1





6 calendar years(T 1-1 thru 1-5 & T 3)

30% Apr. 1, 2017 (1D/1Q 36 mo. following regulatory approval of PRC-005-2)2





12 calendar years(T 1-1 thru 1-5, T 2, T 3)



60% Apr. 1, 2023 (1D/1Q 108 mo. following regulatory app. of PRC-005-2)


100% Apr. 1, 2027 (1D/1Q 156 mo. following regulatory app. of PRC-005-2)


PRC-005-6 R3 and R4 Implementation Timelines

Max. Maintenance Interval

% Compliant By

6 calendar years(T 4-1, 4-2(a), 4-2(b), 4-3, 5)

30% Jan. 1, 2019 (1D/1Q 36 mo. following regulatory approval of PRC-005-6)












• This issue has been reviewed with many Electric Reliability Organization (ERO) personnel across the country

• Is being reviewed at NERC• The strong consensus is that the registered entity must first

maintain the component under the requirements of PRC-005-2 (or later) before the entity can identify the component as being maintained under the “Post PRC-005-1.1b” standards, such as PRC-005-6

Implementation Plan – How Does an Entity Transition to New Standard?


• Don’t be mislead by the Implementation Plan• The dates are NOT dates you must start using the new PRC-005• These dates are the dates that certain percentages of your

population must have been last maintained under the new PRC-005

• Must start as soon as possible (or earlier) to ensure you make the minimum percentages by the target dates

• As an example, for relays which must be maintained every six years, 30% must have been last maintained under PRC-005-6 by April 1, 2017. Will 30% of your unmonitored relays be compliant then?

IMPORTANT – Start Implementing Later PRC-005 Versions Now


The Battery Table – Table 1-4f


Table 1-4f, Simplified

Table1-4f Exclusions for Protection System DC Supply

Maintenance Activities

Attributes Max MaintenanceInterval

Maintenance Activities

Station DC Supply with high and low voltage monitoring

No periodic verification of voltage is required

Any battery station DC supply with level monitoring and alarming

No periodic maintenance specified

No periodic inspection of electrolyte level is required

Any station Dc supply with DC ground monitoring

No inspection for grounds is required


Table 1-4f, Simplified

Table1-4f Maintenance Activities

Attributes Maintenance Activities

Station DC Supply with high and low voltage monitoring

No periodic verification of voltage is required

Any battery station DC supply with level monitoring and alarming

No periodic inspection of electrolyte level is required

Any station DC supply with DC ground monitoring

No inspection for grounds is required


• Requirement 5 in the PRC-005-2 and later standards obligates registered entities to: “Demonstrate efforts to correct identified unresolved maintenance

issues.”

• This requirement has no counterpart in PRC-005-1.1b, or the other legacy standards

• All registered entities are required to maintain components in accordance with either PRC-005-1.1b (and other legacy standards) or the PRC-005 (or later versions), not both

• Consequently, R5 applies only to those components being maintained under PRC-005-2 (or later versions)

PRC-005-6 R5 Applicability


• Ensure contractors follow the standard (if employed)• Create a spreadsheet and make sure all PS Components are

included (including those which affect the BES)• Identify all departments in your entity with PS maintenance

responsibilities (batteries, communication, relays, etc.). Then ensure they all understand the new rules and requirements

• Set up an implementation plan to ensure components are tested before mandatory transition date

Tips


• Identify and train maintenance personnel on any new tasks – to include record keeping on unresolved maintenance activities

• Set up record keeping system for R5 - unresolved maintenance activities

• Ensure that your record keeping system can support proving compliance with the new requirements. Specifically: Does your records system provide for documenting field performance of

the tasks NERC is now requiring? Do you have a method by which all groups performing maintenance can

produce field records reflecting performance of the required tasks.

• Obtain data (perhaps from the BA) needed to identify R/C that must be addressed in the program. Must test under the new rules before you can transition to the new standard

Tips


• Recognize that scope is dictated by “affects the reliability of the BES”

• Categorize the battery systems• Ensure battery maintenance personnel are aware of applicable

requirements – which table applies to which batteries on your system

• Focus on compliance with PRC-005-6. It’s all inclusive of the new PRC-005 series

• Retain the original field testing information for audit purposes and Protection System Maintenance Management

• Manage the transition to ensure compliant by target dates

Tips


• Entities thinking that a physical inspection by a station operator is proper testing

• Entity with newly commissioned equipment (or an entire brand new NERC Compliance Registry (NCR)) not having a maintenance program documented for the new equipment

• Some registered entities are classifying components as monitored without fully reviewing the monitoring characteristics in the PRC-005-2 tables. This causes the registered entities to extend the intervals beyond what they should be and miss the required testing and maintenance associated with unmonitored components

Examples of Violations


• Some registered entities are failing to require verification of correct relay settings as part of the maintenance/test requirements. This verification is specified in Table 1-1 for both unmonitored and monitored protective relays

• Some registered entities struggle to understand what it means to transition a component from one of the legacy standards to PRC-005-2. A component is considered transitioned to PRC-005-2 once it has been tested according to the maximum maintenance intervals and minimum maintenance activities listed in the PRC-005 Tables. A component has not been transitioned simply by designating in the registered entities’ records that a given component is going to be maintained under PRC-005-2 from that point forward

Examples of Violations


• Registered entities using component inventories (spreadsheets) to facilitate maintenance management and auditing

• Regions publishing guidance mandatory inventory spreadsheet formats to facilitate data gathering and review

• TRE Spreadsheets available at: PRC-005-6 Implementation Plan – Calendar View PRC-005-6 Implementation Plan – Requirements View

Good Practices

http://www.texasre.org/CPDL/PRC-005-6%20-%20Implementation%20Plan%20-%20Calendar.pdf

http://www.texasre.org/CPDL/PRC-005-6%20-%20Implementation%20Plan%20-%20Requirements.pdf


TRE Implementation Guide


TRE Implementation Plan


• Supplementary reference and FAQ, PRC-005-2 PS Maintenance, October 2012http://www.nerc.com/pa/Stand/Pages/PRC0052RI.aspx

(Note the above is 4.5 years old)• BES Definition Reference Document Version 2 April 2014

NERC WebsiteProgram Areas & Departments > Reliability Assessment and Performance Analysis > > Bulk Electric System (BES) Definition, Notification, and Exception Process > BES Definitions and Supporting Documents > Bulk Electric System Definition Reference Document Version 2.0 – April 2014

Relevant References

http://www.nerc.com/pa/Stand/Pages/PRC0052RI.aspx

http://www.nerc.com/PA/Pages/default.aspx

http://www.nerc.com/pa/RAPA/Pages/default.aspx


[email protected]

www.encari.com

Simplifying Process Development‐‐ How To Melt The Ice ‐‐

March 2016

www.encari.com 2

Encari, As Part Of The PowerSecure Family, Provides The Consulting Services That Generation And Transmission Utilities, Municipalities, And Cooperatives Need In Order To Attain And Maintain Compliance With The NERC CIP Reliability Standards

PowerSecure Provides Energy Technologies And Services To Electric Utilities And Their Large Industrial, Commercial, Institutional And Municipal Customers

www.encari.com

• Senior NERC CIP Compliance Consultant with Encari, after working with a variety of clients on NERC CIP compliance since 2006.

• Qualified Nuclear Operator in the U.S. Navy serving on a fast attack submarine.

• Worked in civilian nuclear power industry in regulatory compliance dealing with both NRC and DOE at several nuclear facilities.

• Director of Strategic Quality and Director of HR Information Systems for an international IT company.

• Managed the technical writing staff in preparing the license application submittal for DOE’s Yucca Mountain Project. NERC CIP activities include initial gap analysis, staff training, critical asset identification, developing policies and procedures to achieve compliance, audit preparation, and audit support.

• Captain, US Navy (retired)

JACK KUTZERSENIOR NERC CIP COMPLIANCE CONSULTANT

3

GREATTOBEPRESENTINGTODAY

www.encari.com

Objectives• Nostalgia• ‘Melt the ICE’• Procedures – the ‘Large Mosaic’• Simplifying Process Chart & Procedure Development• Procedure Lore• Before & After• Tools Discussion• 5 Key Success Factors

4

WHAT’SITALLABOUT?

Audience• New to Compliance – It all still sounds like ‘alphabet soup’• Doing compliance for 2 years or less – Somewhere in your ‘elephant meal’• Old Hand at Compliance – You remember CIP Version 1, or even 1200

www.encari.com 5

PROCESSCHART– CIRCA1980

PROCEDURE PROCEDUREI. Procedure – Revision ScopingII. Procedure ScopingIII. Procedure DevelopmentIV. Technical ReviewV. Procedure Comment PeriodVI. Procedure ApprovalVII. Approved Procedure Handling

www.encari.com 6

PROCESSCHART– CIRCA1980

www.encari.com 7

PROCESSCHARTINGTOOL– CIRCA1980

www.encari.com 8

MELTTHEICE!

Internal Controls Evaluation (ICE)

• Evaluate registered entity controls for identified risks and associated Reliability Standards and Requirements identified in an IRA

• Entity‐Level Controls: controls which are pervasive across an organization and include culture, values and ethics, governance, transparency and accountability

• Activity‐Level Controls: controls specific to a process or a function; may be manual or automated

• Identify important or key internal controls (such as business processes, practices, policies, and procedures)

• Level of Internal Control Implementation Table – “Documented processes” & “Evidence”

Source: ERO Enterprise Internal Control Evaluation Guide| October 2014

www.encari.com 9

CURRENTPROCEDURESTATUS?

You Don’t Need Procedures For Everything!

Every Single One Of Your Companies Have Procedures!

Procedures Minimize ‘Random Activity’

www.encari.com 10

THE‘LARGEMOSAIC’

Relationship Diagram“Big Boxes”

Process Hierarchy

Process ChartsBusiness Logical Software & Tool Independent

R‐01

P‐03‐01 P‐03‐02

P‐03‐04

Process P‐03

b

V‐01‐001

P‐03‐03

Process P‐02

BP‐02‐01

P‐02‐02 P‐02‐03

a

b

a

E‐01 P‐01‐01A

P‐01‐02aa

P‐01‐04

bb

B

Swimlane 1

Swimlane 2

Process P‐01 Swimlane 1

Swimlane 2

Swimlane 1

Organization

E‐01

R‐03

P‐01

P‐02

P‐03

P‐04

P‐05

E‐02

E‐03

R‐02

R‐01A B

B

V‐03‐00 2V‐03‐00 1

P‐05

V‐01‐001

C‐03‐00 2C‐03‐00 1

C‐01‐00 1

Implementing DocumentsHow To Use Software & Tools

a

E‐01 P‐01‐01A

P‐01‐02aa

P‐01‐04

bb

B

Swimlane 1

Swimlane 2

Process P‐01

Checklists Job Aids

www.encari.com 11

THE‘LARGEMOSAIC’

Relationship Diagram“Big Boxes”

Process Hierarchy

Process ChartsBusiness Logical Software & Tool Independent

R‐01

P‐03‐01 P‐03‐02

P‐03‐04

Process P‐03

b

V‐01‐001

P‐03‐03

Process P‐02

BP‐02‐01

P‐02‐02 P‐02‐03

a

b

a

E‐01 P‐01‐01A

P‐01‐02aa

P‐01‐04

bb

B

Swimlane 1

Swimlane 2

Process P‐01 Swimlane 1

Swimlane 2

Swimlane 1

Organization

E‐01

R‐03

P‐01

P‐02

P‐03

P‐04

P‐05

E‐02

E‐03

R‐02

R‐01A B

B

V‐03‐00 2V‐03‐00 1

P‐05

V‐01‐001

C‐03‐00 2C‐03‐00 1

C‐01‐00 1

Implementing DocumentsHow To Use Software & Tools

a

E‐01 P‐01‐01A

P‐01‐02aa

P‐01‐04

bb

B

Swimlane 1

Swimlane 2

Process P‐01

Checklists Job Aids

www.encari.com 12

PROCESSCHARTVS.PROCEDURE

What Does a Process Chart Accomplish• Management Review Tool• Easy To Evaluate Flow From Triggers To Results• Easy To ‘See’ Roles Of Stakeholders• Shows Where Control Points Could Be Established• Training Tool To Show Overview Of Procedure• Generally, Insufficient Detail To Perform ProcedureWhat Does a Procedure Accomplish• Provides Implementation Details To Perform Procedure• Establishes Specific Control Points• Specifies Records And Documentation To Demonstrate Procedure Completion• Lesson Learned – Don’t Put Process Charts In Procedures

www.encari.com 13

STEP1– “BIGBOX”PROCESS

Procedure Procedure Process

Change Control Process

Employee Hiring Process

www.encari.com 14

STEP2A– IDENTIFYSTAKEHOLDERS <Process Name>

Particip

ant 2

Particip

ant 1

Identify Individuals Or Organizational Units With ‘Roles’ In Procedure• These Become The ‘Swimlanes’

Types Of Roles:• Generic Roles

• Initiator, Employee, Contractor, Supervisor, etc.• Identify With Generic Role Name

• Organizational Units• Information Technology, Operations, Transmission Support, etc.• Identify With Organizational Unit Name

• Specific Individuals• Operations Manager, Plant Manager, Compliance Manager, etc.• Identify With Specific Individual’s Title

www.encari.com 15

STEP2B– ESTABLISHTHEPROCESSBOUNDARIES

Identify Procedure Triggers• Event Based Triggers – Something Happens – Unplanned Outage, Planned Outage,

Hardware Upgrade, Software Update, Predecessor Procedure Ends, etc.• Time Based Triggers – Daily, Weekly, Monthly, Quarterly, Annual, etc.

Identify Procedure Inputs• Information Necessary To Complete A Process, But Is Not Generated By The Procedure

• Hiring Procedure Requires Company Salary Structure• Inputs Are Not Required For A Process

Identify Procedure Outputs Or Results (Pick Your Terminology)• A Deliverable That Ends The Subject Process Flow• May Be An Internal Or External Deliverable• Typically Indicates Collection Or Storage Of Documentation Or Records

www.encari.com 16

STEP3 – IDENTIFYFLOWFROMTRIGGERSTORESULTS

Identify Process Activities• Data Transformation ‐ Source Data Manipulated To An Output Or Deliverable• Value Added – Design, Develop, Install, Replace, etc.• Review And Approve – Review Deliverable From A Predecessor Step

Identify ‘Mid‐Procedure Deliverables ’• Lines Between Process Activities Indicate A ‘Mid‐Procedure Deliverable’• Should ‘Name’ Deliverable Lines Crossing Swimlane Boundaries• Obtain Agreement For Both Sides Of The Deliverable Line

Identify Decisions• Branch On ‘Condition’ In Deliverable• Branch Based On ‘Review And Approval’ Activity• Show All Paths (Happy & Unhappy)

www.encari.com 17

WHATITLOOKSLIKE…

www.encari.com 18

PROCESSCHARTELEMENTS

Process Chart Element Description Typical Symbol1. Triggering Event 2. Input

1. Event that initiates the subject process flow ‐ external stimulus, an output from another process, or a timed‐based event.

2. Information that is necessary to perform or complete activities

1. Output 2. Result

1. A deliverables that ends the subject process flow. 2. Indicates the collection or storage of documentation or

records.

Process Activity Value‐added action or data transformation that is performed by a Stakeholder.

Deliverable Information or document that results from a process activity

Decision Conditional branching step based on the characteristics of a deliverable.

Swimlane (Stakeholder) ‐ An indication of some degree of participation in the process flow.

<Process Name>

Particip

ant 2

Particip

ant 1

www.encari.com 19

METHODOLOGY

Post Meeting Activities• Produce A ‘Clean’ Process Chart For Stakeholder Review• Depending On Review Comment Scope:

• Produce A Final Version Of The Process Chart, Or• Schedule A ‘Review Meeting’ For The Revised Process Chart

Facilitated Stakeholder Meeting• About 1 Hour Per Procedure• Limit To 3 Procedures Per Day, Or Less• Participants Need To Be Able To Make Decisions And Commitments

Use The ‘Final Process Chart’ To Write The Procedure• Write What the ‘Picture’ Shows• Add Details for Activities

www.encari.com 20

PROCEDURECONTENT

Title Page• Table Of Contents1. Purpose – What It Does2. Scope – Where It Applies, Or Doesn’t Apply3. [Overview – Summarizes Regulatory Requirement Implementation]4. Roles & Responsibilities – From Process Chart5. Procedure – From Process Chart6. Definitions 7. References – Internal And External To Organization8. Revision History9. Approval

www.encari.com 21

RULES(“MOREWHATYOU’DCALLGUIDELINES)

Process Chart ‘Rules’• Label Process Chart Elements – Including ‘Deliverable’ Lines Crossing

Boundaries• Process Chart Symbols Should Have 3 To 5 Words – Not Sentences• Process Chart Terminology Should Be ‘Verb‐Noun‐Qualifier’ • Process Chart ‘Flows’ From Left to Right And Top To Bottom• Change Swimlane Order To Get Better ‘Flow Picture’• Eliminate ‘Non‐actions’ – Notify, Email, And Similar (Process Activity Definition)• Process Charts Should Fit On Letter Size Paper• It’s Nice To Have A Large Format Printer• ‘White‐wall’ Rooms Are Nice To Have Available

www.encari.com

Generate – Report – Monthly SalesGenerate – Sales Report – Monthly

Generate – Monthly Report – SalesGenerate – Monthly Sales Report

22


VERB – NOUN – [QUALIFIER]

www.encari.com 23


Procedure Development ‘Rules’• Process Charts Are A Key Tool In Procedure Development• DO NOT Put Process Charts In The Procedure – Insufficient Detail Causes Problems• One Action Per Procedure Step

• ‘Review & Approve’ Is OK As ‘One Action’• Procedure Statements Are Directive, And Terse – It’s Not Literature• Procedure Statements Start With Action Verb, With Conditionals At The End• Procedure Statement Identify Role Performing Step• Don’t Forget ‘Unhappy’ Paths• Create A ‘New’ Process Chart From The Procedure – Does It Match The Initial Chart?• MEGA TIP – Read It Aloud

www.encari.com 24

LEVELOFDETAILINPROCEDURES

Establish An ‘Assumed Level Of Knowledge’• Too Much Detail…• Too Little Detail…• Just Right Detail…

Most Qualified Person Is Not Always The Best Procedure Writer

“The Least Qualified Person That WillUse The Procedure Unsupervised!”

www.encari.com 25

HAVEYOUHEARDTHIS…

Bad Procedure Programs• We have to do it ‘that way’ ‐We can’t do ‘that’ It’s in (not in) our procedure• You have to ask [insert old dog name here] to find out how to do ‘it’• You haven’t been here long enough to understand how we do ‘it’• We’ve always done it ‘this’ way ‐We’ve never done it ‘that’ way• I do ‘this’ when I get it, and send it to [someone] – I don’t know what they do with it• Why can’t [someone] give me what I need the ‘first time’• Is there something documented where I can find out how to do ‘it’?• How did this procedure get ‘stuff’ for me to do when I never saw it before now?

Procedure Roadblocks• I know how to do my job – why do I have to write it down?• I’m the only one that does this – why do I need a procedure?• Don’t you trust me?• Procedures are too ‘limiting’• What I do won’t work as a procedure ‐ I do different things, depending on the situation• I don’t have time to document what I do – I’m too busy doing it…

www.encari.com 26

GOODPROCEDURE PROGRAMS

Benefits• Results Are Consistent ‐ Products And Services• Right Information Comes In – Right Information Goes Out (Anti‐GIGO)• Short Learning Curve – New People Become Effective Team Members Faster• Balanced Workloads – Tasks Can Be Completed By Any Team Member• Personnel Gain Experience – Don’t Get Stuck Because Nobody Else Can Do A Task• Controls, Quality, Evidence, & Documentation Is Built In – NOT AN ‘ADD‐ON’!• Life Is Good & Everyone Lives Happily Ever After!Attributes• Key Activities & Regulatory Activities Are Documented With Procedures• Clear Procedure Hierarchy In Place• Changes And Updates To Procedures Are ‘Easy’• Team Talks About Activities By Referencing Procedures• ‘Event Analysis’ Includes Procedure Questions

www.encari.com 27

CHANGECONTROL‐ BEFORE

www.encari.com 28


www.encari.com 29


www.encari.com 30

CHANGECONTROL‐ AFTER

www.encari.com 31


www.encari.com 32


www.encari.com 33

PROCESSCHARTINGTOOLS

Stand Alone Process Charting Tool• Specific Process Charting Functionality – Makes Charts ‘Pretty’• A Reasonably Cost Effective Solution• Can Have Expandable Capabilities

IT Enterprise Architecture Tool• Typically Has A Process Charting Module Available• Generally Not A Good Process Charting Solution – Correct, But Not ‘Pretty’• May Be A Wide Price Variation• May Have ‘Access’ Issues Internally

Microsoft Visio• Most Companies Have It Available• Basic Functionality – Good Starting Point• Limited Expansion Of Capabilities• WARNING – Installing Visio Does NOT Make You A Process Charting Expert!

www.encari.com 34

PROCEDURETOOLS

MS Word• Readily available in most companies• Issues when developing ‘complex documents’

Adobe InDesign• Handles ‘complex documents’• Likely requires additional cost and approvals

Lotus Word Pro• If you have it, then use it

Bottom line ‐ generally ‐ use what your company has in house.

www.encari.com 35

EATTHEELEPHANT

Next Steps• Take Action!• Create A “Procedure Procedure”• Evaluate What You Have ‐ Critically• Pick A Priority

• Improve Existing ‘Stuff’• Fill In Gaps

Smaller Bites Are Easier To Digest• Don’t Try To Write A ‘Gigantuous’ Procedure• Break It Into Smaller ‘Complete Procedures’• If The Process Chart Is Readable On Letter Size Paper, The Procedure Is About The

Right Size• When You Have Some ‘Procedure Maturity’ You Can Combine The Smaller Bites

www.encari.com 36

5KEYSUCCESSFACTORS

Celebrate Victories

Get Professional Help

Limit Initial Scope To Most Painful Issue

Involve Stakeholders

Senior Manager Sponsorship

www.encari.com

Q A&

QUESTIONSANDANSWERS

37

www.encari.com

QUESTIONSANDANSWERS

38

[email protected]

469‐955‐5763

Jack Kutzer – Senior NERC CIP Consultant, Encari

[email protected]

205‐960‐8832

Wes Stewart – Vice President, Encari

Internal Controls

Presented by Donna Maskil-ThompsonSPP RE Workshop

03/15/2016

Property of KC Board of Public Utilities © - PUBLIC - 2016 1

Internal Controls

• The policies, procedures, practices and organizational structuresdesigned to provide reasonable assurance that business objectiveswill be achieved and undesired events will be prevented ordetected and corrected.

Reference - ISACA Glossary -(formerly known as Information SystemsAudit and Control Association


Internal Control Structure

The dynamic, integrated processes designed to provide reasonableassurance regarding the achievement of the following generalobjectives:

• Effectiveness and efficiency of operations

• Reliability of management

• Compliance with applicable laws, regulations and internal policies

Reference - ISACA Glossary -(formerly known as Information Systems Audit and Control Association)


Internal Control Structure

Management’s strategies for achieving these general objectives areaffected by the design and operation of the following components:

• Control environment

– Integrity

– Ethical values

– Competence – Knowledge and Aptitude

• Information Systems

• Control procedures

Reference - ISACA Glossary -(formerly known as Information Systems Audit and Control Association)


Internal Controls

• Help achieve operational goals

• Provide information on progress meeting goals

– Operating Effectively or are there Exceptions?

• Can only provide reasonable, not absolute, assurance

“An internal control cannot change an inherently poor manager into a goodone…”

- COSO (Committee of Sponsoring Organizations of the Treadway Commission) – Internal Controls


Where to Start?


Effective Risk Management + Audit = Compliance

Where to Start?

• What is the Risk?

• Perform Risk Assessments

– Perform SWOT Analysis

– Business Impact Analysis

– Review Incident Reports


SWOT Analysis

Strengths Weaknesses

Opportunities Threats


Internal

External

• How do you leveragestrengths to minimizeimpacts of threats?

• How do you mitigate orremediate weaknessesto avoid threats?

BPU Policy Framework

• Outlines standards and guidance

• References multiple Authoritative Sources

– National Institute of Standards and Technology (NIST)

– COSO (Committee of Sponsoring Organizations of the TreadwayCommission)

– ISACA (formerly known as Information Systems Audit and ControlAssociation)

• COBIT® 5 – Risk, Process, and Information

Not a “check the box” approach


Using RSAWs

• Yes, we know – Seriously, use them

– Maintain and update (quarterly)

• How are we meeting this requirement? (Self-Assessment)

• Have the SMEs changed?

• What are we missing?

• Identify Training Needs


Controls Assessment

IT General Controls Assessment Yes No Description of

Policy, Process

or Procedure

Program Change Controls – Change Management

1.Does BPU maintain written procedures for controlling program changes through IT management and

programming personnel?

2. Do program change authorization forms or screens prepared by the user (Change Request) include:

Authorizations by management before proposed program changes are made?

Testing program changes?

IT management and user personnel review and approval of testing methodology and test results?

3. Does BPU use library control software or other controls to manage source programs and object

programs, especially production programs?

4. Does BPU have procedures for emergency program changes (or program files)?


Think like an Auditor -


Manage and Measure your Program like an auditor would

Writing Control Objectives

• What is the objective of thiscontrol?

– Prevent

– Detect

– Correct

• How does it effectively mitigaterisk?

– SMART criteria


Monitoring & Controlling- Compliance

• Perform Quarterly Testing

• Identify and Correct Defects –SELF REPORT

• Perform Root Cause Analysis

• Continuous Improvement– DEMING (Plan, Do, Check, Act)

– DMAIC (Define, Measure, Analyze,Improve & Control)

– Kaizen “Change for the Better”


Leadership

Accountability

Identify Risk

Control Risk

ShareKnowledge

ManageChange

Questions?


References

ISACA® and COBIT Online® ,www.isaca.org

Committee of Sponsoring Organizations of the Treadway Commission, www.coso.org

National Institute of Standards and Technology (NIST), Special Publications,http://csrc.nist.gov/publications/PubsSPs.html

– NIST 800-12– NIST 800-14– NIST 800-16– NIST 800-34 (R1)– NIST 800-37– NIST 800-50– NIST 800-53 A (Assessment Guide)– NIST 800-53 (R4)– NIST 800-55– NIST 800-60– NIST-800-61– NIST 800-118– Cybersecurity Framework


Risk Assessment & Internal Controls ITC’s Implementation

2

Topics

• Risk Assessment Development

• Risk Assessment Implementation

• Overview of Internal Controls

• The Internal Controls Process

• ITC’s Internal Controls Program

• OATI Internal Control Module Overview

• OATI Internal Control Module Discussion

3

Internal Control Framework –Convergence of Compliance ProgramsKey compliance efforts integrated into the Internal Controls Framework:

NERC RAI white papers: Changing self-certification to focus on risk and internal controls Add controls from 2014 Audit Lessons Learned – internal survey Regional Entity self-reporting database – creation of self-logging NERC 13 questions and EIE – define program and demonstrate culture Creation of a Corrective Action Program including schedule of IC reviews (e.g. 3-yr Plan),

root cause analysis and lessons learned centrally managed to mitigate SV/AFI/etc.; Monitoring Metrics to Reliability Compliance Steering Committee; Self-report high risk IC

deficiencies

Internal Controls

Audit Lessons Learned

RAI: Change from Self

Certs to IC Reviews

RAI: Self-Reporting Database

(TBD)

13 Questions or NERC EIE

Corrective Action

Program

Monitoring Metrics &

Corp Goals(TBD)

4

NERC Reliability AssuranceInitiative (RAI) Program

“The IRA is a review of potential risks posed by an individual registered entity to the reliability of the bulk power system (BPS).”

NERC ERO Enterprise Inherent Risk Assessment Guide

5

Risk

What is risk?The possibility of an event occurring that will have an

adverse impact of the achievement of objectives (reliability of the Bulk Electric System).

How do we measure risk?Risk is measured in terms of likelihood and impact.

What is a risk assessment?The identification, evaluation, and estimation of the levels of risks involved in a situation, their comparison against benchmarks or standards, and determination of an acceptable level of risk.

6

Inherent Risk Assessment

Objective of a Risk Assessment Model• Identify and prioritize the most important or key areas (what

really matters)

Measure and prioritize risk exposures• The higher the risk exposure, the higher the priority

ITC’s Risk Assessment Model• Scores based on 11 key risk indicators that influence the

likelihood of the risk event and potential impact

• Risk score used to prioritize control reviews

• Full assessment every 3 years; Annual refresh

7

Key Risk Indicators

Key Risk Indicators• Routine vs. Non-Routine• Automation vs. Manual• Cross-Functional (Internal)• 3rd Party Interaction (External)• NERC High Risk Standards • Significance of Changes in Standard or Process• Key Personnel Turnover• NERC VRF• Reliability and/or Reputational Impact• Violation History• Automated Internal Controls

8

ITC’s Risk Assessment Model

How do we calculate the risk score?• Rate each of the risk factors on a scale of “1” to “5”.

o “1” indicating lower risk, and “5” indicating higher risk

• Weight each factor based upon significance of each factor.

• Multiply each factor by it’s risk weight to calculate an overall score.

• Rank each score from high to low

• Focus on the areas with the highest risk score (what really matters).

9

Inherent Risk Assessment

How/where will this information be used?

• Drive the implementation of future controls

• Strengthen specific compliance related processes

• Prioritize training and communication efforts

ITC 2012 Reliability Compliance Risk AssessmentRisk Indicators

Standard Reqmt FunctionsRoutine vs.

Non-RoutineAutomation vs.

Manual

Cross-Functional (internal)

3rd Party Interaction (external)

High Risk Standards

(NERC Tier 1,2,3)

Significance of Changes in Standard or

Process

Key Personnel Turnover

NERC VRF

Reliability and/or Reputational

ImpactViolation History

Automated Internal Controls

Overall Risk Score

CIP-005-3a 2 MISO, LBA, TOP, TO 4 3 4 5 5 4 3 5 5 4.6CIP-005-3a 4 MISO, LBA, TOP, TO 4 3 4 5 5 4 3 5 5 4.6CIP-007-3a 2 MISO, LBA, TOP, TO 4 4 3 4 5 5 3 5 5 4.6CIP-007-3a 3 MISO, LBA, TOP, TO 4 4 3 4 5 5 3 5 5 4.6CIP-009-3 4 MISO, LBA, TOP, TO 3 3 4 5 5 4 3 5 5 4.5CIP-009-3 5 MISO, LBA, TOP, TO 3 3 4 5 5 4 3 5 5 4.5CIP-003-3 1 MISO, LBA, TOP, TO 3 4 5 2 5 4 5 5 5 4.4CIP-003-3 5 MISO, LBA, TOP, TO 3 4 5 2 5 4 5 5 5 4.4CIP-003-3 6 MISO, LBA, TOP, TO 3 4 5 2 5 4 5 5 5 4.4CIP-007-3a 1 MISO, LBA, TOP, TO 4 4 3 4 5 4 3 4 5 4.4CIP-007-3a 5 MISO, LBA, TOP, TO 4 4 3 4 5 4 3 4 5 4.4CIP-007-3a 6 MISO, LBA, TOP, TO 4 4 3 4 5 4 3 4 5 4.4CIP-007-3a 8 MISO, LBA, TOP, TO 4 4 3 4 5 4 3 4 5 4.4CIP-009-3 2 MISO, LBA, TOP, TO 3 3 4 5 5 4 3 4 5 4.4PRC-005-1 2 TO 3 3 3 3 5 5 2 5 5 4.3MOD-004-1 6 BA,TP 5 5 3 5 5 5 5 3 3 4.3MOD-004-1 8 BA,TP 5 5 3 5 5 5 5 3 3 4.3MOD-004-1 9 BA,TP 5 5 3 5 5 5 5 3 3 4.3MOD-004-1 10 BA,TP 5 5 3 5 5 5 5 3 3 4.3MOD-004-1 11 BA,TP 5 5 3 5 5 5 5 3 3 4.3CIP-003-3 4 MISO, LBA, TOP, TO 3 4 4 2 5 4 5 5 5 4.3CIP-005-3a 1 MISO, LBA, TOP, TO 3 3 3 3 5 5 3 5 5 4.3CIP-005-3a 3 MISO, LBA, TOP, TO 4 3 3 3 5 4 3 5 5 4.3CIP-009-3 1 MISO, LBA, TOP, TO 3 3 4 5 4 5 3 5 5 4.3CIP-009-3 3 MISO, LBA, TOP, TO 3 3 4 5 4 5 3 5 5 4.3FAC-003-1 1 TO 3 4 4 2 5 2 5 5 5 4.1EOP-005-1 6 MISO,LBA,TOP 5 5 1 3 5 1 3 5 5 4.1FAC-003-1 2 TO 2 4 5 2 5 2 5 5 5 4.1PER-002-0 3 MISO,LBA,TOP 2 3 4 3 5 3 3 5 5 4.1FAC-009-1 1 TO 3 5 4 4 5 4 3 3 4 4.1EOP-005-1 8 MISO,LBA,TOP 3 5 3 5 4 3 1 5 5 4.1CIP-004-3a 4 MISO, LBA, TOP, TO 3 3 2 3 5 4 3 4 5 4.1CIP-007-3a 4 MISO, LBA, TOP, TO 4 4 3 4 4 5 3 5 4 4.1CIP-008-3 1 MISO, LBA, TOP, TO 4 3 4 5 3 4 3 5 5 4.1EOP-008-0 1 MISO,LBA,TOP 4 5 3 2 5 2 1 5 5 4.0COM-002-2 2 MISO,LBA,TOP 3 5 1 5 5 3 1 3 5 4.0PRC-001-1 4 MISO,LBA,TOP 2 5 4 5 4 2 2 5 5 4.0CIP-005-3a 5 MISO, LBA, TOP, TO 4 3 3 4 4 3 3 4 5 4.0CIP-006-3c 1 MISO, LBA, TOP, TO 4 4 4 4 4 4 4 4 4 4.0CIP-006-3c 4 MISO, LBA, TOP, TO 4 3 4 4 4 4 4 4 4 4.0

10

2016 ITC Inherent Risk Assessment

Risk Score 3.7 - 4.8Risk Score 3.1 - 3.6Risk Score 1.0 - 2.4 Risk Score 2.5 - 3.0

IRO-004-2 R1

PRC-001-1 R3, 4, 5, 6

NUC-001-2.1R2, 3, 4, 6, 9 FAC-003-3

R2

PRC-005-1.1b R3

CIP-008-5 R1, 2, 3

CIP-007-6R1, 2, 3, 4, 5

EOP-005-2R1, 6, 7, 8

CIP-011-1 R1

EOP-008-0R1 COM-002-2 R2

(66 R’s)(57 R’s)

(61 R’s)(43 R’s)

Risk priority for each requirement will be reassessed every 3 years, interim assessment every year.

TOP-004-2 R1,2,3,4

TOP-002-2.1bR11,16, 17

TOP-001-1a R2,3,5,7,8

TOP-007-0R3

FAC-008-3 R3, 6, 8

PRC-023-3 R1

CIP-006-6 R1, 2

EOP-003-2R1, 2, 5, 6, 8

COM-001-1.1 R1, 2

EOP-005-2 R2, 4, 9, 11,13

EOP-001-2.1b R3, 4

TOP-002-2.1b R1, 2, 4, 5,6, 10, 19

TOP-004-2 R5

IRO-001-1.1 R8

TPL-001-4R1, 4, 7, 8

PRC-001-1 R1, 2

IRO-005-3.1a R9,10

PRC-008-0 R2

NUC-001-2 R8

PRC-005-1.1b R5

FAC-002-2 R1, 3, 4

TOP-001-1a R6

IRO-010-1a R3FAC-003-3 R5

TOP-007-0 R1,2

CIP-003-6 R1TOP-008-1 R1, 2, 3

MOD-027-1 R5

PER-005-1 R3

EOP-008-1 R6, 7, 8

IRO-005-3.1a R5

TOP-006-2R3, 6, 7

PRC-017 R1, 2

EOP-004-2 R3

PRC-016-1 R1, 2

FAC-014-2 R5

PRC-015-1b R1, 2, 3

TOP-003-1 R1, 2, 3

CIP-004-6 R1

TOP-004-2 R6

TPL-001-4R2, 5, 6

FAC-014-2 R2

EOP-005-2R3, 5, 10, 12

MOD-018-0 R1MOD-019-0.1R1

COM-002-2 R1

EOP-001-2.1b R2, 5

CIP-003-6 R2,3,4

PRC-017-0 R2

EOP-004-2 R1, 2

CIP-006-6 R3

TOP-005-2a R1, 2

FAC-003-3 R3, 4, 6, 7

PRC-008-0 R1

PRC-023-3 R2, 3, 4, 5

EOP-003-2 R3, 4

PER-001-0.2R1

CIP-014-1 R3, 4

MOD-027-1 R1

VAR-001-4 R1,5

VAR-001-4 R2, 3, 4, 6

MOD-018-0 R2

BAL-005-0.2b R1, 12, 13, 15

MOD-020-0 R1

BAL-006-2 R3, 4

EOP-008-1 R2, 5

COM-001-1.1R4

TOP-001-1a R1

EOP-001-2.1b R6

MOD-021-1 R1, R2, R3

MOD-032-1R1, 2, 3

FAC-002-2 R2

TOP-006-2 R1, 2, 4, 5

PRC-006-1 R10

EOP-010-1 R3

PER-003-1R2

TOP-002-2.1b R18 PRC-004-2.1a

R3

MOD-012-0R1, 2

MOD-010-0 R1, 2

FAC-001-2 R1, 3

EOP-003-2 R7

CIP-011-2 R2

TOP-008-1 R4

PRC-018-1 R1, 2, 3,4, 5, 6

PER-005-1 R1,2

PRC-004-2.1aR1

CIP-010-1R1, 2, 3

CIP-005-5 R1, 2

CIP-009-6 R1, 2

CIP-014-1R1, 2, 4, 5, 6

CIP-004-6R2, 3, 4, 5

CIP-002-5.1 R1, 2

CIP-009-6 R3

COM-001-1.1R3, 5

EOP-008-1 R4

PRC-016-1R3

TPL-001-4 R3

Last Update Feb 5, 2016

11

NERC Reliability AssuranceInitiative (RAI) Program

“As described in the ERO Enterprise Internal Control Evaluation Guide (ICE Guide),3 the ICE may inform whether a registered entity has implemented effective internal controls that provide reasonable assurance of compliance with Reliability Standards associated with areas of risk identified through the IRA.”

NERC Guidance Document: “The Application of Risk-based Compliance Monitoring and Enforcement Program Concepts to CIP Version 5”

12

Internal Controls Framework

People

Functional Processes

Information Systems/Technology

ID and Assess Risks; Establish/Review

Controls

Internal Control Testing and Assurance Review;

Risk Response

Remediation & AFI

Monitoring , Metrics & Reporting

13

Controls

What is a control?A point where you create evidence of compliance

An action [taken by you, me, management, the board of directors, and / or other parties] to manage risk and increase the likelihood that established objectives and goals will be achieved.

Controls should be designed to bring about appropriate responses to risks. In other words, controls help to reduce or mitigate risk.

Controls should address the root cause of a risk event, not the symptom(s).

14

INTERNAL CONTROL CYCLE

Continuous Improvement

15

INTERNAL CONTROL TYPES

Internal Controls should be designed to:

• Prevent undesired outcomes

• Detect deviations in performance

• Correct broken processes

Internal Controls are also of two varieties• Automated – preferred over manual

• Manual – should have additional controls, cannot verify source of data

16

INTERNAL CONTROL EXAMPLES

Preventive Controls

• Policies and Procedures• Training and Awareness• Three-Part Communication• Forward Studies and Day ahead studies• Configuration Documentation• ID badges and door locks• Asset Inventory• Annual Plans (Vegetation Management, SRP, Security)• Operating guides• Defined testing and/or maintenance program

17


Detective Controls

• Review of logged activity for Control Room• Review of phone logs for three-part communication• Review of system access logs• Management Review• Self Certifications and Audits• Activity and Exception Reports

18


Automated Controls

• An automated control will prevent improper activities from occurring

• AdvantagesNo manual interventionReliable Time-stamp Activity is repeatable

Programmed alarms in a system like TMSSystem generated logsPassword Controls over access into a system

19


Manual Controls

Manual controls can often be circumvented

Manual controls are often performed after the fact

• Often time developed in a spreadsheet• Some type of control that is handwritten

20


For an Internal Control to be effective the following should be present

• The control activity should be assigned to a specific function/individual

• The control activity must be executed in a defined time period (daily, weekly, monthly, yearly)

• The control activity should be repeatable

21

Internal Control Development

Document Controls

Design, Test and Evaluate

Implement

Test Design

Test Effectiveness

Identify and Correct

Deficiencies

Review and Improve Design

22

Internal Control Monitoring

Benefits of monitoring the effectiveness of Internal Controls:

• Ensures that there exists a sustainable and repeatable process.

• Identifies potential improvements to process efficiencies and internal control value.

• Provides timely information for improved assessment and management of risk.

• Improves the overall value of internal controls towards compliance efforts as they relate to the reliability of the BES.

• Ensures that there has been no degradation of the controls over time.

• Identification and correction of control deviations and failures.

• Elimination of unnecessary or inefficient controls.

23

INTERNAL CONTROL PROGRAM

Detective Controls

• Review of logged Activity• Training• Three-Part Communication• Forward Studies• Day-ahead Studies

24

ITC Internal Control Program

Tasks Completed:• Conducted initial risk assessment

• Developed Heat Map based on results of risk assessment

• Determined controls to target in initial roll out

• Met with SOs and SMEs to review process and document controls

• Developed workflow for Internal Control process

• Developed Use Cases for loading into OATI Internal Controls Module

• Loaded controls into OATI Internal Controls Module

• Conducted internal testing to validate workflow

• Developed Internal Controls schedule

• Completed Initial Pilot

25

ITC Internal Control Calendar

An Internal Controls calendar has been developed based on:

• Timing of Process/Event

• Frequency of controls

• Relationship to timing of reviews in the Compliance Monitoring Calendar

26

ITC Internal Control Workflow

Following is an example of a typical OATI procedure work flow for Internal Controls. There will generally be 6 steps.

(1) Initial OATI procedure to notify SME to kick-off control activity (e.g., procedure, review, assessment, etc.) and attach/load evidence

(2) Std. Owner approval of evidence sample. (recursive)

(3) If evidence/sample is not approved, send back to SME for new or additional example. (recursive)

(4) Std. Owner approves control evidence review without further action.

(5) Std. Owner approves control evidence review but Corrective Actions are needed. Trigger CA procedure.

Rejected

Resubmit

OR

Approved

Clean Outcome

CA Needed

(6) Control evidence provided to Reliability Assurance for review

End Process

27

Internal Control Execution

• The Internal Control workflow will be initiated by a notification to the Subject Matter Expert (SME) for evidence

• The notification may be based on the calendar, i.e. first day of the quarter, something that is time based

• The notification may be based on the completion of another control procedure, something that is process based

• The SME will load requested evidence into OATI and mark complete

28

Internal Control Execution

• Once the evidence is loaded by the SME it will trigger a review process by the designated Standard Owner (SO)

• The SO will review the evidence and either:• Accept the evidence provided• Request additional evidence from the SME• Initiate a corrective action if the evidence indicates a potential

issue

• Controls in which evidence was Accepted or requiring Corrective Action will be sent to Reliability Assurance for review

• Reliability Assurance will review evidence of Control and complete the workflow

29

OATI – Internal Control Module

• OATI’s Internal Control (IC) Module was developed in response to NERC’s Reliability Assurance Initiative (RAI)

• ITC is one of the Companies that had worked with OATI in the development of the IC module and actively participated in the Beta testing process and Acceptance Testing of the module.

• ITC has worked closely with OATI in the loading of identified controls to the production site

30


• The IC Module is a flexible workflow tool

• The IC module will allow us to record and track controls as they relate to Reliability Requirements

• The IC Module will allow us to show we have controls in place and that we are following these controls

• Reports can be generated from Summary pages

• Future reports will be developed as needs are identified by the User Community

31


OATI webCompliance Main Dashboard

32


Internal Controls Dashboard

33


Task Summary

34


Task Screen

35


Attachment Screen

36


Graph Workflow Display

37


Task Screen – Status Change

38

ITC 2015-2016 Internal Control Roadmap

1

2

QTR 3 & 4 2016• Document medium priority IC’s in OATI• Develop Metrics & Compliance dashboard• Evaluate medium priority ICs in OATI

QTR 1 2017 • Evaluate IC program for effectiveness• Make adjustments as needed

5

QTR 1 & 2 2015• Completed Full-scale Inherent Risk Assessment• Documented high priority IC’s in OATI (Control Monitoring

System)• Evaluated effectiveness of Internal Controls • Conducted SME and SO training on OATI

QTR 3 & 4 2015• Documented formal Inherent Risk Assessment Procedure• Documented additional high priority IC’s in OATI• Performed Internal Control evaluations of completed Controls• Update Compliance Program Manual to include Internal Controls

QTR 1 & 2 2016• Standardize IC Evaluation procedure• Evaluate and refine IC based on IC reviews• Update Inherent Risk Assessment• Develop Reliability Compliance Steering

Committee reporting

3

39

Internal Controls

Questions?

Mitigation OverviewJenny AndersonCompliance [email protected]

1

BREAK-FIX IS OUT, RISK ASSURANCE IS IN

Introduction

2

VIOLATION ANALYSIS FOR MITIGATION

Section 1

3

Scope Analysis

• Determine the full scope of the violation

• Identify all non-compliance– Audit findings may be limited to an individual issue

– Document the scoping activity

– Explain full extent of violation scope

• Include time period (performance period)

4

Determining Scope (Audit finding)• Violation - Issue of non-compliance

“The audit team determined an instance of non-compliance where a patch was not applied to a CCA…”

• Full scope – Identify all non-compliance “A review of all patching for the period MMDDYY to MMDDYY

was completed, and two additional patches were not applied for a total of 3 instances. These were:

1) Patch KB2013014322, released MMDDYY, assessed MMDDYY, but not applied to the Data Historian

2) Antivirus signature files released MMDDYY not applied

3) …”

5

Determining Scope (Self-report)

• Violation - Issue of non-compliance “On January 7, 2016, JPC identified a violation with CIP-006-3c

R1… in review of the daily visitor access logs …noted that egress time was not recorded…”

• Full scope – Identify all non-compliance– Review of all possible instances

“A review of all visitor access logs for the period from MMDDYY until MMDDYY was completed, and three individual instances of incomplete logs were identified. These were:

1) MMDDYY – Egress time not logged for a contractor

2) MMDDYY – Last name not recorded for a visitor

3) …”6

Root Cause Analysis

• Determine root cause(s) of the noncompliance

• Originating cause for non-compliance

• Ask “Why did the noncompliance occur?” “The root cause of the instance(s) of non-compliance was lack

of procedural control to ensure adherence...”

7

Violation Summary

• A violation has 3 parts– Immediate noncompliance

– Scoping activity and additional noncompliance (if any)

– Root cause analysis and root cause(s)

8

MITIGATIONSection 2

9

Determine Mitigation Approach

• Mitigating Activities– Mitigation complete

– Statement of mitigating activities

– Evidence

– Certification of Completion

• Mitigation Plan

– Multi-step implementation

– Future activity

10

MITIGATING ACTIVITIES

11

Mitigating Activities (Self-Report or Certification)

• Self-Report Detail– Description of Mitigating Activities and Preventative

Measure (include scope and root cause)

– Date Mitigating Activities Completed

12

Mitigating Activities Evidence (Self-Report or Certification)

• Submit via EFT, e-mail, or webCDMS

• Certification of Completed Mitigating Activities

• In webCDMS, submit in the Self Report Detail Entity Documents

13

Mitigating Activities (Audit or Spot Check)

• webCDMS does not allow for Mitigating Activities for audit or spot check findings

• Submit via e-mail or the EFT server– Description of mitigating activities and preventative

measures, including scope and root cause

– Evidence to support repair of noncompliance, root cause, and controls

– Certification of Completed Mitigating Activities

• No Mitigation Plan required if Mitigating Activities are acceptable

14

MITIGATION PLAN

15

Documenting Violation – Instance

16

Documenting Violation – Root Cause

• Identify root cause in Violation description of the Plan

• Root cause should be mitigated and controls identified

17

ROOT CAUSE

Documenting Scope – All Instances

18

SCOPE• All instances

during audit period

Mitigation - Plan Details

• Summary of all activity completed and planned– Corrective action for all issues of noncompliance

– Corrective action for the root cause

– Corrective, detective and/or preventive controls to address recurrence

19

Controls

• Mitigation Plan must include one or more controls to– Correct – repair future instances

– Detect – monitor and alert of occurrence

– Prevent – avert further occurrence

20

Mitigation – Milestones

• Future activity only (no past due milestones)

21

Mitigation – Completion Date

• Date all plan activities will be completed

• Not always consistent with violation end date

• Generally date of the last milestone activity

22

Mitigation – Reliability Risk

• Risk(s) – Associated with the Violation and Standard

– To Registered Entity and BPS

• Impact(s)

• Mitigating or Compensating measures

23

Mitigation – Reliability Risk Prevention

• Prevention– Controls (corrective, detective, preventive)

24

Mitigation Plan Summary

• Mitigation Plan should address the 3 parts of a violation:– Immediate noncompliance

– Additional instance(s) of noncompliance (if any)

– Root cause

• Must include controls for correcting, detecting and/or preventing recurrence

25

MITIGATION COMPLETIONSection 3

26

Mitigation Evidence

• Evidence must be provided prior to or with Certification of Completion– Correct all noncompliance

– Repair of root cause

– Implementation of controls

• webCDMS - Upload to Mitigation Plan

• EFT Server

27

Mitigation Plan Evidence – Entity Documents

• CDMS - Upload to Mitigation Plan Entity Documents

28

Mitigation Evidence – EFT Server

• EFT Server– Secure, encrypted electronic file transfer

29

Mitigation Plan – Certification of Completion

• Complete and submit via CDMS– Attached “paper copy” not required

– Completion date should be consistent with proposed plan completion date

30

MITIGATION - RETURNING TO COMPLIANCE

Section 4

31

Returning to Compliance

• End the violation as soon as possible

• Complete other mitigation after ending the violation and returning to compliance

• Violation duration is a direct input to the Penalty Calculation Tool, and delaying the violation end could increase a monetary penalty

32

Violation End Date

• Duration of noncompliance is from the first instance to correction of noncompliance

• Violation end date is the date noncompliance was corrected

• Must be supported by evidence

33

In Summary

• Mitigation is just a piece of a Mitigation Plan

34

References

1. ERO Mitigation Guide , April 2014

2. Cause Analysis Methods for NERC, Regional Entities, and Registered Entities, September 2011

35

http://www.nerc.com/pa/comp/Reliability%20Assurance%20Initiative/ERO%20Mitigation%20Plan%20Guide%20(April%202014).pdf

https://www.wecc.biz/Administrative/2014%20HPWG%20Workshop%202%20Cause%20Analysis%20Methods%20for%20NERC,%20Regional%20Entities,%20and%20Registered%20Entities.pdf

36

Jenny AndersonCompliance Enforcement [email protected]

New Misoperations Reporting Process Thomas Teafatiller

Senior Compliance Engineer - O&P

March 15, 2016 SPP RE workshop

1

2

Misoperations Information Data Analysis System (MIDAS)

• ERO Enterprise system for collecting Misoperations data outlined in 1600 Data Request approved by FERC

• Starting July 1, 2016, Misoperation data will be collected by and maintained at NERC

• System testing to start this week with submission of Misoperations data from three Registered Entities selected from each region

Section 1600 Template Entry (Spreadsheet)

3

Collection of Protection System Operations

Section 1600 Template Entry (Spreadsheet)

4

Collection of Protection System Misoperations

Demo Submission Website

5

Demo Submission Website

6

Once the Entity’s NCR number is entered and Resubmittal Only question is checked as “NO”, the webpage will expand to ask the following questions:

Works as an Attestation for Entities if there are no Protection System Operations or Misoperations

Misoperations Collection Process Flow

7

Submission Site

•Data Submitter receives Submission Site Keycode (ensures security when submitting data)

• Submits excel spreadsheet via Submission website

• Indicates if submission is a resubmittal only

•Can indicate they have no Misops or Protection System Operations (to report

•Upon submittal, Submission Record with spreadsheet attached will be created in xRM

xRM Staging Area

•ERO Enterprise can:

•Open xls attachment from Submission Record, review/modify, then re-attach to Submission Record

•Process Submission Record, which extracts information from xls and creates Misoperation and PS Operation Records in xRM

•View Misoperation & PS Operation records

• See audit trail (i.e. who and when data was submitted)

xRM Records

• RE can edit misop/op records in xRM within its region

• NERC can make edits to all data

• Reporting, exporting, querying can be performed on records

• System will be capable of providing FERC with data (US) if required

xRM Portal

• Data Submitter can:

• Submit excel spreadsheet (bulk upload) or add entry manually

• Make edit when portal is available

Future Release

Submission Site Activity Diagram

8

Misoperations Resubmittals and Reminders • Updates to Misoperations will be submitted on same

Misoperations spreadsheet – Misoperation ID is created by the spreadsheet by

concatenating six fields

– Misoperation ID must be the same each time the Misoperation is updated or the system will create a new Misoperation

– Misoperations should be updated until the Corrective Action Plan is complete or a declaration is made that no cause for the Misoperation can be found

• MIDAS will send email reminders out much like webCDMS to notify Entities of upcoming deadlines

9

Misoperations Spreadsheet and Training

• NERC 1600 Misoperation Spreadsheet • NERC training webinar on data submission in MIDAS

– June 14 at 1 PM CST

– All eight Regional Entity representatives will be available to answer questions

– Reminder in SPP RE newsletter for webinar and training videos

10

http://www.nerc.com/pa/RAPA/Pages/Misoperations.aspx

Misoperations Training Videos

• NERC MIDAS training videos

• Multiple short videos will posted rather than one long video so users can watch specific parts

11

https://vimeopro.com/nerclearning/midas-video-library

MIDAS Training Videos

12

https://vimeopro.com/nerclearning/midas-video-library

Misoperations Summary

• NERC will take over collection of Misoperation data starting July 1, 2016 with 2Q 2016

• NERC training webinar June 14, 2016 at 1 PM

• Misoperations will be collected by submitting Misoperation spreadsheet via NERC hosted website

• xRM Portal will be used to collect Misoperations data in the future – Similar to webCDMS RAPA module

– Will allow single or bulk upload(s) and editing of past Misoperations

13

Registration Process Changes Greg Sorenson

Senior Compliance Engineer - O&P


14

Outline

• Risk-Based Registration

• Elimination of functions

• Change in thresholds

• Bulk Electric System (BES) Definition Change and Exception Requests

• NERC-led review panel and materiality

15

Risk-Based Registration

• Focus on Registered Entities that pose higher risks to Bulk Electric System

• Allows for some flexibility in gray areas

• Burden of evidence is on Registered Entity for exclusion

• Burden of evidence is on the Regional Entity or Transmission Operator/Reliability Coordinator (TOP/RC) for inclusion

• Removed functions that were not essential for reliability

16

Risk-Based Registration • March 19, 2015 FERC Order

– Removed Purchasing Selling Entities and Interchange Authorities

– Generally business functions, covered by NAESB standards

– Low history of reliability violations

– FERC has Market Monitoring arm

– Raised Distribution Provider (DP) threshold from 25 MW to 75 MW

– Introduced UFLS-only DP Removes burden of other standards including CIP from small

DPs

17

Risk-Based Registration

• October 15, 2015 FERC Order – Removed Load Serving Entities as a registered function

– Relies on everyone to continue to follow “Good Utility Practice” (i.e. provide TOPs and RCs what they need to safely operate the system)

• September 19, 2013 FERC Order – Many facilities connecting generation to the

transmission system are “generation Interconnection facilities”

– Many Generator Owner/Transmission Owners (TO) were deactivated as TOs

18

Revised BES Definition

• July 1, 2014 effective date

• Revised and clarified definition – Reactive resources, dispersed power producing

resources, transformers more consistent

• Newly identified elements must be fully compliant on July 1, 2016

• BESnet tool: besnet.eroenterprise.com

• Allows for Exception Request if element meets the BES definition, but is not material to reliability – Facts and circumstances specific

– ERO Exception Request Evaluation Guideline 19

https://identity.eroenterprise.com/account/signin?ReturnUrl=/issue/oauth2/authorize?client_id%3dexceptionsweb%26scope%3durn:ExceptionsWeb%26redirect_uri%3dhttps://besnet.eroenterprise.com/%26response_type%3dcode&client_id=exceptionsweb&scope=urn:ExceptionsWeb&redirect_uri=https://besnet.eroenterprise.com/&response_type=code

http://www.nerc.com/pa/RAPA/BES%20DL/BES%20Exception%20Evaluation%20Guideline%202-4-14%20REMG%20App.pdf

NERC-led review panel and materiality

• Registered Entities that meet registration threshold, but are not material, may request a NERC-led review panel

• Panel has final say

• “One switch TO”, “One breaker TOP”, critical 18 MW generator, etc.

• Facts and circumstances specific

• Elements definitely part of the BES, however, registering the entity provides little to no reliability benefit

20

Registration Reviews

• Registered Entities may ask SPP RE to review their registrations

• Provide information re: why you no longer meet the registration threshold

• Asset sales: Provide agreement of sale – Buyer must register before seller will be removed

– Effective date matches sale date

21

Links

• Statement of Registry Criteria

• BES Definition Reference Document

• BES Exception Request Evaluation Guideline

• Risk-Based Registration Implementation Guidance (covers NERC-led panel and materiality)

22

http://www.nerc.com/FilingsOrders/us/RuleOfProcedureDL/Appendix_5B_RegistrationCriteria_20151015.pdf

http://www.nerc.com/pa/RAPA/BES%20DL/bes_phase2_reference_document_20140325_final_clean.pdf

http://www.nerc.com/pa/RAPA/BES%20DL/BES%20Exception%20Evaluation%20Guideline%202-4-14%20REMG%20App.pdf

http://www.nerc.com/pa/comp/Registration%20and%20Certification%20DL/RBR%20Implementation%2020151217_final.pdf

23

Thomas Teafatiller Greg Sorenson Senior Compliance Engineer – O&P Senior Compliance Engineer - O&P 501.688.2514 501.688.1713 [email protected] [email protected]

March 16, 2016

1

CIP Update

Steven KellerLead Compliance Specialist - [email protected]

Discussion Topics• FERC Actions

– CIP V5 Revisions Order 822

– FERC extension of time

– FERC-led audits

• CIP-002-5 / CIP-014-2 Self-Certification– SPP’s data collection requirement

• SPP RE audit approach

• Outreach

2

FERC Update – CIP V5 Revisions*• FERC Approved CIP V5 Revisions on 1/21/16 (Order

822)– Removed Identify, Assess, and Correct language

– Clarified protections for Low Impact BES Cyber Systems

– New Glossary definitions

– Accepted NERC’s discussion of the term “communication networks” in lieu of a new Glossary definition

– Approved the Implementation Plan, violation risk factors, and violation severity levels

3

* Some are calling this set of revisions “V6”, but the official term is “V5 Revisions”

FERC Update – CIP V5 Revisions

• Order 822 directed:– Develop risk-based modifications to CIP Standards to

address FERC’s concerns regarding: Protection for transient electronic devices used with Low

Impact BES Cyber Systems

Protections for communication links and data communicated between Bulk Electric System Control Centers

– Modify the definition of Low Impact External Routable Connectivity (LERC) consistent with the related discussion found in the Guidelines and Technical Basis section of CIP-003-6

4

FERC Update – CIP V5 Revisions

• Order 822:– Directed NERC to complete and submit by 4/1/17, a

study of the efficacy of the remote access protections afforded by CIP V5

– Deferred further action on Supply Chain Management until after the technical conference held 1/28/16

5

FERC Update – Extension of Time

• Order 822 effective on 7/1/16

• On 2/25/16, FERC issued Docket RM-15-14-000– Granted an extension of time for V5 from 4/1/16 to

7/1/16 to align with Order 822

– SPP RE CIP audit schedule has been changed to reflect new effective date

6

V5 Transition Advisory Group Transfer Document

• Standard Drafting Team (SDT) should consider:– Definition of Cyber Asset and clarify the intent of

“programmable”

– Definition of “BES Cyber Asset” Definition

Term “adverse” in “adverse”

Clarify the double impact criteria

7


• SDT should consider concepts and requirements concerning Electronic Security Perimeters (ESP), External Routable Connectivity (ERC), and Interactive Remote Access (IRA) Clarify the 4.2.3.2 exemption phrase “between discrete

Electronic Security Perimeters.”

Use of the term “associated” and defining the relationship of ERC if no ESP exists

Direct vs. indirect communication (connection to non-BES Cyber System, e.g., Fileserver)

Dial-Up connectivity for IRA

8


• Transmission Owners performing the functional obligations of a Transmission Provider– Clarify applicability of requirements

• Definition of a Control Center– Clarification of “control” (e.g., remote start button in

generation plant)

– Attachment 1 criteria “performing the functional obligations of”

9


• V5 does not address virtualization– Virtual server

– Virtual network

– Virtual storage

• SDT should consider revisions to CIP-005 and definitions of Cyber Asset and Electronic Access Point that:– Make clear the permitted architecture

– Address the security risks of network, server and storage virtualization technologies

10

CIP-002-5.1 / CIP-014-2 Self-Certification

• CIP-002-5.1 Self-Certification with data collection (2/1/16 – 7/15/16) *Deadline Updated– NERC-standardized Excel workbook will be used for data

collection May use EFT Server or data submission mechanism in

webCDMS for data collection

• CIP-014 is a more traditional Self-Certification, using the Self-Certification tool (3/15/16 – 5/2/16)– If you respond “yes” to certain questions, we will

contact you to get additional detail about the identified substations

11

CIP V5 / V6 Audits

• Audit Period– Audit period will start no earlier than 7/1/16

– Audit evidence collection will be for the version of the Standards in effect at the time

– If non-compliance is found for a Requirement/Part with a mostly compatible V3 predecessor Requirement, the audit team will look at compliance with prior versions to determine the start of the violation

• Reliability Gaps– Will identify gaps in the Standards as recommendations

– Will be discussed separately in the audit report12

Tell Your Story

• “See Spot Run” vs. “War and Peace”

• V5 requirements are performance or outcome-based

• Many ways to achieve the desired outcome

• We need you to explain how you achieved the outcome

• Auditors must be able to understand what you did and why you believe that what you did meets expectations

13

• “Show your work”

• We need to know that you got the “right answer” from a correct, repeatable process (not stumbling into the right answer)

• Try to anticipate the auditor's questions and answer them in your narrative

• Find someone with reasonable background knowledge and have that person look at your documentation/ evidence. Do they understand your story and evidence?

14

Tell Your Story

NERC Standardized Request for Information

• Current version of evidence request and user guide posted on NERC web site– Developed in response to industry concerns about

Regional audit consistency

– Developed against CIP V5 Revisions

– Seeking industry review and comment

15

http://www.nerc.com/pa/CI/Documents/CIP%20Version%205%20Evidence%20Request_User%20Guide.zip

NERC Standardized Request for Information

16

SPP RE / NERC Outreach

• SPP RE Outreach– Focus is shifting to Low Impact

Combination of site visits, webinars, CIP Workshop, and phone/email Q&A support

• NERC Outreach– Low Impact focused Small Group Advisory Sessions

August 15 – 17 (location TBD)

October 6-7 -or- October 25-27 (Atlanta)

November 15-17 (location TBD)

17

Helpful Resources

• NERC V5 Page

• SPP RE CIP V5 Guidance Page– FERC CIP Version 5 Filings

– SPP RE presentations, webinars, and videos

18

http://www.nerc.com/pa/CI/Pages/Transition-Program.aspx


http://www.spp.org/regional-entity/cip-v5/

SPP RE CIP Team• Kevin Perry, Director of Critical Infrastructure Protection

(501) 614-3251

• Shon Austin, Lead Compliance Specialist-CIP(501) 614-3273

• Steven Keller, Lead Compliance Specialist-CIP(501) 688-1633

• Jeremy Withers, Senior Compliance Specialist-CIP(501) 688-1676

• Robert Vaughn, Compliance Specialist II-CIP(501) 482-2301

• Sushil Subedi, Compliance Specialist II-CIP(501) 482-2332

19







March 16, 2016

Shon AustinLead Compliance Specialist – CIP

Mike HughesLead Compliance Engineer - O&P

1

SPP RE Audit Processes and Sampling

Outline

• Inherent Risk Assessment (IRA)

• Internal Control Evaluation (ICE)

• Audit Packet

• RAT-STATS

• Pre-Audit

• Audit

• Audit Report

• Feedback Form

2

Inherent Risk Assessment (IRA) Questionnaire and RAT-STATS Sample

• Registered Functions

• Transmission and Generation Facilities

• Underfrequency Load Shedding

• System Network Information

• Control Centers

• Events

• Internal Compliance Program

• RAT-STATS Spreadsheet (first pass)

• SCADA Environment3

IRA Summary Report

• Summary of Risk Factors

– High

– Moderate

– Low

• Compliance Oversight Program

– Standards and Requirements

– Proposed Monitoring Tool for each Standard

Audit; Spot Check; Periodic Data Submittal; Self-Certification

• Offer to perform an Internal Control Evaluation (ICE)

4

Benefits of ICE

• Internal control consultation

• Scalable – Registered Entities pick/choose requirements

• Possible reduction in RAT-STATS sampling

• Possible shift from audit to Self-Certification

• Not an audit; non-binding recommendations

5

145

130 115 100

Start of the monitoring activity

Registered Entity will provide

documentation and SPP RE will evaluate the effectiveness of

the Internal Controls

The Registered Entity will provide

documentation and SPP RE will evaluate

the design of the Internal Control

Upon receiving the IRA Letter, the

Registered Entity will have 10 days to

request an ICE

IRA completed and approved at approx

165 days prior to monitoring activity

and the IRA Letter is sent to the

Registered Entity

90 Days40 Days25 Days10

Days

165 Days 90 Days155 Days 130 Days 0 Days

SPP RE will send the Registered Entity the

monitoring activity notification at 90 days as

stated in the Rules of Procedure

IRA and ICE Timeline

15 Days

180 Days

IRA started at approx 180 days

prior to monitoring

activity

ICE Summary Report

• Review Key Control Design

• Implementation Level of Key Controls

– Fully Implemented <- to -> Missing

– Monitoring Tool for each Standard

Audit; Spot Check; Periodic Data Submittal; Self-Certification

• Impact on Compliance Oversight Plan

– Reduced Fieldwork; Self-Certification

7

145

130 115 100

Start of the monitoring activity

Registered Entity will provide

documentation and SPP RE will evaluate the effectiveness of

the Internal Controls

The Registered Entity will provide

documentation and SPP RE will evaluate

the design of the Internal Control

Upon receiving the IRA Letter, the

Registered Entity will have 10 days to

request an ICE

IRA completed and approved at approx

165 days prior to monitoring activity

and the IRA Letter is sent to the

Registered Entity

90 Days40 Days25 Days10

Days

165 Days 90 Days155 Days 130 Days 0 Days

SPP RE will send the Registered Entity the

monitoring activity notification at 90 days as

stated in the Rules of Procedure

IRA and ICE Timeline

15 Days

180 Days

IRA started at approx 180 days

prior to monitoring

activity

Audit Packet

• Notification letter

• Audit team bios

• Monitoring scope

• Reliability standard audit worksheets (RSAWs)

• Draft agenda (Ops & Planning)

• RAT-STATS spreadsheet

• Webex to review audit packet

9

10

File Structure

RAT-STATS Spreadsheet – O&P

Sent with IRA questionnaire - first pass sampling:

• BES Substations

• BES Transmission Lines

• BES Generation

• Flowgates

11

RAT-STATS Spreadsheet – O&P

Sent with Audit Packet - second pass selected samples:

• Transmission Line FAC-003 Vegetation

• Transmission Line FAC-008 Facility Ratings

• Generation FAC-008 Facility Ratings

• TOP-002 Next-Day/Current Day Studies

12

RAT-STATS PRC-005 Example

• Questionnaire => First pass 22 BES substations

• SPP RE RAT-STATS random sample => 8 substations

• Second pass BES relays => 110 relays

• SPP RE RAT-STATS random sample => 29 relays

• Submit records for Communications, Batteries, Battery Charger, Sensing Devices (PTs & CTs), and DC Circuitry associated with the 29 identified relays

13

CIP RAT-STATS

14

CIP RAT-STATS Spreadsheet

• Spreadsheet is sent with Audit Packet

• Registered Entity submits “population” in spreadsheet, including but not limited to:

– Identified assets containing BES Cyber Systems (BES Assets)

– Cyber Assets (CA)

– BES Cyber Systems (BCS/BCS detail)

– Logical group of Cyber Asset(s) into one or more BES Cyber Systems (CABCS)

15

CIP RAT-STATS Spreadsheet

One week after receiving sampling data, SPP RE selects samples from the population

• EXAMPLE (CIP-004 R3 Part 3.5): For each sampled “Personnel” in Sample Set, provide a redacted copy of personnel risk assessment with only sufficient evidence to demonstrate:

1. Assessment date

2. Identity check was performed

3. Appropriate criminal history check was performed

• EXAMPLE (CIP-010 R1 Part 1.1): For each Cyber Asset selected in Sample Set, please provide the baseline configuration for this Cyber Asset.

• EXAMPLE (CIP-007 R4 Part 4.2) : For each Cyber Asset selected in Sample Set, provide evidence of actual alerts generated, if any, during the audit period.

16

CIP RAT-STATS Example

• Universe: Registered Entity submits 176 Personnel

• Sample: SPP RE selects 57

• Universe: Registered Entity submits 54 Cyber Assets


• Universe: Registered Entity submits 2,026 generated alerts


17

O&P Timeline for Submission of Evidence

• 60 Days before audit:

– Email stating agreement with Draft Audit Agenda– Verification of Recent Employment (potential conflicts

of interest)– First pass RAT-STATS spreadsheet data


– First round of evidence– Responses to RSAWs– Attestation letter


– Objection to audit team members

18

CIP Timeline for Submission of EvidenceAudit Start Date: 10/19/2016 D-nn* End of Week

IRA Start Date 4/22/2016 D-180 180

IRA Notice 5/7/2016 D-165 165

ICE Approval 5/17/2016 D-155 155

V5 Notice 6/6/2016 D-135 135

Initial Notice: 7/17/2016 D-90 94

Initial Evidence Request 7/17/2016 D-90 94

Pre-Sample Evidence Due: 7/31/2016 D-76 80

Sample Selection Request: 8/7/2016 D-69 73

Survey/Team Objections Due: 8/21/2016 D-56 59

RSAWs/Workbooks Due: 8/28/2016 D-48 52

Initial Evidence Due: 8/28/2016 D-48 52

Second Evidence Request: 9/4/2016 D-41 45

Second Evidence Due: 9/25/2016 D-20 24

First Day Onsite: 10/19/2016 D-0 019

* “D” is first day of audit

Pre-Audit

• Audit Team Review of Evidence

• Possible Subject Matter Expert (SME) Interviews

• Evidence Requests

• Periodic Status Reports

20

Audit

• Opening Presentation

• SME Interviews

– Control Center Visit (Transmission Operator)

• Evidence Requests

• Daily Status Reports

• Exit Presentation

• Audit Report Review

• Final Report(s)

• Feedback Survey Link

24

25

Shon Austin Mike HughesLead Compliance Specialist - CIP Lead Compliance Engineer - O&P501.614.3273 [email protected] [email protected]

1

Commonly Violated Standards: What Went WrongMarch 16, 2016

Greg SorensonSenior Compliance Engineer - O&P

Jeremy WithersSenior Compliance Specialist – CIP

Commonly Violated Standards• Standards and requirements with common problems

• What specifically went wrong

• Expectations, best practices, & key points

• What should you check to make sure you are on track?

• Questions and Answers

2

3

Most Violated StandardsBased on rolling 12 months through 12/31/15 [Represents ~ 89% of total violations]

The current period is the most recent 12 months.The previous period is the previous 12 month period.

SPPRE

RankStandard Description

ViolationsCurrent Period

ViolationsPrevious Period

∆ Risk Factor

1 CIP-002 Critical Cyber Asset Identification 22 8 +14 High/Lower

2 CIP-007 Systems Security Management 9 28 (19) Med./Lower

3 CIP-005 Electronic Security Perimeters 9 15 (6) Med./Lower

4 CIP-006 Physical Security - Cyber Assets 8 13 (5) Med./Lower

5 PRC-005 Protection System Maintenance 8 4 +4 High/Lower

6 VAR-002 Network Voltage Schedules 5 6 (1) Med./Lower

7 CIP-004 Personnel & Training 4 8 (4) Med./Lower

8 FAC-008 Facility Ratings 4 11 (7) Med./Lower

9 PRC-006 Automatic UFLS 3 0 +3 High/Lower

10 PRC-008 UFLS Relay Maintenance 3 1 +2 Medium

All SPP RE Total Incoming 84 121 (37)

PRC-005, PRC-008, PRC-006Protection System Maintenance Plan (PSMP)

• Protection System Maintenance and Testing

• Successful implementation of the standard helps prevent misoperations

• 7 Self-Reports, 9 audit findings

• Self-Reports were lower risk, minor issues which indicated stronger culture of compliance

• Audit findings were high risk (> 30% failure of proper testing)

4

PRC-005, PRC-008, PRC-006 Self-Reports

• Missed one communication device cycle

• Discovered communication devices during diagram and program review

• Missed all quarterly battery inspections one quarter (alarming mitigating factor)

• Batteries and chargers not included in PRC-005 program and therefore not tested

• Inventory incomplete (3 different cases)

• New generator – could not find any inventory or tests

5

Good Internal Controls (not required)

• Routine Program Review – Preventative, Detective

• Routine Inventory Review – Preventative, Corrective– Essential for CASCADE (and other database) programs

– Should perform this after upgrades as well

• Engineer reviews test after completion

• Thorough internal review during self-certification

• Include all elements of the Protection System

• Keep commissioning tests/good file management6

Best practices to prevent these… (not required)

PRC-005 Audit findings

• Registered Entity could not produce tests that showed PSMP followed for 66% of sampled relays

• Yearly maintenance not being performed on any Registered Entity battery chargers (quarterly checks were)

• Registered Entity failed to maintain 50% of devices within the interval (interval transition unsuccessful)

7

PRC-005 Best Practices (not required)

• Review some test reports… Are all required tests listed in the PSMP clearly documented? Do the test reports show what kinds of testing/maintenance was performed?

• When transitioning intervals, the last test is the start date. Changing databases can also be challenging.

• Are all maintenance activities being scheduled and performed?

• You can conduct your own NERC sample – Will match the company-wide situation 95% of time

8

PRC-008, PRC-006 Audit findings

• Registered Entity did not test UFLS component of 40% of sampled UFLS relays

• Registered Entity did not perform set point verification for 15% of sampled UFLS relays

• Registered Entity not following Planning Coordinator’s UFLS step plan

• Registered Entity not following Planning Coordinator’s UFLS time delay in 33% of sampled relays

9

PRC-008, PRC-006 Expectations

• Microprocessor PRC-005-6 includes verifying settings

• 81 (UFLS) settings need to be checked, particularly on distribution relays

• Setting includes a frequency and a time delay

• Noting which distribution relays are “NERC” UFLS relays (best practice… not required)

10

VAR-002

• 2 Self-Reports– Off-peak to on-peak and seasonal transitions of voltage

schedules

• 1 Self-Report– Power System Stabilizer not engaged by generation

personnel after start-up (Transmission Operator never notified)

• 1 Self-Report– Automatic Voltage Regulator out of service for 10 days

after a SCADA test (Transmission Operator never notified of initial test or thereafter)

11

FAC-008

• 3 audit violations (4 findings) and 2 Self-Reports (found during Self-Certification review)

• Self-Report:– Relatively new generator did not have generator

methodology (R1)

• Self-Report:– Relatively new generator did not have a final facility

rating (R6)

• Facilities are expected to be compliant when connected to the grid and producing (any) power

12

FAC-008 Audit findings

• Registered Entity– Did not consider jointly owned elements when developing

ratings (that were in series and limiting)

– Did not follow generator step-up transformer methodology when developing ratings

– Did not consider open bus limitations when developing/implementing ratings (R3.2.4 Operating Limitations – new for FAC-008-3)

– Had 20% of transmission lines rated higher than most limiting element Current transformers were wired at less than maximum and

limited circuit13

FAC-008 Expectations

• All elements in a series are considered

• Elements are correctly rated per the methodology

• Check diagrams and nameplates

• Carefully consider impacts of limiting elements in generation facilities

• Verify rating developed and rating reported/used by Reliability Coordinator and Transmission Operator match

• SPP RE audit team assesses materiality of errors– Errors in monitored elements of flowgates are always

material 14

CIP-002 Critical Cyber Asset Identification

• 22 total violations covering all four requirements

• All 22 violations were self-reported

• The Registered Entities did not have a Risk Based Assessment Methodology (RBAM)

• CIP Version 5 Impact Rating Criteria eliminates need for RBAM

• Ensure you have evidence to support Impact Rating Criteria determinations

15

CIP-007 Systems Security Management

• 9 violations covering 5 requirements

• 4 violations were self-reported

• 5 violations were audit findings

• Issues in the following areas:– Test Procedures

– Account Management

– Security Status Monitoring

– Cyber Vulnerability Assessment (CVA)

16

CIP-007 Test Procedures

• Issue– Laptop not tested before it was introduced into the

Electronic Security Perimeter (ESP)

• Discovery method– Self-Report

• Key takeaways– Train employees on CIP standards and Registered Entity

procedures for applicable assets

– CIP-010-2 R4 allows Registered Entities to deploy Transient Cyber Assets

17

CIP-007 Account Management• Issue

– Employees with access to shared database accounts were not identified

• Discovery method– Audit

• Key takeaways– Ensure accounts associated with software included in

the baseline (CIP-010-2 R1 part 1.1.2) are inventoried

– Identify individuals with access to shared accounts

– Review shared accounts once every 15 calendar months

18

CIP-007 Security Status Monitoring

• Issue– Logs were not captured for assets covered under the CIP

program


• Key takeaway– Check log settings after changes as a part of CIP-007

controls testing

– Be cognizant of any asset limitations

19

CIP-007 Cyber Vulnerability Assessment

• Issue– Controls for default database accounts were not

reviewed


• Key takeaway– Ensure accounts associated with software included in

the baseline (CIP-010-2 R1 part 1.1.2) are inventoried

– Review controls for all default accounts during the CVA

20

CIP-005 Electronic Security Perimeters




• Issues in the following areas:– Electronic Security Perimeter

– Electronic Access Control

– Monitoring Electronic Access

– CVA

– Documentation Review and Maintenance

21

CIP-005 Electronic Access Controls• Issue

– Unnecessary firewall rules were present in the configurations


• Key takeaways– Consider evaluating rulesets often for errors and adding

firewall ruleset changes to change management program

– Consider adding a step in the disposal procedure to remove firewall rules associated with the retired asset

– Avoid overly broad rules 22

CIP-006 Physical Security – Cyber Assets




• Issues in the following areas:– Physical Security Plan

– Protection of Physical Access Control Systems

23

CIP-006 Physical Security Plan

• Issue– Multiple instances of visitor logs lacked date of visit and

exit times of visitors


• Key takeaways– Implement a “real-time” review of visitor logs before

entry into a Physical Security Perimeter

– CIP V5 allows you to record the last exit time of visitors

24

CIP-004 Personnel & Training


• All 4 violations were self-reported

• Issues in the following areas:– Personnel Risk Assessment

– Access

25

CIP-004 Personnel Risk Assessment (PRA)

• Issue– An employee’s PRA was not updated within seven years

of the initial PRA


• Key takeaways– Implement automated flags for task completions

– Evaluate flags periodically for accuracy

26

27



SPPRE




∆ Risk Factor

1 CIP-002 Critical Cyber Asset Identification 22 8 +14 High/Lower




5 PRC-005 Protection System Maintenance 8 4 +4 High/Lower

6 VAR-002 Network Voltage Schedules 5 6 (1) Med./Lower




10 PRC-008 UFLS Relay Maintenance 3 1 +2 Medium

All SPP RE Total Incoming 84 121 (37)

28

Jeremy Withers Greg SorensonSenior Compliance Specialist - CIP Senior Compliance Engineer - O&P501.688-1676 [email protected] [email protected]

General Manager’s ReportRon Ciesiel


1

2015 Year-End Report

2

In 2015, SPP RE staff achieved a weighted total metrics achievement of 115%

SPP RE Violations By Year

3

4



SPPRE




∆ Risk Factor

1 PRC-005 Protection System Maintenance 10 2 +8 High/Med.




5 VAR-002 Network Voltage Schedules 5 4 +1 Med./Lower




9 PRC-023 Transmission Relay Loadability 3 1 +2 High/Lower

10 PRC-008 UFLS Relay Maintenance 2 2 0 Medium

All SPP RE Top 10 Total Incoming 55 74 (19)

VEGETATION CONTACTS

REPORTABLE ACTIONABLE

NERC 3Q-2015 3Q-2015(3Q-2015 LAST

OFFICIAL REPORT)

SPP RE 4Q-2015 3Q-2010(4Q-2015 LAST

OFFICIAL REPORT)

5

Total 2015 SPP RE Events• 21 Events (compared to 30 in 2014)

- 1 event reached Category 2 status (3 in 2014)

- 9 events reached Category 1 status (13 in 2014)

- 11 did not reach “Category” status and were not analyzed via the Events Analysis process (14 in 2014)

6

SPP RE Regional Events 4Q 2015• Two category 1h. Partial loss of monitoring or control

at a control center for 30 minutes

• Two category 1a. Outage of multiple elements. (occurred in 2014 but reported late)

SPP RE Misoperation Report as of 3Q 2015

OutreachDecember webinar on Low Impact Cyber Assets had 150 registrants

2016 WorkshopsMay 24-25, CIP Workshop, Little RockSept. 20-21, Fall 2016 Workshop, Oklahoma City

2016 Trustee Meetings April 25, 2016 - Santa FeJuly 25, 2016 - Rapid CityOctober 24, 2016 - Little Rock

8

http://www.spp.org/documents/34072/low%20impact%20webinar%20121015.pdf

http://www.spp.org/events/calendar/cip-spp-re-workshop/

http://www.spp.org/events/calendar/fall-spp-re-workshop/

http://www.spp.org/event_register2.asp?oID=7361



New Standards: July 1, 2016

• BAL-001-2 Real Power Balancing Control Performance

• CIP Version 5 Standards

• COM-002-4 Operating Personnel Communications Protocols

• MOD-025-2 Verification and Data Reporting of Generator Real and Reactive Power Capability and Synchronous Condenser Reactive Power Capability

• MOD-031-1 Demand and Energy Data

• PER-005-2 Operations Personnel Training9

http://www.nerc.com/pa/Stand/Reliability%20Standards/BAL-001-2.pdf


http://www.nerc.com/pa/Stand/Reliability%20Standards/COM-002-4.pdf

http://www.nerc.com/files/MOD-025-2.pdf

http://www.nerc.com/pa/Stand/Reliability%20Standards/MOD-031-1.pdf

http://www.nerc.com/pa/Stand/Reliability%20Standards/PER-005-2.pdf

New Standards: July 1, 2016 (Cont.)

• PRC-002-2 Disturbance Monitoring and Reporting Requirements

• PRC-004-4(i) Protection System Misoperation Identification and Correction

• PRC-019-1&2 Coordination of Generating Unit or Plant Capabilities, Voltage Regulating Controls, and Protection

• PRC-024-1&2 Generator Frequency and Voltage Protective Relay Settings

10

http://www.nerc.com/pa/Stand/Reliability%20Standards/PRC-002-2.pdf




11

New Standards: January 1, 2017

• TOP -003-3 Operational Reliability Data (all Requirements except R5)

• IRO-010-2 Reliability Coordinator Data Specifications and Collection (Requirements R1 and R2)

http://www.nerc.com/pa/Stand/Reliability%20Standards/TOP-003-3.pdf

http://www.nerc.com/pa/Stand/Reliability%20Standards/IRO-010-2.pdf