internal controls practices group - wecc internal... · 2019. 8. 1. · poka-yoke. poka-yoke...
TRANSCRIPT
Internal Controls
Practices Group
August 1, 2019
Ruchi Shah
Director, Risk Assessment
& Mitigation
Purpose
A platform to bring together WECC and
industry leaders to share practices regarding the
development and sustainability of internal
controls programs for NERC Reliability
Standards.
2
Why?
3
Requirement % of PNCs
CIP-007-6 R2. 6%
CIP-004-6 R5. 5%
CIP-004-6 R4. 5%
CIP-010-2 R1. 5%
CIP-006-6 R1. 4%
PRC-005-6 R3. 4%
CIP-002-5.1a R1. 3%
CIP-006-6 R2. 3%
CIP-007-6 R5. 3%
CIP-004-6 R3. 2%
PRC-024-2 R2. 2%
PRC-019-2 R1. 2%
Other 55%
Top 12 Most Violated Requirements
Cause Group Description % PNCs
A3B1 Skill-based error 31%
A3B2 Rule-based error 27%
A4B1 Inadequate management methods 12%
A4B3 Inadequate work organization and
planning
12%
Top PNC Causes
How can Practices Group Help?
4
Understand how failures happen
Identify common ways to address the failures
Reduce Noncompliance
& Improve Reliability &
Security
Key Takeaways
5
▪ Gain a better understanding of Internal Controls concepts
▪ Implement ideas and practices in day-to-day operations
▪ Peer-to-peer learning
Today
▪ Understand the concepts and tools
▪ Practice sharing
▪ Wrap-up
6
Internal Controls
Practice Group
August 1, 2019
Harold Sherrill
Risk Assessment &
Mitigation
▪ Internal Controls Program Components
• Risk Assessment
• Design & Implementation
▪ Concepts
• Poka-Yoke
• Process Failure Mode Effect Analysis (PFMEA)
◦ Application of PFMEA
▪ Practice Sharing
• Introductions
9
Agenda
10
Program Components
Internal Controls Program
Reliability and Security
Ris
k A
sses
smen
t
Des
ign
&Im
ple
men
tati
on
Co
ntr
ols
M
on
ito
rin
g
Co
ntr
ols
E
val
uat
ion
11
Program Components
Internal Controls Program
Reliability and Security
Ris
k A
sses
smen
t
Des
ign
&
Imp
lem
enta
tio
n
Co
ntr
ols
M
on
ito
rin
g
Co
ntr
ols
E
val
uat
ion
▪ Poka-Yoke/Mistake Proofing
▪ Process Failure Mode Effect Analysis
(PFMEA)
12
Concepts & Tools
Poka-Yoke/Mistake Proofing
Mistake proofing is about awareness, detection, and prevention of
errors that damage outcomes (i.e., reliability & security) and
compliance.
▪ Awareness—communicating the potential for mistakes and
designing the process to detect or prevent mistakes.
▪ Detection—allowing the mistake to happen, but providing a
way to uncover the mistake.
Proactive Focus:
▪ Prevention—keeping process mistakes from occurring in the
first place.
13
Poka-Yoke
Poka-Yoke
Exercise
14
Poka-Yoke for Process Improvement
Human Error
An analyst uses a spreadsheet to track upcoming due
dates. They misread a date which caused an
important task to be late.
Possible Solutions
• Highlight near or past-due items
• Add a “days till due” count-down
15
16
How to Build a PFMEA
Sub-Process Action
Sub-Practice Function
Potential Failure Mode
Potential Causes of Failure
Potential Effects of Failure
Step 1)
Create an action statement from the language of the
requirement and place it in the Sub-Process Action
column. For instance, “…shall have documentation
for determining the facilities ratings…”
17
Sub-Process Action
Step 2)
Determine what the requirement is asking you to do.
In this example, you are being required to document
how you determine facility ratings. So, the Sub-
Practice Function is to “develop documentation.”
18
Sub-Practice Function
Step 3)
Detail the “way” in which you might fail to meet the requirement in the Potential Failure Mode column. In this example, you might fail by having “No or poor documentation suitable to effectively capture ratings.”
19
Potential Failure Mode
Step 4)
Now find the “cause” of this potential failure. One
cause of might be that you did not include guidance
on how exactly you will produce and maintain
documentation. In this example, the Potential Causes
of Failure might be “Failure to develop guidance
specifying how [the entity] shall have documentation
for determining Facility Ratings.”
20
Potential Causes of Failure
Step 5)
Finally, you must state the “effect” if you fail to
mitigate the Potential Causes of Failure. In this
example, the effect statement might be “Reliability
issues due to lack of understanding of facilities
ratings and subsequent limits for devices, lines, and
facilities.”
21
Potential Effects of Failure
The potential failure points and guidance questions give direction to registered entities for assessment of risk, while designing internal controls specific to NERC Reliability Standards and Requirements. The Registered Entity may use this document as a starting point in determining entity risk. It is not WECC’s intent to establish a standard or baseline for entity risk assessment or controls design.
Note: Guidance questions help an entity understand and document its controls. Any responses, including lack of affirmative feedback, will have no consequences on an entity’s demonstration of compliance at audit.
*Please send feedback to [email protected] with suggestions on potential failure points and guidance questions.
22
Intent of Failure Points
▪ Internal Controls Program Components
• Risk Assessment
• Design & Implementation
▪ Poka-Yoke/Mistake Proofing
▪ Process Failure Mode Effect Analysis
(PFMEA)
• Application of PFMEA
23
Summary/Questions
24
Break
25
READY,
SET,
SHARE!
Internal Controls Failure Points
CIP-007-6 R1
26
SECURITY OBJECTIVE
To reduce the attack surface of Cyber Assets by disabling or restricting access to all known unnecessary ports.
To be aware of network-accessible (“listening”) ports and associated services accessible on their assets and
systems, whether they are needed for that Cyber Asset’s function, and disable or restrict access to all other
ports.
NIST Special Publication 800-53 (Rev. 4) CM-6
Internal Controls Failure Points
CIP-007-6 R1
27
Potential Failure Point (Part 1.1): Failure to develop a complete list of
Cyber Assets that require a process to identify all logical network-
accessible ports.
Potential Failure Point (Part 1.1): Failure to develop a process to
determine technical feasibility.
Internal Controls Failure Points
CIP-007-6 R1
28
Potential Failure Point (Part 1.1): Failure to develop a process to
identify all logical network-accessible ports.
Potential Failure Point (Part 1.1): Failure to develop a process to identify
which network-accessible ports are needed.
Potential Failure Point (Part 1.1): Failure to have a process to identify
ranges on logical network accessible ports.
Internal Controls Failure Points
CIP-007-6 R1
29
Potential Failure Point (Part 1.1): Failure to document ports identified
as “needed for operation” in configuration baselines.
Potential Failure Point (Part 1.2): Failure to develop a process to identify and
protect physical input/output ports.
Internal Controls Failure Points
CIP-007-6 R1
30
SECURITY OBJECTIVE
• Continuously acquire, assess, and act on new information to identify, remediate, and reduce
opportunities for attack.
• Review proposed configuration-controlled changes to the information system and approve or disapprove
changes considering security impact analyses.
• Proactively monitor and address known security vulnerabilities in software before they can be used to
gain control of or render inoperable a Bulk Electric System (BES) Cyber Asset or BES Cyber System.
NIST Special Publication 800-53 (Rev. 4) (CM-4)
Internal Controls Failure Points
CIP-007-6 R2
31
Potential Failure Point: Failure to have a procedure to update the patch
management process whenever there are changes to the entity’s applicable
Cyber Assets.
Potential Failure Point (Part 2.1): Failure to develop a complete list of
Cyber Assets that require a process to identify and track sources of
patches.
Internal Controls Failure Points
CIP-007-6 R2
32
Potential Failure Point (Part 2.1): Failure to develop a process/procedure
on how to identify and track sources of patches for applicable systems.
Potential Failure Point (Part 2.1): Failure to have a process or procedure to
evaluate patches for all applicable Cyber Assets, Systems, associated
software, firmware, and drivers.
Internal Controls Failure Points
CIP-007-6 R2
33
Potential Failure Point (Part 2.1): Failure to have a process or procedure
to install patches for all applicable Cyber Assets, Systems, associated
software, firmware, and drivers.
Potential Failure Point (Part 2.1): Failure to develop a procedure to
document updates of installed patches in baseline configurations.
Internal Controls Failure Points
CIP-007-6 R2
34
Potential Failure Point (Parts 2.2, 2.4): Failure to define or communicate start/end
dates for monitoring and mitigation timeline(s).
Potential Failure Point (Part 2.3): Failure to have a process for creating a
mitigation plan to properly deal with the vulnerabilities addressed by
each security patch.
Internal Controls Failure Points
CIP-007-6 R2
35
SECURITY OBJECTIVE
Each Responsible Entity must implement documented processes that collectively include:
• Deploying methods to deter, detect, or prevent malicious code;
• Mitigating the threat of detected malicious code; and
• Updating, testing, and installing identified methods that use signatures or patterns.
NIST Special Publication 800-53 (Rev. 4) SI-3
Internal Controls Failure Points
CIP-007-6 R3
36
Potential Failure Point (R3): Failure to develop a complete list of
Cyber Assets that require a process to prevent malicious code.
Potential Failure Point (R3): Failure to have a procedure that shows how the entity
will deploy methods to deter, detect, or prevent malicious code.
Internal Controls Failure Points
CIP-007-6 R3
37
Potential Failure Point (R3): Failure to develop a procedure that shows
how the entity will mitigate the threat of detected malicious code.
Internal Controls Failure Points
CIP-007-6 R3
38
Potential Failure Point (R3): Failure to develop a process to identify
methods in Part 3.1 that use signatures or patterns.
Potential Failure Point (R3): Failure to develop a process to update the
signatures or patterns.
Internal Controls Failure Points
CIP-007-6 R3
39
Potential Failure Point (R3): Failure to develop a procedure that shows
how to address testing and installation of signatures or patterns.
Internal Controls Failure Points
CIP-007-6 R3
40
SECURITY OBJECTIVE
Awareness of access events that report on:
• Successful login attempts;
• A limit of [organization-defined number] consecutive invalid login attempts by a user during a
[organization-defined period];
• A maximum number of unsuccessful login attempts; and
• Awareness of detection of malicious code.
NIST Special Publication 800-53 (Rev. 4) SI-3(1) & AU-12
Internal Controls Failure Points
CIP-007-6 R4
41
Potential Failure Point (R4): Failure to develop a complete list of assets
that require a process to log relevant events.
Potential Failure Point (R4): Failure to develop a procedure or process
that defines events at the device or system level for the specified types.
Internal Controls Failure Points
CIP-007-6 R4
42
Potential Failure Point (R4): Failure to develop a procedure or process that outlines how the entity will capture events.
Potential Failure Point (R4): Failure to develop a procedure or process that
defines an “alert.”
Internal Controls Failure Points
CIP-007-6 R4
43
Potential Failure Point (R4): Failure to develop a procedure or process that
defines a “failure of event logging.”
Potential Failure Point (R4): Failure to develop a policy that requires event
log retention at the device or system level for the specified types.
Internal Controls Failure Points
CIP-007-6 R4
44
Potential Failure Point (R4): Failure to define a qualifying “CIP Exceptional Circumstance.”
Potential Failure Point (R4): Failure to develop a procedure or process
that defines “technical feasibility.”
Potential Failure Point (R4): Failure to define a “summarization” or a
“sample.”
Potential Failure Point (R4): Failure to define an
“undetected Cyber Security Incident.”
Internal Controls Failure Points
CIP-007-6 R4
45
Potential Failure Point (R4): Failure to develop a procedure or process
that outlines how the identification of an undetected Cyber Security
Incident is to occur.
Potential Failure Point (R4): Failure to clearly define or communicate
start and end dates used to establish a period for review of log
outside of alert monitoring.
Internal Controls Failure Points
CIP-007-6 R4
46
Internal Controls Failure Points
CIP-007-6 R5
SECURITY OBJECTIVE
To manage system security by specifying technical, operational, and procedural requirements that protect the
Bulk Electric System (BES) Cyber Systems against compromise that could lead to misoperation or instability
in the BES.
• Enforce authentication of the intended individuals, groups, roles, or devices.
• Disable the identifier after business use is not required.
• Review accounts for compliance with account management requirements.
• Establish a process for protection of shared or group account credentials when individuals are
removed from the group.
• Ensure information systems support individual authenticator management by capability-defined
settings and restrictions for characteristics such as minimum password length, password
composition, etc.
• Enforce a limit on consecutive invalid login attempts by user on devices.
NIST Special Publication 800-53 (Rev. 4) IA-4, IA-5, AC-2, AC-7.
47
Potential Failure Point (R5): Failure to develop a complete list of assets
that require application of security controls outlined in R5.
Internal Controls Failure Points
CIP-007-6 R5
48
Potential Failure Point (R5): Failure to establish methods to enforce
authentication of interactive user access.
Internal Controls Failure Points
CIP-007-6 R5
49
Potential Failure Point (R5): Failure to identify the existence and potential
uses of default or generic account types that could be used to access
devices or introduce vulnerabilities for new and existing accounts.
Potential Failure Point (R5): Failure to identify individuals with access to
shared accounts.
Internal Controls Failure Points
CIP-007-6 R5
50
Potential Failure Point: (R5) Failure to develop a process to identify and
inventory all known default passwords.
Potential Failure Point (R5): Failure to change default passwords.
Internal Controls Failure Points
CIP-007-6 R5
51
Potential Failure Point (R5): Failure to develop methods to enforce password
parameters technically or procedurally.
Internal Controls Failure Points
CIP-007-6 R5
Potential Failure Point (R5): Failure to determine technical feasibility of
password change capability.
52
Potential Failure Point (R5): Failure to clearly define or communicate start
and end dates used to establish a period for password changes
Potential Failure Point (R5): Failure to create a technical feasibility
exception (TFE) and have it reviewed by WECC.
Internal Controls Failure Points
CIP-007-6 R5
53
Potential Failure Point (R5): Failure to establish a procedure on how
lockouts should occur.
Potential Failure Point (R5): Failure to establish lockout thresholds or alert
parameters after a specified number of unsuccessful authentication
attempts.
Internal Controls Failure Points
CIP-007-6 R5
54
Internal Controls Failure Points
FAC-008-3
SECURITY OBJECTIVE
To ensure that Facility Ratings used in the reliable planning and operation of the Bulk Electric System (BES)
are determined based on technically sound principles. A Facility Rating is essential for the determination of
System Operating Limits.
GENERAL FAILURE POINTS
▪ Potential Failure Point: Failure to develop a process
to ensure that the Facility Ratings methodology is
developed and followed.
▪ Potential Failure Point: Failure to develop a process
to track Facility status (i.e., new, existing, modified,
re-rates) and its Ratings.
▪ Potential Failure Point: Failure to develop guidance
specifying how you shall have documentation for
determining Facility Ratings.
55
Internal Controls Failure Points
FAC-008-3
56
Potential Failure Point (R1): Failure to develop guidance specifying how you will
have documentation for determining Facility Ratings.
Internal Controls Failure Points
FAC-008-3 R1
Potential Failure Point (R1): Failure to develop a process to identify
element ownership.
57
Internal Controls Failure Points
FAC-008-3 R1
Potential Failure Point (R1): Failure to develop a process to identify
element connectivity.
58
Potential Failure Point (R1): Failure to train personnel on developed
Facility Ratings.
Potential Failure Point (R1): Failure to develop a process for
identifying the most limiting element in a Facility.
Potential Failure Point (R1): Failure to define, communicate, and apply
technically sound assumptions used in developing Ratings.
Internal Controls Failure Points
FAC-008-3 R1
59
Potential Failure Point (R2): Failure to develop guidance specifying
how you will document methodology for determining Facility Ratings.
Internal Controls Failure Points
FAC-008-3 R2
60
Internal Controls Failure Points
FAC-008-3 R2
Potential Failure Point (R2): Failure to develop a process to identify
element ownership for solely and jointly owned Facilities.
Potential Failure Point (R2): Failure to develop a process to identify
and evaluate element connectivity.
61
Potential Failure Point (R2): Failure to define, communicate, and apply
technically sound assumptions used in developing the methodology.
Internal Controls Failure Points
FAC-008-3 R2
62
Potential Failure Point (R2): Failure to develop a process for
identifying the most limiting element in a Facility.
Potential Failure Point (R2): Failure to develop guidance used in the
Equipment Rating determination process.
Internal Controls Failure Points
FAC-008-3 R2
63
Potential Failure Point (R3): Failure to develop guidance specifying how you
will document methodology for determining Facility Ratings.
Internal Controls Failure Points
FAC-008-3 R3
64
Potential Failure Point (R3): Failure to develop a process to identify element
ownership for solely and jointly owned Facilities.
Internal Controls Failure Points
FAC-008-3 R3
65
Potential Failure Point (R3): Failure to develop a process to identify and
evaluate element connectivity.
Potential Failure Point (R3): Failure to define, communicate, and apply
technically sound assumptions used in developing the methodology.
Internal Controls Failure Points
FAC-008-3 R3
66
Potential Failure Point (R3): Failure to develop a process for identifying the
most limiting element in a Facility.
Potential Failure Point (R3): Failure to develop guidance used in the
Equipment Rating determination process.
Internal Controls Failure Points
FAC-008-3 R3
67
Potential Failure Point (R6): Failure to have a Facility Ratings application
strategy that includes applicable components in R1, R2, and R3.
Internal Controls Failure Points
FAC-008-3 R6
Potential Failure Point (R6): Failure to train personnel who execute and implement Facility Ratings process.
68
Potential Failure Point (R6): Failure to develop a process to identify the
most limiting equipment of a Facility.
Potential Failure Point (R7): Failure to develop a process to track changes to the new, modified, or re rated Facility and its Rating.
Internal Controls Failure Points
FAC-008-3 R6, R7
69
Potential Failure Point (R7): Failure to develop a process to manage
requests for information that you are obligated to provide.
Internal Controls Failure Points
FAC-008-3 R7, R8
70
Potential Failure Point (R7): Failure to develop a process to identify the
most limiting equipment of a Facility.
Potential Failure Point (R8): Failure to develop a process to track changes to
the new, modified, or re rated Facility and its Rating.
Internal Controls Failure Points
FAC-008-3 R7, R8
71
Internal Controls Failure Points
FAC-008-3 R8
Potential Failure Point (R8): Failure to develop a process to manage
request for information that you are obligated to provide.
72
Potential Failure Point (R8): Failure to develop a process to identify a
Facility with a Thermal Rating that limits the use of the Facility under the
requestor’s authority.
Internal Controls Failure Points
FAC-008-3 R8
▪ Failure Points and Guidance Questionshttps://www.wecc.org/Pages/Compliance-UnitedStates.aspx
▪ National Institute of Standards and
Technology – Framework for Improving
Critical Infrastructure Cybersecurity Core
▪ SP-800-53 Security and Privacy Controls for
Federal Information Systems and
Organizations
73
Resources for Good Practices
FAC-003 & CIP-010
November 19, 2019 1:00pm – 5:00pm
November 20, 2019 8:00am – 12:00pm
California ISO
250 Outcropping Way
Folsom, CARegistration Link
74
Next ICPG Meeting