1ISASecure

ISASecure Device Test Development and Execution

ISA99 Standards CommitteeOther Standards Organizations

Marketplace Donors

ISA Security Compliance Institute

Test Execution

ISA99 Security StandardsOther Standards, Regulations

Market Donated IP

Feedback on Gaps and Clarifications from Test

Development and Execution (missing requirements)

ISASecure Test Specifications and Profiles

Feedback to ISA Security Compliance Institute

ISASecure Compliant Products

Feedback to Supplier

Supplier Enhances Products/Systems

Pass

Fail

(See details)

2ISASecure

ISASecure Device Conformance Test Development Path

ISA99 StandardsISA100 StandardsIEC StandardsDHS RequirementsNERC StandardsFERC StandardsOther

ISASecure Conformance Requirements

ISASecure Test Kit Specification

(includes test plan)

ISASecure Test Kit(Test cases, procedures,

tools)

Testing Profiles

Device

System

Device &

System

Testing Profiles

Device

System

Device &

System

Testing Profiles

Device

System

Device &

System

Standards Organizations

ISA Security Compliance Institute

WHAT

WHATHOW with tools and procedures defined

Tools and procedures

3ISASecure

Harmonizing Market Supplied (Donated) IAC Security Conformance Requirements

ISCI TSC Issues a public call for input on ISASecure Conformance requirements (example: network attacks)

Donated conformance requirements are entered into a spreadsheet to identify duplications and gaps for analysis by TSC.

TSC reviews Conformance requirements and gains consensus on requirements to include in ISASecure through a vetting process (2/3 majority).

Formally approved conformance requirements from TSC are sent to Governing Board for formal approval based on 2/3 majority of ALL voting Board Members.

Donated conformance requirements are evaluated for quality, format, completeness. Reject poorly constructed/ unusable requirements.

The harmonization process should follow the Conformance Test Development path with the benefit that specific work products should already exist as part of the donated IP; specifically the Conformance Requirements Document and the corresponding Test Kit

TSC evaluates test kits against conformance requirements for approval as ISASecure test vendor. Forwards Recommendation to Governing Board.

Test vendors update tests based on approved conformance requirements.

Approved conformance requirements submitted to ISA SP99 for consideration in standard.

Governing board votes to approve test vendor for ISASecure (2/3 majority of all board members).

4ISASecure

Harmonizing Market Supplied IAC Security Test Specifications

For Example NetworkAttack Testing

Mu SecurityWurldtech

CodenomiconOther

ISASecure Conformance Requirements

ISASecure Test Kit Specification

(includes test plan)

ISASecure Test Kit(Test cases, procedures,

tools)

Testing Profiles

Device

System

Device &

System

Testing Profiles

Device

System

Device &

System

Testing Profiles

Device

System

Device &

System

Donor Organizations

ISA Security Compliance Institute

WHAT

WHATHOW with tools and procedures defined

Tools and procedures

Evaluate whether the donated specifications include well-written Conformance Requirements (the ‘how’), Test Kit Specification and, the Test Kit

5ISASecure

ISASecure Logo Considerations

• What does compliance mean?– Compliance by testing?– Compliance by verifiable/auditable process?– Other forms of compliance

• Do we start with one with intent to evolve to something else?

6ISASecure

ISASecure Compliance by Testing

• Compliance Testing Approach– Works well for standard protocols

• Fieldbus, OPC, TCP/IP

– Can work for devices• Network connected only?• What about proprietary protocols?

– What about open systems nodes?– What about systems?

7ISASecure

ISASecure Compliance by Testing

• Open systems node compliance– Testing for OS configuration– Testing for enabled services

• What about systems that leverage additional services?

– Testing OS security configuration– For Windows Systems

• Compliance to Windows LOGO?– Which LOGO Standard?– Does this mean using VeriTest?

8ISASecure

ISASecure Compliance by Testing

• System Compliance– Network Infrastructure

• Firewalls, routers, switches

– Compartmentalization– Least privilege security configuration– Transferred risks– Role based security configurations– Application level security– …..

9ISASecure

Conformance Testing Challenges

• Approximately 50% of security issues are code bugs.

• Compliance testing will uncover a majority of those bugs, but not all– Will also only find ones in 1st layer code not

multiple layers down

• Testing catches problems too late in the lifecycle– OK to start there but should drive behavioral

change

10ISASecure

Conformance Requirements

• An additional area that causes security vulnerabilities is deployment errors– 30-40% of security compromises

• Difficult to test deployment

• Better to define deployment process and validate

11ISASecure

Conformance Requirements

• Process driven conformance– Similar to DO-178B for avionics products

• Process conformance requirements• External audits for process conformance

– IEC 61508 and 61511 also contain process conformance

12ISASecure

Conformance by process

• Conformance to Security standards– ISA SP99, others

• Conformance for Security Assurance Levels– More objectives for higher assurance levels

• DO-178B like

– More objectives requiring independence• DO-178B like

• Vendors must prove through evidence that required objectives have been met.