information security audits & certificationkursawe/sio2011/slides/lecture9.pdf · • an...

1

Information security audits

& certification

Security in Organizations

2011

Eric Verheul

2

Literature

Main literature for this lecture:

1. NOREA beroepsregels

http://www.norea.nl/Norea/Thema's/Gedrags-

+en+beroepsregels/Richtlijn+Assurance-opdrachten

2. TTP.NL schema (http://www.ecp.nl/sites/default/files/TTP-

NL_Scheme_version_8.1_final__June_2010_.pdf )

3. Common Criteria part 1

(http://standards.iso.org/ittf/PubliclyAvailableStandards/c050

341_ISO_IEC_15408-1_2009.zip )

Variants on ISO 2700*

http://www.ecp.nl/sites/default/files/TTP-NL_Scheme_version_8.1_final__June_2010_.pdf



http://standards.iso.org/ittf/PubliclyAvailableStandards/c050341_ISO_IEC_15408-1_2009.zip




3

Assignment #5

• Assignment #3 is on Blackboard

• It uses VMWARE image. This is available:

• Through Klaus/DVD

• On-line

sftp://lilo.science.ru.nl/vol/xpsoftware/sio2009/image_1111

09/*.*

• Note: starting the VMWARE image takes time; first start

the image then read the assignment

Variants on ISO 2700*

4

Outline

• Audit introduction

• IT security audits in general

• management system certification audits

• IT security product certification audits (‘common criteria’)

• Recap & Practicum

5

Types of audits

• The audits we are discussing include: • IT security audits in general,

• management systemcertification audits,

• IT security product certification audits

• As there is – as far as we know – no common terminology used for these three types of audits simultaneously, we will introduce our own terminology. This is actually based on a combination of terms taken from these audit types.

Audit introduction

6

Terminology

• An audit is the process in which an competent, impartial judgment (‘opinion’) is formed on one or more aspects of an object (‘criteria’).

• The result of an audit is typically a document in which the auditor expresses his opinion, the supporting findings and the limitations that apply.

• The opinion provides assurance to the auditee itself or to a third party.

• The assurance can be either positive or negative: • Positive assurance - An affirmative statement or opinion given by the

auditor, generally based on a high level of work performed.

• Negative assurance - A statement indicating that nothing came to the auditor's attention indicating that the subject matter in question did not meet a specified criteria.

Audit introduction

7

Terminology

Audit

Object

Audit

Criteria

Audit

Scheme Auditor

Opinion (report)

Independent

overseer

(e.g. association

organization)

Scheme

maintainer

(e.g. association

organization)

Audit introduction

Criteria

maintainer

8

Terminology

• The audit process should be reproducible and should not depend on the (qualified) auditor.

• An opinion can also take the form of a ‘certificate’.

• Audits are historically associated with accounting: a financial audit of the financial accounts (‘jaarrekening audit’) performed by (registered) accountants. In this situation the criteria are based on the laws on accounting (‘Wet op de jaarrekening’). In the accounting context the term ‘audit’ is a very sensitive notion.

Audit introduction

9

Terminology

• The audit is performed for a client, that also sponsors the audit.

• The aspects that form the basis of the audit are formulated as a set of criteria (audit criteria), determined prior to the actual audit and agreed upon with the client. In Dutch these criteria are sometimes ‘de gehanteerde (audit) norm’.

• The set of criteria could be an open standard, a tailored version of it, or even some assertions made by the client management. In the latter case, the opinion can be a statement of the auditor that the assertions are correct.

• The object type can vary, examples are: a person, a product, a process, a system or an organization.

Audit introduction

10

Audit schemes

• Closely linked with the audit criteria is the audit scheme used. These are rules describing how the audits shall be conducted and what requirements should be met by the auditor organization itself

• An audit scheme provides a ‘manual’ for conducting audits and typically answers questions like: • What steps shall an audit have?

• When is a criterion met?

• What qualifications should an auditor have?

• When can the auditor ‘built’ on prior work done by other auditors?

• When can an opinion be provided and what can be part of it?

Audit introduction

11

Audit schemes

Important general topics in audit schemes are:

• impartiality requirements of auditors and the organizations they work for,

• confidentiality,

• providing auditees the opportunity to respond to findings (‘hoor en wederhoor’)

• ethics, e.g., ‘do not audit your own work’,

• quality, e.g. filing of evidence

Audit introduction

12

Audit schemes

The audit scheme can be:

• an open standard itself, e.g., • ISO 19011 ‘Guidelines for quality and/or environmental management

systems auditing’

• ISO/IEC 17021 ‘Requirements for bodies providing audit and certification of management systems,

• and its particularization ISO 27006 ‘Requirements for bodies providing audit and certification of information security management systems’

• a dedicated document, e.g., the TTP-NL scheme ‘Scheme For Certification of Certification Authorities against ETSI TS 101 456’

• or it could be part of the rules of conduct of the professional associations (‘beroepsverenigingen’) of auditors, e.g. of NOREA (http://www.norea.nl/Norea/Thema's/Gedrags-+en+beroepsregels/Richtlijn+Assurance-opdrachten) or ISACA (www.isaca.org).

Audit introduction

http://www.norea.nl/Norea/Thema's/Gedrags-+en+beroepsregels/Richtlijn+Assurance-opdrachten





http://www.isaca.org/

13

Terminology

Obje

ct

Op

inio

n

Crite

ria

Source: https://cert.webtrust.org/SealFile?seal=304&file=pdf

Schem

e

Audit introduction

https://cert.webtrust.org/SealFile?seal=304&file=pdf

14

Terminology

Audit

Object

Audit

Criteria

Audit

Scheme Auditor

Opinion (report)

Independent

overseer

(e.g. association

organization)

Scheme

maintainer

(e.g. association

organization)

Audit introduction

Criteria

maintainer

15

Outline



• management systemcertification audits,



16

IT (security) audits

• An IT security audit is a particular type of an IT audit.

• An IT audit is also known as an EDP audit and focuses on the following aspects of IT systems (cf. COBIT): • Effectiveness

• Efficiency

• Compliance

• Reliability

• Confidentiality

• Integrity

• Availability

• An IT audit can therefore include much more than information security.

IT security audits in general

17

IT audit aspects

• Effectiveness

Deals with information being relevant and pertinent to the business process as well as being delivered in a timely, correct, consistent and usable manner.

• Efficiency

Concerns the provision of information through the optimal (most productive and economical) usage of resources

• Reliability

Relates to systems providing management with appropriate information for it to use in operating the entity, in providing financial reporting to users of the financial information, and in providing information to report to regulatory bodies with regard to compliance with laws and regulations

• Compliance

Deals with complying with those laws, regulations and contractual arrangements to which the business process is subject; i.e., externally imposed business criteria


18

IT effectiveness IT security audits in general

19

IT audit aspects

• Confidentiality

Concerns protection of sensitive information from unauthorized disclosure.

• Integrity

Relates to the accuracy and completeness of information as well as to its validity in accordance with the business' set of values and expectations.

• Availability

Relates to information being available when required by the business process, and hence also concerns the safeguarding of resources.


20

IT security audits

• IT security audits (aka IT security reviews) concentrate on information security aspects, i.e.: • Confidentiality

• Integrity

• Availability

• Sometimes IT security audits are called IT security reviews to prevent confusion with financial audits.

• IT security audits can be: • technically oriented; then the objects are IT systems, e.g., a whole IT

infrastructure, a network, a Windows environment, a specific application

• process oriented; then the objects are IT processes, e.g., a security management process, a change management process.

• The audit criteria are typically formulated in information security objectives or security controls, e.g. based on ISO 27002.


21

Example of technical IT Security criteria IT security audits in general

22

Example of non-technical IT Security criteria IT security audits in general

23

Audit evidence

• Practically speaking, the auditor should: • determine the scope of the audit (e.g., Windows based office

automation network),

• agree the audit criteria with the audit sponsor and put them in a table and compare the criteria with the object setting.

• But what should an auditor accept as compliance evidence?


24

Audit evidence

• What if the IT administrator says in an interview: ‘Sure, we have this password policy and account lockout setting’?

• What if there is an official document stating compliance with these setting?

• When should you believe this setting is actually implemented?


25

Audit evidence

• What if the IT administrator shows you the Windows settings. Can you be sure that this will not be changed tomorrow?


26

The three audit assurance levels

• In IT audits one therefore distinguishes three types of audit assurance ‘levels’: • Design

The auditor has reviewed the relevant design based on documentation and interviews but not on actual inspections. In effect, the auditor can not provide assurance is designed is actually implemented.

• Existence

The auditor has additionally performed inspections of system settings, paper archives and other things providing him with assurance that the design is at least implemented during the audit.

• Operational Effectiveness

The auditor has additionally looked for evidence that the implemented controls were effective for a certain amount of time.

• These audit levels build upon each other, i.e. you can only have Design, Design + Existence or Design + Existence + OE

• The audit level is an integral part of the opinion report!


27

Terminology

Obje

ct

Op

inio

n

Crite

ria

Source: https://cert.webtrust.org/SealFile?seal=304&file=pdf

Schem

e


https://cert.webtrust.org/SealFile?seal=304&file=pdf

28

The three audit assurance levels

• In IT audits one therefore distinguishes three types of audit assurance ‘levels’: • Design

The auditor has reviewed the relevant design based on documentation and interviews but not on actual inspections. In effect, the auditor can not provide assurance is designed is actually implemented.

• Existence

The auditor has additionally performed inspections of system settings, paper archives and other things providing him with assurance that the design is at least implemented during the audit.

• Operational Effectiveness

The auditor has additionally looked for evidence that the implemented controls were effective for a certain amount of time.


29

The opinion

• It is vital that the opinion minimally states:

• For who the audit was conducted (client) by whom (auditor)

• The objective of the audit

• The object and its boundaries

• The period in which the audit was performed

• The followed procedures, e.g., documentation review, interviews, inspections etc.

• The audit criteria used and the related audit scheme

• The assurance level of the audit (‘design’, ‘existence’ or ‘operational effectiveness’

• The opinion itself and any reservations or limitations regarding the opinion.

• Optionally one can supplement the opinion with recommendations however some schemes preclude on grounds of impartiality.


30

Is Penetration Testing ‘auditing’?

• A penetration test is a method of evaluating the security of a computer system or network by simulating an attack from a malicious source. (source: wikipedia).

• One could say a penetration test concentrates on ‘existence’ and ‘operational effectiveness’ of information security and not on documented ‘design’.

• There are implicit ‘criteria’ and ‘frameworks’ such as The Open Source Security Testing Methodology Manual (OSSTMM) and Guideline on Network Security Testing (NIST SP 800-42) and the Testing Guide of the Open Web Application Security Project (OWASP).

• There also exist professional associations of penetration testers.

• Dependence on the competence of penetration tester is higher than in a typical audit, making the reproducibility difficult.


31

Outline



• management systemcertification audits



32

Certification of management systems

• A management system is framework of policies, procedures guidelines and associated resources to achieve the objectives of the organization. (source: ISO 27000)

• An Information Security Management System (ISMS) is that part of the overall management system, based on a business risk approach, to establish, implement, operate, monitor, review, maintain and improve information security. (source: ISO 27001)

• Certification of a management system, such as a quality or environmental management system of an organization, is one means of providing assurance that the organization has implemented a system for the management of the relevant aspects of its activities, in line with its policy. (source: ISO 17021).

management system certification audits

33

Limitation

• We limit our selves to these two IT security (management) systems:

• ISO 27001 ‘Information security management systems’ which we have focused on in the first three lectures

• TTP.NL which relates to the European guideline on electronic signatures.

• Certification ‘auditors’ are typically called Certification Bodies and need to have management systems of their own to reflect the proper process of certification. The basis for the management system for Certification Bodies is described in ISO 17021.


34

IT security management certification framework

Management

System Standard

(ISO, ETSI)

Certification

Scheme Certification

Body

Certificate (report)

Scheme

maintainer

(ISO, ECP-EPN)

Accreditation

Body

(RVA)


Criteria

Maintainer

(ISO, ETSI)

35

Accreditation

• The management systems of the certification Bodies are inspected by the Dutch Accreditation Council (Raad voor Accrediatie, see www.rva.nl) in a similar way as the Certification Bodies perform certification.

• This process is called accreditation. There is actually a standard for Accreditation Bodies themselves (ISO 17011 ‘General requirements for accreditation bodies accrediting conformity assessment bodies’).

• (Nearly) every country has its own ‘Accreditation Council’ and mutual agreements exist.

• The national councils also perform ‘peer reviews’.


http://www.rva.nl/

36

The schemes

ISO 27001 scheme

Criteria: ISO 27001

Scheme:

• maintained by ISO

• ISO 27006 which is based on ISO 17021 (who) and ISO 19011 (how)

• Accreditation Body: RVA

TTP.NL scheme

Criteria: ETSI TS 101 456

Scheme:

• Maintained by ECP-EPN

• TTP.NL scheme which is based on ISO 17021 (who) and ISO 19011 (how)

• Accreditation Body: RVA


37

Audits within a certification

There exist several types of audits within in the context of a certification:

• Trial audits (optional)

• Initial audits (consisting themselves of documentation and implementation audits)

• Follow-up audits

• Surveillance audits

• Recertification audits very similar to initial audits.

• Special audits, when major changes take place in the client’s organization


38

Certification overview

Documentation Audit

Implementation Audit

Certification Decision

Trial-audit (optional)

Surveillance

Audit (yearly)

Surveillance

Audit (yearly)

Recertification

• A certificate is valid for three years.

Follow-up audits


39

Audit plan

• For each type of audit, the audits makes an audit plan prior to conducting it.

• The audit plan describes: • the timing of the audit,

• the topics of the audit (preferably in reference with the criteria)

• the staff (internal/external) that needs to be interviewed (when/where)

• any visits or inspections (in implementation and surveillance audits)

• The audit plan is interactively compiled with the client, typically through email.


40

Opening meetings

• Each audit commences with an opening meeting. Typical things addressed are:

• introduction of team members

• scope and methodology of the audit

• (understanding of) the audit plan

• any unresolved issues from earlier audits

• timing of the closing meeting


41

Closing meetings

• Each audit ends with an closing meeting. In this meeting the lead auditor provides its general impression on the organization and more specifically the negative findings.

• There exist two types of negative findings:

• Minor nonconformities also known as deficiencies

• Major nonconformities

• Each of these is separately documented in a (draft) non-conformity report and discussed with the client.

• The findings are (preferably) formally accepted (‘signed’) in the closing meeting.

• Only negative findings are documented.


42

Documentation audit

• Part of initial audit, also known as Stage I audit in ISO 27006.

• The auditor reviews the documentation and keeps interviews to check consistence with the audit criteria.

• This audit will familiarize the auditor with the organization and will allow him to formulate attention points for the Stage II (‘implementation’) audit.

ISO

27

00

1

ET

SI T

S 1

01456


43

Implementation audit

• Part of initial audit, also known as Stage II audit in ISO 27006.

• The auditor checks the existence of controls in consistence with the documentation.

ISO 27001

ETSI TS 101456


44

Certification decision

• The certification manager certification decides upon certification based on the stage I and stage II reports.

• The certification manager must not have been part of the audit team.

• The client documents for each Non-Conformity a Corrective Action Report (CAR) which includes a cause analysis, a corrective action and its planning.

• It is impossible to be certified if there still exist major non-conformities; these need to be addressed and reassessed (‘follow-up audit’) before certification.

• The certificate is valid for three years and every year surveillance audits are conducted (typically much smaller than an implementation audit).

• After three years a recertification audit is conducted, similar to the initial one.


45

Surveillance audit

• At least once a year, the certification body carries out a surveillance audit consistent with the same requirements the initial audit was conducted.

• The surveillance is a combination of a documentation and implementation audit.

• These periodic assessments serve to make sure all requirements are assessed at least once during the certificates’ period of validity

• Each surveillance audit will address fixed elements as well:

• ‘open’ non-conformities

• the internal audits carried out by the organization,

• the complaints of customers,

• management reviews of the management system


46

Audit time

• To conduct certification audits the auditor has rather limited time, implying that the implementation audits are only of limited depth.

ISO

27

00

6


47

Outline






48

IT security product certification framework

IT product Common

Criteria (ISO15408 )

CC-CEM

(ISO 18045)

Evaluator

(laboratory)

CC Certificate

Scheme

maintainer

(ISO, BSI, TNO)

Accreditation

Body

(BSI, RVA)

IT security product certification audits (‘common criteria’)

49

NL product example (fox-it) IT security product certification audits (‘common criteria’)

http://dl.dropbox.com/u/6343869/My%20First%20Common%20Criteria.pdf

http://dl.dropbox.com/u/6343869/My First Common Criteria.pdf

50

IT product security

• Several governments have early recognized the inherent security risk of computer systems, e.g.:

• the risks of not having the ‘right’ controls in the systems (security functionality)

• the risks of not having adequate assurance that controls are properly implemented (assurance)

• Security of a system is function of security functionality and assurance

Systems A and B could have the same security functionality (e.g. a password based authentication mechanism) but if the system A’s development of A is more thorough than that of system B; system A is probably more ‘secure’ than B.

• What is required is an IT-product security certification framework enabling:

• ‘users’ to formulate their security needs in requirements for IT products

• manufacturers to develop (potential) conformant IT products

• technical laboratories to independently evaluate these IT products against the set requirements

• ‘authorities’ to certify these IT products based on the evaluation report

• ‘users’ to apply these IT products in the right way (accreditation)


51

TCSEC

• In the 1980s the US defense department initiated the Trusted Computer System Evaluation Criteria (TCSEC) program for assessing the effectiveness of computer security controls built into an operation system.

• It is commonly known as the Orange Book based on the color of its cover. The Orange Book focuses on OS and leaves out many important information security aspects (such as networks!). This gave rise to many other (colored) books resulting in what called the Rainbow series.

• The Orange Book distinguishes the following OS types: • D: minimal protection

• C[1-2]: Discretionary protection: users can decide which information is accessible by others’

• B[1-3]: Mandatory protection: information is labeled with classifications, e.g. restricted, confidential, secret, and the system enforces access based on the clearance of users

• A[1]: Verified protection: builds further on B but includes formal design and verification techniques.

• The classes are a rigid combination of security functionality and security assurance.


52

ITSEC

• The critique on TCSEC is that it is rather rigid • TCSEC focuses on confidentiality

• The TSCEC ratings are a fixed combination of functionality and assurance.

• TCSEC does not provide users flexibility in describing security requirements different than those in TCSEC.

• In the 1990s France, Germany, the Netherlands, and the United Kingdom published their own evaluation framework called the Information Technology Security Evaluation Criteria (ITSEC).

• ITSEC is more flexible and allows users more flexibility in describing their security requirements than TCSEC. Moreover ITSEC separates functionality and assurance. ITSEC introduces 7 assurance classes E0 – E6 where E0 represents the lowest and E6 the highest assurance.

• ITSEC suggest a comparison between its assurance classes and the implicit assurance classes in the TCSEC classes (D, C1, C2, B1, B2, B3, A1).


53

ITSEC assurance classes IT security product certification audits (‘common criteria’)

54

Common Criteria

• According to some TCSEC is too hard and ITSEC is too soft.

• The Common Criteria for Information Technology Security Evaluation or simply Common Criteria (or CC) is based on three underlying IT-product security certification frameworks: ITSEC (EU), TCSEC (US) and CTCPEC (Canada).

• The CC are published as ISO standards (ISO/IEC 15408): • Part 1: Introduction and general model

• Part 2: Security functional requirements

• Part 3: Security assurance requirements

• The guidelines for the CC evaluators (Methodology for IT security evaluation) is also published as an ISO standard (ISO 18045)

• They can be freely downloaded from http://standards.iso.org/ittf/PubliclyAvailableStandards/index.html


http://standards.iso.org/ittf/PubliclyAvailableStandards/index.html

55

Short history IT security product certification audits (‘common criteria’)

56

Protection Profile (PP)

• Typically a ‘user community ’ compiles a Protection Profile (PP) for a TOE (Target of Evaluation) type, e.g., a firewall or a smartcard application (e.g., an SSCD).

• A PP defines an implementation-independent set of IT security requirements for a category of TOEs which are intended to meet common consumer needs for IT security.

• A PP contains • The TOE description,

• The TOE environment (including threats)

• Security Functional Requirements (SFRs) as specified in CC-part2

• Security Assurance Requirements (SARs) as specified in CC-part3

• Security Requirements for the IT Environment

• Security Requirements for the non-IT Environment

• A rationale

• Additional SFRs and SARs can be formulated.


57

Security Functional Requirements (SFRs) and Assurance Requirements (SARs)

Security Functional Classes Security Assurance Classes

Class FAU: Security audit Class ACM:Configuration

management

Class FCO: Communication Class ADO:Delivery and

operation

Class FCS: Cryptographic

support

Class ADV:Development

Class FDP: User data

protection

Class AGD:Guidance documents

Class FIA: Identification and

authentication

Class ALC:Life cycle support

Class FMT: Security management Class APE:Protection Profile

evaluation

Class FPR: Privacy Class ASE:Security Target

evaluation

Class FPT: Protection of the

TSF

Class ATE:Tests

Class FRU: Resource

utilisation

Class AVA:Vulnerability

assessment

Class FTA: TOE access

Class FTP: Trusted

path/channels


58

Evaluation Assurance Levels

• A PP also defines an Evaluation Assurance Level which in fact is a package of SARs. The CC distinguishes 7 EAL levels from EAL1 to EAL7.


59

Security Targets

• In some cases the Protection Profiles contains more SARs than necessary for a certain EAL level. In that case one uses the term ‘augmented’. So EAL 4 augmented (or EAL4+) means all SARs required in EAL 4 plus some additional ones.

• When creating a product in compliance with a PP, the manufacturer creates a Security Target for its product. The manufacturer refers to the PP.

• In the evaluation process the product (TOE) is evaluated against the SFRs by the evaluator (‘laboratory’) in accordance with the SARs.

• Based on the evaluation report typically another party certifies the product, but in some schemes it is the laboratory itself. In Germany, the Bundesamt für Sicherheit in der Informationstechnik performs the accreditation of the laboratories (‘Prüfstelle’) and issues the certificates based on the evaluations.


60

PP example IT security product certification audits (‘common criteria’)

61

PP example IT security product certification audits (‘common criteria’)

62

Security Target example IT security product certification audits (‘common criteria’)

63

Security Target example IT security product certification audits (‘common criteria’)

64

Certificate example IT security product certification audits (‘common criteria’)

65

NL Certificate example IT security product certification audits (‘common criteria’)

http://dl.dropbox.com/u/6343869/My%20First%20Common%20Criteria.pdf

http://dl.dropbox.com/u/6343869/My First Common Criteria.pdf

66

Certification of Secure Signature Creation

Devices (SSCDs) • An SSCD is a combination of hardware (‘chip’),

‘generic operation system’ and application.

• Nowadays many chip applications are applets based on the Java platforms (‘Javacards’). In many cases the chip and the Java virtual machine (called ‘JCOP’ for NXP) are separately certified.

• In the Dutch context the certification of the SSCD can be replaced by (roughly) by a suitably certified platform (chip + java VM) and a tested JAVA applet.

• See http://www.ecp.nl/sites/default/files/TTP-NL_GuidanceNote2_June_2010.pdf


http://www.ecp.nl/sites/default/files/TTP-NL_GuidanceNote2_June_2010.pdf



67

Links for certified products

• http://www.commoncriteriaportal.org/products.html

• https://www.bsi.bund.de/cln_134/DE/Themen/ZertifizierungundAkkreditierung/ZertifizierungnachCCundITSEC/ZertifizierteProdukte/zertifizierteprodukte_node.html


http://www.commoncriteriaportal.org/products.html

https://www.bsi.bund.de/cln_134/DE/Themen/ZertifizierungundAkkreditierung/ZertifizierungnachCCundITSEC/ZertifizierteProdukte/zertifizierteprodukte_node.html




68

Outline






69

•Recap & Practicum

•See blackboard

•Please submit to [email protected] before

28 November 2011

• Room: 02.047

information security audits & certificationkursawe/sio2011/slides/lecture9.pdf · • an...

Documents