Download - Evaluation of application systems
Evaluation of application systemsby ISO/IEC TR 19791
JEITA IT Security Center
September 25, 2005
ICCCICCCICCC
1. Outline and Goals
�� ISO/IEC TR 19791ISO/IEC TR 19791 - Security techniques for system evaluation - being developed in order to evaluate suitably a “system”.
�� JEITA ITSCJEITA ITSC has carried out the pilot evaluation of an application system according to the ISO/IEC TR 19791.
�� This presentationThis presentation introduce a notion of how to evaluate the application system.
1.1. Outline
1.2. Goals
�� EstablishingEstablishing trusted security standard for ”IT System evaluation”.
�� PromotingPromoting effectively and efficiently evaluation method for “IT System”.
2. Background
� “System ” is defined as ...
2.1. Definition of “System”
A specific IT installationA specific IT installation, with a particular purpose and operational environment which covered by
personnelpersonnel, proceduresprocedures, processesprocesses, and physical measuresphysical measures.
2.2. Examples of “System”
� Ministry of A, Online application system
� B City, The Resident Register system
� Online Bunking System of C Bunk
� Inventory and logistics management system of D Factory
� :
2.3. Characteristics of “System”
� “System ” consists of several productsseveral products.
� The specific IT installationspecific IT installation. (There is only one in the world.)
� The location and facilities exist actually.
� The peripheral equipments exist actually.
� The operational procedures exist actually.
� The operators are uniquely identified.
(The users may be uniquely identified.)
2.3. Characteristics of “System”
� “System” configuration is changed frequentlychanged frequently.
� ProgramsPrograms are changed frequently.
� EquipmentEquipmentss are changed frequently.
� Roles of personnelRoles of personnel are changed frequently.
� “System” is across the four life cycle stagesfour life cycle stages –i.e. Development/IntegrationDevelopment/Integration, InstallationInstallation, OperationOperation and Maintenance stageMaintenance stage.
2.4. Methodology of “System evaluation”
ISO/IEC TR 19791 is being developed and will be merged into ISO/IEC 15408.
Several products.
Procedures and rules based on security objectives.
Operational environment which changes frequently.
Characteristics of “System”
EvaluateEvaluate
2.5. Feature of the “System Security Target”
� No “Assumption”.
� STOE is operated in an actually existing environment. STOE is located in ““Real siteReal site””.
� No “IT environment”.
� STOE shall be defined as a whole of whole of ““System System ””.
� Identify “Risks” instead of “Threats”.
� ““RiskRisk”” is defined as ““ThreatThreat”” and and ““VulnerabilityVulnerability””in operational environment for STOE.
TR 19791TR 19791
2.6. Structure of “System ST”
STOE Reference, Overview and Description.STOE Reference, Overview and Description.
SST IntroductionSST Introduction
STOE Security problem definitionSTOE Security problem definition
RisksRisks
Organizational
Security
Policy
Organizational
Security
Policy
Security objectives for the STOE
Security objectives for the STOE
Security objectives for the operational env.
Security objectives for the operational env.
Security objectives for the development env.
Security objectives for the development env.
Security objectivesSecurity objectives
Functional requirements
Functional Functional
requirementsrequirementsOperational
requirementsAssurance
requirements
Assurance Assurance
requirementsrequirements
Security functions
Security Security
functionsfunctionsOperational
controlsAssurance measures
Assurance Assurance
measuresmeasures
STOE Summary specification
STOE Summary specification
STOE Security requirements
STOE Security requirements
TR 19791TR 19791
3. Present work
ITSC has been evaluated following model system in order to verify the ISO/IEC TR 19791.
External
operational
systems
Internet
Leased
line
Data storage
system
Backup
system
Intranet
Firewall,
Load
sharing
device
STOE:STOE:
Fire
wall
Database server,
Application server
Virus check server,
WCM server,
Load sharing device
Web server,
Mail server
DMZ
Data Management service
User Data
IDS
User Administrator
Users Personnel
Personnel
Internet Data Center
VaultVault Box server systemBox server system
Monitoring
system
3.1. Feature of system model
Vault BoxVault Box
3.2. Technical requirements and operational requirements
Access controlAccess control
InformationInformation
flow Controlflow Control
Identification andIdentification and
AuthenticationAuthentication
Session LockingSession Locking
Trusted PathTrusted Path
Management of security attributeManagement of security attribute Management of TSF dataManagement of TSF data
Security policySecurity policy Risk management withRisk management with
--in the organizationin the organization
Network SecurityNetwork Security
Audit LogsAudit Logs
Legal adviceLegal advice
Protection of assetsProtection of assets
Impact analysisImpact analysis
Vault BoxVault Box
Protected assetsProtected assets
TR 19791TR 19791Logical scope of the model systemLogical scope of the model system
4. Issues and Suggestions
① Unique identification of a “System ”.� How can uniquely identify a system ?
Configuration items are changing frequently.
② Identification of “Risks ”.� Evaluator cannot determine what level of risks
should be countered by STOE, since there are too many candidates of “Risks”.
③ “Documentation” cannot be provided.� As “System” may contain third party products,
design documents of the third party products cannot be provided.
4.1. What is Problem ?
4.2. Unique identification of a “System”
�� ““System System ”” configurationconfiguration is changed is changed frequently.frequently.
� The programsprograms are updated at any time.
� The equipmentequipmentss are exchanged at any time.
� The personnel changespersonnel changes occur at any time.
How can uniquely identify a system ?How can uniquely identify a system ?
4.2. Unique identification of a “System”
�� Changes of STOEChanges of STOE are able to be accepted, if the configuration management rulesconfiguration management rules or baseline management rulesbaseline management rules are obeyed.
� STOE continues to have an ““Identical Identical versionversion”” easily identified from users within the acceptable changesacceptable changes.
The acceptable changesThe acceptable changes do not have an effect on “Risks”.
� Updating of the programs for bug fix in accordance with baseline management rule.
� Exchange of the hard disk device with disk failure in accordance with assets maintenance rule.
� Personnel change according to office regulations.
Acceptable changesAcceptable changes
� Add new external interface.
� Add new security function, or remove security function.
� Add fundamental device which is not described in SST.
� Change owner in the organization which manages STOE.
Unacceptable changesUnacceptable changes
4.2. Unique identification of a “System”
((Cases when Cases when ““RisksRisks”” are not influenced.)are not influenced.)
((Cases when Cases when ““RisksRisks”” are influenced.)are influenced.)
4.2. Unique identification of a “System”
:::
2005/07/24
ver.1.2
2005/04/28
Ver.1.1
2005/04/02
Ver.1.0
Application program
S/W
2005/03/25DBMS ***S/W
------2005/03/25
Ver.**
Linux ***S/W
:::::
---2005/04/102005/03/15switchH/W
------2005/03/15FirewallH/W
…change date (and version)
change date (and version)
install date (and version)
Item name
Example of “Configuration list” of the model system.
Vault Box Server System version 1.01.0
It isIt is
““Identical versionIdentical version””
within acceptable within acceptable
changes.changes.
4.3. Identification of “Risks ”
Some Threats can be excluded by identifying “assumptions”.
Assumption
(Potential Threats)(Potential Threats)
Product TOE
All Risks have to be identified.(No assumption allied.)
TOSP
T: ThreatR: RiskV: Vulnerability
T T
System TOE
OSPR V
V
R R
V
R
V
V
R
4.3. Identification of “Risks ”
� Various countermeasures are implemented in the environment of STOE. -e.g.
� The “System” is installed in facility which takes account of natural disaster.
� The operators of the “System” possess proficient knowledge of a level.
Evaluator cannot determine what level of Evaluator cannot determine what level of
risks should be countered by STOE, since risks should be countered by STOE, since
there are too many candidates of there are too many candidates of ““RisksRisks””..
4.3. Identification of “Risks ”
� Developer/integrator should describe the result of the risk assessmentresult of the risk assessment in the SST with reasons why the developer/integrator identified or did not identify candidates of risk as risks.
4.3. Identification of “Risks ”
Example of “Security problem definition” in the SST.Result of risk assessment in the model system.
The capture is impossible, since the network is closed leased line.
NoAn attacker may capture data being transferred across a network.
3
STOE is installed in the fifth floor.
NoSTOE may be damaged by flood.
2
A possibility of malice should be considered.
YesThe administrator may do injustice.
1
Reason of the judgeJudgeCandidates of risk
4.4. “Documentation” cannotbe provided
�� NoNo design documents of third party design documents of third party products are provided.products are provided.
� The design documents of multipurpose OS cannot be provided.
� The design documents of general purpose package products cannot be provided.
How should evaluate third party products ?How should evaluate third party products ?
4.4. “Documentation” cannotbe provided
� The portion without evaluation evidence should be separated as a “DomainDomain ”.
� This “DomainDomain ” can be evaluated by using the interfaces between individual products in the “System ”.
TR 19791TR 19791
I.e. TR 19791 defines “System assurance requirements”.
� ASD_SAD: System architecture design
� ASD_IFS: System interface functional specification etc..
4.4. “Documentation” cannotbe provided
Application
programs
COTS
Products(DBMS/OS)
Web
ServerPrint
Server
DB
Server
Clie
nt
Clie
nt
Clie
nt
Domain A
Domain B
Domain C Domain D
STOE
Intra LAN
Ap
pli
cati
on
Serv
er
Requires
EAL4
EAL1 Certificat
ed
Requires
EAL3
Requires
non-EAL
Domain is defined as…
A portion of an operational A portion of an operational
system that implements the system that implements the
same set of security policies.same set of security policies.
TR 19791TR 19791
Philosophy of Philosophy of
““DomainDomain””
4.4. “Documentation” cannotbe provided
The application
programs developed
by own vender.
STOESTOE
Evaluate by the security assurance requirements according to EAL.
Evaluate by the
operational system operational system assurassur
--anceance requirementsrequirements by using interfaces between individual products.
Domain-A
The third vender product.
The third vender product.
TR 19791TR 19791
Domain-B
Example of domain in model system.
Firewall
Virus check server, WCM server,
Load sharing device
Web server
STOE: Vault Box Server SystemSTOE: Vault Box Server System
OS
CGI of http
Application
server
OS, demon
Application
Database
server
clustermanagement
DBMS
DomainDomain--A:A:
Application domain
DomainDomain--B:B:
Infra domainOS
4.4. “Documentation” cannotbe provided
Requires Requires
EAL2EAL2
RequiresRequires
system assurancesystem assurance
requirementsrequirements
TR 19791TR 19791
4.4. “Documentation” cannotbe provided
X system security target
1. SST introduction
2. Conformance claims
3. Security problem definition
4. Security objectives
5. Security requirements
6. STOE summary specification
X system security target
1. SST introduction
2. Conformance claims
3. Security problem definition
4. Security objectives
5. Security requirements
6. STOE summary specification
7.1. Security domain introduction7.2. Security domain Conformance claims7.3. Security domain security problem definition7.4. Security domain security objectives7.5. Security domains security requirements7.6. Security domain summary specification
7.1. Security domain introduction7.2. Security domain Conformance claims7.3. Security domain security problem definition7.4. Security domain security objectives7.5. Security domains security requirements7.6. Security domain summary specification
Domain part
TR 19791TR 19791Domain part
Domain part
Construction of Construction of ““System STSystem ST””
4.5. Conclusion
①Unique identification of a “System ”.� STOE can continue to have an “Identical versiondentical version”
as far as obeying the life cycle regulation.
② Identification of “Risks ”.� For all candidate risks, developer/integrator
should state “result of the risk assessmentresult of the risk assessment”” in SST.
③ “Documentation ” cannot be provided.� The portion without documentation is able to be
evaluated by using the “system assurance system assurance requirementsrequirements””.
5. Epilogue
� ISO/IEC TR 19791 should be converted into an International Standard to support ISO/IEC 15408 specifically for evaluation of operational “System”.
5.1. Effort to the future
JEITA IT Security Center
List of Operational System Functional Requirements
FOS_POL: Policy for IT systemsFOS: IT Systems
FOD_PSN: Personnel administration
FOD_RSM: Risk management administration
FOD_INC: Incident management administration
FOD_ORG: Security organization administration
FOS_CNF: Configuration of IT systems
FOS_PSN: Personnel control of IT systems
FOS_MON: Monitoring of IT systems
FOS_NET: Network security of IT systems
FOD_SER: Service agreements administration
FOD_POL: Policy administrationFOD: Administration
FamilyClass
Annex-A
FOP_SYS: System equipment
FOP_MOB: Mobile equipmentFOP: Facility and Equipment
FOA_PRO: Privacy data protectionFOA: User Assets
FOA_INF: User assets information protection
FOB_POL: Business policiesFOB: Business
FOB_BCN: Business continuity
FOP_MNG: Facility management
FOP_RMT: Remote equipment
FOP_RMM: Removable equipment
FOS_RCD: Records for IT systems
FOS_OAS: Operational systems assets of IT systemsFOS: IT Systems
FOM_PRM: Management of security parametersFOM: Management
FOM_CLS: Management of asset classification
FOM_PSN: Management of personnel security responsibilities
FOM_ORG: Management of security organization
FOM_INC: Management of security reporting
FOT_MNG: Third party management
FOT_COM: Third party commitmentsFOT: Third Parties
AOD_GVR: Guidance document verification
ASD: Operational System Architecture, Design and Configuration Documentation ASD_SSD: Operational system subsystem design
ASD_IFS: Operational system interface functional specification
ASD_SAD: Operational system architecture design
AOD: Operational system guidance document
ASP: System Protection Profile evaluation
AOD_OCD: Operational system configuration specification
AOD_ADM: Administrator guidance for an operational system
AOD_USR: User guidance for an operational system
ASS: System Security Target evaluation
FamilyClass
List of Operational System Assurance Requirements
Annex-B
ASD_COM: Security concept of operations
ASD_GVR: Design document verification
AOC_OBM: Operational system baseline configuration
AOT_FUN: Operational system functional tests
AOT_COV: Operational system test coverage
AOT_IND: Independent testing
AOT_DPT: Operational system depth
AOT: Operational System Test
AOC_NCP: Non-evaluated component products
AOC_ECP: Evaluated component products
AOC: Operational System Configuration Management AOC_PPC: Conformance with PPs
AOT_REG: Regression testing
ASD_IMP: Implementation representation
ASD_CMP: Operational system primitive component design
AOV_SOF: Strength of operational STOE security functions
AOV_MSU: Operational system misuseAOV: Operational System Vulnerability Analysis
AOV_VLA: Vulnerability analysis
ASO_VER: Verification of operational controls
ASO_RCD: Operation records of operational controlsASO: Records on Operational System
ASI_SIC: Site interoperability check
ASI_AWA: Awareness trainingASI: System Security Installation and Delivery
ASI_CMM: Communication
ASO_MON: Monitoring of operational controls
AOL_DVS: Identification of operational security measures
AOL: Operational System Life Cycle Support