requirements for programming languages in safety and security software standards

9
Computer Standards & Interfaces 14 (1992) 433-441 433 North-Holland Requirements for programming languages in safety and security software standards B.A. Wichmann National Physical Laboratory, Teddington, Middlesex, TWll OLW, UK Abstract Wichmann, B.A., Requirements for programming languages in safety and security software standards, Computer Standards & Interfaces 14 (1992) 433-441. The use of modern programming languages can improve software quality. Two standards which require high quality are examined to understand how the use of programming languages is specified with the objective of satisfying safety and security requirements. Differences are noted in the two standards (ITSEC for security, UK Def Std 00-55 for safety) and suggestions are made to improve these standards. Keywords: Safety; security; ITSEC; programming languages; Def Std 00-55. 1. Introduction The requirements for safety-critical software and those for security software both have an underlying requirement for software quality. In 1991, two important draft standards have been published, one in the safety-critical area [20], and the other in the security area [17]. The Interim Defence Standard 00-55 "The Procurement of Safety Critical Software in Defence Equipment" contains the UK Ministry of Defence's sugges- tions for the future procurement of such software in order for them to meet their safety require- ments. The "Information Technology Security Evaluation Criteria" is an informal standard pub- lished by the European Commission which is broadly similar to the US "Orange Book" [11]. The objective is to lay down a framework to enable security products to be evaluated. Both these standards have requirements for programming languages and compilers. However, the approach taken is very different - which is not necessarily a criticism of either standard. The purpose of this paper is to contrast the two stan- Correspondence to: B.A. Wichmann, National Physical Labo- ratory, Teddington, Middlesex, TWll OLW, UK, email: [email protected] dards and suggest an approach that could im- prove consistency. Two other standards could also have been considered: the avionics standard (which is in an early draft), and the generic IEC safety critical software standard [13]. Since the latest draft of the IEC standard has only just become available to the author, and the avionics draft is very in- complete, neither of these documents are consid- ered here. In fact, the current draft of the avion- ics standard largely ignores the issues raised here; and the IEC standard is less detailed (making revision easier). This paper should be of interest to those con- cerned with the use of programming languages within critical systems and how such concerns can be expressed in standards. Recommendations are made about the use of programming languages in revisions of the two standards considered and, by implications, other standards having similar ob- jectives. 2. The two standards The requirements given in both standards are gratifyingly short, and therefore are reproduced in full here. However, one does need to appreci- Elsevier Science Publishers B.V.

Upload: ba-wichmann

Post on 28-Aug-2016

214 views

Category:

Documents


2 download

TRANSCRIPT

Page 1: Requirements for programming languages in safety and security software standards

Computer Standards & Interfaces 14 (1992) 433-441 433 North-Holland

Requirements for programming languages in safety and security software standards

B.A. Wichmann National Physical Laboratory, Teddington, Middlesex, TWll OLW, UK

Abstract

Wichmann, B.A., Requirements for programming languages in safety and security software standards, Computer Standards & Interfaces 14 (1992) 433-441.

The use of modern programming languages can improve software quality. Two standards which require high quality are examined to understand how the use of programming languages is specified with the objective of satisfying safety and security requirements. Differences are noted in the two standards (ITSEC for security, UK Def Std 00-55 for safety) and suggestions are made to improve these standards.

Keywords: Safety; security; ITSEC; programming languages; Def Std 00-55.

1. Introduction

The requirements for safety-critical software and those for security software both have an underlying requirement for software quality. In 1991, two important draft standards have been published, one in the safety-critical area [20], and the other in the security area [17]. The Interim Defence Standard 00-55 "The Procurement of Safety Critical Software in Defence Equipment" contains the UK Ministry of Defence's sugges- tions for the future procurement of such software in order for them to meet their safety require- ments. The "Information Technology Security Evaluation Criteria" is an informal standard pub- lished by the European Commission which is broadly similar to the US "Orange Book" [11]. The objective is to lay down a framework to enable security products to be evaluated.

Both these standards have requirements for programming languages and compilers. However, the approach taken is very different - which is not necessarily a criticism of either standard. The purpose of this paper is to contrast the two stan-

Correspondence to: B.A. Wichmann, National Physical Labo- ratory, Teddington, Middlesex, TWl l OLW, UK, email: [email protected]

dards and suggest an approach that could im- prove consistency.

Two other standards could also have been considered: the avionics standard (which is in an early draft), and the generic IEC safety critical software standard [13]. Since the latest draft of the IEC standard has only just become available to the author, and the avionics draft is very in- complete, neither of these documents are consid- ered here. In fact, the current draft of the avion- ics standard largely ignores the issues raised here; and the IEC standard is less detailed (making revision easier).

This paper should be of interest to those con- cerned with the use of programming languages within critical systems and how such concerns can be expressed in standards. Recommendations are made about the use of programming languages in revisions of the two standards considered and, by implications, other standards having similar ob- jectives.

2. The two standards

The requirements given in both standards are gratifyingly short, and therefore are reproduced in full here. However, one does need to appreci-

Elsevier Science Publishers B.V.

Page 2: Requirements for programming languages in safety and security software standards

434 B.A. Wichmann

ate the different context and approach that the two standards take. 00-55 takes a prescriptive approach, aimed at a very high level of assurance. In contrast, ITSEC makes no prescriptions in terms of development methods, but requires in- formation to allow independent assessment of the security aspects. ITSEC also has six levels of increasing rigour, from E1 to E6, while 00-55 has no such levels.

Due to the different concerns, direct compari- son does not seem appropriate. However, in both cases, one can understand the concerns being addressed, and therefore question whether these different concerns can be satisfied by similar methods.

Due to the lack of levels in 00-55, one must take a view as to what level in ITSEC, the re- quirements of 00-55 most closely correspond. This is clearly the highest levels, say E5 or E6.

2.1. 00-55

The standard has requirements on both lan- guages and compilers. For languages, section 31.1.2 states:

Safety-Critical Software shall be programmed in a language, or a predefined subset of a language, which shall have the following char- acteristics: 1. A formally defined syntax. 2. A means of enforcing the use of any subset

employed. 3. A well-understood semantics and a formal

means of relating code to the Formal De- sign.

4. Block structured. 5. Strongly typed.

The requirements for language translators ap- pears in section 36 and states:

Compilers shall meet the following require- ments:

1. They shall be validated with an approved international or national validation certifi- cate. The Design Authority shall demon- strate that the certificate relates to the ver- sion and use intended in the project.

2. They shall be verified to the extent indi- cated from the hazard analysis and safety risk assessment. As a minimum they shall have been developed within a recognised

Quality Control System (e.g. AQAP-1/13 or the ISO 9000 series).

3. If the object code is not shown by Formal Arguments to be equivalent to the source code, the compiler shall be classified as safety critical and shall have been devel- oped to the requirements of this standard.

2.2. 1TSEC

Requirements for programming languages and compilers are introduced at four levels: E3, E4, E5 and E6. These can be summarised as follows: E3, EA and E5. Any programming languages used

for implementation shall be well-defined, e.g. as in an ISO standard. Any implementation dependent options of the programming lan- guage shall be documented. The definition of the programming languages shall define unam- biguously the meaning of all statements used in the source code.

E4 and E5. For all compilers used, the implemen- tation options selected shall be documented.

E5. The source code of any runtime libraries shall be provided.

E6. Investigate any suspected inconsistencies be- tween the source code and executable code found during testing using the sponsor sup- plied tools (e.g. a disassembler and /or a de- bugger).

3. Analysis

3.1. Use of levels

The approach used by 00-55 of stating that software is either safety-critical or not, and if it is, all the requirements should apply, seems too harsh for the civil sector. Both the other safety- critical standards [23,13] use levels. The effect of no levels in 00-55 is to inhibit its application in the civil sector. Of course, it may well be appro- priate for the Ministry of Defence's internal re- quirements. However, it seems essential of a sin- gle level approach that this should correspond to a specific level of a generic standard. (Obviously,

Page 3: Requirements for programming languages in safety and security software standards

Requirements for programming languages 435

it might be difficult to obtain agreement on this, especially since 00-55 is so specific.)

3.2. Programming language syntax

The specific requirement in 00-55 for a for- mally defined syntax has no analogy in ITSEC. This seems unfortunate, since it specifically leaves open the issue in ITSEC as to whether 'well-de- fined' implies a formally defined syntax '

The use of BNF to provide a formally defined syntax is straightforward and hence there is no technical barrier to providing the requirements of 00-55. However, several commercial compilers have informally defined extensions for which no syntax is given. If these extensions are used, then there are attendant risks. The author believes that one of the motivations behind the require- ment in 00-55 is to allow the independent pro- duction of tools driven by the syntactic descrip- tion of the language. Such tools can perform a useful role in independent auditing of code - a clear benefit.

The author concludes that the cost of this requirement is small, and the benefits potentially quite significant. Hence it is preferable that this should be a requirement (at some level, in a scheme with levels).

3.3. Programming language semantics

The sentence in ITSEC is rather strong: The definition of the programming languages shall define unambiguously the meaning of all statements used in the source code. Unless this sentence is interpreted suitably, it

cannot be met by current language definitions. To see why this is so, consider the Pascal state- ment:

p := q + r;

w h e r e p , q a n d r a r e integer v a r i a b l e s . T h e I S O

standard only defines the meaning of this state- ment when three conditions are met: • The variable q has been assigned a value. • The variable r has been assigned a value. • ] q + r ] < maxint , where maxint is the largest

integer value supported by the particular im- plementation.

It is not clear if assemblers are permit ted with ITSEC.

If these conditions are not met, then the state- ment is not defined and an implementation (con- forming to ISO-Pascal) can do anything it likes. Moreover, this freedom is exploited by imple- mentations to produce efficient code which will fail in bizarre ways if the conditions are indeed violated. Unfortunately, the above conditions are hard to check statically, and hence it may be impractical to check, at least with complex pro- grams. The example with integer addition above is just about the simplest one can produce. At the other end of the spectrum, conditions on parame- ter passing in Fortran and Ada are much more difficult to state, understand and check. Hence it would be advisable to amend the ITSEC state- ment to read:

The definition of the programming languages shall define unambiguously the meaning of all statements used in the source code subject to clearly stated conditions.

In the author's view, it would be a substantial move forward to require the use of languages with a formally defined semantics. Unfortunately, there is no international consensus on this mat- ter. Although ISO has issued a technical report advocating such definitions [16], and the forth- coming standard for VDM would provide a suit- able definitional method, language groups rarely use such methods. The debate in the revision of the Ada 9X standard was against the use of formal methods in spite of the CEC-funded for- mal definition of Ada 83. The Modula-2 ISO standard is being produced using VDM, but this is not complete and objections to this have been raised (mainly in the USA).

A formally defined standard gives substantial advantages to formal verification tools, since the information required by such tools does not have to be guessed for the text of an informal stan- dard. On the other hand, formal definitions are less accessible to the user community. The ap- proach taken in the Modula-2 standard of having both informal and formal text does create a dan- ger in potential conflicts between the two parts. However, whatever method is used, the possibil- ity of internal inconsistencies cannot be avoided.

In consequence of the above, it does not seem feasible to recommend that programming lan-

Page 4: Requirements for programming languages in safety and security software standards

436 B.A. Wichmann

guages should have formally defined semantics for use in the safety and security areas.

3. 4. Use of standardized languages

The requirements of 00-55 are again more demanding in requiring both the use of a stan- dardized language and also in demanding that the implementation be validated.

The author thinks that both requirements are important, and that the small cost involved means that it should be a requirement at the lowest level. Of course, these small costs build upon the large investments made in both the standards and the validation suites.

An important aspect of programming lan- guages is that they should be well understood by the programmers. Good textbooks are essential for this, and these are rarely available for non- standard languages. Moreover, many vendor- specific languages have a disclaimer about changes to the language which makes them un- suitable for long-term projects. For many com- mercial projects, it is important to mount the software on various hardware platforms accord- ing to market requirements. This is virtually im- possible with proprietary products.

The dangers inherent in non-standard dialects are even worse. For an illustration of some of the problems, see Appendix A.

The 'cost' in the use of validated, standard- conforming compilers is mainly in restricting the developer's choice. In view of the dangers noted above it seems, at the very least, that the use of non-standard languages should be justified, and at higher levels, only validated compilers should be used.

There are also important questions to consider in the choice of an appropriate standard lan- guage, see [9,12].

3.5. Use of language subsets

It is widely recognised that language subsets are not only useful for the development of critical software but are virtually essential for the valida- tion of source code [9,7].

The higher levels of ITSEC are based upon examination of the source code, and hence, the omission of any requirement on the language used is surprising. On the other hand, at the

lower levels in which code inspection is not un- dertaken, the use of a subset is not directly rele- vant to evaluation.

One problem with subsets is to ensure that they are enforced. The specific requirement in 00-55 that a tool should enforce the subset is potentially quite demanding. For instance, a sub- set requirement may be that recursion is not used. This would then imply a tool to detect recursion, presumably by performing a transitive closure on the call graph. On the other hand, tools could enforce a subset on the programmer which could prevent the best expression of a program. (A tool may not handle floating point which would clearly be unacceptable for some numeric applications.)

Another problem with subsets is their intelli- gent use. For instance, the subset within a high- level language project may exclude assembler routines. However, many safety critical projects require access to peripherals which may demand the use of assemblers. In this context, the use of features outside the subset should require spe- cific authorization.

No currently standardized language could be recommended without reservation for the most critical applications without subsetting. The only attempt at the design of such a language is still on the drawing board [10]. In consequence, the higher levels should require the definition of a subset, with this being enforced partly by tools and partly by quality control procedures. An im- portant potential advantage of a subset is the ability to define the subset formally, which is difficult for a full language (see the section above).

3. 6. Registration of tool vendors

In the context in which compiler errors could cause significant problems, 00-55 requires that the compiler vendor is registered to ISO 9001 or equivalent [6,14,15]. No such requirement is given in ITSEC. There is a significant issue here, since some vendors do not perform proper regression testing on their products, while others certainly do. For a validated compiler, where there is a validation suite, vendors should be required only to release compilers that pass the validation suite. (Some vendors like to be responsive to their customers by providing a rapid response to an error; however, if the revised product has not

Page 5: Requirements for programming languages in safety and security software standards

Requirements for programming languages 437

undergone regression test, this is unlikely to be in the user's best interests.)

One unfortunate fact about ISO 9001 is that it is not widely used in the USA. This implies that the requirement in 00-55 will cause problems due to the desire to use US produced compilers (due to the special requirement here for compilers). Another problem, especially with advanced tools to support formal methods, is that the product may be a university prototype without proper customer support.

Conceptually, there is no doubt that ISO 9001 is the correct approach. Hence pressure should be applied to see if this could become accepted throughout Europe and thus encourage others along the same path. In practice, some transi- tional arrangements may be needed, but estab- lishing this as the long-term goal seems vital.

3. 7. Implementation options

The reference in ITSEC to ' implementation dependent options' is confusing since it is differ- ent from those of the programming language area. The Pascal standard and the ISO technical report [16] defines two concepts ' implementation- defined' and ' implementation-dependent ' , which appear to be different from the concept in IT- SEC. The three terms are as follows:

Implementation-defined (processor-defined in I S O / T R ) . For Pascal, this refers to those as- pects of an implementation of Pascal which can vary between implementations, but must be defined for any implementation. An exam- ple is the maximum integer value. Implemen- tation-defined features are really a portability problem rather than a security or safety risk. A portability problem can become a security or safety risk when a system is ported onto a different hardware platform from that upon which is was developed.

Implementation-dependent (processor-dependent in I S O / T R ) . For Pascal, this refers to those aspects of an implementation of Pascal which can vary between implementations, but need not be defined for any implementation. An example for Pascal is the order of evaluation of the components of an expression. If a Pascal program depends upon such features, it is in-

correct but need not be detected as such by a compiler. This can give rise to security or safety concerns.

Implementation dependent options. The author assumed that this refers to options provided to the user of a specific implementation. For in- stance, most language systems have options to increase or reduce with amount of checking undertaken at run-time, This is clearly a con- cern with security and safety systems.

The first point above is not of concern, but the second two are. The second issue is an example of a language insecurity. Hence, the quality pro- cedures should be required to show how such insecurities have been avoided. For many stan- dard languages, this problem has been carefully studied and hence the problems have been quan- tified [27].

For user compilation options, several issues arise due to the large effect such options can have: 1. Implementations should be validated with the

same options used during development. This is rarely the case and in consequence, gives rise to additional insecurities, see Appendix B.

2. An option to turn optimization on or off can be very important in terms of the reliability of the object code. However, modern compiler optimizations are complex and hard to test adequately. Some testing has indicated that optimization is indeed unreliable [26], but since almost all optimizations are semantically neu- tral, it is hard to quantify what should be turned off.

3. Many compiler options are immaterial to many of the issues under consideration. For in- stance, turning off the compiler listing should make no difference to the object-code pro- duced.

4. Some systems call a linker automatically so that linker options may not be apparent. Some linkers will happily link programs with unsatis- fied references causing insecurities not visible at the programming language level.

3.8. Object code t,ersus source code

With high level languages, one clearly requires that the object code reflects the correct semantics given by the source code and the language defini-

Page 6: Requirements for programming languages in safety and security software standards

438 B.A. Wichmann

tion. If the language definition is not precise enough, then this correspondence cannot be es- tablished and the language must be regarded as being unsuitable.

For the lower (less demanding) levels, one can assume the compiler is correct so that the only issue is the suitability of the language definition.

For the higher levels, there is a significant problem. Compilers are known to have faults and therefore the development procedures must take this into account. The approach taken in 00-55 of showing the equivalence by 'Formal Arguments', is very demanding and cannot be recommended due to the cost (in many circumstances). More- over, the alternative given in 00-55 of formal development of the entire compiler is certainly not feasible for many languages. Hence it seems that ultra highly reliable software which should require the demonstration of the correspondence between source and object code is currently unattainable, or at best, very expensive.

Various research projects would appear to provide a viable solution to this problem [28,21].

3. 9. Formally verified or trusted compilers

Given the great difficulty of establishing the correctness of object code, for highly critical ap- plications, the ability to trust a compiler would be a very substantial benefit. Code verification could then be undertaken at the source-code level. There are many technical problems to be re- solved with such an undertaking, apart for the production of the compiler to industrial stan- dards.

In several industrial sectors, it would be highly advantageous if claims of reliability could be made. However, Professor Littlewood has pointed out that such claims greater that one in 104 cannot be justified by conventional dynamic test- ing [18]. In the author's view, the solution to this dilemma lies in the production of trusted compil- ers for formally defined languages (probably a subset language, see above), together with match- ing formal verification tools for such languages.

The production of such a system would be a major undertaking which in practice, could not be undertaken without public funding. The result would facilitate the production of software in many areas in which public funding predomi-

nates, such as the military, space and public transport sectors.

3.10. Run-time libraries

ITSEC requires access to the source code of run-time libraries at the highest level. This is problematic for several reasons. Firstly, the infor- mation is certainly commercially sensitive, sec- ondly this could be tightly integrated with an operating system, as with the DEC Ada compiler. Lastly, the actual code used in likely to depend upon the constructs used in the source code, and could depend upon options given to the linker.

It is clearly rather easy to subvert a program by substituting a covert run-time library rather than the correct one. However, other checks must be applied to avoid this as well as identifying the run-time actually included within a program. For safety-critical software, the run-time library has no special role, but is one component that has to be addressed in the validation of the final prod- uct.

Hence the divergence between the two stan- dards can probably be justified here.

3.11. Compiler evaluation

Compiler validation merely determines if a specific compiler satisfies the requirements of the standard. The more general requirement of being suitable for the development of critical software is not addressed.

A significant part of this gap can be filled by compiler evaluation facilities which have been proposed for both Pascal and Ada [4]. The Ada Evaluation Suite was developed by the Ministry of Defence, and its use was a requirement in the previous draft of 00-55.

Without compiler evaluation, there is a risk that inappropriate compilers will be chosen for projects merely because they are validated. The evaluation services proposed would provide ob- jective evidence for the choice of a compiler.

3.12. Code verification

This appears to be the end objective of 00-55, but not ITSEC. However, it would appear that 00-55 and ITSEC level E6 would have similar

Page 7: Requirements for programming languages in safety and security software standards

Requirements for programming languages 439

concerns for the (small) fraction of a secure sys- tem with implements the security features.

The application of code verification methods to the actual object code [8] avoids the depen- dence upon the compiler which is so trouble- some. Research to underpin high-level code veri- fication has a long history [3]. Without formally verified compilers, the use of tools to verify source code is clearly limited by potential errors in the compiler. The CEC-funded ProCoS project [2] makes important progress in this area, but the language is rather small and not widely used.

Another approach, which falls short of code verification, but in practice may well provide a similar level of assurance, is a detailed indepen- dent review of the specification and the code. This method has been applied to a highly critical safety application [1].

8. Funding research work for the construction of provably correct compilers for highly critical application should be considered.

9. The use of compiler evaluation services to provide objective evidence for the choice of compilers should be considered.

Acknowledgements

This paper was produced as part of the re- search at NPL funded by the Department of Trade and Industry's Software Quality Unit. The author wishes to thank those who commented upon early drafts, namely Mr. G. O'Neill (NPL), Dr. D. Schofield (NPL), Dr. P.T. Wilkinson (DTI), Mr. B.L. Meek (King's College, London) and Dr. V. Stenning.

4. Conclusions

This is a summary of the points made above. These points could be used to review any revision of the two standards considered or in construct- ing similar standards (or even company quality manuals). 1. Any revision of 00-55 should consider the in-

troduction of levels, or specifically state that the intent is to meet the requirements of the highest level of the IEC standard.

2. Languages should have a formally defined syn- tax.

3. Use of non-standard or unvalidated compilers should be justified, and at higher levels, only validated compilers should be used.

4. For the higher levels, a subset of a standard language should be defined. Adherence to this subset should be enforced by appropriate means, including tools and quality assurance procedures.

5. Mandatory registration of tool vendors to ISO 9001 should be considered.

6. For higher levels, developers should indicate in their quality procedures how high-level lan- guage insecurities have been avoided.

7. For the higher levels, the compiling options used should be compared with those applied during validation to assess the risk arising from language insecurities.

Appendix A. Use of non-standard dialects

Many people would think that a product which has the name of a standard language in its title supports a dialect of the standard language (in a way which may be permitted by the standard). This is not the case. As an example, consider Turbo-Pascal, the biggest selling Pascal-like sys- tem.

To the non-expert, it may appear that Turbo- Pascal is just an easy to use version of Pascal for the IBM-PC. However, when the then current version (4.0) of the compiler was checked by BSI-QA against the validation suite, over 100 tests failed. Some of these were relatively harm- less and arise because Turbo-Pascal has numer- ous extensions to the standard which cannot be disabled by a compiler option. For the full re- sults, see [5].

A summary of the result of the 'validation' is: • 71 correct Pascal programs failed to compile. • 18 correct Pascal programs compiled but failed

on execution with a diagnostic. • 5 correct Pascal programs produced incorrect

results on execution. • 3 tests which were incorrect programs and

should fail to compile, did compile, and on execution, crashed the machine.

• 62 other incorrect programs compiled.

Page 8: Requirements for programming languages in safety and security software standards

440 B.A. Wichmann

Appendix B. Risks arising from compiler options

One prob lem with the def ini t ion of any pro- g ramming language is that of programs which are

incorrect , but this fact canno t be detec ted at compile time. Dif ferent languages take different

approaches to the problem:

• F O R T R A N ignores the p rob lem completely. • C itemizes the problems in an appendix, bu t

does not encourage implemen ta t ions to provide detec t ion facilities.

• Pascal encourages detect ion. • Modula-2 (draft s tandard) encourages detec-

tion, and requires a mode in which some errors

are detected.

• Ada requires detec t ion for the majori ty of such

errors. For any language implementa t ion , commercia l

pressures require good execution speed, which implies minimis ing the checks under t aken . Hence most commercia l products minimise the checks by default . Even for Ada, most implemen ta t ions have an opt ion to remove all checking. (Ada val idat ion procedures allow such an opt ion to be

presen t in an imp lemen ta t i on but this opt ion

cannot be used dur ing val idat ion test ing since the s tandard requires that the checks be per formed

unless explicitly removed by a pragma.)

In consequence of the above, almost all lan- guage systems will allow the compi la t ion and execution of a p rogram which performs an arbi-

trary store overwrite (the most dangerous error

one can conceive). O n the o ther hand, the mode of opera t ion that

unde r t akes the largest a m o u n t of checking can

detect very many errors before anything happens to u n d e r m i n e the integri ty of the system.

References

[1] G.J.K. Asmis, H.O. Tezel and J.D. Kendall, The Cana- dian process for the regulatory approval of safety critical software in nuclear power reactors, lnternat. Conf. on Control & Instrumentation in Nuclear Installations, Glas- gow (May 1990).

[2] D. Bj0rner, A ProCoS project description: ESPRIT BRA 3104, Bull. EATCS, 39 (1989) 60-73.

[3] R.S. Boyer and J.S. Moore, A Verification Condition Generator for FORTRAN, The Correctness Problem in Computer Science (Academic press, New York, 1981).

[4] Ada Evaluation Suite, BSI-QA, Milton Keynes, 1988. [5] A validation report of seven compilers for the IBM PC

and compatibles under MS-DOS, BSI-Q4, Milton Keynes, May 1989.

[6] Quality Assurance, Handbook 22, British Standards Insti- tution, Fourth ed. (1990).

[7] B.A. Carr6 and T.J. Jennings, SPARK - - The SPADE Ada kernel, University of Southampton, March 1988.

[8] D. Clutterbuck and B.A. Carr6, The verification of low- level code, Software Engrg. J. 2 (1988) 97-111.

[9] W.J. Cullyer, S.J. Goodenough and B.A. Wichmann, The choice of computer languages in safety-critical systems, Software Engrg. J. 6(2) (Mar. 1991) 51-58.

[10] I.F. Currie, New Speak - an unexceptional language, Software Engrg. J. 1 (1987) 170-176.

[11] Trusted Computer Systems Evaluation Criteria, DOD 5200.28-5TD, Department of Defense, Dec. 1985.

[12] A.D. Hill, The choice of programming language for highly reliable software - - a comparison of C and Ada, Part 1, Ada User, Vol 12, pp. 11-32, Part 2, pp. 92-103.

[13] IEC/5C65A/WG9/45, Software for computers in the application of industrial safety-related systems, 3rd draft, June 1989.

[14] ISO 9000, Quality management and quality assurance standards - - Guidelines for selection and use.

[15] ISO 9001: 1987, Quality systems - - Model for quality assurance in production and installation.

[16] ISO/TR 10176:1990, Information technology - - Lan- guages - - Guidelines for the preparation of program- ming language standards.

[17] Information Technology Security Evaluation Criteria, Provisional Harmonised Criteria, Version 1,2. 1991 (UK contact point: CESG Room 2/0805, Fiddlers Green Lane, Cheltenham, Glos, GL52 5A J).

[18] B. Littlewood and L. Strigini, Validation of ultra-high dependability for software-based systems (to be pub- lished).

[19] A. McGettrick, Program Verification Using Ada (Cam- bridge University Press, Cambridge, 1982).

[20] Interim Defence Standard 00-55, The Procurement of Safety Critical Software in Defence Equipment, Ministry of Defence (Part l: Requirements; Part2: Guidance)April 1991.

[21] J.S. Moore, PITON: A verified assembly level language, Computational Logic Inc., Sept. 1988.

[22] I.M. O'Neill, D.L. Clutterbuck, P.F. Farrow, P.G. Sum- mers and W.C. Dolman, The formal verification of safety-critical assembly code, presented at the SAFE- COMP 88 IFAC/IFIP Internat. Symp. on Safety Related Computers (Nov. 1988).

[23] Issued in the USA by the Radio Technical Commission for Aeronautics (document RCTA/D0-178A) and in Eu- rope by the European Organization for Civil Aviation Electronics (EURO-CAE document ED-12A).

[24] Rex, Thompson and Partners Ltd., The capabilities of MALPAS - a software verification and validation tool; RTP/4002, April 1987.

[25] C.I.N. Ruggles, ed. Formal Methods in Standards, A Report from the BCS Working Group (BCS/Springer- Verlag, ISBN 3-540-19577-7, 1990).

[26] B.A. Wichmann and M. Davies, Experience with a com- piler testing tool, NPL Report DITC 138/89, Mar. 1989.

[27] B.A. Wichmann, Insecurities in the Ada programming language, NPL Report DITC 137/89, Jan. 1989.

[28] B.A. Wichmann, Low-Ada: An Ada validation tool, NPL Report DITC 144/89, Aug. 1989.

Page 9: Requirements for programming languages in safety and security software standards

Requirements for programming languages 441

B.A. Wichmann has had a long in- volvement with programming lan- guages, from Algol 60 to Ada. He was the member of the Ada language de- sign team responsible for the numer- ics, and was subsequently the found- ing Chairman of Ada-Europe. His current research is in software qual- ity, but is also contributing to the safety and security annex for the Ada 9X standard.