human factors research supporting the anticipated release … · · 2014-02-28human factors...

©2012 Research Integrations, Inc. 1

Human Factors Research Supporting the Anticipated Release of 14 CFR 25.1302

Jennifer Wilson Greg Harron Emily Jones Bob Hoffa Beth Lyall

Prepared for Federal Aviation Administration

Human Factors Division ANG-C1

Revised December 11, 2012


Contents 1. Introduction and Project Objectives .................................................................................... 4 2. Approach and Methods ....................................................................................................... 4

2.1. Interview Participants .......................................................................................................... 4 2.1.1. Demographics ............................................................................................................................................. 5

2.1.1.1. Aircraft and Avionics Manufacturers .................................................................................................. 5 2.1.1.2. Civil Aviation Authorities .................................................................................................................... 6

2.1.2. Level of Familiarity and Experience ............................................................................................................ 6 2.1.2.1. Level of Familiarity with Proposed 14 CFR 25.1302 ........................................................................... 6 2.1.2.2. Level of Familiarity with Proposed AC 25.1302-X ............................................................................... 7 2.1.2.3. Experience with EASA CS 25.1302 ...................................................................................................... 7

2.2. Conducting Interviews .......................................................................................................... 8 3. Findings ............................................................................................................................... 9

3.1. Information Needed ............................................................................................................. 9 3.1.1. Why Has the Proposed Regulation Been Created? ..................................................................................... 9

3.1.1.1. Specific Questions Asked by Participants ......................................................................................... 10 3.1.1.2. Resources Currently Available .......................................................................................................... 10

3.1.2. When Does the Regulation Apply? ........................................................................................................... 10 3.1.2.1. Specific Questions Asked by Participants ......................................................................................... 12 3.1.2.2. Resources Currently Available .......................................................................................................... 12

3.1.3. What Are the Expectations for Methods and Level of Detail for Compliance? ........................................ 13 3.1.3.1. Guidance on Methods of Compliance and Evaluation ..................................................................... 14 3.1.3.2. Guidance for Addressing Error Identification and Error Management ............................................ 18 3.1.3.3. Information-Sharing Resources within FAA ..................................................................................... 20

3.1.4. What Processes Will the FAA Use to Review and Evaluate Compliance with 14 CFR 25.1302? ............... 21 3.1.4.1. How Will the Review and Evaluation Process Be Performed by the FAA? ....................................... 21 3.1.4.2. How Will the FAA Facilitate Consistency in the Review and Evaluation Process for the Proposed 14

CFR 25.1302 Regulation? .................................................................................................................. 24 3.1.5. How Does 14 CFR 25.1302 Relate to Other Regulations and Processes? ................................................. 26 3.1.6. Who Will Be Responsible and Have Authority to Make Compliance Findings with 14 CFR 25.1302? ...... 28

3.1.6.1. Roles and Responsibilities Within the Civil Aviation Authority ........................................................ 28 3.1.6.2. Human Factors Designees ................................................................................................................ 29

3.1.7. Where Can Useful Resources Be Found? .................................................................................................. 30 3.1.7.1. Specific Questions Asked by Participants ......................................................................................... 30 3.1.7.2. Resources Currently Available .......................................................................................................... 30

3.2. Current and Planned Practices ........................................................................................... 31 3.2.1. Resources and Guidance Material Currently Used ................................................................................... 31

3.2.1.1. Information-Related Resources Currently Used ............................................................................... 31 3.2.1.2. People-Related Resources Currently Used ....................................................................................... 32 3.2.1.3. Tool-Related Resources Currently Used ........................................................................................... 33

3.2.2. Certification Documentation .................................................................................................................... 33 3.2.3. Defining Intended Function and Information Requirements ................................................................... 34 3.2.4. Evaluating Urgency, Frequency, and Duration ......................................................................................... 35 3.2.5. Identifying and Mitigating Errors Reasonably Expected in Service ........................................................... 36 3.2.6. Experience with EASA Local and Global Evaluations ................................................................................ 36 3.2.7. Simulator Use to Demonstrate Compliance with EASA CS 25.1302 ......................................................... 37


4. Conclusion ......................................................................................................................... 37 4.1. Lessons Learned ................................................................................................................. 37

4.1.1. Experience with EASA CS 25.1302 ............................................................................................................ 38 4.1.2. New Rule ................................................................................................................................................... 38 4.1.3. Anticipated Impact.................................................................................................................................... 38 4.1.4. Overall State of Preparedness .................................................................................................................. 39 4.1.5. Commonality of Information Needs ......................................................................................................... 40

4.2. Recommended Next Steps ................................................................................................. 41 References............................................................................................................................. 43

Current and Proposed Federal Aviation Regulations .................................................................................... 43 FAA Advisory Circulars ................................................................................................................................... 43 FAA Policies ................................................................................................................................................... 44 Industry Standards ........................................................................................................................................ 45 Other Resources ............................................................................................................................................ 45

Appendix A: Interview Questions for Manufacturers ............................................................. 46 Appendix B: Interview Questions for Civil Aviation Authority Representatives...................... 50


1. Introduction and Project Objectives The proposed regulation FAA 14 CFR 25.1302 is a general applicability FAA regulation that includes explicit requirements for design attributes related to avoiding and managing flight crew error. While FAA 14 CFR 25.1523 and corresponding Appendix D have addressed some related design attributes, this proposed FAA regulation represents a much more global approach to human factors on the flight deck and will require system and equipment designers to consider human error and feedback in their design and testing of flight deck interfaces.

The new proposed regulation is tentatively planned for issuance by the FAA in late 2012 or early 2013. In preparation for the release of the proposed 14 CFR 25.1302 regulation, the FAA has funded this research project to gather information about the state of industry and FAA understanding and practices that may impact the effective implementation and compliance with the proposed 14 CFR 25.1302 regulation.

The FAA 14 CFR 25.1302 was initially developed as a harmonized regulation with the EASA CS 25.1302 regulation. Since the release of the EASA regulation in 2006, several manufacturers have had projects that required demonstrating compliance with the EASA CS 25.1302 regulation. In addition, other aircraft and avionics manufacturers have integrated human factors-related practices into their design and testing processes in preparation for CS 25.1302 and the likely future release of the proposed 14 CFR 25.1302. All of these experiences can provide useful information in preparation for implementation of the proposed FAA regulation in the U.S.

The research presented here is intended to provide a foundation to support further development of information and other resources that may contribute to the effective implementation of the proposed 14 CFR 25.1302 regulation.

2. Approach and Methods To perform this research, structured interviews were conducted with individuals from both civil aviation authorities and manufacturers within the civil aviation sector to gather information related to demonstrating and evaluating compliance with the proposed 14 CFR 25.1302 regulation.

2.1. Interview Participants Information was gathered from aircraft and avionics manufacturers who will be required to comply with the proposed regulation, as well as personnel at the civil aviation authorities who oversee compliance with the EASA CS 25.1302 and, when released, will oversee compliance with the proposed FAA 14 CFR 25.1302.

In the early stages of the project, the FAA Technical Sponsors identified a sampling of manufacturers and individuals from civil aviation authorities that they felt would have valuable, relevant information to offer related to their compliance experiences, plans, and/or unique


perspectives. Representatives from each company and organization who were suggested by the FAA Technical Sponsors, as well as a sampling of others who had knowledge of the proposed 14 CFR 25.1302 regulation and/or experience with the existing European EASA CS 25.1302 regulation or, were interviewed.

2.1.1. Demographics The interview participants represented a variety of aspects of the aviation industry that will be impacted by the new proposed regulation. A total of 43 people participated in the interviews including 29 individuals from aircraft or avionics manufacturers and 14 individuals from civil aviation authorities. Thirty-six participants represented U.S. companies and organizations and 7 participants represented companies and organizations from non-U.S. countries. The majority of non-U.S. countries represented in the interviews were European.

2.1.1.1. Aircraft and Avionics Manufacturers A sample of individuals from manufacturers representing important distinguishing characteristics of the overall population of manufacturers that will be required to comply with the proposed 14 CFR 25.1302 regulation were interviewed. The sample of individuals from manufacturers included the following:

• Avionics manufacturers • Aircraft manufacturers • Large-sized companies • Medium-sized companies • Small-sized companies • U.S. manufacturers • Non-U.S. manufacturers

Potential participants from the manufacturers were contacted either directly or via an introduction from one of the FAA Technical Sponsors of the project. Most of the individuals from manufacturers who were contacted responded and agreed to participate in the project. A small number of individuals from the manufacturers did not respond to the request. For those individuals who agreed to participate, an interview was set up, and the interview was conducted either in person or over the phone.

A total of 29 individuals representing 10 different aircraft or avionics manufacturers were interviewed as part of this project. The following table presents a breakdown by organization type of the participants from the aircraft and avionics manufacturers.

Organization Type # of Participants # of Organizations Aircraft Manufacturers 10 4 Avionics Manufacturers 19 6

TOTAL = 29 10


The following table presents a breakdown by country of the participants from the aircraft and avionics manufacturers. The majority of non-U.S. countries represented in the interviews were European.

Country # of Participants # of Organizations U.S. 24 7 Non-U.S. 5 3

TOTAL = 29 10

2.1.1.2. Civil Aviation Authorities A sample of individuals from civil aviation authorities representing important distinguishing characteristics of the overall population of individuals who will evaluate compliance with the proposed 14 CFR 25.1302 regulation or who have been involved in evaluating compliance for the EASA CS 25.1302 regulation were interviewed. The sample of individuals from the civil aviation authorities included the following:

• Human Factors Specialists • Flight Test Engineers • Flight Test Pilots • Aircraft Certification Offices (ACOs) • U.S. Civil Aviation Authority (i.e., FAA) • Non-U.S. Civil Aviation Authorities (i.e., EASA and Transport Canada)

Potential participants from the civil aviation authorities were contacted either directly or via an introduction from one of the FAA Technical Sponsors of the project. All individuals from the civil aviation authorities who were contacted responded, and a majority of those individuals agreed to participate in the project. A small number of individuals from civil aviation authorities were unable to make time to participate due to their current workload. For those individuals who agreed to participate, an interview meeting was set up, and the interview was conducted either in person or over the phone.

A total of 14 individuals from the civil aviation authorities were interviewed as part of the project, 12 from the U.S. and 2 from outside the U.S.

2.1.2. Level of Familiarity and Experience This section describes the level of familiarity of the interview participants with the proposed 14 CFR 25.1302 regulation and the proposed AC 25.1302-X as described in their interviews.

2.1.2.1. Level of Familiarity with Proposed 14 CFR 25.1302 All the interview participants had some degree of familiarity with the NPRM language of proposed 14 CFR 25.1302 regulation. Most of the individuals from the civil aviation authorities who participated in the interviews were very familiar with the proposed regulation. However,


there were a few who had only read the proposed 14 CFR 25.1302 once or twice prior to our interview. The level of familiarity of the individuals from the aircraft and avionics manufacturers was more varied than those from the civil aviation authorities, but all had some level of familiarity with the proposed 14 CFR 25.1302 regulation. Over half of the aircraft and avionics manufacturers interviewed were very familiar with the NPRM language of proposed 14 CFR 25.1302. However, a few manufacturers had only read the proposed 14 CFR 25.1302 once or twice prior to our interview.

Several of the individuals interviewed had participated in the Human Factors Harmonization Working Group (HF HWG) that developed the initial draft of the regulation. The HF HWG was an Aviation Rulemaking Advisory Committee (ARAC) under the Transport Airplane and Engine Issues Group (TAEIG). A link to the final report of the HF HWG can be found in the References section of this report.

The sample of individuals interviewed participated in the project because they had an interest in and information and experience to offer related to the proposed 14 CFR 25.1302 regulation. Therefore, it is likely that the level of familiarity of our participants is significantly higher than the general population of individuals from aircraft and avionics manufacturers and from civil aviation authorities.

2.1.2.2. Level of Familiarity with Proposed AC 25.1302-X While almost everyone interviewed had familiarity with the NPRM language of the proposed 14 CFR 25.1302 regulation, only half of the interview participants had some degree of familiarity with the proposed Advisory Circular AC 25.1302-X. Many of the participants who did not have familiarity with the proposed Advisory Circular prior to being approached to participate in our interview expressed that they had tried to find it but were unable to locate a copy.

The majority of the individuals from the aircraft and avionics manufacturers and from the civil aviation authorities who were familiar with the proposed Advisory Circular stated that they were very familiar with it. These participants had either specifically compared it with the EASA Acceptable Means of Compliance AMC 25.1302 and/or had participated in the Human Factors Harmonization Working Group (HF HWG) that developed an initial draft of the 14 CFR 25.1302 regulation and the associated Advisory Circular.

2.1.2.3. Experience with EASA CS 25.1302 There was less experience across industry with the EASA CS 25.1302 than initially expected. Due to the time that typically elapses between the establishment of the Certification Basis and the completion of the certification process for larger products, very few aircraft and avionics manufacturers had experience complying with EASA CS 25.1302. Some aircraft and avionics manufacturers had experience demonstrating compliance with various EASA Certification Review Items (CRIs) that contained some similar requirements to EASA CS 25.1302.


A total of 9 individuals from the aircraft and avionics manufacturers and from the civil aviation authorities who were interviewed as part of the project had specific experience either with EASA CS 25.1302 or with a CRI that contained similar requirements. The following is a breakdown by organization type:

2.2. Conducting Interviews Two similar sets of interview questions were developed to conduct the interviews: one for individuals from the aircraft and avionics manufacturers (see Appendix A) and another for individuals from the civil aviation authorities (see Appendix B). The concepts questioned were the same, but the approach to the questions was appropriate for the roles of those being interviewed. A structured interview approach was used in which the questions were used as a guide for the interviews.

The areas of the proposed 14 CFR 25.1302 regulation that had the greatest potential to cause concern and difficulty for aircraft and avionics manufacturers and civil aviation authorities were defined based on input from the FAA Technical Sponsors. The interview questions were developed to focus on those specific areas of concern instead of systematically going through each paragraph of the regulation. The questions asked covered a variety of information, including the following:

• Demographics and background of our interview participants • Familiarity with the proposed regulation • Experiences with EASA CS 25.1302 and FAA 14 CFR 23/25/27/29.777 regulations • Anticipated impact of the proposed regulation • Resources currently used and additional resources needed • Planned test and evaluation methods to use for demonstrating compliance • Identifying errors reasonably expected in service and demonstrating compliance with

paragraph (d) • Defining information requirements and demonstrating compliance with paragraph (a) • Designing for task urgency, frequency, and duration and demonstrating compliance with

paragraph (b)(2)

The data were gathered by conducting interviews with the individuals from the aircraft and avionics manufacturers and the civil aviation authorities. The interview participants were asked questions as appropriate to their experience, expertise, interests, and the time constraints of the interviews. During the interview, notes were taken and recordings were made (if the participants agreed) to ensure that the relevant information shared was accurately captured. Examples of the

Organization Type # of Participants # of Organizations Aircraft & Avionics Manufacturers 8 4 Civil Aviation Authorities 1 1

TOTAL = 9 5


questions used to guide the interviews are presented in Appendix A (for manufactures) and Appendix B (for civil aviation authorities).

3. Findings This section describes what was learned from the interviews with the individuals from the aircraft and avionics manufacturers and from the civil aviation authorities. Section 4.1 and its subsection describes finding related to information that may be needed to enhance the effectiveness of implementation of the proposed regulation, and Section 4.2 describes what was learned about current practices being used by the interview participants.

3.1. Information Needed This section describes the information identified that the participants need to accurately and adequately understand 14 CFR 25.1302 and the processes necessary to demonstrate, evaluate, and review compliance with it. The section is organized using five primary questions that were raised by participants who were interviewed:

• Why has the proposed regulation been created? (Section 3.1.1) • When does the regulation apply? (Section 3.1.2) • What are the expectations for methods and level of detail for compliance? (Section 3.1.3) • What processes will the FAA use to review and evaluate compliance with proposed 14

CFR 25.1302? (Section 3.1.4) • How does proposed 14 CFR 25.1302 relate to other regulations and processes? (Section

3.1.5) • Who will be responsible and have authority to make compliance decisions with proposed

14 CFR 25.1302? (Section 3.1.6) • Where can useful resources be found? (Section 3.1.7)

3.1.1. Why Has the Proposed Regulation Been Created? Individuals from the aircraft and avionics manufacturers and from the civil aviation authorities stated that it would be useful to better understand the intent and rationale behind the proposed 14 CFR 25.1302 regulation. Specifically participants asked for background information about proposed 14 CFR 25.1302 including a description of intent of the regulation, the rationale behind the regulation, why it was created as a separate regulation, and what the FAA will be looking for and hoping that the proposed regulation will achieve.

Participants felt that a better understanding of the background and rationale would support an accurate understanding of the regulation and allow them to more effectively address it.


3.1.1.1. Specific Questions Asked by Participants The following is a list of questions identified in the interviews related to why the proposed regulation has been created.

• What is the intent and rationale behind the proposed 14 CFR 25.1302 regulation? • Why was proposed 14 CFR 25.1302 created as a separate regulation? • What is the FAA hoping proposed 14 CFR 25.1302 will achieve?

3.1.1.2. Resources Currently Available Resources currently available that include information related to why the proposed 14 CFR 25.1302 was developed include:

• Preamble to proposed 14 CFR 25.1302 The preamble published in the Federal Register as a Notice to Proposed Rulemaking (NPRM) provides background information about the proposed 14 CFR 25.1302 regulation and the reasons why the regulation has been proposed.

• Human Factors Harmonization Working Group (HF HWG) Final Report The HF HWG Final Report submitted to the Transport Airplane and Engine Issues Group (TAEIG) of the Aviation Rulemaking Advisory Committee that contains a description of the work that the group did to develop an initial draft of the proposed 14 CFR 25.1302 regulation and the associated proposed Advisory Circular.

• Proposed Advisory Circular AC 25.1302-X The proposed Advisory Circular AC 25.1302-X includes information on the rationale and intent behind the proposed regulation in the introductory sections.

An assessment may be necessary to determine if these resources contain adequate information to address all of the information needs related to the intent and rationale behind the proposed regulation.

See Reference listing at the end of this report for links to access these documents.

3.1.2. When Does the Regulation Apply? Individuals from the aircraft and avionics manufacturers and from the civil aviation authorities expressed the need to have a better understanding of the scope of the proposed 14 CFR 25.1302 regulation. Participants felt that having a clear definition of when the proposed regulation will apply would help provide clear expectations when planning for new certification projects. It would also ensure that the intended scope of how the proposed 14 CFR 25.1302 regulation is applied is consistent with the understanding and application of what it will be in practice. Additionally, they stated that this definition would provide a shared understanding between the manufacturer and the civil aviation authority of the level of scrutiny and methods of compliance appropriate for demonstrating compliance.


The information requested by participants to gain a better understanding of when the proposed 14 CFR 25.1302 would apply falls into three distinct areas. However, many participants discussed these areas as a single area of concern. These areas include the following:

• Which equipment will be evaluated with additional scrutiny against proposed 14 CFR 25.1302 – Participants asked for more detailed information related to how to determine whether a design feature should be considered highly integrated, complex, and novel and, therefore, should be evaluated with additional scrutiny against the proposed 14 CFR 25.1302. When 14 CFR 25.1302 is included in the Certification Basis, the proposed regulation and proposed advisory circular state that the systems and equipment that will be evaluated with additional scrutiny against 14 CFR 25.1302 will be determined by considering the flight deck controls, information, and system behavior that involve flight crew interaction; the level of integration of the design features with other flight deck systems; the level of complexity of the design features; and the degree of novelty of the design features. As described in the proposed AC 25.1302-X, an agreement between the applicant and the FAA will determine which systems and equipment will be reviewed with additional scrutiny. Therefore, participants wanted to gain a better understanding related to expectations of how the determination will be made about which systems and equipment possess design features considered highly integrated, complex, and novel.

• The extent that integrated systems and equipment will be evaluated for compliance with 14 CFR 25.1302 – Because the systems and equipment that possess highly integrated, complex, and novel design features are connected with other systems and equipment, the interaction between these systems may have implications that would require additional scrutiny of connected systems and equipment with regard to proposed 14 CFR 25.1302. Manufacturers stated concern regarding not having a clear understanding of the other systems and equipment with which the new or modified system is integrated would also be evaluated against the proposed 14 CFR 25.1302, as well as what level of scrutiny these other systems and equipment would require. Several manufacturers expressed the need for the definition of clear boundaries in order to avoid the entire flight deck being opened up to evaluation because of one change being made.

• Impact of the changed product rule – While not specific to the proposed 14 CFR 25.1302, the participants discussed issues related to the changed product rule and asked for information related to determining how much design change would require the Certification Basis to be updated to include the proposed 14 CFR 25.1302. The changed product rule, 14 CFR 21.101, is used to establish the Certification Basis of a certification project that addresses a design modification. The regulation 14 CFR 21.101 and AC 21.101-1A provide criteria and guidance to determine whether or not a specific change should be considered significant for establishing the certification basis for a changed product, and guidance for identifying whether it will be necessary to apply for a new type certificate. There appears to have been confusion among some of the participants about the distinction between the changed product rule (that impacts the Certification Basis)


with the process of making the determination about which systems will be reviewed with additional scrutiny against proposed 14 CFR 25.1302. Because of this, it may be useful to provide information for or access information to available resources that help others understand this process and make such a distinction.

3.1.2.1. Specific Questions Asked by Participants The following is a list of questions identified in the interviews related to when the proposed 14 CFR 25.1302 regulation will apply:

• What is the intended, appropriate interpretation of the proposed regulation? • When might it be required to make changes to previously-approved designs that are

incorporated into a new application? • Which related systems and equipment will the regulation be applied to when a design

modification is made? • What level of change in the design of previously approved equipment will prompt an

update to the certification basis to include the proposed 14 CFR 25.1302 regulation? • Would a previously-approved design be considered to be a novel design if it is integrated

differently with other functions? How will this determination be made? • What level of change will trigger the regulation to be added to the Certification Basis for

a modified design? • How will differences in interpretation be resolved?

3.1.2.2. Resources Currently Available Resources currently available that include information related to when the proposed 14 CFR 25.1302 regulation will apply include the following:

• Preamble to proposed 14 CFR 25.1302 The preamble published in the Federal Register as a Notice to Proposed Rulemaking (NPRM) provides background information about the proposed 14 CFR 25.1302 regulation.

• Proposed Advisory Circular AC 25.1302-X The proposed Advisory Circular AC 25.1302-X provides guidance to determine whether a feature should require more scrutiny against 14 CFR 25.1302. Section 4-2 of the proposed AC 25.1302X provides a discussion of integration, complexity, and novelty.

• Human Factors Harmonization Working Group (HF HWG) Final Report The HF HWG Final Report submitted to the Transport Airplane and Engine Issues Group (TAEIG) of the Aviation Rulemaking Advisory Committee that contains a description of the work that the group did to develop an initial draft of the proposed 14 CFR 25.1302 regulation and the associated proposed Advisory Circular AC 25.1302-X.


• FAA Regulation 14 CFR 21.101, Designation of Applicable Regulations This FAA Regulation provides criteria to be used to determine whether or not a specific change should be considered significant.

• FAA Advisory Circular AC 21.101-1A, Establishing the Certification Basis of Changed Aeronautical Products This Advisory Circular provides guidance for establishing the certification basis for a changed product as well as guidance for identifying if it will be necessary to apply for a new type certificate.

• FAA Order 8110.48, How to Establish the Certification Basis for Changed Aeronautical Products This Order provides information for establishing a certification basis when approval is sought for changed products.

• FAA Order 8110.105, Simple and Complex Hardware Approval Guidance This Order provides information to supplement RTCA DO-254 and includes information about how to define simple and complex hardware for use in the certification process.

An assessment may be necessary to determine if these resources contain adequate information to address all of the information needs related to when the proposed 14 CFR 25.1302 regulation will apply.


3.1.3. What Are the Expectations for Methods and Level of Detail for Compliance? Individuals from the aircraft and avionics manufacturers and from the civil aviation authorities expressed uncertainty about exactly what the expectations would be for showing and finding compliance with the proposed 14 CFR 25.1302 and how they would know when they had done enough to demonstrate compliance. There were concerns that minimum standards for compliance are not yet well-defined and, therefore, may be based on the judgment of different individuals from the civil aviation authority.

Individuals from the aircraft and avionics manufacturers and from the civil aviation authorities discussed the difficulty in developing a clear and consistent understanding of the design attributes related to avoiding and managing flight crew error required by the proposed regulation. Participants were concerned that without clear descriptions and well-defined processes and methods of compliance, evaluating such attributes could be subjective and based on personal preferences. While this may be a similar concern with many regulations, participants noted that some design attributes specified by 14 CFR 25.1302 may be particularly prone to subjectivity. Participants felt that design attributes particularly prone to subjectivity included ensuring that the behavior of the equipment is predictable and ensuring that it is unambiguous. Some participants


stated that because predictability and ambiguity can be difficult to define precisely, evaluation of these attributes may be more prone to subjectivity and personal preferences.

The following subsections describe resources that were discussed in the interviews as those anticipated as being beneficial for helping individuals from aircraft and avionics manufacturers as well as from the civil aviation authorities to successfully address 14 CFR 25.1302 when it is released.

3.1.3.1. Guidance on Methods of Compliance and Evaluation Most of the participants stated that they would benefit from additional advisory and/or guidance material to be able to effectively address compliance with the proposed 14 CFR 25.1302. Much of the information identified in this section would likely be of interest for any new or modified FAA regulation. However, many participants appeared to feel an added sense of need for additional guidance on methods of compliance and evaluation related to the proposed regulation. The added sense of need was likely because a number of manufacturers have relied only on informal processes to address human factors-related design elements, and some participants have the perception that the human factors-related requirements of the proposed regulation are more subjective than in most other regulations. The specific information participants described needing in order to effectively demonstrate or evaluate compliance with the proposed 14 CFR 25.1302 includes the following:

• Specific examples and case studies – Requests included detailed, real or hypothetical, examples related to demonstrating compliance with 14 CFR 25.1302. Participants asked for examples of flight deck designs that are acceptable and designs that would not be compliant with14 CFR 25.1302, examples of methods of compliance, and examples of acceptable compliance documentation.

• Detailed information about specific methods of compliance – Participants asked for detailed method of compliance information that would help them effectively choose and conduct the appropriate means of compliance for the various design elements and requirements of 14 CFR 25.1302. Participants stated that more detail about methods of compliance and evaluation than is currently provided in the evaluation section of the proposed AC 25.1302-X was needed. Specifically, the information that participants asked for related to methods of compliance included the following:

o Benefits and limitations of various means of compliance o How to choose appropriate methods o How much is enough testing and data for any particular design decision o Information about the philosophy behind different methods of compliance used

and the objectives they can be used to achieve o Information about what methods are more likely to give the more credible, value-

added results with less wasted effort o Information about how and when to perform a task analysis


o Information about state-of-the-art metrics, methods to measure, and tools available for things such as workload, error rate, and situational awareness

o Information about evaluation methods that are useful in different phases of the design process

o Information about method, means, procedure or practice that may be used jointly between certification authority and applicant to determine if a test is sensitive enough and sufficient

Much of the information requested by participants in this list of detailed information about specific methods of compliance would not commonly be provided by the FAA or other civil aviation authority, but the list provides an understanding of the type of information that the participants stated they still need, no matter the source that would provide the information. The broad nature of this information may be due to the variety of participants included in the study especially related to the extensive nature and maturity of their current human factors processes.

• Clear definitions of terminology used in the regulation – Some examples mentioned include clarifying definitions of normal, non-normal, predictable, and unambiguous.

• Guidance related to novelty, complexity, and integration – This includes information about the criteria used to judge the novelty, complexity, and level of integration of a piece of equipment.

• Guidance related to non-conformed facilities – Some participants stated that the guidance related to the use of non-conformed facilities would be particularly beneficial for demonstrating compliance with 14 CFR 25.1302. Participants asked for guidance that included information about the benefits of using varying levels of fidelity as the design matures and clarifying, adding emphasis, and expanding information related to the use of non-conformed facilities for possible certification credit.

• Clarification related to when enough has been accomplished to demonstrate compliance – This includes objective criteria to determine what is enough to demonstrate compliance and guidance about how to determine when enough breadth in exploring design for human error may be enough, and information related to the level of analysis required by criticality of associated functions (e.g., will low criticality functions be required to conform to the same rigor as high criticality functions?).

• Guidance related to compliance documentation – This includes information about the type and level of detail to be included in the documentation that will be required to demonstrate compliance, examples of acceptable compliance documentation, and how much written explanation will be appropriate and sufficient for various demonstrations of compliance. Additional guidance on details to be included in the Certification Plan was desired as well as guidance on when a separate Human Factors Certification Plan should be developed and used.

• Guidance about skills needed to demonstrate compliance – This includes the type of skills needed to effectively develop, conduct, and document human factors related evaluations needed to demonstrate compliance with 14 CFR 25.1302. This is another area that would not likely be provided by the FAA or other civil aviation authority, but


reflects the variety and extent of information mentioned as needed by the study participants.

• Information to help non-human factors personnel understand the implications of the regulation – This includes a description, understandable to those not familiar with human factors, to help program teams and various levels of management understand the implications of the regulation and help gain necessary support for addressing human factors issues for future certification projects.

In addition, participants also requested guidance that was not strictly specific to 14 CFR 25.1302, but would be beneficial to demonstrating and evaluating compliance with other regulations as well. This information includes the following:

• Guidance related to the timing of design and certification tasks – This includes guidance about tasks that should be performed during the development stage in order to be compliant in the design and specific information about when applicants should involve the civil aviation authorities.

• Guidance on how to choose a representative pilot population sample – This should include how to handle situations when the pilots needed for testing are not available.

• Guidance on how to develop effective flight test plans – It was noted that additional guidance on how to develop effective flight test plans may also save time for the civil aviation authority flight test pilots who at times have to spend a lot of time providing feedback and input to applicants to help them write flight test plans.

3.1.3.1.1. Specific Questions Asked by Participants The following is a list of questions identified in the interviews related to guidance material or other information needed to help effectively address 14 CFR 25.1302.

Questions related to demonstrating compliance include the following:

• How do you demonstrate that something is predictable and unambiguous? • How much additional testing will be required as a result of 14 CFR 25.1302? • What additional evaluation tools will be required to demonstrate compliance with the

proposed 14 CFR 25.1302? • How do you design a meaningful test method for assessing Human Factors that is

appropriate to identify human factors shortcomings over and above the design review and assessment that has always been part of the design and certification process?

• What will be the impact of 14 CFR 25.1302 on flight test plans? Will they need to be longer and more detailed?

• What are the data requirements for a new or novel product with no service history? • Will paragraph (a) require applicants to analyze equipment interface functions before

demonstration of their functions? (e.g., The ability of one piece of equipment to support the annunciation for a number of different interfacing systems is available in some display units and is controlled through an interface document. With the new regulation,


will the function also need to be analyzed in order to demonstrate compliance with the proposed 14 CFR 25.1302 paragraph (a)?)

Questions related to documentation for demonstrating compliance include:

• What documentation will be required to comply with 14 CFR 25.1302? • Will documenting data from early evaluations using non-conformed facilities result in

certification credit? • Will the amount of documentation of evaluations increase as a result of 14 CFR 25.1302? • Will paragraph (a) require the documentation of all interconnections between flight deck

systems and equipment, such as when a pilot is doing something on one piece of equipment, but the annunciation or the indication is displayed on another piece of equipment?

3.1.3.1.2. Resources Currently Available Resources currently available that may include information related to guidance material or other information requested to help effectively demonstrate and document compliance with proposed 14 CFR 25.1302 include the following:

• Proposed FAA Advisory Circular AC 25.1302-X The proposed Advisory Circular AC 25.1302-X provides guidance to determine whether a feature should require more scrutiny against 14 CFR 25.1302. Section 4-2 of the proposed AC 25.1302-X provides a discussion of integration, complexity, and novelty.

• FAA Advisory Circular AC 25-7C, Flight Test Guide for Certification of Transport Category Airplanes This Advisory Circular provides guidance for the flight text evaluation of transport category airplanes. This AC includes flight test methods and procedures to show compliance with the regulations contained in subpart B of 14 CFR part 25 which address airplane performance and handling characteristics.

• FAA Policy Statement ANM-99-02, Guidance for Reviewing Certification Plans to Address Human Factors for Certification of Transport airplane Flight Decks This Policy provides guidance to FAA Certification Teams that will enable them to conduct an effective review of an applicant’s Human Factors Certification Plan or the human factors components of a general Certification Plan. This guidance describes the sections of a Human Factors Certification Plan and the information that should be included in each section.

• FAA Memorandum ANM-01-03A, Factors to Consider when Reviewing an Applicant’s Proposed Human Factors Methods of Compliance for Flight Deck Certification. This Policy provides guidance to FAA Certification Teams that will enable them to conduct an effective review of an applicant’s Human Factors Certification Plan or the human factors components of a general Certification Plan. The information provided in


the policy statement covers general information on methods of compliance for regulations related to human factors, the identification of design-specific human factors issues, and the identification of regulation-specific human factors issues.

• FAA Order 8110.4C, Type Certification This Order sets procedures for evaluating and approving aircraft, engine, and propeller type design data and changes to approved type design data.

• FAA Notice 8110.98, Addressing Human Factors/Pilot Interface Issues of Complex, Integrated Avionics as Part of the Technical Standard Order (TSO) Process. This Notice provides guidance to facilitate the identification and resolution of human factors/pilot interface issues associated with complex, integrated avionics submitted for new or amended Technical Standard Order Authorization.


An assessment may be necessary to determine if these resources contain adequate information to address all of the information needs related to guidance material or other information needed to help effectively address 14 CFR 25.1302.


3.1.3.2. Guidance for Addressing Error Identification and Error Management Participants requested guidance information for addressing error identification and error management in order to effectively demonstrate, review, and evaluate compliance with proposed 14 CFR 25.1302 paragraph (d). Specific information needed related to errors, error identification, and error management includes the following:

• Clarification of errors reasonably expected in service – This includes explicit definitions of “reasonably expected” and “practicable”.

• Guidance for identifying errors – This includes a definition of the scope of errors to be identified by the proposed 14 CFR 25.1302 regulation, as well as how best to identify an appropriately comprehensive list of errors (or if this is necessary). Participants were particularly interested in guidance for a well-defined scope to address the concern that the unpredictability of human nature makes identifying all possible errors an extremely difficult challenge.

• Clarification of expectations for demonstrating compliance related to errors – This includes the types of data, kinds of analysis, and extent of documentation that may be required for the identification and management of errors.


• Guidance related to error analysis – This includes guidance to better understand the rationale behind the analysis of errors, as well as appropriate approaches to error analysis and when they are best used.

• Guidance related to the timeliness of presenting information and responding when managing errors – This includes the concept that a timely indicator is provided to pilots so they can respond to it in a safe manner to keep the aircraft out of the hazardous situation.

• Clarification about acceptable means of compliance for managing errors – This includes providing information to help create clear expectations as to what will be required to demonstrate compliance with providing a means to manage errors and also specific guidance related to situations when an undo function as a means of compliance will be required.

• Applied definition of human error – This includes various elements such as the amount of time that elapses between the error and a correction (e.g., whether an incorrect switch is activated with an immediate correction is considered an error), and the severity or criticality of the consequences of the error (e.g., does severity and criticality impact whether an action is considered an error).

• Guidance on error and error rates – This includes definitions of acceptable levels of human performance, how the acceptable levels can be determined, and information to define performance references and error references that can be reasonably expected in service.

3.1.3.2.1. Specific Questions Asked by Participants The following is a list of questions identified in the interviews related to addressing errors reasonably expected in service for the proposed 14 CFR 25.1302 paragraph (d):

• How would an applicant show to the FAA how reasonably expected errors are identified, captured, and mitigated?

• How are the terms "reasonably expected" and "practicable" to de defined and used to create boundaries on the scope of errors to be addressed?

• Are applicants required to generate an exhaustive list of all possible errors? • How do you address the large number of operational variables when trying to evaluate

errors reasonably expected in service? Variables cited include: flight crew backgrounds, operational knowledge with particular aircraft, depth of experience, and fatigue.

• How do you determine if human error is caused by the design? • Will an undo function be required for all systems with electronic controls and displays? • When is it appropriate to use training or procedures as a means to address human errors? • What is an acceptable rate of error and how is that determined? • How long ago, and after how many intervening events after the error may have occurred,

must recovery from the error be provided? • Will the severity of the error be considered in determining the acceptable means required

to enable flight crew to manage errors?


3.1.3.2.2. Resources Currently Available Resources currently available that include information related to addressing errors reasonably expected in service for the proposed 14 CFR 25.1302 paragraph (d) include the following:

• Proposed FAA Advisory Circular AC 25.1302-X The proposed Advisory Circular AC 25.1302-X provides information on the rationale and intent behind the proposed regulation in the introductory sections, guidance to determine whether a feature should require more scrutiny against 14 CFR 25.1302, and guidance related to acceptable methods of compliance.

• FAA Advisory Circular AC 25.1523-1, Minimum Flightcrew The Advisory Circular provides guidance related to acceptable methods of compliance with the requirements of 14 CFR 25.1523 which contains certification requirements for the minimum flightcrew on transport category airplanes.


An assessment may be necessary to determine if these resources contain adequate information to address all of the information needs related to addressing errors reasonably expected in service for the proposed 14 CFR 25.1302 paragraph (d).


3.1.3.3. Information-Sharing Resources within FAA Participants also discussed the need to share information within FAA in order to effectively evaluate compliance with 14 CFR 25.1302. Various suggestions of tools and methods to accomplish effective collaboration and standardization information sharing were mentioned:

• Certification personnel forum – This includes the opportunity for certification personnel to have access to experienced colleagues who can share examples and experiences.

• Information about those with experience certifying particular systems – This may include contact information for those with experience certifying similar systems so that they may be used as resources by others in the FAA.

• Human Factors Specialist pushing out information to certification crew members – An example suggested is a 14 CFR 25.1302 email newsletter created by the Human Factors Specialists, in coordination with the FAA Engineers and Test Pilots who work these issues, that could be populated with information relevant to 14 CFR 25.1302. This could include information such as the day-to-day experiences of the Human Factors Specialist with the regulation.


• Information about what others in the agency are doing – This includes projects that are being worked on and their details, what processes, rationale, and determinations that they use to evaluate compliance.

• Technology for collaboration between FAA offices – This includes better use of existing technology such as email and telecons to allow for better collaboration to look for such things as inconsistencies or red flags that may be identified by sharing early design concepts and diagrams with other FAA personnel.

3.1.3.3.1. Specific Questions Asked by Participants The following is a list of questions identified in the interviews related to sharing information within the FAA to more effectively perform compliance reviews and evaluation with 14 CFR 25.1302.

• What processes and technology may be used to better share certification project information and facilitate collaboration within the FAA?

3.1.3.3.2. Resources Currently Available There is a number of information sharing methods that currently exist within the FAA that may provide a platform for sharing information and coordinating on issues related to the proposed 14 CFR 25.1302 regulations. Some of these include workshops that provide a forum for raising awareness and coordinating on issues (e.g., FAA Avionics Workshop) and newsletters produced by each of the FAA Directorates.

3.1.4. What Processes Will the FAA Use to Review and Evaluate Compliance with 14 CFR 25.1302? This section is broken into two subsections. The first addresses the question “How will the review and evaluation process be performed by the FAA?” and describes the information that study participants requested related to this question. The second addresses the question “How will the FAA facilitate consistency in the review and evaluation process for the proposed 14 CFR 25.1302 regulation?” and provides participant input about that information.

3.1.4.1. How Will the Review and Evaluation Process Be Performed by the FAA? Individuals from the aircraft and avionics manufacturers and from the civil aviation authorities wanted to better understand how the review and evaluation process related to the new proposed 14 CFR 25.1302 will be conducted. Participants also expressed concern about the evaluation process being implemented consistently across the industry. Participants felt that a better understanding of the evaluation process would allow them to more clearly understand, predict, and apply proposed 14 CFR 25.1302.


Participants specifically asked for the following information:

• Detailed description of review and evaluation process – Participants asked for a well-defined, detailed and clearly described process for how the FAA will conduct the review and evaluation to determine compliance with proposed 14 CFR 25.1302. This description includes the following:

o A roadmap of review and evaluation timelines and content. including: Specific information about when and how often applicants should

communicate with and involve the civil aviation authorities How early in the design process the FAA expects proposed 14 CFR

25.1302 to be addressed When is most effective for the Human Factors Certification Plan to be

developed and submitted o How the definition of the intended function of an application is used in evaluating

compliance (e.g., move forward with compliance evaluations after coming to agreement on the intended function).

• Tools to facilitate consistent FAA review and evaluation processes – Participants asked for tools to help facilitate consistently performing the FAA’s review and evaluation processes. One example of a tool requested was a human factors checklist or a job aid to help standardize the processes. Participants also asked about whether existing human factors training would be adapted and/or existing training would be develop to teach the necessary skills to perform a consistent evaluation process.

In addition clarification about specific methods that may be used in the evaluation process for proposed 14 CFR 25.1302 was requested by the participants, including:

o Multi Pilot System Usability Evaluations (MPSUE) Participants asked for a description of the best practices for a successful MPSUE process. The description should include information that describes the following: Criteria and guidelines for how to perform a useful and successful

MPSUE. The importance of defining objectives and priorities at the beginning of

the process that are adhered to and fulfilled. The use of test cards to help the focus stay on the objectives. Definition of process for resolving differences in opinion and coming to

consensus.

Some participants expressed concern that more MPSUEs would be required to find compliance with proposed 14 CFR 25.1302. The concern centered on the perception that they have had mixed results (i.e., perceived that some were too subjective and did not always result in certification credit) and that MPSUEs tend to be expensive and time-consuming.


o Pilots used in evaluation Participants were interested in clarification about ways to facilitate appropriate assessment of design through the use of pilots with varied backgrounds.

• Description of the FAA’s approach to evaluating human factors – Participants asked for information that specifically outlines how the FAA will evaluate human factors in determining compliance with the proposed 14 CFR 25.1302. This includes information about what the goals are, how testing will be structured, and specific assessments that will be used.

In addition, participants also requested guidance that was not strictly specific to 14 CFR 25.1302, but would be beneficial to demonstrating and evaluation compliance with other regulations as well. This included the following:

• Description of how compliance decisions will be documented – Participants asked for a description and examples of how compliance decisions will be documented.

• Description of processes likely to be used for coming to agreement and resolving differences – Participants asked for a description of the processes that will likely be used to facilitate coming to agreement between the applicant and the FAA. This includes the process for coming to agreement about the content of the certification plan (e.g., the plans for evaluations and testing), as well as resolving differences in understanding about compliance requirements and whether or not a design adequately meets these requirements.

3.1.4.1.1. Specific Questions Asked by Participants The following is a list of questions identified in the interviews related to how the review and evaluation process related to the new proposed 14 CFR 25.1302 will be conducted.

• How will FAA personnel document that compliance has been shown? • How will compliance findings be determined? • How do you know when you have done enough to adequately demonstrate or evaluate

compliance?

3.1.4.1.2. Resources Currently Available Resources currently available that include information related to how the review and evaluation process related to the new proposed 14 CFR 25.1302 will be conducted include the following:




• Draft FAA Order 8110.4D, Type Certification Multi Pilot System Usability Evaluations (MPSUE) is a new term for a practice that is used by several offices, although the term and the description is not currently documented in any FAA regulatory or guidance material. However, it is planned for MPSUE to be referenced in revised Order 8110.4D Type Certification (C is current version). Version D of the Order is currently in the draft and review process.

An assessment may be necessary to determine if these resources contain adequate information to address all of the information needs related to how the review and evaluation process for the new proposed 14 CFR 25.1302 will be conducted.


3.1.4.2. How Will the FAA Facilitate Consistency in the Review and Evaluation Process for the Proposed 14 CFR 25.1302 Regulation? Individuals from the aircraft and avionics manufacturers and from the civil aviation authorities stated that it would be useful to better understand how the FAA will facilitate ensuring that the proposed 14 CFR 25.1302 regulation is consistently applied as they interact with different applicants. While the consistent application of a regulation may be a concern for any regulation, participants felt that because of the subjective nature of some of the elements of the proposed 14 CFR 25.1302, lack of consistency may be greater issue for this regulation. Therefore, participants felt that certain information and processes to help facilitate consistency in both the understanding of the proposed 14 CFR 25.1302 regulation, as well as in making compliance decisions related to the proposed 14 CFR 25.1302, would be particularly beneficial.

Participants asked about processes and information that would be used to apply the regulation effectively, efficiently, and consistently throughout project, across applicants, and across different ACOs. Participants were interested in how the FAA would facilitate both a consistent and accurate understanding of the regulation and consistent compliance decisions across different certification projects.

Participants expressed concerns about an individual or organization developing an understanding of the proposed 14 CFR 25.1302 that is different from others, and they expressed a desire to reduce this vulnerability. Individuals from the aircraft and avionics manufacturers and from the civil aviation authorities talked about the fact that an individual’s background, skill set, and experiences with the development and typical application of the regulation can impact the way that the individual interprets a regulation. While this is the same for many regulations, participants noted that some design attributes specified by the proposed 14 CFR 25.1302 may be


particularly prone to subjectivity and, therefore, prone to differences in interpretation and application.

3.1.4.2.1. Facilitating Consistent Understanding of Regulation Participants wanted to better understand the processes that the FAA will ensure that individuals and organizations develop a consistent and accurate understanding of the regulation. Specifically, participants asked for the following:

• Well-defined and clear description of the meaning of proposed14 CFR 25.1302 • Specific real or hypothetical examples of how particular aspects or paragraphs of the

regulation are appropriately applied to designs • Training developed so that format, delivery, and content is designed so that it effectively

prepares the participants to form a consistent and accurate understanding of 14 CFR 25.1302. Many participants requested training for representatives from the civil aviation authorities and for aircraft and avionics manufacturers.

• Well-defined and clearly described processes related to coming to a shared understanding of the proposed 14 CFR 25.1302

o Process for resolving differences of understanding and opinion o Process for new representative from the civil aviation authority to join an existing

certification team

3.1.4.2.2. Facilitating Consistent Compliance Decisions Participants wanted to better understand the processes that the FAA will use to ensure that compliance decisions are made consistently throughout project, across applicants, and across different ACOs. As mentioned above, this is relevant for many regulations, but participants noted that some design attributes specified by the proposed 14 CFR 25.1302 may be particularly prone to subjectivity and, therefore, may be more prone to differences in compliance decisions. To address this, specific information asked for by the participants included the following:

• Well-defined and clearly described processes to make compliance decisions consistently • Real or hypothetical examples of acceptable and un-acceptable designs related to the

proposed 14 CFR 25.1302 • Training developed so that format, delivery, and content are designed to effectively

prepare individuals from the civil aviation authorities to find compliance with the proposed 14 CFR 25.1302. Specific content asked for by the participants included the following:

o Information about the interpretation of the regulation o FAA’s approach to human factors evaluations o Human factors information and resources useful in reviewing and evaluating

compliance with the proposed 14 CFR 25.1302 regulation • Well-defined and clearly described processes to use to consistently determine when

enough has been done to demonstrate compliance with proposed 14 CFR 25.1302. This should include a description of the process to determine level of scrutiny and analysis


effort for proposed 14 CFR 25.1302 regulation required and whether this may be dependent on the type and extent of design change made.

3.1.4.2.3. Specific Questions Asked by Participants The following is a list of questions identified in the interviews related to how the FAA will facilitate ensuring that the proposed14 CFR 25.1302 regulation is applied consistently.

• How will situations be resolved in cases where the applicant has formally documented a program with a well-balanced objective and subjective portfolio of data, but the representative from the civil aviation authority disagrees based on their expert judgment call?

• What is the intended, appropriate interpretation of the proposed regulation? • When there is disagreement about compliance, how is it resolved? • How will differences in interpretation be resolved?

3.1.4.2.4. Resources Currently Available Resources currently available that include information related to how the FAA will facilitate ensuring that the proposed 14 CFR 25.1302 regulation is applied consistently include the following:



An assessment may be necessary to determine if these resources contain adequate information to address all of the information needs related to how the FAA will facilitate ensuring that the proposed 14 CFR 25.1302 regulation is applied consistently. See Reference listing at the end of this report for links to access these documents.

3.1.5. How Does 14 CFR 25.1302 Relate to Other Regulations and Processes? Individuals from the aircraft and avionics manufacturers and from the civil aviation authorities wanted to better understand the relationship of the proposed 14 CFR 25.1302 to other regulations and guidance materials. While applicants need to show compliance to each regulation regardless of the overlap with other regulations, participants felt that having a clear understanding of the


similarity, differences, and overlap of proposed14 CFR 25.1302 with other regulations would allow them to more effectively develop a clear understanding and appropriately apply the proposed 14 CFR 25.1302 regulation.

Participants expressed particular concern over how to put the proposed regulation into context with current regulations, and how to sort out current processes used for demonstrating compliance with existing regulations from the new requirements contained in proposed 14 CFR 25.1302. Several manufactures stated that they are uncertain what is new in the proposed 14 CFR 25.1302 regulation.

Some participants thought that it would be useful to provide information that describes the intentional choice of location of the proposed 14 CFR 25.1302 as a general applicability regulation in the equipment section of 14 CFR Part 25 and describe in what ways it builds on the requirements specified in 14 CFR 25.1301. Along with that, participants thought that in addition to the related regulations section of the proposed AC 25.1302-X, it would be useful to have information that clarifies the relationship of the proposed 14 CFR 25.1302 to the following regulations and guidance material:

• Specific regulations: o 14 CFR 25.1301 o 14 CFR 25.1309 o 14 CFR 25.1523

• Regulatory Parts: o 14 CFR 21 o 14 CFR 23 o 14 CFR 121 (and other operations regulation)

• Other Guidance Materials: o AC 25-11A o FAA Policies

3.1.5.3. Specific Questions Asked by Participants The following is a list of questions identified in the interviews related to the relationship of the new proposed 14 CFR 25.1302 to other regulations and guidance materials.

• What is in 14 CFR 25.1302 that is not covered by 14 CFR 25.1301 and 14 CFR 25.1309? • What will happen on each one of the individual regulations that have pilot- or flightcrew-

associated implications? • How will the training and operations regulations interact with 14 CFR 25.1302? • How does paragraph (a) relate to policy, procedures, SOPs, and checklists? There might

be a struggle between aircraft certification and aircraft evaluation. • What is the extent to which 1302 applies in the presence of failures? Which failures? Of

which probability? Of which level of severity and consequences?


3.1.5.4. Resources Currently Available Resources currently available that include information related to the relationship of the new proposed 14 CFR 25.1302 to other regulations and guidance materials include the following:

• Proposed AC 25.1302-X The proposed AC 25.1302-X provides guidance for showing compliance with 14 CFR 25.1302 and several other rules in 14 CFR Part 25 that relate to the installed equipment used by the flight crew in the operation of the aircraft. This proposed Advisory Circular describes acceptable approaches to compliance and also provides recommendations for the design and evaluation of controls, displays, system behavior, and system integration, as well as design guidance for error management.

• Human Factors Harmonization Working Group (HF HWG) Final Report The HF HWG Final Report submitted to the Transport Airplane and Engine Issues Group (TAEIG) of the Aviation Rulemaking Advisory Committee contains a description of the work that the group did to develop an initial draft of the proposed 14 CFR 25.1302 regulation and the associated proposed Advisory Circular.

An assessment may be necessary to determine if these resources contain adequate information to address all of the information needs related to the relationship of the new proposed 14 CFR 25.1302 to other regulations and guidance materials.


3.1.6. Who Will Be Responsible and Have Authority to Make Compliance Findings with 14 CFR 25.1302?

3.1.6.1. Roles and Responsibilities Within the Civil Aviation Authority Individuals from the aircraft and avionics manufacturers and from the civil aviation authorities wanted to better understand the roles and responsibilities related to the proposed 14 CFR 25.1302 regulation. Participants ask for a well-defined and clear description of the roles and responsibilities related to the new regulation within the civil aviation authority. This may be a concern for any new regulation since each office has the authority and responsibility to determine which individual or teams of individuals will be responsible for finding compliance. This process is thought to be inconsistent across offices in terms of the backgrounds or experience required. But while this may be a concern for any new regulation, participants were particularly interested because they felt that the design attributes specified by the proposed 14 CFR 25.1302 may require expertise in multiple disciplines such as piloting and human factors.

Participants asked for a description of the roles and responsibilities that include who and what disciplines will be authorized to interpret and approve compliance for the proposed 14 CFR 25.1302 regulation.


Manufacturers are also looking for the application of the defined roles and responsibilities within the regulatory authority to be consistently applied across projects and throughout the agencies.

3.1.6.1.1. Specific Questions Asked by Participants The following is a list of questions identified in the interviews related to the roles and responsibilities for the proposed 14 CFR 25.1302 regulation:

• Who will be authorized to interpret and approve compliance with 14 CFR 25.1302 within the regulatory authority?

3.1.6.1.2. Resources Currently Available As with any new regulation, the staffing, roles, and responsibilities related to finding of compliance will be determined by the individual offices.

3.1.6.2. Human Factors Designees Individuals from the aircraft and avionics manufacturers and from the civil aviation authorities expressed the need to better understand the potential role of human factors designees in the process of finding compliance with 14 CFR 25.1302. Concern was expressed regarding language in current FAA documents that does not allow for delegation of human factors.

Order 8110.37E Designated Engineering Representative (DER) Guidance Handbook, states:

4-16. Human Factors. There is no human factors authority that can be granted to a DER. DERs who approve data that has an impact on human factors should document and discuss any issues with the ACO.

If the decision is made to authorize human factors designees, participants asked for a description of the roles and responsibilities of the human factors designee that includes the following information:

• Process and timeline for approval of human factors designees • Qualifications that will be required for human factors designees • What aspects, if any, of 14 CFR 25.1302 will human factors designees be authorized to

review for compliance • What training will be developed to prepare human factors designees with the

understanding of the regulation, the knowledge, and the skills necessary to find compliance in manner consistent with others authorized to find compliance with proposed 14 CFR 25.1302, and, specifically, what content will the training include

• Process that will be used to avoid having two potentially competing persons finding compliance for the regulation

3.1.6.2.1. Specific Questions Asked by Participants The following is a list of questions identified in the interviews related to the potential role of human factors designees:


• If a human factors designee is approved, what elements of finding compliance could be delegated to them?

• What qualifications will be required for human factors designees? • How does proposed 14 CFR 25.1302 fit into the current policy that human factors cannot

be delegated?

3.1.6.2.2. Resources Currently Available No resources were identified that include information related to the potential role of human factors designees.

3.1.7. Where Can Useful Resources Be Found? Individuals from the aircraft and avionics manufacturers and from the civil aviation authorities wanted to more effectively and efficiently know how to locate the resources useful for demonstrating, evaluating, and finding compliance with the proposed 14 CRF 25.1302. Many participants talked about the challenge of finding the appropriate resources when those resources are located in many locations. To address this challenge, participants wanted a listing or index of guidance materials and resources related to proposed 14 CFR 25.1302. This includes such information as a listing of guidance materials applicable to specific things like a display (e.g., one participant mentioned wanting a Cliff Notes version of the HFYI Design CoPilotTM), and a collector document that provides a checklist of documents to reference during the evaluation process. Individuals from the aircraft and avionics manufacturers and from the civil aviation authorities also wanted to know what reference information should be used for complying with proposed 14 CFR 25.1302 as well as 25.777, such as which anthropometric database to use, what ergonomic data to use in design, and what resources are available for identifying possible errors.

3.1.7.1. Specific Questions Asked by Participants The following is a list of questions identified in the interviews related to the location of the resources useful for demonstrating, evaluating, and finding compliance with the proposed 14 CFR 25.1302 regulation:

• What guidance materials and resources are useful related to demonstrating or evaluating compliance with the proposed 14 CFR 25.1302?

• Where are useful guidance materials located?

3.1.7.2. Resources Currently Available Resources currently available that include information related to the location of the resources useful for demonstrating, evaluating, and finding compliance with the proposed 14 CFR 25.1302 regulation include the following:


• References section in this Report The Reference section at the end of this report presents a listing of resources and access information useful for demonstrating, evaluating, and finding compliance with the proposed 14 CRF 25.1302.

• HFYI Design CoPilotTM application The HFYI Design CoPilotTM application at http://www.designcopilot.com is an online tool for gathering the information needed to make human factors decisions related to flight deck design and evaluation.

• Various research literature and bibliographies

An assessment may be necessary to determine if these resources contain adequate information to address all of the information needs related to the location of the resources useful for demonstrating, evaluating, and finding compliance with the proposed 14 CFR 25.1302 regulation.

3.2. Current and Planned Practices The following section addresses current practices described by interview participants for addressing human factors aspects of existing regulations, including EASA CS 25.1302, as well as planned practices for compliance with proposed 14 CFR 25.1302.

3.2.1. Resources and Guidance Material Currently Used The following information-related, people-related, and tool-related resources currently used when addressing human factors aspects of existing regulations were discussed in the interviews.

3.2.1.1. Information-Related Resources Currently Used The participants mentioned that they used the following information-related resources when addressing human factors aspects of existing regulations, including EASA CS 25.1302:

• Accident and Incident Reports

• ACs and AMCs – In addition to existing ACs and the AMC for CS 25.1302, some participants stated that they also use the proposed AC 25.1302-X as a reference for the definition of phrases.

• Guidance from other civil aviation regulatory authorities – This includes guidance from EASA, JAA documents, and EUROCONTROL.

• HFYI Design CoPilotTM application – This online application for gathering information needed to make human factors decisions related to flight deck design and evaluation is available at http://www.designcopilot.com.

• Industry Standards – This includes SAE, RTCA, ISO, and military standards. Specific ISO standards mentioned were ISO 92-41 and ISO 13-407. SAE ARP 5056 was also mentioned as a key document that might be enhanced to describe in more detail

http://www.designcopilot.com/�



recommended flight deck design practices and provide useful guidance to inform decisions related to compliance with the FAA proposed regulation 14 CFR 25.1302.

• Internal documentation – This includes internal guidance material, internal case documentation on decisions made in previous projects, and documentation from other related projects.

• Issue Papers

• Civil aviation authorities websites • Reference material – This includes that found in textbooks. • Research literature – This includes research articles published in conference proceedings

and journals. • FAA and EASA Regulations – This includes the regulation language, preamble in the

Federal Register as a Notice to Proposed Rulemaking (NPRM) for a regulation, and any legal interpretation of a regulation that may be available.

• FAA Policies – This includes FAA Memorandum ANM-01-03A which provides general information on methods of compliance for regulations related to human factors, the identification of design-specific human factors issues, and the identification of regulation-specific human factors issues.

3.2.1.2. People-Related Resources Currently Used The participants mentioned that they used the following people-related resources when addressing human factors aspects of existing regulations, including EASA CS 25.1302:

• Customers – This includes feedback that customers have provided about their products. • Pilots – This includes pilots who supply design input, flight operations pilots for

conducting testing using a diverse and representative pool of subjects, and flight test pilots who perform the majority of the evaluation tasks related to human factors aspects of design.

• Flight Test Engineers – The Flight Test Engineers along with the Flight Test Pilots perform the majority of the evaluation tasks related to human factors aspects of design.

• Human Factors Specialists – This includes those at the Aircraft Certification Offices (ACOs), Transport Aircraft Directorate (TAD), FAA headquarters, and internally within manufacturer.

• Individuals with specialized task expertise – This includes linguists and test engineers who specialize in lab and bench testing.

• Other colleagues – This includes colleagues who have worked on similar projects, colleagues who have particular relevant experiences or expertise, and colleagues not involved in current project and, therefore, have a fresh perspective.

• Civil aviation authority representatives – This includes asking civil aviation authority representatives to answer specific questions and for help with interpretation, along with including civil aviation authority representatives as part of the test flight crew.

• Self – This includes one’s own knowledge and previous experiences.


• Workshops and Working Groups – This includes avionics workshops, human factors workshops, and pilot working groups.

• Teams – This includes multi-disciplinary teams, multi-pilot teams (e.g., MPSUE) and human factors teams.

3.2.1.3. Tool-Related Resources Currently Used The participants mentioned that they used the following tool-related resources when addressing human factors aspects of existing regulations, including EASA CS 25.1302:

• Aircraft – This includes the aircraft as used for conduct the flight test. • Error Analysis Worksheets

• HFYI Design CoPilotTM application – This online application for gathering information needed to make human factors decisions related to flight deck design and evaluation is available at http://www.designcopilot.com.

• Mockups – This includes mockups used for interface testing and for demonstrating some aspects of compliance.

• Simulators – This includes fixed-based simulators and simulators with various levels of motion.

• Non-conformed articles – This includes various non-conformed articles that are used for early design iteration testing and decision making. Participants stated that the use of non-conformed articles allows for a wider range of pilots to participate in testing as well as a greater number and variety of conditions to be tested.

• Modeling – This includes the use of modeling as predictive tools during design and development to set design requirements. Participants stated that they use human modeling to control and customize anthropometric dimensions. Two specific tools mentioned were the following:

o Human Builder Modeling Workbench – Used for anthropometrics. o The University of Michigan Strength Model – Used for predicting strength

requirements for tasks such as lifts, presses, pushes, and pulls for the equipment and then validated later with pilots.

• Bedford Scale – This scale is used by some participants in a simulator or with a prototype to perform workload assessment.

• Reason Error Model – The error model developed by John Reason is used by some participants for the identification of possible errors.

3.2.2. Certification Documentation Some participants described current practices related to developing certification documentation for human factors related regulations that may be applicable to showing compliance for the proposed 14 CFR 25.1302.



Current practices described for addressing human factors issues in certification documentation included the following:

• Providing documentation of design decisions throughout development, including o Justification of design decisions o Descriptions of why certain designs were not effective o Descriptions of considerations and alternatives that were evaluated

• Working with the civil aviation authority to agree on a list of novelties. • Listing and describing the human factors issues to be addressed. • Developing flight test plans to explicitly address human factors considerations, such as

whether the equipment operation is consistent with the pilot’s expectations. • Using the HFYI Design CoPilotTM application to find and understand related human

factors considerations.

3.2.3. Defining Intended Function and Information Requirements Based on prior experience with EASA CS 25.1302 and/or experience with human factors related aspects of other regulations, interview participants described their current practices for defining the intended function and information requirements for a system or piece of equipment. These practices are likely to be applicable to demonstrating compliance with paragraph (a) of the proposed 14 CFR 25.1302 regulation.

In the interviews, the proposed paragraph (a) was used as a starting point for discussion about the general topic of defining the intended function and the information requirements for the flight crew. Participants were asked about what processes they would anticipate using to define the information needs of the flight crew, and any anticipated need to develop any new processes to define the information requirements associated with a design to comply with the proposed regulation. As provided in the NPRM published in the Federal Register, the proposed 14 CFR 25.1302 paragraph (a) states:

(a) Flight deck controls must be installed to allow accomplishment of all the tasks required to safely perform the equipment's intended function including providing information to the flightcrew that is necessary to accomplish the defined tasks.

Many participants stated that the concerns of the proposed paragraph (a) are already being evaluated as part of their current practices. The following specific current practices were described by interview participants regarding defining intended function and information requirements:

• Definition of intended function, tasks, and intended user characteristics at the start of development to facilitate successful design process across a large team

• Documentation of intended function as part of the design review and test plan • Use of system safety assessments and functional hazard assessments to define pilot tasks

related to the intended function


• Use of function allocation, followed by a task analysis to determine information requirements (Use of task analysis includes realistic scenarios to define the information needs of the pilot as related to the intended function)

• Use of informal processes including relying on engineering judgment and pilot input to address information requirements

• Use of evaluation testing to ensure the pilot is finding the necessary information and taking appropriate actions to accomplish the tasks based on that information

3.2.4. Evaluating Urgency, Frequency, and Duration Based on prior experience with EASA CS 25.1302 and/or experience with human factors related aspects of other regulations, interview participants described their current practices for evaluating the design of controls and information as suitable for urgency, frequency, or duration of a task for which they are used. These practices are likely to be applicable to demonstrating compliance with paragraph (b)(2) of the proposed 14 CFR 25.1302 regulation.

As provided in the NPRM published in the Federal Register, the proposed 14 CFR 25.1302 paragraph (b)(2) states:

(b)Flight deck controls and information intended for the flightcrew's use must ... (2) Be accessible and usable by the flightcrew in a manner consistent with the urgency, frequency, and duration of their tasks…

All participants who responded to this question stated that urgency, frequency, and duration are already part of their design or evaluation processes, and they did not express any particular anticipated challenges with the proposed paragraph (b)(2).

The following specific current practices were described by interview participants regarding the evaluation of task urgency, frequency, and duration.

• Use of systematic review and prioritization of entire function or feature list to prioritize each task or function

• Use of Failure Modes and Effects Analysis to define the criticality and priority of a function

• Use of subjective judgment-based assessments and informal observations • Use of simulation and flight testing to demonstrate timeliness of access to information,

including use of scenarios that would require pilots to find information quickly • Evaluation of soft controls on electronic screens by measuring number of clicks to access

the control and use of quick access keys • Evaluation of criticality and frequency of information and control access to determine what

components should be included in the primary field of view


3.2.5. Identifying and Mitigating Errors Reasonably Expected in Service Based on prior experience with EASA CS 25.1302 and/or experience with other human factors related aspects of regulations, interview participants offered the following current and planned practices to address the requirement to identify and manage errors reasonably expected in service as described in paragraph (d) of the proposed 14 CFR 25.1302 regulation.

As provided in the NPRM published in the Federal Register, the proposed 14 CFR 25.1302 paragraph (d) states:

... (d) To the extent practicable, installed equipment must incorporate means to enable the flightcrew to manage errors resulting from the kinds of flightcrew interactions with the equipment that can be reasonably expected in service.

It was suggested that an array of evaluation methods and documentation be used to show compliance with the proposed paragraph (d). Concerns were raised that a single method for error identification should not be used as all single taxonomies for identifying errors will have limitations. Some of the methods currently used and/or suggested for future use include the following:

• Human error analysis • Error taxonomies – One specific example given was the Reason Model • Review of incidents and accidents of similar products to determine error tolerance

levels and how to address previous issues in the new design • Review of associated procedures • Surveys and system evaluations • Review of error types that exist in similar products • Identification of potential weaknesses and development of scenarios to test them • Part task analysis early in the design process • Workload testing • Behavior observation methods • Evaluations designed to cultivate pilot errors with post-evaluation questionnaire for

pilots

3.2.6. Experience with EASA Local and Global Evaluations

Two participants described their experience with two types of EASA evaluations: local evaluations and global evaluations. These experiences differed somewhat so both descriptions are provided here for comparison.

One manufacturer stated that a local evaluation is dedicated to avionics and a global evaluation addresses the whole system and system integration. New and novel items receive both global and local evaluations and items with some novel aspects may receive only global evaluations.


Evaluations can change from local to global (and vice versa) depending on what is revealed in the evaluation process.

The second manufacturer described local evaluations as addressing novelties and being organized by main functions. Global evaluations were described as addressing the full flight deck and all novel integrations. Global evaluations include test pilots from the manufacturer and EASA, as well as flight instructors and line pilots from the airlines.

It appears that the distinction between these two types of evaluation provided useful approaches to the interview participants, but the definitions and implementations were not consistent.

3.2.7. Simulator Use to Demonstrate Compliance with EASA CS 25.1302 One manufacturer described their current practice of simulator use to demonstrate compliance with EASA CS 25.1302. Increasing levels of simulation are used as the program advances, starting very early in the process (3.5 to 4 years in advance of a 5 year program with final simulator used around 2.5 years before certification). Simulation evaluations are completed using an observation grid that specifies what is expected from the pilots during the session. A long and thorough debrief is used to establish the basis for HF analysis.

One regulator stated that observation of scenarios in the simulator is the primary method for testing and evaluation that is used by one of the larger manufacturers, and that they anticipate this is what others will do as they implement CS 25.1302. Analysis of simulator observations was described as being based on issues and not specifically on human errors.

Another manufacturer described that in their experience related to EASA CS 25.1302, the level of simulation used has been made on a case-by-case basis and that EASA does not always require flight tests to show human factors compliance.

4. Conclusion This work was conducted to gain an understanding of the current state of industry preparedness for the release of the proposed 14 CFR 25.1302, and to identify the perceived needs for information that the FAA may consider developing to increase the effectiveness of the implementation of the proposed 14 CFR 25.1302. To accomplish this, a range of people who are involved in the certification process were interviewed.

4.1. Lessons Learned This section describes the lessons learned during this project about the current state of understanding and preparedness for the release of the proposed 14 CFR 25.1302 regulation.


4.1.1. Experience with EASA CS 25.1302 There was less experience across industry with the EASA CS 25.1302 than initially expected. Despite the fact that EASA CS 25.1302 was released several years ago, there are still not many organizations experienced with complying with that EASA regulation. Especially for larger projects, several years may elapse between the establishment of the Certification Basis and the completion of the certification process. Therefore, relatively few certification projects in which EASA CS 25.1302 has been included in the Certification Basis have been completed. Only a total of 9 individuals from the aircraft and avionics manufacturers and from the civil aviation authorities that were interviewed as part of the project had experience either with EASA CS 25.1302 or with a CRI that contained some similar requirements. This lack of experience translated into fewer current practices, recommended methods of compliance, and lessons learned than expected at the outset of this work.

4.1.2. New Rule As described earlier in this report, a large range of questions were asked by participants to help them more accurately and adequately understand proposed 14 CFR 25.1302 and the processes necessary to demonstrate, evaluate, and review compliance with it. However, many of the questions are those that would be expected with any new regulation. Some of the uncertainty is increased because proposed 14 CFR 25.1302 regulation and its related AC has not been officially released, and, therefore, the final wording that will be included is uncertain.

Because 14 CFR 25.1302 is a proposed regulation, many participants also had a lack of awareness or familiarity with the already available resources that could help support understanding of the proposed regulation. Many questions asked about the proposed 14 CFR 25.1302 could be addressed at least in part with existing documents and resources. Part of the issue was due to not being able to find a copy of the proposed AC 25.1302-X because it no longer was out for comment and not readily accessible when the participants looked for it in preparation for our interview. Another part of the lack of awareness and familiarity appears to be because useful and relevant information is located in a number of different documents. This report should aid in allowing interested individuals to identify and access currently available resources that may support understanding of the proposed regulation.

However, as with any new regulation, it is likely that some of the questions and feelings of uncertainty will not be resolved until the proposed 14 CFR 25.1302 is in place and people begin to practice real world compliance.

4.1.3. Anticipated Impact Since many aircraft and avionics manufacturers had not yet had a certification project in which EASA CS 25.1302 was included in their Certification Basis, many participants did not draw a distinction between the impact of the proposed FAA 14 CFR 25.1302 and the impact of EASA CS 25.1302, which is already in place. Overall, there was a general impression among the


interview participants that the existing EASA CS 25.1302 and the proposed FAA 14 CFR 25.1302 would result in an increase in resources, time and effort to demonstrate, document, review, and evaluate compliance. However, those aircraft and avionics manufacturers with experience complying with EASA CS 25.1302 believed there would be no additional resources, time, and effort required for addressing compliance with the proposed 14 CFR 25.1302 regulation beyond that required for compliance with EASA CS 25.1302.

A positive outcome anticipated by participants was that the new proposed regulation would result in more structured, systematic, and formal practices related to the design and evaluation process. This was supported by the description of processes that are now in place as a result of CS 25.1302 described by participants who had experience demonstrating and evaluating compliance with that regulation. They stated that the design and certification processes have become more methodical, detailed, and standardized and that the flow of design and testing processes has been made clearer by mapping out the steps required to ensure that everything required by CS 25.1302 has been addressed.

4.1.4. Overall State of Preparedness There appeared to be a wide range of preparedness, understanding and uncertainty among those interviewed. Overall, those aircraft and avionics manufacturers who had confidence in their existing design and testing processes for addressing human factors, were less concerned with their ability to adequately demonstrate compliance with the proposed 14 CFR 25.1302 regulation. In contrast, those aircraft and avionics manufacturers who had not already incorporated formal human factors processes into their overall design and testing processes were more concerned and expressed more uncertainty about how to adequately prepare for demonstrating compliance with EASA CS 25.1302. They also showed concern about what they believed the level of effort, time, and knowledge required for demonstrating compliance with EASA CS 25.1302 and the proposed 14 CFR 25.1302 regulations would be.

Many of the participants wanted to gain a better understanding of various aspects of the regulation as described in earlier sections of this report to help prepare for demonstrating and evaluating compliance with the proposed 14 CFR 25.1302 regulation. The participants general preparedness related to some of the specific aspects of the proposed regulation are the following:

• 14 CFR 25.1302 paragraph (a) – The area of paragraph (a) that participants discussed specifically was related to defining the intended function and information requirements of the design features of the systems and equipment. Some interview participants suggested that some applicants do not adequately understand what is required to define and communicate a product’s intended function. Overall, our findings in this area showed that many of the large manufacturers felt prepared in this area, but some smaller manufacturers could benefit from additional education and process development to appropriately define intended function and information requirements.


• 14 CFR 25.1302 paragraph (b)(2) – The area of paragraph (b)(2) that participants discussed specifically was related to ensuring that the design features of systems and equipment were accessible and usable by the flightcrew in a manner consistent with the urgency, frequency, and duration of their tasks. All of the responding participants said that in one way or another, they are currently engaged in evaluating task urgency, frequency and duration and seemed to feel prepared for addressing these elements as described in paragraph (b)(2) of the proposed 14 CFR 25.1302. Many participants described various methods that are already used to evaluate these characteristics, which were presented in section 4.2.4 of this report.

• 14 CFR 25.1302 paragraph (d) – The area of paragraph (d) that participants discussed specifically was related to identifying errors reasonably expected in service and providing a means to manage those errors. There were a wide range of questions and concerns presented by interview participants regarding how to interpret the requirements related to identifying and mitigating errors reasonably expected in service. Among the participants, there is uncertainty and concern about the expectation of the scope of the errors to be identified, the methods by which they should be identified, and the acceptable means of compliance for those managing errors. Some of the participants stated that it is important to use carefully selected scenarios to identify the errors that result from performance with the systems or equipment in handling those scenarios. These participants stated that they typically would find errors as they emerge using such processes instead of trying to define all the errors beforehand.

4.1.5. Commonality of Information Needs While there appeared to be a range of preparedness, understanding and uncertainty among those we interviewed, there was also commonality among the additional information requested that would likely provide the largest benefit. Some of the additional information that would likely be of benefit to all participants includes the following:

• Additional detail on how to approach Methods of Compliance (MOCs) – All participants would benefit from additional guidance about acceptable methods of compliance for the proposed 14 CFR 25.1302 regulation. In particular, additional guidance related to the use of non-conformed articles and whether tests and evaluation performed in non-conformed articles would be considered by the civil aviation authority for certification credit would be beneficial. As with all guidance, there is a balance between being overly prescriptive and being too subjective.

• Guidance to determine when they have done enough – All participants would benefit from guidance related to criteria and processes to help both the applicant and the civil aviation authority determine when the testing and demonstration conducted is enough to adequately address the requirements of the various aspects of the proposed 14 CFR 25.1302 regulation.


• Additional clarification for identifying which systems and equipment should be reviewed with a higher level of scrutiny against 14 CFR 25.1302 – All participants would benefit from additional clarification and guidance related to the process for identifying which systems and equipment have design features that should be reviewed with additional scrutiny against 14 CFR 25.1302.

• Clarifying expectations about the scope of errors reasonably expected in service to be identified – All participants would benefit from additional clarification and guidance related to the expectations of the scope of errors to be identified to fulfill the requirements of paragraph (d) in the proposed 14 CFR 25.1302 regulation. Additional clarification related to how comprehensive the list of errors should be and guidance about the methods by which these errors should be identified would benefit all participants in preparing for addressing the proposed regulation.

• Clarifying expectations about required documentation – All participants would benefit from clarification about the expectations for adequate documentation for demonstrating compliance with the proposed 14 CFR 25.1302. Many participants thought that examples of the documentation would be beneficial for preparing for the proposed regulation.

• Clarifying who will have authority to make compliance decisions – All participants would benefit from clarification about who will have the authority to make compliance decisions for the proposed 14 CFR 25.1302. They would also benefit from information about whether a position for human factors designees may be created in the future and the general timeline for its creation.

4.2. Recommended Next Steps

Based on the interviews conducted, recommendations of the next steps for making a significant, positive impact on the effective implementation and compliance with the proposed 14 CFR 25.1302 are as follows:

1. Identify existing resources that contain guidance for 14 CFR 25.1302 – While this report provides some resources that address the questions posed by the interview participants, there are likely other resources not included in this report. In addition, a detailed assessment will be necessary to determine if the resources identified contain adequate information to address all of the information needs related to questions posed by the interview participants, and if not, what are the outstanding information needs.

2. Provide additional guidance through FAA documents and Industry Standards – Provide the additional detail and guidance needed as identified in the previous step in documents other than the proposed AC for 14 CFR 25.1302 as appropriate. For example, the following documents may be used to provide additional information:


o FAA Policy – Supplemental information about expectations related to various aspects of the proposed regulation may be provided through the creation of a Policy document. A FAA Policy document may provide an effective mechanism to provide both FAA personnel and aircraft and avionics manufacturers with information to support shared expectations and interpretation of the proposed 14 CFR 25.1302.

o Supplementing SAE ARP 5056 – It was suggested that supplementing the current version of the SAE ARP 5056: Flight Crew Interface Considerations in the Flight Deck Design Process for Part 25 Aircraft industry standard document with additional detail may be an effective way to provide information with guidance related to integrating human factors into the design and evaluation process of flight crew interfaces. The current version of the industry standard may not be detailed enough to provide the needed guidance, but participants suggested that with additional detail, the industry standard could be a very useful resource for 14 CFR 25.1302.

3. Conduct research to assess the effectiveness of suggested compliance methods – This research project demonstrated that there is uncertainty about the appropriate compliance methods to use for detailed human factors assessments and analyses related to the proposed 14 CFR 25.1302 regulation. It would be useful to develop and conduct a future project that would define and assess candidate methods with input from the interviews presented here. The project could result in guidelines for selecting and conducting appropriate evaluation methods, as well as how to effectively document and communicate the results.


References

Current and Proposed Federal Aviation Regulations

14 CFR Part 21 • FAA Regulation 14 CFR 21.101, Designation of Applicable Regulations

http://rgl.faa.gov/Regulatory_and_Guidance_Library/rgPolicy.nsf/0/ebc74c0510ebcae686257483004f2295/$FILE/PS-AIR-110-14%20CFR%2021.101.pdf This FAA Regulation provides criteria to be used to determine whether or not a specific change should be considered significant.

14 CFR Part 25 • Preamble to proposed 14 CFR 25.1302

http://rgl.faa.gov/Regulatory_and_Guidance_Library/rgNPRM.nsf/0/4366D39205C350158625782C00534461?OpenDocument&Highlight=faa-2010-1175;%20notice%20no.%2011-02 The preamble published in the Federal Register as a Notice to Proposed Rulemaking (NPRM) provides background information about the 14 CFR 25.1302 regulation and the reasons why the regulation has been proposed.

• Proposed FAA Regulation 14 CFR 25.1302, Installed systems and equipment for use by the flightcrew http://rgl.faa.gov/Regulatory_and_Guidance_Library/rgNPRM.nsf/0/4366D39205C350158625782C00534461?OpenDocument&Highlight=faa-2010-1175;%20notice%20no.%2011-02 The proposed 14 CFR 25.1302 regulation is a general applicability regulation in subpart F of 14 CFR part 25 that provides requirements for design attributes related to avoiding and managing flight crew error.

• FAA Regulation 14 CFR 25.1301, Function and Installation http://rgl.faa.gov/Regulatory_and_Guidance_Library/rgFAR.nsf/0/1A42288111847ABC862573980074B292?OpenDocument&Highlight=25.1301 This FAA Regulation is a general applicability regulation in subpart F of 14 CFR part 25 that provides requirements related to function and installation of equipment.

FAA Advisory Circulars

14 CFR Part 25 • Proposed FAA Advisory Circular AC 25.1302-X

The proposed Advisory Circular AC 25.1302-X provides guidance describes guidance for showing compliance with 14 CFR 25.1302 and several other rules in 14 CFR Part 25 that

http://rgl.faa.gov/Regulatory_and_Guidance_Library/rgPolicy.nsf/0/ebc74c0510ebcae686257483004f2295/$FILE/PS-AIR-110-14%20CFR%2021.101.pdf�

http://rgl.faa.gov/Regulatory_and_Guidance_Library/rgPolicy.nsf/0/ebc74c0510ebcae686257483004f2295/$FILE/PS-AIR-110-14%20CFR%2021.101.pdf�

http://rgl.faa.gov/Regulatory_and_Guidance_Library/rgNPRM.nsf/0/4366D39205C350158625782C00534461?OpenDocument&Highlight=faa-2010-1175;%20notice%20no.%2011-02�






http://rgl.faa.gov/Regulatory_and_Guidance_Library/rgFAR.nsf/0/1A42288111847ABC862573980074B292?OpenDocument&Highlight=25.1301�

http://rgl.faa.gov/Regulatory_and_Guidance_Library/rgFAR.nsf/0/1A42288111847ABC862573980074B292?OpenDocument&Highlight=25.1301�


relate to the installed equipment used by the flight crew in the operation of the aircraft. This proposed Advisory Circular describes acceptable approaches to compliance and also provides recommendations for the design and evaluation of controls, displays, system behavior, and system integration, as well as design guidance for error management.

• FAA Advisory Circular AC 25-7C, Flight Test Guide for Certification of Transport Category Airplanes http://www.faa.gov/documentLibrary/media/Advisory_Circular/AC%2025-7C%20.pdf This Advisory Circular provides guidance for the flight text evaluation of transport category airplanes. This AC includes flight test methods and procedures to show compliance with the regulations contained in subpart B of 14 CFR part 25 which address airplane performance and handling characteristics.

• FAA Advisory Circular AC 25.1523-1, Minimum Flightcrew http://rgl.faa.gov/Regulatory_and_Guidance_Library/rgAdvisoryCircular.nsf/list/AC%2025.1523-1/$FILE/AC25.1523-1.pdf The Advisory Circular provides guidance related to acceptable methods of compliance with the requirements of 14 CFR 25.1523 which contains certification requirements for the minimum flightcrew on transport category airplanes.

FAA Policies • FAA Policy Statement ANM-99-02, Guidance for Reviewing Certification Plans to

Address Human Factors for Certification of Transport airplane Flight Decks http://rgl.faa.gov/Regulatory_and_Guidance_Library/rgPolicy.nsf/0/4684b8382b57f18e8625774500705cb7/$FILE/PSanm992.pdf This Policy provides guidance to FAA Certification Teams that will enable them to conduct an effective review of an applicant’s Human Factors Certification Plan or the human factors components of a general Certification Plan. This guidance describes the sections of a Human Factors Certification Plan and the information that should be included in each section.

• FAA Memorandum ANM-01-03A, Factors to Consider when Reviewing an Applicant’s Proposed Human Factors Methods of Compliance for Flight Deck Certification. http://rgl.faa.gov/Regulatory_and_Guidance_Library/rgPolicy.nsf/0/4b68cf2dfb0ed36586256d6400548cf5/$FILE/ANM-01-03(A)-FINAL.pdf This Policy provides guidance to FAA Certification Teams that will enable them to conduct an effective review of an applicant’s Human Factors Certification Plan or the human factors components of a general Certification Plan. The information provided in the policy statement covers general information on methods of compliance for regulations related to human factors, the identification of design-specific human factors issues, and the identification of regulation-specific human factors issues.

http://www.faa.gov/documentLibrary/media/Advisory_Circular/AC%2025-7C%20.pdf�

http://rgl.faa.gov/Regulatory_and_Guidance_Library/rgAdvisoryCircular.nsf/list/AC%2025.1523-1/$FILE/AC25.1523-1.pdf�

http://rgl.faa.gov/Regulatory_and_Guidance_Library/rgAdvisoryCircular.nsf/list/AC%2025.1523-1/$FILE/AC25.1523-1.pdf�

http://rgl.faa.gov/Regulatory_and_Guidance_Library/rgPolicy.nsf/0/4684b8382b57f18e8625774500705cb7/$FILE/PSanm992.pdf�

http://rgl.faa.gov/Regulatory_and_Guidance_Library/rgPolicy.nsf/0/4684b8382b57f18e8625774500705cb7/$FILE/PSanm992.pdf�

http://rgl.faa.gov/Regulatory_and_Guidance_Library/rgPolicy.nsf/0/4b68cf2dfb0ed36586256d6400548cf5/$FILE/ANM-01-03(A)-FINAL.pdf�

http://rgl.faa.gov/Regulatory_and_Guidance_Library/rgPolicy.nsf/0/4b68cf2dfb0ed36586256d6400548cf5/$FILE/ANM-01-03(A)-FINAL.pdf�


Industry Standards

• SAE ARP5056, Flight Crew Interface Considerations in the Flight Deck Design Process for Part 25 Aircraft This Aerospace Recommended Practices (ARP) standard defines recommended flight crew interface design processes and methods for new flight deck designs as well as modifications to the flight crew interface of existing flight decks. The processes and methods described in this ARP address the integration of human factors/ergonomics, engineering, and flight operations in the design and/or modification of flight crew interfaces. This standard can be purchased from SAE International at http://engineers.ihs.com/document/abstract/UCWPJBAAAAAAAAAA.

• ISO 9241, Ergonomics of Human-System Interaction This International Standard is a multi-part standard that provides guidance on various aspects of ergonomics and human-system interaction. This standard can be purchased from the International Organization for Standardization at http://www.iso.org.

• ISO 13407, Human Centered Processes for Interactive Systems This International Standard provides guidance on human-centered design activities throughout the life cycle of computer-based interactive systems. It is aimed at those managing design processes and provides guidance on sources of information and standards relevant to the human-centered approach. This standard can be purchased from the International Organization for Standardization at http://www.iso.org.

Other Resources

• Human Factors Harmonization Working Group (HF HWG) Final Report http://www.faa.gov/regulations_policies/rulemaking/committees/documents/media/TAEhfhT1-072299.pdf The HF HWG Final Report submitted to the Transport Airplane and Engine Issues Group (TAEIG) of the Aviation Rulemaking Advisory Committee that contains a description of the work that the group did to develop an initial draft of the proposed 14 CFR 25.1302 regulation and the associated proposed Advisory Circular AC 25.1302-X.

• HFYI Design CoPilotTM application http://www.designcopilot.com The HFYI Design CoPilotTM application at http://www.designcopilot.comis an online tool for gathering the information needed to make human factors decisions related to flight deck design and evaluation.

http://engineers.ihs.com/document/abstract/UCWPJBAAAAAAAAAA�

http://www.iso.org/�

http://www.iso.org/�

http://www.faa.gov/regulations_policies/rulemaking/committees/documents/media/TAEhfhT1-072299.pdf�

http://www.faa.gov/regulations_policies/rulemaking/committees/documents/media/TAEhfhT1-072299.pdf�




Appendix A: Interview Questions for Manufacturers The following represents the questions that were used to structure the interviews with the aircraft and avionics manufacturers.

I. Level of familiarity with proposed FAA 14 CFR 25.1302 and proposed AC 25.1302-X

1. Please describe your familiarity with the proposed FAA regulation 14 CFR 25.1302 and its accompanying proposed AC.

a. Have you read the proposed regulation 14 CFR 25.1302 published in the Federal Register?

b. Have you read the proposed accompanying proposed AC to 14 CFR 25.1302 published in the Federal Register?

c. What has been your level of involvement in the creation of the proposed regulation 14 CFR 25.1302 and proposed AC?

II. Experiences with Existing Regulations

a. EASA CS 25.1302 1. Have you had any experiences complying with EASA CS 25.1302? If so,

what were they? a. What methods of compliance have you used to comply with EASA

CS 25.1302? b. Did you have any challenges showing compliance to EASA CS

25.1302? c. Is there anything that you plan to do differently in the future? d. At what point in the design process would you like to begin to

communicate with the regulatory authority about compliance with the regulation?

b. 14 CFR 25.777 1. Have you had any experiences complying with FAA 14 CFR 25.777? If

so, what were they? a. What methods of compliance have you used to show compliance

with FAA 14 CFR 25.777? b. Did you have any challenges showing compliance with FAA 14

CFR 25.777? c. Is there anything that you plan to do differently in the future? d. At what point in the design process would you like to begin to

communicate with the regulatory authority about compliance with the regulation?


III. Anticipated Impact 1. What do you see as anticipated or real impact of the release of the FAA 14

CFR 25.1302 regulation? 2. Are you doing any planning or preparations in anticipation of the release

of proposed 14 CFR 25.1302? 3. What level of change in your processes do you see as the result of the

release of proposed 14 CFR 25.1302? 4. Is there anything that you are particularly concerned about in the release of

proposed 14 CFR 25.1302? 5. Do you foresee any challenges with showing compliance with the

proposed regulation? 6. Are there any particular aspects, phrases, or paragraphs of the proposed

language of the regulation that are of particular concern to you? 7. Do you anticipate any challenges with consistent interpretation of the

proposed regulation? If so, are there strategies that could be used to address the inconsistency?

IV. Resources

a. Resources currently used 1. What resources do you currently use to show compliance to requirements

related to the Human Factors aspects of existing regulations? • Information • People • Time • Policies, Issue Papers • FAA personnel

2. Do you think those same things will be useful for the proposed FAA 14 CFR 25.1302?

b. Additional resources needed 1. Are there any additional guidance and/or resources that you anticipate

needing to more confidently understand how to adequately comply with the regulations?

2. What information, tools, or other resources do you need (or would like) from the FAA in order to be successful in complying and showing compliance with various aspects of the regulation?

3. Do you anticipate any need for additional personnel or expertise to be added to your team to assist with compliance to this regulation?


V. Detailed questions

a. Planned test and evaluation methods 1. Have you given much thought to how you would demonstrate compliance

with the proposed regulation? 2. Do you already have the systems in place to supply the information that

you will need to demonstrate compliance? If not, do you have plans related to when those systems will be developed and put in place in your organization?

3. At what point in the design process do you expect to begin communication with the FAA about compliance with 14 CFR 25.1302?

4. What challenges do you anticipate in showing compliance with the proposed regulation?

5. In summary, if you had to do this tomorrow, what methods would you use to show compliance with 14 CFR 25.1302?

6. What methods do you currently use to define and document such things as:

• Purpose of certification project • Intended function [*] • Tasks [*] • Intended user characteristics • Operational context to be considered

And do you feel that any of these would change with the introduction of 14 CFR 25.1302? [*] most important items to ask if time is short Definitions: Tasks and Functions may be similar things-- differentiated by who/what is performing them.

• Functions – that is what the systems do • Tasks – that is what the human is doing

b. Identifying errors reasonably expected in service. In the proposed new regulation, paragraph (d) states:

(d) To the extent practicable, installed equipment must incorporate means to enable the flightcrew to manage errors resulting from the kinds of flightcrew interactions with the equipment that can be reasonably expected in service.

1. What information and/or data do you think would be needed to define the errors reasonably expected in service?


2. Are these information and/or data sources that you already have available to you?

3. What challenges do you anticipate would occur with identifying errors reasonably expected in service?

c. Defining information requirements In the proposed new regulation, paragraph (a) states:


1. What process do you anticipate you would use to define the information needs of the flightcrew?

2. Do you anticipate needing to develop any new processes to define the information requirements associated with a design to comply with the proposed new regulation?

d. Design for urgency, frequency, and duration. In the proposed new regulation, paragraph (b)(2) states:

Flight deck controls and information intended for the flightcrew's use must … (b) Be accessible and usable by the flightcrew in a manner consistent with the urgency, frequency, and duration of their tasks…

1. Do you anticipate the need to adapt any new processes for evaluating the urgency, frequency, and duration of design-related tasks?

VI. Wrap up/Other input 1. Overall, how would you describe your level of readiness for the proposed

regulation? a. On a scale of 1 to 10, how ready would you say you are to comply

with the proposed regulation? 2. We are interested in the questions you have about the proposed regulation

that are outstanding. Do you have any other questions that you would like to see answered on a FAQ-type page?

3. What are the highest priorities, concerns, or needs related to preparing for the proposed regulation?

4. Do you have any other thoughts or information that you think would be useful and would like to share with us?


Appendix B: Interview Questions for Civil Aviation Authority Representatives The following represents the questions that were used to structure the interviews with the individuals from the civil aviation authorities.

I. Level of familiarity with proposed FAA 14 CFR 25.1302 and proposed AC 25.1302-X

1. Please describe your familiarity with the proposed FAA regulation 14 CFR 25.1302 and its accompanying proposed AC.

a. Have you read the proposed regulation 14 CFR 25.1302 published in the Federal Register?

b. Have you read the proposed accompanying proposed AC to 14 CFR 25.1302 published in the Federal Register?

c. What has been your level of involvement in the creation of the proposed regulation 14 CFR 25.1302 and proposed AC?

II. Experiences with Existing Regulations

a. EASA CS 25.1302 1. Have you had any experiences supporting the evaluation for compliance for

EASA CS 25.1302? If so, what were they? a. What methods of evaluation have you used to find compliance with EASA

CS 25.1302? b. Did you have any challenges determining compliance with EASA CS

25.1302? c. Is there anything that you plan to do differently in the future? d. At what point in the design process would you like the applicant to initiate

communication with you about compliance with the regulation?

b. 14 CFR 25.777 1. Have you had any experiences evaluating for compliance for FAA 14 CFR

25.777? If so, what were they? a. What methods of evaluation have you used to find compliance with FAA

14 CFR 25.777? b. Did you have any challenges determining compliance with FAA 14 CFR

25.777? c. Is there anything that you plan to do differently in the future? d. At what point in the design process would you like the applicant to initiate

communication with you about compliance with this regulation?


III. Anticipated Impact 1. What do you see as anticipated or real impact of the release of the FAA 14 CFR

25.1302 regulation? 2. Are you doing any planning or preparations in anticipation of the release of

proposed 14 CFR 25.1302? 3. What level of change in your processes do you see as the result of the release of

the proposed 14 CFR 25.1302? 4. Is there anything that you are particularly concerned about in the release of

proposed 14 CFR 25.1302? 5. Do you foresee any challenges with evaluating compliance with the proposed

FAA 14 CFR 25.1302 regulation? 6. Are there any particular aspects, phrases, or paragraphs of the proposed language

of the regulation that are of particular concern to you? 7. Do you anticipate any challenges with consistent interpretation of the proposed

regulation? If so, are there strategies that could be used to address the inconsistency?

IV. Resources

a. Resources currently used 1. What resources do you currently use to ensure compliance to requirements related

to the Human Factors aspects of the design (e.g., compliance with 14 CFR 25.777)?

• Information • People • Time • Policies, Issue Papers • FAA personnel

2. Do you think those same things will be useful for the proposed FAA 14 CFR 25.1302?

b. Additional resources needed 1. Are there any additional guidance and/or resources that you anticipate needing to

more confidently understand how to adequately evaluate compliance with the regulations?

2. What information, tools, or other resources do you need (or would like) in order to be successful in evaluating compliance with various aspects of the regulation?

3. Do you anticipate any need for additional personnel or expertise to be added to your team to assist with evaluating compliance with this regulation?


V. Detailed questions

a. Planned test and evaluation methods 1. At what point in the design process do you hope and expect that the applicant will

initiate communication about compliance with FAA 14 CFR 25.1302? 2. Are there particular test and evaluation methods, that when applied to particular

design situations, are more effective than others in demonstrating compliance? 3. What methods do you currently use to define and document such things as

• Purpose of certification project • Intended function [*] • Tasks [*] • Intended user characteristics • Operational context to be considered

And do you feel that any of these would change with the introduction of 14 CFR 25.1302?

[*] most important items to ask if time is short Definitions: Tasks and Functions may be similar things—differentiated bywho/what is performing them.

• Functions – that is what the systems do • Tasks – that is what the human is doing

b. Identifying errors reasonably expected in service. In the proposed new regulation, paragraph (d) states:

(d) To the extent practicable, installed equipment must incorporate means to enable the flightcrew to manage errors resulting from the kinds of flightcrew interactions with the equipment that can be reasonably expected in service.

1. What information and/or data do you expect to be used to support definitions of errors reasonably expected in service?

c. Defining information requirements In the proposed new regulation, paragraph (a) states:



1. What process do you plan to use to determine if the applicant has adequately defined the information needs of the pilots related to the intended function of the design?

2. Do you anticipate any challenges in evaluating whether the applicants have appropriately defined the information requirements related to a design?

d. Design for urgency, frequency, and duration. In the proposed new regulation, paragraph (b)(2) states:

Flight deck controls and information intended for the flightcrew's use must … (b) Be accessible and usable by the flightcrew in a manner consistent with the urgency, frequency, and duration of their tasks…

1. Do you anticipate the need to adapt any new processes for evaluating whether the applicant’s design appropriately accommodates the urgency, frequency, and duration of design-related tasks?

VI. Wrap up/Other input 1. Overall, how would you describe your level of readiness for the proposed 14 CFR

25.1302 regulation? a. On a scale of 1 to 10, how ready would you say you are to determine

compliance with the proposed regulation? 2. We are interested in the questions you have about the proposed regulation that are

outstanding. Do you have any other questions that you would like to see answered on a FAQ-type page?

3. What are the highest priorities, concerns or needs related to preparing for the proposed regulation?

4. Do you have any other thoughts or information that you think would be useful and would like to share with us?

human factors research supporting the anticipated release … · · 2014-02-28human factors...

Documents