Quality Infrastructure Committee (QIC) Update July 27, 2015 Agenda US Standards System Refresher Registry Approval Process Technical Merit and Standards Development Process Worksheets Whats Next from the QIC 2 US STANDARDS SYSTEM OVERVIEW 3 Definitions of a Standard Common and repeated use of rules, conditions, guidelines or characteristics for products or related processes and production methods, and related management systems practices. (NTTAA of 1995 and OMB Circular A-119 of 1998) Market-driven technical specification for a product, service, person, process or system with which compliance is voluntary. (Anonymous) 4 Types of Standards documentary (technical) standards physical (measurement) standards Certified reference material to aid with calibration of measurements Specific requirements for the operation of a laboratory related to management system and competence 5 The U.S. Standards System The U.S. standards system is voluntary, decentralized, sector and market driven and is, sometimes, competitive and duplicative. The system relies on cooperation and communication among: Industry Private sector standards organizations Stakeholders Government 6 Other Terms for Standards Best practice Guide Guideline Guidance Specification De facto standard Code 7 Codes are Special Standards Building Code Example: Used in the design, build and compliance process to construct safe, sustainable, affordable and resilient structures. Prevents disaster occurrence and manages disaster impact - minimizes the risk and effects. 8 Federal Standards Policy The National Technology Transfer and Advancement Act (NTTAA), and the Office of Management and Budget Circular (OMB) A119: Establishes a preference for federal agency use of voluntary consensus standards over government standards Encourages federal participation in standards development Authorizes the National Institute of Standards and Technology to coordinate conformity assessment activities of the agencies working with state and local government and the private sector9 Key Concepts in Voluntary Consensus Standards Development Openness All stakeholders may participate; no single interest may dominate Transparency Records/ processes open and publicly available Due Process Appeals mechanism Consensus Decisions more than majority but not unanimity 10 Openness Due Process Transparency Consensus American National Standards Institute (ANSI) ANSI Essential Requirements Documented procedures Audits to ensure processes are followed Available guidance documents Balance Lack of dominance OSAC will build off of ANSI members procedures for development and coordination of standards 11 Standards Developing Organizations (SDOs) Who are they? Professional Societies whose members seek to advance their professions, and also develop standards Trade Associations promote their industry's products, and also develop standards International Standards Organizations Country member based organization like the International Organization for Standardization (ISO) and the International Electrotechnical Commission (IEC) Other Types of Organizations Testing laboratories and those who only develop standards 12 Relevant Forensic Science SDOs 13 REGISTRY APPROVAL PROCESSS 14 This Process Relevance to OSAC Objectives and Aims 15 The purpose of the Organization of Scientific Area Committees (OSAC) for Forensic Science is to strengthen the nations use of forensic science by: providing technical leadership necessary to facilitate the development and promulgation of consensus-based documentary standards and guidelines for forensic science promoting standards and guidelines that are fit-for- purpose and based on sound scientific principles promoting the use of OSAC standards and guidelines by accreditation and certification bodies establishing and maintaining working relationships with other similar organizations This Process Relevance to OSAC Objectives and Aims 16 The aims of the OSAC are to: populate the OSAC Registry of Approved Standards and the OSAC Registry of Approved Guidelines promote and improve the communication, dissemination and use of forensic science standards, accreditation, and personnel competencies encourage forensic science service providers in the United States to implement guidelines and standards (e.g., ISO/IEC 17025, etc.) for quality and competency enlist stakeholder involvement from a broad community to provide public comment on OSAC outputs Documentary standards and guidelines that have demonstrated: Technical merit Detailed Scope Fitness for purpose Uncertainty measurement and potential bias Method validation, as appropriate Reasonable standards development process Due Process Consensus Openness Transparency Freedom from undue influence Balance of interests 17 What Goes on the OSAC Registries? OSAC Registry of Approved Standards OSAC Registry of Approved Guidelines Standard mandatory Specifies uniform methods, actions, practices, or processes, protocols Compliance recommended to be mandatory and modified only under unusual circumstances: (shall) Approved by FSSB Guideline strongly recommended Suggested methods, actions, practices, or processes to consider in absence of applicable standards Best practices that are strongly recommended but not required (should) Approved by SAC consensus due process trans- parency technical merit openness balance OSAC: Standards vs. Guidelines 18 Standards/Guidelines Registry Approval Process Overview https://workspace.forensicosac.org/kws/groups/library/documents ?folder_id=700 https://workspace.forensicosac.org/kws/groups/library/documents ?folder_id= Document Number/Identifier & TitlePurpose OSAC Registry Approval Process of Published Standards or Guidelines Process Map V1 Overarching process map and flow chart that demonstrates this processs steps and approvers OSAC Registry Approval Process of Published Standards or Guidelines Process Description V1 Provides additional details on specifics and nuances of each step QIC Form 1: Technical Merit WorksheetTool and checklist to analyze technical merit to include a detailed scope, fitness for purpose, uncertainty measurement, validation, and other details QIC Form 2: Standards Development Process WorksheetTool and checklist to analyze a standard/guidelines due process, consensus, openness, transparency, freedom from undue influence, and balance of interests during the standards and publishing process QIC Form 3: Registry RequestCover sheet to your Registry Request packet, and is a signal to the SAC and FSSB which materials are relevant QIC Form 4: Impact WorksheetTool and checklist to analyze the potential impact on the laboratory or crime scene if the standard / guideline is implemented QIC Template A: Notice of Intent to Add to Registry & QIC Template B: Justification for Non-Approval Templates to use at various stages of the registry approval process 20 RA-100: Complete and Compile Registry Request Packet 21 The subcommittee will then vote on whether to move the full packet forward to the SAC. A 2/3 quorum is required to vote, and a majority vote is required to move a packet to the next stage. RA-200-RA-600: 22 QIC Template A: Notice of Intent to Add to Registry OR QIC Template B: Justification for Non-Approval 23 RA-700 1000: QIC Subcommittee SAC RA : 24 25 RA : QIC Form -01 Technical Merit Worksheet 26 Technical Merit Worksheet - Purpose Use to guide discussions on the technical strength and scientific underpinnings of a standard or guideline. Serve as a starting point for discussions on the appropriateness of a standard or guideline for inclusion into the OSAC registry. How to Use This Form Responses to the questions outlined in the Technical Merit Worksheet are intended to be drafted and finalized during task group and/or Subcommittee deliberations. A technical point of contact should be assigned to each standard or guideline being considered for inclusion into the OSAC Registry. Two potential outcomes: Has technical merit: If the subcommittee has determined the standard/guideline has technical merit, the subcommittee includes this form along with the other documents required for the Registry Approval Packet. Does not have technical merit: If the subcommittee has determined the standard/guidelines does not have technical merit, this form is retained in the Kavi workspace as a record of discussion, or serve as a basis to have the document further edited by its authors. Technical Merit Worksheet 29 Subcommittee Reaches Consensus The subcommittee at large should now review the standard AND the initial draft of the technical merit form. After a reasonable amount of review and deliberation, the subcommittee should finalize the language and technical rating on the form. Gradients of Agreement Tool to reach consensus 1- Wholehearted Endorsement I really like it 2- Support with Reservations I can live with it 3- Abstain This issue does not affect me 4- More Discussion Needed I dont understand the issues well enough 5 Serious Disagreement I am not on board with this dont count one me Unanimity vs Consensus All standards making groups do not require unanimity to make decisions. In the OSAC, 2/3 quorum is required to vote and a majority vote is required to make decisions which will move a document forward to the next stage. With that said, CONSENSUS should be the primary aim. UnanimityConsensus Unanimity refers to agreement by every member in a particular group. In contrast, consensus is the process by which a group arrives at an agreement. QIC Form -02 Standards Development Process Worksheet 33 Standards Development Process Worksheet 34 35 Registry Request Form Indicate which forms you have provided Indicate which registry your packet is intended for For more information or answers to questions Contact your QIC liaison Biology/DNA: Alice Isenberg,Chemistry/Instrumental Analysis: Deborah Friedman,Physics/Pattern: Karin Athanas, Crime Scene/Death Investigation: Sally Aiken,Digital/Multimedia: Kris Cano, Whats Next from the QIC 37 QIC Next Steps Training Materials Complete and make available QIC training modules What is Technical Merit? Technical Merit Worksheet US Standards Development System Standards Development Process Worksheet Registry Request Form Impact Worksheet Harmonization Worksheet TIMELINE: August 2015 QIC Next Steps Appeals Process Completed first draft of Appeals process and received FSSB comments and feedback Currently revising; will finalize and distribute to OSAC Membership TIMELINE: 1 st week of August 2015 QIC Next Steps Comment Adjudication Process Complete first draft of Comment Adjudication Process Comment resolution categories Kavi process Share draft with FSSB for comment and feedback Revise as needed, and distribute to OSAC membership TIMELINE: August 2015 QIC Next Steps Working with and SDO Process and Procedure Confirm current process map is still accurate Document corresponding procedure Share draft process map and procedure with FSSB for comment and feedback Revise as needed, and distribute to OSAC membership TIMELINE: September 2015 43 Karen Reczek Chair, Quality Infrastructure Committee (QIC) Standard Coordination Office, NIST