sw qual joint webinar deck (5)

Beyond FDA Compliance: 5 Hidden

Benefits of Your Trace Matrix

Slide 1Copyright © 2013 Software Quality Consulting Inc.

sponsored by

Speaker Bios

Steven R. Rakitin

President, Software Quality Consulting


Michael C. Sieve

Life Sciences Solution Engineer, Seapine

Software

• 10 years experience in compliance industries, including life

sciences, government, utilities, and aviation.

• Certified Master Black Belt in Lean Six Sigma, Software

Validation, Risk Management, and Negotiation. Well versed in

the General Principles of Software Validation and Part 11

Compliance.

• 35 years experience as a software engineer and software quality manager.

• Senior member of the IEEE Computer Society, ASQ Software Division, ASQ Biomedical Division, and the Association for the Advancement of Medical Instrumentation (AAMI).

• Helped write the first IEEE Software Engineering Standard (IEEE-STD-730 Standard for Software Quality Assurance Plans) and is currently a committee member working on revisions to both IEEE Standard 1012 (Software Verification & Validation) and 730 (Software Quality Assurance).

Topics

• Regulatory Requirements for Traceability

• Traceability Basics

• Requirements Trace Matrix (RTM) Benefits

• RTM Tool Validation

• References


Regulatory Basis for Traceability

• General Principles of Software Validation Guidance

– Software requirements traceability analysis should be conducted to trace software requirements to (and from) system requirements and to risk analysis results.

– Traceability analysis should be conducted to verify that software design implements all of the software requirements.

– As a technique for identifying where requirements are not sufficient, the traceability analysis should also verify that all aspects of the design are traceable to software requirements.


General Principles of Software Validation, FDA, Final Guidance, January 2002

Regulatory Requirements for Traceability

• General Principles of Software Validation Guidance

Source code traceability analysis is an important tool to verify that all code is linked to established specifications and established test procedures.

Source code traceability analysis should be conducted to verify that:

– Every element of software design descriptions (SDD) has been

implemented

– Source code traces back to elements in SDD and to risk analysis

– Tests trace to elements in SDD and to risk analysis

– Tests trace to source code


General Principles of Software Validation, FDA, Final Guidance, January 2002

Regulatory Requirements for Traceability

• Guidance for Pre-market Submissions Containing Software

– Traceability Analysis links together product design requirements, design specifications, and testing requirements.

– Also provides a means of tying together identified hazards with implementation and testing of mitigations.

– Traceability Analysis commonly consists of a matrix with line columns for requirements, specifications and tests, and pointers to hazard mitigations.

– Submit for review explicit traceability among these activities and associated documentation as they are essential to effective product development and to reviewer’s understanding of product design, development and testing, and hazard mitigations.


Guidance for the Content of Premarket Submissions for Software Contained in Medical Devices, May 11, 2005

Traceability Basics


• Forward Traceability

– Trace requirements from User Needs and

System Specification through software

requirements, design, test, and hazard

documents to ensure medical device

addresses needs of clinicians and patients.

– Benefits include ensuring all requirements

are implemented in design and code and

that all requirements are covered by tests.

Traceability Basics


• Backwards Traceability

– Trace each unique work product (e.g., design

element, object/class, source code

unit, test, etc.) back to its associated

requirement source(s).

– Backward traceability verifies that:

• Design and implementation match

specifications and intent

• Requirements are current with changes to

design, hazard analysis, source code, bug

fixes, and tests.

Traceability Basics

• Typical Requirements Trace Matrix (RTM)

• Connection to issue management…


User Needs

Document

System

Spec

SRS SDD Source

Code

Unit

Tests

Integration

Tests

System

Validation

Tests

Design Outputs Design Inputs Tests

RTM Benefits

• Provides a tool for estimating tests…

• Provides evidence all requirements are implemented…

• Provides evidence all requirements have been tested…

• Provides visibility for managing changes throughout product development…

• Provides evidence hazard mitigations are implemented and validated for effectiveness…


RTM as a tool to Estimate Tests

• Use RTM early to estimate tests needed…


User

Needs

System

Spec

SRS Estimated

Validation

Tests Req’d

Types of

tests

Existing

Validation

Tests

New Tests

to be

written

User Need

100

System 200

User Login

SRS 440 10 3-Positive

2-Negative

5-Boundary

VAL 4400

VAL 4500

VAL 4600

User Need

110

System 220

Power-on

SRS 450 8 2-Postive

1-Negative

5-Boundary

None VAL 8000

VAL 8010

VAL 8020

TOTAL ESTIMATED

TESTS

150 80 70

Provides Evidence Requirements Implemented


• Every SRS requirement maps to an SDD and to source code

User

Needs

System

Spec

SRS SDD Source

Code

Unit

Tests

Integration

Tests

System

Validation

Tests

User

Need 100

System 200

User Login

SRS 440 SDD 550 login.c UT 100 INT 330

VAL 4400

VAL 4500

VAL 4600

User

Need 200

System 220

Power-on

SRS 450 SDD 560 bit.c UT 200 INT 440 VAL 8000

VAL 8010

VAL 8020

TOTAL ACTUAL

TESTS

204 139 173

Provides Evidence Requirements Tested

• Tests mapped to source code and requirements…


User

Needs

System

Spec

SRS SDD Source

Code

Unit

Tests

Integration

Tests

System

Validation

Tests

User

Need 100

System 200

User Login


VAL 4400

VAL 4500

VAL 4600

User

Need 200

System 220

Power-on


VAL 8010

VAL 8020

TOTAL ACTUAL

TESTS

204 139 173

Provides Visibility for Managing Change

• Impact of change can be easily assessed…

• During development, use active links…


User

Needs

System

Spec

SRS SDD Source

Code

Unit

Tests

Integration

Tests

System

Validation

Tests

User

Need 100

System 200

User Login


VAL 4400

VAL 4500

VAL 4600

User

Need 200

System 220

Power-on


VAL 8010

VAL 8020

TOTAL ACTUAL

TESTS

204 139 173

Provides Evidence Mitigations Implemented


HardwareFailure

OperatorError

InterfaceError

MemoryLeak Algorithm

error

Usesgeneric

test strip

Doesn’t recognize

units

ExternalFactors

Thermistorfails

R14Shorts

SoftwareFailure

OROROR

Patient becomes hypoglycemic

OR

Incorrect Glucose ResultResult Delayed No Result

http://images.google.com/imgres?imgurl=http://img.alibaba.com/photo/200222026/Blood_Glucose_Meter_for_Health_Medical.jpg&imgrefurl=http://draco.en.alibaba.com/product/200222026-50170404/Medical_Implements/Blood_Glucose_Meter.html&usg=__3wi_VEp3YvGHagvEcvVtefFLgwQ=&h=745&w=634&sz=55&hl=en&start=20&tbnid=DoAI-7lCEymtEM:&tbnh=141&tbnw=120&prev=/images?q=home+use+glucose+meter&gbv=2&hl=en



Basic Events Preliminary Risk Assessment Mitigation Information Residual Risk Assessment

Basic Event

Failure Modes

Severity Likelihood Risk Index

Mitigation Verification

(Implemented)

Validation

(Effective)

Severity Likelihood Risk Index

Memory Leak

Coding error

Critical Frequent Very High

Memory leak

detection added to SRS 4.2.2.3

Refer to code review minutes

dated 8/14/13

System test SYS-2245

shows no memory leaks

Critical Occasional Moderate

Resistor

R12 fails

Age or

wearout Critical Probable High Built-in test

(BIT) to check R12 added to SRS

4.6.5.3.1

Refer to code

review minutes dated 8/15/13

System test

SYS-3020 shows R12 failure detection by

BIT

Critical Very Low Low

Failure Modes and Effect Criticality Analysis (FMECA)

Harm: Patient Becomes HypoglycemicHazard: Incorrect Glucose Result


• Mitigations reflected in the design…

• Active links: specs mitigations in Risk Tables…


User

Needs

System

Spec

SRS SDD Source

Code

Unit

Tests

Integration

Tests

System

Validation

Tests

User

Need 100

System 200

User Login


VAL 4400

VAL 4500

VAL 4600

User

Need 200

System 220

Power-on


VAL 8010

VAL 8020

TOTAL ACTUAL

TESTS

204 139 173

Seapine TestTrack Demonstration


RTM Tool Validation

• All software tools are subject to requirement for software validation, but validation approach used for each application can vary widely.

• Risk-based approach is widely used

AAMI TIR 36:2007 provides guidance on validation of software tools…


Risk Category

Tool Description Some examples

High Tool output directly affects embeddedsoftware structure, supplies data or constants used in device, or affects configuration…

Memory (EPROM or Flash)programming tools, calculationtools (spreadsheets, etc.),

Moderate Tools that support Design Controls and the Quality System

RTM, Doc control, source code control, bug tracking, complaint handling, CAPA, etc.

Low General purpose tools used to support the product development process

Word processing, spreadsheet, presentation

Training Available from SQC

• Software Development for Medical Device Manufacturers

• Medical Device Risk Management

• Software Verification & Validation

• Computer System Validation

• For more information, please visit www.swqual.com


sw qual joint webinar deck (5)

Technology

software quality manager

asq software division

testing requirements

system requirements

topicsregulatory requirements

product design requirements

risk analysis results

design specifications