module 12: quality assurance global laboratory initiative – xpert mtb/rif training package

Module 12: Quality Assurance

Global Laboratory Initiative – Xpert MTB/RIF Training Package

Global Laboratory InitiativeXpert MTB/RIF Training Package-2-

Contents of this moduleContents of this module

What is quality assurance? Key components of Xpert MTB/RIF quality assurance

programme

◦ Training and competence assessment

◦ Instrument verification

◦ Method validation

◦ Quality control and lot testing

◦ Quality indicator monitoring

◦ External quality assurance/assessment (EQA)


Learning ObjectivesLearning Objectives

At the end of this module, you will be able to:

Describe the function of the two internal quality controls in each Xpert MTB/RIF cartridge

Understand and apply the requirements for Xpert MTB/RIF instrument verification

List Xpert MTB/RIF quality indicators to be monitored

Understand and apply the requirements of on-site supervision

Understand and apply requirements for proficiency testing for Xpert MTB/RIF

Understand how to develop and apply a comprehensive approach to quality assurance


What is Quality Assurance (QA)?

“Planned and systematic activities to provide confidence that an organization fulfills requirements for quality.” [CLSI GP26-A4]

“Encompasses a range of activities that enable laboratories to achieve and maintain high levels of accuracy and proficiency despite changes in test methods and the volume of specimens tested.” [www.cdc.gov/labstandards]

Quality assurance is just one part of a Laboratory Quality Management System, required to ensure quality of all its processes

Continuous quality improvement is a critical concept to be adopted by laboratories


Key Components of Xpert MTB/RIF Quality Assurance Program

Environment- Safe and functional- Temperature control

Personnel- Trained and competent staff- Test user is documented- Current SOPs readily available

Equipment- Maintained and serviced

Supplies- Uninterrupted supplies- Appropriate transport and storage conditions

Specimens- Good qualityLabelled with unique ID- Completed request form

Internal quality monitoring- Test working properly

ExternalQuality Assessment (EQA)- Testing site‘s work checked by another site

Accurate and timely reporting- Turnaround timeResults review

Accurate, reliable and timely results


The quality improvement cycle includes the four steps Plan (P), Do (D), Check (C) and Action (A)

Non-conformities identified during routine testing and quality assurance activities should be analysed, corrective actions implemented and monitored

Steps should be repeated over time to ensure continuous improvements in lab processes

Quality improvement cycle


Safe and Functional Laboratory Environment

Bio-safety requirements similar to direct sputum smear microscopy

◦ Refer to Module 2

Secure premises for the equipment to prevent theft of the GeneXpert unit and the computer/laptop

Stable electricity supply or sufficient measures to ensure uninterrupted supply (generator, solar panels, battery/UPS backup, etc.)

Appropriate measures to prevent ambient temperature exceeding 30°C in the room where equipment will be installed (e.g. ventilation, air conditioning)

Appropriate management of biohazardous waste according to local guidelines


Trained and Competent Personnel

Qualification requirements for testing personnel determined by country’s Ministry of Health or other governing body

At least 1-2 staff per testing site◦ Basic computer literacy◦ Knowledge of laboratory registers ◦ Performs testing and interprets results◦ Conducts routine equipment maintenance◦ Troubleshoots common testing problems

Documented initial training and competency assessment

Documented annual competency assessment

◦ Refresher training as needed

Standard Operating Procedures (SOPs) must be up to date and available onsite


Maintained Equipment

Maintain all laboratory equipment in a good working condition

Perform instrument verification upon installation and after calibration/module replacement

Daily, weekly, monthly, and annual maintenance tasks performed

◦ Refer to Module 9

All maintenance tasks recorded on appropriate logs

Troubleshooting testing or instrument failures

Documentation of corrective action(s)

Warranty or service contracts in place and terms and conditions adhered to

Maintain dated service records


Adequate Supplies and Reagents

Plan procurement based on actual test consumption data. Rotate stock to ensure that oldest material is used first

◦ SOPs for supply and reagent procurement and inventory management

Adequate storage of Xpert MTB/RIF cartridges at recommended temperature range (2-28°C)

◦ Daily temperature monitoring

◦ Corrective action documentation if temperature out of range

Label all supplies and reagents with date received, and the date first opened

Any material found to be unsatisfactory should be recorded as such and removed from the laboratory immediately so it is not used

Label prepared disinfectants with name, date prepared, expiry date


Quality Specimens Ensure SOPs are available and patients are instructed in providing

good quality specimens

Ensure specimens are labeled correctly and request forms are completed

Reject specimens that are incorrectly labeled, leaking, in broken containers or of insufficient volume

Record the date specimens arrive in the laboratory

Monitor specimen transport times

Evaluate and record the sputum specimen quality

Specimens should be refrigerated at 2–8°C for a maximum of 10 days

◦ If necessary, specimens can be stored at a maximum of 35°C for up to 3 days, and then refrigerated at 2–8°C for a combined maximum duration of 10 days


GeneXpert users may be designated as admin, detailed or basic users

This designation relates to the functions and level of detail visible on the Xpert MTB/RIF software

Some quality assurance and troubleshooting functions are only available to staff with admin or detailed user access

◦ e.g. Ct values, graphs and error codes

Ensure the correct staff have appropriate access based on their job function, e.g. staff responsible for QA and troubleshooting need detailed or admin access

Functions available to different level users can be customised according to need

GeneXpert user rights and QASlide is optional, for certain audiences


Internal Quality Monitoring Each cartridge contains internal controls (Sample Processing Control

and Probe Check Control)

Positive and negative controls may be run according to local guidelines

Record the results of control samples, troubleshoot any unexpected results and monitor trends over time

Control strains should be from well-characterised (phenotypic and genotypic) clinical isolates

Routinely monitor quality indicators

◦ See quality indicator monitoring slides below


Internal Quality Monitoring: Sample Processing Control Each cartridge includes a Sample Processing Control (SPC), which

contains non-infectious spores in the form of a dry spore cake, to verify adequate processing of MTB

◦ Verifies that lysis of MTB has occurred

◦ Verifies the specimen processing is adequate

◦ Detects specimen-associated inhibition of the real-time PCR assay

SPC must be positive when the result is MTB Not Detected; and

SPC can be negative or positive when the result is MTB Detected

The test result is invalid if the SPC is negative when the test result is negative


Internal Quality Monitoring: Probe Check Control The Probe Check Control is undertaken by the system before the PCR

reaction starts. The system measures the fluorescence signal from the probes to monitor:

◦ bead rehydration

◦ reaction-tube filling

◦ probe integrity, and

◦ fluorescent dye stability

Results are automatically compared to the pre-established factory settings in the software

If Probe Check fails, then the test is stopped, and an Error result is obtained


External Quality Assessment (Assurance) (EQA)

“Inter-laboratory comparisons and other performance evaluations that may extend throughout all phases of the testing cycle, including interpretation of results; determination of individual and collective laboratory performance characteristics of examination procedures by means of inter-laboratory comparison; NOTE: the primary objectives of EQA are educational and may be supported by additional elements.” [CLSI GP27-A2]

EQA:

Is used to improve performance across the laboratory network

Is an important tool for communicating with and motivating staff

Is designed to identify and resolve problems

Is not used to punish staff


Accurate and Timely Reporting

Report Xpert MTB/RIF results within 24 hours after the specimen is received at the testing site to allow rapid treatment initiation

Appropriate sample transportation systems are required to ensure rapid shipment of samples and the reporting of results to referring sites

Record all work performed in standard format in the Laboratory Register

Analyze Xpert MTB/RIF results each month to detect changes which may indicate problems

◦ See recommended quality indicators


Quality Assurance Requirements

GLI recommends the following components for a comprehensive approach to quality assurance of the Xpert MTB/RIF assay:

1. Training and user competence assessment2. Instrument verification3. Quality indicator monitoring4. On-site supervision 5. New lot (incoming) QC testing6. Proficiency testing (PT)


Blinded re-checking (recommended for TB smear microscopy EQA) is not appropriate for Xpert MTB/RIF testing since the entire specimen is usually required to perform the test and no specimen can be saved for later re-checking

If Xpert MTB/RIF is used according to manufacturer’s specifications, method validation is not required

◦ Use of an alternative protocol or sample that is not included in the product insert should be validated by the laboratory prior to use

Quality Assurance Requirements


Instrument Verification Each module should be verified as being “fit for purpose” using

known positive and/or negative material prior to starting testing of clinical specimens

At least one verification test should be performed per module after:

◦ Instrument installation

◦ Post-calibration or swapping of instrument modules

Verification panels are now distributed routinely by Cepheid with each new instrument and with recalibrated modules

◦ Verification panels consist of a card containing 5 Dried Culture Spots (DCS) of a known concentration of whole inactivated Mycobacterium tuberculosis (rifampicin sensitive) bacilli

DCS samples should be processed according to instructions and one sample tested per module. All results are expected to be MTB Detected, RIF resistance Not Detected


If an invalid/error/no result is obtained on any module, repeat the test in that module using the extra DCS sample provided

In case of not obtaining the expected result in any module, refer to the Troubleshooting module (Module 9)

Instrument verification results should be reported to the designated GeneXpert supervisor

Cepheid should be contacted immediately in order to assist with any problems in the verification process

Instrument Verification


Site visits should be planned at regular intervals to assess laboratory/testing site practices

Usually conducted by NRL/NTP and/or partners◦ May be conducted by national level or regional/district level staff◦ Should be integrated with other on-site supervision where possible (e.g. quarterly

NTP site visits) Standardized checklist must be utilized for consistency and completeness

of information Identifies which practices are ‘good’ and which areas need improving Collect information for

◦ Planning & implementation◦ Monitoring◦ Continuous quality improvement

Especially critical during early stages of implementation of new technology Provides motivation and support to staff, especially in peripheral settings

◦ Strong relationships with staff encourages rapid reporting of any problems Enables rapid troubleshooting, re-training and corrective actions

On-site Supervision


A schedule for site visits should be drawn up in advance, preferably integrated with other supervision activities

Responsibilities for on-site supervision may be decentralised to regional/district staff◦ All staff conducting supervision visits need appropriate training and should

use standardised checklists◦ Reports should be shared with the testing site and NTRL/NTP according to

local practices On-site supervision should be conducted in conjunction with collection

and review of quality indicators Proficiency testing and monitoring of quality indicators do not negate

the need for on-site supervision◦ Poorly performing testing sites should be prioritised for on-site visits

On-site Supervision


On-site Supervision Sufficient time should be allocated for supervisory visits, including travel

time All components of Xpert MTB/RIF testing and lab workflow should be

evaluated◦ Comprehensive assessments may be conducted less frequently (e.g. annually) by

expert laboratory staff, with more frequent (e.g. quarterly) visits done by district/regional supervisors or other appropriately trained staff

◦ The extent of evaluation during each visit will depend on frequency of visit, capacity of staff and performance of laboratory (more extensive evaluation is needed in poorly performing sites)

Assess pre- and post-analytical stages (specimen collection, recording & reporting results, confirmatory testing)

Check and analyze trends in Xpert MTB/RIF indicators Run GeneXpert software reports to view frequencies and types of errors Mentor staff on troubleshooting and error reporting during site visits Visits are opportunities to:

◦ learn◦ discuss concerns ◦ solve problems


Quality Indicator Monitoring

Routine monitoring of quality indicators (performance indicators) is a critical element of quality assurance for any diagnostic test and is also an ISO requirement

Every month, each testing site should collect and analyse quality indicators Any unexplained change in quality indicators (such as increase in error rates,

change in MTB positivity rate or RIF resistance rate or significant change in volume of tests conducted) should be documented and investigated

Error rates higher than a pre-determined threshold (e.g. 5%) should be investigated

Quality indicators should be reviewed by the laboratory manager and must always be linked to corrective actions if any unexpected results or trends are observed

A standard set of quality indicators should be used for all sites conducting Xpert MTB/RIF testing

A system should be in place for centralised reporting of monthly quality indicators

Documentation of corrective actions, and subsequent improvement/normalisation of lab indicators following corrective action are critical

To be customized by each country



Each instrument should be monitored on a monthly basis using the following minimum set of indicators to evaluate proper use:

Number of tests performed per month Number and proportion of MTB detected, RIF resistance not detected Number and proportion of MTB detected, RIF resistance detected Number and proportion of MTB detected RIF indeterminate Number and proportion of MTB not detected Number and proportion of errors Number and proportion of invalid results Number and proportion of no results

Where possible, countries should collect disaggregated data according to the population group tested (HIV positive, MDR risk, extrapulmonary)

To be customized by each country


Identifying the number and type of various errors can help with troubleshooting since certain errors may be associated with processing, instrument or environmental conditionsThe following analyses may be performed: The number of errors occurring by instrument module

◦ If a particular module produces more errors over time compared with other modules, it may require repair

The number of errors occurring by user◦ If a user has a high number of errors, investigation of the specific

error types is necessary since some errors are due to incorrect specimen processing

The number of tests lost due to power outages or surges The number, duration and causes of routine interruptions in the Xpert

MTB/RIF testing service◦ Common causes are cartridge stock-outs or expiration, staff not

available, instrument/computer breakdown



Site nameDistrictPerson compiling reportReporting period (Month/ Year)

Xpert results# Xpert tests MTB Not detected (N)

A

# Xpert tests MTB detected, RIF resistance not detected (T)

B

# Xpert tests MTB detected RIFresistance detected (RR)

C

# Xpert tests MTB detected RIF indeterminate (TI) D

# error results (I) E

# invalid results (I) F

# no result (I) G

Total numbers# total Xpert tests A+B+C+D+E+F+G# total successful Xpert tests A+B+C# total unsuccessful Xpert tests D+E+F+G# total Xpert tests MTB detected B+CAnalysisError rate [E]/ [total Xpert tests]

Invalid result rate [F]/ [total Xpert tests]

No result rate [G]/ [total Xpert tests]Total unsuccessful test rate [E+F+G]/ [total Xpert tests]Xpert MTB positivity rate (Xpert MTB detected/all successful Xpert tests)

[B+C]/ [total Xpert tests]

Xpert RIF resistant rate (Xpert MTB detected RIF resistant/all Xpert MTB detected)

C/ [B+C]

Operational issuesNumber of days of routine Xpert service interruption this monthReason(s) for service interruption (enter no. days service interrupted due to each cause)

Combined data for all instruments at one site No. tests

Monthly Report Xpert ResultsTo be customized by each country

Where possible, countries should collect disaggregated data according to the population group tested (e.g. HIV positive, MDR risk, extrapulmonary)


Several systems are being piloted for remote monitoring of GeneXpert instruments

Open-source and proprietary options available Cloud-based servers and in-country servers possible Connectivity via internet or SMS Data accessed via web-based dashboards Remote monitoring can assist with collection of data relating to quality

assurance, procurement and some solutions can be linked to reporting of patient results

Remote monitoring allows near real-time data to be available simultaneously to different types and levels of staff

Remote Monitoring


Incoming batches of Xpert MTB/RIF cartridges should be tested using a sample of cartridges and at least one known positive and negative specimen, to ensure expected performance

QC testing may be centralized at a reference laboratory before distribution of a new batch of cartridges to sites

In addition to new lot QC testing, continuous monitoring at site level of performance indicators of tests, including error rates, is important for the early detection of any problems with cartridge batches due to local storage conditions or other factors

New lot (incoming batch) QC testing


“A program in which multiple specimens are periodically sent to members of a group of laboratories for analysis and/or identification, in which each laboratory’s results are compared with those of other laboratories in the group and/or with an assigned value, and reported to the participating laboratory and others.” [CLSI GP27-A2]

Ideally, a PT program checks key pre-analytical, analytical, and post-analytical processes occurring in the testing site. A number of samples are sent to the laboratory or testing site several

times per year Testing site performs Xpert MTB/RIF as they would with routine patient

specimens and report results. Results indicate quality of personnel performance and test site operations. Results are compared to expected results and across several testing sites. Results are monitored for trends over time. PT does not measure routine laboratory performance but may identify

laboratories with major deficiencies

Proficiency Testing


Proficiency Testing

PT is recommended at least once per year Feedback of PT results should be provided in a timely manner to the

testing sites and supervisory staff◦ Rapid feedback is needed to enable prompt initiation of

correction actions While on-site supervision and routine monitoring of quality

indicators are the most critical components of QA, PT helps to:◦ identify major non-conformities◦ target the most poorly performing laboratories for on-site

supervision Can be used to evaluate technician performance post-training

32


Proficiency Testing

Several options (different formats) have been evaluated and all were found to be acceptable formats◦ For examples, see Scott et al. J. Clin. Microbiol. 2014. 52: 2493-

2499.

National reference laboratories may consider preparing their own PT panels in place of an external PT scheme


Summary

Quality assurance includes all the following aspects:

◦ Safe and functional laboratory environment, trained and competent

personnel, maintained equipment, adequate supplies and reagents,

testing of quality specimens, internal quality monitoring, External

Quality Assessment (EQA), and accurate and timely reporting

A comprehensive quality assurance programme should include the following, and which are needed to fulfil ISO 15189 requirements:

1. Instrument verification using dried culture spot panels

2. Quality indicator monitoring

3. Regular on-site supervision

4. New lot QC testing

5. Proficiency testing


Quality assurance is part of the routine workload and is not a separate activity

All quality assurance activities must be documented

Feedback to testing sites and implementing corrective and preventive measures are the most critical aspects of any quality assurance programme

Quality assurance is needed whether Xpert is placed in a laboratory or non-laboratory setting

Summary


Assessment

Which quality assurance activities are recommended for Xpert MTB/RIF?

How does your country address these requirements?

What role does quality indicator monitoring play in quality assurance?

Can proficiency testing be done in place of on-site supervision?

How would you plan an on-site supervision schedule for sites performing Xpert MTB/RIF in your country?

What should happen to results of quality assurance activities?

Acknowledgements

The Xpert MTB/RIF Training Package has been developed by a consortium of GLI partners, including FIND, KNCV, US CDC, USAID and WHO, with funding from USAID (TB CARE I).The modules are based on materials originally developed by FIND, KNCV and Cepheid.

module 12: quality assurance global laboratory initiative – xpert mtb/rif training package

Documents

quality assurance slide

quality assurance activities

xpert mtbrif cartridge

internal quality controls

quality improvement

lot testing quality

proficiency testing

routine testing