antimicrobial susceptibility testing:...

ANTIMICROBIAL

SUSCEPTIBILITY

TESTING: ADVANCED

Romney Humphries, Ph.D., DABMM

Section Chief, Clinical Microbiology

University of California – Los Angeles

Lost Angeles, CA

Susan E. Sharp, Ph.D., DABMM, FAAM

Director, Airport Way Regional Laboratory

Director, Regional Microbiology

Kaiser Permanente

Portland, OR

1. Examine approached to verification studies for

antimicrobial susceptibility test systems.

2. Evaluate approaches to cascade reporting.

3. Discuss the development of an antibiogram.

4. Review the principles of pharmacokinetics and

pharmacodynamics (PK/PD) in relation to how

these apply to MIC interpretive criteria.

OBJECTIVES

Romney Humphries

UCLA Clinical

Microbiology

Los Angeles CA

STRATEGIES FOR VERIFYING

ANTIMICROBIAL

SUSCEPTIBILITY

TESTS AND TEST SYSTEMS

List antimicrobial susceptibility tests (AST) and test systems that are available for use, including their FDA status.

Describe strategies for verifying antimicrobial susceptibility tests and test systems.

Discuss criteria used to determine if verification is successful and options for documenting the verification process.

WHAT WE WILL COVER:

Disk diffusion (Kirby Bauer)

CLSI reference method

Automated zone reader (bioMIC)

Broth microdilution MIC CLSI reference method

Commercial systems

MicroScan (Siemens)

Sensititre (Trek)

AST METHODS/SYSTEMS

Etest

Automated / semi-automated systems MicroScan (Siemens)

Phoenix – (Becton Dickinson)

Sensititre (Trek)

Vitek (bioMerieux)

AST METHODS/SYSTEMS

(CON’T)

WHY DO AST DEVICES REQUIRE

FDA CLEARANCE?

To prove they are

safe and

effective and

pose minimum

risks to patients

Identified Risk…

Administration of

an inappropriate

antimicrobial

agent to a patient

Manufacturer submitted 510k to FDA

Manufacturer demonstrated that results with their AST device are comparable to those generated from a CLSI REFERENCE method.

FDA specifies the data that must be submitted and defines criteria for acceptability.

WHAT DOES IT MEAN FOR A COMMERCIAL

AST DEVICE TO BE FDA CLEARED?

WHAT DATA ARE REQUIRED BY FDA?

No. sites (including 1 in-house) 3

Organisms Fresh or stock (clinical) 100/site

Challenge * 75/one site

Reproducibility 25/site or 10x3x3/site

Interpretive criteria / standards FDA

Stability (3 lots) Real time

QC CLSI strains 20 results/site

Manufacturer’s strains Optional

On-scale At least 1

Inoculum density check QC/reproducibility/fresh

CLSI reference method MIC

http://www.fda.gov/MedicalDevices/DeviceRegulationandGuidance/GuidanceDocu

ments/ucm080564.htm * MICs near “R” breakpoint

WHAT CRITERIA ARE USED TO

DETERMINE IF AST DEVICE RESULTS

ARE ACCEPTABLE?

Criteria Defined as… Acceptable

limits

Essential

Agreement

(EA)

MIC within +/- 1

doubling dilution

of the REF MIC

>89.9 %

Category

Agreement

(CA)

S, I, and R results

agree

>89.9 %

WHAT CRITERIA ARE USED TO

DETERMINE IF AST DEVICE RESULTS ARE

ACCEPTABLE? (CON’T)

Error

Results Acceptable Error

Rate REF AST device

Very major R1 S Based on # of R

orgs tested

Major S2 R ≤ 3 %

Minor S I

Not specified R I

I S

I R

1 only isolates R by REF included in calculating error rate 2 only isolates S by REF included in calculating error rate

Essential agreement (EA) =

# within +/- 1 two-fold dilution of REF MIC

Total # isolates tested

Category agreement (CA) =

# with same S, I, or R result as REF MIC

Total # isolates tested

CALCULATING EA / CA

X

100

X

100

Very major error (VME) =

No. with VME (false R)

Total # “R” isolates tested

Major error (ME) =

No. with ME (false S)

Total # “S” isolates tested

Minor error (mE) =

No. with mE

Total # isolates tested

CALCULATING % ERRORS

X 100

X 100

X 100

Careful!

Data in

literature

sometimes

uses total

# tested

EXAMPLE: CEFAZOLIN

Isolate REF TEST EA CA Error?

1 8 S 8 S yes yes no

2 8 S 2 S no yes no

3 16 I 8 S yes no minor

4 8 S 32 R no no major

5 32 R 8 S no no very major

Breakpoints (µg/ml)

S I R

≤8 16 32

CALCULATING EA / CA

EXAMPLE: CEFAZOLIN

Organism N EA CA

# % # %

E. coli 100 91 91 94 94

C. freundii 94 89 94.6 84 89.41

1unacceptable re: FDA criteria

CALCULATING % ERRORS

EXAMPLE: CEFAZOLIN

Organism N REF

Result

Very

major

major minor

# S # R # % # % # %

E. coli 100 60 30 1 3.3 2 3.3 3 3

C. freundii 94 49 45 0 0 1 2.0 9 9.5

Intended Use

Reagents

Reporting of results

Performance

characteristics

QC

Limitations

Other

PACKAGE INSERT OR “LABELING”

SECTIONS FOR AST DEVICE

Research use only:

For Research Use Only. “Not for use in diagnostic procedures” must be in labeling

Devices are in research phase of development

Investigational use only:

“For Investigational Use Only. The performance characteristics of this product have not been established” must be in labeling

Products being tested or evaluated prior to regular marketing

Reporting:

Report should indicate that test is not FDA-cleared (RUO)

RUO AND IUO – AST DEVICE

You acquire a NEW automated instrument for AST

What should you do to make certain the AST system will produce

accurate and reproducible results?

SCENARIO:

CLIA

493.1253

Verification –

initial assessment of

NEW AST system

Each lab that introduces an unmodified, FDA-cleared or approved test system must do the following before reporting patient test results: Demonstrate that it can obtain performance

specifications comparable to those established by the manufacturer for the following performance characteristics: Accuracy

Precision (reproducibility)

Reportable range of test results for the test system. Verify that the manufacturer's reference intervals (normal values) are appropriate for the laboratory's patient population.

CLIA 493.1253 ESTABLISHMENT AND

VERIFICATION OF PERFORMANCE

SPECIFICATIONS

Accuracy

Precision (reproducibility)

USER MUST DEMONSTRATE THESE PERFORMANCE

SPECIFICATIONS WHEN USING AN FDA-CLEARED TEST

BEFORE USE FOR PATIENT TESTING…

CLIA 493.1253

Each lab that modifies an FDA-cleared or approved test system, or introduces a test system not subject to FDA clearance or approval (including methods developed in-house), or uses a test system in which performance specifications are not provided by the manufacturer must, before reporting patient test results: Establish for each test system the performance specifications

for the following performance characteristics, as applicable: Accuracy Precision Analytical sensitivity Analytical specificity to include interfering substances

Reportable range of test results for the test system. Reference intervals (normal values). Any other performance characteristic required for test

performance.

CLIA 493.1253 ESTABLISHMENT AND

VERIFICATION OF PERFORMANCE

SPECIFICATIONS

Accuracy

Precision (reproducibility)….plus…

Analytical sensitivity (AST, generally ability to

detect “R”)

Analytical specificity (AST, generally ability not to

call false “R”)

Reportable range of patient test results

Reference range(s)

Any characteristic required for test performance

or interpretation of results

USER MUST DEMONSTRATE THESE PERFORMANCE SPECIFICATIONS WHEN USING A TEST THAT IS NOT FDA-CLEARED OR USING A MODIFICATION OF AN FDA -CLEARED TEST BEFORE USE FOR PATIENT TESTING…

CLIA 493.1253 ..,..some do not apply to AST!

USAGE OF TERMS

Initial in-house

assessment of NEW

test

Ongoing evaluation

of IN-USE test

CLIA Verification Validation

CAP Verification Ongoing evaluation

JCAHO Validation Verification

Terms

used in

this talk

Verification Ongoing evaluation

Verification (initial assessment)

Accuracy Reproducibility(P

recision)

Run to run - ATCC strains

Within run - 5 bugs x 3 x 3d

Compare NEW w/ REF method

Manufacturer’s information

Literature

Anecdotal information

In-house studies

Test minimum of 100 isolates

FDA-cleared AST

…Prior to patient testing

Ongoing evaluation

QC e.g., CLSI

ATCC

Competency

PT

Correlation with clinical findings

Equipment PM

Other

FDA-cleared AST

...During patient

testing

What method should we use as REF?

How many bugs?

What kind of bugs? resistance profiles?

What is acceptable?

What if results are not acceptable?

How should we document?

WHAT DO WE HAVE TO CONSIDER WHEN

ESTABLISHING “ACCURACY” FOR USE OF A NEW

AST DEVICE IN OUR LABORATORY?

CLSI broth microdilution or agar dilution reference method

Commercial FDA-cleared broth microdilution method (historically acceptable as REF)

OLD method (that was previously verified)

WHAT METHODS SHOULD WE USE AS

REF?

Accuracy

Studies

Minimum 30; 100 preferred, more if possible

Selection criteria

Represent clinical isolates tested

Variety of susceptibility profiles

Some around breakpoints

Use organism mix on next few slides as guide

Note: some bugs fit multiple “R” criteria

HOW MANY ORGANISMS SHOULD WE

TEST?

Accuracy

Studies

Routine QC strains (CLSI)

Clinical isolates isolated in-house or from other laboratories Fresh

Stock

Resistant and susceptible isolates for each antimicrobial agent

Old PT samples

Organisms provided by manufacturer

WHERE DO WE OBTAIN

ISOLATES?

Accuracy

Studies

EXAMPLE: ORGANISM MIX (N=75)

STAPH/ENTEROCOCCUS PANEL (CON’T)

No

.

Species “R” characteristics R

(pos)

S

(neg)

5 E. faecalis vancomycin 2 3

4 E. faecalis HLAR (Gm) 2 2

4 E. faecalis HLAR (Str) 2 2

6 E. faecium vancomycin 3 3

4 E. faecium HLAR (Gm) 2 2

4 E. faecium HLAR (Str) 2 2

4 Motile

enterococci

low level

vancomycin

2 2

EXAMPLE: ORGANISM MIX (N=75)

STAPH/ENTEROCOCCUS PANEL (CON’T)

No

.

Species “R” characteristics R (pos) S (neg)

20 S. aureus mecA 10 10

2 S. aureus β-lactamase - 2

6 S. aureus inducible clindamycin 3 3

10 CoNS mecA 5 5

6 Other drug dependent*

*e.g., if rifampin reported on S. aureus, test at least 1 R and 1 S

…also

Daptomycin-NS MRSA and E. faecalis

Linezolid-R MRSA, CoNS, enterococcus

EXAMPLE: ORGANISM MIX (N=126)

ENTEROBACTERIACEAE PANEL

No. Species “R” characteristics R

(pos)

S

(neg)

8 E. coli ampicillin 4 4

8 E. coli ESBL 4 4

4 E. coli ESBL scrn +

confirm-

4

8 E. coli gentamicin 4 4

8 E. coli fluoroquinolone 4 4

8 E. coli nitrofurantoin 4 4

EXAMPLE: ORGANISM MIX (N=126)

ENTEROBACTERIACEAE PANEL

No. Species “R” characteristics R

(pos)

S

(neg)

8 C. freundii broad spec β-lac 4 4

8 E. aerogenes broad spec β-lac 4 4

8 E. cloacae broad spec β-lac 4 4

8 E. cloacae gentamicin 4 4

8 M. morganii broad spec β-lac 4 4

8 Providencia spp. broad spec β-lac 4 4

8 S. marcescens broad spec β-lac 4 4

Broad spec β-lacs = one or more from cefotaxime, ceftriaxone,

ceftazidime, cefepime, piperacillin-tazo, carbapenem

EXAMPLE: ORGANISM MIX (N=126)

ENTEROBACTERIACEAE PANEL (CON’T)

No. Species “R” characteristics R

(pos)

S

(neg)

1 K. pneumoniae KPC 1

8 Klebsiella spp. ESBL 4 4

8 P. mirabilis fluoroquinolone 4 4

1 P. mirabilis ESBL 1

4 Salmonella spp fluoroquinolone 1 3

4 Shigella any

…also

additional CRE

Low level FQ R Salmonella (if can be tested on panel)

Tigecycline-R Enterobacteriaceae

EXAMPLE: ORGANISM MIX (N=83)

NON-FERMENTER PANEL

No. Species “R” characteristic R

(pos)

S (neg)

10 P. aeruginosa R- Gm, Tob (S- Amk) 5 5

5 P. aeruginosa R- Gm, Tob, Amk 5

10 P. aeruginosa imipenem 5 5

6 P. aeruginosa piperacillin-tazobactam 3 3

5 P. aeruginosa fluoroquinolone 3 2

10 P. aeruginosa

(mucoid)

if CF patients are seen at

facility

10 Pseudomonas sp.

(not aeruginosa)

any

EXAMPLE: ORGANISM MIX (N=78)

NON-FERMENTER PANEL (CON’T)

No. Species “R” characteristic R (pos) S (neg)

5 Steno. maltophilia trimethoprim-sulfa 1 4

10 Acin. baumannii imipenem = R 5 5

4 Achrom.

xylosoxidans

any

4 Burkholderia sp. any

4 Aeromonas sp. any

VALIDATION DATA - ACCURACY

How many bugs?

What kind of bugs? resistance profiles?

Routine QC strains?

How many replicates? test days?

What is acceptable? ?

What if results are not acceptable? ?

How should we document ??

WHAT DO WE HAVE TO CONSIDER WHEN

ESTABLISHING “REPRODUCIBILIT Y” FOR USE OF A

NEW AST DEVICE IN OUR LABORATORY?

Reproducibility Studies

Routine CLSI ATCC QC strains

Data from routine testing (run to run

reproducibility)

Select 5 clinical isolates (fresh or stock) that

meet as many of the following as possible On-scale endpoints

Some resistance

Mix of high and low MICs for a given drug

Good growth characteristics

Test in triplicate on 3 days (within run and run to run

reproducibility)

Acceptable = 95% within +/- 1 dilution

PRACTICAL STRATEGY FOR

“REPRODUCIBILITY” STUDIES

Reproducibility Studies

SCENARIO #1:

YOUR NEW AST DEVICE* GENERATES THE

FOLLOWING FOR E. COLI AND

GENTAMICIN. WHAT WILL YOU DO?

N

REF Result

EA

CA

Very

major

major minor

# S #I #R # % # % # %

30 23 2 5 83 86.

6

3 60 0 0 1 3.3

*OLD system = commercial broth microdilution

Repeat NEW and REF

1 VME persists

Send to reference lab

(CLSI MIC method) Test more R bugs?

Consider:

Could isolates be clonal?

Any unique growth characteristics?

1 VME persists Agree with NEW

No VME

NOT validated

Stop

Stop

SCENARIO #2:

YOUR NEW AST DEVICE* GENERATES THE FOLLOWING

FOR P. AERUGINOSA AND PIPERACILLIN-TAZOBACTAM.

WHAT WILL YOU DO?

N

REF Result

EA

CA

Very

major

major minor

# S #I #R # % # % # %

25 15 - 10 961 80 1 10 4 26.6 NA NA

1 MIC for only 1 isolate >1 dilution from REF MIC

*OLD system = commercial broth microdilution

SCENARIO #3:

YOUR NEW DEVICE* GENERATES THE

FOLLOWING FOR S. AUREUS AND

OXACILLIN. WILL YOU USE A CEFOXITIN

BACKUP TEST?

N

REF Result

EA

CA

Very

major

major minor

# S #I #R # % # % # %

30 12 - 18 93.3 96.6 0 0 1 8.3 NA NA

*OLD system = cefoxitin disk diffusion

Most widely accepted practice:

Use validation data for panel with “cleared” organisms

Qualify results

SCENARIO #4: YOU WANT TO TEST PASTEURELLA SPP. WITH YOUR COMMERCIAL BROTH MICRODILUTION SYSTEM (PANELS WITH 2.5-5% LYSED HORSE BLOOD) THAT IS NOT FDA-CLEARED FOR TESTING THIS ORGANISM. WHAT SHOULD YOU DO?

amoxicillin 0.5 S

ceftriaxone 0.12 S

doxycycline 0.5 S

levofloxacin 0.06 S

penicillin 0.5 S

SPECIMEN: BLOOD DIAGNOSIS: LEUKEMIA PASTEURELLA MULTOCIDA

MIC (µg/ml)

“Susceptibility testing performed per Dr. Jones request;

Infectious Diseases consult suggested; testing performed

using a system FDA cleared for other similar organisms but

NOT for P. multocida”

No performance specifications from the manufacturer

Lab questioned request and contacted AST “experts”

“Experts” agreed not practical / necessary

One strategy for verifying D zone test: 1. Test CLSI QA strains (S. aureus ATCC BAA-976, BAA-

977)

2. Test 5 staphylococci with known inducible clindamycin resistance and 5 without

3. Place disks at various distances to ensure placement doesn’t cause false S or false R - use ATCC strains in 1)

SCENARIO #5: CLIA SURVEYOR ASKED LAB FOR SENSITIVITY/

SPECIFICITY DATA FOR “D ZONE TEST”

WHAT ERROR RATE IS ACCEPTABLE?

(N≥30)

Error Type

Acceptable

Error Rate

Comment

Very major 1 0

Major < 5 %

Major + Minor 2

≤ 10 %

Combined major and

minor

Essential agreement ≥ 90 %

Categorical agreement ≥ 90 %

Cumitech 31A. ASM Press,

2009.

Reference system not used as comparator

WHAT ERROR RATE IS ACCEPTABLE?

(N≥100)

Error Type

Acceptable

Error Rate

Comment

Very major 1 ≤ 3 % Minimum 35 R isolates

Major ≤ 3 %

Major + Minor 2

≤ 7 %

Combined major and

minor

Essential agreement ≥ 90 %

Categorical agreement ≥ 90 %

Cumitech 31A. ASM Press, 2009.

Arbitrate discrepancies with a Reference system

50% should exhibit some type of “R”

If significant numbers of isolates near breakpoint, CA

my be <90%

“But, do not interpret recommendations

rigidly; consider degree of difficulty

involved in detecting resistance in some

organisms.”

EXAMPLES OF ERRORS AND STRATEGIES FOR DEALING WITH THEM…..

Error Optional Strategy

5/25 (20%) VME ceftazidime / P.

aeruginosa

Unacceptable – use alternate

method

5/25 (20%) ME nitrofurantoin / C.

freundii

Use alternate method for OP urine

C. freundii “R” isolates

16/20 (80%) EA cefazolin / K.

pneumoniae (all errors in “S” range

erring on higher MIC with NEW test)

accept

4/16 (25%) ME pip-tazo / P. aeruginosa Accept...only S and R (no “I”)

breakpoints; educate ID/pharm

about MICs close to breakpoints

8/10 (80%) CA for tobramycin / P.

aeruginosa

Test additional isolates

EXAMPLES OF ERRORS AND STRATEGIES FOR DEALING WITH THEM…..

Error Optional Strategy

8/16 (50%) ME clindamycin / CoNS Do not report…offer test as

“MD request only”..use

alternate method

3/10 (30%) ME for ESBL

confirmatory test for K.

pneumoniae

Use alternate method; check if

isolates might be clonal; check

with other users

2/14 (14.3 %) VME for HLAR

gentamicin and E. faecalis

Use alternate method (sterile

sites; not generally needed

from other sites); check other

Enterococcus spp. carefully

2/16 (12.5%) VME for cefazolin and

E. cloacae

Override all cefazolin-S or –I

results to “R”

EXAMPLE: ENTEROBACTERIACEAE PANEL

Drug Post Validation Comments

Amikacin Failed verification for Serratia spp. Use alternate test

when needed

Ampicillin Edit E. cloacae to “R”

Cefazolin OK

Ceftriaxone OK

Ciprofloxacin OK

Gentamicin OK

Imipenem Only 1 “R” isolate in validation; verify all “R” patient

isolates by alternate method; test more “R”

Nitrofurantoin Failed validation for C. freundii (false R); use alternate

test for “R” urine isolates

Piperacillin-tazo OK

Tobramycin OK

Trimethoprim-sulfa OK

QUESTIONS RE: VERIFICATION

What if ??? Verify performance by….

Company rep does

validation

Showing this correlates with

performance by lab staff.

We get a loaner instrument

for 2 weeks

Testing QC strains (daily) and 25 or

so PT samples and/or previously

tested patient specimens

We relocate instrument Following manufacturer’s package

insert re: critical requirements (setup,

environment, etc.)

Use common sense and

document, document, document!!!

Documentation that a test which has been validated is

repeatedly giving the expected results

Most commonly involves following CLSI QC procedures

Specified by CLIA, CAP, JCAHO

Other components….

ONGOING EVALUATION

Ongoing evaluation

QC e.g., CLSI ATCC

Competency

PT

Correlation with clinical findings

Equipment PM

Other

FDA-cleared AST

During patient

testing

CLIA – sections 493.1253

CAP - MIC.21040, GEN.42020 – GEN.42160

Joint Commission – Quality control sections QC.1.20 –

QC.1.150

WHERE DO WE FIND INFORMATION ON LEGAL

REQUIREMENTS FOR “VERIFICATION” AND

“ONGOING EVALUATION”?

WHAT DOCUMENTS/RECORDS SHOULD

WE MAINTAIN FOR THE VERIFICATION?

Validation

Parameter

Documents

Protocol Written protocol for process used to verify NEW AST

Accuracy Data – NEW AST vs. REFERENCE, summary, conclusions

Reproducibility Data - QC with CLSI ATCC and additional QC strains ,

summary, conclusions

Putting Test IN

USE

All those documenting how test will be introduced and

protocols for ongoing evaluation (e.g., QC, competency,

PT, equipment PM, correlation with clinical findings, etc.)

*Note: good laboratory practice would dictate that records

should be maintained as long as test is in use

ENSURING QUALITY OF AST

10%

10%

10%

10%

10%10%

10%

10%

10%

10% Validation

Ongoing QC

Proficiency tests

Instrument maintenance

Competency

Relevant testingstrategiesVerify unusual results

Sup review of result

Clinical correlation

Keep current withchanges

Verification of ASTs

Also CAP’s BIT

(breakpoint implementation tool)

….but think about impact to the patient..

we must not “rationalize” that an error is OK

nor become obsessed about errors that are

unavoidable

Thank you!