How Accurately Are Starting Times Documented in the Medical Record? Implications forSurgical Infection Prevention Performance Measurement • Author(s): Michele R. Bozikis, MPH; Barbara I. Braun, PhD; Stephen B. Kritchevsky, PhDSource: Infection Control and Hospital Epidemiology, Vol. 31, No. 3 (March 2010), pp. 307-309Published by: The University of Chicago Press on behalf of The Society for Healthcare Epidemiologyof AmericaStable URL: http://www.jstor.org/stable/10.1086/650759 .

Accessed: 15/05/2014 19:55

Your use of the JSTOR archive indicates your acceptance of the Terms & Conditions of Use, available at .http://www.jstor.org/page/info/about/policies/terms.jsp

.JSTOR is a not-for-profit service that helps scholars, researchers, and students discover, use, and build upon a wide range ofcontent in a trusted digital archive. We use information technology and tools to increase productivity and facilitate new formsof scholarship. For more information about JSTOR, please contact support@jstor.org.

.

The University of Chicago Press and The Society for Healthcare Epidemiology of America are collaboratingwith JSTOR to digitize, preserve and extend access to Infection Control and Hospital Epidemiology.

http://www.jstor.org

This content downloaded from 195.78.109.122 on Thu, 15 May 2014 19:55:18 PMAll use subject to JSTOR Terms and Conditions

research briefs 307

symptoms than those with B. pertussis infection.9 B. holmesiihas been isolated from tracheal aspirate and pleural fluidspecimens, and has been reported to cause pneumonia andempyema.10,11 Thus, it is likely that the cough in our indexpatient was due to a B. holmesii infection.

Large pseudo-outbreaks of pertussis have been reported asa result of false-positive PCR test results.12 False-positive PCRtest results may be the result of laboratory error or, as in ourcase, the use of a nonspecific PCR that detects a closely relatedbacterial species. Healthcare facilities should adhere to properlaboratory procedures when developing and using PCR teststo diagnose pertussis, especially when the standardized kitsapproved by the US Food and Drug Administration are notavailable. Only patients with symptomatic pertussis shouldbe tested, and positive test results using nonspecific primersshould be confirmed by direct sequencing or a second PCRtest that targets a different region of the genome.

David J. Weber, MD, MPH; Melissa B. Miller, PhD;Rebecca H. Brooks, RN; Vickie M. Brown, RN, MPH;

William A. Rutala, PhD, MPH

From the 1Division of Adult Infectious Diseases, Department of Medicine,University of North Carolina at Chapel Hill, and the Departments of 2HospitalEpidemiology and 3Hospital Laboratories, UNC Health Care, Chapel Hill, NorthCarolina. (Present affiliation: Bioinformatics, University of North Carolina atChapel Hill, Chapel Hill, NC [D.J.W.].)

Address reprint requests to David J. Weber, MD, MPH, Bioinformatics, CB7030, University of North Carolina at Chapel Hill, Chapel Hill, NC 27599-7030 (dweber@unch.unc.edu).

Received January 9, 2009; accepted August 19, 2009; electronically publishedJanuary 20, 2010.� 2010 by The Society for Healthcare Epidemiology of America. All rightsreserved. 0899-823X/2010/3103-0013$15.00. DOI: 10.1086/650760

references

1. von Konig CHW, Halperin S, Riffelmann M, Guiso N. Pertussis of adultsand children. Lancet Infect Dis 2002;2:744–750.

2. Cherry JD. The epidemiology of pertussis: a comparison of the epide-miology of the disease pertussis with the epidemiology of Bordetellapertussis infection. Pediatrics 2005;115:1422–1427.

3. Preventing tetanus, diphtheria, and pertussis among adults: use of tetanustoxoid, reduced diphtheria toxoid and acellular pertussis vaccine: rec-ommendations of the Advisory Committee on Immunization Practices(ACIP) and recommendation of ACIP, supported by the Healthcare In-fection Control Practices Advisory Committee (HICPAC), for use ofTdap among health-care personnel. MMWR Recomm Rep 2006;55(RR-17):1–37.

4. Centers for Disease Control and Prevention (CDC). Guidelines for thecontrol of pertussis outbreaks (amendments made in 2005 and 2006). Atlanta,GA: CDC, 2000. http://www.cdc.gov/vaccines/Pubs/pertussis-guide/guide.htm. Accessed April 1, 2009.

5. American Academy of Pediatrics. Pertussis. In: Pickering LK, Baker CJ,Long SS, McMillan JA, eds. Red Book: 2006 Report of the Committee onInfectious Diseases. 27th ed. Elk Grove Village, IL: American Academyof Pediatrics; 2006:498–520.

6. Knorr L, Fox JD, Tilley PAG, Ahmed-Bentley J. Evaluation of real-timePCR for diagnosis of Bordetella pertussis infection. BMC Infect Dis 2006;6:62.

7. Centers for Disease Control and Prevention (CDC). Hospital-acquiredpertussis among newborns—Texas, 2004. MMWR Morb Mortal WklyRep 2008;57:600–603.

8. Bryant KA, Humbaugh K, Brothers K, et al. Measures to control anoutbreak of pertussis in a neonatal intermediate care nursery after ex-posure to a healthcare worker. Infect Control Hosp Epidemiol 2006;27:541–545.

9. Yih WK, Silva EA, Ida J, Harrington N, Lett SM, George H. Bordetellaholmesii-like organisms isolated from Massachusetts patients with per-tussis-like symptoms. Emerg Infect Dis 1999;5:441–443.

10. Russell FM, Davis JM, Whipp MJ, et al. Severe Bordetella holmesii in-fection in a previously healthy adolescent confirmed by gene sequenceanalysis. Clin Infect Dis 2001;33:129–130.

11. Dorbecker C, Licht C, Korber F, et al. Community-acquired pneumoniadue to Bordetella holmesii in a patient with frequently relapsing nephroticsyndrome. J Infect 2007;54:e203–e205.

12. Centers for Disease Control and Prevention (CDC). Outbreaks of respi-ratory illness mistakenly attributed to pertussis—New Hampshire, Mas-sachusetts, and Tennessee, 2004–2006. MMWR Morb Mortal Wkly Rep2007;56:837–842.

How Accurately Are Starting TimesDocumented in the Medical Record?Implications for Surgical InfectionPrevention Performance Measurement

Public reporting of hospital-level performance measure ratesis an effective national strategy for improving care and hasbeen linked to financial rewards or penalties based onperformance.1,2 Despite the importance of accurate data, littleis known about the reliability and validity of most efforts atperformance measure data collection after measures havebeen widely implemented.3(p13-20)

Several publicly reported measures address the time inter-val between an event and the administration of a drug orintervention.4,5 In 2007, the Hospital Quality Alliance rec-ommended5 public reporting of a surgical-site infection pre-vention measure that assesses the time interval between ad-ministration of prophylactic antibiotics and surgical incision.We undertook a prospective study to assess whether startingtimes of antimicrobial prophylaxis administration and sur-gical incision were being accurately documented in the med-ical record. These data elements are important because dis-crepancies of a few minutes can have a major effect on thehospital’s performance rate.

The purpose was to determine the accuracy of startingtimes of documented antimicrobial prophylaxis administra-tion and incision by comparing them with the times observedby independent staff (considered the gold standard). Thisstudy was ancillary to a multicenter cluster randomized trialof a quality collaborative for improving the antimicrobialprophylaxis process in 44 hospitals, the Trial to Reduce An-timicrobial Prophylaxis Errors (TRAPE).6 TRAPE hospitalswere recruited by postal and e-mail communication to mem-

This content downloaded from 195.78.109.122 on Thu, 15 May 2014 19:55:18 PMAll use subject to JSTOR Terms and Conditions

308 infection control and hospital epidemiology march 2010, vol. 31, no. 3

figure 1. Distribution of differences between documented and observed antimicrobial prophylaxis administration times and differencesbetween documented and observed surgical incision times for 96 surgical procedures performed at 25 participating hospitals.

bers of the Society for Healthcare Epidemiology of America.Protocols were approved by institutional review boards at allperformance sites. The hospital study liaison (usually the in-fection prevention specialist) was invited by means of e-mailand telephone to participate in this voluntary time-obser-vation study.

We used a structured form for recording 7 data elements:observed and documented start times of antimicrobial pro-phylaxis (defined as the time an antibiotic dose was admin-istered closest to the incision time), observed and docu-mented incision times (the time of the initial surgical inci-sion for the principal procedure), surgical procedure startdate, type of surgical procedure, and name of antibioticagent administered.

During the period from December 2003 through Febru-ary 2004, liaisons selected a convenience sample of atleast 2 surgical procedures (cardiac procedure, hip or kneereplacement, and/or hysterectomy) to observe directly. Ob-servers were instructed to be unobtrusive and nonspecificabout purpose if asked. After recording observed times onthe form, the liaison reviewed the patient record (usually thenext day) and recorded the times documented by operatingsuite personnel.

TRAPE staff assessed data for completeness and followedup with liaisons as needed. Data were analyzed using Excel(Microsoft) and SAS, version 8.2 (SAS Institute). Differenceswere calculated by subtracting documented times from ob-served times. Mean differences were calculated overall andby type of surgical procedure.

Twenty-five of 44 TRAPE sites participated in this reliabilitystudy. Ninety-six surgical procedures were observed: 33(34%) cardiovascular procedures, 38 (40%) joint replace-ments, and 25 (26%) hysterectomies. At each site, 2–6 pro-cedures were observed.

All but 1 patient received antimicrobial prophylaxis; thestart time of antimicrobial prophylaxis was documented in92 (97%) of 95 instances. The overall mean difference be-tween observed and documented prophylaxis administrationtimes was 0 minutes (standard deviation [SD], 16.3; range,�67 to 115) (Figure 1). Documented times exactly matchedobserved times in 50 (54%) of 92 cases. Documented timeswere later in 18 (20%) cases and earlier in 24 (26%) cases.Documented times were within 5 minutes of observed timesin 77 (84%) cases and within 10 minutes of observed timesin 83 (90%) cases. In only 4 cases were documented timesdifferent from observed times by 30 minutes or more. Themean differences between observed and documented pro-phylaxis administration times were 1.0 minutes (SD, 24.7)for cardiovascular surgeries, 0.6 minutes (SD, 5.2) for hys-terectomies, and �1.2 minutes (SD, 11.2) for arthroplasties.

Surgical incision time was documented for all procedures.The mean difference between observed and documentedtimes was 0.2 minutes (SD, 3.4; range, �10 to 18). In 65(68%) cases, observed and documented times were identical;90 (94%) and 93 (97%) documented times were within 5and 10 minutes of observed times, respectively. The meandifferences between observed and documented incision timeswere 0.2 minutes (SD, 2.6) for cardiovascular procedures, 1.4minutes (SD, 5.0) for hysterectomies, and �0.1 minutes (SD,2.6) for arthroplasties.

This study evaluated the accuracy of documented anti-microbial prophylaxis administration times and surgical in-cision times by comparing them with the time recorded byan independent observer. Documented times were within 5minutes of observed times in most cases. However, there wasgreater variability in the difference between observed anddocumented antimicrobial prophylaxis administration timesthan in the difference for surgical incision times. There was

This content downloaded from 195.78.109.122 on Thu, 15 May 2014 19:55:18 PMAll use subject to JSTOR Terms and Conditions

research briefs 309

little evidence of better or worse documentation practicesacross the 3 types of surgical procedures.

Our study has limitations that should be acknowledged.All study sites volunteered to participate and may not berepresentative of hospitals nationally. The number of surgicalprocedures observed in each hospital was too small to de-termine whether some hospitals performed systematically bet-ter or worse. Although we attempted to minimize the observerinfluence, the presence of an observer may have affected doc-umentation practices.

Implications for practice include the notion that periodicobservation of the antimicrobial administration process is asimple and effective method for auditing the accuracy of data.Observation of a small number of surgical procedures is use-ful to minimize burden yet provides confidence in the ac-curacy of performance measure data. To paraphrase a familiarsaying of former President Ronald Reagan, hospitals maywant to “trust but verify” their documentation of start timesto ensure that publicly reported data are indeed accurate.

Our results suggest that documented antimicrobial pro-phylaxis administration and surgical incision times are min-imally biased and can be reliably used to assess preoperativeprophylaxis timing intervals among participating hospitals.Errors in documented times should have only a modest effecton publicly reported performance measures that assess com-pliance with antimicrobial prophylaxis guidelines.

acknowledgments

We sincerely thank the following participants: Diane Baranowsky, Sandi Baus,Jennifer Belavic, Beth Boersma, Ruth Carrico, Margaret Chambers, NicoleDarr, Wanda Gillespie, Linda Greene, Kathy Harms, Nicholas Kreatsoulas,John LaJoice, Mari Mangino, Sue Marchione, Emily McCracken, Gail Mor-chel, Joyce Nichlas, Delores Nobles, Steve Parenteau, Ramesh Patel, JoycePatton, Linda Riley, Mary Urso, Mary Wisniewski, and Carol Zuckerman.In addition, we gratefully acknowledge the contributions of Cheryl Richards,Linda Kusek, and Richard Koss.

Financial support. Agency for Healthcare Research and Quality (AHRQ)grant RO1 HS11331–01. The study was conducted under an ongoing col-laboration, called the Project to Monitor Indicators, between the Society for

Healthcare Epidemiology of America, the Centers for Disease Control andPrevention, and The Joint Commission.

Potential conflicts of interest. All authors report no conflicts of interestrelevant to this article.

Michele R. Bozikis, MPH; Barbara I. Braun, PhD;Stephen B. Kritchevsky, PhD

From the Division of Quality Measurement and Research, The JointCommission, Oakbrook Terrace, Illinois (M.R.B., B.I.B.); and the Depart-ment of Internal Medicine, Wake Forest University School of Medicine(S.B.K.), and the J. Paul Sticht Center on Aging (S.B.K.), Winston-Salem,North Carolina.

Address reprint requests to Barbara I. Braun, PhD, The Joint Commission,Division of Quality Measurement and Research, One Renaissance Boulevard,Oakbrook Terrace, IL 60181 (bbraun@jointcommission.org).

Received March 27, 2009; accepted August 14, 2009; electronicallypublished January 26, 2010.� 2010 by The Society for Healthcare Epidemiology of America. All rightsreserved. 0899-823X/2010/3103-0014$15.00. DOI: 10.1086/650759

references

1. Williams SC, Watt A, Schmaltz SP, Koss RG, Loeb JM. Assessing thereliability of standardized performance indicators. Int J Qual Health Care2006;18:246–255.

2. Fung CH, Lim Y, Mattke S, Damberg C, Shekelle PG. Systematic review:the evidence that publishing patient care performance data improves qual-ity of care. Ann Intern Med 2008;148:111–123.

3. US Government Accountability Office. CMS Needs More Rigorous Meth-ods to Ensure Reliability of Publicly Released Data. January 2006 (ReportGAO-06–54). Available at: http://www.gao.gov/products/GAO-06-54. Ac-cessed March 18, 2009.

4. Nallamothu BK, Bradley EH, Krumholz HM. Time to treatment in pri-mary percutaneous coronary intervention. N Engl J Med 2007;357:1631–1638.

5. US Department of Health and Human Services. Hospital Compare:Information for Professionals—Hospital Process of Care Measures.http://www.hospitalcompare.hhs.gov/Hospital/Static/InformationforProfessionals_tabset.asp?activeTabp1&LanguagepEnglish&versionpdefault&subTabp7#POC3. Accessed March 19, 2009.

6. Kritchevsky SB, Braun BI, Bush AJ, et al. The effect of a quality improve-ment collaborative to improve antimicrobial prophylaxis in surgical pa-tients: a randomized trial. Ann Intern Med 2008;149:472–480.

This content downloaded from 195.78.109.122 on Thu, 15 May 2014 19:55:18 PMAll use subject to JSTOR Terms and Conditions