1

SUMMARY OF CHANGES TO THE 2018 LEAPFROG HOSPITAL SURVEY & RESPONSES TO PUBLIC COMMENTS

PUBLISHED MARCH 23, 2018

Each year, The Leapfrog Group’s team of researchers, in conjunction with the Armstrong Institute for Patient Safety and

Quality at Johns Hopkins Medicine, review literature and convene national expert panels to ensure the Leapfrog Hospital

Survey aligns with the latest science as well the public reporting needs of purchasers and consumers. We assemble a list of

proposed changes for the next year’s Survey and release those changes for a 30-day public comment period. Comments are

reviewed carefully and used to further refine the Survey. The Survey is then pilot tested with a diverse group of hospitals

across the country. Following the pilot test, Survey content and scoring are finalized for launch on April 1.

Leapfrog received over 150 public comments in response to its proposed changes for the 2018 Leapfrog Hospital Survey.

Those comments, as well as the results from the pilot test, were incorporated into the final content and scoring algorithms

for the Survey. We have summarized the changes in this document, and included summaries and responses to public

comments in Appendix I.

We offer our sincere gratitude to all commenters for the time and thought they gave to the 2018 Leapfrog Hospital

Survey. The submitted comments were invaluable to the development of a high-quality Survey that serves our many

constituents, including purchasers and payers, as well as hospitals and the public at large.

The 2018 Survey will open on April 1, 2018. Leapfrog has already scheduled a number of informative Town Hall Calls.

Hospitals and other stakeholders can register on the Town Hall Calls webpage.

2

CONTENT AND SCORING CHANGES

SECTION 1: BASIC HOSPITAL INFORMATION

To ensure accurate reporting of pediatric admissions, Leapfrog updated the endnote describing the criteria for pediatric

admissions to include pediatric admissions (i.e. <18 years of age) to any inpatient unit, not just dedicated pediatric units.

Hospitals are instructed to exclude normal newborn admissions to the nursery and pediatric patients admitted for

maternity care, behavioral health, or discharged to hospice when reporting on pediatric admissions.

To ensure consistency with other national data sources, Leapfrog will obtain teaching status directly from the 2017 NHSN

Annual Hospital Survey - Patient Safety Component for those hospitals that join Leapfrog’s NHSN Group, provide a valid

NHSN ID in the Profile, and submit Section 1. Hospitals designated as “Major Teaching” or “Graduate” will be designated as

teaching hospitals for the purposes of the Leapfrog Hospital Survey. Teaching status is not used in scoring, but is used to

designate hospitals for the purposes of the Top Hospital Program.

Find instructions on how to join Leapfrog’s NHSN Group and deadlines for the 2018 Survey on the Join NHSN Group

webpage.

SECTION 2: MEDICATION SAFETY - COMPUTERIZED PHYSICIAN ORDER ENTRY (CPOE)

Based on feedback received during the Pilot Test, Leapfrog has added measure specifications for both questions #3 (total

number of inpatient medication orders across all units, including those without CPOE) and question #4 (total number of

those inpatient medication orders included in question #3 that were entered via a qualified CPOE system) for clarity.

Leapfrog is implementing three updates to the CPOE Scoring Algorithm:

We are increasing the target for implementation status (i.e. percentage of inpatient orders entered through an

inpatient CPOE system) from 75% to 85%. This increase will be applied to both adult/general hospitals and

pediatric hospitals.

We are increasing the target for the score on the Adult Inpatient Test via the CPOE Evaluation Tool from 50% to

60%. To achieve “Full Demonstration of National Safety Standard for Decision Support” in 2018, hospitals must

demonstrate that its inpatient CPOE system alerts prescribers to at least 60% of common serious ordering errors.

We are updating the way in which the two elements of the standard (implementation status and Adult Inpatient

Test score from the CPOE Evaluation Tool) are combined together to result in an overall CPOE Score for

adult/general hospitals. In order to be scored as “Fully Meets the Standard” in 2018, hospitals will need to have

the top CPOE test score ( “Full Demonstration of National Safety Standard for Decision Support”) and have one of

the top two implementation statuses (85% or greater or 75-84%).

Please see Appendix II for additional updates to the Adult CPOE Scoring Algorithm and a description of the order checking

categories included in the CPOE Evaluation Tool (v3.5).

Currently, there is no CPOE test available for pediatric hospitals. Therefore, pediatric hospitals will need to have the top

implementation status (85% or greater) in order to be scored as “Fully Meets the Standard.”

SECTION 3: INPATIENT SURGERY (APPLICABLE TO ADULT/GENERAL HOSPITALS ONLY)

3

SECTION 3A: HOSPITAL AND SURGEON VOLUME STANDARD

Based on minimum hospital and surgeon volume standards first published in 2015 by researchers at Dartmouth-Hitchcock

Medical Center, Michigan Medicine (University of Michigan) and Johns Hopkins Medicine, as well as information collected

in 2017 from the Leapfrog Hospital Survey, peer-reviewed literature, and consultation with national experts, Leapfrog is

implementing the following hospital and surgeon volume standards for 2018:

Procedure Hospital Volume (minimum per 12-months or 24-

month annual average)

Surgeon Volume (minimum per 12-months or 24-

month annual average) Bariatric surgery for weight loss 50 20 Esophageal resection for cancer 20 7 Lung resection for cancer 40 15 Pancreatic resection for cancer 20 10 Rectal cancer surgery 16 6 Carotid endarterectomy 20 10 Open abdominal aortic aneurysm repair 15 10 Mitral valve repair and replacement 40 20

Hospitals will be asked to report on their total hospital volume over a 12-month period or their annual average over a 24-

month period based on updated procedures and diagnosis codes that include several recommendations from participating

hospitals and health systems. Hospitals can download an Excel Workbook which includes the list of procedure and diagnosis

codes on the Survey and CPOE Materials webpage beginning April 1.

In addition, the list of high-risk procedures has been reduced from ten to eight. Leapfrog has eliminated total hip and total

knee replacement for the 2018 Survey to allow for additional time in finalizing the recommended hospital and surgeon

volume standards.

We are removing the individual surgeon volume questions due to the challenges hospitals reported in obtaining accurate

volume data on surgeons that perform that surgery at multiple facilities. Instead, hospitals will be asked whether their

process for privileging surgeons requires that the surgeon meet or exceed the minimum surgeon volume standards listed in

the table above. See the updated questions below:

1) Check all procedures that your hospital performs as defined in the Inpatient Surgery Reference Information. If your hospital does not perform the procedure, or ONLY does so when a patient is too unstable for safe transfer, OR ONLY when a procedure is urgent or emergent, do not check the box next to that procedure. If “None of the above,” please skip remaining questions in Section 3A and 3B, and go to the Affirmation of Accuracy.

Carotid endarterectomy Mitral valve repair and replacement Open abdominal aortic aneurysm repair Lung resection for cancer Esophageal resection for cancer Pancreatic resection for cancer Rectal cancer surgery Bariatric surgery for weight loss None of the above

Hospitals will only respond to questions #2 and #3 based on the procedures selected in question #1.

2) Total hospital volume for each selected procedure during the reporting period:

4

Procedure Number of Procedures Performed (12-month count or 24-month annual average)

Carotid endarterectomy

Mitral valve repair and replacement

Open abdominal aortic aneurysm repair

Lung resection for cancer

Esophageal resection for cancer

Pancreatic resection for cancer

Rectal cancer surgery

Bariatric surgery for weight loss

3) Does your hospital’s process for privileging surgeons include the surgeon meeting or exceeding the minimum surgeon volume standard listed below?

Procedure Surgeon Volume Standard

Carotid endarterectomy 10 Yes No

Plan to implement within 12 months

Mitral valve repair and replacement 20 Yes No

Plan to implement within 12 months

Open abdominal aortic aneurysm repair 10 Yes No

Plan to implement within 12 months

Lung resection for cancer 15 Yes No

Plan to implement within 12 months

Esophageal resection for cancer 7 Yes No

Plan to implement within 12 months

Pancreatic resection for cancer 10 Yes No

Plan to implement within 12 months

Rectal cancer surgery 6 Yes No

Plan to implement within 12 months

Bariatric surgery for weight loss 20 Yes No

Plan to implement within 12 months

Scoring Algorithm for the Minimum Hospital and Surgeon Volume Standard

Hospital and Surgeon Volume Standard Score (Performance Category)

For each of the surgeries performed by the hospital…

Fully Meets the Standard (four-filled bars)

The hospital met the minimum hospital volume standard for the surgery; and

5

The hospital’s process for privileging surgeons includes meeting or exceeding the minimum surgeon volume standard.

Substantial Progress (three-filled bars)

The hospital met the minimum hospital volume standard for the surgery; and

The hospital’s process for privileging surgeons does not include meeting or exceeding the minimum surgeon volume standard, but the hospital is committed to doing so within the next 12 months.

Some Progress (two-filled bars)

The hospital did not meet the minimum hospital volume standard for the surgery, but the hospital’s process for privileging surgeons includes meeting or exceeding the minimum surgeon volume standard; OR

The hospital met the minimum hospital volume standard for the surgery, but the hospital’s process for privileging surgeons does not include the minimum surgeon volume standard, and the hospital is not committed to doing so within the next 12 months.

Willing to Report (one-filled bar)

The hospital did not meet the minimum hospital volume standard for the surgery; and

The hospital does not include the minimum surgeon volume standard in its privileging policy, whether or not they are committed to doing so in the next 12 months.

Does Not Apply The hospital does not perform the surgery.

Declined to Respond The hospital did not respond to the questions in this section of the survey or did not submit a survey.

SECTION 3B: SURGICAL APPROPRIATENESS

Questions in this section focus on the hospital’s progress in developing appropriateness criteria based on published

guidelines and input from local surgeons, supporting and monitoring adherence, as well as communicating with surgeons,

hospital leaders, and board members about adherence to the criteria.

In 2018, responses to this subsection will be publicly reported but not scored by Leapfrog. When visitors to Leapfrog’s

public reporting website click the score icon for the surgical volume standard (i.e. four filled bars, three filled bars, etc.),

they will see a statement indicating whether the hospital has processes and protocols in place to ensure surgical

appropriateness. Hospitals that respond “Yes” to all five questions specific to that surgery will be reported as “Yes” and

hospitals that respond “No” to one or more of the five questions will be reported as “Not Yet.” This approach aims to

encourage hospitals to continue implementing processes and protocols to ensure surgical appropriateness while giving

them additional time before the responses are used in scoring, planned for 2019.

SECTION 4: MATERNITY CARE

There are no changes to the questions in this section.

However, there are important updates to the data specifications. For hospitals that do not submit data to The Joint

Commission (TJC) and need to retrospectively collect data using the TJC specifications provided, two of the three TJC

measures included in Section 4, Early Elective Deliveries (PC-01) and NTSV C-sections (PC-02), will use multiple TJC measure

specifications based on the discharge dates of included cases due to updates between each version:

6

v2016B1: Discharges between 01/01/2017 – 06/30/2017

v2017A1: Discharges between 07/01/2017 – 12/31/2017

v2017B1: Discharges between 01/01/2018 – 06/30/2018

Please be sure to refer to the correct specification manual for the discharge date if using TJC measure specifications as

there have been updates to the ICD-10 tables and data elements between releases, including the addition of “history of

stillbirth” in v2017A1 and v2017B1 as an exclusion for Early Elective Deliveries (PC-01).

The Antenatal Steroids (PC-03) measure will only use one set of TJC measure specifications (v2017A1) for both reporting

periods since there were no updates between the releases.

SECTION 5: ICU PHYSICIAN STAFFING (IPS)

Leapfrog has made minor updates to the wording of some of the questions and response options in Section 5 ICU Physician

Staffing to better understand hospitals’ use of tele-intensivists.

In addition, in 2018, Questions #7 and #8 will not be included in the scoring criteria for the “Some Progress” (i.e. two-filled

bars) performance category. This change will ensure consistency between responses to Question #3, which asks if all

patients in these ICUs are at any time managed or co-managed by one or more physicians certified in critical care medicine

either on-site or via telemedicine, and Questions #7 and/or #8, which asks how much time patients are managed or co-

managed by intensivists either on-site or via telemedicine.

Please see Appendix III for the 2018 IPS Scoring Algorithm.

SECTION 6: NQF SAFE PRACTICES SCORE

Due to the absence of national training/educational opportunities to support managers in integrating risk and hazard

information, Leapfrog is removing Safe Practice element 4.3.c, which is listed below, from Safe Practice 4 Risks and Hazards.

No updates to the scoring or weight for Safe Practice 4 Risks and Hazards are proposed.

In regard to developing the ability to appropriately assess risk and hazards to patients, the organization has done the

following or had in place during the last 12 months:

Senior managers have received training in the integration of risk and hazard information across the organization.

Training was documented. (pp. 107-108)

Instructions for reporting on Safe Practices in Section 6 have been updated to include information about collecting key

documentation to support each answer, as Leapfrog’s verification process includes a review of safe practices

documentation from a random selection of hospitals every month during the survey cycle (April 1 to December 31).

Leapfrog would like to thank those organizations that provided feedback on the addition of two unscored, fact-finding

questions in Safe Practice 19 Hand Hygiene related to the use of electronic hand hygiene and/or video monitoring systems.

Leapfrog will not be adding these two unscored questions to Safe Practice 19 in 2018. We are convening a national expert

panel to develop an evidence-based structural measure that represents the best practices in hand hygiene monitoring and

compliance and envision this new measure replacing Safe Practice 19 in the future.

7

SECTION 7: MANAGING SERIOUS ERRORS

SECTION 7A: NEVER EVENTS POLICY STATEMENT

Based on comments received during the public comment period, Leapfrog has made two minor updates to Section 7A

which were not originally proposed. First, we will update the wording of each practice statement to indicate fact rather

than intent. For example, the current policy states that “we will apologize to the patient and/or family effected by the never

event” and will be updated to state, “we apologize to the patient and/or family effected by the never event.” Next, we have

updated the number of days that hospitals are required to report the never event from 10 days to 15 business days. This is

consistent with the Minnesota Department of Health’s Adverse Event Reporting Systems, which has been cited by experts

as a national model for reporting adverse events. More information can be found here.

Leapfrog will implement the following scoring algorithm for Section 7A Never Events which includes the four (4) new

principles added in 2017. These results will be publicly reported in 2018.

Never Events Score (Performance Category)

Description

Fully Meets the Standard (four-filled bars)

The hospital has implemented a policy that adheres to all 9 principles of The Leapfrog Group’s Policy Statement on Serious

Reportable Events/“Never Events.”

Substantial Progress (three-filled bars)

The hospital has implemented a policy that adheres to all of the original 5 principles of The Leapfrog Group’s Policy Statement on Serious Reportable Events/“Never Events,” as well as at least 2

additional principles.

Some Progress (two-filled bars)

The hospital has implemented a policy that adheres to all of the original 5 principles of The Leapfrog Group’s Policy Statement on

Serious Reportable Events/“Never Events.”

Willing to Report (one-filled bar)

The hospital responded to the Leapfrog Hospital Survey questions pertaining to adoption of this policy, but does not yet meet the

criteria for “Some Progress.”

Declined to Respond The hospital did not respond to the questions in this section of

the Survey or did not submit a Survey.

SECTION 7B: HEALTHCARE-ASSOCIATED INFECTIONS

There are no changes to this subsection.

We are pleased that hospital feedback regarding Leapfrog’s removal of burdensome questions related to healthcare-

associated infections has been overwhelmingly positive. Therefore, in 2018, Leapfrog will again draw infections data

directly from the National Healthcare Safety Network (NHSN). To do so, hospitals must join Leapfrog’s NHSN Group,

provide a valid NHSN ID in their Leapfrog Hospital Survey Profile, and submit Section 7 of the Leapfrog Hospital Survey by

the designated deadlines. This is all that is necessary to be scored and publicly reported on the five infection measures:

CLABSI, CAUTI, MRSA, C. Diff. and SSI Colon.

There are no changes to the scoring algorithm for this section.

8

SECTION 7C: HOSPITAL-ACQUIRED CONDITIONS – PRESSURE ULCERS AND INJURIES

Due to feedback Leapfrog received from hospitals in 2017 regarding the feasibility of using the updated ICD-10 measure

specifications to report on the hospital-acquired pressure ulcers and injuries measures, Leapfrog is removing Section 7C

Hospital-Acquired Conditions – Pressure Ulcers and Injuries from the 2018 Survey.

SECTION 7D: ANTIBIOTIC STEWARDSHIP PRACTICES

We are pleased that hospitals will no longer need to devote time to responding to questions in Section 7D. Instead,

Leapfrog will obtain information regarding the hospital’s implantation of antibiotic stewardship practices directly from

CDC’s National Healthcare Safety Network (NHSN).

Hospitals will be required to join Leapfrog’s NHSN Group, provide a valid NHSN ID in their Leapfrog Hospital Survey Profile,

and submit Section 7 of the Leapfrog Hospital Survey by the designated deadlines in order to be scored and publicly

reported on the Antibiotic Stewardship Practices measure.

There are no changes to the scoring algorithm for this section.

SECTION 8: MEDICATION SAFETY

PROPOSED CHANGES TO SECTION 8A BAR CODE MEDICATION ADMINISTRATION

In Section 8A BCMA, Question #15a – 15e, hospitals currently report if they have any of five Processes and Structures to

Prevent Workarounds:

a. Has a formal committee that meets routinely to review data reports on BCMA system use

b. Has back-up systems for BCMA hardware failures

c. Has a Help Desk that provides timely responses to urgent BCMA issues in real-time

d. Conducts real-time observations of users using the BCMA system

e. Engages nursing leadership at the unit level on BCMA use

Leapfrog has added three additional processes to this question in the 2018 Survey:

Which of the following has your hospital done with the data and information identified through items a-e above:

f. In the past 12 months, used the data and information obtained through items a-e to implement quality

improvement projects that have focused on improving the hospital’s BCMA performance

OR

In the past 12 months, used the data and information obtained through items a-e to monitor a previously

implemented quality improvement project focused on improving the hospital’s BCMA performance

g. In the past 12 months, evaluated the results of the quality improvement projects (from f) and demonstrated that

these projects have resulted in higher adherence to our hospital’s standard medication administration process

OR

In the past 12 months, evaluated the results of the quality improvement projects (from f) and demonstrated

continued adherence to your hospital’s standard medication administration process

9

h. Communicated back to end users the resolution of system deficiencies and/or problems that may have

contributed to the workaround.

To meet the Processes and Structures to Prevent Workarounds component of the BCMA standard, hospitals would need to

respond “yes” to 6 out of 8 questions above. No other updates are proposed for Section 8A BCMA.

Please see Appendix IV for the 2018 IPS Scoring Algorithm.

PROPOSED CHANGES TO SECTION 8B MEDICATION RECONCILIATION (APPLICABLE TO ADULT/GENERAL

HOSPITALS ONLY)

In 2017, Leapfrog added a new NQF-endorsed medication reconciliation measure: Number of Unintentional Medication

Discrepancies per Patient (NQF 2456). The measure focuses on the quality and accuracy of the hospital’s medication

reconciliation process. The measure is developed for adult inpatients only, but Leapfrog continues to advocate for an

adaptation for pediatric patients. The data collection protocols in the measure are effective for hospitals’ quality

improvement and the measure stewards provide significant free resources and information for hospitals to implement

these protocols. Information about the significant impact of poor medication reconciliation is available on Leapfrog’s

website.

Hospitals were not scored or publicly reported on this measure in 2017. In 2018, Leapfrog will address feedback received

from hospitals in 2017. First, Leapfrog will give hospitals two options for meeting the data collection requirements for this

measure:

Hospitals that started and have continued to sample 10 patients on a quarterly basis using the 2017 Leapfrog

Hospital Survey measure specifications can use those data when reporting on this section of the Survey.

Hospitals that did not start sampling patients in 2017, can sample in real-time (i.e. after April 1) and start data

collection anytime during the survey cycle by sampling 15 patients.

Next, we will limit sampling for the measure to patients admitted to medical/surgical units only. Lastly, we made the

following updates to the data collection instructions:

Standard language that pharmacists can use to inform patients selected for the measure that the pharmacist is not

normally part of the patient’s care team, but interviewing the patient to ensure that the hospital’s medication

reconciliation process is accurate.

Instructions on how to record unintentional discrepancies that have been corrected prior to the patient’s

discharge. Pharmacists are expected to intervene upon identification of an error on the admission or discharge

orders (i.e. discrepancy between the Gold Standard Medication History the pharmacist obtained from the patient

and the admission or discharge orders), and these discrepancies should still be recorded in the measure.

Hospitals will continue to report the number of unintentional medication discrepancies identified between the Gold

Standard Medication History obtained by a trained pharmacist and the admission and discharge orders, including the

number of additional unintentional medications.

Hospitals who submit this section of the 2018 Leapfrog Hospital Survey, and whose responses are not flagged during

Leapfrog’s monthly data review, will be scored as “Fully Meets the Standard” for having a protocol in place to collect data

10

on the accuracy of the hospital’s medication reconciliation process. Hospitals that submit this section, but whose responses

are flagged during Leapfrog’s monthly data review for potential data entry errors, will be scored as “Willing to Report” for

beginning to put a protocol in place to collect data on the accuracy of the hospital’s medication reconciliation process.

Hospitals that do not submit this section will be scored as “Declined to Respond.” In 2018, Leapfrog does not intend to

publicly report a hospital’s rate of unintentional medication discrepancies per patient.

SECTION 9: PEDIATRIC CARE

SECTION 9A: CAHPS CHILD HOSPITAL SURVEY

Hospitals with at least 500 annual pediatric inpatient admissions (patients <18 years of age) to any unit (adult or pediatric,

including any level ICU) will be asked to administer the CAHPS Child Hospital Survey or be publicly reported as “Declined to

Respond.”

Next, hospitals with at least 100 returned CAHPS Child Hospital Surveys will be asked to report their Top Box Score for each

of the 18 domains of patient experience. Hospitals that are administering the survey, but had fewer than 100 returned

surveys will be reported as “Unable to Calculate Score.”

Results from this section will be scored and publicly reported in 2018. Based on additional feedback from stakeholders

received after the Proposed Changes were published, Leapfrog will publicly report the Top Box Scores for all 18 domains,

but calculate the performance category based on five (5) of the domains, rather than the nine (9) as originally proposed.

Those five will include:

Communication with Parents – Communication about your child’s medicines

Communication with Parents – Keeping you informed about your child’s care

Communication with Children – How well nurses communicate with your child

Communication with Children – How well doctors communicate with your child

Attention to Safety and Comfort – Preventing mistakes and helping you report concerns

Quartile values for each of the 5 domains listed above will be calculated based on the range of hospital performance

reported on 2018 Leapfrog Hospital Surveys submitted by June 30. Hospitals will receive points for each of the 5 domains

based on how they compare to the quartile cut-points. Hospitals that perform in the top quartile will receive 4 points for

that domain; those that perform in the 3rd quartile will receive 3 points, etc. Then, the percentage of points earned over all

domains is calculated to determine the overall performance category. The percentage of points required for each of the

four performance categories (i.e. “Fully Meets the Standard,” “Substantial Progress,” etc.) will be determined by the

distribution of total points earned using Surveys submitted by June 30.

SECTION 9B: PEDIATRIC COMPUTED TOMOGRAPHY (CT) RADIATION DOSE

Leapfrog will limit reporting to head scans and abdomen/pelvis scans given the low frequency of chest and

chest/abdomen/pelvis scans observed in the 2017 Survey. Based on feedback received after the Proposed Changes were

published and concerns about very low median dose values, Leapfrog will continue to ask hospitals to report their 25th, 50th,

and 75th percentile dose values (DLP) for head scans and abdomen/pelvis scans in the five age ranges.

11

Hospital responses will be scored and publicly reported for head scans and abdomen/pelvis scans separately. Leapfrog will

score this section of the Survey by comparing the hospital’s median dose for each anatomic region and age stratum to two

benchmarks. The first benchmark is the Median Benchmark, which will be the median of the median doses reported across

all Leapfrog-reporting hospitals as of June 30, 2018. The second benchmark is the median of the 75th percentile doses

reported across all Leapfrog-reporting hospitals as of June 30, 2018.

Hospitals will receive points based on their reported median dose compared to the benchmarks. If the hospital’s reported

median dose is less than the Median Benchmark, it will receive 2 points. If the hospital’s reported median dose is greater

than or equal to the Median Benchmark and less than the 75th Percentile Benchmark, it will receive 1 point. Otherwise, if

the hospital’s reported median dose is greater than or equal to the 75th Percentile Benchmark, it will not receive points for

that category.

Therefore, for each anatomic region, there are at most 10 possible points. If a hospital had less than 10 CT scans for an age

stratum, that age stratum is not included in scoring. For each anatomic region, the percentage of points awarded is

calculated by summing the points earned and dividing by the total number of possible points (i.e. 2 times the number of age

strata with at least 10 CT scans). This percentage of points earned will be used to assign a performance category according

to the table below:

Pediatric CT Dose Score (Performance Category)

Head Scans

Abdomen/Pelvis Scans

Fully Meets the Standard (four-filled bars)

>= 75% of total possible points >= 75% of total possible points

Substantial Progress (three-filled bars)

>= 50% and < 75% of total possible points >= 50% and < 75% of total possible points

Some Progress (two-filled bars)

>=25% and < 50% of total possible points >=25% and < 50% of total possible points

Willing to Report (one-filled bar)

< 25% of total possible points < 25% of total possible points

Unable to Calculate Score Fewer than 10 CT scans for all age ranges Fewer than 10 CT scans for all age ranges

Does Not Apply The hospital does not perform CT scans on pediatric patients.

Declined to Respond The hospital did not measure pediatric scan doses or did not submit a survey.

12

APPENDIX I: RESPONSES TO PUBLIC COMMENTS

Leapfrog received over 150 public comments in response to the proposed changes to the 2018 Leapfrog Hospital Survey.

Comments were submitted from health care organizations, as well as health care experts, patient advocates, and

purchasers.

Responses to the public comments are organized by survey section below. If you submitted a comment, and do not see a

response, or if you have additional questions, please contact the Help Desk at https://leapfroghosptialsurvey.zendesk.com.

SECTION 1 BASIC HOSPITAL INFORMATION

Several commenters expressed agreement with Leapfrog’s decision to obtain teaching status from the NHSN Patient

Safety Component.

Leapfrog is committed to aligning measures and designations with other national organizations as appropriate. Find a

complete national crosswalk of 2018 Leapfrog Hospital Survey measures here.

Several comments expressed agreement with Leapfrog’s decision to update its definition for “pediatric admissions” to

include admission to any inpatient unit (i.e. adult units).

This change was in direct response to several hospitals that submitted feedback during the 2017 Survey cycle. We

appreciate continued feedback on refining the new definition.

Several commenters had questions about which patients to include when reporting on the new pediatric admissions

question.

The definition of “Total Pediatric Acute-Care Admissions” has been updated to include acute-care medical and surgical

pediatric (aged 17 years or younger) admissions to any inpatient unit. Include transfers from other hospitals as admissions

to your hospital. Include any admissions directly to an ICU or NICU (any level NICU) in your hospital, even if counted in

question #9. Exclude normal newborn admissions to the nursery and pediatric patients admitted for maternity care,

behavioral health, or discharged to hospice.

SECTION 2 COMPUTERIZED PHYSICIAN ORDER ENTRY (CPOE)

Some commenters expressed concern regarding the increased CPOE utilization target due to low utilization by non-

employed physician groups.

While contracting with external physician groups may present unique challenges in engaging physicians to use CPOE,

hospitals’ first and foremost responsibility is the safety of patients. Leapfrog expects hospital leadership to use maximum

leverage to ensure all physicians utilize CPOE for medication ordering.

Some commenters felt that requiring physicians to use CPOE to enter all orders reduces the speed and efficiency of care.

While entering orders through CPOE can add some time to the ordering process, significant evidence suggests the decision

support embedded in CPOE systems provides protection to patients from potentially life-threatening medication errors.

Leapfrog’s expert panel concluded that these safety benefits outweigh the possible inefficiencies introduced by ordering

through CPOE.

13

Some commenters expressed concern that some of their physicians refuse to use CPOE due to the perceived steep

learning curve.

While learning to use CPOE may require a time commitment by physicians, the benefits of CPOE are well established in the

literature as important to patient safety. The adoption of CPOE has been shown to significantly reduce the risk of adverse

drug events, the most common error made in hospitals.

Some commenters noted that community physicians’ CPOE system interfaces may not sync with the hospital’s CPOE

interface due to a lack of financial and personnel resources.

While aligning community physician CPOE system interfaces with hospital CPOE interfaces may require resources, the

added safety that the CPOE provides patients should be a top priority. To the extent this problem threatens patient safety,

hospitals should find immediate solutions, possibly to include assisting community physicians with the human and financial

resources to help implement the needed interfaces.

Some commenters expressed concern over hospitals’ ability to meet the increased targets for both utilization (change

from 75% to 85%) and efficacy (change from 50% to 60%) and suggested we give hospitals additional time to achieve

these thresholds.

While the higher standards may prove difficult for some hospitals to meet initially, results from previous Leapfrog Hospital

Surveys show that most hospitals are already meeting these standards. Leapfrog also believes it is important to continue

encouraging hospitals to improve the safety of their care delivery systems. For these reasons, Leapfrog has decided to move

forward with the proposed changes of increasing the targets for both utilization and efficacy.

Several commenters expressed agreement with the changes to the CPOE Scoring criteria.

Based on an analysis of 2017 Leapfrog Hospital Survey Results, almost 50% of hospitals are already meeting the targets of

the new, higher criteria.

Some commenters requested that the Sample CPOE Test (via the CPOE Evaluation Tool) be expanded to include

additional test scenarios.

The Sample Test is designed to give hospitals, including the licensed prescriber, an opportunity to practice completing the

test process (e.g., time limits and steps, browser compatibility, lab and ADT links, etc.). The Sample Test is an abridged

version of the Adult Inpatient Test, and not fully representative of the Test Patients and Test Orders that will appear in the

Adult Inpatient Test. It is not meant to predict a hospital’s score on the Adult Inpatient Test. Sample Test materials consist

of only two Test Patients and four Test Orders. Leapfrog will publish a guidance document developed by the CPOE

Evaluation Tool developers to provide hospitals with information on how to improve their clinical decision support

capability on the Survey and CPOE Materials webpage on April 1.

A commenter asked how the 10 order-checking categories included in the CPOE Evaluation Tool were selected

The ten order-checking categories included in the CPOE Evaluation Tool were identified by the CPOE Evaluation Tool

developers using several sources: data from the ISMP Medication Error Reporting System, peer reviewed literature, and

data from several large health systems including the Veteran’s Health Administration, Duke Health, Boston University, and

Partners Healthcare. A complete list and description of the order-checking categories is available in Appendix II.

Several commenters expressed frustration with their inability to customize their CPOE system’s clinical decision support

functions and asked if there is a role that the CPOE Evaluation Tool developers can play in communicating safety

requirements to the major vendors.

The CPOE Evaluation Tool developers have developed a guidance document for hospitals that participate in the CPOE

14

Evaluation Tool, which includes information to help hospitals update and improve their clinical decision support system.

Hospitals can share this document with their CPOE vendors. Leapfrog will be publishing this document on the Survey and

CPOE Materials webpage on April 1.

SECTION 3 INPATIENT SURGERY

Some hospitals had questions about how Leapfrog’s new surgeon volume question would affect surgeons who practice

at multiple hospitals.

Based on hospital feedback, in lieu of asking hospitals to report on the exact volume for each surgeon, in the 2018 Leapfrog

Survey Leapfrog will ask hospitals if they require their surgeons to have minimum experience with a procedure to be

privileged at that hospital to perform that procedure. Leapfrog’s expert panel has recommended the appropriate minimum

level of surgeon volume for each procedure. We expect that hospitals will consider total experience in the privileging

process, not just experience gained at a single facility.

Some hospitals expressed confusion regarding how to determine which surgeons are privileged at the hospital to

perform the eight procedures.

Leapfrog recommends checking with surgical leadership at your hospital on the process your hospital uses for privileging

surgeons. Every surgeon must have “privileges” to perform specific procedures.

Several commenters noted that they do not support minimum hospital and/or surgeon volume standards as outcomes

measures are most important to patients.

While Leapfrog recognizes that volume is not a perfect substitute for outcomes, for the eight surgeries being measured in

the 2018 Leapfrog Survey, there is a strong body of evidence linking hospital and surgeon experience and patient outcomes.

Leapfrog continues to explore opportunities to incorporate evidence-based, endorsed outcome measures (e.g., mortality,

morbidity) into the survey.

A commenter expressed concern about Leapfrog implementing the surgical volume standards in 2018 because hospitals

may not have Leapfrog’s minimum volume standards as part of their current privileging process, and putting that in place

will take time.

As described in Section 3A of this document, hospitals will be able to earn partial credit for committing to include Leapfrog’s

Minimum Surgeon Volume Standards in their privileging process within the next 12 months.

Two commenters noted that procedures may be done by a primary and assisting surgeon, and wanted to ensure the

assistant surgeon received credit for the case.

For determining surgeon volume, if a surgeon assists another surgeon during a procedure, the procedure should count for

both surgeons’ procedure totals. This would apply when both surgeons are experienced, practicing surgeons. If one or more

of the surgeons is a resident, fellow, or being proctored by an experienced surgeon, only the experienced surgeon should

receive credit toward her/his procedure total. Surgeons who have just finished their training should receive a 24-month

grace period to build up their experience. After that point, his/her volume should be tracked for the surgeon volume. The

procedures performed by this surgeon during the reporting period should still be counted towards the hospital’s volume

total, as the broader staff still had the experience with the surgery.

Several commenters noted that the bariatric surgery cases should be limited to those undergoing the procedure for

weight loss.

Leapfrog found this comment very helpful and has worked to clarify for the 2018 Survey that bariatric surgery needs to be

15

done explicitly for weight loss. Hospitals will be given two sets of instructions for identifying these cases: (a) use the listed

ICD-10 diagnosis codes to identify those patients who underwent the procedure where obesity was the primary diagnosis

or (b) use the listed diagnosis codes to identify those patients who underwent the procedures, where obesity was not the

primary diagnosis, but chart review indicated the procedure was performed for the purpose of weight loss.

One commenter suggested that Leapfrog give additional credit to hospitals that participate in national clinical registries

(i.e. NSQIP) and/or allow hospitals to share their NSQIP outcomes in place of volume.

Leapfrog appreciates the importance of registry participation, and continues to work with its surgical expert panel to

explore opportunities to add additional outcomes measures into the survey, including clinical outcome measures used in

registries. We welcome feedback from hospitals on potential barriers to participating in a national registry.

Some commenters questioned why Leapfrog is asking hospitals to develop surgical appropriateness criteria only for

these particular eight procedures, even though there may be other procedures with more documented evidence of

overuse.

Given that Leapfrog is measuring hospital performance based on volume for these eight procedures, Leapfrog seeks to

ensure that appropriate safeguards are in place for these eight procedures to ensure there isn’t a perverse incentive for

hospitals to operate when not necessary. We recognize the importance of overuse as a more widespread problem, and

over time, Leapfrog plans to explore additional opportunities to address it.

A commenter expressed concern about the burden of implementing appropriateness criteria without sophisticated EHR

capabilities.

While the implementation of appropriateness criteria can be streamlined and more easily tracked with EHR capabilities,

Leapfrog’s research and discussions with hospitals have indicated that implementation and monitoring of appropriateness

criteria can be achieved without the use of EHR capabilities. Leapfrog has convened an Advisory Workgroup who will be

helping to identify resources for hospitals to begin the work of developing, implementing, and monitoring appropriateness

criteria. We are planning to host a webinar later this spring.

SECTION 4 MATERNITY CARE

No comments were submitted.

SECTION 5 ICU PHYSICIAN STAFFING

Commenters expressed support for Leapfrog’s minor update to the scoring algorithm.

Leapfrog continues to look for opportunities to clarify the questions and scoring algorithm for Section 5. Additional

feedback on the reformatting of the questions in Section 5, as well as the re-formatting of the scoring algorithm are

welcomed.

SECTION 6 NQF SAFE PRACTICES

With regard to Leapfrog’s request for feedback on adding unscored questions to Safe Practice 19 Hand Hygiene on the

use of electronic hand hygiene monitoring, commenters expressed mixed perspectives. Some commenters noted that

electronic monitoring is potentially cost prohibitive for small hospitals while others expressed strong support for

electronic monitoring as a best practice.

Leapfrog shared the comments with its technical experts and determined that further consultation and fact-finding is

16

necessary before adding this to the Survey. Moreover, given the importance of hand hygiene and statistics that point to

poor compliance, Leapfrog will convene a national expert panel to review the entirety of Safe Practice 19 Hand Hygiene,

including reviewing the latest literature and evidence-based best practices for monitoring compliance with hand hygiene

protocols.

SECTION 7 MANAGING SERIOUS ERRORS

Several commenters expressed concern over Leapfrog’s Never Evens Policy Statement that requires hospital to notify an

external agency within 10 days.

Leapfrog’s technical experts have reviewed the comments and recommended that Leapfrog align with the Minnesota

Adverse Event Reporting requirements in 2018. Therefore, Leapfrog has updated the requirement to report to an external

agency on the 2018 Survey from 10 days to 15 business days.

Several commenters supported Leapfrog’s decision to remove the Hospital-Acquired Pressure Ulcers and Injuries

measures from the 2018 Survey, but some commenters noted that these are important safety measures and should not

be dropped from the Survey.

Leapfrog agrees that measures of hospital-acquired conditions are critical to patients, families, and to healthcare

purchasers. However, given the burdens and potential for error reported by hospitals in 2017 in identifying over 3,650 ICD-

10 diagnosis codes for hospital-acquired injuries, Leapfrog must remove these measures for 2018. We will continue to

include the DRA HAC falls with trauma measure and the PSI 3 Pressure Ulcer Rate in the Hospital Safety Grade.

SECTION 8 MEDICATION SAFETY

With regard to Bar Code Medication Administration (BCMA), several commenters expressed concern about Leapfrog

using the three new process/structural measures to monitor and prevent workarounds in scoring in 2018.

Leapfrog has reviewed the comments as well as the proposed scoring algorithm for 2018. In 2018, hospitals will be able to

meet the Leapfrog standard with 6 out of 8 processes or structures in place to prevent workarounds. The updated scoring

algorithm is detailed in Appendix IV.

With regard to Medication Reconciliation, several commenters expressed concern that the sample sizes are too small,

and therefore, not valid.

Leapfrog is taking a phased approach to rolling out this measure on the Survey. In 2017, hospitals were asked to sample 10

patients, and responses were not scored or publicly reported. For 2018, hospitals will be asked to sample 15 patients and

Leapfrog will publicly report whether or not the hospital has a process in place to protocol in place to collect data on the

accuracy of the hospital’s medication reconciliation process. We will not score and publicly report a hospital’s rate of

unintentional medication discrepancies. We will monitor and study the results to determine future reporting and standards

for this endorsed measure, which is the only endorsed measure of its kind.

With regard to Medication Reconciliation, some commenters questioned why pediatric patients are excluded?

The medication reconciliation measure on the Survey is endorsed by the National Quality Forum (NQF 2456: Number of

Unintentional Medication Discrepancies per Patient), and unfortunately this measure is currently specified for adult

patients only. We will advocate with NQF and measure developers to adapt the measure for pediatric patients.

With regard to Medication Reconciliation, some commenters expressed concern that limiting the sampling to patients

admitted to medical/surgical units would not result in a representative sample from the hospital and may exclude

17

patients with multiple chronic conditions and polypharmacy who are most at risk for medication reconciliation errors.

In order to respond to hospital’s feedback regarding the challenges of scheduling the interview with the pharmacist and the

patient to collect the Gold Standard Medication History, Leapfrog is allowing hospitals to restrict sampling to

medical/surgical units in 2018. However, hospitals are welcome to sample for additional units if they choose. Sampling from

medical/surgical units is the minimum requirement.

Two health systems that participated in the national Pilot of the 2018 Leapfrog Hospital Survey noted that they will be

monitoring the time spent on data collection for the Medication Reconciliation measure in 2018.

Any feedback hospitals collect regarding the time commitment to complete data collection would be greatly appreciated. In

addition, hospitals may want to refer to the information from the Medication Reconciliation Technical Assistance Calls

hosted in 2017 where three different hospitals experienced with the measure shared ways to reduce the time burden of

data collection. Materials are posted on the Town Hall Calls webpage.

SECTION 9 PEDIATRIC CARE

Regarding the CAHPS Child Hospital Survey, several commenters expressed concern with administering the survey for

NICU discharges.

The Child CAHPS Hospital Survey was designed to be administered to pediatric discharges including NICU discharges.

Additional details on fielding the CAHPS Child Hospital Survey can be found here. In 2018, hospitals that have been

administering the CAHPS survey without including NICU discharges in their sample can report those results to Leapfrog,

provided they meet the minimum sample size and timing requirements in the Leapfrog Hospital Survey. However, we are

urging those hospitals to begin including NICU discharges-- per the manual guidelines-- immediately, as CAHPS is designed

to include those patients. Hospitals that are just starting to administer the survey in 2018 should include NICU discharges in

their sample per the sampling framework detailed in the manual.

Regarding the CAHPS Child Hospital Survey, some commenters expressed the cost of administering the survey and low

response rates.

Some hospitals have asked about the use of alternative, lower cost modes of survey administration, such as administering

paper surveys as discharge that can then be batched and mailed to a vendor to calculate results. This approach is

potentially an opportunity both lower the cost of administration and increase response rates.

Leapfrog’s Pediatric Expert Panel, has noted that while administering the CAHPS Child Hospital Survey using paper forms at

discharge on the list of AHRQ-approved modes, hospitals that are trying to find ways to administer the survey and increase

response rate should be able to submit results to the 2018 Leapfrog Hospital Survey. That said, the Pediatric Expert Panel

has expressed a desire for these different modes to be tested, and so we cannot guarantee that you will be able to submit

these results for future Leapfrog Hospital Surveys.

Regarding Pediatric CT Radiation Dose, several commenters expressed concern that Leapfrog’s scoring algorithm for the

Pediatric CT Radiation Dose Measures, which gives hospitals credit for having a median average DLP value at or below

the 50th percentile, would drive down doses to ranges that would results in less than optimal scan quality.

The measure developer has found in a large randomized trial that routine review and sharing of dosage by hospitals leads

to dose reductions without compromise to clinical efficacy. However, Leapfrog has decided to continue requiring hospitals

to report on their average 25% DLP and we will carefully monitor the prevalence of low doses. We will work with our

national expert panel to make changes to the scoring algorithm as appropriate in the future.

18

APPENDIX II: CPOE SCORING ALGORITHM FOR ADULT/GENERAL HOSPITALS

Score on Adult Inpatient Test via the CPOE Evaluation Tool

Implementation Status

(from Leapfrog Hospital Survey

Questions #3-4)

Full Demonstration

of National Safety Standard

for Decision Support

(60% or greater of test orders

correct)

Substantial Demonstration

of National Safety Standard

for Decision Support

(50-59% of test orders correct)

Some Demonstration

of National Safety Standard

for Decision Support

(40-49% of test orders correct)

Completed

The Evaluation (Less than 40% of test

orders correct)

Insufficient Evaluation (Hospital was not able to test at least 50% of test orders)

Incomplete Evaluation (Failed deception analysis or timed out) -or- Did not complete an evaluation

85% or greater of all inpatient

medication orders entered through

CPOE System

Fully Meets the Standard

Substantial Progress

(Previously “Fully Meets the

Standard”)

Substantial Progress

Some Progress

(Previously “Substantial Progress”)

Unable to Calculate

Score

Willing to Report

75-84% of all inpatient

medication orders entered through

CPOE System

Fully Meets Standard

(Previously “Substantial Progress”)

Substantial Progress

Some Progress (Previously

“Substantial Progress”)

Some Progress

Unable to Calculate

Score

Willing to Report

50-74% of all inpatient

medication orders entered through

CPOE System

Substantial Progress

Substantial Progress

(Previously “Some

Progress”)

Some Progress

Willing to Report

(Previously “Some

Progress”)

Unable to Calculate

Score

Willing to Report

CPOE implemented in at least one

inpatient unit but <50% of all inpatient

medication orders entered through

CPOE System

Substantial Progress

(Previously “Some Progress”)

Some Progress

Some Progress (Previously “Willing to Report”)

Willing to Report

Unable to Calculate

Score

Willing to Report

CPOE not implemented in at least one inpatient

unit

Cannot take CPOE Evaluation Tool; hospital will be scored as “Willing to Report”

Declined to Respond: The hospital did not respond to this section of the survey or did not complete the survey.

Order Checking Categories Included in the CPOE Evaluation Tool (v3.5)

19

Order Checking Category Description Example Therapeutic Duplication Medication combinations overlap therapeutically

(same agent or same class) Using clonazepam and lorazepam together

Drug-Dose (Single) Specified dose of medication exceeds safe range for single dose

Tenfold overdose of digoxin

Drug-Dose (Daily) Specified frequency of administration results in daily dose that exceeds safe range for daily dose

Ordering ibuprofen regular dose every three hours

Drug-Allergy

Medication (or medication class) is one for which patient allergy has been documented

Penicillin prescribed for patient with documented penicillin allergy

Drug-Route Specified route of administration is inappropriate and potentially harmful

Use of vitamin K intramuscular injection

Drug-Drug

Medications in pair of orders result in known harmful interaction when used in combination

Concurrent linezolid and sumatriptan

Drug-Diagnosis

Medication contraindicated based on documented problem/diagnosis

Nonspecific beta-blocker in patient with asthma

Drug-Age

Medication contraindicated based on patient age Prescribing diazepam for a patient over 65 years old

Drug-Lab

Medication contraindicated based on documented laboratory test results (includes renal status)

Use of enalapril in patient with severe renal failure

Drug Monitoring Medication for which the standard of care includes subsequent monitoring to avoid harm

Prompt to order drug levels when ordering aminoglycoside.

The Tool also includes an “Alert Fatigue” test category, which checks if prescribers are receiving alerts or information for inconsequential medication interactions that clinicians typically ignore. An example would be alerting on the concurrent use of hydrochlorothiazide and captopril. This test category is not included in scoring. The Tool also includes a “Deception Analysis” test category, which checks for “false positives” (e.g., orders that should not have generated any warning in the hospital’s CPOE system). Hospital’s that “fail” the Deception Analysis are scored as “incomplete evaluation” and will not be able to re-take an Adult Inpatient Test for 120 days.

20

APPENDIX III: ICU PHYSICIAN STAFFING SCORING ALGORITHM

IPS Score

(Performance

Category)

Meaning that:

Fully Meets the

Standard

(four-filled bars)

The hospital responded “Yes” or “Not applicable, intensivists present 24/7” to all of the following questions:

Question #3: All critical care patients in adult and pediatric general medical and/or surgical ICU(s) and neuro ICUs are managed or co-managed by one or more physicians who are certified in critical care medicine (i.e. “intensivists”) when these physicians are present (on-site or via telemedicine)

Question #4 or #5: o One or more intensivist(s) is/are present via telemedicine 24 hours per day, 7

days per week, with some on-site intensivist time; o One or more intensivist(s) is/are present in each ICU during daytime hours for

at least 8 hours per day, 7 days per week, providing care exclusively in each ICU during these hours

Question #6: When intensivists are not present (on-site or via telemedicine) in these ICUs, one of them returns more than 95% of calls/pages/texts from these units within five minutes

Question #7: When intensivists are not present (on-site or via telemedicine) in the ICU or not able to physically reach an ICU patient within 5 minutes, another physician, physician assistant, nurse practitioner or FCCS-certified nurse “effector” is on-site at the hospital and able to reach ICU patients within five minutes in more than 95% of the cases

Note: When telemedicine is employed as a substitute for on-site intensivist coverage, it must meet all ten requirements detailed in endnotes in the hard copy of the Survey, which includes some on-site intensivist time to manage the ICU patients’ admission, discharge, and care planning.

Substantial

Progress

(three-filled bars)

The hospital responded “Yes” to all of the following questions:

Question #3: All critical care patients in adult and pediatric medical and/or surgical ICU(s) and neuro ICUs are managed or co-managed by one or more physicians who are certified in critical care medicine (i.e. “intensivists”), when these physicians are present (on-site or via telemedicine)

Question #8 or #12:

One or more intensivist(s) is/are present in each ICU during daytime hours for at least 8 hours per day, 4 days per week or 4 hours per day, 7 days per week;

Clinical pharmacists make daily rounds on all critical care patients in adult and pediatric medical and/or surgical and neuro ICUs 7 days per week

Question #13 or #14:

21

An intensivist leads daily, multi-disciplinary team rounds on-site on all critical care patients in adult and pediatric medical and/or surgical and neuro ICUs 7 days per week;

When intensivists are on-site in adult and pediatric medical and/or surgical and neuro ICUs, they make all admission and discharge decisions

Substantial

Progress

(alternative for

hospitals)

The hospital responded “Yes” to all of the following questions:

Question #3: All critical care patients in adult and pediatric medical and/or surgical ICU(s) and neuro ICUs are managed or co-managed by one or more physicians who are certified in critical care medicine (i.e. “intensivists”), when these physicians are present (on-site or via telemedicine)

Question #9: One or more intensivist(s) is/are present via telemedicine 24 hours per day, 7 days per week, with on-site care planning done by an intensivist, hospitalist, anesthesiologist, or a physician trained in emergency medicine

Note: When telemedicine is employed as a substitute for on-site intensivist coverage, it must meet all nine requirements detailed in endnote in the hard copy of the Survey.

Some Progress

(two-filled bars)

The hospital responded “Yes” to all of the following questions:

Question #3: All critical care patients in adult and pediatric medical and/or surgical ICU(s) and neuro ICUs are managed or co-managed by one or more physicians who are certified in critical care medicine (i.e. “intensivists”), when these physicians are present (on-site or via telemedicine)

Question #10: One or more intensivist(s) is/are present on-site at least 4 days per week to establish or revise daily care plans for all critical care patients

Question #13 or #14:

An intensivist leads daily, multi-disciplinary team rounds on-site on all critical care patients in adult and pediatric medical and/or surgical and neuro ICUs 7 days per week;

When intensivists are on-site in adult and pediatric medical and/or surgical and neuro ICUs, they make all admission and discharge decisions

Or the hospital responded “Yes” to all of the following questions:

Question #11: If not all, at least some critical care patients are managed or co-managed by physicians who are certified in critical care medicine (i.e. “intensivists”), either on-site or via telemedicine

Question #13 or #14:

An intensivist leads daily, multi-disciplinary team rounds on-site on all critical care patients in adult and pediatric medical and/or surgical and neuro ICUs 7 days per week;

When intensivists are on-site in adult and pediatric medical and/or surgical and neuro ICUs, they make all admission and discharge decisions

22

Note: When telemedicine is employed as a substitute for on-site intensivist coverage, it must meet all nine requirements detailed in endnote in the hard copy of the survey.

Willing to Report

(one-filled bar)

The hospital responded to all the Leapfrog survey questions, but it does not yet meet the

criteria for Some Progress.

Does Not Apply The hospital does not operate an adult or pediatric general medical or surgical intensive care

unit or a neuro intensive care unit.

Declined to

Respond The hospital did not respond to this section of the survey, or has not submitted a survey.

23

APPENDIX IV: BAR CODE MEDICATION ADMINISTRATION (BCMA) SCORING ALGORITHM

BCMA Score (Performance Category)

% Units % Compliance Decision Support

Processes & Structures to

Prevent Workarounds

Fully Meets the Standard (four-filled bars)

100% 95% 7 out of 7 6 out of 8

Substantial Progress (three-filled bars)

The hospital meets 3 of the 4 standards

Some Progress (two-filled bars)

The hospital meets 2 of the 4 standards

Willing to Report (one-filled bar)

The hospital meets 1 or 0 of the 4 standards

Declined to Respond The hospital did not respond to the questions in this section of the survey or did not

submit a survey.

Does Not Apply The hospital does not operate an ICU, medical/surgical unit, or labor and delivery

unit.