Evaluating the Impact of an Educational Interventionon Documentation of Decision-making Capacity inan Emergency Medical Services System

Jennifer Riley, MD, Rob Burgess, EMT-P, Brian Schwartz, MD

AbstractObjectives: To compare the documentation of decision-making capacity by advanced life support (ALS) providersand signature acquisition before, one month after, and oneyear after an educational intervention. Methods: The in-tervention comprised a one-and-a-half-hour module onassessment and documentation of decision-making capacity.Ambulance call reports were reviewed for all ALS callsoccurring during three two-month periods, and refusals oftransport were recorded. Provider compliance with docu-mentation of decision-making capacity and signature ac-quisition were determined from a convenience sample of 75reports from each period. Reviewers were blinded to studyperiod. Twenty-percent double data entry was undertakento evaluate accuracy. Ninety-five percent confidence inter-vals were calculated to compare frequencies of cancelledcalls and documentation. Results: From the emergencymedical services database, 7,744 calls before the interven-

tion, 7,444 immediately after, and 7,604 one year later wereidentified. Documentation rates in the second and thirdperiods did not differ from that prior to the intervention(1.3% vs. 0.0% and 0.0% in subsequent periods), nor did therates of signature acquisition differ (85.3% vs. 85.3% and78.6%). The accuracy of data entry was 92.6%. However, thefrequency of call refusals decreased significantly after theintervention (from 9.0% to 2.0% and 6.6% in the respectiveperiods). Conclusions: An educational intervention resultedin no change in the rate of decision-making capacitydocumentation or signature acquisition by ALS providersfor refusal of transport. There was a temporary increase inthe number of transported patients. Key words: refusal oftransport; out-of-hospital; decision-making capacity; refusalof care; emergency medical services; paramedics. ACA-DEMIC EMERGENCY MEDICINE 2004; 11:790–793.

Patients who refuse transport are a challenge foremergency medical services (EMS) providers andphysicians. Admission rates to the hospital followingpatient-initiated refusal of EMS transport range from6% to 13%.1–3 Although in some cases it may be safe forpatients to refuse transport,4 the legal risk to providersin inappropriate instances is well documented.5,6 Priorto allowing a patient to refuse transport, it must bedetermined whether the patient is capable of makingthis decision. Decision-making capacity requires thatthe patient understands 1) the treatment and the al-

ternatives to treatment (and transport), 2) its risks andbenefits, and 3) the likely and possible consequencesof the decision.

Current practice in our jurisdiction requires that theprovider explain the consequences of refusal andacquire a signature on the ‘‘refusal of service’’ sectionof the ambulance call report (ACR). However, thisdoes not provide enough information to determinewhether the patient had the decision-making capacityto provide an informed refusal of transport or that theprovider’s assessment was valid.

This study sought to determine whether an educa-tional module improved paramedic practice in assess-ing and documenting patient decision-makingcapacity and acquiring the patient’s signature.

METHODS

Study Design. This was a prospective observationalstudy of the impact of an educational intervention onsubsequent assessment of decision-making capacityof patients by out-of-hospital providers. The institu-tional research ethics board approved this study.

Study Setting and Population. We reviewed datafrom ACRs completed by advanced life support (ALS)providers in our EMS system. Our urban EMS system

From the Division of Emergency Medicine, Department of Medi-cine, University of Toronto, and Department of Emergency Servicesat St. Michael’s Hospital and the Hospital for Sick Children,Toronto, Ontario, Canada (JR); the Department of EmergencyServices, Division of Prehospital Care, Sunnybrook and Women’sCollege Health Sciences Centre, Toronto, Ontario, Canada (RB); andthe Department of Family and Community Medicine, Division ofEmergency Medicine, University of Toronto, Toronto, Ontario,Canada (BS).Received March 10, 2003; revisions received August 11, 2003, andDecember 31, 2003; accepted January 7, 2004.Presented at the SAEM annual meeting, San Francisco, CA, May2000; and the Canadian Association of Emergency Physiciansannual meeting, Saint John, New Brunswick, Canada, June 2000.Address for correspondence and reprints: Jennifer Riley, MD, St.Michael’s Hospital, 30 Bond Street, Toronto, Ontario M5B 1W8,Canada. Fax: 416-864-5341; e-mail: jennifer.riley@utoronto.ca.doi:10.1197/j.aem.2004.01.005

790 Riley et al. d DECISION-MAKING CAPACITY DOCUMENTATION

is a tiered-response system with basic and ALSparamedics and serves a population of 2.5 millionresidents with an approximate annual call volume of140,000 patient transports (50,000ALS). AnALS crew iscomprised of one emergency medical technician–para-medic (EMT-P) and an emergency medical technician–intermediate (EMT-I) partner.

Study Protocol. Over a four-week period in thespring of 1997, 200 EMT-Ps and EMT-Is attendeda one-and-a-half-hour educational module on assess-ing a patient’s decision-making capacity for refusingtransport. This consisted of a series of case-based,small-group discussions led by the base hospitalmedical director. Topics included the importance ofassessing a patient’s decision-making capacity, thethree essential components of this assessment, anddocumentation requirements. Providers learned thatto allow a refusal, they must demonstrate and docu-ment that patients refusing transport display anunderstanding of each of these three components.Emphasis was also placed on policy requirementsregarding the acquisition of a signature in the refusalsection of the ACR and its inadequacy as a surrogateto the documentation of a patient’s decision-makingcapacity assessment.Three two-month periods were examined to evalu-

ate the effectiveness of the educational module. Thefirst period (January 1, 1997–February 28, 1997) oc-curred prior to the educational module and served asthe reference period. The second (June 1, 1997–July 31,1997) and third (June 1, 1998–July 31, 1998) periodsoccurred one month and one year following comple-tion of the educational module. For period 1, a totalof 698 patient refusals of transport calls occurred outof 7,744 total calls, 150 of 7,444 for period 2, and 504 of7,604 for period 3.

Measures. Compliance with documentation of a pa-tient’s decision-making capacity and signature acqui-sition on the ACR was determined by review of aconvenience sample of 75 patient refusal calls fromeach of periods 1, 2, and 3, respectively (total of 225out of 1,352 refusal-of-transport calls). This was ob-tained by extracting every fourth consecutive ACRfrom those in which the patient refused transport.ACRs were included if they were completed by an

ALS crew (comprised of an EMT-P and either anEMT-P or an EMT-I), and the patient was assessedprior to refusal. Cardiac arrests were excluded. Totalcall volumes and frequency of refusals of transportwere calculated for each period of interest. Retrieveddata included patient characteristics, ambulance callinformation, and chief complaint. Each ACR wasreviewed for a signature in the refusal section anddocumentation of decision-making capacity assess-ment (the three essential components discussed in theeducational module). All charts were reviewed, anddata were entered by one of the study authors (RB),who was blinded to study period. To evaluate accu-racy, 20% of the original sample (15 ACRs per period)was reentered by the principal author, blinded tooriginal data entry and study period.

Data Analysis. The frequency of patient-initiatedtransport refusals was also calculated for each period.Comparisons among the three periods were conductedusing a chi-square test. All statistical analyses wereperformed using STATA 7.0 software (STATA Corpo-ration, College Station, TX).

RESULTS

The most common chief complaints for patients whorefused transport in each of the three periods werediabetes, chest pain, and shortness of breath.

Capacity Documentation. Review of 75 ACRs fromeach period showed no difference in frequency ofdocumentation of a patient’s decision-making capacityin the three periods (Table 1). Similarly, no increase inthe rate of signature acquisition on the refusal section ofthe ACRwas observed in the postintervention periods.

Frequency of Patient Refusals of Transport. Thefrequency of patient refusals of transport in each ofthe three periods is shown in Table 2. One month andone year after the intervention, there were fewerpatient refusals of transport and an increase in totalpatient transports.

Reviewer Accuracy. Of 270 data points reentered bythe second reviewer, 20 were at variance with theoriginal, resulting in an accuracy rate of 92.6%.

TABLE 1. Capacity Documentation and Signature Acquisition According to Time Period

CapacityDocumentedn (%; 95% CI)

% Difference(CI)

SignatureAcquired

n (%; 95% CI)% Difference

(CI)

Preeducation (n = 75) 1 (1.3%; 0.03, 7.2) 64 (85.3%; 75.2, 92.4)One month posteducation (n = 75) 0 (0.0%; 0.0, 4.8)* 1.3% (�1.3, 3.9) 64 (85.3%; 75.2, 92.4) 0.0% (�1.1, 1.1)One year posteducation (n = 75) 0 (0.0%; 0.0, 4.8)* 1.3% (�1.3, 3.9) 59 (78.6%; 67.7, 87.2) 6.7% (�5.6, 18.9)

Note: Reference category for ‘‘% Difference’’ is the preeducation period.*One-sided 97.5% CI.

ACAD EMERG MED d July 2004, Vol. 11, No. 7 d www.aemj.org 791

DISCUSSION

The educational intervention did not increase docu-mentation of decision-making capacity between thethree periods. The almost complete absence of anydocumentation in our study is disturbing. Clearly, oureducational intervention was ineffective in changingparamedic assessment of decision-making capacity.However, paramedic behavior may have changed inthat fewer patients refused transport. Although theeducational module did not achieve the intendedresult, other learning may have occurred, and it wasunclear whether this was related to our intervention.

The results were discussed with the participants.Most felt uncomfortable with making a decision abouta patient’s decision-making capacity and completionof appropriate documentation. They perceived a lackof support for their decision and were concernedabout liability risk should they accept the patient’srefusal and therefore preferred to convince the patientto go to the hospital. This may be the reason for theobserved decrease in the rate of patient refusals oftransport. In some cases, providers may have con-ducted formal assessments with increased frequency,identifying more patients who were incapable ofrefusing and therefore transporting them despite theirrefusal of care. Patients also may have withdrawntheir refusal after becoming aware of the risks ofnontransport. For refusals that were accepted, pro-viders were unwilling to commit an assessment inwriting and were more likely to complete the refusalsection by acquiring a signature. This may be relatedto a policy requirement for signature acquisition.

Based on these observations, we believe that im-provement in documentation might be achieved byincluding a decision-making capacity assessmentchecklist in the refusal section to be completed priorto acquiring the signature. Evaluation of this or othertools could assist in further evaluating and addressingthe lack of documentation of decision-making capacity.

Patient refusal of transport to hospital is a recog-nized area of concern for out-of-hospital care provid-ers. One survey of American EMS systems found that91% of respondents had formal refusal-of-transport

policies. Of these, 83% required an assessment ofdecision-making capacity, most of which consisted of atest of patient orientation (97%). Comprehension ofillness and treatment risks and benefits was requiredin 59% and 48% of patient refusal policies, respec-tively, whereas signature acquisition was required in98%.7 Using patient orientation as a surrogate fordecision-making capacity assessment poses a risk. Itdoes not address patient understanding and compre-hension.8,9 Signature acquisition on an against-medi-cal-advice form is an equally inadequate assessmentof decision-making capacity because it does not reflectdecision-making ability.7,10

Out-of-hospital training programs need to addressmanagement of patient refusals of transport, a recog-nized malpractice risk for EMS systems.6 Providersmust be aware of policy requirements of patient refusalin their jurisdictions,11 and the need for accurate recordkeeping. One study observed adequate documentationin only 65.2% of cases and identified risks of treatmentas the most commonly missed component.12 Anotherfound that documentation was incomplete in 43% ofcases.1Although the authors found that documentationimproved with direct medical control, only 40% ofthe medical control physicians documented patientdecision-making capacity.

Patient understanding and recollection of informa-tion pose other challenges. Patients have been ob-served to recall risks of refusing care and dischargeinformation at rates of 22% and 55%, respectively.13

Therefore, explanations must be thorough, offered insimple language, and documented.14

LIMITATIONS

The sample of ACRs chosen for review was a conve-nience sample; therefore, sampling bias cannot beruled out. In addition, we have no follow-up infor-mation about nontransported patients or patients whoinitially refused treatment and, after assessment, werethen transported. While the frequency of transportrefusals initially declined and then rose a year afterthe intervention, we did not review subsequent timeperiods to determine whether this ultimately reachedpreintervention levels.

CONCLUSIONS

Documentation of a patient’s decision-making capac-ity and signature acquisition on the refusal section ofthe ACR did not change after an educational moduledesigned to improve assessment and documentationamong patients who refuse transport.

References

1. Cone DC, Kim DT, Davidson SJ. Patient-initiated refusals ofprehospital care: ambulance call report documentation, patient

TABLE 2. Frequency of Cancelled Calls Accordingto Time Period

n (%; 95% CI) % Difference (CI)

Preeducation(n = 7,744) 698 (9.0%; 8.4, 9.7)

One monthposteducation(n = 7,444) 150 (2.0%; 1.7, 2.4) 7.0% (6.3, 7.7)

One yearposteducation(n = 7,604) 504 (6.6%; 6.1, 7.2) 2.4% (1.5, 3.2)

Note: Reference category for ‘‘% Difference’’ is the preedu-cation period.

792 Riley et al. d DECISION-MAKING CAPACITY DOCUMENTATION

outcome, and on-line medical command. Prehosp DisasterMed. 1995; 10:22–8.

2. Sucov A, Verdile VP, Garettson D, Paris PM. The outcome ofpatients refusing prehospital transportation. Prehosp DisasterMed. 1992; 7:365–71.

3. Zachariah BS, Bryan D, Pepe PE, Griffin M. Follow-up andoutcome of patients who decline or are denied transport byEMS. Prehosp Disaster Med. 1992; 7:359–64.

4. Socransky SJ, Pirrallo RG, Rubin JM. Out-of-hospital treatmentof hypoglycemia: refusal of transport and patient outcome.Acad Emerg Med. 1998; 5:1080–5.

5. Goldberg RJ, Zautche JL, Koenigsberg MD, et al. A review ofprehospital care litigation in a large metropolitan EMS system.Ann Emerg Med. 1990; 19:557–61.

6. Soler JM, Montes MF, Egol AB, Nateman HR, Donaldson EA,Greene HH. The ten-year malpractice experience of a largeurban EMS system. Ann Emerg Med. 1985; 14:982–5.

7. Weaver J, Brinsfield KH, Dalphond D. Prehospitalrefusal-of-transport policies: adequate legal protection?Prehosp Emerg Care. 2000; 4:53–6.

8. Ayers RJ. Legal considerations in prehospital care. Emerg MedClin North Am. 1993; 11:853–67.

9. Kaufman DM, Zun L. A quantifiable, brief mental statusexamination for emergency patients. J Emerg Med. 1995; 13:449–56.

10. Griglak MJ, Bucci RL. Medicolegal management of theorganically impaired patient in the emergency department.Ann Emerg Med. 1985; 14:685–9.

11. Holroyd B, Shalit M, Kallsen G, Cuthane D, Knop R.Prehospital patients refusing care. Ann Emerg Med. 1988;17:957–63.

12. Selden BS, Schnitzer PG, Nolan FX. Medicolegal documentationof prehospital triage. Ann Emerg Med. 1990; 19:547–51.

13. Schmidt TA, Mann C, Federiuk CS, Atcheson RR, Fuller D,Christie MJ. Do patients refusing transport rememberdescriptions of risks after initial advanced life supportassessment? Acad Emerg Med. 1998; 5:796–801.

14. Mayer D. Refusal of care and discharging ‘‘difficult’’ patientsfrom the emergency department. Ann Emerg Med. 1990;19:1436–46.

ACAD EMERG MED d July 2004, Vol. 11, No. 7 d www.aemj.org 793