PEER REVIEW HISTORY

BMJ Open publishes all reviews undertaken for accepted manuscripts. Reviewers are asked to

complete a checklist review form (see an example) and are provided with free text boxes to elaborate

on their assessment. These free text comments are reproduced below. Some articles will have been

accepted based in part or entirely on reviews undertaken for other BMJ Group journals. These will be

reproduced where possible.

ARTICLE DETAILS

TITLE (PROVISIONAL) Challenges in implementing government-directed VTE guidance for

medical patients; a mixed methods study

AUTHORS Basey, Avril ; Mackridge, Adam; Kennedy, Tom; Krska, Janet

VERSION 1 - REVIEW

REVIEWER Elliott R. Haut, MD, FACS Associate Professor of Surgery, Anesthesiology / Critical Care Medicine (ACCM) and Emergency Medicine, The Johns Hopkins University School of Medicine Core faculty, The Armstrong Institute for Patient Safety and Quality, Johns Hopkins Medicine Baltimore, Maryland, USA Dr. Haut is the Primary Investigator of a Mentored Clinician Scientist Development Award K08 1K08HS017952-01 from the AHRQ entitled “Does Screening Variability Make DVT an Unreliable Quality Measure of Trauma Care?” Dr. Haut receives royalties from Lippincott, Williams, & Wilkins for a book - "Avoiding Common ICU Errors" and has given expert witness testimony in various medical malpractice cases.

REVIEW RETURNED 03-Aug-2012

THE STUDY Can you please give more information about the local initiatives at your hospital? Did you do anything to help translate, clarify, promulgate or educate about the national guidelines locally? Can you give more information about your paper forms and electronic tool? There are two references I think would add to your paper 1- A paper I recently co-authored in BMJ. This paper has similar features to yours, but has come to slightly different conclusions. Perhaps you could compare/contrast to your findings and also bring a more international perspective. Streiff MB, Carolan HT, Hobson DB, Kraus PS, Holzmueller CG, Demski R, Lau BD, Biscup-Horn P, Pronovost PJ, Haut ER. Lessons from the Johns Hopkins Multi-Disciplinary Venous Thromboembolism (VTE) Prevention Collaborative. British Medical Journal. 2012;344:e3935. 2- Another systematic review that shows passive attempts to improve VTE prophylaxis does not work well- similar to your findings. Tooher R, Middleton P, Pham C, Fitridge R, Rowe S, Babidge W, Maddern G. A systematic review of strategies to improve prophylaxis for venous thromboembolism in hospitals. Annals of Surgery 2005;241(3):397-415.

RESULTS & CONCLUSIONS Please add the national vs. local designation next to each line of text in figure 1? This would help- especially for those readers in other countries not familiar with the British national initiatives.

on Novem

ber 9, 2020 by guest. Protected by copyright.

http://bmjopen.bm

j.com/

BM

J Open: first published as 10.1136/bm

jopen-2012-001668 on 6 Novem

ber 2012. Dow

nloaded from

Please add percentage for each “main cause of admission” in table 1? Can you run stats tests (as appropriate) to compare the sex, age, length of stay, and admitting diagnosis for the columns in table 1? This will help us see whether the observed patients truly representative of the “case note review” group. Some of the data presented in table 2 are somewhat confusing. 1- Can it be split into 2 smaller tables (top and bottom parts)? 2- please clarify the exact denominator for each row of the top part of the table. In addition, I have concerns about the numbers/percentage in the bottom half of the table as well. 3- Can you please explain how 221 of 224 patients had VTE risk assessment performed based on the all admissions data (3 not done), but that 15/20 of the observations had VTE risk assessment done (5 not done). How can there be 5 missing from the observations and FEWER missing from the full data of the entire population. 4- I am also confused by the percentages that go with the LMWH prescribed appropriately and inappropriately. Can you give the full number ordered LMWH and then the % that is appropriate vs. inappropriate? I think this should add up to 100% but the way the numbers are in the table they are not adding up to me. 5- can you add p-values for the comparisons performed (as mentioned in the text)? You claim that the local initiatives did not work but he national ones did. Is it possible that the local initiatives took time to show benefit and there is a lag time? Maybe the national ones did nothing and you are just seeing a delayed benefit of the local ones? Also you need to explain what (if anything) you did to make sure the local providers knew about the national top-down approach. You state that relatively few staff knew of the national policies, yet you think this s what made the difference. Did any interview questions ask whether there was a direct link between providers ordering practices and either local or national programs?

REVIEWER Cohen, Alexander Kings College Hospital, Surgery

REVIEW RETURNED 18-Aug-2012

REPORTING & ETHICS Strobe guidelines for observational studies not provided

GENERAL COMMENTS This is an interesting observational study and although I have a strong sense that the conclusions are correct, the study design shows that there were a minimum of three “initiatives” during each intervening time period and hence it is difficult to attribute the success or failure to any particular intervention. I suspect that the interventions impacted on each other and it seems clearer that some collectively failed during a time period, but success is harder to attribute. Major points: 1. It would be important to review the 2 forms for: The CRF for data collection The structured interview We need to know who was interviewed, occupation, job title or level of training etc

on Novem

ber 9, 2020 by guest. Protected by copyright.

http://bmjopen.bm

j.com/

BM

J Open: first published as 10.1136/bm

jopen-2012-001668 on 6 Novem

ber 2012. Dow

nloaded from

2. The authors describe the collection of data on LMWH, did they collect data on unfractionated heparin (UFH) which would have been indicated for some patients with severe renal impairment? Was data on other anticoagulants prophylaxis and mechanical methods collected? 3. More information is needed on the power calculation 4. Table 1. What is meant by “abnormal biochemistry” is this mainly hyperglycaemia or other causes? This term is too vague for a medical journal and needs to be explained 5. Weaknesses, the sample size is limited with around 250 patients reviewed in each time period. This means the confidence intervals (CI) around the estimates of frequencies would be sizeable. The CI’s should be given for the frequencies. The sample size for “Observations” is even smaller and the same with respect to CI’s applies. 6. Table 2: The baseline data from November 2009 seems to have lower rates of VTE and bleeding risk factors, this needs further comment and explanation 7. Table 2: The terminology “appropriately” needs to be defined clearly 8. p9.12-14 The statistical analyses describing the changes do not clearly define what was the analysis. Were these individual comparisons or a chi-square test for trend? 9. p11. Conclusion: This needs tempering: the phrase “strong clinical leadership” should probably be reconsidered, especially as the changes occurred after government intervention with financial penalties/incentives. Perhaps “a consultant led approach” as used before is better. In observational studies “can result in” is unfounded, this should be replaced with “was associated with” Minor points: p10.5-7 and The lack of correlation with training and assessment is interesting, as is the lack of knowledge of risk and guidelines Some of the references need more meat e.g. Ref 20, 24, 31, 32. Web addresses and book details should be provided

VERSION 1 – AUTHOR RESPONSE

This is an interesting observational study and although I have a strong sense that the conclusions are

correct, the study design shows that there were a minimum of three “initiatives” during each

intervening time period and hence it is difficult to attribute the success or failure to any particular

intervention. I suspect that the interventions impacted on each other and it seems clearer that some

collectively failed during a time period, but success is harder to attribute.

Major points:

1. It would be important to review the 2 forms for:

The CRF for data collection

The structured interview

We need to know who was interviewed, occupation, job title or level of training etc

2. The authors describe the collection of data on LMWH, did they collect data on unfractionated

heparin (UFH) which would have been indicated for some patients with severe renal impairment?

Was data on other anticoagulants prophylaxis and mechanical methods collected?

3. More information is needed on the power calculation

4. Table 1. What is meant by “abnormal biochemistry” is this mainly hyperglycaemia or other causes?

This term is too vague for a medical journal and needs to be explained

on Novem

ber 9, 2020 by guest. Protected by copyright.

http://bmjopen.bm

j.com/

BM

J Open: first published as 10.1136/bm

jopen-2012-001668 on 6 Novem

ber 2012. Dow

nloaded from

5. Weaknesses, the sample size is limited with around 250 patients reviewed in each time period.

This means the confidence intervals (CI) around the estimates of frequencies would be sizeable. The

CI’s should be given for the frequencies. The sample size for “Observations” is even smaller and the

same with respect to CI’s applies.

6. Table 2: The baseline data from November 2009 seems to have lower rates of VTE and bleeding

risk factors, this needs further comment and explanation

7. Table 2: The terminology “appropriately” needs to be defined clearly

8. p9.12-14 The statistical analyses describing the changes do not clearly define what was the

analysis. Were these individual comparisons or a chi-square test for trend?

9. p11. Conclusion: This needs tempering: the phrase “strong clinical leadership” should probably be

reconsidered, especially as the changes occurred after government intervention with financial

penalties/incentives. Perhaps “a consultant led approach” as used before is better. In observational

studies “can result in” is unfounded, this should be replaced with “was associated with”

Minor points:

p10.5-7 and The lack of correlation with training and assessment is interesting, as is the lack of

knowledge of risk and guidelines

Some of the references need more meat e.g. Ref 20, 24, 31, 32. Web addresses and book details

should be provided

VERSION 2 – REVIEW

REVIEWER Elliott R. Haut, MD, FACS Associate Professor of Surgery, Anesthesiology / Critical Care Medicine (ACCM) and Emergency Medicine, The Johns Hopkins University School of Medicine Core faculty, The Armstrong Institute for Patient Safety and Quality, Johns Hopkins Medicine Baltimore, Maryland, USA Dr. Haut is the Primary Investigator of a Mentored Clinician Scientist Development Award K08 1K08HS017952-01 from the AHRQ entitled “Does Screening Variability Make DVT an Unreliable Quality Measure of Trauma Care?” Dr. Haut receives royalties from Lippincott, Williams, & Wilkins for a book - "Avoiding Common ICU Errors" and has given expert witness testimony in various medical malpractice cases.

REVIEW RETURNED 21-Sep-2012

THE STUDY overall much clearer data presentation

RESULTS & CONCLUSIONS I am still concerned that you make the assumption that the national initiative is what drove htis imporvement. You only give very little credit to the local initiatives. I would suggest that the nation made people do the local things. The local really made the difference. Unless you can definifly say why this is not the case, you should at least expund upon this possibility. Unfortunately, you cannot say from these data what would have happened with ther national impetus alone.

REVIEWER Alexander (Ander) T Cohen MBBS MSc MD FRACP Honorary Consultant Vascular Physician Vascular Medicine King's College Hospital London SE5 9RS UK

REVIEW RETURNED 24-Sep-2012

on Novem

ber 9, 2020 by guest. Protected by copyright.

http://bmjopen.bm

j.com/

BM

J Open: first published as 10.1136/bm

jopen-2012-001668 on 6 Novem

ber 2012. Dow

nloaded from

GENERAL COMMENTS This manuscript is much improved, however I still have some concerns and comments 1. As there were a minimum of three “initiatives” during each intervening time period it is difficult to attribute the success or failure to any particular intervention, this should be added to the limitations of attributing the "likely" success to financial incentives. 2. There now seems to be 2 power calculations one with 99% power and one with 96% power 3. The "main causes for admission" do not seem typical for medical wards. In the UK and worldwide, most patients in medical wards with risk of VTE have heart failure, respiratory failure and infections and these make up over 80% of such cases (see ENDORSE study, Lancet 2008). Although this may not affect the conclusions, it does not add up and should be mentioned as a limitation. 4. The CRF shows that data on hypersensitivity to heparin and history of HIT (a thrombotic disorder) were collected under the section on "Bleeding Risk Identified". They should not have been counted or analysed as bleeding risks. Also this section did not define "severe renal disease"

VERSION 2 – AUTHOR RESPONSE

Reviewer: Elliott R. Haut, MD, FACS

Associate Professor of Surgery, Anesthesiology / Critical Care Medicine (ACCM) and Emergency

Medicine, The Johns Hopkins University School of Medicine Core faculty, The Armstrong Institute for

Patient Safety and Quality, Johns Hopkins Medicine Baltimore, Maryland, USA

Dr. Haut is the Primary Investigator of a Mentored Clinician Scientist Development Award K08

1K08HS017952-01 from the AHRQ entitled “Does Screening Variability Make DVT an Unreliable

Quality Measure of Trauma Care?” Dr. Haut receives royalties from Lippincott, Williams, & Wilkins for

a book - "Avoiding Common ICU Errors" and has given expert witness testimony in various medical

malpractice cases.

overall much clearer data presentation

I am still concerned that you make the assumption that the national initiative is what drove this

improvement. You only give very little credit to the local initiatives. I would suggest that the nation

made people do the local things. The local really made the difference. Unless you can definitely say

why this is not the case, you should at least expund upon this possibility. Unfortunately, you cannot

say from these data what would have happened with the national impetus alone.

Author response

Added to discussion page 11

Reviewer: Alexander (Ander) T Cohen MBBS MSc MD FRACP Honorary Consultant Vascular

Physician Vascular Medicine King's College Hospital London SE5 9RS

This manuscript is much improved, however I still have some concerns and comments

1. As there were a minimum of three “initiatives” during each intervening time period it is difficult to

attribute the success or failure to any particular intervention, this should be added to the limitations of

attributing the "likely" success to financial incentives.

on Novem

ber 9, 2020 by guest. Protected by copyright.

http://bmjopen.bm

j.com/

BM

J Open: first published as 10.1136/bm

jopen-2012-001668 on 6 Novem

ber 2012. Dow

nloaded from

Author response

Added to discussion page 11

2. There now seems to be 2 power calculations one with 99% power and one with 96% power

Author response

That is correct one power calculation to detect the change in the proportion of patients risk assessed

and a second power calculation to detect the change in the number of patients appropriately

prescribed prophylaxis

3. The "main causes for admission" do not seem typical for medical wards. In the UK and worldwide,

most patients in medical wards with risk of VTE have heart failure, respiratory failure and infections

and these make up over 80% of such cases (see ENDORSE study, Lancet 2008). Although this may

not affect the conclusions, it does not add up and should be mentioned as a limitation.

Author response

This is due to patients with acute cardiac conditions being admitted to the Heart Emergency Centre

rather than the Acute Medical Unit and therefore not included in the study – added to discussion page

11

4. The CRF shows that data on hypersensitivity to heparin and history of HIT (a thrombotic disorder)

were collected under the section on "Bleeding Risk Identified". They should not have been counted or

analysed as bleeding risks. Also this section did not define "severe renal disease"

Author response

None of the patients in the study had a history of HIT; one patient had hypersensitivity to heparin but

this has not been included in the data relating to bleeding risks

Severe renal disease was defined as eGFR <30ml/ml/min added to method page 6

on Novem

ber 9, 2020 by guest. Protected by copyright.

http://bmjopen.bm

j.com/

BM

J Open: first published as 10.1136/bm

jopen-2012-001668 on 6 Novem

ber 2012. Dow

nloaded from