validity of ”antibiotic treatment for suspected pneumonia ... · validity of ”antibiotic...
TRANSCRIPT
![Page 1: Validity of ”antibiotic treatment for suspected pneumonia ... · Validity of ”antibiotic treatment for pneumonia” indicator Professor Harry Campbell, Centre for Global Health](https://reader034.vdocuments.us/reader034/viewer/2022042323/5f0d960b7e708231d43b16cf/html5/thumbnails/1.jpg)
Validity of ”antibiotic treatment for pneumonia” indicator
Professor Harry Campbell, Centre for Global Health Research,
Usher Institute of Population Health Sciences and Informatics, University of Edinburgh
JOGH Collection on Improving Coverage Measurement (ICM), 20 November 2018
![Page 2: Validity of ”antibiotic treatment for suspected pneumonia ... · Validity of ”antibiotic treatment for pneumonia” indicator Professor Harry Campbell, Centre for Global Health](https://reader034.vdocuments.us/reader034/viewer/2022042323/5f0d960b7e708231d43b16cf/html5/thumbnails/2.jpg)
Pneumonia still a major cause of child death840k child pneumonia deaths in 2017:• Wahl B et al. Lancet GH 2018
60% of global child pneumonia deaths occurred outside hospitals:• Nair H at al. Lancet 2015
![Page 3: Validity of ”antibiotic treatment for suspected pneumonia ... · Validity of ”antibiotic treatment for pneumonia” indicator Professor Harry Campbell, Centre for Global Health](https://reader034.vdocuments.us/reader034/viewer/2022042323/5f0d960b7e708231d43b16cf/html5/thumbnails/3.jpg)
The Integrated Global Action Plan for the Prevention and Control of Pneumonia and Diarrhoea
![Page 4: Validity of ”antibiotic treatment for suspected pneumonia ... · Validity of ”antibiotic treatment for pneumonia” indicator Professor Harry Campbell, Centre for Global Health](https://reader034.vdocuments.us/reader034/viewer/2022042323/5f0d960b7e708231d43b16cf/html5/thumbnails/4.jpg)
GAPPD Framework: Protect, Prevent, Treat
![Page 5: Validity of ”antibiotic treatment for suspected pneumonia ... · Validity of ”antibiotic treatment for pneumonia” indicator Professor Harry Campbell, Centre for Global Health](https://reader034.vdocuments.us/reader034/viewer/2022042323/5f0d960b7e708231d43b16cf/html5/thumbnails/5.jpg)
Reporting of population-based coverage data“Coverage of core interventions remains low”
Source: UNICEF's State of the World's Children 2013
![Page 6: Validity of ”antibiotic treatment for suspected pneumonia ... · Validity of ”antibiotic treatment for pneumonia” indicator Professor Harry Campbell, Centre for Global Health](https://reader034.vdocuments.us/reader034/viewer/2022042323/5f0d960b7e708231d43b16cf/html5/thumbnails/6.jpg)
Coverage indicator validation: Study design
Step 1: Observe intervention delivery
Step 2: Wait, based on recall period in DHS/MICS.
Step 3: Conduct household interviews
1) Standard DHS/MICS questions2) Additional or modified questions3) Inclusion of strategies to aid recall
Step 4: Compare, determining validity of respondents’ reports
![Page 7: Validity of ”antibiotic treatment for suspected pneumonia ... · Validity of ”antibiotic treatment for pneumonia” indicator Professor Harry Campbell, Centre for Global Health](https://reader034.vdocuments.us/reader034/viewer/2022042323/5f0d960b7e708231d43b16cf/html5/thumbnails/7.jpg)
![Page 8: Validity of ”antibiotic treatment for suspected pneumonia ... · Validity of ”antibiotic treatment for pneumonia” indicator Professor Harry Campbell, Centre for Global Health](https://reader034.vdocuments.us/reader034/viewer/2022042323/5f0d960b7e708231d43b16cf/html5/thumbnails/8.jpg)
Evaluation of “pneumonia treatment indicator”DESIRED INDICATOR: proportion of young children with pneumonia who receive a recommended antibiotic • Numerator: children with pneumonia who receive an antibiotic• Denominator: children with pneumonia
ACTUAL INDICATOR:• Numerator: children reported by caretaker to have symptoms of respiratory
infection consistent with pneumonia* who report receiving an antibiotic• Denominator: children reported by caretaker to have symptoms of
respiratory infection consistent with pneumonia* in the past 2 weeks
• * [cough and] short, rapid breathing or difficult breathing that is chest-related
![Page 9: Validity of ”antibiotic treatment for suspected pneumonia ... · Validity of ”antibiotic treatment for pneumonia” indicator Professor Harry Campbell, Centre for Global Health](https://reader034.vdocuments.us/reader034/viewer/2022042323/5f0d960b7e708231d43b16cf/html5/thumbnails/9.jpg)
Evaluation of “pneumonia treatment indicator”PROJECT AIMS: to provide information to assess the validity of the indicator as measured by DHS and MICS
Study in Ibadan, Nigeria led by Prof Falade / Dr Ayede, Department of Paediatrics, University of Ibadan• recruited ~300 children with “true pneumonia” and
~300 children with respiratory symptoms (cough / difficulty breathing) but in whom pneumonia was excluded (based on examination by trained doctor)
• conducted follow up survey with current DHS / MICS questions
![Page 10: Validity of ”antibiotic treatment for suspected pneumonia ... · Validity of ”antibiotic treatment for pneumonia” indicator Professor Harry Campbell, Centre for Global Health](https://reader034.vdocuments.us/reader034/viewer/2022042323/5f0d960b7e708231d43b16cf/html5/thumbnails/10.jpg)
Validity of the denominator of the indicator
What is the sensitivity and specificity of the DHS and MICS questionnaire measure of “reported short, rapid breathing or difficult breathing due to a problem in the chest ” for confirmed pneumonia ?
![Page 11: Validity of ”antibiotic treatment for suspected pneumonia ... · Validity of ”antibiotic treatment for pneumonia” indicator Professor Harry Campbell, Centre for Global Health](https://reader034.vdocuments.us/reader034/viewer/2022042323/5f0d960b7e708231d43b16cf/html5/thumbnails/11.jpg)
Validity of the numerator of the indicator
What is the sensitivity and specificity of the DHS and MICS questionnaire measure of reported caregiver recall of antibiotic treatment for antibiotic treatment recorded at the consultation? •does this improve with the use of a pill board use?
![Page 12: Validity of ”antibiotic treatment for suspected pneumonia ... · Validity of ”antibiotic treatment for pneumonia” indicator Professor Harry Campbell, Centre for Global Health](https://reader034.vdocuments.us/reader034/viewer/2022042323/5f0d960b7e708231d43b16cf/html5/thumbnails/12.jpg)
Validity of the denominator* of the indicator* children with fast/difficult breathing due to a problem in the chest past 2 weeks
Sensitivity, specificity and AUC for true pneumonia, by study site
![Page 13: Validity of ”antibiotic treatment for suspected pneumonia ... · Validity of ”antibiotic treatment for pneumonia” indicator Professor Harry Campbell, Centre for Global Health](https://reader034.vdocuments.us/reader034/viewer/2022042323/5f0d960b7e708231d43b16cf/html5/thumbnails/13.jpg)
Bangladesh urban
Nigeria urban
Pakistan urban
Bangladesh rural
0
10
20
30
40
50
60
70
80
90
100
0 10 20 30 40 50 60 70 80 90 100
Sens
itivi
ty (%
)
False positive (%)
Validity of the denominator* for true pneumonia*children with fast/difficult breathing due to a problem in the chest past 2 weeks
The caregiver report is a poor predictor of true pneumonia [AUC ~ 0.6]
The sensitivity / specificity values vary widely so the % of true pneumonia will vary widely across surveys making comparisons difficult
![Page 14: Validity of ”antibiotic treatment for suspected pneumonia ... · Validity of ”antibiotic treatment for pneumonia” indicator Professor Harry Campbell, Centre for Global Health](https://reader034.vdocuments.us/reader034/viewer/2022042323/5f0d960b7e708231d43b16cf/html5/thumbnails/14.jpg)
TRUE PNEUMONIA
REPORTED ARI SYMPTOMS
Present Absent TotalPresentAbsentTotal 170 1700 1870
Validation results applied to a typical household survey10,000 children aged < 5 years
• ~170 cases true pneumonia expected in last 2 weeks (based on 300 cases /1000 child-yrs)• ~1700 cases of cough / difficult breathing
![Page 15: Validity of ”antibiotic treatment for suspected pneumonia ... · Validity of ”antibiotic treatment for pneumonia” indicator Professor Harry Campbell, Centre for Global Health](https://reader034.vdocuments.us/reader034/viewer/2022042323/5f0d960b7e708231d43b16cf/html5/thumbnails/15.jpg)
TRUE PNEUMONIA
REPORTED ARI SYMPTOMS
Present Absent TotalPresent 119 425 544Absent 51 1275 1326Total 170 1700 1870
Validation results applied to a typical household survey10,000 children aged < 5 years
• Apply 70% sensitivity and 75% specificity of DHS / MICS questions for true pneumonia
• 544 children with “ARI symptoms”* of whom 119 (22%) had true pneumonia
• * % of children with reported [cough and] short, rapid breathing or difficult breathing that is chest-related
![Page 16: Validity of ”antibiotic treatment for suspected pneumonia ... · Validity of ”antibiotic treatment for pneumonia” indicator Professor Harry Campbell, Centre for Global Health](https://reader034.vdocuments.us/reader034/viewer/2022042323/5f0d960b7e708231d43b16cf/html5/thumbnails/16.jpg)
TRUE PNEUMONIA
REPORTED ARI SYMPTOMS
Present Absent TotalPresent 119 425 544Absent 51 1275 1326Total 170 1700 1870
• All 119 children with pneumonia were treated and all children that did not have pneumonia were not treated – [all children with pneumonia treated correctly]
• indicator value 119/544 = 22%• No children with pneumonia were treated and 119 children that did not have
pneumonia were treated [no children with pneumonia treated correctly]• indicator value 119/544 = 22%
• This indicator does not give useful information on programmeperformance and 90% is not an appropriate target value – could result in very substantial over-treatment and promote antibiotic resistance
Scenarios with “pneumonia antibiotic treatment rate” indicator
![Page 17: Validity of ”antibiotic treatment for suspected pneumonia ... · Validity of ”antibiotic treatment for pneumonia” indicator Professor Harry Campbell, Centre for Global Health](https://reader034.vdocuments.us/reader034/viewer/2022042323/5f0d960b7e708231d43b16cf/html5/thumbnails/17.jpg)
Validity of the numerator of the indicator
• “pill board” was not labour intensive to develop• “pill board” significantly improved antibiotic treatment
recall• correct recall increased from 63% to 89% [overall kappa
statistic for agreement increased by ~ 0.15]• replicated now in 4 diverse Asian / African settings
![Page 18: Validity of ”antibiotic treatment for suspected pneumonia ... · Validity of ”antibiotic treatment for pneumonia” indicator Professor Harry Campbell, Centre for Global Health](https://reader034.vdocuments.us/reader034/viewer/2022042323/5f0d960b7e708231d43b16cf/html5/thumbnails/18.jpg)
Conclusions• poor discriminative power of DHS/MICS
questions for identifying true childhood pneumonia confirmed in African setting
• as most identified cases are NOT pneumonia, this does not serve as an adequate basis for asking about pneumonia treatment rate
• if programmes aim to increase this “treatment rate” they will drive over-prescription of antibiotics
“Coverage of core interventions remains low”
![Page 19: Validity of ”antibiotic treatment for suspected pneumonia ... · Validity of ”antibiotic treatment for pneumonia” indicator Professor Harry Campbell, Centre for Global Health](https://reader034.vdocuments.us/reader034/viewer/2022042323/5f0d960b7e708231d43b16cf/html5/thumbnails/19.jpg)
“careful assessment of how well countries are progressing on the delivery of proven interventions”
“only a core set of proven indicators are analysed”
![Page 20: Validity of ”antibiotic treatment for suspected pneumonia ... · Validity of ”antibiotic treatment for pneumonia” indicator Professor Harry Campbell, Centre for Global Health](https://reader034.vdocuments.us/reader034/viewer/2022042323/5f0d960b7e708231d43b16cf/html5/thumbnails/20.jpg)
Kruk ME et al. Lancet Global Health 2018
![Page 21: Validity of ”antibiotic treatment for suspected pneumonia ... · Validity of ”antibiotic treatment for pneumonia” indicator Professor Harry Campbell, Centre for Global Health](https://reader034.vdocuments.us/reader034/viewer/2022042323/5f0d960b7e708231d43b16cf/html5/thumbnails/21.jpg)
Global pneumonia report released on World Pneumonia Day 2018
![Page 22: Validity of ”antibiotic treatment for suspected pneumonia ... · Validity of ”antibiotic treatment for pneumonia” indicator Professor Harry Campbell, Centre for Global Health](https://reader034.vdocuments.us/reader034/viewer/2022042323/5f0d960b7e708231d43b16cf/html5/thumbnails/22.jpg)
![Page 23: Validity of ”antibiotic treatment for suspected pneumonia ... · Validity of ”antibiotic treatment for pneumonia” indicator Professor Harry Campbell, Centre for Global Health](https://reader034.vdocuments.us/reader034/viewer/2022042323/5f0d960b7e708231d43b16cf/html5/thumbnails/23.jpg)
![Page 24: Validity of ”antibiotic treatment for suspected pneumonia ... · Validity of ”antibiotic treatment for pneumonia” indicator Professor Harry Campbell, Centre for Global Health](https://reader034.vdocuments.us/reader034/viewer/2022042323/5f0d960b7e708231d43b16cf/html5/thumbnails/24.jpg)
Conclusions and Recommendations• low discriminative power + low pneumonia prevalence so most DHS/ MICS reported episodes don’t have pneumonia• invalid denominator for pneumonia antibiotic treatment indicator
![Page 25: Validity of ”antibiotic treatment for suspected pneumonia ... · Validity of ”antibiotic treatment for pneumonia” indicator Professor Harry Campbell, Centre for Global Health](https://reader034.vdocuments.us/reader034/viewer/2022042323/5f0d960b7e708231d43b16cf/html5/thumbnails/25.jpg)
Conclusions and Recommendations
• Do not attempt to use data on reported antibiotics given for symptoms of ARI as a proxy indicator of antibiotic treatment rate of pneumonia
• DHS and MICS now make their ARI questions very clear and do NOT attempt to measure “pneumonia treatment rate”;
• however, some other agencies still interpret these data as “pneumonia treatment rate”
• this is misleading and likely to lead to incorrect conclusions / decisions
![Page 26: Validity of ”antibiotic treatment for suspected pneumonia ... · Validity of ”antibiotic treatment for pneumonia” indicator Professor Harry Campbell, Centre for Global Health](https://reader034.vdocuments.us/reader034/viewer/2022042323/5f0d960b7e708231d43b16cf/html5/thumbnails/26.jpg)
Global pneumonia report released on World Pneumonia Day 2018
Conclusions and Recommendations
![Page 27: Validity of ”antibiotic treatment for suspected pneumonia ... · Validity of ”antibiotic treatment for pneumonia” indicator Professor Harry Campbell, Centre for Global Health](https://reader034.vdocuments.us/reader034/viewer/2022042323/5f0d960b7e708231d43b16cf/html5/thumbnails/27.jpg)
Conclusions and Recommendations
• Measures of effective coverage of pneumonia treatment should be explored by linking data on care-seeking coverage from household surveys with data on quality of care from health provider surveys
• Indicator validation studies are useful in determining which household survey questions to include and how data should be interpreted; and can give information on how indicators can be improved
![Page 28: Validity of ”antibiotic treatment for suspected pneumonia ... · Validity of ”antibiotic treatment for pneumonia” indicator Professor Harry Campbell, Centre for Global Health](https://reader034.vdocuments.us/reader034/viewer/2022042323/5f0d960b7e708231d43b16cf/html5/thumbnails/28.jpg)
For more details:see JOGH
![Page 29: Validity of ”antibiotic treatment for suspected pneumonia ... · Validity of ”antibiotic treatment for pneumonia” indicator Professor Harry Campbell, Centre for Global Health](https://reader034.vdocuments.us/reader034/viewer/2022042323/5f0d960b7e708231d43b16cf/html5/thumbnails/29.jpg)
Thanks & Acknowledgements
Department of Paediatrics, University of Ibadan and University College Hospital, Ibadan, Nigeria• Adejumoke I. Ayede; Kayode R. Fowobaje; Adebayo A
Bakare; Oladapo B. Oyewole; Oluwaseun B. Olorunfemi; Oluwaseun Kuna; Nkechi T. Iwuala; Abolanle Oguntoye; O. Simeon; Mofeyisade E. Okunlola; Adegoke G. Falade
University of Edinburgh• Amir Kirolos; Linda Williams; Harish Nair; Harry CampbellWHO• Shamim A. Qazi
• Funding for the Improving Coverage Measurement project was provided by the Bill and Melinda Gates Foundation through a grant to the Johns Hopkins University Bloomberg School of Public Health (OPP1084442).
![Page 30: Validity of ”antibiotic treatment for suspected pneumonia ... · Validity of ”antibiotic treatment for pneumonia” indicator Professor Harry Campbell, Centre for Global Health](https://reader034.vdocuments.us/reader034/viewer/2022042323/5f0d960b7e708231d43b16cf/html5/thumbnails/30.jpg)
![Page 31: Validity of ”antibiotic treatment for suspected pneumonia ... · Validity of ”antibiotic treatment for pneumonia” indicator Professor Harry Campbell, Centre for Global Health](https://reader034.vdocuments.us/reader034/viewer/2022042323/5f0d960b7e708231d43b16cf/html5/thumbnails/31.jpg)
Some additional slides
![Page 32: Validity of ”antibiotic treatment for suspected pneumonia ... · Validity of ”antibiotic treatment for pneumonia” indicator Professor Harry Campbell, Centre for Global Health](https://reader034.vdocuments.us/reader034/viewer/2022042323/5f0d960b7e708231d43b16cf/html5/thumbnails/32.jpg)
.
![Page 33: Validity of ”antibiotic treatment for suspected pneumonia ... · Validity of ”antibiotic treatment for pneumonia” indicator Professor Harry Campbell, Centre for Global Health](https://reader034.vdocuments.us/reader034/viewer/2022042323/5f0d960b7e708231d43b16cf/html5/thumbnails/33.jpg)
Caregiver recall of antibiotic treatment