1Quach A, et al. BMJ Global Health 2021;6:e006804. doi:10.1136/bmjgh-2021-006804

Assessing the quality of care for children attending health facilities: a systematic review of assessment tools

Alicia Quach,1,2 Shidan Tosif,1,3 Herfina Nababan,4 Trevor Duke,1,3 Stephen M Graham,1,5 Wilson M Were,6 Moise Muzigaba,6 Fiona M Russell1,2

Original research

To cite: Quach A, Tosif S, Nababan H, et al. Assessing the quality of care for children attending health facilities: a systematic review of assessment tools. BMJ Global Health 2021;6:e006804. doi:10.1136/bmjgh-2021-006804

Handling editor Seye Abimbola

► Additional material is published online only. To view please visit the journal online (http:// dx. doi. org/ 10. 1136/ bmjgh- 2021- 006804).

Received 3 July 2021Accepted 9 September 2021

For numbered affiliations see end of article.

Correspondence toDr Alicia Quach; alicia. quach@ mcri. edu. au

© Author(s) (or their employer(s)) 2021. Re- use permitted under CC BY- NC. No commercial re- use. See rights and permissions. Published by BMJ.

ABSTRACTIntroduction Assessing quality of healthcare is integral in determining progress towards equitable health outcomes worldwide. Using the WHO ‘Standards for improving quality of care for children and young adolescents in health facilities’ as a reference standard, we aimed to evaluate existing tools that assess quality of care for children.Methods We undertook a systematic literature review of publications/reports between 2008 and 2020 that reported use of quality of care assessment tools for children (<15 years) in health facilities. Identified tools were reviewed against the 40 quality statements and 510 quality measures from the WHO Standards to determine the extent each tool was consistent with the WHO Standards. The protocol was registered in PROSPERO ID: CRD42020175652.Results Nine assessment tools met inclusion criteria. Two hospital care tools developed by WHO- Europe and WHO- South- East Asia Offices had the most consistency with the WHO Standards, assessing 291 (57·1%) and 208 (40·8%) of the 510 quality measures, respectively. Remaining tools included between 33 (6·5%) and 206 (40·4%) of the 510 quality measures. The WHO- Europe tool was the only tool to assess all 40 quality statements. The most common quality measures absent were related to experience of care, particularly provision of educational, emotional and psychosocial support to children and families, and fulfilment of children’s rights during care.Conclusion Quality of care assessment tools for children in health facilities are missing some key elements highlighted by the WHO Standards. The WHO Standards are, however, extensive and applying all the quality measures in every setting may not be feasible. A consensus of key indicators to monitor the WHO Standards is required. Existing tools could be modified to include priority indicators to strengthen progress reporting towards delivering quality health services for children. In doing so, a balance between comprehensiveness and practical utility is needed.PROSPERO registration number CRD42020175652.

INTRODUCTIONEnding preventable child deaths by 2030 is a major focus for the Sustainable Development Goals (SDGs).1 A crucial factor to achieving

this is Universal Health Coverage (UHC) which ensures that all people, including children, have access to quality essential healthcare services without being pushed

Key questions

What is already known? ► There are no universally agreed indicators to assess quality of health care.

► Previous reviews on quality of health care for children in low- income and middle- income countries (LMICs) tend to concentrate on system input measures such as physical infrastructure, availability of essential medicines, equipment and human resources.

► There has been no systematic review of existing as-sessment tools for quality of health care for children in health facilities.

What are the new findings? ► This is the first systematic review to compare ex-isting quality of care assessment tools against the WHO ‘Standards for improving the quality of care for children and young adolescents in health facilities’, and found that they do not adequately assess the WHO Standards in its current format.

► Most assessment tools were more comprehensive in assessing provision of care and available human and physical resources, but deficient in assessing experience of care.

► Most assessment tools focused more on input and process measures than outcome measures.

What do the new findings imply? ► There is no existing assessment tool that can com-prehensively assess all the indicators in the ‘WHO Standards’, however, the indicators are extensive and may not be feasible for LMICs to comprehen-sively assess.

► Future endeavours should focus on identifying and obtaining consensus on a selection of key indicators in the assessment of quality of health care for chil-dren in health facilities. Harmonisation of key indica-tors embedded within existing assessment tools will enable regular monitoring and comparable data in order to report progress in the quality of health care for children at local and national levels.

on Novem

ber 23, 2021 by guest. Protected by copyright.

http://gh.bmj.com

/B

MJ G

lob Health: first published as 10.1136/bm

jgh-2021-006804 on 4 October 2021. D

ownloaded from

2 Quach A, et al. BMJ Global Health 2021;6:e006804. doi:10.1136/bmjgh-2021-006804

BMJ Global Health

into financial hardship. Quality healthcare is defined by WHO as health services which are ‘effective, efficient, accessible, patient centred, equitable and safe’.2 To further reduce child deaths, many countries will need to find ways to increase UHC with quality healthcare for children.

Determining progress in quality of healthcare delivery for children requires monitoring and tracking of measur-able indicators. However, there are no universally agreed indicators for quality of care (QoC). To better under-stand the complex multidimensional nature of quality healthcare, WHO developed a framework to identify domains to assess, improve and monitor the quality of paediatric care in health facilities, an extension of the earlier framework for improving maternal and newborn care in health facilities.3 4 The framework encompasses three broad categories of QoC: (A) provision of care—evidence- based practices, effective information systems and referral pathways; (B) experience of care—effec-tive communication, recognition of child rights and appropriate emotional and psychological support; and (C) available human and physical resources to meet the best interests of children. The broad categories are subdivided into eight domains to provide a structured approach when addressing QoC at all levels of the health system (online supplemental appendix A). These eight domains reflect the eight quality standards (QSd) in the WHO ‘Standards for improving the QoC for children and young adolescents in health facilities’ which are further detailed in 40 priority statements and 510 measurable indicators.4 The WHO Standards can, therefore, be used as a standard point of reference when assessing QoC for children in a healthcare facility.

Historically, various tools have been developed to assess QoC for children. We sought to understand if these tools adequately assess all aspects of QoC as outlined by the WHO Standards for children and young adolescents. A recent review identified and compared five existing assess-ment tools to the WHO ‘Standards for improving quality of maternal and newborn care in health facilities’.5 6 The percentage of indicators outlined in the WHO Standards that the five tools were able to assess ranged from 12% to 62%.6 There has been no systematic review of existing assessment tools for QoC for children <15 years. There is an urgent need to better understand the capacity of readily available tools to assess the QoC for children, in order to meet the SDG targets for child health.

The aim of this systematic literature review is to identify existing tools used to assess QoC for children and young adolescents in health facilities and assess the extent to which they represent the domains in the WHO QSd.

METHODSSearch strategy and selection criteriaA systematic review of the literature was undertaken in August 2020 using Preferred Reporting Items for System-atic Reviews and Meta- Analyses reporting guidelines, to

identify assessment tools available globally that evaluate QoC for children attending health facilities.7 MEDLINE (Ovid) database was searched using Medical Subject Heading terms and/or keywords. PubMed was searched using keywords, to retrieve items not indexed on MEDLINE. The PubMed search strategy was adapted for use in Global Health (Commonwealth Agricultural Bureaux direct) database and the International Journal for Quality in Health Care. Additional peer- reviewed publications were identified through handsearching of reference lists of key articles. Grey literature was iden-tified by conducting a keyword search using the World Bank and WHO library databases. The search strategies and results yielded are available from online supple-mental appendix B.

The inclusion criteria for eligibility included publi-cations/reports that: reported the use of an assessment tool to evaluate QoC in a primary, secondary or tertiary level healthcare facility. The assessment tool was deemed eligible if used by more than one country; and included at least one module/component evaluating QoC in chil-dren. A child was defined as aged 0–14 years, to align with the definition of ‘birth up to 15 years’ used in the WHO Standards. To identify tools more likely to be in current use and available globally, the search period was limited to ten years preceding the publication of the WHO Standards to present time (2008–2020), and to those published in English language. Exclusion criteria included publications/reports of assessment tools that: evaluated only newborns (<1 month old) or only adoles-cents (10–19 years old); were developed only for research purposes; evaluated QoC for a specific disease; or a niche component of QoC (eg, antimicrobial prescribing prac-tice). For any assessment tools not publicly available, authors and/or the original developers of the tool were contacted.

The screening process was performed independently by two reviewers using Covidence systematic review soft-ware.8 Titles and abstracts were screened and excluded if inclusion/exclusion criteria were not met. Full texts of remaining publications/reports were assessed for eligi-bility. The assessment tools from full texts were retrieved and further assessed to ensure that they met eligibility criteria. The assessment tools that were identified in multiple reports were grouped together as one unique tool for analysis. Conflicts in determining whether an article/assessment tool met eligibility criteria were resolved by discussion between the two reviewers. If consensus was not reached, a third reviewer was consulted.

Data analysisEach quality assessment tool included was compared against the WHO ‘Standards for improving QoC for chil-dren and young adolescents in health facilities’.4 The WHO Standards comprises of eight overarching ‘QSd’, each one correlating to one domain of the framework for improving quality of paediatric care. Each QSd is composed of priorities or ‘Quality Statements (QSt)’

on Novem

ber 23, 2021 by guest. Protected by copyright.

http://gh.bmj.com

/B

MJ G

lob Health: first published as 10.1136/bm

jgh-2021-006804 on 4 October 2021. D

ownloaded from

Quach A, et al. BMJ Global Health 2021;6:e006804. doi:10.1136/bmjgh-2021-006804 3

BMJ Global Health

(total of 40) for improving QoC for children. The QSt further subdivided into 510 ‘Quality Measures (QM)’, comprised of 235 input, 169 process/output and 106 outcome measures (figure 1). A full list of the QSd, QSt and QM are listed in online supplemental appendix C.

For each assessment tool, we used the most updated version in English in its generic format. Each tool was composed of various modules to assess quality (eg, direct clinical observations, health worker interviews, inventory checklists). We evaluated all modules and excluded those not relevant to our review (eg, antenatal care services). For the remaining modules, we reviewed each question/statement and matched them if applicable to the relevant QM in the WHO Standards. Two paediatricians (AQ and ST) performed the matching process independently to decrease the risk of bias. Any conflicts were discussed between the two reviewers and a third reviewer was consulted for any unresolved conflicts.

We used a similar scoring system as the review of facility assessment tools on maternal and newborn QoC in health facilities performed by Brizuela et al.6 A ques-tion/statement from the tool was considered a match to a WHO QM if any component of the QM was included. If a WHO QM was matched, a score of 1 was allocated. Although multiple questions could match the same QM, each QM could only score a maximum of 1. For a QM that consisted of more than one subcomponent, a ques-tion/statement from the assessment tool only had to fulfil one subcomponent to be considered a match. For example, QM 1.1.1: ‘health facility maintains an up- to- date 24 hours staff duty roster, with a functioning contact mechanism for finding additional support, which ensures that staff responsible for paediatric triage are available at all times’; would be matched by a question asking if

the health facility has a 24- hour staff roster.4 Conversely, a single question/statement could also be matched to more than one QM. For example, an assessment tool with a checklist of available antibiotics in the health facility would match the WHO QM detailing adequate supplies of antibiotics to treat pneumonia (QM 1.3.3), sepsis (QM 1.5.4), neonatal infections (QM 1.2.2); while also matching the QM detailing adequate stocks of essential medicines (QM 8.4.4).

Descriptive statistics were used to calculate the percentage of matched QM, QSt and QSd for each assess-ment tool. The assessment tools were ranked according to the total percentage of WHO QM assessable. Assess-ment tools were also categorised according to whether they were able to completely assess (100%), partially assess (1%–49% and 50%–99%), or not assess (0%) any of the QSd and QSt.

Patient and public involvementNo patients or public were involved in this study.

RESULTSThe search strategy identified 1180 publications/reports, after duplicates were removed (figure 2). The screening process excluded 1035 publications/reports. The remaining 145 full- text manuscripts were assessed for eligibility, with 39 publications/reports being deemed eligible. These publications/reports were further evalu-ated to collate duplications of assessment tools, with 10 unique tools identified as eligible. One tool was not accessible from the author, leaving nine unique assess-ment tools for analysis (figure 2).

Table 1 summarises the nine assessment tools and the modules that were evaluated as part of our analysis. All tools were developed for use in low- income and middle- income countries (LMICs), except for the Child Hospital Consumer Assessment of Healthcare Providers and Systems (HCAHPS).9 All tools were available in English, but could be adapted/translated to local context. All tools were structured questionnaires/interviews with checklist style questions, but varied in length and compo-sition of modules. The shortest tool was the HCAHPS with 62 multichoice questions. Others such as the Service Provision Assessment (SPA) and the WHO Hospital Care assessment tools were detailed with over 100 pages, had multiple modules, with over 100 questions/checklist items per module.10–12

Table 2 summarises the percentages of WHO QM within each QSd assessable by each tool. Overall, QM related to the domains of provision of care and available human and physical resources were more widely assessable than experience of care. QSd 1: ‘Every child receives evidence- based care and management of illness according to WHO guidelines’ was most comprehensively assessable across all tools, apart from the Child HCAHPS which did not assess it at all.4 9 QSd 6: ‘All children and their families are provided with educational, emotional and psychosocial

Figure 1 Structure of the WHO ‘Standards for improving the quality of care for children and youngadolescents in health facilities’.

on Novem

ber 23, 2021 by guest. Protected by copyright.

http://gh.bmj.com

/B

MJ G

lob Health: first published as 10.1136/bm

jgh-2021-006804 on 4 October 2021. D

ownloaded from

4 Quach A, et al. BMJ Global Health 2021;6:e006804. doi:10.1136/bmjgh-2021-006804

BMJ Global Health

support that is sensitive to their needs and strengthens their capability’ was absent from five of the nine included tools.4 All assessment tools were able to partially assess QSd 7 (staff availability) and QSd 8 (health facility phys-ical infrastructure, waste management, supplies and equipment).

Overall, the tools were more comprehensive at assessing the input QM (median 32·8%, IQR (16·4%–45·3%)) and process/output QM (median 21·3%, IQR (12·4–41·7%)) compared with the outcome QM (median 16·0%, IQR (3·8%–28·3%)). Table 3 summarises the proportion of QSt that had at least one each of input, process and outcome QM within each QSd. No single tool was able to fulfil this for all 40 WHO QSt. The percentages of QM that was assessable within each of the 40 QSt are summarised

in figure 3. The WHO (Europe regional office) ‘Hospital care for children: quality assessment and improvement tool’ was the only tool able to partially assess every QSt. Of the remaining tools, 5–27 of 40 QSt were not assess-able at all. The SPA and the Health Resources Available Mapping System (HeRAMS) were the only tools able to assess 100% of any one QSt. The HeRAMS, however, failed to assess any QM of 15 other QSt.

Figure 4 shows the overall percentage of QM assess-able by each tool. The WHO- Europe tool was the most comprehensive with 291 (57·1%) of the 510 WHO QM being assessable.11 The remaining tools varied from 6·5% to 40·8% in their capacities to assess the QM. Table 4 shows the percentage of assessable QM in each QSt. Most tools assessed less than half the QM in any single QSt. The

Figure 2 PRISMA flow diagram for the selection of assessment tools. PRISMA, Preferred Reporting Items for Systematic Reviews and Meta- Analyses.

on Novem

ber 23, 2021 by guest. Protected by copyright.

http://gh.bmj.com

/B

MJ G

lob Health: first published as 10.1136/bm

jgh-2021-006804 on 4 October 2021. D

ownloaded from

Quach A, et al. BMJ Global Health 2021;6:e006804. doi:10.1136/bmjgh-2021-006804 5

BMJ Global Health

Tab

le 1

S

umm

ary

of a

sses

smen

t to

ols

revi

ewed

for

qua

lity

of c

are

for

child

ren

in h

ealth

faci

litie

s

Too

lD

evel

op

men

t/so

urce

Typ

e o

f he

alth

fac

ility

to

ol d

esig

ned

fo

rP

op

ulat

ion

and

ser

vice

s to

ol d

esig

ned

fo

rK

ey t

op

ics

asse

ssed

b

y to

ol

Co

untr

ies

that

hav

e us

ed t

oo

l*Ve

rsio

n as

sess

edM

od

ules

incl

uded

in

asse

ssm

ent

Ser

vice

Pro

visi

on

Ass

essm

ent10

US

AID

-

ME

AS

UR

E

Eva

luat

ion

All

heal

th fa

cilit

y ty

pes

All

ages

Fam

ily p

lann

ing,

mat

erna

l an

d c

hild

hea

lth s

ervi

ces,

co

mm

unic

able

and

non

- co

mm

unic

able

dis

ease

s,

bas

ic s

urge

ry

►

Ava

ilab

ility

of h

ealth

se

rvic

es

►S

ervi

ces

read

ines

s of

he

alth

faci

litie

s

►E

xten

t se

rvic

es fo

llow

ac

cep

ted

sta

ndar

ds

of c

are

►

Sat

isfa

ctio

n le

vels

of

clie

nts

and

ser

vice

p

rovi

der

s

Afg

hani

stan

, B

angl

ades

h, C

ongo

D

R, E

gyp

t, E

thio

pia

, G

hana

, Gua

tem

ala,

G

uyan

a, H

aiti,

Ken

ya,

Mal

awi,

Nam

ibia

, N

epal

, Rw

and

a,

Sen

egal

, Tan

zani

a,

Uga

nda,

Zam

bia

2012

►

Inve

ntor

y

►C

linic

al

obse

rvat

ion—

sick

ch

ild

►E

xit

inte

rvie

w—

care

take

r of

sic

k ch

ild

►H

ealth

wor

ker

inte

rvie

w

Rap

id H

ealth

Fac

ility

A

sses

smen

t32U

SA

ID- M

EA

SU

RE

E

valu

atio

nP

rimar

y he

alth

care

fa

cilit

ies

Wom

en a

nd c

hild

ren

Mat

erna

l, ne

onat

al a

nd

child

hea

lth s

ervi

ces

►

Ass

essm

ent,

d

iagn

osis

and

tr

eatm

ent

of c

omm

on

child

hood

illn

esse

s

►S

ervi

ce r

ead

ines

s of

he

alth

faci

litie

s

►Q

ualit

y of

m

anag

emen

t p

roce

sses

in h

ealth

fa

cilit

ies

Nig

eria

, Sou

th S

udan

2008

►

Clin

ical

ob

serv

atio

n—si

ck

child

►

Exi

t in

terv

iew

—ca

reta

ker

of s

ick

child

►

Hea

lth w

orke

r in

terv

iew

and

re

cord

rev

iew

►

Hea

lth fa

cilit

y ch

eckl

ist

Ser

vice

Ava

ilab

ility

and

R

ead

ines

s A

sses

smen

t19W

HO

All

heal

th fa

cilit

y ty

pes

All

ages

Mat

erna

l and

chi

ld h

ealth

se

rvic

es, c

omm

unic

able

an

d n

on- c

omm

unic

able

d

isea

ses,

sur

gica

l car

e

►

Ser

vice

ava

ilab

ility

- in

fras

truc

ture

, cor

e he

alth

per

sonn

el,

serv

ice

utili

satio

n

►G

ener

al s

ervi

ce

read

ines

s- b

asic

eq

uip

men

t, a

men

ities

, in

fect

ion

pre

vent

ion,

d

iagn

ostic

cap

acity

, es

sent

ial m

edic

ines

►

Ser

vice

- sp

ecifi

c re

adin

ess

Ban

glad

esh,

Ben

in,

Bur

kina

Fas

o,

Cam

bod

ia, C

ongo

D

R, C

ote

D’Iv

oire

, D

jibou

ti, G

hana

, Hai

ti,

Ind

ia, K

enya

, Lao

PD

R,

Lib

ya M

adag

asca

r, M

aurit

ania

, Mya

nmar

, N

amib

ia, N

iger

, R

wan

da,

Sen

egal

, S

ierr

a Le

one,

Som

alia

, S

outh

Sud

an T

anza

nia,

To

go, U

gand

a V

ietn

am,

Zam

bia

, Zim

bab

we

2015

►

Sta

ffing

►

Infr

astr

uctu

re

►A

vaila

ble

ser

vice

s

►D

iagn

ostic

s

►M

edic

ines

and

co

mm

oditi

es

Con

tinue

d

on Novem

ber 23, 2021 by guest. Protected by copyright.

http://gh.bmj.com

/B

MJ G

lob Health: first published as 10.1136/bm

jgh-2021-006804 on 4 October 2021. D

ownloaded from

6 Quach A, et al. BMJ Global Health 2021;6:e006804. doi:10.1136/bmjgh-2021-006804

BMJ Global Health

Too

lD

evel

op

men

t/so

urce

Typ

e o

f he

alth

fac

ility

to

ol d

esig

ned

fo

rP

op

ulat

ion

and

ser

vice

s to

ol d

esig

ned

fo

rK

ey t

op

ics

asse

ssed

b

y to

ol

Co

untr

ies

that

hav

e us

ed t

oo

l*Ve

rsio

n as

sess

edM

od

ules

incl

uded

in

asse

ssm

ent

Hea

lth F

acili

ty S

urve

y-

usin

g In

tegr

ated

M

anag

emen

t C

hild

hood

Ill

ness

as

clin

ical

gu

idel

ines

33

WH

OP

rimar

y le

vel o

utp

atie

nt

heal

th fa

cilit

ies

All

child

ren

Out

pat

ient

car

e fo

r si

ck

child

ren

►

Qua

lity

of c

ase

man

agem

ent

rece

ived

b

y si

ck c

hild

ren

►

Qua

lity

of c

ouns

ellin

g of

car

etak

ers

►

Ava

ilab

ility

of k

ey

heat

h sy

stem

su

pp

orts

- ed

icin

es,

equi

pm

ent,

su

per

visi

on, r

efer

ral

syst

em

►B

arrie

rs t

o ef

fect

ive

inte

grat

ed c

ase

man

agem

ent

Bot

swan

a, C

amb

odia

, G

uate

mal

a, K

enya

, Z

amb

ia, A

fgha

nist

an,

Rw

and

a, C

hina

, M

alaw

i, M

ongo

lia,

Nig

eria

, Pap

ua N

ew

Gui

nea,

Phi

lipp

ines

, V

ietn

am, K

irib

ati,

Lao

PD

R, S

olom

on Is

land

s

2003

►

Clin

ical

ob

serv

atio

n—si

ck

child

►

Exi

t in

terv

iew

—ca

reta

ker

of s

ick

child

►

Re-

exam

inat

ion

of

sick

chi

ld

►E

qui

pm

ent

and

su

pp

ly c

heck

list

Ass

essm

ent

Tool

for

Hos

pita

l Car

e (W

HO

- SE

A

sia

Offi

ce)12

WH

O- S

E A

sia

Offi

ceA

ll ho

spita

lsW

omen

and

chi

ldre

nM

ater

nal,

neon

atal

and

ch

ild h

ealth

ser

vice

s

►

Ser

vice

ava

ilab

ility

an

d r

ead

ines

s-

infr

astr

uctu

re, h

osp

ital

sup

por

t sy

stem

s,

staf

fing,

ess

entia

l m

edic

ines

and

d

iagn

ostic

s

►Q

ualit

y of

med

ical

ca

re p

rovi

ded

for

mat

erna

l, ne

onat

al

and

chi

ld h

ealth

se

rvic

es

►S

atis

fact

ion

leve

ls o

f cl

ient

s an

d s

ervi

ce

pro

vid

ers

Ind

ones

ia, N

epal

2016

►

Gen

eral

hos

pita

l in

form

atio

n

►P

aed

iatr

ic c

are

►

Car

egiv

er in

terv

iew

►

Hea

lth w

orke

r in

terv

iew

►

Pos

tal

que

stio

nnai

re

Tab

le 1

C

ontin

ued

Con

tinue

d

on Novem

ber 23, 2021 by guest. Protected by copyright.

http://gh.bmj.com

/B

MJ G

lob Health: first published as 10.1136/bm

jgh-2021-006804 on 4 October 2021. D

ownloaded from

Quach A, et al. BMJ Global Health 2021;6:e006804. doi:10.1136/bmjgh-2021-006804 7

BMJ Global Health

Too

lD

evel

op

men

t/so

urce

Typ

e o

f he

alth

fac

ility

to

ol d

esig

ned

fo

rP

op

ulat

ion

and

ser

vice

s to

ol d

esig

ned

fo

rK

ey t

op

ics

asse

ssed

b

y to

ol

Co

untr

ies

that

hav

e us

ed t

oo

l*Ve

rsio

n as

sess

edM

od

ules

incl

uded

in

asse

ssm

ent

Hos

pita

l car

e fo

r ch

ildre

n:

qua

lity

asse

ssm

ent

and

im

pro

vem

ent

tool

(WH

O-

Eur

ope

Offi

ce)11

WH

O- E

urop

e O

ffice

All

hosp

itals

All

child

ren,

exc

lud

ing

new

bor

nsP

aed

iatr

ic s

ervi

ces,

ex

clud

ing

neon

atal

he

alth

care

►

Ser

vice

ava

ilab

ility

an

d r

ead

ines

s-

infr

astr

uctu

re, h

osp

ital

sup

por

t sy

stem

s,

heal

th m

anag

emen

t in

form

atio

n sy

stem

s,

staf

fing,

ess

entia

l m

edic

ines

and

eq

uip

men

t

►Q

ualit

y of

med

ical

ca

re p

rovi

ded

fo

r co

mm

on

child

hood

illn

esse

s an

d e

mer

genc

y p

rese

ntat

ions

►

Hos

pita

l pol

ices

and

gu

idel

ines

- inf

ectio

n p

reve

ntio

n, t

rain

ing,

ac

cess

, pat

ient

’s

right

s

►S

atis

fact

ion

leve

ls o

f cl

ient

s an

d s

ervi

ce

pro

vid

ers

Kyr

gyst

an, T

ajik

ista

n,

Eth

iop

ia, A

ngol

a,

Mal

awi,

Moz

amb

ique

2015

►

Hos

pita

l sup

por

t se

rvic

es

►C

ase

man

agem

ent

►

Pol

icie

s an

d

orga

nisa

tion

of

serv

ices

►

Inte

rvie

ws

Hea

lth R

esou

rces

A

vaila

bili

ty M

app

ing

Sys

tem

34

WH

OH

uman

itaria

n an

d

Em

erge

ncy

Res

pon

se

sett

ings

All

ages

Ess

entia

l tra

uma

care

, ch

ild h

ealth

and

nut

ritio

n,

com

mun

icab

le d

isea

ses,

m

enta

l hea

lth, s

exua

l and

re

pro

duc

tive

heal

th

►

Ava

ilab

ility

of h

ealth

fa

cilit

ies

and

ser

vice

s

►R

esou

r ces

for

serv

ice

del

iver

y- in

fras

truc

ture

, hu

man

res

ourc

es,

com

mun

icat

ions

, in

fect

ion

cont

rol

►

Rea

sons

for

gap

s in

se

rvic

e av

aila

bili

ty

Sud

an, U

gand

a20

17

►H

osp

itals

as

sess

men

t to

ol

►H

ealth

cen

tres

as

sess

men

t to

ol

Tab

le 1

C

ontin

ued

Con

tinue

d

on Novem

ber 23, 2021 by guest. Protected by copyright.

http://gh.bmj.com

/B

MJ G

lob Health: first published as 10.1136/bm

jgh-2021-006804 on 4 October 2021. D

ownloaded from

8 Quach A, et al. BMJ Global Health 2021;6:e006804. doi:10.1136/bmjgh-2021-006804

BMJ Global Health

Too

lD

evel

op

men

t/so

urce

Typ

e o

f he

alth

fac

ility

to

ol d

esig

ned

fo

rP

op

ulat

ion

and

ser

vice

s to

ol d

esig

ned

fo

rK

ey t

op

ics

asse

ssed

b

y to

ol

Co

untr

ies

that

hav

e us

ed t

oo

l*Ve

rsio

n as

sess

edM

od

ules

incl

uded

in

asse

ssm

ent

Hea

lth R

esul

ts B

ased

Fi

nanc

ing

imp

act

eval

uatio

n to

olki

t35

Wor

ld B

ank

All

heal

thca

re fa

cilit

ies

(Pro

gram

me

eval

uatin

g th

e im

pac

t of

hea

lth-

rela

ted

res

ults

- bas

ed

finan

cing

ince

ntiv

es)

All

ages

with

focu

s on

w

omen

and

chi

ldre

nM

ater

nal,

neon

atal

an

d c

hild

hea

lth,

com

mun

icab

le d

isea

ses

►

Qua

ntity

of h

ealth

se

rvic

es d

eliv

ered

►

Qua

lity

of h

ealth

se

rvic

es p

rovi

ded

►

Hea

lth o

utco

mes

►

Res

ourc

e m

anag

emen

t at

hea

lth

faci

litie

s

►Im

pac

t on

non

- res

ults

b

ased

fina

ncin

g se

rvic

es d

eliv

ered

►

Dis

aggr

egat

ion

of

imp

act

by

pro

vid

er

and

pop

ulat

ion

char

acte

ristic

s

Tajik

ista

n, B

urki

na

Faso

, Con

go D

R,

Rw

and

a, B

enin

, Le

soth

o, K

yrgy

zsta

n,

Cam

eroo

n, C

entr

al

Afr

ican

Rep

ublic

, The

G

amb

ia,

2012

►

Hea

lth fa

cilit

y as

sess

men

t

►H

ealth

wor

ker

ind

ivid

ual

►

Exi

t in

terv

iew

- ca

regi

ver

for

child

un

der

5

Hos

pita

l Con

sum

er

Ass

essm

ent

of H

ealth

care

P

rovi

der

s an

d S

yste

ms-

C

hild

Ver

sion

9

Bos

ton

Chi

ldre

n’s

Hos

pita

lA

ll ho

spita

lsA

ll ch

ildre

nA

ll p

aed

iatr

ic in

pat

ient

he

alth

ser

vice

s

►

Qua

lity

of

com

mun

icat

ion

with

car

egiv

ers

and

p

atie

nts

►

Att

entio

n to

saf

ety

and

co

mfo

rt

►H

osp

ital e

nviro

nmen

t

Bel

gium

, Can

ada,

U

SA

, Arg

entin

a20

14

►E

ntire

sur

vey

(62

item

s)

*Onl

y in

clud

es c

ount

ries

with

rep

orts

/dat

a p

ublic

ly a

vaila

ble

.

Tab

le 1

C

ontin

ued

on Novem

ber 23, 2021 by guest. Protected by copyright.

http://gh.bmj.com

/B

MJ G

lob Health: first published as 10.1136/bm

jgh-2021-006804 on 4 October 2021. D

ownloaded from

Quach A, et al. BMJ Global Health 2021;6:e006804. doi:10.1136/bmjgh-2021-006804 9

BMJ Global Health

Tab

le 2

P

erce

ntag

e of

WH

O Q

ualit

y S

tand

ard

s as

sess

able

by

each

qua

lity

asse

ssm

ent

tool

Per

cent

age

of

Qua

lity

Mea

sure

s as

sess

able

fo

r ea

ch Q

ualit

y S

tand

ard

WH

O- E

uro

pe

WH

O- S

E A

sia

SPA

HR

BF

HeR

AM

Sr-

HFA

SA

RA

HFS

- IM

CI

HC

AH

PS

- Chi

ld

Sta

ndar

d 1

: Eve

ry c

hild

rec

eive

s ev

iden

ce- b

ased

ca

re a

nd m

anag

emen

t of

illn

ess

acco

rdin

g to

WH

O

guid

elin

es. (

n=22

2)

59%

55%

49%

27%

39%

24%

23%

18%

0%

Sta

ndar

d 2

: The

hea

lth in

form

atio

n sy

stem

ens

ures

th

e co

llect

ion,

ana

lysi

s an

d u

se o

f dat

a to

ens

ure

early

, ap

pro

pria

te a

ctio

n to

imp

rove

the

car

e of

ev

ery

child

. (n=

29)

52%

52%

48%

38%

0%24

%0%

10%

10%

Sta

ndar

d 3

: Eve

ry c

hild

with

con

diti

on(s

) tha

t ca

nnot

be

trea

ted

effe

ctiv

ely

with

the

ava

ilab

le

reso

urce

s re

ceiv

es a

pp

rop

riate

, tim

ely

refe

rral

, with

se

amle

ss c

ontin

uity

of c

are.

(n=

28)

39%

18%

36%

39%

43%

7%7%

18%

0%

Sta

ndar

d 4

: Com

mun

icat

ion

with

chi

ldre

n an

d t

heir

fam

ilies

is e

ffect

ive,

with

mea

ning

ful p

artic

ipat

ion

and

res

pon

ds

to t

heir

need

s an

d p

refe

renc

es.

(n=

45)

53%

22%

29%

24%

9%7%

0%16

%27

%

Sta

ndar

d 5

: Eve

ry c

hild

’s r

ight

s ar

e re

spec

ted

, p

rote

cted

and

fulfi

lled

at

all t

imes

dur

ing

care

, w

ithou

t d

iscr

imin

atio

n. (n

=44

)

45%

11%

20%

14%

7%5%

5%2%

9%

Sta

ndar

d 6

: All

child

ren

and

the

ir fa

mili

es

are

pro

vid

ed w

ith e

duc

atio

nal,

emot

iona

l and

p

sych

osoc

ial s

upp

ort

that

is s

ensi

tive

to t

heir

need

s an

d s

tren

gthe

ns t

heir

cap

abili

ty. (

n=32

)

66%

9%3%

0%0%

0%0%

0%19

%

Sta

ndar

d 7

: For

eve

ry c

hild

, com

pet

ent,

mot

ivat

ed,

emp

athi

c st

aff a

re c

onsi

sten

tly a

vaila

ble

to

pro

vid

e ro

utin

e ca

re a

nd m

anag

emen

t of

com

mon

ch

ildho

od il

lnes

ses.

(n=

44)

70%

34%

52%

59%

2%25

%7%

16%

7%

Sta

ndar

d 8

: The

hea

lth fa

cilit

y ha

s an

ap

pro

pria

te,

child

frie

ndly

phy

sica

l env

ironm

ent,

with

ad

equa

te w

ater

, san

itatio

n, w

aste

man

agem

ent,

en

ergy

sup

ply

, med

icin

es, m

edic

al s

upp

lies

and

eq

uip

men

t fo

r ro

utin

e ca

re a

nd m

anag

emen

t of

co

mm

on c

hild

hood

illn

esse

s. (n

=66

)

56%

50%

41%

39%

35%

20%

32%

3%8%

Red

box

es in

dic

ate

that

the

ass

essm

ent

tool

did

not

ass

ess

any;

ora

nge

box

es in

dic

ate

less

tha

n ha

lf an

d y

ello

w b

oxes

ind

icat

e eq

ual t

o or

mor

e th

an h

alf o

f the

qua

lity

mea

sure

s w

ere

asse

ssab

le in

the

ass

ocia

ted

Qua

lity

Sta

ndar

d.

HC

AH

PS

, Hos

pita

l Con

sum

er A

sses

smen

t of

Hea

lthca

re P

rovi

der

s an

d S

yste

ms;

HeR

AM

S, H

ealth

Res

ourc

es A

vaila

bili

ty M

app

ing

Sys

tem

; HFS

- IM

CI,

Hea

lth F

acili

ty S

urve

y—us

ing

Inte

grat

ed

Man

agem

ent

of C

hild

hood

Illn

ess

clin

ical

gui

del

ines

; HR

BF,

Hea

lth R

esul

ts B

ased

Fin

anci

ng im

pac

t ev

alua

tion

tool

kit;

n, n

umb

er o

f Qua

lity

Mea

sure

s w

ithin

the

ass

ocia

ted

Qua

lity

Sta

ndar

d;

r- H

FA, r

apid

Hea

lth F

acili

ty A

sses

smen

t; S

AR

A, S

ervi

ce A

vaila

bili

ty a

nd R

ead

ines

s A

sses

smen

t; S

PA, S

ervi

ce P

rovi

sion

Ass

essm

ent.

;

on Novem

ber 23, 2021 by guest. Protected by copyright.

http://gh.bmj.com

/B

MJ G

lob Health: first published as 10.1136/bm

jgh-2021-006804 on 4 October 2021. D

ownloaded from

10 Quach A, et al. BMJ Global Health 2021;6:e006804. doi:10.1136/bmjgh-2021-006804

BMJ Global Health

Tab

le 3

P

rop

ortio

n of

WH

O Q

ualit

y S

tate

men

ts w

ith a

t le

ast

one

each

of i

nput

, pro

cess

and

out

put

Qua

lity

Mea

sure

ass

essa

ble

by

each

qua

lity

asse

ssm

ent

tool

Per

cent

age

of

Qua

lity

Mea

sure

s as

sess

able

fo

r ea

ch Q

ualit

y S

tand

ard

WH

O- E

uro

pe

WH

O- S

E A

sia

SPA

HR

BF

HeR

AM

Sr-

HFA

SA

RA

HFS

- IM

CI

HC

AH

PS

- Chi

ld

Sta

ndar

d 1

: Eve

ry c

hild

rec

eive

s ev

iden

ce- b

ased

ca

re a

nd m

anag

emen

t of

illn

ess

acco

rdin

g to

WH

O

guid

elin

es. (

QS

t=15

)

7/15

7/15

6/15

2/15

1/15

0/15

0/15

1/15

0/15

Sta

ndar

d 2

: The

hea

lth in

form

atio

n sy

stem

ens

ures

th

e co

llect

ion,

ana

lysi

s an

d u

se o

f dat

a to

ens

ure

early

, ap

pro

pria

te a

ctio

n to

imp

rove

the

car

e of

eve

ry c

hild

. (Q

St=

3)

2/3

3/3

2/3

2/3

0/3

1/3

0/3

0/3

0/3

Sta

ndar

d 3

: Eve

ry c

hild

with

con

diti

on(s

) tha

t ca

nnot

be

trea

ted

effe

ctiv

ely

with

the

ava

ilab

le r

esou

rces

rec

eive

s ap

pro

pria

te, t

imel

y re

ferr

al, w

ith s

eam

less

con

tinui

ty o

f ca

re. (

QS

t=3)

1/3

0/3

1/3

2/3

1/3

0/3

0/3

0/3

0/3

Sta

ndar

d 4

: Com

mun

icat

ion

with

chi

ldre

n an

d t

heir

fam

ilies

is e

ffect

ive,

with

mea

ning

ful p

artic

ipat

ion,

and

re

spon

ds

to t

heir

need

s an

d p

refe

renc

es (Q

St=

4)

2/4

1/4

1/4

2/4

0/4

0/4

0/4

0/4

1/4

Sta

ndar

d 5

: Eve

ry c

hild

’s r

ight

s ar

e re

spec

ted

, p

rote

cted

and

fulfi

lled

at

all t

imes

dur

ing

care

, with

out

dis

crim

inat

ion.

(QS

t=5)

2/5

0/5

1/5

0/5

0/5

0/5

0/5

0/5

0/5

Sta

ndar

d 6

: All

child

ren

and

the

ir fa

mili

es a

re p

rovi

ded

w

ith e

duc

atio

nal,

emot

iona

l and

psy

chos

ocia

l sup

por

t th

at is

sen

sitiv

e to

the

ir ne

eds

and

str

engt

hens

the

ir ca

pab

ility

. (Q

St=

3)

3/3

0/3

0/3

0/3

0/3

0/3

0/3

0/3

0/3

Sta

ndar

d 7

: For

eve

ry c

hild

, com

pet

ent,

mot

ivat

ed,

emp

athi

c st

aff a

re c

onsi

sten

tly a

vaila

ble

to

pro

vid

e ro

utin

e ca

re a

nd m

anag

emen

t of

com

mon

chi

ldho

od

illne

sses

. (Q

St=

3)

3/3

2/3

2/3

2/3

0/3

1/3

0/3

1/3

0/3

Sta

ndar

d 8

: The

hea

lth fa

cilit

y ha

s an

ap

pro

pria

te,

child

frie

ndly

phy

sica

l env

ironm

ent,

with

ad

equa

te

wat

er, s

anita

tion,

was

te m

anag

emen

t, e

nerg

y su

pp

ly,

med

icin

es, m

edic

al s

upp

lies

and

eq

uip

men

t fo

r ro

utin

e ca

re a

nd m

anag

emen

t of

com

mon

chi

ldho

od il

lnes

ses.

(Q

St=

4)

1/4

1/4

2/4

2/4

0/4

1/4

1/4

0/4

0/4

Tota

l Qua

lity

Sta

tem

ents

=40

21/4

014

/40

15/4

012

/40

2/40

3/40

1/40

2/40

1/40

Red

box

es in

dic

ate

that

the

ass

essm

ent

tool

did

not

incl

ude

any;

ora

nge

box

es in

dic

ate

less

tha

n ha

lf; y

ello

w b

oxes

ind

icat

e eq

ual t

o or

mor

e th

an h

alf a

nd g

reen

box

es in

dic

ate

that

all

of t

he

Qua

lity

Sta

tem

ents

with

in t

he a

ssoc

iate

d Q

ualit

y S

tand

ard

had

at

leas

t on

e ea

ch o

f inp

ut, p

roce

ss a

nd o

utco

me

Qua

lity

Mea

sure

ass

essa

ble

.H

CA

HP

S, H

osp

ital C

onsu

mer

Ass

essm

ent

of H

ealth

care

Pro

vid

ers

and

Sys

tem

s; H

eRA

MS

, Hea

lth R

esou

rces

Ava

ilab

ility

Map

pin

g S

yste

m; H

FS- I

MC

I, H

ealth

Fac

ility

Sur

vey—

usin

g In

tegr

ated

M

anag

emen

t of

Chi

ldho

od Il

lnes

s cl

inic

al g

uid

elin

es; H

RB

F, H

ealth

Res

ults

Bas

ed F

inan

cing

imp

act

eval

uatio

n to

olki

t; Q

St,

num

ber

of Q

ualit

y S

tate

men

ts w

ithin

the

ass

ocia

ted

Qua

lity

Sta

ndar

d; r

- HFA

, rap

id H

ealth

Fac

ility

Ass

essm

ent;

SA

RA

, Ser

vice

Ava

ilab

ility

and

Rea

din

ess

Ass

essm

ent;

SPA

, Ser

vice

Pro

visi

on A

sses

smen

t.;

on Novem

ber 23, 2021 by guest. Protected by copyright.

http://gh.bmj.com

/B

MJ G

lob Health: first published as 10.1136/bm

jgh-2021-006804 on 4 October 2021. D

ownloaded from

Quach A, et al. BMJ Global Health 2021;6:e006804. doi:10.1136/bmjgh-2021-006804 11

BMJ Global Health

WHO- Europe tool and the WHO- SE Asia were the most comprehensive, able to assess more than half the QM in 23 and 13 of the 40 QSt, respectively. The HCAHPS- Child tool had the largest number of gaps, leaving 27 QSt

completely unassessed, but was able to assess 9 (69%) of the 13 QM for QSt 4.1, that is, effective communication given to children and their carers, which is a key objective of the tool.

DISCUSSIONThis is the first systematic review to compare QoC assess-ment tools against the current WHO Standards for chil-dren and young adolescents in health facilities. Three of the nine assessment tools included questions from all eight of the WHO QSd, but only one (WHO- Europe Hospital Care assessment tool) was able to address all 40 QSt. Despite being the most comprehensive tool, the WHO- Europe Hospital Care assessment tool still only included about half of all QM. QSd that included evidence- based management, staffing and physical infra-structure and resources (QSd 1,7 and 8) were more widely covered across the tools than those which encompassed health information systems (HIS), referral processes, communication, psychosocial support and child rights (QSd 2–6).

Previous assessments of QoC in LMICs have often centred on input measures as these are seen to be more tangible, objective measures.6 13–15 This was reflected in this review with most tools assessing more input measures and less process or outcome measures, and no tool assessing at least one input, process and outcome measure in all 40 QSt. Mortality data were largely absent from almost all tools. The Donabedian quality framework describes the relationship between the three components on input/structure, process and outcomes.16 Although structural, input measures are important to healthcare delivery, the flow- on effects to processes such as appro-priate care delivery and adequate communication, and outcome measures such as morbidity and mortality data, and patient satisfaction, determine how successful a health facility is. Including all three components is there-fore crucial for assessing the level of QoC that is being provided for children in hospitals.

The WHO Standards for improving QoC for children and young adolescents were used in this review as a refer-ence standard for assessing QoC for children attending health facilities. They are comprehensive and include all three components of the Donabedian quality framework, while aligning with the SDG emphasis on equity. They have been developed as a resource for healthcare profes-sionals and managers at the health facility level through to government bodies and technical partners responsible for policy and programme development at the national level, to support quality improvement practices for chil-dren.4 How the WHO Standards are implemented in practice and what tools to use when assessing quality of healthcare for children, are to be decided within the local context. However, it is unlikely that a single tool could encompass all 510 QM and be feasible in most settings in LMICs.

Figure 3 Percentage of WHO Quality Statements* assessable by each quality assessment tool. *‘Quality Statements’ are 40 concise statements of the priorities for improving quality of care for children as documented in the WHO Standards. Each quality statement contains from 6 to 22 quality measures.4 Not assessable = the assessment tool did not assess any quality measures in the quality statement. Partially assessable = the assessment tool assessed at least one of the quality measures in the quality statement. Completely assessable = the assessment tool assessed all of the quality measures in the quality statement. HRBF, Health Results Based Financing impact evaluation toolkit; HeRAMS, Health Resources Availability Mapping System; HFS- IMCI, Health Facility Survey—using Integrated Management of Childhood Illness clinical guidelines; HCAHPS, Hospital Consumer Assessment of Healthcare Providers and Systems; r- HFA, rapid Health Facility Assessment; SPA, Service Provision Assessment. SARA, Service Availability and Readiness Assessment.

Figure 4 Overall percentage of WHO Quality Measures* assessable by each quality assessment tool. HRBF, Health Results Based Financing impact evaluation toolkit; HeRAMS, Health Resources Availability Mapping System; HFS- IMCI, Health Facility Survey—using Integrated Management of Childhood Illness clinical guidelines; HCAHPS, Hospital Consumer Assessment of Healthcare Providers and Systems; r- HFA, rapid Health Facility Assessment; SPA, Service Provision Assessment; SARA, Service Availability and Readiness Assessment.

on Novem

ber 23, 2021 by guest. Protected by copyright.

http://gh.bmj.com

/B

MJ G

lob Health: first published as 10.1136/bm

jgh-2021-006804 on 4 October 2021. D

ownloaded from

12 Quach A, et al. BMJ Global Health 2021;6:e006804. doi:10.1136/bmjgh-2021-006804

BMJ Global Health

Tab

le 4

P

erce

ntag

e of

WH

O Q

ualit

y S

tate

men

ts a

sses

sab

le b

y ea

ch q

ualit

y as

sess

men

t to

ol

Per

cent

age

of

Qua

lity

Mea

sure

s as

sess

able

in e

ach

Qua

lity

Sta

tem

ent

WH

O- E

uro

pe

WH

O- S

E A

sia

SPA

HR

BF

HeR

AM

Sr-

HFA

SA

RA

HFS

- IM

CI

HC

AH

PS

- Chi

ld

Sta

ndar

d 1

: Eve

ry c

hild

rec

eive

s ev

iden

ce- b

ased

car

e an

d m

anag

emen

t o

f ill

ness

acc

ord

ing

to

WH

O g

uid

elin

es

Sta

tem

ent

1.1

All

child

ren

are

tria

ged

and

pro

mp

tly

asse

ssed

for

emer

genc

y an

d p

riorit

y si

gns

to d

eter

min

e w

heth

er t

hey

req

uire

res

usci

tatio

n an

d r

ecei

ve

app

rop

riate

car

e ac

cord

ing

to W

HO

gui

del

ines

. (n=

19)

63%

58%

68%

16%

63%

26%

16%

16%

0%

Sta

tem

ent

1.2

All

sick

infa

nts,

esp

ecia

lly s

mal

l new

bor

ns,

are

thor

ough

ly a

sses

sed

for

serio

us b

acte

rial i

nfec

tion

and

rec

eive

ap

pro

pria

te c

are

acco

rdin

g to

WH

O

guid

elin

es (n

=19

)

37%

79%

42%

11%

11%

16%

16%

11%

0%

Sta

tem

ent

1.3

All

child

ren

with

cou

gh o

r d

ifficu

lt b

reat

hing

are

cor

rect

ly a

sses

sed

, cla

ssifi

ed a

nd

inve

stig

ated

and

rec

eive

ap

pro

pria

te c

are

and

/or

antib

iotic

s fo

r p

neum

onia

, acc

ord

ing

to W

HO

gui

del

ines

. (n

=18

)

83%

83%

61%

17%

61%

39%

28%

33%

0%

Sta

tem

ent

1.4

All

child

ren

with

dia

rrho

ea a

re c

orre

ctly

as

sess

ed a

nd c

lass

ified

and

rec

eive

ap

pro

pria

te

rehy

dra

tion

and

car

e, in

clud

ing

cont

inue

d fe

edin

g,

acco

rdin

g to

WH

O g

uid

elin

es. (

n=19

)

74%

68%

68%

32%

74%

53%

16%

42%

0%

Sta

tem

ent

1.5

All

child

ren

with

feve

r ar

e co

rrec

tly

asse

ssed

, cla

ssifi

ed a

nd in

vest

igat

ed a

nd r

ecei

ve

app

rop

riate

car

e ac

cord

ing

to W

HO

gui

del

ines

.(n

=19

)

58%

68%

58%

37%

47%

42%

32%

26%

0%

Sta

tem

ent

1.6

All

infa

nts

and

you

ng c

hild

ren

are

asse

ssed

for

grow

th, b

reas

tfee

din

g an

d n

utrit

ion,

and

th

eir

care

rs r

ecei

ve a

pp

rop

riate

sup

por

t an

d c

ouns

ellin

g,

acco

rdin

g to

WH

O g

uid

elin

es. (

n=12

)

83%

75%

50%

33%

17%

25%

17%

25%

0%

Sta

tem

ent

1.7

All

child

ren

at r

isk

for

acut

e m

alnu

triti

on

and

ana

emia

are

cor

rect

ly a

sses

sed

and

cla

ssifi

ed a

nd

rece

ive

app

rop

riate

car

e ac

cord

ing

to W

HO

gui

del

ines

. (n

=16

)

75%

88%

44%

38%

56%

25%

31%

25%

0%

Sta

tem

ent

1.8

All

child

ren

at r

isk

for

TB a

nd/o

r H

IV

infe

ctio

n ar

e co

rrec

tly a

sses

sed

and

inve

stig

ated

and

re

ceiv

e ap

pro

pria

te m

anag

emen

t ac

cord

ing

to W

HO

gu

idel

ines

. (n=

17)

47%

29%

47%

65%

41%

18%

41%

0%0%

Con

tinue

d

on Novem

ber 23, 2021 by guest. Protected by copyright.

http://gh.bmj.com

/B

MJ G

lob Health: first published as 10.1136/bm

jgh-2021-006804 on 4 October 2021. D

ownloaded from

Quach A, et al. BMJ Global Health 2021;6:e006804. doi:10.1136/bmjgh-2021-006804 13

BMJ Global Health

Per

cent

age

of

Qua

lity

Mea

sure

s as

sess

able

in e

ach

Qua

lity

Sta

tem

ent

WH

O- E

uro

pe

WH

O- S

E A

sia

SPA

HR

BF

HeR

AM

Sr-

HFA

SA

RA

HFS

- IM

CI

HC

AH

PS

- Chi

ld

Sta

tem

ent

1.9

All

child

ren

are

asse

ssed

and

che

cked

for

imm

unis

atio

n st

atus

and

rec

eive

ap

pro

pria

te v

acci

natio

ns

acco

rdin

g to

the

gui

del

ines

of t

he W

HO

exp

and

ed

pro

gram

me

on im

mun

isat

ion.

(n=

12)

17%

58%

100%

75%

42%

50%

58%

58%

0%

Sta

tem

ent

1.10

: All

child

ren

with

chr

onic

con

diti

ons

rece

ive

app

rop

riate

car

e, a

nd t

hey

and

the

ir fa

mili

es a

re

suffi

cien

tly in

form

ed a

bou

t th

eir

cond

ition

(s) a

nd a

re

sup

por

ted

to

optim

ise

thei

r he

alth

, dev

elop

men

t an

d

qua

lity

of li

fe. (

n=12

)

42%

0%33

%8%

17%

0%25

%0%

0%

Sta

tem

ent

1.11

All

child

ren

are

scre

ened

for

evid

ence

of

mal

trea

tmen

t, in

clud

ing

negl

ect

and

vio

lenc

e, a

nd r

ecei

ve

app

rop

riate

car

e. (n

=7)

29%

0%0%

0%29

%0%

0%0%

0%

Sta

tem

ent

1.12

: All

child

ren

with

sur

gica

l con

diti

ons

are

scre

ened

for

surg

ical

em

erge

ncie

s an

d in

jurie

s an

d

rece

ive

app

rop

riate

sur

gica

l car

e. (n

=12

)

33%

17%

17%

8%25

%8%

17%

0%0%

Sta

tem

ent

1.13

: All

sick

chi

ldre

n, e

spec

ially

tho

se

who

are

mos

t se

rious

ly il

l, ar

e ad

equa

tely

mon

itore

d,

reas

sess

ed p

erio

dic

ally

and

rec

eive

sup

por

tive

care

ac

cord

ing

to W

HO

gui

del

ines

. (n=

14)

93%

50%

36%

0%21

%7%

7%0%

0%

Sta

tem

ent

1.14

: All

child

ren

rece

ive

care

with

sta

ndar

d

pre

caut

ions

to

pre

vent

hea

lth c

are-

asso

ciat

ed in

fect

ions

. (n

=14

)

64%

50%

50%

57%

43%

21%

36%

7%0%

Sta

tem

ent

1.15

: All

child

ren

are

pro

tect

ed fr

om

unne

cess

ary

or h

arm

ful p

ract

ices

dur

ing

thei

r ca

re. (

n=12

)67

%33

%17

%0%

0%0%

0%8%

0%

Sta

ndar

d 2

: The

hea

lth

info

rmat

ion

syst

em e

nsur

es t

he c

olle

ctio

n, a

naly

sis

and

use

of

dat

a to

ens

ure

earl

y, a

pp

rop

riat

e ac

tio

n to

imp

rove

the

car

e o

f ev

ery

child

Sta

tem

ent

2.1:

Eve

ry c

hild

has

a c

omp

lete

, acc

urat

e,

stan

dar

dis

ed, u

p- t

o- d

ate

med

ical

rec

ord

, whi

ch is

ac

cess

ible

thr

ough

out

thei

r ca

re, o

n d

isch

arge

and

on

follo

w- u

p. (

n=13

)

62%

38%

23%

15%

0%23

%0%

8%0%

Sta

tem

ent

2.2:

Eve

ry h

ealth

faci

lity

has

a fu

nctio

nal

mec

hani

sm fo

r d

ata

colle

ctio

n, a

naly

sis

and

use

as

par

t of

its

activ

ities

for

mon

itorin

g p

erfo

rman

ce a

nd q

ualit

y im

pro

vem

ent.

(n=

9)

44%

67%

78%

56%

0%44

%0%

0%0%

Sta

tem

ent

2.3:

Eve

ry h

ealth

faci

lity

has

a m

echa

nism

fo

r co

llect

ing,

ana

lysi

ng a

nd p

rovi

din

g fe

edb

ack

on t

he

serv

ices

pro

vid

ed a

nd t

he p

erce

ptio

n of

chi

ldre

n an

d t

heir

fam

ilies

of t

he c

are

rece

ived

. (n=

7)

43%

57%

57%

57%

0%0%

0%29

%43

%

Tab

le 4

C

ontin

ued

Con

tinue

d

on Novem

ber 23, 2021 by guest. Protected by copyright.

http://gh.bmj.com

/B

MJ G

lob Health: first published as 10.1136/bm

jgh-2021-006804 on 4 October 2021. D

ownloaded from

14 Quach A, et al. BMJ Global Health 2021;6:e006804. doi:10.1136/bmjgh-2021-006804

BMJ Global Health

Per

cent

age

of

Qua

lity

Mea

sure

s as

sess

able

in e

ach

Qua

lity

Sta

tem

ent

WH

O- E

uro

pe

WH

O- S

E A

sia

SPA

HR

BF

HeR

AM

Sr-

HFA

SA

RA

HFS

- IM

CI

HC

AH

PS

- Chi

ld

Sta

ndar

d 3

: Eve

ry c

hild

wit

h co

ndit

ion(

s) t

hat

cann

ot

be

trea

ted

eff

ecti

vely

wit

h th

e av

aila

ble

res

our

ces

rece

ives

ap

pro

pri

ate,

tim

ely

refe

rral

, wit

h se

amle

ss

cont

inui

ty o

f ca

re

Sta

tem

ent

3.1:

Eve

ry c

hild

who

req

uire

s re

ferr

al r

ecei

ves

app

rop

riate

pre

refe

rral

car

e, a

nd t

he d

ecis

ion

to r

efer

is

mad

e w

ithou

t d

elay

. (n=

10)

30%

30%

30%

50%

40%

0%0%

20%

0%

Sta

tem

ent

3.2:

Eve

ry c

hild

who

req

uire

s re

ferr

al r

ecei

ves

seam

less

, coo

rdin

ated

car

e an

d r

efer

ral a

ccor

din

g to

a

pla

n th

at e

nsur

es t

imel

ines

s. (n

=12

)

33%

0%17

%42

%17

%8%

8%25

%0%

Sta

tem

ent

3.3:

For

eve

ry c

hild

ref

erre

d o

r co

unte

r-

refe

rred

with

in o

r am

ong

heal

th fa

cilit

ies,

the

re is

ap

pro

pria

te in

form

atio

n ex

chan

ge a

nd fe

edb

ack

to

rele

vant

hea

lthca

re s

taff.

(n=

6)

67%

33%

83%

17%

100%

17%

17%

0%0%

Sta

ndar

d 4

: Co

mm

unic

atio

n w

ith

child

ren

and

the

ir f

amili

es is

eff

ecti

ve, w

ith

mea

ning

ful p

arti

cip

atio

n an

d r

esp

ond

s to

the

ir n

eed

s an

d p

refe

renc

es

Sta

tem

ent

4.1

All

child

ren

and

the

ir ca

rers

are

giv

en

info

rmat

ion

abou

t th

e ch

ild’s

illn

ess

and

car

e ef

fect

ivel

y,

so t

hat

they

und

erst

and

and

cop

e w

ith t

he c

ond

ition

and

th

e ne

cess

ary

trea

tmen

t. (n

=13

)

85%

23%

38%

31%

0%15

%0%

23%

69%

Sta

tem

ent

4.2

All

child

ren

and

the

ir ca

rers

exp

erie

nce

coor

din

ated

car

e, w

ith c

lear

, acc

urat

e in

form

atio

n ex

chan

ge a

mon

g re

leva

nt h

ealth

and

soc

ial c

are

pro

fess

iona

ls a

nd o

ther

sta

ff. (n

=9)

22%

11%

11%

11%

0%0%

0%0%

0%

Sta

tem

ent

4.3

All

child

ren

and

the

ir ca

rers

are

ena

ble

d t

o p

artic

ipat

e ac

tivel

y in

the

chi

ld’s

car

e, in

dec

isio

n m

akin

g,

in e

xerc

isin

g th

e rig

ht t

o in

form

ed c

onse

nt a

nd in

mak

ing

choi

ces,

in a

ccor

dan

ce w

ith t

heir

evol

ving

cap

acity

. (n

=10

)

80%

10%

10%

20%

0%0%

0%0%

20%

Sta

tem

ent

4.4

All

child

ren

and

the

ir ca

rers

rec

eive

ap

pro

pria

te c

ouns

ellin

g an

d h

ealth

ed

ucat

ion,

acc

ord

ing

to t

heir

cap

acity

, ab

out

the

curr

ent

illne

ss a

nd p

rom

otio

n of

the

chi

ld’s

hea

lth a

nd w

ell-

bei

ng. (

n=13

)

23%

38%

46%

31%

31%

8%0%

31%

8%

Sta

ndar

d 5

: Eve

ry c

hild

’s r

ight

s ar

e re

spec

ted

, pro

tect

ed a

nd f

ulfi

lled

at

all t

imes

dur

ing

car

e, w

itho

ut d

iscr

imin

atio

n

Sta

tem

ent

5.1

All

child

ren

have

the

rig

ht t

o ac

cess

he

alth

care

and

ser

vice

s, w

ith n

o d

iscr

imin

atio

n of

any

ki

nd. (

n=9)

78%

11%

44%

22%

11%

0%0%

11%

0%

Tab

le 4

C

ontin

ued

Con

tinue

d

on Novem

ber 23, 2021 by guest. Protected by copyright.

http://gh.bmj.com

/B

MJ G

lob Health: first published as 10.1136/bm

jgh-2021-006804 on 4 October 2021. D

ownloaded from

Quach A, et al. BMJ Global Health 2021;6:e006804. doi:10.1136/bmjgh-2021-006804 15

BMJ Global Health

Per

cent

age

of

Qua

lity

Mea

sure

s as

sess

able

in e

ach

Qua

lity

Sta

tem

ent

WH

O- E

uro

pe

WH

O- S

E A

sia

SPA

HR

BF

HeR

AM

Sr-

HFA

SA

RA

HFS

- IM

CI

HC

AH

PS

- Chi

ld

Sta

tem

ent

5.2

All

child

ren

and

the

ir ca

rers

are

mad

e aw

are

of a

nd g

iven

info

rmat

ion

abou

t ch

ildre

n’s

right

s to

he

alth

and

hea

lthca

re. (

n=9)

44%

0%0%

0%0%

0%0%

0%0%

Sta

tem

ent

5.3

All

child

ren

and

the

ir ca

rers

are

tre

ated

w

ith r

esp

ect

and

dig

nity

, and

the

ir rig

ht t

o p

rivac

y an

d

confi

den

tialit

y is

res

pec

ted

. (n=

7)

57%

14%

43%

43%

0%14

%14

%0%

57%

Sta

tem

ent

5.4

All

child

ren

are

pro

tect

ed fr

om a

ny

viol

atio

n of

the

ir hu

man

rig

hts,

phy

sica

l or

men

tal

viol

ence

, inj

ury,

ab

use,

neg

lect

or

any

othe

r fo

rm o

f m

altr

eatm

ent.

(n=

9)

11%

0%0%

0%22

%0%

0%0%

0%

Sta

tem

ent

5.5

All

child

ren

have

acc

ess

to s

afe,

ad

equa

te

nutr

ition

tha

t is

ap

pro

pria

te fo

r b

oth

thei

r ag

e an

d t

heir

heal

th c

ond

ition

dur

ing

thei

r ca

re in

a fa

cilit

y. (n

=10

)

40%

30%

20%

10%

0%10

%10

%0%

0%

Sta

ndar

d 6

: All

child

ren

and

the

ir f

amili

es a

re p

rovi

ded

wit

h ed

ucat

iona

l, em

oti

ona

l and

psy

cho

soci

al s

upp

ort

tha

t is

sen

siti

ve t

o t

heir

nee

ds

and

str

eng

then

s th

eir

cap

abili

ty

Sta

tem

ent

6.1

All

child

ren

are

allo

wed

to

be

with

the

ir ca

rers

, and

the

rol

e of

car

ers

is r

ecog

nise

d a

nd s

upp

orte

d

at a

ll tim

es d

urin

g ca

re, i

nclu

din

g ro

omin

g- in

dur

ing

the

child

’s h

osp

italis

atio

n. (n

=9)

78%

22%

0%0%

0%0%

0%0%

11%

Sta

tem

ent

6.2

All

child

ren

and

the

ir fa

mili

es a

re g

iven

em

otio

nal s

upp

ort

that

is s

ensi

tive

to t

heir

need

s, w

ith

opp

ortu

nitie

s fo

r p

lay

and

lear

ning

tha

t st

imul

ate

and

st

reng

then

the

ir ca

pab

ility

. (n=

10)

60%

10%

0%0%

0%0%

0%0%

20%

Sta

tem

ent

6.3

Eve

ry c

hild

is a

sses

sed

rou

tinel

y fo

r p

ain

or s

ymp

tom

s of

dis

tres

s an

d r

ecei

ves

app

rop

riate

m

anag

emen

t ac

cord

ing

to W

HO

gui

del

ines

.(n

=13

)

62%

0%8%

0%0%

0%0%

0%23

%

Sta

ndar

d 7

: Fo

r ev

ery

child

, co

mp

eten

t, m

oti

vate

d, e

mp

athi

c st

aff

are

cons

iste

ntly

ava

ilab

le t

o p

rovi

de

rout

ine

care

and

man

agem

ent

of

com

mo

n ch

ildho

od

ill

ness

es

Sta

tem

ent

7.1

All

child

ren

and

the

ir fa

mili

es h

ave

acce

ss

at a

ll tim

es t

o su

ffici

ent

heal

th p

rofe

ssio

nals

and

sup

por

t st

aff f

or r

outin

e ca

re a

nd m

anag

emen

t of

chi

ldho

od

illne

sses

. (n=

15)

67%

27%

40%

53%

0%13

%0%

13%

7%

Sta

tem

ent

7.2

Hea

lth p

rofe

ssio

nals

and

sup

por

t st

aff h

ave

the

app

rop

riate

ski

lls t

o fu

lfil t

he h

ealth

, p

sych

olog

ical

, dev

elop

men

tal,

com

mun

icat

ion

and

cu

ltura

l nee

ds

of c

hild

ren.

(n=

17)

76%

47%

65%

71%

6%35

%18

%24

%6%

Tab

le 4

C

ontin

ued

Con

tinue

d

on Novem

ber 23, 2021 by guest. Protected by copyright.

http://gh.bmj.com

/B

MJ G

lob Health: first published as 10.1136/bm

jgh-2021-006804 on 4 October 2021. D

ownloaded from

16 Quach A, et al. BMJ Global Health 2021;6:e006804. doi:10.1136/bmjgh-2021-006804

BMJ Global Health

Per

cent

age

of

Qua

lity

Mea

sure

s as

sess

able

in e

ach

Qua

lity

Sta

tem

ent

WH

O- E

uro

pe

WH

O- S

E A

sia

SPA

HR

BF

HeR

AM

Sr-

HFA

SA

RA

HFS

- IM

CI

HC

AH

PS

- Chi

ld

Sta

tem

ent

7.3

Eve

ry h

ealth

faci

lity

has

man

ager

ial

lead

ersh

ip t

hat

colle

ctiv

ely

dev

elop

s, im

ple

men

ts a

nd

mon

itors

ap

pro

pria

te p

olic

ies

and

lega

l ent

itlem

ents

th

at fo

ster

an

envi

ronm

ent

for

cont

inuo

us q

ualit

y im

pro

vem

ent.

(n=

12)

67%

25%

50%

50%

0%17

%0%

8%8%

Sta

ndar

d 8

: The

hea

lth

faci

lity

has

an a

pp

rop

riat

e, c

hild

fri

end

ly p

hysi

cal e

nvir

onm

ent,

wit

h ad

equa

te w

ater

, san

itat

ion,

was

te m

anag

emen

t, e

nerg

y su

pp

ly,

med

icin

es, m

edic

al s

upp

lies

and

eq

uip

men

t fo

r ro

utin

e ca

re a

nd m

anag

emen

t o

f co

mm

on

child

hoo

d il

lnes

ses.

Sta

tem

ent

8.1

Chi

ldre

n ar

e ca

red

for

in a

wel

l-

mai

ntai

ned

, saf

e, s

ecur

e p

hysi

cal e

nviro

nmen

t w

ith

an a

deq

uate

ene

rgy

sup

ply

and

whi

ch is

ap

pro

pria

tely

d

esig

ned

, fur

nish

ed a

nd d

ecor

ated

to

mee

t th

eir

need

s,

pre

fere

nces

and

dev

elop

men

tal a

ge. (

n=15

)

40%

47%

40%

33%

20%

13%

33%

0%20

%

Sta

tem

ent

8.2

Chi

ld- f

riend

ly w

ater

, san

itatio

n, h

and

hy

gien

e an

d w

aste

dis

pos

al fa

cilit

ies

are

easi

ly

acce

ssib

le, f

unct

iona

l, re

liab

le, s

afe

and

suf

ficie

nt t

o m

eet

the

need

s of

chi

ldre

n, t

heir

care

rs a

nd s

taff.

(n=

22)

68%

59%

41%

45%

41%

18%

18%

0%0%

Sta

tem

ent

8.3

Chi

ld- f

riend

ly, a

ge- a

pp

rop

riate

eq

uip

men

t d

esig

ned

to

mee

t ch

ildre

n’s

need

s in

med

ical

car

e,

lear

ning

, rec

reat

ion

and

pla

y ar

e av

aila

ble

at

all t

imes

. (n

=15

)

40%

27%

27%

27%

33%

7%33

%0%

13%

Sta

tem

ent

8.4

Ad

equa

te s

tock

s of

chi

ld- f

riend

ly

med

icin

es a

nd m

edic

al s

upp

lies

are

avai

lab

le fo

r th

e ro

utin

e ca

re a

nd m

anag

emen

t of

acu

te a

nd c

hron

ic

child

hood

illn

esse

s an

d c

ond

ition

s. (n

=14

)

71%

64%

57%

50%

43%

43%

50%

14%

0%

Red

box

es in

dic

ate

that

the

ass

essm

ent

tool

did

not

ass

ess

any;

ora

nge

box

es in

dic

ate

less

tha

n ha

lf; y

ello

w b

oxes

ind

icat

e eq

ual t

o or

mor

e th

an h

alf a

nd g

reen

box

es in

dic

ate

all o

f the

q

ualit

y m

easu

res

wer

e as

sess

able

in t

he a

ssoc

iate

d q

ualit

y st

atem

ent.

HC

AH

PS

, Hos

pita

l Con

sum

er A

sses

smen

t of

Hea

lthca

re P

rovi

der

s an

d S

yste

ms;

HeR

AM

S, H

ealth

Res

ourc

es A

vaila

bili

ty M

app

ing

Sys

tem

; HFS

- IM

CI,

Hea

lth F

acili

ty S

urve

y—us

ing

Inte

grat

ed

Man

agem

ent

of C

hild

hood

Illn

ess

clin

ical

gui

del

ines

; HR

BF,

Hea

lth R

esul

ts B

ased

Fin

anci

ng im

pac

t ev

alua

tion

tool

kit;

n, n

umb

er o

f Qua

lity

Mea

sure

s w

ithin

the

ass

ocia

ted

Qua

lity

Sta

tem

ent;

r-

HFA

, rap

id H

ealth

Fac

ility

Ass

essm

ent;

SA

RA

, Ser

vice

Ava

ilab

ility

and

Rea

din

ess

Ass

essm

ent;

SPA

, Ser

vice

Pro

visi

on A

sses

smen

t.;

Tab

le 4

C

ontin

ued

on Novem

ber 23, 2021 by guest. Protected by copyright.

http://gh.bmj.com

/B

MJ G

lob Health: first published as 10.1136/bm

jgh-2021-006804 on 4 October 2021. D

ownloaded from

Quach A, et al. BMJ Global Health 2021;6:e006804. doi:10.1136/bmjgh-2021-006804 17

BMJ Global Health

The purpose and context of the assessment tools needs to be considered when examining how comprehensive they are in comparison to the WHO Standards. The WHO Hospital Care assessment tools were found to be more comprehensive in their ability to capture the WHO Stan-dards. These tools were first developed in 2001 to provide government and stakeholders guidance in performing evaluation of quality of healthcare practices in order to identify key areas to improve on.17 They have since been revised multiple times and adapted to multiple settings with the most recent European version including indica-tors for child rights, communication and alignment with evidence- based practice as outlined in the WHO Pocket Book of Hospital Care for Children.18 The WHO Stan-dards for improving QoC have been modelled on similar frameworks, which may explain the overlap of indicators between the WHO Hospital Care assessment tools and the WHO Standards.

Of the three broad arms of QoC in the WHO Stan-dards: provision of care (QSd 1–3) and available human and physical resources (QSd 7–8), were more widely covered by the tools than experience of care (QSd 4–6). This may be because provision of care and system inputs have more definitive QM, which make them amenable to be assessed through checklist style questionnaires. The SPA and Service Availability and Readiness Assessment were developed to provide nationwide data on the capac-ities of health facilities to provide quality services and have been used in over 30 countries.10 19 These surveys are designed to be repeated at periodic intervals, every 1–5 years, to monitor progress and inform development of national health policies and programmes. It is there-fore not surprising that their strengths lie in assessing availability of concrete measures such as physical infra-structure and human resources, with less emphasis on subjective components such as communication and emotional support. The HeRAMS similarly was developed to collect information on availability of health resources and services. Designed to be implemented in humani-tarian and emergency response settings, rapid reporting is essential to its purpose in order to obtain supplies and resources required for basic healthcare needs.

The child- specific assessment tools such as the WHO hospital tools, HFS- IMCI and Child- HCAHPS, had relative strengths in assessing experience of care when compared with the remaining tools. This may be in recognition that communicating with parents/carers is a large component of paediatric healthcare. The Child- HCAHPS survey was developed for the sole purpose of obtaining parent/guardian feedback on their experience of care of their child in hospital. Health service delivery has traditionally revolved around disease diagnosis and treatment. However, there has been a gradual global shift towards integrated people- centred health services, where people and communities are seen as active participants as well as beneficiaries of their responsive health systems. In 2016, the WHO adopted the ‘Framework on integrated people- centred health services’ to help drive change in

national policies on health services delivery to include cross- sectoral collaboration and community involve-ment and empowerment in decision- making processes.20 Involving people in their own care, especially marginal-ised subpopulations, is considered essential to achieving equitable access and QoC towards UHC. So, although tools such as the Child- HCAHPS may not be a suitable tool to assess all aspects of QoC for children, it can be a useful adjunctive tool to assess communication skills and patient experiences of care, which may otherwise be lacking in existing quality assessment frameworks.

Feasibility was not formally assessed as part of this review. The more comprehensive WHO Hospital Care Assessment tools are extensive and labour intensive and yet only cover about half the QM in the WHO Standards. Our search only identified two original peer- reviewed publications that used the WHO Hospital Care Assess-ment Tools.21 22 It is possible that the tools are used for auditing and quality improvement practices with infor-mation only being disseminated within country. However, there is little anecdotal evidence of this occurring. Without clear quality frameworks in place and limited resources, health facilities would likely find it challenging to implement these assessment tools effectively. In order for LMICs to effectively implement quality improvement practices and to sustainably assess and monitor them, key indicators need to be clear and manageable. We recom-mend that the QM in the WHO Standards be simplified and that key indicators to monitor in each QSd be high-lighted. Key indicators should obtain global consensus and adhere to a measurement framework to ensure that they are relevant, acceptable, achievable and robust. Existing core assessment tools could then be combined and simplified, to incorporate the key indicators, with flexibility for other QM to be included as prioritised by individual health facilities. This will be more achiev-able and constructive at the local level, while assisting in reporting of national progress of uniform child health indicators in LMICs.

An alternative to using explicit QoC assessment tools to evaluate and monitor quality improvement prac-tices, would be to have key indicators embedded in other routine data collection systems. In the USA, there are existing monitoring systems of QMs embedded in HIS. Data are collated from multiple databases by such agencies as the Agency for Healthcare Research and Quality to produce annual National Healthcare Quality and Disparities Reports which include indica-tors overlapping with many of the WHO Standards.23 In LMICs, paper- based surveys and medical records have traditionally been the main way to collect health data. Routine HIS which could potentially monitor key indicators for QoC can be variable in levels of data recording and quality, and are seldom used to evaluate programme interventions and policy changes often due to lack of capacity.24 25 The introduction of elec-tronic health records and platforms are emerging as more reliable sources of data management and analyses

on Novem

ber 23, 2021 by guest. Protected by copyright.

http://gh.bmj.com

/B

MJ G

lob Health: first published as 10.1136/bm

jgh-2021-006804 on 4 October 2021. D

ownloaded from

18 Quach A, et al. BMJ Global Health 2021;6:e006804. doi:10.1136/bmjgh-2021-006804

BMJ Global Health

in LMICs.26 27 However, a range of challenges has meant that electronic- HIS have not yet replaced paper- based records or survey tools in most LMIC settings.28–31 Until routine HIS become more robust and reliable, using existing, comprehensive, low- cost tools to assess QoC, will continue to more feasible.

Our review focused on assessing QoC for children and young adolescents. However, it is important to recog-nise that quality healthcare is required throughout the continuum of the life course, including the antenatal and perinatal periods, to ensure improved outcomes for all children. The WHO Standards for children and adolescents are an extension of the WHO Standards for maternal and newborn care in health facilities, with both developed using the same framework. A previous review comparing existing assessment tools with the WHO Stan-dards for maternal and newborn care drew similar conclu-sions to our assessment—that current tools had gaps in assessing experience of care and that there should be global consensus on core data to be collected.6 Although each life stage has its own unique healthcare needs (eg, obstetric care for women; immunisations for children), comparable themes for quality healthcare practices are applicable across the life course. Having similar frame-works to assess and monitor quality of healthcare across the life course would make quality improvement prac-tices and assessment tools easier to develop. It would also foster collaboration between health sectors in the development of common goals towards achieving better health outcomes for all.

Our systematic review had several limitations. In the matching process, clinical judgement deter-mined whether questions/items from assessment tools matched the QM in the WHO Standards. This subjective process could have led to bias in some indicators. Our selection criteria were aimed at identifying stand- alone assessment tools. This may have inadvertently excluded quality assessment tools integrated within other health data collection activities. We also only included English publications, which may have excluded other existing tools used in non- English speaking countries. Our review only evaluated the survey instruments and did not assess feasibility of implementation, which would include preplanning, training, supervision, evaluation and feedback of data. These pragmatic factors would affect the capacity of a tool to reliably assess the WHO Standards and would need to be considered in future activities assessing QoC.

CONCLUSIONThis review found that although the WHO Standards are comprehensive, no single tool can adequately assess all the QM in its current format. Furthermore, operational use of extensive assessment tools is seldom seen due to lack of resources and organisational frame-works. Existing tools tend to emphasise input measures and few tools adequately assess experience of care.

Consensus and harmonisation of select key indicators from the WHO Standards, integrated into simplified assessment tools would make them more achievable in LMICs. Comparable data on key indicators for moni-toring within and between countries will also assist in national and global reporting on progress of child health outcomes. Further research into the feasibility of modified tools with key indicators to assess QoC and the impact on health outcomes, is, therefore, an impor-tant next step in establishing equitable access to quality healthcare.

Author affiliations1Department of Paediatrics, The University of Melbourne Faculty of Medicine Dentistry and Health Sciences, Melbourne, Victoria, Australia2Asia Pacific Health Group, Murdoch Childrens Research Institute, Parkville, Victoria, Australia3The Royal Children's Hospital Melbourne, Parkville, Victoria, Australia4Health System Strengthening Unit, World Health Organisation Country Office for Indonesia, Jakarta, Indonesia5International Child Health Group, Murdoch Childrens Research Institute, Parkville, Victoria, Australia6Maternal, Newborn, Child and Adolescent Health and Ageing Department, World Health Organization, Geneva, Switzerland

Twitter Alicia Quach @QuachAlicia

Contributors AQ conceived the study, wrote the first draft and is the guarantor of the manuscript. ST and HN were second reviewers for screening and data analysis. FMR and SMG provided feedback on the first draft of the manuscript. All authors including TD, WMW and MM provided feedback on subsequent versions of the manuscript and approved the final version.

Funding The authors have not declared a specific grant for this research from any funding agency in the public, commercial or not- for- profit sectors.

Competing interests We declare no competing interests. The authors alone are responsible for the views expressed in this article and they do not necessarily represent the views, decisions or policies of the institutions with which they are affiliated.

Patient consent for publication Not applicable.

Ethics approval Patient consent and ethics approval for publication not required.

Provenance and peer review Not commissioned; externally peer reviewed.

Data availability statement Data are available on reasonable request.

Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer- reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.

Open access This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY- NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non- commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non- commercial. See: http:// creativecommons. org/ licenses/ by- nc/ 4. 0/.

REFERENCES 1 Sustainable development goals knowledge platform New York:

United nations, 2015. Available: https:// sust aina bled evel opment. un. org/ sdgs [Accessed 30 Mar 2020].

2 World Health Organization. Quality of care : a process for making strategic choices in health systems. Geneva: World Health Organization, 2006.

on Novem

ber 23, 2021 by guest. Protected by copyright.

http://gh.bmj.com

/B

MJ G

lob Health: first published as 10.1136/bm

jgh-2021-006804 on 4 October 2021. D

ownloaded from

Quach A, et al. BMJ Global Health 2021;6:e006804. doi:10.1136/bmjgh-2021-006804 19

BMJ Global Health

3 Tunçalp Ӧ, Were WM, MacLennan C, et al. Quality of care for pregnant women and newborns- the who vision. BJOG 2015;122:1045–9.

4 World Health Organization. Standards for improving the quality of care for children and young adolescents in health facilities. Geneva: World Health Organization, 2018.

5 World Health Organization. Standards for improving quality of maternal and newborn care in health facilities. Geneva: World Health Organization, 2016.

6 Brizuela V, Leslie HH, Sharma J, et al. Measuring quality of care for all women and newborns: how do we know if we are doing it right? A review of facility assessment tools. Lancet Glob Health 2019;7:e624–32.

7 Moher D, Liberati A, Tetzlaff J, et al. Preferred reporting items for systematic reviews and meta- analyses: the PRISMA statement. PLoS Med 2009;6:e1000097.

8 Covidence systematic review software: veritas health innovation, Melbourne, Australia. Available: www. covidence. org [Accessed 01 Oct 2019].

9 CAHPS Hospital child survey: agency for healthcare research and quality. Available: https://www. ahrq. gov/ cahps/ surveys- guidance/ hospital/ about/ child_ hp_ survey. html [Accessed 0Feb 2020].

10 Service provision assessments: the DHS program. Available: https:// dhsprogram. com/ Methodology/ Survey- Types/ SPA. cfm [Accessed 02 Feb 2020].

11 World Health Organization Regional Office for Europe. Hospital care for children: quality assessment and improvement tool. Available: https://www. euro. who. int/__ data/ assets/ pdf_ file/ 0003/ 286185/ Hospital- care- for- children- quality- assessment- and- improvement- tool. pdf [Accessed 02 Feb 2020].

12 World Health Organization Regional Office for South East Asia. Assessment tool for hospital care: improving the quality of care for reproductive, maternal, neonatal, child and adolescent health in south- east Asia. Available: https:// apps. who. int/ iris/ handle/ 10665/ 279767 [Accessed 02 Feb 2020].

13 Dettrick Z, Firth S, Jimenez Soto E. Do strategies to improve quality of maternal and child health care in lower and middle income countries lead to improved outcomes? A review of the evidence. PLoS One 2013;8:e83070.

14 Powell- Jackson T, Acharya A, Mills A. An assessment of the quality of primary health care in India. Economic and Political Weekly 2013;48:53–61.

15 Leslie HH, Sun Z, Kruk ME. Association between infrastructure and observed quality of care in 4 healthcare services: a cross- sectional study of 4,300 facilities in 8 countries. PLoS Med 2017;14:e1002464.

16 Donabedian A. The quality of care. How can it be assessed? JAMA 1988;260:1743–8.

17 Nolan T, Angos P, Cunha AJ, et al. Quality of hospital care for seriously ill children in less- developed countries. Lancet 2001;357:106–10.

18 World Health Organization. Pocket book of hospital care for children: guidelines for the management of common childhood illnesses. 2nd edition. Geneva: World Health Organization, 2013.

19 World Health Organization. Health statistics and information systems: service availability and readiness assessment (SARA). Available: https://www. who. int/ healthinfo/ systems/ sara_ introduction/ en/ [Accessed 02 Feb 2020].

20 World Health Organization. Framework on integrated people- centred health services. Geneva: World Health Organization, 2016. https://www. who. int/ serv iced eliv erys afety/ areas/ people- centred- care/ framework/ en/

21 Sidik NA, Lazuardi L, Agung FH, et al. Assessment of the quality of hospital care for children in Indonesia. Trop Med Int Health 2013;18:407–15.

22 Lazzerini M, Shukurova V, Davletbaeva M, et al. Improving the quality of hospital care for children by supportive supervision: a cluster randomized trial, Kyrgyzstan. Bull World Health Organ 2017;95:397–407.

23 U.S. Department of Health and Human Services, Agency for Healthcare Research and Quality. National healthcare quality and disparities report. Rockville, MD, 2018.

24 Wagenaar BH, Sherr K, Fernandes Q, et al. Using routine health information systems for well- designed health evaluations in low- and middle- income countries. Health Policy Plan 2016;31:129–35.

25 Hung YW, Hoxha K, Irwin BR, et al. Using routine health information data for research in low- and middle- income countries: a systematic review. BMC Health Serv Res 2020;20:790.

26 Khubone T, Tlou B, Mashamba- Thompson TP. Electronic health information systems to improve disease diagnosis and management at point- of- care in low and middle income countries: a narrative review. Diagnostics 2020;10:327.

27 Kiberu VM, Matovu JKB, Makumbi F, et al. Strengthening district- based health reporting through the district health management information software system: the Ugandan experience. BMC Med Inform Decis Mak 2014;14:40.

28 Dornan L, Pinyopornpanish K, Jiraporncharoen W, et al. Utilisation of electronic health records for public health in Asia: a review of success factors and potential challenges. Biomed Res Int 2019;2019:1–9.

29 Fraser HS, Blaya J. Implementing medical information systems in developing countries, what works and what doesn't. AMIA Annu Symp Proc 2010;2010:232–6.

30 Labrique AB, Wadhwani C, Williams KA, et al. Best practices in scaling digital health in low and middle income countries. Global Health 2018;14:103.

31 Dehnavieh R, Haghdoost A, Khosravi A, et al. The district health information system (DHIS2): a literature review and meta- synthesis of its strengths and operational challenges based on the experiences of 11 countries. Health Inf Manag 2019;48:62–75.

32 The rapid health facility assessment (R- HFA): USAID: maternal and child health integrated program. Available: https://www. mchip. net/ technical- resource/ the- rapid- health- facility- assessment- r- hfa/ [Accessed 02 Feb 2020].

33 World Health Organization. Maternal, newborn, child and adolescent health: Health facility survey - tool to evaluate the quality of care delivered to sick children attending outpatient facilities. Available: https://www. who. int/ maternal_ child_ adolescent/ documents/ 9241545860/ en/ [Accessed 02 Feb 2020].

34 World Health Organization. Health resources and services availability monitoring system (HeRAMS), 2020. Available: https://www. who. int/ initiatives/ herams [Accessed 02 Feb 2020].

35 RBF Health. Impact evaluation toolkit. Available: https://www. rbfhealth. org/ resource/ impact- evaluation- toolkit- provides- hands- guidance [Accessed 02 Feb 2020]. on N

ovember 23, 2021 by guest. P

rotected by copyright.http://gh.bm

j.com/

BM

J Glob H

ealth: first published as 10.1136/bmjgh-2021-006804 on 4 O

ctober 2021. Dow

nloaded from