marquis - methods of assessing response to quality improvement strategies marquis: wp4: the...
TRANSCRIPT
MA
RQ
uIS
-M
eth
ods
of
Ass
ess
ing R
esp
on
se
to
Qu
ali
t y I
mpr o
vem
en
t S
trate
gie
s
MARQuIS:WP4: The Questionnaire
Niek Klazinga
Kiki Lombarts
Ines Rupp
MA
RQ
uIS
-M
eth
ods
of
Ass
ess
ing R
esp
on
se
to
Qu
ali
t y I
mpr o
vem
en
t S
trate
gie
s ObjectivesThe strategic objective of work package 4 is to describe in a sample of EU member states how hospitals have applied national quality strategies, to what extend they meet the defined requirements of cross-border patient care and what organizational and/or methodological variables are associated with (non-)compliance of requirements.
Further, WP4 is to validate the statistical results provided by WP3 concerning types and quantity of health services used by patients from another European member state.
MA
RQ
uIS
-M
eth
ods
of
Ass
ess
ing R
esp
on
se
to
Qu
ali
t y I
mpr o
vem
en
t S
trate
gie
s WP 4: Questionnaire Purpose
• measure quality improvement: • the application of Q policies and procedures,
Q governance structure as well as Q activities, used to close the gap between current and expected levels of quality
• measure quality: • quantifying the current levels of performance
or compliance with expected requirements.
MA
RQ
uIS
-M
eth
ods
of
Ass
ess
ing R
esp
on
se
to
Qu
ali
t y I
mpr o
vem
en
t S
trate
gie
s Three main phases
1. Defining a concrete set of measures (development of a questionnaire) to identify quality strategies in hospitals and quality requirement for hospital care delivered to EU cross-border patients
2. Collecting data through a questionnaire survey in an ad random sample of hospitals in 8 EU member states
3. Analysing and reporting the data
MA
RQ
uIS
-M
eth
ods
of
Ass
ess
ing R
esp
on
se
to
Qu
ali
t y I
mpr o
vem
en
t S
trate
gie
s Focus of the questionnaire in
measuring quality (improvement) in hospitals
Strategic level, Hospital-wide policies,
structures and activities
Tactic level, hospital (supporting)
services, policies, structures
and activities
Operational level, Management of patients
suffering from AMI
Operational level, Management of patients
suffering from acute appendicitis
Operational level, Management of women
during pregnancy and labor
MA
RQ
uIS
-M
eth
ods
of
Ass
ess
ing R
esp
on
se
to
Qu
ali
t y I
mpr o
vem
en
t S
trate
gie
s
Section S1 S2 S3 S4
Numberof questions 63 42 48 46
Number of items appr. 500 in total
QuestionnaireDesign
MA
RQ
uIS
-M
eth
ods
of
Ass
ess
ing R
esp
on
se
to
Qu
ali
t y I
mpr o
vem
en
t S
trate
gie
s Recruitment & response
Hospitals visiting quest.
Hospitals responded
UK 41 14
Ireland 29 25
Neth 12 10
Belgium 33 25
France 100 78
Spain 131 113
Poland 84 80
Czech Rep 53 44
Total 483 389
MA
RQ
uIS
-M
eth
ods
of
Ass
ess
ing R
esp
on
se
to
Qu
ali
t y I
mpr o
vem
en
t S
trate
gie
s Challenges for analyses
• Frequency distribution answers• Testing of hypotheses• Contributing to the audit tool• Developing a QI-maturity
classification for• sampling for the audit
• testing of hypotheses
MA
RQ
uIS
-M
eth
ods
of
Ass
ess
ing R
esp
on
se
to
Qu
ali
t y I
mpr o
vem
en
t S
trate
gie
s QI-maturity classificationPolicy, planning, documents 20 items α=0.86
Leadership 36 items α=0.89
Structure 19 items α=0.69
QI activities 8 items α=0.75
QI activities (labs) 20 items α=0.85
Patient involvement 6 items α=0.82
Accountability 4 items α=0.54
→ scores combined in an overall classification score
MA
RQ
uIS
-M
eth
ods
of
Ass
ess
ing R
esp
on
se
to
Qu
ali
t y I
mpr o
vem
en
t S
trate
gie
s
Hospital
response
Hospitals in overall QI classification
Belgium 25 24
Czech Republic 44 38
France 78 65
Ireland 25 23
The Netherlands 10 8
Poland 80 76
Spain 113 105
UK 14 10
Total 389 349
Response QI-maturity classification
MA
RQ
uIS
-M
eth
ods
of
Ass
ess
ing R
esp
on
se
to
Qu
ali
t y I
mpr o
vem
en
t S
trate
gie
s Variance overall
classification scoremin. max.
Belgium (n=24) 2.10 - 3.10
Czech Republic (n=38) 1.50 - 3.00
France (n=65) 1.87 - 3.25
Ireland (n=23) 1.90 - 3.08
The Netherlands (n=8) 2.09 - 2.52
Poland (n=76) 1.86 - 3.45
Spain (n=105) 1.70 - 3.26
UK (n=10) 1.77 - 2.37
Total (n=349) 1.50 - 3.45
MA
RQ
uIS
-M
eth
ods
of
Ass
ess
ing R
esp
on
se
to
Qu
ali
t y I
mpr o
vem
en
t S
trate
gie
s Hospital distribution per
country based on the classification score
Belgium 3 16 5
Czech Republic 10 22 6
France 11 32 22
Ireland 8 12 3
The Netherlands 4 4 -
Poland 22 33 21
Spain 20 55 30
UK 9 1 -
Total 87 175 87*most mature: ≤25e percentile; least mature: > 75e percentile (entire sample)
“least mature”*“most mature”*
MA
RQ
uIS
-M
eth
ods
of
Ass
ess
ing R
esp
on
se
to
Qu
ali
t y I
mpr o
vem
en
t S
trate
gie
s
WP 5: Hospital audits• On-site hospital audits
• Verify questionnaire responses• Obtain details of quality methods
• 105 hospitals, 8 countries• Recruiting hospitals that have returned WP4
questionnaire• Sample based on QI-maturity classification
(most / least mature)• Belgium 6, Czech Republic 15, France 21,
Ireland 8, The Netherlands 4, Poland 15, Spain 30, United Kingdom 6
MA
RQ
uIS
-M
eth
ods
of
Ass
ess
ing R
esp
on
se
to
Qu
ali
t y I
mpr o
vem
en
t S
trate
gie
s Further validation QI-maturity classification
• Reliability study• Consistency over dimensions
• Exploring the “QI-index” in other datasets (e.g., PATH)
• Validity study• Comparison with maturity classification
based on audit
• Linking to data on accreditation / performance indicators (e.g. France, Spain)