Postgraduate Course
7. Evidence-based management:
Research designs
Postgraduate Course
Empirical circle
Postgraduate Course
Research designs: 4 elements
Randomization
Comparison
Before vs after
Prospective vs retrospective
Postgraduate Course
Randomized controlled study
Controlled before-after study
Cohort / panel study
Before-after study
Post-test only study
Survey
Case study
Experiment
Quasi-experiment
Observational study
Longitudinal study
Cross-sectional study
Qualitative study
Research designs: terms
Postgraduate Course
CBA: controlled before-after study
In a controlled before-after study two or more
groups are compared with each other, usually
comprising one group in which an intervention is
carried out (experimental group) and one group
where no or an alternative intervention is
conducted (control group).
Postgraduate Course
=
CBA: controlled before-after study
Postgraduate Course
In the case of randomization, the groups compared with each
other are selected entirely randomly, for example by drawing lots.
This means that each participant (or other unit such as a team,
department or company) has an equal chance of being in the
intervention or control group. In this way, the influence of any
distorting factors is spread over both groups so that these groups
are as comparable as possible with each other with the exception
of the intervention.
Better (but not always feasible): randomization
Postgraduate Course
=
even
oneven
Randomized controlled study (RCT)
Postgraduate Course
Cohort / panel study
Starting point: intervention/exposure (independent variable)
Longitudinal study where large groups of people or companies (cohort / panel) are followed over a long period to see (prospective) whether differences occur among the groups.
=
Postgraduate Course
Case-control study
Starting point: outcome (dependent variable)
Longitudinal study in which one group of people or companies with a particular outcome is compared retrospectively with a group that does not have this outcome.
=
✔
✗
VS
VS
Postgraduate Course
Examples of (flawed) case-control studies
Postgraduate Course
versus
Study in which data are obtained or particular
characteristics are measured of a population before
versus after an intervention / exposure / event, to
measure the effect or correlation.
Before-after study
Postgraduate Course
=
Posttest-only
Postgraduate Course
Cross-sectional study
Study in which data of a statistically significant sample of
a population (managers, CEO’s, employees) is gathered
at one point in time. It provides a snapshot of the current
condition but does not explain cause and effect.
Cross-sectional studies
include surveys
Postgraduate Course
Cross-sectional study
?
?
?
Postgraduate Course
Study in which a large number (qualitative or quantitative)
of aspects of a single case (organization or team) was
investigated in depth over a long period within the case’s
own context.
Case study
Postgraduate Course
Experimental research
‘True’ experiment: RCT
‘Quasi’ experiment: CBA
Experimental research refers to studies where the
researcher manipulates one (or more) variable and
controls the other variable(s) to determine whether there
is a causal relation between the manipulated variable and
the outcome.
Postgraduate Course
Observational research
Natural experiment
Cohort / panel study
Case-control study
Observational research refers to studies where the
researcher merely observes but does not intervene,
with the intention of finding correlations among the
observed data
Postgraduate Course
Qualitative research explores and tries to understand
people's beliefs, experiences, attitudes, behaviour and
interactions. It generates non-numerical data. The best-
known qualitative research-methods include in-depth
interviews, focus groups, documentary analysis and
participant observation.
Qualitative research includes
Case studies
Ethnographic studies
Field research
Grounded theory approach
Qualitative research
Postgraduate Course
Longitudinal research
Cohort / panel study
Repeated measures
Times series
A longitudinal study is an observational study that
involves repeated observations (measurements)
of the same variable(s) over long periods of time
(sometimes years or even decades).
Postgraduate Course
Mixed methods
Postgraduate Course
Better than a single study: a replication study
Better than a replication study:a systematic review / meta analysis
If there were 100 studies, 99 of which gave a ‘negative’ result (where, say, the new intervention appeared to be not effective), while one had a ‘positive’ result (were the intervention appeared effective), it would obviously be a mistake to consider only the single positive study.
But ….
Postgraduate Course
Systematic review / meta-analysis
The intention behind a systematic review is to identify as fully
as possible all the scientific studies of relevance to a particular
subject and to assess the validity and authority of the evidence
of each study separately. As the name indicates, a systematic
review takes a systematic approach to identifying studies and
has the methodological quality critically appraised by multiple
researchers independently of each other, as a consequence of
which the review is transparent and reproducible and can be
monitored. The use of statistical analysis techniques in a
systematic review to pool the results of the individual studies
numerically in order to achieve a more accurate estimate of
the effect is termed a “meta-analysis”.
Postgraduate Course
?
Systematic review / meta-analysis
Postgraduate Course
1. Randomized controlled study
2. Controlled before-after study
3. Cohort / panel study
4. Before-after study
5. Post-test only study
6. Survey
7. Case study
A. Experiment
B. Quasi-experiment
C. Observational study
D. Longitudinal study
E. Cross-sectional study
F. Qualitative study
Which terms belong together?