JBI Library of Systematic Reviews JBL000307 2010;8(18):730-751
Godfrey et al. Use of models or frameworks in evidence synthesis © the authors 2010 page 730
Utilisation of theoretical models and frameworks in the process of
evidence synthesis
Christina M. Godfrey RN MSc1
Margaret B. Harrison RN PhD2
Ian D. Graham PhD3
Amanda Ross-White BA MLIS4
1. Queen‟s Joanna Briggs Collaboration, Queen‟s University, Kingston, Ontario, Canada, K7L 3N6
2. School of Nursing, Queen‟s University, Kingston, Ontario, Canada, K7L 3N6
3. Canadian Institutes of Health Research, Ottawa, Ontario, Canada, K1A 0W9
4. Bracken Health Sciences Library, Queen's University, Kingston, Ontario, Canada, K7L 3N6
Corresponding author: Christina M. Godfrey, Queen‟s Joanna Briggs
Collaboration, Queen‟s University, Kingston, Ontario, Canada, K7L 3N6
Tel: 613-533-6000 ext 78760 Email: [email protected]
Executive summary
Background A systematic review is a comprehensive enquiry or study of secondary
data sources. There is a research question, an a priori articulation of methods and a
set of procedures to focus the investigation. Despite these rigorous structures to
guide the review, synthesising evidence is a challenging, resource intense and time
consuming process. Large volumes of information complicate not only the search
functions, but also the conceptualisation of the evidence needed to create the concise
and integrated results. Use of a theoretical model or framework could serve as an
essential element in effectively focusing the review and designing the methods to
respond to the knowledge question.
Objective This scoping review sought to confirm the value of models or frameworks
used by authors working within traditional methodologies for evidence synthesis.
Inclusion criteria
Types of participants The focus of this review was on the context of health care.
Types of intervention(s)/phenomena of interest All studies that discussed models
or frameworks used specifically to address the process of synthesis were included.
Types of studies Discussion, scholarship or methodology papers and reviews were
included.
Types of outcome All theoretical models or frameworks were described, with
specific attention to the purpose of the framework for each study, and the contribution
of the framework to the process of synthesis.
Search strategy The search strategy aimed to find both published and unpublished
studies. A three-step search strategy was utilised. The databases for published
material included CINAHL; Medline; EMBASE; PsycINFO; AMED; Cochrane; Biomed
JBI Library of Systematic Reviews JBL000307 2010;8(18):730-751
Godfrey et al. Use of models or frameworks in evidence synthesis © the authors 2010 page 731
Central; Scirus; and Mednar. Databases for unpublished material included
Dissertation Abstracts; Sociological Abstracts; Conference proceedings.
Methodological quality The review was a focused scoping review to locate and
describe the contribution of theoretical models or frameworks to the process of
synthesis. The methodological quality of the discussion papers was therefore not
assessed.
Data collection Data was extracted from the discussion papers using an adaptation
of the standardised data extraction tool from the Joanna Briggs Institute Data
Extraction for Narrative, Expert opinion & text (JBI-NOTARI).
Data synthesis Results were discussed in narrative form. The use of frameworks in
each step of the synthesis process was discussed.
Results Eight studies (nine papers) formed the final set included in this review. The
studies targeted the following issues: Child protection; end-of-life care; predictors of
adolescent sexual behaviour and intention; primary care career choice; prognosis of
acute whiplash; reluctance to care; use of Information Technology; young child's
post-divorce adjustment. Frameworks were used in four of the seven steps of
synthesis, and integration of the data indicated that the use of frameworks in the
process of evidence synthesis was valuable and had many advantages.
Conclusion This review illustrates that the addition of structure and guidance
provided by a framework can serve to benefit the process of integration. Studies in
this review indicated that the use of frameworks helped to inform the association
between variables, guide the search strategy, structure and clarify the outcomes,
identify knowledge gaps and indicate areas for future research. Used in this manner,
frameworks could provide a valuable foundation for the process of synthesis.
Implications for practice Evidence from systematic reviews informs practice. The
incorporation of a theoretical model or framework helps to guide the process of
synthesis and clarify the outcomes. This added transparency will facilitate the
assimilation of the evidence by the target audience.
Implications for research Systematic reviews are the highest level of evidence
available at this time. The use of theoretical models or frameworks in the review
process strengthens the rigor and transparency of the integrative method. Further
research into the contribution of theoretical models or conceptual frameworks to the
process of synthesis may be valuable.
Keywords Theoretical model; conceptual framework; theoretical framework;
conceptual model; Systematic review; integrative review; integrative research;
evidence summary
Background
A systematic review is a comprehensive enquiry or study of secondary data sources. There is a
research question, an a priori articulation of methods and a set of procedures to focus the
investigation. Seven formal steps help guide the synthesis process: Stage 1 - Developing the review
protocol; Stage 2 - Developing the review question; Stage 3 - Identifying search criteria;
JBI Library of Systematic Reviews JBL000307 2010;8(18):730-751
Godfrey et al. Use of models or frameworks in evidence synthesis © the authors 2010 page 732
Stage 4 - Generating the search strategy; Stage 5 - Assessing study quality; Stage 6 - Extracting the
data; and Stage 7 - Synthesising the data. Despite these rigorous structures to guide the review,
synthesising evidence is a challenging, resource intense and time consuming process. Large volumes
of information complicate not only the search functions, but also the conceptualisation of the evidence
needed to create the concise and integrated results.
Formal review protocols, such as those used by Cochrane or the Joanna Briggs Institute, are
procedure driven from methodologies that serve to lead the review group and keep them on-track with
their original goals for synthesis of gathered information. Although such protocols are essential, our
experience is that more is needed to form a basis for the effective conduct of reviews. We believe a
critical and key element is a priori conceptualisation of the review. Use of a theoretical model or
conceptual framework could serve as an essential element in effectively focusing the review and
designing the methods to respond to the knowledge question.
Conceptual frameworks are constructed to identify logical or theoretical relationships among variables
and are often represented diagrammatically. As such they have been referred to in the research
literature as 'conceptual maps'. A conceptual map is "a diagram of the relationship among the
variables linking the independent variable to the dependent variables and serves to clarify the
research problem."1,p.379 A conceptual map is regarded as a theoretical model rather than a model
for statistical analysis, and serves to summarise and integrate the existing state of knowledge about a
phenomenon.1 With systematic reviews, this is a critical element in effectively focusing the review and
designing the methods to respond to the knowledge question.
Since the term 'conceptual frameworks' is frequently used interchangeably with conceptual maps,
models, frameworks, and paradigms, it is important to define these terms. The American Heritage
Dictionary2 defines „conceptual‟ as:
1. The ability to form or understand mental concepts and abstractions
2. Something conceived in the mind; a concept, plan, design, idea, or thought and the word
'framework' as:3
1. A structure for supporting or enclosing something else, especially a skeletal support used as
the basis for something being constructed
2. A fundamental structure, as for a written work
3. A set of assumptions, concepts, values, and practices that constitutes a way of viewing
reality.
Thus the term 'conceptual framework' combines both abstract and form, bringing together thoughts
and ideas and providing a supporting structure with which to contain them. A conceptual framework
facilitates the linkage of concepts with previously constructed reality, thereby promoting the
incorporation of different concepts or new ideas into our existing world view. For example, Kim et al.,4
developed their own conceptual framework in order to identify strategies for constructing teaching
cases for a variety of learning settings. After conducting a review of the literature they developed a
conceptual framework that comprised five attributes: Relevant; Realistic; Engaging; Challenging, and
Instructional, which could serve as a menu of teaching case development options for educators.
Although the term theoretical framework is used interchangeably with conceptual framework, a
theoretical framework generally incorporates at least part of a specific theory as the basis for a study,
and usually guides the development and testing of interventions and hypotheses based on the tenets
of the theory.5 As an example, Mock et al.,
5 describe the use of the Levine Conservation Model to
JBI Library of Systematic Reviews JBL000307 2010;8(18):730-751
Godfrey et al. Use of models or frameworks in evidence synthesis © the authors 2010 page 733
guide the investigation of an exercise intervention to mitigate cancer-related fatigue. They used this
model because it included principles that help explain cancer-related fatigue and supported exercise
as a potential intervention for the fatigue.
The beginning of an enquiry is considered to be the conceptual phase of research and as such, lays
the foundation by presenting the „state of knowledge‟ about the phenomenon under study.6 This
phase directs the procedures of data production and analysis as well as the interpretation of the
derived findings. Use of a theoretical model or framework at this stage of the review process,
contributes to identifying and accessing needed clinical and policy expertise to hone and focus a
search methodology. However, the usefulness of a model is not limited to establishing the initial focus
and depth of the research project. Once the results of an investigation are known they can be
illustrated and discussed by relating them back to the elements within the model. This process
facilitates the incorporation of new ideas as the model provides the vehicle by which to enter into the
existing knowledge structure. This iterative process of exploration generates questions such as: How
does what is discovered enrich what is known? What new questions have been generated? Do new
areas require exploration as a result of the study's findings?
As the use of theoretical models or conceptual frameworks is not a prescriptive element in the
performance of systematic reviews, this scoping review will determine the value of such models or
frameworks when performing synthesis of evidence.
Systematic reviews versus scoping reviews
The distinction between systematic and scoping reviews is not frequently discussed in the current
literature. Arksey and O‟Malley‟s7 2005 article titled Scoping Studies: Towards a Methodological
Framework differentiates between these two types of reviews and suggests two important differences:
a) systematic reviews typically focus on a well defined question, whereas scoping reviews tend to
address broader topics; and b) systematic reviews assess the methodological quality of included
studies, whereas scoping reviews do not undertake this assessment.
Further to this, Arksey and O‟Malley7 propose two different ways of thinking about scoping reviews: a)
scoping reviews could be considered part of the ongoing process of reviewing in which scoping
reviews precede systematic reviews and help determine the value of undertaking a full systematic
review; or b) scoping reviews are considered a method in their own right and facilitate an examination
of the literature to identify gaps or to summarise and disseminate research findings.
The purpose of this scoping review was to examine the process of evidence synthesis and the
contribution of the use of frameworks to this process. The methodological rigor of the included studies
was not considered an issue that would influence the contribution of the framework or model.
Therefore a scoping review methodology was undertaken to synthesise the existing evidence and is
perceived according to the second category, as a methodology in its own right, producing a stand-
alone study.
Objective
This scoping review sought to confirm the value of models or frameworks used by authors when
performing the synthesis of evidence.
JBI Library of Systematic Reviews JBL000307 2010;8(18):730-751
Godfrey et al. Use of models or frameworks in evidence synthesis © the authors 2010 page 734
Review question
How does the use of theoretical models or frameworks in the process of synthesis contribute to the
scholarship of integration?
PICO
Problem – complex and challenging process of integrative research and evidence synthesis.
Intervention – use of a model or framework.
Context – conduct of systematic or scoping reviews.
Outcome – description of model(s) or framework(s) currently in use; purpose of framework for the
review; contribution of the framework to the review.
Inclusion criteria
Types of participants
This review did not address a particular set of participants but focused on the context of health care.
Types of intervention(s)/phenomena of interest
This review considered all studies that discussed models or frameworks that were used specifically to
address the process of synthesis. Given the range of terms used to refer to models and frameworks,
the following terms were included:
theoretical models
theoretical frameworks
conceptual models
conceptual frameworks
conceptual maps
Types of studies
This scoping review considered all discussion, scholarship or methodology papers that discussed the
use of models or frameworks to address the process of synthesis.
Types of outcomes
This review described all theoretical models or frameworks used during the process of conducting a
systematic review; discussed the specific purpose of the framework for each study; and the
contribution of the framework to the process of synthesis.
JBI Library of Systematic Reviews JBL000307 2010;8(18):730-751
Godfrey et al. Use of models or frameworks in evidence synthesis © the authors 2010 page 735
Search strategy
The search strategy aimed to find both published and unpublished studies (Appendix I). A copy of the
Medline search is provided as it represents the most complex and comprehensive of all the strategies.
A three-step search strategy was utilised. An initial limited search of MEDLINE and CINAHL was
undertaken followed by analysis of the text words contained in the title and abstract, and of the index
terms used to describe the article. A second search using all identified keywords and index terms was
then undertaken across all included databases. Thirdly, the reference list of all identified reports and
articles was searched for additional studies. The databases included: CINAHL; Medline; EMBASE;
PsycINFO; AMED; Cochrane; Biomed Central; Scirus; Mednar.
Studies were limited to English or French language. No date limits were set on the searches and each
search was performed from the beginning date of each database until July 2009. The search for
unpublished studies included the following sources:
Dissertation Abstracts
Sociological Abstracts
Conference proceedings
Electronic searching resulted in lists of articles with details of title, author, source, and sometimes
abstract. All identified articles were assessed on the basis of the abstract (or title if abstract not
available) and full reports were retrieved for all studies that meet the inclusion criteria for the review.
When in doubt, the full article was retrieved.
Initial subject headings and keywords included:
Theoretical model (subject heading); Conceptual framework (subject heading in CINAHL);
Theoretical framework; Conceptual model; Systematic review (subject heading in CINAHL)
Integrative review; Integrative research; Evidence summary.
Method of the review
The citations and articles were reviewed by the lead author. In collaboration with the review panel,
selected studies were checked to verify those accepted into the review and those excluded.
Methodological quality
This study sought to understand the contribution of theoretical models or conceptual frameworks to
the process of synthesis. We therefore focused on each author‟s description of the framework they
used and their assessment of the value of incorporating the framework into evidence synthesis. Given
that our intent was to understand the process, the rigor of the research used to explicate the
frameworks was not considered relevant to this study.
Data extraction
Data was extracted from the discussion papers using an adaptation of the standardised data
extraction tool from the Joanna Briggs Institute Data Extraction for Narrative, Expert opinion & text -
JBI-NOTARI (Appendix II).
The extraction tool was adapted to allow a full description of the model or framework and its
components; if created to address a particular step in the synthesis process; and how the model or
JBI Library of Systematic Reviews JBL000307 2010;8(18):730-751
Godfrey et al. Use of models or frameworks in evidence synthesis © the authors 2010 page 736
framework contributed to the process of synthesis. The following specific data was extracted:
(modified from Graham8)
1. The origins of the model or framework. Who developed it? Where are they from (institution,
discipline)?
2. The concepts described by the model or framework.
3. The purpose of the framework for the review.
4. The frameworks contribution to the review.
5. The value of the framework for the process of synthesis.
Data synthesis
Results were discussed in narrative form. The contribution of frameworks in each step of the
synthesis process was discussed.
Review results
Description of studies
The search strategy for this review generated 2,795 citations, of which 17 met the inclusion criteria
(Appendix III). An additional six studies were included from hand searching giving a total of 23
studies. Nine papers, comprising eight studies (one companion paper) formed the final set included in
this review (Appendix IV). It is recognised that the numbers in this final set does not reflect the
number of reviews currently using frameworks as part of the process of synthesis. However, the
purpose of this scoping review was to examine the contribution of frameworks to this process of
integration and these nine studies were selected because they specifically addressed this issue and
discussed the value of using models or frameworks as part of their review process. Fourteen studies
were excluded because they did not address the contribution of their framework to the process of
synthesis, or discussed quality assessment models which were used for critical appraisal or statistical
models which were used for meta-analysis (Appendix V). Eight studies9-17
(nine papers) were included
in this review (Appendix VI). Studies ranged in publication date from 1995-2007. Five studies
originated (by location of the lead author) in the United States, one in the United Kingdom, one in
Australia and one in Canada. Disciplines involved included three in public health, two in nursing, one
in medicine, and one in psychology. Six reviews were performed in academic settings and two in
clinical settings, and they targeted the following issues:
Child protection
End-of-life care
Predictors of adolescent sexual behaviour and intention
Primary care career choice
Prognosis of acute whiplash
Reluctance to care
Use of Information Technology
Young child's post-divorce adjustment
JBI Library of Systematic Reviews JBL000307 2010;8(18):730-751
Godfrey et al. Use of models or frameworks in evidence synthesis © the authors 2010 page 737
Results
Frameworks
Three reviews9;11;16
used frameworks that were researcher developed. Two researchers12;13
based
their frameworks on previously created models that addressed the same issues and three
reviews10;14;15
used other researchers‟ frameworks (Table 1).
Table 1: Author list, purpose of framework and the framework origin
Author
Year
Purpose of Framework Framework Origin
Burridge
2007
To organise the findings into four categories revealing contributing factors of
caregiving reluctance.
Researcher developed
Buhi
2006
To reflect which variables might function as primary determinants for any given
behaviour.
National Institute of Mental Health
Theorists' Workshop
Lorenz
2005
To define outcomes for both patients and caregivers in end-of-life care and to
define the relationship between these outcomes.
Donabedian's Quality of Care
Framework
Taylor
2004
The salutogenic framework was used as a way of clarifying what benefit
particular research findings may have in identifying and using factors which
can be associated with protection, safety and well being of children.
Antonovski's Salutogenesis
Framework
Kukafka
2003
Novel framework intended to guide synthesis of more than one theoretical
perspective for the purpose of planning multi-level interventions to enhance IT
use.
Researcher based on Green and
Kreuter's PRECEDE/ PROCEED
model
Cote
2001
A conceptual framework was designed to classify the literature according to
methodologic quality, target population, and phases of investigation.
Researcher based on work by
Altman and Layman (prognosis of
breast cancer)
Whiteside
2000
Model depicting young child adjustment to divorce - Intent was to determine
what evidence supports the hypothesised connections in the model, as well as
what had not been studied.
Researcher developed
Bland
1995 (a)
Bland
1995 (b)
To reflect the variables that determine medical students’ choice of specialties.
The model presents a theory on how multiple variables work together to
determine specialty choice.
Researcher developed
Stages of the review process in which the model or framework was used
All of the studies used their frameworks to assist with the final stage of synthesising the review data
and three studies9;10;13
used their frameworks to help with generating the search strategy. One would
assume that in order to facilitate the synthesis, the data extraction would be performed using the
framework as a guide; however, only two studies9;16
specifically mentioned using their frameworks in
this manner, at this stage (Table 2).
JBI Library of Systematic Reviews JBL000307 2010;8(18):730-751
Godfrey et al. Use of models or frameworks in evidence synthesis © the authors 2010 page 738
Table 2: Stages of the review process and use of frameworks
Stage of the review process Number of studies using frameworks in this stage
Stage 1 - Developing the review protocol -
Stage 2 - Developing the review question -
Stage 3 - Identifying search criteria 1 ( Kukafka )
Stage 4 - Generating the search strategy 3 (Bland; Buhi; Kukafka)
Stage 5 - Assessing study quality -
Stage 6 - Extracting the data 2 (Bland; Whiteside)
Stage 7 - Synthesising the data 8 (All)
Purpose of the framework for the study
Studies used models or frameworks for one of two reasons:
To organise or classify either the literature or the study findings, in so doing, guide the process of
synthesis. For example, in their review on the prognosis of acute whiplash, Côté et al.,12
designed a
conceptual framework “to classify the literature according to methodologic quality, target population,
and phases of investigation.”12
,p.E445
To illustrate how multiple variables work together, thereby defining or clarifying relationships between
outcomes. In their review on the determinants of primary care specialty choice, Bland et al.,9
generated their model to present “a theory on how multiple variables work together to determine
specialty choice.”9,p.622
Overall five studies10-13;16
used their models or frameworks to augment the organisation of the review
and three studies9;14;15
used frameworks to determine or exemplify the relationships between the study
variables.
Contribution of the framework to the study
Three studies14;15;17
described their framework‟s contribution as serving to inform the association
between variables. Bland and colleagues17
concluded that the benefits of their review were
maximised by using a framework because “the model provided a picture of the whole phenomenon
and how its components related to each other.”17
,p.649 In this capacity the framework could also
highlight parts of the larger phenomenon needing further study, and reveal shortcomings in the
review‟s included studies, gaps in the literature, weak or missing links in the literature, or areas for
further research.
The purpose of framework in the study by Kukafka and colleagues13
was to “facilitate the cross-
theoretical integration of behavioral models to guide multi-level IT implementation plans.”13
,p.219 In
this instance the framework‟s contribution was to organise the disparate theories coherently. “A
planner can then identify those theories that are applicable to each user community and to each
identified barrier to behaviour change. In so doing, the full range of theoretical tools and methods
becomes available to tackle the problem.”13
,p.227
The review by Buhi et al.,10
integrated the literature on the predictors of adolescent sexual behaviour
and identified a „negative contribution‟ of their framework. As a limitation in their review they
JBI Library of Systematic Reviews JBL000307 2010;8(18):730-751
Godfrey et al. Use of models or frameworks in evidence synthesis © the authors 2010 page 739
recognised that “there are factors not identified in the integrative theoretical framework that may add
to the explanation of adolescent sexual behaviour, ...”10
,p.19
Value of using frameworks in the process of synthesis
The models or frameworks used in the process of integration demonstrated their value at different
stages of the review process and in different ways. Bland et al.17
noted the value of their framework in
several stages of their review.
“A model or theory provides direction and boundaries to guide the search for relevant literature. At the
summary and conclusion stage, a model enables the synthesiser not only to cluster the literature into
themes but also to suggest how these findings work together to explain the larger phenomenon of
interest. … Finally, a review guided by a model reveals the areas that need further study.”17
,p.645-6
A framework‟s value in revealing gaps in knowledge and areas for further investigation were reported
by four studies.9;10;12;16
In their review on the young child‟s post-divorce adjustment, Whiteside et al.,16
stated:
“Many of the pathways in our models had been examined in only one or two studies and thus provide
results that are suggestive, but not definitive. In addition, other variables that may have powerful
impact on the child‟s postdivorce adjustment were not included. Had we focused only on the primary
predictors and their direct relationships to child outcomes, these gaps would not have been evident;
model-driven meta-analysis facilitated our identification of these weak or missing links.”16
,p.22
Buhi and colleagues10
report on the value of frameworks for the conceptualisation of the concepts in
the review, in addition to indicating directions for future research.
“A theoretical framework provides a structure and context for thinking logically about determinants
and their relationships; assists scholars in diagnosing which variables are under-represented in the
research, and reveals both a potentially important gap in knowledge and future directions for
inquiry.”10
,p.5
For some, use of a framework increased the complexity of their enquiry, such as the study by Kukafka
et al.,13
whose model pulled together several behaviour models to guide their decision making
process. For others, use of frameworks served to „unpack‟ or unravel some of the complex issues in
their study. Taylor and colleagues15
integrated the literature on child protection using the salutogenic
framework. They found the addition of a framework to the review process most valuable, and
concluded:
“Application of a salutogenic framework to the results was further illuminating and has utility for both
systematic review methodology and other child protection explorations. The matrix created a warp
and weft effect that identified gaps in current evidence and practice and was able to disentangle some
of the complexities inherent within failure to thrive situations. By beginning to shed understanding on
such processes, the concept of salutogenesis added further depth and rigour to the analysis.”15
,p.633
Discussion
This review illustrates the value of the use of models or frameworks in the process of synthesis.
Incorporating frameworks into the integrative process contributes positively at many stages, and has a
multitude of advantages. In her discussion on the benefits of a model driven meta-analysis, Becker18
proposes that “model-driven syntheses can produce a synergy among studies by suggesting and
addressing questions not examined in any single study.”18
,p.196 Becker‟s article written in 2001
JBI Library of Systematic Reviews JBL000307 2010;8(18):730-751
Godfrey et al. Use of models or frameworks in evidence synthesis © the authors 2010 page 740
compliments and substantiates the findings of this review. Becker prescribes two objectives for a
model-driven synthesis: a) to determine the extent to which research has investigated all or parts of a
model; and b) to investigate what the current research indicates about paths in the model.18
These
objectives correspond to the purpose of frameworks identified from the review studies: to
organise/classify the literature according to the framework; and to illustrate how framework variables
work together (what Becker calls „the paths in the model‟).
Becker also describes the potential „negative contribution‟ of a model or framework as was reported
by Buhi et al. “Models can limit the generalisations possible from a review if they limit choices of
constructs and variables. Specifically, conclusions can be limited when models do not include an
important construct or operationalisation of that construct.”18
,p.201
Studies in this review indicated that the use of frameworks helped to inform the association between
variables, and identified gaps in the knowledge base as well as areas for further research. The use of
frameworks provided a structure to link constructs and determinants, a guide to the generation of the
search strategy and a mechanism to integrate the review findings. The use of a model or framework
could increase complexity by bringing together several theories under one structure, or assist in the
unravelling of complex issues into their essential determinants.
Conclusion
The use of a model or framework in the process of synthesis is not prescriptive at this time. However,
this review illustrates that the addition of structure and guidance provided by a framework may serve
to benefit the process of integration. The contribution of a framework to inform the association of
variables, guide the search strategy, structure and clarify the outcomes, identify knowledge gaps and
indicate areas for future research could provide a valuable foundation for the process of synthesis.
This obviously has to be balanced with the potential limitations a model or framework could bring to
the research process.
Implications for practice
Evidence from systematic reviews informs practice. The incorporation of theoretical models or
frameworks into the process of synthesis increases the value of this evidence, by providing a solid
underlying structure for the review and by linking outcomes to the theory of a particular model. The
addition of this capacity to guide the process and clarify the outcomes will facilitate the assimilation of
this evidence by the target audience.
Implications for research
Systematic reviews are the highest level of evidence available at this time.19
It is therefore important
to ensure that reviews are undertaken with utmost rigor and transparency. Any opportunity to increase
the strength of integrative research is valuable. The incorporation of theoretical models or frameworks
into this process offers just this enhancement by providing a scaffold on which to anchor the concepts
and processes of the review. Further research into the contribution of theoretical models or
conceptual frameworks to the process of synthesis may be valuable.
Conflict of interest
None.
JBI Library of Systematic Reviews JBL000307 2010;8(18):730-751
Godfrey et al. Use of models or frameworks in evidence synthesis © the authors 2010 page 741
Acknowledgements
The lead author (CG) would like to acknowledge the Canadian Institutes of Health Research (CIHR)
for funding support provided by a PhD Fellowship Knowledge Translation Award (KPD 85181); and
the instructors from the CIHR-OHRI Knowledge Synthesis Training course for support and guidance
in conducting this review.
References
1. Artinian BM. Conceptual mapping: development of the strategy. Western Journal of Nursing
Research 1982;4(4):379-93.
2. The American Heritage® Dictionary of the English Language FE. Conceptual definition. Answers
com 2004 [cited 2006 Sep 4];Available from: URL: http://www.answers.com
3. The American Heritage® Dictionary of the English Language FE. Framework definition. Answers
com 2004 [cited 2006 Sep 4];Available from: URL: http://www.answers.com
4. Kim S, Phillips WR, Pinsky L, Brock D, Phillips K, Keary J. A conceptual framework for developing
teaching cases: a review and synthesis of the literature across disciplines. Medical Education
2006;40:867-76.
5. Mock V, St.Ours C, Hall S, Bositis A, Tillery M, Belcher A, et al. Using a conceptual model in
nursing research - mitigating fatigue in cancer patients. Journal of Advanced Nursing
2007;58(5):503-12.
6. Batey M. Conceptualisation: Knowledge and Logic Guiding Empirical Research. Nursing
Research 1977;26(5):324-9.
7. Arksey H, O'Malley L. Scoping studies: Towards a methodological framework. Int J Social
Research Methodology 2005;8(1):19-32.
8. Graham ID, Tetro J, and the KT Theories Research Group. Some Theoretical Underpinnings of
Knowledge Translation. Academic Emergency Medicine 2007;14:936-41.
9. Bland CJ, Meurer LN, Maldonado G. Determinants of primary care specialty choice: a non-
statistical meta-analysis of the literature. [Article]. Academic Medicine 1995 Jul;70(7):620-41.
10. Buhi ER, Goodson P. Predictors of adolescent sexual behavior and intention: a theory-guided
systematic review. Journal of Adolescent Health 2007;40(1):4-21.
11. Burridge L, Winch S, Clavarino A. Reluctance to care: a systematic review and development of a
conceptual framework. Cancer Nursing 2007 Mar;30(2):E9-19.
12. Cote P, Cassidy JD, Carroll L, Frank JW, Bombardier C. A systematic review of the prognosis of
acute whiplash and a new conceptual framework to synthesise the literature.[see comment].
[Review] [63 refs]. Spine 2001 Oct 1;26(19):E445-E458.
13. Kukafka R, Johnson SB, Linfante A, Allegrante JP. Grounding a new information technology
implementation framework in behavioral science: a systematic analysis of the literature on IT use.
[Review] [25 refs]. Journal of Biomedical Informatics 2003 Jun;36(3):218-27.
14. Lorenz KA, Lynn J, Morton SC, Dy SM, Shugarman LM, Wilkinson A, et al. Methodological
approaches for a systematic review of end-of-life care. J Palliat Med 2005;8 Supp 1:S4-11.:S4-11.
JBI Library of Systematic Reviews JBL000307 2010;8(18):730-751
Godfrey et al. Use of models or frameworks in evidence synthesis © the authors 2010 page 742
15. Taylor JS. Salutogenesis as a framework for child protection: literature review. Journal of
Advanced Nursing 2004 Mar 15;45(6):633-43.
16. Whiteside MF, Becker BJ. Parental Factors and the Young Child's Postdivorce Adjustment: A
Meta-Analysis With Implications for Parenting Arrangements. Journal of Family Psychology
2000;14(1):5-26.
17. Bland CJ, Meurer LN, Maldonado G. A systematic approach to conducting a non-statistical meta-
analysis of research literature. [Review] [14 refs]. Academic Medicine 1995 Jul;70(7):642-53.
18. Becker BJ. Examining theoretical models through research synthesis: the benefits of model-
driven meta-analysis. Evaluation & the Health Professions 2001;24(2):190-217.
19. Pearson A, Wiechula R, Court A, Lockwood C. The JBI model of evidence-based healthcare .
International Journal of Evidence-Based Health Care 2005;3:207-15.
JBI Library of Systematic Reviews JBL000307 2010;8(18):730-751
Godfrey et al. Use of models or frameworks in evidence synthesis © the authors 2010 page 743
Appendix I: Search strategy
Medline Search July 20
Search History (27 searches) # ▼ Searches
1 theoretical model.mp. [mp=title, original title, abstract, name of substance word,
subject heading word]
2 theoretical framework.mp. [mp=title, original title, abstract, name of substance word,
subject heading word]
3 conceptual model.mp. [mp=title, original title, abstract, name of substance word,
subject heading word]
4 conceptual framework.mp. [mp=title, original title, abstract, name of substance word,
subject heading word]
5 conceptual map.mp. [mp=title, original title, abstract, name of substance word,
subject heading word]
6 or/1-5
7 systematic review$.mp. [mp=title, original title, abstract, name of substance word,
subject heading word]
8 integrative research.mp. [mp=title, original title, abstract, name of substance word,
subject heading word]
9 integrative review$.mp. [mp=title, original title, abstract, name of substance word,
subject heading word]
10 evidence summar$.mp. [mp=title, original title, abstract, name of substance word,
subject heading word]
11 narrative summary.mp. [mp=title, original title, abstract, name of substance word,
subject heading word]
12 meta-analysis.mp. [mp=title, original title, abstract, name of substance word, subject
heading word]
13 meta?analysis.mp. [mp=title, original title, abstract, name of substance word, subject
heading word]
14 meta-synthesis.mp. [mp=title, original title, abstract, name of substance word, subject
heading word]
15 meta?synthesis.mp. [mp=title, original title, abstract, name of substance word,
subject heading word]
16 synthesis.mp. [mp=title, original title, abstract, name of substance word, subject
heading word]
17 or/7-16
18 6 and 17
19 Models, Theoretical/
20 models, nursing/ or models, organisational/
21 19 or 20
22 "Review Literature as Topic"/ 23 meta-analysis as topic/ 24 22 or 23 25 21 and 24 26 18 or 25 27 remove duplicates from 26
JBI Library of Systematic Reviews JBL000307 2010;8(18):730-751
Godfrey et al. Use of models or frameworks in evidence synthesis © the authors 2010 page 744
Appendix II: Data Extraction Instruments
NOTARI data extraction instrument
Extraction form adapted from JBI NOTARI data extraction instrument
Author ___________Year ______________ Title: ___________________
Author
Full reference
Publication year
Country
Study details :
Context
Institution
Department
Discipline
Setting
Framework description:
Purpose of framework
Framework details
Concept relationships
Framework author
Framework contribution
Stage of the review
Framework value description
JBI Library of Systematic Reviews JBL000307 2010;8(18):730-751
Godfrey et al. Use of models or frameworks in evidence synthesis © the authors 2010 page 745
Appendix III: Search decision flow diagram
Total # of citations
found 2,795
1950 to July Week 2 2009
Medline 617
1982 –July 28 2009 Cinahl
248
1980 to 2009 Week 29 Embase
278
1985 to July 28 2009 AMED
21
1967 to July Week 3 2009
PsycInfo
1079
2nd Quarter 2009
COCHRANE 94
July 29 2009 Dissertation
24 Socio Abs
4
July 29 2009 Scirus 255
Mednar 175
# of articles
meeting
criteria
10 8 2 0 2 0 0 0
Total number of articles 22 minus 5 duplicates = 17
Hand searching
6
17 plus 6 from hand searching
23
Articles excluded -
models used for critical appraisal or meta-analysis
14
Final number of articles 9
JBI Library of Systematic Reviews JBL000307 2010;8(18):730-751
Godfrey et al. Use of models or frameworks in evidence synthesis © the authors 2010 page 746
Appendix IV: Included studies
1. Bland CJ, Meurer LN, Maldonado G. A systematic approach to conducting a non-statistical meta-analysis
of research literature. [Review] [14 refs]. Academic Medicine 1995 Jul;70(7):642-53.
2. Bland CJ, Meurer LN, Maldonado G. Determinants of primary care specialty choice: a non-statistical meta-
analysis of the literature. [Article]. Academic Medicine 1995 Jul;70(7):620-41.
3. Buhi ER, Goodson P. Predictors of adolescent sexual behavior and intention: a theory-guided systematic
review. Journal of Adolescent Health 2007;40(1):4-21.
4. Burridge L, Winch S, Clavarino A. Reluctance to care: a systematic review and development of a
conceptual framework. Cancer Nursing 2007 Mar;30(2):E9-19.
5. Cote P, Cassidy JD, Carroll L, Frank JW, Bombardier C. A systematic review of the prognosis of acute
whiplash and a new conceptual framework to synthesise the literature.[see comment]. [Review] [63 refs].
Spine 2001 Oct 1;26(19):E445-E458.
6. Kukafka R, Johnson SB, Linfante A, Allegrante JP. Grounding a new information technology
implementation framework in behavioral science: a systematic analysis of the literature on IT use. [Review]
[25 refs]. Journal of Biomedical Informatics 2003 Jun;36(3):218-27.
7. Lorenz KA, Lynn J, Morton SC, Dy SM, Shugarman LM, Wilkinson A, et al. Methodological approaches for
a systematic review of end-of-life care. J Palliat Med 2005;8 Supp 1:S4-11.:S4-11. Taylor JS.
Salutogenesis as a framework for child protection: literature review. Journal of Advanced Nursing 2004 Mar
15;45(6):633-43.
8. Whiteside MF, Becker BJ. Parental Factors and the Young Child's Postdivorce Adjustment: A Meta-
Analysis With Implications for Parenting Arrangements. Journal of Family Psychology 2000;14(1):5-26.
JBI Library of Systematic Reviews JBL000307 2010;8(18):730-751
Godfrey et al. Use of models or frameworks in evidence synthesis © the authors 2010 page 747
Appendix V: Excluded studies
Carlson LE, Feldman-Stewart D, Tishelman C, Brundage MD. Patient-professional communication research in
cancer: an integrative review of research methods in the context of a conceptual framework. Psycho-
Oncology 2005 Oct;14(10):812-28.
Reason for exclusion: Does not address the contribution of the framework to the review process
Classen S, Winter S, Awadzi KD, Garvan CW, Lopez EDS, Sundaram S. Psychometric testing of SPIDER: data
capture tool for systematic literature reviews. American Journal of Occupational Therapy 2008
May;62(3):335-48.
Reason for exclusion: SPIDER is a data capture and quality assessment tool – quality assessment
was outside the scope of this review
Findley TW. Research in physical medicine and rehabilitation. II. The conceptual review of the literature or how
to read more articles than you ever want to see in your entire life.[see comment]. American Journal of
Physical Medicine & Rehabilitation 1989 Apr;68(2):97-102.
Reason for exclusion: Focus of the article was on how to develop a conceptual framework from the
literature
Hersh WR, Hickam DH. How well do physicians use electronic information retrieval systems? A framework for
investigation and systematic review.[see comment]. [Review] [35 refs]. JAMA 1998 Oct
21;280(15):1347-52.
Reason for exclusion: The framework was developed for evaluation and they did not discuss the
contribution of the framework t the review process
Hickam DH, Severance S, Feldstein A, Ray L, Gorman P, Schuldheis S, et al. The effect of health care working
conditions on patient safety. Rockville MD: Prepared by the Oregon Health & Science University
Evidence-based Practice Center Portland, Oregon under contract 290-97-0018 Agency for Healthcare
Research and Quality; 2003. Report No.: Evidence Report/Technology Assessment No. 74. AHRQ
Publication No. 03-E031.
Reason for exclusion: Does not discuss the contribution of models/frameworks to the review process
Kendrick DC, Bu D, Pan E, Middleton B. Crossing the evidence chasm: building evidence bridges from process
changes to clinical outcomes. Journal of the American Medical Informatics Association 2007
May;14(3):329-39.
Reason for exclusion: Mathematical model for analysis
Kim J, Pressler SJ, Jones J, Graves JR. Generating scientific models of knowledge using arcs. Clinical Nurse
Specialist 2008 Nov;22(6):286-92. Godfrey et al Use of models or frameworks in evidence synthesis
page 24
Reason for exclusion: Off topic – arcs is a computer program that builds scientific models
Mayer RE. Models for understanding. Review of Educational Research 1989;59(1):43-64.
Reason for exclusion: Off topic – use of models to increase students‟ understanding
Mulrow C, Langhorne P, Grimshaw J. Integrating heterogeneous pieces of evidence in systematic reviews.
Annals of Internal Medicine 1997 Dec 1;127(11):989-95.
JBI Library of Systematic Reviews JBL000307 2010;8(18):730-751
Godfrey et al. Use of models or frameworks in evidence synthesis © the authors 2010 page 748
Reason for exclusion: Off topic – does not discuss the contribution of models to the process of
synthesis
Penberthy JK, Cox D, Breton M, Robeva R, Kalbfleisch ML, Loboschefski T, et al. Calibration of ADHD
assessments across studies: A meta-analysis tool. Applied Psychophysiology Biofeedback 2005
Mar;30(1):31-51.
Reason for exclusion: Mathematical model for analysis
Russell CL. An overview of the integrative research review. [Review] [15 refs]. Progress in Transplantation
2005 Mar;15(1):8-13.
Reason for exclusion: Description of the integrative process
Rutledge DN, DePalma JA, Cunningham M. A process model for evidence-based literature syntheses.
Oncology Nursing Forum 2004 May;31(3):543-50.
Reason for exclusion: Process model to structure the interaction of individuals during the conduct of a
review
Smith E, Donovan S, Beresford P, Manthorpe J, Brearley S, Sitzia J, et al. Getting ready for user involvement
in a systematic review. Health Expectations 2009 Jun;12(2):197-208.
Reason for exclusion: Representation framework to categorise users
Smith MC, Stullenbarger E. A prototype for integrative review and meta-analysis of nursing research. Journal of
Advanced Nursing 1991 Nov;16(11):1272-83.
Reason for exclusion: A prototype for integrative research
JBI Library of Systematic Reviews JBL000307 2010;8(18):730-751
Godfrey et al. Use of models or frameworks in evidence synthesis © the authors 2010 page 749
Author/Year Country Institution
Purpose of Framework Framework Details Framework Contribution Framework Value
Burridge (2007) Australia University of Queensland
To organise the findings into four categories revealing contributing factors of caregiving reluctance
Conceptual framework consisted of the dimensions of caregiving reluctance which were divided into four dimensions: demographic, physical, psychological and social indicators. The outcomes of caregiving reluctance were identified as: deterioration of carer/patient relationship; reduced quality of care; institutionalisation
Classification of the findings according to the framework helped identify three aspects of the consequences of caregiving reluctance, with important implications for carers, patients, and healthcare services.
Framework assisted in analysis of information and classification of studies
Buhi (2006) U.S.A. University of South Florida
To reflect which variables might function as primary determinants for any given behaviour
Eight factors which account for the variance in any given behaviour: A strong intention (or a commitment to performing a behaviour); the absence of environmental constraints (hindering practice); necessary skills; a positive attitude toward performing the behaviour; social normative pressure to act; consistency.
This review’s purpose was to systematically summarise a decade of literature (1996 –2005) regarding these eight model elements’ predictive or explanatory ability when applied to adolescent sexual behaviors or intentions. Contribution positive: the elements of the framework guided the search for empirical studies and were used to classify each reviewed study’s findings. Contribution negative: there are factors not identified in the integrative theoretical framework that may add to the explanation of adolescent sexual behavior.
A theoretical framework provides a structure and context for thinking logically about determinants and their relationships; assists scholars in diagnosing which variables are under-represented in the research, and reveals both a potentially important gap in knowledge and future directions for inquiry.
Lorenz (2005) U.S.A. VA Greater LA Healthcare System, LA, California.
To define outcomes for both patients and caregivers in end-of-life care and to define the relationship between these outcomes
Donabedian described the relationship between outcomes, processes, and structure of care.
Donabedian’s quality-of-care framework structured the examination of the associations among outcomes considered by the project.
The framework informs the association between satisfaction, quality of care, quality of dying, and quality of life, which are various distal outcomes that apply in varying degrees to both patients and caregivers.
Appendix VI: Details of included studies
JBI Library of Systematic Reviews JBL000307 2010;8(18):730-751
Godfrey et al. Use of models or frameworks in evidence synthesis © the authors 2010 page 750
Author/Year Country Institution
Purpose of Framework Framework Details Framework Contribution Framework Value
Taylor (2004) United Kingdom University of Dundee
The salutogenic framework was used as a way of clarifying what benefit particular research findings may have in identifying and using factors which can be associated with protection, safety and well being of children.
Based on an ecological model of health, salutogenesis basically seeks to explore what keeps us healthy, or in the case of child protection, what is protective.
The salutogenesis framework can clarify what benefit particular research findings may have in identifying and using factors which can be associated with protection and safety and well being of children. It is fitting to use frameworks that enhance multidisciplinary understanding and collaboration and that seek to identify those factors that for children are most protective.
Use of a theoretical framework can strengthen research analysis, particularly in application to practice. The matrix created a warp and weft effect that identified gaps in current evidence and practice and was able to disentangle some of the complexities inherent within failure to thrive situations. By beginning to shed understanding on such processes, the concept of salutogenesis added further depth and rigour to the analysis.
Kukafka (2003) U.S.A. Joseph Mailman School of Public Health, Columbia University, New York, NY, USA
Novel framework intended to guide synthesis of more than one theoretical perspective for the purpose of planning multi-level interventions to enhance IT use.
The application of the proposed framework rests on two propositions: (1) IT use is complex, multi-dimensional, and influenced by a variety of factors at individual and organisational levels. The framework is a structure for applying scientific evidence from empirically tested models so that the approaches implemented are built upon links among multiple levels. (2) Success in achieving change is enhanced by the active participation of members from the target user groups. The framework promotes participatory design through a linkage system of critical assessment phases to ensure that the planners have a structure in place to engage system end-users effectively from the start.
The framework serves to focus the scope, to identify the conceptual variables to be extracted, and then to make explicit their relationship to the synthesising question of the analysis. The proposed framework promotes a distinct, problem- driven behavioral science perspective that brings to bear problem definition, application of theory, and empirical evidence in solving a problem. This is beneficial because rather than beginning with a single theory, the framework organises disparate theories coherently. A planner can then identify those theories that are applicable to each user community and to each identified barrier to behavior change. In so doing, the full range of theoretical tools and methods becomes available to tackle the problem. The framework facilitates the move from a single model to explain IT usage to more suitable multi-level interventions.
The proposed IT implementation framework that we have presented draws on several theoretical perspectives on human behavior. Such a framework can be an important element in the effort to improve the implementation and use of IT. This multiple-factor approach can aid in addressing all determinants involved in the implementation and successful adoption of an information system within a healthcare organisation.
JBI Library of Systematic Reviews JBL000307 2010;8(18):730-751
Godfrey et al. Use of models or frameworks in evidence synthesis © the authors 2010 page 751
Author/Year Country Institution
Purpose of Framework Framework Details Framework Contribution Framework Value
Cote (2001) Canada Institute for Work and Health, Toronto, Canada
A conceptual framework was designed to classify the literature according to methodologic quality, target population, and phases of investigation.
The framework meets the needs of clinicians, policy makers, and researchers and integrates three key attributes of prognostic studies: (1) it includes the assessment of the methodologic merit of a study (internal validity); (2) it incorporates the concept of target population and generalisability (external validity); (3) it includes a hierarchy of knowledge that categorises the strength evidence into mutually exclusive categories.
A conceptual framework is proposed that addresses the stakeholders’ needs while emphasising the principles of scientific rigor. The framework will assist clinicians and policy makers to critically appraise the literature and help them identify populations to whom the results may be generalisable. Moreover, it will provide researchers with a tool to systematically identify gaps in the literature and guide future research while mapping a research agenda that avoids the problems of circular epidemiology
The conceptual framework described here clearly shows that despite the recommendations of the Quebec Task Force (QTF), the recent prognostic literature on whiplash remains largely descriptive and demonstrates that large cohort studies investigating a wide range of prognostic factors are necessary to improve the understanding of this problem.
Whiteside (2000) U.S.A. Ann Arbor Center for the Family, Ann Arbor, Michigan
Model depicting young child adjustment to divorce - Intent was to determine what evidence supports the hypothesised connections in the model, as well as what had not been studied
Model illustrates contextual factors, parenting environment, child characteristics, and child outcomes
Model-driven meta-analysis facilitated the identification of weak or missing links in this area of research
The process reveals which pathways in the model have been frequently examined and which relationships have yet to be studied.
Bland (1995)(a) Bland (1995) (b) U.S.A. University of Minnesota Medical School
To reflect the variables that determine medical student's choice of specialties. The model presents a theory on how multiple variables work together to determine specialty choice.
The model depicts the type of school, its mission and faculty composition. The model also depicts the student characteristics with their incoming values and needs. The model illustrates how the type of school influences the student, their needs and values and how this interaction between the school and student influences the student’s choice of specialty.
The literature was sorted according to the components of the model, and the review addressed each of those components and studies. The model provided a picture of the whole phenomenon and how its components related to each other. Use of a model also revealed shortcomings of individual studies, and highlighted parts of the larger phenomenon needing further study.
“A model or theory provides direction and boundaries to guide the search for relevant literature. At the summary and conclusion stage, a model enables the synthesiser not only to cluster the literature into themes but also to suggest how these findings work together to explain the larger phenomenon of interest. … Finally, a review guided by a model reveals the areas that need further study.”