1/20

REVIEW PROTOCOL

Recommendation 4

The effect of health professionals’ education and training institutions adapting curricula to the evolving health-care needs of their communities.

A systematic review

Organization, City, Country: Faculty of Nursing, Mahidol University, Bangkok,

Thailand Prepared by: Date:

Nantiya Watthayu, Rachel Amiya July 2012

Senior supervisor Nantiya Watthayu Research team members Fongcum Tilokskulchai, Krongdai Unhasuta, Rachel

Amiya

2/20

Table of Contents

1.0 Background ......................................................................................................... 3

2.0 Objective ............................................................................................................. 3

3.0 Review Question ................................................................................................. 3

4.0 Evidence gathering and study selection .............................................................. 4

5.0 Assessment of risk of bias and data extraction ................................................... 8

6.0 Data synthesis ..................................................................................................... 9

7.0 Dissemination ..................................................................................................... 9

8.0 Resource implications ......................................................................................... 9

9.0 References ........................................................................................................... 9

Appendix A: Search strategy ........................................................................................ 11

3/20

1.0 Background

Core competencies have been used to refine curricula across health professional education in recent decades. The definition of competencies has been discussed widely in health professional education and practice (Gonczi et al., 1990; Pearson et al., 2002; Redfern et al., 2002). Broadly, competence is described as the set of characteristics or attributes that underlie and enable capable practice in an occupation (Heywood et al., 1992), while competencies describe a comprehensive profile of the knowledge, skills and professional behaviors that can be expected for a particular professional group (Fullerton et al., 2003).

Specifically, core competencies include the skills that health professionals need to interact effectively with patients and health care teams, recognizing that health professionals’ needs for particular competencies and skills vary based on discipline, specialty, and individual practice settings and patterns. Due to the nature of practice, opportunities for intervention and training needs vary across the different health professional groups, so it is also expected that educators from each discipline will translate the competencies into training curricula specific to their particular discipline.

This review will focus particularly on the process of adapting curricula to evolving needs and standards, and on the impact of such changes in terms of the quality and relevance of health professionals.

2.0 Objective

To assess the effect of adapting curricula to needs through the definition of core competencies and their incorporation into core curricula, in both undergraduate and postgraduate programs , on the quality and relevance of health professionals

3.0 Review Question

For the purposes of this literature review, the population, intervention, comparators and

outcomes (PICO) framework to inform the review objectives are presented below:

Population Intervention Comparison Outcome

Health professional schools

Curricula incorporating defined core competencies, including:

those needed to address local population health needs and expectations (e.g. epidemiology, burden of diseases, preventive services, screening / counseling / family planning, etc.)

those needed for the effective delivery of health services (e.g.

Traditional curricula (i.e. curricula not incorporating defined core competencies)

Quantity, quality and relevance of health professionals

4/20

decision-making, adapt to and initiate change, lead/manage, work in teams, lifelong learning)

those needed for education and training (i.e. to be an educator).

4.0 Evidence gathering and study selection

4.1 Evidence gathering

The evidence gathering approach will have four components:

4.1.1 Searching databases:

The databases in the table below will be searched with a pre-determined strategy as

detailed in Appendix A. In cases where the search results are small in number, search terms

will be reduced to maximize the search sensitivity.

Databases

Databases for biomedicine and health sciences:

PubMed/MEDLINE

Cumulative Index to Nursing and Allied Health Literature (CINAHL)

Cochrane Library (CDSR)

Excerpta Medica Database (EMBASE)

Health Systems Evidence (PPD/CCNC)

Medscape

NHS Evidence

Nursing Consult

Proquest Nursing & Allied Health Source Database

Databases for social sciences:

Campbell Library

ISI Web of Knowledge

Social Science Research Network (SSRN)

Databases for education:

BEME database

Education Research Complete

Educational Resources Information Center (ERIC)

Regional databases for biomedicine and health science:

African Index Medicus

5/20

Australasian Medical Index

Index Medicus for Eastern Mediterranean Region

Index Medicus for South-East Asia Region (IMSEAR)

IndMED

Japan Science and Technology Information Aggregator, Electronic (J-STAGE)

KoreaMed

LILACS

Mahidol University Research Databases

Panteleimon

Western Pacific Region Index Medicus (WPRIM)

Grey Literature

Agency for Healthcare Research and Quality

Academic Archive On-line

Biomed http://www.biomedcentral.com/

Controlled clinical trials database

Digital Dissertation Consortium

FADE Library http://www.fade.nhs.uk/

Grey Literature Network Service http://www.greynet.org

HMIC: Health Management Information Consortium http://library.nhs.uk/help/resource/hmic

International Network for Quality Assurance Agencies in Higher Education (INQAAHE) http://www.inqaahe.org/

INTUTE http://www.intute.ac.uk/

Index to Theses (UK and Ireland) http://www.theses.com

Lancashire Care Library and Information Service

MEDNAR http://mednar.com/mednar/search.html

National Library of Medicine Gateway

National Library of Guidelines

New York Academy of Medicine Grey Literature site http://www.nyam.org/library/online-resources/grey-literature-report/

Netting the Evidence www.nettingtheevidence.org.uk

NHS Evidence

OpenSIGLE System for Information on Grey Literature in Europe www.opengrey.eu

ProQuest Dissertation and Thesis

The Networked Digital Library of Theses and Dissertations

UNESCO Literature Database http://www.unesco.org/new/en/unesco/resources/online-materials/publications/unesdoc-database/

Web of Science

WHO's Library Database (WHOLIS)

WHO portal of clinical trials

World Bank http://www.worldbank.org

General databases:

6/20

Academic Search Complete/EBSCO

Blackwell Synergy

Faculty of 1000 (F1000)

Google

Google Scholar

OpenGrey

4.1.2 Hand searching:

The following journals and websites will be hand-searched for relevant articles:

Resources to be searched by hand:

Academic Medicine

American Academy of Family Physicians (AAFP)

American College of Physicians (ACP)

American Journal of Health Studies

Assessment & Evaluation in Higher Education

Association of American Medical Colleges (AAMC)

Association for Medical Education in Europe (AMEE)

Best Evidence Medical Education (BEME) database

BioMed Central

Council on Graduate Medical Education (COGME)

FADE: The North West Grey Literature Service

Grey Literature Network Services

Health & Social Care in the Community

Health Education Journal

Health Management Information Consortium (HMIC)

Health Services Research (HSR)

Health & Social Care in the Community

HRH Global Resource Center

International Medical Association (IMA)

International Council of Nurses (ICN)

International Journal of Academic Development (IJAD)

International Journal of Nursing Education Scholarship (IJNES)

International Pharmaceutical Federation (FIP)

Intute

Japan International Cooperation Agency (JICA) project reports

Joint Community Journal on Quality Improvement

Journal of Allied Health

Journal of Dental Education (JDE)

Journal of Interprofessional Care

Medical Teacher

New York Academy of Medicine Grey Literature Report

OpenGrey

7/20

SAMSS

Teaching and Teacher Education

UNESCO

World Bank

World Federation of Medical Education (WFME)

WHO’s Library Database (WHOLIS)

4.1.3 Expert network consultations:

A network of health professional education experts will be consulted by email to identify

additional grey literature or research not been found through the above processes.

Experts and networks to be contacted regarding additional articles

UNESCO, Asia-Pacific Academic Consortium for Public Health, Thai Nursing Council

4.1.4 Reference searches:

Bibliographies of those papers that match the eligibility criteria below will be searched by

hand to identify any further, relevant references, which will be subject to the same

screening and selection process.

4.2 Eligibility criteria

After gathering the evidence, the following eligibility criteria will be applied to the results

and all identified references screened independently by the reviewers (Fongcum

Tilokskulchai, Krongdai Unhasuta, Rachel Amiya) using a three-stage approach to reviewing

the title, abstract and full text.

4.2.1 Types of studies:

All types of evaluative study designs are eligible for inclusion, including grey literature.

Studies will not be selected on methodological quality.

4.2.2 Types of participants:

Health Professional Schools graduating:

Medical Doctors, both General and Specialist Practitioners, including Public Health Doctors (ISCO-08 minor group 221)

Nursing Professionals, both Generalist and Advanced nursing professionals, including Public Health Nurses (ISCO-08 minor group 222, unit 2221)

Midwifery Professionals (ISCO-08 minor group 222, unit 2222) Dentists (ISCO-08 minor group 226, unit 2261) Pharmacists (ISCO-08 minor group 226, 2262)

8/20

Paramedical Practitioners (ISCO-08 minor group 226, 2240)

4.2.3 Types of intervention:

Competencies needed to address local population health needs and expectations (e.g. epidemiology, burden of diseases, preventive services, screening / counseling / family planning, etc.)

Competencies needed for the effective delivery of health services (e.g. decision-making, adaptation to and initiation of change, leadership/management, working in teams, lifelong learning)

Competencies needed for education and training (i.e. to be an educator)

4.2.4 Types of outcome measures:

The primary outcomes of interest are the quantity, quality and relevance of practicing health professionals. These are defined by a number of measurable outcomes found in the Outcomes Framework document. Other important outcomes include values and preferences, resource use/costs, benefits and harms, as well as all other unintended effects of the intervention. Studies that include other outcomes should not be excluded at this stage in the evidence retrieval.

4.3 Exclusion criteria

Editorials, newspaper articles and other forms of popular media will be excluded. Failure to meet any one of the above eligibility criteria (section 3.2) will result in exclusion from the review and any apparent discrepancies during the selection process will be resolved by a third, independent reviewer. The number of excluded studies (including reasons for exclusion for those excluded following review of the full text) will be recorded at each stage.

5.0 Assessment of risk of bias and data extraction

Following the initial selection of literature, the reviewers should consult with the WHO

Secretariat regarding the process to be followed for summarizing the studies. Data will be

extracted from relevant papers using predefined evidence summary templates attached in

Appendix B. Data will be collected regarding the reasons for exclusion, characteristics of

included studies, participants, interventions (including comparators) and outcomes. The

final decision for inclusion or exclusion will be made by a team consisting of the WHO

Secretariat, methodologist and researchers conducting the review. Any potential

disagreement will be recorded and resolved by further discussion.

9/20

Risk of bias across studies will be assessed using the approach outlined by the Grading of

Recommendations Assessment Development and Evaluation (GRADE) working group. Any

disagreements will be recorded and resolved by involvement of an additional reviewer.

6.0 Data synthesis

The availability of appropriate data and resources to conduct a meta-analysis will be considered, where feasible.

7.0 Dissemination

A final set of tables including a GRADE Evidence Table and Descriptive Evidence Table will be produced and submitted to the WHO Secretariat as stipulated in the Procedures for the Retrieval of Evidence and Summary of Evidence. In addition, a manuscript will be prepared for submission to a peer-reviewed journal (a more specific dissemination proposal can go here as well).

8.0 Resource implications

The project lead will work closely with the WHO Secretariat to define the scope and methods of the review and facilitate access to unpublished literature, supporting translation of foreign language literature where necessary. Proposed milestones and timescales are outlined below:

MILESTONES

Evidence retrieval protocol finalized

Identifying and retrieving the evidence

Conduct literature review, hand searching and contacts with experts and networks

Summarizing evidence in Descriptive Evidence Tables

9.0 References

Fullerton J, Severino R, Brogan K, et al. The International Confederation of Midwives' study of essential competencies for midwifery practice. Midwifery, 2003, 19: 174-190.

10/20

Gonczi A, Hager P, Oliver E. Establishing competency based standards in the professions. Canberra: Australian Government Publishing Service, 1990.

Heywood L, Gonczi A, Hager P. A Guide to Development of Competency Based Standards for Professions: Research Paper No. 7. Canberra: National Office of Overseas Skills Recognition, Department of Employment Education and Training, 1992.

Pearson AFM, Fitzgerald M, Walsh K. Nurses' views on competency indicators for Australian nursing. Collegian, 2002, 9: 36-40.

Redfern S, Norman I, Calman L, Watson R, Murrells T. Assessing competence to practice in nursing: a review of the literature. Research Papers in Education, 2002, 17: 51-77.

11/20

Appendix A: search strategy

Search terms

Search Concept MeSH headings Keywords/ Text word

Population: Health professional schools/Health professional education

Education, Predental

Education, Premedical

Education, Professional

Education, Continuing

Education, Dental

Education, Graduate

Education, Medical

Education, Nursing

Education, Pharmacy

Education, Public Health Professional

Education, Midwifery

Education, Medical

Education, Medical, Graduate

Education, Medical, Undergraduate

Faculty, Dental

Faculty, Medical

Faculty, Nursing

Internal Medicine/education

Internship and Residency

Family Practice/education

Schools, Dental

Schools, Medical

Schools, Nursing

Schools, Pharmacy

Schools, Public Health

dental school(s)

dentistry school(s)

department(s) of dental medicine

department(s) of dentistry

department(s) of health science(s)

department(s) of medicine

department(s) of midwifery

department(s) of nursing

department(s) of pharmaceutical science(s)

department(s) of pharmacy

facult(y/ies) of dental medicine

facult(y/ies) of dentistry

facult(y/ies) of health science(s)

facult(y/ies) of medicine

facult(y/ies) of midwifery

facult(y/ies) of nursing

facult(y/ies) of pharmaceutical science(s)

facult(y/ies) of pharmacy

graduate school(s)

health professional school(s)

medical school(s)

midwifery school(s)

nursing school(s)

pharmacy school(s)

postgraduate school(s)

post-graduate school(s)

school(s) of dental medicine

school(s) of dentistry

school(s) of health science(s)

school(s) of medicine

school(s) of midwifery

school(s) of nursing

school(s) of pharmaceutical science(s)

12/20

school(s) of pharmacy

Intervention I: Core competencies / Standards

Competency-Based Education/standards

Education, Medical, Graduate/standards

Program Development/standards

Program Evaluation/standards

Professional Competence/standards

Clinical Competence/standards

Needs Assessment/standards

competency-based education

core competencies

core competence

core curriculum

core curricula

core curricular

education standards

educational standards

academic standards

degree standards

common standards

universal standards

Bologna

European Higher Education Area

European Area for Higher Education

European Space for Higher Education

European Higher Education Space

Intervention II: Adaptation / Innovation

Organization Innovation (Simplified list - see various permutations in search strategy below)

curriculum/program adaptation*

curriculum/program reform*

curriculum/program revision*

curriculum/program change*

curriculum/program renewal*

curriculum/program modification*

curriculum/program adjustment*

curriculum/program update*

innovat*

education* reform*

The above MeSH thesaurus headings, keywords and free text terms will be amended as

required in accordance with each different database used to maximize sensitivity. The

scope of each free text search should include the study title and abstract where possible.

13/20

Individual MESH and free text terms will be combined with the Boolean operator OR,

unless otherwise specified. Each search concept should then be combined with the

Boolean operator AND as follows: population AND intervention.

Search limits

Limit category Specified limit

Languages English, Thai

Publication type None

Date of publication 1995-2012

Study design None

Example search output – PubMed

No. Search term Hits

1

POPULATION CONCEPT MeSH terms

Schools, Medical[MeSH] OR Schools, Nursing[MeSH] OR Schools, Dental[MeSH] OR Schools, Pharmacy[MeSH] OR Schools, Public Health[MeSH] OR Education, Dental[MeSH] OR Education, Graduate[MeSH] OR Education, Medical[MeSH] OR Education, Nursing[MeSH] OR Education, Pharmacy[MeSH] OR Family Practice/education[mh] OR Internal Medicine/education[mh] OR Internship and Residency[mh] OR Teaching/methods[MeSH] OR Teaching/organization and administration[MeSH] OR Evidence-Based Medicine/education[MeSH] OR Evidence-Based Dentistry/education[MeSH] OR Evidence-Based Emergency Medicine/education[MeSH] OR Evidence-Based Nursing/education [MeSH] OR Clinical Competence/education[MeSH]

235163

2

POPULATION CONCEPT Keywords

Graduate school[tw] OR Postgraduate school[tw] OR Post-graduate school[tw] OR Graduate schools[tw] OR Postgraduate schools[tw] OR Post-graduate schools[tw] OR Medical school[tw] OR medical schools[tw] OR nursing school[tw] OR nursing schools[tw] OR midwifery school[tw]

273800

14/20

No. Search term Hits

OR midwifery schools[tw] OR dental school[tw] OR dental schools[tw] OR dentistry school[tw] OR dentistry schools[tw]OR pharmacy school[tw] OR pharmacy schools[tw] OR health professional school[tw] OR health professional schools[tw] OR School of medicine[tw] OR Schools of medicine[tw] OR School of nursing[tw] OR Schools of nursing[tw] OR School of midwifery[tw] OR Schools of midwifery[tw] OR School of dentistry[tw] OR Schools of dentistry[tw] OR School of dental medicine[tw] OR Schools of dental medicine[tw] OR School of pharmacy[tw] OR Schools of pharmacy[tw] OR School of pharmaceutical science[tw] OR Schools of pharmaceutical science[tw] OR School of health science[tw] OR Schools of health science[tw] OR School of health sciences[tw] OR Schools of health sciences[tw] OR Faculty of medicine[tw] OR Faculties of medicine[tw] OR Faculty of nursing[tw] OR Faculties of nursing[tw] OR Faculty of midwifery[tw] OR Faculties of midwifery[tw] OR Faculty of dentistry[tw] OR Faculties of dentistry[tw] OR Faculty of dental medicine[tw] OR Faculties of dental medicine[tw] OR Faculty of pharmacy[tw] OR Faculties of pharmacy[tw] OR Faculty of pharmaceutical science[tw] OR Faculties of pharmaceutical science[tw] OR Faculty of health science[tw] OR Faculties of health scienc[tw] OR Faculty of health sciences[tw] OR Faculties of health sciences[tw] OR Department of medicine[tw] OR Departments of medicine[tw] OR Department of nursing[tw] OR Departments of nursing[tw] OR Department of midwifery[tw] OR Departments of midwifery[tw] OR Department of dentistry[tw] OR Departments of dentistry[tw] OR Department of dental medicine[tw] OR Departments of dental medicine[tw] OR Department of pharmacy[tw] OR Departments of pharmacy[tw] OR Department of pharmaceutical science[tw] OR Departments of pharmaceutical science[tw] OR Department of health science[tw] OR Departments of health science[tw] OR Department of health sciences[tw] OR Departments of health sciences[tw] OR midwifery education[tw] OR pre service education[tw] OR preservice education[tw] OR pre licensure education[tw] OR prelicensure education[tw] OR clerkship[tw] OR clerkships[tw] OR clinical rotation[tw] OR clinical rotations[tw] OR clinical placement[tw] OR clinical placements[tw] OR internship[tw] OR internships[tw] OR

15/20

No. Search term Hits

house staff[tw] OR dental education[tw] OR pharmaceutical education[tw] OR clinical competence[tw] OR medical education[tw] OR nursing education[tw] OR midwifery education[tw] OR dentistry education[tw] OR pharmacy education[tw] OR health professional education[tw] OR graduate education[tw] OR graduate program[tw] OR graduate programme[tw] OR postgraduate education[tw] OR postgraduate program[tw] OR postgraduate programme[tw] OR post-graduate education[tw] OR post-graduate program[tw] OR post-graduate programme[tw] OR medical internship[tw] OR medical residency[tw] OR residency[tw] OR residencies[tw]

3 1 OR 2 288134

4

INTERVENTION CONCEPT 1 (Core Competencies/Standards) MeSH terms

Competency-Based Education/standards[MeSH] OR Education, Medical, Graduate/standards[MeSH] OR Program Development/standards[MeSH] OR Program Evaluation/standards[MeSH] OR Professional Competence/standards[MeSH] OR Clinical Competence/standards[MeSH] OR Needs Assessment/standards[MeSH]

17724

5

INTERVENTION CONCEPT 1 (Core Competencies/Standards) Keywords

competency-based education[tw] OR core competencies[tw] OR core competence[tw] OR core curriculum[tw] OR core curricula[tw] OR core curricular[tw] OR education standards[tw] OR educational standards[tw] OR academic standards[tw] OR degree standards[tw] OR common standards[tw] OR universal standards[tw] OR Bologna [tw] OR European Higher Education Area[tw] OR European Area for Higher Education[tw] OR European Space for Higher Education[tw] OR European Higher Education Space[tw]

5894

6 4 OR 5 22829

8 3 AND 6 17232

9

INTERVENTION CONCEPT 3 (Adaptation/Innovation) MeSH terms

Organizational Innovation [MeSH]

20291

10 INTERVENTION CONCEPT 3 (Adaptation/Innovation) 45179

16/20

No. Search term Hits

Keywords

curriculum adaptation [tw] OR curriculum adaptations [tw] OR curricula adaptation [tw] OR curricula adaptations [tw] OR curricular adaptation [tw] OR curricular adaptations [tw] OR adaptation of the curriculum[tw] OR adaptations of the curriculum[tw] OR adaptation of the curricula[tw] OR adaptations of the curricula[tw] OR adaptation to the curricula[tw] OR adaptations to the curricula[tw] OR adaptation to the curriculum[tw] OR adaptations to the curriculum[tw] OR program adaptation[tw] OR program adaptations[tw] OR programme adaptation[tw] OR programme adaptations[tw] OR adaptation of the program[tw] OR adaptations of the program[tw] OR adaptation of the programme[tw] OR adaptations of the programme[tw] OR adaptation to the program[tw] OR adaptations to the program[tw] OR adaptation to the programme[tw] OR adaptations to the programme[tw] OR adapting the curriculum[tw] OR adapting a curriculum[tw] OR adapting curricula[tw] OR adapting the curricula[tw] OR adapting curricular[tw] OR adapting the program[tw] OR adapting a program[tw] OR adapting programs[tw] OR adapting the programs[tw] OR adapting the programme[tw] OR adapting a programme[tw] OR adapting programmes[tw] OR adapting the programmes[tw] OR curriculum reform[tw] OR curriculum reforms[tw] OR curricula reform[tw] OR curricula reforms[tw] OR curricular reform[tw] OR curricular reforms[tw] OR reform of the curriculum[tw] OR reforms of the curriculum[tw] OR reform of the curricula[tw] OR reforms of the curricula[tw] OR reform to the curricula[tw] OR reforms to the curricula[tw] OR reform to the curriculum[tw] OR reforms to the curriculum[tw] OR program reform[tw] OR program reforms[tw] OR programme reform[tw] OR programme reforms[tw] OR reform of the program[tw] OR reforms of the program[tw] OR reform of the programme[tw] OR reforms of the programme[tw] OR reform to the program[tw] OR reforms to the program[tw] OR reform to the programme[tw] OR reforms to the programme[tw] OR reforming the curriculum[tw] OR reforming a curriculum[tw] OR reforming curricula[tw] OR reforming the curricula[tw] OR reforming curricular[tw] OR reforming the program[tw] OR reforming a program[tw] OR reforming programs[tw] OR reforming the programs[tw] OR reforming the

17/20

No. Search term Hits

programme[tw] OR reforming a programme[tw] OR reforming programmes[tw] OR reforming the programmes[tw] OR curriculum revision[tw] OR curriculum revisions[tw] OR curricula revision[tw] OR curricula revisions[tw] OR curricular revision[tw] OR curricular revisions[tw] OR revision of the curriculum[tw] OR revisions of the curriculum[tw] OR revision of the curricula[tw] OR revisions of the curricula[tw] OR revision to the curricula[tw] OR revisions to the curricula[tw] OR revision to the curriculum[tw] OR revisions to the curriculum[tw] OR program revision[tw] OR program revisions[tw] OR programme revision[tw] OR programme revisions[tw] OR revision of the program[tw] OR revisions of the program[tw] OR revision of the programme[tw] OR revisions of the programme [tw] OR revision to the program [tw] OR revisions to the program [tw] OR revision to the programme[tw] OR revisions to the programme[tw] OR revising the curriculum[tw] OR revising a curriculum[tw] OR revising curricula[tw] OR revising the curricula[tw] OR revising curricular[tw] OR revising the program[tw] OR revising a program[tw] OR revising programs[tw] OR revising the programs[tw] OR revising the programme[tw] OR revising a programme[tw] OR revising programmes[tw] OR revising the programmes[tw] OR curriculum change [tw]OR curriculum changes[tw] OR curricula change[tw] OR curricula changes[tw] OR curricular change[tw] OR curricular changes[tw] OR change of the curriculum[tw] OR changes of the curriculum[tw] OR change of the curricula[tw] OR change of the curricula[tw] OR change to the curricula[tw] OR changes to the curricula[tw] OR change to the curriculum[tw] OR changes to the curriculum[tw] OR program change[tw] OR program changes[tw] OR programme change[tw] OR programme changes[tw] OR change of the program[tw] OR changes of the program[tw] OR change of the programme[tw] OR changes of the programme[tw] OR change to the program[tw] OR changes to the program[tw] OR change to the programme[tw] OR changes to the programme[tw] OR changing the curriculum[tw] OR changing a curriculum[tw] OR changing curricula[tw] OR changing the curricula[tw] OR changing curricular[tw] OR changing the program[tw] OR changing a program[tw] OR changing programs[tw] OR changing the programs OR changing the programme OR changing a programme[tw] OR changing programmes[tw] OR

18/20

No. Search term Hits

changing the programmes[tw] OR curriculum renewal[tw] OR curriculum renewals[tw] OR curricula renewal[tw] OR curricula renewals[tw] OR curricular renewal[tw] OR curricular renewals[tw] OR renewal of the curriculum[tw] OR renewals of the curriculum[tw] OR renewal of the curricula[tw] OR renewals of the curricula[tw] OR renewal to the curricula[tw] OR renewals to the curricula[tw] OR renewal to the curriculum[tw] OR renewals to the curriculum[tw] OR program renewal[tw] OR program renewals[tw] OR programme renewal[tw] OR programme renewals[tw] OR renewal of the program[tw] OR renewals of the program[tw] OR renewal of the programme[tw] OR renewals of the programme[tw] OR renewal to the program[tw] OR renewals to the program[tw] OR renewal to the programme[tw] OR renewals to the programme[tw] OR renewing the curriculum[tw] OR renewing a curriculum[tw] OR renewing curricula[tw] OR renewing the curricula[tw] OR renewing curricular[tw] OR renewing the program[tw] OR renewing a program[tw] OR renewing programs[tw] OR renewing the programs[tw] OR renewing the programme[tw] OR renewing a programme[tw] OR renewing programmes[tw] OR renewing the programmes[tw] OR curriculum modification[tw] OR curriculum modifications[tw] OR curricula modification[tw] OR curricula modifications[tw] OR curricular modification[tw] OR curricular modifications[tw] OR modification of the curriculum[tw] OR modifications of the curriculum[tw] OR modification of the curricula[tw] OR modifications of the curricula[tw] OR modification to the curricula[tw] OR modifications to the curricula[tw] OR modification to the curriculum[tw] OR modifications to the curriculum[tw] OR program modification[tw] OR program modifications[tw] OR programme modification[tw] OR programme modifications[tw] OR modification of the program[tw] OR modifications of the program[tw] OR modification of the programme[tw] OR modifications of the programme[tw] OR modification to the program[tw] OR modifications to the program[tw] OR modification to the programme[tw] OR modifications to the programme[tw] OR modifying the curriculum[tw] OR modifying a curriculum[tw] OR modifying curricula[tw] OR modifying the curricula[tw] OR modifying curricular[tw] OR modifying the program[tw] OR modifying a program [tw] OR modifying programs[tw] OR modifying the

19/20

No. Search term Hits

programs[tw] OR modifying the programme[tw] OR modifying a programme[tw] OR modifying programmes[tw] OR modifying the programmes[tw] OR curriculum adjustment[tw] OR curriculum adjustments[tw] OR curricula adjustment[tw] OR curricula adjustments[tw] OR curricular adjustment[tw] OR curricular adjustments[tw] OR adjustment of the curriculum[tw] OR adjustments of the curriculum[tw] OR adjustment of the curricula[tw] OR adjustments of the curricula[tw] OR adjustment to the curricula[tw] OR adjustments to the curricula[tw] OR adjustment to the curriculum[tw] OR adjustments to the curriculum[tw] OR program adjustment[tw] OR program adjustments[tw] OR programme adjustment[tw] OR programme adjustments[tw] OR adjustment of the program[tw] OR adjustments of the program[tw] OR adjustment of the programme[tw] OR adjustments of the programme[tw] OR adjustment to the program[tw] OR adjustments to the program[tw] OR adjustment to the programme[tw] OR adjustments to the programme[tw] OR adjusting the curriculum[tw] OR adjusting a curriculum[tw] OR adjusting curricula[tw] OR adjusting the curricula[tw] OR adjusting curricular[tw] OR adjusting the program[tw] OR adjusting a program[tw] OR adjusting programs[tw] OR adjusting the programs[tw] OR adjusting the programme[tw] OR adjusting a programme[tw] OR adjusting programmes[tw] OR adjusting the programmes[tw] OR curriculum update[tw] OR curriculum updates[tw] OR curricula update[tw] OR curricula updates[tw] OR curricular update[tw] OR curricular updates[tw] OR update of the curriculum[tw] OR updates of the curriculum[tw] OR update of the curricula[tw] OR updates of the curricula[tw] OR update to the curricula[tw] OR updates to the curricula[tw] OR update to the curriculum[tw] OR updates to the curriculum[tw] OR program update[tw] OR program updates[tw] OR programme update[tw] OR programme updates[tw] OR update of the program[tw] OR updates of the program[tw] OR update of the programme[tw] OR updates of the programme[tw] OR update to the program[tw] OR updates to the program[tw] OR update to the programme[tw] OR updates to the programme[tw] OR updating the curriculum[tw] OR updating a curriculum[tw] OR updating curricula[tw] OR updating the curricula[tw] OR updating curricular[tw] OR updating the program[tw] OR updating a

20/20

No. Search term Hits

program[tw] OR updating programs[tw] OR updating the programs[tw] OR updating the programme[tw] OR updating a programme[tw] OR updating programmes[tw] OR updating the programmes[tw] OR innovate[tw] OR innovates[tw] OR innovation[tw] OR innovating[tw] OR educational reform[tw] OR educational reforms[tw] OR education reform[tw] OR education reforms[tw]

11 9 OR 10 46652

12 8 AND 11 742

13

CONTROLLED STUDY DESIGN Filter

randomized controlled trial[pt] OR randomized controlled trials[mh] OR random allocation[mh] OR double-blind method[mh] OR single-blind method[mh] OR clinical trial[pt] OR ((singl*[tw] OR doubl*[tw] OR trebl*[tw] OR tripl*[tw]) AND (mask*[tw] OR blind*[tw])) OR ("latin square"[tw]) OR random*[tw] OR comparative study[pt] OR evaluation studies[pt] OR follow-up studies[mh] OR prospective studies[mh] OR cross-over studies[mh] OR prospectiv*[tw]

3003969

14 3 AND 10 AND 12 117