Digital education

for building health

workforce capacity

Digital education

for building health

workforce capacity

Digital education for building health workforce capacity

ISBN 978-92-4-000047-6

© World Health Organization 2020

Some rights reserved. This work is available under the Creative Commons Attribution-NonCommercial-ShareAlike 3.0 IGO licence (CC BY-NC-SA 3.0 IGO; https://creativecommons.org/licenses/by-nc-sa/3.0/igo).

Under the terms of this licence, you may copy, redistribute and adapt the work for non-commercial purposes, provided the work is appropriately cited, as indicated below. In any use of this work, there should be no suggestion that WHO endorses any specific organization, products or services. The use of the WHO logo is not permitted. If you adapt the work, then you must license your work under the same or equivalent Creative Commons licence. If you create a translation of this work, you should add the following disclaimer along with the suggested citation: “This translation was not created by the World Health Organization (WHO). WHO is not responsible for the content or accuracy of this translation. The original English edition shall be the binding and authentic edition”.

Any mediation relating to disputes arising under the licence shall be conducted in accordance with the mediation rules of the World Intellectual Property Organization.

Suggested citation. Digital education for building health workforce capacity. Geneva: World Health Organization; 2020. Licence: CC BY-NC-SA 3.0 IGO.

Cataloguing-in-Publication (CIP) data. CIP data are available at http://apps.who.int/iris.

Sales, rights and licensing. To purchase WHO publications, see http://apps.who.int/bookorders. To submit requests for commercial use and queries on rights and licensing, see http://www.who.int/about/licensing.

Third-party materials. If you wish to reuse material from this work that is attributed to a third party, such as tables, figures or images, it is your responsibility to determine whether permission is needed for that reuse and to obtain permission from the copyright holder. The risk of claims resulting from infringement of any third-party-owned component in the work rests solely with the user.

General disclaimers. The designations employed and the presentation of the material in this publication do not imply the expression of any opinion whatsoever on the part of WHO concerning the legal status of any country, territory, city or area or of its authorities, or concerning the delimitation of its frontiers or boundaries. Dotted and dashed lines on maps represent approximate border lines for which there may not yet be full agreement.

The mention of specific companies or of certain manufacturers’ products does not imply that they are endorsed or recommended by WHO in preference to others of a similar nature that are not mentioned. Errors and omissions excepted, the names of proprietary products are distinguished by initial capital letters.

All reasonable precautions have been taken by WHO to verify the information contained in this publication. However, the published material is being distributed without warranty of any kind, either expressed or implied. The responsibility for the interpretation and use of the material lies with the reader. In no event shall WHO be liable for damages arising from its use.

Printed in Switzerland.

1

Acknowledgements 2

Abbreviations 3

Executive summary 4

1. Introduction 6

2. Overview of global mandates to address health workforce challenges and their intersection with digital health 7

3. Digital education for health workforce: evidence, potential role and future trends 10

3.1 Potential role of digital health education in the global agenda 10

3.2 Defining the landscape of digital education 11

3.3 Structuring the digital education field 11

3.4 Current evidence synthesis of digital education for health workers 12

3.5 Future trends in digital education 13

4. Health system integration of digital education: requirements and resources 15

4.1 External factors 16

4.2 System-level factors 16

4.3 Institutional factors 17

4.4 Individual factors 18

5. Conclusion 20

Key messages 20

Recommendations 20

References 22

Annex 1: Key themes identified from WHO mandates, resolutions and reports, and representative quotes 25

Annex 2: Questions on digital education requiring further research 27

Contents

2

Acknowledgements

This discussion paper was commissioned by the World Health Organization (WHO) to the Centre for Population Health Sciences at the Lee Kong Chian School of Medicine, Nanyang Technological University (NTU), Singapore (the WHO Collaborating Centre for Digital Health and Health Education) under the guidance of Jim Campbell (Director, Health Workforce Department, WHO). Technical oversight and quality assurance from the WHO were provided by Giorgio Cometto and Onyema Ajuebor.

WHO is grateful to the primary writers, peer reviewers and contributors to this document at NTU Singapore: Josip Car, Geronimo Jimenez Larrain, Bhone Myint Kyaw, Deepali Pavagadhi, Josh Quah Jialong, Yuri Rykov, Pier Spinazze, Swee Chuan Tjin, Elaine Tan Yu Ming and Lorainne Tudor Car.

Other contributors include participants at the informal virtual meeting on Digital health interventions and health workforce capacity building convened by WHO on 9 April 2019: Maya Adam (Stanford University, CA, USA), Magnus Conteh (Last Mile Health, Boston, MA, USA),

Seble Frehywot (George Washington University, Washington, DC, USA), Anna Frellsen (Maternity Foundation, Copenhagen, Denmark), Dai Hozumi (IntraHealth), Karin Kallander (United Nations Children’s Fund [UNICEF]), Luis Lapao (Institute of Hygiene and Tropical Medicine, Lisbon, Portugal), Wendy Leonard (The Ihangane Project, Rwanda), Karoline Linde (Laerdal Global Health, Stavanger, Norway), Jodi Lis (JHPIEGO), Charles G Prober (Stanford University, CA, USA). Participating colleagues from WHO included: Siobhan Fitzpatrick, Tigest Tamrat, Tana Wuliji, Xu Lihui and Diana Zandi.

Other reviewers from WHO include: Karen Daniels and Carey McCarthy. The e-learning case study resource on Hai Phong University, Viet Nam, was provided by Annie Chu (WHO Country Office, Viet Nam) with the support of Hazarika Indrajit (Regional Office for the Western Pacific). Special thanks to Clayton Hamilton (WHO Regional Office Europe) for a critical review of the draft manuscript.

Financial support for this policy brief was provided by the Government of the Federal Republic of Germany.

3

AI artificial intelligenceAR augmented realityCD-ROM compact disc - read only memoryDPST digital psychomotor skills trainersDVD-ROM digital versatile disk - read only memorye-health electronic healthe-learning electronic learningGPW13 Thirteenth General Programme of Work 2019–2023 (WHO)ICT information and communication technologiesILO International Labour OrganizationIP internet protocolIT information technologyLMIC low- and middle-income countriesLMS learning management systemm-health mobile healthm-learning mobile learningMOOC massive open online courseNGO nongovernmental organizationNTU Nanyang Technical University, SingaporeOECD Organisation for Economic Co-operation and DevelopmentRCT randomized controlled trialSDG Sustainable Development GoalsSGG serious gaming and gamificationTCP transmission control protocolUHC universal health coverageUNICEF United Nations Children’s FundUSB universal serial busVLE virtual learning environmentVP virtual patientVR virtual realityWHO World Health Organization

Abbreviations

4

Building health workforce capacity in countries requires major barriers to be overcome in order to achieve the Sustainable Development Goals, universal health coverage, and other health targets proposed in the WHO Thirteenth General Programme of Work (GPW13).

Global health mandates and resolutions have consistently emphasized the need for health workforce strengthening through lifelong learning opportunities. A thematic analysis of recent global health-related international resolutions (including United Nations General Assembly resolutions, World Health Assembly resolutions and other intergovernmental organizational strategies and workforce related strategies) reveals an urgent need to address global health workforce challenges to deliver better health services performance and outcomes. These concerns are presented under key themes and sub-themes that provide a framework for policy directives on digital education (also known as e-learning) to address health workers’ issues. This broad array of educational needs differs by setting. Some relevant examples include the need to increase student enrolment, improve learning outcomes, deliver education to health workers in remote areas, strengthen the competency of educators and enable lifelong learning.

Digital education has the potential to improve the competencies and satisfaction of health professionals. However, the effectiveness of digital methods depends upon the manner of implementation. Published studies that point to the benefits of digital health education have been found to have variable evidence quality and limited generalizability. Effectiveness of digital health education and outcomes vary widely depending on the learning objectives, modality (e.g. mobile phones, online digital education, virtual reality, serious gaming and gamification), delivery mode (e.g. fully digital or blended), instructional method (e.g. simulations, direct instruction), assessment methods (i.e. use of validated or non-validated instruments), learning pedagogies (e.g. digital problem-based learning or digital team-based learning), study population (e.g. nurses, allied health professionals, doctors), and the topic, discipline and health condition being taught (e.g. smoking cessation, diabetes management, domestic violence, antibiotic management, dermatology, child health, elderly care). The phenomenon of the digital divide is also important from an

implementation perspective as it exists both within and between countries, and may be a significant barrier for students, limiting equal access to digital education. Further research, rigorous evaluations, audits, investments and collaborations are required to optimize approaches for the effective use of digital education.

Scaling up and integrating digital tools for health workforce development into broader health systems involves addressing external, system-level, institutional and individual factors. The following framework of building health workforce capacity describes four levels of factors required to embed information and communication technologies (ICT) as foundations for transforming the health education system:

1. External factors include the level of digital and health literacy of the population, the extent to which the target population is receptive to adopting innovations and ICT systems, as well as the degree of commitment and support of governmental and nongovernmental actors. The culture and receptiveness of learning audiences to digital education is important to consider; this pertains to the trust that learners implicitly have (or lack) in digital education methods compared with other means available.

2. System-level factors include the incorporation of health workforce development objectives in long-term plans and evidence-based policy, sufficiency of technical infrastructure, appropriate levels of funding, and robustness of multisectoral collaboration among stakeholders (e.g. ministries of health, education, health academic centres, health care delivery organizations, IT companies). Digital education can address the standardization of the quality of curricula and accreditation mechanisms to allow for a uniform assessment of different educational institutions. Similarly, standardizing user interfaces/formats could play an important part in user acceptance.

3. Institutional factors include the level of organizational ownership, availability of infrastructure, governance, financing and management support, expertise in development of health education curricula and teaching resources using digital tools, deployment of education modules using appropriate digital health technologies, as well as training plans for developing human resources, teachers and administrators.

Executive summary

5

4. Individual factors include the beliefs, attitudes and behaviours of administrators, teachers, students and the support staff involved in the educational and technical processes.

Ensuring that all these factors are addressed is vital for maximizing the potential of digital education and reducing the digital divide.

This paper combines evidence from the scientific literature (including evidence from a collection of systematic reviews on digital tools and health workers’ education), practical suggestions for stakeholders to formulate approaches, and guidance in using digital tools to scale up health workforce education and capacity.

6

1. Introduction

Health systems worldwide are facing an increasing discrepancy between the number of health workers needed to provide health services, the availability of health professionals, and the capacity of countries to employ them. Several studies have demonstrated the direct correlation between the availability of health workers, coverage of health services, and population health outcomes (1–3).

Health workforce challenges (especially the shortage of personnel) are recognized as an important barrier to the achievement of universal health coverage (UHC) and the Sustainable Development Goals (SDG). Globally, there is a shortfall of 18 million health workers to achieve the health-related SDGs by 2030, and measures are required to counter this scarcity (4). At the same time, changing population health needs, renewed emphases on primary health care and UHC, and evolving health care delivery models, such as team-based and patient-centred care, require health workers with the right competencies to deliver safe and effective services. Digital health technologies have been identified as a potentially effective means to help address these growing challenges.

This paper provides an initial framework describing how digital health interventions may be used to strengthen education and build health workforce capacity. Its approach includes the synthesis of broader scientific evidence and practical case studies on the use of digital health education for health workers. The findings draw heavily on evidence from a recently published collection of systematic reviews on digital tools and health workers’ education (5). Additionally, the conditions needed for scaling up digital health interventions, and integrating them into broader health systems application have also been described.

This report comprises the following sections:

• Key global agendas and programmes related to the development of the health workforce.

• Evidence on the effectiveness of, and trends related to, digital health interventions, especially in the education and training of health care professionals.

• Requirements and resources needed for the successful integration and scaling up of digital education for health care professionals.

7

The global health workforce broadly consists of a range of health professions, mid-level occupations, as well as community health workers and other health worker categories. An in-depth analysis of the global mandates and intergovernmental organizations’ programmes highlights interrelated themes and cross-cutting issues for health workforce development. Health workforce strengthening is the most general among the identified themes. The Global Strategy on Human Resources for Health: Workforce 2030 (4) calls on WHO Member States to adopt transformative strategies in the scale-up of health worker education, and harness information and communication technologies (ICT) opportunities where feasible and cost-effective. The report of the United Nations High-Level Commission on Health Employment and Economic Growth (6) reinforces this call by highlighting the potential of digital tools as a common denominator across approaches for tackling health workforce challenges.

Broadly, across the existing policies and strategies, the following have been proposed to strengthen health workforce development across countries:

• increasing student enrolment and graduation (completion rate);

• optimizing organizational management and creating proper working conditions to achieve higher performance of the available health workforce;

• increasing workforce retention and postponing retirement in a balanced way to increase the number of health workers with the capacity to deliver high-quality health services;

• regulating the labour market and managing health workforce migration;

• facilitating cooperation and re-allocation of activities across professional groups of health workers to maximize performance of their skillsets;

• aligning investments in health workforce with population health needs;

• investing in health workforce jobs; and

• facilitating intersectoral coordination.

Further details on the cross-cutting themes emerging from the global mandates and programmes related to health workforce strengthening and digital health can be found in Annex 1. It is important for stakeholders and decision-makers to consider the following topic areas when designing or implementing local policies to strengthen their health workforce’s use of digital tools. These include but are not limited to:

Establishing digital health systems and policies

Digital health or e-health is an “important tool to help countries establish safe, efficient and sustainable health care delivery systems”(7). The rapid development of ICT, artificial intelligence (AI), e-health and m-health initiatives are fostering changes in models of care and health systems worldwide (8). These technologies provide new opportunities and have applications in

2. Overview of global mandates to address health workforce challenges and their intersection with digital health

8

telemedicine, electronic health records, digital education, clinical decision support, patient safety and autonomy, and big data analytics. The increasing use of digital health technologies in health care is an upcoming, evolving and currently underserved area for health workforce training and capacity building. Some of the main policy options in digital health:

• utilize ICT through systematic and evidence-based integration into health systems;

• develop standards and interoperability infrastructure for health data, while addressing current implementation barriers;

• develop evidence-based guidelines for digital health interventions to help countries select, adopt, manage and evaluate digital health solutions;

• build a repository of knowledge, best practices and tools to help countries implement their digital health strategies (for example, by adopting characteristics of strategies that have already succeeded);

• develop legislation and data protection policies; and

• improve population digital literacy.

Strengthening education and training capacity

Educational capacity is a key factor in addressing global health workforce shortages. Health workforce planning models must consider the implications for recruiting students (new entry into different health professions) and balancing their student or workforce profile according to needs. The WHO points out that efforts to scale up health professionals’ education must not only aim to increase the quantity of health workers, but must also address issues of quality and relevance to address population health needs (9). The increasing mobility of health workers and the implication of this for lifelong learning also needs to be considered.

Addressing faculty development and shortage of health professional educators

The shortage of health professional educators is one of the key constraining factors for scaling up health professionals’ education. This difficulty is exacerbated by the multitasking of health professionals, conflicting opportunities, different attitudes and approaches towards teaching, additional training requirements, and insufficient incentives. Strategies and formats for faculty development can vary widely, provided they are adapted to the specific needs of the country, institution and learner (10).

Enhancing competencies and skills development and optimizing health worker roles

The exponential progress in technology, diagnostic tools and treatment methods, as well as changing population demographics and disease burden, make updating and maintaining the knowledge and skills of health workers throughout their professional lives more important than ever (10). It is important for institutions and decision-makers to realize that investment in human resources and time is necessary to properly utilize digitalized systems for improving health care (11). The updating and development of health workforce competencies are crucial to support the shift of health systems towards integrated people-centred health services. Apart from this, in accordance with priorities highlighted in the GPW13 (12), an appropriate mix of skills is needed to improve healthy ageing, child and adolescent health, pregnancy and childbirth care, noncommunicable diseases and emergencies. Therefore, the right mix of continued professional development activities and competencies is needed to ensure an effective health workforce.

Preparing health workers for the future of digital health

Several health systems, especially in high-income countries, are heading for a future of health care that is driven significantly by digital means. For instance, the recent Topol review (13) states that within 20 years, 90% of all jobs in the United Kingdom’s National Health Service will require digital skills to some extent. This trend is likely to be a feature in economies of similar scale, especially for emerging economies. Therefore, a major approach to ensuring health workforce readiness is to embed the culture of lifelong learning in the development of digital health systems. Educators will need to be given change management and transformation support to adapt smoothly to digital learning systems. They need to be conversant with new methods of learning, that incorporate the science of AI and self-learning algorithms to ensure that learning content stays relevant, up to date, and is readily available to learners.

Digital education for building health workforce capacity – goals and policy aim:

• develop and implement evidence-based policies for transforming and scaling up health workforce education;

• develop accreditation, standards and regulation systems to certify and ensure quality of training;

• cover rural and remote areas with opportunities for health professional education to increase health worker retention in these hard to cover areas;

9

• use digital education to scale and improve education in the health professions;

• develop community-based training curricula for health workers;

• facilitate interprofessional education and collaborative practice;

• update health workers’ competencies and related curricula to meet the ever-changing and evolving population health needs;

• increase relevance of health professionals’ abilities;

• implement and promote continued professional development and lifelong learning.

10

3.1 Potential role of digital health education in the global agenda

The issues highlighted in the global health workforce agenda could be addressed by a range of measures that incorporate digital health technologies and tools. Box 1 below lists several potential capabilities

of digital education to transform the health workforce. These potential capabilities are contingent upon the successful implementation of digital education programmes as a driver. Hence, the effective use of digital education to train health workers is limited to certain situations and determined by learning objectives and contexts of application (e.g. whether being applied to pre- or in-service education).

3. Digital education for health workforce: evidence, potential role and future trends

Box 1. Potential of digital education for health professionals: possible impacts

� Increase student enrolment by making learning opportunities available to those who cannot physically attend classes for various

reasons, e.g. due to disabilities or distance form learning facility.

� Strengthen education and training capacity and optimize the roles and workloads (depending on the teaching activities) of educators,

so that shortages can be addressed. Simultaneously, digital education is attractive for employed health professionals for convenience

reasons and this can foster positive changes in their attitude towards increasing their learning.

� Remove geographical barriers by enabling the training of health workers in remote/isolated areas, and in community-based settings.

This can help reduce the geographical maldistribution of health workforce especially in settings where learning facilities are limited and

concentrated in urban locations. However, not all disciplines and skills can be learned exclusively through digital means.

� Improve resource use/efficiency by reducing the cost per learner (upfront cost of developing the course content versus delivery cost)

borne by the content providers. Digital education also has the potential to reduce other general costs as well as the opportunity costs to

learners in settings where education is acquired through a self-paced digital content course rather than traditional classroom modalities.

� Support lifelong learning by making relevant learning courses readily available for health workers who for valid reasons may find it

difficult to interrupt their work for significant amounts of time to go back to school. This will also enhance “on demand” learning by the

individual, i.e. learning in a time and place of their choosing, with its associated convenience.

� Improve learning outcomes by offering higher quality, standardized teaching with more breadth in clinical cases. Self-paced learning can

allow students more time for reflection/reading on areas where they may require more time for understanding.

� Foster peer teaching by facilitating communities of practice for peer teaching, knowledge sharing and interprofessional learning among

health workers.

� Improve gender balance and inclusion of hard-to-reach learner groups by breaking down social barriers that are sometimes

entrenched in physical systems.

11

In applying digital health interventions to workforce capacity building, it is important to consider some basic guiding principles that may apply. Findings from systematic reviews of peer-reviewed academic literature, including the conceptualization and operationalization of digital education, has revealed several challenges (14–22). These include a lack of clear categorization of digital education for health professionals in the literature, overarching terms with constantly evolving concepts, pedagogies or theories, as well as a variety of methods, features, technologies and/or delivery settings.

To map how digital education can address and complement the mandates and programmes for strengthening the global health workforce, a wider view of stakeholder contributions is needed. The involvement of educators, content experts, digital technology experts, methodological experts, IT specialists and statisticians, helps provide structure and clarity in this field. Defining the different types of educational modalities (Box 2), systematically reviewing the evidence, and identifying future trends is a sequential process, which has resulted in the findings outlined below.

3.2 Defining the landscape of digital education

A variety of definitions exist (23), hence working definitions and several conceptual frameworks of digital education have to be developed (14).

3.3 Structuring the digital education field

The field can be structured in different ways. For instance, digital education can be for specific:

• topics or health conditions (e.g. smoking cessation, diabetes management, domestic violence, antibiotic management);

• disciplines (e.g. dermatology, paediatrics, geriatrics, leadership and management);

• health worker roles (e.g. nurses, pharmacists, medical students, pre-registration (undergraduate) and post-registration (postgraduate) health care professionals);

• pedagogical approaches with relevant technology applications (e.g. digital problem-based learning, digital team-based learning); and

• type of outcomes (e.g. cognitive skills, such as diagnostic skills; procedural/psychomotor skills, such as surgical skills; or clinical procedures ability).

One approach to structuring digital education (including blended learning efforts) could be around clinical or public health competencies, which are often a combination of knowledge, skills and professional attitudes. For instance, effective communication

Box 2. Definitions of digital, traditional and blended education

Digital educationDigital education (also known as e-learning or digital learning) is defined as the act of teaching and learning by means of digital technologies

(14). Digital education is also characterized by specific pedagogies and instructional methods, contexts of provision and technical

affordances of hardware and software. Modalities of digital education range from the basic conversion of content into digital formats to

complex deployment of digital technologies (14) (Table 3.1).

Traditional educationTraditional education is any act of teaching and learning based on non-digital educational material (for example, textbook model); or in-

person human interaction (face-to-face learning). Traditional education in the form of face-to-face interaction can also include non-digital

and digital educational aids (e.g. images, charts, maps, objects, boards and videos) (14).

Blended educationBlended education (also known as blended learning) is the act of teaching and learning that integrates aspects of traditional and digital

education. Blended education can take diverse formats, depending on the type as well as the share of digital and traditional education

employed in the blended educational approach. The digital component of blended learning includes online digital education, as well as the

use of other digital education modalities. However, education delivered via in-person human interaction supported by digital educational aids

(e.g. images, charts, maps, objects, boards) is considered traditional education and not blended education (14).

12

between a health professional and a patient is a complex competency in the clinical context, which includes knowledge of ethics, psychology, and/or sociology, combined with interpersonal skills and attitudes. It is believed that significant levels of communication competencies can be delivered via digital education, e.g. through online digital education, virtual patients (VP) or virtual reality (VR) (24,27). On the other hand, training skills that involve physical contact with patients, such as auscultation, pose challenges that are difficult to address with digital education. Therefore, the type of health profession, the content required and the level of expertise will determine what curricula can be delivered in a fully digital or in a blended format (5).

3.4 Current evidence synthesis of digital education for health workers

The application of digital education for health workers, like any other intervention, requires rigorous and critical evaluation. Despite all the aforementioned challenges, the effectiveness of several digital health education modalities has been corroborated in this report. For instance, offline digital education may improve educational

outcomes, i.e. knowledge of medical students (n=37, where n represents the number of randomized controlled trials (RCTs) evaluating an educational intervention) (17), doctors (n=27) (15), and various allied health professions (n=27) (18). Studies suggest that digital education is at least as effective as traditional education in improving health professionals’ knowledge, skills, satisfaction with the educational approach and professional attitudes (28).

Digital education for health workers can be used as an adjunct to traditional education, usually in a blended learning model. However, the evidence is often inconclusive. For instance, it appears that end-to-end digital education is more effective in improving knowledge, whereas blended learning is more effective in improving skills, but this also depends on the learning objectives or assessment methods.

Studies on a variety of health care professions (postgraduates) and pre-registration, undergraduate students of health-related university degrees, or basic health-related vocational training programmes show professionals or students underwent a plethora of courses, modules, seminars etc., involving multiple academic topics or

Table 3.1 Digital education modalities and definitions

DIGITAL EDUCATION MODALITY WORKING DEFINITIONS/DESCRIPTION

Offline digital education

An intervention that requires no internet or local area network connection and can be delivered through external media including CD-ROM, external hard disc and USB stick (17).

Online digital education

An intervention that requires the use of a “transmission control protocol” (TCP) and an “internet protocol” (IP) as a standard for the learning activities; also referred to as “online”, “web-based” or “networked” (15). It is important to note that some online digital education modalities could have offline applicability. An example of this is cloud-syncable courses offered offline through platforms such as the Moodle open-source learning management system.

Serious gaming and gamification (SGG)

A competitive activity in which students are given educational goals intended to promote knowledge acquisition. The games may either be designed to promote learning or the development of cognitive skills, or they may take the form of simulations allowing learners to practise their skills in a virtual environment (19). Although these concepts may be considered distinct, they are grouped together as a broader category for the purposes of this report. More specifically, gamification can be defined as “the application of the characteristics and benefits of games to real world processes or problems” and serious games can be defined as games designed specifically for the “serious” purpose of providing health professional education via a digital device.

Massive open online course (MOOC)

An online course that is designed for the participation of large numbers of geographically dispersed students.

Virtual reality (VR) VR is a technology that allows the user to explore and manipulate computer-generated real or artificial three-dimensional (3D) multimedia sensory environments in real time. It allows for a first-person active learning experience through different levels of immersion, i.e. a perception of the digital world as real and the ability to interact with objects and/or perform a series of actions in this digital world (20).

Virtual patient (VP) Interactive computer simulations of real-life clinical scenarios for the purpose of medical training, education or assessment (24).

Digital psychomotor skills trainers (DPST)

An intervention involving training procedural skills that includes mental and motor activities required to execute a manual task (25).

Mobile learning (m-learning)

Learning across multiple contexts, through social and content interactions, using portable, networked devices (26).

13

conditions. However, it is unclear from the evaluated evidence whether digital education is more effective amongst postgraduates or undergraduates.

Most studies of digital education (of RCT standard) to date have targeted medical students or doctors (approximately 90%). The remaining 10% pertains to other health professions including nurses, midwives, dentists, physiotherapists, radiographers, pharmacists, audiologists, speech therapists, nutritionists, and optometrists/opticians. Given the importance highlighted by the programmes’ mandates, efforts should be placed on developing digital education initiatives to balance these proportions and increase the number of allied health professionals targeted.

The field is highly diverse in terms of evaluated technologies/platforms and these are sometimes “intertwined” with one another, e.g. VR or MOOC available via m-learning. Older technologies, such as DVD-ROMs or CD-ROMs, or other offline digital health education, such as distributing content via USBs, may still have an important role in settings and contexts with low or cost-prohibitive internet connectivity. Newer and emerging technologies have also been evaluated with similar outcomes. For example, SGG (n=30) (19), VR (n=31) (20) and VP scenarios (n=51) (24) show promise in improving educational outcomes of health professions.

Again, the conclusions are not definite and some gaps in the evidence have been identified, such as the lack of high-quality RCT studies from low- and middle-income countries (LMIC), and cost–effectiveness studies. Despite this, evidence from the main aggregation of findings (n=93 studies) suggests that online digital education already plays a noteworthy role in training medical doctors (15), significantly improving learning outcomes compared with self-directed or face-to-face learning. In the case of antibiotic management, digital education was shown to improve the prescription behaviours of practising physicians better than those who undertook only traditional learning (29). Although, these conclusions must be interpreted cautiously, as the quality of the evidence in the studies evaluated was often low. Blended learning may be more suitable for health care training, which commonly needs to combine practical hands on, skill-based training and self-directed digital education, as emphasized above.

Taken in context with the earlier mentioned considerations, digital technologies possess distinct advantages. For instance, the portability of some devices, e.g. mobile phones, makes m-learning unique compared with other forms of digital education for health professionals. Likewise, deploying MOOCs on such portable devices could offer greater accessibility to learning compared with other digital health education tools/modalities. Anyone with access to the

internet can enrol and learn. Most technologies have a high degree of interactivity between learners and the educational materials provided, and between fellow learners and educators, and this can be applied to health professional education. A high level of customization allows online digital education to be adjusted to learners’ needs, requirements and topics; while accessibility enables students to learn at any time and pace.

In general, the quality of the evidence was suboptimal (mostly low and very low). The included studies were often poorly reported, of limited validity and sometimes small in size. Furthermore, there were significant variations in the evaluated populations, interventions, comparator groups and outcomes evaluated. Overall, for most digital health education modalities except online digital education, future research is recommended to improve the completeness, consistency and applicability of the findings (17,18,20).

With uncertainty also comes opportunity – a myriad of questions remains unanswered in assessing the effectiveness of digital education for health workers (see Annex 2); and addressing these questions will guide future research and facilitate implementation. Besides effectiveness, other unanswered questions relating to scalability, sustainability and fidelity of digital education are included.

3.5 Future trends in digital education

Looking forward, digital education’s ability to overlap with a variety of educational approaches, concepts and methods, offers new avenues of acquiring and developing knowledge or skills.

Future trends in digital education include, but are not limited to:

• virtual, augmented, mixed reality in education for skills development, clinical scenario role playing and facilitation of procedures;

• personalized learning adapted to the health needs of individuals and populations;

• AI and intelligent tutoring systems facilitating the customization of learning experiences;

• redesigned learning spaces to accommodate the workforce approaches of future health professions;

• SGG for simulation and incentivized learning;

• utilizing social media or communication platforms, such as Facebook, WhatsApp, Skype, LinkedIn or Twitter, for knowledge sharing and interdisciplinary communication;

14

• big data and learning analytics (including regulatory and security issues); findings from learning analytics are useful to determine areas needing reinforcement, with a view to improve learning experience.

• MOOCs, virtual learning environments (VLEs) and learning management systems (LMSs) for training a larger audience as part of lifelong learning (continued medical development/continued professional development) programmes;

• developing standards for reporting digital health education intervention trials; and

• validity evidence on measurements of instruments used in digital health education trials.

These trends or innovative approaches to education have the potential to transform health professional training and provide a broader, more comprehensive picture. They have the potential to offer novel methods for bridging the educational divide between the developing and developed world but conversely may also not be feasible in resource-limited settings where basic infrastructure and training are lacking. As Box 3 highlights, the full potential of these digital trends will need to be explored and will require further investment not only financially but also in terms of future research, legislative, regulatory and administrative efforts.

Box 3. Evolving “big picture” outlook on implementing digital education for health workers

� Align standards of digital education and accreditation across health worker cultures, occupational groups and education systems.

Government ministries of education or other competent authorities should oversee and audit this process to ensure participation and

compliance by all stakeholders.

� Decision-makers in government authorities and ministries (i.e. education, health, finance technology, etc.) should promote collaboration by

engaging the private sector and other relevant stakeholders to increase the scale and reduce the costs of digital education.

� The monitoring (and portability where equivalent) of professional qualifications obtained through digital education is needed to ensure that

such workers are competent to provide high-quality health services.

� Learning contents and approaches to digital education should be flexible to allow for adaptations at regional/country levels according to

diverse learners’ needs and systems capacity.

� Digital education for health workers has relevance for both pre-service and lifelong learning (continuing professional development) and

should be deployed as such.

� Health systems need to embrace the culture of lifelong learning by putting in place educational systems and mechanisms that will allow

for seamless production and capacity building of digitally ready or deployable health workers. Such policy frameworks and action plans are

best operationalized through national digital health strategies as part of broader national health strategies and policies.

15

4. Health system integration of digital education: requirements and resources

The benefits of research on digital education for health workers can only be realized if a country or community is able to successfully incorporate relevant findings in their health systems. Integrating digital education into the planning and development of the broader health education system is an evolving process, which directly involves three of the six WHO health system building blocks: health

workforce, financing and leadership/governance (30). In order for this to occur, several health system components, including appropriate financing and governance, need to be mobilized across all levels, as described below. Fig. 4.1 presents a framework describing the different level elements of health workforce capacity building, and how it may impact health system outcomes.

Fig. 4.1 Conceptual framework for the use of e-learning for building health workforce capacity in improving health system outcomes

Source: Adapted from Tudor Car L, Kyaw BM, Atun R. The role of eLearning in health management and leadership capacity building in health system: a systematic review. Human Resources for Health. 2018;16(1):44 (31).

Health system outcomes:

patient health outcomes

population health outcomes

financial protection

user satisfaction

equity

efficiency

effectiveness

responsiveness

Health workforce competencies:

Health workforce structure:

Health workforce quantity

System-level capacity building: stewardship, infrastructure, governance, resource generation and allocation, policies, guidelines, strategic partnerships, human resources management, academic system, accreditation, licensing

Organizational-level capacity building: ownership, infrastructure, governance, financing, change management, organizational tools and standard operating systems, information technology systems, performance management, strategic collaborations, organizational training systems

Individual- and team-level capacity building: traditional education, e-learning, blended learning (curriculum design, pre-services and in-service training, formal degrees, workshops, courses, on-the-job learning, mentoring, coaching, job challenge, secondments, one-off training and continuous training)

External factors: health needs, support systems, environment (cultural, social, economic, political, legal)

Health workforce leadership and managemennt capacity building: strategic thinking, problem-solving, management, governance, leadership, political analysis, dialogue, community engagement, active learning, self- management skills

Health workforce technical capacity building: medical knowledge, patient-centred care, communication skills, professionalism. practice-based learning and improvement, team-based care, continuous learning and quality improvement, community engagement

16

In planning for the integration of ICT into health education and health systems, stakeholders are encouraged to ensure that a full understanding of the following are in place:

• What are the current and future needs, and what vision do we have in place for realizing the full impact of digital education for health workers?

• Who are the key players that should be involved in crafting and implementing this vision (32)?

A number of concepts and principles should also be considered (see Box 4).

4.1 External factors

Very wide disparities still persist in ICT access, both within countries and between countries (34,35). This disparity, commonly referred to as the “digital divide”, threatens individuals, groups and countries who have poor access to ICT. Digital inequalities are thought to mirror existing social inequalities and may result in vulnerable groups not reaping the same benefits from the technology as more privileged groups, therefore putting them at a further disadvantage (34). Gaps in access, however, are not just about digital connectivity and open information borders. Suber identifies four barriers (36),

all of which are likely to threaten e-learning interventions if not adequately addressed, even after open access has been achieved. These include:

• Filtering and censorship barriers: limitations by governments and other institutions as to what can be seen or accessed.

• Language barriers: language limitations of online materials; contents often available only in English.

• Handicap access barriers: websites and other technologies not offering disability access.

• Connectivity barriers: consumer digital technology and infrastructural access not available to people across different global income groups.

There are important external elements, above and beyond the reach of the relevant ministries and health systems, that need to be considered when evaluating the possibility of embedding digital education into the health system, including:

• population health and digital literacy;

• receptivity towards IT systems (31);

• presence of a supportive policy environment;

• political and government buy-in for digital education initiatives (to ensure long-term commitment) (37);

• strong technological supporting systems, including: internet capabilities, wireless technology, mobile applications and IT security, and implementation (33);

• physical and financial access to computers and internet: affordable low-cost internet access and computer technology with low user charges (33,38);

• culture and receptiveness of learning audiences to digital education is important to consider, just as they need to have trust and confidence in the effectiveness of digital education compared with other means that are available to them.

4.2 System-level factors

Requirements involving governmental structures, such as the ministries of health and education, the health system, and multisector collaboration, include:

Box 4. Considerations for scaling up digital education for health professionals

� Understand your health systems’ context and heath labour

market approaches.

� Apply people-centred principles and co-design with health

workers (users) to improve uptake.

� Use appropriate pedagogic frameworks including the use

of appropriate digital communication frameworks (with

the understanding that digital communication skills could

be different from those required for traditional classroom

training).

� Scale up investments and intersectoral cooperation.

� Understand knowledge as an ongoing, seamless product of

services and research, and digital technology as the engine for

continuous improvement.

� Build a shared learning environment and leverage existing

programmes and policies.

� Embed services and research into a continuous learning loop.

� Keep user barriers low and complexity incremental (33).

17

• Development of long-term plans, closely aligned to the national health goals and needs, which require: § national health (care) needs assessment, and how education

may be able to address them; § identification of priority areas for support; § planning of policies and guidelines; § a set of achievable goals with a corresponding timeline.

• Governance coordination and needs (33,39), including: § a shared vision and guiding principles; § definition of the participants’ roles and responsibilities; § stewardship of processes and protocols, including management

support for specific innovations, a plan on implementation phasing, and consistency and reliability in reporting for regulators.

• Appropriate funding, including: § adequate resource mobilization plans for resource generation

and allocation, which take into account cost of physical infrastructure, teacher training, content development, maintenance and ongoing support of all other aspects of operating an ICT system (37);

§ stimulation of government investment by digital health stakeholders demonstrating how digital health education can help address national health care issues (40);

§ demonstration that the deployment of e-learning in health has associated financial benefits (39).

• Development of infrastructure programmes and projects to support digital health education systems: § taking advantage of the systems already in place as opposed to

building new infrastructure (33); § having the appropriate technical support for the ongoing

development and operation of the selected innovations (39).

• Collaboration among different stakeholders, emphasizing active cooperation and role delineation among the many organizations, companies and agencies involved (33,37,40), including: § government ministries, for effective planning and

implementation, including setting up a coordinating agency between ministries and divisions;

§ involvement of health academic centres and health care delivery organizations, who have invested substantially in the creation of advanced digital resources;

§ health information vendors, who have emerged to meet the growing demand in virtually every aspect of health and health care;

§ multistakeholder alignment and coordination strategies for public, private and independent sector initiatives.

These partnerships among governments, technologists, nongovernmental organizations (NGOs), academia and industry are particularly relevant for LMIC as such pooling and alignment of the existing resources increases the likelihood of effective and sustainable programmes and initiatives.

• Academic system involvement for accreditation and licensing processes (31).

• Effective and systematic monitoring and evaluation of ICT systems to provide insights on the elements working well and those that need further development (37).

• Standardization of the quality of curricula and accreditation mechanisms to allow for a uniform assessment of different educational institutions. Similarly, standardizing user interfaces/formats could play an important part in user acceptance.

4.3 Institutional factors

Requirements at the level of academic and research institutions, IT companies, NGOs and other organizations involved in the reconceptualization of the pedagogical discourse (42), include:

Case study 1

It is important for countries to ensure deliberate investments

in operational mechanisms that will control how digital

technologies and data will be used to drive transformational

change. The Government of the United Kingdom and Northern

Ireland has learned from this and other international experiences

and has reprioritized digital maturity as well as clinical and data

science capabilities in the NHS through the creation of four

training organizations: the NHS Digital Academy to train chief

clinical information officers and chief information officers in

data science skills and leadership, the Farr Institute, Health Data

Research UK and the Alan Turing Institute.

These organizations are simultaneously developing academic

health data science capabilities on a national scale and working

with universities across the country and with partners such as

the life sciences industry to stimulate data-driven innovation

and job and wealth creation. This is already paying dividends by

helping the NHS embrace the technological changes required

to serve both its health workforce and population health needs

better in the area of digital health.

Source: Colclough et al, 2018 (41).

18

• Need for organizational-level ownership, infrastructure, governance, financing, change management, organizational tools and standard operating systems, IT systems, performance management, strategic collaborations, and organizational training systems, and finding the corresponding resources to enable all these processes (31).

• Development of projects and programmes for the roll out of teacher training, e-content development and administration (37): § Development of appropriate and actionable curriculum and

course contents based on already successful examples, understanding that a novel pedagogical approach that should be flexible, engaging and learner-centred.

§ Consideration of which courses can be delivered wholly online and which are the best to adopt a blended model of learning (38).

§ Human resources and teacher development, and clear administrative processes.

• Planning of educational modules, considering: § Type of educational module: pre-service and in-service training,

formal degrees, workshops, courses, on-the-job learning, mentoring, coaching, job challenge, secondments, one-off training and continuous training (31).

§ Health professional content to be delivered: medical knowledge, patient-centred care, communication skills, professionalism, practice-based learning and improvement, team-based care, continuous learning and quality improvement, community engagement.

• Planning and deployment of the appropriate technology, in particular, software and interface design (38), including: § Technical support and wider involvement in the development

and operation of the innovations over time. § Managing the stability, reliability, availability and affordability of

the required technology.

4.4 Individual factors

Considerations at the level of school administrators, teachers, students, etc. (42) include:

• Administrator-related factors for leadership and management, such as strategic thinking, problem-solving, political analysis and community engagement (31).

• Teacher-related factors, such as e-learning initiatives’ ability to save teachers’ time, training availability, ease of use, consistency of

Box 5. Nuances of digital education implementation

With traditional learning, a workshop for continuing health

education may involve a trip to a different city or country, a

couple of days off from work, extra payments for workshop

expenses, stipends, etc., in addition to the knowledge acquired.

If the traditional learning workshop is replaced with an

e-learning workshop, the learner will not receive some of the

perceived additional benefits of traditional learning workshops

and may even show dissatisfaction.

For the digital learning initiative to be successful, there needs

to be a motivated learner. Therefore, benefits such as incentives

and time off work should be considered. Additionally, the learner

should have a minimum level of digital literacy, and access to a

digital device and to the internet. It may make sense financially

for organizations or hospital administrators to replace traditional

training with digital learning, but they need to be aware of

learners’ related attitudes that come into play.

Case study 2

Hai Phong University of Medicine and Pharmacy, Viet Nam, has

recently developed and piloted e-learning modules for pre-

service and in-service medical education aimed at improving

the quality of and access to training for health workers,

particularly in primary health care in the northern coastal region

of the country. The project has adopted an innovative e-learning

approach and is funded by the governments of India, Brazil and

South Africa, with technical support from WHO and the Ministry

of Health, Viet Nam.

Essential for its success were effective collaboration and

planning to procure e-learning infrastructure in four remote sites

and to train faculty and adjunct staff on e-learning methods and

application. The pilot programme enabled: training delivery to 60

students, including on noncommunicable diseases; enrolment

of 600 students in the LMS; and connection across a network of

130 remote sites. An additional 600 students have been enrolled

in the LMS to date.

Accessibility, flexibility and adaptability of the learning approach

were some of the key positive aspects of the learners’

experience. The programme aims to build on the strong

commitment of all stakeholders involved and its alignment with

government priorities and strategic direction as a foundation for

further scale up.

Source: WHO, 2018 (43).

19

pedagogical approaches, wider adoption, and the formal recognition of the teacher’s role (39).

• Student-related factors and characteristics, such as: § learner’s age and gender are factors reported to make a

difference (38); § previous use of technology and level of digital literacy (39); § learner’s motivation, academic confidence and satisfaction also

play important roles in their success with e-learning courses (38,39);

§ learner’s economy and economic requirements for enrolment (38), and the possible inclusion of appropriate incentives;

§ e-learning initiatives’ ability to save students’ time, ease of use, improvement on student learning (39).

Several of the aspects highlighted in this section overlap and complement the big picture elements of digital health education uncovered by the evidence. Taking into account the needs and efforts identified from global mandates and programmes, the evidence supporting digital education for overcoming issues and improving the health workforce, and the requirements for its successful integration, several key messages sum up what is needed for digital education to achieve its full potential for health workforce capacity building.

20

5. Conclusion

Key messages

• Digital education strategies, if properly designed and implemented, have the potential to strengthen health workforce capacity in a number of areas, including: increasing student enrolment in health professions’ programmes, improving learning outcomes, delivering education to remote areas, enabling lifelong learning, facilitating communities of practice among health workers, and balancing the gender composition of health workforce.

• Digital education can be as effective as traditional education in terms of improving the knowledge and skills of health professionals, under certain circumstances and in the areas identified through the evidence reviewed. However, a combination of learning approaches in different contexts and environments will lead to optimal education outcomes. These combinations will require clear pedagogic and evaluation frameworks.

• There is evidence showing that m-learning, virtual patients and virtual reality may be more effective than other digital teaching modalities in terms of improving health professional skills; other technologies such as serious games, and virtual patients also hold promise in improving educational outcomes of health professions. It is still unclear from the available evidence whether digital education is, or how to make it, cost-effective, and how it affects health profession behaviours or patient outcomes.

• Scientifically, the field is fraught with challenges such as low methodological quality of evidence, often high or unclear risk of bias and high heterogeneity or low validity of the studies. These need to be addressed to ensure robustness of recommendations and ultimately cost–effectiveness of digital technologies for health professionals’ education. As a result, further research is needed to clearly determine which components of health education can be better delivered digitally, what role digital education can play alongside traditional education, what the cost-benefit is, and how learning outcomes can be assessed.

• The potential success of digital education to improve health workers’ competencies can be better realized if the digital divide facing countries with poor access to ICT is recognized and addressed in

a manner that promotes equity, rather than further disadvantaging those already struggling with socioeconomic and other hardships.

• The choice of digital education delivery should be driven by the context and availability of resources with a focus on scalability, sustainability, fidelity and adoption, rather than by the novelty of the interventions.

• Feasibility and acceptability of digital education interventions from the perspective of policy-makers, educators and students should be better understood.

Recommendations

Member States and other stakeholders are called upon to consider the following opportunities and recommendations in view of the background, evidence and key messages considered in this report:

1. Ensure that national digital heath strategies incorporate costed strategies and actionable policy options to strengthen health workers’ education and other capacities using digital means where favourable. Such policies and plans should be backed by the availability of functional and operational mechanisms coupled with technical leadership at policy-making levels and extending downstream to include support for faculty members and educators in health education institutions.

2. Undertake long-term evaluation of digital education programmes integrated into existing health professions’ curricula. There should be standardization of the follow-up duration for various digital education modalities with potential inclusion of deferred measurement to assess retention of knowledge. The incorporation of data analytics into evaluation and monitoring efforts to improve learners’ experience is useful in settings where feasible. Common evaluation and reporting guidelines should also be adopted, making sure to include other health professionals in addition to physicians (i.e. nurses, midwives, other allied health professionals).

3. Map current and planned programmes to strengthen health workers’ lifelong acquisition and maintenance of competencies using digital education, with a view to highlighting and prioritizing the delivery

21

of programmes where digital or blended formats offer greater effectiveness compared with other means. This is also a critical step to ensure that health workers are prepared for the future, to deliver health services in digital ecosystems where learning needs are constantly evolving and requiring quicker adaption times.

4. Invest in research to determine how digital education can be properly implemented, and which components of health

education are best delivered in a “pure” digital format versus a blended learning experience, as the latter may be more suitable for scenarios where hands-on training is required. The research objectives should include realist reviews that consider the relationship between contexts, mechanisms and outcomes in interventions; as well as qualitative evidence syntheses that consider experience of health worker education through e-learning, across various settings and circumstances.

22

1. Campbell J, Buchan J, Cometto G, David B, Dussault G, Fogstad H et al. Human resources for health and universal health coverage: fostering equity and effective coverage. Bull World Health Organ. 2013;91(11):853–63.

2. Anand S, Bärnighausen T. Health workers and vaccination coverage in developing countries: an econometric analysis. Lancet 2007;369(9569):1277–85 (https://www.sciencedirect.com/science/article/pii/S0140673607605996, accessed 22 November 2019).

3. Scheffler R, Cometto G, Tulenko K, Bruckner T, Liu J, Keuffel EL et al. Health workforce requirements for universal health coverage and the Sustainable Development Goals – Background paper No.1 to the WHO Global Strategy on Human Resources for Health: Workforce 2030. Hum Resour Health Obs. 2016;(17).

4. WHO. Global Strategy on Human Resources for Health: Workforce 2030. Geneva: World Health Organization; 2016; (https://apps.who.int/iris/bitstream/handle/10665/250368/9789241511131-eng.pdf?sequence=1, accessed 22 November 2019).

5. Health professions digital education: what works best, when, for whom and at what cost? J Med Internet Res. 2018; (https://www.jmir.org/themes/762, accessed 22 November 2019).

6. WHO. Working for health and growth: investing in the health workforce. Report of the High-Level Commission on Health Employment and Economic Growth. Geneva: World Health Organization; 2017.

7. WHO. Global diffusion of eHealth: making universal health coverage achievable. Report of the third global survey on eHealth. Geneva: World Health Organization; 2016.

8. Dussault G, Lapão L. The contribution of ehealth and mhealth to improving the performance of the health workforce: a review. Public Health Panorama. 2017;3(3):463–71 (https://novaresearch.unl.pt/en/publications/the-contribution-of-ehealth-and-mhealth-toimproving-the-performa, accessed 22 November 2019).

9. WHO. Transforming and scaling up health professionals’ education and training. Geneva: World Health Organization 2013.

10. WHO. Transforming and scaling up health professional education and training. Policy brief on faculty development. Geneva: World Health Organization; 2013.

11. Lapão LV. The future of healthcare: the impact of digitalization on healthcare services performance. In: Neto A, Flynn M, editors. The internet and health in Brazil. Switzerland: Springer; 2019;435–49.

12. WHO. The Thirteenth General Programme of Work, 2019–2023. Geneva: World Health Organization; 2018.

13. Topol E. Preparing the healthcare workforce to deliver the digital future. An independent review on behalf of the Secretary of State for Health and Social Care. London: Health Education England; 2019.

14. Car J, Carlstedt-Duke J, Tudor Car L, Posadzki P, Whiting P, Zary N et al. Digital education in health professions: The need for overarching evidence synthesis. J Med Internet Res. 2019;21(2):e12913 (http://www.jmir.org/2019/2/e12913/, accessed 22 November 2019).

15. George PP, Zhabenko O, Kyaw BM, Antoniou P, Posadzki P, Saxena N et al. Online digital education for postregistration training of medical doctors: systematic review by the Digital Health Education Collaboration. J Med Internet Res. 2019;21(2):e13269 (http:// www.ncbi.nlm.nih.gov/pubmed/30801252, accessed 22 November 2019).

16. Dunleavy G, Nikolaou CK, Nifakos S, Atun R, Law GCY, Tudor Car L. Mobile digital education for health professions: systematic review and meta-analysis by the Digital Health Education Collaboration. J Med Internet Res. 2019;21(2):e12937 (http://www.jmir.org/2019/2/e12937/, accessed 22 November 2019).

17. Kyaw BM, Posadzki P, Dunleavy G, Semwal M, Divakar U, Hervatis V et al. Offline digital education for medical students: systematic review and meta-analysis by the Digital Health Education Collaboration. J Med Internet Res. 2019;21(3):e13165 (http://www.jmir.org/2019/3/e13165/, accessed 22 November 2019).

References

23

18. Posadzki P, Bala MM, Kyaw BM, Semwal M, Divakar U, Koperny M et al. Offline digital education for postregistration health professions: systematic review and meta-analysis by the Digital Health Education Collaboration. J Med Internet Res. 2019;21(4):e12968 (http://www.ncbi.nlm.nih.gov/pubmed/31017584, accessed 22 November 2019).

19. Gentry SV, Gauthier A, L’Estrade Ehrstrom B, Wortley D, Lilienthal A, Tudor Car L et al. Serious gaming and gamification education in health professions: systematic review. J Med Internet Res. 2019;21(3):e12994 (http://www.jmir.org/2019/3/e12994/, accessed 22 November 2019).

20. Kyaw BM, Saxena N, Posadzki P, Vseteckova J, Nikolaou CK, George PP et al. Virtual reality for health professions education: systematic review and meta-analysis by the Digital Health Education Collaboration. J Med Internet Res. 2019;21(1):e12959. (http://www.ncbi.nlm.nih.gov/pubmed/30668519, accessed 22 November 2019).

21. Wutoh R, Boren SA, Balas AE. eLearning: a review of internet-based continuing medical education. J Contin Educ Health Prof. 2004;24(1):20–30 (http://www.ncbi.nlm.nih.gov/pubmed/15069909, accessed 22 November 2019).

22. Wang R, DeMaria S, Goldberg A, Katz D. A systematic review of serious games in training health care professionals. Simul Healthc. 2016;11(1):41–51 (http://www.ncbi.nlm.nih.gov/pubmed/26536340, accessed 22 November 2019).

23. Peek R. Web-based digital education opportunities. JJAL. 2000;26(1):49–52 (https://www.sciencedirect.com/science/article/abs/pii/S0099133399001238, accessed 22 November 2019).

24. Kononowicz AA, Woodham LA, Edelbring S, Stathakarou N, Davies D, Saxena N et al. Virtual patient simulations in health professions education: systematic review and meta-analysis by the Digital Health Education Collaboration. J Med Internet Res. 2019;21(7):e14676 (http://www.ncbi.nlm.nih.gov/pubmed/31267981, accessed 22 November 2019).

25. Foley RP Smilanksy J. Teaching techniques. A handbook for health professionals. New York (NY): McGraw Hill; 1980:71–91.

26. Crompton H. A historical overview of m-learning: toward learner-centered education. In: Berge ZL, Muilenburg LY, editors. Handbook of mobile learning. New York & London: Routledge; 2013:3–14.

27. Kron FW, Fetters MD, Scerbo MW, White CB, Lypson ML, Padilla MA et al. Using a computer simulation for teaching communication skills: a blinded multisite mixed methods randomized controlled trial. PEC. 2017;100(4):748–59 (https://www.sciencedirect.com/science/article/pii/S0738399116304943?via%3Dihub, accessed 22 November 2019).

28. Al-Shorbaji N, Atun R, Car J, Majeed A, Wheeler E. elearning for undergraduate health professional education: a systematic review informing a radical transformation of health workforce development. Geneva; World Health Organization; 2015. (https://www.who.int/hrh/documents/14126-eLearningReport.pdf, accessed 2 January 2020).

29. Kyaw BM, Tudor Car L, van Galen LS, van Agtmael MA, Costelloe CE, Ajuebor O et al. Health professions digital education on antibiotic management: systematic review and meta-analysis by the Digital Health Education Collaboration. J Med Internet Res. 2019;21(9):e14984 (http://www.jmir.org/2019/9/e14984/, accessed 22 November 2019).

30. WHO. Everybody’s business: strengthening health systems to improve health outcomes : WHO’s framework for action. Geneva: World Health Organization; 2007 (https://www.who.int/healthsystems/strategy/everybodys_business.pdf, accessed 22 November 2019).

31. Tudor Car L, Kyaw BM, Atun R. The role of eLearning in health management and leadership capacity building in health system: a systematic review. Hum Resour Health. 2018;16(1):44 (http://www.ncbi.nlm.nih.gov/pubmed/30176899, accessed 22 November 2019).

32. Park J. Trends and development of ICT in education policy. Knowledge sharing program on ICT in education. UNESCO; 2016 (https://events.development.asia/system/files/materials/2016/11/201611-trends-and-development-ict-educationpolicy.pdf, accessed 22 November 2019).

33. IOM (Institute of Medicine). Digital infrastructure for the learning health system: the foundation for continuous improvement in health and health care: workshop series summary. Washington (DC): National Academies Press; 2011.

34. Lutz C. Digital inequalities in the age of artificial intelligence and big data. Hum Behav Emerg Technol. 2019;1(2):141–8 (https://onlinelibrary.wiley.com/doi/abs/10.1002/hbe2.140, accessed 22 November 2019).

24

35. OECD. Bridging the digital divide. (http://www.oecd.org/site/schoolingfortomorrowknowledgebase/themes/ict/bridgingthedigitaldivide.htm, accessed 22 November 2019).

36. Suber P. Open access overview. 2014 (http://legacy.earlham.edu/~peters/fos/overview.htm, accessed 22 November 2019).

37. Chua C. ICT masterplans in the Singapore Education System. Singapore: Ministry of Education; 2016 (http://www.unesco.org/new/fileadmin/MULTIMEDIA/HQ/ED/images/singapore.pdf, accessed 22 November 2019).

38. Safie N, Aljunid S, Safie N, Aljunid S. E-Learning initiative capacity building for healthcare workforce of developing countries. J Comput Sci. 2013;9(5):583–91 (http://thescipub.com/abstract/10.3844/jcssp.2013.583.591, accessed 22 November 2019).

39. McGill TJ, Klobas JE, Renzi S. Critical success factors for the continuation of e-learning initiatives. Internet High Educ. 2014;22:24–36 (https://www.sciencedirect.com/science/article/pii/S1096751614000190?via%3Dihub, accessed 22 November 2019).

40. Long LA, Pariyo G, Kallander K. Digital technologies for health workforce development in low- and middle-income countries: a scoping review. Glob Heal Sci Pract. 2018;6(1):S41–8 (http://www.ncbi.nlm.nih.gov/pubmed/30305338, accessed 22 November 2019).

41. Colclough G, Dorling G, Riahi F, Ghafur S, Sheikh A. Harnessing data science and AI in healthcare: from policy to practice. Doha, Qatar: World Innovation Summit for Health. 2018 (https://www.wish.org.qa/wp-content/uploads/2018/11/IMPJ6078-WISH-2018-Data-Science-181015.pdf, accessed 22 November 2019).

42. So HJ. Shaping the future of education with technology: comparison of Korea and Singapore for ICT in education. ADB Knowledge Sharing Program on ICT in Education. 2016.

43. WHO. E-learning for health. WHO: WHO Western Pacific Region; 2018 (http://www.wpro.who.int/vietnam/mediacentre/features/E_learning_for_Health/en/, accessed 22 November 2019).

25

Annex 1: Key themes identified from WHO mandates, resolutions and reports, and representative quotes

KEY THEMES REFERENCES

Health workforce strengthening“Increase the quantity, quality and relevance of health professionals.” (from reference 3.)

1. Transforming and scaling up health professional education and training. WHO guidelines. WHO, 2013.2. General Assembly Resolution 70/1 (2015). Transforming our world: the 2030 Agenda for Sustainable

Development.3. World Health Assembly 69/19 (2016). Global Strategy on Human Resources for Health: Workforce 2030.4. Global Strategy on Human Resources for Health: Workforce 2030. WHO, 2016.5. Health workforce policies in OECD countries: right jobs, right skills, right places. OECD Health Policy

Studies. OECD, 2016.6. Health workforce requirements for universal health coverage and the Sustainable Development Goals.

Human Resources for Health Observer Series No. 17. WHO, 2016.7. World Health Assembly 71/4 (2018). Draft Thirteenth General Programme of Work, 2019–2023.8. Primary health care: transforming vision into action – operational framework (draft for consultation).

WHO/UNICEF, 2018.

Strengthening nursing and allied health professions“Nurses and midwives constitute more than 50% of the health workforce in many countries.” (from reference 1.)

1. Global strategic directions for strengthening nursing and midwifery 2016–2020. WHO, 2016.2. Final report of the expert group to the High-Level Commission on Health Employment and Economic

Growth. WHO, 2016.3. Maier CB, Aiken LH, Busse R. Nurses in advanced roles in primary care: policy levers for

implementation. OECD Health Working Papers, No. 98. OECD, 2017.

Health workers employment and labour marketCountries’ capacity to employ health workers determines the demand for health workforce, which is crucial for health systems and populations.

“Health workforce investments coupled with the right policy action could unleash enormous socioeconomic gains in quality education, gender equality, decent work, inclusive economic growth, and health and well-being.” (from reference 2.)

1. World Health Assembly 63/16 (2010). WHO Global Code of Practice on the International Recruitment of Health Personnel.

2. Ono T, Lafortune G, Schoenstein M. Health workforce planning in OECD countries: a review of 26 projection models from 18 countries. OECD Health Working Papers, No. 62. OECD, 2013.

3. OECD. Health workforce policies in OECD countries: right jobs, right skills, right places. OECD Health Policy Studies. OECD, 2016.

4. Improving employment and working conditions in health services: Report for discussion at the Tripartite Meeting on Improving Employment and Working Conditions in Health Services. Geneva, 24–28 April 2017. ILO, 2017.

5. World Health Assembly 70/6 (2017). Human resources for health and implementation of the outcomes of the United Nations’ High-Level Commission on Health Employment and Economic Growth.

6. Five-year action plan for health employment and inclusive economic growth (2017–2021). WHO, 2018.7. World Health Assembly 71/4 (2018). Draft Thirteenth General Programme of Work, 2019–2023.

Digital health“An important tool to help countries establish safe, efficient and sustainable health care delivery systems.” (from reference 1.)

1. World Health Assembly 66/24 (2013). eHealth standardization and interoperability.2. Global diffusion of eHealth: making universal health coverage achievable. Report of the third global

survey on eHealth. WHO, 2016.3. World Health Assembly 71/1 (2018). Digital health. Draft resolution.4. World Health Assembly 71/20 (2018). mHealth. Use of appropriate digital technologies for public health.5. Classification of digital health interventions. WHO, 2018.

Integrated people-centred health servicesThe new model of health care provision requires strengthening the health workforce to enable the shift from inpatient to comprehensive outpatient and ambulatory care and equipping it with an appropriate skills mix to meet population health needs and preferences.

1. World Health Assembly 69/24 (2016). Strengthening integrated, people-centred health services.2. World Health Assembly 69/39 (2016). Framework on integrated, people-centred health services. Report

by the Secretariat.

26

KEY THEMES REFERENCES

Monitoring and assessment of population health needs“The key components of health assessment tools should address the needs of the target population as well as reflect the competencies required by health workers and the ability of institutions to impart the necessary training and education.” (from reference 4.)

WHO launched initiatives to monitor and assess population health indicators and resources of national health systems to provide information to support policy development and implementation.

Main programmes and agencies include: � Global Health Observatory � National Health Workforce Accounts � Global Observatory for eHealth � Global Health Workforce Network (intersectoral coordination agency)

1. World Health Assembly 66/23 (2013). Transforming health workforce education in support of universal health coverage.

2. Transforming the health workforce in support of universal health coverage: a global toolkit for evaluating health workforce education. Report of the first meeting, 4–5 December 2013. WHO, 2013.

3. Assessing financing, education, management and policy context for strategic planning of human resources for health WHO, 2017.

4. World Health Assembly 70/6 (2017). Human resources for health and implementation of the outcomes of the United Nations’ High-Level Commission on Health Employment and Economic Growth.

5. National health workforce accounts: a handbook. WHO, 2017.6. World Health Assembly 71/4 (2018). Draft Thirteenth General Programme of Work, 2019–2023.

Strengthening education and training capacity“Efforts to scale up health professionals’ education must not only increase the quantity of health workers, but also address issues of quality and relevance in order to address population health needs.” (from reference 3.)

1. World Health Assembly 66/23 (2013). Transforming health workforce education in support of universal health coverage.

2. Transforming and scaling up health professional education and training. WHO guidelines. WHO, 2013.3. World Health Assembly 70/6 (2017). Human resources for health and implementation of the outcomes

of the United Nations’ High-Level Commission on Health Employment and Economic Growth.4. World Health Assembly 71/4 (2018). Draft Thirteenth General Programme of Work, 2019–2023.

Faculty development and addressing shortage of medical educators“Strategies and formats for faculty development can vary widely as long as they are adapted to the specific needs of the country, institution and learner.” (from reference 1.)

1. Transforming and scaling up health professional education and training. Policy brief on faculty development. WHO, 2013.

Enhancing competencies and skills development and optimizing health worker roles“The exponential progress in technology, diagnostic tools and treatment methods, as well as changing population demographics and disease burden, makes updating and maintaining the knowledge and skills of health workers throughout their professional life more important than ever.” (from reference 1.)

1. WHO eBook on integrating a social determinants of health approach into health workforce education and training. WHO, 2014.

2. Health workforce policies in OECD countries: right jobs, right skills, right places. OECD Health Policy Studies. OECD, 2016.

3. World Health Assembly 69/24 (2016). Strengthening integrated, people-centred health services.4. World Health Assembly 69/39 (2016). Framework on integrated, people-centred health services. Report

by the Secretariat.5. Maier CB, Aiken LH, Busse R. Nurses in advanced roles in primary care: policy levers for

implementation. OECD Health Working Papers, No. 98. OECD, 2017.6. World Health Assembly 71/4 (2018). Draft Thirteenth General Programme of Work, 2019–2023.

27

Annex 2: Questions on digital education requiring further research

• What is the economic impact of digital education?

• How does digital education affect patient-centred outcomes?

• How does the level of interactivity, accessibility, customization, learner control, cooperation, immersion, frequency, duration and intensity of delivery affect educational outcomes?

• How valid and reliable are the available measurement tools quantifying outcomes and how the outcomes are reported?

• Is digital education effective in low- to middle-income countries; and in different geographical and cultural settings? How can it be made such?

• Are learning outcomes transferable across different practice settings?

• How linked to established learning theories are digital educational interventions?

• What are the barriers and facilitators to successful implementation of digital education?

• How long does it take (and how much does it cost) to educate a health care professional using digital technologies/blended learning?

• Are there any adverse effects of the digital education to health workers or students?

• Which learning outcomes can be delivered using different digital technologies?

• What are the costs/benefits of different modes of digital instruction as they relate to health professional education?

• What other opportunities and aspects of health worker education can digital technologies be leveraged for? For example, curriculum planning, assessment, student records of achievements/portfolios, sharing case studies/best practice?

• What is the long-term effectiveness (retention) of health worker competencies gained through digital education?

• How does digital education affect behaviour change (change in clinical practice of health workers) outcomes?

Health Workforce DepartmentWorld Health Organization20 Avenue AppiaCH1211 Geneva 27 Switzerlandwww.who.int/hrh

ISBN 978-92-4-000047-6