Evidence synthesis – Preferred modalities for delivering continuing education to the public health workforce: a scoping review

Anya Archer, MPH; Isha Berry, MSc; Uttam Bajwa, PhD; Robyn Kalda, MES; Erica Di Ruggiero, PhD

https://doi.org/10.24095/hpcdp.40.4.03

This article has been peer reviewed.

Author references

Dalla Lana School of Public Health, University of Toronto, Toronto, Ontario, Canada

Correspondence: Anya Archer, Dalla Lana School of Public Health, 155 College Street, Toronto, ON  M5T 1P8; Email: anya.archer@alum.utoronto.ca

Abstract

Introduction: Continuing education (CE) can help public health professionals maintain and further develop their knowledge and skills to adapt to the changing public health landscape. This scoping review aims to identify the preferred modalities for delivering CE to public health professionals and to determine how equity has been incorporated into public health training.

Methods: Using the PRISMA extension for Scoping Reviews as a guide, we searched four databases for peer-reviewed primary research studies that evaluated public health workforce CE modalities.

Results: The review included 33 studies published between 1 January 2000 and 6 August 2019 from over 11 countries. Most articles broadly described their training audience as public health professionals employed by government or non-governmental organizations. Delivery methods included online, in-person or blended learning (combining online and in-person instruction). Learners strongly preferred self-directed approaches. Organizational support, including protected time for professional development during work hours, was an important enabler of training completion. Commonly cited barriers included course duration and a high number of contact hours.

Conclusion: Findings suggest that there is no single preferred training modality. We identified three elements that influence modality preference: design, delivery and organizational support. Modality should be determined by participants’ location, needs and previous experiences to ensure the content is relevant and delivered in a way that equips learners to apply the knowledge gained.

Keywords: public health, continuing education, training, capacity building, health human resources, professional development, scoping review

Highlights

  • It is important that the public health workforce has continuing education (CE) opportunities relevant to the context of their daily work and overarching organizational priorities.
  • CE opportunities should be equitably accessible.
  • Training that takes into account the context in which it will be delivered helps to create practical content. Such CE increases learners’ knowledge retention and their ability to apply new knowledge and skills in their professional settings.
  • Specific enablers of completion of CE programs are leadership support and protected time to participate in training.

Introduction

The Ottawa Charter for Health Promotion describes the purpose of public health as improving population health while enabling individuals’ increased control over their own health.Footnote 1 This requires the public health workforce to stay responsive to changing health needs while considering the social, cultural, environmental and economic contexts for good health.Footnote 2 As a result, it is important that public health professionals at all levels—frontline staff as well as middle and senior management—have the resources and organizational supports necessary to keep abreast of new developments in the field.

These resources and supports can be delivered in part through continuing education (CE) courses.Footnote 3 In Canada, for example, reviews of the public health system in the early 2000s called for a renewed commitment to public health at federal, provincial and territorial levels to strengthen core functions and prepare for the challenges of the 21st century.Footnote 4

While training is one way to stay current and meet these challenges, organizations that employ public health professionals are not always able to prioritize training due to resource and time constraints.Footnote 5 These factors must be taken into consideration during the planning and designing phases of training development.

Although public health professionals assume a variety of roles and responsibilities, the entire workforce requires a foundational understanding of population health and the social and ecological determinants of health.Footnote 6 The workforce must also be equipped with the competencies needed to collect and analyze population health data and evaluate interventions, particularly ones that reduce and eliminate health inequities.Footnote 6 These competencies include undertaking evidence-informed practice, using surveillance data and communicating information on risks to various audiences.Footnote 7

Public health professionals also need to understand and appreciate the sociopolitical and economic context in which they are working to ensure they do not exacerbate inequities.Footnote 8 As a result, equity-focussed training is a crucial priority.Footnote 6 Equity-focussed content refers to considering and presenting on how certain practices and ways of thinking in all types of public health work can amplify inequities, from collecting data to designing and evaluating interventions.Footnote 6 Training that incorporates concepts of equity questions the status quo to ensure diversity and inclusion are considered in all work.

CE can be delivered in a number of different ways—online, in-person and as a combination of the two, which is known as blended learning.Footnote 9 CE continues to evolve as technologies allow for more interaction with content and peers outside of a traditional classroom setting. CE should be delivered equitably, meaning that developers adopt an inclusive approach that reduces barriers to participation, for example, by offering training at times and places that are convenient.Footnote 10 This approach requires adapting to the audience’s accessibility needs.Footnote 11

We undertook a scoping review to assess the current state of literature on public health CE and to better understand which modalities professionals prefer when receiving CE on population health topics and approaches. The specific objectives of this scoping review were to identify preferred modalities to provide CE to public health professionals; identify the enablers and barriers to training; and determine how equity has been incorporated into public health training, including both equity as a training topic and its equitable delivery.

Methods

We developed a research question based on a preliminary review of the academic literature and consultation with public health workforce training experts. Our research question was as follows: “What are preferred delivery methods for training the public health workforce in population health topics and approaches?”

Using a scoping review methodology, we mapped concepts broadly, identified key sources and evaluated the types of evidence available in the public health field.Footnote 12Footnote 13 We used the PRISMA-ScR (PRISMA extension for Scoping Reviews) to guide reporting.Footnote 13

Data sources and searches

We searched four databases—MEDLINE (Epub Ahead of Print, In-Process & Other Non-Indexed Citations, Ovid MEDLINE® Daily and Ovid MEDLINE®); Embase (Embase Classic+Embase); CINAHL Plus with Full Text; and ERIC (ProQuest)—to identify all peer-reviewed primary research articles published between 1 January 2000 and 6 August 2019. The literature search was first performed on 17 August 2018 and repeated on 6 August 2019 to ensure inclusion of relevant articles published in the year since the original search. The search start date was chosen to best capture the changing context and scope of public health and the growing number of academic training programs.

We included original research (i.e. empirical research) that evaluated modalities, meaning the method for delivering training content. Furthermore, information and communication technologies have evolved rapidly in the 21st century, with Internet access expanding the ways CE programs can be delivered.Footnote 14 To ensure that the review was comprehensive, public health experts were consulted to identify additional articles.Footnote 12 The experts included public health professionals and academics with experience in developing CE; all were provided with our research question.

Search strategy

A focussed search strategy using a set of key search terms guided the scoping review. Table 1 lists the search terms used for MEDLINE and Embase. The search strategies for the other databases were similar, with formatting adaptations made based on their respective requirements (details available upon request). In keeping with scoping review methodology, search terms as well as inclusion and exclusion criteria were iteratively developed as the reviewers examined the literature.Footnote 12

Table 1. Search terms used in MEDLINE and Embase
Parameters Search term
Context Public health (Health ADJ1 promotion) OR (Population ADJ1 health) OR (public ADJ1 health)
Population Public health workforce (public health workforce) OR (Practitioner*) OR (Health personnel) OR (Health professional*) OR (Front#line public health practitioner*) OR (Community#health worker*) OR (Employee*) OR (Human resource*) OR (Staff) OR (Adult learner*)
Activity Training (training) OR (Professional ADJ2 development) OR (Continuing ADJ2 education) OR (Capacity ADJ2 building) OR (Competency#based ADJ2 education) OR (Education) OR (Technical ADJ2 assistance) OR (Staff ADJ2 development) OR (continuing#professional#development)
Focus Delivery method (delivery method*) OR (MOOC*) OR (Massive Open Online Course) OR (Online platform*) OR (Webinar*) OR (Workshop*) OR (Course*) OR (Module*) OR (online#learning*) OR (Institute*) OR (Community of practice) OR (peer#learning) OR (train#the#trainer) OR (study#groups) OR (hybrid ADJ1 learning)

Search terms fell into four key domains: public health (the context); the public health workforce (the population); the training activity; and delivery method.

Study selection and eligibility criteria

The results were exported into Covidence, a literature review management software program. Two reviewers (AA and IB) independently screened titles and abstracts and then screened full texts of the relevant articles. Differences between the reviewers were resolved through discussion until a consensus was reached.

We reviewed the full texts of studies that evaluated CE training programs delivered to the public health workforce. The populations of interest also included the medical and nursing sectors, non-governmental organizations and government departments. Articles were included if they were written in English and explicitly evaluated training delivery methods. Articles were excluded if they focussed on university-level education and training (i.e. courses for undergraduate or graduate degree programs); were literature reviews; or did not explicitly evaluate the modes of delivering training content (Table 2). The search was not restricted by geography.

Table 2. Inclusion and exclusion criteria for study articles
Parameter Inclusion criteria Exclusion criteria
Title/abstract screening Related to public health training
Published in English
Peer-reviewed primary research articles
Not related to public health training
Focussed on university-level education and training, i.e. courses for degree programs
Not published in English
Not peer reviewed
Literature reviews
Full text screening Related to public health training
Evaluated modes of delivering training
Evaluated the uptake, reach, participant feedback, training objectives reached and any external evaluations of the training
Focussed on university-level education and training, i.e. courses for degree programs
No discussion of the evaluation of modes of delivering training materials/content in sufficient detail
Not relevant
Full-text was not found/not available

Data charting process

In an abstraction form co-developed by all authors, three reviewers (AA, IB and RK) charted article characteristics (e.g. study funder, objectives); numbers of individuals completing the training; training populations (e.g. country, organization); development and delivery of training; topics and modalities of trainings (including contact hours); equity (whether considered in both delivery and content); evaluation methods and outcomes for training; and recommendations for future practice.

All authors piloted the extraction form by each charting data from one article, which informed the final version of the form. This descriptive-analytical method assisted the analysis and reporting of results by identifying standard information from each study for extraction.Footnote 12 Differences in abstraction were resolved through discussion. Consistent with the proposed scoping review methodology,Footnote 15 risk of bias for individual studies was not assessed.

Results

Of the 4251 articles identified through our search, 143 were selected for full-text screening, including 20 identified by the consulted experts. Of the 143 full-text articles identified, 33 met our inclusion criteria and were included in this review for evidence synthesis (Figure 1).

Figure 1. PRISMA-ScR flowchart of included and excluded studies

figure 1
Figure 1 - Text Equivalent
Figure 1. PRISMA-ScR flowchart of included and excluded studies
Level
(Top to Bottom)
Position (Left to Right) Text
1 1 Records identified through database searching (n = 3766)
1 2 Expert-identified articles (n = 20)
1 3 Additional articles identified in 2019 update (n = 465)
2 1 Total articles identified (n = 4251)
2 2 Duplicates removed (n = 424)
3 1 Title and abstracts screened (n = 3827)
3 2 Title and abstracts excluded (n = 3684)
4 1 Full-text articles assessed for eligibility (n = 143)
4 2 Full-text articles excluded (n = 109)aFootnote a; Full-text article not found (n = 1)
5 1 Studies included (n = 33)
Footnote a

Full-text excluded: Did not discuss evaluation of models of delivering training materials/content in sufficient detail (n = 76); not peer reviewed (n = 9); not original research (n = 1); focussed on university-level education and training (n = 6); not English (n = 4); not relevant to training the public health workforce (n = 12); article retracted (n = 1)

Return to footnote a referrer

a Full-text excluded: Did not discuss evaluation of models of delivering training materials/content in sufficient detail (n = 76); not peer reviewed (= 9); not original research (= 1); focussed on university-level education and training (n = 6); not English (= 4); not relevant to training the public health workforce (= 12); article retracted (= 1).

Public health training program characteristics

We identified public health training programs delivered in over 11 countries: Australia,Footnote 10Footnote 16Footnote 17 Bolivia,Footnote 18 Brazil,Footnote 19 Canada,Footnote 20Footnote 21Footnote 22 Greece,Footnote 23 the Islamic Republic of Iran,Footnote 24 Mexico,Footnote 25 Rwanda,Footnote 26Footnote 27 Uganda,Footnote 28 the United KingdomFootnote 29 and the United States of America (USA).Footnote 30Footnote 31Footnote 32Footnote 33Footnote 34Footnote 35Footnote 36Footnote 37Footnote 38Footnote 39Footnote 40Footnote 41Footnote 42Footnote 43Footnote 44Footnote 45 One program offered trainings in over 77 countries,Footnote 46 and another trained participants in Austria, Lithuania and the Netherlands.Footnote 9 Within the past 7 years, 19 of these articles were published.Footnote 9Footnote 16Footnote 18Footnote 19Footnote 20Footnote 21Footnote 23Footnote 25Footnote 26Footnote 27Footnote 28Footnote 29Footnote 34Footnote 39Footnote 40Footnote 43Footnote 44Footnote 45Footnote 46

A number of organizations developed non-degree public health training programs: universities/academic institutions, such as the University of North Carolina; non-profit organizations, such as the USA-based group Management Sciences for Health; and government agencies, such as the Public Health Agency of Canada. Organizations that developed training programs also often delivered the content to the intended audiences. We found that 36% (12/33) of the articles reported using frameworks and competencies; most of these used frameworks developed by governmental organizations such as the Centers for Disease Control and Prevention (USA) and the Public Health Agency of Canada (Canada) to inform the design and content of the trainings.

Online training included on-demand and live webinars, tele- or videoconferencing and course modules (a few short segments of the course offered at a time). In-person training included traditional didactic lectures, facilitated discussions and case studies (Table 3).

Table 3. Public health training program characteristics
Characteristics Articles, n (%) Reference
Year study published
 2000–2005 7 (21.0) 10, 32, 36-38, 41, 42
 2006–2011 7 (21.0) 17, 22, 24, 30, 31, 33, 35
 2012–2019 19 (58.0) 9, 16, 18-21, 23, 25-29, 34, 39, 40, 43-46
Organizations responsible for training development
Universities/academic institutions 13 (40.0) 9, 10, 24, 26, 29, 31, 32, 34-36, 38, 41, 43
Non-profit organizations 1 (3.0) 46
Government agencies 12 (36.0) 17, 19-22, 25, 27, 28, 33, 37, 42, 45
Universities and government agencies together 7 (21.0) 16, 18, 23, 30, 39, 40, 44
Use of frameworks and competencies
Yes 12 (36.0) 9, 20, 23, 27, 28, 30, 31, 35, 38, 40, 44, 45
No 21 (64.0) 10, 16-19, 21, 22, 24-26, 29, 32-34, 36, 37, 39, 41-43, 46
Intended training audience
Public health professionals employed by governments or health non-governmental organizations 25 (76.0) 10, 17, 20, 22, 23, 25-29, 31, 33-46
Health educators 2 (6.0) 9,18
Public health nurses 1 (3.0) 30
Medical professionals 1 (3.0) 19
A combination of the health professions listed above 4 (12.0) 16, 21, 24, 32
Training modalities
Online 9 (27.3) 17, 21, 25, 32, 33, 35-38
In-person 8 (24.2) 10, 16, 22, 24, 27, 28, 43, 44
Blended learning 13 (39.4) 9, 18-20, 26, 29-31, 34, 39, 41, 45, 46
Participant choice 3 (9.1) 23, 40, 42
Training topics
Public health approaches 11 (33.3) 10, 16, 23, 32, 33, 35, 39, 40, 43-45
Epidemiological methods 8 (24.2) 17, 20, 21, 30, 31, 36, 37, 42
Management and leadership 4 (12.1) 9, 24, 29, 46
Research principles 2 (6.1) 22, 41
Specific topics (injury prevention, emergency preparedness, global health, ethics, quality improvement, health and safety in hospitals) 8 (24.2) 18, 19, 25-28, 34, 38
Equity (in delivery of training)
Yes 6 (18.0) 19, 20, 25, 27, 38, 46
No 27 (82.0) 9, 10, 16-18, 21-24, 26, 28-37, 39-45
Equity (in topic)
Yes 8 (24.0) 10, 16, 18, 23, 24, 35-37
No 25 (76.0) 9, 17, 19-22, 25-34, 38-46

Online modalities tended to be shorter, from 20 minutes to 20 hours per module. In-person trainings were generally described as workshops and occurred over a full day or multiple days, often over a number of weeks and occasionally over the course of a year. Blended training programs included various components of these modalities, with the most common being online tutorials complemented by in-person facilitated group sessions. Individuals could choose between participating online or in-person.Footnote 23Footnote 40Footnote 42 The number of individuals being trained varied, with the smallest programs reaching 12 participantsFootnote 17Footnote 36 and the largest delivered to over 4000 participants.Footnote 46

Only six studies explicitly considered ways to deliver their training equitably. These six addressed equity by offering training at no cost to participants;Footnote 38 providing multiple methods of engaging with the material to allow participants with poor Internet connectivity to participate;Footnote 25Footnote 46 donating resources (e.g. computers) to participating sites to overcome technical barriers to participation;Footnote 19 or ensuring that content was adapted to reflect the local context and the participants’ level of literacy.Footnote 20Footnote 27 The other 27 studies did not specify whether equity was a consideration in designing the training programs. Only eight studies included trainings where issues around equity, diversity and inclusion were part of the course content.Footnote 10Footnote 16Footnote 18Footnote 23Footnote 24Footnote 35Footnote 36Footnote 37 Topics included reducing health inequities and instruction related to conducting research with vulnerable populations.

Evaluation of trainings

The objectives of and the methods used to gather evaluations varied across studies. Approaches to evaluation included surveys;Footnote 26Footnote 27Footnote 38Footnote 42Footnote 43 pre/post questionnaires;Footnote 30Footnote 31Footnote 33Footnote 41 participant completion statistics;Footnote 19 interviews;Footnote 28Footnote 45 and mixed methods (a combination of either focus groups or interviews and surveys or questionnaires with participants).Footnote 9Footnote 10Footnote 16Footnote 17Footnote 18Footnote 20Footnote 21Footnote 22Footnote 23Footnote 24Footnote 25Footnote 29Footnote 32Footnote 34Footnote 35Footnote 36Footnote 39Footnote 40Footnote 44

Evaluations after participation captured participants’ demographic information, subject knowledge before and after training, satisfaction with and post-training use of their newly gained knowledge. Post-training evaluations were administered anywhere from immediately after the training concluded—the majority of evaluations—to 6-months post-training.Footnote 41 Two studies did not describe their evaluation approach in detail.Footnote 37Footnote 46

Although the 33 included studies evaluated different aspects of their programs, we identified common enabling factors as well as barriers to completing training (Table 4). Where measured, participants generally rated high satisfaction with the training and noted increases in knowledge, perceived skill and self-efficacy. Self-efficacy was commonly defined by evaluators as increasing confidence in one’s ability to perform specific behaviours.Footnote 36

Table 4. Enablers and barriers to participating in and completing training identified across studies
Enablers to completing training Barriers to completing training
Interactive, hands-on training
Opportunity to collaborate and network with colleagues from other organizations
Self-directed online courses that allow for varying learning paces
Locally adapted content
Choice between various modalities
Certification upon completion
Organizational support (i.e. resources to complete training)
Technological challenges associated with online learning
Scheduling time to complete course work
Courses that lasted a year or longer
Lack of funding from organization to participate in training

Enablers to completing public health training

Participants generally preferred a hands-on, interactive, “learning by doing” approachFootnote 18Footnote 27Footnote 31Footnote 40Footnote 41 that included time for the practical application of content on projects relevant to participants’ roles within their organization.Footnote 22Footnote 36Footnote 46 When training did not take a practical approach, participants said the content was too focussed on theory and that they were not ready to apply the content in their workplace.Footnote 16Footnote 24Footnote 43

A commonly cited enabler to engaging with training content was the opportunity to meet colleagues within and outside of their organizations to discuss training materials.Footnote 17Footnote 18Footnote 22Footnote 24Footnote 36Footnote 46 Networking was perceived as a strong benefit of trainings that used this collaborative approach.

For online training programs, participants highly rated the opportunity to engage in training at a self-directed pace.Footnote 18Footnote 32Footnote 40Footnote 46 Participants also noted the convenience of online trainingFootnote 29 and onsite training,Footnote 17 with both decreasing travel time. Other factors that increased accessibility to public health professionals included content adapted to the context of the community in which the learners workFootnote 20Footnote 27Footnote 45 and when a variety of modalities was used to deliver the content.Footnote 9 One study noted participants’ preference for a formally certified course.Footnote 24 Employer support, for instance by providing tools such as computers and designated time to complete the training, was an essential factor in facilitating program completion.Footnote 10Footnote 19Footnote 20Footnote 22Footnote 25Footnote 44Footnote 45

Barriers to completing public health training

In the context of online training, technological challenges such as sound quality and network issues were cited as one of largest barriers to completing training,Footnote 18Footnote 20Footnote 32 especially in rural locations.Footnote 17Footnote 26 In one study, some training participants did not use computers in their daily work so they were not familiar with the technologies used for the course, which resulted in poor uptake.Footnote 19

The lack of time to work on modules or assignments while working full-time was another commonly reported barrier,Footnote 18Footnote 20Footnote 39Footnote 41Footnote 45 especially when planning group work.Footnote 22 This concern cut across the literature. Some participants found it difficult to continually engage with material during long training programs.Footnote 10Footnote 16Footnote 37 One study noted that a year-long course had a high drop-out rate.Footnote 28 Another study noted that participants preferred courses that were divided into shorter units.Footnote 29 A lack of employer funding to register for courses also inhibited participation.Footnote 16Footnote 39

Recommended practices

Most studies concluded with recommendations (i.e. lessons learned or best practices) for designing training based on their evaluation findings (Table 5). Recommendations included designing interactive sessions to facilitate collaboration between participants from different organizations; gaining employer support for staff participation; and adapting content to local contexts. In addition, studies recommended that training be delivered in short, easily digestible segments with support from technical specialists and be regularly evaluated for revisions and improvements. Other studies identified online learning as being the most flexible for participants and also recommended “train-the-trainer” approaches as preferred delivery methods within the public health workforce.

Table 5. Recommended practices based on evaluation results
Recommended practice Number of mentions Reference
Gaining employer support for staff to participate in training 11 10, 16, 19, 20, 22, 25, 38, 39, 41, 44, 45
Using interactive learning strategies with an opportunity for application throughout the training 6 18, 22, 27, 28, 34, 40
Adapting training content and delivery method to local context 5 20, 23, 24, 27, 45
Building teams or hosting collaborative training sessions across organizations 4 16, 18, 35, 39
Employing blended learning techniques where multiple modalities are used to deliver the content 4 9, 26, 30, 46
Dedicating a technical support person or providing pre-training course on how to access and use the online platform 4 17, 27, 35, 45
Shorter, just-in-time modules to better meet the evolving needs of public health professionals 3 10, 36, 37
Ongoing evaluation to revise and improve training 2 24, 40
Posting content online to provide participants with flexibility to complete the work when most convenient 1 18
Train-the-trainer approach as an effective delivery method to train the public health workforce 1 23

Discussion

We conducted a scoping review to identify the preferred delivery modes of training the public health workforce and the extent to which equity is considered in training content and delivery. Our findings indicate that while there is no single preferred delivery method for training public health professionals, online content allows for increased access to the material and opportunity to collaborate with individuals across organizations. We found that most of the training programs did not consider equity, but some paid attention to the equitable delivery of content. We also analyzed the enablers and barriers to completing the training to better understand what types of factors affect learners’ preference of training modality. From this analysis, we identified three elements that influence overall preference: design, delivery and organizational support.

Design

Although only one study mentioned the importance of conducting a needs assess­ment,Footnote 20 literature not included in the scop­ing review suggests that before designing a CE course, developers should conduct a comprehensive needs assessment to iden­tify suitable modes of delivery.Footnote 47Footnote 48 This process helps gauge participant readiness and ability to engage and assess issues to do with computer literacy and organiza­tions’ technological capacities.Footnote 47Footnote 48 With this information, appropriate and accessible considerations can be built into training upfront, rather than adapting delivery mode content after the fact.

Role of competencies in training design

Over one-third of the studies indicated that competencies were used when designing training. Many of these studies illustrated that anchoring content in existing competencies helped participants understand how their new skills and knowledge can be applied in their work.Footnote 23Footnote 30Footnote 31 Implementing common competencies across organizations helps to create a shared language and develops transferrable skills that are valued by employers.Footnote 49 By integrating these concepts and standards into CE, participants can be more confident in their increased ability to fulfill their agency’s goals and mandates.

Built-in feedback mechanisms

Our scoping review found that participants appreciated having their learning needs and feedback iteratively incorporated into the training as the course progressed. Having dedicated support allowed individuals to actively participate, highlighting the need for training programs that incorporate ongoing mechanisms for evaluation and continuous quality improvement.Footnote 40

Participatory approach

Reviewed studies highlighted problem-based learning as beneficial for applying the knowledge post training. Participants considered it highly desirable to develop tangible skills that are directly applicable in the workplace.Footnote 10Footnote 18Footnote 24Footnote 28Footnote 36Footnote 39Footnote 43 Using current, relevant issues or dilemmas that a professional might encounter allows them to engage with the training material in more depth and better understand how to apply the skills in their daily public health practice.Footnote 47

Participants preferred participatory approaches, including facilitating team-building and learning through collaboration with other health professionals. Some studies suggested that teams of two or three individuals from an agency should attend a course together, creating a “critical mass” of trained staff who subsequently disseminate the knowledge.Footnote 36Footnote 39Footnote 46 This approach enhances the likelihood of the skills being implemented and adopted throughout the organization.Footnote 36 Team learning also strengthens relationships between colleagues as they work towards a common goal.Footnote 46

Other trainings created teams of individuals across participating organizations to complete the training as a group. This method encourages resource sharing (e.g. of finances and staff time), especially for agencies with limited resources.Footnote 50 The development of interorganizational teams facilitates improved knowledge management and strengthens strategies for addressing population health across geographies.Footnote 48 Team-based learning that integrates a collaborative approach is essential to public health work and provides opportunities for the practical application of the skills learned in CE programs.

Equity

Equity should be an overarching objective for public health programs, not an independent component.Footnote 51 Our literature search did not identify many training programs that focussed on equity as a consideration in the design and delivery of training initiatives. There should be a stronger emphasis on incorporating equity into the assessment of participants’ needs. This can help guide the development and delivery of training content, as well as the evaluation of this work, so that best practices can be shared within the public health community. Incorporating equity into the delivery of training includes ensuring the participation of a diverse cross-section of the public health workforce, providing content in multiple languages as well as addressing potential barriers to people’s participation, particularly the engagement of underrepresented groups.Footnote 30Footnote 31Footnote 36

Delivery

Online training with a complementary in-person component was the most popular modality based on evaluation results. Options for engaging with online content have vastly expanded in recent years. The emphasis is on courses that are user-friendly and engaging.Footnote 40 Interactive exercises provide immediate feedback, assisting with knowledge retention.Footnote 52 Combining online content with in-person approaches allows participants to ask questions, collaborate and understand the content better than in more passive engagement (e.g. reading static content on a screen).Footnote 9Footnote 26Footnote 30Footnote 46

It is crucial that trainers adapt the delivery modality to the type and complexity of content while being mindful of the skills, goals and experiences of their intended audience.Footnote 18Footnote 20 Modality should be chosen based on content, participant capacity and resource availability. When learners have the autonomy to choose when and how to engage with material, known as self-directed learning, they are able to tailor their learning to their own professional goals and motivations.Footnote 53 This is an important aspect of adult-learning that allows individuals to decide their level of self-direction based on their own experiences, time constraints and learning goals.Footnote 47

Technological support

The availability of technological support to facilitate the timely resolution of technology-related challenges is important.Footnote 35 Before starting the course, participants should be familiar with the training modality, for instance via a website,Footnote 36 to minimize technical issues. In general, programs and the platforms need to be accessible and easy to use. When issues do arise, there should be an obvious way for individuals to obtain technical assistance.Footnote 17Footnote 18Footnote 20Footnote 35Footnote 37 Developers must ensure that the chosen delivery method is compatible with the technology available to potential participants wherever they are completing the training.

Organizational-level support

The design and delivery of a course is only as effective as the way in which it is promoted in the workplace. Based on our findings, managerial support, such as time to complete training during work hours and use of office equipment, enables staff to complete training.Footnote 10Footnote 16Footnote 19Footnote 20Footnote 22 In order to successfully recruit participants, those offering training programs must ensure that potential participants and their managers see the purpose and benefits of training. Certification (only mentioned in one studyFootnote 10) can incentivize completion for both participants and their managers when ideally aligned with competencies.Footnote 8 As shorter courses were generally more manageable for full-time staff to complete,Footnote 10Footnote 22Footnote 28Footnote 36Footnote 37Footnote 41 course developers should consider how much time is needed for participants to become certified in a specific topic.Footnote 8 Communicating incentives like certification early on can increase buy-in from all stakeholders.

Connected to managerial support, research on capacity building conducted by Joffres et al.Footnote 54 found that organizational-level support can promote the ability of participants to incorporate knowledge and skills gained into their daily work. An organization’s strategic direction begins at the senior management level, signalling to staff where priorities lie.Footnote 50 Organizational cultures that support the development of staff skills are essential for addressing emerging public health challenges in the face of rapidly changing contexts. As those in senior management positions are the stewards of systems-level change, their promotion of CE courses can help align the training offerings with organizational priorities so that training can be framed as part of existing work responsibilities, rather than as additional and new tasks that may burden an employee.Footnote 54 This approach to communicating the benefits of training also reflects an understanding of the context in which CE courses are offered and the value of using competencies to design relevant content. Senior and middle managers should work together towards a culture of professional development and organizational competence, motivating, incentivizing and supporting the public health workforce to engage in meaningful CE.Footnote 50

Limitations

We did not extend our search to grey literature despite that agencies that offer public health workforce training do not always publish in the peer-reviewed literature. Given our specific focus on training programs that were explicitly evaluated afterwards, we considered that a review of the peer-reviewed literature was appropriate.

Having restricted our search to start in the year 2000, we may be missing further training courses offered before this date.

In addition, we only included programs with evaluations. Although we may have missed some innovative programs that have yet to be evaluated, this inclusion criterion allowed us to capture participant-preferred modalities and better understand lessons learned and best practices identified.

Finally, this being a scoping review, we did not assess the methodological quality of evaluation findings within the individual studies.Footnote 12 Rather, we used the evaluations across the studies to assess the commonly reported results, both positive and negative, of the various ways of delivering CE content.

Conclusion

The aim of this scoping review was to identify preferred modalities used to deliver CE to the public health workforce while examining how equity (in the form of inclusive approaches to training and health equity as a content area) has been incorporated in training. Our findings suggest that while there is no preferred modality, using both in-person and online engagement strategies, when possible, is highly beneficial for learner knowledge retention and application of content. Adult-learning principles should be used to design training and engage public health professionals.Footnote 53 This includes preparing the course with the understanding that adult-learners are motivated by the desire to solve problems and know why they are learning before undertaking training.Footnote 47 Similarly, CE should be developed with special consideration for participants’ needs, previous experiences and technological and financial resources.Footnote 47

Only a few studies detailed strategies for delivering training equitably or teaching health equity as a content area. This highlights the need for greater emphasis to be placed on incorporating health equity into the design and delivery of CE material for this audience. Future research assessing the quality of literature on evaluating modalities employed would be beneficial to further understand the landscape of CE within the public health sector.

Acknowledgements

Vincci Lui, Information Specialist at the University of Toronto, provided consultation on the initial search strategy. Public Health Training for Equitable Systems Change (PHESC) partners provided input on the initial search strategy and additional relevant literature. Partners include Dalla Lana School of Public Health, University of Toronto; Alliance for Healthier Communities; National Collaborating Centre for Determinants of Health; National Collaborating Centre for Healthy Public Policy; National Collaborating Centre for Methods and Tools; Ontario Public Health Association; Public Health Ontario; and Wellesley Institute.

Funding

This paper was made possible in part thanks to funding from a Ministry of Health and Long-Term Care (Ontario) Health and Well-Being Grant. The funders had no role in study design, data collection and analysis, decision to publish or preparation of the manuscript.

Conflict of interest

The contributing authors declare no potential conflicts of interest with respect to the research, authorship and/or publication of this article.

Authors’ contributions and statement

Conceptualization and design: AA, IB, UB, RK, EDR; acquisition of data: AA, IB; pilot of data charting: AA, IB, UB, RK, EDR; data charting: AA, IB, RK; formal analysis and interpretation of data: AA, IB; writing – original draft: AA, IB; writing – review and editing: AA, IB, UB, RK, EDR; writing – journal revisions: AA, IB, UB, RK, EDR; final approval: AA, IB, UB, RK, EDR; project administration: AA.

The content and views expressed in this article are those of the authors and do not necessarily reflect those of the Government of Canada.

References

Footnote 1

World Health Organization. Ottawa charter for health promotion. Ottawa (ON); 1986.

Return to footnote 1 referrer

Footnote 2

DeSalvo KB, Wang YC, Harris A, Auerbach J, Koo D, O'Carroll P. Public Health 3.0: A call to action for public health to meet the challenges of the 21st century. Prev Chronic Dis. 2017;14:E78. doi:10.5888/pcd14.170017.

Return to footnote 2 referrer

Footnote 3

DeCorby-Watson K, Mensah G, Bergeron K, Abdi S, Rempel B, Manson H. Effectiveness of capacity building interventions relevant to public health practice: a systematic review. BMC Public Health. 2018;18(1):684. doi:10.1186/s12889-018-5591-6.

Return to footnote 3 referrer

Footnote 4

National Advisory Committee on SARS and Public Health, Naylor D. Learning from SARS: renewal of public health in Canada: a report of the National Advisory Committee on SARS and Public Health. Ottawa (ON): Health Canada; 2003.

Return to footnote 4 referrer

Footnote 5

Mowat D, Moloughney BW. Developing the public health workforce in Canada: a summary of regional workshops on workforce education and training. Can J Public Health. 2004;95(3):186-7.

Return to footnote 5 referrer

Footnote 6

Gonzales G, Quinones N, Martin M. Health equity curricula within health policy and management concentrations in US public health graduate programs. Pedagogy Health Promot. 2018;5(4):276-82. doi:10.1177/2373379918814024.

Return to footnote 6 referrer

Footnote 7

Ministry of Health and Long-Term Care. Ontario public health standards: requirements for programs, services, and accountability (standards). Toronto (ON): Ministry of Health and Long-Term Care; 2018.

Return to footnote 7 referrer

Footnote 8

Lichtveld MY, Cioffi JP. Public health workforce development: progress, challenges, and opportunities. J Public Health Manag Pract. 2003;9(6):443-50. doi:10.1097/00124784-200311000-00003.

Return to footnote 8 referrer

Footnote 9

Könings KD, de Jong N, Lohrmann C, et al. Is blended learning and problem-based learning course design suited to develop future public health leaders? An explorative European study. Public Health Rev. 2018;39(1):13. doi:10.1186/s40985-018-0090-y.

Return to footnote 9 referrer

Footnote 10

Keleher H, Round R, Marshall B, Murphy B. Impact evaluation of a five-day Short Course in Health Promotion: workforce development in action. Health Promot J Austr. 2005;16(2):110-5. doi:10.1071/HE05110.

Return to footnote 10 referrer

Footnote 11

Thomson G. AODA training requirements: who needs it and why? Accessibility for Ontarians with Disabilities Act; 2018 [accessed 2019 Aug 30]. Available from: https://www.aoda.ca/aoda-training-requirements-who-needs-it-and-why/

Return to footnote 11 referrer

Footnote 12

Arksey H, O'Malley L. Scoping studies: towards a methodological framework. Int J Soc Res Methodol. 2005;8(1):19-32. doi:10.1080/1364557032000119616.

Return to footnote 12 referrer

Footnote 13

Tricco AC, Lillie E, Zarin W, et al. PRISMA extension for scoping reviews (PRISMA-ScR): checklist and explanation. Ann Intern Med. 2018;169(7):467-73. doi:10.7326/M18-0850.

Return to footnote 13 referrer

Footnote 14

Narain JP, Ofrin R. Role of modern technology in public health: opportunities and challenges. WHO South-East Asia J Public Health. 2012;1(2):125-7. doi:10.4103/2224-3151.206924.

Return to footnote 14 referrer

Footnote 15

Peters MD, Godfrey CM, Khalil H, McInerney P, Parker D, Soares CB. Guidance for conducting systematic scoping reviews. Int J Evid-Based Healthc. 2015;13(3):141-6. doi:10.1097/XEB.0000000000000050.

Return to footnote 15 referrer

Footnote 16

Naccarella L, Greenstock L, Butterworth I. Evaluation of population health short courses: implications for developing and evaluating population health professional development initiatives. Aust J Prim Health. 2016;22(3):218-25. doi:10.1071/PY14140.

Return to footnote 16 referrer

Footnote 17

Naylor CJ, Madden DL, Neville L, Oong DJ. Pilot study of using a web and teleconference for the delivery of an Epi Info training session to public health units in NSW, 2005. New South Wales Public Health Bulletin Supplementary Series. 2009;20(2):22-37. doi:10.1071/NB09S22.

Return to footnote 17 referrer

Footnote 18

Aalborg A, Sullivan S, Cortes J, Basagoitia A, Illanes D, Green M. Research ethics training of trainers: developing capacity of Bolivian health science and civil society leaders. Acta Bioeth. 2016;22(2):281-91. doi:10.4067/S1726-569X2016000200015.

Return to footnote 18 referrer

Footnote 19

Bussotti EA, Leite MT, Alves AC, Cristensen K. Online training for health professionals in three regions of Brazil. Rev Bras Enferm. 2016;69(5):981-5. doi:10.1590/0034-7167.2016690506.

Return to footnote 19 referrer

Footnote 20

Bell M, MacDougall K. Adapting online learning for Canada's Northern public health workforce. Int J Circumpolar Health. 2013;72(1):1-7. doi:10.3402/ijch.v72i0.21345.

Return to footnote 20 referrer

Footnote 21

Chan L, Mackintosh J, Dobbins M. How the “Understanding Research Evidence” web-based video series from the National Collaborating Centre for Methods and Tools contributes to public health capacity to practice evidence-informed decision making: mixed-methods evaluation. J Med Internet Res. 2017;19(9):e286.
doi:10.2196/jmir.6958.

Return to footnote 21 referrer

Footnote 22

Mathews M, Lynch A. Increasing research skills in rural health boards: an evaluation of a training program from Western Newfoundland. Can J Program Eval. 2007;22(2):41-56.

Return to footnote 22 referrer

Footnote 23

Andrioti D, Charalambous G, Skitsou A, Kyriopoulos J. An international partnership interdisciplinary training programme on public health. Health Sci J. 2015;9(2):1.

Return to footnote 23 referrer

Footnote 24

Omar M, Gerein N, Tarin E, Butcher C, Pearson S, Heidari G. Training evaluation: a case study of training Iranian health managers. Hum Resour Health. 2009;7(1):20. doi:10.1186/1478-4491-7-20.

Return to footnote 24 referrer

Footnote 25

Magana-Valladares L, Rosas-Magallanes C, Montoya-Rodriguez A, Calvillo-Jacobo G, Alpuche-Arande CM, García-Saisó S. A MOOC as an immediate strategy to train health personnel in the cholera outbreak in Mexico. BMC Med Educ. 2018;18(1):111. doi:10.1186/s12909-018-1215-1.

Return to footnote 25 referrer

Footnote 26

Warugaba C, Naughton B, Gauthier BH, Muhirwa E, Amoroso CL. Experience with a massive open online course in rural Rwanda. Int Rev Res Open Dis. 2016;17(2). doi:10.19173/irrodl.v17i2.2401.

Return to footnote 26 referrer

Footnote 27
 

Muhimpundu MA, Joseph KT, Husain MJ, et al. Road map for leadership and management in public health: a case study on noncommunicable diseases program managers' training in Rwanda. Int J Health Promot Educ. 2019;57(2):82-97. doi:10.1080/14635240.2018.1552178.

Return to footnote 27 referrer

Footnote 28

Matovu JK, Wanyenze RK, Mawemuko S, Okui O, Bazeyo W, Serwadda D. Strengthening health workforce capacity through work-based training. BMC Int Health Human Rights. 2013;13(1):8. doi:10.1186/1472-698X-13-8.

Return to footnote 28 referrer

Footnote 29

de Jong N, Könings KD, Czabanowska K. The development of innovative online problem-based learning: a leadership course for leaders in European public health. J Univ Teach Learn Pract. 2014;11(3):3.

Return to footnote 29 referrer

Footnote 30

Alexander LK, Dail K, Horney JA, et al. Partnering to meet training needs: a communicable-disease continuing education course for public health nurses in North Carolina. Public Health Rep. 2008;123(2):36-43. doi:10.1177/00333549081230S206.

Return to footnote 30 referrer

Footnote 31

Baseman JG, Marsden-Haug N, Holt VL, Stergachis A, Goldoft M, Gale JL. Epidemiology competency development and application to training for local and regional public health practitioners. Public Health Rep. 2008;123(1 Suppl 1):44-52. doi:10.1177/00333549081230S111.

Return to footnote 31 referrer

Footnote 32

Bernhardt JM, Runyan CW, Bou-Saada I, Felter EM. Implementation and evaluation of a Web-based continuing education course in injury prevention and control. Health Promot Pract. 2003;4(2):120-8. doi:10.1177/1524839902250758.

Return to footnote 32 referrer

Footnote 33

Davis MV, Fernandez CP, Porter J, McMullin K. UNC certificate program in core public health concepts: lessons learned. J Public Health Manag Pract. 2006;12(3):288-95. doi:10.1097/00124784-200605000-00010.

Return to footnote 33 referrer

Footnote 34

Davis MV, Vincus A, Eggers M, et al. Effectiveness of public health quality improvement training approaches: application, application, application. J Public Health Manag Pract. 2012;18(1):E1-E7. doi:10.1097/PHH.0b013e3182249505.

Return to footnote 34 referrer

Footnote 35

Demers AL, Mamary E, Ebin VJ. Creating opportunities for training California's public health workforce. J Contin Educ Health Prof. 2011;31(1):64-9. doi:10.1002/chp.20102.

Return to footnote 35 referrer

Footnote 36

Farel A, Umble K, Polhamus B. Impact of an online analytic skills course. Eval Health Prof. 2001;24(4):446-59. doi:10.1177/01632780122035019.

Return to footnote 36 referrer

Footnote 37

Farel AM, Polhamus B. Ensuring public health professionals go the distance: a case study from an online data skills course. J Public Health Manag Pract. 2001;7(3):82-5. doi:10.1097/00124784-200107030-00013.

Return to footnote 37 referrer

Footnote 38

Horney JA, MacDonald P, Rothney EE, Alexander LK. User patterns and satisfaction with on-line trainings completed on the North Carolina Center for Public Health Preparedness Training Web Site. J Public Health Manag Pract. 2005;11(6):S90-4. doi:10.1097/00124784-200511001-00015.

Return to footnote 38 referrer

Footnote 39

Jacobs JA, Duggan K, Erwin P, et al. Capacity building for evidence-based decision making in local health departments: scaling up an effective training approach. Implement Sci. 2014;9(1):124. doi:10.1186/s13012-014-0124-x.

Return to footnote 39 referrer

Footnote 40

Kenefick HW, Ravid S, MacVarish K, et al. On your time: online training for the public health workforce. Health Promot Pract. 2014;15(1 Suppl):48S-55S. doi:10.1177/1524839913509270.

Return to footnote 40 referrer

Footnote 41

Steckler A, Farel A, Bontempi JB, Umble K, Polhamus B, Trester A. Can health professionals learn qualitative evaluation methods on the World Wide Web? A case example. Health Educ Res. 2001;16(6):735-45. doi:10.1093/her/16.6.735.

Return to footnote 41 referrer

Footnote 42

Umble KE, Cervero RM, Yang B, Atkinson WL. Effects of traditional classroom and distance continuing education: a theory-driven evaluation of a vaccine-preventable diseases course. Am J Public Health. 2000;90(8):1218-24. doi:10.2105/AJPH.90.8.1218.

Return to footnote 42 referrer

Footnote 43

Yarber L, Brownson CA, Jacob RR, et al. Evaluating a train-the-trainer approach for improving capacity for evidence-based decision making in public health. BMC Health Serv Res. 2015;15(1):547. doi:10.1186/s12913-015-1224-2.

Return to footnote 43 referrer

Footnote 44

Allen P, O'Connor JC, Best LA, Lakshman M, Jacob RR, Brownson RC. Management practices to build evidence-based decision-making capacity for chronic disease prevention in Georgia: a case study. Prev Chronic Dis. 2018;15:E92. doi:10.5888/pcd15.170482.

Return to footnote 44 referrer

Footnote 45

Brownson CA, Allen P, Yang SC, Bass K, Brownson RC. Scaling up evidence-based public health training. Prev Chronic Dis. 2018;15:E145. doi:10.5888/pcd15.180315.

Return to footnote 45 referrer

Footnote 46

Chio KS. Effective practices in providing online, in-service training to health professionals in low-resource settings. Int J Train Dev. 2012;16(3):228-34. doi:10.1111/j.1468-2419.2012.00406.x.

Return to footnote 46 referrer

Footnote 47

Bryan RL, Kreuter MW, Brownson RC. Integrating adult learning principles into training for public health practice. Health Promot Pract. 2009;10(4):557-63. doi:10.1177/1524839907308117.

Return to footnote 47 referrer

Footnote 48

Ng E, de Colombani P. Framework for selecting best practices in public health: a systematic literature review. J Public Health Res. 2015;4(3)577. doi:10.4081/jphr.2015.577.

Return to footnote 48 referrer

Footnote 49

Bryant B, Ward M. A strategic approach to workforce development for local public health. Can J Public Health. 2017;108(4):e403-8.

Return to footnote 49 referrer

Footnote 50

Peirson L, Ciliska D, Dobbins M, Mowat D. Building capacity for evidence informed decision making in public health: a case study of organizational change. BMC Public Health. 2012;12(1):137. doi:10.1186/1471-2458-12-137.

Return to footnote 50 referrer

Footnote 51

Potvin L, Jones CM. Twenty-five years after the Ottawa Charter: the critical role of health promotion for public health. Can J Public Health. 2011;102(4):244-8. doi:10.1007/BF03404041.

Return to footnote 51 referrer

Footnote 52

Abd-Hamid NH, Walkner L. Evidence-based best practices in designing and developing quality eLearning for the public health and health care workforce. Pedagog Health Promot. 2017;3(1S):35S-39S.

Return to footnote 52 referrer

Footnote 53

Merriam SB. Andragogy and self-directed learning: pillars of adult learning theory. New Dir Adult Contin Educ. 2001;2001(89):3-14. doi:10.1002/ace.3.

Return to footnote 53 referrer

Footnote 54

Joffres C, Heath S, Farquharson J, Barkhouse K, Latter C, MacLean DR. Facilitators and challenges to organizational capacity building in heart health promotion. Qual Health Res. 2004;14(1):39-60. doi:10.1177/1049732303259802.

Return to footnote 54 referrer

Page details

Date modified: