Evaluation of National Microbiology Laboratory 2014-15 to 2018-19

Cover thumbnail

Download the alternative format
(PDF format, 780 KB, 34 pages)

Organization: Public Health Agency of Canada

Published: February 2021

Prepared by
the Office of Audit and Evaluation
Health Canada and the Public Health Agency of Canada

February 2021

Table of Contents

List of Figures

List of Acronyms

US CDC
United States Centers for Disease Control and Prevention
CCDIC
Centre for Immunization and Respiratory Infectious Diseases
CFEZID
Centre for Food-borne, Environmental and Zoonotic Infectious Diseases
CFIA
Canadian Food Inspection Agency
CIHR
Canadian Institute for Health Research
CIRID
Centre for Immunization and Respiratory Infectious Diseases
CNPHI
Canadian Network for Public Health Intelligence
CPHLN
Canadian Public Health Laboratory Network
HSIB
Health Infrastructure and Security Branch
IDPC
Infectious Disease Prevention and Control Branch
IRIDA
Integrated Rapid Infectious Disease Analysis
IT
Information Technology
NML
National Microbiology Laboratory
PHAC
Public Health Agency of Canada
WHO
World Health Organization

Executive Summary

Program Context

As part of the Public Health Agency of Canada (PHAC), the National Microbiology Laboratory (NML) works with provincial, national, and international public health and laboratory partners to prevent the spread of infectious diseases. It does so by conducting laboratory-based surveillance and applied and discovery research, as well as helping to prepare for public health emergencies and responding to outbreaks. The NML is also part of multiple networks. One of its key functions is to provide leadership and capacity development in the area of infectious diseases, both domestically and internationally.

The NML's main laboratory, the Canadian Science Centre for Human and Animal Health facility in Winnipeg (Manitoba) houses Canada's only Containment Level 4 laboratories in Canada. The NML has additional laboratories in Guelph (Ontario), and Saint-Hyacinthe (Quebec).

The NML had a budget of about $80M and 550 employees in 2018-19.

About this evaluation: This evaluation and the related management response and action plan were completed prior to the COVID-19 outbreak in Canada. While reviewing the findings of this evaluation, readers should keep in mind that the Canadian response to the pandemic has significantly affected the activities of the NML, as well as the context in which it operates.

Key Findings

The NML is a highly credible and relevant organization. It is well valued and recognized by its external and internal partners for its ability to provide core services to the public health sector and to innovate in order to remain relevant and effective at accomplishing its mandate of protecting Canadians against infectious diseases.

The changing laboratory environment (i.e., transition to "omics technologies"), and emerging public health problems (e.g., climate change, antimicrobial resistance, new infectious diseases) are two pressures faced by the NML. To address these challenges, the NML has been adaptive and has innovated to modernize its laboratory equipment and techniques. It has also implemented new informatics, data-sharing solutions, and diagnostic approaches. It has conducted discovery research that led to the creation of novel vaccines and improved its ability to diagnose pathogens.

Innovation is, however, not without challenges, as it requires NML scientists to use new skill sets and adapt their IT infrastructure. The transfer of innovation requires the NML to support its partners in developing capacity (e.g., helping to acquire equipment, providing training and access to IT platforms). In a context where resources are limited, the NML and PHAC, as a whole, also have to find the right balance between creating the space necessary to foster a culture of innovation, while meeting core public health functions and priorities. Currently, the NML does not have consistent mechanisms in place to prioritize its research activities, but at the time of the evaluation, work was underway to improve the planning of NML's research activities. Systematic planning of research activities also occurred in selected groups within the laboratory.

Collaborating with a range of internal and external partners allows the NML to focus its work on areas of relevance for public health and gather intelligence on upcoming priorities or threats. Collaboration with external partners is well established, but internal collaboration with PHAC Centres tends to be more ad hoc. Both the NML and PHAC Centres perceived that more work is required to improve communication and collaboration at the working level. Collaboration was successful in some cases where there was a more formalized process in place to support ongoing working relationships between PHAC Centres and the NML.

The NML's work aligns generally to public health and PHAC priorities. The NML relies on insight from its various partners and networks to identify emerging issues of priority. It also participates in PHAC's various corporate governance mechanisms. However, the evaluation did not find evidence of a process at the Centre or Agency levels to foster strategic discussions on priorities that are common to both science and PHAC programs and policy. Neither did it find processes to measure and report on the performance of both the NML and PHAC Centres in using laboratory science to inform public health interventions.

Overall, the quality of the NML's work and its ability to be nimble has allowed it to make a key contribution to society, and the public health sector in Canada and internationally. It develops expertise that provincial laboratories do not have the means to develop. It occupies specialized niches among laboratories in Canada and undertakes research or innovation to better understand, prevent, and respond to infectious diseases. It also helps coordinate the sharing of expertise among partners in the public health system.

Recommendations

The quality, credibility, and relevance of NML's work is well established, but improvements to internal practices in the areas of collaboration, communication, and strategic planning would strengthen the reach of the NML's work and of PHAC as whole. These findings led to the following recommendations:

1. Implement consistent practices to plan research and innovation in order to create the space necessary to foster innovative thinking while also addressing key priorities related to NML's mandate.

The evaluation found that the NML does not have formalized and systematic processes to inform the prioritization of its research efforts, although such processes are in place in some areas of the laboratory. While it is important to provide space to scientists to enable them to be innovative, resources are limited and the NML's mandate is to address other public health priorities that do not necessarily call for innovation. Processes that are more consistent would help ensure that the NML can balance the need to be innovative with the need to fulfill its core public health priorities. This would also likely help the NML to communicate on its work and engagement.

2. Improve communication with internal and external partners on the services and scientific expertise delivered by the NML. As part of these efforts, both the NML and the Centres from the Infectious Disease Prevention and Control Branch should examine best practices already in place and emulate what worked well in order to foster a more systematic and proactive two-way collaboration and communication.

Navigating the laboratory can be challenge for external partners, who also reported a need for the NML to communicate better on services offered. Partners, especially from PHAC Centres, do not appear to understand well how the Canadian Network for Public Health Intelligence operates and plans its activities. A few internal key informants also suggested that PHAC Centres do not always understand the NML's activities.

There is a shared perception in PHAC Centres and the NML that more could be done to increase internal collaboration. Often, the NML and PHAC Centres are not aware of what the other is working on. As a result, they do not always know about opportunities for laboratory scientists and Centre epidemiologists to collaborate in order to achieve greater impacts on public health. That being said, the PHAC Centre responsible for responding to food-borne and water-borne enteric illnesses and the enteric pathogen division at the NML have set up a best practice to collaborate and coordinate the transition to whole-genome-sequencing and the use of data generated through this technology to inform surveillance and outbreak investigations.

3. Identify opportunities to formalize discussions and expectations between the NML and PHAC Centres on strategic priorities to connect laboratory science to policy.

The evaluation did not find evidence of a process at the program or Agency levels to foster strategic discussions on priorities that are common to both science and policy. Evaluation evidence also suggests that there is a need for PHAC to clarify expectations around the contribution of laboratory science to informing programs and policies.

This evaluation examined how the Public Health Agency of Canada's (PHAC) National Microbiology Laboratory (NML) fosters and manages innovation while also balancing its core public health functions. In doing so, the evaluation examined NML's core functions, its ability to innovate as well as factors that support and challenge NML's ability to manage and transfer innovation. The evaluation examined all NML activities from April 2014 to September 2019. Data was collected from a review of documents, files, financial data, key informant interviewsFootnote i, and case studies (See appendix A for more details on the methodology).

About this evaluation: This evaluation and the related management response and action plan were completed prior to the COVID-19 outbreak in Canada. The Canadian response to the pandemic has significantly affected the activities of the NML operates as well as the context in which it operates. Throughout the response, the NML has provided leadership in Canada in the area of laboratory testing for COVID-19. At this time, it is not possible to know what long-term mark the COVID-19 response will leave on the NML's operations.

NML Organization

The NML had 550 employees and a budget of about $80M in 2018-19Footnote ii representing about 12% of the $700M budget allocated to PHAC in 2018-19. During that year, about 22% of PHAC's employees were working at the NMLFootnote 1. Organizationally, the NML is part of PHAC's Infectious Disease Prevention and Control Branch (IDPC) (See Figure 1 for information about Centres within the Branch).

As shown in Figure 2, the NML's activities are organized across various scientific divisions and programs; each specialized by disease or pathogen type. In addition to these core activities, it hosts the secretariat for the Canadian Public Health Laboratory Network (CPHLN), a national network of federal and provincial public health laboratories that provides a forum to share knowledge and standardize laboratory activities. The NML also leads the Canadian Network for Public Health Intelligence (CNPHI), a secure and web-based scientific informatics and bio-surveillance platform. CNPHI helps public health professionals gather and share strategic information, resources, and expertise quickly and in real-time, as a way of advancing knowledge sharing across public health laboratories.

Figure 1. IDPC Branch Organizations

IDPC addresses persistent and emerging infectious diseases through the work of the following Centres:

  • National Microbiology Laboratory;
  • Centre for Immunization and Respiratory Infectious Diseases (CIRID): prevents, reduces, or eliminates vaccine preventable and infectious respiratory diseases, reduces the negative impact of emerging and re-emerging respiratory infections, facilitates pandemic preparedness and response, and strengthens confidence in immunization programs;
  • Centre for Communicable Diseases and Infection Control (CCDIC): creates and shares credible knowledge, and facilitates coherent, national action for the prevention and control of specific communicable diseases for all Canadians with a focus on key populations at risk of and living with communicable diseases;
  • Centre for Food-borne, Environmental and Zoonotic Infectious Diseases (CFEZID): assesses, prevents, reduces, responds, and provides upstream surveillance on the risks of food-borne, water-borne, and zoonotic diseases; and
  • Office of the Chief Science Officer: provides leadership, oversight, and support on science excellence, science policy integration, and science promotion.

Source: IDPC Business Operational Plan 2017-18 to 2019-20

The NML's activities are conducted in three locations: Winnipeg (Manitoba), Guelph (Ontario), and Saint-Hyacinthe (Quebec). The main laboratory (the Canadian Science Centre for Human and Animal Health) in Winnipeg is a high-security bio-containment facility co-located with the Canadian Food Inspection Agency (CFIA). The facility houses the only two Level 4 laboratories in Canada (i.e., PHAC and CFIA). These laboratories are equipped to test the most dangerous pathogens.

NML's Functions and Activities

NML's key mission is to work with provincial, national, and international public health and laboratory partners to prevent the spread of infectious diseases.

The laboratory accomplishes its mission through the following primary functions:

Laboratory-based surveillance: Collects and analyzes data on pathogens to inform public health decision making at different levels of government.

Specialized reference laboratory services: Conducts specialized diagnostic services for infectious diseases that are difficult to identify or diagnose, as well as new and emerging diseases. These services are offered to provincial laboratories across Canada.

Applied and discovery research: Performs research activities to discover new therapies, treatments, and diagnostic tests, and develops new and cutting-edge laboratory tools.

Leadership, networking, and capacity development: Contributes nationally through the CPHLN and CNPHI platform. The NML also contributes internationally to global laboratory networks as a designated World Health Organization (WHO) regional reference.

Emergency preparedness and outbreak response: Prepares and responds to biological threats and outbreaks in Canada, and assists in emergency events in other countries, as per international agreements.

Figure 2. NML Divisions and Programs with their 2018-19 BudgetFootnote *

Scientific Divisions

Bacterial Pathogens ($10.5M): offers diagnostic testing services and laboratory responses for human threats, including antimicrobial resistant organisms, vaccine-preventable bacterial diseases, rare and emerging bacterial organisms, tuberculosis and non-tuberculous mycobacteria, bacterial sexually transmitted infections, and biological security responses to acts of bio-terrorism.

Enteric Diseases ($8.5M): coordinates national diagnostic and testing services for provincial laboratories in cases of food-borne and water-borne illness. It also tracks and monitors food-borne diseases.

Zoonotic Diseases and Special Pathogens ($8M): tracks, diagnoses, and controls zoonotic and other rare or emerging pathogens spread between animals and people. This includes rabies, Lyme disease, West Nile encephalitis, and Zika virus, as well as zoonotic agents that are highly infectious or associated with severe diseases (e.g., Lassa fever, Ebola virus disease, Nipah virus encephalitis, hantavirus pulmonary syndrome).

National HIV and Retrovirology Laboratories ($7M): provides national laboratory testing, quality assurance, and develops new tools for diagnosis of Human Immunodeficiency Virus (HIV).

Viral Diseases ($6.1M): performs specialized testing for viral diseases, such as poxvirus, influenza and respiratory viruses, enteroviruses and enteric viruses, viral sexually transmitted diseases, blood-borne pathogens and hepatitis, and skin rashes caused by fever or diseases, such as measles, mumps, or rubella.

Science Technology Cores and Services ($3.7): translates global scientific progress into better public health response and preparedness using advanced investigative methods (e.g., genomics, mass spectrometry and proteomics, bioinformatics, molecular pathobiology).

Public Health Risk Sciences Division ($3.2M): works to prevent and control infectious diseases that arise from contact between humans and animals.

Other Divisions and Operational Functions

  • CNPHI ($2.4M)
  • Network and Resilience Development, including CPHNL Secretariat ($0.4M)
  • Biorisk and Occupational Safety ($1.9M)
  • Facility and Property Management ($16M)
  • Scientific Informatic Services ($6.7M)
  • Other support services: Office of Intellectual Property, Business Development, Science Supports, Clients Services and Director General Office ($11M)
Footnote - Figure 2

Source: Financial data provided by the NML and its annual plans.

Includes employee and student salaries, employee benefit plans established at 20%, Operations and Management, and Capital. Data excludes targeted funding for Ebola projects and external funding sources (e.g., Genomic Research and Development Initiative).

Return to footnote * referrer

Collaborators

The NML carries out its work through various domestic and international collaborations, including with PHAC Centres.

As explained by many NML key informants, collaborating with a range of partners allows the NML to focus its work on areas of relevance for public health and to gather intelligence on upcoming priorities or threats. The NML's main collaborators are described below (also see Figure 4).

Provincial public health laboratories, through the CPHLN. The network was developed in response to the events of September 11, 2001 and subsequent anthrax threats. The CPHLN works to develop and maintain strong functional bonds between federal and provincial public health laboratories in Canada, and held 126 meetings in 2018-19. The CPHLN provides a nationwide forum for provincial public health laboratories to share knowledge on issues related to diagnostics, and to emergency preparedness and response to public health events of national and international concern.

International organizations and stakeholders, through a number of partnerships, including the Global Health Security Action Group. This group was established to develop and implement concrete actions to improve global health security. The NML's Office of Networks and Resilience Development leads the group and develops relationships with other international partners.

In addition, four of the NML programs serve as WHO Collaborating Centres, reference centres, and regional references laboratories, as follows:

As an example, Figure 3 describes how the NML's functions are involved in preventing and responding to the Zika virus in Canada.

Figure 3. Example of NML Activities: Work on Zika

Specialized reference laboratory services: The NML conducts the bulk of the Zika virus testing across Canada and plays a support role for provinces that do not have the resources or expertise themselves. The Viral Zoonoses section at NML performs reference services and applied and basic research for zoonotic pathogens such as mosquito and tick-borne arboviruses. They conduct molecular detection and serology tests targeted at Zika virus infectionFootnote 2.

Surveillance: The Public Health Risk Sciences Division at NML uses advanced epidemiological analyses to synthesize laboratory, environmental, animal, and human population data, to create public health intelligence for a range of outputs. They use risk modelling to predict future risks and how best to address them. Risk modelling aims to obtain a quantitative relationship between values for key drivers, such as weather or climate, and the occurrence of infectious disease risk. These tools are used in conjunction with field research to provide a more complete picture of the Canadian landscape for the Zika Virus.

Research: The NML, in collaboration with the Centre for Biotechnology at Brock University, conducted an assessment on the feasibility of the Zika virus being carried in mosquitoes that are native to Canada. The study aimed to determine if mosquitoes in Canada could acquire the slow-growing Zika virus and transmit it to humans while surviving in Canadian temperatures. The study found it is unlikely that any of the mosquito species tested would be involved in any large-scale transmission of the Zika virus. Further studies are planned to determine the capacity of a broader range of mosquito species to acquire and transmit the Zika virus in Canada.

Outbreak response: PHAC has published information about the NML's guidelines and testing recommendations for the Zika virus targeted towards Canadian travelers in Zika affected areas, health professionals, and laboratory testing recommendations. The guidelines cover prevention and treatment recommendations for individuals travelling to Zika-affected areas and give targeted advice for higher risk travelers such as pregnant women. The guidance document includes a decision tree that considers travel history, population at risk, presence of symptoms consistent with Zika virus infection, and possible non-travel related exposures.

The NML collaborates with various countries and organizations on different files, including:

Through the CPHLN secretariat, it networks with various countries or regions, including Southeast Asia and the Caribbean, for surveillance purposes, data sharing, and helping other laboratories to develop their surveillance programs.

Figure 4. NML Internal and External Collaborators
Description Figure 4:

This image depicts the range of NML collaborators. There is a circle at the center which represents the NML. This circle is divided into three sections: programs; core services; and, information sciences. Around the center, there is a listing of all sub-divisions of the NML. Revolving around this center, the range of different collaborators is depicted including:

  • Provincial and territorial laboratories.
  • PHAC branches
  • International collaborators
  • Universities
  • Other government departments (i.e., GAC; AAFC; CFIA; HC and other).

Innovation at the NML

The laboratory environment faces pressures that are driving changes to the way science is conducted.

As characterized by some NML and external key informants, the NML is now operating in a paradigm shift where laboratories around the world are implementing advanced "omics technologies" (see definition in Figure 5). These computer-based technologies are transformative, as they allow scientists to understand the DNA of a pathogen, which then permits for faster and more precise identification of these pathogens. As a result, outbreaks can be resolved with greater speed, accuracy, and confidence. These technologies also support vaccine development and, in some cases, can identify if certain pathogens are drug resistant.

Figure 5. What are "Omics"?

"Omics" refers to the technologies used to characterize pools of biological molecules and explore their roles and relationships in the cells of a living creature. The "omics" suffix describes the use of these technologies to examine genomes (genomics), proteins (proteomics), and the chemical processes involving metabolites (metabolomics), among others. Genomics was the first "omics" terminology used and is the most advanced practice.

As mentioned by several of its key informants, the NML has to keep up with global players who are investing heavily in the transition towards "omics", as these technologies are accepted globally as the de facto gold standard. Reviewed internal documents and many internal and external key informants noted that rapidly advancing laboratory technologies are driving the need for a coordinated long-term laboratory modernization strategy that will transform public health investigations for Canadians.

At the same time, some internal and external key informants noted that new and emerging public health problems are appearing while the resources to address them are limited. As explained by many external and internal key informants, and further corroborated by reviewed internal documents, laboratory and public health practitioners must now address numerous global challenges and security concerns that increase the complexity of the environment in which they operate, and will likely affect various population groups differently. Among others, these challenges include climate change, antimicrobial resistance, the risk of bioterrorism, the emergence or re-emergence of infectious diseases, and the increased spread of these diseases due to global travel or migration.

As outlined in the "One Health" approachFootnote 3, addressing these complex problems requires multidisciplinary input that taps into expertise from all aspects of health (human, animal, and environmental). Most provincial partners interviewed explained that addressing these challenges requires collaboration and partnerships between the NML and the provinces to address unequal capacity across jurisdictions, and to develop the necessary Canadian expertise to respond to emerging infectious diseases.

In this context, the NML has been adaptive and innovative in order to remain relevant. It has put in place numerous innovations to address these challenges (see Figure 6).

In regards to laboratory modernization, the NML has started to transition toward the use of "omics". It has achieved a full implementation of whole genome sequencing (a technique to obtain the complete genetic blueprint of an organism) for the surveillance and detection of food-borne enteric illnesses including salmonella and listeria.

According to internal documents, the use of whole genome sequencing enhances real-time surveillance and outbreak responses by answering scientific questions related to the source and transmission patterns of pathogens. Some external key informants noted that the NML's PulseNet Canada Genomics Implementation Roadmap (2016) played an important role in delivering a phased-in transition to this modernized (genomics) approach by ensuring consistency across provincial laboratory partners, and supporting capacity building and quality assurance.

Following up on the implementation of genomics for food-borne illnesses, the NML continued implementing genomics for other infectious diseases (e.g., tuberculosis and influenza) through the PHAC Laboratory Technology Modernization Strategy and with provincial health laboratory partners through the CPHLN Pathogen Public Health Genomics StrategyFootnote 4. In 2018-19, the NML sequenced 43,198 genomes, almost double from the previous year (22,576).

Figure 6. NML Innovation Highlights
Description Figure 6:

This text box lists few examples of NML innovations. These are organised around genomic; diagnostic; equipment and apps; database; vaccine; and, prevention and treatment of priority pathogens.

The NML also continues to improve its informatics solutions and data sharing with its partners in support of rapid communication and improved coordination of public health information to inform decision making at both the local and national levels. For example:

The NML has established an Innovative Diagnostics Program to develop and deploy technological solutions to conduct infectious disease testing in underserved communities. The objective is to improve testing using novel, culturally safe and community-led methodsFootnote 6.

Innovations in the field of discovery research also led the NML to develop promising vaccines, like the Ebola experimental vaccine (see section 6.2). As shown in recent academic research, although the vaccine was licensed to a private company, its development, from the early research stage to its clinical trials during the Ebola epidemic, was largely due to the combined efforts of the NML and other public organizationsFootnote 7.

Furthermore, NML scientists, in collaboration with the National Research Council and the Canadian Institutes for Health Research (CIHR), developed a vaccine for a deadly and emerging influenza strain that is especially dangerous for at-risk children in northern Indigenous populations (i.e., Hemophilus influenza type a). The experimental vaccine is currently licensed for clinical trialsFootnote 8.

To continue improving laboratories' ability to diagnose pathogens, the NML undertook research to identify the most effective Zika diagnostic testing tool. NML scientists carried out an assessment of numerous commercially available Zika diagnostic kits that resulted in the identification of the most efficient testing practices. This can potentially reduce the burden on laboratory resources and improve testing turnaround times should a Zika outbreak ariseFootnote 9.

Overall, most external partners praised the NML for its ability to innovate and remain nimble in face of emerging public health and laboratory challenges. Moreover, as noted in Figure 7, the 2017 Client Satisfaction and Needs Assessment Survey showed that 89% of the respondents agreed that the NML's research and development of new technologies is appropriate, a significant increase from the 2011 response of 77%.

Figure 7. Level of Client Satisfaction with NML's Services (2017)
Description Figure 7:

This figure depicts a horizontal bar graph showing the proportion of clients who were strongly or somewhat satisfied with NML services in response to questions from NML's 2017 Client Satisfaction and Needs Assessment Survey. There were 43 respondents.

The proportion of respondents who answered strongly or somewhat satisfied is as follows:

  • Provides appropriate training to its clients when new technologies are deployed: 75%;
  • NML research and development of new technologies are appropriate: 89%;
  • Timely consultations/expert opinions are provided through telephone and email communications: 78%;
  • Provides appropriate proficiency panel programs: 83%;
  • Provides urgent/rush services in a timely fashion: 85%;
  • Offers high quality assays: 77%; and,
  • Provides laboratory reference services within stated turn-around times: 72%

Balancing Core Functions with Innovation

Satisfaction with NML Services

Key informants interviewed from provinces, academia, and PHAC Centres were satisfied with the work of NML divisions.

Many international partners, PHAC Centres, academics, and other federal departments interviewed perceived the NML as providing high-quality services, and as being a credible and helpful organization. Most provincial partners interviewed echoed that sentiment.

These perceptions echoed findings from the 2017 Client Satisfaction and Needs Assessment Survey, where 98% of respondents strongly or somewhat agreed that NML programs provide satisfactory services and meet the requirements of clients. Overall, client satisfaction steadily increased over time from 81% in 2008, when the survey was first conducted, to its peak of 98% in 2017, when it was last conducted.

As shown in Figure 7, the majority of 2017 survey respondents were satisfied with NML services. The highest level of satisfaction was with the NML's research and development of new technologies (89%), while the lowest level was regarding turnaround times for laboratory reference services (72%). Echoing this result, a few provincial key informants noted concerns with the timeliness of reference testingFootnote iii.

Of note, some key informants from PHAC Centres, provincial partners, and other federal government departments or agencies interviewed expressed that the CNPHI is a valuable tool to them. These key informants mentioned that CNPHI allows for better disease tracking, improves collaboration and timeliness in reporting, and helps inform evidence-based public health decision making.

Collaboration with Internal and External Partners

Collaborations with external partners are well valued.

Overall, most of the NML's external partners interviewed (i.e., academic, international experts, provinces, other federal departments) described the laboratory as a collaborative organization driven by its leadership, responsiveness, and openness to collaborating at all levels. This perception aligns with the 2017 Client Satisfaction and Needs Assessment Survey, where 87% of respondents strongly or somewhat agreed that the NML collaborates with its national, provincial, and international partners.

The majority of provincial partners interviewed credited the NML in being instrumental to providing leadership at the federal and provincial levels through the CPHLN. The CPHLN, including its working groups, was seen by many provincial and NML key informants as effective in supporting the identification of current and emerging public health issues and threats through the sharing of insights with provinces, which are at the front line of infectious disease detection.

Most academic and half of provincial key informants indicated that the NML's international collaborations are valuable as they allow the sharing of information and best practices, and the development of networks. Half of NML key informants echoed the importance of international collaborations and networks for the exchange of information and expertise.

Of note, while the level of collaboration from NML staff was generally seen as positive, some provincial partners noted variability in the level of collaboration obtained from laboratory scientists. However, considering the breadth of NML work and the size of its organization, it may be natural that working relationships differ depending on the division with which a key informant interacted.

Some provincial key informants noted that they would welcome greater transparency with respect to work undertaken by the NML. In particular, those who are not part of the CPHLN decision-making bodies reported being less aware of NML activities and unsure of how to engage with its staff to ensure the alignment between each other's work. A few provincial key informants noted that it was difficult for staff in the provincial laboratories to navigate the NML's structure, and to know where to direct their questions.

In addition, when asked about the NML's communication and issue resolution, 65% of the respondents from the 2017 Client Satisfaction and Needs Assessment Survey agreed that communication about NML services could be improved.

NML has been helping provincial partners to take up laboratory innovations.

Scaling up innovation with provincial and territorial partners is an ongoing challenge for the NML, as provinces do not always have the capacity to adopt innovations. This means that the NML's ability to transfer diagnostic testing to the provinces, once they become high volume or routine, depends on the province's capacity to adopt the technology required to conduct these tests.

In this context, the NML works with CPHLN members to coordinate the integration of bioinformatics and whole genome sequencing laboratory processes and procedures into provincial public health laboratories. The CPHLN recently undertook a Laboratory Omics Readiness Assessment, which involved consultations with NML partners to review the status of "omics" technologies within their laboratories. A few NML and provincial key informants viewed these consultations as important for developing a better understanding of the gaps and challenges with "omics" technologies, and preparing actions to address them.

Moreover, to facilitate the implementation of whole genome sequencing, the NML provided provinces with training, equipment purchases (e.g., MiSeq sequencers), as well as IT platforms (e.g., IRIDA). In addition, the NML's Laboratory Liaison Technician Officer Program funds the positions of Government of Canada employees working in eight provincial laboratories.

Overall, many provincial key informants noted that the NML has provided assistance to help them adopt innovations and spoke highly of this support. The sentiment was echoed in the 2017 Client Satisfaction and Needs Assessment Survey as shown in Figure 7.

While the NML appears to have strong and well valued external collaborations, its level of interactions with PHAC Centres could be improved.

Several PHAC Centre key informants and more than half of NML key informants mentioned that they collaborate with each other and these collaborations yield positive results. However, there is a shared perception among PHAC Centre and NML key informants that more could be done to increase their level of collaboration and communication.

NML divisions and PHAC Centres interact regularly though conferences calls or meetings, but these interactions are often informal. Several PHAC Centre key informants indicated that personal relationships often drive the collaborations between their programs and NML divisions. Furthermore, as explained by one key informant and shown in Figure 8, PHAC Centres have to interact with multiple divisions within the NML, which can be challenging.

Both NML and PHAC Centre key informants raised concerns about the frequency of communications at the working level. Many key informants from various PHAC Centres explained that they are often informed of laboratory decisions after the fact, even though these decisions can affect their work. Furthermore, most of the PHAC Centre key informants explained that they were often not aware of what the NML was working on, and it was up to them to find out where collaboration with laboratory staff could be beneficial.

Various PHAC Centre key informants interviewed shared the perception about the lack of communication. These key informants provided different examples of what they perceived to be a lack of communication. Among other things, one PHAC Centre key informant noted that the NML unilaterally makes most, if not all, of the decisions made around laboratory activities that support their program.

Another key informant noted that their Centre does not receive NML division work plans. Also mentioned was that PHAC Centres often learn about upcoming NML publications because of comments made in passing or that they are asked to review the upcoming publication in its final stage. In some cases, external stakeholders have been informing PHAC Centres about publications or initiatives coming from the NML.

Figure 8. Key interactions between NML science divisions, and PHAC Centres and branches
Description Figure 8:

This image illustrates that each NML program is having interactions with different Centers and Branches within PHAC. For example, the NML bacterial pathogens division interacts with CFEZID, CCDIC and HSIB within PHAC.

Source: Analysis based on evaluation data

Some NML representatives also share the same sentiment. They mentioned that PHAC Centres do not always communicate with them in a timely manner on initiatives that affect the laboratory's work.

The evaluation has not found conclusive evidence that the perceived lack of communication has negatively affected NML's or the PHAC Centre's work, but a few key informants explained that it has probably resulted in missed opportunities to join forces in order to achieve a greater impact on public health. More collaboration would also allow PHAC to take a coordinated approach on international files more consistently.

In some instances, collaboration between the NML and PHAC Centres is more formalized and perceived as successful.

Some PHAC Centre key informants identified a few examples where collaboration with the NML was successful. For example, interactions between epidemiologists and NML laboratory staff for the Canadian Nosocomial Infection Surveillance Program and the National Advisory Committee on Immunization were identified as positive. Another example is the working group put in place between the Enteric Diseases Division and the Centre for Food-Borne, Environmental and Zoonotic Infectious Diseases (CFEZID) to discuss the implementation of whole genome sequencing for the identification of food-borne enteric illness pathogens. This working group met weekly during the implementation phase of the new technology. To this day, it continues to meet every week to discuss issues with respect to surveillance, cluster detection, and outbreak responses. According to a PHAC Centre key informant, the implementation of the working group has fostered more collaboration and communication between laboratory staff and epidemiologists.

Various reasons may explain the lack of formalized collaboration and communication between PHAC Centres and NML divisions.

Some key informants from NML programs and PHAC Centres noted a lack of integration between epidemiologists and laboratory scientists, as well as a disconnect between the two domains where one does not necessarily see how its work affects the other. As well, these key informants pointed to a difference in culture between PHAC Centres and the NML. For example, there is a perception that some aspects of NML's culture are similar to the academic world where researchers have more freedom to determine their research agenda.

When applying for an internal promotion within PHAC, research scientists, including those working at the NML, have to provide evidence of their achievements. Their publications are accepted as evidence to demonstrate their innovation, the impact of their research, the recognition they have received, as well as their productivity. As noted by one key informant, scientists may be compelled to keep their work to themselves until published. As a result, this may limit proactive communication on research work underway.

A few NML and PHAC Centre key informants suggested that there is a lack of understanding of NML activities within PHAC's National Headquarters. As described by a few NML key informants, there is a perception that the people at the National Headquarters do not fully understand what priorities are important for the NML to work on. Since the laboratory has the mandate to prepare for the next emerging threat, it has to investigate pathogens or work on files that may not be an official priority for the Government of Canada at the current time, but could become one in the future. For example, although it was not one of PHAC's priorities at the time, the work undertaken by the NML on Ebola allowed them to be ready to provide on-the-ground support during recent international outbreaks and to test potential cases in Canada.

To address the need to increase the visibility of its work, the NML implemented various communication initiatives including a Twitter campaign featuring its scientists, and new content for the Canada.ca website. It also developed Science Blog posts and Science Stories, published on the Government of Canada's Science and Technology website and on the PHAC Intranet. The NML Science Stories page received 388 unique page views between January and December 2019 from PHAC or Health Canada employees. This is a relatively low number considering the number of PHAC and Health Canada staff.

Finally, a few key informants noted that the geographic separation between NML sites located in Winnipeg, Guelph, and Saint-Hyacinthe, and the PHAC Centres, mostly located in Ottawa, could be a factor affecting the level of collaboration. A similar challenge was uncovered in the 2019 evaluation of the National Collaborating Centres for Public Health, which shows that PHAC needs to improve its relationships with funded organizations located outside of National HeadquartersFootnote 10.

Priority Setting and Planning

The NML's work is aligned to public health priorities.

External partner key informants generally viewed the NML as being relevant and aligned to public health priorities, a view shared by PHAC Centres. Moreover, an analysis of the NML's work plans and various PHAC priority-setting documentsFootnote 11 showed that the NML's work is well aligned with PHAC priorities; however, many of these priorities are wide ranging.

The NML gathers insights from external partners, especially from provincial laboratories, about potentially emerging issues. It identifies priorities for reference services through discussions with the CPHLN. The Canadian Public Health Laboratory Network Strategic Plan guides CPHLN activitiesFootnote 12.

Within PHAC, there are limited mechanisms in place to discuss common priorities between the Centres and the NML, or to undertake strategic planning at the Agency level.

The NML participates in PHAC corporate planning at both the Branch and the Agency levels. It is involved in the IDPC's Program Planning Group, which provides a forum for the NML to engage with other PHAC Centres on corporate planning and reporting activities (i.e., planning, reporting, performance measurement, risk-management activities). The planning group coordinates input for information that will feed into the departmental planning processes, including parliamentary reports (i.e., Departmental Plan, Departmental Results Report), Corporate Risk Profiles, Operational Plans, and Investment Plans.

The NML also takes part in broader policy and operational discussions at the Agency level. A review of agendas and records of decisions from PHAC's Operations Committee and Policy Committee showed that NML representatives regularly participated in both committees and led discussions on topics like the Centre for Genomics and Bioinformatics Innovation Project and the PHAC Laboratory Technology Modernization Strategy.

A review of records of decisions from the NML's management committees showed that representatives from the National Headquarters sometimes participated in governance meetings internal to the NML. These discussions generally focused on subjects like human resources and wellness, which the NML identified as priority initiatives.

Overall, although the NML and PHAC Centres are involved in joint governance committees, a review of internal documents and responses from many PHAC Centre key informants showed that no mechanism exists at either the Agency or Centre level to foster strategic discussions on: 1) better ways of integrating science into policy; 2) prioritization of publication efforts in order to address stakeholders needs; nor 3) insights that the NML gathers from its international networks on what may be the next priority for addressing infectious diseases in Canada. A few key informants expressed a desire for more formalized processes to support strategic discussions.

The NML's Performance Information Profile states that the laboratory's objective is to provide Canadians with the scientific readiness to respond to infectious disease threats. It also indicates that the NML enables informed public health action through delivery of innovative approaches to advance laboratory science, testing services, lab-based surveillance, outbreak response, and national public health laboratory leadership. However, performance measurement indicators used to measure and report on NML's contribution in this area tend to focus on satisfaction and use of services by other provincial laboratories. None of them captures how NML's science and expertise is expected to support public health interventions by PHAC. Moreover, although the NML is making a real and ongoing contribution by supporting PHAC's public health interventions (e.g., providing testing information to PHAC numerous surveillance systems, as further explained in section 6.1 of this report), the evaluation's review of Performance Information Profiles for PHAC CentresFootnote iv, showed that laboratory science is rarely documented as an input to support their activities. Laboratory input was noted in a few cases (e.g., use of a mobile laboratory for emergency response, submission of enteric samples to the NML via provincial laboratories to mitigate antimicrobial resistant-related human health impacts). As a result, PHAC does not have a plan to systematically measure and report on how laboratory science contributes to support its public health interventions. The lack of official documentation, from both the NML and PHAC Centres, on how laboratory science is informing PHAC programs and policies also suggests that the Agency, as a whole, may have not fully discussed and defined expectations in this area.

Internally, the NML does not have consistent mechanisms in place to prioritize its research activities.

At the time of the evaluation, the NML did not have a process to systematically prioritize its research efforts from an organizational perspective. A few key informants from the NML and PHAC Centres indicated that research is driven by a bottom-up approach, where scientists actively determine what areas to focus on and demonstrate the value of their work. Similarly, an internal document noted a need to engage scientists in science planning in relation with public health priorities.

More systematic planning of research activities tends to happen within the NML when research funding comes from external sources. For example, a more rigorous approach has been used to manage research activities and priorities in the Genomic Research and Development Initiative and the Canadian Safety and Security Program. A formal proposal review process guides funding decisions, based on alignment with PHAC priorities and science excellence. A steering committee or external peer review also provides oversight on these projects.

The NML uses internal steering committees comprised of NML senior executives to provide strategic leadership to guide new initiatives like the Laboratories Canada Initiative or the new Innovative Diagnostics Program. Working groups provide technical and programmatic expertise for these initiatives.

As shown in Figure 9, other science-based organizations have put in place good practices to balance the need to provide scientists with space to innovate and the need to address broader priorities. This balance appears particularly relevant to achieve in a context where the NML, like many other organizations, has to innovate with a limited level of resources.

Processes to prioritize investment in CNPHI are not well understood by partners.

Although CNPHI is seen as a valuable tool, and the quality of its work has received praise, several key informants from PHAC Centres did not understand how CNPHI activities were prioritized and driven. Some of these key informants indicated that CNPHI was not always responsive to their needs when designing a new tool. They stated that consultations regarding CNPHI's priorities were limited, even when developing new tools.

Figure 9. Good Practices put in Place by Other Science-Based Organizations to Plan Innovation

A comparative assessment was undertaken to gather insight as to how other government departments and laboratories, including the National Research Council, the Canadian Institute for Health Research, the US CDC (Enteric Laboratory Disease Branch), and the CFIA (Innovation Centre of Expertise) manage innovation and plan around research and scientific activities.

Overall, there was a movement among these organizations to bring innovation to the forefront and, in some cases, modernize their operations. These organizations consider many variables when undertaking research investment planning, including:

  • identifying where the most pressing need exists and where innovation could be best targeted;
  • identifying where there is potential for the most impactful scientific progress (e.g., existence of promising pathways and qualified investigators);
  • balancing curiosity and scientific freedom with the need to make strategic investments where there is possibility for greatest societal gain; and
  • ensuring alignment of research priorities with higher-level organizational and agency priorities.

These organizations have also put in place different best practices to plan and implement innovation:

  • While planning with a bottom-up approach helps to empower scientists at the working level and stimulate creativity, individual science divisions develop and consolidate business cases across the organization in order to bring together overarching themes and drivers, as well as to foster synergy. The planning process also factors in overarching external trends and pressures (e.g., climate change, aging population).
  • Drawing on the expertise of external advisors on a regular basis to obtain a broader perspective and validation. For example, the US CDC often convenes Blue Ribbon panels of external experts to provide advice on setting priorities. As noted by one key informant, this input is particularly valuable when the US CDC is unfamiliar with a new science area, such as when a panel was assembled in 2011 to provide guidance on the implementation of genomics. In 2019, the CIHR launched an online survey open to a wide audience of health professionals, including those in academia, to solicit input on research priorities. Survey results were presented to external partners at a follow-up consensus workshop and research summit, where further input on defining research themes and funding areas was gatheredFootnote 13.
  • Sharing regularly across research programs and divisions. This helps reduce duplication and allows scientists to see how they can collectively contribute to the greater mandate and vision of the organization. It can also help to avoid working in silos.
  • Driving innovation through international linkages, networks with international partners and participation in global events. This offers opportunities to stimulate research and development, as well as innovation in technological areas of mutual interest.

The evaluation did not find any evidence of a structured process to approve and prioritize projects for the platform, however, a Program Leads Committee, grouping together experts in human, animal, and environmental health from the federal and provincial governments, supports CNPHI. A Terms of Reference for the committee was being developed at the time of the evaluation.

Internal systems and practices were launched recently to improve the planning of NML's activities.

Among other things, in 2018, the NML implemented the Science Planning Information Management System to provide timely and accurate information on NML program activities. It also launched the Laboratory Rounds in 2015-16 as a forum to discuss science activities. As well, at the time of the evaluation, the NML was developing a Strategic Science Plan to articulate its long-term scientific priorities.

Adapting to changes brought by innovations

Innovation put in place at the NML to modernize its science has created a need to adapt its skill set and IT infrastructure.

As noted by some NML key informants, talent recruitment and staff training represents an ongoing pressure, as the use of modern laboratory technologies requires additional expertise in computation biology, data sciences, bioinformatics, social science, and data interpretation. To address this concern, at the time of the evaluation, the NML was developing a Human Resource Plan to establish requirements for staffing, career development, and training activities. According to an internal document, the NML has been able to secure a significant workforce of bioinformaticians but developing and retaining highly specialized informatics expertise remain a challenge.

These advanced technologies have also increased the volume of data generated by the laboratory, which has increased the demand for data storage and subsequently put pressure on the NML's IT infrastructure. However, the NML has made progress to adapt to this new reality through server upgrades, IT architecture improvements, and providing enhanced operational support for software and databases (e.g., installing upgraded versions of bioinformatics software, "bug" fixes).

The NML's Scientific Informatics Services Division has also put in place initiatives to manage the processing of large amounts of data and better meet science-specific IT requirements in the area of whole genome sequencing. Nevertheless, more progress is required to ensure data sharing between federal and provincial laboratories, to ensure surge capacity in the event of significant outbreaks where testing requirements exceed what can be met with regular resources, and to ensure that the IT resource needs of provincial laboratories are met.

A number of NML key informants expressed concerns that there are insufficient resources to support the transition to "omics" technologies. While the evaluation team was not able to find evidence to characterize the extent of the resource gap, an internal document from January 2019 reports a need for $21M to fund genomic epidemiology work in priority programs (i.e., antimicrobial resistant pathogens, tuberculosis and respiratory viruses, sexually transmitted and blood-borne infections, vaccine preventable diseases, vector-borne diseases, and enteric diseases across the One Health spectrum). Among other things, this funding would serve to advance work such as establishing pathogens specific whole genome sequencing databases and continuing parallel testing to validate whole genome sequencing against traditional methods. According to internal documentation, progress was made to fund and establish IT tools and the right infrastructure.

As noted by some NML key informants and internal documents, without additional investment, there is a risk that the NML will not have the modern technological capacity and scientific expertise to use genomics, bioinformatics, and other emerging technologies to their fullest capabilities. Some NML key informants expressed concerns that the NML could lag behind international counterparts that are making substantial investments in this area.

Contributions made by NML's work

Overall, improvements with the internal practices noted earlier would likely help the NML to improve how it fulfills its core functions while innovating. However, as further described below, the NML's work is already of high value. The laboratory has been critical to PHAC's mandate and has made an important contribution to prevent and address infectious diseases in Canada.

Contribution to PHAC Mandate and Activities

As Canada's leading public health laboratory for infectious diseases, the NML assumes a significant role within PHAC.

The NML contributes to PHAC's efforts in preventing and controlling infectious diseases by being "science ready" to address key issues in this area through expertise, partnerships, and technology. In its multi-faceted role, the NML contributes to PHAC through a range of activities, with the most notable areas highlighted below.

As previously mentioned, the NML provides a wide range of reliable testing services to enhance the capacity of regional, provincial, and territorial public health services. As noted in internal documents, these activities contribute directly to the protection of the health of Canadians through leadership, innovation, and action, which are key aspects of PHAC's overall strategic vision and mission. These activities align with PHAC's mandate of promoting cooperation and consultation with provincial and territorial governments in the field of public health, as stipulated in the Public Health Agency of Canada ActFootnote 14.

The NML plays an important role in providing laboratory input to PHAC's more than 50 surveillance systems, including the Canadian Nosocomial Infections Surveillance Program, and FluWatch, among others. Laboratory surveillance activities are important for tracking pathogens and their sources, allowing for effective intervention and response actions. A recent example of this was the Salmonella outbreak in 2017-18 linked to breaded chicken nuggets, where collaboration between integrated laboratory scientists and epidemiologists led to faster identification of the pathogen, resulting in a targeted food recall, and ultimately an effective prevention strategy to reduce illness.

The NML supports PHAC and other federal departments in emergency response. The NML mobilizes its Operations Centre during national public health emergencies to manage the laboratory aspect of the event. As part of this role, the NML liaises with PHAC's Emergency Operations Centre to support a co-ordinated Agency response. The NML also deploys first response laboratory capacity and mobile field units to target outbreaks, as well as bio-terrorism threats anywhere in Canada and around the world. Since 2007, the NML Operations Centre was activated 34 times for domestic and international events, 14 of which were in 2014 to 2019, inclusively. In the time covered by this evaluation, the NML provided services and support within Canada, including assistance during the Syrian refugee response, mobilization and site support at the G7 Summit in Quebec City in 2018, and surge capacity for tuberculosis testing in Nunavut and northern Manitoba. Internationally, the NML deployed three times, including to West Africa during the Ebola virus outbreak from 2014 to 2016. The NML's work in this area supports PHAC's mandate of supporting national readiness for public health threats and of taking measures to address public health emergency preparedness and responseFootnote 15.

NML research helps generate science-based evidence to inform public health decision making, policies, and programs. For example, several recent research studies investigated the geographic patterns of tick distribution in conjunction with social behaviour (i.e., if people are informed and are adopting preventative measures against Lyme disease). According to one key informant from a PHAC Centre, these studies provided valuable knowledge to inform public education material on Lyme disease, and assisted in effectively targeting populations at greatest risk.

Contribution to Society

The NML is perceived as a critical force in giving Canada the ability to prepare for public health threats.

External partners and PHAC Centres found the NML to be valuable. Its value mainly comes from its leadership role and expertise. The NML's key contribution to society is to protect Canadians against infectious diseases by assuming the roles described below.

Levelling the playing field in Canada by supporting provinces with testing, expertise, and training: The majority of provincial key informants noted that the NML supports provinces by providing testing services for exotic and emerging diseases, or where a province might be lacking diagnostic capabilities, or is overwhelmed with the amount of samples (e.g., during the Zika outbreaks).

To illustrate the extent of the NML's reference services work, in 2018-19, it received 89,785 specimens and ran 79,327 accredited tests. This was an increase from 2017-18, when the NML received 83,596 specimens and ran 70,189 accredited tests. The NML also tested 14,139 Zika samples during that year.

Many provincial key informants explained that NML reference services would be too specialized or expensive for them to provide in their own laboratories. The NML's assistance with reference services was of particular importance to smaller provinces, which do not have the technological or financial capacity to conduct all the tests themselves.

The NML's work was seen as helping to improve the understanding of public health issues across Canada. For instance, some external key informants mentioned that the transition to genomics has played an important role in understanding outbreaks and transmission patterns of pathogens (e.g., syphilis, salmonella). In 2018-19, the NML detected 485 food outbreak clusters, which were 148 more than the previous fiscal year.

The genomics work done by scientists at the NML has contributed to a faster response to food-borne illness outbreaks. For example, the use of whole genome sequencing (a technique to obtain the complete blueprint of an organism) reduced testing time for E. coli from 24 hours to three hours, and allowed for the identification of linked cases across the country and around the worldFootnote 16.

External partners interviewed credited the NML for supporting provinces with training and advice (e.g., on bioinformatics). For example, 85% of respondents to the Client Satisfaction and Needs Assessment Survey strongly or somewhat agreed that the NML provides high-quality consultation and expert opinions via telephone or email, while 71% of respondents strongly or somewhat agreed that the NML provides training and education opportunities to its clients.

Some provincial key informants explained that they would like the NML to have a greater advocacy role for public health laboratories across the country. Some provinces noted that the NML had helped them make the case for new investments in modern laboratory technologies (e.g., whole genome sequencing) and they would like to see the NML have a greater role on that front moving forward. Having the NML speak about current federal laboratory priorities can help inform decision making in the province regarding investments in their own laboratories. Canadian public health laboratories operate in a system that is fragmented across various levels of government, and where resources are constrained. In this context, a few key informants from the provinces suggested that the NML could help facilitate more sharing of expertise across the system. Examples of leadership work that the NML could undertake include fostering an increased integration of data held by laboratories in Canada, and increasing coordination among jurisdictions to help laboratories share their expertise and services with those lacking the means.

A recent external evaluation of Canada's public health system conducted by the WHO and the Pan American Health Organization also recommended that Canada consider mechanisms to better understand the impacts of variation in laboratory testing capacity across the countryFootnote 17.

Leadership in preparing for and responding to infectious disease threats: Some key informants underlined the necessity of having an organization in Canada that can develop the expertise and tools necessary to prepare for the next outbreak or public health threat, as not all provinces have the capacity to do so.

As well, most provincial and academic key informants explained that engaging NML leadership (e.g., the secretariat for the CPHLN), regular networking, and information sharing with Canadian jurisdictions, other countries, and national and international organizations allows for better collaboration. It also allows for a more agile response to new and emerging pathogens.

One concrete example of the NML's work to share public health information is the creation of the CNPHI platform in 2004. CNPHI had 138,154 and 125,705 logins in 2017-18 and 2018-19, respectively. CNPHI implemented innovative solutions, such as rapid and adaptive data collection, interactive data interrogation, and data visualization, thus enabling the transformation and modernization of numerous surveillance programs. For example, the CNPHI platform allows for the collection of data and production of analytics on Drinking Water AdvisoriesFootnote 18. According to a few external key informants, CNPHI data has been critical to understanding the causes of Drinking Water Advisories, and has helped lead to appropriate public health interventions. This includes the purchase of power generators for some communities, as some Drinking Water Advisories were related to power outages.

Helping to address the specific needs of populations affected differently by infectious diseases: Some NML and provincial key informants explained how the laboratory worked with Indigenous groups to improve diagnosis, with approaches such as dried blood spot testing for HIV and other sexually transmitted and blood-borne infections in remote communities. In addition, the NML's mobile laboratories have been deployed in Nunavut to do on-site tuberculosis diagnostics, allowing Inuit to receive diagnosis and treatment in their own communitiesFootnote 19.

Advancing knowledge on infectious diseases and laboratory science: The NML's work is helping advance knowledge about infectious diseases, develop cutting-edge laboratory tools, and advance public health interventions. In particular, NML research and publications have made a significant contribution to science and public health. In 2018, the NML published 232 scientific papers on subjects related to infectious diseases, laboratory science, and research work. These papers were referenced 568 times, an increase from 137 papers and 521 citations in 2017.

Examples of work showcased in different publications include the NML's surveillance work on lymphogranuloma venereum (LGV), a sexually transmitted infection that has re-emerged since 2013 among men who have sex with men. This study noted the importance of enhanced surveillance for this disease, in order to develop targeted protection and promotion activities to reduce the risk of transmission. The NML conducted in-depth genotyping identification and DNA sequencing, which assisted in stopping the spread of the diseaseFootnote 20.

Scientists from the NML produced several articles on the Zika virus, one of which examined the susceptibility and ability of several species of mosquitoes to transmit the virus. They provided information on potential risks of Zika to CanadiansFootnote 21.

A citation analysis conducted to gauge the reach of the NML's scientific work showed that publications by NML-affiliated authors had uptake from a wide audience in the scientific community. A search in Scopus found that three of the ten most referenced papers authored by people affiliated with the NML and published from 2014 to the end of October 2019 were about Ebola, including one article on portable genome sequencing for Ebola surveillance, and two articles on Zika. The remaining five documents focused on genomics, taxonomy, food-borne illnesses, and research methodology. The citations came predominantly from academic journals focusing on many different topics (e.g., infectious diseases, genomics, nursing, plants) from various regions (e.g., Africa, Asia, Europe) and countries around the world (e.g., Canada, U.S., Russia, China, Japan).

Figure 10. An Example of NML Contribution to Society: Work on Ebola

The NML's work on Ebola Virus Disease (Ebola) provides a good example of how the laboratory helps to prevent and treat infectious diseases, respond to emergencies on the ground, and advance scientific knowledge.

The NML is known around the world for its invention of a vaccine for Ebola (rVSV-ZEBOV). Work on the vaccine had been going on long before the 2014 to 2016 outbreak in West Africa. In 2010, the vaccine was patented and a license granted to a private company, although the Government of Canada continues to own the intellectual propertyFootnote 22. Clinical trials started in 2015 in Guinea, one of the countries heavily affected by Ebola during the 2014 to 2016 outbreak, using vials of the vaccine donated by the NML and the Government of Canada to the WHOFootnote 23. Since then, the vaccine (now rVSVΔG-ZEBOV-GP) has undergone further trials and used in the most recent outbreak of the Ebola Virus Disease in the Democratic Republic of Congo, under an Expanded Access Protocol or “compassionate use”Footnote 24.

In addition to the vaccine, the NML was involved in the development of ZMapp, a drug treatment for Ebola. This treatment is made of a combination of three monoclonal antibodies, two of which were developed at the NMLFootnote 25. ZMapp was first used in 2014 on a human: an American doctor who contracted Ebola while working in LiberiaFootnote 26.

NML's research on Ebola has been referenced widely in scientific papers. As found in the citation analysis, the article Reversion of advanced Ebola virus disease in nonhuman primates with ZMapp, published in Nature in October 2014, was referenced 472 times (after removing self-citations), out of which 93% were in journal articles, and the other 7% in books and monographs. Fifteen of the 26 co-authors on this article were not affiliated with the NML, and were mainly from various private and public organizations in the United States.

Staff from the NML has served on numerous deployments as part of mobile laboratory units to help provide rapid diagnostic support, including to the outbreaks of Ebola in the Democratic Republic of the Congo in 2003 and 2007, and the outbreak in Liberia, Sierra Leone, and Guinea in 2014 to 2016. NML personnel have also assisted with the current (2018-present) Ebola outbreak in the Democratic Republic of the Congo. ZMapp and the above article received international media attention when findings from the article were made public.

Overall, as demonstrated by an external evaluation conducted by the WHO and the Pan American Health Organization, Canada has a well-established and functioning laboratory system and "possesses a high level of competency to maintain access to and conduct laboratory testing for many communicable diseases"Footnote 27. Figure 10 demonstrates how the NML's work on the Ebola virus has helped address a serious infectious disease threat, both domestically and internationally.

Conclusions

The NML is a highly credible and relevant organization. It is well valued and recognized by its external and internal partners for its ability to provide core services to the public health sector and to innovate in order to remain relevant and effective at accomplishing its mandate of protecting Canadians against infectious diseases.

In recent years, the NML has made significant strides to modernize its tools in order to adopt new laboratory technologies, such as whole genome sequencing. It also implemented different innovations to address public health challenges or improve the detection, prevention, and response to infectious diseases. Innovation is not without challenges, as it requires the NML scientists to use different skills and to adapt the laboratory IT infrastructure. The transfer of innovation also requires the NML to support its partners in developing capacity. In a context where resources are limited, the NML and PHAC, as a whole, need to continue to maintain the right balance between creating the space necessary to foster a culture of innovation, while also meeting core public health functions and priorities.

The NML carries out its work through various networks and collaboration mechanisms. Collaboration with external partners is well established, but internal collaboration with PHAC Centres tends to be more ad hoc. In this regard, both the NML and PHAC Centres perceived that more work is required to improve communication and collaboration at the working level. External partners also noted a need to improve the communications on NML's expertise and services.

The evaluation found that NML's activities generally align with public health and PHAC priorities. The NML relies on insight from its various partners and networks to identify emerging issues of priority. It also participates in PHAC's various corporate governance mechanisms. However, the evaluation did not find evidence of a process at the program or Agency levels to foster strategic discussions on priorities that are common to both science and PHAC programs and policy. This would include measuring and reporting on the performance of both the NML and PHAC Centres at using laboratory science to inform public health interventions.

Overall, the quality of its work and its ability to be nimble has allowed the NML to make key contributions to society and public health in Canada and internationally. It develops expertise that provincial laboratories do not have the means to develop. It occupies specialized niches among laboratories in Canada and undertakes research or innovation to better understand, prevent, and respond to infectious diseases. It helps coordinate the sharing of expertise among partners in the public health system.

Recommendations

The quality, credibility and relevance of NML's work is well established but improvements to internal practices in the areas of collaboration, communication and strategic planning would strengthen the reach of NML's work and of PHAC as whole. These findings led to the following recommendations:

1. Implement consistent practices to plan research and innovation in order to create the space necessary to foster innovative thinking while also addressing key priorities related to NML's mandate.

The evaluation found that the NML does not have formalized and systematic processes to inform the prioritization of its research efforts, although such processes are in place in some areas of the laboratory. While it is important to provide space to scientists to enable them to be innovative, resources are limited and the NML's mandate is to address other public health priorities that are do necessarily call for innovation. Processes that are more consistent would help ensure that the NML can balance the need to be innovative with the need to fulfill core public health priorities. This would also likely help the NML to communicate on its work and engagement.

2. Improve communication with internal and external partners on the services and scientific expertise delivered by the NML. As part of these efforts, both the NML and the Centres from the Infectious Disease Prevention and Control Branch should examine best practices already in place and emulate what worked well in order to foster a more systematic two-way proactive collaboration and communication.

Navigating the laboratory can be challenge for external partners, who also reported a need to for the NML to communicate better on services offered. Partners, especially from PHAC Centres, do not appear to understand well how CNPHI operates and plans its activities. A few internal key informants also suggested that PHAC Centres do not always understand NML's activities.

There is a shared perception in PHAC Centres and the NML that more could be done to increase internal collaboration. Often, the NML and PHAC Centres are not aware of what the other is working on. As a result, they do not always know about opportunities for laboratory scientists and Centre epidemiologists to collaborate in order to achieve greater impacts on public health. That being said, the PHAC Centre responsible for responding to food-borne and water-borne enteric illnesses and the enteric pathogen division at the NML have set up a best practice to collaborate and coordinate the transition to whole-genome-sequencing and the use of data generated through this technology to inform surveillance and outbreak investigations.

3. Identify opportunities to formalize discussions and expectations between the NML and PHAC Centres on strategic priorities to connect laboratory science to policy.

The evaluation did not find evidence of a process at the program or Agency levels to foster strategic discussions on priorities that are common to both science and policy. Evaluation evidence also suggests that there is a need for PHAC to clarify expectations around the contribution of laboratory science to inform programs and policies.

Management Response and Action Plan

Management Response and Action Plan
Recommendations Management response Action Plan Deliverables Expected Completion Date Accountability Resources

Recommendation 1:

Implement consistent practices to plan research and innovation in order to create the space necessary to foster innovative thinking while also addressing key priorities related to NML's mandate.

Agree. NML will develop a science planning process to enable research planning and innovation management that provides a mechanism to identify linkage to priorities and mandate.
  • a) Develop and implement a database that itemizes all NML's research activities and allows linkage to priorities.
  • b) Develop and implement a management review procedure and administrative supports for planned research activities that will foster advanced planning, prioritization and innovation management.
  • c) Note that these deliverables are elements of the "Enabling Scientific Excellence" Initiative underway at the NML.
  • a) October 1, 2020
  • b) October 1, 2021
Director General, NML NIL

Recommendation 2:

Improve communication with internal and external partners on the services and scientific expertise delivered by the NML. As part of these efforts, both the NML and the Centres from the Infectious Disease Prevention and Control Branch should examine best practices already in place and emulate what worked well in order to foster a more systematic two-way proactive collaboration and communication.

Agree. NML will undertake initiatives to communicate information about services and expertise available at the NML directed at both internal and external stakeholders.
  • a) The NML DG will co-present with IDPCB colleagues information on current initiatives at the PHAC Executive Committee at periodic intervals.
  • b) NML will increase its presence on science.gc.ca in order to communicate information about expertise, research and services at NML to all stakeholders by:
    • i. Posting scientist profiles to showcase scientific expertise available at NML
    • ii. Posting 'science stories' (concise plain language summaries of recent publications by NML scientists).
  • c) The NML will extend invitations to IDPC Centres to attend appropriate "Lab Rounds" (a biweekly internal presentation highlighting current NML science) in order to better connect the working levels of NML with Centre counterparts.

Note that these deliverables are elements of the "Enabling Scientific Excellence" Initiative underway at the NML.

*All items for Recommendation 2 are currently paused for the COVID-19 response. The activities are expected resume within 6 months following the return to regular business.

  • a) October 1, 2020
  • b) October 1, 2022
  • c) October 1, 2022
Director General, NML NIL

Recommendation 3:

Identify opportunities to formalize discussions and expectations between the NML and PHAC centres on strategic priorities to connect laboratory science to policy.

Agree. The Infectious Disease Prevention and Control (IDPC) Branch will strengthen opportunities for integrated planning and prioritization.
  • a) The NML DG will co-present with IDPCB colleagues information on current initiatives at the PHAC Executive Committee at periodic intervals.
  • b) NML will update and share its Strategic Science Directions document with IDPC Centres (who may in turn share relevant strategic plans) to better connect the laboratory's science with policy directions.
  • a) October 1, 2020
  • b) October 1, 2022
  • c) October 1, 2022
Vice President, IDPC and Director General, NML NIL

Appendix 1 - Evaluation Methodology

Methodology and Scope

The purpose of the evaluation was to examine how the NML fosters and manages innovation, while also balancing core public health functions. The evaluation covered all of the NML's activities from April 1, 2014 to September 2019. The questions examined are as follows:

Fostering and managing innovation: How does the NML promote, foster, and manage innovation, and what can be learned from this process? What is the impact on other stakeholders, including PHAC Centres, and how can this be improved, if needed?

Data collection activities started in June 2019 and ended in November 2019. The OAE applied a sex- and gender-based analysis plus (SGBA+) lens to its analysis, where appropriate, for the purpose of examining how the NML's activities contribute to addressing the specific needs of diverse groups, including vulnerable and at-risk populations (See sections 4 and 6.2).

Data for the evaluation was collected using various sources of evidence as described in

Figure 11. Evaluation Methods and Lines of Evidence

Review of documents, files and financial data, including a client satisfaction survey, performance measurement data, and documentation from external sources. Financial data from 2014-15 to 2018-19 was examined to assess how resources were allocated and spent across the NML divisions and programs.

Key informant interviews were conducted in a semi-structured manner, based on a predetermined questionnaire. In total, 50 interviews, including a few group interviews, were conducted with the following groups:

  • Canadian academics and Non-Governmental Organizations (n=6)
  • International experts/collaborators (n=4)
  • Other federal departments (n=5)
  • Provinces (n=14, representing 9 provinces)
  • NML staff (n=11)
  • PHAC Centres (n=10)

Findings from interviews were reported based on the share of key informants who responded to a specific question with "a few" representing two or three respondents; "some/several" representing more than three respondents but less than the majority and "the majority/most/almost all" represent the majority of key informants who responded to the question.

Case studies on work carried out by the NML on the Ebola virus, "Omics", and the Zika virus. Case studies were completed using a combination of the methods outlined above, plus a literature review where necessary.

Comparative analysis was undertaken to investigate how other laboratories and similar research-based departments or agencies manage innovation and invest in research. This task involved conducting interviews and gathering relevant documents from the (i) National Research Council, (ii) Canadian Institute for Health Research, (iii) US Center for Disease Control (Enteric Laboratory Disease Branch), and (iv) Canadian Food Inspection Agency (Innovation Centre of Expertise). These organizations were selected due to their research focus and the existence of laboratory activities within their purview.

Citation analysis: The Health Library conducted a search in Scopus for the 10 documents authored by people affiliated with the NML or PHAC referenced most frequently. The search was restricted to 2014 to 2019, inclusively, and PHAC affiliations were narrowed by city (Winnipeg, Guelph, and Saint-Hyacinthe).

Strengths and Limitations

Most evaluations face constraints that may have implications for the validity and reliability of evaluation findings and conclusions. This evaluation encountered three main limitations:

  1. The evaluation relied predominantly on qualitative information and perspectives from key informants. As a result, it is unknown to what extent findings are representative of all NML partners and stakeholders. To mitigate this effect, key informants were selected to be representatives of various NML partners and stakeholders. This allowed capturing a variety of perspectives about the NML.
  2. Some of the key informants suggested by the NML had strong positive views about its activities. This could have introduced a positive bias to the evaluation findings. To mitigate this effect, the evaluation team undertook its own research to identify potential partners and stakeholders of the NML. The research consisted of identifying potential collaborators from publications co-authored by NML staff, horizontal projects involving NML staff, and directories of provincial laboratories. These additional interviews helped corroborate information reported by informants suggested by the NML.
  3. Limited data was available to measure the impacts and contribution of NML's work. Data available on the NML's achievements was generally limited. As a result, the assessment of the NML's contribution to society relied mainly on key informant interviews. The effects of the NML's work for society were only assessed in a descriptive way. Where possible, the evaluation complemented findings from key informant interviews with relevant performance data, concrete examples of the NML's work identified from the document review, and a citation analysis.

Footnote References

Footnote References

While the NML is part of PHAC, in this evaluation key informants working at the NML are referred to as "NML key informants" while those working in the Centres and programs are referred to as "PHAC Centre key informants".

Return to footnote i referrer

The NML's budget for 2018-19, includes employee and students' salaries, employee benefit plan established at 20%, Operation and Management and Capital. Data excludes targeted funding for Ebola projects and external funding sources (e.g., Genomic Research and Development Initiative).

Return to footnote ii referrer

Reference services are available to provincial and territorial laboratories. As such, other groups of key informant interviews do not use these services.

Return to footnote iii referrer

Centre for Food-borne, Environmental and Zoonotic Infectious diseases (CFEZID), Centre for Communicable Disease and Infection Control (CCDIC), Centre for Immunization and Respiratory Infectious Disease (CIRID) and Centre for Emergency Preparedness and Response (CEPR).

Return to footnote iv referrer

Endnotes

Footnote 1

Calculation based on NML resources and FTEs allocations planned in the IDPC Operational Plan 2019-20 and PHAC budget and FTEs reported in the 2019-20 PHAC Departmental Plan. Retrieved from: https://www.canada.ca/en/public-health/corporate/transparency/corporate-management-reporting/reports-plans-priorities/2019-2020-report-plans-priorities.html#a4.2

Return to footnote 1 referrer

Footnote 2

Viral Zoonoses. https://cnphi.canada.ca/gts/laboratory/1022.; Molecular Detection of Zika Virus by Reverse Transcriptase PCR (RT-PCR). Retrieved from: https://cnphi.canada.ca/gts/reference-diagnostic-test/10983?labId=1022

Return to footnote 2 referrer

Footnote 3

World Health Organization. 2017. One Health. Retrieved from: https://www.who.int/features/qa/one-health/en/

Return to footnote 3 referrer

Footnote 4

Canadian Public Health Laboratory Network. 2016. Strategic Plan 2016-2020. Retrieved from: https://www.canada.ca/en/services/health/publications/science-research-data/2016-2020-canadian-public-health-laboratory-network-strategic-plan.html

Return to footnote 4 referrer

Footnote 5

Public Health Agency of Canada. 2018 Compilation of the National Microbiology Laboratory Science Stories.

Return to footnote 5 referrer

Footnote 6

Public Health Agency of Canada. 2019. Accelerating Our Response: GofC 5 year Action Plan for STBB. p.20. Retrieved from: https://www.canada.ca/content/dam/hc-sc/documents/services/reports-publications/accelerating-our-response-five-year-action-plan-sexually-transmitted-blood-borne-infections/stbbi-federal-action-plan-en.pdf

Return to footnote 6 referrer

Footnote 7

Herder, M., Graham, J.E. and Gold, R. 2020-01-16. From discovery to delivery: public sector development of the rVSV-ZEBOV Ebola vaccine.. Journal of Law and Biosciences. 1-14.

Return to footnote 7 referrer

Footnote 8

Public Health Agency of Canada. Leading the way with HIA: Vaccine for the deadly infection gains traction; Blog post, May 29, 2018. Retrieved from: http://www.science.gc.ca/eic/site/063.nsf/eng/97611.html

Return to footnote 8 referrer

Footnote 9

Public Health Agency of Canada. 2018 Compilation of the National Microbiology Laboratory Science Stories "Evaluating of the Diasorin Liaison XL Zika capture igM CMIA for Zika virus serological testing"

Return to footnote 9 referrer

Footnote 10

Public Health Agency of Canada. 2019. Evaluation of the National Collaborating Centres for Public Health Program 2014-15 to 2018-19. Retrieved from: https://www.canada.ca/en/public-health/corporate/transparency/corporate-management-reporting/evaluation/2014-2015-2018-2019-evaluation-report-national-collaborating-Centres-public-health-program.html

Return to footnote 10 referrer

Footnote 11

Public Health Agency of Canada 2018–19 Departmental Plan. Retrieved from: https://www.canada.ca/en/public-health/corporate/transparency/corporate-management-reporting/reports-plans-priorities/2018-2019-report-plans-priorities.html; Minister of Health Mandate Letter. October 04, 2017. ARCHIVED. Retrieved from: https://pm.gc.ca/en/mandate-letters/archived-minister-health-mandate-letter-0; Public Health Agency of Canada. 2019–20 Departmental Plan. Retrieved from: https://www.canada.ca/en/public-health/corporate/transparency/corporate-management-reporting/reports-plans-priorities/2019-2020-report-plans-priorities.html

Return to footnote 11 referrer

Footnote 12

Canadian Public Health Laboratory Network Strategic Plan 2016-2020. Retrieved from: https://www.canada.ca/en/services/health/publications/science-research-data/2016-2020-canadian-public-health-laboratory-network-strategic-plan.html

Return to footnote 12 referrer

Footnote 13

Let's Talk CIHR: What are your health research priorities? Retrieved from: https://letstalk-cihr.ca/letstalk

Return to footnote 13 referrer

Footnote 14

Public Health Agency of Canada Act., S.C. 2006, c. 5. Retrieved from: https://lois-laws.justice.gc.ca/eng/acts/P-29.5/page-1.html

Return to footnote 14 referrer

Footnote 15

Public Health Agency of Canada Act., S.C. 2006, c. 5. Retrieved from: https://lois-laws.justice.gc.ca/eng/acts/P-29.5/page-1.html

Return to footnote 15 referrer

Footnote 16

Public Health Agency of Canada. 2018 Compilation of the National Microbiology Laboratory Science Stories.

Return to footnote 16 referrer

Footnote 17

World Health Organization and Pan American Health Organization. 2018. Joint External Evaluation of IHR Core Capacity of Canada. Mission Report: June 11 to 20 2018, p.25.

Return to footnote 17 referrer

Footnote 18

Health Canada. 2016. Update on Health Canada's Water Quality Activities: Retrieved from: http://www.cwwa.ca/pdf_files/2016Stephanie.pdf

Return to footnote 18 referrer

Footnote 19

Public Health Agency of Canada. 2019. Footprints in the Snow Mobilizing north to stop tuberculosis. Retrieved from http://science.gc.ca/eic/site/063.nsf/eng/97784.html

Return to footnote 19 referrer

Footnote 20

Public Health Agency of Canada. 2018 Compilation of the National Microbiology Laboratory Science Stories; Boutin CA, Venne S, Fiset M, Fortin C, Murphy D, Severini A, Martineau C, Longtin J, Labbé AC. (Feb 2018). Lymphogranuloma venereum in Quebec: re-emergence among men who have sex with men. Can Commun Dis Rep 44(2):55-61. doi: http://doi.org/10.14745/ccdr.v44i02a04.

Return to footnote 20 referrer

Footnote 21

NML Compilation of science stories 2018; Dibernardo A, Turell MJ, Lindsay LR, Loomer C, Iranpour M. (Dec 2017). Vector competence of some mosquito species from Canada for Zika virus. J Am Mosq Control Assoc, 33(4):276-281. doi: https://doi.org/10.2987/17-6664.1.

Return to footnote 21 referrer

Footnote 22

Attaran, A. & Nickerson, J.W. (2014). Is Canada patent deal obstructing Ebola vaccine development? The Lancet, 384, e61. http://dx.doi.org/10.1016/S0140-6736(14)62044-4; CBC News Network. (2014-08-13). Message from Chief of Pathogens at the National Microbiology Laboratory. Retrieved from http://www.infomedia.gc.ca/hc-sc.

Return to footnote 22 referrer

Footnote 23

Pinchin, K. (2019-08-13). Behind the Life-saving Ebola Vaccine is a Story of Missed Opportunity. Retrieved from https://www.pbs.org/wgbh/frontline/; Outhwaite, K. &Taylor, G. (2017-01-16). Canada remains actively engaged in the efforts to fight Ebola, CMAJ, 189 (2), p.E80; WHO Media Centre. 2015-07-31. World on the verge of an effective Ebola vaccine. Retrieved from https://www.who.int/mediaCentre/news/releases/2015/effective-ebola-vaccine/en/; WHO Media Centre. 2015-07-31. World on the verge of an effective Ebola vaccine. Retrieved from https://www.who.int/mediaCentre/news/releases/2015/effective-ebola-vaccine/en/; CBC News Network. 2014-10-01. Ebola news conference in Banff with CPHO and Minister Ambrose. Retrieved from http://www.infomedia.gc.ca/hc-sc.

Return to footnote 23 referrer

Footnote 24

CBC. 2018-05-22. Congo announces 6 new confirmed cases of Ebola virus. Retrieved from https://www.cbc.ca/news/health/ebola-congo-1.4672744; European Medicines Agency (2019-10-18). First vaccine to protect against Ebola. Retrieved from https://www.ema.europa.eu/en/news/first-vaccine-protect-against-ebola; WHO – SAGE (2018-10). Update with the development of Ebola vaccines and implications of emerging evidence to inform future policy recommendations. Retrieved from https://www.who.int/immunization/sage/meetings/2018/october/presentations_background_docs/en/; R&D Blueprint for WHO (2019-10). Overview of the current research, development and use, of vaccines against Ebola. Retrieved from https://www.who.int/immunization/sage/meetings/2019/october/presentations_background_docs/en/.

Return to footnote 24 referrer

Footnote 25

Murin, C.D., Fusco, M.L., Bornholdt, Z.A., Qiu, X., Olinger, G.G., Zeitlin, L., ... Saphire, E.O. (2014). Structures of protective antibodies reveal sites of vulnerability on Ebola virus. Proceedings of the National Academy of Sciences of the United States of America, 111(48); Branswell,H.(2014-08-05). Experimental Ebola drug draws on Canadian research: Antibodies developed at national lab in Winnipeg, Maclean's. Retrieved from https://www.macleans.ca/news/world/experimental-ebola-drug-draws-on-canadian-research/; Tam, T. 2018-05-03. Fifteen years post-SARS: Key milestones in Canada's public health emergency response. Canada Communicable Disease Report, 44(5): 98-101.

Return to footnote 25 referrer

Footnote 26

CBC (2018-04-07) Doctor who survived Ebola visits Winnipeg to thank ZMapp scientists. Retrieved from: https://www.cbc.ca/news/canada/manitoba/brantly-kobinger-ebola-1.4609586

Return to footnote 26 referrer

Footnote 27

World Health Organization and Pan-American Health Organization. 2018. Joint External Evaluation of IHR Core Capacity of Canada. Mission Report: June 11 to 20 2018, p.24.

Return to footnote 27 referrer

Page details

Date modified: