Evaluation of The Food Safety Program 2012-13 to 2017-18

Download the alternative format
(PDF format, 1.2 MB, 56 pages)

Organization: Public Health Agency of Canada

Published: 2019-07-12

Prepared by Office of Audit and Evaluation Health Canada and the Public Health Agency of Canada

March 2019

Executive Summary
Management Response and Action Plan
1.0 Evaluation Purpose
2.0 Program Description
3.0 Evaluation Description
- 3.1 Evaluation Scope, Approach and Design
- 3.2 Limitations and Mitigation Strategies
4.0 Findings
5.0 Conclusions
- 5.1 Achievement of Expected Outcomes
- 5.2 Demonstration of Economy and Efficiency
6.0 Recommendations
Appendix 1: Evaluation Description
Appendix 2: Logic Model
Endnotes

List of Tables and Charts

Table 1: Program Expenditures (2012-13 to 2017-18)
Table 2: Limitations and Mitigation Strategies
Chart 1: Number of HRAs Conducted by Health Canada (2011-2018)
Table 3: Number of Food Directorate Regulatory Pre-Market Assessments Completed within Service Standards
Table 4: Number of VDD Regulatory Pre-Market Assessments Completed within Service Standards
Chart 2: Number of Non-Regulatory Pre-market Assessments by Type (2011-2017)
Chart 3: Annual national Salmonella rates reported to the National Enteric Surveillance Program (NESP, 1997-2017)
Chart 4: Annual national Salmonella Enteritidis rates reported to the National Enteric Surveillance Program (NESP, 1997-2017)
Chart 5: Annual national E. Coli 0157 rates reported to the National Enteric Surveillance Program (NESP, 2006-2017)
Table 5: Planned Spending and Expenditures (2012-13 to 2017-18) ($)
Table 6: Core Evaluation Issues and Questions

List of Acronyms

CFIA: Canadian Food Inspection Agency
DPR: Departmental Performance Report
EBP: Employee Benefit Plan
FSP: Food Safety Program
FTE: Full-Time Equivalent
HPFB: Health Products and Food Branch
HRA: Health Risk Assessment
IBR: Incorporation by Reference
MOU: Memorandum of Understanding
O&M: Operating and Maintenance
OECD: Organisation for Economic Co-operation and Development
P/T: Province/Territory
PHAC: Public Health Agency of Canada
PMRA: Pest Management Regulatory Agency
PPIAD: Policy, Planning and International Affairs Directorate
RAPB: Regional and Program Bureau
RMOD: Resource Management and Operations Directorate
ROEB: Regulatory Operations and Enforcement Branch
RORB: Regulatory Operations and Regions Branch
VDD: Veterinary Drugs Directorate

Executive Summary

The purpose of this evaluation was to assess the performance of the Food Safety Program (FSP) for the period from April 2012 to March 2018. The evaluation was also designed to highlight accomplishments and lessons learned, as well as challenges experienced by the Program.

Program Description

FSP operates under the authority of the Department of Health Act, the Food and Drugs Act and the Food and Drug Regulations, which provide the framework for Health Canada to develop, maintain, and implement a regulatory system associated with food safety and nutrition. The Program is managed by the Health Products and Food Branch (HPFB), with direct support from the Regulatory Operations and Enforcement Branch (ROEB)^Footnote a and the Canadian Food Inspection Agency (CFIA). It encompasses activities undertaken to help ensure that food products used by Canadians are safe for their health. It also includes activities pertaining to veterinary drugs administered to food-producing animals.

The FSP includes the following key activities:

Conducting scientific research to support standard setting;
Developing, publishing, and maintaining food standards, policies, regulations, and guidelines, as well as conducting risk analysis activities;
Conducting outreach with partners and stakeholders, responding to public inquiries and providing information to Canadians;
Conducting pre-market assessments of selected ingredients, processes, and final foods;
Conducting health risk assessments (HRAs) to support the management of food safety incidents; and
Conducting surveillance and monitoring as they relate to food safety and nutrition.

Summary of Results

Achievement of Expected Outcomes

Overall, the Food Safety Program has achieved, or is making significant progress towards achieving its key program objectives. In addition, according to a 2014 Conference Board of Canada report, Canada's food safety system is one of the best in the world. However, this evaluation found some areas where activities could be strengthened.

The Program used a variety of methods to communicate and engage with Canadians, partners, and stakeholders. Overall, stakeholders and partners were generally positive regarding the usefulness and relevance of the information they receive from the Program; however, opinions tended to be somewhat less positive with respect to its timeliness. Furthermore, stakeholders and partners were generally pleased with the Program's engagement efforts; however, some expressed a desire for earlier engagement and more frequent collaboration with the Program in areas such as regulation and policy development, research, and planning.

The FSP reached out to Canadians to try to make them more knowledgeable of various food safety issues and encourage them to practice safe food handling, preparation, and storage. In general, Canadians' knowledge of food safety issues is fairly high and Canadians exhibit a number effective food safety behaviours; however, a few notable knowledge gaps and unsafe behaviours still exist. Most noteworthy is the fact that a clear majority of certain at-risk groups do not consider themselves more at risk for complications from food poisoning than the average person.

Health Risk Assessments (HRAs) and pre-market assessments are important components of the FSP and Health Canada completes both types of assessments according to established service standards. However, Health Portfolio partners noted areas where improvements could be made in terms of the timeliness, clarity, and usefulness of HRAs, especially when there was no clear guidance or policy on a particular issue. Furthermore, there were a number of improvements to the pre-market assessment process over the evaluation period, including greater predictability; however, some industry key informants still believed it is too slow and this can affect companies' willingness to bring new products to Canada.

During the evaluation period, the FSP produced a number of policy and guidance documents for its stakeholders and partners. The evaluation found that these stakeholders and partners generally view these policies and guidelines as being of high quality and high importance, as they often provide greater clarity and certainty to their food safety activities, and they use this information in a variety of different ways (e.g., risk management decisions, guidance development, sharing with members). At the same time, some Portfolio partner key informants felt that Health Canada needed to take a more proactive leadership role in the development of policies and guidance, particularly in response to the transition towards a more outcome-based approach to Canada's food safety system.

Demonstration of Economy and Efficiency

Over the period of the evaluation, actual spending was very similar to planned budgets for the FSP. Furthermore, there were a number of examples of how the FSP had become more efficient over the evaluation period. These included the use of Marketing Authorization and Incorporation by Reference, the development of pre-market submission guides, and the use of the RADAR database and the "Trackers Club" to monitor pre-market submissions.

At the same time, a number of resource constraints (e.g., reliance on targeted funding sources, access to regulatory drafters, focus on mandate commitments, staff turnover) made it challenging for the Food Directorate to make progress on activities that were more proactive in nature. These included regulatory modernization and the development of policies and guidelines to support the transition to a more outcome-based approach to food safety.

Finally, the FSP's performance measurement was focused on key service standards, and performance data for certain key outcomes did not capture the perspectives of Canadians, even though they were explicitly identified in their logic model.

Recommendations

Recommendation 1

Work with Health Portfolio partners to explore ways to better operationalize the current HRA process (e.g., define roles and responsibilities of Food Safety Program partners, review and update existing service standards as required for responsive HRAs, provide mechanisms for ongoing dialogue with Portfolio partners on risk assessments and risk management decisions related to HRA processes, as well as explore timelines to complete longer-term HRAs in collaboration with partners).

Health Risk Assessments (HRAs) conducted by the Food Directorate were completed according to established service standards; however, Portfolio partners noted certain areas where improvements could be made in terms of the timeliness, clarity, and usefulness of the HRAs, especially when there was no clear guidance or policy on a particular issue.

Recommendation 2

Increase coordination and collaboration between Health Canada and Health Portfolio partners at the planning stage to discuss issues such as research plans and the alignment of objectives and priorities across the Health Portfolio.

Portfolio partners expressed a desire for earlier and more regular collaboration with the FSP on regulations, policy, and research. Portfolio partners suggested that more could be done to engage them at the planning stages to discuss issues like research plans, to align objectives and priorities across departments, and to ensure that they understand how they fit within the overall structure of Health Canada's food safety activities. It was also suggested that, since the needs of the various program partners were not always thoroughly discussed prior to the Program establishing its work plans, more interaction and engagement by the Program could help ensure better alignment in key activities (e.g., research) and better leveraging of partner data and activities.

Recommendation 3

Consider increasing outreach and education efforts aimed at Canadians to help address various food safety knowledge and behaviour gaps.

Canadians' knowledge of food safety issues is fairly high; however, a few notable knowledge gaps still remain. Most noteworthy is the fact that a clear majority of certain at-risk groups do not consider themselves to be more at risk for complications from food poisoning than the average person. Furthermore, a significant proportion of Canadians continue to underestimate the risks associated with frozen raw breaded chicken products.

Recommendation 4

Increase efforts to obtain Canadians' perspectives on the timeliness and usefulness of Health Canada information on food safety and the effectiveness of its engagement efforts.

The Program collects and uses performance information; however, it focuses primarily on collecting data related to service standards and, in some areas, does not capture the perspectives of Canadians. For instance, reach is tracked by the number of knowledge products ordered by health professionals and other intermediaries, as well as by targeted mail outs, but very little data is collected on the impact these have on knowledge uptake or behavioural change among Canadians. Additionally, there is no evidence that the Program has collected Canadians' perspectives on the timeliness and usefulness of the information, as per the Program's logic model.

Management Response and Action Plan

Evaluation of the Food Safety Program: 2012-13 to 2017-18
Recommendations	Response	Action Plan	Deliverables	Expected Completion Date	Accountability	Resources
Recommendation as stated in the evaluation report	Identify whether program management agrees, agrees with conditions, or disagrees with the recommendation, and why	Identify what action(s) program management will take to address the recommendation	Identify key deliverables	Identify timeline for implementation of each deliverable	Identify Senior Management and Executive (DG and ADM level) accountable for the implementation of each deliverable	Describe the human and/or financial resources required to complete recommendation, including the source of resources (additional vs. existing budget)
Recommendation 1 Work with Health Portfolio partners to explore ways to better operationalize the current HRA process (e.g., define roles and responsibilities of Food Safety Program partners, review and update existing service standards as required for responsive HRAs, provide mechanisms for ongoing dialogue with Portfolio partners on risk assessments and risk management decisions related to HRA processes, as well as explore timelines to complete longer-term HRAs in collaboration with partners).	Agree	Health Canada's Director General of the Food Directorate to engage counterparts at the Canada Food Inspection Agency and Public Health Agency of Canada, through the DG Committee on Food Safety, to review the current governance mechanisms for maintaining clear responsibilities with respect to Health Risk Assessments.	In collaboration with CFIA and PHAC: 1. Review and, if necessary, initiate process to update the MOU, which describes mandates, roles, and responsibilities of the partner organizations.	1. May, 2019	DG Food Directorate, ADM HPFB	Existing resources will be used for preliminary discussions with CFIA and PHAC.
			2. Establish interdepartmental working groups to discuss opportunities for improvement in the HRA process.	2. September, 2019
			3. Review and update, as required, the Standard Operating Procedures (SOP) for providing Health Risk Assessments to CFIA in the context of Food Safety investigations.	3. January, 2020
Recommendation 2 Increase coordination and collaboration between Health Canada and Health Portfolio partners at the planning stage to discuss issues such as research plans and the alignment of objectives and priorities across the Health Portfolio.	Agree	Health Canada to review existing portfolio coordination of food safety issues and research to maximize collaboration, transparency and timely decision making.	1. Undertake a review of existing Health Portfolio governance structures addressing food, including the Committee on Food Safety, as well as other existing portfolio collaboration mechanisms, to produce a food governance report with recommendations.	1. December, 2019	DG Food Directorate, ADM HPFB	Existing resources
			2. Undertake a review of the planning processes that informs Food Safety Program research priorities and produce a report with recommendations to build on and improve existing collaboration and coordination	2. December, 2019
			3. Implement recommendations that stem from the reviews.	3. December, 2020
Recommendation 3 Consider increasing outreach and education efforts aimed at Canadians to help address various food safety knowledge and behaviour gaps.	Agree	Undertake a review of existing food safety risk communications to develop and implement a Food Safety Risk Communications Action Plan, aligned with CPAB's overarching communications plan.	In collaboration with CFIA and PHAC: 1. Consult with Canadians and health partners on Food Safety Risk Communications	1. October, 2019	DG Food Directorate, ADM HPFB ADM CPAB	Food Safety Risk Communications Action Plan development, and participation in the development of CPAB's overarching communications plan, will use existing resources (estimated 2 FTEs). Additional funding may be needed for consultation with Canadians and implementation of the Food Safety Risk Communications Action Plan.
			2. Develop a draft Food Safety Risk Communications Action Plan to implement CPAB's overarching communications plan, addressing areas of concern to Canadians, knowledge and behaviour gaps, preferred communication channels, partnership opportunities with stakeholder groups, etc.	2. January, 2020
			3. Establish an evaluation framework to assess the effectiveness of Food Safety Risk Communications.	3. September, 2020
			4. Integrate existing and increased Food Safety Risk Communications approaches into Food Directorate Operational Plans.	4. December, 2020
Recommendation 4 Increase efforts to obtain Canadians' perspectives on the timeliness and usefulness of Health Canada information on food safety and the effectiveness of its engagement efforts.	Agree	Additional performance measures will be identified or developed to address Canadians' perceptions related to two short term outcomes in the Food & Nutrition Program Logic Model.	1. Update Food and Nutrition Program Performance Information Profile (PIP) to demonstrate how Canadians' perceptions will be measured.	1. October, 2019	DG Food Directorate, ADM HPFB ADM CPAB	Plan to use existing resources but may need to revisit once options in place for collection of Canadians' perceptions in these areas.
			2. Implement mechanisms to assess Canadians perceptions of the timeliness and usefulness of Health Canada information on food safety and the effectiveness of its engagement efforts.	2. September, 2020
			3. Produce a report on Canadians perceptions.	3. March, 2021

1.0 Evaluation Purpose

2.0 Program Description

2.1 Program Profile

The FSP is the federal health authority responsible for establishing regulations, guidelines, standards, and policies pertaining to food safety and nutrition, as well as conducting reviews and assessments of the safety of food ingredients, veterinary drugs for food-producing animals, food processes, and final foods. The Program conducts risk assessments on the chemical, microbiological, and nutritional safety of foods. In addition, the Program plans and implements surveillance and research initiatives in support of Health Canada (HC)'s mandate of setting food standards.

The Food and Drugs Act and the Food and Drug Regulations provide the regulatory framework for the FSP. The legislative and policy framework under which the FSP operates also includes:

the Canada Health Act;
the Financial Administration Act;
the Access to Information Act and Privacy Act; and
the Public Service Modernization Act.

The Program is managed through the Health Products and Food Branch (HPFB), with direct support from the Regulatory Operations and Enforcement Branch (ROEB)^Footnote b and the Canadian Food Inspection Agency (CFIA). It encompasses activities undertaken to help ensure that food products used by Canadians are safe for their health, as detailed below. It also includes activities pertaining to veterinary drugs administered to food-producing animals.

The FSP includes the following key activities:

Conducting scientific research, including laboratory work, and analytical method development in support of food safety and nutrition activities, such as standard setting.
Developing, publishing, and maintaining food standards, policies, regulations, and guidelines, and conducting risk analysis activities. FSP also conducts standard-setting activities associated with the safety, quality, and effectiveness of drugs used in food-producing animals.
Conducting outreach with partners and stakeholders, responding to public inquiries, and providing information to Canadians. A key feature of this activity is using a variety of communication avenues to inform the public, as well as educating and interacting with partners and stakeholders, about the work of the FSP and the regulatory framework for food safety.
Conducting pre-market assessments of selected ingredients, processes, and final foods. FSP conducts pre-market evaluations of submissions on food additives, flavouring agents, food packaging materials, processing aids, processes (e.g., food irradiation), and novel foods to determine risks to human health and the appropriateness of food product labelling. The Program also assesses industry submissions on veterinary drugs used in food-producing animals.
Conducting health risk assessments (HRAs) to inform the management of food safety investigations. FSP is responsible for conducting HRAs that support food safety investigations by providing essential information that enables regulatory compliance authorities (i.e., CFIA, P/Ts) to make appropriate and consistent risk management decisions, such as food recalls.
Conducting surveillance and monitoring as they relate to food safety and nutrition. FSP develops policies, strategies, and methods, as well as working with program partners and stakeholders to coordinate and implement a consistent approach to surveillance and monitoring as they relate to food safety and nutrition.

2.2 Program Context

There are a number of contextual factors that influence food safety in Canada, including the FSP.

Globalization of production and supply chains: Given the growing number of free trade agreements that Canada has signed, differences between countries have become more evident in relation to regulatory regimes, as well as the production and distribution of food products. Ingredients and finished products are sourced from other countries around the world, where oversight may be less stringent when compared with Canadian standards.^Endnote 1 This could result in the potential for increased health risks associated with foreign pathogens and counterfeit products.^Endnote 2
Rapidly evolving science and technology: Scientific and technological changes relating to food products continue to advance at a rapid pace. New products, formulations, and technologies are continuously entering the market. These changes have put pressure on regulatory bodies, as they increase potential new risks to human and animal health.^Endnote 3
Changing consumer buying patterns and dietary preferences: Consumers are requesting and consuming a much broader range of food products, such as the year-round availability of seasonal fruits and vegetables, more convenient prepared foods (e.g., bagged salads, ready-to-eat meals), and international food products and ingredients.^Endnote 4,^Endnote 5 Furthermore, there are many social factors that will have an impact on food safety in Canada, such as the growth of an aging population that is specifically susceptible to food-borne illness due to weakened or impaired immune systems.^Endnote 6
Changes in the way we communicate: In a world of rapidly evolving technology, communication tools such as text messaging, emailing, photo sharing, and social networking have greatly changed the ways in which information is delivered and shared.^Endnote 7

2.3 Roles and Responsibilities

The core activities of the FSP rest within the Food Directorate of the HPFB at Health Canada, the Veterinary Drugs Directorate (VDD), the Resource Management and Operations Directorate (RMOD), Policy, Planning and International Affairs Directorate (PPIAD) and ROEB.

The roles and responsibilities of these key participants in relation to food safety are the following:

The Food Directorate conducts assessments of food industry submissions; develops, updates and disseminates policies, guidelines, regulations, standards, strategies, and consumer information to support Canadians in making decisions about food; performs HRAs; conducts surveillance and monitoring; conducts research and method development; and coordinates priorities and risk management approaches within Canada's food safety system. In case of a food-borne illness outbreak in more than one province, the Food Directorate supports PHAC and CFIA, as part of the coordinated federal investigation and response, by providing HRAs to inform decision making.
The VDD conducts pre- and post-market assessments of industry submissions on veterinary drugs; performs monitoring and standard-setting activities associated with the safety, quality, and effectiveness of drugs used in animals; and promotes prudent use of veterinary drugs and setting standards for such use. In the area of food safety, VDD develops standards, policies, and regulations concerning the sale of veterinary drugs for use in food-producing animals and resulting drug residues in foods derived from animals, such as meat, milk, eggs, and honey. VDD provides assistance to CFIA in managing food safety incidents related to the presence of veterinary drug residues in food products. VDD also advanced a complementary set of regulatory and policy initiatives to manage antimicrobial resistance risks associated with the use of antimicrobials in food animals, as part of the Government of Canada's "Antimicrobial Resistance and Use in Canada: A Federal Framework for Action".
The RMOD provides HPFB-wide direction, oversight, coordination, guidance, and advice on effective and efficient management of operations and resources for HPFB.
PPIAD provides leadership and support on policy development and coordination of horizontal issues of strategic importance. It also provides leadership with respect to international affairs. Additionally, PPIAD develops legislative and regulatory proposals for the Food and Drugs Act and its Regulations, by working in close collaboration with all program partners.
ROEB is responsible for regional program activities and laboratories. Food-related laboratories exist in three locations in the country: Longueuil, Quebec; Scarborough, Ontario; and Burnaby, British Columbia.

The FSP also collaborates with a wide range of internal and external partners and stakeholders, including Health Canada's Communications and Public Affairs Branch (CPAB), industry, P/Ts, CFIA, PHAC, international regulators, non-governmental organizations, and several international organizations, such as the World Health Organization, the Food Agriculture Organization of the United Nations, and the Codex Alimentarius Commission.

2.4 Program Resources

As shown in Table 1 below, expenditures for the FSP totalled approximately $356M over the fiscal years 2012-13 to 2017-18.

Table 1: Program Expenditures (2012-13 to 2017-18)
Fiscal Year	Actual Spending
Fiscal Year	FTE	Total salary (EBP and Salary	O&M	Capital	Total
Food Directorate
2012-13	270	26,874,016	6,615,050	396,471	33,885,536
2013-14	349	35,805,154	6,448,862	469,968	42,723,984
2014-15	349	36,992,649	5,929,622	837,170	43,759,441
2015-16	345	35,077,136	6,285,927	3,123,427	44,486,490
2016-17	350	36,216,295	6,318,314	3,495,190	46,029,800
2017-18	341	40,292,706	5,796,548	1,852,637	47,941,891
TOTAL	no data	211,257,955	37,394,323	10,174,863	258,827,141
VDD
2012-13	40	4,637,255	482,255	64,682	5,184,192
2013-14	35	4,124,813	205,520	no data	4,330,333
2014-15	32	3,986,722	271,075	no data	4,257,797
2015-16	32	3,856,485	169,303	no data	4,025,788
2016-17	30	3,562,649	186,093	no data	3,748,742
2017-18	30	3,914,330	346,695	no data	4,261,025
TOTAL	no data	24,082,254	1,660,942	64,682	25,807,878
Overhead (ADM, PPIAD, RMOD, Litigation, other)
2012-13	27	4,615,110	252,975	27,829	4,895,914
2013-14	31	3,225,629	476,232	no data	3,701,861
2014-15	33	3,807,892	303,664	no data	4,111,557
2015-16	36	3,860,978	631,347	no data	4,492,324
2016-17	30	3,283,333	290,425	52,317	3,626,074
2017-18	32	3,687,389	292,271	60,096	4,039,755
TOTAL	no data	22,480,330	2,246,914	140,242	24,867,486
Total RAPB/RORB
2012-13	33	4,312,175	769,969	770,495	5,852,639
2013-14	54	5,636,005	622,365	835,860	7,094,229
2014-15	50	5,346,537	782,464	1,309,043	7,438,043
2015-16	48	4,695,404	756,619	172,905	5,624,928
2016-17	48	5,564,255	641,351	11,495	6,217,101
2017-18	38	5,225,154	945,793	1,637,145	7,808,092
TOTAL	no data	30,779,531	4,518,560	4,736,942	40,035,033
CPAB
2012-13	no data	68,339	600,259	no data	668,598
2013-14	no data	42,984	497,605	no data	540,589
2014-15	no data	92,005	461,709	no data	553,714
2015-16	no data	65,721	327,001	no data	392,722
2016-17	no data	72,716	484,950	no data	557,666
2017-18	no data	137,996	309,653	no data	447,649
TOTAL	no data	479,762	2,681,177	no data	3,160,939
CSB - Real Property
2016-17	no data	no data	3,555,293	21,002	3,576,295
Total Health Canada
2012-13	370	40,506,894	8,720,509	1,259,477	50,486,879
2013-14	469	48,834,585	8,250,584	1,305,827	58,390,996
2014-15	464	50,225,805	7,748,534	2,146,212	60,120,552
2015-16	461	47,555,723	8,170,196	3,296,332	59,022,252
2016-17	458	48,699,249	11,476,426	3,580,004	63,755,679
2017-18	440	53,257,577	7,690,960	3,549,877	64,498,414
TOTAL	no data	289,079,833	52,057,209	15,137,730	356,274,772
Numbers for years listed above do not necessarily correspond to the Report on Plans and Priorities (RPP) and Departmental Performance Reports (DPRs), mainly due to the varying percentage of the Employee Benefit Plan (EBP) calculated on salaries. Uncontrollable salaries are also not included in actuals, as the funding is statutory. 2012-13: Actuals are less than the DPR, mainly due to EBP variances and uncontrollable salaries. 2017-18: The increase in salaries from 2016-17 was mainly due to payments for collective agreements (including retroactive pay). Therefore, this amount may drop in 2018-19. RAPB/RORB/ROEB: RORB was officially created in 2016-17 and then renamed ROEB in February 2019. The reorganization resulted in the regionally-based Drug Analysis Service (DAS) and Food and Health Products (formerly the Inspectorate) laboratories being reallocated under the Laboratories Directorate, a national laboratory program. This also resulted in the coding of overhead to the Food Safety program.

3.0 Evaluation Description

3.1 Evaluation Scope, Approach and Design

This evaluation was an impact evaluation that assessed results and outcomes, as well as issues like alternatives and improvements. The evaluation covered key activities undertaken by the Food Directorate, VDD, and ROEB, as related to Food Safety, from April 2012 to March 2018.

The evaluation did not include the major initiatives related to the Healthy Eating Strategy (e.g., regulations related to front-of-package labeling), as the implementation of these initiatives is ongoing and not expected to be completed until 2023. The evaluation also did not include activities undertaken by HPFB's Office of Nutrition Policy and Promotion (ONPP), and by program partners, such as CFIA, PMRA, PHAC, P/Ts, and international regulators.

The evaluation used multiple lines of evidence, including a review of literature, program documents, program files, and financial data, as well as key informant interviews and two case studies: Marketing Authorization (MA)/Incorporation by Reference (IBR), and Health Canada's Guidance Document on E. coli 0157 in Raw Beef (see Appendix 1 for further details).

Furthermore, in support of Health Canada's Sex and Gender Action Plan, the evaluation used a Health Equity Lens to understand how the Program considered sex, gender, or socio-economic population groups in its design.

Data was analyzed by triangulating information gathered from the different lines of evidence listed above, with the objective of ensuring the accuracy and reliability of evaluation findings.

3.2 Limitations and Mitigation Strategies

The following table outlines the limitations encountered in the implementation of the data collection methods selected for this evaluation. Also noted are the mitigation strategies put in place to help ensure that evaluation findings could be used with confidence in guiding program planning and decision making.

Table 2: Limitations and Mitigation Strategies
Limitation	Impact	Mitigation Strategy
Retrospective nature of interviews and reliance on interviews for some indicators	As interviews were retrospective in nature, this led to the provision of recent perspectives on past events. This can affect the validity of assessing activities or results relating to improvements in the program area. Some indicators also relied heavily on interview data, and thus, findings related to these indicators are primarily opinion-based and thus subjective.	Triangulation of other lines of evidence where possible to substantiate or provide further information on data received in interviews. Take into consideration input from multiple stakeholders.
Key informant representation	Given the number of different categories of key informants, it was only possible to conduct a few interviews for some of them (especially for case studies).	Interviews were triangulated with other data sources where possible.
The 2016 survey of stakeholders and external partners did not include internal partners (i.e., PHAC, CFIA, AAFC).	It was not possible to use the survey data to understand the overall perspective of both internal and external partners.	The evaluation included interviews of internal partners, including PHAC, CFIA, and AAFC.
Difficulty related to attribution	Attribution is difficult in some areas; for example, in measuring Health Canada's Guidance Document on E. coli's contribution to the decline in E. coli incidence rates.	Difficulties related to attribution were highlighted in the report.
Lack of performance data	Some performance data was limited to a few years, thus not allowing for trend analysis.	Other lines of evidence, such as file and document review and key informant interviews, were used to help provide as clear of a picture as possible as to the impact of activities.

4.0 Findings

The following sections of the report are organized according to the Food Safety Program's logic model (see Appendix 2). This logic model outlines the theory of change that defines how the Program will achieve its desired results. The theory is as follows: through information sharing and engagement, the Program contributes to ensuring that Canadians have the knowledge, skills, and behaviour to make informed decisions pertaining to food safety, nutrition, and healthy eating. In addition, by providing regulatory partners and industry with the tools they need (e.g., HRAs, pre-market assessments, policies and guidelines, regulations) to address food safety and nutrition issues, the Program contributes to partners integrating nutrition, healthy eating, and food safety considerations into their respective policies, programs, and initiatives. In total, these activities help ensure that Canada maintains a world-class food safety system.

4.1 Information Sharing and Engagement

4.1.1 Information Sharing

The Program uses a variety of communication methods to disseminate food safety and nutrition information to Canadians, partners, and stakeholders. Key informants and survey respondents were generally positive regarding the usefulness and relevance of this information; however, opinions tended to be somewhat less positive with regards to its timeliness.

The Food Safety Program disseminates information via a number of different communication methods and vehicles, including the Canada.ca website, electronic mailing lists, partner/stakeholder and committee meetings, and various presentations and publications. For example, the Food Directorate developed 432 outreach publications, attended 68 outreach conferences, and responded to 12,745 inquiries over the evaluation period.^Endnote 8 VDD responded to 672 inquiries in 2017-18.^Endnote 9

Program research scientists also published hundreds of research papers, many of which appeared in academic journals such as the Journal of Food Protection and the Journal of Food Science. Furthermore, the Food Directorate completed 410 scientific presentations during the evaluation period.^Endnote 10 Overall, key informants from different groups felt that Health Canada's scientific publications were of high quality.

Health Canada's Marketing Division assists the Program with education and communication efforts to Canadians. The Division has undertaken various communications activities, such as developing print and web-based content for various audiences, posting to social media, targeting activities to health professionals and other intermediaries, creating national multimedia advertising campaigns, and promoting food safety messages through partners. Over the last number of years, the Program has targeted Canadians deemed most at risk (i.e., seniors, those with compromised immune systems, pregnant women, children under five years of age) and has developed specific products (e.g., pamphlets and posters) to reach these audiences.^Footnote c Targeting seniors is particularly important due to the growth of an aging population in Canada and seniors' vulnerability to infections due to weakened immune systems and the presence of existing diseases.^Endnote 11 The Program also disseminates food safety information for seasonal occasions (e.g., summer barbequing, holiday cooking).

Relevance, Quality, Usefulness, and Timeliness of Program Information

Overall, key informants and respondents to the 2016 Food Safety and Nutrition Survey of stakeholders and external partners^Footnote d,^Endnote 12 were generally positive in their assessments of the quality, usefulness, and relevance of the information provided by the Program. Surveyed stakeholders and external partners, as well as Health Portfolio partner key informants, suggested that information contained in policies and guidelines (e.g., listeria policy, E. coli guidance) was of high quality, useful, and relevant to their work. External expert key informants noted that referencing Health Canada increases their credibility because Health Canada is known to be a reputable science-based organization. One of these experts described Health Canada's food safety research by stating:

"In terms of research coming out of Health Canada [...] I think they do a good job...it tends to be very high quality."

Furthermore, according to the 2016 survey, almost two-thirds (65%) of stakeholders and external partners believed that Health Canada provided them with useful information on food safety and nutrition. For surveyed stakeholders and external partners, the Health Canada website was the most popular source of information (71%), followed by stakeholder/partner meetings (45%), conferences (44%), and newsletters (41%).

With respect to the timeliness of food safety and nutrition information, opinions were somewhat less positive. The 2016 survey of stakeholders and external partners found that opinions were mixed on the timeliness of food safety and nutrition information. Half of survey respondents (56%) said that Health Canada had provided them with timely information on food safety and nutrition, one-third (33%) were neutral, and 11% disagreed that information was timely. Similarly, while a few external key informants thought that the information was timely, some key informants from all groups suggested that information from the Program was not timely, especially if they were not on the Program's electronic mailing lists. For example, one industry key informant who had a positive opinion of the Program's timeliness noted the importance of being on an electronic mailing list in order to receive information in a timely manner:

"So if you are on the circulation list for Health Canada, you get information in a really timely fashion. But if you don't know what you don't know and, for instance, you don't know that there's a circulation list and what you need to be on and what the various lists are within Health Canada, within the Food Directorate, your information access is highly limited."

Other examples provided by partner and stakeholder key informants of situations where information was not provided in a timely manner included research on mycotoxins^Footnote e, food processing-induced contaminants (e.g., acrylamide), and the supplement monograph. Also, external partners expressed that the Program had informed them very late about discussions and concerns related to frozen raw breaded chicken products, and that little information on this issue was made available to them. However, it should be noted that some key informants (Portfolio partners and industry) had the impression that the Program's heavy workload and having to deal with controversial issues could sometimes have had a negative impact on the timeliness of the information they provided.

Education

Both key informant interviews and a 2018 survey of Canadians^Footnote f,^Endnote 13 showed a clear demand for more food safety information, especially among at-risk groups (i.e., pregnant women, seniors, immunocompromised people). This was particularly evident for information on safe food handling practices, with approximately one-third (31-33%) of each at-risk group citing this as information that they needed. This demand for information was also made clear by the fact that the most viewed web pages on Canada.ca for food safety were those containing general food safety tips, such as safe internal cooking temperatures (46% of total visits from November 2017 to November 2018).

Key informants, particularly Portfolio partners and external key informants, highlighted that they would like to have received more surveillance information and research results. They also felt that they would like to have seen more systematic sharing of information. For example, Portfolio partners expressed a desire for better access to Health Canada's surveillance and research, in an effort to reduce overlap among partners and maximize resources.

Some key informants, particularly those from industry, felt that there was too much reliance on web postings for food safety education. They noted that this information does not reach many individuals and is difficult to find. As noted by one external expert:

"I know where to look now, but I've used [Canada.ca] a lot. It's kind of hard to find things on it."

Other key informants, especially program partners and experts, would like to have seen a greater presence of food safety-related educational materials across a variety of platforms (e.g., traditional media, social media, and point-of-sale printouts). Other key informants expressed concern around focusing too much on web-based food safety education.

All of the E. coli 0157 case study key informants felt that education is a key component of an effective food safety system. However, it was also felt that changing Canadians' behaviours related to food safety is difficult and has not necessarily been successful in the past. One program key informant noted,

"So it's an interesting challenge though, because the information that we have shows that consumer behaviours around food safety practices are not changing a lot. In fact, in some cases they are getting worse."

In terms of E. coli in ground beef, another program key informant stated that one of the reasons why Health Canada developed the new Guidance Document on E. coli 0157 in Raw Beef was that consumers have continued to demonstrate unsafe behaviour in relation to this type of meat product.

One case study key informant mentioned a lack of social science research to help understand why consumers exhibit certain behaviours, such as undercooking ground beef, and what can be done to change this type of behaviour. Another case study key informant felt that there has not been enough explicit analysis and discussion aimed at developing an overall approach across the Health Portfolio for meat products, including educating Canadians. The same key informant noted that Health Canada's efforts on education have been much too low, particularly in relation to frozen raw breaded chicken.

With a limited marketing budget of approximately $575K per year, the Program relies heavily on social media to provide information to Canadians.^Endnote 14 This approach appears to be appropriate as, according to the 2018 survey of Canadians, 83% reported using some form of social media in 2018. A growing proportion of users feel that social media is effective at providing them with information on safe food handling (63% in 2018 vs. 43% in 2010). As such, the Internet and websites are now the most popular source of food safety information for Canadians. However, while reliance on traditional media (i.e., TV, radio, newspapers) has declined, it remains a highly relevant vehicle for circulating information to the public during an outbreak of a food-borne illness (63% using traditional media as the main source vs. 17% using the Internet and websites).

Portfolio and industry key informants also noted that the responsibility for educating Canadians about safe food handling does not lie solely with the Program. They mentioned shared responsibilities between other government departments, industry, and provinces and territories. For example, they suggested that food safety and safe food handling practices could be integrated into school curricula and, in general, be the subject of more active and targeted outreach campaigns.

While the Communications Branch evaluates their food safety campaigns, it is less clear if the impact or effectiveness of information dissemination is currently being measured. For instance, reach is tracked by number of website visits, by number of knowledge products ordered by health professionals and other intermediaries, and by targeted mail outs, but very little data has been collected in regards to the impact these have on knowledge uptake or behavioural change. Additionally, there is no evidence that the Program has collected Canadians' perspectives on the timeliness and usefulness of the information, as per their logic model. Recommendations from the previous evaluation of this Program^Endnote 15 included conducting an impact assessment of its public outreach initiatives to determine uptake. This evaluation found that the Program's ability to assess consumers' uptake continues to be a challenge, although this is more specifically related to assessing the impact of food safety information disseminated by the Program.

Finally, in terms of better understanding the challenges around education, the Director General Committee on Food Safety, under the strategic direction of the Deputy Heads, is currently conducting an analysis of consumer behaviour and education efforts across the Health Portfolio^Endnote 16. In cooperation with the Food Directorate, the Public Health Agency of Canada is leading this review, which includes the following activities:

Reviewing existing consumer behavior education initiatives across the Health Portfolio to assess activities and determine gaps, including emerging issues.
Exploring opportunities to address those gaps through collaboration across the Health Portfolio, as well as with PTs, industry, and consumer education stakeholders.
Strengthening the evidence base on consumer behaviour to support the evaluation of current Health Portfolio-led and other education initiatives (i.e., targeting the right populations: parents of young children, seniors, and pregnant women).^Endnote 17

Results from this review will be available in winter 2019.

4.1.2 Program Engagement

Stakeholders and partners were generally pleased with the Program's engagement efforts; however, some expressed a desire for earlier engagement and more frequent collaboration with the Program.

The FSP engages with stakeholders and partners via Canada.ca, emails to stakeholder groups, stakeholder conferences, regular Food Safety Committee meetings at the ADM, DG, ED, and working levels, and face-to-face sessions with certain partners and stakeholders for certain topics (e.g., emerging issues and high-profile subjects, such as allergens). For example, the Food Directorate organized 539 outreach consultations over the course of the evaluation period.^Endnote 18

Many key informants, especially Portfolio partners and external stakeholders, including industry, were appreciative of the Program's engagement efforts, mentioning that there had been more engagement in recent years. They highlighted various examples of what they felt had been productive consultations led by the Program (e.g., listeria, pre-market assessment guidelines, trans fats). Other program partners also mentioned that, from their point of view, the Program had done a good job of engaging with industry on a great number of issues. Industry and expert key informants described the engagement as being of high value, noting that it "fosters collaboration and information exchange". Engagements led by the Veterinary Drugs Directorate (VDD) were particularly noteworthy for the positive feedback they received. Portfolio partner and external key informants mentioned that VDD's engagement efforts related to antimicrobial use surveillance were successful because Health Canada had embraced collaboration with Portfolio partners and other stakeholders. One industry key informant mentioned: "I don't think I'd seen that level of collaboration before, and it seemed to work."

According to the 2016 stakeholder and external partner survey, six in ten (59%) respondents felt that the collaborative approach to maintaining and promoting food safety and nutrition in Canada was effective. Furthermore, a majority (67%) reported that they had effectively engaged with the Program in order to be equipped to meet food safety requirements. Almost half (48%) of respondents said that they had been engaged through outreach activities that promote awareness regarding food safety and also to provide input in the development of the Program's products (risk assessments, guidelines, etc.) (46%).

A number of Portfolio partner key informants noted that they generally had good working relationships with the FSP, yet there were some areas that could be improved. For example, these key informants felt that the quality of the Program's engagements were dependent on a number of factors, such as the bureau they were dealing with, and the design and nature of an engagement. Some industry key informants suggested that online consultations did not always allow them the opportunity to express their point of view on a particular issue. A number of industry key informants also mentioned that they often did not feel comfortable providing confidential business information in online consultations because it was not made clear to them why it was being collected, nor what Health Canada would be doing with this information. In addition, industry association members and Portfolio partner key informants mentioned that the Program was not always transparent as it relates to the results of consultations. The Program published lessons learned from consultations, yet industry association member and Portfolio partner key informants often did not see a clear link between the results of the consultations and final decisions made by the Program. For example, one key informant noted:

"Share the thought process. So it would be helpful if we understand the thought process, then maybe we would say, 'Oh yeah, that makes a little sense', as opposed to, 'Okay, we don't know where it comes from. We don't know the context and where it's going.'"

External key informants from industry, the provinces and territories, and most notably Portfolio partners, all expressed a desire for earlier and more frequent collaboration with the FSP on regulations, policy, and research. Some Portfolio key informants noted that their relationship with the Program was more client-based, which limits their ability to work collaboratively across the Health Portfolio towards the common goal of food safety.

Portfolio partners suggested that more could be done to engage them in the planning stages to discuss issues like research plans, to align objectives and priorities across departments, and to ensure that they understand how they fit within the overall structure of Health Canada's food safety activities. It was also suggested that, since the needs of the various program partners were not always thoroughly discussed prior to the Program establishing their work plans, more interaction and engagement by the Program could help ensure better alignment in key activities (e.g., research) and better leveraging of partner data and activities.

Opinions were mixed concerning the Program's responsiveness to efforts made by stakeholders to reach out and engage the Program in discussions of interest or concern. Portfolio partner and industry key informants noted that the effectiveness of these interactions varied, in large part, on the bureau or directorate with whom they were dealing, or the topic of discussion. For example, VDD was seen as particularly responsive in terms of its consultations related to antimicrobial resistance. Industry and ROEB key informants thought that previous successful consultation exercises could serve as a model for future consultations (e.g., trans fats consultations led by the Bureau of Nutritional Science, Safe Food for Canadians Act consultations led by CFIA). On the other hand, industry key informants mentioned that they were not consulted very much about nutritional labeling. Furthermore, a few industry key informants suggested that the Program was a "black box" and that they did not always receive a response to their outreach efforts. They also noted that it was often a challenge to find out who to contact.

4.2 Canadians' Knowledge and Behaviour Related to Food Safety

4.2.1 Canadians Have the Knowledge and Skills to Make Informed Decisions

Canadians' knowledge of food safety issues is fairly high; however, a few notable knowledge gaps remain. Most noteworthy is the fact that clear majorities of certain at-risk groups do not consider themselves to be more at risk for complications from food poisoning than the average person.

According to the 2018 survey of Canadians, awareness of food safety-related issues was fairly high. This was especially evident when it came to issues like the importance of handwashing (96% of respondents reported a 4 or 5 on a five-point scale, where 5 means that they have heard a great deal about the subject), proper cooking temperatures (76%), proper cooking and cooling instructions (73%), and proper storage of foods (72%). However, awareness has dropped from 2010 levels in a few notable areas including proper cooking and cooling instructions (73% in 2018 vs. 79% in 2010), safe food handling (63% vs. 74%) and listeria (36% vs. 54%). The decline in the level of awareness of listeria is not surprising, as it is to be expected that awareness of specific food-borne pathogens would fluctuate over time, depending on the severity of outbreaks and extent of national or regional media attention on this issue.

Canadians have also demonstrated relatively high levels of knowledge in a number of food safety-related areas:

recognition that food poisoning can be severe (97% agreed with the statement that food poisoning can be mild or severe, and can sometimes send people to the hospital);
the sources of food poisoning;
certain groups are at greater risk of developing complications from food-borne illness (89% indicated this as true); and
that most food-borne illness can be prevented by cooking food thoroughly (84% indicated this as true).

Findings from the 2018 survey on Canadians' perceptions on food safety indicated that Canadians continue to feel that they have sufficient information on food safety and how to protect themselves and their family from food-borne illnesses and food poisoning (72% vs. 76% in 2010).

Canadians are generally knowledgeable of the issues related to food-borne illnesses, yet there continue to be some gaps. Significant percentages of those who self-identified as being in an at-risk group did not consider themselves to be at any greater risk of complications from food poisoning than the average person (seniors at 73%, pregnant women at 59%, immunocompromised people at 43%).^Footnote g Furthermore, a modest level of confusion is also apparent when it comes to Canadians' understanding of proper refrigerator temperatures (25% did not know proper refrigerator temperatures) and whether the look, taste, or smell of a food is any indication that it could cause a food-borne illness (52% of Canadians incorrectly believe it is an indication).

Of particular relevance to recent outbreaks, significantly fewer Canadians (as reported in 2018) were aware that frozen raw breaded chicken products represent a high risk for contamination, as compared to their awareness of risk in regular raw chicken (53% for frozen raw breaded chicken vs. 89% for regular raw chicken). Furthermore, one-third of the general public (35%) and almost one-half of seniors (46%) mistakenly believed frozen raw breaded chicken products only require reheating. Similar findings emerged from the Public Health Agency of Canada's FoodBook study^Endnote 19, which found that, although 86% of surveyed Canadians were aware of general risks associated with chicken, only 23% of respondents were aware of risks associated with raw chicken nuggets. These findings, combined with the fact that an estimated 44,109 Canadians have become ill from handling or consuming frozen raw breaded chicken products since May 2017^Endnote 20, suggest that continued education on safe food handling and appropriate storage and cooking temperatures for these products is important. It should be noted, however, that since the survey was conducted in 2018, the FSP has implemented a number of measures aimed at increasing the knowledge of Canadians on issues related to frozen raw breaded chicken products.

4.2.2 Canadians Make Safe and Healthy Eating Choices

Canadians are generally conducting themselves appropriately when it comes to handling and preparing foods. However, there are some exceptions.

Overall, survey results suggest that Canadians are generally conducting themselves appropriately when it comes to handling and preparing foods and these positive behaviours are in line with 2010 results. The most common safe food handling practices include handwashing before preparing food or after handling raw meat or fish (97% reported 'always' or 'often' doing this), cleaning food preparation surfaces (96%), not refreezing foods which have already been completely thawed (92%), washing fruits and vegetables before consuming them (89%), and following cooking instructions (85%).

At the same time, there is still a significant number of Canadians who continue to engage in unsafe food handling, preparation, and storage activities. For example, many people do not make a regular practice of washing reusable grocery bags (63% reported 'rarely' or 'never'). A small, but notable, number of Canadians continue to eat eggs with runny yolks (30% reported 'always' or 'often') and defrost meat or poultry at room temperature rather than in the fridge (22%). Other practices, such as putting meat, poultry, and fresh produce in the same shopping bag (21%) and keeping leftovers after they have been reheated (20%), are also common among a minority of Canadians.

While the rates of the above behaviours tend to be quite consistent with 2010 levels, there were a few notable changes, both positive and negative. In terms of positive examples, more Canadians are using a food thermometer (49% reported 'always' or 'often' doing this in 2018 vs. 28% in 2010) and fewer rinse poultry before cooking it (62% vs. 75%). Conversely, more Canadians keep remaining leftover food after having reheated it once (20% vs. 13%). In addition, the practice of defrosting meat or poultry at room temperature rather than in the fridge has increased since 2010 among pregnant women (39% in 2018 vs. 21% in 2010) and parents of young children (39% vs. 27%). These rates are particularly high among these at-risk groups.

4.3 Regulatory Partners and Industry Have the Tools They Need to Address Food Safety and Nutrition Issues

4.3.1 Health Risk Assessments

Health Risk Assessments (HRAs) conducted by the Food Directorate are completed according to established service standards. However, Portfolio partners noted certain issues with respect to the timeliness, clarity, and usefulness of HRAs, especially when there is no clear guidance or policy on a particular issue.

Background

An HRA involves determining the likelihood that a specific adverse health effect will occur in an individual or a population following exposure to a substance or microorganism in food (e.g., chemical contaminants, natural toxins, allergens, unapproved food additives, bacteria, viruses or parasites). If it is found that a substance or microorganism in food poses a human health risk, risk management actions are taken to reduce, and if possible, eliminate any risk that is posed to people who consume the food in question.^Endnote 21

The roles and responsibilities related to HRAs of the Government of Canada's Health Portfolio are governed by a Memorandum of Understanding (MOU) between Health Canada, PHAC, and CFIA for common issues related to Human Health.^Endnote 22 This MOU outlines that CFIA is responsible for conducting HRAs for "foods for which HC guidelines, policies, and standards are in place and no policy interpretation is required." An example of this would be the case of E. coli in raw beef, as in 2014 Health Canada published an official guidance document on E. coli in raw beef. Health Canada is responsible for conducting HRAs "in support of HC policy development, upon request from CFIA and other external organizations, and to support pre-market evaluations of industry submissions according to regulatory requirements".^Endnote 23

Health Canada conducts two types of HRAs. Short-term (or "responsive") HRAs are used to inform CFIA in its decision making around risk management. These types of HRAs are often required under very short timelines, as immediate action (i.e., food recalls) may be required. CFIA and Health Canada have agreed to a set of service standards for responsive HRAs, which are based on potential level of risk. As described in the related interdepartmental standard operating procedure, potential Health Risk 1 situations must be completed within eight hours, potential Health Risk 2 situations must be completed within 24 hours, and potential Health Risk 3 situations should be completed within 48 hours (on business days). VDD is required to provide HRAs within 24 hours.

Long-term risk assessments are used by Health Canada and other government departments, such as CFIA, PHAC, and the provinces, to support policy and guidance development. This type of HRA does not have service standards, but rather Health Canada develops a work plan and timelines in consultation with the associated departments or governments.

Health Canada is Meeting its Responsive HRA Service Standards

From 2011 to 2018, the Food Directorate completed a total of 2,093 HRAs, with an average of 321 per year (with 2011 and 2018 prorated, as neither contains a full year of data), and a low of 177 in 2017, and a high of 521 in 2014 (see Chart 1 below).^Endnote 24, VDD completed ten HRAs in 2016-17 and eight in 2017-18.^Footnote h It is not known why there was a peak in HRAs conducted by the Bureau of Chemical Safety in 2014, since there were no changes to the Program that would have significantly affected the distribution of reported numbers. As noted in the chart, the number of recalls reported by CFIA follows a similar pattern to the number of HRAs.

In the fall of 2013, the Office of the Auditor General conducted an assessment of Canada's food recall system, which concluded that Health Canada provided timely HRAs to CFIA. It states:

"We found that Health Canada had established and followed standard operating procedures for its HRAs, which were conducted according to international principles. Health Canada conducts an HRA whenever the CFIA issues a request, including during evenings and weekends. We found that Health Canada met its time standards by assessing urgent concerns within eight hours."^Endnote 25

Chart 1: Number of HRAs Conducted by Health Canada (2011-2018). Text description follows. — **Chart 1: Number of HRAs Conducted by Health Canada (2011-2018)**

* 2011 only includes nine months of data; **2018 only includes six months of data.

Over the evaluation period, both the Food Directorate and VDD provided 100% of HRAs to Portfolio partners within service standards. By meeting these standards, Health Canada has helped contribute to timely responses to food safety incidents by providing information necessary for CFIA and other partners to make critical decisions related to food safety.

Despite meeting these service standards, a number of concerns were raised by Portfolio partners that represent opportunities for program improvement. Below, we highlight the concerns that were noted.

Service Standards and Timeliness

A number of Portfolio key informants raised concerns that the service standards for responsive HRAs may not be adequate, or may need to be reconsidered. There was the perception that, while Health Canada regularly responds within the established service standards, the response time could be shorter in urgent situations. Partners also recognized that in more complex cases, it may be preferable to negotiate a longer service standard. As stated by a couple of Portfolio partner key informants:

"I think this is an incorrect target. The, 'as fast as possible' is missing. We have service standards, but the service standard is so long that anybody can meet it. If you can respond in half an hour, do it. So I think that 'as soon as possible to a maximum of eight hours', that part is missing."

"If you need to reach out to your experts, so that's why I find that the service standard is not really helpful because, at the end of the day, what we want is good expertise, fast when it can be fast, and when it needs more thinking, then it needs to take the time."

Concerns were also raised about how the "time clock" for service standards is reset when questions or additional information is requested, as it was felt that it would be more reasonable that the "clock" be paused for these types of questions, rather than re-started. Although the Food Directorate develops work plans for long-term HRAs in coordination with partners and these HRAs are generally perceived to be high quality, a number of Portfolio partner key informants raised concerns that the time that it takes the Food Directorate to complete longer-term HRAs limits Portfolio partners' ability to use these HRAs for decision making.

Clarity and Usefulness of Health Canada HRAs

A number of Portfolio partner key informants felt that the Food Directorate could improve the approach for conducting HRAs in situations where there is no official Health Canada guidance or policy on a particular issue. It was felt that, in these situations, HRAs would often indicate outcomes of "no increased risk" or "unable to assess", which then contributed to difficulties in making decisions on risk management. In some instances, it was perceived that the approach to assessing the risk was too purist or rigid, and that greater consideration to precautionary risk positions should be considered.

For example, one Portfolio partner key informant noted that HRAs were clear when there was a ground beef contamination issue, as Health Canada has a policy of no E. coli 0157 in ground beef. However, there was no official guidance for salmonella in frozen raw breaded chicken products, and, as a result, it was not clear to all partners why a series of HRAs all came back with the conclusion of "no increased risk". Some key informants felt that these HRAs were too purist in their approach and did not consider a strong enough precautionary risk position.

HRA Roles and Responsibilities

The roles and responsibilities with respect to HRAs are outlined in the trilateral MOU between Health Canada, CFIA, and PHAC (see Background in Section 4.3.1); however, challenges have been noted by all partners in their execution. For example, responsibilities for some activities in practice have slowly moved away from the MOU, which then makes it difficult for Portfolio partners to clearly understand each other's role, especially now that they all report to the same minister. This was referred to by one Portfolio partner key informant:

"That's all done through an MOU [defining roles and responsibilities], as opposed to being something that's in legislation [...] -- scope creep happens a lot, just in the context of day-to-day work, because -- and the fact that everybody is now reporting into the same minister, in some sense it makes that easier. But it also can make it a little bit more complicated, because the federal family is very close then, and it gets very challenging as to who is actually supposed to be doing that."

In addition, according to representatives from CFIA, the challenges with the HRA process noted above have contributed to CFIA considering, in consultation with Health Canada, conducting more short-term HRAs in-house where policies and guidance exist (as outlined in the MOU), and have Health Canada focus on more complex HRAs and the longer-term HRAs.

4.3.2 Pre-Market Assessments

There have been a number of improvements to the pre-market assessment process over the evaluation period and service standards are being met by the Food Directorate and VDD. However, industry still believes that the process is too slow and that this affects companies' willingness to bring new products to Canada.

Since 2013-14, the Food Directorate has been meeting its service standard of completing 80% of regulatory pre-market assessments (for infant formula, food additives, and novel foods) in less than 410 days (see Table #3 below). Similarly, VDD has also been meeting its Total Time to Decision service standards^Footnote i (See Table #4). VDD is now currently focusing on reducing the time to complete the first review component of pre-market assessments, in preparation for the renewal of cost recovery regulations, which will make VDD subject to penalties for not meeting service standards related to this review. Given their past strong performance, both the Food Directorate and VDD have increased their target for completing service standards to 90%, starting in 2018-19.

Table 3: Number of Food Directorate Regulatory Pre-Market Assessments Completed within Service Standards
Fiscal Year	Total Decisions	Completed <410 days	Completed >410 days	Service Standard Met
2013-14^{Table 3 footnote *}	4	4	0	100%
2014-15	81	80	1	99%
2015-16	80	65	15	81%
2016-17	80	65	15	81%
2017-18	84	79	5	92%
2018-19^{Table 3 footnote **}	21	19	2	90%
Total	351	311	38	89%
Table 3 footnote * Only seven months of data available Return to Table 3 footnote * referrer Table 3 footnote Only four months of data available^Endnote 26 Return to Table 3 footnote referrer

Table 4: Number of VDD Regulatory Pre-Market Assessments Completed within Service Standards
Fiscal Year	Total Decisions	Service Standard Met	Service Standard Not Met	Service Standard Met (%)
2013-14	45	41	4	91%
2014-15	47	41	6	87%
2015-16	58	56	2	97%
2016-17	53	48	5	91%
2017-18	53	49	4	92%
2018-19^{Table 4 footnote *}	22	21	1	95%
Total	278	256	22	92%
Table 4 footnote * Only seven months of data available Return to Table 4 footnote * referrer

In addition to regulatory pre-market assessments, the Food Directorate has completed 9,897 non-regulatory pre-market assessments since April 2011, with an average of 1,468 annually (2011 was prorated), a low of 1,049 in 2017, and a high of 1,791 in 2012 (see Chart #2 below).^Endnote 27

Chart 2: Number of Non-Regulatory Pre-market Assessments by Type (2011-2017). Text description follows. — **Chart 2: Number of Non-Regulatory Pre-market Assessments by Type (2011-2017)**

According to the Food Directorate, the lower number of non-regulatory pre-market assessments conducted by Health Canada after 2014 was most likely due to regulatory changes made by the CFIA in 2014. These changes stipulate that only antimicrobial meat and poultry washes require a Letter of No Objection (LONO) (if a LONO is required, a pre-market assessment is also required), rather than all processing aids, incidental additives, and packaging materials.

According to various administrative data and documents, as well as program and industry key informants, there were a number of improvements made to the pre-market assessment process over the evaluation period, including the following:^Footnote j

The backlog of regulatory pre-market assessments was eliminated. Around 2012, a decision was made to increase staff resources in order to eliminate the backlog of regulatory pre-market assessments. As of March 2015, there were 15 submissions that were received prior to September 1, 2013. By June 2016 there was only one remaining, and by February 2017 there were none.^Endnote 28
The Food Directorate began to use a database called RADAR to track pre-market assessments. Staff resources were dedicated to running the database and dealing with petitioners' inquiries related to the status of their submissions. There is also a "Trackers Club" that meets every two weeks to help ensure that submissions are being completed according to service standards.
A detailed Pre-market Submission Guide was developed by the Food Directorate^Endnote 29 and, according to program and industry key informants, it has improved the quality of submissions, as petitioners know exactly what they need to include in their submission, as well as the detailed steps of the process.
Regular meetings with companies were initiated to understand what they were working on, the type and number of future submissions, and to answer any questions.

The pre-market assessment service standards set in regulations are 45 to 90 days. However, through extensive consultations with stakeholders, a 410-day timeline for completion of the submission management process was established. In doing so, the Program provides greater certainty to industry, aligns with the evaluation practices of other food regulatory agencies in similar jurisdictions, reflects a more realistic timeline for pre-market assessments, and considers current resourcing of the Directorate. Although the change in the service standard has been viewed as positive by industry, as it provides more certainty for when assessments are provided, most industry key informants were also strong in their opinions that pre-market assessments take too long and that the length of time affects companies' decisions to bring products to Canada. Industry key informants stated that greater harmonization of regulations (for example, the process for approving a new product) with those of the U.S. would help increase incentives for companies to bring products to the Canadian market. One industry key informant noted:

"It seems to take an inordinate amount of time to get chemicals approved for use for pathogen control and I do know that American companies have sort of abandoned the process altogether because it's so bogged-down in red tape, it's not even worth them going through the effort to do this because it -- it's just much more efficient in the States. So as a result, American processors have accessed all of these different interventions, then Canadian processors continue to complain about not having them."

It should be noted that, in response to industry concerns about the time required to obtain approval for the use of antimicrobial processing aids in meat processing establishments (the example provided above), the Food Directorate has developed a policy for issuing interim Letters of No Objection (iLONOs), based partly on the approved use in another recognized, similar food regulatory agency. Typically, iLONOs are issued within 60 calendar days from the receipt of a completed submission.

Health Canada recognizes the need for greater harmonization with international regulators. According to the HPFB Strategic Plan 2016-2021:

"The globalization of markets is challenging our conventional oversight mechanisms, emphasizing the need for increased international regulatory cooperation and harmonization in order to maximize the efficiency of our processes and ensure Canadians continue to have timely access to safe, effective, and high-quality health products and food."^Endnote 30

4.3.3 Policies, Guidance, and Legislation

Stakeholders and partners view the Food Safety Program's policy and guideline documents as being of high quality and of high importance. There are many examples of such documents that were developed over the evaluation period. However, some Portfolio partner key informants felt that Health Canada needed to take a more proactive leadership role in the development of policy and guidance documents, particularly in response to the transition towards a more outcome-based approach to Canada's food safety system.

Health Canada developed 32 policy and guidance documents over the evaluation period.^Endnote 31 Industry, Portfolio partner, and external partner key informants generally felt that these documents were of high quality and critical to Canada's food safety system. An example is Health Canada's Guidance Document on E. coli 0157 in Raw Beef.^Endnote 32 Key informants felt that this guidance document has helped to bring more certainty to industry on what is expected in terms of addressing E. coli 0157, has aligned Canadian standards with U.S. standards, and has incentivized industry to improve processing hygiene techniques^Footnote k,^Footnote l for reducing E. coli 0157 in raw beef. One case study key informant stated that the guidance document's explicit and systematic focus on controlling processing hygiene, in addition to the economic incentives for improving processing hygiene due to testing, was of particular importance.

Independent and internal research has provided further evidence to support key informant views. One published study found that overall changes to industry processing hygiene over the last decade have been correlated with a decline in E. coli 0157 prevalence rates.^Endnote 33 A related study notes that, "these changes in industry practice have occurred during a period in which reported Shiga toxin-producing Escherichia coli (STEC) illness in Canada, including E. coli 0157, has fallen from an average of 4.5 per/100,000 in the period of 1993 to 1999 to an average of 1.8 per/100,000 in the period of 2009 to 2019".^Endnote 34 Although it is not possible to measure the impact of the guidance document on the decline in prevalence rates of E. coli 0157, by providing industry with incentives to make changes to processing hygiene, it is likely that the guidance document contributed to a reduction in E. coli 0157.

In addition to key informant views, approximately seven out of ten (71%) stakeholders and external partners surveyed either agreed or strongly agreed that they had the tools (policies, guidelines, regulations) they needed to address food safety and nutrition issues. Furthermore, slightly more than eight in ten (82%) also reported that they were aware of regulatory and non-regulatory requirements pertaining to food safety and nutrition.

Although there are a number of positive examples of guidance and policy documents developed by Health Canada, some Portfolio partner and provincial key informants stated that they would like Health Canada to take a stronger role in this area. These key informants stated that, with the changes associated with the Safe Food for Canadians Act (i.e., placing more responsibility on industry for the safety of their products and moving towards a more outcome-based approach to food safety), Health Canada needs to be more proactive in terms of developing guidance and policy. These key informants felt that Health Canada has been slow to make this transition. Although the Safe Food for Canadians Act is not part of the FSP, these key informants felt that this overall change in context, where industry now holds greater responsibility for food safety, makes Health Canada's guidance and policy work that much more important.

One Health Portfolio key informant stated:

"These things [guidance and policy] become, I think, more critically important as we shift to a space where we're saying we're not going to regulate you [as related to the Safe Food for Canadians Act]. We're going to put the responsibility for safety into your hands. Those people with that responsibility will need better guidance."

Examples of guidance areas where some key informants felt that Health Canada could have played a stronger role include salmonella in poultry, E. coli in pork, and Vibrio parahaemolyticus in oysters.

Internal and external key informants generally believed that the overall regulatory framework for food safety has improved. However, they also identified a number of smaller regulatory changes that they felt needed to be addressed, including the following:

Section 4(1)(a) of the Food and Drugs Act should be amended from a zero percent tolerance to the amount of toxic substances allowed in food, to a more risk-based approach that recognizes that the human body can tolerate very tiny trace amounts of toxic materials without harm. There are many naturally-occurring materials, including cadmium, lead, arsenic, and mycotoxins that exist in food and can never be completely eliminated. Companies who sell foods that have those kinds of naturally-occurring contaminants cannot currently comply with the Food and Drugs Act. However, it should be noted that, while literal interpretation of Section 4(1)(a) of the Food and Drugs Act can imply a zero tolerance, any compliance and enforcement actions by the CFIA would be based on risk (identified health risks).
There is a gap in the regulations allowing human milk fortifier products to be sold in Canada. It is felt that human milk fortifier products should adhere to the same pre-market assessment process as infant formula. There is an understanding with paediatric neonatal intensive care units and CFIA that these type of products are essential for newborn and premature infants, and that they should be permitted under a doctor's supervision.
There is overlap between the Food and Drugs Act and the Safe Food for Canadians Act. What should be in the domain of Health Canada's domain and what should be in CFIA's domain is not always clear, thus making it difficult to implement regulatory changes.
Other regulatory changes suggested by interviewees included modernized regulations for infant formula, raw milk cheese, and meal replacements.

4.3.4 Examples of how partners and stakeholders integrate nutrition and food safety considerations into their respective policies, programs, and initiatives

According to survey results and key informants, Health Canada's food safety partners and stakeholders integrate nutrition and food safety considerations into their respective policies, programs, and initiatives.

A large majority of surveyed stakeholders and external partners (80%) expressed that they use information provided by Health Canada to support decisions pertaining to food safety and nutrition. Furthermore, industry and consumer associations, Portfolio partners, and international regulator key informants provided a number of examples of how their organizations use Health Canada information, including the following:

Health Portfolio partners use a wide range of Health Canada documents, such as published material, guidance and policy documents, and HRAs to support decision making.
Industry and consumer association groups re-package Health Canada information to send to their membership by email or through newsletters, reference Health Canada material on social media forums, share Health Canada website links with clients and membership, use Health Canada information to increase their organization's understanding of related federal frameworks and structures, and gain extra credibility by linking their organization's concerns about food safety with science conducted by Health Canada.
An international regulator provided an example of using Health Canada information on genetically modified foods to develop guidance in this area.

4.4 World-Class Food Safety System

Canada was ranked by the Conference Board of Canada in 2014 as second out of 16 OECD countries in terms of food safety performance. Furthermore, rates of reported food-borne illnesses are either stable or decreasing in Canada, other than the rate of Salmonella Enteritidis.

According to the Conference Board of Canada's 2014 report^Endnote 35, Canada excelled in a number of areas, including the following:

rate of use of agricultural chemicals;
incidence of reported illness caused by food-borne pathogens;
national capacity to respond to food safety and other emergencies;
response score for food recalls;
radionuclides standards;
food labelling; and
public trust.

However, the report also mentioned a few areas where Canada could improve, including the following:

greater frequency of reporting on food safety chemical risks through total diet studies;
greater frequency of national food consumption or dietary intake surveys;
developing national supply chain traceability regulations, notably for commodities and products outside animal production; and
incorporating the direction for use and storage on labeling.

The 2018 survey of Canadians, although based on Canadians' perceptions, found that the confidence of the Canadian population in the food safety system continues to be quite high: two-thirds (66% in 2018 vs. 67% in 2010) expressed great to complete confidence in Canada's food safety system, while only eight per cent expressed little to no confidence (8% vs. 6%). That said, confidence decreased significantly between 2010 and 2018 within three out of four groups with specific health-related vulnerabilities: immunocompromised Canadians (54% vs. 70%), pregnant women (61% and 74%), and parents of young children (63% and 70%).

In terms of reported food-borne illnesses, although the overall rate for Salmonella has been steady, illness rates related to Salmonella Enteritidis have increased by a total of 307 percent from 1997 to 2017 (see Chart 3 and 4 below).^Footnote l According to PHAC's 2015-16 Departmental Performance Report, while the overall rate of Salmonella has decreased slightly in recent years, the overall upward trend is due to the ongoing occurrence of Salmonella Enteritidis illness associated with poultry products (e.g., frozen raw breaded chicken products, fresh boneless chicken breast).^Endnote 36

In contrast to Salmonella Enteritidis, the rate of E. coli 0157 has been steadily decreasing (see Chart 5 below). There are a number of factors that may have contributed to this decline. These include food safety interventions at meat processing plants, as discussed previously in relation to the case study on the E. coli 0157 Guidance Document in Section 4.3.3, ongoing campaigns for food safety education, and a number of high-profile outbreaks and recalls in the last few decades that have helped raise public awareness on the potential hazards of E. coli in ground beef002E

Chart 3: Annual national Salmonella rates reported to the National Enteric Surveillance Program (NESP, 1997-2017). Text description follows. — **Chart 3: Annual national Salmonella rates reported to the National Enteric Surveillance Program (NESP, 1997-2017)**

Chart 4: Annual national Salmonella Enteritidis rates reported to the National Enteric Surveillance Program (NESP, 1997-2017). Text description follows. — **Chart 4: Annual national Salmonella Enteritidis rates reported to the National Enteric Surveillance Program (NESP, 1997-2017)**

Chart 5: Annual national E. coli 0157 rates reported to the National Enteric Surveillance Program (NESP, 2006-2017). Text description follows. — **Chart 5: Annual national E. *coli* 0157 rates reported to the National Enteric Surveillance Program (NESP, 2006-2017)**

*Campylobacter are not routinely reported to the provincial or central reference laboratories and are greatly under-represented in NESP.
** Reporting of Listeria monocytogenes to NESP began in July 2010.^Endnote 37

Finally, according to the 2018 survey of Canadians, fewer than one-in-five (16%) members of the general public have experienced an illness over the past year that they thought was related to something they had eaten. However, the proportion of self-reported incidents of food poisoning is higher among those who are expecting or anticipating becoming pregnant (29%), and is slightly above that for the general population among parents of younger children (23%) and those with compromised immune systems (22%).

4.5 Efficiency and Economy

4.5.1 Program Resources

Over the evaluation period, overall spending was similar to budgets for the Food Safety Program. However, there are a number of resource constraints that have made it challenging for the Food Directorate to make progress on activities that are more proactive in nature.

Over the evaluation period, the Food Directorate's actual spending was in line with budgeted spending (99.7%) (see Table 5 below). Within this period, there was only one year where the Food Directorate spent slightly less than planned (93.9% in 2012-13). According to the Food Directorate, the reason for this lower spending, as compared to the budget for 2012-13, was that there was a department-wide freeze on hiring and spending in both Salary and Operating and Maintenance (O&M).

Over the same time period, VDD's total expenditures also nearly met its budget expectations (99.3% of planned budget spent). However, there were large fluctuations from year to year: 75.8% in 2012-13, 111.9% in 2014-15, 113.3% in 2015-16, and 112.6% in 2017-18. The evaluation was not able to determine the rationale for these fluctuations.

Table 5: Planned Spending and Expenditures (2012-13 to 2017-18) ($)
Fiscal Year	Budget				Expenditures				Budget vx. Expenditures
Fiscal Year	Total salary	O&M	Capital	Total	Total salary	O&M	Capital	Total	Variance	% of Planned Budget Spent
Food Directorate
2012-13	27,930,424	7,694,343	453,585	36,078,352	26,874,016	6,615,050	396,471	33,885,536	2,192,816	93.9%
2013-14	35,149,472	5,902,253	483,000	41,534,726	35,805,154	6,448,862	469,968	42,723,984	-1,189,258	102.9%
2014-15	34,318,548	7,526,070	548,000	42,392,618	36,992,649	5,929,622	837,170	43,759,441	-1,366,822	103.2%
2015-16	35,462,912	6,146,534	3,014,841	44,624,287	35,077,136	6,285,927	3,123,427	44,486,490	137,797	99.7%
2016-17	37,484,724	6,303,495	3,501,479	47,289,698	36,216,295	6,318,314	3,495,190	46,029,800	1,259,899	97.3%
2017-18	40,753,890	5,047,964	1,861,425	47,663,279	40,292,706	5,796,548	1,852,637	47,941,891	-278,613	100.6%
TOTAL	211,099,970	38,620,660	9,862,330	259,582,960	211,257,955	37,394,323	10,174,863	258,827,141	755,819	99.7%
VDD
2012-13	6,229,896	606,606	no data	6,836,502	4,637,255	482,255	64,682	5,184,192	1,652,310	75.8%
2013-14	4,007,890	252,520	no data	4,260,410	4,124,813	205,520	no data	4,330,333	-69,924	101.6%
2014-15	3,579,125	226,166	no data	3,805,291	3,986,722	271,075	no data	4,257,797	- 452,506	111.9%
2015-16	3,353,141	199,500	no data	3,552,641	3,856,485	169,303	no data	4,025,788	- 473,147	113.3%
2016-17	3,550,842	206,740	no data	3,757,582	3,562,649	186,093	no data	3,748,742	8,840	99.8%
2017-18	3,590,253	195,214	no data	3,785,467	3,914,330	346,695	no data	4,261,025	-475,558	112.6%
TOTAL	24,311,147	1,686,746	no data	25,997,892	24,082,254	1,660,942	64,682	25,807,878	190,014	99.3%
Overhead (ADM, PPIAD, RMOD, Litigation, other)
2012-13	3,216,390	363,051	39,000	3,618,440	4,615,110	252,975	27,829	4,895,914	-1,277,474	135.3%
2013-14	3,608,492	1,243,400	no data	4,851,892	3,225,629	476,232	no data	3,701,861	1,150,031	76.3%
2014-15	5,089,398	1,137,414	299,000	6,525,812	3,807,892	303,664	no data	4,111,557	2,414,255	63.0%
2015-16	2,095,687	779,147	123,332	2,998,166	3,860,978	631,347	no data	4,492,324	-1,494,158	149.8%
2016-17	3,146,711	857,098	46,028	4,049,837	3,283,333	290,425	52,317	3,626,074	423,763	89.5%
2017-18	3,052,264	113,598	no data	3,165,862	3,687,389	292,271	60,096	4,039,755	-873,893	127.6%
TOTAL	20,208,943	4,493,707	507,360	25,210,010	22,480,330	2,246,914	140,242	24,867,486	342,525	98.6%
Total RAPB/RORB
2012-13	6,366,632	333,750	no data	6,700,382	4,312,175	769,969	770,495	5,852,639	847,743	87.3%
2013-14	5,786,650	502,522	442,260	6,731,432	5,636,005	622,365	835,860	7,094,229	-362,798	105.4%
2014-15	5,245,654	700,001	1,350,246	7,295,901	5,346,537	782,464	1,309,043	7,438,043	-142,143	101.9%
2015-16	4,814,542	741,678	194,073	5,750,293	4,695,404	756,619	172,905	5,624,928	125,365	97.8%
2016-17	5,565,484	975,021	344,454	6,884,959	5,564,255	641,351	11,495	6,217,101	667,858	90.3%
2017-18	5,590,261	745,639	1,659,365	7,995,265	5,225,154	945,793	1,637,145	7,808,092	187,173	97.7%
TOTAL	33,369,222	3,998,611	3,990,398	41,358,231	30,779,531	4,518,560	4,736,942	40,035,033	1,323,198	96.8%
CPAB
2012-13	no data	598,177	no data	598,177	68,339	600,259	no data	668,598	70,420	111.8%
2013-14	no data	597,332	no data	597,332	42,984	497,605	no data	540,589	-56,743	90.5%
2014-15	16,376	598,179	no data	614,555	92,005	461,709	no data	553,714	60,841	90.1%
2015-16	16,376	597,699	no data	614,075	65,721	327,001	no data	392,722	221,353	64.0%
2016-17	no data	581,350	no data	581,350	72,716	484,950	no data	557,666	23,684	95.9%
2017-18	no data	581,351	no data	581,351	137,996	309,653	no data	447,649	133,702	77.0%
TOTAL	32,753	3,554,088	no data	3,586,841	479,762	2,681,176	no data	3,160,938	425,904	88.1%
CSB - Real Property
2016-17	no data	no data	no data	no data	no data	3,555,293	21,002	3,576,295	3,576,295	no data
Total Health Canada
2012-13	43,743,342	9,595,927	492,585	53,831,854	40,506,894	8,720,509	1,259,477	50,486,879	3,344,975	93.8%
2013-14	48,552,504	8,498,027	925,260	57,975,791	48,834,586	8,250,583	1,305,827	58,390,996	-415,205	100.7%
2014-15	48,249,101	10,187,830	2,197,246	60,634,177	50,225,805	7,748,534	2,146,212	60,120,552	513,625	99.2%
2015-16	45,742,658	8,464,558	3,332,246	57,539,462	47,555,723	8,170,196	3,296,332	59,022,252	-1,482,789	102.6%
2016-17	49,747,760	8,923,704	3,891,961	62,563,426	48,699,248	11,476,426	3,580,004	63,755,678	-1,192,252	101.9%
2017-18	52,986,668	6,683,766	3,520,790	63,191,224	53,257,577	7,690,960	3,549,877	64,498,413	-1,307,189	102.1%
TOTAL	289,022,034	52,353,813	14,360,088	355,735,935	289,079,833	52,057,208	15,137,730	356,274,771	-538,835	100.2%
Notes: All years' budgets and actuals do not equal the RPP and Departmental Performance Reports (DPRs), mainly due to the varying percentage of the Employee Benefit Plan (EBP) calculated on salaries. Uncontrollable salaries are also not included in actuals as the funding is statutory. 2012-13: Actuals are less than the DPR mainly due to EBP variances and uncontrollable salaries. 2017-18: The increase in salaries from 2016-17 was mainly due to payments for collective agreements, including retro pay. Therefore, this amount may drop in 2018-19. RAPB/RORB/ROEB: RORB was officially created in 2016-17 and then renamed ROEB in February 2019. The reorganization resulted in the regionally based Drug Analysis Service (DAS), and Food and Health Products (formerly the Inspectorate) laboratories being reallocated under the Laboratories Directorate, a national laboratory program. It also resulted in the coding of overhead to the Food Safety program.

In terms of Health Canada's overall spending on the Food Safety Program, total expenditures compared to budgets over the six-year evaluation period are similar (100.2% of planned budget spent). However, the combination of both the Food Directorate's and VDD's underspending in 2012-13, due to the reasons discussed above, led to an overall spending of 93.8% of the allocated budget in this year.

There have been a number of resource constraints that make it challenging for the Food Directorate to progress on activities that are more proactive in nature, such as regulatory modernization, and the development of policies and guidelines to support the transition to a more outcome-based approach to food safety.

A number of program key informants stated that targeted funding sources are driving the types of activities that Health Canada prioritizes, leaving fewer resources for other core-mandate activities. For example, a significant amount of the Bureau of Chemical Safety's funding is resourced from targeted funding for the Chemicals Management Plan. There is also a significant amount of funding tied to horizontal funding initiatives led by CFIA, particularly funding related to HRAs. The amount of targeted funding over the evaluation period as a percentage of total funding has varied from year-to-year, ranging from one third to almost one half of total funding: 40% in 2013-14, 39% in 2014-15, 46% in 2015-16, 30% in 2016-17, and 34% in 2017-18.

A 2014 HPFB report entitled Food Directorate Operations Review and Capacity Assessment expressed similar conclusions as program key informants, suggesting a more long-term issue in that the large amount of targeted funding from 2009-10 to 2013-14 may have broadened the scope of the Food Directorate due to additional responsibilities associated with the targeted funding. This has made it difficult for the Food Directorate to plan strategically and maintain the core capacities and equipment necessary to fulfill its organizational mandate.^Endnote 38

As mentioned in Section 4.3.3, a few Portfolio partner key informants were strong in their opinions that not enough resources have been allocated towards the transition to an outcome-based approach to food safety. It is thought that, by placing more responsibility on industry for the safety of their products, industry will require more guidance. These key informants felt that the Food Directorate has not been resourced adequately to assume this role.

A number of program key informants also raised the concern that a lack of legal resources for drafting regulations has a significant impact on the Food Directorate's ability to complete necessary regulatory work. It was also noted that this is a departmental-level issue, partially due to Health Canada's extensive regulatory agenda, and that the Department is looking into options for improving the situation, such as providing funds to Justice Canada to hire additional legal drafters.

A few program and Portfolio partner key informants also noted that the heavy workload related to mandate commitments makes it challenging to accomplish other core mandate priorities. For example, one key informant stated that they are not able to update regulations, such as for meal replacements, because most of the resources, both human and financial, are being diverted to mandate commitments. A number of program, partner, and provincial key informants mentioned that staff turnover, mainly due to retirement and competition with other government departments such as CFIA, had a significant impact on capacity within the Food Directorate. Furthermore, the Directorate faces challenges in hiring specialists, such as scientists, statisticians, and toxicologists.

In terms of ROEB's role in supporting the FSP, challenges have been noted related to the maintenance and renewal of capital equipment. According to departmental officials, the Department currently faces challenges with respect to the maintenance and renewal of laboratory capital equipment assets. In the past year, a centralization approach was endorsed at the departmental level for the administration of capital budgets. While this centralization will allow for the prioritization of capital across the Department, it may also increase the difficulty for all laboratories to in obtaining necessary equipment for the FSP, as investments will now be prioritized across four asset classes (fleet vehicles, lab equipment, IM/IT applications, and real property).

As a separate issue, challenges were noted related to ensuring that Food Directorate project requests to ROEB, and thus funding as well, match ROEB's expectations of FTE requirements. Documentation identified that ROEB had 30.8 full-time equivalents (FTEs) available to deliver projects in 2016, including management and administration, but that the Food Directorate had not been able to allocate enough work to use 30.8 FTEs in the food laboratories over the previous four years.^Endnote 39 ROEB and the Food Directorate agreed that the Food Directorate would commit to provide ROEB ongoing resources (16 FTEs worth of work and $220K O&M) for projects to be identified and confirmed annually. This agreement aligned with the Management Response and Action Plan (MRAP) for recommendation #2 for the Audit of Regional Laboratory Activities, which states, "establishing the recurrent minimal level of laboratory services and resources required to support the Food Program."^Endnote 40,^Endnote 41

4.5.2 Program Efficiency

There are a number of examples of how the Food Safety Program has become more efficient over the evaluation period.

According to various program data, as well as a number of internal and external key informants, including those from the case study, "Marketing Authorizations (MA)" and "Incorporation by Reference (IBR)" help the Government of Canada to adjust food safety regulations in a more timely fashion, and in turn, keep pace with rapidly evolving technology, science, and business practices.^Footnote m Whereas some types of regulatory amendments pertaining to food safety once took years to formulate under the traditional Governor in Council process, they can now take less than six months under a ministerial process, once Health Canada scientists have made a safety decision. For example, an analysis of enzyme food additive submissions prepared by the Chemical Health Hazard Assessment Division on October 11, 2018 found that the 24 previous submissions took an average of 103 days to complete.^Endnote 42

Since the inception of MAs and IBRs, the FSP has implemented 17 marketing authorizations and has used IBR related to the following:

30 Notices of Proposals (NOPs) published in relation to proposed modifications to the food additives lists;
5 NOPs have been published for proposed modifications to the contaminants and adulterants list;
124 Notices of Modification (NOMs) have been published for the food additive lists (28 of these were preceded by an NOP, two of those NOPs are still in the notification and comment phase);
3 NOMs have been published for the contaminants and adulterants list.^Endnote 43

As previously mentioned, the Food Directorate developed a Pre-market Submission Guide, which, according to program key informants, has improved the quality of submissions. Furthermore, a database was developed to track Food Directorate pre-market assessments and a Trackers Club meets every two weeks to help ensure that submissions are being completed according to service standards. These measures, in addition to increased staff for pre-market assessments, have resulted in the elimination of the pre-market backlog.

Health Canada is leading a review of food safety governance through the Deputy Head Committee on Food Safety. Activities in this review include:

Developing an inventory of existing food safety and related federal, provincial, and territorial sub-committees;
Examining scope, roles and responsibilities, and effectiveness of the existing structure;
Identifying opportunities for alignment, ways to reduce gaps or overlaps, and oversight needs for emerging issues.^Endnote 44

Although an analysis of cost recovery related to Food Directorate pre-market assessments is not yet complete, program key informants felt overall that cost recovery may not be viable due to the relatively low number of pre-market assessments completed by the Program.

To facilitate access to safe, effective, and quality veterinary drugs in Canada, the Veterinary Drugs Directorate (VDD) works with the U.S. Food and Drug Administration's Center for Veterinary Medicine to conduct simultaneous reviews of veterinary drug applications. Companies that wish to market a veterinary drug in Canada and the U.S. can submit applications to have their product reviewed by both agencies at the same time. This process allows veterinary drugs to be available at the same time on either side of the border. VDD is also exploring ways to review veterinary drug submissions jointly with their international partners. For example, VDD has collaborated with the Australian Pesticides and Veterinary Medicines Authority, and the New Zealand Ministry of Primary Industries, to jointly review and enable simultaneous access to a new animal drug in three major markets, leading to improved animal health and food safety.

In preparation for moving towards a revised approach to cost recovery, VDD has added an additional Regulatory Project Officer, who project manages drug reviews, in order to better manage submission review times.

Finally, with the Veterinary Health Products program, VDD has introduced a regulatory pathway for companies to import and sell low-risk veterinary health products, such as vitamins and minerals that are used as additional health management tools in food-producing animals to help reduce the need to use conventional drugs, including antimicrobials.

4.5.3 Performance Measurement

The Program is collecting and using performance information; however, it primarily focuses on collecting data related to service standards and, in some areas, does not capture the perspectives of Canadians, specifically as related to the usefulness and timeliness of information and the effectiveness of engagement efforts.

In terms of performance measurement, the Program seems to focus on collecting data related to the various service standards (e.g., HRAs, pre-market assessments). This information is rolled up into dashboards that are shared with senior management and, if service standards are not being met, the Program must explain why this is the case. Beyond these areas, it was less clear, especially to program key informants, what performance information was being collected and how it might be used.

In addition, performance data for certain key outcomes did not capture the perspectives of Canadians (e.g., no information on Canadians' perceptions of the timeliness and usefulness of information or the effectiveness of engagement efforts), nor did it always use information that was already collected (e.g., could roll up feedback collected from various outreach and engagement efforts). Furthermore, as previously mentioned, greater efforts could have been made in analyzing the effectiveness of public education campaigns.

5.0 Conclusions

5.1 Achievement of Expected Outcomes

Overall, the Food Safety Program has achieved or is making significant progress towards achieving its key program objectives. In addition, according to a 2014 Conference Board of Canada report, Canada's food safety system is one of the best in the world. However, this evaluation found some areas where activities could be strengthened.

The FSP reached out to Canadians to try and make them more knowledgeable of various food safety issues and encourage them to practice to safe food handling, preparation and storage. In general, Canadians' knowledge of food safety issues is fairly high and Canadians exhibit a number effective food safety behaviours; however, a few notable knowledge gaps and unsafe behaviours still exist. Most noteworthy is the fact that a clear majority of certain at-risk groups do not consider themselves more at risk for complications from food poisoning than the average person.

5.2 Demonstration of Economy and Efficiency

Finally, the FSP's performance measurement was focused on key service standards, and performance data for certain key outcomes did not capture the perspectives of Canadians even though they were explicitly identified in their logic model.

6.0 Recommendations

Recommendation 1

Recommendation 2

Portfolio partners expressed a desire for earlier and more regular collaboration with the FSP on regulations, policy, and research. Portfolio partners suggested that more could be done to engage them at the planning stages to discuss issues like research plans, to align objectives and priorities across departments, and to ensure that they understand how they fit within the overall structure of Health Canada's food safety activities. It was also suggested that since the needs of the various program partners were not always thoroughly discussed prior to the Program establishing its work plans, more interaction and engagement by the Program could help ensure better alignment in key activities (e.g., research) and better leveraging of partner data and activities.

Recommendation 3

Consider increasing outreach and education efforts aimed at Canadians to help address various food safety knowledge and behaviour gaps.

Recommendation 4

Increase efforts to obtain Canadians' perspectives on the timeliness and usefulness of Health Canada information on food safety and the effectiveness of its engagement efforts.

The Program collects and uses performance information; however, it focuses primarily on collecting data related to service standards and, in some areas, does not capture the perspectives of Canadians. For instance, reach is tracked by the number of knowledge products ordered by health professionals and other intermediaries, as well as targeted mail outs, but very little data is collected on the impact these have on knowledge uptake or behavioural change among Canadians. Additionally, there is no evidence that the Program has collected Canadians' perspectives on the timeliness and usefulness of the information, as per the Program's logic model.

Appendix 1: Evaluation Description

The evaluation covered all activities undertaken by the Food Directorate, the Veterinary Drugs Directorate (VDD) and the Resource Management and Operations Directorate (RMOD), as related to Food Safety, from April 2012 to March 2018. However, the evaluation did not include activities undertaken by HPFB's Office of Nutrition Policy and Promotion (ONPP) and by program partners such as CFIA, the Pest Management Regulatory Agency (PMRA), the Public Health Agency of Canada (PHAC), provincial and territorial governments, and international regulators.

The specific evaluation questions were based on achievement of expected outcomes and the demonstration of efficiency and economy (see Table 6 below).

Table 6: Core Evaluation Issues and Questions
Core Issues	Evaluation Questions
Performance (effectiveness, efficiency, and economy)
Achievement of Expected Outcomes (Effectiveness)	What contextual factors (positive or negative) may have influenced the achievement of outcomes? How has the FSP responded to those factors? To what extent has the FSP produced expected outcomes? Short Term To what extent do Canadians, partners, and stakeholders have access to timely, useful, and relevant information? To what extent are Canadians, partners, and stakeholders engaged effectively by the Program? From a Health Canada perspective, to what extent are food safety incidents, including food-borne illness outbreaks, responded to in a timely manner? To what extent are partners and stakeholders informed of Program policies, guidelines, and regulations? Medium Term To what extent do partners and stakeholders integrate nutrition, healthy eating, and food safety considerations into their respective policies, programs, and initiatives? To what extent do regulatory partners and industry have the tools they need to address food safety and nutrition issues? To what extent do Canadians have the knowledge and skills to make informed decisions pertaining to food safety, nutrition, and healthy eating? Long Term To what extent does Canada maintain a world-class food safety system? To what extent do Canadians make safe and healthy eating choices? How do roles and responsibilities impact program results? How have challenges identified in the previous evaluation been addressed?
Demonstration of Efficiency and Economy	Has the Program undertaken its activities in the most economical and efficient manner? Is there appropriate performance measurement in place and how is this used to inform decision making? What changes in the Program's design might produce better outcomes or lead to better impacts?

The Evaluation approach

Evaluators collected and analyzed data from multiple sources. Data for the evaluation was collected using the following methods:

Literature review – A short literature review was conducted to obtain information on the main contextual factors that affect food safety in Canada, including the FSP.
Program document and file review – Approximately 500 documents were reviewed to obtain information on all aspects of the FSP.
Financial data review – A review of financial data from 2012-13 to 2017-18 was conducted, including budgeted and actual expenditures.
Key informant interviews^Footnote n – Interviews were conducted with 45 (64) stakeholders:
- Program (i.e., FD, VDD, CPAB, RMOD, PPIAD, and ROEB): n=15 (23)
- Portfolio Partners (i.e., CFIA, PMRA, and PHAC): n=11 (17)
- External Partners (i.e., AAFC, provincial and territorial governments, and international regulators): n=7 (11)
- External stakeholders (i.e., Industry Associations and Experts): n=12 (13)
Case studies – Two case studies were completed: Marketing Authorization and Incorporation by Reference, and Health Canada's Guidance Document on E. coli 0157 in Raw Beef. In total, nine interviews were conducted as part of the case studies: Program (n=5); industry (n=3); external partner (n=1).

In support of Health Canada's Sex and Gender Action Plan, the evaluation also used a Health Equity Lens aimed at understanding how the Program considered sex, gender, or socio-economic population groups in program design.

Data was analyzed by triangulating information gathered from the different methods listed above. The use of multiple lines of evidence and triangulation were intended to increase the reliability and credibility of the evaluation findings and conclusions.

Appendix 2: Logic Model

Figure 6. Food and Nutrition Program Logic Model, 2017-06-15. Text description follows. — **Figure 6. Food and Nutrition Program Logic Model, 2017-06-15**

Footnotes

Footnote a

In February 2019 the Regulatory Operations and Regions Branch (RORB) was renamed the Regulatory Operations and Enforcement Branch (ROEB).

Return to footnote a referrer

Footnote b

In February 2019 the Regulatory Operations and Regions Branch (RORB) was renamed the Regulatory Operations and Enforcement Branch (ROEB).

Return to footnote b referrer

Footnote c

For example, the Program has developed and distributed Safe Food Handling Guides and printed advertisements for pregnant women, children aged five and under, immunocompromised individuals, and adults aged 60 and over.

Return to footnote c referrer

Footnote d

The Food Safety and Nutrition Survey was administered online to stakeholders and external partners, such as provincial governments, to determine if the Program is achieving its intended results. Sampling consisted of subscribers to the Consultation and Stakeholder Information Management System (CSIMS). Of the 852 successfully administered surveys, 211 were completed representing a response rate of 25%.

Return to footnote d referrer

Footnote e

A mycotoxin is a toxic secondary metabolite produced by organisms of the fungus kingdom and is capable of causing disease and death in both humans and other animals.

Return to footnote e referrer

Footnote f

2018 Public opinion research was conducted by the Strategic Counsel, on behalf of Health Canada, to assess public awareness, attitudes, knowledge, and behaviours related to food safety and food-borne illnesses. The research study was conducted between December 2017 and January 2018, with a total of 2,814 Canadians representing an 8.5% response rate. Sampling included individuals from each identified vulnerable group: seniors aged 60+ (406), parents of children aged five and younger (302), pregnant women or those who anticipate they will become pregnant within the next year (301), and those with compromised immune systems (300). Results were tracked against the original 2010 baseline survey.

Return to footnote f referrer

Footnote g

It should be noted that these percentages are slightly down from 2010.

Return to footnote g referrer

Footnote h

Data for other years is not currently available.

Return to footnote h referrer

Footnote i

Service Standards include New Drug Submission – 748 days, Abbreviated New Drug Submission – 748 days, Supplemental New Drug Submission – 658 days, Supplemental Abbreviated New Drug Submission – 658 days.

Return to footnote i referrer

Footnote j

A number of these measures were included in the Management Response and Action Plan from Recommendation #1 of the previous evaluation of this Program: "The Food Directorate should examine options to enhance the efficiency and transparency of its food pre-market submission activities".

Return to footnote j referrer

Footnote k

Meat processing hygiene refers to the hygienic measures taken during the various processing steps of the manufacture of meat products.

Return to footnote k referrer

Footnote l

Even though the rate of Salmonella has been increasing, as per the Conference Board of Canada's 2014 report, Canada is still a strong performer compared to other countries in terms of the incidence of reported illness caused by food-borne pathogens. According to the report, along with Canada, countries with the strongest performance include Austria, France, Ireland, Japan, the U.K., and the U.S.

Return to first footnote l referrer

Footnote m

According to HPFB Strategic Plan 2016-2021, novel health products and food, new trends in medicine, fast-paced scientific discoveries, and innovative business models are the new normal. This challenges conventional definitions of health products and food, and affects the relevance and effectiveness of related legislative and regulatory frameworks.

Return to footnote m referrer

Footnote n

Some key informant interviews included multiple participants, therefore the first number represents the interview count and the second number, in parentheses, represents the participant count.

Return to footnote n referrer

Endnotes

Endnote 1

Health Canada, (May 31, 2016), Performance Measurement Strategy for the Food Safety and Nutrition Program.

Return to endnote 1 referrer

Endnote 2

Health Canada, (October 24, 2014), Food Directorate Operations Review and Capacity Assessment: Report 2 Environmental Scan Report Final, Version 3.0.

Return to endnote 2 referrer

Endnote 3

Health Canada, (May 31, 2016), Performance Measurement Strategy for the Food Safety and Nutrition Program.

Return to endnote 3 referrer

Endnote 4

Health Canada, (May 31, 2016), Performance Measurement Strategy for the Food Safety and Nutrition Program.

Return to endnote 4 referrer

Endnote 5

Health Canada, (October 24, 2014), Food Directorate Operations Review and Capacity Assessment: Report 2 Environmental Scan Report Final, Version 3.0.

Return to endnote 5 referrer

Endnote 6

Charlebois, Sylvain and Amit Summan, (2014), "Determinants of Future Microbial Food Safety in Canada for Risk Communication", Journal of Food Safety, 35, p.303-317.

Return to endnote 6 referrer

Endnote 7

Health Canada, (May 31, 2016), Performance Measurement Strategy for the Food Safety and Nutrition Program.

Return to endnote 7 referrer

Endnote 8

Program landscape data provided by the Food Directorate, 2018.

Return to endnote 8 referrer

Endnote 9

Program landscape data provided by the Food Directorate, 2018; inquiries statistics provided by VDD, 2018.

Return to endnote 9 referrer

Endnote 10

Program landscape data provided by the Food Directorate, 2018.

Return to endnote 10 referrer

Endnote 11

Charlebois, Sylvain and Amit Summan, (2014), "Determinants of Future Microbial Food Safety in Canada for Risk Communication", Journal of Food Safety, 35, p.303-317.

Return to endnote 11 referrer

Endnote 12

Health Canada, (2016), Food Safety and Nutrition Survey of Partners and Stakeholders.

Return to endnote 12 referrer

Endnote 13

Final Report: Survey of Canadian's Knowledge and Behaviours Related to Food Safety, (May 2018), Health Canada and The Strategic Council

Return to endnote 13 referrer

Endnote 14

Safe Food Handling for At-Risk Populations Past Marketing Campaigns 2009-2018, (no date), Health Canada

Return to endnote 14 referrer

Endnote 15

Health Canada, (2014), Evaluation of the Food Safety and Nutrition Quality Program 1999-2000 to 2011-2012

Return to endnote 15 referrer

Endnote 16

Health Canada and the Public Health Agency of Canada, (September 2017), Deputy Heads Committee on Food Safety Future Areas of Focus

Return to endnote 16 referrer

Endnote 17

Health Canada, (January, 2018), Committees on Food Safety – Areas of Focus: Consumer Behavior and Education Work Plan Update

Return to endnote 17 referrer

Endnote 18

Program landscape data provided by the Food Directorate, 2018.

Return to endnote 18 referrer

Endnote 19

Public Health Agency of Canada (2015). Foodbook Report. Retrieved from: https://www.canada.ca/en/public-health/services/publications/food-nutrition/foodbook-report.html

Return to endnote 19 referrer

Endnote 20

Health Canada (August 2018), "Food Safety 2018-19 Social Marketing Strategy", (internal deck).

Return to endnote 20 referrer

Endnote 21

Health Canada, Overview of Health Risk Assessments, retrieved from: https://www.canada.ca/en/health-canada/services/food-nutrition/food-safety/chemical-contaminants/food-related-health-risk-assessment.html, accessed: 2018.

Return to endnote 21 referrer

Endnote 22

Memorandum of Understanding Between: Health Canada and the Public Health Agency of Canada and the Canadian Food Inspection Agency for Common Issues Related to Human Health, April 7, 2008.

Return to endnote 22 referrer

Endnote 23

Memorandum of Understanding Between: Health Canada and the Public Health Agency of Canada and the Canadian Food Inspection Agency for Common Issues Related to Human Health, April 7, 2008.

Return to endnote 23 referrer

Endnote 24

Food Directorate, VDD HRA program data; recalls: http://www.inspection.gc.ca/about-the-cfia/newsroom/food-recall-warnings/complete-listing/eng/1351519587174/1351519588221

Return to endnote 24 referrer

Endnote 25

Office of the Auditor General of Canada (Fall 2013), Chapter 4: Canada's Food Recall System, p. 13. Completed 24 July 2013.

Return to endnote 25 referrer

Endnote 26

Food Directorate regulatory pre-market assessment program data

Return to endnote 26 referrer

Endnote 27

Food Directorate non-regulatory pre-market assessment program data

Return to endnote 27 referrer

Endnote 28

Health Canada, "Briefing Note to the Associate Assistant Deputy Minister: Update on Food Directorate Performance on mandatory pre-market submissions (novel food, food additives and infant formula)".

Return to endnote 28 referrer

Endnote 29

Health Canada, (2018), "The Food Directorate's Pre-Market Submission Management Process for Food Additives, Infant Formulas and Novel Foods", retrieved from https://www.canada.ca/en/health-canada/services/food-nutrition/public-involvement-partnerships/food-directorate-market-submission-management-process-food-additives-infant-formulas-novel-foods.html

Return to endnote 29 referrer

Endnote 30

Health Canada, HPFB Strategic Plan 2016-2021.

Return to endnote 30 referrer

Endnote 31

Program landscape data provided by the Food Directorate, 2018.

Return to endnote 31 referrer

Endnote 32

Health Canada (2018), "Health Canada's Guidance Document on E. coli O157:H7 and E. coli O157:NM in Raw Beef", retrieved from https://www.canada.ca/en/health-canada/services/food-nutrition/legislation-guidelines/guidance-documents/guidance-document-coli-0157-coli-0157-beef-2014.html

Return to endnote 32 referrer

Endnote 33

Pollari, Frank et al., (2017), "Evidence for the benefits of food chain interventions on E. coli 0157:H7/NM prevalence in retail ground beef and human disease incidence: A success story". Canadian Journal of Public Health, Vol. 108, No.1.

Return to endnote 33 referrer

Endnote 34

Gill, Alex, (2018), "The decline in Chiga toxin producing Escherichia coli illness in Canada: evidence for a role in changes in beef industry practices". Health Canada, Bureau of Microbial Hazards.

Return to endnote 34 referrer

Endnote 35

Le Vallée, Jean-Charles and Sylvain Charlebois, (2014), "2014 World Ranking: Food Safety Performance", The Conference Board of Canada, 68 pages.

Return to endnote 35 referrer

Endnote 36

PHAC, 2015/16 Departmental Performance Report.

Return to endnote 36 referrer

Endnote 37

National Enteric Surveillance Program (NESP), annual summary reports, 2006-2012.

Return to endnote 37 referrer

Endnote 38

Health Canada, (2014), Food Directorate Operations Review and Capacity Assessment.

Return to endnote 38 referrer

Endnote 39

Health Canada, (July 13, 2017) 2.2.1 Food Safety, RORB.

Return to endnote 39 referrer

Endnote 40

Health Canada, (2016) Current Status and Future Options for FD Needs and RORB Regional Labs.

Return to endnote 40 referrer

Endnote 41

Health Canada (July 2016), Final Report: Audit of Regional Laboratory Activities.

Return to endnote 41 referrer

Endnote 42

Program data analyzing enzyme food additive submissions prepared by the Chemical Health Hazard Assessment Division on October 11, 2018.

Return to endnote 42 referrer

Endnote 43

Health Canada, Overview of Incorporation by Reference and Marketing Authorization, retrieved from: https://www.canada.ca/en/health-canada/services/food-nutrition/legislation-guidelines/acts-regulations/incorporation-reference.html and https://www.canada.ca/en/health-canada/services/food-nutrition/legislation-guidelines/acts-regulations/marketing-authorizations-acts-regulations-food-nutrition.html, accessed: 2018.

Return to endnote 43 referrer

Endnote 44

Health Canada, (September 28, 2017), "Deputy Heads Committee on Food Safety Future Areas of Focus".

Return to endnote 44 referrer

Page details

Date modified:: 2019-09-19

Evaluation of The Food Safety Program 2012-13 to 2017-18

Table of Contents

List of Tables and Charts

List of Acronyms

Executive Summary

Program Description

Summary of Results

Achievement of Expected Outcomes

Demonstration of Economy and Efficiency

Recommendations

Recommendation 1

Recommendation 2

Recommendation 3

Recommendation 4

Management Response and Action Plan

1.0 Evaluation Purpose

2.0 Program Description

2.1 Program Profile

2.2 Program Context

2.3 Roles and Responsibilities

2.4 Program Resources

3.0 Evaluation Description

3.1 Evaluation Scope, Approach and Design

3.2 Limitations and Mitigation Strategies

4.0 Findings

4.1 Information Sharing and Engagement

4.1.1 Information Sharing

Relevance, Quality, Usefulness, and Timeliness of Program Information

Education

4.1.2 Program Engagement

4.2 Canadians' Knowledge and Behaviour Related to Food Safety

4.2.1 Canadians Have the Knowledge and Skills to Make Informed Decisions

4.2.2 Canadians Make Safe and Healthy Eating Choices

4.3 Regulatory Partners and Industry Have the Tools They Need to Address Food Safety and Nutrition Issues

4.3.1 Health Risk Assessments

Background

Health Canada is Meeting its Responsive HRA Service Standards

Service Standards and Timeliness

Clarity and Usefulness of Health Canada HRAs

HRA Roles and Responsibilities

4.3.2 Pre-Market Assessments

4.3.3 Policies, Guidance, and Legislation

4.3.4 Examples of how partners and stakeholders integrate nutrition and food safety considerations into their respective policies, programs, and initiatives

4.4 World-Class Food Safety System

4.5 Efficiency and Economy

4.5.1 Program Resources

4.5.2 Program Efficiency

4.5.3 Performance Measurement

5.0 Conclusions

5.1 Achievement of Expected Outcomes

5.2 Demonstration of Economy and Efficiency

6.0 Recommendations

Recommendation 1

Recommendation 2

Recommendation 3

Recommendation 4

Appendix 1: Evaluation Description

The Evaluation approach

Appendix 2: Logic Model

Footnotes

Endnotes

Page details