Summative Evaluation of the Bovine Spongiform Encephalopathy (BSE) I and II Initiatives

Prepared by Evaluation Directorate, Health Canada and the Public Health Agency of Canada

June 2013

List of Acronyms

AAFC
Agriculture and Agri-food Canada
ADM
Assistant Deputy Minister
BGTD
Biologics and Genetic Therapies Directorate
BSE
Bovine Spongiform Encephalopathy
BVCRT
Beef Value Chain Roundtable
CDC
Centers for Disease Control and Prevention
CEP
Certificate of Suitability
CFIA
Canadian Food Inspection Agency
CJD
Creutzfeldt-Jakob Disease
CJDSS
Creutzfeldt-Jakob Disease Surveillance System
CNS
Central Nervous System
CPSD
Consumer Products Safety Directorate
CWD
Chronic Wasting Disease
DFAIT
Department of Foreign Affairs and International Trade
DFO
Department of Fisheries and Oceans
DG
Director General
DIAND
Department of Indian Affairs and Northern Development
DM
Deputy Minister
DPD
Drug Product Database
DPMED
Departmental Performance Measurement and Evaluation Directorate
EC
Environment Canada
EDQM
European Directorate for the Quality of Medicine and HealthCare
EFSA
European Food Safety Authority
EMA
European Medicines Agency
EU
European Union
FBS
fetal bovine serum
FD
Food Directorate
FNIHB
First Nations and Inuit Health Branch
GHTF
Global Harmonization Task Force
GMP
Good Manufacturing Practices
HECSB
Healthy Environments and Consumer Safety Branch
HPFB
Health Products and Food Branch
HPFBI
Health Products and Food Branch Inspectorate
ICH
International Conference on Harmonisation of Technical Requirements for Registration of Pharmaceuticals for Human Use
IRCH
International Regulatory Cooperation for Herbal Medicines
LNHPD
Licensed Natural Health Product Database
MDALL
Medical Devices Active License Listing
MDB
Medical Devices Bureau
MHPD
Marketed Health Products Directorate
MOU
Memorandum of Understanding
NHPD
Natural Health Products Directorate
NPN
Natural Product Number
OIE
Organisation for Animal Health
PAA
Program Activity Architecture
PCO
Privy Council Office
PHAC
Public Health Agency of Canada
PM
Performance Measurement
PPHB
Population and Public Health Branch
PPIAD
Policy, Planning, and International Affairs Directorate
RMOD
Resource Management and Operations Directorate
SIPD
Submission Information and Policy Division
SRM
Specified Risk Material
TPD
Therapeutic Products Directorate
TSE
Transmissible Spongiform Encephalopathy
UK
United Kingdom
US
United States
VBS
Veterinary Biologics Section
vCJD
variant Creutzfeldt-Jakob Disease
VDD
Veterinary Drugs Directorate
VICH
International Cooperation on Harmonisation of Technical Requirements for Registration of Veterinary Products
VLA
Veterinary Laboratories Agency
WHO
World Health Organization

Table of Contents

Executive Summary

In 2003, in response to the discovery of a domestic case of Bovine Spongiform Encephalopathy (BSE), the Government of Canada announced the federal BSE strategy, involving Health Canada, the Canadian Food Inspection Agency (CFIA), Agriculture and Agri-food Canada (AAFC), and the Public Health Agency of Canada (PHAC). The strategy is intended to safeguard the health of Canadians by minimizing the risk of exposure to the BSE agent. Within the federal strategy, Health Canada executed its responsibilities through the BSE I and BSE II Initiatives.

  • BSE I (Augmenting Health Canada's Response to BSE, 2003-2008) consisted of product assessment, tracking and tracing, and compliance and enforcement activities.
  • BSE II (Further Measures on BSE in Areas of Risk Assessment and Targeted Research, 2004-2009) consisted of risk assessments and targeted scientific research.

An evaluation of the BSE I and BSE II Initiatives is part of Health Canada's Five-Year Evaluation Plan. The evaluation assessed the relevance and performance (effectiveness, efficiency, and economy) of the Initiatives, in accordance with Treasury Board of Canada's Policy on Evaluation. The evaluation was conducted by PRA Inc., an independent evaluation consulting firm, on behalf of Health Canada.

The evaluation used multiple lines of evidence, including literature review; document review; administrative data review; key informant interviews completed in two rounds; a survey of industry stakeholders; and one case study.

Findings

Relevance

The evaluation confirmed an ongoing need for intervention to manage the risks to human health associated with BSE/TSEs. Based on a review of the scientific literature, there clearly remain many unknowns and uncertainties in the field of BSE/TSE science, with important future implications for public health. These include ongoing uncertainty with respect to human-to-human transmission; atypical BSE; and TSEs other than BSE, such as chronic wasting disease (CWD) -- a uniquely North American phenomenon that may require Canada to be the first jurisdiction to develop an appropriate policy and regulatory response. The potential health impacts stemming from the greater risk of exposure to CWD by some Canadians, including First Nations and Inuit populations, and sport and subsistence hunters, further implicate Health Canada in ongoing efforts to understand and manage the associated risks.

Recommendation 1: Health Canada should continue to play a role in BSE/TSE risk assessment and research to inform policy and regulatory development. Particular attention should be paid to the evolving science on CWD, given its potential health implications for all Canadians.

The BSE Initiatives are clearly aligned with Health Canada's roles and responsibilities as described in federal statutes and regulations, in particular the Food and Drugs Act and Regulations and the Department of Health Act, and with government priorities to strengthen food safety expressed in the 2009 Budget and 2010 Speech from the Throne. An ongoing role for Health Canada in managing BSE/TSE-related risks seems warranted on this basis.

Performance

Governance

The evaluation found that governance of the BSE I and II Initiatives had several weaknesses that likely affected program implementation, including limited collaboration and coordination among internal and external partners and the lack of a coherent profile for the TSE Secretariat, created to coordinate Health Canada's BSE/TSE activities. While the Secretariat performed an important role in monitoring and disseminating scientific information in the early years of the government's response to BSE/TSE, its current role is not clear, particularly given the maturation of the field and Health Canada's more sophisticated understanding of BSE/TSE science and policy. Finally, performance measurement and reporting did not occur, possibly due to a lack of clarity regarding responsibilities for these activities. Since the federal government is continuing to fund Health Canada for BSE-related activities through BSE III, some clarification of these roles and responsibilities seems warranted.

Recommendation 2:  Health Canada should consider whether there is still a necessary role for the TSE Secretariat, internally and externally, in coordinating the federal government's overall approach to BSE/TSE.

Recommendation 3:  Health Canada should take steps to ensure that performance measurement takes place for BSE III and for future funded initiatives, including clarifying internal roles and responsibilities for coordinating performance measurement and reporting.

Design

While the evaluation found that program design was appropriate, to the extent that it was based on the scientific evidence available at the time and informed by risk-based analysis and some consultation with stakeholders, it also found that some Health Canada partners whose mandated responsibilities extend to BSE/TSE risk management and control efforts were not included in the Initiatives. These include the Consumer Products Safety Directorate (CPSD), which is responsible for regulating cosmetics and personal care products, and the Marketed Health Products Directorate (MHPD), which is responsible for surveillance activities for health products regulated by Health Canada. While the evaluation did not find any evidence that their exclusion had a detrimental impact on Health Canada's ability to achieve its expected outcomes, it does raise questions about the adequacy and comprehensiveness of Health Canada's overall approach to BSE/TSE. With respect to the potential role of MHPD in BSE/TSE-related surveillance, it is important to note that the PHAC was funded for and carries out prospective surveillance of all types of human prion disease through the Canadian Creutzfeldt-Jakob Disease Surveillance System (CJDSS).

Recommendation 4:  Health Canada should determine whether the CPSD and the MHPD have a role to play in its overall BSE/TSE strategy. With respect to surveillance activities, due consideration should be given to the role already performed by the PHAC through the CJDSS.

Implementation

Based on the available evidence, not all of the BSE I and II Initiatives activities were implemented as planned. For instance, the evaluation found that several targeted research projects were undertaken by the Food Directorate (FD) and the Biologics and Genetic Therapies Directorate (BGTD), including some involving international collaboration. However, other activities were not as clearly fulfilled. Although risk assessment, product assessment, and tracking and tracing were conceptualized as distinct activities with discrete funding allocations in the original planning documents, in practice these activities were not and are not necessarily distinguished from one another by all of the directorates that received funding for them. As a result, it was difficult for the evaluators to determine the extent of implementation for these activities during the time period under evaluation. While all four evaluation directorates -- Biologics and Genetic Therapies Directorate (BGTD), Therapeutic Products Directorate (TPD), Natural Health Products Directorate (NHPD), and Veterinary Drugs Directorate (VDD) -- now require sponsors to identify and provide information on animal-sourced ingredients in new drugsFootnote 1 as part of the product application process, most have not published policies and guidance documents for industry pertaining specifically to the reduction of BSE/TSE-related risks, though they may apply internal policies when reviewing product submissions.

Moreover, despite an apparent interest in a Branch-level policy on reducing BSE/TSE-related risks in the products regulated by HPFB, such a policy, though drafted, has never been finalized. While the inability to arrive at a consensus on a Branch-level policy may be a function of limited collaboration and coordination among internal partners in the Initiatives, it could also be indicative of valid differences among the regulated industries that render an overarching policy unrealistic or unfeasible.

Recommendation 5:  Health Canada should revisit the feasibility of developing a Department-wide policy on reducing BSE/TSE-related risks for the consumer and health products it regulates.

With respect to compliance and enforcement, the absence of any documentation of inspection activities specifically related to BSE/TSE, and the fact that some of the funds allocated for this purpose were evidently redirected to other Health Products and Food Branch Inspectorate (HPFBI) activities, suggest that planned compliance and enforcement activities were not fully implemented.

Recommendation 6:  Health Canada should take steps to document its inspection activities for BSE/TSE-related risks in health products regulated by HPFB, as well as the outcomes of these inspections (i.e., non-compliances found and actions taken in response to non-compliance).

Outcomes

In the absence of performance measurement data, there is some evidence from other data collection methods that some progress has been made toward expected outcomes. The evaluation found that at least some of Health Canada's regulatory and policy responses -- such as the Specified Risk Material (SRM) removal policy and amendments to the blood donor deferral policy -- were timely, based on scientific evidence, and informed by risk assessment as well as consultation with stakeholders. These examples are illustrative of an improved regulatory/policy response to control and prevent risks associated with BSE/TSE. However, based on feedback from some federal partners as well as some respondents to the industry survey, some stakeholders do not believe Health Canada consulted adequately with them when developing its regulatory and policy response.

While internal and external key informants believe that, generally speaking, awareness and understanding of BSE/TSE-related risks has increased in Canada over the past two decades as the field of BSE/TSE science has matured, results from the industry survey suggest opportunities to strengthen awareness and understanding among industry stakeholders. Most notably, even though about half of survey respondents had received information from Health Canada regarding policies and regulations affecting their industry, only one-quarter assessed their organization as having a strong understanding of Health Canada's BSE/TSE-related policies and regulations affecting them. Some internal key informants admitted that industry may not, in all cases, have a clear understanding of the BSE/TSE regulatory framework affecting them, since some of the directorates responsible for regulating health products have not published guidance documents or policies specifically pertaining to BSE, and/or make references on their websites to policies and guidance documents that are not in effect.

Recommendation 7:  Health Canada should take steps to improve the transparency of its BSE/TSE regulatory framework for health products, with a view to strengthening industry awareness and understanding.

Within Health Canada, and based primarily on qualitative evidence from internal and external key informants, expertise and knowledge of BSE/TSE science has increased over the past decade, paralleling the growth and diversification of the field over the same period. Similarly, while both internal and external key informants believe Health Canada has always taken an evidence-based and risk-based approach to decision-making, they also noted that the scientific evidence base for BSE/TSE-related decision-making is now much stronger than it was 10 years ago. They believe that in that sense, knowledge-based decision-making within Health Canada has increased. However, due to a lack of documentation describing the basis for various Health Canada policies and/or decisions, the evaluation had difficulty assessing the relative weight given to scientific knowledge versus other factors in the Department's decision-making process. Thus, the evaluation could not draw a definitive conclusion on this question.

Likewise, the evaluation could not determine whether there has been increased adherence to acts, regulations, and other guidance documents on the part of industry. For health products, barring fraudulent applications, compliance is presumed to be 100%; however, there has never been any systematic attempt to verify and authenticate the claims made by industry through the product submission process and Health Canada has no objective information on industry compliance. The absence of specific policies and guidance documents further complicates the picture, since it is not immediately obvious what industry is expected to be compliant with. For food products, an inspection program is operated by the CFIA. Key informants reported that efforts are ongoing to develop an information-sharing agreement between Health Canada and the CFIA. Such an agreement would enable Health Canada to access CFIA compliance data, which it requires to fully understand industry compliance with the SRM removal policy and update its risk assessments with respect to food products.

Recommendation 8:  Health Canada should endeavour to finalize an information-sharing agreement with the CFIA in the near future. To that end, an action plan with clear milestones and senior management support should be developed and implemented.

The evaluation found that Canada's BSE/TSE regulatory framework is reasonably well aligned with that of other jurisdictions, and to that extent there are internationally harmonized standards and regulations addressing BSE/TSE and related risks. Health Canada and its directorates participate in a variety of collaborative efforts to harmonize standards and regulations in areas covered by the BSE Initiatives, and review of regulatory approaches to BSE/TSE in several jurisdictions revealed considerable similarity among them, though some of the details vary.

Finally, the BSE Initiatives were intended to reduce exposure to the risks associated with the use of animal-sourced materials in food and health products regulated by Health Canada, and reduce the risk of acquiring human TSEs associated with these ingredients, ultimately leading to safer food and health products. Key informants agreed that risks have been reduced as a result of the implementation of control measures -- singling out in particular the SRM removal policy and the blood donor deferral policy -- although several observed that the decrease associated with these measures has been marginal since the risks were very low to begin with. Methodologically, it is extremely difficult to assess the effectiveness of risk mitigation measures, and there is little objective data to support a definitive conclusion on this question. However, data from PHAC's Creutzfeldt-Jakob Disease Surveillance System indicate that Canada does not have either transfusion-related CJD or domestically acquired vCJD.

Efficiency and economy

Based on the available financial information, it appears that a significant proportion of BSE funds were reallocated-- 36% for BSE I and 53% for BSE II. This finding is consistent with the findings of the November 2006 report of the Office of the Auditor General, which recommended that Health Canada monitor sources of program funding to ensure that resources are allocated to the intended purposes and also monitor the impact of reallocations to ensure that ability to meet program objectives is not compromised.

Recommendation 9:  Health Canada should take steps to improve financial oversight and reporting to ensure that allocated funds are used as planned. If reallocation does occur, appropriate justifications should be documented and monitoring should take place to ensure that program objectives are met.

Because Health Canada did not implement all of the BSE/TSE Initiatives as planned and a significant amount of BSE funding was reallocated, an assessment of efficiency and economy is difficult, if not impossible. External key informants were generally of the view that Health Canada's response to BSE/TSE to date has been appropriate and adequate to manage the related risks, and the review of international approaches to minimizing BSE/TSE-related risks did not find any other approaches that are radically different from what has been implemented in Canada. However, the absence of a BSE/TSE risk reduction policy for health products at the Department or Branch level and the fact that a specific inspection program related to BSE/TSE was not fully implemented may well be seen as shortcomings.

The evaluation found general support in the literature and from key informants for continued vigilance and involvement on the part of Health Canada, in light of a changing BSE and prion disease profile in Canada, and in light of significant ongoing scientific uncertainty related to, for example, TSEs other than BSE, especially CWD; atypical BSE; and emergent risks with respect to human-to-human transmission. In this context, key informants believe that Health Canada has an important ongoing role to play, particularly in the areas of risk assessment and BSE/TSE research to inform policy and regulatory development.

Management Action Plan

Bovine Spongiform Encephalopathy (BSE) I and II - Summative Evaluation
Recommendation(s) Response Key Activities Deliverables Responsible Manager Timeframe

Health Canada (HC) should continue to play a role in Bovine Spongiform Encephalopathy (BSE)/Transmissible Spongiform Encephalopathy (TSE) risk assessment and research to inform policy and regulatory development. Particular attention should be paid to the evolving science on chronic wasting disease (CWD), given its potential health implications for all Canadians.

Agree

1) Biologics and Genetic Therapies Directorate (BGTD) will communicate the criteria for performing TSE risk assessments for human blood, tissues, organs and urine-derived products.

Development of a specific guidance document on minimizing risks from human-derived material.

Director, Office of Policy and International Collaboration and Centre for Blood and Tissues Evaluation, BGTD, HPFB, HC

September 2013

2) BGTD will conduct targeted research to inform risk assessments in biotherapeutics.

Generation of new knowledge of the nature of prion structure at the cell membrane to inform risk assessment of biotherapeutics and publication of same in the scientific literature.

Director, Centre for Vaccine Evaluation, BGTD, HPFB, HC

March 2014

Development of method to identify and quantify prion proteins in urine and urine-derived biotherapeutics.

Director, Centre for Vaccine Evaluation, BGTD, HPFB, HC

March 2014

3) Food Directorate (FD) will complete its targeted research for CWD and species barrier so that it can update its risk profile in collaboration with the Canadian Food Inspection Agency (CFIA) and the Public Health Agency of Canada (PHAC). This will include potential impacts on First Nations and Inuit populations.

Update the current HC risk profile on the transmission of CWD and human health using the results of the targeted CWD research which is currently underway.

Director, Bureau of Microbial Hazards, FD, HPFB, HC

March 2014

2) Health Canada should consider whether there is still a necessary role for the TSE Secretariat, internally and externally, in coordinating the federal government's overall approach to BSE/TSE.

Agree

The FD will consider eliminating the TSE Secretariat upon an evaluation of its role.

Director General FD, HPFB, HC

March 2014

3) Health Canada should take steps to ensure that performance measurement takes place for BSE III and for future funded initiatives, including clarifying internal roles and responsibilities for coordinating performance measurement and reporting.

Agree

HPFB Directorates receiving funding under BSE III for risk assessments and targeted research will report on activities, expected outcomes and results in the 2012-13 Departmental Performance Report (DPR) supplementary table (horizontal initiative) specific to this initiative to the CFIA-led BSE III evaluation and to the HPFB Assistant Deputy Minister (ADM). Roles and responsibilities for program reporting have been defined to rest within HPFB's Strategic Planning & Accountability Division (SPAD).

2012-13 DPR Supplementary table will contain performance information for BSE III.

Director, SPAD, Resource Management and Operations Directorate, (RMOD), HPFB, HC

April 2013

4) Health Canada should determine whether the Consumer Product Safety Directorate (CPSD) and the Marketed Health Products Directorate (MHPD) have a role to play in its overall BSE/TSE strategy. With respect to surveillance activities, due consideration should be given to the role already performed by the PHAC through the Creutzfeldt-Jakob Disease Surveillance System (CJDSS).

Agree

In consultation with CPSD and MHPD, HC decided not to allocate funds to these parties under the BSE I & II Treasury Board Submissions.  However, CPSD and MHPD will be included in discussions on future Treasury Board Submissions relating to BSE/TSE.

CPSD and MHPD will be engaged on future BSE/TSE related activities to determine possible involvement/activities pertaining to their mandate.

TSE Secretariat, FD, HPFB, HC in collaboration with PPIAD (Policy, Planning and International Affairs Directorate)/RMOD (renewal of Treasury Board Submission)

Date of future Treasury Board Submission unknown

5) Health Canada should revisit the feasibility of developing a Department-wide policy on reducing BSE/TSE-related risks for the consumer and health products it regulates.

Agree

HPFB will analyze the feasibility of developing a department wide policy on reducing BSE /TSE -related risks for the consumer and health products it regulates.

HPFB will hold a meeting to determine the feasibility of developing a Department-wide policy on BSE/TSE related risks.

TSE Secretariat's Senior scientific advisor for TSE/BSE in the Bureau of Microbial Hazards, FD, HPFB, HC

Sept 2013

6) Health Canada should take steps to document its inspection activities for BSE/TSE-related risks in health products regulated by HPFB, as well as the outcomes of these inspections (i.e., non-compliances found and actions taken in response to non-compliance).

Agree

HPFB Inspectorate will examine the feasibility of developing specific documentation relating to BSE as part of the Inspectorate program.

Review of the feasibility of documenting BSE specific inspection activities and the resulting outcomes.

DG, HPFBI, HPFB, HC

March 2014

7) Health Canada should take steps to improve the transparency of its BSE/TSE regulatory framework for health products, with a view to strengthening industry awareness and understanding.

Agree

HPFB will improve transparency by posting documents relating to the management of BSE/TSE risk on the HC website.

Documents and applicable forms on managing BSE/TSE risk from applicable health product directorates posted on the HC website.

Applicable HPFB health product directorates at HC.

Existing documents posted by December 2012, new documents posted by March 2014

8) Health Canada should endeavour to finalize an information-sharing agreement with the CFIA in the near future. To that end, an action plan with clear milestones and senior management support should be developed and implemented.

Agree

The existing Memorandum of Understanding governing the
roles, responsibilities and interactions between PHAC, CFIA and Health Canada, which has been updated in 2008, offers the cover required for confidential information sharing which would include compliance and enforcement data related to BSE to support Health Risk Assessments. HC and CFIA officials are discussing the means by which the effectiveness of the provisions of this agreement, as they relate to BSE information, is ensured.

Effective means to share information related to BSE between CFIA and HC implemented.

Director, Bureau of Microbial Hazards, FD, HPFB, HC

March 2014

9) Health Canada should take steps to improve financial oversight and reporting to ensure that allocated funds are used as planned. If reallocation does occur, appropriate justifications should be documented and monitoring should take place to ensure that program objectives are met.

Agree

HPFB Directorates receiving funding under BSE III will be coding their 2013-14 actual expenditures to SAP Internal Orders for this initiative. HPFB Directorates transferring funding to other Directorates and/or to other Branches will inform the receiving partner(s) of the reporting requirement and of the use of the Internal Orders.

The 2013-14 DPR on BSE III will report on actual expenditures.

Director, SPAD, RMOD, HPFB, HC in collaboration with FD, BGTD, TPD, and the Natural Health Products Directorate (NHPD).

April 2013

1.0 Introduction

Bovine Spongiform Encephalopathy (BSE) is a fatal prion disease that occurs in cattle and causes a sponge-like degradation of the brain. While BSE represents a significant animal health concern, it is also believed to be linked to a fatal human prion disease known as variant Creutzfeldt-Jakob Disease (vCJD), which may occur after consuming food containing the agent that causes BSE.

The presence of BSE in the United Kingdom (UK), as well as the first case of BSE in a Canadian-born cow in May 2003, has led to a variety of actions by the federal and provincial/territorial governments in response to the threat. The federal BSE strategy involves several delivery partners, including Health Canada, the Canadian Food Inspection Agency (CFIA), Agriculture and Agri-food Canada (AAFC), the Department of Foreign Affairs and International Trade (DFAIT), and the Public Health Agency of Canada (PHAC). The strategy is intended to safeguard the health of Canadians by minimizing the risk of exposure to the BSE agent, protecting the national herd and re-establishing international markets for Canadian beef products.

Health Canada's responsibilities within the federal BSE strategy were executed through the BSE I and BSE II Initiatives, which were designed to complement Health Canada's existing responsibilities and activities.

  • BSE I (Augmenting Health Canada's Response to BSE, 2003-2008) intended to augment product assessment and develop a larger knowledge base to track and trace health products that have increased risk of BSE. Part of this Initiative involved ongoing reviews of food and health products -- including biological products, human and veterinary drugs, natural health products, and medical devices -- with bovine-sourced materials that could pose risks to Canadian consumers. The Initiative also included identification and ongoing surveillance of bovine-sourced materials that may be used in food and health products, as well as compliance and enforcement activities intended to ensure compliance with the Food and Drugs Act and Regulations.
  • BSE II (Further Measures on BSE in Areas of Risk Assessment and Targeted Research, 2004-2009) intended to enhance the risk assessments of consuming bovine-derived products. The risk assessments were meant to determine the risks posed to humans by products that may be potentially infected with the BSE agent, including food and health products. This Initiative also involved funding targeted scientific research to further understand the pathogenesis and biological characteristics of BSE and other Transmissible Spongiform Encephalopathies (TSEs). The goals of the research were to support development of an appropriate regulatory framework for BSE/TSEs and mitigate the public health impacts from animal TSE diseases through improving understanding of BSE pathogenesis; increasing capacity to identify exposure risks through all consumer products; improving standardized diagnostics; and improving understanding of BSE zoonoticFootnote 2 potential.

An evaluation of the BSE I and BSE II Initiatives is part of Health Canada and Public Health Agency of Canada's Five-Year Evaluation Plan. The evaluation assessed the relevance and performance (effectiveness, efficiency, and economy) of the Initiatives, in accordance with Treasury Board of Canada's Policy on Evaluation (Treasury Board of Canada Secretariat, 2009). The evaluation was conducted by PRA Inc., an independent evaluation consulting firm, on behalf of Health Canada. This report presents the evaluation findings, draws conclusions, and makes recommendations.

1.1 Organization of the report

The report is organized in several sections. Section 2 describes the context for the BSE I and II Initiatives and provides a detailed program profile. Section 3 describes the methodology, while Section 4 provides the evaluation findings. Section 5 concludes and makes recommendations. Three appendices accompany the main report. Appendix A contains the evaluation matrix; Appendix B contains the data collection instruments; and Appendix C contains the list of references.

In Section 4, the evaluation findings are organized by evaluation question, and a rating is provided for each question. The ratings are based on a judgment of whether the findings indicate the following:

  • The intended outcome or goal has been fully achieved or met -- labelled as Achieved.
  • Some progress has been made to meet the intended outcome or goal, but attention is still needed -- labelled as Some evidence of progress; attention needed.
  • Some progress has been made to meet the intended outcome or goal, but evidence is insufficient to support a firm conclusion -- labelled as some evidence of progress; insufficient to support firm conclusion.
  • The intended goal or outcome has not been achieved and/or the evaluation found significant issues pertaining to the intended goal or outcome -- labelled as Attention needed.
  • The evaluation could not conclude whether the intended goal or outcome has been achieved due to insufficient evidence -- labelled as insufficient evidence to support conclusion.

Finally, if a rating does not apply to an evaluation question, it is labelled as not applicable.

A summary of the ratings for each evaluation question is provided in Table 1 below.

Table 1: Summary of ratings by evaluation question
Evaluation questions Achieved Some evidence of progress; attention needed Some evidence of progress; insufficient to support firm conclusion Attention needed Insufficient evidence to support conclusion Not applicable

Relevance

Is there a continued need for the BSE I and BSE II Initiatives?

Do the BSE I and BSE II Initiatives align with government priorities?

Do the BSE I and BSE II Initiatives align with federal roles and responsibilities?

Performance

Are the governance structures likely to support the achievement of expected outcomes?

Have the BSE I and BSE II Initiatives been designed appropriately to achieve expected outcomes?

Have the BSE I and BSE II Initiatives been implemented appropriately to achieve expected outcomes?

To what extent is there improved regulation/policy response to control and prevent risks associated with BSE/TSE?

To what extent is there increased awareness and understanding of BSE/TSE risk control efforts, regulations, and policies among partners and stakeholders?

To what extent is there increased adherence to Acts, regulations, and other guidance documents by industry?

To what extent is there increased expertise and knowledge of BSE/TSE science and risk within Health Canada?

To what extent are there internationally harmonized standards and regulations addressing BSE/TSE and related risks?

To what extent is there reduced exposure to the risks associated with the use of animal-sourced materials in food and products regulated by Health Canada?

To what extent is there increased knowledge-based decision-making in Health Canada?

To what extent is there reduced risk of acquiring human TSEs associated with animal-sourced ingredients in food and products regulated by Health Canada?

To what extent are food and health products safer?

Were there any unintended consequences (positive or negative) as a result of the BSE I and BSE II Initiatives?

Were resources deployed at the least cost, consistent with realizing timely outputs that met the requirements of the Initiatives?

Did the outputs of the Initiatives meet needs at the lowest cost?

Are there alternate ways to deliver the Initiatives to achieve similar results at lower cost?

2.0 Context and program profile

BSE is one of a group of prion diseases also known as TSEs. TSEs can afflict both animals and humans. The prions in an infected brain alter proteins and neurons, causing fatal brain damage (Centers for Disease Control and Prevention [CDC], 2010a). While TSEs are believed to have existed in Europe for centuries, prion disease in humans -- Creutzfeldt-Jakob Disease (CJD) --was first identified in the 1920s (Belay & Schonberger, 2005).

Several prion diseases have been identified over time. Animal TSEs include BSE, which occurs in cattle; scrapie, which affects sheep; and Chronic Wasting Disease (CWD), which affects cervids such as deer and elk. Human prion diseases are of three basic types: sporadic CJD, the cause of which is unknown; genetic prion disease; and acquired prion disease (PHAC, 2011a). Acquired human prion diseases are further categorized as either iatrogenic CJD, which is acquired through infectious transmission (i.e., contamination through brain surgery, corneal transplant, dura mater graft, human growth hormone, or through transfusion-associated vCJD transmission) or variant CJD (vCJD), which is acquired through exposure to BSE (PHAC, 2011). Currently, there are no available blood tests to screen for vCJD. BSE is likely the best-known prion disease because of its devastating effect in 1986 on the cattle industry in the UK. Scientists initially assumed it did not pose a risk to humans, since it likely originated from scrapie-infected sheep used in cattle feed, and scrapie has not been shown to be transmissible to humans. Due to a general lack of knowledge surrounding BSE, the disease had become an epidemic in cattle by the time the UK implemented the 1988 animal feed ban, followed in 1989 by the Specified Bovine Offal Ban. The UK also developed a policy to destroy all cattle showing signs of BSE. Although these policies almost certainly helped reduce the impact of the epidemic, approximately 12,000 cattle born after the ban in 1988 and another 12,000 in 1989 developed clinical signs of BSE; and these did not include cattle with BSE that were slaughtered before developing symptoms (Phillips, Bridgemen, & Ferguson-Smith, 2000, para. 10). The cross-contamination of animal feeds led to ongoing infection of cattle throughout the UK that, at its peak in 1993, included almost 1,000 new cases per week, and a total of almost 185,000 cases by 2008 (CDC, 2010b).

Following the appearance of BSE, vCJD was identified in humans in the UK, and subsequent testing found that "the most likely cause of vCJD is exposure to the BSE agent, most plausibly due to dietary contamination by affected bovine central nervous system tissue" (World Health Organization [WHO], 2002, para. 22).

2.1 Canada's response to BSE

International organizations such as the WHO and the Organisation for Animal Health (OIE), as well as various nations including Canada, have taken action in response to BSE and human TSEs. Major elements of Canada's policy response were introduced between 1990 and 2002 -- before the BSE I and BSE II Initiatives were implemented -- and are summarized in Table 2 below.

Table 2: Chronology of major elements of Government of Canada's response to BSE/TSEs
Year Description

1989

Ban on all UK cattle imports and subsequently on other countries known to represent a BSE risk

1990

BSE made a reportable disease in Canada; any suspected case must be reported to a federal veterinarian

1992

Implementation of a national BSE surveillance program, including training and education components

1992

Health Canada introduces requirement for manufacturers to provide information on animal tissues used in pharmaceutical products

1995

Recall of blood products by the Canadian Red Cross, based on information that two previous donors had received diagnoses of probable classic CJD (Clark, 1995)

1997

Ban on the use of rendered animal proteins of ruminant origin (excluding milk, blood, and fat) in feed for ruminants

1998

Health Canada establishes CJD/vJCD surveillance network

1998

Revisions to the animal compensation program

1998

Policies on bovine-derived ingredients used in health products introduced

1998

Trilateral country classification system

1998

Import restrictions on ruminant products; specific articulation of BSE policy

1999

Health Canada contacts the pharmaceutical industry to collect information on the use of animal-derived materials in all human and animal health products for which a Drug Identification Number (DIN) has been issued

1999

First blood donor deferral policy introduced, deferring blood donations from any individual who had spent six months or longer in the UK during the period of 1980 to 1996

2000

Second blood donor deferral policy introduced, deferring blood donations from individuals who had spent six months in France from 1980 to 1996

2001

Health Canada again contacts the pharmaceutical industry to collect information on the use of animal-derived materials in all human and animal health products for which a Drug Identification Number (DIN) has been issued

2001

Canadian Cattle Identification Program introduced, enabling Canada to trace individual animal movements from the herd of origin to slaughter

2001

Government of Canada creates the TSE Secretariat within Health Canada, in response to BSE actions directed at Brazil

2001

Third blood donor deferral policy (enhanced)

2002

vCJD detected in Canada; attributed to residence in UK at the height of the epidemic

2002

Infection Control Guidelines: Creutzfeldt-Jakob Disease in Canada introduced

2002

Suspension of the license for, and recall of, Tutoplast Dura (a commercially processed dura mater) following confirmation of a case of classical CJD in a patient who had received a Tutoplast Dura graft (PHAC, 2006)

Source: Information provided by HPFB unless otherwise noted.

In May 2003, a single Canadian-born cow was discovered to have acquired BSE, dispelling the notion that the livestock production system in Canada was BSE-free. This incident triggered widespread border closure to the export of both live cattle and boxed beef, throwing the entire beef production cycle into crisis and resulting in a rapid drop in price and loss of sales. Canada responded by requiring the removal of bovine Specified Risk Material (SRM) and diverting SRM from the food supply (Health Canada, 2003a). The SRM removal policy was announced in July 2003 and changes to the Food and Drug Regulations came into effect in August 2003. Section B.01.047.1 (1) of the Food and Drug Regulations (Government of Canada, 2011a) provides that:

  • "...specified risk material" means
  • (a) the skull, brain, trigeminal ganglia, eyes, tonsils, spinal cord and dorsal root ganglia of cattle aged 30 months or older; and
  • (b) the distal ileum of cattle of all ages.
  • (2) No person shall sell or import for sale food that contains specified risk material."

Development of the SRM removal policy was the responsibility of Health Canada, while its enforcement is the responsibility of the CFIA, as per their respective legislated mandates and authorities.

2.2 The federal BSE strategy

The 2003 Canadian BSE incident and further cases of indigenous BSE led to a decision by the federal government to create an integrated strategy for managing the risks of the disease. The strategy involved the CFIA, AAFC, and Health Canada (Health Canada, 2010, p. 3). The strategy was designed to reduce the risk of BSE to human and animal health through a variety of activities, such as removal of SRM from food; health product assessment; cattle surveillance and identification; and targeted investment in BSE-related research and risk assessment.

AAFC, Health Canada, and the CFIA each have specific roles and fund separate initiatives within the federal strategy. AAFC specializes in supporting the Canadian beef and cattle industry, and compensates stakeholders that have been negatively affected by the BSE crisis. Health Canada's responsibilities within the federal strategy consist, broadly speaking, of implementing a tracking and surveillance system and conducting scientific research and enhanced BSE risk assessments, and its funding also includes a portion allocated to PHAC for research activities.Footnote 3 The CFIA, along with Health Canada, concentrates on increasing the knowledge base regarding BSE and TSEs, while implementing enhanced risk mitigating measures to protect human and animal health.

In addition, DFAIT, along with AAFC, focuses on the importing and exporting market and any transfers that involve food-producing animals or animal-based foods. The provincial and territorial governments work with the partners of the BSE I and II Initiatives on BSE and TSE issues within their respective jurisdictions.

It is important to note that this evaluation is focusing only on Health Canada's activities under the federal BSE strategy.

2.3 Profile of Health Canada's BSE I and BSE II Initiatives

Health Canada's responsibilities within the BSE federal strategy are executed through the BSE I and II Initiatives, which are designed to complement previous Health Canada activities and responsibilities.

Activities

The BSE Initiatives consist of five interrelated activities aimed at developing, implementing, and maintaining the BSE/TSE regulatory framework in order to maintain and improve the health of Canadians. The Initiatives and activities, along with the rationale for each activity according to government planning documents, are described below.

BSE I (Augmenting Health Canada's Response to BSE, 2003-2008) provided funding for three main activities:

  • Product assessment. This activity consisted of the ongoing review of priority food and health products (up to 20%) with the potential to present a risk to the health of Canadians, due to bovine-sourced materials or other ingredients that may pose a BSE infection risk. Although the federal government's assumption prior to May 2003 was that all bovine-derived food and health products sourced from Canada or elsewhere and destined for human use could be reasonably expected to be safe, the confirmation of BSE in a domestic Canadian cow led the federal government to adopt the assumption that Canada may have a low and previously undetected level of BSE prevalence.
  • Tracking and tracing. This activity consisted of the identification and ongoing surveillance of bovine-sourced materials that may be used as ingredients in foods, biological/therapeutic products, drugs for human or animal use, natural health products, medical devices, or cosmetics. Prior to the discovery of BSE within Canada, Health Canada did not have a comprehensive system of collecting and tracking information on the origins, nature, processing, and use of these materials. Access to this information was expected to enhance Health Canada's ability to address its regulatory responsibilities with respect to managing safety risks.
  • Compliance and enforcement. This activity consisted of monitoring industry in order to verify acceptable sources of raw materials and ensure its compliance with the Food and Drugs Act and Regulations concerning the creation of food products, biological/ therapeutic products, medical devices, and cosmetics; as well as taking appropriate action to enforce the regulations if non-compliance was found. This component also consisted of preventative measures including setting restrictions on the import of food or animal products that are determined to present a BSE/TSE risk and enforcing these restrictions. Under BSE I, Health Canada is responsible for compliance and enforcement activities with respect to health products, while the CFIA is responsible for compliance and enforcement activities with respect to food products.

BSE II (Further Measures on BSE in Areas of Risk Assessment and Targeted Research, 2004-2009) was launched one year after BSE I, and provided funding for two main activities:

  • Targeted research. This component was expected to increase understanding of the science underpinning BSE/TSE, in order to inform risk assessments. More specifically, this component consisted of targeted research into the characteristics of BSE/TSE, in order to improve understanding of BSE pathogenesis, increase capacity to identify exposure risks through all consumer products; improve standardized diagnostics; develop a better understanding of BSE zoonotic potential; and mitigate the public health impacts from animal TSE diseases. Since 2005-2006, Health Canada's funding under BSE II included a portion allocated to PHAC for targeted research.
  • Risk assessment. This activity consisted of risk assessments to determine the potential prion disease risks to humans posed by identified food and health products and ingredients. This component also includes activities related to the development of guidelines and protocols to improve the methods for selecting and screening products. The completed risk assessments would provide the necessary evidence base to inform BSE/TSE-related regulatory actions to protect human health.  

Two additional components of the BSE I and BSE II Initiatives are also identified in the program logic model (see page 12). These include:

  • Coordination and communication in issues related to BSE/TSE, in order to share knowledge and increase awareness pertaining to BSE with the general public, health officials, and provincial and international governments.
  • Regulatory framework includes developing, implementing, and maintaining policies and regulations for products that have been derived from animal material.

In 2009-2010, Canada provided additional funding for BSE-related activities to Health Canada, the CFIA, and PHAC under BSE III.

Partners

Responsibility for the BSE I and BSE II Initiatives resides within the Health Products and Food Branch (HPFB) of Health Canada, the mandate of which is to manage the health-related benefits and risks of food and health products using an integrated approach. Its goals are to minimize health risk factors while also maximizing the safety of food and health products; and to promote conditions to support information dissemination to Canadians so that they may make informed health-related decisions.

Within HPFB, a number of partners received funding and are responsible for activities under BSE I and BSE II. These internal partners include:

  • Biologics and Genetic Therapies Directorate (BGTD).The BGTD regulates drugs and products derived from living sources, and radiopharmaceuticals. BGTD is responsible for approving and monitoring blood and blood products, vaccines, gene therapy products, tissues, and organs. In addition, information generated by BGTD is used by health care professionals to make recommendations to their patients about biological products.
  • Food Directorate (FD). The FD's role is to set policies, create standards, and provide information on the nutritional value and safety of food. The Directorate undertakes a variety of activities, including scientific research, risk assessments, policy development, and evaluation of submissions from the food industry. Its areas of focus involve food contamination (by microbiological or chemical agents), food additives, food processing, nutrition, and TSEs.
  • Health Products and Food Branch Inspectorate (HPFBI). HPFBI is responsible for the delivery of a national compliance and enforcement program for all products under the mandate of HPFB (except food). This program is intended to support the Branch's risk management approach to decision making and its goal of a comprehensive regulatory strategy across all product classes. Inspectorate activities include inspection, compliance verification and investigation, establishment licensing, and related laboratory functions.
  • Natural Health Products Directorate (NHPD).The NHPD regulates natural health products in Canadian markets. The Directorate is responsible for assessing the safety, effectiveness, and quality of these products. Natural health products are often made from plants, but can also contain parts from animals and microorganisms, which can pose health risks to Canadians. Some examples of natural health products are herbal medicine, homeopathic medicine, vitamins, minerals, probiotics, and traditional Chinese medicine.
  • Policy, Planning, and International Affairs Directorate (PPIAD). The PPIAD provides leadership and support on strategic planning, as well as policy development and planning on horizontal issues of strategic importance. During the period of the BSE Initiatives, it also assisted the Branch Executive Committee in making the strategic planning process the basis for Branch decision making on resource allocation, performance measurement and reporting, and communicating with government and stakeholders on strategic directions and performance plans and results.
  • Therapeutic Products Directorate (TPD). The TPD regulates pharmaceutical drugs and medical devices for human use.The Directorate undertakes risk assessments and policy development, and evaluates submissions from manufacturers regarding the safety, efficacy and quality of pharmaceutical products and medical devices.
  • Transmissible Spongiform Encephalopathy (TSE) Secretariat. The TSE Secretariat was created in 2001, following a Government of Canada decision, and located within the FD. It was intended primarily as a scientific management, monitoring, and dissemination body for the federal government's TSE-related activities. The TSE Secretariat moved to the Veterinary Drugs Directorate (VDD) in 2006, and returned to the FD in 2010.
  • Veterinary Drugs Directorate (VDD).The VDD sets standards for veterinary drugs and evaluates and monitors their safety, effectiveness, and quality. The Directorate monitors drugs for both food-producing animals and pets. Its work involves reviewing veterinary drug submissions and conducting risk analyses, as well as information management, policy development, and communications with Canadians involving veterinary drugs.

It is important to note that two Health Canada directorates whose mandates and responsibilities would seem to implicate them in the Department's BSE activities were not funded under the BSE Initiatives. The Consumer Product Safety Directorate within the Healthy Environment and Consumer Safety Branch (HECSB), whose responsibilities include regulating cosmetics and personal care products, was identified in government planning documents as having a role in BSE-funded activities with respect to cosmetics, but did not receive any BSE funding. Similarly, the Marketed Health Products Directorate, which is responsible for post-approval safety surveillance, assessment of signals, and safety trends and risk communications concerning all regulated marketed health products --including those implicated in the BSE Initiatives -- did not receive any BSE funding.

It is also important to note that under BSE I, Health Canada is responsible for compliance and enforcement activities with respect to health products, while the CFIA is responsible for compliance and enforcement activities (e.g., inspection) with respect to food products. The CFIA activities are not included in the scope of this evaluation.

Table 3 below depicts the roles and responsibilities of the various Health Canada partners in carrying out the activities funded through the BSE I and BSE II Initiatives. The TSE Secretariat is not included in the figure, since it has been transferred several times among directorates. Other partners such as the CFIA and PHAC are also not included in this table.

Table 3: Roles and responsibilities by Health Canada partners
Roles and Responsibilities Health Canada
Health Products and Food Branch
BGTD FOOD NHPD TPD VDD HPFBI PPIAD
(PPIAD - formerly known as PSPD)

BSE I

Product Assessment

X

X

X

X

X

Tracking & Tracing

X

X

X

X

Compliance & Enforcement

X

BSE II

Risk Assessment

X

X

X

X

X

Targeted Research

X

X

Evaluation

X

2.3.3 Expected outcomes

The implementation of the activities described above corresponds with specific immediate, intermediate, and long-term outcomes of the Initiatives. Greater communication practices between stakeholders, agencies, and other governments are expected to lead to improved regulations related to BSE, as well as an increased awareness of control and risk measures (Health Canada 2010, p. 38). Health Canada (2010) also suggests that the compliance and enforcement activities conducted by the HPFBI and the CFIA, combined with a greater understanding of BSE regulations and policies, will improve adherence by industry (Health Canada 2010, p. 39). In addition, the activities involving risk and product assessments, tracing and tracking, and research on animal ingredients in food and health products are intended to increase expertise and knowledge of BSE and TSE science with Health Canada. This could lead to improved TSE detection methods in animals and humans, improved data screening processes, better product assessments of high-risk products, and a greater understanding of the use of animal materials in food and health products (Health Canada, 2010, p. 39).

The achievement of these immediate outcomes is expected to lead to intermediate outcomes of a reduced risk of using animal material in products, internationally harmonized BSE standards and regulations, and an increase in knowledge-based decision-making within Health Canada. As a result of these intermediate outcomes, Health Canada expects to reduce the risk for humans to contract BSE through products containing animal material, and to have an increased level of safe food and health products available to the Canadian public. All of these outcomes incorporate Health Canada's ultimate program goal of "improving the health and well-being of Canadians" (Health Canada, 2010). A summary of the expected outcomes of the BSE I and BSE II Initiatives is listed below, and a logic model, depicting the linkages between BSE I and BSE II activities, outputs, and expected outcomes, is in Table 4.

Expected outcomes of the BSE I and BSE II Initiatives

Immediate outcomes

  • improved regulation/policy response to control and prevent risks associated with BSE/TSE
  • increased awareness and understanding of BSE/TSE risk control efforts, regulations, and policies among partners and stakeholders
  • increased adherence to acts, regulations, and other guidance documents
  • increased expertise and knowledge of BSE/TSE science, risks, and surveillance

Intermediate outcomes

  • internationally harmonized standards and regulations addressing BSE/TSE and related risks
  • reduced exposure to the risks associated with the use of animal-sourced ingredients and food and products regulated by Health Canada
  • increased knowledge-based decision-making

Long-term outcomes

  • reduced risk of acquiring human TSEs associated with animal-sourced ingredients in food and health products regulated by Health Canada
  • safer food and health products

Ultimate outcome

  • health status of Canadians maintained or improved
Table 4: Logic Model for BSE I and BSE II
BSE I & II project funding
Input Personnel (expertise) Facilities, infrastructure Acts, regulations, policies, priorities Stakeholders

Activities

Research Table 1 footnote 1
Targeted laboratory studies of BSE

Risk Assessment1
Assess potential risks to human health of bovine & other ruminant materials in products

Product Assessment
Evaluate food & health products & ingredients containing animal-sourced material

Tracking/Tracing
Identify & track products & ingredients containing animal-sourced materials

Coordination/
Communication
Coordinate BSE-related activities & communication with stakeholders

Compliance/
Enforcement
Conduct compliance & enforcement activities

Outputs

  • Priority, domestic, & international products assessed
  • Test reports, certificates
  • Policies
  • Tracking information for animal-sourced material
  • New & enhanced tracking tools
  • Policies
  • Documents
  • Coordinated initiatives
  • International contacts
  • Common performance measures
  • Consumer outreach
  • Communications plans
  • Policies
  • Inspection reports
  • Investigation reports
  • Policies

Knowledge Transfer & Information Sharing

Academic scientists, Research networks, Government, Industry, general public, international partners

Level 1
Outcomes

Increased research capacity, information & surveillance regarding BSE risk to humans 1

Increased information, traceability & surveillance of Products

Improved coordination & communication of BSE control efforts

Improved industry
awareness

Level 2
Outcomes

Assess effectiveness & strengthen BSE risk-mitigation measures in Canada

Enhanced response to BSE-related issues

Increased public understanding
of BSE issues & risks

Increased industry compliance with enhanced BSE requirements

Ultimate
Results

  • Safer food & health products
  • Increased public confidence in food & health product safety
  • Health status of Canadians maintained or improved

Table 1 footnotes

Table 1 footnote 1

BSE II activities. The remaining are BSE I activities.
Updated: June 27. 2007

Return to table 1 footnote 1 referrer

2.3.4 Resources

A total of $10.38 million was allocated to BSE I between 2003-2004 and 2007-2008. The Initiative began with 9.33 FTEs (5.6 for Product Assessment, 2.8 for Tracking and Tracing, and 0.93 for Compliance and Enforcement) in its first year, rising to 17.26 for all subsequent years (10.4 for Product Assessment, 5.2 for Tracking and Tracing, and 1.66 for Compliance and Enforcement).

Table 5: Planned annual funding for the BSE I Initiative
Component 2003-2004 2004-2005 2005-2006 2006-2007 2007-2008 Total

Product Assessment

$756,000

$1,368,000

$1,368,000

$1,368,000

$1,368,000

$6,228,000

Tracking and Tracing

$378,000

$684,000

$684,000

$684,000

$684,000

$3,114,000

Compliance and Enforcement

$126,000

$228,000

$228,000

$228,000

$228,000

$1,038,000

Total

$1,260,000

$2,280,000

$2,280,000

$2,280,000

$2,280,000

$10,380,000

Source: Government documents.

BSE II was originally planned to run for only two years, but was sustained with a one-year and subsequent two-year extension through to 2008-2009, increasing its total allocation to $35 million over five years. The Initiative maintained a strategy of temporary and short-term contracts rather than long-term increases to personnel capacity. Initially, a total of 27.7 FTEs were allocated to the Initiative (25.4 for Risk Assessment and 2.3 for Research), which was increased to 29.8 (25 for Risk Assessment and 4.8 for Research) by 2008-2009.

Over the period of 2004-2005 to 2006-2007, funding was allocated in the amount of $5 million to Risk Assessment and $2 million to Research in each year (Health Canada, 2007a, pp. 46-47). It is not clear from available information whether these allocations remained the same in 2007-2008 and 2008-2009. However, total funding for both risk assessment and targeted research was approximately $5.7 million in the 2007-2008 and 2008-2009 fiscal years.

Table 6: Planned annual funding for the BSE II Initiative
Organization 2004-2005 2005-2006 2006-2007 2007-2008 2008-2009 Total

Health Canada

$7,000,000

$6,222,200

$6,222,200

$6,222,200

$6,222,200

$31,888,800

PHAC

-

$777,800

$777,000

$777,800

$777,800

$3,110,400

Total

$7,000,000

$7,000,000

$6,999,200

$7,000,000

$7,000,000

$34,999,200

Source: Government documents.

3.0 Methodology

This section of the report provides a detailed description of the evaluation methodology. The section includes:

  • a list of the evaluation issues and questions
  • a description of the overall evaluation design
  • a description of each data collection method
  • a description of the approach to data analysis
  • a discussion of the limitations of the methodology, as well as mitigation strategies

3.1 Evaluation issues and questions

The evaluation addressed 10 key questions focusing on relevance and performance.

Relevance

  • 1. Is there a continued need for the BSE I and BSE II Initiatives?
  • 2. Do the BSE I and BSE II Initiatives align with government priorities?
  • 3. Do the BSE I and BSE II Initiatives align with federal roles and responsibilities?

Performance

  • 4. Are the governance structures likely to support the achievement of expected outcomes?
    1. Are the Health Canada BSE Initiative partners' roles and responsibilities clearly articulated and understood?
    2. To what extent is there communication and collaboration among Health Canada BSE Initiative partners?
  • 5. Have the BSE I and BSE II Initiatives been designed and implemented appropriately to achieve expected outcomes?
    1. Was the design of the Initiatives supported by evidence?
    2. Were the Initiatives implemented as planned?
    3. Has implementation of the Initiatives been supported by evidence?
  • 6. To what extent has progress towards expected outcomes (immediate, intermediate, and long term) been achieved with reference to the BSE I and BSE II Initiatives' performance targets (where applicable) and reach?
    1. To what extent is there improved regulation/policy response to control and prevent risks associated with BSE/TSE?
    2. To what extent is there increased awareness and understanding of BSE/TSE risk control efforts, regulations, and policies among partners and stakeholders?
    3. To what extent is there increased adherence to acts, regulations, and other guidance documents by the regulatees?
    4. To what extent is there increased expertise and knowledge of BSE/TSE science and risk within Health Canada?
    5. To what extent are there internationally harmonized standards and regulations addressing BSE/TSE and related risks?
    6. To what extent is there reduced exposure to the risks associated with the use of animal-sourced materials in food and products regulated by Health Canada?
    7. To what extent is there increased knowledge-based decision-making in Health Canada?
    8. To what extent is there reduced risk of acquiring human TSEs associated with animal-sourced ingredients in food and products regulated by Health Canada?
    9. To what extent are food and health products safer?
  • 7. Were there any unintended consequences (positive or negative) as a result of the BSE I and BSE II Initiatives?
  • 8. Were resources deployed at the least cost, consistent with realizing timely outputs that met requirements of the BSE I and BSE II Initiatives (economy)?
    1. Were expenditures within budget? What accounted for overruns or lower than planned expenditures?
    2. Are processes for allocating staff to activities clearly documented, streamlined and well-understood by all staff?
  • 9. Did the outputs of the BSE I and BSE II Initiatives meet needs at the lowest cost (efficiency)?
    1. Were the outputs of sufficient quality to achieve the immediate outcomes?
  • 10. Are there alternate ways to deliver the BSE I and BSE II Initiatives to achieve similar results at lower cost?

Appendix A contains a detailed evaluation matrix that links each question to a set of indicators, data sources, and collection methods. The matrix conforms to the Treasury Board of Canada's Policy on Evaluation.

Evaluation design and data collection methods

The evaluation design was developed based on the findings of an evaluability assessment completed as a first step in the evaluation. The purpose of the evaluability assessment was to determine the extent to which the data sources -- and in particular documents and administrative data -- identified in the framework would be available to the evaluation, as well as to identify gaps in data that would need to be addressed through other data collection activities.

To complete the evaluability assessment, PRA undertook three main activities:

  1. Preliminary interviews with key program stakeholders. PRA interviewed 10 key program stakeholders, representing the TSE Secretariat and each of the internal Health Canada directorates. These preliminary interviews were intended to gain an understanding of the BSE I and II Initiatives and to explore the extent to which the documents and data identified in the evaluation framework document would be available to the evaluators. Key informants were also asked to suggest possible key informants and case studies.
  2. Review and assessment of available documents. PRA reviewed numerous documents related to BSE/TSE provided by Health Canada to determine their usefulness to the evaluation, and conducted an Internet search to locate additional relevant documents. Some key informants identified and provided additional documentation to PRA for review.
  3. Review and assessment of administrative data. PRA reviewed the extent to which performance measurement and other administrative data would be available and assessed the usefulness and relevance of these data to the evaluation.

The evaluability assessment found that performance measurement data were severely limited, necessitating an alternative approach to generating outcome information. Accordingly, a survey of industry stakeholders, which was not part of Health Canada's original evaluation design, was introduced as a primary means of filling the gap. Overall, the evaluation methodology consisted of six data collection methods:

  • literature review
  • document review
  • administrative data review
  • case study
  • survey of industry
  • key informant interviews

Each of these methods is described in detail below.

3.2.1 Literature review

The literature review focused on examining the science base of the BSE I and BSE II Initiatives and outlined available evidence related to the following evaluation questions:

  • Is there a continued need for the BSE I and BSE II Initiatives?
  • To what extent are there internationally harmonized standards and regulations addressing BSE/TSE and related risks?
  • To what extent are food and health products safer?
  • Are there alternate ways to deliver the BSE I and BSE II Initiatives to achieve similar results at lower cost?

The literature review gathered information from both peer-reviewed (scientific and other academic) journals and grey literature, such as industry journals, newspapers, magazines, and websites. The review involved four key stages: 1) establishing the scope of the review; 2) conducting the search; 3) organizing and classifying research materials and drawing out key insights; and 4) reporting on the literature review findings.

The scope of the literature review (i.e., the specific evaluation questions to be addressed) was established during the evaluation design phase. The second stage of the review involved conducting a comprehensive search of the literature. Key journals in the field were scanned and Google Scholar (http://scholar.google.com) was used to conduct keyword searches of the academic and technical literature, using a list of search terms developed by PRA and approved by Health Canada. Abstracts of the key articles and reports were reviewed to determine their relevance, and their bibliographies were used to identify additional material for review.

In the third stage, the material collected was organized and additional key insights were identified. Two valuable tools used during this stage were Zotero, an open-source software package facilitating the collection, organization, and retrieval of research materials, and NVivo, a commercial software application that constitutes the current industry standard for qualitative data analysis. Documents were reviewed and coded in NVivo, which was ultimately used to group information presented in the literature by theme and to identify important trends.

A literature review report was prepared as a stand-alone deliverable, and key findings from the literature were integrated with those from the document review and the administrative data review to produce the first technical report on findings, which was submitted to Health Canada as an interim deliverable.

3.2.2 Document review

The document review provided important historical and contextual information for the BSE I and II Initiatives and addressed virtually all of the evaluation questions, including:

  • Is there a continued need for the BSE I and BSE II Initiatives?
  • Does the program align with the priorities of the federal government?
  • Does the program align with federal roles and responsibilities?
  • Are the program's governance structures likely to support the achievement of expected outcomes?
  • Has the program been designed and implemented appropriately to achieve expected outcomes?
  • To what extent has progress towards expected outcomes (immediate, intermediate, and ultimate) been achieved?
  • Are processes for allocating staff to activities clearly documented, streamlined, and well-understood by all staff?
  • Were the outputs of sufficient quality to achieve the immediate outcomes?

Most of the documents reviewed were provided by Health Canada, although some additional publicly available documents were used where further information was needed. Findings from the document review were integrated with those from the literature review and the administrative data review in the first technical report of findings.

3.2.3 Administrative data review

Administrative data were expected to provide information to support conclusions on the evaluation questions related to program effectiveness and efficiency/economy. However, with two exceptions (data on voluntary changes made by industry, provided by BGTD, as well as financial information on planned and actual spending), the evaluation did not receive any administrative data.

3.2.4 Case study

One in-depth case study was conducted of a firm in an industry affected by Health Canada's BSE regulatory framework. The firm, a meat processor, was selected by Health Canada. The case study addressed the following evaluation questions:

  • To what extent is there improved regulation/policy response to control and prevent risks associated with BSE/TSE?
  • To what extent is there increased awareness and understanding of BSE/TSE risk control efforts, regulations, and policies among partners and stakeholders?
  • To what extent is there increased adherence to acts, regulations, and other guidance documents?

The information for the case study was gathered through a key informant interview with a representative of the firm and a document review of publicly available sources. A case study report was prepared, reviewed by the participant, and revised based on feedback received.

3.2.5 Industry survey

The industry survey targeted companies in industries regulated by Health Canada, as well as industry associations. The main purpose of the survey was to generate information to support conclusions on the extent to which the BSE I and BSE II Initiatives have achieved their expected outcomes. The survey was also intended to provide an opportunity for a more extensive range and number of stakeholders to provide input into the evaluation than relying on key informants alone.

The survey addressed the following evaluation questions:

  • To what extent is there improved regulation/policy response to control and prevent risks associated with BSE/TSE (transmissible spongiform encephalopathy)?
  • To what extent is there increased awareness and understanding of BSE/TSE risk control efforts, regulations, and policies among partners and stakeholders?
  • To what extent is there increased adherence to acts, regulations, and other guidance documents?
  • To what extent is there increased expertise and knowledge of BSE/TSE science, risk, and surveillance?
  • To what extent is there reduced exposure to the risks associated with the use of animal-sourced materials in food and products regulated by Health Canada?
  • To what extent is there increased knowledge-based decision-making?
  • To what extent is there reduced risk of acquiring human TSEs associated with animal-sourced ingredients in food and products regulated by Health Canada?
  • To what extent are food and health products safer?
  • Were there any unintended consequences, either positive or negative, of the program?

The survey questionnaire was designed by PRA in consultation with the Departmental Performance Measurement and Evaluation Directorate (DPMED). Internal Health Canada program representatives were given an opportunity to provide input on a draft version of the questionnaire and their feedback was incorporated into the final version.

Sample development

The survey sample was developed by PRA, in consultation with Health Canada. The survey sample could only be drawn from those industries and association that had direct contact with the Department, either through the product submission process or through consultation or communication with Health Canada. As described in more detail below, various sources were used to develop the survey sample.

a. Pharmaceuticals, biologicals and radiopharmaceuticals, and veterinary drugs

Health Canada's Drug Product Database (DPD) was used as the source of firms in the pharmaceutical, biological/radiopharmaceutical, and veterinary drug industries.Footnote 5 The DPD contains information on all firms licensed to produce, import, or distribute these products for sale in Canada. To derive the survey sample, PRA downloaded a file from Health Canada's website, which contained the names and addresses of all firms currently licensed to sell these products in Canada. This produced a list of 1,053 individual firms. PRA then eliminated firms based outside of Canada from the sample, leaving a total of 655.

As no contact information was available for the firms in the DPD, PRA conducted an Internet search to locate telephone numbers for each company. Telephone numbers were located for 608 companies.

The Therapeutic Products Directorate, which administers the DPD, distributed an initial communication to its stakeholders, advising them that the evaluation was taking place and that PRA may contact them to identify an individual within their organization to complete the survey and obtain contact information (telephone number and email address). Following the initial communication, PRA contacted each of the 608 companies by telephone. Email addresses were secured for 296 companies. These 296 companies (or 45% of all companies based in Canada in the DPD database) were included in the survey sample.

It is important to note that the original dataset downloaded by PRA from Health Canada's website did not identify the specific industry or industries with which firms were involved, so it was not possible to know how many firms in the original dataset or in the final DPD sample were active in each industry.

b. Natural health products

Health Canada's Licensed Natural Health Product Database (LNHPD) was used as the source of firms in the natural health products industry. The LNHPD contains information on all firms licensed to produce, import, or distribute these products for sale in Canada. To derive the survey sample, PRA used a similar process as that used with the DPD. A file was downloaded from Health Canada's website containing the names and addresses of all firms currently licensed to sell these products in Canada. This produced a list of 1,532 firms. Eliminating firms based outside of Canada left a total of 1,084 in the sample.

As contact information was not available for firms in the LNHPD, PRA conducted an Internet search to locate telephone numbers for each company. Telephone numbers were located for 828 companies.

The Natural Health Products Directorate, which administers the LNHPD, distributed an initial communication to its stakeholders, advising them that the evaluation was taking place and that PRA may contact them to identify an individual within their organization to complete the survey and obtain contact information (telephone number and email address). Following the initial communication, PRA contacted each of the 828 companies by telephone. Email addresses were secured for 304 companies. These 304 companies (or 28% of all Canadian-based companies in the LNHPD) were included in the survey sample.

c. Medical devices

Health Canada's Medical Devices Active License Listing (MDALL) was used as the source of firms in the medical devices industry. The MDALL contains information on all firms licensed to produce, import, or distribute these products for sale in Canada. In this case, Health Canada provided PRA with an Excel spreadsheet containing the names of all firms based in Canada that are currently licensed to sell medical devices in Canada, along with contact names, telephone numbers, and email addresses. Health Canada also sent the initial communication to its stakeholders, advising them that the survey would be conducted.

A list of 413 firms was provided to PRA. Of these, 404 (98% of all Canadian-based firms in the MDALL) had email addresses and were included in the survey sample.

d. Livestock producers and food manufacturers/processors

Unlike drug products, natural health products, and medical devices, a comprehensive database containing livestock producers and food manufacturers/processors was not available. Accordingly, Health Canada's SIMS database was used to derive this component of the survey sample. The SIMS database is a list of organizations and individuals that have been engaged with Health Canada. Health Canada provided PRA with an extract from the SIMS database containing over 5,000 contacts. Of these, just over 1,300 were categorized as "industry" contacts; in some cases, the database included multiple contacts for a single organization. To identify appropriate producers and firms for inclusion in the sample, PRA applied the following rules:

  • Producers of ruminant livestock were included; producers of other types of livestock were excluded.
  • Firms involved in meat processing/packing were included, as were producers of dairy products; firms involved in the manufacturing or processing of other types of food products were excluded.
  • Each organization (producer or business) was included only once.

In many cases, making an appropriate determination regarding whether or not to include a firm or producer required PRA to research the organization online. Through this process, 118 livestock producers and food manufacturers/processors were identified for inclusion in the survey. Health Canada circulated an initial email to these organizations, although email addresses were available in SIMS for only 79 of the firms in the sample. PRA followed up on the initial communication by phoning all of the firms to verify contact information. In total, email addresses were secured for 89 producers and firms, and these were included in the survey sample.

6.Industry associations

The SIMS database was used to generate a list of relevant industry associations, supplemented by a small number of additional industry associations identified by Health Canada. The industry associations received an initial email communication from Health Canada, which was followed by a telephone call by PRA to verify contact information. In total, 154 industry associations were identified, and email addresses were secured for 146 associations. These associations were included in the sample.

Final sample

Because many firms (and email addresses) were present in two or more of the source databases, the final step in developing the sample was to merge the respondent lists derived from the four source databases, with a view to eliminating duplicate or multiple appearances of the same firms and email addresses.Footnote 6 Duplicate and multiple entries were mainly found among the DPD, LNHPD, and MDALL databases; as might be expected, there was little overlap of these databases with SIMS. After this process was completed, the final sample consisted of 1,219 unique email addresses.

Field operations

The survey was programmed online by Nooro, a survey web hosting company based in Ontario. PRA provided Nooro with the survey sample (email addresses only) and a brief introductory email in English and French. Nooro was responsible for emailing the invitation to potential respondents. The invitation contained a link to the survey and a unique ID number that each respondent could use to log in to the survey. Nooro was also responsible for sending out reminder emails, prepared by PRA, in both official languages. Three rounds of reminder emails were sent out.

The survey was launched on October 12, 2011. The first reminder was sent on October 19, the second on October 27, and the final on November 2. The survey closed on November 5.

Response rate and respondent profile

Of the 1, 219 email invitations that were sent out, 0 bounced back; that is, all email addresses included in the sample were valid. This is an exceptional result; typically 5% or more email addresses within a sample bounce back.

The survey achieved 191 responses, for a response rate of 15.7% (the actual response rate may have been higher, since an unknown percentage of the email invitations were likely diverted by spam filters). Of these, 117 respondents completed the entire survey, producing a completion rate of 9.6%. This is in-line with online industry surveys in PRA's experience, which typically achieve completion rates of approximately 10%. A total of 74 respondents, or 38.7% of those who responded to the survey, did not complete the entire survey.

Table 7 below compares respondents who completed the survey with those who did not complete the survey, based on type of organization and product type (self-report data). These data show some differences between the two groups based on product type, with proportionally more representation of organizations involved in the natural health products and dairy products sectors among respondents who did not complete their surveys, as compared to those who did. The reverse is true for organizations involved with medical devices, pharmaceuticals, disinfectant products, livestock, and biologicals/radiopharmaceuticals.

Table 7: Overview of respondents
Organizations and Product Types Completes Incompletes

Type of organization

n=117

%

n=74

%

A company

99

85%

52

70%

An industry association

12

10%

5

7%

A company and an industry association

6

5%

2

3%

No response

--

--

15

20%

Product type

Medical devices

40

34%

16

22%

Natural health products

35

30%

27

37%

Pharmaceutical drugs

23

20%

8

11%

Cosmetics and/or personal care products

20

17%

12

16%

Disinfectant products

19

16%

7

10%

Livestock

18

15%

3

4%

Meat products

12

10%

8

11%

Biologicals and/or radiopharmaceuticals

11

9%

2

3%

Veterinary drugs

5

4%

2

3%

Dairy products

4

3%

6

8%

General cleaning products

3

3%

2

3%

Industrial supplies

2

2%

--

--

Other

11

9%

6

8%

Note: For type of organization, totals may not sum to 100% due to rounding. For product type, totals do not sum to 100% due to multiple responses.

Survey programming and analysis

The survey results were analysed using SPSS, a statistical software package commonly used in social science research. Overall frequencies and (where appropriate) frequencies pertaining to a subset of survey respondents were produced.

In addition, cross-tabulations comparing respondents who self-identified as being involved only in the food industry (i.e., livestock, meat products, dairy products) (n=21) with respondents who self-identified as being involved in any of the other industries (n=96) were performed for select questions (questions 7, 8, 9, 10, 12, and 15). Statistically significant differences were found only in response to question 7 and are included in this report; nonetheless caution should be used due to small sample size. Further cross-tabulations by industry/product type were not performed due to the small size of the sample and the fact that many respondents were involved in more than one type of industry (for example, pharmaceutical drugs and natural health products and cosmetics) and could not be categorized as belonging to only one industry for the purpose of cross-tabulations.

3.2.6 Key informant interviews

The key informant interviews were completed in two rounds. During the evaluation planning phase, 10 key BSE program personnel, representing the TSE Secretariat and each of the internal Health Canada directorates, were interviewed to provide an understanding of the activities of the BSE I and II Initiatives and to explore the extent to which the documents and data identified in the evaluation framework document would be available to the evaluation.

In the second round, key informant interviews were completed with Health Canada's federal partners, i.e., representatives of the CFIA, Agriculture and Agri-food Canada, and the Public Health Agency of Canada (n=5) and other external stakeholders (n=5). These interviews sought the perspectives of key informants with respect to relevance and performance.

In addition, a second round of interviews was completed with BSE program personnel (n=16) representing the funded directorates and the TSE Secretariat. The purpose of these interviews was to give program personnel an opportunity to respond to key findings from the other lines of evidence. As such, the interview guide included short summaries of the key evaluation findings as a preface to the interview questions.

All interviews were digitally recorded with the permission of key informants. Interview notes were prepared and provided to key informants for review and revision. This step was intended to enhance the validity and reliability of the data collected. However, some key informants chose not to review the notes from their interview, and some of those who received the notes did not respond within the time provided (generally at least one week).

Approach to data analysis

Data from all lines of evidence were integrated or triangulated in order to arrive at the overall evaluation findings. Triangulation is a process through which answers to research questions generated by different data collection methods are compared. Where different methods produced similar findings, those findings were assumed to have greater validity and therefore greater confidence in the results is warranted. While findings generated by a single method were treated with caution, it is also important to note that in some cases only a single line of evidence was necessary in order to arrive at a valid conclusion. For example, answering the evaluation questions related to alignment of the BSE Initiatives with government priorities and federal responsibilities relied solely on document review, since the use of additional methods (such as key informant interviews) would not have strengthened the evidence in this case.

Review of preliminary findings by internal Health Canada personnel was an important aspect of the data analysis process and was also used to strengthen the accuracy and validity of the findings. Program personnel had an opportunity to review and comment on the technical report of the document, data, and literature review and the technical case study report. In addition, during the second round of interviews, internal Health Canada interviewees were given an opportunity to respond to the preliminary findings, providing alternate interpretations and/or additional data. This process not only added nuance to the interpretation of the data but also helped to validate the evaluation findings.

Limitations of the methodology and mitigation strategies

There are several limitations of the evaluation methodology.

Limitation 1: One limitation stems from the nature of the documents available for review. Many types of documents that are typically reviewed in evaluations, including Terms of Reference, records of actions taken, work plans/operational plans, and performance reports, were unavailable to the evaluation, despite formal requests for such documentation through the preliminary interviews conducted with key Health Canada partners. In addition, although some information on actual spending was provided to the evaluation, there are many caveats associated with these data which limit their usefulness. As a result, the information needed to address certain questions was limited. In particular, it was difficult to establish what activities have been undertaken with BSE funding and whether implementation of the Initiatives occurred as planned.

Mitigation: To the extent possible, key informant interviews were used to fill these gaps.

Limitation 2: Another limitation is the lack of performance measurement and administrative data to support analysis of the extent to which the BSE I and BSE II Initiatives have achieved their expected outcomes.

Mitigation: The survey of industry was introduced to address this gap, and provided some evidence related to outcomes achieved. However, the survey could not entirely compensate for the absence of performance measurement data.

Limitation 3: Although the industry survey was introduced to address gaps in data, it too has limitations. The most significant are related to the way in which the sample for the various industries was derived. Although a "census" approach that surveyed all firms in all relevant industries would have been ideal, such an approach was not possible due to differences in the nature of the contact information that was made available to the evaluation, as described in detail above. As a result of these differences, the medical devices industry is overrepresented in the sample because email addresses were provided to the evaluator by Health Canada and were available for virtually all firms in the sample. Conversely, industries represented in the DPD and LNHPD databases are underrepresented, since contact information for firms in these databases had to be located by the evaluator. The food industry is also underrepresented in the sample, since there was no comprehensive database of firms in the food industry, comparable to the DPD, LNHPD, or MDALL, available to the evaluation. Moreover, due in large part to the fact that a census approach could not be used for all industries, the survey did not achieve a sufficient number of completions to enable comparisons by industry or product type, as described above.

Furthermore, the survey sample consists of only Canadian-based firms. Although foreign-based firms are also subject to Health Canada's regulatory framework, it was not feasible due to budgetary and time constraints to include them in the survey, given the significant amount of resources that were required for sample development (i.e., locating and/or verifying contact information). As a result, the survey results do not reflect input from that stakeholder group.

Despite best efforts to target the survey to firms and associations affected by Health Canada's BSE regulatory framework, some respondents may have felt that the survey or specific questions within it were not applicable to them and either elected not to participate or to drop out partway through. Alternatively, the survey as a whole may have been perceived as too long for some respondents. However, the average time to complete the survey was 10 minutes, which would not normally be considered a lengthy survey.

Finally, the survey, as well as the key informant interviews and the case study, are limited by self-selection bias and possible strategic response bias.

Mitigation: The use of multiple lines of evidence and triangulation of findings from various methods was the main strategy for mitigating the limitations associated with the survey, the key informant interviews, and the case study.

Finally, it is important to note that it is extremely difficult to evaluate the impact of risk mitigation or reduction measures such as Health Canada's BSE Initiatives. One reason is the fact that the desired outcome of a risk mitigation measure, unlike most programs, is the absence, rather than the presence, of an observable event. The second reason is the absence of a counterfactual. That is, a scenario in which Health Canada's BSE/TSE risk management measures were not in place was unavailable to the evaluation as a point of comparison. As a result, conclusions on the extent to which risk mitigation or reduction measures are successful at reducing risk must necessarily rely on indirect or qualitative evidence.

4.0 Findings

This section of the report presents the evaluation findings, based on all lines of evidence. The findings are organized according to the evaluation issues.

4.1 Relevance

Overall, the evaluation confirmed an ongoing need for government intervention to manage the public health risks associated with BSE/TSEs, and found the program to align with federal government priorities and responsibilities.

4.1.1 Ongoing need for the program

Evaluation Question:
• Is there a continued need for the BSE I and BSE II Initiatives?
Indicators:
• Need for initiatives identified/documented.

• Expert/stakeholder assessment of ongoing need.
Rating:
• Achieved.
Summary:
• Based on scientific literature and expert and stakeholder assessments, the evaluation confirmed an ongoing need for intervention, including intervention on the part of Health Canada, to manage the risks to human health associated with BSE/TSEs.

The evaluation confirmed an ongoing need for intervention, including intervention on the part of Health Canada, to manage the risks to human health associated with BSE/TSEs. While Canada's prevention and eradication methods are acknowledged as effective by the OIE, and Canada is recognized by the OIE as a "controlled BSE risk" country (OIE, 2011), it has not achieved "negligible BSE risk" status.Footnote 7 Furthermore, while some of the recent literature and government documents on the subject suggest that primary transmission of vCJD through consumption of BSE-infected beef is now less of a public health concern in many jurisdictions than in the past (European Union [EU], 2005, p. 16; Brown, 2010), it is also clear from the literature and from government documentation in several jurisdictions that other potential channels of TSE transmission are perceived as serious threats to public health. Furthermore, there are many features of TSEs, including BSE, that are not well understood and many areas of uncertainty with potential public health implications. For example:

  • In a recent joint opinion, the European Food Safety Authority (EFSA) and the European Centre for Disease Prevention and Control noted that some data indicate that one of the new atypical BSE agents, the L-BSE or BASE agentFootnote 8, may have a similar or higher zoonotic potential than the Classical BSE agent (EFSA Panel on Biological Hazards, 2011). The opinion also recommended that systematic monitoring of TSE diseases be continued in both humans and animals, in view of a number of ongoing scientific uncertainties with respect to the possibility of animal-to-human transfer.
  • Although there have been no cases of human prion disease associated with Chronic Wasting Disease (CWD) (a type of TSE affecting cervid species such as deer, elk, and moose) to date (James, 2008, p. 7; Ludlam & Turner, 2006, p. 14), the literature strongly encourages a precautionary approachFootnote 9 in managing the potential public health risks (Belay et al., 2004, p. 983; Leiss et al., 2010, p. 386; Wang & Coulthart, 2011, p. 14; WHO, 2002). This is particularly significant as a recent experimental study suggested CWD has the potential to transfer to humans (Aiken, 2011, p. 13), and because risk tolerance for human CWD is very low (Wang & Coulthart, 2011, p. 14). The potential social, environmental, and health implications of CWD for First Nations, Inuit, and Northern populations in Canada -- some of whom have a longstanding tradition of consumption and use of moose, elk, and deer -- has also been raised in the literature (McLachlan, 2011).
  • It is now considered likely that vCJD can be transmitted from person to person through blood and blood products, with four probable cases of vCJD in the UK traced to contaminated blood products (Farrugia, Ironside, & Giangrande, 2005; Ironside, 2010, p. 177; Turner & Ludlam, 2009, p. 144). In addition, many writers have expressed concerns about the possibility of transmission through contaminated medical or dental instruments, although there is no evidence that such transmission has ever actually occurred (Ward & Knight, 2009, p. 130). Preventing person-to-person transmission of vCJD is difficult because of the known resilience of prions to conventional decontamination procedures (Bradley, Collee, & Liberski, 2006, p. 94; Pauli, 2005, p. 195; Sutton, Dickinson, Walker, Raven, & Weinstein, 2006), and the absence of a reliable vCJD screening test (Bailey, 2006; Graziano & Pocchiari, 2009, p. 425; Ironside, 2010, p. 179; NHS Blood and Transplant, 2008), although researchers may recently have solved the latter issue (Adams, 2011; Edgeworth et al., 2011).
  • A large number of pharmaceuticals, biologics, vaccines, natural health products, and cosmetics contain animal-derived materials. For example, bovine-sourced ingredients in pharmaceuticals and biologics include milk and milk derivatives; meat extracts; bovine serum including fetal bovine serum; bovine bone gelatin; bile derivatives; and beef tallow derivatives (WHO, 2006, p. 10), while cosmetics may be manufactured using such materials as albumin, brain extract, brain lipid, cholesterol, fibronectin, sphingolipids, collagen, keratin, and tallow and tallow derivatives (U.S. Food and Drug Administration [US FDA], 2004a, p. 42191). Similarly, many natural health products contain gelatin (used in making soft-gel capsules and dry tablets), and glandulars (Colloton, 2001, pp. 502-505).
  • It is known that people with certain genotypes are more susceptible to vCJD than others, as evidenced by the fact that nearly all cases to date were homozygousFootnote 10 for methionine at codon 129 (Stevenson, Oakley, Chick, & Chalkidou, 2008). However, there are also rare known instances of infection in the other genotypes (i.e., heterozygous at codon 129 - methionine/valine, and homozygous for valine) (Sutton et al., 2006, p. 758; WHO, 2006, pp. 5-6). Some researchers believe that there are many people infected with the disease who are as yet asymptomatic (Collee, Bradley, & Liberski, 2006; Wiggins, 2009). Not only will some of these individuals ultimately be afflicted with vCJD, but in the meanwhile they could pose a serious health risk to others, for example through blood, tissue, or organ donation (Azarpazhooh & Fillery, 2008, p. 1161; Hilton, 2006; Pauli, 2005, p. 195; Sutton et al., 2006, p. 758).
  • BSE may develop spontaneously in cattle, just as sporadic CJD develops spontaneously in humans; if this is so, it may not be possible to eradicate the disease in cattle, requiring ongoing management of the risk of transmission to humans (Budka, 2008, p. 8).
  • There is presently no treatment, cure, or vaccine for vCJD, which is therefore invariably fatal once contracted (Ackerman & Johnecheck, 2008, p. 146; Blanchfield, 2009). There is also an ongoing need for manufacturing techniques capable of reducing infectivity in blood, blood components, and plasma-derived products, as well as new procedures for decontaminating materials and devices (WHO, 2006, p. 15). Until recently, the literature also suggested an urgent need for a test capable of diagnosing vCJD prior to emergence of symptoms or death, although a solution to this problem may now have been found (Edgeworth et al., 2011).

Broadly speaking, the findings from the literature with respect to knowledge gaps and research needs relating to TSEs suggest that there remain many unknowns and uncertainties with important future implications for public health, and that research is needed to help develop new technologies, processes, and treatments. Together, these findings suggest an ongoing need for government intervention to manage the public health risks associated with TSEs. Similarly, external key informants generally agreed that Health Canada should continue to be active in the area of BSE/TSEs, in light of the many scientific uncertainties that remain, especially with respect to human-to-human transmission; atypical BSE; and TSEs other than BSE, such as CWD.Footnote 11 That being said, some questions were raised by key informants about the overall level of funding for BSE/TSE-related activities and the allocation of this funding among federal departments and within Health Canada. This issue is discussed in more detail in section 4.2.6.

4.1.2 Alignment with government priorities and federal responsibilities

Evaluation Questions:
• Do the BSE I and BSE II Initiatives align with government priorities?

• Do the BSE I and BSE II Initiatives align with federal roles and responsibilities?
Indicators:
• Extent to which program objectives are linked to Government priorities.

• Extent to which the Initiatives' objectives are linked to the strategic outcomes of Health Canada/priorities of HPFB.

• Extent to which the Initiatives' objectives are consistent with the legislative framework of the Federal government.

• Extent to which the Initiatives' objectives are consistent with the legislative framework of Health Canada.
Rating:
• Achieved.
Summary:
• The evaluation found that the BSE I and BSE II Initiatives align with government priorities and federal responsibilities.

The evaluation found that Health Canada's BSE I and BSE II Initiatives align with government priorities and federal responsibilities. Although the three most recent Speeches from the Throne (2009, 2010, and 2011) did not mention BSE or TSEs specifically, the Speech from March 3, 2010 indicated that food and health product safety was a priority for the federal government. The government vowed to "reintroduce legislation to protect Canadian families from unsafe food, drug and consumer products" (Government of Canada, 2010) and strengthen Canada's food safety system. Thus, although there is no Speech material specifically devoted to BSE or TSEs, the BSE Initiatives relate closely to the federal priorities identified in this part of the 2010 Speech.

Furthermore, from the funding plans obtained in government documents, it is clear that the federal government plans to maintain its commitment to addressing BSE issues in Canada. In 2009, funding was extended in the amount of $45.7 million per year over five years ($228.5 million in total). This funding covers some major components of the BSE strategy. CFIA will receive $38.7 million of the annual funding, whereas Health Canada will receive $6.2 million and PHAC will receive $0.8 million.

The evaluation found that the Initiatives align with two of Health Canada's strategic outcomes. The BSE Initiatives align most closely with the Department's strategic outcome of Access to Safe and Effective Health Products and Food and Information for Healthy Choices. As per its Program Activity Architecture (PAA), the two activities that support this strategic outcome are Health Products, and Food and Nutrition. The goals of the Health Products activity are to increase the regulatory response to the risks of health products, and raise awareness of health product issues. For Food and Nutrition, the expected results are to lower exposure to food-borne pathogens and chemical contaminants, and improve the level of informed decision-making regarding the health and safety of food products. Thus, the BSE Initiatives align closely with this strategic outcome.

The BSE Initiatives also relate to the strategic outcome of Reduced Health and Environmental Risks from Products and Substances, and Healthy, Sustainable Living and Working Environments, though perhaps only loosely. Under this strategic outcome, Health Canada uses evidence-based research to develop policies, programs, and regulations to promote healthy and safe living. The program activities associated with this outcome involve air quality, drinking water safety, chemicals, substances, tobacco, consumer product safety (such as cosmetics), radiation exposure, and pest control products. The activities of BSE I align with this strategic outcome, as one of its major activities involves compliance and enforcement in health, food, and consumer products.

Similarly, the evaluation found that the BSE I and BSE II Initiatives align with federal roles and responsibilities, as articulated in the Food and Drugs Act and Regulations (1985)and the Department of Health Act (1996).The Food and Drugs Act requires that no person sell a food item that contains harmful substances or is unfit for human consumption (Government of Canada, 2011b, p. 4), and specifies that the federal government has a regulatory role related to food, drugs, cosmetics, and devices that extends to labelling and packaging; size, dimensions, and specifications; standards of composition and purity; the conditions of sale; and the use of any substances as ingredients (p. 13). Health Canada's pre-market and compliance and enforcement activities under the BSE Initiatives closely align with federal responsibilities under the Act with respect to health products.Footnote 12

The BSE Initiatives also align with the Department of Health Act (1996). Under the Act, the Minister's duties involve "all matters over which Parliament has jurisdiction relating to the promotion and preservation of the health of the people of Canada not by law assigned to any other department, board, or agency of the Government of Canada" (Government of Canada, 2011c, p. 1). Such roles the Department (now known as Health Canada) is required to fulfill include promoting the physical, mental, and social well-being of people in Canada; protecting them against health risks and disease; conducting investigations and research in public health, including monitoring diseases; establishing consumer product safety standards; and collecting and distributing health-related information (p. 2). These roles align with the objectives of the BSE Initiatives involving risk prevention, BSE/TSE research, and knowledge development; and safeguarding the health of Canadians. In addition, the Act gives authority to Health Canada to set and enforce regulations regarding the objects of the Act, and contravening the regulations is an offence "punishable on summary conviction" (p. 5), aligning with the compliance and enforcement aspects of the BSE Initiatives.

4.2 Performance

Overall, the evaluation found that although implementation of the BSE I and BSE II Initiatives did not occur as planned, some progress has been made toward achieving the expected outcomes.

4.2.1 Program governance

Evaluation Question:
• Are the governance structures likely to support the achievement of expected outcomes?
Indicators:
• Extent to which internal and interdepartmental partners' roles, responsibilities, accountabilities, and decision-making authorities are documented and understood.

• Existence/use of forums for ongoing communications among internal and interdepartmental partners.

• Extent of collaboration among internal and interdepartmental partners.

• Existence of performance measurement frameworks.

Extent to which performance data are collected and used to support decision making.
Rating:
• Attention needed.
Summary:
• The governance structure for BSE I and BSE II had several weaknesses that likely affected program implementation, including limited collaboration and coordination among partners, the absence of a coherent profile for the TSE Secretariat whose role was to coordinate Health Canada's overall approach to BSE/TSE, and failure to measure and report on performance.

Based on the evidence available to the evaluation, governance of BSE I and BSE II had several weaknesses that likely affected program implementation. The governance structure for the federal government's overall response to BSE/TSE had its origins in 2001. In February of that year, Health Canada organized a two-day Expert Consultation on Prion-Related Diseases, with the objectives of reviewing current understanding of the BSE/vCJD epidemics in Europe and approaches used to control the spread of the disease there; to identify research needs; and to identify and prioritize issues of particular concern relating to the science, control, and prevention of TSEs (Health Canada, 2001a, pp. 1-2).

Following the consultation, Health Canada created a TSE Action Plan (Health Canada, 2002a), setting out the decision-making process by which Health Canada would respond to TSE-related public health issues and risks.

Several observations can be made about program governance based on the evaluation evidence.

1) There is limited evidence of coordination and collaboration among partners

In a December 2000 report, the Auditor General of Canada emphasized coordination among partners as an essential element of horizontal initiatives (Auditor General of Canada, 2000). As horizontal initiatives involving multiple internal Health Canada partners as well as external federal partners, the BSE I and BSE II Initiatives should have, by this standard, involved a degree of coordination and collaboration among partners. However, the evaluation found limited evidence that such coordination occurred.

There was, for example, little evidence of activity on the part of various inter- and intra-departmental committees established to oversee the federal government's response to BSE/TSE. Although Deputy Minister (DM) and Assistant Deputy Minister (ADM) Interagency Advisory Committees on TSEs -- chaired by Health Canada and including representation from the CFIA, AAFC, Environment Canada, Foreign Affairs and International Trade, and the Privy Council Office -- were established in 2001, the evaluation found little documented evidence of activity, such as meeting agendas or minutes, on the part of these committees.Footnote 13 Similarly, within Health Canada, although an HPFB Director General BSE Steering Committee and an HPFB Working Group on BSE were established, the evaluation found virtually no documentation of their work. See Table 8 for a summary of these committee activities.

Health Canada's federal partners reported that although formal mechanisms such as committees and working groups were active in the early days of the Initiatives, they have since been replaced by informal mechanisms and individual, one-on-one relationships -- a situation that was satisfactory for some of these key informants, but not for others.

Health Canada key informants involved in the Initiatives generally acknowledged that collaboration and coordination among internal partners was not as extensive as it could have been, although some noted that in the early days of the Initiatives, internal partners did meet frequently through the meetings of the TSE Science and Policy teams, as discussed in more detail below.
Table 8: Governance of the BSE I and BSE II Initiatives: summary of committee activity
Committee/working group name Summary of activity

Deputy Minister Interagency Advisory Committee on TSEs

The document review did not uncover Terms of Reference, minutes, or any other documentation associated with the DM Committee and it is not clear if, or for how long, the committee was active.

Assistant Deputy Minister Interagency Advisory Committee on TSEs

The draft Terms of Reference for the ADM Committee indicate that it would meet "as required" (Health Canada, 2001b). The document review found minutes from what appears to be the first meeting of the ADM Committee in May 2001 (Health Canada, 2001c); it is not clear if subsequent meetings were held.

HPFB Director General BSE Steering Committee

This committee was mentioned in the Management Action Plan of the formative evaluation (Health Canada, 2007a). However, the document review did not uncover Terms of Reference, minutes, or any other documentation associated with this committee and it is not clear when it was established and how long, if at all, it was ever active.

HPFB Working Group on BSE

This committee was mentioned in the draft Issue Analysis: Bovine Spongiform Encephalopathy and Health Products (Health Canada, 2005a). The document reported on Health Canada's need to develop a mitigation strategy for the risk of BSE transmitted through health products. Health Canada consulted with the HPFB Working Group on BSE on their analysis of options for risk mitigation. The Group consisted of members from various directorates. The HPFB Working Group was responsible for bringing the process to the attention of their management for approval, and recommended the strategy of using risk reduction through case-by-case risk assessment and management guidelines. The document review did not uncover Terms of Reference, minutes, or any other documentation associated with this working group and it is not clear when it was established and how long, if at all, it was ever active.

2) The TSE Science and Policy Teams do not appear to have played the key role in decision-making envisioned for them

Although multidisciplinary TSE Science and Policy Teams were established by the TSE Action Plan to play a key role in Health Canada's decision-making process with respect to BSE/TSEs, the extent to which they fulfilled that key role is not clear. Both teams had a series of "subgroups" to provide assessments of science and policy related to certain issues. The responsibilities of the TSE Science Team, which consisted of Health Canada and CFIA personnel, were to review the risk assessments conducted by its subgroups, review and revise priorities, and provide advice to the Policy Team and File Champions on science issues, including research needs (Health Canada, 2002a, p. 15). The Science subgroups were also to report on scientific developments in TSE research and advise on the type of research needed to address specific issues. The responsibilities of the TSE Policy Team, by contrast, were to review the risk assessments conducted by their subgroups; review issue identification documents; review policy analysis documents; review and revise priorities; and provide advice to the Science Team and File Champions on policy issues (Health Canada, 2002a, pp. 15-16).

Health Canada key informants reported that regular (biweekly) meetings of the TSE Science and Policy Teams were organized by the TSE Secretariat between 2001 and 2007.Footnote 14 Based on key informant accounts, these meetings primarily were concerned with information-sharing among the partners. Some key informants reported that the effectiveness of the meetings and the teams was hampered by a lack of consistency in, and the relative inexperience of, some of the participants over time. After 2007, the meetings of the TSE Science and Policy Teams reportedly became ad hoc and eventually were discontinued altogether. Internal key informants reported that as Health Canada's understanding of BSE/TSE science and policy responses improved, there was no longer a need for these meetings.

3) The TSE Secretariat has lacked a coherent profile within Health Canada

The evaluation evidence suggests that the TSE Secretariat has lacked a coherent profile within Health Canada and that its current role is unclear. The Secretariat was created within the Food Directorate in 2001 to provide Secretariat support to the ADM Committee and specifically, "to coordinate horizontally all issues related to TSE/BSE (Food Safety) within Health Canada" (Health Canada, 2001c). However, the roles and responsibilities of the TSE Secretariat, as articulated in Health Canada documents, have varied over time (see description of roles and responsibilities below). For example, "managing TSE funding, resources, and planning activities" is mentioned as a role of the TSE Secretariat in only one document (Health Canada, n.d.a.), and the evaluation did not uncover any evidence that the Secretariat had any role in resource allocation or management.Footnote 15 These variations could reflect an evolution in the Secretariat's role over time, a lack of clarity regarding the Secretariat's role, or both.

Moreover, the Secretariat has been transferred several times within Health Canada, starting out in the Food Directorate, moving to the Veterinary Drugs Directorate in 2006, and then returning to the Food Directorate in October 2010. Similarly, resourcing for the Secretariat has varied over time. Initially, the Secretariat was resourced for one position by the Food Directorate through the assignment of existing A-base funding (Health Canada, 2007b). The Food Directorate also supplied support and administrative services as needed. In 2003-2004, the Secretariat was funded by the ADM HPFB, and between 2003 and 2009, through the BSE I Initiative (Health Canada, 2007b). In 2006, the Secretariat was moved to the Veterinary Drugs Directorate (VDD). According to a 2006 organizational chart, the TSE Secretariat at that time consisted of the TSE Secretariat Manager, an Administrative Assistant, an Administrative Clerk, a Secretary, a Document Librarian, a Desktop Publisher, and three TSE Coordinators, for Food, Biologics and Therapeutics (Health Canada, 2006a). Of these positions, all were vacant with the exception of the Manager and the Desktop Publisher. Information on current resourcing for the Secretariat was not available, but anecdotal evidence from key informants suggests that the Secretariat today consists of two individuals.

Description of roles and responsibilities of TSE Secretariat in available documents

A document entitled Food Directorate -- Briefing Book (Health Canada, n.d.a.) identifies the responsibilities of the TSE Secretariat as:

  • supporting the DM and ADM Interagency Advisory Committees on TSEs;
  • facilitating and supporting the work of the BSE/TSE Science and Policy Teams within Health Canada;
  • managing TSE funding, resources, and planning activities; and
  • coordinating TSE-related issues and activities with Branches, Departments, and agencies coordinating TSE-related information and its dissemination.
  • A January 2007 presentation on the TSE Secretariat for DMC (Health Canada, 2007b, p. 4) describes the original role of the TSE Secretariat as:
  • providing scientific analysis and documentation, issue management, monitoring, and coordination;
  • facilitating communication between involved Directorates and Branches;
  • ensuring access to timely scientific information, analysis, and
  • expertise playing an instrumental role in the development and evaluation of risk assessments, communications products and guidelines, and other documents concerning BSE/TSE.

The January 2007 presentation on the Secretariat (Health Canada, 2007b, p. 8) describes the Secretariat's role in 2007 as:

  • analysis, assessment, and provision of comprehensive TSE-related information;
  • support to and coordination of multi-disciplinary, inter-agency science and policy teams, and inter-departmental committees;
  • provision of authoritative advice and guidance to senior management and programs;
  • intelligence gathering of national and international TSE policy and regulatory trends and scientific developments; and
  • authoritative departmental representation in national and international TSE missions and conferences provision of consistent and timely ministerial briefing, correspondence and communications products, and support to media relations as well as to other federal departments and provincial governments.

Several key informants, both internal and external to Health Canada, suggested that these changes may have affected the ability of the Secretariat to fulfill its coordinating function. Nevertheless, internal key informants agreed that the Secretariat has been at least partially successful at doing so, particularly in the earlier years of BSE I and BSE II. Based on documentation and key informant accounts, accomplishments of the Secretariat include:

  • coordinating the meetings of the TSE Science and Policy Teams between 2001 and 2007;
  • collaborating with the FD and with various international entities on targeted research projects;
  • disseminating scientific information on BSE/TSE within Health Canada and to external partners via newsletter and more recently via SNARF (a web service);
  • travelling internationally and as a member of the Canadian BSE delegation to present information on completed BSE risk assessments, on the safety of Canadian beef, and on Canada's response to BSE/TSE; and 
  • completing several large risk assessments in collaboration with the FDFootnote 16, and preparing several documents on BSE/TSE science; documents provided to the evaluation are summarized in below.
Documents produced by the TSE Secretariat, 2002 to 2009
  • Chronic Wasting Disease in Cervids: A Human Health Concern? (Health Canada, 2002b) -- A document containing information on Chronic Wasting Disease (CWD) and whether it can be transmitted through deer meat to humans.
  • Chronic Wasting Disease of Deer and Elk: A Canadian Perspective (Health Canada, 2002c) -- A background document on CWD in deer and elk, intended to serve as information for future examination on human health risks.
  • CWD Table of Assumptions and Scientific Facts (Health Canada, 2003b).
  • Transmissible Spongiform Encephalopathies: Developing a Common Understanding -- Scientific Version (Health Canada, 2005b) - This document provides background on TSEs and key issues related to risk assessments and risk mitigation strategies. The document represents the current body of knowledge and will be updated as needed.
  • Transmissible Spongiform Encephalopathies: Developing a Common Understanding -- Layperson Version (Health Canada, 2005c) - A layperson version of the document above.
  • Classical BSE Assumptions and Scientific Facts (Health Canada, 2009) -- Draws on a variety of sources (government and academic) to list a variety of assumptions and facts related to BSE, including those currently undergoing review.

Internal key informants generally agreed that the TSE Secretariat performed an important role in the early years of the government's response to BSE/TSE, particularly in monitoring and disseminating scientific information during what was at the time a rapidly changing field. However, some also questioned whether the Secretariat is still relevant, given the maturation of the field and Health Canada's more sophisticated understanding of BSE/TSE science and policy, including increased capacity at the directorate level to monitor aspects of BSE/TSE science that are specifically relevant to them. The issue of increased expertise and knowledge of BSE/TSE science within Health Canada is discussed in more detail in section 4.2.4.

4) Performance measurement and reporting has not occurred

Finally, the evaluation found that performance measurement and reporting on BSE I and BSE II has not occurred. Performance measurement was first identified as an issue in 2007 by the formative evaluation of the BSE Initiatives (Health Canada, 2007a). That report recommended that the logic model and Performance Measurement (PM) Strategy undergo revisions to ensure that accurate information would be captured for the summative evaluation. While the formative evaluation found the logic model and PM Strategy to be generally acceptable, it stressed the need to clearly define roles and responsibilities regarding data collection, and to begin the data collection process as soon as possible.

In response to these recommendations, Health Canada's Management Response to the formative evaluation identified the TSE Secretariat as playing a key coordinating role with respect to performance measurement, stating that:

  • The TSE Secretariat will continue to coordinate with directorates in reporting on the accomplishments and achievements of the BSE-funded directorates to the HPFB DG BSE Steering Committee, Departmental Performance Report, Report on Programs and Priorities, HPFB Annual Report and Treasury Board Secretariat, when required. (Health Canada, 2007a, p. 3)

The Management Response also noted that "roles and responsibilities for collection of data for the funded directorates, including the TSE Secretariat...will be defined in the revised PM Strategy which is expected to be completed and implemented by June 30, 2007" (Health Canada, 2007a, p. 4).

The Management Response identified the VDD, within which the TSE Secretariat was housed between 2006 and October 2010 (when it returned to the FD), as the lead directorate for implementing both of the above actions. However, as shown in Table 3, within Health Canada PPIAD was to be responsible for evaluation, and in fact only PPIAD received BSE funding for this purpose. Within the documentation, there is therefore some ambiguity regarding responsibilities for performance monitoring, reporting, and evaluation activities. Certainly, Health Canada key informants had different perspectives regarding where the responsibility for these activities resided.

As a result of this lack of clarity, although performance measurement frameworks were developed, they were never actually implemented, i.e., the performance data identified in the frameworks were never collected, monitored, reported on, or used to support decision-making.Footnote 17 As a consequence, performance measurement or administrative data pertaining to outcomes is virtually non-existent, with repercussions for the ability of the evaluation to draw conclusions on the extent to which outcomes have been achieved.

4.2.2 Program design

Evaluation Question:
• Have the BSE I and BSE II Initiatives been designed appropriately to achieve expected outcomes?
Indicators:
• Extent to which key stakeholders/partners were involved in program design.

• Extent to which risk-based analysis and scientific evidence were used in program design.
Rating:
• Some evidence of progress; attention needed.
Summary:
• The design of the BSE Initiatives was based on the scientific evidence available at the time and was informed by risk-based analysis and some consultation with stakeholders, and to that extent was appropriate to achieve the expected outcomes. However, some Health Canada partners whose mandated responsibilities would seem to extend to BSE/TSE risk management and control efforts were not included in the Initiatives, raising questions about the adequacy of the overall program design.

The evaluation evidence suggests that the design of the BSE I and BSE II Initiatives was based on the scientific evidence available at the time, and was informed by risk-based analysis and some consultation with stakeholders. To that extent, the program was designed appropriately to achieve its expected outcomes. On the other hand, the evaluation also found that some Health Canada partners whose mandated responsibilities would seem to extend to BSE/TSE risk management and control efforts were not included in the Initiatives, raising questions about the adequacy of the overall program design.

Involvement of relevant partners

The evaluation found that some internal Health Canada partners with responsibilities that would seem to implicate them in Health Canada's BSE/TSE risk management and control efforts were not involved in BSE I and II. The CPSD, whose responsibilities include regulating cosmetics and personal care products, was identified in government planning documents as having a role in BSE-funded activities with respect to cosmetics, but did not receive any BSE funding for reasons that are not clear. Similarly, the MHPD, which is responsible for post-approval safety surveillance, assessment of signals, and safety trends and risk communications concerning all regulated marketed health products -- including those implicated in the BSE Initiatives -- likewise did not receive any BSE funding.

The evaluation did not find any evidence that the exclusion of CPSD and MHPD had a detrimental impact on Health Canada's ability to achieve its expected outcomes under BSE I and II. However, their exclusion does raise questions about the adequacy and comprehensiveness of Health Canada's overall approach to BSE/TSE. With respect to the potential role of MHPD in BSE/TSE-related surveillance, it is important to note that the PHAC was funded for and carries out prospective surveillance of all types of human prion disease through the Canadian Creutzfeldt-Jakob Disease Surveillance System (CJDSS).

Stakeholder consultation

Stakeholder consultations occurred mainly with Health Canada's federal partners, rather than with stakeholders external to the federal government. Furthermore, the consultations were intended primarily to inform the federal government's overall response to BSE/TSE, rather than the design of the Health Canada component in particular.

The main venue for formal consultation appears to have been the Expert Consultation held in February 2001, which made recommendations to the federal government on specific areas where BSE or TSE activity should be considered. The Expert Consultation was attended primarily by federal government representatives, including 18 from the CFIA and 22 from Health Canada. Experts from the United States (n=3), the United Kingdom (n=3), the World Health Organization (n=1), the provinces of Quebec (n=1) and Ontario (n=2), and three others (Health Canada, 2001a, p. 42) were also in attendance. Industry representatives, Canadian academics, and Canadian researchers working outside of government do not appear to have participated.

There is less evidence of formal consultations specifically to inform the design of Health Canada's activities under BSE I and BSE II. Key informants representing Health Canada's federal partners (CFIA, AAFC, and PHAC) reported that their organization was consulted or at least informed to varying degrees regarding the design of BSE I and BSE II, but was not directly involved in designing the Initiatives or in formulating the Treasury Board submissions for funding. Furthermore, the formative evaluation reported that "there was minimal to no consultation with external stakeholders with respect to the design of the BSE Initiatives" (Health Canada, 2007a, p. 22). Without standards for comparison, it is difficult to assess whether the nature and level of stakeholder consultation in program design was appropriate.

Use of scientific evidence and risk-based analysis

The evaluation found that risk-based analysis and scientific evidence were used to inform the federal government's overall response to BSE and the initial design of the BSE Initiatives. For example, several components of BSE I and BSE II, particularly the development of tracing systems and the ongoing practice of risk assessments, were among those recommended by a group of experts from the World Organisation for Animal Health (OIE) that evaluated Canada's response to the May 2003 case of BSE (Kihm, Hueston, & Heim, 2003, p. 4).

Furthermore, the TSE Action Plan developed following the 2001 Expert Consultation set out a formal process, depicted in Table 9, by which Health Canada would address TSE issues. The process incorporates the three phases of risk-based analysis identified in Health Canada's Decision-Making Framework for Identifying, Assessing, and Managing Health Risks (Health Canada, 2000): namely, issues identification, risk assessment, and risk management. Issues of particular concern relating to the science, control, and prevention of TSEs were identified and prioritized through the Expert Consultation (Health Canada, 2001a, pp. 1-2).

Based on this list of issues, four key priority areas for Health Canada -- food, cosmetics, blood/vaccines/tissues, and therapeutics -- were identified by the TSE Science Team and linked to specific issues or risks (Health Canada, 2002a, p. 17).Footnote 18 These four priority areas were subsequently addressed byHealth Canada's BSE I and BSE II Initiatives. However, as previously noted, the CPSD, which is responsible for regulating cosmetics, did not receive any funding under BSE, even though it was identified in Health Canada's planning documents related to the Initiatives.

External key informants generally agreed that the design of BSE I and BSE II was based on the scientific evidence that was available at the time, and that risk-based analysis was used to inform program design. Several external key informants also noted that in their opinion, Health Canada leads federal government departments in the use of scientific evidence and risk-based analysis to inform decision-making. As discussed in Section 4.2.1, however, it is not clear that the process described in the TSE Action Plan was used on an ongoing basis to make decisions with respect to BSE/TSE regulation and policy under BSE I and II.

Table 9: Health Canada Process to Address TSE Issues - Encapsulated in TSE Action Plan
Identification and Clarification of Specific Issues
(Expert Consultation/Science Review)
Science Team Policy Team
  • (4 sub-groups)
  • Define Issues
  • Document Current Science
  • Develop Risk Assessment
  • Identify Mitigation Options
  • Advise on Research Needs
  • Prepare Template for Policy Analysis
  • Assess Existing Policy
  • Advise on New Policy
  • Recommend Risk Management Actions
  • Analysis of Imports Policy on International
  • Agreements (e.g., WTO)

Ongoing Review Process

Source: Health Canada, 2002a

4.2.3 Program implementation

Evaluation Question:
• Have the BSE I and BSE II Initiatives been implemented appropriately to achieve expected outcomes?
Indicator:
• Extent to which BSE activities were implemented as planned in the TB submissions.
Rating:
• Attention needed.
Summary:
• The evidence available to the evaluation suggests that implementation of the Initiatives did not occur as planned. Risk assessment, product assessment, and tracking and tracing were conceptualized as distinct activities with discrete funding allocations in the original planning documents, but were not distinguished from one another by all of the directorates that received funding for them. Three of the evaluation directorates (BGTD, TPD, and VDD) have not published policies and guidance documents for industry pertaining specifically to the reduction of BSE/TSE-related risks. Despite an apparent interest in a Branch-level policy on reducing BSE/TSE-related risks in the products regulated by HPFB, such a policy, though drafted, has never been finalized. Moreover, it appears that a specific BSE/TSE-related inspection program for health products regulated by Health Canada was not fully implemented. Finally, actual spending was 64% of planned spending for BSE I and 47% of planned spending for BSE II.

The absence of work plans, operational plans, or progress reports related to BSE I and BSE II created challenges in assessing program implementation. That being said, the evidence available to the evaluation suggests that implementation of the Initiatives did not occur as planned.

Table 10 below describes the five activities funded under BSE I and BSE II, and identifies which directorates received funding for these activities. The activities undertaken by each of the funded directorates are summarized following the table, and some general observations on program implementation conclude the section.

Table 10: BSE I and II activities and funded directorates
Activity Description Funded directorates

Risk assessment

This activity was to consist of risk assessments to determine the potential prion disease risks to humans posed by identified food and health products and ingredients. This component also includes activities related to the development of guidelines and protocols to improve the methods for selecting and screening products. The completed risk assessments would provide the necessary evidence base to inform BSE/TSE-related regulatory actions to protect human health

BGTD, FD, NHPD, TPD, VDD

Product assessment

This activity was to consist of the ongoing review of priority food and health products (up to 20%) with the potential to present a risk to the health of Canadians, due to bovine-sourced materials or other ingredients that may post a BSE infection risk.

BGTD, FD, NHPD, TPD, VDD

Tracking and tracing

This activity was to consist of the identification and ongoing surveillance of bovine-sourced materials that may be used as ingredients in foods, biological/therapeutic products, drugs for human or animal use, natural health products, medical devices, or cosmetics.

BGTD, NHPD, TPD, VDD

Targeted research

This component was to consist of targeted research into the characteristics of BSE/TSE, in order to improve understanding of BSE pathogenesis; increase capacity to identify exposure risks through all consumer products; improve standardized diagnostics; develop a better understanding of BSE zoonotic potential; and mitigate the public health impacts from animal TSE diseases.

BGTD, FD

Compliance and enforcement

According to official government documents for BSE I, Health Canada will be increasing its capacity to conduct thorough inspections and investigations with respect to the source of drug raw materials of bovine origin.

Key activities include: inspections of establishments to verify acceptable sources of drug raw materials, confirming information on new drug submissions, carrying out inspections to ensure that Good Manufacturing Practices (GMP) are being followed and investigations where a potential problem has been identified.

The submission notes that "there are three activities or components involved in the process:

The proactive component involves inspections of establishments to verify acceptable sources of drug raw materials, confirming information on new drug submissions. These would occur integrated with regularly scheduled good manufacturing practices submissions.

The reactive component would constitute investigations where the Inspectorate became aware of a potential problem, in addition to requests from industry to confirm raw material sources for business purposes.

Additional resources would enable Health Canada to conduct the new inspections and investigations with respect to the source of drug raw materials of bovine origin.

HPFBI

Biologics and Genetic Therapies Directorate

BGTD received funding for risk assessment, product assessment, tracking and tracing, and research. Key informants reported that risk assessment is a routine aspect of product review and approval. All product applications are required to complete Health Canada Form 3011: Drug Submission Application Form for Human, Veterinary, Disinfectant Drugs, and Clinical Trial Application/Attestation (Health Canada, n.d.b.), which screens for the presence of animal and/or human-sourced materials. Products containing these materials are required to complete Appendix 4 to the Form (Drug Product Formulation Information: Animal and/or Human-Sourced Ingredients/Materials), which requires sponsors to provide information on the source, the tissue or fluid of origin, the age of the animal, and the country of origin of the animal. Although Form 3011 is relatively new, key informants reported that BGTD has always included an assessment of raw materials of biological origin in the review process.Footnote 19

Key informants noted that product approvals are not necessarily static and that products are reassessed in the event of changes to animal-derived ingredients and as new information becomes available. Key informants also noted that within the directorate's product portfolio, there has been a consistent and voluntary progression by industry away from bovine-derived raw materials and towards animal-free materials.

The information captured via Appendix 4 is a key element of BGTD's tracking and tracing system for animal-derived ingredients. In 2001, to capture information on animal-derived ingredients for products already on the market, BGTD sent a letter to all manufacturers that had a product licensed in Canada, asking them to identify all animal-derived ingredients in their products (Health Canada, n.d.c.). Although the response was not 100%, key informants reported that the data were reasonably complete for an acceptable proportion of BGTD's sponsors. However, the data were never validated and key informants were uncertain whether this exercise was ever driven to completion. As a result, BGTD does not have complete information on animal-derived ingredients in biologic products currently on the market.

Beyond the information captured on Appendix 4 and in order to execute on a directive requiring it to recall products if a donor is subsequently found to have had vCJD, key informants reported that the directorate has also implemented a tracking and tracing system for specific lots of human-derived materials, which enables it to track both release of human-derived products onto the Canadian market and use of human-derived excipients in biologics in the Canadian market. This system allows BGTD to locate all the batches of other products that have incorporated, for example, human albumin from a specific lot of human albumin, and recall these products if necessary.

Key informants also reported that the directorate conducts risk assessments in relation to emerging situations or new scientific information, and cited several completed in the past year. For example, in response to recent publications concerning risks associated with urinary products and prion excretion in urine, BGTD established a working group to evaluate the risk of using urine as a starting material for human therapeutics and the capacity of the manufacturing process for these products to effectively clear prions. BGTD also reported that recent risk assessments have addressed whether to release or destroy plasma products upon discovery that a plasma fractionation pool contained plasma from a donor with one of the forms of CJD, as well as specific prion clearance claims related to human plasma.

Risk assessment has also influenced BGTD's blood donor deferral policy, which has been updated twice during the period of BSE funding. In 2005, the policy was updated to include other Western European countries aside from the United Kingdom and France. The policy evolved to defer all blood donations from individuals who have spent a cumulative three months of time between 1980 and 1996 in the United Kingdom or France; or have spent a cumulative five years from 1980 to the present in Western European countries, including Germany, Italy, Netherlands, Switzerland, Austria, Belgium, Spain, Republic of Ireland, Portugal, Denmark, Luxembourg, and Liechtenstein; or have received a transfusion from 1980 to the present involving blood or blood-derived products in the United Kingdom, France, or the aforementioned Western Europe countries (Health Canada, 2005d, para. 2). In 2011, as a result of a patient in Canada with vCJD who was believed to have contracted the disease in Saudi Arabia, Health Canada expanded the policy to defer blood donations from all individuals who have spent a cumulative six months or more between 1980 and 1996 in Saudi Arabia (Health Canada, 2011a, para. 2-3).

In terms of guidance documents for industry, BGTD created the Guidance for Sponsors: Regulatory Requirements for Managing Potential Risks of Transmissible Spongiform Encephalopathies (TSEs) Transmission from Animal-Sourced Excipients, Auxiliary Reagents, and/or Active Pharmaceutical Ingredients in the Manufacture of Schedule D (Biologic) Drugs. This document was never published, although it was used internally and was the basis of a Branch-level guidance document, Minimizing the Risk of Transmission of Non-Human Animal Transmissible Spongiform Encephalopathy (TSE) Agents via Health Products (Health Canada, 2005e).The Branch-level guidance document was never finalized. Key informants reported that the BGTD document is currently being updated, and BGTD has plans to finalize and implement it at the directorate level, as well as bring it forward to the Branch level.

Key informants reported that the directorate has also introduced a variety of other documents intended to provide guidance to industry. These documents include guidance with respect to good manufacturing practices for Schedule D Drugs (Biological Drugs); guidance on Post-Notice of Compliance Changes; a Common Technical Document; a policy on the use of albumin as an excipient in Division 5 drugs; and a regulatory requirement that clinical trial sponsors use an authorized human-sourced excipient or file supporting information. The directorate also uses the WHO's tables on tissue infectivity distribution (WHO, 2010).

In the area of research, the formative evaluation reported that BGTD undertook two research projects, one on the structure of prions as they developed into a diseased state, and the other addressing the susceptibility of components in vaccine and biologics production to prion infection (Health Canada, 2007a). The latter project was designed to produce tools, processes, or technology for use in product assessment activities. A third project on the inactivation of prions was planned, but was cancelled due to overlap with other work.

Food Directorate

The FD received funding for risk assessment, product assessment, and research. In collaboration with the TSE Secretariat, the FD produced two formal risk assessments in 2003, namely: Variant Creutzfeldt-Jakob Disease Risk to Canadians Eating Imported Foods Containing Small Amounts of Processed Ruminant Meat Product (Health Canada, 2003c); and Risk Assessment: Impact of SRM Policies on Potential Levels of BSE Infectivity in Food (Health Canada, 2003d). In 2005, the FD and the TSE Secretariat collaborated on a third formal risk assessment, Quantifying the Potential BSE Infectivity in the Production of Canadian Beef and Beef Products and the Risk to Canadian Consumers of Acquiring vCJD (Health Canada, 2005f). Work on a fourth risk assessment has been underway for the past few years. Key informants reported that efforts are ongoing to develop an information-sharing agreement between Health Canada and the CFIA. Such an agreement would enable Health Canada to access CFIA compliance data, which it requires to fully understand industry compliance with the SRM removal policy and update its risk assessments with respect to food products.

With respect to product assessment, while the FD did not make any changes to the SRM Removal Policy for Food, the TSE Secretariat did undertake a variety of activities related to product assessment, including assessing risks to public health presented by products covered under Interim Regulations to the United States; products covered by the CFIA Import Policy; products covered by the CFIA Manual of Meat Hygiene; products covered by the OIE, and any proposed changes to their chapters concerning BSE/TSE; and the proposed CFIA Enhanced Feed Regulations (Health Canada, 2007b).

Finally, the FD undertook several targeted research projects in collaboration with the TSE Secretariat, though it is not clear if all of these projects were funded under BSE II or if some were funded, at least in part, under BSE III. These research projects include:

  • a collaborative project with Veterinary Laboratories Agency (VLA) in the UK to compare the sensitivity and specificity of methodologies for testing stages of BSE incubation and infection in cattle tissue; this project resulted in a scientific publication in the Journal of General Virology (Arnold et al., 2007);
  • a Bureau of Microbial Hazards research project to compare and develop methodologies to detect prohibited SRMs in food products;
  • a study of TSEs in non-human primates with implications for human health, in collaboration with the Commissariat à l'Energie Atomique France;
  • a study under signed Memorandum of Agreement with the Friedrich-Loeffler Institute (Germany), analyzing abnormal prions and detection of incubating BSE to enhance definition of SRMs;
  • a research project on transmission and infectivity of BSE in Canadian cattle; and
  • three research projects undertaken through a research partnership established with the CFIA through a Letter of Understanding: two on CWD infectivity and transmission in animals, and one to collect BSE surveillance information for the Canadian Animal Surveillance Network data system
Natural Health Products Directorate

NHPD received funding for risk assessment, product assessment, and tracking and tracing. NHPD reported that it has not conducted extensive formal risk assessments internally, but has relied on those completed by other directorates (i.e., the Food Directorate) and internationally.

NHPD reported that within the directorate, risk assessment occurs primarily through the product review process. All product submissions are required to provide information on animal-sourced ingredients via the Animal Tissue Form (Health Canada, 2004a), introduced in 2004 specifically to address BSE/TSE related risks. This is the same year that the directorate was established; prior to that, natural health products were categorized as either food products or drug products. With the establishment of NHPD, all products that fell within the definition of a natural health product, including products that were already on the market, had to be submitted to NHPD in order to obtain a Natural Product Number (NPN). According to key informants, as a result of this requirement, NHPD has information on animal ingredients for all natural health products, regardless of when they were first approved for sale in Canada.

NHPD has developed and implemented several policies and guidance documents for industry, including:

  • Evidence for Safety and Efficacy of Finished Natural Health Products: Guidance Document (Health Canada, 2006b). Section 7.1 describes NHPD's policy on animal-derived ingredients, including a prohibition on the use of SRMs (as defined in the Food and Drug Regulations) for manufacturing and/or in the processing of natural health products.
  • Revised Policy for Gelatin in Natural Health Products (Health Canada, 2006c). This policy is referenced in the above guidance document, although the link to it within the document is broken and the policy is not available on NHPD's website. Nevertheless, key informants reported that the policy is in effect.
  • Draft Policy on Reduction of Transmissible Spongiform Encephalopathy (TSE) Risk in Natural Health Products (Health Canada, 2007c). NHPD key informants reported that this policy is in effect, although it could not be located on NHPD's website. Furthermore, the prohibited tissues listed in this document differ from those identified in the guidance document mentioned above.Footnote 20
Therapeutic Products Directorate

TPD received funding for risk assessment, product assessment, and tracking and tracing. These activities are closely linked within TPD. Key informants reported that TPD updated the submission review process to identify animal-sourced materials in reviewed drugs.Footnote 21 All applications for human drugs are now required to complete Health Canada Form 3011, including Appendix 4 requiring sponsors to provide information on the source, the tissue or fluid of origin, the age of the animal, and the country of origin of the animal. Prior to 2010, reporting of this type of information was voluntary. Since the introduction of the revised Form 3011, the sponsors of Division 1 drugs are required to submit the form including Appendix 4 when they make a change to the formulation of the product. This requirement also applies to veterinary drugs and biologics. The information captured via Appendix 4 is the main mechanism for tracking and tracing animal-sourced ingredients in human drugs.

It was also reported that in 2004, TPD's Submission Information and Policy Division sent a Notice, Guideline, and instructions, as well as an accompanying "Drug Product Information Form" to all DIN holders (for TPD, BGTD, and VDD) asking them to supply information such as specific animal source, tissues used, and country of origin. Although the response rate was not 100%, TPD believes that this information is reasonably complete for an acceptable proportion of DIN holders. Nonetheless, like BGTD, TPD does not have complete information on animal-derived ingredients in all therapeutic products currently on the market.Footnote 22

TPD does not have any specific review and/or risk assessment policies or guidance documents relating to BSE/TSE for human drugs. TPD key informants reported that the directorate had begun drafting a policy, but work on it was put on hold due to the perceived need for a Branch-level BSE/TSE policy. As reported above, a Branch-level policy has never been finalized.

Medical devices are managed by the Medical Devices Bureau (MDB) of TPD. A separate set of application forms has been developed for these products and MDB has developed the Guidance Document on the Regulation of Medical Devices Manufactured from or Incorporating Viable or Non-Viable Animal Tissue or their Derivative(s) (Health Canada, 2004b). The guidance document stipulates that animal materials used in medical devices must be identified according to tissue type, species, and country of origin, and requires that the source country for bovine materials must be considered BSE free and ovine and caprine sources must be free of scrapie.

Veterinary Drugs Directorate

VDD received funding for risk assessment, product assessment, and tracking and tracing. Like BGTD, NHPD, and TPD, VDD reported that risk assessment is done systematically on all incoming submissions. All product applications are required to complete Health Canada Form 3011 including Appendix 4. In addition, according to the VDD's Guidance for Industry: Preparation of Veterinary New Drug Submissions: "pursuant to Section C.08.002(2) of the Food and Drugs Act and Regulations, all initial and supplemental submissions should include an original signed and dated version of the Animal Ingredient Form" (Health Canada, 2007d, p. 16). However, since the introduction of Appendix 4 of the HC-SC 3011 Form, VDD has required sponsors of veterinary drugs to complete this Appendix in lieu of the Animal Ingredient Form for both new drugs and when they make a change to the formulation of veterinary drugs regulated under Division 1.   

As previously described, TPD's Submission and Information Policy Division, on behalf of VDD, undertook an initiative to identify animal-derived ingredients in veterinary drug products that were already on the market before the new submission forms were developed. Since, as described above, the response rate was not 100%, this information is presumably not complete for this group of veterinary drug products.

Beyond general instructions to sponsors to complete Appendix 4 of the HC-SC 3011 form, VDD does not currently have in place any guidance documents specifically related to BSE/TSE. The Directorate had developed a guidance document entitled Guidance for Industry: Minimizing the Potential Risks of Transmission of TSE Agent via Veterinary Therapeutic Products Fabricated from Animal-Sourced Ingredients, and although this document is referenced in Section 6 of Appendix 4 of the HC-SC 3011 form, the link to the document is broken because the guidance is currently not in effect. The Directorate put the publication of that document on hold due to the perceived need for a Branch-level BSE/TSE policy that has never been finalized.

Health Products and Food Branch Inspectorate

HPFBI received funding under BSE I for compliance and enforcement activities. Key informants reported that the Inspectorate undertook the following BSE/TSE-related activities:

  • A dedicated BSE/TSE lead was funded within the Inspectorate, who was also the Inspectorate's representative on the internal working groups;
  • A list of companies that use animal-derived raw material was produced and a plan was developed to follow up during inspections. According to key informants, this list was periodically updated, but is no longer available;
  • BSE/TSE was incorporated into the Inspectorate's regular inspections. For a period of time, as part of Good Manufacturing Practices (GMP) inspections, inspectors asked for BSE-free certificates when looking at gelatine products (i.e., capsules and gel caps). Key informants also noted that while inspectors may still make this request, it is not part of the regular GMP inspection; and
  • Compliance verifications and inspections were conducted for any referrals of BSE risk issues across the country.

The Inspectorate did not provide the evaluation with any documentation of these activities. However, some Inspectorate planning documents from 2003 were provided to the evaluation. These set the context for a potential BSE/TSE inspection program by noting, among other things, that "no inspection program has been conducted to date...to verify in the field the authenticity and accuracy of the information committing the manufacturer and demonstrating that there is no risk of transmitting the agents responsible for TSEs" (Health Canada, 2003e, p. 3). The document also noted that:

  • Collaboration with the evaluation directorates - i.e. TPD and BGTD - must be constant ... these directorates will play a determining role in the development of the inspection program through approval of the priorities and [underline in original] final evaluation of the information collected in the field for decision-making. They will play this role in the following areas in particular:
    • identification of the products and materials that pose the most risk on the basis of the origin of the tissues used and the countries of origin of the animals ... [and]
    • the processing of the information collected during the inspections and considered unsatisfactory by the inspectors, since the final decision regarding the products future is the responsibility of TPD and BGTD on the basis of the balance between risks and benefits. [bold in original] (Health Canada, 2003e, pp. 5-6).

The document went on to identify priority directions for inspection, present an inspection methodology, and propose a schedule for implementation.

A year-end project report for fiscal 2005-2006 noted that the inspection program was in the development and planning stage and that just over one Full-Time Equivalent of unused resources had been reallocated to other inspection requirements (Health Canada, 2006d). An Operational Planning Project report for fiscal 2006-2007 suggests that an inspection program was still contemplated at that time (Health Canada, 2007e), and the 2006-2007 year-end report noted that some planning work had been done, including initiation of contacts with BGTD, TPD and NHPD. However, the latter also noted that "the operational phase of this project has not yet been requested to be initiated" and that "resources dedicated to this project were reallocated" (Health Canada, 2007f, p. 1). The evaluation did not receive planning documents or year-end reports related to the inspection program for subsequent years (2007-2008 and 2008-2009).

Key informants reported that although some planning work was indeed done on a formal BSE/TSE inspection program as described above, a formal program was not implemented, pending direction from the science directorates. Based on the evaluation evidence, it is not clear whether and to what extent there was collaboration between HPFBI and the evaluation directorates (BGTD, TPD, VDD and NHPD) on determining inspection priorities related to BSE/TSE.

Consumer Products Safety Directorate

The CPSD, as previously described, was identified in Health Canada's BSE planning documents but, for reasons that are not clear, did not receive any funding under either BSE I or BSE II. Nonetheless, the CPSD did produce a draft risk assessment in 2003 entitled Transmissible Spongiform Encephalopathy: Risk to Canadians Using Cosmetic and Personal Care Products (Health Canada, 2003f), and conducted a survey of industry regarding their use of animal-derived ingredients, though the timing of that survey could not be determined based on available documents. The CPSD also published a list of prohibited and restricted cosmetic ingredients, or "hotlist", which prohibits SRM from use in cosmetic products (Health Canada, 2011b).

Observations on program implementation

Several broad observations on program implementation can be made on the basis of the evaluation evidence.

  • Although risk assessment, product assessment, and tracking and tracing were conceptualized as distinct activities with discrete funding allocations in the original planning documents, in practice these activities were not and are not necessarily distinguished from one another by all of the directorates that received funding for them.
  • All four drug evaluation directorates (BGTD, TPD, NHPD and VDD) require sponsors to identify animal-sourced ingredients in new drugs as part of the product application process.Footnote 23 These directorates have submission requirements and forms requiring manufacturers to identify and provide information on animal-sourced ingredients in their products. The information captured is also the main way in which the directorates track and trace information. SIPD provided the centralized function for TPD, BGTD and VDD to identify animal-sourced ingredients in products that were already on the market before the new submission forms were developed, although this information is not complete in most cases. In particular, neither BGTD nor TPD has complete information on animal-derived ingredients for products that are currently on the market, and this is presumably also the case for VDD.
  • Despite an apparent interest in a Branch-level policy on reducing BSE/TSE-related risks in the products regulated by HPFB, such a policy -- though drafted -- has never been finalized. This may be illustrative of a lack of collaboration and coordination among internal partners in the Initiatives. At present, among the evaluation directorates involved in regulating health products, NHPD and the MDB within TPD appear to have published policies and guidance documents pertaining specifically to the reduction of BSE/TSE-related risks. Currently, two of the directorates funded through BSE I and BSE II -- the FD and NHPD -- officially prohibit SRM in the products they regulate.
  • Two Health Canada directorates with responsibilities that would seem to implicate them in Health Canada initiatives to reduce BSE/TSE-related risks -- namely the Marketed Health Products Directorate and the Consumer Safety Products Directorate -- were excluded from BSE funding for reasons that are not clear. The evaluation could not determine whether their exclusion had any impact on Health Canada's ability to address BSE/TSE-related risks.
  • According to key informants, some BSE/TSE-related inspection activities were conducted. However, the absence of any documentation of inspection activities specifically related to BSE/TSE suggests that planned compliance and enforcement activities were not fully implemented.

Finally, information on planned and actual spending, provided by the Strategic Planning and Accountability Division within the RMOD, would seem to support the conclusion that program implementation did not occur as planned. These data indicate that both BSE I and BSE II reallocated a significant amount of funds. Actual spending was 64% of planned spending for BSE I and only 47% of planned spending for BSE II. Detailed financial information is presented in Section 4.2.6 below.

4.2.4 Outcomes achieved

The lack of performance measurement and administrative data related to outcomes has limited the extent to which the evaluation can draw conclusions regarding outcomes achieved. That being said, there is some evidence from other data collection methods that some progress towards certain outcomes has been achieved.

a. Improved regulation/policy response to control and prevent risks associated with BSE/TSE
Evaluation Question:
• To what extent is there improved regulation/policy response to control and prevent risks associated with BSE/TSE?
Indicators:
• Reduced time between identification of risk and decision to maintain, update or create new regulation or policy.

• Extent to which regulation/policy responses are based on credible scientific research and assessed risks.

• Extent to which regulation/policy response include consultation with partners and stakeholders.
Rating:
• Some evidence of progress; insufficient to support firm conclusion.
Summary:
• The evaluation found that while some of Health Canada's responses have been based in science, informed by risk assessment, and involved consultation with stakeholders, a lack of documentation prevented the evaluation from drawing general conclusions on the extent to which this outcome has been achieved.

The BSE Initiatives were intended to produce an improved regulatory/policy response to control and prevent risks associated with BSE/TSE, measured by the scientific basis of the response and the use of risk assessment to inform it; the timeliness of the response; and the extent to which the response involved consultation with stakeholders. The evaluation found that some of Health Canada's responses have met these criteria, but due to a lack of documentation could not draw general conclusions on the extent to which this outcome has been achieved.

Health Canada's federal partners and other external stakeholders agreed that Health Canada's regulatory/policy responses to BSE/TSE are based on credible scientific research and assessed risks, and those who felt able to comment on the matter also believe that these responses have been timely. However, on the question of whether Health Canada consulted adequately with them when developing its response, federal partners disagreed.

Similarly inconclusive results were found by the industry survey. Overall, 56% of respondents agreed that, compared to 10 years ago, Health Canada has improved its regulatory and policy response to control and prevent risks associated with BSE/TSE, while slightly fewer agreed that this response is based on assessed risks (50%) and on science (45%). Only 27% thought that Health Canada has consulted adequately with their industry when developing its response to BSE/TSE. However, in all of these cases, a substantial minority (between 38% and 46%) did not know. See Table 11.

Table 11: Industry survey respondents' level of agreement with statements about extent to which Health Canada's regulatory and policy response to BSE/TSE has improved
Industry Survey Questions Percent (n=117)
Agree Disagree Don't know

Compared to 10 years ago, Health Canada has improved its regulatory and policy response to control and prevent risks associated with BSE/TSE

56%

6%

38%

Health Canada's regulatory and policy response to control and prevent risks associated with BSE/TSE is based on assessed risks

50%

8%

42%

Health Canada's regulatory and policy response to control and prevent risks associated with BSE/TSE is based on science

45%

10%

44%

Over the past 10 years, Health Canada has consulted adequately with my industry when developing its regulatory and policy response to BSE/TSE

27%

27%

46%

Source: Survey of industry.
Note: Row totals may not sum to 100% due to rounding.

Despite the inconclusiveness of these findings, the evaluation did find some concrete evidence that Health Canada's regulatory/policy response to BSE/TSE has been timely, consultative, and based on scientific evidence. One example is the Policy on Specified Risk Material in the Food Supply.Footnote 24 In 2003, the Food Directorate and the TSE Secretariat completed two formal risk assessments on the subject of BSE-related risks and food. Following the discovery in May 2003 of the first domestic case of BSE, the SRM removal policy was announced in July 2003 and implemented in August 2003. Extensive consultations were undertaken by Health Canada on the development of the SRM policy, and included consultation with the CFIA; AAFC; provincial/territorial authorities; three federal/provincial/territorial committees (namely, Federal/Provincial/Territorial Committee on Food Safety Policy, Federal/Provincial/Territorial Agri-Food Inspection Committee, and the Canadian Food Inspection System Implementation Group); and the Council of Chief Medical Officers of Health. Additionally, consultation with representatives from various industry sectors (e.g., feed, seed stock, cow-calf, feedlot, primary/secondary processing, rendering, food service, and retail) occurred at the National Beef Industry Value Chain Roundtable on June 23-24, 2003 (Health Canada, 2003a).

Case study of a firm in the Canadian beef industry

Following the discovery of a domestic case of BSE, Firm A participated in a working group consisting of 8 to 12 industry representatives as well as representatives of Health Canada and the CFIA. Communication among working group members took place through teleconferences and in-person meetings. This group developed the policy requiring the removal at slaughter of SRM from cattle older than 30 months.

The firm representative reported a productive working relationship with Health Canada, and specifically pointed out that Health Canada provided valuable scientific justification for the policy approach that was taken. The firm representative also pointed out that Health Canada made itself readily available to the whole industry, clearly outlined what was needed to resolve the crisis, and effectively rallied industry to determine how these needs could be met.

The company representative reported that the firm maintains a good relationship with Health Canada officials. In addition to the productive relationship that characterized the period during which the SRM regulation was being developed, the representative also reported that the firm communicates regularly with Health Canada through the Beef Value Chain Roundtable (BVCRT).

Another example singled out by key informants is Health Canada's BSE/TSE regulatory framework with respect to biological products, blood safety, and tissue transplantation. One component of this framework is the policy on blood donor deferral. As previously reported, Health Canada first implemented a blood donor deferral policy in 1999 to exclude individuals who had spent six months or longer in the UK during the period of 1980 to 1996, and updated the policy a year later to exclude individuals who had spent six months in France from 1980 to 1996. Two subsequent updates occurred during the period of BSE funding. In 2005, the policy was updated to include other Western European countries aside from the United Kingdom and France, and in 2011, it was expanded to defer donations from all individuals who have spent a cumulative six months or more between 1980 and 1996 in Saudi Arabia.

BGTD key informants reported that these updates to the policy involved a geographically-based assessment of risk, and described the directorate's approach as a series of measured decisions in which the potential reduction in risks to recipients of transfusion products were weighed against the likely impact on the blood supply. Key informants also noted that although the initial blood donor deferral policy was based on a theoretical risk of transmission through blood transfusion, in recent years epidemiological evidence of such transmission has confirmed that transmission through blood transfusion is likely.Footnote 25 Key informants both internal and external to Health Canada identified the blood donor deferral policy as a good example of the application of the precautionary principle, particularly in light of the number of Canadians affected or potentially affected by the blood system.

b. Increased awareness and understanding of BSE/TSE risk control efforts, regulations, and policies among partners and stakeholders
Evaluation Question:
• To what extent is there increased awareness and understanding of BSE/TSE risk control efforts, regulations and policies among partners and stakeholders?
Indicators:
• Number/nature of information sources and communication/dissemination plans.

• Number/nature of requests for information by internal and external stakeholders.

• Level of partner and stakeholder awareness and understanding of BSE/TSE risk control efforts, regulations, and policies.
Rating:
• Some evidence of progress; attention needed.
Summary:
• While the qualitative evidence suggests that general awareness and understanding of BSE/TSE-related risks has increased in Canada, results from the industry survey suggest opportunities to strengthen awareness and understanding among industry stakeholders, particularly with respect to the BSE/TSE-related policies and regulations affecting them. This finding is consistent with concerns expressed by some internal key informants regarding the transparency of Health Canada's BSE/TSE regulatory framework.

The qualitative evidence available to the evaluation suggests that, generally speaking, awareness and understanding of BSE/TSE-related risks has increased in Canada over the past two decades. External key informants certainly believe such an increase has occurred as the field of BSE/TSE science has matured. Several noted that the establishment of PrioNet Canada has had a major impact on awareness, as have the efforts of other organizations including the CFIA and the PHAC, as well as Health Canada and industry.Footnote 26 From their perspective, Health Canada has contributed to a general increase in awareness and understanding, but has not been solely responsible for it.

Nevertheless, results from the industry survey suggest that awareness and understanding of BSE/TSE-related risks, and of Health Canada's BSE/TSE risk control efforts, regulations, and policies, could be strengthened among industry stakeholders. Overall, about half (49%) of survey respondents reported that Health Canada is their main source of information related to BSE/TSE, followed by industry; those involved in the food industry were significantly more likely than other respondents to rely on sources other than Health Canada. Three-quarters (76%) of all survey respondents reported having received some type of information from Health Canada, and of these, most agreed that the information they received was useful (72%), of high quality (64%), and timely (63%).

Case study of a firm in the Canadian beef industry

The representative of Firm A reported that in terms of domestic governmental sources of information, the vast majority of its communications regarding BSE/TSE are with the CFIA. In special cases, Health Canada is used as a source as well, but this occurs much less often. In particular, the representative estimated that about 85% of their BSE/TSE information comes from the CFIA, while the remaining 15% comes from Health Canada as well as other sources. The representative mentioned that the company's relationship with Health Canada mostly involves adding new procedures and rules to current practices (e.g., developing standards of removal for dorsal root ganglia and distal ileum). Conversely, communications with the CFIA relate to day-to-day operational issues.

The company representative maintains a very positive view of Health Canada, describing the department's information and communications as timely, of high quality, and useful. The interviewee reported a high level of awareness and understanding within Firm A of Health Canada's BSE/TSE risk control efforts, regulations, and policies.

Just over half of survey respondents had received information on changed or new policies or regulations affecting their industry, introduced by Health Canada, to address BSE/TSE-related risks (53%). Similarly, among the subset of respondents involved in the health products industries (n=75)Footnote 27, about half had received information on submission requirements related to identification of animal-sourced ingredients for manufacturers applying to have a health product approved for sale in Canada (55%); guidance for manufacturers of health products on how to comply with the submission requirements (55%); and Health Canada's regulatory compliance and enforcement activities in their industry (49%). For reasons that are not clear but could be related to the survey sampleFootnote 28, a substantial minority of these respondents (between one-fifth and one-quarter) believe these types of information are not relevant to their industry or organization. Caution should be used when interpreting these results due to small sample size.

Fewer than half of all survey respondents reported receiving other types of information related to BSE/TSE from Health Canada, such as information on BSE/TSE risks affecting their industry, information on BSE/TSE risk management or control measures in their industry, or scientific, technical or research literature on BSE/TSE. See Table 12 below for the details.

Table 12: Percent of industry survey respondents having received various types of information from Health Canada
Type of information Percent of all respondents (n=117)
Yes No Not applicable

Changed or new policies or regulations affecting their industry, introduced by Health Canada, to address BSE/TSE-related risks

53%

47%

--

Information on BSE/TSE risks affecting their industry

40%

60%

--

Information on BSE/TSE risk management or control measures in their industry

30%

70%

--

Scientific, technical, or research literature on BSE/TSE

22%

78%

--

Percent of health product industry respondents (n=75)

Submission requirements related to identification of animal-sourced ingredients for manufacturers applying to have a health product approved for sale in Canada

55%

20%

25%

Guidance for manufacturers of health products on how to comply with the submission requirements

55%

27%

19%

Information on Health Canada's regulatory compliance and enforcement activities, including inspection, investigations, and verifications, in their industry

49%

32%

19%

Source: Survey of industry.
Note: For purpose of this analysis "health product industry respondents" includes all respondents who self-identified as being involved in at least one of the following industries: pharmaceuticals, biologicals/radiopharmaceuticals, medical devices, natural health products, veterinary drugs.
Row totals may not sum to 100% due to rounding.

Overall, the majority of survey respondents assessed their organizations as having a moderate or poor understanding of BSE/TSE. Furthermore only one-quarter believes that their organization has a strong understanding of Health Canada's BSE/TSE-related policies and regulations affecting their industry -- even though more than half had received information from Health Canada on the subject. See Table 13 for additional information.

Table 13: Industry survey respondents' ratings of level of understanding in their organization with respect to BSE/TSE
BSE/TSE Understanding Percent of respondents (n=117)
Rated level of understanding
Strong Moderate Poor

BSE/TSE-related risks affecting their industry

31%

39%

30%

BSE/TSE risk management/control measures for their industry

30%

40%

30%

Health Canada's BSE/TSE-related policies and regulations affecting their industry

24%

39%

38%

BSE/TSE science/research

14%

46%

40%

Source: Survey of industry.
Note: Row totals may not sum to 100% due to rounding.

Respondents involved in the health products industry reported somewhat higher levels of understanding within their organizations of BSE/TSE-related information pertinent specifically to them. About 40% reported strong understanding of Health Canada's submission requirements for health products, while 32% reported strong understanding of its regulatory compliance and enforcement activities. However, in both cases, a substantial proportion said this information is not applicable to their industry or organization. Caution should be used when interpreting these results due to small sample size. See Table 14.

Table 14: Health product industry respondents' ratings of level of understanding in their organization with respect to BSE/TSE
Regulatory Understanding Percent of health product industry respondents (n=75)
Rated level of understanding
Strong Moderate Poor Not applicable

Health Canada's submission requirements for health products, related to identification of animal-sourced ingredients

41%

25%

9%

24%

Health Canada's regulatory compliance and enforcement activities, including inspection, investigations, and verifications, in their industry

32%

28%

21%

19%

Source: Survey of industry.
Note: For purpose of this analysis "health product industry respondents" includes all respondents who self-identified as being involved in at least one of the following industries: pharmaceuticals, biologicals/radiopharmaceuticals, medical devices, natural health products, veterinary drugs.
Row totals may not sum to 100% due to rounding.

Finally, just over one-third (36%) of industry survey respondents said that Health Canada's responsibilities with respect to BSE/TSE are clear to them; many said these responsibilities are not clear (38%) or did not know (27%).

Key informants representing Health Canada expressed two points of view in response to these findings. On the one hand, they questioned the applicability of the results to their particular industry, suggesting that the sponsors they work with are well aware of the relevant requirements. On the other hand, some acknowledged that industry may not have a clear understanding of the BSE/TSE regulatory framework affecting them, due to the lack of specific policies or guidance documents on the subject and/or a lack of transparency regarding Health Canada's position. It was also noted that regulatory issues are very complex, particularly with respect to health products requiring pre-market review, and that Health Canada could do more to educate industry. Finally, it was noted that regulation of health products is very different from regulation of food products. Whereas SRM are prohibited from use in food products, the same is not true for most health products (natural health products and cosmetics are exceptions). In a scenario in which every product submission is subject to risk assessment, it was suggested, achieving complete transparency can be difficult.

c. Improved adherence to acts, regulations, and other guidance documents
Evaluation Question:
• To what extent is there increased adherence to Acts, regulations and other guidance documents by industry?
Indicators:
• Number of compliance and enforcement actions.

• Extent of voluntary actions by industry and timing of actions.

• Changes in general manufacturing practices for handling BSE risks.
Rating:
• Insufficient evidence to support conclusion.
Summary:
• The evaluation could not determine whether this outcome has been achieved. There has never been any systematic attempt to verify and authenticate the claims made by industry through the product submission process and Health Canada has no objective information on industry compliance for health products industries.

Due to a lack of data, the evaluation could not draw conclusions on the extent to which industry is compliant with Health Canada's BSE/TSE regulatory framework. With respect to regulated health products subject to pre-market approval, it could be argued that, barring fraudulent applications, industry compliance with the regulatory framework is 100%, at least to the extent that these products are not approved unless sponsors provide the information required by the product submission process and this information has been assessed by Health Canada. Several Health Canada key informants noted that compliance is presumed to be high because firms in the regulated industries produce high-value products and are motivated by a desire to protect their corporate reputation. In other words, there are substantial risks to industry associated with non-compliance.

That being said, it appears that there has never been any systematic attempt to verify and authenticate the claims made by industry through the submission process. While HPFBI key informants reported that to their recollection, there have been no inspections where BSE-related violations were found, nor has the Inspectorate ever issued a product recall based on a BSE issue, this impression could not be confirmed with administrative data. Furthermore, as reported elsewhere in this report, despite some early planning activity, HPFBI does not appear to have fully implemented inspection activities specifically related to BSE/TSE. Finally, in the absence (in some cases) of clear policies or guidance for industry on managing BSE/TSE risks, it is not immediately obvious what industry is expected to be compliant with, and by extension, what an inspection program might look for.

With respect to food products, an inspection program is operated by the CFIA.  According to key informants, efforts are underway to develop an information-sharing agreement between Health Canada and the CFIA that would enable Health Canada to access CFIA compliance data, which it requires to fully understand industry compliance with the SRM removal policy and update its risk assessments with respect to food products. Although Health Canada has a mandate to assess the effectiveness of the CFIA's activities related to food safety, key informants noted that in practice such assessments are difficult to do.

In spite of a lack of compliance information, there is some evidence of voluntary changes by industry to respond to BSE/TSE-related risks. For example, BGTD reported that between January 1, 2003 and June 10, 2011:

  • There were 3,240 No Objection Letters issued for Notifiable Change submissions (general manufacturing changes).
  • Of this total, 10 submissions (0.003%) were approved for a non-irradiated fetal bovine serum (FBS) to an irradiated FBS.
  • 48 submissions (1.48%) were approved to change an excipient or raw material from an animal-sourced product to a non-animal-sourced product.

These data support the point made by BGTD key informants that the biologics industry is voluntarily moving toward animal-free and lower-risk ingredients. However, similar data were not available from the other directorates. The industry survey found that of 105 respondents representing companies, 38% (n=40) had made changes in the last 10 years to respond to BSE-related risks. Of these, 20 reported that these changes were made voluntarily (including one respondent in the food industry and 19 respondents in all other industries), 14 said the changes were the result of a compliance or enforcement action by Health Canada or the CFIA (including seven respondents in the food industry and seven respondents in all other industries), and 6 did not know why the changes were made. Based on these limited data, it is not possible to draw any conclusions regarding the frequency of voluntary actions by industry, beyond observing that voluntary changes are being made by some firms.

Case study of a firm in the Canadian beef industry

Prior to the May 2003 discovery of the Canadian cattle infected with BSE, Firm A had already taken some action to pre-emptively address BSE-related risks. The company has been participating in the national BSE surveillance program since the program was introduced in 1992. The company also complied with the regulations introduced in 1997, banning the use of rendered animal proteins of ruminant origin (excluding milk, blood, and fat) from feed for ruminants. Furthermore, during the 1998-1999 European BSE outbreaks, the company created and implemented procedures that required producers dealing with the company to sign documents verifying that they were compliant with the ruminant-to-ruminant feed ban. The firm representative reported that this latter measure was entirely voluntary on the part of the company, and that it was motivated by a desire to protect its customers and beef herd. Upon implementation of the SRM regulation, Firm A complied with it as quickly as possible. The firm's representative stated that Health Canada first provided it with the complete list of SRM, and then all stakeholders, including Health Canada, the CFIA, and the industry at large, communicated with one another to determine the best way to apply the new regulations.

d. Increased expertise and knowledge of BSE/TSE science and risk, and increased knowledge-based decision-making
Evaluation Questions:
• To what extent is there increased expertise and knowledge of BSE/TSE science and risk within Health Canada?

• To what extent is there increased knowledge-based decision-making?
Indicators:
• Extent of partners' participation in national and international expert bodies, conferences, and training.

• Number/nature of publications related to BSE/TSE by Health Canada.

• Number/nature of initiatives stemming from joint agreements.

• Internal and external stakeholders' perceptions of changes in level of expertise/knowledge.

• Extent to which Health Canada decisions are informed by scientific evidence/risk assessment.

• Internal and external stakeholders' perceptions of changes in extent to which Health Canada's decision-making is informed by scientific evidence/risk assessment.
Rating:
• Some evidence of progress; insufficient to support firm conclusion.
Summary:
• The evaluation found that Health Canada's expertise and knowledge of BSE/TSE science and risk has increased over the past decade. Internal and external key informants generally believe that, as a corollary, knowledge-based decision-making has also increased. However, due to a lack of documentation describing the basis for various Health Canada policies and/or decisions, the evaluation had difficulty assessing the relative weight given to scientific knowledge versus other factors in the Department's decision-making process.

The evaluation found that Health Canada's expertise and knowledge of BSE/TSE science and risk has increased over the past decade. Internal and external key informants generally believe that, as a corollary, knowledge-based decision-making has also increased. However, in most cases, the evaluation could not confirm this opinion through other lines of evidence.

Internal and external key informants generally believe that expertise and knowledge of BSE/TSE science and risk management has increased within Health Canada over the past decade. Internal key informants noted that, particularly in the early years of the BSE Initiatives, Health Canada's knowledge of BSE/TSE science and risk was -- like the field of BSE/TSE science itself -- in its infancy. In this context, the TSE Secretariat played an important role as a clearinghouse or filter for information related to BSE/TSE in what was at that time a rapidly developing scientific field. The TSE Secretariat's BSE/TSE related documents (see p. 36); the targeted research projects undertaken by the TSE Secretariat, the FD, and BGTD (see section 4.2.3); and the two TSE research meetings organized in 2005 and 2006 (Health Canada, 2005g; Health Canada, 2006e), were also means by which Health Canada's knowledge of BSE/TSE science increased.Footnote 29

Key informants considered the increase in Health Canada's expertise and knowledge to have paralleled the growth of the field of BSE/TSE science over the past decade. As the field has grown, Health Canada's scientific expertise likewise has become more internally specialized and diverse. Similarly, while both internal and external key informants believe Health Canada has always taken an evidence-based and risk-based approach to decision-making, regardless of the field, they also noted that the scientific evidence base for BSE/TSE-related decision-making is now much stronger than it was 10 years ago and that Health Canada has become more sophisticated in its approach to risk assessments.Footnote 30 Key informants observed that there is now more information available on which to base decisions. In that sense, they considered that knowledge-based decision-making within Health Canada has increased.

The evaluation found a few examples of Health Canada policies whose basis in scientific knowledge was documented. The development of the SRM removal policy for food and the changes to the blood donor deferral policy, both of which were described above, are two such examples. However, due to a lack of documentation describing the basis for various Health Canada policies and/or decisions, the evaluation had difficulty assessing the relative weight given to scientific knowledge versus other factors in the Department's decision-making process. For example, it is not clear what factors led Health Canada to decide that an explicit policy on reducing BSE/TSE-related risks was not required for human and veterinary drug products, or that the guidance for industry on reducing BSE/TSE-related risks in veterinary drugs was no longer required. As discussed in section 4.2.1, it is also not clear that the decision-making process originally described in the TSE Action Plan was used on an ongoing basis to make decisions with respect to BSE/TSE regulation and policy under BSE I and II.

A few external key informants wondered whether there is still a need, given developments in the field of BSE/TSE science, for Health Canada to maintain in-house scientific expertise related to BSE/TSE. They noted that over the past decade, PrioNet Canada has assumed some of the role that was originally envisioned for Health Canada, particularly in terms of BSE/TSE research, and has fostered the development of a large pool of experts in the field. These key informants suggested that the existence and growth of PrioNet may have removed the need for Health Canada to maintain internal scientific expertise related to BSE/TSEs. However, most external and internal key informants agreed that, as a science-based regulator, it is essential for Health Canada to maintain internal scientific expertise as well as arms-length impartiality.Footnote 31 Without such expertise, key informants were concerned that Health Canada would be ill-equipped to conduct meaningful risk assessments and make appropriate policy decisions. To that end, key informants believe that Health Canada has an ongoing role in setting research priorities, conducting secondary research such as monitoring BSE/TSE science in the scientific literature and international fora, and in conducting primary research related to products and processes, including research intended to develop the expertise necessary to conduct risk assessments.

e. Internationally harmonized standards and regulations addressing BSE/TSE and related risks
Evaluation Question:
• To what extent are there internationally harmonized standards and regulations addressing BSE/TSE and related risks?
Indicator:
• Extent to which BSE/TSE regulatory framework is harmonized with those in other countries.
Rating:
• Achieved.
Summary:
• The evaluation evidence suggests that Canada's BSE/TSE regulatory framework is reasonably well aligned with that of other jurisdictions.

The evaluation evidence suggests that Canada's BSE/TSE regulatory framework is reasonably well aligned with that of other jurisdictions. Health Canada and its directorates participate in a variety of collaborative efforts to harmonize standards and regulations in areas covered by the BSE Initiatives. For example:

  • VDD participates as an observer with CFIA's Veterinary Biologics Section (VBS) in the International Cooperation on Harmonisation of Technical Requirements for Registration of Veterinary Products (VICH).
  • Health Canada participates as a member of the Steering Committee for the International Conference on Harmonisation of Technical Requirements for Registration of Pharmaceuticals for Human Use (ICH) and the Global Harmonization Task Force (GHTF).Footnote 32
  • Canada is one of the member countries of the International Regulatory Cooperation for Herbal Medicines (IRCH) (WHO, 2011); HPFB participated as the Preliminary Secretariat for the IRCH for a two-year term (Health Canada, 2008).
  • On March 20, 2007, HPFB signed a Memorandum of Understanding (MOU) with the European Directorate for the Quality of Medicines and HealthCare (EDQM) (Health Canada & EDQM, 2007). One of the objectives of the MOU was to work towards the integration of Certificates of Suitability (CEPs) with the Canadian drug approval process (Health Canada, 2007g); among other things, CEPs certify that products with a risk of transmitting TSEs are compliant with the monographs of the European Pharmacopoeia (Health Canada & EDQM, 2007, p. 2).

Given their complexity, it was beyond the scope of the evaluation to conduct an exhaustive review of international BSE/TSE regulatory frameworks for all types of products implicated in BSE I and BSE II. That being said, examination of these frameworks in Canada and other jurisdictions reveals a considerable degree of harmonization among them, as summarized below.

Food products

Two common mechanisms for controlling BSE-related risks in food products are SRM removal policies and feed bans. Canada, the United States (US), and the European Union (EU) have similar regulations in place requiring the removal of SRM from the human food supply. However, the definition of SRM varies slightly by jurisdiction. For example, Canadian regulations require the tonsils of cattle aged 30 months or older to be removed (Government of Canada, 2011c), whereas US and EU regulations require the tonsils of all cattle to be removed (US FDA, 2005; USITC, 2008). Similarly, Canada and the US both require the removal of the distal ileum (part of the small intestine) of cattle of all ages, whereas the EU requires the complete removal of all gut tissue from the stomach to the rectum from all cattle. All three jurisdictions also have feed bans in place, though the requirements vary considerably among jurisdictions (CFIA, 2010; USITC, 2008; EC, 2007).

Therapeutic products

The European Commission requires that all animal by-products used as starting materials in the manufacture of medicinal products should be Category 3 (i.e., safe) materials or equivalent; the use of substances from other, high infectivity materials must be justified by an appropriate benefit/risk evaluation (EU, 2011). The EC also stipulates that where there is a choice, animals should be sourced from countries with the lowest possible BSE risk (negligible risk countries) unless the use of materials from countries with a higher BSE risk is justified (EU, 2011). In Canada, as described in section 4.2.3, sponsors of therapeutic products (including human pharmaceuticals, biologics, medical devices, and veterinary drugs) are required to provide information about animal ingredients, including how they are used in the product, the species of animal they were sourced from, the tissues or fluids of origin, the age of the animal, and the country of origin of the animal. However, unlike the EU, Canada does not have an explicit policy or guidance for manufacturers regarding minimizing the risk of transmitting BSE/TSEs via therapeutic products.

The position of the US with respect to therapeutic products is unclear; the evaluation located a 2007 proposed rule by the FDA that suggests banning certain cattle material in, or in the manufacture (including processing), of medical products for humans and in drugs intended for use in ruminant animals (US FDA, 2007a), but it is not clear if this proposed rule has been implemented.

Blood and blood products

Mechanisms for controlling TSE-related risks in these products include blood donor deferral policies, use of leukodepletion, and recall of blood and blood products traced to donors who later developed vCJD. There is some similarity among donor deferral policies among jurisdictions. For example, the policies for both Canada and the US stipulate exclusion of donors spending a cumulative total of three or more months in the UK between 1980 and 1996 (Wilson et al, 2001; US FDA, 2010). However, it is not clear if the US has updated its policy to exclude donors from other Western European nations and Saudi Arabia, as Canada did in 2005 and 2011, respectively.

Universal leukoreduction (removal of white blood cells) is currently practiced in Canada and the UK (Sher, 1998; UK Blood Transfusion Services & National Institute for Biological Standards and Control, 2010, pp. 3-4),Footnote 33 but appears not to be applied in the US on the grounds that its effectiveness in reducing the transmission risk of vCJD is unproven (US FDA, 2010, p. 15). The position of the EU on leukoreduction of blood and blood products is unclear, but the European Medicines Agency's (EMA) most recent position statement on vCJD in plasma-derived and urine-derived medicinal products does recommend leukoreduction for plasma-derived products (Committee for Medicinal Products for Human Use, 2011, pp. 5-6).

When information is received that a past donor has been diagnosed with vCJD, it is policy in Canada, the US, and the UK alike to recall and destroy any blood products (including plasma derivatives) associated with that donor (PHAC, 2003; UK Blood Transfusion Services & National Institute for Biological Standards and Control, 2010, pp. 3-4; US FDA, 2010, pp. 29-34). This practice is also recommended in the EU as well as by WHO (Committee for Medicinal Products for Human Use, 2011; WHO, 2006, pp. 13-15).

Natural health products

Assessing the level of harmonization of standards/regulations for natural health products is complicated by the range of definitions of these products across jurisdictions, as well as the variety of structures used to divide responsibility for these products between government departments. For example, products described as natural health products in Canada appear to be generally classified as "dietary supplements" in the US (Rudge, 2005, p. 10); in other jurisdictions, such as the EU, the regulatory framework for these products is very similar to or, in the latter case, almost indistinguishable from other therapeutic products (WHO, 2001, p. 156).

However, there are a number of broad similarities between the standards and guidelines used in Canada and those used in other jurisdictions. For example, the country of origin of animals used to obtain materials for manufacturing is one of the factors considered by NHPD in issuing a product licence for a natural health product (Health Canada, 2004a). Similarly, the US and the EU take country of origin into consideration in their regulations/guidelines for natural health products (European Union [EU], 2011, pp. E9-E10; US FDA, 2004a, p. 42260), and WHO emphasizes the importance of adequate documentation of the source of animal-derived materials used in manufacturing homeopathic preparations (WHO, 2009, pp. 9-10).

In addition, restrictions against the use of highly infective animal tissues constitute an important role of the guidelines and/or regulatory frameworks established in each of the jurisdictions in the scan. For example, Canadian natural health products manufacturers are prohibited from using SRM (Health Canada, 2006b) and from using gelatin produced from animal skulls or vertebral columns (Health Canada, 2006c), and the FDA has ruled against the use of high-risk bovine materials in production (US FDA, 2007b, p. 34839). Similarly, the EU guidelines aimed at reducing TSE transmissibility from medicinal products stipulate that high infectivity materials including central nervous system (CNS) tissue and tissue anatomically associated with the CNS must not be included in manufacturing unless justified by a benefit/risk evaluation (EU, 2011, pp. E9-E10).

f. Reduced risks and safer food and health products
Evaluation Questions:
• To what extent is there reduced exposure to the risks associated with the use of animal-sourced materials in food and health products regulated by Health Canada?

• To what extent is there reduced risk of acquiring human TSEs associated with animal-sourced ingredients in food and health products regulated by Health Canada?

• To what extent are food and health products safer?
Indicators:
• Number/type of animal-sourced materials prohibited for use in food and health products regulated by Health Canada.

• Perceived level of risks by experts/stakeholders.

• International OIE designation of BSE risk in Canada.

• Annual number of food and health products with bovine sourced ingredients prevented from entering the market; removed from the market; and prohibited for use.

• Number of food and health products on the market that were reassessed.

• Annual number of documented human TSE cases originated in Canada.

• Number of reported BSE/TSE risks associated with food and health products.

• Level of public confidence in safety of food and health products.
Rating:
• Some evidence of progress; insufficient to support firm conclusion.
Summary:
• Health Canada's BSE/TSE risk management measures are presumed to have reduced risks and led to safer food and health products, but there is insufficient data to support a definitive conclusion on this question. However, from a methodological point of view it is extremely difficult to test the effectiveness of risk management or risk mitigation strategies. PHAC's Creutzfeldt-Jakob Disease Surveillance System (CJDSS) has found two cases of vCJD in Canada (2002 and 2009), both of which are recognized as having been acquired outside of Canada; to date, no indigenous cases of vCJD have been found. CJDSS data also indicated that Canada does not have transfusion-related CJD.

The BSE Initiatives were intended to reduce exposure to the risks associated with the use of animal-sourced materials in food and health products regulated by Health Canada, and reduce the risk of acquiring human TSEs associated with these ingredients, ultimately leading to safer food and health products. There is general agreement among both internal and external key informants that Health Canada's risk management and control measures have contributed to achieving these outcomes, though several also observed that the reduction in risk has been marginal since the risks were very low to begin with.Footnote 34

Although, there is insufficient data to support a definitive conclusion on the question, the following facts are known:

  • BSE risks associated with the human food supply have been reduced through prohibitions on the use of SRM in food, animal feed, pet food, and fertilizers, all of which are internationally acknowledged as effective measures for reducing BSE-related risks to humans and cattle.
  • With the exception of natural health products, SRM have not been officially banned from health products regulated by Health Canada and the directorates involved in regulating these products (BGTD, TPD, and VDD) do not have explicit policies or guidance for industry on reducing BSE/TSE-related risks.Footnote 35 However, all new product submissions are screened for the presence of high-risk materials and products containing these materials are subject to risk assessment. That being said, there has apparently been no systematic attempt to verify the information provided by sponsors through the submission process, and Health Canada has no information on industry compliance with the regulatory framework.
  • Data from PHAC's Creutzfeldt-Jakob Disease Surveillance System (CJDSS) show that the incidence rate of CJD deaths in Canada is similar to the worldwide rate of 1-2 per million population (PHAC, 2012). The number of deaths due to definite and probable CJD has trended upward in the years for which data are complete (1999 to 2009), but this pattern is very likely due to enhanced surveillance, rather than an increase in deaths per se. Furthermore, the vast majority of deaths to date have been due to sporadic CJD, that is, CJD whose cause remains unknown.
  • CJDSS data show that to date, four deaths have been due to iatrogenic CJDFootnote 36, and two deaths have been due to vCJD.Footnote 37 According to PHAC, neuropathological examination revealed that all four of the deaths from iatrogenic CJD were due to dura mater grafts, the use of which in neurosurgical procedures is now banned in Canada. The two deaths due to vCJD (in 2002 and 2009) were both cases acquired outside of Canada. To date, no indigenous cases of vCJD have been reported.
  • Based on CJDSS data, Canada does not appear to have either transfusion-related CJD (acquired through tainted blood) or domestically acquired vCJD (acquired through eating tainted beef).
  • Data from the European Creutzfeldt-Jakob Disease Surveillance Network (EUROCJD) show that the worldwide incidence rate of vCJD has been declining since 2000 (EUROCJD, 2012).
  • Canada is recognized by the OIE as a "controlled BSE risk" country (OIE, 2011) and its prevention and eradication methods are acknowledged as effective by the OIE. This has led to 19 confirmed cases of BSE in Canadian-born cattle as of March 2011 (CDC, 2012).

The evaluation framework for the BSE Initiatives also identified beef consumption patterns and consumer confidence levels as indicators of the level of BSE-related risks. Beef consumption in Canada did not decline significantly following the discovery of BSE in a domestic cow (AAFC, 2011); and Health Canada's public opinion research, conducted between 2003 and 2005, showed that Canadians appeared unconcerned about contracting BSE through tainted meat or other transmission channels, and that most consumers perceived beef as safe even after the second and third domestic cases of BSE (Health Canada, 2004c, 2004d, 2004e, 2004f, 2005h, 2005i). However, these measures are not particularly strong indicators of the level of actual risk, though they are certainly appropriate indicators of the level of perceived risk.

In short, Health Canada's BSE/TSE risk management measures are presumed to have reduced risks and led to safer food and health products, but there is insufficient data to support a definitive conclusion on this question. In any case, as described in the methodology section of this report, from a methodological point of view it is extremely difficult to test the effectiveness of risk management or risk mitigation strategies, necessitating a reliance on indirect and/or qualitative data.

4.2.5 Unintended consequences

Evaluation Question:
• Were there any unintended consequences as a result of the BSE I and BSE II Initiatives?
Indicator:
• Unintended consequences identified by internal and external stakeholders.
Rating:
• Not applicable.
Summary:
• The evaluation found that the main unintended consequences of the Initiatives were economic.

The main unintended consequences of the BSE Initiatives appear to be economic.Footnote 38 A minority of industry survey respondents (18%) identified negative unintended consequences of the BSE Initiatives for their firm or industry, including increased capital and labour costs, reduced ability to produce certain products, inability to export to certain markets, reduced revenues, and difficulties sourcing ingredients that comply with Health Canada regulations. Key informants also reported negative economic consequences of the SRM removal policy on the Canadian beef industry, but on the positive side, identified increased awareness of BSE/TSE-related risks and the rapid recovery of Canada's international markets for beef products.

4.2.6 Efficiency and economy

Evaluation Questions:
• Were resources deployed at the least cost, consistent with realizing timely outputs that met the requirements of the Initiatives?

• Did the outputs of the Initiatives meet needs at the lowest cost?

• Are there alternate ways to deliver the Initiatives to achieve similar results at lower cost?
Indicators:
• Comparison of planned versus actual spending and explanations for variances.

• Extent to which resource allocation processes are documented and understood.

• Enumeration of outputs produced.

• Internal and external stakeholder assessment of quality and usefulness of outputs.

• Approaches used in other countries.

• Internal and external stakeholder assessment of other options for delivery.

• Internal and external stakeholder assessment of appropriateness and adequacy of Canada's approach to BSE/TSE risk management.
Rating:
• Insufficient evidence to support conclusion.
Summary:
• Actual spending was 64% of planned spending for BSE I and only 47% of planned spending for BSE II. Because Health Canada did not implement all of the BSE/TSE Initiatives as planned and a significant amount of BSE funding was reallocated to other priorities, an assessment of efficiency and economy is difficult, if not impossible. Canada's response to BSE/TSE generally is consistent with international approaches. However, due to the absence of a Department- or Branch-level policy on reducing BSE/TSE-related risks and the lack of evidence demonstrating inspection activities for health products regulated by Health Canada, the appropriateness of Health Canada's response needs attention. The evaluation found general support in the literature and from key informants for continued vigilance and involvement on the part of Health Canada, in light of significant ongoing scientific uncertainties and emergent risks related to BSE/TSEs.

Information on planned and actual spending, provided by the Strategic Planning and Accountability Division within the RMOD, indicate that both BSE I and BSE II reallocated a significant amount of funds. As shown in Tables 15 and 16 below, actual spending was 64% of planned spending for BSE I and only 47% of planned spending for BSE II. Similar findings were made by the OAG in its November 2006 report, which found that within Health Canada, and specifically within HPFB and HECSB, funds for special initiatives were not always spent for the purposes approved by the Treasury Board (OAG, 2006). The OAG report recommended that Health Canada monitor sources of program funding to ensure that resources are allocated to the intended purposes and also monitor the impact of reallocations to ensure that ability to meet program objectives is not compromised.

Table 15: Planned versus actual spending for BSE I
Organization Planned Actual Variance

Product Assessment

BGTD

509,446.00

867,058.87

-357,612.87

FOOD

3,098,408.00

2,521,620.00

576,788.00

TSE Secretariat

1,510,628.00

785,866.93

724,761.07

NHPD

478,270.00

52,478.77

425,791.23

TPD

465,110.00

330,230.85

134,879.15

VDD

548,976.00

609,073.99

-60,097.99

Total Product Assessment

6,610,838.00

5,166,329.41

1,444,508.59

Tracking and Tracing

BGTD

408,473.00

45,300.00

363,173.00

NHPD

476,270.00

-

476,270.00

TPD

261,624.00

170,609.96

91,014.04

VDD

333,172.00

-

333,172.00

Total Tracking and Tracing

408,473.00

45,300.00

363,173.00

Compliance and Enforcement

HPFBI

  708,968.00

  255,668.97

  453,299.04

Total Compliance and Enforcement

  708,968.00

  255,668.97

  453,299.04

Total

8,799,345.00

5,637,908.34

3,161,436.66

Sources: Strategic Planning and Accountability Division, Resource Management and Operations Directorate and Food Directorate.

Notes:

  1. The planned amounts stated in the Tables represent net figures and do not take into account the corporate levies that are applied for various corporate support related to the Initiatives.
  2. For both planned and actuals, the amounts shown below includes the sum of regular salaries, students, O&M and capital. EBP and uncontrollable salaries are not included.
  3. Student expenditures are recorded under O&M for this exercise.
Table 16: Planned versus actual spending for BSE II
Organization Planned Actual Variance
Risk Assessment

BGTD

5,024,450.00

1,261,919.52

3,762,530.48

FOOD

6,504,551.00

2,748,721

3,755,830.00

NHPD

2,262,930.00

396,854.39

1,866,075.61

TPD

3,689,102.00

2,782,504.18

906,597.82

VDD

2,969,139.00

67,398.72

2,901,740.28

Total Risk Assessment

20,450,172.00

7,257,397.81

13,192,774.19

Targeted Research
BGTD

1,997,782.00

1,395,510.09

602,271.91

FOOD

1,236,487.00

2,262,214

-1,025,727.00

NML (PHAC)

2,319,844.00

1,487,000.00

832,844.00

Total Targeted Research

5,554,113.00

5,144,724.09

409,388.91

Evaluation

PSPD (now RMOD)

1,084,418.00

336,191.06

748,226.94

Total Evaluation

1,084,418.00

336,191.06

748,226.94

 

Total

27,088,703.00

12,738,312.96

14,350,390.04

Sources: Strategic Planning and Accountability Division, Resource Management and Operations Directorate and Food Directorate.

Notes:

  1. The planned amounts stated in the Tables represent net figures and do not take into account the corporate levies that are applied for various corporate support related to the Initiatives.
  2. For both planned and actuals, the amounts shown below includes the sum of regular salaries, students, O&M and capital. EBP and uncontrollable salaries are not included.
  3. Student expenditures are recorded under O&M for this exercise.

Since Health Canada did not implement all of the BSE/TSE Initiatives as planned and a significant amount of BSE funding has been reallocated, an assessment of efficiency and economy is difficult if not impossible.Footnote 39 That being said, external key informants were generally of the view that Health Canada's response to BSE/TSE to date has been appropriate and adequate to manage the related risks, and the review of international approaches to minimizing BSE/TSE-related risks did not find any other approaches that are radically different from what has been implemented in Canada, though the details may differ (e.g., definitions of SRM may be more or less inclusive).Footnote 40 On the other hand, given the failure to implement the program as planned, the appropriateness and adequacy of Health Canada's response should be revisited. In particular, Health Canada has no overall BSE/TSE risk reduction policy and no objective information on industry compliance with the BSE/TSE regulatory framework for health products industries.

The evaluation found general support in the literature (summarized in section 4.1.1) and from key informants for continued vigilance and involvement on the part of Health Canada, in light of a changing BSE and prion disease profile in Canada, and in light of significant ongoing scientific uncertainty related to, for example, TSEs other than BSE, especially CWD, atypical BSE, and emergent risks with respect to human-to-human transmission.

However, some key informants questioned the overall funding levels and/or the allocation of BSE/TSE-related funding among federal departments. They suggested that some activities for which Health Canada received funding through the BSE Initiatives have become routine aspects of directorate activity, in particular, requiring sponsors to provide information on animal-sourced ingredients through the product submission process. Consequently, this activity may no longer require the same level of funding. On the other hand, most key informants agreed that ongoing funding is required for BSE/TSE research, and that departments other than Health Canada, including PHAC, have an important role to play in this regard. Key informants suggested that Health Canada's role could include setting research priorities, conducting secondary research such as monitoring BSE/TSE science in the scientific literature and international fora, and conducting primary research related to products and processes, including research intended to develop the expertise necessary to conduct risk assessments.

5.0 Conclusions and recommendations

This section of the report summarizes the main findings from the evaluation, draws conclusions, and makes recommendations.

Relevance

Ongoing need

The evaluation confirmed an ongoing need for intervention to manage the risks to human health associated with BSE/TSEs. Based on a review of the scientific literature, there clearly remain many unknowns and uncertainties in the field of BSE/TSE science, with important future implications for public health. These include ongoing uncertainty with respect to human-to-human transmission; atypical BSE; and TSEs other than BSE, such as CWD -- a uniquely North American phenomenon that may require Canada to be the first jurisdiction to develop an appropriate policy and regulatory response. The potential health impacts stemming from the greater risk of exposure to CWD by some Canadians, including First Nations and Inuit populations, and sport and subsistence hunters, further implicate Health Canada in ongoing efforts to understand and manage the associated risks.

Recommendation 1: Health Canada should continue to play a role in BSE/TSE risk assessment and research to inform policy and regulatory development. Particular attention should be paid to the evolving science on CWD, given its potential health implications for all Canadians.

Alignment

The BSE Initiatives are clearly aligned with Health Canada's roles and responsibilities as described in federal statutes and regulations, in particular the Food and Drugs Act and Regulations and the Department of Health Act, and an ongoing role for the department in managing BSE/TSE-related risks seems warranted on this basis. The evaluation also found that the BSE I and II Initiatives are aligned with government priorities to strengthen food safety expressed in the 2009 Budget and 2010 Speech from the Throne. Similarly, the Initiatives align directly with Health Canada's strategic outcome of Access to Safe and Effective Health Products and Food and Information for Healthy Choices, as well as (though perhaps more loosely) with the strategic outcomes of Reduced Health and Environmental Risks from Products and Substances, and Healthy, Sustainable Living and Working Environments.

Performance

Governance

The evaluation found that governance of the BSE I and II Initiatives had several weaknesses that likely affected program implementation. There was, for example, limited evidence of collaboration and coordination among the various internal and external partners involved in the Initiatives, and the TSE Science and Policy Teams, which were envisioned as playing a key role in decision-making related to BSE/TSE, do not appear to have fulfilled that role in practice. Furthermore, the TSE Secretariat, which was intended to coordinate all of Health Canada's BSE/TSE activities, lacked a coherent profile within Health Canada, having been transferred several times among directorates and experiencing some variations over time in its articulated responsibilities. While the Secretariat performed an important role in the early years of the government's response to BSE/TSE, particularly in monitoring and disseminating scientific information during what was at the time a rapidly changing field, its current role is not clear, particularly given the maturation of the field and Health Canada's more sophisticated understanding of BSE/TSE science and policy.

Recommendation 2: Health Canada should consider whether there is still a necessary role for the TSE Secretariat, internally and externally, in coordinating the federal government's overall approach to BSE/TSE.

Finally, although performance measurement frameworks were developed, performance measurement and reporting did not occur, possibly because of a lack of clarity regarding who (i.e., the TSE Secretariat or PPIAD) was to be responsible for these activities. As a consequence, performance measurement data pertaining to outcomes is virtually non-existent, with significant repercussions for the ability of the evaluation to draw conclusions on the extent to which outcomes have been achieved. Given that the federal government is continuing to fund Health Canada for BSE-related activities through BSE III, some clarification of these roles and responsibilities seems warranted.

Recommendation 3: Health Canada should take steps to ensure that performance measurement takes place for BSE III and for future funded initiatives, including clarifying internal roles and responsibilities for coordinating performance measurement and reporting.

Program design

The evaluation evidence suggests that the design of the BSE I and BSE II Initiatives was based on the scientific evidence available at the time, and was informed by risk-based analysis and some consultation with stakeholders. To that extent, the program was designed appropriately to achieve its expected outcomes. On the other hand, the evaluation also found that some Health Canada partners whose mandated responsibilities would seem to extend to BSE/TSE risk management and control efforts were not included in the Initiatives. These include the CPSD, which is responsible for regulating cosmetics and personal care products, and the MHPD, which is responsible for surveillance activities for health products regulated by Health Canada.

While the evaluation did not find any evidence that the exclusion of CPSD and MHPD had a detrimental impact on Health Canada's ability to achieve its expected outcomes under BSE I and II, their exclusion requires attention as part of Health Canada's overall approach to BSE/TSE. With respect to the potential role of MHPD in BSE/TSE-related surveillance, it is important to note that the PHAC was funded for and carries out prospective surveillance of all types of human prion disease through the Canadian Creutzfeldt-Jakob Disease Surveillance System (CJDSS).

Recommendation 4: Health Canada should determine whether the CPSD and the MHPD have a role to play in its overall BSE/TSE strategy. With respect to surveillance activities, due consideration should be given to the role already performed by the PHAC through the CJDSS.

Program implementation

Based on the available evidence, not all of the BSE I and II Initiatives activities were implemented as planned. For instance, the evaluation found that several targeted research projects were undertaken by the Food Directorate (FD) and the Biologics and Genetic Therapies Directorate (BGTD), including some involving international collaboration. However, other activities were not as clearly fulfilled. Although risk assessment, product assessment, and tracking and tracing were conceptualized as distinct activities with discrete funding allocations in the original planning documents, in practice these activities were not and are not necessarily distinguished from one another by all of the directorates that received funding for them. As a result, it was difficult for the evaluators to determine the extent of implementation for these activities during the time period under evaluation. While all four evaluation directorates (BGTD, TPD, NHPD, and VDD) now require sponsors to identify and provide information on animal-sourced ingredients in new drugsFootnote 41 as part of the product application process, most have not published policies and guidance documents for industry pertaining specifically to the reduction of BSE/TSE-related risks, though they may apply internal policies when reviewing product submissions.

Moreover, despite an apparent interest in a Branch-level policy on reducing BSE/TSE-related risks in the products regulated by HPFB, such a policy, though drafted, has never been finalized. While the inability to arrive at a consensus on a Branch-level policy may be a function of limited collaboration and coordination among internal partners in the Initiatives, it could also be indicative of valid differences among the regulated industries that render an overarching policy unrealistic or unfeasible.

Recommendation 5: Health Canada should revisit the feasibility of developing a Department-wide policy on reducing BSE/TSE-related risks for the consumer and health products it regulates.

With respect to compliance and enforcement, the absence of any documentation of inspection activities specifically related to BSE/TSE, and the fact that some of the funds allocated for this purpose were evidently redirected to other Inspectorate activities, suggest that planned compliance and enforcement activities were not fully implemented.

Recommendation 6: Health Canada should take steps to document its inspection activities for BSE/TSE-related risks in health products regulated by HPFB, as well as the outcomes of these inspections (i.e., non-compliances found and actions taken in response to non-compliance).

Outcomes

In the absence of performance measurement data, there is some evidence from other data collection methods that some progress has been made toward expected outcomes. The evaluation found that at least some of Health Canada's regulatory and policy responses -- such as the SRM removal policy and amendments to the blood donor deferral policy -- were timely, based on scientific evidence, and informed by risk assessment as well as consultation with stakeholders. These examples are illustrative of an improved regulatory/policy response to control and prevent risks associated with BSE/TSE. However, based on feedback from some federal partners as well as some respondents to the industry survey, some stakeholders do not believe Health Canada consulted adequately with them when developing its regulatory and policy response.

While internal and external key informants believe that, generally speaking, awareness and understanding of BSE/TSE-related risks has increased in Canada over the past two decades as the field of BSE/TSE science has matured, results from the industry survey suggest opportunities to strengthen awareness and understanding among industry stakeholders. Most notably, even though about half of survey respondents had received information from Health Canada regarding policies and regulations affecting their industry, only one-quarter assessed their organization as having a strong understanding of Health Canada's BSE/TSE-related policies and regulations affecting them. Some internal key informants admitted that industry may not, in all cases, have a clear understanding of the BSE/TSE regulatory framework affecting them, since some of the directorates responsible for regulating health products have not published guidance documents or policies specifically pertaining to BSE, and/or make references on their websites to policies and guidance documents that are not in effect.

Recommendation 7: Health Canada should take steps to improve the transparency of its BSE/TSE regulatory framework for health products, with a view to strengthening industry awareness and understanding.

Within Health Canada, and based primarily on qualitative evidence from internal and external key informants, expertise and knowledge of BSE/TSE science has increased over the past decade, paralleling the growth and diversification of the field over the same period. Similarly, while both internal and external key informants believe Health Canada has always taken an evidence-based and risk-based approach to decision-making, they also noted that the scientific evidence base for BSE/TSE-related decision-making is now much stronger than it was 10 years ago. They believe that in that sense, knowledge-based decision-making within Health Canada has increased. However, due to a lack of documentation describing the basis for various Health Canada policies and/or decisions, the evaluation had difficulty assessing the relative weight given to scientific knowledge versus other factors in the Department's decision-making process. Thus, the evaluation could not draw a definitive conclusion on this question.

Although a few key informants questioned whether there is still a need for Health Canada to maintain internal scientific expertise in BSE/TSE, especially given the existence of PrioNet, most believe that maintaining such expertise is necessary in order to conduct meaningful risk assessments and make appropriate policy decisions.

The evaluation could not determine whether there has been increased adherence to acts, regulations, and other guidance documents on the part of industry. For health products, barring fraudulent applications, compliance is presumed to be 100%; however, there has never been any systematic attempt to verify and authenticate the claims made by industry through the product submission process and Health Canada has no objective information on industry compliance. The absence of specific policies and guidance documents further complicates the picture, since it is not immediately obvious what industry is expected to be compliant with. For food products, an inspection program is operated by the CFIA. Key informants reported that efforts are ongoing to develop an information-sharing agreement between Health Canada and the CFIA. Such an agreement would enable Health Canada to access CFIA compliance data, which it requires to fully understand industry compliance with the SRM removal policy and update its risk assessments with respect to food products.

Recommendation 8: Health Canada should endeavour to finalize an information-sharing agreement with the CFIA in the near future. To that end, an action plan with clear milestones and senior management support should be developed and implemented.

The evaluation found that Canada's BSE/TSE regulatory framework is reasonably well aligned with that of other jurisdictions, and to that extent there are internationally harmonized standards and regulations addressing BSE/TSE and related risks. Health Canada and its directorates participate in a variety of collaborative efforts to harmonize standards and regulations in areas covered by the BSE Initiatives, and review of regulatory approaches to BSE/TSE in several jurisdictions revealed considerable similarity among them, though some of the details vary.

Finally, the BSE Initiatives were intended to reduce exposure to the risks associated with the use of animal-sourced materials in food and health products regulated by Health Canada, and reduce the risk of acquiring human TSEs associated with these ingredients, ultimately leading to safer food and health products. Key informants agreed that risks have been reduced as a result of the implementation of control measures -- singling out in particular the SRM removal policy and the blood donor deferral policy -- although several observed that the decrease associated with these measures has been marginal since the risks were very low to begin with. Methodologically, it is extremely difficult to assess the effectiveness of risk mitigation measures, and there is little objective data to support a definitive conclusion on this question. However, data from PHAC's Creutzfeldt-Jakob Disease Surveillance System indicate that Canada does not have either transfusion-related CJD or domestically acquired vCJD. Moreover, the incidence rate of CJD deaths in Canada is similar to the worldwide rate of 1-2 per million population.

Efficiency and economy

Based on the available financial information, it appears that a significant proportion of BSE funds were reallocated -- 36% for BSE I and 53% for BSE II. This finding is consistent with the findings of the November 2006 report of the Office of the Auditor General, which recommended that Health Canada monitor sources of program funding to ensure that resources are allocated to the intended purposes and also monitor the impact of reallocations to ensure that ability to meet program objectives is not compromised.

Recommendation 9: Health Canada should take steps to improve financial oversight and reporting to ensure that allocated funds are used as planned. If reallocation does occur, appropriate justifications should be documented and monitoring should take place to ensure that program objectives are met.

Because Health Canada did not implement all of the BSE/TSE Initiatives as planned and a significant amount of BSE funding was reallocated, an assessment of efficiency and economy is difficult, if not impossible.

External key informants were generally of the view that Health Canada's response to BSE/TSE to date has been appropriate and adequate to manage the related risks, and the review of international approaches to minimizing BSE/TSE-related risks did not find any other approaches that are radically different from what has been implemented in Canada. However, the absence of a BSE/TSE risk reduction policy for health products at the Department or Branch level and the fact that a specific inspection program related to BSE/TSE was not fully implemented may well be seen as shortcomings.

The evaluation found general support in the literature and from key informants for continued vigilance and involvement on the part of Health Canada, in light of a changing BSE and prion disease profile in Canada, and in light of significant ongoing scientific uncertainty related to, for example, TSEs other than BSE, especially CWD; atypical BSE; and emergent risks with respect to human-to-human transmission. In this context, key informants believe that Health Canada has an important ongoing role to play, particularly in the areas of risk assessment and BSE/TSE research to inform policy and regulatory development.

Appendix A - List of References

  • Ackerman, F., & Johnecheck, W. A. (2008). Mad cows and computer models: The U.S. response to BSE. NEW SOLUTIONS: A Journal of Environmental and Occupational Health Policy, 18(2), 145-156. doi:10.2190/NS.18.2.e
  • Adams, S. (2011, February 3). vCJD blood test developed. The Telegraph.
  • Agriculture and Agri-Food Canada (AAFC). (2011, June 6). Red Meat Market Information.
  • Aiken, J. (2011, February 9). Chronic Wasting Disease Strains. Workshop summary presented at the Management of CWD in Canada: Past practices, current conditions, current science, future risks and options, Edmonton, Alberta.
  • Arnold, M. E., Ryan, J. B. M., Konold, T., Simmons, M. M., Spencer, Y. I., Wear, A., . . . Wells, G. A. H. (2007). Estimating the temporal relationship between PrPSc detection and incubation period in experimental bovine spongiform encephalopathy of cattle. Journal of General Virology, 88, 3918-3208. doi:10.1099/vir.0.82987-0
  • Auditor General of Canada. (2000, December). Report of the Auditor General of Canada. Chapter 20: Managing Departments for Results and Managing Horizontal Issues for Results.
  • Auditor General of Canada. (2006, November). Report of the Auditor General of Canada. Chapter 8: Allocating Funds to Regulatory Programs - Health Canada.
  • Azarpazhooh, A., & Fillery, E. D. (2008). Prion disease: The implications for dentistry. Journal of Endodontics, 34(10), 1158-1166.
  • Bailey, A. (2006). A decade on from discovery: The battle against prions. European BioPharmaceutical Review. Retrieved from www.virusure.com/downloads/publications/14-the-battle.../download.html
  • Belay, E. D., & Schonberger, L. B. (2005). The public health impact of prion diseases. Annual Review of Public Health, 26(1), 191-212. doi:10.1146/annurev.publhealth.26.021304.144536
  • Belay, E. D., Maddox, R. A., Williams, E. S., Miller, M. W., Gambetti, P., & Schonberger, L. B. (2004). Chronic wasting disease and potential transmission to humans. Emerging Infectious Diseases, 10(6), 977-84.
  • Blanchfield, R. (2009, September 25). BSE/vCJD: The European On-going Story. Retrieved from http://www.google.ca/url?sa=t&source=web&cd=1&ved=0CBwQFjAA&url=http%3A%2F%2Fwww.jralphb.co.uk%2FBSE_vCJD_Web.ppt&ei=rgwbTsvHGsXZgQf0psH-Dw&usg=AFQjCNF_XL3eaL2M05XUKyH09M0hOXT8WQ
  • Bradley, R., Collee, J. G., & Liberski, P. P. (2006). Variant CJD (vCJD) and bovine spongiform encephalopathy (BSE): 10 and 20 years on: part 1. Folia Neuropathologica, 44(2), 93.
  • Brown, C. (2010). Mad cow update: Risk now limited. Canadian Medical Association Journal, 182(16), E729.
  • Budka, H. (2008, March 26). Scientific Opening - The Transmissible Spongiform Encephalopathy Saga: Past, Present, and Future. Presented at the seventh session of the Statistics and Risk Assessment Section's International Expert Advisory Group on Risk Modelling: Iterative risk assessment processes for policy development under conditions of uncertainty/emerging infectious diseases: Round IV, Ottawa, Ontario. Retrieved from http://www.phac-aspc.gc.ca/hcai-iamss/sra-ser/seven-sept/pdf/ieag-ccei-07-proceedings-eng.pdf
  • Canadian Food Inspection Agency (CFIA). (2010). Canadian Food Inspection Agency - Canadian Food Inspection Agency Position on Change Over to Specified Risk Material (SRM) Removed Meat and Bone Meal Production in Rendering Plant Operations.
  • Centers for Disease Control and Prevention (CDC). (2010a). Prion diseases.
  • Centers for Disease Control and Prevention (CDC). (2012). BSE (bovine spongiform encephalopathy, or mad cow disease).
  • Clark, C. (1995, August 7). Canadian Red Cross recalls blood tainted with CJD. Blood Weekly, 2.
  • Collee, J. G., Bradley, R., & Liberski, P. P. (2006). Variant CJD (vCJD) and bovine spongiform encephalopathy (BSE): 10 and 20 years on: part 2. Folia Neuropathologica, 44(2), 102.
  • Colloton, M. (2001). Dietary supplements: A challenge facing the FDA in mad cow disease prevention. American University Law Review, 51, 495.
  • Committee for Medicinal Products for Human Use. (2011). CHMP position statement on Creutzfeldt-Jakob disease and plasma-derived and urine-derived medicinal products (No.EMA/CHMP/BWP/303353/2010). London, England: European Medicines Agency.
  • European Commission. (2007). Questions and Answers on BSE.
  • Edgeworth, J. A., Farmer, M., Sicilia, A., Tavares, P., Beck, J., Campbell, T., . . . Jackson, G. S. (2011). Detection of prion infection in variant Creutzfeldt-Jakob disease: A blood-based assay. The Lancet, 377(9764), 487-493.
  • EFSA Panel on Biological Hazards (BIOHAZ). (2011). Joint Scientific Opinion on any possible epidemiological or molecular association between TSEs in animals and humans. EFSA Journal 2011;9(1):1945. doi:10.2903/j.efsa.2011.1945.
  • European Creutzfeld-Jakob Disease Surveillance Network (EUROCJD). (2012). EUROCJD Surveillance Data: vCJD Cases Worldwide (February 2012).
  • European Union (EU). (2005). The TSE Roadmap (Final Report No. COM(2005) 322). Brussels, Belgium.
  • European Union (EU). (2011, July 1). Minimising the risk of transmitting animal spongiform encephalopathy agents via human and veterinary medicinal products.
  • Farrugia, A., Ironside, J. W., & Giangrande, P. (2005). Variant Creutzfeldt-Jakob disease transmission by plasma products: Assessing and communicating risk in an era of scientific uncertainty. Vox Sanguinis, 89(4), 186-192.
  • Government of Canada. (2010, March 3). Speech from the Throne. A Stronger Canada, A Stronger Economy. Now and For the Future.
  • Government of Canada. (2011a). (Current to July 11, 2011). Food and Drug Regulations.
  • Government of Canada. (2011b). (Current to June 22, 2011). Food and Drugs Act.
  • Government of Canada. (2011c). (Current to June 22, 2011). Department of Health Act.
  • Graziano, S., & Pocchiari, M. (2009). Management and prevention of human prion diseases. Current Neurology and Neuroscience Reports, 9(6), 423-429.
  • Health Canada. (n.d.a). Food Directorate - Briefing Book. Subject: TSE Secretariat.
  • Health Canada. (n.d.b.). Health Canada 3011: Drug Submission Application Form for Human, Veterinary, Disinfectant Drugs and Clinical Trial Application/Attestation.
  • Health Canada. (n.d.c). Biologics and Genetic Therapies Directorate (BGTD). Memorandum to the Members of the Branch Executive Committee, Health Products and Food Branch. Subject: Status of TSE Database.
  • Health Canada. (2000). Health Canada Decision-Making Framework for Identifying, Assessing, and Managing Health Risks.
  • Health Canada. (2001a). Expert Consultation on Prion-Related Diseases. [Final Report].
  • Health Canada. (2001b). Terms of Reference: Assistant Deputy Ministers' Interagency Advisory Committee on TSEs and Food. [Draft].
  • Health Canada. (2001c). ADM Committee on TSE - BSE. May 4, 2011. [Meeting Highlights, Draft].
  • Health Canada. (2002a). Transmissible Spongiform Encephalopathy Action Plan.
  • Health Canada. (2002b). Chronic Wasting Disease of cervids: A human health concern?
  • Health Canada. (2002c). Chronic Wasting Disease of deer and elk: A Canadian perspective
  • Health Canada. (2002d). TSE Risk Assessment Template. Version 6.3. Revised March 13, 2002.
  • Health Canada. (2003a). Food Directorate Policy on Specified Risk Material (SRM) in the Food Supply.
  • Health Canada. (2003b). CWD table of assumptions and scientific facts.
  • Health Canada. (2003c). Variant Creutzfeldt-Jakob Disease Risk to Canadians Eating Imported Foods Containing Small Amounts of Processed Ruminant Meat Product.
  • Health Canada. (2003d). Risk Assessment: Impact of SRM Policies on Potential Levels of BSE Infectivity in Food
  • Health Canada. (2003e, November 4). Memorandum to France Dansereau, Manager, Drug Inspection Unit. Subject: TSE Inspection Program.
  • Health Canada. (2003f). Transmissible Spongiform Encephalopathy: Risk to Canadians Using Cosmetic and Personal Care Products.
  • Health Canada. (2004a, March 15). Natural Health Products Directorate. Animal Tissue Form.
  • Health Canada. (2004b). Guidance Document on the Regulation of Medical Devices Manufactured from or Incorporating Viable or Non-Viable Animal Tissue or their Derivative(s).
  • Health Canada. (2004c, March 15). Canadians' views regarding public health.
  • Health Canada. (2004d). Emerging infectious disease issues (Public Opinion Research Issue Paper).
  • Health Canada. (2004e). Food safety issues (Public Opinion Research Issue Paper).
  • Health Canada. (2004f). Food for Thought (Public Opinion Syndicated Research Summary).
  • Health Canada. (2005a, January 21). Issue Analysis: Bovine Spongiform Encephalopathy and Health Products. [Draft].
  • Health Canada. (2005b, updated). Transmissible Spongiform Encephalopathies: Developing a common understanding. [Scientific version].
  • Health Canada. (2005c, updated). Transmissible Spongiform Encephalopathies: Developing a common understanding. [Layperson version].
  • Health Canada. (2005d). Additional Donor Exclusion Measures to Address the Potential Risk of Transmission of variant Creutzfeldt-Jakob Disease (vCJD) through the Blood Supply.
  • Health Canada. (2005e, January 22). Minimizing the Risk of Transmission of Non-Human Animal Transmissible Spongiform Encephalopathy (TSE) Agents via Health Products. [Revision 6].
  • Health Canada. (2005f, May 4). Quantifying the Potential BSE Infectivity in the Production of Canadian Beef and Beef Products and the Risk to Canadian Consumers of Acquiring vCJD. [Final Risk Assessment].
  • Health Canada. (2005g). TSE Research Meeting materials, including agenda, list of attendees, and presentations.
  • Health Canada. (2005h). Eye on health (Public Opinion Research Quarterly Report).
  • Health Canada. (2005i). Eye on health (Public Opinion Research Quarterly Report).
  • Health Canada. (2006a, September). TSE Secretariat VDD Organizational Chart.
  • Health Canada. (2006b). Evidence for Safety and Efficacy of Finished Natural Health Products: Guidance Document.
  • Health Canada. (2006c). Important Notice - Revised Policy Regarding Gelatin in Natural Health Products.
  • Health Canada. (2006d). A2.7 - BSE Compliance. 2005/06 Year End Project Report.
  • Health Canada. (2006e). TSE Research Meeting materials, including agenda, list of attendees, and presentations.
  • Health Canada. (2007a, April). Bovine Spongiform Encephalopathy I and II Formative Evaluation: Final Report.
  • Health Canada. (2007b, January). TSE Secretariat. Presentation for DMC.
  • Health Canada. (2007c). Natural Health Products Directorate (NHPD). Policy on Reduction of Transmissible Spongiform Encephalopathy (TSE) Risk in Natural Health Products. [Draft].
  • Health Canada. (2007d). Guidance for Industry: Preparation of Veterinary New Drug Submissions.
  • Health Canada. (2007e). Operational Planning Project. A2.7- BSE Compliance.
  • Health Canada. (2007f). A2.7 - BSE Compliance. 2006/07 Year End Project Report.
  • Health Canada. (2007g, March). Progress Update: Health Canada's Use of the European Directorate for the Quality of Medicines and Healthcare's Certificate of Suitability and the Signing of a Memorandum of Understandin.
  • Health Canada. (2008). 2008-2009 Report on Plans and Priorities.
  • Health Canada. (2009). Classical BSE Assumptions and Scientific Facts.
  • Health Canada. (2010). Summative Evaluation Framework for BSE I (Augmenting Health Canada's Response to the Bovine Spongiform Encephalopathy) and BSE II (Further Measures on BSE in the Areas of Risk Assessment and Targeted Research). Ottawa, ON: Author.
  • Health Canada. (2011a). Blood donation deferral policy expanded beyond the United Kingdom and Western Europe.
  • Health Canada. (2011b). Cosmetic Ingredient Hotlist - March 2011. Consumer Product Safety Division.
  • Health Canada & EDQM. (2007, March 20). Memorandum of Understanding between the Health Products and Food Branch of the Department of Health of Canada and the European Directorate for the Quality of Medicines and Healthcare of the Council of Europe regarding conformity of substances for pharmaceutical use.
  • Hilton, D. A. (2006). Pathogenesis and prevalence of variant Creutzfeldt-Jakob disease. The Journal of Pathology, 208(2), 134-141. doi:10.1002/path.1880
  • Ironside, J. W. (2010). Variant Creutzfeldt-Jakob disease. Haemophilia, 16, 175-180.
  • James, P. C. (2008). Both sides of the fence: A strategic review of chronic wasting disease management costs and benefits. Canadian Wildlife Federation.
  • Kihm, U., Hueston, W., & Heim, D. (2003). Report on actions taken by Canada in response to the confirmation of an indigenous case of BSE.
  • Leiss, W., Tyshenko, M. G., Krewski, D., Cashman, N., Lemyre, L., Al-Zoughool, M., & Amaratunga, C. (2010). Managing the risks of bovine spongiform encephalopathy: A Canadian perspective. International Journal of Risk Assessment and Management, 14(5), 381-436.
  • Ludlam, C. A., & Turner, M. L. (2006). Managing the risk of transmission of variant Creutzfeldt Jakob disease by blood products. British Journal of Haematology, 132(1), 13-24.
  • McLachlan, Stéphane. (2011, February). Crisis and Opportunity: Chronic Wasting Disease, Indigenous Peoples, and Cross-Cultural Research and Communication Regarding Wildlife and Environmental Health. In Management of CWD in Canada: Past Practices, Current Conditions, Current Science, Future Risks and Options. Workshop Summary. February 9-10, 2011. Edmonton, Alberta.
  • NHS Blood and Transplant. (2008, Spring). Managing the risk of transmission of vCJD by blood and tissues. Blood Matters, (24), 11-12.
  • Pauli, G. (2005). Tissue safety in view of CJD and variant CJD. Cell and Tissue Banking, 6(3), 191-200. doi:10.1007/s10561-005-0336-z
  • Phillips, Lord, of Worth Matravers, Bridgemen, J., & Ferguson-Smith, M. (2000). Findings and conclusions. The BSE Inquiry: The Report, Volume 1, i-xxxi. A report prepared for the Minister of Agriculture, Fisheries and Food, the Secretary of State for Health and the Secretaries of State for Scotland, Wales and Northern Ireland. Retrieved from http://collections.europarchive.org/tna/20060829152214/http://bseinquiry.gov.uk/report/volume1/chapte35.htm
  • PrioNet Canada. (2011). About us. Retrieved July 28, 2011, from http://www.prionetcanada.ca/detail.aspx?menu=2&app=113&cat1=378&tp=2&lk=no&title=About+PrioNet
  • Public Health Agency of Canada (PHAC). (2003, May). Blood safety and variant Creutzfeldt-Jakob disease (variant CJD). Retrieved from http://www.phac-aspc.gc.ca/cjd-mcj/vcjd-blood-eng.php
  • Public Health Agency of Canada (PHAC). (2006, December). Creutzfeld-Jakob Disease. CJD-Surveillance System. Newsletter to physicians. Retrieved December 15, 2011 from http://www.phac-aspc.gc.ca/hcai-iamss/cjd-mcj/cjdss-ssmcj/1206-eng.php
  • Public Health Agency of Canada (PHAC). (n.d.). Evaluation of the Prion Diseases Program.
  • Public Health Agency of Canada (PHAC). (2011). Creutzfeld-Jakob Disease. Prion Disease Information.
  • Public Health Agency of Canada (PHAC). (2012). Creutzfeld-Jakob Disease. CJD-Surveillance System. Statistics.
  • Rudge, T. J. (2005). Increasing regulations for natural health products: An investigation of trade effects. University of Saskatchewan. Retrieved from http://library2.usask.ca/theses/available/etd-08022005-071844/unrestricted/Thesis.pdf
  • Sher, G. D. (1998, October). Leukoreduction of the blood supply.
  • Stevenson, M. D., Oakley, J. E., Chick, S. E., & Chalkidou, K. (2008). The cost-effectiveness of surgical instrument management policies to reduce the risk of vCJD transmission to humans. Journal of the Operational Research Society, 60(4), 506-518. doi:10.1057/palgrave.jors.2602580
  • Sutton, J. M., Dickinson, J., Walker, J. T., Raven, N. D. H., & Weinstein, R. A. (2006). Methods to minimize the risks of Creutzfeldt-Jakob disease transmission by surgical procedures: Where to set the standard? Clinical Infectious Diseases, 43(6), 757-764. doi:10.1086/507030
  • Treasury Board of Canada Secretariat. (2009). Policy on Evaluation. Retrieved from http://www.tbs-sct.gc.ca/pol/doc-eng.aspx?section=text&id=15024
  • Turner, M. L., & Ludlam, C. A. (2009). An update on the assessment and management of the risk of transmission of variant Creutzfeldt-Jakob disease by blood and plasma products. British Journal of Haematology, 144(1), 14-23.
  • UK Blood Transfusion Services, & National Institute for Biological Standards and Control. (2010, January 15). Position statement: Creutzfeldt-Jakob disease.
  • U.S. Food and Drug Administration (FDA). (2004a, July 14). Use of Materials Derived From Cattle in Human Food and Cosmetics; and Recordkeeping Requirements for Human Food and Cosmetics Manufactured From, Processed With, or Otherwise Containing, Material From Cattle; Final Rule and Proposed Rule. Federal Register, 69(134), 42256.
  • U.S Food and Drug Administration (FDA). (2005). BSE Rules Being Strictly Enforced.
  • U.S. Food and Drug Administration (FDA). (2007a). Proposed Rule: Use of Materials Derived from Cattle in Medical Products Intended for Use in Humans and Drugs Intended for Use in Ruminants. Retrieved from http://edocket.access.gpo.gov/2007/E6-22329.htm
  • U.S. Food and Drug Administration (FDA). (2007b, June 25). Current Good Manufacturing Practice in Manufacturing, Packaging, Labeling, or Holding Operations for Dietary Supplements; Final Rule. Federal Register, 72(121), 34751.
  • U.S. Food and Drug Administration (FDA). (2010, May). Guidance for industry: Revised preventive measures to reduce the possible risk of transmission of Creutzfeldt-Jakob Disease (CJD) and variant Creutzfeldt-Jakob Disease (vCJD) by blood and blood products.
  • United States International Trade Commission. (2008, September). Global Beef Trade: Effects of Animal Health, Sanitary, Food Safety, and Other Measures on U.S. Beef Exports. USITC.
  • Wang, Z., & Coulthart, M. B. (2011, February 9). Are We Prepared for Human Chronic Wasting Disease? Workshop summary presented at the Management of CWD in Canada: Past practices, current conditions, current science, future risks and options, Edmonton, Alberta.
  • Ward, H. J. T., & Knight, R. S. G. (2009). Transmissible Spongiform Encephalopathy and Meat Safety. Safety of Meat and Processed Meat, 125-146.
  • Wiggins, R. C. (2009). Prions and the Transmissible Spongiform Encephalopathies. In D. W. McCandless (Ed.), Metabolic Encephalopathy (pp. 531-550). Springer.
  • Wilson, K., Hébert, P. C., Laupacis, A., Dornan, C., Ricketts, M., Ahmad, N., & Graham, I. (2001). A policy analysis of major decisions relating to Creutzfeldt-Jakob disease and the blood supply. Canadian Medical Association Journal, 165(1), 59-65.
  • World Health Organization (WHO). (2001). Legal Status of Traditional Medicine and Complementary/Alternative Medicine: A Worldwide Review.
  • World Health Organization (WHO). (2002). Variant Creutzfeldt-Jakob Disease.
  • World Health Organization (WHO). (2006). WHO guidelines on tissue infectivity distribution in transmissible spongiform encephalopathies.
  • World Health Organization (WHO). (2009). Safety issues in the preparation of homeopathic medicines. Geneva, Switzerland: WHO Press. Retrieved from http://www.who.int/medicines/areas/traditional/Homeopathy.pdf
  • World Health Organization. (WHO). (2010). World Health Organization Tables on Tissue Infectivity Distribution in Transmissible Spongiform Encephalopathies.
  • World Health Organization (WHO). (2011). International Regulatory Cooperation for Herbal Medicines (IRCH).
  • World Organisation for Animal Health (OIE). (2011). Bovine Spongiform Encephalopathy Status of Members.
Report a problem or mistake on this page
Please select all that apply:

Thank you for your help!

You will not receive a reply. For enquiries, contact us.

Date modified: