ARCHIVED - Evaluation of the C-EnterNet Program Final Report

 

Introduction

1.1 Introduction and Purpose of Evaluation

This report presents the results of the evaluation of the C-EnterNet Program.  The evaluation was undertaken by Government Consulting Services (GCS) for the Public Health Agency of Canada (PHAC) between May and October 2009.  The primary purpose of the evaluation was to inform resource allocation decisions of PHAC senior management.  A secondary purpose was to inform future design and delivery decisions of the C-EnterNet Program team.

The evaluation followed the scope and methodology set out in an evaluation plan developed during a planning phase completed prior to the commencement of the evaluation.  The evaluation planning phase was undertaken by GCS between January and April 2009.  The evaluation plan was aligned with the Treasury Board Policy on Evaluation1.

The report is organized as follows:

  • Section 1 presents a profile of the C-EnterNet Program;
  • Section 2 presents the methodology for the evaluation and discusses methodological considerations;
  • Section 3 presents the findings by evaluation issue; and
  • Section 4 presents the overall conclusions and related recommendations.

1.2 Profile of the C-EnterNet Program

The National Integrated Enteric Pathogen Surveillance (C-EnterNet) Program is designed to support activities to reduce the burden of enteric disease in Canada.  Enteric disease refers to gastrointestinal illnesses that result from pathogenic micro-organisms, often coming from food or animal sources.  The program involves systematic and integrated data collection, analysis, interpretation and communication of results. 

C-EnterNet is based on a model of active sentinel site surveillance, as used by the Centers for Disease Control and Prevention (CDC) in the United States for its FoodNet program.  This model involves a limited number of communities acting as reporting (sentinel) sites, from which the information may be extrapolated to the general population. This model is intended to allow for more comprehensive and rigorous surveillance than the passive enteric disease surveillance approaches traditionally employed in Canada.  Active surveillance includes:

  • sustained coordination among all levels of the health care system;
  • specialised laboratory involvement/partnership;
  • standardized microbiological and epidemiological methods; and
  • regular reporting and communication.

The core objectives of the program are to2:

  • detect changes in trends in human enteric disease and in levels of pathogen exposure from food, animal and water sources in a defined population;
  • generate human illness attribution values (i.e., the proportion of human cases due to exposure via water, food and animals); and
  • improve the analysis, interpretation and reporting of laboratory and epidemiological data for public health, water and agri-food purposes.

The business case for the C-EnterNet Program was made in December 2004 and a pilot sentinel site was initiated in June 2005.  While the program design called for five sentinel sites across Canada to meet its pan-Canadian objectives, C-EnterNet has been implemented in one pilot sentinel site to determine the feasibility and usefulness of the program’s design.

The selection of the pilot sentinel site in the Region of Waterloo was based on specific criteria, including population size and local public health capacity.  In the pilot site, C-EnterNet works in collaboration with the Region of Waterloo Public Health (ROWPH) to undertake enhanced human disease surveillance, including standardized case follow-up.  C-EnterNet supports a site coordinator working within the health department to coordinate and lead activities within ROWPH.

In addition to the work undertaken with ROWPH, the C-EnterNet team also manages and coordinates active surveillance (i.e. sample collection) of pathogens in surface water, retail food and on farms, and undertakes analysis of these samples in partnership with laboratories.  A brief overview of the four components of integrated surveillance is provided in Table 1.

Table 1. Overview of C-EnterNet Surveillance Components
Element Description
Human
  • Cases of human enteric illness in the sentinel site are identified based on the province’s passive notifiable disease system, with some modifications as a result of the C-EnterNet Program.
  • Once a sample is identified as positive for enteric illness by a community medical or hospital laboratory, a standardized questionnaire is administered to each case by Region of Waterloo public health inspectors for control and mitigation of the disease and to identify risk factors and exposures.
  • Isolates from suspect cases of enteric illness are further tested by the Ontario Agency for Health Protection and Promotion’s Toronto Public Health Laboratory (OAHPP TPHL) and PHAC’s National Microbiology Laboratory.
  • The laboratory data for each positive case is sent back from OAHPP TPHL to the ROWPH to link with the corresponding case follow-up data.
  • Coordination of data flow is undertaken by the C-EnterNet Site Coordinator located at the ROWPH.
Water
  • Surface water samples are collected bi-weekly from 5 locations within the Grand River watershed, upstream to the drinking water intake site for the sentinel area.
  • Samples are tested for pathogens, and water quality indicators are recorded.
  • Sampling and laboratory work was undertaken by the University of Waterloo and now sampling is undertaken by the Ontario Ministry of the Environment, with the laboratory work by University of Guelph Laboratory Service Division, Hyperion and the Alberta Provincial Laboratory.
Retail (Food)
  • Samples of raw pork chops, chicken, beef and (since April 2009) imported produce are purchased weekly at a random selection of food retailers within the sentinel site.
  • Sample collection is undertaken by part-time employees of the C-EnterNet Program.
  • Laboratory testing for pathogens is undertaken by the University of Guelph Laboratory Service Division, the Laboratory for Foodborne Zoonoses (PHAC) in Guelph and the Bureau of Microbial Hazards, Health Canada, Ottawa.
On-Farm
  • Manure samples from a random selection of swine, beef, dairy, and broiler chicken farms within the sentinel site are collected and tested for the existence of pathogens.
  • Sample collection is undertaken through partnerships with University of Guelph Department of Population Medicine staff.
  • Laboratory work is undertaken by the Laboratory for Foodborne Zoonoses.

Along with the surveillance component, the program includes a source attribution component.  Source attribution determines what proportion of a particular disease is acquired from a specific source (e.g., chicken) and pathway (e.g., food, water, person-to-person transmission)3.

Lastly, the program also undertakes targeted research studies of relevance to the program.  These have included one-time studies related to food consumption, water consumption, and other topics of relevance.

The program is currently staffed at 7.0 Full Time Equivalents (FTEs).  The C-EnterNet Program team is located within PHAC offices in Guelph, Ontario, with the exception of one member based in St-Hyacinthe, Quebec.  Team members are assigned to one or more elements of the program (i.e., human, water, farm or retail components), depending on their roles and areas of expertise.  Team members’ backgrounds include veterinary medicine, epidemiology, public health, environmental microbiology and parasitology.  The core team is supported by a network of partners, including a Site Coordinator based in the ROWPH, ROWPH public health inspectors, sample collectors, laboratory technicians and other support staff.

The C-EnterNet program is jointly managed by the Centre for Food-borne, Environmental and Zoonotic Infectious Diseases (CFEZID) and the Laboratory for Foodborne Zoonoses (LFZ), both located within PHAC.   An Advisory Committee of governmental and non-governmental stakeholders has been developed to provide advice and guidance to the program.  The Advisory Committee is intended to meet in-person on an annual basis, with additional communication done on an as-needed basis.

Table 2 shows the amount and sources of financial resources committed to the program from 2003-04 to 2009-10.  From 2003-04 to 2007-08, C-EnterNet was primarily funded by Agriculture and Agri-food Canada (AAFC) through its Agricultural Policy Framework (APF).  Funding from the APF totalled $3.3 million.  Other contributing funders included the Ontario Ministry of Agriculture, Food and Rural Affairs (OMAFRA) and PHAC.  Since 2008-09, C-EnterNet has been primarily funded through the Food Safety Action Plan (FSAP), with PHAC also providing salary and operating funding through both LFZ and CFEZID.

Table 2. Financial Resources ($’000) Committed to C-EnterNet by Source and Year
  2003-04 2004-05 2005-06 2006-07 2007-08 2008-09 2009/10
AAFCAPF 390 690 660 880 700 -- --
FSAP           544 1,393
OMAFRA -- -- 30 46 46 -- --
PHACLFZ 100 240 240 280 350 350 280
PHACFWZID/CFEZID* -- -- -- 160 160 190 290
Total 490 930 930 1,366 1,366 1,084 1,963

Source: C-EnterNet Program
* Foodborne, Waterborne, Zoonotic Infections Division (within the Centre for Infectious Disease Prevention and Control), now the Centre for Food-borne, Environmental and Zoonotic Infectious Disease, IDEP

A timeline of key C-EnterNet Program events is presented in Table 3.

Table 3. C-EnterNet Program Timeline4
Date Event
2003
  • C-EnterNet Program initiated through funding from the APF
December 2004
  • Submission of Business Case for C-EnterNet
April 2005
  • First sentinel site initiated, in Region of Waterloo
  • On-farm swine sampling launched
June 2005
  • Official launch of the pilot site
  • Retail and water sampling launched
2005
  • C-EnterNet facilitates two outbreak investigations at the request of the local public health department
2005
  • C-EnterNet establishes and leads a national working group on source attribution
June 2006
  • On-farm dairy sampling launched
February 2007
  • On-farm beef sampling launched
June 2007
  • Release of first Annual Report (2006)
August 2007
  • First peer-reviewed article published
October 2007
  • On-farm broiler chicken sampling launched
April 2009
  • Sampling of produce launched
  • Commencement of present evaluation of C-EnterNet Program

1.3 Program Logic Model

A logic model for the C-EnterNet Program was developed during the evaluation planning phase.  The logic model provides a visual representation connecting the program’s activities and outputs with expected results (outcomes).  The logic model formed a framework for assessing the success of the program in the evaluation.

Program Logic Model
Text Equivalent

ARCHIVED - Logic Model

A logic model (such as the C-EnterNet logic model) is a schematic overview of program elements showing what the program supposed to do, the expected results of those activities and the expected affects of those results. The following is a description of the logic model.

C-EnterNet main activities are:
Identification of information needs and research development for enteric disease in Canada;
Data collection and compilation through surveillance and studies;
Data analysis, interpretation and integration;
Knowledge production and dissemination.

The main outputs of C-EnterNet are proposals, agreements, collaborations, presentations, workshops, reports and publications.

In the short term (1 to 5 years), the expected outcomes of CHVI are:
Up-to-date information on enteric diseases and exposures received and exchanged among stakeholders;
Collaborative networks for enteric disease established and developed;
Advancement of source attribution methodologies.

The intermediate (6-10 years) outcomes of the program are:
Increased knowledge of enteric diseases and exposures among stakeholders;
Public health capacity is strengthened;
Programs, policies, practices, and health messages related to enteric diseases and exposures are informed, created or evaluated.

Finally, the long term (11-15 years) outcome of C-EnterNet is to promote and protect the health of Canadians by contributing to reducing the burden of human illness in Canada.

 


1 This evaluation was undertaken using the direction from Treasury Board’s evaluation policy from 2001.  A new evaluation policy came into effect on April 2009, however, at the time of planning this evaluation, no direction on applying the new policy had yet been provided.

2 Public Health Agency of Canada. C-EnterNet 2006 Annual Report.

3 Public Health Agency of Canada.  C-EnterNet Overview.

4 Partly based on information from Evaluation of C-EnterNet by Kate Zinszer, June 30, 2007.

Page details

Date modified: