ARCHIVED - Evaluation of the C-EnterNet Program Final Report

 

Evaluation Findings

This section presents the findings of the evaluation, organized by the four major evaluation areas: relevance; design and delivery; program success; and cost-effectiveness and alternatives.

3.1 Relevance

3.1.1 Continued Need

Finding: Evidence suggests that there is a continued need for the C-EnterNet Program due to the prevalence of enteric illness and the associated costs to Canadian society. Further, food-borne and waterborne illness is a strong public health concern.  The C-EnterNet Program is aligned with national priorities related to food and water safety and with international surveillance initiatives.  All components of the C-EnterNet Program are seen as relevant, and there is strong support for program expansion.

Importance of Infectious Enteric Disease in Canada

Infectious enteric disease is a broad category of illness including numerous bacterial, viral and parasitic enteric illnesses.  These commonly include salmonellosis, campylobacteriosis and giardiasis, as well as verotoxigenic E. coli, shigellosis, yersiniosis, Hepatitis A, cryptosporidiosis, and others.  Symptoms of illness are generally mild, but can range from vomiting and diarrhea to severe chronic conditions or death.

Infectious enteric disease is common in Canada.  It has been estimated that there are 1.3 episodes of acute gastrointestinal illness per person-year in Canada1.  These are often transmitted through food and water.  PHAC has estimated that there are as many as 13 million cases of food-borne illness a year in Canada2, and that there are more than 250 types of food-borne illnesses3.

As a result of its prevalence, the associated costs of enteric illness are significant.  A 2006 study estimated the cost of enteric disease at $115 per Canadian per year, and the cost per case at $1,089.  This represents an estimated annual cost of $3.7 billion4.  These costs included the value of missed employment, costs to the health system, and medical costs.  According to a recent British study, roughly 4% of enteric disease cases in industrialized countries will result in hospitalization5.  As the highest costs of enteric illness have been reported in those older than 65 years of age, the associated costs are predicted to rise with the aging of the population6.

According to interviewed stakeholders, recent events like the Bovine Spongiform Encephalopathy (BSE) crisis, the water pathogen-related deaths in Walkerton, Ontario, and outbreaks of food-borne disease (including listeriosis) have meant that food-borne and waterborne diseases have remained a major public health concern in Canada.  As enteric disease is preventable, there is the expectation among Canadians that governments are working to improve prevention and detection.

Trends in Incidences of Food-borne and Waterborne Illness in Canada

The documented health and economic costs associated with infectious enteric diseases have increased in many countries. Countries that have implemented surveillance systems have noted a significant increase in the incidence of food-borne diseases in the past two decades7.  According to the recent report of the Independent Listeriosis Investigative Review, food-borne illness is the fastest growing class of emerging infectious disease in Canada, and that there are more than 250 types of food-borne illnesses8.

Based on available data, reported cases of enteric illness in Canada have increased, remained relatively stable or seen a small decrease in Canada in recent years, depending on the type of illness9.  Figure 1 shows trends in a select few types of enteric illness.  These figures are, however, based on cases reported through provincial passive surveillance systems only and are therefore an under-reporting of the number of cases. 

Figure 1. Trends in Selected Enteric Illness, Canada, 1994-2004

Figure 1, Trends in Selected Enteric Illness, Canada, 1994-2004

It should be noted that after 1999, not all provinces reported all diseases for all years. In addition, all reporting for some provinces/territories for some years is missing, and data shown is unadjusted for this.

Text Equivalent

ARCHIVED - Figure 1. Trends in Selected Enteric Illness, Canada, 1994-2004 (Incidence rates per 100,000 population)

  Campylobacteriosis Cryptosporidiosis Salmonellosis Verotoxigenic E. coli Yersiniosis
1994 54.1   22.2 4.1  
1995 46.6   21.8 5.1  
1996 43.1   22.2 4.2  
1997 45.1   20.1 4.2  
1998 37.7   18.4 4.9  
1999 37.8 0.3 18.5 4.9 2.3
2000 40.2 2.6 18.4 9.8 2.1
2001 38.3 7.5 19.5 4.3 2.5
2002 36.7 2.4 19.1 3.9 1.9
2003 30.9 2.5 15.6 3.2 1.7
2004 28.9 1.8 15.3 3.2 1.5

It should be noted that after 1999, not all provinces reported all diseases for all years. In addition, all reporting for some provinces/territories for some years is missing, and data shown is unadjusted for this.

Source:  Primary data from the National Notifiable Diseases (NND) database, and National Enteric Surveillance Program database where NND data is unavailable. 

According to PHAC, there is evidence that the Canadian population may be becoming more susceptible to enteric illness, which is likely to result in an increase in the occurrence and severity of enteric disease in the future. This is in part a result of the aging of the Canadian population. In addition, increased urbanization results in a reduction in the population of direct contact with the agri-food sector, which is known to infer natural immunity. Trends suggest immunocompromised people will reflect a larger proportion of the population in the future10

Stakeholders interviewed for this evaluation indicated that there are many recent factors that have influenced the prevalence of enteric illness.  Increased globalization has meant increased travel, and food distribution and consumption patterns have become more complex.  At the same time, food production has become more industrialized.  A single food processing establishment distributing its products to millions of consumers can now result in food-borne illness that is international in scope.  Others noted that climate change will continue to affect the prevalence of enteric disease as environmental changes (such as changing pathogen levels in bodies of water) become more pronounced.

Evidence that C-EnterNet Fills Important Gaps in Addressing Enteric Disease

Well-informed public health activities and policies can prevent and contain outbreaks, and reduce the burden of infectious disease.  The passive surveillance reporting system relies on physicians, laboratories and others to submit case information through provincial and national reporting chains.  This has been shown to have resulted in under-reporting of enteric disease in Canada: a recent report found that one in 313 cases of enteric disease was reported to the Province of Ontario11.

C-EnterNet was designed to address a need for more complete data on infectious enteric illness in terms of the surveillance of enteric pathogen exposures through food animals, food and water, so that source attribution can be established.  Enhanced surveillance is intended to provide reliable data on trends over time for human cases and exposures.  Information on sources of enteric illness is not currently collected or available in Canada.

The information produced by the C-EnterNet Program is intended to inform the development and evaluation of policies and practices related to food and water safety.  This is in line with public health recommendations called for in several recent reports, including: the 2004 Haines report on meat safety (“Report of the Meat Regulatory and Inspection Review”); the 2002 O’Connor reports on water safety produced following deaths from waterborne illness in Walkerton, Ontario; the 2003 Naylor Report, “Renewal of Public Health in Canada”; and 1999, 2002, 2005 Auditor General reports, which emphasized the need for enhanced disease surveillance activity in Canada.

Furthermore, the recent June 2009 Report of the Standing Committee on Agriculture and Agri-Food to the House of Commons, “Beyond the Listeriosis Crisis: Strengthening the Food Safety System”, recommended that “the government enhance the national food-borne illnesses surveillance system by developing programs to gather epidemiological data on food-borne illnesses in Canada and that it initiate discussions with the provinces to add relevant food-borne illnesses to the list of nationally notifiable diseases” 12.

The C-EnterNet Program is aligned with related international efforts.  In 2000, the 53rd World Health Assembly adopted a resolution recognizing the role of food safety in public health, and the Word Health Organization (WHO) encouraged its member states to develop surveillance programs for food-borne diseases13.  Active food surveillance programs are operating in the United States (the CDC’s Foodborne Diseases Active Surveillance Network (FoodNet)), and in Australia (OzFoodNet).  In 2006, the WHO's Department of Food Safety and Zoonoses launched, at an international consultation, an initiative to estimate the global burden of food-borne diseases from all major causes, including chemicals and zoonoses14.

Stakeholders interviewed for the evaluation were, with few exceptions, of the opinion that C-EnterNet helps to address the issue of infectious enteric disease.  Reasons given included:

  • C-EnterNet, through examining the source of enteric illness, allows for the determination of the relative importance of different sources of illness, and therefore provides direction in where resources should be targeted to reduce risks.  C-EnterNet can therefore allow for an impact on practices at the source of the illnesses.
  • C-EnterNet can act as a tool to evaluate the effectiveness of programs/interventions through examining trends over time, and with more sentinel sites in operation, across regions.
  • C-EnterNet is a unique program and the data from the sentinel sites is high-quality, rigorous, and comprehensive.
  • C-EnterNet has helped to increase capacity in the area of infectious enteric illness.  This has been evident not only at the public health department, which has benefited from increased training and research support, but also through the networking and collaboration of the program generally.

Surveyed recipients of C-EnterNet information products were similarly positive on the extent to which C-EnterNet was addressing important gaps in information.  The vast majority (82.0%) of recipients of C-EnterNet information agreed that the program is helping to fill information gaps on enteric disease and exposures (with 44.0% strongly agreeing and 38.0% agreeing).  Only 4.0% disagreed that the program was helping to fill information gaps (the remaining 14.0% of respondents indicated that they did not know).

Views on Relevance of Program Components

All of the various program components are perceived as useful by surveyed information recipients.  As demonstrated in Table 6, information for each of the program components was found useful/very useful by over three-quarters of surveyed information recipients.

Table 6. Perceived Usefulness of Different Program Components
Program Component Useful / Very Useful Not At All / Not Very Useful Don’t Know
Enhanced human disease follow-up information and pathogen sub-typing. 77.9% 7.4% 14.7%
Active surveillance of enteric pathogens in water, aligned with surveillance in humans. 76.8% 11.6% 11.6%
Active surveillance of enteric pathogens in farm/ agriculture, aligned with surveillance in humans. 85.4% 5.2% 9.4%
Active surveillance of enteric pathogens in food, aligned with surveillance in humans. 86.0% 5.0% 9.0%
Integrated surveillance of enteric pathogens (in humans, agri-food, and water) 83.8% 7.1% 9.1%
Work and studies on human illness attribution (source attribution) 80.6% 6.1% 13.3%

Source: C-EnterNet Information Recipient Survey
N=95-100

Active surveillance of enteric pathogens in food, aligned with surveillance in humans, had the biggest share of respondents rating it as useful, at 86.0%, but other components were similarly rated.  Water surveillance was least often rated as useful, although over three-quarters (76.8%) still indicated this component was useful to them or their organization.  This is consistent with the opinion of stakeholders, who were overwhelmingly positive on the extent to which both the integrated surveillance and source attribution components were relevant to the C-EnterNet Program.  No interviewee suggested that either of these parts of the program were not relevant.

With respect to integrated surveillance, it was noted by interviewees that this component has helped to support the improvement of data, improve the ability to identify frequency clusters, and determine the costs resulting from illness, among other benefits.  It was also noted that improved data and collaboration increase overall capacity in the field of enteric disease.  It was emphasized that the source attribution component allows for a better understanding of where resources should be placed in order to effectively tackle enteric illness.

While the evidence suggests that all components are relevant, some concern was expressed by a few interviewed stakeholders that the program lacked a clear conceptual framework.  A conceptual framework would illustrate how the different elements of the program fit together to provide a picture of enteric illness and its sources.  Specifically, it was noted that there has been no explanatory documentation from the program on the assumptions in causality between levels of pathogens and number of cases of human illness.  For example, it was suggested by one interviewee that it was not clear to what extent the model for the program has taken into account the impact of water treatment, or the cooking of meat, on the level of pathogens to which humans are exposed.

Need for Similar Information from Other Regions

The original design of the C-EnterNet Program envisioned five to six sentinel sites operating in communities across Canada. To date, the program includes a single pilot site in Waterloo, Ontario.  C-EnterNet information recipients and interviewed stakeholders both strongly supported expanding the program to other regions of Canada. 

Survey respondents overwhelmingly agreed (84.0%) that expanding the C-EnterNet Program to other sentinel sites across Canada would provide useful information for addressing enteric diseases and exposures.  Only 3.0% disagreed, and the remaining 13.0% indicated they did not know.

Similarly, all interviewed stakeholders with an opinion indicated that they felt the program should be expanded to other sites.  Most noted that the program was designed to operate in multiple sites to allow for national-level estimates.  A few interviewees also noted that having C-EnterNet operating in just one site does not allow for the identification of jurisdictions that appear to employ more effective practices (e.g., with respect to agricultural practices, water, etc.) as evidenced by levels of illness and pathogens.  Interviewees also noted that increasing the number of sites would allow for the increased overall statistical power and rigour afforded by larger sample sizes. 

3.1.2 Alignment with Government of Canada, PHAC Objectives and Priorities and Local Public Health Priorities

Finding: C-EnterNet is aligned with Government of Canada priorities and PHAC objectives, and with local health priorities.  The stated results of the program under its current funding also align with the objectives of the Canadian Food Inspection Agency.

Alignment with Government of Canada Objectives, Priorities

The C-EnterNet Program supports PHAC in achieving the Government of Canada (GoC) outcome of Healthy Canadians.  The 2008 PHAC Reports on Plans and Priorities specifically mentioned alignment to Government of Canada outcomes in regard to Infectious Disease Prevention and Control. In the preface to the same report, former Minister of Health Tony Clement noted that PHAC “supports the Government’s priorities through its expertise and networks on the surveillance of health outcomes. By highlighting links between exposure and illness, this work allows us to direct interventions where needed, and to in turn measure their effectiveness"15.  Furthermore, both the 2007 and 2008 Speeches from the Throne include food safety as a GoC priority.

Alignment with PHAC Objectives, Priorities

The C-EnterNet Program theory is directly aligned and falls under PHAC’s objectives and priorities as defined in the agency’s Program Activity Architecture. Under the overriding strategic objective “Healthier Canadians, reduced health disparities, and a stronger public health capacity” is the program activity “Infectious Disease Prevention and Control,” under which falls the strategic activity "Foodborne, Waterborne and Zoonotic Diseases." In addition, C-EnterNet feeds into the program activity “Strengthen Public Health Capacity" including the strategic activity of "Building Public Health Human Resource Capacity."

However, the C-EnterNet Program has never been primarily funded from “core” PHAC funding.  Under its current Food Safety Action Plan funding, the stated expected results of the C-EnterNet Program are framed within a food safety role.  Expected results in this submission are “information for decision-makers on risks posed by imported commodities” and “information on trends of pathogens on foods and related illnesses”.  While also addressing public health, these appear closely aligned with the mandate of the Canadian Food Inspection Agency (CFIA) in ensuring the safety of Canada’s food supply.

Alignment with Local Health Department Priorities

The ROWPH undertakes health services, programs, assistance and current information to:

  • promote healthier living;
  • protect against health threats; and
  • prevent disease and injury 16.

The C-EnterNet Program supports all three areas with a focus on preventing disease and injury.  All interviewed staff and management at ROWPH were positive on the degree to which C-EnterNet aligned with the priorities of their health department.  This alignment was evidenced in the following C-EnterNet Program activities:

  • co-development and implementation of a comprehensive follow-up questionnaire for cases of enteric illness in the region;
  • facilitating systematic exchange of isolates and information between laboratories and the public health department;
  • organizing/supporting staff through training and professional development workshops;
  • collaborating on the preparation and publication of scientific manuscripts (e.g. Heywood et. al. A community outbreak of travel-acquired Hepatitis A transmitted by an infected food handler.Canada Communicable Disease Report 1 November 2007,Volume 33, Number  12);
  • facilitating outbreak investigations at the request of the local public health department; and
  • the C-EnterNet team working with ROWPH and PHAC’s Outbreak Response and Issues Management section to improve outbreak management in childcare facilities.  This was undertaken through collaborative analysis and publication of an E. coli outbreak in a childcare facility17 and qualitative research related to childcare facility protocols18.

It was also noted in interviews with the ROWPH staff and management and the focus group with ROWPH staff that participation in the C-EnterNet Program had raised the profile of the public health department, and helped it to develop a “culture of excellence”.  The stated goal of developing a culture of excellence was to emphasize innovation, collaboration, knowledge exchange and service excellence within the health department.  Thus, the C-EnterNet Program is helping to advance the goal of the ROWPH in this respect.

3.1.3 Appropriateness of Current Role of Government

Finding: As the C-EnterNet Program is national in scope, and is related to public health and food safety, the current role of the federal government is appropriate.  An expanded role for multiple federal and provincial organizations impacted by the program and its results would help to ensure buy-in and sustainability, and would reflect the collaborative and multi-disciplinary nature of the project.

Evidence of Appropriateness of Role

Multi-disciplinary in nature, the C-EnterNet Program brings together different levels and areas of government and other organizations.  Sentinel sites require partnerships with local public health, laboratories, water and agri-food sectors, as well as provincial and federal organizations responsible for public health, food safety and water safety.  The original design of the program calls for the program to produce national estimates on enteric illness and sources.  In this context, the management of the program by the federal government is appropriate: the federal government is best placed to act as a coordinating body for numerous partners working within, and across, a network of sentinel sites.

The federal government plays a role in the area of food safety, which is an important component of the C-EnterNet Program. Responsibility for food safety is shared across the different levels of government, however, with federal responsibility being largely regulatory in nature.  As outlined in the Report of the Standing Committee on Agriculture and Agri-food, Beyond the Listeriosis Crisis: Strengthening the Food Safety System:"At the federal level, the Food and Drugs Act provides the foundation of Canada’s food safety system. It derives its authority from the federal power to legislate in the area of criminal law and requires that all food sold in the country be fit for human consumption. Some products […] are also covered by separate Acts of Parliament, enacted under the federal jurisdiction over trade and commerce."19  

Under their public health and trade mandates, provincial governments also have a role in food safety, including regulating food retailers and services.  In some jurisdictions, enforcement of provincial regulations is undertaken by the municipal government.  A Federal/Provincial/Territorial Food Safety Committee (FPTFSC), consisting of government officials from health and agriculture ministries across Canada, coordinates the development of national food safety policy options, and implements food safety initiatives, and acts as a forum for discussing scientific, policy and program issues20.

Opinion on Potential Transfer of Components

It was noted by some interviewed stakeholders that other organizations could have a larger role in the C-EnterNet Program.  Given its multi-disciplinary focus and the different players operating in the areas of health, food safety, and the environment, multiple federal organizations acting as funders was suggested as one potential option.  It was also suggested that there may be room for an increased role for provincial organizations, including as significant funding partners.  It was suggested that this would help to ensure the program's sustainability and would help to increase buy-in.  A potential downside noted was that, as a result of data sharing protocols between different levels of government, there may be more administrative time and effort required if the provinces become more involved in data collection for the C-EnterNet Program.

3.1.4 Complementarity, Duplication and Overlap

Finding: C-EnterNet as a whole does not duplicate or overlap the work of other programs or organizations.  Evidence suggests that program complementarity could be improved through closer collaboration with Canadian Integrated Program for Antimicrobial Resistance (CIPARS).

Existence of Similar Programs and Organizations

C-EnterNet is the only program in Canada that collects, analyses and reports on both cases of enteric disease and related pathogens in exposures.  

The primary source of data on cases of enteric illness across Canada is the National Notifiable Diseases (NND) system.  Over 50 diseases are considered notifiable and are under national surveillance, including campylobacteriosis, giardiasis, shigellosis, salmonellosis, verotoxigenic E. coli, and others.  There is a legal obligation for acute cases of enteric illness to be reported by each of the provinces through this system.  The number of cases are compiled and distributed through monthly reports as well as online through the Notifiable Disease On-line database.

In addition to the NND, the National Enteric Surveillance Program (NESP), administered by PHAC, compiles data from provincial public health laboratories and produces weekly reports of results. NESP provides a picture of the status of major enteric infectious diseases in the human population such as salmonellosis, campylobacteriosis, shigellosis, verotoxigenic E. coli, yersiniosis and, more recently, intestinal parasitic organisms such as Giardia, Cryptosporidium, Entamoeba and Cyclospora, as well as enteric viruses such as Norovirus and Rotavirus. This system is known to report only a small proportion of all cases, as it relies upon the patient seeking medical attention, the physician requesting laboratory testing for enteric pathogen, the laboratory testing for the pathogen and the test correctly identifying the pathogen21

C-EnterNet is one of two surveillance programs in Canada where sampling of sources is coupled with human illness sampling.  The other (i.e., CIPARS), tests the antimicrobial susceptibility in enteric pathogens and other organisms at various points along the food chain (at the on-farm, abattoir (slaughterhouse) and retail levels).  CIPARS is more limited in scope than C-EnterNet, in that it examines a fewer number of pathogen.  Also, it does not use a sentinel design.  However, like C-EnterNet, CIPARS does include sampling of food retailers and farms.  CIPARS also does sampling at abattoirs, which was suggested as a potential source of relevant data for C-EnterNet.

In addition to CIPARS, there are a number of other programs and organizations that incorporate surveillance and/or sampling-related activities.  Interviewed stakeholders noted that the CFIA has begun to do testing of produce, bagged lettuce, and bulk lettuce for restaurants.  It was said that testing is done for just three bacterial pathogens, and not those of most interest to C-EnterNet’s scientists.  It was also noted that CFIA also does some testing at abattoirs.  Stakeholders also noted that some testing takes place by provincial organizations, including some testing at abattoirs by the OMAFRA

Some provincial and local health regions also appear to have undertaken their own surveillance activities, but on a comparatively small scale.  For example, one stakeholder noted that one health region in Alberta has undertaken their own human case follow-up, similar to that done in the C-EnterNet Program.  In addition, an integrated surveillance program was being undertaken for Salmonella in BC. 

Similar enteric illness surveillance programs are in operation outside of Canada.  C-EnterNet was modeled on the CDC’s FoodNet program, implemented to reduce the occurrence and impact of food-borne disease in the United States.  OzFoodNet, in Australia, is another food-borne disease surveillance program.  The scope of C-EnterNet is, however, wider than its American and Australian counterparts in that it also includes water and on-farm components. 

Information recipients were asked whether they could obtain the information they receive from C-EnterNet from another source.  A very small proportion of survey respondents (3.0%) indicated that if C-EnterNet did not exist they could obtain the exact same information elsewhere.  More than half of respondents (52.5%) said they could obtain some of the same information from another source, while 15.8% said they could not obtain any of the same information from another source.  More than one-quarter (28.7%) indicated that they did not know. 

Among the respondents who indicated they could obtain some of the same information from other sources, 41.5% indicated that the available information from other sources was not as valuable or of the same quality as C-EnterNet data.  For example, data was not as precise, not collected as frequently, not as detailed or not as relevant. One respondent noted, for example: "Generic information is available elsewhere.  The C-EnterNet advantage is the provision of Canadian data."   Further, 17.0% said that they could only obtain some similar information by way of multiple data sources, which would require a considerable amount of effort. For example, one respondent noted that "partial information gathered by C-EnterNet could be gathered from other sources, however not easily and not in an integrated way."

Finally, several people commented that they could obtain some (though not all) of the data (related to agri-food exposures / retail meat pathogens) from CIPARS.  One respondent noted that: "My primary interest is meat. CIPARS provides similar but less information in terms of enteric pathogens in meat." 

Level of Coordination with Similar Programs, Organizations

The C-EnterNet Program has collaborated extensively with other organizations or individuals.  This is evidenced through the large number of partners and initiatives at the local, provincial, national, and international level. Partners and programs include public and private laboratories, various levels of government, academic institutions, and national and international working groups.  Some of this coordination involves significant levels of responsibility.  For the water component, for example, the Ontario Ministry of Environment conducts water sampling for the C-EnterNet Program.

Beyond participation in the Advisory Committee, formal collaboration and coordination with other programs and organizations within the federal government appears to be limited.  The C-EnterNet Program and CIPARS, both located within the Infectious Disease and Emergency Preparedness (IDEP) Branch at PHAC, have, in the past, provided only informal assistance to each other, such as advice provided to assist in setting up C-EnterNet and sharing of contacts between team members.  From a methodological and operational perspective, there are significant differences between the two programs in terms of sampling, testing and analysis.  However, several interviewed stakeholders indicated that there could be more formal collaboration between C-EnterNet and CIPARS.  It was noted that more collaboration could result in greater cost-effectiveness, and could enhance data sharing and the comprehensiveness of data available to each program. 

Finally, the C-EnterNet Program team has undertaken significant consultation with its international peers.  For example, during design and implementation of C-EnterNet, the program team consulted with CDC FoodNet headquarters in Atlanta, Georgia as well as with FoodNet sites in Minnesota and Connecticut in order to learn of their experiences in implementing their surveillance programs.  A member of the FoodNet program is also a member of the C-EnterNet Advisory Committee.  International collaboration has also been undertaken related to source attribution methodology development.

3.2 Design and Delivery

3.2.1 Factors Facilitating and Impeding Strength

Finding: The skills, knowledge and composition of the C-EnterNet team were identified as key strengths of the program, while the existence of only one sentinel site was viewed as the biggest impediment to program success.

Interviewed stakeholders were asked which factors in the design or delivery of the program had facilitated the success of the program, and which had impeded success.

The composition of the C-EnterNet team was most commonly noted as a factor in the program’s success.  Specifically, interviewees noted:

  • the team has the right skills to foster collaboration, build bridges between organizations, and bring people together;
  • the inclusion of a point-person within the sentinel site’s public health department (as Site Coordinator) who had been previously employed within the same department, has allowed for effective coordination; and
  • the multidisciplinary nature of the team and the fact that the team members each bring their respective and unique networks were very valuable.

Other, less commonly suggested success factors (each mentioned by multiple interviewees) included:

  • the program’s partnership with ROWPH has been structured as a mutually beneficial partnership;
  • the composition of the Advisory Committee has brought together a large and diverse group, and was well chosen; and
  • scientific choices in the program design—including sampling, testing and the overall methodology—were carefully considered and selected.

The pilot nature of the project was the most commonly identified factor impeding the success of the program.  The fact that there is just one sentinel site was mentioned by several stakeholders as having limited the usefulness of the information produced.

It was also noted that there had initially been insufficient communication between C-EnterNet and ROWPH senior management.  This had resulted in a lack of awareness among senior management of C-EnterNet activities involving the use of the data collected through the health department.  The establishment of a Steering Committee has resolved this issue and helped to put into place a formal approval process, according to interviewees.

The lack of predictable, sustainable, long-term funding was mentioned by a few interviewees as an impediment to the program’s success.  It was noted that multiple funding partners would help with the sustainability of the program and would help to increase the level of buy-in from other agencies.  It was also noted that the limited staff resources for the program had been a challenge.

3.2.2 Suggested Potential for a Network of Sentinel Sites across Canada

Finding: The C-EnterNet design and delivery structure is flexible enough to be adapted to different sites across Canada.  The geographic distance between the C-EnterNet team and other potential sites and the time required to establish sentinel sites will be a challenge for any expansion.  Effective management processes and structure, as well as sufficient resources will be critical to success in any expansion efforts.  The current structure of the C-EnterNet team would be inappropriate for expansion. 

The majority of interviewed stakeholders indicated that it would be feasible to replicate the pilot sites to other sites across Canada, while the rest indicated they did not know.  It was noted by some interviewees that there is a degree of flexibility in the delivery of the program, as sentinel sites can, to some extent, be tailored to the capacity and needs of their communities. 

Based on the interviews and the focus group conducted at the ROWPH, there appears to be a very high degree of satisfaction from ROWPH with its involvement in the C-EnterNet Program.  The 2007 review of the C-EnterNet Program similarly concluded that the program’s partners had high levels of satisfaction with their participation in the program, the workload, the level of communication from the C-EnterNet team, and the relevance of the program22.

To achieve the similar success in other sites, expansion would necessitate interest in and capacity for the program in other regions, which would require significant work on the part of the C-EnterNet team.  Some interviewed stakeholders indicated that there has already been significant interest in other provinces in hosting sentinel sites.  This included Western provinces, one of which has commenced initial work with the C-EnterNet team in developing a sentinel site.  Another Western province which had indicated a strong interest in the development of a sentinel site was not able to progress at the time due to a restructuring of the public health system in that province.

Some challenges to expansion were noted by several interviewees.  These included:

  • the development of a pilot site in Waterloo benefited from the proximity of the C-EnterNet team in Guelph.  The logistics of managing a complex program from a greater distance was identified as a significant, though not insurmountable, challenge.  This is especially true given the collaborative nature of C-EnterNet, where there is a need to develop and manage relationships with many different organizations.
  • several interviewees noted that a challenge may be ensuring appropriate and long-term resources.  This was noted with respect to the varying level of available resources in different provinces, as well as related to an overall lack of long-term resources dedicated to the C-EnterNet Program.  The pressure to demonstrate significant results in the short-term in order to sustain funding was mentioned as a challenge.
  • laboratory capacity was also mentioned as a potential barrier to expansion, which was consistent with findings from the 2007 review of C-EnterNet. 

The 2007 review also noted that the workload burden on the C-EnterNet staff was already considerable, and that the energy and dedication required of the team during the implementation and maintenance of the pilot site would not be replicable across additional sites23.  Interviews undertaken for the present evaluation also suggest that the team would find the additional burden of more sentinel sites to be a challenge with the current team structure and number of personnel.

3.2.3 Program Management

Finding: The joint management structure and lack of formal management processes present a risk to continued program success.  The Advisory Committee and ROWPH Steering Committee have proven to be effective mechanisms for information sharing and validation, although less effective at helping to shape program planning.

Management Structure

Senior management responsibility for the C-EnterNet rests with CFEZID and LFZ, both located within PHAC’s Infectious Disease and Emergency Preparedness Branch.

Key informant interviews noted both advantages and disadvantages with the joint management structure.  The shared expertise and the potential to attract funding from either group were considered advantages.  It was noted, however, that the joint management does present a risk in terms of the program maintaining visibility and buy-in within PHAC.  Since the program rests within both centres of PHAC, it risks belonging, in effect, to neither.  As a result, the program risks lacking senior leadership and a “champion” within senior management.  Further, the joint management has lead to a misalignment in organizational hierarchy, as C-EnterNet team members from the LFZ are reporting to a program lead from CFEZID.

Planning, Performance Measurement and Reporting Structures

C-EnterNet management processes appear to have been somewhat ad hoc and unsystematic.  A formal annual program plan was said to have been developed in the early years of the program, but has not been consistently updated in recent years, nor has progress been routinely tracked.  Program progress has instead been monitored through internal team meetings.  There was recognition within the C-EnterNet team that, given numerous priorities and a limited number of staff, formal and consistent program planning has been somewhat ignored.  The 2007 review of the program noted that, “some C-EnterNet staff desire stronger management of the program, which they feel would improve the clarity and focus of C-EnterNet”24.

The program has no performance measurement framework.  Limited annual performance reporting had been undertaken as per the requirements of its APF funding, but this reporting was not used for informing internal program processes or planning.  At the level of outputs, many program activities—including, for example, the number of responses to data requests and presentations made—are logged by the project team.  The program has presented updates on its activities and outputs to its Advisory Committee on an annual basis.  No ongoing tracking or reporting of outcomes is undertaken, however. 

Prior to the present evaluation, the program commissioned three different program reviews: an examination of the appropriateness of the C-EnterNet design relative to identified best practices; an assessment of the public health utility of the program; and a determination of best practices that could be used in the development of other sentinel sites.  There was no evidence that the recommendations in these reviews had been systematically responded to or addressed, which was itself noted in the most recent evaluation25.

Advisory Committee

An Advisory Committee was developed by the C-EnterNet project team to assist in guiding the program.  The Advisory Committee’s objectives, as outlined in its terms of reference, are:

  • to advise on needs, priorities, goals and strategies for C-EnterNet’s activities, related to integrated, enhanced enteric disease sentinel surveillance in Canada; and
  • with collaborators in academia, government, and private industry, to inform the design and implementation of the national surveillance plan, to capture the incidence and exposure to enteric disease in Canada.

The Advisory Committee’s membership includes 27 experts representing various areas of relevance to the C-EnterNet Program.  Membership includes:

  • representatives from federal organizations including PHAC, Health Canada, AAFC, CFIA and the National Microbiology Laboratory;
  • representatives from provincial ministries of health and agriculture;
  • members of regional health bodies (both from the sentinel site and others);
  • researchers from universities and other research institutes;
  • a representative of the CDC FoodNet program; and
  • other representatives, including from private laboratories.

The Advisory Committee provided input on the initial framework and analytical design of the program.  For example, the Advisory Committee determined the criteria to be used for selection of sentinel sites.  Since implementation, it has provided advice on data analysis and scientific issues.  To date, the Advisory Committee has met four times in-person, as well as through regular teleconferences.

According to stakeholders, the Advisory Committee has provided useful feedback on program findings, deliverables and activities, and the C-EnterNet team has effectively acted on the feedback provided.  The considerable expertise of the members, representing a wide range of different fields, was considered a key strength of the committee.

Some Advisory Committee members mentioned, however, that the committee has been primarily reactive in its function.  It was felt by some that the Advisory Committee had not taken on an adequate role with respect to program planning, identifying opportunities for program promotion and knowledge dissemination, or in assisting with program funding.  It was suggested in stakeholder interviews that the Advisory Committee may also lack members at senior levels within their organizations.

Regional Steering Committee

The ROWPH Steering Committee was formed with the objective to coordinate C-EnterNet’s activities in the pilot sentinel site.  According to its terms of reference, the Steering Committee was mandated to provide a forum to discuss and advise on C-EnterNet activities.  Steering Committee members include the site coordinator and management within ROWPH as well as key members of the C-EnterNet team.  The Steering Committee was originally intended to meet quarterly and the C-EnterNet 2006-2007 Accomplishment Report notes that there had been four meetings of the Steering Committee during that year.

According to stakeholder interviews, the ROWPH Steering Committee addressed a perceived lack of communication between C-EnterNet and ROWPH management.  In particular, the committee ensures a venue for input and approval from ROWPH senior management on C-EnterNet research activities involving data from the public health department.  According to all interviewed members, the Steering Committee had successfully addressed initial communication issues and allowed ROWPH management to be regularly apprised of C-EnterNet activities. 

3.3 Program Success

3.3.1 Program Outputs

Finding: The program has produced a series of information products, including annual reports and peer-reviewed articles.  Program members have also organized and attended numerous presentations and other venues for knowledge dissemination.

The C-EnterNet Program has produced a series of information products since 2006. Program outputs have included annual and accomplishment reports on C-EnterNet results (three complete annual reports, one short annual report, and two accomplishment reports) and seven newsletters.  In addition, a total of 12 articles written or co-written by the C-EnterNet team have been published in, or accepted to, peer-reviewed journals.  Publications and reports are listed in Table 7.

Table 7. Articles and Reports Produced by C-EnterNet Program

Peer-Reviewed Articles

Grieg, J., Ravel, A., “Analysis of foodborne outbreak data reported internationally for source attribution,” International Journal of Food Microbiology 2009 130: 77-87.

Heywood, P., Cutler, C., Komorowski, C., Marshall, B., Burrows, K., Wang H.L.,“A Community Outbreak of Travel-Acquired Hepatitis A Transmitted by an Infected Food Handler,” Canada Communicable Disease Report. November 2007; 33 (12).

Hexemer, A.M., Bird, T.M., Garcia, H.P., Pintar, K., Pollari, F., Zentner, S.E., “An Investigation of Bacteriological and Chemical Water Quality and the Barriers to Private Well Water Sampling in a Southwestern Ontario Community,” Journal of Water and Health. 2008; 6 (4): 521-525.

Keegan, V., Majowicz, S., Pearl, D., Marshall, B., Sittler, N., Knowles, L., Wilson, J., “The Epidemiology of Enteric Disease in C-EnterNet’s Pilot Site, Waterloo Region, Ontario, 1990-2004,” Canadian Journal of Infectious Diseases and Medical Microbiology. (In Press).

Mattison, K., Shukla, A., Cook, A., Pollari, F., Friendship, R., Kelton, D., Bidawid, S., Farber, J.M., “Human Noroviruses in Swine and Cattle,” Emerging Infectious Diseases 2007, 13 (8): 1184-1188.

Nesbitt, A., Majowicz, S., Finley, R., Pollari, F., Pintar, K., Marshall, B., Cook, A., Sargeant, J., Wilson, J., Ribble, C., and Knowles, K., “Food Consumption Patterns in the Waterloo Region, Ontario, Canada: a Cross-Sectional Telephone Survey”.  BMC Public Health. 2008; 8 (370).

Pintar, K.D.M., Waltner-Toews, D., Charron, D.F., Pollari, F., McEwen, S.A., Fazil, A., “Adopting Risk Assessment as a Policy Tool for Informing Water Safety in Canada”. 2008. (Submitted).

Pintar, K.D.M., Pollari, F., Waltner-Toews, D., Charron, D.F., McEwen, S.A., Fazil, A., Nesbitt, A., “A Modified Case-Control Study of Cryptosporidiosis (Using Non-Cryptosporidium Infected Enteric Cases as Controls) in a Southwestern, Ontario Community”. Epidemiology and Infection. 2009. (Accepted).

Pintar, K.D.M., Waltner-Toews, D., Charron, D., Pollari, F., Fazil, A., McEwen, S.A., Nesbitt, A., Majowicz, S., “Water Consumption Habits of a Southwestern Ontario community.” 2008. (Submitted).

Pintar, K., Cook, A., Pollari,F.,Ravel, A., Lee, S., Odumeru, J., “Quantitative Effect of Refrigerated Storage Time on the Enumeration of Campylobacter, Listeria, and Salmonella on Artificially Inoculated Raw Chicken Meat,” Journal of Food Protection. 2007; 70 (3): 739-745.

Ravel, A., Grieg, J., Tinga, C., Todd, E., Campbell, G., Cassidy, M., Marshall, B., Pollari, F., “Exploring Historical Canadian Foodborne Outbreak Datasets For Human Illness Attribution Through Food”. Journal of Food Protection. 2009; 72(9): 1963-1976.

Ravel, A., Smolina, E., Sargeant, J.M., Cook, A., Marshall, B., Fleury, M., Pollari, F., “Factors Associated with Seasonality and Human Cases of Salmonellosis”.  (Submitted on September 2009).

Thomas, M.K., Majowicz, S.E., Pollari, F., Sockett, P.N., “Burden of Acute Gastrointestinal Illness in Canada, 1999-2007:Interim summary of NSAGI activities”. Canada Communicable Disease Report 2008; 34 (5): 8-13.

Annual and Other Reports

C-EnterNet 2005 Annual Report

C-EnterNet 2006 Annual Report

C-EnterNet 2007 Annual Report

C-EnterNet 2008 Annual Report (currently in final stage of writing)

C-EnterNet Accomplishment Report 2005-2006

C-EnterNet Accomplishment Report 2006-2007

C-EnterNet Short Report 2007

C-EnterNet Short Report 2008

Other outputs of the C-EnterNet Program included organizing eight workshops at the ROWPH.  These workshops have ranged from between 21 and 34 attendees each, with topics including outbreak management and response and enhancing investigation skills for case follow-up.

According to the program’s records, the C-EnterNet team undertook over 60 presentations to local, provincial and international audiences from 2005-2009, including meetings, poster and conference presentations.

3.3.2 Relevance, Appropriateness and Timeliness of Information Produced

Finding: Recipients of C-EnterNet information find the information useful, relevant, in an appropriate format, and of a high quality.  Timeliness of information appears to be somewhat of an issue, and the lack of pan-Canadian data has limited the relevance of the information being produced.  Most information recipients are making use of the information, and all major types of information are being used by a significant proportion of recipients.  The lack of nationally representative data is seen as the major gap in C-EnterNet information.

Recipients of C-EnterNet Information

The program maintains a database of stakeholder contacts, which it uses to distribute its annual reports and newsletters by email.  At the time of the evaluation there were 284 names in this database with valid email addresses.  As shown in Table 8, the largest proportion of these contacts was federal government workers (42.6%), followed by provincial government workers (20.4%) and academics (16.5%).

Table 8. C-EnterNet Information Recipients – By Stakeholder Group
Stakeholder Group Number Percentage
Federal 121 42.6%
Provincial 58 20.4%
Academic 47 16.5%
Other 43 15.1%
Regional health 10 3.5%
Laboratory (provincial and private) 5 1.8%
Total 284 100%

Source: C-EnterNet Program Information Recipient Database

Among surveyed C-EnterNet information recipients, 58.8% indicated they had received C-EnterNet information from annual reports, 42.2% indicated they had received newsletters, 28.4% had requested specific information or data, 16.7% had participated in a C-EnterNet workshop or training session, while 19.6% said they had obtained program information through other means.

Survey findings suggest that C-EnterNet information is generally being distributed to appropriate stakeholders who can make use of the information provided.  Only 8.3% of survey respondents indicated that C-EnterNet information has not been received by the appropriate person(s) in their organization.  However, 26.0% said they did not know if the information was being received by the appropriate person(s).

C-EnterNet information appears to be largely assisting the work of researchers and those involved in delivering services/programs, while a smaller proportion of information recipients are responsible for shaping or managing public policy or programs.  About one-half (51.0%) of survey respondents described their occupation as "conducting research and/or teaching", one-third (33.3%) indicated a primary involvement in the delivery or management of programs or services, 14.7% indicated they were involved in senior management and 2.9% undertook policy analysis.  In addition, 16.7% said they were involved in “other” types of occupational areas, including, for example, working in industry, for industry associations, and for non-profit groups.

As shown in Figure 2, surveyed respondents were most often working in the areas of public health/human health (49.5%) or in agri-food (32.7%), while the remainder indicated they were primarily engaged in work related to the environment (9.9%) or another field (7.9%).

Figure 2. Occupational Areas of C-EnterNet Information Recipients

Figure 2, Occupational Areas of C-EnterNet Information Recipients

Source: C-EnterNet Information Recipient Survey
N=101

Text Equivalent

ARCHIVED - Figure 2. Occupational Areas of C-EnterNet Information Recipients

Occupational area Percent
Public Health / Human Health 49.5
Agri-food 32.7
Environment 9.9
Other 7.9

Not surprisingly given the large number of stakeholders located in the sentinel site and in the federal government, information recipients were most often located in Ontario: 21.8% were located in either the Regional Municipality of Waterloo or in the National Capital Region, while 37.6% were located elsewhere in Ontario.  In addition, 38.6% were located elsewhere in Canada and two respondents were located outside of Canada. 

Usefulness, Format, Quality, Relevance and Timeliness of Information

Most survey respondents indicated that they have found the information received by C-EnterNet to be personally useful.  Over three-quarters (78.2%) agreed that the information was useful, 7.3% disagreed, and 14.6% indicated that they did not know.  In addition, 71.9% of respondents agreed that the information had been useful to their organization.  The significant proportion of respondents who selected the “Do Not Know” option for many of the survey questions, however, suggests there are gaps in recipient awareness and use of information products.

The vast majority (77.1%) of survey respondents also agreed that the information provided by C-EnterNet was high-quality.  Only 4.2% indicated that the information was not of high quality, while 18.8% said they did not know.  The 2007 review of the C-EnterNet Program similarly concluded, based on its review of a random sample of paper and electronic data records, that the data quality of C-EnterNet was high.  Most respondents (76.0%) also felt that the information was provided in a format that was user-friendly.

With respect to overall relevance, respondents were asked how important the information/data received from C-EnterNet had been to their organization.  Almost three-quarters (73.4%) said the information had been important, 5.3% said the information had been not very important and 21.3% said they did not know.  No respondents indicated that the information had not been important “at all”.

While timeliness of information appears to be satisfactory for most recipients, it did garner the most negative results of all information attributes explored in the survey.  Sixty percent (60.4%) of respondents agreed that the information was provided in a timely manner, while 12.5% disagreed and 27.1% said they did not know.  Further, among those survey respondents who indicated that the information received from C-EnterNet had not been used (either by them or their organization), one-third (33.3%) indicated that this was because the information had not been timely enough to be useful, the most common reason selected.

Figure 3. C-EnterNet Information Recipients’ Views on Information

Figure 3, C-EnterNet Information Recipients' Views on Information

Source: C-EnterNet Information User Survey
N=96

Text Equivalent

ARCHIVED - Figure 3. C-EnterNet Information Recipients' Views on Information

Program Component Strongly Agree / Agree Strongly Disagree / Disagree Don’t Know
Information has been useful to me. 78.20% 7.30% 14.60%
Information has been of high quality. 77.10% 4.20% 18.80%
Information is in a user-friendly format. 76.00% 8.30% 15.60%
Information has been useful to my organization. 71.90% 4.20% 24.00%
Information provided in a timely manner. 60.40% 12.50% 27.10%

Frequency of Use of Information

Most surveyed recipients indicated they were making use of the information they had received from the C-EnterNet Program.  Overall, two-thirds (67.0%) of survey respondents said that the information had been put to use by the respondent themselves or by someone else in their organization.

There were no major types of information that did not appear to be useful to a significant proportion of information recipients.  As shown in Table 9, the most commonly used type of information was “overall number of human cases of enteric diseases, broken down by travel- and outbreak-related cases and endemic cases”.  While the results of episodic studies were the least commonly used, this is consistent with their more targeted research areas (such as food consumption patterns).

Table 9. Proportion of Survey Respondents Indicating that Different Types of Information had been used by them or their Organization
Type of Information Proportion Used
Overall number of human cases of enteric diseases, broken down by travel- and outbreak-related cases and endemic cases 42.2%
Detection/number of pathogens in retail/meat sample 40.2%
Integration of surveillance data and studies on human illness attribution 37.3%
Annual and seasonal trends in human enteric disease 35.3%
Human cases with detailed microbial information (subtyping data) 35.3%
Detailed microbial information (subtyping data) in retail (meat), farm (manure) and/or water samples 32.4%
Detection of pathogens in farm (manure) samples 31.4%
Detection/number of pathogens in water samples 30.4%
Comparisons of detailed microbial information between human isolates and isolates from food, food animals or water 30.4%
Annual and seasonal trends in exposure sources 29.4%
Human endemic cases with exposure data 26.5%
News of C-EnterNet activities 20.6%
Results of episodic (targeted) studies 18.6%

Source: C-EnterNet Information User Survey
N=102

Data Requests

The C-EnterNet Program has maintained a record of data requests it has received from outside organizations.  Between 2007 and 2009, C-EnterNet logged 31 data requests.  These requests were from a range of federal, provincial and municipal government organizations, non-governmental organizations, and academics.  Ten requests for data were made from PulseNet, a virtual electronic network of provincial and federal public health laboratories that tracks the DNA fingerprints of cases of E. coli and Salmonella.  Data requests made to the program were generally responded to quickly, usually within one day.  

Information Gaps

Interviewed stakeholders were asked whether there were any gaps in the information produced by the C-EnterNet program.  The major identified gap was that there is only one sentinel site. As a result, there was a lack of national data, or data that were widely generalizable.

Interviewees noted that many of the information gaps of the early days of the program were now filled, including the lack of case control data (which was collected through a targeted study undertaken by the program in 2009) and data on all critical elements, including, most recently, produce (for which data collection began in April 2009). 

Interviewees less commonly noted other gaps in information, though no major or consistent trends emerged from these responses.  These other responses included progress on source attribution, a conceptual framework, more testing of different types of pathogens in case samples, testing of water in wells and recreational facilities, and additional foods to be tested (fish, seafood, milk products, eggs, food in restaurants, and others).

It was also noted that processes within the sentinel site are not documented.  As a result, loss of human resources would result in significant knowledge loss.  One interviewee also noted that there appeared to be a lack of information geared specifically to policy recommendations coming out of the program.

Responses from the survey of information recipients were similar.  About one-third (31.4%) of surveyed information recipients indicated that there was additional information or data on enteric disease and pathogens that it would be useful for C-EnterNet to provide.  Again, respondents most frequently indicated that data should be more geographically representative through the implementation of more sentinel sites. 

3.3.3 Collaborative Networks

Finding: C-EnterNet has successfully established and developed collaborative networks across multiple disciplines and including several levels of government.  These networks have extended from the local to the international in scope.

Partnerships, both formal and informal, have been a key component of the C-EnterNet Program since its inception.  In addition to its central partnership with the ROWPH in the first sentinel site, partnerships have been developed with a wide range of organizations. According to C-EnterNet administrative data, by 2008 the program had established approximately 40 partnerships with organizations assisting with data and sample collection and testing.  These included national, provincial and private laboratories, other federal government departments, local health units/departments, a hospital, and universities. In addition, there were sixty-five confirmed partnerships for which C-EnterNet shared its data, including laboratories, provincial ministries of health and agriculture, regional and municipal health organizations, other areas within PHAC, other federal organizations, universities, and overseas partners from the United States and Australia. 

The C-EnterNet Program has also provided venues for networking and collaboration through organizing meetings, consultations, workshops, and other venues.  The program has organized six Advisory Committee meetings and eight workshops at the Region of Waterloo.  In addition, according to program data, the program participated in 62 presentations and meetings from 2005-2008, although not all of these meetings were organized by the program itself.

Among interviewed stakeholders, there was consensus that C-EnterNet has successfully developed collaborative networks related to enteric disease.  Interviewees often noted that the Advisory Committee had brought together a diverse group of professionals, including individuals from different levels of government and with different areas of expertise. C-EnterNet has also furthered collaboration between practitioners and labs through, for example, organizing a 2007 meeting of nine laboratories supporting the program to discuss methodological issues related to the program.

The eight workshops at the ROWPH organized by C-EnterNet had resulted in public health inspectors and other staff members making connections with experts brought in for presentations.  Having the support of the C-EnterNet team has also allowed the ROWPH to have access to C-EnterNet's own networks of experts, which was said to have been useful in connecting with epidemiologists and seeking advice from other experts.  Participation in C-EnterNet has also led to the ROWPH connecting with other health units/departments in Ontario to share information and best practices related to enteric illness, as ROWPH has come to be seen as a leader in this area within the province.

Other forms of collaborative networks facilitated by the program mentioned by interviewees include:

  • the source attribution working group;
  • information-sharing with experts in source attribution in Denmark and in the US, and regular contact with CDC FoodNet; and
  • episodic research activities have led to collaboration, including research that has brought together C-EnterNet, PulseNet, Grand River Hospital, and others.

The 2007 review of the C-EnterNet program similarly concluded that the program had been effective in establishing and facilitating communication and collaboration among organizations that otherwise would not have occurred26.

3.3.4 Advancement of Source Attribution Methodologies

Finding: C-EnterNet has undertaken work to bring together scientists to advance source attribution and to review source attribution methodologies developed outside Canada.  The program has also undertaken some preliminary work to adapt approaches to the Canadian context.

Source attribution has been defined as the partitioning of the human disease burden of one or more food-borne infections to specific sources, where the term source includes animal reservoirs and vehicles (e.g., foods)27.  One of the key objectives of the C-EnterNet Program is to determine the proportion of human cases of enteric illness that are due to water, food and animal contact.  Determining the sources of illness is important in helping to shape and inform effective and well-targeted prevention and control activities.

According to interviewed stakeholders, there was no substantial work being undertaken in Canada on source attribution prior to C-EnterNet.  C-EnterNet has undertaken a number of related activities, and was said to have taken a position of international leadership in this area, along with Denmark.

In order to assist in advancing source attribution methodologies, C-EnterNet has brought together and led an international source attribution working group.  This group includes scientists from PHAC, OMAFRA, the British Columbia Centre for Disease Control, and Michigan State University28.  In addition to the working group, C-EnterNet has also collaborated with Danish and American scientists working in this area.

In order to be able to build on the work already completed in this area, C-EnterNet undertook a review of source attribution methodologies used internationally.  The review has allowed the program to clarify terminology and concepts, and to assess what is appropriate for the Canadian context.  According to one stakeholder, the working group is “grappling with how to do the best it can with the data that exists.”  Seven different methodological approaches to source attribution have been identified, and work remains ongoing using several of these approaches.  This includes utilizing a microbial subtyping design through adaptation of a Danish model to available Canadian data.

In an internal review of the C-EnterNet Program undertaken in early 2008, surveyed C-EnterNet team members and other stakeholders indicated that source attribution methodology development had not, at that time, been adequate.  It was suggested that priority and resources at that time had been focused on surveillance activities at the expense of this component.  It was noted that considerable time is required to select a best available technique for source attribution.  During the present evaluation, a few stakeholders noted that progress has been relatively slow in this area, though there was recognition of the complexities inherent in this work.

As a result of the work done to-date in source attribution, five associated papers have been prepared for publication (in collaboration with experts outside the program).  According to interviewees, three of the five have been published, and work on the remaining two is ongoing.

3.3.5 Increased Knowledge of Enteric Diseases and Exposures

Finding: Overall, C-EnterNet has had a positive impact on the level of knowledge of enteric diseases and exposures among stakeholders.

Survey findings demonstrate that C-EnterNet is perceived to have increased levels of knowledge of enteric disease and exposures among recipients of information products.  Most (78.2%) surveyed recipients of C-EnterNet information agreed that the program had increased their level of knowledge, while 11.5% disagreed and 10.4% said they did not know.  Further, respondents were asked to what degree the information had increased the level of knowledge of enteric disease and exposures in their organization.  A majority (57.3%) agreed that it had, while 32.3% said they did not know, and 10.4% indicated the information had not increased the level of knowledge.

Interviewed stakeholders were similarly positive on the degree to which the level of knowledge had been raised by the program.  Examples cited as evidence included:

  • the level of knowledge of ROWPH front-line staff has been raised by improved and more consistent training provided through the program;
  • the early findings presented in program deliverables were promising and had begun to advance the field.  This included the finding that travel-related cases comprised about 30% of all cases of enteric illness; and
  • the success of the Advisory Committee as a venue for information sharing.

Many interviewees did, however, note that more time and additional sentinel sites were needed to begin to see major trends and findings from C-EnterNet data that would be of wide interest.

ROWPH staff also cited examples in which C-EnterNet has helped increase their knowledge on risk factors for enteric illness.  One staff person noted that: “many risk factors have been confirmed, as we have more data to back them up, especially with seasonal trends, food handling, and issues around using barbeques.  This information can then be used for public education pieces or campaigns.”

The wide range and large number of presentations undertaken by the project team can reasonably be expected to have increased knowledge in the field.  C-EnterNet has also contributed to increasing knowledge in the field through disseminating its research in academic journals.  To date, 12 articles have been published or accepted for publication in peer-reviewed journals.

3.3.6 Strengthening Public Health Capacity

Finding: C-EnterNet has strengthened public health capacity within the Region of Waterloo and, to a lesser degree, outside the sentinel site area.

An important medium-term outcome of the C-EnterNet Program is strengthening public health capacity.  Results of the evaluation suggest that the C-EnterNet Program has increased public health capacity within the ROWPH in several ways.

The C-EnterNet Program developed a partnership with ROWPH that is mutually beneficial.  As a result of its participation in the program, ROWPH has:

  • developed new case follow-up questionnaires, which allow the public health department to collect richer data on risk factors and exposures among identified cases of enteric illness in the region.  Although not all members of management within the public health department were satisfied with the questionnaire and it remains a work-in-progress, it was widely considered to have improved the amount of data available.  This has led to the public health department having identified groups who are at an increased risk of enteric illness within its region (including members of an ethnic minority who were consuming unpasteurized milk) in order to better target its preventative health messages;
  • improved training for public health inspectors and other staff.  Both in interviews and in the focus group with staff, members of the public health department emphasized that the training and workshops provided through the program have increased their knowledge of enteric illness and exposures; and
  • developed new “fact sheets” related to enteric illness, which are distributed to those who have a reported illness or would like more information on this topic.

The C-EnterNet team also worked with ROWPH and PHAC’s Outbreak Response and Issues Management section to improve outbreak management in childcare facilities.  This was undertaken through collaborative analysis and publication of an E. coli outbreak in a childcare facility29 and qualitative research related to childcare facility protocols30.

There was more limited evidence that C-EnterNet was helping to strengthen public health capacity outside the sentinel site area.  Some members of the Advisory Committee and recipients of C-EnterNet information products are employed in public health outside of the Region of Waterloo, including some located outside of Ontario.  One stakeholder at the Ontario Ministry of Health and Long-Term Care responsible for outbreak investigation was working with the C-EnterNet Program to further disseminate program findings, and had made use of C-EnterNet data (including from the food consumption survey) in its analysis and in presentations to its own stakeholders (such as food industry associations). 

There was some suggestion that more could be done to inform other health regions within Ontario of what was happening within the C-EnterNet sentinel site, perhaps led by the Ontario Ministry of Health and Long-Term Care.  It was suggested that this could lead to further adoption of best practices in case follow-up and related data collection.

Finally, public health capacity has been supported through the program having provided additional data and opportunities for networking and collaboration among laboratories.  For example, the National Microbiology Laboratory has had access to isolates with which it can undertake additional analysis beyond that directly related to C-EnterNet’s needs.

3.3.7 Uses of Information

Finding: The information produced by C-EnterNet is largely being used to support research activities, and to inform the development of programs.

The information produced by the C-EnterNet Program is expected to inform, create or help to evaluate programs, policies or health messages related to enteric illness. 

Two-thirds (67.0%) of survey respondents said that the information received from C-EnterNet had been put to use by the respondent themselves or by someone else in their organization.  A somewhat higher proportion of academics said that the information had been put to use (87.5%), compared to regional health workers (75.0%), federal government employees (69.6%), or provincial employees (57.9%).  A significant proportion of all groups indicated they did not know if the information had been put to use.

Among those respondents who said that C-EnterNet information had been used by their organization, 70.7% said that the information had been used to inform the development of programs.  Nearly one-half (48.7%) said that the information had been used to support the development of policies and practices (48.7%), while one-third (33.3%) said that the information had supported the development of health messages.

C-EnterNet data was also providing support for evaluation functions: 43.6% said that the information had supported the evaluation of programs, 27.8% said the information had supported the evaluation of health messages, and 27.0% said that it had supported the evaluation of policies or practices.

Table 10. Frequency of Different Usages of C-EnterNet Information, Among Respondents Stating They/Their Organization Had Made Use of C-EnterNet Information
Type of Use % saying information used for this purpose at least once
To support the development of programs 70.7%
To support the development of policies and practices 48.7%
To support the development of health messages 33.3%
To support the evaluation of programs 43.6%
To support the evaluation of policies or practices 27.0%
To support the evaluation of health messages 27.8%

Source: C-EnterNet Information User Survey
N=36-41

When asked to elaborate on how information had been used, respondents most commonly provided examples that can be grouped into three broad areas: risk assessment/analysis and source attribution; developing or undertaking research and/or assisting with teaching; and to assist in program development.  In addition, respondents mentioned a large number of other uses, including that the information had been used for policy development, reviewing lab procedures, and, in one noted instance, to inform outbreak response.

Among the many specific examples of uses of the data provided in survey responses included:

  • “Information and data from C-EnterNet has been used to support research proposals that our group has submitted to various granting agencies […]  The data has been used to support project reports (Canada Water Network and the Natural Sciences & Engineering Research Council), journal publications and conference presentations by our group.  In addition, C-EnterNet data has been used when reviewing proposed federal government guidelines on drinking water.”
  • “I have used the information in a number of conference presentations regarding zoonotic transmission.”
  • “Information on sources of Salmonella Enteritidis from various meats used by the Ontario Salmonella Enteritidis Working Group.”
  • “The data has been used in models to set food safety targets.”
  • “Provide media with incidence rate of microbial food-borne illness in Canada.”

When asked to describe the major impacts of the C-EnterNet information, respondents, again, commonly indicated the program had helped to support research and identify research gaps.  Some related comments included that the information had supported “future directions in research,” “being able to reflect the actual occurrence of pathogens in Canada […] in a research and teaching setting”.

Interviewed stakeholders were evenly split on whether the information produced by C-EnterNet had resulted in their making any changes to programs, policies or practices.  Some academics noted that the program would not apply to them in this way.  Others noted that there was still time needed before the findings would reveal substantial trends.

Among those who said that there had been changes made, examples included:

  • staff training practices had improved at ROWPH through staff being provided with additional training by the C-EnterNet team;
  • ROWPH had altered its messaging with respect to the problems in its area around raw milk consumption; and
  • C-EnterNet data had been used by government workers in another province to estimate the cost of enteric illness to their province, which was successfully used to lobby for additional funding in this area.

It was noted by a few stakeholders that the program could benefit from better marketing and needed to identify key decision-makers who would benefit from the information produced.  It was also suggested that the implications of the findings on policy and practice were not, as yet, being widely considered or communicated.

3.3.8 Source Attribution Estimates

Finding: Progress has been made in developing source attribution estimates, but comprehensive source attribution using the integrated data from the sentinel site has not been completed to-date.

Producing source attribution estimates is dependent on progress made in the development of source attribution methodologies.  C-EnterNet’s review of methodologies identified seven major types of potential approaches.  A June 2009 update on C-EnterNet’s activities and outputs related to source attribution outlined work completed or in-progress in five of these seven types: microbial subtyping; case-control studies; quantitative microbial risk assessment; analysis of data from outbreaks; and expert elicitation.

Source attribution activities using the data collected from the C-EnterNet sentinel site started with the descriptive comparisons of sub-typing data for various pathogens from human cases of illness and potential sources.  These data have been presented in the annual reports produced by the program.  The application and adaptation to the Canadian data of a salmonellosis source attribution model developed in Denmark, using subtyping data from human cases and from various potential sources of salmonellosis, has been undertaken.  This project uses data collected by the C-EnterNet Program as well as data coming from other national surveillance systems (CIPARS, CFIA).

C-EnterNet data was also used to inform analysis of risks of the targeted enteric diseases in its annual reports using a case-cases comparison approach.  A more formal analysis of cryptosporidiosis was conducted and results published in a peer-reviewed publication.  With four years of data, a special analysis of C-EnterNet data started in August 2009 to describe the burden of travel-related cases of enteric diseases, which was considered quite high (i.e., 28% of non outbreak related cases).

The quantitative microbial risk assessment approach was finalized on cryptosporidiosis in relation to recreational water and drinking water and the results were submitted for publication to two peer-reviewed journals. 

Other available data sources have been used by the C-EnterNet team and the source attribution working group to develop preliminary source attribution estimates for Canada.  Two projects dealt with food-borne outbreak data.  One used three sets of food-borne outbreak data in Canada covering 30 years (1976 to 2005) provided by Health Canada and PHAC.  It showed changes over time in source attribution and provided the most up-to-date source attribution estimates based on outbreak data for Canada.  The paper was recently published by the Journal of Food Protection31.  The second project used records of outbreaks from all over the world between 1998 and 2007 and showed some geographical variation in source attributions for some diseases.

Expert opinion is another approach to develop estimates on source attribution.  An expert elicitation survey was conducted in 2008.  It identified 154 food safety experts and asked them, with respect to 10 enteric diseases, their best estimates for the proportion of disease being food-borne and associated with 12 food categories.  The analysis is currently in progress and a first paper is currently being written highlighting areas of uncertainty and disagreement between experts.

It was noted in the 2007 review of the program that partners were eager to see results on source attribution and that, in relation to the surveillance component of the program, progress had been slow.  It was concluded that acceptability of the program among its partners and advisors would suffer if source attribution was not adequately addressed, as it was a major interest of those interviewed32.  It should be noted that though Denmark, the Netherlands, UK, Australia, New Zealand, and CDC FoodNet in the US have been working on source attribution over the past decade, the results have been limited (except related to salmonellosis in Denmark and the Netherlands) due to methodological challenges and data limitations.

3.3.9 Contribution to Reducing the Burden of Human Enteric Illness

Finding: Evidence from the evaluation suggests that the program has the potential to contribute to reducing the burden of human enteric illness if allowed more time and with additional sentinel sites in place.

The ultimate outcome of the C-EnterNet Program is to contribute to reducing the burden of human enteric illness in Canada.  As indicated by most interviewed stakeholders, it is not yet reasonable to expect the program to have had a substantial impact in this area.  Most stakeholders felt that the program required more time and more years of data to begin to have its full impact.  Many noted that there is a need for additional sentinel sites for the program to make substantial progress.

The evaluation findings do, however, suggest that a foundation has been laid for future progress in the achievement of this outcome.  In building collaboration and in testing its approach and methodology in its pilot phase, the program has shown positive indications that future expansion would advance the science and strengthen public health capacity related to enteric illness.  Findings from across the evaluation questions suggest that implementation of a network of sentinel sites would make a contribution to reducing the burden of human enteric illness. 

3.4 Cost-effectiveness and Alternatives

3.4.1 Alternative Design/Delivery Approaches

Finding: The program design and delivery approach appears to have been developed to ensure maximum cost-efficiency given the methodology selected for the C-EnterNet Program.  Moving forward, the program should examine the feasibility of other cost-sharing arrangements, or cost efficiencies from further collaboration with other organizations or programs.

C-EnterNet’s use of a sentinel site approach attempts to strike an appropriate balance between cost and scientific rigour.  By concentrating resources in a limited number of communities, data can be gathered that is richer and more detailed than those available from broader national surveillance programs, while still remaining cost-effective given the limited geographic areas of the sentinel sites.  Further, potential cost-efficiencies have been factored into the selection criteria for sentinel sites, which includes local capacity for sample collection and laboratory.  In the delivery of the program, the C-EnterNet team has included a broad range of partners in order to help minimize costs.

Interviewed stakeholders were asked for suggestions on any modifications to the design and delivery of the program that could improve cost-effectiveness.  The majority of suggestions involved C-EnterNet examining the possibility of working with other agencies to reduce its own role in coordinating the sampling and testing.  Several interviewed stakeholders suggested that increased level of collaboration between C-EnterNet may be possible with provincial ministries of environment and agriculture, and potentially with other federal organizations, including AAFC, and CFIA.

Some interviewees indicated that relying on provincial government organizations to assist in data collection may be too time-consuming and could lead to data collected not being fully aligned with C-EnterNet’s needs.  However, it was suggested that the program could examine increasing collaboration with others organizations or programs, such as CIPARS.  Documentation on CIPARS similarly suggests that there is potential for increased collaboration.  For example, CIPARS is collecting data that could support the retail and on-farm components of the program.  Furthermore, the original business plan for C-EnterNet indicated that data from CIPARS would be used for C-EnterNet’s food/retail component33

A few interviewees emphasized that, as the strength and quantity of the information increases in the future, more effort is required in formulating analysis and policy implications of the findings, as well as disseminating the information to decision-makers.  It was suggested that this could be done by collaborating on knowledge exchange with CIPARS.

It was also suggested that, although it may not have an impact on the total costs, there could be a broader sharing of costs across different organizations. The CDC FoodNet Program was raised as a potential funding model, as its costs are shared between the federal government and the states hosting sentinel sites.

3.4.2 Value for Cost

Finding: The C-EnterNet Program has provided good value for costs, as it has attracted free support from partners and appears to have kept overhead expenses to a reasonable level.

The evaluation included a costing of the program using program financial data.  Costs were categorized according to whether they were direct or indirect.  Direct costs are those that link directly to one or more of four major components of the program (i.e., human, water, farm and retail surveillance components), to the development of publications or to the operation of the C-EnterNet Advisory Committee.  They include related salaries, employee benefits, expenses and contracts.   Table 11 provides a detailed breakdown of the direct costs of the C-EnterNet Program for one year (based on data from 2006-2007).

Table 11. Annual Direct Costs of C-EnterNet Program
Program Component Direct Costs* ($’s)
Salaries Employee Benefit Plan Expenses Contract In-Kind Support Total
Water 84,217.20 16,843.44 -- 60,000.00 106,683.38 267,744.02
Human 138,350.40 27,670.08 -- 188,933.00 -- 354,953.48
On-Farm 44,643.60 8,928.72 32,455.00 68,660.00 4,000.00 158,687.32
Retail 55,485.60 11,097.12 28,150.00 82,631.00 -- 177,363.72
Newsletter Publication   -- 17,737.00     17,737.00
Advisory Council   -- 11,152.00     11,152.00
Total 322,696.80 64,539.36 89,494.00 400,224.00 110,683.38 987,637.54

* Employee Benefit Plan (EBP) expenses are employee benefit expenses and are assessed at 20% of salary expenses, as per Treasury Board guidelines.  Expenses include other items used to support operations.  Contracts were also used for sample collection and analysis.  Several partners provide related support for free, and the estimated value of this support has been included under in-kind support.
Source: C-EnterNet Program Financial Data, Costs Survey

Indirect costs are costs not directly linked to the program components.  They include accommodation, office expenses, management costs including the salaries of managers and assistants, and other program support.  Table 12 provides a detailed breakdown of indirect costs.

Table 12. Annual Indirect Costs of C-EnterNet Program
Indirect Costs* ($’s)
Accommodation Salaries Employee Benefit Plan Expenses Contracts In-Kind Support Total
68,513.95 204,333.60 40,866.72< 102,712.00 12,798.00 1,100.00 430,324.27

* Accommodation costs are calculated at a standard rate of 13% of salary expenses, while employee benefit plan (EBP) are calculated at 20% of salaries, as per Treasury Board guidelines.  Several contracts for exploratory work, and in-kind support for general program support, have been included as indirect expenses.
Source: C-EnterNet Program Financial Data, Costs Survey

Overall, indirect costs make up 31% of total program expenses.  This is a relatively low proportion and suggests the program is demonstrating value for money in this regard, as most funds are used to directly support program activities.

Table 13 illustrates a breakdown of total program costs by program component.  In these figures, total indirect costs have been estimated for each of the program component34

Table 13. Annual Costs per Program Component
Program Component Total Cost ($’s), Including In-kind Support from Partners Proportion of Total Costs
Human 539,447.21 38.0%
Water 383,102.70 26.8%
Retail 253,366.55 17.7%
On-Farm 219,839.55 15.4%
Newsletter Publication 17,737.00 1.3%
Advisory Council 11,152.00 0.8%
  1,429,659.81 100%

Source: C-EnterNet Program Financial Data, Costs Survey

As demonstrated, the enhanced human surveillance component constituted the largest proportion of total costs (at 38.0%), followed by the water component (26.8%).  Costs for the human component included a contract with the ROWPH for staffing the Site Coordinator position within the public health department, and laboratory analysis for suspected cases of enteric illness.

Many partners have found value in the program and have provided in-kind support, free of charge, to support program activities.  Over $110,000 of in-kind support is being received annually, mainly for analytical activities in support of the water component.  Other partners have provided time to support the program in terms of strategic advice and speaking at presentations.  Private laboratories have identified samples of interest to the C-EnterNet Program and have not charged for this work.  Significant time has also been provided by ROWPH management and staff without financial compensation. 

Current expenses do not appear to include activities that do not support the efficient achievement of expected outcomes.  It is unknown if C-EnterNet could develop capacity to undertake the work currently contracted out, or whether it could undertake the work for the same cost.


1 Thomas, M.K., et al., “Burden of Acute Gastrointestinal Illness in Canada, 1999-2007: Interim Summary of NSAGI Activities,” Canada Communicable Disease Report Vol. 34, No. 5 (May 2008).  Accessed online at: http://www.phac-aspc.gc.ca/publicat/ccdr-rmtc/08vol34/dr-rm3405b-eng.php

2 Standing Committee on Agriculture and Agri-Food.  Report to the House of Commons: Beyond the Listeriosis Crisis: Strengthening the Food Safety System, June 2009.  Page 7.

3 Government of Canada.  Report of the Independent Investigator of the 2008 Listeria Outbreak, June 2009. Accessed online at: http://news.gc.ca/web/article-eng.do?m=/index&nid=468909 (External link)

4 Majowicz, S.E. et al.  “Burden and Cost of Gastroenteritis in a Canadian Community,” Journal of Food Protection, Vol. 69, No. 3 (2006): 651-659.

5 Wheeler, J.G. et al.  Study of Infectious Intestinal Disease in England : Rates in the Community, Presenting to General Practice, and Reported to National Surveillance.  The Infectious Intestinal Disease Executive. BMJ, 318 (7190) (1999): 1046-50.

6 Majowicz, S.E. et al.  “Burden and Cost of Gastroenteritis in a Canadian Community,” Journal of Food Protection, Vol. 69, No. 3 (2006): 651-659. Page 658.

7 El Allaki, Farouk.  Evaluation Report for C-EnterNet Based on Health Surveillance Theory, March 2005. Page 2.

8 Government of Canada.  Report of the Independent Investigator of the 2008 Listeria Outbreak, June 2009. Accessed online at: http://news.gc.ca/web/article-eng.do?m=/index&nid=468909 (External link)

9 Public Health Agency of Canada.  National Enteric Disease Rates, 1994-2004.

10 Public Health Agency of Canada.  Establishing C-EnterNet: A Business Case, November 1, 2004.

11 Public Health Agency of Canada.  C-EnterNet Overview. Page 4.

12 Standing Committee on Agriculture and Agri-Food.  Report to the House of Commons: Beyond the Listeriosis Crisis: Strengthening the Food Safety System, June 2009.  Page 35.

13 El Allaki, Farouk.  Evaluation Report for C-EnterNet Based on Health Surveillance Theory, March 2005. Page 2.

14 World Health Organization Consultation to Develop a Strategy to Estimate the Global Burden of Foodborne Diseases 25-27 September 2006.  Accessed online at: http://www.who.int/foodsafety/publications/foodborne_disease/burden_sept06/en (External link)

15 Government of Canada.  Public Health Agency of Canada Reports on Plans and Priorities 2008-2009.  Accessed online at: http://www.tbs-sct.gc.ca/rpp/2008-2009/inst/ahs/ahs01-eng.asp (External link)

16 Region of Waterloo Public Health website, accessed at:
http://chd.region.waterloo.on.ca/web/health.nsf/DocID
/D908471BF1AF7A7B85256F04005707D9?OpenDocument (External link)

17 Gilbert, M. et al., “Screening policies for daycare attendees: lessons learned from an outbreak of E. coli O157:H7 in a daycare in Waterloo, Ontario,” Canadian Journal of Public Health July-August 2008; 99 (4): 281-5.

18 Taylor, M. et al., Master’s thesis, 2008.

19 Standing Committee on Agriculture and Agri-Food.  Report to the House of Commons: Beyond the Listeriosis Crisis: Strengthening the Food Safety System, June 2009.  Page 3.

20 Standing Committee on Agriculture and Agri-Food.  Report to the House of Commons: Beyond the Listeriosis Crisis: Strengthening the Food Safety System, June 2009.  Page 4.

21 Public Health Agency of Canada.  Establishing C-EnterNet:  A Business Case, November 1, 2004.  Page 6.

22 Zinszer, Kate.  Evaluation of C-EnterNet, June 30, 2007.

23 Zinszer, Kate.  Evaluation of C-EnterNet, June 30, 2007.

24 Zinszer, Kate.  Evaluation of C-EnterNet, June 30, 2007. Page 7.

25 Framst, Gordon.  C-EnterNet Pilot Sentinel Site Evaluation, March 2008.

26 Zinszer, Kate.  Evaluation of C-EnterNet, June 30, 2007. Page 7.

27 Pires, S. M. et al. “Attributing the Human Disease Burden of Foodborne Infections to Specific Sources,” Foodborne Pathogens and Disease 6 (2009): 417-424.

28 Public Health Agency of Canada.  C-EnterNet Accomplishment Report 2006-2007. Page 14.

29 Gilbert, M. et al., “Screening policies for daycare attendees: lessons learned from an outbreak of E. coli O157:H7 in a daycare in Waterloo, Ontario,” Canadian Journal of Public Health July-August 2008; 99 (4): 281-5.

30 Taylor, M. et al., Master’s thesis, 2008.

31 Ravel, A., Greig, et al.  “Exploring Historical Canadian Foodborne Outbreak Data Sets for Human Illness Attribution”. Journal of Food Protection, 2009; 72(9): 1963-1976.

32 Zinszer, Kate.  Evaluation of C-EnterNet, June 30, 2007.

33 Public Health Agency of Canada.  A Business Plan for Launching the First C-EnterNet Sentinel Site, September 2004. Page 9.

34 Indirect expenses are allocated to program components through the calculation of an overhead ratio.  This is done by dividing total indirect expenses by salaries within the program components.  This ratio is then multiplied by salaries for each program component, and these allocated indirect costs are added to the program component.  For C-EnterNet, total indirect costs are $442,022 divided by salaries of $322,969, for an overhead rate of $1.37 per salary dollar.

Page details

Date modified: