ARCHIVED - Evaluation of the C-EnterNet Program Final Report

Conclusions and Recommendations

The following section provides a summary of the overall conclusions of the evaluation, with related recommendations. 

Relevance

The evaluation found a strong rationale for the C-EnterNet Program.  Enteric disease is prevalent and costly to society, and its economic burden is likely to increase given current trends in food consumption and distribution, demographics and environmental changes.  The C-EnterNet Program is aligned with national priorities related to food and water safety and with international surveillance initiatives such as the CDC FoodNet in the United States. 

C-EnterNet’s design includes integrated surveillance of human cases, as well as of pathogens in water, retail and farm animal sources.  The program also involves the development of source attribution methodologies and estimates to link cases of illness with their specific sources.  All components of the program are widely seen as relevant to informing future initiatives related to prevention and control.

While C-EnterNet involves partners from many types of organizations and from all levels of government, federal leadership of the program is appropriate given the program’s national focus and its alignment with federal jurisdictions in health and food safety.  While C-EnterNet is located within PHAC, its activities and impact extend beyond public health into areas of food and water safety, the environment, and agriculture.  Indeed, its primary funding has been from AAFC and, more recently, from federal funding dedicated to enhancing food safety.  Given the multi-disciplinary nature of the program and the range of areas potentially benefitting from its scientific results, future funding from other federal and provincial organizations and programs would be appropriate and in line with the funding model used by CDC FoodNet.

While the C-EnterNet Program does not, as a whole, duplicate the work of other programs or organizations, program complementarity would likely be improved through closer collaboration with CIPARS and, potentially, with other organizations.  While the program has developed a variety of partnerships to assist in sampling, testing, and analysis, a comprehensive assessment of potential future partners would help to support planning, particularly as the program is working to implement a second sentinel site in 2009-2010.

Recommendation #1: The potential for increased collaboration with other federal and provincial organizations and programs should be assessed through a formal and systematic review.  The review should examine whether activities can be achieved through pre-existing processes in use by other organizations, and whether increased complementarity can be pursued with CIPARS and others.  This review should also seek to identify any potential future funding partners.  

Design and Delivery

The pilot nature of the program, with only one sentinel site in operation, is widely seen as the biggest impediment to program success.  Without a network of sentinel sites as envisioned in the original program design, data are not as statistically powerful, able to be used for comparisons across regions, or able to generate national estimates.

The distance and time required to establish and maintain additional sentinel sites across Canada will be challenge, particularly given the limited size of the C-EnterNet team and its heavy involvement in operational matters in the pilot site.  While the C-EnterNet team was widely praised by stakeholders and partners for its composition and skills, expansion to additional sentinel sites will not be possible without additional resources or a re-conceptualization of C-EnterNet team roles and responsibilities.  Similar to CDC FoodNet, an increased role in data coordination, cleaning, and day-to-day operations at the local/provincial level may be appropriate.  A more decentralized operational structure would allow the core C-EnterNet team to undertake more analysis, communication and strategic planning.

Recommendation #2: A review of the structure, roles and responsibilities of the C-EnterNet team should be undertaken, particularly in light of future program expansion.  In the absence of additional funding, the program should examine ways to reduce the scope of activities undertaken by the C-EnterNet team through more extensive operational involvement by partners.

Currently, senior management responsibility for C-EnterNet is located within two centres of PHAC.  Day-to-day management has been undertaken at the level of the program lead, and management processes have been largely ad hoc and informal.  While the joint management structure and the lack of formal management processes have not been an impediment to success in program implementation, they do present a risk to continued success, particularly as the program expands.  Effective management processes and a more clear management structure will be critical in moving forward.

Recommendation #3: Senior management responsibility for C-EnterNet should be placed solely within one centre at PHAC. 

The program has established an Advisory Committee, made up of scientists and stakeholders from different government organizations and with different areas of expertise.  While effective as an advisory body, the Advisory Committee has not taken a strong role in strategic planning.

Recommendation #4: C-EnterNet should undertake a formal, annual program planning exercise.  This should involve input from Advisory Committee members to help identify future directions and opportunities for program marketing and collaboration.

Program Success

The C-EnterNet Program is complex and is based on the input and assistance from a complex network of partners in multiple capacities.  Much of the effort in the implementation of the program has involved developing and structuring this network and in setting up delivery processes.  C-EnterNet has successfully established and developed collaborative networks across multiple disciplines and several levels of government.  These networks extend from the local to the international in scope. 

Based on data from its surveillance activities to-date, the C-EnterNet Program has developed and distributed a series of information products, including annual reports and newsletters.  The program team has also authored or co-authored several peer-reviewed articles on source attribution and other aspects of the program’s research.  Recipients of C-EnterNet information appear to demonstrate a high degree of satisfaction with the data and information produced by the program.  Timeliness of the information appears to be somewhat of an issue, however.  Program members have also organized and attended numerous presentations and other venues for knowledge dissemination. 

Recommendation #5:  The C-EnterNet team should develop a strategy for improving the timeliness of its information products.

Most recipients of C-EnterNet information recipients appear to be making use of the information provided.  Stakeholders and recipients of information products felt the major gap in the data was that it was from only one sentinel site, and therefore not nationally representative or able to be used for regional comparisons.

Recommendation #6: Additional sentinel sites should be pursued, as envisioned in the original program design.  However, establishing additional sites should be contingent on identifying additional funding partners or arrangements, as expansion should not risk having a negative impact on the pilot site.

C-EnterNet information appears to be most often feeding into the work of researchers and those involved in services/program delivery, and less often senior management or policy analysts.  As additional years of data become available and trends emerge from the data analysis, it is expected that the implications of the data in shaping policy and practices will become more apparent.  To date, C-EnterNet information products have not addressed potential implications for policy/practice of the data presented.  This has been exacerbated by the heavy workload of the core team during implementation of the pilot.

Furthermore, based on the survey results, a proportion of C-EnterNet information recipients lack awareness of the program, is not reviewing information products they are sent, and is not making use of the data.  Additional marketing of the program would raise awareness of its applicability to stakeholders and their work.

Recommendation #7:  C-EnterNet should develop a marketing and knowledge dissemination strategy.  This strategy should work to identify: 1) key messages derived from program findings that could contribute to informing programs and policies; 2) the appropriate audiences or stakeholders with whom these messages should be shared to have maximum impact; and 3) venues/products most effective for reaching these audiences/stakeholders.  Again, the Advisory Committee should be solicited for input in this activity.  The program should examine the possibility for collaboration with CIPARS in this area.  A clear conceptual framework document for the program should also be developed, which will help with marketing and knowledge dissemination.

With respect to the source attribution component of the program, C-EnterNet has brought together scientists to advance source attribution through a working group, and has contributed to a review of source attribution methodologies developed outside of Canada.  The program has begun to adapt approaches used elsewhere to Canadian data.  The lack of nationally representative data from C-EnterNet has meant that other data sources have been used for estimates undertaken to-date.  Preliminary source attribution estimates have been undertaken, but, relative to the integrated surveillance component, progress appears to be somewhat limited in this area.

Overall, C-EnterNet appears to have had a positive impact on the level of knowledge of enteric illness and exposures among information recipients and staff at the ROWPH.  To date, information has been largely used to support other research activities, but other uses have also been made at the level of informing programs and practices.  The research-oriented function reflect the fact that the results represent only one sentinel site, but also demonstrates the largely scientific impact that can be expected of the program.  However, in order for its findings to have a significant impact on policy and practice, the implications of the data must be made clear and widely communicated to decision-makers outside the scientific community.

The C-EnterNet Program has increased public health capacity through its partnership with ROWPH.  This has included through developing a new case follow-up questionnaire and factsheets on enteric illness and through improved training and workshops with public health inspectors and other staff in the health department.  There is minimal evidence that public health capacity has been strengthened significantly outside of the sentinel site at this time.

Evidence from across the evaluation points to the potential for the program to help reduce the burden of human enteric illness in the future.  Additional time and a network of sentinel sites are, however, critical to achieving this long-term goal.

Cost-effectiveness and Alternatives

The program design and delivery approach appears to have been developed to ensure cost-efficiency within the methodology selected for the program.  As discussed, the feasibility of other cost-sharing arrangements or cost-efficiencies from partnerships should continue to be pursued.  To date, the program has benefitted from substantial in-kind support, particularly in relation to its water component and appears to provide value relative to its costs.  However, the full value of the program will not be fully realized until the program becomes pan-Canadian in scope.