ARCHIVED - Evaluation of the C-EnterNet Program Final Report
Annex A: Evaluation Matrix
Evaluation Issues / Questions | Indicators | Methodologies | |||||||||
---|---|---|---|---|---|---|---|---|---|---|---|
Interviews | Document Review | User Survey | Admin Data | Costs Survey | PHU Focus Group | ||||||
C-Net team | PHAC senior mgm’t | ROWPH Steering Committee | Sample of AC members | Experts | |||||||
1.0 RELEVANCE | |||||||||||
1.1 Is there a continued need for C-EnterNet (C-Net)? | 1.1.1 Evidence that infectious enteric disease is an important issue in Canada (costs to Canadian society of enteric disease, number of cases, number of deaths, increasing risk environment due to factors such as globalization) | ■ | ■ | ■ | ■ | ■ | ■ | ||||
1.1.2 Trends in incidences of food-borne and waterborne illness in Canada | ■ | ||||||||||
1.1.3 Evidence that C-Net fills important gaps in addressing enteric disease | ■ | ■ | ■ | ■ | ■ | ■ | ■ | ||||
1.1.4 Views that C-Net components are relevant – i.e., human and pathogen (farm, retail, water) | ■ | ■ | ■ | ■ | ■ | ■ | |||||
1.1.5 Views on whether similar information is needed from other regions | ■ | ■ | ■ | ■ | ■ | ||||||
1.2 Does C-EnterNet align with Government of Canada, PHAC objectives and priorities, and local public health priorities? | 1.2.1 Degree of alignment of C-Net with Government of Canada objectives, priorities | ■ | ■ | ■ | |||||||
1.2.2 Degree of alignment of C-Net with PHAC objectives, priorities, mission and vision | ■ | ■ | ■ | ||||||||
1.2.3. Degree of alignment of C-Net with local health priorities | ■ | ■ | ■ | ||||||||
1.3 Is the current role of the federal government appropriate? | 1.3.1 Evidence of the appropriateness / fit of the current role | ■ | |||||||||
1.3.2 Stakeholder opinion on the transfer of C-Net or some of its components to another level of government or the private sector | ■ | ■ | |||||||||
1.4 Does C-EnterNet complement, duplicate or overlap the work of other programs or organizations? | 1.4.1 Existence of other programs and organizations providing similar or the same information, services | ■ | ■ | ■ | ■ | ■ | ■ | ■ | |||
1.4.2 Level of coordination with other similar programs, organizations | ■ | ■ | ■ | ■ | |||||||
2.0 DESIGN AND DELIVERY | |||||||||||
2.1 What factors in the C-EnterNet design and delivery facilitate or impede the success of the program? | 2.1.1 Stakeholder opinions on design and delivery issues impacting success of program | ■ | ■ | ■ | ■ | ||||||
2.2 What does the implementation of C-EnterNet in the pilot site suggest in terms of the potential for having a network of sentinel sites across Canada? | 2.2.1 Opinions on the feasibility of adapting the pilot’s design and delivery structure, processes, and policies to include a network of sentinel sites across Canada | ■ | ■ | ■ | ■ | ■ | ■ | ||||
2.2.2 Views on barriers to implementation of sites in other locations | ■ | ■ | ■ | ■ | ■ | ■ | |||||
2.3 To what extent is the C-EnterNet program well-managed? | 2.3.1 Management structure for C-EnterNet program is effective | ■ | ■ | ||||||||
2.3.2 Annual planning, performance measurement and reporting structure is in place | ■ | ■ | ■ | ||||||||
2.3.3 Advisory Committee has been established and functions well | ■ | ■ | ■ | ||||||||
2.3.4 Regional Steering Committee has been established and functions well | ■ | ■ | ■ | ||||||||
2.3.5 Opinions on successes and challenges in managing the C-EnterNet program | ■ | ■ | |||||||||
3.0 PROGRAM SUCCESS | |||||||||||
OUTPUTS | |||||||||||
3.1 Did the C-EnterNet Pilot generate the intended information (outputs)? | 3.1.1 # and types of outputs generated by C-Net (e.g., workshops, presentations, reports) | ■ | ■ | ■ | |||||||
IMMEDIATE OUTCOMES | |||||||||||
3.2 Has up-to-date and relevant information on enteric diseases and exposures been received by the right stakeholders in a timely manner? | 3.2.1 Evidence that C-Net-produced information is received by appropriate stakeholders | ■ | ■ | ||||||||
3.2.2 Views on the relevance, timeliness, usefulness, quality of information received from C-Net | ■ | ■ | ■ | ||||||||
3.2.3 % of recipients who made use of information, and types of uses for information | ■ | ||||||||||
3.2.4 # of data requests and information inquiries made to C-Net, including by topic area and by stakeholder group | ■ | ||||||||||
3.2.5 Evidence that C-Net effectively responded to requests for data and information in a timely manner | ■ | ||||||||||
3.2.6 Gaps in the information being delivered to stakeholders | ■ | ■ | ■ | ||||||||
3.3 Has C-EnterNet resulted in the establishment and development of collaborative networks related to enteric disease? | 3.3.1 # of new partnerships, networks developed as a result of C-Net | ■ | |||||||||
3.3.2 # of meetings, consultations, workshops, and other venues for collaboration and networking organized by C-Net | ■ | ||||||||||
3.3.3 Views that C-Net resulted in collaborative networks, including multidisciplinary networks | ■ | ■ | ■ | ■ | |||||||
3.4 What contribution has C-EnterNet made to the advancement of source attribution methodologies for Canada? | 3.4.1 Source attribution methodologies are tailored to the Canadian context | ■ | |||||||||
3.4.2 Views on C-Net’s contribution to source attribution methodologies | ■ | ■ | ■ | ||||||||
INTERMEDIATE OUTCOMES | |||||||||||
3.5 Has C-EnterNet resulted in increased knowledge of enteric diseases and exposures among stakeholders? | 3.5.1 % of information recipients that state that he information resulted in increased knowledge | ■ | |||||||||
3.5.2 Views on extent to which C-Net has increased knowledge among stakeholders | ■ | ■ | ■ | ■ | |||||||
3.5.3 # of peer-reviewed publications | ■ | ||||||||||
3.5.4 # of citations of C-Net documents in publications | ■ | ■ | |||||||||
3.6 To what extent has C-EnterNet strengthened public health capacity? | 3.6.1 Improved accuracy in notifiable disease data at the local health department | ■ | ■ | ||||||||
3.6.2 Use of standardized tools at the local health department | ■ | ■ | |||||||||
3.6.3 Other examples of ways in which public health capacity has been strengthened, including at laboratories | ■ | ■ | ■ | ■ | |||||||
3.7 Did the information produced by C-EnterNet inform, create or help to evaluate programs, policies, practices (e.g., risk assessments) or health messages related to infectious diseases and exposures? | 3.7.1 # of programs, policies, practices or health messages informed, created or evaluated by the information produced by C-Net | ■ | ■ | ■ | ■ | ||||||
3.7.2 Views on the extent to which the information produced by C-Net has informed, created or evaluated programs, policies, practices or health messages | ■ | ■ | ■ | ■ | ■ | ■ | ■ | ||||
3.8 Has C-EnterNet established any Canadian estimates on source attribution? | 3.8.1 # of source attribution estimates established | ■ | ■ | ||||||||
ULTIMATE OUTCOMES | |||||||||||
3.9 Is C-EnterNet making progress in achieving its ultimate goal of contributing to reducing the burden of human enteric illness in Canada? | 3.9.1 Evidence that C-Net is contributing to reducing the burden of enteric illness in Canada (results from evaluation) | ■ | ■ | ■ | ■ | ■ | |||||
3.10 Has the program resulted in any unintended impacts? | 3.10.1 Evidence of any identified unintended impacts | ■ | ■ | ■ | ■ | ■ | |||||
4.0 COST-EFFECTIVENESS AND ALTERNATIVES | |||||||||||
4.1 Are there alternative design/delivery approaches/models or modifications that might be more cost-effective and still achieve similar or better outcomes? | 4.1.1 Evidence and views of alternative design approaches / models (within or outside of Canada) (e.g., could the same data be collected more cost-effectively in other ways or by other groups?) | ■ | ■ | ■ | ■ | ■ | |||||
4.1.2 Evidence and views of delivery modifications to C-Net that would improve cost-effectiveness | ■ | ■ | ■ | ■ | ■ | ■ | |||||
4.2 Is C-EnterNet providing value for its cost? | 4.2.1 Ratio of administrative versus operating costs (compared to benchmarks) | ■ | |||||||||
4.2.2 $ value of funds leveraged from partnerships (including in-kind support) | ■ | ||||||||||
4.2.3 Examples of cost-savings realized at other agencies as a result of C-Net activities | ■ | ||||||||||
4.2.4 Evidence that C-Net is providing value (results from across the evaluation) | ■ | ■ | ■ | ■ | ■ | ■ | ■ | ■ | ■ |
Page details
- Date modified: