ARCHIVED - Conclusions
The overall intent of this formative evaluation is to guide and inform stakeholders regarding “how far and how fast” program implementation of the Surveillance Functional Component has proceeded, and what outputs and early outcomes have been generated. The evaluation questions have addressed:
- relevance of the program when compared with need and government priorities;
- appropriateness of the program as a means toward achieving its objectives; and
- the program’s effectiveness in generating outputs, without unintended consequences, given the resources provided.
The Surveillance Functional Component has demonstrated a high level of consistency with federal requirements and joint F/P/T priorities, directives and plans for enhancing chronic disease surveillance capacity. The findings from this evaluation illustrate PHAC has shown sensitivity and respect for the roles of other public health organizations in an evolving national policy context, where changes in government and the creation of a disease-specific agency outside of government presented challenges in implementing the original vision of the ISHLCD.
The Surveillance Division and their partners have faced the challenge of increases in simultaneous expectations for greater breadth and depth of surveillance information. The establishment of disease-specific initiatives within the broader context of healthy living and chronic disease prevention necessitates continued leadership by PHAC to monitor a growing number of partnerships and surveillance information requirements. The monitoring and evaluation of the functions and relationships of the NDCSS governance structure will be necessary to ensure ongoing alignment with priorities as the scope and capacity of the NDCSS develops.
Design and Delivery
While the implementation of the Surveillance Functional Component has been delayed compared to the original schedule established with Treasury Board, this was largely due to internal and external constraints beyond the immediate control of program management. The fact that the Surveillance Functional Component may be able to accelerate program implementation over the next several fiscal years to be on schedule will represent a significant program accomplishment.
The systematic approach of documenting and tracking used to support the management of the Gs&Cs process may provide a model for determining and coordinating the management of the other outputs of the Surveillance Functional Component. Outside of the Gs&Cs program, most outputs are not being measured systematically (e.g., only half of those who respond to requests for information are contributing to the monitoring of these requests). The continued development and implementation of the surveillance indicator framework as a surveillance planning and reporting tool should be considered as a significant resource in the planning of a performance measurement system.
Progress towards accomplishing the outputs described in the Surveillance Logic Model has been reported via a number of mechanisms. In particular, a number of comprehensive chronic disease-specific epidemiologic reports have been generated as part of the Strategy. Similarly, activities are underway to improve analytic capacities for existing chronic disease data, although delays in implementation have had an impact on the realization of some outputs.
Among the lessons learned during the implementation of the Surveillance Functional Component is that enhanced communication with stakeholders to help manage their expectations for surveillance data, and to identify and communicate the many steps required to coordinate and deliver enhanced surveillance capacity, would clearly benefit the Strategy.
One challenge in measuring progress of the Surveillance Functional Component has been the issue of conceptually defining and measuring engagement. Given the need to develop and pilot a qualitative measure of engagement to support the assessment of the sole immediate outcome, this evaluation’s assessment of engagement is preliminary and intended to be supportive of future attempts to assess this concept. The results of this evaluation indicate engagement with stakeholders by the Surveillance Functional Component has been successful in increasing trust and the sharing of information.
While outside the scope of this evaluation, the fact that the operationalization of all ISHLCD Functional Components is based on the application of matrix management concepts (i.e. the ISHLCD Operational Matrix) necessitates that any future evaluation of ISHLCD should identify well known critical successes, limiting factors and any standard measures of matrix management functions. Respondents reported challenges to implementation of this matrix approach, identifying that the work has been “siloed”, “stove-piped”, or there was a poor connection between the ISHLCD matrix approach and the organizational hierarchy.
Among the intermediate outcomes described in the Logic Model, respondents identified progress in the areas of data quality and providing chronic disease surveillance information, as well as trends in chronic diseases, risk factors and disease outcomes. Internal respondents reported more positive perceptions than external respondents. This finding further identifies the need to communicate advances in chronic disease surveillance capacity with a broader stakeholder audience.
Finally, assessing longer-term outcomes, such as the impact of surveillance on decision-making, can be regarded as premature given the short period of time the Surveillance Functional Component has had to implement its work. Future work would also need to be informed by best practices identified by the KDED Component of the ISHLCD.
Report a problem or mistake on this page
- Date modified: