Section 5: Evaluation of the surveillance function at the Public Health Agency of Canada – Evaluation conclusions and recommendations

5. Evaluation conclusions and recommendations

The analysis of the information gathered for this evaluation resulted in findings and conclusions about the relevance and performance (summarized in Table 1) which led to three key recommendations.

5.1 Conclusions

5.1.1 Relevance

There is a clear, continued need for public health surveillance due to emerging and re-emerging infections, chronic diseases, and other population health risk factors that continue to pose risks to the health of Canadians, with associated costs to Canadian society. Public health surveillance is needed to identify risk factors and determinants of health, to monitor these risks to the health of Canadians, and to inform policy and program decisions for upstream interventions and other prevention and control measures. By extension, an effective and responsive public health surveillance function requires that the surveillance infrastructure, which consists of the structures and processes required to enable the core public health function of surveillance, continues to be supported.

The Public Health Agency’s surveillance function aligns with the Government of Canada’s objectives and priorities by working towards the objective of healthy Canadians. Surveillance activities also enable the Public Health Agency to attain its strategic outcome: “Canada is able to promote health, reduce health inequalities, prevent and mitigate disease and injury”,^{Footnote 104} and support national priorities related to communicable and non-communicable diseases. The public health surveillance function also aligns with objective 2 of The Public Health Agency of Canada - Strategic Plan: 2007 – 2012, Information, Knowledge, Action^{Footnote 105}: “Ensure actions are supported by integrated information and knowledge functions”.

The Government of Canada and the Public Health Agency of Canada have a mandate and role in public health surveillance. Various acts, including the Department of Health Act^{Footnote 106} and the Public Health Agency of Canada Act^{Footnote 107}, confer authority on the federal government to carry out public health surveillance. Under the International Health Regulations^{Footnote 108}, Canada also has an obligation to carry out surveillance. Although a mandate exists, there is no commonly held understanding of the mandate and its associated vision, roles and responsibilities and priorities for public health surveillance, particularly among provincial and territorial partners.

The federal government has assumed a role in:

• monitoring inter-jurisdictional outbreaks
• providing the national evidence on disease and epidemiologic trends
• liaising with international counterparts
• supporting federal partners to fulfill their surveillance mandates
• building provincial and territorial surveillance and epidemiological capacity
• coordinating national surveillance activities.

These are appropriate roles for the Government of Canada and the Public Health Agency that align with the needs and expectations of the provinces, territories and other stakeholders.

While the public health surveillance function as a whole was not seen to duplicate the work of other federal organizations, the respective roles and responsibilities of the players are not clear. Program complementarity would be improved by closer collaboration among federal partners such as Statistics Canada, Health Canada and the Canadian Food Inspection Agency, as well as with organizations such as the Canadian Institute for Health Information.

5.1.2 Performance

Both the Pan-Canadian Public Health Network and the Public Health Agency have priority-setting processes. However, the Public Health Agency’s formal surveillance priority-setting process is not widely visible because priority setting occurs at the program or centre level.”

Performance of the surveillance function was assessed in terms of the Public Health Agency’s progress in aligning its activities to achieve the goals of the 2007 Surveillance Strategic Plan^{Footnote 109}.

Overall, the Public Health Agency has made progress in implementing the initiatives associated with the goals of the 2007 Surveillance Strategic Plan. However, gaps remain in key areas of the public health surveillance infrastructure, including:

• information-sharing agreements
• coordinated and standardized data collection processes, procedures, tools and methods to improve the collection of comparable information
• robust national and interoperable systems
• the provision of timely, accurate and complete information to public health decision makers and Canadians.

Given the decentralized nature of surveillance within the Public Health Agency, analysis was complicated and the evaluation team was unable to assess the overall efficiency and economy of the surveillance function. However, key informants anticipate that greater efficiency and economy could be achieved through:

• standardization, coordination and integration
• data management
• setting surveillance and publication priorities.

5.2 Recommendations

The Public Health Agency should take a more strategic approach to public health surveillance including:

• assuming a lead role in developing a shared federal/provincial/territorial vision for the public health surveillance function in Canada
• establishing a formal mechanism and criteria to identify relative priorities for surveillance investments
• developing a more strategic approach to the approval and dissemination of surveillance products and information.

Policy framework for a federated model for public health surveillance in Canada

During the period covered by the evaluation, the Public Health Agency’s surveillance activities were guided by the 2007 Surveillance Strategic Plan^{Footnote 110}. However, the Public Health Agency and its provincial and territorial partners would benefit from an overarching policy framework for a ‘federated’ model for public health surveillance in Canada that sets out a shared vision and the essential elements of a national surveillance function. This framework should cover both infectious and chronic diseases and include, but not be limited to, a description of:

• federal, provincial, territorial and local mandates
• roles and responsibilities for surveillance
• goals, objectives and operating principles
• surveillance priorities.

It is appropriate that the Public Health Agency take a lead role in developing this framework. However, the cooperation of the provinces and territories is essential to establishing an effective public health surveillance system. Therefore, to achieve a common understanding of a national system for public health surveillance in Canada, this framework should be developed collaboratively and disseminated widely.

Public Health Agency priority setting

Within the Public Health Agency, funding for surveillance is program- or centre-based and priority setting occurs at this level. The Public Health Agency could be more strategic about its surveillance investments by using a formal mechanism, including predefined criteria, to establish relative Public Health Agency-level surveillance priorities. These priorities should align with and support the achievement of the objectives of the Public Health Agency’s Strategic Plan^{Footnote 111}, as well as provincial and territorial and program priorities and information needs. Such a process would assist Public Health Agency staff in making decisions about resource allocation, and would ensure that surveillance investments are directed to areas of priority.

Knowledge management

The Public Health Agency’s role is to provide timely national analysis and surveillance products to public health decision makers and Canadians in a format that is useful to them. The Public Health Agency produces a wide range of knowledge products that are distributed in various formats and at various frequencies. However, several factors were found to affect the Public Health Agency’s ability to produce and disseminate surveillance products in a timely enough manner to assist in the detection of outbreaks or to inform policy and program decision making. These factors include timeliness and accuracy of information provided by the provinces and territories and other data providers, internal content approval processes and communications processes. To ensure that surveillance investments produce useful surveillance information that meets the needs of internal and external audiences, Public Health Agency programs should routinely assess user needs and satisfaction with their surveillance outputs and systematically collect data on the use of their surveillance products and information in policy and program interventions.

In addition, the Public Health Agency should:

• prioritize its surveillance publications and website surveillance content
• determine appropriate frequencies for dissemination based on user needs
• examine alternate approaches for timely approval, production and dissemination of surveillance products.

RECOMMENDATION 1

Take a strategic approach to surveillance that includes, but is not limited to:

1. working with partners and stakeholders to develop a policy framework for a ‘federated’ model for public health surveillance, and an approach for its implementation
2. establishing a formal mechanism and criteria for surveillance priority setting within the Public Health Agency that links surveillance priorities to public health, provincial, territorial and Public Health Agency strategic priorities
3. developing a surveillance knowledge management approach that identifies priority and appropriate surveillance products; examines approaches for their timely approval, production and dissemination; and reflects the needs of multiple audiences.

Integrated surveillance

The term ‘integrated surveillance’ must be defined and communicated. Despite a lack of common understanding, Public Health Agency surveillance managers and staff support integration, if it represents:

• greater coordination
• enhanced internal communications across the divisions and centres
• standardization and common approaches to surveillance within the Public Health Agency, including integration and pooling of data, priority setting criteria and processes
• data analysis and reporting practices.

All levels of government recognize that greater standardization and harmonization will lead to the production of comparable data across the country. Without a harmonized approach to surveillance, systems are not likely to be interoperable. Areas of opportunity include, but are not limited to, working with surveillance partners on:

• data content, including standards and definitions
• data management processes, including secure transfer and storage of data and data access
• technological platforms and tools
• electronic health records.

Once developed, the Public Health Agency should continue to work with its partners to identify and resolve implementation issues.

RECOMMENDATION 2

The Public Health Agency should assume a leadership role in working with surveillance partners towards greater consistency of surveillance practices and processes, and where feasible, explore opportunities within the Public Health Agency for integration and linkages among its own surveillance systems and programs.

The Public Health Agency can further strengthen its surveillance function by improving its governance and performance measurement functions.

Governance

The Public Health Agency’s internal surveillance governance structure, in particular the Surveillance Integration Team, is effective as a forum for horizontal information exchange among programs and for issue management. The Surveillance Integration Team, comprising surveillance experts from across the Public Health Agency, works closely with the Surveillance Coordination Unit to develop a collaborative approach to surveillance. However, it could play a more effective role in defining surveillance priorities and promoting consistent implementation of surveillance initiatives across the Public Health Agency. While there is no requirement to significantly change the existing governance structure, consistent with the findings of the mid-term governance review, key informants noted that there are opportunities to strengthen the key components of the governance structure, including:

• strengthening the role of the Chief Health Surveillance Officer (formerly the Senior Surveillance Advisor) to provide authority for Public Health Agency-wide implementation of surveillance transformation and to ensure program accountability to routinely report progress on implementation of surveillance priorities
• providing the Surveillance Integration Team with a stronger advisory role
• ensuring that the Surveillance Coordination Unit is structured and adequately resourced to support the functions of the Chief Health Surveillance Officer, as well as to coordinate major horizontal initiatives.

Performance measurement

The 2008 May Report of the Auditor General of Canada^{Footnote 112} identified that the Public Health Agency needed to measure the performance of its surveillance systems. To address gaps in performance data, the Public Health Agency has developed a performance measurement framework for its surveillance systems. The Public Health Agency should finalize and implement the framework, and develop a plan to monitor economy and efficiency.

Generally, Public Health Agency programs already collect some data to measure the performance of their surveillance systems to guide the ongoing improvement and evaluation of these systems. By implementing the framework for its surveillance systems, programs would report using the core set of indicators which would allow for an assessment of the surveillance function. This assessment would determine if the Public Health Agency’s surveillance function is operating effectively, meeting the needs of stakeholders and providing quality data for public health decision making.

RECOMMENDATION 3

Strengthen the governance and accountability of the Public Health Agency’s surveillance function by:

1. providing the components of the governance structure with appropriate authority, accompanied by clearly defined accountabilities for programs
2. finalizing and implementing a Public Health Agency-wide performance measurement framework for its surveillance systems, and developing the necessary tools to monitor economy and efficiency.