ARCHIVED - Introduction and Context

1.1 Program Description

This formative evaluation focuses on the Surveillance Functional Component of the Public Health Agency of Canada’s Integrated Strategy on Healthy Living and Chronic Disease (ISHLCD). The Surveillance Functional Component is one of six Functional Components in the ISHLCD.

Chronic diseases, including cancer, cardiovascular disease, diabetes, arthritis, asthma and mental illness, are a major challenge in Canada, affecting the quality of life of Canadians and costing society billions of dollars in lost productivity and health care costs (Health Canada, 2002; Levitt, 2008, p. 1-2; Minister of Health, 2007-2008).

Overview of the Integrated Strategy on Healthy Living and Chronic Disease (ISHLCD)

In the 2005 federal budget, $300 million over five years and $74.4 million per year in ongoing funding was provided to the Public Health Agency of Canada (PHAC) for the ISHLCD (Public Health Agency of Canada, 2007c, p. 1). This funding ensured Canada had an integrated approach to addressing major chronic diseases through addressing risk factors, as well as through complementary disease-specific work (Public Health Agency of Canada, October 20, 2005). The ISHLCD design involves various groups within PHAC working together with their public health partners across Canada to set the stage for promoting healthy living and better management of chronic disease.

This strategy consists of three pillars: 1) promoting health; 2) preventing chronic disease by minimizing risk, and 3) early detection and management of chronic disease (Public Health Agency of Canada, 2007c, p. 2), as well as disease-specific strategies on diabetes (non-Aboriginal elements), cancer, and cardiovascular disease (CVD).

Six Functional Components within the ISHLCD were identified to address the underlying factors influencing these three pillars:

• Surveillance;
• Knowledge development, exchange & dissemination (KDED);
• Community-based programming and community capacity building;
• Public information;
• Leadership, coordination and strategic policy development; and
• Monitoring and evaluation (Public Health Agency of Canada, 2007c, p. 5).

These Functional Components also reflect the public health work conducted at the P/T and local/regional levels (Stewart, 2006).

Guiding Principles

The strategic principles of the overall Integrated Strategy are:

• “The Integrated Strategy should span the full spectrum of integrated actions in public health from promotion to prevention to early detection and management and balance between actions that will help promote healthy living to all Canadians and specific actions that will target those at highest risk for chronic disease;
• Initiatives should be evidence-based and efforts should be made to establish evidence where gaps exist;
• Actions and outcomes should be a shared responsibility of all partners;
• The federal government will play a stewardship role in public health, providing the leadership and drawing in players from across many sectors;
• The approach should be comprehensive, making use of all policy and program tools and levers;
• Undertakings should be knowledge-advancing, supported by surveillance, monitoring and research, and evaluated to determine the efficacy of the approach; and
• The Integrated Strategy should be adaptive, changing in response to the achievement of health goals, evaluation, changing risk factors and new knowledge and opportunities” (Public Health Agency of Canada, 2007c, p. 1).

Surveillance Functional Component

The scope of this evaluation is the Surveillance Functional Component of the ISHLCD.  The ISHLCD, including the Surveillance Functional Component, is in the implementation phase, and hence the focus of this evaluation is on examining if the foundation has been laid for its success.

Surveillance involves the “tracking and forecasting of any health event or health determinant through the continuous collection of high-quality data, the integration, analysis and interpretation of those data into surveillance products (such as reports, advisories, warnings) and the dissemination of those surveillance products to those who need to know” (Advisory Committee on Population Health and Health Security: Surveillance Systems for Chronic Disease Risk Factors Task Group, 2005, p. 5; National Advisory Committee on SARS and Public Health, 2003). In other words, surveillance answers the questions: What happened? To what extent? To whom? When? Where?

Surveillance is a key knowledge tool for decision-makers to understand what is going on in the population by providing ongoing, timely information on the health of the population, the nature and scope of health problems, and the factors that need to be addressed in the population to improve health. These factors include health behaviours, risk conditions, use of health services, and environmental and community factors that influence health. Surveillance information on these factors helps policy-makers decide which policies, programs and services are needed to improve health, and to evaluate progress on health goals and targets (Public Health Agency of Canada, 2007a, p. 1).

The Surveillance Functional Component works with partners to enhance the timeliness, quality and use of surveillance information in decision-making about policies, programs and services to promote healthy living, to reduce risk factors, and to detect and manage chronic disease (Public Health Agency of Canada, 2007a, p. 1). This includes continued work on a number of advanced surveillance systems such as diabetes, as well as emerging surveillance work in areas such as arthritis, chronic respiratory diseases, CVD, mental illness and surveillance with Aboriginal Peoples. The budget for the Surveillance Functional Component is $36.5 million over five years with $10.5 million ongoing.

In terms of the role of PHAC’s regions in the Surveillance Functional Component, at the time of this evaluation, two regional surveillance officers had been hired, with plans for the hiring of additional surveillance staff at the regional level to support the work of the ISHLCD. Following a consultation with PHAC regional representatives, a four-fold vision (Tyler, 2007) for a regional role in surveillance was identified as:

• Providing a regional perspective on national-led surveillance projects;
• Contributing to the development of surveillance indicators, standards, and new data sources;
• Increasing local knowledge, skills and capacity to analyze, interpret and use surveillance data; and
• Providing ongoing communication to facilitate collaborative surveillance activities, fostering appropriate dissemination of data, tracking the utilization of data, and identifying target audiences and needs.

Surveillance Program Areas

The Surveillance Functional Component is led by the Surveillance Division of the Centre for Chronic Disease Prevention and Control (CCDPC) and includes the following program areas:

• Public Health Network Task Group on Chronic Disease and Injury: This Task Group, co-chaired by the Director of the Surveillance Division, provides an F/P/T forum to coordinate, plan and evaluate surveillance activity across the phases of surveillance, which includes: all aspects of indicator development and use; data collection and collation; analysis, interpretation and product development; dissemination; and support for public health action. The Task Group is an essential link to the public health governance structure in Canada.
• Expansion of the National Diabetes and Chronic Disease Surveillance System (NDCSS): As the embodiment of an integrated approach to surveillance, the expansion of the NDCSS (2008-2012) will provide a consistent infrastructure supporting hypertension, respiratory disease, mental illness, cardiovascular diseases, arthritis, renal disease, neurological and bowel disease. The governance of NDCSS will provide linkages between specific disease content area advice, strategic direction and scientific and technical advice in the formation of enhanced national chronic disease surveillance capacity. The NDCSS will feature annual data collection from P/Ts, annual publication of methodology on the Internet, and comprehensive reports for each disease area within a 5 year timeframe.
• Adult Cancer:  Adult cancer surveillance analyzes data from the Canadian Cancer Registry (managed by Statistics Canada) and population surveys to develop surveillance information products on cancer risk factors incidence, prevalence, survival, and mortality in reports such as the annual Canadian Cancer Statistics.  The program also provides training support for P/T cancer registries in the areas of cancer epidemiological analysis and staging of tumours.  Cancer statistics are also made available via the PHAC Website using the Cancer Surveillance On-Line Web tool.  The work of the program is coordinated with others working in this area including Statistics Canada, The Canadian Council of Cancer Registries, and the Canadian Partnership Against Cancer (CPAC) through the Surveillance Action Group of CPAC.
• Childhood Cancer: The purpose of the Child and Youth Cancer Surveillance Program is to track children and youth with cancer from initial diagnosis into adulthood.  The data are used to develop an evidence-based approach to risk management policy and control strategies. Primary monitoring data will be collected via an on-line Web-based collection tool (currently under development) from 17 childhood cancer centres across Canada. The program is conducted in collaboration with C17 (the Directors of Pediatric Haematology and Oncology of the 17 Childhood Cancer Centres in Canada).
• Aboriginal: The program works with First Nations, Inuit and Métis organizations and provinces and territories to track chronic disease and its risk factors using existing data sources, principally the Canadian Cancer Registry and P/T health administrative databases. Information from current databases is included in the Division's reports.
• Development of Dissemination Tools: The Chronic Disease Surveillance Division of PHAC is currently pilot testing software that enables the dissemination of online analytical health indicator data for analysis and research by target audience.  This technology will allow Internet users to manipulate and display subset summary health indicator data using their Web browsers (without the need of special software add-ons). Data-cubes are dynamic tables and charts that allow target audiences, such as policy-makers and health professionals, to access and manipulate various aggregate health indicator data (such as risk factors and disease status) by other demographic dimensions (such as geography, age group and gender) using their Internet browsers.  The user decides what categories in the table they want to view, and the software dynamically populates the table for them.
• Sentinel Surveillance: Sentinel surveillance explores the use of surveillance information from primary care to provide information on chronic disease indicators. Currently, PHAC provides epidemiological support on methods for sentinel surveillance in conjunction with key collaborators.
• The Canadian Alliance for Regional Risk Factor Surveillance (CARRFS): This network was formed in September 2008 based on a task group recommendation in 2005. CARRFS is a national network of public health stakeholders interested in strengthening regional/local area chronic disease risk factor surveillance in Canada. Representation on CARRFS comes from local, P/T, and federal public health stakeholders across Canada.
• Survey Enhancement and Development: This program focuses on enhancing existing national surveys to: fill knowledge gaps on the impact of chronic disease on individuals; understand how people with chronic disease manage their health condition; and identify emerging public health issues. PHAC works in collaboration with Statistics Canada and other experts to develop questionnaires focusing on specific chronic diseases. The current focus is to develop disease-specific questionnaires for inclusion in the Canadian Community Health Survey (CCHS) rapid response, as well as developing the Survey on Living with Chronic Disease in Canada (SLCDC).

Program Objectives

For the Surveillance Functional Component, the three action areas are:

• Enhance the analysis, interpretation and use of existing data sources for surveillance;
• Expand data sources for surveillance; and
• Coordinate, plan and evaluate surveillance activities (Public Health Agency of Canada, 2007a, p. 2).

These action areas will lead to improved surveillance systems with:

• More information on the tracking of chronic disease in Canada;
• Enhanced quality and comparability of data;
• Increased access to data and metadata; and
• Enhanced knowledge, skills and resources to do surveillance.

Figure 1 illustrates the Logic Model for the Surveillance Functional Component. The three action areas and outcomes identified above are depicted in this Logic Model.

Intended Beneficiaries

The key stakeholders and beneficiaries for the Surveillance Functional Component include:

• PHAC and the other federal government departments;
• P/T governments;
• Local/regional public health and planning bodies;
• Non-government organizations (NGOs), including professional organizations, disease or risk factor organizations, and Aboriginal organizations;
• Universities and colleges (researchers, educators, students);
• Database managers e.g., Statistics Canada and the Canadian Institute for Health Information (CIHI); and
• Canadian Institutes of Health Research (CIHR) and other funding bodies (Public Health Agency of Canada, 2007a, p. 3).

In addition, specific stakeholders and beneficiaries eligible for the Grants and Contributions (Gs&Cs) are NGOs, universities, P/T governments, and local/regional public health organizations and networks (Public Health Agency of Canada, 2007a, p. 7).

Figure 1: ISHLCD – Surveillance Functional Component Logic Model

Figure 1: ISHLCD – Surveillance Functional Component Logic Model

1.2 Evaluation Context

Chronic Disease Surveillance Context

For the Surveillance Functional Component, external events have played an important role in implementation.

The biggest overall impact experienced by the Strategy was the change in government in November 2005, compressing timelines for funding and the completion of work.

The external policy issue that has had the largest impact on surveillance was the creation of the Canadian Partnership Against Cancer (CPAC). The purpose of the new non-profit corporation is to implement the Canadian Strategy for Cancer Control. Budget 2006 committed new investments in CPAC of $52 million per year outside of the ISHLCD (Public Health Agency of Canada, 2007a, p. 2; Public Health Agency of Canada, 2007c, p. 11). The creation of this new entity has raised issues of roles and responsibilities between PHAC and CPAC.

Other initiatives that have influenced the Surveillance Functional Component included the Diabetes Policy Review, the creation of the Mental Health Commission and the National Lung Health Framework, and new surveillance requirements in diseases such as arthritis, neurological diseases and autism.

Corporate Support for Evaluation of Surveillance within PHAC

Priorities in the PHAC Surveillance Strategic Plan 2007-2012 (PSSP) include support for the performance measurement, quality assurance, and evaluation requirements of surveillance.

Overall Purpose of the Evaluation

The evaluation of the Surveillance Functional Component was conducted as part of the commitment in the ISHLCD RMAF to conduct a formative evaluation of this Component (Public Health Agency of Canada, 2007c, p. 18).

The overall intent of this formative evaluation is to guide and inform stakeholders regarding “how far and how fast” program implementation of the Surveillance Functional Component has proceeded, and what early outcomes have been generated for the action areas.

 

Objectives and Scope of the Evaluation

This evaluation focuses on design, implementation and delivery, as well as progress towards early outcomes, between 2007 and early 2009. This evaluation aims to provide feedback on program performance and capture performance stories in relation to integration and engagement.  The work portrays the current state of program performance, and provides information for supporting program improvement decisions.

In particular, this evaluation examines the Surveillance Functional Component’s progress in: assessing need for the program, aligning with government priorities and direction, implementing activities, and enhancing engagement. The emphasis of the implementation of the ISHLCD since its inception in 2007 has been on the ISHLCD Operational Matrix and enhancing external engagement (Public Health Agency of Canada, 2008a). Therefore, this evaluation has examined the concepts of:

• Integration: The ISHLCD uses a matrix system (the ISHLCD Operational Matrix) to implement the Strategy in an integrated way. Coordination and accountability run in two directions: horizontally for the 14 Program Components and vertically for the 6 Functional Components (please see Annex A). Integration across these Components, including coordination, is key to the overall Strategy and to the Surveillance Functional Component. Integration occurs at several levels, including coordination across the six Functional Components, across risk factors, across levels of engagement (e.g., with P/Ts), within and outside of the health system, and in the determinants of health (Public Health Agency of Canada, 2007c, p. 7-8).
• Engagement: Working with a diversity of partners, the ISHLCD works to engage the right internal and external stakeholders at the right time with the right intensity. Improving engagement, including between and within organizations, within and outside the health sector, and across jurisdictions, is a critical focus of the ISHLCD overall and the Surveillance Functional Component specifically.  Hence, engagement has been identified as an immediate outcome.

 

As a result, the ISHLCD Operational Matrix and Surveillance Logic Model were explicitly referenced during interviews with key informants as prompts to examine these two concepts with respondents.

The evaluation findings from this Surveillance Functional Component will also be one of the inputs synthesized to create an overarching evaluation for the ISHLCD (Public Health Agency of Canada, 2007c, p. 17).

Evaluation Questions

The following questions relating to program design, delivery, success and relevance were examined in this evaluation:

• Does the Surveillance Functional Component continue to be consistent with departmental and government-wide priorities, and does it realistically address an actual need?
• Are the most appropriate means being used to achieve objectives?
• Is the Surveillance Functional Component effective in generating outputs, given its resources, and without unintended consequences?