Evaluation of the Canadian Partnership Against Cancer Activities 2012–2013 to 2015–2016
Prepared by
Office of Audit and Evaluation
Health Canada and the Public Health Agency of Canada
January 2016
The Public Health Agency of Canada and Health Canada Office of Audit and Evaluation acknowledges and thanks all individuals who gave their time and input for this evaluation. In particular, the Office of Audit and Evaluation thanks the consulting firm of Bell, Browne, Molnar and Delicate who were hired by the Canadian Partnership Against Cancer to conduct the evaluation.
Table of Contents
- Executive Summary
- Management Response and Action Plan for Health Canada
- Management Response and Action Plan for CPAC
- 1.0 Evaluation Purpose
- 2.0 Description of the Partnership
- 3.0 Evaluation Description
- 4.0 Findings
- 4.1 Relevance: Issue #1 - Continued Need for the Program
- 4.2 Relevance: Issue #2 - Alignment with Government Priorities
- 4.3 Relevance: Issue #3 - Alignment with Federal Roles and Responsibilities
- 4.4 Performance: Issue #4 - Achievement of Expected Outcomes (Effectiveness)
- 4.5 Performance: Issue #5 - Demonstration of Economy and Efficiency
- 5.0 Conclusions
- 6.0 Recommendations
- Appendix 1 - References
- Appendix 2 - Recommendations from Previous Evaluations
- Appendix 3 - Strategic Priorities and Outcomes
- Appendix 4 - Summary of Findings
- Appendix 5 - Evaluation Description
List of Tables
Executive Summary
Evaluation Purpose and Scope
The purpose of this independent performance evaluation was to assess the relevance and performance of the Canadian Partnership Against Cancer (the Partnership) from April 2012 to June 2015. It was conducted to fulfil a requirement of the Partnership's 2012-2017 funding agreement with Health Canada. Moreover, it was an opportunity for the Partnership to take stock, be able to understand more fully its successes and areas where improvement or recalibration may be necessary, communicate these to its stakeholders, and use the findings and conclusions as input to its own decision-making. The evaluation was conducted in accordance with the Treasury Board of Canada's Policy on Evaluation (2009).
Program Description
The Partnership was implemented as a means to advance the 2006 Canadian Strategy for Cancer ControlFootnote 1 and accelerate action on cancer control across Canada. The Partnership's initial funding agreement with Health Canada was for $250 million for the period 2007-2012. Areas of focus and action were guided by the Partnership's 2007-2012 Strategic Plan. The Partnership's second funding agreement with Health Canada was for $241 million for the period 2012-2017, during which it is guided by its 2012-2017 Strategic Plan.
Conclusions - Relevance
Continued Need
Need for Sustained Coordinated Effort
There is a continued need for a sustained (i.e., ongoing) and coordinated effort in transforming and improving the cancer control system across Canada. This need arises because of a number of factors: the ongoing problem of cancer and its impacts on Canadians; the gaps in and opportunities for improvement that still exist in the cancer control system today; the length of time that it takes between making changes to the system and realizing the benefits; and the challenges posed by Canada's federated model for health care.
Need for the Partnership
The Partnership has played and will continue to play an important and unique role in the sustained and coordinated effort at the national level. There will continue to be a need for and stakeholder support for the "honest broker" role that the Partnership has been uniquely positioned to take, going into a third mandate and even beyond.
Need for the Partnership's Model
The Partnership's model of convening, integrating, catalyzing and brokering knowledge will continue to be appropriate for its role in the sustained and coordinated effort. Ideas from stakeholders as to how the Partnership's positioning and model could be revised moving forward will need to be vetted carefully against the Partnership's context and value as perceived by stakeholders in the cancer control system. For example, changes in the model or in its application need to consider potential implications for the Partnership's positioning as an honest broker that keeps true to the model, as a catalyst rather than "owner" of change, and as a nimble, adaptive and flexible organization that moves its focus and portfolio of initiatives to where it can provide maximum value.
Alignment with Government Priorities
The Partnership is aligned with the priorities of the federal government and the priorities and strategic outcomes of Health Canada. It also has good alignment with the priorities of its key partners. This alignment is well understood by key stakeholders engaged in this evaluation, with mechanisms in place to support the sharing of information about priorities.
Alignment with Federal Roles and Responsibilities
The Partnership's mandate is well aligned with federal roles and responsibilities related to health care innovation, collaboration across jurisdictions, and chronic disease prevention and mitigation. This conclusion is supported by key stakeholders engaged in this evaluation, who voiced their expectation that the federal government would have ongoing roles and responsibilities related to the cancer control system and the Canadian Cancer Control Strategy, and that the Partnership, as an arm's length organization, was supportive of federal roles and responsibilities.
The Partnership's mandate has also been aligned with and has supported the roles and responsibilities of its partners and stakeholders. Alignment in this case is not equivalent to "same as". Rather it is better viewed as meaning "complementary" or "compatible". Ongoing alignment is supported by the Partnership's positioning as an honest broker, focusing on supporting stakeholders to improve and change to the cancer control system.
Conclusions - Performance
Achievement of Expected Outcomes (Effectiveness)
Demonstration of Results on Immediate Outcomes
Prevention and Screening
Building upon work begun or with roots in the Partnership's first mandate, significant progress continued regarding improved access to evidence-based prevention strategies and improved quality of and participation in screening. Further, the Partnership has made important contributions to the progress made.
The Coalitions Linking Action and Science for Prevention (CLASP) initiative has shown successful pilot/demonstration projects and more are underway in this mandate. The challenge will be to sustain the current CLASPs and then scale the CLASP approach to engage a much greater percentage of the Canadian population. This raises questions about project selection and knowledge transfer and exchange approaches going forward.
Population-based screening is a major success story for the Partnership and its partners. Through its support for cancer-specific screening frameworks and screening networks, the inclusion of specific performance indicators in its cancer system performance reports, and its Cancer Risk Management Model (CRMM), the Partnership has helped accelerate the implementation of organized population-based screening programs across and within jurisdictions. These are important functions for the Partnership to continue to play in screening programs. Looking ahead, the Partnership may be able to help launch, on a pan-Canadian scale, screening programs for other cancers where scientific evidence is available to support new screening efforts. The Partnership may also be a good vehicle for bringing together stakeholders from across the cancer-specific screening networks to address issues related to screening of underserved populations - e.g., new immigrants, rural and remote, those facing socio-economic challenges - such as it is doing through its upcoming (September 2015) workshop on Screening in Underserved Populations to Expand Reach.
Diagnosis and Clinical Care
Progress has been made on the immediate outcome of more consistent actions to enhanced quality of diagnostic and clinical care. The Partnership's major effort has been the National Staging Initiative in its first mandate and the Synoptic Reporting and Staging Initiative in this second mandate. Staging is now mature. Efforts on synoptic reporting should continue for this mandate but should require much less Partnership involvement, other than performance reporting and benefits analysis, in the next mandate.
The Partnership has also made contributions to quality improvement in diagnosis and clinical care through its work with the Canadian Partnership for Quality Radiotherapy on quality assurance and with Accreditation Canada on accreditation standards.
Patient Needs
Although there is evidence that progress has been made on improving the capacity of the cancer control system to respond to patient needs, there is also evidence that much still remains to be done. There is a general consensus that although early steps towards achieving this outcome have been taken, there has been only limited impact on patients. Although the cancer control system, including the Partnership, has taken steps to improve the capacity of the cancer control system to respond to patient needs, actual change will need to come from stakeholders and partners who work directly with patients, i.e., the Partnership is able to facilitate change but not directly bring it about.
Research Coordination and Capacity
The Canadian Cancer Research Alliance (CCRA) and the Canadian Partnership for Tomorrow Project (CPTP) have resulted in enhanced coordination of cancer research and improved population-health research capacity. The Partnership has contributed to the CCRA through support of the CCRA's executive office and through its membership in the CCRA. The Partnership has also contributed to several collaborative projects (e.g., CPTP and the Canadian Clinical Trials Network) included in the CCRA's research plan. Although the Partnership has contributed to progress on this outcome, there is evidence of some ambiguity concerning the role of the Partnership in research - whether the Partnership coordinates cancer research itself versus supporting and being a member of the CCRA, which is the coordinating body.
Cancer Control for First Nations, Inuit and Métis Peoples
The development of the First Nations, Inuit and Métis Action Plan on Cancer Control represents an important step towards improving cancer control in First Nations, Inuit and Métis communities. The Partnership played a critical role in facilitating the development of the Action Plan. Although it is up to provincial, territorial, federal and other stakeholders to implement changes to improve cancer control in First Nations, Inuit and Métis communities, there is evidence of change. In 2015, eight jurisdictions were participating in and implementing initiatives to improve the continuity of care in remote and rural locations. Further, cancer agencies or their equivalents in three jurisdictions (Northern British Columbia, Manitoba and Ontario) had in place First Nations, Inuit and Métis specific cancer control strategies, compared to one since the Partnership was first initiated. The delivery of the Saint Elizabeth Cancer Course has increased awareness of culturally relevant approaches to cancer control and care among healthcare providers.
Cancer System Performance
Evidence from the evaluation indicates that the objective of improved analysis and reporting on cancer control system performance has been achieved. The Partnership, in collaboration with its partners and stakeholders, has provided both annual and special cancer system performance reports, some of which have delved into areas of interest to the Partnership's partners. The Partnership has brought forward the pan-Canadian approach to the cancer system performance reports and facilitated the identification of comparable indicators from across Canada. There is evidence that the reports are being used by cancer agencies and other stakeholders to improve cancer control across Canada. However, provinces and territories face challenges in finding the financial and technological resources required to collect the data in support of the System Performance Reporting Initiative. The current approach to system performance reporting, making use of a mix of annual system performance reports on a small core set of indicators, and special reports to consider specific issues, appears to be appropriate looking ahead, as the Initiative matures and provinces meet their performance targets.
Information, Knowledge, Tools and Resources
There is evidence of progress on enhancing access to high-quality information, knowledge, tools and resources through collaboration across numerous players in the cancer control community. The Partnership has increased access to resources through its web presence via cancerview.ca and its corporate site, partnershipagainstcancer.ca. In addition, the Partnership has developed tools such as the Cancer Risk Management Model, which is expected to lead to system wide improvements in cancer control. It has worked on building analytical capacity in jurisdictions and supporting use and analysis through subject-specific websites (e.g., CPTP portal) and collaboration spaces (e.g., cancerview.ca). The maturing of more initiatives will put additional demands on the Partnership's knowledge transfer and exchange efforts.
Public and Patient Awareness and Engagement
Although the Partnership has taken actions to facilitate the enhancement of public and patient awareness and engagement about the Canadian Cancer Control Strategy and cancer control issues, the evaluation found that it is still early days in terms of outcomes. Even though the Partnership, since 2013, has focused its efforts upon engaging the interested public (e.g., cancer community, those in the public to be engaged for a particular outcome such as screening) and embedding the patient perspective in initiatives, rather than broadly engaging the general public and patients, there were key informants who questioned the Partnership's involvement in public awareness. Dialogue with partners and stakeholders to make clear the Partnership's role and intentions related to this outcome appears to be required.
Impact of Immediate Outcomes on Intermediate and Ultimate Outcomes
Although there is agreement that progress on immediate outcomes has been made and that these immediate outcomes are the building blocks for achieving the longer-term outcomes, there is also agreement that the achievement of the outcomes is dependent on action by the jurisdictions. The federated system of healthcare delivery means that there is a need for a strong coordinating body such as the Partnership to facilitate the achievement of the immediate, intermediate and longer-term outcomes of the Canadian Cancer Control Strategy.
There may also be a need for the Partnership to continue to focus on those areas where it can provide the greatest value and drive the most impact in the longer term.
Unintended Impacts
The work carried out by the Partnership has resulted in relationships and approaches which have been used to advantage in other situations, for example, in the broader health care system.
Projects being supported by the Partnership may raise expectations that the cancer control system will change, even in jurisdictions that may not be participating in the projects.
Lessons Learned
The key lesson learned identified by a broad range of interviewees is that the collaborative model employed by the Partnership is effective, i.e., it works. External interviewees consistently expressed satisfaction with how the Partnership has engaged its partners and stakeholders in order to advance the Canadian Cancer Control Strategy. A related lesson learned is the approach taken to engagement with First Nations, Inuit and Métis communities in the development of specific initiatives for each community. Some interviewees noted this approach as a model for other federal departments and agencies for engaging with First Nations, Inuit and Métis.
Internal and external interviewees noted that there is a need for the Partnership to take into consideration the funding and resources available in jurisdictions to participate in projects and initiatives, and later the sustainability of these projects or initiatives as the Partnership moves forward to its next mandate. There is evidence of a need for the Partnership to establish clear points where it will start to withdraw from particular activities and initiatives.
Internal and external interviewees noted the need for the Partnership to involve key collaborators in the planning process for initiatives. These interviewees feel that in some cases the Partnership has brought in key partners only after the priorities have been set.
Demonstration of Economy and Efficiency
Ability to Act More Quickly in Response to Evidence
The Partnership has supported the cancer control system to more quickly respond to evidence. It has supported the ability of jurisdictions without the resources and/or expertise to respond to evidence by providing financial and technical support. It has brought together leaders in various aspects of the cancer control system to discuss and communicate evidence of positive impacts related to these issues and facilitated the transfer of this knowledge through Knowledge Transfer and Exchange (KTE) events and the various networks that have been established (e.g., screening networks).
The Partnership has supported the implementation of quality improvements through the development of best practice guidelines (e.g., screening for distress, radiation therapy, chemotherapy, etc.). The synoptic reporting initiative has resulted in improvements in pathology and the jurisdictional involvement in the CLASP initiatives has supported improvements in planning and policy through sharing knowledge and information from research, practice and policy applications.
The Partnership has supported the cancer control system to make more informed decisions by improving the quality and quantity of data available for decision makers. The annual system performance reports and associated special studies are particularly important examples demonstrating how jurisdictional decision-making has been affected.
Ability to Do More with Same Resources; Acceleration of Achievement of Results; and Alternatives
The Partnership has facilitated the ability to do more with the same resources and accelerate change. The sharing of information and materials has eliminated or reduced the duplication of efforts across jurisdictions to recreate the information and materials. It has accelerated the achievements of results as evidenced by the increased speed with which successive population-based screening programs have been implemented on a pan-Canadian basis, down from 50 years to implement cervical cancer screening to just 3 years to implement colorectal cancer screening.
Interviewees generally agreed that the current model, given the federated model of healthcare in Canada, is the most appropriate model for achieving the desired outcomes. It was further suggested that the Partnership model has been so successful that it could/should be implemented in other areas such as a national strategy for seniors' health, a national strategy for dementia, and a national strategy for cardiovascular health.
Contribution of Partnership to Economy and Efficiency Improvements
The Partnership has contributed to increasing the efficiency and economy of the cancer control system. The key mechanism through which this has occurred is via collaboration and sharing of information across jurisdictions. This sharing has occurred through the development of agendas and frameworks, sharing of best practices, development of protocols and tools and the development of analytical tools (e.g., CRMM), which will help jurisdictions make more efficient and cost-effective decisions. It was noted by interviewees, however, that there were some adverse impacts related to economy that centered on the increased costs associated with the increases in cancer screening and the additional costs relating to data collection and reporting.
Interviewees were in general agreement that the Partnership has made important contributions in addressing some of the tough issues in cancer control in Canada. There was also general agreement that a number of tough issues remain to be addressed, both existent and emerging. Existing issues included programs and services serving the needs of new immigrants and those living in rural and remote areas as well as a further focus on the person-centred perspective. Emerging issues identified were in the areas of costs of cancer care treatment (e.g., cancer drug and technology costs), as well as the impacts of the aging population on the cancer control system.
Recommendations
The evaluation proposed six recommendations directed at CPAC (see annex) and one recommendation directed at Health Canada. Overall the CPAC recommendations cover planning, engagement and role clarification.
Health Canada Recommendation 1
Health Canada should monitor implementation of CPAC's Action Plan to address the evaluation recommendations. Moreover Health Canada should ensure, when appropriate, that revisions are made to CPAC deliverables in any future contribution agreement in response to the evaluation recommendations.
CPAC Recommendation 1
The Partnership model of convening, integrating, catalyzing, and brokering knowledge should continue to be the cornerstone of the Partnership's approach during the remainder of this mandate and in any future mandates. In applying the model, the Partnership should be diligent that it continues to play its current "honest broker" role rather than moving to an advocacy role.
The current model of convening, integrating, catalyzing, and brokering knowledge has provided the Partnership during the current mandate, as it did in the first mandate, significant "space" within which to add value and support changes across the cancer control system, working with a broad range of partners in a variety of arrangements. For these reasons, the evaluation team supports the model's continued use in the future. Moving into an advocacy position, or being perceived as doing so, will jeopardize the value and unique positioning of the Partnership.
CPAC Recommendation 2
In order to continue to be a nimble, adaptive and flexible organization involved in a portfolio of initiatives which shifts to where the Partnership can provide maximum value in addressing needs, the Partnership should have an explicit exit strategy/plan for each initiative/project in which it is involved, and make the exit strategy/plan clear to partners/stakeholders as part of initial initiative/project planning.
In parallel, the Partnership should continue its emphasis upon each initiative/project having a sustainability plan, with targets, so that initiatives/projects complete their life cycle to benefits realization, even if the Partnership itself is no longer involved.
This recommendation emphasizes the importance of the Partnership knowing when and how it will end its involvement in an initiative or project, and that it can do so knowing that the initiative or project will continue to be sustained. In a sense there is already an implicit exit strategy (i.e., what happens if the Partnership is not renewed for a third mandate). An explicit exit strategy needs to consider this scenario, but more generally exit strategies are about keeping the Partnership's portfolio renewed and refreshed, and directed where the Partnership can provide most value. As the Partnership has matured and the initiatives/projects in which it has been involved have grown, a larger amount of resources are required for their ongoing operations and maintenance. However, the Partnership should not be in the operations and maintenance business which is inconsistent with the Partnership's mandate and model. Operations and maintenance should rest with those in the delivery part of the system.
It would also be appropriate for other partners/stakeholders in an initiative/project to have an explicit exit strategy if their involvement is time or resource limited or is expected to end before the completion of the initiative's or project's complete life cycle.
CPAC Recommendation 3
The Partnership should engage with other organizations with a mandate for chronic disease prevention, including cancer, to develop an approach as to how the CLASP model, or something similar, can be scaled to engage the Canadian population on a much broader basis than has taken place to date. The approach would include defining a future role, if any, appropriate for the Partnership.
This recommendation is based upon the advantages that could arise from scaling the CLASP approach to engage a much greater percentage of the Canadian population. However, there is also recognition that risk factors for chronic diseases, including cancer, can cover a range of health determinants, including socio-economic factors and life style choices and behaviours, and that there are numerous organizations with a mandate for chronic disease prevention. The Partnership's experience in navigating such complex landscapes, as an honest broker, may be beneficial. It may also have a role to play as a leader or team member in supporting resulting chronic disease prevention initiatives.
CPAC Recommendation 4
The Partnership should clarify and communicate its current and ongoing role with respect to research coordination - that the Canadian Cancer Research Alliance is the cancer research coordinating body and the Partnership is a member of the Alliance.
This recommendation is intended to clarify the ambiguity that some key stakeholders have concerning the Partnership's role in research coordination.
CPAC Recommendation 5
The Partnership should communicate with stakeholders to clarify that its focus in Public and Patient Awareness and Engagement has been and will continue to be on engaging the interested public and embedding the patient perspective in initiatives, rather than broadly engaging the general public and patients.
This recommendation is intended to reinforce external understanding of the Partnership's focus on engaging the interested public and embedding the patient perspective in initiatives, so that stakeholders understand that the Partnership does not wish to broadly engage the general public.
CPAC Recommendation 6
The Partnership should continue to ensure that partners are engaged early in the planning process for initiatives and projects so that there is buy-in to the plan.
This recommendation responds to the concern expressed by some interviewees that in some cases the Partnership has brought in key partners only after priorities for initiatives and projects have been set.
Management Response and Action Plan for Health Canada
Recommendations | Management Response | Action Plan | Deliverables | Expected Completion Date | Accountability | Resources |
---|---|---|---|---|---|---|
Health Canada should monitor implementation of CPAC's Action Plan to address the evaluation recommendations. Health Canada should ensure, when appropriate, that revisions are made to CPAC deliverables in any future contribution agreement in response to the evaluation recommendations. | Agree | Health Canada will work with CPAC's Chief Executive Officer and Director, Strategy, Evaluation and Analytics, to ensure actions identified in CPAC's response to the evaluation are implemented appropriately | Reports on Progress to the Office of Audit and Evaluation | May 2016 November 2016 |
Director General, Health Care Programs and Policy Directorate | Existing FTEs within the Health Care Programs and Policy Directorate |
Management Response and Action Plan for CPAC
Recommendations | Management Response | Action Plan | Deliverables | Expected Completion Date | Accountability | Required Collaboration |
---|---|---|---|---|---|---|
1. The Partnership model of convening, integrating, catalyzing, and brokering knowledge should continue to be the cornerstone of the Partnership's approach during the remainder of this mandate and in any future mandates. In applying the model, the Partnership should be diligent that it continues to play its current "honest broker" role rather than moving to an advocacy role. |
Agree |
We have reaffirmed the Partnership's model of convening, integrating, catalyzing, and brokering knowledge ourselves with partners and stakeholders as part of our current strategic planning process. This model will continue to be the foundation of our work and reflected in the next strategic plan. The relative emphasis placed on any one of these four components does evolve as the organization matures and the needs of the cancer control system change. The emphasis was on convening in the Partnership's first mandate; on catalyzing and integrating in the second; and would likely be on brokering knowledge in a third. We have confirmed with our partners and stakeholders the value of the Partnership's role as an "honest broker" within the cancer control system. We believe advocacy is best left to others in the system, enabling us to protect that honest broker role. However, others playing an advocacy role can and do use the information developed and disseminated by the Partnership to ensure their efforts are based on evidence. Action: The Partnership will enhance the role of "honest broker" through enhanced focus on knowledge mobilization (KMb) driving the uptake and use of evidence. This will be done by providing active support for KMb throughout the life cycle of initiatives, beginning with planning and contracts to ensure the development of project or initiative-specific KMb plans, through their deployment and evaluation. The Partnership will also determine how to best leverage our digital properties, including cancerview.ca, to support our role as an "honest broker." |
KMb criteria embedded in initiative plans and project management documents |
April 2016 - ongoing |
Canadian Partnership Against Cancer |
Partner organizations, including provincial cancer programs & agencies, National Aboriginal Organizations, PHAC, provincial & territorial governments, relevant NGOs |
KT metrics refreshed to ensure they are accurately assessing knowledge transfer and update; report at portfolio level |
April 2016 |
|||||
2. In order to continue to be a nimble, adaptive and flexible organization involved in a portfolio of initiatives which shifts to where the Partnership can provide maximum value in addressing needs, the Partnership should have an explicit exit strategy/plan for each initiative/project in which it is involved, and make the exit strategy/plan clear to partners/stakeholders as part of initial initiative/project planning. In parallel, the Partnership should continue its emphasis upon each initiative/project having a sustainability plan, with targets, so that initiatives/projects complete their life cycle to benefits realization, even if the Partnership itself is no longer involved. |
Agree |
We agree that planning for sustainability and/or exiting a particular program of work is ideally embedded from the outset. As the Partnership matures, we are getting better at doing this; an example is how we embedded sustainability planning in the RFP process associated with the First Nations, Inuit and Métis initiative. This has become our standard approach, as applicable. Action: The Partnership is looking at its existing portfolio as part of strategic planning to determine what work will be concluded, continued or evolve. Sustainability planning and exit strategies will be discussed as appropriate, including plans, risks, etc. during quarterly reporting for initiatives that are concluding. For new initiatives, exit strategies and sustainability planning will be integrated into initiative life cycle planning from the outset. This will be managed by the Delivery Management Office (DMO) as part of the annual cycle of planning and quarterly reporting, with accountability shared by the program areas, DMO and Strategic Management Committee. |
Review of existing, concluding and potential new initiatives to determine which need sustainability plans and which need exit plans |
June 2016 |
Canadian Partnership Against Cancer |
Health Canada;Partner organizations, including provincial cancer programs & agencies, National Aboriginal Organizations, PHAC, provincial & territorial governments, relevant NGOs |
Development and implementation of exit strategies for discontinued initiatives |
October 2016 |
|||||
Sustainability planning for continuing and new initiatives embedded into annual planning and quarterly review cycles |
Commencing with fiscal year 2017-2018 |
|||||
3. The Partnership should engage with other organizations with a mandate for chronic disease prevention, including cancer, to develop an approach as to how the CLASP model, or something similar, can be scaled to engage the Canadian population on a much broader basis than has taken place to date. The approach would include defining a future role, if any, appropriate for the Partnership. |
Agree |
Where initiatives have been effective, we will continue to look for ways to scale up or expand the scope of coverage to achieve larger-scale impact across the entire country. The Coalitions Linking Action and Science in Prevention (CLASP) initiative is one such example. As noted in the response to recommendation #1, the emphasis in a third mandate would likely be on brokering knowledge. This is in recognition of the fact that there are many opportunities for the Partnership to disseminate knowledge gained during previous mandates, and a key element of scaling up successful initiatives. A pillar of the Partnership's approach has been to work collaboratively with partners with a chronic disease mandate (an example is the Public Health Agency of Canada). We will continue to work with and learn from these partners; however, our focus will be on cancer control specifically, where we will look for areas in which we can add the most value, accelerating progress towards shared cancer control goals without duplicating efforts. Discussions with partners on future initiatives in prevention are also focusing on the potential for broader impact on cancer control outcomes; the successes of CLASP and current partnerships are important foundational elements upon which we can now build. Action: As part of strategic and business planning in 2015 and the first half of 2016, CPAC will review its portfolio of initiatives to assess opportunities for scale-up. By January 2016, CPAC will have determined at a high level how it is partnering with the Public Health Agency of Canada and Canadian Cancer Society; this will be embedded in the strategic plan. More detailed planning completed by July 2016. |
Strategic plan |
January 2016 |
Canadian Partnership Against Cancer |
Health Canada, PHAC, CCS and other partners |
Business plan |
July 2016 |
|||||
4. The Partnership should clarify and communicate its current and ongoing role with respect to research coordination - that the Canadian Cancer Research Alliance is the cancer research coordinating body and the Partnership is a member of the Alliance. |
Agree |
The Partnership is not a traditional funder of research operating grants; the Canadian Institute of Health Research (CIHR) is the agency mandated to support research. However, at the time of the Partnership's creation, there were a number of research coordination gaps that could not have been addressed with the funding models available through traditional research funders at that time. Through the research initiatives supported by the Partnership - the Canadian Partnership for Tomorrow Project (CPTP), the Canadian Cancer Research Alliance (CCRA) and 3CTN - the Partnership is supporting research collaboration across the cancer control continuum. CIHR is supportive of the Partnership's role in this capacity, and is an active member of CCRA. CPTP is a major cohort study that includes five regional cohorts. CPTP was a one-time project for the Partnership to drive the development and launch of this research platform, and the plan has always been to migrate the platform to another organization once launched. We are in active discussions with parties that are most appropriate to manage the platform and maximize its use for research. CCRA convenes the major funders of cancer research to determine shared priorities. The group exists with the Partnership's financial and infrastructural support, and its executive director is a Partnership staff member. CCRA has been effective in avoiding duplication of efforts and encouraging strategic investments in complimentary areas of research. While the Partnership will continue to support CCRA in this coordinating role in a third mandate, it may be appropriate for CCRA to expand to include other activities not funded by the Partnership. The Partnership is the first of eight funders of 3CTN, a pan-Canadian initiative to improve the efficiency and quality of clinical trials in Canada through a network of regional centres. The Partnership was well situated to take on the initiation of this process, which involved identification of a host agency and the adjudication of a business case. The host agency is now the Ontario Institute for Cancer Research. The Partnership is represented on the steering committee, and several partners are cancer control agencies, rather than traditional research funders, reflecting this initiative's desire to be grounded in impact on cancer care. Action: The Partnership will communicate its research-related activities and their context more effectively; for example, through the next strategic plan and as the Partnership develops and rolls out research coordination activities. Through its own communications, CCRA will convey its mission and relationship with the Partnership. |
Description of relationship to the Partnership added to CCRA's corporate description (i.e., in external-facing communications materials, including the website) |
January 2016 |
Canadian Partnership Against Cancer |
CCRA members, CIHR, other partners |
Description of the Partnership's role in research, including CCRA, included in strategic plan |
January 2016 |
|||||
5. The Partnership should communicate with stakeholders to clarify that its focus in Public and Patient Awareness and Engagement has been and will continue to be on engaging the interested public and embedding the patient perspective in initiatives, rather than broadly engaging the general public and patients. |
Agree |
The Partnership made the decision a few years ago that our primary audiences are those engaged directly in cancer control, such as system administrators, health professionals and researchers. This was done through a very deliberate process that included surveying the roles played by others in cancer control with respect to communicating with the public (e.g., Canadian Cancer Society, Canadian Cancer Action Network), and with direction from the Board of Directors. While we avoid reaching out directly to the general public through our communications products, it is vital that our work is informed by and reflects the needs and perspectives of individuals affected by cancer. As such, we use a variety of approaches to engage effectively with this "interested public" and get input into our work. This includes the use of deliberative engagement as part of several projects that have an impact on patients, such as a study on high-volume cancer surgery and the Canadian Partnership for Quality Radiotherapy. This is consistent with the emphasis in the original Canadian Strategy for Cancer Control on embedding the "patient voice" in the Partnership's work. Action: The Partnership will continue to clarify communications about the role of the Partnership in public awareness and patient engagement. This will be done in part by developing a description of the parameters within which the Partnership engages with the public and the rationale for doing so for inclusion in communications materials related to the Public Engagement and Outreach initiative and, where appropriate, in corporate materials. We will also seek opportunities to engage with the public on relevant issues, as appropriate, using methods such as deliberative engagement and polling. This work will be defined each year as part of the Partnership's annual plan. |
Clarifying statement regarding when and how the Partnership communicates directly with the public developed and plan for embedding the statement in relevant materials defined (e.g., corporate website, annual report) |
January 2016 |
Canadian Partnership Against Cancer |
|
We will continue to seek opportunities to leverage partner efforts in this area to inform our work. Specifically, the Partnership will continue to partner with the Canadian Association of Provincial Cancer Agencies and the Canadian Cancer Society to communicate the parameters within which the Partnership communicates to the public. The Partnership will work with these and other partners to leverage opportunities for shared communications that highlight joint actions to improve cancer control for Canadians. |
Partnered work with the Canadian Association of Provincial Cancer Agencies and the Canadian Cancer Society embedded in the Partnership's strategic plan and/or business plan |
January 2016 - July 2016 |
||||
6. The Partnership should continue to ensure that partners are engaged early in the planning process for initiatives and projects so that there is buy-in to the plan. |
Agree |
The Partnership is privileged to be the recipient of funding to support programming on behalf of the cancer control community. This means all our work must be co-created with our partners. We recognized early in the second mandate that there was some unevenness in how we engaged with key partners and have addressed this in a number of ways, including the development and use of a stakeholder map to identify key relationships and ensure we are inclusive of all our partners. The Partnership's approach to engagement with key partners has evolved over the last few years, partly related to the fact that a significant portion of our work is done in collaboration with the same partners: particularly the provincial cancer agencies, National Aboriginal Organizations, provincial/territorial governments, and other federally-funded health organizations with a pan-Canadian mandate. We discuss new programmatic work with the Canadian Association of Provincial Cancer Agencies (CAPCA) members and the National Aboriginal Organizations in the developmental phase. If the work is not supported by these organizations, it does not proceed. In recognition of the fact that much of our work is done with the same partners, we are also actively rethinking our approach to procurement to more explicitly facilitate co-creation. A recent example of co-creation is the development of a Partnership-funded initiative to support smoking cessation programs in the context of tobacco control efforts. Recognizing that tobacco control is a key driver of cancer control outcomes, the Partnership convened a meeting with representatives of organizations and initiatives involved in tobacco control. The aim was to determine whether there was an appropriate role to be played by the Partnership, given the evidence, gaps, and efforts of these other organizations. After discussion over the course of several stakeholder planning sessions, the Partnership's Integrated Evidence-Based Tobacco Cessation as a Cancer Care Quality Improvement Initiative was co-created with the full support of all partners. We will evaluate the success of this model and anticipate integrating lessons learned while deploying a similar approach to defining new work moving forward. Action: The Partnership will continue to use its stakeholder map to ensure key partners are engaged in strategic planning. |
Specific programs of work with key partners defined as part of the strategic planning process and embedded in the Partnership's strategic plan and business plan |
January 2016 - July 2016 |
Canadian Partnership Against Cancer |
Partner organizations, including provincial cancer programs & agencies, National Aboriginal Organizations, PHAC, provincial & territorial governments, relevant NGOs |
The Partnership is undertaking a procurement review to evaluate its current approach to procurement and determine opportunities to streamline the process. |
Procurement review report and action plan |
March 2016 |
1.0 Evaluation Purpose
The purpose of this independent performance evaluation was to assess the relevance and performance of the Canadian Partnership Against Cancer (the Partnership or CPAC) from April 2012 to June 2015. It was conducted to fulfil a requirement of the Partnership's 2012-2017 funding agreement with Health Canada. Moreover, it was an opportunity for the Partnership to take stock, be able to understand more fully its successes and areas where improvement or recalibration may be necessary, communicate these to its stakeholders, and use the findings and conclusions as input to its own decision-making. The evaluation was conducted in accordance with the Treasury Board of Canada's Policy on Evaluation (2009).
2.0 Description of the Partnership
2.1 Context
2.1.1 Historical Background
The Partnership was implemented as a means to advance the 2006 Canadian Strategy for Cancer ControlFootnote 2 (CSCC) and accelerate action on cancer control across Canada. The CSCC was built by more than 700 cancer groups, experts, patients and survivors.
The Partnership's initial funding agreement with Health Canada was for $250 million for the period 2007-2012. Areas of focus and action were guided by the Partnership's 2007-2012 Strategic Plan. The objectives of the Partnership's strategy were to:
- Reduce gaps in knowledge to enhance cancer control;
- Facilitate and accelerate implementation of best available knowledge;
- Optimize quality and access; and
- Improve the cancer experience for Canadians.
The Partnership's second funding agreement with Health Canada was for $241 million for the period 2012-2017. During this period, which is the Partnership's second mandate in advancing the CSCC, the Partnership is guided by its 2012-2017 Strategic Plan. The 2012-2017 Strategic Plan, contained in Sustaining Action Toward a Shared Vision, "articulates the high-level road map that will guide the Partnership's next phase of work to advance Canada's cancer strategy and corresponding desired outcomes". At the time of this evaluation, the Partnership was mid-way through its second mandate. More about the Partnership's second mandate is included in section 2.2.
2.1.2 Previous Evaluations
In accordance with its initial funding agreement, an independent evaluation of the Partnership was published in 2009.Footnote 3 That evaluation made a number of recommendations related to achievement of the CSCC, building the organization, core frameworks and corporate enablers, and implementation activities. Although examined within the scope of the evaluation, no recommendations were made related to overall governance and accountability mechanisms, or to refinement and focusing of the CSCC. The recommendations of this evaluation are summarized in Appendix 2.
An organizational evaluation of the Partnership was conducted by Health Canada and published in May 2010.Footnote 4 The evaluation made recommendations in the areas of design and delivery, success, and governance. The recommendations are listed in Appendix 2.
The Partnership acted on the recommendations of these two former evaluations, as well as evaluations of specific initiatives undertaken by the Partnership, during its first and second mandate. For example, its 2012-2017 Strategic Plan for the second mandate reflects recommendations made in the 2010 organizational evaluation. Initiatives to report on cancer system performance, impacts and benefits realization, partner and stakeholder engagement and focus on First Nations, Inuit and Métis all have responded to recommendations from the organizational-level evaluations.
2.2 Profile
The nine outcomes for 2017 are embedded in the Immediate Outcomes in the Partnership's logic model for the CSCCFootnote 5. Eight of these are shared outcomes for the cancer control system, and one - increased efficiency and acceleration of cancer control in Canada - "belongs" to the Partnership. In order to contribute to the achievement of these outcomes, the Partnership's 2012-2017 Strategic Plan outlines five strategic priorities and three core enabling functions. The strategic priorities are:
- Develop high-impact, population-based prevention and cancer screening approaches;
- Advance high-quality early detection and clinical care;
- Embed a person-centred perspective throughout the cancer journey;
- Enable targeted research to augment our knowledge and understanding of cancer and related chronic diseases; and
- Advance cancer control with and for First Nations, Inuit and Métis communities.
The three core enabling functions are:
- System performance analysis and reporting;
- Knowledge management through tools, technology, connections and resources; and
- Public engagement and outreach.
The 2012-2017 Business Plan that accompanies the 2012-2017 Strategic Plan outlines the initiatives that are taking place in each of the five strategic priorities and three core enabling functions. A listing of the initiatives currently underway is provided in Appendix 3.
An enhanced Performance Measurement Strategy (PMS) was developed in 2012 to provide the basis for understanding progress and impact. The PMS is tied to the logic model for the CSCC and includes a comprehensive set of performance indicators, many of which have already been implemented. The Partnership was proactive in developing this system and integrating the information it provides into its management and governance, and its external reporting. Quarterly outcome reports are produced for use by management and the Board. Many of the performance indicators also feed into the annual cancer system performance reports produced by the Partnership. The available performance data provided a rich and readily accessible source of information for this evaluation.
Annual corporate plans provide more precision on the initiatives. Annual reports communicate progress against the plans and impacts to date.
2.3 Program Narrative
The program logic and narrative for the CSCC, including the Partnership's contribution to it, are fully described in the Performance Measurement Strategy document.Footnote 6 The implementation of the PMS took place during the course of this evaluation. The outcome statements are introduced in the findings section 4.4 of this report.
As noted in the PMS:
Given the complexity of the cancer control landscape and the Partnership's collaborative model for translating knowledge into action, quantifying and attributing changes to the system is challenging. To mitigate this challenge within its Performance Measurement Strategy, the Partnership has distinguished between measurement of the shared outcomes of the collective efforts of all who are engaged in cancer control, and measurement of the specific contributions made by the Partnership to those outcomes: increased efficiency and acceleration of cancer control in Canada.
The Partnership consults and collaborates with the cancer control community. It also engages the public including those with an interest in or affected by cancer, and First Nations, Inuit and Métis partners. Partners and stakeholders include:
- Provincial cancer agencies and programs;
- Health delivery organizations;
- Federal, provincial and territorial governments;
- Cancer control and health experts;
- Clinicians, health-care professionals, researchers and health professional organizations;
- Federal agencies and funded organizations (e.g., Canadian Institute for Health Information (CIHI), Canada Health Infoway, Canadian Institutes of Health Research (CIHR), Statistics Canada, Public Health Agency of Canada (PHAC), Canadian Agency for Drugs and Technologies in Health (CADTH));
- Patients, survivors and family members, as well as patient groups;
- National aboriginal organizations (NAOs);
- Non-governmental organizations (NGOs);
- Public health agencies and where appropriate, provincial government programs; and
- Research funding organizations.Footnote 7
According to the program logic model, the Canadian Partnership Against Cancer has four activity areas: convene stakeholders, integrate solutions into existing portfolios, catalyze the adoption of best evidence programs, and broker knowledge and information. These activities are expected to lead to the following immediate, intermediate and long-term outcomes.
Immediate Outcomes
It is anticipated that by 2017, program activities will improve: access to evidence-based prevention strategies; the capacity to respond to patient needs; cancer control for First Nations, Inuit and Métis peoples and partners; and analysis and reporting on cancer system performance. Additionally, these activities will enhance access to high-quality information, tools and resources; the coordination of cancer research and the population research capacity; and public and patient awareness and engagement. Further, it is expected that there will be an increased number of consistent actions to enhance the quality of diagnosis and clinical care.
Intermediate Outcome
Achieving immediate outcomes is expected to enhance: population-based prevention and screening; the quality of diagnosis and clinical care; and the cancer control system and synergies with the broader health system. It will also improve the cancer experience with, and for, Canadians. The program estimates that these outcomes should be achieved between 2018 and 2027.
Ultimate Outcomes
By 2037, the Canadian Partnership Against Cancer anticipates that achieving the above outcomes will result in reduced incidence of cancer; lessened likelihood of Canadians dying from cancer and enhanced quality of life for those affected by cancer.
2.4 Alignment and Resources
The federal government's support for the Partnership is aligned with Health Canada's Strategic Outcome 1 and related Program 1.1 and Sub-Program 1.1.1 in Health Canada's Program Alignment Architecture (PAA) for 2013-2014.Footnote 8
- Strategic Outcome 1: A health system responsive to the needs of Canadians.
- Program 1.1: Canadian Health System Policy - program objective is "to support innovative health care policy and programs to help Canadians maintain and improve their health"
- Sub-Program 1.1.1: Health System Priorities - program objective is "to ensure that Canadians have access to quality and cost-effective health care services". [Note that the funding of the Partnership is described under this sub-program in Health Canada's 2013-2014 Departmental Performance Report (DPR) submitted to Parliament.]
- Program 1.1: Canadian Health System Policy - program objective is "to support innovative health care policy and programs to help Canadians maintain and improve their health"
The allocation (budget and actual) of funding across the Partnership's strategic priorities and core enabling functions, as well as for its own operations (i.e., corporate support), is shown in Table 1 on the next page. The exhibit shows significant re-profiling of the 2013-2014 budget. In particular this reflected 2013-2014 expenditures related to three Strategic Priorities - Diagnosis and Clinical Care, Person-Centred Perspective; and Targeted Research (in particular, the Canadian Partnership for Tomorrow Project). The reasons for re-profiling were that initiatives are developed and implemented together with partners and at times this results in unplanned (i.e., when the original budget was set) changes to the pace of initiatives and associated procurements and spending within the multi-year contractual commitments with these partners. The majority of the under-spending was pushed forward to future years.
CPAC Strategic Priorities and Core Enabling Functions | Budget 12/13 | Actual 12/13 | Variance 12/13 | % Budget spent 12/13 | Budget 13/14 | Actual 13/14 | Variance 13/14 | % Budget spent 13/14 |
---|---|---|---|---|---|---|---|---|
|
||||||||
Population based Prevention and Screening | $6,203,611 | $4,962,642 | $1,240,969 | 80.00% | $7,771,607 | $6,507,840 | $1,263,767 | 83.74% |
Diagnosis and Clinical Care | $3,229,298 | $1,743,559 | $1,485,739 | 53.99% | $8,267,645 | $2,237,972 | $6,029,673Table 1 footnote 2 | 27.07% |
Person-centred Perspective | $1,598,978 | $1,590,760 | $8,218 | 99.49% | $5,008,825 | $1,500,112 | $3,508,713Table 1 footnote 3 | 29.95% |
Targeted Research | $9,471,589 | $5,998,184 | $3,473,405 | 63.33% | $8,963,343 | $7,432,454 | $1,530,889Table 1 footnote 4 | 82.92% |
Advance cancer control, First Nations, Inuit and Métis communities | $1,633,008 | $865,182 | $767,826 | 52.98% | $2,150,000 | $2,015,508 | $134,492 | 93.74% |
System Performance | $1,523,839 | $974,443 | $549,396 | 63.95% | $1,397,066 | $1,110,966 | $286,100 | 79.52% |
Knowledge Management and Public Engagement and Outreach | $8,807,098 | $8,197,497 | $609,601 | 93.08% | $9,564,533 | $7,314,308 | $2,250,225 | 76.47% |
Corporate Support to Programs | $9,020,583 | $7,604,260 | $1,416,323 | 84.30% | $7,288,742 | $7,974,289 | -$685,547 | 109.41% |
One-Time Costs - 2nd Mandate Establishment | $1,945,657 | ($1,945,657) | ||||||
Total CPAC | $41,488,004 | $33,882,182 | $7,605,822 | 81.67% | $50,411,761 | $36,093,449 | $14,318,312 | 71.60% |
3.0 Evaluation Description
3.1 Evaluation Scope, Approach and Design
The scope of the evaluation had both a "big picture" goal:
...the impact since 2007 of a coordinated, collaborative approach to national cancer control, including the groundwork that has been laid, what has been achieved that will directly affect patients by 2017 and how we will see these efforts expanding their reach and impact for all Canadians over the next two decades with sustained effort...
and a "performance of the Partnership" goal:
... the Partnership's performance in 2012-2017 in delivering on its mandate.
These two goals were further articulated through five objectives set in the evaluation's terms of reference by the Partnership's Board of Directors.
- Demonstrate that the Partnership's 2012-2017 programs and initiatives are being implemented and managed to achieve the immediate outcomes and lay the foundation for achieving the intermediate and ultimate outcomes detailed in the Performance Measurement Strategy and logic model for the CSCC.
- Assess the Partnership's progress toward achieving the immediate outcomes (by 2017) relative to the overall strategic directions outlined in the 2012-2017 strategic plan and annual corporate plans.
- Evaluate the Partnership's ability to accelerate evidence-based change and introduce efficiencies to the cancer control system through its collaborative approach.
- Examine mid-way through the Partnership's second mandate (eighth year since inception) what is working well and areas for improvement to ensure continued success in achieving the stated outcomes (immediate, intermediate and ultimate) of the CSCC.
- Assess whether activities and outcomes that are achieved are likely to result in impact on the intermediate and ultimate outcomes described in the logic model.
In meeting these objectives, the evaluation considered the five core evaluation issues in the 2009 Treasury Board Directive on Evaluation. The evaluation questions are shown in Appendix 4.
The evaluation was conducted in three phases, each with milestones and deliverables.
- Phase 1: Develop project plan and confirm existing evaluation framework;
- Phase 2: Conduct evaluation and draft report; and
- Phase 3: Report on evaluation findings.
More about the methods used to conduct the evaluation is provided in Appendix 4.
3.2 Limitations and Mitigation Strategies
Limitation | Impact | Mitigation Strategy |
---|---|---|
Key Informant Interviews - interviews retrospective in nature. | Interviewees may provide recent perspectives on past events. Can impact validity of assessing activities or results. | Interviewees asked to provide and describe specific examples during the time period under review. Also, triangulation with other lines of evidence. |
Key Informant Interviews - selection of interviewees. | Program partners and stakeholders with particular views may be missed. | Candidates for interviews were selected from across the categories of partners and stakeholders. Within each category, there was a purposeful identification of candidates. |
4.0 Findings
This section presents findings related to each of the evaluation issues and evaluation questions (see Appendix 4). The findings, shaded in gray, are given at the beginning of each sub-section, and the supporting information and evidence follows. Conclusions that derive from the findings are presented in section 5.0 of this report, and recommendations are made in section 6.0.
4.1 Relevance: Issue #1 - Continued Need for the Program
Findings
The current demographics in Canada (i.e., aging population, individuals living longer with cancer) and increasing incidences of cancer and its impacts on Canadians are indicative of an ongoing need for a coordinated approach to cancer control. The gaps and opportunities for improvement that, despite significant progress made to date, still exist in the cancer control system, as well as the long time period to benefits realization and Canada's federated model for health care, demonstrate the continued need for a sustained (i.e., ongoing) and coordinated effort in transforming (e.g., major changes) and improving the cancer control system across Canada.
The Partnership has played and continues to play an important and unique role in the sustained and coordinated effort for cancer control. Its positioning, in the words of many of those interviewed in this evaluation, as an "honest broker" has facilitated jurisdictions to take pan-Canadian perspectives and develop and implement multi-jurisdictional solutions across the components of the cancer control continuum. There is strong support among stakeholders interviewed for the evaluation for the Partnership's value proposition going into a third mandate and even beyond.
The Partnership's model of convening, integrating, catalyzing and brokering knowledge has been appropriate to its role and supported its value proposition. The model has clearly delivered value, which is recognized and appreciated by key informants representing important stakeholder groups. The Partnership's reputation as an "honest broker" has been built by its careful and deliberate use of the model to guide what it does and what it does not do. Almost all key informants expressed a continued need for the model, with some emphasizing that the Partnership must "stay true to the model". Others suggested ideas such as framing questions to challenge the system and to force discussion on some key issues (e.g., balance between prevention and treatment investments) that may stretch simple definitions of the components of the model. Still others emphasized the need for the Partnership to be a nimble, adaptive and flexible organization involved in a portfolio of initiatives which shifts to where it can provide maximum value. This means consideration of sustainability for initiatives launched under the umbrella of the CSCC. It also means consideration of an exit strategy by the Partnership regarding its own role and involvement in those initiatives so that it continues to be able to adapt and re-profile its portfolio of initiatives where the system needs and Partnership's value have the greatest impact.
4.1.1 Continued Need for/Importance of a Sustained Coordinated Effort
Cancer is, and will continue to be, a serious problem with significant impacts upon the lives of Canadians. For example, cancer incidence continues to rise. Between 2007 and 2031, it is expected that new cancer cases will increase by 71 per cent, while the Canadian population will increase by only 19 per cent over the same time period.Footnote 9 Cancer incidence is a particular concern for First Nations, Inuit and Métis populations. Although research is limited, existing studies invariably show that cancer incidence has risen dramatically in each of the First Nations, Inuit and Métis populations over the past few decades. From being nearly unknown a few generations ago, cancer is now among the top three causes of death among First Nations, Inuit and Métis peoples.Footnote 10 At the same time as cancer incidence continues to rise, advances in the cancer control system mean that survival is also increasing. By 2031, it is estimated that 2.5 times as many Canadians (2.2 million) will be living with a cancer diagnosis compared to 2007.Footnote 11
Experience has shown that actions can be taken (e.g., reduction in smoking rates, screening programs) that will have significant impacts on cancer control in the future. Such areas of action and impact were defined in the CSCC, and more precisely elaborated in the Partnership's Strategic Plans for 2008-2012 and 2012-2017, and associated annual corporate plans. The results of actions taken can be seen in progress made on outputs and immediate outcomes in the logic model for the CSCC. There are initiatives such as cancer screening and the Canadian Partnership for Tomorrow project which started in the Partnership's first mandate and are more mature and further along in their development. Others such as the person-centred perspective and cancer control for First Nations, Inuit and Métis have been more the focus of the Partnership's current mandate and are still in early days in terms of outcome achievement. This experience to date and plans for the future demonstrate that there is a time lag (sometimes many years) between action taken and impacts/benefits being realized. Dealing with such time lags requires a sustained, i.e., ongoing, effort.
As will be described in section 4.4, there have been improvements across the cancer control continuum. However, gaps/issues still remain. Some are national-level issues such as cancer control for First Nations, Inuit and Métis peoples and communities, as well as underserved populations. Some relate to sharing best practice approaches and tools across jurisdictions. Others focus upon practitioners, patients and the public. When these pan-Canadian gaps and issues are superimposed upon Canada's federated model of health care, in which jurisdictions may vary considerably in their priorities, delivery approaches and available capacity/resources, there is a strong argument and clear need for a coordinated effort at a pan-Canadian level.
Key informants across all stakeholder groups supported these arguments about the need for a sustained coordinated effort. They noted that such efforts help to identify areas upon which to focus improvement initiatives, to deal with the lengthy time periods required for implementation and benefits realization, to provide transparency and order to spending, to improve efficiency and to inform jurisdictional priority setting and decision-making. Without a sustained coordinated approach, they believe that efforts would be less deliberate, take longer to achieve results and be more fragmented. Smaller jurisdictions and pan-Canadian efforts would be particularly affected.
Finally, there appears to be ongoing broad public support for a national strategy for a sustained coordinated approach. In 2011, 93% of Canadians felt there should be a national cancer strategy to reduce the risk of cancer, lessen the likelihood of people dying from cancer, and enhance the quality of life for those affected by cancer.Footnote 12 In a more recent poll, conducted in late 2014, 94% of Canadians supported the national cancer strategy.Footnote 13
4.1.2 Continued Need for Partnership to Achieve Intermediate and Ultimate Outcomes
The Partnership as a legal and governance entity was launched from the original CSCC.Footnote 14 Therein it was recommended that the best vehicle for moving the CSCC forward ("the model of governance and management will ultimately determine the effectiveness and success of the CSCC") while adhering to key principles was "an independent stand-alone model of governance". The key principles related to:
- The CSCC must be efficient. Decisions should be made faster and take into account the needs of patients and stakeholders; knowledge must be transferred across the country efficiently and effectively; and duplication of effort in cancer control must be minimized.
- The CSCC must be transparent. Decisions and rationale should be communicated and discussed publicly; anyone should have the ability to review investment against targets, goals and outcomes; the CSCC must support the achievement of real and measurable results.
- The CSCC must be accountable. The CSCC should be accountable for decisions and actions to both its own governing body and to external organizations that have a substantive interest in one or more of the programs, including governments (federal and provincial), cancer agencies, non-governmental organizations and industry.
- The CSCC must be inclusive. Canada will be one of the first jurisdictions worldwide to give cancer patients/survivors and stakeholders an active role in both decision-making and policy setting. Establishing Priorities for action should be done by all stakeholders, based on the needs of patients and the knowledge of experts in cancer from across the country.
- The CSCC must be flexible. It should enhance our ability to adapt more quickly to evidence and knowledge as it becomes available. The CSCC must allow all stakeholders, including governments, cancer agencies, medical professionals and patients to access and use our best evidence and knowledge to better manage and control cancer in Canada.
These principles continue to be reflected in the Partnership's governance, management, plans and processes today. For example, elements of the principles are embedded into the logic model for the CSCC, wherein the immediate outcome for which the Partnership is specifically accountable is: "Increased efficiency and acceleration of cancer control in Canada".
The Partnership is seen by key informants as having brought value to the cancer control community and system. It is considered to have been an important part of the system and a unique contributor to the sustained coordinated approach, especially in areas such as prevention and screening where it has helped accelerate uptake and implementation. In its positioning and support for pan-Canadian perspectives, it is described in terms such as "honest broker". It is credited with helping provincial and territorial jurisdictions move to implementation (i.e., knowledge to action) with a better balance across the various components of the cancer control continuum, better knowledge and understanding influencing decision-making, and more efficient approaches. This is especially the case for smaller jurisdictions without the scale and resources to "go it alone".
The above needs identified by key informants as the value proposition delivered by the Partnership are expected to continue in provincial and territorial jurisdictions, cancer agencies, and non-governmental organizations. This was articulated through the support for a third mandate (2017-2022) for the Partnership by most key informants.
4.1.3 Continued Need for Partnership's Model of Convening, Integrating, Catalyzing and Brokering Knowledge
The coordinated and sustained approach to cancer control upon which the CSCC is based contains internationally recognized, as per published research, elements and features of effective cancer control. According to the literature reviewed for this evaluation, these elements and features include collaboration, knowledge translation, governance that is inclusive of partners, and integration of lessons learned. The Partnership's specific role in the sustained coordinated approach is exemplified in its model of "convening, integrating, catalyzing and brokering knowledge with and through partners".Footnote 15 This model has helped define the scope of the Partnership's role, that is, what it does and, equally importantly, what it does not do.
There are strong examples, such as introduction of screening programs and synoptic reporting, where the Partnership's model has accelerated progress. Such examples are further described in section 4.4. They are the types of examples mentioned by the majority of both external and internal key informants in their support for the Partnership's model and its contributions to date.
Going forward, key informants noted their continued support for the model, although there were, quite naturally, different opinions on the components of the model that should be emphasized. For example, many key informants noted the success that the Partnership had achieved in "convening" during its first mandate and now in its second mandate. Moving forward, some suggested that the Partnership should have more active ownership of some issues, by convening key stakeholders to "force" these issues to be discussed. In terms of "integrating", some key informants thought that more integrated solutions across chronic diseases would offer benefits, while others thought that organizing around themes such as the patient perspective would be beneficial. Many key informants noted the issues around sustainability of initiatives, especially when scarce resources contribute to the implementation challenges faced by jurisdictions. Sustainability is really the next challenge beyond "catalyzing", and raises questions about not only planning for sustainability but also for the Partnership's exit at an appropriate stage in each initiative so that it can focus on other areas offering the greatest opportunities for value. In "brokering knowledge" some key informants thought that the Partnership should do more than provide evidence such as the annual system performance reports, and move into defining questions about the relevance of cancer practices and approaches, that would lead jurisdictions to examine their individual practices in cancer control. Some key informants also noted the need to "double back" to encourage the spread of changes and new programs and practices across jurisdictions, for example, to those not involved in the initial initiative, and to identify and address higher-level issues and barriers to change.
4.2 Relevance: Issue #2 - Alignment with Government Priorities
Findings
The Partnership is aligned with the priorities of the federal government and the priorities and strategic outcomes of Health Canada. The alignment is demonstrated by statements made by the former Prime Minister (2011) and the former Minister of Health (2014). It is also demonstrated by how Health Canada's funding support for the Partnership fits within Health Canada's Program Alignment Architecture (2013-2014) and Departmental Performance Report (DPR) (2013-2014) to Parliament. The Partnership has carried this alignment through to its own strategic plans for its first and second mandates, and its annual corporate plans and annual reports.
The Partnership also has good alignment with the priorities of its key partners, including jurisdictions, cancer agencies, NGOs and patient groups. In some cases, these are reflected in shared ultimate outcomes, and in others the same priorities expressed in forward looking documents such as strategic plans.
4.2.1 Alignment with Federal Government and Health Canada
The Partnership continues to be aligned with federal government priorities and with Health Canada's priorities and strategic outcomes. This is demonstrated by statements made by the former Prime Minister and former Minister of Health in announcing support for the Partnership's second mandate and later during the third mandate.
Every year, millions of Canadians are affected by cancer, either through personal struggle, or by supporting a family member, friend or neighbour who suffers from this devastating disease. The Government's commitment [is] to keep Canadians and their families healthy, to help doctors detect cancer sooner and give health-care workers, support groups and survivors the help they need to fight back. (Former Prime Minister Stephen Harper - March 10, 2011)Footnote 16
Today, the Government announced its plan to renew funding to the Partnership over five years - 2012 to 2017 - to help it continue its invaluable work. The former Prime Minister Harper said "The funding announced today is part of our Government's commitment to keep Canadians and their families healthy, to help doctors detect cancer sooner and give health-care workers, support groups and survivors the help they need to fight back."Footnote 17
Cancer has touched the lives of Canadians across the country. Our Government is pleased to support the Canadian Partnership Against Cancer and the Tomorrow Project, which aims to find the causes of cancer and help us save lives. Let's continue to work together and share in the hope of finding a cure and work together to raise awareness about cancer. (Rona Ambrose, former Minister of Health, February 3, 2014)Footnote 18
Alignment is also demonstrated in Health Canada's 2013-2014 Program Alignment Architecture and its reporting to Parliament. Health Canada's funding of the Partnership is connected to Health Canada's PAA at the Sub-Program level, as was described in section 2.4.
Further, a review of the Partnership's strategic plans for its first and second mandates, as well as its annual corporate plans and annual reports, shows a strong alignment with the CSCC which is the basis of the Partnership's funding agreement and as such reflects federal government and Health Canada priorities.
Key informants corroborated these findings. There was consensus amongst federal government key informants that the Partnership is aligned with federal government priorities, especially with respect to its focus on healthy Canadians, reducing health care costs, needs of First Nations, Inuit and Métis, and working more in collaboration with the provinces/territories and other organizations. All federal government key informants also thought that there is strong alignment of the Partnership with Health Canada's priorities and strategic outcomes. Health Canada's priorities of safety, strengthening the health care system and universal access to programs, as well as First Nations, Inuit and Métis work and focus on access to high quality interventions (through screening, diagnosis and evidence-based treatment) were identified as covering and relating to many of the Partnership's priorities.
4.2.2 Alignment with Partners
In addition to the core evaluation issue concerning alignment with federal government priorities, the evaluation also gathered information about alignment with the priorities of other key partners. Overall, most key informants, both internal and external, felt that there is alignment (in some cases stated as "strong alignment") of the Partnership with the priorities and objectives of partners, including jurisdictions, cancer agencies, NGOs such as the Canadian Cancer Society and patient groups. Such alignment is deemed by most interviewees to be necessary for change to occur.
Cancer agencies are the main partners and the Partnership reflects some or all of their priorities, and as such also jurisdictional priorities. The CPAC and Provincial Cancer Agency/Program Council, as well as Canadian Association of Provincial Cancer Agencies (CAPCA) meetings, are mechanisms for sharing and understanding each other's priorities. Some cancer agencies noted that they may not be as involved as other cancer agencies in Partnership initiatives. They suggested that this should not be viewed as non-alignment or lack of support, but rather a reflection of the realities of resource availability and allocation at the jurisdictional level.
National charities and NGOs are also key partners, and the Partnership reflects their mandates and priorities such as prevention and screening, and person-centred care. Although mainly aligned, some feel the Partnership sometimes steps into areas where NGOs are already present (e.g., NGOs such as the Canadian Cancer Society are heavily involved in public engagement).
The First Nations, Inuit and Métis Action PlanFootnote 19 is well aligned with the priorities of the National Aboriginal Organizations who were partners in creating the plan and now are members of the Partnership's First Nations, Inuit and Métis Advisory Committee. The priorities of patient groups also are important as the Partnership brings a person-centred perspective and patient view to its advisory groups and initiatives. However, progress in this area is less advanced than in some others.
4.3 Relevance: Issue #3 - Alignment with Federal Roles and Responsibilities
Findings
The Partnership's mandate is aligned with federal roles and responsibilities. In particular, there is a connection with the federal roles related to "supporting health care innovation and collaboration across jurisdictions" and to chronic disease prevention and mitigation. All key informants voiced their expectation that the federal government would have ongoing roles and responsibilities related to the cancer control system and the CSCC. They also indicated that the Partnership and its mandate were appropriate and supportive of the federal roles and responsibilities. For example, an arms-length organization such as the Partnership provides value to Health Canada, allowing it to play its role in the federated health care system and support multi-jurisdictional initiatives.
The Partnership's mandate is also aligned with and supports the roles and responsibilities of its partners and stakeholders. Some key informants emphasized that alignment depended upon the Partnership remaining true to its model and limiting its scope where others are already involved in the cancer control continuum. Some others suggested a more active leadership role for the Partnership around specific issues.
4.3.1 Alignment of Partnership's Mandate with Federal Roles and Responsibilities
The Partnership's mandate is aligned with federal roles and responsibilities. A federal role is expressed in Health Canada's 2013-2014 DPR, in the context of the department's priority to "Promote Health System Innovation". As illustrated by the underlined text (underlining done by evaluation team), the connection to the Partnership's mandate is through "supporting health care innovation and collaboration across jurisdictions".
The health care system is vital to addressing the health needs of Canadians. Although health care delivery is primarily under provincial jurisdiction, the federal government has an ongoing role in providing financial support for provincial and territorial health insurance plans, maintaining the core principles of the Canada Health Act, and supporting health care innovation and collaboration across jurisdictions. Health Canada can contribute to improving the quality and sustainability of health care as the system continues to evolve in a context of technological change, demographic shifts and fiscal pressures.Footnote 20
The Partnership's mandate is also aligned with roles and responsibilities of the Public Health Agency of Canada. PHAC's Chronic (non-communicable) Disease and Injury Prevention Sub-Program has the goal that "policies and programs support healthy living, decrease chronic disease rates and reduce the impact of these diseases on Canada's population".Footnote 21 Further, PHAC identifies and disseminates best practices in chronic disease prevention and mitigation, and facilitates collaboration among stakeholders to increase the efficiency and effectiveness of chronic disease prevention and mitigation.
There was widespread agreement among key informants about an expectation of a federal role given the federated model of health care, the need for coordination, the need for leadership to promote quality and access to health care services (universality), the need for federal funding to help implement the CSCC, national level reporting and working with First Nations, Inuit and Métis communities and organizations.
Further, key informants saw the Partnership's mandate as being appropriate and supportive of the federal role. Reasons given included:
- It allows the federal government to be involved while not directly managing healthcare and while exercising its other responsibilities such as regulation;
- It cuts across partners and stakeholders at the pan-Canadian level, and brings them together in networks and for specific initiatives;
- It supports multi-jurisdictional initiatives such as pan-Canadian standards and consistency and quality of treatment (note that on this point, a provincial key informant cautioned that the Partnership needs to avoid raising expectations as to what provinces will provide); and
- It fulfills the important brokerage role.
4.3.2 Alignment of Partnership's Mandate with Partners' Roles and Responsibilities
In addition to the core evaluation issue concerning alignment of the Partnership's mandate with federal roles and responsibilities, the evaluation also gathered information about alignment with the roles and responsibilities of other key partners.
Overall, most key informants, both internal and external, felt that the Partnership's mandate is aligned with and supports the roles and responsibilities of its partners and stakeholders. This alignment is supported by the Partnership's model of convening, integrating, catalyzing and brokering model, as well as the CSCC and the Partnership's strategic plans, which help determine what the Partnership does and does not do. For example, the fact that health care delivery is primarily under provincial jurisdiction means the Partnership must work through jurisdictional partners (e.g., ministries, cancer agencies) when initiatives touch upon these provincial roles and responsibilities.
In this context, as noted earlier, some key informants emphasized that alignment in an everyday practical sense depends upon the Partnership staying true to its model, and limiting its scope where others are already involved. Some others thought that the Partnership would provide benefits if it took on a more active leadership role around specific issues, that would go beyond presenting evidence to formulating questions to be discussed regarding an issue, convening the discussion about the questions/issue, and even advocating for change. Other key informants expressed concern about the Partnership taking on an advocacy function as this would diminish its perceived role as an "honest broker".
4.4 Performance: Issue #4 - Achievement of Expected Outcomes (Effectiveness)
This section presents findings related to the achievement of expected outcomes (as per the logic model). The first eight sub-sections walk through each of the immediate outcomes in the logic model for the CSCC. In each of these sub-sections, findings are provided concerning progress and the Partnership's contribution to the achievement of results. The findings are followed by a more detailed description of actions and progress for the initiatives primarily contributing to the immediate outcome.
4.4.1 Immediate Outcome - Improved Access to Evidence-Based Prevention Strategies and Improved the Quality of, and Participation in, Screening
Findings
Through initiatives such as Coalitions Linking Action and Science for Prevention (CLASP), Population-Based Screening, Healthy Public Policy and CAREX Canada, all begun or with roots in the Partnership's first mandate, progress has continued in the second mandate on improving access to evidence-based prevention strategies and improving quality of and participation in screening.
To date, CLASP projects have already exceeded targets set for 2017 in terms of the total population engaged, the number of locales engaged, and the number of locales adopting evidence-informed prevention policies and evidence-informed prevention practices. Types of changes included patient screening and development of new or amended municipal bylaws. Immediate impacts were felt at local levels, and particularly for rural and remote locations. The Partnership's contribution to the CLASP projects through its initial launch of the CLASP initiative and its funding and other support were acknowledged. Such multi-jurisdictional projects would not likely have happened without the Partnership's leadership. At the same time, issues of sustainability of the formal coalitions and ability to monitor the impacts of policy and practice changes into the future were highlighted due to funding pressures.
Screening has been a success story for the Partnership and its partners. During this mandate, the Partnership's screening initiative expanded beyond cervical and colorectal screening, to also include breast and lung cancer. The screening networks have been effective in sharing information and best practices, which has led to the spread of screening programs and increased screening rates and quality on a pan-Canadian basis. Screening rates for these targeted cancers are at or moving towards the targets set by the screening networks.
Performance indicators related to screening, as reported in the annual System Performance Reports prepared by the Partnership, have allowed jurisdictions to compare their approaches to cancer control to other jurisdictions, and balance their investments between screening and other parts of the cancer continuum. The Partnership's Cancer Risk Management Model (CRMM) has enabled analysis of the broader impacts of screening upon outcomes related to cancer incidence, cancer mortality, quality of life and socio-economic factors. Furthermore, the Partnership's network approach and work program are considered to have greatly accelerated the implementation of colorectal cancer screening, compared to earlier cervical and breast cancer screening programs. Challenges are considered to still exist in screening of specific population segments such as new immigrants, those living in rural and remote areas, those facing socio-economic challenges and First Nations, Inuit and Métis. The Partnership's upcoming workshop, in September 2015, on Screening in Underserved Populations to Expand Reach is a step towards addressing this issue.
Relationship to the 2012-2017 Strategic Plan
The activities and outputs undertaken in the coordinated strategy for cancer control related to this immediate outcome are closely tied to the following strategic priority (in the 2012-2017 Strategic Plan) and initiatives.
Strategic Priority
Develop high-impact, population-based prevention and cancer screening approaches.
Initiative(s)
- Coalitions Linking Action and Science for Prevention (CLASP)
- Population-Based Screening
- Healthy Public Policy
- CAREX Canada
Coalitions Linking Action and Science for Prevention (CLASP)
CLASP have been aimed at improving "the health of communities and Canadians by bringing together organizations from two or more provinces and territories to form research, practice and policy coalitions to address common risk factors for cancer and other chronic diseases".Footnote 22 The coalitions integrate lessons from science with those from practice and policy. The first phase of CLASP (CLASP1) took place from 2009 to 2012 during the Partnership's first mandate. Three of the CLASP1 projects (CLASP1R) were renewed for two years at the start of the Partnership's second mandate. The renewals broadened the reach from five to eleven provinces and territories. As well, the Partnership funded five more CLASP projects (CLASP2) in 2013. These included partners in ten provinces and territories. Funding of the CLASP2 projects is for $10.8 million over 3 years, including a commitment of $1.2 million from the Heart and Stroke Foundation.
As of March 2015, targets set for 2017 for the CLASP projects had been exceeded. For example, the total population engaged in CLASP1 projects was 142,111 compared to a 2017 target of 100,000. The total number of locales engaged in CLASP1 renewals (CLASP1R) was 970 compared to a target of 100. Sixty-one locales had adopted evidence-informed prevention policies compared to a target of 20. And 1,722 locales had adopted evidence-informed prevention practices compared to a target of 700.
A recent evaluation of the CLASP1 renewal projectsFootnote 23 showed that the changes in prevention policies and practices arising from those projects varied depending on the specific activities that catalyzed these changes. They ranged from changes in patient screening to the development of new or amended municipal bylaws, but the volume and breadth of these changes reflected positively on the CLASP model. The evaluation also noted the pan-Canadian reach of and number of locales included in CLASP projects. Further, it validated that the design intent of CLASP, i.e., to integrate actions of coalitions in multiple jurisdictions, was demonstrated in the projects that were implemented. In terms of impacts, the data supported that there are immediate impacts felt at local levels where activities were implemented.
The CLASP1R evaluation also pointed to the sustainability of activities and results as remaining a key and critical question. Although evidence pointed to the sustaining of policy and practice changes, there was a limited ability, due to absence of funding, to continue to monitor these changes, and their resulting impacts in the future.
Many key informants interviewed in the current evaluation also identified the CLASP initiatives as being key to the establishment of evidence-based prevention strategies in chronic disease. They noted that, through the various coalitions established (involving 11 of 13 provinces and territories), jurisdictions now share knowledge and information from research, practice and policy applications and can improve their planning and policies in encouraging the adoption of healthy lifestyles and environments and reducing the risks associated with cancer (e.g., smoking cessation, healthy eating). One CLASP project cited was Healthy Canada by Design, which involved changes to the municipal transportation infrastructure that would positively affect increased physical activity and therefore reduce chronic disease rates.
Population-Based Screening
Cancer screening, that is, checking people for certain cancers before there are any symptoms, helps to save lives. The challenge is "to make sure the right people get the right tests at the rights times and that the screening programs continue to be of the highest possible quality", with high-quality follow-up of people's test results.Footnote 24 The programs of work key to this initiative are: national networks that promote active pan-Canadian engagement and connect stakeholders in breast, cervical, colorectal and lung cancer; screening program evaluation activities that enable regular reporting and analysis of quality indicators; and quality improvement opportunities identified through the evaluation work.
The Partnership was involved in cervical cancer screening and colorectal cancer screening during its first mandate. These initiatives were rolled into the Population-Based Screening Initiative for the second mandate. Work continued with the established integrated pan-Canadian networks for these cancers.
During this second mandate, the Canadian Breast Cancer Screening Network was successfully transitioned from the Public Health Agency of Canada to the Partnership. The Pan-Canadian Lung Cancer Screening Network was created in response to new evidence suggesting screening may be appropriate for a high-risk population. To help guide jurisdictions in their deliberations and discussions, the network developed a lung cancer screening framework for Canada.
Performance data shows progress is being made. Data from the 2012 CCHS shows screening rates for the targeted cancers are at or moving towards the 2017 targets. Screening within guidelines of targeted populations for cervical cancer was 76.7% (2017 target of >80%), for breast cancer was 72.4% (2017 target of >70%) and for colorectal cancer was 44.3% (2017 target of >60%). Twelve national targets related to colorectal and cervical cancers had been agreed upon by all provinces and territories, compared to a 2017 target of 10.
All of the groups interviewed for this evaluation identified screening as a major success story for the Partnership. The interviewees noted that the strengthening of existing networks through the sharing of information and best practices has increased the level of screening overall and the quality of screening through the establishment of various protocols and frameworks (e.g. lung cancer screening framework for Canada). In the past, each province had its own approach to screening (breast, colorectal, cervical). Under this initiative, the Partnership, together with partners and stakeholders, was able to look at evidence in best practices and the current state of cancer screening in Canada and transfer this knowledge back to the pan-Canadian networks. More provinces and territories now participate in screening programs than before with national targets established for core quality performance measurement, providing more access for Canadians to high quality CRC screening programs.
In terms of overall screening, many interviewees identified that work still needs to be done in reaching underserved populations (First Nations, Inuit, Métis, newly arrived immigrants and those residing in rural and remote areas of Canada). It was acknowledged by most interviewees that expansion of screening programs is highly dependent on a jurisdiction's ability to fund and resource these programs.
These findings were further supported by those of a case study on colorectal cancer (CRC) screening and the National Colorectal Cancer Screening Network (NCCSN) prepared for this evaluation (see the Technical Report - Case Studies Report, (to obtain a copy, use the following e-mail"Evaluation Reports HC - Rapports Evaluation@hc-sc.gc.ca")). A summary of findings is given in the case study below.
Case Study - Colorectal Cancer Screening and the National Colorectal Cancer Screening Network
Healthy Public Policy
The Healthy Public Policy Initiative is centred on the Prevention Policies DirectoryFootnote 25 that brings together over 1,500cancer and chronic disease prevention policies from hundreds of Canadian sources in a searchable online tool. This initiative continued from the Partnership's first mandate.
During this mandate, the Partnership continued to target knowledge transfer and exchange activities with Canadian research, practice and policy specialists working on the built environment and, more specifically, active transportation at the municipal level. For example, in 2013-14, building on engagement with the Urban Public Health Network, outreach was conducted to highlight the directory's new municipal content and the collaborative map on active transportation policies. A partnership was established with the Propel Centre for Population Health Impact to conduct a multiple-case study on active transportation policy to inform future active transportation policy work across Canada.
CAREX Canada
Funded by the Partnership, CAREX Canada provides national, provincial and territorial occupational and environmental carcinogen exposure surveillance information and tools. CAREX was among the initial projects launched by the Partnership in its first mandate and work has continued through CAREX Canada. During this mandate, CAREX Canada is focused on a "knowledge mobilization program to make CAREX information available and accessible to Canada's cancer prevention and policy arena".Footnote 26 For example, in 2013-2014, the project focused on developing targeted outreach and knowledge products in partnership with WorkSafe B.C., the Government of Alberta, the Occupational Cancer Research Centre, First Nations Environmental Health Innovation Network, the Institut national de santé publique du Québec and the Nova Scotia Ministry of Health. An online version of eWORK (a tool that allows users to access occupational carcinogen exposure estimates) was launched in January 2014 with training materials and webinars. Other tools include the Emissions Mapping Project and the eRISK tool.
Several key informants for the current evaluation identified the CAREX project as contributing to evidence-based prevention strategies.
4.4.2 Immediate Outcome - More Consistent Actions to Enhanced Quality of Diagnosis and Clinical Care
Findings
Progress has been made on the immediate outcome of more consistent actions to enhanced quality of diagnosis and clinical care. Synoptic reporting and staging, started in the first mandate and continuing into the second mandate, has been the priority initiative in support of this outcome. Targets for 2017 have been or are close to being met, in terms of the number of jurisdictions meeting targets for the percentage of stage data captured, implementing synoptic pathology reporting and implementing synoptic surgery reporting. In the case of surgical synoptic reporting, an evaluation in 2014 found improvements in access, quality and productivity. Throughout this initiative, the Partnership is credited with having had an important and evolving role, as acknowledged by its partners and validated in evaluations of the National Staging Initiative and Synoptic Surgery Initiative. Looking ahead the challenges appear to be to: expand ("scale") the implementation of synoptic reporting within jurisdictions to more institutions, and to more jurisdictions; bring more pathologists and surgeons on board through demonstrating value and providing transition / change management support; and put in place by jurisdictions the funding and resources to move to much more widespread use of an IT intensive approach.
Important work has also taken place with the Canadian Partnership for Quality Radiotherapy on the gathering of evidence in radiotherapy quality assurance and associated accreditation standards (with Accreditation Canada). This followed work in the first mandate on chemotherapy guidelines and accreditation standards.
Relationship to the 2012-2017 Strategic Plan
The activities and outputs undertaken in the coordinated strategy for cancer control related to this immediate outcome are closely tied to the following strategic priority (in the 2012-2017 Strategic Plan) and initiatives.
Strategic Priority
Advance high-quality diagnosis and clinical care.
Initiative(s)
- Embedding Evidence in Care - Synoptic Reporting and Staging
- Emerging Screening and Early Detection
- Quality Implementation Initiative
- Enhancing Canadian Cancer Clinical
Embedding Evidence in Care - Synoptic Reporting and Staging
Cancer "stage" refers to a cancer's extent of disease within the body. Stage data can be used at the individual level to develop treatment plans and at the population level to inform cancer control strategies through analysis of factors related to screening, diagnostic timeliness and stage-specific evidence-based care. Collaborative stage is the preferred method for cancer surveillance in Canada and allows for detailed analysis of cancer stage.
Synoptic reporting is a "structured, systematic method for preparing and submitting healthcare reports and includes the ability to incorporate evidence-based best practices and scientifically validated data elements to improve data capture, facilitate clinical decision making and lead to improved outcomes."Footnote 27 Its value includes more consistent, complete and timely data collection and greater accessibility of patient information by the healthcare team.
The Partnership collaborated with partners in the cancer control system on staging data and synoptic pathology reporting under the umbrella of the National Staging Initiative during its first mandate. Work also started on synoptic surgery reporting (co-funded with Canada Health Infoway) near the end of the first mandate. Synoptic pathology reporting and synoptic surgery reporting continued to be important in the Partnership's second mandate. A detailed description of the work undertaken in staging and synoptic reporting is contained in the case study prepared for this evaluation and shown in the Technical Report - Case Studies Report, (to obtain a copy, use the following e-mail"Evaluation Reports HC - Rapports Evaluation@hc-sc.gc.ca").
Performance data illustrates the progress that has been made. Currently, six provinces are implementing synoptic pathology reporting, against a target of six by 2017. Four provinces are implementing synoptic surgery reporting, meeting the target for 2017. As of 2012, nine jurisdictions had met targets for staging (i.e., 90% of stage data captured for the four major cancers) compared to five in 2007, six in 2008 and a target of nine for 2017.
Many key informants interviewed for this evaluation identified synoptic reporting and the cancer staging initiative as key elements in achieving an enhanced quality of diagnostic and clinical care. They also noted the dependency on jurisdictional funding for the implementation of synoptic reporting. Their input is captured in the case study findings/conclusions shown in below.
Case Study - Embedding Evidence in Care - Synoptic Reporting and Staging
Emerging Screening and Early Detection
The Lung Cancer Screening Framework for Canada (September 2014)Footnote 28, developed by the Pan-Canadian Lung Cancer Screening Network, is an example of work done by the Partnership in this initiative. The Partnership's Vice-President of Cancer Control chairs the network. Biannually, the Partnership also prepares an environmental scan of lung cancer screening guidelinesFootnote 29 across Canada, which compares current guidelines and evidence-based recommendations.
Quality Implementation Initiative
The Quality Implementation Initiative uses evidence and system performance measures to inform coordinated action to enhance quality in patient care. The goal of the initiative is to build on the foundation laid in the first mandate to leverage a national approach to identifying gaps in quality, synthesize the evidence about the gaps and identify potential quality initiatives. The Partnership is working with and funding the Canadian Partnership for Quality Radiotherapy (CPQR) to promote the gathering of evidence in radiotherapy quality assurance. This followed earlier work in the first mandate on the development of chemotherapy guidelines. The Partnership was involved with Accreditation Canada in both of these initiatives. Accreditation Canada spoke positively about their experience in working with the Partnership, and noted the access that the Partnership provided to key stakeholders in the cancer control system. The Partnership was involved with the Quality Initiative in Interpretive Pathology (QIIP) to develop and implement a national framework. The Partnership is also supporting four multi-jurisdictional projects, chosen through a request for proposals process and started in 2014-15, to implement quality initiatives across Canada.Footnote 30
Enhancing Canadian Cancer Clinical Trials
Clinical trials are a key step in evaluating new approaches to cancer treatment, palliation and support that lead to improved patient outcomes, including increased survival and improved quality of life. The Partnership has invested in a pan-Canadian approach to strengthen Canada's ability to conduct practice-changing clinical trials and improve cancer outcomes for Canadians.
The Canadian Cancer Clinical Trials Network (3CTN) is a pan-Canadian initiative to improve the efficiency and quality of clinical trials in Canada. 3CTN provides support and coordination for a network of teams at cancer treatment centres and hospitals. With regional participation, 3CTN enables sites to increase their capacity and capability to conduct academic trials.Footnote 31 The Ontario Institute for Cancer Research (OICR) was identified as the host of the 3CTN through a competitive process that was funded by the Partnership. The Partnership now supports the OICR's 3CTN work as one of a consortium of funders.Footnote 32
Other - Tobacco Control and Cancer Control Initiative
In late 2014, the Partnership established a new initiative to support better integration of tobacco control and cancer control resources across Canada. The initiative will:
- Convene experts to improve and expand smoking cessation programs within cancer diagnosis and treatment settings;
- Explore expanding smoking cessation services into other cancer care service areas, such as cancer screening; and
- Explore opportunities for collective action on emerging tobacco control issues, such as e-cigarettes.Footnote 33
Some federal key informants pointed to the Partnership's already established scans of the success of smoking cessation programs and the distribution of the data as an example of supporting the enhancement of diagnosis and clinical care. By demonstrating the overall impact of various smoking cessation programs, including those for cancer patients, provinces and organizations can now look to best practices in this area and improve the success rates of their own programs.
4.4.3 Immediate Outcome - Improved Capacity to Respond to Patient Needs
Findings
Progress has been made on the immediate outcome of improved capacity to respond to patient needs. Projects have been launched related to: patient experience and outcomes; survivorship; palliative and end-of-life care; and primary and cancer care integration. However, it is acknowledged by the Partnership and its partners that much of this work is still in its early days and much work remains to be done before significant impacts will be realized. The Partnership has supported this work through funded projects related to different parts of the cancer journey (as noted above), as well as performance indicator work and patient representation on its working and advisory groups that are intended to embed a person-centred perspective into the Partnership's initiatives across the cancer control continuum. For example, eight jurisdictions are now using a common set of indicators to measure patient-reported experience and patient-reported outcomes. This is considered to be an important step in helping to ensure Canadians have equal access to consistent standards of cancer care.
Capacity is seen as having many dimensions (e.g., knowledge, tools, resources, funding, human resources and technology) and there are vast differences in capacity across provinces and territories. Success is considered to be highly dependent upon the resources available in jurisdictions to effect the changes required to reflect a patient-centred perspective. Projects being supported also vary across the country so new networks such as the Palliative and End-of-Life National Network are promising for the sharing of best practices and information. The use of narrative stories that reflect the face of cancer (e.g., The Truth of It) are also considered important to remind those in the community why they are all involved in cancer care from the patients' viewpoint.
Relationship to the 2012-2017 Strategic Plan
The activities and outputs undertaken in the coordinated strategy for cancer control related to this immediate outcome are closely tied to the following strategic priority (in the 2012-2017 Strategic Plan) and initiatives.
Strategic Priority
Embed a person-centred perspective throughout the cancer journey.
Initiative(s)
- Patient Experience and Outcomes
- Survivorship
- Palliative and End-of-Life Care
- Primary and Cancer Care Integration
Patient Experience and Outcomes
A hallmark of quality cancer care is the provision of person-centred care that is respectful of and responsive to individual patients' preferences, needs and values. The goal of the Patient Experience and Outcomes Initiative is to improve the patient experience across the cancer journey through standardized measurement that accelerates optimal care and measures impact (health-related outcomes for patients) across Canada. The aim is for all participating jurisdictions to establish a measurement and reporting cycle for patient experience that facilitates action to guide improvement through use of standardized screening and assessment tools and implementation of programmatic interventions.Footnote 34
During this mandate, a national steering committee was established in 2013-2014 and subsequent work was conducted by the steering committee and through other stakeholder meetings on information sharing and development of indicators to measure patient experience, with the goals of common approaches and pan-Canadian reporting. Indicators include both patient-reported outcome measures (PROMs) and patient-reported experience measures (PREMs).
Four projects were approved and launched in 2013-2014 as a result of a request for proposal process.
More detail about the work program and implementation of this initiative is provided in the case study prepared for this evaluation (see the Technical Report - Case Study Report, (to obtain a copy, use the following e-mail"Evaluation Reports HC - Rapports Evaluation@hc-sc.gc.ca").
Progress on this initiative is demonstrated by performance indicator data. As of midway through 2015, eight jurisdictions were using a common set of indicators to measure patient experience and outcomes, which meets the 2017 target of eight. This compares to no jurisdictions in 2007. Eight jurisdictions were using a standardized symptom screening tool for at least a portion of patients, compared to the 2017 target of ten.
Key informant input and summary findings for this initiative are expressed in the findings/conclusions section of the case study, which is provided below. It should be noted that many of the comments relate to the person-centred perspective in a broader sense, and therefore apply to other of the initiatives related to this outcome.
Case Study - Patient Experience and Patient Reported Outcomes
Survivorship
In Canada, there are currently more than a million people living as cancer survivors. With further advances in prevention, screening and treatment, as well as the expected increase in the number of cancer diagnoses, the total number of cancer survivors is expected to increase in the future.Footnote 35 The Canadian health care system will see a higher number of people living with survivorship issues. It is therefore critical to understand the unique needs of this growing population. In the Partnership's first mandate, The Cancer Journey Action Group identified the development of improved approaches to cancer survivorship as a key priority.Footnote 36
Two multi-jurisdictional projects focused on survivorship care are being supported by the Partnership.
Palliative and End-of-Life Care
Given that many patients will still eventually die from their cancer, it is important to consider palliative and end-of-life care as an integral part of a cancer control strategy. The Palliative and End-of-Life Care Initiative supports multi-jurisdictional efforts aimed at improvements in professional education, early introduction of palliative and end-of-life care, system integration, incorporation of patients' and families' expressed wishes for place of death, advanced care planning, improvement in rural and remote access, and First Nations, Inuit and Métis engagement.Footnote 37
There are five projects being supported through this initiative.
Primary and Cancer Care Integration
Demand for cancer and chronic disease health care services is increasing due to the growing and aging population. With advances in cancer screening and treatment, more people are living with a diagnosis of cancer for longer periods of time. While this is a good thing, it can also be a challenge. The need to maximize the quality of cancer and chronic disease care while ensuring long-term sustainability of provincially-based health care systems is essential. Created in collaboration with the College of Family Physicians of Canada and the Canadian Association of Provincial Cancer Agencies, the goal of the Primary and Cancer Care Integration Initiative is to investigate leading practices that improve transitions of care between cancer specialists and primary care settings, and ultimately improve the patient experience, and enhance the efficiency and the quality of care of the primary care and cancer care systems. The focus of these projects is the period after primary cancer treatment is complete and follow-up care has begun.Footnote 38
Three projects are being supported through this initiative. The projects relate to dialogue and story work in support of First Nations, Inuit and Métis cancer patients throughout their oncology and primary care transition experience (Northwest Territories, British Columbia, Alberta and Manitoba); leveraging a suite of existing tools to support patients and health care professionals in the post-treatment period (British Columbia, Yukon, Nova Scotia); and improving clinical, functional and vertical integration for providers of cancer care (Ontario, Manitoba, British Columbia).
4.4.4 Immediate Outcome - Enhanced Co-ordination of Cancer Research and Improved Population Research Capacity
Findings
Substantial progress has been made on enhanced co-ordination of cancer research through the Canadian Cancer Research Alliance (CCRA) and on improved population-health research capacity through the Canadian Partnership for Tomorrow Project (CPTP).
The CCRA has been guided by its Pan-Canadian Cancer Research Strategy, 2010-2014. A new strategy is being developed for launch in 2015. During the previous strategy, key accomplishments included the biennial Canadian Cancer Research Conferences, first held in 2011, and the first Pan-Canadian Framework for cancer prevention research, which spawned increased focus on and inclusion of prevention research in the overall funding environment. The Partnership has contributed to the CCRA through its support for the CCRA's executive office, its membership in the CCRA and several projects included in the research strategy, namely the CPTP and the Canadian Cancer Clinical Trials Network (see section 4.4.2). Key informant input on the impacts on the coordination of cancer research varied. For example, interviewees from cancer agencies identified only a modest level of impact, while those from NGOs indicated that coordination had improved.
Input from some key informants suggests lack of clarity concerning the extent to which the Partnership coordinates cancer research itself versus the Partnership supporting and being a member of the CCRA, which is the coordinating body.
The CPTP is a major success story for the Partnership and its partners. The Partnership is credited with bringing together a harmonized project, from what would instead have been five separate cohort projects. Reach was extended into the cardiovascular research community in partnership with the Heart and Stroke Foundation. Participation targets have been met and the CPTP platform has been tested and launched.
Looking ahead, the CPTP is seen as an important source for researchers examining the causes of cancer and other related chronic diseases. However, there are issues concerning a long-term home and sustaining funding for the CPTP platform/infrastructure, beyond 2017.
Relationship to the 2012-2017 Strategic Plan
The activities and outputs undertaken in the coordinated strategy for cancer control related to this immediate outcome are closely tied to the following strategic priority (in the 2012-2017 Strategic Plan) and initiatives.
Strategic Priority
Enable targeted research to augment our knowledge and understanding of cancer and related chronic diseases.
Initiative(s)
- Canadian Partnership for Tomorrow Project
- Canadian Cancer Research Alliance
Canadian Partnership for Tomorrow Project
The Canadian Partnership for Tomorrow Project is a longitudinal, long-term cohort involving approximately 300,000 Canadians, ages 35 to 69, who agree to share their health and lifestyle information over their adult lives. The CPTP also provides the infrastructure necessary to make this information available to researchers. In addition to questionnaire data, over 138,000 blood specimens and thousands of additional biological specimens have been gathered and stored for future analysis. By analyzing and comparing the data collected through this project, researchers can identify clues as to why some people develop cancer and other chronic diseases, and others do not, and explore and better understand regional and national patterns and trends. These studies will also help to confirm whether emerging international evidence and information is applicable to Canada's population. The project has resulted in a "population laboratory" of a size and scale not previously seen in Canada. The project is currently made up of five regional cohorts: the BC Generations Project, Alberta's Tomorrow Project, the Ontario Health Study, Quebec's CARTa-GENE and Atlantic PATH.Footnote 39
In 2013-2014, the cardiovascular disease component of the Canadian Partnership for Tomorrow Project was initiated in partnership with the Heart and Stroke Foundation. The Canadian Alliance for Healthy Hearts and Mindsis an important partnership between the cancer and cardiovascular research communities that will strengthen understanding of chronic disease.
Harmonization of core datasets from the regional cohorts was initiated and a data access pilot was successfully completed in March 2014. The CPTP portalFootnote 40 was launched in June 2015, for use by researchers. Researchers can request access to questionnaire data for over 202,000 participants. Other data, for additional participants, physical measures and other types of data, as well as information related to biosamples, is to be made available to researchers later in 2015-2016 and as more data is collected and harmonized. More about the CPTP work program and implementation is contained in the CPTP case study prepared for this evaluation (see the Technical Report - Case Studies Report, (to obtain a copy, use the following e-mail"Evaluation Reports HC - Rapports Evaluation@hc-sc.gc.ca")).
Key informant input and summary findings for this initiative are given in the findings/conclusions section of the case study, which is provided below.
Case Study - The Canadian Partnership for Tomorrow Project
Canadian Cancer Research Alliance
The Canadian Cancer Research Alliance (CCRA) is "an alliance of organizations that collectively fund most of the cancer research conducted in Canada - research that will lead to better ways to prevent, diagnose, and treat cancer and improve survivor outcomes".Footnote 41 Members include federal research funding programs and agencies, provincial cancer care agencies, cancer charities and other voluntary associations. It is intended that, through effective collaboration, Canadian cancer research funding organizations can "maximize their collective impact on cancer control and accelerate discovery for the ultimate benefit of Canadians affected by cancer".Footnote 42
During the period covered by this evaluation, CCRA was guided by the Pan-Canadian Cancer Research Strategy, 2010-2014. In the final report (March 2015) for the research strategyFootnote 43, the Partnership is credited with having supported the executive office of the CCRA as well as being a CCRA member organization and having supported several projects included in the research strategy, namely the CPTP and the Canadian Cancer Clinical Trials Network.
Some of the initiatives considered of note in the final report are:
- The biennial Canadian Cancer Research Conference, first held in 2011 and then attended by over 1000 cancer researchers in 2013. Another is planned for Montreal in November 2015.
- Two multi-million dollar Canadian cancer genome sequencing projects in prostate and pancreatic cancer as Canada's contribution to the International Cancer Genome Consortium (note that the CCRA coordinated the discussion for these projects but did not fund them).
- The first Pan-Canadian Framework for cancer prevention research, which in turn spawned new collaborative funding for cancer prevention grants and capacity building awards, as well as an increased focus on, and inclusion of, prevention within the Canadian cancer research funding environment.
- The 3CTN mentioned above.
A second pan-Canadian cancer research strategy is being developed for launch in 2015.Footnote 44
Among key informants, there was a great range of opinions expressed in terms of the Partnership's contribution to enhanced coordination of research and apparent confusion on the role that the Partnership plays in this area.Footnote 45 Representatives from both Health Canada and the Public Health Agency of Canada did not see why the Partnership is involved with the coordination of cancer research (i.e., they viewed it more as the mandate of CIHR). CIHR interviewees viewed the Partnership's involvement in research coordination through the creation of the CCRA as a great success. It was noted that CIHR deliberately passed the role of research coordination to the Partnership to avoid the perception within the cancer community that, through the sheer volume of funding, CIHR could dictate the research areas that become priorities in Canadian cancer research. An example provided of the coordination role played by the Partnership and CCRA is in the shortage of radio-nucleotides (isotopes) that Canada faced a few years ago. With CIHR as the lead and chair, and through CCRA and Partnership participation, alternate sources for these nucleotides were found much faster and at a lower cost than if provinces and territories had attempted to do it themselves.
Cancer agencies identified only a modest level of impact on the coordination of cancer research. They provided no specific examples of the impact that the Partnership and CCRA have had on the overall coordination of cancer research in Canada today.
NGOs expressed their opinion that the coordination of cancer research has improved with CCRA bringing various funding and research agencies together to discuss and identify areas of cancer research and avoid duplication of efforts.
Interviewees representing various initiatives contributing to this outcome indicated a number of real benefits to standardization and knowledge sharing in both staging and screening and noted the importance of bringing together stakeholders to address and understand cancer risk. One interviewee pointed to the fact that this is an ongoing outcome; i.e., it will continue to be important in ongoing work given that data is beginning to be taken up by the research community in a coordinated way in order to produce new evidence for policy and programming. Another interviewee pointed to the usefulness of the Analytic Capacity Initiative to support capacity development in the research community to appropriately and effectively use the data. Another key informant identified unintended, spin-off benefits resulting from their work; specifically that they were able to leverage research findings to partner with other organizations to expand the use of data in other related areas of inquiry; i.e., using environmental data to consider children's health and working with First Nations, Inuit and Métis organizations to produce environmental health-related information.
4.4.5 Immediate Outcome - Improved First Nations, Inuit and Métis Cancer Control with and for First Nations, Inuit and Métis Peoples
Findings
The development of the First Nations, Inuit and Métis Action Plan on Cancer Control is viewed as a significant first step in advancing cancer control for First Nations, Inuit and Métis communities.
Responding to the major concern of access to health care services in remote and rural areas, in 2015, eight jurisdictions were participating in and implementing initiatives to improve the continuity of care in remote and rural locations. Further, cancer agencies or their equivalents in three jurisdictions (Northern British Columbia, Manitoba and Ontario) had in place First Nations, Inuit and Métis specific cancer control strategies, compared to one since the Partnership was first initiated.
The delivery of the Saint Elizabeth Cancer Course has increased awareness among healthcare providers of culturally relevant approaches to cancer control and increased awareness amongst First Nations service providers, of cancer control issues and challenges.
Most key informants interviewed for the evaluation at large acknowledged the significant role that the Partnership has played in establishing the baselines and the Action Plan. It was noted that the process used involved a great deal of consultation and engagement of First Nations, Inuit and Métis). It is considered to have worked well and could be a useful model for federal departments and agencies to employ in their other interactions with Aboriginal groups. The Partnership's representatives identified that there is now increased collaboration between cancer control stakeholders in terms of programming for First Nations, Inuit and Métis without the facilitation or involvement of the Partnership. Health Canada representatives identified the unanticipated benefit of now having better relations with First Nations and Inuit groups as a result of the consultation and engagement process used by the Partnership.
It was acknowledged by interviewees that it is still premature to assess the extent of the success in advancing cancer control within First Nations, Inuit and Métis communities as the implementation of the initiatives is still in the early stages. It was also recognized that the success of the initiatives will depend on the appropriate level of funding and resources being made available from the respective jurisdictions (federal, provincial, territorial levels). This led federal representatives to question whether there is an on-going role the Partnership to play with respect to Aboriginal groups in that moving ahead with the Action Plan will require efforts on the part of the federal, provincial and territorial governments as well as their funding and resourcing of activities and programs within the Action Plan.
An on-going challenge is the capacity (human and financial) within the NAOs that has made it challenging for ITK, MNC and, to a lesser extent, AFN to be as engaged as they would like. While health funding has remained stable, all three organizations have seen other sources of federal funding decreased (in the case of ITK and AFN) or eliminated (in the case of MNC) in recent years. Representatives from all three NAOs noted that the Partnership has been very supportive and has provided financial resources in order to make it possible for the NAOs to stay engaged and fully contribute to the implementation of the Action Plan. Notably, capacity is also a challenge for Nunavut, which is not actively participating in CPAC funded First Nations, Inuit and Métis initiatives. ITK and the Partnership have made efforts to keep Nunavut engaged until they are able to find the resources to become more involved.
Relationship to the 2012-2017 Strategic Plan
The activities and outputs undertaken in the coordinated strategy for cancer control related to this immediate outcome are closely tied to the following strategic priority (in the 2012-2017 Strategic Plan) and initiatives.
Strategic Priority
Advance cancer control with and for First Nations, Inuit and Métis peoples and partners.
Initiative(s)
- First Nations, Inuit and Métis Action Plan on Cancer Control
First Nations, Inuit and Métis Action Plan on Cancer Control
The Partnership's focus on working with and for Canada's First Nations, Inuit and Métis peoples is a recognition of cancer rates and issues for First Nations, Inuit and Métis people and communities. As stated in the Partnership's 2011-2012 Annual Report:
Cancer rates among Canada's First Nations, Inuit and Métis peoples are increasing faster than overall Canadian cancer rates, yet at the community level there remains a gap in awareness about cancer and its causes. The need for culturally relevant educational materials and expertise contributes to the challenge of disease awareness, prevention and care. In addition, broader determinants of health, including factors such as geography and access to health services, must be considered in the development and implementation of strategic initiatives to improve cancer control.Footnote 46
The Partnership hosted a National Forum on First Nations, Inuit and Métis Cancer Control in 2009. Specific challenges identified by the Forum were related to: system integration relative to the practical and cultural needs of First Nations, Inuit and Métis populations; primary prevention including levels of awareness about cancer and cancer risk factors, and addressing socio-economic determinants of health; organized and systematic screening programs that are relevant to the needs of First Nations, Inuit and Métis populations; support through the cancer journey that integrates traditional practices and is culturally appropriate; health human resources in the communities; and cancer control data specific to First Nations, Inuit and Métis populations.
In 2010-2011, the First Nations, Inuit and Métis Action Plan on Cancer Control was developed to address these issues and build on work already being done by First Nations, Inuit and Métis communities, governments and the cancer control community across Canada. The Advisory Committee for the plan included: National Aboriginal Organizations; Public Health Agency of Canada; Health Canada's First Nations and Inuit Health Branch (FNIHB); Canadian Cancer Society; and Canadian Association of Provincial Cancer Agencies. The Assembly of First Nations, the Inuit Tapiriit Kanatami, and the Métis National Council endorsed the plan. It was released in June 2011.
The Action Plan focuses on four distinct areas:
- Community-based health human resource skills and capacity and community awareness;
- Culturally responsive resources and services;
- First Nations, Inuit and Métis patient identification; and
- Access to care in remote and rural communities.
More about the development and early implementation of the Action Plan during the Partnership's first mandate is described in the case study presented in the Technical Report - Case Studies Report, (to obtain a copy, use the following e-mail"Evaluation Reports HC - Rapports Evaluation@hc-sc.gc.ca").
During this second mandate, First Nations, Inuit and Métis Cancer Control baseline documents were completed and shared with stakeholders in 2013-2014. As well, the Métis Cancer Pathways Environmental Scans were completed and shared with stakeholders and partners. Multi-jurisdictional networks and working groups were convened starting in 2013-2014, with the aim of supporting knowledge exchange and collaboration.
Additional Partnership activities in 2014-2015 included convening a "cross-functional group" supporting multiple themes, including prevention, person-centred perspective and screening, with respect to quality engagement and relationship development with First Peoples and a "traditional health supports planning group to advance specific lines of work". An evaluation metrics working group was also convened.
In 2013-2014, the Partnership, issued a request for proposals to the jurisdictions for initiatives intended to respond to needs identified in the Action Plan focused on the cancer journey. In order to be considered for funding, jurisdictions were required to partner with First Nations, Inuit and Métis partners. Ten jurisdictions submitted proposals, nine initiatives were funded and eight are currently underway. The eight active projects are in the Yukon Territory, Northwest Territories, British Columbia, Alberta, Saskatchewan, Manitoba, Quebec, and Newfoundland and Labrador.
4.4.6 Immediate Outcome - Improved Analysis and Reporting on Cancer System Performance
Findings
Improved analysis and reporting on cancer system performance was thought by most interviewees to be an area where the Partnership has already achieved the outcome. Both "annual" and "special" cancer system performance reports have been produced. A set of core indicators has been identified and most jurisdictions are participating. Some of the special reports have delved into shedding light on variations in performance (e.g., What is the explanation for differences in performance across jurisdictions? - the "so what").
The Partnership was viewed by most interviewees as having done an exemplary job of introducing the reports and adapting them to respond to additional deeper analysis as needed (e.g. special studies). The pan-Canadian approach brought visibility to cancer system performance reporting and enabled smaller provinces to engage, thereby helping to establish comparable indicators and data across Canada.
Cancer agencies identified the use of the data in their own planning and operations and the fact that this level of reporting could only be achieved through a pan-Canadian approach. They also noted the use of the data and information as a tool for leveraging funding from their own provinces (motivational factor for those at the bottom of the rankings). For example, as was also noted in the case study for colorectal cancer screening, a jurisdictional key informant described how the multi-jurisdictional data on screening rates from the cancer system performance reports, plus the Partnership's work on the net benefits of screening, led jurisdictional decision-makers to fund more screening, when they had initially focused upon funding for treatment.
A recent (July 2015) evaluation of the System Performance Reporting Initiative provided additional examples of cases where the cancer system performance reports have had impacts upon jurisdictional capacity related to system performance, improved clinical practice, improved program or service planning and/or delivery, improved policy or legislation or funding, and increased public awareness.
Work to develop indicators around person-centred care and patient experience and outcomes is supported by key stakeholders. Further cancer organizations expressed the need to clearly demonstrate concrete evidence of the impacts of investments made in cancer care (value and what the investments are actually achieving in terms of making the cancer care system sustainable).
The main challenge identified for provinces and territories is the funding and resources necessary for data collection in support of the System Performance Reporting Initiative. The information technology (IT) infrastructure and level of connectivity varies greatly across the jurisdictions. The levels of investment required to capture the data and put in place the associated electronic infrastructure, as well as the human resources to implement the necessary changes, are highly dependent on the level of funding available and the various health care system priorities established in each province or territory.
Looking ahead, as cancer system performance reporting continues to mature and as jurisdictions meet the targets that have been set for indicators, there will be a need, as expressed by some key informants, to decide upon the ongoing purpose(s) of cancer system performance reporting. Will the purpose be to measure where the system is today and highlight gaps between actual performance and the targets that have been set? This means a focus on target-setting in order to make the current system work better. Or will the purpose be to use indicators to drive system change in areas where needs/issues have been identified? This means a focus on introducing new indicators that would help drive and measure perhaps more transformative change. Such indicators could be system, practitioner or person-focused. Or will it be a mix of the two purposes, which appears to be the case today?
Relationship to the 2012-2017 Strategic Plan
The activities and outputs undertaken in the coordinated strategy for cancer control related to these immediate outcomes are closely tied to the following core enabling function (in the 2012-2017 Strategic Plan) and initiatives.
Core Enabling Function
System performance analysis and reporting.
Initiative(s)
- System Performance Reporting
System Performance Reporting
The Partnership's System Performance Reporting Initiative has been:
...a national effort to identify the aspects of the cancer control system that need to be measured, define and collect valid and comparable data needed for the measurement, and present results in an integrated report that allows for synthesis of results and interpretation of patterns in a manner designed to inform quality improvement strategies.Footnote 47
National level collaboration on system performance reporting supports evidence-based planning, management and policy development through sharing of best practices and increased understanding of how Canada's performance compares to that of other developed countries.
The first annual cancer system performance report was published in 2009. During the Partnership's first mandate two subsequent annual and more focused reports were released in 2010 and 2011. Through work on, release of and feedback on these reports, progress was made on development of the appropriate set of core performance indicators, consistent and robust data sources and collection, and reporting approaches.
During the Partnership's current mandate, annual cancer system performance reports have been published in 2012, 2014 and 2015, and one is planned for summer 2016. By the 2015 report, it was agreed that the focus would be on a group of 17 "dashboard" indicators, from across the continuum of cancer control. The dashboard indicators are described as "a group of well-established metrics with a strong evidence base that address important aspects of key cancer control domains... [they] can help determine clear pathways for reducing the burden of disease and improving care for Canadians".Footnote 48 The 2015 annual cancer system performance report also contained more detailed information on three special features. In addition to the annual reports, during the second mandate, five special focus reports (or "spotlight" reports) and three special studies were produced. Two spotlight reports are scheduled for release in 2015-2016.
A system performance web applicationFootnote 49 was launched in June 2014 to provide access to the system performance reports, as well as downloadable content and a dashboard with cancer control domain indicators, disease site indicators and provincial/territorial indicators (not yet implemented).
By 2015, substantial progress had been made as measured by the performance indicators related to this core enabling function (and immediate outcomes). Ten jurisdictions were participating in the development of the core indicators (compared to a 2017 target of 13). Eight jurisdictions had taken at least one system level action based on the system performance reporting tools (compared to a 2017 target of 13). Almost 60% of the core indicators had consensus-based targets developed that were being used in reporting (compared to a 2017 target of 100%), and 13 indicators had been developed in under-measured domains such as system efficiency, person centred care, end-of-life care, and research (compared to a 2017 target of 20).
More about the development, implementation and outputs from this initiative are described in the case study presented in the Technical Report - Case Study Report, (to obtain a copy, use the following e-mail"Evaluation Reports HC - Rapports Evaluation@hc-sc.gc.ca").
4.4.7 Immediate Outcome - Enhanced Access to High-Quality Information, Knowledge, Tools and Resources
Findings
Progress continues to be made on the immediate outcome of enhanced access to high-quality information, knowledge, tools and resources. Contributions are being made from players across the cancer control community, for example, through the types of materials presented on the websites of provincial cancer agencies, cancer NGOs, and others.
The Partnership has contributed to increased access through its own web presence - through cancerview.ca, and its corporate site partnershipagainstcancer.ca which is an entry point to various parts of cancerview.ca, and through its support for websites and portals tied to specific initiatives such as the CPTP and system performance reporting.
The Partnership has gone beyond simply enhancing access. It has built tools - e.g., Cancer Risk Management Model (CRMM) - and worked on gathering and organizing information - e.g., through its internal Evidence, Synthesis, Guidelines Initiative, and though Co-ordinated Data Development. It has also worked on building capacity to not only access but transfer and use/analyze the information - e.g., through its Partnership Knowledge Transfer and Adoption Initiative, its Analytic Capacity Building and the collaborative work spaces on cancerview.ca
As more of the initiatives supported by the Partnership in other areas (e.g., screening) mature, there is more demand on sharing the tools, knowledge and resources that they produced. The need for knowledge transfer and exchange then becomes greater.
Relationship to the 2012-2017 Strategic Plan
The activities and outputs undertaken in the coordinated strategy for cancer control related to this immediate outcome are closely tied to the following core enabling function (in the 2012-2017 Strategic Plan) and initiatives.
Core Enabling Function
Knowledge management through tools, technology, connections and resources.
Initiative(s)
- Cancerview.ca
- Cancer Risk Management Modelling and Economic Analysis
- Partnership Knowledge Transfer and Adoption
- Analytic Capacity Building and Co-ordinated Data Development
- Evidence, Synthesis, Guidelines
Cancerview.ca
Launched in 2009, cancerview.ca is a knowledge hub and online community that offers trusted, evidence-based content and a wide array of tools and resources. It provides specialized platforms that support planning and collaboration in cancer control across jurisdictions to guide decision-making at the clinical, management and policy levels. The website also provides professionals working in cancer control with timely access to trusted information and evidence from a variety of partner organizations across Canada. As a platform for virtual collaboration, it allows experts and colleagues from across Canada to easily connect and work together regardless of their geographic location.Footnote 50
As an example, the work program for 2013-2014Footnote 51 for cancerview.ca included refining and formalizing the content strategy "to target health professionals (including those in primary care), to curate and deliver knowledge products that synthesize and provide expert perspectives on cancer evidence, and to better align cancerview.ca offerings with other strategic priorities". Further refinement of this work took place in 2014-2015. In 2014-2015, eight topics were presented in the 1 in 3 cancerview digest, which had been launched in 2013-2014. Digest topics included "colorectal screening, radon gas exposure in homes and workplaces, smoking cessation, and cancer in adolescents and young adults.Footnote 52
In 2014-2015, cancerview.ca's reach (i.e., number of unique visitors to cancerview.ca including the 1 in 3 blog) was 39,828. There was no data available concerning either usefulness or intent to use the information on the website.
Cancer Risk Management Modelling and Economic Analysis
The Cancer Risk Management Model platform draws on population-based data sources to provide health and economic cancer projections via a web-based platform. The model was developed by the Partnership in collaboration with Statistics Canada and teams of clinicians, health economists and other experts. It is in keeping with the Partnership's mandate to drive system-wide improvements through the synthesis and dissemination of the best available information. The model allows users to project health and economic outcomes of various interventions - whether in prevention, screening, or a new therapy - that would ordinarily take years to realize.Footnote 53
During the current mandate, the Partnership has continued to update and use the CRMM and develop new models. For example, in 2013-2014, enhancements were completed to the cervical, colorectal, HPV and lung models to meet the needs of the pan-Canadian screening networks, and the models were run to inform policy questions in jurisdictions. Work began on development of the breast cancer model and this continued in 2014-2015. The Partnership also reported that model results were increasingly used by partners and stakeholders. Four examples were provided in the 2014-2015 Annual Reports - use of the colorectal, cervical and lung models by the Canadian Cancer Society in its Canadian Cancer Statistics 2015 Special Topics Report; use of the lung model by Alberta Health in its economic evaluation of low-dose computed tomography screening for lung cancer; evaluation of the CRMM by the Public Health Agency of Canada on behalf of the Canadian Task Force on Preventive Health Care to assess the model's use in supporting upcoming screening guidelines; and ongoing use by the lung, colorectal and cervical screening networks to inform decision-making.
Preparation of manuscripts and academic publications took place in 2013-2014 and 2014-2015. In 2013-2014, two foundational papers on the lung model were published, and work was initiated on six more manuscripts. In 2014-2015, four papers focusing on model validation, new lung cancer treatments, lung screening in asbestos-exposed populations and colorectal screening modalities were accepted for publication, and two other papers on the cervical model and the cost-effectiveness of lung screening were developed.Footnote 54
Partnership Knowledge Transfer and Adoption
The Knowledge Transfer and Adoption Initiative supports the coordination of the Partnership's approaches to knowledge transfer, exchange and adoption across the cancer control strategy. It includes the creation of toolkits, resources and evaluation instruments to help Partnership initiatives measure their impact from a knowledge management perspective.Footnote 55
In 2014-2015, work continued on refining the Knowledge Transfer and Exchange (KTE) toolkit, including the standard KTE survey. Use and uptake of the survey tripled in 2014-2015. Ninety-three percent of KTE participants/target audience members (compared to a 2017 target of 75%) reported that the Partnership's KTE activity (product/event) was useful, and 80.1% (compared to a 2017 target of 75%) reported that they intended to use the Partnership's KTE product/information provided.
Many key informants interviewed for this evaluation also generally found the materials and information developed and distributed through cancerview.ca, partnershipagainstcancer.ca and knowledge brokering events to be useful. Federal representatives identified that they use the information and knowledge in their planning and operations within their departments. Cancerview.ca was identified as being the most up to date source of cancer information and data in Canada today. Cancer agencies also acknowledged the high quality of materials produced (in particular the quality assurance documents). However, the cancer agencies reported that use of the material varied, with some stating that they make extensive use and find it very beneficial to their work and others indicating they don't use the information on the websites at all. It was noted by a few interviewees that the real success in this area occurs when the information is being used by Canadians, partners and stakeholders to implement changes, but it was also noted that in a federated model of health care such as Canada's, you can produce and provide access to the information, but it is very difficult to measure the impact of the use and implementation of the information at the provincial levels.
Analytic Capacity Building and Co-ordinated Data Development
The Analytic Capacity Building and Coordinated Data Development Initiative was launched in 2013-2014 to identify and fill key data gaps and to support analytic capacity within Canada. The objectives of this initiative are twofold: to enhance and create new capacity to support analysis and use of evidence to improve the cancer system, and to enable consistent pan-Canadian reporting related to cancer treatment.
The Analytic Capacity Building stream of work is increasing training for both cancer data analysts and cancer data coders, building a pan-Canadian analytic network and increasing the supply of cancer-data trained analysts. As examples of work in this stream, in 2014-2015, a steering committee was established to advise on a strategy for analytic capacity development. As well, projects included: the launch of Analytically Yours, a collaborative online community of practice for the Canadian analyst community; and a joint analytic project on relative survival by stage to train data analysts from nine provinces in survival-by-stage analysis, with associated mentoring and webinars (two webinars attracted over 350 Canadian and international participants).
The Coordinated Data Development Initiative aims to identify a core set of treatment data elements, seek provincial alignment to common data definitions, and test the feasibility of obtaining treatment data or creating linked data sets. After a call for Expressions of Interest, six pilot projects have been funded. All projects focus specifically on addressing gaps in the availability of, or access to, one or more types of treatment data.Footnote 56
Evidence, Synthesis, Guidelines
The body of evidence to inform clinical and policy decisions is rapidly expanding. The Evidence, Synthesis and Guidelines Initiative builds on the Partnership's earlier work to enable stakeholders to use evidence in practice through knowledge synthesis, resources and toolkits for action. The aim of this work is to:
- Facilitate access to evidence syntheses and other knowledge products to support both efforts to advance the cancer strategy and those working in cancer control broadly;
- Develop models of collaboration with partners to provide an evidence synthesis function; and
- Provide supports, such as training, tools and resources, to foster evidence-informed practice within the Partnership and the cancer control community.Footnote 57
As an example of work in this initiative, in 2014-2015 the focus was on: improving the Partnership's own use, application and production of evidence-based resources; preparing additional guidelines for addition to the Standards and Guidelines Evidence (SAGE) Directory of Cancer Guidelines; and collaborating with Pan-Canadian Oncology Drug Review (pCODR) to update and translate a tutorial for patients, caregivers and interested public on how cancer drug funding decisions are made.
4.4.8 Immediate Outcome - Enhanced Public and Patient Awareness and Engagement about the Cancer Strategy and Cancer Control Issues
Findings
The Partnership's initiative in support of the immediate outcome of enhanced public and patient awareness and engagement about the cancer strategy and cancer control issues is in its early days. The initiative is targeted on engaging the interested public and embedding the patient perspective in initiatives, rather than broadly engaging the general public and patients. It is designed to ensure the patient perspective is captured, that integrated approaches across multiple communications channels and platforms are used, that existing partnerships are built upon, and innovative opportunities are explored.
The Partnership has launched initiatives to strengthen engagement with the public, patient groups and professional associations. The Partnership has also incorporated the patient perspective on all of its advisory committees. There is agreement that the Partnership's role with respect to engagement of health care professionals and others working in cancer control is appropriate.
However, there were concerns raised from key partners that the Partnership should not be trying to directly engage with patients and the public in order to raise awareness, rather than supporting other organizations to do this. These comments suggest a disconnect between what stakeholders interviewed assume is within scope of the Partnership's "public engagement and outreach" and the actual focus that the Partnership is taking in this initiative.
Relationship to the 2012-2017 Strategic Plan
The activities and outputs undertaken in the coordinated strategy for cancer control related to this immediate outcome are closely tied to the following core enabling function (in the 2012-2017 Strategic Plan) and initiatives.
Core Enabling Function
Public engagement and outreach.
Initiative(s)
- Public Engagement and Outreach
Public Engagement and Outreach
The Partnership launched its integrated Public Engagement and Outreach Initiative to further strengthen engagement with professional audiences, the patient community as well as the general public.Footnote 58 More precisely, the direction of the initiative is on engaging the interested public and embedding the patient perspective in initiatives, rather than broadly engaging the general public and patients. This direction was approved by the Board of Directors in April 2013. At that time, the Board also supported the new direction for cancerview.ca - that its content should "target professionals and those active within the health system, but the information, such as that proposed in the monthly digest, would then also be used to support public outreach and engagement strategies...".Footnote 59
Key principles of the Public Engagement and Outreach Initiative are as follows:
- Ensuring that patient perspectives are captured to inform system design and direct the Partnership's efforts.
- Using integrated approaches across multiple communications channels and platforms to reflect the collective efforts of the Partnership's work with partners and its progress toward shared goals in cancer control.
- Building on existing partnerships and seeking innovative opportunities to expand the Partnership's reach and impact.
The work program has included: a pan-Canadian Cancer Communications Committee, comprised of communications leads from provincial cancer agencies; increased collaboration with Canadian Cancer Action Network (CCAN) and other national health organizations; addition of patient and family representatives to the Partnership's advisory group structures; and increased social media presence on Twitter and Facebook.
Internal key informants for this evaluation noted that the patient perspective is now captured with patients or family members represented on all of the advisory committees and actively participating in planning groups and working groups. There were also examples provided of activities and initiatives focusing on the patient perspective that are now getting underway. A majority of internal interviewees questioned any greater role that the Partnership should play in terms of public awareness and engagement since the bulk of the Partnerships work is focused on providing information and data without interpretation, mostly aimed at policy makers and decision makers at the federal and provincial levels in terms of the cancer care system.
Federal representatives noted that it is critical to engage the public and patients, but questioned whether the Partnership should be focusing on public awareness. It was also identified that for cancer patients and the public, there is a tendency to seek out information and support at the local level or through branded organizations (e.g., Canadian Cancer Society).
Most representatives of cancer agencies noted that in general, there is an increased awareness and engagement of Canadians in cancer issues and concerns, but these interviewees questioned whether there is a role for the Partnership to play in awareness and engagement of patients and the public.
Key interviewees at the initiative level indicated that the tools and resources developed as a result of projects targeted to increase public and patient engagement supported patients to better understand their cancer experience.
4.4.9 Impact on the Intermediate and Ultimate Outcomes
Findings
The approach being taken is consistent with those well-recognized internationally as leading to the intermediate and ultimate outcomes. Key informants agreed that the immediate outcomes are necessary to achieve the intermediate and ultimate outcomes. All key informants also agreed that, in Canada's federated health care system, achievement of the outcomes is dependent on action by the jurisdictions. At the same time, this also reinforces the importance of a strong coordinated body such as the Partnership to facilitate achievement of the outcomes across the country.
Many key informants suggested that, looking ahead, the Partnership will need to focus on areas where, given national cancer issues (e.g., related to trends and demographics), it can drive the most impact in the longer term (e.g., focus on prevention, lowering the downstream incidence of cancer and related costs, variability of cancer control systems across Canada, and innovation).
Immediate Outcomes as Precursor to Intermediate and Ultimate Outcomes
The literature review confirmed that the approach contains elements and building blocks that are well-recognized internationally as contributing to the achievement of cancer outcomes. It points to the importance of sustained efforts. It also shows that the CRMM is consistent with what others are doing to understand causal changes in a complex cancer control environment.
There was general agreement among interviewees that the immediate outcomes are necessary to achieve the longer-term outcomes and have a pan-Canadian approach to cancer control. There was some question as to the Partnership's ongoing role in certain initiatives (e.g., cancer control for First Nations, Inuit and Métis given that moving ahead with the Action Plan will require efforts on the part of the federal, provincial and territorial governments as well as their funding and resourcing of activities and programs within the Action Plan (see Findings in section 4.4.5), patient and public awareness). Cancer agencies generally stated that the current mix of activities and initiatives is required to achieve the longer-term goals. However, all interviewees representing the cross section of initiatives cautioned that while the intermediate and ultimate outcomes of the CSCC are clearly within the Partnership's mandate, it can only wield so much influence given that, at the end of the day, the Partnership is not a health care delivery agent and as such will be limited in terms of what it can do to achieve these outcomes.
No interviewees identified whether certain intermediate and/or longer-term outcomes were more likely to be achieved than others.
Achieving Impact on the Intermediate and Ultimate Outcomes
The federal perspective is that the Partnership needs to look at what is coming on the national cancer stage (trends, demographics etc.) and focus on the areas that drive the most impact in the longer term (e.g., greater focus on prevention, lowering the downstream incidence and costs of cancer control).
Interviewees representing cancer agencies noted the Partnership has demonstrated its capacity to evolve and add new areas where it can bring value and where there is an appropriate role for them to play. A challenge identified was the variability of cancer control environments across Canada. A question raised by some of these interviewees was whether as a pan-Canadian approach, the focus should be on moving bringing all provinces up to the standards of the leading provinces, or on getting the "biggest bang for the buck" in cancer, which may mean differing levels of treatment and service across the country (e.g., smoking and lung cancer).
4.4.10 Unintended Impacts, Either Positive or Negative
Findings
There have been a number of positive unintended impacts as a result of the Partnership's activities and initiatives. These generally relate to spill-over benefits, where relationships and approaches developed during work on the cancer control system have been able to be used in other situations. Examples include impacts on the consistency of views and approaches at the federal level, on chronic disease at a more general level, on stakeholders working together outside of the Partnership's own involvement, and on other communities of healthcare professionals.
The primary negative unintended impact raised by key informants is that projects being carried out with support from the Partnership may be raising expectations of change, even in jurisdictions which may not be participating in the projects or are not placing a priority of such change.
There were a number of examples provided through the interviews of positive unintended impacts created through the Partnership's activities and initiatives. These impacts generally relate to spill-over benefits, where relationships and approaches related to work on the cancer control system have been implemented in other areas. Examples include:
- The Partnership has resulted in PHAC/HC/CIHR having more consistent views and approaches from the federal level as a result of collaboration through the Partnership;
- The pan-Canadian model for funding projects contributed to PHAC changing the way it funds projects. PHAC is now using a more pan-Canadian approach modelled on the approach used by the Partnership for CLASP. Although the pan-Canadian approach is not being used for all PHAC projects, it is being used more than previously;
- Through its prevention initiatives, the Partnership has helped build a chronic disease network across the country and developed national standards;
- While CCRA is facilitating the formation of coalitions of research funders, there have been quite a few bilateral agreements which have evolved independently of the Partnership. Organizations have also leveraged research findings to partner with other organizations to expand the use of data in other related areas of inquiry (e.g., working with First Nations, Inuit and Métis organizations to produce environmental health related information);
- There has been increased collaboration, without the facilitation or participation of the Partnership, among other cancer control system stakeholders in terms of programming for First Nations, Inuit and Métis;
- There has been an impact on other health care professionals (e.g., gastroenterology) in how they organize and collaborate amongst their communities;
- Leveraging CPAC-funded meetings and events by "piggybacking" additional meetings (e.g., tobacco strategy) onto an existing meeting or event where the same players are involved.
Negative unintended impacts voiced by jurisdictional key informants were that:
- The Partnership may be creating demands on the resources and capacity of provincial partners (e.g., through system performance reporting) and driving costs associated with increased screening; and
- Reliance on federal funding for a significant part of the "change agenda" in many jurisdictions (especially smaller ones) has led to a lot of planning on short-term projects, and sustainability issues.
4.4.11 Lessons Learned from Implementing Pan-Canadian Collaborative Approach
Findings
Lessons learned from implementing the Pan-Canadian collaborative approach, based upon input from key informants, included the following.
- The model of collaboration is effective (it works).
- It is important to align not just with jurisdictions' priorities, but also with their ability to fund and resource projects and initiatives. Also the timing of projects needs to be flexible given jurisdictional control of implementation.
- The model of engagement of First Nations, Inuit and Métis on the First Nations, Inuit and Métis Action Plan worked well, and there may be potential for its use elsewhere.
- The sustainability of initiatives is important, and associated with this, the Partnership's deliberative exit strategy for each initiative.
- It is important that all collaborators to be involved early in the planning process.
One of the key lessons learned identified by most internal and external interviewees is that the collaboration model employed by the Partnership works. The Partnership has done a very good job of bringing the various organizations together to discuss and engage around specific issues. An area for further consideration is in the planning phases of initiatives and projects. It is important to recognize that each province and territory is at its own starting point in all aspects of cancer control, and has its own way of doing things at its own speed. Facilitation of adoption and implementation needs to be aligned with the jurisdiction's ability to fund and resource associated projects and initiatives.
Another example provided of lessons learned is the approach the Partnership took on the engagement and participation of First Nations, Inuit and Métis in the development of the specific action plans developed for these groups. It was pointed out that the approach used could become a model for how other federal departments and agencies should engage aboriginal groups. Further, the Partnership's representatives identified that there is now increased collaboration among cancer control stakeholders in terms of programming for First Nations, Inuit and Métis without the facilitation or involvement of the Partnership. Health Canada representatives identified the unanticipated benefit of now having better relations with First Nations and Inuit groups as a result of the consultation and engagement process used by the Partnership.
Internal and external interviewees identified a need for the consideration of sustainability. Specifically, they noted that the Partnership should not be responsible for particular projects or areas indefinitely. These interviewees felt that the Partnership needs to identify early in an initiative when it can withdraw from being the focal point for convening, catalyzing and brokering knowledge and establish indicators that point to when to start to withdraw from an initiative or activity.
Internal and external interviewees noted that collaborators need to be involved in the process early, including the planning process, in order to effect change (key decision-makers, partners and stakeholders, clinical leaders who influence their community, etc.). These interviewees believe that in some cases the Partnership has brought in partners after the priorities for particular initiatives have been set. These interviewees argue that partners should be involved in the priority setting process as well to ensure their priorities are well reflected.
4.5 Performance: Issue #5 - Demonstration of Economy and Efficiency
4.5.1 Ability to Act More Quickly in Response to Evidence
Findings
There are strong examples where the Partnership has made partners more able to act quickly in response to evidence. Notable examples include:
- Spread and quality of screening programs across jurisdictions- e.g., colorectal;
- Quality improvements - e.g., best practices guidelines for distress screening of patients, synoptic reporting, CLASP coalitions; and
- More informed decisions - e.g., System Performance Reports supporting jurisdictional decision-making, HPV vaccination for girls based on better staging data.
Partners' Ability to More Quickly Respond to Evidence
Examples of how the Partnership has supported the cancer control system to more quickly respond to evidence include the following:
- The Partnership has supported the needs for well-funded evaluation and research for those jurisdictions without the resources and/or expertise available for these functions, as well as supporting National Aboriginal Organizations to fully engage and participate in Partnership initiatives.
- The Partnership was able to quickly bring together leaders in lung cancer diagnosis and treatment and based on their input communicate evidence of positive impacts related to lung cancer screening every other year, ideally combined with smoking cessation.
- Communication related to the benefits of screening as well as inappropriate screening (e.g., prostate cancer).
- In the past, each province had its own approach to screening (breast, colorectal, cervical). Through the creation of the screening networks, the Partnership was able to collectively look at evidence, analyze it and transfer the information and knowledge to the pan-Canadian networks in cancer screening. More provinces and territories now participate in screening programs than before with national targets established for core cervical and colorectal cancer screening, providing more access by Canadians to high quality cancer screening programs.
Impacts - Implementation of Quality Improvements
Examples of how evidence has helped the cancer control system to implement quality improvement include the following:
- The development and implementation of best practice guidelines for screening patients for distress, depression. This is seen as directly improving quality of care for patients because stress impacts patients' care, and their ability to comply with treatments.
- Synoptic reporting has resulted in quality improvements in pathology.
- The coalitions established in CLASP initiatives have been important for jurisdictions sharing knowledge and information from research, practice and policy applications, and thereby improving their planning and policies.
- Support for the development of radiation therapy guidelines and earlier chemotherapy guidelines.
Impacts - More Informed Decision-Making
Examples of how evidence has helped the cancer control system to make more informed decisions include:
- Facing high rates of colorectal cancer, a jurisdiction was initially hesitant to fund more screening, preferring instead to increase funding for treatment. The use of data and materials developed through the Partnership provided decision-makers with information that made them decide to fund more screening.
- Increased data collected and distributed by the Partnership has improved the quality and quantity of data available for decisions makers, thus allowing for more informed decision-making. The System Performance Reports and associated special studies are particularly important examples demonstrating how jurisdictional capacity and decision-making have been affected.Footnote 60
- Better staging data resulted in improved decision-making with respect to HPV vaccination for girls.
- Prospective research findings from use of the CPTP platform and database.
4.5.2 Ability to Do More with Same Resources; Acceleration of Achievement of Results; and Alternatives
Findings
There are strong examples of how the Partnership has facilitated the ability to do more with the same resources and accelerate change.
- Sharing of materials has eliminated the need for everyone to recreate the same materials. Duplication of effort has been eliminated or minimized.
- Increased speed with which successive population-based screening programs have been implemented - 50 years for cervical cancer, 10 years for breast cancer, three years for colorectal cancer.
- Sharing of information - e.g., mapping of active transportation infrastructure in collaboration with urban planners.
The Partnership model was strongly endorsed as the best alternative, given Canada's federated model. There were some suggestions that the approach could have a broader focus than just cancer (like the National Institute for Health and Clinical Care Excellence (NICE) in the United Kingdom). In general, these were contemplative suggestions rather than strongly held opinions that there was something wrong with the current approach. There were some other suggestions that the model be implemented in other issue areas such as seniors' health, cardiovascular health and dementia.
Facilitation of the Ability to Do More with the Same Resources and Accelerate Change
There is broad consensus among interviewees who were able to respond that the Partnership and its model of coordination and collaboration have facilitated the ability of the cancer control system to do more with the same resources. Most interviewees agreed that the ability among the Partnership's partners and stakeholders to share materials (guidelines, protocols, etc.) has eliminated the need for everyone to recreate the same materials. In this sense, the Partnership has eliminated or minimized at least some duplication of effort across jurisdictions and other CPAC partners and stakeholders. Other examples of collaborative approaches which have enabled partners to achieve more with the same level of resources or to accelerate change are the same as those for increasing the efficiency and economy of the cancer control system (see section 4.5.3). Perhaps the most cited example is the increased speed at which successive population-based screening programs have been implemented across Canada - 50 years for cervical cancer, 10 years for breast cancer, and three years for colorectal cancer.
Another example provided by key informants involves the sharing of information. The Partnership is mapping active transportation infrastructure in Canada in collaboration with Canadian urban planners who identify items for the Partnership to add to the map. This map is available on cancerview.ca. This has facilitated the sharing of information and best practices among jurisdictions (including provinces and municipalities) and eliminated the need for each jurisdiction to develop its own map.
A further example is the case of the recent shortage of radio-nucleotides (isotopes) that Canada faced a few years ago. Through CCRA and the Partnership, alternates sources for these nucleotides were found much faster and at a lower cost than if every province and territory had attempted to do it themselves.
Alternative Approaches to the Partnership Model
Few interviewees were able to offer alternatives to the Partnership model that would achieve the same or similar outcomes. Interviewees generally agreed that the Partnership is the most appropriate model for achieving the desired outcomes given the federated model of healthcare in Canada. A few of these interviewees (both internal and external) further commented that in the absence of a single healthcare system, the Partnership model is the best alternative. A few interviewees noted that the Partnership model has been so successful that it could/should be implemented in other areas such as a national strategy for seniors' health, a national strategy for dementia, and a national strategy for cardiovascular health.
Very few alternative models were suggested by interviewees. The National Institute for Health and Clinical Care Excellence in the United Kingdom was suggested as an alternative with a broader focus than just cancer. Another alternative proposed was the Canada Health Infoway model whereby partners/stakeholders contribute some funding. However, this interviewee further noted that this approach would not have worked initially since one of the big draws of the Partnership was the amount of funding available.
4.5.3 Contribution of Partnership to Economy and Efficiency Improvements
Findings
The Partnership has contributed to improved efficiency and economy of the cancer control system. Examples include:
- Collaboration and sharing of information across jurisdictions;
- Agenda / frameworks for discussions which help stakeholders work through issues earlier and accelerate change;
- Sharing of best practices;
- Development of common protocols and tools; and
- Development and operation of specific analytical tools such as the CRMM which will help jurisdictions make more efficient and cost-effective decisions.
The Partnership has done so while also addressing some of the tough issues such as cancer control for First Nations, Inuit and Métis and instilling a person-centred perspective. Both of these are at early stages and will continue to be tough issues for the cancer control system moving forward. Other tough issues relate to other underserved populations such as new immigrants and those living in rural and remote areas, and to the aging population, and costs of cancer care, drugs and technologies.
Improved Efficiency and Economy of the Cancer Control System
Internal and external interviewees were able to provide examples of where the Partnership has contributed to increasing the efficiency and economy of the cancer control system. The key mechanism through which this has occurred is via collaboration and sharing of information across jurisdictions. The information shared includes information developed by the Partnership in collaboration with its partners as well as information on best practices in jurisdictions across Canada, the sharing of which is facilitated by the Partnership. An internal interviewee further noted that there is also the potential to further share information, and thus increase the efficiency and economy of cancer control in Canada through international collaboration and information sharing.
Although there was broad agreement that the Partnership has directly contributed to the overall efficiency and economy of the cancer control system, it was also noted that the activities of the Partnership have the potential to adversely impact the system costs of other stakeholders. For example, while acknowledging the benefits of screening, cancer agency representatives noted that increased screening results in additional costs at the jurisdictional level, as does increased data collection and reporting.
Examples of how the Partnership has contributed to the overall efficiency and economy in the cancer control system include:
- The Partnership has provided an agenda/framework for discussions with and among provinces and cancer agencies, which helps in working through issues earlier, and accelerating change.
- The sharing of best practices related to screening for depression, distress, etc. in cancer patients meant that these issues could be dealt with earlier in the context of treatment and thus providing efficiencies at the treatment level. At a broader system level, the sharing of best practices in this area meant that jurisdictions were able to benefit from the experience and knowledge of others and thus more efficiently develop their own approach to screening for depression, distress, etc.
- The development of protocols and tools for screening for colorectal cancer and synoptic reporting meant that individual jurisdictions did not need to develop the necessary tools/documents wholly on their own - they were able to benefit from tools and other documents developed through the Partnership.
- The cancer risk management data being aggregated by the Partnership is expected to provide information on the potential costs, benefits and impact on lives saved, thus providing jurisdictions with guidance on making the most efficient and cost-effective decisions.
- The coordination of the Canadian Partnership for Tomorrow Project by the Partnership meant that duplication of effort at the provincial level was minimized.
- The development of a mechanism that will track incident reporting for radiation therapy was made possible by the Partnership, saving time and money by bringing together the right people/stakeholders; i.e., those who are able to implement policy changes to bring about change.
Addressing the Tough Issues in Cancer Control
Although there is general agreement across all categories of interviewees that the Partnership has made important contributions in addressing some of the tough issue in cancer control in Canada, there is also agreement that a number of tough issues remain to be addressed - issues that are both existent and emerging.
The Partnership's "anticipatory science" population-based approach has helped the cancer community look at existing evidence, understand what is known, and develop the national position and messages that should be going forward to Canadians to reduce cancer risk.
In terms of existing issues that remain to be addressed, interviewees identified underserved populations such as new immigrants, those living in rural and remote areas, and First Nations, Inuit and Métis (although most of these interviewees recognized that much progress has been made in recent years) as requiring attention. Screening was specifically noted as an area requiring attention for these populations.
Another set of issues identified by interviewees as remaining to be addressed relates to the person-centred perspective, including issues such as palliative care, treatment, and cancer navigation. Interviewees who noted these issues generally indicated that the Partnership has made little progress in addressing the person-centred perspective in its current mandate.
Emerging issues identified by interviewees which will require attention in the future include the aging population and the corresponding increase in the number of cancer cases, the increasing costs of cancer care, cancer drugs and technologies and the implications for patients, families and healthcare systems.
5.0 Conclusions
5.1 Relevance Conclusions
5.1.1 Continued Need
Need for Sustained Coordinated Effort
There is a continued need for a sustained (i.e., ongoing) and coordinated effort in transforming and improving the cancer control system across Canada. This need arises because of a number of factors: the ongoing problem of cancer and its impacts on Canadians; the gaps in and opportunities for improvement that still exist in the cancer control system today; the length of time that it takes between making changes to the system and realizing the benefits; and the challenges posed by Canada's federated model for health care.
Need for the Partnership
The Partnership has played and will continue to play an important and unique role in the sustained and coordinated effort at the national level. There will continue to be a need for and stakeholder support for the "honest broker" role that the Partnership has been uniquely positioned to take, going into a third mandate and even beyond.
Need for the Partnership's Model
The Partnership's model of convening, integrating, catalyzing and brokering knowledge will continue to be appropriate for its role in the sustained and coordinated effort. Ideas from stakeholders as to how the Partnership's positioning and model could be revised moving forward will need to be vetted carefully against the Partnership's context and value as perceived by stakeholders in the cancer control system. For example, changes in the model or in its application need to consider potential implications for the Partnership's positioning as an honest broker that keeps true to the model, as a catalyst rather than "owner" of change, and as a nimble, adaptive and flexible organization that moves its focus and portfolio of initiatives to where it can provide maximum value.
5.1.2 Alignment with Government Priorities
The Partnership is aligned with the priorities of the federal government and the priorities and strategic outcomes of Health Canada. It also has good alignment with the priorities of its key partners. This alignment is well understood by key stakeholders engaged in this evaluation, with mechanisms in place to support the sharing of information about priorities.
5.1.3 Alignment with Federal Roles and Responsibilities
The Partnership's mandate is well aligned with federal roles and responsibilities related to health care innovation, collaboration across jurisdictions, and chronic disease prevention and mitigation. This conclusion is supported by key stakeholders engaged in this evaluation, who voiced their expectation that the federal government would have ongoing roles and responsibilities related to the cancer control system and the CSCC, and that the Partnership, as an arm's length organization, was supportive of federal roles and responsibilities.
The Partnership's mandate has also been aligned with and has supported the roles and responsibilities of its partners and stakeholders. Alignment in this case is not equivalent to "same as". Rather it is better viewed as meaning "complementary" or "compatible". Ongoing alignment is supported by the Partnership's positioning as an honest broker, focusing on supporting stakeholders to improve and change to the cancer control system.
5.2 Performance Conclusions
5.2.1 Achievement of Expected Outcomes (Effectiveness)
Demonstration of Results on Immediate Outcomes
Prevention and Screening
Building upon work begun or with roots in the Partnership's first mandate, significant progress continued regarding improved access to evidence-based prevention strategies and improved quality of and participation in screening. Further, the Partnership has made important contributions to the progress made.
The CLASP initiative has shown successful pilot/demonstration projects and more are underway in this mandate. The challenge will be to sustain the current CLASPs and then scale the CLASP approach to engage a much greater percentage of the Canadian population. This raises questions about project selection and knowledge transfer and exchange approaches going forward.
Population-based screening is a major success story for the Partnership and its partners. Through its support for cancer-specific screening frameworks and screening networks, the inclusion of specific performance indicators in its cancer system performance reports, and its Cancer Risk Management Model, the Partnership has helped accelerate the implementation of organized population-based screening programs across and within jurisdictions. These are important functions for the Partnership to continue to play in screening programs. Looking ahead, the Partnership may be able to help launch, on a pan-Canadian scale, screening programs for other cancers where scientific evidence is available to support new screening efforts. The Partnership may also be a good vehicle for bringing together stakeholders from across the cancer-specific screening networks to address issues related to screening of underserved populations - e.g., new immigrants, rural and remote, those facing socio-economic challenges - such as it is doing through its upcoming (September 2015) workshop on Screening in Underserved Populations to Expand Reach.
Diagnosis and Clinical Care
Progress has been made on the immediate outcome of more consistent actions to enhanced quality of diagnostic and clinical care. The Partnership's major effort has been the National Staging Initiative in its first mandate and the Synoptic Reporting and Staging Initiative in this second mandate. Staging is now mature. Efforts on synoptic reporting should continue for this mandate but should require much less Partnership involvement, other than performance reporting and benefits analysis, in the next mandate.
The Partnership has also made contributions to quality improvement in diagnosis and clinical care through its work with the Canadian Partnership for Quality Radiotherapy on quality assurance and with Accreditation Canada on accreditation standards.
Patient Needs
Although there is evidence that progress has been made on improving the capacity of the cancer control system to respond to patient needs, there is also evidence that much still remains to be done. There is a general consensus that although early steps towards achieving this outcome have been taken, there has been only limited impact on patients. Although the cancer control system, including the Partnership, has taken steps to improve the capacity of the cancer control system to respond to patient needs, actual change will need to come from stakeholders and partners who work directly with patients, i.e., the Partnership is able to facilitate change but not directly bring it about.
Research Coordination and Capacity
The CCRA and the CPTP have resulted in enhanced coordination of cancer research and improved population-health research capacity. The Partnership has contributed to the CCRA through support of the CCRA's executive office and through its membership in the CCRA. The Partnership has also contributed to several collaborative projects (e.g., CPTP and the Canadian Clinical Trials Network) included in the CCRA's research plan. Although the Partnership has contributed to progress on this outcome, there is evidence of some ambiguity concerning the role of the Partnership in research - whether the Partnership coordinates cancer research itself versus supporting and being a member of the CCRA, which is the coordinating body.
Cancer Control for First Nations, Inuit and Métis Peoples
The development of the First Nations, Inuit and Métis Action Plan on Cancer Control represents an important step towards improving cancer control in First Nations, Inuit and Métis communities. The Partnership played a critical role in facilitating the development of the Action Plan. Although it is up to provincial, territorial, federal and other stakeholders to implement changes to improve cancer control in First Nations, Inuit and Métis communities, there is evidence of change. In 2015, eight jurisdictions were participating in and implementing initiatives to improve the continuity of care in remote and rural locations. Further, cancer agencies or their equivalents in three jurisdictions (Northern British Columbia, Manitoba and Ontario) had in place First Nations, Inuit and Métis specific cancer control strategies, compared to one since the Partnership was first initiated. The delivery of the Saint Elizabeth Cancer Course has increased awareness of culturally relevant approaches to cancer control and care among healthcare providers.
Cancer System Performance
Evidence from the evaluation indicates that the objective of improved analysis and reporting on cancer control system performance has been achieved. The Partnership, in collaboration with its partners and stakeholders, has provided both annual and special cancer system performance reports, some of which have delved into areas of interest to the Partnership's partners. The Partnership has brought forward the pan-Canadian approach to the cancer system performance reports and facilitated the identification of comparable indicators from across Canada. There is evidence that the reports are being used by cancer agencies and other stakeholders to improve cancer control across Canada. However, provinces and territories face challenges in finding the financial and technological resources required to collect the data in support of the System Performance Reporting Initiative. The current approach to system performance reporting, making use of a mix of annual system performance reports on a small core set of indicators, and special reports to consider specific issues, appears to be appropriate looking ahead, as the Initiative matures and provinces meet their performance targets.
Information, Knowledge, Tools and Resources
There is evidence of progress on enhancing access to high-quality information, knowledge, tools and resources through collaboration across numerous players in the cancer control community. The Partnership has increased access to resources through its web presence via cancerview.ca and its corporate site, partnershipagainstcancer.ca. In addition, the Partnership has developed tools such as the Cancer Risk Management Model, which is expected to lead to system wide improvements in cancer control. It has worked on building analytical capacity in jurisdictions and supporting use and analysis through subject-specific websites (e.g., CPTP portal) and collaboration spaces (e.g., cancerview.ca). The maturing of more initiatives will put additional demands on the Partnership's knowledge transfer and exchange efforts.
Public and Patient Awareness and Engagement
Although the Partnership has taken actions to facilitate the enhancement of public and patient awareness and engagement about the CSCC and cancer control issues, the evaluation found that it is still early days in terms of outcomes. Even though the Partnership, since 2013, has focused its efforts upon engaging the interested public (e.g., cancer community, those in the public to be engaged for a particular outcome such as screening) and embedding the patient perspective in initiatives, rather than broadly engaging the general public and patients, there were key informants who questioned the Partnership's involvement in public awareness. Dialogue with partners and stakeholders to make clear the Partnership's role and intentions related to this outcome appears to be required.
Impact of Immediate Outcomes on Intermediate and Ultimate Outcomes
Although there is agreement that progress on immediate outcomes has been made and that these immediate outcomes are the building blocks for achieving the longer-term outcomes, there is also agreement that the achievement of the outcomes is dependent on action by the jurisdictions. The federated system of healthcare delivery means that there is a need for a strong coordinating body such as the Partnership to facilitate the achievement of the immediate, intermediate and longer-term outcomes of the Canadian Cancer Control Strategy.
There may also be a need for the Partnership to continue to focus on those areas where it can provide the greatest value and drive the most impact in the longer term.
Unintended Impacts
The work carried out by the Partnership has resulted in relationships and approaches which have been used to advantage in other situations, for example, in the broader health care system.
Projects being supported by the Partnership may raise expectations that the cancer control system will change, even in jurisdictions that may not be participating in the projects.
Lessons Learned
The key lesson learned identified by a broad range of interviewees is that the collaborative model employed by the Partnership is effective, i.e., it works. External interviewees consistently expressed satisfaction with how the Partnership has engaged its partners and stakeholders in order to advance the CSCC. A related lesson learned is the approach taken to engagement with First Nations, Inuit and Métis communities in the development of specific initiatives for each community. Some interviewees noted this approach as a model for other federal departments and agencies for engaging with First Nations, Inuit and Métis.
Internal and external interviewees noted that there is a need for the Partnership to take into consideration the funding and resources available in jurisdictions to participate in projects and initiatives, and later the sustainability of these projects or initiatives as the Partnership moves forward to its next mandate. There is evidence of a need for the Partnership to establish clear points where it will start to withdraw from particular activities and initiatives.
Internal and external interviewees noted the need for the Partnership to involve key collaborators in the planning process for initiatives. These interviewees feel that in some cases the Partnership has brought in key partners only after the priorities have been set.
5.2.2 Demonstration of Economy and Efficiency
Ability to Act More Quickly in Response to Evidence
The Partnership has supported the cancer control system to more quickly respond to evidence. It has supported the ability of jurisdictions without the resources and/or expertise to respond to evidence by providing financial and technical support. It has brought together leaders in various aspects of the cancer control system to discuss and communicate evidence of positive impacts related to these issues and facilitated the transfer of this knowledge through KTE events and the various networks that have been established (e.g., screening networks).
The Partnership has supported the implementation of quality improvements through the development of best practice guidelines (e.g., screening for distress, radiation therapy, chemotherapy, etc.). The synoptic reporting initiative has resulted in improvements in pathology and the jurisdictional involvement in the CLASP initiatives has supported improvements in planning and policy through sharing knowledge and information from research, practice and policy applications.
The Partnership has supported the cancer control system to make more informed decisions by improving the quality and quantity of data available for decision makers. The annual cancer system performance reports and associated special studies are particularly important examples demonstrating how jurisdictional decision-making has been affected.
Ability to Do More with Same Resources; Acceleration of Achievement of Results; and Alternatives
The Partnership has facilitated the ability to do more with the same resources and accelerate change. The sharing of information and materials has eliminated or reduced the duplication of efforts across jurisdictions to recreate the information and materials. It has accelerated the achievements of results as evidenced by the increased speed with which successive population-based screening programs have been implemented on a pan-Canadian basis, down from 50 years to implement cervical cancer screening to just 3 years to implement colorectal cancer screening.
Interviewees generally agreed that the current model, given the federated model of healthcare in Canada, is the most appropriate model for achieving the desired outcomes. It was further suggested that the Partnership model has been so successful that it could/should be implemented in other areas such as a national strategy for seniors' health, a national strategy for dementia, and a national strategy for cardiovascular health.
Contribution of Partnership to Economy and Efficiency Improvements
The Partnership has contributed to increasing the efficiency and economy of the cancer control system. The key mechanism through which this has occurred is via collaboration and sharing of information across jurisdictions. This sharing has occurred through the development of agendas and frameworks, sharing of best practices, development of protocols and tools and the development of analytical tools (e.g., CRMM), which will help jurisdictions make more efficient and cost-effective decisions. It was noted by interviewees, however, that there were some adverse impacts related to economy that centered on the increased costs associated with the increases in cancer screening and the additional costs relating to data collection and reporting.
Interviewees were in general agreement that the Partnership has made important contributions in addressing some of the tough issues in cancer control in Canada. There was also general agreement that a number of tough issues remain to be addressed, both existent and emerging. Existing issues included programs and services serving the needs of new immigrants and those living in rural and remote areas as well as a further focus on the person-centred perspective. Emerging issues identified were in the areas of costs of cancer care treatment (e.g., cancer drug and technology costs), as well as the impacts of the aging population on the cancer control system.
6.0 Recommendations
Seven recommendations came out of this evaluation, six recommendations are directed at CPAC and one recommendation is directed at Health Canada.
Health Canada Recommendation 1
Health Canada should monitor implementation of CPAC's Action Plan to address the evaluation recommendations. Moreover Health Canada should ensure, when appropriate, that revisions are made to CPAC deliverables in any future contribution agreement in response to the evaluation recommendations.
CPAC Recommendation 1
The Partnership model of convening, integrating, catalyzing, and brokering knowledge should continue to be the cornerstone of the Partnership's approach during the remainder of this mandate and in any future mandates. In applying the model, the Partnership should be diligent that it continues to play its current "honest broker" role rather than moving to an advocacy role.
The current model of convening, integrating, catalyzing, and brokering knowledge has provided the Partnership during the current mandate, as it did in the first mandate, significant "space" within which to add value and support changes across the cancer control system, working with a broad range of partners in a variety of arrangements. For these reasons, the evaluation team supports the model's continued use in the future. Moving into an advocacy position, or being perceived as doing so, will jeopardize the value and unique positioning of the Partnership.
CPAC Recommendation 2
In order to continue to be a nimble, adaptive and flexible organization involved in a portfolio of initiatives which shifts to where the Partnership can provide maximum value in addressing needs, the Partnership should have an explicit exit strategy/plan for each initiative/project in which it is involved, and make the exit strategy/plan clear to partners/stakeholders as part of initial initiative/project planning.
In parallel, the Partnership should continue its emphasis upon each initiative/project having a sustainability plan, with targets, so that initiatives/projects complete their life cycle to benefits realization, even if the Partnership itself is no longer involved.
This recommendation emphasizes the importance of the Partnership knowing when and how it will end its involvement in an initiative or project, and that it can do so knowing that the initiative or project will continue to be sustained. In a sense there is already an implicit exit strategy (i.e., what happens if the Partnership is not renewed for a third mandate). An explicit exit strategy needs to consider this scenario, but more generally exit strategies are about keeping the Partnership's portfolio renewed and refreshed, and directed where the Partnership can provide most value. As the Partnership has matured and the initiatives/projects in which it has been involved have grown, a larger amount of resources are required for their ongoing operations and maintenance. However, the Partnership should not be in the operations and maintenance business which is inconsistent with the Partnership's mandate and model. Operations and maintenance should rest with those in the delivery part of the system.
It would also be appropriate for other partners/stakeholders in an initiative/project to have an explicit exit strategy if their involvement is time or resource limited or is expected to end before the completion of the initiative's or project's complete life cycle.
CPAC Recommendation 3
The Partnership should engage with other organizations with a mandate for chronic disease prevention, including cancer, to develop an approach as to how the CLASP model, or something similar, can be scaled to engage the Canadian population on a much broader basis than has taken place to date. The approach would include defining a future role, if any, appropriate for the Partnership.
This recommendation is based upon the advantages that could arise from scaling the CLASP approach to engage a much greater percentage of the Canadian population. However, there is also recognition that risk factors for chronic diseases, including cancer, can cover a range of health determinants, including socio-economic factors and life style choices and behaviours, and that there are numerous organizations with a mandate for chronic disease prevention. The Partnership's experience in navigating such complex landscapes, as an honest broker, may be beneficial. It may also have a role to play as a leader or team member in supporting resulting chronic disease prevention initiatives.
CPAC Recommendation 4
The Partnership should clarify and communicate its current and ongoing role with respect to research coordination - that the Canadian Cancer Research Alliance is the cancer research coordinating body and the Partnership is a member of the Alliance.
This recommendation is intended to clarify the ambiguity that some key stakeholders have concerning the Partnership's role in research coordination.
CPAC Recommendation 5
The Partnership should communicate with stakeholders to clarify that its focus in Public and Patient Awareness and Engagement has been and will continue to be on engaging the interested public and embedding the patient perspective in initiatives, rather than broadly engaging the general public and patients.
This recommendation is intended to reinforce external understanding of the Partnership's focus on engaging the interested public and embedding the patient perspective in initiatives, so that stakeholders understand that the Partnership does not wish to broadly engage the general public.
CPAC Recommendation 6
The Partnership should continue to ensure that partners are engaged early in the planning process for initiatives and projects so that there is buy-in to the plan.
This recommendation responds to the concern expressed by some interviewees that in some cases the Partnership has brought in key partners only after priorities for initiatives and projects have been set.
Appendix 1 - References
Bell Browne Molnar & Delicate Consulting Inc (2009). Independent Evaluation of the Canadian Partnership Against Cancer, Final Report. April 22, 2009.
Canadian Cancer Research Alliance (2015). Pan-Canadian Cancer Research Strategy, 2010-2014: Final Report. March 27, 2015. From http://www.ccra-acrc.ca/index.php/publications-en/strategy/item/pan-canadian-cancer-research-strategy-2010-2014-final-report. Last viewed August 13, 2015.
Canadian Partnership Against Cancer (2012). Annual Report 2011/12. From http://www.partnershipagainstcancer.ca/wp-content/uploads/sites/5/2015/03/Annual-report_2011-2012_accessible.pdf. Last viewed August 13, 2015.
Canadian Partnership Against Cancer (2012). Performance Measurement Strategy. October 30, 2012.
Canadian Partnership Against Cancer (2014). Annual Report 2013/14. From http://www.partnershipagainstcancer.ca/wp-content/uploads/sites/5/2015/03/Annual-report_2013-2014_accessible.pdf. Last viewed August 13, 2015.
Canadian Partnership Against Cancer (2014). Pre-Canadian Lung Cancer Screening Initiative: Lung Cancer Screening Framework for Canada. September 2014. From http://www.cancerview.ca/idc/groups/public/documents/webcontent/lung_framework_en.pdf. Last viewed August 13, 2015.
Canadian Partnership Against Cancer (2014). Prevention policies directory. Last updated March 20, 2014. From http://www.cancerview.ca/cv/portal/Home/PreventionAndScreening/PSProfessionals/PSPrevention/PreventionPoliciesDirectory?_afrLoop=11152468260992614&_afrWindowMode=0&_adf.ctrl-state=16047xojfb_4. Last viewed August 13, 2015.
Canadian Partnership Against Cancer (2014). Synoptic Surgery Reporting Initiative Benefits Evaluation: Final Report. December 2, 2014. Prepared by Intelligent Improvement Consultants for Canadian Partnership Against Cancer. From http://www.cancerview.ca/idc/groups/public/documents/webcontent/ssr_benefits_evaluation_report.pdf. Last viewed August 10, 2015.
Canadian Partnership Against Cancer (2015). 2014/15 Annual Report. From http://www.partnershipagainstcancer.ca/wp-content/uploads/sites/5/2015/07/CPAC_Annual-Report_Accessible-Version_En.pdf. Last viewed August 13, 2015.
Canadian Partnership Against Cancer (2015). 2015/16 Corporate Plan. January 30, 2015.
Canadian Partnership Against Cancer (2015). Lung Cancer Screening Guidelines Across Canada - Environmental Scan. February 2015. From http://www.cancerview.ca/idc/groups/public/documents/webcontent/lung_cancer_enviro_scan.pdf. Last viewed August 13, 2015.
Canadian Partnership Against Cancer(2015). Our Work: Diagnosis and Clinical Care. From http://www.partnershipagainstcancer.ca/what-we-do/diagnosis-clinical-care. Last viewed August 13, 2015.
Canadian Partnership Against Cancer(2015). Our Work: Knowledge Management. From http://www.partnershipagainstcancer.ca/what-we-do/knowledge-management. Last viewed August 13, 2015.
Canadian Partnership Against Cancer(2015). Our Work: Person-Centred Perspective. From http://www.partnershipagainstcancer.ca/what-we-do/person-centred-perspective. Last viewed August 13, 2015.
Canadian Partnership Against Cancer(2015). Our Work: Prevention and Screening. From http://www.partnershipagainstcancer.ca/what-we-do/prevention-and-screening. Last viewed August 13, 2015.
Canadian Partnership Against Cancer(2015). Our Work: Public Engagement & Outreach. From http://www.partnershipagainstcancer.ca/what-we-do/public-engagement-outreach. Last viewed August 13, 2015.
Canadian Partnership Against Cancer (2015). The 2015 Cancer System Performance Report. June 2015. From http://www.cancerview.ca/idc/groups/public/documents/webcontent/the_2015_cancer_system_performance_report_en.pdf. Last viewed August 13, 2015.
Canadian Partnership Against Cancer (2015). The Partnership Promotes Greater Integration of Tobacco Control and Cancer Control. January 19, 2015. From http://www.partnershipagainstcancer.ca/the-partnership-promotes-greater-integration-of-tobacco-control-and-cancer-control/. Last viewed August 13, 2015.
Canadian Partnership Against Cancer. Review of 2007-12 Achievements and Delivery to Original Objectives. Date unknown.
Canadian Strategy for Cancer Control Governing Council (2006). The Canadian Strategy for Cancer Control: A Cancer Plan for Canada. July 2006. From http://www.partnershipagainstcancer.ca/wp-content/uploads/sites/5/2015/03/The-Canadian-Strategy-for-Cancer-Control-A-Cancer-Plan-for-Canada_accessible.pdf. Last viewed August 13, 2015.
CAREX Canada (2015). What is CAREX?. From http://www.carexcanada.ca/en/about/. Last viewed August 13, 2015.
Health Canada (2010). Canadian Partnership Against Cancer Corporation, Evaluation, Final Report. May 2010. Prepared by EKOS Research Associates for Health Canada.
Health Canada (2014). Health Canada 2013-2014 Departmental Performance Report. Last modified November 7, 2014. From http://www.hc-sc.gc.ca/ahc-asc/performance/estim-previs/dpr-rmr/2013-2014/report-rapport-eng.php. Last viewed August 11, 2015.
Health Canada (2014). Landmark study aims to find the causes of cancer and other diseases - Will help improve prevention efforts by understanding why some people get cancer. February 3, 2014. From http://www.hc-sc.gc.ca/ahc-asc/media/nr-cp/_2014/2014-013-eng.php. Last viewed August 13, 2015.
KPMG (2015). Canadian Partnership Against Cancer, Coalitions Linking Action and Science for Prevention, CLASP1 Renewal Evaluation Report, Final Report. February 6, 2015.
Prime Minister of Canada (2011). PM renews support for national strategy to fight cancer. March 10, 2011. From http://pm.gc.ca/eng/news/2011/03/10/pm-renews-support-national-strategy-fight-cancer. Last viewed August 13, 2015.
Public Health Agency of Canada (2013). Cancer - About Us. Last modified February 6, 2013. From http://www.phac-aspc.gc.ca/cd-mc/cancer/about-apropos-eng.php. Last viewed August 13, 2015.
Appendix 2 - Recommendations from Previous Evaluations
In accordance with its initial funding agreement, an independent evaluation of the Partnership was published in 2009.Footnote 61 That evaluation made a number of recommendations related to achievement of the Strategy (recommendations 1 through 3), building the organization (recommendations 4 and 5), core frameworks and corporate enablers (recommendation 6) , and implementation activities (recommendations 7 and 8). Although examined within the scope of the evaluation, no recommendations were made related to overall governance and accountability mechanisms, or to refinement and focusing of the Strategy for Cancer Control.
Recommendation 1: The Partnership is making progress and achieving results through itsrefined Partnership strategy, and appears well positioned to continue to do so. These achievements are endorsed by the majority of stakeholders. Therefore, the Partnership should continue to implement its strategy, with adjustments made when necessary to deal with new opportunities or performance gaps.
Recommendation 2: Impacts and benefits for the cancer control domain and its stakeholders will need to be shown to maintain support. Therefore, work on cancer risk management and cancer control system performance needs to continue so that information about benefits / impacts can be gathered, analyzed and disseminated.
Recommendation 3: The Partnership needs to continue to work with and through partners and collaboration for maximum longer-term impact even when there may be alternative approaches that might speed up the achievement of short term results that demonstrate quick successes.
Recommendation 4: The Partnership needs to keep its eyes firmly on the target, and at this time work with and through its current corporate and advisory structures, and delivery approaches, except when barriers present significant risk. With this in mind, it is recommended that the role of the Advisory Council be clarified in relation to the other advisory mechanisms that have been put into place for specific priorities and initiatives (e.g., Cancer Risk Management Advisory Committee, Canadian Colorectal Screening Network).
Recommendation 5: When there are new initiatives, priorities and opportunities in pursuit of the achievement of the CSCC objectives, it would be appropriate for the Partnership to put in place new advisory and delivery approaches if the existing ones are inadequate.
Recommendation 6: The Partnership should develop a comprehensive performance measurement framework based upon a logic model (i.e., outcomes, outputs, activities) such as the one developed for this evaluation. Current initiatives - the Enterprise Performance and Risk Management scorecard, cancer risk management model, cancer control system performance - would feed into this performance measurement framework. Any additional gaps in the ability to tell the full performance story should be identified and filled, as appropriate.
Recommendation 7: The Partnership should periodically review the roles, composition and activities of Action Groups and Pan-Canadian networks to ensure that they continue to provide net benefits. However, as noted in Recommendation 4, the priority should be to work with and through current structures and delivery approaches, except when barriers present significant risk. When conducting a review, the best role, composition and activities for Action Groups and Pan-Canadian networks need to be looked at on a case by case basis.
Recommendation 8: It is an appropriate time for the Partnership to put in place a stakeholder engagement strategy. The strategy should include the notion of an engagement continuum, with stakeholders being engaged in a manner appropriate to their roles and the impact that they can bring to the change agenda. The strategy should consider stakeholders that have not been engaged significantly over the last almost two years. This includes the public, aboriginal groups and other potential stakeholder/partners. The strategy should also promote the use by stakeholders of their own networks for communications out to broader audiences.
An organizational evaluation of the Partnership was also conducted by Health Canada and published in May 2010.Footnote 62 The evaluation made recommendations in the areas of design and delivery, success, and governance. The recommendations follow.
- Design and Delivery
- CPACC should continue to facilitate the integration and coordination amongst the eight strategic priorities and two supporting activities wherever appropriate. Encouraging more integrated approaches to developing initiatives will facilitate coordination and impact.
- CPACC must continue in its recently increased efforts to address the perspectives and needs of First Nations, Inuit and Métis in all of its activities.
- Success
- CPACC should develop formal mechanisms for assessing the usefulness of the data and information it is providing. Stakeholders and users of CPACC data and information should be consulted on a regular basis to gauge the usefulness, credibility and accessibility of CPACC data and information. The results of these consultations would be used to facilitate ongoing improvements to CPACC knowledge transfer/knowledge exchange.
- CPACC should develop mechanisms for communicating with stakeholders who are not currently engaged with CPACC but who work in the area of cancer control. This could be done through attendance and presentations at conferences and other such events.
- It is recommended that CPACC assess mechanisms for increasing its regional presence. This could include options such as affiliation with university-based partners. An increased regional presence would better enable CPACC staff to network and develop relationships with regional cancer control organizations. This is particularly critical in the context of the Canadian healthcare system and for CPACC to ensure needs are being met at the jurisdictional level.
- CPACC must ensure that the needs of jurisdictions are reflected in all of CPACC activities and initiatives, as their buy-in and active engagement are required for CPACC to fulfill its objectives.
- It is recommended that CPACC work to clarify its roles and responsibilities and those of its stakeholders on an on-going basis, to ensure that all individuals affiliated with stakeholder organizations are aware of CPACC and their organization's relationship with CPACC.
- Governance
- CPACC must develop and implement a performance monitoring system using both qualitative and quantitative measures appropriate to the current stage of its development, which should include measuring outcomes. As a new organization it is clear that early on the focus of performance monitoring will be on outputs (# of meetings, #of reports produced, etc.); however, as CPACC evolves the emphasis should move away from measuring outputs to measuring outcomes. This will require the full engagement of the federal government and jurisdictions.
- It is recommended that CPACC put in place a transparent and clearly articulated mechanism for soliciting and selecting projects. There must also be a mechanism in place for communicating the results of decisions made.
- It is recommended that CPACC work to increase awareness of itself among the cancer control community as well as the Canadian public.
Appendix 3 - Strategic Priorities, Core Enabling Functions and Associated Initiatives
According to the program's strategic priorities and outcomes, the Canadian Partnership Against Cancer has eight priority initiatives:
- Coalitions Linking Action and Science for Prevention,
- Embedding Evidence in Care - Synoptic Reporting and Staging,
- Patient Experience and Outcomes,
- Canadian Partnership for Tomorrow Project,
- First Nations, Inuit and Métis Action Plan on Cancer Control,
- System Performance Reporting,
- Cancerview.ca, and
- Public Engagement and Outreach.
Most of these initiatives have associated initiatives, for example:
- Coalitions Linking Action and Science for Prevention has three associated initiatives:
- Population-based Screening,
- Healthy Public Policy, and
- CAREX Canada.
- Embedding Evidence in Care - Synoptic Reporting and Staging Initiative has three associated initiatives:
- Emerging Screening and Early Detection,
- Quality Implementation Initiative, and
- Enhancing Canadian Cancer Clinical Trials.
- Patient Experience and Outcomes has three associated initiatives:
- Survivorship,
- Palliative and End-of-Life Care, and
- Primary and Cancer Care Integration.
- Canadian Partnership for Tomorrow Project has one associated initiative:
- Canadian Cancer Research Alliance.
- Cancerview.ca has four associated initiatives:
- Cancer Risk Management Modelling and Economic Analysis,
- Partnership Knowledge Transfer and Adoption,
- Analytic Capacity Building and Co-ordinated Data Development, and
- Evidence Synthesis Guidelines.
These initiatives support the following Strategic Priorities and Core Enabling Functions:
- develop high-impact, population-based prevention and cancer screening approaches;
- advance high-quality diagnosis and clinical care;
- embed a person-centred perspective throughout the cancer journey;
- enable targeted research to augment our knowledge and understanding of cancer and related chronic diseases;
- advance cancer control with and for First Nations, Inuit and Métis peoples and partners;
- system performance analysis and reporting;
- knowledge management through tools, technology, connections and resources; and
- public engagement and outreach.
It is anticipated that these initiatives and strategic priorities and core enabling functions will lead to the following outcomes:
- improved access to evidence-based prevention strategies and quality of, and participation in, screening;
- more consistent actions to enhance quality of diagnosis and clinical care;
- improved capacity to respond to patient needs;
- enhanced coordination of cancer research and improved population research capacity;
- improved First Nations, Inuit and Métis cancer control with and for First Nations, Inuit and Métis peoples and partners;
- improved analysis and reporting on cancer system performance;
- enhanced access to high-quality information, tools and resources; and
- enhanced public and patient awareness and engagement.
By 2017, the program expects that these priority initiatives, associated initiatives, strategic priorities, core enabling functions and outcomes will result in increased efficiency and acceleration of cancer control in Canada.
Appendix 4 - Summary of Findings
Rating of Findings
Ratings have been provided to indicate the degree to which each evaluation questions and issues have been addressed.
Relevance Rating Symbols and Significance
A summary of Relevance ratings is presented in Table 1 below. A description of the Relevance Ratings Symbols and Significance can be found in the Legend.
Questions | Indicators | Overall Rating | Summary |
---|---|---|---|
Legend - Relevance Rating Symbols and Significance:
|
|||
1. Continued Need for the Program | |||
What is the current picture of cancer control and the likely impact of cancer on Canadians with and without a sustained, coordinated effort? |
|
High |
There is a continued need for a sustained (i.e., ongoing) and coordinated effort in transforming and improving the cancer control system across Canada. This need arises because of a number of factors: the ongoing problem of cancer and its impacts on Canadians; the gaps in and opportunities for improvement that still exist in the cancer control system today; the length of time that it takes between making changes to the system and realizing the benefits; and the challenges posed by Canada's federated model for health care. Cancer is, and will continue to be, a serious problem with significant impacts upon the lives of Canadians. For example, cancer incidence continues to rise. Between 2007 and 2031, it is expected that new cancer cases will increase by 71 per cent, while the Canadian population will increase by only 19 per cent over the same time period. Cancer incidence is a particular concern for First Nations, Inuit and Métis populations. Although research is limited, existing studies invariably show that cancer incidence has risen dramatically in each of the First Nations, Inuit and Métis populations over the past few decades. By 2031, it is estimated that 2.5 times as many Canadians (2.2 million) will be living with a cancer diagnosis compared to 2007. Key informants across all stakeholder groups supported these arguments about the need for a sustained coordinated effort. They noted that such efforts help to identify areas upon which to focus improvement initiatives, to deal with the lengthy time periods required for implementation and benefits realization, to provide transparency and order to spending, to improve efficiency and to inform jurisdictional priority setting and decision-making. In a recent poll, conducted in late 2014, 94% of Canadians supported the national cancer strategy |
What is the continued need for the Partnership to achieve intermediate and ultimate outcomes in cancer control in Canada? |
|
High |
It was recommended that the best vehicle for moving the Canadian Strategy for Cancer Control (CSCC) forward ("the model of governance and management will ultimately determine the effectiveness and success of the CSCC") while adhering to key principles was "an independent stand-alone model of governance". The Partnership is seen by key informants as having brought value to the cancer control community and system. It is considered to have been an important part of the system and a unique contributor to the sustained coordinated approach, especially in areas such as prevention and screening where it has helped accelerate uptake and implementation. In its positioning and support for pan-Canadian perspectives, it is described in terms such as "honest broker". It is credited with helping provincial and territorial jurisdictions move to implementation (i.e., knowledge to action) with a better balance across the various components of the cancer control continuum, better knowledge and understanding influencing decision-making, and more efficient approaches. This is especially the case for smaller jurisdictions without the scale and resources to "go it alone". |
Is the Partnership, through its model of convening, integrating, catalyzing, and brokering knowledge with and through partners, accelerating cancer control in Canada? |
|
High |
The coordinated and sustained approach to cancer control upon which the CSCC is based contains internationally recognized, as per published research, elements and features of effective cancer control. According to the literature reviewed for this evaluation, these elements and features include collaboration, knowledge translation, governance that is inclusive of partners, and integration of lessons learned. The Partnership's specific role in the sustained coordinated approach is exemplified in its model of "convening, integrating, catalyzing and brokering knowledge with and through partners. There are strong examples, such as the introduction of screening programs and synoptic reporting, where the Partnership's model has accelerated progress. These are the types of examples mentioned by the majority of both external and internal key informants in their support for the Partnership's model and its contributions to date. Going forward, key informants noted their continued support for the model, although there were, quite naturally, different opinions on the components of the model that should be emphasized. |
2. Alignment with Government Priorities | |||
Is the Partnership aligned with federal government priorities and with Health Canada's priorities and strategic outcomes? |
|
High |
The Partnership is aligned with the priorities of the federal government and the priorities and strategic outcomes of Health Canada. The alignment is demonstrated by statements made by the former Prime Minister (2011) and the former Minister of Health (2014). It is also demonstrated by how Health Canada's funding support for the Partnership fits within Health Canada's Program Alignment Architecture (2013-2014) and Departmental Performance Report (DPR) (2013-2014) to Parliament. The Partnership has carried this alignment through to its own strategic plans for its first and second mandates, and its annual corporate plans and annual reports. |
Is the Partnership aligned with priorities and objectives of the partners? |
|
High |
The Partnership is well aligned with the priorities of its key partners, including jurisdictions, cancer agencies, NGOs and patient groups. In some cases, alignment is reflected in shared ultimate outcomes, and in others the same priorities expressed in forward looking documents such as strategic plans. |
3. Alignment with Federal Roles and Responsibilities | |||
Is the Partnership aligned with federal roles and responsibilities? |
|
High |
The Partnership's mandate is aligned with federal roles and responsibilities. In particular, there is a connection with the federal roles related to "supporting health care innovation and collaboration across jurisdictions" and to chronic disease prevention and mitigation. All key informants voiced their expectation that the federal government would have ongoing roles and responsibilities related to the cancer control system and the CSCC. They also indicated that the Partnership and its mandate were appropriate and supportive of the federal roles and responsibilities. For example, an arms-length organization such as the Partnership provides value to Health Canada, allowing it to play its role in the federated health care system and support multi-jurisdictional initiatives. |
Is the Partnership's mandate aligned with the roles and responsibilities of the partners? |
|
High |
The Partnership's mandate is aligned with and supports the roles and responsibilities of its partners and stakeholders. Some key informants emphasized that alignment depended upon the Partnership remaining true to its model and limiting its scope where others are already involved in the cancer control continuum. Some others suggested a more active leadership role for the Partnership around specific issues. |
Performance Rating Symbols and Significance:
A summary of Performance Ratings is presented in Table 2 below. A description of the Performance Ratings Symbols and Significance can be found in the Legend.
Questions | Indicators | Overall Rating | Summary |
---|---|---|---|
Legend - Performance Rating Symbols and Significance:
|
|||
4. Achievement of Expected Outcomes (Effectiveness) | |||
Are the efforts of the coordinated strategy for cancer control demonstrating results on the immediate outcomes of: | |||
Improved access to evidence‐based prevention strategies and improved the quality of, and participation in, screening |
|
Progress Made; Further Work Warranted |
Through initiatives such as Coalitions Linking Action and Science for Prevention (CLASP), Population-Based Screening, Healthy Public Policy and CAREX Canada, all of which begun during or have with roots in the Partnership's first mandate, progress has continued in the second mandate on improving access to evidence-based prevention strategies and improving quality of and participation in screening. To date, CLASP projects have already exceeded the targets set for 2017 in terms of the total population engaged, the number of locales engaged, and the number of locales adopting evidence-informed prevention policies and evidence-informed prevention practices. Types of changes included patient screening and development of new or amended municipal bylaws. Immediate impacts were felt at local levels, and particularly for rural and remote locations. The Partnership's contribution to the CLASP projects through its initial launch of the CLASP initiative and its funding and other support were acknowledged. Such multi-jurisdictional projects would not likely have happened without the Partnership's leadership. At the same time, funding pressures resulted in highlighting issues of sustainability of the formal coalitions and ability to monitor the impacts of policy and practice changes into the future. Screening has been a success story for the Partnership and its partners. During this mandate, the Partnership's screening initiative expanded beyond cervical and colorectal screening, to include breast and lung cancer. The screening networks have been effective in sharing information and best practices, which has led to the spread of screening programs and increased screening rates and quality on a pan-Canadian basis. Screening rates for these targeted cancers are at or moving towards the targets set by the screening networks. |
More consistent actions to enhanced quality of diagnosis and clinical care |
|
Progress Made; Further Work Warranted |
Progress has been made on the immediate outcome of more consistent actions to enhanced quality of diagnosis and clinical care. Synoptic reporting and staging, which started in the first mandate and continued into the second mandate, has been the priority initiative in support of this outcome. Targets for 2017 have been or are close to being met, in terms of the number of jurisdictions meeting targets for the percentage of stage data captured, implementing synoptic pathology reporting and implementing synoptic surgery reporting. In the case of surgical synoptic reporting, an evaluation in 2014 found improvements in access, quality and productivity. Throughout this initiative, the Partnership is credited with having had an important and evolving role, as acknowledged by its partners and validated in evaluations of the National Staging Initiative and Synoptic Surgery Initiative. Looking ahead the challenges appear to be to: expand ("scale") the implementation of synoptic reporting within jurisdictions to more institutions, and to more jurisdictions; bring more pathologists and surgeons on board through demonstrating value and providing transition / change management support; and put in place, by jurisdictions, the funding and resources to move to much more widespread use of an information technology intensive approach. Important work has also taken place with the Canadian Partnership for Quality Radiotherapy on the gathering of evidence in radiotherapy quality assurance and associated accreditation standards (with Accreditation Canada). This followed work in the first mandate on chemotherapy guidelines and accreditation standards. |
Improved capacity to respond to patient needs |
|
Little Progress; Priority for Attention |
Progress has been made on the immediate outcome of improved capacity to respond to patient needs. Projects have been launched related to: patient experience and outcomes; survivorship; palliative and end-of-life care; and primary and cancer care integration. However, it is acknowledged by the Partnership and its partners that much of this work is still in its early days and much work remains to be done before significant impacts will be realized. The Partnership has supported this work through funded projects related to different parts of the cancer journey (as noted above), as well as performance indicator work and patient representation on its working and advisory groups that are intended to embed a person-centred perspective into the Partnership's initiatives across the cancer control continuum. For example, eight jurisdictions are now using a common set of indicators to measure patient-reported experience and patient-reported outcomes. This is considered to be an important step in helping to ensure Canadians have equal access to consistent standards of cancer care. Capacity is seen as having many dimensions (e.g., knowledge, tools, resources, funding, human resources and technology) and there are vast differences in capacity across provinces and territories. Success is considered to be highly dependent upon the resources available in jurisdictions to effect the changes required to reflect a patient-centred perspective. Projects being supported varies across the country, therefore new networks such as the Palliative and End-of-Life National Network are promising for the sharing of best practices and information. The use of narrative stories that reflect the face of cancer (e.g., The Truth of It) are also considered important to remind those in the community why they are all involved in cancer care from the patients' viewpoint. |
Enhanced coordination of cancer research and improved population research capacity |
|
Progress Made; Further Work Warranted |
Substantial progress has been made on enhanced co-ordination of cancer research through the Canadian Cancer Research Alliance (CCRA) and on improved population-health research capacity through the Canadian Partnership for Tomorrow Project (CPTP). The CCRA has been guided by its Pan-Canadian Cancer Research Strategy, 2010-2014. A new strategy is being developed for launch in 2015. During the previous strategy, key accomplishments included the biennial Canadian Cancer Research Conferences, first held in 2011, and the first Pan-Canadian Framework for cancer prevention research, which spawned increased focus on and inclusion of prevention research in the overall funding environment. The Partnership has contributed to the CCRA through its support for the CCRA's executive office, its membership in the CCRA and several projects included in the research strategy, namely the CPTP and the Canadian Cancer Clinical Trials Network. Key informant views on the impacts on the coordination of cancer research varied. For example, interviewees from cancer agencies identified only a modest level of impact, while those from NGOs indicated that coordination had improved. Some key informants suggested that there is a lack of clarity concerning the extent to which the Partnership coordinates cancer research itself versus supporting and being a member of the CCRA, which is the coordinating body. The CPTP is a major success story for the Partnership and its partners. The Partnership is credited with bringing together a harmonized project, from what would instead have been five separate cohort projects. Reach was extended into the cardiovascular research community in partnership with the Heart and Stroke Foundation. Participation targets have been met and the CPTP platform has been tested and launched. Looking ahead, the CPTP is seen as an important source for researchers examining the causes of cancer and other related chronic diseases. However, there are issues concerning a long-term home and sustaining funding for the CPTP platform/infrastructure, beyond 2017. |
Improved First Nations, Inuit and Métis cancer control with and for First Nations, Inuit and Métis peoples |
|
Little Progress; Priority for Attention |
The development of the First Nations, Inuit and Métis Action Plan on Cancer Control is viewed as a significant first step in advancing cancer control for First Nations, Inuit and Métis communities. Responding to the major concern of access to health care services in remote and rural areas, in 2015, eight jurisdictions participated in and implemented initiatives to improve the continuity of care in remote and rural locations. Further, cancer agencies or their equivalents in three jurisdictions (Northern British Columbia, Manitoba and Ontario) had First Nations, Inuit and Métis specific cancer control strategies in place, compared to one since the Partnership was first initiated. The delivery of the Saint Elizabeth Cancer Course has increased awareness among healthcare providers of culturally relevant approaches to cancer control and increased awareness amongst First Nations service providers, of cancer control issues and challenges. Most key informants interviewed for the evaluation acknowledged the significant role that the Partnership has played in establishing the baselines and the Action Plan. It was noted that the process used involved a great deal of consultation and engagement of First Nations, Inuit and Métis. This process is considered to have worked well and could be a good model for federal departments and agencies to use in their other interactions with Aboriginal groups. The Partnership's representatives identified that there is now increased collaboration between cancer control stakeholders in terms of programming for First Nations, Inuit and Métis without the facilitation or involvement of the Partnership. It was acknowledged by interviewees that it is still premature to assess the extent of the success in advancing cancer control within First Nations, Inuit and Métis communities as the implementation of the initiatives is still in the early stages. It was also recognized that the success of the initiatives will depend on the appropriate level of funding and resources being made available from the respective jurisdictions (federal, provincial, territorial levels). An on-going challenge is the capacity (human and financial) within the National aboriginal organizations (NAOs) that has made it challenging for the Inuit Tapiriit Kanatami (ITK), Métis National Council (MNC) and, to a lesser extent, the Assembly of First Nations (AFN) to be as engaged as they would like. All three organizations have seen their federal funding decreased (in the case of ITK and AFN) or eliminated (in the case of MNC) in recent years. Representatives from all three NAOs noted that the Partnership has been very supportive and has provided financial resources in order to make it possible for the NAOs to stay engaged and fully contribute to the implementation of the Action Plan. Notably, capacity is also a challenge for Nunavut, which is not actively participating in CPAC-funded First Nations, Inuit and Métis initiatives. ITK and the Partnership have made efforts to keep Nunavut engaged until they are able to find the resources to become more involved. |
Improved analysis and reporting on cancer system performance |
|
Progress Made; Further Work Warranted |
Improved analysis and reporting on cancer system performance was thought, by most interviewees, to be an area where the Partnership has already achieved the outcome. Both "annual" and "special" cancer system performance reports have been produced. A set of core indicators has been identified and most jurisdictions are participating. Some of the special reports have delved into shedding light on variations in performance. The Partnership was viewed by most interviewees as having done an exemplary job of introducing the reports and adapting them to respond to additional deeper analysis as needed (e.g. special studies). The pan-Canadian approach brought visibility to cancer system performance reporting and enabled smaller provinces to engage, thereby helping to establish comparable indicators and data across Canada. Cancer agencies identified the use of the data in their own planning and operations and the fact that this level of reporting could only be achieved through a pan-Canadian approach. They also noted the use of the data and information as a tool for leveraging funding from their own provinces (motivational factor for those at the bottom of the rankings). For example, as was also noted in the case study for colorectal cancer screening, a jurisdictional key informant described how the multi-jurisdictional data on screening rates from the cancer system performance reports, plus the Partnership's work on the net benefits of screening, led jurisdictional decision-makers to fund more screening, when they had initially focused upon funding for treatment. A recent (July 2015) evaluation of the System Performance Reporting Initiative provided additional examples of cases where the cancer system performance reports have had impacts upon jurisdictional capacity related to system performance, improved clinical practice, improved program or service planning and/or delivery, improved policy or legislation or funding, and increased public awareness. Work to develop indicators around person-centred care and patient experience and outcomes is supported by key stakeholders. Further cancer organizations expressed the need to clearly demonstrate concrete evidence of the impacts of investments made in cancer care (value and what the investments are actually achieving in terms of making the cancer care system sustainable). The main challenge identified for provinces and territories is the funding and resources necessary for data collection in support of the System Performance Reporting Initiative. The information technology infrastructure and level of connectivity varies greatly across the jurisdictions. The levels of investment required to capture the data and put in place the associated electronic infrastructure, as well as the human resources to implement the necessary changes, are highly dependent on the level of funding available and the various health care system priorities established in each province or territory. Looking ahead, as cancer system performance reporting continues to mature and as jurisdictions meet the targets that have been set for indicators, there will be a need, as expressed by some key informants, to decide upon the ongoing purpose(s) of cancer system performance reporting. Will the purpose be to measure where the system is today and highlight gaps between actual performance and the targets that have been set? This means a focus on target-setting in order to make the current system work better. Or will the purpose be to use indicators to drive system change in areas where needs/issues have been identified? This means a focus on introducing new indicators that would help drive and measure perhaps more transformative change. Such indicators could be system, practitioner or person-focused. Or will it be a mix of the two purposes, which appears to be the case today? |
Enhanced access to high‐quality information, knowledge, tools and resources |
|
Progress Made; Further Work Warranted |
Progress continues to be made on the immediate outcome of enhanced access to high-quality information, knowledge, tools and resources. Contributions are being made from players across the cancer control community, for example, through the types of materials presented on the websites of provincial cancer agencies, cancer NGOs, and others. The Partnership has contributed to increased access through its own web presence - through cancerview.ca, and its corporate site partnershipagainstcancer.ca which is an entry point to various parts of cancerview.ca, as well as through its support for websites and portals tied to specific initiatives such as the CPTP and system performance reporting. The Partnership has gone beyond simply enhancing access. It has built tools - e.g., Cancer Risk Management Model (CRMM) - and worked on gathering and organizing information - e.g., through its internal Evidence, Synthesis, Guidelines Initiative, and though Co-ordinated Data Development. It has also worked on building capacity to not only access but transfer and use/analyze the information - e.g., through its Partnership Knowledge Transfer and Adoption Initiative, its Analytic Capacity Building and the collaborative work spaces on cancerview.ca As more of the initiatives supported by the Partnership in other areas (e.g., screening) mature, there is more demand on sharing the tools, knowledge and resources that they produced. The need for knowledge transfer and exchange then becomes greater. |
Enhanced public and patient awareness and engagement about the cancer strategy and cancer control issues |
|
Little Progress; Priority for Attention |
The Partnership's initiative in support of the immediate outcome of enhanced public and patient awareness and engagement about the cancer strategy and cancer control issues is in its early days. The initiative is targeted on engaging the interested public and embedding the patient perspective in initiatives, rather than broadly engaging the general public and patients. It is designed to ensure the patient perspective is captured, that integrated approaches across multiple communications channels and platforms are used, that existing partnerships are built upon, and innovative opportunities are explored. The Partnership has launched initiatives to strengthen engagement with the public, patient groups and professional associations. The Partnership has also incorporated the patient perspective on all of its advisory committees. There is agreement that the Partnership's role with respect to engagement of health care professionals and others working in cancer control is appropriate. However, there were concerns raised from key partners that the Partnership should not be trying to directly engage with patients and the public in order to raise awareness, rather than supporting other organizations to do this. These comments suggest a disconnect between what stakeholders interviewed assume is within scope of the Partnership's "public engagement and outreach" and the actual focus that the Partnership is taking in this initiative. |
5. Demonstration of Economy and Efficiency | |||
To what extent has the Partnership model resulted in an ability to act more quickly in response to evidence on the part of partners? |
|
Progress Made; Further Work Warranted |
There are strong examples where the Partnership has made partners more able to act quickly in response to evidence. Notable examples include:
|
To what extent has the Partnership model resulted in an ability to do more with the same resources? By working together through collaborative approaches are partners able to do more, and accelerate achievement of change as a result? |
|
Progress Made; Further Work Warranted |
There are strong examples of how the Partnership has facilitated the ability to do more with the same resources and accelerate change.
The Partnership model was strongly endorsed as the best alternative, given Canada's federated model. There were some suggestions that the approach could have a broader focus than just cancer (like the National Institute for Health and Clinical Care Excellence (NICE) in the United Kingdom). In general, these were contemplative suggestions rather than strongly held opinions that there was something wrong with the current approach. There were some other suggestions that the model be implemented in other issue areas such as seniors' health, cardiovascular health and dementia. |
What has been the contribution of the Partnership to improvements to the economy and efficiency of the cancer control system? |
|
Progress Made; Further Work Warranted |
The Partnership has contributed to improved efficiency and economy of the cancer control system. Examples include:
The Partnership has done so while also addressing some of the tough issues such as cancer control for First Nations, Inuit and Métis and instilling a person-centred perspective. Both of these are at early stages and will continue to be tough issues for the cancer control system moving forward. Other tough issues relate to other underserved populations such as new immigrants and those living in rural and remote areas, and to the aging population, and costs of cancer care, drugs and technologies. |
Appendix 5 - Evaluation Description
Core Issues | Evaluation Questions |
---|---|
Relevance | |
Issue #1: Continued need for program |
Assessment of the extent to which the program continues to address a demonstrable need and is responsive to the needs of Canadians.
|
Issue #2: Alignment with Government Priorities |
Assessment of the linkages between program objectives and (i) federal government priorities and (ii) departmental strategic outcomes.
|
Issue #3: Alignment with Federal Roles and Responsibilities |
Assessment of the role and responsibilities of the federal government in delivering the program.
|
Performance (effectiveness, efficiency and economy) | |
Issue #4: Achievement of expected outcomes |
Assessment of progress toward expected outcomes (including immediate, intermediate and ultimate outcomes) with reference to performance targets and program reach, program design, including the linkage and contribution of outputs to outcomes
|
Issue # 5: Demonstration of efficiency and economy |
Assessment of resource utilization in relation to the production of outputs and progress toward expected outcomes
|
Methods of Data Collection
Evaluators collected and analyzed data from multiple sources in order to address each of the evaluation issues and questions listed above.
Sources of information included literature review, document review including performance data, interviews and case studies.
The literature review involved identification of appropriate publications, and then synthesis of information from over 80 articles against those evaluation indicators for which the literature review was identified as a potential / likely data source. Those indicators were related, in particular, to the current picture of cancer control in Canada, the impact of cancer on Canadians, the factors which accelerate cancer control, the Partnership's model, and potential alternatives. Findings were presented in a literature review working paper.
The data and document review was based upon corporate, program and initiative level documents, as well as performance data, provided by the Partnership. Information was synthesized from over 80 documents against those evaluation indicators for which the data and document review was identified as a potential/likely data source. Those indicators were widely related to the relevance and performance issues. Findings were presented in a data and document review working paper.
Key informant interviews were conducted with 92 individuals, distributed across categories of key informants, as shown below. This number included interviews in support of the case studies. Findings were presented in a key informant interviews working paper.
- The Partnership (total 24)
- Chair and Members of the Board (4)
- Senior executives and Directors in the Partnership (10)
- Senior Scientific Leads (4)
- Expert Leads (6)
- Chairs and/or other selected members of Advisory Groups to the Partnership (total 15)
- Diagnosis and Clinical Care (2)
- Person-Centred Perspective (4)
- Population Health (2)
- Research (3)
- First Nation, Inuit and Métis Cancer Control (4)
- External (total 53)
- Federal / provincial / territorial governments (19)
- Cancer agencies (10)
- Non-Government organizations, such as national and PT cancer societies, patient/survivor groups, health associations, research groups (10)
- Key Initiative Partner Organizations (14)
Six case studies were prepared to highlight the development and results of a number of priority initiatives during the Partnership's first and second mandates. These case studies are included as appendices to this evaluation report.
- Colorectal Cancer Screening and the National Colorectal Cancer Screening Network
- Embedding Evidence in Care - Synoptic Reporting and Screening
- Patient Experience and Patient Reported Outcomes
- The Canadian Partnership for Tomorrow Project
- First Nations, Inuit and Métis Action Plan on Cancer Control
- System Performance Reporting
List of Acronyms
- AFN
- Assembly of First Nations
- CADTH
- Canadian Agency for Drugs and Technologies in Health
- CAPCA
- Canadian Association of Provincial Cancer Agencies
- CCAN
- Canadian Cancer Action Network
- CCRA
- Canadian Cancer Research Alliance
- CCTN
- Canadian Cancer Clinical Trials Network
- CHI
- Canada Health Infoway
- CIHI
- Canadian Institute for Health Information
- CIHR
- Canadian Institutes of Health Research
- CLASP
- Coalitions Linking Action and Science for Prevention
- CPAC
- Canadian Partnership Against Cancer
- CPQR
- Canadian Partnership for Quality Radiotherapy
- CPTP
- Canadian Partnership for Tomorrow Project
- CRC
- Colorectal cancer
- CRMM
- Cancer Risk Management Model
- CSCC
- Canadian Strategy for Cancer Control
- DPR
- Departmental Performance Report
- FN/I/M
- First Nations, Inuit and Métis
- FNIHB
- Health Canada's First Nations and Inuit Health Branch
- HC
- Health Canada
- ITK
- Inuit Tapiriit Kanatami
- KTE
- Knowledge transfer and exchange
- MNC
- Métis National Council
- NAO
- National aboriginal organization
- NCCSN
- National Colorectal Cancer Screening Network
- NGO
- Non-governmental organization
- NICE
- National Institute for Health and Clinical Care Excellence
- OICR
- Ontario Institute for Cancer Research
- PAA
- Program Alignment Architecture
- pCODR
- Pan-Canadian Oncology Drug Review
- PHAC
- Public Health Agency of Canada
- PMS
- Performance Measurement Strategy
- PREM
- Patient-reported experience measure
- PROM
- Patient-reported outcome measure
- QIIP
- Quality Initiative in Interpretive Pathology
- SAGE
- Standards and Guidelines Evidence
Footnotes
Page details
- Date modified: